mirror of
https://github.com/RayLabsHQ/gitea-mirror.git
synced 2026-11-10 08:22:21 +01:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a18f262ca7 | ||
|
|
3798456f5d | ||
|
|
73f1609117 | ||
|
|
588567931a | ||
|
|
5c1317c759 | ||
|
|
5f1c37b320 | ||
|
|
083b342f38 | ||
|
|
92bb38b122 | ||
|
|
e7ac54a72a | ||
|
|
2ea250f081 | ||
|
|
c1712bc670 | ||
|
|
c4550196e9 | ||
|
|
8cb8fd6fe1 | ||
|
|
4b4ea9614b | ||
|
|
1644505043 | ||
|
|
e142524bfc | ||
|
|
8fac30fc02 |
No files matched your search
@@ -46,6 +46,14 @@ BETTER_AUTH_URL=http://localhost:4321
|
||||
PUBLIC_BETTER_AUTH_URL=http://localhost:4321
|
||||
# BETTER_AUTH_TRUSTED_ORIGINS=
|
||||
|
||||
# ===========================================
|
||||
# HTTPS / TLS (Optional)
|
||||
# ===========================================
|
||||
# Set BOTH to have the server terminate TLS directly (no reverse proxy needed).
|
||||
# Leave unset when TLS is handled upstream by Nginx/Traefik/Caddy.
|
||||
# SERVER_CERT_PATH=/etc/ssl/gitea-mirror/cert.pem
|
||||
# SERVER_KEY_PATH=/etc/ssl/gitea-mirror/key.pem
|
||||
|
||||
# ===========================================
|
||||
# DOCKER CONFIGURATION (Optional)
|
||||
# ===========================================
|
||||
@@ -65,6 +73,12 @@ DOCKER_TAG=latest
|
||||
# GITHUB_TOKEN=your-github-personal-access-token
|
||||
# GITHUB_TYPE=personal # Options: personal, organization
|
||||
|
||||
# GitHub Enterprise (GHES / GHEC with data residency)
|
||||
# Leave unset for standard github.com. Examples:
|
||||
# GHES (self-hosted): https://ghe.example.com/api/v3
|
||||
# GHEC data residency: https://api.TENANT.ghe.com
|
||||
# GH_API_URL=https://ghe.example.com/api/v3
|
||||
|
||||
# Repository Selection
|
||||
# PRIVATE_REPOSITORIES=false
|
||||
# PUBLIC_REPOSITORIES=true
|
||||
|
||||
@@ -33,7 +33,7 @@ jobs:
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v1
|
||||
with:
|
||||
bun-version: '1.3.10'
|
||||
bun-version: '1.3.13'
|
||||
|
||||
- name: Check lockfile and install dependencies
|
||||
run: |
|
||||
|
||||
@@ -40,7 +40,7 @@ env:
|
||||
FAKE_GITHUB_PORT: 4580
|
||||
GIT_SERVER_PORT: 4590
|
||||
APP_PORT: 4321
|
||||
BUN_VERSION: "1.3.10"
|
||||
BUN_VERSION: "1.3.13"
|
||||
|
||||
jobs:
|
||||
e2e-tests:
|
||||
|
||||
+3
-3
@@ -1,6 +1,6 @@
|
||||
# syntax=docker/dockerfile:1.4
|
||||
|
||||
FROM oven/bun:1.3.10-debian AS base
|
||||
FROM oven/bun:1.3.13-debian AS base
|
||||
WORKDIR /app
|
||||
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install-recommends \
|
||||
python3 make g++ gcc wget sqlite3 openssl ca-certificates \
|
||||
@@ -32,7 +32,7 @@ FROM debian:trixie-slim AS git-lfs-builder
|
||||
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install-recommends \
|
||||
wget ca-certificates git make \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
ARG GO_VERSION=1.25.8
|
||||
ARG GO_VERSION=1.25.9
|
||||
ARG GIT_LFS_VERSION=3.7.1
|
||||
RUN ARCH="$(dpkg --print-architecture)" \
|
||||
&& wget -qO /tmp/go.tar.gz "https://go.dev/dl/go${GO_VERSION}.linux-${ARCH}.tar.gz" \
|
||||
@@ -49,7 +49,7 @@ RUN git clone --branch "v${GIT_LFS_VERSION}" --depth 1 https://github.com/git-lf
|
||||
&& install -m 755 /tmp/git-lfs/bin/git-lfs /usr/local/bin/git-lfs
|
||||
|
||||
# ----------------------------
|
||||
FROM oven/bun:1.3.10-debian AS runner
|
||||
FROM oven/bun:1.3.13-debian AS runner
|
||||
WORKDIR /app
|
||||
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install-recommends \
|
||||
git wget sqlite3 openssl ca-certificates \
|
||||
|
||||
@@ -29,6 +29,7 @@ First user signup becomes admin. Configure GitHub and Gitea/Forgejo through the
|
||||
## ✨ Features
|
||||
|
||||
- 🔁 Mirror public, private, and starred GitHub repos to Gitea/Forgejo
|
||||
- 🏛️ **GitHub Enterprise support** - Works with GHES and GHEC with data residency via `GH_API_URL`
|
||||
- 🏢 Mirror entire organizations with flexible strategies
|
||||
- 🎯 Custom destination control for repos and organizations
|
||||
- 📦 **Git LFS support** - Mirror large files with Git LFS
|
||||
@@ -296,6 +297,20 @@ CLEANUP_DRY_RUN=false # Set to true to test without changes
|
||||
- **The Whole Point of Backups**: Your Gitea/Forgejo mirrors are preserved even when GitHub sources disappear - that's why you have backups!
|
||||
- **Strongly Recommended**: Always use `CLEANUP_ORPHANED_REPO_ACTION=archive` (default) instead of `delete`
|
||||
|
||||
### GitHub Enterprise (GHES / GHEC with Data Residency)
|
||||
|
||||
Gitea Mirror works with non-`github.com` GitHub deployments. Point the client at your Enterprise API via the `GH_API_URL` environment variable:
|
||||
|
||||
```bash
|
||||
# GitHub Enterprise Server (self-hosted)
|
||||
GH_API_URL=https://ghe.example.com/api/v3
|
||||
|
||||
# GitHub Enterprise Cloud with data residency
|
||||
GH_API_URL=https://api.TENANT.ghe.com
|
||||
```
|
||||
|
||||
Standard GitHub Enterprise Cloud on `github.com` needs no override. Use a token issued by the target Enterprise instance for `GITHUB_TOKEN`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Reverse Proxy Configuration
|
||||
@@ -303,15 +318,18 @@ CLEANUP_DRY_RUN=false # Set to true to test without changes
|
||||
If you run behind a reverse proxy on a subpath (for example `https://git.example.com/mirror`), configure:
|
||||
|
||||
```bash
|
||||
# BASE_URL handles the path prefix — auth URLs stay as origin only
|
||||
BASE_URL=/mirror
|
||||
BETTER_AUTH_URL=https://git.example.com
|
||||
PUBLIC_BETTER_AUTH_URL=https://git.example.com
|
||||
BETTER_AUTH_TRUSTED_ORIGINS=https://git.example.com
|
||||
# → Auth endpoints resolve to: https://git.example.com/mirror/api/auth/*
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `BASE_URL` sets the application path prefix.
|
||||
- `BETTER_AUTH_TRUSTED_ORIGINS` should contain origins only (no path).
|
||||
- `BETTER_AUTH_URL` and `PUBLIC_BETTER_AUTH_URL` should be **origin only** (e.g. `https://git.example.com`). Do not include the base path — it is applied automatically from `BASE_URL`. Any path accidentally included is stripped.
|
||||
- `BETTER_AUTH_TRUSTED_ORIGINS` should also contain origins only (no path).
|
||||
- `BASE_URL` is runtime configuration, so prebuilt registry images can be reused across different subpaths.
|
||||
|
||||
### Mirror Token Rotation (GitHub Token Changed)
|
||||
|
||||
@@ -45,7 +45,7 @@
|
||||
"clsx": "^2.1.1",
|
||||
"cmdk": "^1.1.1",
|
||||
"dotenv": "^17.3.1",
|
||||
"drizzle-orm": "^0.45.1",
|
||||
"drizzle-orm": "^0.45.2",
|
||||
"fuse.js": "^7.1.0",
|
||||
"jsonwebtoken": "^9.0.3",
|
||||
"lucide-react": "^0.577.0",
|
||||
@@ -82,9 +82,14 @@
|
||||
},
|
||||
"overrides": {
|
||||
"@esbuild-kit/esm-loader": "npm:tsx@^4.21.0",
|
||||
"@xmldom/xmldom": "^0.8.12",
|
||||
"defu": "^6.1.7",
|
||||
"devalue": "^5.6.4",
|
||||
"fast-xml-parser": "^5.5.6",
|
||||
"kysely": "^0.28.16",
|
||||
"lodash": "^4.18.1",
|
||||
"node-forge": "^1.3.3",
|
||||
"picomatch": "^4.0.4",
|
||||
"rollup": ">=4.59.0",
|
||||
"svgo": "^4.0.1",
|
||||
},
|
||||
@@ -721,7 +726,7 @@
|
||||
|
||||
"@xmldom/is-dom-node": ["@xmldom/is-dom-node@1.0.1", "", {}, "sha512-CJDxIgE5I0FH+ttq/Fxy6nRpxP70+e2O048EPe85J2use3XKdatVM7dDVvFNjQudd9B49NPoZ+8PG49zj4Er8Q=="],
|
||||
|
||||
"@xmldom/xmldom": ["@xmldom/xmldom@0.8.11", "", {}, "sha512-cQzWCtO6C8TQiYl1ruKNn2U6Ao4o4WBBcbL61yJl84x+j5sOWWFU9X7DpND8XZG3daDppSsigMdfAIl2upQBRw=="],
|
||||
"@xmldom/xmldom": ["@xmldom/xmldom@0.8.12", "", {}, "sha512-9k/gHF6n/pAi/9tqr3m3aqkuiNosYTurLLUtc7xQ9sxB/wm7WPygCv8GYa6mS0fLJEHhqMC1ATYhz++U/lRHqg=="],
|
||||
|
||||
"acorn": ["acorn@8.16.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw=="],
|
||||
|
||||
@@ -859,7 +864,7 @@
|
||||
|
||||
"decode-named-character-reference": ["decode-named-character-reference@1.3.0", "", { "dependencies": { "character-entities": "^2.0.0" } }, "sha512-GtpQYB283KrPp6nRw50q3U9/VfOutZOe103qlN7BPP6Ad27xYnOIWv4lPzo8HCAL+mMZofJ9KEy30fq6MfaK6Q=="],
|
||||
|
||||
"defu": ["defu@6.1.4", "", {}, "sha512-mEQCMmwJu317oSz8CwdIOdwf3xMif1ttiM8LTufzc3g6kR+9Pe236twL8j3IYT1F7GfRgGcW6MWxzZjLIkuHIg=="],
|
||||
"defu": ["defu@6.1.7", "", {}, "sha512-7z22QmUWiQ/2d0KkdYmANbRUVABpZ9SNYyH5vx6PZ+nE5bcC0l7uFvEfHlyld/HcGBFTL536ClDt3DEcSlEJAQ=="],
|
||||
|
||||
"depd": ["depd@2.0.0", "", {}, "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw=="],
|
||||
|
||||
@@ -893,7 +898,7 @@
|
||||
|
||||
"drizzle-kit": ["drizzle-kit@0.31.9", "", { "dependencies": { "@drizzle-team/brocli": "^0.10.2", "@esbuild-kit/esm-loader": "^2.5.5", "esbuild": "^0.25.4", "esbuild-register": "^3.5.0" }, "bin": { "drizzle-kit": "bin.cjs" } }, "sha512-GViD3IgsXn7trFyBUUHyTFBpH/FsHTxYJ66qdbVggxef4UBPHRYxQaRzYLTuekYnk9i5FIEL9pbBIwMqX/Uwrg=="],
|
||||
|
||||
"drizzle-orm": ["drizzle-orm@0.45.1", "", { "peerDependencies": { "@aws-sdk/client-rds-data": ">=3", "@cloudflare/workers-types": ">=4", "@electric-sql/pglite": ">=0.2.0", "@libsql/client": ">=0.10.0", "@libsql/client-wasm": ">=0.10.0", "@neondatabase/serverless": ">=0.10.0", "@op-engineering/op-sqlite": ">=2", "@opentelemetry/api": "^1.4.1", "@planetscale/database": ">=1.13", "@prisma/client": "*", "@tidbcloud/serverless": "*", "@types/better-sqlite3": "*", "@types/pg": "*", "@types/sql.js": "*", "@upstash/redis": ">=1.34.7", "@vercel/postgres": ">=0.8.0", "@xata.io/client": "*", "better-sqlite3": ">=7", "bun-types": "*", "expo-sqlite": ">=14.0.0", "gel": ">=2", "knex": "*", "kysely": "*", "mysql2": ">=2", "pg": ">=8", "postgres": ">=3", "sql.js": ">=1", "sqlite3": ">=5" }, "optionalPeers": ["@aws-sdk/client-rds-data", "@cloudflare/workers-types", "@electric-sql/pglite", "@libsql/client", "@libsql/client-wasm", "@neondatabase/serverless", "@op-engineering/op-sqlite", "@opentelemetry/api", "@planetscale/database", "@prisma/client", "@tidbcloud/serverless", "@types/better-sqlite3", "@types/pg", "@types/sql.js", "@upstash/redis", "@vercel/postgres", "@xata.io/client", "better-sqlite3", "bun-types", "expo-sqlite", "gel", "knex", "kysely", "mysql2", "pg", "postgres", "sql.js", "sqlite3"] }, "sha512-Te0FOdKIistGNPMq2jscdqngBRfBpC8uMFVwqjf6gtTVJHIQ/dosgV/CLBU2N4ZJBsXL5savCba9b0YJskKdcA=="],
|
||||
"drizzle-orm": ["drizzle-orm@0.45.2", "", { "peerDependencies": { "@aws-sdk/client-rds-data": ">=3", "@cloudflare/workers-types": ">=4", "@electric-sql/pglite": ">=0.2.0", "@libsql/client": ">=0.10.0", "@libsql/client-wasm": ">=0.10.0", "@neondatabase/serverless": ">=0.10.0", "@op-engineering/op-sqlite": ">=2", "@opentelemetry/api": "^1.4.1", "@planetscale/database": ">=1.13", "@prisma/client": "*", "@tidbcloud/serverless": "*", "@types/better-sqlite3": "*", "@types/pg": "*", "@types/sql.js": "*", "@upstash/redis": ">=1.34.7", "@vercel/postgres": ">=0.8.0", "@xata.io/client": "*", "better-sqlite3": ">=7", "bun-types": "*", "expo-sqlite": ">=14.0.0", "gel": ">=2", "knex": "*", "kysely": "*", "mysql2": ">=2", "pg": ">=8", "postgres": ">=3", "sql.js": ">=1", "sqlite3": ">=5" }, "optionalPeers": ["@aws-sdk/client-rds-data", "@cloudflare/workers-types", "@electric-sql/pglite", "@libsql/client", "@libsql/client-wasm", "@neondatabase/serverless", "@op-engineering/op-sqlite", "@opentelemetry/api", "@planetscale/database", "@prisma/client", "@tidbcloud/serverless", "@types/better-sqlite3", "@types/pg", "@types/sql.js", "@upstash/redis", "@vercel/postgres", "@xata.io/client", "better-sqlite3", "bun-types", "expo-sqlite", "gel", "knex", "kysely", "mysql2", "pg", "postgres", "sql.js", "sqlite3"] }, "sha512-kY0BSaTNYWnoDMVoyY8uxmyHjpJW1geOmBMdSSicKo9CIIWkSxMIj2rkeSR51b8KAPB7m+qysjuHme5nKP+E5Q=="],
|
||||
|
||||
"dset": ["dset@3.1.4", "", {}, "sha512-2QF/g9/zTaPDc3BjNcVTGoBbXBgYfMTTceLaYcFJ/W9kggFUkhxD/hMEeuLKbugyef9SqAx8cpgwlIP/jinUTA=="],
|
||||
|
||||
@@ -1085,7 +1090,7 @@
|
||||
|
||||
"kleur": ["kleur@4.1.5", "", {}, "sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ=="],
|
||||
|
||||
"kysely": ["kysely@0.28.12", "", {}, "sha512-kWiueDWXhbCchgiotwXkwdxZE/6h56IHAeFWg4euUfW0YsmO9sxbAxzx1KLLv2lox15EfuuxHQvgJ1qIfZuHGw=="],
|
||||
"kysely": ["kysely@0.28.16", "", {}, "sha512-3i5pmOiZvMDj00qhrIVbH0AnioVTx22DMP7Vn5At4yJO46iy+FM8Y/g61ltenLVSo3fiO8h8Q3QOFgf/gQ72ww=="],
|
||||
|
||||
"lightningcss": ["lightningcss@1.32.0", "", { "dependencies": { "detect-libc": "^2.0.3" }, "optionalDependencies": { "lightningcss-android-arm64": "1.32.0", "lightningcss-darwin-arm64": "1.32.0", "lightningcss-darwin-x64": "1.32.0", "lightningcss-freebsd-x64": "1.32.0", "lightningcss-linux-arm-gnueabihf": "1.32.0", "lightningcss-linux-arm64-gnu": "1.32.0", "lightningcss-linux-arm64-musl": "1.32.0", "lightningcss-linux-x64-gnu": "1.32.0", "lightningcss-linux-x64-musl": "1.32.0", "lightningcss-win32-arm64-msvc": "1.32.0", "lightningcss-win32-x64-msvc": "1.32.0" } }, "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ=="],
|
||||
|
||||
@@ -1111,7 +1116,7 @@
|
||||
|
||||
"lightningcss-win32-x64-msvc": ["lightningcss-win32-x64-msvc@1.32.0", "", { "os": "win32", "cpu": "x64" }, "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q=="],
|
||||
|
||||
"lodash": ["lodash@4.17.21", "", {}, "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg=="],
|
||||
"lodash": ["lodash@4.18.1", "", {}, "sha512-dMInicTPVE8d1e5otfwmmjlxkZoUpiVLwyeTdUsi/Caj/gfzzblBcCE5sRHV/AsjuCmxWrte2TNGSYuCeCq+0Q=="],
|
||||
|
||||
"lodash.includes": ["lodash.includes@4.3.0", "", {}, "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w=="],
|
||||
|
||||
@@ -1325,7 +1330,7 @@
|
||||
|
||||
"picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="],
|
||||
|
||||
"picomatch": ["picomatch@4.0.3", "", {}, "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q=="],
|
||||
"picomatch": ["picomatch@4.0.4", "", {}, "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A=="],
|
||||
|
||||
"playwright": ["playwright@1.58.2", "", { "dependencies": { "playwright-core": "1.58.2" }, "optionalDependencies": { "fsevents": "2.3.2" }, "bin": { "playwright": "cli.js" } }, "sha512-vA30H8Nvkq/cPBnNw4Q8TWz1EJyqgpuinBcHET0YVJVFldr8JDNiU9LaWAE1KqSkRYazuaBhTpB5ZzShOezQ6A=="],
|
||||
|
||||
@@ -1745,8 +1750,6 @@
|
||||
|
||||
"@testing-library/dom/dom-accessibility-api": ["dom-accessibility-api@0.5.16", "", {}, "sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg=="],
|
||||
|
||||
"anymatch/picomatch": ["picomatch@2.3.1", "", {}, "sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA=="],
|
||||
|
||||
"astro/esbuild": ["esbuild@0.27.4", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.27.4", "@esbuild/android-arm": "0.27.4", "@esbuild/android-arm64": "0.27.4", "@esbuild/android-x64": "0.27.4", "@esbuild/darwin-arm64": "0.27.4", "@esbuild/darwin-x64": "0.27.4", "@esbuild/freebsd-arm64": "0.27.4", "@esbuild/freebsd-x64": "0.27.4", "@esbuild/linux-arm": "0.27.4", "@esbuild/linux-arm64": "0.27.4", "@esbuild/linux-ia32": "0.27.4", "@esbuild/linux-loong64": "0.27.4", "@esbuild/linux-mips64el": "0.27.4", "@esbuild/linux-ppc64": "0.27.4", "@esbuild/linux-riscv64": "0.27.4", "@esbuild/linux-s390x": "0.27.4", "@esbuild/linux-x64": "0.27.4", "@esbuild/netbsd-arm64": "0.27.4", "@esbuild/netbsd-x64": "0.27.4", "@esbuild/openbsd-arm64": "0.27.4", "@esbuild/openbsd-x64": "0.27.4", "@esbuild/openharmony-arm64": "0.27.4", "@esbuild/sunos-x64": "0.27.4", "@esbuild/win32-arm64": "0.27.4", "@esbuild/win32-ia32": "0.27.4", "@esbuild/win32-x64": "0.27.4" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-Rq4vbHnYkK5fws5NF7MYTU68FPRE1ajX7heQ/8QXXWqNgqqJ/GkmmyxIzUnf2Sr/bakf8l54716CcMGHYhMrrQ=="],
|
||||
|
||||
"astro/vite": ["vite@7.3.1", "", { "dependencies": { "esbuild": "^0.27.0", "fdir": "^6.5.0", "picomatch": "^4.0.3", "postcss": "^8.5.6", "rollup": "^4.43.0", "tinyglobby": "^0.2.15" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", "jiti": ">=1.21.0", "less": "^4.0.0", "lightningcss": "^1.21.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", "sugarss": "^5.0.0", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "jiti", "less", "lightningcss", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-w+N7Hifpc3gRjZ63vYBXA56dvvRlNWRczTdmCBBa+CotUzAPf5b7YMdMR/8CQoeYE5LX3W4wj6RYTgonm1b9DA=="],
|
||||
|
||||
@@ -16,6 +16,7 @@ When environment variables are set:
|
||||
## Table of Contents
|
||||
|
||||
- [Core Configuration](#core-configuration)
|
||||
- [HTTPS / TLS](#https--tls)
|
||||
- [GitHub Configuration](#github-configuration)
|
||||
- [Gitea Configuration](#gitea-configuration)
|
||||
- [Mirror Options](#mirror-options)
|
||||
@@ -36,11 +37,35 @@ Essential application settings required for running Gitea Mirror.
|
||||
| `BASE_URL` | Application base path. Use `/` for root deployments, or a prefix such as `/mirror` when serving behind a reverse-proxy path prefix. | `/` | No |
|
||||
| `DATABASE_URL` | Database connection URL | `sqlite://data/gitea-mirror.db` | No |
|
||||
| `BETTER_AUTH_SECRET` | Secret key for session signing (generate with: `openssl rand -base64 32`) | - | Yes |
|
||||
| `BETTER_AUTH_URL` | Primary base URL for authentication. This should be the main URL where your application is accessed. | `http://localhost:4321` | No |
|
||||
| `PUBLIC_BETTER_AUTH_URL` | Client-side auth URL for multi-origin access. Set this to your primary domain when you need to access the app from different origins (e.g., both IP and domain). The client will use this URL for all auth requests instead of the current browser origin. | - | No |
|
||||
| `BETTER_AUTH_URL` | Authentication origin (scheme + host only, e.g. `https://git.example.com`). Do **not** include a path — any path is automatically stripped, and `BASE_URL` is applied separately. | `http://localhost:4321` | No |
|
||||
| `PUBLIC_BETTER_AUTH_URL` | Client-side auth origin for multi-origin access (same rule: origin only, no path). Set this to your primary domain when you need to access the app from different origins (e.g., both IP and domain). The client will use this URL for all auth requests instead of the current browser origin. | - | No |
|
||||
| `BETTER_AUTH_TRUSTED_ORIGINS` | Trusted origins for authentication requests. Comma-separated list of URLs. Use this to specify additional access URLs (e.g., local IP + domain: `http://10.10.20.45:4321,https://gitea-mirror.mydomain.tld`), SSO providers, reverse proxies, etc. | - | No |
|
||||
| `ENCRYPTION_SECRET` | Optional encryption key for tokens (generate with: `openssl rand -base64 48`) | - | No |
|
||||
|
||||
## HTTPS / TLS
|
||||
|
||||
Gitea Mirror can terminate TLS directly via the underlying `@astrojs/node` adapter — useful when you don't want a separate reverse proxy. When both variables below are set, the server starts as a real HTTPS listener instead of HTTP.
|
||||
|
||||
| Variable | Description | Default | Required |
|
||||
|----------|-------------|---------|----------|
|
||||
| `SERVER_CERT_PATH` | Absolute path to the TLS certificate (PEM). Set together with `SERVER_KEY_PATH` to enable HTTPS. | - | No |
|
||||
| `SERVER_KEY_PATH` | Absolute path to the TLS private key (PEM). Set together with `SERVER_CERT_PATH` to enable HTTPS. | - | No |
|
||||
|
||||
**Example (systemd or `.env`):**
|
||||
|
||||
```bash
|
||||
SERVER_CERT_PATH=/etc/ssl/gitea-mirror/cert.pem
|
||||
SERVER_KEY_PATH=/etc/ssl/gitea-mirror/key.pem
|
||||
PORT=443
|
||||
BETTER_AUTH_URL=https://mirror.example.com
|
||||
BETTER_AUTH_TRUSTED_ORIGINS=https://mirror.example.com
|
||||
```
|
||||
|
||||
Notes:
|
||||
- The process must have read access to both files. When binding to `PORT=443`, grant the binary the `CAP_NET_BIND_SERVICE` capability (or run as a user allowed to bind privileged ports) rather than running as root.
|
||||
- If you already terminate TLS at a reverse proxy (nginx, Traefik, Caddy), leave these unset and let the proxy handle certificates.
|
||||
- Works in Docker too — mount your certs and set both paths to locations inside the container.
|
||||
|
||||
## GitHub Configuration
|
||||
|
||||
Settings for connecting to and configuring GitHub repository sources.
|
||||
@@ -52,6 +77,21 @@ Settings for connecting to and configuring GitHub repository sources.
|
||||
| `GITHUB_USERNAME` | Your GitHub username | - | - |
|
||||
| `GITHUB_TOKEN` | GitHub personal access token (requires repo and admin:org scopes) | - | - |
|
||||
| `GITHUB_TYPE` | GitHub account type | `personal` | `personal`, `organization` |
|
||||
| `GH_API_URL` | GitHub API base URL. Override this to point at GitHub Enterprise Server or Enterprise Cloud with data residency. | `https://api.github.com` | e.g. `https://ghe.example.com/api/v3`, `https://api.TENANT.ghe.com` |
|
||||
|
||||
### GitHub Enterprise (GHES / GHEC with data residency)
|
||||
|
||||
Set `GH_API_URL` to point Octokit at a non-`github.com` API endpoint:
|
||||
|
||||
```bash
|
||||
# GitHub Enterprise Server (self-hosted)
|
||||
GH_API_URL=https://ghe.example.com/api/v3
|
||||
|
||||
# GitHub Enterprise Cloud with data residency
|
||||
GH_API_URL=https://api.TENANT.ghe.com
|
||||
```
|
||||
|
||||
Standard GitHub Enterprise Cloud on `github.com` works with the default — no override needed. Use a personal access token issued by the target Enterprise instance for `GITHUB_TOKEN`.
|
||||
|
||||
### Repository Selection
|
||||
|
||||
@@ -377,14 +417,17 @@ This setup allows you to:
|
||||
If you serve Gitea Mirror under a subpath such as `https://git.example.com/mirror`, set:
|
||||
|
||||
```bash
|
||||
# BASE_URL handles the path prefix — auth URLs stay as origin only
|
||||
BASE_URL=/mirror
|
||||
BETTER_AUTH_URL=https://git.example.com
|
||||
PUBLIC_BETTER_AUTH_URL=https://git.example.com
|
||||
BETTER_AUTH_TRUSTED_ORIGINS=https://git.example.com
|
||||
# → Auth endpoints resolve to: https://git.example.com/mirror/api/auth/*
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `BETTER_AUTH_TRUSTED_ORIGINS` must contain origins only (no path).
|
||||
- `BETTER_AUTH_URL` and `PUBLIC_BETTER_AUTH_URL` must be **origin only** (scheme + host). Do not include the base path — it is applied automatically from `BASE_URL`. Any path accidentally included is stripped.
|
||||
- `BETTER_AUTH_TRUSTED_ORIGINS` must also contain origins only (no path).
|
||||
- `BASE_URL` is applied at runtime, so prebuilt images can be reused with different path prefixes.
|
||||
|
||||
### Trusted Origins
|
||||
|
||||
+9
-4
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "gitea-mirror",
|
||||
"type": "module",
|
||||
"version": "3.15.0",
|
||||
"version": "3.15.7",
|
||||
"engines": {
|
||||
"bun": ">=1.2.9"
|
||||
},
|
||||
@@ -45,11 +45,16 @@
|
||||
},
|
||||
"overrides": {
|
||||
"@esbuild-kit/esm-loader": "npm:tsx@^4.21.0",
|
||||
"@xmldom/xmldom": "^0.8.12",
|
||||
"defu": "^6.1.7",
|
||||
"devalue": "^5.6.4",
|
||||
"fast-xml-parser": "^5.5.6",
|
||||
"kysely": "^0.28.16",
|
||||
"lodash": "^4.18.1",
|
||||
"node-forge": "^1.3.3",
|
||||
"svgo": "^4.0.1",
|
||||
"rollup": ">=4.59.0"
|
||||
"picomatch": "^4.0.4",
|
||||
"rollup": ">=4.59.0",
|
||||
"svgo": "^4.0.1"
|
||||
},
|
||||
"dependencies": {
|
||||
"@astrojs/check": "^0.9.7",
|
||||
@@ -92,7 +97,7 @@
|
||||
"clsx": "^2.1.1",
|
||||
"cmdk": "^1.1.1",
|
||||
"dotenv": "^17.3.1",
|
||||
"drizzle-orm": "^0.45.1",
|
||||
"drizzle-orm": "^0.45.2",
|
||||
"fuse.js": "^7.1.0",
|
||||
"jsonwebtoken": "^9.0.3",
|
||||
"lucide-react": "^0.577.0",
|
||||
|
||||
@@ -6,7 +6,9 @@ import {
|
||||
CardHeader,
|
||||
CardTitle,
|
||||
} from "@/components/ui/card";
|
||||
import { giteaApi } from "@/lib/api";
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert";
|
||||
import { AlertTriangle } from "lucide-react";
|
||||
import { giteaApi, type GiteaServerInfo } from "@/lib/api";
|
||||
import type { GiteaConfig, MirrorStrategy } from "@/types/config";
|
||||
import { toast } from "sonner";
|
||||
import { OrganizationStrategy } from "./OrganizationStrategy";
|
||||
@@ -23,6 +25,7 @@ interface GiteaConfigFormProps {
|
||||
|
||||
export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, githubUsername }: GiteaConfigFormProps) {
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
const [serverInfo, setServerInfo] = useState<GiteaServerInfo | null>(null);
|
||||
|
||||
// Derive the mirror strategy from existing config for backward compatibility
|
||||
const getMirrorStrategy = (): MirrorStrategy => {
|
||||
@@ -128,13 +131,16 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
|
||||
try {
|
||||
const result = await giteaApi.testConnection(config.url, config.token);
|
||||
if (result.success) {
|
||||
setServerInfo(result.serverInfo ?? null);
|
||||
toast.success("Successfully connected to Gitea!");
|
||||
} else {
|
||||
setServerInfo(null);
|
||||
toast.error(
|
||||
"Failed to connect to Gitea. Please check your URL and token."
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
setServerInfo(null);
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "An unknown error occurred"
|
||||
);
|
||||
@@ -162,6 +168,31 @@ export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving, g
|
||||
</CardHeader>
|
||||
|
||||
<CardContent className="flex flex-col gap-y-6 flex-1">
|
||||
{serverInfo?.type === "forgejo" && serverInfo.hasMirrorCredBug && (
|
||||
<Alert variant="warning">
|
||||
<AlertTriangle className="h-4 w-4" />
|
||||
<AlertTitle>
|
||||
Forgejo {serverInfo.version} has a known mirror-credential bug
|
||||
</AlertTitle>
|
||||
<AlertDescription>
|
||||
<p>
|
||||
Pull-mirror credentials sent via Forgejo's migrate API aren't persisted on this version, so subsequent syncs of private repos fail with <code className="text-xs font-mono bg-amber-100 dark:bg-amber-900/40 px-1 py-0.5 rounded">terminal prompts disabled</code>. Fixed in Forgejo 15.0.0 (
|
||||
<a
|
||||
href="https://codeberg.org/forgejo/forgejo/pulls/11909"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="underline underline-offset-2"
|
||||
>
|
||||
PR #11909
|
||||
</a>
|
||||
).
|
||||
</p>
|
||||
<p>
|
||||
Upgrade Forgejo to 15.0.0 or later, then delete and re-mirror affected repos — or open each repo's Settings → Mirror Settings in Forgejo and re-enter the GitHub token once.
|
||||
</p>
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
<div>
|
||||
<label
|
||||
htmlFor="gitea-username"
|
||||
|
||||
@@ -65,14 +65,16 @@ export function useConfigStatus(): ConfigStatus {
|
||||
if (isCacheValid && hasCheckedRef.current) {
|
||||
const configResponse = configCache.data!;
|
||||
|
||||
const isGitHubConfigured = !!(
|
||||
configResponse?.githubConfig?.username &&
|
||||
configResponse?.githubConfig?.token
|
||||
);
|
||||
// Only token/url are actually required at runtime: the GitHub token is
|
||||
// self-authenticating for listForAuthenticatedUser, and a Gitea username
|
||||
// isn't needed under single-org / flat mirror strategies. Users who
|
||||
// configure via env vars without GITHUB_USERNAME / GITEA_USERNAME set
|
||||
// (or who otherwise left those blank) were being locked out of the
|
||||
// dashboard even though mirroring worked fine (see issue #271).
|
||||
const isGitHubConfigured = !!configResponse?.githubConfig?.token;
|
||||
|
||||
const isGiteaConfigured = !!(
|
||||
configResponse?.giteaConfig?.url &&
|
||||
configResponse?.giteaConfig?.username &&
|
||||
configResponse?.giteaConfig?.token
|
||||
);
|
||||
|
||||
@@ -108,14 +110,16 @@ export function useConfigStatus(): ConfigStatus {
|
||||
userId: user.id
|
||||
};
|
||||
|
||||
const isGitHubConfigured = !!(
|
||||
configResponse?.githubConfig?.username &&
|
||||
configResponse?.githubConfig?.token
|
||||
);
|
||||
// Only token/url are actually required at runtime: the GitHub token is
|
||||
// self-authenticating for listForAuthenticatedUser, and a Gitea username
|
||||
// isn't needed under single-org / flat mirror strategies. Users who
|
||||
// configure via env vars without GITHUB_USERNAME / GITEA_USERNAME set
|
||||
// (or who otherwise left those blank) were being locked out of the
|
||||
// dashboard even though mirroring worked fine (see issue #271).
|
||||
const isGitHubConfigured = !!configResponse?.githubConfig?.token;
|
||||
|
||||
const isGiteaConfigured = !!(
|
||||
configResponse?.giteaConfig?.url &&
|
||||
configResponse?.giteaConfig?.username &&
|
||||
configResponse?.giteaConfig?.token
|
||||
);
|
||||
|
||||
|
||||
+14
-4
@@ -87,12 +87,22 @@ export const githubApi = {
|
||||
};
|
||||
|
||||
// Gitea API
|
||||
export interface GiteaServerInfo {
|
||||
type: "forgejo" | "gitea";
|
||||
version: string;
|
||||
raw: string;
|
||||
hasMirrorCredBug: boolean;
|
||||
}
|
||||
|
||||
export const giteaApi = {
|
||||
testConnection: (url: string, token: string) =>
|
||||
apiRequest<{ success: boolean }>("/gitea/test-connection", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ url, token }),
|
||||
}),
|
||||
apiRequest<{ success: boolean; serverInfo?: GiteaServerInfo; message?: string }>(
|
||||
"/gitea/test-connection",
|
||||
{
|
||||
method: "POST",
|
||||
body: JSON.stringify({ url, token }),
|
||||
}
|
||||
),
|
||||
};
|
||||
|
||||
// Health API
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
*/
|
||||
|
||||
import { db, configs, users } from '@/lib/db';
|
||||
import { eq, and } from 'drizzle-orm';
|
||||
import { eq, and, sql } from 'drizzle-orm';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import { encrypt } from '@/lib/utils/encryption';
|
||||
|
||||
@@ -224,10 +224,12 @@ export async function initializeConfigFromEnv(): Promise<void> {
|
||||
|
||||
console.log('[ENV Config Loader] Found environment configuration, initializing...');
|
||||
|
||||
// Get the first user (admin user)
|
||||
// Get the first user (admin user) — deterministic order so we always pick the
|
||||
// same row across restarts even if multiple users exist.
|
||||
const firstUser = await db
|
||||
.select()
|
||||
.from(users)
|
||||
.orderBy(sql`${users.createdAt} ASC`)
|
||||
.limit(1);
|
||||
|
||||
if (firstUser.length === 0) {
|
||||
@@ -237,11 +239,14 @@ export async function initializeConfigFromEnv(): Promise<void> {
|
||||
|
||||
const userId = firstUser[0].id;
|
||||
|
||||
// Check if config already exists for this user
|
||||
// Check if config already exists for this user — prefer the active config and
|
||||
// fall back to most-recently-updated so we never write env values into a stale
|
||||
// inactive stub while the populated active row sits untouched (see issue #271).
|
||||
const existingConfig = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
// Determine mirror strategy based on environment variables or use explicit value
|
||||
@@ -308,6 +313,13 @@ export async function initializeConfigFromEnv(): Promise<void> {
|
||||
mirrorPullRequests: envConfig.mirror.mirrorPullRequests ?? existingConfig?.[0]?.giteaConfig?.mirrorPullRequests ?? false,
|
||||
mirrorLabels: envConfig.mirror.mirrorLabels ?? existingConfig?.[0]?.giteaConfig?.mirrorLabels ?? false,
|
||||
mirrorMilestones: envConfig.mirror.mirrorMilestones ?? existingConfig?.[0]?.giteaConfig?.mirrorMilestones ?? false,
|
||||
// Backup options — preserve existing values so UI-configured settings survive restart
|
||||
backupStrategy: existingConfig?.[0]?.giteaConfig?.backupStrategy ?? 'on-force-push',
|
||||
backupBeforeSync: existingConfig?.[0]?.giteaConfig?.backupBeforeSync ?? true,
|
||||
backupRetentionCount: existingConfig?.[0]?.giteaConfig?.backupRetentionCount ?? 5,
|
||||
backupRetentionDays: existingConfig?.[0]?.giteaConfig?.backupRetentionDays ?? 30,
|
||||
backupDirectory: existingConfig?.[0]?.giteaConfig?.backupDirectory || undefined,
|
||||
blockSyncOnBackupFailure: existingConfig?.[0]?.giteaConfig?.blockSyncOnBackupFailure ?? true,
|
||||
};
|
||||
|
||||
// Build schedule config with support for interval as string or number
|
||||
|
||||
@@ -789,7 +789,7 @@ describe("Enhanced Gitea Operations", () => {
|
||||
expect(mockMirrorGitRepoLabelsToGitea).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("continues incremental issue and PR syncing when metadata was previously synced", async () => {
|
||||
test("skips issues and PRs when metadata shows they were already synced", async () => {
|
||||
const config: Partial<Config> = {
|
||||
userId: "user123",
|
||||
githubConfig: {
|
||||
@@ -848,9 +848,10 @@ describe("Enhanced Gitea Operations", () => {
|
||||
}
|
||||
);
|
||||
|
||||
// All metadata components were previously synced, so none should be called again
|
||||
expect(mockMirrorGitHubReleasesToGitea).not.toHaveBeenCalled();
|
||||
expect(mockMirrorGitRepoIssuesToGitea).toHaveBeenCalledTimes(1);
|
||||
expect(mockMirrorGitRepoPullRequestsToGitea).toHaveBeenCalledTimes(1);
|
||||
expect(mockMirrorGitRepoIssuesToGitea).not.toHaveBeenCalled();
|
||||
expect(mockMirrorGitRepoPullRequestsToGitea).not.toHaveBeenCalled();
|
||||
expect(mockMirrorGitRepoLabelsToGitea).not.toHaveBeenCalled();
|
||||
expect(mockMirrorGitRepoMilestonesToGitea).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -8,7 +8,8 @@
|
||||
|
||||
import type { Config } from "@/types/config";
|
||||
import type { Repository } from "./db/schema";
|
||||
import { Octokit } from "@octokit/rest";
|
||||
import type { Octokit } from "@octokit/rest";
|
||||
import { createGitHubClient } from "./github";
|
||||
import { createMirrorJob } from "./helpers";
|
||||
import { decryptConfigTokens } from "./utils/config-encryption";
|
||||
import { httpPost, httpGet, httpPatch, HttpError } from "./http-client";
|
||||
@@ -431,7 +432,7 @@ export async function syncGiteaRepoEnhanced({
|
||||
try {
|
||||
const decryptedGithubToken = decryptedConfig.githubConfig?.token;
|
||||
if (decryptedGithubToken) {
|
||||
const fpOctokit = new Octokit({ auth: decryptedGithubToken });
|
||||
const fpOctokit = createGitHubClient(decryptedGithubToken);
|
||||
const detectionResult = await detectForcePush({
|
||||
giteaUrl: config.giteaConfig.url,
|
||||
giteaToken: decryptedConfig.giteaConfig.token,
|
||||
@@ -556,6 +557,9 @@ export async function syncGiteaRepoEnhanced({
|
||||
}
|
||||
|
||||
// Update mirror interval if needed
|
||||
// NOTE: Gitea/Forgejo's PATCH /repos/{owner}/{repo} API does not support
|
||||
// updating mirror credentials (mirror_username/mirror_password). Repos that
|
||||
// were originally migrated without credentials must be deleted and re-mirrored.
|
||||
if (config.giteaConfig?.mirrorInterval) {
|
||||
try {
|
||||
console.log(`[Sync] Updating mirror interval for ${repoOwner}/${repoName} to ${config.giteaConfig.mirrorInterval}`);
|
||||
@@ -593,9 +597,7 @@ export async function syncGiteaRepoEnhanced({
|
||||
if (!decryptedConfig.githubConfig?.token) {
|
||||
return null;
|
||||
}
|
||||
metadataOctokit = new Octokit({
|
||||
auth: decryptedConfig.githubConfig.token,
|
||||
});
|
||||
metadataOctokit = createGitHubClient(decryptedConfig.githubConfig.token);
|
||||
return metadataOctokit;
|
||||
};
|
||||
|
||||
@@ -603,10 +605,12 @@ export async function syncGiteaRepoEnhanced({
|
||||
!!config.giteaConfig?.mirrorReleases && !skipMetadataForStarred;
|
||||
const shouldMirrorIssuesThisRun =
|
||||
!!config.giteaConfig?.mirrorIssues &&
|
||||
!skipMetadataForStarred;
|
||||
!skipMetadataForStarred &&
|
||||
!metadataState.components.issues;
|
||||
const shouldMirrorPullRequests =
|
||||
!!config.giteaConfig?.mirrorPullRequests &&
|
||||
!skipMetadataForStarred;
|
||||
!skipMetadataForStarred &&
|
||||
!metadataState.components.pullRequests;
|
||||
const shouldMirrorLabels =
|
||||
!!config.giteaConfig?.mirrorLabels &&
|
||||
!skipMetadataForStarred &&
|
||||
@@ -680,6 +684,13 @@ export async function syncGiteaRepoEnhanced({
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if (
|
||||
config.giteaConfig?.mirrorIssues &&
|
||||
metadataState.components.issues
|
||||
) {
|
||||
console.log(
|
||||
`[Sync] Issues already mirrored for ${repository.name}; skipping to avoid duplicates`
|
||||
);
|
||||
}
|
||||
|
||||
if (shouldMirrorPullRequests) {
|
||||
@@ -710,6 +721,13 @@ export async function syncGiteaRepoEnhanced({
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if (
|
||||
config.giteaConfig?.mirrorPullRequests &&
|
||||
metadataState.components.pullRequests
|
||||
) {
|
||||
console.log(
|
||||
`[Sync] Pull requests already mirrored for ${repository.name}; skipping`
|
||||
);
|
||||
}
|
||||
|
||||
if (shouldMirrorLabels) {
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
/**
|
||||
* Regression test for issue #268.
|
||||
*
|
||||
* `let migrateSucceeded = false;` was declared *inside* the try block
|
||||
* of mirrorGithubRepoToGitea and mirrorGitHubRepoToGiteaOrg, but the
|
||||
* catch block referenced it. `let` is block-scoped to the try, so any
|
||||
* error inside try made the catch crash with `ReferenceError:
|
||||
* migrateSucceeded is not defined` before reaching the DB update that
|
||||
* marks the repo "failed". Result: repos stuck in "mirroring" forever
|
||||
* with no entry in the activity log (see issue logs).
|
||||
*
|
||||
* This test asserts the declaration is hoisted above the try block in
|
||||
* both functions. It deliberately reads the source rather than calling
|
||||
* the functions, because behavioral tests for these functions require
|
||||
* heavy module mocks that pollute other test files (bun's mock.module
|
||||
* is process-wide and persists across files).
|
||||
*/
|
||||
import { describe, test, expect } from "bun:test";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
const SOURCE = readFileSync(
|
||||
join(import.meta.dir, "gitea.ts"),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
/**
|
||||
* Locate the body of a function declaration by name. Walks from the
|
||||
* declaration, balances parens to skip the parameter list (which can
|
||||
* contain destructured object literals with their own braces), then
|
||||
* finds the body's opening brace and its matching close.
|
||||
*/
|
||||
function extractFunctionBody(source: string, declarationStart: RegExp): string {
|
||||
const match = source.match(declarationStart);
|
||||
if (!match) {
|
||||
throw new Error(`Could not locate declaration ${declarationStart}`);
|
||||
}
|
||||
let i = match.index! + match[0].length;
|
||||
// Skip whitespace until the opening paren of the parameter list.
|
||||
while (i < source.length && source[i] !== "(") i++;
|
||||
if (source[i] !== "(") {
|
||||
throw new Error(`No '(' after ${declarationStart}`);
|
||||
}
|
||||
// Balance parens to find the end of the parameter list. Braces inside
|
||||
// the parameter list (e.g. destructured `{ foo, bar }`) are allowed
|
||||
// and ignored.
|
||||
let parenDepth = 0;
|
||||
for (; i < source.length; i++) {
|
||||
if (source[i] === "(") parenDepth++;
|
||||
else if (source[i] === ")") {
|
||||
parenDepth--;
|
||||
if (parenDepth === 0) {
|
||||
i++;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
// Skip return-type annotation, => arrow, whitespace, until the body's `{`.
|
||||
while (i < source.length && source[i] !== "{") i++;
|
||||
if (source[i] !== "{") {
|
||||
throw new Error(`No body '{' for ${declarationStart}`);
|
||||
}
|
||||
// Balance braces for the body.
|
||||
let braceDepth = 0;
|
||||
const startIdx = i;
|
||||
for (; i < source.length; i++) {
|
||||
if (source[i] === "{") braceDepth++;
|
||||
else if (source[i] === "}") {
|
||||
braceDepth--;
|
||||
if (braceDepth === 0) {
|
||||
return source.slice(startIdx, i + 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
throw new Error(`Unterminated body for ${declarationStart}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Confirm that within a function body, the first `let migrateSucceeded`
|
||||
* declaration occurs BEFORE the function's outermost `try {`.
|
||||
*
|
||||
* If the declaration is inside the try block, the catch block can't see
|
||||
* it (ReferenceError in production = repo stuck mirroring).
|
||||
*/
|
||||
function assertMigrateSucceededDeclaredBeforeTry(body: string, label: string) {
|
||||
const declIdx = body.indexOf("let migrateSucceeded");
|
||||
expect(declIdx, `${label}: 'let migrateSucceeded' should exist`).toBeGreaterThanOrEqual(0);
|
||||
|
||||
// The function's outermost try is the first standalone `try {` in
|
||||
// the body — assignments and inner try/catches don't share its name.
|
||||
const tryIdx = body.search(/\btry\s*\{/);
|
||||
expect(tryIdx, `${label}: outermost 'try {' should exist`).toBeGreaterThanOrEqual(0);
|
||||
|
||||
expect(
|
||||
declIdx,
|
||||
`${label}: 'let migrateSucceeded' must be declared BEFORE the try block ` +
|
||||
`so the catch block can read it. If declared inside try, it's block-scoped ` +
|
||||
`and the catch will throw ReferenceError, leaving repos stuck in 'mirroring'. ` +
|
||||
`See issue #268.`
|
||||
).toBeLessThan(tryIdx);
|
||||
|
||||
// And it should still be assigned to true after the migrate call —
|
||||
// otherwise the catch can't tell whether to clear mirroredLocation.
|
||||
expect(
|
||||
body.includes("migrateSucceeded = true"),
|
||||
`${label}: 'migrateSucceeded = true' assignment should exist after the migrate call`
|
||||
).toBe(true);
|
||||
|
||||
// And the catch must read it.
|
||||
expect(
|
||||
body.includes("if (!migrateSucceeded)"),
|
||||
`${label}: catch block should read 'migrateSucceeded' to decide whether to clear mirroredLocation`
|
||||
).toBe(true);
|
||||
}
|
||||
|
||||
describe("issue #268 — migrateSucceeded scoping regression", () => {
|
||||
test("mirrorGithubRepoToGitea declares migrateSucceeded outside try", () => {
|
||||
const body = extractFunctionBody(
|
||||
SOURCE,
|
||||
/export const mirrorGithubRepoToGitea = async\b/
|
||||
);
|
||||
assertMigrateSucceededDeclaredBeforeTry(body, "mirrorGithubRepoToGitea");
|
||||
});
|
||||
|
||||
test("mirrorGitHubRepoToGiteaOrg declares migrateSucceeded outside try", () => {
|
||||
const body = extractFunctionBody(
|
||||
SOURCE,
|
||||
/export async function mirrorGitHubRepoToGiteaOrg\b/
|
||||
);
|
||||
assertMigrateSucceededDeclaredBeforeTry(body, "mirrorGitHubRepoToGiteaOrg");
|
||||
});
|
||||
});
|
||||
+83
-11
@@ -539,6 +539,11 @@ export const mirrorGithubRepoToGitea = async ({
|
||||
repository: Repository;
|
||||
config: Partial<Config>;
|
||||
}): Promise<any> => {
|
||||
// Declared here (not inside try) so the catch block can read it.
|
||||
// `let` is block-scoped — declaring inside try makes it inaccessible
|
||||
// from catch, which previously caused a ReferenceError that swallowed
|
||||
// the real error and left repos stuck in "mirroring" state.
|
||||
let migrateSucceeded = false;
|
||||
try {
|
||||
if (!config.userId || !config.githubConfig || !config.giteaConfig) {
|
||||
throw new Error("github config and gitea config are required.");
|
||||
@@ -815,8 +820,10 @@ export const mirrorGithubRepoToGitea = async ({
|
||||
service: "git",
|
||||
};
|
||||
|
||||
// Add authentication for private repositories
|
||||
if (repository.isPrivate) {
|
||||
// Always send authentication credentials so Gitea/Forgejo stores them
|
||||
// for subsequent mirror fetches. This prevents "terminal prompts disabled"
|
||||
// errors on public repos and raises GitHub API rate limits.
|
||||
{
|
||||
const githubOwner =
|
||||
(
|
||||
config.githubConfig as typeof config.githubConfig & {
|
||||
@@ -835,10 +842,6 @@ export const mirrorGithubRepoToGitea = async ({
|
||||
);
|
||||
}
|
||||
|
||||
// Track whether the Gitea migrate call succeeded so the catch block
|
||||
// knows whether to clear mirroredLocation (only safe before migrate succeeds)
|
||||
let migrateSucceeded = false;
|
||||
|
||||
const response = await httpPost(
|
||||
apiUrl,
|
||||
migratePayload,
|
||||
@@ -1319,6 +1322,9 @@ export async function mirrorGitHubRepoToGiteaOrg({
|
||||
giteaOrgId: number;
|
||||
orgName: string;
|
||||
}) {
|
||||
// Declared here (not inside try) so the catch block can read it.
|
||||
// See note in mirrorGithubRepoToGitea for the scoping bug this prevents.
|
||||
let migrateSucceeded = false;
|
||||
try {
|
||||
if (
|
||||
!config.giteaConfig?.url ||
|
||||
@@ -1501,10 +1507,13 @@ export async function mirrorGitHubRepoToGiteaOrg({
|
||||
lfs: config.giteaConfig?.lfs || false,
|
||||
private: repository.isPrivate,
|
||||
description: repository.description?.trim() || "",
|
||||
service: "git",
|
||||
};
|
||||
|
||||
// Add authentication for private repositories
|
||||
if (repository.isPrivate) {
|
||||
// Always send authentication credentials so Gitea/Forgejo stores them
|
||||
// for subsequent mirror fetches. This prevents "terminal prompts disabled"
|
||||
// errors on public repos and raises GitHub API rate limits.
|
||||
{
|
||||
const githubOwner =
|
||||
(
|
||||
config.githubConfig as typeof config.githubConfig & {
|
||||
@@ -1523,8 +1532,6 @@ export async function mirrorGitHubRepoToGiteaOrg({
|
||||
);
|
||||
}
|
||||
|
||||
let migrateSucceeded = false;
|
||||
|
||||
const migrateRes = await httpPost(
|
||||
apiUrl,
|
||||
migratePayload,
|
||||
@@ -2715,7 +2722,7 @@ export async function mirrorGitHubReleasesToGitea({
|
||||
if (existingNote !== releaseNote || existingRelease.name !== (release.name || release.tag_name)) {
|
||||
console.log(`[Releases] Updating existing release ${release.tag_name} with new changelog/title`);
|
||||
|
||||
await httpPut(
|
||||
await httpPatch(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${existingRelease.id}`,
|
||||
{
|
||||
tag_name: release.tag_name,
|
||||
@@ -2829,6 +2836,71 @@ export async function mirrorGitHubReleasesToGitea({
|
||||
}
|
||||
|
||||
console.log(`✅ Mirrored/Updated ${mirroredCount} releases to Gitea (${skippedCount} already up-to-date)`);
|
||||
|
||||
// Enforce release retention limit by removing the oldest excess releases from Gitea
|
||||
try {
|
||||
// Paginate to fetch ALL Gitea releases (API max is 100 per page)
|
||||
const allGiteaReleases: Array<{ id: number; tag_name: string; created_at: string }> = [];
|
||||
let cleanupPage = 1;
|
||||
while (true) {
|
||||
const pageResponse = await httpGet(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases?per_page=100&page=${cleanupPage}`,
|
||||
{
|
||||
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
|
||||
}
|
||||
).catch(() => null);
|
||||
|
||||
if (!pageResponse?.data || !Array.isArray(pageResponse.data) || pageResponse.data.length === 0) {
|
||||
break;
|
||||
}
|
||||
|
||||
allGiteaReleases.push(...pageResponse.data);
|
||||
|
||||
if (pageResponse.data.length < 100) {
|
||||
break;
|
||||
}
|
||||
cleanupPage++;
|
||||
}
|
||||
|
||||
if (allGiteaReleases.length > releaseLimit) {
|
||||
const excessCount = allGiteaReleases.length - releaseLimit;
|
||||
|
||||
// Sort by created_at ascending (oldest first) so we delete the oldest excess
|
||||
const sorted = [...allGiteaReleases].sort(
|
||||
(a, b) => new Date(a.created_at).getTime() - new Date(b.created_at).getTime()
|
||||
);
|
||||
|
||||
const toDelete = sorted.slice(0, excessCount);
|
||||
|
||||
console.log(
|
||||
`[Releases] Enforcing retention limit (${releaseLimit}): ${allGiteaReleases.length} releases found, removing ${toDelete.length} oldest excess release(s)`
|
||||
);
|
||||
|
||||
for (const excess of toDelete) {
|
||||
try {
|
||||
await httpDelete(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${excess.id}`,
|
||||
{
|
||||
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
|
||||
}
|
||||
);
|
||||
console.log(`[Releases] Deleted excess release: ${excess.tag_name}`);
|
||||
} catch (deleteError) {
|
||||
console.error(
|
||||
`[Releases] Failed to delete excess release ${excess.tag_name}: ${
|
||||
deleteError instanceof Error ? deleteError.message : String(deleteError)
|
||||
}`
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (cleanupError) {
|
||||
console.warn(
|
||||
`[Releases] Release retention cleanup failed: ${
|
||||
cleanupError instanceof Error ? cleanupError.message : String(cleanupError)
|
||||
}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export async function mirrorGitRepoPullRequestsToGitea({
|
||||
|
||||
@@ -3,7 +3,7 @@ import type { NotificationEvent } from "./providers/ntfy";
|
||||
import { sendNtfyNotification } from "./providers/ntfy";
|
||||
import { sendAppriseNotification } from "./providers/apprise";
|
||||
import { db, configs } from "@/lib/db";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { decrypt } from "@/lib/utils/encryption";
|
||||
|
||||
function sanitizeTestNotificationError(error: unknown): string {
|
||||
@@ -120,11 +120,13 @@ export async function triggerJobNotification({
|
||||
return;
|
||||
}
|
||||
|
||||
// Fetch user's config from database
|
||||
// Fetch user's config from database — prefer active and most-recently-updated
|
||||
// to avoid picking a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const configResults = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (configResults.length === 0) {
|
||||
|
||||
+7
-3
@@ -5,7 +5,7 @@
|
||||
|
||||
import { findInterruptedJobs, resumeInterruptedJob } from './helpers';
|
||||
import { db, repositories, organizations, mirrorJobs, configs } from './db';
|
||||
import { eq, and, lt, inArray } from 'drizzle-orm';
|
||||
import { eq, and, lt, inArray, sql } from 'drizzle-orm';
|
||||
import { mirrorGithubRepoToGitea, mirrorGitHubOrgRepoToGiteaOrg, syncGiteaRepo } from './gitea';
|
||||
import { createGitHubClient } from './github';
|
||||
import { processWithResilience } from './utils/concurrency';
|
||||
@@ -216,11 +216,13 @@ async function recoverMirrorJob(job: any, remainingItemIds: string[]) {
|
||||
console.log(`Recovering mirror job ${job.id} with ${remainingItemIds.length} remaining items`);
|
||||
|
||||
try {
|
||||
// Get the config for this user with better error handling
|
||||
// Get the config for this user — prefer active and most-recently-updated
|
||||
// to avoid picking a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const userConfigs = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, job.userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (userConfigs.length === 0) {
|
||||
@@ -347,11 +349,13 @@ async function recoverSyncJob(job: any, remainingItemIds: string[]) {
|
||||
console.log(`Recovering sync job ${job.id} with ${remainingItemIds.length} remaining items`);
|
||||
|
||||
try {
|
||||
// Get the config for this user with better error handling
|
||||
// Get the config for this user — prefer active and most-recently-updated
|
||||
// to avoid picking a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const userConfigs = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, job.userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (userConfigs.length === 0) {
|
||||
|
||||
@@ -58,6 +58,51 @@ describe("Scheduler Service - Ignored Repository Handling", () => {
|
||||
expect(shouldMirrorRepository(oldSyncedRepo)).toBe(true);
|
||||
});
|
||||
|
||||
test("auto-mirror filter respects autoMirror and autoMirrorStarred independently", () => {
|
||||
// Mirrors the inline filter at scheduler-service.ts L228-233 / L609-614:
|
||||
// a repo is "starred from another owner" iff isStarred && owner !== githubOwner.
|
||||
// Such repos are gated by autoMirrorStarred; everything else is gated by autoMirror.
|
||||
const githubOwner = "Alice".toLowerCase();
|
||||
const filterRepos = (
|
||||
repos: Array<{ name: string; isStarred: boolean; owner: string }>,
|
||||
autoMirror: boolean,
|
||||
autoMirrorStarred: boolean,
|
||||
) =>
|
||||
repos.filter(repo => {
|
||||
const isStarredFromOther = repo.isStarred && repo.owner.toLowerCase() !== githubOwner;
|
||||
return isStarredFromOther ? autoMirrorStarred : autoMirror;
|
||||
});
|
||||
|
||||
// "ALICE" tests case-insensitive owner match — GitHub usernames are case-insensitive,
|
||||
// so a self-starred repo stored with different casing must still count as owned.
|
||||
const repos = [
|
||||
{ name: "owned-repo", isStarred: false, owner: "alice" },
|
||||
{ name: "self-starred", isStarred: true, owner: "ALICE" },
|
||||
{ name: "starred-from-bob", isStarred: true, owner: "bob" },
|
||||
];
|
||||
|
||||
// Both off: nothing mirrors
|
||||
expect(filterRepos(repos, false, false).map(r => r.name)).toEqual([]);
|
||||
|
||||
// Only autoMirror: owned + self-starred, not third-party stars
|
||||
expect(filterRepos(repos, true, false).map(r => r.name)).toEqual([
|
||||
"owned-repo",
|
||||
"self-starred",
|
||||
]);
|
||||
|
||||
// Only autoMirrorStarred: just third-party stars (the bug fix — used to be empty)
|
||||
expect(filterRepos(repos, false, true).map(r => r.name)).toEqual([
|
||||
"starred-from-bob",
|
||||
]);
|
||||
|
||||
// Both on: everything
|
||||
expect(filterRepos(repos, true, true).map(r => r.name)).toEqual([
|
||||
"owned-repo",
|
||||
"self-starred",
|
||||
"starred-from-bob",
|
||||
]);
|
||||
});
|
||||
|
||||
test("should validate all repository status enum values", () => {
|
||||
const validStatuses = [
|
||||
"imported",
|
||||
|
||||
@@ -99,15 +99,14 @@ async function runScheduledSync(config: any): Promise<void> {
|
||||
if (scheduleConfig.autoImport !== false) {
|
||||
console.log(`[Scheduler] Checking for new GitHub repositories for user ${userId}...`);
|
||||
try {
|
||||
const { getGithubRepositories, getGithubStarredRepositories } = await import('@/lib/github');
|
||||
const { getGithubRepositories, getGithubStarredRepositories, createGitHubClient } = await import('@/lib/github');
|
||||
const { v4: uuidv4 } = await import('uuid');
|
||||
const { getDecryptedGitHubToken } = await import('@/lib/utils/config-encryption');
|
||||
|
||||
// Create GitHub client
|
||||
|
||||
// Create GitHub client (honors GH_API_URL for GHES / GHEC data residency)
|
||||
const decryptedToken = getDecryptedGitHubToken(config);
|
||||
const { Octokit } = await import('@octokit/rest');
|
||||
const octokit = new Octokit({ auth: decryptedToken });
|
||||
|
||||
const octokit = createGitHubClient(decryptedToken, userId, config.githubConfig?.owner);
|
||||
|
||||
// Fetch GitHub data
|
||||
const [basicAndForkedRepos, starredRepos] = await Promise.all([
|
||||
getGithubRepositories({ octokit, config }),
|
||||
@@ -117,7 +116,7 @@ async function runScheduledSync(config: any): Promise<void> {
|
||||
]);
|
||||
const allGithubRepos = mergeGitReposPreferStarred(basicAndForkedRepos, starredRepos);
|
||||
const mirrorableGithubRepos = allGithubRepos.filter(isMirrorableGitHubRepo);
|
||||
|
||||
|
||||
// Check for new repositories
|
||||
const existingRepos = await db
|
||||
.select({ normalizedFullName: repositories.normalizedFullName })
|
||||
@@ -204,10 +203,14 @@ async function runScheduledSync(config: any): Promise<void> {
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-mirror: Mirror imported/pending/failed repositories if enabled
|
||||
if (scheduleConfig.autoMirror) {
|
||||
// Auto-mirror: Mirror imported/pending/failed repositories if enabled.
|
||||
// autoMirror covers owned repos; autoMirrorStarred covers starred repos from other owners.
|
||||
// Either flag on its own is enough to enter this phase.
|
||||
const autoMirrorOwned = !!scheduleConfig.autoMirror;
|
||||
const autoMirrorStarred = !!config.githubConfig?.autoMirrorStarred;
|
||||
if (autoMirrorOwned || autoMirrorStarred) {
|
||||
try {
|
||||
console.log(`[Scheduler] Auto-mirror enabled - checking for repositories to mirror for user ${userId}...`);
|
||||
console.log(`[Scheduler] Auto-mirror enabled (owned=${autoMirrorOwned}, starred=${autoMirrorStarred}) - checking for repositories to mirror for user ${userId}...`);
|
||||
let reposNeedingMirror = await db
|
||||
.select()
|
||||
.from(repositories)
|
||||
@@ -222,26 +225,25 @@ async function runScheduledSync(config: any): Promise<void> {
|
||||
)
|
||||
);
|
||||
|
||||
// Filter out starred repos from auto-mirror when autoMirrorStarred is disabled
|
||||
if (!config.githubConfig?.autoMirrorStarred) {
|
||||
const githubOwner = config.githubConfig?.owner || '';
|
||||
const beforeCount = reposNeedingMirror.length;
|
||||
reposNeedingMirror = reposNeedingMirror.filter(
|
||||
repo => !repo.isStarred || repo.owner === githubOwner
|
||||
);
|
||||
const skippedCount = beforeCount - reposNeedingMirror.length;
|
||||
if (skippedCount > 0) {
|
||||
console.log(`[Scheduler] Skipped ${skippedCount} starred repositories from auto-mirror (autoMirrorStarred is disabled)`);
|
||||
}
|
||||
const githubOwner = (config.githubConfig?.owner || '').toLowerCase();
|
||||
const beforeCount = reposNeedingMirror.length;
|
||||
reposNeedingMirror = reposNeedingMirror.filter(repo => {
|
||||
// GitHub usernames are case-insensitive; lowercase both sides to avoid misclassifying self-starred repos.
|
||||
const isStarredFromOther = repo.isStarred && repo.owner.toLowerCase() !== githubOwner;
|
||||
return isStarredFromOther ? autoMirrorStarred : autoMirrorOwned;
|
||||
});
|
||||
const skippedCount = beforeCount - reposNeedingMirror.length;
|
||||
if (skippedCount > 0) {
|
||||
console.log(`[Scheduler] Skipped ${skippedCount} repositories from auto-mirror (autoMirror=${autoMirrorOwned}, autoMirrorStarred=${autoMirrorStarred})`);
|
||||
}
|
||||
|
||||
if (reposNeedingMirror.length > 0) {
|
||||
console.log(`[Scheduler] Found ${reposNeedingMirror.length} repositories that need initial mirroring`);
|
||||
|
||||
// Prepare Octokit client
|
||||
// Prepare Octokit client (honors GH_API_URL for GHES / GHEC data residency)
|
||||
const decryptedToken = getDecryptedGitHubToken(config);
|
||||
const { Octokit } = await import('@octokit/rest');
|
||||
const octokit = new Octokit({ auth: decryptedToken });
|
||||
const { createGitHubClient } = await import('@/lib/github');
|
||||
const octokit = createGitHubClient(decryptedToken, userId, config.githubConfig?.owner);
|
||||
|
||||
// Process repositories in batches
|
||||
const batchSize = scheduleConfig.batchSize || 10;
|
||||
@@ -482,13 +484,12 @@ async function performInitialAutoStart(): Promise<void> {
|
||||
try {
|
||||
// Step 1: Import repositories from GitHub
|
||||
console.log(`[Scheduler] Step 1: Importing repositories from GitHub for user ${config.userId}...`);
|
||||
const { getGithubRepositories, getGithubStarredRepositories } = await import('@/lib/github');
|
||||
const { getGithubRepositories, getGithubStarredRepositories, createGitHubClient } = await import('@/lib/github');
|
||||
const { v4: uuidv4 } = await import('uuid');
|
||||
|
||||
// Create GitHub client
|
||||
|
||||
// Create GitHub client (honors GH_API_URL for GHES / GHEC data residency)
|
||||
const decryptedToken = getDecryptedGitHubToken(config);
|
||||
const { Octokit } = await import('@octokit/rest');
|
||||
const octokit = new Octokit({ auth: decryptedToken });
|
||||
const octokit = createGitHubClient(decryptedToken, config.userId, config.githubConfig?.owner);
|
||||
|
||||
// Fetch GitHub data
|
||||
const [basicAndForkedRepos, starredRepos] = await Promise.all([
|
||||
@@ -576,10 +577,12 @@ async function performInitialAutoStart(): Promise<void> {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Step 2: Trigger mirror for all repositories that need mirroring
|
||||
// Only auto-mirror if autoMirror is enabled in schedule config
|
||||
if (!config.scheduleConfig?.autoMirror) {
|
||||
console.log(`[Scheduler] Step 2: Skipping initial mirror - autoMirror is disabled for user ${config.userId}`);
|
||||
// Step 2: Trigger mirror for all repositories that need mirroring.
|
||||
// autoMirror covers owned repos; autoMirrorStarred covers starred repos from other owners.
|
||||
const autoMirrorOwned = !!config.scheduleConfig?.autoMirror;
|
||||
const autoMirrorStarred = !!config.githubConfig?.autoMirrorStarred;
|
||||
if (!autoMirrorOwned && !autoMirrorStarred) {
|
||||
console.log(`[Scheduler] Step 2: Skipping initial mirror - autoMirror and autoMirrorStarred are both disabled for user ${config.userId}`);
|
||||
|
||||
// Still update schedule config timestamps
|
||||
const currentTime2 = new Date();
|
||||
@@ -589,7 +592,7 @@ async function performInitialAutoStart(): Promise<void> {
|
||||
continue;
|
||||
}
|
||||
|
||||
console.log(`[Scheduler] Step 2: Triggering mirror for repositories that need mirroring...`);
|
||||
console.log(`[Scheduler] Step 2: Triggering mirror for repositories that need mirroring (owned=${autoMirrorOwned}, starred=${autoMirrorStarred})...`);
|
||||
let reposNeedingMirror = await db
|
||||
.select()
|
||||
.from(repositories)
|
||||
@@ -604,17 +607,16 @@ async function performInitialAutoStart(): Promise<void> {
|
||||
)
|
||||
);
|
||||
|
||||
// Filter out starred repos from auto-mirror when autoMirrorStarred is disabled
|
||||
if (!config.githubConfig?.autoMirrorStarred) {
|
||||
const githubOwner = config.githubConfig?.owner || '';
|
||||
const beforeCount = reposNeedingMirror.length;
|
||||
reposNeedingMirror = reposNeedingMirror.filter(
|
||||
repo => !repo.isStarred || repo.owner === githubOwner
|
||||
);
|
||||
const skippedCount = beforeCount - reposNeedingMirror.length;
|
||||
if (skippedCount > 0) {
|
||||
console.log(`[Scheduler] Skipped ${skippedCount} starred repositories from initial auto-mirror (autoMirrorStarred is disabled)`);
|
||||
}
|
||||
const githubOwner = (config.githubConfig?.owner || '').toLowerCase();
|
||||
const beforeCount = reposNeedingMirror.length;
|
||||
reposNeedingMirror = reposNeedingMirror.filter(repo => {
|
||||
// GitHub usernames are case-insensitive; lowercase both sides to avoid misclassifying self-starred repos.
|
||||
const isStarredFromOther = repo.isStarred && repo.owner.toLowerCase() !== githubOwner;
|
||||
return isStarredFromOther ? autoMirrorStarred : autoMirrorOwned;
|
||||
});
|
||||
const skippedCount = beforeCount - reposNeedingMirror.length;
|
||||
if (skippedCount > 0) {
|
||||
console.log(`[Scheduler] Skipped ${skippedCount} repositories from initial auto-mirror (autoMirror=${autoMirrorOwned}, autoMirrorStarred=${autoMirrorStarred})`);
|
||||
}
|
||||
|
||||
if (reposNeedingMirror.length > 0) {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { db, configs } from "@/lib/db";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { v4 as uuidv4 } from "uuid";
|
||||
import { encrypt } from "@/lib/utils/encryption";
|
||||
import { getNextScheduledRun, normalizeTimezone } from "@/lib/utils/schedule-utils";
|
||||
@@ -25,11 +25,13 @@ export interface DefaultConfigOptions {
|
||||
* Environment variables can override these defaults
|
||||
*/
|
||||
export async function createDefaultConfig({ userId, envOverrides = {} }: DefaultConfigOptions) {
|
||||
// Check if config already exists
|
||||
// Check if config already exists — prefer active and most-recently-updated
|
||||
// to avoid returning a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const existingConfig = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (existingConfig.length > 0) {
|
||||
|
||||
@@ -52,12 +52,12 @@ describe("buildGithubSourceAuthPayload", () => {
|
||||
expect(auth.auth_token).toBe("ghp_trimmed");
|
||||
});
|
||||
|
||||
test("throws when token is missing", () => {
|
||||
expect(() =>
|
||||
buildGithubSourceAuthPayload({
|
||||
token: " ",
|
||||
githubUsername: "user",
|
||||
})
|
||||
).toThrow("GitHub token is required to mirror private repositories.");
|
||||
test("returns empty object when token is missing", () => {
|
||||
const result = buildGithubSourceAuthPayload({
|
||||
token: " ",
|
||||
githubUsername: "user",
|
||||
});
|
||||
|
||||
expect(result).toEqual({});
|
||||
});
|
||||
});
|
||||
@@ -11,6 +11,8 @@ export interface GithubSourceAuthPayload {
|
||||
auth_token: string;
|
||||
}
|
||||
|
||||
export type GithubSourceAuthPayloadOrEmpty = GithubSourceAuthPayload | Record<string, never>;
|
||||
|
||||
const DEFAULT_GITHUB_AUTH_USERNAME = "x-access-token";
|
||||
|
||||
function normalize(value?: string | null): string {
|
||||
@@ -18,18 +20,19 @@ function normalize(value?: string | null): string {
|
||||
}
|
||||
|
||||
/**
|
||||
* Build source credentials for private GitHub repository mirroring.
|
||||
* Build source credentials for GitHub repository mirroring.
|
||||
* GitHub expects username + token-as-password over HTTPS (not the GitLab-style "oauth2" username).
|
||||
* Returns an empty object when no token is available, allowing callers to use it unconditionally.
|
||||
*/
|
||||
export function buildGithubSourceAuthPayload({
|
||||
token,
|
||||
githubOwner,
|
||||
githubUsername,
|
||||
repositoryOwner,
|
||||
}: BuildGithubSourceAuthPayloadParams): GithubSourceAuthPayload {
|
||||
}: BuildGithubSourceAuthPayloadParams): GithubSourceAuthPayloadOrEmpty {
|
||||
const normalizedToken = normalize(token);
|
||||
if (!normalizedToken) {
|
||||
throw new Error("GitHub token is required to mirror private repositories.");
|
||||
return {};
|
||||
}
|
||||
|
||||
const authUsername =
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db, configs, users } from "@/lib/db";
|
||||
import { v4 as uuidv4 } from "uuid";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { createSecureErrorResponse } from "@/lib/utils";
|
||||
import {
|
||||
mapUiToDbConfig,
|
||||
@@ -83,11 +83,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch existing config
|
||||
// Fetch existing config — prefer the active config; fall back to most-recently-updated
|
||||
// so a stale inactive stub never wins over a populated active row (see issue #271).
|
||||
const existingConfigResult = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
const existingConfig = existingConfigResult[0];
|
||||
@@ -255,11 +257,14 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
if ("response" in authResult) return authResult.response;
|
||||
const userId = authResult.userId;
|
||||
|
||||
// Fetch the configuration for the user
|
||||
// Fetch the configuration for the user — prefer the active config; fall back to
|
||||
// most-recently-updated so a stale inactive stub never wins over a populated
|
||||
// active row (see issue #271).
|
||||
const config = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (config.length === 0) {
|
||||
|
||||
@@ -38,7 +38,12 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
.where(eq(mirrorJobs.userId, userId))
|
||||
.orderBy(sql`${mirrorJobs.timestamp} DESC`)
|
||||
.limit(10),
|
||||
db.select().from(configs).where(eq(configs.userId, userId)).limit(1),
|
||||
db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1),
|
||||
db
|
||||
.select({ value: count() })
|
||||
.from(repositories)
|
||||
|
||||
@@ -2,6 +2,25 @@ import type { APIRoute } from 'astro';
|
||||
import { httpGet, HttpError } from '@/lib/http-client';
|
||||
import { createSecureErrorResponse } from '@/lib/utils';
|
||||
|
||||
// Forgejo reports `15.0.0+gitea-1.22.0`; pure Gitea reports just `1.22.0`.
|
||||
// Forgejo < 15.0.0 has a known bug where pull-mirror credentials sent via
|
||||
// /api/v1/repos/migrate are not persisted, so subsequent sync of private
|
||||
// repos fails with `terminal prompts disabled`. Fixed upstream in v15.0.0
|
||||
// via PR #11909 (codeberg.org/forgejo/forgejo/pulls/11909).
|
||||
function parseServerInfo(versionString: string) {
|
||||
const forgejoMatch = versionString.match(/^(\d+)\.(\d+)\.(\d+)\+gitea-/);
|
||||
if (forgejoMatch) {
|
||||
const major = Number(forgejoMatch[1]);
|
||||
return {
|
||||
type: 'forgejo' as const,
|
||||
version: `${forgejoMatch[1]}.${forgejoMatch[2]}.${forgejoMatch[3]}`,
|
||||
raw: versionString,
|
||||
hasMirrorCredBug: major < 15,
|
||||
};
|
||||
}
|
||||
return { type: 'gitea' as const, version: versionString, raw: versionString, hasMirrorCredBug: false };
|
||||
}
|
||||
|
||||
export const POST: APIRoute = async ({ request }) => {
|
||||
try {
|
||||
const body = await request.json();
|
||||
@@ -49,7 +68,18 @@ export const POST: APIRoute = async ({ request }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Return success response with user data
|
||||
let serverInfo: ReturnType<typeof parseServerInfo> | undefined;
|
||||
try {
|
||||
const versionResp = await httpGet(`${baseUrl}/api/v1/version`, {
|
||||
'Accept': 'application/json',
|
||||
});
|
||||
if (typeof versionResp.data?.version === 'string') {
|
||||
serverInfo = parseServerInfo(versionResp.data.version);
|
||||
}
|
||||
} catch {
|
||||
// Version probe is best-effort; older or non-standard servers may not expose it.
|
||||
}
|
||||
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: true,
|
||||
@@ -59,6 +89,7 @@ export const POST: APIRoute = async ({ request }) => {
|
||||
name: data.full_name,
|
||||
avatar_url: data.avatar_url,
|
||||
},
|
||||
serverInfo,
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db, configs } from "@/lib/db";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import {
|
||||
createGitHubClient,
|
||||
getGithubStarredListNames,
|
||||
@@ -15,10 +15,13 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
if ("response" in authResult) return authResult.response;
|
||||
const userId = authResult.userId;
|
||||
|
||||
// Prefer active and most-recently-updated config to avoid picking a stale
|
||||
// inactive stub when multiple rows exist (see issue #271).
|
||||
const [config] = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (!config) {
|
||||
|
||||
@@ -1,39 +1,51 @@
|
||||
import { describe, test, expect, mock, beforeEach, afterEach } from "bun:test";
|
||||
import { POST } from "./test-connection";
|
||||
import { Octokit } from "@octokit/rest";
|
||||
|
||||
// Mock the Octokit class
|
||||
mock.module("@octokit/rest", () => {
|
||||
// createGitHubClient returns this stub. Tests mutate `getAuthenticatedImpl`
|
||||
// to steer the behavior without re-calling mock.module (which is fragile
|
||||
// once the route module has already captured a live binding).
|
||||
let getAuthenticatedImpl: () => Promise<any> = () =>
|
||||
Promise.resolve({
|
||||
data: {
|
||||
login: "testuser",
|
||||
name: "Test User",
|
||||
avatar_url: "https://example.com/avatar.png",
|
||||
},
|
||||
});
|
||||
|
||||
mock.module("@/lib/github", () => {
|
||||
return {
|
||||
Octokit: mock(function() {
|
||||
return {
|
||||
users: {
|
||||
getAuthenticated: mock(() => Promise.resolve({
|
||||
data: {
|
||||
login: "testuser",
|
||||
name: "Test User",
|
||||
avatar_url: "https://example.com/avatar.png"
|
||||
}
|
||||
}))
|
||||
}
|
||||
};
|
||||
})
|
||||
createGitHubClient: mock(() => ({
|
||||
users: {
|
||||
getAuthenticated: mock(() => getAuthenticatedImpl()),
|
||||
},
|
||||
})),
|
||||
};
|
||||
});
|
||||
|
||||
import { POST } from "./test-connection";
|
||||
|
||||
describe("GitHub Test Connection API", () => {
|
||||
// Mock console.error to prevent test output noise
|
||||
let originalConsoleError: typeof console.error;
|
||||
|
||||
|
||||
beforeEach(() => {
|
||||
originalConsoleError = console.error;
|
||||
console.error = mock(() => {});
|
||||
// Reset to the success stub before each test so tests are independent
|
||||
getAuthenticatedImpl = () =>
|
||||
Promise.resolve({
|
||||
data: {
|
||||
login: "testuser",
|
||||
name: "Test User",
|
||||
avatar_url: "https://example.com/avatar.png",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
afterEach(() => {
|
||||
console.error = originalConsoleError;
|
||||
});
|
||||
|
||||
|
||||
test("returns 400 if token is missing", async () => {
|
||||
const request = new Request("http://localhost/api/github/test-connection", {
|
||||
method: "POST",
|
||||
@@ -42,16 +54,16 @@ describe("GitHub Test Connection API", () => {
|
||||
},
|
||||
body: JSON.stringify({})
|
||||
});
|
||||
|
||||
|
||||
const response = await POST({ request } as any);
|
||||
|
||||
|
||||
expect(response.status).toBe(400);
|
||||
|
||||
|
||||
const data = await response.json();
|
||||
expect(data.success).toBe(false);
|
||||
expect(data.message).toBe("GitHub token is required");
|
||||
});
|
||||
|
||||
|
||||
test("returns 200 with user data on successful connection", async () => {
|
||||
const request = new Request("http://localhost/api/github/test-connection", {
|
||||
method: "POST",
|
||||
@@ -62,11 +74,11 @@ describe("GitHub Test Connection API", () => {
|
||||
token: "valid-token"
|
||||
})
|
||||
});
|
||||
|
||||
|
||||
const response = await POST({ request } as any);
|
||||
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
|
||||
|
||||
const data = await response.json();
|
||||
expect(data.success).toBe(true);
|
||||
expect(data.message).toBe("Successfully connected to GitHub as testuser");
|
||||
@@ -76,7 +88,7 @@ describe("GitHub Test Connection API", () => {
|
||||
avatar_url: "https://example.com/avatar.png"
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
test("returns 400 if username doesn't match authenticated user", async () => {
|
||||
const request = new Request("http://localhost/api/github/test-connection", {
|
||||
method: "POST",
|
||||
@@ -88,29 +100,19 @@ describe("GitHub Test Connection API", () => {
|
||||
username: "differentuser"
|
||||
})
|
||||
});
|
||||
|
||||
|
||||
const response = await POST({ request } as any);
|
||||
|
||||
|
||||
expect(response.status).toBe(400);
|
||||
|
||||
|
||||
const data = await response.json();
|
||||
expect(data.success).toBe(false);
|
||||
expect(data.message).toBe("Token belongs to testuser, not differentuser");
|
||||
});
|
||||
|
||||
|
||||
test("handles authentication errors", async () => {
|
||||
// Mock Octokit to throw an error
|
||||
mock.module("@octokit/rest", () => {
|
||||
return {
|
||||
Octokit: mock(function() {
|
||||
return {
|
||||
users: {
|
||||
getAuthenticated: mock(() => Promise.reject(new Error("Bad credentials")))
|
||||
}
|
||||
};
|
||||
})
|
||||
};
|
||||
});
|
||||
// Swap the stub to throw an auth error for this test only
|
||||
getAuthenticatedImpl = () => Promise.reject(new Error("Bad credentials"));
|
||||
|
||||
const request = new Request("http://localhost/api/github/test-connection", {
|
||||
method: "POST",
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { Octokit } from "@octokit/rest";
|
||||
import { createGitHubClient } from "@/lib/github";
|
||||
import { createSecureErrorResponse } from "@/lib/utils";
|
||||
|
||||
export const POST: APIRoute = async ({ request }) => {
|
||||
@@ -22,10 +22,10 @@ export const POST: APIRoute = async ({ request }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Create an Octokit instance with the provided token
|
||||
const octokit = new Octokit({
|
||||
auth: token,
|
||||
});
|
||||
// Create an Octokit instance with the provided token.
|
||||
// Uses createGitHubClient so GH_API_URL / GITHUB_API_URL routes the call
|
||||
// to the correct endpoint for GHES / GHEC with data residency.
|
||||
const octokit = createGitHubClient(token);
|
||||
|
||||
// Test the connection by fetching the authenticated user
|
||||
const { data } = await octokit.users.getAuthenticated();
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db, configs, repositories } from "@/lib/db";
|
||||
import { and, eq, inArray } from "drizzle-orm";
|
||||
import { and, eq, inArray, sql } from "drizzle-orm";
|
||||
import { repositoryVisibilityEnum, repoStatusEnum } from "@/types/Repository";
|
||||
import { syncGiteaRepoEnhanced } from "@/lib/gitea-enhanced";
|
||||
import { createSecureErrorResponse } from "@/lib/utils";
|
||||
@@ -38,11 +38,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Fetch config
|
||||
// Fetch config — prefer active and most-recently-updated to avoid picking
|
||||
// a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const configResult = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
const config = configResult[0];
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import type { MirrorOrgRequest, MirrorOrgResponse } from "@/types/mirror";
|
||||
import { db, configs, organizations } from "@/lib/db";
|
||||
import { and, eq, inArray } from "drizzle-orm";
|
||||
import { and, eq, inArray, sql } from "drizzle-orm";
|
||||
import { createGitHubClient } from "@/lib/github";
|
||||
import { mirrorGitHubOrgToGitea } from "@/lib/gitea";
|
||||
import { repoStatusEnum } from "@/types/Repository";
|
||||
@@ -41,11 +41,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Fetch config
|
||||
// Fetch config — prefer active and most-recently-updated to avoid picking
|
||||
// a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const configResult = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
const config = configResult[0];
|
||||
|
||||
@@ -3,6 +3,46 @@ import type { MirrorRepoRequest } from "@/types/mirror";
|
||||
import { POST } from "./mirror-repo";
|
||||
|
||||
// Mock the database module
|
||||
const mockConfigRow = [{
|
||||
id: "config-id",
|
||||
userId: "user-id",
|
||||
githubConfig: {
|
||||
token: "github-token",
|
||||
preserveOrgStructure: false,
|
||||
mirrorIssues: false
|
||||
},
|
||||
giteaConfig: {
|
||||
url: "https://gitea.example.com",
|
||||
token: "gitea-token",
|
||||
username: "giteauser"
|
||||
}
|
||||
}];
|
||||
|
||||
const mockRepoRows = [
|
||||
{
|
||||
id: "repo-id-1",
|
||||
name: "test-repo-1",
|
||||
visibility: "public",
|
||||
status: "pending",
|
||||
organization: null,
|
||||
lastMirrored: null,
|
||||
errorMessage: null,
|
||||
forkedFrom: null,
|
||||
mirroredLocation: ""
|
||||
},
|
||||
{
|
||||
id: "repo-id-2",
|
||||
name: "test-repo-2",
|
||||
visibility: "public",
|
||||
status: "pending",
|
||||
organization: null,
|
||||
lastMirrored: null,
|
||||
errorMessage: null,
|
||||
forkedFrom: null,
|
||||
mirroredLocation: ""
|
||||
}
|
||||
];
|
||||
|
||||
const mockDb = {
|
||||
select: mock(() => ({
|
||||
from: mock((table: any) => ({
|
||||
@@ -10,47 +50,14 @@ const mockDb = {
|
||||
// Return config for configs table
|
||||
if (table === mockConfigs) {
|
||||
return {
|
||||
limit: mock(() => Promise.resolve([{
|
||||
id: "config-id",
|
||||
userId: "user-id",
|
||||
githubConfig: {
|
||||
token: "github-token",
|
||||
preserveOrgStructure: false,
|
||||
mirrorIssues: false
|
||||
},
|
||||
giteaConfig: {
|
||||
url: "https://gitea.example.com",
|
||||
token: "gitea-token",
|
||||
username: "giteauser"
|
||||
}
|
||||
}]))
|
||||
orderBy: mock(() => ({
|
||||
limit: mock(() => Promise.resolve(mockConfigRow))
|
||||
})),
|
||||
limit: mock(() => Promise.resolve(mockConfigRow))
|
||||
};
|
||||
}
|
||||
// Return repositories for repositories table
|
||||
return Promise.resolve([
|
||||
{
|
||||
id: "repo-id-1",
|
||||
name: "test-repo-1",
|
||||
visibility: "public",
|
||||
status: "pending",
|
||||
organization: null,
|
||||
lastMirrored: null,
|
||||
errorMessage: null,
|
||||
forkedFrom: null,
|
||||
mirroredLocation: ""
|
||||
},
|
||||
{
|
||||
id: "repo-id-2",
|
||||
name: "test-repo-2",
|
||||
visibility: "public",
|
||||
status: "pending",
|
||||
organization: null,
|
||||
lastMirrored: null,
|
||||
errorMessage: null,
|
||||
forkedFrom: null,
|
||||
mirroredLocation: ""
|
||||
}
|
||||
]);
|
||||
return Promise.resolve(mockRepoRows);
|
||||
})
|
||||
}))
|
||||
}))
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import type { MirrorRepoRequest, MirrorRepoResponse } from "@/types/mirror";
|
||||
import { db, configs, repositories } from "@/lib/db";
|
||||
import { and, eq, inArray } from "drizzle-orm";
|
||||
import { and, eq, inArray, sql } from "drizzle-orm";
|
||||
import { repositoryVisibilityEnum, repoStatusEnum } from "@/types/Repository";
|
||||
import {
|
||||
mirrorGithubRepoToGitea,
|
||||
@@ -43,11 +43,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Fetch config
|
||||
// Fetch config — prefer active and most-recently-updated to avoid picking
|
||||
// a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const configResult = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
const config = configResult[0];
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { and, eq, inArray } from "drizzle-orm";
|
||||
import { and, eq, inArray, sql } from "drizzle-orm";
|
||||
import { db, configs, repositories } from "@/lib/db";
|
||||
import { repositoryVisibilityEnum, repoStatusEnum } from "@/types/Repository";
|
||||
import type { ResetMetadataRequest, ResetMetadataResponse } from "@/types/reset-metadata";
|
||||
@@ -35,10 +35,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Prefer active and most-recently-updated config to avoid picking a stale
|
||||
// inactive stub when multiple rows exist (see issue #271).
|
||||
const configResult = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
const config = configResult[0];
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db, configs, repositories } from "@/lib/db";
|
||||
import { and, eq, inArray } from "drizzle-orm";
|
||||
import { and, eq, inArray, sql } from "drizzle-orm";
|
||||
import { getGiteaRepoOwnerAsync, isRepoPresentInGitea } from "@/lib/gitea";
|
||||
import {
|
||||
mirrorGithubRepoToGitea,
|
||||
@@ -45,11 +45,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Fetch user config
|
||||
// Fetch user config — prefer active and most-recently-updated to avoid picking
|
||||
// a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const configResult = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
const config = configResult[0];
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db, configs, repositories } from "@/lib/db";
|
||||
import { and, eq, or } from "drizzle-orm";
|
||||
import { and, eq, or, sql } from "drizzle-orm";
|
||||
import { repoStatusEnum, repositoryVisibilityEnum } from "@/types/Repository";
|
||||
import { isRepoPresentInGitea, syncGiteaRepo } from "@/lib/gitea";
|
||||
import type {
|
||||
@@ -19,11 +19,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
|
||||
await request.json().catch(() => ({} as ScheduleSyncRepoRequest));
|
||||
|
||||
// Fetch config for the user
|
||||
// Fetch config for the user — prefer active and most-recently-updated to avoid
|
||||
// picking a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const configResult = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
const config = configResult[0];
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import type { MirrorRepoRequest } from "@/types/mirror";
|
||||
import { db, configs, repositories } from "@/lib/db";
|
||||
import { and, eq, inArray } from "drizzle-orm";
|
||||
import { and, eq, inArray, sql } from "drizzle-orm";
|
||||
import { repositoryVisibilityEnum, repoStatusEnum } from "@/types/Repository";
|
||||
import { syncGiteaRepo } from "@/lib/gitea";
|
||||
import type { SyncRepoResponse } from "@/types/sync";
|
||||
@@ -38,11 +38,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Fetch config
|
||||
// Fetch config — prefer active and most-recently-updated to avoid picking
|
||||
// a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const configResult = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
const config = configResult[0];
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db, rateLimits } from "@/lib/db";
|
||||
import { eq, and, desc } from "drizzle-orm";
|
||||
import { eq, and, desc, sql } from "drizzle-orm";
|
||||
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
|
||||
import { RateLimitManager } from "@/lib/rate-limit-manager";
|
||||
import { createGitHubClient } from "@/lib/github";
|
||||
@@ -19,10 +19,13 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
try {
|
||||
// If refresh is requested, fetch current rate limit from GitHub
|
||||
if (refresh) {
|
||||
// Prefer active and most-recently-updated config to avoid picking a stale
|
||||
// inactive stub when multiple rows exist (see issue #271).
|
||||
const [config] = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (config && config.githubConfig?.token) {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db, organizations, repositories, configs } from "@/lib/db";
|
||||
import { eq, and } from "drizzle-orm";
|
||||
import { eq, and, sql } from "drizzle-orm";
|
||||
import { v4 as uuidv4 } from "uuid";
|
||||
import { createMirrorJob } from "@/lib/helpers";
|
||||
import {
|
||||
@@ -21,10 +21,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
const userId = authResult.userId;
|
||||
|
||||
try {
|
||||
// Prefer active and most-recently-updated config to avoid picking a stale
|
||||
// inactive stub when multiple rows exist (see issue #271).
|
||||
const [config] = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (!config) {
|
||||
|
||||
@@ -2,7 +2,7 @@ import type { APIRoute } from "astro";
|
||||
import { Octokit } from "@octokit/rest";
|
||||
import { configs, db, repositories } from "@/lib/db";
|
||||
import { v4 as uuidv4 } from "uuid";
|
||||
import { and, eq } from "drizzle-orm";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import { type Repository } from "@/lib/db/schema";
|
||||
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
|
||||
import type {
|
||||
@@ -72,11 +72,13 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
});
|
||||
}
|
||||
|
||||
// Get user's active config
|
||||
// Get user's active config — prefer active and most-recently-updated to avoid
|
||||
// picking a stale inactive stub when multiple rows exist (see issue #271).
|
||||
const [config] = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.userId, userId))
|
||||
.orderBy(sql`${configs.isActive} DESC`, sql`${configs.updatedAt} DESC`)
|
||||
.limit(1);
|
||||
|
||||
if (!config) {
|
||||
@@ -88,7 +90,16 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
|
||||
const configId = config.id;
|
||||
|
||||
const octokit = new Octokit(); // No auth for public repos
|
||||
// Unauthenticated one-shot lookup for public repos.
|
||||
// Uses bare Octokit (not createGitHubClient) to preserve fast-fail on the
|
||||
// 60 req/hr public rate limit — this endpoint is user-facing, we don't
|
||||
// want the throttling plugin to wait multiple retry-after windows.
|
||||
// Still respects GH_API_URL / GITHUB_API_URL for GHES / GHEC data residency.
|
||||
const baseUrl =
|
||||
process.env.GH_API_URL ||
|
||||
process.env.GITHUB_API_URL ||
|
||||
"https://api.github.com";
|
||||
const octokit = new Octokit({ baseUrl });
|
||||
|
||||
const { data: repoData } = await octokit.rest.repos.get({
|
||||
owner: trimmedOwner,
|
||||
|
||||
@@ -54,7 +54,7 @@ import { withBase } from '@/lib/base-path';
|
||||
{ var: 'HOST', desc: 'Server host', default: '0.0.0.0' },
|
||||
{ var: 'BASE_URL', desc: 'Application base path ("/" or e.g. "/mirror")', default: '/' },
|
||||
{ var: 'BETTER_AUTH_SECRET', desc: 'Authentication secret key', default: 'Auto-generated' },
|
||||
{ var: 'BETTER_AUTH_URL', desc: 'Authentication base URL', default: 'http://localhost:4321' },
|
||||
{ var: 'BETTER_AUTH_URL', desc: 'Authentication origin (scheme + host, no path). Any path is stripped automatically.', default: 'http://localhost:4321' },
|
||||
{ var: 'NODE_EXTRA_CA_CERTS', desc: 'Path to CA certificate file', default: 'None' },
|
||||
{ var: 'DATABASE_URL', desc: 'SQLite database path', default: './data/gitea-mirror.db' },
|
||||
].map((item, i) => (
|
||||
@@ -230,15 +230,17 @@ BETTER_AUTH_TRUSTED_ORIGINS=https://gitea-mirror.example.com`}</code></pre>
|
||||
</div>
|
||||
<div class="bg-muted/30 rounded p-3 mt-3">
|
||||
<pre class="text-sm"><code>{`# If deployed under a path prefix (example: /mirror):
|
||||
# BASE_URL handles the path — auth URLs stay as origin only
|
||||
BASE_URL=/mirror
|
||||
BETTER_AUTH_URL=https://git.example.com
|
||||
PUBLIC_BETTER_AUTH_URL=https://git.example.com
|
||||
BETTER_AUTH_TRUSTED_ORIGINS=https://git.example.com`}</code></pre>
|
||||
BETTER_AUTH_TRUSTED_ORIGINS=https://git.example.com
|
||||
# → Auth endpoints resolve to: https://git.example.com/mirror/api/auth/*`}</code></pre>
|
||||
</div>
|
||||
<ul class="mt-3 space-y-1 text-sm">
|
||||
<li><code class="bg-red-500/10 px-1 rounded">BASE_URL</code> — Application base path for path-prefix deployments (e.g. <code>/mirror</code>)</li>
|
||||
<li><code class="bg-red-500/10 px-1 rounded">BETTER_AUTH_URL</code> — Server-side auth base URL for callbacks and redirects</li>
|
||||
<li><code class="bg-red-500/10 px-1 rounded">PUBLIC_BETTER_AUTH_URL</code> — Client-side (browser) URL for auth API calls</li>
|
||||
<li><code class="bg-red-500/10 px-1 rounded">BASE_URL</code> — Application base path for path-prefix deployments (e.g. <code>/mirror</code>). This is handled separately from the auth URLs.</li>
|
||||
<li><code class="bg-red-500/10 px-1 rounded">BETTER_AUTH_URL</code> — Server-side auth origin, scheme + host only (e.g. <code>https://git.example.com</code>). Do <strong>not</strong> include the base path — it is applied automatically from <code>BASE_URL</code>.</li>
|
||||
<li><code class="bg-red-500/10 px-1 rounded">PUBLIC_BETTER_AUTH_URL</code> — Client-side (browser) auth origin. Same rule: origin only, no path.</li>
|
||||
<li><code class="bg-red-500/10 px-1 rounded">BETTER_AUTH_TRUSTED_ORIGINS</code> — Comma-separated origins allowed to make auth requests</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
@@ -11,7 +11,7 @@ import { validateGiteaAuth } from "@/lib/gitea-auth-validator";
|
||||
import { getConfigsByUserId } from "@/lib/db/queries/configs";
|
||||
import { db, users, repositories } from "@/lib/db";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { Octokit } from "@octokit/rest";
|
||||
import { createGitHubClient } from "@/lib/github";
|
||||
import type { Repository } from "@/lib/db/schema";
|
||||
|
||||
async function testMetadataMirroringAuth() {
|
||||
@@ -108,10 +108,8 @@ async function testMetadataMirroringAuth() {
|
||||
console.log("\n🔄 Test 4: Testing metadata mirroring authentication...");
|
||||
|
||||
try {
|
||||
// Create Octokit instance
|
||||
const octokit = new Octokit({
|
||||
auth: config.githubConfig.token,
|
||||
});
|
||||
// Create Octokit instance (honors GH_API_URL for GHES / GHEC data residency)
|
||||
const octokit = createGitHubClient(config.githubConfig.token);
|
||||
|
||||
// Test by attempting to fetch labels (lightweight operation)
|
||||
const { httpGet } = await import("@/lib/http-client");
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@
|
||||
"@types/canvas-confetti": "^1.9.0",
|
||||
"@types/react": "^19.2.14",
|
||||
"@types/react-dom": "^19.2.3",
|
||||
"astro": "^6.0.4",
|
||||
"astro": "^6.1.6",
|
||||
"canvas-confetti": "^1.9.4",
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"clsx": "^2.1.1",
|
||||
|
||||
Generated
+389
-326
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user