From 7dd14dbb2ebee8411c42ba171d534a9c5569ae94 Mon Sep 17 00:00:00 2001 From: ARUNAVO RAY Date: Wed, 23 Sep 2026 01:00:20 +0530 Subject: [PATCH] feat(releases): mirror releases from Gitea and Forgejo sources (#446) * feat(releases): mirror releases from Gitea and Forgejo sources (#440) Release mirroring was GitHub only. The list-and-reconcile loop in gitea.ts now works on a normalized release shape that a per-source lister fills, so Gitea, Forgejo and Codeberg repositories mirror their releases and assets the same way GitHub ones do. - GiteaSourceProvider gains listReleases(owner, repo, limit), paginating with limit/page and stopping at x-total-count. listReleases is optional on the SourceProvider interface; GitLab does not implement it. - mirrorGitHubReleasesToGitea keeps its signature and its behaviour, with the "Originally published on GitHub:" header line unchanged byte for byte so existing mirrored bodies are not PATCHed on the next sync. Gitea and Forgejo releases get the same shape with their own label. - mirrorGiteaSourceReleasesToGitea is the new entry point. It honours the same release limit, asset limit, per-destination lock, tag-exists check, retention pruning and asset reconciliation. - reconcileReleaseAssets takes the download Authorization from the caller instead of always sending the primary config's GitHub token: the GitHub path sends its token, a Gitea source sends that instance's token, and a tokenless public source sends no header at all. One host's token can no longer reach another host. - mirrorReleases left GITHUB_ONLY_METADATA_KEYS and is gated on the new sourceKindSupportsReleases instead. Issues, pull requests, labels, milestones and the metadata master switch stay GitHub only, and GitLab still has releases forced off. - The three call sites pick the path: a GitHub source with a client goes through Octokit, a Gitea source through its own API, anything else logs and is skipped. - The release switch and its two limits follow the release-capable check in the settings UI, and the docs, the site and the capability tables say what is true now. * test(releases): run the source aware release suite in an isolated child Other test files replace the HTTP client and related modules with mock.module, which is process-wide in bun and depends on file order. On CI the new suite ran after one of them and every case failed on a mocked destination lookup. Same harness as gitea-org-mirror-destination.test.ts. --- CHANGELOG.md | 4 + README.md | 2 +- docs/ENVIRONMENT_VARIABLES.md | 2 +- docs/SOURCE_PROVIDERS.md | 6 +- src/components/config/GitHubConfigForm.tsx | 2 +- .../config/GitHubMirrorSettings.tsx | 22 +- src/components/config/SourcesCard.tsx | 2 +- src/lib/gitea-enhanced.ts | 85 +++- src/lib/gitea-release-assets.test.ts | 6 +- src/lib/gitea-source-releases.test.ts | 462 ++++++++++++++++++ src/lib/gitea.ts | 383 +++++++++++---- src/lib/source-provider-wiring.test.ts | 27 +- .../gitea-source-releases.test.ts | 203 ++++++++ src/lib/source-providers/gitea-source.ts | 78 ++- src/lib/source-providers/kinds.ts | 20 + src/lib/source-providers/types.ts | 35 ++ src/lib/utils/mirror-overrides.test.ts | 82 ++++ src/lib/utils/mirror-overrides.ts | 27 +- www/src/pages/docs/environment-variables.mdx | 2 +- www/src/pages/docs/sources.mdx | 4 +- 20 files changed, 1314 insertions(+), 140 deletions(-) create mode 100644 src/lib/gitea-source-releases.test.ts create mode 100644 src/lib/source-providers/gitea-source-releases.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 283bb32..cbcd659 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Raised the dependency floors for the advisories published on 2026-09-08: Astro 7.2.8 (remote code execution through AVIF image optimization, and an authorization bypass when stripping the configured base), @xmldom/xmldom 0.8.15 (eight parser and serializer issues), sharp 0.35.4 (libheif), svgo 4.1.0 (removeScripts sanitization) and js-yaml 4.3.2 (merge-key CPU use). Applied to both the application and the documentation site. ### Added +- Releases mirror from Gitea and Forgejo sources, Codeberg included (#440) + - The release mirror lists releases through the repository's own source instead of always through the GitHub API, and keeps the same release limit, asset limit, per-destination lock, tag check and retention pruning + - Release assets are downloaded with the credentials of the host they live on, so a Gitea source uses its own token and a public one needs none + - The release switch and its two limits are enabled for Gitea and Forgejo sources; GitLab stays code only - Sync an organization that is already mirrored (#429) - The organization card menu gains Sync Organization for mirrored and failed organizations; the Mirror button only ever covered the first run - `POST /api/job/sync-org` re-discovers the organization's repositories from its source, mirrors the imported ones and syncs the mirrored, synced and failed ones, leaving rows another run owns alone; the organization is claimed before the response so two clicks cannot both start a run diff --git a/README.md b/README.md index 67bda28..6b2cc63 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ First user signup becomes admin. Configure GitHub and Gitea/Forgejo through the ## ✨ Features - 🔁 Mirror public, private, and starred GitHub repos to Gitea/Forgejo -- 🦊 **Multiple sources (GitLab and Gitea/Forgejo beta)** - Connect several sources at once: GitHub and GitHub Enterprise, GitLab, Gitea/Forgejo, and each repository keeps its source. gitlab.com, Codeberg and self hosted instances mirror code, tags, wiki and LFS (issues, pull requests and releases need a GitHub source). GitHub to Gitea is the supported default; every other source or destination is beta. See [docs/SOURCE_PROVIDERS.md](docs/SOURCE_PROVIDERS.md) +- 🦊 **Multiple sources (GitLab and Gitea/Forgejo beta)** - Connect several sources at once: GitHub and GitHub Enterprise, GitLab, Gitea/Forgejo, and each repository keeps its source. gitlab.com, Codeberg and self hosted instances mirror code, tags, wiki and LFS; Gitea and Forgejo sources mirror releases with their assets too (issues and pull requests need a GitHub source). GitHub to Gitea is the supported default; every other source or destination is beta. See [docs/SOURCE_PROVIDERS.md](docs/SOURCE_PROVIDERS.md) - 🏛️ **GitHub Enterprise support** - Works with GHES and GHEC with data residency via `GH_API_URL` - 🚀 **Forgejo, GitHub and GitLab destinations (beta)** - Forgejo works like Gitea. Pick GitHub or GitLab and the app keeps a bare clone of each repository and pushes its branches and tags there. Code only, and the target is overwritten on every sync. See [docs/PUSH_TARGETS.md](docs/PUSH_TARGETS.md) - 🏢 Mirror entire organizations with flexible strategies, including public organizations from GitHub, GitLab or Gitea/Forgejo with no source account configured diff --git a/docs/ENVIRONMENT_VARIABLES.md b/docs/ENVIRONMENT_VARIABLES.md index 575f025..33cf2f6 100644 --- a/docs/ENVIRONMENT_VARIABLES.md +++ b/docs/ENVIRONMENT_VARIABLES.md @@ -97,7 +97,7 @@ GITHUB_USERNAME=my-codeberg-user GITHUB_TOKEN=... ``` -Code, tags, wiki and LFS are mirrored from every source. Issues, pull requests, releases, labels, milestones and star lists need a GitHub source and are skipped for the others. +Code, tags, wiki and LFS are mirrored from every source, and releases with their assets from GitHub and Gitea/Forgejo sources. Issues, pull requests, labels, milestones and star lists need a GitHub source and are skipped for the others. Once repositories have been imported the source is locked, and once anything has been mirrored the Gitea server URL is locked. A `SOURCE_PROVIDER`, `SOURCE_URL` or `GITEA_URL` value that disagrees with a locked host is ignored on boot with a warning; change the host on the Configuration page, where the change asks for confirmation. diff --git a/docs/SOURCE_PROVIDERS.md b/docs/SOURCE_PROVIDERS.md index bd86b15..4ffadd8 100644 --- a/docs/SOURCE_PROVIDERS.md +++ b/docs/SOURCE_PROVIDERS.md @@ -39,11 +39,11 @@ The mirror itself is done by Gitea's pull mirror, which treats every source as a | Entire organizations (GitLab: top level groups) | yes | yes | yes | | Add a single repository by URL | yes | yes | yes | | Issues, pull requests, labels, milestones | yes | no | no | -| Releases with assets | yes | no | no | +| Releases with assets | yes | no | yes | | Star lists | yes | no | no | | Force push detection | yes | no | no | -The GitHub only rows read the GitHub API. For other sources the corresponding switches are disabled on the Configuration page, and the mirror step skips them. +The GitHub only rows read the GitHub API. For other sources the corresponding switches are disabled on the Configuration page, and the mirror step skips them. Releases are the exception: Gitea and Forgejo expose them at `GET /repos/{owner}/{repo}/releases`, so a Gitea, Forgejo or Codeberg source mirrors releases and their assets like GitHub does, with that instance's own token (public repositories need none). Entire organizations also mirror without a token on every source; the next section covers public mode. @@ -56,7 +56,7 @@ On the Organizations page, **Add organization** has a public mode, which is the Public mode degrades in known ways: - **GitHub metadata is best effort.** Anonymous requests run under GitHub's 60 requests per hour limit, so a large organization can end up with partial metadata. Releases, description and topics mirror anonymously with no account at all. Issues, pull requests, labels and milestones are also fetched anonymously once any source in the account carries a token; with no token anywhere they are skipped, and the mirror itself is unaffected either way. -- **GitLab and Gitea/Forgejo sources mirror code only.** Metadata remains GitHub only, as in the table above, and issues and merge requests from those sources stay a Not yet item with or without a token. +- **GitLab sources mirror code only; Gitea and Forgejo sources add releases.** The rest of the metadata remains GitHub only, as in the table above, and issues and merge requests from those sources stay a Not yet item with or without a token. - **Personal auto discovery and starred repositories need a token.** Neither runs for a public only source. - **Cleanup does not run.** Repositories deleted upstream are not detected for public only sources. diff --git a/src/components/config/GitHubConfigForm.tsx b/src/components/config/GitHubConfigForm.tsx index 882edd7..ce06bed 100644 --- a/src/components/config/GitHubConfigForm.tsx +++ b/src/components/config/GitHubConfigForm.tsx @@ -326,7 +326,7 @@ export function GitHubConfigForm({ ? "Where your repositories are pulled from" : provider === "gitlab" ? "Beta. Code, tags, wiki and LFS are mirrored. Issues, merge requests and releases need a GitHub source." - : "Beta. Code, tags, wiki and LFS are mirrored. Issues, pull requests and releases need a GitHub source."} + : "Beta. Code, tags, releases, wiki and LFS are mirrored. Issues and pull requests need a GitHub source."}

{sourceLock?.locked && ( 0 ? sources.some((source) => source.provider === "github") : sourceProvider === "github"; const isGithubSource = hasGithubSource && !isPushTarget; + // Releases are listed through the source API, which Gitea and Forgejo + // expose as well (#440), so they follow their own capability check. + const hasReleaseCapableSource = + sources && sources.length > 0 + ? sources.some((source) => sourceKindSupportsReleases(source.provider)) + : sourceKindSupportsReleases(sourceProvider); + const isReleaseCapableSource = hasReleaseCapableSource && !isPushTarget; const sourceLabel = SOURCE_PROVIDER_LABELS[sourceProvider]; const orgNoun = SOURCE_PROVIDER_ORG_NOUNS[sourceProvider]; const [starListsOpen, setStarListsOpen] = React.useState(false); @@ -881,7 +889,9 @@ export function GitHubMirrorSettings({ ? `${destinationLabel} destinations receive branches and tags only` : isGithubSource ? "Pull requests are mirrored as issues due to Gitea API limits" - : "Issues, pull requests and releases are mirrored for GitHub sources only" + : isReleaseCapableSource + ? "Releases are mirrored. Issues, pull requests, labels and milestones need a GitHub source" + : "Issues, pull requests and releases are mirrored for GitHub sources only" } /> } @@ -906,13 +916,13 @@ export function GitHubMirrorSettings({ description={ isPushTarget ? pushTargetReason - : !isGithubSource - ? "Release assets need a GitHub source. Tags always come with the code" + : !isReleaseCapableSource + ? "Release assets need a GitHub or Gitea/Forgejo source. Tags always come with the code" : mirrorOptions.mirrorReleases ? "Includes tags. Leave the assets box empty to upload assets for every mirrored release, or set it to 0 for release notes only" : "Includes release assets and tags" } - disabled={!isGithubSource} + disabled={!isReleaseCapableSource} checked={mirrorOptions.mirrorReleases} onCheckedChange={(checked) => handleMirrorChange('mirrorReleases', checked)} extra={ diff --git a/src/components/config/SourcesCard.tsx b/src/components/config/SourcesCard.tsx index ee29043..143a76a 100644 --- a/src/components/config/SourcesCard.tsx +++ b/src/components/config/SourcesCard.tsx @@ -119,7 +119,7 @@ function providerHelp(provider: SourceProvider): string { if (provider === "github") return "Where your repositories are pulled from"; if (provider === "gitlab") return "Beta. Code, tags, wiki and LFS are mirrored. Issues, merge requests and releases need a GitHub source."; - return "Beta. Code, tags, wiki and LFS are mirrored. Issues, pull requests and releases need a GitHub source."; + return "Beta. Code, tags, releases, wiki and LFS are mirrored. Issues and pull requests need a GitHub source."; } function sourceHost(source: SourceApiRecord): string { diff --git a/src/lib/gitea-enhanced.ts b/src/lib/gitea-enhanced.ts index 6535e0b..2c72899 100644 --- a/src/lib/gitea-enhanced.ts +++ b/src/lib/gitea-enhanced.ts @@ -7,7 +7,11 @@ */ import type { Config } from "@/types/config"; -import { getRepositorySource } from "@/lib/source-providers/kinds"; +import { + getRepositorySource, + SOURCE_PROVIDER_LABELS, +} from "@/lib/source-providers/kinds"; +import { sourceConnectionFromSource } from "@/lib/source-providers"; import { decryptSourceToken, findSourceForRepository, @@ -46,6 +50,12 @@ type SyncDependencies = { getGiteaRepoOwnerAsync: typeof import("./gitea")["getGiteaRepoOwnerAsync"]; syncRepositoryMetadataToGitea: typeof import("./gitea")["syncRepositoryMetadataToGitea"]; mirrorGitHubReleasesToGitea: typeof import("./gitea")["mirrorGitHubReleasesToGitea"]; + /** + * Optional so the many existing test doubles of this dependency bag keep + * type checking; the real module always carries it and the sync path falls + * back to importing it when a double leaves it out. + */ + mirrorGiteaSourceReleasesToGitea?: typeof import("./gitea")["mirrorGiteaSourceReleasesToGitea"]; mirrorGitRepoIssuesToGitea: typeof import("./gitea")["mirrorGitRepoIssuesToGitea"]; mirrorGitRepoPullRequestsToGitea: typeof import("./gitea")["mirrorGitRepoPullRequestsToGitea"]; mirrorGitRepoLabelsToGitea: typeof import("./gitea")["mirrorGitRepoLabelsToGitea"]; @@ -472,7 +482,8 @@ export async function syncGiteaRepoEnhanced({ const sources = config.userId ? await listSources(config.userId) : []; const repoSource = findSourceForRepository(repository, sources); const repoSourceToken = repoSource ? decryptSourceToken(repoSource.token) : ""; - const repoIsGitHub = getRepositorySource(repository).provider === "github"; + const repoSourceKind = getRepositorySource(repository).provider; + const repoIsGitHub = repoSourceKind === "github"; if (usesPushEngine(config)) { throw new Error("The configured destination is a push target; syncing goes through the push engine."); @@ -1002,36 +1013,72 @@ export async function syncGiteaRepoEnhanced({ const shouldMirrorMilestones = mirrorOptions.mirrorMilestones; if (shouldMirrorReleases) { - const octokit = ensureOctokit(); - if (!octokit) { - console.warn( - `[Sync] Skipping release mirroring for ${repository.name}: Missing GitHub token` - ); - } else { - try { - await dependencies.mirrorGitHubReleasesToGitea({ + // GitHub reads releases through Octokit; Gitea and Forgejo sources + // read them from their own instance with their own token (#440). + // Any other source has no release listing and is skipped. + try { + let mirrored = false; + + if (repoIsGitHub) { + const octokit = ensureOctokit(); + if (!octokit) { + console.warn( + `[Sync] Skipping release mirroring for ${repository.name}: Missing GitHub token` + ); + } else { + await dependencies.mirrorGitHubReleasesToGitea({ + config, + octokit, + repository, + sourceToken: repoSourceToken, + giteaOwner: repoOwner, + giteaRepoName: repoName, + releaseLimit: mirrorOptions.releaseLimit, + releaseAssetLimit: mirrorOptions.releaseAssetLimit, + }); + mirrored = true; + } + } else if (repoSourceKind === "gitea" && repoSource) { + const mirrorGiteaSourceReleases = + dependencies.mirrorGiteaSourceReleasesToGitea ?? + (await import("./gitea")).mirrorGiteaSourceReleasesToGitea; + await mirrorGiteaSourceReleases({ config, - octokit, repository, + // Carries that instance's own decrypted token, so the release + // listing and the asset downloads authenticate against it. + connection: sourceConnectionFromSource(repoSource, { + userId: config.userId, + }), giteaOwner: repoOwner, giteaRepoName: repoName, releaseLimit: mirrorOptions.releaseLimit, releaseAssetLimit: mirrorOptions.releaseAssetLimit, }); + mirrored = true; + } else { + console.warn( + `[Sync] Skipping release mirroring for ${repository.name}: ${ + SOURCE_PROVIDER_LABELS[repoSourceKind] + } sources cannot list releases` + ); + } + + if (mirrored) { metadataState.components.releases = true; metadataUpdated = true; console.log( `[Sync] Mirrored releases for ${repository.name} after sync` ); - } catch (releaseError) { - console.error( - `[Sync] Failed to mirror releases for ${repository.name}: ${ - releaseError instanceof Error - ? releaseError.message - : String(releaseError) - }` - ); } + } catch (releaseError) { + console.error( + `[Sync] Failed to mirror releases for ${repository.name}: ${ + releaseError instanceof Error + ? releaseError.message + : String(releaseError) + }` + ); } } diff --git a/src/lib/gitea-release-assets.test.ts b/src/lib/gitea-release-assets.test.ts index 6020215..7b57034 100644 --- a/src/lib/gitea-release-assets.test.ts +++ b/src/lib/gitea-release-assets.test.ts @@ -138,7 +138,7 @@ function createFakeDestination(options: FakeDestinationOptions = {}) { function reconcile( fetchImpl: typeof fetch, - githubAssets: Array<{ name: string; size: number; browser_download_url: string }> + sourceAssets: Array<{ name: string; size: number; browser_download_url: string }> ) { return reconcileReleaseAssets({ config, @@ -146,7 +146,9 @@ function reconcile( repoOwner: OWNER, repoName: REPO, giteaReleaseId: RELEASE_ID, - githubAssets, + sourceAssets, + // The GitHub path's credential, which is what these cases model. + downloadAuthorization: `token ${decryptedConfig.githubConfig!.token}`, tagName: "v1.0.0", fetchImpl, }); diff --git a/src/lib/gitea-source-releases.test.ts b/src/lib/gitea-source-releases.test.ts new file mode 100644 index 0000000..813f787 --- /dev/null +++ b/src/lib/gitea-source-releases.test.ts @@ -0,0 +1,462 @@ +/** + * Behavioural tests for source aware release mirroring (#440). + * + * Release mirroring used to call the GitHub API unconditionally and download + * every asset with the primary config's GitHub token. A Gitea or Forgejo + * source (Codeberg included) now lists its own releases and downloads its own + * assets with its own credentials, while the GitHub path keeps its exact + * previous behaviour, including the "Originally published on GitHub:" header + * line, which must stay byte for byte or every mirrored release body would + * differ and be PATCHed on the next sync. + * + * Both entry points are driven through a fake global fetch, which is what the + * destination client, the source adapter and the asset reconciliation all use. + * Other test files replace @/lib/http-client and friends with mock.module, + * which is process-wide in bun and depends on file order, so the suites here + * run in an isolated child process (same harness as + * gitea-org-mirror-destination.test.ts) where the real modules are loaded. + */ + +import { afterEach, describe, expect, it, test } from "bun:test"; +import type { Config } from "@/types/config"; +import type { Repository } from "@/lib/db/schema"; +import type { SourceConnection } from "@/lib/source-providers/types"; + +const CHILD_FLAG = "GM_SOURCE_RELEASES_ISOLATED"; +const isChild = !!process.env[CHILD_FLAG]; + +if (!isChild) { + test("source aware release mirroring - isolated child suite", () => { + const res = Bun.spawnSync({ + cmd: [process.execPath, "test", import.meta.path], + env: { ...process.env, [CHILD_FLAG]: "1" }, + stdout: "pipe", + stderr: "pipe", + }); + if (res.exitCode !== 0) { + console.error(res.stdout.toString()); + console.error(res.stderr.toString()); + } + expect(res.exitCode).toBe(0); + }, 60_000); +} + +const { mirrorGiteaSourceReleasesToGitea, mirrorGitHubReleasesToGitea } = isChild + ? await import("@/lib/gitea") + : ({} as typeof import("@/lib/gitea")); +const GITEA_URL = "https://gitea.example.com"; +const SOURCE_URL = "https://codeberg.org"; +const DEST_OWNER = "mirror-owner"; +const DEST_REPO = "demo"; +const DEST_API = `${GITEA_URL}/api/v1/repos/${DEST_OWNER}/${DEST_REPO}`; + +const realFetch = globalThis.fetch; + +afterEach(() => { + globalThis.fetch = realFetch; +}); + +const config = { + userId: "user-1", + githubConfig: { owner: "acme", token: "github-token" }, + giteaConfig: { url: GITEA_URL, token: "gitea-token", defaultOwner: DEST_OWNER }, +} as unknown as Partial; + +const repository = { + id: "repo-1", + name: DEST_REPO, + fullName: "acme/demo", + owner: "acme", + isStarred: false, + mirrorOverrides: null, + organization: null, +} as unknown as Repository; + +function giteaConnection(token: string): SourceConnection { + return { provider: "gitea", url: SOURCE_URL, username: "acme", token }; +} + +interface Recorded { + method: string; + url: string; + authorization: string | null; + body?: string; +} + +interface DestinationOptions { + /** Releases the source reports, newest first. */ + sourceReleases: Array>; + /** Release already on the destination, matched by tag. */ + existingRelease?: { id: number; name: string; body: string; tag_name: string }; + /** Tags the destination already has. Defaults to every source tag. */ + destinationTags?: string[]; +} + +function fakeWorld(options: DestinationOptions) { + const requests: Recorded[] = []; + const tags = + options.destinationTags ?? + options.sourceReleases.map((release) => String(release.tag_name)); + + globalThis.fetch = (async (input: any, init?: any) => { + const url = typeof input === "string" ? input : String(input); + const method = String(init?.method ?? "GET").toUpperCase(); + const headers = new Headers(init?.headers ?? {}); + const body = typeof init?.body === "string" ? init.body : undefined; + requests.push({ method, url, authorization: headers.get("authorization"), body }); + + const json = (payload: unknown, status = 200) => + new Response(JSON.stringify(payload), { + status, + headers: { "content-type": "application/json" }, + }); + + // The source lists its releases. + if (url.startsWith(`${SOURCE_URL}/api/v1/`)) { + return json(options.sourceReleases); + } + + // Downloading an asset from the source host. + if (url.startsWith(`${SOURCE_URL}/attachments/`) || url.startsWith("https://github.com/")) { + return new Response(new Uint8Array([1, 2, 3, 4]), { status: 200 }); + } + + // The destination repository exists. + if (method === "GET" && url === DEST_API) { + return json({ id: 1, name: DEST_REPO }); + } + + // Does the release already exist for this tag? + if (method === "GET" && url.startsWith(`${DEST_API}/releases/tags/`)) { + return options.existingRelease + ? json(options.existingRelease) + : new Response("not found", { status: 404 }); + } + + // Is the git tag synced yet? + if (method === "GET" && url.startsWith(`${DEST_API}/tags/`)) { + const tag = decodeURIComponent(url.slice(`${DEST_API}/tags/`.length)); + return tags.includes(tag) + ? json({ name: tag }) + : new Response("not found", { status: 404 }); + } + + // Existing attachments of a release. + if (method === "GET" && /\/releases\/\d+\/assets$/.test(url)) { + return json([]); + } + + // Uploading an attachment. + if (method === "POST" && url.includes("/assets?name=")) { + return json({ id: 900 }, 201); + } + + // Creating a release. + if (method === "POST" && url === `${DEST_API}/releases`) { + return json({ id: 77 }, 201); + } + + // Updating a release. + if (method === "PATCH" && /\/releases\/\d+$/.test(url)) { + return json({ id: 77 }); + } + + // The retention pass lists everything the destination has. + if (method === "GET" && url.startsWith(`${DEST_API}/releases?`)) { + return json([]); + } + + throw new Error(`unexpected request: ${method} ${url}`); + }) as unknown as typeof fetch; + + return { + requests, + creates: () => requests.filter((r) => r.method === "POST" && r.url === `${DEST_API}/releases`), + patches: () => requests.filter((r) => r.method === "PATCH"), + uploads: () => requests.filter((r) => r.method === "POST" && r.url.includes("/assets?name=")), + downloads: () => + requests.filter( + (r) => r.url.startsWith(`${SOURCE_URL}/attachments/`) || r.url.startsWith("https://github.com/") + ), + }; +} + +function giteaRelease(tag: string, extra: Record = {}) { + return { + id: 5, + tag_name: tag, + name: `Release ${tag}`, + body: `notes for ${tag}`, + draft: false, + prerelease: false, + created_at: "2026-01-01T00:00:00Z", + published_at: "2026-01-01T00:00:00Z", + assets: [], + ...extra, + }; +} + +function mirrorFromGitea(connection: SourceConnection) { + return mirrorGiteaSourceReleasesToGitea({ + config, + repository, + connection, + giteaOwner: DEST_OWNER, + giteaRepoName: DEST_REPO, + releaseLimit: 10, + releaseAssetLimit: null, + }); +} + +describe.skipIf(!isChild)("mirrorGiteaSourceReleasesToGitea", () => { + it("creates a missing release with the Gitea/Forgejo header", async () => { + const world = fakeWorld({ sourceReleases: [giteaRelease("v1.0.0")] }); + + await mirrorFromGitea(giteaConnection("source-token")); + + const [create] = world.creates(); + expect(create).toBeDefined(); + const payload = JSON.parse(create.body!); + expect(payload.tag_name).toBe("v1.0.0"); + expect(payload.name).toBe("Release v1.0.0"); + expect(payload.body).toBe( + "> 📅 **Originally published on Gitea / Forgejo:** Thu, 01 Jan 2026 00:00:00 GMT\n\nnotes for v1.0.0" + ); + // The destination is always addressed with the destination token. + expect(create.authorization).toBe("token gitea-token"); + }); + + it("updates an existing release whose body drifted", async () => { + const world = fakeWorld({ + sourceReleases: [giteaRelease("v1.0.0")], + existingRelease: { + id: 77, + tag_name: "v1.0.0", + name: "Release v1.0.0", + body: "stale body", + }, + }); + + await mirrorFromGitea(giteaConnection("source-token")); + + expect(world.creates()).toHaveLength(0); + const [patch] = world.patches(); + expect(patch.url).toBe(`${DEST_API}/releases/77`); + expect(JSON.parse(patch.body!).body).toContain( + "**Originally published on Gitea / Forgejo:**" + ); + }); + + it("leaves a release alone when the body and title already match", async () => { + const world = fakeWorld({ + sourceReleases: [giteaRelease("v1.0.0")], + existingRelease: { + id: 77, + tag_name: "v1.0.0", + name: "Release v1.0.0", + body: + "> 📅 **Originally published on Gitea / Forgejo:** Thu, 01 Jan 2026 00:00:00 GMT\n\nnotes for v1.0.0", + }, + }); + + await mirrorFromGitea(giteaConnection("source-token")); + + expect(world.creates()).toHaveLength(0); + expect(world.patches()).toHaveLength(0); + }); + + it("skips a release whose git tag has not reached the destination yet", async () => { + const world = fakeWorld({ + sourceReleases: [giteaRelease("v1.0.0")], + destinationTags: [], + }); + + await mirrorFromGitea(giteaConnection("source-token")); + + expect(world.creates()).toHaveLength(0); + }); + + it("downloads assets with the source instance's token, never the GitHub one", async () => { + const world = fakeWorld({ + sourceReleases: [ + giteaRelease("v1.0.0", { + assets: [ + { + id: 1, + name: "firmware.bin", + size: 4, + browser_download_url: `${SOURCE_URL}/attachments/abc`, + }, + ], + }), + ], + }); + + await mirrorFromGitea(giteaConnection("source-token")); + + const [download] = world.downloads(); + expect(download.url).toBe(`${SOURCE_URL}/attachments/abc`); + expect(download.authorization).toBe("token source-token"); + expect(world.uploads()).toHaveLength(1); + // No request anywhere carried the GitHub token. + expect( + world.requests.some((r) => r.authorization === "token github-token") + ).toBe(false); + }); + + it("downloads assets with no Authorization header from a tokenless public source", async () => { + const world = fakeWorld({ + sourceReleases: [ + giteaRelease("v1.0.0", { + assets: [ + { + id: 1, + name: "firmware.bin", + size: 4, + browser_download_url: `${SOURCE_URL}/attachments/abc`, + }, + ], + }), + ], + }); + + await mirrorFromGitea(giteaConnection("")); + + const [download] = world.downloads(); + expect(download.authorization).toBeNull(); + // The source listing is anonymous too. + const listing = world.requests.find((r) => r.url.startsWith(`${SOURCE_URL}/api/v1/`)); + expect(listing?.authorization).toBeNull(); + }); +}); + +describe.skipIf(!isChild)("mirrorGitHubReleasesToGitea keeps its own header and credentials", () => { + function githubOctokit(releases: Array>) { + return { + rest: { + repos: { + listReleases: async ({ page }: { page: number }) => ({ + data: page === 1 ? releases : [], + }), + }, + }, + } as any; + } + + it("writes the GitHub header byte for byte and downloads with the GitHub token", async () => { + const world = fakeWorld({ + sourceReleases: [], + destinationTags: ["v1.0.0"], + }); + + await mirrorGitHubReleasesToGitea({ + config, + octokit: githubOctokit([ + { + tag_name: "v1.0.0", + name: "Release v1.0.0", + body: "notes for v1.0.0", + draft: false, + prerelease: false, + created_at: "2026-01-01T00:00:00Z", + published_at: "2026-01-01T00:00:00Z", + assets: [ + { + name: "firmware.bin", + size: 4, + browser_download_url: + "https://github.com/acme/demo/releases/download/v1.0.0/firmware.bin", + }, + ], + }, + ]), + repository, + giteaOwner: DEST_OWNER, + giteaRepoName: DEST_REPO, + releaseLimit: 10, + releaseAssetLimit: null, + }); + + const [create] = world.creates(); + expect(JSON.parse(create.body!).body).toBe( + "> 📅 **Originally published on GitHub:** Thu, 01 Jan 2026 00:00:00 GMT\n\nnotes for v1.0.0" + ); + + const [download] = world.downloads(); + expect(download.authorization).toBe("token github-token"); + }); + + it("sends no Authorization header when the GitHub source has no token", async () => { + const world = fakeWorld({ sourceReleases: [], destinationTags: ["v1.0.0"] }); + + await mirrorGitHubReleasesToGitea({ + config: { + ...config, + githubConfig: { ...(config.githubConfig as any), token: "" }, + } as Partial, + octokit: githubOctokit([ + { + tag_name: "v1.0.0", + name: "Release v1.0.0", + body: "", + draft: false, + prerelease: false, + created_at: "2026-01-01T00:00:00Z", + published_at: "2026-01-01T00:00:00Z", + assets: [ + { + name: "firmware.bin", + size: 4, + browser_download_url: + "https://github.com/acme/demo/releases/download/v1.0.0/firmware.bin", + }, + ], + }, + ]), + repository, + giteaOwner: DEST_OWNER, + giteaRepoName: DEST_REPO, + releaseLimit: 10, + releaseAssetLimit: null, + }); + + const [download] = world.downloads(); + expect(download.authorization).toBeNull(); + }); + + it("prefers the repository's own GitHub source token over the primary config token", async () => { + const world = fakeWorld({ sourceReleases: [], destinationTags: ["v1.0.0"] }); + + await mirrorGitHubReleasesToGitea({ + config, + octokit: githubOctokit([ + { + tag_name: "v1.0.0", + name: "Release v1.0.0", + body: "", + draft: false, + prerelease: false, + created_at: "2026-01-01T00:00:00Z", + published_at: "2026-01-01T00:00:00Z", + assets: [ + { + name: "firmware.bin", + size: 4, + browser_download_url: + "https://github.com/acme/demo/releases/download/v1.0.0/firmware.bin", + }, + ], + }, + ]), + repository, + sourceToken: "ghes-token", + giteaOwner: DEST_OWNER, + giteaRepoName: DEST_REPO, + releaseLimit: 10, + releaseAssetLimit: null, + }); + + const [download] = world.downloads(); + expect(download.authorization).toBe("token ghes-token"); + }); +}); diff --git a/src/lib/gitea.ts b/src/lib/gitea.ts index 7e13b64..69259fa 100644 --- a/src/lib/gitea.ts +++ b/src/lib/gitea.ts @@ -20,6 +20,9 @@ import { resolveSourceConnection, sourceConnectionFromSource, } from "./source-providers"; +import { GiteaSourceProvider } from "./source-providers/gitea-source"; +import { SOURCE_PROVIDER_LABELS } from "./source-providers/kinds"; +import type { SourceConnection, SourceRelease } from "./source-providers/types"; import { decryptSourceToken, findSourceForOrganization, @@ -43,6 +46,82 @@ import { resolveDestinationIdentity } from "./destination-connection"; import { withKeyedLock } from "./utils/keyed-mutex"; import { claimRepositoryForInFlightWork } from "./utils/repo-status-claim"; +/** + * Run the release mirror for one repository through whichever source it came + * from (#440). + * + * GitHub goes through the Octokit path, Gitea and Forgejo through the source + * instance's REST API with that instance's own credentials. Any other source + * (GitLab today) has no release listing, so the caller logs and moves on; + * `resolveMirrorOptions` already keeps the switch off for it, this is the + * belt to that braces. + * + * Returns true when a release pass actually ran. + */ +export async function mirrorRepositoryReleases({ + config, + repository, + sourceConnection, + sourceToken, + octokit, + giteaOwner, + giteaRepoName, + releaseLimit, + releaseAssetLimit, +}: { + config: Partial; + repository: Repository; + sourceConnection: SourceConnection; + /** Decrypted token of the repository's own source row. */ + sourceToken: string; + /** Only used for a GitHub source; null means no client could be built. */ + octokit: Octokit | null; + giteaOwner: string; + giteaRepoName: string; + releaseLimit?: number; + releaseAssetLimit?: number | null; +}): Promise { + if (sourceConnection.provider === "github") { + if (!octokit) { + console.warn( + `[Releases] Skipping release mirroring for ${repository.name}: no GitHub client for its source` + ); + return false; + } + await mirrorGitHubReleasesToGitea({ + config, + octokit, + repository, + sourceToken, + giteaOwner, + giteaRepoName, + releaseLimit, + releaseAssetLimit, + }); + return true; + } + + if (sourceConnection.provider === "gitea") { + await mirrorGiteaSourceReleasesToGitea({ + config, + repository, + connection: { ...sourceConnection, token: sourceToken }, + giteaOwner, + giteaRepoName, + releaseLimit, + releaseAssetLimit, + }); + return true; + } + + console.warn( + `[Releases] Skipping release mirroring for ${repository.name}: ${ + SOURCE_PROVIDER_LABELS[sourceConnection.provider] + } sources cannot list releases` + ); + return false; +} + /** * Helper function to get organization configuration including destination override */ @@ -1050,7 +1129,8 @@ export const mirrorGithubRepoToGitea = async ({ // Mirror releases if enabled (always allowed to rerun for updates). // mirrorOptions already accounts for org/repo overrides and starredCodeOnly. - const shouldMirrorReleases = mirrorOptions.mirrorReleases && octokit !== null; + // Which source path runs is decided inside mirrorRepositoryReleases (#440). + const shouldMirrorReleases = mirrorOptions.mirrorReleases; console.log( `[Metadata] Release mirroring check: mirrorReleases=${mirrorOptions.mirrorReleases}, isStarred=${repository.isStarred}, starredCodeOnly=${config.githubConfig?.starredCodeOnly}, shouldMirrorReleases=${shouldMirrorReleases}` @@ -1058,20 +1138,24 @@ export const mirrorGithubRepoToGitea = async ({ if (shouldMirrorReleases) { try { - await mirrorGitHubReleasesToGitea({ + const mirrored = await mirrorRepositoryReleases({ config, - octokit, repository, + sourceConnection, + sourceToken: repoSourceToken, + octokit, giteaOwner: repoOwner, giteaRepoName: targetRepoName, releaseLimit: mirrorOptions.releaseLimit, releaseAssetLimit: mirrorOptions.releaseAssetLimit, }); - metadataState.components.releases = true; - metadataUpdated = true; - console.log( - `[Metadata] Successfully mirrored releases for ${repository.name}` - ); + if (mirrored) { + metadataState.components.releases = true; + metadataUpdated = true; + console.log( + `[Metadata] Successfully mirrored releases for ${repository.name}` + ); + } } catch (error) { console.error( `[Metadata] Failed to mirror releases for ${repository.name}: ${ @@ -1819,7 +1903,8 @@ export async function mirrorGitHubRepoToGiteaOrg({ let metadataUpdated = false; // mirrorOptions already accounts for org/repo overrides and starredCodeOnly. - const shouldMirrorReleases = mirrorOptions.mirrorReleases && octokit !== null; + // Which source path runs is decided inside mirrorRepositoryReleases (#440). + const shouldMirrorReleases = mirrorOptions.mirrorReleases; console.log( `[Metadata] Release mirroring check: mirrorReleases=${mirrorOptions.mirrorReleases}, isStarred=${repository.isStarred}, starredCodeOnly=${config.githubConfig?.starredCodeOnly}, shouldMirrorReleases=${shouldMirrorReleases}` @@ -1827,20 +1912,24 @@ export async function mirrorGitHubRepoToGiteaOrg({ if (shouldMirrorReleases) { try { - await mirrorGitHubReleasesToGitea({ + const mirrored = await mirrorRepositoryReleases({ config, - octokit, repository, + sourceConnection, + sourceToken: repoSourceToken, + octokit, giteaOwner: orgName, giteaRepoName: targetRepoName, releaseLimit: mirrorOptions.releaseLimit, releaseAssetLimit: mirrorOptions.releaseAssetLimit, }); - metadataState.components.releases = true; - metadataUpdated = true; - console.log( - `[Metadata] Successfully mirrored releases for ${repository.name}` - ); + if (mirrored) { + metadataState.components.releases = true; + metadataUpdated = true; + console.log( + `[Metadata] Successfully mirrored releases for ${repository.name}` + ); + } } catch (error) { console.error( `[Metadata] Failed to mirror releases for ${repository.name}: ${ @@ -3053,7 +3142,7 @@ export function classifyReleasesForReconciliation( * hitting the network (regression guard for #331 and #417). */ export function classifyAssetsForReconciliation( - githubAssets: Array<{ name: string; size: number }>, + sourceAssets: Array<{ name: string; size: number }>, giteaAssets: Array<{ id: number; name: string; size: number }> ): { toUpload: string[]; @@ -3080,7 +3169,7 @@ export function classifyAssetsForReconciliation( const toDelete: number[] = []; const seen = new Set(); - for (const asset of githubAssets) { + for (const asset of sourceAssets) { if (seen.has(asset.name)) { continue; } @@ -3144,7 +3233,8 @@ export async function reconcileReleaseAssets({ repoOwner, repoName, giteaReleaseId, - githubAssets, + sourceAssets, + downloadAuthorization, tagName, fetchImpl, }: { @@ -3153,7 +3243,16 @@ export async function reconcileReleaseAssets({ repoOwner: string; repoName: string; giteaReleaseId: number; - githubAssets: Array<{ name: string; size: number; browser_download_url: string }>; + sourceAssets: Array<{ name: string; size: number; browser_download_url: string }>; + /** + * Authorization header value for the download requests, or null for none. + * The caller supplies it because the assets live on the repository's own + * source: a GitHub token for a GitHub release, that instance's token for a + * Gitea or Forgejo release, and nothing for a tokenless public source. This + * function must never pick a credential itself, or one host's token would + * end up on another host (#440). + */ + downloadAuthorization: string | null; tagName: string; /** Injected in tests; production uses the global fetch. */ fetchImpl?: typeof fetch; @@ -3162,7 +3261,7 @@ export async function reconcileReleaseAssets({ let failed = 0; let skipped = 0; - if (!githubAssets || githubAssets.length === 0) { + if (!sourceAssets || sourceAssets.length === 0) { return { uploaded, failed, skipped }; } @@ -3190,13 +3289,13 @@ export async function reconcileReleaseAssets({ console.error( `[Releases] Could not list the existing assets of ${tagName}: ${ listError instanceof Error ? listError.message : String(listError) - }. Skipping all ${githubAssets.length} asset(s) for this release so no duplicate is created; the next sync retries.` + }. Skipping all ${sourceAssets.length} asset(s) for this release so no duplicate is created; the next sync retries.` ); - return { uploaded, failed: githubAssets.length, skipped }; + return { uploaded, failed: sourceAssets.length, skipped }; } const { toUpload, toSkip, toDelete } = classifyAssetsForReconciliation( - githubAssets, + sourceAssets, existingAssets ); skipped = toSkip.length; @@ -3234,10 +3333,10 @@ export async function reconcileReleaseAssets({ } } - const githubByName = new Map(githubAssets.map((a) => [a.name, a])); + const sourceByName = new Map(sourceAssets.map((a) => [a.name, a])); for (const name of toUpload) { - const asset = githubByName.get(name)!; + const asset = sourceByName.get(name)!; if (undeletedNames.has(name)) { console.warn( @@ -3248,16 +3347,16 @@ export async function reconcileReleaseAssets({ } try { - // Download from GitHub. fetch strips the Authorization header on the - // cross-host redirect to GitHub's object storage, so this works for both - // public and private release assets. + // Download from the source host. fetch strips the Authorization header + // on a cross-host redirect (GitHub redirects to its object storage), so + // this works for both public and private release assets. console.log( `[Releases] Downloading asset: ${asset.name} (${asset.size} bytes) for ${tagName}` ); const assetResponse = await doFetch(asset.browser_download_url, { headers: { Accept: "application/octet-stream", - Authorization: `token ${decryptedConfig.githubConfig!.token}`, + ...(downloadAuthorization ? { Authorization: downloadAuthorization } : {}), }, }); @@ -3306,6 +3405,26 @@ export async function reconcileReleaseAssets({ return { uploaded, failed, skipped }; } +/** + * Where one release mirror pass gets its releases and its download credentials. + * + * The list-and-reconcile logic below is identical for every source; only the + * listing call, the header label and the Authorization header used to download + * assets differ (#440). Keeping those three in one object is what stops a + * GitHub token from ever being sent to a Gitea host, or the other way round. + */ +type ReleaseSourceAdapter = { + /** Host label for the release body header, e.g. "GitHub". */ + label: string; + /** + * Authorization header value for asset downloads, or null for no header. + * Always a credential for the host the assets live on, never for another. + */ + downloadAuthorization: string | null; + /** The newest `limit` releases of the source repository, newest first. */ + listReleases: (owner: string, repo: string, limit: number) => Promise; +}; + type MirrorReleasesParams = { octokit: Octokit; repository: Repository; @@ -3318,6 +3437,12 @@ type MirrorReleasesParams = { * here so direct callers still honor per-repo overrides. */ releaseLimit?: number; + /** + * Decrypted token of the repository's own GitHub source, used to download + * assets. Absent or empty, the primary config's GitHub token is used, which + * is the historical behaviour for single-source setups. + */ + sourceToken?: string; /** * Already-resolved asset limit (#311): assets are uploaded only for the * newest N of the mirrored releases. `null` means every release, 0 means @@ -3326,6 +3451,11 @@ type MirrorReleasesParams = { releaseAssetLimit?: number | null; }; +/** The same parameters with the source adapter in place of the GitHub client. */ +type MirrorSourceReleasesParams = Omit & { + source: ReleaseSourceAdapter; +}; + /** * Lock key for one destination repository's releases: the destination base URL * plus the owner and repository name it is mirrored as, lowercased. @@ -3355,7 +3485,7 @@ export function buildReleaseTargetLockKey( * both upload, leaving duplicates. The lock makes the read and the write one * unit per destination repository. */ -export async function mirrorGitHubReleasesToGitea(params: MirrorReleasesParams) { +async function mirrorSourceReleasesToGitea(params: MirrorSourceReleasesParams) { const { config, repository, giteaOwner, giteaRepoName } = params; if ( @@ -3374,7 +3504,7 @@ export async function mirrorGitHubReleasesToGitea(params: MirrorReleasesParams) return withKeyedLock( buildReleaseTargetLockKey(config.giteaConfig.url, repoOwner, repoName), () => - mirrorGitHubReleasesToGiteaLocked({ + mirrorSourceReleasesToGiteaLocked({ ...params, giteaOwner: repoOwner, giteaRepoName: repoName, @@ -3382,15 +3512,110 @@ export async function mirrorGitHubReleasesToGitea(params: MirrorReleasesParams) ); } -async function mirrorGitHubReleasesToGiteaLocked({ - octokit, +/** + * Mirror releases from a GitHub source. + * + * The historical entry point: same signature and same behaviour as before the + * mirror was made source agnostic, including the "Originally published on + * GitHub:" header line, which must stay byte for byte or every already + * mirrored release body would differ and be PATCHed on the next sync. + */ +export async function mirrorGitHubReleasesToGitea(params: MirrorReleasesParams) { + const { octokit, ...rest } = params; + + // The source token downloads the assets; the destination token is read + // inside the locked body. An empty token means an anonymous public client, + // and an empty `token ` header is worse than sending none at all. + const decryptedConfig = decryptConfigTokens(params.config as Config); + const githubToken = + params.sourceToken?.trim() || decryptedConfig.githubConfig?.token?.trim() || ""; + + return mirrorSourceReleasesToGitea({ + ...rest, + source: { + label: SOURCE_PROVIDER_LABELS.github, + downloadAuthorization: githubToken ? `token ${githubToken}` : null, + listReleases: async (owner, repo, limit) => { + // GitHub API max per page is 100; paginate until we reach the limit. + const releases: Awaited< + ReturnType + >["data"] = []; + let page = 1; + const perPage = Math.min(100, limit); + + while (releases.length < limit) { + const response = await octokit.rest.repos.listReleases({ + owner, + repo, + per_page: perPage, + page, + }); + + if (response.data.length === 0) { + break; + } + + releases.push(...response.data); + + if (response.data.length < perPage) { + break; + } + + page++; + } + + return releases.slice(0, limit).map((release) => ({ + tag_name: release.tag_name, + name: release.name, + body: release.body, + draft: release.draft, + prerelease: release.prerelease, + created_at: release.created_at, + published_at: release.published_at, + assets: (release.assets ?? []).map((asset) => ({ + name: asset.name, + size: asset.size, + browser_download_url: asset.browser_download_url, + })), + })); + }, + }, + }); +} + +/** + * Mirror releases from a Gitea or Forgejo source, Codeberg included (#440). + * + * The connection is the repository's own source row, so assets are downloaded + * with that instance's token when it has one and anonymously when it does not. + * Public repositories need no token at all. + */ +export async function mirrorGiteaSourceReleasesToGitea( + params: Omit & { connection: SourceConnection } +) { + const { connection, ...rest } = params; + const provider = new GiteaSourceProvider(connection); + const token = connection.token.trim(); + + return mirrorSourceReleasesToGitea({ + ...rest, + source: { + label: SOURCE_PROVIDER_LABELS.gitea, + downloadAuthorization: token ? `token ${token}` : null, + listReleases: (owner, repo, limit) => provider.listReleases(owner, repo, limit), + }, + }); +} + +async function mirrorSourceReleasesToGiteaLocked({ + source, repository, config, giteaOwner, giteaRepoName, releaseLimit: releaseLimitOverride, releaseAssetLimit: releaseAssetLimitOverride, -}: MirrorReleasesParams) { +}: MirrorSourceReleasesParams) { if ( !config.giteaConfig?.defaultOwner || !config.giteaConfig?.token || @@ -3406,13 +3631,13 @@ async function mirrorGitHubReleasesToGiteaLocked({ const repoOwner = giteaOwner || (await getGiteaRepoOwnerAsync({ config, repository })); const repoName = giteaRepoName || repository.name; - // Derive GITHUB coordinates from fullName, matching the issues/PRs/labels/ + // Derive the SOURCE coordinates from fullName, matching the issues/PRs/labels/ // milestones mirror functions (`const [owner, repo] = repository.fullName.split("/")`). - // repository.name/owner can drift from the GitHub source (e.g. Gitea-side - // renames), so fullName is authoritative; fall back only if it's malformed. + // repository.name/owner can drift from the source (e.g. Gitea-side renames), + // so fullName is authoritative; fall back only if it's malformed. const [fullNameOwner, fullNameRepo] = (repository.fullName || "").split("/"); - const githubOwner = fullNameOwner && fullNameRepo ? fullNameOwner : repository.owner; - const githubRepo = fullNameOwner && fullNameRepo ? fullNameRepo : repository.name; + const sourceOwner = fullNameOwner && fullNameRepo ? fullNameOwner : repository.owner; + const sourceRepo = fullNameOwner && fullNameRepo ? fullNameRepo : repository.name; // Verify the repository exists in Gitea before attempting to mirror releases console.log(`[Releases] Verifying repository ${repoName} exists at ${repoOwner}`); @@ -3421,7 +3646,7 @@ async function mirrorGitHubReleasesToGiteaLocked({ owner: repoOwner, repoName: repoName, }); - + if (!repoExists) { console.error(`[Releases] Repository ${repository.name} not found at ${repoOwner}. Cannot mirror releases.`); throw new Error(`Repository ${repository.name} does not exist in Gitea at ${repoOwner}. Please ensure the repository is mirrored first.`); @@ -3448,35 +3673,11 @@ async function mirrorGitHubReleasesToGiteaLocked({ ? (normalizeReleaseAssetLimit(releaseAssetLimitOverride) ?? null) : (await resolveOptions()).releaseAssetLimit; - // GitHub API max per page is 100; paginate until we reach the configured limit. - const releases: Awaited< - ReturnType - >["data"] = []; - let page = 1; - const perPage = Math.min(100, releaseLimit); - - while (releases.length < releaseLimit) { - const response = await octokit.rest.repos.listReleases({ - owner: githubOwner, - repo: githubRepo, - per_page: perPage, - page, - }); - - if (response.data.length === 0) { - break; - } - - releases.push(...response.data); - - if (response.data.length < perPage) { - break; - } - - page++; - } - - const limitedReleases = releases.slice(0, releaseLimit); + // The source lists its own newest releases; everything below this point is + // the same reconciliation whichever host they came from. + const limitedReleases = ( + await source.listReleases(sourceOwner, sourceRepo, releaseLimit) + ).slice(0, releaseLimit); console.log( `[Releases] Found ${limitedReleases.length} releases (limited to latest ${releaseLimit}; ${ @@ -3498,7 +3699,7 @@ async function mirrorGitHubReleasesToGiteaLocked({ let totalAssetsFailed = 0; let releasesPastAssetLimit = 0; - // Process releases in their GitHub API order (newest first by default) + // Process releases in the source API's order (newest first by default) const releasesToProcess = limitedReleases.slice(); console.log(`[Releases] Processing ${releasesToProcess.length} releases for ${repository.fullName}`); @@ -3523,34 +3724,34 @@ async function mirrorGitHubReleasesToGiteaLocked({ } ).catch(() => null); - // Prepare release body with GitHub original date header - const githubPublishedDate = release.published_at || release.created_at; - const githubTagCreatedDate = release.created_at; + // Prepare release body with the source's original date header + const sourcePublishedDate = release.published_at || release.created_at; + const sourceTagCreatedDate = release.created_at; - let githubDateHeader = ''; - if (githubPublishedDate) { - githubDateHeader = `> 📅 **Originally published on GitHub:** ${new Date(githubPublishedDate).toUTCString()}`; + let sourceDateHeader = ''; + if (sourcePublishedDate) { + sourceDateHeader = `> 📅 **Originally published on ${source.label}:** ${new Date(sourcePublishedDate).toUTCString()}`; // If the tag was created on a different date than the release was published, // show both dates (helps with repos that create multiple tags from the same commit) if (release.published_at && release.created_at && release.published_at !== release.created_at) { - githubDateHeader += `\n> 🏷️ **Git tag created:** ${new Date(githubTagCreatedDate).toUTCString()}`; + sourceDateHeader += `\n> 🏷️ **Git tag created:** ${new Date(sourceTagCreatedDate).toUTCString()}`; } - githubDateHeader += '\n\n'; + sourceDateHeader += '\n\n'; } const originalReleaseNote = release.body || ""; - const releaseNote = githubDateHeader + originalReleaseNote; + const releaseNote = sourceDateHeader + originalReleaseNote; if (existingReleasesResponse) { // Update existing release if the changelog/body differs const existingRelease = existingReleasesResponse.data; const existingNote = existingRelease.body || ""; - + if (existingNote !== releaseNote || existingRelease.name !== (release.name || release.tag_name)) { console.log(`[Releases] Updating existing release ${release.tag_name} with new changelog/title`); - + await httpPatch( `${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${existingRelease.id}`, buildGiteaReleasePayload(release, releaseNote), @@ -3560,9 +3761,9 @@ async function mirrorGitHubReleasesToGiteaLocked({ ); if (originalReleaseNote) { - console.log(`[Releases] Updated changelog for ${release.tag_name} (${originalReleaseNote.length} characters + GitHub date header)`); + console.log(`[Releases] Updated changelog for ${release.tag_name} (${originalReleaseNote.length} characters + ${source.label} date header)`); } else { - console.log(`[Releases] Updated release ${release.tag_name} with GitHub date header`); + console.log(`[Releases] Updated release ${release.tag_name} with ${source.label} date header`); } mirroredCount++; } else { @@ -3584,7 +3785,8 @@ async function mirrorGitHubReleasesToGiteaLocked({ repoOwner, repoName, giteaReleaseId: existingRelease.id, - githubAssets: release.assets || [], + sourceAssets: release.assets || [], + downloadAuthorization: source.downloadAuthorization, tagName: release.tag_name, }); if (assetResult.uploaded > 0) { @@ -3626,11 +3828,11 @@ async function mirrorGitHubReleasesToGiteaLocked({ continue; } - // Create new release with changelog/body content (includes GitHub date header) + // Create new release with changelog/body content (includes the date header) if (originalReleaseNote) { - console.log(`[Releases] Including changelog for ${release.tag_name} (${originalReleaseNote.length} characters + GitHub date header)`); + console.log(`[Releases] Including changelog for ${release.tag_name} (${originalReleaseNote.length} characters + ${source.label} date header)`); } else { - console.log(`[Releases] Creating release ${release.tag_name} with GitHub date header (no changelog)`); + console.log(`[Releases] Creating release ${release.tag_name} with ${source.label} date header (no changelog)`); } const createReleaseResponse = await httpPost( @@ -3640,7 +3842,7 @@ async function mirrorGitHubReleasesToGiteaLocked({ Authorization: `token ${decryptedConfig.giteaConfig.token}`, } ); - + // Mirror release assets if they exist (idempotent — see reconcileReleaseAssets) // and the release sits inside the asset limit (#311). if (release.assets && release.assets.length > 0) { @@ -3652,7 +3854,8 @@ async function mirrorGitHubReleasesToGiteaLocked({ repoOwner, repoName, giteaReleaseId: createReleaseResponse.data.id, - githubAssets: release.assets, + sourceAssets: release.assets, + downloadAuthorization: source.downloadAuthorization, tagName: release.tag_name, }); totalAssetsUploaded += assetResult.uploaded; diff --git a/src/lib/source-provider-wiring.test.ts b/src/lib/source-provider-wiring.test.ts index 6c4e822..ad1a90d 100644 --- a/src/lib/source-provider-wiring.test.ts +++ b/src/lib/source-provider-wiring.test.ts @@ -44,7 +44,17 @@ describe("gitea.ts migrate sites", () => { test("the mirror entry points accept a null GitHub client and skip metadata without one", () => { expect(count(source, "octokit: Octokit | null;")).toBeGreaterThanOrEqual(4); - expect(count(source, "&& octokit !== null;")).toBe(10); + // Releases no longer gate on a GitHub client here: both mirror paths hand + // the decision to mirrorRepositoryReleases, which picks the source's own + // release path (#440). Everything else still needs Octokit. + expect(count(source, "&& octokit !== null;")).toBe(8); + }); + + test("both mirror paths route releases through the repository's own source", () => { + expect(count(source, "await mirrorRepositoryReleases({")).toBe(2); + expect(count(source, "sourceToken: repoSourceToken,")).toBe(2); + expect(source).toContain('if (sourceConnection.provider === "gitea") {'); + expect(source).toContain("await mirrorGiteaSourceReleasesToGitea({"); }); }); @@ -77,14 +87,25 @@ describe("discovery and housekeeping go through the source provider", () => { expect(recovery).toContain("findSourceForRepository(repo, sources)"); expect(recovery).toContain("repoSource?.provider === 'github'"); const enhanced = read("gitea-enhanced.ts"); - expect(enhanced).toContain("getRepositorySource(repository).provider === \"github\""); + expect(enhanced).toContain("const repoSourceKind = getRepositorySource(repository).provider;"); + expect(enhanced).toContain('const repoIsGitHub = repoSourceKind === "github";'); expect(enhanced).toContain("findSourceForRepository(repository, sources)"); + // The sync path mirrors releases from a Gitea/Forgejo source too (#440). + expect(enhanced).toContain('repoSourceKind === "gitea" && repoSource'); + expect(enhanced).toContain("mirrorGiteaSourceReleasesToGitea"); }); test("the metadata resolver clamps GitHub only options for other sources", () => { const source = read("utils", "mirror-overrides.ts"); expect(source).toContain("GITHUB_ONLY_METADATA_KEYS"); - expect(source).toContain('normalizeSourceProviderKind(repository.sourceProvider) !== "github"'); + expect(source).toContain("const sourceKind = normalizeSourceProviderKind(repository.sourceProvider);"); + expect(source).toContain('if (sourceKind !== "github") {'); + // Releases have their own gate: GitHub and Gitea/Forgejo can list them, + // GitLab cannot, so they left the GitHub only set (#440). + expect(source).toContain( + 'export const GITHUB_ONLY_METADATA_KEYS = [\n "mirrorMetadata",' + ); + expect(source).toContain("if (!sourceKindSupportsReleases(sourceKind)) {"); }); test("the discovery routes use the provider, match the pasted host against connected sources and stamp the source on inserted rows", () => { diff --git a/src/lib/source-providers/gitea-source-releases.test.ts b/src/lib/source-providers/gitea-source-releases.test.ts new file mode 100644 index 0000000..84666fd --- /dev/null +++ b/src/lib/source-providers/gitea-source-releases.test.ts @@ -0,0 +1,203 @@ +/** + * Unit tests for the Gitea/Forgejo source adapter's release listing (#440). + * + * Release mirroring used to be GitHub only. Gitea and Forgejo expose + * `GET /repos/{owner}/{repo}/releases` with the same limit/page pagination and + * x-total-count header as their other list endpoints, which is what lets a + * Codeberg or self hosted repository mirror its releases. + * + * The global fetch is swapped rather than the module, because bun's + * mock.module is process-wide and leaks into other test files. + */ + +import { afterEach, describe, expect, it } from "bun:test"; +import { GiteaSourceProvider } from "./gitea-source"; +import type { SourceConnection } from "./types"; + +const SOURCE_URL = "https://codeberg.org"; +const RELEASES_PATH = `${SOURCE_URL}/api/v1/repos/acme/demo/releases`; + +const realFetch = globalThis.fetch; + +afterEach(() => { + globalThis.fetch = realFetch; +}); + +function connection(token = "source-token"): SourceConnection { + return { + provider: "gitea", + url: SOURCE_URL, + username: "acme", + token, + }; +} + +function release(tag: string, extra: Record = {}) { + return { + id: Number(tag.replace(/\D/g, "")) || 1, + tag_name: tag, + name: `Release ${tag}`, + body: `notes for ${tag}`, + draft: false, + prerelease: false, + created_at: "2026-01-01T00:00:00Z", + published_at: "2026-01-02T00:00:00Z", + assets: [], + ...extra, + }; +} + +interface FakeOptions { + /** Pages returned in order, newest first overall. */ + pages: Array>>; + /** Value for the x-total-count header, or null to omit it. */ + totalCount?: number | null; +} + +function fakeSource(options: FakeOptions) { + const requests: Array<{ url: string; authorization: string | null }> = []; + + globalThis.fetch = (async (input: any, init?: any) => { + const url = typeof input === "string" ? input : String(input); + const headers = new Headers(init?.headers ?? {}); + requests.push({ url, authorization: headers.get("authorization") }); + + const page = Number(new URL(url).searchParams.get("page") ?? "1"); + const body = options.pages[page - 1] ?? []; + const responseHeaders: Record = { + "content-type": "application/json", + }; + if (options.totalCount !== null && options.totalCount !== undefined) { + responseHeaders["x-total-count"] = String(options.totalCount); + } + return new Response(JSON.stringify(body), { + status: 200, + headers: responseHeaders, + }); + }) as unknown as typeof fetch; + + return requests; +} + +describe("GiteaSourceProvider.listReleases", () => { + it("asks for the newest releases with limit and page", async () => { + const requests = fakeSource({ pages: [[release("v1.0.0"), release("v0.9.0")]], totalCount: 2 }); + + const releases = await new GiteaSourceProvider(connection()).listReleases("acme", "demo", 10); + + expect(releases.map((r) => r.tag_name)).toEqual(["v1.0.0", "v0.9.0"]); + expect(requests).toHaveLength(1); + expect(requests[0].url).toBe(`${RELEASES_PATH}?limit=10&page=1`); + }); + + it("caps the page size at the requested limit and never returns more", async () => { + const requests = fakeSource({ + pages: [[release("v3"), release("v2"), release("v1")]], + totalCount: 99, + }); + + const releases = await new GiteaSourceProvider(connection()).listReleases("acme", "demo", 2); + + expect(requests[0].url).toBe(`${RELEASES_PATH}?limit=2&page=1`); + // The instance may over-serve; the limit still wins. + expect(releases.map((r) => r.tag_name)).toEqual(["v3", "v2"]); + expect(requests).toHaveLength(1); + }); + + it("pages until the limit is reached, capping the page size at 50", async () => { + const firstPage = Array.from({ length: 50 }, (_, i) => release(`v${100 - i}`)); + const secondPage = [release("v50"), release("v49")]; + const requests = fakeSource({ pages: [firstPage, secondPage], totalCount: 120 }); + + const releases = await new GiteaSourceProvider(connection()).listReleases("acme", "demo", 52); + + expect(releases).toHaveLength(52); + expect(requests.map((r) => r.url)).toEqual([ + `${RELEASES_PATH}?limit=50&page=1`, + `${RELEASES_PATH}?limit=2&page=2`, + ]); + }); + + it("stops at x-total-count instead of asking for a page that cannot exist", async () => { + const firstPage = Array.from({ length: 50 }, (_, i) => release(`v${100 - i}`)); + const requests = fakeSource({ pages: [firstPage, []], totalCount: 50 }); + + const releases = await new GiteaSourceProvider(connection()).listReleases("acme", "demo", 100); + + expect(releases).toHaveLength(50); + expect(requests).toHaveLength(1); + }); + + it("falls back to a short page when the instance omits x-total-count", async () => { + const firstPage = Array.from({ length: 50 }, (_, i) => release(`v${100 - i}`)); + const requests = fakeSource({ pages: [firstPage, [release("v50")]], totalCount: null }); + + const releases = await new GiteaSourceProvider(connection()).listReleases("acme", "demo", 100); + + expect(releases).toHaveLength(51); + expect(requests).toHaveLength(2); + }); + + it("sends the source token when the connection has one", async () => { + const requests = fakeSource({ pages: [[release("v1")]], totalCount: 1 }); + + await new GiteaSourceProvider(connection("source-token")).listReleases("acme", "demo", 5); + + expect(requests[0].authorization).toBe("token source-token"); + }); + + it("sends no Authorization header for a tokenless public source", async () => { + const requests = fakeSource({ pages: [[release("v1")]], totalCount: 1 }); + + await new GiteaSourceProvider(connection("")).listReleases("acme", "demo", 5); + + expect(requests[0].authorization).toBeNull(); + }); + + it("normalizes assets and drops entries with nothing to download", async () => { + fakeSource({ + pages: [ + [ + release("v1.0.0", { + assets: [ + { + id: 1, + name: "firmware.bin", + size: 2048, + browser_download_url: `${SOURCE_URL}/attachments/abc`, + }, + { id: 2, name: "no-url.bin", size: 10 }, + { id: 3, size: 10, browser_download_url: `${SOURCE_URL}/attachments/def` }, + ], + }), + ], + ], + totalCount: 1, + }); + + const [mirrored] = await new GiteaSourceProvider(connection()).listReleases( + "acme", + "demo", + 5 + ); + + expect(mirrored.assets).toEqual([ + { + name: "firmware.bin", + size: 2048, + browser_download_url: `${SOURCE_URL}/attachments/abc`, + }, + ]); + expect(mirrored.name).toBe("Release v1.0.0"); + expect(mirrored.body).toBe("notes for v1.0.0"); + expect(mirrored.published_at).toBe("2026-01-02T00:00:00Z"); + }); + + it("drops releases without a tag, which cannot be mirrored", async () => { + fakeSource({ pages: [[release("v1"), { ...release("x"), tag_name: "" }]], totalCount: 2 }); + + const releases = await new GiteaSourceProvider(connection()).listReleases("acme", "demo", 5); + + expect(releases.map((r) => r.tag_name)).toEqual(["v1"]); + }); +}); diff --git a/src/lib/source-providers/gitea-source.ts b/src/lib/source-providers/gitea-source.ts index d36e88d..67e40ec 100644 --- a/src/lib/source-providers/gitea-source.ts +++ b/src/lib/source-providers/gitea-source.ts @@ -24,6 +24,7 @@ import type { SourceFailedOrganization, SourceOrganizationResult, SourceProvider, + SourceRelease, SourceRepositoryPath, } from "./types"; @@ -60,6 +61,50 @@ export interface GiteaRepository { updated_at?: string; } +export interface GiteaReleaseAsset { + id?: number; + name?: string; + size?: number; + browser_download_url?: string; +} + +export interface GiteaRelease { + id?: number; + tag_name?: string; + name?: string | null; + body?: string | null; + draft?: boolean; + prerelease?: boolean; + created_at?: string; + published_at?: string | null; + assets?: GiteaReleaseAsset[] | null; +} + +/** + * Normalize one Gitea/Forgejo release into the shape the release mirror works + * on. Assets without a name or a download URL are dropped: there is nothing to + * fetch or to name the attachment after. + */ +export function mapGiteaRelease(release: GiteaRelease): SourceRelease { + const createdAt = release.created_at || release.published_at || ""; + return { + tag_name: release.tag_name || "", + name: release.name ?? null, + body: release.body ?? "", + draft: Boolean(release.draft), + prerelease: Boolean(release.prerelease), + created_at: createdAt, + published_at: release.published_at ?? null, + assets: (release.assets ?? []) + .filter((asset) => !!asset?.name && !!asset?.browser_download_url) + .map((asset) => ({ + name: asset.name!, + size: Number(asset.size) || 0, + browser_download_url: asset.browser_download_url!, + })), + }; +} + export interface GiteaOrganization { id?: number; username?: string; @@ -167,24 +212,31 @@ export class GiteaSourceProvider implements SourceProvider { private async paginate( path: string, - params: Record = {} + params: Record = {}, + /** Stop once this many items are collected. Absent means every page. */ + maxItems?: number ): Promise { const items: T[] = []; for (let page = 1; page <= MAX_PAGES; page += 1) { + const pageSize = + maxItems === undefined ? PAGE_SIZE : Math.min(PAGE_SIZE, maxItems - items.length); + if (pageSize <= 0) break; + const url = withQuery(`${this.apiRoot}${path}`, { ...params, - limit: PAGE_SIZE, + limit: pageSize, page, }); const { data, response } = await sourceFetch(url, { headers: this.headers() }); if (!Array.isArray(data) || data.length === 0) break; items.push(...data); + if (maxItems !== undefined && items.length >= maxItems) break; const header = response.headers.get("x-total-count"); const total = header === null ? Number.NaN : Number(header); if (Number.isFinite(total)) { if (items.length >= total) break; - } else if (data.length < PAGE_SIZE) { + } else if (data.length < pageSize) { break; } } @@ -364,6 +416,26 @@ export class GiteaSourceProvider implements SourceProvider { } } + /** + * The newest `limit` releases of a repository (#440). + * + * Gitea and Forgejo return releases newest first and accept the same + * limit/page pagination as every other list endpoint. Public repositories + * answer without a token, so a tokenless source can mirror releases too. + */ + async listReleases(owner: string, repo: string, limit: number): Promise { + const max = Math.max(1, Math.floor(limit) || 1); + const releases = await this.paginate( + `/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/releases`, + {}, + max + ); + return releases + .slice(0, max) + .map(mapGiteaRelease) + .filter((release) => release.tag_name.length > 0); + } + async testConnection(): Promise { if (!this.hasToken) { throw new Error("An access token is required."); diff --git a/src/lib/source-providers/kinds.ts b/src/lib/source-providers/kinds.ts index a6be2a4..7783ab0 100644 --- a/src/lib/source-providers/kinds.ts +++ b/src/lib/source-providers/kinds.ts @@ -16,6 +16,26 @@ export function isBetaSourceProvider(kind: SourceProviderKind): boolean { return kind !== DEFAULT_SOURCE_PROVIDER; } +/** + * Sources whose API can list releases with their assets. + * + * GitHub has always been able to. Gitea and Forgejo expose the same shape at + * `GET /repos/{owner}/{repo}/releases`, so Codeberg and self hosted instances + * can mirror releases too (#440). GitLab is code only: its release API is not + * wired up, so the resolver still forces releases off for it. + */ +export const RELEASE_CAPABLE_SOURCE_KINDS = [ + "github", + "gitea", +] as const satisfies readonly SourceProviderKind[]; + +/** True when a source kind's API can list releases. Unknown values are GitHub rows. */ +export function sourceKindSupportsReleases(kind: unknown): boolean { + return (RELEASE_CAPABLE_SOURCE_KINDS as readonly string[]).includes( + normalizeSourceProviderKind(kind) + ); +} + /** Base URL used when the config has no instance URL for the provider. */ export const SOURCE_PROVIDER_DEFAULT_URLS: Record = { github: "https://github.com", diff --git a/src/lib/source-providers/types.ts b/src/lib/source-providers/types.ts index 96fd01b..5591cd8 100644 --- a/src/lib/source-providers/types.ts +++ b/src/lib/source-providers/types.ts @@ -51,6 +51,32 @@ export interface SourceRepositoryPath { repo: string; } +/** One file attached to a release on a source host. */ +export interface SourceReleaseAsset { + name: string; + size: number; + /** Direct download URL on the source host. */ + browser_download_url: string; +} + +/** + * One release, in the shape the release mirror works on. + * + * Field names follow the GitHub and Gitea payloads (both use snake_case and + * agree on every field here), so the GitHub path needed no mapping when the + * mirror was made source agnostic (#440). + */ +export interface SourceRelease { + tag_name: string; + name?: string | null; + body?: string | null; + draft?: boolean; + prerelease?: boolean; + created_at: string; + published_at?: string | null; + assets: SourceReleaseAsset[]; +} + /** * The operations the mirror pipeline needs from a source host. * @@ -92,6 +118,15 @@ export interface SourceProvider { /** Whether the account currently stars the repository. Throws on errors other than 404. */ isRepositoryStarred(owner: string, name: string): Promise; + /** + * The newest `limit` releases of a repository, newest first. + * + * Optional: only hosts whose API exposes releases implement it (GitHub and + * Gitea/Forgejo). GitLab is code only, so it leaves this out and + * `sourceKindSupportsReleases` keeps the release switch off for it. + */ + listReleases?(owner: string, repo: string, limit: number): Promise; + /** Verify the token and return the account it belongs to. */ testConnection(): Promise; diff --git a/src/lib/utils/mirror-overrides.test.ts b/src/lib/utils/mirror-overrides.test.ts index 3093a9f..9978128 100644 --- a/src/lib/utils/mirror-overrides.test.ts +++ b/src/lib/utils/mirror-overrides.test.ts @@ -822,6 +822,7 @@ import { GITHUB_ONLY_METADATA_KEYS, resolveMirrorOptions as resolveOptionsForSource, } from "./mirror-overrides"; +import { sourceKindSupportsReleases } from "@/lib/source-providers/kinds"; describe("resolveMirrorOptions for non-GitHub sources", () => { const config = { @@ -870,6 +871,87 @@ describe("resolveMirrorOptions for non-GitHub sources", () => { expect(resolved.mirrorIssues).toBe(true); expect(resolved.mirrorReleases).toBe(true); }); + + // #440: Gitea and Forgejo expose releases through their own API, so the + // release switch is no longer part of the GitHub only clamp. + test("a Gitea repository keeps releases while the GitHub only metadata is clamped", () => { + const resolved = resolveOptionsForSource({ + config, + repository: { isStarred: false, mirrorOverrides: null, sourceProvider: "gitea" } as any, + }); + expect(resolved.mirrorReleases).toBe(true); + expect(resolved.releaseLimit).toBe(5); + expect(resolved.mirrorIssues).toBe(false); + expect(resolved.mirrorPullRequests).toBe(false); + expect(resolved.mirrorLabels).toBe(false); + expect(resolved.mirrorMilestones).toBe(false); + expect(resolved.mirrorMetadata).toBe(false); + }); + + test("forgejo and codeberg spellings resolve to the Gitea kind and keep releases", () => { + for (const sourceProvider of ["forgejo", "codeberg"]) { + const resolved = resolveOptionsForSource({ + config, + repository: { isStarred: false, mirrorOverrides: null, sourceProvider } as any, + }); + expect(resolved.mirrorReleases).toBe(true); + expect(resolved.mirrorIssues).toBe(false); + } + }); + + test("a GitLab repository still has releases and every other metadata flag clamped", () => { + const resolved = resolveOptionsForSource({ + config, + repository: { + isStarred: false, + mirrorOverrides: { mirrorReleases: true }, + sourceProvider: "gitlab", + } as any, + }); + expect(resolved.mirrorReleases).toBe(false); + for (const key of GITHUB_ONLY_METADATA_KEYS) { + expect(resolved[key]).toBe(false); + } + }); + + test("releases left the GitHub only set, so the constant no longer lists them", () => { + expect(GITHUB_ONLY_METADATA_KEYS as readonly string[]).not.toContain("mirrorReleases"); + }); + + test("a Gitea repository still loses releases on a push destination", () => { + const resolved = resolveOptionsForSource({ + config: { + ...config, + giteaConfig: { ...config.giteaConfig, provider: "github" }, + } as any, + repository: { isStarred: false, mirrorOverrides: null, sourceProvider: "gitea" } as any, + }); + expect(resolved.mirrorReleases).toBe(false); + }); + + test("starredCodeOnly still outranks the Gitea release capability", () => { + const resolved = resolveOptionsForSource({ + config: { ...config, githubConfig: { starredCodeOnly: true } } as any, + repository: { isStarred: true, mirrorOverrides: null, sourceProvider: "gitea" } as any, + }); + expect(resolved.mirrorReleases).toBe(false); + }); +}); + +describe("sourceKindSupportsReleases", () => { + test("GitHub and Gitea can list releases, GitLab cannot", () => { + expect(sourceKindSupportsReleases("github")).toBe(true); + expect(sourceKindSupportsReleases("gitea")).toBe(true); + expect(sourceKindSupportsReleases("forgejo")).toBe(true); + expect(sourceKindSupportsReleases("codeberg")).toBe(true); + expect(sourceKindSupportsReleases("gitlab")).toBe(false); + }); + + test("an unknown value is a GitHub row, as everywhere else", () => { + expect(sourceKindSupportsReleases(undefined)).toBe(true); + expect(sourceKindSupportsReleases(null)).toBe(true); + expect(sourceKindSupportsReleases("bitbucket")).toBe(true); + }); }); describe("release asset limit (#311)", () => { diff --git a/src/lib/utils/mirror-overrides.ts b/src/lib/utils/mirror-overrides.ts index 523e6e2..2743651 100644 --- a/src/lib/utils/mirror-overrides.ts +++ b/src/lib/utils/mirror-overrides.ts @@ -1,7 +1,10 @@ import type { Config } from "@/types/config"; import type { MirrorOverrides, Repository } from "@/lib/db/schema"; import { mirrorOverridesSchema } from "@/lib/db/schema"; -import { normalizeSourceProviderKind } from "@/lib/source-providers/kinds"; +import { + normalizeSourceProviderKind, + sourceKindSupportsReleases, +} from "@/lib/source-providers/kinds"; import { isPushDestinationKind, normalizeDestinationProviderKind } from "@/lib/destination-kinds"; /** @@ -139,13 +142,16 @@ export const STARRED_CLAMPED_KEYS = [ /** * Flags that only work for GitHub sources. * - * Issue, pull request, release, label and milestone mirroring read the - * GitHub API. A repository from GitLab or Gitea gets code, wiki and LFS - * through the Gitea pull mirror and nothing else, so the resolver forces - * these off for it regardless of what any tier asked for. + * Issue, pull request, label and milestone mirroring read the GitHub API. A + * repository from GitLab or Gitea gets code, wiki and LFS through the Gitea + * pull mirror, so the resolver forces these off for it regardless of what any + * tier asked for. + * + * `mirrorReleases` is deliberately not in this list since #440: releases are + * gated on `sourceKindSupportsReleases` instead, which also allows Gitea and + * Forgejo sources. GitLab still ends up with releases off, through that gate. */ export const GITHUB_ONLY_METADATA_KEYS = [ - "mirrorReleases", "mirrorMetadata", "mirrorIssues", "mirrorPullRequests", @@ -531,12 +537,19 @@ export function resolveMirrorOptions({ // Metadata mirroring needs the GitHub API. Rows without a source // provider predate the column and came from GitHub. - if (normalizeSourceProviderKind(repository.sourceProvider) !== "github") { + const sourceKind = normalizeSourceProviderKind(repository.sourceProvider); + if (sourceKind !== "github") { for (const key of GITHUB_ONLY_METADATA_KEYS) { resolved[key] = false; } } + // Releases are listed through the source API, which GitHub and + // Gitea/Forgejo both expose (#440). GitLab is code only, so it lands here. + if (!sourceKindSupportsReleases(sourceKind)) { + resolved.mirrorReleases = false; + } + // A push target receives branches and tags only. if (isPushDestinationConfig(config)) { for (const key of PUSH_TARGET_CLAMPED_KEYS) { diff --git a/www/src/pages/docs/environment-variables.mdx b/www/src/pages/docs/environment-variables.mdx index cbcac30..5cc693d 100644 --- a/www/src/pages/docs/environment-variables.mdx +++ b/www/src/pages/docs/environment-variables.mdx @@ -167,7 +167,7 @@ Only used when `DESTINATION_PROVIDER` is `github` or `gitlab`. > **Note:** Concurrency above `1` speeds up metadata mirroring but issues and pull requests may be created out of chronological order, since Gitea assigns numbers on creation. Set both to `1` if exact ordering matters more than throughput. -Issues, pull requests, labels, milestones and releases are read from the GitHub API and need a GitHub source. With a GitHub or GitLab destination none of the metadata options apply, and neither do wiki and LFS: push targets receive branches and tags only. +Issues, pull requests, labels and milestones are read from the GitHub API and need a GitHub source. Releases are read from the source's own API, so they mirror from GitHub and from Gitea/Forgejo sources, but not from GitLab. With a GitHub or GitLab destination none of the metadata options apply, and neither do wiki and LFS: push targets receive branches and tags only. ## Scheduling diff --git a/www/src/pages/docs/sources.mdx b/www/src/pages/docs/sources.mdx index 55fa056..a6255df 100644 --- a/www/src/pages/docs/sources.mdx +++ b/www/src/pages/docs/sources.mdx @@ -58,11 +58,11 @@ The code itself is copied by the destination: Gitea and Forgejo pull from the so | Private repositories | yes | yes | yes | | Add a single repository by URL | yes | yes | yes | | Issues, pull requests, labels, milestones | yes | no | no | -| Releases with assets | yes | no | no | +| Releases with assets | yes | no | yes | | Star lists | yes | no | no | | Force push detection | yes | no | no | -The GitHub only rows read the GitHub API. With another source the corresponding switches in the mirror settings are disabled and say why. Wiki and LFS also depend on the destination: they travel to Gitea and Forgejo, not to GitHub or GitLab. See [Destinations](./destinations.md). +The GitHub only rows read the GitHub API. With another source the corresponding switches in the mirror settings are disabled and say why. Releases are the exception: Gitea and Forgejo expose them through their own API, so a Gitea, Forgejo or Codeberg source mirrors releases and their assets with that instance's token, or anonymously for a public repository. Wiki and LFS also depend on the destination: they travel to Gitea and Forgejo, not to GitHub or GitLab. See [Destinations](./destinations.md). ## Adding repositories