Compare commits

...
5 Commits
Author SHA1 Message Date
Arunavo Ray 35b275035d chore: bump version to 3.49.0 2026-10-30 00:41:25 +05:30
ARUNAVO RAY ecdba3cad7 fix(ui): sort the Activity Log name filter case-insensitively too (#490)
Follow-up to #487: the owner, organization and Activity Log name filters share one compareNames (an Intl.Collator built once), so the Activity Log filter no longer puts capitalized names first.
2026-10-30 00:40:56 +05:30
Florian Zevedei 2485513973 fix(activities): accept the warning status in the activity feed (#488)
The feed parsed job statuses with the repository status enum, which has no "warning", so one release filter warning turned GET /api/activities into a 500. It now parses with activityStatusEnum. Adds a route test with warning and mirrored rows.
2026-10-30 00:32:56 +05:30
Florian Zevedei 43581f3e8b feat(releases): opt-in pruning of release assets past the asset limit (#489)
A global Remove old release assets switch (PRUNE_OLD_RELEASE_ASSETS) deletes the attachments of mirrored releases ranked past each repository's asset limit on every sync. Releases, notes and tags stay. Off by default. A release whose attachments cannot be listed is left alone, and releases the listing already shows without attachments are skipped.

Closes #485
2026-10-30 00:30:56 +05:30
Florian Zevedei 7e70e1005d fix(ui): sort owner and organization filters case-insensitively (#487)
Owner and organization filter options are sorted with Intl.Collator, case and accent insensitive, instead of a plain sort that put "Zebra" before "apple".
2026-10-30 00:25:43 +05:30
23 changed files with 534 additions and 20 deletions

No files matched your search

+3
View File
@@ -9,12 +9,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **Refresh sizes** on the Storage page measures every mirror now instead of waiting for its next sync, so the page fills in on a fresh install. It only reads the destination: one repository listing per destination owner, the single repository GET for a mirror the listing does not have, and the release listing of repositories that mirror releases. It runs in the background with its progress on the button (`POST /api/storage/refresh`)
- **Remove old release assets** brings the attachments of older releases under a release asset limit. Turning the limit on, or lowering it, used to leave every file uploaded earlier in place, so the limit only held for the future. With the switch on, a kept release past the newest N releases that get assets loses its attachments on the next sync; the release, its notes and its tag stay, and a release whose attachments cannot be listed is left alone. It is a global setting (`PRUNE_OLD_RELEASE_ASSETS`, off by default) because it deletes files from the destination, while the limit it works with stays configurable per organization and repository
### Changed
- Pages keep what they last showed in this browser tab. Going back to the Dashboard, Repositories, Organizations, Storage or Activity Log shows that data at once and refreshes it in the background, instead of starting from a skeleton and waiting for the whole list again. Nothing is written to browser storage; a reload or a new tab starts fresh
### Fixed
- The Activity Log came back as an error instead of the feed once a sync had finished with a warning. A run whose release filter matched nothing on the destination leaves a job whose status is `warning`, and the feed validated the status with the repository status list, which has no such value
- Long lists no longer put every row in the page. The Repositories page built a card for every repository even on desktop, where the cards are hidden, and the Storage table rendered every row; with 3,000 repositories the Repositories page held over 120,000 elements. Only the layout on screen is rendered now, and the mobile cards and Storage rows are virtualized like the desktop table, so a few hundred elements are on the page whatever the count
- The owner and organization filters on the Repositories page and the name filter on the Activity Log list names in alphabetical order regardless of case (#487). A plain sort put every capitalized name before the lowercase ones and accented names last
- Scrolling the Repositories table on desktop dropped frames on slower machines. Each scroll step rendered the filters and every visible row again, and each row coming into view forced a style recalculation for its name. Only the rows coming into view render now; with the CPU slowed 4x, a scroll frame went from about 31 ms to about 13 ms
- Destination release listings stopped after 30 releases. Gitea and Forgejo ignore `per_page` and answer 30 a page, and the listing stopped on any page shorter than the 100 it asked for. The release limit only trimmed within the newest 30 on each sync, the Storage page counted at most 30 releases per repository, and the release filter preview showed only the newest 30 tags. Listings now page with `limit` until the total the destination reports
- The sync button on an archived repository says **Sync** like every other row instead of **Manual Sync**, so the wider label no longer pushes the row's status badge out of line. Hovering it explains that automatic syncs are off for archived mirrors
+1 -1
View File
@@ -242,7 +242,7 @@ Transfer complete repository metadata from GitHub to Gitea/Forgejo:
Releases have two limits and two filters. The release limit keeps only the newest N releases in Gitea/Forgejo. The release asset limit uploads assets only for the newest N of those, so a repository with years of releases can keep every release note and tag while only the recent binaries take up space. Leave the asset limit empty to upload assets for every mirrored release, or set it to 0 for release notes only. The **Pre-releases** switch skips everything the source marks as a pre-release, and **Custom tag rules** narrows releases by tag name (`v1.*`, `*-rc.*`, or a regular expression in slashes) for projects that ship candidates without marking them.
All of it can be overridden per organization and per repository in the **Mirror options** dialog, and lowering the asset limit never deletes assets that were already uploaded.
All of it can be overridden per organization and per repository in the **Mirror options** dialog, and lowering the asset limit never deletes assets that were already uploaded. Enable **Remove old release assets** if you want each sync to delete the attachments of releases past that count instead.
Enable in Settings → Mirror Options → Mirror metadata
+2 -1
View File
@@ -217,7 +217,8 @@ Control what content gets mirrored from GitHub to Gitea.
| `RELEASE_TYPE` | Which releases are candidates: `all` mirrors everything, `stable` skips what the source marks as a pre-release. With a filter active the release limit counts the releases that match, and releases of the other kind are removed from Gitea on the next sync | `all` | `all`, `stable` |
| `RELEASE_INCLUDE_TAGS` | Comma separated tag patterns; only releases whose tag matches are mirrored. A pattern is a glob over the whole tag (`v1.*`, `*-rc.*`, `?` for one character) or a regular expression in slashes (`/^v\d+\.\d+\.\d+$/`), matched case-insensitively. Commas inside the slashes belong to the expression (`/^v\d{1,3}\.\d+$/`) | empty | any list of globs or `/regex/` |
| `RELEASE_EXCLUDE_TAGS` | Comma separated tag patterns in the same syntax; releases whose tag matches are skipped, and an excluded tag wins over `RELEASE_INCLUDE_TAGS` | empty | any list of globs or `/regex/` |
| `RELEASE_ASSET_LIMIT` | Upload release assets only for the newest N mirrored releases. Older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means release notes only. Lowering it never deletes assets already uploaded | unset (all) | Number (0 or more) |
| `RELEASE_ASSET_LIMIT` | Upload release assets only for the newest N mirrored releases. Older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means release notes only. Lowering it never deletes assets already uploaded unless `PRUNE_OLD_RELEASE_ASSETS` is on | unset (all) | Number (0 or more) |
| `PRUNE_OLD_RELEASE_ASSETS` | Opt-in cleanup for what the asset limit left behind. Off, a sync only ever uploads and attachments already on Gitea stay. On, every sync also deletes the attachments of each mirrored release ranked past `RELEASE_ASSET_LIMIT`; the releases themselves and their notes stay | `false` | `true`, `false` |
| `MIRROR_WIKI` | Mirror wiki content | `false` | `true`, `false` |
| `MIRROR_METADATA` | Master toggle for metadata mirroring | `false` | `true`, `false` |
| `MIRROR_ISSUES` | Mirror issues (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
+2 -2
View File
@@ -2,8 +2,8 @@ apiVersion: v2
name: gitea-mirror
description: Kubernetes helm chart for gitea-mirror
type: application
version: 3.48.0
appVersion: "3.48.0"
version: 3.49.0
appVersion: "3.49.0"
icon: https://github.com/RayLabsHQ/gitea-mirror/blob/main/.github/assets/logo.png
keywords:
- git
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "gitea-mirror",
"type": "module",
"version": "3.48.0",
"version": "3.49.0",
"engines": {
"bun": ">=1.2.9"
},
@@ -15,6 +15,7 @@ import {
} from "@/components/ui/popover";
import { selectTriggerClassName } from "@/components/ui/select";
import { cn } from "@/lib/utils";
import { compareNames } from "@/lib/utils/compare-names";
type ActivityNameComboboxProps = {
activities: any[];
@@ -39,7 +40,7 @@ export function ActivityNameCombobox({
if (a.repositoryName) set.add(a.repositoryName);
if (a.organizationName) set.add(a.organizationName);
});
return Array.from(set).sort();
return Array.from(set).sort(compareNames);
}, [activities]);
const [open, setOpen] = React.useState(false);
@@ -39,6 +39,7 @@ import {
UserX,
X,
FlaskConical,
Trash2,
} from "lucide-react";
import type { GitHubConfig, MirrorOptions, AdvancedOptions, DuplicateNameStrategy, DestinationProvider } from "@/types/config";
import { DESTINATION_PROVIDER_LABELS, isPushDestinationKind } from "@/lib/destination-kinds";
@@ -1055,6 +1056,24 @@ export function GitHubMirrorSettings({
/>
)}
{/* Deleting what a lowered limit already uploaded is destructive, so it
stays off until the user asks for it. */}
{mirrorOptions.mirrorReleases && (
<SwitchRow
icon={Trash2}
className="pl-6"
label="Remove old release assets"
description={
isPushTarget
? pushTargetReason
: "Off by default: attachments stay on the destination even when a release falls outside \u201cAssets for latest\u201d. Turning this on deletes the attachments of every mirrored release past that count"
}
disabled={!isReleaseCapableSource}
checked={mirrorOptions.pruneOldReleaseAssets === true}
onCheckedChange={(checked) => handleMirrorChange('pruneOldReleaseAssets', checked)}
/>
)}
<SwitchRow
icon={HardDrive}
label="Git LFS objects"
+3 -2
View File
@@ -53,6 +53,7 @@ import { useConfigStatus } from "@/hooks/useConfigStatus";
import { useNavigation } from "@/components/layout/MainLayout";
import { withBase } from "@/lib/base-path";
import { getRepositorySource, normalizeSourceUrl } from "@/lib/source-providers/kinds";
import { compareNames } from "@/lib/utils/compare-names";
type RepositoriesPageCache = {
repositories: Repository[];
@@ -917,14 +918,14 @@ export default function Repository() {
new Set(
repositories.map((repo) => repo.owner).filter((v): v is string => !!v)
)
).sort();
).sort(compareNames);
const orgOptions = Array.from(
new Set(
repositories
.map((repo) => repo.organization)
.filter((v): v is string => !!v)
)
).sort();
).sort(compareNames);
const handleConfirmDuplicateRepository = async () => {
if (!duplicateRepoCandidate) {
+7
View File
@@ -153,6 +153,13 @@ export const giteaConfigSchema = z.object({
// Upload assets only for the newest N mirrored releases. null/absent means
// every mirrored release gets its assets; 0 means release notes only.
releaseAssetLimit: z.number().nullable().optional(),
// Opt-in cleanup for what that limit left behind. Off by default, because
// the limit alone only gates uploads: attachments a release already has are
// never removed. Turning this on deletes the attachments of every mirrored
// release ranked past `releaseAssetLimit`. Global only - there is no
// per-repository tier for it, the ranks still come from that repository's
// own effective asset limit.
pruneOldReleaseAssets: z.boolean().default(false),
// Which releases are candidates: every one, or only the ones not flagged as a
// pre-release. The limit is applied after filtering.
releaseType: releaseTypeEnum.default("all"),
+11
View File
@@ -66,6 +66,7 @@ interface EnvConfig {
mirrorMetadata?: boolean;
releaseLimit?: number;
releaseAssetLimit?: number;
pruneOldReleaseAssets?: boolean;
releaseType?: string;
releaseIncludeTags?: string[];
releaseExcludeTags?: string[];
@@ -197,6 +198,10 @@ function parseEnvConfig(): EnvConfig {
mirrorMetadata: process.env.MIRROR_METADATA === 'true',
releaseLimit: process.env.RELEASE_LIMIT ? parseInt(process.env.RELEASE_LIMIT, 10) : undefined,
releaseAssetLimit: process.env.RELEASE_ASSET_LIMIT ? parseInt(process.env.RELEASE_ASSET_LIMIT, 10) : undefined,
pruneOldReleaseAssets:
process.env.PRUNE_OLD_RELEASE_ASSETS === undefined
? undefined
: process.env.PRUNE_OLD_RELEASE_ASSETS === 'true',
releaseType: process.env.RELEASE_TYPE,
releaseIncludeTags,
releaseExcludeTags,
@@ -397,6 +402,12 @@ export async function initializeConfigFromEnv(): Promise<void> {
normalizeReleaseAssetLimit(envConfig.mirror.releaseAssetLimit) ??
existingConfig?.[0]?.giteaConfig?.releaseAssetLimit ??
null,
// Unset env var keeps what the UI stored; anything but "true" is false,
// matching how the other boolean mirror switches here are read.
pruneOldReleaseAssets:
envConfig.mirror.pruneOldReleaseAssets ??
existingConfig?.[0]?.giteaConfig?.pruneOldReleaseAssets ??
false,
// all | stable. An unusable value (a typo in the env var)
// falls through to the stored value rather than storing a filter.
releaseType:
+140 -1
View File
@@ -157,6 +157,12 @@ interface DestinationOptions {
destinationPageCap?: number;
/** Destination release ids whose DELETE fails. */
failDeleteIds?: number[];
/** Attachments a destination release has, keyed by release id. */
destinationAssets?: Record<string, Array<{ id: number; name: string; size: number }>>;
/** Attachment ids whose DELETE fails. */
failAssetDeleteIds?: number[];
/** Release ids whose attachment listing answers with an error. */
failAssetListingFor?: number[];
}
function fakeWorld(options: DestinationOptions) {
@@ -210,7 +216,19 @@ function fakeWorld(options: DestinationOptions) {
// Existing attachments of a release.
if (method === "GET" && /\/releases\/\d+\/assets$/.test(url)) {
return json([]);
const releaseId = Number(url.match(/\/releases\/(\d+)\/assets$/)?.[1]);
if (options.failAssetListingFor?.includes(releaseId)) {
return new Response("boom", { status: 500 });
}
return json(options.destinationAssets?.[String(releaseId)] ?? []);
}
// Pruning the attachments of a release that is past the asset limit.
if (method === "DELETE" && /\/releases\/\d+\/assets\/\d+$/.test(url)) {
const assetId = Number(url.slice(url.lastIndexOf("/") + 1));
return options.failAssetDeleteIds?.includes(assetId)
? new Response("boom", { status: 500 })
: new Response(null, { status: 204 });
}
// Uploading an attachment.
@@ -267,6 +285,10 @@ function fakeWorld(options: DestinationOptions) {
requests.filter(
(r) => r.url.startsWith(`${SOURCE_URL}/attachments/`) || r.url.startsWith("https://github.com/")
),
assetListings: () =>
requests.filter((r) => r.method === "GET" && /\/releases\/\d+\/assets$/.test(r.url)),
assetDeletes: () =>
requests.filter((r) => r.method === "DELETE" && /\/releases\/\d+\/assets\/\d+$/.test(r.url)),
};
}
@@ -959,3 +981,120 @@ describe.skipIf(!isChild)("mirrorGitHubReleasesToGitea keeps its own header and
expect(download.authorization).toBe("token ghes-token");
});
});
describe.skipIf(!isChild)("pruning release assets past the asset limit", () => {
function configWithPruning(prune: boolean) {
return {
...config,
giteaConfig: {
...(config as any).giteaConfig,
pruneOldReleaseAssets: prune,
},
} as unknown as Partial<Config>;
}
const destinationReleases = [
{
id: 1,
tag_name: "v3.0.0",
created_at: "2026-03-01T00:00:00Z",
prerelease: false,
assets: [{ id: 11, name: "new.bin", size: 7 }],
},
{
id: 2,
tag_name: "v2.0.0",
created_at: "2026-02-01T00:00:00Z",
prerelease: false,
assets: [{ id: 21, name: "old.bin", size: 5 }],
},
{
id: 3,
tag_name: "v1.0.0",
created_at: "2026-01-01T00:00:00Z",
prerelease: false,
assets: [{ id: 31, name: "older.bin", size: 3 }],
},
];
/** Three releases on both sides; an asset limit of 1 covers only v3.0.0. */
function run(prune: boolean, extra: Partial<DestinationOptions> = {}) {
const world = fakeWorld({
sourceReleases: [
giteaRelease("v3.0.0"),
giteaRelease("v2.0.0"),
giteaRelease("v1.0.0"),
],
destinationReleases,
destinationAssets: {
"1": [{ id: 11, name: "new.bin", size: 7 }],
"2": [{ id: 21, name: "old.bin", size: 5 }],
"3": [{ id: 31, name: "older.bin", size: 3 }],
},
...extra,
});
return mirrorGiteaSourceReleasesToGitea({
config: configWithPruning(prune),
repository,
connection: giteaConnection("source-token"),
giteaOwner: DEST_OWNER,
giteaRepoName: DEST_REPO,
releaseLimit: 3,
releaseAssetLimit: 1,
}).then(() => world);
}
it("keeps attachments a looser limit uploaded before", async () => {
const world = await run(false);
expect(world.assetListings()).toHaveLength(0);
expect(world.assetDeletes()).toHaveLength(0);
expect(storageRecords.at(-1)?.summary).toEqual({ count: 3, bytes: 15 });
});
it("deletes the attachments of releases past the asset limit when enabled", async () => {
const world = await run(true);
expect(world.assetDeletes().map((r) => r.url)).toEqual([
`${DEST_API}/releases/2/assets/21`,
`${DEST_API}/releases/3/assets/31`,
]);
// The newest release stays inside the limit and keeps its file.
expect(world.assetListings().map((r) => r.url)).toEqual([
`${DEST_API}/releases/2/assets`,
`${DEST_API}/releases/3/assets`,
]);
// The storage summary follows the deletions instead of reporting the old total.
expect(storageRecords.at(-1)?.summary).toEqual({ count: 1, bytes: 7 });
});
it("skips releases the listing shows without attachments", async () => {
// v1.0.0 was pruned on an earlier sync: it is not listed or deleted again.
const world = await run(true, {
destinationReleases: destinationReleases.map((release) =>
release.id === 3 ? { ...release, assets: [] } : release
),
destinationAssets: {
"1": [{ id: 11, name: "new.bin", size: 7 }],
"2": [{ id: 21, name: "old.bin", size: 5 }],
"3": [],
},
});
expect(world.assetListings().map((r) => r.url)).toEqual([
`${DEST_API}/releases/2/assets`,
]);
expect(world.assetDeletes().map((r) => r.url)).toEqual([
`${DEST_API}/releases/2/assets/21`,
]);
expect(storageRecords.at(-1)?.summary).toEqual({ count: 1, bytes: 7 });
});
it("deletes nothing when a release's attachments cannot be listed", async () => {
const world = await run(true, { failAssetListingFor: [2, 3] });
expect(world.assetDeletes()).toHaveLength(0);
expect(storageRecords.at(-1)?.summary).toEqual({ count: 3, bytes: 15 });
});
});
+151 -4
View File
@@ -3807,6 +3807,90 @@ export async function reconcileReleaseAssets({
return { uploaded, failed, skipped };
}
/**
* Delete every attachment one release currently has on the destination.
*
* Used by the opt-in pruning in the retention pass, which removes what the
* asset limit left behind: the uploads a release got before the limit was set
* or lowered. Like `reconcileReleaseAssets` this reads before it writes, and a
* listing that cannot be read deletes nothing, so a transient API error never
* wipes a release that still had its assets.
*
* Returns what was removed so the caller can log it and keep the storage
* totals honest. A release whose deletes all succeeded reports `failed: 0`.
*/
export async function pruneReleaseAttachments({
baseUrl,
token,
repoOwner,
repoName,
giteaReleaseId,
tagName,
fetchImpl,
}: {
baseUrl: string;
token: string;
repoOwner: string;
repoName: string;
giteaReleaseId: number;
tagName: string;
/** Injected in tests; production uses the global fetch. */
fetchImpl?: typeof fetch;
}): Promise<{ removed: number; failed: number }> {
const doFetch: typeof fetch = fetchImpl ?? fetch;
const giteaAuth = { Authorization: `token ${token}` };
const assetsUrl = `${baseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${giteaReleaseId}/assets`;
let existingAssets: Array<{ id: number; name: string }>;
try {
const listResponse = await doFetch(assetsUrl, { headers: giteaAuth });
if (!listResponse.ok) {
throw new Error(`HTTP ${listResponse.status} ${listResponse.statusText}`);
}
const body = await listResponse.json();
if (!Array.isArray(body)) {
throw new Error("the assets endpoint did not return a list");
}
existingAssets = body;
} catch (listError) {
console.error(
`[Releases] Could not list the attachments of ${tagName} to prune them: ${
listError instanceof Error ? listError.message : String(listError)
}. Deleted nothing; the next sync retries.`
);
return { removed: 0, failed: 0 };
}
let removed = 0;
let failed = 0;
for (const asset of existingAssets) {
try {
const deleteResponse = await doFetch(`${assetsUrl}/${asset.id}`, {
method: "DELETE",
headers: giteaAuth,
});
// 404 means it is already gone, which is the state we wanted.
if (!deleteResponse.ok && deleteResponse.status !== 404) {
throw new Error(`HTTP ${deleteResponse.status} ${deleteResponse.statusText}`);
}
removed++;
console.log(
`[Releases] Pruned attachment ${asset.name} (id ${asset.id}) from ${tagName} (past the release asset limit)`
);
} catch (deleteError) {
failed++;
console.error(
`[Releases] Failed to prune attachment ${asset.name} (id ${asset.id}) from ${tagName}: ${
deleteError instanceof Error ? deleteError.message : String(deleteError)
}`
);
}
}
return { removed, failed };
}
/**
* Where one release mirror pass gets its releases and its download credentials.
*
@@ -4093,13 +4177,19 @@ async function mirrorSourceReleasesToGiteaLocked({
// Which of those releases also get their assets (#311). Older releases
// past this point are still created with notes and tag, just without the
// uploads. Assets already in Gitea are never removed by this limit; it only
// decides what a sync uploads.
// uploads. The limit alone only decides what a sync uploads - attachments
// that already reached Gitea stay put, unless the opt-in pruning below is
// enabled and takes them away again.
const releaseAssetLimit =
releaseAssetLimitOverride !== undefined
? (normalizeReleaseAssetLimit(releaseAssetLimitOverride) ?? null)
: (await resolveOptions()).releaseAssetLimit;
// Opt-in retroactive pruning (see `pruneOldReleaseAssets` on the config).
// Global only; it prunes by the ranks this repository's own asset limit
// produces, so a per-repo asset limit of 3 prunes that repo's 4th release.
const pruneOldReleaseAssets = (await resolveOptions()).pruneOldReleaseAssets;
// Which releases are candidates at all: every one, or only the ones the source
// does not flag as a pre-release.
const releaseType =
@@ -4280,7 +4370,11 @@ async function mirrorSourceReleasesToGiteaLocked({
totalAssetsFailed += assetResult.failed;
} else if (release.assets && release.assets.length > 0) {
console.log(
`[Releases] Skipping ${release.assets.length} asset(s) for ${release.tag_name}: outside the newest ${releaseAssetLimit} release(s) that get assets (anything already uploaded is kept)`
`[Releases] Skipping ${release.assets.length} asset(s) for ${release.tag_name}: outside the newest ${releaseAssetLimit} release(s) that get assets${
pruneOldReleaseAssets
? " (their existing attachments are pruned by the retention pass)"
: " (anything already uploaded is kept)"
}`
);
releasesPastAssetLimit++;
}
@@ -4514,6 +4608,55 @@ async function mirrorSourceReleasesToGiteaLocked({
}
}
// Opt-in cleanup of what the asset limit left behind. The limit gates
// uploads, so releases that got their attachments before the limit was set
// or lowered still carry them. Ranked by the same newest-first order the
// uploads were gated on, so a destination release the source does not list
// keeps its attachments - retention removes the release itself if it is
// past the release limit.
const prunedReleaseIds = new Set<number>();
if (pruneOldReleaseAssets && releaseAssetLimit !== null) {
const rankByTag = new Map(
limitedReleases.map((rel, index) => [rel.tag_name, index])
);
const prunable = retainedReleases.filter((rel) => {
if (deletedReleaseIds.has(rel.id)) return false;
// The listing carries each release's attachments, so a release that
// has none (one already pruned on an earlier sync) costs no request.
if (Array.isArray(rel.assets) && rel.assets.length === 0) return false;
const rank = rankByTag.get(rel.tag_name);
return rank !== undefined && !releaseGetsAssets(rank, releaseAssetLimit);
});
if (prunable.length > 0) {
console.log(
`[Releases] Pruning release assets: ${prunable.length} kept release(s) sit past the newest ${releaseAssetLimit} release(s) that get assets`
);
let prunedAssets = 0;
for (const rel of prunable) {
const pruneResult = await pruneReleaseAttachments({
baseUrl: giteaBaseUrl,
token: giteaToken,
repoOwner,
repoName,
giteaReleaseId: rel.id,
tagName: rel.tag_name,
});
prunedAssets += pruneResult.removed;
// Only a release whose attachments actually went is worth zeroing out
// in the storage summary below. A failed listing removed nothing, and
// a partial failure left some behind; both keep the old totals, which
// are closer to the truth than assuming they are gone.
if (pruneResult.removed > 0 && pruneResult.failed === 0) {
prunedReleaseIds.add(rel.id);
}
}
console.log(
`[Releases] Removed ${prunedAssets} attachment(s) from releases past the asset limit for ${repository.fullName}`
);
}
}
// The listing already carries every asset's size, so the storage page
// gets its release numbers without another request (#479). Releases whose
// delete failed are still on the destination and still count.
@@ -4521,7 +4664,11 @@ async function mirrorSourceReleasesToGiteaLocked({
await recordReleaseAssetStorage(
repository.id,
summarizeReleaseAssets(
allGiteaReleases.filter((rel) => !deletedReleaseIds.has(rel.id))
allGiteaReleases
.filter((rel) => !deletedReleaseIds.has(rel.id))
.map((rel) =>
prunedReleaseIds.has(rel.id) ? { ...rel, assets: [] } : rel
)
)
);
}
+28
View File
@@ -0,0 +1,28 @@
import { describe, expect, test } from "bun:test";
import { compareNames } from "./compare-names";
describe("compareNames", () => {
test("ignores case", () => {
expect(["Zebra", "apple", "Mango"].sort(compareNames)).toEqual([
"apple",
"Mango",
"Zebra",
]);
});
test("sorts accented names with their base letter", () => {
expect(["Zeta", "Ärger", "Bravo"].sort(compareNames)).toEqual([
"Ärger",
"Bravo",
"Zeta",
]);
});
test("compares numbers by value", () => {
expect(["repo-10", "repo-2", "repo-1"].sort(compareNames)).toEqual([
"repo-1",
"repo-2",
"repo-10",
]);
});
});
+13
View File
@@ -0,0 +1,13 @@
/**
* Order for owner, organization and repository names in filter lists: case
* and accent insensitive, with numbers compared by value. A plain `.sort()`
* compares code units, so "Zebra" lands before "apple" and umlauts sort last.
*/
const nameCollator = new Intl.Collator(undefined, {
sensitivity: "base",
numeric: true,
});
export function compareNames(a: string, b: string): number {
return nameCollator.compare(a, b);
}
+4
View File
@@ -171,6 +171,9 @@ export function mapUiToDbConfig(
// null is a real value here ("assets for every mirrored release"), so
// it is stored as such rather than dropped.
releaseAssetLimit: normalizeReleaseAssetLimit(mirrorOptions.releaseAssetLimit) ?? null,
// Opt-in pruning: only ever a plain boolean, so an absent switch stores the
// safe default (never delete what the destination already has).
pruneOldReleaseAssets: mirrorOptions.pruneOldReleaseAssets === true,
// An unusable value falls back to "all" rather than storing a filter that
// would hide releases from the user.
releaseType: normalizeReleaseType(mirrorOptions.releaseType) ?? DEFAULT_RELEASE_TYPE,
@@ -251,6 +254,7 @@ export function mapDbToUiConfig(dbConfig: any): {
mirrorReleases: dbConfig.giteaConfig?.mirrorReleases || false,
releaseLimit: dbConfig.giteaConfig?.releaseLimit || 10,
releaseAssetLimit: normalizeReleaseAssetLimit(dbConfig.giteaConfig?.releaseAssetLimit) ?? null,
pruneOldReleaseAssets: dbConfig.giteaConfig?.pruneOldReleaseAssets === true,
releaseType: normalizeReleaseType(dbConfig.giteaConfig?.releaseType) ?? DEFAULT_RELEASE_TYPE,
releaseIncludeTags: normalizeReleaseTagPatterns(dbConfig.giteaConfig?.releaseIncludeTags) ?? [],
releaseExcludeTags: normalizeReleaseTagPatterns(dbConfig.giteaConfig?.releaseExcludeTags) ?? [],
+12 -2
View File
@@ -112,6 +112,10 @@ export const MAX_RELEASE_TAG_PATTERN_LENGTH = 200;
export type ResolvedMirrorOptions = Record<MirrorOverrideKey, boolean> & {
releaseLimit: number;
releaseAssetLimit: number | null;
// Global only: a cleanup policy for the destination rather than something a
// single repository argues about. The ranks it prunes by still come from that
// repository's own effective `releaseAssetLimit`.
pruneOldReleaseAssets: boolean;
releaseType: ReleaseType;
releaseIncludeTags: string[];
releaseExcludeTags: string[];
@@ -424,8 +428,10 @@ const PATTERN_NORMALIZERS: Record<
/**
* Whether the release at `index` in the newest-first list gets its assets
* uploaded. `null` means no limit. A release past the limit still gets its
* notes and tag, and assets that already reached Gitea are never removed
* because of this rule; it only decides whether a sync uploads.
* notes and tag, and assets that already reached Gitea are not removed by this
* rule on its own; it only decides whether a sync uploads. The opt-in
* `pruneOldReleaseAssets` config switch is what applies the same rank rule in
* reverse and deletes what an earlier, looser limit had uploaded.
*/
export function releaseGetsAssets(
index: number,
@@ -895,6 +901,10 @@ export function resolveMirrorOptions({
normalizeReleaseTagPatterns(globalConfig?.releaseExcludeTags) ??
[];
// No per-repo or per-org tier for the pruning switch, and no stored value
// means the safe default: never delete what the destination already holds.
resolved.pruneOldReleaseAssets = globalConfig?.pruneOldReleaseAssets === true;
// Starred repos with starredCodeOnly mirror code and nothing else. This
// clamp intentionally outranks explicit per-repo overrides: the setting
// exists to stop starred repos from dragging in metadata wholesale.
+118
View File
@@ -0,0 +1,118 @@
/**
* GET /api/activities: the feed must accept every status a mirror job can hold.
*
* "warning" is an activity-only status (see mirrorJobSchema) written when all
* releases were filtered out of a repository. repoStatusEnum does not contain
* it, and parsing with it turned the whole feed into a 500.
*
* mock.module is process-wide in bun, so this file re-runs itself in an
* isolated child (same harness as ../dashboard/index.test.ts).
*/
import { expect, mock, test } from "bun:test";
import { mirrorJobs } from "@/lib/db/schema";
const CHILD_FLAG = "GM_ACTIVITIES_ROUTE_ISOLATED";
const isChild = !!process.env[CHILD_FLAG];
if (!isChild) {
test("activities route - isolated child suite", () => {
const res = Bun.spawnSync({
cmd: [process.execPath, "test", import.meta.path],
env: { ...process.env, [CHILD_FLAG]: "1" },
stdout: "pipe",
stderr: "pipe",
});
if (res.exitCode !== 0) {
console.error(res.stdout.toString());
console.error(res.stderr.toString());
}
expect(res.exitCode).toBe(0);
}, 120_000);
}
if (isChild) {
// Shaped like what the sync writes when the release filters empty a
// destination (gitea.ts), typed as table rows so they track the schema.
const rows: Array<typeof mirrorJobs.$inferSelect> = [
{
id: "job-warning",
userId: "user-1",
repositoryId: "repo-1",
repositoryName: "ollama",
organizationId: null,
organizationName: null,
details:
"12 releases deleted by the release filters; the destination now has none. Check the include/exclude tag patterns and the pre-release setting.",
status: "warning",
message: "All releases removed from mirrors/ollama",
timestamp: new Date("2026-10-29T12:00:05.000Z"),
jobType: "mirror",
batchId: null,
totalItems: null,
completedItems: 0,
itemIds: null,
completedItemIds: [],
inProgress: false,
startedAt: null,
completedAt: null,
lastCheckpoint: null,
},
{
id: "job-mirrored",
userId: "user-1",
repositoryId: "repo-2",
repositoryName: "other",
organizationId: null,
organizationName: null,
details: null,
status: "mirrored",
message: "Successfully mirrored repository: other",
timestamp: new Date("2026-10-29T11:00:02.000Z"),
jobType: "mirror",
batchId: null,
totalItems: null,
completedItems: 0,
itemIds: null,
completedItemIds: [],
inProgress: false,
startedAt: null,
completedAt: null,
lastCheckpoint: null,
},
];
mock.module("@/lib/db", () => ({
db: {
select: () => {
const query: any = {
from: () => query,
where: () => query,
orderBy: () => query,
limit: async () => rows,
then: (resolve: (value: unknown) => void) => resolve(rows),
};
return query;
},
},
mirrorJobs,
}));
mock.module("@/lib/auth-guards", () => ({
requireAuthenticatedUserId: async () => ({ userId: "user-1" }),
}));
const { GET } = await import("./index");
test("activity feed returns warning jobs instead of failing", async () => {
const response = await GET({
request: new Request("http://localhost/api/activities?userId=user-1"),
locals: {},
} as any);
expect(response.status).toBe(200);
const body = await response.json();
expect(body.success).toBe(true);
expect(body.activities.map((entry: any) => entry.status)).toEqual(["warning", "mirrored"]);
expect(body.activities[0].repositoryName).toBe("ollama");
expect(body.activities[0].message).toBe("All releases removed from mirrors/ollama");
});
}
+5 -2
View File
@@ -3,7 +3,7 @@ import { db, mirrorJobs } from "@/lib/db";
import { eq, sql } from "drizzle-orm";
import { createSecureErrorResponse } from "@/lib/utils";
import type { MirrorJob } from "@/lib/db/schema";
import { repoStatusEnum } from "@/types/Repository";
import { activityStatusEnum } from "@/types/Repository";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
export const GET: APIRoute = async ({ request, locals }) => {
@@ -28,7 +28,10 @@ export const GET: APIRoute = async ({ request, locals }) => {
repositoryName: job.repositoryName ?? undefined,
organizationId: job.organizationId ?? undefined,
organizationName: job.organizationName ?? undefined,
status: repoStatusEnum.parse(job.status),
// Job/activity statuses include the activity-only "warning" (see mirrorJobSchema).
// repoStatusEnum only covers repository statuses, so parsing "warning" with
// it threw a ZodError and turned the whole feed into a 500.
status: activityStatusEnum.parse(job.status),
details: job.details ?? undefined,
message: job.message,
timestamp: job.timestamp,
+1
View File
@@ -86,6 +86,7 @@ export interface MirrorOptions {
mirrorReleases: boolean;
releaseLimit?: number; // Limit number of releases to mirror (default: 10)
releaseAssetLimit?: number | null; // Upload assets only for the newest N releases; null or absent means all, 0 means none
pruneOldReleaseAssets?: boolean; // Also delete attachments already on releases past releaseAssetLimit (default: false)
releaseType?: ReleaseType; // "all" | "stable": which releases are mirror candidates before the limit applies
releaseIncludeTags?: string[]; // Glob or /regex/ tag patterns; when set, only matching tags are mirror candidates
releaseExcludeTags?: string[]; // Glob or /regex/ tag patterns; matching tags are skipped, and exclude wins over include
+1 -1
View File
@@ -157,7 +157,7 @@ export const useCases: UseCase[] = [
title: 'Control Release Asset Storage',
summary: 'Keep release notes for every mirrored release but upload binaries only for the newest few, globally or per organization and repository.',
painPoint: 'A project that ships builds for five platforms every fortnight turns a release mirror into gigabytes of installers nobody will run.',
outcome: 'Two independent limits decide how many releases exist on Gitea and how many of those carry assets, and nothing already uploaded is removed.',
outcome: 'Two independent limits decide how many releases exist on Gitea and how many of those carry assets, and an opt-in switch also deletes the attachments of the older ones.',
tags: ['Releases', 'Storage', 'Overrides'],
},
];
+2
View File
@@ -38,6 +38,8 @@ Below the connection cards, the mirror settings decide what travels with the cod
Releases have two numbers under the switch. **Latest** is how many releases are mirrored per repository, newest first, ten by default. **Assets for latest** is how many of those get their assets uploaded: leave it empty to upload assets for every mirrored release, set it to `0` for release notes and tags only, or set it to a number to keep assets for the newest N releases while older ones keep their notes and tag. The limit never deletes assets that were already uploaded.
The **Remove old release assets** switch under them turns that last sentence around. Off, which is the default, a sync only ever uploads. On, every sync also deletes the attachments of each mirrored release ranked past **Assets for latest**, so lowering the limit actually frees the disk. It is global rather than per repository, it never removes a release or its notes, and a release whose attachments cannot be listed is left alone and retried on the next sync.
The **Pre-releases** switch under them decides which releases are candidates at all: left on it mirrors every release, turned off it skips everything the source marks as a pre-release. With a filter active the **Latest** count applies to the releases that match, so ten stable releases are kept even when a project ships betas in between, and releases of the other kind are removed from Gitea on the next sync. A release the destination has no pre-release flag for is left alone. The per organization and per repository **Mirror options** dialog offers the same choice under its **Releases** row - **All**, **Stable only**, or **Inherit**, where **Inherit** means the level above - and the release type is `RELEASE_TYPE` in the environment.
**Custom tag rules...** beside that switch opens a dialog with **Include tags** and **Exclude tags**, two comma separated pattern lists that catch the projects shipping release candidates without marking them as pre-releases. The same lists sit under the **Releases** row in the per organization and per repository dialog, where leaving both lists empty means the level above, and in the environment they are `RELEASE_INCLUDE_TAGS` and `RELEASE_EXCLUDE_TAGS`.
+2 -1
View File
@@ -165,7 +165,8 @@ Only used when `DESTINATION_PROVIDER` is `github` or `gitlab`.
| `RELEASE_TYPE` | `all` | Which releases are candidates: `all` or `stable` (skip what the source marks as a pre-release). With a filter active the limit counts the releases that match, and releases of the other kind are removed from Gitea on the next sync. |
| `RELEASE_INCLUDE_TAGS` | empty | Comma separated tag patterns; only releases whose tag matches are mirrored. A pattern is a glob over the whole tag (`v1.*`, `*-rc.*`) or a regular expression in slashes (`/^v\d+\.\d+\.\d+$/`), matched case-insensitively. |
| `RELEASE_EXCLUDE_TAGS` | empty | Comma separated tag patterns in the same syntax; matching releases are skipped, and an excluded tag wins over `RELEASE_INCLUDE_TAGS`. |
| `RELEASE_ASSET_LIMIT` | unset | Upload release assets only for the newest N mirrored releases; older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means notes only. Never deletes assets already uploaded. |
| `RELEASE_ASSET_LIMIT` | unset | Upload release assets only for the newest N mirrored releases; older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means notes only. Lowering it never deletes assets already uploaded unless `PRUNE_OLD_RELEASE_ASSETS` is on. |
| `PRUNE_OLD_RELEASE_ASSETS` | `false` | Opt-in cleanup for what the asset limit left behind. On, every sync also deletes the attachments of each mirrored release ranked past `RELEASE_ASSET_LIMIT`; the releases and their notes stay. |
| `MIRROR_WIKI` | `false` | Clone wiki content. |
| `MIRROR_ISSUE_CONCURRENCY` | `3` | Issues processed in parallel. |
| `MIRROR_PULL_REQUEST_CONCURRENCY` | `5` | Pull requests processed in parallel. |
@@ -45,10 +45,12 @@ Patterns are comma separated, and each one is either a glob over the whole tag o
Matching ignores case, and a pattern that cannot compile is ignored rather than failing the run. A tag filter widens the search exactly like the type filter does, so **Latest 1** with `v*` still finds the newest real release behind a wall of nightlies. Releases whose tag no longer matches are removed from Gitea on the next sync, and unlike the type filter a tag pattern can clean up releases the destination has no pre-release flag for, because it only needs the tag name.
## Nothing already uploaded is deleted
## Nothing already uploaded is deleted, unless you ask
The asset limit only decides whether a sync uploads. Lowering it later never removes assets that already reached Gitea. If you mirrored everything for a year and then set the limit to 2, the old files stay until you delete them in Gitea yourself. That is the safe direction for a backup tool, and it means the setting is cheap to experiment with.
The **Remove old release assets** switch under the release settings changes that on purpose. With it on, a sync also prunes: the attachments of every mirrored release ranked past **Assets for latest** are deleted from Gitea, so lowering the limit actually shrinks what is on disk. It is a global switch, it stays off until you turn it on, and it only touches the files - the release and its notes stay. A release whose attachment listing cannot be read keeps its files and is retried on the next sync.
## Step-by-step
### 1. Set a global default
@@ -71,6 +73,7 @@ For deployments configured from the environment:
MIRROR_RELEASES=true
RELEASE_LIMIT=25
RELEASE_ASSET_LIMIT=3 # unset for all, 0 for notes only
PRUNE_OLD_RELEASE_ASSETS=true # also delete attachments past that count
RELEASE_TYPE=stable # all or stable
RELEASE_INCLUDE_TAGS=v* # comma separated globs or /regex/
RELEASE_EXCLUDE_TAGS=*-rc.*
@@ -82,6 +85,8 @@ RELEASE_EXCLUDE_TAGS=*-rc.*
Take a project that publishes a release every two weeks with five platform builds of 40 MB each. That is 200 MB per release and about 5 GB per year of assets, for one repository. With the release limit at 25 and no asset limit, the first mirror pulls about 5 GB and every sync afterwards adds the newest release's 200 MB. With **Assets for latest** at 3, the first mirror pulls 600 MB. Each later release still adds 200 MB, because a new release is inside the limit and gets its files, and the files of the release that dropped out of the top three stay on disk until you remove them. The ongoing growth is the same; what you save is the backfill of every historical build.
With **Remove old release assets** on, the picture changes: when a release drops out of the top three, its 200 MB go with it, so the destination hovers around 600 MB instead of growing without bound.
Multiply by the number of repositories that ship binaries and compare with the free space on the Gitea host. Gitea stores release attachments in its own attachment storage, separate from the git objects and from LFS, so check that path when you plan capacity.
## Where releases apply