mirror of
https://github.com/RayLabsHQ/gitea-mirror.git
synced 2026-11-01 23:15:46 +01:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4a3bba890b | ||
|
|
344ca9f9ac | ||
|
|
e0d20d1c1a | ||
|
|
4a4550b94f | ||
|
|
5e95fa7d41 | ||
|
|
f6342de307 | ||
|
|
35b275035d | ||
|
|
ecdba3cad7 | ||
|
|
2485513973 | ||
|
|
43581f3e8b | ||
|
|
7e70e1005d |
No files matched your search
@@ -8,13 +8,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **Stalled sync warning** (#493). Gitea and Forgejo answer a mirror sync request by queueing it, and say OK before it runs, so a stuck mirror queue looked like a working sync: gitea-mirror kept reporting syncs while Gitea fetched nothing, and new releases waited on tags that never arrived. Each sync now compares Gitea's last mirror run with the one it saw before the previous request. When a request has gone unrun for over an hour, the Activity Log gets a warning for that repository that points at Gitea's mirror queue, and a notification goes out through the **Sync errors** switch, at most once every six hours per user. It reads the repository lookup the sync already makes, so it costs no extra requests
|
||||
- **Issue filters** narrow issue mirroring (#491). A state filter creates only open or only closed issues; an issue that is already mirrored keeps following GitHub, so closing it there still closes the copy. Comments can be skipped, which also skips the comment listing for each issue, and images, videos and attached files can be left out of issue bodies and new comments, replaced by `[media omitted]`. All three sit under the metadata options and can be overridden per organization and repository. With media off, mirrored issue bodies lose their media as they are next synced; comments already mirrored stay as they are. Changing the state filter runs a full issue pass on the next sync
|
||||
- **Refresh sizes** on the Storage page measures every mirror now instead of waiting for its next sync, so the page fills in on a fresh install. It only reads the destination: one repository listing per destination owner, the single repository GET for a mirror the listing does not have, and the release listing of repositories that mirror releases. It runs in the background with its progress on the button (`POST /api/storage/refresh`)
|
||||
- **Remove old release assets** brings the attachments of older releases under a release asset limit. Turning the limit on, or lowering it, used to leave every file uploaded earlier in place, so the limit only held for the future. With the switch on, a kept release past the newest N releases that get assets loses its attachments on the next sync; the release, its notes and its tag stay, and a release whose attachments cannot be listed is left alone. It is a global setting (`PRUNE_OLD_RELEASE_ASSETS`, off by default) because it deletes files from the destination, while the limit it works with stays configurable per organization and repository
|
||||
- **Destination filter** on the Repositories page lists repositories by where they mirror to, worked out the way the mirror does it: the starred organization, the repository's own destination, its organization's destination, then the mirror strategy (#492). The Organization column names that same destination now. It used to work it out in the browser and got some cases wrong, such as personal repositories under the mixed strategy, which it showed under the Gitea user instead of the organization they mirror into
|
||||
|
||||
### Changed
|
||||
- The Dashboard's Repositories and Recent Activity lists run down to the bottom of the screen instead of stopping at eight rows, so a tall screen shows more and a short one fewer, up to 50. They follow the window as it is resized. On narrow screens, where the two lists stack, they still show eight
|
||||
- Pages keep what they last showed in this browser tab. Going back to the Dashboard, Repositories, Organizations, Storage or Activity Log shows that data at once and refreshes it in the background, instead of starting from a skeleton and waiting for the whole list again. Nothing is written to browser storage; a reload or a new tab starts fresh
|
||||
|
||||
### Fixed
|
||||
- Organization cards cut off the buttons on their bottom row when the grid's columns came out narrower than that row, as on a 4K screen with five columns. Columns are now at least 30rem wide instead of 27rem, which fits the row, and a longer label such as Forgejo moves the link buttons onto a second line instead of past the card's edge
|
||||
- The Activity Log came back as an error instead of the feed once a sync had finished with a warning. A run whose release filter matched nothing on the destination leaves a job whose status is `warning`, and the feed validated the status with the repository status list, which has no such value
|
||||
- Long lists no longer put every row in the page. The Repositories page built a card for every repository even on desktop, where the cards are hidden, and the Storage table rendered every row; with 3,000 repositories the Repositories page held over 120,000 elements. Only the layout on screen is rendered now, and the mobile cards and Storage rows are virtualized like the desktop table, so a few hundred elements are on the page whatever the count
|
||||
- The owner and organization filters on the Repositories page and the name filter on the Activity Log list names in alphabetical order regardless of case (#487). A plain sort put every capitalized name before the lowercase ones and accented names last
|
||||
- Scrolling the Repositories table on desktop dropped frames on slower machines. Each scroll step rendered the filters and every visible row again, and each row coming into view forced a style recalculation for its name. Only the rows coming into view render now; with the CPU slowed 4x, a scroll frame went from about 31 ms to about 13 ms
|
||||
- Destination release listings stopped after 30 releases. Gitea and Forgejo ignore `per_page` and answer 30 a page, and the listing stopped on any page shorter than the 100 it asked for. The release limit only trimmed within the newest 30 on each sync, the Storage page counted at most 30 releases per repository, and the release filter preview showed only the newest 30 tags. Listings now page with `limit` until the total the destination reports
|
||||
- The sync button on an archived repository says **Sync** like every other row instead of **Manual Sync**, so the wider label no longer pushes the row's status badge out of line. Hovering it explains that automatic syncs are off for archived mirrors
|
||||
|
||||
@@ -242,7 +242,7 @@ Transfer complete repository metadata from GitHub to Gitea/Forgejo:
|
||||
|
||||
Releases have two limits and two filters. The release limit keeps only the newest N releases in Gitea/Forgejo. The release asset limit uploads assets only for the newest N of those, so a repository with years of releases can keep every release note and tag while only the recent binaries take up space. Leave the asset limit empty to upload assets for every mirrored release, or set it to 0 for release notes only. The **Pre-releases** switch skips everything the source marks as a pre-release, and **Custom tag rules** narrows releases by tag name (`v1.*`, `*-rc.*`, or a regular expression in slashes) for projects that ship candidates without marking them.
|
||||
|
||||
All of it can be overridden per organization and per repository in the **Mirror options** dialog, and lowering the asset limit never deletes assets that were already uploaded.
|
||||
All of it can be overridden per organization and per repository in the **Mirror options** dialog, and lowering the asset limit never deletes assets that were already uploaded. Enable **Remove old release assets** if you want each sync to delete the attachments of releases past that count instead.
|
||||
|
||||
Enable in Settings → Mirror Options → Mirror metadata
|
||||
|
||||
|
||||
@@ -217,7 +217,8 @@ Control what content gets mirrored from GitHub to Gitea.
|
||||
| `RELEASE_TYPE` | Which releases are candidates: `all` mirrors everything, `stable` skips what the source marks as a pre-release. With a filter active the release limit counts the releases that match, and releases of the other kind are removed from Gitea on the next sync | `all` | `all`, `stable` |
|
||||
| `RELEASE_INCLUDE_TAGS` | Comma separated tag patterns; only releases whose tag matches are mirrored. A pattern is a glob over the whole tag (`v1.*`, `*-rc.*`, `?` for one character) or a regular expression in slashes (`/^v\d+\.\d+\.\d+$/`), matched case-insensitively. Commas inside the slashes belong to the expression (`/^v\d{1,3}\.\d+$/`) | empty | any list of globs or `/regex/` |
|
||||
| `RELEASE_EXCLUDE_TAGS` | Comma separated tag patterns in the same syntax; releases whose tag matches are skipped, and an excluded tag wins over `RELEASE_INCLUDE_TAGS` | empty | any list of globs or `/regex/` |
|
||||
| `RELEASE_ASSET_LIMIT` | Upload release assets only for the newest N mirrored releases. Older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means release notes only. Lowering it never deletes assets already uploaded | unset (all) | Number (0 or more) |
|
||||
| `RELEASE_ASSET_LIMIT` | Upload release assets only for the newest N mirrored releases. Older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means release notes only. Lowering it never deletes assets already uploaded unless `PRUNE_OLD_RELEASE_ASSETS` is on | unset (all) | Number (0 or more) |
|
||||
| `PRUNE_OLD_RELEASE_ASSETS` | Opt-in cleanup for what the asset limit left behind. Off, a sync only ever uploads and attachments already on Gitea stay. On, every sync also deletes the attachments of each mirrored release ranked past `RELEASE_ASSET_LIMIT`; the releases themselves and their notes stay | `false` | `true`, `false` |
|
||||
| `MIRROR_WIKI` | Mirror wiki content | `false` | `true`, `false` |
|
||||
| `MIRROR_METADATA` | Master toggle for metadata mirroring | `false` | `true`, `false` |
|
||||
| `MIRROR_ISSUES` | Mirror issues (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
|
||||
|
||||
@@ -105,7 +105,7 @@ The secret is encrypted at rest.
|
||||
|
||||
| Event | Default | Description |
|
||||
|-------|---------|-------------|
|
||||
| Sync errors | On | A mirror job failed |
|
||||
| Sync errors | On | A mirror job failed, the release filters removed every release of a destination, or Gitea accepted a sync request and has not run it for over an hour (at most one such alert every six hours) |
|
||||
| Sync success | Off | A mirror job completed successfully |
|
||||
| New repositories discovered | Off | New repositories were discovered and imported; the three switches below select which ones |
|
||||
| In your account | On | Repositories owned by your account |
|
||||
|
||||
@@ -2,8 +2,8 @@ apiVersion: v2
|
||||
name: gitea-mirror
|
||||
description: Kubernetes helm chart for gitea-mirror
|
||||
type: application
|
||||
version: 3.48.0
|
||||
appVersion: "3.48.0"
|
||||
version: 3.51.0
|
||||
appVersion: "3.51.0"
|
||||
icon: https://github.com/RayLabsHQ/gitea-mirror/blob/main/.github/assets/logo.png
|
||||
keywords:
|
||||
- git
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "gitea-mirror",
|
||||
"type": "module",
|
||||
"version": "3.48.0",
|
||||
"version": "3.51.0",
|
||||
"engines": {
|
||||
"bun": ">=1.2.9"
|
||||
},
|
||||
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
} from "@/components/ui/popover";
|
||||
import { selectTriggerClassName } from "@/components/ui/select";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { compareNames } from "@/lib/utils/compare-names";
|
||||
|
||||
type ActivityNameComboboxProps = {
|
||||
activities: any[];
|
||||
@@ -39,7 +40,7 @@ export function ActivityNameCombobox({
|
||||
if (a.repositoryName) set.add(a.repositoryName);
|
||||
if (a.organizationName) set.add(a.organizationName);
|
||||
});
|
||||
return Array.from(set).sort();
|
||||
return Array.from(set).sort(compareNames);
|
||||
}, [activities]);
|
||||
|
||||
const [open, setOpen] = React.useState(false);
|
||||
|
||||
@@ -96,6 +96,9 @@ export function ConfigTabs() {
|
||||
milestones: false,
|
||||
wiki: false,
|
||||
},
|
||||
issueState: "all",
|
||||
issueComments: true,
|
||||
issueMedia: true,
|
||||
},
|
||||
advancedOptions: {
|
||||
skipForks: false,
|
||||
|
||||
@@ -39,6 +39,7 @@ import {
|
||||
UserX,
|
||||
X,
|
||||
FlaskConical,
|
||||
Trash2,
|
||||
} from "lucide-react";
|
||||
import type { GitHubConfig, MirrorOptions, AdvancedOptions, DuplicateNameStrategy, DestinationProvider } from "@/types/config";
|
||||
import { DESTINATION_PROVIDER_LABELS, isPushDestinationKind } from "@/lib/destination-kinds";
|
||||
@@ -479,7 +480,7 @@ export function GitHubMirrorSettings({
|
||||
<ChevronDown className="ml-2 h-3 w-3 opacity-50" />
|
||||
</Button>
|
||||
</PopoverTrigger>
|
||||
<PopoverContent align="end" className="w-64">
|
||||
<PopoverContent align="end" className="w-72">
|
||||
<div className="space-y-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="text-sm font-medium">Metadata to mirror</div>
|
||||
@@ -528,6 +529,63 @@ export function GitHubMirrorSettings({
|
||||
</Label>
|
||||
</div>
|
||||
|
||||
{isGithubSource && mirrorOptions.metadataComponents.issues && (
|
||||
<div className="ml-8 space-y-2 border-l pl-3">
|
||||
<div className="flex items-center justify-between gap-2">
|
||||
<Label htmlFor="issue-state-popup" className="text-xs font-normal">
|
||||
State
|
||||
</Label>
|
||||
<Select
|
||||
value={mirrorOptions.issueState ?? "all"}
|
||||
onValueChange={(value) =>
|
||||
handleMirrorChange("issueState", value)
|
||||
}
|
||||
>
|
||||
<SelectTrigger id="issue-state-popup" className="h-8 w-28 text-xs">
|
||||
<SelectValue />
|
||||
</SelectTrigger>
|
||||
<SelectContent align="end">
|
||||
<SelectItem value="all" className="text-xs">All issues</SelectItem>
|
||||
<SelectItem value="open" className="text-xs">Open only</SelectItem>
|
||||
<SelectItem value="closed" className="text-xs">Closed only</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center space-x-2">
|
||||
<Checkbox
|
||||
id="issue-comments-popup"
|
||||
checked={mirrorOptions.issueComments ?? true}
|
||||
onCheckedChange={(checked) =>
|
||||
handleMirrorChange("issueComments", !!checked)
|
||||
}
|
||||
/>
|
||||
<Label
|
||||
htmlFor="issue-comments-popup"
|
||||
className="text-xs font-normal cursor-pointer"
|
||||
>
|
||||
Copy comments
|
||||
</Label>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center space-x-2">
|
||||
<Checkbox
|
||||
id="issue-media-popup"
|
||||
checked={mirrorOptions.issueMedia ?? true}
|
||||
onCheckedChange={(checked) =>
|
||||
handleMirrorChange("issueMedia", !!checked)
|
||||
}
|
||||
/>
|
||||
<Label
|
||||
htmlFor="issue-media-popup"
|
||||
className="text-xs font-normal cursor-pointer"
|
||||
>
|
||||
Copy images and attachments
|
||||
</Label>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex items-center space-x-3 py-1 px-1 rounded hover:bg-accent">
|
||||
<Checkbox
|
||||
id="metadata-prs-popup"
|
||||
@@ -1055,6 +1113,24 @@ export function GitHubMirrorSettings({
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Deleting what a lowered limit already uploaded is destructive, so it
|
||||
stays off until the user asks for it. */}
|
||||
{mirrorOptions.mirrorReleases && (
|
||||
<SwitchRow
|
||||
icon={Trash2}
|
||||
className="pl-6"
|
||||
label="Remove old release assets"
|
||||
description={
|
||||
isPushTarget
|
||||
? pushTargetReason
|
||||
: "Off by default: attachments stay on the destination even when a release falls outside \u201cAssets for latest\u201d. Turning this on deletes the attachments of every mirrored release past that count"
|
||||
}
|
||||
disabled={!isReleaseCapableSource}
|
||||
checked={mirrorOptions.pruneOldReleaseAssets === true}
|
||||
onCheckedChange={(checked) => handleMirrorChange('pruneOldReleaseAssets', checked)}
|
||||
/>
|
||||
)}
|
||||
|
||||
<SwitchRow
|
||||
icon={HardDrive}
|
||||
label="Git LFS objects"
|
||||
|
||||
@@ -20,11 +20,13 @@ import {
|
||||
import { ChevronRight } from "lucide-react";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { withBase } from "@/lib/base-path";
|
||||
import type { MirrorOverrides, ReleaseType } from "@/lib/db/schema";
|
||||
import type { IssueStateFilter, MirrorOverrides, ReleaseType } from "@/lib/db/schema";
|
||||
import {
|
||||
compileReleaseTagPattern,
|
||||
DEFAULT_ISSUE_STATE,
|
||||
getMirrorOverrideGating,
|
||||
MIRROR_OVERRIDE_LABELS,
|
||||
normalizeIssueState,
|
||||
normalizeReleaseAssetLimit,
|
||||
normalizeReleaseLimit,
|
||||
normalizeReleaseTagPatterns,
|
||||
@@ -48,6 +50,16 @@ type TriState = "inherit" | "on" | "off";
|
||||
*/
|
||||
type ReleaseTypeChoice = "inherit" | ReleaseType;
|
||||
|
||||
/** Issue state is a choice too, so "inherit" persists as absent. */
|
||||
type IssueStateChoice = "inherit" | IssueStateFilter;
|
||||
|
||||
/** How the issue-state row describes the value it inherits. */
|
||||
const ISSUE_STATE_LABELS: Record<IssueStateFilter, string> = {
|
||||
all: "all issues",
|
||||
open: "open only",
|
||||
closed: "closed only",
|
||||
};
|
||||
|
||||
/**
|
||||
* What the single Releases row offers. It covers both stored fields at once:
|
||||
* "off" pins the `mirrorReleases` flag false, the two kinds pin it true
|
||||
@@ -203,6 +215,9 @@ export function MirrorOverridesDialog({
|
||||
const [releaseTypeDraft, setReleaseTypeDraft] = useState<ReleaseTypeChoice>(
|
||||
() => buildReleaseTypeDraft(value)
|
||||
);
|
||||
const [issueStateDraft, setIssueStateDraft] = useState<IssueStateChoice>(
|
||||
() => buildIssueStateDraft(value)
|
||||
);
|
||||
const [includeTagsDraft, setIncludeTagsDraft] = useState(() =>
|
||||
buildReleaseTagDraft(value, "releaseIncludeTags")
|
||||
);
|
||||
@@ -225,6 +240,7 @@ export function MirrorOverridesDialog({
|
||||
setReleaseLimitDraft(buildReleaseLimitDraft(value));
|
||||
setReleaseAssetLimitDraft(buildReleaseAssetLimitDraft(value));
|
||||
setReleaseTypeDraft(buildReleaseTypeDraft(value));
|
||||
setIssueStateDraft(buildIssueStateDraft(value));
|
||||
setIncludeTagsDraft(buildReleaseTagDraft(value, "releaseIncludeTags"));
|
||||
setExcludeTagsDraft(buildReleaseTagDraft(value, "releaseExcludeTags"));
|
||||
}
|
||||
@@ -317,6 +333,9 @@ export function MirrorOverridesDialog({
|
||||
const pinnedReleaseType: ReleaseType | undefined =
|
||||
releaseTypeDraft === "inherit" ? undefined : normalizeReleaseType(releaseTypeDraft);
|
||||
|
||||
const pinnedIssueState: IssueStateFilter | undefined =
|
||||
issueStateDraft === "inherit" ? undefined : normalizeIssueState(issueStateDraft);
|
||||
|
||||
// One control drives two stored fields. A stored "releases on" with no kind
|
||||
// pinned still reads as Inherit and keeps its pin on save, so an override set
|
||||
// before the kind filter existed is never dropped by merely opening the dialog.
|
||||
@@ -356,6 +375,7 @@ export function MirrorOverridesDialog({
|
||||
(draft.mirrorReleases !== "inherit" || releaseTypeDraft !== "inherit"
|
||||
? 1
|
||||
: 0) +
|
||||
(pinnedIssueState !== undefined ? 1 : 0) +
|
||||
(pinnedReleaseLimit !== undefined ? 1 : 0) +
|
||||
(pinnedReleaseAssetLimit !== undefined ? 1 : 0) +
|
||||
(includeTags.patterns !== undefined ? 1 : 0) +
|
||||
@@ -380,6 +400,9 @@ export function MirrorOverridesDialog({
|
||||
next.releaseAssetLimit = pinnedReleaseAssetLimit;
|
||||
}
|
||||
if (pinnedReleaseType !== undefined) next.releaseType = pinnedReleaseType;
|
||||
if (pinnedIssueState !== undefined) {
|
||||
next.mirrorIssueState = pinnedIssueState;
|
||||
}
|
||||
if (includeTags.patterns !== undefined) {
|
||||
next.releaseIncludeTags = includeTags.patterns;
|
||||
}
|
||||
@@ -398,6 +421,7 @@ export function MirrorOverridesDialog({
|
||||
setReleaseLimitDraft("");
|
||||
setReleaseAssetLimitDraft("");
|
||||
setReleaseTypeDraft("inherit");
|
||||
setIssueStateDraft("inherit");
|
||||
setIncludeTagsDraft("");
|
||||
setExcludeTagsDraft("");
|
||||
};
|
||||
@@ -469,6 +493,59 @@ export function MirrorOverridesDialog({
|
||||
tagPatternCount === 0 &&
|
||||
inheritedTagText !== "";
|
||||
|
||||
const issueStateReason = gating.mirrorIssueState;
|
||||
const issueStateDisabled = !!issueStateReason;
|
||||
const effectiveIssueState =
|
||||
pinnedIssueState ?? inheritedFrom?.mirrorIssueState ?? DEFAULT_ISSUE_STATE;
|
||||
const showIssueStateHint =
|
||||
!inheritedLoading && !issueStateDisabled && issueStateDraft === "inherit";
|
||||
|
||||
// Rendered under the Issues row, with the other issue options.
|
||||
const issueStateRow = (
|
||||
<div key="mirrorIssueState" className="space-y-1">
|
||||
<div className="flex items-center justify-between gap-4">
|
||||
<Label
|
||||
htmlFor="override-mirrorIssueState"
|
||||
className={
|
||||
issueStateDisabled ? "flex-1 text-muted-foreground" : "flex-1"
|
||||
}
|
||||
>
|
||||
{MIRROR_OVERRIDE_LABELS.mirrorIssueState}
|
||||
{showIssueStateHint && (
|
||||
<span className="ml-2 text-xs font-normal text-muted-foreground">
|
||||
currently {ISSUE_STATE_LABELS[effectiveIssueState]}
|
||||
</span>
|
||||
)}
|
||||
</Label>
|
||||
<Select
|
||||
value={issueStateDraft}
|
||||
disabled={issueStateDisabled}
|
||||
onValueChange={(next) =>
|
||||
setIssueStateDraft(next as IssueStateChoice)
|
||||
}
|
||||
>
|
||||
<SelectTrigger
|
||||
id="override-mirrorIssueState"
|
||||
className="w-32"
|
||||
>
|
||||
<SelectValue />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value="inherit">Inherit</SelectItem>
|
||||
<SelectItem value="all">All issues</SelectItem>
|
||||
<SelectItem value="open">Open only</SelectItem>
|
||||
<SelectItem value="closed">Closed only</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
{issueStateReason && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{issueStateReason}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={onOpenChange}>
|
||||
<DialogContent className="sm:max-w-md">
|
||||
@@ -668,7 +745,7 @@ export function MirrorOverridesDialog({
|
||||
draft[key] === "inherit" &&
|
||||
inherited !== undefined;
|
||||
|
||||
return (
|
||||
const row = (
|
||||
<div key={key} className="space-y-1">
|
||||
<div className="flex items-center justify-between gap-4">
|
||||
<Label
|
||||
@@ -706,6 +783,7 @@ export function MirrorOverridesDialog({
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
return key === "mirrorIssues" ? [row, issueStateRow] : row;
|
||||
})}
|
||||
|
||||
<p className="text-xs text-muted-foreground">
|
||||
@@ -793,3 +871,9 @@ function buildReleaseTypeDraft(
|
||||
): ReleaseTypeChoice {
|
||||
return normalizeReleaseType(value?.releaseType) ?? "inherit";
|
||||
}
|
||||
|
||||
function buildIssueStateDraft(
|
||||
value: MirrorOverrides | null | undefined
|
||||
): IssueStateChoice {
|
||||
return normalizeIssueState(value?.mirrorIssueState) ?? "inherit";
|
||||
}
|
||||
@@ -476,7 +476,7 @@ export function NotificationSettings({
|
||||
<CardSection>
|
||||
<SwitchRow
|
||||
label="Sync errors"
|
||||
description="Notify when a mirror job fails"
|
||||
description="Notify when a mirror job fails or needs attention"
|
||||
checked={notificationConfig.notifyOnSyncError}
|
||||
onCheckedChange={(checked) =>
|
||||
onNotificationChange({
|
||||
|
||||
@@ -7,7 +7,8 @@ import type { MirrorJob, Organization, Repository } from "@/lib/db/schema";
|
||||
import { readPageCache, writePageCache } from "@/lib/page-cache";
|
||||
import { useAuth } from "@/hooks/useAuth";
|
||||
import { apiRequest, showErrorToast } from "@/lib/utils";
|
||||
import type { DashboardApiResponse } from "@/types/dashboard";
|
||||
import { DASHBOARD_LIST_LIMIT, type DashboardApiResponse } from "@/types/dashboard";
|
||||
import { useFittingRowCount } from "@/hooks/useFittingRowCount";
|
||||
import { useSSE } from "@/hooks/useSEE";
|
||||
import { toast } from "sonner";
|
||||
import { useEffect as useEffectForToasts } from "react";
|
||||
@@ -87,6 +88,15 @@ export function Dashboard() {
|
||||
const [mirroredCount, setMirroredCount] = useState<number>(cached?.mirroredCount ?? 0);
|
||||
const [lastSync, setLastSync] = useState<Date | null>(cached?.lastSync ?? null);
|
||||
|
||||
// The repository and activity lists end at the bottom of the screen: more
|
||||
// rows on a tall screen, fewer on a short one.
|
||||
const [listsElement, setListsElement] = useState<HTMLDivElement | null>(null);
|
||||
const visibleRows = useFittingRowCount(listsElement, {
|
||||
min: 5,
|
||||
max: DASHBOARD_LIST_LIMIT,
|
||||
fallback: 8,
|
||||
});
|
||||
|
||||
// Dashboard auto-refresh timer (30 seconds)
|
||||
const dashboardTimerRef = useRef<NodeJS.Timeout | null>(null);
|
||||
const DASHBOARD_REFRESH_INTERVAL = 30000; // 30 seconds
|
||||
@@ -360,13 +370,13 @@ export function Dashboard() {
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="flex flex-col lg:flex-row gap-6">
|
||||
<div ref={setListsElement} className="flex flex-col lg:flex-row gap-6">
|
||||
<div className="w-full lg:w-1/2">
|
||||
<RepositoryList repositories={repositories.slice(0, 8)} />
|
||||
<RepositoryList repositories={repositories.slice(0, visibleRows)} />
|
||||
</div>
|
||||
|
||||
<div className="w-full lg:w-1/2">
|
||||
<RecentActivity activities={activities.slice(0, 8)} />
|
||||
<RecentActivity activities={activities.slice(0, visibleRows)} />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -69,7 +69,7 @@ export function RecentActivity({ activities }: RecentActivityProps) {
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
<div className="flex flex-col divide-y divide-border">
|
||||
<div data-fit-rows className="flex flex-col divide-y divide-border">
|
||||
{activities.map((activity, index) => {
|
||||
const { Icon, color } = activityIcon(activity.status);
|
||||
return (
|
||||
|
||||
@@ -65,7 +65,7 @@ export function RepositoryList({ repositories }: RepositoryListProps) {
|
||||
</Button>
|
||||
</div>
|
||||
) : (
|
||||
<div className="flex flex-col divide-y divide-border">
|
||||
<div data-fit-rows className="flex flex-col divide-y divide-border">
|
||||
{repositories.map((repo, index) => (
|
||||
<div
|
||||
key={index}
|
||||
|
||||
@@ -292,7 +292,7 @@ export function OrganizationList({
|
||||
))}
|
||||
</div>
|
||||
) : (
|
||||
<div className="grid grid-cols-1 sm:grid-cols-[repeat(auto-fill,minmax(27rem,1fr))] gap-4">
|
||||
<div className="grid grid-cols-1 sm:grid-cols-[repeat(auto-fill,minmax(30rem,1fr))] gap-4">
|
||||
{Array.from({ length: 5 }).map((_, i) => (
|
||||
<Skeleton key={i} className="h-[11.25rem] w-full" />
|
||||
))}
|
||||
@@ -350,7 +350,7 @@ export function OrganizationList({
|
||||
{overridesDialog}
|
||||
</>
|
||||
) : (
|
||||
<div className="grid grid-cols-1 sm:grid-cols-[repeat(auto-fill,minmax(27rem,1fr))] gap-4 pb-20 sm:pb-0">
|
||||
<div className="grid grid-cols-1 sm:grid-cols-[repeat(auto-fill,minmax(30rem,1fr))] gap-4 pb-20 sm:pb-0">
|
||||
{filteredOrganizations.map((org, index) => {
|
||||
const isLoading = loadingOrgIds.has(org.id ?? "");
|
||||
const statusBadge = getStatusBadge(org.status);
|
||||
@@ -748,8 +748,9 @@ export function OrganizationList({
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Desktop Actions */}
|
||||
<div className="hidden sm:flex items-center justify-between mt-4">
|
||||
{/* Desktop Actions. The grid's 30rem column fits this row with
|
||||
"Successfully mirrored"; a longer label wraps the links below. */}
|
||||
<div className="hidden sm:flex flex-wrap items-center justify-between gap-y-2 mt-4">
|
||||
<div className="flex items-center gap-2">
|
||||
{org.status === "ignored" ? (
|
||||
<Button
|
||||
|
||||
@@ -31,6 +31,12 @@ interface InlineDestinationEditorProps {
|
||||
* row would otherwise name the strategy's owner instead (#416).
|
||||
*/
|
||||
organizationDestination?: string | null;
|
||||
/**
|
||||
* Where the server resolved the mirror to go, with the same precedence and
|
||||
* strategy rules the mirror itself uses. The local guess below gets some
|
||||
* strategies wrong, such as personal repositories under "mixed".
|
||||
*/
|
||||
resolvedDestination?: string;
|
||||
onUpdate: (
|
||||
repoId: string,
|
||||
newDestination: string | null,
|
||||
@@ -48,6 +54,7 @@ export function InlineDestinationEditor({
|
||||
repository,
|
||||
giteaConfig,
|
||||
organizationDestination,
|
||||
resolvedDestination,
|
||||
onUpdate,
|
||||
isUpdating = false,
|
||||
className,
|
||||
@@ -105,7 +112,10 @@ export function InlineDestinationEditor({
|
||||
}
|
||||
};
|
||||
|
||||
const defaultDestination = getDefaultDestination();
|
||||
// A repository pinned to its own destination resolves to that pin, so its
|
||||
// default still comes from the local guess.
|
||||
const defaultDestination =
|
||||
(!repository.destinationOrg && resolvedDestination) || getDefaultDestination();
|
||||
const currentDestination = repository.destinationOrg || defaultDestination;
|
||||
// A stored destination is a pin even when it spells out the default, so the
|
||||
// badge follows the stored value rather than comparing against it (#416).
|
||||
|
||||
@@ -43,7 +43,11 @@ import { useSSE } from "@/hooks/useSEE";
|
||||
import { useFilterParams } from "@/hooks/useFilterParams";
|
||||
import { toast } from "sonner";
|
||||
import type { SyncRepoRequest, SyncRepoResponse } from "@/types/sync";
|
||||
import { OwnerCombobox, OrganizationCombobox } from "./RepositoryComboboxes";
|
||||
import {
|
||||
OwnerCombobox,
|
||||
OrganizationCombobox,
|
||||
DestinationCombobox,
|
||||
} from "./RepositoryComboboxes";
|
||||
import type { RetryRepoRequest, RetryRepoResponse } from "@/types/retry";
|
||||
import type { ResetMetadataRequest, ResetMetadataResponse } from "@/types/reset-metadata";
|
||||
import AddRepositoryDialog from "./AddRepositoryDialog";
|
||||
@@ -53,10 +57,12 @@ import { useConfigStatus } from "@/hooks/useConfigStatus";
|
||||
import { useNavigation } from "@/components/layout/MainLayout";
|
||||
import { withBase } from "@/lib/base-path";
|
||||
import { getRepositorySource, normalizeSourceUrl } from "@/lib/source-providers/kinds";
|
||||
import { compareNames } from "@/lib/utils/compare-names";
|
||||
|
||||
type RepositoriesPageCache = {
|
||||
repositories: Repository[];
|
||||
organizationDestinations: Record<string, string>;
|
||||
destinationOwners: Record<string, string>;
|
||||
};
|
||||
|
||||
export default function Repository() {
|
||||
@@ -70,6 +76,10 @@ export default function Repository() {
|
||||
const [organizationDestinations, setOrganizationDestinations] = useState<
|
||||
Record<string, string>
|
||||
>(cached?.organizationDestinations ?? {});
|
||||
// Repository id -> the destination owner/org the mirror will really use.
|
||||
const [destinationOwners, setDestinationOwners] = useState<
|
||||
Record<string, string>
|
||||
>(cached?.destinationOwners ?? {});
|
||||
const [isInitialLoading, setIsInitialLoading] = useState(!cached);
|
||||
// True once there are rows to show, from the cache or a fetch.
|
||||
const hasLoadedRef = useRef(!!cached);
|
||||
@@ -80,6 +90,7 @@ export default function Repository() {
|
||||
searchTerm: "",
|
||||
status: "",
|
||||
organization: "",
|
||||
destinationOrganization: "",
|
||||
owner: "",
|
||||
sort: "imported-desc",
|
||||
});
|
||||
@@ -158,6 +169,7 @@ export default function Repository() {
|
||||
hasLoadedRef.current = true;
|
||||
setRepositories(response.repositories);
|
||||
setOrganizationDestinations(response.organizationDestinations ?? {});
|
||||
setDestinationOwners(response.destinationOwners ?? {});
|
||||
return true;
|
||||
} else {
|
||||
// Only show error toast for manual refreshes to avoid spam during live updates
|
||||
@@ -189,8 +201,12 @@ export default function Repository() {
|
||||
|
||||
useEffect(() => {
|
||||
if (!hasLoadedRef.current) return;
|
||||
writePageCache<RepositoriesPageCache>(user?.id, "repositories", { repositories, organizationDestinations });
|
||||
}, [user?.id, repositories, organizationDestinations]);
|
||||
writePageCache<RepositoriesPageCache>(user?.id, "repositories", {
|
||||
repositories,
|
||||
organizationDestinations,
|
||||
destinationOwners,
|
||||
});
|
||||
}, [user?.id, repositories, organizationDestinations, destinationOwners]);
|
||||
|
||||
// Register with global live refresh system
|
||||
useEffect(() => {
|
||||
@@ -917,14 +933,22 @@ export default function Repository() {
|
||||
new Set(
|
||||
repositories.map((repo) => repo.owner).filter((v): v is string => !!v)
|
||||
)
|
||||
).sort();
|
||||
).sort(compareNames);
|
||||
const orgOptions = Array.from(
|
||||
new Set(
|
||||
repositories
|
||||
.map((repo) => repo.organization)
|
||||
.filter((v): v is string => !!v)
|
||||
)
|
||||
).sort();
|
||||
).sort(compareNames);
|
||||
// The destination owner each row resolves to, not the source organization.
|
||||
const destinationOptions = Array.from(
|
||||
new Set(
|
||||
repositories
|
||||
.map((repo) => (repo.id ? destinationOwners[repo.id] : undefined))
|
||||
.filter((v): v is string => !!v)
|
||||
)
|
||||
).sort(compareNames);
|
||||
|
||||
const handleConfirmDuplicateRepository = async () => {
|
||||
if (!duplicateRepoCandidate) {
|
||||
@@ -1075,8 +1099,19 @@ export default function Repository() {
|
||||
const actionCounts = getActionCounts();
|
||||
|
||||
// Check if any filters are active
|
||||
const hasActiveFilters = !!(filter.owner || filter.organization || filter.status);
|
||||
const activeFilterCount = [filter.owner, filter.organization, filter.status, filter.hasOverrides].filter(Boolean).length;
|
||||
const hasActiveFilters = !!(
|
||||
filter.owner ||
|
||||
filter.organization ||
|
||||
filter.destinationOrganization ||
|
||||
filter.status
|
||||
);
|
||||
const activeFilterCount = [
|
||||
filter.owner,
|
||||
filter.organization,
|
||||
filter.destinationOrganization,
|
||||
filter.status,
|
||||
filter.hasOverrides,
|
||||
].filter(Boolean).length;
|
||||
|
||||
// Clear all filters
|
||||
const clearFilters = () => {
|
||||
@@ -1084,6 +1119,7 @@ export default function Repository() {
|
||||
searchTerm: filter.searchTerm,
|
||||
status: "",
|
||||
organization: "",
|
||||
destinationOrganization: "",
|
||||
owner: "",
|
||||
hasOverrides: "",
|
||||
sort: filter.sort || "imported-desc",
|
||||
@@ -1189,6 +1225,25 @@ export default function Repository() {
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* Destination Filter */}
|
||||
<div className="space-y-2">
|
||||
<label className="text-sm font-medium flex items-center gap-2">
|
||||
<span className="text-muted-foreground">By</span> Destination
|
||||
{filter.destinationOrganization && (
|
||||
<span className="ml-auto text-xs text-muted-foreground">
|
||||
Selected
|
||||
</span>
|
||||
)}
|
||||
</label>
|
||||
<DestinationCombobox
|
||||
options={destinationOptions}
|
||||
value={filter.destinationOrganization || ""}
|
||||
onChange={(destinationOrganization: string) =>
|
||||
setFilter((prev) => ({ ...prev, destinationOrganization }))
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* Status Filter */}
|
||||
<div className="space-y-2">
|
||||
<label className="text-sm font-medium flex items-center gap-2">
|
||||
@@ -1634,6 +1689,8 @@ export default function Repository() {
|
||||
<RepositoryTable
|
||||
repositories={repositories}
|
||||
organizationDestinations={organizationDestinations}
|
||||
destinationOwners={destinationOwners}
|
||||
destinationOptions={destinationOptions}
|
||||
isLoading={isInitialLoading || (!connected && !hasLoadedRef.current)}
|
||||
filter={filter}
|
||||
setFilter={setFilter}
|
||||
|
||||
@@ -22,6 +22,7 @@ type ComboboxProps = {
|
||||
onChange: (value: string) => void;
|
||||
placeholder?: string;
|
||||
label?: string;
|
||||
className?: string;
|
||||
};
|
||||
|
||||
/** These stay comboboxes rather than Selects because the owner and
|
||||
@@ -35,6 +36,7 @@ function FilterCombobox({
|
||||
emptyLabel,
|
||||
searchPlaceholder,
|
||||
emptyMessage,
|
||||
className,
|
||||
}: ComboboxProps & {
|
||||
emptyLabel: string;
|
||||
searchPlaceholder: string;
|
||||
@@ -52,7 +54,7 @@ function FilterCombobox({
|
||||
type="button"
|
||||
role="combobox"
|
||||
aria-expanded={open}
|
||||
className={cn(selectTriggerClassName, "h-10 w-full lg:w-50")}
|
||||
className={cn(selectTriggerClassName, "h-10 w-full lg:w-50", className)}
|
||||
>
|
||||
<span className={cn("truncate", !value && "text-muted-foreground")}>
|
||||
{value || emptyLabel}
|
||||
@@ -129,3 +131,14 @@ export function OrganizationCombobox(props: ComboboxProps) {
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
export function DestinationCombobox(props: ComboboxProps) {
|
||||
return (
|
||||
<FilterCombobox
|
||||
{...props}
|
||||
emptyLabel="All destinations"
|
||||
searchPlaceholder="Search destinations..."
|
||||
emptyMessage="No destinations found."
|
||||
/>
|
||||
);
|
||||
}
|
||||
@@ -32,11 +32,13 @@ import {
|
||||
} from "@/components/ui/tooltip";
|
||||
import { toast } from "sonner";
|
||||
import { InlineDestinationEditor, type DestinationUpdateOptions } from "./InlineDestinationEditor";
|
||||
import { DestinationCombobox } from "./RepositoryComboboxes";
|
||||
import { MarqueeText, MarqueeTrigger } from "@/components/ui/marquee-text";
|
||||
import { MirrorOverridesDialog } from "@/components/config/MirrorOverridesDialog";
|
||||
import {
|
||||
hasMirrorOverrides,
|
||||
mirrorOptionsToFlags,
|
||||
normalizeIssueState,
|
||||
normalizeReleaseAssetLimit,
|
||||
normalizeReleaseLimit,
|
||||
normalizeReleaseTagPatterns,
|
||||
@@ -74,6 +76,10 @@ interface RepositoryTableProps {
|
||||
repositories: Repository[];
|
||||
/** Organization name -> its destination override (#416). */
|
||||
organizationDestinations?: Record<string, string>;
|
||||
/** Repository id -> the destination owner/org the mirror will use. */
|
||||
destinationOwners?: Record<string, string>;
|
||||
/** The values of `destinationOwners`, sorted, for the destination filter. */
|
||||
destinationOptions?: string[];
|
||||
isLoading: boolean;
|
||||
filter: FilterParams;
|
||||
setFilter: (filter: FilterParams) => void;
|
||||
@@ -117,6 +123,8 @@ function getTableSorting(sortOrder: string | undefined): SortingState {
|
||||
export default function RepositoryTable({
|
||||
repositories,
|
||||
organizationDestinations,
|
||||
destinationOwners,
|
||||
destinationOptions = [],
|
||||
isLoading,
|
||||
filter,
|
||||
setFilter,
|
||||
@@ -198,6 +206,8 @@ export default function RepositoryTable({
|
||||
if (orgIncludeTags !== undefined) globals.releaseIncludeTags = orgIncludeTags;
|
||||
const orgExcludeTags = normalizeReleaseTagPatterns(orgOverrides.releaseExcludeTags);
|
||||
if (orgExcludeTags !== undefined) globals.releaseExcludeTags = orgExcludeTags;
|
||||
const orgIssueState = normalizeIssueState(orgOverrides.mirrorIssueState);
|
||||
if (orgIssueState !== undefined) globals.mirrorIssueState = orgIssueState;
|
||||
return globals;
|
||||
}, [mirrorOptions, orgOverrides]);
|
||||
|
||||
@@ -304,6 +314,7 @@ export default function RepositoryTable({
|
||||
filter.status,
|
||||
filter.owner,
|
||||
filter.organization,
|
||||
filter.destinationOrganization,
|
||||
filter.hasOverrides,
|
||||
// val?.toString() is undefined for an unset filter, and undefined !== ""
|
||||
// is true, so this used to report a filter on a completely clean view.
|
||||
@@ -324,12 +335,25 @@ export default function RepositoryTable({
|
||||
next.push({ id: "organization", value: filter.organization });
|
||||
}
|
||||
|
||||
if (filter.destinationOrganization) {
|
||||
next.push({
|
||||
id: "destinationOrganization",
|
||||
value: filter.destinationOrganization,
|
||||
});
|
||||
}
|
||||
|
||||
if (filter.hasOverrides) {
|
||||
next.push({ id: "hasOverrides", value: filter.hasOverrides });
|
||||
}
|
||||
|
||||
return next;
|
||||
}, [filter.status, filter.owner, filter.organization, filter.hasOverrides]);
|
||||
}, [
|
||||
filter.status,
|
||||
filter.owner,
|
||||
filter.organization,
|
||||
filter.destinationOrganization,
|
||||
filter.hasOverrides,
|
||||
]);
|
||||
|
||||
const sorting = useMemo(() => getTableSorting(filter.sort), [filter.sort]);
|
||||
|
||||
@@ -349,6 +373,13 @@ export default function RepositoryTable({
|
||||
accessorFn: (row) => row.organization ?? "",
|
||||
filterFn: "equalsString",
|
||||
},
|
||||
{
|
||||
id: "destinationOrganization",
|
||||
accessorFn: (row) =>
|
||||
row.id ? destinationOwners?.[row.id] ?? "" : "",
|
||||
filterFn: "equalsString",
|
||||
enableGlobalFilter: false,
|
||||
},
|
||||
{
|
||||
id: "status",
|
||||
accessorFn: (row) => row.status,
|
||||
@@ -374,7 +405,7 @@ export default function RepositoryTable({
|
||||
enableColumnFilter: false,
|
||||
},
|
||||
],
|
||||
[]
|
||||
[destinationOwners]
|
||||
);
|
||||
|
||||
const table = useReactTable({
|
||||
@@ -783,7 +814,9 @@ export default function RepositoryTable({
|
||||
searchTerm: "",
|
||||
status: "",
|
||||
organization: "",
|
||||
destinationOrganization: "",
|
||||
owner: "",
|
||||
hasOverrides: "",
|
||||
sort: filter.sort || "imported-desc",
|
||||
})
|
||||
}
|
||||
@@ -794,6 +827,17 @@ export default function RepositoryTable({
|
||||
</div>
|
||||
|
||||
<div className="hidden flex-wrap items-center gap-2 lg:flex">
|
||||
{/* Here rather than beside the owner and organization filters, which
|
||||
leave no room for it at 1280px and below. */}
|
||||
<DestinationCombobox
|
||||
className="h-9"
|
||||
options={destinationOptions}
|
||||
value={filter.destinationOrganization || ""}
|
||||
onChange={(destinationOrganization: string) =>
|
||||
setFilter({ ...filter, destinationOrganization })
|
||||
}
|
||||
/>
|
||||
|
||||
<Select
|
||||
value={filter.status || "all"}
|
||||
onValueChange={(value) =>
|
||||
@@ -1035,6 +1079,7 @@ export default function RepositoryTable({
|
||||
selectedRepoIds={selectedRepoIds}
|
||||
loadingRepoIds={loadingRepoIds}
|
||||
organizationDestinations={organizationDestinations}
|
||||
destinationOwners={destinationOwners}
|
||||
context={rowContext}
|
||||
/>
|
||||
</div>
|
||||
@@ -1098,12 +1143,14 @@ function RepositoryTableRows({
|
||||
selectedRepoIds,
|
||||
loadingRepoIds,
|
||||
organizationDestinations,
|
||||
destinationOwners,
|
||||
context,
|
||||
}: {
|
||||
repositories: Repository[];
|
||||
selectedRepoIds: Set<string>;
|
||||
loadingRepoIds: Set<string>;
|
||||
organizationDestinations?: Record<string, string>;
|
||||
destinationOwners?: Record<string, string>;
|
||||
context: RepositoryRowContext;
|
||||
}) {
|
||||
const scrollRef = useRef<HTMLDivElement>(null);
|
||||
@@ -1144,6 +1191,7 @@ function RepositoryTableRows({
|
||||
organizationDestination={
|
||||
repo.organization ? organizationDestinations?.[repo.organization] : undefined
|
||||
}
|
||||
resolvedDestination={repo.id ? destinationOwners?.[repo.id] : undefined}
|
||||
context={context}
|
||||
/>
|
||||
);
|
||||
@@ -1161,6 +1209,7 @@ const RepositoryTableRow = memo(function RepositoryTableRow({
|
||||
selected,
|
||||
loading,
|
||||
organizationDestination,
|
||||
resolvedDestination,
|
||||
context,
|
||||
}: {
|
||||
repo: Repository;
|
||||
@@ -1170,6 +1219,7 @@ const RepositoryTableRow = memo(function RepositoryTableRow({
|
||||
selected: boolean;
|
||||
loading: boolean;
|
||||
organizationDestination?: string;
|
||||
resolvedDestination?: string;
|
||||
context: RepositoryRowContext;
|
||||
}) {
|
||||
return (
|
||||
@@ -1252,6 +1302,7 @@ const RepositoryTableRow = memo(function RepositoryTableRow({
|
||||
repository={repo}
|
||||
giteaConfig={context.giteaConfig}
|
||||
organizationDestination={organizationDestination}
|
||||
resolvedDestination={resolvedDestination}
|
||||
onUpdate={context.onUpdateDestination}
|
||||
isUpdating={loading}
|
||||
/>
|
||||
|
||||
@@ -7,6 +7,7 @@ const FILTER_KEYS: (keyof FilterParams)[] = [
|
||||
"membershipRole",
|
||||
"owner",
|
||||
"organization",
|
||||
"destinationOrganization",
|
||||
"sort",
|
||||
"type",
|
||||
"name",
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
import { useLayoutEffect, useState } from "react";
|
||||
import { useNavigation } from "@/components/layout/MainLayout";
|
||||
|
||||
/** Matches Tailwind's `lg`, where the dashboard lists sit side by side. */
|
||||
const SIDE_BY_SIDE_QUERY = "(min-width: 1024px)";
|
||||
|
||||
/**
|
||||
* How many rows the dashboard lists can show before they run past the bottom
|
||||
* of the screen, so a tall screen shows more and a short one fewer.
|
||||
*
|
||||
* `container` holds the lists, each marked with `data-fit-rows`. A list is
|
||||
* measured as it is rendered: its first row, the average of the rows after it
|
||||
* (they carry the divider), and the padding and borders between the list and
|
||||
* the container. None of that depends on how many rows are shown, so a new
|
||||
* count measures the same as the last one and the layout settles in one pass.
|
||||
*
|
||||
* Below `lg` the lists stack, so they cannot both end at the screen's bottom,
|
||||
* and `fallback` is used.
|
||||
*/
|
||||
export function useFittingRowCount(
|
||||
container: HTMLElement | null,
|
||||
{ min, max, fallback }: { min: number; max: number; fallback: number },
|
||||
): number {
|
||||
const { pageScrollElement } = useNavigation();
|
||||
const [count, setCount] = useState(fallback);
|
||||
|
||||
useLayoutEffect(() => {
|
||||
if (!container || !pageScrollElement) return;
|
||||
|
||||
const measure = () => {
|
||||
if (!window.matchMedia(SIDE_BY_SIDE_QUERY).matches) {
|
||||
setCount(fallback);
|
||||
return;
|
||||
}
|
||||
|
||||
const scrollStyle = getComputedStyle(pageScrollElement);
|
||||
const bottom =
|
||||
pageScrollElement.getBoundingClientRect().bottom - parseFloat(scrollStyle.paddingBottom);
|
||||
|
||||
let fit: number | undefined;
|
||||
for (const list of container.querySelectorAll<HTMLElement>("[data-fit-rows]")) {
|
||||
const rows = list.children;
|
||||
if (rows.length === 0) continue;
|
||||
|
||||
const listRect = list.getBoundingClientRect();
|
||||
const first = rows[0].getBoundingClientRect().height;
|
||||
const pitch = rows.length > 1 ? (listRect.height - first) / (rows.length - 1) : first + 1;
|
||||
if (first <= 0 || pitch <= 0) continue;
|
||||
|
||||
// Padding and borders below the list, up to the container.
|
||||
let below = 0;
|
||||
for (let el = list.parentElement; el && el !== container; el = el.parentElement) {
|
||||
const style = getComputedStyle(el);
|
||||
below += parseFloat(style.paddingBottom) + parseFloat(style.borderBottomWidth);
|
||||
}
|
||||
|
||||
// Where the list starts with the page scrolled to the top.
|
||||
const top = listRect.top + pageScrollElement.scrollTop;
|
||||
const space = bottom - top - below;
|
||||
const rowsThatFit = space < first ? 0 : 1 + Math.floor((space - first) / pitch);
|
||||
fit = fit === undefined ? rowsThatFit : Math.min(fit, rowsThatFit);
|
||||
}
|
||||
|
||||
if (fit !== undefined) setCount(Math.min(max, Math.max(min, fit)));
|
||||
};
|
||||
|
||||
measure();
|
||||
const observer = new ResizeObserver(measure);
|
||||
observer.observe(pageScrollElement);
|
||||
observer.observe(container);
|
||||
return () => observer.disconnect();
|
||||
}, [container, pageScrollElement, min, max, fallback]);
|
||||
|
||||
return count;
|
||||
}
|
||||
@@ -58,6 +58,12 @@ export const backupStrategyEnum = z.enum([
|
||||
export const releaseTypeEnum = z.enum(["all", "stable"]);
|
||||
export type ReleaseType = z.infer<typeof releaseTypeEnum>;
|
||||
|
||||
// Which source issues are candidates for mirroring. "all" keeps the historical
|
||||
// behavior; the other values narrow the GitHub listing before anything is
|
||||
// created in the destination.
|
||||
export const issueStateFilterEnum = z.enum(["all", "open", "closed"]);
|
||||
export type IssueStateFilter = z.infer<typeof issueStateFilterEnum>;
|
||||
|
||||
/**
|
||||
* Per-object overrides for the global mirror options in `giteaConfigSchema`.
|
||||
*
|
||||
@@ -94,6 +100,11 @@ export type ReleaseType = z.infer<typeof releaseTypeEnum>;
|
||||
* list of strings and ignores everything else, and an empty list is not a pin:
|
||||
* clearing the field means inherit.
|
||||
*
|
||||
* `mirrorIssueState` follows the same lenient-string rule as `releaseType`
|
||||
* (`normalizeIssueState` sanitizes it). `mirrorIssueComments` and
|
||||
* `mirrorIssueMedia` narrow what an issue pass copies; both default to on when
|
||||
* no tier pins them.
|
||||
*
|
||||
* `skipForks` is organization-tier only: it opts an organization out of
|
||||
* importing and mirroring forked repositories without touching the global
|
||||
* `githubConfig.skipForks` switch. Null/absent inherits that global value.
|
||||
@@ -107,6 +118,9 @@ export const mirrorOverridesSchema = z.object({
|
||||
mirrorReleases: z.boolean().nullable().optional(),
|
||||
mirrorMetadata: z.boolean().nullable().optional(),
|
||||
mirrorIssues: z.boolean().nullable().optional(),
|
||||
mirrorIssueComments: z.boolean().nullable().optional(),
|
||||
mirrorIssueMedia: z.boolean().nullable().optional(),
|
||||
mirrorIssueState: z.string().nullable().optional(),
|
||||
mirrorPullRequests: z.boolean().nullable().optional(),
|
||||
mirrorLabels: z.boolean().nullable().optional(),
|
||||
mirrorMilestones: z.boolean().nullable().optional(),
|
||||
@@ -153,6 +167,13 @@ export const giteaConfigSchema = z.object({
|
||||
// Upload assets only for the newest N mirrored releases. null/absent means
|
||||
// every mirrored release gets its assets; 0 means release notes only.
|
||||
releaseAssetLimit: z.number().nullable().optional(),
|
||||
// Opt-in cleanup for what that limit left behind. Off by default, because
|
||||
// the limit alone only gates uploads: attachments a release already has are
|
||||
// never removed. Turning this on deletes the attachments of every mirrored
|
||||
// release ranked past `releaseAssetLimit`. Global only - there is no
|
||||
// per-repository tier for it, the ranks still come from that repository's
|
||||
// own effective asset limit.
|
||||
pruneOldReleaseAssets: z.boolean().default(false),
|
||||
// Which releases are candidates: every one, or only the ones not flagged as a
|
||||
// pre-release. The limit is applied after filtering.
|
||||
releaseType: releaseTypeEnum.default("all"),
|
||||
@@ -164,6 +185,11 @@ export const giteaConfigSchema = z.object({
|
||||
releaseExcludeTags: z.array(z.string()).default([]),
|
||||
mirrorMetadata: z.boolean().default(false),
|
||||
mirrorIssues: z.boolean().default(false),
|
||||
// Which issues the pass lists, and how much of each one it copies. Defaults
|
||||
// preserve the behavior from before these options existed.
|
||||
mirrorIssueState: issueStateFilterEnum.default("all"),
|
||||
mirrorIssueComments: z.boolean().default(true),
|
||||
mirrorIssueMedia: z.boolean().default(true),
|
||||
mirrorPullRequests: z.boolean().default(false),
|
||||
mirrorLabels: z.boolean().default(false),
|
||||
mirrorMilestones: z.boolean().default(false),
|
||||
|
||||
@@ -66,6 +66,7 @@ interface EnvConfig {
|
||||
mirrorMetadata?: boolean;
|
||||
releaseLimit?: number;
|
||||
releaseAssetLimit?: number;
|
||||
pruneOldReleaseAssets?: boolean;
|
||||
releaseType?: string;
|
||||
releaseIncludeTags?: string[];
|
||||
releaseExcludeTags?: string[];
|
||||
@@ -197,6 +198,10 @@ function parseEnvConfig(): EnvConfig {
|
||||
mirrorMetadata: process.env.MIRROR_METADATA === 'true',
|
||||
releaseLimit: process.env.RELEASE_LIMIT ? parseInt(process.env.RELEASE_LIMIT, 10) : undefined,
|
||||
releaseAssetLimit: process.env.RELEASE_ASSET_LIMIT ? parseInt(process.env.RELEASE_ASSET_LIMIT, 10) : undefined,
|
||||
pruneOldReleaseAssets:
|
||||
process.env.PRUNE_OLD_RELEASE_ASSETS === undefined
|
||||
? undefined
|
||||
: process.env.PRUNE_OLD_RELEASE_ASSETS === 'true',
|
||||
releaseType: process.env.RELEASE_TYPE,
|
||||
releaseIncludeTags,
|
||||
releaseExcludeTags,
|
||||
@@ -397,6 +402,12 @@ export async function initializeConfigFromEnv(): Promise<void> {
|
||||
normalizeReleaseAssetLimit(envConfig.mirror.releaseAssetLimit) ??
|
||||
existingConfig?.[0]?.giteaConfig?.releaseAssetLimit ??
|
||||
null,
|
||||
// Unset env var keeps what the UI stored; anything but "true" is false,
|
||||
// matching how the other boolean mirror switches here are read.
|
||||
pruneOldReleaseAssets:
|
||||
envConfig.mirror.pruneOldReleaseAssets ??
|
||||
existingConfig?.[0]?.giteaConfig?.pruneOldReleaseAssets ??
|
||||
false,
|
||||
// all | stable. An unusable value (a typo in the env var)
|
||||
// falls through to the stored value rather than storing a filter.
|
||||
releaseType:
|
||||
@@ -421,6 +432,10 @@ export async function initializeConfigFromEnv(): Promise<void> {
|
||||
: existingConfig?.[0]?.giteaConfig?.pullRequestConcurrency ?? 5,
|
||||
mirrorMetadata: envConfig.mirror.mirrorMetadata ?? (envConfig.mirror.mirrorIssues || envConfig.mirror.mirrorPullRequests || envConfig.mirror.mirrorLabels || envConfig.mirror.mirrorMilestones) ?? existingConfig?.[0]?.giteaConfig?.mirrorMetadata ?? false,
|
||||
mirrorIssues: envConfig.mirror.mirrorIssues ?? existingConfig?.[0]?.giteaConfig?.mirrorIssues ?? false,
|
||||
// No env vars for these; keep what the UI stored across restarts.
|
||||
mirrorIssueState: existingConfig?.[0]?.giteaConfig?.mirrorIssueState ?? 'all',
|
||||
mirrorIssueComments: existingConfig?.[0]?.giteaConfig?.mirrorIssueComments ?? true,
|
||||
mirrorIssueMedia: existingConfig?.[0]?.giteaConfig?.mirrorIssueMedia ?? true,
|
||||
mirrorPullRequests: envConfig.mirror.mirrorPullRequests ?? existingConfig?.[0]?.giteaConfig?.mirrorPullRequests ?? false,
|
||||
mirrorLabels: envConfig.mirror.mirrorLabels ?? existingConfig?.[0]?.giteaConfig?.mirrorLabels ?? false,
|
||||
mirrorMilestones: envConfig.mirror.mirrorMilestones ?? existingConfig?.[0]?.giteaConfig?.mirrorMilestones ?? false,
|
||||
|
||||
@@ -1059,6 +1059,131 @@ describe("Enhanced Gitea Operations", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("stalled mirror sync (#493)", () => {
|
||||
const stallConfig = {
|
||||
userId: "user123",
|
||||
githubConfig: { username: "testuser", token: "github-token", privateRepositories: false, mirrorStarred: true },
|
||||
giteaConfig: { url: "https://gitea.example.com", token: "encrypted-token", defaultOwner: "testuser", mirrorReleases: false },
|
||||
} as unknown as Partial<Config>;
|
||||
const stallDependencies = {
|
||||
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
|
||||
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
|
||||
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
|
||||
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
|
||||
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
|
||||
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
|
||||
syncRepositoryMetadataToGitea: mockSyncRepositoryMetadataToGitea,
|
||||
};
|
||||
const repoUrl = "https://gitea.example.com/api/v1/repos/starred/test-repo";
|
||||
const mirrorUpdated = "2026-10-20T07:55:12Z";
|
||||
const stallRepository = (metadata?: unknown): Repository => ({
|
||||
id: "repo-stall", name: "test-repo", fullName: "ceph/test-repo", owner: "ceph",
|
||||
cloneUrl: "https://github.com/ceph/test-repo.git", isPrivate: false, isStarred: true,
|
||||
status: repoStatusEnum.parse("synced"), visibility: "public", userId: "user123",
|
||||
mirroredLocation: "starred/test-repo", createdAt: new Date(), updatedAt: new Date(),
|
||||
metadata: metadata === undefined ? null : JSON.stringify(metadata),
|
||||
} as Repository);
|
||||
const warningJobs = () =>
|
||||
mockCreateMirrorJob.mock.calls
|
||||
.map((call: any[]) => call[0])
|
||||
.filter((job: any) => job.status === "warning");
|
||||
const storedRequest = () => {
|
||||
const successUpdate = dbUpdateSetCalls.find((data) => "lastMirrored" in data);
|
||||
return JSON.parse(successUpdate.metadata).mirrorSyncRequest;
|
||||
};
|
||||
|
||||
const withMirrorUpdated = async (value: string, run: () => Promise<void>) => {
|
||||
const defaultGet = mockHttpGet.getMockImplementation()!;
|
||||
mockHttpGet.mockImplementation(async (url: string, headers?: any) => {
|
||||
const response: any = await defaultGet(url, headers);
|
||||
return url === repoUrl
|
||||
? { ...response, data: { ...response.data, mirror_updated: value } }
|
||||
: response;
|
||||
});
|
||||
try {
|
||||
await run();
|
||||
} finally {
|
||||
mockHttpGet.mockImplementation(defaultGet);
|
||||
}
|
||||
};
|
||||
|
||||
test("records the request against the mirror_updated read before it", async () => {
|
||||
await withMirrorUpdated(mirrorUpdated, async () => {
|
||||
const result = await syncGiteaRepoEnhanced(
|
||||
{ config: stallConfig, repository: stallRepository() },
|
||||
stallDependencies
|
||||
);
|
||||
expect(result).toEqual({ success: true });
|
||||
});
|
||||
|
||||
expect(storedRequest()).toEqual({
|
||||
requestedAt: expect.any(String),
|
||||
mirrorUpdatedBefore: mirrorUpdated,
|
||||
});
|
||||
expect(warningJobs()).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("warns once when Gitea never ran the previous request", async () => {
|
||||
const requestedAt = new Date(Date.now() - 8 * 60 * 60 * 1000).toISOString();
|
||||
|
||||
await withMirrorUpdated(mirrorUpdated, async () => {
|
||||
await syncGiteaRepoEnhanced(
|
||||
{
|
||||
config: stallConfig,
|
||||
repository: stallRepository({ mirrorSyncRequest: { requestedAt, mirrorUpdatedBefore: mirrorUpdated } }),
|
||||
},
|
||||
stallDependencies
|
||||
);
|
||||
});
|
||||
|
||||
const warnings = warningJobs();
|
||||
expect(warnings).toHaveLength(1);
|
||||
expect(warnings[0].message).toBe("Gitea has not run the requested sync for starred/test-repo");
|
||||
expect(warnings[0].details).toContain("requested 8 hours ago");
|
||||
expect(warnings[0].details).toContain(`still ${mirrorUpdated}`);
|
||||
expect(warnings[0].details).toContain("Monitoring > Queues > mirror");
|
||||
expect(warnings[0].skipNotification).toBe(true);
|
||||
// The original request is kept, now marked as reported.
|
||||
expect(storedRequest()).toEqual({
|
||||
requestedAt,
|
||||
mirrorUpdatedBefore: mirrorUpdated,
|
||||
warnedAt: expect.any(String),
|
||||
});
|
||||
|
||||
// The next sync finds the same request still waiting and stays quiet.
|
||||
mockCreateMirrorJob.mockClear();
|
||||
const warnedMetadata = dbUpdateSetCalls.find((data) => "lastMirrored" in data).metadata;
|
||||
dbUpdateSetCalls.length = 0;
|
||||
await withMirrorUpdated(mirrorUpdated, async () => {
|
||||
await syncGiteaRepoEnhanced(
|
||||
{ config: stallConfig, repository: stallRepository(JSON.parse(warnedMetadata)) },
|
||||
stallDependencies
|
||||
);
|
||||
});
|
||||
expect(warningJobs()).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("starts over once Gitea ran the previous request", async () => {
|
||||
const requestedAt = new Date(Date.now() - 8 * 60 * 60 * 1000).toISOString();
|
||||
const ranAt = "2026-10-20T08:00:03Z";
|
||||
|
||||
await withMirrorUpdated(ranAt, async () => {
|
||||
await syncGiteaRepoEnhanced(
|
||||
{
|
||||
config: stallConfig,
|
||||
repository: stallRepository({ mirrorSyncRequest: { requestedAt, mirrorUpdatedBefore: mirrorUpdated } }),
|
||||
},
|
||||
stallDependencies
|
||||
);
|
||||
});
|
||||
|
||||
expect(warningJobs()).toHaveLength(0);
|
||||
const record = storedRequest();
|
||||
expect(record.mirrorUpdatedBefore).toBe(ranAt);
|
||||
expect(Date.parse(record.requestedAt)).toBeGreaterThan(Date.parse(requestedAt));
|
||||
});
|
||||
});
|
||||
|
||||
describe("GitHub App sources", () => {
|
||||
const appConfig = {
|
||||
userId: "user123",
|
||||
|
||||
@@ -45,6 +45,12 @@ import {
|
||||
} from "./repo-backup";
|
||||
import { detectForcePush } from "./utils/force-push-detection";
|
||||
import { destinationSizeColumns } from "./repository-storage";
|
||||
import {
|
||||
assessMirrorSyncRequest,
|
||||
formatWaitedTime,
|
||||
STUCK_MIRROR_QUEUE_HINT,
|
||||
} from "./mirror-sync-health";
|
||||
import { triggerMirrorSyncStalledNotification } from "./notification-service";
|
||||
import { sanitizeRepoNameAlphaDashDot, type GiteaMilestoneIds } from "./gitea";
|
||||
import {
|
||||
parseRepositoryMetadataState,
|
||||
@@ -91,6 +97,9 @@ export interface GiteaRepoInfo {
|
||||
size?: number;
|
||||
// Set while a transfer waits for the new owner to accept it (issue #400).
|
||||
repo_transfer?: { recipient?: { login?: string; username?: string } | null } | null;
|
||||
// Last pull mirror run Gitea processed, failed runs included. Tells whether
|
||||
// the previous mirror-sync request ever ran (#493).
|
||||
mirror_updated?: string;
|
||||
}
|
||||
|
||||
export interface GiteaTransferResult {
|
||||
@@ -957,6 +966,60 @@ export async function syncGiteaRepoEnhanced({
|
||||
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
|
||||
});
|
||||
|
||||
// Gitea queues the pull and answers 200 before running it, so whether
|
||||
// the previous request ever ran shows only in mirror_updated (#493).
|
||||
// repoInfo is from the GET before this request, so the value it holds
|
||||
// is what this request is measured against next time.
|
||||
const mirrorSyncCheck = assessMirrorSyncRequest({
|
||||
previous: metadataState.mirrorSyncRequest,
|
||||
mirrorUpdated: repoInfo.mirror_updated,
|
||||
});
|
||||
const mirrorSyncRequestChanged =
|
||||
JSON.stringify(mirrorSyncCheck.next) !== JSON.stringify(metadataState.mirrorSyncRequest);
|
||||
metadataState.mirrorSyncRequest = mirrorSyncCheck.next;
|
||||
|
||||
if (mirrorSyncCheck.stalled) {
|
||||
const { mirrorUpdated, waitedMs } = mirrorSyncCheck.stalled;
|
||||
const destination = `${repoOwner}/${repoName}`;
|
||||
const waited = formatWaitedTime(waitedMs);
|
||||
console.warn(
|
||||
`[Sync] Gitea has not run the sync requested for ${destination} ${waited} ago (mirror_updated still ${mirrorUpdated})`
|
||||
);
|
||||
|
||||
void triggerMirrorSyncStalledNotification({
|
||||
userId: config.userId,
|
||||
repositoryName: repository.name,
|
||||
destination,
|
||||
waited,
|
||||
mirrorUpdated,
|
||||
});
|
||||
|
||||
// Reported here rather than at the end: the record above is marked as
|
||||
// warned and can be persisted by a metadata pass even if the rest of
|
||||
// the sync fails. Same shape as the all-releases-removed alert: an
|
||||
// activity-only "warning" row, kept out of the job notifications (the
|
||||
// trigger above sends its own) and out of the event stream so it does
|
||||
// not swallow the completion event.
|
||||
await createMirrorJob({
|
||||
userId: config.userId,
|
||||
repositoryId: repository.id,
|
||||
repositoryName: repository.name,
|
||||
status: "warning",
|
||||
skipNotification: true,
|
||||
skipDuplicateEvent: true,
|
||||
message: `Gitea has not run the requested sync for ${destination}`,
|
||||
details:
|
||||
`The sync was requested ${waited} ago, but Gitea's last mirror run is still ${mirrorUpdated}. ` +
|
||||
STUCK_MIRROR_QUEUE_HINT,
|
||||
}).catch((activityError) => {
|
||||
console.error(
|
||||
`[Sync] Failed to record the stalled sync activity for ${repository.name}: ${
|
||||
activityError instanceof Error ? activityError.message : String(activityError)
|
||||
}`
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
// metadataState + metadataUpdated are hoisted above the backup
|
||||
// strategy block so force-push detection can read/write the
|
||||
// acknowledged-deletions list. Don't shadow them here.
|
||||
@@ -1166,6 +1229,9 @@ export async function syncGiteaRepoEnhanced({
|
||||
repository,
|
||||
giteaOwner: repoOwner,
|
||||
giteaRepoName: repoName,
|
||||
issueState: mirrorOptions.mirrorIssueState,
|
||||
includeComments: mirrorOptions.mirrorIssueComments,
|
||||
includeMedia: mirrorOptions.mirrorIssueMedia,
|
||||
syncCursor: metadataState.syncCursors.issues,
|
||||
passProgress: metadataState.passProgress.issues,
|
||||
onPassProgress: async (progress) => {
|
||||
@@ -1292,7 +1358,7 @@ export async function syncGiteaRepoEnhanced({
|
||||
lastMirrored: new Date(),
|
||||
...(isArchivedRepo ? {} : { errorMessage: null }),
|
||||
mirroredLocation: `${repoOwner}/${repoName}`,
|
||||
metadata: metadataUpdated
|
||||
metadata: metadataUpdated || mirrorSyncRequestChanged
|
||||
? serializeRepositoryMetadataState(metadataState)
|
||||
: repository.metadata ?? null,
|
||||
// From the GET that resolved the target, so before this sync's
|
||||
|
||||
@@ -195,7 +195,11 @@ function fakeOctokit(data: GitHubData) {
|
||||
const issues = {
|
||||
listForRepo: async (params: any) => {
|
||||
calls.push({ endpoint: "issues.listForRepo", params });
|
||||
return { data: data.issueListing };
|
||||
// GitHub answers a state filter with only the issues in that state.
|
||||
const state = params.state ?? "open";
|
||||
return {
|
||||
data: data.issueListing.filter((issue) => state === "all" || issue.state === state),
|
||||
};
|
||||
},
|
||||
listComments: async (params: any) => {
|
||||
calls.push({ endpoint: "issues.listComments", params });
|
||||
@@ -983,3 +987,92 @@ describe.skipIf(!isChild)("milestone assignment (#460)", () => {
|
||||
expect(cursor).toBeDefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe.skipIf(!isChild)("mirrorGitRepoIssuesToGitea content options", () => {
|
||||
const baseArgs = (octokit: any, extra: Record<string, unknown> = {}) => ({
|
||||
config,
|
||||
octokit,
|
||||
repository,
|
||||
giteaOwner: DEST_OWNER,
|
||||
giteaRepoName: DEST_REPO,
|
||||
...extra,
|
||||
});
|
||||
|
||||
it("lists every state and creates only the issues the filter keeps", async () => {
|
||||
const gitea = fakeGitea([]);
|
||||
const github = fakeOctokit({
|
||||
issueListing: [ghIssue(1), ghIssue(2, { state: "closed" })],
|
||||
});
|
||||
|
||||
const cursor = await mirrorGitRepoIssuesToGitea(
|
||||
baseArgs(github.octokit, { issueState: "open" })
|
||||
);
|
||||
|
||||
expect(github.callsTo("issues.listForRepo")[0]?.params.state).toBe("all");
|
||||
expect(gitea.createdTitles()).toEqual(["[GH-ISSUE #1] Issue 1"]);
|
||||
expect(github.callsTo("issues.listComments").map((c) => c.params.issue_number)).toEqual([1]);
|
||||
expect(cursor?.issueState).toBe("open");
|
||||
});
|
||||
|
||||
it("closes the copy of a mirrored issue that was closed on GitHub", async () => {
|
||||
const gitea = fakeGitea([{ number: 21, title: "[GH-ISSUE #1] Issue 1" }]);
|
||||
const github = fakeOctokit({ issueListing: [ghIssue(1, { state: "closed" })] });
|
||||
|
||||
await mirrorGitRepoIssuesToGitea(baseArgs(github.octokit, { issueState: "open" }));
|
||||
|
||||
const edit = gitea.requests.find(
|
||||
(r) => r.method === "PATCH" && r.url === `${DEST_API}/issues/21`
|
||||
);
|
||||
expect(JSON.parse(edit!.body!).state).toBe("closed");
|
||||
});
|
||||
|
||||
it("runs a full pass when the filter differs from the last pass", async () => {
|
||||
fakeGitea([{ number: 21, title: "[GH-ISSUE #1] Issue 1" }]);
|
||||
const changed = fakeOctokit({ issueListing: [] });
|
||||
await mirrorGitRepoIssuesToGitea(
|
||||
baseArgs(changed.octokit, { issueState: "all", syncCursor: { ...recentCursor(), issueState: "open" } })
|
||||
);
|
||||
expect(changed.callsTo("issues.listForRepo")[0]?.params.since).toBeUndefined();
|
||||
|
||||
fakeGitea([{ number: 21, title: "[GH-ISSUE #1] Issue 1" }]);
|
||||
const same = fakeOctokit({ issueListing: [] });
|
||||
const cursor = recentCursor();
|
||||
await mirrorGitRepoIssuesToGitea(
|
||||
baseArgs(same.octokit, { issueState: "open", syncCursor: { ...cursor, issueState: "open" } })
|
||||
);
|
||||
expect(same.callsTo("issues.listForRepo")[0]?.params.since).toBe(expectedSince(cursor));
|
||||
});
|
||||
|
||||
it("skips comments when the comments option is off", async () => {
|
||||
fakeGitea([]);
|
||||
const github = fakeOctokit({ issueListing: [ghIssue(1)] });
|
||||
|
||||
await mirrorGitRepoIssuesToGitea(baseArgs(github.octokit, { includeComments: false }));
|
||||
|
||||
expect(github.callsTo("issues.listComments")).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("keeps media by default and strips it when the media option is off", async () => {
|
||||
const issue = ghIssue(1, {
|
||||
body: "Before  after",
|
||||
});
|
||||
|
||||
const giteaWithMedia = fakeGitea([]);
|
||||
const withMedia = fakeOctokit({ issueListing: [issue] });
|
||||
await mirrorGitRepoIssuesToGitea(baseArgs(withMedia.octokit));
|
||||
|
||||
const giteaWithoutMedia = fakeGitea([]);
|
||||
const withoutMedia = fakeOctokit({ issueListing: [issue] });
|
||||
await mirrorGitRepoIssuesToGitea(
|
||||
baseArgs(withoutMedia.octokit, { includeMedia: false })
|
||||
);
|
||||
|
||||
expect(giteaWithMedia.createdBodies()[0]).toContain(
|
||||
"user-images.githubusercontent.com"
|
||||
);
|
||||
expect(giteaWithoutMedia.createdBodies()[0]).toContain("[media omitted]");
|
||||
expect(giteaWithoutMedia.createdBodies()[0]).not.toContain(
|
||||
"user-images.githubusercontent.com"
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -157,6 +157,12 @@ interface DestinationOptions {
|
||||
destinationPageCap?: number;
|
||||
/** Destination release ids whose DELETE fails. */
|
||||
failDeleteIds?: number[];
|
||||
/** Attachments a destination release has, keyed by release id. */
|
||||
destinationAssets?: Record<string, Array<{ id: number; name: string; size: number }>>;
|
||||
/** Attachment ids whose DELETE fails. */
|
||||
failAssetDeleteIds?: number[];
|
||||
/** Release ids whose attachment listing answers with an error. */
|
||||
failAssetListingFor?: number[];
|
||||
}
|
||||
|
||||
function fakeWorld(options: DestinationOptions) {
|
||||
@@ -210,7 +216,19 @@ function fakeWorld(options: DestinationOptions) {
|
||||
|
||||
// Existing attachments of a release.
|
||||
if (method === "GET" && /\/releases\/\d+\/assets$/.test(url)) {
|
||||
return json([]);
|
||||
const releaseId = Number(url.match(/\/releases\/(\d+)\/assets$/)?.[1]);
|
||||
if (options.failAssetListingFor?.includes(releaseId)) {
|
||||
return new Response("boom", { status: 500 });
|
||||
}
|
||||
return json(options.destinationAssets?.[String(releaseId)] ?? []);
|
||||
}
|
||||
|
||||
// Pruning the attachments of a release that is past the asset limit.
|
||||
if (method === "DELETE" && /\/releases\/\d+\/assets\/\d+$/.test(url)) {
|
||||
const assetId = Number(url.slice(url.lastIndexOf("/") + 1));
|
||||
return options.failAssetDeleteIds?.includes(assetId)
|
||||
? new Response("boom", { status: 500 })
|
||||
: new Response(null, { status: 204 });
|
||||
}
|
||||
|
||||
// Uploading an attachment.
|
||||
@@ -267,6 +285,10 @@ function fakeWorld(options: DestinationOptions) {
|
||||
requests.filter(
|
||||
(r) => r.url.startsWith(`${SOURCE_URL}/attachments/`) || r.url.startsWith("https://github.com/")
|
||||
),
|
||||
assetListings: () =>
|
||||
requests.filter((r) => r.method === "GET" && /\/releases\/\d+\/assets$/.test(r.url)),
|
||||
assetDeletes: () =>
|
||||
requests.filter((r) => r.method === "DELETE" && /\/releases\/\d+\/assets\/\d+$/.test(r.url)),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -959,3 +981,120 @@ describe.skipIf(!isChild)("mirrorGitHubReleasesToGitea keeps its own header and
|
||||
expect(download.authorization).toBe("token ghes-token");
|
||||
});
|
||||
});
|
||||
|
||||
describe.skipIf(!isChild)("pruning release assets past the asset limit", () => {
|
||||
function configWithPruning(prune: boolean) {
|
||||
return {
|
||||
...config,
|
||||
giteaConfig: {
|
||||
...(config as any).giteaConfig,
|
||||
pruneOldReleaseAssets: prune,
|
||||
},
|
||||
} as unknown as Partial<Config>;
|
||||
}
|
||||
|
||||
const destinationReleases = [
|
||||
{
|
||||
id: 1,
|
||||
tag_name: "v3.0.0",
|
||||
created_at: "2026-03-01T00:00:00Z",
|
||||
prerelease: false,
|
||||
assets: [{ id: 11, name: "new.bin", size: 7 }],
|
||||
},
|
||||
{
|
||||
id: 2,
|
||||
tag_name: "v2.0.0",
|
||||
created_at: "2026-02-01T00:00:00Z",
|
||||
prerelease: false,
|
||||
assets: [{ id: 21, name: "old.bin", size: 5 }],
|
||||
},
|
||||
{
|
||||
id: 3,
|
||||
tag_name: "v1.0.0",
|
||||
created_at: "2026-01-01T00:00:00Z",
|
||||
prerelease: false,
|
||||
assets: [{ id: 31, name: "older.bin", size: 3 }],
|
||||
},
|
||||
];
|
||||
|
||||
/** Three releases on both sides; an asset limit of 1 covers only v3.0.0. */
|
||||
function run(prune: boolean, extra: Partial<DestinationOptions> = {}) {
|
||||
const world = fakeWorld({
|
||||
sourceReleases: [
|
||||
giteaRelease("v3.0.0"),
|
||||
giteaRelease("v2.0.0"),
|
||||
giteaRelease("v1.0.0"),
|
||||
],
|
||||
destinationReleases,
|
||||
destinationAssets: {
|
||||
"1": [{ id: 11, name: "new.bin", size: 7 }],
|
||||
"2": [{ id: 21, name: "old.bin", size: 5 }],
|
||||
"3": [{ id: 31, name: "older.bin", size: 3 }],
|
||||
},
|
||||
...extra,
|
||||
});
|
||||
|
||||
return mirrorGiteaSourceReleasesToGitea({
|
||||
config: configWithPruning(prune),
|
||||
repository,
|
||||
connection: giteaConnection("source-token"),
|
||||
giteaOwner: DEST_OWNER,
|
||||
giteaRepoName: DEST_REPO,
|
||||
releaseLimit: 3,
|
||||
releaseAssetLimit: 1,
|
||||
}).then(() => world);
|
||||
}
|
||||
|
||||
it("keeps attachments a looser limit uploaded before", async () => {
|
||||
const world = await run(false);
|
||||
|
||||
expect(world.assetListings()).toHaveLength(0);
|
||||
expect(world.assetDeletes()).toHaveLength(0);
|
||||
expect(storageRecords.at(-1)?.summary).toEqual({ count: 3, bytes: 15 });
|
||||
});
|
||||
|
||||
it("deletes the attachments of releases past the asset limit when enabled", async () => {
|
||||
const world = await run(true);
|
||||
|
||||
expect(world.assetDeletes().map((r) => r.url)).toEqual([
|
||||
`${DEST_API}/releases/2/assets/21`,
|
||||
`${DEST_API}/releases/3/assets/31`,
|
||||
]);
|
||||
// The newest release stays inside the limit and keeps its file.
|
||||
expect(world.assetListings().map((r) => r.url)).toEqual([
|
||||
`${DEST_API}/releases/2/assets`,
|
||||
`${DEST_API}/releases/3/assets`,
|
||||
]);
|
||||
// The storage summary follows the deletions instead of reporting the old total.
|
||||
expect(storageRecords.at(-1)?.summary).toEqual({ count: 1, bytes: 7 });
|
||||
});
|
||||
|
||||
it("skips releases the listing shows without attachments", async () => {
|
||||
// v1.0.0 was pruned on an earlier sync: it is not listed or deleted again.
|
||||
const world = await run(true, {
|
||||
destinationReleases: destinationReleases.map((release) =>
|
||||
release.id === 3 ? { ...release, assets: [] } : release
|
||||
),
|
||||
destinationAssets: {
|
||||
"1": [{ id: 11, name: "new.bin", size: 7 }],
|
||||
"2": [{ id: 21, name: "old.bin", size: 5 }],
|
||||
"3": [],
|
||||
},
|
||||
});
|
||||
|
||||
expect(world.assetListings().map((r) => r.url)).toEqual([
|
||||
`${DEST_API}/releases/2/assets`,
|
||||
]);
|
||||
expect(world.assetDeletes().map((r) => r.url)).toEqual([
|
||||
`${DEST_API}/releases/2/assets/21`,
|
||||
]);
|
||||
expect(storageRecords.at(-1)?.summary).toEqual({ count: 1, bytes: 7 });
|
||||
});
|
||||
|
||||
it("deletes nothing when a release's attachments cannot be listed", async () => {
|
||||
const world = await run(true, { failAssetListingFor: [2, 3] });
|
||||
|
||||
expect(world.assetDeletes()).toHaveLength(0);
|
||||
expect(storageRecords.at(-1)?.summary).toEqual({ count: 3, bytes: 15 });
|
||||
});
|
||||
});
|
||||
+100
-1
@@ -1,7 +1,12 @@
|
||||
import { describe, test, expect, mock, beforeEach, afterEach } from "bun:test";
|
||||
import { Octokit } from "@octokit/rest";
|
||||
import { repoStatusEnum } from "@/types/Repository";
|
||||
import { getOrCreateGiteaOrg, getGiteaRepoOwner, getGiteaRepoOwnerAsync } from "./gitea";
|
||||
import {
|
||||
getOrCreateGiteaOrg,
|
||||
getGiteaRepoOwner,
|
||||
getGiteaRepoOwnerAsync,
|
||||
resolveDestinationOwner,
|
||||
} from "./gitea";
|
||||
import type { Config, Repository, Organization } from "./db/schema";
|
||||
import { createMockResponse, mockFetch } from "@/tests/mock-fetch";
|
||||
|
||||
@@ -751,3 +756,97 @@ describe("getGiteaRepoOwner - Organization Override Tests", () => {
|
||||
expect(result).toBe("nice-user");
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveDestinationOwner", () => {
|
||||
const config = {
|
||||
githubConfig: {
|
||||
owner: "testuser",
|
||||
username: "testuser",
|
||||
token: "token",
|
||||
starredReposOrg: "starred",
|
||||
starredReposMode: "dedicated-org",
|
||||
mirrorStrategy: "preserve",
|
||||
},
|
||||
giteaConfig: {
|
||||
defaultOwner: "giteauser",
|
||||
url: "https://gitea.example.com",
|
||||
token: "gitea-token",
|
||||
organization: "github-mirrors",
|
||||
visibility: "public",
|
||||
preserveVisibility: false,
|
||||
},
|
||||
} as unknown as Partial<Config>;
|
||||
|
||||
const repo = (overrides: Partial<Repository> = {}) =>
|
||||
({
|
||||
fullName: "acme/demo",
|
||||
owner: "acme",
|
||||
organization: "acme",
|
||||
destinationOrg: null,
|
||||
isStarred: false,
|
||||
...overrides,
|
||||
}) as Repository;
|
||||
|
||||
test("uses the starred organization in dedicated mode", () => {
|
||||
expect(
|
||||
resolveDestinationOwner({
|
||||
config,
|
||||
repository: repo({ isStarred: true }),
|
||||
})
|
||||
).toBe("starred");
|
||||
});
|
||||
|
||||
test("preserves the owner namespace for starred repos in preserve-owner mode", () => {
|
||||
expect(
|
||||
resolveDestinationOwner({
|
||||
config: {
|
||||
...config,
|
||||
githubConfig: {
|
||||
...config.githubConfig!,
|
||||
starredReposMode: "preserve-owner",
|
||||
},
|
||||
},
|
||||
repository: repo({ isStarred: true, owner: "other", organization: undefined }),
|
||||
})
|
||||
).toBe("other");
|
||||
});
|
||||
|
||||
test("a repository override wins over the organization override", () => {
|
||||
expect(
|
||||
resolveDestinationOwner({
|
||||
config,
|
||||
repository: repo({ destinationOrg: "repo-target" }),
|
||||
organizationDestination: "org-target",
|
||||
})
|
||||
).toBe("repo-target");
|
||||
});
|
||||
|
||||
test("the organization override wins over the strategy", () => {
|
||||
expect(
|
||||
resolveDestinationOwner({
|
||||
config,
|
||||
repository: repo(),
|
||||
organizationDestination: "org-target",
|
||||
})
|
||||
).toBe("org-target");
|
||||
});
|
||||
|
||||
test("preserve keeps the source organization when no override exists", () => {
|
||||
expect(
|
||||
resolveDestinationOwner({
|
||||
config,
|
||||
repository: repo(),
|
||||
})
|
||||
).toBe("acme");
|
||||
});
|
||||
|
||||
test("personal repos of the configured source user go to defaultOwner", () => {
|
||||
expect(
|
||||
resolveDestinationOwner({
|
||||
config,
|
||||
repository: repo({ owner: "testuser", organization: undefined }),
|
||||
sourceUsername: "testuser",
|
||||
})
|
||||
).toBe("giteauser");
|
||||
});
|
||||
});
|
||||
+374
-123
@@ -7,9 +7,10 @@ import {
|
||||
import { membershipRoleEnum } from "@/types/organizations";
|
||||
import { Octokit } from "@octokit/rest";
|
||||
import type { Config } from "@/types/config";
|
||||
import type { Organization, ReleaseType, Repository } from "./db/schema";
|
||||
import type { IssueStateFilter, Organization, ReleaseType, Repository } from "./db/schema";
|
||||
import { httpPost, httpGet, httpDelete, httpPut, httpPatch } from "./http-client";
|
||||
import { createMirrorJob } from "./helpers";
|
||||
import { applyIssueMediaOption, stripIssueMedia } from "./utils/issue-media";
|
||||
import { db, organizations, repositories } from "./db";
|
||||
import { eq, and, ne } from "drizzle-orm";
|
||||
import { decryptConfigTokens } from "./utils/config-encryption";
|
||||
@@ -218,38 +219,36 @@ export const getGiteaRepoOwnerAsync = async ({
|
||||
throw new Error("User ID is required for organization overrides.");
|
||||
}
|
||||
|
||||
// Check if repository is starred
|
||||
if (repository.isStarred) {
|
||||
const starredReposMode = config.githubConfig.starredReposMode || "dedicated-org";
|
||||
if (starredReposMode === "preserve-owner") {
|
||||
return repository.organization || repository.owner;
|
||||
}
|
||||
return config.githubConfig.starredReposOrg || "starred";
|
||||
}
|
||||
|
||||
// Check for repository-specific override (second highest priority)
|
||||
if (repository.destinationOrg) {
|
||||
console.log(`Using repository override: ${repository.fullName} -> ${repository.destinationOrg}`);
|
||||
return repository.destinationOrg;
|
||||
}
|
||||
|
||||
// Check for organization-specific override
|
||||
if (repository.organization) {
|
||||
// The organization override is the only tier that needs a database read,
|
||||
// and only when the starred mode or a repository override has not already
|
||||
// decided. Everything else is shared with `resolveDestinationOwner` below,
|
||||
// which the repositories list uses to filter by the real destination
|
||||
// without one query per row.
|
||||
let organizationDestination: string | undefined;
|
||||
if (!repository.isStarred && !repository.destinationOrg && repository.organization) {
|
||||
const orgConfig = await getOrganizationConfig({
|
||||
orgName: repository.organization,
|
||||
userId: config.userId,
|
||||
});
|
||||
|
||||
if (orgConfig?.destinationOrg) {
|
||||
console.log(`Using organization override: ${repository.organization} -> ${orgConfig.destinationOrg}`);
|
||||
return orgConfig.destinationOrg;
|
||||
}
|
||||
organizationDestination = orgConfig?.destinationOrg ?? undefined;
|
||||
}
|
||||
|
||||
// For personal repos (not organization repos), fall back to the default strategy
|
||||
const owner = resolveDestinationOwner({
|
||||
config,
|
||||
repository,
|
||||
organizationDestination,
|
||||
sourceUsername,
|
||||
});
|
||||
|
||||
// Fall back to existing strategy logic
|
||||
return getGiteaRepoOwner({ config, repository, sourceUsername });
|
||||
// Logged here rather than in the resolver, which the repositories list
|
||||
// runs for every row on every load.
|
||||
if (!repository.isStarred && repository.destinationOrg) {
|
||||
console.log(`Using repository override: ${repository.fullName} -> ${repository.destinationOrg}`);
|
||||
} else if (organizationDestination) {
|
||||
console.log(`Using organization override: ${repository.organization} -> ${organizationDestination}`);
|
||||
}
|
||||
|
||||
return owner;
|
||||
};
|
||||
|
||||
export const getGiteaRepoOwner = ({
|
||||
@@ -347,6 +346,53 @@ export const getGiteaRepoOwner = ({
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve the destination owner with the same precedence the mirror uses:
|
||||
* starred mode > repository override > organization override > strategy.
|
||||
*
|
||||
* The organization override is passed in rather than fetched so callers that
|
||||
* already hold the organization rows (the repositories list) can resolve many
|
||||
* repositories without a database round-trip per row.
|
||||
*/
|
||||
export const resolveDestinationOwner = ({
|
||||
config,
|
||||
repository,
|
||||
organizationDestination,
|
||||
sourceUsername,
|
||||
}: {
|
||||
config: Partial<Config>;
|
||||
repository: Repository;
|
||||
organizationDestination?: string | null;
|
||||
/** Account name of the repository's own source, when already resolved. */
|
||||
sourceUsername?: string;
|
||||
}): string => {
|
||||
if (!config.githubConfig || !config.giteaConfig) {
|
||||
throw new Error("GitHub or Gitea config is required.");
|
||||
}
|
||||
|
||||
if (!config.giteaConfig.defaultOwner) {
|
||||
throw new Error("Gitea username is required.");
|
||||
}
|
||||
|
||||
if (repository.isStarred) {
|
||||
const starredReposMode = config.githubConfig.starredReposMode || "dedicated-org";
|
||||
if (starredReposMode === "preserve-owner") {
|
||||
return repository.organization || repository.owner;
|
||||
}
|
||||
return config.githubConfig.starredReposOrg || "starred";
|
||||
}
|
||||
|
||||
if (repository.destinationOrg) {
|
||||
return repository.destinationOrg;
|
||||
}
|
||||
|
||||
if (repository.organization && organizationDestination) {
|
||||
return organizationDestination;
|
||||
}
|
||||
|
||||
return getGiteaRepoOwner({ config, repository, sourceUsername });
|
||||
};
|
||||
|
||||
export const isRepoPresentInGitea = async ({
|
||||
config,
|
||||
owner,
|
||||
@@ -1272,6 +1318,9 @@ export const mirrorGithubRepoToGitea = async ({
|
||||
repository,
|
||||
giteaOwner: repoOwner,
|
||||
giteaRepoName: targetRepoName,
|
||||
issueState: mirrorOptions.mirrorIssueState,
|
||||
includeComments: mirrorOptions.mirrorIssueComments,
|
||||
includeMedia: mirrorOptions.mirrorIssueMedia,
|
||||
syncCursor: metadataState.syncCursors.issues,
|
||||
passProgress: metadataState.passProgress.issues,
|
||||
onPassProgress: async (progress) => {
|
||||
@@ -2095,6 +2144,9 @@ export async function mirrorGitHubRepoToGiteaOrg({
|
||||
repository,
|
||||
giteaOwner: orgName,
|
||||
giteaRepoName: targetRepoName,
|
||||
issueState: mirrorOptions.mirrorIssueState,
|
||||
includeComments: mirrorOptions.mirrorIssueComments,
|
||||
includeMedia: mirrorOptions.mirrorIssueMedia,
|
||||
syncCursor: metadataState.syncCursors.issues,
|
||||
passProgress: metadataState.passProgress.issues,
|
||||
onPassProgress: async (progress) => {
|
||||
@@ -2809,12 +2861,26 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
passProgress,
|
||||
onPassProgress,
|
||||
milestoneIds,
|
||||
issueState = "all",
|
||||
includeComments = true,
|
||||
includeMedia = true,
|
||||
}: {
|
||||
config: Partial<Config>;
|
||||
octokit: Octokit;
|
||||
repository: Repository;
|
||||
giteaOwner: string;
|
||||
giteaRepoName?: string;
|
||||
/**
|
||||
* Which source issues get created. "all" keeps the historical behavior.
|
||||
* The listing still asks for every state, so an issue that is already
|
||||
* mirrored keeps following GitHub when it leaves the filter: closing it
|
||||
* there closes the copy. Nothing is deleted.
|
||||
*/
|
||||
issueState?: IssueStateFilter;
|
||||
/** Copy issue comments. Off skips the per-issue comment listing entirely. */
|
||||
includeComments?: boolean;
|
||||
/** Keep images/videos/attachment links in issue bodies and comments. */
|
||||
includeMedia?: boolean;
|
||||
/**
|
||||
* Gitea milestone ids by title from the milestones pass that ran just
|
||||
* before (#460). Without it the issues' milestones are left untouched.
|
||||
@@ -2888,9 +2954,19 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
}
|
||||
// GitHub numbers this pass has finished, kept if it does not complete.
|
||||
const completedNumbers = new Set<number>();
|
||||
// Issues an earlier state filter left out have not changed since, so an
|
||||
// incremental listing would never return them. A pass under a different
|
||||
// filter lists everything once.
|
||||
if (plan.mode === "incremental" && (syncCursor?.issueState ?? "all") !== issueState) {
|
||||
console.log(
|
||||
`[Issues] The issue state filter for ${repository.fullName} changed; running a full pass`
|
||||
);
|
||||
plan = { mode: "full", reason: "the issue state filter changed" };
|
||||
}
|
||||
const finishPass = async () => {
|
||||
if (resume) await onPassProgress?.(undefined);
|
||||
return advanceMetadataSyncCursor(syncCursor, plan.mode, passStartedAt);
|
||||
const cursor = advanceMetadataSyncCursor(syncCursor, plan.mode, passStartedAt);
|
||||
return issueState === "all" ? cursor : { ...cursor, issueState };
|
||||
};
|
||||
const keepPassProgress = async () => {
|
||||
await onPassProgress?.(buildPassProgress(plan, passStartedAt, completedNumbers, resume));
|
||||
@@ -3068,6 +3144,7 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
}
|
||||
|
||||
let unchangedIssueCount = 0;
|
||||
let filteredOutIssueCount = 0;
|
||||
|
||||
// Process issues in parallel with concurrency control
|
||||
let mirroredIssues: unknown[];
|
||||
@@ -3088,6 +3165,17 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
return issue;
|
||||
}
|
||||
|
||||
// The state filter only decides what gets created. Not counted as done,
|
||||
// so a resumed pass under another filter looks at the issue again.
|
||||
if (
|
||||
issueState !== "all" &&
|
||||
issue.state !== issueState &&
|
||||
!giteaIssueByGitHubNumber.has(issue.number)
|
||||
) {
|
||||
filteredOutIssueCount++;
|
||||
return issue;
|
||||
}
|
||||
|
||||
const githubLabelNames =
|
||||
issue.labels
|
||||
?.map((l) => (typeof l === "string" ? l : l.name))
|
||||
@@ -3133,7 +3221,7 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
}.`;
|
||||
const issueMarker = `[GH-ISSUE #${issue.number}]`;
|
||||
const mirroredTitle = `${issueMarker} ${issue.title}`;
|
||||
const issueBody = `${issueOriginHeader}\nOriginal GitHub issue: ${issue.html_url}${originalAssignees}\n\n${issue.body ?? ""}`;
|
||||
const issueBody = `${issueOriginHeader}\nOriginal GitHub issue: ${issue.html_url}${originalAssignees}\n\n${applyIssueMediaOption(issue.body, includeMedia)}`;
|
||||
|
||||
const wantedMilestoneId = resolveGiteaMilestoneId(milestoneIds, issue.milestone);
|
||||
const issuePayload: any = {
|
||||
@@ -3331,106 +3419,119 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
});
|
||||
}
|
||||
|
||||
// Clone comments
|
||||
const comments = await octokit.paginate(
|
||||
octokit.rest.issues.listComments,
|
||||
{
|
||||
owner,
|
||||
repo,
|
||||
issue_number: issue.number,
|
||||
per_page: 100,
|
||||
},
|
||||
(res) => res.data
|
||||
);
|
||||
|
||||
// Ensure comments are applied in chronological order to preserve discussion flow
|
||||
const sortedComments = comments
|
||||
.slice()
|
||||
.sort(
|
||||
(a, b) =>
|
||||
new Date(a.created_at || 0).getTime() -
|
||||
new Date(b.created_at || 0).getTime()
|
||||
if (includeComments) {
|
||||
// Clone comments
|
||||
const comments = await octokit.paginate(
|
||||
octokit.rest.issues.listComments,
|
||||
{
|
||||
owner,
|
||||
repo,
|
||||
issue_number: issue.number,
|
||||
per_page: 100,
|
||||
},
|
||||
(res) => res.data
|
||||
);
|
||||
|
||||
// Process comments sequentially to preserve historical ordering
|
||||
if (sortedComments.length > 0) {
|
||||
const existingComments: any[] = [];
|
||||
let commentsPage = 1;
|
||||
const commentsPerPage = 100;
|
||||
while (true) {
|
||||
const existingCommentsRes = await httpGet(
|
||||
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${targetIssueNumber}/comments?page=${commentsPage}&limit=${commentsPerPage}`,
|
||||
{
|
||||
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
|
||||
}
|
||||
// Ensure comments are applied in chronological order to preserve discussion flow
|
||||
const sortedComments = comments
|
||||
.slice()
|
||||
.sort(
|
||||
(a, b) =>
|
||||
new Date(a.created_at || 0).getTime() -
|
||||
new Date(b.created_at || 0).getTime()
|
||||
);
|
||||
const pageComments = Array.isArray(existingCommentsRes.data)
|
||||
? existingCommentsRes.data
|
||||
: [];
|
||||
if (!pageComments.length) break;
|
||||
existingComments.push(...pageComments);
|
||||
// Use the Link header to decide whether more pages exist.
|
||||
// See note on the existing-issues pagination above; the
|
||||
// same Gitea behaviors (MAX_RESPONSE_ITEMS cap and
|
||||
// repeated-data on out-of-bound pages) apply here.
|
||||
const commentsLinkHeader =
|
||||
existingCommentsRes.headers.get("link") || "";
|
||||
if (!/\brel="next"/.test(commentsLinkHeader)) break;
|
||||
commentsPage += 1;
|
||||
}
|
||||
const mirroredCommentIds = new Set<number>();
|
||||
const existingCommentBodies = new Set<string>();
|
||||
for (const existingComment of existingComments) {
|
||||
const body = String(existingComment.body || "");
|
||||
if (body) existingCommentBodies.add(body);
|
||||
const marker = String(existingComment.body || "").match(
|
||||
/<!--\s*gh-comment-id:(\d+)\s*-->/i
|
||||
);
|
||||
if (marker?.[1]) {
|
||||
const parsed = Number.parseInt(marker[1], 10);
|
||||
if (Number.isFinite(parsed)) mirroredCommentIds.add(parsed);
|
||||
}
|
||||
}
|
||||
|
||||
await processWithRetry(
|
||||
sortedComments,
|
||||
async (comment) => {
|
||||
if (mirroredCommentIds.has(comment.id)) {
|
||||
return comment;
|
||||
}
|
||||
const commenter = comment.user?.login ?? "unknown";
|
||||
const commentDate = formatDateShort(comment.created_at);
|
||||
const commentHeader = `@${commenter} commented on GitHub${
|
||||
commentDate ? ` (${commentDate})` : ""
|
||||
}:`;
|
||||
const legacyBody = `${commentHeader}\n\n${comment.body ?? ""}`;
|
||||
const markedBody = `<!-- gh-comment-id:${comment.id} -->\n${legacyBody}`;
|
||||
if (existingCommentBodies.has(legacyBody) || existingCommentBodies.has(markedBody)) {
|
||||
return comment;
|
||||
}
|
||||
|
||||
await httpPost(
|
||||
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${targetIssueNumber}/comments`,
|
||||
{
|
||||
body: markedBody,
|
||||
},
|
||||
// Process comments sequentially to preserve historical ordering
|
||||
if (sortedComments.length > 0) {
|
||||
const existingComments: any[] = [];
|
||||
let commentsPage = 1;
|
||||
const commentsPerPage = 100;
|
||||
while (true) {
|
||||
const existingCommentsRes = await httpGet(
|
||||
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${targetIssueNumber}/comments?page=${commentsPage}&limit=${commentsPerPage}`,
|
||||
{
|
||||
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
|
||||
}
|
||||
);
|
||||
return comment;
|
||||
},
|
||||
{
|
||||
concurrencyLimit: 1,
|
||||
maxRetries: 2,
|
||||
retryDelay: 1000,
|
||||
onRetry: (_comment, error, attempt) => {
|
||||
console.log(
|
||||
`Retrying comment (attempt ${attempt}): ${error.message}`
|
||||
);
|
||||
},
|
||||
const pageComments = Array.isArray(existingCommentsRes.data)
|
||||
? existingCommentsRes.data
|
||||
: [];
|
||||
if (!pageComments.length) break;
|
||||
existingComments.push(...pageComments);
|
||||
// Use the Link header to decide whether more pages exist.
|
||||
// See note on the existing-issues pagination above; the
|
||||
// same Gitea behaviors (MAX_RESPONSE_ITEMS cap and
|
||||
// repeated-data on out-of-bound pages) apply here.
|
||||
const commentsLinkHeader =
|
||||
existingCommentsRes.headers.get("link") || "";
|
||||
if (!/\brel="next"/.test(commentsLinkHeader)) break;
|
||||
commentsPage += 1;
|
||||
}
|
||||
);
|
||||
const mirroredCommentIds = new Set<number>();
|
||||
const existingCommentBodies = new Set<string>();
|
||||
for (const existingComment of existingComments) {
|
||||
const body = String(existingComment.body || "");
|
||||
if (body) {
|
||||
existingCommentBodies.add(body);
|
||||
// When media is off, compare against the stripped form too so a
|
||||
// comment mirrored before the option existed is not re-created
|
||||
// as a duplicate with the same text and no media.
|
||||
if (!includeMedia) {
|
||||
existingCommentBodies.add(stripIssueMedia(body));
|
||||
}
|
||||
}
|
||||
const marker = String(existingComment.body || "").match(
|
||||
/<!--\s*gh-comment-id:(\d+)\s*-->/i
|
||||
);
|
||||
if (marker?.[1]) {
|
||||
const parsed = Number.parseInt(marker[1], 10);
|
||||
if (Number.isFinite(parsed)) mirroredCommentIds.add(parsed);
|
||||
}
|
||||
}
|
||||
|
||||
await processWithRetry(
|
||||
sortedComments,
|
||||
async (comment) => {
|
||||
if (mirroredCommentIds.has(comment.id)) {
|
||||
return comment;
|
||||
}
|
||||
const commenter = comment.user?.login ?? "unknown";
|
||||
const commentDate = formatDateShort(comment.created_at);
|
||||
const commentHeader = `@${commenter} commented on GitHub${
|
||||
commentDate ? ` (${commentDate})` : ""
|
||||
}:`;
|
||||
const legacyBody = `${commentHeader}\n\n${applyIssueMediaOption(comment.body, includeMedia)}`;
|
||||
const markedBody = `<!-- gh-comment-id:${comment.id} -->\n${legacyBody}`;
|
||||
if (
|
||||
existingCommentBodies.has(legacyBody) ||
|
||||
existingCommentBodies.has(markedBody)
|
||||
) {
|
||||
return comment;
|
||||
}
|
||||
|
||||
await httpPost(
|
||||
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${targetIssueNumber}/comments`,
|
||||
{
|
||||
body: markedBody,
|
||||
},
|
||||
{
|
||||
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
|
||||
}
|
||||
);
|
||||
return comment;
|
||||
},
|
||||
{
|
||||
concurrencyLimit: 1,
|
||||
maxRetries: 2,
|
||||
retryDelay: 1000,
|
||||
onRetry: (_comment, error, attempt) => {
|
||||
console.log(
|
||||
`Retrying comment (attempt ${attempt}): ${error.message}`
|
||||
);
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
completedNumbers.add(issue.number);
|
||||
@@ -3471,6 +3572,9 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
`Completed mirroring ${filteredIssues.length} issues for ${repository.fullName}` +
|
||||
(unchangedIssueCount > 0
|
||||
? ` (${unchangedIssueCount} unchanged since the last pass, no GitHub calls)`
|
||||
: "") +
|
||||
(filteredOutIssueCount > 0
|
||||
? ` (${filteredOutIssueCount} not ${issueState}, skipped)`
|
||||
: "")
|
||||
);
|
||||
|
||||
@@ -3807,6 +3911,90 @@ export async function reconcileReleaseAssets({
|
||||
return { uploaded, failed, skipped };
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete every attachment one release currently has on the destination.
|
||||
*
|
||||
* Used by the opt-in pruning in the retention pass, which removes what the
|
||||
* asset limit left behind: the uploads a release got before the limit was set
|
||||
* or lowered. Like `reconcileReleaseAssets` this reads before it writes, and a
|
||||
* listing that cannot be read deletes nothing, so a transient API error never
|
||||
* wipes a release that still had its assets.
|
||||
*
|
||||
* Returns what was removed so the caller can log it and keep the storage
|
||||
* totals honest. A release whose deletes all succeeded reports `failed: 0`.
|
||||
*/
|
||||
export async function pruneReleaseAttachments({
|
||||
baseUrl,
|
||||
token,
|
||||
repoOwner,
|
||||
repoName,
|
||||
giteaReleaseId,
|
||||
tagName,
|
||||
fetchImpl,
|
||||
}: {
|
||||
baseUrl: string;
|
||||
token: string;
|
||||
repoOwner: string;
|
||||
repoName: string;
|
||||
giteaReleaseId: number;
|
||||
tagName: string;
|
||||
/** Injected in tests; production uses the global fetch. */
|
||||
fetchImpl?: typeof fetch;
|
||||
}): Promise<{ removed: number; failed: number }> {
|
||||
const doFetch: typeof fetch = fetchImpl ?? fetch;
|
||||
const giteaAuth = { Authorization: `token ${token}` };
|
||||
const assetsUrl = `${baseUrl}/api/v1/repos/${repoOwner}/${repoName}/releases/${giteaReleaseId}/assets`;
|
||||
|
||||
let existingAssets: Array<{ id: number; name: string }>;
|
||||
try {
|
||||
const listResponse = await doFetch(assetsUrl, { headers: giteaAuth });
|
||||
if (!listResponse.ok) {
|
||||
throw new Error(`HTTP ${listResponse.status} ${listResponse.statusText}`);
|
||||
}
|
||||
const body = await listResponse.json();
|
||||
if (!Array.isArray(body)) {
|
||||
throw new Error("the assets endpoint did not return a list");
|
||||
}
|
||||
existingAssets = body;
|
||||
} catch (listError) {
|
||||
console.error(
|
||||
`[Releases] Could not list the attachments of ${tagName} to prune them: ${
|
||||
listError instanceof Error ? listError.message : String(listError)
|
||||
}. Deleted nothing; the next sync retries.`
|
||||
);
|
||||
return { removed: 0, failed: 0 };
|
||||
}
|
||||
|
||||
let removed = 0;
|
||||
let failed = 0;
|
||||
|
||||
for (const asset of existingAssets) {
|
||||
try {
|
||||
const deleteResponse = await doFetch(`${assetsUrl}/${asset.id}`, {
|
||||
method: "DELETE",
|
||||
headers: giteaAuth,
|
||||
});
|
||||
// 404 means it is already gone, which is the state we wanted.
|
||||
if (!deleteResponse.ok && deleteResponse.status !== 404) {
|
||||
throw new Error(`HTTP ${deleteResponse.status} ${deleteResponse.statusText}`);
|
||||
}
|
||||
removed++;
|
||||
console.log(
|
||||
`[Releases] Pruned attachment ${asset.name} (id ${asset.id}) from ${tagName} (past the release asset limit)`
|
||||
);
|
||||
} catch (deleteError) {
|
||||
failed++;
|
||||
console.error(
|
||||
`[Releases] Failed to prune attachment ${asset.name} (id ${asset.id}) from ${tagName}: ${
|
||||
deleteError instanceof Error ? deleteError.message : String(deleteError)
|
||||
}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return { removed, failed };
|
||||
}
|
||||
|
||||
/**
|
||||
* Where one release mirror pass gets its releases and its download credentials.
|
||||
*
|
||||
@@ -4093,13 +4281,19 @@ async function mirrorSourceReleasesToGiteaLocked({
|
||||
|
||||
// Which of those releases also get their assets (#311). Older releases
|
||||
// past this point are still created with notes and tag, just without the
|
||||
// uploads. Assets already in Gitea are never removed by this limit; it only
|
||||
// decides what a sync uploads.
|
||||
// uploads. The limit alone only decides what a sync uploads - attachments
|
||||
// that already reached Gitea stay put, unless the opt-in pruning below is
|
||||
// enabled and takes them away again.
|
||||
const releaseAssetLimit =
|
||||
releaseAssetLimitOverride !== undefined
|
||||
? (normalizeReleaseAssetLimit(releaseAssetLimitOverride) ?? null)
|
||||
: (await resolveOptions()).releaseAssetLimit;
|
||||
|
||||
// Opt-in retroactive pruning (see `pruneOldReleaseAssets` on the config).
|
||||
// Global only; it prunes by the ranks this repository's own asset limit
|
||||
// produces, so a per-repo asset limit of 3 prunes that repo's 4th release.
|
||||
const pruneOldReleaseAssets = (await resolveOptions()).pruneOldReleaseAssets;
|
||||
|
||||
// Which releases are candidates at all: every one, or only the ones the source
|
||||
// does not flag as a pre-release.
|
||||
const releaseType =
|
||||
@@ -4280,7 +4474,11 @@ async function mirrorSourceReleasesToGiteaLocked({
|
||||
totalAssetsFailed += assetResult.failed;
|
||||
} else if (release.assets && release.assets.length > 0) {
|
||||
console.log(
|
||||
`[Releases] Skipping ${release.assets.length} asset(s) for ${release.tag_name}: outside the newest ${releaseAssetLimit} release(s) that get assets (anything already uploaded is kept)`
|
||||
`[Releases] Skipping ${release.assets.length} asset(s) for ${release.tag_name}: outside the newest ${releaseAssetLimit} release(s) that get assets${
|
||||
pruneOldReleaseAssets
|
||||
? " (their existing attachments are pruned by the retention pass)"
|
||||
: " (anything already uploaded is kept)"
|
||||
}`
|
||||
);
|
||||
releasesPastAssetLimit++;
|
||||
}
|
||||
@@ -4514,6 +4712,55 @@ async function mirrorSourceReleasesToGiteaLocked({
|
||||
}
|
||||
}
|
||||
|
||||
// Opt-in cleanup of what the asset limit left behind. The limit gates
|
||||
// uploads, so releases that got their attachments before the limit was set
|
||||
// or lowered still carry them. Ranked by the same newest-first order the
|
||||
// uploads were gated on, so a destination release the source does not list
|
||||
// keeps its attachments - retention removes the release itself if it is
|
||||
// past the release limit.
|
||||
const prunedReleaseIds = new Set<number>();
|
||||
if (pruneOldReleaseAssets && releaseAssetLimit !== null) {
|
||||
const rankByTag = new Map(
|
||||
limitedReleases.map((rel, index) => [rel.tag_name, index])
|
||||
);
|
||||
const prunable = retainedReleases.filter((rel) => {
|
||||
if (deletedReleaseIds.has(rel.id)) return false;
|
||||
// The listing carries each release's attachments, so a release that
|
||||
// has none (one already pruned on an earlier sync) costs no request.
|
||||
if (Array.isArray(rel.assets) && rel.assets.length === 0) return false;
|
||||
const rank = rankByTag.get(rel.tag_name);
|
||||
return rank !== undefined && !releaseGetsAssets(rank, releaseAssetLimit);
|
||||
});
|
||||
|
||||
if (prunable.length > 0) {
|
||||
console.log(
|
||||
`[Releases] Pruning release assets: ${prunable.length} kept release(s) sit past the newest ${releaseAssetLimit} release(s) that get assets`
|
||||
);
|
||||
let prunedAssets = 0;
|
||||
for (const rel of prunable) {
|
||||
const pruneResult = await pruneReleaseAttachments({
|
||||
baseUrl: giteaBaseUrl,
|
||||
token: giteaToken,
|
||||
repoOwner,
|
||||
repoName,
|
||||
giteaReleaseId: rel.id,
|
||||
tagName: rel.tag_name,
|
||||
});
|
||||
prunedAssets += pruneResult.removed;
|
||||
// Only a release whose attachments actually went is worth zeroing out
|
||||
// in the storage summary below. A failed listing removed nothing, and
|
||||
// a partial failure left some behind; both keep the old totals, which
|
||||
// are closer to the truth than assuming they are gone.
|
||||
if (pruneResult.removed > 0 && pruneResult.failed === 0) {
|
||||
prunedReleaseIds.add(rel.id);
|
||||
}
|
||||
}
|
||||
console.log(
|
||||
`[Releases] Removed ${prunedAssets} attachment(s) from releases past the asset limit for ${repository.fullName}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// The listing already carries every asset's size, so the storage page
|
||||
// gets its release numbers without another request (#479). Releases whose
|
||||
// delete failed are still on the destination and still count.
|
||||
@@ -4521,7 +4768,11 @@ async function mirrorSourceReleasesToGiteaLocked({
|
||||
await recordReleaseAssetStorage(
|
||||
repository.id,
|
||||
summarizeReleaseAssets(
|
||||
allGiteaReleases.filter((rel) => !deletedReleaseIds.has(rel.id))
|
||||
allGiteaReleases
|
||||
.filter((rel) => !deletedReleaseIds.has(rel.id))
|
||||
.map((rel) =>
|
||||
prunedReleaseIds.has(rel.id) ? { ...rel, assets: [] } : rel
|
||||
)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
@@ -44,6 +44,7 @@ describe("sync cursor parsing", () => {
|
||||
state.syncCursors.issues = {
|
||||
lastPassStartedAt: "2026-09-20T10:00:00.000Z",
|
||||
lastFullPassStartedAt: "2026-09-18T10:00:00.000Z",
|
||||
issueState: "open",
|
||||
};
|
||||
state.syncCursors.pullRequests = {
|
||||
lastPassStartedAt: "2026-09-20T11:00:00.000Z",
|
||||
@@ -263,3 +264,42 @@ describe("unfinished pass progress (#449 follow-up)", () => {
|
||||
expect(incremental.since).toBe("2026-09-25T08:50:00.000Z");
|
||||
});
|
||||
});
|
||||
|
||||
describe("mirror sync request record (#493)", () => {
|
||||
test("the record survives a round trip, with and without warnedAt", () => {
|
||||
const state = createDefaultMetadataState();
|
||||
state.mirrorSyncRequest = {
|
||||
requestedAt: "2026-10-20T08:00:00.000Z",
|
||||
mirrorUpdatedBefore: "2026-10-20T07:55:12Z",
|
||||
};
|
||||
expect(
|
||||
parseRepositoryMetadataState(serializeRepositoryMetadataState(state)).mirrorSyncRequest
|
||||
).toEqual(state.mirrorSyncRequest);
|
||||
|
||||
state.mirrorSyncRequest.warnedAt = "2026-10-20T16:00:00.000Z";
|
||||
expect(
|
||||
parseRepositoryMetadataState(serializeRepositoryMetadataState(state)).mirrorSyncRequest
|
||||
).toEqual(state.mirrorSyncRequest);
|
||||
});
|
||||
|
||||
test("a row without the record, or with a malformed one, loads without it", () => {
|
||||
expect(parseRepositoryMetadataState({ components: {} }).mirrorSyncRequest).toBeUndefined();
|
||||
expect(
|
||||
parseRepositoryMetadataState({
|
||||
mirrorSyncRequest: { requestedAt: "2026-10-20T08:00:00.000Z", mirrorUpdatedBefore: 0 },
|
||||
}).mirrorSyncRequest
|
||||
).toBeUndefined();
|
||||
expect(
|
||||
parseRepositoryMetadataState({
|
||||
mirrorSyncRequest: {
|
||||
requestedAt: "2026-10-20T08:00:00.000Z",
|
||||
mirrorUpdatedBefore: "2026-10-20T07:55:12Z",
|
||||
warnedAt: "soon",
|
||||
},
|
||||
}).mirrorSyncRequest
|
||||
).toEqual({
|
||||
requestedAt: "2026-10-20T08:00:00.000Z",
|
||||
mirrorUpdatedBefore: "2026-10-20T07:55:12Z",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -29,6 +29,11 @@ export interface MetadataSyncCursor {
|
||||
lastPassStartedAt: string;
|
||||
/** Start of the last full pass that completed cleanly. */
|
||||
lastFullPassStartedAt: string;
|
||||
/**
|
||||
* Issue state filter the last completed issues pass ran with, when it was
|
||||
* not "all". Issues a filter left out are only found again by a full pass.
|
||||
*/
|
||||
issueState?: string;
|
||||
}
|
||||
|
||||
export interface MetadataSyncCursors {
|
||||
@@ -63,12 +68,29 @@ export interface MetadataPassProgressByKind {
|
||||
pullRequests?: MetadataPassProgress;
|
||||
}
|
||||
|
||||
/**
|
||||
* The oldest mirror sync request Gitea has not run yet (#493). Gitea
|
||||
* answers `mirror-sync` by queueing the pull and returns 200 before it
|
||||
* runs, so the only sign that a request was dropped or is stuck is that
|
||||
* the repository's `mirror_updated` never moves past the value read
|
||||
* before the request.
|
||||
*/
|
||||
export interface MirrorSyncRequestState {
|
||||
/** When the request was accepted, on this host's clock. */
|
||||
requestedAt: string;
|
||||
/** Gitea's `mirror_updated` just before the request, on Gitea's clock. */
|
||||
mirrorUpdatedBefore: string;
|
||||
/** Set once the activity log and notifications have reported this request. */
|
||||
warnedAt?: string;
|
||||
}
|
||||
|
||||
export interface RepositoryMetadataState {
|
||||
components: MetadataComponentsState;
|
||||
lastSyncedAt?: string;
|
||||
acknowledgedDeletions: AcknowledgedDeletion[];
|
||||
syncCursors: MetadataSyncCursors;
|
||||
passProgress: MetadataPassProgressByKind;
|
||||
mirrorSyncRequest?: MirrorSyncRequestState;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -115,14 +137,32 @@ function isValidTimestamp(value: unknown): value is string {
|
||||
|
||||
function parseSyncCursor(raw: unknown): MetadataSyncCursor | undefined {
|
||||
if (!raw || typeof raw !== "object") return undefined;
|
||||
const { lastPassStartedAt, lastFullPassStartedAt } = raw as {
|
||||
const { lastPassStartedAt, lastFullPassStartedAt, issueState } = raw as {
|
||||
lastPassStartedAt?: unknown;
|
||||
lastFullPassStartedAt?: unknown;
|
||||
issueState?: unknown;
|
||||
};
|
||||
if (!isValidTimestamp(lastPassStartedAt) || !isValidTimestamp(lastFullPassStartedAt)) {
|
||||
return undefined;
|
||||
}
|
||||
return { lastPassStartedAt, lastFullPassStartedAt };
|
||||
return typeof issueState === "string"
|
||||
? { lastPassStartedAt, lastFullPassStartedAt, issueState }
|
||||
: { lastPassStartedAt, lastFullPassStartedAt };
|
||||
}
|
||||
|
||||
function parseMirrorSyncRequest(raw: unknown): MirrorSyncRequestState | undefined {
|
||||
if (!raw || typeof raw !== "object") return undefined;
|
||||
const { requestedAt, mirrorUpdatedBefore, warnedAt } = raw as {
|
||||
requestedAt?: unknown;
|
||||
mirrorUpdatedBefore?: unknown;
|
||||
warnedAt?: unknown;
|
||||
};
|
||||
if (!isValidTimestamp(requestedAt) || !isValidTimestamp(mirrorUpdatedBefore)) {
|
||||
return undefined;
|
||||
}
|
||||
return isValidTimestamp(warnedAt)
|
||||
? { requestedAt, mirrorUpdatedBefore, warnedAt }
|
||||
: { requestedAt, mirrorUpdatedBefore };
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -381,6 +421,9 @@ export function parseRepositoryMetadataState(
|
||||
if (pullRequests) base.passProgress.pullRequests = pullRequests;
|
||||
}
|
||||
|
||||
const mirrorSyncRequest = parseMirrorSyncRequest(parsed.mirrorSyncRequest);
|
||||
if (mirrorSyncRequest) base.mirrorSyncRequest = mirrorSyncRequest;
|
||||
|
||||
return base;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import {
|
||||
assessMirrorSyncRequest,
|
||||
formatWaitedTime,
|
||||
MIRROR_SYNC_STALL_THRESHOLD_MS,
|
||||
} from "./mirror-sync-health";
|
||||
|
||||
const HOUR = 60 * 60 * 1000;
|
||||
|
||||
describe("assessMirrorSyncRequest", () => {
|
||||
const requestedAt = "2026-10-20T08:00:00.000Z";
|
||||
const mirrorUpdatedBefore = "2026-10-20T07:55:12Z";
|
||||
const previous = { requestedAt, mirrorUpdatedBefore };
|
||||
|
||||
test("records the first request with the mirror_updated read before it", () => {
|
||||
const now = new Date("2026-10-20T08:00:00.000Z");
|
||||
const result = assessMirrorSyncRequest({ mirrorUpdated: mirrorUpdatedBefore, now });
|
||||
|
||||
expect(result).toEqual({
|
||||
next: { requestedAt: now.toISOString(), mirrorUpdatedBefore },
|
||||
});
|
||||
});
|
||||
|
||||
test("starts a new record once Gitea ran the previous request", () => {
|
||||
const now = new Date("2026-10-20T16:00:00.000Z");
|
||||
const result = assessMirrorSyncRequest({
|
||||
previous,
|
||||
mirrorUpdated: "2026-10-20T08:00:03Z",
|
||||
now,
|
||||
});
|
||||
|
||||
expect(result.stalled).toBeUndefined();
|
||||
expect(result.next).toEqual({
|
||||
requestedAt: now.toISOString(),
|
||||
mirrorUpdatedBefore: "2026-10-20T08:00:03Z",
|
||||
});
|
||||
});
|
||||
|
||||
test("keeps a waiting request without warning inside the threshold", () => {
|
||||
const now = new Date(Date.parse(requestedAt) + MIRROR_SYNC_STALL_THRESHOLD_MS - 1);
|
||||
const result = assessMirrorSyncRequest({ previous, mirrorUpdated: mirrorUpdatedBefore, now });
|
||||
|
||||
expect(result).toEqual({ next: previous });
|
||||
});
|
||||
|
||||
test("reports a request Gitea has not run past the threshold", () => {
|
||||
const now = new Date(Date.parse(requestedAt) + 8 * HOUR);
|
||||
const result = assessMirrorSyncRequest({ previous, mirrorUpdated: mirrorUpdatedBefore, now });
|
||||
|
||||
expect(result.stalled).toEqual({
|
||||
requestedAt,
|
||||
mirrorUpdated: mirrorUpdatedBefore,
|
||||
waitedMs: 8 * HOUR,
|
||||
});
|
||||
expect(result.next).toEqual({ ...previous, warnedAt: now.toISOString() });
|
||||
});
|
||||
|
||||
test("keeps the age of the oldest waiting request across frequent syncs", () => {
|
||||
// Syncs every 20 minutes: each one finds the same request still waiting
|
||||
// and keeps it, so the third hour reports instead of resetting forever.
|
||||
let record = previous;
|
||||
let reported = 0;
|
||||
for (let minutes = 20; minutes <= 180; minutes += 20) {
|
||||
const result = assessMirrorSyncRequest({
|
||||
previous: record,
|
||||
mirrorUpdated: mirrorUpdatedBefore,
|
||||
now: new Date(Date.parse(requestedAt) + minutes * 60_000),
|
||||
});
|
||||
if (result.stalled) reported++;
|
||||
record = result.next!;
|
||||
expect(record.requestedAt).toBe(requestedAt);
|
||||
}
|
||||
|
||||
expect(reported).toBe(1);
|
||||
});
|
||||
|
||||
test("reports each waiting request once", () => {
|
||||
const warned = { ...previous, warnedAt: "2026-10-20T16:00:00.000Z" };
|
||||
const result = assessMirrorSyncRequest({
|
||||
previous: warned,
|
||||
mirrorUpdated: mirrorUpdatedBefore,
|
||||
now: new Date(Date.parse(requestedAt) + 48 * HOUR),
|
||||
});
|
||||
|
||||
expect(result).toEqual({ next: warned });
|
||||
});
|
||||
|
||||
test("clears the record when Gitea does not report mirror_updated", () => {
|
||||
expect(assessMirrorSyncRequest({ previous, mirrorUpdated: undefined })).toEqual({});
|
||||
expect(assessMirrorSyncRequest({ previous, mirrorUpdated: "not a date" })).toEqual({});
|
||||
});
|
||||
});
|
||||
|
||||
describe("formatWaitedTime", () => {
|
||||
test("rounds down to minutes, hours or days", () => {
|
||||
expect(formatWaitedTime(10_000)).toBe("1 minute");
|
||||
expect(formatWaitedTime(45 * 60_000)).toBe("45 minutes");
|
||||
expect(formatWaitedTime(HOUR)).toBe("1 hour");
|
||||
expect(formatWaitedTime(47 * HOUR + 59 * 60_000)).toBe("47 hours");
|
||||
expect(formatWaitedTime(7 * 24 * HOUR)).toBe("7 days");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,79 @@
|
||||
import type { MirrorSyncRequestState } from "./metadata-state";
|
||||
|
||||
/**
|
||||
* How long a mirror sync request may wait before it counts as stalled.
|
||||
* A healthy Gitea runs a queued pull within seconds; this leaves room for
|
||||
* a long queue after a large import or a slow fetch of a big repository.
|
||||
*/
|
||||
export const MIRROR_SYNC_STALL_THRESHOLD_MS = 60 * 60 * 1000;
|
||||
|
||||
export interface MirrorSyncAssessment {
|
||||
/** What to store after this sync's request; undefined clears the record. */
|
||||
next?: MirrorSyncRequestState;
|
||||
/** Set when a request has waited past the threshold and was not reported yet. */
|
||||
stalled?: {
|
||||
requestedAt: string;
|
||||
mirrorUpdated: string;
|
||||
waitedMs: number;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare Gitea's `mirror_updated` with the value read before the last
|
||||
* request (#493). Gitea moves it on every mirror run it processes, failed
|
||||
* runs included, so an unchanged value means the request never ran. Both
|
||||
* values come from Gitea's clock, which keeps clock skew between the two
|
||||
* hosts out of the comparison.
|
||||
*
|
||||
* A request that is still waiting is kept as is, so repeated syncs do not
|
||||
* reset its age, and it is reported once.
|
||||
*/
|
||||
export function assessMirrorSyncRequest({
|
||||
previous,
|
||||
mirrorUpdated,
|
||||
now = new Date(),
|
||||
}: {
|
||||
previous?: MirrorSyncRequestState;
|
||||
/** `mirror_updated` from the repository GET that came before this request. */
|
||||
mirrorUpdated?: string | null;
|
||||
now?: Date;
|
||||
}): MirrorSyncAssessment {
|
||||
if (typeof mirrorUpdated !== "string" || !Number.isFinite(Date.parse(mirrorUpdated))) {
|
||||
return {};
|
||||
}
|
||||
|
||||
const waiting =
|
||||
previous !== undefined &&
|
||||
Date.parse(mirrorUpdated) <= Date.parse(previous.mirrorUpdatedBefore);
|
||||
|
||||
if (!waiting) {
|
||||
return {
|
||||
next: { requestedAt: now.toISOString(), mirrorUpdatedBefore: mirrorUpdated },
|
||||
};
|
||||
}
|
||||
|
||||
const waitedMs = now.getTime() - Date.parse(previous.requestedAt);
|
||||
if (previous.warnedAt || waitedMs < MIRROR_SYNC_STALL_THRESHOLD_MS) {
|
||||
return { next: previous };
|
||||
}
|
||||
|
||||
return {
|
||||
next: { ...previous, warnedAt: now.toISOString() },
|
||||
stalled: { requestedAt: previous.requestedAt, mirrorUpdated, waitedMs },
|
||||
};
|
||||
}
|
||||
|
||||
/** "45 minutes", "3 hours", "6 days": rounded down, for alert text. */
|
||||
export function formatWaitedTime(ms: number): string {
|
||||
const minutes = Math.max(1, Math.floor(ms / 60_000));
|
||||
if (minutes < 60) return `${minutes} minute${minutes === 1 ? "" : "s"}`;
|
||||
const hours = Math.floor(minutes / 60);
|
||||
if (hours < 48) return `${hours} hour${hours === 1 ? "" : "s"}`;
|
||||
const days = Math.floor(hours / 24);
|
||||
return `${days} days`;
|
||||
}
|
||||
|
||||
/** Where to look in Gitea or Forgejo, shared by the activity row and the notification. */
|
||||
export const STUCK_MIRROR_QUEUE_HINT =
|
||||
"Gitea accepts sync requests before running them, so this usually means its mirror queue is stuck." +
|
||||
' In Gitea, open Site Administration > Monitoring > Queues > mirror; "Remove all" clears a stuck queue.';
|
||||
@@ -18,7 +18,7 @@ mock.module("@/lib/utils/encryption", () => ({
|
||||
}));
|
||||
|
||||
// Import after mocks are set up — db is already mocked via setup.bun.ts
|
||||
import { sendNotification, testNotification, buildNewRepositoriesNotificationEvent, buildAllReleasesRemovedNotificationEvent, triggerAllReleasesRemovedNotification, isNewRepositoryGroupEnabled, MAX_LISTED_NEW_REPOSITORIES, STARRED_GROUP_LABEL } from "./notification-service";
|
||||
import { sendNotification, testNotification, buildNewRepositoriesNotificationEvent, buildAllReleasesRemovedNotificationEvent, triggerAllReleasesRemovedNotification, buildMirrorSyncStalledNotificationEvent, claimMirrorSyncStalledNotification, resetMirrorSyncStalledNotifications, triggerMirrorSyncStalledNotification, MIRROR_SYNC_STALLED_NOTIFICATION_INTERVAL_MS, isNewRepositoryGroupEnabled, MAX_LISTED_NEW_REPOSITORIES, STARRED_GROUP_LABEL } from "./notification-service";
|
||||
import type { NotificationConfig } from "@/types/config";
|
||||
|
||||
describe("sendNotification", () => {
|
||||
@@ -559,3 +559,57 @@ describe("triggerAllReleasesRemovedNotification", () => {
|
||||
expect(mockFetch).toHaveBeenCalledTimes(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildMirrorSyncStalledNotificationEvent", () => {
|
||||
test("names the destination, how long it waited and where to look in Gitea", () => {
|
||||
const event = buildMirrorSyncStalledNotificationEvent({
|
||||
repositoryName: "demo",
|
||||
destination: "mirrors/demo",
|
||||
waited: "8 hours",
|
||||
mirrorUpdated: "2026-10-20T07:55:12Z",
|
||||
});
|
||||
|
||||
expect(event.title).toBe("Mirror Sync Stalled: demo");
|
||||
expect(event.type).toBe("sync_error");
|
||||
expect(event.message).toContain("sync requested for mirrors/demo 8 hours ago");
|
||||
expect(event.message).toContain("still 2026-10-20T07:55:12Z");
|
||||
expect(event.message).toContain("Site Administration > Monitoring > Queues > mirror");
|
||||
});
|
||||
});
|
||||
|
||||
describe("claimMirrorSyncStalledNotification", () => {
|
||||
beforeEach(() => resetMirrorSyncStalledNotifications());
|
||||
|
||||
test("allows one alert per user per window", () => {
|
||||
const start = Date.parse("2026-10-20T08:00:00.000Z");
|
||||
|
||||
expect(claimMirrorSyncStalledNotification("user-1", start)).toBe(true);
|
||||
expect(claimMirrorSyncStalledNotification("user-1", start + 1000)).toBe(false);
|
||||
expect(claimMirrorSyncStalledNotification("user-2", start + 1000)).toBe(true);
|
||||
expect(
|
||||
claimMirrorSyncStalledNotification("user-1", start + MIRROR_SYNC_STALLED_NOTIFICATION_INTERVAL_MS - 1)
|
||||
).toBe(false);
|
||||
expect(
|
||||
claimMirrorSyncStalledNotification("user-1", start + MIRROR_SYNC_STALLED_NOTIFICATION_INTERVAL_MS)
|
||||
).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("triggerMirrorSyncStalledNotification", () => {
|
||||
beforeEach(() => resetMirrorSyncStalledNotifications());
|
||||
|
||||
test("never throws and keeps the alert slot when the user has no notification config", async () => {
|
||||
await triggerMirrorSyncStalledNotification({
|
||||
userId: "user-1",
|
||||
repositoryName: "demo",
|
||||
destination: "mirrors/demo",
|
||||
waited: "8 hours",
|
||||
mirrorUpdated: "2026-10-20T07:55:12Z",
|
||||
});
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledTimes(0);
|
||||
// Nothing was sent, so the slot is still free for a user who turns
|
||||
// notifications on later.
|
||||
expect(claimMirrorSyncStalledNotification("user-1")).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -7,6 +7,7 @@ import { sendWebhookNotification } from "./providers/webhook";
|
||||
import { db, configs } from "@/lib/db";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { decrypt } from "@/lib/utils/encryption";
|
||||
import { STUCK_MIRROR_QUEUE_HINT } from "./mirror-sync-health";
|
||||
|
||||
function sanitizeTestNotificationError(error: unknown): string {
|
||||
if (!(error instanceof Error)) {
|
||||
@@ -429,3 +430,91 @@ export async function triggerAllReleasesRemovedNotification(args: {
|
||||
console.error("[NotificationService] All-releases-removed notification failed:", error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The "Gitea never ran the requested sync" event (#493).
|
||||
*
|
||||
* A stuck mirror queue holds back every repository at once, so the text
|
||||
* points at the queue rather than the one repository that happened to be
|
||||
* checked first.
|
||||
*/
|
||||
export function buildMirrorSyncStalledNotificationEvent({
|
||||
repositoryName,
|
||||
destination,
|
||||
waited,
|
||||
mirrorUpdated,
|
||||
}: {
|
||||
repositoryName: string;
|
||||
destination: string;
|
||||
/** How long ago the request was accepted, already formatted. */
|
||||
waited: string;
|
||||
mirrorUpdated: string;
|
||||
}): NotificationEvent {
|
||||
return {
|
||||
title: `Mirror Sync Stalled: ${repositoryName}`,
|
||||
message: [
|
||||
`Gitea has not run the sync requested for ${destination} ${waited} ago; its last mirror run is still ${mirrorUpdated}.`,
|
||||
` ${STUCK_MIRROR_QUEUE_HINT}`,
|
||||
" Other repositories held back by the same queue are listed in the Activity Log.",
|
||||
].join(""),
|
||||
type: "sync_error",
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* At most one stalled-sync alert per user in this window. Every repository
|
||||
* reports its own stall in the activity log; the push channels only need to
|
||||
* hear that something is wrong.
|
||||
*/
|
||||
export const MIRROR_SYNC_STALLED_NOTIFICATION_INTERVAL_MS = 6 * 60 * 60 * 1000;
|
||||
|
||||
const lastMirrorSyncStalledNotification = new Map<string, number>();
|
||||
|
||||
/**
|
||||
* Takes this user's alert slot when the last alert is outside the window.
|
||||
* Kept in memory: a restart allows one more alert, which is fine.
|
||||
*/
|
||||
export function claimMirrorSyncStalledNotification(userId: string, now: number = Date.now()): boolean {
|
||||
const last = lastMirrorSyncStalledNotification.get(userId);
|
||||
if (last !== undefined && now - last < MIRROR_SYNC_STALLED_NOTIFICATION_INTERVAL_MS) {
|
||||
return false;
|
||||
}
|
||||
lastMirrorSyncStalledNotification.set(userId, now);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Test hook: forget when each user was last alerted. */
|
||||
export function resetMirrorSyncStalledNotifications(): void {
|
||||
lastMirrorSyncStalledNotification.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Warns that Gitea accepted a sync request but never ran it. The existing
|
||||
* "Sync errors" switch gates the alert.
|
||||
*
|
||||
* Called fire-and-forget from the sync path, so it NEVER throws.
|
||||
*/
|
||||
export async function triggerMirrorSyncStalledNotification(args: {
|
||||
userId: string;
|
||||
repositoryName: string;
|
||||
destination: string;
|
||||
waited: string;
|
||||
mirrorUpdated: string;
|
||||
}): Promise<void> {
|
||||
try {
|
||||
const notificationConfig = await loadNotificationConfig(args.userId);
|
||||
|
||||
if (!notificationConfig?.enabled || !notificationConfig.notifyOnSyncError) {
|
||||
return;
|
||||
}
|
||||
if (!claimMirrorSyncStalledNotification(args.userId)) {
|
||||
return;
|
||||
}
|
||||
|
||||
const event = buildMirrorSyncStalledNotificationEvent(args);
|
||||
|
||||
await sendNotification(decryptNotificationConfig(notificationConfig), event);
|
||||
} catch (error) {
|
||||
console.error("[NotificationService] Mirror sync stalled notification failed:", error);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { compareNames } from "./compare-names";
|
||||
|
||||
describe("compareNames", () => {
|
||||
test("ignores case", () => {
|
||||
expect(["Zebra", "apple", "Mango"].sort(compareNames)).toEqual([
|
||||
"apple",
|
||||
"Mango",
|
||||
"Zebra",
|
||||
]);
|
||||
});
|
||||
|
||||
test("sorts accented names with their base letter", () => {
|
||||
expect(["Zeta", "Ärger", "Bravo"].sort(compareNames)).toEqual([
|
||||
"Ärger",
|
||||
"Bravo",
|
||||
"Zeta",
|
||||
]);
|
||||
});
|
||||
|
||||
test("compares numbers by value", () => {
|
||||
expect(["repo-10", "repo-2", "repo-1"].sort(compareNames)).toEqual([
|
||||
"repo-1",
|
||||
"repo-2",
|
||||
"repo-10",
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,13 @@
|
||||
/**
|
||||
* Order for owner, organization and repository names in filter lists: case
|
||||
* and accent insensitive, with numbers compared by value. A plain `.sort()`
|
||||
* compares code units, so "Zebra" lands before "apple" and umlauts sort last.
|
||||
*/
|
||||
const nameCollator = new Intl.Collator(undefined, {
|
||||
sensitivity: "base",
|
||||
numeric: true,
|
||||
});
|
||||
|
||||
export function compareNames(a: string, b: string): number {
|
||||
return nameCollator.compare(a, b);
|
||||
}
|
||||
@@ -41,6 +41,9 @@ function buildMinimalUiConfigs(overrides: { includeCollaboratorRepos?: boolean }
|
||||
milestones: false,
|
||||
wiki: false,
|
||||
},
|
||||
issueState: "all",
|
||||
issueComments: true,
|
||||
issueMedia: true,
|
||||
};
|
||||
const advancedOptions: AdvancedOptions = {
|
||||
skipForks: false,
|
||||
|
||||
@@ -17,7 +17,14 @@ import { normalizeDestinationProviderKind } from "@/lib/destination-kinds";
|
||||
import { z } from "zod";
|
||||
import { githubConfigSchema, giteaConfigSchema, scheduleConfigSchema, cleanupConfigSchema } from "@/lib/db/schema";
|
||||
import { parseInterval } from "@/lib/utils/duration-parser";
|
||||
import { normalizeReleaseAssetLimit, normalizeReleaseTagPatterns, normalizeReleaseType, DEFAULT_RELEASE_TYPE } from "@/lib/utils/mirror-overrides";
|
||||
import {
|
||||
DEFAULT_ISSUE_STATE,
|
||||
normalizeIssueState,
|
||||
normalizeReleaseAssetLimit,
|
||||
normalizeReleaseTagPatterns,
|
||||
normalizeReleaseType,
|
||||
DEFAULT_RELEASE_TYPE,
|
||||
} from "@/lib/utils/mirror-overrides";
|
||||
import { buildClockCronExpression, normalizeTimezone, parseClockCronExpression } from "@/lib/utils/schedule-utils";
|
||||
|
||||
// Use the actual database schema types
|
||||
@@ -171,6 +178,9 @@ export function mapUiToDbConfig(
|
||||
// null is a real value here ("assets for every mirrored release"), so
|
||||
// it is stored as such rather than dropped.
|
||||
releaseAssetLimit: normalizeReleaseAssetLimit(mirrorOptions.releaseAssetLimit) ?? null,
|
||||
// Opt-in pruning: only ever a plain boolean, so an absent switch stores the
|
||||
// safe default (never delete what the destination already has).
|
||||
pruneOldReleaseAssets: mirrorOptions.pruneOldReleaseAssets === true,
|
||||
// An unusable value falls back to "all" rather than storing a filter that
|
||||
// would hide releases from the user.
|
||||
releaseType: normalizeReleaseType(mirrorOptions.releaseType) ?? DEFAULT_RELEASE_TYPE,
|
||||
@@ -180,6 +190,9 @@ export function mapUiToDbConfig(
|
||||
releaseExcludeTags: normalizeReleaseTagPatterns(mirrorOptions.releaseExcludeTags) ?? [],
|
||||
mirrorMetadata: mirrorOptions.mirrorMetadata,
|
||||
mirrorIssues: mirrorOptions.mirrorMetadata && mirrorOptions.metadataComponents.issues,
|
||||
mirrorIssueState: normalizeIssueState(mirrorOptions.issueState) ?? DEFAULT_ISSUE_STATE,
|
||||
mirrorIssueComments: mirrorOptions.issueComments,
|
||||
mirrorIssueMedia: mirrorOptions.issueMedia,
|
||||
mirrorPullRequests: mirrorOptions.mirrorMetadata && mirrorOptions.metadataComponents.pullRequests,
|
||||
mirrorLabels: mirrorOptions.mirrorMetadata && mirrorOptions.metadataComponents.labels,
|
||||
mirrorMilestones: mirrorOptions.mirrorMetadata && mirrorOptions.metadataComponents.milestones,
|
||||
@@ -251,6 +264,7 @@ export function mapDbToUiConfig(dbConfig: any): {
|
||||
mirrorReleases: dbConfig.giteaConfig?.mirrorReleases || false,
|
||||
releaseLimit: dbConfig.giteaConfig?.releaseLimit || 10,
|
||||
releaseAssetLimit: normalizeReleaseAssetLimit(dbConfig.giteaConfig?.releaseAssetLimit) ?? null,
|
||||
pruneOldReleaseAssets: dbConfig.giteaConfig?.pruneOldReleaseAssets === true,
|
||||
releaseType: normalizeReleaseType(dbConfig.giteaConfig?.releaseType) ?? DEFAULT_RELEASE_TYPE,
|
||||
releaseIncludeTags: normalizeReleaseTagPatterns(dbConfig.giteaConfig?.releaseIncludeTags) ?? [],
|
||||
releaseExcludeTags: normalizeReleaseTagPatterns(dbConfig.giteaConfig?.releaseExcludeTags) ?? [],
|
||||
@@ -263,6 +277,10 @@ export function mapDbToUiConfig(dbConfig: any): {
|
||||
milestones: dbConfig.giteaConfig?.mirrorMilestones || false,
|
||||
wiki: dbConfig.giteaConfig?.wiki || false,
|
||||
},
|
||||
issueState:
|
||||
normalizeIssueState(dbConfig.giteaConfig?.mirrorIssueState) ?? DEFAULT_ISSUE_STATE,
|
||||
issueComments: dbConfig.giteaConfig?.mirrorIssueComments ?? true,
|
||||
issueMedia: dbConfig.giteaConfig?.mirrorIssueMedia ?? true,
|
||||
};
|
||||
|
||||
// Map advanced options
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { applyIssueMediaOption, stripIssueMedia } from "./issue-media";
|
||||
|
||||
describe("stripIssueMedia", () => {
|
||||
test("removes Markdown images but keeps the surrounding text", () => {
|
||||
const body = "Before\n\n\n\nAfter";
|
||||
|
||||
expect(stripIssueMedia(body)).toBe("Before\n\n[media omitted]\n\nAfter");
|
||||
});
|
||||
|
||||
test("removes images nested inside links", () => {
|
||||
const body = "[](https://example.com)";
|
||||
|
||||
expect(stripIssueMedia(body)).toBe("[media omitted]");
|
||||
});
|
||||
|
||||
test("removes HTML media tags", () => {
|
||||
const body = 'Text\n<img src="https://example.com/a.png" />\n<video controls><source src="a.mp4"></video>\nEnd';
|
||||
|
||||
const result = stripIssueMedia(body);
|
||||
expect(result).toContain("Text");
|
||||
expect(result).toContain("End");
|
||||
expect(result).not.toContain("<img");
|
||||
expect(result).not.toContain("<video");
|
||||
});
|
||||
|
||||
test("removes GitHub attachment links", () => {
|
||||
const body = "See https://github.com/user-attachments/assets/abc123 and https://user-images.githubusercontent.com/2/b.png";
|
||||
|
||||
expect(stripIssueMedia(body)).toBe("See [media omitted] and [media omitted]");
|
||||
});
|
||||
|
||||
test("removes attached files, on the current and the older upload paths", () => {
|
||||
const body =
|
||||
"Logs: [debug.log](https://github.com/user-attachments/files/123/debug.log) and [dump.zip](https://github.com/acme/demo/files/456/dump.zip)\nVideo: https://github.com/acme/demo/assets/789/0d1e2f3a-aaaa-bbbb-cccc-1234567890ab";
|
||||
|
||||
expect(stripIssueMedia(body)).toBe(
|
||||
"Logs: [media omitted] and [media omitted]\nVideo: [media omitted]"
|
||||
);
|
||||
});
|
||||
|
||||
test("keeps links to repository content on other GitHub hosts", () => {
|
||||
const body =
|
||||
"Run https://raw.githubusercontent.com/acme/demo/main/install.sh, see [the config](https://raw.githubusercontent.com/acme/demo/main/config.yml) and https://gist.githubusercontent.com/u/abc/raw/log.txt";
|
||||
|
||||
expect(stripIssueMedia(body)).toBe(body);
|
||||
});
|
||||
|
||||
test("keeps tilde fenced code blocks", () => {
|
||||
const body = "~~~md\n\n~~~";
|
||||
|
||||
expect(stripIssueMedia(body)).toBe(body);
|
||||
});
|
||||
|
||||
test("keeps normal links and code blocks", () => {
|
||||
const body = "See [the docs](https://example.com/docs).\n\n```\n\n```";
|
||||
|
||||
expect(stripIssueMedia(body)).toBe(body);
|
||||
});
|
||||
|
||||
test("returns an empty string for missing bodies", () => {
|
||||
expect(stripIssueMedia(null)).toBe("");
|
||||
expect(stripIssueMedia(undefined)).toBe("");
|
||||
});
|
||||
});
|
||||
|
||||
describe("applyIssueMediaOption", () => {
|
||||
test("leaves the body untouched when media is enabled", () => {
|
||||
const body = "";
|
||||
expect(applyIssueMediaOption(body, true)).toBe(body);
|
||||
});
|
||||
|
||||
test("strips media when the option is disabled", () => {
|
||||
const body = "";
|
||||
expect(applyIssueMediaOption(body, false)).toBe("[media omitted]");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,90 @@
|
||||
const MEDIA_PLACEHOLDER = "[media omitted]";
|
||||
|
||||
/**
|
||||
* Where GitHub keeps what people upload to issues and comments: images,
|
||||
* videos and attached files, on the current hosts and the older per-repository
|
||||
* paths. Other GitHub hosts such as raw.githubusercontent.com serve repository
|
||||
* content, not uploads, and are left alone.
|
||||
*/
|
||||
const GITHUB_UPLOAD_URL =
|
||||
String.raw`https?:\/\/(?:(?:private-)?user-images\.githubusercontent\.com\/|github\.com\/user-attachments\/(?:assets|files)\/|github\.com\/[\w.-]+\/[\w.-]+\/(?:assets|files)\/\d+\/)`;
|
||||
|
||||
const GITHUB_MEDIA_URL = new RegExp(String.raw`${GITHUB_UPLOAD_URL}[^\s)>\]]+`, "gi");
|
||||
|
||||
/**
|
||||
* Markdown links to an upload. This runs before the bare URL pass so
|
||||
* `[screenshot](https://user-images...)` becomes one placeholder instead of a
|
||||
* broken empty link.
|
||||
*/
|
||||
const MARKDOWN_MEDIA_LINK = new RegExp(
|
||||
String.raw`\[([^\]]*)\]\(${GITHUB_UPLOAD_URL}[^\s)]+(?:\s+"[^"]*")?\)`,
|
||||
"gi"
|
||||
);
|
||||
|
||||
/** Inline Markdown images, with optional title. */
|
||||
const MARKDOWN_IMAGE = /!\[[^\]]*\]\([^)]*\)/g;
|
||||
|
||||
/** HTML media blocks/tags that can be embedded in Markdown. */
|
||||
const HTML_MEDIA = [
|
||||
/<picture\b[^>]*>[\s\S]*?<\/picture>/gi,
|
||||
/<video\b[^>]*>[\s\S]*?<\/video>/gi,
|
||||
/<audio\b[^>]*>[\s\S]*?<\/audio>/gi,
|
||||
/<iframe\b[^>]*>[\s\S]*?<\/iframe>/gi,
|
||||
/<(?:img|source)\b[^>]*\/?>/gi,
|
||||
];
|
||||
|
||||
/**
|
||||
* Remove images, videos and attachment links from an issue body or comment.
|
||||
*
|
||||
* With the "issue media" option off we still want the text of the issue, so
|
||||
* this drops only the media references and leaves everything else intact. It
|
||||
* does not download anything; it just prevents the mirrored copy from pointing
|
||||
* at GitHub-hosted media that would be loaded every time the issue is opened.
|
||||
*/
|
||||
export function stripIssueMedia(value: string | null | undefined): string {
|
||||
if (!value) return "";
|
||||
|
||||
// Code blocks may document Markdown/HTML media syntax. Protect them so the
|
||||
// cleanup only touches content the author actually rendered as media.
|
||||
const codeBlocks: string[] = [];
|
||||
let result = value.replace(/(```|~~~)[\s\S]*?\1|`[^`\n]*`/g, (block) => {
|
||||
const index = codeBlocks.push(block) - 1;
|
||||
return `\u0000${index}\u0000`;
|
||||
});
|
||||
|
||||
// Images first: their target URL is often a media host, and running the
|
||||
// link pass first would leave the leading `![...]` behind.
|
||||
result = result.replace(MARKDOWN_IMAGE, MEDIA_PLACEHOLDER);
|
||||
result = result.replace(MARKDOWN_MEDIA_LINK, MEDIA_PLACEHOLDER);
|
||||
for (const pattern of HTML_MEDIA) {
|
||||
result = result.replace(pattern, MEDIA_PLACEHOLDER);
|
||||
}
|
||||
result = result.replace(GITHUB_MEDIA_URL, MEDIA_PLACEHOLDER);
|
||||
|
||||
// An image wrapped in a link leaves either `[](url)` or a link whose text is
|
||||
// just the placeholder behind; collapse both so the body does not keep a
|
||||
// meaningless link.
|
||||
result = result.replace(
|
||||
/\[\[media omitted\]\]\([^)]*\)/g,
|
||||
MEDIA_PLACEHOLDER
|
||||
);
|
||||
result = result.replace(/\[\]\([^)]*\)/g, MEDIA_PLACEHOLDER);
|
||||
|
||||
// Removing block-level media leaves runs of blank lines behind. Keep the
|
||||
// author's paragraph spacing but avoid growing the body with empty space.
|
||||
result = result.replace(/\n{3,}/g, "\n\n");
|
||||
|
||||
result = result.replace(/\u0000(\d+)\u0000/g, (_match, index: string) => {
|
||||
return codeBlocks[Number.parseInt(index, 10)] ?? "";
|
||||
});
|
||||
|
||||
return result.trim();
|
||||
}
|
||||
|
||||
/** Apply the media option without changing bodies when media is enabled. */
|
||||
export function applyIssueMediaOption(
|
||||
value: string | null | undefined,
|
||||
includeMedia: boolean
|
||||
): string {
|
||||
return includeMedia ? value ?? "" : stripIssueMedia(value);
|
||||
}
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
MIRROR_OVERRIDE_LIMIT_KEYS,
|
||||
MIRROR_OVERRIDE_PATTERN_KEYS,
|
||||
mirrorOptionsToFlags,
|
||||
normalizeIssueState,
|
||||
normalizeMirrorOverrides,
|
||||
normalizeReleaseAssetLimit,
|
||||
normalizeReleaseLimit,
|
||||
@@ -446,6 +447,8 @@ describe("UI_MIRROR_OVERRIDE_KEYS", () => {
|
||||
"lfs",
|
||||
"wiki",
|
||||
"mirrorIssues",
|
||||
"mirrorIssueComments",
|
||||
"mirrorIssueMedia",
|
||||
"mirrorPullRequests",
|
||||
"mirrorReleases",
|
||||
"mirrorLabels",
|
||||
@@ -1702,15 +1705,20 @@ describe("release type filter", () => {
|
||||
});
|
||||
|
||||
test("the choice keys and labels agree", () => {
|
||||
expect(MIRROR_OVERRIDE_CHOICE_KEYS).toEqual(["releaseType"]);
|
||||
expect(MIRROR_OVERRIDE_CHOICE_KEYS).toEqual([
|
||||
"releaseType",
|
||||
"mirrorIssueState",
|
||||
]);
|
||||
for (const key of MIRROR_OVERRIDE_CHOICE_KEYS) {
|
||||
expect(MIRROR_OVERRIDE_LABELS[key]).toBeTruthy();
|
||||
}
|
||||
});
|
||||
|
||||
test("the choice keys are not walked as flags or limits", () => {
|
||||
expect(MIRROR_OVERRIDE_KEYS).not.toContain("releaseType");
|
||||
expect(MIRROR_OVERRIDE_LIMIT_KEYS).not.toContain("releaseType");
|
||||
for (const key of MIRROR_OVERRIDE_CHOICE_KEYS) {
|
||||
expect(MIRROR_OVERRIDE_KEYS).not.toContain(key);
|
||||
expect(MIRROR_OVERRIDE_LIMIT_KEYS).not.toContain(key);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1770,3 +1778,57 @@ describe("release type filter", () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("issue content options", () => {
|
||||
test("normalizes the issue state filter", () => {
|
||||
expect(normalizeIssueState(" OPEN ")).toBe("open");
|
||||
expect(normalizeIssueState("closed")).toBe("closed");
|
||||
expect(normalizeIssueState("all")).toBe("all");
|
||||
expect(normalizeIssueState("merged")).toBeUndefined();
|
||||
expect(normalizeIssueState(null)).toBeUndefined();
|
||||
});
|
||||
|
||||
test("resolves issue state per tier, most specific first", () => {
|
||||
const resolved = resolveMirrorOptions({
|
||||
config: makeConfig({ mirrorIssueState: "closed" }),
|
||||
repository: makeRepo({
|
||||
mirrorOverrides: { mirrorIssueState: "all" },
|
||||
}),
|
||||
orgOverrides: { mirrorIssueState: "open" },
|
||||
});
|
||||
|
||||
expect(resolved.mirrorIssueState).toBe("all");
|
||||
});
|
||||
|
||||
test("inherits issue state from the organization when the repository has none", () => {
|
||||
const resolved = resolveMirrorOptions({
|
||||
config: makeConfig({ mirrorIssueState: "closed" }),
|
||||
repository: makeRepo(),
|
||||
orgOverrides: { mirrorIssueState: "open" },
|
||||
});
|
||||
|
||||
expect(resolved.mirrorIssueState).toBe("open");
|
||||
});
|
||||
|
||||
test("keeps comments and media on when no tier pins them", () => {
|
||||
const resolved = resolveMirrorOptions({
|
||||
config: makeConfig({}),
|
||||
repository: makeRepo(),
|
||||
});
|
||||
|
||||
expect(resolved.mirrorIssueComments).toBe(true);
|
||||
expect(resolved.mirrorIssueMedia).toBe(true);
|
||||
expect(resolved.mirrorIssueState).toBe("all");
|
||||
});
|
||||
|
||||
test("the content options are disabled while issues are off", () => {
|
||||
const gating = getMirrorOverrideGating({
|
||||
targetKind: "repository",
|
||||
effective: { mirrorIssues: false },
|
||||
});
|
||||
|
||||
expect(gating.mirrorIssueComments).toBe(MIRROR_GATING_REASONS.issuesOff);
|
||||
expect(gating.mirrorIssueMedia).toBe(MIRROR_GATING_REASONS.issuesOff);
|
||||
expect(gating.mirrorIssueState).toBe(MIRROR_GATING_REASONS.issuesOff);
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,10 @@
|
||||
import type { Config } from "@/types/config";
|
||||
import type { MirrorOverrides, ReleaseType, Repository } from "@/lib/db/schema";
|
||||
import type {
|
||||
IssueStateFilter,
|
||||
MirrorOverrides,
|
||||
ReleaseType,
|
||||
Repository,
|
||||
} from "@/lib/db/schema";
|
||||
import { mirrorOverridesSchema } from "@/lib/db/schema";
|
||||
import {
|
||||
normalizeSourceProviderKind,
|
||||
@@ -18,6 +23,8 @@ export const MIRROR_OVERRIDE_KEYS = [
|
||||
"mirrorReleases",
|
||||
"mirrorMetadata",
|
||||
"mirrorIssues",
|
||||
"mirrorIssueComments",
|
||||
"mirrorIssueMedia",
|
||||
"mirrorPullRequests",
|
||||
"mirrorLabels",
|
||||
"mirrorMilestones",
|
||||
@@ -44,9 +51,13 @@ export type MirrorOverrideLimitKey = (typeof MIRROR_OVERRIDE_LIMIT_KEYS)[number]
|
||||
* tri-state toggles and the numeric rows in the dialog assume those shapes.
|
||||
*
|
||||
* `releaseType` decides which releases are candidates at all; `releaseLimit`
|
||||
* then decides how many of the matching ones are kept.
|
||||
* then decides how many of the matching ones are kept. `mirrorIssueState`
|
||||
* selects which source issues the issue pass lists.
|
||||
*/
|
||||
export const MIRROR_OVERRIDE_CHOICE_KEYS = ["releaseType"] as const;
|
||||
export const MIRROR_OVERRIDE_CHOICE_KEYS = [
|
||||
"releaseType",
|
||||
"mirrorIssueState",
|
||||
] as const;
|
||||
|
||||
export type MirrorOverrideChoiceKey =
|
||||
(typeof MIRROR_OVERRIDE_CHOICE_KEYS)[number];
|
||||
@@ -86,6 +97,19 @@ export const DEFAULT_RELEASE_LIMIT = 10;
|
||||
/** Matches `giteaConfigSchema.releaseType`'s default. */
|
||||
export const DEFAULT_RELEASE_TYPE: ReleaseType = "all";
|
||||
|
||||
/** Matches `giteaConfigSchema.mirrorIssueState`'s default. */
|
||||
export const DEFAULT_ISSUE_STATE: IssueStateFilter = "all";
|
||||
|
||||
/**
|
||||
* Flags whose absent global value means "on". The schema defaults cover saved
|
||||
* configs, but the resolver also runs against partial configs (tests, older
|
||||
* callers), where an absent key must not silently turn issue content off.
|
||||
*/
|
||||
const FLAG_DEFAULTS: Partial<Record<MirrorOverrideKey, boolean>> = {
|
||||
mirrorIssueComments: true,
|
||||
mirrorIssueMedia: true,
|
||||
};
|
||||
|
||||
/**
|
||||
* How many releases the mirror looks at when any release filter is active.
|
||||
*
|
||||
@@ -112,9 +136,14 @@ export const MAX_RELEASE_TAG_PATTERN_LENGTH = 200;
|
||||
export type ResolvedMirrorOptions = Record<MirrorOverrideKey, boolean> & {
|
||||
releaseLimit: number;
|
||||
releaseAssetLimit: number | null;
|
||||
// Global only: a cleanup policy for the destination rather than something a
|
||||
// single repository argues about. The ranks it prunes by still come from that
|
||||
// repository's own effective `releaseAssetLimit`.
|
||||
pruneOldReleaseAssets: boolean;
|
||||
releaseType: ReleaseType;
|
||||
releaseIncludeTags: string[];
|
||||
releaseExcludeTags: string[];
|
||||
mirrorIssueState: IssueStateFilter;
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -128,6 +157,7 @@ export type InheritedMirrorOptions = Partial<Record<MirrorOverrideKey, boolean>>
|
||||
releaseType?: ReleaseType;
|
||||
releaseIncludeTags?: string[];
|
||||
releaseExcludeTags?: string[];
|
||||
mirrorIssueState?: IssueStateFilter;
|
||||
skipForks?: boolean;
|
||||
};
|
||||
|
||||
@@ -156,6 +186,20 @@ export function normalizeReleaseAssetLimit(value: unknown): number | undefined {
|
||||
return limit >= 0 ? limit : undefined;
|
||||
}
|
||||
|
||||
const ISSUE_STATES: readonly IssueStateFilter[] = ["all", "open", "closed"];
|
||||
|
||||
/**
|
||||
* Sanitize an issue-state filter from any tier into one of the known values,
|
||||
* or `undefined` when it is absent or unusable so the next tier out is
|
||||
* consulted. Case is folded so an env var or a hand-edited row behaves like
|
||||
* the value the UI writes.
|
||||
*/
|
||||
export function normalizeIssueState(value: unknown): IssueStateFilter | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const state = value.trim().toLowerCase();
|
||||
return ISSUE_STATES.find((candidate) => candidate === state);
|
||||
}
|
||||
|
||||
const RELEASE_TYPES: readonly ReleaseType[] = ["all", "stable"];
|
||||
|
||||
/**
|
||||
@@ -407,9 +451,10 @@ const LIMIT_NORMALIZERS: Record<
|
||||
/** Per-key sanitizers for the choice overrides. */
|
||||
const CHOICE_NORMALIZERS: Record<
|
||||
MirrorOverrideChoiceKey,
|
||||
(value: unknown) => ReleaseType | undefined
|
||||
(value: unknown) => ReleaseType | IssueStateFilter | undefined
|
||||
> = {
|
||||
releaseType: normalizeReleaseType,
|
||||
mirrorIssueState: normalizeIssueState,
|
||||
};
|
||||
|
||||
/** Per-key sanitizers for the tag-pattern overrides. */
|
||||
@@ -424,8 +469,10 @@ const PATTERN_NORMALIZERS: Record<
|
||||
/**
|
||||
* Whether the release at `index` in the newest-first list gets its assets
|
||||
* uploaded. `null` means no limit. A release past the limit still gets its
|
||||
* notes and tag, and assets that already reached Gitea are never removed
|
||||
* because of this rule; it only decides whether a sync uploads.
|
||||
* notes and tag, and assets that already reached Gitea are not removed by this
|
||||
* rule on its own; it only decides whether a sync uploads. The opt-in
|
||||
* `pruneOldReleaseAssets` config switch is what applies the same rank rule in
|
||||
* reverse and deletes what an earlier, looser limit had uploaded.
|
||||
*/
|
||||
export function releaseGetsAssets(
|
||||
index: number,
|
||||
@@ -447,6 +494,8 @@ export const STARRED_CLAMPED_KEYS = [
|
||||
"mirrorReleases",
|
||||
"mirrorMetadata",
|
||||
"mirrorIssues",
|
||||
"mirrorIssueComments",
|
||||
"mirrorIssueMedia",
|
||||
"mirrorPullRequests",
|
||||
"mirrorLabels",
|
||||
"mirrorMilestones",
|
||||
@@ -467,6 +516,8 @@ export const STARRED_CLAMPED_KEYS = [
|
||||
export const GITHUB_ONLY_METADATA_KEYS = [
|
||||
"mirrorMetadata",
|
||||
"mirrorIssues",
|
||||
"mirrorIssueComments",
|
||||
"mirrorIssueMedia",
|
||||
"mirrorPullRequests",
|
||||
"mirrorLabels",
|
||||
"mirrorMilestones",
|
||||
@@ -485,6 +536,8 @@ export const PUSH_TARGET_CLAMPED_KEYS = [
|
||||
"mirrorReleases",
|
||||
"mirrorMetadata",
|
||||
"mirrorIssues",
|
||||
"mirrorIssueComments",
|
||||
"mirrorIssueMedia",
|
||||
"mirrorPullRequests",
|
||||
"mirrorLabels",
|
||||
"mirrorMilestones",
|
||||
@@ -509,6 +562,8 @@ export const UI_MIRROR_OVERRIDE_KEYS: MirrorOverrideKey[] = [
|
||||
"lfs",
|
||||
"wiki",
|
||||
"mirrorIssues",
|
||||
"mirrorIssueComments",
|
||||
"mirrorIssueMedia",
|
||||
"mirrorPullRequests",
|
||||
"mirrorReleases",
|
||||
"mirrorLabels",
|
||||
@@ -551,6 +606,9 @@ export function mirrorOptionsToFlags(
|
||||
releaseIncludeTags?: string[];
|
||||
releaseExcludeTags?: string[];
|
||||
mirrorMetadata?: boolean;
|
||||
issueState?: IssueStateFilter;
|
||||
issueComments?: boolean;
|
||||
issueMedia?: boolean;
|
||||
metadataComponents?: {
|
||||
issues?: boolean;
|
||||
pullRequests?: boolean;
|
||||
@@ -570,6 +628,10 @@ export function mirrorOptionsToFlags(
|
||||
mirrorReleases: !!mirrorOptions?.mirrorReleases,
|
||||
wiki: !!components?.wiki,
|
||||
mirrorIssues: !!components?.issues,
|
||||
mirrorIssueComments: mirrorOptions?.issueComments ?? true,
|
||||
mirrorIssueMedia: mirrorOptions?.issueMedia ?? true,
|
||||
mirrorIssueState:
|
||||
normalizeIssueState(mirrorOptions?.issueState) ?? DEFAULT_ISSUE_STATE,
|
||||
mirrorPullRequests: !!components?.pullRequests,
|
||||
mirrorLabels: !!components?.labels,
|
||||
mirrorMilestones: !!components?.milestones,
|
||||
@@ -590,6 +652,7 @@ export const MIRROR_GATING_REASONS = {
|
||||
starredCodeOnly:
|
||||
"Starred repos mirror code only (Advanced Options > starred code only)",
|
||||
labelsFollowIssues: "Issues mirroring already syncs labels",
|
||||
issuesOff: "Issues are not being mirrored",
|
||||
releasesOff: "Releases are not being mirrored",
|
||||
pushTarget: "Not available for GitHub and GitLab destinations: only branches and tags are pushed",
|
||||
} as const;
|
||||
@@ -645,6 +708,7 @@ export function getMirrorOverrideGating({
|
||||
gating.releaseLimit = MIRROR_GATING_REASONS.pushTarget;
|
||||
gating.releaseAssetLimit = MIRROR_GATING_REASONS.pushTarget;
|
||||
gating.releaseType = MIRROR_GATING_REASONS.pushTarget;
|
||||
gating.mirrorIssueState = MIRROR_GATING_REASONS.pushTarget;
|
||||
gating.releaseIncludeTags = MIRROR_GATING_REASONS.pushTarget;
|
||||
gating.releaseExcludeTags = MIRROR_GATING_REASONS.pushTarget;
|
||||
return gating;
|
||||
@@ -666,6 +730,18 @@ export function getMirrorOverrideGating({
|
||||
gating.mirrorLabels = MIRROR_GATING_REASONS.labelsFollowIssues;
|
||||
}
|
||||
|
||||
// Issue content filters only mean something while issues are mirrored.
|
||||
// Inherit the starred/push reason when that is what disabled issues.
|
||||
if (gating.mirrorIssues) {
|
||||
gating.mirrorIssueComments = gating.mirrorIssues;
|
||||
gating.mirrorIssueMedia = gating.mirrorIssues;
|
||||
gating.mirrorIssueState = gating.mirrorIssues;
|
||||
} else if (effective.mirrorIssues === false) {
|
||||
gating.mirrorIssueComments = MIRROR_GATING_REASONS.issuesOff;
|
||||
gating.mirrorIssueMedia = MIRROR_GATING_REASONS.issuesOff;
|
||||
gating.mirrorIssueState = MIRROR_GATING_REASONS.issuesOff;
|
||||
}
|
||||
|
||||
// The release limit and the release filters ride along with releases:
|
||||
// inherit the starred reason when that is what disabled releases, otherwise
|
||||
// report releases being off. Only an explicit `false` counts; an unknown
|
||||
@@ -698,6 +774,9 @@ export const MIRROR_OVERRIDE_LABELS: Record<
|
||||
mirrorReleases: "Releases",
|
||||
mirrorMetadata: "Metadata",
|
||||
mirrorIssues: "Issues",
|
||||
mirrorIssueComments: "Issue comments",
|
||||
mirrorIssueMedia: "Issue images and attachments",
|
||||
mirrorIssueState: "Issue state",
|
||||
mirrorPullRequests: "Pull requests",
|
||||
mirrorLabels: "Labels",
|
||||
mirrorMilestones: "Milestones",
|
||||
@@ -848,7 +927,9 @@ export function resolveMirrorOptions({
|
||||
} else if (typeof orgValue === "boolean") {
|
||||
resolved[key] = orgValue;
|
||||
} else {
|
||||
resolved[key] = !!globalConfig?.[key];
|
||||
const globalValue = globalConfig?.[key];
|
||||
resolved[key] =
|
||||
typeof globalValue === "boolean" ? globalValue : FLAG_DEFAULTS[key] ?? !!globalValue;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -879,6 +960,15 @@ export function resolveMirrorOptions({
|
||||
normalizeReleaseType(globalConfig?.releaseType) ??
|
||||
DEFAULT_RELEASE_TYPE;
|
||||
|
||||
// Issue state follows the same precedence and the same "an unusable value
|
||||
// falls through to the next tier" rule. Nothing pinned means mirror every
|
||||
// issue, like every configuration from before this option existed.
|
||||
resolved.mirrorIssueState =
|
||||
normalizeIssueState(repo?.mirrorIssueState) ??
|
||||
normalizeIssueState(org?.mirrorIssueState) ??
|
||||
normalizeIssueState(globalConfig?.mirrorIssueState) ??
|
||||
DEFAULT_ISSUE_STATE;
|
||||
|
||||
// And the same for the tag lists. No tier pins one means no filter, which is
|
||||
// what every configuration predating the option does. A tier that pins an
|
||||
// empty or unusable list is not a pin either, so the next tier out still
|
||||
@@ -895,6 +985,10 @@ export function resolveMirrorOptions({
|
||||
normalizeReleaseTagPatterns(globalConfig?.releaseExcludeTags) ??
|
||||
[];
|
||||
|
||||
// No per-repo or per-org tier for the pruning switch, and no stored value
|
||||
// means the safe default: never delete what the destination already holds.
|
||||
resolved.pruneOldReleaseAssets = globalConfig?.pruneOldReleaseAssets === true;
|
||||
|
||||
// Starred repos with starredCodeOnly mirror code and nothing else. This
|
||||
// clamp intentionally outranks explicit per-repo overrides: the setting
|
||||
// exists to stop starred repos from dragging in metadata wholesale.
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
/**
|
||||
* GET /api/activities: the feed must accept every status a mirror job can hold.
|
||||
*
|
||||
* "warning" is an activity-only status (see mirrorJobSchema) written when all
|
||||
* releases were filtered out of a repository. repoStatusEnum does not contain
|
||||
* it, and parsing with it turned the whole feed into a 500.
|
||||
*
|
||||
* mock.module is process-wide in bun, so this file re-runs itself in an
|
||||
* isolated child (same harness as ../dashboard/index.test.ts).
|
||||
*/
|
||||
import { expect, mock, test } from "bun:test";
|
||||
import { mirrorJobs } from "@/lib/db/schema";
|
||||
|
||||
const CHILD_FLAG = "GM_ACTIVITIES_ROUTE_ISOLATED";
|
||||
const isChild = !!process.env[CHILD_FLAG];
|
||||
|
||||
if (!isChild) {
|
||||
test("activities route - isolated child suite", () => {
|
||||
const res = Bun.spawnSync({
|
||||
cmd: [process.execPath, "test", import.meta.path],
|
||||
env: { ...process.env, [CHILD_FLAG]: "1" },
|
||||
stdout: "pipe",
|
||||
stderr: "pipe",
|
||||
});
|
||||
if (res.exitCode !== 0) {
|
||||
console.error(res.stdout.toString());
|
||||
console.error(res.stderr.toString());
|
||||
}
|
||||
expect(res.exitCode).toBe(0);
|
||||
}, 120_000);
|
||||
}
|
||||
|
||||
if (isChild) {
|
||||
// Shaped like what the sync writes when the release filters empty a
|
||||
// destination (gitea.ts), typed as table rows so they track the schema.
|
||||
const rows: Array<typeof mirrorJobs.$inferSelect> = [
|
||||
{
|
||||
id: "job-warning",
|
||||
userId: "user-1",
|
||||
repositoryId: "repo-1",
|
||||
repositoryName: "ollama",
|
||||
organizationId: null,
|
||||
organizationName: null,
|
||||
details:
|
||||
"12 releases deleted by the release filters; the destination now has none. Check the include/exclude tag patterns and the pre-release setting.",
|
||||
status: "warning",
|
||||
message: "All releases removed from mirrors/ollama",
|
||||
timestamp: new Date("2026-10-29T12:00:05.000Z"),
|
||||
jobType: "mirror",
|
||||
batchId: null,
|
||||
totalItems: null,
|
||||
completedItems: 0,
|
||||
itemIds: null,
|
||||
completedItemIds: [],
|
||||
inProgress: false,
|
||||
startedAt: null,
|
||||
completedAt: null,
|
||||
lastCheckpoint: null,
|
||||
},
|
||||
{
|
||||
id: "job-mirrored",
|
||||
userId: "user-1",
|
||||
repositoryId: "repo-2",
|
||||
repositoryName: "other",
|
||||
organizationId: null,
|
||||
organizationName: null,
|
||||
details: null,
|
||||
status: "mirrored",
|
||||
message: "Successfully mirrored repository: other",
|
||||
timestamp: new Date("2026-10-29T11:00:02.000Z"),
|
||||
jobType: "mirror",
|
||||
batchId: null,
|
||||
totalItems: null,
|
||||
completedItems: 0,
|
||||
itemIds: null,
|
||||
completedItemIds: [],
|
||||
inProgress: false,
|
||||
startedAt: null,
|
||||
completedAt: null,
|
||||
lastCheckpoint: null,
|
||||
},
|
||||
];
|
||||
|
||||
mock.module("@/lib/db", () => ({
|
||||
db: {
|
||||
select: () => {
|
||||
const query: any = {
|
||||
from: () => query,
|
||||
where: () => query,
|
||||
orderBy: () => query,
|
||||
limit: async () => rows,
|
||||
then: (resolve: (value: unknown) => void) => resolve(rows),
|
||||
};
|
||||
return query;
|
||||
},
|
||||
},
|
||||
mirrorJobs,
|
||||
}));
|
||||
|
||||
mock.module("@/lib/auth-guards", () => ({
|
||||
requireAuthenticatedUserId: async () => ({ userId: "user-1" }),
|
||||
}));
|
||||
|
||||
const { GET } = await import("./index");
|
||||
|
||||
test("activity feed returns warning jobs instead of failing", async () => {
|
||||
const response = await GET({
|
||||
request: new Request("http://localhost/api/activities?userId=user-1"),
|
||||
locals: {},
|
||||
} as any);
|
||||
expect(response.status).toBe(200);
|
||||
const body = await response.json();
|
||||
expect(body.success).toBe(true);
|
||||
expect(body.activities.map((entry: any) => entry.status)).toEqual(["warning", "mirrored"]);
|
||||
expect(body.activities[0].repositoryName).toBe("ollama");
|
||||
expect(body.activities[0].message).toBe("All releases removed from mirrors/ollama");
|
||||
});
|
||||
}
|
||||
@@ -3,7 +3,7 @@ import { db, mirrorJobs } from "@/lib/db";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { createSecureErrorResponse } from "@/lib/utils";
|
||||
import type { MirrorJob } from "@/lib/db/schema";
|
||||
import { repoStatusEnum } from "@/types/Repository";
|
||||
import { activityStatusEnum } from "@/types/Repository";
|
||||
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
|
||||
|
||||
export const GET: APIRoute = async ({ request, locals }) => {
|
||||
@@ -28,7 +28,10 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
repositoryName: job.repositoryName ?? undefined,
|
||||
organizationId: job.organizationId ?? undefined,
|
||||
organizationName: job.organizationName ?? undefined,
|
||||
status: repoStatusEnum.parse(job.status),
|
||||
// Job/activity statuses include the activity-only "warning" (see mirrorJobSchema).
|
||||
// repoStatusEnum only covers repository statuses, so parsing "warning" with
|
||||
// it threw a ZodError and turned the whole feed into a 500.
|
||||
status: activityStatusEnum.parse(job.status),
|
||||
details: job.details ?? undefined,
|
||||
message: job.message,
|
||||
timestamp: job.timestamp,
|
||||
|
||||
@@ -2,7 +2,7 @@ import type { APIRoute } from "astro";
|
||||
import { db, repositories, organizations, mirrorJobs, configs } from "@/lib/db";
|
||||
import { eq, count, and, sql, or, isNotNull } from "drizzle-orm";
|
||||
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
|
||||
import type { DashboardApiResponse } from "@/types/dashboard";
|
||||
import { DASHBOARD_LIST_LIMIT, type DashboardApiResponse } from "@/types/dashboard";
|
||||
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
|
||||
|
||||
export const GET: APIRoute = async ({ request, locals }) => {
|
||||
@@ -26,7 +26,7 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
.from(repositories)
|
||||
.where(eq(repositories.userId, userId))
|
||||
.orderBy(sql`${repositories.updatedAt} DESC`)
|
||||
.limit(10),
|
||||
.limit(DASHBOARD_LIST_LIMIT),
|
||||
db
|
||||
.select()
|
||||
.from(organizations)
|
||||
@@ -40,7 +40,7 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
// Timestamps are stored to the second; rowid keeps events written in
|
||||
// the same second newest first too (#454).
|
||||
.orderBy(sql`${mirrorJobs.timestamp} DESC, ${mirrorJobs}.rowid DESC`)
|
||||
.limit(10),
|
||||
.limit(DASHBOARD_LIST_LIMIT),
|
||||
db
|
||||
.select()
|
||||
.from(configs)
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db, repositories, configs, organizations } from "@/lib/db";
|
||||
import { db, repositories, configs, organizations, sources } from "@/lib/db";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import { resolveDestinationOwner } from "@/lib/gitea";
|
||||
import type { Repository } from "@/lib/db/schema";
|
||||
import {
|
||||
repositoryVisibilityEnum,
|
||||
repoStatusEnum,
|
||||
@@ -63,17 +65,54 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
.where(eq(organizations.userId, userId));
|
||||
|
||||
const organizationDestinations: Record<string, string> = {};
|
||||
const organizationDestinationByName = new Map<string, string>();
|
||||
for (const row of orgDestinationRows) {
|
||||
if (row.destinationOrg) {
|
||||
organizationDestinations[row.name] = row.destinationOrg;
|
||||
}
|
||||
if (!row.destinationOrg) continue;
|
||||
organizationDestinations[row.name] = row.destinationOrg;
|
||||
organizationDestinationByName.set(row.name, row.destinationOrg);
|
||||
}
|
||||
|
||||
const response: RepositoryApiResponse = {
|
||||
success: true,
|
||||
message: "Repositories fetched successfully",
|
||||
organizationDestinations,
|
||||
repositories: rawRepositories.map((repo) => ({
|
||||
// The repository's own source username is part of the strategy fallback for
|
||||
// personal repos, so resolve it up front instead of per row.
|
||||
const sourceRows = await db
|
||||
.select({ id: sources.id, username: sources.username })
|
||||
.from(sources)
|
||||
.where(eq(sources.userId, userId));
|
||||
const sourceUsernames = new Map(sourceRows.map((row) => [row.id, row.username]));
|
||||
|
||||
// The real destination each row mirrors into. This is what the repositories
|
||||
// page filters by; `organizationDestinations` above remains for the
|
||||
// destination column's existing per-organization fallback (#416).
|
||||
const destinationOwners: Record<string, string> = {};
|
||||
const repositoriesWithDestination = rawRepositories.map((repo) => {
|
||||
// Matched by exact name, like getOrganizationConfig on the mirror path.
|
||||
const organizationDestination = repo.organization
|
||||
? organizationDestinationByName.get(repo.organization)
|
||||
: undefined;
|
||||
|
||||
try {
|
||||
destinationOwners[repo.id] = resolveDestinationOwner({
|
||||
config: {
|
||||
userId,
|
||||
githubConfig: config.githubConfig as any,
|
||||
giteaConfig: config.giteaConfig as any,
|
||||
},
|
||||
repository: repo as unknown as Repository,
|
||||
organizationDestination,
|
||||
sourceUsername: repo.sourceId
|
||||
? sourceUsernames.get(repo.sourceId)
|
||||
: undefined,
|
||||
});
|
||||
} catch (error) {
|
||||
// A row that cannot be resolved still belongs in the list; it simply
|
||||
// has no destination filter value.
|
||||
console.warn(
|
||||
`[Repositories] Could not resolve destination owner for ${repo.fullName}:`,
|
||||
error
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
...repo,
|
||||
organization: repo.organization ?? undefined,
|
||||
lastMirrored: repo.lastMirrored ?? undefined,
|
||||
@@ -81,7 +120,15 @@ export const GET: APIRoute = async ({ request, locals }) => {
|
||||
forkedFrom: repo.forkedFrom ?? undefined,
|
||||
status: repoStatusEnum.parse(repo.status),
|
||||
visibility: repositoryVisibilityEnum.parse(repo.visibility),
|
||||
})),
|
||||
};
|
||||
});
|
||||
|
||||
const response: RepositoryApiResponse = {
|
||||
success: true,
|
||||
message: "Repositories fetched successfully",
|
||||
organizationDestinations,
|
||||
destinationOwners,
|
||||
repositories: repositoriesWithDestination,
|
||||
};
|
||||
|
||||
return jsonResponse({
|
||||
|
||||
@@ -44,6 +44,12 @@ export interface RepositoryApiSuccessResponse {
|
||||
* list needs it to name a repository's real default (#416).
|
||||
*/
|
||||
organizationDestinations?: Record<string, string>;
|
||||
/**
|
||||
* Repository id -> the destination owner/org the mirror will actually use.
|
||||
* Resolved with the same precedence as the mirror path, so the repositories
|
||||
* page can filter by destination rather than by source organization.
|
||||
*/
|
||||
destinationOwners?: Record<string, string>;
|
||||
}
|
||||
|
||||
export interface RepositoryApiErrorResponse {
|
||||
|
||||
+12
-1
@@ -1,4 +1,8 @@
|
||||
import { type Config as ConfigType, type ReleaseType } from "@/lib/db/schema";
|
||||
import {
|
||||
type Config as ConfigType,
|
||||
type IssueStateFilter,
|
||||
type ReleaseType,
|
||||
} from "@/lib/db/schema";
|
||||
|
||||
export type GiteaOrgVisibility = "public" | "private" | "limited";
|
||||
export type MirrorStrategy = "preserve" | "single-org" | "flat-user" | "mixed";
|
||||
@@ -86,6 +90,7 @@ export interface MirrorOptions {
|
||||
mirrorReleases: boolean;
|
||||
releaseLimit?: number; // Limit number of releases to mirror (default: 10)
|
||||
releaseAssetLimit?: number | null; // Upload assets only for the newest N releases; null or absent means all, 0 means none
|
||||
pruneOldReleaseAssets?: boolean; // Also delete attachments already on releases past releaseAssetLimit (default: false)
|
||||
releaseType?: ReleaseType; // "all" | "stable": which releases are mirror candidates before the limit applies
|
||||
releaseIncludeTags?: string[]; // Glob or /regex/ tag patterns; when set, only matching tags are mirror candidates
|
||||
releaseExcludeTags?: string[]; // Glob or /regex/ tag patterns; matching tags are skipped, and exclude wins over include
|
||||
@@ -98,6 +103,12 @@ export interface MirrorOptions {
|
||||
milestones: boolean;
|
||||
wiki: boolean;
|
||||
};
|
||||
/** Which source issues are listed: all, open only, or closed only. */
|
||||
issueState: IssueStateFilter;
|
||||
/** Copy comments along with each issue. */
|
||||
issueComments: boolean;
|
||||
/** Keep images, videos and attachment links in issue bodies and comments. */
|
||||
issueMedia: boolean;
|
||||
}
|
||||
|
||||
export interface AdvancedOptions {
|
||||
|
||||
@@ -2,6 +2,12 @@ import type { Repository } from "@/lib/db/schema";
|
||||
import type { Organization } from "@/lib/db/schema";
|
||||
import type { MirrorJob } from "@/lib/db/schema";
|
||||
|
||||
/**
|
||||
* The most repositories and activities the dashboard API returns. The
|
||||
* dashboard shows as many as fit on the screen, up to this.
|
||||
*/
|
||||
export const DASHBOARD_LIST_LIMIT = 50;
|
||||
|
||||
export interface DashboardApiSuccessResponse {
|
||||
success: true;
|
||||
message: string;
|
||||
|
||||
+2
-1
@@ -6,7 +6,8 @@ export interface FilterParams {
|
||||
status?: ActivityStatus | ""; // repos, activity and orgs status
|
||||
membershipRole?: MembershipRole | ""; //membership role in orgs
|
||||
owner?: string; // owner of the repos
|
||||
organization?: string; // organization of the repos
|
||||
organization?: string; // source organization of the repos
|
||||
destinationOrganization?: string; // resolved organization/owner on the destination
|
||||
sort?: string; // repository sort order
|
||||
hasOverrides?: "overridden" | "default" | ""; // repos with custom mirror options
|
||||
type?: string; //types in activity log
|
||||
|
||||
@@ -157,7 +157,7 @@ export const useCases: UseCase[] = [
|
||||
title: 'Control Release Asset Storage',
|
||||
summary: 'Keep release notes for every mirrored release but upload binaries only for the newest few, globally or per organization and repository.',
|
||||
painPoint: 'A project that ships builds for five platforms every fortnight turns a release mirror into gigabytes of installers nobody will run.',
|
||||
outcome: 'Two independent limits decide how many releases exist on Gitea and how many of those carry assets, and nothing already uploaded is removed.',
|
||||
outcome: 'Two independent limits decide how many releases exist on Gitea and how many of those carry assets, and an opt-in switch also deletes the attachments of the older ones.',
|
||||
tags: ['Releases', 'Storage', 'Overrides'],
|
||||
},
|
||||
];
|
||||
@@ -38,6 +38,8 @@ Below the connection cards, the mirror settings decide what travels with the cod
|
||||
|
||||
Releases have two numbers under the switch. **Latest** is how many releases are mirrored per repository, newest first, ten by default. **Assets for latest** is how many of those get their assets uploaded: leave it empty to upload assets for every mirrored release, set it to `0` for release notes and tags only, or set it to a number to keep assets for the newest N releases while older ones keep their notes and tag. The limit never deletes assets that were already uploaded.
|
||||
|
||||
The **Remove old release assets** switch under them turns that last sentence around. Off, which is the default, a sync only ever uploads. On, every sync also deletes the attachments of each mirrored release ranked past **Assets for latest**, so lowering the limit actually frees the disk. It is global rather than per repository, it never removes a release or its notes, and a release whose attachments cannot be listed is left alone and retried on the next sync.
|
||||
|
||||
The **Pre-releases** switch under them decides which releases are candidates at all: left on it mirrors every release, turned off it skips everything the source marks as a pre-release. With a filter active the **Latest** count applies to the releases that match, so ten stable releases are kept even when a project ships betas in between, and releases of the other kind are removed from Gitea on the next sync. A release the destination has no pre-release flag for is left alone. The per organization and per repository **Mirror options** dialog offers the same choice under its **Releases** row - **All**, **Stable only**, or **Inherit**, where **Inherit** means the level above - and the release type is `RELEASE_TYPE` in the environment.
|
||||
|
||||
**Custom tag rules...** beside that switch opens a dialog with **Include tags** and **Exclude tags**, two comma separated pattern lists that catch the projects shipping release candidates without marking them as pre-releases. The same lists sit under the **Releases** row in the per organization and per repository dialog, where leaving both lists empty means the level above, and in the environment they are `RELEASE_INCLUDE_TAGS` and `RELEASE_EXCLUDE_TAGS`.
|
||||
|
||||
@@ -165,7 +165,8 @@ Only used when `DESTINATION_PROVIDER` is `github` or `gitlab`.
|
||||
| `RELEASE_TYPE` | `all` | Which releases are candidates: `all` or `stable` (skip what the source marks as a pre-release). With a filter active the limit counts the releases that match, and releases of the other kind are removed from Gitea on the next sync. |
|
||||
| `RELEASE_INCLUDE_TAGS` | empty | Comma separated tag patterns; only releases whose tag matches are mirrored. A pattern is a glob over the whole tag (`v1.*`, `*-rc.*`) or a regular expression in slashes (`/^v\d+\.\d+\.\d+$/`), matched case-insensitively. |
|
||||
| `RELEASE_EXCLUDE_TAGS` | empty | Comma separated tag patterns in the same syntax; matching releases are skipped, and an excluded tag wins over `RELEASE_INCLUDE_TAGS`. |
|
||||
| `RELEASE_ASSET_LIMIT` | unset | Upload release assets only for the newest N mirrored releases; older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means notes only. Never deletes assets already uploaded. |
|
||||
| `RELEASE_ASSET_LIMIT` | unset | Upload release assets only for the newest N mirrored releases; older releases still get their notes and tag. Unset means assets for every mirrored release, `0` means notes only. Lowering it never deletes assets already uploaded unless `PRUNE_OLD_RELEASE_ASSETS` is on. |
|
||||
| `PRUNE_OLD_RELEASE_ASSETS` | `false` | Opt-in cleanup for what the asset limit left behind. On, every sync also deletes the attachments of each mirrored release ranked past `RELEASE_ASSET_LIMIT`; the releases and their notes stay. |
|
||||
| `MIRROR_WIKI` | `false` | Clone wiki content. |
|
||||
| `MIRROR_ISSUE_CONCURRENCY` | `3` | Issues processed in parallel. |
|
||||
| `MIRROR_PULL_REQUEST_CONCURRENCY` | `5` | Pull requests processed in parallel. |
|
||||
|
||||
@@ -26,7 +26,7 @@ The **Notification Events** card controls which job outcomes produce a notificat
|
||||
|
||||
| Event | Fires when | Default |
|
||||
|---|---|---|
|
||||
| Sync errors | A mirror job ends with status `failed` | On |
|
||||
| Sync errors | A mirror job ends with status `failed`, the release filters remove every release of a destination, or Gitea has not run a sync it accepted for over an hour (at most one such alert every six hours) | On |
|
||||
| Sync success | A mirror job ends with status `mirrored` or `synced` | Off |
|
||||
| New repository discovered | Not yet implemented, the switch is disabled and marked SOON | Off |
|
||||
|
||||
|
||||
@@ -45,10 +45,12 @@ Patterns are comma separated, and each one is either a glob over the whole tag o
|
||||
|
||||
Matching ignores case, and a pattern that cannot compile is ignored rather than failing the run. A tag filter widens the search exactly like the type filter does, so **Latest 1** with `v*` still finds the newest real release behind a wall of nightlies. Releases whose tag no longer matches are removed from Gitea on the next sync, and unlike the type filter a tag pattern can clean up releases the destination has no pre-release flag for, because it only needs the tag name.
|
||||
|
||||
## Nothing already uploaded is deleted
|
||||
## Nothing already uploaded is deleted, unless you ask
|
||||
|
||||
The asset limit only decides whether a sync uploads. Lowering it later never removes assets that already reached Gitea. If you mirrored everything for a year and then set the limit to 2, the old files stay until you delete them in Gitea yourself. That is the safe direction for a backup tool, and it means the setting is cheap to experiment with.
|
||||
|
||||
The **Remove old release assets** switch under the release settings changes that on purpose. With it on, a sync also prunes: the attachments of every mirrored release ranked past **Assets for latest** are deleted from Gitea, so lowering the limit actually shrinks what is on disk. It is a global switch, it stays off until you turn it on, and it only touches the files - the release and its notes stay. A release whose attachment listing cannot be read keeps its files and is retried on the next sync.
|
||||
|
||||
## Step-by-step
|
||||
|
||||
### 1. Set a global default
|
||||
@@ -71,6 +73,7 @@ For deployments configured from the environment:
|
||||
MIRROR_RELEASES=true
|
||||
RELEASE_LIMIT=25
|
||||
RELEASE_ASSET_LIMIT=3 # unset for all, 0 for notes only
|
||||
PRUNE_OLD_RELEASE_ASSETS=true # also delete attachments past that count
|
||||
RELEASE_TYPE=stable # all or stable
|
||||
RELEASE_INCLUDE_TAGS=v* # comma separated globs or /regex/
|
||||
RELEASE_EXCLUDE_TAGS=*-rc.*
|
||||
@@ -82,6 +85,8 @@ RELEASE_EXCLUDE_TAGS=*-rc.*
|
||||
|
||||
Take a project that publishes a release every two weeks with five platform builds of 40 MB each. That is 200 MB per release and about 5 GB per year of assets, for one repository. With the release limit at 25 and no asset limit, the first mirror pulls about 5 GB and every sync afterwards adds the newest release's 200 MB. With **Assets for latest** at 3, the first mirror pulls 600 MB. Each later release still adds 200 MB, because a new release is inside the limit and gets its files, and the files of the release that dropped out of the top three stay on disk until you remove them. The ongoing growth is the same; what you save is the backfill of every historical build.
|
||||
|
||||
With **Remove old release assets** on, the picture changes: when a release drops out of the top three, its 200 MB go with it, so the destination hovers around 600 MB instead of growing without bound.
|
||||
|
||||
Multiply by the number of repositories that ship binaries and compare with the free space on the Gitea host. Gitea stores release attachments in its own attachment storage, separate from the git objects and from LFS, so check that path when you plan capacity.
|
||||
|
||||
## Where releases apply
|
||||
|
||||
Reference in new issue
Block a user