mirror of
https://github.com/RayLabsHQ/gitea-mirror.git
synced 2026-10-30 22:21:58 +01:00
feat: fill API gaps for integrations that add repositories and organizations (#476)
- JSON 404 for an /api path or method without an endpoint, instead of the HTML page. - The 409 from sync/repository and sync/organization carries the stored record. - role is optional on sync/organization (default member, force keeps the stored role, unknown role is 400); the not-found message mentions user accounts. - GET /api/repositories/:id and GET /api/organizations/:id. - API docs: organization endpoints, host and path for pasted URLs, and the Content-Type header every write needs.
This commit is contained in:
ARUNAVO RAY authored and GitHub committed
1 parent
2189254e38
commit
f3767235f7
13 files changed
+527
-30
No files matched your search
+31
-6
@@ -25,6 +25,14 @@ A missing, malformed, expired or revoked key gets `401 Unauthorized`:
|
||||
{ "success": false, "error": "Unauthorized" }
|
||||
```
|
||||
|
||||
Send `Content-Type: application/json` on every `POST`, `PUT`, `PATCH` and `DELETE`, also when the call has no body. Without it a script gets `403` with `Cross-site POST form submissions are forbidden`: the app's protection against cross-site form posts treats a write with no content type, or a form content type, as a form unless its `Origin` header matches the app. curl's `-d` sends a form content type unless you set the header.
|
||||
|
||||
A path or method that has no endpoint answers `404` with a JSON body naming both, so it reads differently from a `404` for an id or name that was not found:
|
||||
|
||||
```json
|
||||
{ "success": false, "error": "No API endpoint for GET /api/organizations. Check the path and the HTTP method." }
|
||||
```
|
||||
|
||||
If the app is served under a path prefix (`BASE_URL=/mirror`), prefix every URL with it: `/mirror/api/...`.
|
||||
|
||||
## Endpoints
|
||||
@@ -62,6 +70,8 @@ Despite the name it lists every repository the account tracks, whatever the conf
|
||||
|
||||
`status` is one of `imported`, `mirroring`, `mirrored`, `failed`, `syncing`, `synced`, `skipped`, `deleted`, `archived`.
|
||||
|
||||
`GET /api/repositories/{id}` returns one of them in the same shape as `{ "success": true, "repository": { ... } }`, or `404` when the id is not this account's. Use it to watch a single mirror without fetching the whole list.
|
||||
|
||||
### Add a repository
|
||||
|
||||
`POST /api/sync/repository`
|
||||
@@ -72,14 +82,16 @@ Despite the name it lists every repository the account tracks, whatever the conf
|
||||
|
||||
Optional fields: `destinationOrg` to override the Gitea organization for this repository, `force: true` to refresh the metadata of a repository that is already tracked.
|
||||
|
||||
When the request comes from a pasted URL, also send its host as `host` and its path segments as `path`, for example `["octocat", "hello-world", "tree", "main"]`. The host picks the connected source with that host (`400` when none has it), and the source reads the repository from the segments: GitHub and Gitea take the first two, GitLab takes the nested groups up to `/-/`. `owner` and `repo` are still required.
|
||||
|
||||
The repository is looked up on the configured source and added with status `imported`. It is not mirrored until you start a mirror (next call) or the scheduler picks it up.
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `200` | Added. `repository.id` is what the mirror call needs. |
|
||||
| `400` | `owner` or `repo` missing. |
|
||||
| `400` | `owner` or `repo` missing, `host` matches no connected source, or no source is connected. |
|
||||
| `404` | No configuration for this account, or the repository does not exist on the source. |
|
||||
| `409` | Already tracked. Send `force: true` to refresh it instead. |
|
||||
| `409` | Already tracked. `repository` is the stored record, so `repository.id` works as on `200`. Send `force: true` to refresh its metadata instead. |
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -97,7 +109,7 @@ The repository is looked up on the configured source and added with status `impo
|
||||
{ "repositoryIds": ["9b2f..."] }
|
||||
```
|
||||
|
||||
Starts mirroring in the background and returns straight away. Poll the list endpoint to watch `status` move from `mirroring` to `mirrored` or `failed`.
|
||||
Starts mirroring in the background and returns straight away. Poll `GET /api/repositories/{id}`, or the list, to watch `status` move from `mirroring` to `mirrored` or `failed`.
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
@@ -112,10 +124,21 @@ Starts mirroring in the background and returns straight away. Poll the list endp
|
||||
`POST /api/sync/organization` adds an organization the same way:
|
||||
|
||||
```json
|
||||
{ "org": "my-org", "role": "member" }
|
||||
{ "org": "my-org" }
|
||||
```
|
||||
|
||||
`role` is the account's role in that organization (`member`, `admin`, `owner` or `billing_manager`). `409` means it is already tracked; `force: true` refreshes it. Then `POST /api/job/mirror-org` with `{ "organizationIds": ["..."] }` mirrors its repositories.
|
||||
`role` is optional. It is the account's role in that organization (`member`, `admin`, `owner` or `billing_manager`) and only labels the organization on the Organizations page. Without it a new organization is added as `member`, and a forced re-add keeps the stored role.
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `200` | Added, with every repository the source lists in it. `organization.id` is what the mirror call needs. |
|
||||
| `400` | `org` missing, an unknown `role` or `provider`, or no source is connected. |
|
||||
| `404` | No configuration for this account, or the source has no organization of that name. A user account is not an organization: add its repositories one at a time with `POST /api/sync/repository`. |
|
||||
| `409` | Already tracked. `organization` is the stored record. Send `force: true` to refresh it instead. |
|
||||
|
||||
Then `POST /api/job/mirror-org` with `{ "organizationIds": ["..."] }` mirrors its repositories. It takes ids, not names, and answers `404` when none of them belong to this account.
|
||||
|
||||
`GET /api/github/organizations` lists every tracked organization with its `id`, `name`, `status` and repository counts. `GET /api/organizations/{id}` returns one as `{ "success": true, "organization": { ... } }`, without the public, private and fork counts, or `404` when the id is not this account's. An organization's `status` turns `mirrored` once every repository in it has been processed; each repository keeps its own `status`.
|
||||
|
||||
When several sources are connected, an optional `sourceId` picks which one the organization imports and mirrors from (the first connected source is the default):
|
||||
|
||||
@@ -153,7 +176,7 @@ Compares what the destination (Gitea or Forgejo) holds with the repositories thi
|
||||
{ "dryRun": true, "adoptUntracked": false, "relocateMoved": false, "resetMissing": false }
|
||||
```
|
||||
|
||||
The body is optional and every field defaults to the value above, so an empty `POST` is a dry run. With `dryRun: false`:
|
||||
The body is optional and every field defaults to the value above, so an empty `POST` (with the JSON content type, see [Send the key](#send-the-key)) is a dry run. With `dryRun: false`:
|
||||
|
||||
- `adoptUntracked: true` creates a row for each untracked mirror from its source URL, so scheduled sync and the cleanup service include it from then on. The row keeps the mirror where it is, even when the strategy would put it elsewhere.
|
||||
- `relocateMoved: true` points each moved row at the owner that holds its mirror now, so sync follows the transfer instead of failing. A row that failed because its mirror was not at the recorded place goes back to `mirrored`. Nothing changes on the destination.
|
||||
@@ -263,6 +286,8 @@ curl -sS -X POST "$BASE/api/job/mirror-repo" \
|
||||
-d "{\"repositoryIds\":[\"$id\"]}"
|
||||
```
|
||||
|
||||
`.repository.id` is set on both `200` and `409`, so the script works the same for a repository that is already tracked.
|
||||
|
||||
## Managing keys from the API
|
||||
|
||||
Manage keys through the settings UI or with a browser session. The key endpoints need a session cookie, not a key, so a leaked key cannot mint more keys, and like every cookie-authenticated `POST` to `/api/auth/*` they also need an `Origin` header (browsers send it on their own; a script must add `Origin: https://mirror.example.com`). The key itself is only for the app routes above. The endpoints live under `/api/auth/api-key/`:
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { apiNotFoundResponse, isApiNotFoundPage } from "./api-not-found";
|
||||
|
||||
function page(status: number, contentType: string) {
|
||||
return new Response("<html></html>", { status, headers: { "Content-Type": contentType } });
|
||||
}
|
||||
|
||||
describe("isApiNotFoundPage", () => {
|
||||
test("matches Astro's HTML 404 for an /api path", () => {
|
||||
expect(isApiNotFoundPage("/api/organizations", page(404, "text/html"))).toBe(true);
|
||||
expect(isApiNotFoundPage("/api", page(404, "text/html; charset=utf-8"))).toBe(true);
|
||||
});
|
||||
|
||||
test("leaves JSON 404s from route handlers alone", () => {
|
||||
expect(isApiNotFoundPage("/api/organizations/x", page(404, "application/json"))).toBe(false);
|
||||
});
|
||||
|
||||
test("leaves pages and other statuses alone", () => {
|
||||
expect(isApiNotFoundPage("/no-such-page", page(404, "text/html"))).toBe(false);
|
||||
expect(isApiNotFoundPage("/apis", page(404, "text/html"))).toBe(false);
|
||||
expect(isApiNotFoundPage("/api/health", page(200, "text/html"))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("apiNotFoundResponse", () => {
|
||||
test("names the method and path in a JSON 404", async () => {
|
||||
const response = apiNotFoundResponse("GET", "/api/sync/organization");
|
||||
|
||||
expect(response.status).toBe(404);
|
||||
expect(response.headers.get("content-type")).toBe("application/json");
|
||||
expect(await response.json()).toEqual({
|
||||
success: false,
|
||||
error: "No API endpoint for GET /api/sync/organization. Check the path and the HTTP method.",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
import { stripBasePath } from "./base-path";
|
||||
|
||||
/**
|
||||
* Astro answers a request under /api that matches no route, or a route that
|
||||
* does not export the request's method, with the HTML "Page Not Found" page.
|
||||
* Scripts and integrations get a JSON error naming the method and path
|
||||
* instead, so a wrong method reads differently from an id that was not found.
|
||||
*/
|
||||
export function isApiNotFoundPage(pathname: string, response: Response): boolean {
|
||||
if (response.status !== 404) return false;
|
||||
const contentType = response.headers.get("content-type") ?? "";
|
||||
if (!contentType.includes("text/html")) return false;
|
||||
const path = stripBasePath(pathname);
|
||||
return path === "/api" || path.startsWith("/api/");
|
||||
}
|
||||
|
||||
export function apiNotFoundResponse(method: string, pathname: string): Response {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
error: `No API endpoint for ${method} ${pathname}. Check the path and the HTTP method.`,
|
||||
}),
|
||||
{ status: 404, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
}
|
||||
+6
-1
@@ -11,6 +11,7 @@ import { mintSessionFromHeaders } from './lib/auth-header-bridge';
|
||||
import { initializeConfigFromEnv } from './lib/env-config-loader';
|
||||
import { db, users } from './lib/db';
|
||||
import { getBasePath } from './lib/base-path';
|
||||
import { apiNotFoundResponse, isApiNotFoundPage } from './lib/api-not-found';
|
||||
|
||||
const ASTRO_INTERNAL_ASSET_PATH_PATTERN = /(["'])\/(_astro\/|_server-islands\/|_image\b)/g;
|
||||
|
||||
@@ -272,7 +273,11 @@ export const onRequest = defineMiddleware(async (context, next) => {
|
||||
}
|
||||
|
||||
// Continue with the request
|
||||
const response = await next();
|
||||
let response = await next();
|
||||
|
||||
if (isApiNotFoundPage(context.url.pathname, response)) {
|
||||
response = apiNotFoundResponse(context.request.method, context.url.pathname);
|
||||
}
|
||||
|
||||
// Forward any Set-Cookie headers minted by the header-auth bridge
|
||||
// onto the outbound response. Done before the early returns below so
|
||||
|
||||
@@ -35,6 +35,8 @@ let sourceRows: any[] = [];
|
||||
let orgRows: any[] = [];
|
||||
/** Leaf values of each delete's where() condition (drizzle conditions are cyclic). */
|
||||
let deleteWhereValues: unknown[][] = [];
|
||||
/** Leaf values of each select's where() condition. */
|
||||
let selectWhereValues: unknown[][] = [];
|
||||
|
||||
function conditionLeafValues(node: any, depth = 0): unknown[] {
|
||||
if (node == null || depth > 8) return [];
|
||||
@@ -62,9 +64,10 @@ if (isChild) {
|
||||
db: {
|
||||
select: () => ({
|
||||
from: () => ({
|
||||
where: () => ({
|
||||
limit: async () => orgRows.slice(0, 1),
|
||||
}),
|
||||
where: (cond: any) => {
|
||||
selectWhereValues.push(conditionLeafValues(cond));
|
||||
return { limit: async () => orgRows.slice(0, 1) };
|
||||
},
|
||||
}),
|
||||
}),
|
||||
update: () => ({
|
||||
@@ -102,9 +105,17 @@ if (isChild) {
|
||||
}));
|
||||
}
|
||||
|
||||
const { PATCH, DELETE } = isChild
|
||||
const { GET, PATCH, DELETE } = isChild
|
||||
? await import("./[id]")
|
||||
: { PATCH: undefined as any, DELETE: undefined as any };
|
||||
: { GET: undefined as any, PATCH: undefined as any, DELETE: undefined as any };
|
||||
|
||||
function get() {
|
||||
return GET({
|
||||
params: { id: "org-1" },
|
||||
request: new Request("http://localhost/api/organizations/org-1"),
|
||||
locals: { session: { userId: "user-1" } },
|
||||
} as any);
|
||||
}
|
||||
|
||||
function patch(body: Record<string, unknown>) {
|
||||
return PATCH({
|
||||
@@ -130,6 +141,7 @@ function remove() {
|
||||
|
||||
beforeEach(() => {
|
||||
deleteWhereValues = [];
|
||||
selectWhereValues = [];
|
||||
sourceRows = [
|
||||
{
|
||||
id: "source-1",
|
||||
@@ -246,3 +258,30 @@ describe.skipIf(!isChild)("DELETE /api/organizations/:id with a source pin", ()
|
||||
expect(repoDeleteLeaves()).not.toContain("source-gone");
|
||||
});
|
||||
});
|
||||
|
||||
describe.skipIf(!isChild)("GET /api/organizations/:id", () => {
|
||||
test("returns the organization, looked up by id and owner", async () => {
|
||||
orgRows[0].status = "mirrored";
|
||||
orgRows[0].membershipRole = "member";
|
||||
orgRows[0].lastMirrored = null;
|
||||
orgRows[0].errorMessage = null;
|
||||
|
||||
const response = await get();
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
const data = await response.json();
|
||||
expect(data.success).toBe(true);
|
||||
expect(data.organization).toMatchObject({ id: "org-1", name: "acme", status: "mirrored" });
|
||||
expect(data.organization).not.toHaveProperty("lastMirrored");
|
||||
expect(selectWhereValues[0]).toEqual(expect.arrayContaining(["org-1", "user-1"]));
|
||||
});
|
||||
|
||||
test("404s when the organization is not this user's", async () => {
|
||||
orgRows = [];
|
||||
|
||||
const response = await get();
|
||||
|
||||
expect(response.status).toBe(404);
|
||||
expect((await response.json()).error).toBe("Organization not found");
|
||||
});
|
||||
});
|
||||
@@ -6,6 +6,8 @@ import { createSecureErrorResponse } from "@/lib/utils";
|
||||
import { requireAuth } from "@/lib/utils/auth-helpers";
|
||||
import { mirrorOverridesSchema } from "@/lib/db/schema";
|
||||
import { normalizeMirrorOverrides } from "@/lib/utils/mirror-overrides";
|
||||
import { membershipRoleEnum } from "@/types/organizations";
|
||||
import { repoStatusEnum } from "@/types/Repository";
|
||||
|
||||
/**
|
||||
* Partial update. Each field is optional and only written when present in the
|
||||
@@ -17,6 +19,56 @@ const patchBodySchema = z.object({
|
||||
sourceId: z.string().nullable().optional(),
|
||||
});
|
||||
|
||||
/**
|
||||
* One tracked organization, so a client watching an organization mirror does
|
||||
* not have to fetch every row. The public, private and fork breakdowns stay
|
||||
* on GET /api/github/organizations, which computes them per request.
|
||||
*/
|
||||
export const GET: APIRoute = async (context) => {
|
||||
try {
|
||||
const { user, response } = await requireAuth(context);
|
||||
if (response) return response;
|
||||
|
||||
const userId = user!.id;
|
||||
const orgId = context.params.id;
|
||||
if (!orgId) {
|
||||
return new Response(JSON.stringify({ error: "Organization ID is required" }), {
|
||||
status: 400,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
const [org] = await db
|
||||
.select()
|
||||
.from(organizations)
|
||||
.where(and(eq(organizations.id, orgId), eq(organizations.userId, userId)))
|
||||
.limit(1);
|
||||
|
||||
if (!org) {
|
||||
return new Response(JSON.stringify({ error: "Organization not found" }), {
|
||||
status: 404,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: true,
|
||||
organization: {
|
||||
...org,
|
||||
status: repoStatusEnum.parse(org.status),
|
||||
membershipRole: membershipRoleEnum.parse(org.membershipRole),
|
||||
lastMirrored: org.lastMirrored ?? undefined,
|
||||
errorMessage: org.errorMessage ?? undefined,
|
||||
},
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
} catch (error) {
|
||||
return createSecureErrorResponse(error, "organization fetch", 500);
|
||||
}
|
||||
};
|
||||
|
||||
export const PATCH: APIRoute = async (context) => {
|
||||
try {
|
||||
// Check authentication
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
/**
|
||||
* Route tests for GET /api/repositories/:id: the single-repository lookup
|
||||
* integrations poll after starting a mirror.
|
||||
*
|
||||
* NOTE: these tests replace @/lib/db and @/lib/utils/auth-helpers with mocks.
|
||||
* bun's mock.module is process-wide and leaks into other test files, so in
|
||||
* the shared test process this file registers NOTHING: it re-runs itself in
|
||||
* an isolated child process (the same harness as organizations/[id].test.ts).
|
||||
*/
|
||||
import { describe, test, expect, mock, beforeEach } from "bun:test";
|
||||
|
||||
const CHILD_FLAG = "GM_REPO_ID_ROUTE_ISOLATED";
|
||||
const isChild = !!process.env[CHILD_FLAG];
|
||||
|
||||
if (!isChild) {
|
||||
test("repository [id] route — isolated child suite", () => {
|
||||
const res = Bun.spawnSync({
|
||||
cmd: [process.execPath, "test", import.meta.path],
|
||||
env: { ...process.env, [CHILD_FLAG]: "1" },
|
||||
stdout: "pipe",
|
||||
stderr: "pipe",
|
||||
});
|
||||
if (res.exitCode !== 0) {
|
||||
console.error(res.stdout.toString());
|
||||
console.error(res.stderr.toString());
|
||||
}
|
||||
expect(res.exitCode).toBe(0);
|
||||
}, 60_000);
|
||||
}
|
||||
|
||||
let repoRows: any[] = [];
|
||||
/** Leaf values of each select's where() condition (drizzle conditions are cyclic). */
|
||||
let selectWhereValues: unknown[][] = [];
|
||||
|
||||
function conditionLeafValues(node: any, depth = 0): unknown[] {
|
||||
if (node == null || depth > 8) return [];
|
||||
if (typeof node !== "object") return [node];
|
||||
const leaves: unknown[] = [];
|
||||
if (Array.isArray(node.value)) leaves.push(...node.value);
|
||||
for (const chunk of node.queryChunks ?? []) {
|
||||
leaves.push(...conditionLeafValues(chunk, depth + 1));
|
||||
}
|
||||
return leaves;
|
||||
}
|
||||
|
||||
if (isChild) {
|
||||
mock.module("@/lib/utils/auth-helpers", () => ({
|
||||
requireAuth: mock(async () => ({ user: { id: "user-1" }, response: null })),
|
||||
}));
|
||||
|
||||
mock.module("@/lib/db", () => ({
|
||||
db: {
|
||||
select: () => ({
|
||||
from: () => ({
|
||||
where: (cond: any) => {
|
||||
selectWhereValues.push(conditionLeafValues(cond));
|
||||
return { limit: async () => repoRows.slice(0, 1) };
|
||||
},
|
||||
}),
|
||||
}),
|
||||
},
|
||||
organizations: {},
|
||||
repositories: {},
|
||||
configs: {},
|
||||
users: {},
|
||||
events: {},
|
||||
mirrorJobs: {},
|
||||
sessions: {},
|
||||
accounts: {},
|
||||
verificationTokens: {},
|
||||
verifications: {},
|
||||
oauthClients: {},
|
||||
oauthAccessTokens: {},
|
||||
oauthRefreshTokens: {},
|
||||
oauthConsents: {},
|
||||
jwkss: {},
|
||||
ssoProviders: {},
|
||||
apikeys: {},
|
||||
rateLimits: {},
|
||||
sources: {},
|
||||
}));
|
||||
}
|
||||
|
||||
const { GET } = isChild ? await import("./[id]") : { GET: undefined as any };
|
||||
|
||||
function get() {
|
||||
return GET({
|
||||
params: { id: "repo-1" },
|
||||
request: new Request("http://localhost/api/repositories/repo-1"),
|
||||
locals: { session: { userId: "user-1" } },
|
||||
} as any);
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
selectWhereValues = [];
|
||||
repoRows = [
|
||||
{
|
||||
id: "repo-1",
|
||||
userId: "user-1",
|
||||
name: "hello-world",
|
||||
fullName: "octocat/hello-world",
|
||||
organization: null,
|
||||
status: "mirroring",
|
||||
visibility: "public",
|
||||
lastMirrored: null,
|
||||
errorMessage: null,
|
||||
forkedFrom: null,
|
||||
},
|
||||
];
|
||||
});
|
||||
|
||||
describe.skipIf(!isChild)("GET /api/repositories/:id", () => {
|
||||
test("returns the repository, looked up by id and owner", async () => {
|
||||
const response = await get();
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
const data = await response.json();
|
||||
expect(data.success).toBe(true);
|
||||
expect(data.repository).toMatchObject({
|
||||
id: "repo-1",
|
||||
fullName: "octocat/hello-world",
|
||||
status: "mirroring",
|
||||
});
|
||||
// Same shape as the list endpoint: nulls become absent fields.
|
||||
expect(data.repository).not.toHaveProperty("lastMirrored");
|
||||
expect(data.repository).not.toHaveProperty("organization");
|
||||
expect(selectWhereValues[0]).toEqual(expect.arrayContaining(["repo-1", "user-1"]));
|
||||
});
|
||||
|
||||
test("404s when the repository is not this user's", async () => {
|
||||
repoRows = [];
|
||||
|
||||
const response = await get();
|
||||
|
||||
expect(response.status).toBe(404);
|
||||
expect((await response.json()).error).toBe("Repository not found");
|
||||
});
|
||||
});
|
||||
@@ -6,6 +6,7 @@ import { createSecureErrorResponse } from "@/lib/utils";
|
||||
import { requireAuth } from "@/lib/utils/auth-helpers";
|
||||
import { mirrorOverridesSchema } from "@/lib/db/schema";
|
||||
import { normalizeMirrorOverrides } from "@/lib/utils/mirror-overrides";
|
||||
import { repositoryVisibilityEnum, repoStatusEnum } from "@/types/Repository";
|
||||
|
||||
/**
|
||||
* Partial update. Each field is optional and only written when present in the
|
||||
@@ -17,6 +18,57 @@ const patchBodySchema = z.object({
|
||||
mirrorOverrides: mirrorOverridesSchema.nullable().optional(),
|
||||
});
|
||||
|
||||
/**
|
||||
* One tracked repository, in the shape GET /api/github/repositories lists it,
|
||||
* so a client watching a single mirror does not have to fetch every row.
|
||||
*/
|
||||
export const GET: APIRoute = async (context) => {
|
||||
try {
|
||||
const { user, response } = await requireAuth(context);
|
||||
if (response) return response;
|
||||
|
||||
const userId = user!.id;
|
||||
const repoId = context.params.id;
|
||||
if (!repoId) {
|
||||
return new Response(JSON.stringify({ error: "Repository ID is required" }), {
|
||||
status: 400,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
const [repo] = await db
|
||||
.select()
|
||||
.from(repositories)
|
||||
.where(and(eq(repositories.id, repoId), eq(repositories.userId, userId)))
|
||||
.limit(1);
|
||||
|
||||
if (!repo) {
|
||||
return new Response(JSON.stringify({ error: "Repository not found" }), {
|
||||
status: 404,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: true,
|
||||
repository: {
|
||||
...repo,
|
||||
organization: repo.organization ?? undefined,
|
||||
lastMirrored: repo.lastMirrored ?? undefined,
|
||||
errorMessage: repo.errorMessage ?? undefined,
|
||||
forkedFrom: repo.forkedFrom ?? undefined,
|
||||
status: repoStatusEnum.parse(repo.status),
|
||||
visibility: repositoryVisibilityEnum.parse(repo.visibility),
|
||||
},
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
} catch (error) {
|
||||
return createSecureErrorResponse(error, "repository fetch", 500);
|
||||
}
|
||||
};
|
||||
|
||||
export const PATCH: APIRoute = async (context) => {
|
||||
try {
|
||||
// Check authentication
|
||||
|
||||
@@ -574,3 +574,75 @@ describe.skipIf(!isChild)("POST /api/sync/organization public add-org", () => {
|
||||
expect(sourceRows).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe.skipIf(!isChild)("POST /api/sync/organization for integrations", () => {
|
||||
test("a request without role adds the organization as member", async () => {
|
||||
const response = await postOrg({ org: "acme", provider: "gitlab" });
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(insertedOrgRecords).toHaveLength(1);
|
||||
expect(insertedOrgRecords[0].membershipRole).toBe("member");
|
||||
});
|
||||
|
||||
test("an unknown role is rejected with 400 before anything is created", async () => {
|
||||
const response = await postOrg({ org: "acme", role: "maintainer", provider: "gitlab" });
|
||||
|
||||
expect(response.status).toBe(400);
|
||||
const data = await response.json();
|
||||
expect(data.error).toBe("role must be one of member, admin, owner, billing_manager");
|
||||
expect(sourceRows).toHaveLength(0);
|
||||
expect(insertedOrgRecords).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("the 409 for a tracked organization carries its record", async () => {
|
||||
orgRows = [
|
||||
{
|
||||
id: "org-1",
|
||||
userId: "user-1",
|
||||
name: "acme",
|
||||
normalizedName: "acme",
|
||||
membershipRole: "admin",
|
||||
sourceId: null,
|
||||
},
|
||||
];
|
||||
|
||||
const response = await postOrg({ org: "Acme" });
|
||||
|
||||
expect(response.status).toBe(409);
|
||||
const data = await response.json();
|
||||
expect(data.success).toBe(false);
|
||||
expect(data.organization).toMatchObject({ id: "org-1", name: "acme" });
|
||||
});
|
||||
|
||||
test("force:true keeps the stored role unless the request names one", async () => {
|
||||
sourceRows = [makeSourceRow({ username: "octocat", token: "enc:gh-token" })];
|
||||
orgRows = [
|
||||
{
|
||||
id: "org-1",
|
||||
userId: "user-1",
|
||||
name: "acme",
|
||||
normalizedName: "acme",
|
||||
membershipRole: "owner",
|
||||
sourceId: null,
|
||||
},
|
||||
];
|
||||
|
||||
const kept = await postOrg({ org: "acme", force: true });
|
||||
expect(kept.status).toBe(200);
|
||||
expect(orgRows[0].membershipRole).toBe("owner");
|
||||
|
||||
const changed = await postOrg({ org: "acme", role: "admin", force: true });
|
||||
expect(changed.status).toBe(200);
|
||||
expect(orgRows[0].membershipRole).toBe("admin");
|
||||
});
|
||||
|
||||
test("an organization the source does not know points at user accounts", async () => {
|
||||
orgMetadataResult = null;
|
||||
|
||||
const response = await postOrg({ org: "torvalds", provider: "github" });
|
||||
|
||||
expect(response.status).toBe(404);
|
||||
const data = await response.json();
|
||||
expect(data.error).toContain("A user account cannot be added as an organization");
|
||||
});
|
||||
});
|
||||
@@ -2,10 +2,12 @@ import type { APIRoute } from "astro";
|
||||
import { configs, db, organizations, repositories } from "@/lib/db";
|
||||
import { and, eq } from "drizzle-orm";
|
||||
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
|
||||
import type {
|
||||
AddOrganizationApiRequest,
|
||||
AddOrganizationApiResponse,
|
||||
GitOrg,
|
||||
import {
|
||||
membershipRoleEnum,
|
||||
type AddOrganizationApiRequest,
|
||||
type AddOrganizationApiResponse,
|
||||
type GitOrg,
|
||||
type MembershipRole,
|
||||
} from "@/types/organizations";
|
||||
import { v4 as uuidv4 } from "uuid";
|
||||
import {
|
||||
@@ -31,7 +33,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
const userId = authResult.userId;
|
||||
|
||||
const body: AddOrganizationApiRequest = await request.json();
|
||||
const { role, org, force = false } = body;
|
||||
const { org, force = false } = body;
|
||||
if (body.sourceId !== undefined && typeof body.sourceId !== "string") {
|
||||
return jsonResponse({
|
||||
data: { success: false, error: "sourceId must be a string" },
|
||||
@@ -53,13 +55,31 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
});
|
||||
}
|
||||
|
||||
if (!org || !role) {
|
||||
if (!org) {
|
||||
return jsonResponse({
|
||||
data: { success: false, error: "Missing org or role" },
|
||||
data: { success: false, error: "Missing org" },
|
||||
status: 400,
|
||||
});
|
||||
}
|
||||
|
||||
// The role only labels the organization in the UI, and integrations
|
||||
// rarely know it: a new organization without one is added as "member",
|
||||
// and a forced re-add without one keeps the stored role.
|
||||
let role: MembershipRole | undefined;
|
||||
if (body.role !== undefined) {
|
||||
const parsedRole = membershipRoleEnum.safeParse(body.role);
|
||||
if (!parsedRole.success) {
|
||||
return jsonResponse({
|
||||
data: {
|
||||
success: false,
|
||||
error: `role must be one of ${membershipRoleEnum.options.join(", ")}`,
|
||||
},
|
||||
status: 400,
|
||||
});
|
||||
}
|
||||
role = parsedRole.data;
|
||||
}
|
||||
|
||||
const trimmedOrg = org.trim();
|
||||
const normalizedOrg = trimmedOrg.toLowerCase();
|
||||
|
||||
@@ -102,11 +122,14 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
// The 409 carries the stored record, so a client can go straight to
|
||||
// mirroring it without listing every organization to find the id.
|
||||
if (existingOrg && !force) {
|
||||
return jsonResponse({
|
||||
data: {
|
||||
success: false,
|
||||
error: "Organization already exists for this user",
|
||||
organization: existingOrg,
|
||||
},
|
||||
status: 409,
|
||||
});
|
||||
@@ -164,7 +187,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
const [updatedOrg] = await db
|
||||
.update(organizations)
|
||||
.set({
|
||||
membershipRole: role,
|
||||
...(role ? { membershipRole: role } : {}),
|
||||
normalizedName: normalizedOrg,
|
||||
// Only a request that named a source (an explicit pin, or public
|
||||
// mode's provider) re-pins; a bare force re-add must leave the
|
||||
@@ -189,6 +212,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
data: {
|
||||
success: false,
|
||||
error: "Organization already exists for this user",
|
||||
organization: existingOrg,
|
||||
},
|
||||
status: 409,
|
||||
});
|
||||
@@ -236,7 +260,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
return jsonResponse({
|
||||
data: {
|
||||
success: false,
|
||||
error: `Organization ${trimmedOrg} was not found on the configured source`,
|
||||
error: `Organization ${trimmedOrg} was not found on the configured source. A user account cannot be added as an organization; add its repositories one at a time instead.`,
|
||||
},
|
||||
status: 404,
|
||||
});
|
||||
@@ -248,7 +272,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
const orgData: GitOrg = orgMetadata ?? {
|
||||
name: trimmedOrg,
|
||||
avatarUrl: "",
|
||||
membershipRole: role,
|
||||
membershipRole: role ?? "member",
|
||||
isIncluded: false,
|
||||
status: "imported",
|
||||
repositoryCount: orgRepos.length,
|
||||
@@ -314,7 +338,7 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
normalizedName: normalizedOrg,
|
||||
// Column is NOT NULL; the rate-limit fallback record carries "".
|
||||
avatarUrl: orgData.avatarUrl || "",
|
||||
membershipRole: role,
|
||||
membershipRole: role ?? "member",
|
||||
isIncluded: false,
|
||||
sourceId: source.id,
|
||||
status: "imported" as const,
|
||||
|
||||
@@ -127,12 +127,15 @@ export const POST: APIRoute = async ({ request, locals }) => {
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
// The 409 carries the stored record, so a client can go straight to
|
||||
// mirroring it without listing every repository to find the id.
|
||||
if (existingRepo && !force) {
|
||||
return jsonResponse({
|
||||
data: {
|
||||
success: false,
|
||||
error:
|
||||
"Repository with this name and owner already exists for this user",
|
||||
repository: existingRepo,
|
||||
},
|
||||
status: 409,
|
||||
});
|
||||
|
||||
@@ -45,7 +45,8 @@ export interface GitOrg {
|
||||
export interface AddOrganizationApiRequest {
|
||||
userId: string;
|
||||
org: string;
|
||||
role: MembershipRole;
|
||||
/** The account's role in the organization; defaults to "member". */
|
||||
role?: MembershipRole;
|
||||
force?: boolean;
|
||||
/** Which connected source to import from; defaults to the primary source. */
|
||||
sourceId?: string;
|
||||
|
||||
@@ -33,6 +33,14 @@ A missing, malformed, expired or revoked key gets `401 Unauthorized`:
|
||||
{ "success": false, "error": "Unauthorized" }
|
||||
```
|
||||
|
||||
Send `Content-Type: application/json` on every `POST`, `PUT`, `PATCH` and `DELETE`, also when the call has no body. Without it a script gets `403` with `Cross-site POST form submissions are forbidden`: the app's protection against cross-site form posts treats a write with no content type, or a form content type, as a form unless its `Origin` header matches the app. curl's `-d` sends a form content type unless you set the header.
|
||||
|
||||
A path or method that has no endpoint answers `404` with a JSON body naming both, so it reads differently from a `404` for an id or name that was not found:
|
||||
|
||||
```json
|
||||
{ "success": false, "error": "No API endpoint for GET /api/organizations. Check the path and the HTTP method." }
|
||||
```
|
||||
|
||||
If the app is served under a path prefix (`BASE_URL=/mirror`), prefix every URL with it: `/mirror/api/...`.
|
||||
|
||||
## Endpoints
|
||||
@@ -74,6 +82,8 @@ Despite the name it lists every repository the account tracks, whatever the conf
|
||||
|
||||
`status` is one of `imported`, `mirroring`, `mirrored`, `failed`, `syncing`, `synced`, `skipped`, `deleted`, `archived`.
|
||||
|
||||
`GET /api/repositories/{id}` returns one of them in the same shape as `{ "success": true, "repository": { ... } }`, or `404` when the id is not this account's. Use it to watch a single mirror without fetching the whole list.
|
||||
|
||||
### Add a repository
|
||||
|
||||
`POST /api/sync/repository`
|
||||
@@ -84,14 +94,16 @@ Despite the name it lists every repository the account tracks, whatever the conf
|
||||
|
||||
Optional fields: `destinationOrg` to override the destination organization for this repository, and `force: true` to refresh the metadata of a repository that is already tracked.
|
||||
|
||||
When the request comes from a pasted URL, also send its host as `host` and its path segments as `path`, for example `["octocat", "hello-world", "tree", "main"]`. The host picks the connected source with that host (`400` when none has it), and the source reads the repository from the segments: GitHub and Gitea take the first two, GitLab takes the nested groups up to `/-/`. `owner` and `repo` are still required.
|
||||
|
||||
The repository is looked up on the configured source and added with status `imported`. It is not mirrored until you start a mirror (next call) or the scheduler picks it up.
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `200` | Added. `repository.id` is what the mirror call needs. |
|
||||
| `400` | `owner` or `repo` missing. |
|
||||
| `400` | `owner` or `repo` missing, `host` matches no connected source, or no source is connected. |
|
||||
| `404` | No configuration for this account, or the repository does not exist on the source. |
|
||||
| `409` | Already tracked. Send `force: true` to refresh it instead. |
|
||||
| `409` | Already tracked. `repository` is the stored record, so `repository.id` works as on `200`. Send `force: true` to refresh its metadata instead. |
|
||||
|
||||
### Start a mirror
|
||||
|
||||
@@ -101,7 +113,7 @@ The repository is looked up on the configured source and added with status `impo
|
||||
{ "repositoryIds": ["9b2f..."] }
|
||||
```
|
||||
|
||||
Starts mirroring in the background and returns straight away. Poll the list endpoint to watch `status` move from `mirroring` to `mirrored` or `failed`.
|
||||
Starts mirroring in the background and returns straight away. Poll `GET /api/repositories/{id}`, or the list, to watch `status` move from `mirroring` to `mirrored` or `failed`.
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
@@ -116,10 +128,21 @@ Starts mirroring in the background and returns straight away. Poll the list endp
|
||||
`POST /api/sync/organization` adds an organization or group the same way:
|
||||
|
||||
```json
|
||||
{ "org": "my-org", "role": "member" }
|
||||
{ "org": "my-org" }
|
||||
```
|
||||
|
||||
`role` is the account's role in that organization (`member`, `admin`, `owner` or `billing_manager`). `409` means it is already tracked; `force: true` refreshes it. Then `POST /api/job/mirror-org` with `{ "organizationIds": ["..."] }` mirrors its repositories.
|
||||
`role` is optional. It is the account's role in that organization (`member`, `admin`, `owner` or `billing_manager`) and only labels the organization on the Organizations page. Without it a new organization is added as `member`, and a forced re-add keeps the stored role.
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `200` | Added, with every repository the source lists in it. `organization.id` is what the mirror call needs. |
|
||||
| `400` | `org` missing, an unknown `role` or `provider`, or no source is connected. |
|
||||
| `404` | No configuration for this account, or the source has no organization of that name. A user account is not an organization: add its repositories one at a time with `POST /api/sync/repository`. |
|
||||
| `409` | Already tracked. `organization` is the stored record. Send `force: true` to refresh it instead. |
|
||||
|
||||
Then `POST /api/job/mirror-org` with `{ "organizationIds": ["..."] }` mirrors its repositories. It takes ids, not names, and answers `404` when none of them belong to this account.
|
||||
|
||||
`GET /api/github/organizations` lists every tracked organization with its `id`, `name`, `status` and repository counts. `GET /api/organizations/{id}` returns one as `{ "success": true, "organization": { ... } }`, without the public, private and fork counts, or `404` when the id is not this account's. An organization's `status` turns `mirrored` once every repository in it has been processed; each repository keeps its own `status`.
|
||||
|
||||
When several sources are connected, an optional `sourceId` picks which one the organization imports and mirrors from (the first connected source is the default):
|
||||
|
||||
@@ -143,7 +166,7 @@ Compares what the destination (Gitea or Forgejo) holds with the repositories thi
|
||||
{ "dryRun": true, "adoptUntracked": false, "resetMissing": false }
|
||||
```
|
||||
|
||||
The body is optional and every field defaults to the value above, so an empty `POST` is a dry run. With `dryRun: false`:
|
||||
The body is optional and every field defaults to the value above, so an empty `POST` (with the JSON content type, see [Send the key](#send-the-key)) is a dry run. With `dryRun: false`:
|
||||
|
||||
- `adoptUntracked: true` creates a row for each untracked mirror from its source URL, so scheduled sync and the cleanup service include it from then on. The row keeps the mirror where it is, even when the strategy would put it elsewhere.
|
||||
- `resetMissing: true` sets each missing row back to `imported` so the next mirror run recreates the mirror. Rows are never deleted.
|
||||
@@ -201,6 +224,8 @@ curl -sS -X POST "$BASE/api/job/mirror-repo" \
|
||||
-d "{\"repositoryIds\":[\"$id\"]}"
|
||||
```
|
||||
|
||||
`.repository.id` is set on both `200` and `409`, so the script works the same for a repository that is already tracked.
|
||||
|
||||
## Managing keys from the API
|
||||
|
||||
Manage keys through the settings card or with a browser session. The key endpoints need a session cookie, not a key, so a leaked key cannot mint more keys. Like every cookie-authenticated `POST` to `/api/auth/*` they also need an `Origin` header; browsers send it on their own, a script must add `Origin: https://mirror.example.com`. The endpoints live under `/api/auth/api-key/`:
|
||||
|
||||
Reference in new issue
Block a user