diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml index 413e0c50..0b00e894 100644 --- a/.github/workflows/python.yml +++ b/.github/workflows/python.yml @@ -31,7 +31,8 @@ on: # tests/test_configure_script.py runs it. - 'configure.sh' - 'web/lib/potato_mesh/config.rb' - # tests/test_dockerfile_sync.py compares these two files. + # tests/test_dockerfile_sync.py compares these two files; + # tests/test_dockerfile_chromium.py runs their Chromium step. - 'Dockerfile' - 'web/Dockerfile' # tests/test_dependency_manifests.py reads the web Gemfile and its lock. diff --git a/CHANGELOG.md b/CHANGELOG.md index 2295fafb..e1741260 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -71,6 +71,7 @@ * CI: the env-surface check runs in pytest whenever a deployment surface changes, and also checks names with digits, names read through helpers, and retired names (SPEC PX6/PX8) * CI: test results reach Codecov also when tests fail; workflows use codecov-action v7 and checkout v7; Dependabot opens up to 20 GitHub Actions update PRs at a time (SPEC DW1/DW2/DW4) * Docker: the root `Dockerfile` builds the same image as `web/Dockerfile`, including Chromium for Open Graph previews (skip it with `--build-arg WITH_OG_IMAGE=0`), the custom pages directory, and the `APP_VERSION` build arg (SPEC DF1) +* Docker: the `linux-armv7` web image has no Chromium, as Alpine no longer builds it for armv7: `/og-image.png` serves its fallback image unless `OG_IMAGE_URL` is set (SPEC RB3) * Docs: `SECURITY.md` says how to report a vulnerability privately (SPEC SEC1) * Web: `/metrics` no longer exports nodes that opted out or that private mode hides; series exported before an opt-out stop at once (SPEC PM1/PM2) * Web: with `PRIVATE=1`, a hidden client (`CLIENT_HIDDEN`) is left out of positions, telemetry, neighbors, traces, destinations, ingestors, the stats counts, the `/version` and well-known freshness hints, the preview image and its node page, not only the node list; a trace it relayed between other nodes keeps its other hops (SPEC HC1-HC4, HC7) diff --git a/Dockerfile b/Dockerfile index 4550a8a5..1e7c3e78 100644 --- a/Dockerfile +++ b/Dockerfile @@ -63,13 +63,20 @@ ENV WITH_OG_IMAGE=${WITH_OG_IMAGE} # Install runtime dependencies. Chromium powers the runtime Open Graph # preview generator; the accompanying font and library packages are the # minimum set required to render the dashboard headlessly on Alpine. +# Alpine builds Chromium for x86_64 and aarch64 only (no armv7 since +# 2026-10-15), so any other architecture builds as with WITH_OG_IMAGE=0 +# (SPEC RB3). RUN apk add --no-cache \ python3 \ sqlite \ tzdata \ curl \ && if [ "$WITH_OG_IMAGE" = "1" ]; then \ - apk add --no-cache chromium nss freetype harfbuzz ttf-freefont; \ + arch="$(apk --print-arch)"; \ + case "$arch" in \ + x86_64|aarch64) apk add --no-cache chromium nss freetype harfbuzz ttf-freefont ;; \ + *) echo "Alpine builds no Chromium for $arch; Open Graph previews use the packaged default image" ;; \ + esac; \ fi # Create non-root user diff --git a/README.md b/README.md index e802d70a..63ce1733 100644 --- a/README.md +++ b/README.md @@ -115,7 +115,7 @@ The web app can be configured with environment variables (defaults shown): | `EVENTS` | `1` | Set to `0` to disable the live-update SSE stream (`GET /api/events`); clients then fall back to polling at the refresh interval. | | `MIN_THREADS` | `16` | Minimum Puma worker threads kept warm. | | `MAX_THREADS` | `96` | Maximum Puma worker threads. Each active `/api/events` SSE stream pins one thread, so keep this above your peak concurrent SSE clients plus API/ingest headroom. | -| `OG_IMAGE_URL` | _unset_ | Absolute `http(s)://` URL for the social preview image; other schemes are ignored. Replaces the generated `/og-image.png`. Use HTTPS - most platforms won't render an HTTP preview. | +| `OG_IMAGE_URL` | _unset_ | Absolute `http(s)://` URL for the social preview image; other schemes are ignored. Replaces the generated `/og-image.png`. Use HTTPS - most platforms won't render an HTTP preview. The `linux-armv7` web image has no Chromium and serves the fallback image unless this is set. | | `OG_IMAGE_NO_SANDBOX` | `0` | Set to `1` to run the Chromium that renders `/og-image.png` without its sandbox. The Docker images set `1`. With `0` where Chromium cannot start its sandbox, no preview is captured and `/og-image.png` serves its fallback image. | | `PAGES_DIR` | `./pages` | The directory for static, custom-content pages. | | `PROM_REPORT_IDS` | _unset_ | Comma-separated node ids to expose as per-node Prometheus gauges. Empty exports none. Opted-out nodes, and hidden clients under `PRIVATE=1`, are never exported. | diff --git a/tests/test_dockerfile_chromium.py b/tests/test_dockerfile_chromium.py new file mode 100644 index 00000000..8e56133e --- /dev/null +++ b/tests/test_dockerfile_chromium.py @@ -0,0 +1,254 @@ +# Copyright © 2025-26 l5yth & contributors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Run the web image's Chromium step for every platform the release builds. + +Alpine 3.24 stopped building Chromium for armv7 (aports 7e8aa9ca, 2026-10-15), +and the v1.0.0 release build failed on the ``linux/arm/v7`` web image with +``chromium (no such package)``. No pull request builds an image, so this test +runs the production stage's ``RUN`` instruction from both Dockerfiles in a +shell, with a fake ``apk`` that answers for one Alpine architecture at a time +from the package sets below. +""" + +from __future__ import annotations + +import os +import re +import subprocess +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parents[1] + +DOCKERFILES = ("web/Dockerfile", "Dockerfile") + +RELEASE_WORKFLOW = ".github/workflows/docker.yml" + +# Docker platform -> the architecture ``apk --print-arch`` reports for it. +APK_ARCH = {"linux/amd64": "x86_64", "linux/arm64": "aarch64", "linux/arm/v7": "armv7"} + +# The Alpine 3.24 packages the production stage installs, per architecture. +# ``chromium`` is built for aarch64 and x86_64 only (its APKBUILD's ``arch``). +_COMMON_PACKAGES = { + "python3", + "sqlite", + "tzdata", + "curl", + "nss", + "freetype", + "harfbuzz", + "ttf-freefont", +} +ALPINE_PACKAGES = { + "x86_64": _COMMON_PACKAGES | {"chromium"}, + "aarch64": _COMMON_PACKAGES | {"chromium"}, + "armv7": set(_COMMON_PACKAGES), +} + +# A POSIX ``apk`` stand-in: ``--print-arch`` prints ``$FAKE_APK_ARCH``; ``add`` +# records each package in ``$FAKE_APK_LOG`` and fails as apk does for a package +# missing from ``$FAKE_APK_PACKAGES`` (space separated). +FAKE_APK = """#!/bin/sh +if [ "$1" = "--print-arch" ]; then + echo "$FAKE_APK_ARCH" + exit 0 +fi +if [ "$1" != "add" ]; then + echo "fake apk: unsupported arguments: $*" >&2 + exit 2 +fi +shift +for package in "$@"; do + case "$package" in -*) continue ;; esac + case " $FAKE_APK_PACKAGES " in + *" $package "*) echo "$package" >> "$FAKE_APK_LOG" ;; + *) + echo "ERROR: unable to select packages:" >&2 + echo " $package (no such package):" >&2 + exit 1 + ;; + esac +done +""" + + +def _instructions(text: str) -> list[tuple[str, str]]: + """Split a Dockerfile into its instructions, each with its build stage. + + Continuation lines (a trailing backslash) are joined the way Docker joins + them, without the backslash and the newline. Comment and blank lines + between instructions are dropped. + + Args: + text: Full contents of a Dockerfile. + + Returns: + ``(stage, instruction)`` pairs in file order, where ``stage`` is the + ``AS`` name of the enclosing ``FROM`` (empty before the first one) and + ``instruction`` the joined instruction text. + """ + + pairs: list[tuple[str, str]] = [] + stage = "" + pending = "" + for line in text.splitlines(): + if not pending and (not line.strip() or line.lstrip().startswith("#")): + continue + if line.endswith("\\"): + pending += line[:-1] + continue + instruction = pending + line + pending = "" + named = re.match(r"\s*FROM\s+\S+\s+AS\s+(\S+)", instruction, re.IGNORECASE) + if named: + stage = named.group(1) + pairs.append((stage, instruction.strip())) + return pairs + + +def _chromium_step(dockerfile: str) -> str: + """Return the shell command of the production stage's Chromium ``RUN``. + + Args: + dockerfile: Repository-relative path of the Dockerfile. + + Returns: + The command after ``RUN``, as Docker passes it to ``/bin/sh -c``. + """ + + text = (REPO_ROOT / dockerfile).read_text(encoding="utf-8") + steps = [ + instruction[len("RUN") :].strip() + for stage, instruction in _instructions(text) + if stage == "production" + and instruction.startswith("RUN ") + and "chromium" in instruction + ] + assert ( + len(steps) == 1 + ), f"{dockerfile}: expected one production RUN installing chromium, found {len(steps)}" + return steps[0] + + +def _release_platforms() -> list[str]: + """Return the platforms the release workflow builds images for. + + Returns: + Each ``platform:`` value of the ``build-and-push`` matrix, in order. + """ + + workflow = (REPO_ROOT / RELEASE_WORKFLOW).read_text(encoding="utf-8") + return re.findall(r"platform:\s*(linux/[\w/]+)", workflow) + + +def _run_step( + command: str, arch: str, with_og_image: str, tmp_path: Path +) -> tuple[subprocess.CompletedProcess, list[str]]: + """Run a ``RUN`` command against the fake ``apk`` for one architecture. + + Args: + command: Shell command of the ``RUN`` instruction. + arch: Alpine architecture the fake ``apk`` answers for. + with_og_image: Value of the ``WITH_OG_IMAGE`` build argument. + tmp_path: Directory for the fake ``apk`` and its install log. + + Returns: + The finished shell process and the packages it installed, in order. + """ + + bin_dir = tmp_path / f"bin-{arch}-{with_og_image}" + bin_dir.mkdir() + apk = bin_dir / "apk" + apk.write_text(FAKE_APK, encoding="utf-8") + apk.chmod(0o755) + log = tmp_path / f"installed-{arch}-{with_og_image}.txt" + log.touch() + env = { + "PATH": f"{bin_dir}{os.pathsep}{os.environ['PATH']}", + "FAKE_APK_ARCH": arch, + "FAKE_APK_LOG": str(log), + "FAKE_APK_PACKAGES": " ".join(sorted(ALPINE_PACKAGES[arch])), + "WITH_OG_IMAGE": with_og_image, + } + process = subprocess.run( + ["/bin/sh", "-c", command], env=env, capture_output=True, text=True, check=False + ) + return process, log.read_text(encoding="utf-8").split() + + +@pytest.mark.parametrize("dockerfile", DOCKERFILES) +def test_chromium_step_builds_on_every_release_platform( + dockerfile: str, tmp_path: Path +) -> None: + """Each release platform builds; Chromium comes where Alpine builds it.""" + + command = _chromium_step(dockerfile) + platforms = _release_platforms() + assert platforms == ["linux/amd64", "linux/arm64", "linux/arm/v7"] + + for platform in platforms: + arch = APK_ARCH[platform] + process, installed = _run_step(command, arch, "1", tmp_path) + + assert process.returncode == 0, f"{dockerfile} on {platform}: {process.stderr}" + assert ("chromium" in installed) is ("chromium" in ALPINE_PACKAGES[arch]), ( + platform, + installed, + ) + assert {"python3", "sqlite", "tzdata", "curl"} <= set(installed), ( + platform, + installed, + ) + + +@pytest.mark.parametrize("dockerfile", DOCKERFILES) +def test_chromium_step_installs_no_chromium_without_og_images( + dockerfile: str, tmp_path: Path +) -> None: + """``WITH_OG_IMAGE=0`` builds every platform without Chromium.""" + + command = _chromium_step(dockerfile) + + for arch in ALPINE_PACKAGES: + process, installed = _run_step(command, arch, "0", tmp_path) + + assert process.returncode == 0, f"{dockerfile} on {arch}: {process.stderr}" + assert installed == ["python3", "sqlite", "tzdata", "curl"], (arch, installed) + + +def test_fake_apk_refuses_a_package_its_architecture_lacks(tmp_path: Path) -> None: + """The stand-in fails like apk, so a step that asks for Chromium on armv7 fails.""" + + process, installed = _run_step( + "apk add --no-cache curl chromium", "armv7", "1", tmp_path + ) + + assert process.returncode == 1 + assert "chromium (no such package)" in process.stderr + assert installed == ["curl"] + + +def test_instructions_join_continuations_and_track_stages() -> None: + """Continuations join without the backslash; each instruction keeps its stage.""" + + text = "# header\nFROM alpine AS builder\nRUN a \\\n b\n\nFROM alpine AS production\n# note\nRUN c\n" + + assert _instructions(text) == [ + ("builder", "FROM alpine AS builder"), + ("builder", "RUN a b"), + ("production", "FROM alpine AS production"), + ("production", "RUN c"), + ] diff --git a/web/Dockerfile b/web/Dockerfile index 54f0459b..157b2166 100644 --- a/web/Dockerfile +++ b/web/Dockerfile @@ -60,13 +60,20 @@ ENV WITH_OG_IMAGE=${WITH_OG_IMAGE} # Install runtime dependencies. Chromium powers the runtime Open Graph # preview generator; the accompanying font and library packages are the # minimum set required to render the dashboard headlessly on Alpine. +# Alpine builds Chromium for x86_64 and aarch64 only (no armv7 since +# 2026-10-15), so any other architecture builds as with WITH_OG_IMAGE=0 +# (SPEC RB3). RUN apk add --no-cache \ python3 \ sqlite \ tzdata \ curl \ && if [ "$WITH_OG_IMAGE" = "1" ]; then \ - apk add --no-cache chromium nss freetype harfbuzz ttf-freefont; \ + arch="$(apk --print-arch)"; \ + case "$arch" in \ + x86_64|aarch64) apk add --no-cache chromium nss freetype harfbuzz ttf-freefont ;; \ + *) echo "Alpine builds no Chromium for $arch; Open Graph previews use the packaged default image" ;; \ + esac; \ fi # Create non-root user diff --git a/web/lib/potato_mesh/application/federation/domain_names.rb b/web/lib/potato_mesh/application/federation/domain_names.rb index c6693057..8fe3121a 100644 --- a/web/lib/potato_mesh/application/federation/domain_names.rb +++ b/web/lib/potato_mesh/application/federation/domain_names.rb @@ -217,7 +217,9 @@ module PotatoMesh # trailing dots, each label that is not ASCII in Punycode (NFKC and # lower case first), and the port kept. A bracketed IPv6 literal keeps # its form here, for FB10 to refuse. Text longer than any hostname, - # and a label longer than 63 characters, do not sanitize. + # and a label longer than 63 characters, do not sanitize. BINARY text + # is read as UTF-8 (SPEC RB1); text that is not valid UTF-8 does not + # sanitize. # # @param value [Object] a domain as received, with an optional port. # @return [String, nil] the canonical domain, nil when it does not @@ -226,6 +228,11 @@ module PotatoMesh text = string_or_nil(value) return nil if text.nil? || text.length > MAX_DOMAIN_TEXT_LENGTH + # Addrinfo#ip_address and Resolv.getname return BINARY text, and so + # does the INSTANCE_DOMAIN detected from them; NFKC refuses that + # encoding whatever the bytes. Read the bytes as UTF-8, as a JSON body + # or a stored row already is. + text = text.dup.force_encoding(Encoding::UTF_8) if text.encoding == Encoding::BINARY match = text.match(/\A(?.*?)(?::(?\d+))?\z/m) host = match[:host] unless host.start_with?("[") diff --git a/web/spec/instance_domain_boot_spec.rb b/web/spec/instance_domain_boot_spec.rb new file mode 100644 index 00000000..2a5ff12b --- /dev/null +++ b/web/spec/instance_domain_boot_spec.rb @@ -0,0 +1,68 @@ +# Copyright © 2025-26 l5yth & contributors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# frozen_string_literal: true + +require "spec_helper" +require "open3" + +# An instance without INSTANCE_DOMAIN takes its domain from the host, and the +# socket and resolver libraries return that text as BINARY +# (Addrinfo#ip_address, Resolv.getname). From v1.0.0-rc6 the canonical form +# (SPEC FB3) refused that encoding, so every production boot without +# INSTANCE_DOMAIN exited 1 with "INSTANCE_DOMAIN could not be determined". +RSpec.describe "An instance domain detected from the host" do + let(:application) { PotatoMesh::Application } + + # Ruby run before the application loads: the host has one private address + # and no reverse DNS, as a container on Docker's default bridge has. + # Addrinfo builds the address, so its text carries the encoding a real + # host's does. + let(:private_host_prelude) do + <<~RUBY + require "socket" + require "resolv" + Socket.singleton_class.define_method(:ip_address_list) { [Addrinfo.ip("10.0.2.15")] } + Resolv.singleton_class.define_method(:getname) { |_address| raise Resolv::ResolvError } + RUBY + end + + it "reads the BINARY text of the socket and resolver libraries in its canonical form (FB3)" do + address = Addrinfo.ip("10.0.2.15").ip_address + + expect(address.encoding).to eq(Encoding::BINARY) + expect(application.federation_canonical_domain(address)).to eq("10.0.2.15") + expect(application.federation_canonical_domain("Mesh.Example.org.:8443".b)).to eq("mesh.example.org:8443") + expect(application.federation_canonical_domain("мэш.рф".b)).to eq("xn--l1ayo.xn--p1ai") + expect(application.federation_canonical_domain("mesh.\xC3.org".b)).to be_nil + end + + it "boots in production with INSTANCE_DOMAIN unset" do + Dir.mktmpdir do |dir| + env = { + "XDG_CONFIG_HOME" => File.join(dir, "config"), "XDG_DATA_HOME" => File.join(dir, "data"), + "FEDERATION" => "0", "APP_ENV" => "production", "RACK_ENV" => "production", "INSTANCE_DOMAIN" => nil, + } + + # The application loads in a child process, as app.rb loads it. + _stdout, stderr, status = Open3.capture3( + env, RbConfig.ruby, "-e", "#{private_host_prelude}require \"./lib/potato_mesh/application\"", + chdir: File.expand_path("..", __dir__), + ) + + expect(stderr).not_to include("INSTANCE_DOMAIN could not be determined") + expect(status.success?).to be(true), stderr + end + end +end