auth-ui native end-to-end tests

This directory contains the opt-in, serial browser suite for auth-ui. It builds the current application, starts native Ory processes with in-memory SQLite databases, and drives the existing UI with Playwright-managed headless Chromium. Docker and PostgreSQL are not prerequisites.

Pinned runtime

The harness accepts no latest or system-browser fallback:

• Go 1.22 or newer (the module directive is exactly go 1.22)
• Kratos v26.2.0, SQLite release archive
• Hydra v26.2.0, SQLite release archive
• github.com/mxschmitt/playwright-go v0.6100.0
• embedded Playwright CLI 1.61.1
• managed Chromium revision 1228, Chromium 149.0.7827.55
• managed FFmpeg revision 1011

The Playwright binding, CLI, browser, and installer command are one matched version set. The suite never selects Chrome from PATH.

Ory archives and checksums

Hashes are pinned in artifacts.go from the official Kratos and Hydra release checksums.txt files.

The Kratos fixture declares version: v26.2.0; executable identity is independently enforced by archive checksum and binary version verification. These in-memory SQLite tests validate auth-ui compatibility, not production database migrations or rollback.

These four Linux/macOS architecture combinations are the supported selection matrix. Windows and every other GOOS/GOARCH fail before downloads, builds, or service startup. Selection and compile checks do not constitute a native real-browser result. The complete native online/offline suite and automated artifact-format checks have been exercised on Linux amd64; Linux arm64 and both macOS rows remain pending and must not be described as empirically green.

Prerequisites and installation

Required locally:

1. Go 1.22 or newer, with access to the normal Go module cache on first use.
2. GitHub network access for an uncached Ory archive.
3. Playwright-managed Chromium and FFmpeg.
4. On Linux, the host libraries required by Chromium.

Install the exactly matched browser runtime:

From the auth-ui checkout directory:

cd /path/to/repository/auth-ui
make install-e2e-browser

On a Linux host that lacks Chromium system libraries, install the browser and Playwright-recommended OS packages (the package-manager step may require privileges):

make install-e2e-browser-deps

No installer runs during ordinary go test ./..., go vet ./..., make test, builds, or image publishing.

Commands

Fast, untagged source gates (no Ory or browser process startup):

make test
make vet
# or run the complete source-only aggregate:
make check

Install/check the pinned browser, then run the complete tagged suite:

make test-e2e

Run from already populated caches without invoking the online browser installer target:

make test-e2e-offline

The suite is deliberately serial: it starts one Kratos process, one Hydra process, one fresh auth-ui binary, one Playwright driver, and one Chromium process, then creates an isolated browser context for each test. It does not call t.Parallel. Make keeps Go's hard -timeout=10m contract. The harness starts its 9-minute internal deadline before stack setup, so the deadline covers archive preparation, application build, service/browser/UI setup, and test execution before timeout cleanup begins, even though Go starts its own alarm inside m.Run. The harness therefore begins timeout cleanup at least one minute before Go's alarm. The shorter AUTH_UI_E2E_WATCHDOG_TIMEOUT override is reserved for the harness's timeout subprocess regressions; ordinary runs should retain the documented 9-minute deadline. A warm-cache full run is now approximately 60–65 seconds on the implementation Linux amd64 host because the two nested real watchdog regressions deliberately consume most of that duration; first runs and other machines can be slower.

For focused development:

go test -count=1 ./e2e
go test -tags=e2e -count=1 -run '^TestHydra' -timeout=10m -v ./e2e

The first command runs helper unit tests only. The second requires the installed browser and Ory archives.

Downloads, caches, and offline behavior

On a cache miss, the harness downloads the selected official Kratos and Hydra release archives from GitHub, enforces a size bound, verifies the pinned SHA-256, and publishes atomically. Every cache reuse recomputes the hash.

The default Ory archive cache is repository-local:

e2e/cache/ory/<service>/<version>/<archive>

Override its root when needed:

AUTH_UI_E2E_CACHE_DIR=/absolute/cache make test-e2e
AUTH_UI_E2E_CACHE_DIR=/absolute/cache make test-e2e-offline

AUTH_UI_E2E_OFFLINE=1 disables Ory downloads and reports the missing/invalid path and expected hash. make test-e2e-offline intentionally does not depend on install-e2e-browser; therefore the Go modules, Playwright driver, Chromium, FFmpeg, Ory archives, and Linux host libraries must already be available. A first-ever run needs network access for every missing Go-module, Ory, Playwright-driver, or browser cache entry.

Playwright's default cache locations are:

The Linux paths follow the normal user cache root (for example, $XDG_CACHE_HOME when configured). PLAYWRIGHT_DRIVER_PATH and PLAYWRIGHT_BROWSERS_PATH may point to pre-populated matching caches. Do not point them at a different Playwright release or a system browser. Go honors its standard GOMODCACHE/GOCACHE settings.

Workspaces and overrides

Each invocation creates a temporary workspace using the operating system's temporary directory, named like:

<os-temp>/auth-ui-e2e-*/
  bin/{auth-ui,kratos,hydra}
  config/{identity.schema.json,kratos.yml,hydra.yml}
  logs/{auth-ui.log,kratos.log,hydra.log}

The workspace is removed after success. It is retained after setup/test failure, and AUTH_UI_E2E_KEEP_TMP=1 retains it after success. The harness prints retained paths when relevant. All generated service and callback URLs use dynamically allocated 127.0.0.1 ports; no fixed client, identity, port, or external origin is required.

Retained artifacts and sensitivity

Every tagged test retains one artifact set on success and failure under:

e2e/artifacts/<run-id>/
  run.json
  services/{auth-ui,kratos,hydra}.log
  <test-name>/
    screenshots/*.png
    video.webm
    trace.zip
    session.json

Use AUTH_UI_E2E_ARTIFACT_DIR=/absolute/path to change the run-artifact root. Screenshots, video, traces, DOM/network snapshots, and upstream logs can contain generated credentials, cookies, OAuth challenges, codes, or tokens. Treat the entire artifact tree as sensitive local test data; do not publish or attach it without review. Both the default cache and artifact directories are ignored by Git.

After inspecting representative PNG, WebM, trace, and metadata files, remove only default retained runs with:

make clean-e2e-artifacts

That target deletes exactly e2e/artifacts/. It does not clear e2e/cache, Playwright caches, an AUTH_UI_E2E_ARTIFACT_DIR override, fixtures, or ordinary build output. No test or ordinary clean target deletes artifacts automatically.

Cleanup and troubleshooting

The harness registers aggregate lifecycle ownership before setup and records partial stack and Playwright ownership as soon as each becomes available. Service creation and cleanup use one ownership lock across the cleanup-state check, OS process start, and slot publication, so cleanup either prevents a new spawn or observes it. After m.Run, normal cleanup must atomically stop and claim the watchdog before cleanup begins; if the timer already fired, timeout cleanup owns the terminal status and exit. Browser sessions use the same claim model: a normal finalization claimed before timeout finishes exactly once and is awaited, while a timeout-claimed session is finalized as failed with attempted failure/final screenshots, trace closure, context/video finalization, and failed metadata. Timeout cleanup then closes owned callback listeners, stops services and browser/driver, copies bounded redacted service logs, records watchdog_timeout in run.json, retains available workspace/artifact paths, and exits nonzero. Both timeout-triggered cleanup and normal post-m.Run cleanup give protocol cleanup 45 seconds. If that bound expires, the emergency path directly kills every recorded service group and every discoverable driver/browser descendant without relying on Playwright protocols. Emergency termination has a separate grace of at most 5 seconds, with an unconditional exit 125 guard armed before descendant enumeration so even a blocked external ps command cannot delay final exit indefinitely.

This pre-timeout cleanup covers stalls while the Go test process and its cleanup goroutine can still run. It cannot guarantee cleanup after SIGKILL, host shutdown, kernel failure, or another abrupt OS-level termination that prevents the harness from executing. After such an interrupted run, inspect the printed workspace and service logs and verify no process whose command references that workspace remains before deleting it.

Common failures:

• Unsupported E2E platform: use Linux/macOS on amd64/arm64. Windows is intentionally unsupported.
• Offline cache missing or invalid: run make test-e2e online once with the same AUTH_UI_E2E_CACHE_DIR, or place the exact official archive at the reported path; the hash must match.
• Playwright driver/browser missing or mismatched: run make install-e2e-browser with the repository's Go environment. Do not substitute system Chrome.
• Chromium cannot launch on Linux: run make install-e2e-browser-deps with appropriate package-manager privileges.
• GitHub or Go proxy unavailable: retry when online or populate the Ory, Playwright, and Go caches in advance.
• Readiness timeout or early process exit: inspect services/*.log and the retained workspace logs. Port bind conflicts are retried only for owned startup attempts.
• Trace inspection: use the trace viewer from the same Playwright 1.61.1 toolchain; traces from this suite are sensitive.
• Artifact growth: inspect what is needed, then run the explicit cleanup target. Caches remain reusable offline.

Intentional non-goals

This suite does not cover or introduce CI, Docker/Compose, PostgreSQL, Windows, system browsers, Firefox/WebKit, parallel stacks, pixel baselines, accessibility audits, recovery, verification, MFA, social login, registration-disabled mode, PKCE, refresh tokens, revocation, introspection, consent rejection, Hydra logout, device/client-credentials flows, or a public password-change API. It does not change product handlers, templates, selectors, styles, or static assets.