core/auth/ui/e2e/README.md

# auth-ui native end-to-end tests

This directory contains the opt-in, serial browser suite for `auth-ui`. It builds the current application, starts native Ory processes with in-memory SQLite databases, and drives the existing UI with Playwright-managed headless Chromium. Docker and PostgreSQL are not prerequisites.

## Pinned runtime

The harness accepts no `latest` or system-browser fallback:

- Go **1.22 or newer** (the module directive is exactly `go 1.22`)
- Kratos **v1.1.0**, SQLite release archive
- Hydra **v2.2.0**, SQLite release archive
- `github.com/mxschmitt/playwright-go` **v0.6100.0**
- embedded Playwright CLI **1.61.1**
- managed Chromium revision **1228**, Chromium **149.0.7827.55**
- managed FFmpeg revision **1011**

The Playwright binding, CLI, browser, and installer command are one matched version set. The suite never selects Chrome from `PATH`.

### Ory archives and checksums

Hashes are pinned in `artifacts.go` from the official Kratos and Hydra release `checksums.txt` files.

| Platform | Kratos archive / SHA-256 | Hydra archive / SHA-256 |
|---|---|---|
| linux/amd64 | `kratos_1.1.0-linux_sqlite_64bit.tar.gz` / `6fb3077252dde7578c3100d2cd4eb52364ca6b3c1b0b76987e6d586e29008cbd` | `hydra_2.2.0-linux_sqlite_64bit.tar.gz` / `0fe0539fa452496ac5d98b558f93eb2dbb4cf43733da0b09f8f2bdb4445fc31e` |
| linux/arm64 | `kratos_1.1.0-linux_sqlite_arm64.tar.gz` / `fde8a1a1aebd153baff88b1232e0c2a34fdaaafe90b5364f4ea580151e74898e` | `hydra_2.2.0-linux_sqlite_arm64.tar.gz` / `c499ffdaae0f2ab85eff0567214734515b741a393bef89115c16018f4dc0560d` |
| darwin/amd64 | `kratos_1.1.0-macOS_sqlite_64bit.tar.gz` / `ebdc94f27cb6e6a3087ed756accfb7837465ac8e30af9433b4414101814f7769` | `hydra_2.2.0-macOS_sqlite_64bit.tar.gz` / `3d40ca8e99e2a6d840130928d5e0245212dba0eea9c26a0d7186ebb4382e673d` |
| darwin/arm64 | `kratos_1.1.0-macOS_sqlite_arm64.tar.gz` / `6681d7b15dd04686d10764750ce3ad69672b3962553223399a3a315ba5370517` | `hydra_2.2.0-macOS_sqlite_arm64.tar.gz` / `89732ad1494c57ea39348f62dc5ef5c48de129cd205b17cb12bb67ad27094bb7` |

These four Linux/macOS architecture combinations are the supported selection matrix. Windows and every other `GOOS/GOARCH` fail before downloads, builds, or service startup. Selection and compile checks do not constitute a native real-browser result. The complete native online/offline suite and automated artifact-format checks have been exercised on Linux amd64; Linux arm64 and both macOS rows remain pending and must not be described as empirically green.

## Prerequisites and installation

Required locally:

1. Go 1.22 or newer, with access to the normal Go module cache on first use.
2. GitHub network access for an uncached Ory archive.
3. Playwright-managed Chromium and FFmpeg.
4. On Linux, the host libraries required by Chromium.

Install the exactly matched browser runtime:

From the `auth-ui` checkout directory:

```sh
cd /path/to/repository/auth-ui
make install-e2e-browser
```

On a Linux host that lacks Chromium system libraries, install the browser and Playwright-recommended OS packages (the package-manager step may require privileges):

```sh
make install-e2e-browser-deps
```

No installer runs during ordinary `go test ./...`, `go vet ./...`, `make test`, builds, or image publishing.

## Commands

Fast, untagged tests and vet (no Ory or browser process startup):

```sh
make test
```

Install/check the pinned browser, then run the complete tagged suite:

```sh
make test-e2e
```

Run from already populated caches without invoking the online browser installer target:

```sh
make test-e2e-offline
```

The suite is deliberately serial: it starts one Kratos process, one Hydra process, one fresh `auth-ui` binary, one Playwright driver, and one Chromium process, then creates an isolated browser context for each test. It does not call `t.Parallel`. Make keeps Go's hard `-timeout=10m` contract. The harness starts its **9-minute internal deadline before stack setup**, so the deadline covers archive preparation, application build, service/browser/UI setup, and test execution before timeout cleanup begins, even though Go starts its own alarm inside `m.Run`. The harness therefore begins timeout cleanup at least one minute before Go's alarm. The shorter `AUTH_UI_E2E_WATCHDOG_TIMEOUT` override is reserved for the harness's timeout subprocess regressions; ordinary runs should retain the documented 9-minute deadline. A warm-cache full run is now approximately 60–65 seconds on the implementation Linux amd64 host because the two nested real watchdog regressions deliberately consume most of that duration; first runs and other machines can be slower.

For focused development:

```sh
go test -count=1 ./e2e
go test -tags=e2e -count=1 -run '^TestHydra' -timeout=10m -v ./e2e
```

The first command runs helper unit tests only. The second requires the installed browser and Ory archives.

## Downloads, caches, and offline behavior

On a cache miss, the harness downloads the selected official Kratos and Hydra release archives from GitHub, enforces a size bound, verifies the pinned SHA-256, and publishes atomically. Every cache reuse recomputes the hash.

The default Ory archive cache is repository-local:

```text
e2e/cache/ory/<service>/<version>/<archive>
```

Override its root when needed:

```sh
AUTH_UI_E2E_CACHE_DIR=/absolute/cache make test-e2e
AUTH_UI_E2E_CACHE_DIR=/absolute/cache make test-e2e-offline
```

`AUTH_UI_E2E_OFFLINE=1` disables Ory downloads and reports the missing/invalid path and expected hash. `make test-e2e-offline` intentionally does not depend on `install-e2e-browser`; therefore the Go modules, Playwright driver, Chromium, FFmpeg, Ory archives, and Linux host libraries must already be available. A first-ever run needs network access for every missing Go-module, Ory, Playwright-driver, or browser cache entry.

Playwright's default cache locations are:

| Host | Playwright driver | Managed browsers/FFmpeg |
|---|---|---|
| Linux | `~/.cache/ms-playwright-go/1.61.1` | `~/.cache/ms-playwright` |
| macOS | `~/Library/Caches/ms-playwright-go/1.61.1` | `~/Library/Caches/ms-playwright` |

The Linux paths follow the normal user cache root (for example, `$XDG_CACHE_HOME` when configured). `PLAYWRIGHT_DRIVER_PATH` and `PLAYWRIGHT_BROWSERS_PATH` may point to pre-populated matching caches. Do not point them at a different Playwright release or a system browser. Go honors its standard `GOMODCACHE`/`GOCACHE` settings.

## Workspaces and overrides

Each invocation creates a temporary workspace using the operating system's temporary directory, named like:

```text
<os-temp>/auth-ui-e2e-*/
  bin/{auth-ui,kratos,hydra}
  config/{identity.schema.json,kratos.yml,hydra.yml}
  logs/{auth-ui.log,kratos.log,hydra.log}
```

The workspace is removed after success. It is retained after setup/test failure, and `AUTH_UI_E2E_KEEP_TMP=1` retains it after success. The harness prints retained paths when relevant. All generated service and callback URLs use dynamically allocated `127.0.0.1` ports; no fixed client, identity, port, or external origin is required.

## Retained artifacts and sensitivity

Every tagged test retains one artifact set on success and failure under:

```text
e2e/artifacts/<run-id>/
  run.json
  services/{auth-ui,kratos,hydra}.log
  <test-name>/
    screenshots/*.png
    video.webm
    trace.zip
    session.json
```

Use `AUTH_UI_E2E_ARTIFACT_DIR=/absolute/path` to change the run-artifact root. Screenshots, video, traces, DOM/network snapshots, and upstream logs can contain generated credentials, cookies, OAuth challenges, codes, or tokens. Treat the entire artifact tree as **sensitive local test data**; do not publish or attach it without review. Both the default cache and artifact directories are ignored by Git.

After inspecting representative PNG, WebM, trace, and metadata files, remove only default retained runs with:

```sh
make clean-e2e-artifacts
```

That target deletes exactly `e2e/artifacts/`. It does not clear `e2e/cache`, Playwright caches, an `AUTH_UI_E2E_ARTIFACT_DIR` override, fixtures, or ordinary build output. No test or ordinary clean target deletes artifacts automatically.

## Cleanup and troubleshooting

The harness registers aggregate lifecycle ownership before setup and records partial stack and Playwright ownership as soon as each becomes available. Service creation and cleanup use one ownership lock across the cleanup-state check, OS process start, and slot publication, so cleanup either prevents a new spawn or observes it. After `m.Run`, normal cleanup must atomically stop and claim the watchdog before cleanup begins; if the timer already fired, timeout cleanup owns the terminal status and exit. Browser sessions use the same claim model: a normal finalization claimed before timeout finishes exactly once and is awaited, while a timeout-claimed session is finalized as failed with attempted failure/final screenshots, trace closure, context/video finalization, and failed metadata. Timeout cleanup then closes owned callback listeners, stops services and browser/driver, copies bounded redacted service logs, records `watchdog_timeout` in `run.json`, retains available workspace/artifact paths, and exits nonzero. Both timeout-triggered cleanup and normal post-`m.Run` cleanup give protocol cleanup 45 seconds. If that bound expires, the emergency path directly kills every recorded service group and every discoverable driver/browser descendant without relying on Playwright protocols. Emergency termination has a separate grace of at most 5 seconds, with an unconditional exit 125 guard armed before descendant enumeration so even a blocked external `ps` command cannot delay final exit indefinitely.

This pre-timeout cleanup covers stalls while the Go test process and its cleanup goroutine can still run. It cannot guarantee cleanup after `SIGKILL`, host shutdown, kernel failure, or another abrupt OS-level termination that prevents the harness from executing. After such an interrupted run, inspect the printed workspace and service logs and verify no process whose command references that workspace remains before deleting it.

Common failures:

- **Unsupported E2E platform:** use Linux/macOS on amd64/arm64. Windows is intentionally unsupported.
- **Offline cache missing or invalid:** run `make test-e2e` online once with the same `AUTH_UI_E2E_CACHE_DIR`, or place the exact official archive at the reported path; the hash must match.
- **Playwright driver/browser missing or mismatched:** run `make install-e2e-browser` with the repository's Go environment. Do not substitute system Chrome.
- **Chromium cannot launch on Linux:** run `make install-e2e-browser-deps` with appropriate package-manager privileges.
- **GitHub or Go proxy unavailable:** retry when online or populate the Ory, Playwright, and Go caches in advance.
- **Readiness timeout or early process exit:** inspect `services/*.log` and the retained workspace logs. Port bind conflicts are retried only for owned startup attempts.
- **Trace inspection:** use the trace viewer from the same Playwright 1.61.1 toolchain; traces from this suite are sensitive.
- **Artifact growth:** inspect what is needed, then run the explicit cleanup target. Caches remain reusable offline.

## Intentional non-goals

This suite does not cover or introduce CI, Docker/Compose, PostgreSQL, Windows, system browsers, Firefox/WebKit, parallel stacks, pixel baselines, accessibility audits, recovery, verification, settings UI, MFA, social login, registration-disabled mode, PKCE, refresh tokens, revocation, introspection, consent rejection, Hydra logout, device/client-credentials flows, or a public password-change API. It does not change product handlers, templates, selectors, styles, or static assets.