feat: add repeatable Web UX inspection workbench

This commit is contained in:
2026-09-01 22:48:26 +09:00
parent 409245cb52
commit 4a4a01b730
20 changed files with 2307 additions and 0 deletions
+2
View File
@@ -5,3 +5,5 @@
.worktree
*.local*
.env
.web-ux/
artifacts/web-ux/
+4
View File
@@ -4,11 +4,15 @@ pkgs.mkShell {
nixfmt
deno
git
playwright-driver.browsers
rustc
cargo
pkgs.sccache
];
PLAYWRIGHT_BROWSERS_PATH = "${pkgs.playwright-driver.browsers}";
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD = "1";
# sccache is additive to Cargo's shared build-dir, so keep its disk usage bounded.
RUSTC_WRAPPER = "${pkgs.sccache}/bin/sccache";
SCCACHE_CACHE_SIZE = "5G";
+172
View File
@@ -0,0 +1,172 @@
# Web UX inspection workbench
`tools/web-ux` is a development-only Playwright workbench for repeatable visual inspection of the
real Web Workspace. It does not add a Yoi product Skill, Flow, Runtime capability, or browser
automation route.
The workbench produces a **review context bundle** rather than treating a screenshot as evidence by
itself. Every capture records the persona, route, viewport, theme, intended user goal, expected data
state, sanitized document URL/status, console/page/request failures, screenshot hashes, an
accessibility snapshot, source revision, and browser version.
## Environment
Enter the repository dev shell. The shell supplies the Nix-pinned Chromium build and sets
`PLAYWRIGHT_BROWSERS_PATH`; Playwright does not download a browser at runtime.
```sh
nix develop
cd tools/web-ux
deno task check
deno task test
deno task test:browser
```
`test:browser` starts a deterministic fixture server owned by the test, captures distinct owner and
non-owner contexts, verifies the review bundle, and proves server/browser cleanup. It must run
inside `nix develop` so it uses the pinned browser.
The npm Playwright version in `deno.json` must match `pkgs.playwright-driver.version` in the pinned
Nixpkgs input. Update both as one toolchain change.
## Scenario contract
Scenarios are reviewed JSON files under `scenarios/`. A scenario fixes:
- personas and whether each uses an isolated anonymous context or a local Playwright storage-state
file;
- explicit routes and user goals;
- expected data state, viewports, theme, locale, timezone, and reduced-motion mode;
- an explicit readiness condition for every route and optional interaction/capture-point conditions;
- selectors and exact environment-derived text that must be redacted;
- optional processes owned by the capture command, including an HTTP readiness URL.
`${UPPER_CASE_ENV}` values are expanded at runtime. URLs with embedded credentials are rejected.
Route readiness is bounded and retried twice; it never relies on a fixed sleep. `network-idle` is
available but should be used only for screens whose contract actually reaches idle. Prefer a stable
screen-owned selector.
`workspace-control-plane.json` expects:
```sh
export WEB_UX_BASE_URL='http://127.0.0.1:5173'
export WORKSPACE_ID='<workspace-id>'
```
## Authentication fixtures
Authentication state is local sensitive material. `.web-ux/` is gitignored, files are written with
mode `0600`, state contents are never copied into a review bundle, and the CLI never prints cookies
or credentials.
For an interactive Passkey/browser login:
```sh
deno task web-ux auth \
--scenario scenarios/workspace-control-plane.json \
--persona owner
```
The command opens Chromium at the configured login route, waits up to five minutes for the
scenario's success URL, saves `storageState`, and closes the browser in `finally`. Repeat for
`non-owner` using a real account with that permission projection.
A test fixture may already provide Playwright-compatible `{ cookies, origins }` state. Import it
without putting its value on the command line:
```sh
deno task web-ux auth \
--scenario scenarios/workspace-control-plane.json \
--persona owner \
--import-state /private/path/owner-state.json
```
Do not place passwords, bearer tokens, private keys, WebAuthn material, or inline cookies in a
scenario, process arguments, a Repository URL, or `redact.text`. `redact.text` is only a final
defense for a secret already supplied through an environment-owned fixture; it is not a credential
transport.
## Capture and inspect
Capture a stable multi-persona bundle:
```sh
deno task web-ux capture \
--scenario scenarios/workspace-control-plane.json \
--output ../../artifacts/web-ux \
--run-id before-change
```
Use filters for a bounded feedback loop:
```sh
deno task web-ux capture \
--scenario scenarios/workspace-control-plane.json \
--output ../../artifacts/web-ux \
--run-id ticket-list-after \
--personas owner,non-owner \
--routes tickets \
--viewports desktop
```
The command exits `2` when it produced evidence but observed UI/tool errors, and exits `1` when
capture itself failed. It continues other route/persona captures after a bounded route failure.
Inspect:
- `review-context.json` for the exact context, hashes, HTTP status, and failures;
- `contact-sheet.png` with an image-capable reviewer for composition, hierarchy, density, clipping,
empty/error states, and permission-specific affordances;
- each `accessibility.md` for landmark/name/state evidence that a screenshot cannot prove;
- `process-logs/` when the scenario owns a server process. Logs are redacted before writing.
The implementing agent must inspect the actual contact sheet (for example with `ViewImage`), record
concrete findings, fix them, recapture under the same persona/route/viewport filters, and inspect
the new evidence. Playwright success alone is not visual acceptance.
## Compare before and after
```sh
deno task web-ux compare \
--before ../../artifacts/web-ux/before-change/review-context.json \
--after ../../artifacts/web-ux/after-change/review-context.json \
--output ../../artifacts/web-ux/before-vs-after
```
`comparison.html` and `comparison.png` show before, after, and pixel diff side by side.
`comparison.json` records changed-pixel counts, dimension mismatches, unmatched capture keys, and
diff hashes. Pixel differences are orientation evidence, not a correctness verdict; explain expected
animation/font/data changes and inspect the actual UI.
Capture keys are stable across runs: `persona / route / viewport / capture-point`. Keep those
identities unchanged when comparing the same user task.
## Process and artifact cleanup
The capture command owns only processes declared in its scenario. It starts them without a shell,
records bounded/redacted output, and terminates the process and descendants on success, capture
failure, or interruption observed by the command. It never stops an existing Yoi Server or Runtime
that it did not start.
Old complete review bundles can be removed without touching auth state or arbitrary directories:
```sh
deno task web-ux cleanup --output ../../artifacts/web-ux --keep 5 --older-than-days 14 --dry-run
deno task web-ux cleanup --output ../../artifacts/web-ux --keep 5 --older-than-days 14
```
Cleanup recognizes only directories containing `review-context.json`. `.web-ux/` and
`artifacts/web-ux/` are ignored by Git. Keep a bundle outside Git or publish it through the approved
immutable artifact channel when durable review evidence is required.
## Adding a scenario
1. Name the concrete user task and expected data state; do not write “looks correct”.
2. Use the smallest persona/route/viewport matrix that proves the intended contract, including
owner/non-owner/anonymous boundaries when permissions affect composition.
3. Choose a screen-owned readiness selector or response. Avoid arbitrary sleeps.
4. Add capture points only for meaningful visual states (initial, expanded detail, error, empty, and
so on).
5. Mark sensitive DOM regions with `[data-web-ux-redact]` or scenario selectors; never use review
artifacts to transport secrets.
6. Run `deno task check`, `deno task test`, one real capture, and inspect `contact-sheet.png` plus
`review-context.json`.
@@ -0,0 +1,99 @@
import { assertEquals, assertRejects } from "@std/assert";
import { join } from "@std/path";
import { capture } from "../src/capture.ts";
async function freePort(): Promise<number> {
const listener = Deno.listen({ hostname: "127.0.0.1", port: 0 });
const port = (listener.addr as Deno.NetAddr).port;
listener.close();
return port;
}
Deno.test("browser smoke captures distinct owner and non-owner evidence and cleans its server", async () => {
const directory = await Deno.makeTempDir();
const port = await freePort();
const baseUrl = `http://127.0.0.1:${port}`;
try {
const authDirectory = join(directory, "auth");
await Deno.mkdir(authDirectory);
for (const persona of ["owner", "non-owner"]) {
await Deno.writeTextFile(
join(authDirectory, `${persona}.json`),
JSON.stringify({
cookies: [{
name: "persona",
value: persona,
domain: "127.0.0.1",
path: "/",
expires: -1,
httpOnly: true,
secure: false,
sameSite: "Lax",
}],
origins: [],
}),
);
}
const scenarioPath = join(directory, "scenario.json");
await Deno.writeTextFile(
scenarioPath,
JSON.stringify({
schemaVersion: 1,
id: "browser-smoke",
title: "Browser smoke",
baseUrl,
personas: [
{ id: "owner", label: "Owner", auth: { kind: "storage-state", path: "auth/owner.json" } },
{
id: "non-owner",
label: "Non-owner",
auth: { kind: "storage-state", path: "auth/non-owner.json" },
},
],
viewports: [{ label: "desktop", width: 1000, height: 700 }],
routes: [{
id: "repositories",
label: "Repositories",
path: "/screen",
goal: "Verify permission-specific composition",
dataState: "Deterministic fixture repository",
ready: { kind: "selector", selector: "main" },
capturePoints: [{ id: "initial", label: "Initial" }],
}],
processes: [{
id: "fixture-server",
command: Deno.execPath(),
args: [
"run",
"--allow-net",
join(Deno.cwd(), "browser-tests/fixture_server.ts"),
String(port),
],
readyUrl: `${baseUrl}/health`,
}],
}),
);
const manifest = await capture({
scenarioPath,
outputDirectory: join(directory, "artifacts"),
runId: "multi-persona",
});
assertEquals(manifest.status, "completed");
assertEquals(manifest.captures.map((item) => item.persona.id), ["owner", "non-owner"]);
assertEquals(manifest.captures.every((item) => item.screenshots.length === 1), true);
assertEquals(manifest.contactSheet.png, "contact-sheet.png");
const runDirectory = join(directory, "artifacts", "multi-persona");
const reviewContext = await Deno.readTextFile(join(runDirectory, "review-context.json"));
assertEquals(reviewContext.includes('"cookies"'), false);
if (Deno.build.os !== "windows") {
const screenshot = join(runDirectory, manifest.captures[0].screenshots[0].path);
assertEquals((await Deno.stat(screenshot)).mode! & 0o777, 0o600);
}
await assertRejects(
() => fetch(`${baseUrl}/health`, { signal: AbortSignal.timeout(500) }),
TypeError,
);
} finally {
await Deno.remove(directory, { recursive: true });
}
});
@@ -0,0 +1,17 @@
const port = Number(Deno.args[0]);
if (!Number.isInteger(port) || port <= 0) throw new Error("port is required");
Deno.serve({ hostname: "127.0.0.1", port }, (request) => {
const url = new URL(request.url);
if (url.pathname === "/health") return new Response("ok");
const cookie = request.headers.get("cookie") ?? "";
const owner = cookie.includes("persona=owner");
const title = owner ? "Owner repository settings" : "Repository settings";
const action = owner
? '<button type="button">Add repository</button>'
: '<p role="note">Ask a Workspace owner to change repository access.</p>';
return new Response(
`<!doctype html><html><head><title>${title}</title><style>body{font:16px system-ui;margin:0}main{max-width:800px;margin:40px auto}header{border-bottom:1px solid #ccc;padding:16px}section{border:1px solid #ccc;padding:20px}button{background:#06c;color:white;padding:10px 20px}</style></head><body><header>Workspace</header><main><h1>${title}</h1><section><h2>main</h2><p>SSH repository access is configured.</p>${action}</section></main></body></html>`,
{ headers: { "content-type": "text/html; charset=utf-8" } },
);
});
+163
View File
@@ -0,0 +1,163 @@
#!/usr/bin/env -S deno run --allow-env --allow-net --allow-read --allow-write --allow-run --allow-sys
import { authenticate, cleanup } from "./src/lifecycle.ts";
import { capture, describeCapture } from "./src/capture.ts";
import { compare } from "./src/compare.ts";
const HELP = `Web UX inspection workbench
Usage:
deno task web-ux auth --scenario <file> --persona <id> [--base-url <url>] [--import-state <file>] [--headless]
deno task web-ux capture --scenario <file> --output <directory> [--base-url <url>] [--run-id <id>] [--personas <ids>] [--routes <ids>] [--viewports <ids>] [--headed]
deno task web-ux compare --before <review-context.json> --after <review-context.json> --output <directory> [--threshold <0..1>]
deno task web-ux cleanup --output <directory> [--keep <count>] [--older-than-days <days>] [--dry-run]
Comma-separate persona, route, and viewport ids. Auth state is local, mode 0600, and must not be committed.
`;
type Arguments = { command: string; values: Map<string, string[]>; flags: Set<string> };
export function parseArguments(args: string[]): Arguments {
const command = args.shift() ?? "help";
const values = new Map<string, string[]>();
const flags = new Set<string>();
for (let index = 0; index < args.length; index++) {
const token = args[index];
if (!token.startsWith("--")) throw new Error(`unexpected argument: ${token}`);
const name = token.slice(2);
const next = args[index + 1];
if (next === undefined || next.startsWith("--")) {
flags.add(name);
} else {
const items = values.get(name) ?? [];
items.push(next);
values.set(name, items);
index++;
}
}
return { command, values, flags };
}
function optional(args: Arguments, name: string): string | undefined {
const values = args.values.get(name);
if (!values) return undefined;
if (values.length !== 1) throw new Error(`--${name} must be specified once`);
return values[0];
}
function required(args: Arguments, name: string): string {
const value = optional(args, name);
if (!value) throw new Error(`--${name} is required`);
return value;
}
function integer(args: Arguments, name: string, fallback?: number): number | undefined {
const value = optional(args, name);
if (value === undefined) return fallback;
const parsed = Number(value);
if (!Number.isInteger(parsed) || parsed < 0) {
throw new Error(`--${name} must be a non-negative integer`);
}
return parsed;
}
function list(args: Arguments, name: string): string[] | undefined {
const value = optional(args, name);
return value?.split(",").map((item) => item.trim()).filter(Boolean);
}
function rejectUnknown(args: Arguments, allowedValues: string[], allowedFlags: string[]): void {
for (const name of args.values.keys()) {
if (!allowedValues.includes(name)) throw new Error(`unsupported option: --${name}`);
}
for (const name of args.flags) {
if (!allowedFlags.includes(name)) throw new Error(`unsupported flag: --${name}`);
}
}
export async function main(rawArgs: string[]): Promise<number> {
const args = parseArguments([...rawArgs]);
if (args.command === "help" || args.flags.has("help")) {
console.log(HELP);
return 0;
}
if (args.command === "auth") {
rejectUnknown(args, ["scenario", "persona", "base-url", "import-state", "timeout-ms"], [
"headless",
]);
const path = await authenticate({
scenarioPath: required(args, "scenario"),
personaId: required(args, "persona"),
baseUrl: optional(args, "base-url"),
importState: optional(args, "import-state"),
timeoutMs: integer(args, "timeout-ms"),
headless: args.flags.has("headless"),
});
console.log(`auth state saved: ${path}`);
return 0;
}
if (args.command === "capture") {
rejectUnknown(args, [
"scenario",
"output",
"base-url",
"run-id",
"personas",
"routes",
"viewports",
], ["headed"]);
const outputDirectory = required(args, "output");
const manifest = await capture({
scenarioPath: required(args, "scenario"),
outputDirectory,
baseUrl: optional(args, "base-url"),
runId: optional(args, "run-id"),
personas: list(args, "personas"),
routes: list(args, "routes"),
viewports: list(args, "viewports"),
headed: args.flags.has("headed"),
});
console.log(describeCapture(manifest, outputDirectory));
return manifest.status === "completed" ? 0 : 2;
}
if (args.command === "compare") {
rejectUnknown(args, ["before", "after", "output", "threshold"], []);
const thresholdValue = optional(args, "threshold");
const threshold = thresholdValue === undefined ? undefined : Number(thresholdValue);
if (
threshold !== undefined && (!Number.isFinite(threshold) || threshold < 0 || threshold > 1)
) {
throw new Error("--threshold must be between 0 and 1");
}
const report = await compare({
before: required(args, "before"),
after: required(args, "after"),
outputDirectory: required(args, "output"),
threshold,
});
console.log(`comparison saved: ${report}`);
return 0;
}
if (args.command === "cleanup") {
rejectUnknown(args, ["output", "keep", "older-than-days"], ["dry-run"]);
const removed = await cleanup({
outputDirectory: required(args, "output"),
keep: integer(args, "keep", 5)!,
olderThanDays: integer(args, "older-than-days"),
dryRun: args.flags.has("dry-run"),
});
for (const path of removed) {
console.log(`${args.flags.has("dry-run") ? "would remove" : "removed"}: ${path}`);
}
return 0;
}
throw new Error(`unknown command: ${args.command}\n\n${HELP}`);
}
if (import.meta.main) {
try {
Deno.exit(await main(Deno.args));
} catch (error) {
console.error(error instanceof Error ? error.message : String(error));
Deno.exit(1);
}
}
+19
View File
@@ -0,0 +1,19 @@
{
"lock": true,
"imports": {
"@std/assert": "jsr:@std/assert@1.0.19",
"@std/path": "jsr:@std/path@1.1.4",
"pixelmatch": "npm:pixelmatch@7.1.0",
"playwright": "npm:playwright@1.59.1",
"pngjs": "npm:pngjs@7.0.0"
},
"tasks": {
"web-ux": "deno run --allow-env --allow-net --allow-read --allow-write --allow-run --allow-sys cli.ts",
"check": "deno check cli.ts src/*.ts tests/*.ts browser-tests/*.ts",
"test": "deno test --allow-env --allow-read --allow-write --allow-run --allow-sys tests",
"test:browser": "deno test --allow-env --allow-net --allow-read --allow-write --allow-run --allow-sys browser-tests/capture_smoke_test.ts"
},
"fmt": {
"lineWidth": 100
}
}
+68
View File
@@ -0,0 +1,68 @@
{
"version": "5",
"specifiers": {
"jsr:@std/assert@1.0.19": "1.0.19",
"jsr:@std/internal@^1.0.12": "1.0.14",
"jsr:@std/path@1.1.4": "1.1.4",
"npm:pixelmatch@7.1.0": "7.1.0",
"npm:playwright@1.59.1": "1.59.1",
"npm:pngjs@7.0.0": "7.0.0"
},
"jsr": {
"@std/assert@1.0.19": {
"integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
"dependencies": [
"jsr:@std/internal"
]
},
"@std/internal@1.0.14": {
"integrity": "291516b3d4c35024d6ffbc0a9df5bf4c64116e05b50012cf846710152d2ffdf7"
},
"@std/path@1.1.4": {
"integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
"dependencies": [
"jsr:@std/internal"
]
}
},
"npm": {
"fsevents@2.3.2": {
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"os": ["darwin"],
"scripts": true
},
"pixelmatch@7.1.0": {
"integrity": "sha512-1wrVzJ2STrpmONHKBy228LM1b84msXDUoAzVEl0R8Mz4Ce6EPr+IVtxm8+yvrqLYMHswREkjYFaMxnyGnaY3Ng==",
"dependencies": [
"pngjs"
],
"bin": true
},
"playwright-core@1.59.1": {
"integrity": "sha512-HBV/RJg81z5BiiZ9yPzIiClYV/QMsDCKUyogwH9p3MCP6IYjUFu/MActgYAvK0oWyV9NlwM3GLBjADyWgydVyg==",
"bin": true
},
"playwright@1.59.1": {
"integrity": "sha512-C8oWjPR3F81yljW9o5OxcWzfh6avkVwDD2VYdwIGqTkl+OGFISgypqzfu7dOe4QNLL2aqcWBmI3PMtLIK233lw==",
"dependencies": [
"playwright-core"
],
"optionalDependencies": [
"fsevents"
],
"bin": true
},
"pngjs@7.0.0": {
"integrity": "sha512-LKWqWJRhstyYo9pGvgor/ivk2w94eSjE3RGVuzLGlr3NmD8bf7RcYGze1mNdEHRP6TRP6rMuDHk5t44hnTRyow=="
}
},
"workspace": {
"dependencies": [
"jsr:@std/assert@1.0.19",
"jsr:@std/path@1.1.4",
"npm:pixelmatch@7.1.0",
"npm:playwright@1.59.1",
"npm:pngjs@7.0.0"
]
}
}
@@ -0,0 +1,28 @@
{
"schemaVersion": 1,
"id": "anonymous-entry",
"title": "Anonymous entry and authentication review",
"baseUrl": "${WEB_UX_BASE_URL}",
"locale": "en-US",
"timezone": "UTC",
"colorScheme": "light",
"reducedMotion": "reduce",
"personas": [
{ "id": "anonymous", "label": "Anonymous visitor", "auth": { "kind": "anonymous" } }
],
"viewports": [
{ "label": "desktop", "width": 1440, "height": 1000, "deviceScaleFactor": 1 },
{ "label": "mobile", "width": 390, "height": 844, "deviceScaleFactor": 1 }
],
"routes": [
{
"id": "entry",
"label": "Authentication entry",
"path": "/",
"goal": "Understand the product and begin authentication without seeing Workspace-private content.",
"dataState": "Fresh browser context with no cookies, local storage, or session state.",
"ready": { "kind": "network-idle", "timeoutMs": 15000 },
"capturePoints": [{ "id": "initial", "label": "Anonymous entry", "fullPage": true }]
}
]
}
@@ -0,0 +1,70 @@
{
"schemaVersion": 1,
"id": "workspace-control-plane",
"title": "Workspace control-plane owner and non-owner review",
"baseUrl": "${WEB_UX_BASE_URL}",
"locale": "en-US",
"timezone": "UTC",
"colorScheme": "light",
"reducedMotion": "reduce",
"redact": {
"selectors": ["[data-web-ux-redact]", "input[type=password]"],
"text": []
},
"personas": [
{
"id": "owner",
"label": "Workspace owner",
"auth": { "kind": "storage-state", "path": "../../../.web-ux/auth/owner.json" },
"login": { "path": "/", "successUrl": "/w/" }
},
{
"id": "non-owner",
"label": "Authenticated non-owner",
"auth": { "kind": "storage-state", "path": "../../../.web-ux/auth/non-owner.json" },
"login": { "path": "/", "successUrl": "/w/" }
}
],
"viewports": [
{ "label": "desktop", "width": 1440, "height": 1000, "deviceScaleFactor": 1 },
{ "label": "narrow", "width": 900, "height": 900, "deviceScaleFactor": 1 }
],
"routes": [
{
"id": "workspace-home",
"label": "Workspace overview",
"path": "/w/${WORKSPACE_ID}",
"goal": "Orient the user and expose the highest-value Workspace actions without internal authority noise.",
"dataState": "Dogfood Workspace with current Runtime and Ticket data.",
"ready": { "kind": "selector", "selector": "main", "timeoutMs": 20000 },
"capturePoints": [{ "id": "initial", "label": "Initial viewport", "fullPage": true }]
},
{
"id": "tickets",
"label": "Ticket lanes",
"path": "/w/${WORKSPACE_ID}/tickets",
"goal": "Scan actionable Ticket lanes and reach the primary authoring action in the initial viewport.",
"dataState": "Planning, ready, queued, in-progress, and completed Tickets from the selected Workspace.",
"ready": { "kind": "selector", "selector": "main", "timeoutMs": 30000 },
"capturePoints": [{ "id": "initial", "label": "Loaded Ticket lanes", "fullPage": true }]
},
{
"id": "workers",
"label": "Workers",
"path": "/w/${WORKSPACE_ID}/workers",
"goal": "Find current Worker state and the new-Worker action without exposing transport internals as the primary content.",
"dataState": "Current Workspace Worker projection with mixed lifecycle states.",
"ready": { "kind": "selector", "selector": "main", "timeoutMs": 20000 },
"capturePoints": [{ "id": "initial", "label": "Worker list", "fullPage": true }]
},
{
"id": "repositories",
"label": "Repository settings",
"path": "/w/${WORKSPACE_ID}/settings/repositories",
"goal": "Review repository access as an owner and verify non-owner composition does not expose unusable owner actions.",
"dataState": "Workspace repository catalog projected through current permissions.",
"ready": { "kind": "selector", "selector": "main", "timeoutMs": 20000 },
"capturePoints": [{ "id": "initial", "label": "Repository settings", "fullPage": true }]
}
]
}
+78
View File
@@ -0,0 +1,78 @@
import { dirname, relative, resolve } from "@std/path";
const SECRET_PATTERNS: RegExp[] = [
/\b(authorization|cookie|set-cookie|x-csrf-token)\b\s*[:=]\s*[^\s,;]+/gi,
/\b(bearer)\s+[a-z0-9._~+\/-]+=*/gi,
/\b(session|token|credential|password|passkey|private[_ -]?key)\b\s*[:=]\s*["']?[^\s,"'};]+/gi,
];
export function redactText(value: string, exactSecrets: string[] = []): string {
let result = value;
for (const secret of exactSecrets) {
if (secret) result = result.replaceAll(secret, "[REDACTED]");
}
for (const pattern of SECRET_PATTERNS) result = result.replaceAll(pattern, "$1=[REDACTED]");
return result;
}
export function bounded(value: string, maximum = 1000): string {
const normalized = value.replaceAll(/\s+/g, " ").trim();
return normalized.length <= maximum ? normalized : `${normalized.slice(0, maximum - 1)}`;
}
export function safeUrl(value: string, baseUrl?: string): string {
try {
const url = new URL(value, baseUrl);
url.username = "";
url.password = "";
for (const key of [...url.searchParams.keys()]) url.searchParams.set(key, "[REDACTED]");
url.hash = "";
return url.toString();
} catch {
return "[invalid-url]";
}
}
export function assertBundleIsSecretFree(serialized: string, exactSecrets: string[] = []): void {
const lower = serialized.toLowerCase();
for (const forbidden of ["authorization:", "set-cookie:", "cookie:", "bearer "]) {
if (lower.includes(forbidden)) {
throw new Error(`review bundle contains forbidden secret marker: ${forbidden}`);
}
}
for (const secret of exactSecrets) {
if (secret && serialized.includes(secret)) {
throw new Error("review bundle contains configured secret text");
}
}
}
export async function ensurePrivateDirectory(path: string): Promise<void> {
await Deno.mkdir(path, { recursive: true, mode: 0o700 });
if (Deno.build.os !== "windows") await Deno.chmod(path, 0o700);
}
export async function makePrivate(path: string): Promise<void> {
if (Deno.build.os !== "windows") await Deno.chmod(path, 0o600);
}
export async function writePrivateJson(path: string, value: unknown): Promise<void> {
await ensurePrivateDirectory(dirname(path));
await Deno.writeTextFile(path, `${JSON.stringify(value, null, 2)}\n`, { mode: 0o600 });
if (Deno.build.os !== "windows") await Deno.chmod(path, 0o600);
}
export function logicalPath(repositoryRoot: string, path: string): string {
const absolute = resolve(path);
const logical = relative(repositoryRoot, absolute);
if (logical === "" || (!logical.startsWith("..") && !logical.startsWith("/"))) {
return logical || ".";
}
return absolute;
}
export async function sha256File(path: string): Promise<string> {
const bytes = await Deno.readFile(path);
const digest = await crypto.subtle.digest("SHA-256", bytes);
return [...new Uint8Array(digest)].map((byte) => byte.toString(16).padStart(2, "0")).join("");
}
+540
View File
@@ -0,0 +1,540 @@
import { basename, dirname, join, relative, resolve } from "@std/path";
import { type Browser, chromium, type Page, type Response } from "playwright";
import {
assertBundleIsSecretFree,
bounded,
ensurePrivateDirectory,
makePrivate,
redactText,
safeUrl,
sha256File,
} from "./artifacts.ts";
import { type RunningProcess, startOwnedProcesses, stopOwnedProcesses } from "./processes.ts";
import {
interpolateEnvironment,
loadScenario,
resolveScenarioPath,
validateBaseUrl,
} from "./scenario.ts";
import type {
CaptureError,
CaptureEvidence,
CapturePoint,
Interaction,
Persona,
ReadyCondition,
ReviewContext,
RouteScenario,
Scenario,
ScreenshotEvidence,
Viewport,
} from "./types.ts";
export type CaptureOptions = {
scenarioPath: string;
outputDirectory: string;
baseUrl?: string;
runId?: string;
personas?: string[];
routes?: string[];
viewports?: string[];
headed?: boolean;
};
type SourceState = { revision: string | null; dirty: boolean | null };
function slug(value: string): string {
return value.replaceAll(/[^a-zA-Z0-9.-]+/g, "-").replaceAll(/^-+|-+$/g, "").toLowerCase();
}
function viewportId(viewport: Viewport): string {
return viewport.label ?? `${viewport.width}x${viewport.height}`;
}
function timestampId(): string {
return new Date().toISOString().replaceAll(/[:.]/g, "-");
}
async function sourceState(): Promise<SourceState> {
try {
const [revision, status] = await Promise.all([
new Deno.Command("git", { args: ["rev-parse", "HEAD"], stdout: "piped", stderr: "null" })
.output(),
new Deno.Command("git", { args: ["status", "--porcelain"], stdout: "piped", stderr: "null" })
.output(),
]);
return {
revision: revision.success ? new TextDecoder().decode(revision.stdout).trim() : null,
dirty: status.success ? new TextDecoder().decode(status.stdout).trim().length > 0 : null,
};
} catch {
return { revision: null, dirty: null };
}
}
function selectById<T extends { id: string }>(
values: T[],
requested: string[] | undefined,
kind: string,
): T[] {
if (!requested || requested.length === 0) return values;
const requestedSet = new Set(requested);
const selected = values.filter((value) => requestedSet.has(value.id));
const missing = [...requestedSet].filter((id) => !selected.some((value) => value.id === id));
if (missing.length > 0) throw new Error(`unknown ${kind}: ${missing.join(", ")}`);
return selected;
}
function selectViewports(values: Viewport[], requested: string[] | undefined): Viewport[] {
if (!requested || requested.length === 0) return values;
const requestedSet = new Set(requested);
const selected = values.filter((value) => requestedSet.has(viewportId(value)));
const missing = [...requestedSet].filter((id) =>
!selected.some((value) => viewportId(value) === id)
);
if (missing.length > 0) throw new Error(`unknown viewports: ${missing.join(", ")}`);
return selected;
}
function responseMatches(
response: Response,
ready: Extract<ReadyCondition, { kind: "response" }>,
): boolean {
const pattern = new RegExp(ready.urlPattern);
return pattern.test(response.url()) &&
(ready.status === undefined || response.status() === ready.status);
}
async function waitReady(
page: Page,
ready: ReadyCondition,
navigation?: Response | null,
): Promise<void> {
const timeout = ready.timeoutMs ?? 15_000;
if (ready.kind === "selector") {
await page.locator(ready.selector).first().waitFor({ state: "visible", timeout });
return;
}
if (ready.kind === "network-idle") {
await page.waitForLoadState("networkidle", { timeout });
return;
}
if (navigation && responseMatches(navigation, ready)) return;
await page.waitForResponse((response) => responseMatches(response, ready), { timeout });
}
async function performInteraction(page: Page, interaction: Interaction): Promise<void> {
if (interaction.action === "wait") return await waitReady(page, interaction.ready);
const locator = page.locator(interaction.selector).first();
const timeout = interaction.timeoutMs ?? 10_000;
if (interaction.action === "click") return await locator.click({ timeout });
if (interaction.action === "fill") {
return await locator.fill(interpolateEnvironment(interaction.value), { timeout });
}
await locator.press(interaction.key, { timeout });
}
async function retry<T>(label: string, operation: () => Promise<T>): Promise<T> {
let last: unknown;
for (let attempt = 1; attempt <= 2; attempt++) {
try {
return await operation();
} catch (error) {
last = error;
if (attempt < 2) await new Promise((resolve) => setTimeout(resolve, 350));
}
}
throw new Error(
`${label} failed after 2 attempts: ${last instanceof Error ? last.message : String(last)}`,
);
}
async function hideRedactedSelectors(page: Page, selectors: string[]): Promise<void> {
if (selectors.length === 0) return;
const escaped = selectors.join(",\n");
await page.addStyleTag({ content: `${escaped} { visibility: hidden !important; }` });
}
export function isVisibleUiErrorText(content: string): boolean {
return /\b(error|failed|unauthorized|forbidden|not found)\b/i.test(content);
}
async function collectVisibleUiErrors(
page: Page,
errors: CaptureError[],
secrets: string[],
): Promise<void> {
const alerts = page.locator('[role="alert"], [aria-live="assertive"]');
for (let index = 0; index < await alerts.count(); index++) {
const alert = alerts.nth(index);
if (!await alert.isVisible().catch(() => false)) continue;
const content = (await alert.innerText().catch(() => "")).trim();
if (!isVisibleUiErrorText(content)) continue;
const message = `visible UI error: ${bounded(redactText(content, secrets), 500)}`;
if (!errors.some((error) => error.kind === "document" && error.message === message)) {
errors.push({ kind: "document", message });
}
}
}
async function capturePoint(
page: Page,
runDirectory: string,
persona: Persona,
route: RouteScenario,
viewport: Viewport,
point: CapturePoint,
documentResponse: Response | null,
errors: CaptureError[],
scenario: Scenario,
): Promise<CaptureEvidence> {
const startedAt = new Date().toISOString();
for (const interaction of point.interaction ?? []) await performInteraction(page, interaction);
if (point.ready) await waitReady(page, point.ready);
await hideRedactedSelectors(page, scenario.redact?.selectors ?? []);
await collectVisibleUiErrors(page, errors, scenario.redact?.text ?? []);
const directory = join(
runDirectory,
"captures",
persona.id,
route.id,
viewportId(viewport),
point.id,
);
await ensurePrivateDirectory(directory);
const viewportScreenshot = join(directory, "viewport.png");
await page.screenshot({ path: viewportScreenshot, fullPage: false, animations: "disabled" });
await makePrivate(viewportScreenshot);
const screenshots: ScreenshotEvidence[] = [{
kind: "viewport",
path: relative(runDirectory, viewportScreenshot),
sha256: await sha256File(viewportScreenshot),
}];
if (point.fullPage) {
const fullPageScreenshot = join(directory, "full-page.png");
await page.screenshot({ path: fullPageScreenshot, fullPage: true, animations: "disabled" });
await makePrivate(fullPageScreenshot);
screenshots.push({
kind: "full-page",
path: relative(runDirectory, fullPageScreenshot),
sha256: await sha256File(fullPageScreenshot),
});
}
let snapshotPath: string | null = null;
try {
const snapshot = await page.locator("body").ariaSnapshot({ timeout: 5_000 });
const redacted = redactText(snapshot, scenario.redact?.text ?? []);
const target = join(directory, "accessibility.md");
await Deno.writeTextFile(target, redacted, { mode: 0o600 });
snapshotPath = relative(runDirectory, target);
} catch (error) {
errors.push({
kind: "tool",
message: `accessibility snapshot failed: ${
bounded(error instanceof Error ? error.message : String(error))
}`,
});
}
return {
persona: { id: persona.id, label: persona.label },
route: { id: route.id, path: route.path, goal: route.goal, dataState: route.dataState },
viewport,
theme: scenario.colorScheme ?? "light",
capturePoint: { id: point.id, label: point.label },
document: { url: safeUrl(page.url()), status: documentResponse?.status() ?? null },
screenshots,
snapshotPath,
errors: [...errors],
startedAt,
finishedAt: new Date().toISOString(),
};
}
function screenshotDataUrl(bytes: Uint8Array): string {
let binary = "";
for (const byte of bytes) binary += String.fromCharCode(byte);
return `data:image/png;base64,${btoa(binary)}`;
}
function escapeHtml(value: string): string {
return value.replaceAll("&", "&amp;").replaceAll("<", "&lt;").replaceAll(">", "&gt;").replaceAll(
'"',
"&quot;",
);
}
async function createContactSheet(
browser: Browser,
runDirectory: string,
captures: CaptureEvidence[],
): Promise<{ html: string | null; png: string | null }> {
const cells: string[] = [];
for (const capture of captures) {
const screenshot = capture.screenshots.find((item) => item.kind === "viewport") ??
capture.screenshots[0];
if (!screenshot) continue;
const bytes = await Deno.readFile(join(runDirectory, screenshot.path));
cells.push(
`<figure><img src="${screenshotDataUrl(bytes)}"><figcaption><strong>${
escapeHtml(capture.persona.label)
} · ${escapeHtml(capture.route.id)}</strong><br>${
escapeHtml(viewportId(capture.viewport))
} · ${escapeHtml(capture.capturePoint.label)}<br><small>${
escapeHtml(capture.route.dataState)
}</small>${
capture.errors.length > 0
? `<br><strong class="errors">${capture.errors.length} captured error(s)</strong>`
: ""
}</figcaption></figure>`,
);
}
if (cells.length === 0) return { html: null, png: null };
const html =
`<!doctype html><meta charset="utf-8"><title>Web UX review contact sheet</title><style>body{margin:0;padding:20px;background:#e8e8e8;color:#111;font:14px system-ui}main{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:20px}figure{margin:0;background:white;border:1px solid #aaa;padding:10px;box-shadow:0 2px 8px #0002}img{width:100%;height:auto;display:block;border:1px solid #ddd}figcaption{padding-top:8px;line-height:1.45}small{color:#555}.errors{color:#b42318}</style><main>${
cells.join("")
}</main>`;
const htmlPath = join(runDirectory, "contact-sheet.html");
const pngPath = join(runDirectory, "contact-sheet.png");
await Deno.writeTextFile(htmlPath, html, { mode: 0o600 });
const page = await browser.newPage({ viewport: { width: 1600, height: 1000 } });
try {
await page.setContent(html, { waitUntil: "load" });
await page.screenshot({ path: pngPath, fullPage: true, animations: "disabled" });
await makePrivate(pngPath);
} finally {
await page.close();
}
return { html: relative(runDirectory, htmlPath), png: relative(runDirectory, pngPath) };
}
export async function capture(options: CaptureOptions): Promise<ReviewContext> {
const scenarioPath = resolve(options.scenarioPath);
const scenario = await loadScenario(scenarioPath);
const baseUrl = validateBaseUrl(
interpolateEnvironment(
options.baseUrl ?? Deno.env.get("WEB_UX_BASE_URL") ?? scenario.baseUrl ?? "",
),
);
const personas = selectById(scenario.personas, options.personas, "personas");
const routes = selectById(scenario.routes, options.routes, "routes");
const viewports = selectViewports(scenario.viewports, options.viewports);
const runId = slug(options.runId ?? `${scenario.id}-${timestampId()}`);
const runDirectory = resolve(options.outputDirectory, runId);
try {
await Deno.stat(runDirectory);
throw new Error(`run directory already exists: ${runDirectory}`);
} catch (error) {
if (!(error instanceof Deno.errors.NotFound)) throw error;
}
await ensurePrivateDirectory(runDirectory);
const secrets = scenario.redact?.text ?? [];
let browser: Browser | null = null;
let processes: RunningProcess[] = [];
const captures: CaptureEvidence[] = [];
const diagnostics: CaptureError[] = [];
let contactSheet = { html: null as string | null, png: null as string | null };
let browserVersion = "unknown";
let status: ReviewContext["status"] = "completed";
try {
processes = await startOwnedProcesses(
scenario.processes ?? [],
scenarioPath,
join(runDirectory, "process-logs"),
secrets,
);
browser = await chromium.launch({ headless: !options.headed });
browserVersion = browser.version();
for (const persona of personas) {
const storageState = persona.auth.kind === "storage-state"
? resolveScenarioPath(scenarioPath, persona.auth.path)
: undefined;
if (storageState) await Deno.stat(storageState);
for (const viewport of viewports) {
const context = await browser.newContext({
storageState,
viewport: { width: viewport.width, height: viewport.height },
deviceScaleFactor: viewport.deviceScaleFactor ?? 1,
locale: scenario.locale,
timezoneId: scenario.timezone,
colorScheme: scenario.colorScheme,
reducedMotion: scenario.reducedMotion,
});
try {
for (const route of routes) {
const routeErrors: CaptureError[] = [];
const page = await context.newPage();
page.on("console", (message) => {
if (message.type() === "error") {
routeErrors.push({
kind: "console",
message: bounded(redactText(message.text(), secrets)),
});
}
});
page.on(
"pageerror",
(error) =>
routeErrors.push({
kind: "page",
message: bounded(redactText(error.message, secrets)),
}),
);
page.on(
"requestfailed",
(request) =>
routeErrors.push({
kind: "request",
message: bounded(
redactText(request.failure()?.errorText ?? "request failed", secrets),
),
url: safeUrl(request.url()),
}),
);
page.on("response", (response) => {
if (response.status() >= 400) {
routeErrors.push({
kind: "request",
message: `HTTP ${response.status()}`,
url: safeUrl(response.url()),
status: response.status(),
});
}
});
try {
const routePath = interpolateEnvironment(route.path);
const targetUrl = new URL(routePath, `${baseUrl}/`).toString();
const response = await retry(`navigate ${route.id}`, async () => {
const ready = route.ready;
const responseReady = ready.kind === "response"
? page.waitForResponse(
(candidate) => responseMatches(candidate, ready),
{ timeout: ready.timeoutMs ?? 15_000 },
)
: null;
try {
const navigation = await page.goto(targetUrl, {
waitUntil: "domcontentloaded",
timeout: 20_000,
});
if (responseReady) await responseReady;
else await waitReady(page, ready, navigation);
return navigation;
} catch (error) {
responseReady?.catch(() => undefined);
throw error;
}
});
if (response && response.status() >= 400) {
routeErrors.push({
kind: "document",
message: `document returned HTTP ${response.status()}`,
url: safeUrl(response.url()),
status: response.status(),
});
}
for (const point of route.capturePoints) {
captures.push(
await capturePoint(
page,
runDirectory,
persona,
route,
viewport,
point,
response,
routeErrors,
scenario,
),
);
}
} catch (error) {
status = "completed-with-errors";
routeErrors.push({
kind: "tool",
message: bounded(
redactText(error instanceof Error ? error.message : String(error), secrets),
),
});
captures.push({
persona: { id: persona.id, label: persona.label },
route: {
id: route.id,
path: route.path,
goal: route.goal,
dataState: route.dataState,
},
viewport,
theme: scenario.colorScheme ?? "light",
capturePoint: { id: "failed", label: "Capture failed" },
document: { url: safeUrl(page.url()), status: null },
screenshots: [],
snapshotPath: null,
errors: routeErrors,
startedAt: new Date().toISOString(),
finishedAt: new Date().toISOString(),
});
} finally {
await page.close();
}
}
} finally {
await context.close();
}
}
}
contactSheet = await createContactSheet(browser, runDirectory, captures);
if (captures.some((capture) => capture.errors.length > 0)) status = "completed-with-errors";
} catch (error) {
status = "failed";
diagnostics.push({
kind: "tool",
message: bounded(redactText(error instanceof Error ? error.message : String(error), secrets)),
});
} finally {
if (browser) {
await browser.close().catch((error) =>
diagnostics.push({
kind: "tool",
message: `browser cleanup failed: ${bounded(String(error))}`,
})
);
}
diagnostics.push(...await stopOwnedProcesses(processes));
}
if (diagnostics.length > 0 && status === "completed") status = "completed-with-errors";
const manifest: ReviewContext = {
schemaVersion: 1,
runId,
scenario: {
id: scenario.id,
title: scenario.title,
sourcePath: relative(Deno.cwd(), scenarioPath),
},
source: await sourceState(),
baseUrl: safeUrl(baseUrl),
browser: { name: "chromium", version: browserVersion },
createdAt: new Date().toISOString(),
status,
filters: {
personas: personas.map((item) => item.id),
routes: routes.map((item) => item.id),
viewports: viewports.map(viewportId),
},
captures,
contactSheet,
diagnostics,
};
const serialized = `${JSON.stringify(manifest, null, 2)}\n`;
assertBundleIsSecretFree(serialized, secrets);
await Deno.writeTextFile(join(runDirectory, "review-context.json"), serialized, { mode: 0o600 });
if (status === "failed") {
throw new Error(`capture failed; inspect ${join(runDirectory, "review-context.json")}`);
}
return manifest;
}
export function describeCapture(manifest: ReviewContext, outputDirectory: string): string {
return `${manifest.status}: ${manifest.captures.length} capture(s); ${
join(outputDirectory, manifest.runId, "review-context.json")
}`;
}
+173
View File
@@ -0,0 +1,173 @@
import { Buffer } from "node:buffer";
import { basename, dirname, join, relative, resolve } from "@std/path";
import { chromium } from "playwright";
import pixelmatch from "pixelmatch";
import { PNG } from "pngjs";
import { ensurePrivateDirectory, makePrivate, sha256File } from "./artifacts.ts";
import type { CaptureEvidence, ReviewContext } from "./types.ts";
export type CompareOptions = {
before: string;
after: string;
outputDirectory: string;
threshold?: number;
};
type Pair = {
key: string;
before: CaptureEvidence;
after: CaptureEvidence;
beforePath: string;
afterPath: string;
diffPath: string;
changedPixels: number;
totalPixels: number;
dimensionMismatch: boolean;
};
function key(capture: CaptureEvidence): string {
const viewport = capture.viewport.label ?? `${capture.viewport.width}x${capture.viewport.height}`;
return [capture.persona.id, capture.route.id, viewport, capture.capturePoint.id].join("/");
}
async function readManifest(path: string): Promise<ReviewContext> {
const parsed = JSON.parse(await Deno.readTextFile(path));
if (parsed.schemaVersion !== 1 || !Array.isArray(parsed.captures)) {
throw new Error(`not a web-ux review context: ${path}`);
}
return parsed;
}
function viewportScreenshot(capture: CaptureEvidence): string | null {
return capture.screenshots.find((item) => item.kind === "viewport")?.path ??
capture.screenshots[0]?.path ?? null;
}
function dataUrl(bytes: Uint8Array): string {
let binary = "";
for (const byte of bytes) binary += String.fromCharCode(byte);
return `data:image/png;base64,${btoa(binary)}`;
}
function escapeHtml(value: string): string {
return value.replaceAll("&", "&amp;").replaceAll("<", "&lt;").replaceAll(">", "&gt;").replaceAll(
'"',
"&quot;",
);
}
export async function compare(options: CompareOptions): Promise<string> {
const beforeManifestPath = resolve(options.before);
const afterManifestPath = resolve(options.after);
const [before, after] = await Promise.all([
readManifest(beforeManifestPath),
readManifest(afterManifestPath),
]);
const outputDirectory = resolve(options.outputDirectory);
await ensurePrivateDirectory(outputDirectory);
const afterByKey = new Map(after.captures.map((capture) => [key(capture), capture]));
const pairs: Pair[] = [];
const unmatchedBefore: string[] = [];
for (const earlier of before.captures) {
const later = afterByKey.get(key(earlier));
const earlierScreenshot = viewportScreenshot(earlier);
const laterScreenshot = later ? viewportScreenshot(later) : null;
if (!later || !earlierScreenshot || !laterScreenshot) {
unmatchedBefore.push(key(earlier));
continue;
}
afterByKey.delete(key(earlier));
const beforePath = resolve(dirname(beforeManifestPath), earlierScreenshot);
const afterPath = resolve(dirname(afterManifestPath), laterScreenshot);
const [beforeImage, afterImage] = [
PNG.sync.read(Buffer.from(Deno.readFileSync(beforePath))),
PNG.sync.read(Buffer.from(Deno.readFileSync(afterPath))),
];
const dimensionMismatch = beforeImage.width !== afterImage.width ||
beforeImage.height !== afterImage.height;
const width = Math.max(beforeImage.width, afterImage.width);
const height = Math.max(beforeImage.height, afterImage.height);
const diff = new PNG({ width, height, fill: true });
let changedPixels = width * height;
if (!dimensionMismatch) {
changedPixels = pixelmatch(beforeImage.data, afterImage.data, diff.data, width, height, {
threshold: options.threshold ?? 0.1,
includeAA: false,
});
}
const diffPath = join(outputDirectory, "diffs", `${key(earlier).replaceAll("/", "--")}.png`);
await ensurePrivateDirectory(dirname(diffPath));
await Deno.writeFile(diffPath, PNG.sync.write(diff), { mode: 0o600 });
await makePrivate(diffPath);
pairs.push({
key: key(earlier),
before: earlier,
after: later,
beforePath,
afterPath,
diffPath,
changedPixels,
totalPixels: width * height,
dimensionMismatch,
});
}
const cells: string[] = [];
for (const pair of pairs) {
const [beforeBytes, afterBytes, diffBytes] = await Promise.all([
Deno.readFile(pair.beforePath),
Deno.readFile(pair.afterPath),
Deno.readFile(pair.diffPath),
]);
cells.push(
`<section><h2>${escapeHtml(pair.key)}</h2><p>${
pair.dimensionMismatch
? "dimension mismatch"
: `${pair.changedPixels} / ${pair.totalPixels} pixels changed`
}</p><div class="row"><figure><img src="${
dataUrl(beforeBytes)
}"><figcaption>before</figcaption></figure><figure><img src="${
dataUrl(afterBytes)
}"><figcaption>after</figcaption></figure><figure><img src="${
dataUrl(diffBytes)
}"><figcaption>diff</figcaption></figure></div></section>`,
);
}
const html =
`<!doctype html><meta charset="utf-8"><title>Web UX comparison</title><style>body{margin:0;padding:20px;background:#e8e8e8;color:#111;font:14px system-ui}section{background:white;border:1px solid #aaa;margin:0 0 24px;padding:12px}.row{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:12px}figure{margin:0}img{width:100%;height:auto;border:1px solid #ddd}figcaption{text-align:center;padding:6px}h2{font-size:16px;margin:0}p{color:#555}</style>${
cells.join("")
}`;
const htmlPath = join(outputDirectory, "comparison.html");
const pngPath = join(outputDirectory, "comparison.png");
await Deno.writeTextFile(htmlPath, html, { mode: 0o600 });
const browser = await chromium.launch({ headless: true });
try {
const page = await browser.newPage({ viewport: { width: 1800, height: 1000 } });
await page.setContent(html, { waitUntil: "load" });
await page.screenshot({ path: pngPath, fullPage: true, animations: "disabled" });
await makePrivate(pngPath);
} finally {
await browser.close();
}
const report = {
schemaVersion: 1,
before: beforeManifestPath,
after: afterManifestPath,
createdAt: new Date().toISOString(),
threshold: options.threshold ?? 0.1,
pairs: await Promise.all(pairs.map(async (pair) => ({
key: pair.key,
changedPixels: pair.changedPixels,
totalPixels: pair.totalPixels,
changedRatio: pair.totalPixels === 0 ? 0 : pair.changedPixels / pair.totalPixels,
dimensionMismatch: pair.dimensionMismatch,
diff: relative(outputDirectory, pair.diffPath),
diffSha256: await sha256File(pair.diffPath),
}))),
unmatchedBefore,
unmatchedAfter: [...afterByKey.keys()],
contactSheet: { html: basename(htmlPath), png: basename(pngPath) },
};
const reportPath = join(outputDirectory, "comparison.json");
await Deno.writeTextFile(reportPath, `${JSON.stringify(report, null, 2)}\n`, { mode: 0o600 });
return reportPath;
}
+114
View File
@@ -0,0 +1,114 @@
import { dirname, resolve } from "@std/path";
import { chromium } from "playwright";
import { ensurePrivateDirectory, writePrivateJson } from "./artifacts.ts";
import {
interpolateEnvironment,
loadScenario,
resolveScenarioPath,
validateBaseUrl,
} from "./scenario.ts";
export type AuthOptions = {
scenarioPath: string;
personaId: string;
baseUrl?: string;
importState?: string;
timeoutMs?: number;
headless?: boolean;
};
function validateStorageState(value: unknown): { cookies: unknown[]; origins: unknown[] } {
if (typeof value !== "object" || value === null || Array.isArray(value)) {
throw new Error("storage state must be an object");
}
const source = value as Record<string, unknown>;
if (!Array.isArray(source.cookies) || !Array.isArray(source.origins)) {
throw new Error("storage state must contain cookies and origins arrays");
}
return { cookies: source.cookies, origins: source.origins };
}
export async function authenticate(options: AuthOptions): Promise<string> {
const scenarioPath = resolve(options.scenarioPath);
const scenario = await loadScenario(scenarioPath);
const persona = scenario.personas.find((candidate) => candidate.id === options.personaId);
if (!persona) throw new Error(`unknown persona: ${options.personaId}`);
if (persona.auth.kind !== "storage-state") {
throw new Error(`persona ${persona.id} is anonymous and has no auth state`);
}
const outputPath = resolveScenarioPath(scenarioPath, persona.auth.path);
await ensurePrivateDirectory(dirname(outputPath));
if (options.importState) {
const imported = validateStorageState(
JSON.parse(await Deno.readTextFile(resolve(options.importState))),
);
await writePrivateJson(outputPath, imported);
return outputPath;
}
if (!persona.login) {
throw new Error(`persona ${persona.id} needs login configuration or --import-state`);
}
const baseUrl = validateBaseUrl(
interpolateEnvironment(
options.baseUrl ?? Deno.env.get("WEB_UX_BASE_URL") ?? scenario.baseUrl ?? "",
),
);
const browser = await chromium.launch({ headless: options.headless ?? false });
try {
const context = await browser.newContext();
const page = await context.newPage();
const loginUrl = new URL(interpolateEnvironment(persona.login.path ?? "/"), `${baseUrl}/`)
.toString();
await page.goto(loginUrl, { waitUntil: "domcontentloaded", timeout: 30_000 });
const success = new RegExp(interpolateEnvironment(persona.login.successUrl));
if (!success.test(page.url())) {
await page.waitForURL((url) => success.test(url.toString()), {
timeout: options.timeoutMs ?? 300_000,
});
}
await context.storageState({ path: outputPath });
if (Deno.build.os !== "windows") await Deno.chmod(outputPath, 0o600);
return outputPath;
} finally {
await browser.close();
}
}
export type CleanupOptions = {
outputDirectory: string;
keep: number;
olderThanDays?: number;
dryRun?: boolean;
};
export async function cleanup(options: CleanupOptions): Promise<string[]> {
const outputDirectory = resolve(options.outputDirectory);
const candidates: { path: string; modified: number }[] = [];
try {
for await (const entry of Deno.readDir(outputDirectory)) {
if (!entry.isDirectory) continue;
const path = resolve(outputDirectory, entry.name);
try {
await Deno.stat(resolve(path, "review-context.json"));
const stat = await Deno.stat(path);
candidates.push({ path, modified: stat.mtime?.getTime() ?? 0 });
} catch {
// Only complete review bundle directories are owned by cleanup.
}
}
} catch (error) {
if (error instanceof Deno.errors.NotFound) return [];
throw error;
}
candidates.sort((left, right) => right.modified - left.modified);
const cutoff = options.olderThanDays === undefined
? Number.POSITIVE_INFINITY
: Date.now() - options.olderThanDays * 24 * 60 * 60 * 1000;
const removed: string[] = [];
for (const [index, candidate] of candidates.entries()) {
if (index < options.keep || candidate.modified > cutoff) continue;
removed.push(candidate.path);
if (!options.dryRun) await Deno.remove(candidate.path, { recursive: true });
}
return removed;
}
+175
View File
@@ -0,0 +1,175 @@
import { dirname, isAbsolute, resolve } from "@std/path";
import { bounded, redactText } from "./artifacts.ts";
import type { CaptureError, OwnedProcess } from "./types.ts";
export type RunningProcess = {
id: string;
pid: number;
child: Deno.ChildProcess;
output: Promise<void>;
};
async function appendOutput(
stream: ReadableStream<Uint8Array>,
destination: string,
secrets: string[],
): Promise<void> {
const file = await Deno.open(destination, {
create: true,
append: true,
write: true,
mode: 0o600,
});
try {
const reader = stream.pipeThrough(new TextDecoderStream()).getReader();
while (true) {
const { value, done } = await reader.read();
if (done) break;
await file.write(new TextEncoder().encode(redactText(value, secrets)));
}
} finally {
file.close();
}
}
async function waitForReady(url: string, timeoutMs: number): Promise<void> {
const deadline = Date.now() + timeoutMs;
let lastError = "not attempted";
while (Date.now() < deadline) {
try {
const response = await fetch(url, { redirect: "manual", signal: AbortSignal.timeout(2_000) });
if (response.status < 500) return;
lastError = `HTTP ${response.status}`;
} catch (error) {
lastError = error instanceof Error ? error.message : String(error);
}
await new Promise((resolve) => setTimeout(resolve, 200));
}
throw new Error(`process readiness timed out for ${url}: ${bounded(lastError, 200)}`);
}
export async function startOwnedProcesses(
specifications: OwnedProcess[],
scenarioPath: string,
logsDirectory: string,
secrets: string[],
): Promise<RunningProcess[]> {
const running: RunningProcess[] = [];
await Deno.mkdir(logsDirectory, { recursive: true });
try {
for (const specification of specifications) {
const cwd = specification.cwd === undefined
? dirname(resolve(scenarioPath))
: isAbsolute(specification.cwd)
? specification.cwd
: resolve(dirname(scenarioPath), specification.cwd);
const child = new Deno.Command(specification.command, {
args: specification.args ?? [],
cwd,
env: specification.env,
stdin: "null",
stdout: "piped",
stderr: "piped",
}).spawn();
const stdout = appendOutput(
child.stdout,
`${logsDirectory}/${specification.id}.stdout.log`,
secrets,
);
const stderr = appendOutput(
child.stderr,
`${logsDirectory}/${specification.id}.stderr.log`,
secrets,
);
const process = {
id: specification.id,
pid: child.pid,
child,
output: Promise.all([stdout, stderr]).then(() => undefined),
};
running.push(process);
if (specification.readyUrl) {
await Promise.race([
waitForReady(specification.readyUrl, specification.readyTimeoutMs ?? 30_000),
child.status.then((status) => {
throw new Error(
`owned process ${specification.id} exited before readiness: ${status.code}`,
);
}),
]);
}
}
return running;
} catch (error) {
await stopOwnedProcesses(running);
throw error;
}
}
async function descendantPids(parentPid: number): Promise<number[]> {
if (Deno.build.os === "windows") return [];
try {
const result = await new Deno.Command("ps", {
args: ["-eo", "pid=,ppid="],
stdout: "piped",
stderr: "null",
}).output();
if (!result.success) return [];
const rows = new TextDecoder().decode(result.stdout).trim().split("\n").map((line) =>
line.trim().split(/\s+/).map(Number)
);
const descendants: number[] = [];
const queue = [parentPid];
while (queue.length > 0) {
const parent = queue.shift()!;
for (const [pid, ppid] of rows) {
if (ppid === parent && !descendants.includes(pid)) {
descendants.push(pid);
queue.push(pid);
}
}
}
return descendants.reverse();
} catch {
return [];
}
}
function tryKill(pid: number, signal: Deno.Signal): void {
try {
Deno.kill(pid, signal);
} catch (error) {
if (!(error instanceof Deno.errors.NotFound)) throw error;
}
}
export async function stopOwnedProcesses(processes: RunningProcess[]): Promise<CaptureError[]> {
const diagnostics: CaptureError[] = [];
for (const process of [...processes].reverse()) {
try {
for (const pid of await descendantPids(process.pid)) tryKill(pid, "SIGTERM");
tryKill(process.pid, "SIGTERM");
let timer: number | undefined;
const exited = await Promise.race([
process.child.status.then(() => true),
new Promise<boolean>((resolve) => {
timer = setTimeout(() => resolve(false), 3_000);
}),
]).finally(() => clearTimeout(timer));
if (!exited) {
for (const pid of await descendantPids(process.pid)) tryKill(pid, "SIGKILL");
tryKill(process.pid, "SIGKILL");
await process.child.status;
}
await process.output;
} catch (error) {
diagnostics.push({
kind: "tool",
message: `failed to clean process ${process.id}: ${
bounded(error instanceof Error ? error.message : String(error), 500)
}`,
});
}
}
return diagnostics;
}
+270
View File
@@ -0,0 +1,270 @@
import { isAbsolute, join, resolve } from "@std/path";
import type {
CapturePoint,
Persona,
ReadyCondition,
RouteScenario,
Scenario,
Viewport,
} from "./types.ts";
function record(value: unknown, at: string): Record<string, unknown> {
if (typeof value !== "object" || value === null || Array.isArray(value)) {
throw new Error(`${at} must be an object`);
}
return value as Record<string, unknown>;
}
function text(value: unknown, at: string): string {
if (typeof value !== "string" || value.trim() === "") throw new Error(`${at} must be text`);
return value;
}
function positiveInteger(value: unknown, at: string): number {
if (!Number.isInteger(value) || (value as number) <= 0) {
throw new Error(`${at} must be a positive integer`);
}
return value as number;
}
function identifier(value: unknown, at: string): string {
const result = text(value, at);
if (!/^[a-z0-9][a-z0-9-]*$/.test(result)) {
throw new Error(`${at} must contain lowercase ASCII letters, digits, or hyphens`);
}
return result;
}
function stringArray(value: unknown, at: string): string[] {
if (value === undefined) return [];
if (!Array.isArray(value)) throw new Error(`${at} must be an array`);
return value.map((item, index) => text(item, `${at}[${index}]`));
}
function parseReady(value: unknown, at: string): ReadyCondition {
const source = record(value, at);
const kind = text(source.kind, `${at}.kind`);
const timeoutMs = source.timeoutMs === undefined
? undefined
: positiveInteger(source.timeoutMs, `${at}.timeoutMs`);
if (kind === "selector") {
return { kind, selector: text(source.selector, `${at}.selector`), timeoutMs };
}
if (kind === "response") {
const status = source.status === undefined
? undefined
: positiveInteger(source.status, `${at}.status`);
return { kind, urlPattern: text(source.urlPattern, `${at}.urlPattern`), status, timeoutMs };
}
if (kind === "network-idle") return { kind, timeoutMs };
throw new Error(`${at}.kind is unsupported: ${kind}`);
}
function parseCapturePoint(value: unknown, at: string): CapturePoint {
const source = record(value, at);
const result: CapturePoint = {
id: identifier(source.id, `${at}.id`),
label: text(source.label, `${at}.label`),
fullPage: source.fullPage === undefined ? false : Boolean(source.fullPage),
};
if (source.ready !== undefined) result.ready = parseReady(source.ready, `${at}.ready`);
if (source.interaction !== undefined) {
if (!Array.isArray(source.interaction)) throw new Error(`${at}.interaction must be an array`);
result.interaction = source.interaction.map((raw, index) => {
const action = record(raw, `${at}.interaction[${index}]`);
const name = text(action.action, `${at}.interaction[${index}].action`);
if (name === "wait") {
return {
action: name,
ready: parseReady(action.ready, `${at}.interaction[${index}].ready`),
};
}
const selector = text(action.selector, `${at}.interaction[${index}].selector`);
const timeoutMs = action.timeoutMs === undefined
? undefined
: positiveInteger(action.timeoutMs, `${at}.interaction[${index}].timeoutMs`);
if (name === "click") return { action: name, selector, timeoutMs };
if (name === "fill") {
return {
action: name,
selector,
value: text(action.value, `${at}.interaction[${index}].value`),
timeoutMs,
};
}
if (name === "press") {
return {
action: name,
selector,
key: text(action.key, `${at}.interaction[${index}].key`),
timeoutMs,
};
}
throw new Error(`${at}.interaction[${index}].action is unsupported: ${name}`);
});
}
return result;
}
function parsePersona(value: unknown, at: string): Persona {
const source = record(value, at);
const auth = record(source.auth, `${at}.auth`);
const kind = text(auth.kind, `${at}.auth.kind`);
const persona: Persona = {
id: identifier(source.id, `${at}.id`),
label: text(source.label, `${at}.label`),
auth: kind === "anonymous"
? { kind }
: kind === "storage-state"
? { kind, path: text(auth.path, `${at}.auth.path`) }
: (() => {
throw new Error(`${at}.auth.kind is unsupported: ${kind}`);
})(),
};
if (source.login !== undefined) {
const login = record(source.login, `${at}.login`);
persona.login = {
path: login.path === undefined ? "/" : text(login.path, `${at}.login.path`),
successUrl: text(login.successUrl, `${at}.login.successUrl`),
};
}
return persona;
}
function parseViewport(value: unknown, at: string): Viewport {
const source = record(value, at);
return {
width: positiveInteger(source.width, `${at}.width`),
height: positiveInteger(source.height, `${at}.height`),
label: source.label === undefined ? undefined : identifier(source.label, `${at}.label`),
deviceScaleFactor: source.deviceScaleFactor === undefined
? 1
: positiveInteger(source.deviceScaleFactor, `${at}.deviceScaleFactor`),
};
}
function parseRoute(value: unknown, at: string): RouteScenario {
const source = record(value, at);
if (!Array.isArray(source.capturePoints) || source.capturePoints.length === 0) {
throw new Error(`${at}.capturePoints must have at least one item`);
}
return {
id: identifier(source.id, `${at}.id`),
label: text(source.label, `${at}.label`),
path: text(source.path, `${at}.path`),
goal: text(source.goal, `${at}.goal`),
dataState: text(source.dataState, `${at}.dataState`),
ready: parseReady(source.ready, `${at}.ready`),
capturePoints: source.capturePoints.map((point, index) =>
parseCapturePoint(point, `${at}.capturePoints[${index}]`)
),
};
}
function uniqueIds(values: { id: string }[], at: string): void {
const seen = new Set<string>();
for (const value of values) {
if (seen.has(value.id)) throw new Error(`${at} contains duplicate id: ${value.id}`);
seen.add(value.id);
}
}
export function interpolateEnvironment(value: string, environment = Deno.env.toObject()): string {
return value.replaceAll(/\$\{([A-Z][A-Z0-9_]*)\}/g, (_match, name: string) => {
const replacement = environment[name];
if (replacement === undefined) {
throw new Error(`required environment variable is missing: ${name}`);
}
return replacement;
});
}
export function validateBaseUrl(value: string): string {
const url = new URL(value);
if (url.protocol !== "http:" && url.protocol !== "https:") {
throw new Error("base URL must use http or https");
}
if (url.username || url.password) throw new Error("base URL must not contain credentials");
url.pathname = url.pathname.replace(/\/$/, "");
return url.toString().replace(/\/$/, "");
}
export function resolveScenarioPath(sourcePath: string, value: string): string {
const expanded = interpolateEnvironment(value);
return isAbsolute(expanded) ? expanded : resolve(join(sourcePath, "..", expanded));
}
export async function loadScenario(sourcePath: string): Promise<Scenario> {
const parsed = JSON.parse(await Deno.readTextFile(sourcePath));
const source = record(parsed, "scenario");
if (source.schemaVersion !== 1) throw new Error("scenario.schemaVersion must equal 1");
if (!Array.isArray(source.personas) || source.personas.length === 0) {
throw new Error("scenario.personas must have at least one item");
}
if (!Array.isArray(source.viewports) || source.viewports.length === 0) {
throw new Error("scenario.viewports must have at least one item");
}
if (!Array.isArray(source.routes) || source.routes.length === 0) {
throw new Error("scenario.routes must have at least one item");
}
const personas = source.personas.map((value, index) =>
parsePersona(value, `scenario.personas[${index}]`)
);
const routes = source.routes.map((value, index) =>
parseRoute(value, `scenario.routes[${index}]`)
);
const scenario: Scenario = {
schemaVersion: 1,
id: identifier(source.id, "scenario.id"),
title: text(source.title, "scenario.title"),
baseUrl: source.baseUrl === undefined ? undefined : text(source.baseUrl, "scenario.baseUrl"),
locale: source.locale === undefined ? "en-US" : text(source.locale, "scenario.locale"),
timezone: source.timezone === undefined ? "UTC" : text(source.timezone, "scenario.timezone"),
colorScheme: source.colorScheme === "dark" ? "dark" : "light",
reducedMotion: source.reducedMotion === "no-preference" ? "no-preference" : "reduce",
redact: source.redact === undefined ? undefined : (() => {
const redact = record(source.redact, "scenario.redact");
return {
selectors: stringArray(redact.selectors, "scenario.redact.selectors"),
text: stringArray(redact.text, "scenario.redact.text").map((value) =>
interpolateEnvironment(value)
),
};
})(),
personas,
viewports: source.viewports.map((value, index) =>
parseViewport(value, `scenario.viewports[${index}]`)
),
routes,
};
if (source.processes !== undefined) {
if (!Array.isArray(source.processes)) throw new Error("scenario.processes must be an array");
scenario.processes = source.processes.map((value, index) => {
const at = `scenario.processes[${index}]`;
const process = record(value, at);
const env = process.env === undefined ? undefined : record(process.env, `${at}.env`);
return {
id: identifier(process.id, `${at}.id`),
command: text(process.command, `${at}.command`),
args: stringArray(process.args, `${at}.args`),
cwd: process.cwd === undefined ? undefined : text(process.cwd, `${at}.cwd`),
env: env === undefined ? undefined : Object.fromEntries(
Object.entries(env).map((
[key, raw],
) => [key, interpolateEnvironment(text(raw, `${at}.env.${key}`))]),
),
readyUrl: process.readyUrl === undefined ? undefined : validateBaseUrl(
interpolateEnvironment(text(process.readyUrl, `${at}.readyUrl`)),
),
readyTimeoutMs: process.readyTimeoutMs === undefined
? undefined
: positiveInteger(process.readyTimeoutMs, `${at}.readyTimeoutMs`),
};
});
uniqueIds(scenario.processes, "scenario.processes");
}
uniqueIds(personas, "scenario.personas");
uniqueIds(routes, "scenario.routes");
for (const route of routes) uniqueIds(route.capturePoints, `route ${route.id} capturePoints`);
return scenario;
}
+116
View File
@@ -0,0 +1,116 @@
export type Viewport = {
width: number;
height: number;
label?: string;
deviceScaleFactor?: number;
};
export type Persona = {
id: string;
label: string;
auth: { kind: "anonymous" } | { kind: "storage-state"; path: string };
login?: {
path?: string;
successUrl: string;
};
};
export type ReadyCondition =
| { kind: "selector"; selector: string; timeoutMs?: number }
| { kind: "response"; urlPattern: string; status?: number; timeoutMs?: number }
| { kind: "network-idle"; timeoutMs?: number };
export type Interaction =
| { action: "click"; selector: string; timeoutMs?: number }
| { action: "fill"; selector: string; value: string; timeoutMs?: number }
| { action: "press"; selector: string; key: string; timeoutMs?: number }
| { action: "wait"; ready: ReadyCondition };
export type CapturePoint = {
id: string;
label: string;
interaction?: Interaction[];
ready?: ReadyCondition;
fullPage?: boolean;
};
export type RouteScenario = {
id: string;
label: string;
path: string;
goal: string;
dataState: string;
ready: ReadyCondition;
capturePoints: CapturePoint[];
};
export type OwnedProcess = {
id: string;
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
readyUrl?: string;
readyTimeoutMs?: number;
};
export type Scenario = {
schemaVersion: 1;
id: string;
title: string;
baseUrl?: string;
locale?: string;
timezone?: string;
colorScheme?: "light" | "dark";
reducedMotion?: "reduce" | "no-preference";
redact?: {
selectors?: string[];
text?: string[];
};
personas: Persona[];
viewports: Viewport[];
routes: RouteScenario[];
processes?: OwnedProcess[];
};
export type CaptureError = {
kind: "console" | "page" | "request" | "document" | "tool";
message: string;
url?: string;
status?: number;
};
export type ScreenshotEvidence = {
kind: "viewport" | "full-page";
path: string;
sha256: string;
};
export type CaptureEvidence = {
persona: { id: string; label: string };
route: { id: string; path: string; goal: string; dataState: string };
viewport: Viewport;
theme: string;
capturePoint: { id: string; label: string };
document: { url: string; status: number | null };
screenshots: ScreenshotEvidence[];
snapshotPath: string | null;
errors: CaptureError[];
startedAt: string;
finishedAt: string;
};
export type ReviewContext = {
schemaVersion: 1;
runId: string;
scenario: { id: string; title: string; sourcePath: string };
source: { revision: string | null; dirty: boolean | null };
baseUrl: string;
browser: { name: "chromium"; version: string };
createdAt: string;
status: "completed" | "completed-with-errors" | "failed";
filters: { personas: string[]; routes: string[]; viewports: string[] };
captures: CaptureEvidence[];
contactSheet: { html: string | null; png: string | null };
diagnostics: CaptureError[];
};
+26
View File
@@ -0,0 +1,26 @@
import { assertEquals, assertThrows } from "@std/assert";
import { parseArguments } from "../cli.ts";
import { isVisibleUiErrorText } from "../src/capture.ts";
Deno.test("CLI parses bounded capture filters", () => {
const parsed = parseArguments([
"capture",
"--scenario",
"scenario.json",
"--personas",
"owner,non-owner",
"--headed",
]);
assertEquals(parsed.command, "capture");
assertEquals(parsed.values.get("personas"), ["owner,non-owner"]);
assertEquals(parsed.flags.has("headed"), true);
});
Deno.test("visible UI error classification ignores ordinary status text", () => {
assertEquals(isVisibleUiErrorText("Refresh failed (401 Unauthorized)"), true);
assertEquals(isVisibleUiErrorText("Workspace list loaded"), false);
});
Deno.test("CLI rejects positional and missing option values", () => {
assertThrows(() => parseArguments(["capture", "scenario.json"]), Error, "unexpected argument");
});
+75
View File
@@ -0,0 +1,75 @@
import { assertEquals, assertRejects, assertStringIncludes } from "@std/assert";
import { join } from "@std/path";
import {
assertBundleIsSecretFree,
redactText,
safeUrl,
writePrivateJson,
} from "../src/artifacts.ts";
import { startOwnedProcesses, stopOwnedProcesses } from "../src/processes.ts";
Deno.test("redaction removes common credentials and query values", () => {
const redacted = redactText(
"Authorization: Bearer abc.def cookie=session-value token=secret-value",
["abc.def"],
);
assertStringIncludes(redacted, "[REDACTED]");
assertEquals(redacted.includes("abc.def"), false);
assertEquals(redacted.includes("session-value"), false);
assertEquals(
safeUrl("https://user:pass@example.test/path?token=secret#fragment"),
"https://example.test/path?token=%5BREDACTED%5D",
);
assertRejects(
async () => assertBundleIsSecretFree('{"authorization":"Bearer abc"}'),
Error,
"forbidden secret marker",
);
});
Deno.test("private JSON state uses owner-only permissions", async () => {
const directory = await Deno.makeTempDir();
try {
const path = join(directory, "state", "owner.json");
await writePrivateJson(path, { cookies: [], origins: [] });
assertEquals(JSON.parse(await Deno.readTextFile(path)), { cookies: [], origins: [] });
if (Deno.build.os !== "windows") assertEquals((await Deno.stat(path)).mode! & 0o777, 0o600);
} finally {
await Deno.remove(directory, { recursive: true });
}
});
Deno.test("owned process is terminated and its logs are redacted", async () => {
const directory = await Deno.makeTempDir();
const scenario = join(directory, "scenario.json");
await Deno.writeTextFile(scenario, "{}");
try {
const processes = await startOwnedProcesses(
[{
id: "fixture",
command: Deno.execPath(),
args: ["eval", 'console.log("authorization: secret-value"); setInterval(() => {}, 1000)'],
}],
scenario,
join(directory, "logs"),
["secret-value"],
);
assertEquals(processes.length, 1);
const logPath = join(directory, "logs", "fixture.stdout.log");
for (let attempt = 0; attempt < 20; attempt++) {
try {
if ((await Deno.readTextFile(logPath)).length > 0) break;
} catch {
// The output pump creates the file asynchronously.
}
await new Promise((resolve) => setTimeout(resolve, 10));
}
const diagnostics = await stopOwnedProcesses(processes);
assertEquals(diagnostics, []);
const log = await Deno.readTextFile(logPath);
assertEquals(log.includes("secret-value"), false);
assertStringIncludes(log, "[REDACTED]");
} finally {
await Deno.remove(directory, { recursive: true });
}
});
+98
View File
@@ -0,0 +1,98 @@
import { assertEquals, assertRejects, assertThrows } from "@std/assert";
import { join } from "@std/path";
import { cleanup } from "../src/lifecycle.ts";
import { interpolateEnvironment, loadScenario, validateBaseUrl } from "../src/scenario.ts";
function minimalScenario(extra = ""): string {
return `{
"schemaVersion": 1,
"id": "test-screen",
"title": "Test screen",
"baseUrl": "http://127.0.0.1:5173",
"personas": [{"id":"anonymous","label":"Anonymous","auth":{"kind":"anonymous"}}],
"viewports": [{"label":"desktop","width":1000,"height":800}],
"routes": [{
"id":"home","label":"Home","path":"/","goal":"Inspect home",
"dataState":"Fixture data","ready":{"kind":"selector","selector":"main"},
"capturePoints":[{"id":"initial","label":"Initial"}]
}]${extra}
}`;
}
Deno.test("scenario parser preserves explicit visual review context", async () => {
const directory = await Deno.makeTempDir();
try {
const path = join(directory, "scenario.json");
await Deno.writeTextFile(path, minimalScenario());
const scenario = await loadScenario(path);
assertEquals(scenario.personas[0].auth, { kind: "anonymous" });
assertEquals(scenario.routes[0].goal, "Inspect home");
assertEquals(scenario.routes[0].dataState, "Fixture data");
assertEquals(scenario.routes[0].ready, {
kind: "selector",
selector: "main",
timeoutMs: undefined,
});
assertEquals(scenario.reducedMotion, "reduce");
} finally {
await Deno.remove(directory, { recursive: true });
}
});
Deno.test("scenario parser rejects duplicate persona identity", async () => {
const directory = await Deno.makeTempDir();
try {
const path = join(directory, "scenario.json");
await Deno.writeTextFile(
path,
minimalScenario().replace(
'[{"id":"anonymous","label":"Anonymous","auth":{"kind":"anonymous"}}]',
'[{"id":"same","label":"First","auth":{"kind":"anonymous"}},{"id":"same","label":"Second","auth":{"kind":"anonymous"}}]',
),
);
await assertRejects(() => loadScenario(path), Error, "duplicate id: same");
} finally {
await Deno.remove(directory, { recursive: true });
}
});
Deno.test("base URL rejects embedded credentials and non-http schemes", () => {
assertThrows(
() => validateBaseUrl("https://user:secret@example.test"),
Error,
"must not contain credentials",
);
assertThrows(() => validateBaseUrl("file:///tmp/index.html"), Error, "must use http or https");
});
Deno.test("environment interpolation fails closed", () => {
assertEquals(
interpolateEnvironment("/w/${WORKSPACE_ID}", { WORKSPACE_ID: "W-test" }),
"/w/W-test",
);
assertThrows(
() => interpolateEnvironment("${MISSING}", {}),
Error,
"required environment variable is missing",
);
});
Deno.test("cleanup removes only complete review bundles beyond retention", async () => {
const directory = await Deno.makeTempDir();
try {
for (const name of ["one", "two", "three"]) {
const run = join(directory, name);
await Deno.mkdir(run);
await Deno.writeTextFile(join(run, "review-context.json"), "{}");
await new Promise((resolve) => setTimeout(resolve, 5));
}
const unrelated = join(directory, "auth");
await Deno.mkdir(unrelated);
await Deno.writeTextFile(join(unrelated, "state.json"), "secret");
const removed = await cleanup({ outputDirectory: directory, keep: 1 });
assertEquals(removed.length, 2);
assertEquals(await Deno.readTextFile(join(unrelated, "state.json")), "secret");
} finally {
await Deno.remove(directory, { recursive: true });
}
});