feat: add repeatable Web UX inspection workbench
This commit is contained in:
@@ -5,3 +5,5 @@
|
||||
.worktree
|
||||
*.local*
|
||||
.env
|
||||
.web-ux/
|
||||
artifacts/web-ux/
|
||||
|
||||
@@ -4,11 +4,15 @@ pkgs.mkShell {
|
||||
nixfmt
|
||||
deno
|
||||
git
|
||||
playwright-driver.browsers
|
||||
rustc
|
||||
cargo
|
||||
pkgs.sccache
|
||||
];
|
||||
|
||||
PLAYWRIGHT_BROWSERS_PATH = "${pkgs.playwright-driver.browsers}";
|
||||
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD = "1";
|
||||
|
||||
# sccache is additive to Cargo's shared build-dir, so keep its disk usage bounded.
|
||||
RUSTC_WRAPPER = "${pkgs.sccache}/bin/sccache";
|
||||
SCCACHE_CACHE_SIZE = "5G";
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
# Web UX inspection workbench
|
||||
|
||||
`tools/web-ux` is a development-only Playwright workbench for repeatable visual inspection of the
|
||||
real Web Workspace. It does not add a Yoi product Skill, Flow, Runtime capability, or browser
|
||||
automation route.
|
||||
|
||||
The workbench produces a **review context bundle** rather than treating a screenshot as evidence by
|
||||
itself. Every capture records the persona, route, viewport, theme, intended user goal, expected data
|
||||
state, sanitized document URL/status, console/page/request failures, screenshot hashes, an
|
||||
accessibility snapshot, source revision, and browser version.
|
||||
|
||||
## Environment
|
||||
|
||||
Enter the repository dev shell. The shell supplies the Nix-pinned Chromium build and sets
|
||||
`PLAYWRIGHT_BROWSERS_PATH`; Playwright does not download a browser at runtime.
|
||||
|
||||
```sh
|
||||
nix develop
|
||||
cd tools/web-ux
|
||||
deno task check
|
||||
deno task test
|
||||
deno task test:browser
|
||||
```
|
||||
|
||||
`test:browser` starts a deterministic fixture server owned by the test, captures distinct owner and
|
||||
non-owner contexts, verifies the review bundle, and proves server/browser cleanup. It must run
|
||||
inside `nix develop` so it uses the pinned browser.
|
||||
|
||||
The npm Playwright version in `deno.json` must match `pkgs.playwright-driver.version` in the pinned
|
||||
Nixpkgs input. Update both as one toolchain change.
|
||||
|
||||
## Scenario contract
|
||||
|
||||
Scenarios are reviewed JSON files under `scenarios/`. A scenario fixes:
|
||||
|
||||
- personas and whether each uses an isolated anonymous context or a local Playwright storage-state
|
||||
file;
|
||||
- explicit routes and user goals;
|
||||
- expected data state, viewports, theme, locale, timezone, and reduced-motion mode;
|
||||
- an explicit readiness condition for every route and optional interaction/capture-point conditions;
|
||||
- selectors and exact environment-derived text that must be redacted;
|
||||
- optional processes owned by the capture command, including an HTTP readiness URL.
|
||||
|
||||
`${UPPER_CASE_ENV}` values are expanded at runtime. URLs with embedded credentials are rejected.
|
||||
Route readiness is bounded and retried twice; it never relies on a fixed sleep. `network-idle` is
|
||||
available but should be used only for screens whose contract actually reaches idle. Prefer a stable
|
||||
screen-owned selector.
|
||||
|
||||
`workspace-control-plane.json` expects:
|
||||
|
||||
```sh
|
||||
export WEB_UX_BASE_URL='http://127.0.0.1:5173'
|
||||
export WORKSPACE_ID='<workspace-id>'
|
||||
```
|
||||
|
||||
## Authentication fixtures
|
||||
|
||||
Authentication state is local sensitive material. `.web-ux/` is gitignored, files are written with
|
||||
mode `0600`, state contents are never copied into a review bundle, and the CLI never prints cookies
|
||||
or credentials.
|
||||
|
||||
For an interactive Passkey/browser login:
|
||||
|
||||
```sh
|
||||
deno task web-ux auth \
|
||||
--scenario scenarios/workspace-control-plane.json \
|
||||
--persona owner
|
||||
```
|
||||
|
||||
The command opens Chromium at the configured login route, waits up to five minutes for the
|
||||
scenario's success URL, saves `storageState`, and closes the browser in `finally`. Repeat for
|
||||
`non-owner` using a real account with that permission projection.
|
||||
|
||||
A test fixture may already provide Playwright-compatible `{ cookies, origins }` state. Import it
|
||||
without putting its value on the command line:
|
||||
|
||||
```sh
|
||||
deno task web-ux auth \
|
||||
--scenario scenarios/workspace-control-plane.json \
|
||||
--persona owner \
|
||||
--import-state /private/path/owner-state.json
|
||||
```
|
||||
|
||||
Do not place passwords, bearer tokens, private keys, WebAuthn material, or inline cookies in a
|
||||
scenario, process arguments, a Repository URL, or `redact.text`. `redact.text` is only a final
|
||||
defense for a secret already supplied through an environment-owned fixture; it is not a credential
|
||||
transport.
|
||||
|
||||
## Capture and inspect
|
||||
|
||||
Capture a stable multi-persona bundle:
|
||||
|
||||
```sh
|
||||
deno task web-ux capture \
|
||||
--scenario scenarios/workspace-control-plane.json \
|
||||
--output ../../artifacts/web-ux \
|
||||
--run-id before-change
|
||||
```
|
||||
|
||||
Use filters for a bounded feedback loop:
|
||||
|
||||
```sh
|
||||
deno task web-ux capture \
|
||||
--scenario scenarios/workspace-control-plane.json \
|
||||
--output ../../artifacts/web-ux \
|
||||
--run-id ticket-list-after \
|
||||
--personas owner,non-owner \
|
||||
--routes tickets \
|
||||
--viewports desktop
|
||||
```
|
||||
|
||||
The command exits `2` when it produced evidence but observed UI/tool errors, and exits `1` when
|
||||
capture itself failed. It continues other route/persona captures after a bounded route failure.
|
||||
Inspect:
|
||||
|
||||
- `review-context.json` for the exact context, hashes, HTTP status, and failures;
|
||||
- `contact-sheet.png` with an image-capable reviewer for composition, hierarchy, density, clipping,
|
||||
empty/error states, and permission-specific affordances;
|
||||
- each `accessibility.md` for landmark/name/state evidence that a screenshot cannot prove;
|
||||
- `process-logs/` when the scenario owns a server process. Logs are redacted before writing.
|
||||
|
||||
The implementing agent must inspect the actual contact sheet (for example with `ViewImage`), record
|
||||
concrete findings, fix them, recapture under the same persona/route/viewport filters, and inspect
|
||||
the new evidence. Playwright success alone is not visual acceptance.
|
||||
|
||||
## Compare before and after
|
||||
|
||||
```sh
|
||||
deno task web-ux compare \
|
||||
--before ../../artifacts/web-ux/before-change/review-context.json \
|
||||
--after ../../artifacts/web-ux/after-change/review-context.json \
|
||||
--output ../../artifacts/web-ux/before-vs-after
|
||||
```
|
||||
|
||||
`comparison.html` and `comparison.png` show before, after, and pixel diff side by side.
|
||||
`comparison.json` records changed-pixel counts, dimension mismatches, unmatched capture keys, and
|
||||
diff hashes. Pixel differences are orientation evidence, not a correctness verdict; explain expected
|
||||
animation/font/data changes and inspect the actual UI.
|
||||
|
||||
Capture keys are stable across runs: `persona / route / viewport / capture-point`. Keep those
|
||||
identities unchanged when comparing the same user task.
|
||||
|
||||
## Process and artifact cleanup
|
||||
|
||||
The capture command owns only processes declared in its scenario. It starts them without a shell,
|
||||
records bounded/redacted output, and terminates the process and descendants on success, capture
|
||||
failure, or interruption observed by the command. It never stops an existing Yoi Server or Runtime
|
||||
that it did not start.
|
||||
|
||||
Old complete review bundles can be removed without touching auth state or arbitrary directories:
|
||||
|
||||
```sh
|
||||
deno task web-ux cleanup --output ../../artifacts/web-ux --keep 5 --older-than-days 14 --dry-run
|
||||
deno task web-ux cleanup --output ../../artifacts/web-ux --keep 5 --older-than-days 14
|
||||
```
|
||||
|
||||
Cleanup recognizes only directories containing `review-context.json`. `.web-ux/` and
|
||||
`artifacts/web-ux/` are ignored by Git. Keep a bundle outside Git or publish it through the approved
|
||||
immutable artifact channel when durable review evidence is required.
|
||||
|
||||
## Adding a scenario
|
||||
|
||||
1. Name the concrete user task and expected data state; do not write “looks correct”.
|
||||
2. Use the smallest persona/route/viewport matrix that proves the intended contract, including
|
||||
owner/non-owner/anonymous boundaries when permissions affect composition.
|
||||
3. Choose a screen-owned readiness selector or response. Avoid arbitrary sleeps.
|
||||
4. Add capture points only for meaningful visual states (initial, expanded detail, error, empty, and
|
||||
so on).
|
||||
5. Mark sensitive DOM regions with `[data-web-ux-redact]` or scenario selectors; never use review
|
||||
artifacts to transport secrets.
|
||||
6. Run `deno task check`, `deno task test`, one real capture, and inspect `contact-sheet.png` plus
|
||||
`review-context.json`.
|
||||
@@ -0,0 +1,99 @@
|
||||
import { assertEquals, assertRejects } from "@std/assert";
|
||||
import { join } from "@std/path";
|
||||
import { capture } from "../src/capture.ts";
|
||||
|
||||
async function freePort(): Promise<number> {
|
||||
const listener = Deno.listen({ hostname: "127.0.0.1", port: 0 });
|
||||
const port = (listener.addr as Deno.NetAddr).port;
|
||||
listener.close();
|
||||
return port;
|
||||
}
|
||||
|
||||
Deno.test("browser smoke captures distinct owner and non-owner evidence and cleans its server", async () => {
|
||||
const directory = await Deno.makeTempDir();
|
||||
const port = await freePort();
|
||||
const baseUrl = `http://127.0.0.1:${port}`;
|
||||
try {
|
||||
const authDirectory = join(directory, "auth");
|
||||
await Deno.mkdir(authDirectory);
|
||||
for (const persona of ["owner", "non-owner"]) {
|
||||
await Deno.writeTextFile(
|
||||
join(authDirectory, `${persona}.json`),
|
||||
JSON.stringify({
|
||||
cookies: [{
|
||||
name: "persona",
|
||||
value: persona,
|
||||
domain: "127.0.0.1",
|
||||
path: "/",
|
||||
expires: -1,
|
||||
httpOnly: true,
|
||||
secure: false,
|
||||
sameSite: "Lax",
|
||||
}],
|
||||
origins: [],
|
||||
}),
|
||||
);
|
||||
}
|
||||
const scenarioPath = join(directory, "scenario.json");
|
||||
await Deno.writeTextFile(
|
||||
scenarioPath,
|
||||
JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
id: "browser-smoke",
|
||||
title: "Browser smoke",
|
||||
baseUrl,
|
||||
personas: [
|
||||
{ id: "owner", label: "Owner", auth: { kind: "storage-state", path: "auth/owner.json" } },
|
||||
{
|
||||
id: "non-owner",
|
||||
label: "Non-owner",
|
||||
auth: { kind: "storage-state", path: "auth/non-owner.json" },
|
||||
},
|
||||
],
|
||||
viewports: [{ label: "desktop", width: 1000, height: 700 }],
|
||||
routes: [{
|
||||
id: "repositories",
|
||||
label: "Repositories",
|
||||
path: "/screen",
|
||||
goal: "Verify permission-specific composition",
|
||||
dataState: "Deterministic fixture repository",
|
||||
ready: { kind: "selector", selector: "main" },
|
||||
capturePoints: [{ id: "initial", label: "Initial" }],
|
||||
}],
|
||||
processes: [{
|
||||
id: "fixture-server",
|
||||
command: Deno.execPath(),
|
||||
args: [
|
||||
"run",
|
||||
"--allow-net",
|
||||
join(Deno.cwd(), "browser-tests/fixture_server.ts"),
|
||||
String(port),
|
||||
],
|
||||
readyUrl: `${baseUrl}/health`,
|
||||
}],
|
||||
}),
|
||||
);
|
||||
const manifest = await capture({
|
||||
scenarioPath,
|
||||
outputDirectory: join(directory, "artifacts"),
|
||||
runId: "multi-persona",
|
||||
});
|
||||
assertEquals(manifest.status, "completed");
|
||||
assertEquals(manifest.captures.map((item) => item.persona.id), ["owner", "non-owner"]);
|
||||
assertEquals(manifest.captures.every((item) => item.screenshots.length === 1), true);
|
||||
assertEquals(manifest.contactSheet.png, "contact-sheet.png");
|
||||
const runDirectory = join(directory, "artifacts", "multi-persona");
|
||||
const reviewContext = await Deno.readTextFile(join(runDirectory, "review-context.json"));
|
||||
assertEquals(reviewContext.includes('"cookies"'), false);
|
||||
if (Deno.build.os !== "windows") {
|
||||
const screenshot = join(runDirectory, manifest.captures[0].screenshots[0].path);
|
||||
assertEquals((await Deno.stat(screenshot)).mode! & 0o777, 0o600);
|
||||
}
|
||||
await assertRejects(
|
||||
() => fetch(`${baseUrl}/health`, { signal: AbortSignal.timeout(500) }),
|
||||
TypeError,
|
||||
);
|
||||
} finally {
|
||||
await Deno.remove(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
const port = Number(Deno.args[0]);
|
||||
if (!Number.isInteger(port) || port <= 0) throw new Error("port is required");
|
||||
|
||||
Deno.serve({ hostname: "127.0.0.1", port }, (request) => {
|
||||
const url = new URL(request.url);
|
||||
if (url.pathname === "/health") return new Response("ok");
|
||||
const cookie = request.headers.get("cookie") ?? "";
|
||||
const owner = cookie.includes("persona=owner");
|
||||
const title = owner ? "Owner repository settings" : "Repository settings";
|
||||
const action = owner
|
||||
? '<button type="button">Add repository</button>'
|
||||
: '<p role="note">Ask a Workspace owner to change repository access.</p>';
|
||||
return new Response(
|
||||
`<!doctype html><html><head><title>${title}</title><style>body{font:16px system-ui;margin:0}main{max-width:800px;margin:40px auto}header{border-bottom:1px solid #ccc;padding:16px}section{border:1px solid #ccc;padding:20px}button{background:#06c;color:white;padding:10px 20px}</style></head><body><header>Workspace</header><main><h1>${title}</h1><section><h2>main</h2><p>SSH repository access is configured.</p>${action}</section></main></body></html>`,
|
||||
{ headers: { "content-type": "text/html; charset=utf-8" } },
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,163 @@
|
||||
#!/usr/bin/env -S deno run --allow-env --allow-net --allow-read --allow-write --allow-run --allow-sys
|
||||
import { authenticate, cleanup } from "./src/lifecycle.ts";
|
||||
import { capture, describeCapture } from "./src/capture.ts";
|
||||
import { compare } from "./src/compare.ts";
|
||||
|
||||
const HELP = `Web UX inspection workbench
|
||||
|
||||
Usage:
|
||||
deno task web-ux auth --scenario <file> --persona <id> [--base-url <url>] [--import-state <file>] [--headless]
|
||||
deno task web-ux capture --scenario <file> --output <directory> [--base-url <url>] [--run-id <id>] [--personas <ids>] [--routes <ids>] [--viewports <ids>] [--headed]
|
||||
deno task web-ux compare --before <review-context.json> --after <review-context.json> --output <directory> [--threshold <0..1>]
|
||||
deno task web-ux cleanup --output <directory> [--keep <count>] [--older-than-days <days>] [--dry-run]
|
||||
|
||||
Comma-separate persona, route, and viewport ids. Auth state is local, mode 0600, and must not be committed.
|
||||
`;
|
||||
|
||||
type Arguments = { command: string; values: Map<string, string[]>; flags: Set<string> };
|
||||
|
||||
export function parseArguments(args: string[]): Arguments {
|
||||
const command = args.shift() ?? "help";
|
||||
const values = new Map<string, string[]>();
|
||||
const flags = new Set<string>();
|
||||
for (let index = 0; index < args.length; index++) {
|
||||
const token = args[index];
|
||||
if (!token.startsWith("--")) throw new Error(`unexpected argument: ${token}`);
|
||||
const name = token.slice(2);
|
||||
const next = args[index + 1];
|
||||
if (next === undefined || next.startsWith("--")) {
|
||||
flags.add(name);
|
||||
} else {
|
||||
const items = values.get(name) ?? [];
|
||||
items.push(next);
|
||||
values.set(name, items);
|
||||
index++;
|
||||
}
|
||||
}
|
||||
return { command, values, flags };
|
||||
}
|
||||
|
||||
function optional(args: Arguments, name: string): string | undefined {
|
||||
const values = args.values.get(name);
|
||||
if (!values) return undefined;
|
||||
if (values.length !== 1) throw new Error(`--${name} must be specified once`);
|
||||
return values[0];
|
||||
}
|
||||
|
||||
function required(args: Arguments, name: string): string {
|
||||
const value = optional(args, name);
|
||||
if (!value) throw new Error(`--${name} is required`);
|
||||
return value;
|
||||
}
|
||||
|
||||
function integer(args: Arguments, name: string, fallback?: number): number | undefined {
|
||||
const value = optional(args, name);
|
||||
if (value === undefined) return fallback;
|
||||
const parsed = Number(value);
|
||||
if (!Number.isInteger(parsed) || parsed < 0) {
|
||||
throw new Error(`--${name} must be a non-negative integer`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function list(args: Arguments, name: string): string[] | undefined {
|
||||
const value = optional(args, name);
|
||||
return value?.split(",").map((item) => item.trim()).filter(Boolean);
|
||||
}
|
||||
|
||||
function rejectUnknown(args: Arguments, allowedValues: string[], allowedFlags: string[]): void {
|
||||
for (const name of args.values.keys()) {
|
||||
if (!allowedValues.includes(name)) throw new Error(`unsupported option: --${name}`);
|
||||
}
|
||||
for (const name of args.flags) {
|
||||
if (!allowedFlags.includes(name)) throw new Error(`unsupported flag: --${name}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function main(rawArgs: string[]): Promise<number> {
|
||||
const args = parseArguments([...rawArgs]);
|
||||
if (args.command === "help" || args.flags.has("help")) {
|
||||
console.log(HELP);
|
||||
return 0;
|
||||
}
|
||||
if (args.command === "auth") {
|
||||
rejectUnknown(args, ["scenario", "persona", "base-url", "import-state", "timeout-ms"], [
|
||||
"headless",
|
||||
]);
|
||||
const path = await authenticate({
|
||||
scenarioPath: required(args, "scenario"),
|
||||
personaId: required(args, "persona"),
|
||||
baseUrl: optional(args, "base-url"),
|
||||
importState: optional(args, "import-state"),
|
||||
timeoutMs: integer(args, "timeout-ms"),
|
||||
headless: args.flags.has("headless"),
|
||||
});
|
||||
console.log(`auth state saved: ${path}`);
|
||||
return 0;
|
||||
}
|
||||
if (args.command === "capture") {
|
||||
rejectUnknown(args, [
|
||||
"scenario",
|
||||
"output",
|
||||
"base-url",
|
||||
"run-id",
|
||||
"personas",
|
||||
"routes",
|
||||
"viewports",
|
||||
], ["headed"]);
|
||||
const outputDirectory = required(args, "output");
|
||||
const manifest = await capture({
|
||||
scenarioPath: required(args, "scenario"),
|
||||
outputDirectory,
|
||||
baseUrl: optional(args, "base-url"),
|
||||
runId: optional(args, "run-id"),
|
||||
personas: list(args, "personas"),
|
||||
routes: list(args, "routes"),
|
||||
viewports: list(args, "viewports"),
|
||||
headed: args.flags.has("headed"),
|
||||
});
|
||||
console.log(describeCapture(manifest, outputDirectory));
|
||||
return manifest.status === "completed" ? 0 : 2;
|
||||
}
|
||||
if (args.command === "compare") {
|
||||
rejectUnknown(args, ["before", "after", "output", "threshold"], []);
|
||||
const thresholdValue = optional(args, "threshold");
|
||||
const threshold = thresholdValue === undefined ? undefined : Number(thresholdValue);
|
||||
if (
|
||||
threshold !== undefined && (!Number.isFinite(threshold) || threshold < 0 || threshold > 1)
|
||||
) {
|
||||
throw new Error("--threshold must be between 0 and 1");
|
||||
}
|
||||
const report = await compare({
|
||||
before: required(args, "before"),
|
||||
after: required(args, "after"),
|
||||
outputDirectory: required(args, "output"),
|
||||
threshold,
|
||||
});
|
||||
console.log(`comparison saved: ${report}`);
|
||||
return 0;
|
||||
}
|
||||
if (args.command === "cleanup") {
|
||||
rejectUnknown(args, ["output", "keep", "older-than-days"], ["dry-run"]);
|
||||
const removed = await cleanup({
|
||||
outputDirectory: required(args, "output"),
|
||||
keep: integer(args, "keep", 5)!,
|
||||
olderThanDays: integer(args, "older-than-days"),
|
||||
dryRun: args.flags.has("dry-run"),
|
||||
});
|
||||
for (const path of removed) {
|
||||
console.log(`${args.flags.has("dry-run") ? "would remove" : "removed"}: ${path}`);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
throw new Error(`unknown command: ${args.command}\n\n${HELP}`);
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
try {
|
||||
Deno.exit(await main(Deno.args));
|
||||
} catch (error) {
|
||||
console.error(error instanceof Error ? error.message : String(error));
|
||||
Deno.exit(1);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"lock": true,
|
||||
"imports": {
|
||||
"@std/assert": "jsr:@std/assert@1.0.19",
|
||||
"@std/path": "jsr:@std/path@1.1.4",
|
||||
"pixelmatch": "npm:pixelmatch@7.1.0",
|
||||
"playwright": "npm:playwright@1.59.1",
|
||||
"pngjs": "npm:pngjs@7.0.0"
|
||||
},
|
||||
"tasks": {
|
||||
"web-ux": "deno run --allow-env --allow-net --allow-read --allow-write --allow-run --allow-sys cli.ts",
|
||||
"check": "deno check cli.ts src/*.ts tests/*.ts browser-tests/*.ts",
|
||||
"test": "deno test --allow-env --allow-read --allow-write --allow-run --allow-sys tests",
|
||||
"test:browser": "deno test --allow-env --allow-net --allow-read --allow-write --allow-run --allow-sys browser-tests/capture_smoke_test.ts"
|
||||
},
|
||||
"fmt": {
|
||||
"lineWidth": 100
|
||||
}
|
||||
}
|
||||
Generated
+68
@@ -0,0 +1,68 @@
|
||||
{
|
||||
"version": "5",
|
||||
"specifiers": {
|
||||
"jsr:@std/assert@1.0.19": "1.0.19",
|
||||
"jsr:@std/internal@^1.0.12": "1.0.14",
|
||||
"jsr:@std/path@1.1.4": "1.1.4",
|
||||
"npm:pixelmatch@7.1.0": "7.1.0",
|
||||
"npm:playwright@1.59.1": "1.59.1",
|
||||
"npm:pngjs@7.0.0": "7.0.0"
|
||||
},
|
||||
"jsr": {
|
||||
"@std/assert@1.0.19": {
|
||||
"integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
|
||||
"dependencies": [
|
||||
"jsr:@std/internal"
|
||||
]
|
||||
},
|
||||
"@std/internal@1.0.14": {
|
||||
"integrity": "291516b3d4c35024d6ffbc0a9df5bf4c64116e05b50012cf846710152d2ffdf7"
|
||||
},
|
||||
"@std/path@1.1.4": {
|
||||
"integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
|
||||
"dependencies": [
|
||||
"jsr:@std/internal"
|
||||
]
|
||||
}
|
||||
},
|
||||
"npm": {
|
||||
"fsevents@2.3.2": {
|
||||
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
|
||||
"os": ["darwin"],
|
||||
"scripts": true
|
||||
},
|
||||
"pixelmatch@7.1.0": {
|
||||
"integrity": "sha512-1wrVzJ2STrpmONHKBy228LM1b84msXDUoAzVEl0R8Mz4Ce6EPr+IVtxm8+yvrqLYMHswREkjYFaMxnyGnaY3Ng==",
|
||||
"dependencies": [
|
||||
"pngjs"
|
||||
],
|
||||
"bin": true
|
||||
},
|
||||
"playwright-core@1.59.1": {
|
||||
"integrity": "sha512-HBV/RJg81z5BiiZ9yPzIiClYV/QMsDCKUyogwH9p3MCP6IYjUFu/MActgYAvK0oWyV9NlwM3GLBjADyWgydVyg==",
|
||||
"bin": true
|
||||
},
|
||||
"playwright@1.59.1": {
|
||||
"integrity": "sha512-C8oWjPR3F81yljW9o5OxcWzfh6avkVwDD2VYdwIGqTkl+OGFISgypqzfu7dOe4QNLL2aqcWBmI3PMtLIK233lw==",
|
||||
"dependencies": [
|
||||
"playwright-core"
|
||||
],
|
||||
"optionalDependencies": [
|
||||
"fsevents"
|
||||
],
|
||||
"bin": true
|
||||
},
|
||||
"pngjs@7.0.0": {
|
||||
"integrity": "sha512-LKWqWJRhstyYo9pGvgor/ivk2w94eSjE3RGVuzLGlr3NmD8bf7RcYGze1mNdEHRP6TRP6rMuDHk5t44hnTRyow=="
|
||||
}
|
||||
},
|
||||
"workspace": {
|
||||
"dependencies": [
|
||||
"jsr:@std/assert@1.0.19",
|
||||
"jsr:@std/path@1.1.4",
|
||||
"npm:pixelmatch@7.1.0",
|
||||
"npm:playwright@1.59.1",
|
||||
"npm:pngjs@7.0.0"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"id": "anonymous-entry",
|
||||
"title": "Anonymous entry and authentication review",
|
||||
"baseUrl": "${WEB_UX_BASE_URL}",
|
||||
"locale": "en-US",
|
||||
"timezone": "UTC",
|
||||
"colorScheme": "light",
|
||||
"reducedMotion": "reduce",
|
||||
"personas": [
|
||||
{ "id": "anonymous", "label": "Anonymous visitor", "auth": { "kind": "anonymous" } }
|
||||
],
|
||||
"viewports": [
|
||||
{ "label": "desktop", "width": 1440, "height": 1000, "deviceScaleFactor": 1 },
|
||||
{ "label": "mobile", "width": 390, "height": 844, "deviceScaleFactor": 1 }
|
||||
],
|
||||
"routes": [
|
||||
{
|
||||
"id": "entry",
|
||||
"label": "Authentication entry",
|
||||
"path": "/",
|
||||
"goal": "Understand the product and begin authentication without seeing Workspace-private content.",
|
||||
"dataState": "Fresh browser context with no cookies, local storage, or session state.",
|
||||
"ready": { "kind": "network-idle", "timeoutMs": 15000 },
|
||||
"capturePoints": [{ "id": "initial", "label": "Anonymous entry", "fullPage": true }]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"id": "workspace-control-plane",
|
||||
"title": "Workspace control-plane owner and non-owner review",
|
||||
"baseUrl": "${WEB_UX_BASE_URL}",
|
||||
"locale": "en-US",
|
||||
"timezone": "UTC",
|
||||
"colorScheme": "light",
|
||||
"reducedMotion": "reduce",
|
||||
"redact": {
|
||||
"selectors": ["[data-web-ux-redact]", "input[type=password]"],
|
||||
"text": []
|
||||
},
|
||||
"personas": [
|
||||
{
|
||||
"id": "owner",
|
||||
"label": "Workspace owner",
|
||||
"auth": { "kind": "storage-state", "path": "../../../.web-ux/auth/owner.json" },
|
||||
"login": { "path": "/", "successUrl": "/w/" }
|
||||
},
|
||||
{
|
||||
"id": "non-owner",
|
||||
"label": "Authenticated non-owner",
|
||||
"auth": { "kind": "storage-state", "path": "../../../.web-ux/auth/non-owner.json" },
|
||||
"login": { "path": "/", "successUrl": "/w/" }
|
||||
}
|
||||
],
|
||||
"viewports": [
|
||||
{ "label": "desktop", "width": 1440, "height": 1000, "deviceScaleFactor": 1 },
|
||||
{ "label": "narrow", "width": 900, "height": 900, "deviceScaleFactor": 1 }
|
||||
],
|
||||
"routes": [
|
||||
{
|
||||
"id": "workspace-home",
|
||||
"label": "Workspace overview",
|
||||
"path": "/w/${WORKSPACE_ID}",
|
||||
"goal": "Orient the user and expose the highest-value Workspace actions without internal authority noise.",
|
||||
"dataState": "Dogfood Workspace with current Runtime and Ticket data.",
|
||||
"ready": { "kind": "selector", "selector": "main", "timeoutMs": 20000 },
|
||||
"capturePoints": [{ "id": "initial", "label": "Initial viewport", "fullPage": true }]
|
||||
},
|
||||
{
|
||||
"id": "tickets",
|
||||
"label": "Ticket lanes",
|
||||
"path": "/w/${WORKSPACE_ID}/tickets",
|
||||
"goal": "Scan actionable Ticket lanes and reach the primary authoring action in the initial viewport.",
|
||||
"dataState": "Planning, ready, queued, in-progress, and completed Tickets from the selected Workspace.",
|
||||
"ready": { "kind": "selector", "selector": "main", "timeoutMs": 30000 },
|
||||
"capturePoints": [{ "id": "initial", "label": "Loaded Ticket lanes", "fullPage": true }]
|
||||
},
|
||||
{
|
||||
"id": "workers",
|
||||
"label": "Workers",
|
||||
"path": "/w/${WORKSPACE_ID}/workers",
|
||||
"goal": "Find current Worker state and the new-Worker action without exposing transport internals as the primary content.",
|
||||
"dataState": "Current Workspace Worker projection with mixed lifecycle states.",
|
||||
"ready": { "kind": "selector", "selector": "main", "timeoutMs": 20000 },
|
||||
"capturePoints": [{ "id": "initial", "label": "Worker list", "fullPage": true }]
|
||||
},
|
||||
{
|
||||
"id": "repositories",
|
||||
"label": "Repository settings",
|
||||
"path": "/w/${WORKSPACE_ID}/settings/repositories",
|
||||
"goal": "Review repository access as an owner and verify non-owner composition does not expose unusable owner actions.",
|
||||
"dataState": "Workspace repository catalog projected through current permissions.",
|
||||
"ready": { "kind": "selector", "selector": "main", "timeoutMs": 20000 },
|
||||
"capturePoints": [{ "id": "initial", "label": "Repository settings", "fullPage": true }]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
import { dirname, relative, resolve } from "@std/path";
|
||||
|
||||
const SECRET_PATTERNS: RegExp[] = [
|
||||
/\b(authorization|cookie|set-cookie|x-csrf-token)\b\s*[:=]\s*[^\s,;]+/gi,
|
||||
/\b(bearer)\s+[a-z0-9._~+\/-]+=*/gi,
|
||||
/\b(session|token|credential|password|passkey|private[_ -]?key)\b\s*[:=]\s*["']?[^\s,"'};]+/gi,
|
||||
];
|
||||
|
||||
export function redactText(value: string, exactSecrets: string[] = []): string {
|
||||
let result = value;
|
||||
for (const secret of exactSecrets) {
|
||||
if (secret) result = result.replaceAll(secret, "[REDACTED]");
|
||||
}
|
||||
for (const pattern of SECRET_PATTERNS) result = result.replaceAll(pattern, "$1=[REDACTED]");
|
||||
return result;
|
||||
}
|
||||
|
||||
export function bounded(value: string, maximum = 1000): string {
|
||||
const normalized = value.replaceAll(/\s+/g, " ").trim();
|
||||
return normalized.length <= maximum ? normalized : `${normalized.slice(0, maximum - 1)}…`;
|
||||
}
|
||||
|
||||
export function safeUrl(value: string, baseUrl?: string): string {
|
||||
try {
|
||||
const url = new URL(value, baseUrl);
|
||||
url.username = "";
|
||||
url.password = "";
|
||||
for (const key of [...url.searchParams.keys()]) url.searchParams.set(key, "[REDACTED]");
|
||||
url.hash = "";
|
||||
return url.toString();
|
||||
} catch {
|
||||
return "[invalid-url]";
|
||||
}
|
||||
}
|
||||
|
||||
export function assertBundleIsSecretFree(serialized: string, exactSecrets: string[] = []): void {
|
||||
const lower = serialized.toLowerCase();
|
||||
for (const forbidden of ["authorization:", "set-cookie:", "cookie:", "bearer "]) {
|
||||
if (lower.includes(forbidden)) {
|
||||
throw new Error(`review bundle contains forbidden secret marker: ${forbidden}`);
|
||||
}
|
||||
}
|
||||
for (const secret of exactSecrets) {
|
||||
if (secret && serialized.includes(secret)) {
|
||||
throw new Error("review bundle contains configured secret text");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export async function ensurePrivateDirectory(path: string): Promise<void> {
|
||||
await Deno.mkdir(path, { recursive: true, mode: 0o700 });
|
||||
if (Deno.build.os !== "windows") await Deno.chmod(path, 0o700);
|
||||
}
|
||||
|
||||
export async function makePrivate(path: string): Promise<void> {
|
||||
if (Deno.build.os !== "windows") await Deno.chmod(path, 0o600);
|
||||
}
|
||||
|
||||
export async function writePrivateJson(path: string, value: unknown): Promise<void> {
|
||||
await ensurePrivateDirectory(dirname(path));
|
||||
await Deno.writeTextFile(path, `${JSON.stringify(value, null, 2)}\n`, { mode: 0o600 });
|
||||
if (Deno.build.os !== "windows") await Deno.chmod(path, 0o600);
|
||||
}
|
||||
|
||||
export function logicalPath(repositoryRoot: string, path: string): string {
|
||||
const absolute = resolve(path);
|
||||
const logical = relative(repositoryRoot, absolute);
|
||||
if (logical === "" || (!logical.startsWith("..") && !logical.startsWith("/"))) {
|
||||
return logical || ".";
|
||||
}
|
||||
return absolute;
|
||||
}
|
||||
|
||||
export async function sha256File(path: string): Promise<string> {
|
||||
const bytes = await Deno.readFile(path);
|
||||
const digest = await crypto.subtle.digest("SHA-256", bytes);
|
||||
return [...new Uint8Array(digest)].map((byte) => byte.toString(16).padStart(2, "0")).join("");
|
||||
}
|
||||
@@ -0,0 +1,540 @@
|
||||
import { basename, dirname, join, relative, resolve } from "@std/path";
|
||||
import { type Browser, chromium, type Page, type Response } from "playwright";
|
||||
import {
|
||||
assertBundleIsSecretFree,
|
||||
bounded,
|
||||
ensurePrivateDirectory,
|
||||
makePrivate,
|
||||
redactText,
|
||||
safeUrl,
|
||||
sha256File,
|
||||
} from "./artifacts.ts";
|
||||
import { type RunningProcess, startOwnedProcesses, stopOwnedProcesses } from "./processes.ts";
|
||||
import {
|
||||
interpolateEnvironment,
|
||||
loadScenario,
|
||||
resolveScenarioPath,
|
||||
validateBaseUrl,
|
||||
} from "./scenario.ts";
|
||||
import type {
|
||||
CaptureError,
|
||||
CaptureEvidence,
|
||||
CapturePoint,
|
||||
Interaction,
|
||||
Persona,
|
||||
ReadyCondition,
|
||||
ReviewContext,
|
||||
RouteScenario,
|
||||
Scenario,
|
||||
ScreenshotEvidence,
|
||||
Viewport,
|
||||
} from "./types.ts";
|
||||
|
||||
export type CaptureOptions = {
|
||||
scenarioPath: string;
|
||||
outputDirectory: string;
|
||||
baseUrl?: string;
|
||||
runId?: string;
|
||||
personas?: string[];
|
||||
routes?: string[];
|
||||
viewports?: string[];
|
||||
headed?: boolean;
|
||||
};
|
||||
|
||||
type SourceState = { revision: string | null; dirty: boolean | null };
|
||||
|
||||
function slug(value: string): string {
|
||||
return value.replaceAll(/[^a-zA-Z0-9.-]+/g, "-").replaceAll(/^-+|-+$/g, "").toLowerCase();
|
||||
}
|
||||
|
||||
function viewportId(viewport: Viewport): string {
|
||||
return viewport.label ?? `${viewport.width}x${viewport.height}`;
|
||||
}
|
||||
|
||||
function timestampId(): string {
|
||||
return new Date().toISOString().replaceAll(/[:.]/g, "-");
|
||||
}
|
||||
|
||||
async function sourceState(): Promise<SourceState> {
|
||||
try {
|
||||
const [revision, status] = await Promise.all([
|
||||
new Deno.Command("git", { args: ["rev-parse", "HEAD"], stdout: "piped", stderr: "null" })
|
||||
.output(),
|
||||
new Deno.Command("git", { args: ["status", "--porcelain"], stdout: "piped", stderr: "null" })
|
||||
.output(),
|
||||
]);
|
||||
return {
|
||||
revision: revision.success ? new TextDecoder().decode(revision.stdout).trim() : null,
|
||||
dirty: status.success ? new TextDecoder().decode(status.stdout).trim().length > 0 : null,
|
||||
};
|
||||
} catch {
|
||||
return { revision: null, dirty: null };
|
||||
}
|
||||
}
|
||||
|
||||
function selectById<T extends { id: string }>(
|
||||
values: T[],
|
||||
requested: string[] | undefined,
|
||||
kind: string,
|
||||
): T[] {
|
||||
if (!requested || requested.length === 0) return values;
|
||||
const requestedSet = new Set(requested);
|
||||
const selected = values.filter((value) => requestedSet.has(value.id));
|
||||
const missing = [...requestedSet].filter((id) => !selected.some((value) => value.id === id));
|
||||
if (missing.length > 0) throw new Error(`unknown ${kind}: ${missing.join(", ")}`);
|
||||
return selected;
|
||||
}
|
||||
|
||||
function selectViewports(values: Viewport[], requested: string[] | undefined): Viewport[] {
|
||||
if (!requested || requested.length === 0) return values;
|
||||
const requestedSet = new Set(requested);
|
||||
const selected = values.filter((value) => requestedSet.has(viewportId(value)));
|
||||
const missing = [...requestedSet].filter((id) =>
|
||||
!selected.some((value) => viewportId(value) === id)
|
||||
);
|
||||
if (missing.length > 0) throw new Error(`unknown viewports: ${missing.join(", ")}`);
|
||||
return selected;
|
||||
}
|
||||
|
||||
function responseMatches(
|
||||
response: Response,
|
||||
ready: Extract<ReadyCondition, { kind: "response" }>,
|
||||
): boolean {
|
||||
const pattern = new RegExp(ready.urlPattern);
|
||||
return pattern.test(response.url()) &&
|
||||
(ready.status === undefined || response.status() === ready.status);
|
||||
}
|
||||
|
||||
async function waitReady(
|
||||
page: Page,
|
||||
ready: ReadyCondition,
|
||||
navigation?: Response | null,
|
||||
): Promise<void> {
|
||||
const timeout = ready.timeoutMs ?? 15_000;
|
||||
if (ready.kind === "selector") {
|
||||
await page.locator(ready.selector).first().waitFor({ state: "visible", timeout });
|
||||
return;
|
||||
}
|
||||
if (ready.kind === "network-idle") {
|
||||
await page.waitForLoadState("networkidle", { timeout });
|
||||
return;
|
||||
}
|
||||
if (navigation && responseMatches(navigation, ready)) return;
|
||||
await page.waitForResponse((response) => responseMatches(response, ready), { timeout });
|
||||
}
|
||||
|
||||
async function performInteraction(page: Page, interaction: Interaction): Promise<void> {
|
||||
if (interaction.action === "wait") return await waitReady(page, interaction.ready);
|
||||
const locator = page.locator(interaction.selector).first();
|
||||
const timeout = interaction.timeoutMs ?? 10_000;
|
||||
if (interaction.action === "click") return await locator.click({ timeout });
|
||||
if (interaction.action === "fill") {
|
||||
return await locator.fill(interpolateEnvironment(interaction.value), { timeout });
|
||||
}
|
||||
await locator.press(interaction.key, { timeout });
|
||||
}
|
||||
|
||||
async function retry<T>(label: string, operation: () => Promise<T>): Promise<T> {
|
||||
let last: unknown;
|
||||
for (let attempt = 1; attempt <= 2; attempt++) {
|
||||
try {
|
||||
return await operation();
|
||||
} catch (error) {
|
||||
last = error;
|
||||
if (attempt < 2) await new Promise((resolve) => setTimeout(resolve, 350));
|
||||
}
|
||||
}
|
||||
throw new Error(
|
||||
`${label} failed after 2 attempts: ${last instanceof Error ? last.message : String(last)}`,
|
||||
);
|
||||
}
|
||||
|
||||
async function hideRedactedSelectors(page: Page, selectors: string[]): Promise<void> {
|
||||
if (selectors.length === 0) return;
|
||||
const escaped = selectors.join(",\n");
|
||||
await page.addStyleTag({ content: `${escaped} { visibility: hidden !important; }` });
|
||||
}
|
||||
|
||||
export function isVisibleUiErrorText(content: string): boolean {
|
||||
return /\b(error|failed|unauthorized|forbidden|not found)\b/i.test(content);
|
||||
}
|
||||
|
||||
async function collectVisibleUiErrors(
|
||||
page: Page,
|
||||
errors: CaptureError[],
|
||||
secrets: string[],
|
||||
): Promise<void> {
|
||||
const alerts = page.locator('[role="alert"], [aria-live="assertive"]');
|
||||
for (let index = 0; index < await alerts.count(); index++) {
|
||||
const alert = alerts.nth(index);
|
||||
if (!await alert.isVisible().catch(() => false)) continue;
|
||||
const content = (await alert.innerText().catch(() => "")).trim();
|
||||
if (!isVisibleUiErrorText(content)) continue;
|
||||
const message = `visible UI error: ${bounded(redactText(content, secrets), 500)}`;
|
||||
if (!errors.some((error) => error.kind === "document" && error.message === message)) {
|
||||
errors.push({ kind: "document", message });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function capturePoint(
|
||||
page: Page,
|
||||
runDirectory: string,
|
||||
persona: Persona,
|
||||
route: RouteScenario,
|
||||
viewport: Viewport,
|
||||
point: CapturePoint,
|
||||
documentResponse: Response | null,
|
||||
errors: CaptureError[],
|
||||
scenario: Scenario,
|
||||
): Promise<CaptureEvidence> {
|
||||
const startedAt = new Date().toISOString();
|
||||
for (const interaction of point.interaction ?? []) await performInteraction(page, interaction);
|
||||
if (point.ready) await waitReady(page, point.ready);
|
||||
await hideRedactedSelectors(page, scenario.redact?.selectors ?? []);
|
||||
await collectVisibleUiErrors(page, errors, scenario.redact?.text ?? []);
|
||||
const directory = join(
|
||||
runDirectory,
|
||||
"captures",
|
||||
persona.id,
|
||||
route.id,
|
||||
viewportId(viewport),
|
||||
point.id,
|
||||
);
|
||||
await ensurePrivateDirectory(directory);
|
||||
const viewportScreenshot = join(directory, "viewport.png");
|
||||
await page.screenshot({ path: viewportScreenshot, fullPage: false, animations: "disabled" });
|
||||
await makePrivate(viewportScreenshot);
|
||||
const screenshots: ScreenshotEvidence[] = [{
|
||||
kind: "viewport",
|
||||
path: relative(runDirectory, viewportScreenshot),
|
||||
sha256: await sha256File(viewportScreenshot),
|
||||
}];
|
||||
if (point.fullPage) {
|
||||
const fullPageScreenshot = join(directory, "full-page.png");
|
||||
await page.screenshot({ path: fullPageScreenshot, fullPage: true, animations: "disabled" });
|
||||
await makePrivate(fullPageScreenshot);
|
||||
screenshots.push({
|
||||
kind: "full-page",
|
||||
path: relative(runDirectory, fullPageScreenshot),
|
||||
sha256: await sha256File(fullPageScreenshot),
|
||||
});
|
||||
}
|
||||
let snapshotPath: string | null = null;
|
||||
try {
|
||||
const snapshot = await page.locator("body").ariaSnapshot({ timeout: 5_000 });
|
||||
const redacted = redactText(snapshot, scenario.redact?.text ?? []);
|
||||
const target = join(directory, "accessibility.md");
|
||||
await Deno.writeTextFile(target, redacted, { mode: 0o600 });
|
||||
snapshotPath = relative(runDirectory, target);
|
||||
} catch (error) {
|
||||
errors.push({
|
||||
kind: "tool",
|
||||
message: `accessibility snapshot failed: ${
|
||||
bounded(error instanceof Error ? error.message : String(error))
|
||||
}`,
|
||||
});
|
||||
}
|
||||
return {
|
||||
persona: { id: persona.id, label: persona.label },
|
||||
route: { id: route.id, path: route.path, goal: route.goal, dataState: route.dataState },
|
||||
viewport,
|
||||
theme: scenario.colorScheme ?? "light",
|
||||
capturePoint: { id: point.id, label: point.label },
|
||||
document: { url: safeUrl(page.url()), status: documentResponse?.status() ?? null },
|
||||
screenshots,
|
||||
snapshotPath,
|
||||
errors: [...errors],
|
||||
startedAt,
|
||||
finishedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
function screenshotDataUrl(bytes: Uint8Array): string {
|
||||
let binary = "";
|
||||
for (const byte of bytes) binary += String.fromCharCode(byte);
|
||||
return `data:image/png;base64,${btoa(binary)}`;
|
||||
}
|
||||
|
||||
function escapeHtml(value: string): string {
|
||||
return value.replaceAll("&", "&").replaceAll("<", "<").replaceAll(">", ">").replaceAll(
|
||||
'"',
|
||||
""",
|
||||
);
|
||||
}
|
||||
|
||||
async function createContactSheet(
|
||||
browser: Browser,
|
||||
runDirectory: string,
|
||||
captures: CaptureEvidence[],
|
||||
): Promise<{ html: string | null; png: string | null }> {
|
||||
const cells: string[] = [];
|
||||
for (const capture of captures) {
|
||||
const screenshot = capture.screenshots.find((item) => item.kind === "viewport") ??
|
||||
capture.screenshots[0];
|
||||
if (!screenshot) continue;
|
||||
const bytes = await Deno.readFile(join(runDirectory, screenshot.path));
|
||||
cells.push(
|
||||
`<figure><img src="${screenshotDataUrl(bytes)}"><figcaption><strong>${
|
||||
escapeHtml(capture.persona.label)
|
||||
} · ${escapeHtml(capture.route.id)}</strong><br>${
|
||||
escapeHtml(viewportId(capture.viewport))
|
||||
} · ${escapeHtml(capture.capturePoint.label)}<br><small>${
|
||||
escapeHtml(capture.route.dataState)
|
||||
}</small>${
|
||||
capture.errors.length > 0
|
||||
? `<br><strong class="errors">${capture.errors.length} captured error(s)</strong>`
|
||||
: ""
|
||||
}</figcaption></figure>`,
|
||||
);
|
||||
}
|
||||
if (cells.length === 0) return { html: null, png: null };
|
||||
const html =
|
||||
`<!doctype html><meta charset="utf-8"><title>Web UX review contact sheet</title><style>body{margin:0;padding:20px;background:#e8e8e8;color:#111;font:14px system-ui}main{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:20px}figure{margin:0;background:white;border:1px solid #aaa;padding:10px;box-shadow:0 2px 8px #0002}img{width:100%;height:auto;display:block;border:1px solid #ddd}figcaption{padding-top:8px;line-height:1.45}small{color:#555}.errors{color:#b42318}</style><main>${
|
||||
cells.join("")
|
||||
}</main>`;
|
||||
const htmlPath = join(runDirectory, "contact-sheet.html");
|
||||
const pngPath = join(runDirectory, "contact-sheet.png");
|
||||
await Deno.writeTextFile(htmlPath, html, { mode: 0o600 });
|
||||
const page = await browser.newPage({ viewport: { width: 1600, height: 1000 } });
|
||||
try {
|
||||
await page.setContent(html, { waitUntil: "load" });
|
||||
await page.screenshot({ path: pngPath, fullPage: true, animations: "disabled" });
|
||||
await makePrivate(pngPath);
|
||||
} finally {
|
||||
await page.close();
|
||||
}
|
||||
return { html: relative(runDirectory, htmlPath), png: relative(runDirectory, pngPath) };
|
||||
}
|
||||
|
||||
export async function capture(options: CaptureOptions): Promise<ReviewContext> {
|
||||
const scenarioPath = resolve(options.scenarioPath);
|
||||
const scenario = await loadScenario(scenarioPath);
|
||||
const baseUrl = validateBaseUrl(
|
||||
interpolateEnvironment(
|
||||
options.baseUrl ?? Deno.env.get("WEB_UX_BASE_URL") ?? scenario.baseUrl ?? "",
|
||||
),
|
||||
);
|
||||
const personas = selectById(scenario.personas, options.personas, "personas");
|
||||
const routes = selectById(scenario.routes, options.routes, "routes");
|
||||
const viewports = selectViewports(scenario.viewports, options.viewports);
|
||||
const runId = slug(options.runId ?? `${scenario.id}-${timestampId()}`);
|
||||
const runDirectory = resolve(options.outputDirectory, runId);
|
||||
try {
|
||||
await Deno.stat(runDirectory);
|
||||
throw new Error(`run directory already exists: ${runDirectory}`);
|
||||
} catch (error) {
|
||||
if (!(error instanceof Deno.errors.NotFound)) throw error;
|
||||
}
|
||||
await ensurePrivateDirectory(runDirectory);
|
||||
const secrets = scenario.redact?.text ?? [];
|
||||
let browser: Browser | null = null;
|
||||
let processes: RunningProcess[] = [];
|
||||
const captures: CaptureEvidence[] = [];
|
||||
const diagnostics: CaptureError[] = [];
|
||||
let contactSheet = { html: null as string | null, png: null as string | null };
|
||||
let browserVersion = "unknown";
|
||||
let status: ReviewContext["status"] = "completed";
|
||||
try {
|
||||
processes = await startOwnedProcesses(
|
||||
scenario.processes ?? [],
|
||||
scenarioPath,
|
||||
join(runDirectory, "process-logs"),
|
||||
secrets,
|
||||
);
|
||||
browser = await chromium.launch({ headless: !options.headed });
|
||||
browserVersion = browser.version();
|
||||
for (const persona of personas) {
|
||||
const storageState = persona.auth.kind === "storage-state"
|
||||
? resolveScenarioPath(scenarioPath, persona.auth.path)
|
||||
: undefined;
|
||||
if (storageState) await Deno.stat(storageState);
|
||||
for (const viewport of viewports) {
|
||||
const context = await browser.newContext({
|
||||
storageState,
|
||||
viewport: { width: viewport.width, height: viewport.height },
|
||||
deviceScaleFactor: viewport.deviceScaleFactor ?? 1,
|
||||
locale: scenario.locale,
|
||||
timezoneId: scenario.timezone,
|
||||
colorScheme: scenario.colorScheme,
|
||||
reducedMotion: scenario.reducedMotion,
|
||||
});
|
||||
try {
|
||||
for (const route of routes) {
|
||||
const routeErrors: CaptureError[] = [];
|
||||
const page = await context.newPage();
|
||||
page.on("console", (message) => {
|
||||
if (message.type() === "error") {
|
||||
routeErrors.push({
|
||||
kind: "console",
|
||||
message: bounded(redactText(message.text(), secrets)),
|
||||
});
|
||||
}
|
||||
});
|
||||
page.on(
|
||||
"pageerror",
|
||||
(error) =>
|
||||
routeErrors.push({
|
||||
kind: "page",
|
||||
message: bounded(redactText(error.message, secrets)),
|
||||
}),
|
||||
);
|
||||
page.on(
|
||||
"requestfailed",
|
||||
(request) =>
|
||||
routeErrors.push({
|
||||
kind: "request",
|
||||
message: bounded(
|
||||
redactText(request.failure()?.errorText ?? "request failed", secrets),
|
||||
),
|
||||
url: safeUrl(request.url()),
|
||||
}),
|
||||
);
|
||||
page.on("response", (response) => {
|
||||
if (response.status() >= 400) {
|
||||
routeErrors.push({
|
||||
kind: "request",
|
||||
message: `HTTP ${response.status()}`,
|
||||
url: safeUrl(response.url()),
|
||||
status: response.status(),
|
||||
});
|
||||
}
|
||||
});
|
||||
try {
|
||||
const routePath = interpolateEnvironment(route.path);
|
||||
const targetUrl = new URL(routePath, `${baseUrl}/`).toString();
|
||||
const response = await retry(`navigate ${route.id}`, async () => {
|
||||
const ready = route.ready;
|
||||
const responseReady = ready.kind === "response"
|
||||
? page.waitForResponse(
|
||||
(candidate) => responseMatches(candidate, ready),
|
||||
{ timeout: ready.timeoutMs ?? 15_000 },
|
||||
)
|
||||
: null;
|
||||
try {
|
||||
const navigation = await page.goto(targetUrl, {
|
||||
waitUntil: "domcontentloaded",
|
||||
timeout: 20_000,
|
||||
});
|
||||
if (responseReady) await responseReady;
|
||||
else await waitReady(page, ready, navigation);
|
||||
return navigation;
|
||||
} catch (error) {
|
||||
responseReady?.catch(() => undefined);
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
if (response && response.status() >= 400) {
|
||||
routeErrors.push({
|
||||
kind: "document",
|
||||
message: `document returned HTTP ${response.status()}`,
|
||||
url: safeUrl(response.url()),
|
||||
status: response.status(),
|
||||
});
|
||||
}
|
||||
for (const point of route.capturePoints) {
|
||||
captures.push(
|
||||
await capturePoint(
|
||||
page,
|
||||
runDirectory,
|
||||
persona,
|
||||
route,
|
||||
viewport,
|
||||
point,
|
||||
response,
|
||||
routeErrors,
|
||||
scenario,
|
||||
),
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
status = "completed-with-errors";
|
||||
routeErrors.push({
|
||||
kind: "tool",
|
||||
message: bounded(
|
||||
redactText(error instanceof Error ? error.message : String(error), secrets),
|
||||
),
|
||||
});
|
||||
captures.push({
|
||||
persona: { id: persona.id, label: persona.label },
|
||||
route: {
|
||||
id: route.id,
|
||||
path: route.path,
|
||||
goal: route.goal,
|
||||
dataState: route.dataState,
|
||||
},
|
||||
viewport,
|
||||
theme: scenario.colorScheme ?? "light",
|
||||
capturePoint: { id: "failed", label: "Capture failed" },
|
||||
document: { url: safeUrl(page.url()), status: null },
|
||||
screenshots: [],
|
||||
snapshotPath: null,
|
||||
errors: routeErrors,
|
||||
startedAt: new Date().toISOString(),
|
||||
finishedAt: new Date().toISOString(),
|
||||
});
|
||||
} finally {
|
||||
await page.close();
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
await context.close();
|
||||
}
|
||||
}
|
||||
}
|
||||
contactSheet = await createContactSheet(browser, runDirectory, captures);
|
||||
if (captures.some((capture) => capture.errors.length > 0)) status = "completed-with-errors";
|
||||
} catch (error) {
|
||||
status = "failed";
|
||||
diagnostics.push({
|
||||
kind: "tool",
|
||||
message: bounded(redactText(error instanceof Error ? error.message : String(error), secrets)),
|
||||
});
|
||||
} finally {
|
||||
if (browser) {
|
||||
await browser.close().catch((error) =>
|
||||
diagnostics.push({
|
||||
kind: "tool",
|
||||
message: `browser cleanup failed: ${bounded(String(error))}`,
|
||||
})
|
||||
);
|
||||
}
|
||||
diagnostics.push(...await stopOwnedProcesses(processes));
|
||||
}
|
||||
if (diagnostics.length > 0 && status === "completed") status = "completed-with-errors";
|
||||
const manifest: ReviewContext = {
|
||||
schemaVersion: 1,
|
||||
runId,
|
||||
scenario: {
|
||||
id: scenario.id,
|
||||
title: scenario.title,
|
||||
sourcePath: relative(Deno.cwd(), scenarioPath),
|
||||
},
|
||||
source: await sourceState(),
|
||||
baseUrl: safeUrl(baseUrl),
|
||||
browser: { name: "chromium", version: browserVersion },
|
||||
createdAt: new Date().toISOString(),
|
||||
status,
|
||||
filters: {
|
||||
personas: personas.map((item) => item.id),
|
||||
routes: routes.map((item) => item.id),
|
||||
viewports: viewports.map(viewportId),
|
||||
},
|
||||
captures,
|
||||
contactSheet,
|
||||
diagnostics,
|
||||
};
|
||||
const serialized = `${JSON.stringify(manifest, null, 2)}\n`;
|
||||
assertBundleIsSecretFree(serialized, secrets);
|
||||
await Deno.writeTextFile(join(runDirectory, "review-context.json"), serialized, { mode: 0o600 });
|
||||
if (status === "failed") {
|
||||
throw new Error(`capture failed; inspect ${join(runDirectory, "review-context.json")}`);
|
||||
}
|
||||
return manifest;
|
||||
}
|
||||
|
||||
export function describeCapture(manifest: ReviewContext, outputDirectory: string): string {
|
||||
return `${manifest.status}: ${manifest.captures.length} capture(s); ${
|
||||
join(outputDirectory, manifest.runId, "review-context.json")
|
||||
}`;
|
||||
}
|
||||
@@ -0,0 +1,173 @@
|
||||
import { Buffer } from "node:buffer";
|
||||
import { basename, dirname, join, relative, resolve } from "@std/path";
|
||||
import { chromium } from "playwright";
|
||||
import pixelmatch from "pixelmatch";
|
||||
import { PNG } from "pngjs";
|
||||
import { ensurePrivateDirectory, makePrivate, sha256File } from "./artifacts.ts";
|
||||
import type { CaptureEvidence, ReviewContext } from "./types.ts";
|
||||
|
||||
export type CompareOptions = {
|
||||
before: string;
|
||||
after: string;
|
||||
outputDirectory: string;
|
||||
threshold?: number;
|
||||
};
|
||||
|
||||
type Pair = {
|
||||
key: string;
|
||||
before: CaptureEvidence;
|
||||
after: CaptureEvidence;
|
||||
beforePath: string;
|
||||
afterPath: string;
|
||||
diffPath: string;
|
||||
changedPixels: number;
|
||||
totalPixels: number;
|
||||
dimensionMismatch: boolean;
|
||||
};
|
||||
|
||||
function key(capture: CaptureEvidence): string {
|
||||
const viewport = capture.viewport.label ?? `${capture.viewport.width}x${capture.viewport.height}`;
|
||||
return [capture.persona.id, capture.route.id, viewport, capture.capturePoint.id].join("/");
|
||||
}
|
||||
|
||||
async function readManifest(path: string): Promise<ReviewContext> {
|
||||
const parsed = JSON.parse(await Deno.readTextFile(path));
|
||||
if (parsed.schemaVersion !== 1 || !Array.isArray(parsed.captures)) {
|
||||
throw new Error(`not a web-ux review context: ${path}`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function viewportScreenshot(capture: CaptureEvidence): string | null {
|
||||
return capture.screenshots.find((item) => item.kind === "viewport")?.path ??
|
||||
capture.screenshots[0]?.path ?? null;
|
||||
}
|
||||
|
||||
function dataUrl(bytes: Uint8Array): string {
|
||||
let binary = "";
|
||||
for (const byte of bytes) binary += String.fromCharCode(byte);
|
||||
return `data:image/png;base64,${btoa(binary)}`;
|
||||
}
|
||||
|
||||
function escapeHtml(value: string): string {
|
||||
return value.replaceAll("&", "&").replaceAll("<", "<").replaceAll(">", ">").replaceAll(
|
||||
'"',
|
||||
""",
|
||||
);
|
||||
}
|
||||
|
||||
export async function compare(options: CompareOptions): Promise<string> {
|
||||
const beforeManifestPath = resolve(options.before);
|
||||
const afterManifestPath = resolve(options.after);
|
||||
const [before, after] = await Promise.all([
|
||||
readManifest(beforeManifestPath),
|
||||
readManifest(afterManifestPath),
|
||||
]);
|
||||
const outputDirectory = resolve(options.outputDirectory);
|
||||
await ensurePrivateDirectory(outputDirectory);
|
||||
const afterByKey = new Map(after.captures.map((capture) => [key(capture), capture]));
|
||||
const pairs: Pair[] = [];
|
||||
const unmatchedBefore: string[] = [];
|
||||
for (const earlier of before.captures) {
|
||||
const later = afterByKey.get(key(earlier));
|
||||
const earlierScreenshot = viewportScreenshot(earlier);
|
||||
const laterScreenshot = later ? viewportScreenshot(later) : null;
|
||||
if (!later || !earlierScreenshot || !laterScreenshot) {
|
||||
unmatchedBefore.push(key(earlier));
|
||||
continue;
|
||||
}
|
||||
afterByKey.delete(key(earlier));
|
||||
const beforePath = resolve(dirname(beforeManifestPath), earlierScreenshot);
|
||||
const afterPath = resolve(dirname(afterManifestPath), laterScreenshot);
|
||||
const [beforeImage, afterImage] = [
|
||||
PNG.sync.read(Buffer.from(Deno.readFileSync(beforePath))),
|
||||
PNG.sync.read(Buffer.from(Deno.readFileSync(afterPath))),
|
||||
];
|
||||
const dimensionMismatch = beforeImage.width !== afterImage.width ||
|
||||
beforeImage.height !== afterImage.height;
|
||||
const width = Math.max(beforeImage.width, afterImage.width);
|
||||
const height = Math.max(beforeImage.height, afterImage.height);
|
||||
const diff = new PNG({ width, height, fill: true });
|
||||
let changedPixels = width * height;
|
||||
if (!dimensionMismatch) {
|
||||
changedPixels = pixelmatch(beforeImage.data, afterImage.data, diff.data, width, height, {
|
||||
threshold: options.threshold ?? 0.1,
|
||||
includeAA: false,
|
||||
});
|
||||
}
|
||||
const diffPath = join(outputDirectory, "diffs", `${key(earlier).replaceAll("/", "--")}.png`);
|
||||
await ensurePrivateDirectory(dirname(diffPath));
|
||||
await Deno.writeFile(diffPath, PNG.sync.write(diff), { mode: 0o600 });
|
||||
await makePrivate(diffPath);
|
||||
pairs.push({
|
||||
key: key(earlier),
|
||||
before: earlier,
|
||||
after: later,
|
||||
beforePath,
|
||||
afterPath,
|
||||
diffPath,
|
||||
changedPixels,
|
||||
totalPixels: width * height,
|
||||
dimensionMismatch,
|
||||
});
|
||||
}
|
||||
const cells: string[] = [];
|
||||
for (const pair of pairs) {
|
||||
const [beforeBytes, afterBytes, diffBytes] = await Promise.all([
|
||||
Deno.readFile(pair.beforePath),
|
||||
Deno.readFile(pair.afterPath),
|
||||
Deno.readFile(pair.diffPath),
|
||||
]);
|
||||
cells.push(
|
||||
`<section><h2>${escapeHtml(pair.key)}</h2><p>${
|
||||
pair.dimensionMismatch
|
||||
? "dimension mismatch"
|
||||
: `${pair.changedPixels} / ${pair.totalPixels} pixels changed`
|
||||
}</p><div class="row"><figure><img src="${
|
||||
dataUrl(beforeBytes)
|
||||
}"><figcaption>before</figcaption></figure><figure><img src="${
|
||||
dataUrl(afterBytes)
|
||||
}"><figcaption>after</figcaption></figure><figure><img src="${
|
||||
dataUrl(diffBytes)
|
||||
}"><figcaption>diff</figcaption></figure></div></section>`,
|
||||
);
|
||||
}
|
||||
const html =
|
||||
`<!doctype html><meta charset="utf-8"><title>Web UX comparison</title><style>body{margin:0;padding:20px;background:#e8e8e8;color:#111;font:14px system-ui}section{background:white;border:1px solid #aaa;margin:0 0 24px;padding:12px}.row{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:12px}figure{margin:0}img{width:100%;height:auto;border:1px solid #ddd}figcaption{text-align:center;padding:6px}h2{font-size:16px;margin:0}p{color:#555}</style>${
|
||||
cells.join("")
|
||||
}`;
|
||||
const htmlPath = join(outputDirectory, "comparison.html");
|
||||
const pngPath = join(outputDirectory, "comparison.png");
|
||||
await Deno.writeTextFile(htmlPath, html, { mode: 0o600 });
|
||||
const browser = await chromium.launch({ headless: true });
|
||||
try {
|
||||
const page = await browser.newPage({ viewport: { width: 1800, height: 1000 } });
|
||||
await page.setContent(html, { waitUntil: "load" });
|
||||
await page.screenshot({ path: pngPath, fullPage: true, animations: "disabled" });
|
||||
await makePrivate(pngPath);
|
||||
} finally {
|
||||
await browser.close();
|
||||
}
|
||||
const report = {
|
||||
schemaVersion: 1,
|
||||
before: beforeManifestPath,
|
||||
after: afterManifestPath,
|
||||
createdAt: new Date().toISOString(),
|
||||
threshold: options.threshold ?? 0.1,
|
||||
pairs: await Promise.all(pairs.map(async (pair) => ({
|
||||
key: pair.key,
|
||||
changedPixels: pair.changedPixels,
|
||||
totalPixels: pair.totalPixels,
|
||||
changedRatio: pair.totalPixels === 0 ? 0 : pair.changedPixels / pair.totalPixels,
|
||||
dimensionMismatch: pair.dimensionMismatch,
|
||||
diff: relative(outputDirectory, pair.diffPath),
|
||||
diffSha256: await sha256File(pair.diffPath),
|
||||
}))),
|
||||
unmatchedBefore,
|
||||
unmatchedAfter: [...afterByKey.keys()],
|
||||
contactSheet: { html: basename(htmlPath), png: basename(pngPath) },
|
||||
};
|
||||
const reportPath = join(outputDirectory, "comparison.json");
|
||||
await Deno.writeTextFile(reportPath, `${JSON.stringify(report, null, 2)}\n`, { mode: 0o600 });
|
||||
return reportPath;
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
import { dirname, resolve } from "@std/path";
|
||||
import { chromium } from "playwright";
|
||||
import { ensurePrivateDirectory, writePrivateJson } from "./artifacts.ts";
|
||||
import {
|
||||
interpolateEnvironment,
|
||||
loadScenario,
|
||||
resolveScenarioPath,
|
||||
validateBaseUrl,
|
||||
} from "./scenario.ts";
|
||||
|
||||
export type AuthOptions = {
|
||||
scenarioPath: string;
|
||||
personaId: string;
|
||||
baseUrl?: string;
|
||||
importState?: string;
|
||||
timeoutMs?: number;
|
||||
headless?: boolean;
|
||||
};
|
||||
|
||||
function validateStorageState(value: unknown): { cookies: unknown[]; origins: unknown[] } {
|
||||
if (typeof value !== "object" || value === null || Array.isArray(value)) {
|
||||
throw new Error("storage state must be an object");
|
||||
}
|
||||
const source = value as Record<string, unknown>;
|
||||
if (!Array.isArray(source.cookies) || !Array.isArray(source.origins)) {
|
||||
throw new Error("storage state must contain cookies and origins arrays");
|
||||
}
|
||||
return { cookies: source.cookies, origins: source.origins };
|
||||
}
|
||||
|
||||
export async function authenticate(options: AuthOptions): Promise<string> {
|
||||
const scenarioPath = resolve(options.scenarioPath);
|
||||
const scenario = await loadScenario(scenarioPath);
|
||||
const persona = scenario.personas.find((candidate) => candidate.id === options.personaId);
|
||||
if (!persona) throw new Error(`unknown persona: ${options.personaId}`);
|
||||
if (persona.auth.kind !== "storage-state") {
|
||||
throw new Error(`persona ${persona.id} is anonymous and has no auth state`);
|
||||
}
|
||||
const outputPath = resolveScenarioPath(scenarioPath, persona.auth.path);
|
||||
await ensurePrivateDirectory(dirname(outputPath));
|
||||
if (options.importState) {
|
||||
const imported = validateStorageState(
|
||||
JSON.parse(await Deno.readTextFile(resolve(options.importState))),
|
||||
);
|
||||
await writePrivateJson(outputPath, imported);
|
||||
return outputPath;
|
||||
}
|
||||
if (!persona.login) {
|
||||
throw new Error(`persona ${persona.id} needs login configuration or --import-state`);
|
||||
}
|
||||
const baseUrl = validateBaseUrl(
|
||||
interpolateEnvironment(
|
||||
options.baseUrl ?? Deno.env.get("WEB_UX_BASE_URL") ?? scenario.baseUrl ?? "",
|
||||
),
|
||||
);
|
||||
const browser = await chromium.launch({ headless: options.headless ?? false });
|
||||
try {
|
||||
const context = await browser.newContext();
|
||||
const page = await context.newPage();
|
||||
const loginUrl = new URL(interpolateEnvironment(persona.login.path ?? "/"), `${baseUrl}/`)
|
||||
.toString();
|
||||
await page.goto(loginUrl, { waitUntil: "domcontentloaded", timeout: 30_000 });
|
||||
const success = new RegExp(interpolateEnvironment(persona.login.successUrl));
|
||||
if (!success.test(page.url())) {
|
||||
await page.waitForURL((url) => success.test(url.toString()), {
|
||||
timeout: options.timeoutMs ?? 300_000,
|
||||
});
|
||||
}
|
||||
await context.storageState({ path: outputPath });
|
||||
if (Deno.build.os !== "windows") await Deno.chmod(outputPath, 0o600);
|
||||
return outputPath;
|
||||
} finally {
|
||||
await browser.close();
|
||||
}
|
||||
}
|
||||
|
||||
export type CleanupOptions = {
|
||||
outputDirectory: string;
|
||||
keep: number;
|
||||
olderThanDays?: number;
|
||||
dryRun?: boolean;
|
||||
};
|
||||
|
||||
export async function cleanup(options: CleanupOptions): Promise<string[]> {
|
||||
const outputDirectory = resolve(options.outputDirectory);
|
||||
const candidates: { path: string; modified: number }[] = [];
|
||||
try {
|
||||
for await (const entry of Deno.readDir(outputDirectory)) {
|
||||
if (!entry.isDirectory) continue;
|
||||
const path = resolve(outputDirectory, entry.name);
|
||||
try {
|
||||
await Deno.stat(resolve(path, "review-context.json"));
|
||||
const stat = await Deno.stat(path);
|
||||
candidates.push({ path, modified: stat.mtime?.getTime() ?? 0 });
|
||||
} catch {
|
||||
// Only complete review bundle directories are owned by cleanup.
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof Deno.errors.NotFound) return [];
|
||||
throw error;
|
||||
}
|
||||
candidates.sort((left, right) => right.modified - left.modified);
|
||||
const cutoff = options.olderThanDays === undefined
|
||||
? Number.POSITIVE_INFINITY
|
||||
: Date.now() - options.olderThanDays * 24 * 60 * 60 * 1000;
|
||||
const removed: string[] = [];
|
||||
for (const [index, candidate] of candidates.entries()) {
|
||||
if (index < options.keep || candidate.modified > cutoff) continue;
|
||||
removed.push(candidate.path);
|
||||
if (!options.dryRun) await Deno.remove(candidate.path, { recursive: true });
|
||||
}
|
||||
return removed;
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
import { dirname, isAbsolute, resolve } from "@std/path";
|
||||
import { bounded, redactText } from "./artifacts.ts";
|
||||
import type { CaptureError, OwnedProcess } from "./types.ts";
|
||||
|
||||
export type RunningProcess = {
|
||||
id: string;
|
||||
pid: number;
|
||||
child: Deno.ChildProcess;
|
||||
output: Promise<void>;
|
||||
};
|
||||
|
||||
async function appendOutput(
|
||||
stream: ReadableStream<Uint8Array>,
|
||||
destination: string,
|
||||
secrets: string[],
|
||||
): Promise<void> {
|
||||
const file = await Deno.open(destination, {
|
||||
create: true,
|
||||
append: true,
|
||||
write: true,
|
||||
mode: 0o600,
|
||||
});
|
||||
try {
|
||||
const reader = stream.pipeThrough(new TextDecoderStream()).getReader();
|
||||
while (true) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
await file.write(new TextEncoder().encode(redactText(value, secrets)));
|
||||
}
|
||||
} finally {
|
||||
file.close();
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForReady(url: string, timeoutMs: number): Promise<void> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastError = "not attempted";
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
const response = await fetch(url, { redirect: "manual", signal: AbortSignal.timeout(2_000) });
|
||||
if (response.status < 500) return;
|
||||
lastError = `HTTP ${response.status}`;
|
||||
} catch (error) {
|
||||
lastError = error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 200));
|
||||
}
|
||||
throw new Error(`process readiness timed out for ${url}: ${bounded(lastError, 200)}`);
|
||||
}
|
||||
|
||||
export async function startOwnedProcesses(
|
||||
specifications: OwnedProcess[],
|
||||
scenarioPath: string,
|
||||
logsDirectory: string,
|
||||
secrets: string[],
|
||||
): Promise<RunningProcess[]> {
|
||||
const running: RunningProcess[] = [];
|
||||
await Deno.mkdir(logsDirectory, { recursive: true });
|
||||
try {
|
||||
for (const specification of specifications) {
|
||||
const cwd = specification.cwd === undefined
|
||||
? dirname(resolve(scenarioPath))
|
||||
: isAbsolute(specification.cwd)
|
||||
? specification.cwd
|
||||
: resolve(dirname(scenarioPath), specification.cwd);
|
||||
const child = new Deno.Command(specification.command, {
|
||||
args: specification.args ?? [],
|
||||
cwd,
|
||||
env: specification.env,
|
||||
stdin: "null",
|
||||
stdout: "piped",
|
||||
stderr: "piped",
|
||||
}).spawn();
|
||||
const stdout = appendOutput(
|
||||
child.stdout,
|
||||
`${logsDirectory}/${specification.id}.stdout.log`,
|
||||
secrets,
|
||||
);
|
||||
const stderr = appendOutput(
|
||||
child.stderr,
|
||||
`${logsDirectory}/${specification.id}.stderr.log`,
|
||||
secrets,
|
||||
);
|
||||
const process = {
|
||||
id: specification.id,
|
||||
pid: child.pid,
|
||||
child,
|
||||
output: Promise.all([stdout, stderr]).then(() => undefined),
|
||||
};
|
||||
running.push(process);
|
||||
if (specification.readyUrl) {
|
||||
await Promise.race([
|
||||
waitForReady(specification.readyUrl, specification.readyTimeoutMs ?? 30_000),
|
||||
child.status.then((status) => {
|
||||
throw new Error(
|
||||
`owned process ${specification.id} exited before readiness: ${status.code}`,
|
||||
);
|
||||
}),
|
||||
]);
|
||||
}
|
||||
}
|
||||
return running;
|
||||
} catch (error) {
|
||||
await stopOwnedProcesses(running);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function descendantPids(parentPid: number): Promise<number[]> {
|
||||
if (Deno.build.os === "windows") return [];
|
||||
try {
|
||||
const result = await new Deno.Command("ps", {
|
||||
args: ["-eo", "pid=,ppid="],
|
||||
stdout: "piped",
|
||||
stderr: "null",
|
||||
}).output();
|
||||
if (!result.success) return [];
|
||||
const rows = new TextDecoder().decode(result.stdout).trim().split("\n").map((line) =>
|
||||
line.trim().split(/\s+/).map(Number)
|
||||
);
|
||||
const descendants: number[] = [];
|
||||
const queue = [parentPid];
|
||||
while (queue.length > 0) {
|
||||
const parent = queue.shift()!;
|
||||
for (const [pid, ppid] of rows) {
|
||||
if (ppid === parent && !descendants.includes(pid)) {
|
||||
descendants.push(pid);
|
||||
queue.push(pid);
|
||||
}
|
||||
}
|
||||
}
|
||||
return descendants.reverse();
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function tryKill(pid: number, signal: Deno.Signal): void {
|
||||
try {
|
||||
Deno.kill(pid, signal);
|
||||
} catch (error) {
|
||||
if (!(error instanceof Deno.errors.NotFound)) throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export async function stopOwnedProcesses(processes: RunningProcess[]): Promise<CaptureError[]> {
|
||||
const diagnostics: CaptureError[] = [];
|
||||
for (const process of [...processes].reverse()) {
|
||||
try {
|
||||
for (const pid of await descendantPids(process.pid)) tryKill(pid, "SIGTERM");
|
||||
tryKill(process.pid, "SIGTERM");
|
||||
let timer: number | undefined;
|
||||
const exited = await Promise.race([
|
||||
process.child.status.then(() => true),
|
||||
new Promise<boolean>((resolve) => {
|
||||
timer = setTimeout(() => resolve(false), 3_000);
|
||||
}),
|
||||
]).finally(() => clearTimeout(timer));
|
||||
if (!exited) {
|
||||
for (const pid of await descendantPids(process.pid)) tryKill(pid, "SIGKILL");
|
||||
tryKill(process.pid, "SIGKILL");
|
||||
await process.child.status;
|
||||
}
|
||||
await process.output;
|
||||
} catch (error) {
|
||||
diagnostics.push({
|
||||
kind: "tool",
|
||||
message: `failed to clean process ${process.id}: ${
|
||||
bounded(error instanceof Error ? error.message : String(error), 500)
|
||||
}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return diagnostics;
|
||||
}
|
||||
@@ -0,0 +1,270 @@
|
||||
import { isAbsolute, join, resolve } from "@std/path";
|
||||
import type {
|
||||
CapturePoint,
|
||||
Persona,
|
||||
ReadyCondition,
|
||||
RouteScenario,
|
||||
Scenario,
|
||||
Viewport,
|
||||
} from "./types.ts";
|
||||
|
||||
function record(value: unknown, at: string): Record<string, unknown> {
|
||||
if (typeof value !== "object" || value === null || Array.isArray(value)) {
|
||||
throw new Error(`${at} must be an object`);
|
||||
}
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function text(value: unknown, at: string): string {
|
||||
if (typeof value !== "string" || value.trim() === "") throw new Error(`${at} must be text`);
|
||||
return value;
|
||||
}
|
||||
|
||||
function positiveInteger(value: unknown, at: string): number {
|
||||
if (!Number.isInteger(value) || (value as number) <= 0) {
|
||||
throw new Error(`${at} must be a positive integer`);
|
||||
}
|
||||
return value as number;
|
||||
}
|
||||
|
||||
function identifier(value: unknown, at: string): string {
|
||||
const result = text(value, at);
|
||||
if (!/^[a-z0-9][a-z0-9-]*$/.test(result)) {
|
||||
throw new Error(`${at} must contain lowercase ASCII letters, digits, or hyphens`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function stringArray(value: unknown, at: string): string[] {
|
||||
if (value === undefined) return [];
|
||||
if (!Array.isArray(value)) throw new Error(`${at} must be an array`);
|
||||
return value.map((item, index) => text(item, `${at}[${index}]`));
|
||||
}
|
||||
|
||||
function parseReady(value: unknown, at: string): ReadyCondition {
|
||||
const source = record(value, at);
|
||||
const kind = text(source.kind, `${at}.kind`);
|
||||
const timeoutMs = source.timeoutMs === undefined
|
||||
? undefined
|
||||
: positiveInteger(source.timeoutMs, `${at}.timeoutMs`);
|
||||
if (kind === "selector") {
|
||||
return { kind, selector: text(source.selector, `${at}.selector`), timeoutMs };
|
||||
}
|
||||
if (kind === "response") {
|
||||
const status = source.status === undefined
|
||||
? undefined
|
||||
: positiveInteger(source.status, `${at}.status`);
|
||||
return { kind, urlPattern: text(source.urlPattern, `${at}.urlPattern`), status, timeoutMs };
|
||||
}
|
||||
if (kind === "network-idle") return { kind, timeoutMs };
|
||||
throw new Error(`${at}.kind is unsupported: ${kind}`);
|
||||
}
|
||||
|
||||
function parseCapturePoint(value: unknown, at: string): CapturePoint {
|
||||
const source = record(value, at);
|
||||
const result: CapturePoint = {
|
||||
id: identifier(source.id, `${at}.id`),
|
||||
label: text(source.label, `${at}.label`),
|
||||
fullPage: source.fullPage === undefined ? false : Boolean(source.fullPage),
|
||||
};
|
||||
if (source.ready !== undefined) result.ready = parseReady(source.ready, `${at}.ready`);
|
||||
if (source.interaction !== undefined) {
|
||||
if (!Array.isArray(source.interaction)) throw new Error(`${at}.interaction must be an array`);
|
||||
result.interaction = source.interaction.map((raw, index) => {
|
||||
const action = record(raw, `${at}.interaction[${index}]`);
|
||||
const name = text(action.action, `${at}.interaction[${index}].action`);
|
||||
if (name === "wait") {
|
||||
return {
|
||||
action: name,
|
||||
ready: parseReady(action.ready, `${at}.interaction[${index}].ready`),
|
||||
};
|
||||
}
|
||||
const selector = text(action.selector, `${at}.interaction[${index}].selector`);
|
||||
const timeoutMs = action.timeoutMs === undefined
|
||||
? undefined
|
||||
: positiveInteger(action.timeoutMs, `${at}.interaction[${index}].timeoutMs`);
|
||||
if (name === "click") return { action: name, selector, timeoutMs };
|
||||
if (name === "fill") {
|
||||
return {
|
||||
action: name,
|
||||
selector,
|
||||
value: text(action.value, `${at}.interaction[${index}].value`),
|
||||
timeoutMs,
|
||||
};
|
||||
}
|
||||
if (name === "press") {
|
||||
return {
|
||||
action: name,
|
||||
selector,
|
||||
key: text(action.key, `${at}.interaction[${index}].key`),
|
||||
timeoutMs,
|
||||
};
|
||||
}
|
||||
throw new Error(`${at}.interaction[${index}].action is unsupported: ${name}`);
|
||||
});
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function parsePersona(value: unknown, at: string): Persona {
|
||||
const source = record(value, at);
|
||||
const auth = record(source.auth, `${at}.auth`);
|
||||
const kind = text(auth.kind, `${at}.auth.kind`);
|
||||
const persona: Persona = {
|
||||
id: identifier(source.id, `${at}.id`),
|
||||
label: text(source.label, `${at}.label`),
|
||||
auth: kind === "anonymous"
|
||||
? { kind }
|
||||
: kind === "storage-state"
|
||||
? { kind, path: text(auth.path, `${at}.auth.path`) }
|
||||
: (() => {
|
||||
throw new Error(`${at}.auth.kind is unsupported: ${kind}`);
|
||||
})(),
|
||||
};
|
||||
if (source.login !== undefined) {
|
||||
const login = record(source.login, `${at}.login`);
|
||||
persona.login = {
|
||||
path: login.path === undefined ? "/" : text(login.path, `${at}.login.path`),
|
||||
successUrl: text(login.successUrl, `${at}.login.successUrl`),
|
||||
};
|
||||
}
|
||||
return persona;
|
||||
}
|
||||
|
||||
function parseViewport(value: unknown, at: string): Viewport {
|
||||
const source = record(value, at);
|
||||
return {
|
||||
width: positiveInteger(source.width, `${at}.width`),
|
||||
height: positiveInteger(source.height, `${at}.height`),
|
||||
label: source.label === undefined ? undefined : identifier(source.label, `${at}.label`),
|
||||
deviceScaleFactor: source.deviceScaleFactor === undefined
|
||||
? 1
|
||||
: positiveInteger(source.deviceScaleFactor, `${at}.deviceScaleFactor`),
|
||||
};
|
||||
}
|
||||
|
||||
function parseRoute(value: unknown, at: string): RouteScenario {
|
||||
const source = record(value, at);
|
||||
if (!Array.isArray(source.capturePoints) || source.capturePoints.length === 0) {
|
||||
throw new Error(`${at}.capturePoints must have at least one item`);
|
||||
}
|
||||
return {
|
||||
id: identifier(source.id, `${at}.id`),
|
||||
label: text(source.label, `${at}.label`),
|
||||
path: text(source.path, `${at}.path`),
|
||||
goal: text(source.goal, `${at}.goal`),
|
||||
dataState: text(source.dataState, `${at}.dataState`),
|
||||
ready: parseReady(source.ready, `${at}.ready`),
|
||||
capturePoints: source.capturePoints.map((point, index) =>
|
||||
parseCapturePoint(point, `${at}.capturePoints[${index}]`)
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
function uniqueIds(values: { id: string }[], at: string): void {
|
||||
const seen = new Set<string>();
|
||||
for (const value of values) {
|
||||
if (seen.has(value.id)) throw new Error(`${at} contains duplicate id: ${value.id}`);
|
||||
seen.add(value.id);
|
||||
}
|
||||
}
|
||||
|
||||
export function interpolateEnvironment(value: string, environment = Deno.env.toObject()): string {
|
||||
return value.replaceAll(/\$\{([A-Z][A-Z0-9_]*)\}/g, (_match, name: string) => {
|
||||
const replacement = environment[name];
|
||||
if (replacement === undefined) {
|
||||
throw new Error(`required environment variable is missing: ${name}`);
|
||||
}
|
||||
return replacement;
|
||||
});
|
||||
}
|
||||
|
||||
export function validateBaseUrl(value: string): string {
|
||||
const url = new URL(value);
|
||||
if (url.protocol !== "http:" && url.protocol !== "https:") {
|
||||
throw new Error("base URL must use http or https");
|
||||
}
|
||||
if (url.username || url.password) throw new Error("base URL must not contain credentials");
|
||||
url.pathname = url.pathname.replace(/\/$/, "");
|
||||
return url.toString().replace(/\/$/, "");
|
||||
}
|
||||
|
||||
export function resolveScenarioPath(sourcePath: string, value: string): string {
|
||||
const expanded = interpolateEnvironment(value);
|
||||
return isAbsolute(expanded) ? expanded : resolve(join(sourcePath, "..", expanded));
|
||||
}
|
||||
|
||||
export async function loadScenario(sourcePath: string): Promise<Scenario> {
|
||||
const parsed = JSON.parse(await Deno.readTextFile(sourcePath));
|
||||
const source = record(parsed, "scenario");
|
||||
if (source.schemaVersion !== 1) throw new Error("scenario.schemaVersion must equal 1");
|
||||
if (!Array.isArray(source.personas) || source.personas.length === 0) {
|
||||
throw new Error("scenario.personas must have at least one item");
|
||||
}
|
||||
if (!Array.isArray(source.viewports) || source.viewports.length === 0) {
|
||||
throw new Error("scenario.viewports must have at least one item");
|
||||
}
|
||||
if (!Array.isArray(source.routes) || source.routes.length === 0) {
|
||||
throw new Error("scenario.routes must have at least one item");
|
||||
}
|
||||
const personas = source.personas.map((value, index) =>
|
||||
parsePersona(value, `scenario.personas[${index}]`)
|
||||
);
|
||||
const routes = source.routes.map((value, index) =>
|
||||
parseRoute(value, `scenario.routes[${index}]`)
|
||||
);
|
||||
const scenario: Scenario = {
|
||||
schemaVersion: 1,
|
||||
id: identifier(source.id, "scenario.id"),
|
||||
title: text(source.title, "scenario.title"),
|
||||
baseUrl: source.baseUrl === undefined ? undefined : text(source.baseUrl, "scenario.baseUrl"),
|
||||
locale: source.locale === undefined ? "en-US" : text(source.locale, "scenario.locale"),
|
||||
timezone: source.timezone === undefined ? "UTC" : text(source.timezone, "scenario.timezone"),
|
||||
colorScheme: source.colorScheme === "dark" ? "dark" : "light",
|
||||
reducedMotion: source.reducedMotion === "no-preference" ? "no-preference" : "reduce",
|
||||
redact: source.redact === undefined ? undefined : (() => {
|
||||
const redact = record(source.redact, "scenario.redact");
|
||||
return {
|
||||
selectors: stringArray(redact.selectors, "scenario.redact.selectors"),
|
||||
text: stringArray(redact.text, "scenario.redact.text").map((value) =>
|
||||
interpolateEnvironment(value)
|
||||
),
|
||||
};
|
||||
})(),
|
||||
personas,
|
||||
viewports: source.viewports.map((value, index) =>
|
||||
parseViewport(value, `scenario.viewports[${index}]`)
|
||||
),
|
||||
routes,
|
||||
};
|
||||
if (source.processes !== undefined) {
|
||||
if (!Array.isArray(source.processes)) throw new Error("scenario.processes must be an array");
|
||||
scenario.processes = source.processes.map((value, index) => {
|
||||
const at = `scenario.processes[${index}]`;
|
||||
const process = record(value, at);
|
||||
const env = process.env === undefined ? undefined : record(process.env, `${at}.env`);
|
||||
return {
|
||||
id: identifier(process.id, `${at}.id`),
|
||||
command: text(process.command, `${at}.command`),
|
||||
args: stringArray(process.args, `${at}.args`),
|
||||
cwd: process.cwd === undefined ? undefined : text(process.cwd, `${at}.cwd`),
|
||||
env: env === undefined ? undefined : Object.fromEntries(
|
||||
Object.entries(env).map((
|
||||
[key, raw],
|
||||
) => [key, interpolateEnvironment(text(raw, `${at}.env.${key}`))]),
|
||||
),
|
||||
readyUrl: process.readyUrl === undefined ? undefined : validateBaseUrl(
|
||||
interpolateEnvironment(text(process.readyUrl, `${at}.readyUrl`)),
|
||||
),
|
||||
readyTimeoutMs: process.readyTimeoutMs === undefined
|
||||
? undefined
|
||||
: positiveInteger(process.readyTimeoutMs, `${at}.readyTimeoutMs`),
|
||||
};
|
||||
});
|
||||
uniqueIds(scenario.processes, "scenario.processes");
|
||||
}
|
||||
uniqueIds(personas, "scenario.personas");
|
||||
uniqueIds(routes, "scenario.routes");
|
||||
for (const route of routes) uniqueIds(route.capturePoints, `route ${route.id} capturePoints`);
|
||||
return scenario;
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
export type Viewport = {
|
||||
width: number;
|
||||
height: number;
|
||||
label?: string;
|
||||
deviceScaleFactor?: number;
|
||||
};
|
||||
|
||||
export type Persona = {
|
||||
id: string;
|
||||
label: string;
|
||||
auth: { kind: "anonymous" } | { kind: "storage-state"; path: string };
|
||||
login?: {
|
||||
path?: string;
|
||||
successUrl: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type ReadyCondition =
|
||||
| { kind: "selector"; selector: string; timeoutMs?: number }
|
||||
| { kind: "response"; urlPattern: string; status?: number; timeoutMs?: number }
|
||||
| { kind: "network-idle"; timeoutMs?: number };
|
||||
|
||||
export type Interaction =
|
||||
| { action: "click"; selector: string; timeoutMs?: number }
|
||||
| { action: "fill"; selector: string; value: string; timeoutMs?: number }
|
||||
| { action: "press"; selector: string; key: string; timeoutMs?: number }
|
||||
| { action: "wait"; ready: ReadyCondition };
|
||||
|
||||
export type CapturePoint = {
|
||||
id: string;
|
||||
label: string;
|
||||
interaction?: Interaction[];
|
||||
ready?: ReadyCondition;
|
||||
fullPage?: boolean;
|
||||
};
|
||||
|
||||
export type RouteScenario = {
|
||||
id: string;
|
||||
label: string;
|
||||
path: string;
|
||||
goal: string;
|
||||
dataState: string;
|
||||
ready: ReadyCondition;
|
||||
capturePoints: CapturePoint[];
|
||||
};
|
||||
|
||||
export type OwnedProcess = {
|
||||
id: string;
|
||||
command: string;
|
||||
args?: string[];
|
||||
cwd?: string;
|
||||
env?: Record<string, string>;
|
||||
readyUrl?: string;
|
||||
readyTimeoutMs?: number;
|
||||
};
|
||||
|
||||
export type Scenario = {
|
||||
schemaVersion: 1;
|
||||
id: string;
|
||||
title: string;
|
||||
baseUrl?: string;
|
||||
locale?: string;
|
||||
timezone?: string;
|
||||
colorScheme?: "light" | "dark";
|
||||
reducedMotion?: "reduce" | "no-preference";
|
||||
redact?: {
|
||||
selectors?: string[];
|
||||
text?: string[];
|
||||
};
|
||||
personas: Persona[];
|
||||
viewports: Viewport[];
|
||||
routes: RouteScenario[];
|
||||
processes?: OwnedProcess[];
|
||||
};
|
||||
|
||||
export type CaptureError = {
|
||||
kind: "console" | "page" | "request" | "document" | "tool";
|
||||
message: string;
|
||||
url?: string;
|
||||
status?: number;
|
||||
};
|
||||
|
||||
export type ScreenshotEvidence = {
|
||||
kind: "viewport" | "full-page";
|
||||
path: string;
|
||||
sha256: string;
|
||||
};
|
||||
|
||||
export type CaptureEvidence = {
|
||||
persona: { id: string; label: string };
|
||||
route: { id: string; path: string; goal: string; dataState: string };
|
||||
viewport: Viewport;
|
||||
theme: string;
|
||||
capturePoint: { id: string; label: string };
|
||||
document: { url: string; status: number | null };
|
||||
screenshots: ScreenshotEvidence[];
|
||||
snapshotPath: string | null;
|
||||
errors: CaptureError[];
|
||||
startedAt: string;
|
||||
finishedAt: string;
|
||||
};
|
||||
|
||||
export type ReviewContext = {
|
||||
schemaVersion: 1;
|
||||
runId: string;
|
||||
scenario: { id: string; title: string; sourcePath: string };
|
||||
source: { revision: string | null; dirty: boolean | null };
|
||||
baseUrl: string;
|
||||
browser: { name: "chromium"; version: string };
|
||||
createdAt: string;
|
||||
status: "completed" | "completed-with-errors" | "failed";
|
||||
filters: { personas: string[]; routes: string[]; viewports: string[] };
|
||||
captures: CaptureEvidence[];
|
||||
contactSheet: { html: string | null; png: string | null };
|
||||
diagnostics: CaptureError[];
|
||||
};
|
||||
@@ -0,0 +1,26 @@
|
||||
import { assertEquals, assertThrows } from "@std/assert";
|
||||
import { parseArguments } from "../cli.ts";
|
||||
import { isVisibleUiErrorText } from "../src/capture.ts";
|
||||
|
||||
Deno.test("CLI parses bounded capture filters", () => {
|
||||
const parsed = parseArguments([
|
||||
"capture",
|
||||
"--scenario",
|
||||
"scenario.json",
|
||||
"--personas",
|
||||
"owner,non-owner",
|
||||
"--headed",
|
||||
]);
|
||||
assertEquals(parsed.command, "capture");
|
||||
assertEquals(parsed.values.get("personas"), ["owner,non-owner"]);
|
||||
assertEquals(parsed.flags.has("headed"), true);
|
||||
});
|
||||
|
||||
Deno.test("visible UI error classification ignores ordinary status text", () => {
|
||||
assertEquals(isVisibleUiErrorText("Refresh failed (401 Unauthorized)"), true);
|
||||
assertEquals(isVisibleUiErrorText("Workspace list loaded"), false);
|
||||
});
|
||||
|
||||
Deno.test("CLI rejects positional and missing option values", () => {
|
||||
assertThrows(() => parseArguments(["capture", "scenario.json"]), Error, "unexpected argument");
|
||||
});
|
||||
@@ -0,0 +1,75 @@
|
||||
import { assertEquals, assertRejects, assertStringIncludes } from "@std/assert";
|
||||
import { join } from "@std/path";
|
||||
import {
|
||||
assertBundleIsSecretFree,
|
||||
redactText,
|
||||
safeUrl,
|
||||
writePrivateJson,
|
||||
} from "../src/artifacts.ts";
|
||||
import { startOwnedProcesses, stopOwnedProcesses } from "../src/processes.ts";
|
||||
|
||||
Deno.test("redaction removes common credentials and query values", () => {
|
||||
const redacted = redactText(
|
||||
"Authorization: Bearer abc.def cookie=session-value token=secret-value",
|
||||
["abc.def"],
|
||||
);
|
||||
assertStringIncludes(redacted, "[REDACTED]");
|
||||
assertEquals(redacted.includes("abc.def"), false);
|
||||
assertEquals(redacted.includes("session-value"), false);
|
||||
assertEquals(
|
||||
safeUrl("https://user:pass@example.test/path?token=secret#fragment"),
|
||||
"https://example.test/path?token=%5BREDACTED%5D",
|
||||
);
|
||||
assertRejects(
|
||||
async () => assertBundleIsSecretFree('{"authorization":"Bearer abc"}'),
|
||||
Error,
|
||||
"forbidden secret marker",
|
||||
);
|
||||
});
|
||||
|
||||
Deno.test("private JSON state uses owner-only permissions", async () => {
|
||||
const directory = await Deno.makeTempDir();
|
||||
try {
|
||||
const path = join(directory, "state", "owner.json");
|
||||
await writePrivateJson(path, { cookies: [], origins: [] });
|
||||
assertEquals(JSON.parse(await Deno.readTextFile(path)), { cookies: [], origins: [] });
|
||||
if (Deno.build.os !== "windows") assertEquals((await Deno.stat(path)).mode! & 0o777, 0o600);
|
||||
} finally {
|
||||
await Deno.remove(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
Deno.test("owned process is terminated and its logs are redacted", async () => {
|
||||
const directory = await Deno.makeTempDir();
|
||||
const scenario = join(directory, "scenario.json");
|
||||
await Deno.writeTextFile(scenario, "{}");
|
||||
try {
|
||||
const processes = await startOwnedProcesses(
|
||||
[{
|
||||
id: "fixture",
|
||||
command: Deno.execPath(),
|
||||
args: ["eval", 'console.log("authorization: secret-value"); setInterval(() => {}, 1000)'],
|
||||
}],
|
||||
scenario,
|
||||
join(directory, "logs"),
|
||||
["secret-value"],
|
||||
);
|
||||
assertEquals(processes.length, 1);
|
||||
const logPath = join(directory, "logs", "fixture.stdout.log");
|
||||
for (let attempt = 0; attempt < 20; attempt++) {
|
||||
try {
|
||||
if ((await Deno.readTextFile(logPath)).length > 0) break;
|
||||
} catch {
|
||||
// The output pump creates the file asynchronously.
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
}
|
||||
const diagnostics = await stopOwnedProcesses(processes);
|
||||
assertEquals(diagnostics, []);
|
||||
const log = await Deno.readTextFile(logPath);
|
||||
assertEquals(log.includes("secret-value"), false);
|
||||
assertStringIncludes(log, "[REDACTED]");
|
||||
} finally {
|
||||
await Deno.remove(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,98 @@
|
||||
import { assertEquals, assertRejects, assertThrows } from "@std/assert";
|
||||
import { join } from "@std/path";
|
||||
import { cleanup } from "../src/lifecycle.ts";
|
||||
import { interpolateEnvironment, loadScenario, validateBaseUrl } from "../src/scenario.ts";
|
||||
|
||||
function minimalScenario(extra = ""): string {
|
||||
return `{
|
||||
"schemaVersion": 1,
|
||||
"id": "test-screen",
|
||||
"title": "Test screen",
|
||||
"baseUrl": "http://127.0.0.1:5173",
|
||||
"personas": [{"id":"anonymous","label":"Anonymous","auth":{"kind":"anonymous"}}],
|
||||
"viewports": [{"label":"desktop","width":1000,"height":800}],
|
||||
"routes": [{
|
||||
"id":"home","label":"Home","path":"/","goal":"Inspect home",
|
||||
"dataState":"Fixture data","ready":{"kind":"selector","selector":"main"},
|
||||
"capturePoints":[{"id":"initial","label":"Initial"}]
|
||||
}]${extra}
|
||||
}`;
|
||||
}
|
||||
|
||||
Deno.test("scenario parser preserves explicit visual review context", async () => {
|
||||
const directory = await Deno.makeTempDir();
|
||||
try {
|
||||
const path = join(directory, "scenario.json");
|
||||
await Deno.writeTextFile(path, minimalScenario());
|
||||
const scenario = await loadScenario(path);
|
||||
assertEquals(scenario.personas[0].auth, { kind: "anonymous" });
|
||||
assertEquals(scenario.routes[0].goal, "Inspect home");
|
||||
assertEquals(scenario.routes[0].dataState, "Fixture data");
|
||||
assertEquals(scenario.routes[0].ready, {
|
||||
kind: "selector",
|
||||
selector: "main",
|
||||
timeoutMs: undefined,
|
||||
});
|
||||
assertEquals(scenario.reducedMotion, "reduce");
|
||||
} finally {
|
||||
await Deno.remove(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
Deno.test("scenario parser rejects duplicate persona identity", async () => {
|
||||
const directory = await Deno.makeTempDir();
|
||||
try {
|
||||
const path = join(directory, "scenario.json");
|
||||
await Deno.writeTextFile(
|
||||
path,
|
||||
minimalScenario().replace(
|
||||
'[{"id":"anonymous","label":"Anonymous","auth":{"kind":"anonymous"}}]',
|
||||
'[{"id":"same","label":"First","auth":{"kind":"anonymous"}},{"id":"same","label":"Second","auth":{"kind":"anonymous"}}]',
|
||||
),
|
||||
);
|
||||
await assertRejects(() => loadScenario(path), Error, "duplicate id: same");
|
||||
} finally {
|
||||
await Deno.remove(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
Deno.test("base URL rejects embedded credentials and non-http schemes", () => {
|
||||
assertThrows(
|
||||
() => validateBaseUrl("https://user:secret@example.test"),
|
||||
Error,
|
||||
"must not contain credentials",
|
||||
);
|
||||
assertThrows(() => validateBaseUrl("file:///tmp/index.html"), Error, "must use http or https");
|
||||
});
|
||||
|
||||
Deno.test("environment interpolation fails closed", () => {
|
||||
assertEquals(
|
||||
interpolateEnvironment("/w/${WORKSPACE_ID}", { WORKSPACE_ID: "W-test" }),
|
||||
"/w/W-test",
|
||||
);
|
||||
assertThrows(
|
||||
() => interpolateEnvironment("${MISSING}", {}),
|
||||
Error,
|
||||
"required environment variable is missing",
|
||||
);
|
||||
});
|
||||
|
||||
Deno.test("cleanup removes only complete review bundles beyond retention", async () => {
|
||||
const directory = await Deno.makeTempDir();
|
||||
try {
|
||||
for (const name of ["one", "two", "three"]) {
|
||||
const run = join(directory, name);
|
||||
await Deno.mkdir(run);
|
||||
await Deno.writeTextFile(join(run, "review-context.json"), "{}");
|
||||
await new Promise((resolve) => setTimeout(resolve, 5));
|
||||
}
|
||||
const unrelated = join(directory, "auth");
|
||||
await Deno.mkdir(unrelated);
|
||||
await Deno.writeTextFile(join(unrelated, "state.json"), "secret");
|
||||
const removed = await cleanup({ outputDirectory: directory, keep: 1 });
|
||||
assertEquals(removed.length, 2);
|
||||
assertEquals(await Deno.readTextFile(join(unrelated, "state.json")), "secret");
|
||||
} finally {
|
||||
await Deno.remove(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
Reference in New Issue
Block a user