feat: guarded Worker Restore from shared Web actions (T-701)

docs/architecture/worker-restore-observations.md

@@ -0,0 +1,70 @@
+# Guarded Worker Restore and Web observations
+
+T-701 extends the [registry-bound operation reception](workspace-worker-operations.md), not the Worker creation, Resume or Restore state machines.
+
+## Different tokens protect different things
+
+| Operation | Observed identity / expected input | Atomic validation and execution owner | Conflict / uncertain outcome | Shared client behavior / deliberate difference |
+| --- | --- | --- | --- | --- |
+| Restore | Workspace + Runtime + Worker identity; Runtime-owned `restore_observation_token`; immutable `expected_observation_token` + `request_id` | Existing Runtime per-Worker lifecycle operation lock, durable admission and lifecycle commit. Preparation and execution follow admission, not a frontend GET comparison | Stale new intent is HTTP 409 (`restore_observation_conflict`); operation results remain accepted / rejected / rolled_back / reconciliation_required. A lost reply is unknown, not definite rollback | Both entrances use the same scoped POST/parser/notice classifiers and observation refresh. Never replace a rejected intent's token and automatically retry |
+| Stop | Identity and reason, **not** a catalog revision CAS | Runtime per-Worker operation lock and existing backend Stop settlement; Workspace session finalization only after accepted | Existing flat accepted / unsupported / rejected result with diagnostics | Same encoded target paths and bounded HTTP-error messages; refresh observations instead of writing a guessed local stopped state. Stop does not gain Restore's generation guarantee |
+| Pause / Resume / Cancel / Compact | Connected execution identity and `WorkerCommandEnvelope.command_id` | Existing bound execution/controller command admission | Command acknowledgement: Accepted / StaleCommandId / Conflict / InvalidState, with authoritative state | Command IDs order commands; they are not catalog observation generations. Resume is not Restore; connections do not silently move to a replacement execution |
+| Delete / cleanup | Runtime + selected target IDs; `expected_plan_revision` **and** `expected_plan_digest` | Existing cleanup validation, per-target live/retention revalidation, removal service and catalog commits | Existing cleanup-plan conflicts, per-target status and diagnostics | Preserve plan protection for candidate sets and dependencies. Never substitute a Worker generation. Page plans invalidate at the beginning of every catalog refresh and are unavailable until fresh catalog and plan observations settle |
+| Pin / Unpin | Registry Worker identity and explicit boolean | Registry retention update and ordered projection publication | Existing retention result/errors | Preserve retention semantics and full catalog metadata; refresh shared observations and cleanup plan |
+| Submit / Notify | Authenticated source + durable submission / notification request ID | Existing controller/session durable receipt admission | Existing accepted/rejected receipts and unknown delivery diagnostics | These are not Restore requests; Restore never manufactures input, Resume or a new Worker |
+
+There is no universal Worker command token or shared frontend action lock. Workspace reception resolves a stable target and rechecks caller authorization; Runtime/controller/store owners retain operation-specific atomic boundaries.
+
+## Browser request and result
+
+Normal explicit Restore is enabled only for `availability = observed`, catalog `stopped`, and an available backend observation token. Sidebar display state and foreground `worker_state` are not eligibility evidence: sidebar uses `lifecycleState`; the page uses the full catalog's `state`. Neither UI guesses snapshot, singleton, Workdir provider or authorization eligibility.
+
+Both entrances send the generated two-field `WorkerRestoreRequest` to:
+
+```text
+POST /api/w/{workspace_id}/runtimes/{runtime_id}/workers/{worker_id}/restore
+```
+
+Each ID is independently encoded. No Ticket-assignment query, Submit, Resume or Worker-create call is used. The bounded parser validates wrapper identity, the four states, diagnostics and optional **Workspace WorkerSummary**. A Stop lifecycle result or launch summary is not a Restore response. Unknown states/fields/invalid identities cannot produce a success notice.
+
+Accepted means the existing Restore operation committed, not that the Worker is necessarily Running or a later run succeeded. A returned summary validates result identity; display is updated from fresh shared observations, not an old POST snapshot. Rejected and rolled_back display diagnostics without success. Reconciliation_required and transport/malformed-response uncertainty retain the original immutable request for an explicit **Retry Restore** action. That action sends the same request; it does not create an operation, exchange its observation token, or automatically loop. A conflict clears this local recovery intent and asks the user to review fresh observations before a new explicit operation. A GET alone does not settle Runtime's pending lifecycle operation.
+
+## Backend admission, preparation and recovery
+
+The Workspace `WorkspaceWorker` reception still resolves target identity and rechecks caller authority. Its Restore coordinator first dispatches the public tuple to Runtime without candidate preparation. A terminal owner returns its historical receipt; an admitted pending owner supplies its stored preparation before renewed singleton/Workdir/External eligibility is evaluated. Historical result summaries are read from the current Runtime and are not written back as catalog observations.
+
+For a new intent, Workspace builds read-only preparation references after the normal eligibility checks. Runtime atomically checks the observation token again under the existing per-Worker lock and persists the owner together with `pending_restore` and its existing lifecycle operation ID. Only this admitted operation can proceed to SSH resource issuance, binding/access preparation and execution. A race between the first observation check and admission therefore cannot issue credentials or launch a new Restore from the old intent.
+
+Preparation completion is attached to that same pending operation, not a replacement operation. Retained repository-access snapshots and exact request identity allow interrupted preparation to resume without exchanging candidate bindings. Runtime retries use the original owner's payload; a freshly generated preparation cannot rewrite the admitted target or expected observation. Stop and startup reconciliation continue to use the existing pending lifecycle ownership. When startup/legacy persistence has a pending lifecycle operation but no public receipt owner, an explicit request with the current observation may bind recovery to that exact operation ID, without applying candidate preparation. Once bound, a different request cannot take over; terminal replay does not resurrect a later stopped Worker.
+
+| Caller | Intent identity retained / validation boundary |
+| --- | --- |
+| Web sidebar and page | Immutable two-field request retained locally for explicit uncertain-result recovery; no shared in-flight lock |
+| Workspace alias, scoped and Worker-control HTTP | Mandatory public request; common target/auth reception and the same Runtime admission |
+| Embedded / remote Runtime providers and management HTTP | Typed Runtime request and coordination transport, including original observation and lifecycle operation owner |
+| `WorkerRestore` Tool | Observed token in Tool arguments and durable Tool-call-derived request identity |
+| Existing TUI Restore/resume path | Selected row's observation token and retained request; no new TUI entrance |
+| Internal Coder, Orchestrator, subject reuse and compensation | Domain intent journal pins the request/token before dispatch and retains it through unknown result and restart; it does not own lifecycle status or bypass Runtime validation |
+| Runtime startup / pending reconciliation | Private recovery of the existing persisted Restore owner and operation; not an external unguarded new-intent endpoint |
+
+The Workspace internal intent journal stores client retry identity only. Runtime remains the lifecycle authority; it is not a second Restore state machine, Worker ledger or cross-screen lock.
+
+## Observation and cleanup-plan ordering
+
+`workspaceWorkersStore` retains the existing sidebar stream/projection and also fetches the full Workspace Worker catalog for the page. Full catalog metadata preserves retention, attachments, implementation and targets that are not sidebar entries. Stream updates overlay lifecycle/availability/foreground snapshot/token without interpreting absence as a new page exclusion.
+
+Every stream update or explicit refresh increments a local **observation scheduling version** and publishes `catalogRefreshing` before fetching. This local version fences asynchronous display work; it is **not** Restore admission authority and is never sent to Runtime. A newer event/GET supersedes older GET completion. Old POST summaries and page-loader data cannot overwrite the active catalog. On disposal, requests and subscription callbacks are fenced. Catalog fetch errors have their own stable alert episode rather than clearing transport/Workdir alerts.
+
+The page invalidates cleanup plans immediately when observations refresh. It does not fetch or install plans while `catalogRefreshing` is true. Successful refresh completion triggers plan reads even when the resulting catalog contents did not change. Workspace/lifetime and cleanup epochs reject late previous plan responses. A failed refresh leaves old candidates unavailable instead of treating old page data as fresh.
+
+Both components retain local busy state only. Requests pin their starting Workspace/target, and late replies after Workspace switch/disposal cannot update another Workspace's notices, list or busy state. Sidebar and page may issue competing requests: atomic Runtime admission—not a cross-component client lock—decides their outcomes. Console links and existing sidebar filtering remain unchanged.
+
+## Validation surfaces
+
+- Runtime admission tests prove stale/ABA and serialization, durable same-request replay and pending reconciliation.
+- Runtime management, embedded/remote providers, Workspace aliases/scoped/control routes, Tool and existing TUI callers exercise the guarded API rather than a legacy unguarded Restore.
+- `tests/runtime-workers.test.ts` and `test/sidebar/worker-actions.test.ts` prove response/request and notice contracts; the Runtime parser tests are included in the normal Web test task.
+- Rendered `worker-stop.browser.spec.ts` and `workers-page.browser.spec.ts` cover menu/action eligibility, busy behavior, four outcomes and errors, explicit retry, Workspace/disposal fences, ordinary Console/Pin/Delete, sidebar-origin updates and cleanup epochs.
+- `worker-restore-store.browser.spec.ts` tests the real store's full metadata, GET/event ordering, dependent-plan invalidation, no POST recovery loop, catalog-only target preservation and disposal.
+
+Exact executed validation counts and immutable source/review evidence belong to the repository Merge Request and final approved Ticket handoff.

docs/report/2026-10-06-t701-validation.md

@@ -0,0 +1,49 @@
+# T-701 validation ledger
+
+This records concrete intermediate failures rather than treating later passes as erasing them. Final source and review evidence are on the repository Merge Request.
+
+## Validation approach
+
+Start with Restore/parser/UI/serialization boundary filters, then the eight changed crates and the normal root `cargo check` compile closure. Cargo jobs are bounded to three; full crate test concurrency is bounded to four. No all-workspace test, `--all-targets`, Nix/Docker build, deployment, process restart or live Worker repair is part of this validation.
+
+The Web production bundle, `svelte-check`, normal Deno tests and actual rendered component suites have passed. Those results alone do not prove Backend admission, replay or preparation recovery. Full Rust suites exercise callers, schema migration and dependent inventories that narrow guard filters cannot prove.
+
+## Intermediate failures and disposition
+
+1. Early root checks failed because the Runtime request/preparation DTO integration was not yet present. This was an incomplete implementation, not a baseline defect. A subsequent root check passed after the integration.
+2. Early Server API full tests reported three stale generated artifacts after the OpenAPI contract changed. Regenerating the affected closure artifacts resolved them; the full default suite passed 70 tests. TypeScript-feature tests were also executed after regeneration.
+3. Adding full catalog GETs exposed existing browser fixtures that returned Workdir-only shapes to every endpoint. The fixtures now return the exact `/workers` contract, and the Workdir-error fixture fails only its intended endpoint. Catalog and transport/Workdir failures have separate alert identities.
+4. Parent full `cargo test -p yoi-workspace-server -- --test-threads=4` produced **614 passed / 4 failed** (618 library tests):
+   - `hosts::tests::embedded_runtime_registers_routes_input_without_internal_leaks`: a broad forbidden-word assertion rejected the newly public observation field's word `token`. The private credential/token non-leak contract must be preserved while allowing the explicitly public observation token.
+   - `server::tests::embedded_runtime_api_routes_by_runtime_and_worker_ids_without_leaking_internals`: an old `{}` Restore fixture received 422 because guarded Restore now requires the observed token and request ID. Update the real route fixture, do not restore an unguarded compatibility path.
+   - `workspace_deletion::tests::explicit_purge_inventory_covers_every_workspace_scoped_table`: the new internal Restore intent journal was absent from the explicit purge inventory. Integrate its Workspace deletion lifecycle; do not weaken the exhaustive inventory check.
+   - `server::tests::ticket_checker_suppresses_stale_and_clean_results_and_diagnoses_invalid_output`: the inherited stale-checker `execution.take_inputs().is_empty()` assertion failed. Earlier T-702 evidence reproduced this named instability on an isolated original baseline; see [T-702 validation report](2026-10-06-t702-stale-checker-validation.md). T-701 has not fixed that checker, and this failure is not conflated with the three new integration regressions.
+
+## Follow-up disposition and final pre-publication results
+
+The projection fixtures now permit only the explicit `restore_observation_token` key and validate its UUIDv7 value; private token/credential fields remain forbidden. The HTTP fixture sends the real observed token and request ID. The internal intent table is in the exhaustive Workspace deletion inventory.
+
+The final restart recovery test exposed one obsolete Runtime expectation: an unowned startup/legacy pending operation must be explicitly recoverable, not permanently conflict. Public recovery now binds a receipt to the exact existing lifecycle operation ID, without candidate preparation or a replacement execution. Tests assert a competing request conflicts, recovery preserves the ID, and terminal replay after Stop never launches again. A Tool recovery fixture also used path-shaped IDs rejected by the existing authority-ID validator; it now uses valid authority IDs and proves the two explicit calls send an identical request body.
+
+After those fixes, a full Server run reported **618 passed / 1 failed**, solely the inherited stale-checker assertion above. Its isolated test subsequently passed. An additional whole-crate run with the same four-thread configuration and **no skips** passed **619 library + 11 binary tests**. This does not erase the earlier failure or claim to fix the checker race.
+
+| Executed command | Final pre-publication evidence |
+| --- | --- |
+| root `cargo check` | Passed normal default-member compile closure, including TUI and Server |
+| `cargo test -p worker-runtime -- --test-threads=4` | 332 library + 34 binary tests passed |
+| `cargo test -p yoi-workspace-server -- --test-threads=4` | 619 library + 11 binary tests passed in the additional no-skip run; previous flaky failure retained above |
+| `cargo test -p worker -- --test-threads=4` | 804 library + 109 integration tests passed |
+| `cargo test -p runtime-api` | 9 tests passed |
+| `cargo test -p server-api` / `--features typescript` | 70 / 77 tests passed, including generated artifact freshness |
+| `cargo test -p protocol` / `--features typescript` | 78 / 83 tests passed |
+| `cargo test -p client` | 52 library + 2 integration tests passed |
+| `cargo test -p tui` | 331 tests passed |
+| Web `deno task check` | 0 errors, 0 warnings |
+| Web `deno task test` | 504 tests passed |
+| Web `deno task test:component` | 249 tests in 25 files passed, including the real paired sidebar/page/store suite |
+| Web `deno task build` | Production build passed; existing chunk-size warning remains |
+| `cargo fmt --all -- --check`, `git diff --check HEAD` | Passed |
+
+Additional exploratory Runtime feature checks did **not** pass: `--no-default-features` test/check cannot resolve existing unconditional `workspace_request` / `reqwest` imports; the test target additionally references disabled `fs_store`. HTTP-only check also lacks the existing retention and futures dependencies. Inspection of original HEAD confirms those imports/gates already exist; no isolated baseline build was performed for these configurations. These are recorded as failed exploratory checks, not semantic passes or newly supported configurations. The default Runtime configuration, which contains fs-store and ws-server, is exercised in full above. This Ticket does not redesign unrelated Runtime feature gating.
+
+Raw session validation logs are retained under ignored `target/t701-validation/`, not shipped as product source. Immutable published source, independent verdict and any subsequent fix/review evidence belong to the explicit repository Merge Request.