142 Commits
Author SHA1 Message Date
Hare 2d7da04502 docs: remove redundant Ticket access prohibition 2026-11-07 09:25:42 +09:00
Hare b4099ddda4 fix: wait for remote workdir cleanup confirmation 2026-11-07 09:25:42 +09:00
Hare 04410a5022 fix: allow safe workdir cleanup of FIFOs and Unix sockets 2026-11-07 09:25:42 +09:00
Hare 682747e9f7 fix: remove aggregate history size limit 2026-11-06 04:51:35 +09:00
Hare 00f327da4b fix: remove Flow verifier turn limit 2026-11-05 22:36:18 +09:00
Hare aa7bb6282d fix: reuse scoped validation evidence without reusing approval
T-847: separate complete-candidate review responsibility from rerunning unchanged checks, retain source/environment provenance and unverified outcomes, and align coder-review handoff retry identity with the Host-owned tool contract. Verify fresh Profile, Ticket/MR guidance, and actual builtin Flow input composition. Also covers T-850 instruction wiring; real-model operational comparisons remain outstanding.
2026-11-05 03:19:25 +09:00
Hare 33d7ef8fa0 fix: keep delegated diagnosis within agreed scope
T-850: distinguish deliverable dependencies and actual shared resources from same-named files on independent branches. Avoid artificial approval gates after ordinary partial reports; retain explicit stops, authority limits and bounded diagnosis.
2026-11-05 03:19:08 +09:00
Hare 0ea6308dae feat: add compact Ticket and MR read views
T-848: deduplicate model-facing bodies, preserve exact post reads and paged history, and derive MR status from complete evidence rather than the returned page. Update strict browser parsing and generated contracts.
2026-11-05 03:18:53 +09:00
Hare e7351c16b0 fix: reuse Workdir attachments independently of requested alias 2026-10-30 21:22:00 +09:00
Hare 00d96b7163 fix: scope Workdir removal to explicit requests 2026-10-30 20:33:57 +09:00
Hare 2b4bf4b7bd fix: log Workdir cleanup busy reasons 2026-10-30 20:33:57 +09:00
Hare e7ebc92f83 fix: preserve complete merge request reviews 2026-10-30 20:33:57 +09:00
Hare 181b0d637e docs: define Workspace Plugin API contracts 2026-10-28 12:15:35 +09:00
Hare b0733e53b2 fix: release removed Subject bodies while preserving journal history 2026-10-28 12:15:35 +09:00
Hare 618880d38d fix: merge develop and reconcile schema histories
Preserve both published schema-103 histories and converge their Passkey results and Job input retention features transactionally at schema 104. Retain local Reviewer, integration-event, auth and sidebar fixes while adopting develop restore and E2E updates; regenerate API contracts.

Validation: schema convergence regressions, full Server/Worker/Runtime suites, server-api and merge-request tests, Web unit/component tests, root cargo check, E2E harness tests and restore/auth target compilation, generated artifacts, fmt and naming checks. Product-process E2E not rerun; live services and databases untouched.
2026-10-28 04:01:34 +09:00
Hare 79b76c8789 fix: merge worker restore into develop host lifecycle
Integrate integration/worker-restore without reviving removed Worker socket/discovery paths. Keep restore candidate allocation and canonical effects deferred until activation, with registry reclamation and observer publication fences on the in-process controller.

Preserve develop BackendJob/provenance, immutable Journal body capture and erasure, and Workdir diagnostics. Adapt obsolete socket fault tests to in-process publication and registry drift boundaries.

Validation: root cargo check; ordinary worker, worker-runtime and standalone package tests; Server restore (43) and Journal (49) filters; mcp/server-api/runtime-api/session-store tests; optional Runtime feature check; OpenAPI and six generated TypeScript checks; Web restore unit tests (15); runner tests (21); fmt, naming and diff checks. Full workspace/Server suites, E2E, live/model and fixture generation not run.
2026-10-28 00:06:17 +09:00
Hare 897c025b85 feat(journal): execute frozen body inputs with durable delivery and erasure fences 2026-10-27 08:03:43 +09:00
Hare 5b12c3e085 fix: make mobile sidebar swipes track touch gestures 2026-10-27 06:49:28 +09:00
Hare 623c31615d fix: assign unique event refs to merge integration comments 2026-10-27 06:15:48 +09:00
Hare af719e387d fix: align reviewer guidance with Bash-enabled scope 2026-10-27 06:15:48 +09:00
Hare dddb21bd70 fix: preserve possible repair during v9 Restore migration 2026-10-27 01:24:50 +09:00
Hare e8d4e964cb fix: refresh Workdir API generated contract digest 2026-10-26 21:25:46 +09:00
Hare eb24cb04dc fix: revalidate Host authority when publishing prepared restore controllers 2026-10-26 20:39:28 +09:00
Hare e84057f402 fix: retain exact restore ownership in test execution backends 2026-10-26 20:39:28 +09:00
Hare 15c3006d09 fix: propagate fallible restore publication through server factory adapters 2026-10-26 19:59:56 +09:00
Hare b3209c2012 test: verify real orphan pin repair across owned restore process death 2026-10-26 19:38:24 +09:00
Hare 6f5a424661 fix: reclaim stale allocation authority before restoring committed run artifacts 2026-10-26 19:37:26 +09:00
Hare 8d47d2670b fix: initialize reconciliation fault evidence in offline store construction 2026-10-26 14:43:38 +09:00
Hare b19de45a82 test: verify original catalog projection after receipt reconciliation 2026-10-26 14:40:58 +09:00
Hare 1e88893ef9 fix: preserve non-restore cleanup evidence through bridge refactoring 2026-10-26 14:31:45 +09:00
Hare 1fd2e09971 test: cover real partial publication and terminal receipt durability faults 2026-10-26 14:28:37 +09:00
Hare f85d831104 fix: retain committed controller paired streams across bridge admission failure 2026-10-26 14:10:02 +09:00
Hare 2b24db5a15 test: fence superseded restore replay and old Shutdown after replacement 2026-10-26 13:49:48 +09:00
Hare e1945d977c test: prove real postcommit activation timeout ownership and Stop fencing 2026-10-26 13:40:51 +09:00
Hare a10875d68e fix: retire the exact stopped Controller from runtime observation 2026-10-26 13:36:37 +09:00
Hare f35dcef4d0 test: synchronize owned runtime crashes at restore decision checkpoints 2026-10-26 13:10:54 +09:00
Hare f5ebae1134 test: verify restore response loss and owned runtime restart 2026-10-26 12:27:33 +09:00
Hare 6be4e8388b feat: recover saved restore operations by their durable decision 2026-10-26 11:50:54 +09:00
Hare 9e3f87ac52 fix: runtime retain restore facts when Stop supersedes publication 2026-10-26 11:50:54 +09:00
Hare ceb01f6eaa feat: runtime own restore continuation beyond caller timeout 2026-10-26 11:50:54 +09:00
Hare 1b13563e19 feat: runtime commit restore decisions before execution publication
Persist explicit restore commit evidence, reconcile canonical write uncertainty, and publish the same prepared controller before Accepted. Keep owned cleanup and publication failures distinct. Preserve optional transport/store feature boundaries and adapt external controller constructors.
2026-10-26 11:50:54 +09:00
Hare 848a1e9a5b feat: worker separate restored controller preparation from publication
Prepare the real receiver, full snapshot and paired streams privately; retain activation effects and startup cleanup until the host decision.
2026-10-26 11:50:54 +09:00
Hare a77453338c feat: mcp retain stdio process ownership before initialization
Allow startup cleanup to own and reap the exact spawned client across handshake failure or cancellation.
2026-10-26 11:50:54 +09:00
Hare d12bc2ea0b fix: deduplicate live and committed task updates 2026-10-26 10:13:36 +09:00
Hare a1df3742b1 fix: integrate passkey results with current dogfooding shell
Preserve authenticated home/account navigation and guarded mobile sidebar swipes. Keep both browser task selections. Complete the new account fixture contract and synchronize lifecycle probes on visible registered-key results, without changing their success or security assertions.
2026-10-26 09:50:13 +09:00
Hare d8818b776b feat: open and close mobile sidebar with guarded swipes 2026-10-26 09:39:35 +09:00
Hare 40c36ccb1d fix: restore authenticated home and account navigation 2026-10-26 09:39:35 +09:00
Hare 80c151d084 fix: fence recovered passkey results against cross-tab changes 2026-10-26 07:52:06 +09:00
Hare c6a8759e7b fix: fence activation and setup against delayed cross-tab events
Capture the shared authentication stamp as well as generation before dispatch and across response/confirmation/application awaits. Do not let stale setup failure reconciliation overwrite a newer tab's stamp. Add no-event-yet logout and account-change regressions for activation and setup, including prompt, pre-complete and post-dispatch confirmation boundaries.

Addresses MR AC4_CROSS_TAB_ACTIVATION. Validation: activation21/setup40 narrow tests; Web720 unit+625 component, check0/0/build, activation and hosting browser suites; root check/fmt/diff; both explicit real Backend browser suites including native CTAP2. Existing Server exclusions and 15 identical baseline naming findings remain unchanged. No verifier, binding, migration or rendering contract weakened.
2026-10-26 07:34:04 +09:00
Hare f17b5212c7 fix(auth): confirm passkey operations and preserve safe legacy migration
Fence displayed identity and native prompt generations, classify secret-free outcomes, recover atomic addition receipts without restoring old credentials, and expose truthful own-key verification metadata. Preserve required verifier and enrollment authority boundaries. Include real-store, production browser, native CTAP2 and visible responsive regressions.

Validation: root check/fmt; API 115; passkey security 25; Web unit 712/component 605; auth/entry/activation/hosting browsers; product auth E2E 8 and two explicit browser suites. Server suite: 1262 lib+12 CLI, 3 ignored and 2 environment-specific exclusions. State-name checker retains 15 byte-identical baseline findings; no new exceptions. Physical keys/mobile and hosted CI remain unverified.
2026-10-26 07:18:23 +09:00
Hare 85f70d4bc1 test: adapt fixed Worker restore fixtures to integration lane 2026-10-26 06:20:22 +09:00
Hare b284934e34 test: add captured Codex worker restore fixture 2026-10-24 12:57:28 +09:00
Hare 2d33321fab fix: validate live fixture snapshots and shutdown boundaries 2026-10-24 12:57:28 +09:00
Hare 0975471986 fix: preserve diagnostic context in backend test assertions 2026-10-24 12:51:28 +09:00
Hare 214d6e21dd fix: assert classified workdir session closure in tests 2026-10-24 12:40:37 +09:00
Hare fca5c23345 fix: reject unrestorable and opaque fixture records 2026-10-24 12:39:22 +09:00
Hare f9b1500ba2 fix: align panel tests with current ticket case states 2026-10-24 12:33:31 +09:00
Hare 931bf3b27d test: add isolated saved worker restore scenarios 2026-10-24 12:33:31 +09:00
Hare 6bb671be21 test: add typed stopped worker fixture transport 2026-10-24 12:19:43 +09:00
Hare b8f2fa479c test: add isolated backend fixture seeding 2026-10-24 12:15:10 +09:00
Hare 1639be9841 fix: allow reviewers to read related tickets 2026-10-23 03:22:45 +09:00
Hare 907392cce8 fix: preserve restore effects and fence prepared attachment replay 2026-10-23 00:21:02 +09:00
Hare 6776973e4c fix: preserve original session coordinates through capture selection 2026-10-22 23:17:25 +09:00
Hare 73a3e674c9 feat(worker): separate restored candidates, repair, and activation 2026-10-22 22:58:57 +09:00
Hare 994e75233b feat(session): share immutable captures and bounded body transport 2026-10-22 22:25:47 +09:00
Hare 8b6e3070dd fix: align Workdir session cwd and model environment
Project alias, cwd, scope and command ceilings from actual sessions. Bind initial attachment cwd once in the shared broker/router while keeping root-relative delegation and detach generations. Preserve caller coordinates across reads, searches, checked checkout and commands; keep provider authorization and symlink boundaries.

Connect real SubWorkers, all filesystem Tools, scoped aliases, HTTP providers and limited Host grants in permanent regressions. Validated related normal suites (workdir http-client, tools, worker, worker-runtime, server-api typescript, fs-operation), workspace-server normal suite (1272 passed, 3 existing ignores), root check, product binary builds, rustdoc/doctests, generated contract checks, format and state-contract names. Test roots are checkout-local; Git parent discovery requires GIT_CEILING_DIRECTORIES at TMPDIR, confirmed against exact b860 baseline.
2026-10-22 20:46:10 +09:00
Hare b860c67ea2 fix: align Runtime restore diagnostics and socket test fixtures 2026-10-22 03:46:46 +09:00
Hare 533477c2e7 feat: unify headless execution under the Standalone Host
Expose an optional private same-uid Unix operator adapter and JSONL serve/attach CLI using the production Standalone Profile, store, restore lease and shutdown lifecycle. Remove direct Worker subprocess/discovery/readiness/controller sockets and parent socket callbacks; retain metadata-only peer registration and legacy safety readers.

Validated real Host/client input, events, provenance, disconnect, admitted shutdown and shared production restore/duplicate exclusion; normal standalone/worker/manifest/protocol/client/yoi/tui/server tests, root check, rustdoc, fmt and state-contract audit. Runtime lib: 476 passed, damaged_checkout_does_not_block_restore_but_is_rejected_when_used fails at missing_git diagnostic; unchanged restore logic, not relaxed or retried. Runtime bins/docs pass.
2026-10-22 01:29:39 +09:00
Hare 5aa2dc579a fix: integrate bounded Bash cleanup diagnostics 2026-10-22 00:46:41 +09:00
Hare 280b3d2a25 Merge branch 'develop' of /home/hare/Projects/yoi into work/T-775-cleanup-diagnostics 2026-10-22 00:24:00 +09:00
Hare ec74f4cdc4 fix: preserve Workdir failure diagnostics and audit outcomes 2026-10-22 00:16:11 +09:00
Hare 5af9dc8e4a test: focus checkout hardlink coverage on alias and symlink contracts 2026-10-21 23:57:01 +09:00
Hare ca6d4b4b72 fix: audit validated external workdir operation outcomes 2026-10-21 20:39:46 +09:00
Hare cccecbe7ca fix: integrate command output failure diagnostics 2026-10-21 06:52:23 +09:00
Hare 7a64c38493 fix: merge shared command diagnostic assertions
# Conflicts:
#	crates/workspace-server/src/server.rs
2026-10-21 06:40:26 +09:00
Hare b1361d86e9 fix: distinguish external Workdir admission dispatch and response failures
Integrate the published T773 audit contract without overwriting operation outcomes. Preserve bounded admission, send refusal, response loss/timeout and invalid result reasons independently of audit failures and request stages. Signed HTTP/SQLite/log fault tests pass (10), API logging 14 and relevant focused assertions pass. Root check, docs, generated artifacts, fmt/diff and state-contract naming checks pass. Full six-package normal suite ran with no-fail-fast but is not green: long Unix socket fixture paths and existing shared close assertion plus environment-isolation failures are recorded in target/t774 logs. Isolated focused checks resolved the latter; socket validation requires a granted shorter test root. This is a partial checkpoint, not a reviewed completion.
2026-10-21 06:39:54 +09:00
Hare 6181426abd fix: assert retained checkout outcome diagnostics 2026-10-21 06:27:54 +09:00
Hare cd331b93c6 Merge remote-tracking branch 'origin/work/T-773-workdir-audit' into work/T-774-workdir-failure-diagnostics
# Conflicts:
#	crates/workdir/src/http.rs
2026-10-21 06:13:35 +09:00
Hare 60eabea19a fix: assert retained remote session close diagnostics 2026-10-21 06:13:16 +09:00
Hare 7394ec6784 fix: retain safe Workdir open authorization and response failure causes
Keep bounded reasons and derived failure stages separate from public classification. Reject provider-authored raw error text on transport decoding. Preserve identity mismatch when cleanup fails and retain its handle for explicit close without opening again. Add signed Runtime HTTP, Registry/API logging, remote mismatch, authorization no-dispatch and deferred fault coverage. Focused tests and Workdir suite pass; full-source validation follows T773 audit contract merge.
2026-10-21 06:13:07 +09:00
Hare 28b7cc414b fix: retain external workdir results independently of audit failures 2026-10-21 06:04:39 +09:00
Hare 86d5cf9403 fix: integrate safe Drive and config failure diagnostics 2026-10-21 06:01:51 +09:00
Hare 6f0a23e5cc Merge commit '62e2b4430588bacf90a2812a0a88772fc0a00c58' into work/T-775-cleanup-diagnostics 2026-10-21 05:56:16 +09:00
Hare 2f4a46bbdb Merge commit 'a1748b552c52fd5f68de6ba852421fd778aec010' into work/T-775-cleanup-diagnostics 2026-10-21 05:55:48 +09:00
Hare 4872cf2aac fix: retain independent Bash Drop cleanup diagnostics 2026-10-21 05:55:48 +09:00
Hare 734d4b44ae Merge remote-tracking branch 'origin/work/T-772-command-output-integrity' into work/T-774-workdir-failure-diagnostics 2026-10-21 05:55:33 +09:00
Hare 6810f51f46 fix: retain safe Drive and config adapter failure diagnostics 2026-10-21 05:53:58 +09:00
Hare a1748b552c fix: validate closed-session diagnostics and fault quotas 2026-10-21 05:52:48 +09:00
Hare 62b2d27b4a fix: retain command failure stages and output collection state 2026-10-21 05:40:39 +09:00
Hare 62e2b44305 fix: retain safe Journal compaction failure diagnostics
Keep stage/reason classification in the Journal Feature without changing frozen input, metadata or fallback behavior. Exercise connection, Host read failures, size budget and secret-bearing fixtures through compaction persistence.

Validation: old freeze function with new tests fails connection and snapshot/size diagnostics (3 failures); new code passes all six selected cases and cargo test -p worker. Serialization is currently unreachable for the derived JSON-safe snapshot fields; no artificial runtime failure is claimed.
2026-10-21 05:32:07 +09:00
Hare b5c105545c fix: close unattached deferred sessions in restore rollback
Logical Workdir routing can use a Workspace session instead of the operation's raw Deferred handle, and reconstruction can fail before routing. Close the exact handles frozen in the accepted operation request after router cleanup and before releasing allocation ownership. Do not resolve aliases against a new execution or consult current Host grants for cleanup.

Reproduced on 61c20b099 with diagnostic-only test output: rollback emptied routers with provider opens 0, but raw read succeeded and opened the provider (1). Retain the SessionClosed and zero-open regression, add pre-router Metadata/Reconstruction/Binding recovery and operation-mismatch/same-alias isolation coverage. Focused Profile and deferred cases pass; full exact-source validation follows this commit before publication.
2026-10-21 04:55:38 +09:00
Hare aa0fb2f787 fix: continue workdir output reads after long-poll expiry 2026-10-21 04:48:43 +09:00
Hare 1b6d4dbfd8 fix: preserve workdir failure diagnostics across transport 2026-10-21 01:22:04 +09:00
Hare 61c20b0995 fix: adapt T-769 recovery onto current restore lane checkpoint
UNCOMPILED TRANSPORT / PRESERVATION CHECKPOINT, NOT A VALIDATED CANDIDATE. One-time recovery authorization permits publication before validation so the parent can validate this exact source in a separate Workdir. Prior compilation attempts were interrupted by Workdir-session unavailability; cause and test outcome remain unknown. No Cargo build/check/test was started in this bounded checkpoint turn.

Merge current lane 4ef7b03651 into existing source b7a22146a0. Selectively adapt typed factory failure and same-operation cleanup/replay while preserving current saved Feature restore and operation-time authorization, logical/deferred Workdirs, T765 Host/schema and T768 replay-safe Subject grant. No T770/T771 implementation.

All 19 target-relative paths and added test bodies were source-audited. Workspace fmt check, scoped rustfmt, whitespace/diff checks passed; conflict markers zero. Excluded unrelated formatter changes and untracked compiler artifacts. Local audit: target/t769-checkpoint/AUDIT.md.

Handoff to parent: compile and run focused and package/API regressions on this exact source, fix as necessary, then obtain fresh independent review. Existing MR 01a15606-a6ba-7d23-8d60-0874607399e7 only; no new review requested here. No acceptance, approval, adoption, target/develop push, integration, Ticket closure or service update is asserted.
2026-10-21 00:20:54 +09:00
Hare 4ef7b03651 fix: stop background session checks and loading flicker 2026-10-20 22:35:38 +09:00
Hare af615ab177 fix: retain archive and worker restore failure diagnostics 2026-10-20 22:00:58 +09:00
Hare 338e39bcdb fix: defer restored workdir access until use 2026-10-20 22:00:58 +09:00
Hare 3cfe04be3a fix: preserve saved feature configuration during worker restore 2026-10-20 22:00:39 +09:00
Hare 70d153b365 fix: isolate unreadable archives from runtime startup 2026-10-20 17:39:38 +09:00
Hare ce93849948 chore: merge develop into hare/develop 2026-10-20 15:39:09 +09:00
Hare 3c499719fe feat: default Coder ticket work to review flow 2026-10-20 15:38:50 +09:00
Hare 9e1d3df7ad fix: remove overbroad Worker bypass instructions 2026-10-20 15:38:50 +09:00
Hare 18bac16a23 fix: require replay-safe control lifecycle for Subject delegation 2026-10-20 08:12:26 +09:00
Hare 9005856f77 fix: replay accepted delegation without reserving active Workdirs 2026-10-20 07:24:51 +09:00
Hare e39372a52a Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants 2026-10-20 06:38:12 +09:00
Hare 223a74a9e2 fix: retain byte command transport and explicit authorization fixtures 2026-10-20 06:38:12 +09:00
Hare f38d9a16ea Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants
# Conflicts:
#	crates/worker-runtime/src/worker_backend.rs
#	openapi/server-api.json
#	web/workspace/src/lib/generated/companion-api.ts
#	web/workspace/src/lib/generated/repository-api.ts
#	web/workspace/src/lib/generated/runtime-api.ts
#	web/workspace/src/lib/generated/ticket-api.ts
#	web/workspace/src/lib/generated/worker-launch-api.ts
2026-10-20 06:24:11 +09:00
Hare b135fa5e99 feat: enforce explicit Subject permissions and delegated Worker grants 2026-10-20 06:22:58 +09:00
Hare b7a22146a0 fix: recover confirmed worker restore failures 2026-10-20 06:00:58 +09:00
Hare 07b8b6d41b Merge commit 'b8e548b92d3c785c482dbb418517cc5a2cf9e7e0' into refactor/T-765-session-feature-boundary 2026-10-20 05:34:22 +09:00
Hare 8017c55d82 fix: retain annotated compaction history and honor non-System input policy 2026-10-20 05:34:22 +09:00
Hare b8e548b92d chore: integrate reviewed browser session navigation 2026-10-20 05:31:32 +09:00
Hare 90e38551cf Merge commit '00ad02d04728a8e32a82a4e72747fefcf8a82d4e' into refactor/T-765-session-feature-boundary
# Conflicts:
#	crates/worker/src/feature/builtin.rs
2026-10-20 05:25:16 +09:00
Hare 3e8164e757 fix: fence hosting setup verification against superseded sessions 2026-10-20 04:53:43 +09:00
Hare 3b3381900f fix: preserve role-neutral input policy across compaction and replay 2026-10-20 04:45:50 +09:00
Hare 00ad02d047 chore: integrate bounded shared text page refusal 2026-10-20 03:55:13 +09:00
Hare a774925dec fix: reject byte-truncated line windows before advancing search 2026-10-20 03:53:03 +09:00
Hare 8f759911ab fix: emit Session use policy type in browser bindings 2026-10-20 03:52:55 +09:00
Hare daf1f6ed50 fix: route retained Session migration through Runtime staging 2026-10-20 03:52:55 +09:00
Hare 86602240fa refactor: bind Subject metadata and prompt confirmation to Host lifecycle 2026-10-20 03:52:55 +09:00
Hare bfeee487fd feat(auth): separate anonymous entry from expired session recovery 2026-10-20 03:52:48 +09:00
Hare f9830fe7dd test: reject Bash search pages that would skip an unread long line 2026-10-20 03:40:36 +09:00
Hare de96c5b577 chore: update shared file contracts and validation fixtures
# Conflicts:
#	crates/worker/src/checkout_http_tests.rs
2026-10-20 03:26:56 +09:00
Hare 6e729ed06c refactor: isolate Feature System input migration and compaction policy 2026-10-20 03:21:22 +09:00
Hare f38b2a52b1 fix: record native prior Read only after result validation 2026-10-20 03:21:14 +09:00
Hare dd5384df3b feat: expose retained Bash streams through common WIP file contracts 2026-10-20 03:13:46 +09:00
Hare 6576850b7a chore: connect shared file contracts for Bash validation 2026-10-20 03:03:53 +09:00
Hare 4555aace28 feat: capture bounded Bash output over replayable byte transport 2026-10-20 03:01:22 +09:00
Hare 47b32ef6ab fix: bind capability changes to Object freshness and update contract consumers 2026-10-20 02:53:21 +09:00
Hare bb5c50f9ab feat: persist System input policies through public consumers 2026-10-20 02:44:28 +09:00
Hare 32a409c15d feat: share virtual file contracts and checked binary byte ranges 2026-10-20 02:18:49 +09:00
Hare 19514a5d6a feat: define bounded Feature-owned system input envelopes 2026-10-20 02:14:52 +09:00
Hare 887184796e docs: distinguish commit completion from Feature authority 2026-10-20 02:05:02 +09:00
Hare cd0eb5eb00 fix: allowlist public system input projections 2026-10-20 02:02:28 +09:00
Hare e8dfb3e68d refactor: notify system input producers after durable commit 2026-10-20 02:02:28 +09:00
Hare 434ce34da7 refactor: bind checkout target lifetimes to Object observations 2026-10-20 02:01:22 +09:00
Hare bfb51b7ca9 fix: bound encoded common text results including escape and record overhead 2026-10-20 01:48:23 +09:00
Hare 38d9069632 feat: publish stable root file contracts and target-owned dispatch 2026-10-20 01:46:30 +09:00
Hare e5749b9be0 test: integrate Drive connection and lifetime coverage 2026-10-20 00:37:32 +09:00
Hare 387b5ff7df docs: record Drive connection coverage and fault evidence 2026-10-20 00:35:11 +09:00
Hare ec6c5beda0 test: connect Drive publication to client and resource lifetimes 2026-10-20 00:34:50 +09:00
Hare 8e40fae6f4 fix: recover unpublished Subject creation and preserve auth rejections 2026-10-19 18:56:24 +09:00
446 changed files with 103827 additions and 16595 deletions
Generated
+16 -2
View File
@@ -3990,6 +3990,7 @@ dependencies = [
"protocol",
"serde",
"serde_json",
"session-store",
"tokio",
"workdir",
]
@@ -4783,12 +4784,14 @@ version = "0.1.0"
dependencies = [
"chrono",
"feature-storage",
"job",
"manifest",
"memory",
"rusqlite",
"serde",
"serde_json",
"server-api",
"session-store",
"sha2 0.11.0",
"tempfile",
"thiserror 2.0.18",
@@ -5266,6 +5269,7 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"tracing",
"uuid",
"workdir",
]
@@ -6496,6 +6500,7 @@ name = "workdir"
version = "0.1.0"
dependencies = [
"async-trait",
"axum",
"fs-operation",
"libc",
"manifest",
@@ -6519,7 +6524,6 @@ dependencies = [
"axum",
"base64 0.22.1",
"chrono",
"clap",
"client",
"config-source",
"dotenv",
@@ -6557,6 +6561,7 @@ dependencies = [
"toml",
"tools",
"tracing",
"tracing-subscriber",
"tungstenite 0.28.0",
"uuid",
"wip-client",
@@ -6596,6 +6601,7 @@ dependencies = [
"tokio",
"tokio-tungstenite 0.29.0",
"toml",
"tools",
"tower",
"tracing",
"tracing-subscriber",
@@ -6683,6 +6689,7 @@ dependencies = [
"futures",
"manifest",
"project-record",
"protocol",
"reqwest",
"serde",
"serde_json",
@@ -6698,22 +6705,29 @@ dependencies = [
"tui",
"uuid",
"workdir",
"worker",
]
[[package]]
name = "yoi-e2e"
version = "0.0.0"
dependencies = [
"chrono",
"config-source",
"libc",
"manifest",
"protocol",
"reqwest",
"serde",
"serde_json",
"server-api",
"session-store",
"sha2 0.11.0",
"tempfile",
"tungstenite 0.29.0",
"webauthn-authenticator-rs",
"webauthn-rs-proto",
"worker-runtime",
"yoi-workspace-server",
]
[[package]]
+1
View File
@@ -908,6 +908,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let response = create_backend_worker_with_client(&target, &create, &api)
+1
View File
@@ -567,6 +567,7 @@ impl BackendWorkspaceProductClient {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let workspace_id = self.workspace_id.clone();
let response: BrowserCreateWorkerResponse = self.generated(move |client| async move {
+35
View File
@@ -72,6 +72,41 @@ mod tests {
}
}
#[test]
fn compiled_handoff_uses_host_owned_retry_identity() {
let flow = builtin_flow_source(CODER_REVIEW_FLOW_SLUG)
.unwrap()
.compile()
.unwrap();
let complete = flow
.state(&crate::StateId::new("complete").unwrap())
.unwrap();
assert!(
complete
.instructions
.contains("The Host owns transport retry identity")
);
assert!(
complete
.instructions
.contains("do not supply operation_key")
);
assert!(!complete.instructions.contains("same key and full input"));
let fix = flow.state(&crate::StateId::new("fix").unwrap()).unwrap();
assert!(
fix.instructions
.contains("rerun validation affected by the fixes")
);
assert!(
fix.instructions
.contains("Cite reusable evidence for unaffected scope")
);
assert!(
fix.instructions
.contains("Source movement invalidates the prior verdict")
);
}
#[test]
fn every_builtin_flow_compiles_and_matches_catalog_identity() {
assert!(!builtin_flow_sources().is_empty());
+30
View File
@@ -200,6 +200,7 @@ fn run_read_with_limits(
total_lines,
content_hash: content_hasher.finalize().into(),
truncated: end < total_lines || byte_truncated,
byte_truncated,
})
}
@@ -885,6 +886,34 @@ mod bounded_read_tests {
assert!(error.to_string().contains("exceeds provider limit 4096"));
}
#[test]
fn bounded_read_distinguishes_utf8_byte_cut_from_complete_line_continuation() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
fs::write(root.join("text"), "雪雪needle\nnext").unwrap();
for (offset, max_bytes, bytes, more, cut) in [
(0, 5, "雪", true, true),
(0, 64, "雪雪needle\n", true, false),
(1, 64, "next", false, false),
] {
let result = run_read_bounded(
&root,
ReadRequest {
path: FsPath::new("text").unwrap(),
offset,
limit: 1,
max_bytes,
},
&RootAccess(root.clone()),
BoundedReadLimits::new(1024, 64).unwrap(),
)
.unwrap();
assert_eq!(result.bytes, bytes.as_bytes());
assert_eq!(result.truncated, more);
assert_eq!(result.byte_truncated, cut);
}
}
#[test]
fn bounded_read_caps_retained_response_independently_of_request() {
let temp = tempfile::tempdir().unwrap();
@@ -907,6 +936,7 @@ mod bounded_read_tests {
assert_eq!(result.bytes, b"alpha");
assert_eq!(result.total_lines, 3);
assert!(result.truncated);
assert!(result.byte_truncated);
}
}
+4
View File
@@ -204,7 +204,11 @@ pub struct ReadResult {
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub total_lines: usize,
pub content_hash: ContentHash,
/// More source lines remain, or the selected window exceeded the response byte budget.
pub truncated: bool,
/// The selected line window was cut by its byte budget, possibly within a line.
/// Line-based cursors must not treat such a window as completely scanned.
pub byte_truncated: bool,
}
/// Binary-safe byte range read with an optional whole-file version fence.
+1 -38
View File
@@ -100,21 +100,6 @@ pub fn worker_allocation_path() -> Option<PathBuf> {
worker_allocation_path_from_runtime_dir(runtime_dir())
}
/// `<runtime_dir>/<worker_name>/` — per-Worker runtime directory.
pub fn worker_runtime_dir(worker_name: &str) -> Option<PathBuf> {
worker_runtime_dir_from_runtime_dir(runtime_dir(), worker_name)
}
/// `<runtime_dir>/<worker_name>/sock` — Worker's Unix socket path.
///
/// The `worker` crate's `RuntimeDir::socket_path()` creates the socket inside the
/// Worker process. This function **predicts** the path for external callers that
/// know the Worker name, such as the TUI attach flow. Both should return the same
/// path.
pub fn worker_socket_path(worker_name: &str) -> Option<PathBuf> {
worker_socket_path_from_runtime_dir(runtime_dir(), worker_name)
}
// ---- internals --------------------------------------------------------------
fn resolve_config_dir_from_parts(
@@ -204,20 +189,6 @@ fn worker_allocation_path_from_runtime_dir(runtime_dir: Option<PathBuf>) -> Opti
Some(runtime_dir?.join("workers.json"))
}
fn worker_runtime_dir_from_runtime_dir(
runtime_dir: Option<PathBuf>,
worker_name: &str,
) -> Option<PathBuf> {
Some(runtime_dir?.join(worker_name))
}
fn worker_socket_path_from_runtime_dir(
runtime_dir: Option<PathBuf>,
worker_name: &str,
) -> Option<PathBuf> {
Some(worker_runtime_dir_from_runtime_dir(runtime_dir, worker_name)?.join("sock"))
}
/// Treats empty environment values as unset. `std::env::var` distinguishes
/// `Ok("")` from `Err(NotPresent)`, but path resolution treats both as unset
/// for intuitive behavior.
@@ -455,16 +426,8 @@ mod tests {
PathBuf::from("/sand/sessions")
);
assert_eq!(
worker_allocation_path_from_runtime_dir(runtime_dir.clone()).unwrap(),
worker_allocation_path_from_runtime_dir(runtime_dir).unwrap(),
PathBuf::from("/sand/run/workers.json")
);
assert_eq!(
worker_runtime_dir_from_runtime_dir(runtime_dir.clone(), "foo").unwrap(),
PathBuf::from("/sand/run/foo")
);
assert_eq!(
worker_socket_path_from_runtime_dir(runtime_dir, "foo").unwrap(),
PathBuf::from("/sand/run/foo/sock")
);
}
}
+184 -83
View File
@@ -617,7 +617,8 @@ impl fmt::Display for McpRequestStage {
}
}
/// A running initialized stdio MCP client.
/// An owned stdio MCP process, initialized by `connect` or explicitly after
/// `spawn_uninitialized` when startup cleanup must retain it before any await.
pub struct McpStdioClient {
server_name: String,
limits: McpStdioLimits,
@@ -636,6 +637,9 @@ pub struct McpStdioClient {
impl McpStdioClient {
/// Spawn, initialize, negotiate capabilities, and send notifications/initialized.
/// Initialization errors attempt shutdown, but this convenience API cannot
/// return an unreaped client on failure/cancellation. Its error is not proof
/// of cleanup; use `spawn_uninitialized` with retained ownership when needed.
pub async fn connect(
spec: McpStdioServerSpec,
limits: McpStdioLimits,
@@ -667,6 +671,21 @@ impl McpStdioClient {
async fn spawn(
spec: McpStdioServerSpec,
limits: McpStdioLimits,
) -> Result<Self, McpClientError> {
Self::spawn_uninitialized(spec, limits)
}
/// Spawn without awaiting the handshake, so a cleanup owner can retain the
/// client before initialization can fail or be cancelled. Call
/// `initialize_with_timeout` before discovery; on error/cancellation, shut
/// down this same client rather than spawning a replacement.
///
/// Once the process spawns, this returns its owner even if an expected pipe
/// is missing. Initialization then fails, but cleanup can still reap the
/// actual child rather than relying on kill-on-drop for a post-spawn error.
pub fn spawn_uninitialized(
spec: McpStdioServerSpec,
limits: McpStdioLimits,
) -> Result<Self, McpClientError> {
let redactor = spec.redactor();
let mut command = Command::new(&spec.command);
@@ -688,29 +707,23 @@ impl McpStdioClient {
McpErrorKind::Io(redactor.redact(&err.to_string())),
)
})?;
let stdin = child.stdin.take().ok_or_else(|| {
McpClientError::new(
&spec.name,
McpPhase::Spawn,
McpErrorKind::Protocol("child stdin was not piped".into()),
)
})?;
let stdout = child.stdout.take().ok_or_else(|| {
McpClientError::new(
&spec.name,
McpPhase::Spawn,
McpErrorKind::Protocol("child stdout was not piped".into()),
)
})?;
let stderr = child.stderr.take().ok_or_else(|| {
McpClientError::new(
&spec.name,
McpPhase::Spawn,
McpErrorKind::Protocol("child stderr was not piped".into()),
)
})?;
let stdin = child.stdin.take();
let stdout = child.stdout.take();
let stderr = child.stderr.take();
let missing_pipe = if stdin.is_none() {
Some("stdin")
} else if stdout.is_none() {
Some("stdout")
} else if stderr.is_none() {
Some("stderr")
} else {
None
};
let stdin: SharedStdin = Arc::new(Mutex::new(Some(Box::new(stdin))));
let stdin: SharedStdin =
Arc::new(Mutex::new(stdin.map(|stdin| {
Box::new(stdin) as Box<dyn AsyncWrite + Send + Unpin>
})));
let diagnostics = Arc::new(Mutex::new(BoundedDiagnostics::new(
spec.name.clone(),
limits.max_diagnostic_lines,
@@ -718,16 +731,32 @@ impl McpStdioClient {
)));
let list_changes = Arc::new(Mutex::new(BoundedListChanged::new(spec.name.clone())));
let (tx, rx) = mpsc::channel(16);
let reader_task = spawn_stdout_reader(
spec.name.clone(),
stdout,
stdin.clone(),
tx,
limits.clone(),
redactor.clone(),
list_changes.clone(),
);
let stderr_task = spawn_stderr_reader(stderr, diagnostics.clone(), limits.clone());
// Do not return an error that discards an already spawned child. Queue
// setup failure before reader events so initialization cannot succeed.
if let Some(pipe) = missing_pipe {
tx.try_send(ReaderEvent::Error(McpClientError::new(
&spec.name,
McpPhase::Spawn,
McpErrorKind::Protocol(format!("child {pipe} was not piped")),
)))
.expect("new response channel has capacity");
}
let reader_task = match stdout {
Some(stdout) => spawn_stdout_reader(
spec.name.clone(),
stdout,
stdin.clone(),
tx,
limits.clone(),
redactor.clone(),
list_changes.clone(),
),
None => tokio::spawn(async {}),
};
let stderr_task = match stderr {
Some(stderr) => spawn_stderr_reader(stderr, diagnostics.clone(), limits.clone()),
None => tokio::spawn(async {}),
};
Ok(Self {
server_name: spec.name,
@@ -746,6 +775,25 @@ impl McpStdioClient {
})
}
/// Perform the bounded handshake on an already retained client. Failure or
/// cancellation does not drop or shut down the client: its owner must do so.
/// Do not retry initialization on a partially initialized connection.
pub async fn initialize_with_timeout(&mut self) -> Result<(), McpClientError> {
let started = Instant::now();
match timeout(self.limits.startup_timeout, self.initialize()).await {
Ok(result) => result,
Err(_) => Err(McpClientError::new(
&self.server_name,
McpPhase::Initialize,
McpErrorKind::Timeout {
operation: "startup".to_string(),
elapsed: started.elapsed(),
},
)
.with_diagnostics(self.snapshot_diagnostics().await)),
}
}
async fn initialize(&mut self) -> Result<(), McpClientError> {
let result: InitializeResult = self
.request(
@@ -1080,6 +1128,8 @@ impl McpStdioClient {
}
/// Close stdin and wait for process exit, falling back to terminate and kill.
/// Errors and cancellation retain the child for another attempt on this
/// client. Only confirmed reaping releases the process handle.
pub async fn shutdown(&mut self) -> Result<ShutdownReport, McpClientError> {
self.shutdown_started = true;
let deadline = Instant::now() + self.limits.shutdown_timeout;
@@ -1091,64 +1141,55 @@ impl McpStdioClient {
.await
.is_ok();
let mut child = match self.child.take() {
Some(child) => child,
None => return Ok(ShutdownReport::already_finished()),
};
// Keep the actual process handle in the client across every await and
// error. Taking it before wait loses retry/reaping authority on cancel.
if self.child.is_none() {
return Ok(ShutdownReport::already_finished());
}
if !closed {
self.reader_task.abort();
return self.terminate_then_kill(child).await;
return self.terminate_then_kill().await;
}
let child = self.child.as_mut().expect("checked child");
if let Ok(Some(status)) = child.try_wait() {
self.reader_task.abort();
self.stderr_task.abort();
return Ok(ShutdownReport {
exit_status: Some(status),
terminated: false,
killed: false,
});
return Ok(self.finish_shutdown(status, false, false));
}
match timeout_at(deadline, child.wait()).await {
Ok(Ok(status)) => {
self.reader_task.abort();
self.stderr_task.abort();
Ok(ShutdownReport {
exit_status: Some(status),
terminated: false,
killed: false,
})
}
Ok(Ok(status)) => Ok(self.finish_shutdown(status, false, false)),
Ok(Err(err)) => Err(McpClientError::new(
&self.server_name,
McpPhase::Shutdown,
McpErrorKind::Io(self.redactor.redact(&err.to_string())),
)
.with_diagnostics(self.snapshot_diagnostics().await)),
Err(_) => self.terminate_then_kill(child).await,
Err(_) => self.terminate_then_kill().await,
}
}
async fn terminate_then_kill(
fn finish_shutdown(
&mut self,
mut child: Child,
) -> Result<ShutdownReport, McpClientError> {
let mut terminated = false;
let mut killed = false;
if send_terminate(&mut child).is_ok() {
terminated = true;
status: ExitStatus,
terminated: bool,
killed: bool,
) -> ShutdownReport {
// No await separates confirmed reaping from releasing the handle.
drop(self.child.take());
self.reader_task.abort();
self.stderr_task.abort();
ShutdownReport {
exit_status: Some(status),
terminated,
killed,
}
}
async fn terminate_then_kill(&mut self) -> Result<ShutdownReport, McpClientError> {
let child = self.child.as_mut().expect("shutdown retains child");
let terminated = send_terminate(child).is_ok();
match timeout(self.limits.kill_timeout, child.wait()).await {
Ok(Ok(status)) => {
self.reader_task.abort();
self.stderr_task.abort();
Ok(ShutdownReport {
exit_status: Some(status),
terminated,
killed,
})
}
Ok(Ok(status)) => Ok(self.finish_shutdown(status, terminated, false)),
Ok(Err(err)) => Err(McpClientError::new(
&self.server_name,
McpPhase::Shutdown,
@@ -1163,7 +1204,6 @@ impl McpStdioClient {
McpErrorKind::Io(self.redactor.redact(&err.to_string())),
)
})?;
killed = true;
let status = timeout(self.limits.kill_timeout, child.wait())
.await
.map_err(|_| {
@@ -1183,13 +1223,7 @@ impl McpStdioClient {
McpErrorKind::Io(self.redactor.redact(&err.to_string())),
)
})?;
self.reader_task.abort();
self.stderr_task.abort();
Ok(ShutdownReport {
exit_status: Some(status),
terminated,
killed,
})
Ok(self.finish_shutdown(status, terminated, true))
}
}
}
@@ -1197,10 +1231,10 @@ impl McpStdioClient {
impl Drop for McpStdioClient {
fn drop(&mut self) {
if !self.shutdown_started {
if let Some(child) = &mut self.child {
let _ = child.start_kill();
}
// An attempted shutdown is not proof of exit. Errors and cancellation
// leave the child here so Drop must still make a best-effort kill.
if let Some(child) = &mut self.child {
let _ = child.start_kill();
}
self.reader_task.abort();
self.stderr_task.abort();
@@ -1784,3 +1818,70 @@ fn send_terminate(child: &mut Child) -> Result<(), ()> {
#[cfg(test)]
mod deadline_tests;
#[cfg(all(test, unix))]
mod shutdown_retention_tests {
//! The real client and OS process are used; only the external server is a
//! local shell that acknowledges initialization and ignores TERM/EOF.
use super::*;
use std::future::{Future, poll_fn};
use std::task::Poll;
#[tokio::test]
async fn cancelled_graceful_or_terminate_wait_retries_reaping_the_same_child() {
for cancel_in_fallback in [false, true] {
let script = r#"trap '' TERM
read init || exit 1
printf '%s\n' '{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","capabilities":{},"serverInfo":{"name":"retained","version":"1"}}}'
while :; do :; done
"#;
let mut client = McpStdioClient::connect(
McpStdioServerSpec::new("retained", "/bin/sh").args(["-c", script]),
McpStdioLimits {
shutdown_timeout: if cancel_in_fallback {
Duration::ZERO
} else {
Duration::from_secs(30)
},
kill_timeout: Duration::from_secs(30),
..Default::default()
},
)
.await
.expect("initialize after installing TERM handler");
let pid = client.child.as_ref().unwrap().id();
let mut shutdown = Box::pin(client.shutdown());
poll_fn(|cx| {
assert!(shutdown.as_mut().poll(cx).is_pending());
Poll::Ready(())
})
.await;
drop(shutdown);
assert_eq!(client.child.as_ref().and_then(Child::id), pid);
assert!(client.shutdown_started);
client.limits.shutdown_timeout = Duration::from_millis(20);
client.limits.kill_timeout = Duration::from_millis(50);
let report = timeout(Duration::from_secs(2), client.shutdown())
.await
.expect("retry bounded")
.expect("retry reaps original child");
assert!(
report.exit_status.is_some(),
"not an already-finished receipt"
);
assert!(report.killed, "TERM was ignored");
assert!(client.child.is_none());
let mut status = 0;
assert_eq!(
unsafe { libc::waitpid(pid.unwrap() as libc::pid_t, &mut status, libc::WNOHANG) },
-1
);
assert_eq!(
std::io::Error::last_os_error().raw_os_error(),
Some(libc::ECHILD),
"shutdown, not a later observer, reaped the original process"
);
}
}
}
+18 -11
View File
@@ -210,12 +210,7 @@ impl MergeRequestThreadEvent {
}
fn bound_bodies(&mut self) {
match self {
Self::Review(value) => {
truncate_body(&mut value.body);
for finding in &mut value.findings {
truncate_body(&mut finding.body);
}
}
Self::Review(_) => {}
Self::ReviewRevoked(value) => truncate_body(&mut value.reason),
Self::ReviewCancelled(value) => truncate_body(&mut value.reason),
Self::Comment(value) => truncate_body(&mut value.body),
@@ -939,10 +934,6 @@ impl MergeRequestStore {
&self,
i: SubmitMergeRequestReview,
) -> Result<ReviewEvent, MergeRequestError> {
bounded_body("review body", &i.body)?;
for finding in &i.findings {
bounded_body("review finding", &finding.body)?;
}
let mut c = self.lock()?;
let t = c.transaction()?;
let g: Option<(String, String, String, String, String, String)> = t
@@ -1747,7 +1738,23 @@ fn ticket_event(
) -> Result<(), MergeRequestError> {
let ticket = &mr.ticket_ids[0];
let n:i64=t.query_row("SELECT COALESCE(MAX(event_index),-1)+1 FROM typed_ticket_events WHERE workspace_id=?1 AND ticket_id=?2",params![mr.workspace_id,ticket],|r|r.get(0))?;
t.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,author,at,from_state,to_state,heading,body)VALUES(?1,?2,?3,'comment',?4,?5,NULL,NULL,'Merge Request integrated',?6)",params![mr.workspace_id,ticket,n,format!("worker:{}:{}",e.merged_by.runtime_id,e.merged_by.worker_id),e.created_at.to_rfc3339(),format!("Merge Request `{}` integrated approved source ref `{}`; Ticket remains in progress until guarded Ticket completion.",mr.merge_request_id,e.approved_source_ref)])?;
t.execute(
"INSERT INTO typed_ticket_events(
workspace_id,ticket_id,event_index,kind,author,at,from_state,to_state,heading,body,event_ref
) VALUES(?1,?2,?3,'comment',?4,?5,NULL,NULL,'Merge Request integrated',?6,?7)",
params![
mr.workspace_id,
ticket,
n,
format!("worker:{}:{}", e.merged_by.runtime_id, e.merged_by.worker_id),
e.created_at.to_rfc3339(),
format!(
"Merge Request `{}` integrated approved source ref `{}`; Ticket remains in progress until guarded Ticket completion.",
mr.merge_request_id, e.approved_source_ref
),
format!("{ticket}:{n}"),
],
)?;
for (k, v) in [
("implementation_assignment_id", a),
("merge_request_id", &mr.merge_request_id),
+159 -5
View File
@@ -53,8 +53,8 @@ fn fixture() -> (tempfile::TempDir, MergeRequestStore) {
let d = tempfile::tempdir().unwrap();
let p = d.path().join("db");
let c = Connection::open(&p).unwrap();
c.execute_batch("CREATE TABLE workspaces(workspace_id TEXT PRIMARY KEY);CREATE TABLE repositories(workspace_id TEXT,repository_id TEXT,PRIMARY KEY(workspace_id,repository_id));CREATE TABLE ticket_current_worker_assignments(workspace_id TEXT,ticket_id TEXT,assignment_id TEXT,runtime_id TEXT,worker_id TEXT,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_tickets(workspace_id TEXT,ticket_id TEXT,workflow_state TEXT,workflow_state_explicit INTEGER,updated_at TEXT,title TEXT NOT NULL,body TEXT NOT NULL,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_ticket_targets(workspace_id TEXT,ticket_id TEXT,ordinal INTEGER,repository_key TEXT,ref_selector TEXT,access TEXT);CREATE TABLE typed_ticket_events(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,kind TEXT,author TEXT,at TEXT,from_state TEXT,to_state TEXT,heading TEXT,body TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index));CREATE TABLE typed_ticket_event_attributes(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,key TEXT,value TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index,key));INSERT INTO workspaces VALUES('W');INSERT INTO repositories VALUES('W','R');INSERT INTO repositories VALUES('W','R2');INSERT INTO ticket_current_worker_assignments VALUES('W','T','A','runtime','coder','t');INSERT INTO typed_tickets VALUES('W','T','inprogress',1,'t','Title','Requirements');CREATE VIEW ticket_active_worker_assignments AS SELECT current.* FROM ticket_current_worker_assignments current JOIN typed_tickets ticket ON ticket.workspace_id=current.workspace_id AND ticket.ticket_id=current.ticket_id WHERE ticket.workflow_state NOT IN ('done','closed');").unwrap();
c.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body) VALUES('W','T',-1,'comment','Requirements')",[]).unwrap();
c.execute_batch("CREATE TABLE workspaces(workspace_id TEXT PRIMARY KEY);CREATE TABLE repositories(workspace_id TEXT,repository_id TEXT,PRIMARY KEY(workspace_id,repository_id));CREATE TABLE ticket_current_worker_assignments(workspace_id TEXT,ticket_id TEXT,assignment_id TEXT,runtime_id TEXT,worker_id TEXT,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_tickets(workspace_id TEXT,ticket_id TEXT,workflow_state TEXT,workflow_state_explicit INTEGER,updated_at TEXT,title TEXT NOT NULL,body TEXT NOT NULL,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_ticket_targets(workspace_id TEXT,ticket_id TEXT,ordinal INTEGER,repository_key TEXT,ref_selector TEXT,access TEXT);CREATE TABLE typed_ticket_events(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,kind TEXT,author TEXT,at TEXT,from_state TEXT,to_state TEXT,heading TEXT,body TEXT,event_ref TEXT NOT NULL DEFAULT '',PRIMARY KEY(workspace_id,ticket_id,event_index),UNIQUE(workspace_id,ticket_id,event_ref));CREATE TABLE typed_ticket_event_attributes(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,key TEXT,value TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index,key));INSERT INTO workspaces VALUES('W');INSERT INTO repositories VALUES('W','R');INSERT INTO repositories VALUES('W','R2');INSERT INTO ticket_current_worker_assignments VALUES('W','T','A','runtime','coder','t');INSERT INTO typed_tickets VALUES('W','T','inprogress',1,'t','Title','Requirements');CREATE VIEW ticket_active_worker_assignments AS SELECT current.* FROM ticket_current_worker_assignments current JOIN typed_tickets ticket ON ticket.workspace_id=current.workspace_id AND ticket.ticket_id=current.ticket_id WHERE ticket.workflow_state NOT IN ('done','closed');").unwrap();
c.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body,event_ref) VALUES('W','T',-1,'comment','Requirements','T:-1')",[]).unwrap();
drop(c);
let a = Assignments(Arc::new(Mutex::new(CurrentAssignment {
assignment_id: "A".into(),
@@ -376,6 +376,49 @@ fn approve_for(
fn approve(s: &MergeRequestStore, subject: &str, token: &str) -> ReviewEvent {
approve_for(s, "MR", "R", subject, token)
}
#[test]
fn review_submission_preserves_long_body_and_findings_on_reload() {
for (body, finding_body) in [
("b".repeat(16 * 1024 + 1), "finding".into()),
("review".into(), "指摘".repeat(4096)),
] {
let (_dir, store) = fixture();
open(&store);
request(&store, "source", "token");
let findings = vec![ReviewFinding {
severity: FindingSeverity::Major,
code: None,
path: None,
line: None,
body: finding_body,
}];
let submitted = store
.submit_review(SubmitMergeRequestReview {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
current_subject_ref: "source".into(),
capability_token: "token".into(),
decision: ReviewDecision::RequestChanges,
body: body.clone(),
findings: findings.clone(),
now: at(4),
})
.unwrap();
assert_eq!(submitted.body, body);
assert_eq!(submitted.findings, findings);
let reloaded = store.get_by_id("W", "MR").unwrap();
let saved = reloaded
.thread
.iter()
.find_map(|event| match event {
MergeRequestThreadEvent::Review(review) => Some(review),
_ => None,
})
.unwrap();
assert_eq!(saved, &submitted);
}
}
#[test]
fn review_submission_authorization_rejects_invalid_grants_before_side_effects() {
let (_d, store) = fixture();
@@ -457,6 +500,18 @@ fn integration_preserves_selectors_ticket_state_and_assignment_and_replays_its_m
.unwrap();
assert_eq!(replayed, merged);
let connection = Connection::open(d.path().join("db")).unwrap();
let ticket_refs = connection
.prepare("SELECT event_ref FROM typed_ticket_events WHERE event_index >= 0")
.unwrap()
.query_map([], |row| row.get::<_, String>(0))
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
ticket_refs,
vec!["T:0"],
"replay must not append another post"
);
let (state, active): (String, bool) = connection.query_row(
"SELECT workflow_state, EXISTS(SELECT 1 FROM ticket_active_worker_assignments WHERE workspace_id='W' AND ticket_id='T') FROM typed_tickets WHERE workspace_id='W' AND ticket_id='T'",
[],
@@ -477,6 +532,87 @@ fn integration_preserves_selectors_ticket_state_and_assignment_and_replays_its_m
assert!(!json.contains(banned), "{banned} in {json}")
}
}
#[test]
fn integration_post_failure_rolls_back_and_same_operation_recovers_with_legacy_empty_ref() {
let (dir, store) = fixture();
open(&store);
let review = approve(&store, "source", "approved");
request(&store, "source", "pending");
let completion = CompleteMergeRequest {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
operation_id: "recover-integration".into(),
approval_event_id: review.event_id,
current_subject_ref: "source".into(),
target_ref_before: "before".into(),
target_ref_after: "after".into(),
strategy: MergeStrategy::FastForward,
resolution: ConflictResolution::None,
auth: auth(),
now: at(5),
};
let connection = Connection::open(dir.path().join("db")).unwrap();
// Retain a comment written before integration populated event_ref.
connection.execute_batch(
"INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body,event_ref)
VALUES('W','T',0,'comment','Legacy integration','');
CREATE TRIGGER reject_integration_post BEFORE INSERT ON typed_ticket_events
WHEN NEW.heading='Merge Request integrated'
BEGIN SELECT RAISE(ABORT,'injected integration post failure'); END;",
).unwrap();
let before = store.get_by_id("W", "MR").unwrap();
let error = store.complete(completion.clone()).unwrap_err();
assert!(
error
.to_string()
.contains("injected integration post failure")
);
let failed = store.get_by_id("W", "MR").unwrap();
assert_eq!(failed.state, MergeRequestState::Open);
assert_eq!(failed.thread, before.thread);
assert!(store.authorize_review_submission("MR", "pending").is_ok());
assert_eq!(
connection
.query_row(
"SELECT COUNT(*) FROM typed_ticket_event_attributes",
[],
|row| row.get::<_, i64>(0),
)
.unwrap(),
0
);
connection
.execute_batch("DROP TRIGGER reject_integration_post")
.unwrap();
let merged = store.complete(completion.clone()).unwrap();
assert_eq!(store.complete(completion).unwrap(), merged);
assert_eq!(
store.get_by_id("W", "MR").unwrap().state,
MergeRequestState::Merged
);
assert!(store.authorize_review_submission("MR", "pending").is_err());
let posts = connection
.prepare("SELECT event_ref,body FROM typed_ticket_events WHERE event_index >= 0 ORDER BY event_index")
.unwrap()
.query_map([], |row| Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?)))
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
posts.len(),
2,
"retry and replay must append exactly one post"
);
assert_eq!(posts[0], (String::new(), "Legacy integration".into()));
assert_eq!(posts[1].0, "T:1");
assert!(posts[1].1.contains("`MR`"));
assert_eq!(connection.query_row(
"SELECT value FROM typed_ticket_event_attributes WHERE event_index=1 AND key='operation_id'",
[], |row| row.get::<_, String>(0),
).unwrap(), "recover-integration");
}
#[test]
fn source_move_cancels_submission_and_old_approval_is_reusable_when_source_returns() {
let (_d, s) = fixture();
@@ -955,7 +1091,7 @@ fn ticket_rescope_requires_fresh_review_before_merge_and_remains_recoverable() {
let connection = Connection::open(dir.path().join("db")).unwrap();
connection
.execute_batch(
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL)",
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL,'T:1')",
)
.unwrap();
connection
@@ -1082,7 +1218,7 @@ fn ticket_rescope_after_integration_accepts_fresh_requirement_attestation() {
let connection = Connection::open(dir.path().join("db")).unwrap();
connection
.execute_batch(
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL)",
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL,'T:1')",
)
.unwrap();
connection
@@ -1307,6 +1443,24 @@ fn partial_and_full_integration_retain_ticket_state_and_active_assignment() {
assert_eq!(second_request.merged_result().unwrap(), &second_merge);
assert_eq!(first_request.integration_approval().unwrap(), &first);
assert_eq!(second_request.integration_approval().unwrap(), &second);
let posts = connection
.prepare(
"SELECT event.event_ref, attribute.value
FROM typed_ticket_events event
JOIN typed_ticket_event_attributes attribute USING(workspace_id,ticket_id,event_index)
WHERE attribute.key='merge_request_id' ORDER BY event.event_index",
)
.unwrap()
.query_map([], |row| {
Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?))
})
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
posts,
vec![("T:0".into(), "MR".into()), ("T:1".into(), "MR-2".into())]
);
}
#[test]
@@ -1324,7 +1478,7 @@ fn historical_ticket_completion_payload_remains_readable_and_unchanged_on_migrat
}"#;
connection.execute_batch(
"UPDATE typed_tickets SET workflow_state='done' WHERE workspace_id='W' AND ticket_id='T';
INSERT INTO typed_ticket_events VALUES('W','T',0,'state_changed','worker:runtime:coder','2026-07-26T12:00:07Z','inprogress','done','Ticket requirements completed','historical body');"
INSERT INTO typed_ticket_events VALUES('W','T',0,'state_changed','worker:runtime:coder','2026-07-26T12:00:07Z','inprogress','done','Ticket requirements completed','historical body','T:0');"
).unwrap();
connection
.execute(
+27 -26
View File
@@ -52,7 +52,7 @@ pub enum AuthenticatedInputSource {
/// Assigned whenever a serialized tracked method crosses an untrusted
/// protocol boundary. Receivers must handle it exactly like public input.
UntrustedWire,
/// Stamped only by the operator-owned standalone in-process transport.
/// Stamped only by the operator-owned Standalone Host after local admission.
/// This is not a Backend Account or authority accepted from wire fields.
LocalOperator,
Account {
@@ -399,16 +399,13 @@ pub enum Method {
/// Typed lifecycle events sent from a child Worker to its parent.
///
/// Delivered as `Method::WorkerEvent` over the parent's Unix socket. The
/// parent Controller always applies variant-specific side effects
/// (registry / worker-allocation updates). Agent-visible variants are also
/// queued into the notification buffer; control-plane-only variants are
/// not injected into the parent's LLM context.
/// Delivered in-process to the parent Controller's protocol intake. Agent-visible variants
/// are queued into the notification buffer. Reports are advisory: receivers must check the
/// actual child session before treating its work as complete.
///
/// Transport is fire-and-forget; receivers must tolerate out-of-order
/// delivery (e.g. `TurnEnded` arriving after `ShutDown` for the same
/// child Worker).
#[derive(Debug, Clone, Serialize, Deserialize)]
/// Retained legacy control-only variants can be decoded from saved history, but do not
/// register live children, delegate scope, or establish a socket callback.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
@@ -429,16 +426,8 @@ pub enum WorkerEvent {
/// Child has stopped (controller loop is exiting).
ShutDown { worker_name: String },
/// Child SubWorker sub-delegated scope to a grandchild SubWorker via `SubWorkerSpawn`.
///
/// Control-plane only: receivers apply registry side effects and
/// propagate upward, but do not expose this as an agent notification.
///
/// The parent uses this to add the grandchild to its own
/// `spawned_workers.json` so it can manage the grandchild directly
/// even if the intermediate child dies. The parent then re-fires
/// this event upward (if it has a parent of its own) to maintain
/// the chain to root.
/// Legacy process-child scope report retained for saved Session decoding only.
/// Live Controllers ignore it; only the Host-owned Internal registry grants child scope.
ScopeSubDelegated {
/// Sub-delegating Worker (= the sender itself).
parent_worker: String,
@@ -982,6 +971,17 @@ pub enum SessionEntryProvenance {
LegacyUnknown,
}
/// Public use decisions contain no Feature identity or private payload. Absence
/// denotes the older projection contract, not permission to expose System input.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(deny_unknown_fields)]
pub struct SessionEntryUsePolicy {
pub observation_readable: bool,
pub memory_evidence: bool,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -994,6 +994,8 @@ pub struct SessionSnapshotEntry {
pub provenance: SessionEntryProvenance,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub derived_from: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub use_policy: Option<SessionEntryUsePolicy>,
#[serde(flatten)]
pub data: SessionSnapshotEntryData,
}
@@ -1180,12 +1182,10 @@ pub enum Event {
},
/// One agent-injected system item committed to history.
///
/// Carries the JSON form of `session_store::SystemItem`. Covers
/// `Method::Notify` echoes, child-Worker lifecycle events from
/// `Method::WorkerEvent`, `@<path>` resolution payloads, and any future
/// Clients dispatch on the `kind` tag for typed rendering instead
/// of parsing free-text prefixes like `[Notification] …` or
/// `[File: …]`.
/// Carries the allowlisted JSON form of `session_store::PublicSystemItem`,
/// not the storage record. Its descriptive `kind` is independent from the
/// typed `presentation` used for notification, reminder, or context rendering.
/// Private Feature payloads and prompt provenance are never event data.
///
/// One event per `LogEntry::AnnotatedSystemItem` commit. Disk-side and
/// wire-side are 1:1.
@@ -2621,6 +2621,7 @@ mod tests {
session: SessionSnapshot {
pending_submissions: PendingSubmissionsSnapshot::default(),
entries: vec![SessionSnapshotEntry {
use_policy: None,
entry_id: "entry-1".into(),
timestamp: 1,
provenance: SessionEntryProvenance::HumanInput,
+7 -6
View File
@@ -17,12 +17,12 @@ use crate::{
PasteArtifactRef, PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission,
ReasoningConfig, RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason,
ScopeRule, Segment, SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment, SubmissionDisposition,
SymlinkPolicy, ToolResultDisposition, TurnResult, UploadedFileAvailability, UploadedFileRef,
WorkerBusyState, WorkerCommandAcknowledgement, WorkerCommandDisposition, WorkerCommandEnvelope,
WorkerCommandKind, WorkerEvent, WorkerMaintenanceState, WorkerRunState, WorkerState,
WorkerStateSnapshot, WorkerStatus,
SessionEntryUsePolicy, SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole,
SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment,
SubmissionDisposition, SymlinkPolicy, ToolResultDisposition, TurnResult,
UploadedFileAvailability, UploadedFileRef, WorkerBusyState, WorkerCommandAcknowledgement,
WorkerCommandDisposition, WorkerCommandEnvelope, WorkerCommandKind, WorkerEvent,
WorkerMaintenanceState, WorkerRunState, WorkerState, WorkerStateSnapshot, WorkerStatus,
subscription::{
EventSubscriptionSelector, MAX_CORRELATION_ID_BYTES, MAX_REJECTION_MESSAGE_BYTES,
MAX_RESOURCE_ID_BYTES, MAX_SUBSCRIPTION_COLLECTION_ITEMS,
@@ -117,6 +117,7 @@ pub fn generated_protocol_types() -> String {
push_decl::<InFlightCompaction>(&cfg, &mut output);
push_decl::<InFlightSnapshot>(&cfg, &mut output);
push_decl::<SessionEntryProvenance>(&cfg, &mut output);
push_decl::<SessionEntryUsePolicy>(&cfg, &mut output);
push_decl::<SessionMessageRole>(&cfg, &mut output);
push_decl::<SessionContentPart>(&cfg, &mut output);
push_decl::<SessionToolAttachment>(&cfg, &mut output);
+1
View File
@@ -9,6 +9,7 @@ license.workspace = true
api-macros = { workspace = true, features = ["reqwest", "axum"] }
manifest.workspace = true
protocol.workspace = true
session-store.workspace = true
serde = { workspace = true, features = ["derive"] }
workdir.workspace = true
+121 -7
View File
@@ -86,6 +86,8 @@ pub enum WorkerRestoreState {
Accepted,
Rejected,
RolledBack,
/// A later Stop superseded this restore; neither rollback nor prior Accepted is asserted.
Superseded,
ReconciliationRequired,
}
@@ -518,9 +520,10 @@ pub struct WorkerRestorePreparation {
pub workspace_api: Option<WorkspaceApiRef>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workdir_attachments: Option<Vec<LogicalWorkdirAttachment>>,
/// Retained in saved Restore intents; Restore does not apply Repository access.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access: Vec<WorkingDirectoryRepositoryAccessRequest>,
/// Read-only references resolved by Workspace only after this operation owns admission.
/// Retained references from saved resource preparation, not Restore prerequisites.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access_workdirs: Vec<RepositoryAccessWorkdirReference>,
}
@@ -835,6 +838,34 @@ pub struct SessionMaterialPinResponse {
pub pin_id: String,
}
/// One body-bearing immutable slice. The byte budget includes slice metadata,
/// but not the enclosing availability enum or the caller's durable Job envelope.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionRecordingCaptureRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub start: session_store::capture::RecordingPosition,
pub max_entries: usize,
pub max_serialized_bytes: usize,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionRecordingCaptureAvailability {
NotMaterialized {
session_id: String,
},
Captured {
slice: session_store::capture::RecordingCaptureSlice,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialCatalogRequest {
@@ -1140,8 +1171,36 @@ pub struct SubjectReleaseResponse {
pub state: SubjectReleaseState,
}
/// Irreversibly fence an unpublished failed Subject create. Exact retries must retain
/// all four fields; this is distinct from releasing an existing Subject Controller.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryRequest {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
/// Durable admission-fence proof plus freshly confirmed execution absence.
/// An error (including a lost response) is never evidence of absence.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryResponse {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
#[api(reqwest, axum)]
pub trait RuntimeApi {
#[post("/v1/subject-create-recovery", status = 200, error_status = 400)]
async fn recover_subject_create(
&self,
#[body] request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError>;
#[post("/v1/subject-release", status = 200, error_status = 400)]
async fn subject_release(
&self,
@@ -1217,6 +1276,16 @@ pub trait RuntimeApi {
#[body] request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError>;
#[post(
"/v1/session-public/recording-capture",
status = 200,
error_status = 400
)]
async fn session_recording_capture(
&self,
#[body] request: SessionRecordingCaptureRequest,
) -> Result<SessionRecordingCaptureAvailability, RuntimeApiError>;
#[post(
"/v1/session-public/material/catalog",
status = 200,
@@ -1511,13 +1580,24 @@ mod tests {
)
.unwrap();
assert!(request.preparation.is_none());
for state in [
"accepted",
"rejected",
"rolled_back",
"reconciliation_required",
for (state, wire) in [
(WorkerRestoreState::Accepted, "accepted"),
(WorkerRestoreState::Rejected, "rejected"),
(WorkerRestoreState::RolledBack, "rolled_back"),
(WorkerRestoreState::Superseded, "superseded"),
(
WorkerRestoreState::ReconciliationRequired,
"reconciliation_required",
),
] {
assert!(serde_json::from_value::<WorkerRestoreState>(serde_json::json!(state)).is_ok());
assert_eq!(
serde_json::from_value::<WorkerRestoreState>(serde_json::json!(wire)).unwrap(),
state
);
assert_eq!(
serde_json::to_value(state).unwrap(),
serde_json::json!(wire)
);
}
assert!(
serde_json::from_value::<WorkerRestoreState>(serde_json::json!(
@@ -1675,6 +1755,18 @@ mod tests {
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn recover_subject_create(
&self,
request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError> {
Ok(SubjectCreateRecoveryResponse {
workspace_id: request.workspace_id,
worker_id: request.worker_id,
create_fingerprint: request.create_fingerprint,
operation_id: request.operation_id,
})
}
async fn subject_release(
&self,
_request: SubjectReleaseRequest,
@@ -1689,6 +1781,13 @@ mod tests {
Err(test_error(501))
}
async fn session_recording_capture(
&self,
_request: SessionRecordingCaptureRequest,
) -> Result<SessionRecordingCaptureAvailability, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_catalog(
&self,
_request: SessionMaterialCatalogRequest,
@@ -1820,6 +1919,21 @@ mod tests {
.build()
.unwrap();
let recovery = SubjectCreateRecoveryRequest {
workspace_id: "workspace-a".into(),
worker_id: WorkerId::now_v7(),
create_fingerprint: "failed-create-fingerprint".into(),
operation_id: "recovery-1".into(),
};
let response = client
.recover_subject_create(recovery.clone())
.await
.unwrap();
assert_eq!(response.workspace_id, recovery.workspace_id);
assert_eq!(response.worker_id, recovery.worker_id);
assert_eq!(response.create_fingerprint, recovery.create_fingerprint);
assert_eq!(response.operation_id, recovery.operation_id);
let summary = client.runtime_summary().await.unwrap();
assert_eq!(summary.runtime.display_name.as_deref(), Some("round-trip"));
let workers = client
+475
View File
@@ -0,0 +1,475 @@
//! Neutral explicit-allow authorization contracts. Resource containment never crosses kinds.
//!
//! # Trusted Feature boundary
//!
//! A Profile requests execution configuration, not authority. The Worker asks the
//! Host for each enabled Feature's exact activation name before initialization,
//! then trusted Feature code asks for each protected operation and its semantic
//! resource before access or effects. `ticket` on [`PermissionResource::Feature`]
//! activates Ticket; it implies neither `ticket.create` nor any future operation.
//! `ticket.create` on a Workspace permits creation there but cannot activate the
//! Feature. There are no wildcard permissions or implicit dotted-name inheritance.
//!
//! Identity is absent from the authorization request intentionally: the Host
//! authenticates the bound Runtime/Worker and resolves current source permissions,
//! body ownership and any per-Worker execution grant. A shared client, Profile,
//! remembered receipt or caller-supplied Subject claim is not authority. Features
//! depend on this contract, never on a particular Subject policy implementation.
//!
//! Supported cooperative boundaries are Feature activation, ordinary Worker
//! delegation, Ticket creation (Tool/native WIP/typed service), Workdir filesystem
//! read/write and whole-attachment command execution, and scoped result notify.
//! Workdir providers still enforce resolved paths, traversal, capability, Ticket
//! and provider ceilings. The returned scope is for provider narrowing, not a
//! transferable credential. Scoped checkout/overlap operations are refused until
//! their complete access semantics are adapted. Other Feature operations retain
//! their existing owner checks; they do not automatically acquire this operation
//! policy. Missing calls are trusted Feature bugs, not interception by a universal
//! Tool executor or protection against malicious Feature code. Connection tests
//! must exercise each exposed surface, assert a genuine permitted effect and
//! assert zero downstream calls on denial, including revocation and provider denial.
//!
//! # Delegation callers and recovery
//!
//! Callers such as a chat Feature supply explicit operation/resource selections
//! and retain the same public operation ID and full request after an unknown
//! result. Normal Profile/Worker allocation and control/resource grants are reused;
//! there is no Subject Worker registry. Effective grants bind immutable Profile
//! resolution, the reserved Worker and source assignment. Permission replacement
//! invalidates existing children rather than refreshing them. Body replacement,
//! retirement and restore must recheck current authority; historical result lookup
//! is not new execution permission. Cancellation acceptance is distinct from
//! terminal confirmation and does not undo completed effects.
//!
//! A child receives no Subject body identity, singleton, whole Memory or public
//! conversation-post authority. `worker.result` is explicit notify authority to a
//! named origin Worker, not arbitrary control or publication. Chat delivery and
//! conversation correlation belong to the caller integration, not this contract.
//! Unset existing Subject policies deny new Subject execution/delegation; ordinary
//! non-Subject Workers explicitly retain existing Backend/provider contracts.
//! Standalone non-Workspace activation compatibility grants no Workspace operation.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
pub const MAX_SCOPED_PERMISSIONS: usize = 256;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PermissionResource {
Feature,
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
// Serde's internally tagged unit variants ignore extra fields even with deny_unknown_fields.
// A struct-shaped wire variant preserves the public unit variant while rejecting those claims.
impl<'de> Deserialize<'de> for PermissionResource {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
#[derive(Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
enum Wire {
Feature {},
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
Ok(match Wire::deserialize(deserializer)? {
Wire::Feature {} => Self::Feature,
Wire::Workspace { workspace_id } => Self::Workspace { workspace_id },
Wire::Workdir {
working_directory_id,
path,
recursive,
} => Self::Workdir {
working_directory_id,
path,
recursive,
},
Wire::Worker {
runtime_id,
worker_id,
} => Self::Worker {
runtime_id,
worker_id,
},
})
}
}
impl PermissionResource {
pub fn validate(&self) -> Result<(), String> {
match self {
Self::Feature => Ok(()),
Self::Workspace { workspace_id } => validate_authorization_identifier(workspace_id),
Self::Workdir {
working_directory_id,
path,
..
} => {
validate_authorization_identifier(working_directory_id)?;
validate_relative_permission_path(path)
}
Self::Worker {
runtime_id,
worker_id,
} => {
validate_authorization_identifier(runtime_id)?;
validate_authorization_identifier(worker_id)
}
}
}
pub fn covers(&self, other: &Self) -> bool {
if self.validate().is_err() || other.validate().is_err() {
return false;
}
match (self, other) {
(Self::Feature, Self::Feature) => true,
(Self::Workspace { workspace_id: a }, Self::Workspace { workspace_id: b }) => a == b,
(
Self::Worker {
runtime_id: ar,
worker_id: aw,
},
Self::Worker {
runtime_id: br,
worker_id: bw,
},
) => ar == br && aw == bw,
(
Self::Workdir {
working_directory_id: a,
path: ap,
recursive: ar,
},
Self::Workdir {
working_directory_id: b,
path: bp,
recursive: br,
},
) => {
a == b
&& if *ar {
ap == "."
|| ap == bp
|| bp
.strip_prefix(ap)
.is_some_and(|tail| tail.starts_with('/'))
} else {
ap == bp && !br
}
}
_ => false,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ScopedPermission {
pub operation: String,
pub resource: PermissionResource,
}
impl ScopedPermission {
pub fn validate(&self) -> Result<(), String> {
validate_operation_name(&self.operation)?;
self.resource.validate()?;
if matches!(self.resource, PermissionResource::Feature) && self.operation.contains('.') {
return Err("Feature resources authorize activation names only, not operations".into());
}
Ok(())
}
pub fn covers(&self, other: &Self) -> bool {
self.validate().is_ok()
&& other.validate().is_ok()
&& self.operation == other.operation
&& self.resource.covers(&other.resource)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissions {
pub change_id: String,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsUpdate {
pub operation_id: String,
pub expected_change_id: Option<String>,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerExecutionGrant {
pub subject_id: String,
pub origin_runtime_id: String,
pub origin_worker_id: String,
pub source_change_id: String,
pub runtime_id: String,
pub worker_id: String,
pub profile: String,
pub profile_digest: String,
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerDelegationRequest {
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
impl WorkerDelegationRequest {
pub fn validate(&self) -> Result<(), String> {
validate_scoped_permissions(&self.permissions)?;
validate_scoped_permissions(&self.redelegation)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationRequest {
pub operation: String,
pub resource: PermissionResource,
}
impl OperationAuthorizationRequest {
pub fn validate(&self) -> Result<(), String> {
ScopedPermission {
operation: self.operation.clone(),
resource: self.resource.clone(),
}
.validate()
}
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationResponse {
/// Matching granted scope for provider-resolved checks, not a credential.
/// `None` identifies the explicit non-Subject legacy authorization path.
#[serde(default)]
pub scope: Option<PermissionResource>,
}
pub fn validate_operation_name(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| !value.split('.').all(|part| {
!part.is_empty()
&& part
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'_' || b == b'-')
})
{
return Err(
"permission operation must be an exact, nonempty dot-separated name (no wildcards)"
.into(),
);
}
Ok(())
}
pub fn validate_scoped_permissions(permissions: &[ScopedPermission]) -> Result<(), String> {
if permissions.len() > MAX_SCOPED_PERMISSIONS {
return Err("too many scoped permissions".into());
}
for permission in permissions {
permission.validate()?;
}
Ok(())
}
pub fn validate_authorization_identifier(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| value.trim() != value
|| value.chars().any(|c| c.is_control() || c == '*')
{
return Err("authorization identity must be nonempty, bounded and exact".into());
}
Ok(())
}
/// Paths are already-normalized lexical paths, not host filesystem paths. The Host must
/// still enforce symlink/provider boundaries; lexical containment grants no filesystem access.
pub fn validate_relative_permission_path(path: &str) -> Result<(), String> {
if path == "." {
return Ok(());
}
if path.is_empty()
|| path.len() > 4096
|| path.contains(['\\', ':'])
|| path.chars().any(char::is_control)
|| path
.split('/')
.any(|part| part.is_empty() || part == "." || part == "..")
{
return Err("permission path must be a normalized relative path without escape".into());
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn workdir(path: &str, recursive: bool) -> PermissionResource {
PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
}
}
#[test]
fn lexical_containment_rejects_escape_and_respects_component_and_recursion_boundaries() {
for path in [
"",
"/src",
"../src",
"src/../secret",
"src/./a",
"src//a",
"src/",
"C:/src",
"src\\a",
] {
assert!(workdir(path, true).validate().is_err(), "{path}");
}
let root = workdir(".", true);
let src = workdir("src", true);
assert!(root.covers(&src));
assert!(src.covers(&workdir("src/a", true)));
assert!(!src.covers(&workdir("src-other", false)));
assert!(!workdir("src", false).covers(&src));
assert!(workdir("src", false).covers(&workdir("src", false)));
assert!(!workdir("src", false).covers(&workdir("src/a", false)));
assert!(!src.covers(&PermissionResource::Workdir {
working_directory_id: "other".into(),
path: "src".into(),
recursive: false
}));
}
#[test]
fn operation_hierarchy_never_implies_inheritance_or_feature_activation() {
for name in [
"",
"*",
"ticket.*",
".ticket",
"ticket.",
"ticket..read",
"ticket read",
] {
assert!(validate_operation_name(name).is_err(), "{name}");
}
let activation = ScopedPermission {
operation: "ticket".into(),
resource: PermissionResource::Feature,
};
let create = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "ws".into(),
},
};
let parent_name = ScopedPermission {
operation: "ticket".into(),
resource: create.resource.clone(),
};
assert!(!parent_name.covers(&create));
assert!(!create.covers(&parent_name));
assert!(!activation.covers(&create));
assert!(!create.covers(&activation));
assert!(
ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Feature
}
.validate()
.is_err()
);
assert!(
!PermissionResource::Workspace {
workspace_id: "ws".into()
}
.covers(&workdir(".", true))
);
}
#[test]
fn wire_contract_rejects_self_claimed_identity_and_unknown_fields() {
let request = serde_json::json!({"permissions":[],"redelegation":[],"subject_id":"forged"});
assert!(serde_json::from_value::<WorkerDelegationRequest>(request).is_err());
let resource: PermissionResource =
serde_json::from_value(serde_json::json!({"kind":"feature"})).unwrap();
assert_eq!(resource, PermissionResource::Feature);
assert!(
serde_json::from_value::<PermissionResource>(
serde_json::json!({"kind":"feature","workspace_id":"ws"})
)
.is_err()
);
}
#[test]
fn authorization_response_carries_matching_scope_and_defaults_legacy_absence() {
let legacy: OperationAuthorizationResponse =
serde_json::from_value(serde_json::json!({})).unwrap();
assert_eq!(legacy.scope, None);
assert_eq!(
serde_json::to_value(legacy).unwrap(),
serde_json::json!({"scope":null})
);
let granted = OperationAuthorizationResponse {
scope: Some(workdir("src", true)),
};
let wire = serde_json::to_value(&granted).unwrap();
assert_eq!(
wire,
serde_json::json!({"scope":{"kind":"workdir","working_directory_id":"wd","path":"src","recursive":true}})
);
assert_eq!(
serde_json::from_value::<OperationAuthorizationResponse>(wire).unwrap(),
granted
);
}
}
+10
View File
@@ -134,6 +134,14 @@ pub enum DriveEntryKind {
File,
}
/// Current caller's advisory publication ceiling, never a grant or execution credential.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveCapabilities {
pub write: bool,
}
/// Metadata only: content is returned through bounded text or binary routes.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -491,6 +499,7 @@ impl api_macros::HttpRequestError for DriveApiError {
impl api_macros::openapi::OpenApiSchema for DriveApiError {}
impl api_macros::openapi::OpenApiSchema for DriveEntry {}
impl api_macros::openapi::OpenApiSchema for DriveCapabilities {}
impl api_macros::openapi::OpenApiSchema for DriveEntryQuery {}
impl api_macros::openapi::OpenApiSchema for DriveListQuery {}
impl api_macros::openapi::OpenApiSchema for DriveSearchQuery {}
@@ -518,6 +527,7 @@ pub fn drive_api_typescript() -> String {
DriveEntryRef::decl(&config),
DriveEntryKind::decl(&config),
DriveEntry::decl(&config),
DriveCapabilities::decl(&config),
DriveEntryQuery::decl(&config),
DriveListQuery::decl(&config),
DriveSearchQuery::decl(&config),
+349 -10
View File
@@ -6,6 +6,7 @@ use api_macros::api;
pub use api_macros::axum as server_support;
pub use api_macros::reqwest as client_support;
pub use api_macros::{ApiContract, BinaryBody, HttpMethod, TransportMetadata, WebSocketOperation};
pub mod authorization;
mod drive;
mod notification;
pub use notification::*;
@@ -23,6 +24,34 @@ use webauthn_rs_proto::{
};
pub use workspace_config::*;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsResponse {
pub permissions: Option<authorization::SubjectPermissions>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsMutationResponse {
pub permissions: authorization::SubjectPermissions,
/// New operations are already fenced. These are cancellation requests and
/// Backend command cleanup evidence, not proof that a Runtime turn stopped.
pub interruptions: Vec<SubjectPermissionInterruption>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionInterruption {
pub runtime_id: String,
pub worker_id: String,
pub cancellation_accepted: bool,
pub backend_command_sessions_closed: bool,
pub diagnostics: Vec<String>,
}
#[allow(dead_code)]
#[derive(JsonSchema)]
struct ServerJsonSafeU64(#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))] u64);
@@ -96,6 +125,8 @@ pub struct WorkdirOperationApiError {
pub message: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub diagnostics: Option<Vec<Diagnostic>>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub audit_failures: Vec<workdir::http::WorkdirAuditFailure>,
#[serde(skip, default = "default_repository_error_status")]
status: u16,
/// Server-local metadata retained by generated error response extensions, never the wire body.
@@ -103,6 +134,8 @@ pub struct WorkdirOperationApiError {
pub workdir_log: Option<WorkdirOperationLogContext>,
#[serde(skip)]
pub denial_reason: Option<workdir::WorkdirDenialReason>,
#[serde(skip)]
pub failure_reason: Option<workdir::WorkdirFailureReason>,
}
/// Internal operational correlation. Producers supply fixed-length ID fingerprints and
@@ -125,9 +158,11 @@ impl WorkdirOperationApiError {
code: None,
message: error.message,
diagnostics: Some(error.diagnostics),
audit_failures: Vec::new(),
status: error.status,
workdir_log: None,
denial_reason: None,
failure_reason: None,
}
}
@@ -138,9 +173,11 @@ impl WorkdirOperationApiError {
code: Some(error.code),
message: error.message,
diagnostics: None,
audit_failures: Vec::new(),
status,
workdir_log: None,
denial_reason: error.denial_reason,
failure_reason: error.failure_reason,
}
}
}
@@ -168,6 +205,11 @@ macro_rules! impl_openapi_schema {
}
impl_openapi_schema!(
SubjectPermissionsResponse,
SubjectPermissionsMutationResponse,
SubjectPermissionInterruption,
authorization::SubjectPermissions,
authorization::SubjectPermissionsUpdate,
AuthBootstrapUserRequest,
AuthPublicConfig,
HostingSettings,
@@ -187,6 +229,10 @@ impl_openapi_schema!(
PasskeyLoginOptionsResponse,
PasskeyLoginCompleteRequest,
PasskeyAdditionOptionsRequest,
PasskeySummary,
PasskeyListResponse,
PasskeyAdditionResultRequest,
PasskeyAdditionResultResponse,
PasskeyRegistrationOptionsRequest,
PasskeyRegistrationOptionsResponse,
PasskeyRegistrationCompleteRequest,
@@ -510,8 +556,9 @@ pub struct WorkspaceWorkerSessionHistoryResponse {
/// comparisons do not replace admission checks under the Runtime operation lock; conflicts do
/// not trigger automatic token replacement.
/// [`WorkerRestoreState`] Accepted means Restore committed, not that the subsequent run succeeded.
/// Distinguish read-only preflight Rejected, confirmed RolledBack after side effects, and
/// ReconciliationRequired needing settled-state checks. Response loss does not imply rollback.
/// Distinguish read-only preflight Rejected, confirmed RolledBack after side effects,
/// Superseded by a later Stop, and ReconciliationRequired needing settled-state checks.
/// Response loss does not imply rollback.
#[api(reqwest, axum, openapi)]
pub trait ServerApi {
#[get("/health", status = 200, error_status = 400)]
@@ -610,6 +657,33 @@ pub trait ServerApi {
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthPasskeyRegistrationComplete, RepositoryApiError>;
#[get(
"/api/auth/passkeys",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_list(
&self,
#[extension] context: ServerRequestContext,
) -> Result<PasskeyListResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/result",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_result(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyAdditionResultRequest,
) -> Result<PasskeyAdditionResultResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/options",
status = 200,
@@ -1298,6 +1372,51 @@ pub trait ServerApi {
#[body] request: MemoryConsolidateStagingRequest,
) -> Result<MemoryConsolidationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/authorization/check",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
openapi = false,
)]
async fn operation_authorize(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: authorization::OperationAuthorizationRequest,
) -> Result<authorization::OperationAuthorizationResponse, RepositoryApiError>;
#[get(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_get(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
) -> Result<SubjectPermissionsResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_set(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
#[body] request: authorization::SubjectPermissionsUpdate,
) -> Result<SubjectPermissionsMutationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects",
status = 201,
@@ -2762,6 +2881,12 @@ pub trait ServerApi {
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveEntry, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/capabilities", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_capabilities(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveCapabilities, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/metadata", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_metadata(
&self,
@@ -4025,6 +4150,19 @@ impl RequestActor {
#[serde(deny_unknown_fields)]
pub struct WhoamiResponse {
pub actor: Option<RequestActor>,
pub session: BrowserSessionState,
/// Non-secret comparison evidence, never an authentication credential.
pub authentication_reference: Option<String>,
}
/// Browser navigation evidence, not an authorization grant or a generic 401 classification.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum BrowserSessionState {
Anonymous {},
Authenticated { expires_at: Option<String> },
Expired {},
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -4042,6 +4180,8 @@ pub struct AuthBootstrapUserRequest {
#[serde(deny_unknown_fields)]
pub struct AuthUserResponse {
pub user: AuthenticatedUser,
/// Comparison evidence for the exact newly issued session, not a credential.
pub authentication_reference: String,
}
/// Starts new-account registration only; an existing handle is never interpreted as key addition.
@@ -4058,16 +4198,64 @@ pub struct PasskeyRegistrationOptionsRequest {
pub browser_origin: Option<String>,
}
/// Adds a credential to the current authenticated actor; target identity is never public input.
/// Adds to the current actor. Expected identity/binding fields only fence stale UI;
/// they never select an owner or authorize an addition.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyAdditionOptionsRequest {
pub expected_user_id: String,
pub expected_account_id: String,
pub expected_authentication_reference: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub browser_origin: Option<String>,
}
/// Safe account metadata. Neither the WebAuthn credential ID nor key material is exposed.
/// Normal-login verification is observed verifier success, not inferred discoverability.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeySummary {
pub key_reference: String,
pub created_at: String,
pub last_used_at: Option<String>,
pub normal_login_verified_at: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyListResponse {
pub user: AuthenticatedUser,
pub keys: Vec<PasskeySummary>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyAdditionResultRequest {
pub challenge_id: String,
pub expected_user_id: String,
pub expected_account_id: String,
/// Original options authentication reference; the current same-owner session may differ.
pub expected_authentication_reference: String,
}
/// Unknown is not proof that registration failed: completion may still be in flight.
/// Saved proves the credential commit, not session issuance or current session health.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "status", rename_all = "snake_case", deny_unknown_fields)]
pub enum PasskeyAdditionResultResponse {
Unknown {},
Saved {
user: AuthenticatedUser,
key: PasskeySummary,
},
}
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -6876,6 +7064,8 @@ pub struct WorkingDirectoryCleanupTarget {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkingDirectoryRemovalRequest {
/// Stable for an exact resend; a new explicit request uses a new identity.
pub operation_id: String,
pub reason: String,
}
@@ -7410,7 +7600,13 @@ pub struct CurrentWorkerWorkdirAttachmentResponse {
pub alias: String,
pub working_directory_id: String,
pub capabilities: workdir::WorkdirSessionCapabilities,
/// Present for attachment responses, absent after detach.
#[serde(skip_serializing_if = "Option::is_none")]
pub environment: Option<workdir::WorkdirSessionEnvironment>,
pub attached: bool,
/// The Workdir was already attached to this Worker; alias is its existing name.
#[serde(default)]
pub already_attached: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -7422,7 +7618,9 @@ pub struct CurrentWorkerWorkdirOperationRequest {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(transparent)]
pub struct CurrentWorkerWorkdirOperationResponse(pub workdir::http::WorkdirSessionOperationResult);
pub struct CurrentWorkerWorkdirOperationResponse(
pub workdir::http::WorkdirSessionOperationResponse,
);
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -8780,6 +8978,9 @@ pub struct MergeRequestListQuery {
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct MergeRequestThreadQuery {
/// Show only: summary omits history; detail returns an exact bounded thread page.
/// Omission retains the existing HTTP detail response.
pub view: Option<MergeRequestReadView>,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub after: Option<u64>,
#[schemars(range(min = 0, max = 1000))]
@@ -8807,6 +9008,73 @@ pub struct MergeRequestLinkedTicketResponse {
pub key: Option<String>,
}
#[derive(Debug, Clone, Copy, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum MergeRequestReadView {
Summary,
#[default]
Detail,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct MergeRequestEventReference {
pub event_id: String,
/// Exact event: read detail with after = sequence - 1 and limit = 1.
#[cfg_attr(feature = "typescript", ts(type = "number"))]
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub sequence: u64,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct MergeRequestReviewSummary {
pub event: MergeRequestEventReference,
pub request_event_id: String,
pub subject_ref: String,
pub ticket_content_digest: String,
pub decision: ReviewDecision,
/// Findings retain their order in the exact review event; no finding body is truncated here.
#[cfg_attr(feature = "typescript", ts(type = "number"))]
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub findings_count: usize,
}
/// Computed from complete persisted evidence, never from the returned history page.
/// This observation is not integration authority or current-requirements approval.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct MergeRequestStatusSummary {
/// unknown, none, pending, approved, or request_changes for the observed source.
pub review_status: String,
pub review: Option<MergeRequestReviewSummary>,
pub latest_review_subject_ref: Option<String>,
pub latest_event: Option<MergeRequestEventReference>,
#[cfg_attr(feature = "typescript", ts(type = "number"))]
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub thread_event_count: usize,
/// Existing readiness semantics; review bodies are omitted in this compact projection.
pub readiness: MergeRequestReadinessResponse,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct MergeRequestEventPage {
/// Summary deliberately omits events. Read detail from the beginning to obtain history.
pub omitted: bool,
pub has_more: bool,
#[cfg_attr(feature = "typescript", ts(type = "number | null"))]
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub next_after: Option<u64>,
#[schemars(range(min = 1, max = 200))]
pub limit: usize,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -8816,6 +9084,9 @@ pub struct MergeRequestDetailResponse {
pub source: MergeRequestRefResponse,
pub target: MergeRequestRefResponse,
pub linked_tickets: Vec<MergeRequestLinkedTicketResponse>,
pub view: MergeRequestReadView,
pub summary: MergeRequestStatusSummary,
pub event_page: MergeRequestEventPage,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -10428,6 +10699,10 @@ pub struct CreateWorkspaceWorkerRequest {
/// Backend idempotency key used only for authenticated Worker-owned spawn/control.
#[serde(default)]
pub control_operation_id: Option<String>,
/// Requested least-authority child permissions. Host derives the origin; this
/// input cannot assert Subject identity or create a body/singleton attachment.
#[serde(default)]
pub execution_grant: Option<authorization::WorkerDelegationRequest>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -10466,8 +10741,8 @@ pub enum WorkerOperationState {
/// Cross-layer outcome of an explicit Worker restore operation.
///
/// `Rejected` is reserved for read-only preflight failures. Once live restore
/// work starts, failure is either a confirmed `RolledBack` operation or a
/// `ReconciliationRequired` result whose commit state must be reread/retried.
/// work starts, distinguish confirmed `RolledBack`, unresolved
/// `ReconciliationRequired`, and a later Stop's `Superseded` outcome.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
@@ -10475,6 +10750,8 @@ pub enum WorkerRestoreState {
Accepted,
Rejected,
RolledBack,
/// A later Stop superseded this restore; neither rollback nor prior Accepted is asserted.
Superseded,
ReconciliationRequired,
}
@@ -11533,10 +11810,15 @@ pub fn auth_api_typescript() -> String {
ActorAuthMethod::decl(&config),
AuthenticatedUser::decl(&config),
RequestActor::decl(&config),
BrowserSessionState::decl(&config),
WhoamiResponse::decl(&config),
AuthBootstrapUserRequest::decl(&config),
AuthUserResponse::decl(&config),
PasskeyAdditionOptionsRequest::decl(&config),
PasskeySummary::decl(&config),
PasskeyListResponse::decl(&config),
PasskeyAdditionResultRequest::decl(&config),
PasskeyAdditionResultResponse::decl(&config),
PasskeyRegistrationOptionsRequest::decl(&config),
PasskeyRegistrationOptionsResponse::decl(&config),
PasskeyRegistrationCompleteRequest::decl(&config),
@@ -12567,6 +12849,7 @@ mod tests {
(WorkerRestoreState::Accepted, "accepted"),
(WorkerRestoreState::Rejected, "rejected"),
(WorkerRestoreState::RolledBack, "rolled_back"),
(WorkerRestoreState::Superseded, "superseded"),
(
WorkerRestoreState::ReconciliationRequired,
"reconciliation_required",
@@ -12929,6 +13212,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
})
.unwrap();
assert_eq!(
@@ -12943,6 +13227,7 @@ mod tests {
"workdir_attachments": [],
"feature_connections": {},
"control_operation_id": null,
"execution_grant": null,
})
);
}
@@ -13173,10 +13458,13 @@ mod tests {
alias: "checkout".to_string(),
working_directory_id: "workdir".to_string(),
capabilities: workdir::WorkdirSessionCapabilities::EMPTY,
environment: None,
attached: false,
already_attached: false,
};
let value = serde_json::to_value(&response).unwrap();
assert_eq!(value.as_object().unwrap().len(), 5);
assert_eq!(value.as_object().unwrap().len(), 6);
assert_eq!(value["already_attached"], false);
assert!(value.get("connection_id").is_none());
assert_eq!(
serde_json::from_value::<CurrentWorkerWorkdirAttachmentResponse>(value).unwrap(),
@@ -13961,7 +14249,9 @@ mod tests {
"handle": "hare",
"display_name": "Hare",
"auth_method": "browser_session"
}
},
"session": { "kind": "authenticated", "expires_at": "2026-10-20T00:00:00Z" },
"authentication_reference": "synthetic-session-reference"
});
let decoded = serde_json::from_value::<WhoamiResponse>(whoami.clone())
.expect("server whoami fixture should match shared DTO");
@@ -13983,7 +14273,8 @@ mod tests {
"account_id": "account-1",
"handle": "hare",
"display_name": "Hare"
}
},
"authentication_reference": "synthetic-session-reference"
});
let decoded = serde_json::from_value::<AuthUserResponse>(auth_user.clone())
.expect("server auth-user fixture should match shared DTO");
@@ -14143,11 +14434,36 @@ mod tests {
let unexpected_field = serde_json::json!({
"actor": null,
"session": { "kind": "anonymous" },
"access_token": "must-not-be-accepted"
});
assert!(serde_json::from_value::<WhoamiResponse>(unexpected_field).is_err());
}
#[test]
fn whoami_session_json_distinguishes_anonymous_expired_and_token_auth() {
for session in [
serde_json::json!({"kind":"anonymous"}),
serde_json::json!({"kind":"expired"}),
serde_json::json!({"kind":"authenticated", "expires_at":null}),
] {
let value = serde_json::json!({"actor":null, "session":session, "authentication_reference": null});
let response: WhoamiResponse = serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(response).unwrap(), value);
}
for session in [
serde_json::json!({"kind":"unknown"}),
serde_json::json!({"kind":"expired", "expires_at":null}),
] {
assert!(
serde_json::from_value::<WhoamiResponse>(
serde_json::json!({"actor":null, "session":session})
)
.is_err()
);
}
}
#[test]
fn device_login_debug_output_redacts_secret_material() {
let start = DeviceLoginStartResponse {
@@ -14532,6 +14848,7 @@ mod openapi_artifact_tests {
// identity, or a one-use Reviewer capability. They remain generated Rust client/Axum
// operations, but must not appear as unauthenticated operations in the public OpenAPI.
const SIGNED_INTERNAL: &[&str] = &[
"operation_authorize",
"current_worker_workspace_config_get",
"current_worker_workspace_config_attach",
"current_worker_workspace_config_observe",
@@ -14668,10 +14985,28 @@ mod openapi_artifact_tests {
);
}
assert!(
serde_json::from_value::<PasskeyAdditionOptionsRequest>(serde_json::json!({})).is_ok()
serde_json::from_value::<PasskeyAdditionOptionsRequest>(serde_json::json!({})).is_err()
);
}
#[test]
fn passkey_addition_requires_expected_owner_and_binding() {
let expected = serde_json::json!({
"expected_user_id": "user-a", "expected_account_id": "account-a",
"expected_authentication_reference": "reference"
});
assert!(serde_json::from_value::<PasskeyAdditionOptionsRequest>(expected.clone()).is_ok());
for field in [
"expected_user_id",
"expected_account_id",
"expected_authentication_reference",
] {
let mut missing = expected.clone();
missing.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<PasskeyAdditionOptionsRequest>(missing).is_err());
}
}
#[test]
fn auth_workspace_catalog_and_typed_header_contract_is_strict() {
let document = canonical_openapi_document().expect("canonical OpenAPI contract must build");
@@ -14917,10 +15252,12 @@ mod workdir_error_diagnostic_tests {
#[test]
fn workdir_internal_diagnostics_never_enter_public_error_body_or_schema() {
let mut error = WorkdirOperationApiError::provider(workdir::http::WorkdirTransportError {
failure_reason: None,
code: workdir::http::WorkdirTransportErrorCode::Denied,
message: "Workdir operation was denied".to_string(),
denial_reason: Some(workdir::WorkdirDenialReason::OsPermissionDenied),
});
error.failure_reason = Some(workdir::WorkdirFailureReason::HttpTimeout);
error.workdir_log = Some(WorkdirOperationLogContext {
operation: "write",
workspace_id_hash: "internal-workspace-fingerprint".to_string(),
@@ -14934,10 +15271,12 @@ mod workdir_error_diagnostic_tests {
assert_eq!(api_macros::HttpError::status_code(&error), 403);
let decoded: WorkdirOperationApiError = serde_json::from_value(wire).unwrap();
assert!(decoded.denial_reason.is_none());
assert!(decoded.failure_reason.is_none());
assert!(decoded.workdir_log.is_none());
let schema = serde_json::to_value(schemars::schema_for!(WorkdirOperationApiError)).unwrap();
assert!(schema["properties"].get("workdir_log").is_none());
assert!(schema["properties"].get("denial_reason").is_none());
assert!(schema["properties"].get("failure_reason").is_none());
}
}
+2
View File
@@ -11,5 +11,7 @@ checks or Runtime catalog lifecycle management.
- [`worker_session_store`]: storage boundary for a Session bound to a Worker
- [`worker_metadata`]: current Worker state and active Segment switching
- [`system_item`]: records of system input presented to the model
- [`capture`]: Worker-independent immutable public values and bounded body transport;
authorization, source I/O, and delivery lifecycle stay with the Host
- [`public_snapshot`] / [`public_index`]: public projections of stored history
- [`event_trace`]: diagnostic records independent of history replay
+967
View File
@@ -0,0 +1,967 @@
//! Exploration domain for immutable Session captures, with no Workspace/Memory mutation capability.
//!
//! A capture is an ordered projection of committed entries; exploration excludes reasoning and
//! system prompts, including Host compaction summaries and other derived context. Full inherited
//! summary input belongs to compaction, not public observation or Memory evidence. Overview,
//! search, bounded reads, and evidence conversion share `SessionEntryRef`.
//! `SessionEntryRef` is a contextual UI reference. `ScopedSessionEntryRef` additionally binds
//! the Session and exact immutable capture, and is required for portable checked operations.
//! Appends preserve existing entry identities. Sparse real user/assistant anchors and intervening
//! entry counts index long histories. Search with an empty filter also provides bounded range lists.
//!
//! Host-fixed snapshot exploration, Memory candidate evidence validation, and current Worker
//! observation share only this domain; they do not enable or modify each other's Features.
//! The provider owns authorization to recapture a current Worker. A capture itself proves neither
//! completion nor approval and grants no business-mutation authority.
//!
//! Recording Hosts use the common committed replay through the guarded APIs below.
//! [`recording_catalog_worker_guarded`] and [`recording_catalog_fs_guarded`] expose
//! descriptors only to an admission callback under the caller's existing publication/
//! erasure guard and persisted active selector, without allocating a full body DTO.
//! [`capture_recording_slice_worker_guarded`] and [`capture_recording_slice_fs_guarded`]
//! separately construct bounded body-bearing inputs. Admission descriptors are not
//! sufficient Job inputs; saved recording DTOs are read/sliced without live storage.
use std::sync::Arc;
use crate::{
LoggedSessionHistoryMetadata as SessionHistoryMetadata,
LoggedSessionHistoryOrigin as HistoryOrigin,
};
mod body;
mod recording;
use agen::{HistoryEntry, Item, Role};
pub use body::*;
use protocol::{
SessionContentPart, SessionEntryProvenance, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntryData,
};
pub use recording::*;
use serde::{Deserialize, Serialize};
const DEFAULT_SEARCH_LIMIT: usize = 20;
const MAX_SEARCH_LIMIT: usize = 50;
const DEFAULT_READ_MAX_ITEMS: usize = 40;
const MAX_READ_MAX_ITEMS: usize = 80;
const DEFAULT_READ_MAX_BYTES: usize = 32 * 1024;
const OVERVIEW_ANCHOR_STRIDE: usize = 8;
/// Append-stable contextual UI reference derived from a logical entry identity.
/// Portable consumers must bind it with `SessionCapture::scoped_ref` before use.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(transparent)]
pub struct SessionEntryRef(String);
impl SessionEntryRef {
pub fn from_history_entry_id(entry_id: &crate::LoggedSessionHistoryEntryId) -> Self {
Self(format!("E{}", entry_id.0))
}
pub fn parse(value: &str) -> Option<Self> {
let suffix = value.strip_prefix('E')?;
if suffix.is_empty()
// Legacy Host boundary identities contain both Session and Segment UUIDs.
|| suffix.len() > 256
|| !suffix
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
{
return None;
}
Some(Self(value.to_string()))
}
pub fn as_str(&self) -> &str {
&self.0
}
}
impl std::fmt::Display for SessionEntryRef {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str(&self.0)
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ReferenceKind {
User,
Assistant,
System,
Tool,
}
impl ReferenceKind {
pub fn as_str(self) -> &'static str {
match self {
Self::User => "user",
Self::Assistant => "assistant",
Self::System => "system",
Self::Tool => "tool",
}
}
pub fn parse(value: &str) -> Option<Self> {
match value {
"user" => Some(Self::User),
"assistant" | "agent" => Some(Self::Assistant),
"system" => Some(Self::System),
"tool" => Some(Self::Tool),
_ => None,
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ToolPart {
Input,
Output,
Both,
}
impl ToolPart {
pub fn parse(value: &str) -> Option<Self> {
match value {
"input" => Some(Self::Input),
"output" => Some(Self::Output),
"both" => Some(Self::Both),
_ => None,
}
}
pub fn as_str(self) -> &'static str {
match self {
Self::Input => "input",
Self::Output => "output",
Self::Both => "both",
}
}
pub fn matches(self, actual: ToolPart) -> bool {
matches!(self, ToolPart::Both) || self == actual
}
}
#[derive(Debug, Clone)]
pub struct OverviewItem {
pub id: SessionEntryRef,
pub origin: SessionEntryProvenance,
pub entry_range: [u64; 2],
pub kind: ReferenceKind,
pub label: String,
pub text: String,
pub intervening_entries: usize,
}
#[derive(Debug, Clone)]
pub struct ReferenceEntry {
pub attachment_count: usize,
use_policy: Option<protocol::SessionEntryUsePolicy>,
pub id: SessionEntryRef,
pub origin: SessionEntryProvenance,
pub entry_range: [u64; 2],
pub kind: ReferenceKind,
pub tool_part: Option<ToolPart>,
pub tool_name: Option<String>,
pub label: String,
pub summary: String,
search_text: String,
}
#[derive(Debug, Clone, Default)]
pub struct SearchOptions {
pub query: String,
pub kind: Option<ReferenceKind>,
pub tool_part: Option<ToolPart>,
pub tool_name: Option<String>,
pub limit: Option<usize>,
pub min_entry_index: Option<u64>,
pub from: Option<SessionEntryRef>,
pub through: Option<SessionEntryRef>,
pub offset: usize,
}
#[derive(Debug, Clone)]
pub struct SearchHit {
pub id: SessionEntryRef,
pub origin: SessionEntryProvenance,
pub kind: ReferenceKind,
pub tool_part: Option<ToolPart>,
pub tool_name: Option<String>,
pub entry_range: [u64; 2],
pub label: String,
pub summary: String,
}
#[derive(Debug, Clone, Copy)]
pub enum ReadSelector<'a> {
Id(&'a str),
EntryRange([u64; 2]),
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ReadDetail {
Compact,
Full,
}
#[derive(Debug, Clone)]
pub struct ReadOptions {
pub include_tools: bool,
pub tool_part: ToolPart,
pub detail: ReadDetail,
pub max_items: usize,
pub max_bytes: usize,
}
impl Default for ReadOptions {
fn default() -> Self {
Self {
include_tools: true,
tool_part: ToolPart::Both,
detail: ReadDetail::Compact,
max_items: DEFAULT_READ_MAX_ITEMS,
max_bytes: DEFAULT_READ_MAX_BYTES,
}
}
}
#[derive(Debug, Clone)]
pub struct ReadEntry {
pub id: SessionEntryRef,
pub origin: SessionEntryProvenance,
pub kind: ReferenceKind,
pub tool_part: Option<ToolPart>,
pub tool_name: Option<String>,
pub entry_range: [u64; 2],
pub label: String,
pub text: String,
}
#[derive(Debug, Clone)]
pub struct ReadResult {
pub entries: Vec<ReadEntry>,
pub truncated: bool,
}
#[derive(Debug, Clone)]
pub struct SessionEntryEvidence {
pub segment_id: String,
pub entry_ref: SessionEntryRef,
pub origin: SessionEntryProvenance,
/// Inclusive original-source positions in `segment_id`, not view/UI order.
pub entry_range: [u64; 2],
pub kind: ReferenceKind,
pub tool_part: Option<ToolPart>,
pub label: String,
pub summary: String,
pub excerpt: String,
}
#[derive(Debug, Clone)]
struct CapturedHistoryEntry {
use_policy: Option<protocol::SessionEntryUsePolicy>,
item: Item,
entry_id: crate::LoggedSessionHistoryEntryId,
origin: SessionEntryProvenance,
dto: CaptureEntryDto,
}
#[derive(Debug, Clone)]
pub struct SessionCapture {
session_id: String,
segment_id: String,
capture_id: String,
view_entry_count: u64,
entries: Arc<Vec<CapturedHistoryEntry>>,
overview: Vec<OverviewItem>,
index: Vec<ReferenceEntry>,
}
impl SessionCapture {
pub fn from_session_snapshot(
session_id: impl Into<String>,
segment_id: impl Into<String>,
snapshot: SessionSnapshot,
) -> Self {
let segment_id = segment_id.into();
let view_entry_count = snapshot.entries.len() as u64;
let entries = snapshot
.entries
.into_iter()
.enumerate()
.filter_map(|(index, entry)| {
let policy = CaptureUsePolicy {
recording_material: false,
memory_evidence: entry.use_policy.is_none_or(|p| p.memory_evidence),
observation_readable: entry.use_policy.is_none_or(|p| p.observation_readable),
};
let explicit = entry.use_policy.is_some();
let item = match entry.data {
SessionSnapshotEntryData::UserInput { segments } => {
Item::user_message(protocol::Segment::flatten_to_text(&segments))
}
SessionSnapshotEntryData::Message { role, content } => Item::Message {
id: None,
status: None,
role: match role {
SessionMessageRole::User => Role::User,
SessionMessageRole::Assistant => Role::Assistant,
},
content: content
.into_iter()
.map(|part| match part {
SessionContentPart::Text { text } => {
agen::ContentPart::Text { text }
}
SessionContentPart::Refusal { refusal } => {
agen::ContentPart::Refusal { refusal }
}
})
.collect(),
},
SessionSnapshotEntryData::ToolCall {
call_id,
name,
arguments,
} => Item::tool_call(call_id, name, arguments),
SessionSnapshotEntryData::ToolResult {
call_id,
summary,
content,
is_error,
..
} => Item::tool_result_item(call_id, summary, content, is_error),
SessionSnapshotEntryData::SystemItem { content, .. } if explicit => {
Item::system_message(content)
}
_ => return None,
};
let body = CaptureBody::project(&item, &entry.provenance, explicit)?;
if !policy.any() {
return None;
}
Some(CaptureEntryDto {
entry_id: entry.entry_id,
order: index as u64,
source: CaptureSource {
segment_id: segment_id.clone(),
entry_index: index as u64,
provenance: entry.provenance,
history_origin: None,
derived_from: entry.derived_from,
},
policy,
body,
})
})
.collect();
Self::from_projected(session_id.into(), segment_id, view_entry_count, entries)
}
pub fn new(
session_id: impl Into<String>,
segment_id: impl Into<String>,
items: Vec<Item>,
) -> Self {
let segment_id = segment_id.into();
let view_entry_count = items.len() as u64;
let entries = items
.into_iter()
.enumerate()
.filter_map(|(index, item)| {
let origin = SessionEntryProvenance::LegacyUnknown;
let body = CaptureBody::project(&item, &origin, false)?;
Some(CaptureEntryDto {
entry_id: format!("{index:08}"),
order: index as u64,
source: CaptureSource {
segment_id: segment_id.clone(),
entry_index: index as u64,
provenance: origin,
history_origin: None,
derived_from: Vec::new(),
},
policy: CaptureUsePolicy::PUBLIC,
body,
})
})
.collect();
Self::from_projected(session_id.into(), segment_id, view_entry_count, entries)
}
/// Project a complete Host-fixed history whose source positions start at zero.
/// Select a suffix with `select_order_range` on the resulting value instead
/// of passing a rebased history vector: annotations do not carry source positions.
pub fn from_complete_history(
session_id: impl Into<String>,
segment_id: impl Into<String>,
entries: Vec<HistoryEntry<SessionHistoryMetadata>>,
) -> Self {
let segment_id = segment_id.into();
let view_entry_count = entries.len() as u64;
let entries = entries
.into_iter()
.enumerate()
.filter_map(|(index, entry)| {
let origin = public_provenance(&entry.annotation.origin);
let mut item = entry.item;
let mut explicit = false;
let policy = if let Some(input) = entry.annotation.input_metadata {
explicit = true;
if matches!(
item,
Item::Message {
role: Role::System,
..
}
) {
// Only the explicit public projection crosses this boundary.
let public = input.public_item()?;
item = Item::system_message(public.body.clone());
} else if input.feature_payload().is_some() || input.public_item().is_some() {
return None;
}
CaptureUsePolicy::from(input.usage())
} else {
if matches!(entry.annotation.origin, HistoryOrigin::Provided { .. }) {
return None;
}
CaptureUsePolicy::PUBLIC
};
if !policy.any() {
return None;
}
let body = CaptureBody::project(&item, &origin, explicit)?;
Some(CaptureEntryDto {
entry_id: entry.annotation.entry_id.0,
order: index as u64,
source: CaptureSource {
segment_id: segment_id.clone(),
entry_index: index as u64,
provenance: origin,
history_origin: Some(entry.annotation.origin),
derived_from: entry
.annotation
.derivation
.map(|d| d.sources.into_iter().map(|id| id.0).collect())
.unwrap_or_default(),
},
policy,
body,
})
})
.collect();
Self::from_projected(session_id.into(), segment_id, view_entry_count, entries)
}
fn from_projected(
session_id: String,
segment_id: String,
view_entry_count: u64,
dtos: Vec<CaptureEntryDto>,
) -> Self {
let dto = CaptureDto {
version: CAPTURE_VERSION,
session_id: session_id.clone(),
segment_id: segment_id.clone(),
view_entry_count,
entries: dtos,
};
let capture_id = dto.fingerprint();
let entries: Vec<_> = dto
.entries
.into_iter()
.map(|dto| CapturedHistoryEntry {
use_policy: Some(protocol::SessionEntryUsePolicy {
observation_readable: dto.policy.observation_readable,
memory_evidence: dto.policy.memory_evidence,
}),
item: dto.body.to_item(),
entry_id: crate::LoggedSessionHistoryEntryId(dto.entry_id.clone()),
origin: dto.source.provenance.clone(),
dto,
})
.collect();
let mut overview = Vec::new();
let mut index = Vec::new();
for entry in &entries {
let item = &entry.item;
let entry_range = [entry.dto.order; 2];
let attachment_count = match entry.dto.body {
CaptureBody::ToolResult {
attachment_count, ..
} => attachment_count,
_ => 0,
};
match item {
Item::Message { role, content, .. } => {
let Some(kind) = message_reference_kind(&entry.origin, role, entry.use_policy)
else {
continue;
};
let text = content
.iter()
.map(|p| p.as_text())
.collect::<Vec<_>>()
.join("");
let label = format!("{} message", kind.as_str());
let summary = truncate_chars(&text, 240);
let id = SessionEntryRef::from_history_entry_id(&entry.entry_id);
index.push(ReferenceEntry {
use_policy: entry.use_policy,
attachment_count,
id: id.clone(),
origin: entry.origin.clone(),
entry_range,
kind,
tool_part: None,
tool_name: None,
label: label.clone(),
summary: summary.clone(),
search_text: text.clone(),
});
if entry.use_policy.is_none_or(|p| p.observation_readable)
&& matches!(kind, ReferenceKind::User | ReferenceKind::Assistant)
{
overview.push(OverviewItem {
id: id.clone(),
origin: entry.origin.clone(),
entry_range,
kind,
label,
text,
intervening_entries: 0,
});
}
}
Item::ToolCall {
name, arguments, ..
} => {
let text = format!("{name}\n{arguments}");
index.push(ReferenceEntry {
use_policy: entry.use_policy,
attachment_count,
id: SessionEntryRef::from_history_entry_id(&entry.entry_id),
origin: entry.origin.clone(),
entry_range,
kind: ReferenceKind::Tool,
tool_part: Some(ToolPart::Input),
tool_name: Some(name.clone()),
label: format!("tool input: {name}"),
summary: format!("Tool call {name}"),
search_text: text,
});
}
Item::ToolResult {
summary,
content,
attachments: _,
..
} => {
let attachment_marker = if attachment_count == 0 {
String::new()
} else {
format!("\n[{} image attachment(s)]", attachment_count)
};
let text = format!(
"{summary}\n{}{attachment_marker}",
content.as_deref().unwrap_or_default(),
);
index.push(ReferenceEntry {
use_policy: entry.use_policy,
attachment_count,
id: SessionEntryRef::from_history_entry_id(&entry.entry_id),
origin: entry.origin.clone(),
entry_range,
kind: ReferenceKind::Tool,
tool_part: Some(ToolPart::Output),
tool_name: None,
label: "tool output".to_string(),
summary: truncate_chars(summary, 240),
search_text: text,
});
}
Item::AssistantResponseBoundary { .. } | Item::Reasoning { .. } => {}
}
}
if overview.len() > 2 {
let last = overview.len() - 1;
overview = overview
.into_iter()
.enumerate()
.filter_map(|(index, entry)| {
(index == 0 || index == last || index % OVERVIEW_ANCHOR_STRIDE == 0)
.then_some(entry)
})
.collect();
}
for overview_index in 0..overview.len().saturating_sub(1) {
let current_entry = overview[overview_index].entry_range[0];
let next_entry = overview[overview_index + 1].entry_range[0];
overview[overview_index].intervening_entries = index
.iter()
.filter(|entry| {
let entry_index = entry.entry_range[0];
entry_index > current_entry && entry_index < next_entry
})
.count();
}
Self {
session_id,
segment_id,
capture_id,
view_entry_count,
entries: Arc::new(entries),
overview,
index,
}
}
/// Public source entries are immutable; callers retain their own business decisions.
pub fn entries(&self) -> impl ExactSizeIterator<Item = &CaptureEntryDto> {
self.entries.iter().map(|entry| &entry.dto)
}
pub fn overview(&self) -> &[OverviewItem] {
&self.overview
}
/// Capture-local UI position, not the original source position used by evidence.
pub fn view_order_for_ref(&self, reference: &SessionEntryRef) -> Option<u64> {
self.index
.iter()
.find(|entry| entry.id == *reference)
.map(|entry| entry.entry_range[0])
}
pub fn search(&self, options: &SearchOptions) -> Vec<SearchHit> {
if self.validate_search(options).is_err() {
return Vec::new();
}
let query = options.query.trim().to_lowercase();
let limit = options
.limit
.unwrap_or(DEFAULT_SEARCH_LIMIT)
.clamp(1, MAX_SEARCH_LIMIT);
let tool_name = options.tool_name.as_deref();
let min_entry_index = options
.from
.as_ref()
.and_then(|reference| self.view_order_for_ref(reference))
.unwrap_or_else(|| options.min_entry_index.unwrap_or(0));
let max_entry_index = options
.through
.as_ref()
.and_then(|reference| self.view_order_for_ref(reference))
.unwrap_or(u64::MAX);
let mut skipped = 0usize;
let mut hits = Vec::new();
for entry in &self.index {
if entry.use_policy.is_some_and(|p| !p.observation_readable) {
continue;
}
if entry.entry_range[0] < min_entry_index || entry.entry_range[0] > max_entry_index {
continue;
}
if let Some(kind) = options.kind {
if entry.kind != kind {
continue;
}
}
if let Some(part) = options.tool_part {
if entry.kind != ReferenceKind::Tool {
continue;
}
let Some(actual) = entry.tool_part else {
continue;
};
if !part.matches(actual) {
continue;
}
}
if let Some(name) = tool_name {
if entry.tool_name.as_deref() != Some(name) {
continue;
}
}
if !query.is_empty() && !entry.search_text.to_lowercase().contains(&query) {
continue;
}
if skipped < options.offset {
skipped += 1;
continue;
}
hits.push(SearchHit {
id: entry.id.clone(),
origin: entry.origin.clone(),
kind: entry.kind,
tool_part: entry.tool_part,
tool_name: entry.tool_name.clone(),
entry_range: entry.entry_range,
label: entry.label.clone(),
summary: entry.summary.clone(),
});
if hits.len() >= limit {
break;
}
}
hits
}
pub fn read(&self, selector: ReadSelector<'_>, options: ReadOptions) -> ReadResult {
let max_items = options.max_items.clamp(1, MAX_READ_MAX_ITEMS);
let max_bytes = options.max_bytes.max(1);
let mut entries = Vec::new();
let mut bytes = 0usize;
let mut truncated = false;
let selected: Vec<&ReferenceEntry> = match selector {
ReadSelector::Id(id) => self
.index
.iter()
.filter(|entry| entry.id.as_str() == id)
.collect(),
ReadSelector::EntryRange([start, end]) => self
.index
.iter()
.filter(|entry| entry.entry_range[0] >= start && entry.entry_range[0] <= end)
.collect(),
};
for entry in selected {
if entry.use_policy.is_some_and(|p| !p.observation_readable) {
continue;
}
if entries.len() >= max_items {
truncated = true;
break;
}
if entry.kind == ReferenceKind::Tool {
if !options.include_tools {
continue;
}
if let Some(actual) = entry.tool_part {
if !options.tool_part.matches(actual) {
continue;
}
}
}
let Some(item) = self
.entries
.iter()
.find(|captured| captured.dto.order == entry.entry_range[0])
.map(|entry| &entry.item)
else {
continue;
};
let remaining = max_bytes.saturating_sub(bytes);
if remaining == 0 {
truncated = true;
break;
}
let rendered = render_item(item, entry, options.detail, usize::MAX);
truncated |= rendered.len() > remaining;
let text = truncate_bytes(&rendered, remaining);
bytes = bytes.saturating_add(text.len());
entries.push(ReadEntry {
id: entry.id.clone(),
origin: entry.origin.clone(),
kind: entry.kind,
tool_part: entry.tool_part,
tool_name: entry.tool_name.clone(),
entry_range: entry.entry_range,
label: entry.label.clone(),
text,
});
if truncated {
break;
}
}
ReadResult { entries, truncated }
}
/// Resolves the committed tool-call input for a provider call id. This is
/// used by explicit-memory receipts only after the containing run is durable,
/// so a model-authored claim can cite its own committed tool invocation
/// without inventing a SessionEntryRef.
pub fn evidence_for_tool_call(&self, call_id: &str) -> Option<SessionEntryEvidence> {
let captured = self.entries.iter().find(|entry| {
matches!(
&entry.item,
Item::ToolCall {
call_id: stored_call_id,
..
} if stored_call_id == call_id
)
})?;
let reference = self.index.iter().find(|entry| {
entry.entry_range[0] == captured.dto.order
&& entry.kind == ReferenceKind::Tool
&& entry.tool_part == Some(ToolPart::Input)
})?;
self.evidence_for(reference.id.as_str())
}
pub fn evidence_for(&self, id: &str) -> Option<SessionEntryEvidence> {
let entry = self.index.iter().find(|entry| entry.id.as_str() == id)?;
if entry.use_policy.is_some_and(|p| !p.memory_evidence) {
return None;
}
let captured = self
.entries
.iter()
.find(|captured| captured.dto.order == entry.entry_range[0])?;
let excerpt = render_item(&captured.item, entry, ReadDetail::Compact, 2 * 1024);
Some(SessionEntryEvidence {
segment_id: captured.dto.source.segment_id.clone(),
entry_ref: entry.id.clone(),
origin: entry.origin.clone(),
entry_range: [captured.dto.source.entry_index; 2],
kind: entry.kind,
tool_part: entry.tool_part,
label: entry.label.clone(),
summary: entry.summary.clone(),
excerpt,
})
}
}
pub(crate) fn public_provenance(origin: &HistoryOrigin) -> SessionEntryProvenance {
match origin {
HistoryOrigin::HumanInput { .. } | HistoryOrigin::LocalHumanInput => {
SessionEntryProvenance::HumanInput
}
HistoryOrigin::WorkerInput { .. } => SessionEntryProvenance::WorkerInput,
HistoryOrigin::FlowInstruction { .. } => SessionEntryProvenance::FlowInstruction,
HistoryOrigin::BackendInstruction { .. } | HistoryOrigin::Provided { .. } => {
SessionEntryProvenance::BackendInstruction
}
HistoryOrigin::ModelOutput { .. } => SessionEntryProvenance::ModelOutput,
HistoryOrigin::ToolOutput { .. } => SessionEntryProvenance::ToolOutput,
HistoryOrigin::CompactionSummary
| HistoryOrigin::DerivedSummary
| HistoryOrigin::JournalContext => SessionEntryProvenance::DerivedSummary,
HistoryOrigin::LegacyUnknown => SessionEntryProvenance::LegacyUnknown,
}
}
fn message_reference_kind(
origin: &SessionEntryProvenance,
provider_role: &Role,
use_policy: Option<protocol::SessionEntryUsePolicy>,
) -> Option<ReferenceKind> {
if use_policy.is_some() {
return match provider_role {
Role::System => Some(ReferenceKind::System),
Role::User => Some(ReferenceKind::User),
Role::Assistant => Some(ReferenceKind::Assistant),
};
}
match origin {
SessionEntryProvenance::HumanInput | SessionEntryProvenance::WorkerInput => {
Some(ReferenceKind::User)
}
SessionEntryProvenance::ModelOutput => Some(ReferenceKind::Assistant),
SessionEntryProvenance::ToolOutput => Some(ReferenceKind::Tool),
SessionEntryProvenance::LegacyUnknown => match provider_role {
Role::User => Some(ReferenceKind::User),
Role::Assistant => Some(ReferenceKind::Assistant),
Role::System => None,
},
// Flow/backend/system content remains out of the observation surface
// even when represented with a provider user/system role.
SessionEntryProvenance::FlowInstruction
| SessionEntryProvenance::BackendInstruction
| SessionEntryProvenance::DerivedSummary => None,
}
}
fn render_item(
item: &Item,
entry: &ReferenceEntry,
detail: ReadDetail,
max_bytes: usize,
) -> String {
let text = match item {
Item::Message { role, content, .. } => {
let text = content
.iter()
.map(|p| p.as_text())
.collect::<Vec<_>>()
.join("");
format!("[{} {:?}] {text}", entry.id, role)
}
Item::ToolCall {
name, arguments, ..
} => match detail {
ReadDetail::Compact => format!("[{} ToolInput {name}] (arguments omitted)", entry.id),
ReadDetail::Full => format!("[{} ToolInput {name}]\narguments: {arguments}", entry.id),
},
Item::ToolResult {
summary,
content,
attachments: _,
is_error,
..
} => {
let attachment_line = if entry.attachment_count == 0 {
String::new()
} else {
format!("\nattachments: {} image(s)", entry.attachment_count)
};
match detail {
ReadDetail::Compact => format!(
"[{} ToolOutput{}]\nsummary: {summary}\ncontent: (omitted){attachment_line}",
entry.id,
if *is_error { " error" } else { "" },
),
ReadDetail::Full => format!(
"[{} ToolOutput{}]\nsummary: {summary}\ncontent: {}{attachment_line}",
entry.id,
if *is_error { " error" } else { "" },
content.as_deref().unwrap_or_default(),
),
}
}
Item::AssistantResponseBoundary { .. } => {
format!("[{} Assistant response boundary omitted]", entry.id)
}
Item::Reasoning { .. } => format!("[{} Reasoning omitted]", entry.id),
};
truncate_bytes(&text, max_bytes)
}
fn truncate_chars(text: &str, max_chars: usize) -> String {
if max_chars == 0 {
return "… [truncated]".to_string();
}
if text.chars().count() <= max_chars {
return text.to_string();
}
let mut out = text.chars().take(max_chars).collect::<String>();
out.push_str("… [truncated]");
out
}
fn truncate_bytes(text: &str, max_bytes: usize) -> String {
if text.len() <= max_bytes {
return text.to_owned();
}
let marker = "… [truncated]";
let mut end = max_bytes.saturating_sub(marker.len());
while !text.is_char_boundary(end) {
end -= 1;
}
let mut out = text[..end].to_owned();
if max_bytes >= marker.len() {
out.push_str(marker);
}
out
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,425 @@
//! Host capture and pure canonical slicing. No Job state or live-reader fallback lives here.
use super::*;
use crate::committed_replay::CommittedPublicReplay;
use crate::public_material::{
PublicMaterialDescriptor, PublicMaterialError, PublicMaterialLimits, PublicMaterialUse,
};
use crate::retention::SessionRetentionStore;
use crate::{SegmentId, SessionId};
use std::path::Path;
/// Continuation in canonical Journal order, not an observation/Memory view cursor.
/// A Host must reconcile the descriptor at this position with its saved checkpoint.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RecordingPosition {
pub canonical_order: u64,
pub byte_offset: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RecordingCaptureSlice {
pub capture: CaptureDto,
pub next: Option<RecordingPosition>,
}
/// Inspect recording admission descriptors without constructing a body-bearing DTO.
///
/// The Host must authorize the source and hold the **same** Session publication/erasure
/// guard from before reading its persisted active pointer until this function returns
/// (including the callback). Pass that pointer's Segment, not a filename/mtime-derived
/// selector. This function verifies the guard and stored Session identities but does
/// not authorize the caller or establish that the supplied guard fences this root.
/// No lock is recursively acquired, no active pointer is reread, and no replay handle
/// or body-read capability is exposed to `inspect`.
///
/// The common committed recording replay fixes per-file ends before decoding. It
/// retains only descriptors/locations, decoding one record at a time and streaming
/// canonical bodies into their digests. `replay_limits` bound record allocation,
/// aggregate storage extent, entries and Segments; there is no full-capture text/JSON
/// allocation or `CaptureLimits` ceiling. The callback receives only descriptors in
/// canonical order and may return a Host-specific mapping (including a `Result`). It
/// runs once on success and not at all on replay/identity/resource errors.
///
/// This catalog is for Host admission, **not** a descriptor-only Job input. Construct
/// the Job's bounded body input with the guarded slice API at its own authorized
/// boundary; after saving it, execute only that immutable value. Neither descriptors
/// nor a completed callback authorize delayed adoption after erasure.
///
/// ```no_run
/// # use std::path::Path;
/// # use session_store::{SessionId, SegmentId};
/// # use session_store::retention::SessionRetentionGuard;
/// # use session_store::public_material::PublicMaterialError;
/// # fn admit(root: &Path, session: SessionId, active: SegmentId,
/// # guard: &SessionRetentionGuard) -> Result<Vec<u64>, PublicMaterialError> {
/// // `active` was resolved from persisted Host metadata while holding `guard`.
/// session_store::capture::recording_catalog_worker_guarded(
/// root, session, active, Default::default(), guard,
/// |descriptors| descriptors.iter().map(|d| d.total_bytes).collect(),
/// )
/// # }
/// ```
pub fn recording_catalog_worker_guarded<T>(
session_root: &Path,
expected_session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
guard: &crate::retention::SessionRetentionGuard,
inspect: impl FnOnce(&[PublicMaterialDescriptor]) -> T,
) -> Result<T, PublicMaterialError> {
if guard.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
let replay = CommittedPublicReplay::open_worker_guarded(
session_root,
active_segment,
replay_limits,
None,
PublicMaterialUse::Recording,
)?;
if replay.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
Ok(inspect(&replay.into_recording_descriptors()))
}
/// FsStore-layout counterpart of [`recording_catalog_worker_guarded`].
///
/// `sessions_root` is the directory containing per-Session directories. Invoke this
/// inside the Host's `with_recording_boundary` callback, passing **its existing guard
/// and persisted active Segment**. That same external retention control store must
/// fence publishers, pointer changes, archive moves and erasure. Do not reacquire the
/// guard or use a separately opened legacy public reader inside the boundary.
/// All descriptor-only, allocation and callback guarantees of the Worker variant
/// apply; this function does not construct an unbounded transient `CaptureDto`.
pub fn recording_catalog_fs_guarded<T>(
sessions_root: &Path,
session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
guard: &crate::retention::SessionRetentionGuard,
inspect: impl FnOnce(&[PublicMaterialDescriptor]) -> T,
) -> Result<T, PublicMaterialError> {
if guard.session_id() != session {
return Err(PublicMaterialError::Inconsistent);
}
let replay = CommittedPublicReplay::open_fs_guarded(
sessions_root,
session,
active_segment,
replay_limits,
)?;
Ok(inspect(&replay.into_recording_descriptors()))
}
/// Capture the selected original lineage, including typed Host run boundaries.
/// The caller must hold the Session publication/erasure guard while selecting the
/// active pointer and through this call. Runtime publishers use this same lock
/// for appends and Segment creation. No lock is recursively acquired here.
pub fn capture_recording_worker_guarded(
session_root: &Path,
expected_session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
capture_limits: CaptureLimits,
guard: &crate::retention::SessionRetentionGuard,
) -> Result<CaptureDto, PublicMaterialError> {
if guard.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
let replay = CommittedPublicReplay::open_worker_guarded(
session_root,
active_segment,
replay_limits,
None,
PublicMaterialUse::Recording,
)?;
if replay.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
replay.into_recording_dto(capture_limits)
}
/// Standalone convenience entry; Hosts resolving a mutable active pointer should
/// instead hold the fence while resolving it and call the guarded entry.
pub fn capture_recording_worker(
session_root: &Path,
expected_session: SessionId,
active_segment: SegmentId,
retention: &SessionRetentionStore,
replay_limits: PublicMaterialLimits,
capture_limits: CaptureLimits,
) -> Result<CaptureDto, PublicMaterialError> {
let guard = retention
.lock_live(expected_session)
.map_err(|error| match error {
crate::StoreError::SessionErased(_) => PublicMaterialError::Erased,
_ => PublicMaterialError::Unavailable,
})?;
capture_recording_worker_guarded(
session_root,
expected_session,
active_segment,
replay_limits,
capture_limits,
&guard,
)
}
/// Bounded construction for one Job input: descriptor replay is bounded separately;
/// only at most the requested body budget is materialized from the fixed files.
pub fn capture_recording_slice_worker_guarded(
session_root: &Path,
expected_session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
start: RecordingPosition,
max_entries: usize,
max_serialized_bytes: usize,
guard: &crate::retention::SessionRetentionGuard,
) -> Result<RecordingCaptureSlice, PublicMaterialError> {
if guard.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
let replay = CommittedPublicReplay::open_worker_guarded(
session_root,
active_segment,
replay_limits,
None,
PublicMaterialUse::Recording,
)?;
if replay.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
replay.recording_slice(start, max_entries, max_serialized_bytes)
}
/// The standalone FsStore layout uses the same committed replay and fixed values.
pub fn capture_recording_fs_guarded(
sessions_root: &Path,
session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
capture_limits: CaptureLimits,
guard: &crate::retention::SessionRetentionGuard,
) -> Result<CaptureDto, PublicMaterialError> {
if guard.session_id() != session {
return Err(PublicMaterialError::Inconsistent);
}
CommittedPublicReplay::open_fs_guarded(sessions_root, session, active_segment, replay_limits)?
.into_recording_dto(capture_limits)
}
/// Bounded standalone capture; the caller holds its FsStore retention fence.
pub fn capture_recording_slice_fs_guarded(
sessions_root: &Path,
session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
start: RecordingPosition,
max_entries: usize,
max_serialized_bytes: usize,
guard: &crate::retention::SessionRetentionGuard,
) -> Result<RecordingCaptureSlice, PublicMaterialError> {
if guard.session_id() != session {
return Err(PublicMaterialError::Inconsistent);
}
CommittedPublicReplay::open_fs_guarded(sessions_root, session, active_segment, replay_limits)?
.recording_slice(start, max_entries, max_serialized_bytes)
}
impl CaptureDto {
/// Select the oldest canonical prefix fitting both budgets, including JSON
/// escaping, metadata and continuation. Only immutable owned bytes are read.
/// A later Job can recapture at `next`; never combine live pages into this value.
pub fn recording_slice(
&self,
start: RecordingPosition,
max_entries: usize,
max_serialized_bytes: usize,
) -> Result<RecordingCaptureSlice, CaptureError> {
if max_entries == 0 {
return Err(CaptureError::BudgetTooSmall);
}
if start.canonical_order > self.view_entry_count {
return Err(CaptureError::InvalidRange);
}
let mut result = RecordingCaptureSlice {
capture: CaptureDto {
version: self.version,
session_id: self.session_id.clone(),
segment_id: self.segment_id.clone(),
view_entry_count: self.view_entry_count,
entries: Vec::new(),
},
next: (start.canonical_order < self.view_entry_count).then_some(start),
};
if body::json_size(&result, max_serialized_bytes).is_err() {
return Err(CaptureError::BudgetTooSmall);
}
if start.canonical_order == self.view_entry_count {
if start.byte_offset != 0 {
return Err(CaptureError::InvalidRange);
}
result.capture.validate(CaptureLimits {
max_bytes: max_serialized_bytes,
max_text_bytes: max_serialized_bytes,
max_entries,
..Default::default()
})?;
return Ok(result);
}
let first = self
.entries
.iter()
.position(|entry| entry.order == start.canonical_order)
.ok_or(CaptureError::InvalidRange)?;
let mut expected_order = start.canonical_order;
for entry in self.entries.iter().skip(first).take(max_entries) {
if entry.order != expected_order {
return Err(CaptureError::InvalidRange);
}
let CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
boundary,
} = &entry.body
else {
return Err(CaptureError::Invalid("not a recording capture"));
};
if !entry.policy.recording_material {
return Err(CaptureError::PolicyDenied);
}
let offset = if entry.order == start.canonical_order {
start.byte_offset
} else {
*byte_offset
};
let local = offset
.checked_sub(*byte_offset)
.and_then(|o| usize::try_from(o).ok())
.ok_or(CaptureError::InvalidRange)?;
if local > text.len()
|| !text.is_char_boundary(local)
|| offset >= descriptor.total_bytes
{
return Err(CaptureError::InvalidRange);
}
let item = CaptureEntryDto {
entry_id: entry.entry_id.clone(),
order: entry.order,
source: entry.source.clone(),
policy: entry.policy,
body: CaptureBody::CanonicalPublic {
descriptor: descriptor.clone(),
byte_offset: offset,
text: String::new(),
boundary: *boundary,
},
};
result.capture.entries.push(item);
// At most budget bytes are considered before allocating a trial fragment.
let mut low = 0;
let mut high = text.len().saturating_sub(local).min(max_serialized_bytes);
let prior_next = result.next;
while low < high {
let mid = low + (high - low).div_ceil(2);
let mut end = local + mid;
while !text.is_char_boundary(end) {
end -= 1;
}
if let CaptureBody::CanonicalPublic { text: fragment, .. } =
&mut result.capture.entries.last_mut().unwrap().body
{
*fragment = text[local..end].to_owned();
}
let next_offset = offset + (end - local) as u64;
result.next = if next_offset < descriptor.total_bytes {
Some(RecordingPosition {
canonical_order: entry.order,
byte_offset: next_offset,
})
} else {
(entry.order + 1 < self.view_entry_count).then_some(RecordingPosition {
canonical_order: entry.order + 1,
byte_offset: 0,
})
};
if body::json_size(&result, max_serialized_bytes).is_ok() {
low = mid;
} else {
high = mid - 1;
}
}
let mut end = local + low;
while !text.is_char_boundary(end) {
end -= 1;
}
if end == local {
result.capture.entries.pop();
result.next = prior_next;
if result.capture.entries.is_empty() {
return Err(CaptureError::BudgetTooSmall);
}
break;
}
if let CaptureBody::CanonicalPublic { text: fragment, .. } =
&mut result.capture.entries.last_mut().unwrap().body
{
*fragment = text[local..end].to_owned();
}
let next_offset = offset + (end - local) as u64;
result.next = if next_offset < descriptor.total_bytes {
Some(RecordingPosition {
canonical_order: entry.order,
byte_offset: next_offset,
})
} else {
(entry.order + 1 < self.view_entry_count).then_some(RecordingPosition {
canonical_order: entry.order + 1,
byte_offset: 0,
})
};
if next_offset < descriptor.total_bytes {
break;
}
expected_order = entry.order + 1;
}
body::json_size(&result, max_serialized_bytes).map_err(|_| CaptureError::BudgetTooSmall)?;
result.capture.validate(CaptureLimits {
max_bytes: max_serialized_bytes,
max_text_bytes: max_serialized_bytes,
max_entries,
..Default::default()
})?;
Ok(result)
}
/// Resolve canonical text already in the value. No original storage is consulted.
pub fn recording_text(
&self,
order: u64,
) -> Result<(&crate::public_material::PublicMaterialDescriptor, u64, &str), CaptureError> {
let entry = self
.entries
.iter()
.find(|entry| entry.order == order)
.ok_or(CaptureError::UnknownReference)?;
if !entry.policy.recording_material {
return Err(CaptureError::PolicyDenied);
}
match &entry.body {
CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
..
} => Ok((descriptor, *byte_offset, text)),
_ => Err(CaptureError::Invalid("not canonical recording text")),
}
}
}
File diff suppressed because it is too large Load Diff
+4
View File
@@ -361,6 +361,7 @@ impl Store for FsStore {
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.log_path(session_id, segment_id), &line)
@@ -448,6 +449,9 @@ impl Store for FsStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.retention().lock_live(session_id)?;
let path = self.log_path(session_id, segment_id);
if let Some(parent) = path.parent() {
+202
View File
@@ -67,6 +67,10 @@ pub enum LoggedSessionHistoryOrigin {
ToolOutput {
worker: LoggedWorkerSubject,
},
/// Descriptive producer identity; use policies live in separate metadata.
Provided {
producer: String,
},
/// Host-produced opening context summary; not auto-read or TaskStore material.
CompactionSummary,
/// Frozen Host Journal reference, never independent observation evidence.
@@ -81,19 +85,32 @@ pub struct LoggedHistoryDerivation {
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct LoggedSessionHistoryMetadata {
pub entry_id: LoggedSessionHistoryEntryId,
pub origin: LoggedSessionHistoryOrigin,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub derivation: Option<LoggedHistoryDerivation>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub input_metadata: Option<crate::HistoryInputMetadata>,
}
impl LoggedSessionHistoryMetadata {
pub(crate) fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
if matches!(self.origin, LoggedSessionHistoryOrigin::Provided { .. })
&& self.input_metadata.is_none()
{
return Err(crate::InputMetadataError::MissingMetadata);
}
Ok(())
}
pub fn legacy_unknown() -> Self {
Self {
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
input_metadata: None,
}
}
}
@@ -101,20 +118,146 @@ impl LoggedSessionHistoryMetadata {
/// Persisted item and metadata are one value so transforms cannot reorder or
/// truncate one without the other.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(try_from = "HistoryEntryWire")]
pub struct LoggedHistoryEntry {
pub item: LoggedItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct HistoryEntryWire {
item: LoggedItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<HistoryEntryWire> for LoggedHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: HistoryEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedHistoryEntry {
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
if let Some(input) = &self.metadata.input_metadata {
match &self.item {
LoggedItem::Message {
role: crate::LoggedRole::System,
content,
} => {
let [crate::LoggedContentPart::Text { text }] = content.as_slice() else {
return Err(crate::InputMetadataError::InconsistentRecord);
};
if text.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
// Other logical roles may carry use policy, but cannot masquerade
// as the System-only public/payload extension contract.
_ if input.feature_payload().is_some() || input.public_item().is_some() => {
return Err(crate::InputMetadataError::InconsistentRecord);
}
_ => {}
}
}
Ok(())
}
}
/// Typed system-item history record. The typed system event remains available
/// to client replay while its model-visible projection carries the same stable
/// metadata used by live history.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(try_from = "SystemEntryWire")]
pub struct LoggedSystemHistoryEntry {
pub item: crate::SystemItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct SystemEntryWire {
item: crate::SystemItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<SystemEntryWire> for LoggedSystemHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: SystemEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedSystemHistoryEntry {
pub fn from_input(
origin: LoggedSessionHistoryOrigin,
derivation: Option<LoggedHistoryDerivation>,
input: crate::SystemInputEnvelope,
) -> Self {
let (body, input_metadata) = input.into_parts();
Self {
item: crate::SystemItem::Input { body },
metadata: LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId::new(),
origin,
derivation,
input_metadata: Some(input_metadata),
},
}
}
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
match (&self.item, &self.metadata.input_metadata) {
(crate::SystemItem::Input { body }, Some(_)) => {
if body.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
(crate::SystemItem::Input { .. }, None) => {
return Err(crate::InputMetadataError::MissingMetadata);
}
(_, Some(_)) => return Err(crate::InputMetadataError::InconsistentRecord),
(_, None) => {}
}
Ok(())
}
pub fn to_history_entry(&self) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: self.item.to_history_item().into(),
metadata: self.metadata.clone(),
}
}
pub fn public_item(&self) -> crate::PublicSystemItem {
if matches!(self.item, crate::SystemItem::Input { .. }) {
return self
.metadata
.input_metadata
.as_ref()
.and_then(crate::HistoryInputMetadata::public_item)
.cloned()
.unwrap_or_else(|| self.item.public_item());
}
self.item.public_item()
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -132,6 +275,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -150,6 +294,64 @@ mod tests {
);
}
#[test]
fn external_system_records_reject_missing_policy_role_mismatch_and_overlimit_body() {
let valid = serde_json::json!({
"item": { "kind": "input", "body": "context" },
"metadata": {
"entry_id": "stable-entry", "origin": { "kind": "provided", "producer": "owner" },
"input_metadata": { "usage": {
"recording_material": "exclude", "memory_evidence": "exclude",
"observation": "hidden", "compaction": "reference_input"
} }
}
});
let restored: LoggedSystemHistoryEntry = serde_json::from_value(valid.clone()).unwrap();
let flat = serde_json::to_value(restored.to_history_entry()).unwrap();
assert!(serde_json::from_value::<LoggedHistoryEntry>(flat.clone()).is_ok());
for generic in [true, false] {
let wire = if generic { &valid } else { &flat };
let decode = |wire| {
if generic {
serde_json::from_value::<LoggedSystemHistoryEntry>(wire).is_err()
} else {
serde_json::from_value::<LoggedHistoryEntry>(wire).is_err()
}
};
let mut missing = wire.clone();
missing["metadata"]
.as_object_mut()
.unwrap()
.remove("input_metadata");
assert!(decode(missing));
let mut absent_policy = wire.clone();
absent_policy["metadata"]["input_metadata"]
.as_object_mut()
.unwrap()
.remove("usage");
assert!(decode(absent_policy));
let mut invalid_role = wire.clone();
invalid_role["item"] =
serde_json::to_value(LoggedItem::from(agen::Item::user_message("not system")))
.unwrap();
invalid_role["metadata"]["input_metadata"]["public"] = serde_json::json!({
"kind": "context", "body": "public system projection", "presentation": {"kind": "context"}
});
assert!(decode(invalid_role));
let mut oversized = wire.clone();
if generic {
oversized["item"]["body"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
} else {
oversized["item"]["content"][0]["text"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
}
assert!(decode(oversized));
}
}
#[test]
fn annotated_segment_start_is_restore_visible_without_projecting_metadata() {
let session_id = uuid::Uuid::now_v7();
+563
View File
@@ -0,0 +1,563 @@
//! Validated, Feature-neutral history-input policy and System extensions.
//!
//! The enclosing history annotation owns identity, origin and derivation. None of
//! those descriptive facts selects a use policy here. Model text, independent use
//! policies, private Feature data and an explicit public projection travel as one
//! value. Restoring this value does not load a Feature or invoke domain behavior.
//!
//! Payload validation bounds the decoded value; enclosing log readers must also
//! enforce their record allocation limit before decoding untrusted persisted input.
use agen::Item;
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::PublicSystemItem;
pub const MAX_FEATURE_PAYLOAD_BYTES: usize = 128 * 1024;
pub const MAX_SYSTEM_INPUT_BYTES: usize = 4 * 1024 * 1024;
pub const MAX_SYSTEM_PUBLIC_BYTES: usize = 256 * 1024;
const MAX_SCHEMA_BYTES: usize = 128;
const MAX_JSON_DEPTH: usize = 16;
const MAX_JSON_NODES: usize = 8192;
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum InputMetadataError {
#[error("history input payload schema is invalid")]
InvalidSchema,
#[error("history input payload version must be nonzero")]
InvalidVersion,
#[error("system input resource limit exceeded")]
ResourceLimit,
#[error("history input payload could not be serialized")]
Serialization,
#[error("history input payload schema or version is unsupported by this consumer")]
UnsupportedPayload,
#[error("history input payload does not match the owner's typed contract")]
InvalidTypedPayload,
#[error("provided input is missing explicit metadata")]
MissingMetadata,
#[error("input metadata does not match its logical record")]
InconsistentRecord,
}
/// One private payload owned by a Feature's versioned typed decoder.
/// Unknown valid schemas/versions remain data, never execution instructions.
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(try_from = "PayloadWire")]
pub struct FeatureInputPayload {
schema: String,
version: u32,
value: Value,
}
impl std::fmt::Debug for FeatureInputPayload {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("FeatureInputPayload")
.field("schema", &self.schema)
.field("version", &self.version)
.finish_non_exhaustive()
}
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct PayloadWire {
schema: String,
version: u32,
value: Value,
}
impl TryFrom<PayloadWire> for FeatureInputPayload {
type Error = InputMetadataError;
fn try_from(wire: PayloadWire) -> Result<Self, Self::Error> {
if wire.schema.is_empty()
|| wire.schema.len() > MAX_SCHEMA_BYTES
|| !wire.schema.as_bytes()[0].is_ascii_lowercase()
|| !wire.schema.bytes().all(|byte| {
byte.is_ascii_lowercase() || byte.is_ascii_digit() || b"-_.:/".contains(&byte)
})
{
return Err(InputMetadataError::InvalidSchema);
}
if wire.version == 0 {
return Err(InputMetadataError::InvalidVersion);
}
validate_json(&wire.value, MAX_FEATURE_PAYLOAD_BYTES)?;
Ok(Self {
schema: wire.schema,
version: wire.version,
value: wire.value,
})
}
}
impl FeatureInputPayload {
pub fn encode<T: Serialize>(
schema: impl Into<String>,
version: u32,
value: &T,
) -> Result<Self, InputMetadataError> {
PayloadWire {
schema: schema.into(),
version,
value: serde_json::to_value(value).map_err(|_| InputMetadataError::Serialization)?,
}
.try_into()
}
pub fn schema(&self) -> &str {
&self.schema
}
pub fn version(&self) -> u32 {
self.version
}
/// The owner explicitly selects its contract before decoding. A valid unknown
/// payload can be retained and round-tripped without calling this operation.
pub fn decode<T: DeserializeOwned>(
&self,
schema: &str,
version: u32,
) -> Result<T, InputMetadataError> {
if self.schema != schema || self.version != version {
return Err(InputMetadataError::UnsupportedPayload);
}
serde_json::from_value(self.value.clone())
.map_err(|_| InputMetadataError::InvalidTypedPayload)
}
}
/// Eligibility is supplied independently for recording and Memory evidence.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputEvidenceUse {
Include,
Exclude,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputObservationUse {
Readable,
Hidden,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputCompactionUse {
/// Ordinary history eligible for the selected retained tail.
Retain,
/// A full opening summary consumed separately and superseded by compaction.
OpeningSummary,
/// Full historical context for the compactor, not an accumulating retained tail.
ReferenceInput,
}
/// No permissive defaults: malformed/unknown policies cannot turn into eligibility.
/// Public presentation is deliberately not one of these use decisions.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct HistoryInputUsePolicy {
pub recording_material: InputEvidenceUse,
pub memory_evidence: InputEvidenceUse,
pub observation: InputObservationUse,
pub compaction: InputCompactionUse,
}
/// Use policy is independent of logical role. Non-System derived Tool context
/// carries policy only; private Feature payload and public presentation belong
/// to the System envelope and are rejected on other roles at the history boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "MetadataWire")]
pub struct HistoryInputMetadata {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct MetadataWire {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
impl TryFrom<MetadataWire> for HistoryInputMetadata {
type Error = InputMetadataError;
fn try_from(wire: MetadataWire) -> Result<Self, Self::Error> {
if let Some(public) = &wire.public {
// Kind is a descriptive label, not a Feature dispatch or authority key.
if public.kind.is_empty()
|| public.kind.len() > MAX_SCHEMA_BYTES
|| public.kind.chars().any(char::is_control)
{
return Err(InputMetadataError::InvalidSchema);
}
let value =
serde_json::to_value(public).map_err(|_| InputMetadataError::Serialization)?;
validate_json(&value, MAX_SYSTEM_PUBLIC_BYTES)?;
}
Ok(Self {
usage: wire.usage,
feature_payload: wire.feature_payload,
public: wire.public,
})
}
}
impl HistoryInputMetadata {
pub fn new(
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
) -> Result<Self, InputMetadataError> {
MetadataWire {
usage,
feature_payload,
public,
}
.try_into()
}
pub fn usage(&self) -> HistoryInputUsePolicy {
self.usage
}
pub fn feature_payload(&self) -> Option<&FeatureInputPayload> {
self.feature_payload.as_ref()
}
pub fn public_item(&self) -> Option<&PublicSystemItem> {
self.public.as_ref()
}
pub fn public_use_policy(&self) -> protocol::SessionEntryUsePolicy {
protocol::SessionEntryUsePolicy {
observation_readable: self.usage.observation == InputObservationUse::Readable,
memory_evidence: self.usage.memory_evidence == InputEvidenceUse::Include,
}
}
pub fn public_system_use_policy(&self) -> protocol::SessionEntryUsePolicy {
if self.public.is_some() {
self.public_use_policy()
} else {
protocol::SessionEntryUsePolicy {
observation_readable: false,
memory_evidence: false,
}
}
}
pub fn public_projection(&self) -> PublicSystemItem {
self.public.clone().unwrap_or_else(|| PublicSystemItem {
kind: "input".into(),
body: String::new(),
presentation: crate::PublicSystemPresentation::Ignored,
})
}
}
/// System role is invariant; provenance and use policies are attached data.
/// An absent public projection never implicitly publishes the model body.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "InputWire")]
pub struct SystemInputEnvelope {
body: String,
metadata: HistoryInputMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct InputWire {
body: String,
metadata: HistoryInputMetadata,
}
impl TryFrom<InputWire> for SystemInputEnvelope {
type Error = InputMetadataError;
fn try_from(wire: InputWire) -> Result<Self, Self::Error> {
if wire.body.len() > MAX_SYSTEM_INPUT_BYTES {
return Err(InputMetadataError::ResourceLimit);
}
Ok(Self {
body: wire.body,
metadata: wire.metadata,
})
}
}
impl SystemInputEnvelope {
pub fn new(
body: impl Into<String>,
metadata: HistoryInputMetadata,
) -> Result<Self, InputMetadataError> {
InputWire {
body: body.into(),
metadata,
}
.try_into()
}
pub fn body(&self) -> &str {
&self.body
}
pub fn metadata(&self) -> &HistoryInputMetadata {
&self.metadata
}
pub fn to_history_item(&self) -> Item {
Item::system_message(self.body.clone())
}
pub fn into_parts(self) -> (String, HistoryInputMetadata) {
(self.body, self.metadata)
}
}
fn validate_json(value: &Value, max_bytes: usize) -> Result<(), InputMetadataError> {
fn walk(value: &Value, depth: usize, remaining: &mut usize) -> Result<(), InputMetadataError> {
if depth > MAX_JSON_DEPTH || *remaining == 0 {
return Err(InputMetadataError::ResourceLimit);
}
*remaining -= 1;
match value {
Value::Array(values) => {
for value in values {
walk(value, depth + 1, remaining)?;
}
}
Value::Object(values) => {
for value in values.values() {
walk(value, depth + 1, remaining)?;
}
}
_ => {}
}
Ok(())
}
let mut remaining = MAX_JSON_NODES;
walk(value, 0, &mut remaining)?;
let bytes = serde_json::to_vec(value).map_err(|_| InputMetadataError::Serialization)?;
if bytes.len() > max_bytes {
return Err(InputMetadataError::ResourceLimit);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{LoggedItem, LoggedRole, PublicSystemPresentation};
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct OwnerPayload {
source_digest: String,
omitted: bool,
}
fn reference_policy() -> HistoryInputUsePolicy {
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
}
}
#[test]
fn unknown_valid_payload_round_trips_without_domain_decode_or_public_exposure() {
let payload = FeatureInputPayload::encode(
"future-owner:context",
99,
&OwnerPayload {
source_digest: "private-digest".into(),
omitted: true,
},
)
.unwrap();
let envelope = SystemInputEnvelope::new(
"private model context",
HistoryInputMetadata::new(reference_policy(), Some(payload), None).unwrap(),
)
.unwrap();
let restored: SystemInputEnvelope =
serde_json::from_slice(&serde_json::to_vec(&envelope).unwrap()).unwrap();
assert_eq!(restored.body(), "private model context");
assert!(restored.metadata().public_item().is_none());
let saved_payload = restored.metadata().feature_payload().unwrap();
assert_eq!(saved_payload.schema(), "future-owner:context");
assert_eq!(saved_payload.version(), 99);
assert_eq!(
saved_payload.decode::<OwnerPayload>("known-owner:context", 1),
Err(InputMetadataError::UnsupportedPayload)
);
assert_eq!(
serde_json::to_value(&restored).unwrap(),
serde_json::to_value(&envelope).unwrap()
);
assert!(!format!("{saved_payload:?}").contains("private-digest"));
}
#[test]
fn owner_decodes_typed_payload_only_for_its_explicit_contract() {
let input = OwnerPayload {
source_digest: "rev".into(),
omitted: false,
};
let payload = FeatureInputPayload::encode("owner:context", 1, &input).unwrap();
assert_eq!(
payload.decode::<OwnerPayload>("owner:context", 1).unwrap(),
input
);
let invalid = FeatureInputPayload::encode(
"owner:context",
1,
&serde_json::json!({ "omitted": "not a bool" }),
)
.unwrap();
assert_eq!(
invalid.decode::<OwnerPayload>("owner:context", 1),
Err(InputMetadataError::InvalidTypedPayload)
);
}
#[test]
fn malformed_or_oversized_payload_is_rejected_at_construction_and_restore() {
for (schema, version, value) in [
("", 1, Value::Null),
("invalid schema", 1, Value::Null),
("owner:context", 0, Value::Null),
(
"owner:context",
1,
Value::String("x".repeat(MAX_FEATURE_PAYLOAD_BYTES)),
),
(
"owner:context",
1,
Value::Array(vec![Value::Null; MAX_JSON_NODES]),
),
(
"owner:context",
1,
(0..=MAX_JSON_DEPTH).fold(Value::Null, |inner, _| Value::Array(vec![inner])),
),
] {
assert!(FeatureInputPayload::encode(schema, version, &value).is_err());
assert!(
serde_json::from_value::<FeatureInputPayload>(
serde_json::json!({ "schema": schema, "version": version, "value": value })
)
.is_err()
);
}
}
#[test]
fn policy_is_required_and_unknown_or_malformed_fields_fail_closed() {
let valid = serde_json::to_value(reference_policy()).unwrap();
for field in [
"recording_material",
"memory_evidence",
"observation",
"compaction",
] {
let mut absent = valid.clone();
absent.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<HistoryInputUsePolicy>(absent).is_err());
let mut unknown = valid.clone();
unknown[field] = Value::String("future-allow".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(unknown).is_err());
}
let mut extra = valid;
extra["feature_name"] = Value::String("infer-policy-from-me".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(extra).is_err());
}
#[test]
fn feature_name_and_model_body_do_not_select_policies_or_public_projection() {
for (schema, body) in [
("owner:journal", "past context"),
("unrelated:skill", "do not extract this; legitimate wording"),
] {
let public = PublicSystemItem {
kind: "context-label".into(),
body: "explicit public text".into(),
presentation: PublicSystemPresentation::Context,
};
let metadata = HistoryInputMetadata::new(
reference_policy(),
Some(
FeatureInputPayload::encode(
schema,
1,
&OwnerPayload {
source_digest: "private".into(),
omitted: false,
},
)
.unwrap(),
),
Some(public),
)
.unwrap();
let input = SystemInputEnvelope::new(body, metadata).unwrap();
assert_eq!(input.metadata().usage(), reference_policy());
let history: LoggedItem = input.to_history_item().into();
assert!(matches!(
history,
LoggedItem::Message {
role: LoggedRole::System,
..
}
));
assert_eq!(input.to_history_item().as_text(), Some(body));
let public = serde_json::to_string(input.metadata().public_item().unwrap()).unwrap();
assert!(!public.contains("private"));
assert!(!public.contains(schema));
assert!(!public.contains(body));
}
}
#[test]
fn use_policies_remain_independent_of_each_other_and_of_public_display() {
let usage = HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Include,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Readable,
compaction: InputCompactionUse::Retain,
};
let metadata = HistoryInputMetadata::new(usage, None, None).unwrap();
let restored: HistoryInputMetadata =
serde_json::from_value(serde_json::to_value(&metadata).unwrap()).unwrap();
assert_eq!(restored.usage(), usage);
assert!(restored.public_item().is_none());
assert!(restored.feature_payload().is_none());
}
#[test]
fn model_body_and_public_projection_have_independent_resource_limits() {
let metadata = HistoryInputMetadata::new(reference_policy(), None, None).unwrap();
assert!(
SystemInputEnvelope::new("x".repeat(MAX_SYSTEM_INPUT_BYTES + 1), metadata).is_err()
);
let public = PublicSystemItem {
kind: "context".into(),
body: "x".repeat(MAX_SYSTEM_PUBLIC_BYTES),
presentation: PublicSystemPresentation::Context,
};
assert!(HistoryInputMetadata::new(reference_policy(), None, Some(public)).is_err());
let invalid = serde_json::json!({ "body": "small", "metadata": { "public": null, "feature_payload": null } });
assert!(serde_json::from_value::<SystemInputEnvelope>(invalid).is_err());
}
}
+355 -22
View File
@@ -1,12 +1,12 @@
//! Versioned decoder for Session schemas that predate canonical annotated history.
//! Versioned decoder for Session schemas v1-5, including frozen Feature wire.
//!
//! These types are intentionally private to `session-store`. Current writers,
//! replay, and public projections use [`crate::LogEntry`] exclusively; only the
//! Worker Session schema migration is allowed to deserialize these shapes.
//! explicit Session schema migration is allowed to deserialize these shapes.
use agen::llm_client::types::RequestConfig;
use protocol::Segment;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
#[cfg(test)]
mod tests;
@@ -17,6 +17,212 @@ use crate::{
SegmentOrigin, SessionExtension, SessionId, SystemItem,
};
// Frozen v1-4 Feature wire. These types are deliberately not exported and are
// never used by LogEntry/SystemItem's current deserializers. Payload structs
// retain the owner's v1 shape rather than erasing typed fields into arbitrary JSON.
#[derive(Debug, Serialize, Deserialize)]
struct JournalStartupPayload {
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: crate::PromptRenderProvenance,
}
#[derive(Debug, Serialize, Deserialize)]
struct ResidentSummaryPayload {
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SubjectBehaviorPayload {
#[serde(default)]
behavior_md: String,
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SkillActivationPayload {
name: String,
}
#[derive(Debug, Serialize, Deserialize)]
struct InvocationResultPayload {
result: protocol::FeatureInvocationResult,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyFeatureSystemItem {
JournalStartup {
body: String,
#[serde(flatten)]
payload: JournalStartupPayload,
},
ResidentSummaryRefresh {
body: String,
#[serde(flatten)]
payload: ResidentSummaryPayload,
},
SubjectBehaviorRefresh {
body: String,
#[serde(flatten)]
payload: SubjectBehaviorPayload,
},
SkillActivation {
body: String,
#[serde(flatten)]
payload: SkillActivationPayload,
},
FeatureInvocationResult {
body: String,
#[serde(flatten)]
payload: InvocationResultPayload,
},
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacySystemItem {
Feature(LegacyFeatureSystemItem),
// Includes already-migrated Input records for interrupted migration retry.
Core(SystemItem),
}
impl LegacySystemItem {
fn into_entry(
self,
mut metadata: LoggedSessionHistoryMetadata,
) -> Result<LoggedSystemHistoryEntry, crate::InputMetadataError> {
use crate::InputCompactionUse::{ReferenceInput, Retain};
let item = match self {
Self::Core(item) => item,
Self::Feature(feature) => {
metadata.validate_input_metadata()?;
if metadata.input_metadata.is_some() {
return Err(crate::InputMetadataError::InconsistentRecord);
}
let (body, input_metadata) = match feature {
LegacyFeatureSystemItem::JournalStartup { body, payload } => {
migrated_feature_input(
body,
"journal_startup",
"journal:startup",
payload,
ReferenceInput,
)?
}
LegacyFeatureSystemItem::ResidentSummaryRefresh { body, payload } => {
migrated_feature_input(
body,
"resident_summary_refresh",
"memory:resident-summary",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SubjectBehaviorRefresh { body, payload } => {
migrated_feature_input(
body,
"subject_behavior_refresh",
"subject:behavior",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SkillActivation { body, payload } => {
migrated_feature_input(
body,
"skill_activation",
"skill:activation",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::FeatureInvocationResult { body, payload } => {
migrated_feature_input(
body,
"feature_invocation_result",
"feature:invocation-result",
payload,
Retain,
)?
}
};
// Identity/origin/derivation are retained, never re-inferred.
metadata.input_metadata = Some(input_metadata);
SystemItem::Input { body }
}
};
let entry = LoggedSystemHistoryEntry { item, metadata };
entry.validate_input_metadata()?;
Ok(entry)
}
}
fn migrated_feature_input<T: Serialize>(
body: String,
label: &str,
schema: &str,
payload: T,
compaction: crate::InputCompactionUse,
) -> Result<(String, crate::HistoryInputMetadata), crate::InputMetadataError> {
let metadata = crate::HistoryInputMetadata::new(
migration_policy(compaction),
Some(crate::FeatureInputPayload::encode(schema, 1, &payload)?),
Some(crate::PublicSystemItem {
kind: label.into(),
body: body.clone(),
presentation: crate::PublicSystemPresentation::Context,
}),
)?;
Ok((body, metadata))
}
/// Annotated old wire used by v3/v4, and by already-converted files when an
/// older marker survived an interrupted migration. No ordinary decoder uses it.
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyAnnotatedLogEntry {
AnnotatedSystemItem {
ts: u64,
entry: LegacySystemHistoryEntry,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct LegacySystemHistoryEntry {
item: LegacySystemItem,
metadata: LoggedSessionHistoryMetadata,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacyAnnotatedOrCurrent {
Annotated(LegacyAnnotatedLogEntry),
Current(LogEntry),
}
impl LegacyAnnotatedOrCurrent {
fn into_current(self) -> Result<LogEntry, crate::InputMetadataError> {
match self {
Self::Annotated(LegacyAnnotatedLogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
}) => Ok(LogEntry::AnnotatedSystemItem {
ts,
entry: entry.item.into_entry(entry.metadata)?,
extensions,
}),
Self::Current(entry) => Ok(entry),
}
}
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyHistoryLogEntry {
@@ -47,7 +253,9 @@ enum LegacyHistoryLogEntry {
},
SystemItem {
ts: u64,
item: SystemItem,
item: LegacySystemItem,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
@@ -57,7 +265,7 @@ enum LegacyHistoryLogEntry {
#[serde(untagged)]
enum LegacySessionLogEntryV1 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
/// Schema v2 retained the v1 history shapes while adding non-history records.
@@ -67,7 +275,7 @@ enum LegacySessionLogEntryV1 {
#[serde(untagged)]
enum LegacySessionLogEntryV2 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
pub(crate) fn decode_entry(
@@ -77,24 +285,75 @@ pub(crate) fn decode_entry(
segment_id: SegmentId,
line_index: usize,
) -> Result<LogEntry, serde_json::Error> {
let mut wire: serde_json::Value = serde_json::from_str(line)?;
migrate_legacy_metadata_field(&mut wire)?;
let entry = match schema_version {
1 => match serde_json::from_str::<LegacySessionLogEntryV1>(line)? {
1 => match serde_json::from_value::<LegacySessionLogEntryV1>(wire)? {
LegacySessionLogEntryV1::History(entry) => Entry::History(entry),
LegacySessionLogEntryV1::Current(entry) => Entry::Current(entry),
},
2 => match serde_json::from_str::<LegacySessionLogEntryV2>(line)? {
2 => match serde_json::from_value::<LegacySessionLogEntryV2>(wire)? {
LegacySessionLogEntryV2::History(entry) => Entry::History(entry),
LegacySessionLogEntryV2::Current(entry) => Entry::Current(entry),
},
3 => Entry::Current(serde_json::from_str(line)?),
3 | 4 => Entry::Current(serde_json::from_value(wire)?),
5 => return serde_json::from_value(wire),
_ => unreachable!("legacy decoder called for unsupported schema {schema_version}"),
};
Ok(match entry {
match entry {
Entry::History(entry) => {
canonicalize_history_entry(session_id, segment_id, line_index, entry)
}
Entry::Current(entry) => entry,
})
Entry::Current(entry) => entry.into_current(),
}
.map_err(serde::de::Error::custom)
}
// The v5 System-only metadata field was widened to role-neutral input metadata
// in v6. Touch only known history slots, never arbitrary Feature payload objects.
// Both keys are an ambiguous source, not permission to discard either value.
fn migrate_legacy_metadata_field(record: &mut serde_json::Value) -> Result<(), serde_json::Error> {
fn entry(value: &mut serde_json::Value) -> Result<(), serde_json::Error> {
if let Some(metadata) = value
.get_mut("metadata")
.and_then(serde_json::Value::as_object_mut)
{
if let Some(old) = metadata.remove("system_input") {
if metadata.contains_key("input_metadata") {
return Err(serde::de::Error::custom(
"ambiguous legacy input metadata fields",
));
}
metadata.insert("input_metadata".into(), old);
}
}
Ok(())
}
let kind = record
.get("kind")
.and_then(serde_json::Value::as_str)
.unwrap_or_default()
.to_owned();
if matches!(
kind.as_str(),
"annotated_system_item"
| "annotated_assistant_item"
| "annotated_tool_result"
| "tool_result_corrected"
) {
if let Some(value) = record.get_mut("entry") {
entry(value)?;
}
}
if let Some(history) = record
.get_mut("history")
.and_then(serde_json::Value::as_array_mut)
{
for value in history {
entry(value)?;
}
}
Ok(())
}
/// Schema 3 used DerivedSummary for Host compaction seeds. Only the trusted
@@ -152,9 +411,81 @@ pub(crate) fn migrate_compaction_origins(
}
}
/// Explicit legacy-to-current boundary. No Feature inference belongs in ordinary
/// readers/restoration. Invalid or oversized payloads fail migration; the generic
/// payload/body/public limits are deliberately not relaxed for legacy data.
pub(crate) fn migrate_system_inputs(entry: &mut LogEntry) -> Result<(), crate::InputMetadataError> {
entry.validate_input_metadata()?;
match entry {
// Feature wire has already been converted by the explicit old-schema
// decoder. Ordinary LogEntry can contain only generic Input/core events.
LogEntry::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
LogEntry::AnnotatedSegmentStart { history, .. }
| LogEntry::AnnotatedUserInput { history, .. } => {
for entry in history {
migrate_flattened_system_input(entry)?;
}
}
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. }
| LogEntry::ToolResultCorrected { entry, .. } => {
migrate_flattened_system_input(entry)?;
}
_ => {}
}
entry.validate_input_metadata()
}
fn migration_policy(compaction: crate::InputCompactionUse) -> crate::HistoryInputUsePolicy {
crate::HistoryInputUsePolicy {
recording_material: crate::InputEvidenceUse::Exclude,
memory_evidence: crate::InputEvidenceUse::Exclude,
observation: crate::InputObservationUse::Hidden,
compaction,
}
}
fn migrate_flattened_system_input(
entry: &mut LoggedHistoryEntry,
) -> Result<(), crate::InputMetadataError> {
// Check before supplying legacy policy: Provided without metadata is a
// malformed NEW record, not permission to synthesize a producer policy.
entry.validate_input_metadata()?;
if entry.metadata.input_metadata.is_none()
&& (matches!(
entry.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) || matches!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
))
{
let compaction = match entry.metadata.origin {
LoggedSessionHistoryOrigin::JournalContext => crate::InputCompactionUse::ReferenceInput,
LoggedSessionHistoryOrigin::CompactionSummary => {
crate::InputCompactionUse::OpeningSummary
}
_ => crate::InputCompactionUse::Retain,
};
// Legacy derived context was excluded from independent evidence. Preserve
// that established decision explicitly, including synthetic Tool roles;
// never guess from Tool names or body text. Payload/public stay absent.
entry.metadata.input_metadata = Some(crate::HistoryInputMetadata::new(
migration_policy(compaction),
None,
None,
)?);
}
entry.validate_input_metadata()
}
enum Entry {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
fn legacy_metadata(
@@ -167,6 +498,7 @@ fn legacy_metadata(
identity.extend_from_slice(&(line_index as u64).to_be_bytes());
identity.extend_from_slice(&(item_index as u64).to_be_bytes());
LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(format!(
"l-{}",
base64::Engine::encode(&base64::engine::general_purpose::URL_SAFE_NO_PAD, identity)
@@ -181,8 +513,8 @@ fn canonicalize_history_entry(
segment_id: SegmentId,
line_index: usize,
entry: LegacyHistoryLogEntry,
) -> LogEntry {
match entry {
) -> Result<LogEntry, crate::InputMetadataError> {
Ok(match entry {
LegacyHistoryLogEntry::SegmentStart {
ts,
session_id,
@@ -236,13 +568,14 @@ fn canonicalize_history_entry(
metadata: legacy_metadata(segment_id, line_index, 0),
},
},
LegacyHistoryLogEntry::SystemItem { ts, item } => LogEntry::AnnotatedSystemItem {
LegacyHistoryLogEntry::SystemItem {
ts,
entry: LoggedSystemHistoryEntry {
item,
metadata: legacy_metadata(segment_id, line_index, 0),
},
extensions: Vec::new(),
item,
extensions,
} => LogEntry::AnnotatedSystemItem {
ts,
entry: item.into_entry(legacy_metadata(segment_id, line_index, 0))?,
extensions,
},
}
})
}
@@ -1,5 +1,7 @@
use super::*;
mod v5;
fn seed(
origin: LoggedSessionHistoryOrigin,
compacted: bool,
@@ -22,6 +24,7 @@ fn seed(
content: vec![crate::LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("summary".into()),
origin,
derivation: Some(crate::LoggedHistoryDerivation {
@@ -0,0 +1,3 @@
{"kind":"system_item","ts":1,"item":{"kind":"skill_activation","name":"legacy-skill","body":"legacy skill body"}}
{"kind":"user_input","ts":2,"segments":[{"kind":"text","content":"legacy user body"}],"extensions":[{"domain":"old","payload":{"kept":true}}]}
{"kind":"assistant_item","ts":3,"item":{"kind":"message","role":"assistant","content":[{"kind":"text","text":"legacy reply"}]}}
@@ -0,0 +1,8 @@
{"kind":"annotated_system_item","ts":101,"entry":{"item":{"kind":"journal_startup","body":"journal body\nunchanged","snapshot":{"entries":[{"text":"private snapshot"}]},"snapshot_sha256":"snapshot-digest","prompt_provenance":{"workspace_id":"workspace","source_digest":"source","projection_digest":"projection","logical_name":"journal_startup"}},"metadata":{"entry_id":"journal-id","origin":{"kind":"legacy_unknown"},"derivation":{"sources":["source-id"]}}},"extensions":[{"domain":"migration-fixture","payload":{"kept":true}}]}
{"kind":"annotated_system_item","ts":102,"entry":{"item":{"kind":"resident_summary_refresh","body":"resident body"},"metadata":{"entry_id":"resident-id","origin":{"kind":"derived_summary"}}}}
{"kind":"annotated_system_item","ts":103,"entry":{"item":{"kind":"subject_behavior_refresh","body":"rendered subject body","behavior_md":"exact behavior","prompt_provenance":{"source_digest":"subject-source","projection_digest":"subject-projection","logical_name":"subject_behavior"}},"metadata":{"entry_id":"subject-id","origin":{"kind":"backend_instruction","operation_id":"op"}}}}
{"kind":"annotated_system_item","ts":104,"entry":{"item":{"kind":"skill_activation","body":"skill body","name":"example-skill"},"metadata":{"entry_id":"skill-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":105,"entry":{"item":{"kind":"feature_invocation_result","body":"result body","result":{"invocation_id":"invoke-id","identity":"builtin:test/prepare","status":"outcome_unknown","message":"may have committed","context":null}},"metadata":{"entry_id":"result-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_user_input","ts":106,"segments":[],"history":[{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Journal] prefix is not Feature evidence"}]},"metadata":{"entry_id":"flat-journal","origin":{"kind":"journal_context"},"derivation":{"sources":["journal-id"]}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"summary body"}]},"metadata":{"entry_id":"flat-summary","origin":{"kind":"compaction_summary"}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Skill: example] never infer Feature"}]},"metadata":{"entry_id":"flat-other","origin":{"kind":"legacy_unknown"}}}]}
{"kind":"annotated_system_item","ts":107,"entry":{"item":{"kind":"notification","message":"raw message","body":"wrapped message"},"metadata":{"entry_id":"core-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":108,"entry":{"item":{"kind":"input","body":"already generic body"},"metadata":{"entry_id":"generic-id","origin":{"kind":"provided","producer":"future-owner"},"derivation":{"sources":["source-id"]},"system_input":{"usage":{"recording_material":"include","memory_evidence":"exclude","observation":"readable","compaction":"retain"},"feature_payload":{"schema":"future:unknown","version":99,"value":{"opaque":"retained"}},"public":null}}}}
@@ -0,0 +1,5 @@
{"kind":"annotated_system_item","ts":201,"entry":{"item":{"kind":"input","body":"frozen v5 reference"},"metadata":{"entry_id":"reference-v5","origin":{"kind":"provided","producer":"owner-v5"},"system_input":{"usage":{"recording_material":"exclude","memory_evidence":"exclude","observation":"hidden","compaction":"reference_input"},"feature_payload":{"schema":"future:owner","version":8,"value":{"system_input":"opaque owner field must not be renamed"}},"public":null}}}}
{"kind":"annotated_assistant_item","ts":202,"entry":{"item":{"kind":"tool_call","call_id":"original-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"original-call-entry","origin":{"kind":"model_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_tool_result","ts":203,"entry":{"item":{"kind":"tool_result","call_id":"original-call","summary":"original result"},"metadata":{"entry_id":"original-result-entry","origin":{"kind":"tool_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_assistant_item","ts":204,"entry":{"item":{"kind":"tool_call","call_id":"derived-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"derived-call-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
{"kind":"annotated_tool_result","ts":205,"entry":{"item":{"kind":"tool_result","call_id":"derived-call","summary":"derived retrieval result"},"metadata":{"entry_id":"derived-result-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
@@ -0,0 +1,709 @@
//! Fixed old wire bytes, not fixtures generated by the current serializer.
use super::*;
use crate::{FsStore, InputCompactionUse, Store, WorkerSessionStore};
use serde_json::{Value, json};
use std::{fs, path::Path};
const V4: &str = include_str!("fixtures/v4-system-inputs.jsonl");
const V1: &str = include_str!("fixtures/v1-system-inputs.jsonl");
fn decode_fixture(version: u32, source: &str, segment: SegmentId) -> Vec<LogEntry> {
source
.lines()
.enumerate()
.map(|(index, line)| decode_entry(version, line, SessionId::nil(), segment, index).unwrap())
.collect()
}
fn encode(entries: &[LogEntry]) -> Vec<u8> {
let mut bytes = Vec::new();
for entry in entries {
serde_json::to_writer(&mut bytes, entry).unwrap();
bytes.push(b'\n');
}
bytes
}
fn migrate(entries: &mut [LogEntry]) {
for entry in entries {
migrate_system_inputs(entry).unwrap();
entry.validate_input_metadata().unwrap();
}
}
fn write_worker(root: &Path, version: u32, source: &[u8]) -> (SessionId, SegmentId) {
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
fs::create_dir_all(root.join("segments")).unwrap();
fs::write(
root.join("session.json"),
serde_json::to_vec(&json!({
"schema_version": version, "session_id": session,
}))
.unwrap(),
)
.unwrap();
fs::write(
root.join("segments").join(format!("{segment}.jsonl")),
source,
)
.unwrap();
(session, segment)
}
fn assert_marker(root: &Path, version: u32) {
let manifest: Value =
serde_json::from_slice(&fs::read(root.join("session.json")).unwrap()).unwrap();
assert_eq!(manifest["schema_version"], version);
}
#[test]
fn v4_fixed_wire_preserves_identity_body_payload_public_and_independent_policy() {
let mut entries = decode_fixture(4, V4, SegmentId::nil());
let before = entries.clone();
migrate(&mut entries);
let schemas = [
"journal:startup",
"memory:resident-summary",
"subject:behavior",
"skill:activation",
"feature:invocation-result",
];
let old_wire: Vec<Value> = V4
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
for index in 0..5 {
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = &entries[index]
else {
panic!()
};
let old = &old_wire[index];
let old_item = &old["entry"]["item"];
let old_metadata: LoggedSessionHistoryMetadata =
serde_json::from_value(old["entry"]["metadata"].clone()).unwrap();
let old_ext: Vec<crate::SessionExtension> = old
.get("extensions")
.map(|value| serde_json::from_value(value.clone()).unwrap())
.unwrap_or_default();
assert_eq!(*ts, old["ts"].as_u64().unwrap());
assert_eq!(extensions, &old_ext);
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
old_item["body"].as_str().unwrap()
);
assert_eq!(entry.metadata.entry_id, old_metadata.entry_id);
assert_eq!(entry.metadata.origin, old_metadata.origin);
assert_eq!(entry.metadata.derivation, old_metadata.derivation);
let metadata = entry.metadata.input_metadata.as_ref().unwrap();
let public = metadata.public_item().unwrap();
assert_eq!(public.kind, old_item["kind"].as_str().unwrap());
assert_eq!(public.body, old_item["body"].as_str().unwrap());
assert_eq!(
public.presentation,
crate::PublicSystemPresentation::Context
);
assert_eq!(
metadata.usage(),
migration_policy(if index == 0 {
InputCompactionUse::ReferenceInput
} else {
InputCompactionUse::Retain
})
);
let payload = metadata.feature_payload().unwrap();
assert_eq!(payload.schema(), schemas[index]);
assert_eq!(payload.version(), 1);
let value: Value = payload.decode(schemas[index], 1).unwrap();
// Independently require each frozen owner's typed v1 payload contract.
match index {
0 => {
let _: JournalStartupPayload = payload.decode(schemas[index], 1).unwrap();
}
1 => {
let _: ResidentSummaryPayload = payload.decode(schemas[index], 1).unwrap();
}
2 => {
let _: SubjectBehaviorPayload = payload.decode(schemas[index], 1).unwrap();
}
3 => {
let _: SkillActivationPayload = payload.decode(schemas[index], 1).unwrap();
}
4 => {
let _: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
}
_ => unreachable!(),
}
let mut expected = old_item.clone();
expected.as_object_mut().unwrap().remove("kind");
expected.as_object_mut().unwrap().remove("body");
// Optional provenance is an explicit Option in the v1 payload contract.
if index == 1 {
expected["prompt_provenance"] = Value::Null;
}
if index == 4 {
let result: protocol::FeatureInvocationResult =
serde_json::from_value(old_item["result"].clone()).unwrap();
let typed: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
assert_eq!(typed.result, result);
// Preserve the protocol's typed serialization contract, including
// its canonical representation of optional result context.
expected["result"] = serde_json::to_value(result).unwrap();
}
assert_eq!(value, expected);
assert!(value.get("body").is_none());
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[5] else {
panic!()
};
let LogEntry::AnnotatedUserInput { history: old, .. } = &before[5] else {
panic!()
};
for (index, compaction) in [
InputCompactionUse::ReferenceInput,
InputCompactionUse::OpeningSummary,
InputCompactionUse::Retain,
]
.into_iter()
.enumerate()
{
assert_eq!(history[index].item, old[index].item);
assert_eq!(
history[index].metadata.entry_id,
old[index].metadata.entry_id
);
assert_eq!(history[index].metadata.origin, old[index].metadata.origin);
assert_eq!(
history[index].metadata.derivation,
old[index].metadata.derivation
);
let metadata = history[index].metadata.input_metadata.as_ref().unwrap();
assert_eq!(metadata.usage(), migration_policy(compaction));
assert!(metadata.public_item().is_none());
assert!(metadata.feature_payload().is_none());
}
// Core notification is unchanged; existing generic Input and unknown valid
// owner payload are retained without decoding/reclassifying a Feature.
assert_eq!(encode(&entries[6..]), encode(&before[6..]));
let once = encode(&entries);
migrate(&mut entries);
assert_eq!(encode(&entries), once);
}
#[test]
fn v1_through_v4_worker_migration_reopen_and_mixed_file_retry() {
for version in 1..=4 {
let source = if version <= 2 { V1 } else { V4 };
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), version, source.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
// Simulate an interrupted publication: v5 records precede legacy records
// in one file, and a fully new file coexists with an untouched old file.
let mut converted = decode_fixture(version, source, segment);
migrate(&mut converted);
let mut mixed = encode(&converted[..1]);
mixed.extend_from_slice(source.split_once('\n').unwrap().1.as_bytes());
fs::write(&path, &mixed).unwrap();
let new_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let new_bytes = encode(&converted);
fs::write(&new_path, &new_bytes).unwrap();
let old_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(3)));
fs::write(&old_path, source).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
WorkerSessionStore::validate_migration(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), mixed);
assert_eq!(fs::read(&old_path).unwrap(), source.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let store = WorkerSessionStore::new(root.path()).unwrap();
assert_marker(root.path(), 6);
let restored = store.read_all(session, segment).unwrap();
assert_eq!(encode(&restored), new_bytes);
assert_eq!(fs::read(&new_path).unwrap(), new_bytes);
let once = fs::read(&path).unwrap();
let old_once = fs::read(&old_path).unwrap();
drop(store);
let reopened = WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(encode(&reopened.read_all(session, segment).unwrap()), once);
assert_eq!(fs::read(&path).unwrap(), once);
assert_eq!(fs::read(&old_path).unwrap(), old_once);
}
}
#[test]
fn all_segments_preflight_keeps_old_bytes_on_decode_or_conversion_failure() {
let mut oversized: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
oversized["entry"]["item"]["snapshot"] =
json!({"large": "x".repeat(crate::input_metadata::MAX_FEATURE_PAYLOAD_BYTES + 1)});
let mut unknown: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
unknown["entry"]["item"]["kind"] = json!("unsupported_future_feature");
let mut malformed = unknown.clone();
malformed["entry"]["item"] = json!({"kind":"journal_startup", "body":"missing snapshot"});
let mut missing_metadata: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing_metadata["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let mut missing_input = missing_metadata.clone();
missing_input["entry"]["metadata"]["origin"] = json!({"kind":"legacy_unknown"});
let mut invalid_policy: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
invalid_policy["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("unknown");
let mut invalid_payload = invalid_policy.clone();
invalid_payload["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("retain");
invalid_payload["entry"]["metadata"]["system_input"]["feature_payload"]["version"] = json!(0);
let mut flat_provided: Value = serde_json::from_str(V4.lines().nth(5).unwrap()).unwrap();
flat_provided["history"][0]["metadata"]["origin"] =
json!({"kind":"provided", "producer":"owner"});
for bad in [
"{broken-json}\n".to_owned(),
format!("{}\n", oversized),
format!("{}\n", unknown),
format!("{}\n", malformed),
format!("{}\n", missing_metadata),
format!("{}\n", invalid_policy),
format!("{}\n", invalid_payload),
format!("{}\n", flat_provided),
format!("{}\n", missing_input),
] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 4, V4.as_bytes());
let first = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&later, &bad).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::validate_migration(root.path()).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read(&later).unwrap(), bad.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn unsupported_schema_preserves_all_old_bytes() {
for version in [0, 7, 99] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V4.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), V4.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn standalone_store_uses_same_v1_through_v4_migration_and_reopens() {
// None represents the existing unmarked annotated standalone layout (v3).
for version in [None, Some(1), Some(2), Some(3), Some(4)] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let source = if version.is_some_and(|v| v <= 2) {
V1
} else {
V4
};
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, source).unwrap();
if let Some(version) = version {
fs::write(dir.join("session-schema.json"), version.to_string()).unwrap();
}
let store = FsStore::new(root.path()).unwrap();
let entries = store.read_all(session, segment).unwrap();
for entry in entries {
entry.validate_input_metadata().unwrap();
}
assert_eq!(
fs::read_to_string(dir.join("session-schema.json"))
.unwrap()
.trim(),
"6"
);
let once = fs::read(&path).unwrap();
drop(store);
FsStore::new(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), once);
}
}
#[test]
fn compaction_origin_upgrade_is_retained_only_for_pre_v4_sources() {
for version in [3, 4] {
let root = tempfile::tempdir().unwrap();
let creation = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
true,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let (session, segment) =
write_worker(root.path(), version, &encode(&[creation, inherited]));
let store = WorkerSessionStore::new(root.path()).unwrap();
for record in store.read_all(session, segment).unwrap() {
let LogEntry::AnnotatedSegmentStart { history, .. } = record else {
panic!()
};
let metadata = &history[0].metadata;
assert_eq!(metadata.entry_id.0, "summary");
assert_eq!(metadata.derivation.as_ref().unwrap().sources[0].0, "source");
assert_eq!(
metadata.origin,
if version == 3 {
LoggedSessionHistoryOrigin::CompactionSummary
} else {
LoggedSessionHistoryOrigin::DerivedSummary
}
);
assert_eq!(
metadata.input_metadata.as_ref().unwrap().usage().compaction,
if version == 3 {
InputCompactionUse::OpeningSummary
} else {
InputCompactionUse::Retain
}
);
}
}
}
#[test]
fn standalone_preflight_and_unsupported_marker_leave_sources_unchanged() {
for version in [4, 99] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let first = dir.join(format!("{}.jsonl", SegmentId::from_u128(1)));
let later = dir.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&first, V4).unwrap();
fs::write(&later, "{broken-json}\n").unwrap();
let marker = version.to_string();
fs::write(dir.join("session-schema.json"), &marker).unwrap();
assert!(FsStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read_to_string(&later).unwrap(), "{broken-json}\n");
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
marker
);
}
}
#[test]
fn current_v5_rejects_all_legacy_feature_kinds_without_rewriting_sources() {
for line in V4.lines().take(5) {
// Current serde, writable open and read-only restore share the same
// current wire contract. None may dispatch to the old Feature decoder.
assert!(serde_json::from_str::<LogEntry>(line).is_err());
let bytes = format!("{line}\n");
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), 6, bytes.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
let readonly = WorkerSessionStore::open_read_only(root.path()).unwrap();
assert!(readonly.read_all_read_only(session, segment).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let standalone = tempfile::tempdir().unwrap();
let dir = standalone.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, &bytes).unwrap();
fs::write(dir.join("session-schema.json"), "5").unwrap();
assert!(FsStore::new(standalone.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
"5"
);
}
}
#[test]
fn v1_v2_feature_wire_uses_the_same_typed_payloads_and_stable_legacy_identity() {
let segment = SegmentId::from_u128(42);
for version in [1, 2] {
for (index, line) in V4.lines().take(5).enumerate() {
let annotated: Value = serde_json::from_str(line).unwrap();
let wire = json!({
"kind": "system_item", "ts": annotated["ts"],
"item": annotated["entry"]["item"],
"extensions": [{"domain":"old-owner", "payload":{"kept":true}}],
})
.to_string();
let entry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
let retry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
assert_eq!(encode(&[entry.clone()]), encode(&[retry]));
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = entry
else {
panic!()
};
assert_eq!(ts, annotated["ts"].as_u64().unwrap());
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
annotated["entry"]["item"]["body"].as_str().unwrap()
);
assert_eq!(
entry.metadata.entry_id,
legacy_metadata(segment, index, 0).entry_id
);
assert_eq!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::LegacyUnknown
);
assert!(entry.metadata.derivation.is_none());
assert_eq!(extensions[0].domain, "old-owner");
assert_eq!(extensions[0].payload, json!({"kept":true}));
let LogEntry::AnnotatedSystemItem { entry: v4, .. } =
decode_entry(4, line, SessionId::nil(), segment, index).unwrap()
else {
panic!()
};
assert_eq!(entry.metadata.input_metadata, v4.metadata.input_metadata);
}
}
}
#[test]
fn migration_rejects_malformed_typed_feature_fields_instead_of_retaining_unchecked_json() {
let mut items: Vec<Value> = V4
.lines()
.take(5)
.map(|line| serde_json::from_str::<Value>(line).unwrap()["entry"]["item"].clone())
.collect();
items[0]["snapshot_sha256"] = json!(17);
items[1]["prompt_provenance"] = json!({"logical_name":"missing digests"});
items[2]["behavior_md"] = json!(["not a string"]);
items[3]["name"] = json!({"not":"a name"});
items[4]["result"]["status"] = json!("unsupported_status");
for version in 1..=4 {
for item in &items {
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"malformed-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).is_err());
}
}
}
#[test]
fn frozen_behavior_counter_is_accepted_only_at_the_old_schema_boundary() {
for version in 1..=4 {
let item = json!({
"kind":"subject_behavior_refresh", "body":"exact archived body",
"behavior_revision":7,
});
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"behavior-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(serde_json::from_str::<LogEntry>(&wire).is_err());
let LogEntry::AnnotatedSystemItem { entry, .. } =
decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).unwrap()
else {
panic!()
};
assert_eq!(entry.item.history_text(), "exact archived body");
let payload = entry
.metadata
.input_metadata
.as_ref()
.unwrap()
.feature_payload()
.unwrap();
let typed: SubjectBehaviorPayload = payload.decode("subject:behavior", 1).unwrap();
assert_eq!(typed.behavior_md, "");
assert!(typed.prompt_provenance.is_none());
assert_eq!(
serde_json::to_value(typed).unwrap(),
json!({
"behavior_md":"", "prompt_provenance":null,
})
);
}
}
#[test]
fn current_schema_rejects_generic_input_without_metadata_instead_of_migrating_it() {
let mut missing: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let bytes = format!("{}\n", missing);
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 6, bytes.as_bytes());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(
fs::read(
root.path()
.join("segments")
.join(format!("{segment}.jsonl"))
)
.unwrap(),
bytes.as_bytes()
);
assert_marker(root.path(), 6);
}
const V5: &str = include_str!("fixtures/v5-role-metadata.jsonl");
#[test]
fn v5_metadata_and_derived_tool_context_upgrade_without_changing_roles_or_original_evidence() {
for standalone in [false, true] {
let root = tempfile::tempdir().unwrap();
let (session, segment) = if standalone {
let session = SessionId::now_v7();
let segment = SegmentId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
fs::write(dir.join("session-schema.json"), b"5").unwrap();
fs::write(dir.join(format!("{segment}.jsonl")), V5).unwrap();
(session, segment)
} else {
write_worker(root.path(), 5, V5.as_bytes())
};
let entries = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
let originals: Vec<Value> = V5
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
assert_eq!(entries.len(), originals.len());
for (index, (entry, old)) in entries.iter().zip(originals).enumerate() {
let wire = serde_json::to_value(entry).unwrap();
assert_eq!(wire["ts"], old["ts"]);
assert_eq!(wire["entry"]["item"], old["entry"]["item"]);
for field in ["entry_id", "origin", "derivation"] {
assert_eq!(
wire["entry"]["metadata"][field],
old["entry"]["metadata"][field]
);
}
let metadata = &wire["entry"]["metadata"];
if index == 0 {
assert_eq!(
metadata["input_metadata"],
old["entry"]["metadata"]["system_input"]
);
assert_eq!(
metadata["input_metadata"]["feature_payload"]["value"]["system_input"],
"opaque owner field must not be renamed"
);
} else if index < 3 {
assert!(
metadata.get("input_metadata").is_none(),
"ordinary model Tool originals must not acquire exclusion policy"
);
} else {
assert_eq!(
metadata["input_metadata"]["usage"],
json!({
"recording_material":"exclude", "memory_evidence":"exclude", "observation":"hidden", "compaction":"retain"
})
);
}
}
let before = encode(&entries);
let reopened = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
assert_eq!(encode(&reopened), before);
}
}
#[test]
fn legacy_metadata_alias_conflict_fails_preflight_without_dropping_either_value() {
let mut bad: Value = serde_json::from_str(V5.lines().next().unwrap()).unwrap();
bad["entry"]["metadata"]["input_metadata"] = bad["entry"]["metadata"]["system_input"].clone();
for version in [4, 5] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V5.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let bad_bytes = format!("{bad}\n");
fs::write(&later, &bad_bytes).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(path).unwrap(), V5.as_bytes());
assert_eq!(fs::read(later).unwrap(), bad_bytes.as_bytes());
assert_marker(root.path(), version);
}
}
+9 -1
View File
@@ -38,9 +38,12 @@ pub(crate) fn read_without_atime_bounded(path: &Path, max_bytes: u64) -> io::Res
}
}
pub mod capture;
mod committed_replay;
pub mod event_trace;
pub mod fs_store;
pub mod history;
pub mod input_metadata;
mod legacy_session_log;
pub mod logged_item;
mod paste_artifact;
@@ -65,6 +68,10 @@ pub use history::{
LoggedSessionHistoryMetadata, LoggedSessionHistoryOrigin, LoggedSystemHistoryEntry,
LoggedWorkerSubject,
};
pub use input_metadata::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputMetadataError, InputObservationUse, SystemInputEnvelope,
};
pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged};
pub use paste_artifact::PasteArtifactLimits;
pub use public_index::{
@@ -98,7 +105,8 @@ pub use segment::{
pub use segment_log::{LogEntry, RestoredState, SegmentOrigin, SessionExtension, collect_state};
pub use store::{Store, StoreError};
pub use system_item::{
PromptRenderProvenance, SystemItem, SystemReminder, SystemReminderSource, render_worker_event,
PromptRenderProvenance, PublicSystemItem, PublicSystemPresentation, SystemItem, SystemReminder,
SystemReminderSource, render_worker_event,
};
pub use uploaded_file::{
DEFAULT_MAX_FILES_PER_SUBMISSION, DEFAULT_MAX_SESSION_ARTIFACT_BYTES,
+2
View File
@@ -1296,6 +1296,7 @@ mod tests {
LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.to_owned()),
origin,
derivation: None,
@@ -1537,6 +1538,7 @@ mod tests {
prompt_provenance: None,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("system-item".into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction {
operation_id: None,
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+73 -21
View File
@@ -27,7 +27,7 @@ use thiserror::Error;
use crate::{
LogEntry, LoggedContentPart, LoggedHistoryEntry, LoggedItem, LoggedRole,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError, SystemItem,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError,
WorkerAggregateStore, WorkerSessionStore, WorkerStoreError,
};
@@ -1165,7 +1165,7 @@ fn capture_tool_correction(
let LogEntry::ToolResultCorrected { entry, .. } = record else {
return false;
};
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
// We read backwards: the first correction seen is the latest one.
corrections
.entry(entry.metadata.entry_id.0.clone())
@@ -1600,12 +1600,12 @@ pub(crate) fn project_session_snapshot_for_segment(
} => extend_history(&mut entries, history, Some(segments), *ts),
LogEntry::AnnotatedAssistantItem { ts, entry }
| LogEntry::AnnotatedToolResult { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
entries.push(history_entry(entry, *ts, data));
}
}
LogEntry::ToolResultCorrected { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
if let Some(existing) = entries
.iter_mut()
.find(|existing| existing.entry_id == entry.metadata.entry_id.0)
@@ -1618,11 +1618,16 @@ pub(crate) fn project_session_snapshot_for_segment(
}
}
LogEntry::AnnotatedSystemItem { ts, entry, .. } => entries.push(system_entry(
&entry.item,
&entry.public_item(),
entry.metadata.entry_id.0.clone(),
*ts,
provenance(&entry.metadata.origin),
derivation_ids(entry),
entry
.metadata
.input_metadata
.as_ref()
.map(crate::HistoryInputMetadata::public_system_use_policy),
)),
LogEntry::RunYielded {
ts,
@@ -1775,7 +1780,7 @@ fn extend_history(
segments: input_segments.cloned().unwrap_or_default(),
}
} else {
let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) else {
let Some(data) = project_history_item(entry) else {
continue;
};
data
@@ -1790,6 +1795,19 @@ fn history_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: entry.metadata.input_metadata.as_ref().map(|input| {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
input.public_system_use_policy()
} else {
input.public_use_policy()
}
}),
entry_id: entry.metadata.entry_id.0.clone(),
timestamp,
provenance: provenance(&entry.metadata.origin),
@@ -1833,6 +1851,7 @@ fn run_transition_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: persisted_entry_id
.map(|entry_id| entry_id.0.clone())
.unwrap_or_else(|| legacy_entry_id(session_key, segment_id, log_index, 0)),
@@ -1852,6 +1871,7 @@ fn legacy_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: legacy_entry_id(session_key, segment_id, log_index, item_index),
timestamp,
provenance: SessionEntryProvenance::LegacyUnknown,
@@ -1884,9 +1904,8 @@ fn provenance(origin: &LoggedSessionHistoryOrigin) -> SessionEntryProvenance {
LoggedSessionHistoryOrigin::FlowInstruction { .. } => {
SessionEntryProvenance::FlowInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. } => {
SessionEntryProvenance::BackendInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. }
| LoggedSessionHistoryOrigin::Provided { .. } => SessionEntryProvenance::BackendInstruction,
LoggedSessionHistoryOrigin::ModelOutput { .. } => SessionEntryProvenance::ModelOutput,
LoggedSessionHistoryOrigin::ToolOutput { .. } => SessionEntryProvenance::ToolOutput,
LoggedSessionHistoryOrigin::CompactionSummary
@@ -1914,6 +1933,35 @@ pub fn session_tool_attachment_id(
URL_SAFE_NO_PAD.encode(hash.finalize())
}
fn project_history_item(entry: &LoggedHistoryEntry) -> Option<SessionSnapshotEntryData> {
if let Some(input) = &entry.metadata.input_metadata {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
let public = input.public_projection();
return Some(SessionSnapshotEntryData::SystemItem {
item_kind: public.kind.clone(),
content: public.body.clone(),
data: Some(
serde_json::to_value(public).expect("public system input is serializable"),
),
});
}
let usage = input.usage();
if usage.recording_material == crate::InputEvidenceUse::Exclude
&& usage.memory_evidence == crate::InputEvidenceUse::Exclude
&& usage.observation == crate::InputObservationUse::Hidden
{
return None;
}
}
project_item(&entry.metadata.entry_id.0, &entry.item)
}
fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntryData> {
match item {
LoggedItem::Message { role, content } => {
@@ -1986,30 +2034,26 @@ fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntr
}
fn system_entry(
item: &SystemItem,
item: &crate::PublicSystemItem,
entry_id: String,
timestamp: u64,
provenance: SessionEntryProvenance,
derived_from: Vec<String>,
use_policy: Option<protocol::SessionEntryUsePolicy>,
) -> SessionSnapshotEntry {
let mut data = serde_json::to_value(item).ok();
if let Some(serde_json::Value::Object(object)) = data.as_mut() {
object.remove("prompt_provenance");
}
let item_kind = data
.as_ref()
.and_then(|value| value.get("kind"))
.and_then(serde_json::Value::as_str)
.unwrap_or("system_item")
.to_owned();
let public = item.clone();
let data = Some(serde_json::to_value(&public).expect("public system input is serializable"));
let item_kind = public.kind;
let content = public.body;
SessionSnapshotEntry {
use_policy,
entry_id,
timestamp,
provenance,
derived_from,
data: SessionSnapshotEntryData::SystemItem {
item_kind,
content: item.history_text(),
content,
data,
},
}
@@ -2291,6 +2335,7 @@ mod tests {
content: vec![LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(entry_id.into()),
origin,
derivation: None,
@@ -3589,6 +3634,7 @@ mod tests {
.map(|item| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -3643,6 +3689,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id,
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -3712,6 +3759,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::FlowInstruction {
selector: "builtin:coder-review".into(),
@@ -3731,6 +3779,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id.clone(),
origin: origin.clone(),
derivation: Some(LoggedHistoryDerivation {
@@ -3761,6 +3810,7 @@ mod tests {
fn annotated_projection_preserves_identity_and_provenance() {
let session_id = crate::new_session_id();
let metadata = LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::ModelOutput {
worker: LoggedWorkerSubject {
@@ -4323,6 +4373,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("ancestor-image".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4417,6 +4468,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("tool-images".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -103,6 +103,7 @@ fn finish(
stats.insert(
anchor.clone(),
SessionSnapshotEntry {
use_policy: None,
entry_id: format!("run-stats:{anchor}"),
timestamp: ended_at,
provenance: SessionEntryProvenance::LegacyUnknown,
+11 -1
View File
@@ -46,6 +46,13 @@ pub struct SessionRetentionStore {
/// Do not recursively acquire the same Session lock inside a guarded callback.
pub struct SessionRetentionGuard {
_file: File,
session_id: SessionId,
}
impl SessionRetentionGuard {
pub fn session_id(&self) -> SessionId {
self.session_id
}
}
impl SessionRetentionStore {
@@ -71,7 +78,10 @@ impl SessionRetentionStore {
}
let file = options.open(self.root.join(format!("{session}.lock")))?;
FileExt::lock_exclusive(&file)?;
Ok(SessionRetentionGuard { _file: file })
Ok(SessionRetentionGuard {
_file: file,
session_id: session,
})
}
fn read(&self, session: SessionId) -> Result<RetentionState, StoreError> {
+18
View File
@@ -254,6 +254,23 @@ pub struct SegmentOrigin {
}
impl LogEntry {
/// Validate generic System records before a Store publishes any bytes.
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
match self {
Self::AnnotatedSegmentStart { history, .. }
| Self::AnnotatedUserInput { history, .. } => {
for entry in history {
entry.validate_input_metadata()?;
}
}
Self::AnnotatedAssistantItem { entry, .. }
| Self::AnnotatedToolResult { entry, .. }
| Self::ToolResultCorrected { entry, .. } => entry.validate_input_metadata()?,
Self::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
_ => {}
}
Ok(())
}
/// Allocate the durable public identity for a newly committed Run
/// transition. Calling this repeatedly is idempotent so the exact value
/// written to storage is also the value published to live subscribers.
@@ -537,6 +554,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
+3
View File
@@ -27,6 +27,9 @@ pub enum StoreError {
#[error("serialization error: {0}")]
Serde(#[from] serde_json::Error),
#[error("invalid history input metadata: {0}")]
InvalidInputMetadata(#[from] crate::InputMetadataError),
#[error("segment not found: {0}")]
NotFound(SegmentId),
+92 -103
View File
@@ -86,20 +86,16 @@ pub struct PromptRenderProvenance {
/// `notify_wrapper` template) re-shape the live rendering on a later
/// resume.
///
/// New variants get added here as fresh injection kinds come online
/// (e.g. `TaskReminder`). The `kind` JSON tag is the snake_case form of
/// the variant name.
/// Feature-owned inputs use `Input` plus explicit annotated metadata; adding a
/// Feature must not extend this enum or ordinary replay. Only core events have
/// dedicated variants. Pre-v5 Feature kinds exist exclusively in the migration
/// decoder, not in this current wire contract.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum SystemItem {
/// One-time derived Journal reference context. The bounded Host snapshot is
/// retained as JSON independently of its rendered body and prompt projection.
JournalStartup {
body: String,
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: PromptRenderProvenance,
},
/// Generic System body; private metadata and public presentation are supplied
/// independently by the producer in the same annotated history record.
Input { body: String },
/// Free-form notification sent in by an external caller via
/// `Method::Notify`. `message` is the raw caller-supplied text;
/// `body` is the wrapped LLM-context form (Worker renders it via
@@ -122,14 +118,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Result of an explicitly selected chat Feature invocation. The typed
/// result is retained for clients; `body` is the exact bounded context made
/// visible to the Worker model.
FeatureInvocationResult {
result: protocol::FeatureInvocationResult,
body: String,
},
/// `@<path>` file reference resolution. `body` is the rendered
/// LLM-context text (`[File: <path>]\n…` for regular files,
/// `[Dir: <path>]\n…` for directory listings, possibly with a
@@ -137,10 +125,6 @@ pub enum SystemItem {
/// byte-identical to what was sent.
FileAttachment { path: String, body: String },
/// Explicit Agent Skill activation. `body` is the exact LLM-facing
/// Skill text committed before any subsequent LLM request can use it.
SkillActivation { name: String, body: String },
/// Historical persisted Knowledge reference resolution. Knowledge is no
/// longer active, so restored sessions ignore this item instead of replaying
/// archived record text into model context.
@@ -164,29 +148,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Restore-boundary refresh of the current resident Memory projection.
/// The exact rendered body is retained so replay is byte-identical; a
/// refresh with no ready summary explicitly supersedes older resident
/// context without deleting or rewriting history.
ResidentSummaryRefresh {
body: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// A Host-fetched change to user-managed Subject behavior. Each changed
/// body is appended rather than rewriting prior prompt/session content.
SubjectBehaviorRefresh {
body: String,
/// Exact user-managed behavior acknowledged after durable commit. Old
/// logs omit this value; an empty value is not an acknowledgement of a
/// nonempty behavior and does not replace their stored rendered body.
#[serde(default)]
behavior_md: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Synthetic note inserted after an interrupted turn before the next
/// user input. `body` is the exact LLM-context text explaining that the
/// previous turn was cut short.
@@ -197,24 +158,67 @@ pub enum SystemItem {
},
}
/// Allowlisted presentation data shared by live events and retained snapshots.
/// Private Feature payloads and render provenance never cross this boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct PublicSystemItem {
pub kind: String,
pub body: String,
pub presentation: PublicSystemPresentation,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PublicSystemPresentation {
Notification { message: String },
WorkerEvent { event: WorkerEvent },
Context,
TaskReminder,
Ignored,
}
impl SystemItem {
pub fn public_item(&self) -> PublicSystemItem {
let presentation = match self {
Self::Notification { message, .. } => PublicSystemPresentation::Notification {
message: message.clone(),
},
Self::WorkerEvent { event, .. } => PublicSystemPresentation::WorkerEvent {
event: event.clone(),
},
Self::TaskReminder { .. } => PublicSystemPresentation::TaskReminder,
Self::Input { .. }
| Self::LegacyKnowledgeIgnored { .. }
| Self::LegacyIgnored { .. } => PublicSystemPresentation::Ignored,
Self::FileAttachment { .. } | Self::Interrupt { .. } => {
PublicSystemPresentation::Context
}
};
PublicSystemItem {
kind: self.kind_label().to_owned(),
body: if matches!(presentation, PublicSystemPresentation::Ignored) {
String::new()
} else {
self.history_text()
},
presentation,
}
}
/// Free-text body the LLM sees inside its `role:system` message
/// for this item. Returns the variant's stored `body` verbatim.
pub fn history_text(&self) -> String {
match self {
SystemItem::JournalStartup { body, .. } => body.clone(),
SystemItem::Input { body } => body.clone(),
SystemItem::Notification { body, .. } => body.clone(),
SystemItem::WorkerEvent { body, .. } => body.clone(),
SystemItem::FeatureInvocationResult { body, .. } => body.clone(),
SystemItem::FileAttachment { body, .. } => body.clone(),
SystemItem::SkillActivation { body, .. } => body.clone(),
SystemItem::LegacyKnowledgeIgnored { .. } => String::new(),
SystemItem::LegacyIgnored { slug } => {
format!("Ignored legacy procedure item: /{slug}")
}
SystemItem::TaskReminder { body, .. } => body.clone(),
SystemItem::ResidentSummaryRefresh { body, .. } => body.clone(),
SystemItem::SubjectBehaviorRefresh { body, .. } => body.clone(),
SystemItem::Interrupt { body, .. } => body.clone(),
}
}
@@ -229,17 +233,13 @@ impl SystemItem {
/// wire — keep flexible.
pub fn kind_label(&self) -> &'static str {
match self {
SystemItem::JournalStartup { .. } => "journal_startup",
SystemItem::Input { .. } => "input",
SystemItem::Notification { .. } => "notification",
SystemItem::WorkerEvent { .. } => "worker_event",
SystemItem::FeatureInvocationResult { .. } => "feature_invocation_result",
SystemItem::FileAttachment { .. } => "file_attachment",
SystemItem::SkillActivation { .. } => "skill_activation",
SystemItem::LegacyKnowledgeIgnored { .. } => "legacy_knowledge_ignored",
SystemItem::LegacyIgnored { .. } => "legacy_ignored",
SystemItem::TaskReminder { .. } => "task_reminder",
SystemItem::ResidentSummaryRefresh { .. } => "resident_summary_refresh",
SystemItem::SubjectBehaviorRefresh { .. } => "subject_behavior_refresh",
SystemItem::Interrupt { .. } => "interrupt",
}
}
@@ -297,33 +297,45 @@ mod tests {
}
#[test]
fn subject_behavior_refresh_round_trips_with_exact_body() {
let item = SystemItem::SubjectBehaviorRefresh {
body: "current Subject context".to_string(),
behavior_md: "exact behavior".to_string(),
prompt_provenance: None,
};
let raw = serde_json::to_string(&item).unwrap();
let parsed: SystemItem = serde_json::from_str(&raw).unwrap();
assert_eq!(parsed.history_text(), "current Subject context");
assert!(
matches!(parsed, SystemItem::SubjectBehaviorRefresh { behavior_md, .. } if behavior_md == "exact behavior")
);
fn current_system_item_rejects_frozen_feature_kinds() {
// Even well-formed old payloads cannot enter normal v5 decode/replay.
for line in include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl")
.lines()
.take(5)
{
let record: serde_json::Value = serde_json::from_str(line).unwrap();
assert!(serde_json::from_value::<SystemItem>(record["entry"]["item"].clone()).is_err());
assert!(
serde_json::from_value::<crate::LoggedSystemHistoryEntry>(record["entry"].clone())
.is_err()
);
assert!(serde_json::from_str::<crate::LogEntry>(line).is_err());
}
}
#[test]
fn frozen_subject_behavior_log_reads_old_counter_without_republishing_it() {
let old = r#"{"kind":"subject_behavior_refresh","body":"exact archived body","behavior_revision":7}"#;
let item: SystemItem = serde_json::from_str(old).unwrap();
assert_eq!(item.history_text(), "exact archived body");
assert_eq!(
serde_json::to_value(item).unwrap(),
serde_json::json!({
"kind": "subject_behavior_refresh",
"body": "exact archived body",
"behavior_md": ""
})
);
fn public_system_input_decoder_rejects_private_fields_and_unknown_presentations() {
let valid = serde_json::json!({
"kind": "new-feature-label",
"body": "explicit public content",
"presentation": { "kind": "context" },
});
let public: PublicSystemItem = serde_json::from_value(valid.clone()).unwrap();
assert_eq!(public.body, "explicit public content");
for private_field in [
"payload",
"prompt_provenance",
"policy",
"snapshot",
"behavior_md",
] {
let mut malformed = valid.clone();
malformed[private_field] = serde_json::json!({ "private": true });
assert!(serde_json::from_value::<PublicSystemItem>(malformed).is_err());
}
let mut unknown = valid;
unknown["presentation"] = serde_json::json!({ "kind": "execute_feature_payload" });
assert!(serde_json::from_value::<PublicSystemItem>(unknown).is_err());
}
#[test]
@@ -419,29 +431,6 @@ mod tests {
}
}
#[test]
fn invocation_result_retains_typed_outcome_and_exact_context() {
let result = protocol::FeatureInvocationResult {
invocation_id: "stable-invoke".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
status: protocol::FeatureInvocationStatus::OutcomeUnknown,
message: "operation may have committed".into(),
context: None,
};
let body = "Outcome unknown; do not retry automatically";
let item = SystemItem::FeatureInvocationResult {
result: result.clone(),
body: body.into(),
};
let restored: SystemItem =
serde_json::from_str(&serde_json::to_string(&item).unwrap()).unwrap();
assert_eq!(restored.history_text(), body);
assert_eq!(restored.kind_label(), "feature_invocation_result");
assert!(
matches!(restored, SystemItem::FeatureInvocationResult { result: saved, .. } if saved == result)
);
}
#[test]
fn round_trip_worker_event() {
let item = SystemItem::WorkerEvent {
+21 -43
View File
@@ -210,18 +210,6 @@ impl WorkerMetadata {
}
}
fn replace_active_session(metadata: &mut WorkerMetadata, active: Option<WorkerActiveSegmentRef>) {
if let Some(session_id) = active.as_ref().map(|active| active.session_id)
&& metadata
.subjektiv_session_attribution
.as_ref()
.is_some_and(|state| state.session_id() != session_id)
{
metadata.subjektiv_session_attribution = None;
}
metadata.active = active;
}
/// Sync persistence backend for Worker metadata.
pub trait WorkerMetadataStore: Send + Sync {
/// Create or replace metadata for its `worker_name` key.
@@ -297,7 +285,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -317,7 +305,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_id: Option<String>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -334,7 +322,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_root) = workspace_root {
metadata.workspace_root = Some(workspace_root);
@@ -342,18 +330,6 @@ pub trait WorkerMetadataStore: Send + Sync {
})
}
/// Set the Host-owned Subjektiv Session attribution lifecycle state while
/// preserving the active Session pointer and every unrelated field.
fn set_subjektiv_session_attribution(
&self,
worker_name: &str,
state: SubjektivSessionAttributionState,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.subjektiv_session_attribution = Some(state);
})
}
/// Set spawned-child registry state while preserving active pointer and manifest snapshot.
fn set_spawned_children(
&self,
@@ -1087,13 +1063,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
});
})
.unwrap();
store
.set_active(
@@ -1117,7 +1093,7 @@ mod tests {
}
#[test]
fn new_active_session_clears_prior_session_attribution_state() {
fn generic_active_update_does_not_interpret_host_attribution() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let old_session_id = crate::new_session_id();
@@ -1129,13 +1105,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
});
})
.unwrap();
store
@@ -1151,7 +1127,9 @@ mod tests {
restored.active,
Some(WorkerActiveSegmentRef::pending_segment(new_session_id))
);
assert!(restored.subjektiv_session_attribution.is_none());
assert!(
matches!(restored.subjektiv_session_attribution, Some(SubjektivSessionAttributionState::Confirmed { session_id, .. }) if session_id == old_session_id)
);
}
#[test]
+241 -15
View File
@@ -36,7 +36,9 @@ use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::SystemTime;
const SESSION_SCHEMA_VERSION: u32 = 4;
const SESSION_SCHEMA_VERSION: u32 = 6;
const SYSTEM_INPUT_SESSION_SCHEMA_VERSION: u32 = 5;
const COMPACTION_SESSION_SCHEMA_VERSION: u32 = 4;
const ANNOTATED_SESSION_SCHEMA_VERSION: u32 = 3;
const PREVIOUS_SESSION_SCHEMA_VERSION: u32 = 2;
const LEGACY_SESSION_SCHEMA_VERSION: u32 = 1;
@@ -192,6 +194,21 @@ impl WorkerSessionStore {
Ok(store)
}
/// Open a current retained Session without creating or migrating anything.
/// The live constraint is checked here and again by every later publication;
/// attaching the control store itself does not acquire publication authority.
pub fn open_existing_with_retention(
root: impl Into<PathBuf>,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, StoreError> {
let mut store = Self::open_read_only(root)?;
if let Some(session) = *store.session_id.lock().unwrap() {
retention.ensure_live(session)?;
}
store.retention = Some(retention);
Ok(store)
}
fn lock_publication(
&self,
session: SessionId,
@@ -204,7 +221,15 @@ impl WorkerSessionStore {
/// Open the Session store rooted at `<worker-aggregate>/session`.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
Self::new_with_migration_writer(root.into(), atomic_write_bytes)
}
// Keep fault injection at the storage publication boundary, without global
// flags or a public alternate restore API. Production always uses atomic writes.
fn new_with_migration_writer(
root: PathBuf,
write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<Self, StoreError> {
fs::create_dir_all(root.join(SEGMENTS_DIR))?;
let session_id = match fs::read(root.join(SESSION_FILE)) {
Ok(bytes) => {
@@ -215,11 +240,14 @@ impl WorkerSessionStore {
}
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_v4(
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths(&root)?,
manifest.session_id,
manifest.schema_version,
write_segment,
)?;
manifest.schema_version = SESSION_SCHEMA_VERSION;
atomic_write_json(&root.join(SESSION_FILE), &manifest)?;
@@ -246,6 +274,33 @@ impl WorkerSessionStore {
})
}
/// Validate an existing Session's conversion without publishing any bytes.
/// Hosts can isolate unreadable archives before admitting a staged migration;
/// the versioned decoder and conversion checks remain owned by this store.
pub fn validate_migration(root: &Path) -> Result<(), StoreError> {
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(root.join(SESSION_FILE))?)?;
match manifest.schema_version {
SESSION_SCHEMA_VERSION => validate_canonical_segment_logs(root),
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => migrate_segment_logs_to_current(
segment_log_paths(root)?,
manifest.session_id,
manifest.schema_version,
|_, _| Ok(()),
),
version => Err(StoreError::Corrupt {
line: 0,
message: format!(
"unsupported Worker Session schema version {version}, expected {SESSION_SCHEMA_VERSION}"
),
}),
}
}
/// Open an already-retained Session without creating directories or migrating
/// persisted data. Observation paths must never mutate retained state.
pub fn open_read_only(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
@@ -742,6 +797,7 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_log_entry(&self.log_path(segment_id), entry)
@@ -800,6 +856,9 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let mut content = Vec::new();
@@ -1158,10 +1217,11 @@ fn segment_log_paths_in_directory(
Ok(paths)
}
fn migrate_segment_logs_to_v4(
fn migrate_segment_logs_to_current(
paths: Vec<(SegmentId, PathBuf)>,
session_id: SessionId,
source_schema_version: u32,
mut write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<(), StoreError> {
struct MigrationPlan {
path: PathBuf,
@@ -1192,12 +1252,19 @@ fn migrate_segment_logs_to_v4(
}
// Resolve trusted creation boundaries first, then update inherited copies by
// logical identity too (fork/rewind seeds retain the same entry id).
let summaries = crate::legacy_session_log::compaction_summary_ids(
plans.iter().flat_map(|plan| plan.entries.iter()),
);
let summaries = if source_schema_version < COMPACTION_SESSION_SCHEMA_VERSION {
crate::legacy_session_log::compaction_summary_ids(
plans.iter().flat_map(|plan| plan.entries.iter()),
)
} else {
// v4 already made the explicit origin migration. Preserve its origins.
Default::default()
};
for plan in &mut plans {
crate::legacy_session_log::migrate_compaction_origins(&mut plan.entries, &summaries);
for entry in std::mem::take(&mut plan.entries) {
for mut entry in std::mem::take(&mut plan.entries) {
crate::legacy_session_log::migrate_system_inputs(&mut entry)?;
entry.validate_input_metadata()?;
serde_json::to_writer(&mut plan.output, &entry)?;
plan.output.push(b'\n');
}
@@ -1219,7 +1286,7 @@ fn migrate_segment_logs_to_v4(
}
for plan in plans {
atomic_write_bytes(&plan.path, &plan.output)?;
write_segment(&plan.path, &plan.output)?;
}
Ok(())
}
@@ -1236,9 +1303,20 @@ pub(crate) fn migrate_fs_session(root: &Path, session_id: SessionId) -> Result<(
Err(error) => return Err(error.into()),
};
match version {
SESSION_SCHEMA_VERSION => Ok(()),
ANNOTATED_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_v4(segment_log_paths_in_directory(root)?, session_id, version)?;
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_paths(segment_log_paths_in_directory(root)?)
}
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths_in_directory(root)?,
session_id,
version,
atomic_write_bytes,
)?;
atomic_write_json(&marker, &SESSION_SCHEMA_VERSION)
}
_ => Err(StoreError::Corrupt {
@@ -1249,8 +1327,15 @@ pub(crate) fn migrate_fs_session(root: &Path, session_id: SessionId) -> Result<(
}
fn validate_canonical_segment_logs(root: &Path) -> Result<(), StoreError> {
for (_, path) in segment_log_paths(root)? {
let _: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
validate_canonical_segment_paths(segment_log_paths(root)?)
}
fn validate_canonical_segment_paths(paths: Vec<(SegmentId, PathBuf)>) -> Result<(), StoreError> {
for (_, path) in paths {
let entries: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
for entry in entries {
entry.validate_input_metadata()?;
}
}
Ok(())
}
@@ -1430,6 +1515,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -2017,6 +2103,146 @@ mod tests {
);
}
#[test]
fn v5_migration_retries_after_second_file_publication_fails() {
fn identity_and_body(entries: &[LogEntry]) -> Vec<serde_json::Value> {
entries
.iter()
.flat_map(|record| match record {
LogEntry::AnnotatedSystemItem { entry, .. } => vec![entry.to_history_entry()],
LogEntry::AnnotatedUserInput { history, .. } => history.clone(),
_ => panic!("unexpected record in fixed migration fixture"),
})
.map(|entry| {
serde_json::json!({
"entry_id": entry.metadata.entry_id,
"origin": entry.metadata.origin,
"derivation": entry.metadata.derivation,
"item": entry.item,
})
})
.collect()
}
const SOURCE: &str =
include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl");
let root = tempfile::tempdir().unwrap();
let session_id = new_session_id();
let first_id = SegmentId::from_u128(1);
let second_id = SegmentId::from_u128(2);
fs::create_dir_all(root.path().join(SEGMENTS_DIR)).unwrap();
let marker_path = root.path().join(SESSION_FILE);
atomic_write_json(
&marker_path,
&SessionManifest {
schema_version: COMPACTION_SESSION_SCHEMA_VERSION,
session_id,
},
)
.unwrap();
let old_marker = fs::read(&marker_path).unwrap();
let first_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{first_id}.jsonl"));
let second_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{second_id}.jsonl"));
let second_source = SOURCE.replace("-id", "-second-id");
fs::write(&first_path, SOURCE).unwrap();
fs::write(&second_path, &second_source).unwrap();
let old_first = fs::read(&first_path).unwrap();
let old_second = fs::read(&second_path).unwrap();
let first_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, first_id, &old_first).unwrap());
let second_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, second_id, &old_second).unwrap());
let mut writes = 0;
let mut first_published = None;
let failure = WorkerSessionStore::new_with_migration_writer(
root.path().to_path_buf(),
|path, bytes| {
writes += 1;
match writes {
1 => {
assert_eq!(path, first_path.as_path());
// Both files are still old when the first publication starts.
assert_eq!(fs::read(&first_path).unwrap(), old_first);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
atomic_write_bytes(path, bytes)?;
first_published = Some(bytes.to_vec());
Ok(())
}
2 => {
assert_eq!(path, second_path.as_path());
assert_eq!(
fs::read(&first_path).unwrap(),
*first_published.as_ref().unwrap()
);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
// Fail only this atomic-write boundary, after complete
// preflight and a real successful first-file publication.
Err(std::io::Error::other("injected second migration write failure").into())
}
_ => panic!("publication continued after its write failure"),
}
},
);
let error = match failure {
Ok(_) => panic!("the injected publication failure must reject store open"),
Err(error) => error,
};
assert!(
error
.to_string()
.contains("injected second migration write failure")
);
assert_eq!(writes, 2);
let first_published = first_published.unwrap();
assert_ne!(first_published, old_first);
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
assert_eq!(fs::read(&marker_path).unwrap(), old_marker);
let updated_first: Vec<LogEntry> = parse_jsonl(&first_published).unwrap();
for entry in &updated_first {
entry.validate_input_metadata().unwrap();
}
assert!(matches!(
&updated_first[0],
LogEntry::AnnotatedSystemItem {
entry: crate::LoggedSystemHistoryEntry {
item: crate::SystemItem::Input { .. },
..
},
..
}
));
assert_eq!(identity_and_body(&updated_first), first_identity);
// Retry via the public production constructor, without any injection or
// hand-built mixed fixture. The old marker drives the same migration.
let store = WorkerSessionStore::new(root.path()).unwrap();
let first = store.read_all(session_id, first_id).unwrap();
let second = store.read_all(session_id, second_id).unwrap();
assert_eq!(identity_and_body(&first), first_identity);
assert_eq!(identity_and_body(&second), second_identity);
for entry in first.iter().chain(second.iter()) {
entry.validate_input_metadata().unwrap();
}
assert_eq!(fs::read(&first_path).unwrap(), first_published);
let second_published = fs::read(&second_path).unwrap();
assert_ne!(second_published, old_second);
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(&marker_path).unwrap()).unwrap();
assert_eq!(manifest.schema_version, SESSION_SCHEMA_VERSION);
drop(store);
WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), second_published);
}
#[test]
fn current_jsonl_requires_annotations_across_append_rewrite_and_reopen() {
let root = tempfile::tempdir().unwrap();
@@ -0,0 +1,711 @@
//! Public consumer boundary: real capture adapters, checked DTO codec and pure readers.
//! No Worker, persistence, live Session provider, or substituted capture implementation.
use agen::{HistoryEntry, Item};
use protocol::{
SessionEntryProvenance, SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData,
};
use session_store::capture::*;
use session_store::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputObservationUse, LoggedHistoryDerivation, LoggedSessionHistoryEntryId,
LoggedSessionHistoryMetadata, LoggedSessionHistoryOrigin, PublicSystemItem,
PublicSystemPresentation,
};
fn history(
id: &str,
item: Item,
input_metadata: Option<HistoryInputMetadata>,
) -> HistoryEntry<LoggedSessionHistoryMetadata> {
HistoryEntry::new(
item,
LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::LocalHumanInput,
derivation: Some(LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("earlier-entry".into())],
}),
input_metadata,
},
)
}
fn usage(recording: bool, memory: bool, observation: bool) -> HistoryInputUsePolicy {
HistoryInputUsePolicy {
recording_material: if recording {
InputEvidenceUse::Include
} else {
InputEvidenceUse::Exclude
},
memory_evidence: if memory {
InputEvidenceUse::Include
} else {
InputEvidenceUse::Exclude
},
observation: if observation {
InputObservationUse::Readable
} else {
InputObservationUse::Hidden
},
compaction: InputCompactionUse::Retain,
}
}
fn reference(capture: &SessionCapture, id: &str) -> ScopedSessionEntryRef {
capture
.scoped_ref(&SessionEntryRef::parse(id).unwrap())
.unwrap()
}
#[test]
fn public_consumer_roundtrip_retains_body_original_source_order_origin_and_independent_policy() {
let entries = vec![
history(
"recording",
Item::user_message("record-only"),
Some(HistoryInputMetadata::new(usage(true, false, false), None, None).unwrap()),
),
history(
"memory",
Item::user_message("memory-only"),
Some(HistoryInputMetadata::new(usage(false, true, false), None, None).unwrap()),
),
history(
"observation",
Item::user_message("observation-only"),
Some(HistoryInputMetadata::new(usage(false, false, true), None, None).unwrap()),
),
];
let capture = SessionCapture::from_complete_history("synthetic-session", "selected", entries);
let mut dto = capture.to_dto();
// The checked boundary also accepts entries retained from different original segments.
dto.entries[0].source.segment_id = "original-a".into();
dto.entries[0].source.entry_index = 80;
dto.entries[1].source.segment_id = "original-b".into();
dto.entries[1].source.entry_index = 3;
let capture = SessionCapture::try_from_dto(dto.clone(), CaptureLimits::default()).unwrap();
let encoded = capture.encode(CaptureLimits::default()).unwrap();
let restored = SessionCapture::decode(&encoded, CaptureLimits::default()).unwrap();
assert_eq!(restored.to_dto(), dto);
assert_eq!(restored.capture_id(), capture.capture_id());
for (id, allowed) in [
("Erecording", CapturePurpose::Recording),
("Ememory", CapturePurpose::MemoryEvidence),
("Eobservation", CapturePurpose::Observation),
] {
let reference = reference(&capture, id);
assert_eq!(
restored.resolve_ref(&reference, allowed).unwrap(),
capture.resolve_ref(&reference, allowed).unwrap()
);
for purpose in [
CapturePurpose::Recording,
CapturePurpose::MemoryEvidence,
CapturePurpose::Observation,
] {
assert_eq!(
restored.resolve_ref(&reference, purpose).is_ok(),
purpose == allowed
);
}
}
assert_eq!(restored.overview().len(), 1);
assert_eq!(restored.search(&SearchOptions::default()).len(), 1);
assert!(restored.evidence_for("Erecording").is_none());
let evidence = restored.evidence_for("Ememory").unwrap();
assert_eq!(evidence.segment_id, "original-b");
assert_eq!(evidence.entry_range, [3, 3]);
assert!(evidence.excerpt.contains("memory-only"));
let recorded = restored
.slice(CapturePurpose::Recording, None, 10, 16_000)
.unwrap();
assert_eq!(recorded.slice().items.len(), 1);
assert_eq!(recorded.slice().items[0].text, "record-only");
for (purpose, expected) in [
(CapturePurpose::MemoryEvidence, "memory-only"),
(CapturePurpose::Observation, "observation-only"),
] {
let slice = restored.slice(purpose, None, 10, 16_000).unwrap();
assert_eq!(slice.slice().items.len(), 1);
assert_eq!(slice.slice().items[0].text, expected);
assert!(
restored
.read_raw(
&reference(&restored, "Erecording"),
purpose,
RawField::MessagePart(0),
0,
100
)
.is_err()
);
}
}
#[test]
fn common_data_drops_reasoning_raw_system_and_private_payload_even_when_unindexed() {
let payload = FeatureInputPayload::encode(
"capture-test",
1,
&serde_json::json!({"secret": "PRIVATE-PAYLOAD"}),
)
.unwrap();
let public = PublicSystemItem {
kind: "notice".into(),
body: "public notice".into(),
presentation: PublicSystemPresentation::Ignored,
};
let system =
HistoryInputMetadata::new(usage(true, true, true), Some(payload.clone()), Some(public))
.unwrap();
let private = HistoryInputMetadata::new(usage(true, true, true), Some(payload), None).unwrap();
let capture = SessionCapture::from_complete_history(
"s",
"segment",
vec![
history("raw", Item::system_message("RAW-SYSTEM-PROMPT"), None),
history("reasoning", Item::reasoning("HIDDEN-REASONING"), None),
history(
"private",
Item::system_message("PRIVATE-SYSTEM-BODY"),
Some(private),
),
history(
"public",
Item::system_message("MODEL-SYSTEM-BODY"),
Some(system),
),
history("ordinary", Item::user_message("ordinary"), None),
],
);
let encoded = String::from_utf8(capture.encode(CaptureLimits::default()).unwrap()).unwrap();
let retained_debug = format!("{capture:?}");
for secret in [
"RAW-SYSTEM-PROMPT",
"HIDDEN-REASONING",
"PRIVATE-SYSTEM-BODY",
"MODEL-SYSTEM-BODY",
"PRIVATE-PAYLOAD",
] {
assert!(!encoded.contains(secret));
assert!(!retained_debug.contains(secret));
}
assert_eq!(capture.to_dto().entries.len(), 2);
assert!(encoded.contains("public notice"));
// Filtered entries do not renumber the public source positions.
assert_eq!(capture.to_dto().entries[0].order, 3);
assert_eq!(
capture.view_order_for_ref(&SessionEntryRef::parse("Eordinary").unwrap()),
Some(4)
);
}
#[test]
fn protocol_projection_does_not_invent_recording_permission() {
let capture = SessionCapture::from_session_snapshot(
"s",
"seg",
SessionSnapshot {
pending_submissions: Default::default(),
entries: vec![SessionSnapshotEntry {
entry_id: "projected".into(),
timestamp: 0,
provenance: SessionEntryProvenance::HumanInput,
derived_from: vec!["source".into()],
use_policy: Some(protocol::SessionEntryUsePolicy {
observation_readable: true,
memory_evidence: false,
}),
data: SessionSnapshotEntryData::Message {
role: protocol::SessionMessageRole::User,
content: vec![protocol::SessionContentPart::Text {
text: "projected".into(),
}],
},
}],
},
);
let reference = reference(&capture, "Eprojected");
assert!(matches!(
capture.resolve_ref(&reference, CapturePurpose::Recording),
Err(CaptureError::PolicyDenied)
));
assert!(
capture
.resolve_ref(&reference, CapturePurpose::Observation)
.is_ok()
);
assert!(capture.evidence_for("Eprojected").is_none());
}
#[test]
fn malformed_dtos_and_oversized_representations_are_rejected() {
let capture = SessionCapture::new("s", "seg", vec![Item::user_message("body")]);
let original = serde_json::to_value(capture.to_dto()).unwrap();
let mut invalids = Vec::new();
let mut value = original.clone();
value["version"] = 999.into();
invalids.push(value);
let mut value = original.clone();
value["unexpected"] = true.into();
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["policy"]["recording_material"] = "include".into();
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["policy"]
.as_object_mut()
.unwrap()
.remove("memory_evidence");
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["body"] = serde_json::json!({"kind":"reasoning","text":"secret"});
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["body"]["feature_payload"] = serde_json::json!({"private":true});
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["body"]["role"] = "system".into();
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["source"]["unknown"] = true.into();
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["source"]["history_origin"] =
serde_json::json!({"kind":"legacy_unknown","unknown":true});
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["entry_id"] = "bad/id".into();
invalids.push(value);
let mut value = original.clone();
value["entries"]
.as_array_mut()
.unwrap()
.push(original["entries"][0].clone());
invalids.push(value);
for invalid in invalids {
let bytes = serde_json::to_vec(&invalid).unwrap();
assert!(
SessionCapture::decode(&bytes, CaptureLimits::default()).is_err(),
"accepted {invalid}"
);
}
let bytes = capture.encode(CaptureLimits::default()).unwrap();
assert!(
SessionCapture::decode(
&bytes,
CaptureLimits {
max_bytes: bytes.len() - 1,
..CaptureLimits::default()
}
)
.is_err()
);
assert!(
capture
.encode(CaptureLimits {
max_entries: 0,
..CaptureLimits::default()
})
.is_err()
);
assert!(
capture
.encode(CaptureLimits {
max_text_bytes: 3,
..CaptureLimits::default()
})
.is_err()
);
let mut reversed = capture.to_dto();
reversed.view_entry_count = 2;
let mut second = reversed.entries[0].clone();
second.entry_id = "second".into();
second.source.entry_index = 4;
second.order = 0;
reversed.entries[0].order = 1;
reversed.entries.push(second);
assert!(SessionCapture::try_from_dto(reversed, CaptureLimits::default()).is_err());
let mut duplicate_source = capture.to_dto();
duplicate_source.view_entry_count = 2;
let mut second = duplicate_source.entries[0].clone();
second.entry_id = "distinct-id".into();
second.order = 1;
duplicate_source.entries.push(second);
assert!(matches!(
SessionCapture::try_from_dto(duplicate_source, CaptureLimits::default()),
Err(CaptureError::Invalid("duplicate source position"))
));
}
#[test]
fn raw_utf8_tool_input_and_result_are_lossless_without_preview_markers() {
let arguments = "\"quoted\"\\path\n日本語🙂".repeat(300);
let result = "result\n\t\\\"λ🙂".repeat(400);
let capture = SessionCapture::new(
"s",
"seg",
vec![
Item::tool_call("call", "tool", arguments.clone()),
Item::tool_result_item("call", "summary", Some(result.clone()), false),
],
);
for (id, field, expected) in [
("E00000000", RawField::ToolArguments, arguments),
("E00000001", RawField::ToolContent, result),
] {
let reference = reference(&capture, id);
let mut offset = 0;
let mut actual = String::new();
loop {
let part = capture
.read_raw(&reference, CapturePurpose::Observation, field, offset, 17)
.unwrap();
assert!(part.text.len() <= 17);
assert_eq!(part.offset, offset);
actual.push_str(&part.text);
match part.next_offset {
Some(next) => {
assert!(next > offset);
offset = next;
}
None => break,
}
}
assert_eq!(actual, expected);
}
let unicode = SessionCapture::new("s", "seg", vec![Item::tool_call("call", "tool", "🙂")]);
let reference = reference(&unicode, "E00000000");
assert!(matches!(
unicode.read_raw(
&reference,
CapturePurpose::Observation,
RawField::ToolArguments,
1,
8
),
Err(CaptureError::InvalidRange)
));
assert!(matches!(
unicode.read_raw(
&reference,
CapturePurpose::Observation,
RawField::ToolArguments,
0,
3
),
Err(CaptureError::BudgetTooSmall)
));
assert!(matches!(
unicode.read_raw(
&reference,
CapturePurpose::Observation,
RawField::ToolArguments,
5,
8
),
Err(CaptureError::InvalidRange)
));
}
#[test]
fn escaped_json_slice_budget_includes_metadata_and_bound_continuation() {
let body = "\"\\\n\t🙂日本語".repeat(500);
let capture = SessionCapture::new(
"synthetic-session",
"segment",
vec![Item::tool_call("call", "tool", body.clone())],
);
let other_session = SessionCapture::new(
"other-session",
"segment",
vec![Item::tool_call("call", "tool", body.clone())],
);
let changed = SessionCapture::new(
"synthetic-session",
"segment",
vec![Item::tool_call("call", "tool", "changed")],
);
let mut cursor = None;
let mut actual = String::new();
let budget = 1400;
loop {
let part = capture
.slice(CapturePurpose::Recording, cursor.as_ref(), 1, budget)
.unwrap();
assert!(part.encoded_len() <= budget);
assert_eq!(part.encoded_len(), part.as_json().len());
assert_eq!(serde_json::to_vec(part.slice()).unwrap(), part.as_json());
assert_eq!(part.slice().items.len(), 1);
let item = &part.slice().items[0];
assert_eq!(item.offset, actual.len());
actual.push_str(&item.text);
cursor = part.slice().next_cursor.clone();
if let Some(cursor) = &cursor {
assert!(matches!(
other_session.slice(CapturePurpose::Recording, Some(cursor), 1, budget),
Err(CaptureError::ForeignReference)
));
assert!(matches!(
changed.slice(CapturePurpose::Recording, Some(cursor), 1, budget),
Err(CaptureError::ForeignReference)
));
assert!(matches!(
capture.slice(CapturePurpose::Observation, Some(cursor), 1, budget),
Err(CaptureError::ForeignReference)
));
} else {
break;
}
}
assert_eq!(actual, body);
assert!(matches!(
capture.slice(CapturePurpose::Recording, None, 1, 8),
Err(CaptureError::BudgetTooSmall)
));
assert!(matches!(
capture.slice(CapturePurpose::Recording, None, 0, budget),
Err(CaptureError::BudgetTooSmall)
));
}
#[test]
fn snapshot_immutability_and_scoped_operations_reject_foreign_or_unknown_refs_and_bad_ranges() {
let mut source = vec![
Item::user_message("before"),
Item::assistant_message("answer"),
];
let capture = SessionCapture::new("s", "seg", source.clone());
let fixed_bytes = capture.encode(CaptureLimits::default()).unwrap();
let first = reference(&capture, "E00000000");
let second = reference(&capture, "E00000001");
source.push(Item::user_message("appended"));
source[0] = Item::user_message("compacted");
source.clear();
assert_eq!(
capture.encode(CaptureLimits::default()).unwrap(),
fixed_bytes
);
assert_eq!(
capture.clone().encode(CaptureLimits::default()).unwrap(),
fixed_bytes
);
let foreign = SessionCapture::new(
"different",
"seg",
vec![
Item::user_message("before"),
Item::assistant_message("answer"),
],
);
assert!(matches!(
foreign.resolve_ref(&first, CapturePurpose::Observation),
Err(CaptureError::ForeignReference)
));
assert!(matches!(
capture.checked_range(&second, &first, CapturePurpose::Observation),
Err(CaptureError::InvalidRange)
));
assert_eq!(
capture
.checked_range(&first, &second, CapturePurpose::Observation)
.unwrap()
.len(),
2
);
let unknown = SessionEntryRef::parse("E99999999").unwrap();
assert_eq!(capture.view_order_for_ref(&unknown), None);
assert!(matches!(
capture.scoped_ref(&unknown),
Err(CaptureError::UnknownReference)
));
assert!(
capture
.search_checked(&SearchOptions {
from: Some(unknown.clone()),
..Default::default()
})
.is_err()
);
assert!(
capture
.search(&SearchOptions {
through: Some(unknown),
..Default::default()
})
.is_empty()
);
assert!(
capture
.read_checked(ReadSelector::EntryRange([1, 0]), ReadOptions::default())
.is_err()
);
assert!(
capture
.read_checked(ReadSelector::EntryRange([0, 99]), ReadOptions::default())
.is_err()
);
}
#[test]
fn slices_preserve_empty_and_long_result_fields_under_item_and_metadata_budgets() {
let summary = "summary\n\"🙂".repeat(100);
let content = "content\t\\日本語".repeat(200);
let capture = SessionCapture::new(
"s",
"seg",
vec![
Item::tool_result_item("first", summary.clone(), Some(content.clone()), true),
Item::tool_result_item("empty", "", Some(String::new()), false),
],
);
let mut cursor = None;
let mut fields: Vec<(String, RawField, String)> = Vec::new();
loop {
let part = capture
.slice(CapturePurpose::Recording, cursor.as_ref(), 2, 1500)
.unwrap();
assert!(part.slice().items.len() <= 2);
assert!(serde_json::to_vec(part.slice()).unwrap().len() <= 1500);
for item in &part.slice().items {
let id = item.reference.entry_ref.to_string();
if let Some((last_id, last_field, text)) = fields.last_mut() {
if *last_id == id && *last_field == item.field {
assert_eq!(text.len(), item.offset);
text.push_str(&item.text);
continue;
}
}
assert_eq!(item.offset, 0);
fields.push((id, item.field, item.text.clone()));
}
cursor = part.slice().next_cursor.clone();
if cursor.is_none() {
break;
}
}
assert_eq!(
fields,
vec![
("E00000000".into(), RawField::ToolSummary, summary),
("E00000000".into(), RawField::ToolContent, content),
("E00000001".into(), RawField::ToolSummary, String::new()),
("E00000001".into(), RawField::ToolContent, String::new()),
]
);
let metadata_heavy = SessionCapture::new(
"s",
"seg",
vec![Item::tool_call("call", "name".repeat(200), "x")],
);
assert!(matches!(
metadata_heavy.slice(CapturePurpose::Recording, None, 1, 700),
Err(CaptureError::BudgetTooSmall)
));
}
#[test]
fn evidence_uses_original_segment_positions_after_roundtrip_and_view_selection() {
let full = SessionCapture::new(
"s",
"selected",
vec![
Item::user_message("first"),
Item::assistant_message("second"),
],
);
let mut dto = full.to_dto();
dto.entries[0].source.segment_id = "original-a".into();
dto.entries[0].source.entry_index = 80;
dto.entries[1].source.segment_id = "original-b".into();
dto.entries[1].source.entry_index = 3;
let encoded = dto.encode(CaptureLimits::default()).unwrap();
let restored = SessionCapture::decode(&encoded, CaptureLimits::default()).unwrap();
for (id, segment, index, order) in [
("E00000000", "original-a", 80, 0),
("E00000001", "original-b", 3, 1),
] {
let evidence = restored.evidence_for(id).unwrap();
assert_eq!(evidence.segment_id, segment);
assert_eq!(evidence.entry_range, [index; 2]);
let read = restored.read(ReadSelector::Id(id), ReadOptions::default());
assert_eq!(read.entries[0].entry_range, [order; 2]);
}
let selected = restored.select_order_range(1..2).unwrap();
assert_eq!(selected.overview()[0].entry_range, [0, 0]);
let evidence = selected.evidence_for("E00000001").unwrap();
assert_eq!(evidence.segment_id, "original-b");
assert_eq!(evidence.entry_range, [3, 3]);
let roundtrip = SessionCapture::decode(
&selected.encode(CaptureLimits::default()).unwrap(),
CaptureLimits::default(),
)
.unwrap();
assert_eq!(roundtrip.to_dto(), selected.to_dto());
assert_eq!(
roundtrip.evidence_for("E00000001").unwrap().entry_range,
[3, 3]
);
}
#[test]
fn fixed_history_selection_preserves_original_coordinates_and_checks_private_position_bounds() {
let history = vec![
history("old", Item::user_message("old"), None),
history("hidden", Item::reasoning("PRIVATE"), None),
history("selected", Item::user_message("selected"), None),
history("private-tail", Item::system_message("PRIVATE"), None),
];
let full = SessionCapture::from_complete_history("s", "seg", history);
let suffix = full.select_order_range(1..4).unwrap();
let original = full
.entries()
.find(|entry| entry.entry_id == "selected")
.unwrap();
let selected = suffix.entries().next().unwrap();
assert_eq!(selected.entry_id, original.entry_id);
assert_eq!(selected.source, original.source);
assert_eq!(selected.body, original.body);
assert_eq!(selected.policy, original.policy);
assert!(suffix.evidence_for("Eold").is_none());
assert_eq!(suffix.search(&SearchOptions::default()).len(), 1);
assert_eq!(
suffix.view_order_for_ref(&SessionEntryRef::parse("Eselected").unwrap()),
Some(1)
);
assert_eq!(selected.source.entry_index, 2);
assert_eq!(selected.order, 1);
assert_eq!(suffix.to_dto().view_entry_count, 3);
assert_eq!(
suffix.evidence_for("Eselected").unwrap().entry_range,
[2, 2]
);
let nested = suffix.select_order_range(1..2).unwrap();
assert_eq!(nested.entries().next().unwrap().order, 0);
assert_eq!(nested.entries().next().unwrap().source, original.source);
let restored = SessionCapture::decode(
&suffix.encode(CaptureLimits::default()).unwrap(),
CaptureLimits::default(),
)
.unwrap();
assert_eq!(restored.to_dto(), suffix.to_dto());
assert!(full.select_order_range(4..5).is_err());
let reversed = std::ops::Range { start: 3, end: 2 };
assert!(full.select_order_range(reversed).is_err());
let hidden_only = full.select_order_range(3..4).unwrap();
assert_eq!(hidden_only.entries().len(), 0);
assert_eq!(hidden_only.to_dto().view_entry_count, 1);
let mut invalid = full.to_dto();
invalid.view_entry_count = 2; // the selected entry's view order is outside this extent
assert!(SessionCapture::try_from_dto(invalid, CaptureLimits::default()).is_err());
assert_eq!(
full.retain_entries(|entry| entry.entry_id == "selected")
.entries()
.next()
.unwrap()
.source,
original.source
);
assert_eq!(full.to_dto().view_entry_count, 4);
}
+175
View File
@@ -11,6 +11,7 @@ fn annotated(item: Item) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -421,3 +422,177 @@ fn lookup_session_of_finds_owning_session() {
assert_eq!(store.lookup_session_of(segid).unwrap(), Some(sid));
}
// Storage/projection component test: both production layouts, temporary originals,
// no Feature, model, runtime or Host authority is substituted into this boundary.
#[test]
fn generic_system_append_and_fork_preserve_metadata_without_public_payload_leak() {
use session_store::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputObservationUse, LoggedHistoryDerivation, LoggedSystemHistoryEntry,
PublicSystemItem, PublicSystemPresentation, SystemInputEnvelope, WorkerSessionStore,
};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let fork = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
store
.create_segment(sid, segment, &[nil_session_start(1, sid)])
.unwrap();
let entry = LoggedSystemHistoryEntry::from_input(
LoggedSessionHistoryOrigin::Provided {
producer: "new-feature".into(),
},
Some(LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source-entry".into())],
}),
SystemInputEnvelope::new(
"private model body",
HistoryInputMetadata::new(
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
},
Some(
FeatureInputPayload::encode(
"new-feature:context",
101,
&serde_json::json!({"private": "secret-detail"}),
)
.unwrap(),
),
Some(PublicSystemItem {
kind: "context".into(),
body: "explicit public body".into(),
presentation: PublicSystemPresentation::Context,
}),
)
.unwrap(),
)
.unwrap(),
);
store
.append(
sid,
segment,
&LogEntry::AnnotatedSystemItem {
ts: 2,
entry: entry.clone(),
extensions: vec![],
},
)
.unwrap();
let saved = store.read_all(sid, segment).unwrap();
let LogEntry::AnnotatedSystemItem {
entry: restored, ..
} = &saved[1]
else {
panic!("missing system input")
};
assert_eq!(restored.to_history_entry(), entry.to_history_entry());
assert_eq!(
collect_state(&saved).history[0].as_text(),
Some("private model body")
);
let mut fork_seed = nil_session_start(3, sid);
if let LogEntry::AnnotatedSegmentStart {
history,
forked_from,
..
} = &mut fork_seed
{
history.push(restored.to_history_entry());
*forked_from = Some(session_store::SegmentOrigin {
segment_id: segment,
at_turn_index: 0,
});
}
store.create_segment(sid, fork, &[fork_seed]).unwrap();
let fork_saved = store.read_all(sid, fork).unwrap();
let LogEntry::AnnotatedSegmentStart { history, .. } = &fork_saved[0] else {
unreachable!()
};
assert_eq!(history, &[entry.to_history_entry()]);
let original = session_store::public_snapshot::project_session_snapshot(sid, &saved);
let inherited = session_store::public_snapshot::project_session_snapshot(sid, &fork_saved);
let public = serde_json::to_string(&original.entries).unwrap();
assert!(public.contains("explicit public body"));
for private in ["private model body", "secret-detail", "new-feature:context"] {
assert!(!public.contains(private), "private field leaked: {private}");
}
assert_eq!(original.entries.len(), 1);
assert_eq!(inherited.entries.len(), 1);
assert_eq!(original.entries[0].entry_id, inherited.entries[0].entry_id);
assert_eq!(
original.entries[0].derived_from,
inherited.entries[0].derived_from
);
assert_eq!(
original.entries[0].provenance,
inherited.entries[0].provenance
);
assert_eq!(
serde_json::to_value(&original.entries[0].data).unwrap(),
serde_json::to_value(&inherited.entries[0].data).unwrap()
);
}
}
#[test]
fn invalid_generic_system_input_rejects_append_create_and_external_json_without_changing_originals()
{
use session_store::{StoreError, SystemItem, WorkerSessionStore};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
let seed = nil_session_start(1, sid);
store
.create_segment(sid, segment, std::slice::from_ref(&seed))
.unwrap();
let path = if worker_layout {
dir.path().join("segments").join(format!("{segment}.jsonl"))
} else {
dir.path()
.join(sid.to_string())
.join(format!("{segment}.jsonl"))
};
let before = std::fs::read(&path).unwrap();
let invalid = LogEntry::AnnotatedSystemItem {
ts: 2,
extensions: vec![],
entry: session_store::LoggedSystemHistoryEntry {
item: SystemItem::Input {
body: "no policy".into(),
},
metadata: LoggedSessionHistoryMetadata::legacy_unknown(),
},
};
assert!(matches!(
store.append(sid, segment, &invalid),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(matches!(
store.create_segment(sid, segment, &[seed, invalid.clone()]),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(
serde_json::from_value::<LogEntry>(serde_json::to_value(invalid).unwrap()).is_err()
);
assert_eq!(std::fs::read(path).unwrap(), before);
assert_eq!(store.read_all(sid, segment).unwrap().len(), 1);
}
}
@@ -24,6 +24,7 @@ fn annotated(items: &[Item]) -> Vec<session_store::LoggedHistoryEntry> {
.map(|item| session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
+1 -1
View File
@@ -23,7 +23,7 @@ serde.workspace = true
serde_json.workspace = true
session-store.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["rt", "sync", "time"] }
tokio = { workspace = true, features = ["rt", "sync", "time", "net", "io-util", "io-std", "signal"] }
uuid = { workspace = true, features = ["v7"] }
worker.workspace = true
+2 -1
View File
@@ -4,7 +4,8 @@ A Host that starts existing Workers in the same process. Assembles interactive W
## Reference index
- [`host`] — bootstrap and confirmed shutdown.
- [`host`] — bootstrap and confirmed shutdown, with or without a TUI.
- [`unix`] — optional operator transport; disconnect does not release execution ownership.
- [`launch`] — Profile, cwd, and Subject validation before launch.
- [`store`] — Worker records and leases.
- [`jobs`] — Job execution and independent result and cleanup lifetimes.
+42 -23
View File
@@ -23,10 +23,7 @@ use session_store::{
CombinedStore, FsStore, FsWorkerStore, WorkerActiveSegmentRef, WorkerMetadataStore,
};
use thiserror::Error;
use worker::bootstrap::{
WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout, bash_output_dir_for_worker_id,
};
use worker::controller::WorkerControllerTransport;
use worker::bootstrap::{WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout};
use worker::ipc::protocol_session::{
WorkerProtocolSessionStreams, dispatch_worker_protocol_method, live_log_entry_event,
subscribe_worker_protocol_session,
@@ -216,7 +213,6 @@ impl StandaloneHost {
WorkerFilesystemAuthority::local(launch.cwd.clone(), launch.cwd.clone());
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
bootstrap_manifest,
@@ -224,11 +220,7 @@ impl StandaloneHost {
launch.prompt_catalog,
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerControllerTransport::InProcess,
WorkerBootstrapLayout::Direct { runtime_base },
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
@@ -428,7 +420,6 @@ impl StandaloneHost {
);
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
manifest,
@@ -436,11 +427,7 @@ impl StandaloneHost {
worker::PromptCatalogSource::builtins_only(),
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerControllerTransport::InProcess,
WorkerBootstrapLayout::Direct { runtime_base },
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
@@ -530,10 +517,7 @@ impl StandaloneHost {
/// Working events, committed session entries, alert snapshots, and the
/// initial history snapshot are merged behind the client boundary.
pub fn connect(&self) -> Client<InProcessSocket> {
let streams = subscribe_worker_protocol_session(&self.handle);
let (socket, peer) = InProcessSocket::pair();
tokio::spawn(run_protocol_session(self.handle.clone(), streams, peer));
Client::new(socket)
Client::new(connect_protocol(self.handle.clone()))
}
/// Logical Subject disconnection, not Worker shutdown or completion of its represented work.
@@ -575,6 +559,35 @@ impl StandaloneHost {
}
}
/// Run without a TUI until an admitted protocol Shutdown, controller exit, or process signal.
/// Client EOF only disconnects that client. All exits confirm the same production shutdown
/// and lease release as the in-process UI; an unconfirmed shutdown retains the lease.
pub async fn serve_unix(
self,
endpoint: crate::unix::UnixEndpoint,
) -> Result<(), crate::HeadlessError> {
let mut adapter = crate::unix::UnixAdapter::start(endpoint, self.handle.clone());
let signal = async {
let mut terminate =
tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())?;
tokio::select! {
result = tokio::signal::ctrl_c() => result,
_ = terminate.recv() => Ok(()),
}
};
let result = tokio::select! {
_ = self.handle.protocol_closed() => Ok(()),
result = signal => result,
result = adapter.ended() => result,
};
// Reclaim the public entry and every connection before releasing execution ownership.
let transport = adapter.shutdown().await;
self.shutdown().await?;
result?;
transport?;
Ok(())
}
pub fn with_shutdown_timeout(mut self, shutdown_timeout: Duration) -> Self {
self.shutdown_timeout = shutdown_timeout;
self
@@ -657,9 +670,8 @@ impl Drop for StandaloneHost {
}
}
// The in-process channel is owned by the local operator's Host, not an
// arbitrary socket or Backend account. decode_method discards wire provenance;
// only this boundary can stamp accepted local human input.
// Both in-process views and credential-checked Unix clients are admitted by this operator's
// Host. decode_method discards wire provenance; only this boundary stamps local human input.
fn authorize_local_input(method: protocol::Method) -> protocol::Method {
use protocol::{AuthenticatedInputSource, Method};
match method {
@@ -702,6 +714,13 @@ fn authorize_local_input(method: protocol::Method) -> protocol::Method {
}
}
pub(crate) fn connect_protocol(handle: worker::WorkerHandle) -> InProcessSocket {
let streams = subscribe_worker_protocol_session(&handle);
let (socket, peer) = InProcessSocket::pair();
tokio::spawn(run_protocol_session(handle, streams, peer));
socket
}
async fn run_protocol_session(
handle: worker::WorkerHandle,
streams: WorkerProtocolSessionStreams,
+529 -20
View File
@@ -25,7 +25,46 @@ use thiserror::Error;
#[path = "job_store/legacy_migrations.rs"]
mod legacy_migrations;
const SCHEMA_VERSION: i64 = 4;
const SCHEMA_VERSION: i64 = 5;
/// Availability of the immutable input body, independent of execution outcome.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JobInputDisposition {
Available,
Reclaimed,
Erased,
Invalidated,
}
impl JobInputDisposition {
pub fn as_str(self) -> &'static str {
match self {
Self::Available => "available",
Self::Reclaimed => "reclaimed",
Self::Erased => "erased",
Self::Invalidated => "invalidated",
}
}
}
/// Host-authorized retirement; erasure and legacy invalidation do not change outcomes.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum JobInputRetirementReason {
Completed,
ExplicitErasure,
LegacyInvalidation,
}
impl JobInputRetirementReason {
pub fn disposition(self) -> JobInputDisposition {
match self {
Self::Completed => JobInputDisposition::Reclaimed,
Self::ExplicitErasure => JobInputDisposition::Erased,
Self::LegacyInvalidation => JobInputDisposition::Invalidated,
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
@@ -67,7 +106,11 @@ impl JobAttemptState {
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct JobSnapshot {
/// Receipt metadata. After retirement `input` is null; use the verified loader to execute.
pub request: JobRequest,
pub input_digest: String,
pub intent_fingerprint: String,
pub input_disposition: JobInputDisposition,
/// Durable Host-issued domain delegation, not model input or Profile identity.
pub domain_grant: Option<Value>,
pub state: JobState,
@@ -77,14 +120,27 @@ pub struct JobSnapshot {
}
impl JobSnapshot {
/// Load only the original available body, checking both content and immutable intent.
pub fn load_executable_request(&self) -> Result<&JobRequest, JobStoreError> {
if self.input_disposition != JobInputDisposition::Available {
return Err(JobStoreError::InvalidTransition("Job input is retired"));
}
self.request.validate()?;
if self.request.input_digest()? != self.input_digest
|| self.request.fingerprint()? != self.intent_fingerprint
{
return Err(JobStoreError::Corrupt(
"immutable input or intent fingerprint mismatch".into(),
));
}
Ok(&self.request)
}
/// Shared Feature-facing read contract; no Workspace or execution identity leaks.
pub fn outcome(&self) -> job::JobOutcome {
job::JobOutcome {
job_id: self.request.job_id.clone(),
input_digest: self
.request
.input_digest()
.expect("validated immutable Job input"),
input_digest: self.input_digest.clone(),
attempt_id: self.attempt.attempt_id.clone(),
attempt: self.attempt.number,
state: match self.state {
@@ -204,9 +260,23 @@ impl JobStore {
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
SCHEMA_VERSION => {}
4 | SCHEMA_VERSION => {}
other => return Err(JobStoreError::UnsupportedSchema(other)),
}
if version < 5 {
tx.execute_batch(INPUT_RETENTION_SCHEMA)?;
let has_legacy_archive: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_schema WHERE type='table' AND name='legacy_job_archive')", [], |r| r.get(0))?;
if has_legacy_archive {
tx.execute_batch("DROP TRIGGER legacy_job_archive_immutable;
CREATE TRIGGER legacy_job_archive_immutable BEFORE UPDATE OF job_id,fingerprint,attempts_json ON legacy_job_archive
BEGIN SELECT RAISE(ABORT, 'immutable legacy Job archive'); END;
CREATE TRIGGER legacy_job_archive_input_retirement BEFORE UPDATE OF request_json ON legacy_job_archive
WHEN NOT EXISTS (SELECT 1 FROM job_intents WHERE job_id=OLD.job_id AND input_disposition<>'available')
OR (json_remove(NEW.request_json,'$.input') = json_remove(OLD.request_json,'$.input') AND json_type(NEW.request_json,'$.input')='null') IS NOT TRUE
BEGIN SELECT RAISE(ABORT, 'immutable legacy Job input'); END;")?;
}
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
// An accepted result is the only evidence of success. Reserved attempts
// were never started and remain pending; cancelled attempts never resume.
tx.execute_batch(
@@ -249,10 +319,7 @@ impl JobStore {
[&request.job_id],
|row| row.get(0),
)?;
if stored != fingerprint
|| snapshot.request != request
|| snapshot.domain_grant != grant
{
if stored != fingerprint || snapshot.domain_grant != grant {
return Err(JobStoreError::IntentConflict(request.job_id));
}
tx.commit()?;
@@ -334,6 +401,7 @@ impl JobStore {
.conn
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let previous = require_snapshot(&tx, job_id)?;
previous.load_executable_request()?;
if previous.state != JobState::Failed || previous.attempt.state != JobAttemptState::Failed {
return Err(JobStoreError::InvalidTransition(
"retry requires known failure",
@@ -354,6 +422,81 @@ impl JobStore {
Ok(snapshot)
}
/// Historical snapshots for one domain Session, including terminal and retired Jobs.
pub(crate) fn list_session_snapshots(
&self,
session_id: &str,
after: Option<&str>,
limit: usize,
) -> Result<Vec<JobSnapshot>, JobStoreError> {
let mut stmt = self.conn.prepare("SELECT i.job_id FROM job_intents i JOIN job_domain_grants g ON g.job_id=i.job_id WHERE json_extract(g.grant_json,'$.session_id')=?1 AND (?2 IS NULL OR i.job_id>?2) ORDER BY i.job_id LIMIT ?3")?;
let ids = stmt
.query_map(
params![session_id, after, limit.clamp(1, 100) as i64],
|r| r.get::<_, String>(0),
)?
.collect::<rusqlite::Result<Vec<_>>>()?;
ids.into_iter()
.map(|id| require_snapshot(&self.conn, &id))
.collect()
}
/// The Host must fence domain mutation and delivery before erasure/invalidation.
/// This atomically prevents dispatch/retry/result acceptance, without rewriting outcome.
pub(crate) fn reclaim_input(
&mut self,
job_id: &str,
expected_input_digest: &str,
reason: JobInputRetirementReason,
) -> Result<JobSnapshot, JobStoreError> {
let tx = self
.conn
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, job_id)?;
if snapshot.input_digest != expected_input_digest {
return Err(JobStoreError::DigestMismatch);
}
let desired = reason.disposition();
if snapshot.input_disposition == desired {
tx.commit()?;
reclaim_retired_input_storage(&self.conn)?;
return Ok(snapshot);
}
if reason == JobInputRetirementReason::Completed
&& (snapshot.state != JobState::Completed
|| snapshot.attempt.state != JobAttemptState::Completed
|| snapshot.result.is_none())
{
return Err(JobStoreError::InvalidTransition(
"reclaim requires known completed result",
));
}
if snapshot.input_disposition != JobInputDisposition::Available
&& reason != JobInputRetirementReason::ExplicitErasure
{
return Err(JobStoreError::InvalidTransition(
"retired input cannot be restored or reclassified",
));
}
if reason == JobInputRetirementReason::Completed {
snapshot.load_executable_request()?;
}
let mut metadata = snapshot.request.clone();
metadata.input = Value::Null;
tx.execute(
"UPDATE job_intents SET request_json=?2, input_disposition=?3 WHERE job_id=?1",
params![job_id, serde_json::to_string(&metadata)?, desired.as_str()],
)?;
let has_legacy_archive: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_schema WHERE type='table' AND name='legacy_job_archive')", [], |r| r.get(0))?;
if has_legacy_archive {
tx.execute("UPDATE legacy_job_archive SET request_json=json_set(request_json,'$.input',json('null')) WHERE job_id=?1", [job_id])?;
}
let snapshot = require_snapshot(&tx, job_id)?;
tx.commit()?;
reclaim_retired_input_storage(&self.conn)?;
Ok(snapshot)
}
pub(crate) fn get(&self, job_id: &str) -> Result<JobSnapshot, JobStoreError> {
require_snapshot(&self.conn, job_id)
}
@@ -383,7 +526,7 @@ impl JobStore {
let mut statement = self.conn.prepare(
"SELECT i.job_id FROM job_intents i JOIN job_attempts a
ON a.job_id = i.job_id AND a.number = i.current_attempt
WHERE i.state = 'pending' AND a.state = 'reserved' ORDER BY i.rowid",
WHERE i.state = 'pending' AND i.input_disposition='available' AND a.state = 'reserved' ORDER BY i.rowid",
)?;
let ids = statement
.query_map([], |row| row.get::<_, String>(0))?
@@ -406,6 +549,7 @@ impl JobStore {
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, job_id)?;
check_attempt(&snapshot, attempt_id)?;
snapshot.load_executable_request()?;
if snapshot.state != JobState::Pending
|| snapshot.attempt.state != JobAttemptState::Reserved
{
@@ -447,7 +591,7 @@ impl JobStore {
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, &bound.job_id)?;
check_attempt(&snapshot, &bound.attempt_id)?;
if snapshot.request.input_digest()? != bound.input_digest {
if snapshot.input_digest != bound.input_digest {
return Err(JobStoreError::DigestMismatch);
}
let (digest, encoded) =
@@ -466,6 +610,7 @@ impl JobStore {
tx.commit()?;
return Ok(snapshot);
}
snapshot.load_executable_request()?;
if snapshot.state != JobState::Pending
|| snapshot.attempt.state != JobAttemptState::Dispatched
{
@@ -649,7 +794,7 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
let row = conn
.query_row(
"SELECT i.request_json, i.state, i.acknowledged, a.attempt_id, a.number,
a.state, a.failure, a.result_json FROM job_intents i JOIN job_attempts a
a.state, a.failure, a.result_json, a.input_digest, i.fingerprint, i.input_disposition FROM job_intents i JOIN job_attempts a
ON a.job_id = i.job_id AND a.number = i.current_attempt WHERE i.job_id = ?1",
[job_id],
|row| {
@@ -662,12 +807,27 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
row.get::<_, String>(5)?,
row.get::<_, Option<String>>(6)?,
row.get::<_, Option<String>>(7)?,
row.get::<_, String>(8)?,
row.get::<_, String>(9)?,
row.get::<_, String>(10)?,
))
},
)
.optional()?;
row.map(
|(request, state, acknowledged, attempt_id, number, attempt_state, failure, result)| {
|(
request,
state,
acknowledged,
attempt_id,
number,
attempt_state,
failure,
result,
input_digest,
intent_fingerprint,
disposition,
)| {
let state = match state.as_str() {
"pending" => JobState::Pending,
"completed" => JobState::Completed,
@@ -693,8 +853,17 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
)));
}
};
let request_value: Value = serde_json::from_str(&request)?;
if request_value.get("input").is_none() {
return Err(JobStoreError::Corrupt(
"stored Job input field is missing".into(),
));
}
Ok(JobSnapshot {
request: serde_json::from_str(&request)?,
request: serde_json::from_value(request_value)?,
input_digest,
intent_fingerprint,
input_disposition: serde_json::from_value(Value::String(disposition))?,
domain_grant: conn
.query_row(
"SELECT grant_json FROM job_domain_grants WHERE job_id = ?1",
@@ -719,6 +888,37 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
.transpose()
}
// Retirement is already durable before this maintenance step. Failure must be
// retried with the same Job/digest/reason; it must never restore input. VACUUM
// removes historical free-page copies, including stores written before secure_delete.
fn reclaim_retired_input_storage(conn: &Connection) -> Result<(), JobStoreError> {
conn.execute_batch("PRAGMA secure_delete=ON; VACUUM;")?;
let busy: i64 = conn.query_row("PRAGMA wal_checkpoint(TRUNCATE)", [], |r| r.get(0))?;
if busy != 0 {
return Err(JobStoreError::InvalidTransition(
"input retired; SQLite body cleanup blocked by a reader, retry retirement",
));
}
Ok(())
}
const INPUT_RETENTION_SCHEMA: &str = "
ALTER TABLE job_intents ADD COLUMN input_disposition TEXT NOT NULL DEFAULT 'available'
CHECK (input_disposition IN ('available','reclaimed','erased','invalidated'));
DROP TRIGGER job_intent_immutable;
CREATE TRIGGER job_intent_immutable BEFORE UPDATE OF
job_id, fingerprint, serialization_key, max_concurrent ON job_intents
BEGIN SELECT RAISE(ABORT, 'immutable Job intent'); END;
CREATE TRIGGER job_input_retirement BEFORE UPDATE OF request_json, input_disposition ON job_intents
WHEN (
((OLD.input_disposition = 'available' AND NEW.input_disposition IN ('reclaimed','erased','invalidated'))
OR (OLD.input_disposition IN ('reclaimed','invalidated') AND NEW.input_disposition = 'erased'))
AND json_remove(NEW.request_json,'$.input') = json_remove(OLD.request_json,'$.input')
AND json_type(NEW.request_json,'$.input') = 'null'
) IS NOT TRUE
BEGIN SELECT RAISE(ABORT, 'immutable Job input retirement'); END;
";
const RECONCILIATION_SCHEMA: &str = "
DROP INDEX job_serialization_singleflight;
CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key)
@@ -885,7 +1085,11 @@ mod tests {
Some("unconfirmed"),
)
.unwrap();
store.conn.execute_batch("DROP TRIGGER job_reconciliation_immutable; DROP TRIGGER job_reconciliation_retained; DROP TABLE job_domain_reconciliations; DROP INDEX job_serialization_singleflight; CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key) WHERE serialization_key IS NOT NULL AND state IN ('pending','unknown'); PRAGMA user_version=3;").unwrap();
store.conn.execute_batch("DROP TRIGGER job_input_retirement; DROP TRIGGER job_intent_immutable;
ALTER TABLE job_intents DROP COLUMN input_disposition;
CREATE TRIGGER job_intent_immutable BEFORE UPDATE OF job_id,request_json,fingerprint,serialization_key,max_concurrent ON job_intents
BEGIN SELECT RAISE(ABORT, 'immutable Job intent'); END;
DROP TRIGGER job_reconciliation_immutable; DROP TRIGGER job_reconciliation_retained; DROP TABLE job_domain_reconciliations; DROP INDEX job_serialization_singleflight; CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key) WHERE serialization_key IS NOT NULL AND state IN ('pending','unknown'); PRAGMA user_version=3;").unwrap();
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(reopened.get("first").unwrap(), unknown);
@@ -911,10 +1115,7 @@ mod tests {
JobResultSubmission {
job_id: snapshot.request.job_id.clone(),
attempt_id: snapshot.attempt.attempt_id.clone(),
input_digest: snapshot
.request
.input_digest()
.expect("validated immutable Job input"),
input_digest: snapshot.input_digest.clone(),
result,
}
}
@@ -926,6 +1127,314 @@ mod tests {
.unwrap()
}
#[test]
fn input_reclaim_reopens_as_receipt_and_exact_replay_never_rehydrates() {
let (dir, mut store) = fixture();
let original = request("retained");
let grant = Some(json!({"session_id":"session-a", "consumer":"fixture"}));
let reserved = store
.reserve_granted(original.clone(), grant.clone())
.unwrap();
let running = store
.dispatch("retained", &reserved.attempt.attempt_id, 8)
.unwrap();
let result = submission(&running, json!({"saved":true}));
let completed = store.accept(&result).unwrap();
store.ack("retained").unwrap();
let retired = store
.reclaim_input(
"retained",
&completed.input_digest,
JobInputRetirementReason::Completed,
)
.unwrap();
assert_eq!(retired.input_disposition, JobInputDisposition::Reclaimed);
assert!(retired.request.input.is_null());
assert!(retired.load_executable_request().is_err());
assert_eq!(
retired.outcome().input_digest,
original.input_digest().unwrap()
);
assert_eq!(retired.result, completed.result);
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(
reopened
.reserve_granted(original.clone(), grant.clone())
.unwrap(),
retired
);
assert_eq!(reopened.accept(&result).unwrap(), retired);
assert_eq!(
reopened
.list_session_snapshots("session-a", None, 1)
.unwrap(),
vec![retired.clone()]
);
assert!(
reopened
.list_session_snapshots("session-b", None, 100)
.unwrap()
.is_empty()
);
assert!(
reopened
.list_session_snapshots("session-a", Some("retained"), 100)
.unwrap()
.is_empty()
);
let mut changed = original.clone();
changed.input = json!({"changed":true});
assert!(matches!(
reopened.reserve_granted(changed, grant),
Err(JobStoreError::IntentConflict(_))
));
assert!(reopened.retry("retained").is_err());
assert!(
reopened
.dispatch("retained", &retired.attempt.attempt_id, 8)
.is_err()
);
assert!(reopened.conn.execute("UPDATE job_intents SET request_json=?1,input_disposition='available' WHERE job_id='retained'", [serde_json::to_string(&original).unwrap()]).is_err());
let erased = reopened
.reclaim_input(
"retained",
&retired.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert_eq!(erased.input_disposition, JobInputDisposition::Erased);
assert_eq!(erased.state, JobState::Completed);
assert!(erased.acknowledged);
assert_eq!(
reopened
.reserve_granted(original, erased.domain_grant.clone())
.unwrap(),
erased
);
}
#[test]
fn unfinished_unknown_and_failed_inputs_are_kept_until_explicit_invalidation() {
let (_dir, mut store) = fixture();
for (id, state) in [
("pending", None),
("unknown", Some(JobAttemptState::Unknown)),
("failed", Some(JobAttemptState::Failed)),
] {
let original = request(id);
let snapshot = store.reserve(original.clone()).unwrap();
if let Some(state) = state {
store.dispatch(id, &snapshot.attempt.attempt_id, 8).unwrap();
store
.finish(
id,
&snapshot.attempt.attempt_id,
state,
Some("not complete"),
)
.unwrap();
}
let before = store.get(id).unwrap();
assert!(
store
.reclaim_input(
id,
&before.input_digest,
JobInputRetirementReason::Completed
)
.is_err()
);
assert_eq!(store.get(id).unwrap(), before);
assert_eq!(before.load_executable_request().unwrap(), &original);
assert!(
store
.reclaim_input(id, "wrong", JobInputRetirementReason::ExplicitErasure)
.is_err()
);
let retired = store
.reclaim_input(
id,
&before.input_digest,
JobInputRetirementReason::LegacyInvalidation,
)
.unwrap();
assert_eq!(retired.state, before.state);
assert_eq!(retired.attempt, before.attempt);
assert_eq!(retired.input_disposition, JobInputDisposition::Invalidated);
assert!(store.retry(id).is_err());
assert!(store.dispatch(id, &before.attempt.attempt_id, 8).is_err());
assert_eq!(store.reserve(original).unwrap(), retired);
}
}
#[test]
fn erased_dispatched_input_refuses_delayed_result_without_rewriting_unknown() {
let (dir, mut store) = fixture();
let running = dispatched(&mut store, request("erase-running"));
let result = submission(&running, json!({"late":true}));
let erased = store
.reclaim_input(
"erase-running",
&running.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert!(store.accept(&result).is_err());
assert_eq!(erased.state, JobState::Pending);
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
let recovered = reopened.get("erase-running").unwrap();
assert_eq!(recovered.state, JobState::Unknown);
assert_eq!(recovered.input_disposition, JobInputDisposition::Erased);
assert_eq!(recovered.input_digest, running.input_digest);
assert!(reopened.accept(&result).is_err());
assert!(reopened.retry("erase-running").is_err());
}
#[test]
fn available_input_loader_refuses_tamper_missing_body_and_envelope_changes() {
for mutation in [
"json_set(request_json,'$.input',json('{\"tampered\":true}'))",
"json_remove(request_json,'$.input')",
"json_set(request_json,'$.instruction','changed')",
] {
let (_dir, mut store) = fixture();
store.reserve(request("tamper")).unwrap();
// Simulate external corruption bypassing SQLite's normal write guard.
store
.conn
.execute_batch("DROP TRIGGER job_input_retirement")
.unwrap();
store
.conn
.execute_batch(&format!("UPDATE job_intents SET request_json={mutation}"))
.unwrap();
match store.get("tamper") {
Ok(snapshot) => {
assert!(snapshot.load_executable_request().is_err());
assert!(
store
.dispatch("tamper", &snapshot.attempt.attempt_id, 8)
.is_err()
);
}
Err(JobStoreError::Json(_) | JobStoreError::Corrupt(_)) => {}
other => panic!("unexpected corrupt input read: {other:?}"),
}
}
}
#[test]
fn input_retirement_cleanup_failure_is_durable_and_exact_retry_finishes_without_rehydration() {
let (dir, mut store) = fixture();
let mut original = request("physical-erasure");
let private_body = "private-input-retirement-original-bytes".repeat(1000);
original.input = json!({"body":private_body});
let reserved = store.reserve(original.clone()).unwrap();
// A reader retaining the old SQLite snapshot prevents truncation of the old WAL.
let reader = Connection::open(dir.path().join("jobs.sqlite3")).unwrap();
reader
.execute_batch("BEGIN; SELECT request_json FROM job_intents;")
.unwrap();
store.conn.busy_timeout(std::time::Duration::ZERO).unwrap();
assert!(
store
.reclaim_input(
"physical-erasure",
&reserved.input_digest,
JobInputRetirementReason::ExplicitErasure
)
.is_err()
);
let retired = store.get("physical-erasure").unwrap();
assert_eq!(retired.input_disposition, JobInputDisposition::Erased);
assert!(retired.load_executable_request().is_err());
reader.execute_batch("ROLLBACK").unwrap();
drop(reader);
let replay = store
.reclaim_input(
"physical-erasure",
&reserved.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert_eq!(replay, retired);
assert_eq!(store.reserve(original).unwrap(), retired);
let needle = b"private-input-retirement-original-bytes";
for entry in std::fs::read_dir(dir.path()).unwrap() {
let path = entry.unwrap().path();
if path.is_file() {
let bytes = std::fs::read(&path).unwrap();
assert!(
!bytes.windows(needle.len()).any(|bytes| bytes == needle),
"retired body remained in {}",
path.display()
);
}
}
}
#[test]
fn missing_original_null_input_is_not_treated_as_available_null() {
let (_dir, mut store) = fixture();
let mut original = request("null-original");
original.input = Value::Null;
let reserved = store.reserve(original).unwrap();
assert!(reserved.load_executable_request().is_ok());
store.conn.execute_batch("DROP TRIGGER job_input_retirement; UPDATE job_intents SET request_json=json_remove(request_json,'$.input')").unwrap();
assert!(store.get("null-original").is_err());
}
#[test]
fn input_retirement_migration_preserves_receipts_and_rolls_back_failed_ddl() {
let (dir, mut store) = fixture();
let original = store.reserve(request("upgrade")).unwrap();
store.conn.execute_batch("DROP TRIGGER job_input_retirement; DROP TRIGGER job_intent_immutable;
ALTER TABLE job_intents DROP COLUMN input_disposition;
CREATE TRIGGER job_intent_immutable BEFORE UPDATE OF job_id,request_json,fingerprint,serialization_key,max_concurrent ON job_intents
BEGIN SELECT RAISE(ABORT, 'immutable Job intent'); END;
PRAGMA user_version=4;
CREATE TRIGGER job_input_retirement BEFORE UPDATE ON job_attempts BEGIN SELECT RAISE(ABORT,'collision'); END;").unwrap();
drop(store);
let path = dir.path().join("jobs.sqlite3");
assert!(JobStore::open(&path).is_err());
let conn = Connection::open(&path).unwrap();
let version: i64 = conn
.pragma_query_value(None, "user_version", |r| r.get(0))
.unwrap();
assert_eq!(version, 4);
let present: bool = conn.query_row("SELECT EXISTS(SELECT 1 FROM pragma_table_info('job_intents') WHERE name='input_disposition')",[],|r|r.get(0)).unwrap();
assert!(!present);
conn.execute_batch("DROP TRIGGER job_input_retirement")
.unwrap();
drop(conn);
let mut reopened = JobStore::open(&path).unwrap();
assert_eq!(reopened.get("upgrade").unwrap(), original);
assert_eq!(
reopened
.get("upgrade")
.unwrap()
.load_executable_request()
.unwrap(),
&request("upgrade")
);
let retired = reopened
.reclaim_input(
"upgrade",
&original.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert_eq!(retired.state, JobState::Pending);
assert!(
reopened
.conn
.execute_batch("UPDATE job_intents SET input_disposition='available'")
.is_err()
);
}
#[test]
fn database_configuration_schema_and_identity_constraints() {
let (_dir, mut store) = fixture();
@@ -143,6 +143,42 @@ mod tests {
(original, receipt, grant)
}
#[test]
fn legacy_archive_input_erasure_preserves_fingerprints_and_attempt_receipts() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("jobs.sqlite3");
seed(&path, 2);
let mut store = JobStore::open(&path).unwrap();
let before: (String, String) = store
.conn
.query_row(
"SELECT fingerprint,attempts_json FROM legacy_job_archive WHERE job_id='done'",
[],
|r| Ok((r.get(0)?, r.get(1)?)),
)
.unwrap();
let snapshot = store.get("done").unwrap();
store
.reclaim_input(
"done",
&snapshot.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
let (body, fingerprint, attempts): (String,String,String) = store.conn.query_row("SELECT request_json,fingerprint,attempts_json FROM legacy_job_archive WHERE job_id='done'",[],|r|Ok((r.get(0)?,r.get(1)?,r.get(2)?))).unwrap();
assert!(serde_json::from_str::<Value>(&body).unwrap()["input"].is_null());
assert_eq!((fingerprint, attempts), before);
assert!(store.conn.execute_batch("UPDATE legacy_job_archive SET request_json=json_set(request_json,'$.input','restored') WHERE job_id='done'").is_err());
drop(store);
let mut reopened = JobStore::open(&path).unwrap();
let receipt = reopened
.reserve_granted(request("done"), snapshot.domain_grant)
.unwrap();
assert_eq!(receipt.state, JobState::Completed);
assert_eq!(receipt.input_disposition, JobInputDisposition::Erased);
assert!(receipt.load_executable_request().is_err());
}
#[test]
fn legacy_versions_preserve_intent_receipts_attempts_grants_and_replay() {
for version in [1, 2] {
+113 -48
View File
@@ -26,7 +26,10 @@ use job::{JobRequest, JobResultSubmission};
use tokio::sync::watch;
use crate::job_store::JobStore;
pub use crate::job_store::{JobAttempt, JobAttemptState, JobSnapshot, JobState, JobStoreError};
pub use crate::job_store::{
JobAttempt, JobAttemptState, JobInputDisposition, JobInputRetirementReason, JobSnapshot,
JobState, JobStoreError,
};
#[derive(Debug, thiserror::Error)]
pub enum JobServiceError {
@@ -40,6 +43,8 @@ pub enum JobServiceError {
Capacity,
#[error("standalone Job cleanup failed: {0}")]
Cleanup(String),
#[error("standalone Job shutdown is incomplete; execution resources are still owned")]
ShutdownIncomplete,
}
pub(crate) trait JobAttemptFence: Send + Sync {
@@ -173,6 +178,36 @@ impl StandaloneJobs {
Ok(self.with_store(|store| store.reserve(request))?)
}
/// Enumerate historical domain snapshots for a Session, including retired input receipts.
pub fn list_session_snapshots(
&self,
session_id: &str,
after: Option<&str>,
limit: usize,
) -> Result<Vec<JobSnapshot>, JobServiceError> {
self.with_store(|store| store.list_session_snapshots(session_id, after, limit))
}
/// Retire body only. The caller fences domain effects/delivered snapshots before erasure.
/// Normal reclamation also requires that execution cleanup has finished.
pub fn reclaim_input(
&self,
job_id: &str,
expected_input_digest: &str,
reason: JobInputRetirementReason,
) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if reason == JobInputRetirementReason::Completed
&& executions
.jobs
.get(job_id)
.is_some_and(|execution| !execution.task.is_finished())
{
return Err(JobServiceError::Capacity);
}
self.with_store(|store| store.reclaim_input(job_id, expected_input_digest, reason))
}
pub fn get(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
Ok(self.with_store(|store| store.get(job_id))?)
}
@@ -274,6 +309,12 @@ impl StandaloneJobs {
job_id: &str,
client: Option<Box<dyn LlmClient>>,
) -> Result<(), JobServiceError> {
// Preparation owns copies of the immutable input and domain adapter too. Exclude
// owner shutdown before creating them, not only before publishing the task.
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
let client = client.or_else(|| {
self.model_client
.lock()
@@ -282,16 +323,14 @@ impl StandaloneJobs {
.map(|c| c.clone_boxed())
});
let snapshot = self.get(job_id)?;
snapshot.load_executable_request()?;
let timeout = Duration::from_secs(u64::from(snapshot.request.limits.timeout_seconds));
let deadline = tokio::time::Instant::now() + timeout;
let fence: Arc<dyn JobAttemptFence> = Arc::new(BoundAttemptFence {
store: self.store.clone(),
job_id: job_id.into(),
attempt_id: snapshot.attempt.attempt_id.clone(),
input_digest: snapshot
.request
.input_digest()
.map_err(JobStoreError::from)?,
input_digest: snapshot.input_digest.clone(),
deadline,
});
let grant = match (
@@ -311,10 +350,6 @@ impl StandaloneJobs {
let prepared =
worker::job::prepare_job_with_grant(&snapshot.request, &self.cwd, client, grant)
.map_err(|error| JobServiceError::Configuration(error.to_string()))?;
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
executions.jobs.retain(|_, run| {
!(run.task.is_finished() && matches!(run.complete.borrow().as_ref(), Some(Ok(()))))
});
@@ -367,10 +402,7 @@ impl StandaloneJobs {
store: store.clone(),
job_id: id.clone(),
attempt_id: attempt_id.clone(),
input_digest: snapshot
.request
.input_digest()
.map_err(JobStoreError::from)?,
input_digest: snapshot.input_digest.clone(),
deadline,
});
let (cancel, mut cancellation) = watch::channel(false);
@@ -446,23 +478,25 @@ impl StandaloneJobs {
.clone()
.expect("completion");
result.map_err(JobServiceError::Cleanup)?;
let task = {
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions
.jobs
.get(job_id)
.is_some_and(|run| run.attempt_id == attempt)
{
executions.jobs.remove(job_id)
} else {
None
}
};
if let Some(execution) = task {
execution
.task
.await
.map_err(|error| JobServiceError::Cleanup(error.to_string()))??;
// Completion is published immediately before the task returns. Do not remove
// its ownership record until the runtime confirms that return (and future drop).
while self
.executions
.lock()
.expect("Job executions poisoned")
.jobs
.get(job_id)
.is_some_and(|run| run.attempt_id == attempt && !run.task.is_finished())
{
tokio::task::yield_now().await;
}
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions
.jobs
.get(job_id)
.is_some_and(|run| run.attempt_id == attempt)
{
executions.jobs.remove(job_id);
}
}
self.get(job_id)
@@ -476,33 +510,63 @@ impl StandaloneJobs {
}
}
pub(crate) async fn shutdown(&self) -> Result<(), JobServiceError> {
/// Initiate owner shutdown without blocking a synchronous caller. Success confirms all
/// local executions have returned, including request/history and Feature cleanup. An
/// incomplete/error result retains the service resources and must retain the Host lease.
pub(crate) fn try_shutdown(&self) -> Result<(), JobServiceError> {
self.close();
let tasks = {
let mut executions = self.executions.lock().expect("Job executions poisoned");
std::mem::take(&mut executions.jobs)
};
let mut error = None;
for execution in tasks.into_values() {
match execution.task.await {
Ok(Ok(())) => {}
Ok(Err(failure)) => {
error = Some(failure);
}
Err(failure) => {
error = Some(JobServiceError::Cleanup(failure.to_string()));
let mut executions = self.executions.lock().expect("Job executions poisoned");
for execution in executions.jobs.values() {
if !execution.task.is_finished() {
return Err(JobServiceError::ShutdownIncomplete);
}
match execution.complete.borrow().as_ref() {
Some(Ok(())) => {}
Some(Err(error)) => return Err(JobServiceError::Cleanup(error.clone())),
None => {
return Err(JobServiceError::Cleanup(
"Internal Worker ended without cleanup confirmation".into(),
));
}
}
}
// Close SQLite before the Host releases its lease. Cloned handles may
// outlive the Host but cannot mutate or retain the live connection.
executions.jobs.clear();
// Retained external handles share these slots; they cannot retain a live store,
// provider or model client after successful owner shutdown.
self.store.lock().expect("Job store poisoned").take();
self.domain.lock().expect("Job domain poisoned").take();
self.model_client
.lock()
.expect("Job model transport poisoned")
.take();
error.map_or(Ok(()), Err)
Ok(())
}
pub(crate) async fn shutdown(&self) -> Result<(), JobServiceError> {
self.close();
let completions: Vec<_> = self
.executions
.lock()
.expect("Job executions poisoned")
.jobs
.values()
.map(|execution| execution.complete.clone())
.collect();
// Never take execution ownership out of the shared service across an await: a
// cancelled shutdown future or a concurrent close must still see the barrier.
for mut completion in completions {
completion
.wait_for(Option::is_some)
.await
.map_err(|error| JobServiceError::Cleanup(error.to_string()))?;
}
loop {
match self.try_shutdown() {
// Only the task's final return follows its completion notification.
Err(JobServiceError::ShutdownIncomplete) => tokio::task::yield_now().await,
result => return result,
}
}
}
}
@@ -574,7 +638,8 @@ impl JobAttemptFence for BoundAttemptFence {
if snapshot.state != JobState::Pending
|| snapshot.attempt.state != JobAttemptState::Dispatched
|| snapshot.attempt.attempt_id != self.attempt_id
|| snapshot.request.input_digest().map_err(|e| e.to_string())? != self.input_digest
|| snapshot.input_digest != self.input_digest
|| snapshot.input_disposition != JobInputDisposition::Available
{
return Err("Job domain grant is not bound to the current live attempt".into());
}
+101
View File
@@ -468,3 +468,104 @@ async fn cleanup_cannot_reclassify_success_or_durable_cancellation_and_stale_han
assert!(matches!(operation, Err(JobServiceError::Closed)));
}
}
#[tokio::test]
async fn cancelled_shutdown_future_keeps_execution_barrier_and_unknown_input() {
// Actual generic runner + SQLite; only the provider stream is scripted/blocked.
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("jobs.sqlite3");
let jobs = StandaloneJobs::open(&path, temp.path().into()).unwrap();
let reserved = jobs.request(request("cancel-shutdown-wait")).unwrap();
let model = DomainTestClient {
stall: true,
..Default::default()
};
jobs.start_with_model_client(&reserved.request.job_id, model.clone())
.await
.unwrap();
await_model(&model).await;
{
let shutdown = jobs.shutdown();
tokio::pin!(shutdown);
assert!(futures::poll!(shutdown.as_mut()).is_pending());
// Drop the caller's waiter while cancellation/cleanup is still pending.
}
assert!(matches!(
jobs.try_shutdown(),
Err(JobServiceError::ShutdownIncomplete)
));
assert!(jobs.store.lock().unwrap().is_some());
assert!(matches!(
jobs.request(request("after-shutdown")),
Err(JobServiceError::Closed)
));
tokio::time::timeout(Duration::from_secs(5), jobs.shutdown())
.await
.unwrap()
.unwrap();
assert!(jobs.store.lock().unwrap().is_none());
let reopened = StandaloneJobs::open(&path, temp.path().into()).unwrap();
let unknown = reopened.get(&reserved.request.job_id).unwrap();
assert_eq!(unknown.attempt.state, JobAttemptState::Unknown);
assert_eq!(unknown.input_disposition, JobInputDisposition::Available);
assert_eq!(unknown.request, reserved.request);
assert_eq!(unknown.input_digest, reserved.input_digest);
assert!(reopened.retry(&unknown.request.job_id).is_err());
reopened.shutdown().await.unwrap();
}
#[tokio::test]
async fn input_retirement_service_fences_live_attempt_and_preserves_session_receipts() {
// Production service + SQLite + attempt/result fences, without a model or Worker fixture.
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("jobs.sqlite3");
let jobs = StandaloneJobs::open(&path, temp.path().to_path_buf()).unwrap();
let original = request("input-retirement");
let grant = json!({"session_id":"session-a"});
let reserved = jobs
.request_granted(original.clone(), grant.clone())
.unwrap();
jobs.with_store(|store| store.dispatch("input-retirement", &reserved.attempt.attempt_id, 8))
.unwrap();
let fence = BoundAttemptFence {
store: jobs.store.clone(),
job_id: "input-retirement".into(),
attempt_id: reserved.attempt.attempt_id.clone(),
input_digest: reserved.input_digest.clone(),
deadline: tokio::time::Instant::now() + Duration::from_secs(10),
};
assert!(fence.ensure_live().is_ok());
assert!(
jobs.reclaim_input(
"input-retirement",
&reserved.input_digest,
JobInputRetirementReason::Completed
)
.is_err()
);
let erased = jobs
.reclaim_input(
"input-retirement",
&reserved.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert!(fence.ensure_live().is_err());
assert!(
sink(&jobs, "input-retirement", &reserved.attempt.attempt_id)
.submit(result("input-retirement", &reserved.attempt.attempt_id))
.is_err()
);
assert!(jobs.start("input-retirement").await.is_err());
assert_eq!(
jobs.list_session_snapshots("session-a", None, 100).unwrap(),
vec![erased.clone()]
);
assert!(
jobs.list_session_snapshots("session-b", None, 100)
.unwrap()
.is_empty()
);
assert_eq!(jobs.request_granted(original, grant).unwrap(), erased);
jobs.shutdown().await.unwrap();
}
+38
View File
@@ -7,6 +7,44 @@ pub mod launch;
pub mod store;
pub mod subjektiv;
mod subjektiv_sessions;
pub mod unix;
pub use unix::{UnixEndpoint, relay_unix_client, run_unix_client};
/// Restore reads the saved cwd and Profile, not a newly resolved launch recipe.
pub enum HeadlessLaunch {
Fresh(ResolvedStandaloneLaunch),
Restore {
state_dir: std::path::PathBuf,
worker_id: WorkerId,
},
}
#[derive(Debug, thiserror::Error)]
pub enum HeadlessError {
#[error(transparent)]
Transport(#[from] std::io::Error),
#[error(transparent)]
Startup(#[from] StandaloneStartupError),
#[error(transparent)]
Shutdown(#[from] StandaloneShutdownError),
}
/// Reserve the optional public transport before creating or restoring execution state.
pub async fn run_headless(
launch: HeadlessLaunch,
endpoint: std::path::PathBuf,
) -> Result<(), HeadlessError> {
let endpoint = UnixEndpoint::bind(endpoint)?;
let host = match launch {
HeadlessLaunch::Fresh(launch) => StandaloneHost::start(launch).await?,
HeadlessLaunch::Restore {
state_dir,
worker_id,
} => StandaloneHost::restore(state_dir, worker_id).await?,
};
host.serve_unix(endpoint).await
}
pub use host::{
StandaloneHost, StandaloneShutdownError, StandaloneStartupError,
+11 -1
View File
@@ -54,6 +54,8 @@ pub enum SubjectError {
Journal(#[from] subjektiv::journal::JournalError),
#[error(transparent)]
Storage(#[from] feature_storage::FeatureStorageError),
#[error("local Subject Job shutdown: {0}")]
JobShutdown(#[from] crate::jobs::JobServiceError),
#[error("local Subject storage metadata: {0}")]
Json(#[from] serde_json::Error),
#[error("local Subject storage permission or identity is invalid: {0}")]
@@ -338,8 +340,16 @@ impl SubjectConnection {
}
}
pub fn close(&mut self) -> Result<(), SubjectError> {
self.host.jobs.lock().expect("Subject Jobs poisoned").take();
self.host.live.store(false, Ordering::Release);
{
let mut jobs = self.host.jobs.lock().expect("Subject Jobs poisoned");
if let Some(service) = jobs.as_ref() {
// Signal cancellation, but retain the handle and exclusive lease until
// the generic owner confirms local request/history/resource teardown.
service.try_shutdown()?;
}
jobs.take();
}
self.host.catalog.manager.shutdown()?;
if let Some(file) = &self.lease {
file.unlock()?;
+498 -231
View File
@@ -5,9 +5,12 @@
//! recovery authority; shutdown does not delete either original Sessions or pending requests.
use super::*;
use crate::jobs::{JobAttemptState, JobSnapshot, JobState};
use crate::jobs::{
JobAttemptState, JobInputDisposition, JobInputRetirementReason, JobSnapshot, JobState,
};
use session_store::capture::{CaptureDto, CaptureLimits, RecordingPosition};
use subjektiv::api::JobAttemptBinding;
use subjektiv::journal::recording::JournalMaterial;
use subjektiv::journal::recording::JournalInput;
use subjektiv::journal::transport::*;
use subjektiv::journal::*;
use worker::subjektiv::{JournalJobFinalizer, JournalJobHost, JournalJobToolsFeature};
@@ -16,7 +19,6 @@ use worker::subjektiv::{JournalJobFinalizer, JournalJobHost, JournalJobToolsFeat
mod tests;
pub(super) const PURPOSE: &str = "subjektiv_journal";
const INPUT_BUDGET: usize = 48 * 1024;
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
@@ -27,6 +29,7 @@ struct LocalJournalGrant {
session_id: String,
journal_id: String,
ranges: Vec<JournalSourceRange>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
origins: Vec<SourceOrigin>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
@@ -35,11 +38,6 @@ struct SourceOrigin {
pointer: JournalSourcePointer,
origin: session_store::public_material::PublicMaterialOrigin,
}
impl LocalJournalGrant {
fn input(&self) -> serde_json::Value {
serde_json::json!({"selected_ranges": self.ranges, "origins": self.origins})
}
}
impl StandaloneSubjects {
/// Explicit operator erasure, separate from ordinary Worker removal. Requires the
@@ -84,32 +82,29 @@ impl StandaloneSubjects {
self.state_root.clone(),
)
.map_err(|error| SubjectError::InvalidScope(error.to_string()))?;
for snapshot in jobs
.pending()
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?
.into_iter()
.chain(
jobs.unknown_granted()
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?,
)
{
if snapshot.domain_grant.as_ref().is_some_and(|grant| {
grant.get("session_id").and_then(|v| v.as_str()) == Some(session_id)
}) {
let mut after = None;
loop {
let page = jobs
.list_session_snapshots(session_id, after.as_deref(), 100)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
if page.is_empty() {
break;
}
for snapshot in &page {
jobs.cancel(&snapshot.request.job_id)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
jobs.reclaim_input(
&snapshot.request.job_id,
&snapshot.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
if snapshot.state == JobState::Completed && !snapshot.acknowledged {
jobs.acknowledge(&snapshot.request.job_id)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
}
}
}
for snapshot in jobs
.unacknowledged_results()
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?
{
if snapshot.domain_grant.as_ref().is_some_and(|grant| {
grant.get("session_id").and_then(|v| v.as_str()) == Some(session_id)
}) {
jobs.acknowledge(&snapshot.request.job_id)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
}
after = page.last().map(|job| job.request.job_id.clone());
}
session_store::FsStore::new_with_retention(
path,
@@ -590,17 +585,28 @@ impl LocalSubjectHost {
self.ensure_live()?;
let jobs = self.jobs()?;
for saved in jobs.unacknowledged_results().map_err(service_error)? {
if self.journal_grant(&saved).is_ok() {
if self.journal_grant_scope(&saved).is_ok() {
self.complete_journal_job(&jobs, &saved)?;
}
}
for pending in jobs.pending().map_err(service_error)? {
if self.journal_grant(&pending).is_ok() {
if pending.request.purpose != PURPOSE {
continue;
}
self.journal_grant_scope(&pending)?;
if pending.request.input.get("capture").is_some() {
self.journal_grant(&pending)?;
return Ok(Some((jobs, pending.request.job_id)));
}
// Old descriptor jobs never acquire new execution authority. Their immutable
// identity/input is retired, not upgraded in place; saved checkpoint wins.
let replacement = self.replace_legacy_journal_job(&jobs, &pending)?;
if let Some(id) = replacement {
return Ok(Some((jobs, id)));
}
}
for unknown in jobs.unknown_granted().map_err(service_error)? {
let Ok(grant) = self.journal_grant(&unknown) else {
let Ok(grant) = self.journal_grant_scope(&unknown) else {
continue;
};
let context = authority(self, &grant.session_id)?;
@@ -685,32 +691,96 @@ impl LocalSubjectHost {
}
}
}
let materials = self.select_journal_materials(&context, &journal.id)?;
if materials.is_empty() {
let mut retained = None;
// Saved body snapshots always win over descriptor migration. In particular,
// a failed continuation must not recapture its original text even if an
// invalidated legacy grant still bounds a larger unprocessed suffix.
for legacy_pass in [false, true] {
let mut after_job = None;
loop {
let page = jobs
.list_session_snapshots(
&journal.source_session_id,
after_job.as_deref(),
100,
)
.map_err(service_error)?;
if page.is_empty() {
break;
}
for saved in &page {
if saved.request.purpose != PURPOSE {
continue;
}
let legacy = saved.input_disposition
== JobInputDisposition::Invalidated
|| (saved.input_disposition == JobInputDisposition::Available
&& saved.request.input.get("capture").is_none());
if legacy != legacy_pass
|| (!legacy
&& saved.input_disposition != JobInputDisposition::Available)
{
continue;
}
let saved_grant = self.journal_grant_scope(saved)?;
if saved_grant.journal_id != journal.id {
continue;
}
if legacy {
if let Some(id) = self.replace_legacy_journal_job(&jobs, saved)? {
return Ok(Some((jobs, id)));
}
continue;
}
self.journal_grant(saved)?;
let original = JournalInput::decode(&saved.request.input)
.map_err(journal_error)?;
let continuation = self.build_journal_input(
&context,
&journal.id,
Some(&original.capture),
)?;
if !continuation.selected_ranges.is_empty() {
retained = Some(continuation);
break;
}
if saved.state == JobState::Completed && saved.acknowledged {
// Recover the acknowledgement-before-reclamation gap without
// turning a retained receipt into an executable empty input.
jobs.reclaim_input(
&saved.request.job_id,
&saved.input_digest,
JobInputRetirementReason::Completed,
)
.map_err(service_error)?;
}
}
if retained.is_some() {
break;
}
after_job = page.last().map(|job| job.request.job_id.clone());
}
if retained.is_some() {
break;
}
}
let input = match retained {
Some(input) => input,
None => self.build_journal_input(&context, &journal.id, None)?,
};
if input.selected_ranges.is_empty() {
continue;
}
let ranges = materials
.into_iter()
.map(|material| material.range)
.collect::<Vec<_>>();
let origins = self.journal_origins(&ranges)?;
let grant = LocalJournalGrant {
writer_binding_id: Some(self.writer.binding_id.clone()),
binding: self.binding.clone(),
session_id: journal.source_session_id,
journal_id: journal.id.clone(),
ranges,
origins,
};
let request = job::JobRequest {
job_id: format!("subjektiv-journal:{}", uuid::Uuid::now_v7()),
purpose: PURPOSE.into(), input_ref: format!("subjektiv://{}/journal/{}", self.binding.subject_id, journal.id),
input: grant.input(),
instruction: "ReadJournalState first. Input descriptors are the frozen canonical public ranges authorized for this Job, not already read material. Read every selected source byte with bounded ReadJournalText before recording; then read initial/current position and open text in bounded pages. Reconcile saved operation receipts before new creates. Save sequential Create/Rewrite/Fix and UpdateJournalPosition operations, then CheckpointJournal backed by saved record/no-change receipts for all selected input. Do not fix an ongoing event just because this Job/turn ended. SubmitJobResult with result: {} only after the entire selected prefix is reflected. Never infer tool success or user intent not evidenced by public input.".into(),
profile: "builtin:subjektiv-journal-recording".into(),
serialization_key: Some(format!("subjektiv-journal:{}", journal.id)),
limits: job::JobLimits { timeout_seconds: 600, ..Default::default() },
ranges: input.selected_ranges.clone(),
origins: vec![],
};
let request = self.journal_request(&journal.id, &input)?;
let saved = jobs
.request_granted(
request,
@@ -727,6 +797,138 @@ impl LocalSubjectHost {
}
}
fn replace_legacy_journal_job(
&self,
jobs: &crate::jobs::StandaloneJobs,
old: &JobSnapshot,
) -> Result<Option<String>, SubjektivHostError> {
let grant = self.journal_grant_scope(old)?;
if old.input_disposition == JobInputDisposition::Available {
old.load_executable_request()
.map_err(|e| SubjektivHostError::InvalidArgument(e.to_string()))?;
if old.request.input
!= serde_json::json!({"selected_ranges":grant.ranges,"origins":grant.origins})
{
return Err(SubjektivHostError::InvalidArgument(
"invalid legacy Journal descriptor input".into(),
));
}
} else if old.input_disposition != JobInputDisposition::Invalidated {
return Err(SubjektivHostError::InvalidArgument(
"legacy Journal input is unavailable".into(),
));
}
let context = authority(self, &grant.session_id)?;
if let Some(active) = self
.catalog
.store
.active_journal_generation(&context, &grant.journal_id)
.map_err(journal_error)?
{
if active.job.job_id != old.request.job_id {
return Err(SubjektivHostError::InvalidArgument(
"another generation owns legacy migration".into(),
));
}
self.catalog
.store
.close_journal_generation(&context, &active.id, &active.job)
.map_err(journal_error)?;
}
jobs.cancel(&old.request.job_id).map_err(service_error)?;
let current = self
.catalog
.store
.read_journal(&context, &grant.journal_id)
.map_err(journal_error)?;
let end = grant
.ranges
.last()
.ok_or_else(|| SubjektivHostError::InvalidArgument("empty legacy selection".into()))?;
let end_position = if end.bytes.end == end.source.total_bytes {
(end.source.order + 1, 0)
} else {
(end.source.order, end.bytes.end)
};
let replacement = if (current.process_order, current.process_byte) >= end_position {
None
} else {
let mut capture = self.recording_capture(&grant.session_id)?;
capture
.entries
.retain(|entry| grant.ranges.iter().any(|r| r.source.order == entry.order));
for entry in &mut capture.entries {
let range = grant
.ranges
.iter()
.find(|r| r.source.order == entry.order)
.unwrap();
let session_store::capture::CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
..
} = &mut entry.body
else {
unreachable!()
};
if source_descriptor(&grant.session_id, descriptor) != range.source
|| !grant.origins.iter().any(|origin| {
origin.pointer == range.source.pointer && origin.origin == descriptor.origin
})
|| !text.is_char_boundary(range.bytes.start as usize)
|| !text.is_char_boundary(range.bytes.end as usize)
{
return Err(SubjektivHostError::InvalidArgument(
"legacy original source identity/range changed".into(),
));
}
*text = text[range.bytes.start as usize..range.bytes.end as usize].to_owned();
*byte_offset = range.bytes.start;
}
let input = self.build_journal_input(&context, &grant.journal_id, Some(&capture))?;
let request = self.journal_request(&grant.journal_id, &input)?;
let next_grant = LocalJournalGrant {
ranges: input.selected_ranges.clone(),
origins: vec![],
..grant
};
Some(
jobs.request_granted(
request,
serde_json::to_value(next_grant)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?,
)
.map_err(service_error)?
.request
.job_id,
)
};
jobs.reclaim_input(
&old.request.job_id,
&old.input_digest,
JobInputRetirementReason::LegacyInvalidation,
)
.map_err(service_error)?;
Ok(replacement)
}
fn journal_request(
&self,
journal_id: &str,
input: &JournalInput,
) -> Result<job::JobRequest, SubjektivHostError> {
Ok(job::JobRequest {
job_id: format!("subjektiv-journal:{}", uuid::Uuid::now_v7()), purpose: PURPOSE.into(),
input_ref: format!("subjektiv://{}/journal/{journal_id}", self.binding.subject_id),
input: serde_json::to_value(input).map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?,
instruction: "Input contains the frozen canonical public source bodies, provenance and byte ranges. Use only this saved input, never re-fetch Source. ReadJournalState first and read initial/current Journal position/open text in bounded pages if needed. Reconcile saved operation receipts before new creates. Save sequential Create/Rewrite/Fix and UpdateJournalPosition operations, then CheckpointJournal backed by saved record/no-change receipts for all selected input. Do not fix an ongoing event just because this Job/turn ended. SubmitJobResult with result: {} only after the entire selected prefix is reflected. Never infer tool success or user intent not evidenced by public input.".into(),
profile: "builtin:subjektiv-journal-recording".into(),
serialization_key: Some(format!("subjektiv-journal:{journal_id}")),
limits: job::JobLimits { timeout_seconds: 600, ..Default::default() },
})
}
fn pin_journal_session(&self, session: &str) -> Result<(), SubjektivHostError> {
let path = crate::subjektiv_sessions::authorized_sessions_path(
&self.catalog.state_root,
@@ -869,11 +1071,16 @@ impl LocalSubjectHost {
Ok(())
}
fn material_reader(
fn with_recording_boundary<T>(
&self,
session: &str,
) -> Result<session_store::public_material::SessionPublicMaterialReader, SubjektivHostError>
{
read: impl FnOnce(
&Path,
session_store::SessionId,
session_store::SegmentId,
&session_store::retention::SessionRetentionGuard,
) -> Result<T, SubjektivHostError>,
) -> Result<T, SubjektivHostError> {
let sessions = crate::subjektiv_sessions::authorized_sessions_path(
&self.catalog.state_root,
&self.binding.scope_id,
@@ -906,6 +1113,11 @@ impl LocalSubjectHost {
.ok_or_else(|| SubjektivHostError::InvalidArgument("invalid Session storage".into()))?
.join("worker")
.join(&record.storage_key);
let session_id = session
.parse()
.map_err(|_| SubjektivHostError::InvalidArgument("invalid Session".into()))?;
let retention = crate::store::session_retention(&self.catalog.state_root);
let guard = retention.lock_live(session_id).map_err(source_error)?;
// This is the persisted active selector, not the newest filename/mtime or a model claim.
let metadata =
session_store::WorkerAggregateStore::open_read_only(metadata_root, &record.storage_key)
@@ -933,154 +1145,203 @@ impl LocalSubjectHost {
"no attested active lineage for the retained Session".into(),
)
})?;
session_store::public_material::SessionPublicMaterialReader::open_fs_with_retention(
&sessions,
session
.parse()
.map_err(|_| SubjektivHostError::InvalidArgument("invalid Session".into()))?,
active,
session_store::public_material::PublicMaterialLimits {
max_segments: 256,
max_entries: 100_000,
max_record_bytes: 64 * 1024 * 1024,
},
crate::store::session_retention(&self.catalog.state_root),
)
.map_err(|error| SubjektivHostError::Unavailable(error.to_string()))
read(&sessions, session_id, active, &guard)
}
fn recording_capture(&self, session: &str) -> Result<CaptureDto, SubjektivHostError> {
self.with_recording_boundary(session, |sessions, session_id, active, guard| {
session_store::capture::capture_recording_fs_guarded(
sessions,
session_id,
active,
session_store::public_material::PublicMaterialLimits::default(),
CaptureLimits {
max_bytes: 128 * 1024 * 1024,
max_entries: 100_000,
max_text_bytes: 64 * 1024 * 1024,
..Default::default()
},
guard,
)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))
})
}
fn journal_sources(
&self,
session: &str,
) -> Result<Vec<JournalSourceRange>, SubjektivHostError> {
let reader = self.material_reader(session)?;
Ok(reader
.descriptors()
.map(|descriptor| JournalSourceRange {
source: source_descriptor(session, descriptor),
bytes: JournalByteRange {
start: 0,
end: descriptor.total_bytes,
self.with_recording_boundary(session, |sessions, session_id, active, guard| {
session_store::capture::recording_catalog_fs_guarded(
sessions,
session_id,
active,
session_store::public_material::PublicMaterialLimits::default(),
guard,
|descriptors| {
descriptors
.iter()
.map(|descriptor| JournalSourceRange {
source: source_descriptor(session, descriptor),
bytes: JournalByteRange {
start: 0,
end: descriptor.total_bytes,
},
})
.collect()
},
})
.collect())
}
fn source_reader(
&self,
source: &JournalSource,
) -> Result<session_store::public_material::SessionPublicMaterialReader, SubjektivHostError>
{
let sessions = crate::subjektiv_sessions::authorized_sessions_path(
&self.catalog.state_root,
&self.binding.scope_id,
&self.binding.subject_id,
&self.catalog.store,
&source.session_id,
)?;
// Frozen, already admitted original pointers do not follow the Worker's later active
// Session/Segment. This reads that original lineage and still checks descriptor identity,
// canonical order and digest below; the shared service enforces the frozen byte range.
session_store::public_material::SessionPublicMaterialReader::open_fs_with_retention(
&sessions,
source
.session_id
.parse()
.map_err(|_| SubjektivHostError::InvalidArgument("invalid Session".into()))?,
source.pointer.segment_id,
session_store::public_material::PublicMaterialLimits {
max_segments: 256,
max_entries: 100_000,
max_record_bytes: 64 * 1024 * 1024,
},
crate::store::session_retention(&self.catalog.state_root),
)
.map_err(|error| SubjektivHostError::Unavailable(error.to_string()))
}
fn journal_origins(
&self,
ranges: &[JournalSourceRange],
) -> Result<Vec<SourceOrigin>, SubjektivHostError> {
if ranges.is_empty() {
return Err(SubjektivHostError::InvalidArgument(
"empty material selection".into(),
));
}
ranges
.iter()
.map(|range| {
let reader = self.source_reader(&range.source)?;
reader
.descriptors()
.find(|descriptor| {
source_descriptor(&range.source.session_id, descriptor) == range.source
})
.map(|descriptor| SourceOrigin {
pointer: range.source.pointer.clone(),
origin: descriptor.origin.clone(),
})
.ok_or_else(|| {
SubjektivHostError::InvalidArgument(
"source provenance/identity changed".into(),
)
})
})
.collect()
}
fn journal_source_window(
&self,
source: &JournalSource,
offset: u64,
max_bytes: usize,
) -> Result<String, SubjektivHostError> {
let reader = self.source_reader(source)?;
let descriptor = reader.descriptors().find(|descriptor| source_descriptor(&source.session_id, descriptor) == *source)
.ok_or_else(|| SubjektivHostError::InvalidArgument("canonical source identity/order/digest changed or is outside the authorized Session".into()))?;
reader
.read_content(descriptor, offset, max_bytes)
.map(|window| window.content)
.map_err(|error| SubjektivHostError::Unavailable(error.to_string()))
}
fn journal_source_material(
&self,
source: &JournalSource,
start: u64,
limit: usize,
) -> JournalResult<JournalMaterial> {
let text = self
.journal_source_window(source, start, limit.max(4))
.map_err(|error| JournalError::Conflict(error.to_string()))?;
Ok(JournalMaterial {
range: JournalSourceRange {
source: source.clone(),
bytes: JournalByteRange {
start,
end: start + text.len() as u64,
},
},
text,
)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))
})
}
fn select_journal_materials(
fn build_journal_input(
&self,
context: &JournalHostContext,
id: &str,
) -> Result<Vec<JournalMaterial>, SubjektivHostError> {
subjektiv::journal::recording::select_materials_bounded(
&self.catalog.store,
context,
id,
INPUT_BUDGET,
|source, start, limit| self.journal_source_material(source, start, limit),
)
saved: Option<&CaptureDto>,
) -> Result<JournalInput, SubjektivHostError> {
let session = self
.catalog
.store
.read_journal(context, id)
.map_err(journal_error)?
.source_session_id;
JournalInput::build(&self.catalog.store, context, id, |order, offset, budget| {
let capture = match saved {
Some(capture) => capture.clone(),
None => {
let capture = self
.with_recording_boundary(&session, |sessions, session_id, active, guard| {
session_store::capture::capture_recording_slice_fs_guarded(
sessions,
session_id,
active,
session_store::public_material::PublicMaterialLimits::default(),
RecordingPosition {
canonical_order: order,
byte_offset: offset,
},
MAX_JOURNAL_RANGES,
budget,
guard,
)
.map(|slice| slice.capture)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))
})
.map_err(|e| JournalError::Conflict(e.to_string()))?;
// Admission and capture are separate saves. An append between them
// may be in replay but has not yet been accepted for this Journal.
return self.accepted_capture_prefix(context, id, capture);
}
};
let end = capture
.entries
.last()
.map(|entry| {
capture
.recording_text(entry.order)
.map(|(descriptor, start, text)| {
if start + text.len() as u64 == descriptor.total_bytes {
RecordingPosition {
canonical_order: entry.order + 1,
byte_offset: 0,
}
} else {
RecordingPosition {
canonical_order: entry.order,
byte_offset: start + text.len() as u64,
}
}
})
})
.transpose()
.map_err(|e| JournalError::Conflict(e.to_string()))?;
if end.is_none_or(|end| (order, offset) >= (end.canonical_order, end.byte_offset)) {
return Ok(CaptureDto {
entries: vec![],
..capture
});
}
capture
.recording_slice(
RecordingPosition {
canonical_order: order,
byte_offset: offset,
},
MAX_JOURNAL_RANGES,
budget,
)
.map(|slice| slice.capture)
.map_err(|e| JournalError::Conflict(e.to_string()))
})
.map_err(journal_error)
}
fn journal_grant(
fn accepted_capture_prefix(
&self,
context: &JournalHostContext,
id: &str,
mut capture: CaptureDto,
) -> JournalResult<CaptureDto> {
let mut selected = Vec::new();
for mut entry in capture.entries {
let status = match self
.catalog
.store
.read_journal_source(context, id, entry.order)
{
Ok(status) => status,
Err(JournalError::NotFound(_)) => break,
Err(error) => return Err(error),
};
let session_store::capture::CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
..
} = &mut entry.body
else {
return Err(JournalError::InvalidInput("non-recording capture".into()));
};
if source_descriptor(&capture.session_id, descriptor) != status.source {
return Err(JournalError::Conflict(
"captured source differs from admitted identity".into(),
));
}
let Some(accepted) = status
.accepted
.iter()
.find(|range| range.start <= *byte_offset && *byte_offset < range.end)
else {
break;
};
let end = status
.unavailable
.iter()
.filter(|range| range.end > *byte_offset)
.fold(accepted.end, |end, range| {
end.min(range.start.max(*byte_offset))
});
let mut length = text.len().min(end.saturating_sub(*byte_offset) as usize);
while !text.is_char_boundary(length) {
length -= 1;
}
if length == 0 {
break;
}
text.truncate(length);
let partial = *byte_offset + (text.len() as u64) < descriptor.total_bytes;
selected.push(entry);
if partial {
break;
}
}
capture.entries = selected;
Ok(capture)
}
fn journal_grant_scope(
&self,
snapshot: &JobSnapshot,
) -> Result<LocalJournalGrant, SubjektivHostError> {
@@ -1097,7 +1358,6 @@ impl LocalSubjectHost {
|| snapshot.request.purpose != PURPOSE
|| snapshot.request.serialization_key.as_deref()
!= Some(&format!("subjektiv-journal:{}", grant.journal_id))
|| snapshot.request.input != grant.input()
{
return Err(SubjektivHostError::InvalidArgument(
"Job input/grant does not match this Journal capability".into(),
@@ -1135,12 +1395,33 @@ impl LocalSubjectHost {
Ok(grant)
}
fn journal_grant(
&self,
snapshot: &JobSnapshot,
) -> Result<LocalJournalGrant, SubjektivHostError> {
snapshot
.load_executable_request()
.map_err(|e| SubjektivHostError::InvalidArgument(e.to_string()))?;
let grant = self.journal_grant_scope(snapshot)?;
let input = JournalInput::decode(&snapshot.request.input).map_err(journal_error)?;
if input.subject_id != self.binding.subject_id
|| input.journal_id != grant.journal_id
|| input.capture.session_id != grant.session_id
|| input.selected_ranges != grant.ranges
{
return Err(SubjektivHostError::InvalidArgument(
"saved Journal body identity/ranges differ from grant".into(),
));
}
Ok(grant)
}
fn complete_journal_job(
&self,
jobs: &crate::jobs::StandaloneJobs,
snapshot: &JobSnapshot,
) -> Result<(), SubjektivHostError> {
let grant = self.journal_grant(snapshot)?;
let grant = self.journal_grant_scope(snapshot)?;
let context = authority(self, &grant.session_id)?;
let completion: JournalJobCompletion =
serde_json::from_value(snapshot.result.clone().ok_or_else(|| {
@@ -1174,6 +1455,12 @@ impl LocalSubjectHost {
}
jobs.acknowledge(&snapshot.request.job_id)
.map_err(service_error)?;
jobs.reclaim_input(
&snapshot.request.job_id,
&snapshot.input_digest,
JobInputRetirementReason::Completed,
)
.map_err(service_error)?;
if let Err(error) = self.prune_retained_workers() {
// Pruning is independent of a successful checkpoint/result. Keep the originals
// and continue draining other saved prefixes; a retention error is not model failure.
@@ -1192,47 +1479,18 @@ pub(super) fn grant(
.journal_grant(snapshot)
.map_err(|error| error.to_string())?;
let context = authority(&host, &grant.session_id).map_err(|error| error.to_string())?;
// Re-fetch selected identities before opening mutation authority. Only descriptors are
// retained in Job storage; the model must fetch actual source bytes with bounded tools.
for range in &grant.ranges {
let length = usize::try_from(
range
.bytes
.end
.checked_sub(range.bytes.start)
.ok_or("invalid source range")?,
)
.map_err(|_| "source range overflow")?;
if length > INPUT_BUDGET {
return Err("source range exceeds frozen input budget".into());
}
let text = host
.journal_source_window(&range.source, range.bytes.start, length.max(4))
.map_err(|error| error.to_string())?;
if text.len() != length {
return Err("frozen Journal source byte range changed".into());
}
}
if host
.journal_origins(&grant.ranges)
.map_err(|error| error.to_string())?
!= grant.origins
{
return Err(
"frozen Journal provenance does not match the authorized original Session".into(),
);
}
let input = JournalInput::decode(&snapshot.request.input).map_err(|e| e.to_string())?;
let input_digest = snapshot.input_digest.clone();
let generation = host
.catalog
.store
.prepare_journal_generation(
.prepare_journal_input(
&context,
&grant.journal_id,
&input,
JobAttemptBinding {
job_id: snapshot.request.job_id.clone(),
attempt_id: snapshot.attempt.attempt_id.clone(),
},
grant.ranges.clone(),
)
.map_err(|error| error.to_string())?;
let job_context = JournalJobContext::validated(
@@ -1242,12 +1500,17 @@ pub(super) fn grant(
generation.job.clone(),
)
.map_err(|error| error.to_string())?;
input
.validate_generation(&generation)
.map_err(|e| e.to_string())?;
let adapter = Arc::new(LocalJournalHost {
host,
grant,
context,
job_context,
#[cfg(test)]
generation,
input_digest,
fence,
});
Ok(worker::job::JobFeatureGrant {
@@ -1263,7 +1526,9 @@ struct LocalJournalHost {
grant: LocalJournalGrant,
context: JournalHostContext,
job_context: JournalJobContext,
#[cfg(test)]
generation: JournalGeneration,
input_digest: String,
fence: Arc<dyn crate::jobs::JobAttemptFence>,
}
impl LocalJournalHost {
@@ -1296,18 +1561,20 @@ impl JournalJobHost for LocalJournalHost {
request: JournalJobRequest,
) -> Result<JournalJobResponse, SubjektivHostError> {
let context = self.live()?;
subjektiv::journal::service::execute_bounded(
// The generic runner has supplied this exact immutable input before it can
// invoke this bound tool. Preparation alone is not evidence of delivery.
subjektiv::journal::service::attest_input_delivery(
&self.host.catalog.store,
&context,
&self.input_digest,
&self.grant.ranges,
)
.map_err(journal_error)?;
subjektiv::journal::service::execute_snapshot(
&self.host.catalog.store,
&context,
&self.input_digest,
request,
|source, start, limit| {
if source.session_id != self.generation.initial_journal.source_session_id {
return Err(JournalError::PermissionDenied(
"foreign original Session".into(),
));
}
self.host.journal_source_material(source, start, limit)
},
)
.map_err(journal_error)
}
+748 -112
View File
@@ -1,6 +1,6 @@
//! Integration of standalone attribution, original FsStore material, shared Journal storage
//! and the local Job adapter. Only the outer model/execution lease is replaced in these cases.
//! The generic Job runner is exercised separately by the standalone Jobs integration suite.
//! The scripted model also exercises saved direct input through the actual generic Job runner.
use super::*;
use crate::store::{StaleLeasePolicy, StandaloneWorkerAllocation, StandaloneWorkerStore};
@@ -144,8 +144,8 @@ impl Fixture {
.unwrap()
.contains(sentinel)
);
// Include SQLite/WAL/journal files: immutable request/grant storage must never hold
// original text, even in freed pages left behind by explicit source erasure.
// Include SQLite/WAL/journal files: erasure must scrub retired original bodies,
// including any freed pages or journal left by the immutable request store.
for entry in fs::read_dir(self.connection.jobs_path.parent().unwrap()).unwrap() {
let entry = entry.unwrap();
if entry.file_type().unwrap().is_file() {
@@ -194,9 +194,9 @@ impl Fixture {
context,
job_context,
generation,
input_digest: snapshot.input_digest.clone(),
fence: fence.clone(),
};
adapter.read_all_sources();
(adapter, fence, snapshot)
}
}
@@ -212,6 +212,7 @@ fn message(id: &str, text: &str) -> session_store::LoggedHistoryEntry {
content: vec![session_store::LoggedContentPart::Text { text: text.into() }],
},
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId(id.into()),
origin: session_store::LoggedSessionHistoryOrigin::LocalHumanInput,
derivation: None,
@@ -219,37 +220,6 @@ fn message(id: &str, text: &str) -> session_store::LoggedHistoryEntry {
}
}
impl LocalJournalHost {
fn read_all_sources(&self) {
for range in &self.generation.selected_ranges {
let mut offset = range.bytes.start as usize;
let mut content_token = None;
loop {
let response = self
.execute(JournalJobRequest::ReadText {
read: JournalTextRead {
target: JournalTextTarget::Source {
pointer: range.source.pointer.clone(),
},
offset,
limit: MAX_JOURNAL_READ_BYTES,
content_token: content_token.clone(),
},
})
.unwrap();
let JournalJobResponse::Text(text) = response else {
panic!()
};
match text.next_byte {
Some(next) => {
assert!(next > offset);
offset = next;
content_token = Some(text.content_token);
}
None => break,
}
}
}
}
fn state(&self) -> Result<JournalJobState, SubjektivHostError> {
match self.execute(JournalJobRequest::ReadState)? {
JournalJobResponse::State(state) => Ok(state),
@@ -361,8 +331,8 @@ fn duplicate_admission_reserves_one_job_and_excludes_private_prompt() {
assert_eq!(jobs.pending().unwrap().len(), 1);
let input = jobs.get(&id).unwrap().request.input.to_string();
assert!(
!input.contains("short request"),
"Job storage retains descriptors, never a second copy of original text"
input.contains("short request"),
"Job storage supplies the canonical body directly"
);
assert!(!input.contains("PRIVATE PROMPT"));
let foreign = SubjektivHostContext {
@@ -497,14 +467,16 @@ fn generation_input_stays_frozen_and_next_prefix_includes_last_input() {
&adapter.generation.job,
)
.unwrap();
let materials = fixture
let next = fixture
.host()
.select_journal_materials(&adapter.context, &adapter.grant.journal_id)
.build_journal_input(&adapter.context, &adapter.grant.journal_id, None)
.unwrap();
assert_eq!(materials[0].range.source.pointer.entry_ref, "Elast");
assert_eq!(next.selected_ranges[0].source.pointer.entry_ref, "Elast");
assert!(
materials[0]
.text
next.capture
.recording_text(next.selected_ranges[0].source.order)
.unwrap()
.2
.contains("last input while generation runs")
);
}
@@ -546,7 +518,7 @@ fn large_original_is_chunked_without_claiming_unread_suffix() {
let (adapter, _, _) = fixture.adapter();
let range = &adapter.grant.ranges[0];
assert!(range.bytes.end < range.source.total_bytes);
assert!(range.bytes.end - range.bytes.start < INPUT_BUDGET as u64);
assert!(range.bytes.end - range.bytes.start < job::MAX_JOB_INPUT_BYTES as u64);
let snapshot = fixture
.host()
.jobs()
@@ -556,7 +528,7 @@ fn large_original_is_chunked_without_claiming_unread_suffix() {
.pop()
.unwrap();
assert!(serde_json::to_vec(&snapshot.request.input).unwrap().len() < 128 * 1024);
assert!(!snapshot.request.input.to_string().contains("漢字"));
assert!(snapshot.request.input.to_string().contains("漢字"));
assert!(adapter.finish().is_err());
}
@@ -627,43 +599,35 @@ impl JournalScript {
)
} else if let Some(state) = result("state") {
let state = &state["value"];
for (index, range) in state["selected_ranges"]
.as_array()
.unwrap()
let input: serde_json::Value = request
.items
.iter()
.enumerate()
{
let mut offset = range["bytes"]["start"].as_u64().unwrap();
let mut token = Value::Null;
loop {
let id = format!("source-{index}-{offset}");
if let Some(page) = result(&id) {
let page = &page["value"];
let key = format!("{}:{id}", state["generation_id"].as_str().unwrap());
let text = page["text"].as_str().unwrap().to_owned();
if let Some(previous) =
self.source_reads.lock().unwrap().insert(key, text.clone())
{
assert_eq!(
previous, text,
"frozen source read changed within a generation"
);
}
if let Some(next) = page["next_byte"].as_u64() {
assert!(next > offset);
offset = next;
token = page["content_token"].clone();
} else {
break;
}
} else {
let input = json!({"target":{"target":"source","pointer":range["source"]["pointer"]},"offset":offset,"limit":MAX_JOURNAL_READ_BYTES,"content_token":token});
return vec![
Event::tool_use_start(0, &id, "ReadJournalText"),
Event::tool_input_delta(0, input.to_string()),
Event::tool_use_stop(0),
];
}
.filter_map(Item::as_text)
.find_map(|text| {
text.lines()
.find_map(|line| line.strip_prefix("input_json: "))
})
.map(|json| serde_json::from_str(json).unwrap())
.expect("direct saved Journal input");
let input = JournalInput::decode(&input).unwrap();
assert_eq!(
serde_json::to_value(&input.selected_ranges).unwrap(),
state["selected_ranges"]
);
for entry in &input.capture.entries {
let (_, offset, text) = input.capture.recording_text(entry.order).unwrap();
let key = format!(
"{}:{}:{offset}",
state["generation_id"].as_str().unwrap(),
entry.order
);
if let Some(previous) = self
.source_reads
.lock()
.unwrap()
.insert(key, text.to_owned())
{
assert_eq!(previous, text, "saved source changed within generation");
}
}
let (name, action) = if state["current_open"].is_null() {
@@ -1165,6 +1129,11 @@ async fn shutdown_reopen_reconciles_partial_saved_operations_without_duplicate_c
.items;
assert_eq!(entries.len(), 1);
let saved_entry = entries[0].id.clone();
let original_input = jobs.get(&id).unwrap().request.input;
fixture.append(
"newer-than-snapshot",
"NEW INPUT MUST NOT MIX INTO RESUMED SNAPSHOT",
);
jobs.shutdown().await.unwrap();
fixture.connection.close().unwrap();
let mut reopened = StandaloneSubjects::open_existing(&state)
@@ -1175,8 +1144,29 @@ async fn shutdown_reopen_reconciles_partial_saved_operations_without_duplicate_c
.unwrap();
reopened.host.bind_jobs(jobs.clone()).unwrap();
assert_eq!(jobs.get(&id).unwrap().state, JobState::Unknown);
// Domain reconciliation inspects the now-fenced prior attempt and preserves its receipts.
// Deny every original-source read while preparing and running the replacement.
let source_path = fixture
.sessions
.root_dir()
.join(&fixture.context.session_id);
let denied = fixture
._temp
.path()
.join("temporarily-unavailable-original");
fs::rename(&source_path, &denied).unwrap();
let (_, replacement) = reopened.host.next_journal_job().unwrap().unwrap();
let replacement_input = jobs.get(&replacement).unwrap().request.input;
assert_eq!(replacement_input["capture"], original_input["capture"]);
assert!(
!replacement_input
.to_string()
.contains("NEW INPUT MUST NOT MIX")
);
assert_eq!(jobs.get(&id).unwrap().request.input, original_input);
assert_eq!(
jobs.get(&id).unwrap().input_disposition,
JobInputDisposition::Available
);
assert_ne!(
replacement, id,
"unknown external execution is not blindly replayed"
@@ -1204,8 +1194,26 @@ async fn shutdown_reopen_reconciles_partial_saved_operations_without_duplicate_c
.items;
assert_eq!(entries.len(), 1);
assert_eq!(entries[0].id, saved_entry);
fs::rename(&denied, &source_path).unwrap();
let journal = reopened
.host
.catalog
.store
.read_journal(&context, &journal_id)
.unwrap();
let original = JournalInput::decode(&original_input).unwrap();
let end = original.selected_ranges.last().unwrap();
assert_eq!(
(journal.process_order, journal.process_byte),
if end.bytes.end == end.source.total_bytes {
(end.source.order + 1, 0)
} else {
(end.source.order, end.bytes.end)
}
);
assert!(reopened.host.admit_journal(&fixture.context).unwrap());
assert!(
reopened
!reopened
.host
.session_fully_reflected(&fixture.context.session_id)
.unwrap()
@@ -1301,9 +1309,9 @@ fn missing_original_material_blocks_worker_removal_without_erasing_state() {
}
#[test]
fn frozen_original_remains_readable_after_worker_switches_its_active_session() {
fn frozen_input_survives_active_session_change_and_source_tool_remains_refused() {
let fixture = Fixture::new("old Session original");
let (adapter, _, _) = fixture.adapter();
let (adapter, _, snapshot) = fixture.adapter();
let workers = StandaloneWorkerStore::open_existing(&fixture.host().catalog.state_root).unwrap();
let worker = fixture._allocation.as_ref().unwrap().worker_id();
let record = workers.load(worker).unwrap();
@@ -1317,22 +1325,30 @@ fn frozen_original_remains_readable_after_worker_switches_its_active_session() {
)),
))
.unwrap();
let response = adapter
.execute(JournalJobRequest::ReadText {
read: JournalTextRead {
target: JournalTextTarget::Source {
pointer: adapter.generation.selected_ranges[0].source.pointer.clone(),
},
offset: 0,
limit: MAX_JOURNAL_READ_BYTES,
content_token: None,
},
})
.unwrap();
let JournalJobResponse::Text(page) = response else {
panic!()
};
assert!(page.text.contains("old Session original"));
assert!(
adapter
.execute(JournalJobRequest::ReadText {
read: JournalTextRead {
target: JournalTextTarget::Source {
pointer: adapter.generation.selected_ranges[0].source.pointer.clone()
},
offset: 0,
limit: MAX_JOURNAL_READ_BYTES,
content_token: None
}
})
.is_err(),
"original Source lookup is explicitly unavailable"
);
let input = JournalInput::decode(&snapshot.request.input).unwrap();
assert!(
input
.capture
.recording_text(0)
.unwrap()
.2
.contains("old Session original")
);
assert!(
fixture
.host()
@@ -1511,14 +1527,18 @@ fn relocated_erasure_fences_old_and_retained_readers_publishers_and_late_pins()
const SENTINEL: &str = "original-plaintext-sentinel-must-never-enter-immutable-job-storage";
let mut fixture = Fixture::new(SENTINEL);
let (_, _, snapshot) = fixture.adapter();
fixture.assert_job_storage_excludes(&snapshot, SENTINEL);
assert!(snapshot.request.input.to_string().contains(SENTINEL));
let state = fixture.host().catalog.state_root.clone();
let binding = fixture.connection.binding.clone();
let session: session_store::SessionId = fixture.context.session_id.parse().unwrap();
let old_reader = fixture
.host()
.material_reader(&fixture.context.session_id)
.unwrap();
let old_reader = SessionPublicMaterialReader::open_fs_with_retention(
fixture.sessions.root_dir(),
session,
fixture.segment,
PublicMaterialLimits::default(),
crate::store::session_retention(&state),
)
.unwrap();
let descriptor = old_reader.descriptors().next().unwrap().clone();
let (worker, workers) = fixture.stop_worker();
assert!(workers.delete(worker).is_err());
@@ -1758,14 +1778,12 @@ async fn late_terminal_boundary_after_reflected_task_prefix_is_read_by_next_job(
let snapshot = jobs.get(&next).unwrap();
let grant = fixture.host().journal_grant(&snapshot).unwrap();
assert_eq!(grant.ranges.len(), 1);
assert_eq!(
grant.origins[0].origin,
PublicMaterialOrigin::HostRunBoundary
);
let content = fixture
.host()
.journal_source_window(&grant.ranges[0].source, 0, 4096)
let input = JournalInput::decode(&snapshot.request.input).unwrap();
let (descriptor, _, content) = input
.capture
.recording_text(grant.ranges[0].source.order)
.unwrap();
assert_eq!(descriptor.origin, PublicMaterialOrigin::HostRunBoundary);
let value: serde_json::Value = serde_json::from_str(&content).unwrap();
assert_eq!(value["kind"], "run_boundary");
assert_eq!(value["boundary"]["kind"], "errored");
@@ -1790,3 +1808,621 @@ async fn late_terminal_boundary_after_reflected_task_prefix_is_read_by_next_job(
assert!(fixture.host().next_journal_job().unwrap().is_none());
jobs.shutdown().await.unwrap();
}
/// Original FsStore -> bounded common capture -> persisted Job -> actual generic runner ->
/// sequential Journal tools. Only the LLM transport is scripted. Source files are deliberately
/// unavailable after persistence, and input bodies are inspected in the provider request.
#[tokio::test]
async fn saved_body_jobs_process_over_a_hundred_entries_and_utf8_without_live_source_reads() {
let fixture = Fixture::new("oldest original");
for index in 0..115 {
fixture.append(
&format!("entry-{index}"),
&format!("public original {index}"),
);
}
fixture.append("long-utf8", &"漢字🙂\n\"".repeat(24_000));
fixture.host().admit_journal(&fixture.context).unwrap();
let expected = fixture
.host()
.recording_capture(&fixture.context.session_id)
.unwrap();
let source_path = fixture
.sessions
.root_dir()
.join(&fixture.context.session_id);
let denied = fixture._temp.path().join("denied-original-source");
let mut seen: std::collections::BTreeMap<u64, String> = Default::default();
let mut jobs_processed = 0;
loop {
let Some((jobs, id)) = fixture.host().next_journal_job().unwrap() else {
break;
};
let saved = jobs.get(&id).unwrap();
let input = JournalInput::decode(&saved.request.input).unwrap();
assert!(
serde_json::to_vec(&saved.request.input).unwrap().len() <= job::MAX_JOB_INPUT_BYTES
);
assert!(input.capture.entries.len() <= MAX_JOURNAL_RANGES);
for entry in &input.capture.entries {
let (_, offset, text) = input.capture.recording_text(entry.order).unwrap();
let previous = seen.entry(entry.order).or_default();
assert_eq!(
previous.len() as u64,
offset,
"no gap, preview or duplicate prefix"
);
previous.push_str(text);
}
// Deny all catalogs/reads, including grant preparation. The model still receives
// exactly the persisted body, and the Host must not follow the active pointer.
fs::rename(&source_path, &denied).unwrap();
let model = JournalScript::default();
jobs.start_with_model_client(&id, model.clone())
.await
.unwrap();
let result = tokio::time::timeout(std::time::Duration::from_secs(20), jobs.wait(&id))
.await
.unwrap()
.unwrap();
assert_eq!(result.state, JobState::Completed);
assert!(
!model.source_reads.lock().unwrap().is_empty(),
"provider saw saved source bodies"
);
fixture.host().complete_journal_job(&jobs, &result).unwrap();
assert_eq!(
jobs.get(&id).unwrap().input_disposition,
JobInputDisposition::Reclaimed
);
fs::rename(&denied, &source_path).unwrap();
jobs_processed += 1;
assert!(jobs_processed < 20, "bounded prefix drain must progress");
}
assert!(
jobs_processed > 2,
"entry count and long text both require continuation"
);
assert_eq!(seen.len(), expected.entries.len());
for entry in &expected.entries {
assert_eq!(
seen[&entry.order],
expected.recording_text(entry.order).unwrap().2
);
}
fixture.host().jobs().unwrap().shutdown().await.unwrap();
}
#[tokio::test]
async fn descriptor_job_migrates_to_new_identity_preserving_input_and_successful_records() {
let fixture = Fixture::new("original descriptor-only input");
let (adapter, _, modern) = fixture.adapter();
let jobs = fixture.host().jobs().unwrap();
fixture
.host()
.catalog
.store
.close_journal_generation(
&adapter.context,
&adapter.generation.id,
&adapter.generation.job,
)
.unwrap();
jobs.cancel(&modern.request.job_id).unwrap();
let input = JournalInput::decode(&modern.request.input).unwrap();
let origins: Vec<_> = input
.capture
.entries
.iter()
.map(|entry| {
let (descriptor, _, _) = input.capture.recording_text(entry.order).unwrap();
SourceOrigin {
pointer: source_descriptor(&fixture.context.session_id, descriptor).pointer,
origin: descriptor.origin.clone(),
}
})
.collect();
let mut old_grant = adapter.grant.clone();
old_grant.origins = origins;
old_grant.writer_binding_id = None;
let mut request = modern.request.clone();
request.job_id = format!("legacy-descriptor:{}", uuid::Uuid::now_v7());
request.input =
serde_json::json!({"selected_ranges":old_grant.ranges,"origins":old_grant.origins});
let old = jobs
.request_granted(request, serde_json::to_value(&old_grant).unwrap())
.unwrap();
assert!(
fixture.host().journal_grant(&old).is_err(),
"descriptor jobs must not execute as snapshots"
);
let generation = fixture
.host()
.catalog
.store
.prepare_journal_generation(
&adapter.context,
&old_grant.journal_id,
JobAttemptBinding {
job_id: old.request.job_id.clone(),
attempt_id: old.attempt.attempt_id.clone(),
},
old_grant.ranges.clone(),
)
.unwrap();
let context = JournalJobContext::validated(
&fixture.host().catalog.store,
&adapter.context,
&generation.id,
generation.job.clone(),
)
.unwrap();
let saved = fixture
.host()
.catalog
.store
.execute_journal(
&context,
JournalOperation {
request_id: "legacy-record-success-response-lost".into(),
expected_state: generation.initial_journal.state_token.clone(),
action: JournalAction::Create {
title: "already saved".into(),
body: "legacy successful operation".into(),
ongoing: "unfinished".into(),
state: JournalEntryState::Open,
source_end: None,
correction_refs: vec![],
},
covered_ranges: generation.selected_ranges.clone(),
},
)
.unwrap();
fixture.append(
"new-not-selected",
"LATEST SOURCE MUST WAIT UNTIL OLD INPUT FINISHES",
);
fixture.host().admit_journal(&fixture.context).unwrap();
let (_, next) = fixture.host().next_journal_job().unwrap().unwrap();
assert_ne!(next, old.request.job_id);
let replacement = jobs.get(&next).unwrap();
let replacement_input = JournalInput::decode(&replacement.request.input).unwrap();
assert!(
!replacement
.request
.input
.to_string()
.contains("LATEST SOURCE MUST WAIT")
);
assert_eq!(replacement_input.selected_ranges, old_grant.ranges);
let retired = jobs.get(&old.request.job_id).unwrap();
assert_eq!(retired.input_disposition, JobInputDisposition::Invalidated);
assert_eq!(retired.input_digest, old.input_digest);
assert_eq!(retired.intent_fingerprint, old.intent_fingerprint);
assert!(retired.request.input.is_null());
assert!(
fixture
.host()
.catalog
.store
.execute_journal(
&context,
JournalOperation {
request_id: "legacy-delayed".into(),
expected_state: saved.state_token,
action: JournalAction::ConfirmUnchanged,
covered_ranges: old_grant.ranges,
}
)
.is_err()
);
jobs.start_with_model_client(&next, JournalScript::default())
.await
.unwrap();
let completed = tokio::time::timeout(std::time::Duration::from_secs(20), jobs.wait(&next))
.await
.unwrap()
.unwrap();
assert_eq!(completed.state, JobState::Completed);
fixture
.host()
.complete_journal_job(&jobs, &completed)
.unwrap();
let entries = fixture
.host()
.catalog
.store
.list_journal_entries(&adapter.context, &adapter.grant.journal_id, None, 32)
.unwrap();
assert_eq!(entries.items.len(), 1);
assert_eq!(entries.items[0].id, saved.entry_id.unwrap());
jobs.shutdown().await.unwrap();
}
#[test]
fn explicit_erase_retires_every_historical_body_across_job_history_pages() {
const SENTINEL: &str = "historical-input-erasure-sentinel-781";
let mut fixture = Fixture::new("ordinary original");
let (adapter, _, first) = fixture.adapter();
let jobs = fixture.host().jobs().unwrap();
for index in 0..105 {
let mut request = first.request.clone();
request.job_id = format!("historical-{index:03}");
request.purpose = "historical-other-domain-job".into();
request.serialization_key = None;
request.input = serde_json::json!({"body":SENTINEL, "index":index});
let saved = jobs
.request_granted(
request,
serde_json::json!({"session_id":fixture.context.session_id}),
)
.unwrap();
jobs.cancel(&saved.request.job_id).unwrap();
}
let state = fixture.host().catalog.state_root.clone();
let subject = fixture.connection.binding.subject_id.clone();
fixture.connection.close().unwrap();
StandaloneSubjects::open_existing(&state)
.unwrap()
.erase_session_completely(&subject, &fixture.context.session_id, "erase-all-history")
.unwrap();
let jobs = crate::jobs::StandaloneJobs::open(
&fixture.connection.jobs_path,
fixture._temp.path().into(),
)
.unwrap();
let mut after = None;
let mut count = 0;
loop {
let page = jobs
.list_session_snapshots(&fixture.context.session_id, after.as_deref(), 100)
.unwrap();
if page.is_empty() {
break;
}
for saved in &page {
assert_eq!(saved.input_disposition, JobInputDisposition::Erased);
assert!(saved.request.input.is_null());
assert!(saved.load_executable_request().is_err());
}
count += page.len();
after = page.last().map(|job| job.request.job_id.clone());
}
assert_eq!(count, 106);
assert!(
adapter.execute(JournalJobRequest::ReadState).is_err(),
"prepared local adapter is invalidated"
);
fixture.assert_job_storage_excludes(&jobs.get(&first.request.job_id).unwrap(), SENTINEL);
}
#[test]
fn saved_snapshot_missing_tampered_foreign_or_changed_initial_state_is_not_recaptured() {
let fixture = Fixture::new("saved immutable original");
fixture.host().admit_journal(&fixture.context).unwrap();
let (jobs, id) = fixture.host().next_journal_job().unwrap().unwrap();
let saved = jobs.get(&id).unwrap();
for changed in [serde_json::Value::Null, serde_json::json!({"capture":{}})] {
let mut malformed = saved.clone();
malformed.request.input = changed;
assert!(fixture.host().journal_grant(&malformed).is_err());
}
let original = JournalInput::decode(&saved.request.input).unwrap();
for field in ["subject_id", "journal_id", "initial_state_token"] {
let mut changed = saved.request.clone();
changed.job_id = format!("changed-{field}");
changed.serialization_key = None;
changed.input[field] = serde_json::json!("foreign-or-altered");
// Even a separately reserved immutable Job may not adopt the original capability.
let changed = jobs
.request_granted(changed, saved.domain_grant.clone().unwrap())
.unwrap();
assert!(
grant(
fixture.host().clone(),
&changed,
Arc::new(TestFence(AtomicBool::new(true)))
)
.is_err()
);
}
let mut generation = fixture
.host()
.catalog
.store
.prepare_journal_generation(
&authority(fixture.host(), &fixture.context.session_id).unwrap(),
&original.journal_id,
JobAttemptBinding {
job_id: id,
attempt_id: saved.attempt.attempt_id.clone(),
},
original.selected_ranges.clone(),
)
.unwrap();
generation.initial_journal.state_token = "changed-after-snapshot".into();
assert!(original.validate_generation(&generation).is_err());
assert_eq!(
saved.request.input,
jobs.get(&saved.request.job_id).unwrap().request.input
);
}
#[tokio::test]
async fn append_between_admission_and_body_capture_does_not_publish_unaccepted_input() {
let fixture = Fixture::new("accepted oldest prefix");
fixture.host().admit_journal(&fixture.context).unwrap();
fixture.append("not-yet-admitted", "newly saved but not accepted");
let (jobs, id) = fixture.host().next_journal_job().unwrap().unwrap();
let input = jobs.get(&id).unwrap().request.input;
assert!(input.to_string().contains("accepted oldest prefix"));
assert!(!input.to_string().contains("newly saved but not accepted"));
jobs.start_with_model_client(&id, JournalScript::default())
.await
.unwrap();
let completed = tokio::time::timeout(std::time::Duration::from_secs(20), jobs.wait(&id))
.await
.unwrap()
.unwrap();
assert_eq!(completed.state, JobState::Completed);
fixture
.host()
.complete_journal_job(&jobs, &completed)
.unwrap();
fixture.host().admit_journal(&fixture.context).unwrap();
let (_, next) = fixture.host().next_journal_job().unwrap().unwrap();
let remaining = jobs.get(&next).unwrap().request.input;
assert!(
remaining
.to_string()
.contains("newly saved but not accepted")
);
assert!(!remaining.to_string().contains("accepted oldest prefix"));
jobs.shutdown().await.unwrap();
}
/// Generic Job preparation must not attest delivery. The saved body is supplied by the
/// runner before a bound Journal tool may execute; no model-provided digest is trusted.
#[tokio::test]
async fn actual_bound_tool_attests_delivery_but_prepared_blocked_job_does_not() {
let fixture = Fixture::new("body delivered through generic runner");
fixture.host().admit_journal(&fixture.context).unwrap();
let (jobs, id) = fixture.host().next_journal_job().unwrap().unwrap();
let saved = jobs.get(&id).unwrap();
let model = JournalScript {
gate_first: true,
pause_after_record: true,
..Default::default()
};
jobs.start_with_model_client(&id, model.clone())
.await
.unwrap();
tokio::time::timeout(std::time::Duration::from_secs(20), model.entered.notified())
.await
.unwrap();
let host_context = authority(fixture.host(), &fixture.context.session_id).unwrap();
let journal_id = fixture
.host()
.catalog
.store
.journal_for_session(&host_context)
.unwrap()
.unwrap();
let generation = fixture
.host()
.catalog
.store
.active_journal_generation(&host_context, &journal_id)
.unwrap()
.unwrap();
let context = JournalJobContext::validated(
&fixture.host().catalog.store,
&host_context,
&generation.id,
generation.job.clone(),
)
.unwrap();
let undelivered = subjektiv::journal::service::execute_snapshot(
&fixture.host().catalog.store,
&context,
&saved.input_digest,
JournalJobRequest::ReadState,
)
.unwrap_err();
assert!(
undelivered.to_string().contains("delivery"),
"prepared grant must not satisfy delivery gate: {undelivered}"
);
let before = fixture
.host()
.catalog
.store
.read_journal(&host_context, &journal_id)
.unwrap();
assert_eq!((before.process_order, before.process_byte), (0, 0));
model.release.notify_one();
tokio::time::timeout(
std::time::Duration::from_secs(20),
model.record_saved.notified(),
)
.await
.unwrap();
assert!(
subjektiv::journal::service::execute_snapshot(
&fixture.host().catalog.store,
&context,
&saved.input_digest,
JournalJobRequest::ReadState
)
.is_ok(),
"actual bound tool execution attests exact saved input"
);
model.release.notify_one();
let completed = tokio::time::timeout(std::time::Duration::from_secs(20), jobs.wait(&id))
.await
.unwrap()
.unwrap();
assert_eq!(completed.state, JobState::Completed);
fixture
.host()
.complete_journal_job(&jobs, &completed)
.unwrap();
jobs.shutdown().await.unwrap();
}
#[derive(Clone, Default)]
struct BlockedJournalRequest {
entered: Arc<tokio::sync::Notify>,
dropped: Arc<tokio::sync::Notify>,
request_dropped: Arc<AtomicBool>,
}
struct OwnedBlockedJournalRequest {
// A local request copy, including the saved input, must be dropped before erasure
// can acquire the lease. This does not claim remote/provider bytes are retractable.
_request: agen::llm_client::Request,
model: BlockedJournalRequest,
}
impl Drop for OwnedBlockedJournalRequest {
fn drop(&mut self) {
self.model.request_dropped.store(true, Ordering::Release);
self.model.dropped.notify_one();
}
}
#[async_trait::async_trait]
impl agen::llm_client::LlmClient for BlockedJournalRequest {
async fn stream(
&self,
request: agen::llm_client::Request,
) -> Result<
std::pin::Pin<
Box<
dyn futures::Stream<
Item = Result<
agen::llm_client::event::Event,
agen::llm_client::error::ClientError,
>,
> + Send,
>,
>,
agen::llm_client::error::ClientError,
> {
let owned = OwnedBlockedJournalRequest {
_request: request,
model: self.clone(),
};
assert!(
owned
._request
.items
.iter()
.filter_map(agen::llm_client::types::Item::as_text)
.any(|text| text.contains("local blocked source body"))
);
self.entered.notify_one();
std::future::pending::<()>().await;
drop(owned);
unreachable!()
}
fn clone_boxed(&self) -> Box<dyn agen::llm_client::LlmClient> {
Box::new(self.clone())
}
}
/// Real generic Jobs and a retained public handle, with only the provider transport blocked.
/// Connection close must initiate cancellation and preserve the exclusive lease until local
/// request/history ownership is gone (or return an explicit incomplete barrier).
#[tokio::test]
async fn close_cancels_retained_local_job_before_erase_can_acquire_exclusive_lease() {
let mut fixture = Fixture::new("local blocked source body");
fixture.host().admit_journal(&fixture.context).unwrap();
let (jobs, id) = fixture.host().next_journal_job().unwrap().unwrap();
let retained = jobs.clone();
let model = BlockedJournalRequest::default();
retained
.start_with_model_client(&id, model.clone())
.await
.unwrap();
tokio::time::timeout(std::time::Duration::from_secs(20), model.entered.notified())
.await
.unwrap();
let state = fixture.host().catalog.state_root.clone();
let subject = fixture.connection.binding.subject_id.clone();
let close_error = fixture.connection.close().unwrap_err();
assert!(matches!(
close_error,
SubjectError::JobShutdown(crate::jobs::JobServiceError::ShutdownIncomplete)
));
assert!(fixture.connection.lease.is_some());
assert!(fixture.host().jobs.lock().unwrap().is_some());
assert!(matches!(
retained.start(&id).await,
Err(crate::jobs::JobServiceError::Closed)
));
let locally_dropped_before_erase = model.request_dropped.load(Ordering::Acquire);
let early_erasure = StandaloneSubjects::open_existing(&state)
.unwrap()
.erase_session_completely(
&subject,
&fixture.context.session_id,
"erase-blocked-local-run",
);
let early_lease_was_retained = matches!(early_erasure, Err(SubjectError::Active(_)));
let teardown_from_close = model.request_dropped.load(Ordering::Acquire)
|| tokio::time::timeout(std::time::Duration::from_secs(5), model.dropped.notified())
.await
.is_ok();
// Always tear down before asserting the observed violation, so a failed regression
// cannot leak the blocked model future into another test.
let _ = retained.shutdown().await;
assert!(model.request_dropped.load(Ordering::Acquire));
fixture.connection.close().unwrap();
assert!(fixture.connection.lease.is_none());
assert!(fixture.host().jobs.lock().unwrap().is_none());
// Cancellation is not completion/erasure. Reopening retains the same unknown
// attempt's immutable body until the explicit erasure transaction retires it.
let reopened = crate::jobs::StandaloneJobs::open(
&fixture.connection.jobs_path,
fixture._temp.path().into(),
)
.unwrap();
let unknown = reopened.get(&id).unwrap();
assert_eq!(unknown.attempt.state, crate::jobs::JobAttemptState::Unknown);
assert_eq!(unknown.input_disposition, JobInputDisposition::Available);
assert!(
unknown
.request
.input
.to_string()
.contains("local blocked source body")
);
reopened.shutdown().await.unwrap();
StandaloneSubjects::open_existing(&state)
.unwrap()
.erase_session_completely(
&subject,
&fixture.context.session_id,
"erase-blocked-local-run",
)
.unwrap();
let erased_store = crate::jobs::StandaloneJobs::open(
&fixture.connection.jobs_path,
fixture._temp.path().into(),
)
.unwrap();
let erased = erased_store.get(&id).unwrap();
assert_eq!(erased.input_disposition, JobInputDisposition::Erased);
fixture.assert_job_storage_excludes(&erased, "local blocked source body");
erased_store.shutdown().await.unwrap();
assert!(
locally_dropped_before_erase || early_lease_was_retained,
"close released exclusive lease while a retained generic Job still owned local input/history"
);
assert!(
teardown_from_close,
"connection close did not initiate cancellation of the externally retained local Job"
);
assert!(
retained.get(&id).is_err(),
"retained handles are closed after teardown"
);
}
@@ -215,6 +215,7 @@ async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_a
}
protocol::Event::TextDelta { text } if text.contains(PARTIAL) => break,
protocol::Event::RunEnd { .. } => panic!("dialog ended before cancellation"),
protocol::Event::SessionEntryCommitted { entry } if matches!(entry.data, protocol::SessionSnapshotEntryData::RunError { .. }) => panic!("dialog failed before cancellation: {entry:?}"),
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
@@ -294,6 +295,32 @@ async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_a
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
for history in raw.iter().filter_map(|record| match record {
session_store::LogEntry::AnnotatedSegmentStart { history, compacted_from: Some(_), .. } => Some(history),
_ => None,
}) {
let mut calls = std::collections::HashSet::new();
let mut results = 0;
for entry in history {
let is_tool = matches!(entry.item, session_store::LoggedItem::ToolCall { .. } | session_store::LoggedItem::ToolResult { .. });
let Some(metadata) = entry.metadata.input_metadata.as_ref().filter(|_| is_tool) else { continue; };
assert_eq!(metadata.usage().recording_material, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().memory_evidence, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().observation, session_store::InputObservationUse::Hidden);
assert!(metadata.public_item().is_none());
assert!(entry.metadata.derivation.is_some());
match &entry.item {
session_store::LoggedItem::ToolCall { call_id, .. } => { calls.insert(call_id.clone()); }
session_store::LoggedItem::ToolResult { call_id, .. } => {
assert!(calls.contains(call_id), "derived Tool result must follow its real logical call");
results += 1;
}
_ => unreachable!(),
}
}
assert!(!calls.is_empty());
assert!(results >= calls.len());
}
assert_eq!(
raw.iter()
.filter(|entry| matches!(
@@ -11,7 +11,7 @@ use agen::llm_client::{
types::Item,
};
use futures::Stream;
use session_store::{CombinedStore, FsWorkerStore, LogEntry, LoggedSessionHistoryOrigin};
use session_store::{CombinedStore, FsWorkerStore, LogEntry};
use std::pin::Pin;
const BEFORE: &str = "FROZEN_POSITION: B partially implemented; tests not run; result unconfirmed";
@@ -126,8 +126,7 @@ async fn local_host_compaction_freezes_pending_journal_through_request_and_durab
worker::PromptCatalogSource::builtins_only(),
worker::WorkerWorkspaceContext::no_workspace(),
worker::WorkerFilesystemAuthority::None,
worker::WorkerBootstrapLayout::Direct { runtime_base: fixture._temp.path().join("runtime"), bash_output_dir: fixture._temp.path().join("bash") },
worker::WorkerControllerTransport::InProcess,
worker::WorkerBootstrapLayout::Direct { runtime_base: fixture._temp.path().join("runtime") },
).with_model_client(model.clone()).with_subjektiv_host(fixture.host().clone()).prepare_restored("journal-test").await.unwrap();
assert_eq!(prepared.worker().session_id().to_string(), fixture.context.session_id);
let source_segment = prepared.worker().segment_id();
@@ -139,14 +138,14 @@ async fn local_host_compaction_freezes_pending_journal_through_request_and_durab
let worker = prepared.worker();
assert_ne!(worker.segment_id(), source_segment);
let seed = worker.store().read_all(worker.session_id(), worker.segment_id()).unwrap().into_iter().find_map(|entry| match entry { LogEntry::AnnotatedSegmentStart { history, .. } => Some(history), _ => None }).expect("durable compacted seed");
let journal_items: Vec<_> = seed.iter().filter(|entry| entry.metadata.origin == LoggedSessionHistoryOrigin::JournalContext).collect();
let journal_items: Vec<_> = seed.iter().filter(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::ReferenceInput)).collect();
assert_eq!(journal_items.len(), 1);
let journal_item = Item::from(journal_items[0].item.clone());
let body = journal_item.as_text().unwrap();
assert!(body.contains(&expected_json), "durable seed contains the full selected JSON and processing checkpoint");
assert!(body.contains(BEFORE));
assert!(!body.contains(AFTER));
assert!(seed.iter().any(|entry| entry.metadata.origin == LoggedSessionHistoryOrigin::CompactionSummary && Item::from(entry.item.clone()).as_text().is_some_and(|text| text.contains(COMPLEMENT))));
assert!(seed.iter().any(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::OpeningSummary) && Item::from(entry.item.clone()).as_text().is_some_and(|text| text.contains(COMPLEMENT))));
let requests = model.requests.lock().unwrap();
let input = requests[0].items.iter().find_map(|item| match item {
Item::Message { role: agen::llm_client::types::Role::User, .. } => item.as_text().filter(|text| text.contains("## Conversation overview/index")),
@@ -79,7 +79,7 @@ async fn recorded_idle_body_handoff_preserves_open_after_source_deletion_and_see
let sessions = session_store::FsStore::new_with_retention(workers.sessions_dir(second.worker_id()), crate::store::session_retention(&state)).unwrap();
let pointer = session_store::FsWorkerStore::new(workers.worker_metadata_dir(second.worker_id())).unwrap().read_by_name(&second.record().storage_key).unwrap().unwrap().active.unwrap();
let entries = sessions.read_all(session, pointer.segment_id.unwrap()).unwrap();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if matches!(entry.item, session_store::SystemItem::JournalStartup { .. }))).count();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if entry.metadata.input_metadata.as_ref().and_then(|input| input.feature_payload()).is_some_and(|payload| payload.schema() == "journal:startup"))).count();
assert_eq!(journal_items, 1);
assert_eq!(catalog.store.subject(&subject.id).unwrap().unwrap().behavior_md, "BEHAVIOR_ORIGINAL_UNCHANGED");
second.shutdown().await.unwrap();
@@ -1621,6 +1621,7 @@ allow = []
let history = |id: &str, item: LoggedItem| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction { operation_id: None },
derivation: None,
+296
View File
@@ -0,0 +1,296 @@
//! Optional operator transport owned by the Standalone Host, never by a Worker.
//!
//! The endpoint's existing parent must be private to the current Unix user. Binding never
//! removes an existing entry, including stale sockets. Recovery is an explicit operator action.
//! Peers must have that same uid; admitted methods still cross the normal local protocol
//! authorization boundary. Disconnect only drops a view, not the Host or its execution lease.
use std::io;
use std::os::unix::fs::{FileTypeExt, MetadataExt, PermissionsExt};
use std::path::{Path, PathBuf};
use client::transport::Socket;
use protocol::stream::{JsonLineWriter, decode_event, decode_method};
use tokio::io::{AsyncBufReadExt, BufReader};
use tokio::net::{UnixListener, UnixStream};
use tokio::sync::oneshot;
use tokio::task::{JoinHandle, JoinSet};
/// A bound endpoint reservation. Drop removes only the exact socket created by this bind.
/// It can be reserved before starting a Host, so bind failures do not allocate Workers.
pub struct UnixEndpoint {
listener: UnixListener,
path: PathBuf,
device: u64,
inode: u64,
}
impl UnixEndpoint {
pub fn bind(path: impl AsRef<Path>) -> io::Result<Self> {
let path = path.as_ref();
let name = path.file_name().ok_or_else(|| {
io::Error::new(io::ErrorKind::InvalidInput, "endpoint requires a filename")
})?;
let parent = std::fs::canonicalize(
path.parent()
.filter(|p| !p.as_os_str().is_empty())
.unwrap_or(Path::new(".")),
)?;
let metadata = std::fs::metadata(&parent)?;
// SAFETY: geteuid has no preconditions and does not mutate process identity.
let uid = unsafe { libc::geteuid() };
if !metadata.is_dir() || metadata.uid() != uid || metadata.mode() & 0o077 != 0 {
return Err(io::Error::new(
io::ErrorKind::PermissionDenied,
"endpoint parent must be a current-user-owned private directory (mode 0700)",
));
}
let path = parent.join(name);
let listener = UnixListener::bind(&path)?;
let metadata = std::fs::symlink_metadata(&path)?;
let endpoint = Self {
listener,
path,
device: metadata.dev(),
inode: metadata.ino(),
};
std::fs::set_permissions(&endpoint.path, std::fs::Permissions::from_mode(0o600))?;
Ok(endpoint)
}
pub fn path(&self) -> &Path {
&self.path
}
}
impl Drop for UnixEndpoint {
fn drop(&mut self) {
if let Ok(current) = std::fs::symlink_metadata(&self.path)
&& current.file_type().is_socket()
&& current.dev() == self.device
&& current.ino() == self.inode
{
let _ = std::fs::remove_file(&self.path);
}
}
}
pub(crate) struct UnixAdapter {
task: Option<JoinHandle<io::Result<()>>>,
stop: Option<oneshot::Sender<()>>,
}
impl UnixAdapter {
pub(crate) fn start(endpoint: UnixEndpoint, handle: worker::WorkerHandle) -> Self {
let (stop, mut stopped) = oneshot::channel();
let task = tokio::spawn(async move {
let mut connections = JoinSet::new();
let result = loop {
tokio::select! {
_ = &mut stopped => break Ok(()),
accepted = endpoint.listener.accept(), if connections.len() < 64 => {
match accepted {
Ok((stream, _)) => {
// The filesystem gate protects connection establishment; credentials
// also fence open connections to the Host's operator identity.
let allowed = stream.peer_cred().is_ok_and(|cred| {
// SAFETY: geteuid has no preconditions.
cred.uid() == unsafe { libc::geteuid() }
});
if allowed {
let socket = crate::host::connect_protocol(handle.clone());
connections.spawn(bridge(stream, socket));
}
}
Err(error) => break Err(error),
}
}
_ = connections.join_next(), if !connections.is_empty() => {}
}
};
connections.shutdown().await;
// Keep the bound endpoint alive until every connection has been reclaimed.
drop(endpoint);
result
});
Self {
task: Some(task),
stop: Some(stop),
}
}
pub(crate) async fn ended(&mut self) -> io::Result<()> {
let result = self.task.as_mut().expect("adapter task").await;
self.task.take();
result.map_err(io::Error::other)?
}
pub(crate) async fn shutdown(mut self) -> io::Result<()> {
if let Some(stop) = self.stop.take() {
let _ = stop.send(());
}
if self.task.is_some() {
self.ended().await?;
}
Ok(())
}
}
impl Drop for UnixAdapter {
fn drop(&mut self) {
if let Some(stop) = self.stop.take() {
let _ = stop.send(());
}
// Sending stop lets the task join its connection set and drop its endpoint.
// A Host dropped without confirmed shutdown keeps the production execution lease.
}
}
async fn bridge(
stream: UnixStream,
mut socket: client::transport::in_process::Socket,
) -> io::Result<()> {
let (reader, writer) = stream.into_split();
let mut reader = BufReader::new(reader).lines();
let mut writer = JsonLineWriter::new(writer);
loop {
tokio::select! {
line = reader.next_line() => {
let Some(message) = line? else { return Ok(()); };
if message.trim().is_empty() { continue; }
socket.send(message).await.map_err(io::Error::other)?;
}
event = socket.next() => {
let Some(message) = event.map_err(io::Error::other)? else { return Ok(()); };
let event = decode_event(&message).map_err(io::Error::other)?;
writer.write_event(&event).await?;
}
}
}
}
/// Connect a JSONL operator client. EOF disconnects without issuing Shutdown.
/// Input is decoded as Worker Methods and output contains Worker Events only.
pub async fn run_unix_client(endpoint: PathBuf) -> io::Result<()> {
relay_unix_client(endpoint, tokio::io::stdin(), tokio::io::stdout()).await
}
/// The CLI's stream relay, also usable by embedders that own their input/output streams.
pub async fn relay_unix_client<R, W>(endpoint: PathBuf, input: R, output: W) -> io::Result<()>
where
R: tokio::io::AsyncRead + Unpin,
W: tokio::io::AsyncWrite + Unpin,
{
let stream = UnixStream::connect(endpoint).await?;
let (reader, writer) = stream.into_split();
let mut input = BufReader::new(input).lines();
let mut to_host = JsonLineWriter::new(writer);
let mut from_host = BufReader::new(reader).lines();
let mut output = JsonLineWriter::new(output);
loop {
tokio::select! {
line = input.next_line() => {
let Some(line) = line? else { return Ok(()); };
if line.trim().is_empty() { continue; }
let method = decode_method(&line).map_err(io::Error::other)?;
to_host.write(&method).await?;
}
line = from_host.next_line() => {
let Some(line) = line? else { return Ok(()); };
let event = decode_event(&line).map_err(io::Error::other)?;
output.write_event(&event).await?;
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn endpoint_requires_private_parent_and_is_owner_only() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("host.sock");
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o755)).unwrap();
let error = UnixEndpoint::bind(&path)
.err()
.expect("public parent rejected");
assert_eq!(error.kind(), io::ErrorKind::PermissionDenied);
assert!(!path.exists());
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let endpoint = UnixEndpoint::bind(&path).unwrap();
assert_eq!(
std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
0o600
);
let client = UnixStream::connect(&path).await.unwrap();
let (server, _) = endpoint.listener.accept().await.unwrap();
assert_eq!(
client.peer_cred().unwrap().uid(),
server.peer_cred().unwrap().uid()
);
drop(endpoint);
assert!(!path.exists());
}
#[tokio::test]
async fn bind_failure_precedes_worker_state_and_startup_failure_releases_only_its_endpoint() {
let directory = tempfile::tempdir().unwrap();
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let path = directory.path().join("host.sock");
let owner = UnixEndpoint::bind(&path).unwrap();
let state_dir = directory.path().join("must-not-be-opened");
let error = crate::run_headless(
crate::HeadlessLaunch::Restore {
state_dir: state_dir.clone(),
worker_id: protocol::WorkerId::now_v7(),
},
path.clone(),
)
.await
.unwrap_err();
assert!(matches!(error, crate::HeadlessError::Transport(_)));
assert!(!state_dir.exists());
assert!(path.exists());
drop(owner);
let error = crate::run_headless(
crate::HeadlessLaunch::Restore {
state_dir,
worker_id: protocol::WorkerId::now_v7(),
},
path.clone(),
)
.await
.unwrap_err();
assert!(matches!(error, crate::HeadlessError::Startup(_)));
assert!(
!path.exists(),
"a failed Host startup must release its own bind"
);
}
#[tokio::test]
async fn bind_failure_and_old_endpoint_drop_preserve_another_owner() {
let directory = tempfile::tempdir().unwrap();
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let path = directory.path().join("host.sock");
let first = UnixEndpoint::bind(&path).unwrap();
let first_inode = std::fs::metadata(&path).unwrap().ino();
assert_eq!(
UnixEndpoint::bind(&path).err().unwrap().kind(),
io::ErrorKind::AddrInUse
);
assert_eq!(std::fs::metadata(&path).unwrap().ino(), first_inode);
std::fs::rename(&path, directory.path().join("retained.sock")).unwrap();
let second = UnixEndpoint::bind(&path).unwrap();
let second_inode = std::fs::metadata(&path).unwrap().ino();
assert_ne!(first_inode, second_inode);
drop(first);
assert_eq!(std::fs::metadata(&path).unwrap().ino(), second_inode);
let _client = UnixStream::connect(&path).await.unwrap();
let _connection = second.listener.accept().await.unwrap();
drop(second);
assert!(!path.exists());
}
}
+233
View File
@@ -608,6 +608,239 @@ async fn standalone_metadata_fails_closed_on_incomplete_or_newer_records() -> Te
Ok(())
}
#[tokio::test]
async fn headless_unix_client_uses_production_history_lease_and_local_authority() -> TestResult {
use protocol::stream::{JsonLineReader, encode_method};
use std::os::unix::fs::PermissionsExt;
use tokio::io::AsyncWriteExt;
let temp = tempfile::tempdir()?;
// Shorten only the owned endpoint's components, preserving the harness TMPDIR
// even when it is checkout-local. No process-global cwd or external /tmp fallback.
let socket_dir = tempfile::Builder::new()
.prefix("")
.rand_bytes(4)
.tempdir()?;
std::fs::set_permissions(socket_dir.path(), std::fs::Permissions::from_mode(0o700))?;
let endpoint_path = socket_dir.path().join("s");
let endpoint = standalone::UnixEndpoint::bind(&endpoint_path)?;
let state_dir = temp.path().join("state");
let launch = StandaloneLaunchConfig::new(
temp.path(),
&state_dir,
manifest::ProfileSelector::Default,
"headless-operator",
)
.resolve()?;
let model = ScriptedClient::new(vec![vec![
LlmEvent::text_block_start(0),
LlmEvent::text_delta(0, "headless production response"),
LlmEvent::text_block_stop(0, Some(StopReason::EndTurn)),
]]);
let inspection = model.clone();
let host = StandaloneHost::start_with_model_client(launch, model).await?;
let worker_id = host.worker_id();
let session_id = host.record().active_session_id;
let profile = host.record().manifest.clone();
let serve = tokio::spawn(host.serve_unix(endpoint));
// Exercise the real external client's stdin/stdout relay over the actual Unix adapter.
let (mut input, relay_input) = tokio::io::duplex(64 * 1024);
let (relay_output, output) = tokio::io::duplex(64 * 1024);
let relay = tokio::spawn(standalone::relay_unix_client(
endpoint_path.clone(),
relay_input,
relay_output,
));
let mut events = JsonLineReader::new(output);
let snapshot = tokio::time::timeout(Duration::from_secs(10), events.next::<Event>())
.await??
.expect("snapshot");
assert!(matches!(snapshot, Event::Snapshot { .. }));
let request = Method::SubmitTracked {
submission_request_id: protocol::new_submission_request_id(),
input: vec![protocol::Segment::text("headless production input")],
source: protocol::AuthenticatedInputSource::Backend {
operation_id: "forged-backend-source".into(),
},
};
let wire = encode_method(&request)?;
println!("headless input: {wire}");
input.write_all(wire.as_bytes()).await?;
input.write_all(b"\n").await?;
let mut saw_user = false;
let mut saw_response = false;
let mut saw_state = false;
tokio::time::timeout(Duration::from_secs(20), async {
loop {
let event = events.next::<Event>().await?.expect("live event");
match &event {
Event::UserMessage { segments, .. } => {
saw_user |= format!("{segments:?}").contains("headless production input");
}
Event::TextDelta { text } => {
saw_response |= text == "headless production response";
}
Event::WorkerState { .. } => saw_state = true,
Event::RunEnd { .. } => break,
_ => {}
}
println!(
"headless output: {}",
protocol::stream::encode_event(&event)?
);
}
Ok::<_, Box<dyn std::error::Error>>(())
})
.await??;
assert!(saw_user && saw_response && saw_state);
assert_eq!(inspection.requests().len(), 1);
assert!(matches!(
StandaloneHost::restore_with_model_client(
state_dir.clone(),
worker_id,
ScriptedClient::new(vec![])
)
.await,
Err(StandaloneStartupError::WorkerActive)
));
// EOF is a disconnect, not a shutdown; another view can reconnect to the same history.
drop(input);
tokio::time::timeout(Duration::from_secs(10), relay).await???;
assert!(!serve.is_finished());
let mut malformed = tokio::net::UnixStream::connect(&endpoint_path).await?;
malformed.write_all(b"not-json\n").await?;
let mut discarded = Vec::new();
use tokio::io::AsyncReadExt;
tokio::time::timeout(
Duration::from_secs(10),
malformed.read_to_end(&mut discarded),
)
.await??;
assert!(
!serve.is_finished(),
"invalid client input must not stop the Host"
);
let mut socket_client =
Client::new(client::transport::unix_socket::Socket::connect(&endpoint_path).await?);
let snapshot = tokio::time::timeout(Duration::from_secs(10), async {
loop {
let event = socket_client.next_event().await?.expect("reconnect event");
if matches!(event, Event::Snapshot { .. }) {
break Ok::<_, Box<dyn std::error::Error>>(event);
}
}
})
.await??;
let snapshot_text = format!("{snapshot:?}");
assert!(
snapshot_text.contains("headless production input"),
"{snapshot_text}"
);
assert!(
snapshot_text.contains("headless production response"),
"{snapshot_text}"
);
socket_client
.send(&Method::Shutdown {
command: protocol::WorkerCommandEnvelope::new(0),
})
.await?;
tokio::time::timeout(Duration::from_secs(10), async {
loop {
if let Some(Event::CommandAcknowledged { acknowledgement }) =
socket_client.next_event().await?
{
assert_eq!(
acknowledgement.disposition,
protocol::WorkerCommandDisposition::StaleCommandId
);
break;
}
}
Ok::<_, Box<dyn std::error::Error>>(())
})
.await??;
assert!(
!serve.is_finished(),
"stale Shutdown must not bypass Controller admission"
);
socket_client
.send(&Method::Shutdown {
command: protocol::WorkerCommandEnvelope::new(100),
})
.await?;
tokio::time::timeout(Duration::from_secs(15), serve).await???;
assert!(!endpoint_path.exists());
let store = StandaloneWorkerStore::open(&state_dir)?;
let saved = store.load(worker_id)?;
assert_eq!(saved.status, StandaloneWorkerStatus::Stopped);
use session_store::Store;
let history = session_store::FsStore::new(store.sessions_dir(worker_id))?.read_all(
saved.active_session_id,
saved.active_segment_id.expect("active segment"),
)?;
let origins = history
.iter()
.filter_map(|entry| match entry {
session_store::LogEntry::AnnotatedUserInput { history, .. } => Some(history),
_ => None,
})
.flatten()
.map(|entry| &entry.metadata.origin)
.collect::<Vec<_>>();
assert!(origins.contains(&&session_store::LoggedSessionHistoryOrigin::LocalHumanInput));
assert!(!format!("{origins:?}").contains("forged-backend-source"));
// This is the same restoration path called by the TUI, not a headless catalog/import.
let restored = StandaloneHost::restore_with_model_client(
state_dir.clone(),
worker_id,
ScriptedClient::new(vec![]),
)
.await?;
assert_eq!(restored.worker_id(), worker_id);
assert_eq!(restored.record().active_session_id, session_id);
assert_eq!(
serde_json::to_value(&restored.record().manifest)?,
serde_json::to_value(&profile)?
);
assert_eq!(
restored.record().cwd.canonical_path,
temp.path().canonicalize()?
);
let mut in_process = restored.connect();
let snapshot = tokio::time::timeout(Duration::from_secs(10), async {
loop {
let event = in_process
.next_event()
.await?
.expect("in-process restored event");
if matches!(event, Event::Snapshot { .. }) {
break Ok::<_, Box<dyn std::error::Error>>(event);
}
}
})
.await??;
assert!(format!("{snapshot:?}").contains("headless production response"));
assert!(
!endpoint_path.exists(),
"in-process restore must not recreate the adapter"
);
let runtime = state_dir
.join(worker_id.to_string())
.join("runtime")
.join(format!("standalone-{worker_id}"));
assert!(!runtime.join("sock").exists());
assert!(!runtime.join("pid").exists());
restored.shutdown().await?;
println!(
"verdict: input/state/response/disconnect/explicit shutdown, shared production restore and concurrent lease exclusion passed for {worker_id}"
);
Ok(())
}
async fn wait_for_run_end(client: &mut Client<InProcessSocket>) -> TestResult {
tokio::time::timeout(Duration::from_secs(10), async {
loop {
+21 -31
View File
@@ -479,38 +479,28 @@ fn journal_response(request: &Request) -> Vec<LlmEvent> {
}),
);
}
for (index, range) in state["selected_ranges"]
.as_array()
.unwrap()
let input: Value = request
.items
.iter()
.enumerate()
{
let mut offset = range["bytes"]["start"].as_u64().unwrap();
let mut token = Value::Null;
loop {
let id = format!("journal-source-{index}-{offset}");
match result(&id) {
Some(page) => match page["value"]["next_byte"].as_u64() {
Some(next) => {
assert!(next > offset);
offset = next;
token = page["value"]["content_token"].clone();
}
None => break,
},
None => {
return tool(
&id,
"ReadJournalText",
json!({
"target":{"target":"source","pointer":range["source"]["pointer"]},
"offset":offset,"limit":subjektiv::journal::MAX_JOURNAL_READ_BYTES,"content_token":token
}),
);
}
}
}
}
.filter_map(Item::as_text)
.find_map(|text| {
text.lines()
.find_map(|line| line.strip_prefix("input_json: "))
})
.map(|json| serde_json::from_str(json).unwrap())
.expect("saved direct Journal input");
let input = subjektiv::journal::recording::JournalInput::decode(&input).unwrap();
assert_eq!(
serde_json::to_value(&input.selected_ranges).unwrap(),
state["selected_ranges"]
);
assert!(
input
.capture
.entries
.iter()
.all(|entry| input.capture.recording_text(entry.order).is_ok())
);
let (name, action) = if state["current_open"].is_null() {
(
"CreateJournalEntry",
+2
View File
@@ -15,6 +15,8 @@ serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
sha2.workspace = true
server-api.workspace = true
session-store.workspace = true
job.workspace = true
thiserror.workspace = true
uuid = { workspace = true, features = ["v7"] }
+8
View File
@@ -1299,6 +1299,14 @@ pub type OperationResult<T> = std::result::Result<T, OperationError>;
pub fn subjektiv_store_error(error: crate::SubjektivError) -> OperationError {
match error {
crate::SubjektivError::AuthorizationDenied(reason) => {
OperationError::PermissionDenied(reason)
}
crate::SubjektivError::SubjectPermissionsConflict(_)
| crate::SubjektivError::PermissionOperationConflict(_)
| crate::SubjektivError::WorkerExecutionGrantConflict(_) => {
OperationError::Conflict(error.to_string())
}
crate::SubjektivError::SubjectScopeMismatch { .. } => {
OperationError::PermissionDenied(format!("subject_scope_mismatch: {error}"))
}
+123
View File
@@ -67,6 +67,27 @@ impl SubjektivStore {
})
}
/// Absence proof for failed body creation, including bindings never attached to a Session
/// and released lifetimes. The Host must hold its lifecycle fence across this observation
/// and recovery; absence alone does not fence a delayed Runtime create.
pub fn journal_body_is_unpublished(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row(
"SELECT NOT EXISTS(SELECT 1 FROM journal_writer_bindings
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)
AND NOT EXISTS(SELECT 1 FROM subject_session_attributions
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)",
params![subject, runtime, worker],
|row| row.get(0),
)?)
})
}
pub fn journal_worker_was_released(
&self,
subject: &str,
@@ -173,6 +194,29 @@ impl SubjektivStore {
})
}
/// End a body after the Host has confirmed its Runtime removal. This is not a
/// completed Journal handoff: source coverage, positions and saved receipts stay
/// unchanged. Revocation and closing generations are atomic, so no retained Job
/// can write after deletion even while Backend Job cleanup is being retried.
pub fn release_removed_journal_body(&self, host: &JournalHostContext) -> JournalResult<()> {
self.database.try_transaction(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
let binding: Option<String> = tx.query_row(
"SELECT binding_id FROM journal_writer_bindings WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3 ORDER BY rowid DESC LIMIT 1",
params![host.subject_id, attr.runtime_id, attr.worker_id], |r| r.get(0),
).optional()?;
let binding = binding.unwrap_or_else(|| crate::issued_id("journal-removed-body"));
tx.execute("INSERT OR IGNORE INTO journal_writer_bindings(binding_id,subject_id,runtime_id,worker_id,active) VALUES(?1,?2,?3,?4,0)",params![binding,host.subject_id,attr.runtime_id,attr.worker_id])?;
tx.execute("UPDATE journal_writer_bindings SET active=0 WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3",params![host.subject_id,attr.runtime_id,attr.worker_id])?;
// Bind legacy attributed Sessions to an inactive lifetime, never to the
// successor. Retained unfinished history is not an active writer.
tx.execute("INSERT OR IGNORE INTO journal_session_bindings(session_id,binding_id) SELECT session_id,?4 FROM subject_session_attributions WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3",params![host.subject_id,attr.runtime_id,attr.worker_id,binding])?;
tx.execute("UPDATE journal_generations SET active=0 WHERE journal_id IN (SELECT j.id FROM journals j JOIN subject_session_attributions a ON a.session_id=j.source_session_id WHERE a.subject_id=?1 AND a.runtime_id=?2 AND a.worker_id=?3)",params![host.subject_id,attr.runtime_id,attr.worker_id])?;
Ok(())
})
}
/// The Host must hold its lifecycle fence, confirm Idle, reconcile all committed original
/// Sessions (including the save/notification gap), and exclude queued/running/Unknown Jobs.
/// This transaction rejects retained writers, pending source bytes and active generations.
@@ -203,3 +247,82 @@ impl SubjektivStore {
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn unpublished_body_rejects_orphan_and_released_bindings_and_attribution_only() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _workspace, store) = crate::tests::open_store(temp.path(), "absence-tests");
let subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
let other_subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
// No Session was attached: an orphan binding is still published body evidence.
let binding = store
.claim_journal_writer(&subject.id, "runtime", "worker")
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
for (subject_id, runtime, worker) in [
(other_subject.id.as_str(), "runtime", "worker"),
(subject.id.as_str(), "other-runtime", "worker"),
(subject.id.as_str(), "runtime", "other-worker"),
] {
assert!(
store
.journal_body_is_unpublished(subject_id, runtime, worker)
.unwrap()
);
}
store.release_journal_writer(&binding).unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
store
.record_session_attribution(
SubjectSessionAttribution::new(
&subject.id,
"runtime",
"attribution-only",
"session",
)
.unwrap(),
)
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "other-runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&other_subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "never-published")
.unwrap()
);
}
}
+213
View File
@@ -5,8 +5,221 @@
//! original canonical identity while streaming and return only the requested bounded fragment.
use super::*;
use session_store::capture::{CaptureDto, CaptureLimits};
use sha2::{Digest, Sha256};
/// A Job owns only this bounded original-body prefix. The initial content condition binds
/// the separate generation snapshot (whose potentially large Journal text remains paged).
/// Neither possessing these bytes nor delivering them is a processing checkpoint.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalInput {
pub version: u32,
pub subject_id: String,
pub journal_id: String,
pub initial_state_token: String,
pub initial_open_id: Option<String>,
pub previous_fixed_end: Option<JournalSourcePointer>,
pub selected_ranges: Vec<JournalSourceRange>,
pub capture: CaptureDto,
}
impl JournalInput {
/// Capture callback runs once for this immutable Job, never once per page. Fitting uses
/// only that captured value and measures the complete serialized input, including the
/// initial Journal condition and ranges duplicated beside the capture's descriptors.
pub fn build(
store: &SubjektivStore,
host: &JournalHostContext,
journal_id: &str,
capture_once: impl FnOnce(u64, u64, usize) -> JournalResult<CaptureDto>,
) -> JournalResult<Self> {
let (journal, open, previous_fixed_end) = store.database.try_with_connection(|tx| {
super::store::check_host(store, tx, host, true)?;
Ok::<_, JournalError>((
super::store::journal(tx, host, journal_id, true)?,
super::store::open(tx, journal_id)?,
super::store::previous_fixed(tx, journal_id)?.map(|p| p.0),
))
})?;
let mut input = Self {
version: 1,
subject_id: journal.subject_id.clone(),
journal_id: journal_id.into(),
initial_state_token: journal.state_token,
initial_open_id: open.map(|e| e.id),
previous_fixed_end,
selected_ranges: Vec::new(),
capture: CaptureDto {
version: 1,
session_id: journal.source_session_id,
segment_id: "pending".into(),
view_entry_count: 0,
entries: Vec::new(),
},
};
let envelope = serde_json::to_vec(&input)?.len();
// Capture once at a fixed storage boundary. Any adjustment for the Journal envelope
// below operates on this value only, never by making another live capture request.
let budget = job::MAX_JOB_INPUT_BYTES.saturating_sub(envelope);
input.capture = capture_once(journal.process_order, journal.process_byte, budget)?;
input.selected_ranges = capture_ranges(&input.capture)?;
if input.selected_ranges.is_empty() {
return Ok(input);
}
input.fit()?;
let value = serde_json::to_value(&input)?;
Self::decode(&value)
}
fn fit(&mut self) -> JournalResult<()> {
let original = self.capture.clone();
let first = self
.selected_ranges
.first()
.ok_or_else(|| invalid("empty recording selection"))?;
let start = session_store::capture::RecordingPosition {
canonical_order: first.source.order,
byte_offset: first.bytes.start,
};
let mut budget = serde_json::to_vec(&original)?.len();
loop {
let bytes = serde_json::to_vec(self)?.len();
if bytes <= job::MAX_JOB_INPUT_BYTES {
return Ok(());
}
budget = budget
.checked_sub(bytes - job::MAX_JOB_INPUT_BYTES)
.ok_or_else(|| invalid("Journal envelope cannot fit source bytes"))?;
self.capture = original
.recording_slice(start, MAX_JOURNAL_RANGES, budget)
.map_err(|e| invalid(&e.to_string()))?
.capture;
self.selected_ranges = capture_ranges(&self.capture)?;
if self.selected_ranges.is_empty() {
return Err(invalid("Journal envelope cannot fit one source scalar"));
}
}
}
pub fn decode(value: &serde_json::Value) -> JournalResult<Self> {
if serde_json::to_vec(value)?.len() > job::MAX_JOB_INPUT_BYTES {
return Err(invalid(
"Journal input exceeds whole serialized Job input limit",
));
}
let input: Self = serde_json::from_value(value.clone())?;
input
.capture
.validate(CaptureLimits {
max_bytes: job::MAX_JOB_INPUT_BYTES,
max_entries: MAX_JOURNAL_RANGES,
max_text_bytes: job::MAX_JOB_INPUT_BYTES,
..CaptureLimits::default()
})
.map_err(|e| invalid(&e.to_string()))?;
if input.version != 1
|| input.subject_id.is_empty()
|| input.journal_id.is_empty()
|| input.initial_state_token.is_empty()
|| input.selected_ranges.is_empty()
|| input.selected_ranges.len() > MAX_JOURNAL_RANGES
|| input
.selected_ranges
.iter()
.any(|r| r.source.session_id != input.capture.session_id)
{
return Err(invalid("invalid Journal input identity or selection"));
}
if capture_ranges(&input.capture)? != input.selected_ranges {
return Err(conflict(
"Journal body snapshot does not match selected canonical ranges",
));
}
Ok(input)
}
pub fn validate_generation(&self, generation: &JournalGeneration) -> JournalResult<()> {
if generation.journal_id != self.journal_id
|| generation.initial_journal.subject_id != self.subject_id
|| generation.initial_journal.source_session_id != self.capture.session_id
|| generation.initial_journal.state_token != self.initial_state_token
|| generation.initial_open.as_ref().map(|e| &e.id) != self.initial_open_id.as_ref()
|| generation.previous_fixed_end != self.previous_fixed_end
|| generation.selected_ranges != self.selected_ranges
{
return Err(conflict(
"saved Journal input differs from its initial generation snapshot",
));
}
Ok(())
}
}
/// Converts the shared recording projection into Journal's coverage language without
/// interpreting model text or constructing a second capture representation.
pub fn capture_ranges(capture: &CaptureDto) -> JournalResult<Vec<JournalSourceRange>> {
use session_store::capture::CaptureBody;
let mut ranges = Vec::new();
for entry in &capture.entries {
let CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
..
} = &entry.body
else {
return Err(invalid(
"Journal requires canonical original recording capture",
));
};
if !entry.policy.recording_material
|| text.is_empty()
|| descriptor.identity.segment_id.to_string() != entry.source.segment_id
|| descriptor.identity.entry_ref != format!("E{}", entry.entry_id)
|| descriptor.canonical_order != entry.order
|| byte_offset
.checked_add(text.len() as u64)
.is_none_or(|end| end > descriptor.total_bytes)
{
return Err(conflict(
"recording body identity, policy or range mismatch",
));
}
let source = JournalSource {
session_id: capture.session_id.clone(),
pointer: JournalSourcePointer {
segment_id: descriptor.identity.segment_id,
entry_ref: descriptor.identity.entry_ref.clone(),
},
order: descriptor.canonical_order,
content_sha256: descriptor.sha256.clone(),
total_bytes: descriptor.total_bytes,
};
if *byte_offset == 0 && text.len() as u64 == descriptor.total_bytes {
verify_canonical_text(&source, text)?;
}
ranges.push(JournalSourceRange {
source,
bytes: JournalByteRange {
start: *byte_offset,
end: *byte_offset + text.len() as u64,
},
});
}
for pair in ranges.windows(2) {
if pair[0].source.order.checked_add(1) != Some(pair[1].source.order)
|| pair[0].bytes.end != pair[0].source.total_bytes
|| pair[1].bytes.start != 0
{
return Err(conflict(
"recording capture is not an oldest contiguous prefix",
));
}
}
Ok(ranges)
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalMaterial {
+141 -10
View File
@@ -66,8 +66,8 @@ fn text_page(
})
}
/// Adapter for Host captures bounded by their storage contract. Production source reads should
/// use [`execute_bounded`] to avoid allocating an entire source for one small read.
/// Legacy adapter for Host captures bounded by their storage contract.
/// Production Hosts use [`attest_input_delivery`] and [`execute_snapshot`] instead.
pub fn execute(
store: &SubjektivStore,
context: &JournalJobContext,
@@ -101,23 +101,149 @@ pub fn execute(
})
}
/// Reads are delivered through this service before mutation authority is enabled. The durable
/// delivery ledger stores byte ranges only: Jobs may retain immutable descriptors without
/// accumulating a permanent second copy of the original Session in their request ledgers.
/// The Host verifies original provenance/digest/length and returns a bounded UTF-8 fragment.
pub(crate) fn migrate_input_delivery(
tx: &rusqlite::Transaction<'_>,
) -> feature_storage::Result<()> {
tx.execute_batch(
"CREATE TABLE journal_generation_input_deliveries (
generation_id TEXT PRIMARY KEY NOT NULL REFERENCES journal_generations(id) ON DELETE CASCADE,
input_digest TEXT NOT NULL CHECK(length(input_digest)=71 AND substr(input_digest,1,7)='sha256:' AND substr(input_digest,8) NOT GLOB '*[^0-9a-f]*')
);
CREATE TRIGGER journal_generation_input_delivery_no_update
BEFORE UPDATE ON journal_generation_input_deliveries
BEGIN SELECT RAISE(ABORT, 'Journal input delivery bindings are immutable'); END;",
)?;
Ok(())
}
fn validate_input_digest(input_digest: &str) -> JournalResult<()> {
// Preserve JobRequest::input_digest() verbatim, including its algorithm prefix.
if input_digest.strip_prefix("sha256:").is_none_or(|hex| {
hex.len() != 64
|| !hex
.bytes()
.all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b))
}) {
return Err(JournalError::InvalidInput(
"immutable Job input digest must be sha256: followed by 64 lowercase hex digits".into(),
));
}
Ok(())
}
/// Trusted Host-only delivery evidence, never a model tool or a claim of understanding.
/// Before calling, the Host must validate the saved, immutable body-bearing input against the
/// bound Job's input digest, its Session/provenance/ranges, and the actual dispatch plan that
/// supplies that exact input. The service cannot verify bytes it does not own. This transaction
/// stores only the digest bound to the generation's immutable selection; it neither copies the
/// body nor changes records, position, reflection or checkpoint. Exact replay is allowed only
/// while the same generation and writer binding remain active; changed digests/ranges conflict.
pub fn attest_input_delivery(
store: &SubjektivStore,
context: &JournalJobContext,
input_digest: &str,
selected_ranges: &[JournalSourceRange],
) -> JournalResult<()> {
validate_input_digest(input_digest)?;
store.database.try_transaction(|tx| {
super::lifecycle::reject_erased(tx, &context.host)?;
let generation = super::store::job(store, tx, context, true)?;
if generation.selected_ranges != selected_ranges {
return Err(JournalError::Conflict(
"immutable Job input selection differs from generation".into(),
));
}
let saved: Option<String> = tx.query_row(
"SELECT input_digest FROM journal_generation_input_deliveries WHERE generation_id=?1",
[&generation.id], |row| row.get(0),
).optional()?;
if let Some(saved) = saved {
if saved != input_digest {
return Err(JournalError::Conflict(
"generation is bound to a different immutable Job input digest".into(),
));
}
} else {
tx.execute(
"INSERT INTO journal_generation_input_deliveries(generation_id,input_digest) VALUES(?1,?2)",
params![generation.id, input_digest],
)?;
}
Ok(())
})
}
fn require_input_delivery(
tx: &rusqlite::Connection,
context: &JournalJobContext,
input_digest: &str,
) -> JournalResult<()> {
validate_input_digest(input_digest)?;
let saved: Option<String> = tx
.query_row(
"SELECT input_digest FROM journal_generation_input_deliveries WHERE generation_id=?1",
[&context.generation_id],
|row| row.get(0),
)
.optional()?;
if saved.as_deref() != Some(input_digest) {
return Err(JournalError::Conflict(
"trusted Host delivery of this immutable Job input is required".into(),
));
}
Ok(())
}
/// Production sequential execution over Journal state and the Host-attested immutable input.
/// The digest is taken from the bound saved Job by the Host, not from model arguments. Every
/// request requires the matching delivery binding and an active generation/writer. Source text
/// is already in immutable input: Source targets fail explicitly, with no Session reader or
/// fallback. Receipt-backed record, position and checkpoint constraints remain unchanged.
pub fn execute_snapshot(
store: &SubjektivStore,
context: &JournalJobContext,
input_digest: &str,
request: JournalJobRequest,
) -> JournalResult<JournalJobResponse> {
execute_inner(store, context, request, Some(input_digest), |_, _, _| {
Err(JournalError::PermissionDenied(
"Source reads are unavailable for immutable Journal input".into(),
))
})
}
/// Legacy/test source-read adapter. Production Hosts must use [`execute_snapshot`].
/// This ledger attests bounded Source responses only and cannot satisfy the input-digest gate.
pub fn execute_bounded(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
read_source: impl FnMut(&JournalSource, u64, usize) -> JournalResult<recording::JournalMaterial>,
) -> JournalResult<JournalJobResponse> {
execute_inner(store, context, request, None, read_source)
}
fn execute_inner(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
input_digest: Option<&str>,
mut read_source: impl FnMut(&JournalSource, u64, usize) -> JournalResult<recording::JournalMaterial>,
) -> JournalResult<JournalJobResponse> {
let generation = store
.database
.try_with_connection(|tx| super::store::job(store, tx, context, true))?;
let generation = store.database.try_with_connection(|tx| {
super::lifecycle::reject_erased(tx, &context.host)?;
let generation = super::store::job(store, tx, context, true)?;
if let Some(input_digest) = input_digest {
require_input_delivery(tx, context, input_digest)?;
}
Ok::<_, JournalError>(generation)
})?;
let mut delivered = None;
let response = match request {
JournalJobRequest::Execute { operation } => {
require_delivery(store, context, &generation.selected_ranges)?;
if input_digest.is_none() {
require_delivery(store, context, &generation.selected_ranges)?;
}
let receipt = if matches!(operation.action, JournalAction::Checkpoint { .. }) {
crate::api::checkpoint_journal(store, context, operation)?
} else {
@@ -197,6 +323,11 @@ pub fn execute_bounded(
JournalJobRequest::ReadText { read } => {
let text = match &read.target {
JournalTextTarget::Source { pointer } => {
if input_digest.is_some() {
return Err(JournalError::PermissionDenied(
"Source target is unavailable: raw source bodies are in immutable Job input".into(),
));
}
let range = generation
.selected_ranges
.iter()
+81 -1
View File
@@ -202,7 +202,7 @@ pub(super) fn open(tx: &Connection, journal_id: &str) -> JournalResult<Option<Jo
.optional()?;
id.map(|id| entry(tx, journal_id, &id)).transpose()
}
fn previous_fixed(
pub(super) fn previous_fixed(
tx: &Connection,
journal_id: &str,
) -> JournalResult<Option<(JournalSourcePointer, u64)>> {
@@ -569,6 +569,60 @@ impl SubjektivStore {
Ok(snapshot)
})
}
/// Rebind a saved body input to a new attempt without resampling its initial state.
/// The first binding freezes the full initial Journal snapshot; retries reuse it and
/// retain operation receipts. Different Jobs cannot borrow that snapshot as authority.
pub fn prepare_journal_input(
&self,
host: &JournalHostContext,
input: &super::recording::JournalInput,
binding: JobAttemptBinding,
) -> JournalResult<JournalGeneration> {
label(&binding.job_id)?;
label(&binding.attempt_id)?;
self.database.try_transaction(|tx| {
check_host(self, tx, host, false)?;
check_session_host(self, tx, host)?;
let current = journal(tx, host, &input.journal_id, true)?;
let existing: Option<String> = tx.query_row(
"SELECT snapshot_json FROM journal_generations WHERE journal_id=?1 AND job_id=?2 AND attempt_id=?3",
params![input.journal_id, binding.job_id, binding.attempt_id], |r| r.get(0)).optional()?;
if let Some(raw) = existing {
let g = serde_json::from_str(&raw)?;
input.validate_generation(&g)?;
return Ok(g);
}
// Retained receipt lookup does not reacquire writer authority. Only a new
// generation requires the current active Subject/Session writer lifetime.
check_host(self, tx, host, true)?;
if tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_generations WHERE journal_id=?1 AND active=1)", [&input.journal_id], |r| r.get::<_, bool>(0))? {
return Err(conflict("another active generation owns this Journal"));
}
let prior: Option<String> = tx.query_row(
"SELECT snapshot_json FROM journal_generations WHERE journal_id=?1 AND job_id=?2 ORDER BY rowid LIMIT 1",
params![input.journal_id, binding.job_id], |r| r.get(0)).optional()?;
let mut g = if let Some(raw) = prior {
serde_json::from_str::<JournalGeneration>(&raw)?
} else {
JournalGeneration { id: String::new(), journal_id: input.journal_id.clone(), job: binding.clone(),
selected_ranges: input.selected_ranges.clone(), initial_journal: current.clone(),
initial_open: open(tx, &input.journal_id)?, previous_fixed_end: previous_fixed(tx, &input.journal_id)?.map(|p| p.0) }
};
input.validate_generation(&g)?;
check_ranges(tx, &current, &input.selected_ranges, true)?;
let end = prefix(&g.initial_journal, &input.selected_ranges)?;
let cursor = (current.process_order, current.process_byte);
if cursor < (g.initial_journal.process_order, g.initial_journal.process_byte) || cursor > end {
return Err(conflict("saved input no longer owns the current processing cursor"));
}
g.id = issued_id("journal-generation");
g.job = binding;
tx.execute("INSERT INTO journal_generations(id,journal_id,job_id,attempt_id,request_json,snapshot_json,active) VALUES(?1,?2,?3,?4,?5,?6,1)",
params![g.id,g.journal_id,g.job.job_id,g.job.attempt_id,serde_json::to_string(&g.selected_ranges)?,serde_json::to_string(&g)?])?;
Ok(g)
})
}
pub(super) fn validate_journal_job(&self, ctx: &JournalJobContext) -> JournalResult<()> {
self.database.try_with_connection(|tx| {
job(self, tx, ctx, true)?;
@@ -597,6 +651,32 @@ impl SubjektivStore {
Ok(())
})
}
/// Read the exact retained Job/attempt snapshot without granting writer
/// authority or creating a generation. Used for outcome recovery after removal.
pub fn retained_journal_generation(
&self,
host: &JournalHostContext,
id: &str,
binding: &JobAttemptBinding,
) -> JournalResult<Option<JournalGeneration>> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
check_session_host(self, tx, host)?;
journal(tx, host, id, true)?;
let raw: Option<String> = tx.query_row(
"SELECT snapshot_json FROM journal_generations WHERE journal_id=?1 AND job_id=?2 AND attempt_id=?3",
params![id, binding.job_id, binding.attempt_id], |r| r.get(0),
).optional()?;
raw.map(|raw| {
let generation: JournalGeneration = serde_json::from_str(&raw)?;
if generation.journal_id != id || &generation.job != binding {
return Err(conflict("retained generation identity mismatch"));
}
Ok(generation)
}).transpose()
})
}
pub fn read_journal_generation(
&self,
host: &JournalHostContext,
File diff suppressed because it is too large Load Diff
+8 -5
View File
@@ -1,8 +1,10 @@
//! Bounded, Host-bound Journal recording and sequential Job transport.
//!
//! Subject, Journal and generation authority never come from these model inputs.
//! Hosts validate live execution and public source bytes before selection; reading a
//! preview must not select the enclosing entry. Operation retries retain the entire
//! Hosts validate live execution and public source bytes before selection, save body-bearing
//! immutable Job input, and attest its exact digest and dispatch plan through the service.
//! Model requests cannot attest input delivery; Source targets are rejected by snapshot execution.
//! Initial/current Journal reads remain separate from input delivery. Operation retries retain the entire
//! JournalOperation, including its original expected_state and covered_ranges.
use serde::{Deserialize, Serialize};
@@ -50,8 +52,8 @@ pub struct JournalJobState {
pub current_journal: JournalMetadata,
pub current_open: Option<JournalEntrySummary>,
pub previous_fixed_end: Option<JournalSourcePointer>,
/// Frozen permission to fetch this prefix, not evidence of delivery or reflection.
/// The sequential service gates mutations on its durable Source-read delivery ledger.
/// Frozen source selection delivered in immutable Job input, not evidence of reflection.
/// Production execution requires the trusted Host's matching input-digest attestation.
pub selected_ranges: Vec<JournalSourceRange>,
}
@@ -83,7 +85,8 @@ pub enum JournalTextTarget {
OpenOngoing {
snapshot: JournalSnapshot,
},
/// The Host restricts this pointer to this Job's frozen public source input.
/// Legacy source-read adapter only. Production snapshot execution rejects this target;
/// raw source bodies must already be present in the immutable Job input.
Source {
pointer: JournalSourcePointer,
},
+1 -1
View File
@@ -929,7 +929,7 @@ mod tests {
let (_manager, _workspace, store) = open_store(&root, "workspace-a");
let migrated_subject = store.subject(&subject_id).unwrap().unwrap();
assert!(migrated_subject.behavior_md.is_empty());
assert!(migrated_subject.behavior_md.is_empty());
assert!(store.subject_permissions(&subject_id).unwrap().is_none());
let lifecycle_attribution = attribution(
&subject_id,
"runtime-1",
+21
View File
@@ -4,6 +4,7 @@ pub mod api;
pub mod job;
pub mod journal;
mod legacy_migrations;
pub mod permissions;
mod schema;
use legacy_migrations::{
add_candidate_decision_receipts, add_subject_behavior, add_subject_session_attribution,
@@ -82,6 +83,16 @@ pub enum SubjektivError {
},
#[error("subject `{subject_id}` behavior content changed")]
SubjectBehaviorConflict { subject_id: String },
#[error("subject `{0}` permission source changed")]
SubjectPermissionsConflict(String),
#[error("permission operation `{0}` was reused with different input")]
PermissionOperationConflict(String),
#[error("authorization denied: {0}")]
AuthorizationDenied(String),
#[error("Worker execution grant binding conflicts for `{0}`")]
WorkerExecutionGrantConflict(String),
#[error("Subject `{subject_id}` has more than {limit} retained Worker execution grants")]
WorkerExecutionGrantListOverflow { subject_id: String, limit: usize },
#[error("surface generation conflict: {0}")]
SurfaceGenerationConflict(String),
#[error("memory `{memory_id}` cannot transition from {from:?} to {to:?}")]
@@ -623,6 +634,16 @@ static MIGRATIONS: &[FeatureMigration] = &[
"add immutable candidate Journal context links",
journal::memory_links::migrate,
),
FeatureMigration::new(
12,
"add explicit Subject permission sources and immutable child execution grants",
permissions::migrate,
),
FeatureMigration::new(
13,
"bind Journal generation delivery to immutable Job input digest",
journal::service::migrate_input_delivery,
),
];
pub const REGISTRATION: FeatureRegistration =
+288
View File
@@ -0,0 +1,288 @@
//! Host-administered permission sources and immutable child grants, separate from behavior.
//!
//! The Host authenticates administrators before source writes and binds verified Worker/Profile
//! identities before grant writes. These records are not a Worker registry. The Host alone
//! authenticates root execution ownership against its canonical Backend singleton and checks
//! every live lineage edge; Journal bindings and retained grants cannot establish ownership.
//! Replays return historical evidence without restoring a source or granting current authority.
use super::*;
use server_api::authorization::{
PermissionResource, ScopedPermission, SubjectPermissions, SubjectPermissionsUpdate,
WorkerExecutionGrant, validate_authorization_identifier, validate_scoped_permissions,
};
use sha2::{Digest, Sha256};
pub const MAX_SUBJECT_WORKER_EXECUTION_GRANTS: usize = 1_000;
pub(super) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("permissions.sql"))?;
Ok(())
}
impl SubjektivStore {
/// Unset sources are `None` (deny), including migrated Subjects. This is an admin read,
/// not an effective-authorization decision; retired Subjects retain their source evidence.
pub fn subject_permissions(&self, subject_id: &str) -> Result<Option<SubjectPermissions>> {
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
source_in_connection(connection, subject_id)
})
}
/// Host-admin-only replacement. Every replacement issues a fresh change ID, including
/// identical contents. Revocation is an explicit empty replacement. A scope-wide operation
/// ID binds Subject, CAS input, and both permission vectors; result and receipt commit together.
pub fn set_subject_permissions(
&self,
subject_id: &str,
update: &SubjectPermissionsUpdate,
) -> Result<SubjectPermissions> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(&update.operation_id)
.map_err(SubjektivError::InvalidRecord)?;
if let Some(change) = &update.expected_change_id {
validate_authorization_identifier(change).map_err(SubjektivError::InvalidRecord)?;
}
validate_scope(self.scope_id(), &update.execution)?;
validate_scope(self.scope_id(), &update.delegation)?;
let fingerprint: String = Sha256::digest(serde_json::to_vec(&(subject_id, update))?)
.iter()
.map(|byte| format!("{byte:02x}"))
.collect();
self.database.try_transaction(|tx| {
require_subject(tx, subject_id)?;
let receipt: Option<(String, String)> = tx.query_row(
"SELECT request_fingerprint, change_id FROM subject_permission_receipts WHERE operation_id=?1",
[&update.operation_id], |row| Ok((row.get(0)?, row.get(1)?)),
).optional()?;
if let Some((saved, change_id)) = receipt {
if saved != fingerprint {
return Err(SubjektivError::PermissionOperationConflict(update.operation_id.clone()));
}
let raw: String = tx.query_row(
"SELECT record_json FROM subject_permission_changes WHERE subject_id=?1 AND change_id=?2",
params![subject_id, change_id], |row| row.get(0),
)?;
return Ok(serde_json::from_str(&raw)?);
}
require_active_subject(tx, subject_id)?;
let current = source_in_connection(tx, subject_id)?;
if current.as_ref().map(|source| &source.change_id) != update.expected_change_id.as_ref() {
return Err(SubjektivError::SubjectPermissionsConflict(subject_id.into()));
}
let source = SubjectPermissions {
change_id: issued_id("permissions"),
execution: update.execution.clone(),
delegation: update.delegation.clone(),
};
tx.execute("INSERT INTO subject_permission_changes(subject_id,change_id,record_json) VALUES(?1,?2,?3)",
params![subject_id, source.change_id, serde_json::to_string(&source)?])?;
tx.execute("INSERT INTO subject_permissions(subject_id,change_id) VALUES(?1,?2) ON CONFLICT(subject_id) DO UPDATE SET change_id=excluded.change_id",
params![subject_id, source.change_id])?;
tx.execute("INSERT INTO subject_permission_receipts(operation_id,subject_id,request_fingerprint,change_id) VALUES(?1,?2,?3,?4)",
params![update.operation_id, subject_id, fingerprint, source.change_id])?;
Ok(source)
})
}
/// Binds child grants only. Exact retries recover an immutable binding even after revocation;
/// that success is not authorization to execute or redelegate. New bindings require the
/// current source, an active Subject and explicit delegation containment. The Host must
/// authenticate root singleton ownership and every live lineage edge before calling;
/// a missing retained parent is not ownership evidence. Profile resolution, provider scope,
/// Worker creation and caller authentication also remain Host duties.
pub fn bind_worker_execution_grant(&self, grant: &WorkerExecutionGrant) -> Result<()> {
validate_grant(self.scope_id(), grant)?;
self.database.try_transaction(|tx| {
if let Some(existing) = grant_in_connection(tx, &grant.runtime_id, &grant.worker_id)? {
return if existing == *grant { Ok(()) } else {
Err(SubjektivError::WorkerExecutionGrantConflict(format!("{}/{}", grant.runtime_id, grant.worker_id)))
};
}
require_active_subject(tx, &grant.subject_id)?;
let source = source_in_connection(tx, &grant.subject_id)?
.ok_or_else(|| denied("Subject permissions are unset"))?;
if source.change_id != grant.source_change_id {
return Err(denied("Subject permission source is no longer current"));
}
require_covered(&source.delegation, &grant.permissions)?;
require_covered(&source.delegation, &grant.redelegation)?;
// A child grant cannot turn a body (including a released historical body) into a
// delegated child or attach a Worker historically attributed to a different Subject.
let invalid_target: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![grant.runtime_id, grant.worker_id, grant.subject_id], |row| row.get(0),
)?;
if invalid_target { return Err(denied("target is a body or belongs to another Subject")); }
let origin = require_retained_origin(tx, grant)?;
if let Some(origin) = origin {
require_covered(&origin.redelegation, &grant.permissions)?;
require_covered(&origin.redelegation, &grant.redelegation)?;
}
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![grant.runtime_id, grant.worker_id, grant.subject_id, grant.source_change_id, serde_json::to_string(grant)?])?;
Ok(())
})
}
/// Returns this Subject's retained grants, including stale grants needed for cancellation
/// and reporting. Never an ownership list or effective-authorization decision. Overflow is
/// explicit rather than silently omitting Workers from a cancellation/reporting pass.
pub fn list_subject_worker_execution_grants(
&self,
subject_id: &str,
) -> Result<Vec<WorkerExecutionGrant>> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
let mut statement = connection.prepare(
"SELECT record_json FROM worker_execution_grants WHERE subject_id=?1 ORDER BY runtime_id,worker_id LIMIT ?2",
)?;
let mut rows = statement.query(params![subject_id, MAX_SUBJECT_WORKER_EXECUTION_GRANTS + 1])?;
let mut grants = Vec::new();
while let Some(row) = rows.next()? {
if grants.len() == MAX_SUBJECT_WORKER_EXECUTION_GRANTS {
return Err(SubjektivError::WorkerExecutionGrantListOverflow {
subject_id: subject_id.into(),
limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS,
});
}
grants.push(serde_json::from_str(&row.get::<_, String>(0)?)?);
}
Ok(grants)
})
}
/// Returns retained binding evidence, not current authorization. The Host must compare the
/// active Subject, source change, current origin ownership, Profile and provider constraints.
pub fn worker_execution_grant(
&self,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
validate_authorization_identifier(runtime).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(worker).map_err(SubjektivError::InvalidRecord)?;
self.database
.try_with_connection(|connection| grant_in_connection(connection, runtime, worker))
}
}
fn source_in_connection(
connection: &Connection,
subject: &str,
) -> Result<Option<SubjectPermissions>> {
let raw: Option<String> = connection.query_row(
"SELECT c.record_json FROM subject_permissions p JOIN subject_permission_changes c ON c.subject_id=p.subject_id AND c.change_id=p.change_id WHERE p.subject_id=?1",
[subject], |row| row.get(0),
).optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn grant_in_connection(
connection: &Connection,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
let raw: Option<String> = connection
.query_row(
"SELECT record_json FROM worker_execution_grants WHERE runtime_id=?1 AND worker_id=?2",
params![runtime, worker],
|row| row.get(0),
)
.optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn validate_scope(scope: &str, permissions: &[ScopedPermission]) -> Result<()> {
validate_scoped_permissions(permissions).map_err(SubjektivError::InvalidRecord)?;
for permission in permissions {
if let PermissionResource::Workspace { workspace_id } = &permission.resource
&& workspace_id != scope
{
return Err(denied("Workspace permission is outside this store scope"));
}
}
Ok(())
}
fn validate_grant(scope: &str, grant: &WorkerExecutionGrant) -> Result<()> {
for value in [
&grant.subject_id,
&grant.origin_runtime_id,
&grant.origin_worker_id,
&grant.source_change_id,
&grant.runtime_id,
&grant.worker_id,
&grant.profile,
&grant.profile_digest,
] {
validate_authorization_identifier(value).map_err(SubjektivError::InvalidRecord)?;
}
if grant.origin_runtime_id == grant.runtime_id && grant.origin_worker_id == grant.worker_id {
return Err(denied("a child grant cannot originate from itself"));
}
validate_scope(scope, &grant.permissions)?;
validate_scope(scope, &grant.redelegation)
}
fn require_covered(ceiling: &[ScopedPermission], requested: &[ScopedPermission]) -> Result<()> {
if requested
.iter()
.all(|permission| ceiling.iter().any(|allowed| allowed.covers(permission)))
{
Ok(())
} else {
Err(denied(
"requested permission or redelegation exceeds its explicit ceiling",
))
}
}
fn require_retained_origin(
tx: &Transaction<'_>,
grant: &WorkerExecutionGrant,
) -> Result<Option<WorkerExecutionGrant>> {
let mut runtime = grant.origin_runtime_id.clone();
let mut worker = grant.origin_worker_id.clone();
let mut direct_origin = None;
let mut visited = HashSet::from([(grant.runtime_id.clone(), grant.worker_id.clone())]);
// Validate retained lineage only. Absence terminates at a Host-authenticated root;
// neither Journal active state nor absence of a grant proves current singleton ownership.
for _ in 0..64 {
if !visited.insert((runtime.clone(), worker.clone())) {
return Err(denied("cyclic grant origin"));
}
let foreign_attribution: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![runtime, worker, grant.subject_id], |row| row.get(0),
)?;
if foreign_attribution {
return Err(denied("grant origin is attributed to another Subject"));
}
let Some(origin) = grant_in_connection(tx, &runtime, &worker)? else {
return Ok(direct_origin);
};
if origin.subject_id != grant.subject_id
|| origin.source_change_id != grant.source_change_id
{
return Err(denied(
"grant origin belongs to another Subject or permission source",
));
}
if direct_origin.is_none() {
direct_origin = Some(origin.clone());
}
runtime = origin.origin_runtime_id;
worker = origin.origin_worker_id;
}
Err(denied("grant origin exceeds maximum delegation depth"))
}
fn denied(reason: &str) -> SubjektivError {
SubjektivError::AuthorizationDenied(reason.into())
}
#[cfg(test)]
mod tests;
+45
View File
@@ -0,0 +1,45 @@
CREATE TABLE subject_permission_changes (
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL CHECK (length(change_id) > 0),
record_json TEXT NOT NULL,
PRIMARY KEY (subject_id, change_id)
);
CREATE TABLE subject_permissions (
subject_id TEXT PRIMARY KEY REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE subject_permission_receipts (
operation_id TEXT PRIMARY KEY CHECK (length(operation_id) > 0),
subject_id TEXT NOT NULL,
request_fingerprint TEXT NOT NULL,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE worker_execution_grants (
runtime_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
source_change_id TEXT NOT NULL,
record_json TEXT NOT NULL,
PRIMARY KEY (runtime_id, worker_id),
FOREIGN KEY (subject_id, source_change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TRIGGER subject_permission_changes_no_update BEFORE UPDATE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are immutable');
END;
CREATE TRIGGER subject_permission_changes_no_delete BEFORE DELETE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are retained');
END;
CREATE TRIGGER subject_permission_receipts_no_update BEFORE UPDATE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are immutable');
END;
CREATE TRIGGER subject_permission_receipts_no_delete BEFORE DELETE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are retained');
END;
CREATE TRIGGER worker_execution_grants_no_update BEFORE UPDATE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are immutable');
END;
CREATE TRIGGER worker_execution_grants_no_delete BEFORE DELETE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are retained');
END;
+656
View File
@@ -0,0 +1,656 @@
//! Real SQLite component tests through store APIs; Host authentication, canonical singleton
//! ownership and live lineage checks are outside this suite. Root origins are Host-attested inputs;
//! Journal fixtures exercise defensive attribution only, never execution authority.
use super::*;
use crate::tests::{open_store, role};
use feature_storage::FeatureStorage;
use std::path::Path;
fn permission(operation: &str, path: &str, recursive: bool) -> ScopedPermission {
ScopedPermission {
operation: operation.into(),
resource: PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
},
}
}
fn update(
operation: &str,
expected: Option<&str>,
delegation: Vec<ScopedPermission>,
) -> SubjectPermissionsUpdate {
SubjectPermissionsUpdate {
operation_id: operation.into(),
expected_change_id: expected.map(Into::into),
execution: Vec::new(),
delegation,
}
}
fn grant(subject: &str, source: &str, worker: &str) -> WorkerExecutionGrant {
WorkerExecutionGrant {
subject_id: subject.into(),
origin_runtime_id: "runtime".into(),
origin_worker_id: "body".into(),
source_change_id: source.into(),
runtime_id: "runtime".into(),
worker_id: worker.into(),
profile: "builtin:coder".into(),
profile_digest: "resolved-profile-sha256".into(),
permissions: vec![permission("file.read", "src", false)],
redelegation: Vec::new(),
}
}
fn schema(store: &SubjektivStore) -> Vec<(String, String)> {
store.database.try_with_connection::<_, SubjektivError>(|connection| {
let mut statement = connection.prepare("SELECT name,sql FROM sqlite_master WHERE name NOT LIKE 'sqlite_%' AND sql IS NOT NULL ORDER BY name")?;
Ok(statement.query_map([], |row| Ok((row.get(0)?, row.get(1)?)))?.collect::<std::result::Result<_, _>>()?)
}).unwrap()
}
fn open_version(
root: &Path,
version: usize,
) -> (FeatureStorage, ScopedFeatureStorage, SubjektivStore) {
let manager = FeatureStorage::new(root);
let scope = manager.scope("workspace-a").unwrap();
let registration = scope
.register(FeatureRegistration::new(
SUBJEKTIV_FEATURE_ID,
&MIGRATIONS[..version],
))
.unwrap();
let store = SubjektivStore::open(&scope, &registration).unwrap();
(manager, scope, store)
}
#[test]
fn replacements_are_cas_fenced_replayable_and_never_restore_old_permissions() {
let temp = tempfile::tempdir().unwrap();
let subject;
let first;
let first_update = update("allow", None, vec![permission("file.read", ".", true)]);
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
subject = store.create_subject(role()).unwrap().id;
assert!(store.subject_permissions(&subject).unwrap().is_none());
first = store
.set_subject_permissions(&subject, &first_update)
.unwrap();
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert!(matches!(
store.set_subject_permissions(&subject, &update("stale", None, Vec::new())),
Err(SubjektivError::SubjectPermissionsConflict(_))
));
assert!(matches!(
store.set_subject_permissions(&subject, &update("allow", None, Vec::new())),
Err(SubjektivError::PermissionOperationConflict(_))
));
let other = store.create_subject(role()).unwrap();
assert!(matches!(
store.set_subject_permissions(&other.id, &first_update),
Err(SubjektivError::PermissionOperationConflict(_))
));
let same = store
.set_subject_permissions(
&subject,
&update("same", Some(&first.change_id), first.delegation.clone()),
)
.unwrap();
assert_ne!(same.change_id, first.change_id);
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&same.change_id), Vec::new()),
)
.unwrap();
assert!(revoked.delegation.is_empty());
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(revoked));
}
let (_reopened_manager, _reopened_scope, reopened) = open_store(temp.path(), "workspace-a");
assert!(
reopened
.subject_permissions(&subject)
.unwrap()
.unwrap()
.delegation
.is_empty()
);
assert_eq!(
reopened
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
}
#[test]
fn failed_receipt_insert_rolls_back_source_history_and_projection() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let first = store
.set_subject_permissions(&subject, &update("first", None, Vec::new()))
.unwrap();
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_permission_receipt BEFORE INSERT ON subject_permission_receipts BEGIN SELECT RAISE(ABORT,'injected receipt failure'); END;")?;
Ok(())
}).unwrap();
let replacement = update(
"second",
Some(&first.change_id),
vec![permission("file.read", ".", true)],
);
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_err()
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(first));
store.database.try_transaction::<_, SubjektivError>(|tx| {
let counts: (u32, u32) = tx.query_row("SELECT (SELECT count(*) FROM subject_permission_changes),(SELECT count(*) FROM subject_permission_receipts)", [], |r| Ok((r.get(0)?, r.get(1)?)))?;
assert_eq!(counts, (1, 1));
tx.execute_batch("DROP TRIGGER fail_permission_receipt")?;
Ok(())
}).unwrap();
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_ok()
);
}
#[test]
fn scope_and_validation_rejections_do_not_issue_permission_sources() {
let temp = tempfile::tempdir().unwrap();
let (_a_manager, _a_scope, a) = open_store(&temp.path().join("a"), "workspace-a");
let (_b_manager, _b_scope, b) = open_store(&temp.path().join("b"), "workspace-b");
let subject = a.create_subject(role()).unwrap().id;
assert!(
b.set_subject_permissions(&subject, &update("cross-subject", None, Vec::new()))
.is_err()
);
let cross_workspace = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "workspace-b".into(),
},
};
assert!(
a.set_subject_permissions(
&subject,
&update("cross-workspace", None, vec![cross_workspace])
)
.is_err()
);
for (operation, path) in [
("file.*", "src"),
("file..read", "src"),
("file.read", "../src"),
("file.read", "src/../escape"),
] {
assert!(
a.set_subject_permissions(
&subject,
&update("invalid", None, vec![permission(operation, path, true)])
)
.is_err()
);
}
assert!(a.subject_permissions(&subject).unwrap().is_none());
}
#[test]
fn child_grants_are_immutable_bounded_and_do_not_create_subject_ownership() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", "src", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&child).unwrap();
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child.clone())
);
assert!(
store
.journal_body_is_unpublished(&subject, "runtime", "child")
.unwrap()
);
let mut changed = child.clone();
changed.profile_digest = "changed-profile".into();
assert!(matches!(
store.bind_worker_execution_grant(&changed),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
for (worker, requested) in [
("write", permission("file.write", "src", false)),
("escape", permission("file.read", ".", true)),
("prefix", permission("file.read", "src-other", false)),
] {
let mut candidate = grant(&subject, &source.change_id, worker);
candidate.permissions = vec![requested];
assert!(store.bind_worker_execution_grant(&candidate).is_err());
assert!(
store
.worker_execution_grant("runtime", worker)
.unwrap()
.is_none()
);
}
let mut self_origin = child.clone();
self_origin.worker_id = "body".into();
assert!(store.bind_worker_execution_grant(&self_origin).is_err());
let other = store.create_subject(role()).unwrap();
store
.claim_journal_writer(&other.id, "runtime", "other-body")
.unwrap();
let mut invalid = grant(&subject, &source.change_id, "other-body");
assert!(store.bind_worker_execution_grant(&invalid).is_err());
invalid.worker_id = "forged-child".into();
invalid.origin_worker_id = "other-body".into();
assert!(store.bind_worker_execution_grant(&invalid).is_err());
}
#[test]
fn redelegation_requires_current_source_and_explicit_retained_parent_ceiling() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = vec![permission("file.read", "src", true)];
store.bind_worker_execution_grant(&child).unwrap();
let mut descendant = grant(&subject, &source.change_id, "grandchild");
descendant.origin_worker_id = "child".into();
store.bind_worker_execution_grant(&descendant).unwrap();
let mut excessive = descendant.clone();
excessive.worker_id = "excessive".into();
excessive.redelegation = vec![permission("file.read", ".", true)];
assert!(store.bind_worker_execution_grant(&excessive).is_err());
let mut nondelegator = descendant.clone();
nondelegator.origin_worker_id = "grandchild".into();
nondelegator.worker_id = "great-grandchild".into();
assert!(store.bind_worker_execution_grant(&nondelegator).is_err());
let mut stale = descendant.clone();
stale.worker_id = "after-source-replacement".into();
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
assert_ne!(revoked.change_id, source.change_id);
stale.origin_worker_id = "new-body".into();
assert!(store.bind_worker_execution_grant(&stale).is_err());
// Exact retries/readback recover evidence but cannot revive the source or origin.
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
store.retire_subject(&subject).unwrap();
stale.source_change_id = revoked.change_id;
assert!(store.bind_worker_execution_grant(&stale).is_err());
}
#[test]
fn journal_state_neither_authenticates_roots_nor_bypasses_retained_child_ceilings() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
// The Host has authenticated root ownership outside this store. No Journal record is
// required; changing Journal writer state cannot supply or invalidate that authentication.
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "without-journal"))
.unwrap();
let journal = store
.claim_journal_writer(&subject, "runtime", "body")
.unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-active-journal"))
.unwrap();
store.release_journal_writer(&journal).unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-released-journal"))
.unwrap();
// An active Journal binding must never upgrade a child's empty redelegation ceiling.
store
.claim_journal_writer(&subject, "runtime", "without-journal")
.unwrap();
let mut descendant = grant(&subject, &source.change_id, "descendant");
descendant.origin_worker_id = "without-journal".into();
assert!(matches!(
store.bind_worker_execution_grant(&descendant),
Err(SubjektivError::AuthorizationDenied(_))
));
assert!(
store
.worker_execution_grant("runtime", "descendant")
.unwrap()
.is_none()
);
}
#[test]
fn retained_origins_reject_cycles_foreign_subjects_and_stale_parent_sources() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = source.delegation.clone();
store.bind_worker_execution_grant(&child).unwrap();
// Closing a retained chain back through its previously unbound root must not create a cycle.
let mut cycle = grant(&subject, &source.change_id, "body");
cycle.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&cycle),
Err(SubjektivError::AuthorizationDenied(_))
));
let other = store.create_subject(role()).unwrap().id;
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let mut foreign = grant(&other, &other_source.change_id, "foreign-child");
foreign.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&foreign),
Err(SubjektivError::AuthorizationDenied(_))
));
let current = store
.set_subject_permissions(
&subject,
&update("replace", Some(&source.change_id), source.delegation),
)
.unwrap();
let mut stale_parent = grant(&subject, &current.change_id, "stale-parent-child");
stale_parent.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&stale_parent),
Err(SubjektivError::AuthorizationDenied(_))
));
}
#[test]
fn failed_grant_insert_leaves_no_binding_and_can_retry_same_grant() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_grant BEFORE INSERT ON worker_execution_grants BEGIN SELECT RAISE(ABORT,'injected grant failure'); END;")?;
Ok(())
}).unwrap();
assert!(store.bind_worker_execution_grant(&child).is_err());
assert!(
store
.worker_execution_grant("runtime", "child")
.unwrap()
.is_none()
);
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("DROP TRIGGER fail_grant")?;
Ok(())
})
.unwrap();
store.bind_worker_execution_grant(&child).unwrap();
for sql in [
"UPDATE worker_execution_grants SET source_change_id='forged'",
"DELETE FROM worker_execution_grants",
"UPDATE subject_permission_changes SET record_json='{}'",
"DELETE FROM subject_permission_receipts",
] {
assert!(
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch(sql)?;
Ok(())
})
.is_err(),
"{sql}"
);
}
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
}
#[test]
fn current_schema_migration_preserves_subject_and_matches_new_database() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("migrated");
let original;
{
let (_old_manager, _old_scope, old) = open_version(&root, 11);
original = old
.create_subject_with_behavior(role(), "Keep behavior separate".into())
.unwrap();
}
let (_migrated_manager, _migrated_scope, migrated) = open_store(&root, "workspace-a");
assert_eq!(
migrated.subject(&original.id).unwrap(),
Some(original.clone())
);
assert!(
migrated
.subject_permissions(&original.id)
.unwrap()
.is_none()
);
let (_fresh_manager, _fresh_scope, fresh) =
open_store(&temp.path().join("fresh"), "workspace-a");
assert_eq!(schema(&migrated), schema(&fresh));
let source = migrated
.set_subject_permissions(&original.id, &update("new-source", None, Vec::new()))
.unwrap();
assert!(!source.change_id.is_empty());
assert_eq!(migrated.subject(&original.id).unwrap(), Some(original));
}
#[test]
fn unset_or_execution_only_source_cannot_issue_child_grants_and_saved_grants_survive_reopen() {
let temp = tempfile::tempdir().unwrap();
let saved;
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
assert!(
store
.bind_worker_execution_grant(&grant(&subject, "unset", "child"))
.is_err()
);
let mut execution_only = update("execution", None, Vec::new());
execution_only.execution = vec![permission("file.read", ".", true)];
let source = store
.set_subject_permissions(&subject, &execution_only)
.unwrap();
assert!(
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "child"))
.is_err()
);
let source = store
.set_subject_permissions(
&subject,
&update(
"delegation",
Some(&source.change_id),
execution_only.execution,
),
)
.unwrap();
saved = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&saved).unwrap();
}
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(saved.clone())
);
store.bind_worker_execution_grant(&saved).unwrap();
let mut conflicting = saved;
conflicting.profile = "builtin:default".into();
assert!(matches!(
store.bind_worker_execution_grant(&conflicting),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
}
#[test]
fn subject_grant_listing_is_scoped_retained_ordered_and_fails_on_overflow() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let other = store.create_subject(role()).unwrap().id;
assert!(
store
.list_subject_worker_execution_grants(&subject)
.unwrap()
.is_empty()
);
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let other_grant = grant(&other, &other_source.change_id, "other-child");
store.bind_worker_execution_grant(&other_grant).unwrap();
for index in (0..MAX_SUBJECT_WORKER_EXECUTION_GRANTS).rev() {
store
.bind_worker_execution_grant(&grant(
&subject,
&source.change_id,
&format!("child-{index:04}"),
))
.unwrap();
}
store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
store.retire_subject(&subject).unwrap();
let listed = store
.list_subject_worker_execution_grants(&subject)
.unwrap();
assert_eq!(listed.len(), MAX_SUBJECT_WORKER_EXECUTION_GRANTS);
assert!(
listed
.iter()
.all(|grant| grant.subject_id == subject && grant.source_change_id == source.change_id)
);
assert!(
listed
.windows(2)
.all(|pair| (&pair[0].runtime_id, &pair[0].worker_id)
< (&pair[1].runtime_id, &pair[1].worker_id))
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant.clone()]
);
// Seed one more retained row directly: the retired Subject cannot issue new grants.
// Listing must report overflow rather than returning a seemingly complete cancellation set.
let overflow = grant(&subject, &source.change_id, "overflow");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![overflow.runtime_id, overflow.worker_id, overflow.subject_id, overflow.source_change_id, serde_json::to_string(&overflow)?])?;
Ok(())
}).unwrap();
assert!(
matches!(store.list_subject_worker_execution_grants(&subject), Err(SubjektivError::WorkerExecutionGrantListOverflow { subject_id, limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS }) if subject_id == subject)
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant]
);
let (_foreign_manager, _foreign_scope, foreign) =
open_store(&temp.path().join("foreign"), "workspace-b");
assert!(matches!(
foreign.list_subject_worker_execution_grants(&subject),
Err(SubjektivError::SubjectNotFound(_))
));
}
#[test]
fn failed_schema_migration_rolls_back_new_tables_and_preserves_old_data() {
let temp = tempfile::tempdir().unwrap();
let original;
{
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
original = old.create_subject(role()).unwrap();
old.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TABLE subject_permissions (collision TEXT)")?;
Ok(())
})
.unwrap();
}
{
let manager = FeatureStorage::new(temp.path());
let scope = manager.scope("workspace-a").unwrap();
let registration = SubjektivStore::register(&scope).unwrap();
assert!(SubjektivStore::open(&scope, &registration).is_err());
}
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
assert_eq!(old.database.schema_version().unwrap(), 11);
assert_eq!(old.subject(&original.id).unwrap(), Some(original));
old.database.try_with_connection::<_, SubjektivError>(|connection| {
let created: bool = connection.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE name='subject_permission_changes')", [], |row| row.get(0))?;
assert!(!created);
Ok(())
}).unwrap();
}
+319 -38
View File
@@ -132,9 +132,7 @@ System events and case judgments cannot be edited. The Workspace backend supplie
const LIST_DESCRIPTION: &str = "Query Tickets from the configured typed Ticket backend as a bounded \
overview. Workspace-backed Workers replace this definition with the richer authoritative \
text/event/evidence/relation/Objective/time/attention query.";
const SHOW_DESCRIPTION: &str = "Show one Ticket by id or exact query through the configured typed \
Ticket backend. Output includes the initial comment and recent events with stable event_ref, author/time and edit metadata, bounded Markdown body, resolution, and artifact \
metadata; Workspace-backed Workers replace this definition with the richer authoritative evidence projection.";
const SHOW_DESCRIPTION: &str = "Read one Ticket by id or exact query. Use view=summary for status/CAS without prose or history; detail (default) for canonical initial_post and bounded recent discussion; history with event_cursor for older thread pages; post with an observed event_ref for the exact full post. Never use truncated bodies for edit CAS or guess event identities. Workspace-backed Workers retain the same views with authoritative delegation and MR evidence.";
const COMMENT_DESCRIPTION: &str = "Append a typed Ticket comment event. `body` is Markdown.";
const PLAN_DESCRIPTION: &str = "Append a typed Ticket plan event. `body` is Markdown.";
const DECISION_DESCRIPTION: &str = "Append a typed Ticket decision event. `body` is Markdown.";
@@ -549,8 +547,50 @@ impl QueryTicketParams {
}
}
/// Model-facing read modes shared by generic Ticket and Workspace Tool/WIP reads.
#[derive(
Debug, Default, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, schemars::JsonSchema,
)]
#[serde(rename_all = "snake_case")]
pub enum ShowTicketView {
Summary,
#[default]
Detail,
History,
Post,
}
impl ShowTicketView {
pub fn validate(
self,
event_cursor: Option<&str>,
event_ref: Option<&str>,
) -> Result<(), String> {
if (self == Self::Post) != event_ref.is_some()
|| event_ref.is_some_and(|value| value.trim().is_empty())
{
return Err(
"event_ref is required only for view=post; use an observed identity".into(),
);
}
if event_cursor.is_some() && !matches!(self, Self::Detail | Self::History) {
return Err("event_cursor is only valid for detail/history".into());
}
Ok(())
}
}
#[derive(Debug, Deserialize, schemars::JsonSchema)]
struct ShowTicketParams {
/// summary omits prose/history; detail is the default; history returns only a thread page; post returns an exact full post up to 1 MiB (larger posts fail, never truncate).
#[serde(default)]
view: ShowTicketView,
/// Opaque continuation from event_page. Valid for detail/history only.
#[serde(default)]
event_cursor: Option<String>,
/// Exact observed post identity. Required only for view=post.
#[serde(default)]
event_ref: Option<String>,
/// Ticket reference. Prefer `T-*`; canonical internal ids remain accepted for compatibility. Exactly one of `id` or `query` must be provided.
#[serde(default)]
id: Option<String>,
@@ -1096,6 +1136,10 @@ impl Tool for ShowTicketTool {
_ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
let params: ShowTicketParams = parse_input("ShowTicket", input_json)?;
params
.view
.validate(params.event_cursor.as_deref(), params.event_ref.as_deref())
.map_err(ToolError::InvalidArgument)?;
let query = id_or_query(params.id, params.query)?;
let event_limit = bounded(params.event_limit, DEFAULT_EVENT_LIMIT, MAX_EVENT_LIMIT);
let artifact_limit = bounded(
@@ -1117,10 +1161,45 @@ impl Tool for ShowTicketTool {
ticket.meta.id,
ticket.meta.workflow_state.as_str()
);
Ok(json_output(
summary,
ticket_json(&ticket, event_limit, artifact_limit, body_max_bytes),
))
let value = match params.view {
ShowTicketView::Summary => ticket_status_json(&ticket),
ShowTicketView::Post => {
let event = ticket
.events
.iter()
.find(|event| Some(event.event_ref.as_str()) == params.event_ref.as_deref())
.ok_or_else(|| {
ToolError::InvalidArgument(
"Observed post identity not found in this Ticket".into(),
)
})?;
// Match the Workspace full-post route: never silently truncate CAS input.
if event.body.as_str().len() > 1024 * 1024 {
return Err(ToolError::ExecutionFailed(
"Exact post exceeds the 1 MiB full-read limit; do not edit from a truncated preview".into(),
));
}
json!({"ticket": ticket.meta.id, "post": ticket_event_json(event, usize::MAX)})
}
ShowTicketView::Detail | ShowTicketView::History => {
let (events, page) = ticket_event_page(
&ticket,
params.event_cursor.as_deref(),
event_limit,
body_max_bytes,
)?;
if params.view == ShowTicketView::History {
json!({"ticket": ticket.meta.id, "thread": events, "event_page": page})
} else {
let mut detail =
ticket_json(&ticket, event_limit, artifact_limit, body_max_bytes);
detail["events"] = events;
detail["event_page"] = page;
detail
}
}
};
Ok(json_output(summary, value))
}
}
@@ -1694,8 +1773,70 @@ fn ticket_relations_json(ticket: &Ticket) -> Value {
})
}
fn ticket_event_json(event: &crate::TicketEvent, body_max_bytes: usize) -> Value {
fn ticket_status_json(ticket: &Ticket) -> Value {
json!({
"ticket": ticket.meta.resource_key.as_deref().unwrap_or(&ticket.meta.id),
"title": ticket.meta.title,
"state": ticket.meta.workflow_state,
"content_digest": crate::ticket_content_digest(ticket),
"closure_operation_key": ticket.closure_operation_key,
"current_closure": ticket.current_closure.as_ref().map(|j| json!({
"operation_key": j.operation_key, "reason": j.reason, "author": j.author, "at": j.at
})),
"targets": ticket.meta.targets,
"tags": ticket.tags,
"assignee": ticket.meta.assignee,
"readiness": ticket.meta.readiness,
"updated_at": ticket.meta.updated_at,
"latest_event_ref": ticket.events.last().map(|event| &event.event_ref),
})
}
// Cursor is the persisted identity of the oldest returned event, never an offset.
// Appending newer events between pages therefore cannot shift the continuation.
fn ticket_event_page(
ticket: &Ticket,
cursor: Option<&str>,
limit: usize,
body_max_bytes: usize,
) -> Result<(Value, Value), ToolError> {
let events: Vec<_> = ticket
.events
.iter()
.filter(|event| !event.is_initial)
.collect();
let end = match cursor {
None => events.len(),
Some(cursor) => events
.iter()
.position(|event| event.event_ref == cursor)
.ok_or_else(|| {
ToolError::InvalidArgument(
"Unknown Ticket event_cursor; reuse a returned cursor".into(),
)
})?,
};
let start = end.saturating_sub(limit.max(1));
let items: Vec<_> = events[start..end]
.iter()
.map(|event| ticket_event_json(event, body_max_bytes))
.collect();
let next_cursor = if start > 0 {
Some(&events[start].event_ref)
} else {
None
};
Ok((
json!({"count": events.len(), "returned": items.len(), "truncated": start > 0, "items": items}),
json!({"next_cursor": next_cursor, "has_more": start > 0}),
))
}
fn ticket_event_json(event: &crate::TicketEvent, body_max_bytes: usize) -> Value {
use sha2::{Digest, Sha256};
json!({
"body_digest": Sha256::digest(event.body.as_str().as_bytes()).iter().map(|byte| format!("{byte:02x}")).collect::<String>(),
"editable": event.kind == TicketEventKind::Comment,
"event_ref": event.event_ref,
"is_initial": event.is_initial,
"kind": event.kind.as_str(),
@@ -1722,13 +1863,8 @@ fn ticket_json(
artifact_limit: usize,
body_max_bytes: usize,
) -> Value {
let event_count = ticket.events.len();
let events: Vec<_> = ticket
.events
.iter()
.skip(event_count.saturating_sub(event_limit))
.map(|event| ticket_event_json(event, body_max_bytes))
.collect();
let (events, event_page) = ticket_event_page(ticket, None, event_limit, body_max_bytes)
.expect("first thread page has no cursor to reject");
let artifact_count = ticket.artifacts.len();
let artifacts: Vec<_> = ticket
.artifacts
@@ -1753,17 +1889,12 @@ fn ticket_json(
},
"content_digest": crate::ticket_content_digest(ticket),
"closure_operation_key": ticket.closure_operation_key,
"current_closure": ticket.current_closure.as_ref().map(|j| json!({"operation_key": j.operation_key, "reason": j.reason, "explanation": truncate_text(&j.explanation, body_max_bytes), "author": j.author, "references": j.references, "at": j.at})),
"case_history": { "count": ticket.case_history.len(), "items": ticket.case_history.iter().rev().take(event_limit).map(|j| json!({"operation_key": j.operation_key, "previous_closure_operation_key": j.previous_closure_operation_key, "action": j.action, "explanation": truncate_text(&j.explanation, body_max_bytes), "author": j.author, "references": j.references, "at": j.at})).collect::<Vec<_>>() },
"current_closure": ticket.current_closure.as_ref().map(|j| json!({"operation_key": j.operation_key, "reason": j.reason, "author": j.author, "references": j.references, "at": j.at})),
"case_history": { "count": ticket.case_history.len(), "items": ticket.case_history.iter().rev().take(event_limit).map(|j| json!({"operation_key": j.operation_key, "previous_closure_operation_key": j.previous_closure_operation_key, "action": j.action, "author": j.author, "references": j.references, "at": j.at})).collect::<Vec<_>>() },
"legacy_workflow_snapshot": ticket.legacy_workflow_snapshot,
"initial_post": ticket.events.iter().find(|event| event.is_initial).map(|event| ticket_event_json(event, body_max_bytes)),
"body": truncate_text(ticket.document.body.as_str(), body_max_bytes),
"events": {
"count": event_count,
"returned": events.len(),
"truncated": event_count > events.len(),
"items": events,
},
"events": events,
"event_page": event_page,
"artifacts": {
"count": artifact_count,
"returned": artifacts.len(),
@@ -2063,13 +2194,17 @@ mod tests {
.await
.unwrap();
let value: Value = serde_json::from_str(&output.content.unwrap()).unwrap();
assert_eq!(value["events"]["count"], 2);
let projected = value["events"]["items"]
.as_array()
.unwrap()
.iter()
.find(|e| e["event_ref"] == post.event_ref)
.unwrap();
assert_eq!(value["events"]["count"], 1);
let projected = if post.is_initial {
&value["initial_post"]
} else {
value["events"]["items"]
.as_array()
.unwrap()
.iter()
.find(|e| e["event_ref"] == post.event_ref)
.unwrap()
};
assert_eq!(projected["author"], json!(post.author));
assert_eq!(projected["at"], json!(post.at));
assert_eq!(projected["is_initial"], post.is_initial);
@@ -2104,7 +2239,8 @@ mod tests {
value["initial_post"]["event_ref"],
before.events[0].event_ref
);
assert_eq!(value["initial_post"]["body"], value["body"]);
assert_eq!(value["initial_post"]["body"], "Edited Initial body");
assert!(value.get("body").is_none());
assert_eq!(value["meta"]["created_by"], "creator");
assert_eq!(value["initial_post"]["author"], "creator");
let initial_keys = value["initial_post"]
@@ -2518,7 +2654,10 @@ mod tests {
.unwrap();
let value: Value = serde_json::from_str(&output.content.unwrap()).unwrap();
assert_eq!(
value["body"].as_str().unwrap().trim_start_matches('\n'),
value["initial_post"]["body"]
.as_str()
.unwrap()
.trim_start_matches('\n'),
"ONE\ntwo\nONE\n"
);
@@ -2980,13 +3119,13 @@ mod tests {
assert_eq!(value["closure_operation_key"], "tool-correct");
assert_eq!(value["case_history"]["count"], 2);
assert_eq!(value["case_history"]["items"].as_array().unwrap().len(), 1);
assert!(value["current_closure"].get("explanation").is_none());
assert!(
value["current_closure"]["explanation"]
.as_str()
.unwrap()
.len()
< 256
value["case_history"]["items"][0]
.get("explanation")
.is_none()
);
assert_eq!(value["events"]["items"][0]["body_truncated"], true);
assert!(value["content_digest"].as_str().unwrap().len() == 64);
let mut reopen = correction;
reopen["expected_closure_operation_key"] = json!("tool-correct");
@@ -3015,6 +3154,148 @@ mod tests {
);
}
#[tokio::test]
async fn show_views_deduplicate_bound_history_and_read_exact_unicode_posts() {
let temp = TempDir::new().unwrap();
let backend = backend(&temp);
let body = "long-requirement-".repeat(700);
let mut input = NewTicket::new("Read projections");
input.body = MarkdownText::new(body.clone());
let reference = backend.create(input).unwrap();
for index in 0..5 {
backend
.add_event(
reference.id.clone().into(),
NewTicketEvent::new(TicketEventKind::Comment, format!("comment-{index}")),
)
.unwrap();
}
let before = backend.show(reference.id.clone().into()).unwrap();
let detail = ticket_json(&before, 20, 20, DEFAULT_BODY_MAX_BYTES);
assert_eq!(detail.to_string().matches(&body).count(), 1);
assert!(detail.get("body").is_none());
assert!(
detail["events"]["items"]
.as_array()
.unwrap()
.iter()
.all(|event| event["is_initial"] == false)
);
let mut larger = before.clone();
larger.document.body = MarkdownText::new("prose".repeat(100_000));
larger.events[0].body = larger.document.body.clone();
larger
.events
.extend(vec![before.events.last().unwrap().clone(); 1000]);
let small_summary = ticket_status_json(&before).to_string();
let large_summary = ticket_status_json(&larger).to_string();
assert_eq!(small_summary.len(), large_summary.len());
eprintln!(
"Ticket generic bytes: detail={}, summary={} (unchanged with 500KB body / 1000 extra events)",
detail.to_string().len(),
small_summary.len()
);
let show = tool_by_name(backend.clone(), "ShowTicket");
let mut cursor = Value::Null;
let mut seen = Vec::new();
loop {
let output = show.execute(&json!({"id": reference.id, "view": "history", "event_limit": 2, "event_cursor": cursor}).to_string(), Default::default()).await.unwrap();
let page: Value = serde_json::from_str(&output.content.unwrap()).unwrap();
assert_eq!(page.as_object().unwrap().len(), 3);
for event in page["thread"]["items"].as_array().unwrap() {
seen.push(event["event_ref"].as_str().unwrap().to_owned());
}
if cursor.is_null() {
backend
.add_event(
reference.id.clone().into(),
NewTicketEvent::new(TicketEventKind::Comment, "newer append"),
)
.unwrap();
}
cursor = page["event_page"]["next_cursor"].clone();
if cursor.is_null() {
break;
}
}
seen.sort();
let mut expected: Vec<_> = before
.events
.iter()
.filter(|event| !event.is_initial)
.map(|event| event.event_ref.clone())
.collect();
expected.sort();
assert_eq!(seen, expected);
assert!(
show.execute(
&json!({"id": reference.id, "view":"history", "event_cursor":"guessed"})
.to_string(),
Default::default()
)
.await
.is_err()
);
let unicode = "全文🦀é\n".repeat(20_000);
let post = &before.events[0];
let edit = tool_by_name(backend.clone(), "TicketEditPost");
let edit_args = json!({"ticket":reference.id, "event_ref":post.event_ref, "expected_body":post.body.as_str(), "body":unicode});
edit.execute(&edit_args.to_string(), Default::default())
.await
.unwrap();
let full = show.execute(&json!({"id":reference.id, "view":"post", "event_ref":post.event_ref, "body_max_bytes":1}).to_string(), Default::default()).await.unwrap();
let full: Value = serde_json::from_str(&full.content.unwrap()).unwrap();
assert_eq!(full["post"]["body"], unicode);
assert_eq!(full["post"]["body_truncated"], false);
assert_eq!(full["post"]["editable"], true);
assert!(
edit.execute(&edit_args.to_string(), Default::default())
.await
.is_err()
);
for args in [
json!({"view":"post"}),
json!({"view":"summary", "event_ref":post.event_ref}),
json!({"view":"summary", "event_cursor":"x"}),
] {
let mut args = args;
args["id"] = json!(reference.id);
assert!(
show.execute(&args.to_string(), Default::default())
.await
.is_err()
);
}
}
#[tokio::test]
async fn exact_post_read_rejects_oversize_instead_of_truncating_edit_input() {
let temp = TempDir::new().unwrap();
let backend = backend(&temp);
let mut input = NewTicket::new("Full-post limit");
input.body = MarkdownText::new("x".repeat(1024 * 1024 + 1));
let reference = backend.create(input).unwrap();
let observed = backend.show(reference.id.clone().into()).unwrap();
let event_ref = &observed
.events
.iter()
.find(|event| event.is_initial)
.unwrap()
.event_ref;
let show = tool_by_name(backend.clone(), "ShowTicket");
let error = show
.execute(
&json!({"id":reference.id, "view":"post", "event_ref":event_ref}).to_string(),
Default::default(),
)
.await
.unwrap_err();
assert!(error.to_string().contains("1 MiB"));
assert_eq!(backend.show(reference.id.into()).unwrap(), observed);
}
#[tokio::test]
async fn case_tool_rejects_legacy_states_and_unrecognized_input_fields() {
let temp = TempDir::new().unwrap();
+2 -1
View File
@@ -23,7 +23,8 @@ thiserror = { workspace = true }
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tracing = { workspace = true }
workdir = { workspace = true }
uuid = { workspace = true, features = ["v7"] }
[dev-dependencies]
filetime = "0.2.27"
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "test-util"] }
+246 -232
View File
@@ -1,24 +1,26 @@
//! Bash Tool using WorkdirSession command capability.
//!
//! The selected session owns cwd, process launch, bounded output, timeout, and cancellation.
//! Workdir owns launch and byte transport; Bash owns bounded preview and retained evidence.
//! Bash is not a sandbox; the Runtime process and OS user define the execution boundary.
//! Per-execution handles and nonces prevent cancellation of another command. Dropping a future
//! does not confirm termination; active registrations remain until cleanup requests cancellation
//! and observes terminal output.
//! does not confirm termination. Registrations remain during bounded cancellation/drain cleanup;
//! an unconfirmed provider outcome leaves partial evidence, never a fabricated terminal fence.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use crate::bash_output::{BashOutputMetadata, BashOutputView, BashOutputWriter};
use agen::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use async_trait::async_trait;
use schemars::JsonSchema;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
use workdir::{
CommandHandle, CommandOutputRequest, CommandRequest, WorkdirPath, WorkdirSessionHandle,
WorkdirSessionRouter,
};
mod cleanup;
const DEFAULT_TIMEOUT_SECS: u64 = 120;
const MAX_TIMEOUT_SECS: u64 = 600;
const INLINE_BYTE_BUDGET: usize = 12 * 1024;
@@ -28,16 +30,18 @@ struct BashParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
target_workdir: Option<String>,
/// Workdir-relative directory in which to execute the command.
/// Directory in which to execute the command.
// Keep provider path-coordinate documentation out of the model-facing schema.
#[schemars(with = "String")]
cwd: WorkdirPath,
command: String,
#[serde(default)]
timeout: Option<u64>,
}
pub(crate) struct BashTool {
pub struct BashTool {
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
state: Arc<Mutex<BashExecutionState>>,
}
@@ -63,6 +67,8 @@ struct CommandGuard {
execution_id: String,
execution_nonce: u64,
handle: Option<CommandHandle>,
cursor: u64,
retained: Option<BashOutputWriter>,
}
impl Drop for CommandGuard {
@@ -70,46 +76,69 @@ impl Drop for CommandGuard {
let Some(handle) = self.handle.take() else {
return;
};
let workdir = self.session.clone();
let state = Arc::clone(&self.state);
let execution_id = self.execution_id.clone();
let execution_nonce = self.execution_nonce;
// A dropped provider future is not terminal confirmation. Keep the live
// execution registered until cleanup has both requested cancellation and
// observed terminal command output, so cancellation/session teardown
// cannot race with an apparently empty registry.
tokio::spawn(async move {
let _ = workdir.cancel_command(handle.clone()).await;
let _ = workdir
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
.await;
let mut state = state.lock().unwrap();
if state
.active
.get(&execution_id)
.is_some_and(|active| active.execution_nonce == execution_nonce)
{
state.active.remove(&execution_id);
state.cancellation_requested.remove(&execution_id);
}
});
let cleanup = cleanup::Cleanup {
session: self.session.clone(),
state: Arc::clone(&self.state),
execution_id: self.execution_id.clone(),
execution_nonce: self.execution_nonce,
handle,
cursor: self.cursor,
retained: self
.retained
.take()
.expect("started command owns output writer"),
};
tokio::spawn(cleanup.run());
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
// Output pages are replayable only with the exact same command/cursor/limit.
// Retry read transport failures, never command start, and cap both attempts and
// backoff. The caller's deadline bounds a disconnected or unresponsive provider.
async fn command_page(
session: &workdir::WorkdirSessionHandle,
request: CommandOutputRequest,
) -> Result<workdir::CommandOutput, workdir::WorkdirError> {
for attempt in 0..3 {
match session.command_output(request.clone()).await {
Err(error)
if attempt < 2
&& matches!(
error.classification_source(),
workdir::WorkdirError::Transport(_)
| workdir::WorkdirError::Unavailable(_)
| workdir::WorkdirError::OutcomeUnknown(_)
) =>
{
tokio::time::sleep(std::time::Duration::from_millis(100 << attempt)).await;
}
result => return result,
}
}
unreachable!("bounded retry loop returns its last result")
}
impl BashTool {
pub fn input_schema() -> serde_json::Value {
serde_json::to_value(schemars::schema_for!(BashParams)).expect("Bash schema serialization")
}
pub fn new(router: Arc<WorkdirSessionRouter>, output: BashOutputView) -> Self {
Self {
router,
output,
state: Arc::new(Mutex::new(BashExecutionState::default())),
}
}
pub async fn execute_result(
&self,
input_json: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
) -> Result<BashResult, ToolError> {
let params: BashParams = serde_json::from_str(input_json)
.map_err(|error| ToolError::InvalidArgument(format!("invalid Bash input: {error}")))?;
let retained = self.output.begin().map_err(ToolError::ExecutionFailed)?;
let selected = crate::routing::resolve_session(
&self.router,
params.target_workdir.as_deref(),
@@ -134,14 +163,15 @@ impl Tool for BashTool {
execution_id: execution_id.clone(),
execution_nonce,
handle: None,
cursor: 0,
retained: Some(retained),
};
let handle = session
.start_command(CommandRequest {
command: params.command,
timeout_secs,
output_limit: INLINE_BYTE_BUDGET,
transport_byte_budget: self.output.byte_budget(),
cwd: params.cwd,
spill_dir: Some(self.output_dir.clone()),
tool_call_id: Some(call_id.clone()),
})
.await
@@ -167,15 +197,101 @@ impl Tool for BashTool {
.await
.map_err(crate::ToolsError::from)?;
}
let output = session
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
let mut cursor = 0;
let mut stdout_preview = Vec::new();
let mut stderr_preview = Vec::new();
let mut seen = [0_u64; 2];
let mut empty_polls = 0;
let deadline =
tokio::time::Instant::now() + std::time::Duration::from_secs(timeout_secs + 30);
let terminal = loop {
let page = tokio::time::timeout_at(
deadline,
command_page(
&session,
CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
),
)
.await
.map_err(|_| {
ToolError::ExecutionFailed(
"command byte transport deadline exceeded; terminal outcome unknown".into(),
)
})?
.map_err(crate::ToolsError::from)?;
let mut next = cursor;
for chunk in &page.chunks {
if chunk.sequence != next || chunk.bytes.is_empty() {
return Err(ToolError::ExecutionFailed(
"command byte transport sequence gap".into(),
));
}
next = next
.checked_add(chunk.bytes.len() as u64)
.filter(|n| *n <= self.output.byte_budget())
.ok_or_else(|| {
ToolError::ExecutionFailed(
"command byte transport exceeds delivery budget".into(),
)
})?;
let index = if chunk.stream == workdir::CommandStream::Stdout {
0
} else {
1
};
if chunk.offset != seen[index] {
return Err(ToolError::ExecutionFailed(
"command byte transport offset gap".into(),
));
}
seen[index] += chunk.bytes.len() as u64;
guard
.retained
.as_mut()
.unwrap()
.append_chunk(chunk)
.map_err(ToolError::ExecutionFailed)?;
let available =
INLINE_BYTE_BUDGET.saturating_sub(stdout_preview.len() + stderr_preview.len());
let preview = if index == 0 {
&mut stdout_preview
} else {
&mut stderr_preview
};
preview.extend_from_slice(&chunk.bytes[..available.min(chunk.bytes.len())]);
}
if page.next_cursor != next {
return Err(ToolError::ExecutionFailed(
"command byte transport cursor does not fence the page".into(),
));
}
if page.chunks.is_empty() && page.terminal.is_none() {
empty_polls += 1;
if empty_polls > 32 {
return Err(ToolError::ExecutionFailed(
"command byte transport made no progress after bounded polls".into(),
));
}
} else {
empty_polls = 0;
}
cursor = page.next_cursor;
guard.cursor = cursor;
if let Some(terminal) = page.terminal {
break terminal;
}
};
let metadata = guard
.retained
.as_mut()
.unwrap()
.finish_transport(&terminal)
.map_err(ToolError::ExecutionFailed)?;
let cancellation_requested = {
let mut state = self.state.lock().unwrap();
let owns_registration = state
@@ -193,41 +309,84 @@ impl Tool for BashTool {
};
guard.handle = None;
let timed_out = output.timed_out;
let summary = if cancellation_requested {
format!("$ {cmd_summary} (cancelled)")
} else if output.timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match output.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
let content = if output.content.is_empty() {
None
} else if output.truncated {
let notice = match output.output_path {
Some(path) => format!(
"[showing bounded WorkdirSession command output; full output saved to {}]",
path.display()
),
None => "[showing bounded WorkdirSession command output; additional output was truncated]"
.to_owned(),
let timed_out = terminal.status == workdir::CommandStatus::TimedOut;
let mut summary =
if cancellation_requested || terminal.status == workdir::CommandStatus::Cancelled {
format!("$ {cmd_summary} (cancelled)")
} else if timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match terminal.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
Some(format!("{notice}\n{}", output.content))
} else {
Some(output.content)
};
let output = ToolOutput {
if matches!(
terminal.output_collection,
workdir::CommandOutputCollection::Incomplete { .. }
) {
summary.push_str(" (output incomplete)");
}
Ok(BashResult {
summary,
content,
attachments: Vec::new(),
};
if cancellation_requested {
stdout_preview: String::from_utf8_lossy(&stdout_preview).into_owned(),
stderr_preview: String::from_utf8_lossy(&stderr_preview).into_owned(),
preview_omitted: terminal.stdout_bytes.saturating_add(terminal.stderr_bytes)
> (stdout_preview.len() + stderr_preview.len()) as u64,
metadata,
cancelled: cancellation_requested
|| terminal.status == workdir::CommandStatus::Cancelled,
})
}
}
#[derive(Debug, Serialize)]
pub struct BashResult {
pub summary: String,
pub stdout_preview: String,
pub stderr_preview: String,
pub preview_omitted: bool,
pub metadata: BashOutputMetadata,
pub cancelled: bool,
}
impl BashResult {
pub fn stdout_entry(&self) -> String {
format!("/bash-output/{}/stdout", self.metadata.id)
}
pub fn stderr_entry(&self) -> String {
format!("/bash-output/{}/stderr", self.metadata.id)
}
pub fn order_entry(&self) -> String {
format!("/bash-output/{}/order", self.metadata.id)
}
pub fn tool_output(&self) -> ToolOutput {
let content = serde_json::json!({
"stdout_preview": self.stdout_preview, "stderr_preview": self.stderr_preview,
"preview_omitted": self.preview_omitted, "output": self.metadata,
"stdout_entry": self.stdout_entry(), "stderr_entry": self.stderr_entry(), "order_entry": self.order_entry(),
});
ToolOutput {
summary: self.summary.clone(),
content: Some(content.to_string()),
attachments: vec![],
}
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
&self,
input: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
let result = self.execute_result(input, ctx).await?;
let output = result.tool_output();
if result.cancelled {
Err(ToolError::Cancelled(output))
} else if timed_out {
} else if result.metadata.terminal.status == Some(workdir::CommandStatus::TimedOut) {
Err(ToolError::Interrupted(output))
} else {
Ok(output)
@@ -288,164 +447,19 @@ fn truncate_for_summary(command: &str) -> String {
summary
}
pub fn bash_tool(session: WorkdirSessionHandle, output_dir: PathBuf) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output_dir)
pub fn bash_tool(session: WorkdirSessionHandle, output: BashOutputView) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output)
}
pub(crate) fn routed_bash_tool(
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
) -> ToolDefinition {
Arc::new(move || {
let schema = schemars::schema_for!(BashParams);
let meta = ToolMeta::new("Bash")
.description("Execute a shell command in one selected Workdir. Process start, bounded inline output, full-output spill, timeout and cancellation are owned by that WorkdirSession provider. This is not a sandbox.")
.input_schema(serde_json::to_value(schema).expect("Bash schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(BashTool {
router: router.clone(),
output_dir: output_dir.clone(),
state: Arc::new(Mutex::new(BashExecutionState::default())),
});
.description("Execute a shell command in one selected Workdir. Workdir owns launch, lossless bounded byte transport, timeout and cancel; Bash owns preview, quota-limited ephemeral output and read-only virtual references. This is not a sandbox.")
.input_schema(BashTool::input_schema());
let tool: Arc<dyn Tool> = Arc::new(BashTool::new(router.clone(), output.clone()));
(meta, tool)
})
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use tempfile::TempDir;
use workdir::{LocalWorkdirSession, WorkdirSessionHandle};
use super::bash_tool;
use crate::{grep::grep_tool, read::read_tool, tracker::Tracker};
fn session_with_output_scope(root: &TempDir, output: &TempDir) -> WorkdirSessionHandle {
let scope = Scope::from_config(&ScopeConfig {
allow: vec![
ScopeRule {
target: root.path().to_path_buf(),
permission: Permission::Write,
recursive: true,
symlink_policy: Default::default(),
},
ScopeRule {
target: output.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
},
],
deny: Vec::new(),
})
.unwrap();
Arc::new(LocalWorkdirSession::new(scope, root.path().to_path_buf()))
}
#[tokio::test]
async fn long_output_is_spilled_and_available_to_read_and_grep() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session.clone(), output.path().to_path_buf())();
let command = "i=0; while [ $i -lt 2000 ]; do printf 'line-%04d\\n' \"$i\"; i=$((i+1)); done; printf 'FINAL-NEEDLE\\n'";
let result = bash
.execute(
&serde_json::json!({ "command": command, "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
let rendered = result.content.expect("bounded Bash output");
let artifact = std::fs::read_dir(output.path())
.unwrap()
.next()
.expect("artifact entry")
.unwrap()
.path();
assert!(rendered.contains("full output saved to"));
assert!(rendered.contains(&artifact.display().to_string()));
let retained = std::fs::read_to_string(&artifact).unwrap();
assert!(retained.starts_with("line-0000\n"));
assert!(retained.ends_with("FINAL-NEEDLE\n"));
assert_eq!(retained.lines().count(), 2001);
let (_, read) = read_tool(session.clone(), Tracker::new())();
let read_result = read
.execute(
&serde_json::json!({
"file_path": artifact,
"offset": 2000,
"limit": 1,
})
.to_string(),
Default::default(),
)
.await
.unwrap();
assert!(
read_result
.content
.expect("Read content")
.contains("FINAL-NEEDLE")
);
let (_, grep) = grep_tool(session)();
let grep_result = grep
.execute(
&serde_json::json!({
"pattern": "FINAL-NEEDLE",
"path": artifact,
"output_mode": "content",
})
.to_string(),
Default::default(),
)
.await
.unwrap();
let grep_content = grep_result.content.expect("Grep content");
assert!(
grep_content.contains("FINAL-NEEDLE"),
"unexpected Grep content: {grep_content:?}"
);
}
#[tokio::test]
async fn missing_cwd_is_rejected_before_command_start() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let error = bash
.execute(
&serde_json::json!({ "command": "printf hidden" }).to_string(),
Default::default(),
)
.await
.unwrap_err();
assert!(error.to_string().contains("missing field `cwd`"));
}
#[tokio::test]
async fn short_output_does_not_leave_a_spill_artifact() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let result = bash
.execute(
&serde_json::json!({ "command": "printf short", "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
assert_eq!(result.content.as_deref(), Some("short"));
assert_eq!(std::fs::read_dir(output.path()).unwrap().count(), 0);
}
}
+121
View File
@@ -0,0 +1,121 @@
//! Finite Drop cleanup. A cancel acknowledgement is not termination evidence; only a
//! validated terminal fence confirms the process result. No provider text is logged.
use super::*;
use crate::bash_output::{
CleanupCancel, CleanupDiagnostics, CleanupDrain, CleanupFailure, CleanupProviderFailure,
CleanupRegistration,
};
use std::time::Duration;
const CLEANUP_TIMEOUT: Duration = Duration::from_secs(10);
// Reserve time for independent terminal observation even if cancel never responds.
const CANCEL_TIMEOUT: Duration = Duration::from_secs(2);
pub(super) struct Cleanup {
pub session: WorkdirSessionHandle,
pub state: Arc<Mutex<BashExecutionState>>,
pub execution_id: String,
pub execution_nonce: u64,
pub handle: CommandHandle,
pub cursor: u64,
pub retained: BashOutputWriter,
}
impl Cleanup {
pub async fn run(mut self) {
let deadline = tokio::time::Instant::now() + CLEANUP_TIMEOUT;
let cancel = match tokio::time::timeout(
CANCEL_TIMEOUT,
self.session.cancel_command(self.handle.clone()),
)
.await
{
Ok(Ok(())) => CleanupCancel::Acknowledged,
Ok(Err(error)) => CleanupCancel::Failed {
reason: CleanupProviderFailure::from(&error),
},
Err(_) => CleanupCancel::DeadlineExceeded,
};
let drain = match tokio::time::timeout_at(deadline, self.drain()).await {
Ok(Ok(())) => CleanupDrain::TerminalConfirmed,
Ok(Err(reason)) => CleanupDrain::Failed { reason },
Err(_) => CleanupDrain::DeadlineExceeded,
};
let deadline_expired = matches!(drain, CleanupDrain::DeadlineExceeded);
// Finalize evidence before exposing an empty registry. Keep the lock through
// publication so an older cleanup can never remove a newer generation.
let mut state = self.state.lock().unwrap();
let registration = if state
.active
.get(&self.execution_id)
.is_some_and(|active| active.execution_nonce == self.execution_nonce)
{
state.active.remove(&self.execution_id);
state.cancellation_requested.remove(&self.execution_id);
CleanupRegistration::Released
} else {
CleanupRegistration::Superseded
};
let diagnostics = CleanupDiagnostics {
execution_nonce: self.execution_nonce,
cancel,
drain,
deadline_expired,
registration,
};
tracing::info!(
output_id = %self.retained.id(),
cleanup = ?diagnostics,
"Bash Drop cleanup finished; cancellation and terminal observation are independent"
);
self.retained.finish_cleanup(diagnostics);
drop(self.retained);
}
async fn drain(&mut self) -> Result<(), CleanupFailure> {
loop {
let page = command_page(
&self.session,
CommandOutputRequest {
handle: self.handle.clone(),
cursor: self.cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
)
.await
.map_err(|error| CleanupFailure::Provider {
reason: CleanupProviderFailure::from(&error),
})?;
let mut next = self.cursor;
for chunk in &page.chunks {
if chunk.sequence != next {
return Err(CleanupFailure::SequenceGap);
}
next = next
.checked_add(chunk.bytes.len() as u64)
.filter(|next| *next <= crate::bash_output::COMMAND_BYTES as u64)
.ok_or(CleanupFailure::DeliveryBudgetExceeded)?;
self.retained
.append_chunk(chunk)
.map_err(|_| CleanupFailure::RetainedChunkInvalid)?;
}
if page.next_cursor != next {
return Err(CleanupFailure::CursorGap);
}
self.cursor = next;
if let Some(terminal) = page.terminal {
self.retained
.finish_transport(&terminal)
.map_err(|_| CleanupFailure::TerminalMetadataInvalid)?;
return Ok(());
}
// An immediately-ready provider must not starve the cleanup deadline.
tokio::task::yield_now().await;
}
}
}
#[cfg(test)]
mod tests;
+397
View File
@@ -0,0 +1,397 @@
//! Component tests enter through the production CommandGuard Drop, cleanup, page
//! validation and retained store. Only the Workdir provider is replaced; paused time
//! and a cancel barrier make faults/deadlines deterministic without OS timing.
use super::*;
use std::sync::atomic::{AtomicBool, Ordering};
use workdir::*;
const PRIVATE: &str = "SECRET command /private/host/path output password=poison";
#[derive(Clone, Copy, Debug)]
enum Fault {
Cancel,
CancelHang,
Drain,
TerminalTimeout,
Sequence,
Cursor,
RetainedChunk,
TerminalMetadata,
CollectionIncomplete,
Complete,
}
#[derive(Debug)]
struct Provider {
local: LocalWorkdirSession,
fault: Fault,
cancelling: tokio::sync::Notify,
allow_drain: AtomicBool,
}
#[async_trait]
impl WorkdirSession for Provider {
fn workdir(&self) -> &Workdir {
self.local.workdir()
}
fn capabilities(&self) -> WorkdirSessionCapabilities {
WorkdirSessionCapabilities::ALL
}
async fn stat(&self, _: StatRequest) -> Result<StatResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn read(&self, _: ReadRequest) -> Result<ReadResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn write(&self, _: WriteRequest) -> Result<WriteResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn edit(&self, _: EditRequest) -> Result<EditResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn list(&self, _: ListRequest) -> Result<ListResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn glob(&self, _: GlobRequest) -> Result<GlobResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn grep(&self, _: GrepRequest) -> Result<GrepResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn start_command(&self, _: CommandRequest) -> Result<CommandHandle, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn command_status(&self, _: CommandHandle) -> Result<CommandStatus, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn command_output(
&self,
request: CommandOutputRequest,
) -> Result<CommandOutput, WorkdirError> {
assert_eq!(request.cursor, 3);
while !self.allow_drain.load(Ordering::Acquire) {
tokio::task::yield_now().await;
}
match self.fault {
Fault::Drain => {
return Err(WorkdirError::Transport(PRIVATE.into())
.with_failure_reason(WorkdirFailureReason::HttpTransfer));
}
Fault::TerminalTimeout => return std::future::pending().await,
_ => {}
}
let mut chunk = CommandByteChunk {
sequence: 3,
offset: 3,
stream: CommandStream::Stdout,
bytes: b"tail".to_vec(),
};
if matches!(self.fault, Fault::Sequence) {
chunk.sequence = 4;
}
if matches!(self.fault, Fault::RetainedChunk) {
chunk.offset = 42;
}
let mut terminal = terminal();
if matches!(self.fault, Fault::TerminalMetadata) {
terminal.stdout_delivered_bytes = 6;
}
if matches!(self.fault, Fault::CollectionIncomplete) {
terminal.output_collection = CommandOutputCollection::Incomplete {
failure: CommandFailure {
stage: CommandFailureStage::StdoutDrain,
reason: CommandIoErrorKind::Other,
os_code: None,
},
};
}
Ok(CommandOutput {
chunks: vec![chunk],
next_cursor: if matches!(self.fault, Fault::Cursor) {
8
} else {
7
},
terminal: Some(terminal),
})
}
async fn cancel_command(&self, _: CommandHandle) -> Result<(), WorkdirError> {
self.cancelling.notify_one();
match self.fault {
Fault::Cancel => Err(WorkdirError::Unavailable(PRIVATE.into())),
Fault::CancelHang => std::future::pending().await,
_ => Ok(()),
}
}
async fn close(&self) -> Result<(), WorkdirError> {
Ok(())
}
}
fn terminal() -> CommandTerminal {
CommandTerminal {
status: CommandStatus::Completed,
exit_code: Some(0),
process_failure: None,
output_collection: CommandOutputCollection::Complete,
stdout_bytes: 7,
stderr_bytes: 0,
stdout_delivered_bytes: 7,
stderr_delivered_bytes: 0,
truncation_reason: None,
}
}
struct Fixture {
_root: tempfile::TempDir,
provider: Arc<Provider>,
state: Arc<Mutex<BashExecutionState>>,
view: BashOutputView,
id: String,
}
impl Fixture {
fn start(fault: Fault) -> (Self, CommandGuard) {
let base =
std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../target/cleanup-fixtures");
std::fs::create_dir_all(&base).unwrap();
let root = tempfile::tempdir_in(base).unwrap();
let provider = Arc::new(Provider {
local: LocalWorkdirSession::new(
manifest::Scope::writable(root.path()).unwrap(),
root.path().into(),
),
fault,
cancelling: Default::default(),
allow_drain: AtomicBool::new(false),
});
let view = BashOutputView::new();
let mut retained = view.begin().unwrap();
retained.append(CommandStream::Stdout, b"pre");
let id = retained.id();
let state = Arc::new(Mutex::new(BashExecutionState::default()));
let handle = CommandHandle(PRIVATE.into());
state.lock().unwrap().active.insert(
PRIVATE.into(),
ActiveCommand {
call_id: PRIVATE.into(),
execution_nonce: 1,
session: provider.clone(),
handle: handle.clone(),
},
);
state
.lock()
.unwrap()
.cancellation_requested
.insert(PRIVATE.into());
let guard = CommandGuard {
session: provider.clone(),
state: state.clone(),
execution_id: PRIVATE.into(),
execution_nonce: 1,
handle: Some(handle),
cursor: 3,
retained: Some(retained),
};
(
Self {
_root: root,
provider,
state,
view,
id,
},
guard,
)
}
async fn finish(&self) -> BashOutputMetadata {
self.provider.allow_drain.store(true, Ordering::Release);
// The provider's read retry backoff and cleanup deadlines are virtual time.
for _ in 0..4 {
tokio::task::yield_now().await;
tokio::time::advance(Duration::from_secs(3)).await;
}
for _ in 0..1000 {
if let Some(metadata) = self.view.metadata(&self.id) {
return metadata;
}
tokio::task::yield_now().await;
}
panic!("finite cleanup did not publish partial evidence");
}
fn assert_evidence(&self, m: &BashOutputMetadata, terminal: bool, bytes: &[u8]) {
assert_eq!(
self.view
.read(&self.id, CommandStream::Stdout, 0, 64)
.unwrap(),
bytes
);
assert_eq!(m.stdout_saved, bytes.len());
assert_eq!(m.stdout_delivered, bytes.len() as u64);
assert_eq!(m.stdout_seen, bytes.len() as u64);
assert_eq!(
m.terminal.status,
terminal.then_some(CommandStatus::Completed)
);
assert_eq!(m.terminal.exit_code, terminal.then_some(0));
if !terminal {
assert!(m.terminal.output_collection.is_none());
assert!(m.terminal.process_failure.is_none());
assert!(!m.terminal.transport_complete);
}
let diagnostics = m.cleanup.as_ref().unwrap();
assert_eq!(diagnostics.execution_nonce, 1);
let json = serde_json::to_string(diagnostics).unwrap();
let debug = format!("{diagnostics:?}");
for forbidden in [
PRIVATE, "SECRET", "/private", "password", "poison", "tail", "pre",
] {
assert!(!json.contains(forbidden), "diagnostics leaked data: {json}");
assert!(
!debug.contains(forbidden),
"tracing payload leaked data: {debug}"
);
}
}
}
#[tokio::test(start_paused = true)]
async fn cancel_failure_does_not_preclude_confirmed_terminal() {
let (f, guard) = Fixture::start(Fault::Cancel);
drop(guard);
f.provider.cancelling.notified().await;
assert!(f.state.lock().unwrap().active.contains_key(PRIVATE));
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(m.terminal.transport_complete);
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::Failed { .. }
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(!m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
assert!(f.state.lock().unwrap().cancellation_requested.is_empty());
}
#[tokio::test(start_paused = true)]
async fn acknowledged_cancel_drain_faults_keep_distinct_reasons_and_partial_bytes() {
for (fault, reason, bytes) in [
(Fault::Drain, "provider", b"pre".as_slice()),
(Fault::Sequence, "sequence_gap", b"pre"),
(Fault::Cursor, "cursor_gap", b"pretail"),
(Fault::RetainedChunk, "retained_chunk_invalid", b"pre"),
(
Fault::TerminalMetadata,
"terminal_metadata_invalid",
b"pretail",
),
] {
let (f, guard) = Fixture::start(fault);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, false, bytes);
let diagnostic = serde_json::to_value(m.cleanup.as_ref().unwrap()).unwrap();
assert_eq!(diagnostic["cancel"]["outcome"], "acknowledged");
assert_eq!(diagnostic["drain"]["outcome"], "failed");
assert_eq!(diagnostic["drain"]["reason"]["kind"], reason);
assert_eq!(diagnostic["registration"], "released");
assert_eq!(diagnostic["deadline_expired"], false);
assert!(f.state.lock().unwrap().active.is_empty());
}
}
#[tokio::test(start_paused = true)]
async fn terminal_deadline_never_claims_stop_or_leaves_active_collector() {
let (f, guard) = Fixture::start(Fault::TerminalTimeout);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, false, b"pre");
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::Acknowledged
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::DeadlineExceeded
));
assert!(m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
}
#[tokio::test(start_paused = true)]
async fn cancel_deadline_still_attempts_independent_terminal_observation() {
let (f, guard) = Fixture::start(Fault::CancelHang);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::DeadlineExceeded
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(!m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
}
#[tokio::test(start_paused = true)]
async fn old_cleanup_preserves_new_execution_and_cancellation_registration() {
let (f, guard) = Fixture::start(Fault::Complete);
drop(guard);
f.provider.cancelling.notified().await;
f.state
.lock()
.unwrap()
.active
.get_mut(PRIVATE)
.unwrap()
.execution_nonce = 2;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(matches!(
m.cleanup.as_ref().unwrap().registration,
CleanupRegistration::Superseded
));
assert_eq!(f.state.lock().unwrap().active[PRIVATE].execution_nonce, 2);
assert!(
f.state
.lock()
.unwrap()
.cancellation_requested
.contains(PRIVATE)
);
}
#[tokio::test(start_paused = true)]
async fn terminal_confirmation_is_independent_of_output_collection_completeness() {
let (f, guard) = Fixture::start(Fault::CollectionIncomplete);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(!m.terminal.transport_complete);
assert!(matches!(
m.terminal.output_collection,
Some(CommandOutputCollection::Incomplete { .. })
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(f.state.lock().unwrap().active.is_empty());
}
+725
View File
@@ -0,0 +1,725 @@
//! Bash-owned bounded output, independent of Workdir transport and Console subscribers.
//!
//! Output is ephemeral evidence, not a filesystem grant. Every lookup checks the live view;
//! children can see their own subtree and ancestors can retain evidence after child revocation.
//! Root teardown clears all bytes. A new root (including restore) has a new incarnation and
//! never reconstructs grants from history. Expiry is measured from admission, enforced on
//! lookup and by a weak reaper once commands run in Tokio. Parent and all descendants share
//! one pool: admission reserves a whole command budget, evicts the oldest terminal evidence
//! first, and refuses rather than evicting a live collector. Revoking a child denies access
//! but leaves ancestor evidence until pool eviction/expiry; root teardown clears the pool.
use std::collections::VecDeque;
use std::sync::{
Arc, Mutex, Weak,
atomic::{AtomicBool, Ordering},
};
use std::time::{Duration, Instant};
use serde::Serialize;
use workdir::{CommandStatus, CommandStream};
pub(crate) const COMMAND_BYTES: usize = 4 * 1024 * 1024;
const STORE_BYTES: usize = 32 * 1024 * 1024;
const STORE_COUNT: usize = 64;
const ORDER_COUNT: usize = 4096;
const RETENTION: Duration = Duration::from_secs(3600);
#[derive(Clone)]
pub struct BashOutputView {
store: Arc<Mutex<Store>>,
lineage: Vec<String>,
active: Arc<AtomicBool>,
authority: Arc<dyn Fn() -> bool + Send + Sync>,
}
struct Store {
records: VecDeque<Arc<Mutex<Record>>>,
reserved: usize,
retention: Duration,
reaper_started: bool,
_shutdown: tokio::sync::watch::Sender<()>,
}
struct Record {
id: String,
owner: Vec<String>,
created: Instant,
stdout: Vec<u8>,
stderr: Vec<u8>,
order: Vec<OutputRange>,
stdout_seen: u64,
stderr_seen: u64,
truncated: bool,
limit: Option<RetentionLimit>,
transport_counts: Option<(u64, u64)>,
transport_truncation: Option<workdir::CommandOutputTruncation>,
terminal: Option<OutputTerminal>,
cleanup: Option<CleanupDiagnostics>,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputRange {
pub stream: CommandStream,
pub start: u64,
pub end: u64,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputTerminal {
pub status: Option<CommandStatus>,
pub exit_code: Option<i32>,
pub process_failure: Option<workdir::CommandFailure>,
pub output_collection: Option<workdir::CommandOutputCollection>,
/// True only for confirmed EOF collection (quota is reported separately). A terminal
/// fence or exit zero alone does not confirm the unobserved process tail.
pub transport_complete: bool,
}
#[derive(Clone, Copy, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum RetentionLimit {
ByteQuota,
OrderQuota,
}
#[derive(Clone, Debug, Serialize)]
pub struct BashOutputMetadata {
pub id: String,
pub stdout_saved: usize,
pub stderr_saved: usize,
pub stdout_seen: u64,
pub stderr_seen: u64,
pub stdout_delivered: u64,
pub stderr_delivered: u64,
pub retention_limit: Option<RetentionLimit>,
pub transport_truncation: Option<workdir::CommandOutputTruncation>,
pub storage_truncated: bool,
pub terminal: OutputTerminal,
pub cleanup: Option<CleanupDiagnostics>,
pub text_encoding: &'static str,
pub ordering: &'static str,
}
/// Drop cleanup observations, not assertions that cancellation stopped a process.
/// Correlate with the enclosing output UUID and this host-assigned execution generation;
/// caller identifiers, provider handles, commands and error text are deliberately excluded.
#[derive(Clone, Debug, Serialize)]
pub struct CleanupDiagnostics {
pub execution_nonce: u64,
pub cancel: CleanupCancel,
pub drain: CleanupDrain,
pub deadline_expired: bool,
pub registration: CleanupRegistration,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "outcome", rename_all = "snake_case")]
pub enum CleanupCancel {
Acknowledged,
Failed { reason: CleanupProviderFailure },
DeadlineExceeded,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "outcome", rename_all = "snake_case")]
pub enum CleanupDrain {
TerminalConfirmed,
Failed { reason: CleanupFailure },
DeadlineExceeded,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum CleanupFailure {
Provider { reason: CleanupProviderFailure },
SequenceGap,
CursorGap,
DeliveryBudgetExceeded,
RetainedChunkInvalid,
TerminalMetadataInvalid,
}
#[derive(Clone, Debug, Serialize)]
pub struct CleanupProviderFailure {
pub failure_reason: Option<workdir::WorkdirFailureReason>,
pub denial_reason: Option<workdir::WorkdirDenialReason>,
}
impl From<&workdir::WorkdirError> for CleanupProviderFailure {
fn from(error: &workdir::WorkdirError) -> Self {
Self {
failure_reason: error.failure_reason(),
denial_reason: error.denial_reason(),
}
}
}
#[derive(Clone, Copy, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum CleanupRegistration {
Released,
Superseded,
}
/// A bounded reservation acquired before command start. Drop terminalizes partial evidence.
pub struct BashOutputWriter {
record: Arc<Mutex<Record>>,
}
impl Default for BashOutputView {
fn default() -> Self {
Self::new()
}
}
impl BashOutputView {
pub fn new() -> Self {
let (shutdown, _) = tokio::sync::watch::channel(());
let store = Arc::new(Mutex::new(Store {
records: VecDeque::new(),
reserved: 0,
retention: RETENTION,
reaper_started: false,
_shutdown: shutdown,
}));
let view = Self {
store,
lineage: vec![uuid::Uuid::now_v7().to_string()],
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(|| true),
};
view.start_reaper();
view
}
fn start_reaper(&self) {
let Ok(runtime) = tokio::runtime::Handle::try_current() else {
return;
};
let mut store = self.store.lock().unwrap();
if store.reaper_started {
return;
}
let receiver = store._shutdown.subscribe();
store.reaper_started = true;
let weak = Arc::downgrade(&self.store);
runtime.spawn(reap(weak, receiver));
}
pub fn byte_budget(&self) -> u64 {
COMMAND_BYTES as u64
}
pub fn child(&self, authority: Arc<dyn Fn() -> bool + Send + Sync>) -> Self {
let parent = self.clone();
let mut lineage = self.lineage.clone();
lineage.push(uuid::Uuid::now_v7().to_string());
Self {
store: self.store.clone(),
lineage,
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(move || parent.is_active() && authority()),
}
}
pub fn is_active(&self) -> bool {
self.active.load(Ordering::Acquire) && (self.authority)()
}
pub fn is_root(&self) -> bool {
self.lineage.len() == 1
}
pub fn revoke(&self) {
self.active.store(false, Ordering::Release);
if self.lineage.len() == 1 {
let mut store = self.store.lock().unwrap();
store.records.clear();
store.reserved = 0;
}
}
pub fn begin(&self) -> Result<BashOutputWriter, String> {
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
// A host may construct the Feature before entering Tokio. Command admission
// happens inside the runtime, so idle expiry must start here as well.
self.start_reaper();
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
store.expire(Instant::now());
while store.records.len() >= STORE_COUNT || store.reserved + COMMAND_BYTES > STORE_BYTES {
let Some(index) = store
.records
.iter()
.position(|r| r.lock().unwrap().terminal.is_some())
else {
return Err("Bash output capacity is occupied by live commands".into());
};
store.records.remove(index);
store.reserved -= COMMAND_BYTES;
}
let record = Arc::new(Mutex::new(Record {
id: uuid::Uuid::now_v7().to_string(),
owner: self.lineage.clone(),
created: Instant::now(),
stdout: Vec::new(),
stderr: Vec::new(),
order: Vec::new(),
stdout_seen: 0,
stderr_seen: 0,
truncated: false,
limit: None,
transport_counts: None,
transport_truncation: None,
terminal: None,
cleanup: None,
}));
store.records.push_back(record.clone());
store.reserved += COMMAND_BYTES;
Ok(BashOutputWriter { record })
}
fn record(&self, id: &str) -> Option<Arc<Mutex<Record>>> {
if !self.is_active() {
return None;
}
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return None;
}
store.expire(Instant::now());
store
.records
.iter()
.find(|r| {
let r = r.lock().unwrap();
r.id == id && r.owner.starts_with(&self.lineage) && r.terminal.is_some()
})
.cloned()
}
pub fn metadata(&self, id: &str) -> Option<BashOutputMetadata> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.metadata())
}
/// Copies at most the caller's bounded range; never publishes output bodies in descriptors.
pub fn read(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Vec<u8>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let start = offset.min(bytes.len());
Some(bytes[start..start.saturating_add(limit.min(64 * 1024)).min(bytes.len())].to_vec())
}
/// Scan the bounded raw source without cloning it, decoding only the requested line window.
/// A long single line uses the common byte reader instead of allocating an oversized text page.
pub fn line_window(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Result<(String, bool), String>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let mut lines = bytes.split_inclusive(|b| *b == b'\n').skip(offset);
let mut window = Vec::new();
for line in lines.by_ref().take(limit.min(2000)) {
if window.len() + line.len() > 64 * 1024 {
return Some(Err(
"text source window exceeds 64 KiB; use the common byte reader".into(),
));
}
window.extend_from_slice(line);
}
let more = lines.next().is_some();
Some(Ok((String::from_utf8_lossy(&window).into_owned(), more)))
}
pub fn order(&self, id: &str) -> Option<Vec<OutputRange>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.order.clone())
}
}
impl Record {
fn metadata(&self) -> BashOutputMetadata {
BashOutputMetadata {
id: self.id.clone(),
stdout_saved: self.stdout.len(),
stderr_saved: self.stderr.len(),
stdout_seen: self.transport_counts.map_or(self.stdout_seen, |c| c.0),
stderr_seen: self.transport_counts.map_or(self.stderr_seen, |c| c.1),
stdout_delivered: self.stdout_seen,
stderr_delivered: self.stderr_seen,
retention_limit: self.limit,
transport_truncation: self.transport_truncation.clone(),
storage_truncated: self.truncated || self.transport_truncation.is_some(),
terminal: self.terminal.clone().expect("finalized record"),
cleanup: self.cleanup.clone(),
text_encoding: "utf8_lossy",
ordering: "provider_read_observation",
}
}
}
impl Store {
fn expire(&mut self, now: Instant) {
let before = self.records.len();
self.records
.retain(|r| now.duration_since(r.lock().unwrap().created) < self.retention);
self.reserved -= (before - self.records.len()) * COMMAND_BYTES;
}
}
async fn reap(weak: Weak<Mutex<Store>>, mut shutdown: tokio::sync::watch::Receiver<()>) {
loop {
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(30)) => {},
_ = shutdown.changed() => return,
}
let Some(store) = weak.upgrade() else {
return;
};
store.lock().unwrap().expire(Instant::now());
}
}
impl BashOutputWriter {
pub(crate) fn id(&self) -> String {
self.record.lock().unwrap().id.clone()
}
pub(crate) fn finish_cleanup(&mut self, diagnostics: CleanupDiagnostics) {
self.record.lock().unwrap().cleanup = Some(diagnostics);
}
pub fn append_chunk(&mut self, chunk: &workdir::CommandByteChunk) -> Result<(), String> {
let expected = {
let r = self.record.lock().unwrap();
match chunk.stream {
CommandStream::Stdout => r.stdout_seen,
CommandStream::Stderr => r.stderr_seen,
}
};
if chunk.offset != expected || chunk.bytes.is_empty() {
return Err("command byte transport stream gap or empty chunk".into());
}
self.append(chunk.stream, &chunk.bytes);
Ok(())
}
pub fn append(&mut self, stream: CommandStream, bytes: &[u8]) {
let mut r = self.record.lock().unwrap();
let seen = match stream {
CommandStream::Stdout => &mut r.stdout_seen,
CommandStream::Stderr => &mut r.stderr_seen,
};
let start = *seen;
*seen = seen.saturating_add(bytes.len() as u64);
let available = COMMAND_BYTES.saturating_sub(r.stdout.len() + r.stderr.len());
let n = if r.truncated || r.order.len() >= ORDER_COUNT {
0
} else {
available.min(bytes.len())
};
if n < bytes.len() {
r.truncated = true;
if r.limit.is_none() {
r.limit = Some(if r.order.len() >= ORDER_COUNT {
RetentionLimit::OrderQuota
} else {
RetentionLimit::ByteQuota
});
}
}
if n > 0 {
match stream {
CommandStream::Stdout => r.stdout.extend_from_slice(&bytes[..n]),
CommandStream::Stderr => r.stderr.extend_from_slice(&bytes[..n]),
}
if let Some(last) = r
.order
.last_mut()
.filter(|last| last.stream == stream && last.end == start)
{
last.end += n as u64;
} else {
r.order.push(OutputRange {
stream,
start,
end: start + n as u64,
});
}
}
}
pub fn finish_transport(
&mut self,
terminal: &workdir::CommandTerminal,
) -> Result<BashOutputMetadata, String> {
let mut r = self.record.lock().unwrap();
if !terminal.metadata_valid()
|| terminal.stdout_delivered_bytes != r.stdout_seen
|| terminal.stderr_delivered_bytes != r.stderr_seen
{
return Err("command terminal counts do not fence delivered bytes".into());
}
r.transport_counts = Some((terminal.stdout_bytes, terminal.stderr_bytes));
r.transport_truncation = terminal.truncation_reason.clone();
r.terminal = Some(OutputTerminal {
status: Some(terminal.status),
exit_code: terminal.exit_code,
transport_complete: matches!(
terminal.output_collection,
workdir::CommandOutputCollection::Complete
),
process_failure: terminal.process_failure,
output_collection: Some(terminal.output_collection),
});
Ok(r.metadata())
}
pub fn finish(&mut self, terminal: OutputTerminal) -> BashOutputMetadata {
let mut r = self.record.lock().unwrap();
r.terminal = Some(terminal);
r.metadata()
}
}
impl Drop for BashOutputWriter {
fn drop(&mut self) {
let mut r = self.record.lock().unwrap();
if r.terminal.is_none() {
r.terminal = Some(OutputTerminal {
status: None,
exit_code: None,
transport_complete: false,
process_failure: None,
output_collection: None,
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn finish(view: &BashOutputView, text: &[u8]) -> String {
let mut writer = view.begin().unwrap();
writer.append(CommandStream::Stdout, text);
writer
.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
process_failure: None,
output_collection: Some(workdir::CommandOutputCollection::Complete),
})
.id
}
#[test]
fn revoked_child_is_denied_while_parent_keeps_evidence_and_sibling_is_denied() {
let root = BashOutputView::new();
let lease = Arc::new(AtomicBool::new(true));
let child = root.child({
let lease = lease.clone();
Arc::new(move || lease.load(Ordering::Acquire))
});
let sibling = root.child(Arc::new(|| true));
let id = finish(&child, b"child");
assert_eq!(
child.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
assert!(sibling.metadata(&id).is_none());
assert!(BashOutputView::new().metadata(&id).is_none());
lease.store(false, Ordering::Release);
assert!(child.metadata(&id).is_none());
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
root.revoke();
let restored = BashOutputView::new();
assert!(root.metadata(&id).is_none());
assert!(restored.metadata(&id).is_none());
let new_id = finish(&restored, b"new incarnation");
assert_ne!(new_id, id);
assert!(child.metadata(&new_id).is_none());
assert!(root.begin().is_err());
}
#[test]
fn parent_and_descendants_share_admission_without_evicting_live_evidence() {
let root = BashOutputView::new();
let child = root.child(Arc::new(|| true));
let sibling = root.child(Arc::new(|| true));
let mut writers = (0..STORE_BYTES / COMMAND_BYTES)
.map(|index| {
if index % 2 == 0 {
root.begin()
} else {
child.begin()
}
.unwrap()
})
.collect::<Vec<_>>();
assert!(sibling.begin().is_err());
let mut first = writers.remove(0);
first.append(CommandStream::Stdout, b"finished");
let metadata = first.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
process_failure: None,
output_collection: Some(workdir::CommandOutputCollection::Complete),
});
assert!(root.metadata(&metadata.id).is_some());
let _replacement = sibling.begin().unwrap();
assert!(root.metadata(&metadata.id).is_none());
assert!(root.begin().is_err());
assert_eq!(writers.len(), STORE_BYTES / COMMAND_BYTES - 1);
}
#[test]
fn quota_and_forced_drop_are_distinct_from_preview_omission() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, &vec![b'x'; COMMAND_BYTES + 1]);
writer.append(CommandStream::Stderr, b"tail");
let id = writer.record.lock().unwrap().id.clone();
drop(writer);
let m = root.metadata(&id).unwrap();
assert_eq!(m.stdout_saved, COMMAND_BYTES);
assert_eq!(m.stdout_seen, COMMAND_BYTES as u64 + 1);
assert_eq!(m.stderr_seen, 4);
assert!(m.storage_truncated);
assert!(!m.terminal.transport_complete);
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, usize::MAX)
.unwrap()
.len(),
64 * 1024
);
}
#[test]
fn expiry_and_count_eviction_never_reuse_old_references() {
let root = BashOutputView::new();
let id = finish(&root, b"old");
root.store.lock().unwrap().retention = Duration::ZERO;
assert!(root.metadata(&id).is_none());
root.store.lock().unwrap().retention = RETENTION;
let first = finish(&root, b"first");
for _ in 0..STORE_COUNT {
finish(&root, b"later");
}
assert!(root.metadata(&first).is_none());
assert!(root.store.lock().unwrap().reserved <= STORE_BYTES);
}
#[tokio::test(start_paused = true)]
async fn expiry_reclaims_bytes_without_another_read_and_reaper_ends_with_owner() {
// Exercise host construction outside a runtime, followed by real async admission.
let root = std::thread::spawn(BashOutputView::new).join().unwrap();
finish(&root, b"expires without lookup");
let mut closed = root.store.lock().unwrap()._shutdown.subscribe();
tokio::task::yield_now().await;
// The store uses monotonic wall time for publication and Tokio time only for scheduling.
// Age the record deterministically rather than delaying this test in real time.
root.store.lock().unwrap().records[0]
.lock()
.unwrap()
.created = Instant::now() - RETENTION;
tokio::time::advance(Duration::from_secs(31)).await;
tokio::task::yield_now().await;
assert_eq!(root.store.lock().unwrap().reserved, 0);
drop(root);
assert!(closed.changed().await.is_err());
}
#[test]
fn terminal_counts_must_fence_the_received_prefix_before_marking_evidence_complete() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, b"abc");
let mut terminal = serde_json::json!({"status":"completed","exit_code":0,"process_failure":null,"output_collection":{"state":"complete"},"stdout_bytes":9,"stderr_bytes":0,"stdout_delivered_bytes":4,"stderr_delivered_bytes":0,"truncation_reason":"byte_budget"});
assert!(
writer
.finish_transport(&serde_json::from_value(terminal.clone()).unwrap())
.is_err()
);
terminal["stdout_delivered_bytes"] = serde_json::json!(3);
let metadata = writer
.finish_transport(&serde_json::from_value(terminal).unwrap())
.unwrap();
assert_eq!(metadata.stdout_saved, 3);
assert_eq!(metadata.stdout_delivered, 3);
assert_eq!(metadata.stdout_seen, 9);
assert!(metadata.storage_truncated);
assert!(metadata.retention_limit.is_none());
assert_eq!(
metadata.transport_truncation,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
}
#[test]
fn incomplete_collection_with_quota_retains_both_causes_without_claiming_complete_output() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, b"abc");
let terminal: workdir::CommandTerminal = serde_json::from_value(serde_json::json!({
"status":"completed", "exit_code":0, "process_failure":null,
"output_collection":{"state":"incomplete","failure":{"stage":"stdout_drain","reason":"other","os_code":null}},
"stdout_bytes":9, "stderr_bytes":0, "stdout_delivered_bytes":3, "stderr_delivered_bytes":0,
"truncation_reason":"byte_budget"
})).unwrap();
let metadata = writer.finish_transport(&terminal).unwrap();
assert!(!metadata.terminal.transport_complete);
assert_eq!(metadata.terminal.exit_code, Some(0));
assert_eq!(
metadata.terminal.output_collection,
Some(terminal.output_collection)
);
assert_eq!(
metadata.transport_truncation,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
assert_eq!(metadata.stdout_seen, 9);
assert_eq!(metadata.stdout_saved, 3);
assert!(metadata.storage_truncated);
assert!(metadata.retention_limit.is_none());
assert_eq!(
root.read(&metadata.id, CommandStream::Stdout, 0, 100)
.unwrap(),
b"abc"
);
assert_eq!(
serde_json::to_value(root.metadata(&metadata.id).unwrap()).unwrap(),
serde_json::to_value(metadata).unwrap()
);
}
}
+68
View File
@@ -403,3 +403,71 @@ fn check_pattern(pattern: &str) -> Result<(), ToolError> {
#[cfg(test)]
#[path = "checkout_tests.rs"]
mod tests;
/// Read a bounded native text window through the same checked target.
/// The returned one-shot action records prior-Read only after the caller's presentation succeeds.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_text(
router: Arc<WorkdirSessionRouter>,
tracker: Tracker,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: usize,
limit: usize,
max_bytes: usize,
) -> Result<(workdir::ReadResult, Option<Vec<u8>>, impl FnOnce()), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let tracker = tracker.scoped_attachment(&selected.alias, selected.generation);
let target = FileTarget {
session: selected.session.clone(),
path: path.clone(),
validator: Some(validator),
};
let (result, validator) = target.read(offset, limit, max_bytes).await?;
let hash = result.content_hash.clone();
let total_lines = result.total_lines;
Ok((result, validator, move || {
tracker.record_workdir_observation(&path, hash, total_lines);
}))
}
/// Binary-safe checked byte window. This does not establish a text prior-Read observation.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_bytes(
router: Arc<WorkdirSessionRouter>,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: u64,
max_bytes: usize,
) -> Result<(workdir::ReadBytesResult, Vec<u8>), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let result = selected
.session
.checkout_execute(workdir::CheckoutRequest {
target: path.clone(),
validator,
operation: workdir::CheckoutOperation::ReadBytes { offset, max_bytes },
})
.await
.map_err(crate::file_target::checked_error)?;
match result.output {
workdir::CheckoutOutput::ReadBytes(read)
if read.path == path && read.offset == offset && read.bytes.len() <= max_bytes =>
{
Ok((read, result.observation.validator))
}
_ => Err(crate::file_target::readonly_result_error("ReadBytes")),
}
}
+1 -1
View File
@@ -21,7 +21,7 @@ pub(crate) struct EditParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
pub target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
/// Existing file in which to replace text.
pub file_path: String,
#[serde(flatten)]
pub edit: fs_operation::text::EditArgs,
+4 -1
View File
@@ -147,7 +147,9 @@ impl FileTarget {
pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
use workdir::WorkdirError;
let code = match error.classification_source() {
WorkdirError::DenialContext { .. } => unreachable!("diagnostic context unwrapped"),
WorkdirError::DenialContext { .. } | WorkdirError::FailureContext { .. } => {
unreachable!("diagnostic context unwrapped")
}
WorkdirError::Conflict(_) | WorkdirError::NotFound(_) => "checkout_stale",
WorkdirError::Denied { .. }
| WorkdirError::OutOfScope { .. }
@@ -165,6 +167,7 @@ pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
| WorkdirError::IsDirectory { .. }
| WorkdirError::InvalidGlob { .. }
| WorkdirError::InvalidRegex { .. } => "checkout_invalid",
WorkdirError::NotDispatched => "checkout_not_dispatched",
WorkdirError::Unavailable { .. } | WorkdirError::SessionClosed => "checkout_unavailable",
// IO/transport failures may arrive after effects have begun. Do not infer
// no-effects from their text or strip explicit OutcomeUnknown evidence.
+2 -2
View File
@@ -17,7 +17,7 @@ pub(crate) struct GlobParams {
pub target_workdir: Option<String>,
/// Glob pattern, for example `**/*.rs` or `src/**/test_*.py`.
pub pattern: String,
/// Logical Workdir-relative directory. Defaults to the Workdir root.
/// Directory to search. Defaults to `.`.
#[serde(default)]
pub path: Option<String>,
}
@@ -106,7 +106,7 @@ pub(crate) fn routed_glob_tool(router: Arc<WorkdirSessionRouter>) -> ToolDefinit
Arc::new(move || {
let schema = schemars::schema_for!(GlobParams);
let meta = ToolMeta::new("Glob")
.description("Find files matching a glob pattern inside the selected Workdir. Results are sorted and capped at 1000 entries. Paths are Workdir-relative.")
.description("Find files matching a glob pattern inside the selected Workdir. Results are sorted and capped at 1000 entries.")
.input_schema(serde_json::to_value(schema).expect("Glob schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(GlobTool {
router: router.clone(),
+2 -2
View File
@@ -27,7 +27,7 @@ pub(crate) struct GrepParams {
#[serde(default)]
target_workdir: Option<String>,
pattern: String,
/// Workdir-relative path, or an absolute path covered by readable scope. Defaults to the Workdir root.
/// File or directory to search. Defaults to `.`.
#[serde(default)]
pub path: Option<String>,
#[serde(default)]
@@ -160,7 +160,7 @@ pub(crate) fn routed_grep_tool(router: Arc<WorkdirSessionRouter>) -> ToolDefinit
Arc::new(move || {
let schema = schemars::schema_for!(GrepParams);
let meta = ToolMeta::new("Grep")
.description("Search a file or directory in the selected Workdir with a regex. Content results group lines by file; `>` marks matching lines and unmarked lines are context. Directory traversal executes inside the WorkdirSession provider. Results are bounded and Workdir-relative.")
.description("Search a file or directory in the selected Workdir with a regex. Content results group lines by file; `>` marks matching lines and unmarked lines are context. Directory traversal executes inside the WorkdirSession provider. Results are bounded.")
.input_schema(serde_json::to_value(schema).expect("Grep schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(GrepTool {
router: router.clone(),
+10 -9
View File
@@ -5,6 +5,7 @@ pub mod error;
pub mod tracker;
mod bash;
pub mod bash_output;
mod edit;
mod file_target;
mod glob;
@@ -16,7 +17,7 @@ pub mod view_image;
mod web;
mod write;
pub use bash::bash_tool;
pub use bash::{BashResult, BashTool, bash_tool};
pub use checkout::{CheckoutToolOutput, execute_checkout_tool};
pub use edit::edit_tool;
pub use error::ToolsError;
@@ -35,7 +36,7 @@ pub use write::write_tool;
pub fn core_builtin_tools(
session: workdir::WorkdirSessionHandle,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -58,7 +59,7 @@ pub fn core_builtin_tools(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -71,7 +72,7 @@ pub fn core_builtin_tools(
pub fn routed_builtin_tools(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
vec![
read::routed_read_tool(router.clone(), tracker.clone()),
@@ -79,14 +80,14 @@ pub fn routed_builtin_tools(
edit::routed_edit_tool(router.clone(), tracker),
glob::routed_glob_tool(router.clone()),
grep::routed_grep_tool(router.clone()),
bash::routed_bash_tool(router, bash_output_dir),
bash::routed_bash_tool(router, bash_output),
]
}
pub fn routed_builtin_tools_for_current_capabilities(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -108,7 +109,7 @@ pub fn routed_builtin_tools_for_current_capabilities(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -121,7 +122,7 @@ pub fn read_only_builtin_tools(
workdir::WorkdirSessionCapabilities::READ_ONLY,
"read-only tool projection requires a read-only Workdir session"
);
core_builtin_tools(session, Tracker::new(), std::path::PathBuf::new())
core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
}
pub fn read_only_routed_builtin_tools(
@@ -173,7 +174,7 @@ mod workdir_tool_tests {
SharedScope::new(Scope::writable(dir.path()).unwrap()),
WorkdirSessionCapabilities::READ_ONLY,
));
let names = core_builtin_tools(session, Tracker::new(), dir.path().join("output"))
let names = core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
.into_iter()
.map(|definition| definition().0.name)
.collect::<Vec<_>>();
+2 -2
View File
@@ -13,7 +13,7 @@ use workdir::{WorkdirPath, WorkdirSessionHandle, WorkdirSessionRouter};
const DESCRIPTION: &str = "Read a text file from a Workdir attachment selected by its Worker-local alias. \
Supports offset/limit for large files. Returns line-numbered output (1-based). \
Directories cannot be read. The file must be read before Write or Edit can \
modify it. Paths are Workdir-relative unless an absolute path is explicitly readable. \
modify it. \
When exactly one Workdir is attached, target_workdir may be omitted.";
const DEFAULT_LIMIT: usize = 2000;
@@ -24,7 +24,7 @@ pub(crate) struct ReadParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
pub target_workdir: Option<String>,
/// Workdir-relative path, or an absolute path covered by readable scope.
/// Text file to read; directories are not supported.
pub file_path: String,
#[serde(flatten)]
pub read: fs_operation::text::LineReadArgs,
+2 -2
View File
@@ -20,7 +20,7 @@ pub const MAX_IMAGE_BYTES: usize = 10 * 1024 * 1024;
const IMAGE_READ_CHUNK_BYTES: usize = 1024 * 1024;
const DESCRIPTION: &str = "Attach an image from the selected Workdir attachment to the next model request. \
The path must be logical and Workdir-relative. Supported formats: PNG, JPEG, GIF, and WebP. \
Supported formats: PNG, JPEG, GIF, and WebP. \
When exactly one Workdir is attached, target_workdir may be omitted.";
#[derive(Debug, Deserialize, schemars::JsonSchema)]
@@ -28,7 +28,7 @@ struct ViewImageParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
/// Image file to attach to the next model request.
path: String,
}
+1 -1
View File
@@ -20,7 +20,7 @@ pub(crate) struct WriteParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
pub target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
/// File to create or overwrite.
pub file_path: String,
#[serde(flatten)]
pub write: fs_operation::text::WriteArgs,
+19 -27
View File
@@ -3,7 +3,7 @@
use std::sync::Arc;
use agen::tool::{Tool, ToolDefinition};
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use manifest::Scope;
use serde_json::json;
use tempfile::TempDir;
use tools::{Tracker, core_builtin_tools};
@@ -28,31 +28,23 @@ impl Registry {
}
}
fn setup() -> (TempDir, TempDir, Registry) {
fn setup() -> (TempDir, Registry) {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let base = Scope::writable(dir.path()).unwrap();
let mut config = ScopeConfig {
allow: base.allow_rules(),
deny: base.deny_rules(),
};
config.allow.push(ScopeRule {
target: spill.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
});
let scope = Scope::from_config(&config).unwrap();
let scope = Scope::writable(dir.path()).unwrap();
let fs: WorkdirSessionHandle =
std::sync::Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let tracker = Tracker::new();
let reg = Registry::new(core_builtin_tools(fs, tracker, spill.path().to_path_buf()));
(dir, spill, reg)
let reg = Registry::new(core_builtin_tools(
fs,
tracker,
tools::bash_output::BashOutputView::new(),
));
(dir, reg)
}
#[tokio::test]
async fn unicode_path_and_content() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("日本語ファイル.txt");
let content = "こんにちは 🦀 世界\nabc\n";
@@ -87,7 +79,7 @@ async fn unicode_path_and_content() {
async fn symlink_to_outside_scope_is_rejected_for_write() {
use std::os::unix::fs::symlink;
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let outside = TempDir::new().unwrap();
let outside_target = outside.path().join("secret.txt");
std::fs::write(&outside_target, "secret").unwrap();
@@ -151,7 +143,7 @@ async fn symlink_to_outside_scope_is_rejected_for_write() {
async fn broken_symlink_reports_target_and_repair_hint() {
use std::os::unix::fs::symlink;
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let link = dir.path().join("external-project");
let target = dir.path().join("missing-target");
symlink(&target, &link).unwrap();
@@ -173,7 +165,7 @@ async fn broken_symlink_reports_target_and_repair_hint() {
#[tokio::test]
async fn empty_file_read_and_edit() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("empty.txt");
std::fs::write(&file, "").unwrap();
@@ -207,7 +199,7 @@ async fn empty_file_read_and_edit() {
#[tokio::test]
async fn very_long_single_line() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("long.txt");
let big: String = "x".repeat(1024 * 1024); // 1 MiB, no newlines
std::fs::write(&file, &big).unwrap();
@@ -226,7 +218,7 @@ async fn very_long_single_line() {
#[tokio::test]
async fn absolute_path_requires_matching_read_scope() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let outside = tempfile::tempdir().unwrap();
let outside_file = outside.path().join("outside.txt");
std::fs::write(&outside_file, "secret").unwrap();
@@ -247,7 +239,7 @@ async fn absolute_path_requires_matching_read_scope() {
#[tokio::test]
async fn directory_target_is_rejected_for_read() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let read = reg.get("Read");
let err = read
.execute(&json!({ "file_path": "." }).to_string(), Default::default())
@@ -258,7 +250,7 @@ async fn directory_target_is_rejected_for_read() {
#[tokio::test]
async fn deeply_nested_new_file_is_created() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let deep = dir.path().join("a/b/c/d/e/deep.txt");
let write = reg.get("Write");
write
@@ -277,7 +269,7 @@ async fn deeply_nested_new_file_is_created() {
#[tokio::test]
async fn replace_preserves_unicode() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("u.txt");
std::fs::write(&file, "🦀 rust 🦀\n").unwrap();
@@ -306,7 +298,7 @@ async fn replace_preserves_unicode() {
#[tokio::test]
async fn grep_handles_unicode_pattern() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("u.txt");
std::fs::write(&file, "English\n日本語\nрусский\n").unwrap();
File diff suppressed because it is too large Load Diff
+36 -20
View File
@@ -3013,7 +3013,8 @@ impl App {
/// `[File: …]` parsing path: each kind maps directly to a typed
/// block (`Block::Notify`, `Block::WorkerEvent`, …).
fn apply_system_item(&mut self, value: &serde_json::Value) {
let Ok(item) = serde_json::from_value::<session_store::SystemItem>(value.clone()) else {
let Ok(item) = serde_json::from_value::<session_store::PublicSystemItem>(value.clone())
else {
// Unknown / forward-compat shape: fall back to rendering the
// raw text payload (if any) as a generic system message.
if let Some(text) = value.get("body").and_then(|b| b.as_str()) {
@@ -3024,29 +3025,23 @@ impl App {
}
return;
};
match item {
session_store::SystemItem::Notification { message, .. } => {
let body = item.body;
match item.presentation {
session_store::PublicSystemPresentation::Notification { message } => {
self.blocks.push(Block::Notify { message });
}
session_store::SystemItem::WorkerEvent { event, .. } => {
session_store::PublicSystemPresentation::WorkerEvent { event } => {
self.blocks.push(Block::WorkerEvent { event });
}
session_store::SystemItem::FeatureInvocationResult { body, .. }
| session_store::SystemItem::FileAttachment { body, .. }
| session_store::SystemItem::SkillActivation { body, .. }
| session_store::SystemItem::JournalStartup { body, .. }
| session_store::SystemItem::ResidentSummaryRefresh { body, .. }
| session_store::SystemItem::SubjectBehaviorRefresh { body, .. }
| session_store::SystemItem::Interrupt { body, .. } => {
session_store::PublicSystemPresentation::Context => {
self.task_store.apply_system_message_text(&body);
self.blocks.push(Block::SystemMessage { text: body });
}
session_store::SystemItem::TaskReminder { body, .. } => {
session_store::PublicSystemPresentation::TaskReminder => {
self.task_store.apply_system_message_text(&body);
self.blocks.push(Block::TaskReminder { text: body });
}
session_store::SystemItem::LegacyIgnored { .. } => {}
session_store::SystemItem::LegacyKnowledgeIgnored { .. } => {}
session_store::PublicSystemPresentation::Ignored => {}
}
}
@@ -3572,6 +3567,7 @@ mod completion_flow_tests {
session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4329,6 +4325,7 @@ mod completion_flow_tests {
}));
app.handle_worker_event(Event::SessionEntryCommitted {
entry: protocol::SessionSnapshotEntry {
use_policy: None,
entry_id: "run-failure".into(),
timestamp: 2,
provenance: protocol::SessionEntryProvenance::LegacyUnknown,
@@ -4450,12 +4447,12 @@ mod completion_flow_tests {
}
#[test]
fn live_legacy_workflow_system_item_is_ignored() {
fn retired_system_input_is_ignored() {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "workflow",
"slug": "build",
"body": "[Workflow /build]\nRun the build",
"kind": "retired_input",
"body": "",
"presentation": { "kind": "ignored" },
});
app.handle_worker_event(Event::SystemItem {
entry_id: None,
@@ -4861,12 +4858,30 @@ mod completion_flow_tests {
);
}
#[test]
fn feature_label_does_not_select_system_input_presentation() {
for label in ["journal_startup", "new-unregistered-feature"] {
let mut app = App::new("test".into());
app.handle_worker_event(Event::SystemItem {
entry_id: None,
item: serde_json::json!({
"kind": label,
"body": "explicit public content",
"presentation": { "kind": "context" },
}),
});
assert!(
matches!(app.blocks.as_slice(), [Block::SystemMessage { text }] if text == "explicit public content")
);
}
}
#[test]
fn live_system_item_notification_appends_notify_block() {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "notification",
"message": "hi",
"presentation": { "kind": "notification", "message": "hi" },
"body": "[Notification] hi",
});
app.handle_worker_event(Event::SystemItem {
@@ -4884,7 +4899,7 @@ mod completion_flow_tests {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "worker_event",
"event": { "kind": "turn_ended", "worker_name": "child" },
"presentation": { "kind": "worker_event", "event": { "kind": "turn_ended", "worker_name": "child" } },
"body": "[Notification] worker `child` finished a turn",
});
app.handle_worker_event(Event::SystemItem {
@@ -5285,6 +5300,7 @@ mod completion_flow_tests {
item: serde_json::json!({
"kind": "task_reminder",
"body": snapshot,
"presentation": { "kind": "task_reminder" },
}),
});
+1
View File
@@ -162,6 +162,7 @@ fn request_from_selection(selection: Selection) -> BackendCreateWorkerRequest {
ticket_assignment: None,
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
}
}
+63 -31
View File
@@ -785,6 +785,12 @@ mod tests {
#[derive(Default)]
struct PendingRuntimeBackend {
operations: std::sync::Mutex<Vec<worker_runtime::execution::WorkerLifecycleOperationId>>,
owners: std::sync::Mutex<
std::collections::HashMap<
worker_runtime::execution::WorkerLifecycleOperationId,
worker_runtime::identity::WorkerRef,
>,
>,
}
impl worker_runtime::execution::WorkerExecutionBackend for PendingRuntimeBackend {
@@ -806,6 +812,21 @@ mod tests {
)
}
fn restore_recovery_disposition(
&self,
operation_id: worker_runtime::execution::WorkerLifecycleOperationId,
worker_ref: &worker_runtime::identity::WorkerRef,
) -> Option<worker_runtime::execution::RestoreRecoveryDisposition> {
use worker_runtime::execution::RestoreRecoveryDisposition as D;
Some(
if self.owners.lock().unwrap().get(&operation_id) == Some(worker_ref) {
D::Continue
} else {
D::Absent
},
)
}
fn restore_worker(
&self,
request: worker_runtime::execution::WorkerExecutionRestoreRequest,
@@ -813,6 +834,10 @@ mod tests {
use worker_runtime::execution::*;
let mut operations = self.operations.lock().unwrap();
operations.push(request.operation_id);
self.owners
.lock()
.unwrap()
.insert(request.operation_id, request.worker_ref.clone());
if operations.len() == 1 {
WorkerExecutionSpawnResult::ReconciliationRequired {
result: WorkerExecutionResult::errored(
@@ -1119,39 +1144,45 @@ mod tests {
}
#[tokio::test]
async fn rejected_resume_does_not_observe_or_open_retained_state() {
let lifecycle = FakeBackendWorkerLifecycle {
restore_response: restore_response(BackendWorkerRestoreState::Rejected),
observation: session_observation(serde_json::json!({
"availability": "unavailable",
"reason": "retention_missing",
"message": "must not be opened"
})),
calls: std::sync::Mutex::new(Vec::new()),
};
let target = client::BackendRuntimeTarget::new(
"http://127.0.0.1:3000",
"workspace-a",
"runtime-a",
"worker-a",
);
async fn settled_nonaccepted_resume_does_not_retry_observe_or_open_retained_state() {
for state in [
BackendWorkerRestoreState::Rejected,
BackendWorkerRestoreState::RolledBack,
BackendWorkerRestoreState::Superseded,
] {
let lifecycle = FakeBackendWorkerLifecycle {
restore_response: restore_response(state),
observation: session_observation(serde_json::json!({
"availability": "unavailable",
"reason": "retention_missing",
"message": "must not be opened"
})),
calls: std::sync::Mutex::new(Vec::new()),
};
let target = client::BackendRuntimeTarget::new(
"http://127.0.0.1:3000",
"workspace-a",
"runtime-a",
"worker-a",
);
let error = prepare_selected_worker(
&lifecycle,
target,
BackendWorkerPickerIntent::Resume,
Some(client::BackendWorkerRestoreRequest {
expected_observation_token: "observed-worker-state".into(),
request_id: "resume-operation".into(),
}),
|_| Ok(false),
)
.await
.unwrap_err()
.to_string();
let error = prepare_selected_worker(
&lifecycle,
target,
BackendWorkerPickerIntent::Resume,
Some(client::BackendWorkerRestoreRequest {
expected_observation_token: "observed-worker-state".into(),
request_id: "resume-operation".into(),
}),
|_| panic!("settled nonaccepted restore must not offer recovery retry"),
)
.await
.unwrap_err()
.to_string();
assert!(error.contains("Rejected"));
assert_eq!(*lifecycle.calls.lock().unwrap(), ["restore"]);
assert!(error.contains(&format!("{state:?}")));
assert_eq!(*lifecycle.calls.lock().unwrap(), ["restore"]);
}
}
#[test]
@@ -1176,6 +1207,7 @@ mod tests {
for state in [
BackendWorkerRestoreState::Rejected,
BackendWorkerRestoreState::RolledBack,
BackendWorkerRestoreState::Superseded,
BackendWorkerRestoreState::ReconciliationRequired,
] {
let mut response = restore_response(state);
+2 -1
View File
@@ -20,8 +20,9 @@ serde_json.workspace = true
sha2.workspace = true
tempfile.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tokio = { workspace = true, features = ["io-util", "process", "rt", "sync", "time"] }
[dev-dependencies]
axum.workspace = true
serde_json.workspace = true
tempfile.workspace = true
+10 -6
View File
@@ -6,7 +6,7 @@
//! hashes alone do not accept same-content inode replacements. Callers own Read line numbering
//! and per-attachment read history.
//!
//! Create paths are checkout-root-relative and must be strict descendants of the observed target
//! Create paths use the target's session coordinates and must be strict descendants of the observed target
//! directory. Existing destinations are never overwritten; results return the updated observation
//! of the retained target directory, not the created file. Read results also return the validator
//! of the state actually read, never a replacement from a later independent stat.
@@ -18,7 +18,8 @@
//!
//! Search delegates to shared List/Glob/Grep engines. Scope rules form unions within layers and
//! intersections across layers, provider authority, and output roots. Wrappers transform coordinates
//! once; returned paths and Grep content are relative to the final session's logical root.
//! once; returned paths and Grep content are relative to the composed output root (the caller's
//! cwd by default).
//! Paths are not reverse-parsed from rendered filenames; non-UTF-8 names are not lossily aliased.
//! Unsupported providers reject search rather than fall back to broader trait searches. Search is
//! live traversal, not an atomic snapshot of all descendants or exclusion of external editors.
@@ -48,6 +49,10 @@ pub struct CheckoutRequest {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(tag = "operation", rename_all = "snake_case", deny_unknown_fields)]
pub enum CheckoutOperation {
ReadBytes {
offset: u64,
max_bytes: usize,
},
Read {
offset: usize,
limit: usize,
@@ -71,7 +76,7 @@ pub enum CheckoutOperation {
impl CheckoutOperation {
pub fn capability(&self) -> WorkdirSessionCapability {
match self {
Self::Read { .. } => WorkdirSessionCapability::Read,
Self::Read { .. } | Self::ReadBytes { .. } => WorkdirSessionCapability::Read,
Self::Edit { .. } => WorkdirSessionCapability::Edit,
Self::Write { .. } | Self::Create { .. } => WorkdirSessionCapability::Write,
}
@@ -86,6 +91,7 @@ impl CheckoutOperation {
)]
pub enum CheckoutOutput {
Read(ReadResult),
ReadBytes(crate::ReadBytesResult),
Write(WriteResult),
Edit(EditResult),
}
@@ -551,15 +557,13 @@ mod tests {
));
}
#[tokio::test]
async fn checkout_hardlink_save_isolates_alias_and_rejects_symlinks_special_files() {
async fn checkout_hardlink_save_isolates_alias_and_rejects_symlinks() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join("a"), "original").unwrap();
std::fs::hard_link(root.path().join("a"), root.path().join("alias")).unwrap();
std::os::unix::fs::symlink("a", root.path().join("link")).unwrap();
let _socket = std::os::unix::net::UnixListener::bind(root.path().join("socket")).unwrap();
let session = session(root.path());
assert!(session.checkout_observe(path("link")).await.is_err());
assert!(session.checkout_observe(path("socket")).await.is_err());
let read = checked_read(&session, session.checkout_observe(path("a")).await.unwrap()).await;
session
.checkout_execute(CheckoutRequest {
@@ -0,0 +1,43 @@
//! Bounded collection for tests whose subject is command routing/lifecycle,
//! rather than the byte-page protocol itself.
use crate::*;
#[derive(Debug)]
pub(crate) struct CollectedCommand {
pub content: String,
pub status: CommandStatus,
pub exit_code: Option<i32>,
pub timed_out: bool,
}
#[async_trait::async_trait]
pub(crate) trait CollectCommand: WorkdirSession {
async fn collect_command_output(
&self,
mut request: CommandOutputRequest,
) -> Result<CollectedCommand, WorkdirError> {
let mut bytes = Vec::new();
request.wait = true;
loop {
let page = self.command_output(request.clone()).await?;
for chunk in page.chunks {
bytes.extend(chunk.bytes);
assert!(
bytes.len() <= 16 * 1024 * 1024,
"fixture output exceeded bounded test collection"
);
}
request.cursor = page.next_cursor;
if let Some(terminal) = page.terminal {
return Ok(CollectedCommand {
content: String::from_utf8_lossy(&bytes).into_owned(),
status: terminal.status,
exit_code: terminal.exit_code,
timed_out: terminal.status == CommandStatus::TimedOut,
});
}
}
}
}
impl<T: WorkdirSession + ?Sized> CollectCommand for T {}

Some files were not shown because too many files have changed in this diff Show More