134 Commits
Author SHA1 Message Date
Hare 8b6e3070dd fix: align Workdir session cwd and model environment
Project alias, cwd, scope and command ceilings from actual sessions. Bind initial attachment cwd once in the shared broker/router while keeping root-relative delegation and detach generations. Preserve caller coordinates across reads, searches, checked checkout and commands; keep provider authorization and symlink boundaries.

Connect real SubWorkers, all filesystem Tools, scoped aliases, HTTP providers and limited Host grants in permanent regressions. Validated related normal suites (workdir http-client, tools, worker, worker-runtime, server-api typescript, fs-operation), workspace-server normal suite (1272 passed, 3 existing ignores), root check, product binary builds, rustdoc/doctests, generated contract checks, format and state-contract names. Test roots are checkout-local; Git parent discovery requires GIT_CEILING_DIRECTORIES at TMPDIR, confirmed against exact b860 baseline.
2026-10-22 20:46:10 +09:00
Hare b860c67ea2 fix: align Runtime restore diagnostics and socket test fixtures 2026-10-22 03:46:46 +09:00
Hare 533477c2e7 feat: unify headless execution under the Standalone Host
Expose an optional private same-uid Unix operator adapter and JSONL serve/attach CLI using the production Standalone Profile, store, restore lease and shutdown lifecycle. Remove direct Worker subprocess/discovery/readiness/controller sockets and parent socket callbacks; retain metadata-only peer registration and legacy safety readers.

Validated real Host/client input, events, provenance, disconnect, admitted shutdown and shared production restore/duplicate exclusion; normal standalone/worker/manifest/protocol/client/yoi/tui/server tests, root check, rustdoc, fmt and state-contract audit. Runtime lib: 476 passed, damaged_checkout_does_not_block_restore_but_is_rejected_when_used fails at missing_git diagnostic; unchanged restore logic, not relaxed or retried. Runtime bins/docs pass.
2026-10-22 01:29:39 +09:00
Hare 5aa2dc579a fix: integrate bounded Bash cleanup diagnostics 2026-10-22 00:46:41 +09:00
Hare 280b3d2a25 Merge branch 'develop' of /home/hare/Projects/yoi into work/T-775-cleanup-diagnostics 2026-10-22 00:24:00 +09:00
Hare ec74f4cdc4 fix: preserve Workdir failure diagnostics and audit outcomes 2026-10-22 00:16:11 +09:00
Hare 5af9dc8e4a test: focus checkout hardlink coverage on alias and symlink contracts 2026-10-21 23:57:01 +09:00
Hare ca6d4b4b72 fix: audit validated external workdir operation outcomes 2026-10-21 20:39:46 +09:00
Hare cccecbe7ca fix: integrate command output failure diagnostics 2026-10-21 06:52:23 +09:00
Hare 7a64c38493 fix: merge shared command diagnostic assertions
# Conflicts:
#	crates/workspace-server/src/server.rs
2026-10-21 06:40:26 +09:00
Hare b1361d86e9 fix: distinguish external Workdir admission dispatch and response failures
Integrate the published T773 audit contract without overwriting operation outcomes. Preserve bounded admission, send refusal, response loss/timeout and invalid result reasons independently of audit failures and request stages. Signed HTTP/SQLite/log fault tests pass (10), API logging 14 and relevant focused assertions pass. Root check, docs, generated artifacts, fmt/diff and state-contract naming checks pass. Full six-package normal suite ran with no-fail-fast but is not green: long Unix socket fixture paths and existing shared close assertion plus environment-isolation failures are recorded in target/t774 logs. Isolated focused checks resolved the latter; socket validation requires a granted shorter test root. This is a partial checkpoint, not a reviewed completion.
2026-10-21 06:39:54 +09:00
Hare 6181426abd fix: assert retained checkout outcome diagnostics 2026-10-21 06:27:54 +09:00
Hare cd331b93c6 Merge remote-tracking branch 'origin/work/T-773-workdir-audit' into work/T-774-workdir-failure-diagnostics
# Conflicts:
#	crates/workdir/src/http.rs
2026-10-21 06:13:35 +09:00
Hare 60eabea19a fix: assert retained remote session close diagnostics 2026-10-21 06:13:16 +09:00
Hare 7394ec6784 fix: retain safe Workdir open authorization and response failure causes
Keep bounded reasons and derived failure stages separate from public classification. Reject provider-authored raw error text on transport decoding. Preserve identity mismatch when cleanup fails and retain its handle for explicit close without opening again. Add signed Runtime HTTP, Registry/API logging, remote mismatch, authorization no-dispatch and deferred fault coverage. Focused tests and Workdir suite pass; full-source validation follows T773 audit contract merge.
2026-10-21 06:13:07 +09:00
Hare 28b7cc414b fix: retain external workdir results independently of audit failures 2026-10-21 06:04:39 +09:00
Hare 86d5cf9403 fix: integrate safe Drive and config failure diagnostics 2026-10-21 06:01:51 +09:00
Hare 6f0a23e5cc Merge commit '62e2b4430588bacf90a2812a0a88772fc0a00c58' into work/T-775-cleanup-diagnostics 2026-10-21 05:56:16 +09:00
Hare 2f4a46bbdb Merge commit 'a1748b552c52fd5f68de6ba852421fd778aec010' into work/T-775-cleanup-diagnostics 2026-10-21 05:55:48 +09:00
Hare 4872cf2aac fix: retain independent Bash Drop cleanup diagnostics 2026-10-21 05:55:48 +09:00
Hare 734d4b44ae Merge remote-tracking branch 'origin/work/T-772-command-output-integrity' into work/T-774-workdir-failure-diagnostics 2026-10-21 05:55:33 +09:00
Hare 6810f51f46 fix: retain safe Drive and config adapter failure diagnostics 2026-10-21 05:53:58 +09:00
Hare a1748b552c fix: validate closed-session diagnostics and fault quotas 2026-10-21 05:52:48 +09:00
Hare 62b2d27b4a fix: retain command failure stages and output collection state 2026-10-21 05:40:39 +09:00
Hare 62e2b44305 fix: retain safe Journal compaction failure diagnostics
Keep stage/reason classification in the Journal Feature without changing frozen input, metadata or fallback behavior. Exercise connection, Host read failures, size budget and secret-bearing fixtures through compaction persistence.

Validation: old freeze function with new tests fails connection and snapshot/size diagnostics (3 failures); new code passes all six selected cases and cargo test -p worker. Serialization is currently unreachable for the derived JSON-safe snapshot fields; no artificial runtime failure is claimed.
2026-10-21 05:32:07 +09:00
Hare aa0fb2f787 fix: continue workdir output reads after long-poll expiry 2026-10-21 04:48:43 +09:00
Hare 1b6d4dbfd8 fix: preserve workdir failure diagnostics across transport 2026-10-21 01:22:04 +09:00
Hare 4ef7b03651 fix: stop background session checks and loading flicker 2026-10-20 22:35:38 +09:00
Hare af615ab177 fix: retain archive and worker restore failure diagnostics 2026-10-20 22:00:58 +09:00
Hare 338e39bcdb fix: defer restored workdir access until use 2026-10-20 22:00:58 +09:00
Hare 3cfe04be3a fix: preserve saved feature configuration during worker restore 2026-10-20 22:00:39 +09:00
Hare 70d153b365 fix: isolate unreadable archives from runtime startup 2026-10-20 17:39:38 +09:00
Hare ce93849948 chore: merge develop into hare/develop 2026-10-20 15:39:09 +09:00
Hare 3c499719fe feat: default Coder ticket work to review flow 2026-10-20 15:38:50 +09:00
Hare 9e1d3df7ad fix: remove overbroad Worker bypass instructions 2026-10-20 15:38:50 +09:00
Hare 18bac16a23 fix: require replay-safe control lifecycle for Subject delegation 2026-10-20 08:12:26 +09:00
Hare 9005856f77 fix: replay accepted delegation without reserving active Workdirs 2026-10-20 07:24:51 +09:00
Hare e39372a52a Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants 2026-10-20 06:38:12 +09:00
Hare 223a74a9e2 fix: retain byte command transport and explicit authorization fixtures 2026-10-20 06:38:12 +09:00
Hare f38d9a16ea Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants
# Conflicts:
#	crates/worker-runtime/src/worker_backend.rs
#	openapi/server-api.json
#	web/workspace/src/lib/generated/companion-api.ts
#	web/workspace/src/lib/generated/repository-api.ts
#	web/workspace/src/lib/generated/runtime-api.ts
#	web/workspace/src/lib/generated/ticket-api.ts
#	web/workspace/src/lib/generated/worker-launch-api.ts
2026-10-20 06:24:11 +09:00
Hare b135fa5e99 feat: enforce explicit Subject permissions and delegated Worker grants 2026-10-20 06:22:58 +09:00
Hare 07b8b6d41b Merge commit 'b8e548b92d3c785c482dbb418517cc5a2cf9e7e0' into refactor/T-765-session-feature-boundary 2026-10-20 05:34:22 +09:00
Hare 8017c55d82 fix: retain annotated compaction history and honor non-System input policy 2026-10-20 05:34:22 +09:00
Hare b8e548b92d chore: integrate reviewed browser session navigation 2026-10-20 05:31:32 +09:00
Hare 90e38551cf Merge commit '00ad02d04728a8e32a82a4e72747fefcf8a82d4e' into refactor/T-765-session-feature-boundary
# Conflicts:
#	crates/worker/src/feature/builtin.rs
2026-10-20 05:25:16 +09:00
Hare 3e8164e757 fix: fence hosting setup verification against superseded sessions 2026-10-20 04:53:43 +09:00
Hare 3b3381900f fix: preserve role-neutral input policy across compaction and replay 2026-10-20 04:45:50 +09:00
Hare 00ad02d047 chore: integrate bounded shared text page refusal 2026-10-20 03:55:13 +09:00
Hare a774925dec fix: reject byte-truncated line windows before advancing search 2026-10-20 03:53:03 +09:00
Hare 8f759911ab fix: emit Session use policy type in browser bindings 2026-10-20 03:52:55 +09:00
Hare daf1f6ed50 fix: route retained Session migration through Runtime staging 2026-10-20 03:52:55 +09:00
Hare 86602240fa refactor: bind Subject metadata and prompt confirmation to Host lifecycle 2026-10-20 03:52:55 +09:00
Hare bfeee487fd feat(auth): separate anonymous entry from expired session recovery 2026-10-20 03:52:48 +09:00
Hare f9830fe7dd test: reject Bash search pages that would skip an unread long line 2026-10-20 03:40:36 +09:00
Hare de96c5b577 chore: update shared file contracts and validation fixtures
# Conflicts:
#	crates/worker/src/checkout_http_tests.rs
2026-10-20 03:26:56 +09:00
Hare 6e729ed06c refactor: isolate Feature System input migration and compaction policy 2026-10-20 03:21:22 +09:00
Hare f38b2a52b1 fix: record native prior Read only after result validation 2026-10-20 03:21:14 +09:00
Hare dd5384df3b feat: expose retained Bash streams through common WIP file contracts 2026-10-20 03:13:46 +09:00
Hare 6576850b7a chore: connect shared file contracts for Bash validation 2026-10-20 03:03:53 +09:00
Hare 4555aace28 feat: capture bounded Bash output over replayable byte transport 2026-10-20 03:01:22 +09:00
Hare 47b32ef6ab fix: bind capability changes to Object freshness and update contract consumers 2026-10-20 02:53:21 +09:00
Hare bb5c50f9ab feat: persist System input policies through public consumers 2026-10-20 02:44:28 +09:00
Hare 32a409c15d feat: share virtual file contracts and checked binary byte ranges 2026-10-20 02:18:49 +09:00
Hare 19514a5d6a feat: define bounded Feature-owned system input envelopes 2026-10-20 02:14:52 +09:00
Hare 887184796e docs: distinguish commit completion from Feature authority 2026-10-20 02:05:02 +09:00
Hare cd0eb5eb00 fix: allowlist public system input projections 2026-10-20 02:02:28 +09:00
Hare e8dfb3e68d refactor: notify system input producers after durable commit 2026-10-20 02:02:28 +09:00
Hare 434ce34da7 refactor: bind checkout target lifetimes to Object observations 2026-10-20 02:01:22 +09:00
Hare bfb51b7ca9 fix: bound encoded common text results including escape and record overhead 2026-10-20 01:48:23 +09:00
Hare 38d9069632 feat: publish stable root file contracts and target-owned dispatch 2026-10-20 01:46:30 +09:00
Hare e5749b9be0 test: integrate Drive connection and lifetime coverage 2026-10-20 00:37:32 +09:00
Hare 387b5ff7df docs: record Drive connection coverage and fault evidence 2026-10-20 00:35:11 +09:00
Hare ec6c5beda0 test: connect Drive publication to client and resource lifetimes 2026-10-20 00:34:50 +09:00
Hare 8e40fae6f4 fix: recover unpublished Subject creation and preserve auth rejections 2026-10-19 18:56:24 +09:00
Hare c961f244ab fix: preserve feature metadata for concise Journal startup context 2026-10-19 18:45:32 +09:00
Hare cea0225a6e fix: migrate legacy session history and retained archives 2026-10-19 13:52:43 +09:00
Hare b3189d8e85 fix: preserve worker retention failure diagnostics 2026-10-19 13:52:43 +09:00
Hare 24bc45a31e chore: merge develop into hare/develop 2026-10-19 01:31:23 +09:00
Hare b02360fd75 fix: restore orchestrator target push requirement 2026-10-19 01:31:01 +09:00
Hare 41c9d2d5c7 docs: record restored parent regression evidence and cost 2026-10-19 00:02:02 +09:00
Hare cc66bd5052 test: connect restored Runtime parent tools to child session 2026-10-19 00:02:02 +09:00
Hare d1408d3a2f fix: project runtime list subjects for Worker control tools 2026-10-19 00:02:02 +09:00
Hare 19122fb5ea fix: use public Worker control entry in regression design 2026-10-19 00:02:02 +09:00
Hare 1fb216e6ae fix: refresh Ticket regression design evidence 2026-10-19 00:02:02 +09:00
Hare 4b14f7e6b3 docs: design prioritized regression coverage 2026-10-19 00:02:02 +09:00
Hare 67767eeee7 test: integrate Ticket public regression coverage 2026-10-18 02:35:16 +09:00
Hare 1bfb4ce119 fix: align Ticket regression design with Host retry identity 2026-10-18 02:10:15 +09:00
Hare 651820dc6d test: connect Ticket public regressions to real Backend 2026-10-18 02:10:15 +09:00
Hare 75b1557d63 feat: browse Subject Journals and persistent Memory links 2026-10-18 02:03:51 +09:00
Hare 23ff518209 fix: clarify Ticket regression fixture boundaries 2026-10-18 01:26:52 +09:00
Hare fab9e0f4b5 docs: design focused Ticket regression coverage 2026-10-18 01:26:52 +09:00
Hare 5ee42b8db8 docs: record Journal compaction validation and evaluation limits 2026-10-17 12:29:47 +09:00
Hare daf438701f docs: explain shared Journal snapshot boundary guarantees 2026-10-17 12:25:24 +09:00
Hare 23ee401287 fix: disambiguate historical Journal retention instructions 2026-10-17 12:25:24 +09:00
Hare 8f598da931 test: verify frozen Journal compaction across Hosts and restores 2026-10-17 12:25:24 +09:00
Hare 956aeed6b2 feat: freeze Host Journal context across compaction 2026-10-17 12:19:37 +09:00
Hare adfa422973 chore: merge develop into hare/develop 2026-10-17 12:06:05 +09:00
Hare 10d200d9ba chore: align Journal references with integrated develop 2026-10-17 03:52:11 +09:00
Hare 7a5228b8e6 fix: reconcile Memory Journal links with handoff fencing 2026-10-17 03:52:11 +09:00
Hare 0eabdf3a1f chore: integrate Journal-gated Subject handoff 2026-10-17 03:47:11 +09:00
Hare b1808b7c4d fix: unblock MR follow-up and simplify orchestration guidance 2026-10-17 03:35:58 +09:00
Hare 6645ea0a47 fix: observe original presence through saved legacy Journal links 2026-10-17 00:10:35 +09:00
Hare b544e53bd7 feat: connect Memory provenance to durable Journal context 2026-10-16 23:54:54 +09:00
Hare 04a4e59100 feat: fence Subject handoff on recorded Journal and seed new bodies once 2026-10-16 20:51:08 +09:00
Hare 52e914eebc feat: offer ID-free passkey login and explicit legacy migration 2026-10-16 19:01:43 +09:00
Hare 9c12fb4372 feat: verify discoverable passkeys against their saved owners 2026-10-16 19:01:43 +09:00
Hare 3c18916118 test: wait for complete isolated server address records 2026-10-16 19:01:43 +09:00
Hare be1c030350 chore: merge develop into hare/develop 2026-10-16 15:08:40 +09:00
Hare a7faf2cea6 docs: record test optimization results and runner validation 2026-10-16 15:04:52 +09:00
Hare 5cc5ece997 feat: add bounded parallel runner for the normal test suite 2026-10-16 15:04:52 +09:00
Hare ffaaf2ecb0 test: isolate local rules and reduce repeated fixture setup 2026-10-16 15:04:52 +09:00
Hare 6667985883 fix: preserve test completion and fixture invariants under concurrency 2026-10-16 15:04:42 +09:00
Hare 22dcd8a29f fix: make reverse session reads linear for large records 2026-10-16 15:04:42 +09:00
Hare a2816a053d fix: isolate Workspace notifications from public entry 2026-10-16 11:30:00 +09:00
Hare d74660d1d1 fix: retain workspace creation state across session rechecks 2026-10-16 11:08:27 +09:00
Hare 24c6f772f7 fix: serve entry shells independently of workspace catalog 2026-10-16 10:48:51 +09:00
Hare a060ffada8 feat: add public entry and authenticated workspace dashboard 2026-10-16 10:48:51 +09:00
Hare b8b12442a0 feat: separate login registration and own account flows 2026-10-16 09:43:57 +09:00
Hare d22899d39e feat: activate provisional users with scoped passkey enrollment 2026-10-16 08:48:35 +09:00
Hare af008f802d feat: explicit hosting administrators and persistent self-registration policy 2026-10-16 07:26:21 +09:00
Hare 7042ec7d50 chore: merge develop into hare/develop 2026-10-16 05:52:36 +09:00
Hare a2cec5afd1 fix: preserve console alert order and improve alert colors 2026-10-16 05:28:17 +09:00
Hare b5a307811b fix: reconcile erased Archive removals and exercise live compaction recording
Reconcile immutable Runtime removal input under a confirmed permanent erase fence. A confirmed source_removed receipt is audited without republishing erased archives; unknown receipt plus attested remaining aggregate permits an audited stale plan and proof-bound purge replacement. Unknown receipt and inventory remain unresolved. Distinguish domain tombstones from scoped Runtime physical erasure acknowledgements.

Erase diagnostic originals and interrupted staging bound to the same archived Session/removal operation, validating Workspace/Worker/fingerprint and saved receipts without rewriting their historical retained flags. Finalize Backend removal only after physical acknowledgement, including previous Retain disposition.

Connect public standalone protocol, real Worker request-threshold compaction twice within one turn, cancellation and real pre-AI input rollback, original canonical readers, existing Jobs and sequential Journal tools. Gate generation while compaction completes; verify all durable originals were served, summary/rolled-back input exclusion, exact final cursor, no extra input for successor drain, and one ongoing open entry. External model streams/usage are scripted, not semantic evaluation.

Validation: Backend1112lib+11bin (existing ignore1), Runtime429lib+38bin, standalone72lib+27integration (existing model ignore1), runtime-api10, root check/doc/fmt/diff/naming, Server/Runtime builds, isolated worker_environment2 passed. Earlier broad suites remain recorded on the prior source. Fixtures and injected boundaries are not arbitrary crash/remote-transport-loss coverage; worker_environment only verifies startup/connection/cleanup.

Acceptance clarification: T-734 assignment answer event22 explicitly defers real-model semantic condition6 to user use, excluding it from this completion scope. It remains unexecuted and unevaluated, not passed. Technical review findings still require a fresh independent verdict on this source.
2026-10-16 04:58:53 +09:00
Hare 5bd49760e5 fix: fence Journal originals before publication and reconcile removal intent
Protect attributed originals before the first public commit; distinguish pending Session material from missing committed sources. Gate removal admission by the Journal Feature and resolve Subject ownership from durable singleton authority. Resume explicit erasure through signed delete transport and proof-bound purge plans without creating a Journal Job or archive.

Validation: changed-crate suites, root check, doc/fmt/diff/naming checks, Server/Runtime builds and worker_environment isolation suite passed. Public standalone scripted dialog connects Host, automatic Job, sequential tools and drain while dialog continues. Backend public embedded WorkerRemove tests connect removal forwarding, durable Jobs, retention, Runtime and SQLite; Session/attribution/profile fixture setup bypasses creation/auth routes. Runtime signed generated-client tests separately exercise request authentication.

Limits: real-model semantic acceptance condition 6 remains unevaluated without a new explicit model/credential/budget grant. Conditions 1/7/8/11 do not have complete product E2E coverage: worker_environment only verifies isolated startup/connection/cleanup. Compaction tests connect local hooks/reader/store rather than the full public dialog-to-Journal chain. Erasure racing a previously prepared Archive removal plan fails closed with conflict; automatic stale-plan replacement is not implemented.
2026-10-16 03:59:45 +09:00
Hare 9debf43645 Merge remote-tracking branch 'origin/develop' into work/T-734-journal-recording 2026-10-16 02:19:24 +09:00
Hare 81af1f085c feat: record confirmed public Session history through durable Journal Jobs 2026-10-16 02:19:24 +09:00
Hare b67aa2de7d fix: simplify Ticket reporting and speed up browser lists
Remove model-supplied assignment and retry identities from implementation reports; retain Host idempotency, authentication and notification evidence.

Add a SQL-only browser projection and decouple auxiliary page loads. Preserve authoritative MR evidence queries. Include regression tests and component benchmarks, documenting the remaining no-match search cost.
2026-10-16 01:58:15 +09:00
Hare 9f1cbbd385 fix: inherit parent profile for default subworkers 2026-10-16 01:57:58 +09:00
Hare 8a3d9d67d2 fix: guard deferred state when leaving workspace routes 2026-10-16 01:57:58 +09:00
Hare 6218c27fb6 fix: separate account registration from authenticated passkey addition
Bind enrollment purpose and allocated/current account to verifier state. Reject existing-handle registration, derive additions from live transport authentication, recheck revocation/expiry in the insert-only transaction, and consume challenges atomically across connections. Derive stable account WebAuthn handles without rewriting legacy credentials; reject implicit-user login and unsafe legacy ceremonies.

Align account callers, DTO/OpenAPI/generated artifacts and migration/recovery rationale. Preserve credential-less legacy identities rather than allowing anonymous reclaim.

Validation: 14 actual-verifier passkey tests; full workspace-server 1086 library + 11 binary passed (Drive ignored case explicitly selected separately and passed); server-api default 99 and TypeScript feature 108 passed; isolated auth/Runtime E2E 1+2 passed; Web 645 + component 394 passed; root cargo check, docs/doctests, Server/Runtime builds, fmt, naming and generated consistency passed. Software authenticator only; hosted CI/hardware login not claimed.
2026-10-16 00:55:16 +09:00
Hare b744efd104 fix: align isolated runtime harness with current trust contract
Keep Workspace trust enrollment and owner-authenticated registration, then verify each Runtime connection with a fresh signed request and match its binding ID. Remove obsolete workspace_trust_id input and cached connection_state assertions.

Validation: cargo test -p yoi-e2e --features e2e --test worker_environment --test auth_passkeys -- --nocapture (2 Runtime harness tests and 1 Passkey product E2E passed); root cargo check.
2026-10-16 00:35:23 +09:00
Hare 18384482f6 feat: integrate durable Journal storage APIs 2026-10-15 20:51:51 +09:00
Hare c2df3041f9 fix: pin Journal source pagination to saved coverage content 2026-10-14 22:18:02 +09:00
Hare aabc502f30 feat: add durable subject Journal operations and bounded reads 2026-10-14 21:57:35 +09:00
510 changed files with 114524 additions and 12508 deletions
Generated
+12 -2
View File
@@ -5266,6 +5266,7 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"tracing",
"uuid",
"workdir",
]
@@ -6496,6 +6497,7 @@ name = "workdir"
version = "0.1.0"
dependencies = [
"async-trait",
"axum",
"fs-operation",
"libc",
"manifest",
@@ -6519,7 +6521,6 @@ dependencies = [
"axum",
"base64 0.22.1",
"chrono",
"clap",
"client",
"config-source",
"dotenv",
@@ -6548,6 +6549,7 @@ dependencies = [
"session-metrics",
"session-store",
"sha2 0.11.0",
"subjektiv",
"tempfile",
"thiserror 2.0.18",
"ticket",
@@ -6556,6 +6558,7 @@ dependencies = [
"toml",
"tools",
"tracing",
"tracing-subscriber",
"tungstenite 0.28.0",
"uuid",
"wip-client",
@@ -6595,6 +6598,7 @@ dependencies = [
"tokio",
"tokio-tungstenite 0.29.0",
"toml",
"tools",
"tower",
"tracing",
"tracing-subscriber",
@@ -6682,6 +6686,7 @@ dependencies = [
"futures",
"manifest",
"project-record",
"protocol",
"reqwest",
"serde",
"serde_json",
@@ -6697,19 +6702,23 @@ dependencies = [
"tui",
"uuid",
"workdir",
"worker",
]
[[package]]
name = "yoi-e2e"
version = "0.0.0"
dependencies = [
"chrono",
"libc",
"reqwest",
"serde",
"serde_json",
"server-api",
"sha2 0.11.0",
"tempfile",
"webauthn-authenticator-rs",
"webauthn-rs-proto",
"yoi-workspace-server",
]
[[package]]
@@ -6770,6 +6779,7 @@ dependencies = [
"uuid",
"webauthn-authenticator-rs",
"webauthn-rs",
"webauthn-rs-proto",
"wip-protocol",
"workdir",
"worker",
+10
View File
@@ -91,6 +91,16 @@ debug = "line-tables-only"
[profile.dev.package."*"]
debug = false
# Keep the real SQLite engine in schema/transaction tests without paying for
# an unoptimized C interpreter on every statement. Rust test assertions stay enabled.
[profile.test.package.libsqlite3-sys]
opt-level = 2
# Schema/authority integration tests execute substantial parsing and validation.
# Optimize that code without disabling test/debug assertions or overflow checks.
[profile.test.package.yoi-workspace-server]
opt-level = 1
[workspace.dependencies]
# Internal crates
client = { path = "crates/client" }
+42 -9
View File
@@ -1,13 +1,38 @@
//! Compile the real public macro through a renamed dependency in an isolated consumer.
//! Only Cargo artifacts are shared; Cargo checks the fresh consumer on every invocation.
use std::{fs, path::PathBuf, process::Command};
struct Fixture(PathBuf);
impl Fixture {
fn new() -> Self {
for attempt in 0..1000 {
let path = std::env::temp_dir().join(format!(
"api-macros-renamed-dependency-{}-{attempt}",
std::process::id()
));
match fs::create_dir(&path) {
Ok(()) => return Self(path),
Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue,
Err(error) => panic!("create renamed dependency fixture: {error}"),
}
}
panic!("could not allocate a renamed dependency fixture directory");
}
}
impl Drop for Fixture {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
#[test]
fn macro_expansion_resolves_a_renamed_support_dependency() {
let fixture = std::env::temp_dir().join(format!(
"api-macros-renamed-dependency-{}",
std::process::id()
));
let _ = fs::remove_dir_all(&fixture);
fs::create_dir_all(fixture.join("src")).expect("create fixture source directory");
let fixture = Fixture::new();
let fixture = &fixture.0;
fs::create_dir(fixture.join("src")).expect("create fixture source directory");
let package_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let package_path = package_path
@@ -49,13 +74,21 @@ pub fn operation_count() -> usize {
)
.expect("write fixture source");
// Keep nested Cargo checks off the outer build's target lock. Cargo owns cache
// invalidation and locking between concurrent consumers, not a saved
// test-success marker.
let executable = std::env::current_exe().expect("current test executable");
let target_dir = executable
.parent()
.and_then(|deps| deps.parent())
.expect("test executable has a profile directory")
.join("compile-tests/api-macros-renamed");
let output = Command::new(env!("CARGO"))
.args(["check", "--quiet", "--offline"])
.current_dir(&fixture)
.env("CARGO_TARGET_DIR", fixture.join("target"))
.current_dir(fixture)
.env("CARGO_TARGET_DIR", target_dir)
.output()
.expect("run cargo check for renamed dependency fixture");
let _ = fs::remove_dir_all(&fixture);
assert!(
output.status.success(),
+11 -1
View File
@@ -81,6 +81,16 @@ pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
resolved_path: "profiles/job.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:subjektiv-journal-recording"),
path: "profiles/subjektiv-journal-recording.dcdl",
source: include_str!("../../../resources/profiles/subjektiv-journal-recording.dcdl"),
description: "Journal recording policy with Host-bound sequential Job tools.",
imports: &[BuiltinProfileImport {
specifier: "./job.dcdl",
resolved_path: "profiles/job.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:ticket-worker"),
path: "profiles/ticket-worker.dcdl",
@@ -123,7 +133,7 @@ pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
selector: Some("builtin:orchestrator"),
path: "profiles/orchestrator.dcdl",
source: include_str!("../../../resources/profiles/orchestrator.dcdl"),
description: "Intent-led Ticket orchestration with separate guarded MR integration authority.",
description: "Outcome-led orchestration, selection of candidate changes, and risk-proportionate validation.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
+1
View File
@@ -908,6 +908,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let response = create_backend_worker_with_client(&target, &create, &api)
+1
View File
@@ -567,6 +567,7 @@ impl BackendWorkspaceProductClient {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let workspace_id = self.workspace_id.clone();
let response: BrowserCreateWorkerResponse = self.generated(move |client| async move {
+30
View File
@@ -200,6 +200,7 @@ fn run_read_with_limits(
total_lines,
content_hash: content_hasher.finalize().into(),
truncated: end < total_lines || byte_truncated,
byte_truncated,
})
}
@@ -885,6 +886,34 @@ mod bounded_read_tests {
assert!(error.to_string().contains("exceeds provider limit 4096"));
}
#[test]
fn bounded_read_distinguishes_utf8_byte_cut_from_complete_line_continuation() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
fs::write(root.join("text"), "雪雪needle\nnext").unwrap();
for (offset, max_bytes, bytes, more, cut) in [
(0, 5, "雪", true, true),
(0, 64, "雪雪needle\n", true, false),
(1, 64, "next", false, false),
] {
let result = run_read_bounded(
&root,
ReadRequest {
path: FsPath::new("text").unwrap(),
offset,
limit: 1,
max_bytes,
},
&RootAccess(root.clone()),
BoundedReadLimits::new(1024, 64).unwrap(),
)
.unwrap();
assert_eq!(result.bytes, bytes.as_bytes());
assert_eq!(result.truncated, more);
assert_eq!(result.byte_truncated, cut);
}
}
#[test]
fn bounded_read_caps_retained_response_independently_of_request() {
let temp = tempfile::tempdir().unwrap();
@@ -907,6 +936,7 @@ mod bounded_read_tests {
assert_eq!(result.bytes, b"alpha");
assert_eq!(result.total_lines, 3);
assert!(result.truncated);
assert!(result.byte_truncated);
}
}
+4
View File
@@ -204,7 +204,11 @@ pub struct ReadResult {
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub total_lines: usize,
pub content_hash: ContentHash,
/// More source lines remain, or the selected window exceeded the response byte budget.
pub truncated: bool,
/// The selected line window was cut by its byte budget, possibly within a line.
/// Line-based cursors must not treat such a window as completely scanned.
pub byte_truncated: bool,
}
/// Binary-safe byte range read with an optional whole-file version fence.
+1 -38
View File
@@ -100,21 +100,6 @@ pub fn worker_allocation_path() -> Option<PathBuf> {
worker_allocation_path_from_runtime_dir(runtime_dir())
}
/// `<runtime_dir>/<worker_name>/` — per-Worker runtime directory.
pub fn worker_runtime_dir(worker_name: &str) -> Option<PathBuf> {
worker_runtime_dir_from_runtime_dir(runtime_dir(), worker_name)
}
/// `<runtime_dir>/<worker_name>/sock` — Worker's Unix socket path.
///
/// The `worker` crate's `RuntimeDir::socket_path()` creates the socket inside the
/// Worker process. This function **predicts** the path for external callers that
/// know the Worker name, such as the TUI attach flow. Both should return the same
/// path.
pub fn worker_socket_path(worker_name: &str) -> Option<PathBuf> {
worker_socket_path_from_runtime_dir(runtime_dir(), worker_name)
}
// ---- internals --------------------------------------------------------------
fn resolve_config_dir_from_parts(
@@ -204,20 +189,6 @@ fn worker_allocation_path_from_runtime_dir(runtime_dir: Option<PathBuf>) -> Opti
Some(runtime_dir?.join("workers.json"))
}
fn worker_runtime_dir_from_runtime_dir(
runtime_dir: Option<PathBuf>,
worker_name: &str,
) -> Option<PathBuf> {
Some(runtime_dir?.join(worker_name))
}
fn worker_socket_path_from_runtime_dir(
runtime_dir: Option<PathBuf>,
worker_name: &str,
) -> Option<PathBuf> {
Some(worker_runtime_dir_from_runtime_dir(runtime_dir, worker_name)?.join("sock"))
}
/// Treats empty environment values as unset. `std::env::var` distinguishes
/// `Ok("")` from `Err(NotPresent)`, but path resolution treats both as unset
/// for intuitive behavior.
@@ -455,16 +426,8 @@ mod tests {
PathBuf::from("/sand/sessions")
);
assert_eq!(
worker_allocation_path_from_runtime_dir(runtime_dir.clone()).unwrap(),
worker_allocation_path_from_runtime_dir(runtime_dir).unwrap(),
PathBuf::from("/sand/run/workers.json")
);
assert_eq!(
worker_runtime_dir_from_runtime_dir(runtime_dir.clone(), "foo").unwrap(),
PathBuf::from("/sand/run/foo")
);
assert_eq!(
worker_socket_path_from_runtime_dir(runtime_dir, "foo").unwrap(),
PathBuf::from("/sand/run/foo/sock")
);
}
}
+2 -1
View File
@@ -1746,7 +1746,7 @@ mod tests {
}
#[test]
fn builtin_orchestrator_keeps_cleanup_tool_providers_enabled() {
fn builtin_orchestrator_enables_reviewer_handoff_and_cleanup_without_self_review() {
let tmp = TempDir::new().unwrap();
let resolved = ProfileResolver::new()
.with_workspace_base(tmp.path())
@@ -1757,6 +1757,7 @@ mod tests {
.unwrap();
let feature = resolved.manifest.feature;
assert!(feature.sub_worker.enabled);
assert!(feature.worker.enabled);
assert!(!feature.worker.direct_spawn);
assert!(feature.manage_workdir.enabled);
+6 -6
View File
@@ -826,7 +826,7 @@ impl MergeRequestStore {
));
}
let mr = self.get_for_operation(&i.auth.workspace_id, &i.merge_request_id, &i.ticket_id)?;
self.assigned(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
if mr.state == MergeRequestState::Closed
|| (mr.state == MergeRequestState::Open && mr.selector_from.is_none())
{
@@ -1035,7 +1035,7 @@ impl MergeRequestStore {
) -> Result<ReviewRevokedEvent, MergeRequestError> {
bounded_body("revocation reason", &i.reason)?;
let mr = self.get_for_operation(&i.auth.workspace_id, &i.merge_request_id, &i.ticket_id)?;
self.assigned(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
let r = mr
.thread
.iter()
@@ -1152,7 +1152,7 @@ impl MergeRequestStore {
"operation fingerprint mismatch".into(),
));
}
self.completion_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
if mr.state != MergeRequestState::Open {
return Err(MergeRequestError::Conflict(
"Merge Request is not open".into(),
@@ -1186,7 +1186,7 @@ impl MergeRequestStore {
));
}
// Case openness is not an integration phase. Exact source approval,
// current Ticket content and active assignment fencing above remain
// current Ticket content and current assignment fencing above remain
// the independent MR integration guards.
Ok(())
}
@@ -1229,7 +1229,7 @@ impl MergeRequestStore {
}
let assignment_is_current: bool = transaction.query_row(
"SELECT EXISTS(
SELECT 1 FROM ticket_active_worker_assignments
SELECT 1 FROM ticket_current_worker_assignments
WHERE workspace_id=?1 AND ticket_id=?2 AND assignment_id=?3
)",
params![i.auth.workspace_id, i.ticket_id, i.auth.assignment_id],
@@ -1548,7 +1548,7 @@ impl MergeRequestStore {
}
Ok(())
}
fn completion_auth(
fn current_assignment_auth(
&self,
a: &MergeRequestAuth,
t: &str,
+64 -11
View File
@@ -1,3 +1,6 @@
//! Check the shipped templates against the real PDK and WIT for the guest target.
//! Each check owns its source and lockfile; Cargo alone shares and validates artifacts.
use std::fs;
use std::path::Path;
use std::process::Command;
@@ -135,36 +138,86 @@ fn embedded_service_template_cargo_checks_for_wasm_target() {
fn cargo_check_template(template: &str) {
let crate_dir = Path::new(env!("CARGO_MANIFEST_DIR"));
let template_dir = crate_dir.join(format!("../../resources/plugin/templates/{template}"));
let manifest_path = template_dir.join("Cargo.toml");
let lock_path = template_dir.join("Cargo.lock");
let _ = fs::remove_file(&lock_path);
let templates_dir = crate_dir.join("../../resources/plugin/templates");
let template_dir = templates_dir.join(template);
let target_dir = tempfile::tempdir().expect("temporary cargo target dir");
let cargo = std::env::var("CARGO").unwrap_or_else(|_| "cargo".to_string());
let output = Command::new(cargo)
// A sibling fixture preserves the shipped relative PDK and WIT paths verbatim.
// Never create or remove a lockfile in the shipped template: another invocation
// may be checking it at the same time, or a developer may own that lockfile.
let fixture = tempfile::Builder::new()
.prefix(".cargo-check-")
.tempdir_in(&templates_dir)
.expect("temporary template fixture");
copy_template_tree(&template_dir, fixture.path(), true);
// Both templates share the PDK dependencies. A separate persistent target
// reuses those artifacts without sharing source or the outer Cargo build lock.
// Cargo fingerprints and its target lock handle changes and concurrent checks.
let executable = std::env::current_exe().expect("current test executable");
let target_dir = executable
.parent()
.and_then(|deps| deps.parent())
.expect("test executable has a profile directory")
.join("compile-tests/plugin-pdk-templates");
let output = Command::new(env!("CARGO"))
.arg("check")
.arg("--manifest-path")
.arg(&manifest_path)
.arg(fixture.path().join("Cargo.toml"))
.arg("--target")
.arg("wasm32-unknown-unknown")
.arg("--offline")
.arg("--target-dir")
.arg(target_dir.path())
.arg(target_dir)
.current_dir(fixture.path())
.env("CARGO_TERM_COLOR", "never")
.output()
.expect("spawn cargo check for embedded template");
let _ = fs::remove_file(&lock_path);
assert!(
output.status.success(),
"template cargo check failed\nstatus: {}\nstdout:\n{}\nstderr:\n{}",
"template {template} cargo check failed\nstatus: {}\nstdout:\n{}\nstderr:\n{}",
output.status,
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
}
fn copy_template_tree(source: &Path, destination: &Path, is_root: bool) {
let metadata = fs::symlink_metadata(source).expect("read template directory metadata");
assert!(
metadata.is_dir() && !metadata.file_type().is_symlink(),
"template input must be a real directory: {}",
source.display()
);
for entry in fs::read_dir(source).expect("read template directory") {
let entry = entry.expect("read template entry");
let path = entry.path();
let kind = entry.file_type().expect("read template entry type");
assert!(
!kind.is_symlink() && (kind.is_dir() || kind.is_file()),
"unsupported template input (including symlinks): {}",
path.display()
);
let name = entry.file_name();
// Preserve all compiler inputs, including build scripts and nested assets.
// Only omit the root build output and lockfile: each isolated consumer must
// resolve its own lockfile, as the original unlocked template check did.
if is_root
&& ((kind.is_dir() && name == "target") || (kind.is_file() && name == "Cargo.lock"))
{
continue;
}
let copied = destination.join(name);
if kind.is_dir() {
fs::create_dir(&copied).expect("create template fixture directory");
copy_template_tree(&path, &copied, false);
} else {
fs::copy(&path, &copied).expect("copy unmodified template input");
}
}
}
#[test]
fn pdk_runtime_dependencies_are_guest_side_only() {
let cargo: Value = toml::from_str(PDK_CARGO).expect("PDK Cargo.toml parses");
+81 -26
View File
@@ -52,7 +52,7 @@ pub enum AuthenticatedInputSource {
/// Assigned whenever a serialized tracked method crosses an untrusted
/// protocol boundary. Receivers must handle it exactly like public input.
UntrustedWire,
/// Stamped only by the operator-owned standalone in-process transport.
/// Stamped only by the operator-owned Standalone Host after local admission.
/// This is not a Backend Account or authority accepted from wire fields.
LocalOperator,
Account {
@@ -399,16 +399,13 @@ pub enum Method {
/// Typed lifecycle events sent from a child Worker to its parent.
///
/// Delivered as `Method::WorkerEvent` over the parent's Unix socket. The
/// parent Controller always applies variant-specific side effects
/// (registry / worker-allocation updates). Agent-visible variants are also
/// queued into the notification buffer; control-plane-only variants are
/// not injected into the parent's LLM context.
/// Delivered in-process to the parent Controller's protocol intake. Agent-visible variants
/// are queued into the notification buffer. Reports are advisory: receivers must check the
/// actual child session before treating its work as complete.
///
/// Transport is fire-and-forget; receivers must tolerate out-of-order
/// delivery (e.g. `TurnEnded` arriving after `ShutDown` for the same
/// child Worker).
#[derive(Debug, Clone, Serialize, Deserialize)]
/// Retained legacy control-only variants can be decoded from saved history, but do not
/// register live children, delegate scope, or establish a socket callback.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
@@ -429,16 +426,8 @@ pub enum WorkerEvent {
/// Child has stopped (controller loop is exiting).
ShutDown { worker_name: String },
/// Child SubWorker sub-delegated scope to a grandchild SubWorker via `SubWorkerSpawn`.
///
/// Control-plane only: receivers apply registry side effects and
/// propagate upward, but do not expose this as an agent notification.
///
/// The parent uses this to add the grandchild to its own
/// `spawned_workers.json` so it can manage the grandchild directly
/// even if the intermediate child dies. The parent then re-fires
/// this event upward (if it has a parent of its own) to maintain
/// the chain to root.
/// Legacy process-child scope report retained for saved Session decoding only.
/// Live Controllers ignore it; only the Host-owned Internal registry grants child scope.
ScopeSubDelegated {
/// Sub-delegating Worker (= the sender itself).
parent_worker: String,
@@ -982,6 +971,17 @@ pub enum SessionEntryProvenance {
LegacyUnknown,
}
/// Public use decisions contain no Feature identity or private payload. Absence
/// denotes the older projection contract, not permission to expose System input.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(deny_unknown_fields)]
pub struct SessionEntryUsePolicy {
pub observation_readable: bool,
pub memory_evidence: bool,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -994,6 +994,8 @@ pub struct SessionSnapshotEntry {
pub provenance: SessionEntryProvenance,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub derived_from: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub use_policy: Option<SessionEntryUsePolicy>,
#[serde(flatten)]
pub data: SessionSnapshotEntryData,
}
@@ -1180,12 +1182,10 @@ pub enum Event {
},
/// One agent-injected system item committed to history.
///
/// Carries the JSON form of `session_store::SystemItem`. Covers
/// `Method::Notify` echoes, child-Worker lifecycle events from
/// `Method::WorkerEvent`, `@<path>` resolution payloads, and any future
/// Clients dispatch on the `kind` tag for typed rendering instead
/// of parsing free-text prefixes like `[Notification] …` or
/// `[File: …]`.
/// Carries the allowlisted JSON form of `session_store::PublicSystemItem`,
/// not the storage record. Its descriptive `kind` is independent from the
/// typed `presentation` used for notification, reminder, or context rendering.
/// Private Feature payloads and prompt provenance are never event data.
///
/// One event per `LogEntry::AnnotatedSystemItem` commit. Disk-side and
/// wire-side are 1:1.
@@ -1497,6 +1497,11 @@ pub enum Event {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct Alert {
/// Stable only within the emitting live Worker; not durable alert history.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub position: Option<AlertPosition>,
pub level: AlertLevel,
pub source: AlertSource,
pub message: String,
@@ -1504,6 +1509,18 @@ pub struct Alert {
pub timestamp_ms: i64,
}
/// Display anchor captured at emission, resolved when pending text is committed.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum AlertPosition {
Start,
AfterEntry { entry_id: String },
AfterToolCall { call_id: String },
AfterInFlight { block_id: u64 },
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -1704,6 +1721,9 @@ pub struct InFlightCompaction {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct InFlightSnapshot {
/// Live-Worker identities parallel to the blocks actually included below.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub block_ids: Vec<u64>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub blocks: Vec<InFlightBlock>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
@@ -2601,6 +2621,7 @@ mod tests {
session: SessionSnapshot {
pending_submissions: PendingSubmissionsSnapshot::default(),
entries: vec![SessionSnapshotEntry {
use_policy: None,
entry_id: "entry-1".into(),
timestamp: 1,
provenance: SessionEntryProvenance::HumanInput,
@@ -2721,6 +2742,7 @@ mod tests {
},
state: WorkerStatus::Running.into(),
in_flight: InFlightSnapshot {
block_ids: Vec::new(),
blocks: vec![
InFlightBlock::Text {
text: "hel".into(),
@@ -3048,9 +3070,42 @@ mod tests {
}
}
#[test]
fn alert_ordering_metadata_is_optional_and_uses_tagged_wire_anchors() {
let legacy = serde_json::json!({ "level": "warn", "source": "worker", "message": "old", "timestamp_ms": 42 });
let alert: Alert = serde_json::from_value(legacy.clone()).unwrap();
assert!(alert.id.is_none());
assert!(alert.position.is_none());
assert_eq!(serde_json::to_value(alert).unwrap(), legacy);
for position in [
serde_json::json!({ "kind": "start" }),
serde_json::json!({ "kind": "after_entry", "entry_id": "entry-1" }),
serde_json::json!({ "kind": "after_tool_call", "call_id": "call-1" }),
serde_json::json!({ "kind": "after_in_flight", "block_id": 7 }),
] {
let mut wire = legacy.clone();
wire["id"] = serde_json::json!("alert-1");
wire["position"] = position;
let alert: Alert = serde_json::from_value(wire.clone()).unwrap();
assert_eq!(serde_json::to_value(alert).unwrap(), wire);
}
let old_snapshot: InFlightSnapshot = serde_json::from_value(serde_json::json!({})).unwrap();
assert!(old_snapshot.block_ids.is_empty());
assert_eq!(
serde_json::to_value(old_snapshot).unwrap(),
serde_json::json!({})
);
let wire =
serde_json::json!({"blocks": [{"kind": "text", "text": "pending"}], "block_ids": [7]});
let snapshot: InFlightSnapshot = serde_json::from_value(wire.clone()).unwrap();
assert_eq!(serde_json::to_value(snapshot).unwrap(), wire);
}
#[test]
fn event_alert_format() {
let event = Event::Alert(Alert {
id: None,
position: None,
level: AlertLevel::Warn,
source: AlertSource::Compactor,
message: "compaction failed".into(),
+21 -19
View File
@@ -4,25 +4,25 @@ use serde_json::{Map, Value};
use ts_rs::{Config, TS};
use crate::{
Alert, AlertLevel, AlertSource, CommandEvent, CommandSnapshot, CommandStatus, CommandStream,
CommandStreamSlice, CompactionLifecycle, CompactionLifecycleState, CompactionPhase,
CompactionTrigger, CompletionContext, CompletionEntry, CompletionKind, ContextTokenSource,
ContextUsage, ErrorCode, Event, FeatureInvocation, FeatureInvocationDescriptor,
FeatureInvocationIdentity, FeatureInvocationResult, FeatureInvocationStatus,
FeatureInvocationSyntax, Greeting, InFlightBlock, InFlightCompaction, InFlightSnapshot,
InFlightToolCallState, InternalWorkerKind, InternalWorkerRef, InternalWorkerSnapshot,
InvocationArgumentDescriptor, InvocationArgumentType, InvocationArgumentValue,
InvocationClientAdapter, InvocationCompletion, InvocationValue, InvokeKind, MemoryWorkerEvent,
Method, PasteArtifactAvailability, PasteArtifactMediaType, PasteArtifactRef,
PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission, ReasoningConfig,
RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason, ScopeRule, Segment,
SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment, SubmissionDisposition,
SymlinkPolicy, ToolResultDisposition, TurnResult, UploadedFileAvailability, UploadedFileRef,
WorkerBusyState, WorkerCommandAcknowledgement, WorkerCommandDisposition, WorkerCommandEnvelope,
WorkerCommandKind, WorkerEvent, WorkerMaintenanceState, WorkerRunState, WorkerState,
WorkerStateSnapshot, WorkerStatus,
Alert, AlertLevel, AlertPosition, AlertSource, CommandEvent, CommandSnapshot, CommandStatus,
CommandStream, CommandStreamSlice, CompactionLifecycle, CompactionLifecycleState,
CompactionPhase, CompactionTrigger, CompletionContext, CompletionEntry, CompletionKind,
ContextTokenSource, ContextUsage, ErrorCode, Event, FeatureInvocation,
FeatureInvocationDescriptor, FeatureInvocationIdentity, FeatureInvocationResult,
FeatureInvocationStatus, FeatureInvocationSyntax, Greeting, InFlightBlock, InFlightCompaction,
InFlightSnapshot, InFlightToolCallState, InternalWorkerKind, InternalWorkerRef,
InternalWorkerSnapshot, InvocationArgumentDescriptor, InvocationArgumentType,
InvocationArgumentValue, InvocationClientAdapter, InvocationCompletion, InvocationValue,
InvokeKind, MemoryWorkerEvent, Method, PasteArtifactAvailability, PasteArtifactMediaType,
PasteArtifactRef, PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission,
ReasoningConfig, RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason,
ScopeRule, Segment, SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionEntryUsePolicy, SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole,
SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment,
SubmissionDisposition, SymlinkPolicy, ToolResultDisposition, TurnResult,
UploadedFileAvailability, UploadedFileRef, WorkerBusyState, WorkerCommandAcknowledgement,
WorkerCommandDisposition, WorkerCommandEnvelope, WorkerCommandKind, WorkerEvent,
WorkerMaintenanceState, WorkerRunState, WorkerState, WorkerStateSnapshot, WorkerStatus,
subscription::{
EventSubscriptionSelector, MAX_CORRELATION_ID_BYTES, MAX_REJECTION_MESSAGE_BYTES,
MAX_RESOURCE_ID_BYTES, MAX_SUBSCRIPTION_COLLECTION_ITEMS,
@@ -65,6 +65,7 @@ pub fn generated_protocol_types() -> String {
);
push_decl::<AlertLevel>(&cfg, &mut output);
push_decl::<AlertPosition>(&cfg, &mut output);
push_decl::<AlertSource>(&cfg, &mut output);
push_decl::<CompletionKind>(&cfg, &mut output);
push_decl::<CompletionContext>(&cfg, &mut output);
@@ -116,6 +117,7 @@ pub fn generated_protocol_types() -> String {
push_decl::<InFlightCompaction>(&cfg, &mut output);
push_decl::<InFlightSnapshot>(&cfg, &mut output);
push_decl::<SessionEntryProvenance>(&cfg, &mut output);
push_decl::<SessionEntryUsePolicy>(&cfg, &mut output);
push_decl::<SessionMessageRole>(&cfg, &mut output);
push_decl::<SessionContentPart>(&cfg, &mut output);
push_decl::<SessionToolAttachment>(&cfg, &mut output);
+319 -3
View File
@@ -425,6 +425,8 @@ pub struct BackendJobExecutionBinding {
pub input_digest: Option<String>,
#[serde(default)]
pub subjektiv_consolidation: bool,
#[serde(default)]
pub subjektiv_journal: bool,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
@@ -516,9 +518,10 @@ pub struct WorkerRestorePreparation {
pub workspace_api: Option<WorkspaceApiRef>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workdir_attachments: Option<Vec<LogicalWorkdirAttachment>>,
/// Retained in saved Restore intents; Restore does not apply Repository access.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access: Vec<WorkingDirectoryRepositoryAccessRequest>,
/// Read-only references resolved by Workspace only after this operation owns admission.
/// Retained references from saved resource preparation, not Restore prerequisites.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access_workdirs: Vec<RepositoryAccessWorkdirReference>,
}
@@ -712,6 +715,8 @@ pub struct WorkerSessionArchiveManifest {
pub source_runtime_id: String,
pub source_worker_id: WorkerId,
pub source_session_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source_active_segment_id: Option<String>,
pub segment_ids: Vec<String>,
pub source_created_at: String,
pub removed_at: String,
@@ -787,6 +792,118 @@ pub enum SessionPublicSource {
Archived { archive_id: String },
}
/// Canonical Session metadata, distinct from Memory's evidence/public-index projection.
/// `origin_json` is the exact serialized saved origin; it is not model-authored evidence.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionMaterialDescriptor {
pub segment_id: String,
pub entry_ref: String,
pub canonical_order: u64,
pub origin_json: String,
pub sha256: String,
pub total_bytes: u64,
}
/// Explicit Host erasure, distinct from ordinary retention policy or pin release.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialEraseRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
}
/// Acknowledges the permanent fence and physical erasure of Session originals,
/// including archived diagnostics bound to the same removal operations. Historical
/// retention receipts remain immutable; their old retained flags are not current availability.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialEraseResponse {
pub session_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialPinRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub pin_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialPinResponse {
pub session_id: String,
pub pin_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialCatalogRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub expected_catalog_sha256: Option<String>,
pub offset: u64,
pub limit: usize,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialCatalogPage {
pub session_id: String,
pub active_segment_id: String,
pub catalog_sha256: String,
pub descriptors: Vec<SessionMaterialDescriptor>,
pub next_offset: Option<u64>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialReadRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
/// Compared with a freshly resolved saved descriptor, never adopted as evidence.
pub expected_descriptor: SessionMaterialDescriptor,
pub byte_offset: u64,
pub max_content_bytes: usize,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialReadPage {
pub descriptor: SessionMaterialDescriptor,
pub content: String,
pub next_offset: u64,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionMaterialCatalogAvailability {
/// The saved active pointer is pending and storage proves no Session material
/// has ever been published. Missing/corrupt materialized sources are not empty.
NotMaterialized {
session_id: String,
},
Page {
page: SessionMaterialCatalogPage,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionMaterialReadAvailability {
Page {
page: SessionMaterialReadPage,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionPublicEntryKind {
@@ -988,8 +1105,78 @@ pub enum WorkerSessionAvailability {
},
}
/// Host-owned Subject release operation. A reservation has no timeout: uncertainty retains
/// occupancy and mutation exclusion until the same operation is explicitly reconciled.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectReleaseRequest {
pub workspace_id: String,
pub worker_id: WorkerId,
pub binding_id: String,
pub operation_id: String,
pub action: SubjectReleaseAction,
}
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum SubjectReleaseAction {
Reserve,
Disconnect,
Cancel,
}
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum SubjectReleaseState {
Reserved,
Disconnected,
Cancelled,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectReleaseResponse {
pub operation_id: String,
pub binding_id: String,
pub state: SubjectReleaseState,
}
/// Irreversibly fence an unpublished failed Subject create. Exact retries must retain
/// all four fields; this is distinct from releasing an existing Subject Controller.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryRequest {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
/// Durable admission-fence proof plus freshly confirmed execution absence.
/// An error (including a lost response) is never evidence of absence.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryResponse {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
#[api(reqwest, axum)]
pub trait RuntimeApi {
#[post("/v1/subject-create-recovery", status = 200, error_status = 400)]
async fn recover_subject_create(
&self,
#[body] request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError>;
#[post("/v1/subject-release", status = 200, error_status = 400)]
async fn subject_release(
&self,
#[body] request: SubjectReleaseRequest,
) -> Result<SubjectReleaseResponse, RuntimeApiError>;
#[get("/v1/ping", status = 200, error_status = 400)]
async fn ping(
&self,
@@ -1029,6 +1216,52 @@ pub trait RuntimeApi {
#[query] request: WorkerSessionHistoryRequest,
) -> Result<WorkerSessionHistoryAvailability, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/erase",
status = 200,
error_status = 400
)]
async fn session_material_erase(
&self,
#[body] request: SessionMaterialEraseRequest,
) -> Result<SessionMaterialEraseResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/pin",
status = 200,
error_status = 400
)]
async fn session_material_pin(
&self,
#[body] request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/release",
status = 200,
error_status = 400
)]
async fn session_material_release(
&self,
#[body] request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/catalog",
status = 200,
error_status = 400
)]
async fn session_material_catalog(
&self,
#[body] request: SessionMaterialCatalogRequest,
) -> Result<SessionMaterialCatalogAvailability, RuntimeApiError>;
#[post("/v1/session-public/material/read", status = 200, error_status = 400)]
async fn session_material_read(
&self,
#[body] request: SessionMaterialReadRequest,
) -> Result<SessionMaterialReadAvailability, RuntimeApiError>;
#[post("/v1/session-public/search", status = 200, error_status = 400)]
async fn session_public_search(
&self,
@@ -1277,6 +1510,22 @@ pub const REMAINING_RUNTIME_ROUTES: &[RemainingRuntimeRoute] = &[
mod tests {
use super::*;
#[test]
fn backend_job_journal_grant_defaults_off_and_does_not_enable_memory_grant() {
let legacy = serde_json::json!({"job_id":"job", "attempt_id":"attempt", "subjektiv_consolidation":true});
let binding: BackendJobExecutionBinding = serde_json::from_value(legacy).unwrap();
assert!(binding.subjektiv_consolidation);
assert!(!binding.subjektiv_journal);
let journal =
serde_json::json!({"job_id":"job", "attempt_id":"attempt", "subjektiv_journal":true});
let binding: BackendJobExecutionBinding = serde_json::from_value(journal.clone()).unwrap();
assert!(binding.subjektiv_journal);
assert!(!binding.subjektiv_consolidation);
let roundtrip: BackendJobExecutionBinding =
serde_json::from_value(serde_json::to_value(&binding).unwrap()).unwrap();
assert_eq!(roundtrip, binding);
}
#[test]
fn restore_request_is_mandatory_and_conflict_is_not_an_operation_state() {
for invalid in [
@@ -1442,6 +1691,59 @@ mod tests {
})
}
async fn session_material_erase(
&self,
_request: SessionMaterialEraseRequest,
) -> Result<SessionMaterialEraseResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_pin(
&self,
_request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn recover_subject_create(
&self,
request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError> {
Ok(SubjectCreateRecoveryResponse {
workspace_id: request.workspace_id,
worker_id: request.worker_id,
create_fingerprint: request.create_fingerprint,
operation_id: request.operation_id,
})
}
async fn subject_release(
&self,
_request: SubjectReleaseRequest,
) -> Result<SubjectReleaseResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_release(
&self,
_request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_catalog(
&self,
_request: SessionMaterialCatalogRequest,
) -> Result<SessionMaterialCatalogAvailability, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_read(
&self,
_request: SessionMaterialReadRequest,
) -> Result<SessionMaterialReadAvailability, RuntimeApiError> {
Err(test_error(501))
}
async fn session_public_search(
&self,
_request: SessionPublicSearchRequest,
@@ -1559,6 +1861,21 @@ mod tests {
.build()
.unwrap();
let recovery = SubjectCreateRecoveryRequest {
workspace_id: "workspace-a".into(),
worker_id: WorkerId::now_v7(),
create_fingerprint: "failed-create-fingerprint".into(),
operation_id: "recovery-1".into(),
};
let response = client
.recover_subject_create(recovery.clone())
.await
.unwrap();
assert_eq!(response.workspace_id, recovery.workspace_id);
assert_eq!(response.worker_id, recovery.worker_id);
assert_eq!(response.create_fingerprint, recovery.create_fingerprint);
assert_eq!(response.operation_id, recovery.operation_id);
let summary = client.runtime_summary().await.unwrap();
assert_eq!(summary.runtime.display_name.as_deref(), Some("round-trip"));
let workers = client
@@ -1603,9 +1920,8 @@ mod tests {
}
#[test]
fn contract_inventory_is_complete_and_unique() {
fn contract_inventory_has_unique_routes_under_api_base_path() {
let operations = RuntimeApiMetadata::OPERATIONS;
assert_eq!(operations.len(), 20);
let mut routes = operations
.iter()
.map(|operation| (format!("{:?}", operation.method), operation.path))
+475
View File
@@ -0,0 +1,475 @@
//! Neutral explicit-allow authorization contracts. Resource containment never crosses kinds.
//!
//! # Trusted Feature boundary
//!
//! A Profile requests execution configuration, not authority. The Worker asks the
//! Host for each enabled Feature's exact activation name before initialization,
//! then trusted Feature code asks for each protected operation and its semantic
//! resource before access or effects. `ticket` on [`PermissionResource::Feature`]
//! activates Ticket; it implies neither `ticket.create` nor any future operation.
//! `ticket.create` on a Workspace permits creation there but cannot activate the
//! Feature. There are no wildcard permissions or implicit dotted-name inheritance.
//!
//! Identity is absent from the authorization request intentionally: the Host
//! authenticates the bound Runtime/Worker and resolves current source permissions,
//! body ownership and any per-Worker execution grant. A shared client, Profile,
//! remembered receipt or caller-supplied Subject claim is not authority. Features
//! depend on this contract, never on a particular Subject policy implementation.
//!
//! Supported cooperative boundaries are Feature activation, ordinary Worker
//! delegation, Ticket creation (Tool/native WIP/typed service), Workdir filesystem
//! read/write and whole-attachment command execution, and scoped result notify.
//! Workdir providers still enforce resolved paths, traversal, capability, Ticket
//! and provider ceilings. The returned scope is for provider narrowing, not a
//! transferable credential. Scoped checkout/overlap operations are refused until
//! their complete access semantics are adapted. Other Feature operations retain
//! their existing owner checks; they do not automatically acquire this operation
//! policy. Missing calls are trusted Feature bugs, not interception by a universal
//! Tool executor or protection against malicious Feature code. Connection tests
//! must exercise each exposed surface, assert a genuine permitted effect and
//! assert zero downstream calls on denial, including revocation and provider denial.
//!
//! # Delegation callers and recovery
//!
//! Callers such as a chat Feature supply explicit operation/resource selections
//! and retain the same public operation ID and full request after an unknown
//! result. Normal Profile/Worker allocation and control/resource grants are reused;
//! there is no Subject Worker registry. Effective grants bind immutable Profile
//! resolution, the reserved Worker and source assignment. Permission replacement
//! invalidates existing children rather than refreshing them. Body replacement,
//! retirement and restore must recheck current authority; historical result lookup
//! is not new execution permission. Cancellation acceptance is distinct from
//! terminal confirmation and does not undo completed effects.
//!
//! A child receives no Subject body identity, singleton, whole Memory or public
//! conversation-post authority. `worker.result` is explicit notify authority to a
//! named origin Worker, not arbitrary control or publication. Chat delivery and
//! conversation correlation belong to the caller integration, not this contract.
//! Unset existing Subject policies deny new Subject execution/delegation; ordinary
//! non-Subject Workers explicitly retain existing Backend/provider contracts.
//! Standalone non-Workspace activation compatibility grants no Workspace operation.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
pub const MAX_SCOPED_PERMISSIONS: usize = 256;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PermissionResource {
Feature,
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
// Serde's internally tagged unit variants ignore extra fields even with deny_unknown_fields.
// A struct-shaped wire variant preserves the public unit variant while rejecting those claims.
impl<'de> Deserialize<'de> for PermissionResource {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
#[derive(Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
enum Wire {
Feature {},
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
Ok(match Wire::deserialize(deserializer)? {
Wire::Feature {} => Self::Feature,
Wire::Workspace { workspace_id } => Self::Workspace { workspace_id },
Wire::Workdir {
working_directory_id,
path,
recursive,
} => Self::Workdir {
working_directory_id,
path,
recursive,
},
Wire::Worker {
runtime_id,
worker_id,
} => Self::Worker {
runtime_id,
worker_id,
},
})
}
}
impl PermissionResource {
pub fn validate(&self) -> Result<(), String> {
match self {
Self::Feature => Ok(()),
Self::Workspace { workspace_id } => validate_authorization_identifier(workspace_id),
Self::Workdir {
working_directory_id,
path,
..
} => {
validate_authorization_identifier(working_directory_id)?;
validate_relative_permission_path(path)
}
Self::Worker {
runtime_id,
worker_id,
} => {
validate_authorization_identifier(runtime_id)?;
validate_authorization_identifier(worker_id)
}
}
}
pub fn covers(&self, other: &Self) -> bool {
if self.validate().is_err() || other.validate().is_err() {
return false;
}
match (self, other) {
(Self::Feature, Self::Feature) => true,
(Self::Workspace { workspace_id: a }, Self::Workspace { workspace_id: b }) => a == b,
(
Self::Worker {
runtime_id: ar,
worker_id: aw,
},
Self::Worker {
runtime_id: br,
worker_id: bw,
},
) => ar == br && aw == bw,
(
Self::Workdir {
working_directory_id: a,
path: ap,
recursive: ar,
},
Self::Workdir {
working_directory_id: b,
path: bp,
recursive: br,
},
) => {
a == b
&& if *ar {
ap == "."
|| ap == bp
|| bp
.strip_prefix(ap)
.is_some_and(|tail| tail.starts_with('/'))
} else {
ap == bp && !br
}
}
_ => false,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ScopedPermission {
pub operation: String,
pub resource: PermissionResource,
}
impl ScopedPermission {
pub fn validate(&self) -> Result<(), String> {
validate_operation_name(&self.operation)?;
self.resource.validate()?;
if matches!(self.resource, PermissionResource::Feature) && self.operation.contains('.') {
return Err("Feature resources authorize activation names only, not operations".into());
}
Ok(())
}
pub fn covers(&self, other: &Self) -> bool {
self.validate().is_ok()
&& other.validate().is_ok()
&& self.operation == other.operation
&& self.resource.covers(&other.resource)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissions {
pub change_id: String,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsUpdate {
pub operation_id: String,
pub expected_change_id: Option<String>,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerExecutionGrant {
pub subject_id: String,
pub origin_runtime_id: String,
pub origin_worker_id: String,
pub source_change_id: String,
pub runtime_id: String,
pub worker_id: String,
pub profile: String,
pub profile_digest: String,
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerDelegationRequest {
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
impl WorkerDelegationRequest {
pub fn validate(&self) -> Result<(), String> {
validate_scoped_permissions(&self.permissions)?;
validate_scoped_permissions(&self.redelegation)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationRequest {
pub operation: String,
pub resource: PermissionResource,
}
impl OperationAuthorizationRequest {
pub fn validate(&self) -> Result<(), String> {
ScopedPermission {
operation: self.operation.clone(),
resource: self.resource.clone(),
}
.validate()
}
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationResponse {
/// Matching granted scope for provider-resolved checks, not a credential.
/// `None` identifies the explicit non-Subject legacy authorization path.
#[serde(default)]
pub scope: Option<PermissionResource>,
}
pub fn validate_operation_name(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| !value.split('.').all(|part| {
!part.is_empty()
&& part
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'_' || b == b'-')
})
{
return Err(
"permission operation must be an exact, nonempty dot-separated name (no wildcards)"
.into(),
);
}
Ok(())
}
pub fn validate_scoped_permissions(permissions: &[ScopedPermission]) -> Result<(), String> {
if permissions.len() > MAX_SCOPED_PERMISSIONS {
return Err("too many scoped permissions".into());
}
for permission in permissions {
permission.validate()?;
}
Ok(())
}
pub fn validate_authorization_identifier(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| value.trim() != value
|| value.chars().any(|c| c.is_control() || c == '*')
{
return Err("authorization identity must be nonempty, bounded and exact".into());
}
Ok(())
}
/// Paths are already-normalized lexical paths, not host filesystem paths. The Host must
/// still enforce symlink/provider boundaries; lexical containment grants no filesystem access.
pub fn validate_relative_permission_path(path: &str) -> Result<(), String> {
if path == "." {
return Ok(());
}
if path.is_empty()
|| path.len() > 4096
|| path.contains(['\\', ':'])
|| path.chars().any(char::is_control)
|| path
.split('/')
.any(|part| part.is_empty() || part == "." || part == "..")
{
return Err("permission path must be a normalized relative path without escape".into());
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn workdir(path: &str, recursive: bool) -> PermissionResource {
PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
}
}
#[test]
fn lexical_containment_rejects_escape_and_respects_component_and_recursion_boundaries() {
for path in [
"",
"/src",
"../src",
"src/../secret",
"src/./a",
"src//a",
"src/",
"C:/src",
"src\\a",
] {
assert!(workdir(path, true).validate().is_err(), "{path}");
}
let root = workdir(".", true);
let src = workdir("src", true);
assert!(root.covers(&src));
assert!(src.covers(&workdir("src/a", true)));
assert!(!src.covers(&workdir("src-other", false)));
assert!(!workdir("src", false).covers(&src));
assert!(workdir("src", false).covers(&workdir("src", false)));
assert!(!workdir("src", false).covers(&workdir("src/a", false)));
assert!(!src.covers(&PermissionResource::Workdir {
working_directory_id: "other".into(),
path: "src".into(),
recursive: false
}));
}
#[test]
fn operation_hierarchy_never_implies_inheritance_or_feature_activation() {
for name in [
"",
"*",
"ticket.*",
".ticket",
"ticket.",
"ticket..read",
"ticket read",
] {
assert!(validate_operation_name(name).is_err(), "{name}");
}
let activation = ScopedPermission {
operation: "ticket".into(),
resource: PermissionResource::Feature,
};
let create = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "ws".into(),
},
};
let parent_name = ScopedPermission {
operation: "ticket".into(),
resource: create.resource.clone(),
};
assert!(!parent_name.covers(&create));
assert!(!create.covers(&parent_name));
assert!(!activation.covers(&create));
assert!(!create.covers(&activation));
assert!(
ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Feature
}
.validate()
.is_err()
);
assert!(
!PermissionResource::Workspace {
workspace_id: "ws".into()
}
.covers(&workdir(".", true))
);
}
#[test]
fn wire_contract_rejects_self_claimed_identity_and_unknown_fields() {
let request = serde_json::json!({"permissions":[],"redelegation":[],"subject_id":"forged"});
assert!(serde_json::from_value::<WorkerDelegationRequest>(request).is_err());
let resource: PermissionResource =
serde_json::from_value(serde_json::json!({"kind":"feature"})).unwrap();
assert_eq!(resource, PermissionResource::Feature);
assert!(
serde_json::from_value::<PermissionResource>(
serde_json::json!({"kind":"feature","workspace_id":"ws"})
)
.is_err()
);
}
#[test]
fn authorization_response_carries_matching_scope_and_defaults_legacy_absence() {
let legacy: OperationAuthorizationResponse =
serde_json::from_value(serde_json::json!({})).unwrap();
assert_eq!(legacy.scope, None);
assert_eq!(
serde_json::to_value(legacy).unwrap(),
serde_json::json!({"scope":null})
);
let granted = OperationAuthorizationResponse {
scope: Some(workdir("src", true)),
};
let wire = serde_json::to_value(&granted).unwrap();
assert_eq!(
wire,
serde_json::json!({"scope":{"kind":"workdir","working_directory_id":"wd","path":"src","recursive":true}})
);
assert_eq!(
serde_json::from_value::<OperationAuthorizationResponse>(wire).unwrap(),
granted
);
}
}
+10
View File
@@ -134,6 +134,14 @@ pub enum DriveEntryKind {
File,
}
/// Current caller's advisory publication ceiling, never a grant or execution credential.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveCapabilities {
pub write: bool,
}
/// Metadata only: content is returned through bounded text or binary routes.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -491,6 +499,7 @@ impl api_macros::HttpRequestError for DriveApiError {
impl api_macros::openapi::OpenApiSchema for DriveApiError {}
impl api_macros::openapi::OpenApiSchema for DriveEntry {}
impl api_macros::openapi::OpenApiSchema for DriveCapabilities {}
impl api_macros::openapi::OpenApiSchema for DriveEntryQuery {}
impl api_macros::openapi::OpenApiSchema for DriveListQuery {}
impl api_macros::openapi::OpenApiSchema for DriveSearchQuery {}
@@ -518,6 +527,7 @@ pub fn drive_api_typescript() -> String {
DriveEntryRef::decl(&config),
DriveEntryKind::decl(&config),
DriveEntry::decl(&config),
DriveCapabilities::decl(&config),
DriveEntryQuery::decl(&config),
DriveListQuery::decl(&config),
DriveSearchQuery::decl(&config),
+419 -4
View File
@@ -6,9 +6,12 @@ use api_macros::api;
pub use api_macros::axum as server_support;
pub use api_macros::reqwest as client_support;
pub use api_macros::{ApiContract, BinaryBody, HttpMethod, TransportMetadata, WebSocketOperation};
pub mod authorization;
mod drive;
mod notification;
pub use notification::*;
mod ticket_browse;
pub use ticket_browse::*;
pub mod repository_openapi_typescript;
pub use drive::*;
mod workspace_config;
@@ -21,6 +24,34 @@ use webauthn_rs_proto::{
};
pub use workspace_config::*;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsResponse {
pub permissions: Option<authorization::SubjectPermissions>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsMutationResponse {
pub permissions: authorization::SubjectPermissions,
/// New operations are already fenced. These are cancellation requests and
/// Backend command cleanup evidence, not proof that a Runtime turn stopped.
pub interruptions: Vec<SubjectPermissionInterruption>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionInterruption {
pub runtime_id: String,
pub worker_id: String,
pub cancellation_accepted: bool,
pub backend_command_sessions_closed: bool,
pub diagnostics: Vec<String>,
}
#[allow(dead_code)]
#[derive(JsonSchema)]
struct ServerJsonSafeU64(#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))] u64);
@@ -94,6 +125,8 @@ pub struct WorkdirOperationApiError {
pub message: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub diagnostics: Option<Vec<Diagnostic>>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub audit_failures: Vec<workdir::http::WorkdirAuditFailure>,
#[serde(skip, default = "default_repository_error_status")]
status: u16,
/// Server-local metadata retained by generated error response extensions, never the wire body.
@@ -101,6 +134,8 @@ pub struct WorkdirOperationApiError {
pub workdir_log: Option<WorkdirOperationLogContext>,
#[serde(skip)]
pub denial_reason: Option<workdir::WorkdirDenialReason>,
#[serde(skip)]
pub failure_reason: Option<workdir::WorkdirFailureReason>,
}
/// Internal operational correlation. Producers supply fixed-length ID fingerprints and
@@ -123,9 +158,11 @@ impl WorkdirOperationApiError {
code: None,
message: error.message,
diagnostics: Some(error.diagnostics),
audit_failures: Vec::new(),
status: error.status,
workdir_log: None,
denial_reason: None,
failure_reason: None,
}
}
@@ -136,9 +173,11 @@ impl WorkdirOperationApiError {
code: Some(error.code),
message: error.message,
diagnostics: None,
audit_failures: Vec::new(),
status,
workdir_log: None,
denial_reason: error.denial_reason,
failure_reason: error.failure_reason,
}
}
}
@@ -166,8 +205,18 @@ macro_rules! impl_openapi_schema {
}
impl_openapi_schema!(
SubjectPermissionsResponse,
SubjectPermissionsMutationResponse,
SubjectPermissionInterruption,
authorization::SubjectPermissions,
authorization::SubjectPermissionsUpdate,
AuthBootstrapUserRequest,
AuthPublicConfig,
HostingSettings,
HostingAccess,
ProvisionalAccountRequest,
ActivationCodeResponse,
ActivationOptionsRequest,
AuthUserResponse,
DeviceLoginApproveRequest,
DeviceLoginApproveResponse,
@@ -179,6 +228,7 @@ impl_openapi_schema!(
PasskeyLoginOptionsRequest,
PasskeyLoginOptionsResponse,
PasskeyLoginCompleteRequest,
PasskeyAdditionOptionsRequest,
PasskeyRegistrationOptionsRequest,
PasskeyRegistrationOptionsResponse,
PasskeyRegistrationCompleteRequest,
@@ -376,6 +426,8 @@ impl_openapi_schema!(
TicketAssignmentDiagnosticsPage,
TicketQueryRequest,
TicketQueryResponse,
TicketBrowseQueryRequest,
TicketBrowseQueryResponse,
TicketQueueResponse,
TicketRecord,
TicketRecordRef,
@@ -510,6 +562,60 @@ pub trait ServerApi {
#[get("/api/auth/config", status = 200, error_status = 400)]
async fn auth_config(&self) -> Result<AuthPublicConfig, RepositoryApiError>;
#[get("/api/auth/registration-policy", status = 200, error_status = 400)]
async fn auth_registration_policy(&self) -> Result<HostingSettings, RepositoryApiError>;
#[get("/api/hosting/access", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_access(
&self,
#[extension] context: ServerRequestContext,
) -> Result<HostingAccess, RepositoryApiError>;
#[get("/api/hosting/settings", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_settings(
&self,
#[extension] context: ServerRequestContext,
) -> Result<HostingSettings, RepositoryApiError>;
#[put("/api/hosting/settings", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_settings_update(
&self,
#[extension] context: ServerRequestContext,
#[body] request: HostingSettings,
) -> Result<HostingSettings, RepositoryApiError>;
#[post("/api/hosting/accounts", status = 200, error_status = 400, additional_error_statuses = [403, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_account_create(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ProvisionalAccountRequest,
) -> Result<ActivationCodeResponse, RepositoryApiError>;
#[post("/api/hosting/accounts/activation-code", status = 200, error_status = 400, additional_error_statuses = [403, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_activation_code_reissue(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ProvisionalAccountRequest,
) -> Result<ActivationCodeResponse, RepositoryApiError>;
#[post("/api/auth/activation/options", status = 200, error_status = 400, additional_error_statuses = [500])]
async fn auth_activation_options(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ActivationOptionsRequest,
) -> Result<PasskeyRegistrationOptionsResponse, RepositoryApiError>;
#[post(
"/api/auth/activation/complete",
responses = [(status = 200, body = AuthUserResponse, headers = [("set-cookie", SetCookieHeader)])],
error_status = 400,
additional_error_statuses = [500]
)]
async fn auth_activation_complete(
&self,
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthActivationComplete, RepositoryApiError>;
#[post(
"/api/auth/bootstrap-user",
status = 200,
@@ -546,6 +652,36 @@ pub trait ServerApi {
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthPasskeyRegistrationComplete, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/options",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500, 502],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_options(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyAdditionOptionsRequest,
) -> Result<PasskeyRegistrationOptionsResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/complete",
responses = [
(status = 200, body = AuthUserResponse, headers = [("set-cookie", SetCookieHeader)])
],
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_complete(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthPasskeyAdditionComplete, RepositoryApiError>;
#[post(
"/api/auth/passkeys/login/options",
status = 200,
@@ -1204,6 +1340,51 @@ pub trait ServerApi {
#[body] request: MemoryConsolidateStagingRequest,
) -> Result<MemoryConsolidationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/authorization/check",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
openapi = false,
)]
async fn operation_authorize(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: authorization::OperationAuthorizationRequest,
) -> Result<authorization::OperationAuthorizationResponse, RepositoryApiError>;
#[get(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_get(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
) -> Result<SubjectPermissionsResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_set(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
#[body] request: authorization::SubjectPermissionsUpdate,
) -> Result<SubjectPermissionsMutationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects",
status = 201,
@@ -2668,6 +2849,12 @@ pub trait ServerApi {
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveEntry, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/capabilities", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_capabilities(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveCapabilities, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/metadata", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_metadata(
&self,
@@ -3033,6 +3220,14 @@ pub trait ServerApi {
#[body] request: TicketTagsSetRequest,
) -> Result<WorkspaceTicketTagList, RepositoryApiError>;
/// Lightweight browser list; deliberately excludes live MR evidence and external observations.
#[post("/api/w/{workspace_id}/tickets/browse", status = 200, error_status = 400, additional_error_statuses = [401, 403, 404, 500], bearer_auth = true, browser_auth = true)]
async fn ticket_browse(
&self,
#[path] workspace_id: String,
#[body] request: TicketBrowseQueryRequest,
) -> Result<TicketBrowseQueryResponse, RepositoryApiError>;
#[post("/api/w/{workspace_id}/tickets/query", status = 200, error_status = 400, additional_error_statuses = [401, 403, 404, 500], bearer_auth = true, browser_auth = true)]
async fn ticket_query(
&self,
@@ -3803,6 +3998,76 @@ pub struct AuthPublicConfig {
pub cookie_name: String,
}
/// The only public registration policy. No administrator identities or host secrets are exposed.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct HostingSettings {
pub self_registration_enabled: bool,
}
/// Hosting authority for the current authenticated human, not Workspace permissions.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct HostingAccess {
pub is_admin: bool,
}
/// A hosting administrator reserves the same normalized ID used by passkey login.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ProvisionalAccountRequest {
pub handle: String,
}
/// The only response allowed to disclose an activation code. Clients must keep it
/// out of URLs, persistent browser storage and diagnostics. A lost response is
/// recovered by reissuing, which invalidates the previous code and its ceremonies.
#[derive(Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ActivationCodeResponse {
pub user: AuthenticatedUser,
pub code: String,
pub expires_at: String,
}
impl std::fmt::Debug for ActivationCodeResponse {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("ActivationCodeResponse")
.field("user", &self.user)
.field("code", &"[REDACTED]")
.field("expires_at", &self.expires_at)
.finish()
}
}
/// Proof for a pending account's initial passkey ceremony, never a bearer token
/// or browser session. No caller-selected account ID or enrollment purpose is accepted.
#[derive(Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ActivationOptionsRequest {
pub handle: String,
pub code: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub browser_origin: Option<String>,
}
impl std::fmt::Debug for ActivationOptionsRequest {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("ActivationOptionsRequest")
.field("handle", &self.handle)
.field("code", &"[REDACTED]")
.field("browser_origin", &self.browser_origin)
.finish()
}
}
/// Authentication method that established the current request actor.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -3853,6 +4118,17 @@ impl RequestActor {
#[serde(deny_unknown_fields)]
pub struct WhoamiResponse {
pub actor: Option<RequestActor>,
pub session: BrowserSessionState,
}
/// Browser navigation evidence, not an authorization grant or a generic 401 classification.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum BrowserSessionState {
Anonymous {},
Authenticated { expires_at: Option<String> },
Expired {},
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -3872,6 +4148,7 @@ pub struct AuthUserResponse {
pub user: AuthenticatedUser,
}
/// Starts new-account registration only; an existing handle is never interpreted as key addition.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -3885,6 +4162,16 @@ pub struct PasskeyRegistrationOptionsRequest {
pub browser_origin: Option<String>,
}
/// Adds a credential to the current authenticated actor; target identity is never public input.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyAdditionOptionsRequest {
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub browser_origin: Option<String>,
}
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -3905,6 +4192,11 @@ pub struct PasskeyRegistrationCompleteRequest {
pub credential: RegisterPublicKeyCredential,
}
/// Normal discoverable login omits the handle: the authenticator selects a credential.
/// An explicit handle selects legacy identified-account login, for non-resident keys or
/// keys registered with an old random WebAuthn user handle. This never adds a key or
/// falls back to another identity. Authenticate first, then enroll an RK from Account.
/// Existing challenge states are short-lived and are not migrated to the new ceremony.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -7222,6 +7514,9 @@ pub struct CurrentWorkerWorkdirAttachmentResponse {
pub alias: String,
pub working_directory_id: String,
pub capabilities: workdir::WorkdirSessionCapabilities,
/// Present for attachment responses, absent after detach.
#[serde(skip_serializing_if = "Option::is_none")]
pub environment: Option<workdir::WorkdirSessionEnvironment>,
pub attached: bool,
}
@@ -7234,7 +7529,9 @@ pub struct CurrentWorkerWorkdirOperationRequest {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(transparent)]
pub struct CurrentWorkerWorkdirOperationResponse(pub workdir::http::WorkdirSessionOperationResult);
pub struct CurrentWorkerWorkdirOperationResponse(
pub workdir::http::WorkdirSessionOperationResponse,
);
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -8058,9 +8355,8 @@ pub enum TicketImplementationResultKind {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct TicketImplementationResultRequest {
/// Host-owned replay identity; model-facing tools derive it from the Tool call.
pub operation_key: String,
pub assignment_id: String,
pub orchestrator_assignment_id: String,
pub kind: TicketImplementationResultKind,
pub body: String,
pub merge_request: Option<MergeRequestReviewSubject>,
@@ -10241,6 +10537,10 @@ pub struct CreateWorkspaceWorkerRequest {
/// Backend idempotency key used only for authenticated Worker-owned spawn/control.
#[serde(default)]
pub control_operation_id: Option<String>,
/// Requested least-authority child permissions. Host derives the origin; this
/// input cannot assert Subject identity or create a body/singleton attachment.
#[serde(default)]
pub execution_grant: Option<authorization::WorkerDelegationRequest>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -11338,12 +11638,19 @@ pub fn auth_api_typescript() -> String {
let config = ts_rs::Config::default();
let declarations = [
AuthPublicConfig::decl(&config),
HostingSettings::decl(&config),
HostingAccess::decl(&config),
ProvisionalAccountRequest::decl(&config),
ActivationCodeResponse::decl(&config),
ActivationOptionsRequest::decl(&config),
ActorAuthMethod::decl(&config),
AuthenticatedUser::decl(&config),
RequestActor::decl(&config),
BrowserSessionState::decl(&config),
WhoamiResponse::decl(&config),
AuthBootstrapUserRequest::decl(&config),
AuthUserResponse::decl(&config),
PasskeyAdditionOptionsRequest::decl(&config),
PasskeyRegistrationOptionsRequest::decl(&config),
PasskeyRegistrationOptionsResponse::decl(&config),
PasskeyRegistrationCompleteRequest::decl(&config),
@@ -12061,6 +12368,16 @@ mod tests {
HttpMethod::Post,
"/api/auth/passkeys/registration/options",
),
(
"auth_passkey_addition_options",
HttpMethod::Post,
"/api/auth/passkeys/addition/options",
),
(
"auth_passkey_addition_complete",
HttpMethod::Post,
"/api/auth/passkeys/addition/complete",
),
(
"auth_passkey_login_options",
HttpMethod::Post,
@@ -12726,6 +13043,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
})
.unwrap();
assert_eq!(
@@ -12740,6 +13058,7 @@ mod tests {
"workdir_attachments": [],
"feature_connections": {},
"control_operation_id": null,
"execution_grant": null,
})
);
}
@@ -12970,6 +13289,7 @@ mod tests {
alias: "checkout".to_string(),
working_directory_id: "workdir".to_string(),
capabilities: workdir::WorkdirSessionCapabilities::EMPTY,
environment: None,
attached: false,
};
let value = serde_json::to_value(&response).unwrap();
@@ -13758,7 +14078,8 @@ mod tests {
"handle": "hare",
"display_name": "Hare",
"auth_method": "browser_session"
}
},
"session": { "kind": "authenticated", "expires_at": "2026-10-20T00:00:00Z" }
});
let decoded = serde_json::from_value::<WhoamiResponse>(whoami.clone())
.expect("server whoami fixture should match shared DTO");
@@ -13940,11 +14261,36 @@ mod tests {
let unexpected_field = serde_json::json!({
"actor": null,
"session": { "kind": "anonymous" },
"access_token": "must-not-be-accepted"
});
assert!(serde_json::from_value::<WhoamiResponse>(unexpected_field).is_err());
}
#[test]
fn whoami_session_json_distinguishes_anonymous_expired_and_token_auth() {
for session in [
serde_json::json!({"kind":"anonymous"}),
serde_json::json!({"kind":"expired"}),
serde_json::json!({"kind":"authenticated", "expires_at":null}),
] {
let value = serde_json::json!({"actor":null, "session":session});
let response: WhoamiResponse = serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(response).unwrap(), value);
}
for session in [
serde_json::json!({"kind":"unknown"}),
serde_json::json!({"kind":"expired", "expires_at":null}),
] {
assert!(
serde_json::from_value::<WhoamiResponse>(
serde_json::json!({"actor":null, "session":session})
)
.is_err()
);
}
}
#[test]
fn device_login_debug_output_redacts_secret_material() {
let start = DeviceLoginStartResponse {
@@ -14312,6 +14658,9 @@ mod openapi_artifact_tests {
const PUBLIC_UNAUTHENTICATED: &[&str] = &[
"health",
"auth_config",
"auth_registration_policy",
"auth_activation_options",
"auth_activation_complete",
"auth_bootstrap_user",
"auth_passkey_registration_options",
"auth_passkey_registration_complete",
@@ -14326,6 +14675,7 @@ mod openapi_artifact_tests {
// identity, or a one-use Reviewer capability. They remain generated Rust client/Axum
// operations, but must not appear as unauthenticated operations in the public OpenAPI.
const SIGNED_INTERNAL: &[&str] = &[
"operation_authorize",
"current_worker_workspace_config_get",
"current_worker_workspace_config_attach",
"current_worker_workspace_config_observe",
@@ -14452,6 +14802,20 @@ mod openapi_artifact_tests {
}
}
#[test]
fn passkey_addition_rejects_caller_selected_target_fields() {
for field in ["handle", "user_id", "account_id"] {
let payload = serde_json::json!({ (field): "other-user" });
assert!(
serde_json::from_value::<PasskeyAdditionOptionsRequest>(payload).is_err(),
"addition must reject caller-selected {field}"
);
}
assert!(
serde_json::from_value::<PasskeyAdditionOptionsRequest>(serde_json::json!({})).is_ok()
);
}
#[test]
fn auth_workspace_catalog_and_typed_header_contract_is_strict() {
let document = canonical_openapi_document().expect("canonical OpenAPI contract must build");
@@ -14465,6 +14829,8 @@ mod openapi_artifact_tests {
("/api/auth/bootstrap-user", "post"),
("/api/auth/passkeys/registration/options", "post"),
("/api/auth/passkeys/registration/complete", "post"),
("/api/auth/passkeys/addition/options", "post"),
("/api/auth/passkeys/addition/complete", "post"),
("/api/auth/passkeys/login/options", "post"),
("/api/auth/passkeys/login/complete", "post"),
("/api/auth/logout", "post"),
@@ -14503,6 +14869,7 @@ mod openapi_artifact_tests {
}
for path in [
"/api/auth/passkeys/registration/complete",
"/api/auth/passkeys/addition/complete",
"/api/auth/passkeys/login/complete",
"/api/auth/logout",
] {
@@ -14694,10 +15061,12 @@ mod workdir_error_diagnostic_tests {
#[test]
fn workdir_internal_diagnostics_never_enter_public_error_body_or_schema() {
let mut error = WorkdirOperationApiError::provider(workdir::http::WorkdirTransportError {
failure_reason: None,
code: workdir::http::WorkdirTransportErrorCode::Denied,
message: "Workdir operation was denied".to_string(),
denial_reason: Some(workdir::WorkdirDenialReason::OsPermissionDenied),
});
error.failure_reason = Some(workdir::WorkdirFailureReason::HttpTimeout);
error.workdir_log = Some(WorkdirOperationLogContext {
operation: "write",
workspace_id_hash: "internal-workspace-fingerprint".to_string(),
@@ -14711,9 +15080,55 @@ mod workdir_error_diagnostic_tests {
assert_eq!(api_macros::HttpError::status_code(&error), 403);
let decoded: WorkdirOperationApiError = serde_json::from_value(wire).unwrap();
assert!(decoded.denial_reason.is_none());
assert!(decoded.failure_reason.is_none());
assert!(decoded.workdir_log.is_none());
let schema = serde_json::to_value(schemars::schema_for!(WorkdirOperationApiError)).unwrap();
assert!(schema["properties"].get("workdir_log").is_none());
assert!(schema["properties"].get("denial_reason").is_none());
assert!(schema["properties"].get("failure_reason").is_none());
}
}
#[cfg(test)]
mod activation_contract_tests {
use super::*;
#[test]
fn activation_inputs_reject_caller_selected_authority_and_debug_redacts_codes() {
for field in [
"account_id",
"user_id",
"purpose",
"is_admin",
"authentication",
] {
let mut body = serde_json::json!({"handle":"new-user","code":"activation-secret"});
body[field] = serde_json::json!("caller-controlled");
assert!(serde_json::from_value::<ActivationOptionsRequest>(body).is_err());
let mut body = serde_json::json!({"handle":"new-user"});
body[field] = serde_json::json!("caller-controlled");
assert!(serde_json::from_value::<ProvisionalAccountRequest>(body).is_err());
}
let request = ActivationOptionsRequest {
handle: "new-user".into(),
code: "activation-secret".into(),
browser_origin: None,
};
assert!(!format!("{request:?}").contains(&request.code));
let response = ActivationCodeResponse {
user: AuthenticatedUser {
user_id: "u".into(),
account_id: "a".into(),
handle: "new-user".into(),
display_name: "new-user".into(),
},
code: "activation-secret".into(),
expires_at: "2030-01-01T00:00:00Z".into(),
};
assert!(!format!("{response:?}").contains(&response.code));
assert_eq!(
serde_json::to_value(&response).unwrap()["code"],
"activation-secret"
);
}
}
@@ -459,6 +459,13 @@ fn validate_ticket_browser_operations(
request: Some("TicketTagSetRequest"),
successes: &[("200", "Array_of_WorkspaceTag")],
},
OperationContract {
path: "/api/w/{workspace_id}/tickets/browse",
method: "post",
operation_id: "ticket_browse",
request: Some("TicketBrowseQueryRequest"),
successes: &[("200", "TicketBrowseQueryResponse")],
},
OperationContract {
path: "/api/w/{workspace_id}/tickets/query",
method: "post",
+65
View File
@@ -0,0 +1,65 @@
//! Lightweight Tickets browser contract, separate from authoritative evidence queries.
use crate::{BrowserTicketState, QueryPage};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryRequest {
pub query: Option<String>,
#[serde(default)]
#[cfg_attr(feature = "typescript", ts(type = "Array<BrowserTicketState>"))]
pub states: Vec<ticket::TicketState>,
/// Every supplied active tag ID must match (AND).
#[serde(default)]
pub tag_ids: Vec<String>,
#[cfg_attr(
feature = "typescript",
ts(type = "TicketAssignmentPrincipalSnapshot | null")
)]
pub assigned_principal: Option<ticket::TicketAssignmentPrincipalSnapshot>,
#[cfg_attr(
feature = "typescript",
ts(type = "TicketAssignmentTrackingStatus | null")
)]
pub handling_status: Option<ticket::TicketAssignmentTrackingStatus>,
/// Exact current closed-case reason; historical reasons do not match this filter.
#[cfg_attr(feature = "typescript", ts(type = "TicketClosureReason | null"))]
pub closure_reason: Option<ticket::TicketClosureReason>,
/// Only updated_desc is supported; defaults to updated_desc.
pub sort: Option<String>,
#[schemars(range(min = 0, max = 1000))]
pub limit: Option<usize>,
/// Browse-only cursor bound to the Workspace, canonical filters, and sort.
pub cursor: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryItem {
pub id: String,
pub resource_key: String,
pub title: String,
pub state: BrowserTicketState,
pub updated_at: Option<String>,
#[cfg_attr(feature = "typescript", ts(type = "Array<WorkspaceTag>"))]
pub tags: Vec<ticket::WorkspaceTag>,
#[cfg_attr(feature = "typescript", ts(type = "TicketClosureJudgment | null"))]
pub current_closure: Option<ticket::TicketClosureJudgment>,
#[cfg_attr(
feature = "typescript",
ts(type = "Array<TicketAssignmentTrackingProjection>")
)]
pub assignment_tracking: Vec<ticket::TicketAssignmentTrackingProjection>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryResponse {
pub items: Vec<TicketBrowseQueryItem>,
pub page: QueryPage,
pub record_authority: String,
}
+104 -6
View File
@@ -48,17 +48,48 @@ pub struct FsStore {
/// `write_all` may have extended the file, so rollback is safe only while
/// no sibling writer can append behind it.
append_lock: Arc<Mutex<()>>,
retention: crate::retention::SessionRetentionStore,
}
impl FsStore {
/// Create a new `FsStore` rooted at the given directory.
/// Creates the directory if it does not exist.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
let retention = crate::retention::fs_retention(&root);
Self::new_with_retention(root, retention)
}
/// Hosts that relocate Worker originals must keep control state outside the relocated
/// aggregate and pass the same store to readers and deletion/transfer operations.
pub fn new_with_retention(
root: impl Into<PathBuf>,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, StoreError> {
let root = root.into();
fs::create_dir_all(&root)?;
for entry in fs::read_dir(&root)? {
let entry = entry?;
if entry.file_type()?.is_dir()
&& let Some(session_id) = entry
.file_name()
.to_str()
.and_then(|name| name.parse().ok())
{
let _guard = match retention.lock_live(session_id) {
Ok(guard) => guard,
// Interrupted erasure may leave bytes behind. Never read or
// upgrade them, and do not block unrelated live Sessions.
Err(StoreError::SessionErased(_)) => continue,
Err(error) => return Err(error),
};
crate::worker_session_store::migrate_fs_session(&entry.path(), session_id)?;
}
}
Ok(Self {
root,
append_lock: Arc::new(Mutex::new(())),
retention,
})
}
@@ -98,12 +129,46 @@ impl FsStore {
/// Returns `Ok(true)` when a Session directory was removed and `Ok(false)`
/// when it was already absent.
pub fn delete_session(&self, session_id: SessionId) -> Result<bool, StoreError> {
let session_dir = self.session_dir(session_id);
match fs::remove_dir_all(&session_dir) {
Ok(()) => Ok(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
Err(error) => Err(error.into()),
}
self.delete_session_with_intent(
session_id,
crate::retention::SessionDeletionIntent::Ordinary,
)
}
pub fn retention(&self) -> crate::retention::SessionRetentionStore {
self.retention.clone()
}
pub fn pin_session(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().pin(session_id, pin, || {
Ok(self.session_dir(session_id).is_dir())
})
}
pub fn release_session_pin(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().release(session_id, pin)
}
pub fn delete_session_with_intent(
&self,
session_id: SessionId,
intent: crate::retention::SessionDeletionIntent,
) -> Result<bool, StoreError> {
self.retention().delete(session_id, intent, || {
match fs::remove_dir_all(self.session_dir(session_id)) {
Ok(()) => Ok(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
Err(error) => Err(error.into()),
}
})
}
fn session_dir(&self, session_id: SessionId) -> PathBuf {
@@ -282,12 +347,22 @@ impl FsStore {
}
impl Store for FsStore {
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().retain_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.log_path(session_id, segment_id), &line)
}
@@ -374,9 +449,14 @@ impl Store for FsStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.retention().lock_live(session_id)?;
let path = self.log_path(session_id, segment_id);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
crate::worker_session_store::migrate_fs_session(parent, session_id)?;
}
let mut content = String::new();
for entry in entries {
@@ -423,6 +503,7 @@ impl Store for FsStore {
content: &str,
limits: PasteArtifactLimits,
) -> Result<PasteArtifactRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -451,6 +532,7 @@ impl Store for FsStore {
content: &[u8],
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -474,6 +556,7 @@ impl Store for FsStore {
context: &UploadedFileUploadContext,
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -510,6 +593,7 @@ impl Store for FsStore {
reference: &UploadedFileRef,
source_entry_id: &str,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -537,6 +621,7 @@ impl Store for FsStore {
reference: &UploadedFileRef,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -550,6 +635,7 @@ impl Store for FsStore {
artifact_id: &str,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -563,6 +649,7 @@ impl Store for FsStore {
artifact_id: &str,
source_entry_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -579,6 +666,7 @@ impl Store for FsStore {
session_id: SessionId,
live_owner_ids: &[String],
) -> Result<u64, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -591,6 +679,7 @@ impl Store for FsStore {
session_id: SessionId,
artifact_id: &str,
) -> Result<bool, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -599,6 +688,7 @@ impl Store for FsStore {
}
fn delete_uncommitted_uploaded_files(&self, session_id: SessionId) -> Result<u64, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -631,6 +721,13 @@ impl Store for FsStore {
source_session_id: SessionId,
target_session_id: SessionId,
) -> Result<u64, StoreError> {
let mut sessions = vec![source_session_id, target_session_id];
sessions.sort();
sessions.dedup();
let _retention = sessions
.into_iter()
.map(|id| self.retention().lock_live(id))
.collect::<Result<Vec<_>, _>>()?;
let _guard = self
.append_lock
.lock()
@@ -647,6 +744,7 @@ impl Store for FsStore {
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.trace_path(session_id, segment_id), &line)
}
+204
View File
@@ -67,8 +67,14 @@ pub enum LoggedSessionHistoryOrigin {
ToolOutput {
worker: LoggedWorkerSubject,
},
/// Descriptive producer identity; use policies live in separate metadata.
Provided {
producer: String,
},
/// Host-produced opening context summary; not auto-read or TaskStore material.
CompactionSummary,
/// Frozen Host Journal reference, never independent observation evidence.
JournalContext,
DerivedSummary,
LegacyUnknown,
}
@@ -79,19 +85,32 @@ pub struct LoggedHistoryDerivation {
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct LoggedSessionHistoryMetadata {
pub entry_id: LoggedSessionHistoryEntryId,
pub origin: LoggedSessionHistoryOrigin,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub derivation: Option<LoggedHistoryDerivation>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub input_metadata: Option<crate::HistoryInputMetadata>,
}
impl LoggedSessionHistoryMetadata {
pub(crate) fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
if matches!(self.origin, LoggedSessionHistoryOrigin::Provided { .. })
&& self.input_metadata.is_none()
{
return Err(crate::InputMetadataError::MissingMetadata);
}
Ok(())
}
pub fn legacy_unknown() -> Self {
Self {
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
input_metadata: None,
}
}
}
@@ -99,20 +118,146 @@ impl LoggedSessionHistoryMetadata {
/// Persisted item and metadata are one value so transforms cannot reorder or
/// truncate one without the other.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(try_from = "HistoryEntryWire")]
pub struct LoggedHistoryEntry {
pub item: LoggedItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct HistoryEntryWire {
item: LoggedItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<HistoryEntryWire> for LoggedHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: HistoryEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedHistoryEntry {
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
if let Some(input) = &self.metadata.input_metadata {
match &self.item {
LoggedItem::Message {
role: crate::LoggedRole::System,
content,
} => {
let [crate::LoggedContentPart::Text { text }] = content.as_slice() else {
return Err(crate::InputMetadataError::InconsistentRecord);
};
if text.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
// Other logical roles may carry use policy, but cannot masquerade
// as the System-only public/payload extension contract.
_ if input.feature_payload().is_some() || input.public_item().is_some() => {
return Err(crate::InputMetadataError::InconsistentRecord);
}
_ => {}
}
}
Ok(())
}
}
/// Typed system-item history record. The typed system event remains available
/// to client replay while its model-visible projection carries the same stable
/// metadata used by live history.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(try_from = "SystemEntryWire")]
pub struct LoggedSystemHistoryEntry {
pub item: crate::SystemItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct SystemEntryWire {
item: crate::SystemItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<SystemEntryWire> for LoggedSystemHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: SystemEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedSystemHistoryEntry {
pub fn from_input(
origin: LoggedSessionHistoryOrigin,
derivation: Option<LoggedHistoryDerivation>,
input: crate::SystemInputEnvelope,
) -> Self {
let (body, input_metadata) = input.into_parts();
Self {
item: crate::SystemItem::Input { body },
metadata: LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId::new(),
origin,
derivation,
input_metadata: Some(input_metadata),
},
}
}
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
match (&self.item, &self.metadata.input_metadata) {
(crate::SystemItem::Input { body }, Some(_)) => {
if body.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
(crate::SystemItem::Input { .. }, None) => {
return Err(crate::InputMetadataError::MissingMetadata);
}
(_, Some(_)) => return Err(crate::InputMetadataError::InconsistentRecord),
(_, None) => {}
}
Ok(())
}
pub fn to_history_entry(&self) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: self.item.to_history_item().into(),
metadata: self.metadata.clone(),
}
}
pub fn public_item(&self) -> crate::PublicSystemItem {
if matches!(self.item, crate::SystemItem::Input { .. }) {
return self
.metadata
.input_metadata
.as_ref()
.and_then(crate::HistoryInputMetadata::public_item)
.cloned()
.unwrap_or_else(|| self.item.public_item());
}
self.item.public_item()
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -130,6 +275,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -148,6 +294,64 @@ mod tests {
);
}
#[test]
fn external_system_records_reject_missing_policy_role_mismatch_and_overlimit_body() {
let valid = serde_json::json!({
"item": { "kind": "input", "body": "context" },
"metadata": {
"entry_id": "stable-entry", "origin": { "kind": "provided", "producer": "owner" },
"input_metadata": { "usage": {
"recording_material": "exclude", "memory_evidence": "exclude",
"observation": "hidden", "compaction": "reference_input"
} }
}
});
let restored: LoggedSystemHistoryEntry = serde_json::from_value(valid.clone()).unwrap();
let flat = serde_json::to_value(restored.to_history_entry()).unwrap();
assert!(serde_json::from_value::<LoggedHistoryEntry>(flat.clone()).is_ok());
for generic in [true, false] {
let wire = if generic { &valid } else { &flat };
let decode = |wire| {
if generic {
serde_json::from_value::<LoggedSystemHistoryEntry>(wire).is_err()
} else {
serde_json::from_value::<LoggedHistoryEntry>(wire).is_err()
}
};
let mut missing = wire.clone();
missing["metadata"]
.as_object_mut()
.unwrap()
.remove("input_metadata");
assert!(decode(missing));
let mut absent_policy = wire.clone();
absent_policy["metadata"]["input_metadata"]
.as_object_mut()
.unwrap()
.remove("usage");
assert!(decode(absent_policy));
let mut invalid_role = wire.clone();
invalid_role["item"] =
serde_json::to_value(LoggedItem::from(agen::Item::user_message("not system")))
.unwrap();
invalid_role["metadata"]["input_metadata"]["public"] = serde_json::json!({
"kind": "context", "body": "public system projection", "presentation": {"kind": "context"}
});
assert!(decode(invalid_role));
let mut oversized = wire.clone();
if generic {
oversized["item"]["body"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
} else {
oversized["item"]["content"][0]["text"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
}
assert!(decode(oversized));
}
}
#[test]
fn annotated_segment_start_is_restore_visible_without_projecting_metadata() {
let session_id = uuid::Uuid::now_v7();
+563
View File
@@ -0,0 +1,563 @@
//! Validated, Feature-neutral history-input policy and System extensions.
//!
//! The enclosing history annotation owns identity, origin and derivation. None of
//! those descriptive facts selects a use policy here. Model text, independent use
//! policies, private Feature data and an explicit public projection travel as one
//! value. Restoring this value does not load a Feature or invoke domain behavior.
//!
//! Payload validation bounds the decoded value; enclosing log readers must also
//! enforce their record allocation limit before decoding untrusted persisted input.
use agen::Item;
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::PublicSystemItem;
pub const MAX_FEATURE_PAYLOAD_BYTES: usize = 128 * 1024;
pub const MAX_SYSTEM_INPUT_BYTES: usize = 4 * 1024 * 1024;
pub const MAX_SYSTEM_PUBLIC_BYTES: usize = 256 * 1024;
const MAX_SCHEMA_BYTES: usize = 128;
const MAX_JSON_DEPTH: usize = 16;
const MAX_JSON_NODES: usize = 8192;
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum InputMetadataError {
#[error("history input payload schema is invalid")]
InvalidSchema,
#[error("history input payload version must be nonzero")]
InvalidVersion,
#[error("system input resource limit exceeded")]
ResourceLimit,
#[error("history input payload could not be serialized")]
Serialization,
#[error("history input payload schema or version is unsupported by this consumer")]
UnsupportedPayload,
#[error("history input payload does not match the owner's typed contract")]
InvalidTypedPayload,
#[error("provided input is missing explicit metadata")]
MissingMetadata,
#[error("input metadata does not match its logical record")]
InconsistentRecord,
}
/// One private payload owned by a Feature's versioned typed decoder.
/// Unknown valid schemas/versions remain data, never execution instructions.
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(try_from = "PayloadWire")]
pub struct FeatureInputPayload {
schema: String,
version: u32,
value: Value,
}
impl std::fmt::Debug for FeatureInputPayload {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("FeatureInputPayload")
.field("schema", &self.schema)
.field("version", &self.version)
.finish_non_exhaustive()
}
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct PayloadWire {
schema: String,
version: u32,
value: Value,
}
impl TryFrom<PayloadWire> for FeatureInputPayload {
type Error = InputMetadataError;
fn try_from(wire: PayloadWire) -> Result<Self, Self::Error> {
if wire.schema.is_empty()
|| wire.schema.len() > MAX_SCHEMA_BYTES
|| !wire.schema.as_bytes()[0].is_ascii_lowercase()
|| !wire.schema.bytes().all(|byte| {
byte.is_ascii_lowercase() || byte.is_ascii_digit() || b"-_.:/".contains(&byte)
})
{
return Err(InputMetadataError::InvalidSchema);
}
if wire.version == 0 {
return Err(InputMetadataError::InvalidVersion);
}
validate_json(&wire.value, MAX_FEATURE_PAYLOAD_BYTES)?;
Ok(Self {
schema: wire.schema,
version: wire.version,
value: wire.value,
})
}
}
impl FeatureInputPayload {
pub fn encode<T: Serialize>(
schema: impl Into<String>,
version: u32,
value: &T,
) -> Result<Self, InputMetadataError> {
PayloadWire {
schema: schema.into(),
version,
value: serde_json::to_value(value).map_err(|_| InputMetadataError::Serialization)?,
}
.try_into()
}
pub fn schema(&self) -> &str {
&self.schema
}
pub fn version(&self) -> u32 {
self.version
}
/// The owner explicitly selects its contract before decoding. A valid unknown
/// payload can be retained and round-tripped without calling this operation.
pub fn decode<T: DeserializeOwned>(
&self,
schema: &str,
version: u32,
) -> Result<T, InputMetadataError> {
if self.schema != schema || self.version != version {
return Err(InputMetadataError::UnsupportedPayload);
}
serde_json::from_value(self.value.clone())
.map_err(|_| InputMetadataError::InvalidTypedPayload)
}
}
/// Eligibility is supplied independently for recording and Memory evidence.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputEvidenceUse {
Include,
Exclude,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputObservationUse {
Readable,
Hidden,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputCompactionUse {
/// Ordinary history eligible for the selected retained tail.
Retain,
/// A full opening summary consumed separately and superseded by compaction.
OpeningSummary,
/// Full historical context for the compactor, not an accumulating retained tail.
ReferenceInput,
}
/// No permissive defaults: malformed/unknown policies cannot turn into eligibility.
/// Public presentation is deliberately not one of these use decisions.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct HistoryInputUsePolicy {
pub recording_material: InputEvidenceUse,
pub memory_evidence: InputEvidenceUse,
pub observation: InputObservationUse,
pub compaction: InputCompactionUse,
}
/// Use policy is independent of logical role. Non-System derived Tool context
/// carries policy only; private Feature payload and public presentation belong
/// to the System envelope and are rejected on other roles at the history boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "MetadataWire")]
pub struct HistoryInputMetadata {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct MetadataWire {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
impl TryFrom<MetadataWire> for HistoryInputMetadata {
type Error = InputMetadataError;
fn try_from(wire: MetadataWire) -> Result<Self, Self::Error> {
if let Some(public) = &wire.public {
// Kind is a descriptive label, not a Feature dispatch or authority key.
if public.kind.is_empty()
|| public.kind.len() > MAX_SCHEMA_BYTES
|| public.kind.chars().any(char::is_control)
{
return Err(InputMetadataError::InvalidSchema);
}
let value =
serde_json::to_value(public).map_err(|_| InputMetadataError::Serialization)?;
validate_json(&value, MAX_SYSTEM_PUBLIC_BYTES)?;
}
Ok(Self {
usage: wire.usage,
feature_payload: wire.feature_payload,
public: wire.public,
})
}
}
impl HistoryInputMetadata {
pub fn new(
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
) -> Result<Self, InputMetadataError> {
MetadataWire {
usage,
feature_payload,
public,
}
.try_into()
}
pub fn usage(&self) -> HistoryInputUsePolicy {
self.usage
}
pub fn feature_payload(&self) -> Option<&FeatureInputPayload> {
self.feature_payload.as_ref()
}
pub fn public_item(&self) -> Option<&PublicSystemItem> {
self.public.as_ref()
}
pub fn public_use_policy(&self) -> protocol::SessionEntryUsePolicy {
protocol::SessionEntryUsePolicy {
observation_readable: self.usage.observation == InputObservationUse::Readable,
memory_evidence: self.usage.memory_evidence == InputEvidenceUse::Include,
}
}
pub fn public_system_use_policy(&self) -> protocol::SessionEntryUsePolicy {
if self.public.is_some() {
self.public_use_policy()
} else {
protocol::SessionEntryUsePolicy {
observation_readable: false,
memory_evidence: false,
}
}
}
pub fn public_projection(&self) -> PublicSystemItem {
self.public.clone().unwrap_or_else(|| PublicSystemItem {
kind: "input".into(),
body: String::new(),
presentation: crate::PublicSystemPresentation::Ignored,
})
}
}
/// System role is invariant; provenance and use policies are attached data.
/// An absent public projection never implicitly publishes the model body.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "InputWire")]
pub struct SystemInputEnvelope {
body: String,
metadata: HistoryInputMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct InputWire {
body: String,
metadata: HistoryInputMetadata,
}
impl TryFrom<InputWire> for SystemInputEnvelope {
type Error = InputMetadataError;
fn try_from(wire: InputWire) -> Result<Self, Self::Error> {
if wire.body.len() > MAX_SYSTEM_INPUT_BYTES {
return Err(InputMetadataError::ResourceLimit);
}
Ok(Self {
body: wire.body,
metadata: wire.metadata,
})
}
}
impl SystemInputEnvelope {
pub fn new(
body: impl Into<String>,
metadata: HistoryInputMetadata,
) -> Result<Self, InputMetadataError> {
InputWire {
body: body.into(),
metadata,
}
.try_into()
}
pub fn body(&self) -> &str {
&self.body
}
pub fn metadata(&self) -> &HistoryInputMetadata {
&self.metadata
}
pub fn to_history_item(&self) -> Item {
Item::system_message(self.body.clone())
}
pub fn into_parts(self) -> (String, HistoryInputMetadata) {
(self.body, self.metadata)
}
}
fn validate_json(value: &Value, max_bytes: usize) -> Result<(), InputMetadataError> {
fn walk(value: &Value, depth: usize, remaining: &mut usize) -> Result<(), InputMetadataError> {
if depth > MAX_JSON_DEPTH || *remaining == 0 {
return Err(InputMetadataError::ResourceLimit);
}
*remaining -= 1;
match value {
Value::Array(values) => {
for value in values {
walk(value, depth + 1, remaining)?;
}
}
Value::Object(values) => {
for value in values.values() {
walk(value, depth + 1, remaining)?;
}
}
_ => {}
}
Ok(())
}
let mut remaining = MAX_JSON_NODES;
walk(value, 0, &mut remaining)?;
let bytes = serde_json::to_vec(value).map_err(|_| InputMetadataError::Serialization)?;
if bytes.len() > max_bytes {
return Err(InputMetadataError::ResourceLimit);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{LoggedItem, LoggedRole, PublicSystemPresentation};
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct OwnerPayload {
source_digest: String,
omitted: bool,
}
fn reference_policy() -> HistoryInputUsePolicy {
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
}
}
#[test]
fn unknown_valid_payload_round_trips_without_domain_decode_or_public_exposure() {
let payload = FeatureInputPayload::encode(
"future-owner:context",
99,
&OwnerPayload {
source_digest: "private-digest".into(),
omitted: true,
},
)
.unwrap();
let envelope = SystemInputEnvelope::new(
"private model context",
HistoryInputMetadata::new(reference_policy(), Some(payload), None).unwrap(),
)
.unwrap();
let restored: SystemInputEnvelope =
serde_json::from_slice(&serde_json::to_vec(&envelope).unwrap()).unwrap();
assert_eq!(restored.body(), "private model context");
assert!(restored.metadata().public_item().is_none());
let saved_payload = restored.metadata().feature_payload().unwrap();
assert_eq!(saved_payload.schema(), "future-owner:context");
assert_eq!(saved_payload.version(), 99);
assert_eq!(
saved_payload.decode::<OwnerPayload>("known-owner:context", 1),
Err(InputMetadataError::UnsupportedPayload)
);
assert_eq!(
serde_json::to_value(&restored).unwrap(),
serde_json::to_value(&envelope).unwrap()
);
assert!(!format!("{saved_payload:?}").contains("private-digest"));
}
#[test]
fn owner_decodes_typed_payload_only_for_its_explicit_contract() {
let input = OwnerPayload {
source_digest: "rev".into(),
omitted: false,
};
let payload = FeatureInputPayload::encode("owner:context", 1, &input).unwrap();
assert_eq!(
payload.decode::<OwnerPayload>("owner:context", 1).unwrap(),
input
);
let invalid = FeatureInputPayload::encode(
"owner:context",
1,
&serde_json::json!({ "omitted": "not a bool" }),
)
.unwrap();
assert_eq!(
invalid.decode::<OwnerPayload>("owner:context", 1),
Err(InputMetadataError::InvalidTypedPayload)
);
}
#[test]
fn malformed_or_oversized_payload_is_rejected_at_construction_and_restore() {
for (schema, version, value) in [
("", 1, Value::Null),
("invalid schema", 1, Value::Null),
("owner:context", 0, Value::Null),
(
"owner:context",
1,
Value::String("x".repeat(MAX_FEATURE_PAYLOAD_BYTES)),
),
(
"owner:context",
1,
Value::Array(vec![Value::Null; MAX_JSON_NODES]),
),
(
"owner:context",
1,
(0..=MAX_JSON_DEPTH).fold(Value::Null, |inner, _| Value::Array(vec![inner])),
),
] {
assert!(FeatureInputPayload::encode(schema, version, &value).is_err());
assert!(
serde_json::from_value::<FeatureInputPayload>(
serde_json::json!({ "schema": schema, "version": version, "value": value })
)
.is_err()
);
}
}
#[test]
fn policy_is_required_and_unknown_or_malformed_fields_fail_closed() {
let valid = serde_json::to_value(reference_policy()).unwrap();
for field in [
"recording_material",
"memory_evidence",
"observation",
"compaction",
] {
let mut absent = valid.clone();
absent.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<HistoryInputUsePolicy>(absent).is_err());
let mut unknown = valid.clone();
unknown[field] = Value::String("future-allow".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(unknown).is_err());
}
let mut extra = valid;
extra["feature_name"] = Value::String("infer-policy-from-me".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(extra).is_err());
}
#[test]
fn feature_name_and_model_body_do_not_select_policies_or_public_projection() {
for (schema, body) in [
("owner:journal", "past context"),
("unrelated:skill", "do not extract this; legitimate wording"),
] {
let public = PublicSystemItem {
kind: "context-label".into(),
body: "explicit public text".into(),
presentation: PublicSystemPresentation::Context,
};
let metadata = HistoryInputMetadata::new(
reference_policy(),
Some(
FeatureInputPayload::encode(
schema,
1,
&OwnerPayload {
source_digest: "private".into(),
omitted: false,
},
)
.unwrap(),
),
Some(public),
)
.unwrap();
let input = SystemInputEnvelope::new(body, metadata).unwrap();
assert_eq!(input.metadata().usage(), reference_policy());
let history: LoggedItem = input.to_history_item().into();
assert!(matches!(
history,
LoggedItem::Message {
role: LoggedRole::System,
..
}
));
assert_eq!(input.to_history_item().as_text(), Some(body));
let public = serde_json::to_string(input.metadata().public_item().unwrap()).unwrap();
assert!(!public.contains("private"));
assert!(!public.contains(schema));
assert!(!public.contains(body));
}
}
#[test]
fn use_policies_remain_independent_of_each_other_and_of_public_display() {
let usage = HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Include,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Readable,
compaction: InputCompactionUse::Retain,
};
let metadata = HistoryInputMetadata::new(usage, None, None).unwrap();
let restored: HistoryInputMetadata =
serde_json::from_value(serde_json::to_value(&metadata).unwrap()).unwrap();
assert_eq!(restored.usage(), usage);
assert!(restored.public_item().is_none());
assert!(restored.feature_payload().is_none());
}
#[test]
fn model_body_and_public_projection_have_independent_resource_limits() {
let metadata = HistoryInputMetadata::new(reference_policy(), None, None).unwrap();
assert!(
SystemInputEnvelope::new("x".repeat(MAX_SYSTEM_INPUT_BYTES + 1), metadata).is_err()
);
let public = PublicSystemItem {
kind: "context".into(),
body: "x".repeat(MAX_SYSTEM_PUBLIC_BYTES),
presentation: PublicSystemPresentation::Context,
};
assert!(HistoryInputMetadata::new(reference_policy(), None, Some(public)).is_err());
let invalid = serde_json::json!({ "body": "small", "metadata": { "public": null, "feature_payload": null } });
assert!(serde_json::from_value::<SystemInputEnvelope>(invalid).is_err());
}
}
+413 -21
View File
@@ -1,12 +1,15 @@
//! Versioned decoder for Session schemas that predate canonical annotated history.
//! Versioned decoder for Session schemas v1-5, including frozen Feature wire.
//!
//! These types are intentionally private to `session-store`. Current writers,
//! replay, and public projections use [`crate::LogEntry`] exclusively; only the
//! Worker Session schema migration is allowed to deserialize these shapes.
//! explicit Session schema migration is allowed to deserialize these shapes.
use agen::llm_client::types::RequestConfig;
use protocol::Segment;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
#[cfg(test)]
mod tests;
use crate::{
LogEntry, LoggedHistoryEntry, LoggedItem, LoggedSessionHistoryEntryId,
@@ -14,6 +17,212 @@ use crate::{
SegmentOrigin, SessionExtension, SessionId, SystemItem,
};
// Frozen v1-4 Feature wire. These types are deliberately not exported and are
// never used by LogEntry/SystemItem's current deserializers. Payload structs
// retain the owner's v1 shape rather than erasing typed fields into arbitrary JSON.
#[derive(Debug, Serialize, Deserialize)]
struct JournalStartupPayload {
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: crate::PromptRenderProvenance,
}
#[derive(Debug, Serialize, Deserialize)]
struct ResidentSummaryPayload {
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SubjectBehaviorPayload {
#[serde(default)]
behavior_md: String,
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SkillActivationPayload {
name: String,
}
#[derive(Debug, Serialize, Deserialize)]
struct InvocationResultPayload {
result: protocol::FeatureInvocationResult,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyFeatureSystemItem {
JournalStartup {
body: String,
#[serde(flatten)]
payload: JournalStartupPayload,
},
ResidentSummaryRefresh {
body: String,
#[serde(flatten)]
payload: ResidentSummaryPayload,
},
SubjectBehaviorRefresh {
body: String,
#[serde(flatten)]
payload: SubjectBehaviorPayload,
},
SkillActivation {
body: String,
#[serde(flatten)]
payload: SkillActivationPayload,
},
FeatureInvocationResult {
body: String,
#[serde(flatten)]
payload: InvocationResultPayload,
},
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacySystemItem {
Feature(LegacyFeatureSystemItem),
// Includes already-migrated Input records for interrupted migration retry.
Core(SystemItem),
}
impl LegacySystemItem {
fn into_entry(
self,
mut metadata: LoggedSessionHistoryMetadata,
) -> Result<LoggedSystemHistoryEntry, crate::InputMetadataError> {
use crate::InputCompactionUse::{ReferenceInput, Retain};
let item = match self {
Self::Core(item) => item,
Self::Feature(feature) => {
metadata.validate_input_metadata()?;
if metadata.input_metadata.is_some() {
return Err(crate::InputMetadataError::InconsistentRecord);
}
let (body, input_metadata) = match feature {
LegacyFeatureSystemItem::JournalStartup { body, payload } => {
migrated_feature_input(
body,
"journal_startup",
"journal:startup",
payload,
ReferenceInput,
)?
}
LegacyFeatureSystemItem::ResidentSummaryRefresh { body, payload } => {
migrated_feature_input(
body,
"resident_summary_refresh",
"memory:resident-summary",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SubjectBehaviorRefresh { body, payload } => {
migrated_feature_input(
body,
"subject_behavior_refresh",
"subject:behavior",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SkillActivation { body, payload } => {
migrated_feature_input(
body,
"skill_activation",
"skill:activation",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::FeatureInvocationResult { body, payload } => {
migrated_feature_input(
body,
"feature_invocation_result",
"feature:invocation-result",
payload,
Retain,
)?
}
};
// Identity/origin/derivation are retained, never re-inferred.
metadata.input_metadata = Some(input_metadata);
SystemItem::Input { body }
}
};
let entry = LoggedSystemHistoryEntry { item, metadata };
entry.validate_input_metadata()?;
Ok(entry)
}
}
fn migrated_feature_input<T: Serialize>(
body: String,
label: &str,
schema: &str,
payload: T,
compaction: crate::InputCompactionUse,
) -> Result<(String, crate::HistoryInputMetadata), crate::InputMetadataError> {
let metadata = crate::HistoryInputMetadata::new(
migration_policy(compaction),
Some(crate::FeatureInputPayload::encode(schema, 1, &payload)?),
Some(crate::PublicSystemItem {
kind: label.into(),
body: body.clone(),
presentation: crate::PublicSystemPresentation::Context,
}),
)?;
Ok((body, metadata))
}
/// Annotated old wire used by v3/v4, and by already-converted files when an
/// older marker survived an interrupted migration. No ordinary decoder uses it.
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyAnnotatedLogEntry {
AnnotatedSystemItem {
ts: u64,
entry: LegacySystemHistoryEntry,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct LegacySystemHistoryEntry {
item: LegacySystemItem,
metadata: LoggedSessionHistoryMetadata,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacyAnnotatedOrCurrent {
Annotated(LegacyAnnotatedLogEntry),
Current(LogEntry),
}
impl LegacyAnnotatedOrCurrent {
fn into_current(self) -> Result<LogEntry, crate::InputMetadataError> {
match self {
Self::Annotated(LegacyAnnotatedLogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
}) => Ok(LogEntry::AnnotatedSystemItem {
ts,
entry: entry.item.into_entry(entry.metadata)?,
extensions,
}),
Self::Current(entry) => Ok(entry),
}
}
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyHistoryLogEntry {
@@ -44,7 +253,9 @@ enum LegacyHistoryLogEntry {
},
SystemItem {
ts: u64,
item: SystemItem,
item: LegacySystemItem,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
@@ -54,7 +265,7 @@ enum LegacyHistoryLogEntry {
#[serde(untagged)]
enum LegacySessionLogEntryV1 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
/// Schema v2 retained the v1 history shapes while adding non-history records.
@@ -64,7 +275,7 @@ enum LegacySessionLogEntryV1 {
#[serde(untagged)]
enum LegacySessionLogEntryV2 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
pub(crate) fn decode_entry(
@@ -74,28 +285,207 @@ pub(crate) fn decode_entry(
segment_id: SegmentId,
line_index: usize,
) -> Result<LogEntry, serde_json::Error> {
let mut wire: serde_json::Value = serde_json::from_str(line)?;
migrate_legacy_metadata_field(&mut wire)?;
let entry = match schema_version {
1 => match serde_json::from_str::<LegacySessionLogEntryV1>(line)? {
1 => match serde_json::from_value::<LegacySessionLogEntryV1>(wire)? {
LegacySessionLogEntryV1::History(entry) => Entry::History(entry),
LegacySessionLogEntryV1::Current(entry) => Entry::Current(entry),
},
2 => match serde_json::from_str::<LegacySessionLogEntryV2>(line)? {
2 => match serde_json::from_value::<LegacySessionLogEntryV2>(wire)? {
LegacySessionLogEntryV2::History(entry) => Entry::History(entry),
LegacySessionLogEntryV2::Current(entry) => Entry::Current(entry),
},
3 | 4 => Entry::Current(serde_json::from_value(wire)?),
5 => return serde_json::from_value(wire),
_ => unreachable!("legacy decoder called for unsupported schema {schema_version}"),
};
Ok(match entry {
match entry {
Entry::History(entry) => {
canonicalize_history_entry(session_id, segment_id, line_index, entry)
}
Entry::Current(entry) => entry,
})
Entry::Current(entry) => entry.into_current(),
}
.map_err(serde::de::Error::custom)
}
// The v5 System-only metadata field was widened to role-neutral input metadata
// in v6. Touch only known history slots, never arbitrary Feature payload objects.
// Both keys are an ambiguous source, not permission to discard either value.
fn migrate_legacy_metadata_field(record: &mut serde_json::Value) -> Result<(), serde_json::Error> {
fn entry(value: &mut serde_json::Value) -> Result<(), serde_json::Error> {
if let Some(metadata) = value
.get_mut("metadata")
.and_then(serde_json::Value::as_object_mut)
{
if let Some(old) = metadata.remove("system_input") {
if metadata.contains_key("input_metadata") {
return Err(serde::de::Error::custom(
"ambiguous legacy input metadata fields",
));
}
metadata.insert("input_metadata".into(), old);
}
}
Ok(())
}
let kind = record
.get("kind")
.and_then(serde_json::Value::as_str)
.unwrap_or_default()
.to_owned();
if matches!(
kind.as_str(),
"annotated_system_item"
| "annotated_assistant_item"
| "annotated_tool_result"
| "tool_result_corrected"
) {
if let Some(value) = record.get_mut("entry") {
entry(value)?;
}
}
if let Some(history) = record
.get_mut("history")
.and_then(serde_json::Value::as_array_mut)
{
for value in history {
entry(value)?;
}
}
Ok(())
}
/// Schema 3 used DerivedSummary for Host compaction seeds. Only the trusted
/// creation boundary may identify one; arbitrary text and later seed items do not.
/// Include already-upgraded entries so an interrupted multi-file migration resumes.
pub(crate) fn compaction_summary_ids<'a>(
entries: impl Iterator<Item = &'a LogEntry>,
) -> std::collections::HashSet<LoggedSessionHistoryEntryId> {
entries
.filter_map(|entry| {
let LogEntry::AnnotatedSegmentStart {
compacted_from: Some(_),
forked_from: None,
history,
..
} = entry
else {
return None;
};
let first = history.first()?;
(matches!(
first.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
) && matches!(
first.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) && agen::Item::from(first.item.clone())
.as_text()
.is_some_and(|text| text.starts_with("[Compacted context summary]\n\n")))
.then(|| first.metadata.entry_id.clone())
})
.collect()
}
pub(crate) fn migrate_compaction_origins(
entries: &mut [LogEntry],
summaries: &std::collections::HashSet<LoggedSessionHistoryEntryId>,
) {
// Only inherited seed copies can share these identities. Appended input is
// not a construction boundary and must never be reclassified from its text.
for entry in entries {
if let LogEntry::AnnotatedSegmentStart { history, .. } = entry {
for entry in history {
if summaries.contains(&entry.metadata.entry_id)
&& entry.metadata.origin == LoggedSessionHistoryOrigin::DerivedSummary
{
entry.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
}
}
}
}
}
/// Explicit legacy-to-current boundary. No Feature inference belongs in ordinary
/// readers/restoration. Invalid or oversized payloads fail migration; the generic
/// payload/body/public limits are deliberately not relaxed for legacy data.
pub(crate) fn migrate_system_inputs(entry: &mut LogEntry) -> Result<(), crate::InputMetadataError> {
entry.validate_input_metadata()?;
match entry {
// Feature wire has already been converted by the explicit old-schema
// decoder. Ordinary LogEntry can contain only generic Input/core events.
LogEntry::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
LogEntry::AnnotatedSegmentStart { history, .. }
| LogEntry::AnnotatedUserInput { history, .. } => {
for entry in history {
migrate_flattened_system_input(entry)?;
}
}
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. }
| LogEntry::ToolResultCorrected { entry, .. } => {
migrate_flattened_system_input(entry)?;
}
_ => {}
}
entry.validate_input_metadata()
}
fn migration_policy(compaction: crate::InputCompactionUse) -> crate::HistoryInputUsePolicy {
crate::HistoryInputUsePolicy {
recording_material: crate::InputEvidenceUse::Exclude,
memory_evidence: crate::InputEvidenceUse::Exclude,
observation: crate::InputObservationUse::Hidden,
compaction,
}
}
fn migrate_flattened_system_input(
entry: &mut LoggedHistoryEntry,
) -> Result<(), crate::InputMetadataError> {
// Check before supplying legacy policy: Provided without metadata is a
// malformed NEW record, not permission to synthesize a producer policy.
entry.validate_input_metadata()?;
if entry.metadata.input_metadata.is_none()
&& (matches!(
entry.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) || matches!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
))
{
let compaction = match entry.metadata.origin {
LoggedSessionHistoryOrigin::JournalContext => crate::InputCompactionUse::ReferenceInput,
LoggedSessionHistoryOrigin::CompactionSummary => {
crate::InputCompactionUse::OpeningSummary
}
_ => crate::InputCompactionUse::Retain,
};
// Legacy derived context was excluded from independent evidence. Preserve
// that established decision explicitly, including synthetic Tool roles;
// never guess from Tool names or body text. Payload/public stay absent.
entry.metadata.input_metadata = Some(crate::HistoryInputMetadata::new(
migration_policy(compaction),
None,
None,
)?);
}
entry.validate_input_metadata()
}
enum Entry {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
fn legacy_metadata(
@@ -108,6 +498,7 @@ fn legacy_metadata(
identity.extend_from_slice(&(line_index as u64).to_be_bytes());
identity.extend_from_slice(&(item_index as u64).to_be_bytes());
LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(format!(
"l-{}",
base64::Engine::encode(&base64::engine::general_purpose::URL_SAFE_NO_PAD, identity)
@@ -122,8 +513,8 @@ fn canonicalize_history_entry(
segment_id: SegmentId,
line_index: usize,
entry: LegacyHistoryLogEntry,
) -> LogEntry {
match entry {
) -> Result<LogEntry, crate::InputMetadataError> {
Ok(match entry {
LegacyHistoryLogEntry::SegmentStart {
ts,
session_id,
@@ -177,13 +568,14 @@ fn canonicalize_history_entry(
metadata: legacy_metadata(segment_id, line_index, 0),
},
},
LegacyHistoryLogEntry::SystemItem { ts, item } => LogEntry::AnnotatedSystemItem {
LegacyHistoryLogEntry::SystemItem {
ts,
entry: LoggedSystemHistoryEntry {
item,
metadata: legacy_metadata(segment_id, line_index, 0),
},
extensions: Vec::new(),
item,
extensions,
} => LogEntry::AnnotatedSystemItem {
ts,
entry: item.into_entry(legacy_metadata(segment_id, line_index, 0))?,
extensions,
},
}
})
}
@@ -0,0 +1,105 @@
use super::*;
mod v5;
fn seed(
origin: LoggedSessionHistoryOrigin,
compacted: bool,
role: crate::LoggedRole,
text: &str,
) -> LogEntry {
LogEntry::AnnotatedSegmentStart {
ts: 1,
session_id: SessionId::now_v7(),
system_prompt: None,
config: Default::default(),
forked_from: None,
compacted_from: compacted.then(|| SegmentOrigin {
segment_id: SegmentId::now_v7(),
at_turn_index: 0,
}),
history: vec![LoggedHistoryEntry {
item: LoggedItem::Message {
role,
content: vec![crate::LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("summary".into()),
origin,
derivation: Some(crate::LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source".into())],
}),
},
}],
}
}
#[test]
fn only_trusted_compaction_seed_is_migrated_and_retries_update_inherited_copies() {
let text = "[Compacted context summary]\n\nsummary";
for (compacted, role, text) in [
(false, crate::LoggedRole::System, text),
(true, crate::LoggedRole::User, text),
(
true,
crate::LoggedRole::System,
"[Session TaskStore snapshot] tasks",
),
] {
let entry = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
compacted,
role,
text,
);
assert!(compaction_summary_ids([&entry].into_iter()).is_empty());
}
for origin in [
LoggedSessionHistoryOrigin::DerivedSummary,
LoggedSessionHistoryOrigin::CompactionSummary,
] {
let creation = seed(origin, true, crate::LoggedRole::System, text);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
text,
);
let LogEntry::AnnotatedSegmentStart { history, .. } = &inherited else {
panic!()
};
let mut pasted = history[0].clone();
pasted.metadata.entry_id = LoggedSessionHistoryEntryId("pasted".into());
let appended = LogEntry::AnnotatedUserInput {
ts: 2,
history: vec![pasted.clone()],
segments: vec![],
extensions: vec![],
};
let mut entries = vec![creation, inherited, appended];
let ids = compaction_summary_ids(entries.iter());
migrate_compaction_origins(&mut entries, &ids);
for entry in &entries[..2] {
let LogEntry::AnnotatedSegmentStart { history, .. } = entry else {
panic!()
};
assert_eq!(
history[0].metadata.origin,
LoggedSessionHistoryOrigin::CompactionSummary
);
assert_eq!(history[0].metadata.entry_id.0, "summary");
assert_eq!(
history[0].metadata.derivation.as_ref().unwrap().sources[0].0,
"source"
);
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[2] else {
panic!()
};
assert_eq!(history, &[pasted]);
let once = serde_json::to_vec(&entries).unwrap();
migrate_compaction_origins(&mut entries, &ids);
assert_eq!(serde_json::to_vec(&entries).unwrap(), once);
}
}
@@ -0,0 +1,3 @@
{"kind":"system_item","ts":1,"item":{"kind":"skill_activation","name":"legacy-skill","body":"legacy skill body"}}
{"kind":"user_input","ts":2,"segments":[{"kind":"text","content":"legacy user body"}],"extensions":[{"domain":"old","payload":{"kept":true}}]}
{"kind":"assistant_item","ts":3,"item":{"kind":"message","role":"assistant","content":[{"kind":"text","text":"legacy reply"}]}}
@@ -0,0 +1,8 @@
{"kind":"annotated_system_item","ts":101,"entry":{"item":{"kind":"journal_startup","body":"journal body\nunchanged","snapshot":{"entries":[{"text":"private snapshot"}]},"snapshot_sha256":"snapshot-digest","prompt_provenance":{"workspace_id":"workspace","source_digest":"source","projection_digest":"projection","logical_name":"journal_startup"}},"metadata":{"entry_id":"journal-id","origin":{"kind":"legacy_unknown"},"derivation":{"sources":["source-id"]}}},"extensions":[{"domain":"migration-fixture","payload":{"kept":true}}]}
{"kind":"annotated_system_item","ts":102,"entry":{"item":{"kind":"resident_summary_refresh","body":"resident body"},"metadata":{"entry_id":"resident-id","origin":{"kind":"derived_summary"}}}}
{"kind":"annotated_system_item","ts":103,"entry":{"item":{"kind":"subject_behavior_refresh","body":"rendered subject body","behavior_md":"exact behavior","prompt_provenance":{"source_digest":"subject-source","projection_digest":"subject-projection","logical_name":"subject_behavior"}},"metadata":{"entry_id":"subject-id","origin":{"kind":"backend_instruction","operation_id":"op"}}}}
{"kind":"annotated_system_item","ts":104,"entry":{"item":{"kind":"skill_activation","body":"skill body","name":"example-skill"},"metadata":{"entry_id":"skill-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":105,"entry":{"item":{"kind":"feature_invocation_result","body":"result body","result":{"invocation_id":"invoke-id","identity":"builtin:test/prepare","status":"outcome_unknown","message":"may have committed","context":null}},"metadata":{"entry_id":"result-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_user_input","ts":106,"segments":[],"history":[{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Journal] prefix is not Feature evidence"}]},"metadata":{"entry_id":"flat-journal","origin":{"kind":"journal_context"},"derivation":{"sources":["journal-id"]}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"summary body"}]},"metadata":{"entry_id":"flat-summary","origin":{"kind":"compaction_summary"}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Skill: example] never infer Feature"}]},"metadata":{"entry_id":"flat-other","origin":{"kind":"legacy_unknown"}}}]}
{"kind":"annotated_system_item","ts":107,"entry":{"item":{"kind":"notification","message":"raw message","body":"wrapped message"},"metadata":{"entry_id":"core-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":108,"entry":{"item":{"kind":"input","body":"already generic body"},"metadata":{"entry_id":"generic-id","origin":{"kind":"provided","producer":"future-owner"},"derivation":{"sources":["source-id"]},"system_input":{"usage":{"recording_material":"include","memory_evidence":"exclude","observation":"readable","compaction":"retain"},"feature_payload":{"schema":"future:unknown","version":99,"value":{"opaque":"retained"}},"public":null}}}}
@@ -0,0 +1,5 @@
{"kind":"annotated_system_item","ts":201,"entry":{"item":{"kind":"input","body":"frozen v5 reference"},"metadata":{"entry_id":"reference-v5","origin":{"kind":"provided","producer":"owner-v5"},"system_input":{"usage":{"recording_material":"exclude","memory_evidence":"exclude","observation":"hidden","compaction":"reference_input"},"feature_payload":{"schema":"future:owner","version":8,"value":{"system_input":"opaque owner field must not be renamed"}},"public":null}}}}
{"kind":"annotated_assistant_item","ts":202,"entry":{"item":{"kind":"tool_call","call_id":"original-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"original-call-entry","origin":{"kind":"model_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_tool_result","ts":203,"entry":{"item":{"kind":"tool_result","call_id":"original-call","summary":"original result"},"metadata":{"entry_id":"original-result-entry","origin":{"kind":"tool_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_assistant_item","ts":204,"entry":{"item":{"kind":"tool_call","call_id":"derived-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"derived-call-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
{"kind":"annotated_tool_result","ts":205,"entry":{"item":{"kind":"tool_result","call_id":"derived-call","summary":"derived retrieval result"},"metadata":{"entry_id":"derived-result-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
@@ -0,0 +1,709 @@
//! Fixed old wire bytes, not fixtures generated by the current serializer.
use super::*;
use crate::{FsStore, InputCompactionUse, Store, WorkerSessionStore};
use serde_json::{Value, json};
use std::{fs, path::Path};
const V4: &str = include_str!("fixtures/v4-system-inputs.jsonl");
const V1: &str = include_str!("fixtures/v1-system-inputs.jsonl");
fn decode_fixture(version: u32, source: &str, segment: SegmentId) -> Vec<LogEntry> {
source
.lines()
.enumerate()
.map(|(index, line)| decode_entry(version, line, SessionId::nil(), segment, index).unwrap())
.collect()
}
fn encode(entries: &[LogEntry]) -> Vec<u8> {
let mut bytes = Vec::new();
for entry in entries {
serde_json::to_writer(&mut bytes, entry).unwrap();
bytes.push(b'\n');
}
bytes
}
fn migrate(entries: &mut [LogEntry]) {
for entry in entries {
migrate_system_inputs(entry).unwrap();
entry.validate_input_metadata().unwrap();
}
}
fn write_worker(root: &Path, version: u32, source: &[u8]) -> (SessionId, SegmentId) {
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
fs::create_dir_all(root.join("segments")).unwrap();
fs::write(
root.join("session.json"),
serde_json::to_vec(&json!({
"schema_version": version, "session_id": session,
}))
.unwrap(),
)
.unwrap();
fs::write(
root.join("segments").join(format!("{segment}.jsonl")),
source,
)
.unwrap();
(session, segment)
}
fn assert_marker(root: &Path, version: u32) {
let manifest: Value =
serde_json::from_slice(&fs::read(root.join("session.json")).unwrap()).unwrap();
assert_eq!(manifest["schema_version"], version);
}
#[test]
fn v4_fixed_wire_preserves_identity_body_payload_public_and_independent_policy() {
let mut entries = decode_fixture(4, V4, SegmentId::nil());
let before = entries.clone();
migrate(&mut entries);
let schemas = [
"journal:startup",
"memory:resident-summary",
"subject:behavior",
"skill:activation",
"feature:invocation-result",
];
let old_wire: Vec<Value> = V4
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
for index in 0..5 {
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = &entries[index]
else {
panic!()
};
let old = &old_wire[index];
let old_item = &old["entry"]["item"];
let old_metadata: LoggedSessionHistoryMetadata =
serde_json::from_value(old["entry"]["metadata"].clone()).unwrap();
let old_ext: Vec<crate::SessionExtension> = old
.get("extensions")
.map(|value| serde_json::from_value(value.clone()).unwrap())
.unwrap_or_default();
assert_eq!(*ts, old["ts"].as_u64().unwrap());
assert_eq!(extensions, &old_ext);
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
old_item["body"].as_str().unwrap()
);
assert_eq!(entry.metadata.entry_id, old_metadata.entry_id);
assert_eq!(entry.metadata.origin, old_metadata.origin);
assert_eq!(entry.metadata.derivation, old_metadata.derivation);
let metadata = entry.metadata.input_metadata.as_ref().unwrap();
let public = metadata.public_item().unwrap();
assert_eq!(public.kind, old_item["kind"].as_str().unwrap());
assert_eq!(public.body, old_item["body"].as_str().unwrap());
assert_eq!(
public.presentation,
crate::PublicSystemPresentation::Context
);
assert_eq!(
metadata.usage(),
migration_policy(if index == 0 {
InputCompactionUse::ReferenceInput
} else {
InputCompactionUse::Retain
})
);
let payload = metadata.feature_payload().unwrap();
assert_eq!(payload.schema(), schemas[index]);
assert_eq!(payload.version(), 1);
let value: Value = payload.decode(schemas[index], 1).unwrap();
// Independently require each frozen owner's typed v1 payload contract.
match index {
0 => {
let _: JournalStartupPayload = payload.decode(schemas[index], 1).unwrap();
}
1 => {
let _: ResidentSummaryPayload = payload.decode(schemas[index], 1).unwrap();
}
2 => {
let _: SubjectBehaviorPayload = payload.decode(schemas[index], 1).unwrap();
}
3 => {
let _: SkillActivationPayload = payload.decode(schemas[index], 1).unwrap();
}
4 => {
let _: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
}
_ => unreachable!(),
}
let mut expected = old_item.clone();
expected.as_object_mut().unwrap().remove("kind");
expected.as_object_mut().unwrap().remove("body");
// Optional provenance is an explicit Option in the v1 payload contract.
if index == 1 {
expected["prompt_provenance"] = Value::Null;
}
if index == 4 {
let result: protocol::FeatureInvocationResult =
serde_json::from_value(old_item["result"].clone()).unwrap();
let typed: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
assert_eq!(typed.result, result);
// Preserve the protocol's typed serialization contract, including
// its canonical representation of optional result context.
expected["result"] = serde_json::to_value(result).unwrap();
}
assert_eq!(value, expected);
assert!(value.get("body").is_none());
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[5] else {
panic!()
};
let LogEntry::AnnotatedUserInput { history: old, .. } = &before[5] else {
panic!()
};
for (index, compaction) in [
InputCompactionUse::ReferenceInput,
InputCompactionUse::OpeningSummary,
InputCompactionUse::Retain,
]
.into_iter()
.enumerate()
{
assert_eq!(history[index].item, old[index].item);
assert_eq!(
history[index].metadata.entry_id,
old[index].metadata.entry_id
);
assert_eq!(history[index].metadata.origin, old[index].metadata.origin);
assert_eq!(
history[index].metadata.derivation,
old[index].metadata.derivation
);
let metadata = history[index].metadata.input_metadata.as_ref().unwrap();
assert_eq!(metadata.usage(), migration_policy(compaction));
assert!(metadata.public_item().is_none());
assert!(metadata.feature_payload().is_none());
}
// Core notification is unchanged; existing generic Input and unknown valid
// owner payload are retained without decoding/reclassifying a Feature.
assert_eq!(encode(&entries[6..]), encode(&before[6..]));
let once = encode(&entries);
migrate(&mut entries);
assert_eq!(encode(&entries), once);
}
#[test]
fn v1_through_v4_worker_migration_reopen_and_mixed_file_retry() {
for version in 1..=4 {
let source = if version <= 2 { V1 } else { V4 };
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), version, source.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
// Simulate an interrupted publication: v5 records precede legacy records
// in one file, and a fully new file coexists with an untouched old file.
let mut converted = decode_fixture(version, source, segment);
migrate(&mut converted);
let mut mixed = encode(&converted[..1]);
mixed.extend_from_slice(source.split_once('\n').unwrap().1.as_bytes());
fs::write(&path, &mixed).unwrap();
let new_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let new_bytes = encode(&converted);
fs::write(&new_path, &new_bytes).unwrap();
let old_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(3)));
fs::write(&old_path, source).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
WorkerSessionStore::validate_migration(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), mixed);
assert_eq!(fs::read(&old_path).unwrap(), source.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let store = WorkerSessionStore::new(root.path()).unwrap();
assert_marker(root.path(), 6);
let restored = store.read_all(session, segment).unwrap();
assert_eq!(encode(&restored), new_bytes);
assert_eq!(fs::read(&new_path).unwrap(), new_bytes);
let once = fs::read(&path).unwrap();
let old_once = fs::read(&old_path).unwrap();
drop(store);
let reopened = WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(encode(&reopened.read_all(session, segment).unwrap()), once);
assert_eq!(fs::read(&path).unwrap(), once);
assert_eq!(fs::read(&old_path).unwrap(), old_once);
}
}
#[test]
fn all_segments_preflight_keeps_old_bytes_on_decode_or_conversion_failure() {
let mut oversized: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
oversized["entry"]["item"]["snapshot"] =
json!({"large": "x".repeat(crate::input_metadata::MAX_FEATURE_PAYLOAD_BYTES + 1)});
let mut unknown: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
unknown["entry"]["item"]["kind"] = json!("unsupported_future_feature");
let mut malformed = unknown.clone();
malformed["entry"]["item"] = json!({"kind":"journal_startup", "body":"missing snapshot"});
let mut missing_metadata: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing_metadata["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let mut missing_input = missing_metadata.clone();
missing_input["entry"]["metadata"]["origin"] = json!({"kind":"legacy_unknown"});
let mut invalid_policy: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
invalid_policy["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("unknown");
let mut invalid_payload = invalid_policy.clone();
invalid_payload["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("retain");
invalid_payload["entry"]["metadata"]["system_input"]["feature_payload"]["version"] = json!(0);
let mut flat_provided: Value = serde_json::from_str(V4.lines().nth(5).unwrap()).unwrap();
flat_provided["history"][0]["metadata"]["origin"] =
json!({"kind":"provided", "producer":"owner"});
for bad in [
"{broken-json}\n".to_owned(),
format!("{}\n", oversized),
format!("{}\n", unknown),
format!("{}\n", malformed),
format!("{}\n", missing_metadata),
format!("{}\n", invalid_policy),
format!("{}\n", invalid_payload),
format!("{}\n", flat_provided),
format!("{}\n", missing_input),
] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 4, V4.as_bytes());
let first = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&later, &bad).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::validate_migration(root.path()).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read(&later).unwrap(), bad.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn unsupported_schema_preserves_all_old_bytes() {
for version in [0, 7, 99] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V4.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), V4.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn standalone_store_uses_same_v1_through_v4_migration_and_reopens() {
// None represents the existing unmarked annotated standalone layout (v3).
for version in [None, Some(1), Some(2), Some(3), Some(4)] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let source = if version.is_some_and(|v| v <= 2) {
V1
} else {
V4
};
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, source).unwrap();
if let Some(version) = version {
fs::write(dir.join("session-schema.json"), version.to_string()).unwrap();
}
let store = FsStore::new(root.path()).unwrap();
let entries = store.read_all(session, segment).unwrap();
for entry in entries {
entry.validate_input_metadata().unwrap();
}
assert_eq!(
fs::read_to_string(dir.join("session-schema.json"))
.unwrap()
.trim(),
"6"
);
let once = fs::read(&path).unwrap();
drop(store);
FsStore::new(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), once);
}
}
#[test]
fn compaction_origin_upgrade_is_retained_only_for_pre_v4_sources() {
for version in [3, 4] {
let root = tempfile::tempdir().unwrap();
let creation = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
true,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let (session, segment) =
write_worker(root.path(), version, &encode(&[creation, inherited]));
let store = WorkerSessionStore::new(root.path()).unwrap();
for record in store.read_all(session, segment).unwrap() {
let LogEntry::AnnotatedSegmentStart { history, .. } = record else {
panic!()
};
let metadata = &history[0].metadata;
assert_eq!(metadata.entry_id.0, "summary");
assert_eq!(metadata.derivation.as_ref().unwrap().sources[0].0, "source");
assert_eq!(
metadata.origin,
if version == 3 {
LoggedSessionHistoryOrigin::CompactionSummary
} else {
LoggedSessionHistoryOrigin::DerivedSummary
}
);
assert_eq!(
metadata.input_metadata.as_ref().unwrap().usage().compaction,
if version == 3 {
InputCompactionUse::OpeningSummary
} else {
InputCompactionUse::Retain
}
);
}
}
}
#[test]
fn standalone_preflight_and_unsupported_marker_leave_sources_unchanged() {
for version in [4, 99] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let first = dir.join(format!("{}.jsonl", SegmentId::from_u128(1)));
let later = dir.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&first, V4).unwrap();
fs::write(&later, "{broken-json}\n").unwrap();
let marker = version.to_string();
fs::write(dir.join("session-schema.json"), &marker).unwrap();
assert!(FsStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read_to_string(&later).unwrap(), "{broken-json}\n");
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
marker
);
}
}
#[test]
fn current_v5_rejects_all_legacy_feature_kinds_without_rewriting_sources() {
for line in V4.lines().take(5) {
// Current serde, writable open and read-only restore share the same
// current wire contract. None may dispatch to the old Feature decoder.
assert!(serde_json::from_str::<LogEntry>(line).is_err());
let bytes = format!("{line}\n");
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), 6, bytes.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
let readonly = WorkerSessionStore::open_read_only(root.path()).unwrap();
assert!(readonly.read_all_read_only(session, segment).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let standalone = tempfile::tempdir().unwrap();
let dir = standalone.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, &bytes).unwrap();
fs::write(dir.join("session-schema.json"), "5").unwrap();
assert!(FsStore::new(standalone.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
"5"
);
}
}
#[test]
fn v1_v2_feature_wire_uses_the_same_typed_payloads_and_stable_legacy_identity() {
let segment = SegmentId::from_u128(42);
for version in [1, 2] {
for (index, line) in V4.lines().take(5).enumerate() {
let annotated: Value = serde_json::from_str(line).unwrap();
let wire = json!({
"kind": "system_item", "ts": annotated["ts"],
"item": annotated["entry"]["item"],
"extensions": [{"domain":"old-owner", "payload":{"kept":true}}],
})
.to_string();
let entry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
let retry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
assert_eq!(encode(&[entry.clone()]), encode(&[retry]));
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = entry
else {
panic!()
};
assert_eq!(ts, annotated["ts"].as_u64().unwrap());
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
annotated["entry"]["item"]["body"].as_str().unwrap()
);
assert_eq!(
entry.metadata.entry_id,
legacy_metadata(segment, index, 0).entry_id
);
assert_eq!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::LegacyUnknown
);
assert!(entry.metadata.derivation.is_none());
assert_eq!(extensions[0].domain, "old-owner");
assert_eq!(extensions[0].payload, json!({"kept":true}));
let LogEntry::AnnotatedSystemItem { entry: v4, .. } =
decode_entry(4, line, SessionId::nil(), segment, index).unwrap()
else {
panic!()
};
assert_eq!(entry.metadata.input_metadata, v4.metadata.input_metadata);
}
}
}
#[test]
fn migration_rejects_malformed_typed_feature_fields_instead_of_retaining_unchecked_json() {
let mut items: Vec<Value> = V4
.lines()
.take(5)
.map(|line| serde_json::from_str::<Value>(line).unwrap()["entry"]["item"].clone())
.collect();
items[0]["snapshot_sha256"] = json!(17);
items[1]["prompt_provenance"] = json!({"logical_name":"missing digests"});
items[2]["behavior_md"] = json!(["not a string"]);
items[3]["name"] = json!({"not":"a name"});
items[4]["result"]["status"] = json!("unsupported_status");
for version in 1..=4 {
for item in &items {
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"malformed-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).is_err());
}
}
}
#[test]
fn frozen_behavior_counter_is_accepted_only_at_the_old_schema_boundary() {
for version in 1..=4 {
let item = json!({
"kind":"subject_behavior_refresh", "body":"exact archived body",
"behavior_revision":7,
});
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"behavior-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(serde_json::from_str::<LogEntry>(&wire).is_err());
let LogEntry::AnnotatedSystemItem { entry, .. } =
decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).unwrap()
else {
panic!()
};
assert_eq!(entry.item.history_text(), "exact archived body");
let payload = entry
.metadata
.input_metadata
.as_ref()
.unwrap()
.feature_payload()
.unwrap();
let typed: SubjectBehaviorPayload = payload.decode("subject:behavior", 1).unwrap();
assert_eq!(typed.behavior_md, "");
assert!(typed.prompt_provenance.is_none());
assert_eq!(
serde_json::to_value(typed).unwrap(),
json!({
"behavior_md":"", "prompt_provenance":null,
})
);
}
}
#[test]
fn current_schema_rejects_generic_input_without_metadata_instead_of_migrating_it() {
let mut missing: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let bytes = format!("{}\n", missing);
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 6, bytes.as_bytes());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(
fs::read(
root.path()
.join("segments")
.join(format!("{segment}.jsonl"))
)
.unwrap(),
bytes.as_bytes()
);
assert_marker(root.path(), 6);
}
const V5: &str = include_str!("fixtures/v5-role-metadata.jsonl");
#[test]
fn v5_metadata_and_derived_tool_context_upgrade_without_changing_roles_or_original_evidence() {
for standalone in [false, true] {
let root = tempfile::tempdir().unwrap();
let (session, segment) = if standalone {
let session = SessionId::now_v7();
let segment = SegmentId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
fs::write(dir.join("session-schema.json"), b"5").unwrap();
fs::write(dir.join(format!("{segment}.jsonl")), V5).unwrap();
(session, segment)
} else {
write_worker(root.path(), 5, V5.as_bytes())
};
let entries = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
let originals: Vec<Value> = V5
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
assert_eq!(entries.len(), originals.len());
for (index, (entry, old)) in entries.iter().zip(originals).enumerate() {
let wire = serde_json::to_value(entry).unwrap();
assert_eq!(wire["ts"], old["ts"]);
assert_eq!(wire["entry"]["item"], old["entry"]["item"]);
for field in ["entry_id", "origin", "derivation"] {
assert_eq!(
wire["entry"]["metadata"][field],
old["entry"]["metadata"][field]
);
}
let metadata = &wire["entry"]["metadata"];
if index == 0 {
assert_eq!(
metadata["input_metadata"],
old["entry"]["metadata"]["system_input"]
);
assert_eq!(
metadata["input_metadata"]["feature_payload"]["value"]["system_input"],
"opaque owner field must not be renamed"
);
} else if index < 3 {
assert!(
metadata.get("input_metadata").is_none(),
"ordinary model Tool originals must not acquire exclusion policy"
);
} else {
assert_eq!(
metadata["input_metadata"]["usage"],
json!({
"recording_material":"exclude", "memory_evidence":"exclude", "observation":"hidden", "compaction":"retain"
})
);
}
}
let before = encode(&entries);
let reopened = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
assert_eq!(encode(&reopened), before);
}
}
#[test]
fn legacy_metadata_alias_conflict_fails_preflight_without_dropping_either_value() {
let mut bad: Value = serde_json::from_str(V5.lines().next().unwrap()).unwrap();
bad["entry"]["metadata"]["input_metadata"] = bad["entry"]["metadata"]["system_input"].clone();
for version in [4, 5] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V5.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let bad_bytes = format!("{bad}\n");
fs::write(&later, &bad_bytes).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(path).unwrap(), V5.as_bytes());
assert_eq!(fs::read(later).unwrap(), bad_bytes.as_bytes());
assert_marker(root.path(), version);
}
}
+9 -1
View File
@@ -41,11 +41,14 @@ pub(crate) fn read_without_atime_bounded(path: &Path, max_bytes: u64) -> io::Res
pub mod event_trace;
pub mod fs_store;
pub mod history;
pub mod input_metadata;
mod legacy_session_log;
pub mod logged_item;
mod paste_artifact;
pub mod public_index;
pub mod public_material;
pub mod public_snapshot;
pub mod retention;
pub mod segment;
pub mod segment_log;
pub mod store;
@@ -63,6 +66,10 @@ pub use history::{
LoggedSessionHistoryMetadata, LoggedSessionHistoryOrigin, LoggedSystemHistoryEntry,
LoggedWorkerSubject,
};
pub use input_metadata::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputMetadataError, InputObservationUse, SystemInputEnvelope,
};
pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged};
pub use paste_artifact::PasteArtifactLimits;
pub use public_index::{
@@ -96,7 +103,8 @@ pub use segment::{
pub use segment_log::{LogEntry, RestoredState, SegmentOrigin, SessionExtension, collect_state};
pub use store::{Store, StoreError};
pub use system_item::{
PromptRenderProvenance, SystemItem, SystemReminder, SystemReminderSource, render_worker_event,
PromptRenderProvenance, PublicSystemItem, PublicSystemPresentation, SystemItem, SystemReminder,
SystemReminderSource, render_worker_event,
};
pub use uploaded_file::{
DEFAULT_MAX_FILES_PER_SUBMISSION, DEFAULT_MAX_SESSION_ARTIFACT_BYTES,
+7 -1
View File
@@ -1296,6 +1296,7 @@ mod tests {
LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.to_owned()),
origin,
derivation: None,
@@ -1537,6 +1538,7 @@ mod tests {
prompt_provenance: None,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("system-item".into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction {
operation_id: None,
@@ -1602,6 +1604,7 @@ mod tests {
segment_id(20),
"completed",
LogEntry::RunCompleted {
entry_id: None,
ts: 4,
interrupted: false,
result: EngineResult::Finished,
@@ -1640,7 +1643,10 @@ mod tests {
(
segment_id(24),
"abandoned",
LogEntry::PausedTurnAbandoned { ts: 4 },
LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 4,
},
),
];
for (segment_id, text, outcome) in outcomes {
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+76 -21
View File
@@ -27,7 +27,7 @@ use thiserror::Error;
use crate::{
LogEntry, LoggedContentPart, LoggedHistoryEntry, LoggedItem, LoggedRole,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError, SystemItem,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError,
WorkerAggregateStore, WorkerSessionStore, WorkerStoreError,
};
@@ -1165,7 +1165,7 @@ fn capture_tool_correction(
let LogEntry::ToolResultCorrected { entry, .. } = record else {
return false;
};
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
// We read backwards: the first correction seen is the latest one.
corrections
.entry(entry.metadata.entry_id.0.clone())
@@ -1600,12 +1600,12 @@ pub(crate) fn project_session_snapshot_for_segment(
} => extend_history(&mut entries, history, Some(segments), *ts),
LogEntry::AnnotatedAssistantItem { ts, entry }
| LogEntry::AnnotatedToolResult { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
entries.push(history_entry(entry, *ts, data));
}
}
LogEntry::ToolResultCorrected { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
if let Some(existing) = entries
.iter_mut()
.find(|existing| existing.entry_id == entry.metadata.entry_id.0)
@@ -1618,11 +1618,16 @@ pub(crate) fn project_session_snapshot_for_segment(
}
}
LogEntry::AnnotatedSystemItem { ts, entry, .. } => entries.push(system_entry(
&entry.item,
&entry.public_item(),
entry.metadata.entry_id.0.clone(),
*ts,
provenance(&entry.metadata.origin),
derivation_ids(entry),
entry
.metadata
.input_metadata
.as_ref()
.map(crate::HistoryInputMetadata::public_system_use_policy),
)),
LogEntry::RunYielded {
ts,
@@ -1705,6 +1710,7 @@ pub(crate) fn project_session_snapshot_for_segment(
interrupted,
result,
active_run_turn_count: persisted_count,
..
} => {
if *interrupted && matches!(result, agen::EngineResult::Yielded) {
let count = (*persisted_count)
@@ -1774,7 +1780,7 @@ fn extend_history(
segments: input_segments.cloned().unwrap_or_default(),
}
} else {
let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) else {
let Some(data) = project_history_item(entry) else {
continue;
};
data
@@ -1789,6 +1795,19 @@ fn history_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: entry.metadata.input_metadata.as_ref().map(|input| {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
input.public_system_use_policy()
} else {
input.public_use_policy()
}
}),
entry_id: entry.metadata.entry_id.0.clone(),
timestamp,
provenance: provenance(&entry.metadata.origin),
@@ -1832,6 +1851,7 @@ fn run_transition_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: persisted_entry_id
.map(|entry_id| entry_id.0.clone())
.unwrap_or_else(|| legacy_entry_id(session_key, segment_id, log_index, 0)),
@@ -1851,6 +1871,7 @@ fn legacy_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: legacy_entry_id(session_key, segment_id, log_index, item_index),
timestamp,
provenance: SessionEntryProvenance::LegacyUnknown,
@@ -1883,12 +1904,12 @@ fn provenance(origin: &LoggedSessionHistoryOrigin) -> SessionEntryProvenance {
LoggedSessionHistoryOrigin::FlowInstruction { .. } => {
SessionEntryProvenance::FlowInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. } => {
SessionEntryProvenance::BackendInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. }
| LoggedSessionHistoryOrigin::Provided { .. } => SessionEntryProvenance::BackendInstruction,
LoggedSessionHistoryOrigin::ModelOutput { .. } => SessionEntryProvenance::ModelOutput,
LoggedSessionHistoryOrigin::ToolOutput { .. } => SessionEntryProvenance::ToolOutput,
LoggedSessionHistoryOrigin::CompactionSummary
| LoggedSessionHistoryOrigin::JournalContext
| LoggedSessionHistoryOrigin::DerivedSummary => SessionEntryProvenance::DerivedSummary,
LoggedSessionHistoryOrigin::LegacyUnknown => SessionEntryProvenance::LegacyUnknown,
}
@@ -1912,6 +1933,35 @@ pub fn session_tool_attachment_id(
URL_SAFE_NO_PAD.encode(hash.finalize())
}
fn project_history_item(entry: &LoggedHistoryEntry) -> Option<SessionSnapshotEntryData> {
if let Some(input) = &entry.metadata.input_metadata {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
let public = input.public_projection();
return Some(SessionSnapshotEntryData::SystemItem {
item_kind: public.kind.clone(),
content: public.body.clone(),
data: Some(
serde_json::to_value(public).expect("public system input is serializable"),
),
});
}
let usage = input.usage();
if usage.recording_material == crate::InputEvidenceUse::Exclude
&& usage.memory_evidence == crate::InputEvidenceUse::Exclude
&& usage.observation == crate::InputObservationUse::Hidden
{
return None;
}
}
project_item(&entry.metadata.entry_id.0, &entry.item)
}
fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntryData> {
match item {
LoggedItem::Message { role, content } => {
@@ -1984,30 +2034,26 @@ fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntr
}
fn system_entry(
item: &SystemItem,
item: &crate::PublicSystemItem,
entry_id: String,
timestamp: u64,
provenance: SessionEntryProvenance,
derived_from: Vec<String>,
use_policy: Option<protocol::SessionEntryUsePolicy>,
) -> SessionSnapshotEntry {
let mut data = serde_json::to_value(item).ok();
if let Some(serde_json::Value::Object(object)) = data.as_mut() {
object.remove("prompt_provenance");
}
let item_kind = data
.as_ref()
.and_then(|value| value.get("kind"))
.and_then(serde_json::Value::as_str)
.unwrap_or("system_item")
.to_owned();
let public = item.clone();
let data = Some(serde_json::to_value(&public).expect("public system input is serializable"));
let item_kind = public.kind;
let content = public.body;
SessionSnapshotEntry {
use_policy,
entry_id,
timestamp,
provenance,
derived_from,
data: SessionSnapshotEntryData::SystemItem {
item_kind,
content: item.history_text(),
content,
data,
},
}
@@ -2037,6 +2083,7 @@ mod tests {
fn complete(ts: u64) -> LogEntry {
LogEntry::RunCompleted {
entry_id: None,
ts,
interrupted: false,
result: agen::EngineResult::Finished,
@@ -2288,6 +2335,7 @@ mod tests {
content: vec![LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(entry_id.into()),
origin,
derivation: None,
@@ -3586,6 +3634,7 @@ mod tests {
.map(|item| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -3640,6 +3689,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id,
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -3709,6 +3759,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::FlowInstruction {
selector: "builtin:coder-review".into(),
@@ -3728,6 +3779,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id.clone(),
origin: origin.clone(),
derivation: Some(LoggedHistoryDerivation {
@@ -3758,6 +3810,7 @@ mod tests {
fn annotated_projection_preserves_identity_and_provenance() {
let session_id = crate::new_session_id();
let metadata = LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::ModelOutput {
worker: LoggedWorkerSubject {
@@ -4320,6 +4373,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("ancestor-image".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4414,6 +4468,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("tool-images".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -103,6 +103,7 @@ fn finish(
stats.insert(
anchor.clone(),
SessionSnapshotEntry {
use_policy: None,
entry_id: format!("run-stats:{anchor}"),
timestamp: ended_at,
provenance: SessionEntryProvenance::LegacyUnknown,
+398
View File
@@ -0,0 +1,398 @@
//! Durable, domain-neutral Session retention pins and permanent erasure fences.
//!
//! Hosts choose a control root whose lifetime outlasts the physical Session.
//! Every deletion path must execute its physical removal inside `delete`, and
//! publishers must hold `lock_live` while creating or appending source data.
//! A check followed by unlocked removal is not a retention guarantee.
//!
//! Pins have explicit owners and no timer/expiry: domain code releases its pin
//! only after its durable consumer no longer needs originals. A permanent erase
//! persists its fence before physical removal, invalidates pins, and rejects
//! late pins/publication even after reopening the control store. Retrying an
//! interrupted physical erase uses the same intent and is safe.
use std::collections::BTreeSet;
use std::fs::{self, File, OpenOptions};
use std::path::{Path, PathBuf};
use fs4::fs_std::FileExt;
use serde::{Deserialize, Serialize};
use crate::{SessionId, StoreError};
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(transparent)]
pub struct SessionRetentionPin(pub uuid::Uuid);
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SessionDeletionIntent {
Ordinary,
ErasePermanently,
}
#[derive(Debug, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct RetentionState {
erased: bool,
pins: BTreeSet<SessionRetentionPin>,
}
#[derive(Debug, Clone)]
pub struct SessionRetentionStore {
root: PathBuf,
}
/// A process-independent lock; dropping it releases the filesystem lock.
/// Do not recursively acquire the same Session lock inside a guarded callback.
pub struct SessionRetentionGuard {
_file: File,
}
impl SessionRetentionStore {
pub fn new(control_root: impl Into<PathBuf>) -> Self {
Self {
root: control_root.into(),
}
}
fn state_path(&self, session: SessionId) -> PathBuf {
self.root.join(format!("{session}.json"))
}
fn lock(&self, session: SessionId) -> Result<SessionRetentionGuard, StoreError> {
fs::create_dir_all(&self.root)?;
let mut options = OpenOptions::new();
options.read(true).write(true).create(true).truncate(false);
#[cfg(any(target_os = "linux", target_os = "android"))]
{
use std::os::unix::fs::OpenOptionsExt;
// Control roots are Host-owned; never follow a replaced lock file.
options.custom_flags(libc::O_NOFOLLOW);
}
let file = options.open(self.root.join(format!("{session}.lock")))?;
FileExt::lock_exclusive(&file)?;
Ok(SessionRetentionGuard { _file: file })
}
fn read(&self, session: SessionId) -> Result<RetentionState, StoreError> {
match fs::read(self.state_path(session)) {
Ok(bytes) => Ok(serde_json::from_slice(&bytes)?),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
Ok(RetentionState::default())
}
Err(error) => Err(error.into()),
}
}
fn save(&self, session: SessionId, state: &RetentionState) -> Result<(), StoreError> {
crate::worker_session_store::atomic_write_bytes(
&self.state_path(session),
&serde_json::to_vec(state)?,
)
}
/// Observational fence check; unlike `lock_live`, never creates storage.
/// Hosts still fence result adoption in their own durable consumer: a read
/// finishing before erasure is not authority for a delayed domain write.
pub fn ensure_live(&self, session: SessionId) -> Result<(), StoreError> {
if self.read(session)?.erased {
return Err(StoreError::SessionErased(session));
}
Ok(())
}
pub fn lock_live(&self, session: SessionId) -> Result<SessionRetentionGuard, StoreError> {
let guard = self.lock(session)?;
if self.read(session)?.erased {
return Err(StoreError::SessionErased(session));
}
Ok(guard)
}
/// Protect a Host-owned Session before its first source publication. This is
/// retention intent, not proof that any original material exists. The owner must
/// establish it before accepting input and use the same control store for writes
/// and deletion. Source-serving APIs must still verify actual material separately.
pub fn retain_before_publication(
&self,
session: SessionId,
pin: SessionRetentionPin,
) -> Result<(), StoreError> {
let _guard = self.lock_live(session)?;
let mut state = self.read(session)?;
if state.pins.insert(pin) {
self.save(session, &state)?;
}
Ok(())
}
/// Call `source_exists` under the same lock used by deletion. Replaying the
/// same pin is idempotent; missing source never becomes a successful lease.
pub fn pin(
&self,
session: SessionId,
pin: SessionRetentionPin,
source_exists: impl FnOnce() -> Result<bool, StoreError>,
) -> Result<(), StoreError> {
let _guard = self.lock_live(session)?;
if !source_exists()? {
return Err(StoreError::SessionSourceMissing(session));
}
let mut state = self.read(session)?;
if state.pins.insert(pin) {
self.save(session, &state)?;
}
Ok(())
}
pub fn release(&self, session: SessionId, pin: SessionRetentionPin) -> Result<(), StoreError> {
let _guard = self.lock(session)?;
let mut state = self.read(session)?;
if state.pins.remove(&pin) {
self.save(session, &state)?;
}
Ok(())
}
pub fn pins(&self, session: SessionId) -> Result<Vec<SessionRetentionPin>, StoreError> {
let _guard = self.lock(session)?;
Ok(self.read(session)?.pins.into_iter().collect())
}
/// Execute physical removal while fencing concurrent pins and publishers.
/// An erasure fence is retained even if removal fails or the source is
/// already absent. Ordinary deletion after erasure is rejected: recovery
/// must preserve the caller's explicit erase intent.
pub fn delete<T>(
&self,
session: SessionId,
intent: SessionDeletionIntent,
remove: impl FnOnce() -> Result<T, StoreError>,
) -> Result<T, StoreError> {
let _guard = self.lock(session)?;
let mut state = self.read(session)?;
match intent {
SessionDeletionIntent::Ordinary => {
if state.erased {
return Err(StoreError::SessionErased(session));
}
if !state.pins.is_empty() {
return Err(StoreError::SessionPinned(session));
}
}
SessionDeletionIntent::ErasePermanently => {
if !state.erased || !state.pins.is_empty() {
state.erased = true;
state.pins.clear();
self.save(session, &state)?;
}
}
}
remove()
}
}
/// FsStore's control state is outside all per-Session source directories.
pub(crate) fn fs_retention(root: &Path) -> SessionRetentionStore {
SessionRetentionStore::new(root.join(".retention"))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn publication_intent_survives_reopen_without_attesting_missing_originals() {
let temp = tempfile::tempdir().unwrap();
let source = temp.path().join("source");
let controls = temp.path().join("controls");
let session = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let retention = SessionRetentionStore::new(&controls);
retention.retain_before_publication(session, pin).unwrap();
let reopened = SessionRetentionStore::new(&controls);
assert_eq!(reopened.pins(session).unwrap(), [pin]);
// An early owner is not a successful source-availability receipt.
assert!(
matches!(reopened.pin(session, pin, || Ok(source.is_file())), Err(StoreError::SessionSourceMissing(id)) if id == session)
);
{
let _publication = reopened.lock_live(session).unwrap();
fs::write(&source, "committed before notification").unwrap();
}
assert!(
matches!(reopened.delete(session, SessionDeletionIntent::Ordinary, || { fs::remove_file(&source)?; Ok(()) }), Err(StoreError::SessionPinned(id)) if id == session)
);
assert_eq!(
fs::read_to_string(&source).unwrap(),
"committed before notification"
);
reopened
.delete(session, SessionDeletionIntent::ErasePermanently, || {
fs::remove_file(&source)?;
Ok(())
})
.unwrap();
assert!(
matches!(retention.retain_before_publication(session, pin), Err(StoreError::SessionErased(id)) if id == session)
);
}
#[test]
fn pin_survives_reopen_blocks_delete_and_releases_idempotently() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let store = SessionRetentionStore::new(temp.path());
store.pin(id, pin, || Ok(true)).unwrap();
store.pin(id, pin, || Ok(true)).unwrap();
let reopened = SessionRetentionStore::new(temp.path());
assert_eq!(reopened.pins(id).unwrap(), [pin]);
assert!(matches!(
reopened.delete(id, SessionDeletionIntent::Ordinary, || Ok(())),
Err(StoreError::SessionPinned(_))
));
reopened.release(id, pin).unwrap();
reopened.release(id, pin).unwrap();
assert_eq!(
reopened
.delete(id, SessionDeletionIntent::Ordinary, || Ok(7))
.unwrap(),
7
);
}
#[test]
fn erase_failure_persists_fence_invalidates_pin_and_rejects_late_publication() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let store = SessionRetentionStore::new(temp.path());
store.pin(id, pin, || Ok(true)).unwrap();
assert!(
store
.delete(
id,
SessionDeletionIntent::ErasePermanently,
|| Err::<(), _>(std::io::Error::other("injected removal failure").into())
)
.is_err()
);
let reopened = SessionRetentionStore::new(temp.path());
assert!(reopened.pins(id).unwrap().is_empty());
assert!(matches!(
reopened.pin(id, pin, || Ok(true)),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.lock_live(id),
Err(StoreError::SessionErased(_))
));
reopened
.delete(id, SessionDeletionIntent::ErasePermanently, || Ok(()))
.unwrap();
reopened.release(id, pin).unwrap();
assert!(matches!(
reopened.lock_live(id),
Err(StoreError::SessionErased(_))
));
}
#[test]
fn fs_normal_delete_obeys_pin_and_permanent_erase_fences_all_source_writes() {
use crate::{FsStore, LogEntry, Store};
let temp = tempfile::tempdir().unwrap();
let store = FsStore::new(temp.path()).unwrap();
let session = crate::new_session_id();
let segment = crate::new_segment_id();
let entry = LogEntry::TurnEnd {
ts: 1,
turn_count: 1,
};
store
.create_segment(session, segment, &[entry.clone()])
.unwrap();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
store.pin_session(session, pin).unwrap();
assert!(matches!(
store.delete_session(session),
Err(StoreError::SessionPinned(_))
));
assert!(store.exists(session, segment).unwrap());
store
.delete_session_with_intent(session, SessionDeletionIntent::ErasePermanently)
.unwrap();
let reopened = FsStore::new(temp.path()).unwrap();
assert!(!reopened.exists(session, segment).unwrap());
assert!(matches!(
reopened.pin_session(session, pin),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.append(session, segment, &entry),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.create_segment(session, segment, &[entry]),
Err(StoreError::SessionErased(_))
));
assert!(
!reopened
.delete_session_with_intent(session, SessionDeletionIntent::ErasePermanently)
.unwrap()
);
assert!(!reopened.exists(session, segment).unwrap());
}
#[test]
fn independent_handles_serialize_delete_with_late_pin() {
let temp = tempfile::tempdir().unwrap();
let source = temp.path().join("source");
fs::create_dir(&source).unwrap();
let control = temp.path().join("control");
let session = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let (entered_tx, entered_rx) = std::sync::mpsc::channel();
let (release_tx, release_rx) = std::sync::mpsc::channel();
let control2 = control.clone();
let source2 = source.clone();
let delete = std::thread::spawn(move || {
SessionRetentionStore::new(control2).delete(
session,
SessionDeletionIntent::Ordinary,
|| {
entered_tx.send(()).unwrap();
release_rx
.recv_timeout(std::time::Duration::from_secs(10))
.unwrap();
fs::remove_dir(&source2)?;
Ok(())
},
)
});
entered_rx
.recv_timeout(std::time::Duration::from_secs(10))
.unwrap();
let late = std::thread::spawn(move || {
SessionRetentionStore::new(control).pin(session, pin, || Ok(source.is_dir()))
});
release_tx.send(()).unwrap();
delete.join().unwrap().unwrap();
assert!(matches!(
late.join().unwrap(),
Err(StoreError::SessionSourceMissing(_))
));
}
#[test]
fn missing_source_cannot_acquire_pin() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let store = SessionRetentionStore::new(temp.path());
assert!(matches!(
store.pin(id, SessionRetentionPin(uuid::Uuid::now_v7()), || Ok(false)),
Err(StoreError::SessionSourceMissing(_))
));
assert!(store.pins(id).unwrap().is_empty());
}
}
+1
View File
@@ -329,6 +329,7 @@ pub fn save_run_completed(
session_id,
segment_id,
LogEntry::RunCompleted {
entry_id: None,
ts: segment_log::now_millis(),
interrupted,
result,
+43 -6
View File
@@ -122,6 +122,8 @@ pub enum LogEntry {
/// Replay restores both interruption state and any resumable logical-run
/// turn budget.
RunCompleted {
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
ts: u64,
interrupted: bool,
result: EngineResult,
@@ -194,7 +196,11 @@ pub enum LogEntry {
/// A paused interrupted turn was explicitly abandoned without calling
/// `run()` or `resume()` again. Replay clears the interrupted marker so
/// the restored Worker is idle and future user input starts a normal new turn.
PausedTurnAbandoned { ts: u64 },
PausedTurnAbandoned {
ts: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
},
/// `RequestConfig` changed.
ConfigChanged { ts: u64, config: RequestConfig },
@@ -248,12 +254,31 @@ pub struct SegmentOrigin {
}
impl LogEntry {
/// Validate generic System records before a Store publishes any bytes.
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
match self {
Self::AnnotatedSegmentStart { history, .. }
| Self::AnnotatedUserInput { history, .. } => {
for entry in history {
entry.validate_input_metadata()?;
}
}
Self::AnnotatedAssistantItem { entry, .. }
| Self::AnnotatedToolResult { entry, .. }
| Self::ToolResultCorrected { entry, .. } => entry.validate_input_metadata()?,
Self::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
_ => {}
}
Ok(())
}
/// Allocate the durable public identity for a newly committed Run
/// transition. Calling this repeatedly is idempotent so the exact value
/// written to storage is also the value published to live subscribers.
pub fn ensure_session_entry_id(&mut self) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
Self::RunCompleted { entry_id, .. }
| Self::PausedTurnAbandoned { entry_id, .. }
| Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
@@ -275,7 +300,9 @@ impl LogEntry {
record_start: u64,
) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
Self::RunCompleted { entry_id, .. }
| Self::PausedTurnAbandoned { entry_id, .. }
| Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
@@ -527,6 +554,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -586,6 +614,7 @@ mod tests {
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 3200,
interrupted: false,
result: EngineResult::Finished,
@@ -893,12 +922,16 @@ mod tests {
compacted_from: None,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 100,
interrupted: true,
result: EngineResult::Paused,
active_run_turn_count: Some(1),
},
LogEntry::PausedTurnAbandoned { ts: 200 },
LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 200,
},
]);
assert!(!state.last_run_interrupted);
assert_eq!(state.active_run_turn_count, None);
@@ -1056,6 +1089,7 @@ mod tests {
turn_count: 2,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 300,
interrupted: true,
result: EngineResult::LimitReached,
@@ -1069,13 +1103,16 @@ mod tests {
#[test]
fn paused_turn_abandoned_entry_round_trip_via_json() {
let entry = LogEntry::PausedTurnAbandoned { ts: 12345 };
let entry = LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 12345,
};
let json = serde_json::to_string(&entry).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["kind"], "paused_turn_abandoned");
let decoded: LogEntry = serde_json::from_str(&json).unwrap();
match decoded {
LogEntry::PausedTurnAbandoned { ts } => assert_eq!(ts, 12345),
LogEntry::PausedTurnAbandoned { ts, .. } => assert_eq!(ts, 12345),
other => panic!("expected PausedTurnAbandoned, got {other:?}"),
}
}
+26
View File
@@ -27,12 +27,27 @@ pub enum StoreError {
#[error("serialization error: {0}")]
Serde(#[from] serde_json::Error),
#[error("invalid history input metadata: {0}")]
InvalidInputMetadata(#[from] crate::InputMetadataError),
#[error("segment not found: {0}")]
NotFound(SegmentId),
#[error("log corrupted at line {line}: {message}")]
Corrupt { line: usize, message: String },
#[error("session source is missing: {0}")]
SessionSourceMissing(SessionId),
#[error("session original material is retained by a durable consumer: {0}")]
SessionPinned(SessionId),
#[error("session was permanently erased: {0}")]
SessionErased(SessionId),
#[error("session source retention is unsupported by this store")]
SessionRetentionUnsupported,
#[error("retained observation read limit exceeded")]
ReadLimitExceeded,
@@ -80,6 +95,17 @@ pub enum StoreError {
/// take `(SessionId, SegmentId)` so segments can be physically grouped
/// per Session on disk (or per session_id in a DB).
pub trait Store: Send + Sync {
/// Establish durable retention intent before the Host accepts public input.
/// This does not attest source availability; it must use the same control store
/// as publishers and physical deletion. Unsupported stores fail closed.
fn retain_session_before_publication(
&self,
_session_id: SessionId,
_pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
Err(StoreError::SessionRetentionUnsupported)
}
/// Append a single log entry to the segment log.
///
/// One committed line per successful call. Implementations must not expose
+92 -93
View File
@@ -86,12 +86,16 @@ pub struct PromptRenderProvenance {
/// `notify_wrapper` template) re-shape the live rendering on a later
/// resume.
///
/// New variants get added here as fresh injection kinds come online
/// (e.g. `TaskReminder`). The `kind` JSON tag is the snake_case form of
/// the variant name.
/// Feature-owned inputs use `Input` plus explicit annotated metadata; adding a
/// Feature must not extend this enum or ordinary replay. Only core events have
/// dedicated variants. Pre-v5 Feature kinds exist exclusively in the migration
/// decoder, not in this current wire contract.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum SystemItem {
/// Generic System body; private metadata and public presentation are supplied
/// independently by the producer in the same annotated history record.
Input { body: String },
/// Free-form notification sent in by an external caller via
/// `Method::Notify`. `message` is the raw caller-supplied text;
/// `body` is the wrapped LLM-context form (Worker renders it via
@@ -114,14 +118,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Result of an explicitly selected chat Feature invocation. The typed
/// result is retained for clients; `body` is the exact bounded context made
/// visible to the Worker model.
FeatureInvocationResult {
result: protocol::FeatureInvocationResult,
body: String,
},
/// `@<path>` file reference resolution. `body` is the rendered
/// LLM-context text (`[File: <path>]\n…` for regular files,
/// `[Dir: <path>]\n…` for directory listings, possibly with a
@@ -129,10 +125,6 @@ pub enum SystemItem {
/// byte-identical to what was sent.
FileAttachment { path: String, body: String },
/// Explicit Agent Skill activation. `body` is the exact LLM-facing
/// Skill text committed before any subsequent LLM request can use it.
SkillActivation { name: String, body: String },
/// Historical persisted Knowledge reference resolution. Knowledge is no
/// longer active, so restored sessions ignore this item instead of replaying
/// archived record text into model context.
@@ -156,29 +148,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Restore-boundary refresh of the current resident Memory projection.
/// The exact rendered body is retained so replay is byte-identical; a
/// refresh with no ready summary explicitly supersedes older resident
/// context without deleting or rewriting history.
ResidentSummaryRefresh {
body: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// A Host-fetched change to user-managed Subject behavior. Each changed
/// body is appended rather than rewriting prior prompt/session content.
SubjectBehaviorRefresh {
body: String,
/// Exact user-managed behavior acknowledged after durable commit. Old
/// logs omit this value; an empty value is not an acknowledgement of a
/// nonempty behavior and does not replace their stored rendered body.
#[serde(default)]
behavior_md: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Synthetic note inserted after an interrupted turn before the next
/// user input. `body` is the exact LLM-context text explaining that the
/// previous turn was cut short.
@@ -189,23 +158,67 @@ pub enum SystemItem {
},
}
/// Allowlisted presentation data shared by live events and retained snapshots.
/// Private Feature payloads and render provenance never cross this boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct PublicSystemItem {
pub kind: String,
pub body: String,
pub presentation: PublicSystemPresentation,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PublicSystemPresentation {
Notification { message: String },
WorkerEvent { event: WorkerEvent },
Context,
TaskReminder,
Ignored,
}
impl SystemItem {
pub fn public_item(&self) -> PublicSystemItem {
let presentation = match self {
Self::Notification { message, .. } => PublicSystemPresentation::Notification {
message: message.clone(),
},
Self::WorkerEvent { event, .. } => PublicSystemPresentation::WorkerEvent {
event: event.clone(),
},
Self::TaskReminder { .. } => PublicSystemPresentation::TaskReminder,
Self::Input { .. }
| Self::LegacyKnowledgeIgnored { .. }
| Self::LegacyIgnored { .. } => PublicSystemPresentation::Ignored,
Self::FileAttachment { .. } | Self::Interrupt { .. } => {
PublicSystemPresentation::Context
}
};
PublicSystemItem {
kind: self.kind_label().to_owned(),
body: if matches!(presentation, PublicSystemPresentation::Ignored) {
String::new()
} else {
self.history_text()
},
presentation,
}
}
/// Free-text body the LLM sees inside its `role:system` message
/// for this item. Returns the variant's stored `body` verbatim.
pub fn history_text(&self) -> String {
match self {
SystemItem::Input { body } => body.clone(),
SystemItem::Notification { body, .. } => body.clone(),
SystemItem::WorkerEvent { body, .. } => body.clone(),
SystemItem::FeatureInvocationResult { body, .. } => body.clone(),
SystemItem::FileAttachment { body, .. } => body.clone(),
SystemItem::SkillActivation { body, .. } => body.clone(),
SystemItem::LegacyKnowledgeIgnored { .. } => String::new(),
SystemItem::LegacyIgnored { slug } => {
format!("Ignored legacy procedure item: /{slug}")
}
SystemItem::TaskReminder { body, .. } => body.clone(),
SystemItem::ResidentSummaryRefresh { body, .. } => body.clone(),
SystemItem::SubjectBehaviorRefresh { body, .. } => body.clone(),
SystemItem::Interrupt { body, .. } => body.clone(),
}
}
@@ -220,16 +233,13 @@ impl SystemItem {
/// wire — keep flexible.
pub fn kind_label(&self) -> &'static str {
match self {
SystemItem::Input { .. } => "input",
SystemItem::Notification { .. } => "notification",
SystemItem::WorkerEvent { .. } => "worker_event",
SystemItem::FeatureInvocationResult { .. } => "feature_invocation_result",
SystemItem::FileAttachment { .. } => "file_attachment",
SystemItem::SkillActivation { .. } => "skill_activation",
SystemItem::LegacyKnowledgeIgnored { .. } => "legacy_knowledge_ignored",
SystemItem::LegacyIgnored { .. } => "legacy_ignored",
SystemItem::TaskReminder { .. } => "task_reminder",
SystemItem::ResidentSummaryRefresh { .. } => "resident_summary_refresh",
SystemItem::SubjectBehaviorRefresh { .. } => "subject_behavior_refresh",
SystemItem::Interrupt { .. } => "interrupt",
}
}
@@ -287,33 +297,45 @@ mod tests {
}
#[test]
fn subject_behavior_refresh_round_trips_with_exact_body() {
let item = SystemItem::SubjectBehaviorRefresh {
body: "current Subject context".to_string(),
behavior_md: "exact behavior".to_string(),
prompt_provenance: None,
};
let raw = serde_json::to_string(&item).unwrap();
let parsed: SystemItem = serde_json::from_str(&raw).unwrap();
assert_eq!(parsed.history_text(), "current Subject context");
assert!(
matches!(parsed, SystemItem::SubjectBehaviorRefresh { behavior_md, .. } if behavior_md == "exact behavior")
);
fn current_system_item_rejects_frozen_feature_kinds() {
// Even well-formed old payloads cannot enter normal v5 decode/replay.
for line in include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl")
.lines()
.take(5)
{
let record: serde_json::Value = serde_json::from_str(line).unwrap();
assert!(serde_json::from_value::<SystemItem>(record["entry"]["item"].clone()).is_err());
assert!(
serde_json::from_value::<crate::LoggedSystemHistoryEntry>(record["entry"].clone())
.is_err()
);
assert!(serde_json::from_str::<crate::LogEntry>(line).is_err());
}
}
#[test]
fn frozen_subject_behavior_log_reads_old_counter_without_republishing_it() {
let old = r#"{"kind":"subject_behavior_refresh","body":"exact archived body","behavior_revision":7}"#;
let item: SystemItem = serde_json::from_str(old).unwrap();
assert_eq!(item.history_text(), "exact archived body");
assert_eq!(
serde_json::to_value(item).unwrap(),
serde_json::json!({
"kind": "subject_behavior_refresh",
"body": "exact archived body",
"behavior_md": ""
})
);
fn public_system_input_decoder_rejects_private_fields_and_unknown_presentations() {
let valid = serde_json::json!({
"kind": "new-feature-label",
"body": "explicit public content",
"presentation": { "kind": "context" },
});
let public: PublicSystemItem = serde_json::from_value(valid.clone()).unwrap();
assert_eq!(public.body, "explicit public content");
for private_field in [
"payload",
"prompt_provenance",
"policy",
"snapshot",
"behavior_md",
] {
let mut malformed = valid.clone();
malformed[private_field] = serde_json::json!({ "private": true });
assert!(serde_json::from_value::<PublicSystemItem>(malformed).is_err());
}
let mut unknown = valid;
unknown["presentation"] = serde_json::json!({ "kind": "execute_feature_payload" });
assert!(serde_json::from_value::<PublicSystemItem>(unknown).is_err());
}
#[test]
@@ -409,29 +431,6 @@ mod tests {
}
}
#[test]
fn invocation_result_retains_typed_outcome_and_exact_context() {
let result = protocol::FeatureInvocationResult {
invocation_id: "stable-invoke".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
status: protocol::FeatureInvocationStatus::OutcomeUnknown,
message: "operation may have committed".into(),
context: None,
};
let body = "Outcome unknown; do not retry automatically";
let item = SystemItem::FeatureInvocationResult {
result: result.clone(),
body: body.into(),
};
let restored: SystemItem =
serde_json::from_str(&serde_json::to_string(&item).unwrap()).unwrap();
assert_eq!(restored.history_text(), body);
assert_eq!(restored.kind_label(), "feature_invocation_result");
assert!(
matches!(restored, SystemItem::FeatureInvocationResult { result: saved, .. } if saved == result)
);
}
#[test]
fn round_trip_worker_event() {
let item = SystemItem::WorkerEvent {
+30 -43
View File
@@ -210,18 +210,6 @@ impl WorkerMetadata {
}
}
fn replace_active_session(metadata: &mut WorkerMetadata, active: Option<WorkerActiveSegmentRef>) {
if let Some(session_id) = active.as_ref().map(|active| active.session_id)
&& metadata
.subjektiv_session_attribution
.as_ref()
.is_some_and(|state| state.session_id() != session_id)
{
metadata.subjektiv_session_attribution = None;
}
metadata.active = active;
}
/// Sync persistence backend for Worker metadata.
pub trait WorkerMetadataStore: Send + Sync {
/// Create or replace metadata for its `worker_name` key.
@@ -297,7 +285,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -317,7 +305,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_id: Option<String>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -334,7 +322,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_root) = workspace_root {
metadata.workspace_root = Some(workspace_root);
@@ -342,18 +330,6 @@ pub trait WorkerMetadataStore: Send + Sync {
})
}
/// Set the Host-owned Subjektiv Session attribution lifecycle state while
/// preserving the active Session pointer and every unrelated field.
fn set_subjektiv_session_attribution(
&self,
worker_name: &str,
state: SubjektivSessionAttributionState,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.subjektiv_session_attribution = Some(state);
})
}
/// Set spawned-child registry state while preserving active pointer and manifest snapshot.
fn set_spawned_children(
&self,
@@ -806,6 +782,15 @@ where
S: crate::Store,
W: Send + Sync,
{
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), crate::StoreError> {
self.session_store
.retain_session_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
@@ -1078,13 +1063,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
});
})
.unwrap();
store
.set_active(
@@ -1108,7 +1093,7 @@ mod tests {
}
#[test]
fn new_active_session_clears_prior_session_attribution_state() {
fn generic_active_update_does_not_interpret_host_attribution() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let old_session_id = crate::new_session_id();
@@ -1120,13 +1105,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
});
})
.unwrap();
store
@@ -1142,7 +1127,9 @@ mod tests {
restored.active,
Some(WorkerActiveSegmentRef::pending_segment(new_session_id))
);
assert!(restored.subjektiv_session_attribution.is_none());
assert!(
matches!(restored.subjektiv_session_attribution, Some(SubjektivSessionAttributionState::Confirmed { session_id, .. }) if session_id == old_session_id)
);
}
#[test]
+529 -45
View File
@@ -36,7 +36,10 @@ use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::SystemTime;
const SESSION_SCHEMA_VERSION: u32 = 3;
const SESSION_SCHEMA_VERSION: u32 = 6;
const SYSTEM_INPUT_SESSION_SCHEMA_VERSION: u32 = 5;
const COMPACTION_SESSION_SCHEMA_VERSION: u32 = 4;
const ANNOTATED_SESSION_SCHEMA_VERSION: u32 = 3;
const PREVIOUS_SESSION_SCHEMA_VERSION: u32 = 2;
const LEGACY_SESSION_SCHEMA_VERSION: u32 = 1;
const SESSION_FILE: &str = "session.json";
@@ -48,6 +51,7 @@ pub struct WorkerSessionStore {
root: PathBuf,
session_id: Arc<Mutex<Option<SessionId>>>,
append_lock: Arc<Mutex<()>>,
retention: Option<crate::retention::SessionRetentionStore>,
}
#[derive(Debug, Serialize, Deserialize)]
@@ -72,6 +76,9 @@ pub(crate) struct RetainedSegmentReader {
read_end: u64,
buffer_start: u64,
buffer: Vec<u8>,
// Newest-to-oldest fragments of a record spanning multiple read chunks.
// Keep them separate until its boundary is found to avoid repeated prepend copies.
record_suffix: Vec<Vec<u8>>,
}
impl RetainedSegmentReader {
@@ -82,13 +89,19 @@ impl RetainedSegmentReader {
let mut read_bytes = 0_u64;
loop {
if !self.buffer.is_empty() {
if self.buffer.last() != Some(&b'\n') {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment cursor is not at a record boundary".to_string(),
});
}
let prior_newline = self.buffer[..self.buffer.len() - 1]
let search_end = if self.record_suffix.is_empty() {
if self.buffer.last() != Some(&b'\n') {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment cursor is not at a record boundary"
.to_string(),
});
}
self.buffer.len() - 1
} else {
self.buffer.len()
};
let prior_newline = self.buffer[..search_end]
.iter()
.rposition(|byte| *byte == b'\n');
if let Some(newline) =
@@ -100,9 +113,13 @@ impl RetainedSegmentReader {
newline + 1
};
let record_start = self.buffer_start + record_start_in_buffer as u64;
let record_end = self.buffer_start + self.buffer.len() as u64;
let record =
self.buffer[record_start_in_buffer..self.buffer.len() - 1].to_vec();
let record_end = self.read_end;
let mut record = Vec::with_capacity((record_end - record_start) as usize);
record.extend_from_slice(&self.buffer[record_start_in_buffer..]);
for fragment in self.record_suffix.drain(..).rev() {
record.extend_from_slice(&fragment);
}
record.pop(); // The final newline was checked before collecting fragments.
self.buffer.truncate(record_start_in_buffer);
self.read_end = record_start;
if record.is_empty() {
@@ -126,24 +143,17 @@ impl RetainedSegmentReader {
read_bytes,
));
}
// This chunk has no earlier boundary. Never scan or copy it again
// until the complete record can be assembled once.
self.record_suffix.push(std::mem::take(&mut self.buffer));
} else if self.read_end == 0 {
return Ok((None, read_bytes));
}
if self.buffer_start == 0 && !self.buffer.is_empty() {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment log does not end at a record boundary".to_string(),
});
}
if read_bytes >= max_additional_bytes {
return Err(StoreError::ReadLimitExceeded);
}
let end = if self.buffer.is_empty() {
self.read_end
} else {
self.buffer_start
};
let end = self.buffer_start;
let available = max_additional_bytes - read_bytes;
let chunk_len = end
.min(RETAINED_BACKWARD_READ_CHUNK_BYTES as u64)
@@ -156,7 +166,6 @@ impl RetainedSegmentReader {
let mut chunk = vec![0_u8; chunk_len as usize];
self.file.read_exact(&mut chunk)?;
read_bytes += chunk_len;
chunk.extend_from_slice(&self.buffer);
self.buffer = chunk;
self.buffer_start = start;
}
@@ -164,9 +173,48 @@ impl RetainedSegmentReader {
}
impl WorkerSessionStore {
/// Publishes through the same external control store used by Runtime retention.
/// Existing manifest migration is fenced too; future writes keep the guard until their
/// physical publication finishes. The control root must outlive this aggregate.
pub fn new_with_retention(
root: impl Into<PathBuf>,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, StoreError> {
let root = root.into();
let existing = match fs::read(root.join(SESSION_FILE)) {
Ok(bytes) => Some(serde_json::from_slice::<SessionManifest>(&bytes)?.session_id),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => return Err(error.into()),
};
let _guard = existing
.map(|session| retention.lock_live(session))
.transpose()?;
let mut store = Self::new(root)?;
store.retention = Some(retention);
Ok(store)
}
fn lock_publication(
&self,
session: SessionId,
) -> Result<Option<crate::retention::SessionRetentionGuard>, StoreError> {
self.retention
.as_ref()
.map(|store| store.lock_live(session))
.transpose()
}
/// Open the Session store rooted at `<worker-aggregate>/session`.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
Self::new_with_migration_writer(root.into(), atomic_write_bytes)
}
// Keep fault injection at the storage publication boundary, without global
// flags or a public alternate restore API. Production always uses atomic writes.
fn new_with_migration_writer(
root: PathBuf,
write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<Self, StoreError> {
fs::create_dir_all(root.join(SEGMENTS_DIR))?;
let session_id = match fs::read(root.join(SESSION_FILE)) {
Ok(bytes) => {
@@ -175,11 +223,16 @@ impl WorkerSessionStore {
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_logs(&root)?;
}
PREVIOUS_SESSION_SCHEMA_VERSION | LEGACY_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_v3(
&root,
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths(&root)?,
manifest.session_id,
manifest.schema_version,
write_segment,
)?;
manifest.schema_version = SESSION_SCHEMA_VERSION;
atomic_write_json(&root.join(SESSION_FILE), &manifest)?;
@@ -202,9 +255,37 @@ impl WorkerSessionStore {
root,
session_id: Arc::new(Mutex::new(session_id)),
append_lock: Arc::new(Mutex::new(())),
retention: None,
})
}
/// Validate an existing Session's conversion without publishing any bytes.
/// Hosts can isolate unreadable archives before admitting a staged migration;
/// the versioned decoder and conversion checks remain owned by this store.
pub fn validate_migration(root: &Path) -> Result<(), StoreError> {
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(root.join(SESSION_FILE))?)?;
match manifest.schema_version {
SESSION_SCHEMA_VERSION => validate_canonical_segment_logs(root),
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => migrate_segment_logs_to_current(
segment_log_paths(root)?,
manifest.session_id,
manifest.schema_version,
|_, _| Ok(()),
),
version => Err(StoreError::Corrupt {
line: 0,
message: format!(
"unsupported Worker Session schema version {version}, expected {SESSION_SCHEMA_VERSION}"
),
}),
}
}
/// Open an already-retained Session without creating directories or migrating
/// persisted data. Observation paths must never mutate retained state.
pub fn open_read_only(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
@@ -224,6 +305,7 @@ impl WorkerSessionStore {
root,
session_id: Arc::new(Mutex::new(Some(manifest.session_id))),
append_lock: Arc::new(Mutex::new(())),
retention: None,
})
}
@@ -456,6 +538,7 @@ impl WorkerSessionStore {
read_end: before_offset,
buffer_start: before_offset,
buffer: Vec::new(),
record_suffix: Vec::new(),
})
}
@@ -682,12 +765,25 @@ impl WorkerSessionStore {
}
impl Store for WorkerSessionStore {
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention
.as_ref()
.ok_or(StoreError::SessionRetentionUnsupported)?
.retain_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_log_entry(&self.log_path(segment_id), entry)
}
@@ -710,6 +806,11 @@ impl Store for WorkerSessionStore {
}
fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError> {
// A pending Worker can have metadata before its atomic initial seed.
// Absence is queryable without materializing or binding this store.
if self.session_id()?.is_none() {
return Ok(Vec::new());
}
self.ensure_session(session_id, false)?;
let mut segments: Vec<SegmentId> = Vec::new();
for entry in fs::read_dir(self.root.join(SEGMENTS_DIR))? {
@@ -740,6 +841,10 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let mut content = Vec::new();
for entry in entries {
@@ -751,6 +856,9 @@ impl Store for WorkerSessionStore {
}
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
if self.session_id()?.is_none() {
return Ok(false);
}
self.ensure_session(session_id, false)?;
Ok(self.log_path(segment_id).exists())
}
@@ -788,6 +896,7 @@ impl Store for WorkerSessionStore {
content: &str,
limits: PasteArtifactLimits,
) -> Result<PasteArtifactRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -818,6 +927,7 @@ impl Store for WorkerSessionStore {
content: &[u8],
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -842,6 +952,7 @@ impl Store for WorkerSessionStore {
context: &UploadedFileUploadContext,
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -881,6 +992,7 @@ impl Store for WorkerSessionStore {
reference: &UploadedFileRef,
source_entry_id: &str,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -909,6 +1021,7 @@ impl Store for WorkerSessionStore {
reference: &UploadedFileRef,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -923,6 +1036,7 @@ impl Store for WorkerSessionStore {
artifact_id: &str,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -937,6 +1051,7 @@ impl Store for WorkerSessionStore {
artifact_id: &str,
source_entry_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -954,6 +1069,7 @@ impl Store for WorkerSessionStore {
session_id: SessionId,
live_owner_ids: &[String],
) -> Result<u64, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -967,6 +1083,7 @@ impl Store for WorkerSessionStore {
session_id: SessionId,
artifact_id: &str,
) -> Result<bool, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -976,6 +1093,7 @@ impl Store for WorkerSessionStore {
}
fn delete_uncommitted_uploaded_files(&self, session_id: SessionId) -> Result<u64, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -1010,6 +1128,7 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_line(&self.trace_path(segment_id), &serde_json::to_string(entry)?)
}
@@ -1035,7 +1154,12 @@ fn open_retained_file(path: &Path) -> std::io::Result<File> {
}
fn segment_log_paths(root: &Path) -> Result<Vec<(SegmentId, PathBuf)>, StoreError> {
let segments = root.join(SEGMENTS_DIR);
segment_log_paths_in_directory(&root.join(SEGMENTS_DIR))
}
fn segment_log_paths_in_directory(
segments: &Path,
) -> Result<Vec<(SegmentId, PathBuf)>, StoreError> {
if !segments.exists() {
return Ok(Vec::new());
}
@@ -1078,21 +1202,23 @@ fn segment_log_paths(root: &Path) -> Result<Vec<(SegmentId, PathBuf)>, StoreErro
Ok(paths)
}
fn migrate_segment_logs_to_v3(
root: &Path,
fn migrate_segment_logs_to_current(
paths: Vec<(SegmentId, PathBuf)>,
session_id: SessionId,
source_schema_version: u32,
mut write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<(), StoreError> {
struct MigrationPlan {
path: PathBuf,
source: Vec<u8>,
entries: Vec<LogEntry>,
output: Vec<u8>,
}
// Phase 1 is strictly read-only. Every segment must parse and canonicalize
// successfully before the first authoritative byte is replaced.
let mut plans = Vec::new();
for (segment_id, path) in segment_log_paths(root)? {
for (segment_id, path) in paths {
let source = fs::read(&path)?;
let canonical = parse_legacy_jsonl(source_schema_version, session_id, segment_id, &source)
.map_err(|error| StoreError::Corrupt {
@@ -1102,17 +1228,32 @@ fn migrate_segment_logs_to_v3(
path.display()
),
})?;
let mut output = Vec::new();
for entry in canonical {
serde_json::to_writer(&mut output, &entry)?;
output.push(b'\n');
}
plans.push(MigrationPlan {
path,
source,
output,
entries: canonical,
output: Vec::new(),
});
}
// Resolve trusted creation boundaries first, then update inherited copies by
// logical identity too (fork/rewind seeds retain the same entry id).
let summaries = if source_schema_version < COMPACTION_SESSION_SCHEMA_VERSION {
crate::legacy_session_log::compaction_summary_ids(
plans.iter().flat_map(|plan| plan.entries.iter()),
)
} else {
// v4 already made the explicit origin migration. Preserve its origins.
Default::default()
};
for plan in &mut plans {
crate::legacy_session_log::migrate_compaction_origins(&mut plan.entries, &summaries);
for mut entry in std::mem::take(&mut plan.entries) {
crate::legacy_session_log::migrate_system_inputs(&mut entry)?;
entry.validate_input_metadata()?;
serde_json::to_writer(&mut plan.output, &entry)?;
plan.output.push(b'\n');
}
}
// Fence the complete preflight snapshot before starting phase 2. Session
// open is the exclusive restore boundary; this additionally fails closed
@@ -1130,14 +1271,56 @@ fn migrate_segment_logs_to_v3(
}
for plan in plans {
atomic_write_bytes(&plan.path, &plan.output)?;
write_segment(&plan.path, &plan.output)?;
}
Ok(())
}
/// Upgrade the standalone per-Session layout using the same persisted format
/// migration as Worker aggregates. Called only at writable Host store startup.
pub(crate) fn migrate_fs_session(root: &Path, session_id: SessionId) -> Result<(), StoreError> {
let marker = root.join("session-schema.json");
let version: u32 = match fs::read(&marker) {
Ok(bytes) => serde_json::from_slice(&bytes)?,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
ANNOTATED_SESSION_SCHEMA_VERSION
}
Err(error) => return Err(error.into()),
};
match version {
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_paths(segment_log_paths_in_directory(root)?)
}
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths_in_directory(root)?,
session_id,
version,
atomic_write_bytes,
)?;
atomic_write_json(&marker, &SESSION_SCHEMA_VERSION)
}
_ => Err(StoreError::Corrupt {
line: 0,
message: format!("unsupported standalone Session schema {version}"),
}),
}
}
fn validate_canonical_segment_logs(root: &Path) -> Result<(), StoreError> {
for (_, path) in segment_log_paths(root)? {
let _: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
validate_canonical_segment_paths(segment_log_paths(root)?)
}
fn validate_canonical_segment_paths(paths: Vec<(SegmentId, PathBuf)>) -> Result<(), StoreError> {
for (_, path) in paths {
let entries: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
for entry in entries {
entry.validate_input_metadata()?;
}
}
Ok(())
}
@@ -1151,17 +1334,26 @@ fn parse_legacy_jsonl(
let text = std::str::from_utf8(bytes).map_err(|error| {
serde_json::Error::io(std::io::Error::new(std::io::ErrorKind::InvalidData, error))
})?;
text.lines()
let mut offset = 0_u64;
text.split_inclusive('\n')
.enumerate()
.filter(|(_, line)| !line.trim().is_empty())
.map(|(line_index, line)| {
crate::legacy_session_log::decode_entry(
.filter_map(|(line_index, line)| {
let start = offset;
offset += line.len() as u64;
(!line.trim().is_empty()).then_some((line_index, line, start))
})
.map(|(line_index, line, start)| {
let mut entry = crate::legacy_session_log::decode_entry(
schema_version,
line,
session_id,
segment_id,
line_index,
)
)?;
// Byte offsets change during migration. Persist identities previously
// synthesized from the OLD offsets before rewriting any record.
entry.ensure_legacy_session_entry_id(session_id, segment_id, start);
Ok(entry)
})
.collect()
}
@@ -1308,6 +1500,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -1315,6 +1508,139 @@ mod tests {
}
}
fn retained_reader(bytes: &[u8]) -> RetainedSegmentReader {
let mut file = tempfile::tempfile().unwrap();
file.write_all(bytes).unwrap();
RetainedSegmentReader {
file,
session_id: new_session_id(),
segment_id: new_segment_id(),
read_end: bytes.len() as u64,
buffer_start: bytes.len() as u64,
buffer: Vec::new(),
record_suffix: Vec::new(),
}
}
fn checkpoint_line(text: String) -> Vec<u8> {
let mut bytes = serde_json::to_vec(&LogEntry::InputSegmentsCheckpoint {
ts: 1,
user_segments: vec![vec![protocol::Segment::Text { content: text }]],
})
.unwrap();
bytes.push(b'\n');
bytes
}
#[test]
fn retained_reader_preserves_records_and_offsets_across_chunk_boundaries() {
let chunk = RETAINED_BACKWARD_READ_CHUNK_BYTES;
let overhead = checkpoint_line(String::new()).len();
for line_size in [chunk - 1, chunk, chunk + 1, chunk * 3] {
let lines = [
checkpoint_line("最初\nの行".into()),
checkpoint_line("x".repeat(line_size - overhead)),
checkpoint_line("最後".into()),
];
let bytes = lines.concat();
let mut reader = retained_reader(&bytes);
let mut end = bytes.len() as u64;
let mut total_read = 0;
for line in lines.iter().rev() {
let (record, read) = reader.previous_record(u64::MAX).unwrap();
total_read += read;
let record = record.unwrap();
assert_eq!(record.end_offset, end);
end -= line.len() as u64;
assert_eq!(record.start_offset, end);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
line[..line.len() - 1]
);
}
assert_eq!(total_read, bytes.len() as u64);
let (record, read) = reader.previous_record(0).unwrap();
assert!(record.is_none());
assert_eq!(read, 0);
}
}
#[test]
fn retained_reader_resumes_after_budget_exhaustion_and_serves_buffered_records() {
let first = checkpoint_line("first".into());
let last = checkpoint_line("大きな行".repeat(RETAINED_BACKWARD_READ_CHUNK_BYTES));
let mut reader = retained_reader(&[first.clone(), last.clone()].concat());
let budget = RETAINED_BACKWARD_READ_CHUNK_BYTES as u64;
assert!(matches!(
reader.previous_record(budget),
Err(StoreError::ReadLimitExceeded)
));
assert!(matches!(
reader.previous_record(0),
Err(StoreError::ReadLimitExceeded)
));
let (record, read) = reader.previous_record(u64::MAX).unwrap();
let record = record.unwrap();
assert_eq!(record.start_offset, first.len() as u64);
assert_eq!(record.end_offset, (first.len() + last.len()) as u64);
assert_eq!(read + budget, record.end_offset);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
last[..last.len() - 1]
);
let (record, read) = reader.previous_record(0).unwrap();
assert_eq!(record.unwrap().start_offset, 0);
assert_eq!(read, 0);
assert!(reader.previous_record(0).unwrap().0.is_none());
}
#[test]
fn retained_reader_requires_the_prior_boundary_byte_at_exact_chunk_edges() {
let chunk = RETAINED_BACKWARD_READ_CHUNK_BYTES;
let overhead = checkpoint_line(String::new()).len();
for size in [chunk - 1, chunk, chunk + 1] {
let first = checkpoint_line("first".into());
let last = checkpoint_line("x".repeat(size - overhead));
let bytes = [first.clone(), last.clone()].concat();
let mut reader = retained_reader(&bytes);
// The final record alone is insufficient: its preceding newline
// must also be read before its start can be established.
assert!(matches!(
reader.previous_record(size as u64),
Err(StoreError::ReadLimitExceeded)
));
let (record, read) = reader.previous_record(1).unwrap();
let record = record.unwrap();
assert_eq!(read, 1);
assert_eq!(record.start_offset, first.len() as u64);
assert_eq!(record.end_offset, bytes.len() as u64);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
last[..last.len() - 1]
);
let (record, read) = reader.previous_record(u64::MAX).unwrap();
assert_eq!(record.unwrap().start_offset, 0);
assert_eq!(read, (first.len() - 1) as u64);
}
}
#[test]
fn retained_reader_rejects_empty_malformed_and_unterminated_records() {
for bytes in [
b"\n".as_slice(),
b"{bad json}\n",
b"{\"kind\":\"input_segments_checkpoint\"}",
] {
assert!(retained_reader(bytes).previous_record(u64::MAX).is_err());
}
let mut bytes = checkpoint_line("valid".into());
bytes.push(b'\n');
assert!(matches!(
retained_reader(&bytes).previous_record(u64::MAX),
Err(StoreError::Corrupt { .. })
));
}
#[test]
fn canonical_combined_store_supports_attachment_uploads() {
let root = tempfile::tempdir().unwrap();
@@ -1762,6 +2088,146 @@ mod tests {
);
}
#[test]
fn v5_migration_retries_after_second_file_publication_fails() {
fn identity_and_body(entries: &[LogEntry]) -> Vec<serde_json::Value> {
entries
.iter()
.flat_map(|record| match record {
LogEntry::AnnotatedSystemItem { entry, .. } => vec![entry.to_history_entry()],
LogEntry::AnnotatedUserInput { history, .. } => history.clone(),
_ => panic!("unexpected record in fixed migration fixture"),
})
.map(|entry| {
serde_json::json!({
"entry_id": entry.metadata.entry_id,
"origin": entry.metadata.origin,
"derivation": entry.metadata.derivation,
"item": entry.item,
})
})
.collect()
}
const SOURCE: &str =
include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl");
let root = tempfile::tempdir().unwrap();
let session_id = new_session_id();
let first_id = SegmentId::from_u128(1);
let second_id = SegmentId::from_u128(2);
fs::create_dir_all(root.path().join(SEGMENTS_DIR)).unwrap();
let marker_path = root.path().join(SESSION_FILE);
atomic_write_json(
&marker_path,
&SessionManifest {
schema_version: COMPACTION_SESSION_SCHEMA_VERSION,
session_id,
},
)
.unwrap();
let old_marker = fs::read(&marker_path).unwrap();
let first_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{first_id}.jsonl"));
let second_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{second_id}.jsonl"));
let second_source = SOURCE.replace("-id", "-second-id");
fs::write(&first_path, SOURCE).unwrap();
fs::write(&second_path, &second_source).unwrap();
let old_first = fs::read(&first_path).unwrap();
let old_second = fs::read(&second_path).unwrap();
let first_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, first_id, &old_first).unwrap());
let second_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, second_id, &old_second).unwrap());
let mut writes = 0;
let mut first_published = None;
let failure = WorkerSessionStore::new_with_migration_writer(
root.path().to_path_buf(),
|path, bytes| {
writes += 1;
match writes {
1 => {
assert_eq!(path, first_path.as_path());
// Both files are still old when the first publication starts.
assert_eq!(fs::read(&first_path).unwrap(), old_first);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
atomic_write_bytes(path, bytes)?;
first_published = Some(bytes.to_vec());
Ok(())
}
2 => {
assert_eq!(path, second_path.as_path());
assert_eq!(
fs::read(&first_path).unwrap(),
*first_published.as_ref().unwrap()
);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
// Fail only this atomic-write boundary, after complete
// preflight and a real successful first-file publication.
Err(std::io::Error::other("injected second migration write failure").into())
}
_ => panic!("publication continued after its write failure"),
}
},
);
let error = match failure {
Ok(_) => panic!("the injected publication failure must reject store open"),
Err(error) => error,
};
assert!(
error
.to_string()
.contains("injected second migration write failure")
);
assert_eq!(writes, 2);
let first_published = first_published.unwrap();
assert_ne!(first_published, old_first);
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
assert_eq!(fs::read(&marker_path).unwrap(), old_marker);
let updated_first: Vec<LogEntry> = parse_jsonl(&first_published).unwrap();
for entry in &updated_first {
entry.validate_input_metadata().unwrap();
}
assert!(matches!(
&updated_first[0],
LogEntry::AnnotatedSystemItem {
entry: crate::LoggedSystemHistoryEntry {
item: crate::SystemItem::Input { .. },
..
},
..
}
));
assert_eq!(identity_and_body(&updated_first), first_identity);
// Retry via the public production constructor, without any injection or
// hand-built mixed fixture. The old marker drives the same migration.
let store = WorkerSessionStore::new(root.path()).unwrap();
let first = store.read_all(session_id, first_id).unwrap();
let second = store.read_all(session_id, second_id).unwrap();
assert_eq!(identity_and_body(&first), first_identity);
assert_eq!(identity_and_body(&second), second_identity);
for entry in first.iter().chain(second.iter()) {
entry.validate_input_metadata().unwrap();
}
assert_eq!(fs::read(&first_path).unwrap(), first_published);
let second_published = fs::read(&second_path).unwrap();
assert_ne!(second_published, old_second);
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(&marker_path).unwrap()).unwrap();
assert_eq!(manifest.schema_version, SESSION_SCHEMA_VERSION);
drop(store);
WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), second_published);
}
#[test]
fn current_jsonl_requires_annotations_across_append_rewrite_and_reopen() {
let root = tempfile::tempdir().unwrap();
@@ -1906,6 +2372,24 @@ mod tests {
assert!(matches!(error, StoreError::Corrupt { .. }));
}
#[test]
fn pending_session_probes_are_empty_without_binding_and_foreign_materialized_session_is_rejected()
{
let root = tempfile::tempdir().unwrap();
let session_id = new_session_id();
let segment_id = new_segment_id();
let store = WorkerSessionStore::new(root.path()).unwrap();
assert!(!store.exists(session_id, segment_id).unwrap());
assert!(store.list_segments(session_id).unwrap().is_empty());
assert_eq!(store.session_id().unwrap(), None);
let reopened = WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(reopened.session_id().unwrap(), None);
store.create_segment(session_id, segment_id, &[]).unwrap();
let foreign = new_session_id();
assert!(store.exists(foreign, segment_id).is_err());
assert!(store.list_segments(foreign).is_err());
}
#[test]
fn reopen_preserves_session_and_segment_ids() {
let root = tempfile::tempdir().unwrap();
+179 -2
View File
@@ -11,6 +11,7 @@ fn annotated(item: Item) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -62,6 +63,7 @@ fn round_trip_write_and_read() {
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 3200,
interrupted: false,
result: EngineResult::Finished,
@@ -157,9 +159,10 @@ fn create_segment_replacement_keeps_open_old_log_and_commits_complete_new_log()
assert_eq!(reopened.read_entry_count(sid, segid).unwrap(), new.len());
assert_eq!(
std::fs::read_dir(path.parent().unwrap()).unwrap().count(),
1,
"successful replacement must not leave temporary files"
2,
"successful replacement leaves only the segment and Session schema marker"
);
assert!(path.parent().unwrap().join("session-schema.json").is_file());
}
#[test]
@@ -419,3 +422,177 @@ fn lookup_session_of_finds_owning_session() {
assert_eq!(store.lookup_session_of(segid).unwrap(), Some(sid));
}
// Storage/projection component test: both production layouts, temporary originals,
// no Feature, model, runtime or Host authority is substituted into this boundary.
#[test]
fn generic_system_append_and_fork_preserve_metadata_without_public_payload_leak() {
use session_store::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputObservationUse, LoggedHistoryDerivation, LoggedSystemHistoryEntry,
PublicSystemItem, PublicSystemPresentation, SystemInputEnvelope, WorkerSessionStore,
};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let fork = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
store
.create_segment(sid, segment, &[nil_session_start(1, sid)])
.unwrap();
let entry = LoggedSystemHistoryEntry::from_input(
LoggedSessionHistoryOrigin::Provided {
producer: "new-feature".into(),
},
Some(LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source-entry".into())],
}),
SystemInputEnvelope::new(
"private model body",
HistoryInputMetadata::new(
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
},
Some(
FeatureInputPayload::encode(
"new-feature:context",
101,
&serde_json::json!({"private": "secret-detail"}),
)
.unwrap(),
),
Some(PublicSystemItem {
kind: "context".into(),
body: "explicit public body".into(),
presentation: PublicSystemPresentation::Context,
}),
)
.unwrap(),
)
.unwrap(),
);
store
.append(
sid,
segment,
&LogEntry::AnnotatedSystemItem {
ts: 2,
entry: entry.clone(),
extensions: vec![],
},
)
.unwrap();
let saved = store.read_all(sid, segment).unwrap();
let LogEntry::AnnotatedSystemItem {
entry: restored, ..
} = &saved[1]
else {
panic!("missing system input")
};
assert_eq!(restored.to_history_entry(), entry.to_history_entry());
assert_eq!(
collect_state(&saved).history[0].as_text(),
Some("private model body")
);
let mut fork_seed = nil_session_start(3, sid);
if let LogEntry::AnnotatedSegmentStart {
history,
forked_from,
..
} = &mut fork_seed
{
history.push(restored.to_history_entry());
*forked_from = Some(session_store::SegmentOrigin {
segment_id: segment,
at_turn_index: 0,
});
}
store.create_segment(sid, fork, &[fork_seed]).unwrap();
let fork_saved = store.read_all(sid, fork).unwrap();
let LogEntry::AnnotatedSegmentStart { history, .. } = &fork_saved[0] else {
unreachable!()
};
assert_eq!(history, &[entry.to_history_entry()]);
let original = session_store::public_snapshot::project_session_snapshot(sid, &saved);
let inherited = session_store::public_snapshot::project_session_snapshot(sid, &fork_saved);
let public = serde_json::to_string(&original.entries).unwrap();
assert!(public.contains("explicit public body"));
for private in ["private model body", "secret-detail", "new-feature:context"] {
assert!(!public.contains(private), "private field leaked: {private}");
}
assert_eq!(original.entries.len(), 1);
assert_eq!(inherited.entries.len(), 1);
assert_eq!(original.entries[0].entry_id, inherited.entries[0].entry_id);
assert_eq!(
original.entries[0].derived_from,
inherited.entries[0].derived_from
);
assert_eq!(
original.entries[0].provenance,
inherited.entries[0].provenance
);
assert_eq!(
serde_json::to_value(&original.entries[0].data).unwrap(),
serde_json::to_value(&inherited.entries[0].data).unwrap()
);
}
}
#[test]
fn invalid_generic_system_input_rejects_append_create_and_external_json_without_changing_originals()
{
use session_store::{StoreError, SystemItem, WorkerSessionStore};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
let seed = nil_session_start(1, sid);
store
.create_segment(sid, segment, std::slice::from_ref(&seed))
.unwrap();
let path = if worker_layout {
dir.path().join("segments").join(format!("{segment}.jsonl"))
} else {
dir.path()
.join(sid.to_string())
.join(format!("{segment}.jsonl"))
};
let before = std::fs::read(&path).unwrap();
let invalid = LogEntry::AnnotatedSystemItem {
ts: 2,
extensions: vec![],
entry: session_store::LoggedSystemHistoryEntry {
item: SystemItem::Input {
body: "no policy".into(),
},
metadata: LoggedSessionHistoryMetadata::legacy_unknown(),
},
};
assert!(matches!(
store.append(sid, segment, &invalid),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(matches!(
store.create_segment(sid, segment, &[seed, invalid.clone()]),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(
serde_json::from_value::<LogEntry>(serde_json::to_value(invalid).unwrap()).is_err()
);
assert_eq!(std::fs::read(path).unwrap(), before);
assert_eq!(store.read_all(sid, segment).unwrap().len(), 1);
}
}
@@ -24,6 +24,7 @@ fn annotated(items: &[Item]) -> Vec<session_store::LoggedHistoryEntry> {
.map(|item| session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
+1 -1
View File
@@ -23,7 +23,7 @@ serde.workspace = true
serde_json.workspace = true
session-store.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["rt", "sync", "time"] }
tokio = { workspace = true, features = ["rt", "sync", "time", "net", "io-util", "io-std", "signal"] }
uuid = { workspace = true, features = ["v7"] }
worker.workspace = true
+2 -1
View File
@@ -4,7 +4,8 @@ A Host that starts existing Workers in the same process. Assembles interactive W
## Reference index
- [`host`] — bootstrap and confirmed shutdown.
- [`host`] — bootstrap and confirmed shutdown, with or without a TUI.
- [`unix`] — optional operator transport; disconnect does not release execution ownership.
- [`launch`] — Profile, cwd, and Subject validation before launch.
- [`store`] — Worker records and leases.
- [`jobs`] — Job execution and independent result and cleanup lifetimes.
+250 -40
View File
@@ -23,10 +23,7 @@ use session_store::{
CombinedStore, FsStore, FsWorkerStore, WorkerActiveSegmentRef, WorkerMetadataStore,
};
use thiserror::Error;
use worker::bootstrap::{
WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout, bash_output_dir_for_worker_id,
};
use worker::controller::WorkerControllerTransport;
use worker::bootstrap::{WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout};
use worker::ipc::protocol_session::{
WorkerProtocolSessionStreams, dispatch_worker_protocol_method, live_log_entry_event,
subscribe_worker_protocol_session,
@@ -57,6 +54,15 @@ pub struct StandaloneHost {
lease: Option<StandaloneWorkerLease>,
jobs: crate::jobs::StandaloneJobs,
subject: Option<crate::subjektiv::SubjectConnection>,
subject_release: Option<worker::SubjectReleaseReservation>,
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum StandaloneSubjectDisconnectError {
#[error("Subject release requires an actual Idle Controller without pending work: {0}")]
Controller(String),
#[error("Subject release remains incomplete; retained binding/lease must be reconciled: {0}")]
Incomplete(String),
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
@@ -89,6 +95,10 @@ pub enum StandaloneStartupError {
FeatureComposition,
#[error("the in-process Worker controller could not start")]
Controller,
#[error(
"mandatory initial context remains unconfirmed; restore retained Worker {worker_id} to retry this Session"
)]
InitialContextPending { worker_id: WorkerId },
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
@@ -171,7 +181,7 @@ impl StandaloneHost {
let jobs = match crate::jobs::StandaloneJobs::open(&jobs_path, launch.cwd.clone()) {
Ok(jobs) => jobs,
Err(_) => {
let _ = store.abandon_allocation(allocation);
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(StandaloneStartupError::StateStore);
}
};
@@ -179,8 +189,11 @@ impl StandaloneHost {
if let Some(client) = &model_client {
jobs.bind_model_client(client.clone_boxed());
}
if let Some(subject) = &subject {
subject.host.bind_jobs(jobs.clone())?;
if let Some(connection) = &subject
&& let Err(error) = connection.host.bind_jobs(jobs.clone())
{
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(error);
}
// WorkerId is the stable identity. The current Worker store remains
// name-keyed, so keep its derived storage key separate from the
@@ -192,7 +205,7 @@ impl StandaloneHost {
let (backing_store, worker_store) = match backing_store(&store, worker_id) {
Ok(stores) => stores,
Err(error) => {
let _ = store.abandon_allocation(allocation);
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(error);
}
};
@@ -200,7 +213,6 @@ impl StandaloneHost {
WorkerFilesystemAuthority::local(launch.cwd.clone(), launch.cwd.clone());
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
bootstrap_manifest,
@@ -208,11 +220,7 @@ impl StandaloneHost {
launch.prompt_catalog,
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerControllerTransport::InProcess,
WorkerBootstrapLayout::Direct { runtime_base },
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
@@ -220,12 +228,61 @@ impl StandaloneHost {
if let Some(model_client) = model_client {
bootstrap = bootstrap.with_model_client(model_client);
}
let prepared = match bootstrap.prepare().await {
Ok(prepared) => prepared,
Err(error) => {
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(classify_startup_error(error));
}
};
// Reserve a recoverable identity before mandatory initial-context persistence.
// A failed/unknown seed must be reopened as this Session, never abandoned and
// replaced by a newly fetched context under an invisible ghost binding.
let pending_record = if subject.is_some() {
match store.commit_created_connected(
&allocation,
manifest.clone(),
storage_key.clone(),
prepared.worker().session_id(),
None,
subject.as_ref().map(|s| s.binding.clone()),
) {
Ok(record) => Some(record),
Err(_) => {
if let Some(subject) = &mut subject {
subject.retain();
}
allocation.into_lease().retain();
return Err(StandaloneStartupError::StateStore);
}
}
} else {
None
};
if let Some(subject) = &mut subject {
subject.arm();
}
let started = match bootstrap.start().await {
let started = match prepared.start().await {
Ok(started) => started,
Err(error) => {
if let Some(record) = &pending_record {
let cleanup = jobs.shutdown().await;
let saved = store.mark_stopped(
record,
record.active_session_id,
record.active_segment_id,
StandaloneShutdownReason::StartupFailed,
);
let closed = cleanup.is_ok()
&& saved.is_ok()
&& subject.as_mut().is_some_and(|s| s.close().is_ok());
if closed {
let _ = allocation.into_lease().release();
} else {
allocation.into_lease().retain();
}
return Err(StandaloneStartupError::InitialContextPending { worker_id });
}
let _ = store.abandon_allocation(allocation);
return Err(classify_startup_error(error));
}
@@ -234,22 +291,35 @@ impl StandaloneHost {
Ok(active) => active,
Err(error) => {
stop_started_worker(started).await;
let _ = store.abandon_allocation(allocation);
if pending_record.is_some() {
allocation.into_lease().retain();
} else {
let _ = store.abandon_allocation(allocation);
}
return Err(error);
}
};
let record = match store.commit_created_connected(
&allocation,
manifest,
storage_key,
active.session_id,
active.segment_id,
subject.as_ref().map(|s| s.binding.clone()),
) {
let committed = if let Some(record) = &pending_record {
store.update_active_pointer(record, active.session_id, active.segment_id)
} else {
store.commit_created_connected(
&allocation,
manifest,
storage_key,
active.session_id,
active.segment_id,
None,
)
};
let record = match committed {
Ok(record) => record,
Err(_) => {
stop_started_worker(started).await;
let _ = store.abandon_allocation(allocation);
if pending_record.is_some() {
allocation.into_lease().retain();
} else {
let _ = store.abandon_allocation(allocation);
}
return Err(StandaloneStartupError::StateStore);
}
};
@@ -350,7 +420,6 @@ impl StandaloneHost {
);
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
manifest,
@@ -358,11 +427,7 @@ impl StandaloneHost {
worker::PromptCatalogSource::builtins_only(),
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerControllerTransport::InProcess,
WorkerBootstrapLayout::Direct { runtime_base },
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
@@ -427,6 +492,7 @@ impl StandaloneHost {
lease: Some(lease),
jobs,
subject,
subject_release: None,
}
}
@@ -451,10 +517,75 @@ impl StandaloneHost {
/// Working events, committed session entries, alert snapshots, and the
/// initial history snapshot are merged behind the client boundary.
pub fn connect(&self) -> Client<InProcessSocket> {
let streams = subscribe_worker_protocol_session(&self.handle);
let (socket, peer) = InProcessSocket::pair();
tokio::spawn(run_protocol_session(self.handle.clone(), streams, peer));
Client::new(socket)
Client::new(connect_protocol(self.handle.clone()))
}
/// Logical Subject disconnection, not Worker shutdown or completion of its represented work.
/// A successful release keeps the old Controller fenced; a fresh Worker/Session may then
/// acquire the Subject. Failures before durable revocation allow normal recovery and retry.
pub async fn disconnect_subject(&mut self) -> Result<(), StandaloneSubjectDisconnectError> {
let Some(subject) = &mut self.subject else {
return Ok(());
};
let mut reservation = match self.subject_release.take() {
Some(reservation) => reservation,
None => self
.handle
.reserve_subject_release()
.await
.map_err(StandaloneSubjectDisconnectError::Controller)?,
};
let release = match subject.release_for_handoff(&self.record, &mut reservation) {
Ok(()) => match self.jobs.shutdown().await {
Ok(()) => subject.close().map_err(|error| error.to_string()),
Err(error) => Err(error.to_string()),
},
Err(error) => Err(error),
};
match release {
Ok(()) => {
self.subject_release = Some(reservation);
self.subject = None;
Ok(())
}
Err(error) => {
// If revocation committed, keep the actual Controller reserved even if closing
// storage/unlocking failed. Dropping a reservation must not undo revocation.
if subject.writer_released_or_unknown() {
self.subject_release = Some(reservation);
}
Err(StandaloneSubjectDisconnectError::Incomplete(error))
}
}
}
/// Run without a TUI until an admitted protocol Shutdown, controller exit, or process signal.
/// Client EOF only disconnects that client. All exits confirm the same production shutdown
/// and lease release as the in-process UI; an unconfirmed shutdown retains the lease.
pub async fn serve_unix(
self,
endpoint: crate::unix::UnixEndpoint,
) -> Result<(), crate::HeadlessError> {
let mut adapter = crate::unix::UnixAdapter::start(endpoint, self.handle.clone());
let signal = async {
let mut terminate =
tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())?;
tokio::select! {
result = tokio::signal::ctrl_c() => result,
_ = terminate.recv() => Ok(()),
}
};
let result = tokio::select! {
_ = self.handle.protocol_closed() => Ok(()),
result = signal => result,
result = adapter.ended() => result,
};
// Reclaim the public entry and every connection before releasing execution ownership.
let transport = adapter.shutdown().await;
self.shutdown().await?;
result?;
transport?;
Ok(())
}
pub fn with_shutdown_timeout(mut self, shutdown_timeout: Duration) -> Self {
@@ -539,9 +670,8 @@ impl Drop for StandaloneHost {
}
}
// The in-process channel is owned by the local operator's Host, not an
// arbitrary socket or Backend account. decode_method discards wire provenance;
// only this boundary can stamp accepted local human input.
// Both in-process views and credential-checked Unix clients are admitted by this operator's
// Host. decode_method discards wire provenance; only this boundary stamps local human input.
fn authorize_local_input(method: protocol::Method) -> protocol::Method {
use protocol::{AuthenticatedInputSource, Method};
match method {
@@ -584,6 +714,13 @@ fn authorize_local_input(method: protocol::Method) -> protocol::Method {
}
}
pub(crate) fn connect_protocol(handle: worker::WorkerHandle) -> InProcessSocket {
let streams = subscribe_worker_protocol_session(&handle);
let (socket, peer) = InProcessSocket::pair();
tokio::spawn(run_protocol_session(handle, streams, peer));
socket
}
async fn run_protocol_session(
handle: worker::WorkerHandle,
streams: WorkerProtocolSessionStreams,
@@ -619,6 +756,11 @@ async fn run_protocol_session(
event = events.recv() => {
match event {
Ok(event) => {
let Ok(event) = handle.alerter.freshen_event(event) else {
// The retained anchor was evicted; reconnect must obtain
// a paired session/Alert snapshot rather than a stale row.
return;
};
if !send_protocol_event(&peer, event).await {
return;
}
@@ -690,12 +832,32 @@ async fn send_protocol_event(peer: &InProcessPeer, event: Event) -> bool {
peer.send(message).await.is_ok()
}
fn cancel_unpublished_start(
store: &StandaloneWorkerStore,
subject: &mut Option<crate::subjektiv::SubjectConnection>,
allocation: crate::store::StandaloneWorkerAllocation,
) -> Result<(), StandaloneStartupError> {
if let Some(connection) = subject {
if connection.abort_unpublished_start().is_err() {
connection.retain();
allocation.into_lease().retain();
return Err(StandaloneStartupError::StateStore);
}
}
store
.abandon_allocation(allocation)
.map_err(classify_store_startup_error)
}
fn backing_store(
store: &StandaloneWorkerStore,
worker_id: WorkerId,
) -> Result<(StandaloneBackingStore, FsWorkerStore), StandaloneStartupError> {
let session_store = FsStore::new(store.sessions_dir(worker_id))
.map_err(|_| StandaloneStartupError::StateStore)?;
let session_store = FsStore::new_with_retention(
store.sessions_dir(worker_id),
crate::store::session_retention(store.root()),
)
.map_err(|_| StandaloneStartupError::StateStore)?;
let worker_store = FsWorkerStore::new(store.worker_metadata_dir(worker_id))
.map_err(|_| StandaloneStartupError::StateStore)?;
Ok((
@@ -808,3 +970,51 @@ mod local_input_tests {
}
}
}
#[cfg(test)]
mod source_retention_tests {
//! Host backing-store assembly uses the production publisher and an isolated external
//! erasure fence. No controller/model is substituted for the source authority under test.
use super::*;
use session_store::{LogEntry, Store, StoreError};
#[test]
fn backing_publisher_and_reopen_use_the_host_wide_erasure_fence() {
let temp = tempfile::tempdir().unwrap();
let workers = StandaloneWorkerStore::open(temp.path().join("state")).unwrap();
let allocation = workers
.allocate(temp.path(), StaleLeasePolicy::Reject)
.unwrap();
let worker = allocation.worker_id();
let (backing, _) = backing_store(&workers, worker).unwrap();
let session = session_store::SessionId::now_v7();
let segment = uuid::Uuid::now_v7();
let entry = LogEntry::TurnEnd {
ts: 1,
turn_count: 1,
};
backing
.create_segment(session, segment, &[entry.clone()])
.unwrap();
let retention = crate::store::session_retention(workers.root());
assert!(
retention
.delete(
session,
session_store::retention::SessionDeletionIntent::ErasePermanently,
|| Err::<(), StoreError>(
std::io::Error::other("injected physical erase failure").into()
)
)
.is_err()
);
assert!(
matches!(backing.append(session, segment, &entry), Err(StoreError::SessionErased(id)) if id == session)
);
let (reopened, _) = backing_store(&workers, worker).unwrap();
assert!(
matches!(reopened.append(session, segment, &entry), Err(StoreError::SessionErased(id)) if id == session)
);
assert!(!workers.sessions_dir(worker).join(".retention").exists());
}
}
+172 -3
View File
@@ -25,7 +25,7 @@ use thiserror::Error;
#[path = "job_store/legacy_migrations.rs"]
mod legacy_migrations;
const SCHEMA_VERSION: i64 = 3;
const SCHEMA_VERSION: i64 = 4;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
@@ -189,6 +189,7 @@ impl JobStore {
0 => {
tx.execute_batch(SCHEMA)?;
tx.execute_batch(GRANT_SCHEMA)?;
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
1 | 2 => {
@@ -196,6 +197,11 @@ impl JobStore {
tx.execute_batch(GRANT_SCHEMA)?;
}
legacy_migrations::migrate_legacy_jobs(&tx)?;
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
3 => {
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
SCHEMA_VERSION => {}
@@ -278,6 +284,49 @@ impl JobStore {
Ok(snapshot)
}
/// Domain Host attests that this fenced attempt's only effects are reconciled in its
/// authoritative durable consumer. This releases serialization, not the original Unknown
/// history, and cannot authorize retry or change its result. The service checks cleanup.
pub(crate) fn reconcile_granted_attempt(
&mut self,
job_id: &str,
attempt_id: &str,
domain_receipt: &str,
) -> Result<(), JobStoreError> {
if domain_receipt.is_empty()
|| domain_receipt.len() > 256
|| domain_receipt.chars().any(char::is_control)
{
return Err(JobStoreError::InvalidTransition(
"invalid domain reconciliation receipt",
));
}
let tx = self
.conn
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, job_id)?;
check_attempt(&snapshot, attempt_id)?;
if snapshot.domain_grant.is_none()
|| snapshot.attempt.state != JobAttemptState::Unknown
|| !matches!(snapshot.state, JobState::Unknown | JobState::Cancelled)
{
return Err(JobStoreError::InvalidTransition(
"domain reconciliation requires a fenced unknown granted attempt",
));
}
let old: Option<String> = tx.query_row("SELECT domain_receipt FROM job_domain_reconciliations WHERE job_id=?1 AND attempt_id=?2",
params![job_id,attempt_id], |row| row.get(0)).optional()?;
if old.as_deref().is_some_and(|old| old != domain_receipt) {
return Err(JobStoreError::InvalidTransition(
"domain reconciliation receipt changed",
));
}
tx.execute("INSERT OR IGNORE INTO job_domain_reconciliations(job_id,attempt_id,domain_receipt) VALUES(?1,?2,?3)",
params![job_id,attempt_id,domain_receipt])?;
tx.commit()?;
Ok(())
}
/// Only definitive failure can create a new immutable attempt. Unknown
/// outcomes require caller reconciliation, never an automatic replay.
pub(crate) fn retry(&mut self, job_id: &str) -> Result<JobSnapshot, JobStoreError> {
@@ -319,6 +368,16 @@ impl JobStore {
.collect()
}
pub(crate) fn unknown_granted(&self) -> Result<Vec<JobSnapshot>, JobStoreError> {
let mut statement = self.conn.prepare("SELECT i.job_id FROM job_intents i JOIN job_attempts a ON a.job_id=i.job_id AND a.number=i.current_attempt JOIN job_domain_grants g ON g.job_id=i.job_id WHERE a.state='unknown' AND NOT EXISTS (SELECT 1 FROM job_domain_reconciliations r WHERE r.job_id=a.job_id AND r.attempt_id=a.attempt_id) ORDER BY i.rowid")?;
let ids = statement
.query_map([], |row| row.get::<_, String>(0))?
.collect::<rusqlite::Result<Vec<_>>>()?;
ids.into_iter()
.map(|id| require_snapshot(&self.conn, &id))
.collect()
}
/// Only unstarted current attempts are eligible for service startup/resume.
pub(crate) fn pending(&self) -> Result<Vec<JobSnapshot>, JobStoreError> {
let mut statement = self.conn.prepare(
@@ -527,8 +586,10 @@ fn check_serialization(
if let Some(key) = key {
let conflicting: bool = conn.query_row(
"SELECT EXISTS(SELECT 1 FROM job_intents i WHERE serialization_key = ?1
AND job_id <> ?2 AND (state IN ('pending', 'unknown') OR EXISTS
(SELECT 1 FROM job_attempts a WHERE a.job_id = i.job_id AND a.state = 'unknown')))",
AND job_id <> ?2 AND (state = 'pending' OR EXISTS
(SELECT 1 FROM job_attempts a WHERE a.job_id = i.job_id AND a.state = 'unknown'
AND NOT EXISTS (SELECT 1 FROM job_domain_reconciliations r
WHERE r.job_id=a.job_id AND r.attempt_id=a.attempt_id))))",
params![key, job_id],
|row| row.get(0),
)?;
@@ -658,6 +719,22 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
.transpose()
}
const RECONCILIATION_SCHEMA: &str = "
DROP INDEX job_serialization_singleflight;
CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key)
WHERE serialization_key IS NOT NULL AND state = 'pending';
CREATE TABLE job_domain_reconciliations (
job_id TEXT NOT NULL REFERENCES job_intents(job_id),
attempt_id TEXT NOT NULL,
domain_receipt TEXT NOT NULL,
PRIMARY KEY (job_id, attempt_id)
);
CREATE TRIGGER job_reconciliation_immutable BEFORE UPDATE ON job_domain_reconciliations
BEGIN SELECT RAISE(ABORT, 'immutable domain reconciliation'); END;
CREATE TRIGGER job_reconciliation_retained BEFORE DELETE ON job_domain_reconciliations
BEGIN SELECT RAISE(ABORT, 'retained domain reconciliation'); END;
";
const GRANT_SCHEMA: &str = "
CREATE TABLE job_domain_grants (
job_id TEXT PRIMARY KEY NOT NULL REFERENCES job_intents(job_id),
@@ -732,6 +809,98 @@ mod tests {
}
}
#[test]
fn granted_unknown_reconciliation_releases_serialization_but_preserves_history() {
let (dir, mut store) = fixture();
let mut first = request("first");
first.serialization_key = Some("journal".into());
let reserved = store
.reserve_granted(first, Some(json!({"domain":"journal"})))
.unwrap();
store
.dispatch("first", &reserved.attempt.attempt_id, 8)
.unwrap();
let unknown = store
.finish(
"first",
&reserved.attempt.attempt_id,
JobAttemptState::Unknown,
Some("response lost"),
)
.unwrap();
let mut second = request("second");
second.serialization_key = Some("journal".into());
assert!(matches!(
store.reserve(second.clone()),
Err(JobStoreError::SerializationConflict(_))
));
assert!(
store
.reconcile_granted_attempt("first", "foreign-attempt", "domain-fenced")
.is_err()
);
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
assert!(
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "different-proof")
.is_err()
);
assert_eq!(store.get("first").unwrap(), unknown);
store.reserve(second).unwrap();
assert!(store.retry("first").is_err());
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(reopened.get("first").unwrap(), unknown);
assert!(reopened.unknown_granted().unwrap().is_empty());
assert!(
reopened
.conn
.execute("DELETE FROM job_domain_reconciliations", [])
.is_err()
);
assert!(reopened.retry("first").is_err());
}
#[test]
fn schema_three_migration_preserves_unknown_grant_and_its_serialization_fence() {
let (dir, mut store) = fixture();
let mut first = request("first");
first.serialization_key = Some("journal".into());
let reserved = store
.reserve_granted(first, Some(json!({"domain":"journal"})))
.unwrap();
store
.dispatch("first", &reserved.attempt.attempt_id, 8)
.unwrap();
let unknown = store
.finish(
"first",
&reserved.attempt.attempt_id,
JobAttemptState::Unknown,
Some("unconfirmed"),
)
.unwrap();
store.conn.execute_batch("DROP TRIGGER job_reconciliation_immutable; DROP TRIGGER job_reconciliation_retained; DROP TABLE job_domain_reconciliations; DROP INDEX job_serialization_singleflight; CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key) WHERE serialization_key IS NOT NULL AND state IN ('pending','unknown'); PRAGMA user_version=3;").unwrap();
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(reopened.get("first").unwrap(), unknown);
let mut next = request("next");
next.serialization_key = Some("journal".into());
assert!(matches!(
reopened.reserve(next.clone()),
Err(JobStoreError::SerializationConflict(_))
));
reopened
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
reopened.reserve(next).unwrap();
}
fn fixture() -> (TempDir, JobStore) {
let dir = tempfile::tempdir().unwrap();
let store = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
+54
View File
@@ -66,6 +66,15 @@ struct Executions {
jobs: HashMap<String, Execution>,
}
pub(crate) struct QuiescentJobClosure<'a> {
executions: std::sync::MutexGuard<'a, Executions>,
}
impl QuiescentJobClosure<'_> {
pub(crate) fn commit(&mut self) {
self.executions.closed = true;
}
}
/// Cloneable Feature-facing access to a Host-owned service. Cloning this handle
/// neither extends the Host's authority nor permits opening/recovering its store.
#[derive(Clone)]
@@ -79,6 +88,27 @@ pub struct StandaloneJobs {
}
impl StandaloneJobs {
/// Excludes request/start/result acknowledgement while a Subject Host checks its final
/// saved obligations and releases ownership. Accepted results alone are insufficient:
/// every spawned execution must have actually finished successful cleanup.
pub(crate) fn reserve_quiescent_closure(
&self,
) -> Result<QuiescentJobClosure<'_>, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
if executions.jobs.values().any(|execution| {
!execution.task.is_finished()
|| !matches!(execution.complete.borrow().as_ref(), Some(Ok(())))
}) {
return Err(JobServiceError::Cleanup(
"delegated Job execution/cleanup remains running or unconfirmed".into(),
));
}
Ok(QuiescentJobClosure { executions })
}
pub(crate) fn open(path: &Path, cwd: PathBuf) -> Result<Self, JobServiceError> {
Ok(Self {
store: Arc::new(Mutex::new(Some(JobStore::open(path)?))),
@@ -157,6 +187,30 @@ impl StandaloneJobs {
self.with_store(|store| store.unacknowledged())
}
pub(crate) fn unknown_granted(&self) -> Result<Vec<JobSnapshot>, JobServiceError> {
self.with_store(|store| store.unknown_granted())
}
/// Release only an Unknown attempt's serialization reservation after the bound domain
/// has reconciled its receipts and durably fenced further effects. Never a generic retry.
pub(crate) fn reconcile_granted_attempt(
&self,
job_id: &str,
attempt_id: &str,
domain_receipt: &str,
) -> Result<(), JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
if executions.jobs.contains_key(job_id) {
return Err(JobServiceError::Cleanup(
"domain reconciliation requires observed execution cleanup".into(),
));
}
self.with_store(|store| store.reconcile_granted_attempt(job_id, attempt_id, domain_receipt))
}
/// Explicit new attempt, bounded by immutable intent limits. Unknown
/// outcomes cannot be retried without domain reconciliation/new intent.
pub fn retry(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
+42 -1
View File
@@ -7,8 +7,49 @@ pub mod launch;
pub mod store;
pub mod subjektiv;
mod subjektiv_sessions;
pub mod unix;
pub use host::{StandaloneHost, StandaloneShutdownError, StandaloneStartupError};
pub use unix::{UnixEndpoint, relay_unix_client, run_unix_client};
/// Restore reads the saved cwd and Profile, not a newly resolved launch recipe.
pub enum HeadlessLaunch {
Fresh(ResolvedStandaloneLaunch),
Restore {
state_dir: std::path::PathBuf,
worker_id: WorkerId,
},
}
#[derive(Debug, thiserror::Error)]
pub enum HeadlessError {
#[error(transparent)]
Transport(#[from] std::io::Error),
#[error(transparent)]
Startup(#[from] StandaloneStartupError),
#[error(transparent)]
Shutdown(#[from] StandaloneShutdownError),
}
/// Reserve the optional public transport before creating or restoring execution state.
pub async fn run_headless(
launch: HeadlessLaunch,
endpoint: std::path::PathBuf,
) -> Result<(), HeadlessError> {
let endpoint = UnixEndpoint::bind(endpoint)?;
let host = match launch {
HeadlessLaunch::Fresh(launch) => StandaloneHost::start(launch).await?,
HeadlessLaunch::Restore {
state_dir,
worker_id,
} => StandaloneHost::restore(state_dir, worker_id).await?,
};
host.serve_unix(endpoint).await
}
pub use host::{
StandaloneHost, StandaloneShutdownError, StandaloneStartupError,
StandaloneSubjectDisconnectError,
};
pub use launch::{ResolvedStandaloneLaunch, StandaloneLaunchConfig, StandaloneLaunchError};
pub use protocol::WorkerId;
pub use store::{
+26 -1
View File
@@ -119,6 +119,16 @@ pub enum StaleLeasePolicy {
Recover,
}
/// Source control outlives Worker relocation and physical Session pruning. Always derive this
/// from the Host state root, never from an individual live or retained Worker directory.
pub(crate) fn session_retention(
state_root: &Path,
) -> session_store::retention::SessionRetentionStore {
session_store::retention::SessionRetentionStore::new(
state_root.join("retention/session-sources"),
)
}
#[derive(Debug, Clone)]
pub struct StandaloneWorkerStore {
root: PathBuf,
@@ -359,7 +369,19 @@ impl StandaloneWorkerStore {
}
pub fn delete(&self, id: WorkerId) -> Result<(), StandaloneStoreError> {
let record = self.load(id)?;
let record = match self.load(id) {
Err(StandaloneStoreError::WorkerNotFound(_)) => {
let retained_root = self.root.join("subjektiv").join("retained-workers");
let retained =
Self::open_existing(retained_root).and_then(|store| store.load(id))?;
if retained.status == StandaloneWorkerStatus::Stopped && retained.subject.is_some()
{
return Ok(());
}
return Err(StandaloneStoreError::InvalidRecord(id));
}
result => result?,
};
if record.status != StandaloneWorkerStatus::Stopped {
return Err(StandaloneStoreError::DeleteActive(id));
}
@@ -374,6 +396,7 @@ impl StandaloneWorkerStore {
LeaseLiveness::Unknown => StandaloneStoreError::LeaseLivenessUnknown(id),
});
}
crate::subjektiv::prepare_worker_deletion(&self.root, &record)?;
fs::remove_dir_all(self.worker_dir(id)).map_err(StandaloneStoreError::Io)?;
sync_directory(&self.root)
}
@@ -793,6 +816,8 @@ pub enum StandaloneStoreError {
LeaseOwnershipLost,
#[error("standalone Worker {0} must be stopped before deletion")]
DeleteActive(WorkerId),
#[error("Journal deletion handoff/retention is unconfirmed: {0}")]
JournalRetention(String),
#[error("standalone Worker {0} metadata changed since it was read")]
MetadataChanged(WorkerId),
#[error("system clock is before the Unix epoch or out of range")]
+225 -10
View File
@@ -5,9 +5,11 @@
//! non-private nodes; delegates schema/migration to `subjektiv` and connection policy to `feature-storage`.
//!
//! The Host holds each selected Subject's exclusive kernel lease through body execution and
//! confirmed delegated Job cleanup. Process death releases kernel locks; uncertain in-process
//! cleanup retains leases. Restore validates saved Subject/storage bindings and reacquires the
//! same Subject. Launches without a selected Subject do not implicitly connect this store.
//! confirmed delegated Job cleanup. Kernel lock release never clears durable body ownership:
//! shutdown/process death permits only the same body to restore, not a replacement to start.
//! Explicit disconnection requires actual Controller quiescence and final Journal completion;
//! storage then revokes that writer lifetime before the lease is released. Uncertain cleanup
//! retains leases. Launches without a selected Subject do not implicitly connect this store.
//!
//! The selected Subject's Job store is shared across Workers; the generic Jobs service owns execution
//! and cleanup. Consolidation grants allow only immutable candidate batches and limited operations,
@@ -15,6 +17,9 @@
//! Unknown attempts are not rerun; delivery failures for accepted results do not trigger model
//! reexecution. Memory application and surface repair use shared-domain CAS/receipts.
mod journal;
pub(crate) use journal::prepare_worker_deletion;
use std::fs::{self, File, OpenOptions};
use std::io::Write;
use std::path::{Path, PathBuf};
@@ -46,6 +51,8 @@ pub enum SubjectError {
#[error(transparent)]
Domain(#[from] subjektiv::SubjektivError),
#[error(transparent)]
Journal(#[from] subjektiv::journal::JournalError),
#[error(transparent)]
Storage(#[from] feature_storage::FeatureStorageError),
#[error("local Subject storage metadata: {0}")]
Json(#[from] serde_json::Error),
@@ -220,7 +227,51 @@ impl StandaloneSubjects {
scope_id: self.scope_id.clone(),
subject_id: subject_id.to_string(),
};
let host = Arc::new(LocalSubjectHost {
if self.store.journal_worker_was_released(
subject_id,
&self.scope_id,
&worker_id.to_string(),
)? {
return Err(SubjectError::InvalidScope("this body's Subject binding was released; start a new Worker/Session rather than restore its old authority".into()));
}
// Before writer lifetimes existed, a saved body record was the only durable
// attachment evidence. Never interpret its stopped status as a release on upgrade.
let workers = crate::store::StandaloneWorkerStore::open_existing(&self.state_root)
.map_err(|error| SubjectError::InvalidScope(error.to_string()))?;
for entry in fs::read_dir(&self.state_root)? {
let entry = entry?;
let Ok(prior_id) = entry
.file_name()
.to_string_lossy()
.parse::<protocol::WorkerId>()
else {
continue;
};
if prior_id == worker_id {
continue;
}
let prior = match workers.load(prior_id) {
Ok(prior) => prior,
Err(crate::store::StandaloneStoreError::WorkerNotFound(_)) => continue,
Err(error) => return Err(SubjectError::InvalidScope(error.to_string())),
};
if prior.subject.as_ref() == Some(&binding)
&& !self.store.journal_worker_was_released(
subject_id,
&self.scope_id,
&prior_id.to_string(),
)?
{
return Err(SubjectError::InvalidScope("saved Subject body has no explicit release; restore it and complete the Idle/Journal handoff, rather than treating inactive storage as unoccupied".into()));
}
}
let writer =
self.store
.claim_journal_writer(subject_id, &self.scope_id, &worker_id.to_string())?;
let host = Arc::new_cyclic(|weak| LocalSubjectHost {
writer,
self_weak: weak.clone(),
journal_dispatch: Mutex::new(false),
language: self.memory_language.clone(),
catalog: self,
binding: binding.clone(),
@@ -247,6 +298,37 @@ pub(crate) struct SubjectConnection {
armed: bool,
}
impl SubjectConnection {
/// Construction failed before any Controller or mandatory startup context was published.
/// This is cancellation of an unpublished reservation, not Idle inferred from a stopped body.
pub(crate) fn abort_unpublished_start(&mut self) -> Result<(), SubjectError> {
self.host
.catalog
.store
.release_journal_writer(&self.host.writer)?;
self.close()
}
pub(crate) fn writer_released_or_unknown(&self) -> bool {
self.host
.catalog
.store
.journal_writer_active(&self.host.writer)
.map(|active| !active)
.unwrap_or(true)
}
pub(crate) fn release_for_handoff(
&mut self,
record: &crate::store::StandaloneWorkerRecord,
reservation: &mut worker::SubjectReleaseReservation,
) -> Result<(), String> {
if let Err(error) = self.host.release_for_handoff(record, reservation) {
self.host.wake_journals();
return Err(error.to_string());
}
reservation.commit_disconnect()
}
pub fn arm(&mut self) {
self.armed = true;
}
@@ -277,6 +359,9 @@ impl Drop for SubjectConnection {
}
pub(crate) struct LocalSubjectHost {
writer: subjektiv::journal::JournalWriterBinding,
self_weak: std::sync::Weak<Self>,
journal_dispatch: Mutex<bool>,
catalog: StandaloneSubjects,
binding: StandaloneSubjectBinding,
worker_id: protocol::WorkerId,
@@ -293,6 +378,16 @@ impl LocalSubjectHost {
"local Subject Host is closed".into(),
));
}
if !self
.catalog
.store
.journal_writer_active(&self.writer)
.map_err(journal::journal_error)?
{
return Err(SubjektivHostError::Unavailable(
"Subject writer binding was released".into(),
));
}
Ok(())
}
fn body(&self, context: &SubjektivHostContext) -> Result<(), SubjektivHostError> {
@@ -332,6 +427,9 @@ impl LocalSubjectHost {
"Session does not belong to this local Subject/Worker scope".into(),
));
}
store
.bind_journal_session(&self.writer, record.clone())
.map_err(journal::journal_error)?;
return Ok(record);
}
if !create {
@@ -348,9 +446,13 @@ impl LocalSubjectHost {
&context.session_id,
)
.map_err(domain_error)?;
store
let record = store
.record_session_attribution(record)
.map_err(domain_error)
.map_err(domain_error)?;
store
.bind_journal_session(&self.writer, record.clone())
.map_err(journal::journal_error)?;
Ok(record)
}
}
@@ -551,10 +653,11 @@ impl LocalSubjectHost {
.and_then(|v| serde_json::from_value::<LocalConsolidationGrant>(v.clone()).ok())
.is_some_and(|g| g.binding == self.binding)
{
schedule_job(jobs.clone(), job.request.job_id);
schedule_job(jobs.clone(), job.request.job_id, self.self_weak.clone());
}
}
}
self.recover_journals();
}
fn validate_evidence(
&self,
@@ -690,6 +793,107 @@ impl LocalSubjectHost {
}
}
impl SubjektivHost for LocalSubjectHost {
fn memory_journal(
&self,
context: &SubjektivHostContext,
request: subjektiv::journal::memory_links::MemoryJournalRequest,
) -> Result<subjektiv::journal::memory_links::MemoryJournalResponse, SubjektivHostError> {
self.body(context)?;
let reader = subjektiv::journal::JournalHostContext::validated_reader(
&self.catalog.store,
&self.binding.subject_id,
)
.map_err(journal::journal_error)?;
self.catalog
.store
.execute_memory_journal_with_source_presence(&reader, request, |session| {
crate::subjektiv_sessions::original_session_presence(
&self.catalog.state_root,
&self.binding.scope_id,
&self.binding.subject_id,
&self.catalog.store,
session,
)
})
.map_err(journal::journal_error)
}
fn journal_startup_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn journal_startup_context(
&self,
context: &SubjektivHostContext,
) -> Result<subjektiv::journal::startup::JournalStartupSnapshot, SubjektivHostError> {
self.body(context)?;
let reader = subjektiv::journal::JournalHostContext::validated_reader(
&self.catalog.store,
&self.binding.subject_id,
)
.map_err(journal::journal_error)?;
let predecessor = self
.catalog
.store
.journal_predecessor_session(&self.writer)
.map_err(journal::journal_error)?;
self.catalog
.store
.journal_startup_snapshot(&reader, predecessor.as_deref())
.map_err(journal::journal_error)
}
fn journal_compaction_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn journal_compaction_context(
&self,
context: &SubjektivHostContext,
) -> Result<subjektiv::journal::startup::JournalStartupSnapshot, SubjektivHostError> {
self.body(context)?;
let attribution = self
.catalog
.store
.session_attribution(&context.session_id)
.map_err(domain_error)?
.filter(|attr| {
attr.subject_id == self.binding.subject_id
&& attr.runtime_id == self.binding.scope_id
&& attr.worker_id == self.worker_id.to_string()
})
.ok_or_else(|| {
SubjektivHostError::InvalidArgument(
"foreign or unattributed compaction Session".into(),
)
})?;
let host = subjektiv::journal::JournalHostContext::validated_session(
&self.catalog.store,
attribution,
)
.map_err(journal::journal_error)?;
if host.writer_binding_id() != Some(self.writer.binding_id.as_str()) {
return Err(SubjektivHostError::InvalidArgument(
"compaction writer lifetime mismatch".into(),
));
}
self.catalog
.store
.journal_compaction_snapshot(&host)
.map_err(journal::journal_error)
}
fn journal_recording_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn request_journal_recording(
&self,
context: &SubjektivHostContext,
_reason: subjektiv::journal::transport::JournalRecordingReason,
) -> Result<subjektiv::journal::transport::JournalRecordingResponse, SubjektivHostError> {
let admitted = self.admit_journal(context)?;
self.wake_journals();
Ok(if admitted {
subjektiv::journal::transport::JournalRecordingResponse::Accepted
} else {
subjektiv::journal::transport::JournalRecordingResponse::NoNewSources
})
}
fn settings(&self) -> SubjektivHostSettings {
SubjektivHostSettings {
language: self.language.clone(),
@@ -788,6 +992,7 @@ impl SubjektivHost for LocalSubjectHost {
create_if_missing: bool,
) -> Result<SubjektivRecordSessionResponse, SubjektivHostError> {
let attribution = self.attribution(context, create_if_missing)?;
self.reserve_journal_session(context)?;
Ok(SubjektivRecordSessionResponse {
subject_id: attribution.subject_id,
session_id: attribution.session_id,
@@ -811,7 +1016,7 @@ impl SubjektivHost for LocalSubjectHost {
.is_some_and(|g| g.binding == self.binding)
})
{
schedule_job(jobs, pending.request.job_id);
schedule_job(jobs, pending.request.job_id, self.self_weak.clone());
let (count, bytes) = self
.catalog
.store
@@ -874,7 +1079,7 @@ impl SubjektivHost for LocalSubjectHost {
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?,
)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?;
schedule_job(jobs, snapshot.request.job_id);
schedule_job(jobs, snapshot.request.job_id, self.self_weak.clone());
Ok(memory::backend::MemoryConsolidationOutput { status:"started".into(),summary:"Requested a durable generic standalone consolidation Job; processing stops on Host shutdown".into(),candidate_count:count,total_bytes:bytes })
}
}
@@ -894,6 +1099,9 @@ impl crate::jobs::JobDomainProvider for LocalDomainProvider {
) -> Result<worker::job::JobFeatureGrant, String> {
let host = self.0.upgrade().ok_or("Subject Host unavailable")?;
host.ensure_live().map_err(|e| e.to_string())?;
if snapshot.request.purpose == journal::PURPOSE {
return journal::grant(host, snapshot, fence);
}
let grant: LocalConsolidationGrant = serde_json::from_value(
snapshot
.domain_grant
@@ -1097,7 +1305,11 @@ impl worker::job::JobResultFinalizer for LocalConsolidationFinalizer {
}
}
fn schedule_job(jobs: crate::jobs::StandaloneJobs, id: String) {
fn schedule_job(
jobs: crate::jobs::StandaloneJobs,
id: String,
host: std::sync::Weak<LocalSubjectHost>,
) {
// Scheduling/consumer acknowledgement has no Internal Worker ownership.
// The generic service fences and retains every execution until cleanup.
tokio::spawn(async move {
@@ -1114,6 +1326,9 @@ fn schedule_job(jobs: crate::jobs::StandaloneJobs, id: String) {
"standalone Subject Job {id} cleanup/acknowledgement is unconfirmed: {error}"
),
}
if let Some(host) = host.upgrade() {
host.wake_journals();
}
});
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,463 @@
//! Public standalone protocol through real mid-turn compaction and Journal Jobs.
//! Only external model streams are scripted. No history, attribution, hook or Job
//! result is injected; cancellation and rollback use the controller's public commands.
//! Reported model usage and a low configured request threshold force two rewrites.
//! The test attests durable public originals, not unfinished display deltas or meaning.
use super::*;
use agen::llm_client::{
ClientError, LlmClient, Request,
event::{Event, ResponseStatus, StatusEvent, StopReason, UsageEvent},
types::Item,
};
use futures::{Stream, StreamExt};
use std::pin::Pin;
use std::sync::atomic::AtomicUsize;
type ResponseStream = Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>;
const SUMMARY: &str = "DERIVED_COMPACTION_SENTINEL_NOT_AN_ORIGINAL";
const ROLLED_BACK: &str = "ROLLED_BACK_INPUT_MUST_NOT_REACH_JOURNAL";
const PARTIAL: &str = "public partial answer before cancellation";
#[derive(Clone, Default)]
struct CompactingDialog {
shared: DialogAndJournal,
dialog_calls: Arc<AtomicUsize>,
compactions: Arc<AtomicUsize>,
rollback_entered: Arc<tokio::sync::Notify>,
partial_stream_paused: Arc<tokio::sync::Notify>,
}
fn events(events: Vec<Event>) -> ResponseStream {
Box::pin(futures::stream::iter(events.into_iter().map(Ok)))
}
fn done() -> ResponseStream {
events(vec![
Event::text_block_start(0),
Event::text_delta(0, "done"),
Event::text_block_stop(0, Some(StopReason::EndTurn)),
])
}
#[async_trait::async_trait]
impl LlmClient for CompactingDialog {
async fn stream(&self, request: Request) -> Result<ResponseStream, ClientError> {
if request
.tools
.iter()
.any(|tool| tool.name == "write_summary")
{
if request.items.iter().any(|item| matches!(item, Item::ToolResult { call_id, .. } if call_id == "compact-summary")) {
return Ok(done());
}
let count = self.compactions.fetch_add(1, Ordering::SeqCst);
assert!(count < 2, "unexpected additional compaction");
return Ok(events(vec![
Event::tool_use_start(0, "compact-summary", "write_summary"),
Event::tool_input_delta(0, serde_json::json!({"text": SUMMARY}).to_string()),
Event::tool_use_stop(0),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]));
}
if request.tools.iter().any(|tool| {
matches!(
tool.name.as_str(),
"ReadJournalState" | "SubmitMemorySurface" | "SubmitJobResult"
)
}) {
return self.shared.stream(request).await;
}
match self.dialog_calls.fetch_add(1, Ordering::SeqCst) {
step @ (0 | 1) => {
assert!(request.tools.iter().any(|tool| tool.name == "TaskCreate"));
Ok(events(vec![
Event::tool_use_start(0, format!("public-task-{step}"), "TaskCreate"),
Event::tool_input_delta(
0,
serde_json::json!({
"subject": format!("original activity {step}"),
"description": format!("original task result before compaction {step}")
})
.to_string(),
),
Event::tool_use_stop(0),
Event::Usage(UsageEvent {
input_tokens: Some(100_000),
output_tokens: Some(10),
total_tokens: Some(100_010),
..Default::default()
}),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]))
}
2 => {
let paused = self.partial_stream_paused.clone();
Ok(Box::pin(
events(vec![
Event::text_block_start(0),
Event::text_delta(0, PARTIAL),
Event::text_block_stop(0, None),
])
.chain(futures::stream::once(async move {
paused.notify_one();
std::future::pending::<Result<Event, ClientError>>().await
})),
))
}
3 => {
// No AI event: cancelling this request must roll back the submitted input.
self.rollback_entered.notify_one();
std::future::pending::<()>().await;
unreachable!()
}
other => panic!("unexpected dialog request {other}"),
}
}
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
}
async fn cancel_and_wait(
client: &mut client::Client<client::transport::in_process::Socket>,
sequence: u64,
) {
client
.send(&protocol::Method::Cancel {
command: protocol::WorkerCommandEnvelope::new(sequence),
})
.await
.unwrap();
tokio::time::timeout(std::time::Duration::from_secs(20), async {
loop {
match client.next_event().await.unwrap().unwrap() {
protocol::Event::RunEnd { result } => {
assert_ne!(result, protocol::RunResult::Finished);
return;
}
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
}
})
.await
.unwrap();
}
#[tokio::test]
async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_and_rollback() {
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("compaction-journal", None).unwrap();
let model = CompactingDialog {
shared: DialogAndJournal {
journal: JournalScript {
gate_first: true,
..Default::default()
},
..Default::default()
},
..Default::default()
};
let mut launch = crate::StandaloneLaunchConfig::new(
temp.path(),
&state,
manifest::ProfileSelector::source_named(
manifest::ProfileRegistrySource::Builtin,
"standalone-subjektiv",
),
"public-compaction-journal",
)
.with_subject(subject.id.clone())
.resolve()
.unwrap();
launch.profile.manifest.compaction = Some(manifest::CompactionConfig {
threshold: None,
request_threshold: Some(50_000),
retained_tokens: 0,
..Default::default()
});
let mut host = crate::StandaloneHost::start_with_model_client(launch, model.clone())
.await
.unwrap();
let session = host.record().active_session_id;
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
let source_store = session_store::FsStore::new_with_retention(
workers.sessions_dir(host.record().worker_id),
crate::store::session_retention(&state),
)
.unwrap();
let mut client = host.connect();
client
.send(&protocol::Method::submit_text(
protocol::new_submission_request_id(),
"original long turn survives compaction",
))
.await
.unwrap();
let mut completed_compactions = 0;
loop {
match client.next_event().await.unwrap().unwrap() {
protocol::Event::CompactionProgress {
compaction: Some(progress),
} if progress.phase == protocol::CompactionPhase::Committing => {
assert_eq!(
progress.trigger,
protocol::CompactionTrigger::RequestThreshold
);
completed_compactions += 1;
}
protocol::Event::TextDelta { text } if text.contains(PARTIAL) => break,
protocol::Event::RunEnd { .. } => panic!("dialog ended before cancellation"),
protocol::Event::SessionEntryCommitted { entry } if matches!(entry.data, protocol::SessionSnapshotEntryData::RunError { .. }) => panic!("dialog failed before cancellation: {entry:?}"),
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
}
assert_eq!(model.compactions.load(Ordering::SeqCst), 2);
assert_eq!(completed_compactions, 2);
model.shared.journal.entered.notified().await;
let context = JournalHostContext::validated_session(
&catalog.store,
catalog
.store
.session_attribution(&session.to_string())
.unwrap()
.unwrap(),
)
.unwrap();
let reader = JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap();
let journal = catalog
.store
.list_journals(&reader, None, 1)
.unwrap()
.items
.pop()
.unwrap();
let first = catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap();
assert!(
catalog
.store
.list_journal_entries(&reader, &journal.id, None, 10)
.unwrap()
.items
.is_empty(),
"compaction must finish before generation is released"
);
model.partial_stream_paused.notified().await;
cancel_and_wait(&mut client, 1).await;
client
.send(&protocol::Method::submit_text(
protocol::new_submission_request_id(),
ROLLED_BACK,
))
.await
.unwrap();
model.rollback_entered.notified().await;
let before_rollback: Vec<_> = source_store
.list_segments(session)
.unwrap()
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
assert!(
serde_json::to_string(&before_rollback)
.unwrap()
.contains(ROLLED_BACK),
"the submitted input must reach the actual durable rollback boundary"
);
cancel_and_wait(&mut client, 2).await;
assert_eq!(
catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap(),
first,
"compaction/cancellation must not alter the running generation's frozen input"
);
assert!(matches!(source_store.delete_session(session),
Err(session_store::StoreError::SessionPinned(id)) if id == session));
let segments = source_store.list_segments(session).unwrap();
assert_eq!(segments.len(), 3);
let raw: Vec<_> = segments
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
for history in raw.iter().filter_map(|record| match record {
session_store::LogEntry::AnnotatedSegmentStart { history, compacted_from: Some(_), .. } => Some(history),
_ => None,
}) {
let mut calls = std::collections::HashSet::new();
let mut results = 0;
for entry in history {
let is_tool = matches!(entry.item, session_store::LoggedItem::ToolCall { .. } | session_store::LoggedItem::ToolResult { .. });
let Some(metadata) = entry.metadata.input_metadata.as_ref().filter(|_| is_tool) else { continue; };
assert_eq!(metadata.usage().recording_material, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().memory_evidence, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().observation, session_store::InputObservationUse::Hidden);
assert!(metadata.public_item().is_none());
assert!(entry.metadata.derivation.is_some());
match &entry.item {
session_store::LoggedItem::ToolCall { call_id, .. } => { calls.insert(call_id.clone()); }
session_store::LoggedItem::ToolResult { call_id, .. } => {
assert!(calls.contains(call_id), "derived Tool result must follow its real logical call");
results += 1;
}
_ => unreachable!(),
}
}
assert!(!calls.is_empty());
assert!(results >= calls.len());
}
assert_eq!(
raw.iter()
.filter(|entry| matches!(
entry,
session_store::LogEntry::AnnotatedSegmentStart {
compacted_from: Some(_),
..
}
))
.count(),
2
);
assert!(
!serde_json::to_string(&raw).unwrap().contains(ROLLED_BACK),
"pre-AI cancellation must roll back the speculative input"
);
assert!(
serde_json::to_string(&raw).unwrap().contains(SUMMARY),
"real rewrite did not persist its summary"
);
let metadata =
session_store::FsWorkerStore::new(workers.worker_metadata_dir(host.record().worker_id))
.unwrap()
.read_by_name(&host.record().storage_key)
.unwrap()
.unwrap();
let originals =
session_store::public_material::SessionPublicMaterialReader::open_fs_with_retention(
&workers.sessions_dir(host.record().worker_id),
session,
metadata.active.unwrap().segment_id.unwrap(),
session_store::public_material::PublicMaterialLimits::default(),
crate::store::session_retention(&state),
)
.unwrap();
let final_original = originals.descriptors().last().unwrap();
let expected_cursor = (final_original.canonical_order + 1, 0);
let canonical: Vec<_> = originals
.descriptors()
.map(|descriptor| {
let page = originals
.read_content(descriptor, 0, MAX_JOURNAL_READ_BYTES)
.unwrap();
assert_eq!(page.next_offset, descriptor.total_bytes);
page.content
})
.collect();
assert!(!canonical.join("\n").contains(ROLLED_BACK));
assert!(!canonical.join("\n").contains(SUMMARY));
// Resume the existing runner, then its one durable successor, with no more input.
let jobs = host.jobs();
model.shared.journal.release.notify_one();
jobs.wait(&first.job.job_id).await.unwrap();
model.shared.journal.entered.notified().await;
let second = catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap();
assert_ne!(first.id, second.id);
model.shared.journal.release.notify_one();
jobs.wait(&second.job.job_id).await.unwrap();
assert!(!catalog.store.journal_retention_required(&context).unwrap());
let (saved, open) = catalog
.store
.journal_resume_state(&context, &journal.id)
.unwrap();
let open = open.unwrap();
assert!(!open.ongoing.is_empty());
let entries = catalog
.store
.list_journal_entries(&reader, &journal.id, None, 10)
.unwrap();
assert_eq!(
entries.items.len(),
1,
"compaction/turn boundaries must not fix or duplicate the open entry"
);
assert_eq!(entries.items[0].id, open.id);
let supplied = model
.shared
.journal
.source_reads
.lock()
.unwrap()
.values()
.cloned()
.collect::<Vec<_>>()
.join("\n");
for original in [
"original long turn survives compaction",
"original activity 0",
"original activity 1",
] {
assert!(
supplied.contains(original),
"Journal did not receive {original}"
);
}
for original in canonical {
assert!(
supplied.contains(&original),
"a durable public original was skipped"
);
}
assert!(
!supplied.contains(SUMMARY),
"derived summary substituted for original input"
);
assert!(
!supplied.contains(ROLLED_BACK),
"cancelled input was admitted after rollback"
);
assert_eq!((saved.process_order, saved.process_byte), expected_cursor);
let position = catalog
.store
.read_journal_text(
&reader,
&journal.id,
None,
JournalTextField::CurrentPosition,
None,
0,
MAX_JOURNAL_READ_BYTES,
)
.unwrap();
assert!(!position.text.is_empty());
loop {
match host.disconnect_subject().await {
Ok(()) => break,
Err(crate::StandaloneSubjectDisconnectError::Incomplete(message)) if message.contains("dispatcher") || message.contains("cleanup") => tokio::task::yield_now().await,
Err(error) => panic!("recorded compaction/cancellation lineage must remain releasable while work is unfinished: {error}"),
}
}
host.shutdown().await.unwrap();
})
.await
.expect("bounded public compaction/Journal integration");
}
@@ -0,0 +1,170 @@
//! Real LocalSubjectHost/Journal storage -> PreparedWorker -> compactor Request -> durable seed.
//! Only external LLM responses and the fixture's Job execution fence are substituted. The
//! in-flight generation is retained while its real adapter updates storage during generation.
//! Temporary storage and the parent fixture own cleanup; ordinary standalone lib tests select
//! this suite. Predetermined summaries attest transport and persistence, not model understanding.
use super::*;
use agen::llm_client::{
ClientError, LlmClient, Request,
event::{Event, ResponseStatus, StatusEvent},
types::Item,
};
use futures::Stream;
use session_store::{CombinedStore, FsWorkerStore, LogEntry};
use std::pin::Pin;
const BEFORE: &str = "FROZEN_POSITION: B partially implemented; tests not run; result unconfirmed";
const AFTER: &str = "NEXT_POSITION: background Journal changed during compactor generation";
const COMPLEMENT: &str =
"Unrecorded request: keep the user's constraint and finish the remaining checks.";
#[derive(Clone)]
struct MockClient {
adapter: Arc<LocalJournalHost>,
requests: Arc<Mutex<Vec<Request>>>,
updated: Arc<AtomicBool>,
}
#[async_trait::async_trait]
impl LlmClient for MockClient {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(
&self,
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>, ClientError> {
assert!(
request
.tools
.iter()
.any(|tool| tool.name == "write_summary")
);
let first = self.requests.lock().unwrap().is_empty();
self.requests.lock().unwrap().push(request);
let events = if first {
assert!(!self.updated.swap(true, Ordering::SeqCst));
let state = self.adapter.state().unwrap();
save(
&self.adapter,
"position-during-compaction",
&state.current_journal.state_token,
JournalAction::UpdatePosition {
current_position: AFTER.into(),
},
vec![],
);
vec![
Event::tool_use_start(0, "summary", "write_summary"),
Event::tool_input_delta(0, serde_json::json!({ "text": COMPLEMENT }).to_string()),
Event::tool_use_stop(0),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
} else {
vec![
Event::text_block_start(0),
Event::text_delta(0, "summary saved"),
Event::text_block_stop(0, None),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
};
Ok(Box::pin(futures::stream::iter(events.into_iter().map(Ok))))
}
}
#[tokio::test]
async fn local_host_compaction_freezes_pending_journal_through_request_and_durable_seed_during_update()
{
tokio::time::timeout(std::time::Duration::from_secs(30), async {
let fixture = Fixture::new("Keep the user's constraint; work remains unfinished and unconfirmed.");
let (adapter, _, job) = fixture.adapter();
let created = save(
&adapter,
"record-partial-work",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "A corrected to B".into(),
body: "B partially implemented; tests not run; result unconfirmed".into(),
ongoing: "Finish remaining checks".into(),
state: JournalEntryState::Open,
source_end: None,
correction_refs: vec![],
},
adapter.generation.selected_ranges.clone(),
);
save(&adapter, "position-before-compaction", &created.state_token,
JournalAction::UpdatePosition { current_position: BEFORE.into() }, vec![]);
let before = adapter.state().unwrap();
let expected = fixture.host().journal_compaction_context(&fixture.context).unwrap();
let journal = &expected.journals[0];
assert_eq!(journal.source_session_id, fixture.context.session_id);
assert_eq!(journal.current_position, BEFORE);
assert_eq!(journal.entries[0].state, JournalEntryState::Open);
assert!(!before.selected_ranges.is_empty());
assert_eq!(journal.process_order, before.current_journal.process_order);
assert_eq!(journal.process_byte, before.current_journal.process_byte);
assert_eq!(journal.process_order, 0);
assert_eq!(journal.process_byte, 0, "saved open records do not advance processing checkpoint");
let expected_json = serde_json::to_string(&expected).unwrap();
let adapter = Arc::new(adapter);
let model = MockClient { adapter: adapter.clone(), requests: Arc::default(), updated: Arc::new(AtomicBool::new(false)) };
let workers = StandaloneWorkerStore::open_existing(&fixture.host().catalog.state_root).unwrap();
let worker_id = fixture._allocation.as_ref().unwrap().worker_id();
let store = CombinedStore::new(fixture.sessions.clone(), FsWorkerStore::new(workers.worker_metadata_dir(worker_id)).unwrap());
let mut manifest = manifest::WorkerManifest::from_toml(&format!("[worker]\nname='{}'\n[model]\nscheme='anthropic'\nmodel_id='scripted'\n[engine]\n[scope]\nallow=[]", fixture.context.worker_id)).unwrap();
manifest.feature.subjektiv.profile.enabled = true;
manifest.feature.subjektiv.profile.extraction.enabled = false;
let mut prepared = worker::WorkerBootstrap::new(
manifest,
store,
worker::PromptCatalogSource::builtins_only(),
worker::WorkerWorkspaceContext::no_workspace(),
worker::WorkerFilesystemAuthority::None,
worker::WorkerBootstrapLayout::Direct { runtime_base: fixture._temp.path().join("runtime") },
).with_model_client(model.clone()).with_subjektiv_host(fixture.host().clone()).prepare_restored("journal-test").await.unwrap();
assert_eq!(prepared.worker().session_id().to_string(), fixture.context.session_id);
let source_segment = prepared.worker().segment_id();
let original = fixture.sessions.read_all(prepared.worker().session_id(), source_segment).unwrap();
let generation = fixture.host().catalog.store.active_journal_generation(&adapter.context, &adapter.grant.journal_id).unwrap().unwrap();
assert!(!fixture.host().session_fully_reflected(&fixture.context.session_id).unwrap());
prepared.worker_mut().compact(0).await.unwrap();
assert!(model.updated.load(Ordering::SeqCst));
let worker = prepared.worker();
assert_ne!(worker.segment_id(), source_segment);
let seed = worker.store().read_all(worker.session_id(), worker.segment_id()).unwrap().into_iter().find_map(|entry| match entry { LogEntry::AnnotatedSegmentStart { history, .. } => Some(history), _ => None }).expect("durable compacted seed");
let journal_items: Vec<_> = seed.iter().filter(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::ReferenceInput)).collect();
assert_eq!(journal_items.len(), 1);
let journal_item = Item::from(journal_items[0].item.clone());
let body = journal_item.as_text().unwrap();
assert!(body.contains(&expected_json), "durable seed contains the full selected JSON and processing checkpoint");
assert!(body.contains(BEFORE));
assert!(!body.contains(AFTER));
assert!(seed.iter().any(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::OpeningSummary) && Item::from(entry.item.clone()).as_text().is_some_and(|text| text.contains(COMPLEMENT))));
let requests = model.requests.lock().unwrap();
let input = requests[0].items.iter().find_map(|item| match item {
Item::Message { role: agen::llm_client::types::Role::User, .. } => item.as_text().filter(|text| text.contains("## Conversation overview/index")),
_ => None,
}).expect("actual compactor user request");
assert!(input.ends_with(body), "exact same frozen reference goes to actual compactor and seed");
assert_eq!(input.matches(&expected_json).count(), 1);
assert!(!input.contains(AFTER));
drop(requests);
let latest = fixture.host().journal_compaction_context(&fixture.context).unwrap();
assert_eq!(latest.journals[0].current_position, AFTER);
assert_ne!(latest.journals[0].state_token, expected.journals[0].state_token);
assert_eq!(latest.journals[0].process_order, expected.journals[0].process_order);
assert_eq!(latest.journals[0].process_byte, expected.journals[0].process_byte);
assert_eq!(latest.journals[0].entries, expected.journals[0].entries);
assert_eq!(fixture.host().catalog.store.active_journal_generation(&adapter.context, &adapter.grant.journal_id).unwrap(), Some(generation));
assert_eq!(fixture.host().jobs().unwrap().get(&job.request.job_id).unwrap().state, job.state);
let unchanged: Vec<_> = fixture.sessions.read_all(worker.session_id(), source_segment).unwrap().into_iter().filter(|entry| !matches!(entry, LogEntry::Extension { domain, .. } if domain == "metrics")).collect();
assert_eq!(serde_json::to_value(unchanged).unwrap(), serde_json::to_value(original).unwrap());
assert!(!fixture.host().session_fully_reflected(&fixture.context.session_id).unwrap());
}).await.expect("compaction must finish without recording Job completion");
}
@@ -0,0 +1,228 @@
//! Real standalone Subject/Controller/Journal Job/initial input connection. Only external
//! model streams are scripted. The scenario checks persisted data and ordering, not whether
//! a real model understands the correction or can autonomously continue the work.
use super::*;
use crate::{StandaloneHost, StandaloneLaunchConfig, StandaloneSubjectDisconnectError};
use agen::llm_client::types::Item;
fn launch(root: &Path, state: &Path, subject: &str, name: &str) -> crate::ResolvedStandaloneLaunch {
StandaloneLaunchConfig::new(
root,
state,
manifest::ProfileSelector::source_named(
manifest::ProfileRegistrySource::Builtin,
"standalone-subjektiv",
),
name,
)
.with_subject(subject.into())
.resolve()
.unwrap()
}
#[tokio::test]
async fn recorded_idle_body_handoff_preserves_open_after_source_deletion_and_seeds_next_body_once()
{
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("continuity", Some("BEHAVIOR_ORIGINAL_UNCHANGED")).unwrap();
let model = DialogAndJournal { journal: JournalScript { gate_first: true, ..Default::default() }, ..Default::default() };
let mut first = StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "first"), model.clone()).await.unwrap();
let old_worker = first.worker_id();
let old_session = first.record().active_session_id;
let old_writer = catalog.store.claim_journal_writer(&subject.id, &catalog.scope_id, &old_worker.to_string()).unwrap();
submit_dialog(&first, "A was proposed, user corrected its constraint; B selected because A is unusable. One step performed; tests not performed; external outcome Unknown. Work is unfinished.").await;
model.journal.entered.notified().await;
assert!(matches!(first.disconnect_subject().await, Err(StandaloneSubjectDisconnectError::Incomplete(_))));
assert!(catalog.store.journal_writer_active(&old_writer).unwrap());
assert!(StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "premature"), DialogAndJournal::default()).await.is_err());
let context = JournalHostContext::validated_session(&catalog.store, catalog.store.session_attribution(&old_session.to_string()).unwrap().unwrap()).unwrap();
let id = catalog.store.journal_for_session(&context).unwrap().unwrap();
let generation = catalog.store.active_journal_generation(&context, &id).unwrap().unwrap();
model.journal.release.notify_one();
first.jobs().wait(&generation.job.job_id).await.unwrap();
// Observe the authoritative gate after cleanup; yielding is cooperative scheduling,
// never elapsed time used as proof of recording completion.
loop {
match first.disconnect_subject().await {
Ok(()) => break,
Err(StandaloneSubjectDisconnectError::Incomplete(message)) if message.contains("dispatcher") || message.contains("cleanup") => tokio::task::yield_now().await,
Err(error) => panic!("unexpected handoff blocker: {error}"),
}
}
assert!(!catalog.store.journal_writer_active(&old_writer).unwrap());
assert!(catalog.store.accept_journal_sources(&context, "late old activity", &[generation.selected_ranges[0].clone()]).is_err());
first.shutdown().await.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
workers.delete(old_worker).unwrap();
assert!(!workers.sessions_dir(old_worker).exists());
let reader = JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap();
let historical = catalog.store.journal_startup_snapshot(&reader, Some(&old_session.to_string())).unwrap();
assert_eq!(historical.journals[0].entries[0].state, JournalEntryState::Open);
assert!(historical.journals[0].entries[0].ongoing.contains("unknown"));
assert!(catalog.store.list_memories(&subject.id).unwrap().is_empty());
let second_model = DialogAndJournal::default();
let second = StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "second"), second_model.clone()).await.unwrap();
submit_dialog(&second, "Continue B; describe what is still unperformed and unconfirmed.").await;
let requests = second_model.dialog_requests.lock().unwrap();
let initial = &requests[0];
let context_index = initial.items.iter().position(|item| item.as_text().is_some_and(|text| text.contains("Proposed A; user corrected it to B"))).expect("saved Journal visible to first model request");
let user_index = initial.items.iter().position(|item| matches!(item, Item::Message { role: agen::llm_client::types::Role::User, .. })).unwrap();
assert!(context_index < user_index);
assert_eq!(initial.items.iter().filter(|item| item.as_text().is_some_and(|text| text.contains("Proposed A; user corrected it to B"))).count(), 1);
drop(requests);
let session = second.record().active_session_id;
let sessions = session_store::FsStore::new_with_retention(workers.sessions_dir(second.worker_id()), crate::store::session_retention(&state)).unwrap();
let pointer = session_store::FsWorkerStore::new(workers.worker_metadata_dir(second.worker_id())).unwrap().read_by_name(&second.record().storage_key).unwrap().unwrap().active.unwrap();
let entries = sessions.read_all(session, pointer.segment_id.unwrap()).unwrap();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if entry.metadata.input_metadata.as_ref().and_then(|input| input.feature_payload()).is_some_and(|payload| payload.schema() == "journal:startup"))).count();
assert_eq!(journal_items, 1);
assert_eq!(catalog.store.subject(&subject.id).unwrap().unwrap().behavior_md, "BEHAVIOR_ORIGINAL_UNCHANGED");
second.shutdown().await.unwrap();
}).await.expect("bounded complete Subject handoff connection");
}
#[tokio::test]
async fn saved_public_history_without_admission_notification_blocks_release_until_actual_recording()
{
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("saved-before-notified", None).unwrap();
let model = DialogAndJournal {
journal: JournalScript {
gate_first: true,
..Default::default()
},
..Default::default()
};
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "gap"),
model.clone(),
)
.await
.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
let record = host.record().clone();
let writer = catalog
.store
.claim_journal_writer(
&subject.id,
&catalog.scope_id,
&host.worker_id().to_string(),
)
.unwrap();
let sessions = session_store::FsStore::new_with_retention(
workers.sessions_dir(host.worker_id()),
crate::store::session_retention(&state),
)
.unwrap();
// The production saved-history boundary, with its admission callback deliberately
// omitted. The real Host release must discover these bytes, not trust Job status.
sessions
.append(
record.active_session_id,
record.active_segment_id.unwrap(),
&session_store::LogEntry::AnnotatedUserInput {
ts: 3,
segments: vec![],
extensions: vec![],
history: vec![message(
"missed-admission",
"A corrected to B, partially performed, outcome Unknown",
)],
},
)
.unwrap();
assert!(host.disconnect_subject().await.is_err());
assert!(catalog.store.journal_writer_active(&writer).unwrap());
model.journal.entered.notified().await;
let context = JournalHostContext::validated_session(
&catalog.store,
catalog
.store
.session_attribution(&record.active_session_id.to_string())
.unwrap()
.unwrap(),
)
.unwrap();
let id = catalog
.store
.journal_for_session(&context)
.unwrap()
.unwrap();
let generation = catalog
.store
.active_journal_generation(&context, &id)
.unwrap()
.unwrap();
model.journal.release.notify_one();
host.jobs().wait(&generation.job.job_id).await.unwrap();
loop {
match host.disconnect_subject().await {
Ok(()) => break,
Err(StandaloneSubjectDisconnectError::Incomplete(message))
if message.contains("dispatcher") || message.contains("cleanup") =>
{
tokio::task::yield_now().await
}
Err(error) => {
panic!("unexpected release blocker after saved-history recording: {error}")
}
}
}
assert!(!catalog.store.journal_writer_active(&writer).unwrap());
let snapshot = catalog
.store
.journal_startup_snapshot(
&JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap(),
None,
)
.unwrap();
assert!(!snapshot.journals[0].current_position.is_empty());
assert_eq!(
snapshot.journals[0].entries[0].state,
JournalEntryState::Open
);
host.shutdown().await.unwrap();
})
.await
.expect("saved-history/notification-gap handoff is bounded");
}
#[tokio::test]
async fn shutdown_and_normal_deletion_cannot_release_subject_occupancy() {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("retained-owner", None).unwrap();
let host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "retained"),
DialogAndJournal::default(),
)
.await
.unwrap();
let worker = host.worker_id();
host.shutdown().await.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
assert!(workers.delete(worker).is_err());
assert!(
StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "not-an-owner"),
DialogAndJournal::default()
)
.await
.is_err()
);
let mut restored =
StandaloneHost::restore_with_model_client(state, worker, DialogAndJournal::default())
.await
.unwrap();
restored.disconnect_subject().await.unwrap();
restored.shutdown().await.unwrap();
}
@@ -0,0 +1,596 @@
//! Connected standalone Host -> Subjektiv Memory/Journal -> real FsStore coverage.
//! Candidate and Journal decisions are explicit test inputs; no model, Runtime or live DB
//! is used. The parent Journal fixture owns isolated temporary storage and binding cleanup.
//! Selected by the ordinary standalone lib tests, with no additional feature or service.
use super::*;
use memory::{
backend::MemoryStageCandidateOperation,
extract::{CandidateKind, ExtractedCandidate, StagingEvidence},
schema::{EvidenceKind, EvidenceOriginKind, SourceEvidenceRef, SourceRef},
};
use subjektiv::{
CandidateDecision, CandidateDecisionRequest, MemoryChangeTarget, MemoryDraft, MemoryRecord,
StagingResolution, SubjectStagingRecord,
journal::memory_links::{
JournalTargetAvailability, MemoryJournalLinks, MemoryJournalRequest, MemoryJournalResponse,
MemoryJournalSelector, SourceConnection, UnconnectedReason,
},
};
fn adopt_original(fixture: &Fixture) -> (MemoryRecord, SubjectStagingRecord, StagingResolution) {
let evidence_index = session_store::public_index::read_fs_session_evidence_index(
fixture.sessions.root_dir(),
fixture.context.session_id.parse().unwrap(),
session_store::SessionPublicIndexLimits {
max_bytes: 8 * 1024 * 1024,
max_entries: 20_000,
max_segments: 64,
},
)
.unwrap();
let range = *evidence_index
.ranges
.get(&(fixture.segment.to_string(), "Efirst".into()))
.unwrap()
.first()
.unwrap();
let staged = fixture
.host()
.stage_candidate(
&fixture.context,
MemoryStageCandidateOperation {
source: SourceRef {
segment_id: fixture.segment.to_string(),
range,
},
extract_run_id: "explicit-test-extraction-input".into(),
candidate: ExtractedCandidate {
kind: CandidateKind::Preference,
claim: "Keep correction history".into(),
why_useful: "Preserves the reason for later decisions".into(),
staleness: None,
evidence_ids: vec!["Efirst".into()],
},
evidence: vec![StagingEvidence {
id: "Efirst".into(),
kind: EvidenceKind::new(EvidenceKind::MESSAGE),
entry_range: Some(range),
origin: None,
excerpt: None,
summary: None,
}],
source_refs: vec![SourceEvidenceRef {
session_id: Some(fixture.context.session_id.clone()),
segment_id: Some(fixture.segment.to_string()),
entry_range: Some(range),
evidence_id: Some("Efirst".into()),
..Default::default()
}],
},
)
.unwrap();
let store = &fixture.host().catalog.store;
let subject = &fixture.connection.binding.subject_id;
let candidate = store
.staging_candidate(subject, &staged.staging_id)
.unwrap()
.unwrap();
let receipt = store
.decide_candidate(
subject,
CandidateDecisionRequest {
request_id: "explicit-test-adoption-input".into(),
candidate_id: staged.staging_id,
reason: "Adopt the committed original evidence, not the Journal summary".into(),
decision: CandidateDecision::Apply {
target: MemoryChangeTarget::Create,
draft: MemoryDraft::active(
CandidateKind::Preference,
"Keep correction history",
"Preserve earlier decisions and their corrections",
"Preserves the reason for later decisions",
"Adopt original HumanInput",
),
},
},
)
.unwrap();
(receipt.memory.unwrap(), candidate, receipt.resolution)
}
fn create_entry(adapter: &LocalJournalHost, state: JournalEntryState) -> JournalReceipt {
save(
adapter,
"record-original-context",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "A proposal was corrected".into(),
body: "Proposal A changed to B; only part was performed, outcome unconfirmed.".into(),
ongoing: if state == JournalEntryState::Open {
"Remaining work is not complete".into()
} else {
String::new()
},
state,
source_end: (state == JournalEntryState::Fixed).then(|| {
adapter
.generation
.selected_ranges
.last()
.unwrap()
.source
.pointer
.clone()
}),
correction_refs: vec![],
},
adapter.generation.selected_ranges.clone(),
)
}
fn assert_originals_unchanged(
fixture: &Fixture,
memory: &MemoryRecord,
candidate: &SubjectStagingRecord,
resolution: &StagingResolution,
) {
let catalog = StandaloneSubjects::open_existing(&fixture.host().catalog.state_root).unwrap();
let store = &catalog.store;
let subject = &fixture.connection.binding.subject_id;
assert_eq!(
serde_json::to_value(store.memory(subject, &memory.id).unwrap().unwrap()).unwrap(),
serde_json::to_value(memory).unwrap(),
);
assert_eq!(
serde_json::to_value(
store
.staging_candidate(subject, &candidate.id)
.unwrap()
.unwrap()
)
.unwrap(),
serde_json::to_value(candidate).unwrap(),
);
assert_eq!(
serde_json::to_value(
store
.staging_resolution(subject, &candidate.id)
.unwrap()
.unwrap()
)
.unwrap(),
serde_json::to_value(resolution).unwrap(),
);
assert_eq!(store.list_memories(subject).unwrap().len(), 1);
}
fn selector(memory: &MemoryRecord, candidate: &SubjectStagingRecord) -> MemoryJournalSelector {
MemoryJournalSelector {
memory_id: memory.id.clone(),
change_id: memory.change_id.clone(),
candidate_id: candidate.id.clone(),
}
}
fn links(
fixture: &Fixture,
selector: &MemoryJournalSelector,
reconcile: bool,
) -> MemoryJournalLinks {
let request = if reconcile {
MemoryJournalRequest::Reconcile {
selector: selector.clone(),
}
} else {
MemoryJournalRequest::Links {
selector: selector.clone(),
}
};
let MemoryJournalResponse::Links(links) = fixture
.host()
.memory_journal(&fixture.context, request)
.unwrap()
else {
panic!("expected Memory Journal links")
};
links
}
fn read_request(
selector: &MemoryJournalSelector,
journal: &str,
entry: &str,
) -> MemoryJournalRequest {
MemoryJournalRequest::Read {
selector: selector.clone(),
source_index: 0,
journal_id: journal.into(),
entry_id: entry.into(),
expected_token: None,
byte_offset: 0,
max_bytes: MAX_JOURNAL_READ_BYTES,
}
}
fn assert_link(links: &MemoryJournalLinks, journal: &str, entry: &str, reconsideration: bool) {
assert_eq!(links.sources.len(), 1);
assert_eq!(links.sources[0].source_index, 0);
assert_eq!(
links.sources[0].connection,
SourceConnection::Linked {
journal_id: journal.into(),
entry_id: entry.into(),
target: JournalTargetAvailability::Available,
reconsideration,
},
);
}
#[test]
fn saved_link_reads_after_original_worker_and_session_deletion_from_replacement_host() {
let mut fixture = Fixture::new("Please preserve the reasons for my corrected decisions");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
assert_eq!(
candidate.evidence[0].origin.as_ref().unwrap().kind,
EvidenceOriginKind::HumanInput
);
assert_eq!(
candidate.source_refs[0].origin,
candidate.evidence[0].origin
);
assert!(
candidate.evidence[0]
.excerpt
.as_ref()
.unwrap()
.contains("corrected decisions")
);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Fixed);
let entry = recorded.entry_id.clone().unwrap();
let journal = adapter.grant.journal_id.clone();
let connected = links(&fixture, &selector, true);
assert_link(&connected, &journal, &entry, false);
let request = read_request(&selector, &journal, &entry);
let before = fixture
.host()
.memory_journal(&fixture.context, request.clone())
.unwrap();
let positioned = save(
&adapter,
"position-original",
&recorded.state_token,
JournalAction::UpdatePosition {
current_position: "Recorded correction; remaining outcome unconfirmed".into(),
},
vec![],
);
save(
&adapter,
"checkpoint-original",
&positioned.state_token,
JournalAction::Checkpoint {
record_operation_ids: vec![recorded.request_id],
},
adapter.generation.selected_ranges.clone(),
);
adapter.finish().unwrap();
fixture
.host()
.catalog
.store
.close_journal_generation(
&adapter.context,
&adapter.generation.id,
&adapter.generation.job,
)
.unwrap();
let state = fixture.host().catalog.state_root.clone();
let subject = fixture.connection.binding.subject_id.clone();
let original_session = fixture.context.session_id.clone();
// This Host/storage suite has no Controller. Complete the explicit persisted
// writer-release boundary before deletion; the sibling handoff suite exercises
// the production Idle reservation + Host lifecycle that authorizes that release.
fixture
.host()
.catalog
.store
.release_journal_writer(&fixture.host().writer)
.unwrap();
let (worker, workers) = fixture.stop_worker();
workers.delete(worker).unwrap();
assert!(matches!(
workers.load(worker),
Err(crate::store::StandaloneStoreError::WorkerNotFound(_))
));
assert!(!fixture.sessions.root_dir().join(&original_session).exists());
assert!(
!retained_workers_root(&state)
.join(worker.to_string())
.exists()
);
let replacement_worker = protocol::WorkerId::now_v7();
let mut replacement = StandaloneSubjects::open_existing(&state)
.unwrap()
.connect(&subject, replacement_worker)
.unwrap();
let replacement_context = SubjektivHostContext {
worker_id: replacement_worker.to_string(),
session_id: session_store::SessionId::now_v7().to_string(),
};
let MemoryJournalResponse::Links(saved) = replacement
.host
.memory_journal(
&replacement_context,
MemoryJournalRequest::Links {
selector: selector.clone(),
},
)
.unwrap()
else {
panic!("expected saved links")
};
assert_link(&saved, &journal, &entry, false);
let after = replacement
.host
.memory_journal(&replacement_context, request)
.unwrap();
let MemoryJournalResponse::Read(read) = &after else {
panic!("expected saved Journal text")
};
assert!(
read.page
.as_ref()
.unwrap()
.text
.contains("outcome unconfirmed")
);
assert_eq!(
read.source.original_source,
subjektiv::journal::memory_links::OriginalSourceAvailability::Missing
);
// Reading the saved edge does not depend on a live Session fetch or a replacement Job.
let MemoryJournalResponse::Read(before) = before else {
panic!("expected Journal text")
};
assert_eq!(before.page, read.page);
assert!(
replacement.host.jobs().is_err(),
"the reader has no model/Job executor"
);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
replacement.close().unwrap();
}
#[test]
fn late_journal_fix_connects_existing_memory_without_using_checkpoint_as_event_end() {
let fixture = Fixture::new("Keep the reasons behind corrected decisions");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
assert_eq!(
links(&fixture, &selector, false).sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::NotReconciled
}
);
assert_eq!(
links(&fixture, &selector, true).sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::SourceNotAdmitted
}
);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Open);
let entry = recorded.entry_id.clone().unwrap();
let positioned = save(
&adapter,
"position-open",
&recorded.state_token,
JournalAction::UpdatePosition {
current_position: "Work remains open; no fixed endpoint yet".into(),
},
vec![],
);
// Checkpoint a proper prefix, leaving source bytes for the later Fix.
// A fully checkpointed generation cannot admit another activity write.
let mut reflected = adapter.generation.selected_ranges.clone();
reflected.last_mut().unwrap().bytes.end -= 1;
let checkpoint = save(
&adapter,
"checkpoint-open",
&positioned.state_token,
JournalAction::Checkpoint {
record_operation_ids: vec![recorded.request_id],
},
reflected.clone(),
);
let unconnected = links(&fixture, &selector, true);
assert_eq!(
unconnected.sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::JournalNotFixed
}
);
assert_eq!(links(&fixture, &selector, true), unconnected);
let state = adapter.state().unwrap();
let open = state.current_open.unwrap();
assert_eq!(open.id, entry);
assert_eq!(open.state, JournalEntryState::Open);
assert!(open.source_end.is_none());
let last = reflected.last().unwrap();
assert_eq!(state.current_journal.process_order, last.source.order);
assert_eq!(state.current_journal.process_byte, last.bytes.end);
save(
&adapter,
"fix-later",
&checkpoint.state_token,
JournalAction::Fix {
entry_id: entry.clone(),
source_end: adapter
.generation
.selected_ranges
.last()
.unwrap()
.source
.pointer
.clone(),
},
adapter.generation.selected_ranges.clone(),
);
// A plain saved-state read must not silently perform reconciliation.
assert_eq!(links(&fixture, &selector, false), unconnected);
let connected = links(&fixture, &selector, true);
assert_link(&connected, &adapter.grant.journal_id, &entry, false);
assert_eq!(links(&fixture, &selector, true), connected);
assert_eq!(links(&fixture, &selector, false), connected);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
#[test]
fn host_rejects_foreign_subject_worker_context_and_closed_connection_for_all_link_operations() {
let mut fixture = Fixture::new("Keep correction history");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Fixed);
let journal = adapter.grant.journal_id.clone();
let entry = recorded.entry_id.unwrap();
assert_link(&links(&fixture, &selector, true), &journal, &entry, false);
let requests = [
MemoryJournalRequest::Reconcile {
selector: selector.clone(),
},
MemoryJournalRequest::Links {
selector: selector.clone(),
},
read_request(&selector, &journal, &entry),
MemoryJournalRequest::Related {
journal_id: journal,
entry_id: entry,
after: None,
limit: 32,
},
];
let catalog = StandaloneSubjects::open_existing(&fixture.host().catalog.state_root).unwrap();
let foreign_subject = catalog.create("different-reader", None).unwrap();
let foreign_worker = protocol::WorkerId::now_v7();
let mut foreign = catalog
.connect(&foreign_subject.id, foreign_worker)
.unwrap();
let foreign_context = SubjektivHostContext {
worker_id: foreign_worker.to_string(),
session_id: session_store::SessionId::now_v7().to_string(),
};
let wrong_worker = SubjektivHostContext {
worker_id: foreign_worker.to_string(),
..fixture.context.clone()
};
for request in &requests {
assert!(
foreign
.host
.memory_journal(&foreign_context, request.clone())
.is_err()
);
assert!(
fixture
.host()
.memory_journal(&wrong_worker, request.clone())
.is_err()
);
}
foreign.close().unwrap();
fixture.connection.close().unwrap();
for request in requests {
assert!(
fixture
.host()
.memory_journal(&fixture.context, request)
.is_err()
);
}
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
#[test]
fn journal_correction_finds_related_memory_without_revising_or_retargeting_its_evidence() {
let fixture = Fixture::new("Keep correction history");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
fixture.append("correction", "The earlier result was not confirmed");
let (adapter, _, _) = fixture.adapter();
let ranges = &adapter.generation.selected_ranges;
assert_eq!(ranges.len(), 2);
let original = save(
&adapter,
"original-event",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "Original context".into(),
body: "A change was proposed; execution remained uncertain".into(),
ongoing: String::new(),
state: JournalEntryState::Fixed,
source_end: Some(ranges[0].source.pointer.clone()),
correction_refs: vec![],
},
vec![ranges[0].clone()],
);
let entry = original.entry_id.unwrap();
let journal = &adapter.grant.journal_id;
assert_link(&links(&fixture, &selector, true), journal, &entry, false);
let corrected = save(
&adapter,
"correction-event",
&original.state_token,
JournalAction::Create {
title: "Correction".into(),
body: "The earlier result was unconfirmed; reconsider its associated Memory".into(),
ongoing: String::new(),
state: JournalEntryState::Fixed,
source_end: Some(ranges[1].source.pointer.clone()),
correction_refs: vec![entry.clone()],
},
vec![ranges[1].clone()],
);
assert_link(&links(&fixture, &selector, false), journal, &entry, true);
let correction_entry = corrected.entry_id.unwrap();
for target in [&entry, &correction_entry] {
let MemoryJournalResponse::Related(related) = fixture
.host()
.memory_journal(
&fixture.context,
MemoryJournalRequest::Related {
journal_id: journal.clone(),
entry_id: target.clone(),
after: None,
limit: 32,
},
)
.unwrap()
else {
panic!("expected related Memory")
};
assert_eq!(related.items.len(), 1);
assert_eq!(related.items[0].selector, selector);
assert_eq!(related.items[0].source_index, 0);
assert!(related.items[0].reconsideration);
assert!(related.next.is_none());
}
let MemoryJournalResponse::Read(read) = fixture
.host()
.memory_journal(&fixture.context, read_request(&selector, journal, &entry))
.unwrap()
else {
panic!("expected original context")
};
assert!(
read.page
.unwrap()
.text
.contains("execution remained uncertain")
);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
+12 -6
View File
@@ -37,9 +37,8 @@ fn subject_kernel_lease_is_not_worker_or_profile_scoped() {
let catalog = StandaloneSubjects::open(temp.path().join("state")).unwrap();
let a = catalog.create("a", None).unwrap();
let b = catalog.create("b", None).unwrap();
let mut first = catalog
.connect(&a.id, protocol::WorkerId::now_v7())
.unwrap();
let first_id = protocol::WorkerId::now_v7();
let mut first = catalog.connect(&a.id, first_id).unwrap();
assert!(matches!(
StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
@@ -51,11 +50,18 @@ fn subject_kernel_lease_is_not_worker_or_profile_scoped() {
.connect(&b.id, protocol::WorkerId::now_v7())
.unwrap();
first.close().unwrap();
let mut replacement = StandaloneSubjects::open_existing(&temp.path().join("state"))
assert!(
StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&a.id, protocol::WorkerId::now_v7())
.is_err(),
"kernel lease release is not durable body release"
);
let mut resumed = StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&a.id, protocol::WorkerId::now_v7())
.connect(&a.id, first_id)
.unwrap();
replacement.close().unwrap();
resumed.close().unwrap();
second.close().unwrap();
}
+106 -3
View File
@@ -155,6 +155,81 @@ pub(crate) fn authorized_index(
)
}
pub(crate) fn authorized_sessions_path(
state_root: &Path,
scope_id: &str,
subject_id: &str,
store: &SubjektivStore,
session_id: &str,
) -> Result<PathBuf, SubjektivHostError> {
let backend = Backend {
state_root,
scope_id,
subject_id,
store,
reader: session_store::public_index::read_fs_session_public_index,
};
(|| {
backend.check_scope()?;
required_selector(session_id)?;
let attribution = backend.attribution(session_id)?;
Ok(backend.authorize(attribution)?.sessions_path)
})()
.map_err(|error: Failure| SubjektivHostError::Unavailable(error.message))
}
/// Checks only saved ownership and filesystem metadata, never Session bytes or a Worker restore.
/// Inaccessible/corrupt storage stays unknown rather than being reported as absent.
pub(crate) fn original_session_presence(
state_root: &Path,
scope_id: &str,
subject_id: &str,
store: &SubjektivStore,
session_id: &str,
) -> Option<bool> {
let attribution = store.session_attribution(session_id).ok()??;
if store.scope_id() != scope_id
|| attribution.subject_id != subject_id
|| attribution.runtime_id != scope_id
{
return None;
}
let worker: WorkerId = attribution.worker_id.parse().ok()?;
let session: SessionId = session_id.parse().ok()?;
let active = StandaloneWorkerStore::open_existing(state_root).ok()?;
let (workers, record) = match active.load(worker) {
Ok(record) => (active, record),
Err(StandaloneStoreError::WorkerNotFound(_)) => {
let root = state_root.join("subjektiv").join("retained-workers");
match std::fs::metadata(&root) {
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Some(false),
Err(_) => return None,
Ok(_) => {}
}
let retained = StandaloneWorkerStore::open_existing(root).ok()?;
match retained.load(worker) {
Ok(record) => (retained, record),
Err(StandaloneStoreError::WorkerNotFound(_)) => return Some(false),
Err(_) => return None,
}
}
Err(_) => return None,
};
if record.worker_id != worker
|| !record
.subject
.as_ref()
.is_some_and(|binding| binding.scope_id == scope_id && binding.subject_id == subject_id)
{
return None;
}
match std::fs::metadata(workers.sessions_dir(worker).join(session.to_string())) {
Ok(metadata) if metadata.is_dir() => Some(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Some(false),
_ => None,
}
}
fn authorized_index_with_reader(
state_root: &Path,
scope_id: &str,
@@ -221,13 +296,28 @@ impl Backend<'_> {
"Attributed standalone Worker state is missing",
));
}
let workers = StandaloneWorkerStore::open_existing(self.state_root).map_err(|_| {
let mut workers = StandaloneWorkerStore::open_existing(self.state_root).map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RuntimeUnavailable,
"Standalone Worker storage is inaccessible",
)
})?;
let record = workers.load(id).map_err(|error| match error {
let record = match workers.load(id) {
Err(StandaloneStoreError::WorkerNotFound(_)) => {
workers = StandaloneWorkerStore::open_existing(
self.state_root.join("subjektiv").join("retained-workers"),
)
.map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Original Session is not retained",
)
})?;
workers.load(id)
}
result => result,
}
.map_err(|error| match error {
StandaloneStoreError::WorkerNotFound(_) => failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Attributed standalone Worker record is missing",
@@ -262,6 +352,13 @@ impl Backend<'_> {
"Attributed Session identity is invalid",
)
})?;
let retention = crate::store::session_retention(self.state_root);
let _guard = retention.lock_live(id).map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Original Session is erased or its retention fence is unavailable",
)
})?;
let index = (self.reader)(&source.sessions_path, id, limits).map_err(index_failure)?;
if index.session_id != source.attribution.session_id {
return Err(failure(
@@ -1524,6 +1621,7 @@ allow = []
let history = |id: &str, item: LoggedItem| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction { operation_id: None },
derivation: None,
@@ -1539,7 +1637,11 @@ allow = []
)
};
let sessions = f.index_path.parent().unwrap();
let fs_store = session_store::FsStore::new(sessions).unwrap();
let fs_store = session_store::FsStore::new_with_retention(
sessions,
crate::store::session_retention(&f.temp.path().join("workers")),
)
.unwrap();
let segment: SessionId = f.segment.parse().unwrap();
fs_store
.create_segment(
@@ -1592,6 +1694,7 @@ allow = []
// A valid JSON record without its final newline is not committed.
log.write_all(
&serde_json::to_vec(&LogEntry::RunCompleted {
entry_id: None,
ts: 4,
interrupted: false,
result: agen::EngineResult::Finished,
+296
View File
@@ -0,0 +1,296 @@
//! Optional operator transport owned by the Standalone Host, never by a Worker.
//!
//! The endpoint's existing parent must be private to the current Unix user. Binding never
//! removes an existing entry, including stale sockets. Recovery is an explicit operator action.
//! Peers must have that same uid; admitted methods still cross the normal local protocol
//! authorization boundary. Disconnect only drops a view, not the Host or its execution lease.
use std::io;
use std::os::unix::fs::{FileTypeExt, MetadataExt, PermissionsExt};
use std::path::{Path, PathBuf};
use client::transport::Socket;
use protocol::stream::{JsonLineWriter, decode_event, decode_method};
use tokio::io::{AsyncBufReadExt, BufReader};
use tokio::net::{UnixListener, UnixStream};
use tokio::sync::oneshot;
use tokio::task::{JoinHandle, JoinSet};
/// A bound endpoint reservation. Drop removes only the exact socket created by this bind.
/// It can be reserved before starting a Host, so bind failures do not allocate Workers.
pub struct UnixEndpoint {
listener: UnixListener,
path: PathBuf,
device: u64,
inode: u64,
}
impl UnixEndpoint {
pub fn bind(path: impl AsRef<Path>) -> io::Result<Self> {
let path = path.as_ref();
let name = path.file_name().ok_or_else(|| {
io::Error::new(io::ErrorKind::InvalidInput, "endpoint requires a filename")
})?;
let parent = std::fs::canonicalize(
path.parent()
.filter(|p| !p.as_os_str().is_empty())
.unwrap_or(Path::new(".")),
)?;
let metadata = std::fs::metadata(&parent)?;
// SAFETY: geteuid has no preconditions and does not mutate process identity.
let uid = unsafe { libc::geteuid() };
if !metadata.is_dir() || metadata.uid() != uid || metadata.mode() & 0o077 != 0 {
return Err(io::Error::new(
io::ErrorKind::PermissionDenied,
"endpoint parent must be a current-user-owned private directory (mode 0700)",
));
}
let path = parent.join(name);
let listener = UnixListener::bind(&path)?;
let metadata = std::fs::symlink_metadata(&path)?;
let endpoint = Self {
listener,
path,
device: metadata.dev(),
inode: metadata.ino(),
};
std::fs::set_permissions(&endpoint.path, std::fs::Permissions::from_mode(0o600))?;
Ok(endpoint)
}
pub fn path(&self) -> &Path {
&self.path
}
}
impl Drop for UnixEndpoint {
fn drop(&mut self) {
if let Ok(current) = std::fs::symlink_metadata(&self.path)
&& current.file_type().is_socket()
&& current.dev() == self.device
&& current.ino() == self.inode
{
let _ = std::fs::remove_file(&self.path);
}
}
}
pub(crate) struct UnixAdapter {
task: Option<JoinHandle<io::Result<()>>>,
stop: Option<oneshot::Sender<()>>,
}
impl UnixAdapter {
pub(crate) fn start(endpoint: UnixEndpoint, handle: worker::WorkerHandle) -> Self {
let (stop, mut stopped) = oneshot::channel();
let task = tokio::spawn(async move {
let mut connections = JoinSet::new();
let result = loop {
tokio::select! {
_ = &mut stopped => break Ok(()),
accepted = endpoint.listener.accept(), if connections.len() < 64 => {
match accepted {
Ok((stream, _)) => {
// The filesystem gate protects connection establishment; credentials
// also fence open connections to the Host's operator identity.
let allowed = stream.peer_cred().is_ok_and(|cred| {
// SAFETY: geteuid has no preconditions.
cred.uid() == unsafe { libc::geteuid() }
});
if allowed {
let socket = crate::host::connect_protocol(handle.clone());
connections.spawn(bridge(stream, socket));
}
}
Err(error) => break Err(error),
}
}
_ = connections.join_next(), if !connections.is_empty() => {}
}
};
connections.shutdown().await;
// Keep the bound endpoint alive until every connection has been reclaimed.
drop(endpoint);
result
});
Self {
task: Some(task),
stop: Some(stop),
}
}
pub(crate) async fn ended(&mut self) -> io::Result<()> {
let result = self.task.as_mut().expect("adapter task").await;
self.task.take();
result.map_err(io::Error::other)?
}
pub(crate) async fn shutdown(mut self) -> io::Result<()> {
if let Some(stop) = self.stop.take() {
let _ = stop.send(());
}
if self.task.is_some() {
self.ended().await?;
}
Ok(())
}
}
impl Drop for UnixAdapter {
fn drop(&mut self) {
if let Some(stop) = self.stop.take() {
let _ = stop.send(());
}
// Sending stop lets the task join its connection set and drop its endpoint.
// A Host dropped without confirmed shutdown keeps the production execution lease.
}
}
async fn bridge(
stream: UnixStream,
mut socket: client::transport::in_process::Socket,
) -> io::Result<()> {
let (reader, writer) = stream.into_split();
let mut reader = BufReader::new(reader).lines();
let mut writer = JsonLineWriter::new(writer);
loop {
tokio::select! {
line = reader.next_line() => {
let Some(message) = line? else { return Ok(()); };
if message.trim().is_empty() { continue; }
socket.send(message).await.map_err(io::Error::other)?;
}
event = socket.next() => {
let Some(message) = event.map_err(io::Error::other)? else { return Ok(()); };
let event = decode_event(&message).map_err(io::Error::other)?;
writer.write_event(&event).await?;
}
}
}
}
/// Connect a JSONL operator client. EOF disconnects without issuing Shutdown.
/// Input is decoded as Worker Methods and output contains Worker Events only.
pub async fn run_unix_client(endpoint: PathBuf) -> io::Result<()> {
relay_unix_client(endpoint, tokio::io::stdin(), tokio::io::stdout()).await
}
/// The CLI's stream relay, also usable by embedders that own their input/output streams.
pub async fn relay_unix_client<R, W>(endpoint: PathBuf, input: R, output: W) -> io::Result<()>
where
R: tokio::io::AsyncRead + Unpin,
W: tokio::io::AsyncWrite + Unpin,
{
let stream = UnixStream::connect(endpoint).await?;
let (reader, writer) = stream.into_split();
let mut input = BufReader::new(input).lines();
let mut to_host = JsonLineWriter::new(writer);
let mut from_host = BufReader::new(reader).lines();
let mut output = JsonLineWriter::new(output);
loop {
tokio::select! {
line = input.next_line() => {
let Some(line) = line? else { return Ok(()); };
if line.trim().is_empty() { continue; }
let method = decode_method(&line).map_err(io::Error::other)?;
to_host.write(&method).await?;
}
line = from_host.next_line() => {
let Some(line) = line? else { return Ok(()); };
let event = decode_event(&line).map_err(io::Error::other)?;
output.write_event(&event).await?;
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn endpoint_requires_private_parent_and_is_owner_only() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("host.sock");
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o755)).unwrap();
let error = UnixEndpoint::bind(&path)
.err()
.expect("public parent rejected");
assert_eq!(error.kind(), io::ErrorKind::PermissionDenied);
assert!(!path.exists());
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let endpoint = UnixEndpoint::bind(&path).unwrap();
assert_eq!(
std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
0o600
);
let client = UnixStream::connect(&path).await.unwrap();
let (server, _) = endpoint.listener.accept().await.unwrap();
assert_eq!(
client.peer_cred().unwrap().uid(),
server.peer_cred().unwrap().uid()
);
drop(endpoint);
assert!(!path.exists());
}
#[tokio::test]
async fn bind_failure_precedes_worker_state_and_startup_failure_releases_only_its_endpoint() {
let directory = tempfile::tempdir().unwrap();
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let path = directory.path().join("host.sock");
let owner = UnixEndpoint::bind(&path).unwrap();
let state_dir = directory.path().join("must-not-be-opened");
let error = crate::run_headless(
crate::HeadlessLaunch::Restore {
state_dir: state_dir.clone(),
worker_id: protocol::WorkerId::now_v7(),
},
path.clone(),
)
.await
.unwrap_err();
assert!(matches!(error, crate::HeadlessError::Transport(_)));
assert!(!state_dir.exists());
assert!(path.exists());
drop(owner);
let error = crate::run_headless(
crate::HeadlessLaunch::Restore {
state_dir,
worker_id: protocol::WorkerId::now_v7(),
},
path.clone(),
)
.await
.unwrap_err();
assert!(matches!(error, crate::HeadlessError::Startup(_)));
assert!(
!path.exists(),
"a failed Host startup must release its own bind"
);
}
#[tokio::test]
async fn bind_failure_and_old_endpoint_drop_preserve_another_owner() {
let directory = tempfile::tempdir().unwrap();
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let path = directory.path().join("host.sock");
let first = UnixEndpoint::bind(&path).unwrap();
let first_inode = std::fs::metadata(&path).unwrap().ino();
assert_eq!(
UnixEndpoint::bind(&path).err().unwrap().kind(),
io::ErrorKind::AddrInUse
);
assert_eq!(std::fs::metadata(&path).unwrap().ino(), first_inode);
std::fs::rename(&path, directory.path().join("retained.sock")).unwrap();
let second = UnixEndpoint::bind(&path).unwrap();
let second_inode = std::fs::metadata(&path).unwrap().ino();
assert_ne!(first_inode, second_inode);
drop(first);
assert_eq!(std::fs::metadata(&path).unwrap().ino(), second_inode);
let _client = UnixStream::connect(&path).await.unwrap();
let _connection = second.listener.accept().await.unwrap();
drop(second);
assert!(!path.exists());
}
}
+229
View File
@@ -608,6 +608,235 @@ async fn standalone_metadata_fails_closed_on_incomplete_or_newer_records() -> Te
Ok(())
}
#[tokio::test]
async fn headless_unix_client_uses_production_history_lease_and_local_authority() -> TestResult {
use protocol::stream::{JsonLineReader, encode_method};
use std::os::unix::fs::PermissionsExt;
use tokio::io::AsyncWriteExt;
let temp = tempfile::tempdir()?;
// Only the transport endpoint needs a short private pathname. Persistence remains the
// same production store exercised by the in-process Host tests above.
std::fs::set_permissions(temp.path(), std::fs::Permissions::from_mode(0o700))?;
let endpoint_path = temp.path().join("operator.sock");
let endpoint = standalone::UnixEndpoint::bind(&endpoint_path)?;
let state_dir = temp.path().join("state");
let launch = StandaloneLaunchConfig::new(
temp.path(),
&state_dir,
manifest::ProfileSelector::Default,
"headless-operator",
)
.resolve()?;
let model = ScriptedClient::new(vec![vec![
LlmEvent::text_block_start(0),
LlmEvent::text_delta(0, "headless production response"),
LlmEvent::text_block_stop(0, Some(StopReason::EndTurn)),
]]);
let inspection = model.clone();
let host = StandaloneHost::start_with_model_client(launch, model).await?;
let worker_id = host.worker_id();
let session_id = host.record().active_session_id;
let profile = host.record().manifest.clone();
let serve = tokio::spawn(host.serve_unix(endpoint));
// Exercise the real external client's stdin/stdout relay over the actual Unix adapter.
let (mut input, relay_input) = tokio::io::duplex(64 * 1024);
let (relay_output, output) = tokio::io::duplex(64 * 1024);
let relay = tokio::spawn(standalone::relay_unix_client(
endpoint_path.clone(),
relay_input,
relay_output,
));
let mut events = JsonLineReader::new(output);
let snapshot = tokio::time::timeout(Duration::from_secs(10), events.next::<Event>())
.await??
.expect("snapshot");
assert!(matches!(snapshot, Event::Snapshot { .. }));
let request = Method::SubmitTracked {
submission_request_id: protocol::new_submission_request_id(),
input: vec![protocol::Segment::text("headless production input")],
source: protocol::AuthenticatedInputSource::Backend {
operation_id: "forged-backend-source".into(),
},
};
let wire = encode_method(&request)?;
println!("headless input: {wire}");
input.write_all(wire.as_bytes()).await?;
input.write_all(b"\n").await?;
let mut saw_user = false;
let mut saw_response = false;
let mut saw_state = false;
tokio::time::timeout(Duration::from_secs(20), async {
loop {
let event = events.next::<Event>().await?.expect("live event");
match &event {
Event::UserMessage { segments, .. } => {
saw_user |= format!("{segments:?}").contains("headless production input");
}
Event::TextDelta { text } => {
saw_response |= text == "headless production response";
}
Event::WorkerState { .. } => saw_state = true,
Event::RunEnd { .. } => break,
_ => {}
}
println!(
"headless output: {}",
protocol::stream::encode_event(&event)?
);
}
Ok::<_, Box<dyn std::error::Error>>(())
})
.await??;
assert!(saw_user && saw_response && saw_state);
assert_eq!(inspection.requests().len(), 1);
assert!(matches!(
StandaloneHost::restore_with_model_client(
state_dir.clone(),
worker_id,
ScriptedClient::new(vec![])
)
.await,
Err(StandaloneStartupError::WorkerActive)
));
// EOF is a disconnect, not a shutdown; another view can reconnect to the same history.
drop(input);
tokio::time::timeout(Duration::from_secs(10), relay).await???;
assert!(!serve.is_finished());
let mut malformed = tokio::net::UnixStream::connect(&endpoint_path).await?;
malformed.write_all(b"not-json\n").await?;
let mut discarded = Vec::new();
use tokio::io::AsyncReadExt;
tokio::time::timeout(
Duration::from_secs(10),
malformed.read_to_end(&mut discarded),
)
.await??;
assert!(
!serve.is_finished(),
"invalid client input must not stop the Host"
);
let mut socket_client =
Client::new(client::transport::unix_socket::Socket::connect(&endpoint_path).await?);
let snapshot = tokio::time::timeout(Duration::from_secs(10), async {
loop {
let event = socket_client.next_event().await?.expect("reconnect event");
if matches!(event, Event::Snapshot { .. }) {
break Ok::<_, Box<dyn std::error::Error>>(event);
}
}
})
.await??;
let snapshot_text = format!("{snapshot:?}");
assert!(
snapshot_text.contains("headless production input"),
"{snapshot_text}"
);
assert!(
snapshot_text.contains("headless production response"),
"{snapshot_text}"
);
socket_client
.send(&Method::Shutdown {
command: protocol::WorkerCommandEnvelope::new(0),
})
.await?;
tokio::time::timeout(Duration::from_secs(10), async {
loop {
if let Some(Event::CommandAcknowledged { acknowledgement }) =
socket_client.next_event().await?
{
assert_eq!(
acknowledgement.disposition,
protocol::WorkerCommandDisposition::StaleCommandId
);
break;
}
}
Ok::<_, Box<dyn std::error::Error>>(())
})
.await??;
assert!(
!serve.is_finished(),
"stale Shutdown must not bypass Controller admission"
);
socket_client
.send(&Method::Shutdown {
command: protocol::WorkerCommandEnvelope::new(100),
})
.await?;
tokio::time::timeout(Duration::from_secs(15), serve).await???;
assert!(!endpoint_path.exists());
let store = StandaloneWorkerStore::open(&state_dir)?;
let saved = store.load(worker_id)?;
assert_eq!(saved.status, StandaloneWorkerStatus::Stopped);
use session_store::Store;
let history = session_store::FsStore::new(store.sessions_dir(worker_id))?.read_all(
saved.active_session_id,
saved.active_segment_id.expect("active segment"),
)?;
let origins = history
.iter()
.filter_map(|entry| match entry {
session_store::LogEntry::AnnotatedUserInput { history, .. } => Some(history),
_ => None,
})
.flatten()
.map(|entry| &entry.metadata.origin)
.collect::<Vec<_>>();
assert!(origins.contains(&&session_store::LoggedSessionHistoryOrigin::LocalHumanInput));
assert!(!format!("{origins:?}").contains("forged-backend-source"));
// This is the same restoration path called by the TUI, not a headless catalog/import.
let restored = StandaloneHost::restore_with_model_client(
state_dir.clone(),
worker_id,
ScriptedClient::new(vec![]),
)
.await?;
assert_eq!(restored.worker_id(), worker_id);
assert_eq!(restored.record().active_session_id, session_id);
assert_eq!(
serde_json::to_value(&restored.record().manifest)?,
serde_json::to_value(&profile)?
);
assert_eq!(
restored.record().cwd.canonical_path,
temp.path().canonicalize()?
);
let mut in_process = restored.connect();
let snapshot = tokio::time::timeout(Duration::from_secs(10), async {
loop {
let event = in_process
.next_event()
.await?
.expect("in-process restored event");
if matches!(event, Event::Snapshot { .. }) {
break Ok::<_, Box<dyn std::error::Error>>(event);
}
}
})
.await??;
assert!(format!("{snapshot:?}").contains("headless production response"));
assert!(
!endpoint_path.exists(),
"in-process restore must not recreate the adapter"
);
let runtime = state_dir
.join(worker_id.to_string())
.join("runtime")
.join(format!("standalone-{worker_id}"));
assert!(!runtime.join("sock").exists());
assert!(!runtime.join("pid").exists());
restored.shutdown().await?;
println!(
"verdict: input/state/response/disconnect/explicit shutdown, shared production restore and concurrent lease exclusion passed for {worker_id}"
);
Ok(())
}
async fn wait_for_run_end(client: &mut Client<InProcessSocket>) -> TestResult {
tokio::time::timeout(Duration::from_secs(10), async {
loop {
+151 -6
View File
@@ -1,5 +1,7 @@
//! T-709: real standalone Hosts and test-owned processes, with scripted models.
//! These tests do not create a Workspace or contact a model provider.
//! Integration of real standalone Subject lifecycle, dialog, Journal and Memory services.
//! Only model streams are scripted; temporary storage and test-owned subprocesses isolate
//! persistence and kernel leases. Run with `cargo test -p standalone --test subjektiv`.
//! No Workspace or live model provider is used; this is not a continuity meaning evaluation.
use std::collections::{BTreeMap, VecDeque};
use std::io::{Read, Write};
@@ -354,6 +356,17 @@ impl LlmClient for ScriptedClient {
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<LlmEvent, ClientError>> + Send>>, ClientError>
{
// Journal is a separate scripted model boundary. Successful production recording
// is required before these Memory scenarios may hand the Subject to a new body.
if request
.tools
.iter()
.any(|tool| tool.name == "ReadJournalState")
{
return Ok(Box::pin(stream::iter(
journal_response(&request).into_iter().map(Ok),
)));
}
// Lifecycle-triggered consolidation uses a separate Internal Worker. It
// must not consume the dialog script or silently become a live request.
let job = request
@@ -411,6 +424,134 @@ impl LlmClient for ScriptedClient {
}
}
// The outer model script is not a continuity meaning evaluation. Persistence, complete
// source delivery and the handoff gate remain the production implementations.
fn journal_response(request: &Request) -> Vec<LlmEvent> {
let result = |id: &str| -> Option<Value> {
request.items.iter().find_map(|item| match item {
Item::ToolResult {
call_id,
content,
is_error,
summary,
..
} if call_id == id => {
assert!(!is_error, "{id}: {summary}; {content:?}");
Some(
content
.as_deref()
.map(|v| serde_json::from_str(v).unwrap())
.unwrap_or(Value::Null),
)
}
_ => None,
})
};
if result("journal-submit").is_some() {
return prose();
}
if result("journal-checkpoint").is_some() {
return tool("journal-submit", "SubmitJobResult", json!({"result":{}}));
}
let Some(state) = result("journal-state") else {
return tool("journal-state", "ReadJournalState", json!({}));
};
let state = &state["value"];
let generation = state["generation_id"].as_str().unwrap();
if let Some(position) = result("journal-position") {
return tool(
"journal-checkpoint",
"CheckpointJournal",
json!({
"request_id":format!("checkpoint:{generation}"), "expected_state":position["value"]["state_token"],
"action":{"action":"checkpoint","record_operation_ids":[format!("record:{generation}")]},
"covered_ranges":state["selected_ranges"]
}),
);
}
if let Some(record) = result("journal-record") {
return tool(
"journal-position",
"UpdateJournalPosition",
json!({
"request_id":format!("position:{generation}"), "expected_state":record["value"]["state_token"],
"action":{"action":"update_position","current_position":"Memory scenario recorded; represented work may remain unfinished"}, "covered_ranges":[]
}),
);
}
for (index, range) in state["selected_ranges"]
.as_array()
.unwrap()
.iter()
.enumerate()
{
let mut offset = range["bytes"]["start"].as_u64().unwrap();
let mut token = Value::Null;
loop {
let id = format!("journal-source-{index}-{offset}");
match result(&id) {
Some(page) => match page["value"]["next_byte"].as_u64() {
Some(next) => {
assert!(next > offset);
offset = next;
token = page["value"]["content_token"].clone();
}
None => break,
},
None => {
return tool(
&id,
"ReadJournalText",
json!({
"target":{"target":"source","pointer":range["source"]["pointer"]},
"offset":offset,"limit":subjektiv::journal::MAX_JOURNAL_READ_BYTES,"content_token":token
}),
);
}
}
}
}
let (name, action) = if state["current_open"].is_null() {
(
"CreateJournalEntry",
json!({"action":"create","title":"Memory conversation","body":"Conversation recorded by the scripted model fixture","ongoing":"Unfinished work is not declared complete","state":"open","source_end":null,"correction_refs":[]}),
)
} else {
(
"RewriteJournalEntry",
json!({"action":"rewrite","entry_id":state["current_open"]["id"],"title":"Memory conversation","body":"Further conversation recorded by the scripted model fixture","ongoing":"Unfinished work is not declared complete"}),
)
};
tool(
"journal-record",
name,
json!({
"request_id":format!("record:{generation}"), "expected_state":state["current_journal"]["state_token"],
"action":action,"covered_ranges":state["selected_ranges"]
}),
)
}
async fn disconnect_recorded(host: &mut StandaloneHost) {
let mut last_error = None;
let completed = tokio::time::timeout(Duration::from_secs(30), async {
loop {
match host.disconnect_subject().await {
Ok(()) => return,
Err(error) => {
last_error = Some(error.to_string());
tokio::task::yield_now().await;
}
}
}
})
.await;
assert!(
completed.is_ok(),
"actual Idle/Journal completion is required before Memory scenario handoff: {last_error:?}"
);
}
fn prose() -> Vec<LlmEvent> {
vec![
LlmEvent::text_block_start(0),
@@ -750,7 +891,7 @@ async fn explicit_remember_is_committed_staging_and_sessions_follow_subject_acro
tool("query-restored", "SubjektivMemoryQuery", json!({})),
prose(),
]);
let restored =
let mut restored =
StandaloneHost::restore_with_model_client(state.clone(), worker_id, restored_model.clone())
.await
.unwrap();
@@ -761,6 +902,7 @@ async fn explicit_remember_is_committed_staging_and_sessions_follow_subject_acro
assert!(results["recall"].to_string().contains(claim));
assert_eq!(results["query-restored"]["items"], json!([]));
assert!(format!("{:?}", restored_model.requests()[0].items).contains("Remember this lesson"));
disconnect_recorded(&mut restored).await;
restored.shutdown().await.unwrap();
let second_model = ScriptedClient::new(vec![
@@ -980,7 +1122,7 @@ async fn kernel_subject_lease_excludes_other_processes_and_recovers_after_abnorm
let independent = catalog.create("independent", None).unwrap();
drop(catalog);
let ready = temp.path().join("ready");
let parent = StandaloneHost::start_with_model_client(
let mut parent = StandaloneHost::start_with_model_client(
launch(&parent_cwd, &state, &subject.id, "parent-owner"),
ScriptedClient::new(Vec::new()),
)
@@ -1002,6 +1144,7 @@ async fn kernel_subject_lease_excludes_other_processes_and_recovers_after_abnorm
.await
.unwrap();
independent_host.shutdown().await.unwrap();
disconnect_recorded(&mut parent).await;
parent.shutdown().await.unwrap();
let mut owner = OwnedChild(
@@ -1070,7 +1213,7 @@ async fn automatic_generic_consolidation_applies_candidate_publishes_surface_and
drop(catalog);
let claim = "Validate the immutable result before reporting success";
let model = ScriptedClient::consolidating(remember_script(claim), false);
let host = StandaloneHost::start_with_model_client(
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "consolidation-body"),
model.clone(),
)
@@ -1145,6 +1288,7 @@ async fn automatic_generic_consolidation_applies_candidate_publishes_surface_and
.exists(),
"selected Subject owns the shared generic Job ledger"
);
disconnect_recorded(&mut host).await;
host.shutdown().await.unwrap();
let memory = inspect_store(&state, |store| {
assert!(
@@ -1224,7 +1368,7 @@ async fn transport_loss_after_candidate_apply_preserves_memory_and_later_job_onl
drop(catalog);
let claim = "Candidate receipts survive transport failure";
let losing = ScriptedClient::consolidating(remember_script(claim), true);
let host = StandaloneHost::start_with_model_client(
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "losing-body"),
losing.clone(),
)
@@ -1249,6 +1393,7 @@ async fn transport_loss_after_candidate_apply_preserves_memory_and_later_job_onl
failed.result.is_none(),
"partial apply is not an accepted Job result"
);
disconnect_recorded(&mut host).await;
host.shutdown().await.unwrap();
let original = inspect_store(&state, |store| {
assert!(
+4 -1
View File
@@ -1,7 +1,9 @@
# subjektiv
Shared Host domain implementation for per-Subject Memory, change history, candidate dispositions, and derived surfaces.
Shared Host domain implementation for per-Subject Memory, change history, candidate dispositions, derived surfaces, and Session activity Journals.
`feature-storage` manages physical SQLite storage; the calling Host handles authentication and execution authority.
Journal storage does not generate prose, launch Jobs, or fetch Session history.
Hosts retain body ownership across inactivity, reserve actual execution quiescence, and reconcile final committed history before releasing a Journal writer lifetime. [`journal::binding`] fences new mutations inside storage transactions; historical attribution and exact receipt recovery cannot renew a released lifetime. [`journal::startup`] selects bounded reference context independently of source retention and long-term Memory.
## Main entry points
@@ -9,3 +11,4 @@ Shared Host domain implementation for per-Subject Memory, change history, candid
- [`MemoryRecord`], [`SurfaceSnapshot`]: change history and derived context
- [`api`]: shared Host operations and bounded responses
- [`job`]: consolidation selection rules and result validation
- [`journal`]: individually committed activity operations, source coverage, retry receipts, and bounded reading
+12 -2
View File
@@ -11,8 +11,10 @@
mod context;
mod operation;
pub use context::{HostOperationContext, JobAttemptBinding};
pub use operation::execute;
/// Shared Journal domain operations; no model tools or Host-specific transport.
pub use crate::journal;
pub use context::{HostOperationContext, JobAttemptBinding, JournalHostContext, JournalJobContext};
pub use operation::{checkpoint_journal, execute, execute_journal};
use serde::{Deserialize, Serialize};
use std::collections::HashSet;
@@ -1297,6 +1299,14 @@ pub type OperationResult<T> = std::result::Result<T, OperationError>;
pub fn subjektiv_store_error(error: crate::SubjektivError) -> OperationError {
match error {
crate::SubjektivError::AuthorizationDenied(reason) => {
OperationError::PermissionDenied(reason)
}
crate::SubjektivError::SubjectPermissionsConflict(_)
| crate::SubjektivError::PermissionOperationConflict(_)
| crate::SubjektivError::WorkerExecutionGrantConflict(_) => {
OperationError::Conflict(error.to_string())
}
crate::SubjektivError::SubjectScopeMismatch { .. } => {
OperationError::PermissionDenied(format!("subject_scope_mismatch: {error}"))
}
+2 -1
View File
@@ -1,8 +1,9 @@
//! Trusted Host operation context, never deserialized from model input.
use super::{OperationError, OperationResult, subjektiv_store_error};
pub use crate::journal::{JournalHostContext, JournalJobContext};
use crate::{SubjectSessionAttribution, SubjektivStore};
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct JobAttemptBinding {
pub job_id: String,
pub attempt_id: String,
+18
View File
@@ -1,4 +1,22 @@
use super::*;
/// Commits one Journal effect with its exact retry response, using Journal-only authority.
pub fn execute_journal(
store: &crate::SubjektivStore,
context: &JournalJobContext,
operation: journal::JournalOperation,
) -> journal::JournalResult<journal::JournalReceipt> {
store.execute_journal(context, operation)
}
/// Advances only selected input backed by already committed Journal effects.
pub fn checkpoint_journal(
store: &crate::SubjektivStore,
context: &JournalJobContext,
operation: journal::JournalOperation,
) -> journal::JournalResult<journal::JournalReceipt> {
store.checkpoint_journal(context, operation)
}
use server_api::{
SubjektivMemoryBackendOperation as Op, SubjektivMemoryBackendResponse as Response,
};
+305
View File
@@ -0,0 +1,305 @@
//! Durable Journal writer lifetime, issued only from the Host's body binding. Kernel leases
//! and Runtime liveness do not end ownership. The immutable binding ID fences retained contexts
//! even when the same Worker later reconnects. Hosts serialize release with body lifecycle and
//! reconcile final committed history before calling release; this store checks saved effects.
use super::*;
use rusqlite::{Connection, OptionalExtension, params};
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalWriterBinding {
pub subject_id: String,
pub runtime_id: String,
pub worker_id: String,
pub binding_id: String,
}
pub(crate) fn migrate(tx: &Transaction<'_>) -> Result<(), FeatureStorageError> {
tx.execute_batch("CREATE TABLE journal_writer_bindings (
binding_id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
runtime_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
active INTEGER NOT NULL CHECK(active IN (0,1))
);
CREATE UNIQUE INDEX journal_one_body_binding ON journal_writer_bindings(subject_id) WHERE active=1;
CREATE TABLE journal_session_bindings (
session_id TEXT PRIMARY KEY REFERENCES subject_session_attributions(session_id) ON DELETE RESTRICT,
binding_id TEXT NOT NULL REFERENCES journal_writer_bindings(binding_id) ON DELETE RESTRICT
);
CREATE INDEX journal_body_history ON journal_writer_bindings(subject_id,active);
CREATE INDEX journal_body_sessions ON journal_session_bindings(binding_id);
CREATE INDEX journal_recent_subject ON journals(subject_id,updated_at DESC,id DESC);")?;
Ok(())
}
fn denied() -> JournalError {
JournalError::PermissionDenied("Journal writer binding is absent, released or replaced".into())
}
pub(super) fn check_writer(tx: &Connection, host: &JournalHostContext) -> JournalResult<()> {
let attr = host.attribution.as_ref().ok_or_else(denied)?;
let binding = host.writer_binding_id.as_ref().ok_or_else(denied)?;
if !tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings b JOIN journal_session_bindings s ON s.binding_id=b.binding_id WHERE b.binding_id=?1 AND b.subject_id=?2 AND b.runtime_id=?3 AND b.worker_id=?4 AND b.active=1 AND s.session_id=?5)", params![binding,host.subject_id,attr.runtime_id,attr.worker_id,attr.session_id], |r| r.get::<_,bool>(0))? {
return Err(denied());
}
Ok(())
}
pub(super) fn require_saved_position(tx: &Connection, generation: &str) -> JournalResult<()> {
let saved: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_operations p WHERE p.generation_id=?1 AND json_extract(p.request_json,'$.action.action')='update_position' AND p.rowid > COALESCE((SELECT MAX(r.rowid) FROM journal_operations r WHERE r.generation_id=?1 AND json_extract(r.request_json,'$.action.action') IN ('create','rewrite','fix','confirm_unchanged')),0))",[generation],|r|r.get(0))?;
if !saved {
return Err(JournalError::Conflict("Journal current position must be saved after the generation's activity records before completion".into()));
}
Ok(())
}
impl SubjektivStore {
/// The last released body's most recently updated Journal Session, independent of source
/// retention. Session attachment order may differ from the order of recorded activity.
pub fn journal_predecessor_session(
&self,
binding: &JournalWriterBinding,
) -> JournalResult<Option<String>> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT j.source_session_id FROM journals j JOIN journal_session_bindings s ON s.session_id=j.source_session_id WHERE j.subject_id=?1 AND s.binding_id=(SELECT binding_id FROM journal_writer_bindings WHERE subject_id=?1 AND active=0 ORDER BY rowid DESC LIMIT 1) ORDER BY j.updated_at DESC,j.id DESC LIMIT 1", [&binding.subject_id], |r| r.get(0)).optional()?)
})
}
/// Absence proof for failed body creation, including bindings never attached to a Session
/// and released lifetimes. The Host must hold its lifecycle fence across this observation
/// and recovery; absence alone does not fence a delayed Runtime create.
pub fn journal_body_is_unpublished(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row(
"SELECT NOT EXISTS(SELECT 1 FROM journal_writer_bindings
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)
AND NOT EXISTS(SELECT 1 FROM subject_session_attributions
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)",
params![subject, runtime, worker],
|row| row.get(0),
)?)
})
}
pub fn journal_worker_was_released(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3 AND active=0)", params![subject,runtime,worker], |r| r.get(0))?)
})
}
pub fn journal_writer_active(&self, binding: &JournalWriterBinding) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE binding_id=?1 AND subject_id=?2 AND runtime_id=?3 AND worker_id=?4 AND active=1)", params![binding.binding_id,binding.subject_id,binding.runtime_id,binding.worker_id], |r| r.get(0))?)
})
}
pub fn journal_session_writer_matches(
&self,
binding: &JournalWriterBinding,
session: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_session_bindings WHERE binding_id=?1 AND session_id=?2)", params![binding.binding_id,session], |r| r.get(0))?)
})
}
/// Called under the existing Host Subject lifecycle authority. A stopped or unavailable
/// owner remains the owner; only the exact same body may resume an unreleased binding.
pub fn claim_journal_writer(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<JournalWriterBinding> {
for identifier in [subject, runtime, worker] {
if identifier.is_empty()
|| identifier.len() > 128
|| identifier.trim() != identifier
|| identifier.chars().any(char::is_control)
{
return Err(JournalError::InvalidInput(
"invalid Journal binding identity".into(),
));
}
}
self.database.try_transaction(|tx| {
let state: Option<String> = tx.query_row("SELECT state FROM subjects WHERE subject_id=?1", [subject], |r| r.get(0)).optional()?;
if state.as_deref() != Some("active") { return Err(JournalError::PermissionDenied("Subject is absent or retired".into())); }
let current: Option<JournalWriterBinding> = tx.query_row("SELECT subject_id,runtime_id,worker_id,binding_id FROM journal_writer_bindings WHERE subject_id=?1 AND active=1", [subject], |r| Ok(JournalWriterBinding { subject_id:r.get(0)?,runtime_id:r.get(1)?,worker_id:r.get(2)?,binding_id:r.get(3)? })).optional()?;
if let Some(current) = current {
if current.runtime_id != runtime || current.worker_id != worker {
return Err(JournalError::Conflict("Subject is still occupied; explicit Idle and Journal-complete release required".into()));
}
return Ok(current);
}
// Upgraded T-734 data can contain a removed body's durable recording handoff.
// Retention is not permission for a different body to occupy that Subject.
let legacy_busy: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journals j JOIN subject_session_attributions a ON a.session_id=j.source_session_id WHERE j.subject_id=?1 AND (a.runtime_id<>?2 OR a.worker_id<>?3) AND NOT EXISTS(SELECT 1 FROM journal_session_bindings b WHERE b.session_id=j.source_session_id) AND (EXISTS(SELECT 1 FROM journal_generations g WHERE g.journal_id=j.id AND g.active=1) OR EXISTS(SELECT 1 FROM journal_sources s,json_each(s.accepted_json) r WHERE s.journal_id=j.id AND json_extract(r.value,'$.end')>s.reflected_bytes)))", params![subject,runtime,worker], |r| r.get(0))?;
if legacy_busy { return Err(JournalError::Conflict("previous body's retained Journal handoff is not complete".into())); }
let binding = JournalWriterBinding { subject_id:subject.into(),runtime_id:runtime.into(),worker_id:worker.into(),binding_id:crate::issued_id("journal-binding") };
tx.execute("INSERT INTO journal_writer_bindings(binding_id,subject_id,runtime_id,worker_id,active) VALUES(?1,?2,?3,?4,1)", params![binding.binding_id,subject,runtime,worker])?;
Ok(binding)
})
}
/// Binds a new Session once. Reconnecting a Worker under a later binding cannot acquire
/// mutation authority for its old Session. Attribution and binding are Host-only input.
pub fn bind_journal_session(
&self,
binding: &JournalWriterBinding,
attribution: SubjectSessionAttribution,
) -> JournalResult<JournalHostContext> {
let mut host = JournalHostContext::validated_session(self, attribution)?;
self.database.try_transaction(|tx| {
let attr = host.attribution.as_ref().expect("validated Session");
if binding.subject_id != attr.subject_id || binding.runtime_id != attr.runtime_id || binding.worker_id != attr.worker_id {
return Err(denied());
}
tx.execute("INSERT OR IGNORE INTO journal_session_bindings(session_id,binding_id) SELECT ?1,binding_id FROM journal_writer_bindings WHERE binding_id=?2 AND subject_id=?3 AND runtime_id=?4 AND worker_id=?5 AND active=1",params![attr.session_id,binding.binding_id,attr.subject_id,attr.runtime_id,attr.worker_id])?;
host.writer_binding_id = Some(binding.binding_id.clone());
check_writer(tx,&host)?;
Ok(host.clone())
})
}
/// Destructive release is distinct from a continuity handoff: every attributed Session
/// of this body must already have a permanent erasure tombstone. This also records a
/// released lifetime for pre-binding bodies, without granting them new mutation authority.
pub fn release_erased_journal_body(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_transaction(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
let remaining: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM subject_session_attributions a WHERE a.subject_id=?1 AND a.runtime_id=?2 AND a.worker_id=?3 AND NOT EXISTS(SELECT 1 FROM journal_erased_sessions e WHERE e.session_id=a.session_id AND e.subject_id=a.subject_id))", params![host.subject_id,attr.runtime_id,attr.worker_id], |r| r.get(0))?;
if remaining {
return Ok(false);
}
let existing: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)", params![host.subject_id,attr.runtime_id,attr.worker_id], |r| r.get(0))?;
tx.execute("UPDATE journal_writer_bindings SET active=0 WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3",params![host.subject_id,attr.runtime_id,attr.worker_id])?;
if !existing {
tx.execute("INSERT INTO journal_writer_bindings(binding_id,subject_id,runtime_id,worker_id,active) VALUES(?1,?2,?3,?4,0)", params![crate::issued_id("journal-erased-body"),host.subject_id,attr.runtime_id,attr.worker_id])?;
}
Ok(true)
})
}
/// The Host must hold its lifecycle fence, confirm Idle, reconcile all committed original
/// Sessions (including the save/notification gap), and exclude queued/running/Unknown Jobs.
/// This transaction rejects retained writers, pending source bytes and active generations.
/// It never fixes an open entry or declares the represented work complete.
pub fn release_journal_writer(&self, binding: &JournalWriterBinding) -> JournalResult<()> {
self.database.try_transaction(|tx| {
let active: Option<bool> = tx.query_row("SELECT active FROM journal_writer_bindings WHERE binding_id=?1 AND subject_id=?2 AND runtime_id=?3 AND worker_id=?4",params![binding.binding_id,binding.subject_id,binding.runtime_id,binding.worker_id],|r|r.get(0)).optional()?;
let Some(active) = active else { return Err(denied()); };
if !active { return Ok(()); }
let unbound: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM subject_session_attributions a WHERE a.subject_id=?1 AND a.runtime_id=?2 AND a.worker_id=?3 AND NOT EXISTS(SELECT 1 FROM journal_session_bindings s WHERE s.session_id=a.session_id) AND NOT EXISTS(SELECT 1 FROM journal_erased_sessions e WHERE e.session_id=a.session_id AND e.subject_id=a.subject_id))",params![binding.subject_id,binding.runtime_id,binding.worker_id],|r|r.get(0))?;
if unbound { return Err(JournalError::Conflict("body has legacy Sessions whose committed history has not been reconciled under its writer binding".into())); }
let busy: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_generations g JOIN journals j ON j.id=g.journal_id JOIN journal_session_bindings s ON s.session_id=j.source_session_id WHERE s.binding_id=?1 AND g.active=1)",[&binding.binding_id],|r|r.get(0))?;
if busy { return Err(JournalError::Conflict("Journal generation remains active or its result is unconfirmed".into())); }
let mut stmt = tx.prepare("SELECT s.accepted_json,s.reflected_bytes,s.unavailable_json FROM journal_sources s JOIN journals j ON j.id=s.journal_id JOIN journal_session_bindings b ON b.session_id=j.source_session_id WHERE b.binding_id=?1")?;
let mut rows = stmt.query([&binding.binding_id])?;
while let Some(row) = rows.next()? {
let accepted: Vec<JournalByteRange> = serde_json::from_str(&row.get::<_,String>(0)?)?;
let reflected: u64 = row.get(1)?;
if accepted.iter().any(|r| r.end > reflected) {
return Err(JournalError::Conflict("Journal activity/position/checkpoint has unreflected source bytes".into()));
}
}
let mut checkpoints = tx.prepare("SELECT p.generation_id FROM journal_operations p JOIN journals j ON j.id=p.journal_id JOIN journal_session_bindings b ON b.session_id=j.source_session_id WHERE b.binding_id=?1 AND p.rowid=(SELECT MAX(c.rowid) FROM journal_operations c WHERE c.journal_id=j.id AND json_extract(c.request_json,'$.action.action')='checkpoint')")?;
let generations = checkpoints.query_map([&binding.binding_id], |r| r.get::<_,String>(0))?;
for generation in generations { require_saved_position(tx, &generation?)?; }
tx.execute("UPDATE journal_writer_bindings SET active=0 WHERE binding_id=?1 AND active=1",[&binding.binding_id])?;
Ok(())
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn unpublished_body_rejects_orphan_and_released_bindings_and_attribution_only() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _workspace, store) = crate::tests::open_store(temp.path(), "absence-tests");
let subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
let other_subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
// No Session was attached: an orphan binding is still published body evidence.
let binding = store
.claim_journal_writer(&subject.id, "runtime", "worker")
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
for (subject_id, runtime, worker) in [
(other_subject.id.as_str(), "runtime", "worker"),
(subject.id.as_str(), "other-runtime", "worker"),
(subject.id.as_str(), "runtime", "other-worker"),
] {
assert!(
store
.journal_body_is_unpublished(subject_id, runtime, worker)
.unwrap()
);
}
store.release_journal_writer(&binding).unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
store
.record_session_attribution(
SubjectSessionAttribution::new(
&subject.id,
"runtime",
"attribution-only",
"session",
)
.unwrap(),
)
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "other-runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&other_subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "never-published")
.unwrap()
);
}
}
+181
View File
@@ -0,0 +1,181 @@
//! Bounded saved-Journal browser projections. No history fetch, reconciliation or execution.
use super::store::{check_host, entry, fit_page, journal, summary};
use super::*;
use rusqlite::params;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct EntryCursor {
scope: String,
subject: String,
journal: Option<String>,
query: Option<String>,
correcting: Option<String>,
created_at: String,
id: String,
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
impl SubjektivStore {
pub fn journal_entry_summary(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
) -> JournalResult<JournalEntrySummary> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
summary(tx, entry(tx, journal_id, entry_id)?)
})
}
/// Correction targets are capped at sixteen by the writer contract, and may be in
/// another Journal of this Subject. Include actual Journal IDs rather than guessing.
pub fn journal_correction_targets(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
) -> JournalResult<Vec<JournalEntrySummary>> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
entry(tx, journal_id, entry_id)?;
let mut stmt = tx.prepare(
"SELECT e.journal_id,e.id FROM journal_corrections c
JOIN journal_entries e ON e.subject_id=c.subject_id AND e.id=c.target_id
WHERE c.subject_id=?1 AND c.entry_id=?2 ORDER BY e.created_at,e.id LIMIT 16",
)?;
let rows = stmt
.query_map(params![host.subject_id, entry_id], |r| {
Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?))
})?
.collect::<Result<Vec<_>, _>>()?;
rows.into_iter()
.map(|(journal_id, id)| summary(tx, entry(tx, &journal_id, &id)?))
.collect()
})
}
/// Recent entries use immutable creation time and ID as a stable descending keyset.
/// Search is literal substring matching (not SQL LIKE wildcards), bounded to 256 bytes.
/// A correcting selector filters saved correction edges, not inferred text or intervals.
pub fn browse_journal_entries(
&self,
host: &JournalHostContext,
journal_id: Option<&str>,
query: Option<&str>,
correcting: Option<&str>,
after: Option<&str>,
limit: usize,
) -> JournalResult<JournalPage<JournalEntrySummary>> {
if !(1..=MAX_JOURNAL_PAGE).contains(&limit) {
return Err(invalid("page limit must be 1..32"));
}
if query.is_some_and(|q| q.len() > 256 || q.chars().any(char::is_control)) {
return Err(invalid(
"search query must be at most 256 bytes without controls",
));
}
if correcting.is_some() && journal_id.is_none() {
return Err(invalid("correction lookup requires a Journal"));
}
let cursor: Option<EntryCursor> = after
.map(|raw| {
if raw.len() > 4096 {
return Err(invalid("entry cursor exceeds byte limit"));
}
let cursor: EntryCursor =
serde_json::from_str(raw).map_err(|_| invalid("invalid entry cursor"))?;
if cursor.scope != host.scope_id
|| cursor.subject != host.subject_id
|| cursor.journal.as_deref() != journal_id
|| cursor.query.as_deref() != query
|| cursor.correcting.as_deref() != correcting
{
return Err(invalid(
"entry cursor belongs to another scope, Subject or filter",
));
}
Ok(cursor)
})
.transpose()?;
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
if let Some(id) = journal_id {
journal(tx, host, id, false)?;
}
if let Some(id) = correcting {
entry(tx, journal_id.unwrap(), id)?;
}
let mut stmt = tx.prepare(
"SELECT e.journal_id,e.id,e.created_at FROM journal_entries e
WHERE e.subject_id=?1 AND (?4 IS NOT NULL OR ?2 IS NULL OR e.journal_id=?2)
AND (?3 IS NULL OR instr(lower(e.title),lower(?3))>0
OR instr(lower(e.body),lower(?3))>0 OR instr(lower(e.ongoing),lower(?3))>0)
AND (?4 IS NULL OR EXISTS(SELECT 1 FROM journal_corrections c
WHERE c.subject_id=?1 AND c.entry_id=e.id AND c.target_id=?4))
AND (?5 IS NULL OR (e.created_at,e.id)<(?5,?6))
ORDER BY e.created_at DESC,e.id DESC LIMIT ?7",
)?;
let rows = stmt
.query_map(
params![
host.subject_id,
journal_id,
query,
correcting,
cursor.as_ref().map(|c| &c.created_at),
cursor.as_ref().map(|c| &c.id),
limit + 1
],
|r| {
Ok((
r.get::<_, String>(0)?,
r.get::<_, String>(1)?,
r.get::<_, String>(2)?,
))
},
)?
.collect::<Result<Vec<_>, _>>()?;
let more = rows.len() > limit;
let mut items = Vec::new();
for (journal_id_value, id, created_at) in rows.into_iter().take(limit) {
let next = serde_json::to_string(&EntryCursor {
scope: host.scope_id.clone(),
subject: host.subject_id.clone(),
journal: journal_id.map(str::to_owned),
query: query.map(str::to_owned),
correcting: correcting.map(str::to_owned),
created_at,
id: id.clone(),
})?;
items.push((next, summary(tx, entry(tx, &journal_id_value, &id)?)?));
}
fit_page(items, more)
})
}
/// True only for admitted source bytes which the durable checkpoint has not reflected.
/// False does not claim that all original Session activity was admitted or recorded.
pub fn journal_has_unreflected_sources(
&self,
host: &JournalHostContext,
journal_id: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
Ok(tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_sources s, json_each(s.accepted_json) b
WHERE s.journal_id=?1 AND json_extract(b.value,'$.end')>s.reflected_bytes)",
[journal_id],
|r| r.get(0),
)?)
})
}
}
+153
View File
@@ -0,0 +1,153 @@
//! Journal-specific retention and explicit erasure. Hosts reconcile the final committed public
//! Session before consulting retention; absence from the source ledger is not proof of no work.
//! Ordinary retention never deletes Journal text. Explicit erasure leaves a content-free Session
//! tombstone in the same transaction as removal, fencing delayed admission and Job operations.
use super::*;
use crate::now;
use rusqlite::{Connection, OptionalExtension, params};
pub(crate) fn migrate(tx: &Transaction<'_>) -> Result<(), FeatureStorageError> {
tx.execute_batch(
"ALTER TABLE journal_generations ADD COLUMN completion_json TEXT;
CREATE TABLE journal_generation_deliveries (
generation_id TEXT NOT NULL REFERENCES journal_generations(id) ON DELETE CASCADE,
source_order INTEGER NOT NULL,
ranges_json TEXT NOT NULL,
PRIMARY KEY(generation_id, source_order)
);
CREATE TABLE journal_erased_sessions (
session_id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL,
request_id TEXT NOT NULL UNIQUE,
erased_at TEXT NOT NULL
);
DROP TRIGGER journal_operations_no_delete;
CREATE TRIGGER journal_operations_no_delete BEFORE DELETE ON journal_operations
WHEN NOT EXISTS (
SELECT 1 FROM journals j JOIN journal_erased_sessions e
ON e.session_id=j.source_session_id AND e.subject_id=j.subject_id
WHERE j.id=OLD.journal_id
)
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are retained'); END;",
)?;
Ok(())
}
pub(super) fn reject_erased(tx: &Connection, host: &JournalHostContext) -> JournalResult<()> {
let session = host
.attribution
.as_ref()
.ok_or_else(|| JournalError::PermissionDenied("Session authority required".into()))?;
if tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1)",
[&session.session_id],
|row| row.get::<_, bool>(0),
)? {
return Err(JournalError::PermissionDenied(
"Session was explicitly erased".into(),
));
}
Ok(())
}
impl SubjektivStore {
/// Reads the content-free erasure fence under the original Session attribution.
/// Hosts use this before source reconciliation so ordinary removal of an erased Session
/// neither fetches erased material nor forces a new recording generation.
pub fn journal_session_erased(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
Ok(tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1 AND subject_id=?2)",
params![attr.session_id, host.subject_id],
|row| row.get(0),
)?)
})
}
/// Finds this Host's immutable source Session without scanning unrelated Journals.
pub fn journal_for_session(&self, host: &JournalHostContext) -> JournalResult<Option<String>> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
reject_erased(tx, host)?;
let attr = host
.attribution
.as_ref()
.expect("checked Session authority");
Ok(tx
.query_row(
"SELECT id FROM journals WHERE source_session_id=?1 AND subject_id=?2",
params![attr.session_id, host.subject_id],
|row| row.get(0),
)
.optional()?)
})
}
/// True while any admitted bytes still need reflection, including unavailable bytes.
/// A Host must first reconcile storage against admission under its deletion/append fence.
pub fn journal_retention_required(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
if tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1)", [&attr.session_id], |r| r.get::<_,bool>(0))? {
return Ok(false);
}
let mut stmt = tx.prepare("SELECT s.accepted_json,s.reflected_bytes FROM journal_sources s JOIN journals j ON j.id=s.journal_id WHERE j.source_session_id=?1 AND j.subject_id=?2 ORDER BY s.source_order")?;
let mut rows = stmt.query(params![attr.session_id,host.subject_id])?;
while let Some(row) = rows.next()? {
let accepted: Vec<JournalByteRange> = serde_json::from_str(&row.get::<_, String>(0)?)?;
let reflected: u64 = row.get(1)?;
if accepted.iter().any(|range| range.end > reflected) {
return Ok(true);
}
}
Ok(false)
})
}
/// Explicit Host-authorized complete erasure, not an ordinary Worker removal policy.
/// No generation is forced. The tombstone remains even if no Journal was admitted yet.
/// Exact retries converge; a request ID cannot be reused to erase a different Session.
pub fn erase_journal_session(
&self,
host: &JournalHostContext,
request_id: &str,
) -> JournalResult<()> {
if request_id.is_empty()
|| request_id.len() > 128
|| request_id.chars().any(char::is_control)
{
return Err(JournalError::InvalidInput(
"invalid erasure request identifier".into(),
));
}
self.database.try_transaction(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
let previous: Option<(String,String)> = tx.query_row("SELECT session_id,subject_id FROM journal_erased_sessions WHERE request_id=?1", [request_id], |r| Ok((r.get(0)?,r.get(1)?))).optional()?;
if previous.as_ref().is_some_and(|(session,subject)| session!=&attr.session_id || subject!=&host.subject_id) {
return Err(JournalError::Conflict("erasure request belongs to a different Session".into()));
}
let erased_by: Option<String> = tx.query_row("SELECT request_id FROM journal_erased_sessions WHERE session_id=?1", [&attr.session_id], |row| row.get(0)).optional()?;
if erased_by.as_deref().is_some_and(|key| key != request_id) {
return Err(JournalError::Conflict("Session already erased by a different request".into()));
}
tx.execute("INSERT OR IGNORE INTO journal_erased_sessions(session_id,subject_id,request_id,erased_at) VALUES(?1,?2,?3,?4)", params![attr.session_id,host.subject_id,request_id,now()])?;
let journal: Option<String> = tx.query_row("SELECT id FROM journals WHERE source_session_id=?1 AND subject_id=?2",params![attr.session_id,host.subject_id],|r|r.get(0)).optional()?;
if let Some(id) = journal {
// The erasure marker is the only condition permitting receipt deletion. Normal
// record/checkpoint/retention paths retain their immutable receipts.
tx.execute("DELETE FROM journal_operations WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_generations WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_corrections WHERE entry_id IN (SELECT id FROM journal_entries WHERE journal_id=?1) OR target_id IN (SELECT id FROM journal_entries WHERE journal_id=?1)",[&id])?;
tx.execute("DELETE FROM journal_entries WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_sources WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journals WHERE id=?1",[&id])?;
}
Ok(())
})
}
}
@@ -0,0 +1,668 @@
//! Additive, immutable provenance-to-context links, not additional evidence for a Memory claim.
//!
//! Reconciliation reads only saved candidate anchors and the Host-attested Journal source ledger.
//! It never fetches history, starts Workers, changes extraction/consolidation, fixes an open event,
//! or mutates Memory. A stable evidence reference identifies a public entry, not its legacy numeric
//! display range. Missing stable identity stays unconnected; numeric ranges are not guessed to be
//! canonical order. Saved edges remain usable without the original runtime and cannot be retargeted.
//! Journal content is never copied here. Explicit erasure retains edges and exposes the tombstone.
use super::{
JournalError, JournalHostContext, JournalResult, JournalSourcePointer, JournalTextField,
JournalTextPage, MAX_JOURNAL_PAGE, MAX_JOURNAL_RESPONSE_BYTES,
};
use crate::{MAX_STAGING_ANCHORS, SubjectStagingRecord, SubjektivStore};
use memory::schema::SourceEvidenceRef;
use rusqlite::{Connection, OptionalExtension, Transaction, params};
use serde::{Deserialize, Serialize};
#[cfg(test)]
mod tests;
pub(crate) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("schema.sql"))?;
Ok(())
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalSelector {
pub memory_id: String,
pub change_id: String,
pub candidate_id: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case", deny_unknown_fields)]
pub enum MemoryJournalRequest {
Reconcile {
selector: MemoryJournalSelector,
},
/// Reads saved link state only; does not implicitly retry reconciliation.
Links {
selector: MemoryJournalSelector,
},
/// Continues reconciliation over immutable source indices, at most MAX_STAGING_ANCHORS per call.
ReconcilePage {
selector: MemoryJournalSelector,
source_offset: usize,
limit: usize,
},
/// Reads another saved source page without reconciliation.
LinksPage {
selector: MemoryJournalSelector,
source_offset: usize,
limit: usize,
},
Read {
selector: MemoryJournalSelector,
source_index: usize,
journal_id: String,
entry_id: String,
expected_token: Option<String>,
byte_offset: usize,
max_bytes: usize,
},
Related {
journal_id: String,
entry_id: String,
after: Option<RelatedMemoryCursor>,
limit: usize,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(
tag = "kind",
content = "value",
rename_all = "snake_case",
deny_unknown_fields
)]
pub enum MemoryJournalResponse {
Links(MemoryJournalLinks),
Read(MemoryJournalRead),
Related(RelatedMemories),
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum OriginalSourceAvailability {
/// No saved Host attestation. None of these values imply a live history fetch.
Unknown,
Missing,
Available,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalTargetAvailability {
Available,
Unavailable,
Erased,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum UnconnectedReason {
NotReconciled,
InsufficientSource,
SourceNotAdmitted,
AmbiguousSource,
JournalNotFixed,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "state", rename_all = "snake_case", deny_unknown_fields)]
pub enum SourceConnection {
Unconnected {
reason: UnconnectedReason,
},
Linked {
journal_id: String,
entry_id: String,
target: JournalTargetAvailability,
/// A saved correction invites reconsideration, never an automatic Memory edit.
reconsideration: bool,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalSourceLink {
pub source_index: usize,
pub original_source: OriginalSourceAvailability,
pub connection: SourceConnection,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalLinks {
pub selector: MemoryJournalSelector,
pub sources: Vec<MemoryJournalSourceLink>,
pub total_sources: usize,
pub next_source_index: Option<usize>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalRead {
pub source: MemoryJournalSourceLink,
/// Absent for an unavailable or explicitly erased target; never falls back to another entry.
pub page: Option<JournalTextPage>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemoryCursor {
pub memory_id: String,
pub change_id: String,
pub candidate_id: String,
pub source_index: usize,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemory {
pub selector: MemoryJournalSelector,
pub source_index: usize,
pub reconsideration: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemories {
pub items: Vec<RelatedMemory>,
pub next: Option<RelatedMemoryCursor>,
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
fn identifier(value: &str) -> JournalResult<()> {
if value.is_empty() || value.len() > 256 || value.chars().any(char::is_control) {
return Err(invalid("invalid Memory/Journal selector identifier"));
}
Ok(())
}
fn selector_valid(selector: &MemoryJournalSelector) -> JournalResult<()> {
identifier(&selector.memory_id)?;
identifier(&selector.change_id)?;
identifier(&selector.candidate_id)
}
fn candidate(
store: &SubjektivStore,
tx: &Connection,
host: &JournalHostContext,
selector: &MemoryJournalSelector,
) -> JournalResult<SubjectStagingRecord> {
super::store::check_host(store, tx, host, false)?;
selector_valid(selector)?;
// The immutable change-candidate edge, not the current Memory projection or a supplied
// candidate ID alone, authorizes this provenance selector.
let raw: Option<String> = tx
.query_row(
"SELECT s.record_json FROM staging_records s JOIN memory_change_candidates c
ON c.subject_id=s.subject_id AND c.candidate_id=s.candidate_id
WHERE c.subject_id=?1 AND c.memory_id=?2 AND c.change_id=?3 AND c.candidate_id=?4",
params![
host.subject_id,
selector.memory_id,
selector.change_id,
selector.candidate_id
],
|r| r.get(0),
)
.optional()?;
let record: SubjectStagingRecord = serde_json::from_str(
&raw.ok_or_else(|| JournalError::NotFound("Memory change candidate membership".into()))?,
)?;
Ok(record)
}
#[derive(Debug)]
struct SavedLink {
reason: String,
journal_id: Option<String>,
entry_id: Option<String>,
session_id: Option<String>,
order: Option<u64>,
}
fn saved(
tx: &Connection,
host: &JournalHostContext,
candidate_id: &str,
index: usize,
) -> JournalResult<Option<SavedLink>> {
Ok(tx.query_row(
"SELECT reason,journal_id,entry_id,source_session_id,source_order FROM memory_journal_links
WHERE subject_id=?1 AND candidate_id=?2 AND source_index=?3",
params![host.subject_id, candidate_id, index],
|r| Ok(SavedLink { reason:r.get(0)?,journal_id:r.get(1)?,entry_id:r.get(2)?,session_id:r.get(3)?,order:r.get(4)? }),
).optional()?)
}
fn original_availability(
tx: &Connection,
journal: &str,
order: u64,
) -> JournalResult<OriginalSourceAvailability> {
let unavailable: Option<String> = tx
.query_row(
"SELECT unavailable_json FROM journal_sources WHERE journal_id=?1 AND source_order=?2",
params![journal, order],
|r| r.get(0),
)
.optional()?;
Ok(match unavailable {
None => OriginalSourceAvailability::Unknown,
Some(raw) => {
let ranges: Vec<super::JournalByteRange> = serde_json::from_str(&raw)?;
if ranges.is_empty() {
OriginalSourceAvailability::Available
} else {
OriginalSourceAvailability::Missing
}
}
})
}
fn project(
tx: &Connection,
host: &JournalHostContext,
candidate_id: &str,
index: usize,
) -> JournalResult<MemoryJournalSourceLink> {
let Some(link) = saved(tx, host, candidate_id, index)? else {
return Ok(MemoryJournalSourceLink {
source_index: index,
original_source: OriginalSourceAvailability::Unknown,
connection: SourceConnection::Unconnected {
reason: UnconnectedReason::NotReconciled,
},
});
};
if link.reason != "linked" {
let reason = match link.reason.as_str() {
"insufficient_source" => UnconnectedReason::InsufficientSource,
"source_not_admitted" => UnconnectedReason::SourceNotAdmitted,
"ambiguous_source" => UnconnectedReason::AmbiguousSource,
"journal_not_fixed" => UnconnectedReason::JournalNotFixed,
_ => return Err(invalid("invalid saved connection reason")),
};
return Ok(MemoryJournalSourceLink {
source_index: index,
original_source: OriginalSourceAvailability::Unknown,
connection: SourceConnection::Unconnected { reason },
});
}
let journal_id = link
.journal_id
.ok_or_else(|| invalid("saved link lacks Journal"))?;
let entry_id = link
.entry_id
.ok_or_else(|| invalid("saved link lacks entry"))?;
let session = link
.session_id
.ok_or_else(|| invalid("saved link lacks Session"))?;
let order = link
.order
.ok_or_else(|| invalid("saved link lacks source order"))?;
let erased: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE subject_id=?1 AND session_id=?2)",
params![host.subject_id, session], |r| r.get(0),
)?;
let exists: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_entries e JOIN journals j ON j.id=e.journal_id
WHERE e.subject_id=?1 AND e.journal_id=?2 AND e.id=?3 AND j.source_session_id=?4)",
params![host.subject_id, journal_id, entry_id, session],
|r| r.get(0),
)?;
let reconsideration = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_corrections WHERE subject_id=?1 AND target_id=?2)",
params![host.subject_id, entry_id],
|r| r.get(0),
)?;
Ok(MemoryJournalSourceLink {
source_index: index,
original_source: original_availability(tx, &journal_id, order)?,
connection: SourceConnection::Linked {
journal_id,
entry_id,
target: if erased {
JournalTargetAvailability::Erased
} else if exists {
JournalTargetAvailability::Available
} else {
JournalTargetAvailability::Unavailable
},
reconsideration,
},
})
}
struct Resolved {
journal: String,
entry: String,
session: String,
order: u64,
}
fn resolve(
tx: &Connection,
host: &JournalHostContext,
source: &SourceEvidenceRef,
) -> JournalResult<Result<Resolved, &'static str>> {
let Some(evidence) = source.evidence_id.as_deref() else {
return Ok(Err("insufficient_source"));
};
if !evidence.starts_with('E')
|| evidence.len() < 2
|| evidence.len() > 129
|| !evidence[1..]
.bytes()
.all(|c| c.is_ascii_alphanumeric() || c == b'-')
|| (source.session_id.is_none() && source.segment_id.is_none())
{
return Ok(Err("insufficient_source"));
}
// Segment identity may be omitted only when an immutable Session attribution disambiguates it.
// A candidate's primary legacy source range is not substituted for an individual anchor.
let segment = match source.segment_id.as_deref() {
Some(raw) => match uuid::Uuid::parse_str(raw) {
Ok(id) => Some(id),
Err(_) => return Ok(Err("insufficient_source")),
},
None => None,
};
let mut stmt = tx.prepare(
"SELECT j.id,j.source_session_id,s.source_order,s.pointer_json FROM journal_sources s
JOIN journals j ON j.id=s.journal_id
JOIN subject_session_attributions a ON a.subject_id=j.subject_id AND a.session_id=j.source_session_id
WHERE j.subject_id=?1 AND s.entry_ref=?2
AND (?3 IS NULL OR j.source_session_id=?3)
AND (?4 IS NULL OR json_extract(s.pointer_json,'$.segment_id')=?4)
LIMIT 2",
)?;
let found = stmt
.query_map(
params![
host.subject_id,
evidence,
source.session_id,
segment.map(|s| s.to_string())
],
|r| {
Ok((
r.get::<_, String>(0)?,
r.get::<_, String>(1)?,
r.get::<_, u64>(2)?,
r.get::<_, String>(3)?,
))
},
)?
.collect::<Result<Vec<_>, _>>()?;
if found.is_empty() {
return Ok(Err("source_not_admitted"));
}
if found.len() != 1 {
return Ok(Err("ambiguous_source"));
}
let (journal, session, order, pointer) = &found[0];
let pointer: JournalSourcePointer = serde_json::from_str(pointer)?;
if pointer.entry_ref != evidence || segment.is_some_and(|s| s != pointer.segment_id) {
return Ok(Err("ambiguous_source"));
}
// A fixed endpoint covers (previous_fixed_end, source_end]. The Session start is the first
// interval's lower boundary. Processing checkpoints and open entries cannot supply an end.
let mut stmt = tx.prepare(
"SELECT e.id FROM journal_entries e WHERE e.subject_id=?1 AND e.journal_id=?2
AND e.state='fixed' AND e.source_order>=?3
AND NOT EXISTS(SELECT 1 FROM journal_entries p WHERE p.journal_id=e.journal_id
AND p.position<e.position AND p.state='fixed' AND p.source_order>=?3)
ORDER BY e.position LIMIT 2",
)?;
let entries = stmt
.query_map(params![host.subject_id, journal, order], |r| {
r.get::<_, String>(0)
})?
.collect::<Result<Vec<_>, _>>()?;
match entries.as_slice() {
[entry] => Ok(Ok(Resolved {
journal: journal.clone(),
entry: entry.clone(),
session: session.clone(),
order: *order,
})),
[] => Ok(Err("journal_not_fixed")),
_ => Ok(Err("ambiguous_source")),
}
}
fn reconcile(
tx: &Connection,
host: &JournalHostContext,
record: &SubjectStagingRecord,
indices: std::ops::Range<usize>,
) -> JournalResult<()> {
for index in indices {
let source = &record.source_refs[index];
if saved(tx, host, &record.id, index)?.is_some_and(|s| s.reason == "linked") {
continue;
}
let (reason, resolved) = match resolve(tx, host, source)? {
Ok(link) => ("linked", Some(link)),
Err(reason) => (reason, None),
};
tx.execute(
"INSERT INTO memory_journal_links(subject_id,candidate_id,source_index,reason,
journal_id,entry_id,source_session_id,source_order) VALUES(?1,?2,?3,?4,?5,?6,?7,?8)
ON CONFLICT(subject_id,candidate_id,source_index) DO UPDATE SET
reason=excluded.reason,journal_id=excluded.journal_id,entry_id=excluded.entry_id,
source_session_id=excluded.source_session_id,source_order=excluded.source_order",
params![
host.subject_id,
record.id,
index,
reason,
resolved.as_ref().map(|r| r.journal.as_str()),
resolved.as_ref().map(|r| r.entry.as_str()),
resolved.as_ref().map(|r| r.session.as_str()),
resolved.as_ref().map(|r| r.order)
],
)?;
}
Ok(())
}
impl SubjektivStore {
/// Scoped Memory provenance bridge. Reconcile is additive and idempotent, so a lost response
/// needs no operation ledger: retry the same selector and reread saved edges. Reads never reconcile.
pub fn execute_memory_journal(
&self,
host: &JournalHostContext,
request: MemoryJournalRequest,
) -> JournalResult<MemoryJournalResponse> {
let response = match request {
MemoryJournalRequest::Reconcile { selector } => {
self.memory_journal_links(host, selector, true, 0, MAX_STAGING_ANCHORS)?
}
MemoryJournalRequest::Links { selector } => {
self.memory_journal_links(host, selector, false, 0, MAX_STAGING_ANCHORS)?
}
MemoryJournalRequest::ReconcilePage {
selector,
source_offset,
limit,
} => self.memory_journal_links(host, selector, true, source_offset, limit)?,
MemoryJournalRequest::LinksPage {
selector,
source_offset,
limit,
} => self.memory_journal_links(host, selector, false, source_offset, limit)?,
MemoryJournalRequest::Read {
selector,
source_index,
journal_id,
entry_id,
expected_token,
byte_offset,
max_bytes,
} => {
identifier(&journal_id)?;
identifier(&entry_id)?;
if !(4..=super::MAX_JOURNAL_READ_BYTES).contains(&max_bytes) {
return Err(invalid("text page must be 4..16384 bytes"));
}
if let Some(token) = expected_token.as_deref() {
identifier(token)?;
}
let mut source = self.database.try_with_connection(|tx| {
let record = candidate(self, tx, host, &selector)?;
if source_index >= record.source_refs.len() {
return Err(invalid("source index outside candidate anchors"));
}
let link = project(tx, host, &selector.candidate_id, source_index)?;
match &link.connection {
SourceConnection::Linked {
journal_id: saved_journal,
entry_id: saved_entry,
..
} if saved_journal == &journal_id && saved_entry == &entry_id => Ok(link),
_ => Err(JournalError::PermissionDenied(
"read requires the exact saved Journal link".into(),
)),
}
})?;
let page = match &source.connection {
SourceConnection::Linked {
target: JournalTargetAvailability::Available,
..
} => {
match self.read_journal_text(
host,
&journal_id,
Some(&entry_id),
JournalTextField::Body,
expected_token.as_deref(),
byte_offset,
max_bytes,
) {
Ok(page) => Some(page),
// Erasure between the saved-edge check and shared text read must not
// fall back, expose cached text, or pretend the link never existed.
Err(JournalError::NotFound(_)) => {
source = self.database.try_with_connection(|tx| {
project(tx, host, &selector.candidate_id, source_index)
})?;
None
}
Err(error) => return Err(error),
}
}
_ => None,
};
MemoryJournalResponse::Read(MemoryJournalRead { source, page })
}
MemoryJournalRequest::Related {
journal_id,
entry_id,
after,
limit,
} => self.related_memories(host, &journal_id, &entry_id, after.as_ref(), limit)?,
};
if serde_json::to_string_pretty(&response)?.len() > MAX_JOURNAL_RESPONSE_BYTES {
return Err(invalid("Memory Journal response exceeds byte budget"));
}
Ok(response)
}
fn memory_journal_links(
&self,
host: &JournalHostContext,
selector: MemoryJournalSelector,
do_reconcile: bool,
source_offset: usize,
limit: usize,
) -> JournalResult<MemoryJournalResponse> {
if !(1..=MAX_STAGING_ANCHORS).contains(&limit) {
return Err(invalid("source page exceeds staging anchor bound"));
}
self.database.try_transaction(|tx| {
let record = candidate(self, tx, host, &selector)?;
let total_sources = record.source_refs.len();
if source_offset > total_sources {
return Err(invalid("source offset outside immutable candidate anchors"));
}
let end = source_offset.saturating_add(limit).min(total_sources);
if do_reconcile {
reconcile(tx, host, &record, source_offset..end)?;
}
let sources = (source_offset..end)
.map(|index| project(tx, host, &selector.candidate_id, index))
.collect::<JournalResult<Vec<_>>>()?;
Ok(MemoryJournalResponse::Links(MemoryJournalLinks {
selector,
sources,
total_sources,
next_source_index: (end < total_sources).then_some(end),
}))
})
}
fn related_memories(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
after: Option<&RelatedMemoryCursor>,
limit: usize,
) -> JournalResult<MemoryJournalResponse> {
identifier(journal_id)?;
identifier(entry_id)?;
if !(1..=MAX_JOURNAL_PAGE).contains(&limit) {
return Err(invalid("related page must be 1..32"));
}
if let Some(cursor) = after {
identifier(&cursor.memory_id)?;
identifier(&cursor.change_id)?;
identifier(&cursor.candidate_id)?;
if i64::try_from(cursor.source_index).is_err() {
return Err(invalid("invalid related source index"));
}
}
self.database.try_with_connection(|tx| {
super::store::check_host(self,tx,host,false)?;
// Saved target identity authorizes reverse lookup after erasure as well as before it.
let known:bool=tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_entries WHERE subject_id=?1 AND journal_id=?2 AND id=?3)
OR EXISTS(SELECT 1 FROM memory_journal_links WHERE subject_id=?1 AND journal_id=?2 AND entry_id=?3)",
params![host.subject_id,journal_id,entry_id],|r|r.get(0))?;
if !known { return Err(JournalError::NotFound("Journal entry".into())); }
let mut stmt=tx.prepare(
"SELECT m.memory_id,m.change_id,l.candidate_id,l.source_index,
EXISTS(SELECT 1 FROM journal_corrections c WHERE c.subject_id=l.subject_id AND c.target_id=l.entry_id)
FROM memory_journal_links l JOIN memory_change_candidates m
ON m.subject_id=l.subject_id AND m.candidate_id=l.candidate_id
WHERE l.subject_id=?1 AND l.reason='linked'
AND ((l.journal_id=?2 AND l.entry_id=?3)
OR EXISTS(SELECT 1 FROM journal_corrections c WHERE c.subject_id=?1 AND c.entry_id=?3 AND c.target_id=l.entry_id))
AND (?4 IS NULL OR (m.memory_id,m.change_id,l.candidate_id,l.source_index)>(?4,?5,?6,?7))
ORDER BY m.memory_id,m.change_id,l.candidate_id,l.source_index LIMIT ?8")?;
let mut items=stmt.query_map(params![host.subject_id,journal_id,entry_id,
after.map(|c|c.memory_id.as_str()),after.map(|c|c.change_id.as_str()),
after.map(|c|c.candidate_id.as_str()),after.map(|c|c.source_index),limit+1],
|r|Ok(RelatedMemory {selector:MemoryJournalSelector {memory_id:r.get(0)?,change_id:r.get(1)?,candidate_id:r.get(2)?},
source_index:r.get(3)?,reconsideration:r.get(4)?}))?.collect::<Result<Vec<_>,_>>()?;
let more=items.len()>limit;
items.truncate(limit);
let next=if more {items.last().map(|last|RelatedMemoryCursor {
memory_id:last.selector.memory_id.clone(),change_id:last.selector.change_id.clone(),
candidate_id:last.selector.candidate_id.clone(),source_index:last.source_index })} else {None};
Ok(MemoryJournalResponse::Related(RelatedMemories {items,next}))
})
}
}
@@ -0,0 +1,33 @@
-- References outlive explicit Journal erasure. Do not add target-row foreign keys:
-- lifecycle erasure deletes Journal content but retains its Session tombstone.
CREATE TABLE memory_journal_links (
subject_id TEXT NOT NULL,
candidate_id TEXT NOT NULL,
source_index INTEGER NOT NULL CHECK(source_index >= 0),
reason TEXT NOT NULL CHECK(reason IN (
'insufficient_source', 'source_not_admitted', 'ambiguous_source',
'journal_not_fixed', 'linked'
)),
journal_id TEXT,
entry_id TEXT,
source_session_id TEXT,
source_order INTEGER,
PRIMARY KEY(subject_id, candidate_id, source_index),
FOREIGN KEY(subject_id, candidate_id)
REFERENCES staging_records(subject_id, candidate_id) ON DELETE RESTRICT,
CHECK((reason = 'linked' AND journal_id IS NOT NULL AND entry_id IS NOT NULL
AND source_session_id IS NOT NULL AND source_order IS NOT NULL AND source_order >= 0)
OR (reason <> 'linked' AND journal_id IS NULL AND entry_id IS NULL
AND source_session_id IS NULL AND source_order IS NULL))
);
CREATE INDEX memory_journal_links_reverse
ON memory_journal_links(subject_id, journal_id, entry_id, candidate_id, source_index);
CREATE TRIGGER memory_journal_links_identity BEFORE UPDATE ON memory_journal_links
WHEN OLD.subject_id <> NEW.subject_id OR OLD.candidate_id <> NEW.candidate_id
OR OLD.source_index <> NEW.source_index
BEGIN SELECT RAISE(ABORT, 'Journal source reference identity is immutable'); END;
CREATE TRIGGER memory_journal_links_connected_immutable BEFORE UPDATE ON memory_journal_links
WHEN OLD.reason = 'linked'
BEGIN SELECT RAISE(ABORT, 'connected Journal references are immutable'); END;
CREATE TRIGGER memory_journal_links_retained BEFORE DELETE ON memory_journal_links
BEGIN SELECT RAISE(ABORT, 'Journal references are retained'); END;
File diff suppressed because it is too large Load Diff
+415
View File
@@ -0,0 +1,415 @@
//! Durable activity records owned by Subjektiv, independent of Session retention.
//!
//! Hosts attest committed public source identity and canonical Session order, not models. Byte
//! ranges are half-open in the Host's canonical UTF-8 public-entry serialization (including
//! framing, so even an empty message has a nonzero length). Compaction copies must retain the
//! original pointer/order/digest; derived summaries are not new sources. Rewind replacement is
//! an identity conflict, never an implicit reset. This module does not fetch or retain history.
//!
//! Each record operation and its exact retry receipt commit together. A separate checkpoint
//! advances only a contiguous, selected prefix backed by saved record/explicit no-change
//! receipts. It cannot use a position update as evidence. Stopping between these boundaries
//! leaves saved records and unreflected input; read receipts before retrying, even across reopen
//! or replacement Jobs. No whole-Job result submission is required. The Host checks live Job
//! execution leases before issuing contexts, and durably closes a generation on cancellation.
//!
//! Selected ranges describe concrete public bytes supplied to generation, not a preview's
//! enclosing history range or a permission to claim unobserved bytes. The Host verifies canonical
//! serialization, SHA-256 identity and UTF-8 slice boundaries before admission/selection. This
//! store validates identities, byte bounds and coverage but cannot attest text it does not fetch.
//! Availability is the latest Host attestation; readers must not equate it with a live fetch.
//! Public text and metadata reads are paged. Full generation/resume snapshots are Host-only
//! construction material bounded by per-field storage caps (three text fields plus current position),
//! not model-facing reads. A restored snapshot never restores an external Job lease.
//!
//! A T-734 caller saves `Create`/`Rewrite`/`Fix` under stable request IDs, carries each returned
//! `state_token` into the next operation, optionally saves `UpdatePosition`, then submits
//! `Checkpoint { record_operation_ids }` for just the reflected prefix. After a lost response it
//! queries the original receipt, status and coverage pages before submitting any new create.
//! On restart it reads the active generation and current resume state, verifies the external Job
//! binding, and continues missing operations; cancellation closes the generation before replacement.
//! A replacement may checkpoint old saved record receipts only for identical selected source bytes.
//! Tokens incorporate prior saved content conditions as well as current content, preventing A→B→A
//! edits from resurrecting an old operation premise after intervening saved effects.
//! Source-ledger pagination uses an opaque, scope/Subject/Journal-bound content cursor: late gap
//! admission, accepted-fragment extension, reflection, or availability changes reject continuation.
//! The fingerprint and bounded projection share one transaction; hashing streams bounded rows.
//! Hosts resuming at the processing cursor can use a bounded source point lookup instead of
//! traversing already reflected pages. Receipt/entry keysets remain append-stable.
pub mod binding;
pub use binding::JournalWriterBinding;
pub(crate) use binding::migrate as migrate_binding;
pub mod browser;
mod lifecycle;
pub mod memory_links;
pub(crate) use lifecycle::migrate as migrate_lifecycle;
pub mod recording;
pub mod service;
mod source_status;
pub mod startup;
mod store;
#[cfg(test)]
mod tests;
pub mod transport;
use crate::{SubjectSessionAttribution, SubjektivStore, api::JobAttemptBinding};
use feature_storage::FeatureStorageError;
use rusqlite::Transaction;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
pub type JournalId = String;
pub type JournalEntryId = String;
pub type JournalResult<T> = Result<T, JournalError>;
pub const MAX_JOURNAL_TEXT_BYTES: usize = 1024 * 1024;
pub const MAX_JOURNAL_RANGES: usize = 64;
pub const MAX_JOURNAL_PAGE: usize = 32;
pub const MAX_JOURNAL_READ_BYTES: usize = 16 * 1024;
/// Whole pretty-serialized public page budget, including escaped text and metadata.
pub const MAX_JOURNAL_RESPONSE_BYTES: usize = 64 * 1024;
/// Stable owner of Backend source retention, shared by attribution-time protection
/// and later source admission/release. This identifier confers no read authority.
/// Local standalone storage uses its scope-exclusive Session UUID as the owner.
pub fn source_retention_pin(workspace: &str, subject: &str, session: &str) -> Uuid {
use sha2::{Digest, Sha256};
let digest = Sha256::digest(
serde_json::to_vec(&(
"subjektiv-journal-retention-v1",
workspace,
subject,
session,
))
.expect("string tuple serialization"),
);
let mut bytes = [0; 16];
bytes.copy_from_slice(&digest[..16]);
Uuid::from_bytes(bytes)
}
#[derive(Debug, thiserror::Error)]
pub enum JournalError {
#[error("journal permission denied: {0}")]
PermissionDenied(String),
#[error("journal conflict: {0}")]
Conflict(String),
#[error("invalid journal input: {0}")]
InvalidInput(String),
#[error("journal record not found: {0}")]
NotFound(String),
#[error(transparent)]
Storage(#[from] FeatureStorageError),
#[error(transparent)]
Json(#[from] serde_json::Error),
#[error(transparent)]
Subject(#[from] crate::SubjektivError),
}
impl From<rusqlite::Error> for JournalError {
fn from(e: rusqlite::Error) -> Self {
Self::Storage(e.into())
}
}
/// Host-issued in-process authority, never a model request DTO. Reader contexts need no live Session.
#[derive(Debug, Clone)]
pub struct JournalHostContext {
scope_id: String,
subject_id: String,
attribution: Option<SubjectSessionAttribution>,
writer_binding_id: Option<String>,
}
impl JournalHostContext {
/// Evidence for a retained Job grant, not permission to mint or renew a binding.
pub fn writer_binding_id(&self) -> Option<&str> {
self.writer_binding_id.as_deref()
}
pub fn validated_reader(store: &SubjektivStore, subject_id: &str) -> JournalResult<Self> {
store
.subject(subject_id)?
.ok_or_else(|| JournalError::NotFound(subject_id.into()))?;
Ok(Self {
scope_id: store.scope_id().into(),
subject_id: subject_id.into(),
attribution: None,
writer_binding_id: None,
})
}
/// External authentication, public-history validation and execution authority are Host duties.
pub fn validated_session(
store: &SubjektivStore,
attribution: SubjectSessionAttribution,
) -> JournalResult<Self> {
if store.session_attribution(&attribution.session_id)?.as_ref() != Some(&attribution) {
return Err(JournalError::PermissionDenied(
"immutable Session attribution mismatch".into(),
));
}
let mut context = Self::validated_reader(store, &attribution.subject_id)?;
use rusqlite::OptionalExtension;
context.writer_binding_id = store.database.try_with_connection(|tx| {
Ok::<_, JournalError>(
tx.query_row(
"SELECT binding_id FROM journal_session_bindings WHERE session_id=?1",
[&attribution.session_id],
|r| r.get(0),
)
.optional()?,
)
})?;
context.attribution = Some(attribution);
Ok(context)
}
}
/// A restricted Journal generation, not Memory consolidation authority.
#[derive(Debug, Clone)]
pub struct JournalJobContext {
host: JournalHostContext,
generation_id: String,
binding: JobAttemptBinding,
}
impl JournalJobContext {
pub fn validated(
store: &SubjektivStore,
host: &JournalHostContext,
generation_id: &str,
binding: JobAttemptBinding,
) -> JournalResult<Self> {
let context = Self {
host: host.clone(),
generation_id: generation_id.into(),
binding,
};
store.validate_journal_job(&context)?;
Ok(context)
}
/// Reconstructs a retained binding for exact completion/receipt recovery. Hosts still check
/// the external attempt lease. New record/checkpoint operations independently require an
/// active generation inside their transaction, so this never revives cancelled authority.
pub fn validated_recovery(
store: &SubjektivStore,
host: &JournalHostContext,
generation_id: &str,
binding: JobAttemptBinding,
) -> JournalResult<Self> {
let context = Self {
host: host.clone(),
generation_id: generation_id.into(),
binding,
};
store.database.try_with_connection(|tx| {
self::store::job(store, tx, &context, false)?;
Ok::<_, JournalError>(())
})?;
Ok(context)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSourcePointer {
pub segment_id: Uuid,
/// Serialized public SessionEntryRef, including its `E` prefix.
pub entry_ref: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSource {
pub session_id: String,
pub pointer: JournalSourcePointer,
/// Host-attested canonical public entry order across Segments, starting at zero.
pub order: u64,
pub content_sha256: String,
pub total_bytes: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalByteRange {
pub start: u64,
pub end: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSourceRange {
pub source: JournalSource,
pub bytes: JournalByteRange,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalEntryState {
Open,
Fixed,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Journal {
pub id: JournalId,
pub subject_id: String,
pub source_session_id: String,
pub title: String,
pub current_position: String,
/// Processing cursor is independent of every fixed event endpoint.
pub process_order: u64,
pub process_byte: u64,
/// Content condition advanced by committed operations, never by acceptance notifications.
pub state_token: String,
pub created_at: String,
pub updated_at: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalMetadata {
pub id: JournalId,
pub subject_id: String,
pub source_session_id: String,
pub title: String,
pub current_position_bytes: usize,
pub process_order: u64,
pub process_byte: u64,
pub state_token: String,
pub created_at: String,
pub updated_at: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalEntry {
pub id: JournalEntryId,
pub journal_id: JournalId,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub body: String,
pub ongoing: String,
pub source_end: Option<JournalSourcePointer>,
pub created_at: String,
pub updated_at: String,
pub fixed_at: Option<String>,
pub correction_refs: Vec<JournalEntryId>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalGeneration {
pub id: String,
pub journal_id: JournalId,
pub job: JobAttemptBinding,
pub selected_ranges: Vec<JournalSourceRange>,
pub initial_journal: Journal,
pub initial_open: Option<JournalEntry>,
pub previous_fixed_end: Option<JournalSourcePointer>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalAction {
Create {
title: String,
body: String,
ongoing: String,
state: JournalEntryState,
source_end: Option<JournalSourcePointer>,
correction_refs: Vec<JournalEntryId>,
},
Rewrite {
entry_id: JournalEntryId,
title: String,
body: String,
ongoing: String,
},
Fix {
entry_id: JournalEntryId,
source_end: JournalSourcePointer,
},
UpdatePosition {
current_position: String,
},
/// Explicitly records the caller's checked conclusion that no entry change is needed.
ConfirmUnchanged,
/// Links durable record/no-change receipts to actual reflected input; IDs are Journal scoped.
Checkpoint {
record_operation_ids: Vec<String>,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalOperation {
pub request_id: String,
pub expected_state: String,
pub action: JournalAction,
pub covered_ranges: Vec<JournalSourceRange>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalReceipt {
pub request_id: String,
pub generation_id: String,
pub entry_id: Option<JournalEntryId>,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalSavedOperation {
pub operation: JournalOperation,
pub receipt: JournalReceipt,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalOperationKind {
Create,
Rewrite,
Fix,
UpdatePosition,
ConfirmUnchanged,
Checkpoint,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalOperationStatus {
pub receipt: JournalReceipt,
pub kind: JournalOperationKind,
pub covered_range_count: usize,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalSourceStatus {
pub source: JournalSource,
pub accepted: Vec<JournalByteRange>,
pub unavailable: Vec<JournalByteRange>,
pub reflected_bytes: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalPage<T> {
pub items: Vec<T>,
pub next: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalEntrySummary {
pub id: JournalEntryId,
pub journal_id: JournalId,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub source_end: Option<JournalSourcePointer>,
/// Exclusive start; None means Session start. The end is inclusive.
pub previous_fixed_end: Option<JournalSourcePointer>,
pub source_available: bool,
pub correction_refs: Vec<JournalEntryId>,
pub content_token: String,
pub created_at: String,
pub updated_at: String,
pub fixed_at: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalTextField {
Body,
Ongoing,
CurrentPosition,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalTextPage {
pub text: String,
pub next_byte: Option<usize>,
pub content_token: String,
}
pub(super) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("schema.sql"))?;
Ok(())
}
+194
View File
@@ -0,0 +1,194 @@
//! Host-side construction of bounded Journal generation input.
//!
//! Admission and byte identity belong to the Session Host. Selection neither starts Jobs nor
//! treats previews, failed fetches, or missing bytes as reflected. Production readers verify
//! original canonical identity while streaming and return only the requested bounded fragment.
use super::*;
use sha2::{Digest, Sha256};
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalMaterial {
pub range: JournalSourceRange,
pub text: String,
}
/// Adapter for Host captures already bounded by their storage read contract. Filesystem/Runtime
/// callers should use [`select_materials_bounded`] rather than allocating a whole large entry.
pub fn select_materials(
store: &SubjektivStore,
host: &JournalHostContext,
journal_id: &str,
max_bytes: usize,
mut read: impl FnMut(&JournalSource) -> JournalResult<String>,
) -> JournalResult<Vec<JournalMaterial>> {
select_materials_bounded(
store,
host,
journal_id,
max_bytes,
|source, start, limit| {
let text = read(source)?;
verify_canonical_text(source, &text)?;
let start = usize::try_from(start).map_err(|_| invalid("source offset overflow"))?;
if !text.is_char_boundary(start) {
return Err(conflict("source offset is not UTF-8 aligned"));
}
let mut end = start.saturating_add(limit).min(text.len());
while !text.is_char_boundary(end) {
end -= 1;
}
Ok(JournalMaterial {
range: JournalSourceRange {
source: source.clone(),
bytes: JournalByteRange {
start: start as u64,
end: end as u64,
},
},
text: text[start..end].into(),
})
},
)
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
fn conflict(message: &str) -> JournalError {
JournalError::Conflict(message.into())
}
/// Selects the oldest contiguous input under a whole serialized materials-array budget.
/// The Host callback verifies original public provenance/digest/length while streaming, and
/// returns at most `limit` canonical UTF-8 bytes at exactly `start`. Its descriptor must equal
/// the admitted source. This in-process attestation must never be deserialized from model input.
pub fn select_materials_bounded(
store: &SubjektivStore,
host: &JournalHostContext,
journal_id: &str,
max_bytes: usize,
mut read: impl FnMut(&JournalSource, u64, usize) -> JournalResult<JournalMaterial>,
) -> JournalResult<Vec<JournalMaterial>> {
let journal = store.read_journal(host, journal_id)?;
let mut order = journal.process_order;
let mut offset = journal.process_byte;
let mut materials = Vec::new();
let mut remaining = max_bytes.saturating_sub(2);
while materials.len() < MAX_JOURNAL_RANGES {
let status = match store.read_journal_source(host, journal_id, order) {
Ok(status) => status,
Err(JournalError::NotFound(_)) => break,
Err(error) => return Err(error),
};
if offset == status.source.total_bytes {
order = order
.checked_add(1)
.ok_or_else(|| invalid("source order exhausted"))?;
offset = 0;
continue;
}
let Some(accepted) = status
.accepted
.iter()
.find(|r| r.start <= offset && offset < r.end)
else {
break;
};
let mut end = accepted.end;
for unavailable in &status.unavailable {
if unavailable.start <= offset && offset < unavailable.end {
end = offset;
break;
}
if offset < unavailable.start {
end = end.min(unavailable.start);
}
}
if end == offset {
break;
}
let empty = JournalMaterial {
range: JournalSourceRange {
source: status.source.clone(),
bytes: JournalByteRange {
start: offset,
end: offset,
},
},
text: String::new(),
};
let envelope = serde_json::to_vec(&empty)?.len() + usize::from(!materials.is_empty());
// Reserve decimal endpoint growth and enough room for one UTF-8 scalar.
let allowance = remaining.saturating_sub(envelope.saturating_add(20));
if allowance < 4 {
if materials.is_empty() {
return Err(invalid(
"input budget cannot fit source metadata and a UTF-8 scalar",
));
}
break;
}
let limit = allowance.min((end - offset).min(usize::MAX as u64) as usize);
let mut material = read(&status.source, offset, limit)?;
if material.range.source != status.source
|| material.range.bytes.start != offset
|| material.range.bytes.end != offset.saturating_add(material.text.len() as u64)
|| material.range.bytes.end > end
|| material.text.is_empty()
|| material.text.len() > limit
{
return Err(conflict(
"Host source chunk does not match the requested canonical byte range",
));
}
let mut allowance = remaining - envelope;
let mut selected_bytes = 0;
for ch in material.text.chars() {
let cost = match ch {
'"' | '\\' | '\u{8}' | '\u{c}' | '\n' | '\r' | '\t' => 2,
ch if ch < '\u{20}' => 6,
ch => ch.len_utf8(),
};
if cost.saturating_add(20) > allowance {
break;
}
allowance -= cost;
selected_bytes += ch.len_utf8();
}
if selected_bytes == 0 {
if materials.is_empty() {
return Err(invalid("input budget cannot fit one escaped UTF-8 scalar"));
}
break;
}
material.text.truncate(selected_bytes);
material.range.bytes.end = offset + selected_bytes as u64;
let cost = serde_json::to_vec(&material)?.len() + usize::from(!materials.is_empty());
if cost > remaining {
return Err(invalid("generation input budget accounting failed"));
}
remaining -= cost;
offset = material.range.bytes.end;
materials.push(material);
if offset < end {
break;
}
}
Ok(materials)
}
/// Verifies the full canonical public role/tool framing of an already bounded Host capture.
pub fn verify_canonical_text(source: &JournalSource, text: &str) -> JournalResult<()> {
if text.len() as u64 != source.total_bytes
|| Sha256::digest(text.as_bytes())
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>()
!= source.content_sha256
{
return Err(conflict("canonical public source bytes changed"));
}
Ok(())
}
+88
View File
@@ -0,0 +1,88 @@
CREATE TABLE journals (
id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
source_session_id TEXT NOT NULL UNIQUE,
title TEXT NOT NULL,
current_position TEXT NOT NULL,
process_order INTEGER NOT NULL DEFAULT 0 CHECK (process_order >= 0),
process_byte INTEGER NOT NULL DEFAULT 0 CHECK (process_byte >= 0),
state_token TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(subject_id, id)
);
CREATE TABLE journal_entries (
id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL,
journal_id TEXT NOT NULL,
position INTEGER NOT NULL CHECK (position >= 0),
state TEXT NOT NULL CHECK (state IN ('open', 'fixed')),
title TEXT NOT NULL,
body TEXT NOT NULL,
ongoing TEXT NOT NULL,
source_end_json TEXT,
source_order INTEGER,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
fixed_at TEXT,
UNIQUE(journal_id, position),
UNIQUE(subject_id, id),
FOREIGN KEY(subject_id, journal_id) REFERENCES journals(subject_id, id) ON DELETE RESTRICT,
CHECK ((state = 'open' AND source_end_json IS NULL AND source_order IS NULL AND fixed_at IS NULL)
OR (state = 'fixed' AND source_end_json IS NOT NULL AND source_order IS NOT NULL AND source_order >= 0 AND fixed_at IS NOT NULL AND ongoing = ''))
);
CREATE UNIQUE INDEX journal_one_open ON journal_entries(journal_id) WHERE state = 'open';
CREATE TRIGGER journal_append_after_open BEFORE INSERT ON journal_entries
WHEN EXISTS (SELECT 1 FROM journal_entries WHERE journal_id = NEW.journal_id AND state = 'open')
BEGIN SELECT RAISE(ABORT, 'journal open must be the tail'); END;
CREATE TRIGGER journal_fixed_immutable BEFORE UPDATE ON journal_entries WHEN OLD.state = 'fixed'
BEGIN SELECT RAISE(ABORT, 'journal fixed entries are immutable'); END;
CREATE TRIGGER journal_entry_identity BEFORE UPDATE ON journal_entries
WHEN OLD.id <> NEW.id OR OLD.subject_id <> NEW.subject_id OR OLD.journal_id <> NEW.journal_id
OR OLD.position <> NEW.position OR OLD.created_at <> NEW.created_at
BEGIN SELECT RAISE(ABORT, 'journal entry identity is immutable'); END;
CREATE TABLE journal_corrections (
subject_id TEXT NOT NULL,
entry_id TEXT NOT NULL,
target_id TEXT NOT NULL,
PRIMARY KEY(entry_id, target_id),
FOREIGN KEY(subject_id, entry_id) REFERENCES journal_entries(subject_id, id) ON DELETE RESTRICT,
FOREIGN KEY(subject_id, target_id) REFERENCES journal_entries(subject_id, id) ON DELETE RESTRICT,
CHECK(entry_id <> target_id)
);
CREATE TABLE journal_sources (
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
source_order INTEGER NOT NULL CHECK(source_order >= 0),
pointer_json TEXT NOT NULL,
entry_ref TEXT NOT NULL,
source_json TEXT NOT NULL,
accepted_json TEXT NOT NULL,
unavailable_json TEXT NOT NULL,
reflected_bytes INTEGER NOT NULL DEFAULT 0 CHECK(reflected_bytes >= 0),
PRIMARY KEY(journal_id, source_order),
UNIQUE(journal_id, pointer_json),
UNIQUE(journal_id, entry_ref)
);
CREATE TABLE journal_generations (
id TEXT PRIMARY KEY,
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
job_id TEXT NOT NULL,
attempt_id TEXT NOT NULL,
request_json TEXT NOT NULL,
snapshot_json TEXT NOT NULL,
active INTEGER NOT NULL CHECK(active IN (0, 1)),
UNIQUE(journal_id, job_id, attempt_id)
);
CREATE UNIQUE INDEX journal_one_generation ON journal_generations(journal_id) WHERE active = 1;
CREATE TABLE journal_operations (
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
request_id TEXT NOT NULL,
generation_id TEXT NOT NULL REFERENCES journal_generations(id) ON DELETE RESTRICT,
request_json TEXT NOT NULL,
receipt_json TEXT NOT NULL,
PRIMARY KEY(journal_id, request_id)
);
CREATE TRIGGER journal_operations_no_update BEFORE UPDATE ON journal_operations
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are immutable'); END;
CREATE TRIGGER journal_operations_no_delete BEFORE DELETE ON journal_operations
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are retained'); END;
+424
View File
@@ -0,0 +1,424 @@
//! Shared sequential Job service. Backend and standalone authenticate and check the live attempt
//! before each call; the store additionally fences closed generations inside mutation transactions.
//! Job completion attests a saved processing checkpoint, not model prose or fixed event state.
use super::transport::*;
use super::*;
use rusqlite::{OptionalExtension, params};
use sha2::{Digest, Sha256};
fn metadata(j: &Journal) -> JournalMetadata {
JournalMetadata {
id: j.id.clone(),
subject_id: j.subject_id.clone(),
source_session_id: j.source_session_id.clone(),
title: j.title.clone(),
current_position_bytes: j.current_position.len(),
process_order: j.process_order,
process_byte: j.process_byte,
state_token: j.state_token.clone(),
created_at: j.created_at.clone(),
updated_at: j.updated_at.clone(),
}
}
fn text_page(
text: &str,
read: &JournalTextRead,
condition: &[u8],
maximum_end: usize,
) -> JournalResult<JournalTextPage> {
if !(1..=MAX_JOURNAL_READ_BYTES).contains(&read.limit)
|| read.offset > maximum_end
|| !text.is_char_boundary(read.offset)
{
return Err(JournalError::InvalidInput(
"invalid Journal UTF-8 read range".into(),
));
}
let token: String = Sha256::digest(condition)
.iter()
.map(|b| format!("{b:02x}"))
.collect();
if read
.content_token
.as_deref()
.is_some_and(|expected| expected != token)
|| (read.offset > 0 && read.content_token.is_none())
{
return Err(JournalError::Conflict(
"Journal text changed or continuation condition missing".into(),
));
}
let mut end = read.offset.saturating_add(read.limit).min(maximum_end);
while !text.is_char_boundary(end) {
end -= 1;
}
if end == read.offset && end < maximum_end {
return Err(JournalError::InvalidInput(
"read limit cannot fit one UTF-8 scalar".into(),
));
}
Ok(JournalTextPage {
text: text[read.offset..end].into(),
next_byte: (end < maximum_end).then_some(end),
content_token: token,
})
}
/// Adapter for Host captures bounded by their storage contract. Production source reads should
/// use [`execute_bounded`] to avoid allocating an entire source for one small read.
pub fn execute(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
mut read_source: impl FnMut(&JournalSource) -> JournalResult<String>,
) -> JournalResult<JournalJobResponse> {
execute_bounded(store, context, request, |source, start, limit| {
let text = read_source(source)?;
super::recording::verify_canonical_text(source, &text)?;
let start = usize::try_from(start)
.map_err(|_| JournalError::InvalidInput("source offset overflow".into()))?;
if !text.is_char_boundary(start) {
return Err(JournalError::InvalidInput(
"source offset is not UTF-8 aligned".into(),
));
}
let mut end = start.saturating_add(limit).min(text.len());
while !text.is_char_boundary(end) {
end -= 1;
}
Ok(recording::JournalMaterial {
range: JournalSourceRange {
source: source.clone(),
bytes: JournalByteRange {
start: start as u64,
end: end as u64,
},
},
text: text[start..end].into(),
})
})
}
/// Reads are delivered through this service before mutation authority is enabled. The durable
/// delivery ledger stores byte ranges only: Jobs may retain immutable descriptors without
/// accumulating a permanent second copy of the original Session in their request ledgers.
/// The Host verifies original provenance/digest/length and returns a bounded UTF-8 fragment.
pub fn execute_bounded(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
mut read_source: impl FnMut(&JournalSource, u64, usize) -> JournalResult<recording::JournalMaterial>,
) -> JournalResult<JournalJobResponse> {
let generation = store
.database
.try_with_connection(|tx| super::store::job(store, tx, context, true))?;
let mut delivered = None;
let response = match request {
JournalJobRequest::Execute { operation } => {
require_delivery(store, context, &generation.selected_ranges)?;
let receipt = if matches!(operation.action, JournalAction::Checkpoint { .. }) {
crate::api::checkpoint_journal(store, context, operation)?
} else {
crate::api::execute_journal(store, context, operation)?
};
JournalJobResponse::Saved(receipt)
}
JournalJobRequest::ReadState => {
let state = store.database.try_with_connection(|tx| {
let generation = super::store::job(store, tx, context, true)?;
let current =
super::store::journal(tx, &context.host, &generation.journal_id, true)?;
let current_open = super::store::open(tx, &generation.journal_id)?;
Ok::<_, JournalError>(JournalJobState {
generation_id: generation.id,
initial_journal: metadata(&generation.initial_journal),
initial_open: generation
.initial_open
.map(|e| super::store::summary(tx, e))
.transpose()?,
current_journal: metadata(&current),
current_open: current_open
.map(|e| super::store::summary(tx, e))
.transpose()?,
previous_fixed_end: generation.previous_fixed_end,
selected_ranges: generation.selected_ranges,
})
})?;
JournalJobResponse::State(state)
}
JournalJobRequest::QueryOperation { request_id } => {
let status = match store.read_journal_operation_status(
&context.host,
&generation.journal_id,
&request_id,
) {
Ok(status) => Some(status),
Err(JournalError::NotFound(_)) => None,
Err(error) => return Err(error),
};
JournalJobResponse::Operation(status)
}
JournalJobRequest::ReadOperationCoverage {
request_id,
offset,
limit,
} => JournalJobResponse::Coverage(store.read_journal_operation_coverage(
&context.host,
&generation.journal_id,
&request_id,
offset,
limit,
)?),
JournalJobRequest::ListOperations { after, limit } => {
let receipts = store.list_journal_operations(
&context.host,
&generation.journal_id,
after.as_deref(),
limit,
)?;
let items = receipts
.items
.iter()
.map(|r| {
store.read_journal_operation_status(
&context.host,
&generation.journal_id,
&r.request_id,
)
})
.collect::<JournalResult<Vec<_>>>()?;
JournalJobResponse::Operations(JournalPage {
items,
next: receipts.next,
})
}
JournalJobRequest::ReadText { read } => {
let text = match &read.target {
JournalTextTarget::Source { pointer } => {
let range = generation
.selected_ranges
.iter()
.find(|r| {
&r.source.pointer == pointer
&& r.bytes.start <= read.offset as u64
&& read.offset as u64 <= r.bytes.end
})
.ok_or_else(|| {
JournalError::PermissionDenied(
"source bytes are outside this generation".into(),
)
})?;
if !(1..=MAX_JOURNAL_READ_BYTES).contains(&read.limit) {
return Err(JournalError::InvalidInput(
"invalid source read limit".into(),
));
}
let condition =
serde_json::to_vec(&(&generation.id, &range.source, range.bytes))?;
let token: String = Sha256::digest(&condition)
.iter()
.map(|b| format!("{b:02x}"))
.collect();
if read
.content_token
.as_deref()
.is_some_and(|expected| expected != token)
|| (read.offset as u64 != range.bytes.start && read.content_token.is_none())
{
return Err(JournalError::Conflict(
"source continuation condition missing or changed".into(),
));
}
let mut material =
read_source(&range.source, read.offset as u64, read.limit.max(4))?;
if material.range.source != range.source
|| material.range.bytes.start != read.offset as u64
|| material.range.bytes.end
!= (read.offset as u64).saturating_add(material.text.len() as u64)
|| material.range.bytes.end > range.source.total_bytes
|| material.text.len() > read.limit.max(4)
{
return Err(JournalError::Conflict(
"source fragment identity or byte range changed".into(),
));
}
let mut length = material
.text
.len()
.min(read.limit)
.min((range.bytes.end - read.offset as u64) as usize);
while !material.text.is_char_boundary(length) {
length -= 1;
}
if length == 0 && (read.offset as u64) < range.bytes.end {
return Err(JournalError::InvalidInput(
"read limit cannot fit a source UTF-8 scalar".into(),
));
}
material.text.truncate(length);
let end = read.offset + length;
if length > 0 {
delivered = Some((
range.source.order,
JournalByteRange {
start: read.offset as u64,
end: end as u64,
},
));
}
JournalTextPage {
text: material.text,
next_byte: ((end as u64) < range.bytes.end).then_some(end),
content_token: token,
}
}
target => {
let (journal, open) = match target {
JournalTextTarget::Position { snapshot }
| JournalTextTarget::OpenBody { snapshot }
| JournalTextTarget::OpenOngoing { snapshot } => match snapshot {
JournalSnapshot::Initial => (
generation.initial_journal.clone(),
generation.initial_open.clone(),
),
JournalSnapshot::Current => {
store.journal_resume_state(&context.host, &generation.journal_id)?
}
},
JournalTextTarget::Source { .. } => unreachable!(),
};
let text = match target {
JournalTextTarget::Position { .. } => journal.current_position.clone(),
JournalTextTarget::OpenBody { .. } => open
.as_ref()
.ok_or_else(|| JournalError::NotFound("no open entry".into()))?
.body
.clone(),
JournalTextTarget::OpenOngoing { .. } => open
.as_ref()
.ok_or_else(|| JournalError::NotFound("no open entry".into()))?
.ongoing
.clone(),
JournalTextTarget::Source { .. } => unreachable!(),
};
let condition = serde_json::to_vec(&(
&generation.id,
&read.target,
&journal.state_token,
&text,
))?;
text_page(&text, &read, &condition, text.len())?
}
};
JournalJobResponse::Text(text)
}
};
if serde_json::to_vec(&response)?.len() > MAX_JOURNAL_RESPONSE_BYTES {
return Err(JournalError::InvalidInput(
"Journal response exceeds bounded tool budget".into(),
));
}
if let Some((order, range)) = delivered {
store.database.try_transaction(|tx| {
// A response that cannot fit the tool budget or loses its live generation must
// not attest bytes. Lost responses can safely reread the identical source page.
super::store::job(store, tx, context, true)?;
let saved: Option<String> = tx.query_row(
"SELECT ranges_json FROM journal_generation_deliveries WHERE generation_id=?1 AND source_order=?2",
params![context.generation_id, order], |row| row.get(0),
).optional()?;
let mut ranges: Vec<JournalByteRange> = saved.map(|raw| serde_json::from_str(&raw)).transpose()?.unwrap_or_default();
ranges.push(range);
tx.execute(
"INSERT INTO journal_generation_deliveries(generation_id,source_order,ranges_json) VALUES(?1,?2,?3) ON CONFLICT(generation_id,source_order) DO UPDATE SET ranges_json=excluded.ranges_json",
params![context.generation_id, order, serde_json::to_string(&super::store::merge(ranges)?)?],
)?;
Ok::<_, JournalError>(())
})?;
}
Ok(response)
}
fn require_delivery(
store: &SubjektivStore,
context: &JournalJobContext,
selected: &[JournalSourceRange],
) -> JournalResult<()> {
store.database.try_with_connection(|tx| {
super::store::job(store, tx, context, true)?;
for range in selected {
let saved: Option<String> = tx.query_row(
"SELECT ranges_json FROM journal_generation_deliveries WHERE generation_id=?1 AND source_order=?2",
params![context.generation_id, range.source.order], |row| row.get(0),
).optional()?;
let delivered: Vec<JournalByteRange> = saved.map(|raw| serde_json::from_str(&raw)).transpose()?.unwrap_or_default();
if !delivered.iter().any(|bytes| bytes.start <= range.bytes.start && range.bytes.end <= bytes.end) {
return Err(JournalError::Conflict(
"read every selected source byte with ReadJournalText before recording or checkpointing".into(),
));
}
}
Ok(())
})
}
/// Atomically saves completion and fences new operations. Exact replay returns the same saved
/// completion even after later generations advance the Journal. Cancelled incomplete generations
/// cannot manufacture completion, and neither fixing an entry nor updating position can substitute
/// for the receipt-backed processing cursor.
pub fn finish(
store: &SubjektivStore,
context: &JournalJobContext,
) -> JournalResult<JournalJobCompletion> {
store.database.try_transaction(|tx| {
let generation = super::store::job(store, tx, context, false)?;
let saved: Option<String> = tx
.query_row(
"SELECT completion_json FROM journal_generations WHERE id=?1",
[&generation.id],
|row| row.get(0),
)
.optional()?
.flatten();
if let Some(saved) = saved {
return Ok(serde_json::from_str(&saved)?);
}
super::store::job(store, tx, context, true)?;
super::binding::require_saved_position(tx, &generation.id)?;
let last = generation
.selected_ranges
.last()
.ok_or_else(|| JournalError::InvalidInput("empty generation selection".into()))?;
let end = if last.bytes.end == last.source.total_bytes {
(
last.source
.order
.checked_add(1)
.ok_or_else(|| JournalError::InvalidInput("source order exhausted".into()))?,
0,
)
} else {
(last.source.order, last.bytes.end)
};
let journal = super::store::journal(tx, &context.host, &generation.journal_id, true)?;
if (journal.process_order, journal.process_byte) != end {
return Err(JournalError::Conflict(
"generation has unreflected selected bytes; save a receipt-backed checkpoint first"
.into(),
));
}
let completion = JournalJobCompletion {
generation_id: generation.id.clone(),
state_token: journal.state_token,
process_order: journal.process_order,
process_byte: journal.process_byte,
};
tx.execute(
"UPDATE journal_generations SET active=0,completion_json=?2 WHERE id=?1",
params![generation.id, serde_json::to_string(&completion)?],
)?;
Ok(completion)
})
}
@@ -0,0 +1,93 @@
//! Optional Host metadata observations for original material, independent of saved Journal reading.
//!
//! Presence checks must not fetch original history or restore a Worker. A failed/unauthorized
//! metadata lookup is unknown, not missing, and cannot prevent a saved Journal body from being read.
use super::memory_links::{
MemoryJournalRequest, MemoryJournalResponse, OriginalSourceAvailability,
};
use super::{JournalHostContext, JournalResult, SubjektivStore};
use rusqlite::{OptionalExtension, params};
impl SubjektivStore {
/// Executes the saved-reference operation first, then annotates original-source absence using
/// Host-owned metadata only. `Some(false)` is confirmed absence; `None` is an unavailable lookup.
/// Presence alone cannot attest that a particular historical entry is still readable.
pub fn execute_memory_journal_with_source_presence(
&self,
host: &JournalHostContext,
request: MemoryJournalRequest,
mut presence: impl FnMut(&str) -> Option<bool>,
) -> JournalResult<MemoryJournalResponse> {
let selector = match &request {
MemoryJournalRequest::Reconcile { selector }
| MemoryJournalRequest::Links { selector }
| MemoryJournalRequest::ReconcilePage { selector, .. }
| MemoryJournalRequest::LinksPage { selector, .. }
| MemoryJournalRequest::Read { selector, .. } => Some(selector.clone()),
MemoryJournalRequest::Related { .. } => None,
};
let mut response = self.execute_memory_journal(host, request)?;
let Some(selector) = selector else {
return Ok(response);
};
// Membership and scoped reader authority were checked by the operation. Candidate JSON is
// immutable; never infer source ownership from its presence without checking attribution.
let candidate = self.staging_candidate(&host.subject_id, &selector.candidate_id)?;
let mut observations = std::collections::HashMap::new();
let mut annotate =
|link: &mut super::memory_links::MemoryJournalSourceLink| -> JournalResult<()> {
let source = candidate
.as_ref()
.and_then(|c| c.source_refs.get(link.source_index));
// Legacy anchors may lack a Session ID. A connected edge already resolved it
// from immutable attribution and stable Segment/entry identity; reuse that saved
// identity rather than fetching history or modifying the original candidate.
let saved_session: Option<String> = self.database.try_with_connection(|tx| {
Ok::<_, super::JournalError>(tx.query_row(
"SELECT source_session_id FROM memory_journal_links
WHERE subject_id=?1 AND candidate_id=?2 AND source_index=?3 AND reason='linked'",
params![host.subject_id, selector.candidate_id, link.source_index],
|row| row.get(0),
).optional()?)
})?;
let session = saved_session
.as_deref()
.or_else(|| source.and_then(|s| s.session_id.as_deref()));
let observed = if let Some(session) = session {
if let Some(value) = observations.get(session) {
*value
} else {
let owned = self
.session_attribution(session)?
.is_some_and(|a| a.subject_id == host.subject_id);
let value = if owned { presence(session) } else { None };
observations.insert(session.to_owned(), value);
value
}
} else {
None
};
link.original_source = match observed {
Some(false) => OriginalSourceAvailability::Missing,
// Preserve explicit saved missing-byte attestations. Directory/registry presence
// does not prove the referenced entry survived rewind or storage corruption.
_ if link.original_source == OriginalSourceAvailability::Missing => {
OriginalSourceAvailability::Missing
}
_ => OriginalSourceAvailability::Unknown,
};
Ok(())
};
match &mut response {
MemoryJournalResponse::Links(page) => {
for source in &mut page.sources {
annotate(source)?;
}
}
MemoryJournalResponse::Read(read) => annotate(&mut read.source)?,
MemoryJournalResponse::Related(_) => {}
}
Ok(response)
}
}
+151
View File
@@ -0,0 +1,151 @@
//! Read-only, bounded Journal context at body startup and compaction boundaries.
//! Selection, text projection, references and processing checkpoints share one storage
//! transaction. Reads neither consult source history/Job liveness nor mutate recording.
//! Startup prefers the Host-retained predecessor; compaction requires a live body writer
//! and prefers its current Session, without waiting for Idle or recording completion.
//! Both supplement only from the same Subject under the same independent budgets.
//! This is derived reference material, never a new observation or an execution capability.
use super::*;
use rusqlite::{Connection, OptionalExtension, params};
pub const STARTUP_MAX_BYTES: usize = 32 * 1024;
const JOURNALS: usize = 3;
const ENTRIES: usize = 3;
const TEXT_BYTES: usize = 2048;
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalStartupSnapshot {
pub subject_id: String,
pub journals: Vec<StartupJournal>,
pub omitted: bool,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct StartupJournal {
pub journal_id: String,
pub source_session_id: String,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
pub current_position: String,
pub entries: Vec<StartupEntry>,
pub omitted: bool,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct StartupEntry {
pub entry_id: String,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub body: String,
pub ongoing: String,
pub source_end: Option<JournalSourcePointer>,
pub correction_refs: Vec<String>,
pub omitted: bool,
}
// Read a bounded UTF-8 prefix at SQLite, not a full megabyte field followed by truncation.
fn text(tx: &Connection, table: &str, id: &str, field: &str) -> JournalResult<(String, bool)> {
let (mut bytes, length): (Vec<u8>, usize) = tx.query_row(
&format!("SELECT COALESCE(substr(CAST({field} AS BLOB),1,?2),X''),length(CAST({field} AS BLOB)) FROM {table} WHERE id=?1"),
params![id, TEXT_BYTES],
|r| Ok((r.get(0)?, r.get(1)?)),
)?;
let length_read = bytes.len();
match std::str::from_utf8(&bytes) {
Ok(_) => {}
Err(error) if error.error_len().is_none() => bytes.truncate(error.valid_up_to()),
Err(_) => {
return Err(JournalError::Conflict(
"stored Journal text is not UTF-8".into(),
));
}
}
let omitted = bytes.len() < length || length_read < length;
Ok((String::from_utf8(bytes).expect("validated UTF-8"), omitted))
}
impl SubjektivStore {
/// `previous_session` comes from the Host's retained predecessor binding, not model input.
/// Recent Subject-local Journals supplement that predecessor under independent fixed budgets.
pub fn journal_startup_snapshot(
&self,
host: &JournalHostContext,
previous_session: Option<&str>,
) -> JournalResult<JournalStartupSnapshot> {
self.context_snapshot(host, previous_session, false)
}
/// Read the saved current-Session Journal without reconciling or advancing it.
/// The active writer check fences released body lifetimes, not non-Idle execution.
pub fn journal_compaction_snapshot(
&self,
host: &JournalHostContext,
) -> JournalResult<JournalStartupSnapshot> {
let session = host.attribution.as_ref().ok_or_else(|| {
JournalError::PermissionDenied("compaction requires a Host-bound body Session".into())
})?;
self.context_snapshot(host, Some(&session.session_id), true)
}
fn context_snapshot(
&self,
host: &JournalHostContext,
preferred_session: Option<&str>,
require_current_writer: bool,
) -> JournalResult<JournalStartupSnapshot> {
self.database.try_transaction(|tx| {
super::store::check_host(self, tx, host, require_current_writer)?;
let mut ids = Vec::new();
if let Some(session) = preferred_session {
let previous: Option<(String, String)> = tx.query_row(
"SELECT id,subject_id FROM journals WHERE source_session_id=?1",
[session], |r| Ok((r.get(0)?, r.get(1)?)),
).optional()?;
if let Some((id, subject)) = previous {
if subject != host.subject_id { return Err(JournalError::PermissionDenied("predecessor Journal belongs to another Subject".into())); }
ids.push(id);
}
}
let mut query = tx.prepare("SELECT id FROM journals WHERE subject_id=?1 ORDER BY updated_at DESC,id DESC LIMIT ?2")?;
let recent = query.query_map(params![host.subject_id, JOURNALS + 1], |r| r.get::<_, String>(0))?;
for id in recent { let id = id?; if !ids.contains(&id) { ids.push(id); } }
let omitted = ids.len() > JOURNALS;
ids.truncate(JOURNALS);
let mut snapshot = JournalStartupSnapshot { subject_id: host.subject_id.clone(), journals: Vec::new(), omitted };
for id in ids {
let (session, token, order, byte) = tx.query_row(
"SELECT source_session_id,state_token,process_order,process_byte FROM journals WHERE id=?1 AND subject_id=?2",
params![id, host.subject_id], |r| Ok((r.get(0)?,r.get(1)?,r.get(2)?,r.get(3)?)),
)?;
let (current_position, mut omitted) = text(tx, "journals", &id, "current_position")?;
let mut stmt = tx.prepare("SELECT id,position,state,source_end_json FROM journal_entries WHERE journal_id=?1 ORDER BY position DESC LIMIT ?2")?;
let rows = stmt.query_map(params![id, ENTRIES + 1], |r| Ok((r.get::<_,String>(0)?,r.get::<_,u64>(1)?,r.get::<_,String>(2)?,r.get::<_,Option<String>>(3)?)))?.collect::<Result<Vec<_>,_>>()?;
omitted |= rows.len() > ENTRIES;
let mut entries = Vec::new();
for (entry_id, position, state, source_end) in rows.into_iter().take(ENTRIES) {
let (title, title_cut) = text(tx, "journal_entries", &entry_id, "title")?;
let (body, body_cut) = text(tx, "journal_entries", &entry_id, "body")?;
let (ongoing, ongoing_cut) = text(tx, "journal_entries", &entry_id, "ongoing")?;
let mut refs = tx.prepare("SELECT target_id FROM journal_corrections WHERE entry_id=?1 ORDER BY target_id LIMIT 17")?;
let mut correction_refs = refs.query_map([&entry_id], |r| r.get::<_,String>(0))?.collect::<Result<Vec<_>,_>>()?;
let omitted = title_cut || body_cut || ongoing_cut || correction_refs.len() > 16;
correction_refs.truncate(16);
entries.push(StartupEntry { entry_id, position, state: match state.as_str() { "open" => JournalEntryState::Open, "fixed" => JournalEntryState::Fixed, _ => return Err(JournalError::Conflict("invalid stored Journal entry state".into())) }, title, body, ongoing, source_end: source_end.map(|s| serde_json::from_str(&s)).transpose()?, correction_refs, omitted });
}
snapshot.journals.push(StartupJournal { journal_id: id, source_session_id: session, state_token: token, process_order: order, process_byte: byte, current_position, entries, omitted });
}
// JSON escaping also consumes the independent output budget. Drop least-priority
// entries/Journals rather than returning an over-budget or silently partial document.
while serde_json::to_vec(&snapshot)?.len() > STARTUP_MAX_BYTES {
snapshot.omitted = true;
let journal = snapshot.journals.last_mut().ok_or_else(|| JournalError::InvalidInput("startup metadata exceeds budget".into()))?;
if journal.entries.pop().is_some() { journal.omitted = true; } else { snapshot.journals.pop(); }
}
Ok(snapshot)
})
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+135
View File
@@ -0,0 +1,135 @@
//! Bounded, Host-bound Journal recording and sequential Job transport.
//!
//! Subject, Journal and generation authority never come from these model inputs.
//! Hosts validate live execution and public source bytes before selection; reading a
//! preview must not select the enclosing entry. Operation retries retain the entire
//! JournalOperation, including its original expected_state and covered_ranges.
use serde::{Deserialize, Serialize};
use super::{
JournalEntrySummary, JournalMetadata, JournalOperation, JournalOperationStatus, JournalPage,
JournalReceipt, JournalSourcePointer, JournalSourceRange, JournalTextPage,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalRecordingReason {
RunCommitted,
TaskCompleted,
BeforeCompact,
WorkerRestored,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalCompactionRequest {
pub session_id: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalRecordingRequest {
pub reason: JournalRecordingReason,
}
/// Admission is durable Host evidence, not proof that generation finished.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "status", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalRecordingResponse {
NoNewSources,
Accepted,
}
/// No unbounded initial/current body or position is included in model state.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalJobState {
pub generation_id: String,
pub initial_journal: JournalMetadata,
pub initial_open: Option<JournalEntrySummary>,
pub current_journal: JournalMetadata,
pub current_open: Option<JournalEntrySummary>,
pub previous_fixed_end: Option<JournalSourcePointer>,
/// Frozen permission to fetch this prefix, not evidence of delivery or reflection.
/// The sequential service gates mutations on its durable Source-read delivery ledger.
pub selected_ranges: Vec<JournalSourceRange>,
}
/// Host-verified saved progress only; not an aggregate record submission.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalJobCompletion {
pub generation_id: String,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalSnapshot {
Initial,
Current,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "target", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalTextTarget {
Position {
snapshot: JournalSnapshot,
},
OpenBody {
snapshot: JournalSnapshot,
},
OpenOngoing {
snapshot: JournalSnapshot,
},
/// The Host restricts this pointer to this Job's frozen public source input.
Source {
pointer: JournalSourcePointer,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalTextRead {
pub target: JournalTextTarget,
pub offset: usize,
pub limit: usize,
/// Required continuation condition; None only for the first chunk.
pub content_token: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalJobRequest {
Execute {
operation: JournalOperation,
},
ReadState,
QueryOperation {
request_id: String,
},
ReadOperationCoverage {
request_id: String,
offset: usize,
limit: usize,
},
ListOperations {
after: Option<String>,
limit: usize,
},
ReadText {
read: JournalTextRead,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "result", content = "value", rename_all = "snake_case")]
pub enum JournalJobResponse {
Saved(JournalReceipt),
State(JournalJobState),
Operation(Option<JournalOperationStatus>),
Coverage(JournalPage<JournalSourceRange>),
Operations(JournalPage<JournalOperationStatus>),
Text(JournalTextPage),
}
+27 -1
View File
@@ -929,7 +929,7 @@ mod tests {
let (_manager, _workspace, store) = open_store(&root, "workspace-a");
let migrated_subject = store.subject(&subject_id).unwrap().unwrap();
assert!(migrated_subject.behavior_md.is_empty());
assert!(migrated_subject.behavior_md.is_empty());
assert!(store.subject_permissions(&subject_id).unwrap().is_none());
let lifecycle_attribution = attribution(
&subject_id,
"runtime-1",
@@ -953,6 +953,32 @@ mod tests {
store.session_attribution("session-1").unwrap().unwrap(),
stored_attribution
);
let binding = store
.claim_journal_writer(&subject_id, "runtime-1", "worker-1")
.unwrap();
let host = store
.bind_journal_session(&binding, stored_attribution.clone())
.unwrap();
let range = crate::journal::JournalSourceRange {
source: crate::journal::JournalSource {
session_id: stored_attribution.session_id.clone(),
pointer: crate::journal::JournalSourcePointer {
segment_id: Uuid::now_v7(),
entry_ref: "Eafter-v1-migration".into(),
},
order: 0,
content_sha256: "ab".repeat(32),
total_bytes: 8,
},
bytes: crate::journal::JournalByteRange { start: 0, end: 8 },
};
let journal_id = store
.accept_journal_sources(&host, "After oldest supported migration", &[range])
.unwrap();
assert_eq!(
store.read_journal(&host, &journal_id).unwrap().subject_id,
subject_id
);
}
#[test]
+37
View File
@@ -2,7 +2,9 @@
pub mod api;
pub mod job;
pub mod journal;
mod legacy_migrations;
pub mod permissions;
mod schema;
use legacy_migrations::{
add_candidate_decision_receipts, add_subject_behavior, add_subject_session_attribution,
@@ -81,6 +83,16 @@ pub enum SubjektivError {
},
#[error("subject `{subject_id}` behavior content changed")]
SubjectBehaviorConflict { subject_id: String },
#[error("subject `{0}` permission source changed")]
SubjectPermissionsConflict(String),
#[error("permission operation `{0}` was reused with different input")]
PermissionOperationConflict(String),
#[error("authorization denied: {0}")]
AuthorizationDenied(String),
#[error("Worker execution grant binding conflicts for `{0}`")]
WorkerExecutionGrantConflict(String),
#[error("Subject `{subject_id}` has more than {limit} retained Worker execution grants")]
WorkerExecutionGrantListOverflow { subject_id: String, limit: usize },
#[error("surface generation conflict: {0}")]
SurfaceGenerationConflict(String),
#[error("memory `{memory_id}` cannot transition from {from:?} to {to:?}")]
@@ -602,6 +614,31 @@ static MIGRATIONS: &[FeatureMigration] = &[
"replace counters with immutable changes and content conditions",
schema::migrate,
),
FeatureMigration::new(
8,
"add durable journal operations and processing ranges",
journal::migrate,
),
FeatureMigration::new(
9,
"fence erased journal Sessions and permit explicit content erasure",
journal::migrate_lifecycle,
),
FeatureMigration::new(
10,
"fence journal writers by explicit body binding lifetime",
journal::migrate_binding,
),
FeatureMigration::new(
11,
"add immutable candidate Journal context links",
journal::memory_links::migrate,
),
FeatureMigration::new(
12,
"add explicit Subject permission sources and immutable child execution grants",
permissions::migrate,
),
];
pub const REGISTRATION: FeatureRegistration =
+288
View File
@@ -0,0 +1,288 @@
//! Host-administered permission sources and immutable child grants, separate from behavior.
//!
//! The Host authenticates administrators before source writes and binds verified Worker/Profile
//! identities before grant writes. These records are not a Worker registry. The Host alone
//! authenticates root execution ownership against its canonical Backend singleton and checks
//! every live lineage edge; Journal bindings and retained grants cannot establish ownership.
//! Replays return historical evidence without restoring a source or granting current authority.
use super::*;
use server_api::authorization::{
PermissionResource, ScopedPermission, SubjectPermissions, SubjectPermissionsUpdate,
WorkerExecutionGrant, validate_authorization_identifier, validate_scoped_permissions,
};
use sha2::{Digest, Sha256};
pub const MAX_SUBJECT_WORKER_EXECUTION_GRANTS: usize = 1_000;
pub(super) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("permissions.sql"))?;
Ok(())
}
impl SubjektivStore {
/// Unset sources are `None` (deny), including migrated Subjects. This is an admin read,
/// not an effective-authorization decision; retired Subjects retain their source evidence.
pub fn subject_permissions(&self, subject_id: &str) -> Result<Option<SubjectPermissions>> {
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
source_in_connection(connection, subject_id)
})
}
/// Host-admin-only replacement. Every replacement issues a fresh change ID, including
/// identical contents. Revocation is an explicit empty replacement. A scope-wide operation
/// ID binds Subject, CAS input, and both permission vectors; result and receipt commit together.
pub fn set_subject_permissions(
&self,
subject_id: &str,
update: &SubjectPermissionsUpdate,
) -> Result<SubjectPermissions> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(&update.operation_id)
.map_err(SubjektivError::InvalidRecord)?;
if let Some(change) = &update.expected_change_id {
validate_authorization_identifier(change).map_err(SubjektivError::InvalidRecord)?;
}
validate_scope(self.scope_id(), &update.execution)?;
validate_scope(self.scope_id(), &update.delegation)?;
let fingerprint: String = Sha256::digest(serde_json::to_vec(&(subject_id, update))?)
.iter()
.map(|byte| format!("{byte:02x}"))
.collect();
self.database.try_transaction(|tx| {
require_subject(tx, subject_id)?;
let receipt: Option<(String, String)> = tx.query_row(
"SELECT request_fingerprint, change_id FROM subject_permission_receipts WHERE operation_id=?1",
[&update.operation_id], |row| Ok((row.get(0)?, row.get(1)?)),
).optional()?;
if let Some((saved, change_id)) = receipt {
if saved != fingerprint {
return Err(SubjektivError::PermissionOperationConflict(update.operation_id.clone()));
}
let raw: String = tx.query_row(
"SELECT record_json FROM subject_permission_changes WHERE subject_id=?1 AND change_id=?2",
params![subject_id, change_id], |row| row.get(0),
)?;
return Ok(serde_json::from_str(&raw)?);
}
require_active_subject(tx, subject_id)?;
let current = source_in_connection(tx, subject_id)?;
if current.as_ref().map(|source| &source.change_id) != update.expected_change_id.as_ref() {
return Err(SubjektivError::SubjectPermissionsConflict(subject_id.into()));
}
let source = SubjectPermissions {
change_id: issued_id("permissions"),
execution: update.execution.clone(),
delegation: update.delegation.clone(),
};
tx.execute("INSERT INTO subject_permission_changes(subject_id,change_id,record_json) VALUES(?1,?2,?3)",
params![subject_id, source.change_id, serde_json::to_string(&source)?])?;
tx.execute("INSERT INTO subject_permissions(subject_id,change_id) VALUES(?1,?2) ON CONFLICT(subject_id) DO UPDATE SET change_id=excluded.change_id",
params![subject_id, source.change_id])?;
tx.execute("INSERT INTO subject_permission_receipts(operation_id,subject_id,request_fingerprint,change_id) VALUES(?1,?2,?3,?4)",
params![update.operation_id, subject_id, fingerprint, source.change_id])?;
Ok(source)
})
}
/// Binds child grants only. Exact retries recover an immutable binding even after revocation;
/// that success is not authorization to execute or redelegate. New bindings require the
/// current source, an active Subject and explicit delegation containment. The Host must
/// authenticate root singleton ownership and every live lineage edge before calling;
/// a missing retained parent is not ownership evidence. Profile resolution, provider scope,
/// Worker creation and caller authentication also remain Host duties.
pub fn bind_worker_execution_grant(&self, grant: &WorkerExecutionGrant) -> Result<()> {
validate_grant(self.scope_id(), grant)?;
self.database.try_transaction(|tx| {
if let Some(existing) = grant_in_connection(tx, &grant.runtime_id, &grant.worker_id)? {
return if existing == *grant { Ok(()) } else {
Err(SubjektivError::WorkerExecutionGrantConflict(format!("{}/{}", grant.runtime_id, grant.worker_id)))
};
}
require_active_subject(tx, &grant.subject_id)?;
let source = source_in_connection(tx, &grant.subject_id)?
.ok_or_else(|| denied("Subject permissions are unset"))?;
if source.change_id != grant.source_change_id {
return Err(denied("Subject permission source is no longer current"));
}
require_covered(&source.delegation, &grant.permissions)?;
require_covered(&source.delegation, &grant.redelegation)?;
// A child grant cannot turn a body (including a released historical body) into a
// delegated child or attach a Worker historically attributed to a different Subject.
let invalid_target: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![grant.runtime_id, grant.worker_id, grant.subject_id], |row| row.get(0),
)?;
if invalid_target { return Err(denied("target is a body or belongs to another Subject")); }
let origin = require_retained_origin(tx, grant)?;
if let Some(origin) = origin {
require_covered(&origin.redelegation, &grant.permissions)?;
require_covered(&origin.redelegation, &grant.redelegation)?;
}
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![grant.runtime_id, grant.worker_id, grant.subject_id, grant.source_change_id, serde_json::to_string(grant)?])?;
Ok(())
})
}
/// Returns this Subject's retained grants, including stale grants needed for cancellation
/// and reporting. Never an ownership list or effective-authorization decision. Overflow is
/// explicit rather than silently omitting Workers from a cancellation/reporting pass.
pub fn list_subject_worker_execution_grants(
&self,
subject_id: &str,
) -> Result<Vec<WorkerExecutionGrant>> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
let mut statement = connection.prepare(
"SELECT record_json FROM worker_execution_grants WHERE subject_id=?1 ORDER BY runtime_id,worker_id LIMIT ?2",
)?;
let mut rows = statement.query(params![subject_id, MAX_SUBJECT_WORKER_EXECUTION_GRANTS + 1])?;
let mut grants = Vec::new();
while let Some(row) = rows.next()? {
if grants.len() == MAX_SUBJECT_WORKER_EXECUTION_GRANTS {
return Err(SubjektivError::WorkerExecutionGrantListOverflow {
subject_id: subject_id.into(),
limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS,
});
}
grants.push(serde_json::from_str(&row.get::<_, String>(0)?)?);
}
Ok(grants)
})
}
/// Returns retained binding evidence, not current authorization. The Host must compare the
/// active Subject, source change, current origin ownership, Profile and provider constraints.
pub fn worker_execution_grant(
&self,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
validate_authorization_identifier(runtime).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(worker).map_err(SubjektivError::InvalidRecord)?;
self.database
.try_with_connection(|connection| grant_in_connection(connection, runtime, worker))
}
}
fn source_in_connection(
connection: &Connection,
subject: &str,
) -> Result<Option<SubjectPermissions>> {
let raw: Option<String> = connection.query_row(
"SELECT c.record_json FROM subject_permissions p JOIN subject_permission_changes c ON c.subject_id=p.subject_id AND c.change_id=p.change_id WHERE p.subject_id=?1",
[subject], |row| row.get(0),
).optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn grant_in_connection(
connection: &Connection,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
let raw: Option<String> = connection
.query_row(
"SELECT record_json FROM worker_execution_grants WHERE runtime_id=?1 AND worker_id=?2",
params![runtime, worker],
|row| row.get(0),
)
.optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn validate_scope(scope: &str, permissions: &[ScopedPermission]) -> Result<()> {
validate_scoped_permissions(permissions).map_err(SubjektivError::InvalidRecord)?;
for permission in permissions {
if let PermissionResource::Workspace { workspace_id } = &permission.resource
&& workspace_id != scope
{
return Err(denied("Workspace permission is outside this store scope"));
}
}
Ok(())
}
fn validate_grant(scope: &str, grant: &WorkerExecutionGrant) -> Result<()> {
for value in [
&grant.subject_id,
&grant.origin_runtime_id,
&grant.origin_worker_id,
&grant.source_change_id,
&grant.runtime_id,
&grant.worker_id,
&grant.profile,
&grant.profile_digest,
] {
validate_authorization_identifier(value).map_err(SubjektivError::InvalidRecord)?;
}
if grant.origin_runtime_id == grant.runtime_id && grant.origin_worker_id == grant.worker_id {
return Err(denied("a child grant cannot originate from itself"));
}
validate_scope(scope, &grant.permissions)?;
validate_scope(scope, &grant.redelegation)
}
fn require_covered(ceiling: &[ScopedPermission], requested: &[ScopedPermission]) -> Result<()> {
if requested
.iter()
.all(|permission| ceiling.iter().any(|allowed| allowed.covers(permission)))
{
Ok(())
} else {
Err(denied(
"requested permission or redelegation exceeds its explicit ceiling",
))
}
}
fn require_retained_origin(
tx: &Transaction<'_>,
grant: &WorkerExecutionGrant,
) -> Result<Option<WorkerExecutionGrant>> {
let mut runtime = grant.origin_runtime_id.clone();
let mut worker = grant.origin_worker_id.clone();
let mut direct_origin = None;
let mut visited = HashSet::from([(grant.runtime_id.clone(), grant.worker_id.clone())]);
// Validate retained lineage only. Absence terminates at a Host-authenticated root;
// neither Journal active state nor absence of a grant proves current singleton ownership.
for _ in 0..64 {
if !visited.insert((runtime.clone(), worker.clone())) {
return Err(denied("cyclic grant origin"));
}
let foreign_attribution: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![runtime, worker, grant.subject_id], |row| row.get(0),
)?;
if foreign_attribution {
return Err(denied("grant origin is attributed to another Subject"));
}
let Some(origin) = grant_in_connection(tx, &runtime, &worker)? else {
return Ok(direct_origin);
};
if origin.subject_id != grant.subject_id
|| origin.source_change_id != grant.source_change_id
{
return Err(denied(
"grant origin belongs to another Subject or permission source",
));
}
if direct_origin.is_none() {
direct_origin = Some(origin.clone());
}
runtime = origin.origin_runtime_id;
worker = origin.origin_worker_id;
}
Err(denied("grant origin exceeds maximum delegation depth"))
}
fn denied(reason: &str) -> SubjektivError {
SubjektivError::AuthorizationDenied(reason.into())
}
#[cfg(test)]
mod tests;
+45
View File
@@ -0,0 +1,45 @@
CREATE TABLE subject_permission_changes (
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL CHECK (length(change_id) > 0),
record_json TEXT NOT NULL,
PRIMARY KEY (subject_id, change_id)
);
CREATE TABLE subject_permissions (
subject_id TEXT PRIMARY KEY REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE subject_permission_receipts (
operation_id TEXT PRIMARY KEY CHECK (length(operation_id) > 0),
subject_id TEXT NOT NULL,
request_fingerprint TEXT NOT NULL,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE worker_execution_grants (
runtime_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
source_change_id TEXT NOT NULL,
record_json TEXT NOT NULL,
PRIMARY KEY (runtime_id, worker_id),
FOREIGN KEY (subject_id, source_change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TRIGGER subject_permission_changes_no_update BEFORE UPDATE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are immutable');
END;
CREATE TRIGGER subject_permission_changes_no_delete BEFORE DELETE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are retained');
END;
CREATE TRIGGER subject_permission_receipts_no_update BEFORE UPDATE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are immutable');
END;
CREATE TRIGGER subject_permission_receipts_no_delete BEFORE DELETE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are retained');
END;
CREATE TRIGGER worker_execution_grants_no_update BEFORE UPDATE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are immutable');
END;
CREATE TRIGGER worker_execution_grants_no_delete BEFORE DELETE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are retained');
END;
+656
View File
@@ -0,0 +1,656 @@
//! Real SQLite component tests through store APIs; Host authentication, canonical singleton
//! ownership and live lineage checks are outside this suite. Root origins are Host-attested inputs;
//! Journal fixtures exercise defensive attribution only, never execution authority.
use super::*;
use crate::tests::{open_store, role};
use feature_storage::FeatureStorage;
use std::path::Path;
fn permission(operation: &str, path: &str, recursive: bool) -> ScopedPermission {
ScopedPermission {
operation: operation.into(),
resource: PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
},
}
}
fn update(
operation: &str,
expected: Option<&str>,
delegation: Vec<ScopedPermission>,
) -> SubjectPermissionsUpdate {
SubjectPermissionsUpdate {
operation_id: operation.into(),
expected_change_id: expected.map(Into::into),
execution: Vec::new(),
delegation,
}
}
fn grant(subject: &str, source: &str, worker: &str) -> WorkerExecutionGrant {
WorkerExecutionGrant {
subject_id: subject.into(),
origin_runtime_id: "runtime".into(),
origin_worker_id: "body".into(),
source_change_id: source.into(),
runtime_id: "runtime".into(),
worker_id: worker.into(),
profile: "builtin:coder".into(),
profile_digest: "resolved-profile-sha256".into(),
permissions: vec![permission("file.read", "src", false)],
redelegation: Vec::new(),
}
}
fn schema(store: &SubjektivStore) -> Vec<(String, String)> {
store.database.try_with_connection::<_, SubjektivError>(|connection| {
let mut statement = connection.prepare("SELECT name,sql FROM sqlite_master WHERE name NOT LIKE 'sqlite_%' AND sql IS NOT NULL ORDER BY name")?;
Ok(statement.query_map([], |row| Ok((row.get(0)?, row.get(1)?)))?.collect::<std::result::Result<_, _>>()?)
}).unwrap()
}
fn open_version(
root: &Path,
version: usize,
) -> (FeatureStorage, ScopedFeatureStorage, SubjektivStore) {
let manager = FeatureStorage::new(root);
let scope = manager.scope("workspace-a").unwrap();
let registration = scope
.register(FeatureRegistration::new(
SUBJEKTIV_FEATURE_ID,
&MIGRATIONS[..version],
))
.unwrap();
let store = SubjektivStore::open(&scope, &registration).unwrap();
(manager, scope, store)
}
#[test]
fn replacements_are_cas_fenced_replayable_and_never_restore_old_permissions() {
let temp = tempfile::tempdir().unwrap();
let subject;
let first;
let first_update = update("allow", None, vec![permission("file.read", ".", true)]);
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
subject = store.create_subject(role()).unwrap().id;
assert!(store.subject_permissions(&subject).unwrap().is_none());
first = store
.set_subject_permissions(&subject, &first_update)
.unwrap();
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert!(matches!(
store.set_subject_permissions(&subject, &update("stale", None, Vec::new())),
Err(SubjektivError::SubjectPermissionsConflict(_))
));
assert!(matches!(
store.set_subject_permissions(&subject, &update("allow", None, Vec::new())),
Err(SubjektivError::PermissionOperationConflict(_))
));
let other = store.create_subject(role()).unwrap();
assert!(matches!(
store.set_subject_permissions(&other.id, &first_update),
Err(SubjektivError::PermissionOperationConflict(_))
));
let same = store
.set_subject_permissions(
&subject,
&update("same", Some(&first.change_id), first.delegation.clone()),
)
.unwrap();
assert_ne!(same.change_id, first.change_id);
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&same.change_id), Vec::new()),
)
.unwrap();
assert!(revoked.delegation.is_empty());
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(revoked));
}
let (_reopened_manager, _reopened_scope, reopened) = open_store(temp.path(), "workspace-a");
assert!(
reopened
.subject_permissions(&subject)
.unwrap()
.unwrap()
.delegation
.is_empty()
);
assert_eq!(
reopened
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
}
#[test]
fn failed_receipt_insert_rolls_back_source_history_and_projection() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let first = store
.set_subject_permissions(&subject, &update("first", None, Vec::new()))
.unwrap();
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_permission_receipt BEFORE INSERT ON subject_permission_receipts BEGIN SELECT RAISE(ABORT,'injected receipt failure'); END;")?;
Ok(())
}).unwrap();
let replacement = update(
"second",
Some(&first.change_id),
vec![permission("file.read", ".", true)],
);
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_err()
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(first));
store.database.try_transaction::<_, SubjektivError>(|tx| {
let counts: (u32, u32) = tx.query_row("SELECT (SELECT count(*) FROM subject_permission_changes),(SELECT count(*) FROM subject_permission_receipts)", [], |r| Ok((r.get(0)?, r.get(1)?)))?;
assert_eq!(counts, (1, 1));
tx.execute_batch("DROP TRIGGER fail_permission_receipt")?;
Ok(())
}).unwrap();
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_ok()
);
}
#[test]
fn scope_and_validation_rejections_do_not_issue_permission_sources() {
let temp = tempfile::tempdir().unwrap();
let (_a_manager, _a_scope, a) = open_store(&temp.path().join("a"), "workspace-a");
let (_b_manager, _b_scope, b) = open_store(&temp.path().join("b"), "workspace-b");
let subject = a.create_subject(role()).unwrap().id;
assert!(
b.set_subject_permissions(&subject, &update("cross-subject", None, Vec::new()))
.is_err()
);
let cross_workspace = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "workspace-b".into(),
},
};
assert!(
a.set_subject_permissions(
&subject,
&update("cross-workspace", None, vec![cross_workspace])
)
.is_err()
);
for (operation, path) in [
("file.*", "src"),
("file..read", "src"),
("file.read", "../src"),
("file.read", "src/../escape"),
] {
assert!(
a.set_subject_permissions(
&subject,
&update("invalid", None, vec![permission(operation, path, true)])
)
.is_err()
);
}
assert!(a.subject_permissions(&subject).unwrap().is_none());
}
#[test]
fn child_grants_are_immutable_bounded_and_do_not_create_subject_ownership() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", "src", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&child).unwrap();
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child.clone())
);
assert!(
store
.journal_body_is_unpublished(&subject, "runtime", "child")
.unwrap()
);
let mut changed = child.clone();
changed.profile_digest = "changed-profile".into();
assert!(matches!(
store.bind_worker_execution_grant(&changed),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
for (worker, requested) in [
("write", permission("file.write", "src", false)),
("escape", permission("file.read", ".", true)),
("prefix", permission("file.read", "src-other", false)),
] {
let mut candidate = grant(&subject, &source.change_id, worker);
candidate.permissions = vec![requested];
assert!(store.bind_worker_execution_grant(&candidate).is_err());
assert!(
store
.worker_execution_grant("runtime", worker)
.unwrap()
.is_none()
);
}
let mut self_origin = child.clone();
self_origin.worker_id = "body".into();
assert!(store.bind_worker_execution_grant(&self_origin).is_err());
let other = store.create_subject(role()).unwrap();
store
.claim_journal_writer(&other.id, "runtime", "other-body")
.unwrap();
let mut invalid = grant(&subject, &source.change_id, "other-body");
assert!(store.bind_worker_execution_grant(&invalid).is_err());
invalid.worker_id = "forged-child".into();
invalid.origin_worker_id = "other-body".into();
assert!(store.bind_worker_execution_grant(&invalid).is_err());
}
#[test]
fn redelegation_requires_current_source_and_explicit_retained_parent_ceiling() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = vec![permission("file.read", "src", true)];
store.bind_worker_execution_grant(&child).unwrap();
let mut descendant = grant(&subject, &source.change_id, "grandchild");
descendant.origin_worker_id = "child".into();
store.bind_worker_execution_grant(&descendant).unwrap();
let mut excessive = descendant.clone();
excessive.worker_id = "excessive".into();
excessive.redelegation = vec![permission("file.read", ".", true)];
assert!(store.bind_worker_execution_grant(&excessive).is_err());
let mut nondelegator = descendant.clone();
nondelegator.origin_worker_id = "grandchild".into();
nondelegator.worker_id = "great-grandchild".into();
assert!(store.bind_worker_execution_grant(&nondelegator).is_err());
let mut stale = descendant.clone();
stale.worker_id = "after-source-replacement".into();
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
assert_ne!(revoked.change_id, source.change_id);
stale.origin_worker_id = "new-body".into();
assert!(store.bind_worker_execution_grant(&stale).is_err());
// Exact retries/readback recover evidence but cannot revive the source or origin.
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
store.retire_subject(&subject).unwrap();
stale.source_change_id = revoked.change_id;
assert!(store.bind_worker_execution_grant(&stale).is_err());
}
#[test]
fn journal_state_neither_authenticates_roots_nor_bypasses_retained_child_ceilings() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
// The Host has authenticated root ownership outside this store. No Journal record is
// required; changing Journal writer state cannot supply or invalidate that authentication.
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "without-journal"))
.unwrap();
let journal = store
.claim_journal_writer(&subject, "runtime", "body")
.unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-active-journal"))
.unwrap();
store.release_journal_writer(&journal).unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-released-journal"))
.unwrap();
// An active Journal binding must never upgrade a child's empty redelegation ceiling.
store
.claim_journal_writer(&subject, "runtime", "without-journal")
.unwrap();
let mut descendant = grant(&subject, &source.change_id, "descendant");
descendant.origin_worker_id = "without-journal".into();
assert!(matches!(
store.bind_worker_execution_grant(&descendant),
Err(SubjektivError::AuthorizationDenied(_))
));
assert!(
store
.worker_execution_grant("runtime", "descendant")
.unwrap()
.is_none()
);
}
#[test]
fn retained_origins_reject_cycles_foreign_subjects_and_stale_parent_sources() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = source.delegation.clone();
store.bind_worker_execution_grant(&child).unwrap();
// Closing a retained chain back through its previously unbound root must not create a cycle.
let mut cycle = grant(&subject, &source.change_id, "body");
cycle.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&cycle),
Err(SubjektivError::AuthorizationDenied(_))
));
let other = store.create_subject(role()).unwrap().id;
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let mut foreign = grant(&other, &other_source.change_id, "foreign-child");
foreign.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&foreign),
Err(SubjektivError::AuthorizationDenied(_))
));
let current = store
.set_subject_permissions(
&subject,
&update("replace", Some(&source.change_id), source.delegation),
)
.unwrap();
let mut stale_parent = grant(&subject, &current.change_id, "stale-parent-child");
stale_parent.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&stale_parent),
Err(SubjektivError::AuthorizationDenied(_))
));
}
#[test]
fn failed_grant_insert_leaves_no_binding_and_can_retry_same_grant() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_grant BEFORE INSERT ON worker_execution_grants BEGIN SELECT RAISE(ABORT,'injected grant failure'); END;")?;
Ok(())
}).unwrap();
assert!(store.bind_worker_execution_grant(&child).is_err());
assert!(
store
.worker_execution_grant("runtime", "child")
.unwrap()
.is_none()
);
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("DROP TRIGGER fail_grant")?;
Ok(())
})
.unwrap();
store.bind_worker_execution_grant(&child).unwrap();
for sql in [
"UPDATE worker_execution_grants SET source_change_id='forged'",
"DELETE FROM worker_execution_grants",
"UPDATE subject_permission_changes SET record_json='{}'",
"DELETE FROM subject_permission_receipts",
] {
assert!(
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch(sql)?;
Ok(())
})
.is_err(),
"{sql}"
);
}
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
}
#[test]
fn current_schema_migration_preserves_subject_and_matches_new_database() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("migrated");
let original;
{
let (_old_manager, _old_scope, old) = open_version(&root, 11);
original = old
.create_subject_with_behavior(role(), "Keep behavior separate".into())
.unwrap();
}
let (_migrated_manager, _migrated_scope, migrated) = open_store(&root, "workspace-a");
assert_eq!(
migrated.subject(&original.id).unwrap(),
Some(original.clone())
);
assert!(
migrated
.subject_permissions(&original.id)
.unwrap()
.is_none()
);
let (_fresh_manager, _fresh_scope, fresh) =
open_store(&temp.path().join("fresh"), "workspace-a");
assert_eq!(schema(&migrated), schema(&fresh));
let source = migrated
.set_subject_permissions(&original.id, &update("new-source", None, Vec::new()))
.unwrap();
assert!(!source.change_id.is_empty());
assert_eq!(migrated.subject(&original.id).unwrap(), Some(original));
}
#[test]
fn unset_or_execution_only_source_cannot_issue_child_grants_and_saved_grants_survive_reopen() {
let temp = tempfile::tempdir().unwrap();
let saved;
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
assert!(
store
.bind_worker_execution_grant(&grant(&subject, "unset", "child"))
.is_err()
);
let mut execution_only = update("execution", None, Vec::new());
execution_only.execution = vec![permission("file.read", ".", true)];
let source = store
.set_subject_permissions(&subject, &execution_only)
.unwrap();
assert!(
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "child"))
.is_err()
);
let source = store
.set_subject_permissions(
&subject,
&update(
"delegation",
Some(&source.change_id),
execution_only.execution,
),
)
.unwrap();
saved = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&saved).unwrap();
}
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(saved.clone())
);
store.bind_worker_execution_grant(&saved).unwrap();
let mut conflicting = saved;
conflicting.profile = "builtin:default".into();
assert!(matches!(
store.bind_worker_execution_grant(&conflicting),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
}
#[test]
fn subject_grant_listing_is_scoped_retained_ordered_and_fails_on_overflow() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let other = store.create_subject(role()).unwrap().id;
assert!(
store
.list_subject_worker_execution_grants(&subject)
.unwrap()
.is_empty()
);
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let other_grant = grant(&other, &other_source.change_id, "other-child");
store.bind_worker_execution_grant(&other_grant).unwrap();
for index in (0..MAX_SUBJECT_WORKER_EXECUTION_GRANTS).rev() {
store
.bind_worker_execution_grant(&grant(
&subject,
&source.change_id,
&format!("child-{index:04}"),
))
.unwrap();
}
store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
store.retire_subject(&subject).unwrap();
let listed = store
.list_subject_worker_execution_grants(&subject)
.unwrap();
assert_eq!(listed.len(), MAX_SUBJECT_WORKER_EXECUTION_GRANTS);
assert!(
listed
.iter()
.all(|grant| grant.subject_id == subject && grant.source_change_id == source.change_id)
);
assert!(
listed
.windows(2)
.all(|pair| (&pair[0].runtime_id, &pair[0].worker_id)
< (&pair[1].runtime_id, &pair[1].worker_id))
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant.clone()]
);
// Seed one more retained row directly: the retired Subject cannot issue new grants.
// Listing must report overflow rather than returning a seemingly complete cancellation set.
let overflow = grant(&subject, &source.change_id, "overflow");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![overflow.runtime_id, overflow.worker_id, overflow.subject_id, overflow.source_change_id, serde_json::to_string(&overflow)?])?;
Ok(())
}).unwrap();
assert!(
matches!(store.list_subject_worker_execution_grants(&subject), Err(SubjektivError::WorkerExecutionGrantListOverflow { subject_id, limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS }) if subject_id == subject)
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant]
);
let (_foreign_manager, _foreign_scope, foreign) =
open_store(&temp.path().join("foreign"), "workspace-b");
assert!(matches!(
foreign.list_subject_worker_execution_grants(&subject),
Err(SubjektivError::SubjectNotFound(_))
));
}
#[test]
fn failed_schema_migration_rolls_back_new_tables_and_preserves_old_data() {
let temp = tempfile::tempdir().unwrap();
let original;
{
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
original = old.create_subject(role()).unwrap();
old.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TABLE subject_permissions (collision TEXT)")?;
Ok(())
})
.unwrap();
}
{
let manager = FeatureStorage::new(temp.path());
let scope = manager.scope("workspace-a").unwrap();
let registration = SubjektivStore::register(&scope).unwrap();
assert!(SubjektivStore::open(&scope, &registration).is_err());
}
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
assert_eq!(old.database.schema_version().unwrap(), 11);
assert_eq!(old.subject(&original.id).unwrap(), Some(original));
old.database.try_with_connection::<_, SubjektivError>(|connection| {
let created: bool = connection.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE name='subject_permission_changes')", [], |row| row.get(0))?;
assert!(!created);
Ok(())
}).unwrap();
}
+2 -1
View File
@@ -23,7 +23,8 @@ thiserror = { workspace = true }
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tracing = { workspace = true }
workdir = { workspace = true }
uuid = { workspace = true, features = ["v7"] }
[dev-dependencies]
filetime = "0.2.27"
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "test-util"] }
+246 -232
View File
@@ -1,24 +1,26 @@
//! Bash Tool using WorkdirSession command capability.
//!
//! The selected session owns cwd, process launch, bounded output, timeout, and cancellation.
//! Workdir owns launch and byte transport; Bash owns bounded preview and retained evidence.
//! Bash is not a sandbox; the Runtime process and OS user define the execution boundary.
//! Per-execution handles and nonces prevent cancellation of another command. Dropping a future
//! does not confirm termination; active registrations remain until cleanup requests cancellation
//! and observes terminal output.
//! does not confirm termination. Registrations remain during bounded cancellation/drain cleanup;
//! an unconfirmed provider outcome leaves partial evidence, never a fabricated terminal fence.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use crate::bash_output::{BashOutputMetadata, BashOutputView, BashOutputWriter};
use agen::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use async_trait::async_trait;
use schemars::JsonSchema;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
use workdir::{
CommandHandle, CommandOutputRequest, CommandRequest, WorkdirPath, WorkdirSessionHandle,
WorkdirSessionRouter,
};
mod cleanup;
const DEFAULT_TIMEOUT_SECS: u64 = 120;
const MAX_TIMEOUT_SECS: u64 = 600;
const INLINE_BYTE_BUDGET: usize = 12 * 1024;
@@ -28,16 +30,18 @@ struct BashParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
target_workdir: Option<String>,
/// Workdir-relative directory in which to execute the command.
/// Directory in which to execute the command.
// Keep provider path-coordinate documentation out of the model-facing schema.
#[schemars(with = "String")]
cwd: WorkdirPath,
command: String,
#[serde(default)]
timeout: Option<u64>,
}
pub(crate) struct BashTool {
pub struct BashTool {
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
state: Arc<Mutex<BashExecutionState>>,
}
@@ -63,6 +67,8 @@ struct CommandGuard {
execution_id: String,
execution_nonce: u64,
handle: Option<CommandHandle>,
cursor: u64,
retained: Option<BashOutputWriter>,
}
impl Drop for CommandGuard {
@@ -70,46 +76,69 @@ impl Drop for CommandGuard {
let Some(handle) = self.handle.take() else {
return;
};
let workdir = self.session.clone();
let state = Arc::clone(&self.state);
let execution_id = self.execution_id.clone();
let execution_nonce = self.execution_nonce;
// A dropped provider future is not terminal confirmation. Keep the live
// execution registered until cleanup has both requested cancellation and
// observed terminal command output, so cancellation/session teardown
// cannot race with an apparently empty registry.
tokio::spawn(async move {
let _ = workdir.cancel_command(handle.clone()).await;
let _ = workdir
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
.await;
let mut state = state.lock().unwrap();
if state
.active
.get(&execution_id)
.is_some_and(|active| active.execution_nonce == execution_nonce)
{
state.active.remove(&execution_id);
state.cancellation_requested.remove(&execution_id);
}
});
let cleanup = cleanup::Cleanup {
session: self.session.clone(),
state: Arc::clone(&self.state),
execution_id: self.execution_id.clone(),
execution_nonce: self.execution_nonce,
handle,
cursor: self.cursor,
retained: self
.retained
.take()
.expect("started command owns output writer"),
};
tokio::spawn(cleanup.run());
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
// Output pages are replayable only with the exact same command/cursor/limit.
// Retry read transport failures, never command start, and cap both attempts and
// backoff. The caller's deadline bounds a disconnected or unresponsive provider.
async fn command_page(
session: &workdir::WorkdirSessionHandle,
request: CommandOutputRequest,
) -> Result<workdir::CommandOutput, workdir::WorkdirError> {
for attempt in 0..3 {
match session.command_output(request.clone()).await {
Err(error)
if attempt < 2
&& matches!(
error.classification_source(),
workdir::WorkdirError::Transport(_)
| workdir::WorkdirError::Unavailable(_)
| workdir::WorkdirError::OutcomeUnknown(_)
) =>
{
tokio::time::sleep(std::time::Duration::from_millis(100 << attempt)).await;
}
result => return result,
}
}
unreachable!("bounded retry loop returns its last result")
}
impl BashTool {
pub fn input_schema() -> serde_json::Value {
serde_json::to_value(schemars::schema_for!(BashParams)).expect("Bash schema serialization")
}
pub fn new(router: Arc<WorkdirSessionRouter>, output: BashOutputView) -> Self {
Self {
router,
output,
state: Arc::new(Mutex::new(BashExecutionState::default())),
}
}
pub async fn execute_result(
&self,
input_json: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
) -> Result<BashResult, ToolError> {
let params: BashParams = serde_json::from_str(input_json)
.map_err(|error| ToolError::InvalidArgument(format!("invalid Bash input: {error}")))?;
let retained = self.output.begin().map_err(ToolError::ExecutionFailed)?;
let selected = crate::routing::resolve_session(
&self.router,
params.target_workdir.as_deref(),
@@ -134,14 +163,15 @@ impl Tool for BashTool {
execution_id: execution_id.clone(),
execution_nonce,
handle: None,
cursor: 0,
retained: Some(retained),
};
let handle = session
.start_command(CommandRequest {
command: params.command,
timeout_secs,
output_limit: INLINE_BYTE_BUDGET,
transport_byte_budget: self.output.byte_budget(),
cwd: params.cwd,
spill_dir: Some(self.output_dir.clone()),
tool_call_id: Some(call_id.clone()),
})
.await
@@ -167,15 +197,101 @@ impl Tool for BashTool {
.await
.map_err(crate::ToolsError::from)?;
}
let output = session
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
let mut cursor = 0;
let mut stdout_preview = Vec::new();
let mut stderr_preview = Vec::new();
let mut seen = [0_u64; 2];
let mut empty_polls = 0;
let deadline =
tokio::time::Instant::now() + std::time::Duration::from_secs(timeout_secs + 30);
let terminal = loop {
let page = tokio::time::timeout_at(
deadline,
command_page(
&session,
CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
),
)
.await
.map_err(|_| {
ToolError::ExecutionFailed(
"command byte transport deadline exceeded; terminal outcome unknown".into(),
)
})?
.map_err(crate::ToolsError::from)?;
let mut next = cursor;
for chunk in &page.chunks {
if chunk.sequence != next || chunk.bytes.is_empty() {
return Err(ToolError::ExecutionFailed(
"command byte transport sequence gap".into(),
));
}
next = next
.checked_add(chunk.bytes.len() as u64)
.filter(|n| *n <= self.output.byte_budget())
.ok_or_else(|| {
ToolError::ExecutionFailed(
"command byte transport exceeds delivery budget".into(),
)
})?;
let index = if chunk.stream == workdir::CommandStream::Stdout {
0
} else {
1
};
if chunk.offset != seen[index] {
return Err(ToolError::ExecutionFailed(
"command byte transport offset gap".into(),
));
}
seen[index] += chunk.bytes.len() as u64;
guard
.retained
.as_mut()
.unwrap()
.append_chunk(chunk)
.map_err(ToolError::ExecutionFailed)?;
let available =
INLINE_BYTE_BUDGET.saturating_sub(stdout_preview.len() + stderr_preview.len());
let preview = if index == 0 {
&mut stdout_preview
} else {
&mut stderr_preview
};
preview.extend_from_slice(&chunk.bytes[..available.min(chunk.bytes.len())]);
}
if page.next_cursor != next {
return Err(ToolError::ExecutionFailed(
"command byte transport cursor does not fence the page".into(),
));
}
if page.chunks.is_empty() && page.terminal.is_none() {
empty_polls += 1;
if empty_polls > 32 {
return Err(ToolError::ExecutionFailed(
"command byte transport made no progress after bounded polls".into(),
));
}
} else {
empty_polls = 0;
}
cursor = page.next_cursor;
guard.cursor = cursor;
if let Some(terminal) = page.terminal {
break terminal;
}
};
let metadata = guard
.retained
.as_mut()
.unwrap()
.finish_transport(&terminal)
.map_err(ToolError::ExecutionFailed)?;
let cancellation_requested = {
let mut state = self.state.lock().unwrap();
let owns_registration = state
@@ -193,41 +309,84 @@ impl Tool for BashTool {
};
guard.handle = None;
let timed_out = output.timed_out;
let summary = if cancellation_requested {
format!("$ {cmd_summary} (cancelled)")
} else if output.timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match output.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
let content = if output.content.is_empty() {
None
} else if output.truncated {
let notice = match output.output_path {
Some(path) => format!(
"[showing bounded WorkdirSession command output; full output saved to {}]",
path.display()
),
None => "[showing bounded WorkdirSession command output; additional output was truncated]"
.to_owned(),
let timed_out = terminal.status == workdir::CommandStatus::TimedOut;
let mut summary =
if cancellation_requested || terminal.status == workdir::CommandStatus::Cancelled {
format!("$ {cmd_summary} (cancelled)")
} else if timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match terminal.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
Some(format!("{notice}\n{}", output.content))
} else {
Some(output.content)
};
let output = ToolOutput {
if matches!(
terminal.output_collection,
workdir::CommandOutputCollection::Incomplete { .. }
) {
summary.push_str(" (output incomplete)");
}
Ok(BashResult {
summary,
content,
attachments: Vec::new(),
};
if cancellation_requested {
stdout_preview: String::from_utf8_lossy(&stdout_preview).into_owned(),
stderr_preview: String::from_utf8_lossy(&stderr_preview).into_owned(),
preview_omitted: terminal.stdout_bytes.saturating_add(terminal.stderr_bytes)
> (stdout_preview.len() + stderr_preview.len()) as u64,
metadata,
cancelled: cancellation_requested
|| terminal.status == workdir::CommandStatus::Cancelled,
})
}
}
#[derive(Debug, Serialize)]
pub struct BashResult {
pub summary: String,
pub stdout_preview: String,
pub stderr_preview: String,
pub preview_omitted: bool,
pub metadata: BashOutputMetadata,
pub cancelled: bool,
}
impl BashResult {
pub fn stdout_entry(&self) -> String {
format!("/bash-output/{}/stdout", self.metadata.id)
}
pub fn stderr_entry(&self) -> String {
format!("/bash-output/{}/stderr", self.metadata.id)
}
pub fn order_entry(&self) -> String {
format!("/bash-output/{}/order", self.metadata.id)
}
pub fn tool_output(&self) -> ToolOutput {
let content = serde_json::json!({
"stdout_preview": self.stdout_preview, "stderr_preview": self.stderr_preview,
"preview_omitted": self.preview_omitted, "output": self.metadata,
"stdout_entry": self.stdout_entry(), "stderr_entry": self.stderr_entry(), "order_entry": self.order_entry(),
});
ToolOutput {
summary: self.summary.clone(),
content: Some(content.to_string()),
attachments: vec![],
}
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
&self,
input: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
let result = self.execute_result(input, ctx).await?;
let output = result.tool_output();
if result.cancelled {
Err(ToolError::Cancelled(output))
} else if timed_out {
} else if result.metadata.terminal.status == Some(workdir::CommandStatus::TimedOut) {
Err(ToolError::Interrupted(output))
} else {
Ok(output)
@@ -288,164 +447,19 @@ fn truncate_for_summary(command: &str) -> String {
summary
}
pub fn bash_tool(session: WorkdirSessionHandle, output_dir: PathBuf) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output_dir)
pub fn bash_tool(session: WorkdirSessionHandle, output: BashOutputView) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output)
}
pub(crate) fn routed_bash_tool(
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
) -> ToolDefinition {
Arc::new(move || {
let schema = schemars::schema_for!(BashParams);
let meta = ToolMeta::new("Bash")
.description("Execute a shell command in one selected Workdir. Process start, bounded inline output, full-output spill, timeout and cancellation are owned by that WorkdirSession provider. This is not a sandbox.")
.input_schema(serde_json::to_value(schema).expect("Bash schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(BashTool {
router: router.clone(),
output_dir: output_dir.clone(),
state: Arc::new(Mutex::new(BashExecutionState::default())),
});
.description("Execute a shell command in one selected Workdir. Workdir owns launch, lossless bounded byte transport, timeout and cancel; Bash owns preview, quota-limited ephemeral output and read-only virtual references. This is not a sandbox.")
.input_schema(BashTool::input_schema());
let tool: Arc<dyn Tool> = Arc::new(BashTool::new(router.clone(), output.clone()));
(meta, tool)
})
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use tempfile::TempDir;
use workdir::{LocalWorkdirSession, WorkdirSessionHandle};
use super::bash_tool;
use crate::{grep::grep_tool, read::read_tool, tracker::Tracker};
fn session_with_output_scope(root: &TempDir, output: &TempDir) -> WorkdirSessionHandle {
let scope = Scope::from_config(&ScopeConfig {
allow: vec![
ScopeRule {
target: root.path().to_path_buf(),
permission: Permission::Write,
recursive: true,
symlink_policy: Default::default(),
},
ScopeRule {
target: output.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
},
],
deny: Vec::new(),
})
.unwrap();
Arc::new(LocalWorkdirSession::new(scope, root.path().to_path_buf()))
}
#[tokio::test]
async fn long_output_is_spilled_and_available_to_read_and_grep() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session.clone(), output.path().to_path_buf())();
let command = "i=0; while [ $i -lt 2000 ]; do printf 'line-%04d\\n' \"$i\"; i=$((i+1)); done; printf 'FINAL-NEEDLE\\n'";
let result = bash
.execute(
&serde_json::json!({ "command": command, "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
let rendered = result.content.expect("bounded Bash output");
let artifact = std::fs::read_dir(output.path())
.unwrap()
.next()
.expect("artifact entry")
.unwrap()
.path();
assert!(rendered.contains("full output saved to"));
assert!(rendered.contains(&artifact.display().to_string()));
let retained = std::fs::read_to_string(&artifact).unwrap();
assert!(retained.starts_with("line-0000\n"));
assert!(retained.ends_with("FINAL-NEEDLE\n"));
assert_eq!(retained.lines().count(), 2001);
let (_, read) = read_tool(session.clone(), Tracker::new())();
let read_result = read
.execute(
&serde_json::json!({
"file_path": artifact,
"offset": 2000,
"limit": 1,
})
.to_string(),
Default::default(),
)
.await
.unwrap();
assert!(
read_result
.content
.expect("Read content")
.contains("FINAL-NEEDLE")
);
let (_, grep) = grep_tool(session)();
let grep_result = grep
.execute(
&serde_json::json!({
"pattern": "FINAL-NEEDLE",
"path": artifact,
"output_mode": "content",
})
.to_string(),
Default::default(),
)
.await
.unwrap();
let grep_content = grep_result.content.expect("Grep content");
assert!(
grep_content.contains("FINAL-NEEDLE"),
"unexpected Grep content: {grep_content:?}"
);
}
#[tokio::test]
async fn missing_cwd_is_rejected_before_command_start() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let error = bash
.execute(
&serde_json::json!({ "command": "printf hidden" }).to_string(),
Default::default(),
)
.await
.unwrap_err();
assert!(error.to_string().contains("missing field `cwd`"));
}
#[tokio::test]
async fn short_output_does_not_leave_a_spill_artifact() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let result = bash
.execute(
&serde_json::json!({ "command": "printf short", "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
assert_eq!(result.content.as_deref(), Some("short"));
assert_eq!(std::fs::read_dir(output.path()).unwrap().count(), 0);
}
}
+121
View File
@@ -0,0 +1,121 @@
//! Finite Drop cleanup. A cancel acknowledgement is not termination evidence; only a
//! validated terminal fence confirms the process result. No provider text is logged.
use super::*;
use crate::bash_output::{
CleanupCancel, CleanupDiagnostics, CleanupDrain, CleanupFailure, CleanupProviderFailure,
CleanupRegistration,
};
use std::time::Duration;
const CLEANUP_TIMEOUT: Duration = Duration::from_secs(10);
// Reserve time for independent terminal observation even if cancel never responds.
const CANCEL_TIMEOUT: Duration = Duration::from_secs(2);
pub(super) struct Cleanup {
pub session: WorkdirSessionHandle,
pub state: Arc<Mutex<BashExecutionState>>,
pub execution_id: String,
pub execution_nonce: u64,
pub handle: CommandHandle,
pub cursor: u64,
pub retained: BashOutputWriter,
}
impl Cleanup {
pub async fn run(mut self) {
let deadline = tokio::time::Instant::now() + CLEANUP_TIMEOUT;
let cancel = match tokio::time::timeout(
CANCEL_TIMEOUT,
self.session.cancel_command(self.handle.clone()),
)
.await
{
Ok(Ok(())) => CleanupCancel::Acknowledged,
Ok(Err(error)) => CleanupCancel::Failed {
reason: CleanupProviderFailure::from(&error),
},
Err(_) => CleanupCancel::DeadlineExceeded,
};
let drain = match tokio::time::timeout_at(deadline, self.drain()).await {
Ok(Ok(())) => CleanupDrain::TerminalConfirmed,
Ok(Err(reason)) => CleanupDrain::Failed { reason },
Err(_) => CleanupDrain::DeadlineExceeded,
};
let deadline_expired = matches!(drain, CleanupDrain::DeadlineExceeded);
// Finalize evidence before exposing an empty registry. Keep the lock through
// publication so an older cleanup can never remove a newer generation.
let mut state = self.state.lock().unwrap();
let registration = if state
.active
.get(&self.execution_id)
.is_some_and(|active| active.execution_nonce == self.execution_nonce)
{
state.active.remove(&self.execution_id);
state.cancellation_requested.remove(&self.execution_id);
CleanupRegistration::Released
} else {
CleanupRegistration::Superseded
};
let diagnostics = CleanupDiagnostics {
execution_nonce: self.execution_nonce,
cancel,
drain,
deadline_expired,
registration,
};
tracing::info!(
output_id = %self.retained.id(),
cleanup = ?diagnostics,
"Bash Drop cleanup finished; cancellation and terminal observation are independent"
);
self.retained.finish_cleanup(diagnostics);
drop(self.retained);
}
async fn drain(&mut self) -> Result<(), CleanupFailure> {
loop {
let page = command_page(
&self.session,
CommandOutputRequest {
handle: self.handle.clone(),
cursor: self.cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
)
.await
.map_err(|error| CleanupFailure::Provider {
reason: CleanupProviderFailure::from(&error),
})?;
let mut next = self.cursor;
for chunk in &page.chunks {
if chunk.sequence != next {
return Err(CleanupFailure::SequenceGap);
}
next = next
.checked_add(chunk.bytes.len() as u64)
.filter(|next| *next <= crate::bash_output::COMMAND_BYTES as u64)
.ok_or(CleanupFailure::DeliveryBudgetExceeded)?;
self.retained
.append_chunk(chunk)
.map_err(|_| CleanupFailure::RetainedChunkInvalid)?;
}
if page.next_cursor != next {
return Err(CleanupFailure::CursorGap);
}
self.cursor = next;
if let Some(terminal) = page.terminal {
self.retained
.finish_transport(&terminal)
.map_err(|_| CleanupFailure::TerminalMetadataInvalid)?;
return Ok(());
}
// An immediately-ready provider must not starve the cleanup deadline.
tokio::task::yield_now().await;
}
}
}
#[cfg(test)]
mod tests;
+397
View File
@@ -0,0 +1,397 @@
//! Component tests enter through the production CommandGuard Drop, cleanup, page
//! validation and retained store. Only the Workdir provider is replaced; paused time
//! and a cancel barrier make faults/deadlines deterministic without OS timing.
use super::*;
use std::sync::atomic::{AtomicBool, Ordering};
use workdir::*;
const PRIVATE: &str = "SECRET command /private/host/path output password=poison";
#[derive(Clone, Copy, Debug)]
enum Fault {
Cancel,
CancelHang,
Drain,
TerminalTimeout,
Sequence,
Cursor,
RetainedChunk,
TerminalMetadata,
CollectionIncomplete,
Complete,
}
#[derive(Debug)]
struct Provider {
local: LocalWorkdirSession,
fault: Fault,
cancelling: tokio::sync::Notify,
allow_drain: AtomicBool,
}
#[async_trait]
impl WorkdirSession for Provider {
fn workdir(&self) -> &Workdir {
self.local.workdir()
}
fn capabilities(&self) -> WorkdirSessionCapabilities {
WorkdirSessionCapabilities::ALL
}
async fn stat(&self, _: StatRequest) -> Result<StatResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn read(&self, _: ReadRequest) -> Result<ReadResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn write(&self, _: WriteRequest) -> Result<WriteResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn edit(&self, _: EditRequest) -> Result<EditResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn list(&self, _: ListRequest) -> Result<ListResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn glob(&self, _: GlobRequest) -> Result<GlobResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn grep(&self, _: GrepRequest) -> Result<GrepResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn start_command(&self, _: CommandRequest) -> Result<CommandHandle, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn command_status(&self, _: CommandHandle) -> Result<CommandStatus, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn command_output(
&self,
request: CommandOutputRequest,
) -> Result<CommandOutput, WorkdirError> {
assert_eq!(request.cursor, 3);
while !self.allow_drain.load(Ordering::Acquire) {
tokio::task::yield_now().await;
}
match self.fault {
Fault::Drain => {
return Err(WorkdirError::Transport(PRIVATE.into())
.with_failure_reason(WorkdirFailureReason::HttpTransfer));
}
Fault::TerminalTimeout => return std::future::pending().await,
_ => {}
}
let mut chunk = CommandByteChunk {
sequence: 3,
offset: 3,
stream: CommandStream::Stdout,
bytes: b"tail".to_vec(),
};
if matches!(self.fault, Fault::Sequence) {
chunk.sequence = 4;
}
if matches!(self.fault, Fault::RetainedChunk) {
chunk.offset = 42;
}
let mut terminal = terminal();
if matches!(self.fault, Fault::TerminalMetadata) {
terminal.stdout_delivered_bytes = 6;
}
if matches!(self.fault, Fault::CollectionIncomplete) {
terminal.output_collection = CommandOutputCollection::Incomplete {
failure: CommandFailure {
stage: CommandFailureStage::StdoutDrain,
reason: CommandIoErrorKind::Other,
os_code: None,
},
};
}
Ok(CommandOutput {
chunks: vec![chunk],
next_cursor: if matches!(self.fault, Fault::Cursor) {
8
} else {
7
},
terminal: Some(terminal),
})
}
async fn cancel_command(&self, _: CommandHandle) -> Result<(), WorkdirError> {
self.cancelling.notify_one();
match self.fault {
Fault::Cancel => Err(WorkdirError::Unavailable(PRIVATE.into())),
Fault::CancelHang => std::future::pending().await,
_ => Ok(()),
}
}
async fn close(&self) -> Result<(), WorkdirError> {
Ok(())
}
}
fn terminal() -> CommandTerminal {
CommandTerminal {
status: CommandStatus::Completed,
exit_code: Some(0),
process_failure: None,
output_collection: CommandOutputCollection::Complete,
stdout_bytes: 7,
stderr_bytes: 0,
stdout_delivered_bytes: 7,
stderr_delivered_bytes: 0,
truncation_reason: None,
}
}
struct Fixture {
_root: tempfile::TempDir,
provider: Arc<Provider>,
state: Arc<Mutex<BashExecutionState>>,
view: BashOutputView,
id: String,
}
impl Fixture {
fn start(fault: Fault) -> (Self, CommandGuard) {
let base =
std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../target/cleanup-fixtures");
std::fs::create_dir_all(&base).unwrap();
let root = tempfile::tempdir_in(base).unwrap();
let provider = Arc::new(Provider {
local: LocalWorkdirSession::new(
manifest::Scope::writable(root.path()).unwrap(),
root.path().into(),
),
fault,
cancelling: Default::default(),
allow_drain: AtomicBool::new(false),
});
let view = BashOutputView::new();
let mut retained = view.begin().unwrap();
retained.append(CommandStream::Stdout, b"pre");
let id = retained.id();
let state = Arc::new(Mutex::new(BashExecutionState::default()));
let handle = CommandHandle(PRIVATE.into());
state.lock().unwrap().active.insert(
PRIVATE.into(),
ActiveCommand {
call_id: PRIVATE.into(),
execution_nonce: 1,
session: provider.clone(),
handle: handle.clone(),
},
);
state
.lock()
.unwrap()
.cancellation_requested
.insert(PRIVATE.into());
let guard = CommandGuard {
session: provider.clone(),
state: state.clone(),
execution_id: PRIVATE.into(),
execution_nonce: 1,
handle: Some(handle),
cursor: 3,
retained: Some(retained),
};
(
Self {
_root: root,
provider,
state,
view,
id,
},
guard,
)
}
async fn finish(&self) -> BashOutputMetadata {
self.provider.allow_drain.store(true, Ordering::Release);
// The provider's read retry backoff and cleanup deadlines are virtual time.
for _ in 0..4 {
tokio::task::yield_now().await;
tokio::time::advance(Duration::from_secs(3)).await;
}
for _ in 0..1000 {
if let Some(metadata) = self.view.metadata(&self.id) {
return metadata;
}
tokio::task::yield_now().await;
}
panic!("finite cleanup did not publish partial evidence");
}
fn assert_evidence(&self, m: &BashOutputMetadata, terminal: bool, bytes: &[u8]) {
assert_eq!(
self.view
.read(&self.id, CommandStream::Stdout, 0, 64)
.unwrap(),
bytes
);
assert_eq!(m.stdout_saved, bytes.len());
assert_eq!(m.stdout_delivered, bytes.len() as u64);
assert_eq!(m.stdout_seen, bytes.len() as u64);
assert_eq!(
m.terminal.status,
terminal.then_some(CommandStatus::Completed)
);
assert_eq!(m.terminal.exit_code, terminal.then_some(0));
if !terminal {
assert!(m.terminal.output_collection.is_none());
assert!(m.terminal.process_failure.is_none());
assert!(!m.terminal.transport_complete);
}
let diagnostics = m.cleanup.as_ref().unwrap();
assert_eq!(diagnostics.execution_nonce, 1);
let json = serde_json::to_string(diagnostics).unwrap();
let debug = format!("{diagnostics:?}");
for forbidden in [
PRIVATE, "SECRET", "/private", "password", "poison", "tail", "pre",
] {
assert!(!json.contains(forbidden), "diagnostics leaked data: {json}");
assert!(
!debug.contains(forbidden),
"tracing payload leaked data: {debug}"
);
}
}
}
#[tokio::test(start_paused = true)]
async fn cancel_failure_does_not_preclude_confirmed_terminal() {
let (f, guard) = Fixture::start(Fault::Cancel);
drop(guard);
f.provider.cancelling.notified().await;
assert!(f.state.lock().unwrap().active.contains_key(PRIVATE));
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(m.terminal.transport_complete);
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::Failed { .. }
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(!m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
assert!(f.state.lock().unwrap().cancellation_requested.is_empty());
}
#[tokio::test(start_paused = true)]
async fn acknowledged_cancel_drain_faults_keep_distinct_reasons_and_partial_bytes() {
for (fault, reason, bytes) in [
(Fault::Drain, "provider", b"pre".as_slice()),
(Fault::Sequence, "sequence_gap", b"pre"),
(Fault::Cursor, "cursor_gap", b"pretail"),
(Fault::RetainedChunk, "retained_chunk_invalid", b"pre"),
(
Fault::TerminalMetadata,
"terminal_metadata_invalid",
b"pretail",
),
] {
let (f, guard) = Fixture::start(fault);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, false, bytes);
let diagnostic = serde_json::to_value(m.cleanup.as_ref().unwrap()).unwrap();
assert_eq!(diagnostic["cancel"]["outcome"], "acknowledged");
assert_eq!(diagnostic["drain"]["outcome"], "failed");
assert_eq!(diagnostic["drain"]["reason"]["kind"], reason);
assert_eq!(diagnostic["registration"], "released");
assert_eq!(diagnostic["deadline_expired"], false);
assert!(f.state.lock().unwrap().active.is_empty());
}
}
#[tokio::test(start_paused = true)]
async fn terminal_deadline_never_claims_stop_or_leaves_active_collector() {
let (f, guard) = Fixture::start(Fault::TerminalTimeout);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, false, b"pre");
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::Acknowledged
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::DeadlineExceeded
));
assert!(m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
}
#[tokio::test(start_paused = true)]
async fn cancel_deadline_still_attempts_independent_terminal_observation() {
let (f, guard) = Fixture::start(Fault::CancelHang);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::DeadlineExceeded
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(!m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
}
#[tokio::test(start_paused = true)]
async fn old_cleanup_preserves_new_execution_and_cancellation_registration() {
let (f, guard) = Fixture::start(Fault::Complete);
drop(guard);
f.provider.cancelling.notified().await;
f.state
.lock()
.unwrap()
.active
.get_mut(PRIVATE)
.unwrap()
.execution_nonce = 2;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(matches!(
m.cleanup.as_ref().unwrap().registration,
CleanupRegistration::Superseded
));
assert_eq!(f.state.lock().unwrap().active[PRIVATE].execution_nonce, 2);
assert!(
f.state
.lock()
.unwrap()
.cancellation_requested
.contains(PRIVATE)
);
}
#[tokio::test(start_paused = true)]
async fn terminal_confirmation_is_independent_of_output_collection_completeness() {
let (f, guard) = Fixture::start(Fault::CollectionIncomplete);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(!m.terminal.transport_complete);
assert!(matches!(
m.terminal.output_collection,
Some(CommandOutputCollection::Incomplete { .. })
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(f.state.lock().unwrap().active.is_empty());
}
+725
View File
@@ -0,0 +1,725 @@
//! Bash-owned bounded output, independent of Workdir transport and Console subscribers.
//!
//! Output is ephemeral evidence, not a filesystem grant. Every lookup checks the live view;
//! children can see their own subtree and ancestors can retain evidence after child revocation.
//! Root teardown clears all bytes. A new root (including restore) has a new incarnation and
//! never reconstructs grants from history. Expiry is measured from admission, enforced on
//! lookup and by a weak reaper once commands run in Tokio. Parent and all descendants share
//! one pool: admission reserves a whole command budget, evicts the oldest terminal evidence
//! first, and refuses rather than evicting a live collector. Revoking a child denies access
//! but leaves ancestor evidence until pool eviction/expiry; root teardown clears the pool.
use std::collections::VecDeque;
use std::sync::{
Arc, Mutex, Weak,
atomic::{AtomicBool, Ordering},
};
use std::time::{Duration, Instant};
use serde::Serialize;
use workdir::{CommandStatus, CommandStream};
pub(crate) const COMMAND_BYTES: usize = 4 * 1024 * 1024;
const STORE_BYTES: usize = 32 * 1024 * 1024;
const STORE_COUNT: usize = 64;
const ORDER_COUNT: usize = 4096;
const RETENTION: Duration = Duration::from_secs(3600);
#[derive(Clone)]
pub struct BashOutputView {
store: Arc<Mutex<Store>>,
lineage: Vec<String>,
active: Arc<AtomicBool>,
authority: Arc<dyn Fn() -> bool + Send + Sync>,
}
struct Store {
records: VecDeque<Arc<Mutex<Record>>>,
reserved: usize,
retention: Duration,
reaper_started: bool,
_shutdown: tokio::sync::watch::Sender<()>,
}
struct Record {
id: String,
owner: Vec<String>,
created: Instant,
stdout: Vec<u8>,
stderr: Vec<u8>,
order: Vec<OutputRange>,
stdout_seen: u64,
stderr_seen: u64,
truncated: bool,
limit: Option<RetentionLimit>,
transport_counts: Option<(u64, u64)>,
transport_truncation: Option<workdir::CommandOutputTruncation>,
terminal: Option<OutputTerminal>,
cleanup: Option<CleanupDiagnostics>,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputRange {
pub stream: CommandStream,
pub start: u64,
pub end: u64,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputTerminal {
pub status: Option<CommandStatus>,
pub exit_code: Option<i32>,
pub process_failure: Option<workdir::CommandFailure>,
pub output_collection: Option<workdir::CommandOutputCollection>,
/// True only for confirmed EOF collection (quota is reported separately). A terminal
/// fence or exit zero alone does not confirm the unobserved process tail.
pub transport_complete: bool,
}
#[derive(Clone, Copy, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum RetentionLimit {
ByteQuota,
OrderQuota,
}
#[derive(Clone, Debug, Serialize)]
pub struct BashOutputMetadata {
pub id: String,
pub stdout_saved: usize,
pub stderr_saved: usize,
pub stdout_seen: u64,
pub stderr_seen: u64,
pub stdout_delivered: u64,
pub stderr_delivered: u64,
pub retention_limit: Option<RetentionLimit>,
pub transport_truncation: Option<workdir::CommandOutputTruncation>,
pub storage_truncated: bool,
pub terminal: OutputTerminal,
pub cleanup: Option<CleanupDiagnostics>,
pub text_encoding: &'static str,
pub ordering: &'static str,
}
/// Drop cleanup observations, not assertions that cancellation stopped a process.
/// Correlate with the enclosing output UUID and this host-assigned execution generation;
/// caller identifiers, provider handles, commands and error text are deliberately excluded.
#[derive(Clone, Debug, Serialize)]
pub struct CleanupDiagnostics {
pub execution_nonce: u64,
pub cancel: CleanupCancel,
pub drain: CleanupDrain,
pub deadline_expired: bool,
pub registration: CleanupRegistration,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "outcome", rename_all = "snake_case")]
pub enum CleanupCancel {
Acknowledged,
Failed { reason: CleanupProviderFailure },
DeadlineExceeded,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "outcome", rename_all = "snake_case")]
pub enum CleanupDrain {
TerminalConfirmed,
Failed { reason: CleanupFailure },
DeadlineExceeded,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum CleanupFailure {
Provider { reason: CleanupProviderFailure },
SequenceGap,
CursorGap,
DeliveryBudgetExceeded,
RetainedChunkInvalid,
TerminalMetadataInvalid,
}
#[derive(Clone, Debug, Serialize)]
pub struct CleanupProviderFailure {
pub failure_reason: Option<workdir::WorkdirFailureReason>,
pub denial_reason: Option<workdir::WorkdirDenialReason>,
}
impl From<&workdir::WorkdirError> for CleanupProviderFailure {
fn from(error: &workdir::WorkdirError) -> Self {
Self {
failure_reason: error.failure_reason(),
denial_reason: error.denial_reason(),
}
}
}
#[derive(Clone, Copy, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum CleanupRegistration {
Released,
Superseded,
}
/// A bounded reservation acquired before command start. Drop terminalizes partial evidence.
pub struct BashOutputWriter {
record: Arc<Mutex<Record>>,
}
impl Default for BashOutputView {
fn default() -> Self {
Self::new()
}
}
impl BashOutputView {
pub fn new() -> Self {
let (shutdown, _) = tokio::sync::watch::channel(());
let store = Arc::new(Mutex::new(Store {
records: VecDeque::new(),
reserved: 0,
retention: RETENTION,
reaper_started: false,
_shutdown: shutdown,
}));
let view = Self {
store,
lineage: vec![uuid::Uuid::now_v7().to_string()],
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(|| true),
};
view.start_reaper();
view
}
fn start_reaper(&self) {
let Ok(runtime) = tokio::runtime::Handle::try_current() else {
return;
};
let mut store = self.store.lock().unwrap();
if store.reaper_started {
return;
}
let receiver = store._shutdown.subscribe();
store.reaper_started = true;
let weak = Arc::downgrade(&self.store);
runtime.spawn(reap(weak, receiver));
}
pub fn byte_budget(&self) -> u64 {
COMMAND_BYTES as u64
}
pub fn child(&self, authority: Arc<dyn Fn() -> bool + Send + Sync>) -> Self {
let parent = self.clone();
let mut lineage = self.lineage.clone();
lineage.push(uuid::Uuid::now_v7().to_string());
Self {
store: self.store.clone(),
lineage,
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(move || parent.is_active() && authority()),
}
}
pub fn is_active(&self) -> bool {
self.active.load(Ordering::Acquire) && (self.authority)()
}
pub fn is_root(&self) -> bool {
self.lineage.len() == 1
}
pub fn revoke(&self) {
self.active.store(false, Ordering::Release);
if self.lineage.len() == 1 {
let mut store = self.store.lock().unwrap();
store.records.clear();
store.reserved = 0;
}
}
pub fn begin(&self) -> Result<BashOutputWriter, String> {
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
// A host may construct the Feature before entering Tokio. Command admission
// happens inside the runtime, so idle expiry must start here as well.
self.start_reaper();
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
store.expire(Instant::now());
while store.records.len() >= STORE_COUNT || store.reserved + COMMAND_BYTES > STORE_BYTES {
let Some(index) = store
.records
.iter()
.position(|r| r.lock().unwrap().terminal.is_some())
else {
return Err("Bash output capacity is occupied by live commands".into());
};
store.records.remove(index);
store.reserved -= COMMAND_BYTES;
}
let record = Arc::new(Mutex::new(Record {
id: uuid::Uuid::now_v7().to_string(),
owner: self.lineage.clone(),
created: Instant::now(),
stdout: Vec::new(),
stderr: Vec::new(),
order: Vec::new(),
stdout_seen: 0,
stderr_seen: 0,
truncated: false,
limit: None,
transport_counts: None,
transport_truncation: None,
terminal: None,
cleanup: None,
}));
store.records.push_back(record.clone());
store.reserved += COMMAND_BYTES;
Ok(BashOutputWriter { record })
}
fn record(&self, id: &str) -> Option<Arc<Mutex<Record>>> {
if !self.is_active() {
return None;
}
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return None;
}
store.expire(Instant::now());
store
.records
.iter()
.find(|r| {
let r = r.lock().unwrap();
r.id == id && r.owner.starts_with(&self.lineage) && r.terminal.is_some()
})
.cloned()
}
pub fn metadata(&self, id: &str) -> Option<BashOutputMetadata> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.metadata())
}
/// Copies at most the caller's bounded range; never publishes output bodies in descriptors.
pub fn read(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Vec<u8>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let start = offset.min(bytes.len());
Some(bytes[start..start.saturating_add(limit.min(64 * 1024)).min(bytes.len())].to_vec())
}
/// Scan the bounded raw source without cloning it, decoding only the requested line window.
/// A long single line uses the common byte reader instead of allocating an oversized text page.
pub fn line_window(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Result<(String, bool), String>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let mut lines = bytes.split_inclusive(|b| *b == b'\n').skip(offset);
let mut window = Vec::new();
for line in lines.by_ref().take(limit.min(2000)) {
if window.len() + line.len() > 64 * 1024 {
return Some(Err(
"text source window exceeds 64 KiB; use the common byte reader".into(),
));
}
window.extend_from_slice(line);
}
let more = lines.next().is_some();
Some(Ok((String::from_utf8_lossy(&window).into_owned(), more)))
}
pub fn order(&self, id: &str) -> Option<Vec<OutputRange>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.order.clone())
}
}
impl Record {
fn metadata(&self) -> BashOutputMetadata {
BashOutputMetadata {
id: self.id.clone(),
stdout_saved: self.stdout.len(),
stderr_saved: self.stderr.len(),
stdout_seen: self.transport_counts.map_or(self.stdout_seen, |c| c.0),
stderr_seen: self.transport_counts.map_or(self.stderr_seen, |c| c.1),
stdout_delivered: self.stdout_seen,
stderr_delivered: self.stderr_seen,
retention_limit: self.limit,
transport_truncation: self.transport_truncation.clone(),
storage_truncated: self.truncated || self.transport_truncation.is_some(),
terminal: self.terminal.clone().expect("finalized record"),
cleanup: self.cleanup.clone(),
text_encoding: "utf8_lossy",
ordering: "provider_read_observation",
}
}
}
impl Store {
fn expire(&mut self, now: Instant) {
let before = self.records.len();
self.records
.retain(|r| now.duration_since(r.lock().unwrap().created) < self.retention);
self.reserved -= (before - self.records.len()) * COMMAND_BYTES;
}
}
async fn reap(weak: Weak<Mutex<Store>>, mut shutdown: tokio::sync::watch::Receiver<()>) {
loop {
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(30)) => {},
_ = shutdown.changed() => return,
}
let Some(store) = weak.upgrade() else {
return;
};
store.lock().unwrap().expire(Instant::now());
}
}
impl BashOutputWriter {
pub(crate) fn id(&self) -> String {
self.record.lock().unwrap().id.clone()
}
pub(crate) fn finish_cleanup(&mut self, diagnostics: CleanupDiagnostics) {
self.record.lock().unwrap().cleanup = Some(diagnostics);
}
pub fn append_chunk(&mut self, chunk: &workdir::CommandByteChunk) -> Result<(), String> {
let expected = {
let r = self.record.lock().unwrap();
match chunk.stream {
CommandStream::Stdout => r.stdout_seen,
CommandStream::Stderr => r.stderr_seen,
}
};
if chunk.offset != expected || chunk.bytes.is_empty() {
return Err("command byte transport stream gap or empty chunk".into());
}
self.append(chunk.stream, &chunk.bytes);
Ok(())
}
pub fn append(&mut self, stream: CommandStream, bytes: &[u8]) {
let mut r = self.record.lock().unwrap();
let seen = match stream {
CommandStream::Stdout => &mut r.stdout_seen,
CommandStream::Stderr => &mut r.stderr_seen,
};
let start = *seen;
*seen = seen.saturating_add(bytes.len() as u64);
let available = COMMAND_BYTES.saturating_sub(r.stdout.len() + r.stderr.len());
let n = if r.truncated || r.order.len() >= ORDER_COUNT {
0
} else {
available.min(bytes.len())
};
if n < bytes.len() {
r.truncated = true;
if r.limit.is_none() {
r.limit = Some(if r.order.len() >= ORDER_COUNT {
RetentionLimit::OrderQuota
} else {
RetentionLimit::ByteQuota
});
}
}
if n > 0 {
match stream {
CommandStream::Stdout => r.stdout.extend_from_slice(&bytes[..n]),
CommandStream::Stderr => r.stderr.extend_from_slice(&bytes[..n]),
}
if let Some(last) = r
.order
.last_mut()
.filter(|last| last.stream == stream && last.end == start)
{
last.end += n as u64;
} else {
r.order.push(OutputRange {
stream,
start,
end: start + n as u64,
});
}
}
}
pub fn finish_transport(
&mut self,
terminal: &workdir::CommandTerminal,
) -> Result<BashOutputMetadata, String> {
let mut r = self.record.lock().unwrap();
if !terminal.metadata_valid()
|| terminal.stdout_delivered_bytes != r.stdout_seen
|| terminal.stderr_delivered_bytes != r.stderr_seen
{
return Err("command terminal counts do not fence delivered bytes".into());
}
r.transport_counts = Some((terminal.stdout_bytes, terminal.stderr_bytes));
r.transport_truncation = terminal.truncation_reason.clone();
r.terminal = Some(OutputTerminal {
status: Some(terminal.status),
exit_code: terminal.exit_code,
transport_complete: matches!(
terminal.output_collection,
workdir::CommandOutputCollection::Complete
),
process_failure: terminal.process_failure,
output_collection: Some(terminal.output_collection),
});
Ok(r.metadata())
}
pub fn finish(&mut self, terminal: OutputTerminal) -> BashOutputMetadata {
let mut r = self.record.lock().unwrap();
r.terminal = Some(terminal);
r.metadata()
}
}
impl Drop for BashOutputWriter {
fn drop(&mut self) {
let mut r = self.record.lock().unwrap();
if r.terminal.is_none() {
r.terminal = Some(OutputTerminal {
status: None,
exit_code: None,
transport_complete: false,
process_failure: None,
output_collection: None,
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn finish(view: &BashOutputView, text: &[u8]) -> String {
let mut writer = view.begin().unwrap();
writer.append(CommandStream::Stdout, text);
writer
.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
process_failure: None,
output_collection: Some(workdir::CommandOutputCollection::Complete),
})
.id
}
#[test]
fn revoked_child_is_denied_while_parent_keeps_evidence_and_sibling_is_denied() {
let root = BashOutputView::new();
let lease = Arc::new(AtomicBool::new(true));
let child = root.child({
let lease = lease.clone();
Arc::new(move || lease.load(Ordering::Acquire))
});
let sibling = root.child(Arc::new(|| true));
let id = finish(&child, b"child");
assert_eq!(
child.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
assert!(sibling.metadata(&id).is_none());
assert!(BashOutputView::new().metadata(&id).is_none());
lease.store(false, Ordering::Release);
assert!(child.metadata(&id).is_none());
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
root.revoke();
let restored = BashOutputView::new();
assert!(root.metadata(&id).is_none());
assert!(restored.metadata(&id).is_none());
let new_id = finish(&restored, b"new incarnation");
assert_ne!(new_id, id);
assert!(child.metadata(&new_id).is_none());
assert!(root.begin().is_err());
}
#[test]
fn parent_and_descendants_share_admission_without_evicting_live_evidence() {
let root = BashOutputView::new();
let child = root.child(Arc::new(|| true));
let sibling = root.child(Arc::new(|| true));
let mut writers = (0..STORE_BYTES / COMMAND_BYTES)
.map(|index| {
if index % 2 == 0 {
root.begin()
} else {
child.begin()
}
.unwrap()
})
.collect::<Vec<_>>();
assert!(sibling.begin().is_err());
let mut first = writers.remove(0);
first.append(CommandStream::Stdout, b"finished");
let metadata = first.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
process_failure: None,
output_collection: Some(workdir::CommandOutputCollection::Complete),
});
assert!(root.metadata(&metadata.id).is_some());
let _replacement = sibling.begin().unwrap();
assert!(root.metadata(&metadata.id).is_none());
assert!(root.begin().is_err());
assert_eq!(writers.len(), STORE_BYTES / COMMAND_BYTES - 1);
}
#[test]
fn quota_and_forced_drop_are_distinct_from_preview_omission() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, &vec![b'x'; COMMAND_BYTES + 1]);
writer.append(CommandStream::Stderr, b"tail");
let id = writer.record.lock().unwrap().id.clone();
drop(writer);
let m = root.metadata(&id).unwrap();
assert_eq!(m.stdout_saved, COMMAND_BYTES);
assert_eq!(m.stdout_seen, COMMAND_BYTES as u64 + 1);
assert_eq!(m.stderr_seen, 4);
assert!(m.storage_truncated);
assert!(!m.terminal.transport_complete);
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, usize::MAX)
.unwrap()
.len(),
64 * 1024
);
}
#[test]
fn expiry_and_count_eviction_never_reuse_old_references() {
let root = BashOutputView::new();
let id = finish(&root, b"old");
root.store.lock().unwrap().retention = Duration::ZERO;
assert!(root.metadata(&id).is_none());
root.store.lock().unwrap().retention = RETENTION;
let first = finish(&root, b"first");
for _ in 0..STORE_COUNT {
finish(&root, b"later");
}
assert!(root.metadata(&first).is_none());
assert!(root.store.lock().unwrap().reserved <= STORE_BYTES);
}
#[tokio::test(start_paused = true)]
async fn expiry_reclaims_bytes_without_another_read_and_reaper_ends_with_owner() {
// Exercise host construction outside a runtime, followed by real async admission.
let root = std::thread::spawn(BashOutputView::new).join().unwrap();
finish(&root, b"expires without lookup");
let mut closed = root.store.lock().unwrap()._shutdown.subscribe();
tokio::task::yield_now().await;
// The store uses monotonic wall time for publication and Tokio time only for scheduling.
// Age the record deterministically rather than delaying this test in real time.
root.store.lock().unwrap().records[0]
.lock()
.unwrap()
.created = Instant::now() - RETENTION;
tokio::time::advance(Duration::from_secs(31)).await;
tokio::task::yield_now().await;
assert_eq!(root.store.lock().unwrap().reserved, 0);
drop(root);
assert!(closed.changed().await.is_err());
}
#[test]
fn terminal_counts_must_fence_the_received_prefix_before_marking_evidence_complete() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, b"abc");
let mut terminal = serde_json::json!({"status":"completed","exit_code":0,"process_failure":null,"output_collection":{"state":"complete"},"stdout_bytes":9,"stderr_bytes":0,"stdout_delivered_bytes":4,"stderr_delivered_bytes":0,"truncation_reason":"byte_budget"});
assert!(
writer
.finish_transport(&serde_json::from_value(terminal.clone()).unwrap())
.is_err()
);
terminal["stdout_delivered_bytes"] = serde_json::json!(3);
let metadata = writer
.finish_transport(&serde_json::from_value(terminal).unwrap())
.unwrap();
assert_eq!(metadata.stdout_saved, 3);
assert_eq!(metadata.stdout_delivered, 3);
assert_eq!(metadata.stdout_seen, 9);
assert!(metadata.storage_truncated);
assert!(metadata.retention_limit.is_none());
assert_eq!(
metadata.transport_truncation,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
}
#[test]
fn incomplete_collection_with_quota_retains_both_causes_without_claiming_complete_output() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, b"abc");
let terminal: workdir::CommandTerminal = serde_json::from_value(serde_json::json!({
"status":"completed", "exit_code":0, "process_failure":null,
"output_collection":{"state":"incomplete","failure":{"stage":"stdout_drain","reason":"other","os_code":null}},
"stdout_bytes":9, "stderr_bytes":0, "stdout_delivered_bytes":3, "stderr_delivered_bytes":0,
"truncation_reason":"byte_budget"
})).unwrap();
let metadata = writer.finish_transport(&terminal).unwrap();
assert!(!metadata.terminal.transport_complete);
assert_eq!(metadata.terminal.exit_code, Some(0));
assert_eq!(
metadata.terminal.output_collection,
Some(terminal.output_collection)
);
assert_eq!(
metadata.transport_truncation,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
assert_eq!(metadata.stdout_seen, 9);
assert_eq!(metadata.stdout_saved, 3);
assert!(metadata.storage_truncated);
assert!(metadata.retention_limit.is_none());
assert_eq!(
root.read(&metadata.id, CommandStream::Stdout, 0, 100)
.unwrap(),
b"abc"
);
assert_eq!(
serde_json::to_value(root.metadata(&metadata.id).unwrap()).unwrap(),
serde_json::to_value(metadata).unwrap()
);
}
}
+68
View File
@@ -403,3 +403,71 @@ fn check_pattern(pattern: &str) -> Result<(), ToolError> {
#[cfg(test)]
#[path = "checkout_tests.rs"]
mod tests;
/// Read a bounded native text window through the same checked target.
/// The returned one-shot action records prior-Read only after the caller's presentation succeeds.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_text(
router: Arc<WorkdirSessionRouter>,
tracker: Tracker,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: usize,
limit: usize,
max_bytes: usize,
) -> Result<(workdir::ReadResult, Option<Vec<u8>>, impl FnOnce()), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let tracker = tracker.scoped_attachment(&selected.alias, selected.generation);
let target = FileTarget {
session: selected.session.clone(),
path: path.clone(),
validator: Some(validator),
};
let (result, validator) = target.read(offset, limit, max_bytes).await?;
let hash = result.content_hash.clone();
let total_lines = result.total_lines;
Ok((result, validator, move || {
tracker.record_workdir_observation(&path, hash, total_lines);
}))
}
/// Binary-safe checked byte window. This does not establish a text prior-Read observation.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_bytes(
router: Arc<WorkdirSessionRouter>,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: u64,
max_bytes: usize,
) -> Result<(workdir::ReadBytesResult, Vec<u8>), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let result = selected
.session
.checkout_execute(workdir::CheckoutRequest {
target: path.clone(),
validator,
operation: workdir::CheckoutOperation::ReadBytes { offset, max_bytes },
})
.await
.map_err(crate::file_target::checked_error)?;
match result.output {
workdir::CheckoutOutput::ReadBytes(read)
if read.path == path && read.offset == offset && read.bytes.len() <= max_bytes =>
{
Ok((read, result.observation.validator))
}
_ => Err(crate::file_target::readonly_result_error("ReadBytes")),
}
}
+1 -1
View File
@@ -21,7 +21,7 @@ pub(crate) struct EditParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
pub target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
/// Existing file in which to replace text.
pub file_path: String,
#[serde(flatten)]
pub edit: fs_operation::text::EditArgs,
+4 -1
View File
@@ -147,7 +147,9 @@ impl FileTarget {
pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
use workdir::WorkdirError;
let code = match error.classification_source() {
WorkdirError::DenialContext { .. } => unreachable!("diagnostic context unwrapped"),
WorkdirError::DenialContext { .. } | WorkdirError::FailureContext { .. } => {
unreachable!("diagnostic context unwrapped")
}
WorkdirError::Conflict(_) | WorkdirError::NotFound(_) => "checkout_stale",
WorkdirError::Denied { .. }
| WorkdirError::OutOfScope { .. }
@@ -165,6 +167,7 @@ pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
| WorkdirError::IsDirectory { .. }
| WorkdirError::InvalidGlob { .. }
| WorkdirError::InvalidRegex { .. } => "checkout_invalid",
WorkdirError::NotDispatched => "checkout_not_dispatched",
WorkdirError::Unavailable { .. } | WorkdirError::SessionClosed => "checkout_unavailable",
// IO/transport failures may arrive after effects have begun. Do not infer
// no-effects from their text or strip explicit OutcomeUnknown evidence.
+2 -2
View File
@@ -17,7 +17,7 @@ pub(crate) struct GlobParams {
pub target_workdir: Option<String>,
/// Glob pattern, for example `**/*.rs` or `src/**/test_*.py`.
pub pattern: String,
/// Logical Workdir-relative directory. Defaults to the Workdir root.
/// Directory to search. Defaults to `.`.
#[serde(default)]
pub path: Option<String>,
}
@@ -106,7 +106,7 @@ pub(crate) fn routed_glob_tool(router: Arc<WorkdirSessionRouter>) -> ToolDefinit
Arc::new(move || {
let schema = schemars::schema_for!(GlobParams);
let meta = ToolMeta::new("Glob")
.description("Find files matching a glob pattern inside the selected Workdir. Results are sorted and capped at 1000 entries. Paths are Workdir-relative.")
.description("Find files matching a glob pattern inside the selected Workdir. Results are sorted and capped at 1000 entries.")
.input_schema(serde_json::to_value(schema).expect("Glob schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(GlobTool {
router: router.clone(),
+2 -2
View File
@@ -27,7 +27,7 @@ pub(crate) struct GrepParams {
#[serde(default)]
target_workdir: Option<String>,
pattern: String,
/// Workdir-relative path, or an absolute path covered by readable scope. Defaults to the Workdir root.
/// File or directory to search. Defaults to `.`.
#[serde(default)]
pub path: Option<String>,
#[serde(default)]
@@ -160,7 +160,7 @@ pub(crate) fn routed_grep_tool(router: Arc<WorkdirSessionRouter>) -> ToolDefinit
Arc::new(move || {
let schema = schemars::schema_for!(GrepParams);
let meta = ToolMeta::new("Grep")
.description("Search a file or directory in the selected Workdir with a regex. Content results group lines by file; `>` marks matching lines and unmarked lines are context. Directory traversal executes inside the WorkdirSession provider. Results are bounded and Workdir-relative.")
.description("Search a file or directory in the selected Workdir with a regex. Content results group lines by file; `>` marks matching lines and unmarked lines are context. Directory traversal executes inside the WorkdirSession provider. Results are bounded.")
.input_schema(serde_json::to_value(schema).expect("Grep schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(GrepTool {
router: router.clone(),
+10 -9
View File
@@ -5,6 +5,7 @@ pub mod error;
pub mod tracker;
mod bash;
pub mod bash_output;
mod edit;
mod file_target;
mod glob;
@@ -16,7 +17,7 @@ pub mod view_image;
mod web;
mod write;
pub use bash::bash_tool;
pub use bash::{BashResult, BashTool, bash_tool};
pub use checkout::{CheckoutToolOutput, execute_checkout_tool};
pub use edit::edit_tool;
pub use error::ToolsError;
@@ -35,7 +36,7 @@ pub use write::write_tool;
pub fn core_builtin_tools(
session: workdir::WorkdirSessionHandle,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -58,7 +59,7 @@ pub fn core_builtin_tools(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -71,7 +72,7 @@ pub fn core_builtin_tools(
pub fn routed_builtin_tools(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
vec![
read::routed_read_tool(router.clone(), tracker.clone()),
@@ -79,14 +80,14 @@ pub fn routed_builtin_tools(
edit::routed_edit_tool(router.clone(), tracker),
glob::routed_glob_tool(router.clone()),
grep::routed_grep_tool(router.clone()),
bash::routed_bash_tool(router, bash_output_dir),
bash::routed_bash_tool(router, bash_output),
]
}
pub fn routed_builtin_tools_for_current_capabilities(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -108,7 +109,7 @@ pub fn routed_builtin_tools_for_current_capabilities(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -121,7 +122,7 @@ pub fn read_only_builtin_tools(
workdir::WorkdirSessionCapabilities::READ_ONLY,
"read-only tool projection requires a read-only Workdir session"
);
core_builtin_tools(session, Tracker::new(), std::path::PathBuf::new())
core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
}
pub fn read_only_routed_builtin_tools(
@@ -173,7 +174,7 @@ mod workdir_tool_tests {
SharedScope::new(Scope::writable(dir.path()).unwrap()),
WorkdirSessionCapabilities::READ_ONLY,
));
let names = core_builtin_tools(session, Tracker::new(), dir.path().join("output"))
let names = core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
.into_iter()
.map(|definition| definition().0.name)
.collect::<Vec<_>>();
+2 -2
View File
@@ -13,7 +13,7 @@ use workdir::{WorkdirPath, WorkdirSessionHandle, WorkdirSessionRouter};
const DESCRIPTION: &str = "Read a text file from a Workdir attachment selected by its Worker-local alias. \
Supports offset/limit for large files. Returns line-numbered output (1-based). \
Directories cannot be read. The file must be read before Write or Edit can \
modify it. Paths are Workdir-relative unless an absolute path is explicitly readable. \
modify it. \
When exactly one Workdir is attached, target_workdir may be omitted.";
const DEFAULT_LIMIT: usize = 2000;
@@ -24,7 +24,7 @@ pub(crate) struct ReadParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
pub target_workdir: Option<String>,
/// Workdir-relative path, or an absolute path covered by readable scope.
/// Text file to read; directories are not supported.
pub file_path: String,
#[serde(flatten)]
pub read: fs_operation::text::LineReadArgs,
+46 -4
View File
@@ -20,7 +20,7 @@ pub const MAX_IMAGE_BYTES: usize = 10 * 1024 * 1024;
const IMAGE_READ_CHUNK_BYTES: usize = 1024 * 1024;
const DESCRIPTION: &str = "Attach an image from the selected Workdir attachment to the next model request. \
The path must be logical and Workdir-relative. Supported formats: PNG, JPEG, GIF, and WebP. \
Supported formats: PNG, JPEG, GIF, and WebP. \
When exactly one Workdir is attached, target_workdir may be omitted.";
#[derive(Debug, Deserialize, schemars::JsonSchema)]
@@ -28,7 +28,7 @@ struct ViewImageParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
/// Image file to attach to the next model request.
path: String,
}
@@ -360,10 +360,52 @@ mod tests {
}
#[tokio::test]
async fn bounded_binary_read_accepts_arbitrary_bytes_at_caller_limit_and_rejects_larger_files()
{
async fn bounded_binary_read_accepts_caller_limits_above_the_image_tool_limit() {
// The caller's limit is a reader policy, not a LocalWorkdir storage contract.
// Supply immutable chunks here; the separate component case exercises the
// real provider across a chunk boundary without repeatedly hashing 16 MiB.
let dir = TempDir::new().unwrap();
let bytes = vec![0xff; 16 * 1024 * 1024];
let hash = Sha256::digest(&bytes).into();
let responses = bytes
.chunks(IMAGE_READ_CHUNK_BYTES)
.enumerate()
.map(|(index, bytes)| {
chunk(
bytes,
(index * IMAGE_READ_CHUNK_BYTES) as u64,
16 * 1024 * 1024,
hash,
)
})
.collect();
let session = binary_session(&dir, responses);
let path = WorkdirPath::new("artifact.bin").unwrap();
assert_eq!(
read_workdir_bytes(&session, path.clone(), bytes.len())
.await
.unwrap(),
bytes
);
let session = binary_session(
&dir,
vec![chunk(
&bytes[..IMAGE_READ_CHUNK_BYTES],
0,
bytes.len() as u64,
hash,
)],
);
assert!(matches!(
read_workdir_bytes(&session, path, bytes.len() - 1).await,
Err(WorkdirError::InvalidArgument(_))
));
}
#[tokio::test]
async fn bounded_binary_read_reassembles_provider_chunks_and_enforces_the_caller_limit() {
let dir = TempDir::new().unwrap();
let bytes = vec![0xff; IMAGE_READ_CHUNK_BYTES + 1];
std::fs::write(dir.path().join("artifact.bin"), &bytes).unwrap();
let session =
LocalWorkdirSession::new(Scope::writable(dir.path()).unwrap(), dir.path().into());

Some files were not shown because too many files have changed in this diff Show More