129 Commits
Author SHA1 Message Date
Hare 5b12c3e085 fix: make mobile sidebar swipes track touch gestures 2026-10-27 06:49:28 +09:00
Hare 623c31615d fix: assign unique event refs to merge integration comments 2026-10-27 06:15:48 +09:00
Hare af719e387d fix: align reviewer guidance with Bash-enabled scope 2026-10-27 06:15:48 +09:00
Hare d12bc2ea0b fix: deduplicate live and committed task updates 2026-10-26 10:13:36 +09:00
Hare a1df3742b1 fix: integrate passkey results with current dogfooding shell
Preserve authenticated home/account navigation and guarded mobile sidebar swipes. Keep both browser task selections. Complete the new account fixture contract and synchronize lifecycle probes on visible registered-key results, without changing their success or security assertions.
2026-10-26 09:50:13 +09:00
Hare d8818b776b feat: open and close mobile sidebar with guarded swipes 2026-10-26 09:39:35 +09:00
Hare 40c36ccb1d fix: restore authenticated home and account navigation 2026-10-26 09:39:35 +09:00
Hare 80c151d084 fix: fence recovered passkey results against cross-tab changes 2026-10-26 07:52:06 +09:00
Hare c6a8759e7b fix: fence activation and setup against delayed cross-tab events
Capture the shared authentication stamp as well as generation before dispatch and across response/confirmation/application awaits. Do not let stale setup failure reconciliation overwrite a newer tab's stamp. Add no-event-yet logout and account-change regressions for activation and setup, including prompt, pre-complete and post-dispatch confirmation boundaries.

Addresses MR AC4_CROSS_TAB_ACTIVATION. Validation: activation21/setup40 narrow tests; Web720 unit+625 component, check0/0/build, activation and hosting browser suites; root check/fmt/diff; both explicit real Backend browser suites including native CTAP2. Existing Server exclusions and 15 identical baseline naming findings remain unchanged. No verifier, binding, migration or rendering contract weakened.
2026-10-26 07:34:04 +09:00
Hare f17b5212c7 fix(auth): confirm passkey operations and preserve safe legacy migration
Fence displayed identity and native prompt generations, classify secret-free outcomes, recover atomic addition receipts without restoring old credentials, and expose truthful own-key verification metadata. Preserve required verifier and enrollment authority boundaries. Include real-store, production browser, native CTAP2 and visible responsive regressions.

Validation: root check/fmt; API 115; passkey security 25; Web unit 712/component 605; auth/entry/activation/hosting browsers; product auth E2E 8 and two explicit browser suites. Server suite: 1262 lib+12 CLI, 3 ignored and 2 environment-specific exclusions. State-name checker retains 15 byte-identical baseline findings; no new exceptions. Physical keys/mobile and hosted CI remain unverified.
2026-10-26 07:18:23 +09:00
Hare b284934e34 test: add captured Codex worker restore fixture 2026-10-24 12:57:28 +09:00
Hare 2d33321fab fix: validate live fixture snapshots and shutdown boundaries 2026-10-24 12:57:28 +09:00
Hare 0975471986 fix: preserve diagnostic context in backend test assertions 2026-10-24 12:51:28 +09:00
Hare 214d6e21dd fix: assert classified workdir session closure in tests 2026-10-24 12:40:37 +09:00
Hare fca5c23345 fix: reject unrestorable and opaque fixture records 2026-10-24 12:39:22 +09:00
Hare f9b1500ba2 fix: align panel tests with current ticket case states 2026-10-24 12:33:31 +09:00
Hare 931bf3b27d test: add isolated saved worker restore scenarios 2026-10-24 12:33:31 +09:00
Hare 6bb671be21 test: add typed stopped worker fixture transport 2026-10-24 12:19:43 +09:00
Hare b8f2fa479c test: add isolated backend fixture seeding 2026-10-24 12:15:10 +09:00
Hare 1639be9841 fix: allow reviewers to read related tickets 2026-10-23 03:22:45 +09:00
Hare aa0fb2f787 fix: continue workdir output reads after long-poll expiry 2026-10-21 04:48:43 +09:00
Hare 1b6d4dbfd8 fix: preserve workdir failure diagnostics across transport 2026-10-21 01:22:04 +09:00
Hare 4ef7b03651 fix: stop background session checks and loading flicker 2026-10-20 22:35:38 +09:00
Hare af615ab177 fix: retain archive and worker restore failure diagnostics 2026-10-20 22:00:58 +09:00
Hare 338e39bcdb fix: defer restored workdir access until use 2026-10-20 22:00:58 +09:00
Hare 3cfe04be3a fix: preserve saved feature configuration during worker restore 2026-10-20 22:00:39 +09:00
Hare 70d153b365 fix: isolate unreadable archives from runtime startup 2026-10-20 17:39:38 +09:00
Hare ce93849948 chore: merge develop into hare/develop 2026-10-20 15:39:09 +09:00
Hare 3c499719fe feat: default Coder ticket work to review flow 2026-10-20 15:38:50 +09:00
Hare 9e1d3df7ad fix: remove overbroad Worker bypass instructions 2026-10-20 15:38:50 +09:00
Hare 18bac16a23 fix: require replay-safe control lifecycle for Subject delegation 2026-10-20 08:12:26 +09:00
Hare 9005856f77 fix: replay accepted delegation without reserving active Workdirs 2026-10-20 07:24:51 +09:00
Hare e39372a52a Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants 2026-10-20 06:38:12 +09:00
Hare 223a74a9e2 fix: retain byte command transport and explicit authorization fixtures 2026-10-20 06:38:12 +09:00
Hare f38d9a16ea Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants
# Conflicts:
#	crates/worker-runtime/src/worker_backend.rs
#	openapi/server-api.json
#	web/workspace/src/lib/generated/companion-api.ts
#	web/workspace/src/lib/generated/repository-api.ts
#	web/workspace/src/lib/generated/runtime-api.ts
#	web/workspace/src/lib/generated/ticket-api.ts
#	web/workspace/src/lib/generated/worker-launch-api.ts
2026-10-20 06:24:11 +09:00
Hare b135fa5e99 feat: enforce explicit Subject permissions and delegated Worker grants 2026-10-20 06:22:58 +09:00
Hare 07b8b6d41b Merge commit 'b8e548b92d3c785c482dbb418517cc5a2cf9e7e0' into refactor/T-765-session-feature-boundary 2026-10-20 05:34:22 +09:00
Hare 8017c55d82 fix: retain annotated compaction history and honor non-System input policy 2026-10-20 05:34:22 +09:00
Hare b8e548b92d chore: integrate reviewed browser session navigation 2026-10-20 05:31:32 +09:00
Hare 90e38551cf Merge commit '00ad02d04728a8e32a82a4e72747fefcf8a82d4e' into refactor/T-765-session-feature-boundary
# Conflicts:
#	crates/worker/src/feature/builtin.rs
2026-10-20 05:25:16 +09:00
Hare 3e8164e757 fix: fence hosting setup verification against superseded sessions 2026-10-20 04:53:43 +09:00
Hare 3b3381900f fix: preserve role-neutral input policy across compaction and replay 2026-10-20 04:45:50 +09:00
Hare 00ad02d047 chore: integrate bounded shared text page refusal 2026-10-20 03:55:13 +09:00
Hare a774925dec fix: reject byte-truncated line windows before advancing search 2026-10-20 03:53:03 +09:00
Hare 8f759911ab fix: emit Session use policy type in browser bindings 2026-10-20 03:52:55 +09:00
Hare daf1f6ed50 fix: route retained Session migration through Runtime staging 2026-10-20 03:52:55 +09:00
Hare 86602240fa refactor: bind Subject metadata and prompt confirmation to Host lifecycle 2026-10-20 03:52:55 +09:00
Hare bfeee487fd feat(auth): separate anonymous entry from expired session recovery 2026-10-20 03:52:48 +09:00
Hare f9830fe7dd test: reject Bash search pages that would skip an unread long line 2026-10-20 03:40:36 +09:00
Hare de96c5b577 chore: update shared file contracts and validation fixtures
# Conflicts:
#	crates/worker/src/checkout_http_tests.rs
2026-10-20 03:26:56 +09:00
Hare 6e729ed06c refactor: isolate Feature System input migration and compaction policy 2026-10-20 03:21:22 +09:00
Hare f38b2a52b1 fix: record native prior Read only after result validation 2026-10-20 03:21:14 +09:00
Hare dd5384df3b feat: expose retained Bash streams through common WIP file contracts 2026-10-20 03:13:46 +09:00
Hare 6576850b7a chore: connect shared file contracts for Bash validation 2026-10-20 03:03:53 +09:00
Hare 4555aace28 feat: capture bounded Bash output over replayable byte transport 2026-10-20 03:01:22 +09:00
Hare 47b32ef6ab fix: bind capability changes to Object freshness and update contract consumers 2026-10-20 02:53:21 +09:00
Hare bb5c50f9ab feat: persist System input policies through public consumers 2026-10-20 02:44:28 +09:00
Hare 32a409c15d feat: share virtual file contracts and checked binary byte ranges 2026-10-20 02:18:49 +09:00
Hare 19514a5d6a feat: define bounded Feature-owned system input envelopes 2026-10-20 02:14:52 +09:00
Hare 887184796e docs: distinguish commit completion from Feature authority 2026-10-20 02:05:02 +09:00
Hare cd0eb5eb00 fix: allowlist public system input projections 2026-10-20 02:02:28 +09:00
Hare e8dfb3e68d refactor: notify system input producers after durable commit 2026-10-20 02:02:28 +09:00
Hare 434ce34da7 refactor: bind checkout target lifetimes to Object observations 2026-10-20 02:01:22 +09:00
Hare bfb51b7ca9 fix: bound encoded common text results including escape and record overhead 2026-10-20 01:48:23 +09:00
Hare 38d9069632 feat: publish stable root file contracts and target-owned dispatch 2026-10-20 01:46:30 +09:00
Hare e5749b9be0 test: integrate Drive connection and lifetime coverage 2026-10-20 00:37:32 +09:00
Hare 387b5ff7df docs: record Drive connection coverage and fault evidence 2026-10-20 00:35:11 +09:00
Hare ec6c5beda0 test: connect Drive publication to client and resource lifetimes 2026-10-20 00:34:50 +09:00
Hare 8e40fae6f4 fix: recover unpublished Subject creation and preserve auth rejections 2026-10-19 18:56:24 +09:00
Hare c961f244ab fix: preserve feature metadata for concise Journal startup context 2026-10-19 18:45:32 +09:00
Hare cea0225a6e fix: migrate legacy session history and retained archives 2026-10-19 13:52:43 +09:00
Hare b3189d8e85 fix: preserve worker retention failure diagnostics 2026-10-19 13:52:43 +09:00
Hare 24bc45a31e chore: merge develop into hare/develop 2026-10-19 01:31:23 +09:00
Hare b02360fd75 fix: restore orchestrator target push requirement 2026-10-19 01:31:01 +09:00
Hare 41c9d2d5c7 docs: record restored parent regression evidence and cost 2026-10-19 00:02:02 +09:00
Hare cc66bd5052 test: connect restored Runtime parent tools to child session 2026-10-19 00:02:02 +09:00
Hare d1408d3a2f fix: project runtime list subjects for Worker control tools 2026-10-19 00:02:02 +09:00
Hare 19122fb5ea fix: use public Worker control entry in regression design 2026-10-19 00:02:02 +09:00
Hare 1fb216e6ae fix: refresh Ticket regression design evidence 2026-10-19 00:02:02 +09:00
Hare 4b14f7e6b3 docs: design prioritized regression coverage 2026-10-19 00:02:02 +09:00
Hare 67767eeee7 test: integrate Ticket public regression coverage 2026-10-18 02:35:16 +09:00
Hare 1bfb4ce119 fix: align Ticket regression design with Host retry identity 2026-10-18 02:10:15 +09:00
Hare 651820dc6d test: connect Ticket public regressions to real Backend 2026-10-18 02:10:15 +09:00
Hare 75b1557d63 feat: browse Subject Journals and persistent Memory links 2026-10-18 02:03:51 +09:00
Hare 23ff518209 fix: clarify Ticket regression fixture boundaries 2026-10-18 01:26:52 +09:00
Hare fab9e0f4b5 docs: design focused Ticket regression coverage 2026-10-18 01:26:52 +09:00
Hare 5ee42b8db8 docs: record Journal compaction validation and evaluation limits 2026-10-17 12:29:47 +09:00
Hare daf438701f docs: explain shared Journal snapshot boundary guarantees 2026-10-17 12:25:24 +09:00
Hare 23ee401287 fix: disambiguate historical Journal retention instructions 2026-10-17 12:25:24 +09:00
Hare 8f598da931 test: verify frozen Journal compaction across Hosts and restores 2026-10-17 12:25:24 +09:00
Hare 956aeed6b2 feat: freeze Host Journal context across compaction 2026-10-17 12:19:37 +09:00
Hare adfa422973 chore: merge develop into hare/develop 2026-10-17 12:06:05 +09:00
Hare 10d200d9ba chore: align Journal references with integrated develop 2026-10-17 03:52:11 +09:00
Hare 7a5228b8e6 fix: reconcile Memory Journal links with handoff fencing 2026-10-17 03:52:11 +09:00
Hare 0eabdf3a1f chore: integrate Journal-gated Subject handoff 2026-10-17 03:47:11 +09:00
Hare b1808b7c4d fix: unblock MR follow-up and simplify orchestration guidance 2026-10-17 03:35:58 +09:00
Hare 6645ea0a47 fix: observe original presence through saved legacy Journal links 2026-10-17 00:10:35 +09:00
Hare b544e53bd7 feat: connect Memory provenance to durable Journal context 2026-10-16 23:54:54 +09:00
Hare 04a4e59100 feat: fence Subject handoff on recorded Journal and seed new bodies once 2026-10-16 20:51:08 +09:00
Hare 52e914eebc feat: offer ID-free passkey login and explicit legacy migration 2026-10-16 19:01:43 +09:00
Hare 9c12fb4372 feat: verify discoverable passkeys against their saved owners 2026-10-16 19:01:43 +09:00
Hare 3c18916118 test: wait for complete isolated server address records 2026-10-16 19:01:43 +09:00
Hare be1c030350 chore: merge develop into hare/develop 2026-10-16 15:08:40 +09:00
Hare a7faf2cea6 docs: record test optimization results and runner validation 2026-10-16 15:04:52 +09:00
Hare 5cc5ece997 feat: add bounded parallel runner for the normal test suite 2026-10-16 15:04:52 +09:00
Hare ffaaf2ecb0 test: isolate local rules and reduce repeated fixture setup 2026-10-16 15:04:52 +09:00
Hare 6667985883 fix: preserve test completion and fixture invariants under concurrency 2026-10-16 15:04:42 +09:00
Hare 22dcd8a29f fix: make reverse session reads linear for large records 2026-10-16 15:04:42 +09:00
Hare a2816a053d fix: isolate Workspace notifications from public entry 2026-10-16 11:30:00 +09:00
Hare d74660d1d1 fix: retain workspace creation state across session rechecks 2026-10-16 11:08:27 +09:00
Hare 24c6f772f7 fix: serve entry shells independently of workspace catalog 2026-10-16 10:48:51 +09:00
Hare a060ffada8 feat: add public entry and authenticated workspace dashboard 2026-10-16 10:48:51 +09:00
Hare b8b12442a0 feat: separate login registration and own account flows 2026-10-16 09:43:57 +09:00
Hare d22899d39e feat: activate provisional users with scoped passkey enrollment 2026-10-16 08:48:35 +09:00
Hare af008f802d feat: explicit hosting administrators and persistent self-registration policy 2026-10-16 07:26:21 +09:00
Hare 7042ec7d50 chore: merge develop into hare/develop 2026-10-16 05:52:36 +09:00
Hare a2cec5afd1 fix: preserve console alert order and improve alert colors 2026-10-16 05:28:17 +09:00
Hare b5a307811b fix: reconcile erased Archive removals and exercise live compaction recording
Reconcile immutable Runtime removal input under a confirmed permanent erase fence. A confirmed source_removed receipt is audited without republishing erased archives; unknown receipt plus attested remaining aggregate permits an audited stale plan and proof-bound purge replacement. Unknown receipt and inventory remain unresolved. Distinguish domain tombstones from scoped Runtime physical erasure acknowledgements.

Erase diagnostic originals and interrupted staging bound to the same archived Session/removal operation, validating Workspace/Worker/fingerprint and saved receipts without rewriting their historical retained flags. Finalize Backend removal only after physical acknowledgement, including previous Retain disposition.

Connect public standalone protocol, real Worker request-threshold compaction twice within one turn, cancellation and real pre-AI input rollback, original canonical readers, existing Jobs and sequential Journal tools. Gate generation while compaction completes; verify all durable originals were served, summary/rolled-back input exclusion, exact final cursor, no extra input for successor drain, and one ongoing open entry. External model streams/usage are scripted, not semantic evaluation.

Validation: Backend1112lib+11bin (existing ignore1), Runtime429lib+38bin, standalone72lib+27integration (existing model ignore1), runtime-api10, root check/doc/fmt/diff/naming, Server/Runtime builds, isolated worker_environment2 passed. Earlier broad suites remain recorded on the prior source. Fixtures and injected boundaries are not arbitrary crash/remote-transport-loss coverage; worker_environment only verifies startup/connection/cleanup.

Acceptance clarification: T-734 assignment answer event22 explicitly defers real-model semantic condition6 to user use, excluding it from this completion scope. It remains unexecuted and unevaluated, not passed. Technical review findings still require a fresh independent verdict on this source.
2026-10-16 04:58:53 +09:00
Hare 5bd49760e5 fix: fence Journal originals before publication and reconcile removal intent
Protect attributed originals before the first public commit; distinguish pending Session material from missing committed sources. Gate removal admission by the Journal Feature and resolve Subject ownership from durable singleton authority. Resume explicit erasure through signed delete transport and proof-bound purge plans without creating a Journal Job or archive.

Validation: changed-crate suites, root check, doc/fmt/diff/naming checks, Server/Runtime builds and worker_environment isolation suite passed. Public standalone scripted dialog connects Host, automatic Job, sequential tools and drain while dialog continues. Backend public embedded WorkerRemove tests connect removal forwarding, durable Jobs, retention, Runtime and SQLite; Session/attribution/profile fixture setup bypasses creation/auth routes. Runtime signed generated-client tests separately exercise request authentication.

Limits: real-model semantic acceptance condition 6 remains unevaluated without a new explicit model/credential/budget grant. Conditions 1/7/8/11 do not have complete product E2E coverage: worker_environment only verifies isolated startup/connection/cleanup. Compaction tests connect local hooks/reader/store rather than the full public dialog-to-Journal chain. Erasure racing a previously prepared Archive removal plan fails closed with conflict; automatic stale-plan replacement is not implemented.
2026-10-16 03:59:45 +09:00
Hare 9debf43645 Merge remote-tracking branch 'origin/develop' into work/T-734-journal-recording 2026-10-16 02:19:24 +09:00
Hare 81af1f085c feat: record confirmed public Session history through durable Journal Jobs 2026-10-16 02:19:24 +09:00
Hare b67aa2de7d fix: simplify Ticket reporting and speed up browser lists
Remove model-supplied assignment and retry identities from implementation reports; retain Host idempotency, authentication and notification evidence.

Add a SQL-only browser projection and decouple auxiliary page loads. Preserve authoritative MR evidence queries. Include regression tests and component benchmarks, documenting the remaining no-match search cost.
2026-10-16 01:58:15 +09:00
Hare 9f1cbbd385 fix: inherit parent profile for default subworkers 2026-10-16 01:57:58 +09:00
Hare 8a3d9d67d2 fix: guard deferred state when leaving workspace routes 2026-10-16 01:57:58 +09:00
Hare 6218c27fb6 fix: separate account registration from authenticated passkey addition
Bind enrollment purpose and allocated/current account to verifier state. Reject existing-handle registration, derive additions from live transport authentication, recheck revocation/expiry in the insert-only transaction, and consume challenges atomically across connections. Derive stable account WebAuthn handles without rewriting legacy credentials; reject implicit-user login and unsafe legacy ceremonies.

Align account callers, DTO/OpenAPI/generated artifacts and migration/recovery rationale. Preserve credential-less legacy identities rather than allowing anonymous reclaim.

Validation: 14 actual-verifier passkey tests; full workspace-server 1086 library + 11 binary passed (Drive ignored case explicitly selected separately and passed); server-api default 99 and TypeScript feature 108 passed; isolated auth/Runtime E2E 1+2 passed; Web 645 + component 394 passed; root cargo check, docs/doctests, Server/Runtime builds, fmt, naming and generated consistency passed. Software authenticator only; hosted CI/hardware login not claimed.
2026-10-16 00:55:16 +09:00
Hare b744efd104 fix: align isolated runtime harness with current trust contract
Keep Workspace trust enrollment and owner-authenticated registration, then verify each Runtime connection with a fresh signed request and match its binding ID. Remove obsolete workspace_trust_id input and cached connection_state assertions.

Validation: cargo test -p yoi-e2e --features e2e --test worker_environment --test auth_passkeys -- --nocapture (2 Runtime harness tests and 1 Passkey product E2E passed); root cargo check.
2026-10-16 00:35:23 +09:00
Hare 18384482f6 feat: integrate durable Journal storage APIs 2026-10-15 20:51:51 +09:00
Hare c2df3041f9 fix: pin Journal source pagination to saved coverage content 2026-10-14 22:18:02 +09:00
Hare aabc502f30 feat: add durable subject Journal operations and bounded reads 2026-10-14 21:57:35 +09:00
494 changed files with 120140 additions and 6842 deletions
Generated
+14
View File
@@ -5266,6 +5266,7 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"tracing",
"uuid",
"workdir",
]
@@ -6548,6 +6549,7 @@ dependencies = [
"session-metrics",
"session-store",
"sha2 0.11.0",
"subjektiv",
"tempfile",
"thiserror 2.0.18",
"ticket",
@@ -6704,12 +6706,23 @@ dependencies = [
name = "yoi-e2e"
version = "0.0.0"
dependencies = [
"chrono",
"config-source",
"libc",
"manifest",
"protocol",
"reqwest",
"serde",
"serde_json",
"server-api",
"session-store",
"sha2 0.11.0",
"tempfile",
"tungstenite 0.29.0",
"webauthn-authenticator-rs",
"webauthn-rs-proto",
"worker-runtime",
"yoi-workspace-server",
]
[[package]]
@@ -6770,6 +6783,7 @@ dependencies = [
"uuid",
"webauthn-authenticator-rs",
"webauthn-rs",
"webauthn-rs-proto",
"wip-protocol",
"workdir",
"worker",
+10
View File
@@ -91,6 +91,16 @@ debug = "line-tables-only"
[profile.dev.package."*"]
debug = false
# Keep the real SQLite engine in schema/transaction tests without paying for
# an unoptimized C interpreter on every statement. Rust test assertions stay enabled.
[profile.test.package.libsqlite3-sys]
opt-level = 2
# Schema/authority integration tests execute substantial parsing and validation.
# Optimize that code without disabling test/debug assertions or overflow checks.
[profile.test.package.yoi-workspace-server]
opt-level = 1
[workspace.dependencies]
# Internal crates
client = { path = "crates/client" }
+42 -9
View File
@@ -1,13 +1,38 @@
//! Compile the real public macro through a renamed dependency in an isolated consumer.
//! Only Cargo artifacts are shared; Cargo checks the fresh consumer on every invocation.
use std::{fs, path::PathBuf, process::Command};
struct Fixture(PathBuf);
impl Fixture {
fn new() -> Self {
for attempt in 0..1000 {
let path = std::env::temp_dir().join(format!(
"api-macros-renamed-dependency-{}-{attempt}",
std::process::id()
));
match fs::create_dir(&path) {
Ok(()) => return Self(path),
Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue,
Err(error) => panic!("create renamed dependency fixture: {error}"),
}
}
panic!("could not allocate a renamed dependency fixture directory");
}
}
impl Drop for Fixture {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
#[test]
fn macro_expansion_resolves_a_renamed_support_dependency() {
let fixture = std::env::temp_dir().join(format!(
"api-macros-renamed-dependency-{}",
std::process::id()
));
let _ = fs::remove_dir_all(&fixture);
fs::create_dir_all(fixture.join("src")).expect("create fixture source directory");
let fixture = Fixture::new();
let fixture = &fixture.0;
fs::create_dir(fixture.join("src")).expect("create fixture source directory");
let package_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let package_path = package_path
@@ -49,13 +74,21 @@ pub fn operation_count() -> usize {
)
.expect("write fixture source");
// Keep nested Cargo checks off the outer build's target lock. Cargo owns cache
// invalidation and locking between concurrent consumers, not a saved
// test-success marker.
let executable = std::env::current_exe().expect("current test executable");
let target_dir = executable
.parent()
.and_then(|deps| deps.parent())
.expect("test executable has a profile directory")
.join("compile-tests/api-macros-renamed");
let output = Command::new(env!("CARGO"))
.args(["check", "--quiet", "--offline"])
.current_dir(&fixture)
.env("CARGO_TARGET_DIR", fixture.join("target"))
.current_dir(fixture)
.env("CARGO_TARGET_DIR", target_dir)
.output()
.expect("run cargo check for renamed dependency fixture");
let _ = fs::remove_dir_all(&fixture);
assert!(
output.status.success(),
+11 -1
View File
@@ -81,6 +81,16 @@ pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
resolved_path: "profiles/job.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:subjektiv-journal-recording"),
path: "profiles/subjektiv-journal-recording.dcdl",
source: include_str!("../../../resources/profiles/subjektiv-journal-recording.dcdl"),
description: "Journal recording policy with Host-bound sequential Job tools.",
imports: &[BuiltinProfileImport {
specifier: "./job.dcdl",
resolved_path: "profiles/job.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:ticket-worker"),
path: "profiles/ticket-worker.dcdl",
@@ -123,7 +133,7 @@ pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
selector: Some("builtin:orchestrator"),
path: "profiles/orchestrator.dcdl",
source: include_str!("../../../resources/profiles/orchestrator.dcdl"),
description: "Intent-led Ticket orchestration with separate guarded MR integration authority.",
description: "Outcome-led orchestration, selection of candidate changes, and risk-proportionate validation.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
+1
View File
@@ -908,6 +908,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let response = create_backend_worker_with_client(&target, &create, &api)
+1
View File
@@ -567,6 +567,7 @@ impl BackendWorkspaceProductClient {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let workspace_id = self.workspace_id.clone();
let response: BrowserCreateWorkerResponse = self.generated(move |client| async move {
+30
View File
@@ -200,6 +200,7 @@ fn run_read_with_limits(
total_lines,
content_hash: content_hasher.finalize().into(),
truncated: end < total_lines || byte_truncated,
byte_truncated,
})
}
@@ -885,6 +886,34 @@ mod bounded_read_tests {
assert!(error.to_string().contains("exceeds provider limit 4096"));
}
#[test]
fn bounded_read_distinguishes_utf8_byte_cut_from_complete_line_continuation() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
fs::write(root.join("text"), "雪雪needle\nnext").unwrap();
for (offset, max_bytes, bytes, more, cut) in [
(0, 5, "雪", true, true),
(0, 64, "雪雪needle\n", true, false),
(1, 64, "next", false, false),
] {
let result = run_read_bounded(
&root,
ReadRequest {
path: FsPath::new("text").unwrap(),
offset,
limit: 1,
max_bytes,
},
&RootAccess(root.clone()),
BoundedReadLimits::new(1024, 64).unwrap(),
)
.unwrap();
assert_eq!(result.bytes, bytes.as_bytes());
assert_eq!(result.truncated, more);
assert_eq!(result.byte_truncated, cut);
}
}
#[test]
fn bounded_read_caps_retained_response_independently_of_request() {
let temp = tempfile::tempdir().unwrap();
@@ -907,6 +936,7 @@ mod bounded_read_tests {
assert_eq!(result.bytes, b"alpha");
assert_eq!(result.total_lines, 3);
assert!(result.truncated);
assert!(result.byte_truncated);
}
}
+4
View File
@@ -204,7 +204,11 @@ pub struct ReadResult {
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub total_lines: usize,
pub content_hash: ContentHash,
/// More source lines remain, or the selected window exceeded the response byte budget.
pub truncated: bool,
/// The selected line window was cut by its byte budget, possibly within a line.
/// Line-based cursors must not treat such a window as completely scanned.
pub byte_truncated: bool,
}
/// Binary-safe byte range read with an optional whole-file version fence.
+2 -1
View File
@@ -1746,7 +1746,7 @@ mod tests {
}
#[test]
fn builtin_orchestrator_keeps_cleanup_tool_providers_enabled() {
fn builtin_orchestrator_enables_reviewer_handoff_and_cleanup_without_self_review() {
let tmp = TempDir::new().unwrap();
let resolved = ProfileResolver::new()
.with_workspace_base(tmp.path())
@@ -1757,6 +1757,7 @@ mod tests {
.unwrap();
let feature = resolved.manifest.feature;
assert!(feature.sub_worker.enabled);
assert!(feature.worker.enabled);
assert!(!feature.worker.direct_spawn);
assert!(feature.manage_workdir.enabled);
+23 -7
View File
@@ -826,7 +826,7 @@ impl MergeRequestStore {
));
}
let mr = self.get_for_operation(&i.auth.workspace_id, &i.merge_request_id, &i.ticket_id)?;
self.assigned(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
if mr.state == MergeRequestState::Closed
|| (mr.state == MergeRequestState::Open && mr.selector_from.is_none())
{
@@ -1035,7 +1035,7 @@ impl MergeRequestStore {
) -> Result<ReviewRevokedEvent, MergeRequestError> {
bounded_body("revocation reason", &i.reason)?;
let mr = self.get_for_operation(&i.auth.workspace_id, &i.merge_request_id, &i.ticket_id)?;
self.assigned(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
let r = mr
.thread
.iter()
@@ -1152,7 +1152,7 @@ impl MergeRequestStore {
"operation fingerprint mismatch".into(),
));
}
self.completion_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
if mr.state != MergeRequestState::Open {
return Err(MergeRequestError::Conflict(
"Merge Request is not open".into(),
@@ -1186,7 +1186,7 @@ impl MergeRequestStore {
));
}
// Case openness is not an integration phase. Exact source approval,
// current Ticket content and active assignment fencing above remain
// current Ticket content and current assignment fencing above remain
// the independent MR integration guards.
Ok(())
}
@@ -1229,7 +1229,7 @@ impl MergeRequestStore {
}
let assignment_is_current: bool = transaction.query_row(
"SELECT EXISTS(
SELECT 1 FROM ticket_active_worker_assignments
SELECT 1 FROM ticket_current_worker_assignments
WHERE workspace_id=?1 AND ticket_id=?2 AND assignment_id=?3
)",
params![i.auth.workspace_id, i.ticket_id, i.auth.assignment_id],
@@ -1548,7 +1548,7 @@ impl MergeRequestStore {
}
Ok(())
}
fn completion_auth(
fn current_assignment_auth(
&self,
a: &MergeRequestAuth,
t: &str,
@@ -1747,7 +1747,23 @@ fn ticket_event(
) -> Result<(), MergeRequestError> {
let ticket = &mr.ticket_ids[0];
let n:i64=t.query_row("SELECT COALESCE(MAX(event_index),-1)+1 FROM typed_ticket_events WHERE workspace_id=?1 AND ticket_id=?2",params![mr.workspace_id,ticket],|r|r.get(0))?;
t.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,author,at,from_state,to_state,heading,body)VALUES(?1,?2,?3,'comment',?4,?5,NULL,NULL,'Merge Request integrated',?6)",params![mr.workspace_id,ticket,n,format!("worker:{}:{}",e.merged_by.runtime_id,e.merged_by.worker_id),e.created_at.to_rfc3339(),format!("Merge Request `{}` integrated approved source ref `{}`; Ticket remains in progress until guarded Ticket completion.",mr.merge_request_id,e.approved_source_ref)])?;
t.execute(
"INSERT INTO typed_ticket_events(
workspace_id,ticket_id,event_index,kind,author,at,from_state,to_state,heading,body,event_ref
) VALUES(?1,?2,?3,'comment',?4,?5,NULL,NULL,'Merge Request integrated',?6,?7)",
params![
mr.workspace_id,
ticket,
n,
format!("worker:{}:{}", e.merged_by.runtime_id, e.merged_by.worker_id),
e.created_at.to_rfc3339(),
format!(
"Merge Request `{}` integrated approved source ref `{}`; Ticket remains in progress until guarded Ticket completion.",
mr.merge_request_id, e.approved_source_ref
),
format!("{ticket}:{n}"),
],
)?;
for (k, v) in [
("implementation_assignment_id", a),
("merge_request_id", &mr.merge_request_id),
+116 -5
View File
@@ -53,8 +53,8 @@ fn fixture() -> (tempfile::TempDir, MergeRequestStore) {
let d = tempfile::tempdir().unwrap();
let p = d.path().join("db");
let c = Connection::open(&p).unwrap();
c.execute_batch("CREATE TABLE workspaces(workspace_id TEXT PRIMARY KEY);CREATE TABLE repositories(workspace_id TEXT,repository_id TEXT,PRIMARY KEY(workspace_id,repository_id));CREATE TABLE ticket_current_worker_assignments(workspace_id TEXT,ticket_id TEXT,assignment_id TEXT,runtime_id TEXT,worker_id TEXT,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_tickets(workspace_id TEXT,ticket_id TEXT,workflow_state TEXT,workflow_state_explicit INTEGER,updated_at TEXT,title TEXT NOT NULL,body TEXT NOT NULL,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_ticket_targets(workspace_id TEXT,ticket_id TEXT,ordinal INTEGER,repository_key TEXT,ref_selector TEXT,access TEXT);CREATE TABLE typed_ticket_events(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,kind TEXT,author TEXT,at TEXT,from_state TEXT,to_state TEXT,heading TEXT,body TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index));CREATE TABLE typed_ticket_event_attributes(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,key TEXT,value TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index,key));INSERT INTO workspaces VALUES('W');INSERT INTO repositories VALUES('W','R');INSERT INTO repositories VALUES('W','R2');INSERT INTO ticket_current_worker_assignments VALUES('W','T','A','runtime','coder','t');INSERT INTO typed_tickets VALUES('W','T','inprogress',1,'t','Title','Requirements');CREATE VIEW ticket_active_worker_assignments AS SELECT current.* FROM ticket_current_worker_assignments current JOIN typed_tickets ticket ON ticket.workspace_id=current.workspace_id AND ticket.ticket_id=current.ticket_id WHERE ticket.workflow_state NOT IN ('done','closed');").unwrap();
c.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body) VALUES('W','T',-1,'comment','Requirements')",[]).unwrap();
c.execute_batch("CREATE TABLE workspaces(workspace_id TEXT PRIMARY KEY);CREATE TABLE repositories(workspace_id TEXT,repository_id TEXT,PRIMARY KEY(workspace_id,repository_id));CREATE TABLE ticket_current_worker_assignments(workspace_id TEXT,ticket_id TEXT,assignment_id TEXT,runtime_id TEXT,worker_id TEXT,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_tickets(workspace_id TEXT,ticket_id TEXT,workflow_state TEXT,workflow_state_explicit INTEGER,updated_at TEXT,title TEXT NOT NULL,body TEXT NOT NULL,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_ticket_targets(workspace_id TEXT,ticket_id TEXT,ordinal INTEGER,repository_key TEXT,ref_selector TEXT,access TEXT);CREATE TABLE typed_ticket_events(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,kind TEXT,author TEXT,at TEXT,from_state TEXT,to_state TEXT,heading TEXT,body TEXT,event_ref TEXT NOT NULL DEFAULT '',PRIMARY KEY(workspace_id,ticket_id,event_index),UNIQUE(workspace_id,ticket_id,event_ref));CREATE TABLE typed_ticket_event_attributes(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,key TEXT,value TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index,key));INSERT INTO workspaces VALUES('W');INSERT INTO repositories VALUES('W','R');INSERT INTO repositories VALUES('W','R2');INSERT INTO ticket_current_worker_assignments VALUES('W','T','A','runtime','coder','t');INSERT INTO typed_tickets VALUES('W','T','inprogress',1,'t','Title','Requirements');CREATE VIEW ticket_active_worker_assignments AS SELECT current.* FROM ticket_current_worker_assignments current JOIN typed_tickets ticket ON ticket.workspace_id=current.workspace_id AND ticket.ticket_id=current.ticket_id WHERE ticket.workflow_state NOT IN ('done','closed');").unwrap();
c.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body,event_ref) VALUES('W','T',-1,'comment','Requirements','T:-1')",[]).unwrap();
drop(c);
let a = Assignments(Arc::new(Mutex::new(CurrentAssignment {
assignment_id: "A".into(),
@@ -457,6 +457,18 @@ fn integration_preserves_selectors_ticket_state_and_assignment_and_replays_its_m
.unwrap();
assert_eq!(replayed, merged);
let connection = Connection::open(d.path().join("db")).unwrap();
let ticket_refs = connection
.prepare("SELECT event_ref FROM typed_ticket_events WHERE event_index >= 0")
.unwrap()
.query_map([], |row| row.get::<_, String>(0))
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
ticket_refs,
vec!["T:0"],
"replay must not append another post"
);
let (state, active): (String, bool) = connection.query_row(
"SELECT workflow_state, EXISTS(SELECT 1 FROM ticket_active_worker_assignments WHERE workspace_id='W' AND ticket_id='T') FROM typed_tickets WHERE workspace_id='W' AND ticket_id='T'",
[],
@@ -477,6 +489,87 @@ fn integration_preserves_selectors_ticket_state_and_assignment_and_replays_its_m
assert!(!json.contains(banned), "{banned} in {json}")
}
}
#[test]
fn integration_post_failure_rolls_back_and_same_operation_recovers_with_legacy_empty_ref() {
let (dir, store) = fixture();
open(&store);
let review = approve(&store, "source", "approved");
request(&store, "source", "pending");
let completion = CompleteMergeRequest {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
operation_id: "recover-integration".into(),
approval_event_id: review.event_id,
current_subject_ref: "source".into(),
target_ref_before: "before".into(),
target_ref_after: "after".into(),
strategy: MergeStrategy::FastForward,
resolution: ConflictResolution::None,
auth: auth(),
now: at(5),
};
let connection = Connection::open(dir.path().join("db")).unwrap();
// Retain a comment written before integration populated event_ref.
connection.execute_batch(
"INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body,event_ref)
VALUES('W','T',0,'comment','Legacy integration','');
CREATE TRIGGER reject_integration_post BEFORE INSERT ON typed_ticket_events
WHEN NEW.heading='Merge Request integrated'
BEGIN SELECT RAISE(ABORT,'injected integration post failure'); END;",
).unwrap();
let before = store.get_by_id("W", "MR").unwrap();
let error = store.complete(completion.clone()).unwrap_err();
assert!(
error
.to_string()
.contains("injected integration post failure")
);
let failed = store.get_by_id("W", "MR").unwrap();
assert_eq!(failed.state, MergeRequestState::Open);
assert_eq!(failed.thread, before.thread);
assert!(store.authorize_review_submission("MR", "pending").is_ok());
assert_eq!(
connection
.query_row(
"SELECT COUNT(*) FROM typed_ticket_event_attributes",
[],
|row| row.get::<_, i64>(0),
)
.unwrap(),
0
);
connection
.execute_batch("DROP TRIGGER reject_integration_post")
.unwrap();
let merged = store.complete(completion.clone()).unwrap();
assert_eq!(store.complete(completion).unwrap(), merged);
assert_eq!(
store.get_by_id("W", "MR").unwrap().state,
MergeRequestState::Merged
);
assert!(store.authorize_review_submission("MR", "pending").is_err());
let posts = connection
.prepare("SELECT event_ref,body FROM typed_ticket_events WHERE event_index >= 0 ORDER BY event_index")
.unwrap()
.query_map([], |row| Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?)))
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
posts.len(),
2,
"retry and replay must append exactly one post"
);
assert_eq!(posts[0], (String::new(), "Legacy integration".into()));
assert_eq!(posts[1].0, "T:1");
assert!(posts[1].1.contains("`MR`"));
assert_eq!(connection.query_row(
"SELECT value FROM typed_ticket_event_attributes WHERE event_index=1 AND key='operation_id'",
[], |row| row.get::<_, String>(0),
).unwrap(), "recover-integration");
}
#[test]
fn source_move_cancels_submission_and_old_approval_is_reusable_when_source_returns() {
let (_d, s) = fixture();
@@ -955,7 +1048,7 @@ fn ticket_rescope_requires_fresh_review_before_merge_and_remains_recoverable() {
let connection = Connection::open(dir.path().join("db")).unwrap();
connection
.execute_batch(
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL)",
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL,'T:1')",
)
.unwrap();
connection
@@ -1082,7 +1175,7 @@ fn ticket_rescope_after_integration_accepts_fresh_requirement_attestation() {
let connection = Connection::open(dir.path().join("db")).unwrap();
connection
.execute_batch(
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL)",
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL,'T:1')",
)
.unwrap();
connection
@@ -1307,6 +1400,24 @@ fn partial_and_full_integration_retain_ticket_state_and_active_assignment() {
assert_eq!(second_request.merged_result().unwrap(), &second_merge);
assert_eq!(first_request.integration_approval().unwrap(), &first);
assert_eq!(second_request.integration_approval().unwrap(), &second);
let posts = connection
.prepare(
"SELECT event.event_ref, attribute.value
FROM typed_ticket_events event
JOIN typed_ticket_event_attributes attribute USING(workspace_id,ticket_id,event_index)
WHERE attribute.key='merge_request_id' ORDER BY event.event_index",
)
.unwrap()
.query_map([], |row| {
Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?))
})
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
posts,
vec![("T:0".into(), "MR".into()), ("T:1".into(), "MR-2".into())]
);
}
#[test]
@@ -1324,7 +1435,7 @@ fn historical_ticket_completion_payload_remains_readable_and_unchanged_on_migrat
}"#;
connection.execute_batch(
"UPDATE typed_tickets SET workflow_state='done' WHERE workspace_id='W' AND ticket_id='T';
INSERT INTO typed_ticket_events VALUES('W','T',0,'state_changed','worker:runtime:coder','2026-07-26T12:00:07Z','inprogress','done','Ticket requirements completed','historical body');"
INSERT INTO typed_ticket_events VALUES('W','T',0,'state_changed','worker:runtime:coder','2026-07-26T12:00:07Z','inprogress','done','Ticket requirements completed','historical body','T:0');"
).unwrap();
connection
.execute(
+64 -11
View File
@@ -1,3 +1,6 @@
//! Check the shipped templates against the real PDK and WIT for the guest target.
//! Each check owns its source and lockfile; Cargo alone shares and validates artifacts.
use std::fs;
use std::path::Path;
use std::process::Command;
@@ -135,36 +138,86 @@ fn embedded_service_template_cargo_checks_for_wasm_target() {
fn cargo_check_template(template: &str) {
let crate_dir = Path::new(env!("CARGO_MANIFEST_DIR"));
let template_dir = crate_dir.join(format!("../../resources/plugin/templates/{template}"));
let manifest_path = template_dir.join("Cargo.toml");
let lock_path = template_dir.join("Cargo.lock");
let _ = fs::remove_file(&lock_path);
let templates_dir = crate_dir.join("../../resources/plugin/templates");
let template_dir = templates_dir.join(template);
let target_dir = tempfile::tempdir().expect("temporary cargo target dir");
let cargo = std::env::var("CARGO").unwrap_or_else(|_| "cargo".to_string());
let output = Command::new(cargo)
// A sibling fixture preserves the shipped relative PDK and WIT paths verbatim.
// Never create or remove a lockfile in the shipped template: another invocation
// may be checking it at the same time, or a developer may own that lockfile.
let fixture = tempfile::Builder::new()
.prefix(".cargo-check-")
.tempdir_in(&templates_dir)
.expect("temporary template fixture");
copy_template_tree(&template_dir, fixture.path(), true);
// Both templates share the PDK dependencies. A separate persistent target
// reuses those artifacts without sharing source or the outer Cargo build lock.
// Cargo fingerprints and its target lock handle changes and concurrent checks.
let executable = std::env::current_exe().expect("current test executable");
let target_dir = executable
.parent()
.and_then(|deps| deps.parent())
.expect("test executable has a profile directory")
.join("compile-tests/plugin-pdk-templates");
let output = Command::new(env!("CARGO"))
.arg("check")
.arg("--manifest-path")
.arg(&manifest_path)
.arg(fixture.path().join("Cargo.toml"))
.arg("--target")
.arg("wasm32-unknown-unknown")
.arg("--offline")
.arg("--target-dir")
.arg(target_dir.path())
.arg(target_dir)
.current_dir(fixture.path())
.env("CARGO_TERM_COLOR", "never")
.output()
.expect("spawn cargo check for embedded template");
let _ = fs::remove_file(&lock_path);
assert!(
output.status.success(),
"template cargo check failed\nstatus: {}\nstdout:\n{}\nstderr:\n{}",
"template {template} cargo check failed\nstatus: {}\nstdout:\n{}\nstderr:\n{}",
output.status,
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
}
fn copy_template_tree(source: &Path, destination: &Path, is_root: bool) {
let metadata = fs::symlink_metadata(source).expect("read template directory metadata");
assert!(
metadata.is_dir() && !metadata.file_type().is_symlink(),
"template input must be a real directory: {}",
source.display()
);
for entry in fs::read_dir(source).expect("read template directory") {
let entry = entry.expect("read template entry");
let path = entry.path();
let kind = entry.file_type().expect("read template entry type");
assert!(
!kind.is_symlink() && (kind.is_dir() || kind.is_file()),
"unsupported template input (including symlinks): {}",
path.display()
);
let name = entry.file_name();
// Preserve all compiler inputs, including build scripts and nested assets.
// Only omit the root build output and lockfile: each isolated consumer must
// resolve its own lockfile, as the original unlocked template check did.
if is_root
&& ((kind.is_dir() && name == "target") || (kind.is_file() && name == "Cargo.lock"))
{
continue;
}
let copied = destination.join(name);
if kind.is_dir() {
fs::create_dir(&copied).expect("create template fixture directory");
copy_template_tree(&path, &copied, false);
} else {
fs::copy(&path, &copied).expect("copy unmodified template input");
}
}
}
#[test]
fn pdk_runtime_dependencies_are_guest_side_only() {
let cargo: Value = toml::from_str(PDK_CARGO).expect("PDK Cargo.toml parses");
+73 -7
View File
@@ -408,7 +408,7 @@ pub enum Method {
/// Transport is fire-and-forget; receivers must tolerate out-of-order
/// delivery (e.g. `TurnEnded` arriving after `ShutDown` for the same
/// child Worker).
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
@@ -982,6 +982,17 @@ pub enum SessionEntryProvenance {
LegacyUnknown,
}
/// Public use decisions contain no Feature identity or private payload. Absence
/// denotes the older projection contract, not permission to expose System input.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(deny_unknown_fields)]
pub struct SessionEntryUsePolicy {
pub observation_readable: bool,
pub memory_evidence: bool,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -994,6 +1005,8 @@ pub struct SessionSnapshotEntry {
pub provenance: SessionEntryProvenance,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub derived_from: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub use_policy: Option<SessionEntryUsePolicy>,
#[serde(flatten)]
pub data: SessionSnapshotEntryData,
}
@@ -1180,12 +1193,10 @@ pub enum Event {
},
/// One agent-injected system item committed to history.
///
/// Carries the JSON form of `session_store::SystemItem`. Covers
/// `Method::Notify` echoes, child-Worker lifecycle events from
/// `Method::WorkerEvent`, `@<path>` resolution payloads, and any future
/// Clients dispatch on the `kind` tag for typed rendering instead
/// of parsing free-text prefixes like `[Notification] …` or
/// `[File: …]`.
/// Carries the allowlisted JSON form of `session_store::PublicSystemItem`,
/// not the storage record. Its descriptive `kind` is independent from the
/// typed `presentation` used for notification, reminder, or context rendering.
/// Private Feature payloads and prompt provenance are never event data.
///
/// One event per `LogEntry::AnnotatedSystemItem` commit. Disk-side and
/// wire-side are 1:1.
@@ -1497,6 +1508,11 @@ pub enum Event {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct Alert {
/// Stable only within the emitting live Worker; not durable alert history.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub position: Option<AlertPosition>,
pub level: AlertLevel,
pub source: AlertSource,
pub message: String,
@@ -1504,6 +1520,18 @@ pub struct Alert {
pub timestamp_ms: i64,
}
/// Display anchor captured at emission, resolved when pending text is committed.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum AlertPosition {
Start,
AfterEntry { entry_id: String },
AfterToolCall { call_id: String },
AfterInFlight { block_id: u64 },
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -1704,6 +1732,9 @@ pub struct InFlightCompaction {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct InFlightSnapshot {
/// Live-Worker identities parallel to the blocks actually included below.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub block_ids: Vec<u64>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub blocks: Vec<InFlightBlock>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
@@ -2601,6 +2632,7 @@ mod tests {
session: SessionSnapshot {
pending_submissions: PendingSubmissionsSnapshot::default(),
entries: vec![SessionSnapshotEntry {
use_policy: None,
entry_id: "entry-1".into(),
timestamp: 1,
provenance: SessionEntryProvenance::HumanInput,
@@ -2721,6 +2753,7 @@ mod tests {
},
state: WorkerStatus::Running.into(),
in_flight: InFlightSnapshot {
block_ids: Vec::new(),
blocks: vec![
InFlightBlock::Text {
text: "hel".into(),
@@ -3048,9 +3081,42 @@ mod tests {
}
}
#[test]
fn alert_ordering_metadata_is_optional_and_uses_tagged_wire_anchors() {
let legacy = serde_json::json!({ "level": "warn", "source": "worker", "message": "old", "timestamp_ms": 42 });
let alert: Alert = serde_json::from_value(legacy.clone()).unwrap();
assert!(alert.id.is_none());
assert!(alert.position.is_none());
assert_eq!(serde_json::to_value(alert).unwrap(), legacy);
for position in [
serde_json::json!({ "kind": "start" }),
serde_json::json!({ "kind": "after_entry", "entry_id": "entry-1" }),
serde_json::json!({ "kind": "after_tool_call", "call_id": "call-1" }),
serde_json::json!({ "kind": "after_in_flight", "block_id": 7 }),
] {
let mut wire = legacy.clone();
wire["id"] = serde_json::json!("alert-1");
wire["position"] = position;
let alert: Alert = serde_json::from_value(wire.clone()).unwrap();
assert_eq!(serde_json::to_value(alert).unwrap(), wire);
}
let old_snapshot: InFlightSnapshot = serde_json::from_value(serde_json::json!({})).unwrap();
assert!(old_snapshot.block_ids.is_empty());
assert_eq!(
serde_json::to_value(old_snapshot).unwrap(),
serde_json::json!({})
);
let wire =
serde_json::json!({"blocks": [{"kind": "text", "text": "pending"}], "block_ids": [7]});
let snapshot: InFlightSnapshot = serde_json::from_value(wire.clone()).unwrap();
assert_eq!(serde_json::to_value(snapshot).unwrap(), wire);
}
#[test]
fn event_alert_format() {
let event = Event::Alert(Alert {
id: None,
position: None,
level: AlertLevel::Warn,
source: AlertSource::Compactor,
message: "compaction failed".into(),
+21 -19
View File
@@ -4,25 +4,25 @@ use serde_json::{Map, Value};
use ts_rs::{Config, TS};
use crate::{
Alert, AlertLevel, AlertSource, CommandEvent, CommandSnapshot, CommandStatus, CommandStream,
CommandStreamSlice, CompactionLifecycle, CompactionLifecycleState, CompactionPhase,
CompactionTrigger, CompletionContext, CompletionEntry, CompletionKind, ContextTokenSource,
ContextUsage, ErrorCode, Event, FeatureInvocation, FeatureInvocationDescriptor,
FeatureInvocationIdentity, FeatureInvocationResult, FeatureInvocationStatus,
FeatureInvocationSyntax, Greeting, InFlightBlock, InFlightCompaction, InFlightSnapshot,
InFlightToolCallState, InternalWorkerKind, InternalWorkerRef, InternalWorkerSnapshot,
InvocationArgumentDescriptor, InvocationArgumentType, InvocationArgumentValue,
InvocationClientAdapter, InvocationCompletion, InvocationValue, InvokeKind, MemoryWorkerEvent,
Method, PasteArtifactAvailability, PasteArtifactMediaType, PasteArtifactRef,
PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission, ReasoningConfig,
RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason, ScopeRule, Segment,
SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment, SubmissionDisposition,
SymlinkPolicy, ToolResultDisposition, TurnResult, UploadedFileAvailability, UploadedFileRef,
WorkerBusyState, WorkerCommandAcknowledgement, WorkerCommandDisposition, WorkerCommandEnvelope,
WorkerCommandKind, WorkerEvent, WorkerMaintenanceState, WorkerRunState, WorkerState,
WorkerStateSnapshot, WorkerStatus,
Alert, AlertLevel, AlertPosition, AlertSource, CommandEvent, CommandSnapshot, CommandStatus,
CommandStream, CommandStreamSlice, CompactionLifecycle, CompactionLifecycleState,
CompactionPhase, CompactionTrigger, CompletionContext, CompletionEntry, CompletionKind,
ContextTokenSource, ContextUsage, ErrorCode, Event, FeatureInvocation,
FeatureInvocationDescriptor, FeatureInvocationIdentity, FeatureInvocationResult,
FeatureInvocationStatus, FeatureInvocationSyntax, Greeting, InFlightBlock, InFlightCompaction,
InFlightSnapshot, InFlightToolCallState, InternalWorkerKind, InternalWorkerRef,
InternalWorkerSnapshot, InvocationArgumentDescriptor, InvocationArgumentType,
InvocationArgumentValue, InvocationClientAdapter, InvocationCompletion, InvocationValue,
InvokeKind, MemoryWorkerEvent, Method, PasteArtifactAvailability, PasteArtifactMediaType,
PasteArtifactRef, PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission,
ReasoningConfig, RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason,
ScopeRule, Segment, SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionEntryUsePolicy, SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole,
SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment,
SubmissionDisposition, SymlinkPolicy, ToolResultDisposition, TurnResult,
UploadedFileAvailability, UploadedFileRef, WorkerBusyState, WorkerCommandAcknowledgement,
WorkerCommandDisposition, WorkerCommandEnvelope, WorkerCommandKind, WorkerEvent,
WorkerMaintenanceState, WorkerRunState, WorkerState, WorkerStateSnapshot, WorkerStatus,
subscription::{
EventSubscriptionSelector, MAX_CORRELATION_ID_BYTES, MAX_REJECTION_MESSAGE_BYTES,
MAX_RESOURCE_ID_BYTES, MAX_SUBSCRIPTION_COLLECTION_ITEMS,
@@ -65,6 +65,7 @@ pub fn generated_protocol_types() -> String {
);
push_decl::<AlertLevel>(&cfg, &mut output);
push_decl::<AlertPosition>(&cfg, &mut output);
push_decl::<AlertSource>(&cfg, &mut output);
push_decl::<CompletionKind>(&cfg, &mut output);
push_decl::<CompletionContext>(&cfg, &mut output);
@@ -116,6 +117,7 @@ pub fn generated_protocol_types() -> String {
push_decl::<InFlightCompaction>(&cfg, &mut output);
push_decl::<InFlightSnapshot>(&cfg, &mut output);
push_decl::<SessionEntryProvenance>(&cfg, &mut output);
push_decl::<SessionEntryUsePolicy>(&cfg, &mut output);
push_decl::<SessionMessageRole>(&cfg, &mut output);
push_decl::<SessionContentPart>(&cfg, &mut output);
push_decl::<SessionToolAttachment>(&cfg, &mut output);
+319 -3
View File
@@ -425,6 +425,8 @@ pub struct BackendJobExecutionBinding {
pub input_digest: Option<String>,
#[serde(default)]
pub subjektiv_consolidation: bool,
#[serde(default)]
pub subjektiv_journal: bool,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
@@ -516,9 +518,10 @@ pub struct WorkerRestorePreparation {
pub workspace_api: Option<WorkspaceApiRef>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workdir_attachments: Option<Vec<LogicalWorkdirAttachment>>,
/// Retained in saved Restore intents; Restore does not apply Repository access.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access: Vec<WorkingDirectoryRepositoryAccessRequest>,
/// Read-only references resolved by Workspace only after this operation owns admission.
/// Retained references from saved resource preparation, not Restore prerequisites.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access_workdirs: Vec<RepositoryAccessWorkdirReference>,
}
@@ -712,6 +715,8 @@ pub struct WorkerSessionArchiveManifest {
pub source_runtime_id: String,
pub source_worker_id: WorkerId,
pub source_session_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source_active_segment_id: Option<String>,
pub segment_ids: Vec<String>,
pub source_created_at: String,
pub removed_at: String,
@@ -787,6 +792,118 @@ pub enum SessionPublicSource {
Archived { archive_id: String },
}
/// Canonical Session metadata, distinct from Memory's evidence/public-index projection.
/// `origin_json` is the exact serialized saved origin; it is not model-authored evidence.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionMaterialDescriptor {
pub segment_id: String,
pub entry_ref: String,
pub canonical_order: u64,
pub origin_json: String,
pub sha256: String,
pub total_bytes: u64,
}
/// Explicit Host erasure, distinct from ordinary retention policy or pin release.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialEraseRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
}
/// Acknowledges the permanent fence and physical erasure of Session originals,
/// including archived diagnostics bound to the same removal operations. Historical
/// retention receipts remain immutable; their old retained flags are not current availability.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialEraseResponse {
pub session_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialPinRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub pin_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialPinResponse {
pub session_id: String,
pub pin_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialCatalogRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub expected_catalog_sha256: Option<String>,
pub offset: u64,
pub limit: usize,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialCatalogPage {
pub session_id: String,
pub active_segment_id: String,
pub catalog_sha256: String,
pub descriptors: Vec<SessionMaterialDescriptor>,
pub next_offset: Option<u64>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialReadRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
/// Compared with a freshly resolved saved descriptor, never adopted as evidence.
pub expected_descriptor: SessionMaterialDescriptor,
pub byte_offset: u64,
pub max_content_bytes: usize,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialReadPage {
pub descriptor: SessionMaterialDescriptor,
pub content: String,
pub next_offset: u64,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionMaterialCatalogAvailability {
/// The saved active pointer is pending and storage proves no Session material
/// has ever been published. Missing/corrupt materialized sources are not empty.
NotMaterialized {
session_id: String,
},
Page {
page: SessionMaterialCatalogPage,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionMaterialReadAvailability {
Page {
page: SessionMaterialReadPage,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionPublicEntryKind {
@@ -988,8 +1105,78 @@ pub enum WorkerSessionAvailability {
},
}
/// Host-owned Subject release operation. A reservation has no timeout: uncertainty retains
/// occupancy and mutation exclusion until the same operation is explicitly reconciled.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectReleaseRequest {
pub workspace_id: String,
pub worker_id: WorkerId,
pub binding_id: String,
pub operation_id: String,
pub action: SubjectReleaseAction,
}
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum SubjectReleaseAction {
Reserve,
Disconnect,
Cancel,
}
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum SubjectReleaseState {
Reserved,
Disconnected,
Cancelled,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectReleaseResponse {
pub operation_id: String,
pub binding_id: String,
pub state: SubjectReleaseState,
}
/// Irreversibly fence an unpublished failed Subject create. Exact retries must retain
/// all four fields; this is distinct from releasing an existing Subject Controller.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryRequest {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
/// Durable admission-fence proof plus freshly confirmed execution absence.
/// An error (including a lost response) is never evidence of absence.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryResponse {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
#[api(reqwest, axum)]
pub trait RuntimeApi {
#[post("/v1/subject-create-recovery", status = 200, error_status = 400)]
async fn recover_subject_create(
&self,
#[body] request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError>;
#[post("/v1/subject-release", status = 200, error_status = 400)]
async fn subject_release(
&self,
#[body] request: SubjectReleaseRequest,
) -> Result<SubjectReleaseResponse, RuntimeApiError>;
#[get("/v1/ping", status = 200, error_status = 400)]
async fn ping(
&self,
@@ -1029,6 +1216,52 @@ pub trait RuntimeApi {
#[query] request: WorkerSessionHistoryRequest,
) -> Result<WorkerSessionHistoryAvailability, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/erase",
status = 200,
error_status = 400
)]
async fn session_material_erase(
&self,
#[body] request: SessionMaterialEraseRequest,
) -> Result<SessionMaterialEraseResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/pin",
status = 200,
error_status = 400
)]
async fn session_material_pin(
&self,
#[body] request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/release",
status = 200,
error_status = 400
)]
async fn session_material_release(
&self,
#[body] request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/catalog",
status = 200,
error_status = 400
)]
async fn session_material_catalog(
&self,
#[body] request: SessionMaterialCatalogRequest,
) -> Result<SessionMaterialCatalogAvailability, RuntimeApiError>;
#[post("/v1/session-public/material/read", status = 200, error_status = 400)]
async fn session_material_read(
&self,
#[body] request: SessionMaterialReadRequest,
) -> Result<SessionMaterialReadAvailability, RuntimeApiError>;
#[post("/v1/session-public/search", status = 200, error_status = 400)]
async fn session_public_search(
&self,
@@ -1277,6 +1510,22 @@ pub const REMAINING_RUNTIME_ROUTES: &[RemainingRuntimeRoute] = &[
mod tests {
use super::*;
#[test]
fn backend_job_journal_grant_defaults_off_and_does_not_enable_memory_grant() {
let legacy = serde_json::json!({"job_id":"job", "attempt_id":"attempt", "subjektiv_consolidation":true});
let binding: BackendJobExecutionBinding = serde_json::from_value(legacy).unwrap();
assert!(binding.subjektiv_consolidation);
assert!(!binding.subjektiv_journal);
let journal =
serde_json::json!({"job_id":"job", "attempt_id":"attempt", "subjektiv_journal":true});
let binding: BackendJobExecutionBinding = serde_json::from_value(journal.clone()).unwrap();
assert!(binding.subjektiv_journal);
assert!(!binding.subjektiv_consolidation);
let roundtrip: BackendJobExecutionBinding =
serde_json::from_value(serde_json::to_value(&binding).unwrap()).unwrap();
assert_eq!(roundtrip, binding);
}
#[test]
fn restore_request_is_mandatory_and_conflict_is_not_an_operation_state() {
for invalid in [
@@ -1442,6 +1691,59 @@ mod tests {
})
}
async fn session_material_erase(
&self,
_request: SessionMaterialEraseRequest,
) -> Result<SessionMaterialEraseResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_pin(
&self,
_request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn recover_subject_create(
&self,
request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError> {
Ok(SubjectCreateRecoveryResponse {
workspace_id: request.workspace_id,
worker_id: request.worker_id,
create_fingerprint: request.create_fingerprint,
operation_id: request.operation_id,
})
}
async fn subject_release(
&self,
_request: SubjectReleaseRequest,
) -> Result<SubjectReleaseResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_release(
&self,
_request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_catalog(
&self,
_request: SessionMaterialCatalogRequest,
) -> Result<SessionMaterialCatalogAvailability, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_read(
&self,
_request: SessionMaterialReadRequest,
) -> Result<SessionMaterialReadAvailability, RuntimeApiError> {
Err(test_error(501))
}
async fn session_public_search(
&self,
_request: SessionPublicSearchRequest,
@@ -1559,6 +1861,21 @@ mod tests {
.build()
.unwrap();
let recovery = SubjectCreateRecoveryRequest {
workspace_id: "workspace-a".into(),
worker_id: WorkerId::now_v7(),
create_fingerprint: "failed-create-fingerprint".into(),
operation_id: "recovery-1".into(),
};
let response = client
.recover_subject_create(recovery.clone())
.await
.unwrap();
assert_eq!(response.workspace_id, recovery.workspace_id);
assert_eq!(response.worker_id, recovery.worker_id);
assert_eq!(response.create_fingerprint, recovery.create_fingerprint);
assert_eq!(response.operation_id, recovery.operation_id);
let summary = client.runtime_summary().await.unwrap();
assert_eq!(summary.runtime.display_name.as_deref(), Some("round-trip"));
let workers = client
@@ -1603,9 +1920,8 @@ mod tests {
}
#[test]
fn contract_inventory_is_complete_and_unique() {
fn contract_inventory_has_unique_routes_under_api_base_path() {
let operations = RuntimeApiMetadata::OPERATIONS;
assert_eq!(operations.len(), 20);
let mut routes = operations
.iter()
.map(|operation| (format!("{:?}", operation.method), operation.path))
+475
View File
@@ -0,0 +1,475 @@
//! Neutral explicit-allow authorization contracts. Resource containment never crosses kinds.
//!
//! # Trusted Feature boundary
//!
//! A Profile requests execution configuration, not authority. The Worker asks the
//! Host for each enabled Feature's exact activation name before initialization,
//! then trusted Feature code asks for each protected operation and its semantic
//! resource before access or effects. `ticket` on [`PermissionResource::Feature`]
//! activates Ticket; it implies neither `ticket.create` nor any future operation.
//! `ticket.create` on a Workspace permits creation there but cannot activate the
//! Feature. There are no wildcard permissions or implicit dotted-name inheritance.
//!
//! Identity is absent from the authorization request intentionally: the Host
//! authenticates the bound Runtime/Worker and resolves current source permissions,
//! body ownership and any per-Worker execution grant. A shared client, Profile,
//! remembered receipt or caller-supplied Subject claim is not authority. Features
//! depend on this contract, never on a particular Subject policy implementation.
//!
//! Supported cooperative boundaries are Feature activation, ordinary Worker
//! delegation, Ticket creation (Tool/native WIP/typed service), Workdir filesystem
//! read/write and whole-attachment command execution, and scoped result notify.
//! Workdir providers still enforce resolved paths, traversal, capability, Ticket
//! and provider ceilings. The returned scope is for provider narrowing, not a
//! transferable credential. Scoped checkout/overlap operations are refused until
//! their complete access semantics are adapted. Other Feature operations retain
//! their existing owner checks; they do not automatically acquire this operation
//! policy. Missing calls are trusted Feature bugs, not interception by a universal
//! Tool executor or protection against malicious Feature code. Connection tests
//! must exercise each exposed surface, assert a genuine permitted effect and
//! assert zero downstream calls on denial, including revocation and provider denial.
//!
//! # Delegation callers and recovery
//!
//! Callers such as a chat Feature supply explicit operation/resource selections
//! and retain the same public operation ID and full request after an unknown
//! result. Normal Profile/Worker allocation and control/resource grants are reused;
//! there is no Subject Worker registry. Effective grants bind immutable Profile
//! resolution, the reserved Worker and source assignment. Permission replacement
//! invalidates existing children rather than refreshing them. Body replacement,
//! retirement and restore must recheck current authority; historical result lookup
//! is not new execution permission. Cancellation acceptance is distinct from
//! terminal confirmation and does not undo completed effects.
//!
//! A child receives no Subject body identity, singleton, whole Memory or public
//! conversation-post authority. `worker.result` is explicit notify authority to a
//! named origin Worker, not arbitrary control or publication. Chat delivery and
//! conversation correlation belong to the caller integration, not this contract.
//! Unset existing Subject policies deny new Subject execution/delegation; ordinary
//! non-Subject Workers explicitly retain existing Backend/provider contracts.
//! Standalone non-Workspace activation compatibility grants no Workspace operation.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
pub const MAX_SCOPED_PERMISSIONS: usize = 256;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PermissionResource {
Feature,
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
// Serde's internally tagged unit variants ignore extra fields even with deny_unknown_fields.
// A struct-shaped wire variant preserves the public unit variant while rejecting those claims.
impl<'de> Deserialize<'de> for PermissionResource {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
#[derive(Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
enum Wire {
Feature {},
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
Ok(match Wire::deserialize(deserializer)? {
Wire::Feature {} => Self::Feature,
Wire::Workspace { workspace_id } => Self::Workspace { workspace_id },
Wire::Workdir {
working_directory_id,
path,
recursive,
} => Self::Workdir {
working_directory_id,
path,
recursive,
},
Wire::Worker {
runtime_id,
worker_id,
} => Self::Worker {
runtime_id,
worker_id,
},
})
}
}
impl PermissionResource {
pub fn validate(&self) -> Result<(), String> {
match self {
Self::Feature => Ok(()),
Self::Workspace { workspace_id } => validate_authorization_identifier(workspace_id),
Self::Workdir {
working_directory_id,
path,
..
} => {
validate_authorization_identifier(working_directory_id)?;
validate_relative_permission_path(path)
}
Self::Worker {
runtime_id,
worker_id,
} => {
validate_authorization_identifier(runtime_id)?;
validate_authorization_identifier(worker_id)
}
}
}
pub fn covers(&self, other: &Self) -> bool {
if self.validate().is_err() || other.validate().is_err() {
return false;
}
match (self, other) {
(Self::Feature, Self::Feature) => true,
(Self::Workspace { workspace_id: a }, Self::Workspace { workspace_id: b }) => a == b,
(
Self::Worker {
runtime_id: ar,
worker_id: aw,
},
Self::Worker {
runtime_id: br,
worker_id: bw,
},
) => ar == br && aw == bw,
(
Self::Workdir {
working_directory_id: a,
path: ap,
recursive: ar,
},
Self::Workdir {
working_directory_id: b,
path: bp,
recursive: br,
},
) => {
a == b
&& if *ar {
ap == "."
|| ap == bp
|| bp
.strip_prefix(ap)
.is_some_and(|tail| tail.starts_with('/'))
} else {
ap == bp && !br
}
}
_ => false,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ScopedPermission {
pub operation: String,
pub resource: PermissionResource,
}
impl ScopedPermission {
pub fn validate(&self) -> Result<(), String> {
validate_operation_name(&self.operation)?;
self.resource.validate()?;
if matches!(self.resource, PermissionResource::Feature) && self.operation.contains('.') {
return Err("Feature resources authorize activation names only, not operations".into());
}
Ok(())
}
pub fn covers(&self, other: &Self) -> bool {
self.validate().is_ok()
&& other.validate().is_ok()
&& self.operation == other.operation
&& self.resource.covers(&other.resource)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissions {
pub change_id: String,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsUpdate {
pub operation_id: String,
pub expected_change_id: Option<String>,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerExecutionGrant {
pub subject_id: String,
pub origin_runtime_id: String,
pub origin_worker_id: String,
pub source_change_id: String,
pub runtime_id: String,
pub worker_id: String,
pub profile: String,
pub profile_digest: String,
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerDelegationRequest {
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
impl WorkerDelegationRequest {
pub fn validate(&self) -> Result<(), String> {
validate_scoped_permissions(&self.permissions)?;
validate_scoped_permissions(&self.redelegation)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationRequest {
pub operation: String,
pub resource: PermissionResource,
}
impl OperationAuthorizationRequest {
pub fn validate(&self) -> Result<(), String> {
ScopedPermission {
operation: self.operation.clone(),
resource: self.resource.clone(),
}
.validate()
}
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationResponse {
/// Matching granted scope for provider-resolved checks, not a credential.
/// `None` identifies the explicit non-Subject legacy authorization path.
#[serde(default)]
pub scope: Option<PermissionResource>,
}
pub fn validate_operation_name(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| !value.split('.').all(|part| {
!part.is_empty()
&& part
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'_' || b == b'-')
})
{
return Err(
"permission operation must be an exact, nonempty dot-separated name (no wildcards)"
.into(),
);
}
Ok(())
}
pub fn validate_scoped_permissions(permissions: &[ScopedPermission]) -> Result<(), String> {
if permissions.len() > MAX_SCOPED_PERMISSIONS {
return Err("too many scoped permissions".into());
}
for permission in permissions {
permission.validate()?;
}
Ok(())
}
pub fn validate_authorization_identifier(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| value.trim() != value
|| value.chars().any(|c| c.is_control() || c == '*')
{
return Err("authorization identity must be nonempty, bounded and exact".into());
}
Ok(())
}
/// Paths are already-normalized lexical paths, not host filesystem paths. The Host must
/// still enforce symlink/provider boundaries; lexical containment grants no filesystem access.
pub fn validate_relative_permission_path(path: &str) -> Result<(), String> {
if path == "." {
return Ok(());
}
if path.is_empty()
|| path.len() > 4096
|| path.contains(['\\', ':'])
|| path.chars().any(char::is_control)
|| path
.split('/')
.any(|part| part.is_empty() || part == "." || part == "..")
{
return Err("permission path must be a normalized relative path without escape".into());
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn workdir(path: &str, recursive: bool) -> PermissionResource {
PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
}
}
#[test]
fn lexical_containment_rejects_escape_and_respects_component_and_recursion_boundaries() {
for path in [
"",
"/src",
"../src",
"src/../secret",
"src/./a",
"src//a",
"src/",
"C:/src",
"src\\a",
] {
assert!(workdir(path, true).validate().is_err(), "{path}");
}
let root = workdir(".", true);
let src = workdir("src", true);
assert!(root.covers(&src));
assert!(src.covers(&workdir("src/a", true)));
assert!(!src.covers(&workdir("src-other", false)));
assert!(!workdir("src", false).covers(&src));
assert!(workdir("src", false).covers(&workdir("src", false)));
assert!(!workdir("src", false).covers(&workdir("src/a", false)));
assert!(!src.covers(&PermissionResource::Workdir {
working_directory_id: "other".into(),
path: "src".into(),
recursive: false
}));
}
#[test]
fn operation_hierarchy_never_implies_inheritance_or_feature_activation() {
for name in [
"",
"*",
"ticket.*",
".ticket",
"ticket.",
"ticket..read",
"ticket read",
] {
assert!(validate_operation_name(name).is_err(), "{name}");
}
let activation = ScopedPermission {
operation: "ticket".into(),
resource: PermissionResource::Feature,
};
let create = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "ws".into(),
},
};
let parent_name = ScopedPermission {
operation: "ticket".into(),
resource: create.resource.clone(),
};
assert!(!parent_name.covers(&create));
assert!(!create.covers(&parent_name));
assert!(!activation.covers(&create));
assert!(!create.covers(&activation));
assert!(
ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Feature
}
.validate()
.is_err()
);
assert!(
!PermissionResource::Workspace {
workspace_id: "ws".into()
}
.covers(&workdir(".", true))
);
}
#[test]
fn wire_contract_rejects_self_claimed_identity_and_unknown_fields() {
let request = serde_json::json!({"permissions":[],"redelegation":[],"subject_id":"forged"});
assert!(serde_json::from_value::<WorkerDelegationRequest>(request).is_err());
let resource: PermissionResource =
serde_json::from_value(serde_json::json!({"kind":"feature"})).unwrap();
assert_eq!(resource, PermissionResource::Feature);
assert!(
serde_json::from_value::<PermissionResource>(
serde_json::json!({"kind":"feature","workspace_id":"ws"})
)
.is_err()
);
}
#[test]
fn authorization_response_carries_matching_scope_and_defaults_legacy_absence() {
let legacy: OperationAuthorizationResponse =
serde_json::from_value(serde_json::json!({})).unwrap();
assert_eq!(legacy.scope, None);
assert_eq!(
serde_json::to_value(legacy).unwrap(),
serde_json::json!({"scope":null})
);
let granted = OperationAuthorizationResponse {
scope: Some(workdir("src", true)),
};
let wire = serde_json::to_value(&granted).unwrap();
assert_eq!(
wire,
serde_json::json!({"scope":{"kind":"workdir","working_directory_id":"wd","path":"src","recursive":true}})
);
assert_eq!(
serde_json::from_value::<OperationAuthorizationResponse>(wire).unwrap(),
granted
);
}
}
+10
View File
@@ -134,6 +134,14 @@ pub enum DriveEntryKind {
File,
}
/// Current caller's advisory publication ceiling, never a grant or execution credential.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveCapabilities {
pub write: bool,
}
/// Metadata only: content is returned through bounded text or binary routes.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -491,6 +499,7 @@ impl api_macros::HttpRequestError for DriveApiError {
impl api_macros::openapi::OpenApiSchema for DriveApiError {}
impl api_macros::openapi::OpenApiSchema for DriveEntry {}
impl api_macros::openapi::OpenApiSchema for DriveCapabilities {}
impl api_macros::openapi::OpenApiSchema for DriveEntryQuery {}
impl api_macros::openapi::OpenApiSchema for DriveListQuery {}
impl api_macros::openapi::OpenApiSchema for DriveSearchQuery {}
@@ -518,6 +527,7 @@ pub fn drive_api_typescript() -> String {
DriveEntryRef::decl(&config),
DriveEntryKind::decl(&config),
DriveEntry::decl(&config),
DriveCapabilities::decl(&config),
DriveEntryQuery::decl(&config),
DriveListQuery::decl(&config),
DriveSearchQuery::decl(&config),
+516 -4
View File
@@ -6,9 +6,12 @@ use api_macros::api;
pub use api_macros::axum as server_support;
pub use api_macros::reqwest as client_support;
pub use api_macros::{ApiContract, BinaryBody, HttpMethod, TransportMetadata, WebSocketOperation};
pub mod authorization;
mod drive;
mod notification;
pub use notification::*;
mod ticket_browse;
pub use ticket_browse::*;
pub mod repository_openapi_typescript;
pub use drive::*;
mod workspace_config;
@@ -21,6 +24,34 @@ use webauthn_rs_proto::{
};
pub use workspace_config::*;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsResponse {
pub permissions: Option<authorization::SubjectPermissions>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsMutationResponse {
pub permissions: authorization::SubjectPermissions,
/// New operations are already fenced. These are cancellation requests and
/// Backend command cleanup evidence, not proof that a Runtime turn stopped.
pub interruptions: Vec<SubjectPermissionInterruption>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionInterruption {
pub runtime_id: String,
pub worker_id: String,
pub cancellation_accepted: bool,
pub backend_command_sessions_closed: bool,
pub diagnostics: Vec<String>,
}
#[allow(dead_code)]
#[derive(JsonSchema)]
struct ServerJsonSafeU64(#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))] u64);
@@ -101,6 +132,8 @@ pub struct WorkdirOperationApiError {
pub workdir_log: Option<WorkdirOperationLogContext>,
#[serde(skip)]
pub denial_reason: Option<workdir::WorkdirDenialReason>,
#[serde(skip)]
pub failure_reason: Option<workdir::WorkdirFailureReason>,
}
/// Internal operational correlation. Producers supply fixed-length ID fingerprints and
@@ -126,6 +159,7 @@ impl WorkdirOperationApiError {
status: error.status,
workdir_log: None,
denial_reason: None,
failure_reason: None,
}
}
@@ -139,6 +173,7 @@ impl WorkdirOperationApiError {
status,
workdir_log: None,
denial_reason: error.denial_reason,
failure_reason: error.failure_reason,
}
}
}
@@ -166,8 +201,18 @@ macro_rules! impl_openapi_schema {
}
impl_openapi_schema!(
SubjectPermissionsResponse,
SubjectPermissionsMutationResponse,
SubjectPermissionInterruption,
authorization::SubjectPermissions,
authorization::SubjectPermissionsUpdate,
AuthBootstrapUserRequest,
AuthPublicConfig,
HostingSettings,
HostingAccess,
ProvisionalAccountRequest,
ActivationCodeResponse,
ActivationOptionsRequest,
AuthUserResponse,
DeviceLoginApproveRequest,
DeviceLoginApproveResponse,
@@ -179,6 +224,11 @@ impl_openapi_schema!(
PasskeyLoginOptionsRequest,
PasskeyLoginOptionsResponse,
PasskeyLoginCompleteRequest,
PasskeyAdditionOptionsRequest,
PasskeySummary,
PasskeyListResponse,
PasskeyAdditionResultRequest,
PasskeyAdditionResultResponse,
PasskeyRegistrationOptionsRequest,
PasskeyRegistrationOptionsResponse,
PasskeyRegistrationCompleteRequest,
@@ -376,6 +426,8 @@ impl_openapi_schema!(
TicketAssignmentDiagnosticsPage,
TicketQueryRequest,
TicketQueryResponse,
TicketBrowseQueryRequest,
TicketBrowseQueryResponse,
TicketQueueResponse,
TicketRecord,
TicketRecordRef,
@@ -510,6 +562,60 @@ pub trait ServerApi {
#[get("/api/auth/config", status = 200, error_status = 400)]
async fn auth_config(&self) -> Result<AuthPublicConfig, RepositoryApiError>;
#[get("/api/auth/registration-policy", status = 200, error_status = 400)]
async fn auth_registration_policy(&self) -> Result<HostingSettings, RepositoryApiError>;
#[get("/api/hosting/access", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_access(
&self,
#[extension] context: ServerRequestContext,
) -> Result<HostingAccess, RepositoryApiError>;
#[get("/api/hosting/settings", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_settings(
&self,
#[extension] context: ServerRequestContext,
) -> Result<HostingSettings, RepositoryApiError>;
#[put("/api/hosting/settings", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_settings_update(
&self,
#[extension] context: ServerRequestContext,
#[body] request: HostingSettings,
) -> Result<HostingSettings, RepositoryApiError>;
#[post("/api/hosting/accounts", status = 200, error_status = 400, additional_error_statuses = [403, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_account_create(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ProvisionalAccountRequest,
) -> Result<ActivationCodeResponse, RepositoryApiError>;
#[post("/api/hosting/accounts/activation-code", status = 200, error_status = 400, additional_error_statuses = [403, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_activation_code_reissue(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ProvisionalAccountRequest,
) -> Result<ActivationCodeResponse, RepositoryApiError>;
#[post("/api/auth/activation/options", status = 200, error_status = 400, additional_error_statuses = [500])]
async fn auth_activation_options(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ActivationOptionsRequest,
) -> Result<PasskeyRegistrationOptionsResponse, RepositoryApiError>;
#[post(
"/api/auth/activation/complete",
responses = [(status = 200, body = AuthUserResponse, headers = [("set-cookie", SetCookieHeader)])],
error_status = 400,
additional_error_statuses = [500]
)]
async fn auth_activation_complete(
&self,
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthActivationComplete, RepositoryApiError>;
#[post(
"/api/auth/bootstrap-user",
status = 200,
@@ -546,6 +652,63 @@ pub trait ServerApi {
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthPasskeyRegistrationComplete, RepositoryApiError>;
#[get(
"/api/auth/passkeys",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_list(
&self,
#[extension] context: ServerRequestContext,
) -> Result<PasskeyListResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/result",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_result(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyAdditionResultRequest,
) -> Result<PasskeyAdditionResultResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/options",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500, 502],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_options(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyAdditionOptionsRequest,
) -> Result<PasskeyRegistrationOptionsResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/complete",
responses = [
(status = 200, body = AuthUserResponse, headers = [("set-cookie", SetCookieHeader)])
],
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_complete(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthPasskeyAdditionComplete, RepositoryApiError>;
#[post(
"/api/auth/passkeys/login/options",
status = 200,
@@ -1204,6 +1367,51 @@ pub trait ServerApi {
#[body] request: MemoryConsolidateStagingRequest,
) -> Result<MemoryConsolidationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/authorization/check",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
openapi = false,
)]
async fn operation_authorize(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: authorization::OperationAuthorizationRequest,
) -> Result<authorization::OperationAuthorizationResponse, RepositoryApiError>;
#[get(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_get(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
) -> Result<SubjectPermissionsResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_set(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
#[body] request: authorization::SubjectPermissionsUpdate,
) -> Result<SubjectPermissionsMutationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects",
status = 201,
@@ -2668,6 +2876,12 @@ pub trait ServerApi {
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveEntry, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/capabilities", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_capabilities(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveCapabilities, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/metadata", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_metadata(
&self,
@@ -3033,6 +3247,14 @@ pub trait ServerApi {
#[body] request: TicketTagsSetRequest,
) -> Result<WorkspaceTicketTagList, RepositoryApiError>;
/// Lightweight browser list; deliberately excludes live MR evidence and external observations.
#[post("/api/w/{workspace_id}/tickets/browse", status = 200, error_status = 400, additional_error_statuses = [401, 403, 404, 500], bearer_auth = true, browser_auth = true)]
async fn ticket_browse(
&self,
#[path] workspace_id: String,
#[body] request: TicketBrowseQueryRequest,
) -> Result<TicketBrowseQueryResponse, RepositoryApiError>;
#[post("/api/w/{workspace_id}/tickets/query", status = 200, error_status = 400, additional_error_statuses = [401, 403, 404, 500], bearer_auth = true, browser_auth = true)]
async fn ticket_query(
&self,
@@ -3803,6 +4025,76 @@ pub struct AuthPublicConfig {
pub cookie_name: String,
}
/// The only public registration policy. No administrator identities or host secrets are exposed.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct HostingSettings {
pub self_registration_enabled: bool,
}
/// Hosting authority for the current authenticated human, not Workspace permissions.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct HostingAccess {
pub is_admin: bool,
}
/// A hosting administrator reserves the same normalized ID used by passkey login.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ProvisionalAccountRequest {
pub handle: String,
}
/// The only response allowed to disclose an activation code. Clients must keep it
/// out of URLs, persistent browser storage and diagnostics. A lost response is
/// recovered by reissuing, which invalidates the previous code and its ceremonies.
#[derive(Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ActivationCodeResponse {
pub user: AuthenticatedUser,
pub code: String,
pub expires_at: String,
}
impl std::fmt::Debug for ActivationCodeResponse {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("ActivationCodeResponse")
.field("user", &self.user)
.field("code", &"[REDACTED]")
.field("expires_at", &self.expires_at)
.finish()
}
}
/// Proof for a pending account's initial passkey ceremony, never a bearer token
/// or browser session. No caller-selected account ID or enrollment purpose is accepted.
#[derive(Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ActivationOptionsRequest {
pub handle: String,
pub code: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub browser_origin: Option<String>,
}
impl std::fmt::Debug for ActivationOptionsRequest {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("ActivationOptionsRequest")
.field("handle", &self.handle)
.field("code", &"[REDACTED]")
.field("browser_origin", &self.browser_origin)
.finish()
}
}
/// Authentication method that established the current request actor.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -3853,6 +4145,19 @@ impl RequestActor {
#[serde(deny_unknown_fields)]
pub struct WhoamiResponse {
pub actor: Option<RequestActor>,
pub session: BrowserSessionState,
/// Non-secret comparison evidence, never an authentication credential.
pub authentication_reference: Option<String>,
}
/// Browser navigation evidence, not an authorization grant or a generic 401 classification.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum BrowserSessionState {
Anonymous {},
Authenticated { expires_at: Option<String> },
Expired {},
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -3870,8 +4175,11 @@ pub struct AuthBootstrapUserRequest {
#[serde(deny_unknown_fields)]
pub struct AuthUserResponse {
pub user: AuthenticatedUser,
/// Comparison evidence for the exact newly issued session, not a credential.
pub authentication_reference: String,
}
/// Starts new-account registration only; an existing handle is never interpreted as key addition.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -3885,6 +4193,64 @@ pub struct PasskeyRegistrationOptionsRequest {
pub browser_origin: Option<String>,
}
/// Adds to the current actor. Expected identity/binding fields only fence stale UI;
/// they never select an owner or authorize an addition.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyAdditionOptionsRequest {
pub expected_user_id: String,
pub expected_account_id: String,
pub expected_authentication_reference: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub browser_origin: Option<String>,
}
/// Safe account metadata. Neither the WebAuthn credential ID nor key material is exposed.
/// Normal-login verification is observed verifier success, not inferred discoverability.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeySummary {
pub key_reference: String,
pub created_at: String,
pub last_used_at: Option<String>,
pub normal_login_verified_at: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyListResponse {
pub user: AuthenticatedUser,
pub keys: Vec<PasskeySummary>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyAdditionResultRequest {
pub challenge_id: String,
pub expected_user_id: String,
pub expected_account_id: String,
/// Original options authentication reference; the current same-owner session may differ.
pub expected_authentication_reference: String,
}
/// Unknown is not proof that registration failed: completion may still be in flight.
/// Saved proves the credential commit, not session issuance or current session health.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "status", rename_all = "snake_case", deny_unknown_fields)]
pub enum PasskeyAdditionResultResponse {
Unknown {},
Saved {
user: AuthenticatedUser,
key: PasskeySummary,
},
}
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -3905,6 +4271,11 @@ pub struct PasskeyRegistrationCompleteRequest {
pub credential: RegisterPublicKeyCredential,
}
/// Normal discoverable login omits the handle: the authenticator selects a credential.
/// An explicit handle selects legacy identified-account login, for non-resident keys or
/// keys registered with an old random WebAuthn user handle. This never adds a key or
/// falls back to another identity. Authenticate first, then enroll an RK from Account.
/// Existing challenge states are short-lived and are not migrated to the new ceremony.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -8058,9 +8429,8 @@ pub enum TicketImplementationResultKind {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct TicketImplementationResultRequest {
/// Host-owned replay identity; model-facing tools derive it from the Tool call.
pub operation_key: String,
pub assignment_id: String,
pub orchestrator_assignment_id: String,
pub kind: TicketImplementationResultKind,
pub body: String,
pub merge_request: Option<MergeRequestReviewSubject>,
@@ -10241,6 +10611,10 @@ pub struct CreateWorkspaceWorkerRequest {
/// Backend idempotency key used only for authenticated Worker-owned spawn/control.
#[serde(default)]
pub control_operation_id: Option<String>,
/// Requested least-authority child permissions. Host derives the origin; this
/// input cannot assert Subject identity or create a body/singleton attachment.
#[serde(default)]
pub execution_grant: Option<authorization::WorkerDelegationRequest>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -11338,12 +11712,23 @@ pub fn auth_api_typescript() -> String {
let config = ts_rs::Config::default();
let declarations = [
AuthPublicConfig::decl(&config),
HostingSettings::decl(&config),
HostingAccess::decl(&config),
ProvisionalAccountRequest::decl(&config),
ActivationCodeResponse::decl(&config),
ActivationOptionsRequest::decl(&config),
ActorAuthMethod::decl(&config),
AuthenticatedUser::decl(&config),
RequestActor::decl(&config),
BrowserSessionState::decl(&config),
WhoamiResponse::decl(&config),
AuthBootstrapUserRequest::decl(&config),
AuthUserResponse::decl(&config),
PasskeyAdditionOptionsRequest::decl(&config),
PasskeySummary::decl(&config),
PasskeyListResponse::decl(&config),
PasskeyAdditionResultRequest::decl(&config),
PasskeyAdditionResultResponse::decl(&config),
PasskeyRegistrationOptionsRequest::decl(&config),
PasskeyRegistrationOptionsResponse::decl(&config),
PasskeyRegistrationCompleteRequest::decl(&config),
@@ -12061,6 +12446,16 @@ mod tests {
HttpMethod::Post,
"/api/auth/passkeys/registration/options",
),
(
"auth_passkey_addition_options",
HttpMethod::Post,
"/api/auth/passkeys/addition/options",
),
(
"auth_passkey_addition_complete",
HttpMethod::Post,
"/api/auth/passkeys/addition/complete",
),
(
"auth_passkey_login_options",
HttpMethod::Post,
@@ -12726,6 +13121,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
})
.unwrap();
assert_eq!(
@@ -12740,6 +13136,7 @@ mod tests {
"workdir_attachments": [],
"feature_connections": {},
"control_operation_id": null,
"execution_grant": null,
})
);
}
@@ -13758,7 +14155,9 @@ mod tests {
"handle": "hare",
"display_name": "Hare",
"auth_method": "browser_session"
}
},
"session": { "kind": "authenticated", "expires_at": "2026-10-20T00:00:00Z" },
"authentication_reference": "synthetic-session-reference"
});
let decoded = serde_json::from_value::<WhoamiResponse>(whoami.clone())
.expect("server whoami fixture should match shared DTO");
@@ -13780,7 +14179,8 @@ mod tests {
"account_id": "account-1",
"handle": "hare",
"display_name": "Hare"
}
},
"authentication_reference": "synthetic-session-reference"
});
let decoded = serde_json::from_value::<AuthUserResponse>(auth_user.clone())
.expect("server auth-user fixture should match shared DTO");
@@ -13940,11 +14340,36 @@ mod tests {
let unexpected_field = serde_json::json!({
"actor": null,
"session": { "kind": "anonymous" },
"access_token": "must-not-be-accepted"
});
assert!(serde_json::from_value::<WhoamiResponse>(unexpected_field).is_err());
}
#[test]
fn whoami_session_json_distinguishes_anonymous_expired_and_token_auth() {
for session in [
serde_json::json!({"kind":"anonymous"}),
serde_json::json!({"kind":"expired"}),
serde_json::json!({"kind":"authenticated", "expires_at":null}),
] {
let value = serde_json::json!({"actor":null, "session":session, "authentication_reference": null});
let response: WhoamiResponse = serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(response).unwrap(), value);
}
for session in [
serde_json::json!({"kind":"unknown"}),
serde_json::json!({"kind":"expired", "expires_at":null}),
] {
assert!(
serde_json::from_value::<WhoamiResponse>(
serde_json::json!({"actor":null, "session":session})
)
.is_err()
);
}
}
#[test]
fn device_login_debug_output_redacts_secret_material() {
let start = DeviceLoginStartResponse {
@@ -14312,6 +14737,9 @@ mod openapi_artifact_tests {
const PUBLIC_UNAUTHENTICATED: &[&str] = &[
"health",
"auth_config",
"auth_registration_policy",
"auth_activation_options",
"auth_activation_complete",
"auth_bootstrap_user",
"auth_passkey_registration_options",
"auth_passkey_registration_complete",
@@ -14326,6 +14754,7 @@ mod openapi_artifact_tests {
// identity, or a one-use Reviewer capability. They remain generated Rust client/Axum
// operations, but must not appear as unauthenticated operations in the public OpenAPI.
const SIGNED_INTERNAL: &[&str] = &[
"operation_authorize",
"current_worker_workspace_config_get",
"current_worker_workspace_config_attach",
"current_worker_workspace_config_observe",
@@ -14452,6 +14881,38 @@ mod openapi_artifact_tests {
}
}
#[test]
fn passkey_addition_rejects_caller_selected_target_fields() {
for field in ["handle", "user_id", "account_id"] {
let payload = serde_json::json!({ (field): "other-user" });
assert!(
serde_json::from_value::<PasskeyAdditionOptionsRequest>(payload).is_err(),
"addition must reject caller-selected {field}"
);
}
assert!(
serde_json::from_value::<PasskeyAdditionOptionsRequest>(serde_json::json!({})).is_err()
);
}
#[test]
fn passkey_addition_requires_expected_owner_and_binding() {
let expected = serde_json::json!({
"expected_user_id": "user-a", "expected_account_id": "account-a",
"expected_authentication_reference": "reference"
});
assert!(serde_json::from_value::<PasskeyAdditionOptionsRequest>(expected.clone()).is_ok());
for field in [
"expected_user_id",
"expected_account_id",
"expected_authentication_reference",
] {
let mut missing = expected.clone();
missing.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<PasskeyAdditionOptionsRequest>(missing).is_err());
}
}
#[test]
fn auth_workspace_catalog_and_typed_header_contract_is_strict() {
let document = canonical_openapi_document().expect("canonical OpenAPI contract must build");
@@ -14465,6 +14926,8 @@ mod openapi_artifact_tests {
("/api/auth/bootstrap-user", "post"),
("/api/auth/passkeys/registration/options", "post"),
("/api/auth/passkeys/registration/complete", "post"),
("/api/auth/passkeys/addition/options", "post"),
("/api/auth/passkeys/addition/complete", "post"),
("/api/auth/passkeys/login/options", "post"),
("/api/auth/passkeys/login/complete", "post"),
("/api/auth/logout", "post"),
@@ -14503,6 +14966,7 @@ mod openapi_artifact_tests {
}
for path in [
"/api/auth/passkeys/registration/complete",
"/api/auth/passkeys/addition/complete",
"/api/auth/passkeys/login/complete",
"/api/auth/logout",
] {
@@ -14694,10 +15158,12 @@ mod workdir_error_diagnostic_tests {
#[test]
fn workdir_internal_diagnostics_never_enter_public_error_body_or_schema() {
let mut error = WorkdirOperationApiError::provider(workdir::http::WorkdirTransportError {
failure_reason: None,
code: workdir::http::WorkdirTransportErrorCode::Denied,
message: "Workdir operation was denied".to_string(),
denial_reason: Some(workdir::WorkdirDenialReason::OsPermissionDenied),
});
error.failure_reason = Some(workdir::WorkdirFailureReason::HttpTimeout);
error.workdir_log = Some(WorkdirOperationLogContext {
operation: "write",
workspace_id_hash: "internal-workspace-fingerprint".to_string(),
@@ -14711,9 +15177,55 @@ mod workdir_error_diagnostic_tests {
assert_eq!(api_macros::HttpError::status_code(&error), 403);
let decoded: WorkdirOperationApiError = serde_json::from_value(wire).unwrap();
assert!(decoded.denial_reason.is_none());
assert!(decoded.failure_reason.is_none());
assert!(decoded.workdir_log.is_none());
let schema = serde_json::to_value(schemars::schema_for!(WorkdirOperationApiError)).unwrap();
assert!(schema["properties"].get("workdir_log").is_none());
assert!(schema["properties"].get("denial_reason").is_none());
assert!(schema["properties"].get("failure_reason").is_none());
}
}
#[cfg(test)]
mod activation_contract_tests {
use super::*;
#[test]
fn activation_inputs_reject_caller_selected_authority_and_debug_redacts_codes() {
for field in [
"account_id",
"user_id",
"purpose",
"is_admin",
"authentication",
] {
let mut body = serde_json::json!({"handle":"new-user","code":"activation-secret"});
body[field] = serde_json::json!("caller-controlled");
assert!(serde_json::from_value::<ActivationOptionsRequest>(body).is_err());
let mut body = serde_json::json!({"handle":"new-user"});
body[field] = serde_json::json!("caller-controlled");
assert!(serde_json::from_value::<ProvisionalAccountRequest>(body).is_err());
}
let request = ActivationOptionsRequest {
handle: "new-user".into(),
code: "activation-secret".into(),
browser_origin: None,
};
assert!(!format!("{request:?}").contains(&request.code));
let response = ActivationCodeResponse {
user: AuthenticatedUser {
user_id: "u".into(),
account_id: "a".into(),
handle: "new-user".into(),
display_name: "new-user".into(),
},
code: "activation-secret".into(),
expires_at: "2030-01-01T00:00:00Z".into(),
};
assert!(!format!("{response:?}").contains(&response.code));
assert_eq!(
serde_json::to_value(&response).unwrap()["code"],
"activation-secret"
);
}
}
@@ -459,6 +459,13 @@ fn validate_ticket_browser_operations(
request: Some("TicketTagSetRequest"),
successes: &[("200", "Array_of_WorkspaceTag")],
},
OperationContract {
path: "/api/w/{workspace_id}/tickets/browse",
method: "post",
operation_id: "ticket_browse",
request: Some("TicketBrowseQueryRequest"),
successes: &[("200", "TicketBrowseQueryResponse")],
},
OperationContract {
path: "/api/w/{workspace_id}/tickets/query",
method: "post",
+65
View File
@@ -0,0 +1,65 @@
//! Lightweight Tickets browser contract, separate from authoritative evidence queries.
use crate::{BrowserTicketState, QueryPage};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryRequest {
pub query: Option<String>,
#[serde(default)]
#[cfg_attr(feature = "typescript", ts(type = "Array<BrowserTicketState>"))]
pub states: Vec<ticket::TicketState>,
/// Every supplied active tag ID must match (AND).
#[serde(default)]
pub tag_ids: Vec<String>,
#[cfg_attr(
feature = "typescript",
ts(type = "TicketAssignmentPrincipalSnapshot | null")
)]
pub assigned_principal: Option<ticket::TicketAssignmentPrincipalSnapshot>,
#[cfg_attr(
feature = "typescript",
ts(type = "TicketAssignmentTrackingStatus | null")
)]
pub handling_status: Option<ticket::TicketAssignmentTrackingStatus>,
/// Exact current closed-case reason; historical reasons do not match this filter.
#[cfg_attr(feature = "typescript", ts(type = "TicketClosureReason | null"))]
pub closure_reason: Option<ticket::TicketClosureReason>,
/// Only updated_desc is supported; defaults to updated_desc.
pub sort: Option<String>,
#[schemars(range(min = 0, max = 1000))]
pub limit: Option<usize>,
/// Browse-only cursor bound to the Workspace, canonical filters, and sort.
pub cursor: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryItem {
pub id: String,
pub resource_key: String,
pub title: String,
pub state: BrowserTicketState,
pub updated_at: Option<String>,
#[cfg_attr(feature = "typescript", ts(type = "Array<WorkspaceTag>"))]
pub tags: Vec<ticket::WorkspaceTag>,
#[cfg_attr(feature = "typescript", ts(type = "TicketClosureJudgment | null"))]
pub current_closure: Option<ticket::TicketClosureJudgment>,
#[cfg_attr(
feature = "typescript",
ts(type = "Array<TicketAssignmentTrackingProjection>")
)]
pub assignment_tracking: Vec<ticket::TicketAssignmentTrackingProjection>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryResponse {
pub items: Vec<TicketBrowseQueryItem>,
pub page: QueryPage,
pub record_authority: String,
}
+104 -6
View File
@@ -48,17 +48,48 @@ pub struct FsStore {
/// `write_all` may have extended the file, so rollback is safe only while
/// no sibling writer can append behind it.
append_lock: Arc<Mutex<()>>,
retention: crate::retention::SessionRetentionStore,
}
impl FsStore {
/// Create a new `FsStore` rooted at the given directory.
/// Creates the directory if it does not exist.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
let retention = crate::retention::fs_retention(&root);
Self::new_with_retention(root, retention)
}
/// Hosts that relocate Worker originals must keep control state outside the relocated
/// aggregate and pass the same store to readers and deletion/transfer operations.
pub fn new_with_retention(
root: impl Into<PathBuf>,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, StoreError> {
let root = root.into();
fs::create_dir_all(&root)?;
for entry in fs::read_dir(&root)? {
let entry = entry?;
if entry.file_type()?.is_dir()
&& let Some(session_id) = entry
.file_name()
.to_str()
.and_then(|name| name.parse().ok())
{
let _guard = match retention.lock_live(session_id) {
Ok(guard) => guard,
// Interrupted erasure may leave bytes behind. Never read or
// upgrade them, and do not block unrelated live Sessions.
Err(StoreError::SessionErased(_)) => continue,
Err(error) => return Err(error),
};
crate::worker_session_store::migrate_fs_session(&entry.path(), session_id)?;
}
}
Ok(Self {
root,
append_lock: Arc::new(Mutex::new(())),
retention,
})
}
@@ -98,12 +129,46 @@ impl FsStore {
/// Returns `Ok(true)` when a Session directory was removed and `Ok(false)`
/// when it was already absent.
pub fn delete_session(&self, session_id: SessionId) -> Result<bool, StoreError> {
let session_dir = self.session_dir(session_id);
match fs::remove_dir_all(&session_dir) {
Ok(()) => Ok(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
Err(error) => Err(error.into()),
}
self.delete_session_with_intent(
session_id,
crate::retention::SessionDeletionIntent::Ordinary,
)
}
pub fn retention(&self) -> crate::retention::SessionRetentionStore {
self.retention.clone()
}
pub fn pin_session(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().pin(session_id, pin, || {
Ok(self.session_dir(session_id).is_dir())
})
}
pub fn release_session_pin(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().release(session_id, pin)
}
pub fn delete_session_with_intent(
&self,
session_id: SessionId,
intent: crate::retention::SessionDeletionIntent,
) -> Result<bool, StoreError> {
self.retention().delete(session_id, intent, || {
match fs::remove_dir_all(self.session_dir(session_id)) {
Ok(()) => Ok(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
Err(error) => Err(error.into()),
}
})
}
fn session_dir(&self, session_id: SessionId) -> PathBuf {
@@ -282,12 +347,22 @@ impl FsStore {
}
impl Store for FsStore {
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().retain_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.log_path(session_id, segment_id), &line)
}
@@ -374,9 +449,14 @@ impl Store for FsStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.retention().lock_live(session_id)?;
let path = self.log_path(session_id, segment_id);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
crate::worker_session_store::migrate_fs_session(parent, session_id)?;
}
let mut content = String::new();
for entry in entries {
@@ -423,6 +503,7 @@ impl Store for FsStore {
content: &str,
limits: PasteArtifactLimits,
) -> Result<PasteArtifactRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -451,6 +532,7 @@ impl Store for FsStore {
content: &[u8],
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -474,6 +556,7 @@ impl Store for FsStore {
context: &UploadedFileUploadContext,
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -510,6 +593,7 @@ impl Store for FsStore {
reference: &UploadedFileRef,
source_entry_id: &str,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -537,6 +621,7 @@ impl Store for FsStore {
reference: &UploadedFileRef,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -550,6 +635,7 @@ impl Store for FsStore {
artifact_id: &str,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -563,6 +649,7 @@ impl Store for FsStore {
artifact_id: &str,
source_entry_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -579,6 +666,7 @@ impl Store for FsStore {
session_id: SessionId,
live_owner_ids: &[String],
) -> Result<u64, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -591,6 +679,7 @@ impl Store for FsStore {
session_id: SessionId,
artifact_id: &str,
) -> Result<bool, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -599,6 +688,7 @@ impl Store for FsStore {
}
fn delete_uncommitted_uploaded_files(&self, session_id: SessionId) -> Result<u64, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -631,6 +721,13 @@ impl Store for FsStore {
source_session_id: SessionId,
target_session_id: SessionId,
) -> Result<u64, StoreError> {
let mut sessions = vec![source_session_id, target_session_id];
sessions.sort();
sessions.dedup();
let _retention = sessions
.into_iter()
.map(|id| self.retention().lock_live(id))
.collect::<Result<Vec<_>, _>>()?;
let _guard = self
.append_lock
.lock()
@@ -647,6 +744,7 @@ impl Store for FsStore {
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.trace_path(session_id, segment_id), &line)
}
+204
View File
@@ -67,8 +67,14 @@ pub enum LoggedSessionHistoryOrigin {
ToolOutput {
worker: LoggedWorkerSubject,
},
/// Descriptive producer identity; use policies live in separate metadata.
Provided {
producer: String,
},
/// Host-produced opening context summary; not auto-read or TaskStore material.
CompactionSummary,
/// Frozen Host Journal reference, never independent observation evidence.
JournalContext,
DerivedSummary,
LegacyUnknown,
}
@@ -79,19 +85,32 @@ pub struct LoggedHistoryDerivation {
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct LoggedSessionHistoryMetadata {
pub entry_id: LoggedSessionHistoryEntryId,
pub origin: LoggedSessionHistoryOrigin,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub derivation: Option<LoggedHistoryDerivation>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub input_metadata: Option<crate::HistoryInputMetadata>,
}
impl LoggedSessionHistoryMetadata {
pub(crate) fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
if matches!(self.origin, LoggedSessionHistoryOrigin::Provided { .. })
&& self.input_metadata.is_none()
{
return Err(crate::InputMetadataError::MissingMetadata);
}
Ok(())
}
pub fn legacy_unknown() -> Self {
Self {
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
input_metadata: None,
}
}
}
@@ -99,20 +118,146 @@ impl LoggedSessionHistoryMetadata {
/// Persisted item and metadata are one value so transforms cannot reorder or
/// truncate one without the other.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(try_from = "HistoryEntryWire")]
pub struct LoggedHistoryEntry {
pub item: LoggedItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct HistoryEntryWire {
item: LoggedItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<HistoryEntryWire> for LoggedHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: HistoryEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedHistoryEntry {
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
if let Some(input) = &self.metadata.input_metadata {
match &self.item {
LoggedItem::Message {
role: crate::LoggedRole::System,
content,
} => {
let [crate::LoggedContentPart::Text { text }] = content.as_slice() else {
return Err(crate::InputMetadataError::InconsistentRecord);
};
if text.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
// Other logical roles may carry use policy, but cannot masquerade
// as the System-only public/payload extension contract.
_ if input.feature_payload().is_some() || input.public_item().is_some() => {
return Err(crate::InputMetadataError::InconsistentRecord);
}
_ => {}
}
}
Ok(())
}
}
/// Typed system-item history record. The typed system event remains available
/// to client replay while its model-visible projection carries the same stable
/// metadata used by live history.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(try_from = "SystemEntryWire")]
pub struct LoggedSystemHistoryEntry {
pub item: crate::SystemItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct SystemEntryWire {
item: crate::SystemItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<SystemEntryWire> for LoggedSystemHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: SystemEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedSystemHistoryEntry {
pub fn from_input(
origin: LoggedSessionHistoryOrigin,
derivation: Option<LoggedHistoryDerivation>,
input: crate::SystemInputEnvelope,
) -> Self {
let (body, input_metadata) = input.into_parts();
Self {
item: crate::SystemItem::Input { body },
metadata: LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId::new(),
origin,
derivation,
input_metadata: Some(input_metadata),
},
}
}
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
match (&self.item, &self.metadata.input_metadata) {
(crate::SystemItem::Input { body }, Some(_)) => {
if body.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
(crate::SystemItem::Input { .. }, None) => {
return Err(crate::InputMetadataError::MissingMetadata);
}
(_, Some(_)) => return Err(crate::InputMetadataError::InconsistentRecord),
(_, None) => {}
}
Ok(())
}
pub fn to_history_entry(&self) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: self.item.to_history_item().into(),
metadata: self.metadata.clone(),
}
}
pub fn public_item(&self) -> crate::PublicSystemItem {
if matches!(self.item, crate::SystemItem::Input { .. }) {
return self
.metadata
.input_metadata
.as_ref()
.and_then(crate::HistoryInputMetadata::public_item)
.cloned()
.unwrap_or_else(|| self.item.public_item());
}
self.item.public_item()
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -130,6 +275,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -148,6 +294,64 @@ mod tests {
);
}
#[test]
fn external_system_records_reject_missing_policy_role_mismatch_and_overlimit_body() {
let valid = serde_json::json!({
"item": { "kind": "input", "body": "context" },
"metadata": {
"entry_id": "stable-entry", "origin": { "kind": "provided", "producer": "owner" },
"input_metadata": { "usage": {
"recording_material": "exclude", "memory_evidence": "exclude",
"observation": "hidden", "compaction": "reference_input"
} }
}
});
let restored: LoggedSystemHistoryEntry = serde_json::from_value(valid.clone()).unwrap();
let flat = serde_json::to_value(restored.to_history_entry()).unwrap();
assert!(serde_json::from_value::<LoggedHistoryEntry>(flat.clone()).is_ok());
for generic in [true, false] {
let wire = if generic { &valid } else { &flat };
let decode = |wire| {
if generic {
serde_json::from_value::<LoggedSystemHistoryEntry>(wire).is_err()
} else {
serde_json::from_value::<LoggedHistoryEntry>(wire).is_err()
}
};
let mut missing = wire.clone();
missing["metadata"]
.as_object_mut()
.unwrap()
.remove("input_metadata");
assert!(decode(missing));
let mut absent_policy = wire.clone();
absent_policy["metadata"]["input_metadata"]
.as_object_mut()
.unwrap()
.remove("usage");
assert!(decode(absent_policy));
let mut invalid_role = wire.clone();
invalid_role["item"] =
serde_json::to_value(LoggedItem::from(agen::Item::user_message("not system")))
.unwrap();
invalid_role["metadata"]["input_metadata"]["public"] = serde_json::json!({
"kind": "context", "body": "public system projection", "presentation": {"kind": "context"}
});
assert!(decode(invalid_role));
let mut oversized = wire.clone();
if generic {
oversized["item"]["body"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
} else {
oversized["item"]["content"][0]["text"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
}
assert!(decode(oversized));
}
}
#[test]
fn annotated_segment_start_is_restore_visible_without_projecting_metadata() {
let session_id = uuid::Uuid::now_v7();
+563
View File
@@ -0,0 +1,563 @@
//! Validated, Feature-neutral history-input policy and System extensions.
//!
//! The enclosing history annotation owns identity, origin and derivation. None of
//! those descriptive facts selects a use policy here. Model text, independent use
//! policies, private Feature data and an explicit public projection travel as one
//! value. Restoring this value does not load a Feature or invoke domain behavior.
//!
//! Payload validation bounds the decoded value; enclosing log readers must also
//! enforce their record allocation limit before decoding untrusted persisted input.
use agen::Item;
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::PublicSystemItem;
pub const MAX_FEATURE_PAYLOAD_BYTES: usize = 128 * 1024;
pub const MAX_SYSTEM_INPUT_BYTES: usize = 4 * 1024 * 1024;
pub const MAX_SYSTEM_PUBLIC_BYTES: usize = 256 * 1024;
const MAX_SCHEMA_BYTES: usize = 128;
const MAX_JSON_DEPTH: usize = 16;
const MAX_JSON_NODES: usize = 8192;
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum InputMetadataError {
#[error("history input payload schema is invalid")]
InvalidSchema,
#[error("history input payload version must be nonzero")]
InvalidVersion,
#[error("system input resource limit exceeded")]
ResourceLimit,
#[error("history input payload could not be serialized")]
Serialization,
#[error("history input payload schema or version is unsupported by this consumer")]
UnsupportedPayload,
#[error("history input payload does not match the owner's typed contract")]
InvalidTypedPayload,
#[error("provided input is missing explicit metadata")]
MissingMetadata,
#[error("input metadata does not match its logical record")]
InconsistentRecord,
}
/// One private payload owned by a Feature's versioned typed decoder.
/// Unknown valid schemas/versions remain data, never execution instructions.
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(try_from = "PayloadWire")]
pub struct FeatureInputPayload {
schema: String,
version: u32,
value: Value,
}
impl std::fmt::Debug for FeatureInputPayload {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("FeatureInputPayload")
.field("schema", &self.schema)
.field("version", &self.version)
.finish_non_exhaustive()
}
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct PayloadWire {
schema: String,
version: u32,
value: Value,
}
impl TryFrom<PayloadWire> for FeatureInputPayload {
type Error = InputMetadataError;
fn try_from(wire: PayloadWire) -> Result<Self, Self::Error> {
if wire.schema.is_empty()
|| wire.schema.len() > MAX_SCHEMA_BYTES
|| !wire.schema.as_bytes()[0].is_ascii_lowercase()
|| !wire.schema.bytes().all(|byte| {
byte.is_ascii_lowercase() || byte.is_ascii_digit() || b"-_.:/".contains(&byte)
})
{
return Err(InputMetadataError::InvalidSchema);
}
if wire.version == 0 {
return Err(InputMetadataError::InvalidVersion);
}
validate_json(&wire.value, MAX_FEATURE_PAYLOAD_BYTES)?;
Ok(Self {
schema: wire.schema,
version: wire.version,
value: wire.value,
})
}
}
impl FeatureInputPayload {
pub fn encode<T: Serialize>(
schema: impl Into<String>,
version: u32,
value: &T,
) -> Result<Self, InputMetadataError> {
PayloadWire {
schema: schema.into(),
version,
value: serde_json::to_value(value).map_err(|_| InputMetadataError::Serialization)?,
}
.try_into()
}
pub fn schema(&self) -> &str {
&self.schema
}
pub fn version(&self) -> u32 {
self.version
}
/// The owner explicitly selects its contract before decoding. A valid unknown
/// payload can be retained and round-tripped without calling this operation.
pub fn decode<T: DeserializeOwned>(
&self,
schema: &str,
version: u32,
) -> Result<T, InputMetadataError> {
if self.schema != schema || self.version != version {
return Err(InputMetadataError::UnsupportedPayload);
}
serde_json::from_value(self.value.clone())
.map_err(|_| InputMetadataError::InvalidTypedPayload)
}
}
/// Eligibility is supplied independently for recording and Memory evidence.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputEvidenceUse {
Include,
Exclude,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputObservationUse {
Readable,
Hidden,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputCompactionUse {
/// Ordinary history eligible for the selected retained tail.
Retain,
/// A full opening summary consumed separately and superseded by compaction.
OpeningSummary,
/// Full historical context for the compactor, not an accumulating retained tail.
ReferenceInput,
}
/// No permissive defaults: malformed/unknown policies cannot turn into eligibility.
/// Public presentation is deliberately not one of these use decisions.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct HistoryInputUsePolicy {
pub recording_material: InputEvidenceUse,
pub memory_evidence: InputEvidenceUse,
pub observation: InputObservationUse,
pub compaction: InputCompactionUse,
}
/// Use policy is independent of logical role. Non-System derived Tool context
/// carries policy only; private Feature payload and public presentation belong
/// to the System envelope and are rejected on other roles at the history boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "MetadataWire")]
pub struct HistoryInputMetadata {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct MetadataWire {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
impl TryFrom<MetadataWire> for HistoryInputMetadata {
type Error = InputMetadataError;
fn try_from(wire: MetadataWire) -> Result<Self, Self::Error> {
if let Some(public) = &wire.public {
// Kind is a descriptive label, not a Feature dispatch or authority key.
if public.kind.is_empty()
|| public.kind.len() > MAX_SCHEMA_BYTES
|| public.kind.chars().any(char::is_control)
{
return Err(InputMetadataError::InvalidSchema);
}
let value =
serde_json::to_value(public).map_err(|_| InputMetadataError::Serialization)?;
validate_json(&value, MAX_SYSTEM_PUBLIC_BYTES)?;
}
Ok(Self {
usage: wire.usage,
feature_payload: wire.feature_payload,
public: wire.public,
})
}
}
impl HistoryInputMetadata {
pub fn new(
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
) -> Result<Self, InputMetadataError> {
MetadataWire {
usage,
feature_payload,
public,
}
.try_into()
}
pub fn usage(&self) -> HistoryInputUsePolicy {
self.usage
}
pub fn feature_payload(&self) -> Option<&FeatureInputPayload> {
self.feature_payload.as_ref()
}
pub fn public_item(&self) -> Option<&PublicSystemItem> {
self.public.as_ref()
}
pub fn public_use_policy(&self) -> protocol::SessionEntryUsePolicy {
protocol::SessionEntryUsePolicy {
observation_readable: self.usage.observation == InputObservationUse::Readable,
memory_evidence: self.usage.memory_evidence == InputEvidenceUse::Include,
}
}
pub fn public_system_use_policy(&self) -> protocol::SessionEntryUsePolicy {
if self.public.is_some() {
self.public_use_policy()
} else {
protocol::SessionEntryUsePolicy {
observation_readable: false,
memory_evidence: false,
}
}
}
pub fn public_projection(&self) -> PublicSystemItem {
self.public.clone().unwrap_or_else(|| PublicSystemItem {
kind: "input".into(),
body: String::new(),
presentation: crate::PublicSystemPresentation::Ignored,
})
}
}
/// System role is invariant; provenance and use policies are attached data.
/// An absent public projection never implicitly publishes the model body.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "InputWire")]
pub struct SystemInputEnvelope {
body: String,
metadata: HistoryInputMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct InputWire {
body: String,
metadata: HistoryInputMetadata,
}
impl TryFrom<InputWire> for SystemInputEnvelope {
type Error = InputMetadataError;
fn try_from(wire: InputWire) -> Result<Self, Self::Error> {
if wire.body.len() > MAX_SYSTEM_INPUT_BYTES {
return Err(InputMetadataError::ResourceLimit);
}
Ok(Self {
body: wire.body,
metadata: wire.metadata,
})
}
}
impl SystemInputEnvelope {
pub fn new(
body: impl Into<String>,
metadata: HistoryInputMetadata,
) -> Result<Self, InputMetadataError> {
InputWire {
body: body.into(),
metadata,
}
.try_into()
}
pub fn body(&self) -> &str {
&self.body
}
pub fn metadata(&self) -> &HistoryInputMetadata {
&self.metadata
}
pub fn to_history_item(&self) -> Item {
Item::system_message(self.body.clone())
}
pub fn into_parts(self) -> (String, HistoryInputMetadata) {
(self.body, self.metadata)
}
}
fn validate_json(value: &Value, max_bytes: usize) -> Result<(), InputMetadataError> {
fn walk(value: &Value, depth: usize, remaining: &mut usize) -> Result<(), InputMetadataError> {
if depth > MAX_JSON_DEPTH || *remaining == 0 {
return Err(InputMetadataError::ResourceLimit);
}
*remaining -= 1;
match value {
Value::Array(values) => {
for value in values {
walk(value, depth + 1, remaining)?;
}
}
Value::Object(values) => {
for value in values.values() {
walk(value, depth + 1, remaining)?;
}
}
_ => {}
}
Ok(())
}
let mut remaining = MAX_JSON_NODES;
walk(value, 0, &mut remaining)?;
let bytes = serde_json::to_vec(value).map_err(|_| InputMetadataError::Serialization)?;
if bytes.len() > max_bytes {
return Err(InputMetadataError::ResourceLimit);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{LoggedItem, LoggedRole, PublicSystemPresentation};
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct OwnerPayload {
source_digest: String,
omitted: bool,
}
fn reference_policy() -> HistoryInputUsePolicy {
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
}
}
#[test]
fn unknown_valid_payload_round_trips_without_domain_decode_or_public_exposure() {
let payload = FeatureInputPayload::encode(
"future-owner:context",
99,
&OwnerPayload {
source_digest: "private-digest".into(),
omitted: true,
},
)
.unwrap();
let envelope = SystemInputEnvelope::new(
"private model context",
HistoryInputMetadata::new(reference_policy(), Some(payload), None).unwrap(),
)
.unwrap();
let restored: SystemInputEnvelope =
serde_json::from_slice(&serde_json::to_vec(&envelope).unwrap()).unwrap();
assert_eq!(restored.body(), "private model context");
assert!(restored.metadata().public_item().is_none());
let saved_payload = restored.metadata().feature_payload().unwrap();
assert_eq!(saved_payload.schema(), "future-owner:context");
assert_eq!(saved_payload.version(), 99);
assert_eq!(
saved_payload.decode::<OwnerPayload>("known-owner:context", 1),
Err(InputMetadataError::UnsupportedPayload)
);
assert_eq!(
serde_json::to_value(&restored).unwrap(),
serde_json::to_value(&envelope).unwrap()
);
assert!(!format!("{saved_payload:?}").contains("private-digest"));
}
#[test]
fn owner_decodes_typed_payload_only_for_its_explicit_contract() {
let input = OwnerPayload {
source_digest: "rev".into(),
omitted: false,
};
let payload = FeatureInputPayload::encode("owner:context", 1, &input).unwrap();
assert_eq!(
payload.decode::<OwnerPayload>("owner:context", 1).unwrap(),
input
);
let invalid = FeatureInputPayload::encode(
"owner:context",
1,
&serde_json::json!({ "omitted": "not a bool" }),
)
.unwrap();
assert_eq!(
invalid.decode::<OwnerPayload>("owner:context", 1),
Err(InputMetadataError::InvalidTypedPayload)
);
}
#[test]
fn malformed_or_oversized_payload_is_rejected_at_construction_and_restore() {
for (schema, version, value) in [
("", 1, Value::Null),
("invalid schema", 1, Value::Null),
("owner:context", 0, Value::Null),
(
"owner:context",
1,
Value::String("x".repeat(MAX_FEATURE_PAYLOAD_BYTES)),
),
(
"owner:context",
1,
Value::Array(vec![Value::Null; MAX_JSON_NODES]),
),
(
"owner:context",
1,
(0..=MAX_JSON_DEPTH).fold(Value::Null, |inner, _| Value::Array(vec![inner])),
),
] {
assert!(FeatureInputPayload::encode(schema, version, &value).is_err());
assert!(
serde_json::from_value::<FeatureInputPayload>(
serde_json::json!({ "schema": schema, "version": version, "value": value })
)
.is_err()
);
}
}
#[test]
fn policy_is_required_and_unknown_or_malformed_fields_fail_closed() {
let valid = serde_json::to_value(reference_policy()).unwrap();
for field in [
"recording_material",
"memory_evidence",
"observation",
"compaction",
] {
let mut absent = valid.clone();
absent.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<HistoryInputUsePolicy>(absent).is_err());
let mut unknown = valid.clone();
unknown[field] = Value::String("future-allow".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(unknown).is_err());
}
let mut extra = valid;
extra["feature_name"] = Value::String("infer-policy-from-me".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(extra).is_err());
}
#[test]
fn feature_name_and_model_body_do_not_select_policies_or_public_projection() {
for (schema, body) in [
("owner:journal", "past context"),
("unrelated:skill", "do not extract this; legitimate wording"),
] {
let public = PublicSystemItem {
kind: "context-label".into(),
body: "explicit public text".into(),
presentation: PublicSystemPresentation::Context,
};
let metadata = HistoryInputMetadata::new(
reference_policy(),
Some(
FeatureInputPayload::encode(
schema,
1,
&OwnerPayload {
source_digest: "private".into(),
omitted: false,
},
)
.unwrap(),
),
Some(public),
)
.unwrap();
let input = SystemInputEnvelope::new(body, metadata).unwrap();
assert_eq!(input.metadata().usage(), reference_policy());
let history: LoggedItem = input.to_history_item().into();
assert!(matches!(
history,
LoggedItem::Message {
role: LoggedRole::System,
..
}
));
assert_eq!(input.to_history_item().as_text(), Some(body));
let public = serde_json::to_string(input.metadata().public_item().unwrap()).unwrap();
assert!(!public.contains("private"));
assert!(!public.contains(schema));
assert!(!public.contains(body));
}
}
#[test]
fn use_policies_remain_independent_of_each_other_and_of_public_display() {
let usage = HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Include,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Readable,
compaction: InputCompactionUse::Retain,
};
let metadata = HistoryInputMetadata::new(usage, None, None).unwrap();
let restored: HistoryInputMetadata =
serde_json::from_value(serde_json::to_value(&metadata).unwrap()).unwrap();
assert_eq!(restored.usage(), usage);
assert!(restored.public_item().is_none());
assert!(restored.feature_payload().is_none());
}
#[test]
fn model_body_and_public_projection_have_independent_resource_limits() {
let metadata = HistoryInputMetadata::new(reference_policy(), None, None).unwrap();
assert!(
SystemInputEnvelope::new("x".repeat(MAX_SYSTEM_INPUT_BYTES + 1), metadata).is_err()
);
let public = PublicSystemItem {
kind: "context".into(),
body: "x".repeat(MAX_SYSTEM_PUBLIC_BYTES),
presentation: PublicSystemPresentation::Context,
};
assert!(HistoryInputMetadata::new(reference_policy(), None, Some(public)).is_err());
let invalid = serde_json::json!({ "body": "small", "metadata": { "public": null, "feature_payload": null } });
assert!(serde_json::from_value::<SystemInputEnvelope>(invalid).is_err());
}
}
+413 -21
View File
@@ -1,12 +1,15 @@
//! Versioned decoder for Session schemas that predate canonical annotated history.
//! Versioned decoder for Session schemas v1-5, including frozen Feature wire.
//!
//! These types are intentionally private to `session-store`. Current writers,
//! replay, and public projections use [`crate::LogEntry`] exclusively; only the
//! Worker Session schema migration is allowed to deserialize these shapes.
//! explicit Session schema migration is allowed to deserialize these shapes.
use agen::llm_client::types::RequestConfig;
use protocol::Segment;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
#[cfg(test)]
mod tests;
use crate::{
LogEntry, LoggedHistoryEntry, LoggedItem, LoggedSessionHistoryEntryId,
@@ -14,6 +17,212 @@ use crate::{
SegmentOrigin, SessionExtension, SessionId, SystemItem,
};
// Frozen v1-4 Feature wire. These types are deliberately not exported and are
// never used by LogEntry/SystemItem's current deserializers. Payload structs
// retain the owner's v1 shape rather than erasing typed fields into arbitrary JSON.
#[derive(Debug, Serialize, Deserialize)]
struct JournalStartupPayload {
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: crate::PromptRenderProvenance,
}
#[derive(Debug, Serialize, Deserialize)]
struct ResidentSummaryPayload {
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SubjectBehaviorPayload {
#[serde(default)]
behavior_md: String,
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SkillActivationPayload {
name: String,
}
#[derive(Debug, Serialize, Deserialize)]
struct InvocationResultPayload {
result: protocol::FeatureInvocationResult,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyFeatureSystemItem {
JournalStartup {
body: String,
#[serde(flatten)]
payload: JournalStartupPayload,
},
ResidentSummaryRefresh {
body: String,
#[serde(flatten)]
payload: ResidentSummaryPayload,
},
SubjectBehaviorRefresh {
body: String,
#[serde(flatten)]
payload: SubjectBehaviorPayload,
},
SkillActivation {
body: String,
#[serde(flatten)]
payload: SkillActivationPayload,
},
FeatureInvocationResult {
body: String,
#[serde(flatten)]
payload: InvocationResultPayload,
},
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacySystemItem {
Feature(LegacyFeatureSystemItem),
// Includes already-migrated Input records for interrupted migration retry.
Core(SystemItem),
}
impl LegacySystemItem {
fn into_entry(
self,
mut metadata: LoggedSessionHistoryMetadata,
) -> Result<LoggedSystemHistoryEntry, crate::InputMetadataError> {
use crate::InputCompactionUse::{ReferenceInput, Retain};
let item = match self {
Self::Core(item) => item,
Self::Feature(feature) => {
metadata.validate_input_metadata()?;
if metadata.input_metadata.is_some() {
return Err(crate::InputMetadataError::InconsistentRecord);
}
let (body, input_metadata) = match feature {
LegacyFeatureSystemItem::JournalStartup { body, payload } => {
migrated_feature_input(
body,
"journal_startup",
"journal:startup",
payload,
ReferenceInput,
)?
}
LegacyFeatureSystemItem::ResidentSummaryRefresh { body, payload } => {
migrated_feature_input(
body,
"resident_summary_refresh",
"memory:resident-summary",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SubjectBehaviorRefresh { body, payload } => {
migrated_feature_input(
body,
"subject_behavior_refresh",
"subject:behavior",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SkillActivation { body, payload } => {
migrated_feature_input(
body,
"skill_activation",
"skill:activation",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::FeatureInvocationResult { body, payload } => {
migrated_feature_input(
body,
"feature_invocation_result",
"feature:invocation-result",
payload,
Retain,
)?
}
};
// Identity/origin/derivation are retained, never re-inferred.
metadata.input_metadata = Some(input_metadata);
SystemItem::Input { body }
}
};
let entry = LoggedSystemHistoryEntry { item, metadata };
entry.validate_input_metadata()?;
Ok(entry)
}
}
fn migrated_feature_input<T: Serialize>(
body: String,
label: &str,
schema: &str,
payload: T,
compaction: crate::InputCompactionUse,
) -> Result<(String, crate::HistoryInputMetadata), crate::InputMetadataError> {
let metadata = crate::HistoryInputMetadata::new(
migration_policy(compaction),
Some(crate::FeatureInputPayload::encode(schema, 1, &payload)?),
Some(crate::PublicSystemItem {
kind: label.into(),
body: body.clone(),
presentation: crate::PublicSystemPresentation::Context,
}),
)?;
Ok((body, metadata))
}
/// Annotated old wire used by v3/v4, and by already-converted files when an
/// older marker survived an interrupted migration. No ordinary decoder uses it.
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyAnnotatedLogEntry {
AnnotatedSystemItem {
ts: u64,
entry: LegacySystemHistoryEntry,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct LegacySystemHistoryEntry {
item: LegacySystemItem,
metadata: LoggedSessionHistoryMetadata,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacyAnnotatedOrCurrent {
Annotated(LegacyAnnotatedLogEntry),
Current(LogEntry),
}
impl LegacyAnnotatedOrCurrent {
fn into_current(self) -> Result<LogEntry, crate::InputMetadataError> {
match self {
Self::Annotated(LegacyAnnotatedLogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
}) => Ok(LogEntry::AnnotatedSystemItem {
ts,
entry: entry.item.into_entry(entry.metadata)?,
extensions,
}),
Self::Current(entry) => Ok(entry),
}
}
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyHistoryLogEntry {
@@ -44,7 +253,9 @@ enum LegacyHistoryLogEntry {
},
SystemItem {
ts: u64,
item: SystemItem,
item: LegacySystemItem,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
@@ -54,7 +265,7 @@ enum LegacyHistoryLogEntry {
#[serde(untagged)]
enum LegacySessionLogEntryV1 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
/// Schema v2 retained the v1 history shapes while adding non-history records.
@@ -64,7 +275,7 @@ enum LegacySessionLogEntryV1 {
#[serde(untagged)]
enum LegacySessionLogEntryV2 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
pub(crate) fn decode_entry(
@@ -74,28 +285,207 @@ pub(crate) fn decode_entry(
segment_id: SegmentId,
line_index: usize,
) -> Result<LogEntry, serde_json::Error> {
let mut wire: serde_json::Value = serde_json::from_str(line)?;
migrate_legacy_metadata_field(&mut wire)?;
let entry = match schema_version {
1 => match serde_json::from_str::<LegacySessionLogEntryV1>(line)? {
1 => match serde_json::from_value::<LegacySessionLogEntryV1>(wire)? {
LegacySessionLogEntryV1::History(entry) => Entry::History(entry),
LegacySessionLogEntryV1::Current(entry) => Entry::Current(entry),
},
2 => match serde_json::from_str::<LegacySessionLogEntryV2>(line)? {
2 => match serde_json::from_value::<LegacySessionLogEntryV2>(wire)? {
LegacySessionLogEntryV2::History(entry) => Entry::History(entry),
LegacySessionLogEntryV2::Current(entry) => Entry::Current(entry),
},
3 | 4 => Entry::Current(serde_json::from_value(wire)?),
5 => return serde_json::from_value(wire),
_ => unreachable!("legacy decoder called for unsupported schema {schema_version}"),
};
Ok(match entry {
match entry {
Entry::History(entry) => {
canonicalize_history_entry(session_id, segment_id, line_index, entry)
}
Entry::Current(entry) => entry,
})
Entry::Current(entry) => entry.into_current(),
}
.map_err(serde::de::Error::custom)
}
// The v5 System-only metadata field was widened to role-neutral input metadata
// in v6. Touch only known history slots, never arbitrary Feature payload objects.
// Both keys are an ambiguous source, not permission to discard either value.
fn migrate_legacy_metadata_field(record: &mut serde_json::Value) -> Result<(), serde_json::Error> {
fn entry(value: &mut serde_json::Value) -> Result<(), serde_json::Error> {
if let Some(metadata) = value
.get_mut("metadata")
.and_then(serde_json::Value::as_object_mut)
{
if let Some(old) = metadata.remove("system_input") {
if metadata.contains_key("input_metadata") {
return Err(serde::de::Error::custom(
"ambiguous legacy input metadata fields",
));
}
metadata.insert("input_metadata".into(), old);
}
}
Ok(())
}
let kind = record
.get("kind")
.and_then(serde_json::Value::as_str)
.unwrap_or_default()
.to_owned();
if matches!(
kind.as_str(),
"annotated_system_item"
| "annotated_assistant_item"
| "annotated_tool_result"
| "tool_result_corrected"
) {
if let Some(value) = record.get_mut("entry") {
entry(value)?;
}
}
if let Some(history) = record
.get_mut("history")
.and_then(serde_json::Value::as_array_mut)
{
for value in history {
entry(value)?;
}
}
Ok(())
}
/// Schema 3 used DerivedSummary for Host compaction seeds. Only the trusted
/// creation boundary may identify one; arbitrary text and later seed items do not.
/// Include already-upgraded entries so an interrupted multi-file migration resumes.
pub(crate) fn compaction_summary_ids<'a>(
entries: impl Iterator<Item = &'a LogEntry>,
) -> std::collections::HashSet<LoggedSessionHistoryEntryId> {
entries
.filter_map(|entry| {
let LogEntry::AnnotatedSegmentStart {
compacted_from: Some(_),
forked_from: None,
history,
..
} = entry
else {
return None;
};
let first = history.first()?;
(matches!(
first.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
) && matches!(
first.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) && agen::Item::from(first.item.clone())
.as_text()
.is_some_and(|text| text.starts_with("[Compacted context summary]\n\n")))
.then(|| first.metadata.entry_id.clone())
})
.collect()
}
pub(crate) fn migrate_compaction_origins(
entries: &mut [LogEntry],
summaries: &std::collections::HashSet<LoggedSessionHistoryEntryId>,
) {
// Only inherited seed copies can share these identities. Appended input is
// not a construction boundary and must never be reclassified from its text.
for entry in entries {
if let LogEntry::AnnotatedSegmentStart { history, .. } = entry {
for entry in history {
if summaries.contains(&entry.metadata.entry_id)
&& entry.metadata.origin == LoggedSessionHistoryOrigin::DerivedSummary
{
entry.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
}
}
}
}
}
/// Explicit legacy-to-current boundary. No Feature inference belongs in ordinary
/// readers/restoration. Invalid or oversized payloads fail migration; the generic
/// payload/body/public limits are deliberately not relaxed for legacy data.
pub(crate) fn migrate_system_inputs(entry: &mut LogEntry) -> Result<(), crate::InputMetadataError> {
entry.validate_input_metadata()?;
match entry {
// Feature wire has already been converted by the explicit old-schema
// decoder. Ordinary LogEntry can contain only generic Input/core events.
LogEntry::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
LogEntry::AnnotatedSegmentStart { history, .. }
| LogEntry::AnnotatedUserInput { history, .. } => {
for entry in history {
migrate_flattened_system_input(entry)?;
}
}
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. }
| LogEntry::ToolResultCorrected { entry, .. } => {
migrate_flattened_system_input(entry)?;
}
_ => {}
}
entry.validate_input_metadata()
}
fn migration_policy(compaction: crate::InputCompactionUse) -> crate::HistoryInputUsePolicy {
crate::HistoryInputUsePolicy {
recording_material: crate::InputEvidenceUse::Exclude,
memory_evidence: crate::InputEvidenceUse::Exclude,
observation: crate::InputObservationUse::Hidden,
compaction,
}
}
fn migrate_flattened_system_input(
entry: &mut LoggedHistoryEntry,
) -> Result<(), crate::InputMetadataError> {
// Check before supplying legacy policy: Provided without metadata is a
// malformed NEW record, not permission to synthesize a producer policy.
entry.validate_input_metadata()?;
if entry.metadata.input_metadata.is_none()
&& (matches!(
entry.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) || matches!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
))
{
let compaction = match entry.metadata.origin {
LoggedSessionHistoryOrigin::JournalContext => crate::InputCompactionUse::ReferenceInput,
LoggedSessionHistoryOrigin::CompactionSummary => {
crate::InputCompactionUse::OpeningSummary
}
_ => crate::InputCompactionUse::Retain,
};
// Legacy derived context was excluded from independent evidence. Preserve
// that established decision explicitly, including synthetic Tool roles;
// never guess from Tool names or body text. Payload/public stay absent.
entry.metadata.input_metadata = Some(crate::HistoryInputMetadata::new(
migration_policy(compaction),
None,
None,
)?);
}
entry.validate_input_metadata()
}
enum Entry {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
fn legacy_metadata(
@@ -108,6 +498,7 @@ fn legacy_metadata(
identity.extend_from_slice(&(line_index as u64).to_be_bytes());
identity.extend_from_slice(&(item_index as u64).to_be_bytes());
LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(format!(
"l-{}",
base64::Engine::encode(&base64::engine::general_purpose::URL_SAFE_NO_PAD, identity)
@@ -122,8 +513,8 @@ fn canonicalize_history_entry(
segment_id: SegmentId,
line_index: usize,
entry: LegacyHistoryLogEntry,
) -> LogEntry {
match entry {
) -> Result<LogEntry, crate::InputMetadataError> {
Ok(match entry {
LegacyHistoryLogEntry::SegmentStart {
ts,
session_id,
@@ -177,13 +568,14 @@ fn canonicalize_history_entry(
metadata: legacy_metadata(segment_id, line_index, 0),
},
},
LegacyHistoryLogEntry::SystemItem { ts, item } => LogEntry::AnnotatedSystemItem {
LegacyHistoryLogEntry::SystemItem {
ts,
entry: LoggedSystemHistoryEntry {
item,
metadata: legacy_metadata(segment_id, line_index, 0),
},
extensions: Vec::new(),
item,
extensions,
} => LogEntry::AnnotatedSystemItem {
ts,
entry: item.into_entry(legacy_metadata(segment_id, line_index, 0))?,
extensions,
},
}
})
}
@@ -0,0 +1,105 @@
use super::*;
mod v5;
fn seed(
origin: LoggedSessionHistoryOrigin,
compacted: bool,
role: crate::LoggedRole,
text: &str,
) -> LogEntry {
LogEntry::AnnotatedSegmentStart {
ts: 1,
session_id: SessionId::now_v7(),
system_prompt: None,
config: Default::default(),
forked_from: None,
compacted_from: compacted.then(|| SegmentOrigin {
segment_id: SegmentId::now_v7(),
at_turn_index: 0,
}),
history: vec![LoggedHistoryEntry {
item: LoggedItem::Message {
role,
content: vec![crate::LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("summary".into()),
origin,
derivation: Some(crate::LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source".into())],
}),
},
}],
}
}
#[test]
fn only_trusted_compaction_seed_is_migrated_and_retries_update_inherited_copies() {
let text = "[Compacted context summary]\n\nsummary";
for (compacted, role, text) in [
(false, crate::LoggedRole::System, text),
(true, crate::LoggedRole::User, text),
(
true,
crate::LoggedRole::System,
"[Session TaskStore snapshot] tasks",
),
] {
let entry = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
compacted,
role,
text,
);
assert!(compaction_summary_ids([&entry].into_iter()).is_empty());
}
for origin in [
LoggedSessionHistoryOrigin::DerivedSummary,
LoggedSessionHistoryOrigin::CompactionSummary,
] {
let creation = seed(origin, true, crate::LoggedRole::System, text);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
text,
);
let LogEntry::AnnotatedSegmentStart { history, .. } = &inherited else {
panic!()
};
let mut pasted = history[0].clone();
pasted.metadata.entry_id = LoggedSessionHistoryEntryId("pasted".into());
let appended = LogEntry::AnnotatedUserInput {
ts: 2,
history: vec![pasted.clone()],
segments: vec![],
extensions: vec![],
};
let mut entries = vec![creation, inherited, appended];
let ids = compaction_summary_ids(entries.iter());
migrate_compaction_origins(&mut entries, &ids);
for entry in &entries[..2] {
let LogEntry::AnnotatedSegmentStart { history, .. } = entry else {
panic!()
};
assert_eq!(
history[0].metadata.origin,
LoggedSessionHistoryOrigin::CompactionSummary
);
assert_eq!(history[0].metadata.entry_id.0, "summary");
assert_eq!(
history[0].metadata.derivation.as_ref().unwrap().sources[0].0,
"source"
);
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[2] else {
panic!()
};
assert_eq!(history, &[pasted]);
let once = serde_json::to_vec(&entries).unwrap();
migrate_compaction_origins(&mut entries, &ids);
assert_eq!(serde_json::to_vec(&entries).unwrap(), once);
}
}
@@ -0,0 +1,3 @@
{"kind":"system_item","ts":1,"item":{"kind":"skill_activation","name":"legacy-skill","body":"legacy skill body"}}
{"kind":"user_input","ts":2,"segments":[{"kind":"text","content":"legacy user body"}],"extensions":[{"domain":"old","payload":{"kept":true}}]}
{"kind":"assistant_item","ts":3,"item":{"kind":"message","role":"assistant","content":[{"kind":"text","text":"legacy reply"}]}}
@@ -0,0 +1,8 @@
{"kind":"annotated_system_item","ts":101,"entry":{"item":{"kind":"journal_startup","body":"journal body\nunchanged","snapshot":{"entries":[{"text":"private snapshot"}]},"snapshot_sha256":"snapshot-digest","prompt_provenance":{"workspace_id":"workspace","source_digest":"source","projection_digest":"projection","logical_name":"journal_startup"}},"metadata":{"entry_id":"journal-id","origin":{"kind":"legacy_unknown"},"derivation":{"sources":["source-id"]}}},"extensions":[{"domain":"migration-fixture","payload":{"kept":true}}]}
{"kind":"annotated_system_item","ts":102,"entry":{"item":{"kind":"resident_summary_refresh","body":"resident body"},"metadata":{"entry_id":"resident-id","origin":{"kind":"derived_summary"}}}}
{"kind":"annotated_system_item","ts":103,"entry":{"item":{"kind":"subject_behavior_refresh","body":"rendered subject body","behavior_md":"exact behavior","prompt_provenance":{"source_digest":"subject-source","projection_digest":"subject-projection","logical_name":"subject_behavior"}},"metadata":{"entry_id":"subject-id","origin":{"kind":"backend_instruction","operation_id":"op"}}}}
{"kind":"annotated_system_item","ts":104,"entry":{"item":{"kind":"skill_activation","body":"skill body","name":"example-skill"},"metadata":{"entry_id":"skill-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":105,"entry":{"item":{"kind":"feature_invocation_result","body":"result body","result":{"invocation_id":"invoke-id","identity":"builtin:test/prepare","status":"outcome_unknown","message":"may have committed","context":null}},"metadata":{"entry_id":"result-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_user_input","ts":106,"segments":[],"history":[{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Journal] prefix is not Feature evidence"}]},"metadata":{"entry_id":"flat-journal","origin":{"kind":"journal_context"},"derivation":{"sources":["journal-id"]}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"summary body"}]},"metadata":{"entry_id":"flat-summary","origin":{"kind":"compaction_summary"}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Skill: example] never infer Feature"}]},"metadata":{"entry_id":"flat-other","origin":{"kind":"legacy_unknown"}}}]}
{"kind":"annotated_system_item","ts":107,"entry":{"item":{"kind":"notification","message":"raw message","body":"wrapped message"},"metadata":{"entry_id":"core-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":108,"entry":{"item":{"kind":"input","body":"already generic body"},"metadata":{"entry_id":"generic-id","origin":{"kind":"provided","producer":"future-owner"},"derivation":{"sources":["source-id"]},"system_input":{"usage":{"recording_material":"include","memory_evidence":"exclude","observation":"readable","compaction":"retain"},"feature_payload":{"schema":"future:unknown","version":99,"value":{"opaque":"retained"}},"public":null}}}}
@@ -0,0 +1,5 @@
{"kind":"annotated_system_item","ts":201,"entry":{"item":{"kind":"input","body":"frozen v5 reference"},"metadata":{"entry_id":"reference-v5","origin":{"kind":"provided","producer":"owner-v5"},"system_input":{"usage":{"recording_material":"exclude","memory_evidence":"exclude","observation":"hidden","compaction":"reference_input"},"feature_payload":{"schema":"future:owner","version":8,"value":{"system_input":"opaque owner field must not be renamed"}},"public":null}}}}
{"kind":"annotated_assistant_item","ts":202,"entry":{"item":{"kind":"tool_call","call_id":"original-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"original-call-entry","origin":{"kind":"model_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_tool_result","ts":203,"entry":{"item":{"kind":"tool_result","call_id":"original-call","summary":"original result"},"metadata":{"entry_id":"original-result-entry","origin":{"kind":"tool_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_assistant_item","ts":204,"entry":{"item":{"kind":"tool_call","call_id":"derived-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"derived-call-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
{"kind":"annotated_tool_result","ts":205,"entry":{"item":{"kind":"tool_result","call_id":"derived-call","summary":"derived retrieval result"},"metadata":{"entry_id":"derived-result-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
@@ -0,0 +1,709 @@
//! Fixed old wire bytes, not fixtures generated by the current serializer.
use super::*;
use crate::{FsStore, InputCompactionUse, Store, WorkerSessionStore};
use serde_json::{Value, json};
use std::{fs, path::Path};
const V4: &str = include_str!("fixtures/v4-system-inputs.jsonl");
const V1: &str = include_str!("fixtures/v1-system-inputs.jsonl");
fn decode_fixture(version: u32, source: &str, segment: SegmentId) -> Vec<LogEntry> {
source
.lines()
.enumerate()
.map(|(index, line)| decode_entry(version, line, SessionId::nil(), segment, index).unwrap())
.collect()
}
fn encode(entries: &[LogEntry]) -> Vec<u8> {
let mut bytes = Vec::new();
for entry in entries {
serde_json::to_writer(&mut bytes, entry).unwrap();
bytes.push(b'\n');
}
bytes
}
fn migrate(entries: &mut [LogEntry]) {
for entry in entries {
migrate_system_inputs(entry).unwrap();
entry.validate_input_metadata().unwrap();
}
}
fn write_worker(root: &Path, version: u32, source: &[u8]) -> (SessionId, SegmentId) {
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
fs::create_dir_all(root.join("segments")).unwrap();
fs::write(
root.join("session.json"),
serde_json::to_vec(&json!({
"schema_version": version, "session_id": session,
}))
.unwrap(),
)
.unwrap();
fs::write(
root.join("segments").join(format!("{segment}.jsonl")),
source,
)
.unwrap();
(session, segment)
}
fn assert_marker(root: &Path, version: u32) {
let manifest: Value =
serde_json::from_slice(&fs::read(root.join("session.json")).unwrap()).unwrap();
assert_eq!(manifest["schema_version"], version);
}
#[test]
fn v4_fixed_wire_preserves_identity_body_payload_public_and_independent_policy() {
let mut entries = decode_fixture(4, V4, SegmentId::nil());
let before = entries.clone();
migrate(&mut entries);
let schemas = [
"journal:startup",
"memory:resident-summary",
"subject:behavior",
"skill:activation",
"feature:invocation-result",
];
let old_wire: Vec<Value> = V4
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
for index in 0..5 {
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = &entries[index]
else {
panic!()
};
let old = &old_wire[index];
let old_item = &old["entry"]["item"];
let old_metadata: LoggedSessionHistoryMetadata =
serde_json::from_value(old["entry"]["metadata"].clone()).unwrap();
let old_ext: Vec<crate::SessionExtension> = old
.get("extensions")
.map(|value| serde_json::from_value(value.clone()).unwrap())
.unwrap_or_default();
assert_eq!(*ts, old["ts"].as_u64().unwrap());
assert_eq!(extensions, &old_ext);
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
old_item["body"].as_str().unwrap()
);
assert_eq!(entry.metadata.entry_id, old_metadata.entry_id);
assert_eq!(entry.metadata.origin, old_metadata.origin);
assert_eq!(entry.metadata.derivation, old_metadata.derivation);
let metadata = entry.metadata.input_metadata.as_ref().unwrap();
let public = metadata.public_item().unwrap();
assert_eq!(public.kind, old_item["kind"].as_str().unwrap());
assert_eq!(public.body, old_item["body"].as_str().unwrap());
assert_eq!(
public.presentation,
crate::PublicSystemPresentation::Context
);
assert_eq!(
metadata.usage(),
migration_policy(if index == 0 {
InputCompactionUse::ReferenceInput
} else {
InputCompactionUse::Retain
})
);
let payload = metadata.feature_payload().unwrap();
assert_eq!(payload.schema(), schemas[index]);
assert_eq!(payload.version(), 1);
let value: Value = payload.decode(schemas[index], 1).unwrap();
// Independently require each frozen owner's typed v1 payload contract.
match index {
0 => {
let _: JournalStartupPayload = payload.decode(schemas[index], 1).unwrap();
}
1 => {
let _: ResidentSummaryPayload = payload.decode(schemas[index], 1).unwrap();
}
2 => {
let _: SubjectBehaviorPayload = payload.decode(schemas[index], 1).unwrap();
}
3 => {
let _: SkillActivationPayload = payload.decode(schemas[index], 1).unwrap();
}
4 => {
let _: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
}
_ => unreachable!(),
}
let mut expected = old_item.clone();
expected.as_object_mut().unwrap().remove("kind");
expected.as_object_mut().unwrap().remove("body");
// Optional provenance is an explicit Option in the v1 payload contract.
if index == 1 {
expected["prompt_provenance"] = Value::Null;
}
if index == 4 {
let result: protocol::FeatureInvocationResult =
serde_json::from_value(old_item["result"].clone()).unwrap();
let typed: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
assert_eq!(typed.result, result);
// Preserve the protocol's typed serialization contract, including
// its canonical representation of optional result context.
expected["result"] = serde_json::to_value(result).unwrap();
}
assert_eq!(value, expected);
assert!(value.get("body").is_none());
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[5] else {
panic!()
};
let LogEntry::AnnotatedUserInput { history: old, .. } = &before[5] else {
panic!()
};
for (index, compaction) in [
InputCompactionUse::ReferenceInput,
InputCompactionUse::OpeningSummary,
InputCompactionUse::Retain,
]
.into_iter()
.enumerate()
{
assert_eq!(history[index].item, old[index].item);
assert_eq!(
history[index].metadata.entry_id,
old[index].metadata.entry_id
);
assert_eq!(history[index].metadata.origin, old[index].metadata.origin);
assert_eq!(
history[index].metadata.derivation,
old[index].metadata.derivation
);
let metadata = history[index].metadata.input_metadata.as_ref().unwrap();
assert_eq!(metadata.usage(), migration_policy(compaction));
assert!(metadata.public_item().is_none());
assert!(metadata.feature_payload().is_none());
}
// Core notification is unchanged; existing generic Input and unknown valid
// owner payload are retained without decoding/reclassifying a Feature.
assert_eq!(encode(&entries[6..]), encode(&before[6..]));
let once = encode(&entries);
migrate(&mut entries);
assert_eq!(encode(&entries), once);
}
#[test]
fn v1_through_v4_worker_migration_reopen_and_mixed_file_retry() {
for version in 1..=4 {
let source = if version <= 2 { V1 } else { V4 };
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), version, source.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
// Simulate an interrupted publication: v5 records precede legacy records
// in one file, and a fully new file coexists with an untouched old file.
let mut converted = decode_fixture(version, source, segment);
migrate(&mut converted);
let mut mixed = encode(&converted[..1]);
mixed.extend_from_slice(source.split_once('\n').unwrap().1.as_bytes());
fs::write(&path, &mixed).unwrap();
let new_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let new_bytes = encode(&converted);
fs::write(&new_path, &new_bytes).unwrap();
let old_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(3)));
fs::write(&old_path, source).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
WorkerSessionStore::validate_migration(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), mixed);
assert_eq!(fs::read(&old_path).unwrap(), source.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let store = WorkerSessionStore::new(root.path()).unwrap();
assert_marker(root.path(), 6);
let restored = store.read_all(session, segment).unwrap();
assert_eq!(encode(&restored), new_bytes);
assert_eq!(fs::read(&new_path).unwrap(), new_bytes);
let once = fs::read(&path).unwrap();
let old_once = fs::read(&old_path).unwrap();
drop(store);
let reopened = WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(encode(&reopened.read_all(session, segment).unwrap()), once);
assert_eq!(fs::read(&path).unwrap(), once);
assert_eq!(fs::read(&old_path).unwrap(), old_once);
}
}
#[test]
fn all_segments_preflight_keeps_old_bytes_on_decode_or_conversion_failure() {
let mut oversized: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
oversized["entry"]["item"]["snapshot"] =
json!({"large": "x".repeat(crate::input_metadata::MAX_FEATURE_PAYLOAD_BYTES + 1)});
let mut unknown: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
unknown["entry"]["item"]["kind"] = json!("unsupported_future_feature");
let mut malformed = unknown.clone();
malformed["entry"]["item"] = json!({"kind":"journal_startup", "body":"missing snapshot"});
let mut missing_metadata: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing_metadata["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let mut missing_input = missing_metadata.clone();
missing_input["entry"]["metadata"]["origin"] = json!({"kind":"legacy_unknown"});
let mut invalid_policy: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
invalid_policy["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("unknown");
let mut invalid_payload = invalid_policy.clone();
invalid_payload["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("retain");
invalid_payload["entry"]["metadata"]["system_input"]["feature_payload"]["version"] = json!(0);
let mut flat_provided: Value = serde_json::from_str(V4.lines().nth(5).unwrap()).unwrap();
flat_provided["history"][0]["metadata"]["origin"] =
json!({"kind":"provided", "producer":"owner"});
for bad in [
"{broken-json}\n".to_owned(),
format!("{}\n", oversized),
format!("{}\n", unknown),
format!("{}\n", malformed),
format!("{}\n", missing_metadata),
format!("{}\n", invalid_policy),
format!("{}\n", invalid_payload),
format!("{}\n", flat_provided),
format!("{}\n", missing_input),
] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 4, V4.as_bytes());
let first = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&later, &bad).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::validate_migration(root.path()).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read(&later).unwrap(), bad.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn unsupported_schema_preserves_all_old_bytes() {
for version in [0, 7, 99] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V4.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), V4.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn standalone_store_uses_same_v1_through_v4_migration_and_reopens() {
// None represents the existing unmarked annotated standalone layout (v3).
for version in [None, Some(1), Some(2), Some(3), Some(4)] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let source = if version.is_some_and(|v| v <= 2) {
V1
} else {
V4
};
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, source).unwrap();
if let Some(version) = version {
fs::write(dir.join("session-schema.json"), version.to_string()).unwrap();
}
let store = FsStore::new(root.path()).unwrap();
let entries = store.read_all(session, segment).unwrap();
for entry in entries {
entry.validate_input_metadata().unwrap();
}
assert_eq!(
fs::read_to_string(dir.join("session-schema.json"))
.unwrap()
.trim(),
"6"
);
let once = fs::read(&path).unwrap();
drop(store);
FsStore::new(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), once);
}
}
#[test]
fn compaction_origin_upgrade_is_retained_only_for_pre_v4_sources() {
for version in [3, 4] {
let root = tempfile::tempdir().unwrap();
let creation = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
true,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let (session, segment) =
write_worker(root.path(), version, &encode(&[creation, inherited]));
let store = WorkerSessionStore::new(root.path()).unwrap();
for record in store.read_all(session, segment).unwrap() {
let LogEntry::AnnotatedSegmentStart { history, .. } = record else {
panic!()
};
let metadata = &history[0].metadata;
assert_eq!(metadata.entry_id.0, "summary");
assert_eq!(metadata.derivation.as_ref().unwrap().sources[0].0, "source");
assert_eq!(
metadata.origin,
if version == 3 {
LoggedSessionHistoryOrigin::CompactionSummary
} else {
LoggedSessionHistoryOrigin::DerivedSummary
}
);
assert_eq!(
metadata.input_metadata.as_ref().unwrap().usage().compaction,
if version == 3 {
InputCompactionUse::OpeningSummary
} else {
InputCompactionUse::Retain
}
);
}
}
}
#[test]
fn standalone_preflight_and_unsupported_marker_leave_sources_unchanged() {
for version in [4, 99] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let first = dir.join(format!("{}.jsonl", SegmentId::from_u128(1)));
let later = dir.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&first, V4).unwrap();
fs::write(&later, "{broken-json}\n").unwrap();
let marker = version.to_string();
fs::write(dir.join("session-schema.json"), &marker).unwrap();
assert!(FsStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read_to_string(&later).unwrap(), "{broken-json}\n");
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
marker
);
}
}
#[test]
fn current_v5_rejects_all_legacy_feature_kinds_without_rewriting_sources() {
for line in V4.lines().take(5) {
// Current serde, writable open and read-only restore share the same
// current wire contract. None may dispatch to the old Feature decoder.
assert!(serde_json::from_str::<LogEntry>(line).is_err());
let bytes = format!("{line}\n");
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), 6, bytes.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
let readonly = WorkerSessionStore::open_read_only(root.path()).unwrap();
assert!(readonly.read_all_read_only(session, segment).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let standalone = tempfile::tempdir().unwrap();
let dir = standalone.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, &bytes).unwrap();
fs::write(dir.join("session-schema.json"), "5").unwrap();
assert!(FsStore::new(standalone.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
"5"
);
}
}
#[test]
fn v1_v2_feature_wire_uses_the_same_typed_payloads_and_stable_legacy_identity() {
let segment = SegmentId::from_u128(42);
for version in [1, 2] {
for (index, line) in V4.lines().take(5).enumerate() {
let annotated: Value = serde_json::from_str(line).unwrap();
let wire = json!({
"kind": "system_item", "ts": annotated["ts"],
"item": annotated["entry"]["item"],
"extensions": [{"domain":"old-owner", "payload":{"kept":true}}],
})
.to_string();
let entry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
let retry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
assert_eq!(encode(&[entry.clone()]), encode(&[retry]));
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = entry
else {
panic!()
};
assert_eq!(ts, annotated["ts"].as_u64().unwrap());
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
annotated["entry"]["item"]["body"].as_str().unwrap()
);
assert_eq!(
entry.metadata.entry_id,
legacy_metadata(segment, index, 0).entry_id
);
assert_eq!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::LegacyUnknown
);
assert!(entry.metadata.derivation.is_none());
assert_eq!(extensions[0].domain, "old-owner");
assert_eq!(extensions[0].payload, json!({"kept":true}));
let LogEntry::AnnotatedSystemItem { entry: v4, .. } =
decode_entry(4, line, SessionId::nil(), segment, index).unwrap()
else {
panic!()
};
assert_eq!(entry.metadata.input_metadata, v4.metadata.input_metadata);
}
}
}
#[test]
fn migration_rejects_malformed_typed_feature_fields_instead_of_retaining_unchecked_json() {
let mut items: Vec<Value> = V4
.lines()
.take(5)
.map(|line| serde_json::from_str::<Value>(line).unwrap()["entry"]["item"].clone())
.collect();
items[0]["snapshot_sha256"] = json!(17);
items[1]["prompt_provenance"] = json!({"logical_name":"missing digests"});
items[2]["behavior_md"] = json!(["not a string"]);
items[3]["name"] = json!({"not":"a name"});
items[4]["result"]["status"] = json!("unsupported_status");
for version in 1..=4 {
for item in &items {
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"malformed-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).is_err());
}
}
}
#[test]
fn frozen_behavior_counter_is_accepted_only_at_the_old_schema_boundary() {
for version in 1..=4 {
let item = json!({
"kind":"subject_behavior_refresh", "body":"exact archived body",
"behavior_revision":7,
});
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"behavior-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(serde_json::from_str::<LogEntry>(&wire).is_err());
let LogEntry::AnnotatedSystemItem { entry, .. } =
decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).unwrap()
else {
panic!()
};
assert_eq!(entry.item.history_text(), "exact archived body");
let payload = entry
.metadata
.input_metadata
.as_ref()
.unwrap()
.feature_payload()
.unwrap();
let typed: SubjectBehaviorPayload = payload.decode("subject:behavior", 1).unwrap();
assert_eq!(typed.behavior_md, "");
assert!(typed.prompt_provenance.is_none());
assert_eq!(
serde_json::to_value(typed).unwrap(),
json!({
"behavior_md":"", "prompt_provenance":null,
})
);
}
}
#[test]
fn current_schema_rejects_generic_input_without_metadata_instead_of_migrating_it() {
let mut missing: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let bytes = format!("{}\n", missing);
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 6, bytes.as_bytes());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(
fs::read(
root.path()
.join("segments")
.join(format!("{segment}.jsonl"))
)
.unwrap(),
bytes.as_bytes()
);
assert_marker(root.path(), 6);
}
const V5: &str = include_str!("fixtures/v5-role-metadata.jsonl");
#[test]
fn v5_metadata_and_derived_tool_context_upgrade_without_changing_roles_or_original_evidence() {
for standalone in [false, true] {
let root = tempfile::tempdir().unwrap();
let (session, segment) = if standalone {
let session = SessionId::now_v7();
let segment = SegmentId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
fs::write(dir.join("session-schema.json"), b"5").unwrap();
fs::write(dir.join(format!("{segment}.jsonl")), V5).unwrap();
(session, segment)
} else {
write_worker(root.path(), 5, V5.as_bytes())
};
let entries = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
let originals: Vec<Value> = V5
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
assert_eq!(entries.len(), originals.len());
for (index, (entry, old)) in entries.iter().zip(originals).enumerate() {
let wire = serde_json::to_value(entry).unwrap();
assert_eq!(wire["ts"], old["ts"]);
assert_eq!(wire["entry"]["item"], old["entry"]["item"]);
for field in ["entry_id", "origin", "derivation"] {
assert_eq!(
wire["entry"]["metadata"][field],
old["entry"]["metadata"][field]
);
}
let metadata = &wire["entry"]["metadata"];
if index == 0 {
assert_eq!(
metadata["input_metadata"],
old["entry"]["metadata"]["system_input"]
);
assert_eq!(
metadata["input_metadata"]["feature_payload"]["value"]["system_input"],
"opaque owner field must not be renamed"
);
} else if index < 3 {
assert!(
metadata.get("input_metadata").is_none(),
"ordinary model Tool originals must not acquire exclusion policy"
);
} else {
assert_eq!(
metadata["input_metadata"]["usage"],
json!({
"recording_material":"exclude", "memory_evidence":"exclude", "observation":"hidden", "compaction":"retain"
})
);
}
}
let before = encode(&entries);
let reopened = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
assert_eq!(encode(&reopened), before);
}
}
#[test]
fn legacy_metadata_alias_conflict_fails_preflight_without_dropping_either_value() {
let mut bad: Value = serde_json::from_str(V5.lines().next().unwrap()).unwrap();
bad["entry"]["metadata"]["input_metadata"] = bad["entry"]["metadata"]["system_input"].clone();
for version in [4, 5] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V5.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let bad_bytes = format!("{bad}\n");
fs::write(&later, &bad_bytes).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(path).unwrap(), V5.as_bytes());
assert_eq!(fs::read(later).unwrap(), bad_bytes.as_bytes());
assert_marker(root.path(), version);
}
}
+9 -1
View File
@@ -41,11 +41,14 @@ pub(crate) fn read_without_atime_bounded(path: &Path, max_bytes: u64) -> io::Res
pub mod event_trace;
pub mod fs_store;
pub mod history;
pub mod input_metadata;
mod legacy_session_log;
pub mod logged_item;
mod paste_artifact;
pub mod public_index;
pub mod public_material;
pub mod public_snapshot;
pub mod retention;
pub mod segment;
pub mod segment_log;
pub mod store;
@@ -63,6 +66,10 @@ pub use history::{
LoggedSessionHistoryMetadata, LoggedSessionHistoryOrigin, LoggedSystemHistoryEntry,
LoggedWorkerSubject,
};
pub use input_metadata::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputMetadataError, InputObservationUse, SystemInputEnvelope,
};
pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged};
pub use paste_artifact::PasteArtifactLimits;
pub use public_index::{
@@ -96,7 +103,8 @@ pub use segment::{
pub use segment_log::{LogEntry, RestoredState, SegmentOrigin, SessionExtension, collect_state};
pub use store::{Store, StoreError};
pub use system_item::{
PromptRenderProvenance, SystemItem, SystemReminder, SystemReminderSource, render_worker_event,
PromptRenderProvenance, PublicSystemItem, PublicSystemPresentation, SystemItem, SystemReminder,
SystemReminderSource, render_worker_event,
};
pub use uploaded_file::{
DEFAULT_MAX_FILES_PER_SUBMISSION, DEFAULT_MAX_SESSION_ARTIFACT_BYTES,
+7 -1
View File
@@ -1296,6 +1296,7 @@ mod tests {
LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.to_owned()),
origin,
derivation: None,
@@ -1537,6 +1538,7 @@ mod tests {
prompt_provenance: None,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("system-item".into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction {
operation_id: None,
@@ -1602,6 +1604,7 @@ mod tests {
segment_id(20),
"completed",
LogEntry::RunCompleted {
entry_id: None,
ts: 4,
interrupted: false,
result: EngineResult::Finished,
@@ -1640,7 +1643,10 @@ mod tests {
(
segment_id(24),
"abandoned",
LogEntry::PausedTurnAbandoned { ts: 4 },
LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 4,
},
),
];
for (segment_id, text, outcome) in outcomes {
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+76 -21
View File
@@ -27,7 +27,7 @@ use thiserror::Error;
use crate::{
LogEntry, LoggedContentPart, LoggedHistoryEntry, LoggedItem, LoggedRole,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError, SystemItem,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError,
WorkerAggregateStore, WorkerSessionStore, WorkerStoreError,
};
@@ -1165,7 +1165,7 @@ fn capture_tool_correction(
let LogEntry::ToolResultCorrected { entry, .. } = record else {
return false;
};
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
// We read backwards: the first correction seen is the latest one.
corrections
.entry(entry.metadata.entry_id.0.clone())
@@ -1600,12 +1600,12 @@ pub(crate) fn project_session_snapshot_for_segment(
} => extend_history(&mut entries, history, Some(segments), *ts),
LogEntry::AnnotatedAssistantItem { ts, entry }
| LogEntry::AnnotatedToolResult { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
entries.push(history_entry(entry, *ts, data));
}
}
LogEntry::ToolResultCorrected { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
if let Some(existing) = entries
.iter_mut()
.find(|existing| existing.entry_id == entry.metadata.entry_id.0)
@@ -1618,11 +1618,16 @@ pub(crate) fn project_session_snapshot_for_segment(
}
}
LogEntry::AnnotatedSystemItem { ts, entry, .. } => entries.push(system_entry(
&entry.item,
&entry.public_item(),
entry.metadata.entry_id.0.clone(),
*ts,
provenance(&entry.metadata.origin),
derivation_ids(entry),
entry
.metadata
.input_metadata
.as_ref()
.map(crate::HistoryInputMetadata::public_system_use_policy),
)),
LogEntry::RunYielded {
ts,
@@ -1705,6 +1710,7 @@ pub(crate) fn project_session_snapshot_for_segment(
interrupted,
result,
active_run_turn_count: persisted_count,
..
} => {
if *interrupted && matches!(result, agen::EngineResult::Yielded) {
let count = (*persisted_count)
@@ -1774,7 +1780,7 @@ fn extend_history(
segments: input_segments.cloned().unwrap_or_default(),
}
} else {
let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) else {
let Some(data) = project_history_item(entry) else {
continue;
};
data
@@ -1789,6 +1795,19 @@ fn history_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: entry.metadata.input_metadata.as_ref().map(|input| {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
input.public_system_use_policy()
} else {
input.public_use_policy()
}
}),
entry_id: entry.metadata.entry_id.0.clone(),
timestamp,
provenance: provenance(&entry.metadata.origin),
@@ -1832,6 +1851,7 @@ fn run_transition_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: persisted_entry_id
.map(|entry_id| entry_id.0.clone())
.unwrap_or_else(|| legacy_entry_id(session_key, segment_id, log_index, 0)),
@@ -1851,6 +1871,7 @@ fn legacy_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: legacy_entry_id(session_key, segment_id, log_index, item_index),
timestamp,
provenance: SessionEntryProvenance::LegacyUnknown,
@@ -1883,12 +1904,12 @@ fn provenance(origin: &LoggedSessionHistoryOrigin) -> SessionEntryProvenance {
LoggedSessionHistoryOrigin::FlowInstruction { .. } => {
SessionEntryProvenance::FlowInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. } => {
SessionEntryProvenance::BackendInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. }
| LoggedSessionHistoryOrigin::Provided { .. } => SessionEntryProvenance::BackendInstruction,
LoggedSessionHistoryOrigin::ModelOutput { .. } => SessionEntryProvenance::ModelOutput,
LoggedSessionHistoryOrigin::ToolOutput { .. } => SessionEntryProvenance::ToolOutput,
LoggedSessionHistoryOrigin::CompactionSummary
| LoggedSessionHistoryOrigin::JournalContext
| LoggedSessionHistoryOrigin::DerivedSummary => SessionEntryProvenance::DerivedSummary,
LoggedSessionHistoryOrigin::LegacyUnknown => SessionEntryProvenance::LegacyUnknown,
}
@@ -1912,6 +1933,35 @@ pub fn session_tool_attachment_id(
URL_SAFE_NO_PAD.encode(hash.finalize())
}
fn project_history_item(entry: &LoggedHistoryEntry) -> Option<SessionSnapshotEntryData> {
if let Some(input) = &entry.metadata.input_metadata {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
let public = input.public_projection();
return Some(SessionSnapshotEntryData::SystemItem {
item_kind: public.kind.clone(),
content: public.body.clone(),
data: Some(
serde_json::to_value(public).expect("public system input is serializable"),
),
});
}
let usage = input.usage();
if usage.recording_material == crate::InputEvidenceUse::Exclude
&& usage.memory_evidence == crate::InputEvidenceUse::Exclude
&& usage.observation == crate::InputObservationUse::Hidden
{
return None;
}
}
project_item(&entry.metadata.entry_id.0, &entry.item)
}
fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntryData> {
match item {
LoggedItem::Message { role, content } => {
@@ -1984,30 +2034,26 @@ fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntr
}
fn system_entry(
item: &SystemItem,
item: &crate::PublicSystemItem,
entry_id: String,
timestamp: u64,
provenance: SessionEntryProvenance,
derived_from: Vec<String>,
use_policy: Option<protocol::SessionEntryUsePolicy>,
) -> SessionSnapshotEntry {
let mut data = serde_json::to_value(item).ok();
if let Some(serde_json::Value::Object(object)) = data.as_mut() {
object.remove("prompt_provenance");
}
let item_kind = data
.as_ref()
.and_then(|value| value.get("kind"))
.and_then(serde_json::Value::as_str)
.unwrap_or("system_item")
.to_owned();
let public = item.clone();
let data = Some(serde_json::to_value(&public).expect("public system input is serializable"));
let item_kind = public.kind;
let content = public.body;
SessionSnapshotEntry {
use_policy,
entry_id,
timestamp,
provenance,
derived_from,
data: SessionSnapshotEntryData::SystemItem {
item_kind,
content: item.history_text(),
content,
data,
},
}
@@ -2037,6 +2083,7 @@ mod tests {
fn complete(ts: u64) -> LogEntry {
LogEntry::RunCompleted {
entry_id: None,
ts,
interrupted: false,
result: agen::EngineResult::Finished,
@@ -2288,6 +2335,7 @@ mod tests {
content: vec![LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(entry_id.into()),
origin,
derivation: None,
@@ -3586,6 +3634,7 @@ mod tests {
.map(|item| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -3640,6 +3689,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id,
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -3709,6 +3759,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::FlowInstruction {
selector: "builtin:coder-review".into(),
@@ -3728,6 +3779,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id.clone(),
origin: origin.clone(),
derivation: Some(LoggedHistoryDerivation {
@@ -3758,6 +3810,7 @@ mod tests {
fn annotated_projection_preserves_identity_and_provenance() {
let session_id = crate::new_session_id();
let metadata = LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::ModelOutput {
worker: LoggedWorkerSubject {
@@ -4320,6 +4373,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("ancestor-image".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4414,6 +4468,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("tool-images".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -103,6 +103,7 @@ fn finish(
stats.insert(
anchor.clone(),
SessionSnapshotEntry {
use_policy: None,
entry_id: format!("run-stats:{anchor}"),
timestamp: ended_at,
provenance: SessionEntryProvenance::LegacyUnknown,
+398
View File
@@ -0,0 +1,398 @@
//! Durable, domain-neutral Session retention pins and permanent erasure fences.
//!
//! Hosts choose a control root whose lifetime outlasts the physical Session.
//! Every deletion path must execute its physical removal inside `delete`, and
//! publishers must hold `lock_live` while creating or appending source data.
//! A check followed by unlocked removal is not a retention guarantee.
//!
//! Pins have explicit owners and no timer/expiry: domain code releases its pin
//! only after its durable consumer no longer needs originals. A permanent erase
//! persists its fence before physical removal, invalidates pins, and rejects
//! late pins/publication even after reopening the control store. Retrying an
//! interrupted physical erase uses the same intent and is safe.
use std::collections::BTreeSet;
use std::fs::{self, File, OpenOptions};
use std::path::{Path, PathBuf};
use fs4::fs_std::FileExt;
use serde::{Deserialize, Serialize};
use crate::{SessionId, StoreError};
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(transparent)]
pub struct SessionRetentionPin(pub uuid::Uuid);
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SessionDeletionIntent {
Ordinary,
ErasePermanently,
}
#[derive(Debug, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct RetentionState {
erased: bool,
pins: BTreeSet<SessionRetentionPin>,
}
#[derive(Debug, Clone)]
pub struct SessionRetentionStore {
root: PathBuf,
}
/// A process-independent lock; dropping it releases the filesystem lock.
/// Do not recursively acquire the same Session lock inside a guarded callback.
pub struct SessionRetentionGuard {
_file: File,
}
impl SessionRetentionStore {
pub fn new(control_root: impl Into<PathBuf>) -> Self {
Self {
root: control_root.into(),
}
}
fn state_path(&self, session: SessionId) -> PathBuf {
self.root.join(format!("{session}.json"))
}
fn lock(&self, session: SessionId) -> Result<SessionRetentionGuard, StoreError> {
fs::create_dir_all(&self.root)?;
let mut options = OpenOptions::new();
options.read(true).write(true).create(true).truncate(false);
#[cfg(any(target_os = "linux", target_os = "android"))]
{
use std::os::unix::fs::OpenOptionsExt;
// Control roots are Host-owned; never follow a replaced lock file.
options.custom_flags(libc::O_NOFOLLOW);
}
let file = options.open(self.root.join(format!("{session}.lock")))?;
FileExt::lock_exclusive(&file)?;
Ok(SessionRetentionGuard { _file: file })
}
fn read(&self, session: SessionId) -> Result<RetentionState, StoreError> {
match fs::read(self.state_path(session)) {
Ok(bytes) => Ok(serde_json::from_slice(&bytes)?),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
Ok(RetentionState::default())
}
Err(error) => Err(error.into()),
}
}
fn save(&self, session: SessionId, state: &RetentionState) -> Result<(), StoreError> {
crate::worker_session_store::atomic_write_bytes(
&self.state_path(session),
&serde_json::to_vec(state)?,
)
}
/// Observational fence check; unlike `lock_live`, never creates storage.
/// Hosts still fence result adoption in their own durable consumer: a read
/// finishing before erasure is not authority for a delayed domain write.
pub fn ensure_live(&self, session: SessionId) -> Result<(), StoreError> {
if self.read(session)?.erased {
return Err(StoreError::SessionErased(session));
}
Ok(())
}
pub fn lock_live(&self, session: SessionId) -> Result<SessionRetentionGuard, StoreError> {
let guard = self.lock(session)?;
if self.read(session)?.erased {
return Err(StoreError::SessionErased(session));
}
Ok(guard)
}
/// Protect a Host-owned Session before its first source publication. This is
/// retention intent, not proof that any original material exists. The owner must
/// establish it before accepting input and use the same control store for writes
/// and deletion. Source-serving APIs must still verify actual material separately.
pub fn retain_before_publication(
&self,
session: SessionId,
pin: SessionRetentionPin,
) -> Result<(), StoreError> {
let _guard = self.lock_live(session)?;
let mut state = self.read(session)?;
if state.pins.insert(pin) {
self.save(session, &state)?;
}
Ok(())
}
/// Call `source_exists` under the same lock used by deletion. Replaying the
/// same pin is idempotent; missing source never becomes a successful lease.
pub fn pin(
&self,
session: SessionId,
pin: SessionRetentionPin,
source_exists: impl FnOnce() -> Result<bool, StoreError>,
) -> Result<(), StoreError> {
let _guard = self.lock_live(session)?;
if !source_exists()? {
return Err(StoreError::SessionSourceMissing(session));
}
let mut state = self.read(session)?;
if state.pins.insert(pin) {
self.save(session, &state)?;
}
Ok(())
}
pub fn release(&self, session: SessionId, pin: SessionRetentionPin) -> Result<(), StoreError> {
let _guard = self.lock(session)?;
let mut state = self.read(session)?;
if state.pins.remove(&pin) {
self.save(session, &state)?;
}
Ok(())
}
pub fn pins(&self, session: SessionId) -> Result<Vec<SessionRetentionPin>, StoreError> {
let _guard = self.lock(session)?;
Ok(self.read(session)?.pins.into_iter().collect())
}
/// Execute physical removal while fencing concurrent pins and publishers.
/// An erasure fence is retained even if removal fails or the source is
/// already absent. Ordinary deletion after erasure is rejected: recovery
/// must preserve the caller's explicit erase intent.
pub fn delete<T>(
&self,
session: SessionId,
intent: SessionDeletionIntent,
remove: impl FnOnce() -> Result<T, StoreError>,
) -> Result<T, StoreError> {
let _guard = self.lock(session)?;
let mut state = self.read(session)?;
match intent {
SessionDeletionIntent::Ordinary => {
if state.erased {
return Err(StoreError::SessionErased(session));
}
if !state.pins.is_empty() {
return Err(StoreError::SessionPinned(session));
}
}
SessionDeletionIntent::ErasePermanently => {
if !state.erased || !state.pins.is_empty() {
state.erased = true;
state.pins.clear();
self.save(session, &state)?;
}
}
}
remove()
}
}
/// FsStore's control state is outside all per-Session source directories.
pub(crate) fn fs_retention(root: &Path) -> SessionRetentionStore {
SessionRetentionStore::new(root.join(".retention"))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn publication_intent_survives_reopen_without_attesting_missing_originals() {
let temp = tempfile::tempdir().unwrap();
let source = temp.path().join("source");
let controls = temp.path().join("controls");
let session = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let retention = SessionRetentionStore::new(&controls);
retention.retain_before_publication(session, pin).unwrap();
let reopened = SessionRetentionStore::new(&controls);
assert_eq!(reopened.pins(session).unwrap(), [pin]);
// An early owner is not a successful source-availability receipt.
assert!(
matches!(reopened.pin(session, pin, || Ok(source.is_file())), Err(StoreError::SessionSourceMissing(id)) if id == session)
);
{
let _publication = reopened.lock_live(session).unwrap();
fs::write(&source, "committed before notification").unwrap();
}
assert!(
matches!(reopened.delete(session, SessionDeletionIntent::Ordinary, || { fs::remove_file(&source)?; Ok(()) }), Err(StoreError::SessionPinned(id)) if id == session)
);
assert_eq!(
fs::read_to_string(&source).unwrap(),
"committed before notification"
);
reopened
.delete(session, SessionDeletionIntent::ErasePermanently, || {
fs::remove_file(&source)?;
Ok(())
})
.unwrap();
assert!(
matches!(retention.retain_before_publication(session, pin), Err(StoreError::SessionErased(id)) if id == session)
);
}
#[test]
fn pin_survives_reopen_blocks_delete_and_releases_idempotently() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let store = SessionRetentionStore::new(temp.path());
store.pin(id, pin, || Ok(true)).unwrap();
store.pin(id, pin, || Ok(true)).unwrap();
let reopened = SessionRetentionStore::new(temp.path());
assert_eq!(reopened.pins(id).unwrap(), [pin]);
assert!(matches!(
reopened.delete(id, SessionDeletionIntent::Ordinary, || Ok(())),
Err(StoreError::SessionPinned(_))
));
reopened.release(id, pin).unwrap();
reopened.release(id, pin).unwrap();
assert_eq!(
reopened
.delete(id, SessionDeletionIntent::Ordinary, || Ok(7))
.unwrap(),
7
);
}
#[test]
fn erase_failure_persists_fence_invalidates_pin_and_rejects_late_publication() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let store = SessionRetentionStore::new(temp.path());
store.pin(id, pin, || Ok(true)).unwrap();
assert!(
store
.delete(
id,
SessionDeletionIntent::ErasePermanently,
|| Err::<(), _>(std::io::Error::other("injected removal failure").into())
)
.is_err()
);
let reopened = SessionRetentionStore::new(temp.path());
assert!(reopened.pins(id).unwrap().is_empty());
assert!(matches!(
reopened.pin(id, pin, || Ok(true)),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.lock_live(id),
Err(StoreError::SessionErased(_))
));
reopened
.delete(id, SessionDeletionIntent::ErasePermanently, || Ok(()))
.unwrap();
reopened.release(id, pin).unwrap();
assert!(matches!(
reopened.lock_live(id),
Err(StoreError::SessionErased(_))
));
}
#[test]
fn fs_normal_delete_obeys_pin_and_permanent_erase_fences_all_source_writes() {
use crate::{FsStore, LogEntry, Store};
let temp = tempfile::tempdir().unwrap();
let store = FsStore::new(temp.path()).unwrap();
let session = crate::new_session_id();
let segment = crate::new_segment_id();
let entry = LogEntry::TurnEnd {
ts: 1,
turn_count: 1,
};
store
.create_segment(session, segment, &[entry.clone()])
.unwrap();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
store.pin_session(session, pin).unwrap();
assert!(matches!(
store.delete_session(session),
Err(StoreError::SessionPinned(_))
));
assert!(store.exists(session, segment).unwrap());
store
.delete_session_with_intent(session, SessionDeletionIntent::ErasePermanently)
.unwrap();
let reopened = FsStore::new(temp.path()).unwrap();
assert!(!reopened.exists(session, segment).unwrap());
assert!(matches!(
reopened.pin_session(session, pin),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.append(session, segment, &entry),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.create_segment(session, segment, &[entry]),
Err(StoreError::SessionErased(_))
));
assert!(
!reopened
.delete_session_with_intent(session, SessionDeletionIntent::ErasePermanently)
.unwrap()
);
assert!(!reopened.exists(session, segment).unwrap());
}
#[test]
fn independent_handles_serialize_delete_with_late_pin() {
let temp = tempfile::tempdir().unwrap();
let source = temp.path().join("source");
fs::create_dir(&source).unwrap();
let control = temp.path().join("control");
let session = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let (entered_tx, entered_rx) = std::sync::mpsc::channel();
let (release_tx, release_rx) = std::sync::mpsc::channel();
let control2 = control.clone();
let source2 = source.clone();
let delete = std::thread::spawn(move || {
SessionRetentionStore::new(control2).delete(
session,
SessionDeletionIntent::Ordinary,
|| {
entered_tx.send(()).unwrap();
release_rx
.recv_timeout(std::time::Duration::from_secs(10))
.unwrap();
fs::remove_dir(&source2)?;
Ok(())
},
)
});
entered_rx
.recv_timeout(std::time::Duration::from_secs(10))
.unwrap();
let late = std::thread::spawn(move || {
SessionRetentionStore::new(control).pin(session, pin, || Ok(source.is_dir()))
});
release_tx.send(()).unwrap();
delete.join().unwrap().unwrap();
assert!(matches!(
late.join().unwrap(),
Err(StoreError::SessionSourceMissing(_))
));
}
#[test]
fn missing_source_cannot_acquire_pin() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let store = SessionRetentionStore::new(temp.path());
assert!(matches!(
store.pin(id, SessionRetentionPin(uuid::Uuid::now_v7()), || Ok(false)),
Err(StoreError::SessionSourceMissing(_))
));
assert!(store.pins(id).unwrap().is_empty());
}
}
+1
View File
@@ -329,6 +329,7 @@ pub fn save_run_completed(
session_id,
segment_id,
LogEntry::RunCompleted {
entry_id: None,
ts: segment_log::now_millis(),
interrupted,
result,
+43 -6
View File
@@ -122,6 +122,8 @@ pub enum LogEntry {
/// Replay restores both interruption state and any resumable logical-run
/// turn budget.
RunCompleted {
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
ts: u64,
interrupted: bool,
result: EngineResult,
@@ -194,7 +196,11 @@ pub enum LogEntry {
/// A paused interrupted turn was explicitly abandoned without calling
/// `run()` or `resume()` again. Replay clears the interrupted marker so
/// the restored Worker is idle and future user input starts a normal new turn.
PausedTurnAbandoned { ts: u64 },
PausedTurnAbandoned {
ts: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
},
/// `RequestConfig` changed.
ConfigChanged { ts: u64, config: RequestConfig },
@@ -248,12 +254,31 @@ pub struct SegmentOrigin {
}
impl LogEntry {
/// Validate generic System records before a Store publishes any bytes.
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
match self {
Self::AnnotatedSegmentStart { history, .. }
| Self::AnnotatedUserInput { history, .. } => {
for entry in history {
entry.validate_input_metadata()?;
}
}
Self::AnnotatedAssistantItem { entry, .. }
| Self::AnnotatedToolResult { entry, .. }
| Self::ToolResultCorrected { entry, .. } => entry.validate_input_metadata()?,
Self::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
_ => {}
}
Ok(())
}
/// Allocate the durable public identity for a newly committed Run
/// transition. Calling this repeatedly is idempotent so the exact value
/// written to storage is also the value published to live subscribers.
pub fn ensure_session_entry_id(&mut self) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
Self::RunCompleted { entry_id, .. }
| Self::PausedTurnAbandoned { entry_id, .. }
| Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
@@ -275,7 +300,9 @@ impl LogEntry {
record_start: u64,
) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
Self::RunCompleted { entry_id, .. }
| Self::PausedTurnAbandoned { entry_id, .. }
| Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
@@ -527,6 +554,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -586,6 +614,7 @@ mod tests {
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 3200,
interrupted: false,
result: EngineResult::Finished,
@@ -893,12 +922,16 @@ mod tests {
compacted_from: None,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 100,
interrupted: true,
result: EngineResult::Paused,
active_run_turn_count: Some(1),
},
LogEntry::PausedTurnAbandoned { ts: 200 },
LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 200,
},
]);
assert!(!state.last_run_interrupted);
assert_eq!(state.active_run_turn_count, None);
@@ -1056,6 +1089,7 @@ mod tests {
turn_count: 2,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 300,
interrupted: true,
result: EngineResult::LimitReached,
@@ -1069,13 +1103,16 @@ mod tests {
#[test]
fn paused_turn_abandoned_entry_round_trip_via_json() {
let entry = LogEntry::PausedTurnAbandoned { ts: 12345 };
let entry = LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 12345,
};
let json = serde_json::to_string(&entry).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["kind"], "paused_turn_abandoned");
let decoded: LogEntry = serde_json::from_str(&json).unwrap();
match decoded {
LogEntry::PausedTurnAbandoned { ts } => assert_eq!(ts, 12345),
LogEntry::PausedTurnAbandoned { ts, .. } => assert_eq!(ts, 12345),
other => panic!("expected PausedTurnAbandoned, got {other:?}"),
}
}
+26
View File
@@ -27,12 +27,27 @@ pub enum StoreError {
#[error("serialization error: {0}")]
Serde(#[from] serde_json::Error),
#[error("invalid history input metadata: {0}")]
InvalidInputMetadata(#[from] crate::InputMetadataError),
#[error("segment not found: {0}")]
NotFound(SegmentId),
#[error("log corrupted at line {line}: {message}")]
Corrupt { line: usize, message: String },
#[error("session source is missing: {0}")]
SessionSourceMissing(SessionId),
#[error("session original material is retained by a durable consumer: {0}")]
SessionPinned(SessionId),
#[error("session was permanently erased: {0}")]
SessionErased(SessionId),
#[error("session source retention is unsupported by this store")]
SessionRetentionUnsupported,
#[error("retained observation read limit exceeded")]
ReadLimitExceeded,
@@ -80,6 +95,17 @@ pub enum StoreError {
/// take `(SessionId, SegmentId)` so segments can be physically grouped
/// per Session on disk (or per session_id in a DB).
pub trait Store: Send + Sync {
/// Establish durable retention intent before the Host accepts public input.
/// This does not attest source availability; it must use the same control store
/// as publishers and physical deletion. Unsupported stores fail closed.
fn retain_session_before_publication(
&self,
_session_id: SessionId,
_pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
Err(StoreError::SessionRetentionUnsupported)
}
/// Append a single log entry to the segment log.
///
/// One committed line per successful call. Implementations must not expose
+92 -93
View File
@@ -86,12 +86,16 @@ pub struct PromptRenderProvenance {
/// `notify_wrapper` template) re-shape the live rendering on a later
/// resume.
///
/// New variants get added here as fresh injection kinds come online
/// (e.g. `TaskReminder`). The `kind` JSON tag is the snake_case form of
/// the variant name.
/// Feature-owned inputs use `Input` plus explicit annotated metadata; adding a
/// Feature must not extend this enum or ordinary replay. Only core events have
/// dedicated variants. Pre-v5 Feature kinds exist exclusively in the migration
/// decoder, not in this current wire contract.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum SystemItem {
/// Generic System body; private metadata and public presentation are supplied
/// independently by the producer in the same annotated history record.
Input { body: String },
/// Free-form notification sent in by an external caller via
/// `Method::Notify`. `message` is the raw caller-supplied text;
/// `body` is the wrapped LLM-context form (Worker renders it via
@@ -114,14 +118,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Result of an explicitly selected chat Feature invocation. The typed
/// result is retained for clients; `body` is the exact bounded context made
/// visible to the Worker model.
FeatureInvocationResult {
result: protocol::FeatureInvocationResult,
body: String,
},
/// `@<path>` file reference resolution. `body` is the rendered
/// LLM-context text (`[File: <path>]\n…` for regular files,
/// `[Dir: <path>]\n…` for directory listings, possibly with a
@@ -129,10 +125,6 @@ pub enum SystemItem {
/// byte-identical to what was sent.
FileAttachment { path: String, body: String },
/// Explicit Agent Skill activation. `body` is the exact LLM-facing
/// Skill text committed before any subsequent LLM request can use it.
SkillActivation { name: String, body: String },
/// Historical persisted Knowledge reference resolution. Knowledge is no
/// longer active, so restored sessions ignore this item instead of replaying
/// archived record text into model context.
@@ -156,29 +148,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Restore-boundary refresh of the current resident Memory projection.
/// The exact rendered body is retained so replay is byte-identical; a
/// refresh with no ready summary explicitly supersedes older resident
/// context without deleting or rewriting history.
ResidentSummaryRefresh {
body: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// A Host-fetched change to user-managed Subject behavior. Each changed
/// body is appended rather than rewriting prior prompt/session content.
SubjectBehaviorRefresh {
body: String,
/// Exact user-managed behavior acknowledged after durable commit. Old
/// logs omit this value; an empty value is not an acknowledgement of a
/// nonempty behavior and does not replace their stored rendered body.
#[serde(default)]
behavior_md: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Synthetic note inserted after an interrupted turn before the next
/// user input. `body` is the exact LLM-context text explaining that the
/// previous turn was cut short.
@@ -189,23 +158,67 @@ pub enum SystemItem {
},
}
/// Allowlisted presentation data shared by live events and retained snapshots.
/// Private Feature payloads and render provenance never cross this boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct PublicSystemItem {
pub kind: String,
pub body: String,
pub presentation: PublicSystemPresentation,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PublicSystemPresentation {
Notification { message: String },
WorkerEvent { event: WorkerEvent },
Context,
TaskReminder,
Ignored,
}
impl SystemItem {
pub fn public_item(&self) -> PublicSystemItem {
let presentation = match self {
Self::Notification { message, .. } => PublicSystemPresentation::Notification {
message: message.clone(),
},
Self::WorkerEvent { event, .. } => PublicSystemPresentation::WorkerEvent {
event: event.clone(),
},
Self::TaskReminder { .. } => PublicSystemPresentation::TaskReminder,
Self::Input { .. }
| Self::LegacyKnowledgeIgnored { .. }
| Self::LegacyIgnored { .. } => PublicSystemPresentation::Ignored,
Self::FileAttachment { .. } | Self::Interrupt { .. } => {
PublicSystemPresentation::Context
}
};
PublicSystemItem {
kind: self.kind_label().to_owned(),
body: if matches!(presentation, PublicSystemPresentation::Ignored) {
String::new()
} else {
self.history_text()
},
presentation,
}
}
/// Free-text body the LLM sees inside its `role:system` message
/// for this item. Returns the variant's stored `body` verbatim.
pub fn history_text(&self) -> String {
match self {
SystemItem::Input { body } => body.clone(),
SystemItem::Notification { body, .. } => body.clone(),
SystemItem::WorkerEvent { body, .. } => body.clone(),
SystemItem::FeatureInvocationResult { body, .. } => body.clone(),
SystemItem::FileAttachment { body, .. } => body.clone(),
SystemItem::SkillActivation { body, .. } => body.clone(),
SystemItem::LegacyKnowledgeIgnored { .. } => String::new(),
SystemItem::LegacyIgnored { slug } => {
format!("Ignored legacy procedure item: /{slug}")
}
SystemItem::TaskReminder { body, .. } => body.clone(),
SystemItem::ResidentSummaryRefresh { body, .. } => body.clone(),
SystemItem::SubjectBehaviorRefresh { body, .. } => body.clone(),
SystemItem::Interrupt { body, .. } => body.clone(),
}
}
@@ -220,16 +233,13 @@ impl SystemItem {
/// wire — keep flexible.
pub fn kind_label(&self) -> &'static str {
match self {
SystemItem::Input { .. } => "input",
SystemItem::Notification { .. } => "notification",
SystemItem::WorkerEvent { .. } => "worker_event",
SystemItem::FeatureInvocationResult { .. } => "feature_invocation_result",
SystemItem::FileAttachment { .. } => "file_attachment",
SystemItem::SkillActivation { .. } => "skill_activation",
SystemItem::LegacyKnowledgeIgnored { .. } => "legacy_knowledge_ignored",
SystemItem::LegacyIgnored { .. } => "legacy_ignored",
SystemItem::TaskReminder { .. } => "task_reminder",
SystemItem::ResidentSummaryRefresh { .. } => "resident_summary_refresh",
SystemItem::SubjectBehaviorRefresh { .. } => "subject_behavior_refresh",
SystemItem::Interrupt { .. } => "interrupt",
}
}
@@ -287,33 +297,45 @@ mod tests {
}
#[test]
fn subject_behavior_refresh_round_trips_with_exact_body() {
let item = SystemItem::SubjectBehaviorRefresh {
body: "current Subject context".to_string(),
behavior_md: "exact behavior".to_string(),
prompt_provenance: None,
};
let raw = serde_json::to_string(&item).unwrap();
let parsed: SystemItem = serde_json::from_str(&raw).unwrap();
assert_eq!(parsed.history_text(), "current Subject context");
assert!(
matches!(parsed, SystemItem::SubjectBehaviorRefresh { behavior_md, .. } if behavior_md == "exact behavior")
);
fn current_system_item_rejects_frozen_feature_kinds() {
// Even well-formed old payloads cannot enter normal v5 decode/replay.
for line in include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl")
.lines()
.take(5)
{
let record: serde_json::Value = serde_json::from_str(line).unwrap();
assert!(serde_json::from_value::<SystemItem>(record["entry"]["item"].clone()).is_err());
assert!(
serde_json::from_value::<crate::LoggedSystemHistoryEntry>(record["entry"].clone())
.is_err()
);
assert!(serde_json::from_str::<crate::LogEntry>(line).is_err());
}
}
#[test]
fn frozen_subject_behavior_log_reads_old_counter_without_republishing_it() {
let old = r#"{"kind":"subject_behavior_refresh","body":"exact archived body","behavior_revision":7}"#;
let item: SystemItem = serde_json::from_str(old).unwrap();
assert_eq!(item.history_text(), "exact archived body");
assert_eq!(
serde_json::to_value(item).unwrap(),
serde_json::json!({
"kind": "subject_behavior_refresh",
"body": "exact archived body",
"behavior_md": ""
})
);
fn public_system_input_decoder_rejects_private_fields_and_unknown_presentations() {
let valid = serde_json::json!({
"kind": "new-feature-label",
"body": "explicit public content",
"presentation": { "kind": "context" },
});
let public: PublicSystemItem = serde_json::from_value(valid.clone()).unwrap();
assert_eq!(public.body, "explicit public content");
for private_field in [
"payload",
"prompt_provenance",
"policy",
"snapshot",
"behavior_md",
] {
let mut malformed = valid.clone();
malformed[private_field] = serde_json::json!({ "private": true });
assert!(serde_json::from_value::<PublicSystemItem>(malformed).is_err());
}
let mut unknown = valid;
unknown["presentation"] = serde_json::json!({ "kind": "execute_feature_payload" });
assert!(serde_json::from_value::<PublicSystemItem>(unknown).is_err());
}
#[test]
@@ -409,29 +431,6 @@ mod tests {
}
}
#[test]
fn invocation_result_retains_typed_outcome_and_exact_context() {
let result = protocol::FeatureInvocationResult {
invocation_id: "stable-invoke".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
status: protocol::FeatureInvocationStatus::OutcomeUnknown,
message: "operation may have committed".into(),
context: None,
};
let body = "Outcome unknown; do not retry automatically";
let item = SystemItem::FeatureInvocationResult {
result: result.clone(),
body: body.into(),
};
let restored: SystemItem =
serde_json::from_str(&serde_json::to_string(&item).unwrap()).unwrap();
assert_eq!(restored.history_text(), body);
assert_eq!(restored.kind_label(), "feature_invocation_result");
assert!(
matches!(restored, SystemItem::FeatureInvocationResult { result: saved, .. } if saved == result)
);
}
#[test]
fn round_trip_worker_event() {
let item = SystemItem::WorkerEvent {
+30 -43
View File
@@ -210,18 +210,6 @@ impl WorkerMetadata {
}
}
fn replace_active_session(metadata: &mut WorkerMetadata, active: Option<WorkerActiveSegmentRef>) {
if let Some(session_id) = active.as_ref().map(|active| active.session_id)
&& metadata
.subjektiv_session_attribution
.as_ref()
.is_some_and(|state| state.session_id() != session_id)
{
metadata.subjektiv_session_attribution = None;
}
metadata.active = active;
}
/// Sync persistence backend for Worker metadata.
pub trait WorkerMetadataStore: Send + Sync {
/// Create or replace metadata for its `worker_name` key.
@@ -297,7 +285,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -317,7 +305,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_id: Option<String>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -334,7 +322,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_root) = workspace_root {
metadata.workspace_root = Some(workspace_root);
@@ -342,18 +330,6 @@ pub trait WorkerMetadataStore: Send + Sync {
})
}
/// Set the Host-owned Subjektiv Session attribution lifecycle state while
/// preserving the active Session pointer and every unrelated field.
fn set_subjektiv_session_attribution(
&self,
worker_name: &str,
state: SubjektivSessionAttributionState,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.subjektiv_session_attribution = Some(state);
})
}
/// Set spawned-child registry state while preserving active pointer and manifest snapshot.
fn set_spawned_children(
&self,
@@ -806,6 +782,15 @@ where
S: crate::Store,
W: Send + Sync,
{
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), crate::StoreError> {
self.session_store
.retain_session_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
@@ -1078,13 +1063,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
});
})
.unwrap();
store
.set_active(
@@ -1108,7 +1093,7 @@ mod tests {
}
#[test]
fn new_active_session_clears_prior_session_attribution_state() {
fn generic_active_update_does_not_interpret_host_attribution() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let old_session_id = crate::new_session_id();
@@ -1120,13 +1105,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
});
})
.unwrap();
store
@@ -1142,7 +1127,9 @@ mod tests {
restored.active,
Some(WorkerActiveSegmentRef::pending_segment(new_session_id))
);
assert!(restored.subjektiv_session_attribution.is_none());
assert!(
matches!(restored.subjektiv_session_attribution, Some(SubjektivSessionAttributionState::Confirmed { session_id, .. }) if session_id == old_session_id)
);
}
#[test]
+529 -45
View File
@@ -36,7 +36,10 @@ use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::SystemTime;
const SESSION_SCHEMA_VERSION: u32 = 3;
const SESSION_SCHEMA_VERSION: u32 = 6;
const SYSTEM_INPUT_SESSION_SCHEMA_VERSION: u32 = 5;
const COMPACTION_SESSION_SCHEMA_VERSION: u32 = 4;
const ANNOTATED_SESSION_SCHEMA_VERSION: u32 = 3;
const PREVIOUS_SESSION_SCHEMA_VERSION: u32 = 2;
const LEGACY_SESSION_SCHEMA_VERSION: u32 = 1;
const SESSION_FILE: &str = "session.json";
@@ -48,6 +51,7 @@ pub struct WorkerSessionStore {
root: PathBuf,
session_id: Arc<Mutex<Option<SessionId>>>,
append_lock: Arc<Mutex<()>>,
retention: Option<crate::retention::SessionRetentionStore>,
}
#[derive(Debug, Serialize, Deserialize)]
@@ -72,6 +76,9 @@ pub(crate) struct RetainedSegmentReader {
read_end: u64,
buffer_start: u64,
buffer: Vec<u8>,
// Newest-to-oldest fragments of a record spanning multiple read chunks.
// Keep them separate until its boundary is found to avoid repeated prepend copies.
record_suffix: Vec<Vec<u8>>,
}
impl RetainedSegmentReader {
@@ -82,13 +89,19 @@ impl RetainedSegmentReader {
let mut read_bytes = 0_u64;
loop {
if !self.buffer.is_empty() {
if self.buffer.last() != Some(&b'\n') {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment cursor is not at a record boundary".to_string(),
});
}
let prior_newline = self.buffer[..self.buffer.len() - 1]
let search_end = if self.record_suffix.is_empty() {
if self.buffer.last() != Some(&b'\n') {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment cursor is not at a record boundary"
.to_string(),
});
}
self.buffer.len() - 1
} else {
self.buffer.len()
};
let prior_newline = self.buffer[..search_end]
.iter()
.rposition(|byte| *byte == b'\n');
if let Some(newline) =
@@ -100,9 +113,13 @@ impl RetainedSegmentReader {
newline + 1
};
let record_start = self.buffer_start + record_start_in_buffer as u64;
let record_end = self.buffer_start + self.buffer.len() as u64;
let record =
self.buffer[record_start_in_buffer..self.buffer.len() - 1].to_vec();
let record_end = self.read_end;
let mut record = Vec::with_capacity((record_end - record_start) as usize);
record.extend_from_slice(&self.buffer[record_start_in_buffer..]);
for fragment in self.record_suffix.drain(..).rev() {
record.extend_from_slice(&fragment);
}
record.pop(); // The final newline was checked before collecting fragments.
self.buffer.truncate(record_start_in_buffer);
self.read_end = record_start;
if record.is_empty() {
@@ -126,24 +143,17 @@ impl RetainedSegmentReader {
read_bytes,
));
}
// This chunk has no earlier boundary. Never scan or copy it again
// until the complete record can be assembled once.
self.record_suffix.push(std::mem::take(&mut self.buffer));
} else if self.read_end == 0 {
return Ok((None, read_bytes));
}
if self.buffer_start == 0 && !self.buffer.is_empty() {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment log does not end at a record boundary".to_string(),
});
}
if read_bytes >= max_additional_bytes {
return Err(StoreError::ReadLimitExceeded);
}
let end = if self.buffer.is_empty() {
self.read_end
} else {
self.buffer_start
};
let end = self.buffer_start;
let available = max_additional_bytes - read_bytes;
let chunk_len = end
.min(RETAINED_BACKWARD_READ_CHUNK_BYTES as u64)
@@ -156,7 +166,6 @@ impl RetainedSegmentReader {
let mut chunk = vec![0_u8; chunk_len as usize];
self.file.read_exact(&mut chunk)?;
read_bytes += chunk_len;
chunk.extend_from_slice(&self.buffer);
self.buffer = chunk;
self.buffer_start = start;
}
@@ -164,9 +173,48 @@ impl RetainedSegmentReader {
}
impl WorkerSessionStore {
/// Publishes through the same external control store used by Runtime retention.
/// Existing manifest migration is fenced too; future writes keep the guard until their
/// physical publication finishes. The control root must outlive this aggregate.
pub fn new_with_retention(
root: impl Into<PathBuf>,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, StoreError> {
let root = root.into();
let existing = match fs::read(root.join(SESSION_FILE)) {
Ok(bytes) => Some(serde_json::from_slice::<SessionManifest>(&bytes)?.session_id),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => return Err(error.into()),
};
let _guard = existing
.map(|session| retention.lock_live(session))
.transpose()?;
let mut store = Self::new(root)?;
store.retention = Some(retention);
Ok(store)
}
fn lock_publication(
&self,
session: SessionId,
) -> Result<Option<crate::retention::SessionRetentionGuard>, StoreError> {
self.retention
.as_ref()
.map(|store| store.lock_live(session))
.transpose()
}
/// Open the Session store rooted at `<worker-aggregate>/session`.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
Self::new_with_migration_writer(root.into(), atomic_write_bytes)
}
// Keep fault injection at the storage publication boundary, without global
// flags or a public alternate restore API. Production always uses atomic writes.
fn new_with_migration_writer(
root: PathBuf,
write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<Self, StoreError> {
fs::create_dir_all(root.join(SEGMENTS_DIR))?;
let session_id = match fs::read(root.join(SESSION_FILE)) {
Ok(bytes) => {
@@ -175,11 +223,16 @@ impl WorkerSessionStore {
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_logs(&root)?;
}
PREVIOUS_SESSION_SCHEMA_VERSION | LEGACY_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_v3(
&root,
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths(&root)?,
manifest.session_id,
manifest.schema_version,
write_segment,
)?;
manifest.schema_version = SESSION_SCHEMA_VERSION;
atomic_write_json(&root.join(SESSION_FILE), &manifest)?;
@@ -202,9 +255,37 @@ impl WorkerSessionStore {
root,
session_id: Arc::new(Mutex::new(session_id)),
append_lock: Arc::new(Mutex::new(())),
retention: None,
})
}
/// Validate an existing Session's conversion without publishing any bytes.
/// Hosts can isolate unreadable archives before admitting a staged migration;
/// the versioned decoder and conversion checks remain owned by this store.
pub fn validate_migration(root: &Path) -> Result<(), StoreError> {
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(root.join(SESSION_FILE))?)?;
match manifest.schema_version {
SESSION_SCHEMA_VERSION => validate_canonical_segment_logs(root),
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => migrate_segment_logs_to_current(
segment_log_paths(root)?,
manifest.session_id,
manifest.schema_version,
|_, _| Ok(()),
),
version => Err(StoreError::Corrupt {
line: 0,
message: format!(
"unsupported Worker Session schema version {version}, expected {SESSION_SCHEMA_VERSION}"
),
}),
}
}
/// Open an already-retained Session without creating directories or migrating
/// persisted data. Observation paths must never mutate retained state.
pub fn open_read_only(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
@@ -224,6 +305,7 @@ impl WorkerSessionStore {
root,
session_id: Arc::new(Mutex::new(Some(manifest.session_id))),
append_lock: Arc::new(Mutex::new(())),
retention: None,
})
}
@@ -456,6 +538,7 @@ impl WorkerSessionStore {
read_end: before_offset,
buffer_start: before_offset,
buffer: Vec::new(),
record_suffix: Vec::new(),
})
}
@@ -682,12 +765,25 @@ impl WorkerSessionStore {
}
impl Store for WorkerSessionStore {
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention
.as_ref()
.ok_or(StoreError::SessionRetentionUnsupported)?
.retain_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_log_entry(&self.log_path(segment_id), entry)
}
@@ -710,6 +806,11 @@ impl Store for WorkerSessionStore {
}
fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError> {
// A pending Worker can have metadata before its atomic initial seed.
// Absence is queryable without materializing or binding this store.
if self.session_id()?.is_none() {
return Ok(Vec::new());
}
self.ensure_session(session_id, false)?;
let mut segments: Vec<SegmentId> = Vec::new();
for entry in fs::read_dir(self.root.join(SEGMENTS_DIR))? {
@@ -740,6 +841,10 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let mut content = Vec::new();
for entry in entries {
@@ -751,6 +856,9 @@ impl Store for WorkerSessionStore {
}
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
if self.session_id()?.is_none() {
return Ok(false);
}
self.ensure_session(session_id, false)?;
Ok(self.log_path(segment_id).exists())
}
@@ -788,6 +896,7 @@ impl Store for WorkerSessionStore {
content: &str,
limits: PasteArtifactLimits,
) -> Result<PasteArtifactRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -818,6 +927,7 @@ impl Store for WorkerSessionStore {
content: &[u8],
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -842,6 +952,7 @@ impl Store for WorkerSessionStore {
context: &UploadedFileUploadContext,
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -881,6 +992,7 @@ impl Store for WorkerSessionStore {
reference: &UploadedFileRef,
source_entry_id: &str,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -909,6 +1021,7 @@ impl Store for WorkerSessionStore {
reference: &UploadedFileRef,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -923,6 +1036,7 @@ impl Store for WorkerSessionStore {
artifact_id: &str,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -937,6 +1051,7 @@ impl Store for WorkerSessionStore {
artifact_id: &str,
source_entry_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -954,6 +1069,7 @@ impl Store for WorkerSessionStore {
session_id: SessionId,
live_owner_ids: &[String],
) -> Result<u64, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -967,6 +1083,7 @@ impl Store for WorkerSessionStore {
session_id: SessionId,
artifact_id: &str,
) -> Result<bool, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -976,6 +1093,7 @@ impl Store for WorkerSessionStore {
}
fn delete_uncommitted_uploaded_files(&self, session_id: SessionId) -> Result<u64, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -1010,6 +1128,7 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_line(&self.trace_path(segment_id), &serde_json::to_string(entry)?)
}
@@ -1035,7 +1154,12 @@ fn open_retained_file(path: &Path) -> std::io::Result<File> {
}
fn segment_log_paths(root: &Path) -> Result<Vec<(SegmentId, PathBuf)>, StoreError> {
let segments = root.join(SEGMENTS_DIR);
segment_log_paths_in_directory(&root.join(SEGMENTS_DIR))
}
fn segment_log_paths_in_directory(
segments: &Path,
) -> Result<Vec<(SegmentId, PathBuf)>, StoreError> {
if !segments.exists() {
return Ok(Vec::new());
}
@@ -1078,21 +1202,23 @@ fn segment_log_paths(root: &Path) -> Result<Vec<(SegmentId, PathBuf)>, StoreErro
Ok(paths)
}
fn migrate_segment_logs_to_v3(
root: &Path,
fn migrate_segment_logs_to_current(
paths: Vec<(SegmentId, PathBuf)>,
session_id: SessionId,
source_schema_version: u32,
mut write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<(), StoreError> {
struct MigrationPlan {
path: PathBuf,
source: Vec<u8>,
entries: Vec<LogEntry>,
output: Vec<u8>,
}
// Phase 1 is strictly read-only. Every segment must parse and canonicalize
// successfully before the first authoritative byte is replaced.
let mut plans = Vec::new();
for (segment_id, path) in segment_log_paths(root)? {
for (segment_id, path) in paths {
let source = fs::read(&path)?;
let canonical = parse_legacy_jsonl(source_schema_version, session_id, segment_id, &source)
.map_err(|error| StoreError::Corrupt {
@@ -1102,17 +1228,32 @@ fn migrate_segment_logs_to_v3(
path.display()
),
})?;
let mut output = Vec::new();
for entry in canonical {
serde_json::to_writer(&mut output, &entry)?;
output.push(b'\n');
}
plans.push(MigrationPlan {
path,
source,
output,
entries: canonical,
output: Vec::new(),
});
}
// Resolve trusted creation boundaries first, then update inherited copies by
// logical identity too (fork/rewind seeds retain the same entry id).
let summaries = if source_schema_version < COMPACTION_SESSION_SCHEMA_VERSION {
crate::legacy_session_log::compaction_summary_ids(
plans.iter().flat_map(|plan| plan.entries.iter()),
)
} else {
// v4 already made the explicit origin migration. Preserve its origins.
Default::default()
};
for plan in &mut plans {
crate::legacy_session_log::migrate_compaction_origins(&mut plan.entries, &summaries);
for mut entry in std::mem::take(&mut plan.entries) {
crate::legacy_session_log::migrate_system_inputs(&mut entry)?;
entry.validate_input_metadata()?;
serde_json::to_writer(&mut plan.output, &entry)?;
plan.output.push(b'\n');
}
}
// Fence the complete preflight snapshot before starting phase 2. Session
// open is the exclusive restore boundary; this additionally fails closed
@@ -1130,14 +1271,56 @@ fn migrate_segment_logs_to_v3(
}
for plan in plans {
atomic_write_bytes(&plan.path, &plan.output)?;
write_segment(&plan.path, &plan.output)?;
}
Ok(())
}
/// Upgrade the standalone per-Session layout using the same persisted format
/// migration as Worker aggregates. Called only at writable Host store startup.
pub(crate) fn migrate_fs_session(root: &Path, session_id: SessionId) -> Result<(), StoreError> {
let marker = root.join("session-schema.json");
let version: u32 = match fs::read(&marker) {
Ok(bytes) => serde_json::from_slice(&bytes)?,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
ANNOTATED_SESSION_SCHEMA_VERSION
}
Err(error) => return Err(error.into()),
};
match version {
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_paths(segment_log_paths_in_directory(root)?)
}
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths_in_directory(root)?,
session_id,
version,
atomic_write_bytes,
)?;
atomic_write_json(&marker, &SESSION_SCHEMA_VERSION)
}
_ => Err(StoreError::Corrupt {
line: 0,
message: format!("unsupported standalone Session schema {version}"),
}),
}
}
fn validate_canonical_segment_logs(root: &Path) -> Result<(), StoreError> {
for (_, path) in segment_log_paths(root)? {
let _: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
validate_canonical_segment_paths(segment_log_paths(root)?)
}
fn validate_canonical_segment_paths(paths: Vec<(SegmentId, PathBuf)>) -> Result<(), StoreError> {
for (_, path) in paths {
let entries: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
for entry in entries {
entry.validate_input_metadata()?;
}
}
Ok(())
}
@@ -1151,17 +1334,26 @@ fn parse_legacy_jsonl(
let text = std::str::from_utf8(bytes).map_err(|error| {
serde_json::Error::io(std::io::Error::new(std::io::ErrorKind::InvalidData, error))
})?;
text.lines()
let mut offset = 0_u64;
text.split_inclusive('\n')
.enumerate()
.filter(|(_, line)| !line.trim().is_empty())
.map(|(line_index, line)| {
crate::legacy_session_log::decode_entry(
.filter_map(|(line_index, line)| {
let start = offset;
offset += line.len() as u64;
(!line.trim().is_empty()).then_some((line_index, line, start))
})
.map(|(line_index, line, start)| {
let mut entry = crate::legacy_session_log::decode_entry(
schema_version,
line,
session_id,
segment_id,
line_index,
)
)?;
// Byte offsets change during migration. Persist identities previously
// synthesized from the OLD offsets before rewriting any record.
entry.ensure_legacy_session_entry_id(session_id, segment_id, start);
Ok(entry)
})
.collect()
}
@@ -1308,6 +1500,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -1315,6 +1508,139 @@ mod tests {
}
}
fn retained_reader(bytes: &[u8]) -> RetainedSegmentReader {
let mut file = tempfile::tempfile().unwrap();
file.write_all(bytes).unwrap();
RetainedSegmentReader {
file,
session_id: new_session_id(),
segment_id: new_segment_id(),
read_end: bytes.len() as u64,
buffer_start: bytes.len() as u64,
buffer: Vec::new(),
record_suffix: Vec::new(),
}
}
fn checkpoint_line(text: String) -> Vec<u8> {
let mut bytes = serde_json::to_vec(&LogEntry::InputSegmentsCheckpoint {
ts: 1,
user_segments: vec![vec![protocol::Segment::Text { content: text }]],
})
.unwrap();
bytes.push(b'\n');
bytes
}
#[test]
fn retained_reader_preserves_records_and_offsets_across_chunk_boundaries() {
let chunk = RETAINED_BACKWARD_READ_CHUNK_BYTES;
let overhead = checkpoint_line(String::new()).len();
for line_size in [chunk - 1, chunk, chunk + 1, chunk * 3] {
let lines = [
checkpoint_line("最初\nの行".into()),
checkpoint_line("x".repeat(line_size - overhead)),
checkpoint_line("最後".into()),
];
let bytes = lines.concat();
let mut reader = retained_reader(&bytes);
let mut end = bytes.len() as u64;
let mut total_read = 0;
for line in lines.iter().rev() {
let (record, read) = reader.previous_record(u64::MAX).unwrap();
total_read += read;
let record = record.unwrap();
assert_eq!(record.end_offset, end);
end -= line.len() as u64;
assert_eq!(record.start_offset, end);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
line[..line.len() - 1]
);
}
assert_eq!(total_read, bytes.len() as u64);
let (record, read) = reader.previous_record(0).unwrap();
assert!(record.is_none());
assert_eq!(read, 0);
}
}
#[test]
fn retained_reader_resumes_after_budget_exhaustion_and_serves_buffered_records() {
let first = checkpoint_line("first".into());
let last = checkpoint_line("大きな行".repeat(RETAINED_BACKWARD_READ_CHUNK_BYTES));
let mut reader = retained_reader(&[first.clone(), last.clone()].concat());
let budget = RETAINED_BACKWARD_READ_CHUNK_BYTES as u64;
assert!(matches!(
reader.previous_record(budget),
Err(StoreError::ReadLimitExceeded)
));
assert!(matches!(
reader.previous_record(0),
Err(StoreError::ReadLimitExceeded)
));
let (record, read) = reader.previous_record(u64::MAX).unwrap();
let record = record.unwrap();
assert_eq!(record.start_offset, first.len() as u64);
assert_eq!(record.end_offset, (first.len() + last.len()) as u64);
assert_eq!(read + budget, record.end_offset);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
last[..last.len() - 1]
);
let (record, read) = reader.previous_record(0).unwrap();
assert_eq!(record.unwrap().start_offset, 0);
assert_eq!(read, 0);
assert!(reader.previous_record(0).unwrap().0.is_none());
}
#[test]
fn retained_reader_requires_the_prior_boundary_byte_at_exact_chunk_edges() {
let chunk = RETAINED_BACKWARD_READ_CHUNK_BYTES;
let overhead = checkpoint_line(String::new()).len();
for size in [chunk - 1, chunk, chunk + 1] {
let first = checkpoint_line("first".into());
let last = checkpoint_line("x".repeat(size - overhead));
let bytes = [first.clone(), last.clone()].concat();
let mut reader = retained_reader(&bytes);
// The final record alone is insufficient: its preceding newline
// must also be read before its start can be established.
assert!(matches!(
reader.previous_record(size as u64),
Err(StoreError::ReadLimitExceeded)
));
let (record, read) = reader.previous_record(1).unwrap();
let record = record.unwrap();
assert_eq!(read, 1);
assert_eq!(record.start_offset, first.len() as u64);
assert_eq!(record.end_offset, bytes.len() as u64);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
last[..last.len() - 1]
);
let (record, read) = reader.previous_record(u64::MAX).unwrap();
assert_eq!(record.unwrap().start_offset, 0);
assert_eq!(read, (first.len() - 1) as u64);
}
}
#[test]
fn retained_reader_rejects_empty_malformed_and_unterminated_records() {
for bytes in [
b"\n".as_slice(),
b"{bad json}\n",
b"{\"kind\":\"input_segments_checkpoint\"}",
] {
assert!(retained_reader(bytes).previous_record(u64::MAX).is_err());
}
let mut bytes = checkpoint_line("valid".into());
bytes.push(b'\n');
assert!(matches!(
retained_reader(&bytes).previous_record(u64::MAX),
Err(StoreError::Corrupt { .. })
));
}
#[test]
fn canonical_combined_store_supports_attachment_uploads() {
let root = tempfile::tempdir().unwrap();
@@ -1762,6 +2088,146 @@ mod tests {
);
}
#[test]
fn v5_migration_retries_after_second_file_publication_fails() {
fn identity_and_body(entries: &[LogEntry]) -> Vec<serde_json::Value> {
entries
.iter()
.flat_map(|record| match record {
LogEntry::AnnotatedSystemItem { entry, .. } => vec![entry.to_history_entry()],
LogEntry::AnnotatedUserInput { history, .. } => history.clone(),
_ => panic!("unexpected record in fixed migration fixture"),
})
.map(|entry| {
serde_json::json!({
"entry_id": entry.metadata.entry_id,
"origin": entry.metadata.origin,
"derivation": entry.metadata.derivation,
"item": entry.item,
})
})
.collect()
}
const SOURCE: &str =
include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl");
let root = tempfile::tempdir().unwrap();
let session_id = new_session_id();
let first_id = SegmentId::from_u128(1);
let second_id = SegmentId::from_u128(2);
fs::create_dir_all(root.path().join(SEGMENTS_DIR)).unwrap();
let marker_path = root.path().join(SESSION_FILE);
atomic_write_json(
&marker_path,
&SessionManifest {
schema_version: COMPACTION_SESSION_SCHEMA_VERSION,
session_id,
},
)
.unwrap();
let old_marker = fs::read(&marker_path).unwrap();
let first_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{first_id}.jsonl"));
let second_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{second_id}.jsonl"));
let second_source = SOURCE.replace("-id", "-second-id");
fs::write(&first_path, SOURCE).unwrap();
fs::write(&second_path, &second_source).unwrap();
let old_first = fs::read(&first_path).unwrap();
let old_second = fs::read(&second_path).unwrap();
let first_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, first_id, &old_first).unwrap());
let second_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, second_id, &old_second).unwrap());
let mut writes = 0;
let mut first_published = None;
let failure = WorkerSessionStore::new_with_migration_writer(
root.path().to_path_buf(),
|path, bytes| {
writes += 1;
match writes {
1 => {
assert_eq!(path, first_path.as_path());
// Both files are still old when the first publication starts.
assert_eq!(fs::read(&first_path).unwrap(), old_first);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
atomic_write_bytes(path, bytes)?;
first_published = Some(bytes.to_vec());
Ok(())
}
2 => {
assert_eq!(path, second_path.as_path());
assert_eq!(
fs::read(&first_path).unwrap(),
*first_published.as_ref().unwrap()
);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
// Fail only this atomic-write boundary, after complete
// preflight and a real successful first-file publication.
Err(std::io::Error::other("injected second migration write failure").into())
}
_ => panic!("publication continued after its write failure"),
}
},
);
let error = match failure {
Ok(_) => panic!("the injected publication failure must reject store open"),
Err(error) => error,
};
assert!(
error
.to_string()
.contains("injected second migration write failure")
);
assert_eq!(writes, 2);
let first_published = first_published.unwrap();
assert_ne!(first_published, old_first);
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
assert_eq!(fs::read(&marker_path).unwrap(), old_marker);
let updated_first: Vec<LogEntry> = parse_jsonl(&first_published).unwrap();
for entry in &updated_first {
entry.validate_input_metadata().unwrap();
}
assert!(matches!(
&updated_first[0],
LogEntry::AnnotatedSystemItem {
entry: crate::LoggedSystemHistoryEntry {
item: crate::SystemItem::Input { .. },
..
},
..
}
));
assert_eq!(identity_and_body(&updated_first), first_identity);
// Retry via the public production constructor, without any injection or
// hand-built mixed fixture. The old marker drives the same migration.
let store = WorkerSessionStore::new(root.path()).unwrap();
let first = store.read_all(session_id, first_id).unwrap();
let second = store.read_all(session_id, second_id).unwrap();
assert_eq!(identity_and_body(&first), first_identity);
assert_eq!(identity_and_body(&second), second_identity);
for entry in first.iter().chain(second.iter()) {
entry.validate_input_metadata().unwrap();
}
assert_eq!(fs::read(&first_path).unwrap(), first_published);
let second_published = fs::read(&second_path).unwrap();
assert_ne!(second_published, old_second);
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(&marker_path).unwrap()).unwrap();
assert_eq!(manifest.schema_version, SESSION_SCHEMA_VERSION);
drop(store);
WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), second_published);
}
#[test]
fn current_jsonl_requires_annotations_across_append_rewrite_and_reopen() {
let root = tempfile::tempdir().unwrap();
@@ -1906,6 +2372,24 @@ mod tests {
assert!(matches!(error, StoreError::Corrupt { .. }));
}
#[test]
fn pending_session_probes_are_empty_without_binding_and_foreign_materialized_session_is_rejected()
{
let root = tempfile::tempdir().unwrap();
let session_id = new_session_id();
let segment_id = new_segment_id();
let store = WorkerSessionStore::new(root.path()).unwrap();
assert!(!store.exists(session_id, segment_id).unwrap());
assert!(store.list_segments(session_id).unwrap().is_empty());
assert_eq!(store.session_id().unwrap(), None);
let reopened = WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(reopened.session_id().unwrap(), None);
store.create_segment(session_id, segment_id, &[]).unwrap();
let foreign = new_session_id();
assert!(store.exists(foreign, segment_id).is_err());
assert!(store.list_segments(foreign).is_err());
}
#[test]
fn reopen_preserves_session_and_segment_ids() {
let root = tempfile::tempdir().unwrap();
+179 -2
View File
@@ -11,6 +11,7 @@ fn annotated(item: Item) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -62,6 +63,7 @@ fn round_trip_write_and_read() {
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 3200,
interrupted: false,
result: EngineResult::Finished,
@@ -157,9 +159,10 @@ fn create_segment_replacement_keeps_open_old_log_and_commits_complete_new_log()
assert_eq!(reopened.read_entry_count(sid, segid).unwrap(), new.len());
assert_eq!(
std::fs::read_dir(path.parent().unwrap()).unwrap().count(),
1,
"successful replacement must not leave temporary files"
2,
"successful replacement leaves only the segment and Session schema marker"
);
assert!(path.parent().unwrap().join("session-schema.json").is_file());
}
#[test]
@@ -419,3 +422,177 @@ fn lookup_session_of_finds_owning_session() {
assert_eq!(store.lookup_session_of(segid).unwrap(), Some(sid));
}
// Storage/projection component test: both production layouts, temporary originals,
// no Feature, model, runtime or Host authority is substituted into this boundary.
#[test]
fn generic_system_append_and_fork_preserve_metadata_without_public_payload_leak() {
use session_store::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputObservationUse, LoggedHistoryDerivation, LoggedSystemHistoryEntry,
PublicSystemItem, PublicSystemPresentation, SystemInputEnvelope, WorkerSessionStore,
};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let fork = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
store
.create_segment(sid, segment, &[nil_session_start(1, sid)])
.unwrap();
let entry = LoggedSystemHistoryEntry::from_input(
LoggedSessionHistoryOrigin::Provided {
producer: "new-feature".into(),
},
Some(LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source-entry".into())],
}),
SystemInputEnvelope::new(
"private model body",
HistoryInputMetadata::new(
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
},
Some(
FeatureInputPayload::encode(
"new-feature:context",
101,
&serde_json::json!({"private": "secret-detail"}),
)
.unwrap(),
),
Some(PublicSystemItem {
kind: "context".into(),
body: "explicit public body".into(),
presentation: PublicSystemPresentation::Context,
}),
)
.unwrap(),
)
.unwrap(),
);
store
.append(
sid,
segment,
&LogEntry::AnnotatedSystemItem {
ts: 2,
entry: entry.clone(),
extensions: vec![],
},
)
.unwrap();
let saved = store.read_all(sid, segment).unwrap();
let LogEntry::AnnotatedSystemItem {
entry: restored, ..
} = &saved[1]
else {
panic!("missing system input")
};
assert_eq!(restored.to_history_entry(), entry.to_history_entry());
assert_eq!(
collect_state(&saved).history[0].as_text(),
Some("private model body")
);
let mut fork_seed = nil_session_start(3, sid);
if let LogEntry::AnnotatedSegmentStart {
history,
forked_from,
..
} = &mut fork_seed
{
history.push(restored.to_history_entry());
*forked_from = Some(session_store::SegmentOrigin {
segment_id: segment,
at_turn_index: 0,
});
}
store.create_segment(sid, fork, &[fork_seed]).unwrap();
let fork_saved = store.read_all(sid, fork).unwrap();
let LogEntry::AnnotatedSegmentStart { history, .. } = &fork_saved[0] else {
unreachable!()
};
assert_eq!(history, &[entry.to_history_entry()]);
let original = session_store::public_snapshot::project_session_snapshot(sid, &saved);
let inherited = session_store::public_snapshot::project_session_snapshot(sid, &fork_saved);
let public = serde_json::to_string(&original.entries).unwrap();
assert!(public.contains("explicit public body"));
for private in ["private model body", "secret-detail", "new-feature:context"] {
assert!(!public.contains(private), "private field leaked: {private}");
}
assert_eq!(original.entries.len(), 1);
assert_eq!(inherited.entries.len(), 1);
assert_eq!(original.entries[0].entry_id, inherited.entries[0].entry_id);
assert_eq!(
original.entries[0].derived_from,
inherited.entries[0].derived_from
);
assert_eq!(
original.entries[0].provenance,
inherited.entries[0].provenance
);
assert_eq!(
serde_json::to_value(&original.entries[0].data).unwrap(),
serde_json::to_value(&inherited.entries[0].data).unwrap()
);
}
}
#[test]
fn invalid_generic_system_input_rejects_append_create_and_external_json_without_changing_originals()
{
use session_store::{StoreError, SystemItem, WorkerSessionStore};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
let seed = nil_session_start(1, sid);
store
.create_segment(sid, segment, std::slice::from_ref(&seed))
.unwrap();
let path = if worker_layout {
dir.path().join("segments").join(format!("{segment}.jsonl"))
} else {
dir.path()
.join(sid.to_string())
.join(format!("{segment}.jsonl"))
};
let before = std::fs::read(&path).unwrap();
let invalid = LogEntry::AnnotatedSystemItem {
ts: 2,
extensions: vec![],
entry: session_store::LoggedSystemHistoryEntry {
item: SystemItem::Input {
body: "no policy".into(),
},
metadata: LoggedSessionHistoryMetadata::legacy_unknown(),
},
};
assert!(matches!(
store.append(sid, segment, &invalid),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(matches!(
store.create_segment(sid, segment, &[seed, invalid.clone()]),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(
serde_json::from_value::<LogEntry>(serde_json::to_value(invalid).unwrap()).is_err()
);
assert_eq!(std::fs::read(path).unwrap(), before);
assert_eq!(store.read_all(sid, segment).unwrap().len(), 1);
}
}
@@ -24,6 +24,7 @@ fn annotated(items: &[Item]) -> Vec<session_store::LoggedHistoryEntry> {
.map(|item| session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
+211 -30
View File
@@ -23,9 +23,7 @@ use session_store::{
CombinedStore, FsStore, FsWorkerStore, WorkerActiveSegmentRef, WorkerMetadataStore,
};
use thiserror::Error;
use worker::bootstrap::{
WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout, bash_output_dir_for_worker_id,
};
use worker::bootstrap::{WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout};
use worker::controller::WorkerControllerTransport;
use worker::ipc::protocol_session::{
WorkerProtocolSessionStreams, dispatch_worker_protocol_method, live_log_entry_event,
@@ -57,6 +55,15 @@ pub struct StandaloneHost {
lease: Option<StandaloneWorkerLease>,
jobs: crate::jobs::StandaloneJobs,
subject: Option<crate::subjektiv::SubjectConnection>,
subject_release: Option<worker::SubjectReleaseReservation>,
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum StandaloneSubjectDisconnectError {
#[error("Subject release requires an actual Idle Controller without pending work: {0}")]
Controller(String),
#[error("Subject release remains incomplete; retained binding/lease must be reconciled: {0}")]
Incomplete(String),
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
@@ -89,6 +96,10 @@ pub enum StandaloneStartupError {
FeatureComposition,
#[error("the in-process Worker controller could not start")]
Controller,
#[error(
"mandatory initial context remains unconfirmed; restore retained Worker {worker_id} to retry this Session"
)]
InitialContextPending { worker_id: WorkerId },
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
@@ -171,7 +182,7 @@ impl StandaloneHost {
let jobs = match crate::jobs::StandaloneJobs::open(&jobs_path, launch.cwd.clone()) {
Ok(jobs) => jobs,
Err(_) => {
let _ = store.abandon_allocation(allocation);
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(StandaloneStartupError::StateStore);
}
};
@@ -179,8 +190,11 @@ impl StandaloneHost {
if let Some(client) = &model_client {
jobs.bind_model_client(client.clone_boxed());
}
if let Some(subject) = &subject {
subject.host.bind_jobs(jobs.clone())?;
if let Some(connection) = &subject
&& let Err(error) = connection.host.bind_jobs(jobs.clone())
{
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(error);
}
// WorkerId is the stable identity. The current Worker store remains
// name-keyed, so keep its derived storage key separate from the
@@ -192,7 +206,7 @@ impl StandaloneHost {
let (backing_store, worker_store) = match backing_store(&store, worker_id) {
Ok(stores) => stores,
Err(error) => {
let _ = store.abandon_allocation(allocation);
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(error);
}
};
@@ -200,7 +214,6 @@ impl StandaloneHost {
WorkerFilesystemAuthority::local(launch.cwd.clone(), launch.cwd.clone());
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
bootstrap_manifest,
@@ -208,10 +221,7 @@ impl StandaloneHost {
launch.prompt_catalog,
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerBootstrapLayout::Direct { runtime_base },
WorkerControllerTransport::InProcess,
);
if let Some(subject) = &subject {
@@ -220,12 +230,61 @@ impl StandaloneHost {
if let Some(model_client) = model_client {
bootstrap = bootstrap.with_model_client(model_client);
}
let prepared = match bootstrap.prepare().await {
Ok(prepared) => prepared,
Err(error) => {
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(classify_startup_error(error));
}
};
// Reserve a recoverable identity before mandatory initial-context persistence.
// A failed/unknown seed must be reopened as this Session, never abandoned and
// replaced by a newly fetched context under an invisible ghost binding.
let pending_record = if subject.is_some() {
match store.commit_created_connected(
&allocation,
manifest.clone(),
storage_key.clone(),
prepared.worker().session_id(),
None,
subject.as_ref().map(|s| s.binding.clone()),
) {
Ok(record) => Some(record),
Err(_) => {
if let Some(subject) = &mut subject {
subject.retain();
}
allocation.into_lease().retain();
return Err(StandaloneStartupError::StateStore);
}
}
} else {
None
};
if let Some(subject) = &mut subject {
subject.arm();
}
let started = match bootstrap.start().await {
let started = match prepared.start().await {
Ok(started) => started,
Err(error) => {
if let Some(record) = &pending_record {
let cleanup = jobs.shutdown().await;
let saved = store.mark_stopped(
record,
record.active_session_id,
record.active_segment_id,
StandaloneShutdownReason::StartupFailed,
);
let closed = cleanup.is_ok()
&& saved.is_ok()
&& subject.as_mut().is_some_and(|s| s.close().is_ok());
if closed {
let _ = allocation.into_lease().release();
} else {
allocation.into_lease().retain();
}
return Err(StandaloneStartupError::InitialContextPending { worker_id });
}
let _ = store.abandon_allocation(allocation);
return Err(classify_startup_error(error));
}
@@ -234,22 +293,35 @@ impl StandaloneHost {
Ok(active) => active,
Err(error) => {
stop_started_worker(started).await;
let _ = store.abandon_allocation(allocation);
if pending_record.is_some() {
allocation.into_lease().retain();
} else {
let _ = store.abandon_allocation(allocation);
}
return Err(error);
}
};
let record = match store.commit_created_connected(
&allocation,
manifest,
storage_key,
active.session_id,
active.segment_id,
subject.as_ref().map(|s| s.binding.clone()),
) {
let committed = if let Some(record) = &pending_record {
store.update_active_pointer(record, active.session_id, active.segment_id)
} else {
store.commit_created_connected(
&allocation,
manifest,
storage_key,
active.session_id,
active.segment_id,
None,
)
};
let record = match committed {
Ok(record) => record,
Err(_) => {
stop_started_worker(started).await;
let _ = store.abandon_allocation(allocation);
if pending_record.is_some() {
allocation.into_lease().retain();
} else {
let _ = store.abandon_allocation(allocation);
}
return Err(StandaloneStartupError::StateStore);
}
};
@@ -350,7 +422,6 @@ impl StandaloneHost {
);
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
manifest,
@@ -358,10 +429,7 @@ impl StandaloneHost {
worker::PromptCatalogSource::builtins_only(),
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerBootstrapLayout::Direct { runtime_base },
WorkerControllerTransport::InProcess,
);
if let Some(subject) = &subject {
@@ -427,6 +495,7 @@ impl StandaloneHost {
lease: Some(lease),
jobs,
subject,
subject_release: None,
}
}
@@ -457,6 +526,45 @@ impl StandaloneHost {
Client::new(socket)
}
/// Logical Subject disconnection, not Worker shutdown or completion of its represented work.
/// A successful release keeps the old Controller fenced; a fresh Worker/Session may then
/// acquire the Subject. Failures before durable revocation allow normal recovery and retry.
pub async fn disconnect_subject(&mut self) -> Result<(), StandaloneSubjectDisconnectError> {
let Some(subject) = &mut self.subject else {
return Ok(());
};
let mut reservation = match self.subject_release.take() {
Some(reservation) => reservation,
None => self
.handle
.reserve_subject_release()
.await
.map_err(StandaloneSubjectDisconnectError::Controller)?,
};
let release = match subject.release_for_handoff(&self.record, &mut reservation) {
Ok(()) => match self.jobs.shutdown().await {
Ok(()) => subject.close().map_err(|error| error.to_string()),
Err(error) => Err(error.to_string()),
},
Err(error) => Err(error),
};
match release {
Ok(()) => {
self.subject_release = Some(reservation);
self.subject = None;
Ok(())
}
Err(error) => {
// If revocation committed, keep the actual Controller reserved even if closing
// storage/unlocking failed. Dropping a reservation must not undo revocation.
if subject.writer_released_or_unknown() {
self.subject_release = Some(reservation);
}
Err(StandaloneSubjectDisconnectError::Incomplete(error))
}
}
}
pub fn with_shutdown_timeout(mut self, shutdown_timeout: Duration) -> Self {
self.shutdown_timeout = shutdown_timeout;
self
@@ -619,6 +727,11 @@ async fn run_protocol_session(
event = events.recv() => {
match event {
Ok(event) => {
let Ok(event) = handle.alerter.freshen_event(event) else {
// The retained anchor was evicted; reconnect must obtain
// a paired session/Alert snapshot rather than a stale row.
return;
};
if !send_protocol_event(&peer, event).await {
return;
}
@@ -690,12 +803,32 @@ async fn send_protocol_event(peer: &InProcessPeer, event: Event) -> bool {
peer.send(message).await.is_ok()
}
fn cancel_unpublished_start(
store: &StandaloneWorkerStore,
subject: &mut Option<crate::subjektiv::SubjectConnection>,
allocation: crate::store::StandaloneWorkerAllocation,
) -> Result<(), StandaloneStartupError> {
if let Some(connection) = subject {
if connection.abort_unpublished_start().is_err() {
connection.retain();
allocation.into_lease().retain();
return Err(StandaloneStartupError::StateStore);
}
}
store
.abandon_allocation(allocation)
.map_err(classify_store_startup_error)
}
fn backing_store(
store: &StandaloneWorkerStore,
worker_id: WorkerId,
) -> Result<(StandaloneBackingStore, FsWorkerStore), StandaloneStartupError> {
let session_store = FsStore::new(store.sessions_dir(worker_id))
.map_err(|_| StandaloneStartupError::StateStore)?;
let session_store = FsStore::new_with_retention(
store.sessions_dir(worker_id),
crate::store::session_retention(store.root()),
)
.map_err(|_| StandaloneStartupError::StateStore)?;
let worker_store = FsWorkerStore::new(store.worker_metadata_dir(worker_id))
.map_err(|_| StandaloneStartupError::StateStore)?;
Ok((
@@ -808,3 +941,51 @@ mod local_input_tests {
}
}
}
#[cfg(test)]
mod source_retention_tests {
//! Host backing-store assembly uses the production publisher and an isolated external
//! erasure fence. No controller/model is substituted for the source authority under test.
use super::*;
use session_store::{LogEntry, Store, StoreError};
#[test]
fn backing_publisher_and_reopen_use_the_host_wide_erasure_fence() {
let temp = tempfile::tempdir().unwrap();
let workers = StandaloneWorkerStore::open(temp.path().join("state")).unwrap();
let allocation = workers
.allocate(temp.path(), StaleLeasePolicy::Reject)
.unwrap();
let worker = allocation.worker_id();
let (backing, _) = backing_store(&workers, worker).unwrap();
let session = session_store::SessionId::now_v7();
let segment = uuid::Uuid::now_v7();
let entry = LogEntry::TurnEnd {
ts: 1,
turn_count: 1,
};
backing
.create_segment(session, segment, &[entry.clone()])
.unwrap();
let retention = crate::store::session_retention(workers.root());
assert!(
retention
.delete(
session,
session_store::retention::SessionDeletionIntent::ErasePermanently,
|| Err::<(), StoreError>(
std::io::Error::other("injected physical erase failure").into()
)
)
.is_err()
);
assert!(
matches!(backing.append(session, segment, &entry), Err(StoreError::SessionErased(id)) if id == session)
);
let (reopened, _) = backing_store(&workers, worker).unwrap();
assert!(
matches!(reopened.append(session, segment, &entry), Err(StoreError::SessionErased(id)) if id == session)
);
assert!(!workers.sessions_dir(worker).join(".retention").exists());
}
}
+172 -3
View File
@@ -25,7 +25,7 @@ use thiserror::Error;
#[path = "job_store/legacy_migrations.rs"]
mod legacy_migrations;
const SCHEMA_VERSION: i64 = 3;
const SCHEMA_VERSION: i64 = 4;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
@@ -189,6 +189,7 @@ impl JobStore {
0 => {
tx.execute_batch(SCHEMA)?;
tx.execute_batch(GRANT_SCHEMA)?;
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
1 | 2 => {
@@ -196,6 +197,11 @@ impl JobStore {
tx.execute_batch(GRANT_SCHEMA)?;
}
legacy_migrations::migrate_legacy_jobs(&tx)?;
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
3 => {
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
SCHEMA_VERSION => {}
@@ -278,6 +284,49 @@ impl JobStore {
Ok(snapshot)
}
/// Domain Host attests that this fenced attempt's only effects are reconciled in its
/// authoritative durable consumer. This releases serialization, not the original Unknown
/// history, and cannot authorize retry or change its result. The service checks cleanup.
pub(crate) fn reconcile_granted_attempt(
&mut self,
job_id: &str,
attempt_id: &str,
domain_receipt: &str,
) -> Result<(), JobStoreError> {
if domain_receipt.is_empty()
|| domain_receipt.len() > 256
|| domain_receipt.chars().any(char::is_control)
{
return Err(JobStoreError::InvalidTransition(
"invalid domain reconciliation receipt",
));
}
let tx = self
.conn
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, job_id)?;
check_attempt(&snapshot, attempt_id)?;
if snapshot.domain_grant.is_none()
|| snapshot.attempt.state != JobAttemptState::Unknown
|| !matches!(snapshot.state, JobState::Unknown | JobState::Cancelled)
{
return Err(JobStoreError::InvalidTransition(
"domain reconciliation requires a fenced unknown granted attempt",
));
}
let old: Option<String> = tx.query_row("SELECT domain_receipt FROM job_domain_reconciliations WHERE job_id=?1 AND attempt_id=?2",
params![job_id,attempt_id], |row| row.get(0)).optional()?;
if old.as_deref().is_some_and(|old| old != domain_receipt) {
return Err(JobStoreError::InvalidTransition(
"domain reconciliation receipt changed",
));
}
tx.execute("INSERT OR IGNORE INTO job_domain_reconciliations(job_id,attempt_id,domain_receipt) VALUES(?1,?2,?3)",
params![job_id,attempt_id,domain_receipt])?;
tx.commit()?;
Ok(())
}
/// Only definitive failure can create a new immutable attempt. Unknown
/// outcomes require caller reconciliation, never an automatic replay.
pub(crate) fn retry(&mut self, job_id: &str) -> Result<JobSnapshot, JobStoreError> {
@@ -319,6 +368,16 @@ impl JobStore {
.collect()
}
pub(crate) fn unknown_granted(&self) -> Result<Vec<JobSnapshot>, JobStoreError> {
let mut statement = self.conn.prepare("SELECT i.job_id FROM job_intents i JOIN job_attempts a ON a.job_id=i.job_id AND a.number=i.current_attempt JOIN job_domain_grants g ON g.job_id=i.job_id WHERE a.state='unknown' AND NOT EXISTS (SELECT 1 FROM job_domain_reconciliations r WHERE r.job_id=a.job_id AND r.attempt_id=a.attempt_id) ORDER BY i.rowid")?;
let ids = statement
.query_map([], |row| row.get::<_, String>(0))?
.collect::<rusqlite::Result<Vec<_>>>()?;
ids.into_iter()
.map(|id| require_snapshot(&self.conn, &id))
.collect()
}
/// Only unstarted current attempts are eligible for service startup/resume.
pub(crate) fn pending(&self) -> Result<Vec<JobSnapshot>, JobStoreError> {
let mut statement = self.conn.prepare(
@@ -527,8 +586,10 @@ fn check_serialization(
if let Some(key) = key {
let conflicting: bool = conn.query_row(
"SELECT EXISTS(SELECT 1 FROM job_intents i WHERE serialization_key = ?1
AND job_id <> ?2 AND (state IN ('pending', 'unknown') OR EXISTS
(SELECT 1 FROM job_attempts a WHERE a.job_id = i.job_id AND a.state = 'unknown')))",
AND job_id <> ?2 AND (state = 'pending' OR EXISTS
(SELECT 1 FROM job_attempts a WHERE a.job_id = i.job_id AND a.state = 'unknown'
AND NOT EXISTS (SELECT 1 FROM job_domain_reconciliations r
WHERE r.job_id=a.job_id AND r.attempt_id=a.attempt_id))))",
params![key, job_id],
|row| row.get(0),
)?;
@@ -658,6 +719,22 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
.transpose()
}
const RECONCILIATION_SCHEMA: &str = "
DROP INDEX job_serialization_singleflight;
CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key)
WHERE serialization_key IS NOT NULL AND state = 'pending';
CREATE TABLE job_domain_reconciliations (
job_id TEXT NOT NULL REFERENCES job_intents(job_id),
attempt_id TEXT NOT NULL,
domain_receipt TEXT NOT NULL,
PRIMARY KEY (job_id, attempt_id)
);
CREATE TRIGGER job_reconciliation_immutable BEFORE UPDATE ON job_domain_reconciliations
BEGIN SELECT RAISE(ABORT, 'immutable domain reconciliation'); END;
CREATE TRIGGER job_reconciliation_retained BEFORE DELETE ON job_domain_reconciliations
BEGIN SELECT RAISE(ABORT, 'retained domain reconciliation'); END;
";
const GRANT_SCHEMA: &str = "
CREATE TABLE job_domain_grants (
job_id TEXT PRIMARY KEY NOT NULL REFERENCES job_intents(job_id),
@@ -732,6 +809,98 @@ mod tests {
}
}
#[test]
fn granted_unknown_reconciliation_releases_serialization_but_preserves_history() {
let (dir, mut store) = fixture();
let mut first = request("first");
first.serialization_key = Some("journal".into());
let reserved = store
.reserve_granted(first, Some(json!({"domain":"journal"})))
.unwrap();
store
.dispatch("first", &reserved.attempt.attempt_id, 8)
.unwrap();
let unknown = store
.finish(
"first",
&reserved.attempt.attempt_id,
JobAttemptState::Unknown,
Some("response lost"),
)
.unwrap();
let mut second = request("second");
second.serialization_key = Some("journal".into());
assert!(matches!(
store.reserve(second.clone()),
Err(JobStoreError::SerializationConflict(_))
));
assert!(
store
.reconcile_granted_attempt("first", "foreign-attempt", "domain-fenced")
.is_err()
);
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
assert!(
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "different-proof")
.is_err()
);
assert_eq!(store.get("first").unwrap(), unknown);
store.reserve(second).unwrap();
assert!(store.retry("first").is_err());
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(reopened.get("first").unwrap(), unknown);
assert!(reopened.unknown_granted().unwrap().is_empty());
assert!(
reopened
.conn
.execute("DELETE FROM job_domain_reconciliations", [])
.is_err()
);
assert!(reopened.retry("first").is_err());
}
#[test]
fn schema_three_migration_preserves_unknown_grant_and_its_serialization_fence() {
let (dir, mut store) = fixture();
let mut first = request("first");
first.serialization_key = Some("journal".into());
let reserved = store
.reserve_granted(first, Some(json!({"domain":"journal"})))
.unwrap();
store
.dispatch("first", &reserved.attempt.attempt_id, 8)
.unwrap();
let unknown = store
.finish(
"first",
&reserved.attempt.attempt_id,
JobAttemptState::Unknown,
Some("unconfirmed"),
)
.unwrap();
store.conn.execute_batch("DROP TRIGGER job_reconciliation_immutable; DROP TRIGGER job_reconciliation_retained; DROP TABLE job_domain_reconciliations; DROP INDEX job_serialization_singleflight; CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key) WHERE serialization_key IS NOT NULL AND state IN ('pending','unknown'); PRAGMA user_version=3;").unwrap();
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(reopened.get("first").unwrap(), unknown);
let mut next = request("next");
next.serialization_key = Some("journal".into());
assert!(matches!(
reopened.reserve(next.clone()),
Err(JobStoreError::SerializationConflict(_))
));
reopened
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
reopened.reserve(next).unwrap();
}
fn fixture() -> (TempDir, JobStore) {
let dir = tempfile::tempdir().unwrap();
let store = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
+54
View File
@@ -66,6 +66,15 @@ struct Executions {
jobs: HashMap<String, Execution>,
}
pub(crate) struct QuiescentJobClosure<'a> {
executions: std::sync::MutexGuard<'a, Executions>,
}
impl QuiescentJobClosure<'_> {
pub(crate) fn commit(&mut self) {
self.executions.closed = true;
}
}
/// Cloneable Feature-facing access to a Host-owned service. Cloning this handle
/// neither extends the Host's authority nor permits opening/recovering its store.
#[derive(Clone)]
@@ -79,6 +88,27 @@ pub struct StandaloneJobs {
}
impl StandaloneJobs {
/// Excludes request/start/result acknowledgement while a Subject Host checks its final
/// saved obligations and releases ownership. Accepted results alone are insufficient:
/// every spawned execution must have actually finished successful cleanup.
pub(crate) fn reserve_quiescent_closure(
&self,
) -> Result<QuiescentJobClosure<'_>, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
if executions.jobs.values().any(|execution| {
!execution.task.is_finished()
|| !matches!(execution.complete.borrow().as_ref(), Some(Ok(())))
}) {
return Err(JobServiceError::Cleanup(
"delegated Job execution/cleanup remains running or unconfirmed".into(),
));
}
Ok(QuiescentJobClosure { executions })
}
pub(crate) fn open(path: &Path, cwd: PathBuf) -> Result<Self, JobServiceError> {
Ok(Self {
store: Arc::new(Mutex::new(Some(JobStore::open(path)?))),
@@ -157,6 +187,30 @@ impl StandaloneJobs {
self.with_store(|store| store.unacknowledged())
}
pub(crate) fn unknown_granted(&self) -> Result<Vec<JobSnapshot>, JobServiceError> {
self.with_store(|store| store.unknown_granted())
}
/// Release only an Unknown attempt's serialization reservation after the bound domain
/// has reconciled its receipts and durably fenced further effects. Never a generic retry.
pub(crate) fn reconcile_granted_attempt(
&self,
job_id: &str,
attempt_id: &str,
domain_receipt: &str,
) -> Result<(), JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
if executions.jobs.contains_key(job_id) {
return Err(JobServiceError::Cleanup(
"domain reconciliation requires observed execution cleanup".into(),
));
}
self.with_store(|store| store.reconcile_granted_attempt(job_id, attempt_id, domain_receipt))
}
/// Explicit new attempt, bounded by immutable intent limits. Unknown
/// outcomes cannot be retried without domain reconciliation/new intent.
pub fn retry(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
+4 -1
View File
@@ -8,7 +8,10 @@ pub mod store;
pub mod subjektiv;
mod subjektiv_sessions;
pub use host::{StandaloneHost, StandaloneShutdownError, StandaloneStartupError};
pub use host::{
StandaloneHost, StandaloneShutdownError, StandaloneStartupError,
StandaloneSubjectDisconnectError,
};
pub use launch::{ResolvedStandaloneLaunch, StandaloneLaunchConfig, StandaloneLaunchError};
pub use protocol::WorkerId;
pub use store::{
+26 -1
View File
@@ -119,6 +119,16 @@ pub enum StaleLeasePolicy {
Recover,
}
/// Source control outlives Worker relocation and physical Session pruning. Always derive this
/// from the Host state root, never from an individual live or retained Worker directory.
pub(crate) fn session_retention(
state_root: &Path,
) -> session_store::retention::SessionRetentionStore {
session_store::retention::SessionRetentionStore::new(
state_root.join("retention/session-sources"),
)
}
#[derive(Debug, Clone)]
pub struct StandaloneWorkerStore {
root: PathBuf,
@@ -359,7 +369,19 @@ impl StandaloneWorkerStore {
}
pub fn delete(&self, id: WorkerId) -> Result<(), StandaloneStoreError> {
let record = self.load(id)?;
let record = match self.load(id) {
Err(StandaloneStoreError::WorkerNotFound(_)) => {
let retained_root = self.root.join("subjektiv").join("retained-workers");
let retained =
Self::open_existing(retained_root).and_then(|store| store.load(id))?;
if retained.status == StandaloneWorkerStatus::Stopped && retained.subject.is_some()
{
return Ok(());
}
return Err(StandaloneStoreError::InvalidRecord(id));
}
result => result?,
};
if record.status != StandaloneWorkerStatus::Stopped {
return Err(StandaloneStoreError::DeleteActive(id));
}
@@ -374,6 +396,7 @@ impl StandaloneWorkerStore {
LeaseLiveness::Unknown => StandaloneStoreError::LeaseLivenessUnknown(id),
});
}
crate::subjektiv::prepare_worker_deletion(&self.root, &record)?;
fs::remove_dir_all(self.worker_dir(id)).map_err(StandaloneStoreError::Io)?;
sync_directory(&self.root)
}
@@ -793,6 +816,8 @@ pub enum StandaloneStoreError {
LeaseOwnershipLost,
#[error("standalone Worker {0} must be stopped before deletion")]
DeleteActive(WorkerId),
#[error("Journal deletion handoff/retention is unconfirmed: {0}")]
JournalRetention(String),
#[error("standalone Worker {0} metadata changed since it was read")]
MetadataChanged(WorkerId),
#[error("system clock is before the Unix epoch or out of range")]
+225 -10
View File
@@ -5,9 +5,11 @@
//! non-private nodes; delegates schema/migration to `subjektiv` and connection policy to `feature-storage`.
//!
//! The Host holds each selected Subject's exclusive kernel lease through body execution and
//! confirmed delegated Job cleanup. Process death releases kernel locks; uncertain in-process
//! cleanup retains leases. Restore validates saved Subject/storage bindings and reacquires the
//! same Subject. Launches without a selected Subject do not implicitly connect this store.
//! confirmed delegated Job cleanup. Kernel lock release never clears durable body ownership:
//! shutdown/process death permits only the same body to restore, not a replacement to start.
//! Explicit disconnection requires actual Controller quiescence and final Journal completion;
//! storage then revokes that writer lifetime before the lease is released. Uncertain cleanup
//! retains leases. Launches without a selected Subject do not implicitly connect this store.
//!
//! The selected Subject's Job store is shared across Workers; the generic Jobs service owns execution
//! and cleanup. Consolidation grants allow only immutable candidate batches and limited operations,
@@ -15,6 +17,9 @@
//! Unknown attempts are not rerun; delivery failures for accepted results do not trigger model
//! reexecution. Memory application and surface repair use shared-domain CAS/receipts.
mod journal;
pub(crate) use journal::prepare_worker_deletion;
use std::fs::{self, File, OpenOptions};
use std::io::Write;
use std::path::{Path, PathBuf};
@@ -46,6 +51,8 @@ pub enum SubjectError {
#[error(transparent)]
Domain(#[from] subjektiv::SubjektivError),
#[error(transparent)]
Journal(#[from] subjektiv::journal::JournalError),
#[error(transparent)]
Storage(#[from] feature_storage::FeatureStorageError),
#[error("local Subject storage metadata: {0}")]
Json(#[from] serde_json::Error),
@@ -220,7 +227,51 @@ impl StandaloneSubjects {
scope_id: self.scope_id.clone(),
subject_id: subject_id.to_string(),
};
let host = Arc::new(LocalSubjectHost {
if self.store.journal_worker_was_released(
subject_id,
&self.scope_id,
&worker_id.to_string(),
)? {
return Err(SubjectError::InvalidScope("this body's Subject binding was released; start a new Worker/Session rather than restore its old authority".into()));
}
// Before writer lifetimes existed, a saved body record was the only durable
// attachment evidence. Never interpret its stopped status as a release on upgrade.
let workers = crate::store::StandaloneWorkerStore::open_existing(&self.state_root)
.map_err(|error| SubjectError::InvalidScope(error.to_string()))?;
for entry in fs::read_dir(&self.state_root)? {
let entry = entry?;
let Ok(prior_id) = entry
.file_name()
.to_string_lossy()
.parse::<protocol::WorkerId>()
else {
continue;
};
if prior_id == worker_id {
continue;
}
let prior = match workers.load(prior_id) {
Ok(prior) => prior,
Err(crate::store::StandaloneStoreError::WorkerNotFound(_)) => continue,
Err(error) => return Err(SubjectError::InvalidScope(error.to_string())),
};
if prior.subject.as_ref() == Some(&binding)
&& !self.store.journal_worker_was_released(
subject_id,
&self.scope_id,
&prior_id.to_string(),
)?
{
return Err(SubjectError::InvalidScope("saved Subject body has no explicit release; restore it and complete the Idle/Journal handoff, rather than treating inactive storage as unoccupied".into()));
}
}
let writer =
self.store
.claim_journal_writer(subject_id, &self.scope_id, &worker_id.to_string())?;
let host = Arc::new_cyclic(|weak| LocalSubjectHost {
writer,
self_weak: weak.clone(),
journal_dispatch: Mutex::new(false),
language: self.memory_language.clone(),
catalog: self,
binding: binding.clone(),
@@ -247,6 +298,37 @@ pub(crate) struct SubjectConnection {
armed: bool,
}
impl SubjectConnection {
/// Construction failed before any Controller or mandatory startup context was published.
/// This is cancellation of an unpublished reservation, not Idle inferred from a stopped body.
pub(crate) fn abort_unpublished_start(&mut self) -> Result<(), SubjectError> {
self.host
.catalog
.store
.release_journal_writer(&self.host.writer)?;
self.close()
}
pub(crate) fn writer_released_or_unknown(&self) -> bool {
self.host
.catalog
.store
.journal_writer_active(&self.host.writer)
.map(|active| !active)
.unwrap_or(true)
}
pub(crate) fn release_for_handoff(
&mut self,
record: &crate::store::StandaloneWorkerRecord,
reservation: &mut worker::SubjectReleaseReservation,
) -> Result<(), String> {
if let Err(error) = self.host.release_for_handoff(record, reservation) {
self.host.wake_journals();
return Err(error.to_string());
}
reservation.commit_disconnect()
}
pub fn arm(&mut self) {
self.armed = true;
}
@@ -277,6 +359,9 @@ impl Drop for SubjectConnection {
}
pub(crate) struct LocalSubjectHost {
writer: subjektiv::journal::JournalWriterBinding,
self_weak: std::sync::Weak<Self>,
journal_dispatch: Mutex<bool>,
catalog: StandaloneSubjects,
binding: StandaloneSubjectBinding,
worker_id: protocol::WorkerId,
@@ -293,6 +378,16 @@ impl LocalSubjectHost {
"local Subject Host is closed".into(),
));
}
if !self
.catalog
.store
.journal_writer_active(&self.writer)
.map_err(journal::journal_error)?
{
return Err(SubjektivHostError::Unavailable(
"Subject writer binding was released".into(),
));
}
Ok(())
}
fn body(&self, context: &SubjektivHostContext) -> Result<(), SubjektivHostError> {
@@ -332,6 +427,9 @@ impl LocalSubjectHost {
"Session does not belong to this local Subject/Worker scope".into(),
));
}
store
.bind_journal_session(&self.writer, record.clone())
.map_err(journal::journal_error)?;
return Ok(record);
}
if !create {
@@ -348,9 +446,13 @@ impl LocalSubjectHost {
&context.session_id,
)
.map_err(domain_error)?;
store
let record = store
.record_session_attribution(record)
.map_err(domain_error)
.map_err(domain_error)?;
store
.bind_journal_session(&self.writer, record.clone())
.map_err(journal::journal_error)?;
Ok(record)
}
}
@@ -551,10 +653,11 @@ impl LocalSubjectHost {
.and_then(|v| serde_json::from_value::<LocalConsolidationGrant>(v.clone()).ok())
.is_some_and(|g| g.binding == self.binding)
{
schedule_job(jobs.clone(), job.request.job_id);
schedule_job(jobs.clone(), job.request.job_id, self.self_weak.clone());
}
}
}
self.recover_journals();
}
fn validate_evidence(
&self,
@@ -690,6 +793,107 @@ impl LocalSubjectHost {
}
}
impl SubjektivHost for LocalSubjectHost {
fn memory_journal(
&self,
context: &SubjektivHostContext,
request: subjektiv::journal::memory_links::MemoryJournalRequest,
) -> Result<subjektiv::journal::memory_links::MemoryJournalResponse, SubjektivHostError> {
self.body(context)?;
let reader = subjektiv::journal::JournalHostContext::validated_reader(
&self.catalog.store,
&self.binding.subject_id,
)
.map_err(journal::journal_error)?;
self.catalog
.store
.execute_memory_journal_with_source_presence(&reader, request, |session| {
crate::subjektiv_sessions::original_session_presence(
&self.catalog.state_root,
&self.binding.scope_id,
&self.binding.subject_id,
&self.catalog.store,
session,
)
})
.map_err(journal::journal_error)
}
fn journal_startup_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn journal_startup_context(
&self,
context: &SubjektivHostContext,
) -> Result<subjektiv::journal::startup::JournalStartupSnapshot, SubjektivHostError> {
self.body(context)?;
let reader = subjektiv::journal::JournalHostContext::validated_reader(
&self.catalog.store,
&self.binding.subject_id,
)
.map_err(journal::journal_error)?;
let predecessor = self
.catalog
.store
.journal_predecessor_session(&self.writer)
.map_err(journal::journal_error)?;
self.catalog
.store
.journal_startup_snapshot(&reader, predecessor.as_deref())
.map_err(journal::journal_error)
}
fn journal_compaction_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn journal_compaction_context(
&self,
context: &SubjektivHostContext,
) -> Result<subjektiv::journal::startup::JournalStartupSnapshot, SubjektivHostError> {
self.body(context)?;
let attribution = self
.catalog
.store
.session_attribution(&context.session_id)
.map_err(domain_error)?
.filter(|attr| {
attr.subject_id == self.binding.subject_id
&& attr.runtime_id == self.binding.scope_id
&& attr.worker_id == self.worker_id.to_string()
})
.ok_or_else(|| {
SubjektivHostError::InvalidArgument(
"foreign or unattributed compaction Session".into(),
)
})?;
let host = subjektiv::journal::JournalHostContext::validated_session(
&self.catalog.store,
attribution,
)
.map_err(journal::journal_error)?;
if host.writer_binding_id() != Some(self.writer.binding_id.as_str()) {
return Err(SubjektivHostError::InvalidArgument(
"compaction writer lifetime mismatch".into(),
));
}
self.catalog
.store
.journal_compaction_snapshot(&host)
.map_err(journal::journal_error)
}
fn journal_recording_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn request_journal_recording(
&self,
context: &SubjektivHostContext,
_reason: subjektiv::journal::transport::JournalRecordingReason,
) -> Result<subjektiv::journal::transport::JournalRecordingResponse, SubjektivHostError> {
let admitted = self.admit_journal(context)?;
self.wake_journals();
Ok(if admitted {
subjektiv::journal::transport::JournalRecordingResponse::Accepted
} else {
subjektiv::journal::transport::JournalRecordingResponse::NoNewSources
})
}
fn settings(&self) -> SubjektivHostSettings {
SubjektivHostSettings {
language: self.language.clone(),
@@ -788,6 +992,7 @@ impl SubjektivHost for LocalSubjectHost {
create_if_missing: bool,
) -> Result<SubjektivRecordSessionResponse, SubjektivHostError> {
let attribution = self.attribution(context, create_if_missing)?;
self.reserve_journal_session(context)?;
Ok(SubjektivRecordSessionResponse {
subject_id: attribution.subject_id,
session_id: attribution.session_id,
@@ -811,7 +1016,7 @@ impl SubjektivHost for LocalSubjectHost {
.is_some_and(|g| g.binding == self.binding)
})
{
schedule_job(jobs, pending.request.job_id);
schedule_job(jobs, pending.request.job_id, self.self_weak.clone());
let (count, bytes) = self
.catalog
.store
@@ -874,7 +1079,7 @@ impl SubjektivHost for LocalSubjectHost {
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?,
)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?;
schedule_job(jobs, snapshot.request.job_id);
schedule_job(jobs, snapshot.request.job_id, self.self_weak.clone());
Ok(memory::backend::MemoryConsolidationOutput { status:"started".into(),summary:"Requested a durable generic standalone consolidation Job; processing stops on Host shutdown".into(),candidate_count:count,total_bytes:bytes })
}
}
@@ -894,6 +1099,9 @@ impl crate::jobs::JobDomainProvider for LocalDomainProvider {
) -> Result<worker::job::JobFeatureGrant, String> {
let host = self.0.upgrade().ok_or("Subject Host unavailable")?;
host.ensure_live().map_err(|e| e.to_string())?;
if snapshot.request.purpose == journal::PURPOSE {
return journal::grant(host, snapshot, fence);
}
let grant: LocalConsolidationGrant = serde_json::from_value(
snapshot
.domain_grant
@@ -1097,7 +1305,11 @@ impl worker::job::JobResultFinalizer for LocalConsolidationFinalizer {
}
}
fn schedule_job(jobs: crate::jobs::StandaloneJobs, id: String) {
fn schedule_job(
jobs: crate::jobs::StandaloneJobs,
id: String,
host: std::sync::Weak<LocalSubjectHost>,
) {
// Scheduling/consumer acknowledgement has no Internal Worker ownership.
// The generic service fences and retains every execution until cleanup.
tokio::spawn(async move {
@@ -1114,6 +1326,9 @@ fn schedule_job(jobs: crate::jobs::StandaloneJobs, id: String) {
"standalone Subject Job {id} cleanup/acknowledgement is unconfirmed: {error}"
),
}
if let Some(host) = host.upgrade() {
host.wake_journals();
}
});
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,463 @@
//! Public standalone protocol through real mid-turn compaction and Journal Jobs.
//! Only external model streams are scripted. No history, attribution, hook or Job
//! result is injected; cancellation and rollback use the controller's public commands.
//! Reported model usage and a low configured request threshold force two rewrites.
//! The test attests durable public originals, not unfinished display deltas or meaning.
use super::*;
use agen::llm_client::{
ClientError, LlmClient, Request,
event::{Event, ResponseStatus, StatusEvent, StopReason, UsageEvent},
types::Item,
};
use futures::{Stream, StreamExt};
use std::pin::Pin;
use std::sync::atomic::AtomicUsize;
type ResponseStream = Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>;
const SUMMARY: &str = "DERIVED_COMPACTION_SENTINEL_NOT_AN_ORIGINAL";
const ROLLED_BACK: &str = "ROLLED_BACK_INPUT_MUST_NOT_REACH_JOURNAL";
const PARTIAL: &str = "public partial answer before cancellation";
#[derive(Clone, Default)]
struct CompactingDialog {
shared: DialogAndJournal,
dialog_calls: Arc<AtomicUsize>,
compactions: Arc<AtomicUsize>,
rollback_entered: Arc<tokio::sync::Notify>,
partial_stream_paused: Arc<tokio::sync::Notify>,
}
fn events(events: Vec<Event>) -> ResponseStream {
Box::pin(futures::stream::iter(events.into_iter().map(Ok)))
}
fn done() -> ResponseStream {
events(vec![
Event::text_block_start(0),
Event::text_delta(0, "done"),
Event::text_block_stop(0, Some(StopReason::EndTurn)),
])
}
#[async_trait::async_trait]
impl LlmClient for CompactingDialog {
async fn stream(&self, request: Request) -> Result<ResponseStream, ClientError> {
if request
.tools
.iter()
.any(|tool| tool.name == "write_summary")
{
if request.items.iter().any(|item| matches!(item, Item::ToolResult { call_id, .. } if call_id == "compact-summary")) {
return Ok(done());
}
let count = self.compactions.fetch_add(1, Ordering::SeqCst);
assert!(count < 2, "unexpected additional compaction");
return Ok(events(vec![
Event::tool_use_start(0, "compact-summary", "write_summary"),
Event::tool_input_delta(0, serde_json::json!({"text": SUMMARY}).to_string()),
Event::tool_use_stop(0),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]));
}
if request.tools.iter().any(|tool| {
matches!(
tool.name.as_str(),
"ReadJournalState" | "SubmitMemorySurface" | "SubmitJobResult"
)
}) {
return self.shared.stream(request).await;
}
match self.dialog_calls.fetch_add(1, Ordering::SeqCst) {
step @ (0 | 1) => {
assert!(request.tools.iter().any(|tool| tool.name == "TaskCreate"));
Ok(events(vec![
Event::tool_use_start(0, format!("public-task-{step}"), "TaskCreate"),
Event::tool_input_delta(
0,
serde_json::json!({
"subject": format!("original activity {step}"),
"description": format!("original task result before compaction {step}")
})
.to_string(),
),
Event::tool_use_stop(0),
Event::Usage(UsageEvent {
input_tokens: Some(100_000),
output_tokens: Some(10),
total_tokens: Some(100_010),
..Default::default()
}),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]))
}
2 => {
let paused = self.partial_stream_paused.clone();
Ok(Box::pin(
events(vec![
Event::text_block_start(0),
Event::text_delta(0, PARTIAL),
Event::text_block_stop(0, None),
])
.chain(futures::stream::once(async move {
paused.notify_one();
std::future::pending::<Result<Event, ClientError>>().await
})),
))
}
3 => {
// No AI event: cancelling this request must roll back the submitted input.
self.rollback_entered.notify_one();
std::future::pending::<()>().await;
unreachable!()
}
other => panic!("unexpected dialog request {other}"),
}
}
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
}
async fn cancel_and_wait(
client: &mut client::Client<client::transport::in_process::Socket>,
sequence: u64,
) {
client
.send(&protocol::Method::Cancel {
command: protocol::WorkerCommandEnvelope::new(sequence),
})
.await
.unwrap();
tokio::time::timeout(std::time::Duration::from_secs(20), async {
loop {
match client.next_event().await.unwrap().unwrap() {
protocol::Event::RunEnd { result } => {
assert_ne!(result, protocol::RunResult::Finished);
return;
}
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
}
})
.await
.unwrap();
}
#[tokio::test]
async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_and_rollback() {
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("compaction-journal", None).unwrap();
let model = CompactingDialog {
shared: DialogAndJournal {
journal: JournalScript {
gate_first: true,
..Default::default()
},
..Default::default()
},
..Default::default()
};
let mut launch = crate::StandaloneLaunchConfig::new(
temp.path(),
&state,
manifest::ProfileSelector::source_named(
manifest::ProfileRegistrySource::Builtin,
"standalone-subjektiv",
),
"public-compaction-journal",
)
.with_subject(subject.id.clone())
.resolve()
.unwrap();
launch.profile.manifest.compaction = Some(manifest::CompactionConfig {
threshold: None,
request_threshold: Some(50_000),
retained_tokens: 0,
..Default::default()
});
let mut host = crate::StandaloneHost::start_with_model_client(launch, model.clone())
.await
.unwrap();
let session = host.record().active_session_id;
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
let source_store = session_store::FsStore::new_with_retention(
workers.sessions_dir(host.record().worker_id),
crate::store::session_retention(&state),
)
.unwrap();
let mut client = host.connect();
client
.send(&protocol::Method::submit_text(
protocol::new_submission_request_id(),
"original long turn survives compaction",
))
.await
.unwrap();
let mut completed_compactions = 0;
loop {
match client.next_event().await.unwrap().unwrap() {
protocol::Event::CompactionProgress {
compaction: Some(progress),
} if progress.phase == protocol::CompactionPhase::Committing => {
assert_eq!(
progress.trigger,
protocol::CompactionTrigger::RequestThreshold
);
completed_compactions += 1;
}
protocol::Event::TextDelta { text } if text.contains(PARTIAL) => break,
protocol::Event::RunEnd { .. } => panic!("dialog ended before cancellation"),
protocol::Event::SessionEntryCommitted { entry } if matches!(entry.data, protocol::SessionSnapshotEntryData::RunError { .. }) => panic!("dialog failed before cancellation: {entry:?}"),
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
}
assert_eq!(model.compactions.load(Ordering::SeqCst), 2);
assert_eq!(completed_compactions, 2);
model.shared.journal.entered.notified().await;
let context = JournalHostContext::validated_session(
&catalog.store,
catalog
.store
.session_attribution(&session.to_string())
.unwrap()
.unwrap(),
)
.unwrap();
let reader = JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap();
let journal = catalog
.store
.list_journals(&reader, None, 1)
.unwrap()
.items
.pop()
.unwrap();
let first = catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap();
assert!(
catalog
.store
.list_journal_entries(&reader, &journal.id, None, 10)
.unwrap()
.items
.is_empty(),
"compaction must finish before generation is released"
);
model.partial_stream_paused.notified().await;
cancel_and_wait(&mut client, 1).await;
client
.send(&protocol::Method::submit_text(
protocol::new_submission_request_id(),
ROLLED_BACK,
))
.await
.unwrap();
model.rollback_entered.notified().await;
let before_rollback: Vec<_> = source_store
.list_segments(session)
.unwrap()
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
assert!(
serde_json::to_string(&before_rollback)
.unwrap()
.contains(ROLLED_BACK),
"the submitted input must reach the actual durable rollback boundary"
);
cancel_and_wait(&mut client, 2).await;
assert_eq!(
catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap(),
first,
"compaction/cancellation must not alter the running generation's frozen input"
);
assert!(matches!(source_store.delete_session(session),
Err(session_store::StoreError::SessionPinned(id)) if id == session));
let segments = source_store.list_segments(session).unwrap();
assert_eq!(segments.len(), 3);
let raw: Vec<_> = segments
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
for history in raw.iter().filter_map(|record| match record {
session_store::LogEntry::AnnotatedSegmentStart { history, compacted_from: Some(_), .. } => Some(history),
_ => None,
}) {
let mut calls = std::collections::HashSet::new();
let mut results = 0;
for entry in history {
let is_tool = matches!(entry.item, session_store::LoggedItem::ToolCall { .. } | session_store::LoggedItem::ToolResult { .. });
let Some(metadata) = entry.metadata.input_metadata.as_ref().filter(|_| is_tool) else { continue; };
assert_eq!(metadata.usage().recording_material, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().memory_evidence, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().observation, session_store::InputObservationUse::Hidden);
assert!(metadata.public_item().is_none());
assert!(entry.metadata.derivation.is_some());
match &entry.item {
session_store::LoggedItem::ToolCall { call_id, .. } => { calls.insert(call_id.clone()); }
session_store::LoggedItem::ToolResult { call_id, .. } => {
assert!(calls.contains(call_id), "derived Tool result must follow its real logical call");
results += 1;
}
_ => unreachable!(),
}
}
assert!(!calls.is_empty());
assert!(results >= calls.len());
}
assert_eq!(
raw.iter()
.filter(|entry| matches!(
entry,
session_store::LogEntry::AnnotatedSegmentStart {
compacted_from: Some(_),
..
}
))
.count(),
2
);
assert!(
!serde_json::to_string(&raw).unwrap().contains(ROLLED_BACK),
"pre-AI cancellation must roll back the speculative input"
);
assert!(
serde_json::to_string(&raw).unwrap().contains(SUMMARY),
"real rewrite did not persist its summary"
);
let metadata =
session_store::FsWorkerStore::new(workers.worker_metadata_dir(host.record().worker_id))
.unwrap()
.read_by_name(&host.record().storage_key)
.unwrap()
.unwrap();
let originals =
session_store::public_material::SessionPublicMaterialReader::open_fs_with_retention(
&workers.sessions_dir(host.record().worker_id),
session,
metadata.active.unwrap().segment_id.unwrap(),
session_store::public_material::PublicMaterialLimits::default(),
crate::store::session_retention(&state),
)
.unwrap();
let final_original = originals.descriptors().last().unwrap();
let expected_cursor = (final_original.canonical_order + 1, 0);
let canonical: Vec<_> = originals
.descriptors()
.map(|descriptor| {
let page = originals
.read_content(descriptor, 0, MAX_JOURNAL_READ_BYTES)
.unwrap();
assert_eq!(page.next_offset, descriptor.total_bytes);
page.content
})
.collect();
assert!(!canonical.join("\n").contains(ROLLED_BACK));
assert!(!canonical.join("\n").contains(SUMMARY));
// Resume the existing runner, then its one durable successor, with no more input.
let jobs = host.jobs();
model.shared.journal.release.notify_one();
jobs.wait(&first.job.job_id).await.unwrap();
model.shared.journal.entered.notified().await;
let second = catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap();
assert_ne!(first.id, second.id);
model.shared.journal.release.notify_one();
jobs.wait(&second.job.job_id).await.unwrap();
assert!(!catalog.store.journal_retention_required(&context).unwrap());
let (saved, open) = catalog
.store
.journal_resume_state(&context, &journal.id)
.unwrap();
let open = open.unwrap();
assert!(!open.ongoing.is_empty());
let entries = catalog
.store
.list_journal_entries(&reader, &journal.id, None, 10)
.unwrap();
assert_eq!(
entries.items.len(),
1,
"compaction/turn boundaries must not fix or duplicate the open entry"
);
assert_eq!(entries.items[0].id, open.id);
let supplied = model
.shared
.journal
.source_reads
.lock()
.unwrap()
.values()
.cloned()
.collect::<Vec<_>>()
.join("\n");
for original in [
"original long turn survives compaction",
"original activity 0",
"original activity 1",
] {
assert!(
supplied.contains(original),
"Journal did not receive {original}"
);
}
for original in canonical {
assert!(
supplied.contains(&original),
"a durable public original was skipped"
);
}
assert!(
!supplied.contains(SUMMARY),
"derived summary substituted for original input"
);
assert!(
!supplied.contains(ROLLED_BACK),
"cancelled input was admitted after rollback"
);
assert_eq!((saved.process_order, saved.process_byte), expected_cursor);
let position = catalog
.store
.read_journal_text(
&reader,
&journal.id,
None,
JournalTextField::CurrentPosition,
None,
0,
MAX_JOURNAL_READ_BYTES,
)
.unwrap();
assert!(!position.text.is_empty());
loop {
match host.disconnect_subject().await {
Ok(()) => break,
Err(crate::StandaloneSubjectDisconnectError::Incomplete(message)) if message.contains("dispatcher") || message.contains("cleanup") => tokio::task::yield_now().await,
Err(error) => panic!("recorded compaction/cancellation lineage must remain releasable while work is unfinished: {error}"),
}
}
host.shutdown().await.unwrap();
})
.await
.expect("bounded public compaction/Journal integration");
}
@@ -0,0 +1,171 @@
//! Real LocalSubjectHost/Journal storage -> PreparedWorker -> compactor Request -> durable seed.
//! Only external LLM responses and the fixture's Job execution fence are substituted. The
//! in-flight generation is retained while its real adapter updates storage during generation.
//! Temporary storage and the parent fixture own cleanup; ordinary standalone lib tests select
//! this suite. Predetermined summaries attest transport and persistence, not model understanding.
use super::*;
use agen::llm_client::{
ClientError, LlmClient, Request,
event::{Event, ResponseStatus, StatusEvent},
types::Item,
};
use futures::Stream;
use session_store::{CombinedStore, FsWorkerStore, LogEntry};
use std::pin::Pin;
const BEFORE: &str = "FROZEN_POSITION: B partially implemented; tests not run; result unconfirmed";
const AFTER: &str = "NEXT_POSITION: background Journal changed during compactor generation";
const COMPLEMENT: &str =
"Unrecorded request: keep the user's constraint and finish the remaining checks.";
#[derive(Clone)]
struct MockClient {
adapter: Arc<LocalJournalHost>,
requests: Arc<Mutex<Vec<Request>>>,
updated: Arc<AtomicBool>,
}
#[async_trait::async_trait]
impl LlmClient for MockClient {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(
&self,
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>, ClientError> {
assert!(
request
.tools
.iter()
.any(|tool| tool.name == "write_summary")
);
let first = self.requests.lock().unwrap().is_empty();
self.requests.lock().unwrap().push(request);
let events = if first {
assert!(!self.updated.swap(true, Ordering::SeqCst));
let state = self.adapter.state().unwrap();
save(
&self.adapter,
"position-during-compaction",
&state.current_journal.state_token,
JournalAction::UpdatePosition {
current_position: AFTER.into(),
},
vec![],
);
vec![
Event::tool_use_start(0, "summary", "write_summary"),
Event::tool_input_delta(0, serde_json::json!({ "text": COMPLEMENT }).to_string()),
Event::tool_use_stop(0),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
} else {
vec![
Event::text_block_start(0),
Event::text_delta(0, "summary saved"),
Event::text_block_stop(0, None),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
};
Ok(Box::pin(futures::stream::iter(events.into_iter().map(Ok))))
}
}
#[tokio::test]
async fn local_host_compaction_freezes_pending_journal_through_request_and_durable_seed_during_update()
{
tokio::time::timeout(std::time::Duration::from_secs(30), async {
let fixture = Fixture::new("Keep the user's constraint; work remains unfinished and unconfirmed.");
let (adapter, _, job) = fixture.adapter();
let created = save(
&adapter,
"record-partial-work",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "A corrected to B".into(),
body: "B partially implemented; tests not run; result unconfirmed".into(),
ongoing: "Finish remaining checks".into(),
state: JournalEntryState::Open,
source_end: None,
correction_refs: vec![],
},
adapter.generation.selected_ranges.clone(),
);
save(&adapter, "position-before-compaction", &created.state_token,
JournalAction::UpdatePosition { current_position: BEFORE.into() }, vec![]);
let before = adapter.state().unwrap();
let expected = fixture.host().journal_compaction_context(&fixture.context).unwrap();
let journal = &expected.journals[0];
assert_eq!(journal.source_session_id, fixture.context.session_id);
assert_eq!(journal.current_position, BEFORE);
assert_eq!(journal.entries[0].state, JournalEntryState::Open);
assert!(!before.selected_ranges.is_empty());
assert_eq!(journal.process_order, before.current_journal.process_order);
assert_eq!(journal.process_byte, before.current_journal.process_byte);
assert_eq!(journal.process_order, 0);
assert_eq!(journal.process_byte, 0, "saved open records do not advance processing checkpoint");
let expected_json = serde_json::to_string(&expected).unwrap();
let adapter = Arc::new(adapter);
let model = MockClient { adapter: adapter.clone(), requests: Arc::default(), updated: Arc::new(AtomicBool::new(false)) };
let workers = StandaloneWorkerStore::open_existing(&fixture.host().catalog.state_root).unwrap();
let worker_id = fixture._allocation.as_ref().unwrap().worker_id();
let store = CombinedStore::new(fixture.sessions.clone(), FsWorkerStore::new(workers.worker_metadata_dir(worker_id)).unwrap());
let mut manifest = manifest::WorkerManifest::from_toml(&format!("[worker]\nname='{}'\n[model]\nscheme='anthropic'\nmodel_id='scripted'\n[engine]\n[scope]\nallow=[]", fixture.context.worker_id)).unwrap();
manifest.feature.subjektiv.profile.enabled = true;
manifest.feature.subjektiv.profile.extraction.enabled = false;
let mut prepared = worker::WorkerBootstrap::new(
manifest,
store,
worker::PromptCatalogSource::builtins_only(),
worker::WorkerWorkspaceContext::no_workspace(),
worker::WorkerFilesystemAuthority::None,
worker::WorkerBootstrapLayout::Direct { runtime_base: fixture._temp.path().join("runtime") },
worker::WorkerControllerTransport::InProcess,
).with_model_client(model.clone()).with_subjektiv_host(fixture.host().clone()).prepare_restored("journal-test").await.unwrap();
assert_eq!(prepared.worker().session_id().to_string(), fixture.context.session_id);
let source_segment = prepared.worker().segment_id();
let original = fixture.sessions.read_all(prepared.worker().session_id(), source_segment).unwrap();
let generation = fixture.host().catalog.store.active_journal_generation(&adapter.context, &adapter.grant.journal_id).unwrap().unwrap();
assert!(!fixture.host().session_fully_reflected(&fixture.context.session_id).unwrap());
prepared.worker_mut().compact(0).await.unwrap();
assert!(model.updated.load(Ordering::SeqCst));
let worker = prepared.worker();
assert_ne!(worker.segment_id(), source_segment);
let seed = worker.store().read_all(worker.session_id(), worker.segment_id()).unwrap().into_iter().find_map(|entry| match entry { LogEntry::AnnotatedSegmentStart { history, .. } => Some(history), _ => None }).expect("durable compacted seed");
let journal_items: Vec<_> = seed.iter().filter(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::ReferenceInput)).collect();
assert_eq!(journal_items.len(), 1);
let journal_item = Item::from(journal_items[0].item.clone());
let body = journal_item.as_text().unwrap();
assert!(body.contains(&expected_json), "durable seed contains the full selected JSON and processing checkpoint");
assert!(body.contains(BEFORE));
assert!(!body.contains(AFTER));
assert!(seed.iter().any(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::OpeningSummary) && Item::from(entry.item.clone()).as_text().is_some_and(|text| text.contains(COMPLEMENT))));
let requests = model.requests.lock().unwrap();
let input = requests[0].items.iter().find_map(|item| match item {
Item::Message { role: agen::llm_client::types::Role::User, .. } => item.as_text().filter(|text| text.contains("## Conversation overview/index")),
_ => None,
}).expect("actual compactor user request");
assert!(input.ends_with(body), "exact same frozen reference goes to actual compactor and seed");
assert_eq!(input.matches(&expected_json).count(), 1);
assert!(!input.contains(AFTER));
drop(requests);
let latest = fixture.host().journal_compaction_context(&fixture.context).unwrap();
assert_eq!(latest.journals[0].current_position, AFTER);
assert_ne!(latest.journals[0].state_token, expected.journals[0].state_token);
assert_eq!(latest.journals[0].process_order, expected.journals[0].process_order);
assert_eq!(latest.journals[0].process_byte, expected.journals[0].process_byte);
assert_eq!(latest.journals[0].entries, expected.journals[0].entries);
assert_eq!(fixture.host().catalog.store.active_journal_generation(&adapter.context, &adapter.grant.journal_id).unwrap(), Some(generation));
assert_eq!(fixture.host().jobs().unwrap().get(&job.request.job_id).unwrap().state, job.state);
let unchanged: Vec<_> = fixture.sessions.read_all(worker.session_id(), source_segment).unwrap().into_iter().filter(|entry| !matches!(entry, LogEntry::Extension { domain, .. } if domain == "metrics")).collect();
assert_eq!(serde_json::to_value(unchanged).unwrap(), serde_json::to_value(original).unwrap());
assert!(!fixture.host().session_fully_reflected(&fixture.context.session_id).unwrap());
}).await.expect("compaction must finish without recording Job completion");
}
@@ -0,0 +1,228 @@
//! Real standalone Subject/Controller/Journal Job/initial input connection. Only external
//! model streams are scripted. The scenario checks persisted data and ordering, not whether
//! a real model understands the correction or can autonomously continue the work.
use super::*;
use crate::{StandaloneHost, StandaloneLaunchConfig, StandaloneSubjectDisconnectError};
use agen::llm_client::types::Item;
fn launch(root: &Path, state: &Path, subject: &str, name: &str) -> crate::ResolvedStandaloneLaunch {
StandaloneLaunchConfig::new(
root,
state,
manifest::ProfileSelector::source_named(
manifest::ProfileRegistrySource::Builtin,
"standalone-subjektiv",
),
name,
)
.with_subject(subject.into())
.resolve()
.unwrap()
}
#[tokio::test]
async fn recorded_idle_body_handoff_preserves_open_after_source_deletion_and_seeds_next_body_once()
{
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("continuity", Some("BEHAVIOR_ORIGINAL_UNCHANGED")).unwrap();
let model = DialogAndJournal { journal: JournalScript { gate_first: true, ..Default::default() }, ..Default::default() };
let mut first = StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "first"), model.clone()).await.unwrap();
let old_worker = first.worker_id();
let old_session = first.record().active_session_id;
let old_writer = catalog.store.claim_journal_writer(&subject.id, &catalog.scope_id, &old_worker.to_string()).unwrap();
submit_dialog(&first, "A was proposed, user corrected its constraint; B selected because A is unusable. One step performed; tests not performed; external outcome Unknown. Work is unfinished.").await;
model.journal.entered.notified().await;
assert!(matches!(first.disconnect_subject().await, Err(StandaloneSubjectDisconnectError::Incomplete(_))));
assert!(catalog.store.journal_writer_active(&old_writer).unwrap());
assert!(StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "premature"), DialogAndJournal::default()).await.is_err());
let context = JournalHostContext::validated_session(&catalog.store, catalog.store.session_attribution(&old_session.to_string()).unwrap().unwrap()).unwrap();
let id = catalog.store.journal_for_session(&context).unwrap().unwrap();
let generation = catalog.store.active_journal_generation(&context, &id).unwrap().unwrap();
model.journal.release.notify_one();
first.jobs().wait(&generation.job.job_id).await.unwrap();
// Observe the authoritative gate after cleanup; yielding is cooperative scheduling,
// never elapsed time used as proof of recording completion.
loop {
match first.disconnect_subject().await {
Ok(()) => break,
Err(StandaloneSubjectDisconnectError::Incomplete(message)) if message.contains("dispatcher") || message.contains("cleanup") => tokio::task::yield_now().await,
Err(error) => panic!("unexpected handoff blocker: {error}"),
}
}
assert!(!catalog.store.journal_writer_active(&old_writer).unwrap());
assert!(catalog.store.accept_journal_sources(&context, "late old activity", &[generation.selected_ranges[0].clone()]).is_err());
first.shutdown().await.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
workers.delete(old_worker).unwrap();
assert!(!workers.sessions_dir(old_worker).exists());
let reader = JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap();
let historical = catalog.store.journal_startup_snapshot(&reader, Some(&old_session.to_string())).unwrap();
assert_eq!(historical.journals[0].entries[0].state, JournalEntryState::Open);
assert!(historical.journals[0].entries[0].ongoing.contains("unknown"));
assert!(catalog.store.list_memories(&subject.id).unwrap().is_empty());
let second_model = DialogAndJournal::default();
let second = StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "second"), second_model.clone()).await.unwrap();
submit_dialog(&second, "Continue B; describe what is still unperformed and unconfirmed.").await;
let requests = second_model.dialog_requests.lock().unwrap();
let initial = &requests[0];
let context_index = initial.items.iter().position(|item| item.as_text().is_some_and(|text| text.contains("Proposed A; user corrected it to B"))).expect("saved Journal visible to first model request");
let user_index = initial.items.iter().position(|item| matches!(item, Item::Message { role: agen::llm_client::types::Role::User, .. })).unwrap();
assert!(context_index < user_index);
assert_eq!(initial.items.iter().filter(|item| item.as_text().is_some_and(|text| text.contains("Proposed A; user corrected it to B"))).count(), 1);
drop(requests);
let session = second.record().active_session_id;
let sessions = session_store::FsStore::new_with_retention(workers.sessions_dir(second.worker_id()), crate::store::session_retention(&state)).unwrap();
let pointer = session_store::FsWorkerStore::new(workers.worker_metadata_dir(second.worker_id())).unwrap().read_by_name(&second.record().storage_key).unwrap().unwrap().active.unwrap();
let entries = sessions.read_all(session, pointer.segment_id.unwrap()).unwrap();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if entry.metadata.input_metadata.as_ref().and_then(|input| input.feature_payload()).is_some_and(|payload| payload.schema() == "journal:startup"))).count();
assert_eq!(journal_items, 1);
assert_eq!(catalog.store.subject(&subject.id).unwrap().unwrap().behavior_md, "BEHAVIOR_ORIGINAL_UNCHANGED");
second.shutdown().await.unwrap();
}).await.expect("bounded complete Subject handoff connection");
}
#[tokio::test]
async fn saved_public_history_without_admission_notification_blocks_release_until_actual_recording()
{
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("saved-before-notified", None).unwrap();
let model = DialogAndJournal {
journal: JournalScript {
gate_first: true,
..Default::default()
},
..Default::default()
};
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "gap"),
model.clone(),
)
.await
.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
let record = host.record().clone();
let writer = catalog
.store
.claim_journal_writer(
&subject.id,
&catalog.scope_id,
&host.worker_id().to_string(),
)
.unwrap();
let sessions = session_store::FsStore::new_with_retention(
workers.sessions_dir(host.worker_id()),
crate::store::session_retention(&state),
)
.unwrap();
// The production saved-history boundary, with its admission callback deliberately
// omitted. The real Host release must discover these bytes, not trust Job status.
sessions
.append(
record.active_session_id,
record.active_segment_id.unwrap(),
&session_store::LogEntry::AnnotatedUserInput {
ts: 3,
segments: vec![],
extensions: vec![],
history: vec![message(
"missed-admission",
"A corrected to B, partially performed, outcome Unknown",
)],
},
)
.unwrap();
assert!(host.disconnect_subject().await.is_err());
assert!(catalog.store.journal_writer_active(&writer).unwrap());
model.journal.entered.notified().await;
let context = JournalHostContext::validated_session(
&catalog.store,
catalog
.store
.session_attribution(&record.active_session_id.to_string())
.unwrap()
.unwrap(),
)
.unwrap();
let id = catalog
.store
.journal_for_session(&context)
.unwrap()
.unwrap();
let generation = catalog
.store
.active_journal_generation(&context, &id)
.unwrap()
.unwrap();
model.journal.release.notify_one();
host.jobs().wait(&generation.job.job_id).await.unwrap();
loop {
match host.disconnect_subject().await {
Ok(()) => break,
Err(StandaloneSubjectDisconnectError::Incomplete(message))
if message.contains("dispatcher") || message.contains("cleanup") =>
{
tokio::task::yield_now().await
}
Err(error) => {
panic!("unexpected release blocker after saved-history recording: {error}")
}
}
}
assert!(!catalog.store.journal_writer_active(&writer).unwrap());
let snapshot = catalog
.store
.journal_startup_snapshot(
&JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap(),
None,
)
.unwrap();
assert!(!snapshot.journals[0].current_position.is_empty());
assert_eq!(
snapshot.journals[0].entries[0].state,
JournalEntryState::Open
);
host.shutdown().await.unwrap();
})
.await
.expect("saved-history/notification-gap handoff is bounded");
}
#[tokio::test]
async fn shutdown_and_normal_deletion_cannot_release_subject_occupancy() {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("retained-owner", None).unwrap();
let host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "retained"),
DialogAndJournal::default(),
)
.await
.unwrap();
let worker = host.worker_id();
host.shutdown().await.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
assert!(workers.delete(worker).is_err());
assert!(
StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "not-an-owner"),
DialogAndJournal::default()
)
.await
.is_err()
);
let mut restored =
StandaloneHost::restore_with_model_client(state, worker, DialogAndJournal::default())
.await
.unwrap();
restored.disconnect_subject().await.unwrap();
restored.shutdown().await.unwrap();
}
@@ -0,0 +1,596 @@
//! Connected standalone Host -> Subjektiv Memory/Journal -> real FsStore coverage.
//! Candidate and Journal decisions are explicit test inputs; no model, Runtime or live DB
//! is used. The parent Journal fixture owns isolated temporary storage and binding cleanup.
//! Selected by the ordinary standalone lib tests, with no additional feature or service.
use super::*;
use memory::{
backend::MemoryStageCandidateOperation,
extract::{CandidateKind, ExtractedCandidate, StagingEvidence},
schema::{EvidenceKind, EvidenceOriginKind, SourceEvidenceRef, SourceRef},
};
use subjektiv::{
CandidateDecision, CandidateDecisionRequest, MemoryChangeTarget, MemoryDraft, MemoryRecord,
StagingResolution, SubjectStagingRecord,
journal::memory_links::{
JournalTargetAvailability, MemoryJournalLinks, MemoryJournalRequest, MemoryJournalResponse,
MemoryJournalSelector, SourceConnection, UnconnectedReason,
},
};
fn adopt_original(fixture: &Fixture) -> (MemoryRecord, SubjectStagingRecord, StagingResolution) {
let evidence_index = session_store::public_index::read_fs_session_evidence_index(
fixture.sessions.root_dir(),
fixture.context.session_id.parse().unwrap(),
session_store::SessionPublicIndexLimits {
max_bytes: 8 * 1024 * 1024,
max_entries: 20_000,
max_segments: 64,
},
)
.unwrap();
let range = *evidence_index
.ranges
.get(&(fixture.segment.to_string(), "Efirst".into()))
.unwrap()
.first()
.unwrap();
let staged = fixture
.host()
.stage_candidate(
&fixture.context,
MemoryStageCandidateOperation {
source: SourceRef {
segment_id: fixture.segment.to_string(),
range,
},
extract_run_id: "explicit-test-extraction-input".into(),
candidate: ExtractedCandidate {
kind: CandidateKind::Preference,
claim: "Keep correction history".into(),
why_useful: "Preserves the reason for later decisions".into(),
staleness: None,
evidence_ids: vec!["Efirst".into()],
},
evidence: vec![StagingEvidence {
id: "Efirst".into(),
kind: EvidenceKind::new(EvidenceKind::MESSAGE),
entry_range: Some(range),
origin: None,
excerpt: None,
summary: None,
}],
source_refs: vec![SourceEvidenceRef {
session_id: Some(fixture.context.session_id.clone()),
segment_id: Some(fixture.segment.to_string()),
entry_range: Some(range),
evidence_id: Some("Efirst".into()),
..Default::default()
}],
},
)
.unwrap();
let store = &fixture.host().catalog.store;
let subject = &fixture.connection.binding.subject_id;
let candidate = store
.staging_candidate(subject, &staged.staging_id)
.unwrap()
.unwrap();
let receipt = store
.decide_candidate(
subject,
CandidateDecisionRequest {
request_id: "explicit-test-adoption-input".into(),
candidate_id: staged.staging_id,
reason: "Adopt the committed original evidence, not the Journal summary".into(),
decision: CandidateDecision::Apply {
target: MemoryChangeTarget::Create,
draft: MemoryDraft::active(
CandidateKind::Preference,
"Keep correction history",
"Preserve earlier decisions and their corrections",
"Preserves the reason for later decisions",
"Adopt original HumanInput",
),
},
},
)
.unwrap();
(receipt.memory.unwrap(), candidate, receipt.resolution)
}
fn create_entry(adapter: &LocalJournalHost, state: JournalEntryState) -> JournalReceipt {
save(
adapter,
"record-original-context",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "A proposal was corrected".into(),
body: "Proposal A changed to B; only part was performed, outcome unconfirmed.".into(),
ongoing: if state == JournalEntryState::Open {
"Remaining work is not complete".into()
} else {
String::new()
},
state,
source_end: (state == JournalEntryState::Fixed).then(|| {
adapter
.generation
.selected_ranges
.last()
.unwrap()
.source
.pointer
.clone()
}),
correction_refs: vec![],
},
adapter.generation.selected_ranges.clone(),
)
}
fn assert_originals_unchanged(
fixture: &Fixture,
memory: &MemoryRecord,
candidate: &SubjectStagingRecord,
resolution: &StagingResolution,
) {
let catalog = StandaloneSubjects::open_existing(&fixture.host().catalog.state_root).unwrap();
let store = &catalog.store;
let subject = &fixture.connection.binding.subject_id;
assert_eq!(
serde_json::to_value(store.memory(subject, &memory.id).unwrap().unwrap()).unwrap(),
serde_json::to_value(memory).unwrap(),
);
assert_eq!(
serde_json::to_value(
store
.staging_candidate(subject, &candidate.id)
.unwrap()
.unwrap()
)
.unwrap(),
serde_json::to_value(candidate).unwrap(),
);
assert_eq!(
serde_json::to_value(
store
.staging_resolution(subject, &candidate.id)
.unwrap()
.unwrap()
)
.unwrap(),
serde_json::to_value(resolution).unwrap(),
);
assert_eq!(store.list_memories(subject).unwrap().len(), 1);
}
fn selector(memory: &MemoryRecord, candidate: &SubjectStagingRecord) -> MemoryJournalSelector {
MemoryJournalSelector {
memory_id: memory.id.clone(),
change_id: memory.change_id.clone(),
candidate_id: candidate.id.clone(),
}
}
fn links(
fixture: &Fixture,
selector: &MemoryJournalSelector,
reconcile: bool,
) -> MemoryJournalLinks {
let request = if reconcile {
MemoryJournalRequest::Reconcile {
selector: selector.clone(),
}
} else {
MemoryJournalRequest::Links {
selector: selector.clone(),
}
};
let MemoryJournalResponse::Links(links) = fixture
.host()
.memory_journal(&fixture.context, request)
.unwrap()
else {
panic!("expected Memory Journal links")
};
links
}
fn read_request(
selector: &MemoryJournalSelector,
journal: &str,
entry: &str,
) -> MemoryJournalRequest {
MemoryJournalRequest::Read {
selector: selector.clone(),
source_index: 0,
journal_id: journal.into(),
entry_id: entry.into(),
expected_token: None,
byte_offset: 0,
max_bytes: MAX_JOURNAL_READ_BYTES,
}
}
fn assert_link(links: &MemoryJournalLinks, journal: &str, entry: &str, reconsideration: bool) {
assert_eq!(links.sources.len(), 1);
assert_eq!(links.sources[0].source_index, 0);
assert_eq!(
links.sources[0].connection,
SourceConnection::Linked {
journal_id: journal.into(),
entry_id: entry.into(),
target: JournalTargetAvailability::Available,
reconsideration,
},
);
}
#[test]
fn saved_link_reads_after_original_worker_and_session_deletion_from_replacement_host() {
let mut fixture = Fixture::new("Please preserve the reasons for my corrected decisions");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
assert_eq!(
candidate.evidence[0].origin.as_ref().unwrap().kind,
EvidenceOriginKind::HumanInput
);
assert_eq!(
candidate.source_refs[0].origin,
candidate.evidence[0].origin
);
assert!(
candidate.evidence[0]
.excerpt
.as_ref()
.unwrap()
.contains("corrected decisions")
);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Fixed);
let entry = recorded.entry_id.clone().unwrap();
let journal = adapter.grant.journal_id.clone();
let connected = links(&fixture, &selector, true);
assert_link(&connected, &journal, &entry, false);
let request = read_request(&selector, &journal, &entry);
let before = fixture
.host()
.memory_journal(&fixture.context, request.clone())
.unwrap();
let positioned = save(
&adapter,
"position-original",
&recorded.state_token,
JournalAction::UpdatePosition {
current_position: "Recorded correction; remaining outcome unconfirmed".into(),
},
vec![],
);
save(
&adapter,
"checkpoint-original",
&positioned.state_token,
JournalAction::Checkpoint {
record_operation_ids: vec![recorded.request_id],
},
adapter.generation.selected_ranges.clone(),
);
adapter.finish().unwrap();
fixture
.host()
.catalog
.store
.close_journal_generation(
&adapter.context,
&adapter.generation.id,
&adapter.generation.job,
)
.unwrap();
let state = fixture.host().catalog.state_root.clone();
let subject = fixture.connection.binding.subject_id.clone();
let original_session = fixture.context.session_id.clone();
// This Host/storage suite has no Controller. Complete the explicit persisted
// writer-release boundary before deletion; the sibling handoff suite exercises
// the production Idle reservation + Host lifecycle that authorizes that release.
fixture
.host()
.catalog
.store
.release_journal_writer(&fixture.host().writer)
.unwrap();
let (worker, workers) = fixture.stop_worker();
workers.delete(worker).unwrap();
assert!(matches!(
workers.load(worker),
Err(crate::store::StandaloneStoreError::WorkerNotFound(_))
));
assert!(!fixture.sessions.root_dir().join(&original_session).exists());
assert!(
!retained_workers_root(&state)
.join(worker.to_string())
.exists()
);
let replacement_worker = protocol::WorkerId::now_v7();
let mut replacement = StandaloneSubjects::open_existing(&state)
.unwrap()
.connect(&subject, replacement_worker)
.unwrap();
let replacement_context = SubjektivHostContext {
worker_id: replacement_worker.to_string(),
session_id: session_store::SessionId::now_v7().to_string(),
};
let MemoryJournalResponse::Links(saved) = replacement
.host
.memory_journal(
&replacement_context,
MemoryJournalRequest::Links {
selector: selector.clone(),
},
)
.unwrap()
else {
panic!("expected saved links")
};
assert_link(&saved, &journal, &entry, false);
let after = replacement
.host
.memory_journal(&replacement_context, request)
.unwrap();
let MemoryJournalResponse::Read(read) = &after else {
panic!("expected saved Journal text")
};
assert!(
read.page
.as_ref()
.unwrap()
.text
.contains("outcome unconfirmed")
);
assert_eq!(
read.source.original_source,
subjektiv::journal::memory_links::OriginalSourceAvailability::Missing
);
// Reading the saved edge does not depend on a live Session fetch or a replacement Job.
let MemoryJournalResponse::Read(before) = before else {
panic!("expected Journal text")
};
assert_eq!(before.page, read.page);
assert!(
replacement.host.jobs().is_err(),
"the reader has no model/Job executor"
);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
replacement.close().unwrap();
}
#[test]
fn late_journal_fix_connects_existing_memory_without_using_checkpoint_as_event_end() {
let fixture = Fixture::new("Keep the reasons behind corrected decisions");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
assert_eq!(
links(&fixture, &selector, false).sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::NotReconciled
}
);
assert_eq!(
links(&fixture, &selector, true).sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::SourceNotAdmitted
}
);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Open);
let entry = recorded.entry_id.clone().unwrap();
let positioned = save(
&adapter,
"position-open",
&recorded.state_token,
JournalAction::UpdatePosition {
current_position: "Work remains open; no fixed endpoint yet".into(),
},
vec![],
);
// Checkpoint a proper prefix, leaving source bytes for the later Fix.
// A fully checkpointed generation cannot admit another activity write.
let mut reflected = adapter.generation.selected_ranges.clone();
reflected.last_mut().unwrap().bytes.end -= 1;
let checkpoint = save(
&adapter,
"checkpoint-open",
&positioned.state_token,
JournalAction::Checkpoint {
record_operation_ids: vec![recorded.request_id],
},
reflected.clone(),
);
let unconnected = links(&fixture, &selector, true);
assert_eq!(
unconnected.sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::JournalNotFixed
}
);
assert_eq!(links(&fixture, &selector, true), unconnected);
let state = adapter.state().unwrap();
let open = state.current_open.unwrap();
assert_eq!(open.id, entry);
assert_eq!(open.state, JournalEntryState::Open);
assert!(open.source_end.is_none());
let last = reflected.last().unwrap();
assert_eq!(state.current_journal.process_order, last.source.order);
assert_eq!(state.current_journal.process_byte, last.bytes.end);
save(
&adapter,
"fix-later",
&checkpoint.state_token,
JournalAction::Fix {
entry_id: entry.clone(),
source_end: adapter
.generation
.selected_ranges
.last()
.unwrap()
.source
.pointer
.clone(),
},
adapter.generation.selected_ranges.clone(),
);
// A plain saved-state read must not silently perform reconciliation.
assert_eq!(links(&fixture, &selector, false), unconnected);
let connected = links(&fixture, &selector, true);
assert_link(&connected, &adapter.grant.journal_id, &entry, false);
assert_eq!(links(&fixture, &selector, true), connected);
assert_eq!(links(&fixture, &selector, false), connected);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
#[test]
fn host_rejects_foreign_subject_worker_context_and_closed_connection_for_all_link_operations() {
let mut fixture = Fixture::new("Keep correction history");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Fixed);
let journal = adapter.grant.journal_id.clone();
let entry = recorded.entry_id.unwrap();
assert_link(&links(&fixture, &selector, true), &journal, &entry, false);
let requests = [
MemoryJournalRequest::Reconcile {
selector: selector.clone(),
},
MemoryJournalRequest::Links {
selector: selector.clone(),
},
read_request(&selector, &journal, &entry),
MemoryJournalRequest::Related {
journal_id: journal,
entry_id: entry,
after: None,
limit: 32,
},
];
let catalog = StandaloneSubjects::open_existing(&fixture.host().catalog.state_root).unwrap();
let foreign_subject = catalog.create("different-reader", None).unwrap();
let foreign_worker = protocol::WorkerId::now_v7();
let mut foreign = catalog
.connect(&foreign_subject.id, foreign_worker)
.unwrap();
let foreign_context = SubjektivHostContext {
worker_id: foreign_worker.to_string(),
session_id: session_store::SessionId::now_v7().to_string(),
};
let wrong_worker = SubjektivHostContext {
worker_id: foreign_worker.to_string(),
..fixture.context.clone()
};
for request in &requests {
assert!(
foreign
.host
.memory_journal(&foreign_context, request.clone())
.is_err()
);
assert!(
fixture
.host()
.memory_journal(&wrong_worker, request.clone())
.is_err()
);
}
foreign.close().unwrap();
fixture.connection.close().unwrap();
for request in requests {
assert!(
fixture
.host()
.memory_journal(&fixture.context, request)
.is_err()
);
}
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
#[test]
fn journal_correction_finds_related_memory_without_revising_or_retargeting_its_evidence() {
let fixture = Fixture::new("Keep correction history");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
fixture.append("correction", "The earlier result was not confirmed");
let (adapter, _, _) = fixture.adapter();
let ranges = &adapter.generation.selected_ranges;
assert_eq!(ranges.len(), 2);
let original = save(
&adapter,
"original-event",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "Original context".into(),
body: "A change was proposed; execution remained uncertain".into(),
ongoing: String::new(),
state: JournalEntryState::Fixed,
source_end: Some(ranges[0].source.pointer.clone()),
correction_refs: vec![],
},
vec![ranges[0].clone()],
);
let entry = original.entry_id.unwrap();
let journal = &adapter.grant.journal_id;
assert_link(&links(&fixture, &selector, true), journal, &entry, false);
let corrected = save(
&adapter,
"correction-event",
&original.state_token,
JournalAction::Create {
title: "Correction".into(),
body: "The earlier result was unconfirmed; reconsider its associated Memory".into(),
ongoing: String::new(),
state: JournalEntryState::Fixed,
source_end: Some(ranges[1].source.pointer.clone()),
correction_refs: vec![entry.clone()],
},
vec![ranges[1].clone()],
);
assert_link(&links(&fixture, &selector, false), journal, &entry, true);
let correction_entry = corrected.entry_id.unwrap();
for target in [&entry, &correction_entry] {
let MemoryJournalResponse::Related(related) = fixture
.host()
.memory_journal(
&fixture.context,
MemoryJournalRequest::Related {
journal_id: journal.clone(),
entry_id: target.clone(),
after: None,
limit: 32,
},
)
.unwrap()
else {
panic!("expected related Memory")
};
assert_eq!(related.items.len(), 1);
assert_eq!(related.items[0].selector, selector);
assert_eq!(related.items[0].source_index, 0);
assert!(related.items[0].reconsideration);
assert!(related.next.is_none());
}
let MemoryJournalResponse::Read(read) = fixture
.host()
.memory_journal(&fixture.context, read_request(&selector, journal, &entry))
.unwrap()
else {
panic!("expected original context")
};
assert!(
read.page
.unwrap()
.text
.contains("execution remained uncertain")
);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
+12 -6
View File
@@ -37,9 +37,8 @@ fn subject_kernel_lease_is_not_worker_or_profile_scoped() {
let catalog = StandaloneSubjects::open(temp.path().join("state")).unwrap();
let a = catalog.create("a", None).unwrap();
let b = catalog.create("b", None).unwrap();
let mut first = catalog
.connect(&a.id, protocol::WorkerId::now_v7())
.unwrap();
let first_id = protocol::WorkerId::now_v7();
let mut first = catalog.connect(&a.id, first_id).unwrap();
assert!(matches!(
StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
@@ -51,11 +50,18 @@ fn subject_kernel_lease_is_not_worker_or_profile_scoped() {
.connect(&b.id, protocol::WorkerId::now_v7())
.unwrap();
first.close().unwrap();
let mut replacement = StandaloneSubjects::open_existing(&temp.path().join("state"))
assert!(
StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&a.id, protocol::WorkerId::now_v7())
.is_err(),
"kernel lease release is not durable body release"
);
let mut resumed = StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&a.id, protocol::WorkerId::now_v7())
.connect(&a.id, first_id)
.unwrap();
replacement.close().unwrap();
resumed.close().unwrap();
second.close().unwrap();
}
+106 -3
View File
@@ -155,6 +155,81 @@ pub(crate) fn authorized_index(
)
}
pub(crate) fn authorized_sessions_path(
state_root: &Path,
scope_id: &str,
subject_id: &str,
store: &SubjektivStore,
session_id: &str,
) -> Result<PathBuf, SubjektivHostError> {
let backend = Backend {
state_root,
scope_id,
subject_id,
store,
reader: session_store::public_index::read_fs_session_public_index,
};
(|| {
backend.check_scope()?;
required_selector(session_id)?;
let attribution = backend.attribution(session_id)?;
Ok(backend.authorize(attribution)?.sessions_path)
})()
.map_err(|error: Failure| SubjektivHostError::Unavailable(error.message))
}
/// Checks only saved ownership and filesystem metadata, never Session bytes or a Worker restore.
/// Inaccessible/corrupt storage stays unknown rather than being reported as absent.
pub(crate) fn original_session_presence(
state_root: &Path,
scope_id: &str,
subject_id: &str,
store: &SubjektivStore,
session_id: &str,
) -> Option<bool> {
let attribution = store.session_attribution(session_id).ok()??;
if store.scope_id() != scope_id
|| attribution.subject_id != subject_id
|| attribution.runtime_id != scope_id
{
return None;
}
let worker: WorkerId = attribution.worker_id.parse().ok()?;
let session: SessionId = session_id.parse().ok()?;
let active = StandaloneWorkerStore::open_existing(state_root).ok()?;
let (workers, record) = match active.load(worker) {
Ok(record) => (active, record),
Err(StandaloneStoreError::WorkerNotFound(_)) => {
let root = state_root.join("subjektiv").join("retained-workers");
match std::fs::metadata(&root) {
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Some(false),
Err(_) => return None,
Ok(_) => {}
}
let retained = StandaloneWorkerStore::open_existing(root).ok()?;
match retained.load(worker) {
Ok(record) => (retained, record),
Err(StandaloneStoreError::WorkerNotFound(_)) => return Some(false),
Err(_) => return None,
}
}
Err(_) => return None,
};
if record.worker_id != worker
|| !record
.subject
.as_ref()
.is_some_and(|binding| binding.scope_id == scope_id && binding.subject_id == subject_id)
{
return None;
}
match std::fs::metadata(workers.sessions_dir(worker).join(session.to_string())) {
Ok(metadata) if metadata.is_dir() => Some(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Some(false),
_ => None,
}
}
fn authorized_index_with_reader(
state_root: &Path,
scope_id: &str,
@@ -221,13 +296,28 @@ impl Backend<'_> {
"Attributed standalone Worker state is missing",
));
}
let workers = StandaloneWorkerStore::open_existing(self.state_root).map_err(|_| {
let mut workers = StandaloneWorkerStore::open_existing(self.state_root).map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RuntimeUnavailable,
"Standalone Worker storage is inaccessible",
)
})?;
let record = workers.load(id).map_err(|error| match error {
let record = match workers.load(id) {
Err(StandaloneStoreError::WorkerNotFound(_)) => {
workers = StandaloneWorkerStore::open_existing(
self.state_root.join("subjektiv").join("retained-workers"),
)
.map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Original Session is not retained",
)
})?;
workers.load(id)
}
result => result,
}
.map_err(|error| match error {
StandaloneStoreError::WorkerNotFound(_) => failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Attributed standalone Worker record is missing",
@@ -262,6 +352,13 @@ impl Backend<'_> {
"Attributed Session identity is invalid",
)
})?;
let retention = crate::store::session_retention(self.state_root);
let _guard = retention.lock_live(id).map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Original Session is erased or its retention fence is unavailable",
)
})?;
let index = (self.reader)(&source.sessions_path, id, limits).map_err(index_failure)?;
if index.session_id != source.attribution.session_id {
return Err(failure(
@@ -1524,6 +1621,7 @@ allow = []
let history = |id: &str, item: LoggedItem| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction { operation_id: None },
derivation: None,
@@ -1539,7 +1637,11 @@ allow = []
)
};
let sessions = f.index_path.parent().unwrap();
let fs_store = session_store::FsStore::new(sessions).unwrap();
let fs_store = session_store::FsStore::new_with_retention(
sessions,
crate::store::session_retention(&f.temp.path().join("workers")),
)
.unwrap();
let segment: SessionId = f.segment.parse().unwrap();
fs_store
.create_segment(
@@ -1592,6 +1694,7 @@ allow = []
// A valid JSON record without its final newline is not committed.
log.write_all(
&serde_json::to_vec(&LogEntry::RunCompleted {
entry_id: None,
ts: 4,
interrupted: false,
result: agen::EngineResult::Finished,
+151 -6
View File
@@ -1,5 +1,7 @@
//! T-709: real standalone Hosts and test-owned processes, with scripted models.
//! These tests do not create a Workspace or contact a model provider.
//! Integration of real standalone Subject lifecycle, dialog, Journal and Memory services.
//! Only model streams are scripted; temporary storage and test-owned subprocesses isolate
//! persistence and kernel leases. Run with `cargo test -p standalone --test subjektiv`.
//! No Workspace or live model provider is used; this is not a continuity meaning evaluation.
use std::collections::{BTreeMap, VecDeque};
use std::io::{Read, Write};
@@ -354,6 +356,17 @@ impl LlmClient for ScriptedClient {
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<LlmEvent, ClientError>> + Send>>, ClientError>
{
// Journal is a separate scripted model boundary. Successful production recording
// is required before these Memory scenarios may hand the Subject to a new body.
if request
.tools
.iter()
.any(|tool| tool.name == "ReadJournalState")
{
return Ok(Box::pin(stream::iter(
journal_response(&request).into_iter().map(Ok),
)));
}
// Lifecycle-triggered consolidation uses a separate Internal Worker. It
// must not consume the dialog script or silently become a live request.
let job = request
@@ -411,6 +424,134 @@ impl LlmClient for ScriptedClient {
}
}
// The outer model script is not a continuity meaning evaluation. Persistence, complete
// source delivery and the handoff gate remain the production implementations.
fn journal_response(request: &Request) -> Vec<LlmEvent> {
let result = |id: &str| -> Option<Value> {
request.items.iter().find_map(|item| match item {
Item::ToolResult {
call_id,
content,
is_error,
summary,
..
} if call_id == id => {
assert!(!is_error, "{id}: {summary}; {content:?}");
Some(
content
.as_deref()
.map(|v| serde_json::from_str(v).unwrap())
.unwrap_or(Value::Null),
)
}
_ => None,
})
};
if result("journal-submit").is_some() {
return prose();
}
if result("journal-checkpoint").is_some() {
return tool("journal-submit", "SubmitJobResult", json!({"result":{}}));
}
let Some(state) = result("journal-state") else {
return tool("journal-state", "ReadJournalState", json!({}));
};
let state = &state["value"];
let generation = state["generation_id"].as_str().unwrap();
if let Some(position) = result("journal-position") {
return tool(
"journal-checkpoint",
"CheckpointJournal",
json!({
"request_id":format!("checkpoint:{generation}"), "expected_state":position["value"]["state_token"],
"action":{"action":"checkpoint","record_operation_ids":[format!("record:{generation}")]},
"covered_ranges":state["selected_ranges"]
}),
);
}
if let Some(record) = result("journal-record") {
return tool(
"journal-position",
"UpdateJournalPosition",
json!({
"request_id":format!("position:{generation}"), "expected_state":record["value"]["state_token"],
"action":{"action":"update_position","current_position":"Memory scenario recorded; represented work may remain unfinished"}, "covered_ranges":[]
}),
);
}
for (index, range) in state["selected_ranges"]
.as_array()
.unwrap()
.iter()
.enumerate()
{
let mut offset = range["bytes"]["start"].as_u64().unwrap();
let mut token = Value::Null;
loop {
let id = format!("journal-source-{index}-{offset}");
match result(&id) {
Some(page) => match page["value"]["next_byte"].as_u64() {
Some(next) => {
assert!(next > offset);
offset = next;
token = page["value"]["content_token"].clone();
}
None => break,
},
None => {
return tool(
&id,
"ReadJournalText",
json!({
"target":{"target":"source","pointer":range["source"]["pointer"]},
"offset":offset,"limit":subjektiv::journal::MAX_JOURNAL_READ_BYTES,"content_token":token
}),
);
}
}
}
}
let (name, action) = if state["current_open"].is_null() {
(
"CreateJournalEntry",
json!({"action":"create","title":"Memory conversation","body":"Conversation recorded by the scripted model fixture","ongoing":"Unfinished work is not declared complete","state":"open","source_end":null,"correction_refs":[]}),
)
} else {
(
"RewriteJournalEntry",
json!({"action":"rewrite","entry_id":state["current_open"]["id"],"title":"Memory conversation","body":"Further conversation recorded by the scripted model fixture","ongoing":"Unfinished work is not declared complete"}),
)
};
tool(
"journal-record",
name,
json!({
"request_id":format!("record:{generation}"), "expected_state":state["current_journal"]["state_token"],
"action":action,"covered_ranges":state["selected_ranges"]
}),
)
}
async fn disconnect_recorded(host: &mut StandaloneHost) {
let mut last_error = None;
let completed = tokio::time::timeout(Duration::from_secs(30), async {
loop {
match host.disconnect_subject().await {
Ok(()) => return,
Err(error) => {
last_error = Some(error.to_string());
tokio::task::yield_now().await;
}
}
}
})
.await;
assert!(
completed.is_ok(),
"actual Idle/Journal completion is required before Memory scenario handoff: {last_error:?}"
);
}
fn prose() -> Vec<LlmEvent> {
vec![
LlmEvent::text_block_start(0),
@@ -750,7 +891,7 @@ async fn explicit_remember_is_committed_staging_and_sessions_follow_subject_acro
tool("query-restored", "SubjektivMemoryQuery", json!({})),
prose(),
]);
let restored =
let mut restored =
StandaloneHost::restore_with_model_client(state.clone(), worker_id, restored_model.clone())
.await
.unwrap();
@@ -761,6 +902,7 @@ async fn explicit_remember_is_committed_staging_and_sessions_follow_subject_acro
assert!(results["recall"].to_string().contains(claim));
assert_eq!(results["query-restored"]["items"], json!([]));
assert!(format!("{:?}", restored_model.requests()[0].items).contains("Remember this lesson"));
disconnect_recorded(&mut restored).await;
restored.shutdown().await.unwrap();
let second_model = ScriptedClient::new(vec![
@@ -980,7 +1122,7 @@ async fn kernel_subject_lease_excludes_other_processes_and_recovers_after_abnorm
let independent = catalog.create("independent", None).unwrap();
drop(catalog);
let ready = temp.path().join("ready");
let parent = StandaloneHost::start_with_model_client(
let mut parent = StandaloneHost::start_with_model_client(
launch(&parent_cwd, &state, &subject.id, "parent-owner"),
ScriptedClient::new(Vec::new()),
)
@@ -1002,6 +1144,7 @@ async fn kernel_subject_lease_excludes_other_processes_and_recovers_after_abnorm
.await
.unwrap();
independent_host.shutdown().await.unwrap();
disconnect_recorded(&mut parent).await;
parent.shutdown().await.unwrap();
let mut owner = OwnedChild(
@@ -1070,7 +1213,7 @@ async fn automatic_generic_consolidation_applies_candidate_publishes_surface_and
drop(catalog);
let claim = "Validate the immutable result before reporting success";
let model = ScriptedClient::consolidating(remember_script(claim), false);
let host = StandaloneHost::start_with_model_client(
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "consolidation-body"),
model.clone(),
)
@@ -1145,6 +1288,7 @@ async fn automatic_generic_consolidation_applies_candidate_publishes_surface_and
.exists(),
"selected Subject owns the shared generic Job ledger"
);
disconnect_recorded(&mut host).await;
host.shutdown().await.unwrap();
let memory = inspect_store(&state, |store| {
assert!(
@@ -1224,7 +1368,7 @@ async fn transport_loss_after_candidate_apply_preserves_memory_and_later_job_onl
drop(catalog);
let claim = "Candidate receipts survive transport failure";
let losing = ScriptedClient::consolidating(remember_script(claim), true);
let host = StandaloneHost::start_with_model_client(
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "losing-body"),
losing.clone(),
)
@@ -1249,6 +1393,7 @@ async fn transport_loss_after_candidate_apply_preserves_memory_and_later_job_onl
failed.result.is_none(),
"partial apply is not an accepted Job result"
);
disconnect_recorded(&mut host).await;
host.shutdown().await.unwrap();
let original = inspect_store(&state, |store| {
assert!(
+4 -1
View File
@@ -1,7 +1,9 @@
# subjektiv
Shared Host domain implementation for per-Subject Memory, change history, candidate dispositions, and derived surfaces.
Shared Host domain implementation for per-Subject Memory, change history, candidate dispositions, derived surfaces, and Session activity Journals.
`feature-storage` manages physical SQLite storage; the calling Host handles authentication and execution authority.
Journal storage does not generate prose, launch Jobs, or fetch Session history.
Hosts retain body ownership across inactivity, reserve actual execution quiescence, and reconcile final committed history before releasing a Journal writer lifetime. [`journal::binding`] fences new mutations inside storage transactions; historical attribution and exact receipt recovery cannot renew a released lifetime. [`journal::startup`] selects bounded reference context independently of source retention and long-term Memory.
## Main entry points
@@ -9,3 +11,4 @@ Shared Host domain implementation for per-Subject Memory, change history, candid
- [`MemoryRecord`], [`SurfaceSnapshot`]: change history and derived context
- [`api`]: shared Host operations and bounded responses
- [`job`]: consolidation selection rules and result validation
- [`journal`]: individually committed activity operations, source coverage, retry receipts, and bounded reading
+12 -2
View File
@@ -11,8 +11,10 @@
mod context;
mod operation;
pub use context::{HostOperationContext, JobAttemptBinding};
pub use operation::execute;
/// Shared Journal domain operations; no model tools or Host-specific transport.
pub use crate::journal;
pub use context::{HostOperationContext, JobAttemptBinding, JournalHostContext, JournalJobContext};
pub use operation::{checkpoint_journal, execute, execute_journal};
use serde::{Deserialize, Serialize};
use std::collections::HashSet;
@@ -1297,6 +1299,14 @@ pub type OperationResult<T> = std::result::Result<T, OperationError>;
pub fn subjektiv_store_error(error: crate::SubjektivError) -> OperationError {
match error {
crate::SubjektivError::AuthorizationDenied(reason) => {
OperationError::PermissionDenied(reason)
}
crate::SubjektivError::SubjectPermissionsConflict(_)
| crate::SubjektivError::PermissionOperationConflict(_)
| crate::SubjektivError::WorkerExecutionGrantConflict(_) => {
OperationError::Conflict(error.to_string())
}
crate::SubjektivError::SubjectScopeMismatch { .. } => {
OperationError::PermissionDenied(format!("subject_scope_mismatch: {error}"))
}
+2 -1
View File
@@ -1,8 +1,9 @@
//! Trusted Host operation context, never deserialized from model input.
use super::{OperationError, OperationResult, subjektiv_store_error};
pub use crate::journal::{JournalHostContext, JournalJobContext};
use crate::{SubjectSessionAttribution, SubjektivStore};
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct JobAttemptBinding {
pub job_id: String,
pub attempt_id: String,
+18
View File
@@ -1,4 +1,22 @@
use super::*;
/// Commits one Journal effect with its exact retry response, using Journal-only authority.
pub fn execute_journal(
store: &crate::SubjektivStore,
context: &JournalJobContext,
operation: journal::JournalOperation,
) -> journal::JournalResult<journal::JournalReceipt> {
store.execute_journal(context, operation)
}
/// Advances only selected input backed by already committed Journal effects.
pub fn checkpoint_journal(
store: &crate::SubjektivStore,
context: &JournalJobContext,
operation: journal::JournalOperation,
) -> journal::JournalResult<journal::JournalReceipt> {
store.checkpoint_journal(context, operation)
}
use server_api::{
SubjektivMemoryBackendOperation as Op, SubjektivMemoryBackendResponse as Response,
};
+305
View File
@@ -0,0 +1,305 @@
//! Durable Journal writer lifetime, issued only from the Host's body binding. Kernel leases
//! and Runtime liveness do not end ownership. The immutable binding ID fences retained contexts
//! even when the same Worker later reconnects. Hosts serialize release with body lifecycle and
//! reconcile final committed history before calling release; this store checks saved effects.
use super::*;
use rusqlite::{Connection, OptionalExtension, params};
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalWriterBinding {
pub subject_id: String,
pub runtime_id: String,
pub worker_id: String,
pub binding_id: String,
}
pub(crate) fn migrate(tx: &Transaction<'_>) -> Result<(), FeatureStorageError> {
tx.execute_batch("CREATE TABLE journal_writer_bindings (
binding_id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
runtime_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
active INTEGER NOT NULL CHECK(active IN (0,1))
);
CREATE UNIQUE INDEX journal_one_body_binding ON journal_writer_bindings(subject_id) WHERE active=1;
CREATE TABLE journal_session_bindings (
session_id TEXT PRIMARY KEY REFERENCES subject_session_attributions(session_id) ON DELETE RESTRICT,
binding_id TEXT NOT NULL REFERENCES journal_writer_bindings(binding_id) ON DELETE RESTRICT
);
CREATE INDEX journal_body_history ON journal_writer_bindings(subject_id,active);
CREATE INDEX journal_body_sessions ON journal_session_bindings(binding_id);
CREATE INDEX journal_recent_subject ON journals(subject_id,updated_at DESC,id DESC);")?;
Ok(())
}
fn denied() -> JournalError {
JournalError::PermissionDenied("Journal writer binding is absent, released or replaced".into())
}
pub(super) fn check_writer(tx: &Connection, host: &JournalHostContext) -> JournalResult<()> {
let attr = host.attribution.as_ref().ok_or_else(denied)?;
let binding = host.writer_binding_id.as_ref().ok_or_else(denied)?;
if !tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings b JOIN journal_session_bindings s ON s.binding_id=b.binding_id WHERE b.binding_id=?1 AND b.subject_id=?2 AND b.runtime_id=?3 AND b.worker_id=?4 AND b.active=1 AND s.session_id=?5)", params![binding,host.subject_id,attr.runtime_id,attr.worker_id,attr.session_id], |r| r.get::<_,bool>(0))? {
return Err(denied());
}
Ok(())
}
pub(super) fn require_saved_position(tx: &Connection, generation: &str) -> JournalResult<()> {
let saved: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_operations p WHERE p.generation_id=?1 AND json_extract(p.request_json,'$.action.action')='update_position' AND p.rowid > COALESCE((SELECT MAX(r.rowid) FROM journal_operations r WHERE r.generation_id=?1 AND json_extract(r.request_json,'$.action.action') IN ('create','rewrite','fix','confirm_unchanged')),0))",[generation],|r|r.get(0))?;
if !saved {
return Err(JournalError::Conflict("Journal current position must be saved after the generation's activity records before completion".into()));
}
Ok(())
}
impl SubjektivStore {
/// The last released body's most recently updated Journal Session, independent of source
/// retention. Session attachment order may differ from the order of recorded activity.
pub fn journal_predecessor_session(
&self,
binding: &JournalWriterBinding,
) -> JournalResult<Option<String>> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT j.source_session_id FROM journals j JOIN journal_session_bindings s ON s.session_id=j.source_session_id WHERE j.subject_id=?1 AND s.binding_id=(SELECT binding_id FROM journal_writer_bindings WHERE subject_id=?1 AND active=0 ORDER BY rowid DESC LIMIT 1) ORDER BY j.updated_at DESC,j.id DESC LIMIT 1", [&binding.subject_id], |r| r.get(0)).optional()?)
})
}
/// Absence proof for failed body creation, including bindings never attached to a Session
/// and released lifetimes. The Host must hold its lifecycle fence across this observation
/// and recovery; absence alone does not fence a delayed Runtime create.
pub fn journal_body_is_unpublished(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row(
"SELECT NOT EXISTS(SELECT 1 FROM journal_writer_bindings
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)
AND NOT EXISTS(SELECT 1 FROM subject_session_attributions
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)",
params![subject, runtime, worker],
|row| row.get(0),
)?)
})
}
pub fn journal_worker_was_released(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3 AND active=0)", params![subject,runtime,worker], |r| r.get(0))?)
})
}
pub fn journal_writer_active(&self, binding: &JournalWriterBinding) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE binding_id=?1 AND subject_id=?2 AND runtime_id=?3 AND worker_id=?4 AND active=1)", params![binding.binding_id,binding.subject_id,binding.runtime_id,binding.worker_id], |r| r.get(0))?)
})
}
pub fn journal_session_writer_matches(
&self,
binding: &JournalWriterBinding,
session: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_session_bindings WHERE binding_id=?1 AND session_id=?2)", params![binding.binding_id,session], |r| r.get(0))?)
})
}
/// Called under the existing Host Subject lifecycle authority. A stopped or unavailable
/// owner remains the owner; only the exact same body may resume an unreleased binding.
pub fn claim_journal_writer(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<JournalWriterBinding> {
for identifier in [subject, runtime, worker] {
if identifier.is_empty()
|| identifier.len() > 128
|| identifier.trim() != identifier
|| identifier.chars().any(char::is_control)
{
return Err(JournalError::InvalidInput(
"invalid Journal binding identity".into(),
));
}
}
self.database.try_transaction(|tx| {
let state: Option<String> = tx.query_row("SELECT state FROM subjects WHERE subject_id=?1", [subject], |r| r.get(0)).optional()?;
if state.as_deref() != Some("active") { return Err(JournalError::PermissionDenied("Subject is absent or retired".into())); }
let current: Option<JournalWriterBinding> = tx.query_row("SELECT subject_id,runtime_id,worker_id,binding_id FROM journal_writer_bindings WHERE subject_id=?1 AND active=1", [subject], |r| Ok(JournalWriterBinding { subject_id:r.get(0)?,runtime_id:r.get(1)?,worker_id:r.get(2)?,binding_id:r.get(3)? })).optional()?;
if let Some(current) = current {
if current.runtime_id != runtime || current.worker_id != worker {
return Err(JournalError::Conflict("Subject is still occupied; explicit Idle and Journal-complete release required".into()));
}
return Ok(current);
}
// Upgraded T-734 data can contain a removed body's durable recording handoff.
// Retention is not permission for a different body to occupy that Subject.
let legacy_busy: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journals j JOIN subject_session_attributions a ON a.session_id=j.source_session_id WHERE j.subject_id=?1 AND (a.runtime_id<>?2 OR a.worker_id<>?3) AND NOT EXISTS(SELECT 1 FROM journal_session_bindings b WHERE b.session_id=j.source_session_id) AND (EXISTS(SELECT 1 FROM journal_generations g WHERE g.journal_id=j.id AND g.active=1) OR EXISTS(SELECT 1 FROM journal_sources s,json_each(s.accepted_json) r WHERE s.journal_id=j.id AND json_extract(r.value,'$.end')>s.reflected_bytes)))", params![subject,runtime,worker], |r| r.get(0))?;
if legacy_busy { return Err(JournalError::Conflict("previous body's retained Journal handoff is not complete".into())); }
let binding = JournalWriterBinding { subject_id:subject.into(),runtime_id:runtime.into(),worker_id:worker.into(),binding_id:crate::issued_id("journal-binding") };
tx.execute("INSERT INTO journal_writer_bindings(binding_id,subject_id,runtime_id,worker_id,active) VALUES(?1,?2,?3,?4,1)", params![binding.binding_id,subject,runtime,worker])?;
Ok(binding)
})
}
/// Binds a new Session once. Reconnecting a Worker under a later binding cannot acquire
/// mutation authority for its old Session. Attribution and binding are Host-only input.
pub fn bind_journal_session(
&self,
binding: &JournalWriterBinding,
attribution: SubjectSessionAttribution,
) -> JournalResult<JournalHostContext> {
let mut host = JournalHostContext::validated_session(self, attribution)?;
self.database.try_transaction(|tx| {
let attr = host.attribution.as_ref().expect("validated Session");
if binding.subject_id != attr.subject_id || binding.runtime_id != attr.runtime_id || binding.worker_id != attr.worker_id {
return Err(denied());
}
tx.execute("INSERT OR IGNORE INTO journal_session_bindings(session_id,binding_id) SELECT ?1,binding_id FROM journal_writer_bindings WHERE binding_id=?2 AND subject_id=?3 AND runtime_id=?4 AND worker_id=?5 AND active=1",params![attr.session_id,binding.binding_id,attr.subject_id,attr.runtime_id,attr.worker_id])?;
host.writer_binding_id = Some(binding.binding_id.clone());
check_writer(tx,&host)?;
Ok(host.clone())
})
}
/// Destructive release is distinct from a continuity handoff: every attributed Session
/// of this body must already have a permanent erasure tombstone. This also records a
/// released lifetime for pre-binding bodies, without granting them new mutation authority.
pub fn release_erased_journal_body(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_transaction(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
let remaining: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM subject_session_attributions a WHERE a.subject_id=?1 AND a.runtime_id=?2 AND a.worker_id=?3 AND NOT EXISTS(SELECT 1 FROM journal_erased_sessions e WHERE e.session_id=a.session_id AND e.subject_id=a.subject_id))", params![host.subject_id,attr.runtime_id,attr.worker_id], |r| r.get(0))?;
if remaining {
return Ok(false);
}
let existing: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)", params![host.subject_id,attr.runtime_id,attr.worker_id], |r| r.get(0))?;
tx.execute("UPDATE journal_writer_bindings SET active=0 WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3",params![host.subject_id,attr.runtime_id,attr.worker_id])?;
if !existing {
tx.execute("INSERT INTO journal_writer_bindings(binding_id,subject_id,runtime_id,worker_id,active) VALUES(?1,?2,?3,?4,0)", params![crate::issued_id("journal-erased-body"),host.subject_id,attr.runtime_id,attr.worker_id])?;
}
Ok(true)
})
}
/// The Host must hold its lifecycle fence, confirm Idle, reconcile all committed original
/// Sessions (including the save/notification gap), and exclude queued/running/Unknown Jobs.
/// This transaction rejects retained writers, pending source bytes and active generations.
/// It never fixes an open entry or declares the represented work complete.
pub fn release_journal_writer(&self, binding: &JournalWriterBinding) -> JournalResult<()> {
self.database.try_transaction(|tx| {
let active: Option<bool> = tx.query_row("SELECT active FROM journal_writer_bindings WHERE binding_id=?1 AND subject_id=?2 AND runtime_id=?3 AND worker_id=?4",params![binding.binding_id,binding.subject_id,binding.runtime_id,binding.worker_id],|r|r.get(0)).optional()?;
let Some(active) = active else { return Err(denied()); };
if !active { return Ok(()); }
let unbound: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM subject_session_attributions a WHERE a.subject_id=?1 AND a.runtime_id=?2 AND a.worker_id=?3 AND NOT EXISTS(SELECT 1 FROM journal_session_bindings s WHERE s.session_id=a.session_id) AND NOT EXISTS(SELECT 1 FROM journal_erased_sessions e WHERE e.session_id=a.session_id AND e.subject_id=a.subject_id))",params![binding.subject_id,binding.runtime_id,binding.worker_id],|r|r.get(0))?;
if unbound { return Err(JournalError::Conflict("body has legacy Sessions whose committed history has not been reconciled under its writer binding".into())); }
let busy: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_generations g JOIN journals j ON j.id=g.journal_id JOIN journal_session_bindings s ON s.session_id=j.source_session_id WHERE s.binding_id=?1 AND g.active=1)",[&binding.binding_id],|r|r.get(0))?;
if busy { return Err(JournalError::Conflict("Journal generation remains active or its result is unconfirmed".into())); }
let mut stmt = tx.prepare("SELECT s.accepted_json,s.reflected_bytes,s.unavailable_json FROM journal_sources s JOIN journals j ON j.id=s.journal_id JOIN journal_session_bindings b ON b.session_id=j.source_session_id WHERE b.binding_id=?1")?;
let mut rows = stmt.query([&binding.binding_id])?;
while let Some(row) = rows.next()? {
let accepted: Vec<JournalByteRange> = serde_json::from_str(&row.get::<_,String>(0)?)?;
let reflected: u64 = row.get(1)?;
if accepted.iter().any(|r| r.end > reflected) {
return Err(JournalError::Conflict("Journal activity/position/checkpoint has unreflected source bytes".into()));
}
}
let mut checkpoints = tx.prepare("SELECT p.generation_id FROM journal_operations p JOIN journals j ON j.id=p.journal_id JOIN journal_session_bindings b ON b.session_id=j.source_session_id WHERE b.binding_id=?1 AND p.rowid=(SELECT MAX(c.rowid) FROM journal_operations c WHERE c.journal_id=j.id AND json_extract(c.request_json,'$.action.action')='checkpoint')")?;
let generations = checkpoints.query_map([&binding.binding_id], |r| r.get::<_,String>(0))?;
for generation in generations { require_saved_position(tx, &generation?)?; }
tx.execute("UPDATE journal_writer_bindings SET active=0 WHERE binding_id=?1 AND active=1",[&binding.binding_id])?;
Ok(())
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn unpublished_body_rejects_orphan_and_released_bindings_and_attribution_only() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _workspace, store) = crate::tests::open_store(temp.path(), "absence-tests");
let subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
let other_subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
// No Session was attached: an orphan binding is still published body evidence.
let binding = store
.claim_journal_writer(&subject.id, "runtime", "worker")
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
for (subject_id, runtime, worker) in [
(other_subject.id.as_str(), "runtime", "worker"),
(subject.id.as_str(), "other-runtime", "worker"),
(subject.id.as_str(), "runtime", "other-worker"),
] {
assert!(
store
.journal_body_is_unpublished(subject_id, runtime, worker)
.unwrap()
);
}
store.release_journal_writer(&binding).unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
store
.record_session_attribution(
SubjectSessionAttribution::new(
&subject.id,
"runtime",
"attribution-only",
"session",
)
.unwrap(),
)
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "other-runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&other_subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "never-published")
.unwrap()
);
}
}
+181
View File
@@ -0,0 +1,181 @@
//! Bounded saved-Journal browser projections. No history fetch, reconciliation or execution.
use super::store::{check_host, entry, fit_page, journal, summary};
use super::*;
use rusqlite::params;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct EntryCursor {
scope: String,
subject: String,
journal: Option<String>,
query: Option<String>,
correcting: Option<String>,
created_at: String,
id: String,
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
impl SubjektivStore {
pub fn journal_entry_summary(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
) -> JournalResult<JournalEntrySummary> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
summary(tx, entry(tx, journal_id, entry_id)?)
})
}
/// Correction targets are capped at sixteen by the writer contract, and may be in
/// another Journal of this Subject. Include actual Journal IDs rather than guessing.
pub fn journal_correction_targets(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
) -> JournalResult<Vec<JournalEntrySummary>> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
entry(tx, journal_id, entry_id)?;
let mut stmt = tx.prepare(
"SELECT e.journal_id,e.id FROM journal_corrections c
JOIN journal_entries e ON e.subject_id=c.subject_id AND e.id=c.target_id
WHERE c.subject_id=?1 AND c.entry_id=?2 ORDER BY e.created_at,e.id LIMIT 16",
)?;
let rows = stmt
.query_map(params![host.subject_id, entry_id], |r| {
Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?))
})?
.collect::<Result<Vec<_>, _>>()?;
rows.into_iter()
.map(|(journal_id, id)| summary(tx, entry(tx, &journal_id, &id)?))
.collect()
})
}
/// Recent entries use immutable creation time and ID as a stable descending keyset.
/// Search is literal substring matching (not SQL LIKE wildcards), bounded to 256 bytes.
/// A correcting selector filters saved correction edges, not inferred text or intervals.
pub fn browse_journal_entries(
&self,
host: &JournalHostContext,
journal_id: Option<&str>,
query: Option<&str>,
correcting: Option<&str>,
after: Option<&str>,
limit: usize,
) -> JournalResult<JournalPage<JournalEntrySummary>> {
if !(1..=MAX_JOURNAL_PAGE).contains(&limit) {
return Err(invalid("page limit must be 1..32"));
}
if query.is_some_and(|q| q.len() > 256 || q.chars().any(char::is_control)) {
return Err(invalid(
"search query must be at most 256 bytes without controls",
));
}
if correcting.is_some() && journal_id.is_none() {
return Err(invalid("correction lookup requires a Journal"));
}
let cursor: Option<EntryCursor> = after
.map(|raw| {
if raw.len() > 4096 {
return Err(invalid("entry cursor exceeds byte limit"));
}
let cursor: EntryCursor =
serde_json::from_str(raw).map_err(|_| invalid("invalid entry cursor"))?;
if cursor.scope != host.scope_id
|| cursor.subject != host.subject_id
|| cursor.journal.as_deref() != journal_id
|| cursor.query.as_deref() != query
|| cursor.correcting.as_deref() != correcting
{
return Err(invalid(
"entry cursor belongs to another scope, Subject or filter",
));
}
Ok(cursor)
})
.transpose()?;
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
if let Some(id) = journal_id {
journal(tx, host, id, false)?;
}
if let Some(id) = correcting {
entry(tx, journal_id.unwrap(), id)?;
}
let mut stmt = tx.prepare(
"SELECT e.journal_id,e.id,e.created_at FROM journal_entries e
WHERE e.subject_id=?1 AND (?4 IS NOT NULL OR ?2 IS NULL OR e.journal_id=?2)
AND (?3 IS NULL OR instr(lower(e.title),lower(?3))>0
OR instr(lower(e.body),lower(?3))>0 OR instr(lower(e.ongoing),lower(?3))>0)
AND (?4 IS NULL OR EXISTS(SELECT 1 FROM journal_corrections c
WHERE c.subject_id=?1 AND c.entry_id=e.id AND c.target_id=?4))
AND (?5 IS NULL OR (e.created_at,e.id)<(?5,?6))
ORDER BY e.created_at DESC,e.id DESC LIMIT ?7",
)?;
let rows = stmt
.query_map(
params![
host.subject_id,
journal_id,
query,
correcting,
cursor.as_ref().map(|c| &c.created_at),
cursor.as_ref().map(|c| &c.id),
limit + 1
],
|r| {
Ok((
r.get::<_, String>(0)?,
r.get::<_, String>(1)?,
r.get::<_, String>(2)?,
))
},
)?
.collect::<Result<Vec<_>, _>>()?;
let more = rows.len() > limit;
let mut items = Vec::new();
for (journal_id_value, id, created_at) in rows.into_iter().take(limit) {
let next = serde_json::to_string(&EntryCursor {
scope: host.scope_id.clone(),
subject: host.subject_id.clone(),
journal: journal_id.map(str::to_owned),
query: query.map(str::to_owned),
correcting: correcting.map(str::to_owned),
created_at,
id: id.clone(),
})?;
items.push((next, summary(tx, entry(tx, &journal_id_value, &id)?)?));
}
fit_page(items, more)
})
}
/// True only for admitted source bytes which the durable checkpoint has not reflected.
/// False does not claim that all original Session activity was admitted or recorded.
pub fn journal_has_unreflected_sources(
&self,
host: &JournalHostContext,
journal_id: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
Ok(tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_sources s, json_each(s.accepted_json) b
WHERE s.journal_id=?1 AND json_extract(b.value,'$.end')>s.reflected_bytes)",
[journal_id],
|r| r.get(0),
)?)
})
}
}
+153
View File
@@ -0,0 +1,153 @@
//! Journal-specific retention and explicit erasure. Hosts reconcile the final committed public
//! Session before consulting retention; absence from the source ledger is not proof of no work.
//! Ordinary retention never deletes Journal text. Explicit erasure leaves a content-free Session
//! tombstone in the same transaction as removal, fencing delayed admission and Job operations.
use super::*;
use crate::now;
use rusqlite::{Connection, OptionalExtension, params};
pub(crate) fn migrate(tx: &Transaction<'_>) -> Result<(), FeatureStorageError> {
tx.execute_batch(
"ALTER TABLE journal_generations ADD COLUMN completion_json TEXT;
CREATE TABLE journal_generation_deliveries (
generation_id TEXT NOT NULL REFERENCES journal_generations(id) ON DELETE CASCADE,
source_order INTEGER NOT NULL,
ranges_json TEXT NOT NULL,
PRIMARY KEY(generation_id, source_order)
);
CREATE TABLE journal_erased_sessions (
session_id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL,
request_id TEXT NOT NULL UNIQUE,
erased_at TEXT NOT NULL
);
DROP TRIGGER journal_operations_no_delete;
CREATE TRIGGER journal_operations_no_delete BEFORE DELETE ON journal_operations
WHEN NOT EXISTS (
SELECT 1 FROM journals j JOIN journal_erased_sessions e
ON e.session_id=j.source_session_id AND e.subject_id=j.subject_id
WHERE j.id=OLD.journal_id
)
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are retained'); END;",
)?;
Ok(())
}
pub(super) fn reject_erased(tx: &Connection, host: &JournalHostContext) -> JournalResult<()> {
let session = host
.attribution
.as_ref()
.ok_or_else(|| JournalError::PermissionDenied("Session authority required".into()))?;
if tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1)",
[&session.session_id],
|row| row.get::<_, bool>(0),
)? {
return Err(JournalError::PermissionDenied(
"Session was explicitly erased".into(),
));
}
Ok(())
}
impl SubjektivStore {
/// Reads the content-free erasure fence under the original Session attribution.
/// Hosts use this before source reconciliation so ordinary removal of an erased Session
/// neither fetches erased material nor forces a new recording generation.
pub fn journal_session_erased(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
Ok(tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1 AND subject_id=?2)",
params![attr.session_id, host.subject_id],
|row| row.get(0),
)?)
})
}
/// Finds this Host's immutable source Session without scanning unrelated Journals.
pub fn journal_for_session(&self, host: &JournalHostContext) -> JournalResult<Option<String>> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
reject_erased(tx, host)?;
let attr = host
.attribution
.as_ref()
.expect("checked Session authority");
Ok(tx
.query_row(
"SELECT id FROM journals WHERE source_session_id=?1 AND subject_id=?2",
params![attr.session_id, host.subject_id],
|row| row.get(0),
)
.optional()?)
})
}
/// True while any admitted bytes still need reflection, including unavailable bytes.
/// A Host must first reconcile storage against admission under its deletion/append fence.
pub fn journal_retention_required(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
if tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1)", [&attr.session_id], |r| r.get::<_,bool>(0))? {
return Ok(false);
}
let mut stmt = tx.prepare("SELECT s.accepted_json,s.reflected_bytes FROM journal_sources s JOIN journals j ON j.id=s.journal_id WHERE j.source_session_id=?1 AND j.subject_id=?2 ORDER BY s.source_order")?;
let mut rows = stmt.query(params![attr.session_id,host.subject_id])?;
while let Some(row) = rows.next()? {
let accepted: Vec<JournalByteRange> = serde_json::from_str(&row.get::<_, String>(0)?)?;
let reflected: u64 = row.get(1)?;
if accepted.iter().any(|range| range.end > reflected) {
return Ok(true);
}
}
Ok(false)
})
}
/// Explicit Host-authorized complete erasure, not an ordinary Worker removal policy.
/// No generation is forced. The tombstone remains even if no Journal was admitted yet.
/// Exact retries converge; a request ID cannot be reused to erase a different Session.
pub fn erase_journal_session(
&self,
host: &JournalHostContext,
request_id: &str,
) -> JournalResult<()> {
if request_id.is_empty()
|| request_id.len() > 128
|| request_id.chars().any(char::is_control)
{
return Err(JournalError::InvalidInput(
"invalid erasure request identifier".into(),
));
}
self.database.try_transaction(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
let previous: Option<(String,String)> = tx.query_row("SELECT session_id,subject_id FROM journal_erased_sessions WHERE request_id=?1", [request_id], |r| Ok((r.get(0)?,r.get(1)?))).optional()?;
if previous.as_ref().is_some_and(|(session,subject)| session!=&attr.session_id || subject!=&host.subject_id) {
return Err(JournalError::Conflict("erasure request belongs to a different Session".into()));
}
let erased_by: Option<String> = tx.query_row("SELECT request_id FROM journal_erased_sessions WHERE session_id=?1", [&attr.session_id], |row| row.get(0)).optional()?;
if erased_by.as_deref().is_some_and(|key| key != request_id) {
return Err(JournalError::Conflict("Session already erased by a different request".into()));
}
tx.execute("INSERT OR IGNORE INTO journal_erased_sessions(session_id,subject_id,request_id,erased_at) VALUES(?1,?2,?3,?4)", params![attr.session_id,host.subject_id,request_id,now()])?;
let journal: Option<String> = tx.query_row("SELECT id FROM journals WHERE source_session_id=?1 AND subject_id=?2",params![attr.session_id,host.subject_id],|r|r.get(0)).optional()?;
if let Some(id) = journal {
// The erasure marker is the only condition permitting receipt deletion. Normal
// record/checkpoint/retention paths retain their immutable receipts.
tx.execute("DELETE FROM journal_operations WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_generations WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_corrections WHERE entry_id IN (SELECT id FROM journal_entries WHERE journal_id=?1) OR target_id IN (SELECT id FROM journal_entries WHERE journal_id=?1)",[&id])?;
tx.execute("DELETE FROM journal_entries WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_sources WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journals WHERE id=?1",[&id])?;
}
Ok(())
})
}
}
@@ -0,0 +1,668 @@
//! Additive, immutable provenance-to-context links, not additional evidence for a Memory claim.
//!
//! Reconciliation reads only saved candidate anchors and the Host-attested Journal source ledger.
//! It never fetches history, starts Workers, changes extraction/consolidation, fixes an open event,
//! or mutates Memory. A stable evidence reference identifies a public entry, not its legacy numeric
//! display range. Missing stable identity stays unconnected; numeric ranges are not guessed to be
//! canonical order. Saved edges remain usable without the original runtime and cannot be retargeted.
//! Journal content is never copied here. Explicit erasure retains edges and exposes the tombstone.
use super::{
JournalError, JournalHostContext, JournalResult, JournalSourcePointer, JournalTextField,
JournalTextPage, MAX_JOURNAL_PAGE, MAX_JOURNAL_RESPONSE_BYTES,
};
use crate::{MAX_STAGING_ANCHORS, SubjectStagingRecord, SubjektivStore};
use memory::schema::SourceEvidenceRef;
use rusqlite::{Connection, OptionalExtension, Transaction, params};
use serde::{Deserialize, Serialize};
#[cfg(test)]
mod tests;
pub(crate) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("schema.sql"))?;
Ok(())
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalSelector {
pub memory_id: String,
pub change_id: String,
pub candidate_id: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case", deny_unknown_fields)]
pub enum MemoryJournalRequest {
Reconcile {
selector: MemoryJournalSelector,
},
/// Reads saved link state only; does not implicitly retry reconciliation.
Links {
selector: MemoryJournalSelector,
},
/// Continues reconciliation over immutable source indices, at most MAX_STAGING_ANCHORS per call.
ReconcilePage {
selector: MemoryJournalSelector,
source_offset: usize,
limit: usize,
},
/// Reads another saved source page without reconciliation.
LinksPage {
selector: MemoryJournalSelector,
source_offset: usize,
limit: usize,
},
Read {
selector: MemoryJournalSelector,
source_index: usize,
journal_id: String,
entry_id: String,
expected_token: Option<String>,
byte_offset: usize,
max_bytes: usize,
},
Related {
journal_id: String,
entry_id: String,
after: Option<RelatedMemoryCursor>,
limit: usize,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(
tag = "kind",
content = "value",
rename_all = "snake_case",
deny_unknown_fields
)]
pub enum MemoryJournalResponse {
Links(MemoryJournalLinks),
Read(MemoryJournalRead),
Related(RelatedMemories),
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum OriginalSourceAvailability {
/// No saved Host attestation. None of these values imply a live history fetch.
Unknown,
Missing,
Available,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalTargetAvailability {
Available,
Unavailable,
Erased,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum UnconnectedReason {
NotReconciled,
InsufficientSource,
SourceNotAdmitted,
AmbiguousSource,
JournalNotFixed,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "state", rename_all = "snake_case", deny_unknown_fields)]
pub enum SourceConnection {
Unconnected {
reason: UnconnectedReason,
},
Linked {
journal_id: String,
entry_id: String,
target: JournalTargetAvailability,
/// A saved correction invites reconsideration, never an automatic Memory edit.
reconsideration: bool,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalSourceLink {
pub source_index: usize,
pub original_source: OriginalSourceAvailability,
pub connection: SourceConnection,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalLinks {
pub selector: MemoryJournalSelector,
pub sources: Vec<MemoryJournalSourceLink>,
pub total_sources: usize,
pub next_source_index: Option<usize>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalRead {
pub source: MemoryJournalSourceLink,
/// Absent for an unavailable or explicitly erased target; never falls back to another entry.
pub page: Option<JournalTextPage>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemoryCursor {
pub memory_id: String,
pub change_id: String,
pub candidate_id: String,
pub source_index: usize,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemory {
pub selector: MemoryJournalSelector,
pub source_index: usize,
pub reconsideration: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemories {
pub items: Vec<RelatedMemory>,
pub next: Option<RelatedMemoryCursor>,
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
fn identifier(value: &str) -> JournalResult<()> {
if value.is_empty() || value.len() > 256 || value.chars().any(char::is_control) {
return Err(invalid("invalid Memory/Journal selector identifier"));
}
Ok(())
}
fn selector_valid(selector: &MemoryJournalSelector) -> JournalResult<()> {
identifier(&selector.memory_id)?;
identifier(&selector.change_id)?;
identifier(&selector.candidate_id)
}
fn candidate(
store: &SubjektivStore,
tx: &Connection,
host: &JournalHostContext,
selector: &MemoryJournalSelector,
) -> JournalResult<SubjectStagingRecord> {
super::store::check_host(store, tx, host, false)?;
selector_valid(selector)?;
// The immutable change-candidate edge, not the current Memory projection or a supplied
// candidate ID alone, authorizes this provenance selector.
let raw: Option<String> = tx
.query_row(
"SELECT s.record_json FROM staging_records s JOIN memory_change_candidates c
ON c.subject_id=s.subject_id AND c.candidate_id=s.candidate_id
WHERE c.subject_id=?1 AND c.memory_id=?2 AND c.change_id=?3 AND c.candidate_id=?4",
params![
host.subject_id,
selector.memory_id,
selector.change_id,
selector.candidate_id
],
|r| r.get(0),
)
.optional()?;
let record: SubjectStagingRecord = serde_json::from_str(
&raw.ok_or_else(|| JournalError::NotFound("Memory change candidate membership".into()))?,
)?;
Ok(record)
}
#[derive(Debug)]
struct SavedLink {
reason: String,
journal_id: Option<String>,
entry_id: Option<String>,
session_id: Option<String>,
order: Option<u64>,
}
fn saved(
tx: &Connection,
host: &JournalHostContext,
candidate_id: &str,
index: usize,
) -> JournalResult<Option<SavedLink>> {
Ok(tx.query_row(
"SELECT reason,journal_id,entry_id,source_session_id,source_order FROM memory_journal_links
WHERE subject_id=?1 AND candidate_id=?2 AND source_index=?3",
params![host.subject_id, candidate_id, index],
|r| Ok(SavedLink { reason:r.get(0)?,journal_id:r.get(1)?,entry_id:r.get(2)?,session_id:r.get(3)?,order:r.get(4)? }),
).optional()?)
}
fn original_availability(
tx: &Connection,
journal: &str,
order: u64,
) -> JournalResult<OriginalSourceAvailability> {
let unavailable: Option<String> = tx
.query_row(
"SELECT unavailable_json FROM journal_sources WHERE journal_id=?1 AND source_order=?2",
params![journal, order],
|r| r.get(0),
)
.optional()?;
Ok(match unavailable {
None => OriginalSourceAvailability::Unknown,
Some(raw) => {
let ranges: Vec<super::JournalByteRange> = serde_json::from_str(&raw)?;
if ranges.is_empty() {
OriginalSourceAvailability::Available
} else {
OriginalSourceAvailability::Missing
}
}
})
}
fn project(
tx: &Connection,
host: &JournalHostContext,
candidate_id: &str,
index: usize,
) -> JournalResult<MemoryJournalSourceLink> {
let Some(link) = saved(tx, host, candidate_id, index)? else {
return Ok(MemoryJournalSourceLink {
source_index: index,
original_source: OriginalSourceAvailability::Unknown,
connection: SourceConnection::Unconnected {
reason: UnconnectedReason::NotReconciled,
},
});
};
if link.reason != "linked" {
let reason = match link.reason.as_str() {
"insufficient_source" => UnconnectedReason::InsufficientSource,
"source_not_admitted" => UnconnectedReason::SourceNotAdmitted,
"ambiguous_source" => UnconnectedReason::AmbiguousSource,
"journal_not_fixed" => UnconnectedReason::JournalNotFixed,
_ => return Err(invalid("invalid saved connection reason")),
};
return Ok(MemoryJournalSourceLink {
source_index: index,
original_source: OriginalSourceAvailability::Unknown,
connection: SourceConnection::Unconnected { reason },
});
}
let journal_id = link
.journal_id
.ok_or_else(|| invalid("saved link lacks Journal"))?;
let entry_id = link
.entry_id
.ok_or_else(|| invalid("saved link lacks entry"))?;
let session = link
.session_id
.ok_or_else(|| invalid("saved link lacks Session"))?;
let order = link
.order
.ok_or_else(|| invalid("saved link lacks source order"))?;
let erased: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE subject_id=?1 AND session_id=?2)",
params![host.subject_id, session], |r| r.get(0),
)?;
let exists: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_entries e JOIN journals j ON j.id=e.journal_id
WHERE e.subject_id=?1 AND e.journal_id=?2 AND e.id=?3 AND j.source_session_id=?4)",
params![host.subject_id, journal_id, entry_id, session],
|r| r.get(0),
)?;
let reconsideration = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_corrections WHERE subject_id=?1 AND target_id=?2)",
params![host.subject_id, entry_id],
|r| r.get(0),
)?;
Ok(MemoryJournalSourceLink {
source_index: index,
original_source: original_availability(tx, &journal_id, order)?,
connection: SourceConnection::Linked {
journal_id,
entry_id,
target: if erased {
JournalTargetAvailability::Erased
} else if exists {
JournalTargetAvailability::Available
} else {
JournalTargetAvailability::Unavailable
},
reconsideration,
},
})
}
struct Resolved {
journal: String,
entry: String,
session: String,
order: u64,
}
fn resolve(
tx: &Connection,
host: &JournalHostContext,
source: &SourceEvidenceRef,
) -> JournalResult<Result<Resolved, &'static str>> {
let Some(evidence) = source.evidence_id.as_deref() else {
return Ok(Err("insufficient_source"));
};
if !evidence.starts_with('E')
|| evidence.len() < 2
|| evidence.len() > 129
|| !evidence[1..]
.bytes()
.all(|c| c.is_ascii_alphanumeric() || c == b'-')
|| (source.session_id.is_none() && source.segment_id.is_none())
{
return Ok(Err("insufficient_source"));
}
// Segment identity may be omitted only when an immutable Session attribution disambiguates it.
// A candidate's primary legacy source range is not substituted for an individual anchor.
let segment = match source.segment_id.as_deref() {
Some(raw) => match uuid::Uuid::parse_str(raw) {
Ok(id) => Some(id),
Err(_) => return Ok(Err("insufficient_source")),
},
None => None,
};
let mut stmt = tx.prepare(
"SELECT j.id,j.source_session_id,s.source_order,s.pointer_json FROM journal_sources s
JOIN journals j ON j.id=s.journal_id
JOIN subject_session_attributions a ON a.subject_id=j.subject_id AND a.session_id=j.source_session_id
WHERE j.subject_id=?1 AND s.entry_ref=?2
AND (?3 IS NULL OR j.source_session_id=?3)
AND (?4 IS NULL OR json_extract(s.pointer_json,'$.segment_id')=?4)
LIMIT 2",
)?;
let found = stmt
.query_map(
params![
host.subject_id,
evidence,
source.session_id,
segment.map(|s| s.to_string())
],
|r| {
Ok((
r.get::<_, String>(0)?,
r.get::<_, String>(1)?,
r.get::<_, u64>(2)?,
r.get::<_, String>(3)?,
))
},
)?
.collect::<Result<Vec<_>, _>>()?;
if found.is_empty() {
return Ok(Err("source_not_admitted"));
}
if found.len() != 1 {
return Ok(Err("ambiguous_source"));
}
let (journal, session, order, pointer) = &found[0];
let pointer: JournalSourcePointer = serde_json::from_str(pointer)?;
if pointer.entry_ref != evidence || segment.is_some_and(|s| s != pointer.segment_id) {
return Ok(Err("ambiguous_source"));
}
// A fixed endpoint covers (previous_fixed_end, source_end]. The Session start is the first
// interval's lower boundary. Processing checkpoints and open entries cannot supply an end.
let mut stmt = tx.prepare(
"SELECT e.id FROM journal_entries e WHERE e.subject_id=?1 AND e.journal_id=?2
AND e.state='fixed' AND e.source_order>=?3
AND NOT EXISTS(SELECT 1 FROM journal_entries p WHERE p.journal_id=e.journal_id
AND p.position<e.position AND p.state='fixed' AND p.source_order>=?3)
ORDER BY e.position LIMIT 2",
)?;
let entries = stmt
.query_map(params![host.subject_id, journal, order], |r| {
r.get::<_, String>(0)
})?
.collect::<Result<Vec<_>, _>>()?;
match entries.as_slice() {
[entry] => Ok(Ok(Resolved {
journal: journal.clone(),
entry: entry.clone(),
session: session.clone(),
order: *order,
})),
[] => Ok(Err("journal_not_fixed")),
_ => Ok(Err("ambiguous_source")),
}
}
fn reconcile(
tx: &Connection,
host: &JournalHostContext,
record: &SubjectStagingRecord,
indices: std::ops::Range<usize>,
) -> JournalResult<()> {
for index in indices {
let source = &record.source_refs[index];
if saved(tx, host, &record.id, index)?.is_some_and(|s| s.reason == "linked") {
continue;
}
let (reason, resolved) = match resolve(tx, host, source)? {
Ok(link) => ("linked", Some(link)),
Err(reason) => (reason, None),
};
tx.execute(
"INSERT INTO memory_journal_links(subject_id,candidate_id,source_index,reason,
journal_id,entry_id,source_session_id,source_order) VALUES(?1,?2,?3,?4,?5,?6,?7,?8)
ON CONFLICT(subject_id,candidate_id,source_index) DO UPDATE SET
reason=excluded.reason,journal_id=excluded.journal_id,entry_id=excluded.entry_id,
source_session_id=excluded.source_session_id,source_order=excluded.source_order",
params![
host.subject_id,
record.id,
index,
reason,
resolved.as_ref().map(|r| r.journal.as_str()),
resolved.as_ref().map(|r| r.entry.as_str()),
resolved.as_ref().map(|r| r.session.as_str()),
resolved.as_ref().map(|r| r.order)
],
)?;
}
Ok(())
}
impl SubjektivStore {
/// Scoped Memory provenance bridge. Reconcile is additive and idempotent, so a lost response
/// needs no operation ledger: retry the same selector and reread saved edges. Reads never reconcile.
pub fn execute_memory_journal(
&self,
host: &JournalHostContext,
request: MemoryJournalRequest,
) -> JournalResult<MemoryJournalResponse> {
let response = match request {
MemoryJournalRequest::Reconcile { selector } => {
self.memory_journal_links(host, selector, true, 0, MAX_STAGING_ANCHORS)?
}
MemoryJournalRequest::Links { selector } => {
self.memory_journal_links(host, selector, false, 0, MAX_STAGING_ANCHORS)?
}
MemoryJournalRequest::ReconcilePage {
selector,
source_offset,
limit,
} => self.memory_journal_links(host, selector, true, source_offset, limit)?,
MemoryJournalRequest::LinksPage {
selector,
source_offset,
limit,
} => self.memory_journal_links(host, selector, false, source_offset, limit)?,
MemoryJournalRequest::Read {
selector,
source_index,
journal_id,
entry_id,
expected_token,
byte_offset,
max_bytes,
} => {
identifier(&journal_id)?;
identifier(&entry_id)?;
if !(4..=super::MAX_JOURNAL_READ_BYTES).contains(&max_bytes) {
return Err(invalid("text page must be 4..16384 bytes"));
}
if let Some(token) = expected_token.as_deref() {
identifier(token)?;
}
let mut source = self.database.try_with_connection(|tx| {
let record = candidate(self, tx, host, &selector)?;
if source_index >= record.source_refs.len() {
return Err(invalid("source index outside candidate anchors"));
}
let link = project(tx, host, &selector.candidate_id, source_index)?;
match &link.connection {
SourceConnection::Linked {
journal_id: saved_journal,
entry_id: saved_entry,
..
} if saved_journal == &journal_id && saved_entry == &entry_id => Ok(link),
_ => Err(JournalError::PermissionDenied(
"read requires the exact saved Journal link".into(),
)),
}
})?;
let page = match &source.connection {
SourceConnection::Linked {
target: JournalTargetAvailability::Available,
..
} => {
match self.read_journal_text(
host,
&journal_id,
Some(&entry_id),
JournalTextField::Body,
expected_token.as_deref(),
byte_offset,
max_bytes,
) {
Ok(page) => Some(page),
// Erasure between the saved-edge check and shared text read must not
// fall back, expose cached text, or pretend the link never existed.
Err(JournalError::NotFound(_)) => {
source = self.database.try_with_connection(|tx| {
project(tx, host, &selector.candidate_id, source_index)
})?;
None
}
Err(error) => return Err(error),
}
}
_ => None,
};
MemoryJournalResponse::Read(MemoryJournalRead { source, page })
}
MemoryJournalRequest::Related {
journal_id,
entry_id,
after,
limit,
} => self.related_memories(host, &journal_id, &entry_id, after.as_ref(), limit)?,
};
if serde_json::to_string_pretty(&response)?.len() > MAX_JOURNAL_RESPONSE_BYTES {
return Err(invalid("Memory Journal response exceeds byte budget"));
}
Ok(response)
}
fn memory_journal_links(
&self,
host: &JournalHostContext,
selector: MemoryJournalSelector,
do_reconcile: bool,
source_offset: usize,
limit: usize,
) -> JournalResult<MemoryJournalResponse> {
if !(1..=MAX_STAGING_ANCHORS).contains(&limit) {
return Err(invalid("source page exceeds staging anchor bound"));
}
self.database.try_transaction(|tx| {
let record = candidate(self, tx, host, &selector)?;
let total_sources = record.source_refs.len();
if source_offset > total_sources {
return Err(invalid("source offset outside immutable candidate anchors"));
}
let end = source_offset.saturating_add(limit).min(total_sources);
if do_reconcile {
reconcile(tx, host, &record, source_offset..end)?;
}
let sources = (source_offset..end)
.map(|index| project(tx, host, &selector.candidate_id, index))
.collect::<JournalResult<Vec<_>>>()?;
Ok(MemoryJournalResponse::Links(MemoryJournalLinks {
selector,
sources,
total_sources,
next_source_index: (end < total_sources).then_some(end),
}))
})
}
fn related_memories(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
after: Option<&RelatedMemoryCursor>,
limit: usize,
) -> JournalResult<MemoryJournalResponse> {
identifier(journal_id)?;
identifier(entry_id)?;
if !(1..=MAX_JOURNAL_PAGE).contains(&limit) {
return Err(invalid("related page must be 1..32"));
}
if let Some(cursor) = after {
identifier(&cursor.memory_id)?;
identifier(&cursor.change_id)?;
identifier(&cursor.candidate_id)?;
if i64::try_from(cursor.source_index).is_err() {
return Err(invalid("invalid related source index"));
}
}
self.database.try_with_connection(|tx| {
super::store::check_host(self,tx,host,false)?;
// Saved target identity authorizes reverse lookup after erasure as well as before it.
let known:bool=tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_entries WHERE subject_id=?1 AND journal_id=?2 AND id=?3)
OR EXISTS(SELECT 1 FROM memory_journal_links WHERE subject_id=?1 AND journal_id=?2 AND entry_id=?3)",
params![host.subject_id,journal_id,entry_id],|r|r.get(0))?;
if !known { return Err(JournalError::NotFound("Journal entry".into())); }
let mut stmt=tx.prepare(
"SELECT m.memory_id,m.change_id,l.candidate_id,l.source_index,
EXISTS(SELECT 1 FROM journal_corrections c WHERE c.subject_id=l.subject_id AND c.target_id=l.entry_id)
FROM memory_journal_links l JOIN memory_change_candidates m
ON m.subject_id=l.subject_id AND m.candidate_id=l.candidate_id
WHERE l.subject_id=?1 AND l.reason='linked'
AND ((l.journal_id=?2 AND l.entry_id=?3)
OR EXISTS(SELECT 1 FROM journal_corrections c WHERE c.subject_id=?1 AND c.entry_id=?3 AND c.target_id=l.entry_id))
AND (?4 IS NULL OR (m.memory_id,m.change_id,l.candidate_id,l.source_index)>(?4,?5,?6,?7))
ORDER BY m.memory_id,m.change_id,l.candidate_id,l.source_index LIMIT ?8")?;
let mut items=stmt.query_map(params![host.subject_id,journal_id,entry_id,
after.map(|c|c.memory_id.as_str()),after.map(|c|c.change_id.as_str()),
after.map(|c|c.candidate_id.as_str()),after.map(|c|c.source_index),limit+1],
|r|Ok(RelatedMemory {selector:MemoryJournalSelector {memory_id:r.get(0)?,change_id:r.get(1)?,candidate_id:r.get(2)?},
source_index:r.get(3)?,reconsideration:r.get(4)?}))?.collect::<Result<Vec<_>,_>>()?;
let more=items.len()>limit;
items.truncate(limit);
let next=if more {items.last().map(|last|RelatedMemoryCursor {
memory_id:last.selector.memory_id.clone(),change_id:last.selector.change_id.clone(),
candidate_id:last.selector.candidate_id.clone(),source_index:last.source_index })} else {None};
Ok(MemoryJournalResponse::Related(RelatedMemories {items,next}))
})
}
}
@@ -0,0 +1,33 @@
-- References outlive explicit Journal erasure. Do not add target-row foreign keys:
-- lifecycle erasure deletes Journal content but retains its Session tombstone.
CREATE TABLE memory_journal_links (
subject_id TEXT NOT NULL,
candidate_id TEXT NOT NULL,
source_index INTEGER NOT NULL CHECK(source_index >= 0),
reason TEXT NOT NULL CHECK(reason IN (
'insufficient_source', 'source_not_admitted', 'ambiguous_source',
'journal_not_fixed', 'linked'
)),
journal_id TEXT,
entry_id TEXT,
source_session_id TEXT,
source_order INTEGER,
PRIMARY KEY(subject_id, candidate_id, source_index),
FOREIGN KEY(subject_id, candidate_id)
REFERENCES staging_records(subject_id, candidate_id) ON DELETE RESTRICT,
CHECK((reason = 'linked' AND journal_id IS NOT NULL AND entry_id IS NOT NULL
AND source_session_id IS NOT NULL AND source_order IS NOT NULL AND source_order >= 0)
OR (reason <> 'linked' AND journal_id IS NULL AND entry_id IS NULL
AND source_session_id IS NULL AND source_order IS NULL))
);
CREATE INDEX memory_journal_links_reverse
ON memory_journal_links(subject_id, journal_id, entry_id, candidate_id, source_index);
CREATE TRIGGER memory_journal_links_identity BEFORE UPDATE ON memory_journal_links
WHEN OLD.subject_id <> NEW.subject_id OR OLD.candidate_id <> NEW.candidate_id
OR OLD.source_index <> NEW.source_index
BEGIN SELECT RAISE(ABORT, 'Journal source reference identity is immutable'); END;
CREATE TRIGGER memory_journal_links_connected_immutable BEFORE UPDATE ON memory_journal_links
WHEN OLD.reason = 'linked'
BEGIN SELECT RAISE(ABORT, 'connected Journal references are immutable'); END;
CREATE TRIGGER memory_journal_links_retained BEFORE DELETE ON memory_journal_links
BEGIN SELECT RAISE(ABORT, 'Journal references are retained'); END;
File diff suppressed because it is too large Load Diff
+415
View File
@@ -0,0 +1,415 @@
//! Durable activity records owned by Subjektiv, independent of Session retention.
//!
//! Hosts attest committed public source identity and canonical Session order, not models. Byte
//! ranges are half-open in the Host's canonical UTF-8 public-entry serialization (including
//! framing, so even an empty message has a nonzero length). Compaction copies must retain the
//! original pointer/order/digest; derived summaries are not new sources. Rewind replacement is
//! an identity conflict, never an implicit reset. This module does not fetch or retain history.
//!
//! Each record operation and its exact retry receipt commit together. A separate checkpoint
//! advances only a contiguous, selected prefix backed by saved record/explicit no-change
//! receipts. It cannot use a position update as evidence. Stopping between these boundaries
//! leaves saved records and unreflected input; read receipts before retrying, even across reopen
//! or replacement Jobs. No whole-Job result submission is required. The Host checks live Job
//! execution leases before issuing contexts, and durably closes a generation on cancellation.
//!
//! Selected ranges describe concrete public bytes supplied to generation, not a preview's
//! enclosing history range or a permission to claim unobserved bytes. The Host verifies canonical
//! serialization, SHA-256 identity and UTF-8 slice boundaries before admission/selection. This
//! store validates identities, byte bounds and coverage but cannot attest text it does not fetch.
//! Availability is the latest Host attestation; readers must not equate it with a live fetch.
//! Public text and metadata reads are paged. Full generation/resume snapshots are Host-only
//! construction material bounded by per-field storage caps (three text fields plus current position),
//! not model-facing reads. A restored snapshot never restores an external Job lease.
//!
//! A T-734 caller saves `Create`/`Rewrite`/`Fix` under stable request IDs, carries each returned
//! `state_token` into the next operation, optionally saves `UpdatePosition`, then submits
//! `Checkpoint { record_operation_ids }` for just the reflected prefix. After a lost response it
//! queries the original receipt, status and coverage pages before submitting any new create.
//! On restart it reads the active generation and current resume state, verifies the external Job
//! binding, and continues missing operations; cancellation closes the generation before replacement.
//! A replacement may checkpoint old saved record receipts only for identical selected source bytes.
//! Tokens incorporate prior saved content conditions as well as current content, preventing A→B→A
//! edits from resurrecting an old operation premise after intervening saved effects.
//! Source-ledger pagination uses an opaque, scope/Subject/Journal-bound content cursor: late gap
//! admission, accepted-fragment extension, reflection, or availability changes reject continuation.
//! The fingerprint and bounded projection share one transaction; hashing streams bounded rows.
//! Hosts resuming at the processing cursor can use a bounded source point lookup instead of
//! traversing already reflected pages. Receipt/entry keysets remain append-stable.
pub mod binding;
pub use binding::JournalWriterBinding;
pub(crate) use binding::migrate as migrate_binding;
pub mod browser;
mod lifecycle;
pub mod memory_links;
pub(crate) use lifecycle::migrate as migrate_lifecycle;
pub mod recording;
pub mod service;
mod source_status;
pub mod startup;
mod store;
#[cfg(test)]
mod tests;
pub mod transport;
use crate::{SubjectSessionAttribution, SubjektivStore, api::JobAttemptBinding};
use feature_storage::FeatureStorageError;
use rusqlite::Transaction;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
pub type JournalId = String;
pub type JournalEntryId = String;
pub type JournalResult<T> = Result<T, JournalError>;
pub const MAX_JOURNAL_TEXT_BYTES: usize = 1024 * 1024;
pub const MAX_JOURNAL_RANGES: usize = 64;
pub const MAX_JOURNAL_PAGE: usize = 32;
pub const MAX_JOURNAL_READ_BYTES: usize = 16 * 1024;
/// Whole pretty-serialized public page budget, including escaped text and metadata.
pub const MAX_JOURNAL_RESPONSE_BYTES: usize = 64 * 1024;
/// Stable owner of Backend source retention, shared by attribution-time protection
/// and later source admission/release. This identifier confers no read authority.
/// Local standalone storage uses its scope-exclusive Session UUID as the owner.
pub fn source_retention_pin(workspace: &str, subject: &str, session: &str) -> Uuid {
use sha2::{Digest, Sha256};
let digest = Sha256::digest(
serde_json::to_vec(&(
"subjektiv-journal-retention-v1",
workspace,
subject,
session,
))
.expect("string tuple serialization"),
);
let mut bytes = [0; 16];
bytes.copy_from_slice(&digest[..16]);
Uuid::from_bytes(bytes)
}
#[derive(Debug, thiserror::Error)]
pub enum JournalError {
#[error("journal permission denied: {0}")]
PermissionDenied(String),
#[error("journal conflict: {0}")]
Conflict(String),
#[error("invalid journal input: {0}")]
InvalidInput(String),
#[error("journal record not found: {0}")]
NotFound(String),
#[error(transparent)]
Storage(#[from] FeatureStorageError),
#[error(transparent)]
Json(#[from] serde_json::Error),
#[error(transparent)]
Subject(#[from] crate::SubjektivError),
}
impl From<rusqlite::Error> for JournalError {
fn from(e: rusqlite::Error) -> Self {
Self::Storage(e.into())
}
}
/// Host-issued in-process authority, never a model request DTO. Reader contexts need no live Session.
#[derive(Debug, Clone)]
pub struct JournalHostContext {
scope_id: String,
subject_id: String,
attribution: Option<SubjectSessionAttribution>,
writer_binding_id: Option<String>,
}
impl JournalHostContext {
/// Evidence for a retained Job grant, not permission to mint or renew a binding.
pub fn writer_binding_id(&self) -> Option<&str> {
self.writer_binding_id.as_deref()
}
pub fn validated_reader(store: &SubjektivStore, subject_id: &str) -> JournalResult<Self> {
store
.subject(subject_id)?
.ok_or_else(|| JournalError::NotFound(subject_id.into()))?;
Ok(Self {
scope_id: store.scope_id().into(),
subject_id: subject_id.into(),
attribution: None,
writer_binding_id: None,
})
}
/// External authentication, public-history validation and execution authority are Host duties.
pub fn validated_session(
store: &SubjektivStore,
attribution: SubjectSessionAttribution,
) -> JournalResult<Self> {
if store.session_attribution(&attribution.session_id)?.as_ref() != Some(&attribution) {
return Err(JournalError::PermissionDenied(
"immutable Session attribution mismatch".into(),
));
}
let mut context = Self::validated_reader(store, &attribution.subject_id)?;
use rusqlite::OptionalExtension;
context.writer_binding_id = store.database.try_with_connection(|tx| {
Ok::<_, JournalError>(
tx.query_row(
"SELECT binding_id FROM journal_session_bindings WHERE session_id=?1",
[&attribution.session_id],
|r| r.get(0),
)
.optional()?,
)
})?;
context.attribution = Some(attribution);
Ok(context)
}
}
/// A restricted Journal generation, not Memory consolidation authority.
#[derive(Debug, Clone)]
pub struct JournalJobContext {
host: JournalHostContext,
generation_id: String,
binding: JobAttemptBinding,
}
impl JournalJobContext {
pub fn validated(
store: &SubjektivStore,
host: &JournalHostContext,
generation_id: &str,
binding: JobAttemptBinding,
) -> JournalResult<Self> {
let context = Self {
host: host.clone(),
generation_id: generation_id.into(),
binding,
};
store.validate_journal_job(&context)?;
Ok(context)
}
/// Reconstructs a retained binding for exact completion/receipt recovery. Hosts still check
/// the external attempt lease. New record/checkpoint operations independently require an
/// active generation inside their transaction, so this never revives cancelled authority.
pub fn validated_recovery(
store: &SubjektivStore,
host: &JournalHostContext,
generation_id: &str,
binding: JobAttemptBinding,
) -> JournalResult<Self> {
let context = Self {
host: host.clone(),
generation_id: generation_id.into(),
binding,
};
store.database.try_with_connection(|tx| {
self::store::job(store, tx, &context, false)?;
Ok::<_, JournalError>(())
})?;
Ok(context)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSourcePointer {
pub segment_id: Uuid,
/// Serialized public SessionEntryRef, including its `E` prefix.
pub entry_ref: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSource {
pub session_id: String,
pub pointer: JournalSourcePointer,
/// Host-attested canonical public entry order across Segments, starting at zero.
pub order: u64,
pub content_sha256: String,
pub total_bytes: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalByteRange {
pub start: u64,
pub end: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSourceRange {
pub source: JournalSource,
pub bytes: JournalByteRange,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalEntryState {
Open,
Fixed,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Journal {
pub id: JournalId,
pub subject_id: String,
pub source_session_id: String,
pub title: String,
pub current_position: String,
/// Processing cursor is independent of every fixed event endpoint.
pub process_order: u64,
pub process_byte: u64,
/// Content condition advanced by committed operations, never by acceptance notifications.
pub state_token: String,
pub created_at: String,
pub updated_at: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalMetadata {
pub id: JournalId,
pub subject_id: String,
pub source_session_id: String,
pub title: String,
pub current_position_bytes: usize,
pub process_order: u64,
pub process_byte: u64,
pub state_token: String,
pub created_at: String,
pub updated_at: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalEntry {
pub id: JournalEntryId,
pub journal_id: JournalId,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub body: String,
pub ongoing: String,
pub source_end: Option<JournalSourcePointer>,
pub created_at: String,
pub updated_at: String,
pub fixed_at: Option<String>,
pub correction_refs: Vec<JournalEntryId>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalGeneration {
pub id: String,
pub journal_id: JournalId,
pub job: JobAttemptBinding,
pub selected_ranges: Vec<JournalSourceRange>,
pub initial_journal: Journal,
pub initial_open: Option<JournalEntry>,
pub previous_fixed_end: Option<JournalSourcePointer>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalAction {
Create {
title: String,
body: String,
ongoing: String,
state: JournalEntryState,
source_end: Option<JournalSourcePointer>,
correction_refs: Vec<JournalEntryId>,
},
Rewrite {
entry_id: JournalEntryId,
title: String,
body: String,
ongoing: String,
},
Fix {
entry_id: JournalEntryId,
source_end: JournalSourcePointer,
},
UpdatePosition {
current_position: String,
},
/// Explicitly records the caller's checked conclusion that no entry change is needed.
ConfirmUnchanged,
/// Links durable record/no-change receipts to actual reflected input; IDs are Journal scoped.
Checkpoint {
record_operation_ids: Vec<String>,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalOperation {
pub request_id: String,
pub expected_state: String,
pub action: JournalAction,
pub covered_ranges: Vec<JournalSourceRange>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalReceipt {
pub request_id: String,
pub generation_id: String,
pub entry_id: Option<JournalEntryId>,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalSavedOperation {
pub operation: JournalOperation,
pub receipt: JournalReceipt,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalOperationKind {
Create,
Rewrite,
Fix,
UpdatePosition,
ConfirmUnchanged,
Checkpoint,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalOperationStatus {
pub receipt: JournalReceipt,
pub kind: JournalOperationKind,
pub covered_range_count: usize,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalSourceStatus {
pub source: JournalSource,
pub accepted: Vec<JournalByteRange>,
pub unavailable: Vec<JournalByteRange>,
pub reflected_bytes: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalPage<T> {
pub items: Vec<T>,
pub next: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalEntrySummary {
pub id: JournalEntryId,
pub journal_id: JournalId,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub source_end: Option<JournalSourcePointer>,
/// Exclusive start; None means Session start. The end is inclusive.
pub previous_fixed_end: Option<JournalSourcePointer>,
pub source_available: bool,
pub correction_refs: Vec<JournalEntryId>,
pub content_token: String,
pub created_at: String,
pub updated_at: String,
pub fixed_at: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalTextField {
Body,
Ongoing,
CurrentPosition,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalTextPage {
pub text: String,
pub next_byte: Option<usize>,
pub content_token: String,
}
pub(super) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("schema.sql"))?;
Ok(())
}
+194
View File
@@ -0,0 +1,194 @@
//! Host-side construction of bounded Journal generation input.
//!
//! Admission and byte identity belong to the Session Host. Selection neither starts Jobs nor
//! treats previews, failed fetches, or missing bytes as reflected. Production readers verify
//! original canonical identity while streaming and return only the requested bounded fragment.
use super::*;
use sha2::{Digest, Sha256};
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalMaterial {
pub range: JournalSourceRange,
pub text: String,
}
/// Adapter for Host captures already bounded by their storage read contract. Filesystem/Runtime
/// callers should use [`select_materials_bounded`] rather than allocating a whole large entry.
pub fn select_materials(
store: &SubjektivStore,
host: &JournalHostContext,
journal_id: &str,
max_bytes: usize,
mut read: impl FnMut(&JournalSource) -> JournalResult<String>,
) -> JournalResult<Vec<JournalMaterial>> {
select_materials_bounded(
store,
host,
journal_id,
max_bytes,
|source, start, limit| {
let text = read(source)?;
verify_canonical_text(source, &text)?;
let start = usize::try_from(start).map_err(|_| invalid("source offset overflow"))?;
if !text.is_char_boundary(start) {
return Err(conflict("source offset is not UTF-8 aligned"));
}
let mut end = start.saturating_add(limit).min(text.len());
while !text.is_char_boundary(end) {
end -= 1;
}
Ok(JournalMaterial {
range: JournalSourceRange {
source: source.clone(),
bytes: JournalByteRange {
start: start as u64,
end: end as u64,
},
},
text: text[start..end].into(),
})
},
)
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
fn conflict(message: &str) -> JournalError {
JournalError::Conflict(message.into())
}
/// Selects the oldest contiguous input under a whole serialized materials-array budget.
/// The Host callback verifies original public provenance/digest/length while streaming, and
/// returns at most `limit` canonical UTF-8 bytes at exactly `start`. Its descriptor must equal
/// the admitted source. This in-process attestation must never be deserialized from model input.
pub fn select_materials_bounded(
store: &SubjektivStore,
host: &JournalHostContext,
journal_id: &str,
max_bytes: usize,
mut read: impl FnMut(&JournalSource, u64, usize) -> JournalResult<JournalMaterial>,
) -> JournalResult<Vec<JournalMaterial>> {
let journal = store.read_journal(host, journal_id)?;
let mut order = journal.process_order;
let mut offset = journal.process_byte;
let mut materials = Vec::new();
let mut remaining = max_bytes.saturating_sub(2);
while materials.len() < MAX_JOURNAL_RANGES {
let status = match store.read_journal_source(host, journal_id, order) {
Ok(status) => status,
Err(JournalError::NotFound(_)) => break,
Err(error) => return Err(error),
};
if offset == status.source.total_bytes {
order = order
.checked_add(1)
.ok_or_else(|| invalid("source order exhausted"))?;
offset = 0;
continue;
}
let Some(accepted) = status
.accepted
.iter()
.find(|r| r.start <= offset && offset < r.end)
else {
break;
};
let mut end = accepted.end;
for unavailable in &status.unavailable {
if unavailable.start <= offset && offset < unavailable.end {
end = offset;
break;
}
if offset < unavailable.start {
end = end.min(unavailable.start);
}
}
if end == offset {
break;
}
let empty = JournalMaterial {
range: JournalSourceRange {
source: status.source.clone(),
bytes: JournalByteRange {
start: offset,
end: offset,
},
},
text: String::new(),
};
let envelope = serde_json::to_vec(&empty)?.len() + usize::from(!materials.is_empty());
// Reserve decimal endpoint growth and enough room for one UTF-8 scalar.
let allowance = remaining.saturating_sub(envelope.saturating_add(20));
if allowance < 4 {
if materials.is_empty() {
return Err(invalid(
"input budget cannot fit source metadata and a UTF-8 scalar",
));
}
break;
}
let limit = allowance.min((end - offset).min(usize::MAX as u64) as usize);
let mut material = read(&status.source, offset, limit)?;
if material.range.source != status.source
|| material.range.bytes.start != offset
|| material.range.bytes.end != offset.saturating_add(material.text.len() as u64)
|| material.range.bytes.end > end
|| material.text.is_empty()
|| material.text.len() > limit
{
return Err(conflict(
"Host source chunk does not match the requested canonical byte range",
));
}
let mut allowance = remaining - envelope;
let mut selected_bytes = 0;
for ch in material.text.chars() {
let cost = match ch {
'"' | '\\' | '\u{8}' | '\u{c}' | '\n' | '\r' | '\t' => 2,
ch if ch < '\u{20}' => 6,
ch => ch.len_utf8(),
};
if cost.saturating_add(20) > allowance {
break;
}
allowance -= cost;
selected_bytes += ch.len_utf8();
}
if selected_bytes == 0 {
if materials.is_empty() {
return Err(invalid("input budget cannot fit one escaped UTF-8 scalar"));
}
break;
}
material.text.truncate(selected_bytes);
material.range.bytes.end = offset + selected_bytes as u64;
let cost = serde_json::to_vec(&material)?.len() + usize::from(!materials.is_empty());
if cost > remaining {
return Err(invalid("generation input budget accounting failed"));
}
remaining -= cost;
offset = material.range.bytes.end;
materials.push(material);
if offset < end {
break;
}
}
Ok(materials)
}
/// Verifies the full canonical public role/tool framing of an already bounded Host capture.
pub fn verify_canonical_text(source: &JournalSource, text: &str) -> JournalResult<()> {
if text.len() as u64 != source.total_bytes
|| Sha256::digest(text.as_bytes())
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>()
!= source.content_sha256
{
return Err(conflict("canonical public source bytes changed"));
}
Ok(())
}
+88
View File
@@ -0,0 +1,88 @@
CREATE TABLE journals (
id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
source_session_id TEXT NOT NULL UNIQUE,
title TEXT NOT NULL,
current_position TEXT NOT NULL,
process_order INTEGER NOT NULL DEFAULT 0 CHECK (process_order >= 0),
process_byte INTEGER NOT NULL DEFAULT 0 CHECK (process_byte >= 0),
state_token TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(subject_id, id)
);
CREATE TABLE journal_entries (
id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL,
journal_id TEXT NOT NULL,
position INTEGER NOT NULL CHECK (position >= 0),
state TEXT NOT NULL CHECK (state IN ('open', 'fixed')),
title TEXT NOT NULL,
body TEXT NOT NULL,
ongoing TEXT NOT NULL,
source_end_json TEXT,
source_order INTEGER,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
fixed_at TEXT,
UNIQUE(journal_id, position),
UNIQUE(subject_id, id),
FOREIGN KEY(subject_id, journal_id) REFERENCES journals(subject_id, id) ON DELETE RESTRICT,
CHECK ((state = 'open' AND source_end_json IS NULL AND source_order IS NULL AND fixed_at IS NULL)
OR (state = 'fixed' AND source_end_json IS NOT NULL AND source_order IS NOT NULL AND source_order >= 0 AND fixed_at IS NOT NULL AND ongoing = ''))
);
CREATE UNIQUE INDEX journal_one_open ON journal_entries(journal_id) WHERE state = 'open';
CREATE TRIGGER journal_append_after_open BEFORE INSERT ON journal_entries
WHEN EXISTS (SELECT 1 FROM journal_entries WHERE journal_id = NEW.journal_id AND state = 'open')
BEGIN SELECT RAISE(ABORT, 'journal open must be the tail'); END;
CREATE TRIGGER journal_fixed_immutable BEFORE UPDATE ON journal_entries WHEN OLD.state = 'fixed'
BEGIN SELECT RAISE(ABORT, 'journal fixed entries are immutable'); END;
CREATE TRIGGER journal_entry_identity BEFORE UPDATE ON journal_entries
WHEN OLD.id <> NEW.id OR OLD.subject_id <> NEW.subject_id OR OLD.journal_id <> NEW.journal_id
OR OLD.position <> NEW.position OR OLD.created_at <> NEW.created_at
BEGIN SELECT RAISE(ABORT, 'journal entry identity is immutable'); END;
CREATE TABLE journal_corrections (
subject_id TEXT NOT NULL,
entry_id TEXT NOT NULL,
target_id TEXT NOT NULL,
PRIMARY KEY(entry_id, target_id),
FOREIGN KEY(subject_id, entry_id) REFERENCES journal_entries(subject_id, id) ON DELETE RESTRICT,
FOREIGN KEY(subject_id, target_id) REFERENCES journal_entries(subject_id, id) ON DELETE RESTRICT,
CHECK(entry_id <> target_id)
);
CREATE TABLE journal_sources (
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
source_order INTEGER NOT NULL CHECK(source_order >= 0),
pointer_json TEXT NOT NULL,
entry_ref TEXT NOT NULL,
source_json TEXT NOT NULL,
accepted_json TEXT NOT NULL,
unavailable_json TEXT NOT NULL,
reflected_bytes INTEGER NOT NULL DEFAULT 0 CHECK(reflected_bytes >= 0),
PRIMARY KEY(journal_id, source_order),
UNIQUE(journal_id, pointer_json),
UNIQUE(journal_id, entry_ref)
);
CREATE TABLE journal_generations (
id TEXT PRIMARY KEY,
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
job_id TEXT NOT NULL,
attempt_id TEXT NOT NULL,
request_json TEXT NOT NULL,
snapshot_json TEXT NOT NULL,
active INTEGER NOT NULL CHECK(active IN (0, 1)),
UNIQUE(journal_id, job_id, attempt_id)
);
CREATE UNIQUE INDEX journal_one_generation ON journal_generations(journal_id) WHERE active = 1;
CREATE TABLE journal_operations (
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
request_id TEXT NOT NULL,
generation_id TEXT NOT NULL REFERENCES journal_generations(id) ON DELETE RESTRICT,
request_json TEXT NOT NULL,
receipt_json TEXT NOT NULL,
PRIMARY KEY(journal_id, request_id)
);
CREATE TRIGGER journal_operations_no_update BEFORE UPDATE ON journal_operations
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are immutable'); END;
CREATE TRIGGER journal_operations_no_delete BEFORE DELETE ON journal_operations
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are retained'); END;
+424
View File
@@ -0,0 +1,424 @@
//! Shared sequential Job service. Backend and standalone authenticate and check the live attempt
//! before each call; the store additionally fences closed generations inside mutation transactions.
//! Job completion attests a saved processing checkpoint, not model prose or fixed event state.
use super::transport::*;
use super::*;
use rusqlite::{OptionalExtension, params};
use sha2::{Digest, Sha256};
fn metadata(j: &Journal) -> JournalMetadata {
JournalMetadata {
id: j.id.clone(),
subject_id: j.subject_id.clone(),
source_session_id: j.source_session_id.clone(),
title: j.title.clone(),
current_position_bytes: j.current_position.len(),
process_order: j.process_order,
process_byte: j.process_byte,
state_token: j.state_token.clone(),
created_at: j.created_at.clone(),
updated_at: j.updated_at.clone(),
}
}
fn text_page(
text: &str,
read: &JournalTextRead,
condition: &[u8],
maximum_end: usize,
) -> JournalResult<JournalTextPage> {
if !(1..=MAX_JOURNAL_READ_BYTES).contains(&read.limit)
|| read.offset > maximum_end
|| !text.is_char_boundary(read.offset)
{
return Err(JournalError::InvalidInput(
"invalid Journal UTF-8 read range".into(),
));
}
let token: String = Sha256::digest(condition)
.iter()
.map(|b| format!("{b:02x}"))
.collect();
if read
.content_token
.as_deref()
.is_some_and(|expected| expected != token)
|| (read.offset > 0 && read.content_token.is_none())
{
return Err(JournalError::Conflict(
"Journal text changed or continuation condition missing".into(),
));
}
let mut end = read.offset.saturating_add(read.limit).min(maximum_end);
while !text.is_char_boundary(end) {
end -= 1;
}
if end == read.offset && end < maximum_end {
return Err(JournalError::InvalidInput(
"read limit cannot fit one UTF-8 scalar".into(),
));
}
Ok(JournalTextPage {
text: text[read.offset..end].into(),
next_byte: (end < maximum_end).then_some(end),
content_token: token,
})
}
/// Adapter for Host captures bounded by their storage contract. Production source reads should
/// use [`execute_bounded`] to avoid allocating an entire source for one small read.
pub fn execute(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
mut read_source: impl FnMut(&JournalSource) -> JournalResult<String>,
) -> JournalResult<JournalJobResponse> {
execute_bounded(store, context, request, |source, start, limit| {
let text = read_source(source)?;
super::recording::verify_canonical_text(source, &text)?;
let start = usize::try_from(start)
.map_err(|_| JournalError::InvalidInput("source offset overflow".into()))?;
if !text.is_char_boundary(start) {
return Err(JournalError::InvalidInput(
"source offset is not UTF-8 aligned".into(),
));
}
let mut end = start.saturating_add(limit).min(text.len());
while !text.is_char_boundary(end) {
end -= 1;
}
Ok(recording::JournalMaterial {
range: JournalSourceRange {
source: source.clone(),
bytes: JournalByteRange {
start: start as u64,
end: end as u64,
},
},
text: text[start..end].into(),
})
})
}
/// Reads are delivered through this service before mutation authority is enabled. The durable
/// delivery ledger stores byte ranges only: Jobs may retain immutable descriptors without
/// accumulating a permanent second copy of the original Session in their request ledgers.
/// The Host verifies original provenance/digest/length and returns a bounded UTF-8 fragment.
pub fn execute_bounded(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
mut read_source: impl FnMut(&JournalSource, u64, usize) -> JournalResult<recording::JournalMaterial>,
) -> JournalResult<JournalJobResponse> {
let generation = store
.database
.try_with_connection(|tx| super::store::job(store, tx, context, true))?;
let mut delivered = None;
let response = match request {
JournalJobRequest::Execute { operation } => {
require_delivery(store, context, &generation.selected_ranges)?;
let receipt = if matches!(operation.action, JournalAction::Checkpoint { .. }) {
crate::api::checkpoint_journal(store, context, operation)?
} else {
crate::api::execute_journal(store, context, operation)?
};
JournalJobResponse::Saved(receipt)
}
JournalJobRequest::ReadState => {
let state = store.database.try_with_connection(|tx| {
let generation = super::store::job(store, tx, context, true)?;
let current =
super::store::journal(tx, &context.host, &generation.journal_id, true)?;
let current_open = super::store::open(tx, &generation.journal_id)?;
Ok::<_, JournalError>(JournalJobState {
generation_id: generation.id,
initial_journal: metadata(&generation.initial_journal),
initial_open: generation
.initial_open
.map(|e| super::store::summary(tx, e))
.transpose()?,
current_journal: metadata(&current),
current_open: current_open
.map(|e| super::store::summary(tx, e))
.transpose()?,
previous_fixed_end: generation.previous_fixed_end,
selected_ranges: generation.selected_ranges,
})
})?;
JournalJobResponse::State(state)
}
JournalJobRequest::QueryOperation { request_id } => {
let status = match store.read_journal_operation_status(
&context.host,
&generation.journal_id,
&request_id,
) {
Ok(status) => Some(status),
Err(JournalError::NotFound(_)) => None,
Err(error) => return Err(error),
};
JournalJobResponse::Operation(status)
}
JournalJobRequest::ReadOperationCoverage {
request_id,
offset,
limit,
} => JournalJobResponse::Coverage(store.read_journal_operation_coverage(
&context.host,
&generation.journal_id,
&request_id,
offset,
limit,
)?),
JournalJobRequest::ListOperations { after, limit } => {
let receipts = store.list_journal_operations(
&context.host,
&generation.journal_id,
after.as_deref(),
limit,
)?;
let items = receipts
.items
.iter()
.map(|r| {
store.read_journal_operation_status(
&context.host,
&generation.journal_id,
&r.request_id,
)
})
.collect::<JournalResult<Vec<_>>>()?;
JournalJobResponse::Operations(JournalPage {
items,
next: receipts.next,
})
}
JournalJobRequest::ReadText { read } => {
let text = match &read.target {
JournalTextTarget::Source { pointer } => {
let range = generation
.selected_ranges
.iter()
.find(|r| {
&r.source.pointer == pointer
&& r.bytes.start <= read.offset as u64
&& read.offset as u64 <= r.bytes.end
})
.ok_or_else(|| {
JournalError::PermissionDenied(
"source bytes are outside this generation".into(),
)
})?;
if !(1..=MAX_JOURNAL_READ_BYTES).contains(&read.limit) {
return Err(JournalError::InvalidInput(
"invalid source read limit".into(),
));
}
let condition =
serde_json::to_vec(&(&generation.id, &range.source, range.bytes))?;
let token: String = Sha256::digest(&condition)
.iter()
.map(|b| format!("{b:02x}"))
.collect();
if read
.content_token
.as_deref()
.is_some_and(|expected| expected != token)
|| (read.offset as u64 != range.bytes.start && read.content_token.is_none())
{
return Err(JournalError::Conflict(
"source continuation condition missing or changed".into(),
));
}
let mut material =
read_source(&range.source, read.offset as u64, read.limit.max(4))?;
if material.range.source != range.source
|| material.range.bytes.start != read.offset as u64
|| material.range.bytes.end
!= (read.offset as u64).saturating_add(material.text.len() as u64)
|| material.range.bytes.end > range.source.total_bytes
|| material.text.len() > read.limit.max(4)
{
return Err(JournalError::Conflict(
"source fragment identity or byte range changed".into(),
));
}
let mut length = material
.text
.len()
.min(read.limit)
.min((range.bytes.end - read.offset as u64) as usize);
while !material.text.is_char_boundary(length) {
length -= 1;
}
if length == 0 && (read.offset as u64) < range.bytes.end {
return Err(JournalError::InvalidInput(
"read limit cannot fit a source UTF-8 scalar".into(),
));
}
material.text.truncate(length);
let end = read.offset + length;
if length > 0 {
delivered = Some((
range.source.order,
JournalByteRange {
start: read.offset as u64,
end: end as u64,
},
));
}
JournalTextPage {
text: material.text,
next_byte: ((end as u64) < range.bytes.end).then_some(end),
content_token: token,
}
}
target => {
let (journal, open) = match target {
JournalTextTarget::Position { snapshot }
| JournalTextTarget::OpenBody { snapshot }
| JournalTextTarget::OpenOngoing { snapshot } => match snapshot {
JournalSnapshot::Initial => (
generation.initial_journal.clone(),
generation.initial_open.clone(),
),
JournalSnapshot::Current => {
store.journal_resume_state(&context.host, &generation.journal_id)?
}
},
JournalTextTarget::Source { .. } => unreachable!(),
};
let text = match target {
JournalTextTarget::Position { .. } => journal.current_position.clone(),
JournalTextTarget::OpenBody { .. } => open
.as_ref()
.ok_or_else(|| JournalError::NotFound("no open entry".into()))?
.body
.clone(),
JournalTextTarget::OpenOngoing { .. } => open
.as_ref()
.ok_or_else(|| JournalError::NotFound("no open entry".into()))?
.ongoing
.clone(),
JournalTextTarget::Source { .. } => unreachable!(),
};
let condition = serde_json::to_vec(&(
&generation.id,
&read.target,
&journal.state_token,
&text,
))?;
text_page(&text, &read, &condition, text.len())?
}
};
JournalJobResponse::Text(text)
}
};
if serde_json::to_vec(&response)?.len() > MAX_JOURNAL_RESPONSE_BYTES {
return Err(JournalError::InvalidInput(
"Journal response exceeds bounded tool budget".into(),
));
}
if let Some((order, range)) = delivered {
store.database.try_transaction(|tx| {
// A response that cannot fit the tool budget or loses its live generation must
// not attest bytes. Lost responses can safely reread the identical source page.
super::store::job(store, tx, context, true)?;
let saved: Option<String> = tx.query_row(
"SELECT ranges_json FROM journal_generation_deliveries WHERE generation_id=?1 AND source_order=?2",
params![context.generation_id, order], |row| row.get(0),
).optional()?;
let mut ranges: Vec<JournalByteRange> = saved.map(|raw| serde_json::from_str(&raw)).transpose()?.unwrap_or_default();
ranges.push(range);
tx.execute(
"INSERT INTO journal_generation_deliveries(generation_id,source_order,ranges_json) VALUES(?1,?2,?3) ON CONFLICT(generation_id,source_order) DO UPDATE SET ranges_json=excluded.ranges_json",
params![context.generation_id, order, serde_json::to_string(&super::store::merge(ranges)?)?],
)?;
Ok::<_, JournalError>(())
})?;
}
Ok(response)
}
fn require_delivery(
store: &SubjektivStore,
context: &JournalJobContext,
selected: &[JournalSourceRange],
) -> JournalResult<()> {
store.database.try_with_connection(|tx| {
super::store::job(store, tx, context, true)?;
for range in selected {
let saved: Option<String> = tx.query_row(
"SELECT ranges_json FROM journal_generation_deliveries WHERE generation_id=?1 AND source_order=?2",
params![context.generation_id, range.source.order], |row| row.get(0),
).optional()?;
let delivered: Vec<JournalByteRange> = saved.map(|raw| serde_json::from_str(&raw)).transpose()?.unwrap_or_default();
if !delivered.iter().any(|bytes| bytes.start <= range.bytes.start && range.bytes.end <= bytes.end) {
return Err(JournalError::Conflict(
"read every selected source byte with ReadJournalText before recording or checkpointing".into(),
));
}
}
Ok(())
})
}
/// Atomically saves completion and fences new operations. Exact replay returns the same saved
/// completion even after later generations advance the Journal. Cancelled incomplete generations
/// cannot manufacture completion, and neither fixing an entry nor updating position can substitute
/// for the receipt-backed processing cursor.
pub fn finish(
store: &SubjektivStore,
context: &JournalJobContext,
) -> JournalResult<JournalJobCompletion> {
store.database.try_transaction(|tx| {
let generation = super::store::job(store, tx, context, false)?;
let saved: Option<String> = tx
.query_row(
"SELECT completion_json FROM journal_generations WHERE id=?1",
[&generation.id],
|row| row.get(0),
)
.optional()?
.flatten();
if let Some(saved) = saved {
return Ok(serde_json::from_str(&saved)?);
}
super::store::job(store, tx, context, true)?;
super::binding::require_saved_position(tx, &generation.id)?;
let last = generation
.selected_ranges
.last()
.ok_or_else(|| JournalError::InvalidInput("empty generation selection".into()))?;
let end = if last.bytes.end == last.source.total_bytes {
(
last.source
.order
.checked_add(1)
.ok_or_else(|| JournalError::InvalidInput("source order exhausted".into()))?,
0,
)
} else {
(last.source.order, last.bytes.end)
};
let journal = super::store::journal(tx, &context.host, &generation.journal_id, true)?;
if (journal.process_order, journal.process_byte) != end {
return Err(JournalError::Conflict(
"generation has unreflected selected bytes; save a receipt-backed checkpoint first"
.into(),
));
}
let completion = JournalJobCompletion {
generation_id: generation.id.clone(),
state_token: journal.state_token,
process_order: journal.process_order,
process_byte: journal.process_byte,
};
tx.execute(
"UPDATE journal_generations SET active=0,completion_json=?2 WHERE id=?1",
params![generation.id, serde_json::to_string(&completion)?],
)?;
Ok(completion)
})
}
@@ -0,0 +1,93 @@
//! Optional Host metadata observations for original material, independent of saved Journal reading.
//!
//! Presence checks must not fetch original history or restore a Worker. A failed/unauthorized
//! metadata lookup is unknown, not missing, and cannot prevent a saved Journal body from being read.
use super::memory_links::{
MemoryJournalRequest, MemoryJournalResponse, OriginalSourceAvailability,
};
use super::{JournalHostContext, JournalResult, SubjektivStore};
use rusqlite::{OptionalExtension, params};
impl SubjektivStore {
/// Executes the saved-reference operation first, then annotates original-source absence using
/// Host-owned metadata only. `Some(false)` is confirmed absence; `None` is an unavailable lookup.
/// Presence alone cannot attest that a particular historical entry is still readable.
pub fn execute_memory_journal_with_source_presence(
&self,
host: &JournalHostContext,
request: MemoryJournalRequest,
mut presence: impl FnMut(&str) -> Option<bool>,
) -> JournalResult<MemoryJournalResponse> {
let selector = match &request {
MemoryJournalRequest::Reconcile { selector }
| MemoryJournalRequest::Links { selector }
| MemoryJournalRequest::ReconcilePage { selector, .. }
| MemoryJournalRequest::LinksPage { selector, .. }
| MemoryJournalRequest::Read { selector, .. } => Some(selector.clone()),
MemoryJournalRequest::Related { .. } => None,
};
let mut response = self.execute_memory_journal(host, request)?;
let Some(selector) = selector else {
return Ok(response);
};
// Membership and scoped reader authority were checked by the operation. Candidate JSON is
// immutable; never infer source ownership from its presence without checking attribution.
let candidate = self.staging_candidate(&host.subject_id, &selector.candidate_id)?;
let mut observations = std::collections::HashMap::new();
let mut annotate =
|link: &mut super::memory_links::MemoryJournalSourceLink| -> JournalResult<()> {
let source = candidate
.as_ref()
.and_then(|c| c.source_refs.get(link.source_index));
// Legacy anchors may lack a Session ID. A connected edge already resolved it
// from immutable attribution and stable Segment/entry identity; reuse that saved
// identity rather than fetching history or modifying the original candidate.
let saved_session: Option<String> = self.database.try_with_connection(|tx| {
Ok::<_, super::JournalError>(tx.query_row(
"SELECT source_session_id FROM memory_journal_links
WHERE subject_id=?1 AND candidate_id=?2 AND source_index=?3 AND reason='linked'",
params![host.subject_id, selector.candidate_id, link.source_index],
|row| row.get(0),
).optional()?)
})?;
let session = saved_session
.as_deref()
.or_else(|| source.and_then(|s| s.session_id.as_deref()));
let observed = if let Some(session) = session {
if let Some(value) = observations.get(session) {
*value
} else {
let owned = self
.session_attribution(session)?
.is_some_and(|a| a.subject_id == host.subject_id);
let value = if owned { presence(session) } else { None };
observations.insert(session.to_owned(), value);
value
}
} else {
None
};
link.original_source = match observed {
Some(false) => OriginalSourceAvailability::Missing,
// Preserve explicit saved missing-byte attestations. Directory/registry presence
// does not prove the referenced entry survived rewind or storage corruption.
_ if link.original_source == OriginalSourceAvailability::Missing => {
OriginalSourceAvailability::Missing
}
_ => OriginalSourceAvailability::Unknown,
};
Ok(())
};
match &mut response {
MemoryJournalResponse::Links(page) => {
for source in &mut page.sources {
annotate(source)?;
}
}
MemoryJournalResponse::Read(read) => annotate(&mut read.source)?,
MemoryJournalResponse::Related(_) => {}
}
Ok(response)
}
}
+151
View File
@@ -0,0 +1,151 @@
//! Read-only, bounded Journal context at body startup and compaction boundaries.
//! Selection, text projection, references and processing checkpoints share one storage
//! transaction. Reads neither consult source history/Job liveness nor mutate recording.
//! Startup prefers the Host-retained predecessor; compaction requires a live body writer
//! and prefers its current Session, without waiting for Idle or recording completion.
//! Both supplement only from the same Subject under the same independent budgets.
//! This is derived reference material, never a new observation or an execution capability.
use super::*;
use rusqlite::{Connection, OptionalExtension, params};
pub const STARTUP_MAX_BYTES: usize = 32 * 1024;
const JOURNALS: usize = 3;
const ENTRIES: usize = 3;
const TEXT_BYTES: usize = 2048;
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalStartupSnapshot {
pub subject_id: String,
pub journals: Vec<StartupJournal>,
pub omitted: bool,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct StartupJournal {
pub journal_id: String,
pub source_session_id: String,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
pub current_position: String,
pub entries: Vec<StartupEntry>,
pub omitted: bool,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct StartupEntry {
pub entry_id: String,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub body: String,
pub ongoing: String,
pub source_end: Option<JournalSourcePointer>,
pub correction_refs: Vec<String>,
pub omitted: bool,
}
// Read a bounded UTF-8 prefix at SQLite, not a full megabyte field followed by truncation.
fn text(tx: &Connection, table: &str, id: &str, field: &str) -> JournalResult<(String, bool)> {
let (mut bytes, length): (Vec<u8>, usize) = tx.query_row(
&format!("SELECT COALESCE(substr(CAST({field} AS BLOB),1,?2),X''),length(CAST({field} AS BLOB)) FROM {table} WHERE id=?1"),
params![id, TEXT_BYTES],
|r| Ok((r.get(0)?, r.get(1)?)),
)?;
let length_read = bytes.len();
match std::str::from_utf8(&bytes) {
Ok(_) => {}
Err(error) if error.error_len().is_none() => bytes.truncate(error.valid_up_to()),
Err(_) => {
return Err(JournalError::Conflict(
"stored Journal text is not UTF-8".into(),
));
}
}
let omitted = bytes.len() < length || length_read < length;
Ok((String::from_utf8(bytes).expect("validated UTF-8"), omitted))
}
impl SubjektivStore {
/// `previous_session` comes from the Host's retained predecessor binding, not model input.
/// Recent Subject-local Journals supplement that predecessor under independent fixed budgets.
pub fn journal_startup_snapshot(
&self,
host: &JournalHostContext,
previous_session: Option<&str>,
) -> JournalResult<JournalStartupSnapshot> {
self.context_snapshot(host, previous_session, false)
}
/// Read the saved current-Session Journal without reconciling or advancing it.
/// The active writer check fences released body lifetimes, not non-Idle execution.
pub fn journal_compaction_snapshot(
&self,
host: &JournalHostContext,
) -> JournalResult<JournalStartupSnapshot> {
let session = host.attribution.as_ref().ok_or_else(|| {
JournalError::PermissionDenied("compaction requires a Host-bound body Session".into())
})?;
self.context_snapshot(host, Some(&session.session_id), true)
}
fn context_snapshot(
&self,
host: &JournalHostContext,
preferred_session: Option<&str>,
require_current_writer: bool,
) -> JournalResult<JournalStartupSnapshot> {
self.database.try_transaction(|tx| {
super::store::check_host(self, tx, host, require_current_writer)?;
let mut ids = Vec::new();
if let Some(session) = preferred_session {
let previous: Option<(String, String)> = tx.query_row(
"SELECT id,subject_id FROM journals WHERE source_session_id=?1",
[session], |r| Ok((r.get(0)?, r.get(1)?)),
).optional()?;
if let Some((id, subject)) = previous {
if subject != host.subject_id { return Err(JournalError::PermissionDenied("predecessor Journal belongs to another Subject".into())); }
ids.push(id);
}
}
let mut query = tx.prepare("SELECT id FROM journals WHERE subject_id=?1 ORDER BY updated_at DESC,id DESC LIMIT ?2")?;
let recent = query.query_map(params![host.subject_id, JOURNALS + 1], |r| r.get::<_, String>(0))?;
for id in recent { let id = id?; if !ids.contains(&id) { ids.push(id); } }
let omitted = ids.len() > JOURNALS;
ids.truncate(JOURNALS);
let mut snapshot = JournalStartupSnapshot { subject_id: host.subject_id.clone(), journals: Vec::new(), omitted };
for id in ids {
let (session, token, order, byte) = tx.query_row(
"SELECT source_session_id,state_token,process_order,process_byte FROM journals WHERE id=?1 AND subject_id=?2",
params![id, host.subject_id], |r| Ok((r.get(0)?,r.get(1)?,r.get(2)?,r.get(3)?)),
)?;
let (current_position, mut omitted) = text(tx, "journals", &id, "current_position")?;
let mut stmt = tx.prepare("SELECT id,position,state,source_end_json FROM journal_entries WHERE journal_id=?1 ORDER BY position DESC LIMIT ?2")?;
let rows = stmt.query_map(params![id, ENTRIES + 1], |r| Ok((r.get::<_,String>(0)?,r.get::<_,u64>(1)?,r.get::<_,String>(2)?,r.get::<_,Option<String>>(3)?)))?.collect::<Result<Vec<_>,_>>()?;
omitted |= rows.len() > ENTRIES;
let mut entries = Vec::new();
for (entry_id, position, state, source_end) in rows.into_iter().take(ENTRIES) {
let (title, title_cut) = text(tx, "journal_entries", &entry_id, "title")?;
let (body, body_cut) = text(tx, "journal_entries", &entry_id, "body")?;
let (ongoing, ongoing_cut) = text(tx, "journal_entries", &entry_id, "ongoing")?;
let mut refs = tx.prepare("SELECT target_id FROM journal_corrections WHERE entry_id=?1 ORDER BY target_id LIMIT 17")?;
let mut correction_refs = refs.query_map([&entry_id], |r| r.get::<_,String>(0))?.collect::<Result<Vec<_>,_>>()?;
let omitted = title_cut || body_cut || ongoing_cut || correction_refs.len() > 16;
correction_refs.truncate(16);
entries.push(StartupEntry { entry_id, position, state: match state.as_str() { "open" => JournalEntryState::Open, "fixed" => JournalEntryState::Fixed, _ => return Err(JournalError::Conflict("invalid stored Journal entry state".into())) }, title, body, ongoing, source_end: source_end.map(|s| serde_json::from_str(&s)).transpose()?, correction_refs, omitted });
}
snapshot.journals.push(StartupJournal { journal_id: id, source_session_id: session, state_token: token, process_order: order, process_byte: byte, current_position, entries, omitted });
}
// JSON escaping also consumes the independent output budget. Drop least-priority
// entries/Journals rather than returning an over-budget or silently partial document.
while serde_json::to_vec(&snapshot)?.len() > STARTUP_MAX_BYTES {
snapshot.omitted = true;
let journal = snapshot.journals.last_mut().ok_or_else(|| JournalError::InvalidInput("startup metadata exceeds budget".into()))?;
if journal.entries.pop().is_some() { journal.omitted = true; } else { snapshot.journals.pop(); }
}
Ok(snapshot)
})
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+135
View File
@@ -0,0 +1,135 @@
//! Bounded, Host-bound Journal recording and sequential Job transport.
//!
//! Subject, Journal and generation authority never come from these model inputs.
//! Hosts validate live execution and public source bytes before selection; reading a
//! preview must not select the enclosing entry. Operation retries retain the entire
//! JournalOperation, including its original expected_state and covered_ranges.
use serde::{Deserialize, Serialize};
use super::{
JournalEntrySummary, JournalMetadata, JournalOperation, JournalOperationStatus, JournalPage,
JournalReceipt, JournalSourcePointer, JournalSourceRange, JournalTextPage,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalRecordingReason {
RunCommitted,
TaskCompleted,
BeforeCompact,
WorkerRestored,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalCompactionRequest {
pub session_id: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalRecordingRequest {
pub reason: JournalRecordingReason,
}
/// Admission is durable Host evidence, not proof that generation finished.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "status", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalRecordingResponse {
NoNewSources,
Accepted,
}
/// No unbounded initial/current body or position is included in model state.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalJobState {
pub generation_id: String,
pub initial_journal: JournalMetadata,
pub initial_open: Option<JournalEntrySummary>,
pub current_journal: JournalMetadata,
pub current_open: Option<JournalEntrySummary>,
pub previous_fixed_end: Option<JournalSourcePointer>,
/// Frozen permission to fetch this prefix, not evidence of delivery or reflection.
/// The sequential service gates mutations on its durable Source-read delivery ledger.
pub selected_ranges: Vec<JournalSourceRange>,
}
/// Host-verified saved progress only; not an aggregate record submission.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalJobCompletion {
pub generation_id: String,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalSnapshot {
Initial,
Current,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "target", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalTextTarget {
Position {
snapshot: JournalSnapshot,
},
OpenBody {
snapshot: JournalSnapshot,
},
OpenOngoing {
snapshot: JournalSnapshot,
},
/// The Host restricts this pointer to this Job's frozen public source input.
Source {
pointer: JournalSourcePointer,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalTextRead {
pub target: JournalTextTarget,
pub offset: usize,
pub limit: usize,
/// Required continuation condition; None only for the first chunk.
pub content_token: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalJobRequest {
Execute {
operation: JournalOperation,
},
ReadState,
QueryOperation {
request_id: String,
},
ReadOperationCoverage {
request_id: String,
offset: usize,
limit: usize,
},
ListOperations {
after: Option<String>,
limit: usize,
},
ReadText {
read: JournalTextRead,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "result", content = "value", rename_all = "snake_case")]
pub enum JournalJobResponse {
Saved(JournalReceipt),
State(JournalJobState),
Operation(Option<JournalOperationStatus>),
Coverage(JournalPage<JournalSourceRange>),
Operations(JournalPage<JournalOperationStatus>),
Text(JournalTextPage),
}
+27 -1
View File
@@ -929,7 +929,7 @@ mod tests {
let (_manager, _workspace, store) = open_store(&root, "workspace-a");
let migrated_subject = store.subject(&subject_id).unwrap().unwrap();
assert!(migrated_subject.behavior_md.is_empty());
assert!(migrated_subject.behavior_md.is_empty());
assert!(store.subject_permissions(&subject_id).unwrap().is_none());
let lifecycle_attribution = attribution(
&subject_id,
"runtime-1",
@@ -953,6 +953,32 @@ mod tests {
store.session_attribution("session-1").unwrap().unwrap(),
stored_attribution
);
let binding = store
.claim_journal_writer(&subject_id, "runtime-1", "worker-1")
.unwrap();
let host = store
.bind_journal_session(&binding, stored_attribution.clone())
.unwrap();
let range = crate::journal::JournalSourceRange {
source: crate::journal::JournalSource {
session_id: stored_attribution.session_id.clone(),
pointer: crate::journal::JournalSourcePointer {
segment_id: Uuid::now_v7(),
entry_ref: "Eafter-v1-migration".into(),
},
order: 0,
content_sha256: "ab".repeat(32),
total_bytes: 8,
},
bytes: crate::journal::JournalByteRange { start: 0, end: 8 },
};
let journal_id = store
.accept_journal_sources(&host, "After oldest supported migration", &[range])
.unwrap();
assert_eq!(
store.read_journal(&host, &journal_id).unwrap().subject_id,
subject_id
);
}
#[test]
+37
View File
@@ -2,7 +2,9 @@
pub mod api;
pub mod job;
pub mod journal;
mod legacy_migrations;
pub mod permissions;
mod schema;
use legacy_migrations::{
add_candidate_decision_receipts, add_subject_behavior, add_subject_session_attribution,
@@ -81,6 +83,16 @@ pub enum SubjektivError {
},
#[error("subject `{subject_id}` behavior content changed")]
SubjectBehaviorConflict { subject_id: String },
#[error("subject `{0}` permission source changed")]
SubjectPermissionsConflict(String),
#[error("permission operation `{0}` was reused with different input")]
PermissionOperationConflict(String),
#[error("authorization denied: {0}")]
AuthorizationDenied(String),
#[error("Worker execution grant binding conflicts for `{0}`")]
WorkerExecutionGrantConflict(String),
#[error("Subject `{subject_id}` has more than {limit} retained Worker execution grants")]
WorkerExecutionGrantListOverflow { subject_id: String, limit: usize },
#[error("surface generation conflict: {0}")]
SurfaceGenerationConflict(String),
#[error("memory `{memory_id}` cannot transition from {from:?} to {to:?}")]
@@ -602,6 +614,31 @@ static MIGRATIONS: &[FeatureMigration] = &[
"replace counters with immutable changes and content conditions",
schema::migrate,
),
FeatureMigration::new(
8,
"add durable journal operations and processing ranges",
journal::migrate,
),
FeatureMigration::new(
9,
"fence erased journal Sessions and permit explicit content erasure",
journal::migrate_lifecycle,
),
FeatureMigration::new(
10,
"fence journal writers by explicit body binding lifetime",
journal::migrate_binding,
),
FeatureMigration::new(
11,
"add immutable candidate Journal context links",
journal::memory_links::migrate,
),
FeatureMigration::new(
12,
"add explicit Subject permission sources and immutable child execution grants",
permissions::migrate,
),
];
pub const REGISTRATION: FeatureRegistration =
+288
View File
@@ -0,0 +1,288 @@
//! Host-administered permission sources and immutable child grants, separate from behavior.
//!
//! The Host authenticates administrators before source writes and binds verified Worker/Profile
//! identities before grant writes. These records are not a Worker registry. The Host alone
//! authenticates root execution ownership against its canonical Backend singleton and checks
//! every live lineage edge; Journal bindings and retained grants cannot establish ownership.
//! Replays return historical evidence without restoring a source or granting current authority.
use super::*;
use server_api::authorization::{
PermissionResource, ScopedPermission, SubjectPermissions, SubjectPermissionsUpdate,
WorkerExecutionGrant, validate_authorization_identifier, validate_scoped_permissions,
};
use sha2::{Digest, Sha256};
pub const MAX_SUBJECT_WORKER_EXECUTION_GRANTS: usize = 1_000;
pub(super) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("permissions.sql"))?;
Ok(())
}
impl SubjektivStore {
/// Unset sources are `None` (deny), including migrated Subjects. This is an admin read,
/// not an effective-authorization decision; retired Subjects retain their source evidence.
pub fn subject_permissions(&self, subject_id: &str) -> Result<Option<SubjectPermissions>> {
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
source_in_connection(connection, subject_id)
})
}
/// Host-admin-only replacement. Every replacement issues a fresh change ID, including
/// identical contents. Revocation is an explicit empty replacement. A scope-wide operation
/// ID binds Subject, CAS input, and both permission vectors; result and receipt commit together.
pub fn set_subject_permissions(
&self,
subject_id: &str,
update: &SubjectPermissionsUpdate,
) -> Result<SubjectPermissions> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(&update.operation_id)
.map_err(SubjektivError::InvalidRecord)?;
if let Some(change) = &update.expected_change_id {
validate_authorization_identifier(change).map_err(SubjektivError::InvalidRecord)?;
}
validate_scope(self.scope_id(), &update.execution)?;
validate_scope(self.scope_id(), &update.delegation)?;
let fingerprint: String = Sha256::digest(serde_json::to_vec(&(subject_id, update))?)
.iter()
.map(|byte| format!("{byte:02x}"))
.collect();
self.database.try_transaction(|tx| {
require_subject(tx, subject_id)?;
let receipt: Option<(String, String)> = tx.query_row(
"SELECT request_fingerprint, change_id FROM subject_permission_receipts WHERE operation_id=?1",
[&update.operation_id], |row| Ok((row.get(0)?, row.get(1)?)),
).optional()?;
if let Some((saved, change_id)) = receipt {
if saved != fingerprint {
return Err(SubjektivError::PermissionOperationConflict(update.operation_id.clone()));
}
let raw: String = tx.query_row(
"SELECT record_json FROM subject_permission_changes WHERE subject_id=?1 AND change_id=?2",
params![subject_id, change_id], |row| row.get(0),
)?;
return Ok(serde_json::from_str(&raw)?);
}
require_active_subject(tx, subject_id)?;
let current = source_in_connection(tx, subject_id)?;
if current.as_ref().map(|source| &source.change_id) != update.expected_change_id.as_ref() {
return Err(SubjektivError::SubjectPermissionsConflict(subject_id.into()));
}
let source = SubjectPermissions {
change_id: issued_id("permissions"),
execution: update.execution.clone(),
delegation: update.delegation.clone(),
};
tx.execute("INSERT INTO subject_permission_changes(subject_id,change_id,record_json) VALUES(?1,?2,?3)",
params![subject_id, source.change_id, serde_json::to_string(&source)?])?;
tx.execute("INSERT INTO subject_permissions(subject_id,change_id) VALUES(?1,?2) ON CONFLICT(subject_id) DO UPDATE SET change_id=excluded.change_id",
params![subject_id, source.change_id])?;
tx.execute("INSERT INTO subject_permission_receipts(operation_id,subject_id,request_fingerprint,change_id) VALUES(?1,?2,?3,?4)",
params![update.operation_id, subject_id, fingerprint, source.change_id])?;
Ok(source)
})
}
/// Binds child grants only. Exact retries recover an immutable binding even after revocation;
/// that success is not authorization to execute or redelegate. New bindings require the
/// current source, an active Subject and explicit delegation containment. The Host must
/// authenticate root singleton ownership and every live lineage edge before calling;
/// a missing retained parent is not ownership evidence. Profile resolution, provider scope,
/// Worker creation and caller authentication also remain Host duties.
pub fn bind_worker_execution_grant(&self, grant: &WorkerExecutionGrant) -> Result<()> {
validate_grant(self.scope_id(), grant)?;
self.database.try_transaction(|tx| {
if let Some(existing) = grant_in_connection(tx, &grant.runtime_id, &grant.worker_id)? {
return if existing == *grant { Ok(()) } else {
Err(SubjektivError::WorkerExecutionGrantConflict(format!("{}/{}", grant.runtime_id, grant.worker_id)))
};
}
require_active_subject(tx, &grant.subject_id)?;
let source = source_in_connection(tx, &grant.subject_id)?
.ok_or_else(|| denied("Subject permissions are unset"))?;
if source.change_id != grant.source_change_id {
return Err(denied("Subject permission source is no longer current"));
}
require_covered(&source.delegation, &grant.permissions)?;
require_covered(&source.delegation, &grant.redelegation)?;
// A child grant cannot turn a body (including a released historical body) into a
// delegated child or attach a Worker historically attributed to a different Subject.
let invalid_target: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![grant.runtime_id, grant.worker_id, grant.subject_id], |row| row.get(0),
)?;
if invalid_target { return Err(denied("target is a body or belongs to another Subject")); }
let origin = require_retained_origin(tx, grant)?;
if let Some(origin) = origin {
require_covered(&origin.redelegation, &grant.permissions)?;
require_covered(&origin.redelegation, &grant.redelegation)?;
}
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![grant.runtime_id, grant.worker_id, grant.subject_id, grant.source_change_id, serde_json::to_string(grant)?])?;
Ok(())
})
}
/// Returns this Subject's retained grants, including stale grants needed for cancellation
/// and reporting. Never an ownership list or effective-authorization decision. Overflow is
/// explicit rather than silently omitting Workers from a cancellation/reporting pass.
pub fn list_subject_worker_execution_grants(
&self,
subject_id: &str,
) -> Result<Vec<WorkerExecutionGrant>> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
let mut statement = connection.prepare(
"SELECT record_json FROM worker_execution_grants WHERE subject_id=?1 ORDER BY runtime_id,worker_id LIMIT ?2",
)?;
let mut rows = statement.query(params![subject_id, MAX_SUBJECT_WORKER_EXECUTION_GRANTS + 1])?;
let mut grants = Vec::new();
while let Some(row) = rows.next()? {
if grants.len() == MAX_SUBJECT_WORKER_EXECUTION_GRANTS {
return Err(SubjektivError::WorkerExecutionGrantListOverflow {
subject_id: subject_id.into(),
limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS,
});
}
grants.push(serde_json::from_str(&row.get::<_, String>(0)?)?);
}
Ok(grants)
})
}
/// Returns retained binding evidence, not current authorization. The Host must compare the
/// active Subject, source change, current origin ownership, Profile and provider constraints.
pub fn worker_execution_grant(
&self,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
validate_authorization_identifier(runtime).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(worker).map_err(SubjektivError::InvalidRecord)?;
self.database
.try_with_connection(|connection| grant_in_connection(connection, runtime, worker))
}
}
fn source_in_connection(
connection: &Connection,
subject: &str,
) -> Result<Option<SubjectPermissions>> {
let raw: Option<String> = connection.query_row(
"SELECT c.record_json FROM subject_permissions p JOIN subject_permission_changes c ON c.subject_id=p.subject_id AND c.change_id=p.change_id WHERE p.subject_id=?1",
[subject], |row| row.get(0),
).optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn grant_in_connection(
connection: &Connection,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
let raw: Option<String> = connection
.query_row(
"SELECT record_json FROM worker_execution_grants WHERE runtime_id=?1 AND worker_id=?2",
params![runtime, worker],
|row| row.get(0),
)
.optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn validate_scope(scope: &str, permissions: &[ScopedPermission]) -> Result<()> {
validate_scoped_permissions(permissions).map_err(SubjektivError::InvalidRecord)?;
for permission in permissions {
if let PermissionResource::Workspace { workspace_id } = &permission.resource
&& workspace_id != scope
{
return Err(denied("Workspace permission is outside this store scope"));
}
}
Ok(())
}
fn validate_grant(scope: &str, grant: &WorkerExecutionGrant) -> Result<()> {
for value in [
&grant.subject_id,
&grant.origin_runtime_id,
&grant.origin_worker_id,
&grant.source_change_id,
&grant.runtime_id,
&grant.worker_id,
&grant.profile,
&grant.profile_digest,
] {
validate_authorization_identifier(value).map_err(SubjektivError::InvalidRecord)?;
}
if grant.origin_runtime_id == grant.runtime_id && grant.origin_worker_id == grant.worker_id {
return Err(denied("a child grant cannot originate from itself"));
}
validate_scope(scope, &grant.permissions)?;
validate_scope(scope, &grant.redelegation)
}
fn require_covered(ceiling: &[ScopedPermission], requested: &[ScopedPermission]) -> Result<()> {
if requested
.iter()
.all(|permission| ceiling.iter().any(|allowed| allowed.covers(permission)))
{
Ok(())
} else {
Err(denied(
"requested permission or redelegation exceeds its explicit ceiling",
))
}
}
fn require_retained_origin(
tx: &Transaction<'_>,
grant: &WorkerExecutionGrant,
) -> Result<Option<WorkerExecutionGrant>> {
let mut runtime = grant.origin_runtime_id.clone();
let mut worker = grant.origin_worker_id.clone();
let mut direct_origin = None;
let mut visited = HashSet::from([(grant.runtime_id.clone(), grant.worker_id.clone())]);
// Validate retained lineage only. Absence terminates at a Host-authenticated root;
// neither Journal active state nor absence of a grant proves current singleton ownership.
for _ in 0..64 {
if !visited.insert((runtime.clone(), worker.clone())) {
return Err(denied("cyclic grant origin"));
}
let foreign_attribution: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![runtime, worker, grant.subject_id], |row| row.get(0),
)?;
if foreign_attribution {
return Err(denied("grant origin is attributed to another Subject"));
}
let Some(origin) = grant_in_connection(tx, &runtime, &worker)? else {
return Ok(direct_origin);
};
if origin.subject_id != grant.subject_id
|| origin.source_change_id != grant.source_change_id
{
return Err(denied(
"grant origin belongs to another Subject or permission source",
));
}
if direct_origin.is_none() {
direct_origin = Some(origin.clone());
}
runtime = origin.origin_runtime_id;
worker = origin.origin_worker_id;
}
Err(denied("grant origin exceeds maximum delegation depth"))
}
fn denied(reason: &str) -> SubjektivError {
SubjektivError::AuthorizationDenied(reason.into())
}
#[cfg(test)]
mod tests;
+45
View File
@@ -0,0 +1,45 @@
CREATE TABLE subject_permission_changes (
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL CHECK (length(change_id) > 0),
record_json TEXT NOT NULL,
PRIMARY KEY (subject_id, change_id)
);
CREATE TABLE subject_permissions (
subject_id TEXT PRIMARY KEY REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE subject_permission_receipts (
operation_id TEXT PRIMARY KEY CHECK (length(operation_id) > 0),
subject_id TEXT NOT NULL,
request_fingerprint TEXT NOT NULL,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE worker_execution_grants (
runtime_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
source_change_id TEXT NOT NULL,
record_json TEXT NOT NULL,
PRIMARY KEY (runtime_id, worker_id),
FOREIGN KEY (subject_id, source_change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TRIGGER subject_permission_changes_no_update BEFORE UPDATE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are immutable');
END;
CREATE TRIGGER subject_permission_changes_no_delete BEFORE DELETE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are retained');
END;
CREATE TRIGGER subject_permission_receipts_no_update BEFORE UPDATE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are immutable');
END;
CREATE TRIGGER subject_permission_receipts_no_delete BEFORE DELETE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are retained');
END;
CREATE TRIGGER worker_execution_grants_no_update BEFORE UPDATE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are immutable');
END;
CREATE TRIGGER worker_execution_grants_no_delete BEFORE DELETE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are retained');
END;
+656
View File
@@ -0,0 +1,656 @@
//! Real SQLite component tests through store APIs; Host authentication, canonical singleton
//! ownership and live lineage checks are outside this suite. Root origins are Host-attested inputs;
//! Journal fixtures exercise defensive attribution only, never execution authority.
use super::*;
use crate::tests::{open_store, role};
use feature_storage::FeatureStorage;
use std::path::Path;
fn permission(operation: &str, path: &str, recursive: bool) -> ScopedPermission {
ScopedPermission {
operation: operation.into(),
resource: PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
},
}
}
fn update(
operation: &str,
expected: Option<&str>,
delegation: Vec<ScopedPermission>,
) -> SubjectPermissionsUpdate {
SubjectPermissionsUpdate {
operation_id: operation.into(),
expected_change_id: expected.map(Into::into),
execution: Vec::new(),
delegation,
}
}
fn grant(subject: &str, source: &str, worker: &str) -> WorkerExecutionGrant {
WorkerExecutionGrant {
subject_id: subject.into(),
origin_runtime_id: "runtime".into(),
origin_worker_id: "body".into(),
source_change_id: source.into(),
runtime_id: "runtime".into(),
worker_id: worker.into(),
profile: "builtin:coder".into(),
profile_digest: "resolved-profile-sha256".into(),
permissions: vec![permission("file.read", "src", false)],
redelegation: Vec::new(),
}
}
fn schema(store: &SubjektivStore) -> Vec<(String, String)> {
store.database.try_with_connection::<_, SubjektivError>(|connection| {
let mut statement = connection.prepare("SELECT name,sql FROM sqlite_master WHERE name NOT LIKE 'sqlite_%' AND sql IS NOT NULL ORDER BY name")?;
Ok(statement.query_map([], |row| Ok((row.get(0)?, row.get(1)?)))?.collect::<std::result::Result<_, _>>()?)
}).unwrap()
}
fn open_version(
root: &Path,
version: usize,
) -> (FeatureStorage, ScopedFeatureStorage, SubjektivStore) {
let manager = FeatureStorage::new(root);
let scope = manager.scope("workspace-a").unwrap();
let registration = scope
.register(FeatureRegistration::new(
SUBJEKTIV_FEATURE_ID,
&MIGRATIONS[..version],
))
.unwrap();
let store = SubjektivStore::open(&scope, &registration).unwrap();
(manager, scope, store)
}
#[test]
fn replacements_are_cas_fenced_replayable_and_never_restore_old_permissions() {
let temp = tempfile::tempdir().unwrap();
let subject;
let first;
let first_update = update("allow", None, vec![permission("file.read", ".", true)]);
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
subject = store.create_subject(role()).unwrap().id;
assert!(store.subject_permissions(&subject).unwrap().is_none());
first = store
.set_subject_permissions(&subject, &first_update)
.unwrap();
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert!(matches!(
store.set_subject_permissions(&subject, &update("stale", None, Vec::new())),
Err(SubjektivError::SubjectPermissionsConflict(_))
));
assert!(matches!(
store.set_subject_permissions(&subject, &update("allow", None, Vec::new())),
Err(SubjektivError::PermissionOperationConflict(_))
));
let other = store.create_subject(role()).unwrap();
assert!(matches!(
store.set_subject_permissions(&other.id, &first_update),
Err(SubjektivError::PermissionOperationConflict(_))
));
let same = store
.set_subject_permissions(
&subject,
&update("same", Some(&first.change_id), first.delegation.clone()),
)
.unwrap();
assert_ne!(same.change_id, first.change_id);
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&same.change_id), Vec::new()),
)
.unwrap();
assert!(revoked.delegation.is_empty());
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(revoked));
}
let (_reopened_manager, _reopened_scope, reopened) = open_store(temp.path(), "workspace-a");
assert!(
reopened
.subject_permissions(&subject)
.unwrap()
.unwrap()
.delegation
.is_empty()
);
assert_eq!(
reopened
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
}
#[test]
fn failed_receipt_insert_rolls_back_source_history_and_projection() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let first = store
.set_subject_permissions(&subject, &update("first", None, Vec::new()))
.unwrap();
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_permission_receipt BEFORE INSERT ON subject_permission_receipts BEGIN SELECT RAISE(ABORT,'injected receipt failure'); END;")?;
Ok(())
}).unwrap();
let replacement = update(
"second",
Some(&first.change_id),
vec![permission("file.read", ".", true)],
);
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_err()
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(first));
store.database.try_transaction::<_, SubjektivError>(|tx| {
let counts: (u32, u32) = tx.query_row("SELECT (SELECT count(*) FROM subject_permission_changes),(SELECT count(*) FROM subject_permission_receipts)", [], |r| Ok((r.get(0)?, r.get(1)?)))?;
assert_eq!(counts, (1, 1));
tx.execute_batch("DROP TRIGGER fail_permission_receipt")?;
Ok(())
}).unwrap();
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_ok()
);
}
#[test]
fn scope_and_validation_rejections_do_not_issue_permission_sources() {
let temp = tempfile::tempdir().unwrap();
let (_a_manager, _a_scope, a) = open_store(&temp.path().join("a"), "workspace-a");
let (_b_manager, _b_scope, b) = open_store(&temp.path().join("b"), "workspace-b");
let subject = a.create_subject(role()).unwrap().id;
assert!(
b.set_subject_permissions(&subject, &update("cross-subject", None, Vec::new()))
.is_err()
);
let cross_workspace = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "workspace-b".into(),
},
};
assert!(
a.set_subject_permissions(
&subject,
&update("cross-workspace", None, vec![cross_workspace])
)
.is_err()
);
for (operation, path) in [
("file.*", "src"),
("file..read", "src"),
("file.read", "../src"),
("file.read", "src/../escape"),
] {
assert!(
a.set_subject_permissions(
&subject,
&update("invalid", None, vec![permission(operation, path, true)])
)
.is_err()
);
}
assert!(a.subject_permissions(&subject).unwrap().is_none());
}
#[test]
fn child_grants_are_immutable_bounded_and_do_not_create_subject_ownership() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", "src", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&child).unwrap();
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child.clone())
);
assert!(
store
.journal_body_is_unpublished(&subject, "runtime", "child")
.unwrap()
);
let mut changed = child.clone();
changed.profile_digest = "changed-profile".into();
assert!(matches!(
store.bind_worker_execution_grant(&changed),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
for (worker, requested) in [
("write", permission("file.write", "src", false)),
("escape", permission("file.read", ".", true)),
("prefix", permission("file.read", "src-other", false)),
] {
let mut candidate = grant(&subject, &source.change_id, worker);
candidate.permissions = vec![requested];
assert!(store.bind_worker_execution_grant(&candidate).is_err());
assert!(
store
.worker_execution_grant("runtime", worker)
.unwrap()
.is_none()
);
}
let mut self_origin = child.clone();
self_origin.worker_id = "body".into();
assert!(store.bind_worker_execution_grant(&self_origin).is_err());
let other = store.create_subject(role()).unwrap();
store
.claim_journal_writer(&other.id, "runtime", "other-body")
.unwrap();
let mut invalid = grant(&subject, &source.change_id, "other-body");
assert!(store.bind_worker_execution_grant(&invalid).is_err());
invalid.worker_id = "forged-child".into();
invalid.origin_worker_id = "other-body".into();
assert!(store.bind_worker_execution_grant(&invalid).is_err());
}
#[test]
fn redelegation_requires_current_source_and_explicit_retained_parent_ceiling() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = vec![permission("file.read", "src", true)];
store.bind_worker_execution_grant(&child).unwrap();
let mut descendant = grant(&subject, &source.change_id, "grandchild");
descendant.origin_worker_id = "child".into();
store.bind_worker_execution_grant(&descendant).unwrap();
let mut excessive = descendant.clone();
excessive.worker_id = "excessive".into();
excessive.redelegation = vec![permission("file.read", ".", true)];
assert!(store.bind_worker_execution_grant(&excessive).is_err());
let mut nondelegator = descendant.clone();
nondelegator.origin_worker_id = "grandchild".into();
nondelegator.worker_id = "great-grandchild".into();
assert!(store.bind_worker_execution_grant(&nondelegator).is_err());
let mut stale = descendant.clone();
stale.worker_id = "after-source-replacement".into();
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
assert_ne!(revoked.change_id, source.change_id);
stale.origin_worker_id = "new-body".into();
assert!(store.bind_worker_execution_grant(&stale).is_err());
// Exact retries/readback recover evidence but cannot revive the source or origin.
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
store.retire_subject(&subject).unwrap();
stale.source_change_id = revoked.change_id;
assert!(store.bind_worker_execution_grant(&stale).is_err());
}
#[test]
fn journal_state_neither_authenticates_roots_nor_bypasses_retained_child_ceilings() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
// The Host has authenticated root ownership outside this store. No Journal record is
// required; changing Journal writer state cannot supply or invalidate that authentication.
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "without-journal"))
.unwrap();
let journal = store
.claim_journal_writer(&subject, "runtime", "body")
.unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-active-journal"))
.unwrap();
store.release_journal_writer(&journal).unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-released-journal"))
.unwrap();
// An active Journal binding must never upgrade a child's empty redelegation ceiling.
store
.claim_journal_writer(&subject, "runtime", "without-journal")
.unwrap();
let mut descendant = grant(&subject, &source.change_id, "descendant");
descendant.origin_worker_id = "without-journal".into();
assert!(matches!(
store.bind_worker_execution_grant(&descendant),
Err(SubjektivError::AuthorizationDenied(_))
));
assert!(
store
.worker_execution_grant("runtime", "descendant")
.unwrap()
.is_none()
);
}
#[test]
fn retained_origins_reject_cycles_foreign_subjects_and_stale_parent_sources() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = source.delegation.clone();
store.bind_worker_execution_grant(&child).unwrap();
// Closing a retained chain back through its previously unbound root must not create a cycle.
let mut cycle = grant(&subject, &source.change_id, "body");
cycle.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&cycle),
Err(SubjektivError::AuthorizationDenied(_))
));
let other = store.create_subject(role()).unwrap().id;
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let mut foreign = grant(&other, &other_source.change_id, "foreign-child");
foreign.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&foreign),
Err(SubjektivError::AuthorizationDenied(_))
));
let current = store
.set_subject_permissions(
&subject,
&update("replace", Some(&source.change_id), source.delegation),
)
.unwrap();
let mut stale_parent = grant(&subject, &current.change_id, "stale-parent-child");
stale_parent.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&stale_parent),
Err(SubjektivError::AuthorizationDenied(_))
));
}
#[test]
fn failed_grant_insert_leaves_no_binding_and_can_retry_same_grant() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_grant BEFORE INSERT ON worker_execution_grants BEGIN SELECT RAISE(ABORT,'injected grant failure'); END;")?;
Ok(())
}).unwrap();
assert!(store.bind_worker_execution_grant(&child).is_err());
assert!(
store
.worker_execution_grant("runtime", "child")
.unwrap()
.is_none()
);
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("DROP TRIGGER fail_grant")?;
Ok(())
})
.unwrap();
store.bind_worker_execution_grant(&child).unwrap();
for sql in [
"UPDATE worker_execution_grants SET source_change_id='forged'",
"DELETE FROM worker_execution_grants",
"UPDATE subject_permission_changes SET record_json='{}'",
"DELETE FROM subject_permission_receipts",
] {
assert!(
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch(sql)?;
Ok(())
})
.is_err(),
"{sql}"
);
}
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
}
#[test]
fn current_schema_migration_preserves_subject_and_matches_new_database() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("migrated");
let original;
{
let (_old_manager, _old_scope, old) = open_version(&root, 11);
original = old
.create_subject_with_behavior(role(), "Keep behavior separate".into())
.unwrap();
}
let (_migrated_manager, _migrated_scope, migrated) = open_store(&root, "workspace-a");
assert_eq!(
migrated.subject(&original.id).unwrap(),
Some(original.clone())
);
assert!(
migrated
.subject_permissions(&original.id)
.unwrap()
.is_none()
);
let (_fresh_manager, _fresh_scope, fresh) =
open_store(&temp.path().join("fresh"), "workspace-a");
assert_eq!(schema(&migrated), schema(&fresh));
let source = migrated
.set_subject_permissions(&original.id, &update("new-source", None, Vec::new()))
.unwrap();
assert!(!source.change_id.is_empty());
assert_eq!(migrated.subject(&original.id).unwrap(), Some(original));
}
#[test]
fn unset_or_execution_only_source_cannot_issue_child_grants_and_saved_grants_survive_reopen() {
let temp = tempfile::tempdir().unwrap();
let saved;
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
assert!(
store
.bind_worker_execution_grant(&grant(&subject, "unset", "child"))
.is_err()
);
let mut execution_only = update("execution", None, Vec::new());
execution_only.execution = vec![permission("file.read", ".", true)];
let source = store
.set_subject_permissions(&subject, &execution_only)
.unwrap();
assert!(
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "child"))
.is_err()
);
let source = store
.set_subject_permissions(
&subject,
&update(
"delegation",
Some(&source.change_id),
execution_only.execution,
),
)
.unwrap();
saved = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&saved).unwrap();
}
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(saved.clone())
);
store.bind_worker_execution_grant(&saved).unwrap();
let mut conflicting = saved;
conflicting.profile = "builtin:default".into();
assert!(matches!(
store.bind_worker_execution_grant(&conflicting),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
}
#[test]
fn subject_grant_listing_is_scoped_retained_ordered_and_fails_on_overflow() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let other = store.create_subject(role()).unwrap().id;
assert!(
store
.list_subject_worker_execution_grants(&subject)
.unwrap()
.is_empty()
);
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let other_grant = grant(&other, &other_source.change_id, "other-child");
store.bind_worker_execution_grant(&other_grant).unwrap();
for index in (0..MAX_SUBJECT_WORKER_EXECUTION_GRANTS).rev() {
store
.bind_worker_execution_grant(&grant(
&subject,
&source.change_id,
&format!("child-{index:04}"),
))
.unwrap();
}
store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
store.retire_subject(&subject).unwrap();
let listed = store
.list_subject_worker_execution_grants(&subject)
.unwrap();
assert_eq!(listed.len(), MAX_SUBJECT_WORKER_EXECUTION_GRANTS);
assert!(
listed
.iter()
.all(|grant| grant.subject_id == subject && grant.source_change_id == source.change_id)
);
assert!(
listed
.windows(2)
.all(|pair| (&pair[0].runtime_id, &pair[0].worker_id)
< (&pair[1].runtime_id, &pair[1].worker_id))
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant.clone()]
);
// Seed one more retained row directly: the retired Subject cannot issue new grants.
// Listing must report overflow rather than returning a seemingly complete cancellation set.
let overflow = grant(&subject, &source.change_id, "overflow");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![overflow.runtime_id, overflow.worker_id, overflow.subject_id, overflow.source_change_id, serde_json::to_string(&overflow)?])?;
Ok(())
}).unwrap();
assert!(
matches!(store.list_subject_worker_execution_grants(&subject), Err(SubjektivError::WorkerExecutionGrantListOverflow { subject_id, limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS }) if subject_id == subject)
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant]
);
let (_foreign_manager, _foreign_scope, foreign) =
open_store(&temp.path().join("foreign"), "workspace-b");
assert!(matches!(
foreign.list_subject_worker_execution_grants(&subject),
Err(SubjektivError::SubjectNotFound(_))
));
}
#[test]
fn failed_schema_migration_rolls_back_new_tables_and_preserves_old_data() {
let temp = tempfile::tempdir().unwrap();
let original;
{
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
original = old.create_subject(role()).unwrap();
old.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TABLE subject_permissions (collision TEXT)")?;
Ok(())
})
.unwrap();
}
{
let manager = FeatureStorage::new(temp.path());
let scope = manager.scope("workspace-a").unwrap();
let registration = SubjektivStore::register(&scope).unwrap();
assert!(SubjektivStore::open(&scope, &registration).is_err());
}
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
assert_eq!(old.database.schema_version().unwrap(), 11);
assert_eq!(old.subject(&original.id).unwrap(), Some(original));
old.database.try_with_connection::<_, SubjektivError>(|connection| {
let created: bool = connection.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE name='subject_permission_changes')", [], |row| row.get(0))?;
assert!(!created);
Ok(())
}).unwrap();
}
+2 -1
View File
@@ -23,7 +23,8 @@ thiserror = { workspace = true }
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tracing = { workspace = true }
workdir = { workspace = true }
uuid = { workspace = true, features = ["v7"] }
[dev-dependencies]
filetime = "0.2.27"
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "test-util"] }
+277 -212
View File
@@ -1,19 +1,19 @@
//! Bash Tool using WorkdirSession command capability.
//!
//! The selected session owns cwd, process launch, bounded output, timeout, and cancellation.
//! Workdir owns launch and byte transport; Bash owns bounded preview and retained evidence.
//! Bash is not a sandbox; the Runtime process and OS user define the execution boundary.
//! Per-execution handles and nonces prevent cancellation of another command. Dropping a future
//! does not confirm termination; active registrations remain until cleanup requests cancellation
//! and observes terminal output.
//! does not confirm termination. Registrations remain during bounded cancellation/drain cleanup;
//! an unconfirmed provider outcome leaves partial evidence, never a fabricated terminal fence.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use crate::bash_output::{BashOutputMetadata, BashOutputView, BashOutputWriter};
use agen::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use async_trait::async_trait;
use schemars::JsonSchema;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
use workdir::{
CommandHandle, CommandOutputRequest, CommandRequest, WorkdirPath, WorkdirSessionHandle,
WorkdirSessionRouter,
@@ -35,9 +35,9 @@ struct BashParams {
timeout: Option<u64>,
}
pub(crate) struct BashTool {
pub struct BashTool {
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
state: Arc<Mutex<BashExecutionState>>,
}
@@ -63,6 +63,8 @@ struct CommandGuard {
execution_id: String,
execution_nonce: u64,
handle: Option<CommandHandle>,
cursor: u64,
retained: Option<BashOutputWriter>,
}
impl Drop for CommandGuard {
@@ -74,20 +76,66 @@ impl Drop for CommandGuard {
let state = Arc::clone(&self.state);
let execution_id = self.execution_id.clone();
let execution_nonce = self.execution_nonce;
let mut retained = self.retained.take();
let cursor = self.cursor;
// A dropped provider future is not terminal confirmation. Keep the live
// execution registered until cleanup has both requested cancellation and
// observed terminal command output, so cancellation/session teardown
// cannot race with an apparently empty registry.
tokio::spawn(async move {
let _ = workdir.cancel_command(handle.clone()).await;
let _ = workdir
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
.await;
let result = tokio::time::timeout(std::time::Duration::from_secs(10), async {
let _ = workdir.cancel_command(handle.clone()).await;
let mut cursor = cursor;
loop {
let page = command_page(
&workdir,
CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
)
.await?;
if let Some(writer) = &mut retained {
let mut next = cursor;
for chunk in &page.chunks {
if chunk.sequence != next {
return Err(workdir::WorkdirError::Unavailable(
"command cleanup sequence gap".into(),
));
}
writer
.append_chunk(chunk)
.map_err(workdir::WorkdirError::Unavailable)?;
next = next.checked_add(chunk.bytes.len() as u64).ok_or_else(|| {
workdir::WorkdirError::Unavailable(
"command cleanup cursor overflow".into(),
)
})?;
}
if page.next_cursor != next {
return Err(workdir::WorkdirError::Unavailable(
"command cleanup cursor gap".into(),
));
}
}
cursor = page.next_cursor;
if let Some(terminal) = page.terminal {
if let Some(writer) = &mut retained {
writer
.finish_transport(&terminal)
.map_err(workdir::WorkdirError::Unavailable)?;
}
return Ok::<(), workdir::WorkdirError>(());
}
}
})
.await;
// Bounded cleanup can end with an unknown provider outcome. The writer then
// terminalizes partial evidence on drop; retaining a dead collector in `active`
// would leak registrations and falsely suggest it could still collect output.
let _ = result;
let mut state = state.lock().unwrap();
if state
.active
@@ -101,15 +149,53 @@ impl Drop for CommandGuard {
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
// Output pages are replayable only with the exact same command/cursor/limit.
// Retry read transport failures, never command start, and cap both attempts and
// backoff. The caller's deadline bounds a disconnected or unresponsive provider.
async fn command_page(
session: &workdir::WorkdirSessionHandle,
request: CommandOutputRequest,
) -> Result<workdir::CommandOutput, workdir::WorkdirError> {
for attempt in 0..3 {
match session.command_output(request.clone()).await {
Err(error)
if attempt < 2
&& matches!(
error.classification_source(),
workdir::WorkdirError::Transport(_)
| workdir::WorkdirError::Unavailable(_)
| workdir::WorkdirError::OutcomeUnknown(_)
) =>
{
tokio::time::sleep(std::time::Duration::from_millis(100 << attempt)).await;
}
result => return result,
}
}
unreachable!("bounded retry loop returns its last result")
}
impl BashTool {
pub fn input_schema() -> serde_json::Value {
serde_json::to_value(schemars::schema_for!(BashParams)).expect("Bash schema serialization")
}
pub fn new(router: Arc<WorkdirSessionRouter>, output: BashOutputView) -> Self {
Self {
router,
output,
state: Arc::new(Mutex::new(BashExecutionState::default())),
}
}
pub async fn execute_result(
&self,
input_json: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
) -> Result<BashResult, ToolError> {
let params: BashParams = serde_json::from_str(input_json)
.map_err(|error| ToolError::InvalidArgument(format!("invalid Bash input: {error}")))?;
let retained = self.output.begin().map_err(ToolError::ExecutionFailed)?;
let selected = crate::routing::resolve_session(
&self.router,
params.target_workdir.as_deref(),
@@ -134,14 +220,15 @@ impl Tool for BashTool {
execution_id: execution_id.clone(),
execution_nonce,
handle: None,
cursor: 0,
retained: Some(retained),
};
let handle = session
.start_command(CommandRequest {
command: params.command,
timeout_secs,
output_limit: INLINE_BYTE_BUDGET,
transport_byte_budget: self.output.byte_budget(),
cwd: params.cwd,
spill_dir: Some(self.output_dir.clone()),
tool_call_id: Some(call_id.clone()),
})
.await
@@ -167,15 +254,101 @@ impl Tool for BashTool {
.await
.map_err(crate::ToolsError::from)?;
}
let output = session
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
let mut cursor = 0;
let mut stdout_preview = Vec::new();
let mut stderr_preview = Vec::new();
let mut seen = [0_u64; 2];
let mut empty_polls = 0;
let deadline =
tokio::time::Instant::now() + std::time::Duration::from_secs(timeout_secs + 30);
let terminal = loop {
let page = tokio::time::timeout_at(
deadline,
command_page(
&session,
CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
),
)
.await
.map_err(|_| {
ToolError::ExecutionFailed(
"command byte transport deadline exceeded; terminal outcome unknown".into(),
)
})?
.map_err(crate::ToolsError::from)?;
let mut next = cursor;
for chunk in &page.chunks {
if chunk.sequence != next || chunk.bytes.is_empty() {
return Err(ToolError::ExecutionFailed(
"command byte transport sequence gap".into(),
));
}
next = next
.checked_add(chunk.bytes.len() as u64)
.filter(|n| *n <= self.output.byte_budget())
.ok_or_else(|| {
ToolError::ExecutionFailed(
"command byte transport exceeds delivery budget".into(),
)
})?;
let index = if chunk.stream == workdir::CommandStream::Stdout {
0
} else {
1
};
if chunk.offset != seen[index] {
return Err(ToolError::ExecutionFailed(
"command byte transport offset gap".into(),
));
}
seen[index] += chunk.bytes.len() as u64;
guard
.retained
.as_mut()
.unwrap()
.append_chunk(chunk)
.map_err(ToolError::ExecutionFailed)?;
let available =
INLINE_BYTE_BUDGET.saturating_sub(stdout_preview.len() + stderr_preview.len());
let preview = if index == 0 {
&mut stdout_preview
} else {
&mut stderr_preview
};
preview.extend_from_slice(&chunk.bytes[..available.min(chunk.bytes.len())]);
}
if page.next_cursor != next {
return Err(ToolError::ExecutionFailed(
"command byte transport cursor does not fence the page".into(),
));
}
if page.chunks.is_empty() && page.terminal.is_none() {
empty_polls += 1;
if empty_polls > 32 {
return Err(ToolError::ExecutionFailed(
"command byte transport made no progress after bounded polls".into(),
));
}
} else {
empty_polls = 0;
}
cursor = page.next_cursor;
guard.cursor = cursor;
if let Some(terminal) = page.terminal {
break terminal;
}
};
let metadata = guard
.retained
.as_mut()
.unwrap()
.finish_transport(&terminal)
.map_err(ToolError::ExecutionFailed)?;
let cancellation_requested = {
let mut state = self.state.lock().unwrap();
let owns_registration = state
@@ -193,41 +366,78 @@ impl Tool for BashTool {
};
guard.handle = None;
let timed_out = output.timed_out;
let summary = if cancellation_requested {
format!("$ {cmd_summary} (cancelled)")
} else if output.timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match output.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
let content = if output.content.is_empty() {
None
} else if output.truncated {
let notice = match output.output_path {
Some(path) => format!(
"[showing bounded WorkdirSession command output; full output saved to {}]",
path.display()
),
None => "[showing bounded WorkdirSession command output; additional output was truncated]"
.to_owned(),
let timed_out = terminal.status == workdir::CommandStatus::TimedOut;
let summary =
if cancellation_requested || terminal.status == workdir::CommandStatus::Cancelled {
format!("$ {cmd_summary} (cancelled)")
} else if timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match terminal.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
Some(format!("{notice}\n{}", output.content))
} else {
Some(output.content)
};
let output = ToolOutput {
Ok(BashResult {
summary,
content,
attachments: Vec::new(),
};
if cancellation_requested {
stdout_preview: String::from_utf8_lossy(&stdout_preview).into_owned(),
stderr_preview: String::from_utf8_lossy(&stderr_preview).into_owned(),
preview_omitted: terminal.stdout_bytes.saturating_add(terminal.stderr_bytes)
> (stdout_preview.len() + stderr_preview.len()) as u64,
metadata,
cancelled: cancellation_requested
|| terminal.status == workdir::CommandStatus::Cancelled,
})
}
}
#[derive(Debug, Serialize)]
pub struct BashResult {
pub summary: String,
pub stdout_preview: String,
pub stderr_preview: String,
pub preview_omitted: bool,
pub metadata: BashOutputMetadata,
pub cancelled: bool,
}
impl BashResult {
pub fn stdout_entry(&self) -> String {
format!("/bash-output/{}/stdout", self.metadata.id)
}
pub fn stderr_entry(&self) -> String {
format!("/bash-output/{}/stderr", self.metadata.id)
}
pub fn order_entry(&self) -> String {
format!("/bash-output/{}/order", self.metadata.id)
}
pub fn tool_output(&self) -> ToolOutput {
let content = serde_json::json!({
"stdout_preview": self.stdout_preview, "stderr_preview": self.stderr_preview,
"preview_omitted": self.preview_omitted, "output": self.metadata,
"stdout_entry": self.stdout_entry(), "stderr_entry": self.stderr_entry(), "order_entry": self.order_entry(),
});
ToolOutput {
summary: self.summary.clone(),
content: Some(content.to_string()),
attachments: vec![],
}
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
&self,
input: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
let result = self.execute_result(input, ctx).await?;
let output = result.tool_output();
if result.cancelled {
Err(ToolError::Cancelled(output))
} else if timed_out {
} else if result.metadata.terminal.status == Some(workdir::CommandStatus::TimedOut) {
Err(ToolError::Interrupted(output))
} else {
Ok(output)
@@ -288,164 +498,19 @@ fn truncate_for_summary(command: &str) -> String {
summary
}
pub fn bash_tool(session: WorkdirSessionHandle, output_dir: PathBuf) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output_dir)
pub fn bash_tool(session: WorkdirSessionHandle, output: BashOutputView) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output)
}
pub(crate) fn routed_bash_tool(
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
) -> ToolDefinition {
Arc::new(move || {
let schema = schemars::schema_for!(BashParams);
let meta = ToolMeta::new("Bash")
.description("Execute a shell command in one selected Workdir. Process start, bounded inline output, full-output spill, timeout and cancellation are owned by that WorkdirSession provider. This is not a sandbox.")
.input_schema(serde_json::to_value(schema).expect("Bash schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(BashTool {
router: router.clone(),
output_dir: output_dir.clone(),
state: Arc::new(Mutex::new(BashExecutionState::default())),
});
.description("Execute a shell command in one selected Workdir. Workdir owns launch, lossless bounded byte transport, timeout and cancel; Bash owns preview, quota-limited ephemeral output and read-only virtual references. This is not a sandbox.")
.input_schema(BashTool::input_schema());
let tool: Arc<dyn Tool> = Arc::new(BashTool::new(router.clone(), output.clone()));
(meta, tool)
})
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use tempfile::TempDir;
use workdir::{LocalWorkdirSession, WorkdirSessionHandle};
use super::bash_tool;
use crate::{grep::grep_tool, read::read_tool, tracker::Tracker};
fn session_with_output_scope(root: &TempDir, output: &TempDir) -> WorkdirSessionHandle {
let scope = Scope::from_config(&ScopeConfig {
allow: vec![
ScopeRule {
target: root.path().to_path_buf(),
permission: Permission::Write,
recursive: true,
symlink_policy: Default::default(),
},
ScopeRule {
target: output.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
},
],
deny: Vec::new(),
})
.unwrap();
Arc::new(LocalWorkdirSession::new(scope, root.path().to_path_buf()))
}
#[tokio::test]
async fn long_output_is_spilled_and_available_to_read_and_grep() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session.clone(), output.path().to_path_buf())();
let command = "i=0; while [ $i -lt 2000 ]; do printf 'line-%04d\\n' \"$i\"; i=$((i+1)); done; printf 'FINAL-NEEDLE\\n'";
let result = bash
.execute(
&serde_json::json!({ "command": command, "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
let rendered = result.content.expect("bounded Bash output");
let artifact = std::fs::read_dir(output.path())
.unwrap()
.next()
.expect("artifact entry")
.unwrap()
.path();
assert!(rendered.contains("full output saved to"));
assert!(rendered.contains(&artifact.display().to_string()));
let retained = std::fs::read_to_string(&artifact).unwrap();
assert!(retained.starts_with("line-0000\n"));
assert!(retained.ends_with("FINAL-NEEDLE\n"));
assert_eq!(retained.lines().count(), 2001);
let (_, read) = read_tool(session.clone(), Tracker::new())();
let read_result = read
.execute(
&serde_json::json!({
"file_path": artifact,
"offset": 2000,
"limit": 1,
})
.to_string(),
Default::default(),
)
.await
.unwrap();
assert!(
read_result
.content
.expect("Read content")
.contains("FINAL-NEEDLE")
);
let (_, grep) = grep_tool(session)();
let grep_result = grep
.execute(
&serde_json::json!({
"pattern": "FINAL-NEEDLE",
"path": artifact,
"output_mode": "content",
})
.to_string(),
Default::default(),
)
.await
.unwrap();
let grep_content = grep_result.content.expect("Grep content");
assert!(
grep_content.contains("FINAL-NEEDLE"),
"unexpected Grep content: {grep_content:?}"
);
}
#[tokio::test]
async fn missing_cwd_is_rejected_before_command_start() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let error = bash
.execute(
&serde_json::json!({ "command": "printf hidden" }).to_string(),
Default::default(),
)
.await
.unwrap_err();
assert!(error.to_string().contains("missing field `cwd`"));
}
#[tokio::test]
async fn short_output_does_not_leave_a_spill_artifact() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let result = bash
.execute(
&serde_json::json!({ "command": "printf short", "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
assert_eq!(result.content.as_deref(), Some("short"));
assert_eq!(std::fs::read_dir(output.path()).unwrap().count(), 0);
}
}
+605
View File
@@ -0,0 +1,605 @@
//! Bash-owned bounded output, independent of Workdir transport and Console subscribers.
//!
//! Output is ephemeral evidence, not a filesystem grant. Every lookup checks the live view;
//! children can see their own subtree and ancestors can retain evidence after child revocation.
//! Root teardown clears all bytes. A new root (including restore) has a new incarnation and
//! never reconstructs grants from history. Expiry is measured from admission, enforced on
//! lookup and by a weak reaper once commands run in Tokio. Parent and all descendants share
//! one pool: admission reserves a whole command budget, evicts the oldest terminal evidence
//! first, and refuses rather than evicting a live collector. Revoking a child denies access
//! but leaves ancestor evidence until pool eviction/expiry; root teardown clears the pool.
use std::collections::VecDeque;
use std::sync::{
Arc, Mutex, Weak,
atomic::{AtomicBool, Ordering},
};
use std::time::{Duration, Instant};
use serde::Serialize;
use workdir::{CommandStatus, CommandStream};
const COMMAND_BYTES: usize = 4 * 1024 * 1024;
const STORE_BYTES: usize = 32 * 1024 * 1024;
const STORE_COUNT: usize = 64;
const ORDER_COUNT: usize = 4096;
const RETENTION: Duration = Duration::from_secs(3600);
#[derive(Clone)]
pub struct BashOutputView {
store: Arc<Mutex<Store>>,
lineage: Vec<String>,
active: Arc<AtomicBool>,
authority: Arc<dyn Fn() -> bool + Send + Sync>,
}
struct Store {
records: VecDeque<Arc<Mutex<Record>>>,
reserved: usize,
retention: Duration,
reaper_started: bool,
_shutdown: tokio::sync::watch::Sender<()>,
}
struct Record {
id: String,
owner: Vec<String>,
created: Instant,
stdout: Vec<u8>,
stderr: Vec<u8>,
order: Vec<OutputRange>,
stdout_seen: u64,
stderr_seen: u64,
truncated: bool,
limit: Option<RetentionLimit>,
transport_counts: Option<(u64, u64)>,
transport_truncation: Option<workdir::CommandOutputTruncation>,
terminal: Option<OutputTerminal>,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputRange {
pub stream: CommandStream,
pub start: u64,
pub end: u64,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputTerminal {
pub status: Option<CommandStatus>,
pub exit_code: Option<i32>,
/// False on a forcibly dropped collector or failed transport: received bytes are retained,
/// but they are not a claim about the unobserved process tail.
pub transport_complete: bool,
}
#[derive(Clone, Copy, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum RetentionLimit {
ByteQuota,
OrderQuota,
}
#[derive(Clone, Debug, Serialize)]
pub struct BashOutputMetadata {
pub id: String,
pub stdout_saved: usize,
pub stderr_saved: usize,
pub stdout_seen: u64,
pub stderr_seen: u64,
pub stdout_delivered: u64,
pub stderr_delivered: u64,
pub retention_limit: Option<RetentionLimit>,
pub transport_truncation: Option<workdir::CommandOutputTruncation>,
pub storage_truncated: bool,
pub terminal: OutputTerminal,
pub text_encoding: &'static str,
pub ordering: &'static str,
}
/// A bounded reservation acquired before command start. Drop terminalizes partial evidence.
pub struct BashOutputWriter {
record: Arc<Mutex<Record>>,
}
impl Default for BashOutputView {
fn default() -> Self {
Self::new()
}
}
impl BashOutputView {
pub fn new() -> Self {
let (shutdown, _) = tokio::sync::watch::channel(());
let store = Arc::new(Mutex::new(Store {
records: VecDeque::new(),
reserved: 0,
retention: RETENTION,
reaper_started: false,
_shutdown: shutdown,
}));
let view = Self {
store,
lineage: vec![uuid::Uuid::now_v7().to_string()],
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(|| true),
};
view.start_reaper();
view
}
fn start_reaper(&self) {
let Ok(runtime) = tokio::runtime::Handle::try_current() else {
return;
};
let mut store = self.store.lock().unwrap();
if store.reaper_started {
return;
}
let receiver = store._shutdown.subscribe();
store.reaper_started = true;
let weak = Arc::downgrade(&self.store);
runtime.spawn(reap(weak, receiver));
}
pub fn byte_budget(&self) -> u64 {
COMMAND_BYTES as u64
}
pub fn child(&self, authority: Arc<dyn Fn() -> bool + Send + Sync>) -> Self {
let parent = self.clone();
let mut lineage = self.lineage.clone();
lineage.push(uuid::Uuid::now_v7().to_string());
Self {
store: self.store.clone(),
lineage,
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(move || parent.is_active() && authority()),
}
}
pub fn is_active(&self) -> bool {
self.active.load(Ordering::Acquire) && (self.authority)()
}
pub fn is_root(&self) -> bool {
self.lineage.len() == 1
}
pub fn revoke(&self) {
self.active.store(false, Ordering::Release);
if self.lineage.len() == 1 {
let mut store = self.store.lock().unwrap();
store.records.clear();
store.reserved = 0;
}
}
pub fn begin(&self) -> Result<BashOutputWriter, String> {
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
// A host may construct the Feature before entering Tokio. Command admission
// happens inside the runtime, so idle expiry must start here as well.
self.start_reaper();
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
store.expire(Instant::now());
while store.records.len() >= STORE_COUNT || store.reserved + COMMAND_BYTES > STORE_BYTES {
let Some(index) = store
.records
.iter()
.position(|r| r.lock().unwrap().terminal.is_some())
else {
return Err("Bash output capacity is occupied by live commands".into());
};
store.records.remove(index);
store.reserved -= COMMAND_BYTES;
}
let record = Arc::new(Mutex::new(Record {
id: uuid::Uuid::now_v7().to_string(),
owner: self.lineage.clone(),
created: Instant::now(),
stdout: Vec::new(),
stderr: Vec::new(),
order: Vec::new(),
stdout_seen: 0,
stderr_seen: 0,
truncated: false,
limit: None,
transport_counts: None,
transport_truncation: None,
terminal: None,
}));
store.records.push_back(record.clone());
store.reserved += COMMAND_BYTES;
Ok(BashOutputWriter { record })
}
fn record(&self, id: &str) -> Option<Arc<Mutex<Record>>> {
if !self.is_active() {
return None;
}
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return None;
}
store.expire(Instant::now());
store
.records
.iter()
.find(|r| {
let r = r.lock().unwrap();
r.id == id && r.owner.starts_with(&self.lineage) && r.terminal.is_some()
})
.cloned()
}
pub fn metadata(&self, id: &str) -> Option<BashOutputMetadata> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.metadata())
}
/// Copies at most the caller's bounded range; never publishes output bodies in descriptors.
pub fn read(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Vec<u8>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let start = offset.min(bytes.len());
Some(bytes[start..start.saturating_add(limit.min(64 * 1024)).min(bytes.len())].to_vec())
}
/// Scan the bounded raw source without cloning it, decoding only the requested line window.
/// A long single line uses the common byte reader instead of allocating an oversized text page.
pub fn line_window(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Result<(String, bool), String>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let mut lines = bytes.split_inclusive(|b| *b == b'\n').skip(offset);
let mut window = Vec::new();
for line in lines.by_ref().take(limit.min(2000)) {
if window.len() + line.len() > 64 * 1024 {
return Some(Err(
"text source window exceeds 64 KiB; use the common byte reader".into(),
));
}
window.extend_from_slice(line);
}
let more = lines.next().is_some();
Some(Ok((String::from_utf8_lossy(&window).into_owned(), more)))
}
pub fn order(&self, id: &str) -> Option<Vec<OutputRange>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.order.clone())
}
}
impl Record {
fn metadata(&self) -> BashOutputMetadata {
BashOutputMetadata {
id: self.id.clone(),
stdout_saved: self.stdout.len(),
stderr_saved: self.stderr.len(),
stdout_seen: self.transport_counts.map_or(self.stdout_seen, |c| c.0),
stderr_seen: self.transport_counts.map_or(self.stderr_seen, |c| c.1),
stdout_delivered: self.stdout_seen,
stderr_delivered: self.stderr_seen,
retention_limit: self.limit,
transport_truncation: self.transport_truncation.clone(),
storage_truncated: self.truncated || self.transport_truncation.is_some(),
terminal: self.terminal.clone().expect("finalized record"),
text_encoding: "utf8_lossy",
ordering: "provider_read_observation",
}
}
}
impl Store {
fn expire(&mut self, now: Instant) {
let before = self.records.len();
self.records
.retain(|r| now.duration_since(r.lock().unwrap().created) < self.retention);
self.reserved -= (before - self.records.len()) * COMMAND_BYTES;
}
}
async fn reap(weak: Weak<Mutex<Store>>, mut shutdown: tokio::sync::watch::Receiver<()>) {
loop {
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(30)) => {},
_ = shutdown.changed() => return,
}
let Some(store) = weak.upgrade() else {
return;
};
store.lock().unwrap().expire(Instant::now());
}
}
impl BashOutputWriter {
pub fn append_chunk(&mut self, chunk: &workdir::CommandByteChunk) -> Result<(), String> {
let expected = {
let r = self.record.lock().unwrap();
match chunk.stream {
CommandStream::Stdout => r.stdout_seen,
CommandStream::Stderr => r.stderr_seen,
}
};
if chunk.offset != expected || chunk.bytes.is_empty() {
return Err("command byte transport stream gap or empty chunk".into());
}
self.append(chunk.stream, &chunk.bytes);
Ok(())
}
pub fn append(&mut self, stream: CommandStream, bytes: &[u8]) {
let mut r = self.record.lock().unwrap();
let seen = match stream {
CommandStream::Stdout => &mut r.stdout_seen,
CommandStream::Stderr => &mut r.stderr_seen,
};
let start = *seen;
*seen = seen.saturating_add(bytes.len() as u64);
let available = COMMAND_BYTES.saturating_sub(r.stdout.len() + r.stderr.len());
let n = if r.truncated || r.order.len() >= ORDER_COUNT {
0
} else {
available.min(bytes.len())
};
if n < bytes.len() {
r.truncated = true;
if r.limit.is_none() {
r.limit = Some(if r.order.len() >= ORDER_COUNT {
RetentionLimit::OrderQuota
} else {
RetentionLimit::ByteQuota
});
}
}
if n > 0 {
match stream {
CommandStream::Stdout => r.stdout.extend_from_slice(&bytes[..n]),
CommandStream::Stderr => r.stderr.extend_from_slice(&bytes[..n]),
}
if let Some(last) = r
.order
.last_mut()
.filter(|last| last.stream == stream && last.end == start)
{
last.end += n as u64;
} else {
r.order.push(OutputRange {
stream,
start,
end: start + n as u64,
});
}
}
}
pub fn finish_transport(
&mut self,
terminal: &workdir::CommandTerminal,
) -> Result<BashOutputMetadata, String> {
let mut r = self.record.lock().unwrap();
if terminal.stdout_delivered_bytes != r.stdout_seen
|| terminal.stderr_delivered_bytes != r.stderr_seen
|| terminal.stdout_bytes < r.stdout_seen
|| terminal.stderr_bytes < r.stderr_seen
|| ((terminal.stdout_bytes != r.stdout_seen || terminal.stderr_bytes != r.stderr_seen)
!= terminal.truncation_reason.is_some())
{
return Err("command terminal counts do not fence delivered bytes".into());
}
r.transport_counts = Some((terminal.stdout_bytes, terminal.stderr_bytes));
r.transport_truncation = terminal.truncation_reason.clone();
r.terminal = Some(OutputTerminal {
status: Some(terminal.status),
exit_code: terminal.exit_code,
transport_complete: true,
});
Ok(r.metadata())
}
pub fn finish(&mut self, terminal: OutputTerminal) -> BashOutputMetadata {
let mut r = self.record.lock().unwrap();
r.terminal = Some(terminal);
r.metadata()
}
}
impl Drop for BashOutputWriter {
fn drop(&mut self) {
let mut r = self.record.lock().unwrap();
if r.terminal.is_none() {
r.terminal = Some(OutputTerminal {
status: None,
exit_code: None,
transport_complete: false,
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn finish(view: &BashOutputView, text: &[u8]) -> String {
let mut writer = view.begin().unwrap();
writer.append(CommandStream::Stdout, text);
writer
.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
})
.id
}
#[test]
fn revoked_child_is_denied_while_parent_keeps_evidence_and_sibling_is_denied() {
let root = BashOutputView::new();
let lease = Arc::new(AtomicBool::new(true));
let child = root.child({
let lease = lease.clone();
Arc::new(move || lease.load(Ordering::Acquire))
});
let sibling = root.child(Arc::new(|| true));
let id = finish(&child, b"child");
assert_eq!(
child.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
assert!(sibling.metadata(&id).is_none());
assert!(BashOutputView::new().metadata(&id).is_none());
lease.store(false, Ordering::Release);
assert!(child.metadata(&id).is_none());
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
root.revoke();
let restored = BashOutputView::new();
assert!(root.metadata(&id).is_none());
assert!(restored.metadata(&id).is_none());
let new_id = finish(&restored, b"new incarnation");
assert_ne!(new_id, id);
assert!(child.metadata(&new_id).is_none());
assert!(root.begin().is_err());
}
#[test]
fn parent_and_descendants_share_admission_without_evicting_live_evidence() {
let root = BashOutputView::new();
let child = root.child(Arc::new(|| true));
let sibling = root.child(Arc::new(|| true));
let mut writers = (0..STORE_BYTES / COMMAND_BYTES)
.map(|index| {
if index % 2 == 0 {
root.begin()
} else {
child.begin()
}
.unwrap()
})
.collect::<Vec<_>>();
assert!(sibling.begin().is_err());
let mut first = writers.remove(0);
first.append(CommandStream::Stdout, b"finished");
let metadata = first.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
});
assert!(root.metadata(&metadata.id).is_some());
let _replacement = sibling.begin().unwrap();
assert!(root.metadata(&metadata.id).is_none());
assert!(root.begin().is_err());
assert_eq!(writers.len(), STORE_BYTES / COMMAND_BYTES - 1);
}
#[test]
fn quota_and_forced_drop_are_distinct_from_preview_omission() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, &vec![b'x'; COMMAND_BYTES + 1]);
writer.append(CommandStream::Stderr, b"tail");
let id = writer.record.lock().unwrap().id.clone();
drop(writer);
let m = root.metadata(&id).unwrap();
assert_eq!(m.stdout_saved, COMMAND_BYTES);
assert_eq!(m.stdout_seen, COMMAND_BYTES as u64 + 1);
assert_eq!(m.stderr_seen, 4);
assert!(m.storage_truncated);
assert!(!m.terminal.transport_complete);
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, usize::MAX)
.unwrap()
.len(),
64 * 1024
);
}
#[test]
fn expiry_and_count_eviction_never_reuse_old_references() {
let root = BashOutputView::new();
let id = finish(&root, b"old");
root.store.lock().unwrap().retention = Duration::ZERO;
assert!(root.metadata(&id).is_none());
root.store.lock().unwrap().retention = RETENTION;
let first = finish(&root, b"first");
for _ in 0..STORE_COUNT {
finish(&root, b"later");
}
assert!(root.metadata(&first).is_none());
assert!(root.store.lock().unwrap().reserved <= STORE_BYTES);
}
#[tokio::test(start_paused = true)]
async fn expiry_reclaims_bytes_without_another_read_and_reaper_ends_with_owner() {
// Exercise host construction outside a runtime, followed by real async admission.
let root = std::thread::spawn(BashOutputView::new).join().unwrap();
finish(&root, b"expires without lookup");
let mut closed = root.store.lock().unwrap()._shutdown.subscribe();
tokio::task::yield_now().await;
// The store uses monotonic wall time for publication and Tokio time only for scheduling.
// Age the record deterministically rather than delaying this test in real time.
root.store.lock().unwrap().records[0]
.lock()
.unwrap()
.created = Instant::now() - RETENTION;
tokio::time::advance(Duration::from_secs(31)).await;
tokio::task::yield_now().await;
assert_eq!(root.store.lock().unwrap().reserved, 0);
drop(root);
assert!(closed.changed().await.is_err());
}
#[test]
fn terminal_counts_must_fence_the_received_prefix_before_marking_evidence_complete() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, b"abc");
let mut terminal = serde_json::json!({"status":"completed","exit_code":0,"stdout_bytes":9,"stderr_bytes":0,"stdout_delivered_bytes":4,"stderr_delivered_bytes":0,"truncation_reason":"byte_budget"});
assert!(
writer
.finish_transport(&serde_json::from_value(terminal.clone()).unwrap())
.is_err()
);
terminal["stdout_delivered_bytes"] = serde_json::json!(3);
let metadata = writer
.finish_transport(&serde_json::from_value(terminal).unwrap())
.unwrap();
assert_eq!(metadata.stdout_saved, 3);
assert_eq!(metadata.stdout_delivered, 3);
assert_eq!(metadata.stdout_seen, 9);
assert!(metadata.storage_truncated);
assert!(metadata.retention_limit.is_none());
assert_eq!(
metadata.transport_truncation,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
}
}
+68
View File
@@ -403,3 +403,71 @@ fn check_pattern(pattern: &str) -> Result<(), ToolError> {
#[cfg(test)]
#[path = "checkout_tests.rs"]
mod tests;
/// Read a bounded native text window through the same checked target.
/// The returned one-shot action records prior-Read only after the caller's presentation succeeds.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_text(
router: Arc<WorkdirSessionRouter>,
tracker: Tracker,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: usize,
limit: usize,
max_bytes: usize,
) -> Result<(workdir::ReadResult, Option<Vec<u8>>, impl FnOnce()), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let tracker = tracker.scoped_attachment(&selected.alias, selected.generation);
let target = FileTarget {
session: selected.session.clone(),
path: path.clone(),
validator: Some(validator),
};
let (result, validator) = target.read(offset, limit, max_bytes).await?;
let hash = result.content_hash.clone();
let total_lines = result.total_lines;
Ok((result, validator, move || {
tracker.record_workdir_observation(&path, hash, total_lines);
}))
}
/// Binary-safe checked byte window. This does not establish a text prior-Read observation.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_bytes(
router: Arc<WorkdirSessionRouter>,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: u64,
max_bytes: usize,
) -> Result<(workdir::ReadBytesResult, Vec<u8>), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let result = selected
.session
.checkout_execute(workdir::CheckoutRequest {
target: path.clone(),
validator,
operation: workdir::CheckoutOperation::ReadBytes { offset, max_bytes },
})
.await
.map_err(crate::file_target::checked_error)?;
match result.output {
workdir::CheckoutOutput::ReadBytes(read)
if read.path == path && read.offset == offset && read.bytes.len() <= max_bytes =>
{
Ok((read, result.observation.validator))
}
_ => Err(crate::file_target::readonly_result_error("ReadBytes")),
}
}
+3 -1
View File
@@ -147,7 +147,9 @@ impl FileTarget {
pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
use workdir::WorkdirError;
let code = match error.classification_source() {
WorkdirError::DenialContext { .. } => unreachable!("diagnostic context unwrapped"),
WorkdirError::DenialContext { .. } | WorkdirError::FailureContext { .. } => {
unreachable!("diagnostic context unwrapped")
}
WorkdirError::Conflict(_) | WorkdirError::NotFound(_) => "checkout_stale",
WorkdirError::Denied { .. }
| WorkdirError::OutOfScope { .. }
+10 -9
View File
@@ -5,6 +5,7 @@ pub mod error;
pub mod tracker;
mod bash;
pub mod bash_output;
mod edit;
mod file_target;
mod glob;
@@ -16,7 +17,7 @@ pub mod view_image;
mod web;
mod write;
pub use bash::bash_tool;
pub use bash::{BashResult, BashTool, bash_tool};
pub use checkout::{CheckoutToolOutput, execute_checkout_tool};
pub use edit::edit_tool;
pub use error::ToolsError;
@@ -35,7 +36,7 @@ pub use write::write_tool;
pub fn core_builtin_tools(
session: workdir::WorkdirSessionHandle,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -58,7 +59,7 @@ pub fn core_builtin_tools(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -71,7 +72,7 @@ pub fn core_builtin_tools(
pub fn routed_builtin_tools(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
vec![
read::routed_read_tool(router.clone(), tracker.clone()),
@@ -79,14 +80,14 @@ pub fn routed_builtin_tools(
edit::routed_edit_tool(router.clone(), tracker),
glob::routed_glob_tool(router.clone()),
grep::routed_grep_tool(router.clone()),
bash::routed_bash_tool(router, bash_output_dir),
bash::routed_bash_tool(router, bash_output),
]
}
pub fn routed_builtin_tools_for_current_capabilities(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -108,7 +109,7 @@ pub fn routed_builtin_tools_for_current_capabilities(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -121,7 +122,7 @@ pub fn read_only_builtin_tools(
workdir::WorkdirSessionCapabilities::READ_ONLY,
"read-only tool projection requires a read-only Workdir session"
);
core_builtin_tools(session, Tracker::new(), std::path::PathBuf::new())
core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
}
pub fn read_only_routed_builtin_tools(
@@ -173,7 +174,7 @@ mod workdir_tool_tests {
SharedScope::new(Scope::writable(dir.path()).unwrap()),
WorkdirSessionCapabilities::READ_ONLY,
));
let names = core_builtin_tools(session, Tracker::new(), dir.path().join("output"))
let names = core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
.into_iter()
.map(|definition| definition().0.name)
.collect::<Vec<_>>();
+44 -2
View File
@@ -360,10 +360,52 @@ mod tests {
}
#[tokio::test]
async fn bounded_binary_read_accepts_arbitrary_bytes_at_caller_limit_and_rejects_larger_files()
{
async fn bounded_binary_read_accepts_caller_limits_above_the_image_tool_limit() {
// The caller's limit is a reader policy, not a LocalWorkdir storage contract.
// Supply immutable chunks here; the separate component case exercises the
// real provider across a chunk boundary without repeatedly hashing 16 MiB.
let dir = TempDir::new().unwrap();
let bytes = vec![0xff; 16 * 1024 * 1024];
let hash = Sha256::digest(&bytes).into();
let responses = bytes
.chunks(IMAGE_READ_CHUNK_BYTES)
.enumerate()
.map(|(index, bytes)| {
chunk(
bytes,
(index * IMAGE_READ_CHUNK_BYTES) as u64,
16 * 1024 * 1024,
hash,
)
})
.collect();
let session = binary_session(&dir, responses);
let path = WorkdirPath::new("artifact.bin").unwrap();
assert_eq!(
read_workdir_bytes(&session, path.clone(), bytes.len())
.await
.unwrap(),
bytes
);
let session = binary_session(
&dir,
vec![chunk(
&bytes[..IMAGE_READ_CHUNK_BYTES],
0,
bytes.len() as u64,
hash,
)],
);
assert!(matches!(
read_workdir_bytes(&session, path, bytes.len() - 1).await,
Err(WorkdirError::InvalidArgument(_))
));
}
#[tokio::test]
async fn bounded_binary_read_reassembles_provider_chunks_and_enforces_the_caller_limit() {
let dir = TempDir::new().unwrap();
let bytes = vec![0xff; IMAGE_READ_CHUNK_BYTES + 1];
std::fs::write(dir.path().join("artifact.bin"), &bytes).unwrap();
let session =
LocalWorkdirSession::new(Scope::writable(dir.path()).unwrap(), dir.path().into());
+19 -27
View File
@@ -3,7 +3,7 @@
use std::sync::Arc;
use agen::tool::{Tool, ToolDefinition};
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use manifest::Scope;
use serde_json::json;
use tempfile::TempDir;
use tools::{Tracker, core_builtin_tools};
@@ -28,31 +28,23 @@ impl Registry {
}
}
fn setup() -> (TempDir, TempDir, Registry) {
fn setup() -> (TempDir, Registry) {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let base = Scope::writable(dir.path()).unwrap();
let mut config = ScopeConfig {
allow: base.allow_rules(),
deny: base.deny_rules(),
};
config.allow.push(ScopeRule {
target: spill.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
});
let scope = Scope::from_config(&config).unwrap();
let scope = Scope::writable(dir.path()).unwrap();
let fs: WorkdirSessionHandle =
std::sync::Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let tracker = Tracker::new();
let reg = Registry::new(core_builtin_tools(fs, tracker, spill.path().to_path_buf()));
(dir, spill, reg)
let reg = Registry::new(core_builtin_tools(
fs,
tracker,
tools::bash_output::BashOutputView::new(),
));
(dir, reg)
}
#[tokio::test]
async fn unicode_path_and_content() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("日本語ファイル.txt");
let content = "こんにちは 🦀 世界\nabc\n";
@@ -87,7 +79,7 @@ async fn unicode_path_and_content() {
async fn symlink_to_outside_scope_is_rejected_for_write() {
use std::os::unix::fs::symlink;
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let outside = TempDir::new().unwrap();
let outside_target = outside.path().join("secret.txt");
std::fs::write(&outside_target, "secret").unwrap();
@@ -151,7 +143,7 @@ async fn symlink_to_outside_scope_is_rejected_for_write() {
async fn broken_symlink_reports_target_and_repair_hint() {
use std::os::unix::fs::symlink;
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let link = dir.path().join("external-project");
let target = dir.path().join("missing-target");
symlink(&target, &link).unwrap();
@@ -173,7 +165,7 @@ async fn broken_symlink_reports_target_and_repair_hint() {
#[tokio::test]
async fn empty_file_read_and_edit() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("empty.txt");
std::fs::write(&file, "").unwrap();
@@ -207,7 +199,7 @@ async fn empty_file_read_and_edit() {
#[tokio::test]
async fn very_long_single_line() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("long.txt");
let big: String = "x".repeat(1024 * 1024); // 1 MiB, no newlines
std::fs::write(&file, &big).unwrap();
@@ -226,7 +218,7 @@ async fn very_long_single_line() {
#[tokio::test]
async fn absolute_path_requires_matching_read_scope() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let outside = tempfile::tempdir().unwrap();
let outside_file = outside.path().join("outside.txt");
std::fs::write(&outside_file, "secret").unwrap();
@@ -247,7 +239,7 @@ async fn absolute_path_requires_matching_read_scope() {
#[tokio::test]
async fn directory_target_is_rejected_for_read() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let read = reg.get("Read");
let err = read
.execute(&json!({ "file_path": "." }).to_string(), Default::default())
@@ -258,7 +250,7 @@ async fn directory_target_is_rejected_for_read() {
#[tokio::test]
async fn deeply_nested_new_file_is_created() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let deep = dir.path().join("a/b/c/d/e/deep.txt");
let write = reg.get("Write");
write
@@ -277,7 +269,7 @@ async fn deeply_nested_new_file_is_created() {
#[tokio::test]
async fn replace_preserves_unicode() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("u.txt");
std::fs::write(&file, "🦀 rust 🦀\n").unwrap();
@@ -306,7 +298,7 @@ async fn replace_preserves_unicode() {
#[tokio::test]
async fn grep_handles_unicode_pattern() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("u.txt");
std::fs::write(&file, "English\n日本語\nрусский\n").unwrap();
+350 -153
View File
@@ -11,7 +11,7 @@ use agen::tool::{
Tool, ToolDefinition, ToolError, ToolExecutionContext, ToolExecutionHandle,
ToolExecutionTerminal, ToolMeta,
};
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use manifest::Scope;
use serde_json::json;
use tempfile::TempDir;
use tools::{
@@ -23,21 +23,6 @@ use workdir::{
WorkdirToolScopeRule,
};
fn scope_with_spill(workspace: &Path, spill: &Path) -> Scope {
let base = Scope::writable(workspace).unwrap();
let mut config = ScopeConfig {
allow: base.allow_rules(),
deny: base.deny_rules(),
};
config.allow.push(ScopeRule {
target: spill.to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
});
Scope::from_config(&config).unwrap()
}
struct Registry {
entries: Vec<(ToolMeta, Arc<dyn Tool>)>,
}
@@ -61,39 +46,38 @@ impl Registry {
}
}
fn setup() -> (TempDir, TempDir, Registry) {
fn setup() -> (TempDir, Registry) {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let fs: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let tracker = Tracker::new();
let reg = Registry::new(core_builtin_tools(fs, tracker, spill.path().to_path_buf()));
(dir, spill, reg)
let reg = Registry::new(core_builtin_tools(
fs,
tracker,
tools::bash_output::BashOutputView::new(),
));
(dir, reg)
}
fn setup_routed() -> (
TempDir,
TempDir,
TempDir,
Arc<WorkdirSessionRouter>,
Registry,
) {
fn setup_routed() -> (TempDir, TempDir, Arc<WorkdirSessionRouter>, Registry) {
let left = TempDir::new().unwrap();
let right = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let router = Arc::new(WorkdirSessionRouter::new());
for (alias, dir) in [("left", &left), ("right", &right)] {
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
router
.attach(WorkdirAttachmentAlias::new(alias).unwrap(), session)
.unwrap();
}
let definitions =
routed_builtin_tools(router.clone(), Tracker::new(), spill.path().to_path_buf());
(left, right, spill, router, Registry::new(definitions))
let definitions = routed_builtin_tools(
router.clone(),
Tracker::new(),
tools::bash_output::BashOutputView::new(),
);
(left, right, router, Registry::new(definitions))
}
fn valid_gif_with_len(target_len: usize) -> Vec<u8> {
@@ -137,7 +121,7 @@ async fn call_err(tool: &Arc<dyn Tool>, input: serde_json::Value) -> agen::tool:
#[test]
fn core_builtin_tools_registers_full_set() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let mut names = reg.names();
names.sort();
assert_eq!(names, vec!["Bash", "Edit", "Glob", "Grep", "Read", "Write"]);
@@ -145,7 +129,7 @@ fn core_builtin_tools_registers_full_set() {
#[test]
fn meta_has_description_and_schema() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
for (meta, _) in &reg.entries {
assert!(
!meta.description.is_empty(),
@@ -163,7 +147,7 @@ fn meta_has_description_and_schema() {
#[test]
fn routed_tool_schemas_share_optional_target_workdir() {
let (_left, _right, _spill, router, mut reg) = setup_routed();
let (_left, _right, router, mut reg) = setup_routed();
reg.entries.push(routed_view_image_tool(router)());
for (meta, _) in &reg.entries {
let target = &meta.input_schema["properties"]["target_workdir"];
@@ -174,7 +158,7 @@ fn routed_tool_schemas_share_optional_target_workdir() {
#[tokio::test]
async fn routed_tools_require_alias_only_when_attachment_set_is_ambiguous() {
let (_left, _right, _spill, router, reg) = setup_routed();
let (_left, _right, router, reg) = setup_routed();
let error = call_err(&reg.get("Read"), json!({ "file_path": "same.txt" })).await;
let message = error.to_string();
assert!(message.contains("target_workdir_required"), "{message}");
@@ -199,7 +183,7 @@ async fn empty_routed_surface_returns_no_attachment_error() {
let reg = Registry::new(routed_builtin_tools(
router,
Tracker::new(),
Path::new("unused").to_path_buf(),
tools::bash_output::BashOutputView::new(),
));
let message = call_err(&reg.get("Read"), json!({ "file_path": "file.txt" }))
.await
@@ -209,7 +193,7 @@ async fn empty_routed_surface_returns_no_attachment_error() {
#[tokio::test]
async fn target_alias_routes_each_tool_and_isolates_read_before_edit() {
let (left, right, _spill, router, mut reg) = setup_routed();
let (left, right, router, mut reg) = setup_routed();
std::fs::write(left.path().join("same.txt"), "left value\n").unwrap();
std::fs::write(right.path().join("same.txt"), "right value\n").unwrap();
std::fs::write(left.path().join("only-left.log"), "LEFT-NEEDLE\n").unwrap();
@@ -282,7 +266,7 @@ async fn target_alias_routes_each_tool_and_isolates_read_before_edit() {
)
.await;
assert_eq!(
std::fs::canonicalize(bash.content.unwrap().trim()).unwrap(),
std::fs::canonicalize(serde_json::from_str::<serde_json::Value>(&bash.content.unwrap()).unwrap()["stdout_preview"].as_str().unwrap().trim()).unwrap(),
std::fs::canonicalize(right.path()).unwrap()
);
@@ -304,13 +288,12 @@ async fn target_alias_routes_each_tool_and_isolates_read_before_edit() {
async fn selected_attachment_capability_is_checked_without_fallback() {
let left = TempDir::new().unwrap();
let right = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let router = Arc::new(WorkdirSessionRouter::new());
router
.attach(
WorkdirAttachmentAlias::new("writable").unwrap(),
Arc::new(LocalWorkdirSession::new(
scope_with_spill(left.path(), spill.path()),
Scope::writable(left.path()).unwrap(),
left.path().to_path_buf(),
)),
)
@@ -322,7 +305,7 @@ async fn selected_attachment_capability_is_checked_without_fallback() {
workdir::Workdir::new("readonly"),
right.path().to_path_buf(),
right.path().to_path_buf(),
manifest::SharedScope::new(scope_with_spill(right.path(), spill.path())),
manifest::SharedScope::new(Scope::writable(right.path()).unwrap()),
workdir::WorkdirSessionCapabilities::READ_ONLY,
)),
)
@@ -330,7 +313,7 @@ async fn selected_attachment_capability_is_checked_without_fallback() {
let reg = Registry::new(routed_builtin_tools(
router.clone(),
Tracker::new(),
spill.path().to_path_buf(),
tools::bash_output::BashOutputView::new(),
));
let message = call_err(
@@ -352,7 +335,7 @@ async fn selected_attachment_capability_is_checked_without_fallback() {
#[tokio::test]
async fn routed_tools_observe_detach_after_registration() {
let (_left, _right, _spill, router, reg) = setup_routed();
let (_left, _right, router, reg) = setup_routed();
let alias = WorkdirAttachmentAlias::new("left").unwrap();
router.detach(&alias).await.unwrap();
@@ -372,7 +355,6 @@ async fn routed_tools_observe_detach_after_registration() {
async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
let first = TempDir::new().unwrap();
let second = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
std::fs::write(first.path().join("same.txt"), "same bytes\n").unwrap();
std::fs::write(second.path().join("same.txt"), "same bytes\n").unwrap();
@@ -382,7 +364,7 @@ async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
.attach(
alias.clone(),
Arc::new(LocalWorkdirSession::new(
scope_with_spill(first.path(), spill.path()),
Scope::writable(first.path()).unwrap(),
first.path().to_path_buf(),
)),
)
@@ -390,7 +372,7 @@ async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
let reg = Registry::new(routed_builtin_tools(
router.clone(),
Tracker::new(),
spill.path().to_path_buf(),
tools::bash_output::BashOutputView::new(),
));
call(
&reg.get("Read"),
@@ -403,7 +385,7 @@ async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
.attach(
alias.clone(),
Arc::new(LocalWorkdirSession::new(
scope_with_spill(second.path(), spill.path()),
Scope::writable(second.path()).unwrap(),
second.path().to_path_buf(),
)),
)
@@ -432,6 +414,8 @@ async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
struct ExternalValidatingSession {
inner: LocalWorkdirSession,
truncate_first_binary_response: bool,
command_response_losses: std::sync::atomic::AtomicUsize,
lose_terminal_only: bool,
mutate_after_first_binary_response: Option<std::path::PathBuf>,
}
@@ -550,7 +534,24 @@ impl workdir::WorkdirSession for ExternalValidatingSession {
&self,
request: workdir::CommandOutputRequest,
) -> Result<workdir::CommandOutput, workdir::WorkdirError> {
self.inner.command_output(request).await
let output = self.inner.command_output(request).await?;
if (!self.lose_terminal_only || output.terminal.is_some())
&& self
.command_response_losses
.fetch_update(
std::sync::atomic::Ordering::AcqRel,
std::sync::atomic::Ordering::Acquire,
|remaining| remaining.checked_sub(1),
)
.is_ok()
{
// The production provider has committed a page, but its response is lost.
return Err(
workdir::WorkdirError::Transport("injected lost response".into())
.with_failure_reason(workdir::WorkdirFailureReason::HttpTimeout),
);
}
Ok(output)
}
async fn cancel_command(
@@ -568,11 +569,12 @@ impl workdir::WorkdirSession for ExternalValidatingSession {
#[tokio::test]
async fn external_view_image_uses_a_protocol_bounded_request() {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle = Arc::new(ExternalValidatingSession {
inner: LocalWorkdirSession::new(scope, dir.path().to_path_buf()),
truncate_first_binary_response: false,
command_response_losses: std::sync::atomic::AtomicUsize::new(0),
lose_terminal_only: false,
mutate_after_first_binary_response: None,
});
let png = include_bytes!("fixtures/view-image/valid.png");
@@ -590,13 +592,14 @@ async fn external_view_image_uses_a_protocol_bounded_request() {
#[tokio::test]
async fn external_view_image_rejects_truncated_and_cross_version_chunks() {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let image_path = dir.path().join("image.png");
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
std::fs::write(&image_path, include_bytes!("fixtures/view-image/valid.png")).unwrap();
let truncated: WorkdirSessionHandle = Arc::new(ExternalValidatingSession {
inner: LocalWorkdirSession::new(scope.clone(), dir.path().to_path_buf()),
truncate_first_binary_response: true,
command_response_losses: std::sync::atomic::AtomicUsize::new(0),
lose_terminal_only: false,
mutate_after_first_binary_response: None,
});
let (_meta, tool) = view_image_tool(truncated)();
@@ -613,6 +616,8 @@ async fn external_view_image_rejects_truncated_and_cross_version_chunks() {
let changing: WorkdirSessionHandle = Arc::new(ExternalValidatingSession {
inner: LocalWorkdirSession::new(scope, dir.path().to_path_buf()),
truncate_first_binary_response: false,
command_response_losses: std::sync::atomic::AtomicUsize::new(0),
lose_terminal_only: false,
mutate_after_first_binary_response: Some(image_path),
});
let (_meta, tool) = view_image_tool(changing)();
@@ -626,8 +631,7 @@ async fn external_view_image_rejects_truncated_and_cross_version_chunks() {
async fn view_image_enforces_the_ten_mib_boundary_without_partial_attachments() {
const LIMIT: usize = 10 * 1024 * 1024;
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let (_meta, tool) = view_image_tool(session)();
@@ -650,8 +654,7 @@ async fn view_image_enforces_the_ten_mib_boundary_without_partial_attachments()
#[tokio::test]
async fn view_image_attaches_valid_supported_image_fixtures() {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let (_meta, tool) = view_image_tool(session)();
@@ -726,8 +729,7 @@ async fn delegated_view_image_preserves_caller_relative_path_identity() {
#[tokio::test]
async fn view_image_reads_scoped_bytes_into_durable_tool_detail() {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let png = include_bytes!("fixtures/view-image/valid.png");
@@ -752,7 +754,7 @@ async fn view_image_reads_scoped_bytes_into_durable_tool_detail() {
#[tokio::test]
async fn read_then_edit_then_read_roundtrip() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("a.txt");
std::fs::write(&file, "hello world\n").unwrap();
let p = "a.txt";
@@ -784,7 +786,7 @@ async fn read_then_edit_then_read_roundtrip() {
#[tokio::test]
async fn write_then_grep_finds_content() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let write = reg.get("Write");
let grep = reg.get("Grep");
@@ -813,7 +815,7 @@ async fn write_then_grep_finds_content() {
#[tokio::test]
async fn glob_finds_written_files() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let write = reg.get("Write");
let glob = reg.get("Glob");
@@ -837,7 +839,7 @@ async fn glob_finds_written_files() {
#[tokio::test]
async fn absolute_path_is_rejected() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let outside = TempDir::new().unwrap();
let write = reg.get("Write");
@@ -859,7 +861,7 @@ async fn absolute_path_is_rejected() {
#[tokio::test]
async fn write_to_existing_without_read_fails() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("exists.txt");
std::fs::write(&file, "preexisting").unwrap();
@@ -880,7 +882,7 @@ async fn write_to_existing_without_read_fails() {
async fn shared_workdir_across_tools() {
// The key invariant: all builtin tools share the same WorkdirSession instance,
// so read-history set by Read is visible to Edit and Write.
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("shared.txt");
std::fs::write(&file, "one\n").unwrap();
@@ -907,7 +909,7 @@ async fn shared_workdir_across_tools() {
#[tokio::test]
async fn edit_requires_read_across_tools() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("a.txt");
std::fs::write(&file, "foo\n").unwrap();
@@ -928,7 +930,7 @@ async fn edit_requires_read_across_tools() {
#[tokio::test]
async fn deterministic_tool_order_is_registration_order() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
// Registration order from core_builtin_tools(): Read, Write, Edit, Glob, Grep, Bash
let names: Vec<&str> = reg.entries.iter().map(|(m, _)| m.name.as_str()).collect();
assert_eq!(
@@ -940,7 +942,7 @@ async fn deterministic_tool_order_is_registration_order() {
// Regression: tool name capitalization matches Claude Code reference
#[test]
fn tool_names_match_reference_spec() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
for expected in ["Read", "Write", "Edit", "Glob", "Grep", "Bash"] {
assert!(
reg.entries.iter().any(|(m, _)| m.name == expected),
@@ -953,15 +955,14 @@ fn tool_names_match_reference_spec() {
async fn tracker_recent_files_tracks_read_write_edit() {
// Build a fresh registry that shares a tracker we can query afterwards.
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let fs: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let tracker = Tracker::new();
let reg = Registry::new(core_builtin_tools(
fs,
tracker.clone(),
spill.path().to_path_buf(),
tools::bash_output::BashOutputView::new(),
));
let a = dir.path().join("a.txt");
@@ -1003,118 +1004,314 @@ async fn tracker_recent_files_tracks_read_write_edit() {
async fn bash_inherits_workdir_cwd() {
// The Bash tool starts at the WorkdirSession's pwd. Without any `cd`, its
// `pwd` should canonicalize to the workspace root we set up.
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let bash = reg.get("Bash");
let out = call(&bash, json!({ "command": "pwd", "cwd": "." })).await;
let body = out.content.unwrap();
let actual = std::fs::canonicalize(body.trim()).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&body).unwrap();
let actual = std::fs::canonicalize(parsed["stdout_preview"].as_str().unwrap().trim()).unwrap();
let expected = std::fs::canonicalize(dir.path()).unwrap();
assert_eq!(actual, expected);
}
fn output_fixture() -> (
TempDir,
WorkdirSessionHandle,
tools::bash_output::BashOutputView,
Arc<dyn Tool>,
) {
let root = TempDir::new().unwrap();
let session: WorkdirSessionHandle = Arc::new(LocalWorkdirSession::new(
Scope::writable(root.path()).unwrap(),
root.path().into(),
));
let view = tools::bash_output::BashOutputView::new();
let (_, bash) = tools::bash_tool(session.clone(), view.clone())();
(root, session, view, bash)
}
#[tokio::test]
async fn bash_provider_output_exposes_readable_retained_path() {
let (_dir, spill, reg) = setup();
let bash = reg.get("Bash");
async fn bash_output_is_retained_without_a_filesystem_grant_or_telemetry_subscriber() {
let (root, _session, view, bash) = output_fixture();
let out = call(
&bash,
json!({ "command": "printf 'x%.0s' {1..20480}", "cwd": "." }),
json!({ "command": "printf 'x%.0s' {1..20480}; printf '\\377€' >&2; exit 7", "cwd": "." }),
)
.await;
let body = out.content.unwrap();
assert!(body.contains("bounded WorkdirSession command output"));
assert!(body.contains("full output saved to"));
assert!(body.contains(spill.path().to_str().unwrap()));
let artifact = std::fs::read_dir(spill.path())
.unwrap()
.next()
.expect("retained output")
.unwrap()
.path();
assert_eq!(std::fs::metadata(artifact).unwrap().len(), 20_480);
let body: serde_json::Value = serde_json::from_str(&out.content.unwrap()).unwrap();
assert_eq!(body["output"]["terminal"]["exit_code"], 7);
assert_eq!(body["preview_omitted"], true);
assert_eq!(body["output"]["storage_truncated"], false);
let id = body["output"]["id"].as_str().unwrap();
assert_eq!(
view.read(id, workdir::CommandStream::Stdout, 0, 65536)
.unwrap(),
vec![b'x'; 20480]
);
assert_eq!(
view.read(id, workdir::CommandStream::Stderr, 0, 65536)
.unwrap(),
[vec![255], "€".as_bytes().to_vec()].concat()
);
assert!(
body["stdout_entry"]
.as_str()
.unwrap()
.starts_with("/bash-output/")
);
assert!(!body.to_string().contains(root.path().to_str().unwrap()));
assert_eq!(std::fs::read_dir(root.path()).unwrap().count(), 0);
}
async fn await_command_output(
events: &mut tokio::sync::broadcast::Receiver<workdir::CommandEvent>,
) {
tokio::time::timeout(std::time::Duration::from_secs(5), async {
loop {
if matches!(
events.recv().await.unwrap(),
workdir::CommandEvent::Output { .. }
) {
break;
}
}
})
.await
.expect("command output readiness");
}
async fn await_command_terminal(
events: &mut tokio::sync::broadcast::Receiver<workdir::CommandEvent>,
) {
tokio::time::timeout(std::time::Duration::from_secs(5), async {
loop {
if let workdir::CommandEvent::Terminal { status, .. } = events.recv().await.unwrap() {
assert_eq!(status, workdir::CommandStatus::Cancelled);
break;
}
}
})
.await
.expect("command terminal fence");
}
#[tokio::test]
async fn bash_cancellation_returns_bounded_progress_as_terminal_output() {
let (dir, _spill, reg) = setup();
let marker = dir.path().join("must-not-run-after-cancel");
let command = format!(
"printf 'before\\n'; printf 'err-before\\n' >&2; sleep 1; touch {}; printf 'after\\n'",
marker.display()
);
let input = serde_json::to_string(&json!({ "command": command, "cwd": "." })).unwrap();
let (_root, session, _view, bash) = output_fixture();
let mut events = session.subscribe_command_events().unwrap();
let context = ToolExecutionContext::new("call-heavy", "attempt-heavy", 0);
let bash = reg.get("Bash");
let executing = bash.clone();
let execution_context = context.clone();
let execution = tokio::spawn(async move { executing.execute(&input, execution_context).await });
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
bash.cancel_execution(&context)
.await
.expect("signal exact execution cancellation");
let error = tokio::time::timeout(std::time::Duration::from_secs(2), execution)
.await
.expect("cancelled Bash should terminate inside the Engine grace budget")
.expect("Bash task join");
let ToolError::Cancelled(output) = error.expect_err("cancelled command is non-success") else {
panic!("expected typed cancellation result");
let execution = tokio::spawn(async move {
executing.execute(r#"{"command":"printf 'before'; printf 'err-before' >&2; while :; do :; done","cwd":"."}"#, execution_context).await
});
await_command_output(&mut events).await;
bash.cancel_execution(&context).await.unwrap();
let ToolError::Cancelled(output) =
tokio::time::timeout(std::time::Duration::from_secs(5), execution)
.await
.unwrap()
.unwrap()
.unwrap_err()
else {
panic!("expected cancellation");
};
let content = output.content.expect("bounded progress output");
assert!(
content.contains("before"),
"missing pre-cancel stdout: {content}"
);
assert!(
content.contains("err-before"),
"missing pre-cancel stderr: {content}"
);
assert!(
!content.contains("after"),
"post-cancel output leaked: {content}"
);
assert!(content.len() <= 16 * 1024, "output must remain bounded");
tokio::time::sleep(std::time::Duration::from_millis(1_100)).await;
assert!(
!marker.exists(),
"the cancelled command continued executing after terminal confirmation"
);
let body: serde_json::Value = serde_json::from_str(&output.content.unwrap()).unwrap();
assert_eq!(body["stdout_preview"], "before");
assert_eq!(body["stderr_preview"], "err-before");
assert_eq!(body["output"]["terminal"]["status"], "cancelled");
assert_eq!(body["output"]["terminal"]["transport_complete"], true);
}
#[tokio::test]
async fn bash_force_close_cleanup_stops_command_and_keeps_session_reusable() {
let (dir, _spill, reg) = setup();
let marker = dir.path().join("must-not-survive-force-close");
let command = format!("sleep 1; touch {}", marker.display());
let input = serde_json::to_string(&json!({ "command": command, "cwd": "." })).unwrap();
let bash = reg.get("Bash");
let (_root, session, _view, bash) = output_fixture();
let mut events = session.subscribe_command_events().unwrap();
let context = ToolExecutionContext::new("call-force", "attempt-force", 0);
let (handle, terminal) = ToolExecutionHandle::start(bash.clone(), input, context);
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
let (handle, terminal) = ToolExecutionHandle::start(
bash.clone(),
r#"{"command":"printf ready; while :; do :; done","cwd":"."}"#.into(),
context,
);
await_command_output(&mut events).await;
handle.force_close();
assert!(matches!(
terminal.await,
ToolExecutionTerminal::OutcomeUnknown
));
tokio::time::sleep(std::time::Duration::from_millis(1_100)).await;
assert!(
!marker.exists(),
"CommandGuard cleanup allowed a force-closed command to continue"
);
let output = bash
.execute(
r#"{"command":"printf 'reused'","cwd":"."}"#,
Default::default(),
)
.await
.expect("workdir session remains reusable after cleanup");
assert_eq!(output.content.as_deref(), Some("reused"));
await_command_terminal(&mut events).await;
let output = call(&bash, json!({"command":"printf reused", "cwd":"."})).await;
let body: serde_json::Value = serde_json::from_str(&output.content.unwrap()).unwrap();
assert_eq!(body["stdout_preview"], "reused");
}
// Sanity: unused Path import guard
const _: fn() -> &'static Path = || Path::new("/");
#[tokio::test]
async fn parent_and_sibling_broker_commands_keep_output_ownership_after_revoke() {
let (root, source, parent_view, parent_bash) = output_fixture();
let broker = WorkdirToolBroker::new(source);
let mut leases = Vec::new();
let mut views = Vec::new();
let mut commands = Vec::new();
for name in ["left", "right"] {
std::fs::create_dir(root.path().join(name)).unwrap();
let lease = Arc::new(
broker
.scope(WorkdirToolScope {
rules: vec![WorkdirToolScopeRule {
target: WorkdirPath::new(name).unwrap(),
permission: WorkdirToolScopePermission::Write,
recursive: true,
symlink_policy: Default::default(),
}],
cwd: WorkdirPath::new(name).unwrap(),
command: true,
})
.await
.unwrap(),
);
let authority = lease.clone();
let view = parent_view.child(Arc::new(move || authority.is_active()));
let (_, bash) = tools::bash_tool(lease.tool_session(), view.clone())();
leases.push(lease);
views.push(view);
commands.push(bash);
}
let left_input =
json!({"command":"printf 'left-%.0s' {1..10000}; printf left-err >&2", "cwd":"."});
let right_input =
json!({"command":"printf 'right-%.0s' {1..10000}; printf right-err >&2", "cwd":"."});
let parent_input = json!({"command":"printf parent; printf parent-err >&2", "cwd":"."});
let (left, right, parent) = tokio::join!(
call(&commands[0], left_input),
call(&commands[1], right_input),
call(&parent_bash, parent_input)
);
let outputs = [left, right, parent]
.map(|out| serde_json::from_str::<serde_json::Value>(&out.content.unwrap()).unwrap());
for (index, label) in ["left", "right", "parent"].iter().enumerate() {
let id = outputs[index]["output"]["id"].as_str().unwrap();
let bytes = parent_view
.read(id, workdir::CommandStream::Stderr, 0, 65536)
.unwrap();
assert_eq!(bytes, format!("{label}-err").as_bytes());
for (child_index, view) in views.iter().enumerate() {
assert_eq!(view.metadata(id).is_some(), child_index == index);
}
}
let left_id = outputs[0]["output"]["id"].as_str().unwrap();
leases[0].close().await.unwrap();
assert!(views[0].metadata(left_id).is_none());
assert!(parent_view.metadata(left_id).is_some());
assert!(
views[1]
.metadata(outputs[1]["output"]["id"].as_str().unwrap())
.is_some()
);
leases[1].close().await.unwrap();
broker.close().await.unwrap();
parent_view.revoke();
assert!(parent_view.metadata(left_id).is_none());
}
#[tokio::test]
async fn utf8_quota_boundary_preserves_raw_prefix_and_reports_saved_vs_seen() {
let (_root, _session, view, bash) = output_fixture();
let out = call(
&bash,
json!({"command":"printf '%4194303s' ''; printf '€'", "cwd":"."}),
)
.await;
let body: serde_json::Value = serde_json::from_str(&out.content.unwrap()).unwrap();
let id = body["output"]["id"].as_str().unwrap();
let saved = body["output"]["stdout_saved"].as_u64().unwrap() as usize;
assert_eq!(saved, 4 * 1024 * 1024);
assert_eq!(body["output"]["stdout_seen"], 4 * 1024 * 1024 + 2);
assert_eq!(body["output"]["stderr_seen"], 0);
assert_eq!(body["output"]["storage_truncated"], true);
assert_eq!(
view.read(id, workdir::CommandStream::Stdout, saved - 1, 10)
.unwrap(),
vec![0xe2]
);
assert!(
view.line_window(id, workdir::CommandStream::Stdout, 0, 1)
.unwrap()
.is_err()
);
assert!(out.summary.starts_with("$ "));
}
#[tokio::test]
async fn timeout_terminal_fences_retained_stdout_and_stderr() {
let (_root, _session, view, bash) = output_fixture();
let error = tokio::time::timeout(std::time::Duration::from_secs(5), bash.execute(r#"{"command":"printf before; printf err-before >&2; while :; do :; done","cwd":".","timeout":1}"#, Default::default())).await.unwrap().unwrap_err();
let ToolError::Interrupted(out) = error else {
panic!("expected timeout output");
};
let body: serde_json::Value = serde_json::from_str(&out.content.unwrap()).unwrap();
let id = body["output"]["id"].as_str().unwrap();
assert_eq!(body["output"]["terminal"]["status"], "timed_out");
assert_eq!(body["output"]["terminal"]["transport_complete"], true);
assert_eq!(
view.read(id, workdir::CommandStream::Stdout, 0, 65536)
.unwrap(),
b"before"
);
assert_eq!(
view.read(id, workdir::CommandStream::Stderr, 0, 65536)
.unwrap(),
b"err-before"
);
}
// Integration: real process pipes and provider replay into the production Bash collector.
// Only response delivery is fault-injected; Runtime's HTTP suite covers actual wire routing.
#[tokio::test]
async fn lost_page_and_terminal_responses_replay_without_restarting_or_double_appending() {
for lose_terminal_only in [false, true] {
let dir = TempDir::new().unwrap();
let session = Arc::new(ExternalValidatingSession {
inner: LocalWorkdirSession::new(
Scope::writable(dir.path()).unwrap(),
dir.path().into(),
),
truncate_first_binary_response: false,
mutate_after_first_binary_response: None,
command_response_losses: std::sync::atomic::AtomicUsize::new(1),
lose_terminal_only,
});
let view = tools::bash_output::BashOutputView::new();
let definitions = core_builtin_tools(session.clone(), Tracker::new(), view.clone());
let bash = Registry::new(definitions).get("Bash");
let out = call(&bash, json!({"cwd":".","command":"printf once >> effect.txt; printf 'x%.0s' {1..200000}; printf err >&2; exit 3"})).await;
let body: serde_json::Value = serde_json::from_str(&out.content.unwrap()).unwrap();
assert_eq!(
std::fs::read(dir.path().join("effect.txt")).unwrap(),
b"once"
);
assert_eq!(
session
.command_response_losses
.load(std::sync::atomic::Ordering::Acquire),
0
);
assert_eq!(body["output"]["stdout_saved"], 200000);
assert_eq!(body["output"]["stdout_delivered"], 200000);
assert_eq!(body["output"]["stderr_saved"], 3);
assert_eq!(body["output"]["terminal"]["exit_code"], 3);
assert_eq!(body["output"]["terminal"]["transport_complete"], true);
assert_eq!(body["output"]["storage_truncated"], false);
let id = body["output"]["id"].as_str().unwrap();
for offset in (0..200000).step_by(65536) {
assert_eq!(
view.read(id, workdir::CommandStream::Stdout, offset, 65536)
.unwrap(),
vec![b'x'; (200000 - offset).min(65536)]
);
}
use workdir::WorkdirSession as _;
session.close().await.unwrap();
}
}
+39 -19
View File
@@ -3013,7 +3013,8 @@ impl App {
/// `[File: …]` parsing path: each kind maps directly to a typed
/// block (`Block::Notify`, `Block::WorkerEvent`, …).
fn apply_system_item(&mut self, value: &serde_json::Value) {
let Ok(item) = serde_json::from_value::<session_store::SystemItem>(value.clone()) else {
let Ok(item) = serde_json::from_value::<session_store::PublicSystemItem>(value.clone())
else {
// Unknown / forward-compat shape: fall back to rendering the
// raw text payload (if any) as a generic system message.
if let Some(text) = value.get("body").and_then(|b| b.as_str()) {
@@ -3024,28 +3025,23 @@ impl App {
}
return;
};
match item {
session_store::SystemItem::Notification { message, .. } => {
let body = item.body;
match item.presentation {
session_store::PublicSystemPresentation::Notification { message } => {
self.blocks.push(Block::Notify { message });
}
session_store::SystemItem::WorkerEvent { event, .. } => {
session_store::PublicSystemPresentation::WorkerEvent { event } => {
self.blocks.push(Block::WorkerEvent { event });
}
session_store::SystemItem::FeatureInvocationResult { body, .. }
| session_store::SystemItem::FileAttachment { body, .. }
| session_store::SystemItem::SkillActivation { body, .. }
| session_store::SystemItem::ResidentSummaryRefresh { body, .. }
| session_store::SystemItem::SubjectBehaviorRefresh { body, .. }
| session_store::SystemItem::Interrupt { body, .. } => {
session_store::PublicSystemPresentation::Context => {
self.task_store.apply_system_message_text(&body);
self.blocks.push(Block::SystemMessage { text: body });
}
session_store::SystemItem::TaskReminder { body, .. } => {
session_store::PublicSystemPresentation::TaskReminder => {
self.task_store.apply_system_message_text(&body);
self.blocks.push(Block::TaskReminder { text: body });
}
session_store::SystemItem::LegacyIgnored { .. } => {}
session_store::SystemItem::LegacyKnowledgeIgnored { .. } => {}
session_store::PublicSystemPresentation::Ignored => {}
}
}
@@ -3571,6 +3567,7 @@ mod completion_flow_tests {
session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4319,6 +4316,8 @@ mod completion_flow_tests {
let mut app = App::new("test".into());
let message = "mid-run compaction failed: summary unavailable";
app.handle_worker_event(Event::Alert(protocol::Alert {
id: None,
position: None,
level: AlertLevel::Error,
source: AlertSource::Compactor,
message: message.into(),
@@ -4326,6 +4325,7 @@ mod completion_flow_tests {
}));
app.handle_worker_event(Event::SessionEntryCommitted {
entry: protocol::SessionSnapshotEntry {
use_policy: None,
entry_id: "run-failure".into(),
timestamp: 2,
provenance: protocol::SessionEntryProvenance::LegacyUnknown,
@@ -4402,6 +4402,7 @@ mod completion_flow_tests {
},
state: test_worker_state(WorkerStatus::Running),
in_flight: InFlightSnapshot {
block_ids: Vec::new(),
blocks: vec![
InFlightBlock::Thinking {
text: "why".into(),
@@ -4446,12 +4447,12 @@ mod completion_flow_tests {
}
#[test]
fn live_legacy_workflow_system_item_is_ignored() {
fn retired_system_input_is_ignored() {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "workflow",
"slug": "build",
"body": "[Workflow /build]\nRun the build",
"kind": "retired_input",
"body": "",
"presentation": { "kind": "ignored" },
});
app.handle_worker_event(Event::SystemItem {
entry_id: None,
@@ -4857,12 +4858,30 @@ mod completion_flow_tests {
);
}
#[test]
fn feature_label_does_not_select_system_input_presentation() {
for label in ["journal_startup", "new-unregistered-feature"] {
let mut app = App::new("test".into());
app.handle_worker_event(Event::SystemItem {
entry_id: None,
item: serde_json::json!({
"kind": label,
"body": "explicit public content",
"presentation": { "kind": "context" },
}),
});
assert!(
matches!(app.blocks.as_slice(), [Block::SystemMessage { text }] if text == "explicit public content")
);
}
}
#[test]
fn live_system_item_notification_appends_notify_block() {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "notification",
"message": "hi",
"presentation": { "kind": "notification", "message": "hi" },
"body": "[Notification] hi",
});
app.handle_worker_event(Event::SystemItem {
@@ -4880,7 +4899,7 @@ mod completion_flow_tests {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "worker_event",
"event": { "kind": "turn_ended", "worker_name": "child" },
"presentation": { "kind": "worker_event", "event": { "kind": "turn_ended", "worker_name": "child" } },
"body": "[Notification] worker `child` finished a turn",
});
app.handle_worker_event(Event::SystemItem {
@@ -5281,6 +5300,7 @@ mod completion_flow_tests {
item: serde_json::json!({
"kind": "task_reminder",
"body": snapshot,
"presentation": { "kind": "task_reminder" },
}),
});
+1
View File
@@ -162,6 +162,7 @@ fn request_from_selection(selection: Selection) -> BackendCreateWorkerRequest {
ticket_assignment: None,
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
}
}
+1 -1
View File
@@ -20,7 +20,7 @@ serde_json.workspace = true
sha2.workspace = true
tempfile.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tokio = { workspace = true, features = ["io-util", "process", "rt", "sync", "time"] }
[dev-dependencies]
serde_json.workspace = true
+6 -1
View File
@@ -48,6 +48,10 @@ pub struct CheckoutRequest {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(tag = "operation", rename_all = "snake_case", deny_unknown_fields)]
pub enum CheckoutOperation {
ReadBytes {
offset: u64,
max_bytes: usize,
},
Read {
offset: usize,
limit: usize,
@@ -71,7 +75,7 @@ pub enum CheckoutOperation {
impl CheckoutOperation {
pub fn capability(&self) -> WorkdirSessionCapability {
match self {
Self::Read { .. } => WorkdirSessionCapability::Read,
Self::Read { .. } | Self::ReadBytes { .. } => WorkdirSessionCapability::Read,
Self::Edit { .. } => WorkdirSessionCapability::Edit,
Self::Write { .. } | Self::Create { .. } => WorkdirSessionCapability::Write,
}
@@ -86,6 +90,7 @@ impl CheckoutOperation {
)]
pub enum CheckoutOutput {
Read(ReadResult),
ReadBytes(crate::ReadBytesResult),
Write(WriteResult),
Edit(EditResult),
}
@@ -0,0 +1,43 @@
//! Bounded collection for tests whose subject is command routing/lifecycle,
//! rather than the byte-page protocol itself.
use crate::*;
#[derive(Debug)]
pub(crate) struct CollectedCommand {
pub content: String,
pub status: CommandStatus,
pub exit_code: Option<i32>,
pub timed_out: bool,
}
#[async_trait::async_trait]
pub(crate) trait CollectCommand: WorkdirSession {
async fn collect_command_output(
&self,
mut request: CommandOutputRequest,
) -> Result<CollectedCommand, WorkdirError> {
let mut bytes = Vec::new();
request.wait = true;
loop {
let page = self.command_output(request.clone()).await?;
for chunk in page.chunks {
bytes.extend(chunk.bytes);
assert!(
bytes.len() <= 16 * 1024 * 1024,
"fixture output exceeded bounded test collection"
);
}
request.cursor = page.next_cursor;
if let Some(terminal) = page.terminal {
return Ok(CollectedCommand {
content: String::from_utf8_lossy(&bytes).into_owned(),
status: terminal.status,
exit_code: terminal.exit_code,
timed_out: terminal.status == CommandStatus::TimedOut,
});
}
}
}
}
impl<T: WorkdirSession + ?Sized> CollectCommand for T {}
+154 -38
View File
@@ -9,7 +9,8 @@
//! COMMAND remains separate explicit authority. General session capability intersections may form
//! subsets independently of this grant hierarchy; the codec never implicitly expands nonhierarchical
//! grants. Commands run with the shared host's OS permissions, not a sandbox confining processes
//! to a directory. Output is bounded and inline; provider-local spill paths are not exposed.
//! to a directory. Output uses bounded byte pages with cursor acknowledgements;
//! providers never receive caller-selected output paths.
use std::path::{Component, Path};
@@ -18,7 +19,7 @@ use serde::{Deserialize, Serialize};
use crate::http::{WorkdirSessionOperation, WorkdirSessionOperationResult, WorkdirTransportError};
use crate::{BoundedReadLimits, WorkdirId, WorkdirPath, WorkdirSessionCapabilities};
pub const EXTERNAL_WORKDIR_PROVIDER_PROTOCOL_VERSION: u16 = 2;
pub const EXTERNAL_WORKDIR_PROVIDER_PROTOCOL_VERSION: u16 = 3;
/// Maximum serialized Backend-to-provider command frame size.
pub const MAX_EXTERNAL_SERVER_FRAME_BYTES: usize = 1024 * 1024;
/// Maximum serialized provider-to-Backend result frame size. Read bytes are
@@ -33,9 +34,9 @@ const MAX_EXTERNAL_GREP_CONTEXT: usize = 100;
const MAX_EXTERNAL_SCOPE_RULES: usize = 64;
pub const MAX_EXTERNAL_WRITE_BYTES: usize = 512 * 1024;
pub const MAX_EXTERNAL_COMMAND_BYTES: usize = 64 * 1024;
pub const MAX_EXTERNAL_COMMAND_OUTPUT_BYTES: usize = 512 * 1024;
pub const MAX_EXTERNAL_COMMAND_OUTPUT_BYTES: usize = crate::MAX_COMMAND_READ_BYTES;
pub const MAX_EXTERNAL_COMMAND_TIMEOUT_SECS: u64 = 600;
const MAX_EXTERNAL_COMMAND_CURSOR: usize = 1_000_000;
const MAX_EXTERNAL_COMMAND_CURSOR: u64 = crate::MAX_COMMAND_TRANSPORT_BYTES;
fn valid_command_handle(handle: &crate::CommandHandle) -> bool {
!handle.0.is_empty()
@@ -185,6 +186,11 @@ fn validate_external_operation(operation: &WorkdirSessionOperation) -> Result<()
&& !request.validator.is_empty()
&& request.validator.len() <= 256
&& match &request.operation {
crate::CheckoutOperation::ReadBytes { offset, max_bytes } => {
*offset <= 9_007_199_254_740_991
&& *max_bytes > 0
&& *max_bytes <= BoundedReadLimits::EXTERNAL_DEFAULT.max_response_bytes
}
crate::CheckoutOperation::Read {
limit, max_bytes, ..
} => {
@@ -267,11 +273,10 @@ fn validate_external_operation(operation: &WorkdirSessionOperation) -> Result<()
}
WorkdirSessionOperation::CommandStart(request) => {
!request.command.is_empty()
&& request.validate().is_ok()
&& request.command.len() <= MAX_EXTERNAL_COMMAND_BYTES
&& (1..=MAX_EXTERNAL_COMMAND_TIMEOUT_SECS).contains(&request.timeout_secs)
&& (1..=MAX_EXTERNAL_COMMAND_OUTPUT_BYTES).contains(&request.output_limit)
&& is_root_relative(&request.cwd)
&& request.spill_dir.is_none()
&& request.tool_call_id.as_ref().is_none_or(|tool_call_id| {
!tool_call_id.is_empty()
&& tool_call_id.len() <= MAX_PROTOCOL_ID_BYTES
@@ -375,6 +380,13 @@ fn validate_external_result(result: &WorkdirSessionOperationResult) -> Result<()
WorkdirSessionOperationResult::CheckoutExecute(result) => {
valid_checkout_observation(&result.observation)
&& match &result.output {
crate::CheckoutOutput::ReadBytes(read) => {
read.path == result.observation.path
&& validate_external_result(&WorkdirSessionOperationResult::ReadBytes(
read.clone(),
))
.is_ok()
}
crate::CheckoutOutput::Read(read) => {
read.path == result.observation.path
&& validate_external_result(&WorkdirSessionOperationResult::Read(
@@ -450,11 +462,19 @@ fn validate_external_result(result: &WorkdirSessionOperationResult) -> Result<()
WorkdirSessionOperationResult::CommandStart(handle) => valid_command_handle(handle),
WorkdirSessionOperationResult::CommandStatus(_) => true,
WorkdirSessionOperationResult::CommandOutput(output) => {
output.content.len() <= MAX_EXTERNAL_COMMAND_OUTPUT_BYTES
output.transport_bounds_valid()
&& output.chunks.len() <= MAX_EXTERNAL_COMMAND_OUTPUT_BYTES
&& output
.next_cursor
.is_none_or(|cursor| cursor <= MAX_EXTERNAL_COMMAND_CURSOR)
&& output.output_path.is_none()
.chunks
.iter()
.map(|chunk| chunk.bytes.len())
.sum::<usize>()
<= MAX_EXTERNAL_COMMAND_OUTPUT_BYTES
&& output.next_cursor <= MAX_EXTERNAL_COMMAND_CURSOR
&& output
.terminal
.as_ref()
.is_none_or(|terminal| terminal.status != crate::CommandStatus::Running)
}
WorkdirSessionOperationResult::CommandCancel => true,
};
@@ -505,6 +525,8 @@ pub struct ExternalWorkdirOperationError {
code: crate::http::WorkdirTransportErrorCode,
#[serde(default, skip_serializing_if = "Option::is_none")]
denial_reason: Option<crate::WorkdirDenialReason>,
#[serde(default, skip_serializing_if = "Option::is_none")]
failure_reason: Option<crate::WorkdirFailureReason>,
}
impl ExternalWorkdirOperationError {
@@ -514,11 +536,13 @@ impl ExternalWorkdirOperationError {
Self {
code: error.code,
denial_reason: error.denial_reason,
failure_reason: error.failure_reason,
}
}
pub fn into_transport_error(self) -> WorkdirTransportError {
WorkdirTransportError {
failure_reason: self.failure_reason,
code: self.code,
denial_reason: self.denial_reason,
message: format!("External Workdir provider reported {}", self.code.as_str()),
@@ -742,6 +766,7 @@ mod tests {
] {
let external =
ExternalWorkdirOperationError::from_transport_error(WorkdirTransportError {
failure_reason: None,
code: Code::Denied,
message: "/home/operator/private provider-secret".repeat(4096),
denial_reason: reason,
@@ -768,6 +793,7 @@ mod tests {
#[test]
fn provider_failures_are_closed_and_drop_provider_authored_diagnostics() {
let error = ExternalWorkdirOperationError::from_transport_error(WorkdirTransportError {
failure_reason: None,
code: crate::http::WorkdirTransportErrorCode::Internal,
denial_reason: None,
message: "/home/operator/private provider-secret\ncontrol".to_string(),
@@ -801,11 +827,16 @@ mod tests {
#[test]
fn unsupported_protocol_versions_and_invalid_read_limits_fail_closed() {
let future = r#"{
"version": 3,
"message": {"kind": "heartbeat", "generation": 1, "sequence": 1}
}"#;
assert!(serde_json::from_str::<ExternalWorkdirServerFrame>(future).is_err());
for version in [
EXTERNAL_WORKDIR_PROVIDER_PROTOCOL_VERSION - 1,
EXTERNAL_WORKDIR_PROVIDER_PROTOCOL_VERSION + 1,
] {
let unsupported = serde_json::json!({
"version": version,
"message": {"kind": "heartbeat", "generation": 1, "sequence": 1}
});
assert!(serde_json::from_value::<ExternalWorkdirServerFrame>(unsupported).is_err());
}
let invalid_limits = r#"{
"max_source_bytes": 1024,
@@ -825,32 +856,35 @@ mod tests {
assert!(ExternalWorkdirOperation::try_from(absolute).is_err());
let command = WorkdirSessionOperation::CommandStart(CommandRequest {
command: "cat /home/operator/private/session.log".to_string(),
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf unsafe".to_string(),
timeout_secs: 1,
output_limit: 1024,
cwd: WorkdirPath::root(),
spill_dir: Some("/home/operator/private/output".into()),
cwd: serde_json::from_value(serde_json::json!("/absolute")).unwrap(),
tool_call_id: None,
});
assert!(ExternalWorkdirOperation::try_from(command).is_err());
let command = WorkdirSessionOperation::CommandStart(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf bounded".to_string(),
timeout_secs: 30,
output_limit: 12 * 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: Some("tool-call-1".to_string()),
});
assert!(ExternalWorkdirOperation::try_from(command).is_ok());
assert!(
ExternalWorkdirOperation::try_from(WorkdirSessionOperation::CommandStart(
CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "x".repeat(MAX_EXTERNAL_COMMAND_BYTES + 1),
timeout_secs: 30,
output_limit: 12 * 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
}
))
@@ -887,13 +921,17 @@ mod tests {
assert!(
ExternalWorkdirOperationResult::try_from(WorkdirSessionOperationResult::CommandOutput(
crate::CommandOutput {
status: crate::CommandStatus::Completed,
exit_code: Some(0),
timed_out: false,
content: "bounded".to_string(),
next_cursor: None,
truncated: false,
output_path: None,
chunks: vec![],
next_cursor: 0,
terminal: Some(crate::CommandTerminal {
status: crate::CommandStatus::Completed,
exit_code: Some(0),
stdout_bytes: 0,
stderr_bytes: 0,
stdout_delivered_bytes: 0,
stderr_delivered_bytes: 0,
truncation_reason: None,
})
}
))
.is_ok()
@@ -901,13 +939,14 @@ mod tests {
assert!(
ExternalWorkdirOperationResult::try_from(WorkdirSessionOperationResult::CommandOutput(
crate::CommandOutput {
status: crate::CommandStatus::Completed,
exit_code: Some(0),
timed_out: false,
content: "bounded".to_string(),
next_cursor: None,
truncated: true,
output_path: Some("/provider/private/output".into()),
chunks: vec![crate::CommandByteChunk {
sequence: 0,
stream: crate::CommandStream::Stdout,
offset: 0,
bytes: vec![0; MAX_EXTERNAL_COMMAND_OUTPUT_BYTES + 1]
}],
next_cursor: 0,
terminal: None
}
))
.is_err()
@@ -958,7 +997,7 @@ mod tests {
#[test]
fn protocol_rejects_unknown_nested_fields_and_unbounded_requests() {
let unknown = r#"{
"version": 2,
"version": 3,
"message": {
"kind": "heartbeat",
"generation": 1,
@@ -1012,4 +1051,81 @@ mod tests {
assert!(ExternalWorkdirOperationId::new("x".repeat(129)).is_err());
assert!(ExternalWorkdirOperationId::new("operation-1").is_ok());
}
#[test]
fn command_budget_and_page_metadata_obey_external_hard_bounds() {
fn request(budget: u64) -> WorkdirSessionOperation {
WorkdirSessionOperation::CommandStart(CommandRequest {
command: "true".into(),
timeout_secs: 1,
cwd: WorkdirPath::root(),
transport_byte_budget: budget,
tool_call_id: None,
})
}
for budget in [0, crate::MAX_COMMAND_TRANSPORT_BYTES] {
let operation = ExternalWorkdirOperation::try_from(request(budget)).unwrap();
let wire = serde_json::to_value(operation).unwrap();
assert!(serde_json::from_value::<ExternalWorkdirOperation>(wire).is_ok());
}
assert!(
ExternalWorkdirOperation::try_from(request(crate::MAX_COMMAND_TRANSPORT_BYTES + 1))
.is_err()
);
fn page() -> crate::CommandOutput {
crate::CommandOutput {
chunks: vec![crate::CommandByteChunk {
sequence: 0,
stream: crate::CommandStream::Stdout,
offset: 0,
bytes: b"abc".to_vec(),
}],
next_cursor: 3,
terminal: Some(crate::CommandTerminal {
status: crate::CommandStatus::Completed,
exit_code: Some(0),
stdout_bytes: 100 * 1024 * 1024,
stderr_bytes: 7,
stdout_delivered_bytes: 3,
stderr_delivered_bytes: 0,
truncation_reason: Some(crate::CommandOutputTruncation::ByteBudget),
}),
}
}
let accepted = ExternalWorkdirOperationResult::try_from(
WorkdirSessionOperationResult::CommandOutput(page()),
)
.unwrap();
assert!(
serde_json::from_value::<ExternalWorkdirOperationResult>(
serde_json::to_value(accepted).unwrap()
)
.is_ok()
);
for mutate in [
|p: &mut crate::CommandOutput| p.next_cursor = crate::MAX_COMMAND_TRANSPORT_BYTES + 1,
|p: &mut crate::CommandOutput| p.chunks[0].offset = crate::MAX_COMMAND_TRANSPORT_BYTES,
|p: &mut crate::CommandOutput| p.chunks[0].sequence = u64::MAX,
|p: &mut crate::CommandOutput| p.terminal.as_mut().unwrap().stdout_delivered_bytes = 4,
|p: &mut crate::CommandOutput| p.terminal.as_mut().unwrap().stdout_bytes = 2,
|p: &mut crate::CommandOutput| p.terminal.as_mut().unwrap().truncation_reason = None,
] {
let mut invalid = page();
mutate(&mut invalid);
let wire = serde_json::to_value(WorkdirSessionOperationResult::CommandOutput(invalid))
.unwrap();
assert!(serde_json::from_value::<ExternalWorkdirOperationResult>(wire).is_err());
}
assert!(
ExternalWorkdirOperation::try_from(WorkdirSessionOperation::CommandOutput(
crate::CommandOutputRequest {
handle: crate::CommandHandle("command-1".into()),
cursor: crate::MAX_COMMAND_TRANSPORT_BYTES + 1,
limit: 1,
wait: false,
}
))
.is_err()
);
}
}
+13 -7
View File
@@ -1,6 +1,9 @@
#[cfg(test)]
use crate::command_test_support::CollectCommand as _;
use crate::http::{
WorkdirSessionOperation, WorkdirSessionOperationResult, dispatch_workdir_session_operation,
};
use crate::{
BoundedReadLimits, CommandOutputRequest, CommandRequest, EditRequest, ExternalWorkdirRoot,
GlobRequest, GrepOutputMode, GrepRequest, LocalWorkdirSession, ReadRequest, Workdir,
@@ -59,11 +62,12 @@ async fn external_local_provider_is_strictly_read_only() {
let command = WorkdirSession::start_command(
&session,
CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "echo changed > item.txt".to_string(),
timeout_secs: 1,
output_limit: 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
},
)
@@ -145,11 +149,12 @@ async fn external_local_provider_read_write_is_filesystem_only() {
let command = WorkdirSession::start_command(
&session,
CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "touch command-ran".to_string(),
timeout_secs: 1,
output_limit: 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
},
)
@@ -193,17 +198,18 @@ async fn external_command_permission_is_explicit_and_independent_from_file_acces
let handle = session
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf command-ok".to_string(),
timeout_secs: 5,
output_limit: 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
})
.await
.unwrap();
let output = session
.command_output(CommandOutputRequest {
.collect_command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 1024,
+169 -18
View File
@@ -275,6 +275,8 @@ pub struct WorkdirTransportError {
/// Internal provider diagnostics. Public response boundaries may omit this.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub denial_reason: Option<crate::WorkdirDenialReason>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub failure_reason: Option<crate::WorkdirFailureReason>,
}
impl WorkdirTransportError {
@@ -282,6 +284,11 @@ impl WorkdirTransportError {
pub fn from_workdir_error(error: &WorkdirError) -> Self {
use WorkdirTransportErrorCode as Code;
let (code, message) = match error {
WorkdirError::FailureContext { reason, source } => {
let mut transport = Self::from_workdir_error(source);
transport.failure_reason = Some(*reason);
return transport;
}
WorkdirError::DenialContext { reason, source } => {
let mut transport = Self::from_workdir_error(source);
transport.denial_reason = Some(*reason);
@@ -301,6 +308,7 @@ impl WorkdirTransportError {
code: Code::Unsupported,
message: format!("Workdir capability {capability:?} is not available"),
denial_reason: None,
failure_reason: None,
};
}
WorkdirError::UnsupportedOperation(_) => {
@@ -359,6 +367,7 @@ impl WorkdirTransportError {
code,
message: message.to_string(),
denial_reason: error.denial_reason(),
failure_reason: error.failure_reason(),
}
}
@@ -408,6 +417,11 @@ impl WorkdirTransportError {
Code::Transport => WorkdirError::Transport(self.message),
Code::Internal => WorkdirError::OperationFailed,
};
let error = if let Some(reason) = self.failure_reason {
error.with_failure_reason(reason)
} else {
error
};
// Preserve any supplied reason independently of the public classification.
// Without a reason, retain the existing classification's deliberate fallback.
if let Some(reason) = self.denial_reason {
@@ -557,9 +571,10 @@ mod client {
operation: WorkdirSessionOperation,
) -> Result<WorkdirSessionOperationResult, WorkdirError> {
if self.closed.load(Ordering::Acquire) {
return Err(WorkdirError::Unavailable(
"Workdir session is closed".to_string(),
));
return Err(
WorkdirError::Unavailable("Workdir session is closed".to_string())
.with_failure_reason(crate::WorkdirFailureReason::SessionClosed),
);
}
let url = endpoint(
&self.base_url,
@@ -693,6 +708,12 @@ mod client {
let mutation = request.operation.capability() != crate::WorkdirSessionCapability::Read;
let expected_path = request.target.clone();
let expected_output = request.operation.capability();
let byte_range = match &request.operation {
crate::CheckoutOperation::ReadBytes { offset, max_bytes } => {
Some((*offset, *max_bytes))
}
_ => None,
};
match self
.operate(WorkdirSessionOperation::CheckoutExecute(request))
.await
@@ -702,11 +723,30 @@ mod client {
&& !result.observation.validator.is_empty()
&& result.observation.validator.len() <= 256
&& match (&result.output, expected_output) {
(
crate::CheckoutOutput::ReadBytes(read),
crate::WorkdirSessionCapability::Read,
) => byte_range.is_some_and(|(offset, max)| {
read.path == expected_path
&& read.offset == offset
&& read.bytes.len() <= max
&& read
.offset
.checked_add(read.bytes.len() as u64)
.is_some_and(|end| {
end <= read.total_bytes
|| (read.bytes.is_empty() && read.eof)
})
&& read.eof
== (read.offset.saturating_add(read.bytes.len() as u64)
>= read.total_bytes)
}),
(
crate::CheckoutOutput::Read(read),
crate::WorkdirSessionCapability::Read,
) => {
read.path == expected_path
byte_range.is_none()
&& read.path == expected_path
&& read.bytes.len()
<= crate::BoundedReadLimits::EXTERNAL_DEFAULT
.max_response_bytes
@@ -742,6 +782,10 @@ mod client {
let outcome = WorkdirError::OutcomeUnknown(
"checkout transport failed; effects may have occurred".into(),
);
let outcome = match error.failure_reason() {
Some(reason) => outcome.with_failure_reason(reason),
None => outcome,
};
Err(match error.denial_reason() {
Some(reason) => WorkdirError::DenialContext {
reason,
@@ -822,6 +866,7 @@ mod client {
&self,
request: CommandRequest,
) -> Result<CommandHandle, WorkdirError> {
request.validate()?;
match self
.operate(WorkdirSessionOperation::CommandStart(request))
.await?
@@ -848,15 +893,46 @@ mod client {
&self,
request: CommandOutputRequest,
) -> Result<CommandOutput, WorkdirError> {
let wait = request.wait;
if !(1..=crate::MAX_COMMAND_READ_BYTES).contains(&request.limit) {
return Err(WorkdirError::InvalidArgument(
"command byte limit is outside transport bounds".into(),
));
}
loop {
let started = tokio::time::Instant::now();
match self
.operate(WorkdirSessionOperation::CommandOutput(request.clone()))
.await?
{
WorkdirSessionOperationResult::CommandOutput(result)
if wait && result.status == CommandStatus::Running => {}
WorkdirSessionOperationResult::CommandOutput(result) => return Ok(result),
WorkdirSessionOperationResult::CommandOutput(result) => {
let size = result
.chunks
.iter()
.try_fold(0usize, |size, chunk| size.checked_add(chunk.bytes.len()));
if !result.transport_bounds_valid()
|| size.is_none_or(|size| {
size > request.limit
|| request.cursor.checked_add(size as u64)
!= Some(result.next_cursor)
})
{
return Err(Self::mismatch("bounded command byte page"));
}
if request.wait && result.chunks.is_empty() && result.terminal.is_none() {
// A normal long-poll expiry is not a transport failure.
// Continue at the same cursor, without spending the caller's
// error retry budget or acknowledging any bytes. Pace peers
// that return empty pages immediately; a real long poll has
// already consumed this delay. The caller owns cancellation
// and the overall deadline.
tokio::time::sleep_until(
started + std::time::Duration::from_millis(100),
)
.await;
continue;
}
return Ok(result);
}
_ => return Err(Self::mismatch("command_output")),
}
}
@@ -917,24 +993,64 @@ mod client {
response: reqwest::Response,
) -> Result<T, WorkdirError> {
if response.status().is_success() {
response.json().await.map_err(http_unavailable)
let bytes = bounded_response_bytes(response).await?;
serde_json::from_slice(&bytes).map_err(|_| {
WorkdirError::Transport("Runtime Workdir response was invalid".into())
.with_failure_reason(crate::WorkdirFailureReason::HttpResponseInvalid)
})
} else {
Err(decode_error(response).await)
}
}
async fn decode_error(response: reqwest::Response) -> WorkdirError {
response
.json::<WorkdirTransportError>()
.await
.map(WorkdirTransportError::into_workdir_error)
.unwrap_or_else(|_| {
WorkdirError::Transport("Runtime Workdir error response was invalid".to_string())
})
async fn bounded_response_bytes(
mut response: reqwest::Response,
) -> Result<Vec<u8>, WorkdirError> {
const MAX_RESPONSE_BYTES: usize = crate::external::MAX_EXTERNAL_PROVIDER_FRAME_BYTES;
if response
.content_length()
.is_some_and(|size| size > MAX_RESPONSE_BYTES as u64)
{
return Err(WorkdirError::Transport(
"Runtime Workdir response exceeded transport bound".into(),
)
.with_failure_reason(crate::WorkdirFailureReason::HttpResponseTooLarge));
}
let mut bytes = Vec::new();
while let Some(chunk) = response.chunk().await.map_err(http_unavailable)? {
if chunk.len() > MAX_RESPONSE_BYTES.saturating_sub(bytes.len()) {
return Err(WorkdirError::Transport(
"Runtime Workdir response exceeded transport bound".into(),
)
.with_failure_reason(crate::WorkdirFailureReason::HttpResponseTooLarge));
}
bytes.extend_from_slice(&chunk);
}
Ok(bytes)
}
fn http_unavailable(_error: reqwest::Error) -> WorkdirError {
async fn decode_error(response: reqwest::Response) -> WorkdirError {
match bounded_response_bytes(response).await {
Ok(bytes) => serde_json::from_slice::<WorkdirTransportError>(&bytes)
.map(WorkdirTransportError::into_workdir_error)
.unwrap_or_else(|_| {
WorkdirError::Transport("Runtime Workdir error response was invalid".into())
.with_failure_reason(crate::WorkdirFailureReason::HttpResponseInvalid)
}),
Err(error) => error,
}
}
fn http_unavailable(error: reqwest::Error) -> WorkdirError {
let reason = if error.is_timeout() {
crate::WorkdirFailureReason::HttpTimeout
} else if error.is_connect() {
crate::WorkdirFailureReason::HttpConnect
} else {
crate::WorkdirFailureReason::HttpTransfer
};
WorkdirError::Transport("Runtime Workdir HTTP request failed".to_string())
.with_failure_reason(reason)
}
#[cfg(test)]
@@ -942,6 +1058,40 @@ mod client {
use super::*;
use std::sync::Mutex;
#[tokio::test]
async fn http_failure_reasons_do_not_copy_urls_or_credentials() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let client = Client::builder().no_proxy().build().unwrap();
// A listening socket that never replies makes a real HTTP timeout.
let error = client
.get(format!("http://{address}/private?token=secret"))
.timeout(std::time::Duration::from_millis(25))
.send()
.await
.unwrap_err();
assert!(error.is_timeout());
let error = http_unavailable(error);
assert_eq!(
error.failure_reason(),
Some(crate::WorkdirFailureReason::HttpTimeout)
);
assert!(!format!("{error:?}").contains("secret"));
assert!(!format!("{error:?}").contains("/private"));
drop(listener);
let error = client
.get(format!("http://{address}/private?token=secret"))
.timeout(std::time::Duration::from_secs(2))
.send()
.await
.unwrap_err();
assert!(error.is_connect());
assert_eq!(
http_unavailable(error).failure_reason(),
Some(crate::WorkdirFailureReason::HttpConnect)
);
}
#[derive(Debug, Default)]
struct RejectAuthorization {
requests: Arc<Mutex<Vec<(String, String, Vec<u8>)>>>,
@@ -1118,6 +1268,7 @@ mod tests {
(WorkdirTransportErrorCode::Internal, 500, "operation failed"),
] {
let transport = WorkdirTransportError {
failure_reason: None,
code,
denial_reason: None,
message: if code == WorkdirTransportErrorCode::Conflict {

Some files were not shown because too many files have changed in this diff Show More