99 Commits
Author SHA1 Message Date
Hare 897c025b85 feat(journal): execute frozen body inputs with durable delivery and erasure fences 2026-10-27 08:03:43 +09:00
Hare 6776973e4c fix: preserve original session coordinates through capture selection 2026-10-22 23:17:25 +09:00
Hare 994e75233b feat(session): share immutable captures and bounded body transport 2026-10-22 22:25:47 +09:00
Hare 8b6e3070dd fix: align Workdir session cwd and model environment
Project alias, cwd, scope and command ceilings from actual sessions. Bind initial attachment cwd once in the shared broker/router while keeping root-relative delegation and detach generations. Preserve caller coordinates across reads, searches, checked checkout and commands; keep provider authorization and symlink boundaries.

Connect real SubWorkers, all filesystem Tools, scoped aliases, HTTP providers and limited Host grants in permanent regressions. Validated related normal suites (workdir http-client, tools, worker, worker-runtime, server-api typescript, fs-operation), workspace-server normal suite (1272 passed, 3 existing ignores), root check, product binary builds, rustdoc/doctests, generated contract checks, format and state-contract names. Test roots are checkout-local; Git parent discovery requires GIT_CEILING_DIRECTORIES at TMPDIR, confirmed against exact b860 baseline.
2026-10-22 20:46:10 +09:00
Hare b860c67ea2 fix: align Runtime restore diagnostics and socket test fixtures 2026-10-22 03:46:46 +09:00
Hare 533477c2e7 feat: unify headless execution under the Standalone Host
Expose an optional private same-uid Unix operator adapter and JSONL serve/attach CLI using the production Standalone Profile, store, restore lease and shutdown lifecycle. Remove direct Worker subprocess/discovery/readiness/controller sockets and parent socket callbacks; retain metadata-only peer registration and legacy safety readers.

Validated real Host/client input, events, provenance, disconnect, admitted shutdown and shared production restore/duplicate exclusion; normal standalone/worker/manifest/protocol/client/yoi/tui/server tests, root check, rustdoc, fmt and state-contract audit. Runtime lib: 476 passed, damaged_checkout_does_not_block_restore_but_is_rejected_when_used fails at missing_git diagnostic; unchanged restore logic, not relaxed or retried. Runtime bins/docs pass.
2026-10-22 01:29:39 +09:00
Hare 5aa2dc579a fix: integrate bounded Bash cleanup diagnostics 2026-10-22 00:46:41 +09:00
Hare 280b3d2a25 Merge branch 'develop' of /home/hare/Projects/yoi into work/T-775-cleanup-diagnostics 2026-10-22 00:24:00 +09:00
Hare ec74f4cdc4 fix: preserve Workdir failure diagnostics and audit outcomes 2026-10-22 00:16:11 +09:00
Hare 5af9dc8e4a test: focus checkout hardlink coverage on alias and symlink contracts 2026-10-21 23:57:01 +09:00
Hare ca6d4b4b72 fix: audit validated external workdir operation outcomes 2026-10-21 20:39:46 +09:00
Hare cccecbe7ca fix: integrate command output failure diagnostics 2026-10-21 06:52:23 +09:00
Hare 7a64c38493 fix: merge shared command diagnostic assertions
# Conflicts:
#	crates/workspace-server/src/server.rs
2026-10-21 06:40:26 +09:00
Hare b1361d86e9 fix: distinguish external Workdir admission dispatch and response failures
Integrate the published T773 audit contract without overwriting operation outcomes. Preserve bounded admission, send refusal, response loss/timeout and invalid result reasons independently of audit failures and request stages. Signed HTTP/SQLite/log fault tests pass (10), API logging 14 and relevant focused assertions pass. Root check, docs, generated artifacts, fmt/diff and state-contract naming checks pass. Full six-package normal suite ran with no-fail-fast but is not green: long Unix socket fixture paths and existing shared close assertion plus environment-isolation failures are recorded in target/t774 logs. Isolated focused checks resolved the latter; socket validation requires a granted shorter test root. This is a partial checkpoint, not a reviewed completion.
2026-10-21 06:39:54 +09:00
Hare 6181426abd fix: assert retained checkout outcome diagnostics 2026-10-21 06:27:54 +09:00
Hare cd331b93c6 Merge remote-tracking branch 'origin/work/T-773-workdir-audit' into work/T-774-workdir-failure-diagnostics
# Conflicts:
#	crates/workdir/src/http.rs
2026-10-21 06:13:35 +09:00
Hare 60eabea19a fix: assert retained remote session close diagnostics 2026-10-21 06:13:16 +09:00
Hare 7394ec6784 fix: retain safe Workdir open authorization and response failure causes
Keep bounded reasons and derived failure stages separate from public classification. Reject provider-authored raw error text on transport decoding. Preserve identity mismatch when cleanup fails and retain its handle for explicit close without opening again. Add signed Runtime HTTP, Registry/API logging, remote mismatch, authorization no-dispatch and deferred fault coverage. Focused tests and Workdir suite pass; full-source validation follows T773 audit contract merge.
2026-10-21 06:13:07 +09:00
Hare 28b7cc414b fix: retain external workdir results independently of audit failures 2026-10-21 06:04:39 +09:00
Hare 86d5cf9403 fix: integrate safe Drive and config failure diagnostics 2026-10-21 06:01:51 +09:00
Hare 6f0a23e5cc Merge commit '62e2b4430588bacf90a2812a0a88772fc0a00c58' into work/T-775-cleanup-diagnostics 2026-10-21 05:56:16 +09:00
Hare 2f4a46bbdb Merge commit 'a1748b552c52fd5f68de6ba852421fd778aec010' into work/T-775-cleanup-diagnostics 2026-10-21 05:55:48 +09:00
Hare 4872cf2aac fix: retain independent Bash Drop cleanup diagnostics 2026-10-21 05:55:48 +09:00
Hare 734d4b44ae Merge remote-tracking branch 'origin/work/T-772-command-output-integrity' into work/T-774-workdir-failure-diagnostics 2026-10-21 05:55:33 +09:00
Hare 6810f51f46 fix: retain safe Drive and config adapter failure diagnostics 2026-10-21 05:53:58 +09:00
Hare a1748b552c fix: validate closed-session diagnostics and fault quotas 2026-10-21 05:52:48 +09:00
Hare 62b2d27b4a fix: retain command failure stages and output collection state 2026-10-21 05:40:39 +09:00
Hare 62e2b44305 fix: retain safe Journal compaction failure diagnostics
Keep stage/reason classification in the Journal Feature without changing frozen input, metadata or fallback behavior. Exercise connection, Host read failures, size budget and secret-bearing fixtures through compaction persistence.

Validation: old freeze function with new tests fails connection and snapshot/size diagnostics (3 failures); new code passes all six selected cases and cargo test -p worker. Serialization is currently unreachable for the derived JSON-safe snapshot fields; no artificial runtime failure is claimed.
2026-10-21 05:32:07 +09:00
Hare aa0fb2f787 fix: continue workdir output reads after long-poll expiry 2026-10-21 04:48:43 +09:00
Hare 1b6d4dbfd8 fix: preserve workdir failure diagnostics across transport 2026-10-21 01:22:04 +09:00
Hare 4ef7b03651 fix: stop background session checks and loading flicker 2026-10-20 22:35:38 +09:00
Hare af615ab177 fix: retain archive and worker restore failure diagnostics 2026-10-20 22:00:58 +09:00
Hare 338e39bcdb fix: defer restored workdir access until use 2026-10-20 22:00:58 +09:00
Hare 3cfe04be3a fix: preserve saved feature configuration during worker restore 2026-10-20 22:00:39 +09:00
Hare 70d153b365 fix: isolate unreadable archives from runtime startup 2026-10-20 17:39:38 +09:00
Hare ce93849948 chore: merge develop into hare/develop 2026-10-20 15:39:09 +09:00
Hare 3c499719fe feat: default Coder ticket work to review flow 2026-10-20 15:38:50 +09:00
Hare 9e1d3df7ad fix: remove overbroad Worker bypass instructions 2026-10-20 15:38:50 +09:00
Hare 18bac16a23 fix: require replay-safe control lifecycle for Subject delegation 2026-10-20 08:12:26 +09:00
Hare 9005856f77 fix: replay accepted delegation without reserving active Workdirs 2026-10-20 07:24:51 +09:00
Hare e39372a52a Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants 2026-10-20 06:38:12 +09:00
Hare 223a74a9e2 fix: retain byte command transport and explicit authorization fixtures 2026-10-20 06:38:12 +09:00
Hare f38d9a16ea Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants
# Conflicts:
#	crates/worker-runtime/src/worker_backend.rs
#	openapi/server-api.json
#	web/workspace/src/lib/generated/companion-api.ts
#	web/workspace/src/lib/generated/repository-api.ts
#	web/workspace/src/lib/generated/runtime-api.ts
#	web/workspace/src/lib/generated/ticket-api.ts
#	web/workspace/src/lib/generated/worker-launch-api.ts
2026-10-20 06:24:11 +09:00
Hare b135fa5e99 feat: enforce explicit Subject permissions and delegated Worker grants 2026-10-20 06:22:58 +09:00
Hare 07b8b6d41b Merge commit 'b8e548b92d3c785c482dbb418517cc5a2cf9e7e0' into refactor/T-765-session-feature-boundary 2026-10-20 05:34:22 +09:00
Hare 8017c55d82 fix: retain annotated compaction history and honor non-System input policy 2026-10-20 05:34:22 +09:00
Hare b8e548b92d chore: integrate reviewed browser session navigation 2026-10-20 05:31:32 +09:00
Hare 90e38551cf Merge commit '00ad02d04728a8e32a82a4e72747fefcf8a82d4e' into refactor/T-765-session-feature-boundary
# Conflicts:
#	crates/worker/src/feature/builtin.rs
2026-10-20 05:25:16 +09:00
Hare 3e8164e757 fix: fence hosting setup verification against superseded sessions 2026-10-20 04:53:43 +09:00
Hare 3b3381900f fix: preserve role-neutral input policy across compaction and replay 2026-10-20 04:45:50 +09:00
Hare 00ad02d047 chore: integrate bounded shared text page refusal 2026-10-20 03:55:13 +09:00
Hare a774925dec fix: reject byte-truncated line windows before advancing search 2026-10-20 03:53:03 +09:00
Hare 8f759911ab fix: emit Session use policy type in browser bindings 2026-10-20 03:52:55 +09:00
Hare daf1f6ed50 fix: route retained Session migration through Runtime staging 2026-10-20 03:52:55 +09:00
Hare 86602240fa refactor: bind Subject metadata and prompt confirmation to Host lifecycle 2026-10-20 03:52:55 +09:00
Hare bfeee487fd feat(auth): separate anonymous entry from expired session recovery 2026-10-20 03:52:48 +09:00
Hare f9830fe7dd test: reject Bash search pages that would skip an unread long line 2026-10-20 03:40:36 +09:00
Hare de96c5b577 chore: update shared file contracts and validation fixtures
# Conflicts:
#	crates/worker/src/checkout_http_tests.rs
2026-10-20 03:26:56 +09:00
Hare 6e729ed06c refactor: isolate Feature System input migration and compaction policy 2026-10-20 03:21:22 +09:00
Hare f38b2a52b1 fix: record native prior Read only after result validation 2026-10-20 03:21:14 +09:00
Hare dd5384df3b feat: expose retained Bash streams through common WIP file contracts 2026-10-20 03:13:46 +09:00
Hare 6576850b7a chore: connect shared file contracts for Bash validation 2026-10-20 03:03:53 +09:00
Hare 4555aace28 feat: capture bounded Bash output over replayable byte transport 2026-10-20 03:01:22 +09:00
Hare 47b32ef6ab fix: bind capability changes to Object freshness and update contract consumers 2026-10-20 02:53:21 +09:00
Hare bb5c50f9ab feat: persist System input policies through public consumers 2026-10-20 02:44:28 +09:00
Hare 32a409c15d feat: share virtual file contracts and checked binary byte ranges 2026-10-20 02:18:49 +09:00
Hare 19514a5d6a feat: define bounded Feature-owned system input envelopes 2026-10-20 02:14:52 +09:00
Hare 887184796e docs: distinguish commit completion from Feature authority 2026-10-20 02:05:02 +09:00
Hare cd0eb5eb00 fix: allowlist public system input projections 2026-10-20 02:02:28 +09:00
Hare e8dfb3e68d refactor: notify system input producers after durable commit 2026-10-20 02:02:28 +09:00
Hare 434ce34da7 refactor: bind checkout target lifetimes to Object observations 2026-10-20 02:01:22 +09:00
Hare bfb51b7ca9 fix: bound encoded common text results including escape and record overhead 2026-10-20 01:48:23 +09:00
Hare 38d9069632 feat: publish stable root file contracts and target-owned dispatch 2026-10-20 01:46:30 +09:00
Hare e5749b9be0 test: integrate Drive connection and lifetime coverage 2026-10-20 00:37:32 +09:00
Hare 387b5ff7df docs: record Drive connection coverage and fault evidence 2026-10-20 00:35:11 +09:00
Hare ec6c5beda0 test: connect Drive publication to client and resource lifetimes 2026-10-20 00:34:50 +09:00
Hare 8e40fae6f4 fix: recover unpublished Subject creation and preserve auth rejections 2026-10-19 18:56:24 +09:00
Hare c961f244ab fix: preserve feature metadata for concise Journal startup context 2026-10-19 18:45:32 +09:00
Hare cea0225a6e fix: migrate legacy session history and retained archives 2026-10-19 13:52:43 +09:00
Hare b3189d8e85 fix: preserve worker retention failure diagnostics 2026-10-19 13:52:43 +09:00
Hare 24bc45a31e chore: merge develop into hare/develop 2026-10-19 01:31:23 +09:00
Hare b02360fd75 fix: restore orchestrator target push requirement 2026-10-19 01:31:01 +09:00
Hare 41c9d2d5c7 docs: record restored parent regression evidence and cost 2026-10-19 00:02:02 +09:00
Hare cc66bd5052 test: connect restored Runtime parent tools to child session 2026-10-19 00:02:02 +09:00
Hare d1408d3a2f fix: project runtime list subjects for Worker control tools 2026-10-19 00:02:02 +09:00
Hare 19122fb5ea fix: use public Worker control entry in regression design 2026-10-19 00:02:02 +09:00
Hare 1fb216e6ae fix: refresh Ticket regression design evidence 2026-10-19 00:02:02 +09:00
Hare 4b14f7e6b3 docs: design prioritized regression coverage 2026-10-19 00:02:02 +09:00
Hare 67767eeee7 test: integrate Ticket public regression coverage 2026-10-18 02:35:16 +09:00
Hare 1bfb4ce119 fix: align Ticket regression design with Host retry identity 2026-10-18 02:10:15 +09:00
Hare 651820dc6d test: connect Ticket public regressions to real Backend 2026-10-18 02:10:15 +09:00
Hare 75b1557d63 feat: browse Subject Journals and persistent Memory links 2026-10-18 02:03:51 +09:00
Hare 23ff518209 fix: clarify Ticket regression fixture boundaries 2026-10-18 01:26:52 +09:00
Hare fab9e0f4b5 docs: design focused Ticket regression coverage 2026-10-18 01:26:52 +09:00
Hare 5ee42b8db8 docs: record Journal compaction validation and evaluation limits 2026-10-17 12:29:47 +09:00
Hare daf438701f docs: explain shared Journal snapshot boundary guarantees 2026-10-17 12:25:24 +09:00
Hare 23ee401287 fix: disambiguate historical Journal retention instructions 2026-10-17 12:25:24 +09:00
Hare 8f598da931 test: verify frozen Journal compaction across Hosts and restores 2026-10-17 12:25:24 +09:00
Hare 956aeed6b2 feat: freeze Host Journal context across compaction 2026-10-17 12:19:37 +09:00
376 changed files with 67219 additions and 13785 deletions
Generated
+10 -2
View File
@@ -3990,6 +3990,7 @@ dependencies = [
"protocol",
"serde",
"serde_json",
"session-store",
"tokio",
"workdir",
]
@@ -4783,12 +4784,14 @@ version = "0.1.0"
dependencies = [
"chrono",
"feature-storage",
"job",
"manifest",
"memory",
"rusqlite",
"serde",
"serde_json",
"server-api",
"session-store",
"sha2 0.11.0",
"tempfile",
"thiserror 2.0.18",
@@ -5266,6 +5269,7 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"tracing",
"uuid",
"workdir",
]
@@ -6496,6 +6500,7 @@ name = "workdir"
version = "0.1.0"
dependencies = [
"async-trait",
"axum",
"fs-operation",
"libc",
"manifest",
@@ -6519,7 +6524,6 @@ dependencies = [
"axum",
"base64 0.22.1",
"chrono",
"clap",
"client",
"config-source",
"dotenv",
@@ -6557,6 +6561,7 @@ dependencies = [
"toml",
"tools",
"tracing",
"tracing-subscriber",
"tungstenite 0.28.0",
"uuid",
"wip-client",
@@ -6596,6 +6601,7 @@ dependencies = [
"tokio",
"tokio-tungstenite 0.29.0",
"toml",
"tools",
"tower",
"tracing",
"tracing-subscriber",
@@ -6683,6 +6689,7 @@ dependencies = [
"futures",
"manifest",
"project-record",
"protocol",
"reqwest",
"serde",
"serde_json",
@@ -6698,13 +6705,13 @@ dependencies = [
"tui",
"uuid",
"workdir",
"worker",
]
[[package]]
name = "yoi-e2e"
version = "0.0.0"
dependencies = [
"chrono",
"libc",
"reqwest",
"serde",
@@ -6714,6 +6721,7 @@ dependencies = [
"tempfile",
"webauthn-authenticator-rs",
"webauthn-rs-proto",
"yoi-workspace-server",
]
[[package]]
+1
View File
@@ -908,6 +908,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let response = create_backend_worker_with_client(&target, &create, &api)
+1
View File
@@ -567,6 +567,7 @@ impl BackendWorkspaceProductClient {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let workspace_id = self.workspace_id.clone();
let response: BrowserCreateWorkerResponse = self.generated(move |client| async move {
+30
View File
@@ -200,6 +200,7 @@ fn run_read_with_limits(
total_lines,
content_hash: content_hasher.finalize().into(),
truncated: end < total_lines || byte_truncated,
byte_truncated,
})
}
@@ -885,6 +886,34 @@ mod bounded_read_tests {
assert!(error.to_string().contains("exceeds provider limit 4096"));
}
#[test]
fn bounded_read_distinguishes_utf8_byte_cut_from_complete_line_continuation() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
fs::write(root.join("text"), "雪雪needle\nnext").unwrap();
for (offset, max_bytes, bytes, more, cut) in [
(0, 5, "雪", true, true),
(0, 64, "雪雪needle\n", true, false),
(1, 64, "next", false, false),
] {
let result = run_read_bounded(
&root,
ReadRequest {
path: FsPath::new("text").unwrap(),
offset,
limit: 1,
max_bytes,
},
&RootAccess(root.clone()),
BoundedReadLimits::new(1024, 64).unwrap(),
)
.unwrap();
assert_eq!(result.bytes, bytes.as_bytes());
assert_eq!(result.truncated, more);
assert_eq!(result.byte_truncated, cut);
}
}
#[test]
fn bounded_read_caps_retained_response_independently_of_request() {
let temp = tempfile::tempdir().unwrap();
@@ -907,6 +936,7 @@ mod bounded_read_tests {
assert_eq!(result.bytes, b"alpha");
assert_eq!(result.total_lines, 3);
assert!(result.truncated);
assert!(result.byte_truncated);
}
}
+4
View File
@@ -204,7 +204,11 @@ pub struct ReadResult {
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub total_lines: usize,
pub content_hash: ContentHash,
/// More source lines remain, or the selected window exceeded the response byte budget.
pub truncated: bool,
/// The selected line window was cut by its byte budget, possibly within a line.
/// Line-based cursors must not treat such a window as completely scanned.
pub byte_truncated: bool,
}
/// Binary-safe byte range read with an optional whole-file version fence.
+1 -38
View File
@@ -100,21 +100,6 @@ pub fn worker_allocation_path() -> Option<PathBuf> {
worker_allocation_path_from_runtime_dir(runtime_dir())
}
/// `<runtime_dir>/<worker_name>/` — per-Worker runtime directory.
pub fn worker_runtime_dir(worker_name: &str) -> Option<PathBuf> {
worker_runtime_dir_from_runtime_dir(runtime_dir(), worker_name)
}
/// `<runtime_dir>/<worker_name>/sock` — Worker's Unix socket path.
///
/// The `worker` crate's `RuntimeDir::socket_path()` creates the socket inside the
/// Worker process. This function **predicts** the path for external callers that
/// know the Worker name, such as the TUI attach flow. Both should return the same
/// path.
pub fn worker_socket_path(worker_name: &str) -> Option<PathBuf> {
worker_socket_path_from_runtime_dir(runtime_dir(), worker_name)
}
// ---- internals --------------------------------------------------------------
fn resolve_config_dir_from_parts(
@@ -204,20 +189,6 @@ fn worker_allocation_path_from_runtime_dir(runtime_dir: Option<PathBuf>) -> Opti
Some(runtime_dir?.join("workers.json"))
}
fn worker_runtime_dir_from_runtime_dir(
runtime_dir: Option<PathBuf>,
worker_name: &str,
) -> Option<PathBuf> {
Some(runtime_dir?.join(worker_name))
}
fn worker_socket_path_from_runtime_dir(
runtime_dir: Option<PathBuf>,
worker_name: &str,
) -> Option<PathBuf> {
Some(worker_runtime_dir_from_runtime_dir(runtime_dir, worker_name)?.join("sock"))
}
/// Treats empty environment values as unset. `std::env::var` distinguishes
/// `Ok("")` from `Err(NotPresent)`, but path resolution treats both as unset
/// for intuitive behavior.
@@ -455,16 +426,8 @@ mod tests {
PathBuf::from("/sand/sessions")
);
assert_eq!(
worker_allocation_path_from_runtime_dir(runtime_dir.clone()).unwrap(),
worker_allocation_path_from_runtime_dir(runtime_dir).unwrap(),
PathBuf::from("/sand/run/workers.json")
);
assert_eq!(
worker_runtime_dir_from_runtime_dir(runtime_dir.clone(), "foo").unwrap(),
PathBuf::from("/sand/run/foo")
);
assert_eq!(
worker_socket_path_from_runtime_dir(runtime_dir, "foo").unwrap(),
PathBuf::from("/sand/run/foo/sock")
);
}
}
+27 -26
View File
@@ -52,7 +52,7 @@ pub enum AuthenticatedInputSource {
/// Assigned whenever a serialized tracked method crosses an untrusted
/// protocol boundary. Receivers must handle it exactly like public input.
UntrustedWire,
/// Stamped only by the operator-owned standalone in-process transport.
/// Stamped only by the operator-owned Standalone Host after local admission.
/// This is not a Backend Account or authority accepted from wire fields.
LocalOperator,
Account {
@@ -399,16 +399,13 @@ pub enum Method {
/// Typed lifecycle events sent from a child Worker to its parent.
///
/// Delivered as `Method::WorkerEvent` over the parent's Unix socket. The
/// parent Controller always applies variant-specific side effects
/// (registry / worker-allocation updates). Agent-visible variants are also
/// queued into the notification buffer; control-plane-only variants are
/// not injected into the parent's LLM context.
/// Delivered in-process to the parent Controller's protocol intake. Agent-visible variants
/// are queued into the notification buffer. Reports are advisory: receivers must check the
/// actual child session before treating its work as complete.
///
/// Transport is fire-and-forget; receivers must tolerate out-of-order
/// delivery (e.g. `TurnEnded` arriving after `ShutDown` for the same
/// child Worker).
#[derive(Debug, Clone, Serialize, Deserialize)]
/// Retained legacy control-only variants can be decoded from saved history, but do not
/// register live children, delegate scope, or establish a socket callback.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
@@ -429,16 +426,8 @@ pub enum WorkerEvent {
/// Child has stopped (controller loop is exiting).
ShutDown { worker_name: String },
/// Child SubWorker sub-delegated scope to a grandchild SubWorker via `SubWorkerSpawn`.
///
/// Control-plane only: receivers apply registry side effects and
/// propagate upward, but do not expose this as an agent notification.
///
/// The parent uses this to add the grandchild to its own
/// `spawned_workers.json` so it can manage the grandchild directly
/// even if the intermediate child dies. The parent then re-fires
/// this event upward (if it has a parent of its own) to maintain
/// the chain to root.
/// Legacy process-child scope report retained for saved Session decoding only.
/// Live Controllers ignore it; only the Host-owned Internal registry grants child scope.
ScopeSubDelegated {
/// Sub-delegating Worker (= the sender itself).
parent_worker: String,
@@ -982,6 +971,17 @@ pub enum SessionEntryProvenance {
LegacyUnknown,
}
/// Public use decisions contain no Feature identity or private payload. Absence
/// denotes the older projection contract, not permission to expose System input.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(deny_unknown_fields)]
pub struct SessionEntryUsePolicy {
pub observation_readable: bool,
pub memory_evidence: bool,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -994,6 +994,8 @@ pub struct SessionSnapshotEntry {
pub provenance: SessionEntryProvenance,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub derived_from: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub use_policy: Option<SessionEntryUsePolicy>,
#[serde(flatten)]
pub data: SessionSnapshotEntryData,
}
@@ -1180,12 +1182,10 @@ pub enum Event {
},
/// One agent-injected system item committed to history.
///
/// Carries the JSON form of `session_store::SystemItem`. Covers
/// `Method::Notify` echoes, child-Worker lifecycle events from
/// `Method::WorkerEvent`, `@<path>` resolution payloads, and any future
/// Clients dispatch on the `kind` tag for typed rendering instead
/// of parsing free-text prefixes like `[Notification] …` or
/// `[File: …]`.
/// Carries the allowlisted JSON form of `session_store::PublicSystemItem`,
/// not the storage record. Its descriptive `kind` is independent from the
/// typed `presentation` used for notification, reminder, or context rendering.
/// Private Feature payloads and prompt provenance are never event data.
///
/// One event per `LogEntry::AnnotatedSystemItem` commit. Disk-side and
/// wire-side are 1:1.
@@ -2621,6 +2621,7 @@ mod tests {
session: SessionSnapshot {
pending_submissions: PendingSubmissionsSnapshot::default(),
entries: vec![SessionSnapshotEntry {
use_policy: None,
entry_id: "entry-1".into(),
timestamp: 1,
provenance: SessionEntryProvenance::HumanInput,
+7 -6
View File
@@ -17,12 +17,12 @@ use crate::{
PasteArtifactRef, PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission,
ReasoningConfig, RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason,
ScopeRule, Segment, SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment, SubmissionDisposition,
SymlinkPolicy, ToolResultDisposition, TurnResult, UploadedFileAvailability, UploadedFileRef,
WorkerBusyState, WorkerCommandAcknowledgement, WorkerCommandDisposition, WorkerCommandEnvelope,
WorkerCommandKind, WorkerEvent, WorkerMaintenanceState, WorkerRunState, WorkerState,
WorkerStateSnapshot, WorkerStatus,
SessionEntryUsePolicy, SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole,
SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment,
SubmissionDisposition, SymlinkPolicy, ToolResultDisposition, TurnResult,
UploadedFileAvailability, UploadedFileRef, WorkerBusyState, WorkerCommandAcknowledgement,
WorkerCommandDisposition, WorkerCommandEnvelope, WorkerCommandKind, WorkerEvent,
WorkerMaintenanceState, WorkerRunState, WorkerState, WorkerStateSnapshot, WorkerStatus,
subscription::{
EventSubscriptionSelector, MAX_CORRELATION_ID_BYTES, MAX_REJECTION_MESSAGE_BYTES,
MAX_RESOURCE_ID_BYTES, MAX_SUBSCRIPTION_COLLECTION_ITEMS,
@@ -117,6 +117,7 @@ pub fn generated_protocol_types() -> String {
push_decl::<InFlightCompaction>(&cfg, &mut output);
push_decl::<InFlightSnapshot>(&cfg, &mut output);
push_decl::<SessionEntryProvenance>(&cfg, &mut output);
push_decl::<SessionEntryUsePolicy>(&cfg, &mut output);
push_decl::<SessionMessageRole>(&cfg, &mut output);
push_decl::<SessionContentPart>(&cfg, &mut output);
push_decl::<SessionToolAttachment>(&cfg, &mut output);
+1
View File
@@ -9,6 +9,7 @@ license.workspace = true
api-macros = { workspace = true, features = ["reqwest", "axum"] }
manifest.workspace = true
protocol.workspace = true
session-store.workspace = true
serde = { workspace = true, features = ["derive"] }
workdir.workspace = true
+102 -1
View File
@@ -518,9 +518,10 @@ pub struct WorkerRestorePreparation {
pub workspace_api: Option<WorkspaceApiRef>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workdir_attachments: Option<Vec<LogicalWorkdirAttachment>>,
/// Retained in saved Restore intents; Restore does not apply Repository access.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access: Vec<WorkingDirectoryRepositoryAccessRequest>,
/// Read-only references resolved by Workspace only after this operation owns admission.
/// Retained references from saved resource preparation, not Restore prerequisites.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access_workdirs: Vec<RepositoryAccessWorkdirReference>,
}
@@ -835,6 +836,34 @@ pub struct SessionMaterialPinResponse {
pub pin_id: String,
}
/// One body-bearing immutable slice. The byte budget includes slice metadata,
/// but not the enclosing availability enum or the caller's durable Job envelope.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionRecordingCaptureRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub start: session_store::capture::RecordingPosition,
pub max_entries: usize,
pub max_serialized_bytes: usize,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionRecordingCaptureAvailability {
NotMaterialized {
session_id: String,
},
Captured {
slice: session_store::capture::RecordingCaptureSlice,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialCatalogRequest {
@@ -1140,8 +1169,36 @@ pub struct SubjectReleaseResponse {
pub state: SubjectReleaseState,
}
/// Irreversibly fence an unpublished failed Subject create. Exact retries must retain
/// all four fields; this is distinct from releasing an existing Subject Controller.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryRequest {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
/// Durable admission-fence proof plus freshly confirmed execution absence.
/// An error (including a lost response) is never evidence of absence.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryResponse {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
#[api(reqwest, axum)]
pub trait RuntimeApi {
#[post("/v1/subject-create-recovery", status = 200, error_status = 400)]
async fn recover_subject_create(
&self,
#[body] request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError>;
#[post("/v1/subject-release", status = 200, error_status = 400)]
async fn subject_release(
&self,
@@ -1217,6 +1274,16 @@ pub trait RuntimeApi {
#[body] request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError>;
#[post(
"/v1/session-public/recording-capture",
status = 200,
error_status = 400
)]
async fn session_recording_capture(
&self,
#[body] request: SessionRecordingCaptureRequest,
) -> Result<SessionRecordingCaptureAvailability, RuntimeApiError>;
#[post(
"/v1/session-public/material/catalog",
status = 200,
@@ -1675,6 +1742,18 @@ mod tests {
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn recover_subject_create(
&self,
request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError> {
Ok(SubjectCreateRecoveryResponse {
workspace_id: request.workspace_id,
worker_id: request.worker_id,
create_fingerprint: request.create_fingerprint,
operation_id: request.operation_id,
})
}
async fn subject_release(
&self,
_request: SubjectReleaseRequest,
@@ -1689,6 +1768,13 @@ mod tests {
Err(test_error(501))
}
async fn session_recording_capture(
&self,
_request: SessionRecordingCaptureRequest,
) -> Result<SessionRecordingCaptureAvailability, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_catalog(
&self,
_request: SessionMaterialCatalogRequest,
@@ -1820,6 +1906,21 @@ mod tests {
.build()
.unwrap();
let recovery = SubjectCreateRecoveryRequest {
workspace_id: "workspace-a".into(),
worker_id: WorkerId::now_v7(),
create_fingerprint: "failed-create-fingerprint".into(),
operation_id: "recovery-1".into(),
};
let response = client
.recover_subject_create(recovery.clone())
.await
.unwrap();
assert_eq!(response.workspace_id, recovery.workspace_id);
assert_eq!(response.worker_id, recovery.worker_id);
assert_eq!(response.create_fingerprint, recovery.create_fingerprint);
assert_eq!(response.operation_id, recovery.operation_id);
let summary = client.runtime_summary().await.unwrap();
assert_eq!(summary.runtime.display_name.as_deref(), Some("round-trip"));
let workers = client
+475
View File
@@ -0,0 +1,475 @@
//! Neutral explicit-allow authorization contracts. Resource containment never crosses kinds.
//!
//! # Trusted Feature boundary
//!
//! A Profile requests execution configuration, not authority. The Worker asks the
//! Host for each enabled Feature's exact activation name before initialization,
//! then trusted Feature code asks for each protected operation and its semantic
//! resource before access or effects. `ticket` on [`PermissionResource::Feature`]
//! activates Ticket; it implies neither `ticket.create` nor any future operation.
//! `ticket.create` on a Workspace permits creation there but cannot activate the
//! Feature. There are no wildcard permissions or implicit dotted-name inheritance.
//!
//! Identity is absent from the authorization request intentionally: the Host
//! authenticates the bound Runtime/Worker and resolves current source permissions,
//! body ownership and any per-Worker execution grant. A shared client, Profile,
//! remembered receipt or caller-supplied Subject claim is not authority. Features
//! depend on this contract, never on a particular Subject policy implementation.
//!
//! Supported cooperative boundaries are Feature activation, ordinary Worker
//! delegation, Ticket creation (Tool/native WIP/typed service), Workdir filesystem
//! read/write and whole-attachment command execution, and scoped result notify.
//! Workdir providers still enforce resolved paths, traversal, capability, Ticket
//! and provider ceilings. The returned scope is for provider narrowing, not a
//! transferable credential. Scoped checkout/overlap operations are refused until
//! their complete access semantics are adapted. Other Feature operations retain
//! their existing owner checks; they do not automatically acquire this operation
//! policy. Missing calls are trusted Feature bugs, not interception by a universal
//! Tool executor or protection against malicious Feature code. Connection tests
//! must exercise each exposed surface, assert a genuine permitted effect and
//! assert zero downstream calls on denial, including revocation and provider denial.
//!
//! # Delegation callers and recovery
//!
//! Callers such as a chat Feature supply explicit operation/resource selections
//! and retain the same public operation ID and full request after an unknown
//! result. Normal Profile/Worker allocation and control/resource grants are reused;
//! there is no Subject Worker registry. Effective grants bind immutable Profile
//! resolution, the reserved Worker and source assignment. Permission replacement
//! invalidates existing children rather than refreshing them. Body replacement,
//! retirement and restore must recheck current authority; historical result lookup
//! is not new execution permission. Cancellation acceptance is distinct from
//! terminal confirmation and does not undo completed effects.
//!
//! A child receives no Subject body identity, singleton, whole Memory or public
//! conversation-post authority. `worker.result` is explicit notify authority to a
//! named origin Worker, not arbitrary control or publication. Chat delivery and
//! conversation correlation belong to the caller integration, not this contract.
//! Unset existing Subject policies deny new Subject execution/delegation; ordinary
//! non-Subject Workers explicitly retain existing Backend/provider contracts.
//! Standalone non-Workspace activation compatibility grants no Workspace operation.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
pub const MAX_SCOPED_PERMISSIONS: usize = 256;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PermissionResource {
Feature,
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
// Serde's internally tagged unit variants ignore extra fields even with deny_unknown_fields.
// A struct-shaped wire variant preserves the public unit variant while rejecting those claims.
impl<'de> Deserialize<'de> for PermissionResource {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
#[derive(Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
enum Wire {
Feature {},
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
Ok(match Wire::deserialize(deserializer)? {
Wire::Feature {} => Self::Feature,
Wire::Workspace { workspace_id } => Self::Workspace { workspace_id },
Wire::Workdir {
working_directory_id,
path,
recursive,
} => Self::Workdir {
working_directory_id,
path,
recursive,
},
Wire::Worker {
runtime_id,
worker_id,
} => Self::Worker {
runtime_id,
worker_id,
},
})
}
}
impl PermissionResource {
pub fn validate(&self) -> Result<(), String> {
match self {
Self::Feature => Ok(()),
Self::Workspace { workspace_id } => validate_authorization_identifier(workspace_id),
Self::Workdir {
working_directory_id,
path,
..
} => {
validate_authorization_identifier(working_directory_id)?;
validate_relative_permission_path(path)
}
Self::Worker {
runtime_id,
worker_id,
} => {
validate_authorization_identifier(runtime_id)?;
validate_authorization_identifier(worker_id)
}
}
}
pub fn covers(&self, other: &Self) -> bool {
if self.validate().is_err() || other.validate().is_err() {
return false;
}
match (self, other) {
(Self::Feature, Self::Feature) => true,
(Self::Workspace { workspace_id: a }, Self::Workspace { workspace_id: b }) => a == b,
(
Self::Worker {
runtime_id: ar,
worker_id: aw,
},
Self::Worker {
runtime_id: br,
worker_id: bw,
},
) => ar == br && aw == bw,
(
Self::Workdir {
working_directory_id: a,
path: ap,
recursive: ar,
},
Self::Workdir {
working_directory_id: b,
path: bp,
recursive: br,
},
) => {
a == b
&& if *ar {
ap == "."
|| ap == bp
|| bp
.strip_prefix(ap)
.is_some_and(|tail| tail.starts_with('/'))
} else {
ap == bp && !br
}
}
_ => false,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ScopedPermission {
pub operation: String,
pub resource: PermissionResource,
}
impl ScopedPermission {
pub fn validate(&self) -> Result<(), String> {
validate_operation_name(&self.operation)?;
self.resource.validate()?;
if matches!(self.resource, PermissionResource::Feature) && self.operation.contains('.') {
return Err("Feature resources authorize activation names only, not operations".into());
}
Ok(())
}
pub fn covers(&self, other: &Self) -> bool {
self.validate().is_ok()
&& other.validate().is_ok()
&& self.operation == other.operation
&& self.resource.covers(&other.resource)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissions {
pub change_id: String,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsUpdate {
pub operation_id: String,
pub expected_change_id: Option<String>,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerExecutionGrant {
pub subject_id: String,
pub origin_runtime_id: String,
pub origin_worker_id: String,
pub source_change_id: String,
pub runtime_id: String,
pub worker_id: String,
pub profile: String,
pub profile_digest: String,
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerDelegationRequest {
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
impl WorkerDelegationRequest {
pub fn validate(&self) -> Result<(), String> {
validate_scoped_permissions(&self.permissions)?;
validate_scoped_permissions(&self.redelegation)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationRequest {
pub operation: String,
pub resource: PermissionResource,
}
impl OperationAuthorizationRequest {
pub fn validate(&self) -> Result<(), String> {
ScopedPermission {
operation: self.operation.clone(),
resource: self.resource.clone(),
}
.validate()
}
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationResponse {
/// Matching granted scope for provider-resolved checks, not a credential.
/// `None` identifies the explicit non-Subject legacy authorization path.
#[serde(default)]
pub scope: Option<PermissionResource>,
}
pub fn validate_operation_name(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| !value.split('.').all(|part| {
!part.is_empty()
&& part
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'_' || b == b'-')
})
{
return Err(
"permission operation must be an exact, nonempty dot-separated name (no wildcards)"
.into(),
);
}
Ok(())
}
pub fn validate_scoped_permissions(permissions: &[ScopedPermission]) -> Result<(), String> {
if permissions.len() > MAX_SCOPED_PERMISSIONS {
return Err("too many scoped permissions".into());
}
for permission in permissions {
permission.validate()?;
}
Ok(())
}
pub fn validate_authorization_identifier(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| value.trim() != value
|| value.chars().any(|c| c.is_control() || c == '*')
{
return Err("authorization identity must be nonempty, bounded and exact".into());
}
Ok(())
}
/// Paths are already-normalized lexical paths, not host filesystem paths. The Host must
/// still enforce symlink/provider boundaries; lexical containment grants no filesystem access.
pub fn validate_relative_permission_path(path: &str) -> Result<(), String> {
if path == "." {
return Ok(());
}
if path.is_empty()
|| path.len() > 4096
|| path.contains(['\\', ':'])
|| path.chars().any(char::is_control)
|| path
.split('/')
.any(|part| part.is_empty() || part == "." || part == "..")
{
return Err("permission path must be a normalized relative path without escape".into());
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn workdir(path: &str, recursive: bool) -> PermissionResource {
PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
}
}
#[test]
fn lexical_containment_rejects_escape_and_respects_component_and_recursion_boundaries() {
for path in [
"",
"/src",
"../src",
"src/../secret",
"src/./a",
"src//a",
"src/",
"C:/src",
"src\\a",
] {
assert!(workdir(path, true).validate().is_err(), "{path}");
}
let root = workdir(".", true);
let src = workdir("src", true);
assert!(root.covers(&src));
assert!(src.covers(&workdir("src/a", true)));
assert!(!src.covers(&workdir("src-other", false)));
assert!(!workdir("src", false).covers(&src));
assert!(workdir("src", false).covers(&workdir("src", false)));
assert!(!workdir("src", false).covers(&workdir("src/a", false)));
assert!(!src.covers(&PermissionResource::Workdir {
working_directory_id: "other".into(),
path: "src".into(),
recursive: false
}));
}
#[test]
fn operation_hierarchy_never_implies_inheritance_or_feature_activation() {
for name in [
"",
"*",
"ticket.*",
".ticket",
"ticket.",
"ticket..read",
"ticket read",
] {
assert!(validate_operation_name(name).is_err(), "{name}");
}
let activation = ScopedPermission {
operation: "ticket".into(),
resource: PermissionResource::Feature,
};
let create = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "ws".into(),
},
};
let parent_name = ScopedPermission {
operation: "ticket".into(),
resource: create.resource.clone(),
};
assert!(!parent_name.covers(&create));
assert!(!create.covers(&parent_name));
assert!(!activation.covers(&create));
assert!(!create.covers(&activation));
assert!(
ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Feature
}
.validate()
.is_err()
);
assert!(
!PermissionResource::Workspace {
workspace_id: "ws".into()
}
.covers(&workdir(".", true))
);
}
#[test]
fn wire_contract_rejects_self_claimed_identity_and_unknown_fields() {
let request = serde_json::json!({"permissions":[],"redelegation":[],"subject_id":"forged"});
assert!(serde_json::from_value::<WorkerDelegationRequest>(request).is_err());
let resource: PermissionResource =
serde_json::from_value(serde_json::json!({"kind":"feature"})).unwrap();
assert_eq!(resource, PermissionResource::Feature);
assert!(
serde_json::from_value::<PermissionResource>(
serde_json::json!({"kind":"feature","workspace_id":"ws"})
)
.is_err()
);
}
#[test]
fn authorization_response_carries_matching_scope_and_defaults_legacy_absence() {
let legacy: OperationAuthorizationResponse =
serde_json::from_value(serde_json::json!({})).unwrap();
assert_eq!(legacy.scope, None);
assert_eq!(
serde_json::to_value(legacy).unwrap(),
serde_json::json!({"scope":null})
);
let granted = OperationAuthorizationResponse {
scope: Some(workdir("src", true)),
};
let wire = serde_json::to_value(&granted).unwrap();
assert_eq!(
wire,
serde_json::json!({"scope":{"kind":"workdir","working_directory_id":"wd","path":"src","recursive":true}})
);
assert_eq!(
serde_json::from_value::<OperationAuthorizationResponse>(wire).unwrap(),
granted
);
}
}
+10
View File
@@ -134,6 +134,14 @@ pub enum DriveEntryKind {
File,
}
/// Current caller's advisory publication ceiling, never a grant or execution credential.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveCapabilities {
pub write: bool,
}
/// Metadata only: content is returned through bounded text or binary routes.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -491,6 +499,7 @@ impl api_macros::HttpRequestError for DriveApiError {
impl api_macros::openapi::OpenApiSchema for DriveApiError {}
impl api_macros::openapi::OpenApiSchema for DriveEntry {}
impl api_macros::openapi::OpenApiSchema for DriveCapabilities {}
impl api_macros::openapi::OpenApiSchema for DriveEntryQuery {}
impl api_macros::openapi::OpenApiSchema for DriveListQuery {}
impl api_macros::openapi::OpenApiSchema for DriveSearchQuery {}
@@ -518,6 +527,7 @@ pub fn drive_api_typescript() -> String {
DriveEntryRef::decl(&config),
DriveEntryKind::decl(&config),
DriveEntry::decl(&config),
DriveCapabilities::decl(&config),
DriveEntryQuery::decl(&config),
DriveListQuery::decl(&config),
DriveSearchQuery::decl(&config),
+150 -2
View File
@@ -6,6 +6,7 @@ use api_macros::api;
pub use api_macros::axum as server_support;
pub use api_macros::reqwest as client_support;
pub use api_macros::{ApiContract, BinaryBody, HttpMethod, TransportMetadata, WebSocketOperation};
pub mod authorization;
mod drive;
mod notification;
pub use notification::*;
@@ -23,6 +24,34 @@ use webauthn_rs_proto::{
};
pub use workspace_config::*;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsResponse {
pub permissions: Option<authorization::SubjectPermissions>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsMutationResponse {
pub permissions: authorization::SubjectPermissions,
/// New operations are already fenced. These are cancellation requests and
/// Backend command cleanup evidence, not proof that a Runtime turn stopped.
pub interruptions: Vec<SubjectPermissionInterruption>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionInterruption {
pub runtime_id: String,
pub worker_id: String,
pub cancellation_accepted: bool,
pub backend_command_sessions_closed: bool,
pub diagnostics: Vec<String>,
}
#[allow(dead_code)]
#[derive(JsonSchema)]
struct ServerJsonSafeU64(#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))] u64);
@@ -96,6 +125,8 @@ pub struct WorkdirOperationApiError {
pub message: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub diagnostics: Option<Vec<Diagnostic>>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub audit_failures: Vec<workdir::http::WorkdirAuditFailure>,
#[serde(skip, default = "default_repository_error_status")]
status: u16,
/// Server-local metadata retained by generated error response extensions, never the wire body.
@@ -103,6 +134,8 @@ pub struct WorkdirOperationApiError {
pub workdir_log: Option<WorkdirOperationLogContext>,
#[serde(skip)]
pub denial_reason: Option<workdir::WorkdirDenialReason>,
#[serde(skip)]
pub failure_reason: Option<workdir::WorkdirFailureReason>,
}
/// Internal operational correlation. Producers supply fixed-length ID fingerprints and
@@ -125,9 +158,11 @@ impl WorkdirOperationApiError {
code: None,
message: error.message,
diagnostics: Some(error.diagnostics),
audit_failures: Vec::new(),
status: error.status,
workdir_log: None,
denial_reason: None,
failure_reason: None,
}
}
@@ -138,9 +173,11 @@ impl WorkdirOperationApiError {
code: Some(error.code),
message: error.message,
diagnostics: None,
audit_failures: Vec::new(),
status,
workdir_log: None,
denial_reason: error.denial_reason,
failure_reason: error.failure_reason,
}
}
}
@@ -168,6 +205,11 @@ macro_rules! impl_openapi_schema {
}
impl_openapi_schema!(
SubjectPermissionsResponse,
SubjectPermissionsMutationResponse,
SubjectPermissionInterruption,
authorization::SubjectPermissions,
authorization::SubjectPermissionsUpdate,
AuthBootstrapUserRequest,
AuthPublicConfig,
HostingSettings,
@@ -1298,6 +1340,51 @@ pub trait ServerApi {
#[body] request: MemoryConsolidateStagingRequest,
) -> Result<MemoryConsolidationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/authorization/check",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
openapi = false,
)]
async fn operation_authorize(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: authorization::OperationAuthorizationRequest,
) -> Result<authorization::OperationAuthorizationResponse, RepositoryApiError>;
#[get(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_get(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
) -> Result<SubjectPermissionsResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_set(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
#[body] request: authorization::SubjectPermissionsUpdate,
) -> Result<SubjectPermissionsMutationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects",
status = 201,
@@ -2762,6 +2849,12 @@ pub trait ServerApi {
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveEntry, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/capabilities", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_capabilities(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveCapabilities, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/metadata", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_metadata(
&self,
@@ -4025,6 +4118,17 @@ impl RequestActor {
#[serde(deny_unknown_fields)]
pub struct WhoamiResponse {
pub actor: Option<RequestActor>,
pub session: BrowserSessionState,
}
/// Browser navigation evidence, not an authorization grant or a generic 401 classification.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum BrowserSessionState {
Anonymous {},
Authenticated { expires_at: Option<String> },
Expired {},
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -7410,6 +7514,9 @@ pub struct CurrentWorkerWorkdirAttachmentResponse {
pub alias: String,
pub working_directory_id: String,
pub capabilities: workdir::WorkdirSessionCapabilities,
/// Present for attachment responses, absent after detach.
#[serde(skip_serializing_if = "Option::is_none")]
pub environment: Option<workdir::WorkdirSessionEnvironment>,
pub attached: bool,
}
@@ -7422,7 +7529,9 @@ pub struct CurrentWorkerWorkdirOperationRequest {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(transparent)]
pub struct CurrentWorkerWorkdirOperationResponse(pub workdir::http::WorkdirSessionOperationResult);
pub struct CurrentWorkerWorkdirOperationResponse(
pub workdir::http::WorkdirSessionOperationResponse,
);
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -10428,6 +10537,10 @@ pub struct CreateWorkspaceWorkerRequest {
/// Backend idempotency key used only for authenticated Worker-owned spawn/control.
#[serde(default)]
pub control_operation_id: Option<String>,
/// Requested least-authority child permissions. Host derives the origin; this
/// input cannot assert Subject identity or create a body/singleton attachment.
#[serde(default)]
pub execution_grant: Option<authorization::WorkerDelegationRequest>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -11533,6 +11646,7 @@ pub fn auth_api_typescript() -> String {
ActorAuthMethod::decl(&config),
AuthenticatedUser::decl(&config),
RequestActor::decl(&config),
BrowserSessionState::decl(&config),
WhoamiResponse::decl(&config),
AuthBootstrapUserRequest::decl(&config),
AuthUserResponse::decl(&config),
@@ -12929,6 +13043,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
})
.unwrap();
assert_eq!(
@@ -12943,6 +13058,7 @@ mod tests {
"workdir_attachments": [],
"feature_connections": {},
"control_operation_id": null,
"execution_grant": null,
})
);
}
@@ -13173,6 +13289,7 @@ mod tests {
alias: "checkout".to_string(),
working_directory_id: "workdir".to_string(),
capabilities: workdir::WorkdirSessionCapabilities::EMPTY,
environment: None,
attached: false,
};
let value = serde_json::to_value(&response).unwrap();
@@ -13961,7 +14078,8 @@ mod tests {
"handle": "hare",
"display_name": "Hare",
"auth_method": "browser_session"
}
},
"session": { "kind": "authenticated", "expires_at": "2026-10-20T00:00:00Z" }
});
let decoded = serde_json::from_value::<WhoamiResponse>(whoami.clone())
.expect("server whoami fixture should match shared DTO");
@@ -14143,11 +14261,36 @@ mod tests {
let unexpected_field = serde_json::json!({
"actor": null,
"session": { "kind": "anonymous" },
"access_token": "must-not-be-accepted"
});
assert!(serde_json::from_value::<WhoamiResponse>(unexpected_field).is_err());
}
#[test]
fn whoami_session_json_distinguishes_anonymous_expired_and_token_auth() {
for session in [
serde_json::json!({"kind":"anonymous"}),
serde_json::json!({"kind":"expired"}),
serde_json::json!({"kind":"authenticated", "expires_at":null}),
] {
let value = serde_json::json!({"actor":null, "session":session});
let response: WhoamiResponse = serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(response).unwrap(), value);
}
for session in [
serde_json::json!({"kind":"unknown"}),
serde_json::json!({"kind":"expired", "expires_at":null}),
] {
assert!(
serde_json::from_value::<WhoamiResponse>(
serde_json::json!({"actor":null, "session":session})
)
.is_err()
);
}
}
#[test]
fn device_login_debug_output_redacts_secret_material() {
let start = DeviceLoginStartResponse {
@@ -14532,6 +14675,7 @@ mod openapi_artifact_tests {
// identity, or a one-use Reviewer capability. They remain generated Rust client/Axum
// operations, but must not appear as unauthenticated operations in the public OpenAPI.
const SIGNED_INTERNAL: &[&str] = &[
"operation_authorize",
"current_worker_workspace_config_get",
"current_worker_workspace_config_attach",
"current_worker_workspace_config_observe",
@@ -14917,10 +15061,12 @@ mod workdir_error_diagnostic_tests {
#[test]
fn workdir_internal_diagnostics_never_enter_public_error_body_or_schema() {
let mut error = WorkdirOperationApiError::provider(workdir::http::WorkdirTransportError {
failure_reason: None,
code: workdir::http::WorkdirTransportErrorCode::Denied,
message: "Workdir operation was denied".to_string(),
denial_reason: Some(workdir::WorkdirDenialReason::OsPermissionDenied),
});
error.failure_reason = Some(workdir::WorkdirFailureReason::HttpTimeout);
error.workdir_log = Some(WorkdirOperationLogContext {
operation: "write",
workspace_id_hash: "internal-workspace-fingerprint".to_string(),
@@ -14934,10 +15080,12 @@ mod workdir_error_diagnostic_tests {
assert_eq!(api_macros::HttpError::status_code(&error), 403);
let decoded: WorkdirOperationApiError = serde_json::from_value(wire).unwrap();
assert!(decoded.denial_reason.is_none());
assert!(decoded.failure_reason.is_none());
assert!(decoded.workdir_log.is_none());
let schema = serde_json::to_value(schemars::schema_for!(WorkdirOperationApiError)).unwrap();
assert!(schema["properties"].get("workdir_log").is_none());
assert!(schema["properties"].get("denial_reason").is_none());
assert!(schema["properties"].get("failure_reason").is_none());
}
}
+2
View File
@@ -11,5 +11,7 @@ checks or Runtime catalog lifecycle management.
- [`worker_session_store`]: storage boundary for a Session bound to a Worker
- [`worker_metadata`]: current Worker state and active Segment switching
- [`system_item`]: records of system input presented to the model
- [`capture`]: Worker-independent immutable public values and bounded body transport;
authorization, source I/O, and delivery lifecycle stay with the Host
- [`public_snapshot`] / [`public_index`]: public projections of stored history
- [`event_trace`]: diagnostic records independent of history replay
+967
View File
@@ -0,0 +1,967 @@
//! Exploration domain for immutable Session captures, with no Workspace/Memory mutation capability.
//!
//! A capture is an ordered projection of committed entries; exploration excludes reasoning and
//! system prompts, including Host compaction summaries and other derived context. Full inherited
//! summary input belongs to compaction, not public observation or Memory evidence. Overview,
//! search, bounded reads, and evidence conversion share `SessionEntryRef`.
//! `SessionEntryRef` is a contextual UI reference. `ScopedSessionEntryRef` additionally binds
//! the Session and exact immutable capture, and is required for portable checked operations.
//! Appends preserve existing entry identities. Sparse real user/assistant anchors and intervening
//! entry counts index long histories. Search with an empty filter also provides bounded range lists.
//!
//! Host-fixed snapshot exploration, Memory candidate evidence validation, and current Worker
//! observation share only this domain; they do not enable or modify each other's Features.
//! The provider owns authorization to recapture a current Worker. A capture itself proves neither
//! completion nor approval and grants no business-mutation authority.
//!
//! Recording Hosts use the common committed replay through the guarded APIs below.
//! [`recording_catalog_worker_guarded`] and [`recording_catalog_fs_guarded`] expose
//! descriptors only to an admission callback under the caller's existing publication/
//! erasure guard and persisted active selector, without allocating a full body DTO.
//! [`capture_recording_slice_worker_guarded`] and [`capture_recording_slice_fs_guarded`]
//! separately construct bounded body-bearing inputs. Admission descriptors are not
//! sufficient Job inputs; saved recording DTOs are read/sliced without live storage.
use std::sync::Arc;
use crate::{
LoggedSessionHistoryMetadata as SessionHistoryMetadata,
LoggedSessionHistoryOrigin as HistoryOrigin,
};
mod body;
mod recording;
use agen::{HistoryEntry, Item, Role};
pub use body::*;
use protocol::{
SessionContentPart, SessionEntryProvenance, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntryData,
};
pub use recording::*;
use serde::{Deserialize, Serialize};
const DEFAULT_SEARCH_LIMIT: usize = 20;
const MAX_SEARCH_LIMIT: usize = 50;
const DEFAULT_READ_MAX_ITEMS: usize = 40;
const MAX_READ_MAX_ITEMS: usize = 80;
const DEFAULT_READ_MAX_BYTES: usize = 32 * 1024;
const OVERVIEW_ANCHOR_STRIDE: usize = 8;
/// Append-stable contextual UI reference derived from a logical entry identity.
/// Portable consumers must bind it with `SessionCapture::scoped_ref` before use.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(transparent)]
pub struct SessionEntryRef(String);
impl SessionEntryRef {
pub fn from_history_entry_id(entry_id: &crate::LoggedSessionHistoryEntryId) -> Self {
Self(format!("E{}", entry_id.0))
}
pub fn parse(value: &str) -> Option<Self> {
let suffix = value.strip_prefix('E')?;
if suffix.is_empty()
// Legacy Host boundary identities contain both Session and Segment UUIDs.
|| suffix.len() > 256
|| !suffix
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
{
return None;
}
Some(Self(value.to_string()))
}
pub fn as_str(&self) -> &str {
&self.0
}
}
impl std::fmt::Display for SessionEntryRef {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str(&self.0)
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ReferenceKind {
User,
Assistant,
System,
Tool,
}
impl ReferenceKind {
pub fn as_str(self) -> &'static str {
match self {
Self::User => "user",
Self::Assistant => "assistant",
Self::System => "system",
Self::Tool => "tool",
}
}
pub fn parse(value: &str) -> Option<Self> {
match value {
"user" => Some(Self::User),
"assistant" | "agent" => Some(Self::Assistant),
"system" => Some(Self::System),
"tool" => Some(Self::Tool),
_ => None,
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ToolPart {
Input,
Output,
Both,
}
impl ToolPart {
pub fn parse(value: &str) -> Option<Self> {
match value {
"input" => Some(Self::Input),
"output" => Some(Self::Output),
"both" => Some(Self::Both),
_ => None,
}
}
pub fn as_str(self) -> &'static str {
match self {
Self::Input => "input",
Self::Output => "output",
Self::Both => "both",
}
}
pub fn matches(self, actual: ToolPart) -> bool {
matches!(self, ToolPart::Both) || self == actual
}
}
#[derive(Debug, Clone)]
pub struct OverviewItem {
pub id: SessionEntryRef,
pub origin: SessionEntryProvenance,
pub entry_range: [u64; 2],
pub kind: ReferenceKind,
pub label: String,
pub text: String,
pub intervening_entries: usize,
}
#[derive(Debug, Clone)]
pub struct ReferenceEntry {
pub attachment_count: usize,
use_policy: Option<protocol::SessionEntryUsePolicy>,
pub id: SessionEntryRef,
pub origin: SessionEntryProvenance,
pub entry_range: [u64; 2],
pub kind: ReferenceKind,
pub tool_part: Option<ToolPart>,
pub tool_name: Option<String>,
pub label: String,
pub summary: String,
search_text: String,
}
#[derive(Debug, Clone, Default)]
pub struct SearchOptions {
pub query: String,
pub kind: Option<ReferenceKind>,
pub tool_part: Option<ToolPart>,
pub tool_name: Option<String>,
pub limit: Option<usize>,
pub min_entry_index: Option<u64>,
pub from: Option<SessionEntryRef>,
pub through: Option<SessionEntryRef>,
pub offset: usize,
}
#[derive(Debug, Clone)]
pub struct SearchHit {
pub id: SessionEntryRef,
pub origin: SessionEntryProvenance,
pub kind: ReferenceKind,
pub tool_part: Option<ToolPart>,
pub tool_name: Option<String>,
pub entry_range: [u64; 2],
pub label: String,
pub summary: String,
}
#[derive(Debug, Clone, Copy)]
pub enum ReadSelector<'a> {
Id(&'a str),
EntryRange([u64; 2]),
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ReadDetail {
Compact,
Full,
}
#[derive(Debug, Clone)]
pub struct ReadOptions {
pub include_tools: bool,
pub tool_part: ToolPart,
pub detail: ReadDetail,
pub max_items: usize,
pub max_bytes: usize,
}
impl Default for ReadOptions {
fn default() -> Self {
Self {
include_tools: true,
tool_part: ToolPart::Both,
detail: ReadDetail::Compact,
max_items: DEFAULT_READ_MAX_ITEMS,
max_bytes: DEFAULT_READ_MAX_BYTES,
}
}
}
#[derive(Debug, Clone)]
pub struct ReadEntry {
pub id: SessionEntryRef,
pub origin: SessionEntryProvenance,
pub kind: ReferenceKind,
pub tool_part: Option<ToolPart>,
pub tool_name: Option<String>,
pub entry_range: [u64; 2],
pub label: String,
pub text: String,
}
#[derive(Debug, Clone)]
pub struct ReadResult {
pub entries: Vec<ReadEntry>,
pub truncated: bool,
}
#[derive(Debug, Clone)]
pub struct SessionEntryEvidence {
pub segment_id: String,
pub entry_ref: SessionEntryRef,
pub origin: SessionEntryProvenance,
/// Inclusive original-source positions in `segment_id`, not view/UI order.
pub entry_range: [u64; 2],
pub kind: ReferenceKind,
pub tool_part: Option<ToolPart>,
pub label: String,
pub summary: String,
pub excerpt: String,
}
#[derive(Debug, Clone)]
struct CapturedHistoryEntry {
use_policy: Option<protocol::SessionEntryUsePolicy>,
item: Item,
entry_id: crate::LoggedSessionHistoryEntryId,
origin: SessionEntryProvenance,
dto: CaptureEntryDto,
}
#[derive(Debug, Clone)]
pub struct SessionCapture {
session_id: String,
segment_id: String,
capture_id: String,
view_entry_count: u64,
entries: Arc<Vec<CapturedHistoryEntry>>,
overview: Vec<OverviewItem>,
index: Vec<ReferenceEntry>,
}
impl SessionCapture {
pub fn from_session_snapshot(
session_id: impl Into<String>,
segment_id: impl Into<String>,
snapshot: SessionSnapshot,
) -> Self {
let segment_id = segment_id.into();
let view_entry_count = snapshot.entries.len() as u64;
let entries = snapshot
.entries
.into_iter()
.enumerate()
.filter_map(|(index, entry)| {
let policy = CaptureUsePolicy {
recording_material: false,
memory_evidence: entry.use_policy.is_none_or(|p| p.memory_evidence),
observation_readable: entry.use_policy.is_none_or(|p| p.observation_readable),
};
let explicit = entry.use_policy.is_some();
let item = match entry.data {
SessionSnapshotEntryData::UserInput { segments } => {
Item::user_message(protocol::Segment::flatten_to_text(&segments))
}
SessionSnapshotEntryData::Message { role, content } => Item::Message {
id: None,
status: None,
role: match role {
SessionMessageRole::User => Role::User,
SessionMessageRole::Assistant => Role::Assistant,
},
content: content
.into_iter()
.map(|part| match part {
SessionContentPart::Text { text } => {
agen::ContentPart::Text { text }
}
SessionContentPart::Refusal { refusal } => {
agen::ContentPart::Refusal { refusal }
}
})
.collect(),
},
SessionSnapshotEntryData::ToolCall {
call_id,
name,
arguments,
} => Item::tool_call(call_id, name, arguments),
SessionSnapshotEntryData::ToolResult {
call_id,
summary,
content,
is_error,
..
} => Item::tool_result_item(call_id, summary, content, is_error),
SessionSnapshotEntryData::SystemItem { content, .. } if explicit => {
Item::system_message(content)
}
_ => return None,
};
let body = CaptureBody::project(&item, &entry.provenance, explicit)?;
if !policy.any() {
return None;
}
Some(CaptureEntryDto {
entry_id: entry.entry_id,
order: index as u64,
source: CaptureSource {
segment_id: segment_id.clone(),
entry_index: index as u64,
provenance: entry.provenance,
history_origin: None,
derived_from: entry.derived_from,
},
policy,
body,
})
})
.collect();
Self::from_projected(session_id.into(), segment_id, view_entry_count, entries)
}
pub fn new(
session_id: impl Into<String>,
segment_id: impl Into<String>,
items: Vec<Item>,
) -> Self {
let segment_id = segment_id.into();
let view_entry_count = items.len() as u64;
let entries = items
.into_iter()
.enumerate()
.filter_map(|(index, item)| {
let origin = SessionEntryProvenance::LegacyUnknown;
let body = CaptureBody::project(&item, &origin, false)?;
Some(CaptureEntryDto {
entry_id: format!("{index:08}"),
order: index as u64,
source: CaptureSource {
segment_id: segment_id.clone(),
entry_index: index as u64,
provenance: origin,
history_origin: None,
derived_from: Vec::new(),
},
policy: CaptureUsePolicy::PUBLIC,
body,
})
})
.collect();
Self::from_projected(session_id.into(), segment_id, view_entry_count, entries)
}
/// Project a complete Host-fixed history whose source positions start at zero.
/// Select a suffix with `select_order_range` on the resulting value instead
/// of passing a rebased history vector: annotations do not carry source positions.
pub fn from_complete_history(
session_id: impl Into<String>,
segment_id: impl Into<String>,
entries: Vec<HistoryEntry<SessionHistoryMetadata>>,
) -> Self {
let segment_id = segment_id.into();
let view_entry_count = entries.len() as u64;
let entries = entries
.into_iter()
.enumerate()
.filter_map(|(index, entry)| {
let origin = public_provenance(&entry.annotation.origin);
let mut item = entry.item;
let mut explicit = false;
let policy = if let Some(input) = entry.annotation.input_metadata {
explicit = true;
if matches!(
item,
Item::Message {
role: Role::System,
..
}
) {
// Only the explicit public projection crosses this boundary.
let public = input.public_item()?;
item = Item::system_message(public.body.clone());
} else if input.feature_payload().is_some() || input.public_item().is_some() {
return None;
}
CaptureUsePolicy::from(input.usage())
} else {
if matches!(entry.annotation.origin, HistoryOrigin::Provided { .. }) {
return None;
}
CaptureUsePolicy::PUBLIC
};
if !policy.any() {
return None;
}
let body = CaptureBody::project(&item, &origin, explicit)?;
Some(CaptureEntryDto {
entry_id: entry.annotation.entry_id.0,
order: index as u64,
source: CaptureSource {
segment_id: segment_id.clone(),
entry_index: index as u64,
provenance: origin,
history_origin: Some(entry.annotation.origin),
derived_from: entry
.annotation
.derivation
.map(|d| d.sources.into_iter().map(|id| id.0).collect())
.unwrap_or_default(),
},
policy,
body,
})
})
.collect();
Self::from_projected(session_id.into(), segment_id, view_entry_count, entries)
}
fn from_projected(
session_id: String,
segment_id: String,
view_entry_count: u64,
dtos: Vec<CaptureEntryDto>,
) -> Self {
let dto = CaptureDto {
version: CAPTURE_VERSION,
session_id: session_id.clone(),
segment_id: segment_id.clone(),
view_entry_count,
entries: dtos,
};
let capture_id = dto.fingerprint();
let entries: Vec<_> = dto
.entries
.into_iter()
.map(|dto| CapturedHistoryEntry {
use_policy: Some(protocol::SessionEntryUsePolicy {
observation_readable: dto.policy.observation_readable,
memory_evidence: dto.policy.memory_evidence,
}),
item: dto.body.to_item(),
entry_id: crate::LoggedSessionHistoryEntryId(dto.entry_id.clone()),
origin: dto.source.provenance.clone(),
dto,
})
.collect();
let mut overview = Vec::new();
let mut index = Vec::new();
for entry in &entries {
let item = &entry.item;
let entry_range = [entry.dto.order; 2];
let attachment_count = match entry.dto.body {
CaptureBody::ToolResult {
attachment_count, ..
} => attachment_count,
_ => 0,
};
match item {
Item::Message { role, content, .. } => {
let Some(kind) = message_reference_kind(&entry.origin, role, entry.use_policy)
else {
continue;
};
let text = content
.iter()
.map(|p| p.as_text())
.collect::<Vec<_>>()
.join("");
let label = format!("{} message", kind.as_str());
let summary = truncate_chars(&text, 240);
let id = SessionEntryRef::from_history_entry_id(&entry.entry_id);
index.push(ReferenceEntry {
use_policy: entry.use_policy,
attachment_count,
id: id.clone(),
origin: entry.origin.clone(),
entry_range,
kind,
tool_part: None,
tool_name: None,
label: label.clone(),
summary: summary.clone(),
search_text: text.clone(),
});
if entry.use_policy.is_none_or(|p| p.observation_readable)
&& matches!(kind, ReferenceKind::User | ReferenceKind::Assistant)
{
overview.push(OverviewItem {
id: id.clone(),
origin: entry.origin.clone(),
entry_range,
kind,
label,
text,
intervening_entries: 0,
});
}
}
Item::ToolCall {
name, arguments, ..
} => {
let text = format!("{name}\n{arguments}");
index.push(ReferenceEntry {
use_policy: entry.use_policy,
attachment_count,
id: SessionEntryRef::from_history_entry_id(&entry.entry_id),
origin: entry.origin.clone(),
entry_range,
kind: ReferenceKind::Tool,
tool_part: Some(ToolPart::Input),
tool_name: Some(name.clone()),
label: format!("tool input: {name}"),
summary: format!("Tool call {name}"),
search_text: text,
});
}
Item::ToolResult {
summary,
content,
attachments: _,
..
} => {
let attachment_marker = if attachment_count == 0 {
String::new()
} else {
format!("\n[{} image attachment(s)]", attachment_count)
};
let text = format!(
"{summary}\n{}{attachment_marker}",
content.as_deref().unwrap_or_default(),
);
index.push(ReferenceEntry {
use_policy: entry.use_policy,
attachment_count,
id: SessionEntryRef::from_history_entry_id(&entry.entry_id),
origin: entry.origin.clone(),
entry_range,
kind: ReferenceKind::Tool,
tool_part: Some(ToolPart::Output),
tool_name: None,
label: "tool output".to_string(),
summary: truncate_chars(summary, 240),
search_text: text,
});
}
Item::AssistantResponseBoundary { .. } | Item::Reasoning { .. } => {}
}
}
if overview.len() > 2 {
let last = overview.len() - 1;
overview = overview
.into_iter()
.enumerate()
.filter_map(|(index, entry)| {
(index == 0 || index == last || index % OVERVIEW_ANCHOR_STRIDE == 0)
.then_some(entry)
})
.collect();
}
for overview_index in 0..overview.len().saturating_sub(1) {
let current_entry = overview[overview_index].entry_range[0];
let next_entry = overview[overview_index + 1].entry_range[0];
overview[overview_index].intervening_entries = index
.iter()
.filter(|entry| {
let entry_index = entry.entry_range[0];
entry_index > current_entry && entry_index < next_entry
})
.count();
}
Self {
session_id,
segment_id,
capture_id,
view_entry_count,
entries: Arc::new(entries),
overview,
index,
}
}
/// Public source entries are immutable; callers retain their own business decisions.
pub fn entries(&self) -> impl ExactSizeIterator<Item = &CaptureEntryDto> {
self.entries.iter().map(|entry| &entry.dto)
}
pub fn overview(&self) -> &[OverviewItem] {
&self.overview
}
/// Capture-local UI position, not the original source position used by evidence.
pub fn view_order_for_ref(&self, reference: &SessionEntryRef) -> Option<u64> {
self.index
.iter()
.find(|entry| entry.id == *reference)
.map(|entry| entry.entry_range[0])
}
pub fn search(&self, options: &SearchOptions) -> Vec<SearchHit> {
if self.validate_search(options).is_err() {
return Vec::new();
}
let query = options.query.trim().to_lowercase();
let limit = options
.limit
.unwrap_or(DEFAULT_SEARCH_LIMIT)
.clamp(1, MAX_SEARCH_LIMIT);
let tool_name = options.tool_name.as_deref();
let min_entry_index = options
.from
.as_ref()
.and_then(|reference| self.view_order_for_ref(reference))
.unwrap_or_else(|| options.min_entry_index.unwrap_or(0));
let max_entry_index = options
.through
.as_ref()
.and_then(|reference| self.view_order_for_ref(reference))
.unwrap_or(u64::MAX);
let mut skipped = 0usize;
let mut hits = Vec::new();
for entry in &self.index {
if entry.use_policy.is_some_and(|p| !p.observation_readable) {
continue;
}
if entry.entry_range[0] < min_entry_index || entry.entry_range[0] > max_entry_index {
continue;
}
if let Some(kind) = options.kind {
if entry.kind != kind {
continue;
}
}
if let Some(part) = options.tool_part {
if entry.kind != ReferenceKind::Tool {
continue;
}
let Some(actual) = entry.tool_part else {
continue;
};
if !part.matches(actual) {
continue;
}
}
if let Some(name) = tool_name {
if entry.tool_name.as_deref() != Some(name) {
continue;
}
}
if !query.is_empty() && !entry.search_text.to_lowercase().contains(&query) {
continue;
}
if skipped < options.offset {
skipped += 1;
continue;
}
hits.push(SearchHit {
id: entry.id.clone(),
origin: entry.origin.clone(),
kind: entry.kind,
tool_part: entry.tool_part,
tool_name: entry.tool_name.clone(),
entry_range: entry.entry_range,
label: entry.label.clone(),
summary: entry.summary.clone(),
});
if hits.len() >= limit {
break;
}
}
hits
}
pub fn read(&self, selector: ReadSelector<'_>, options: ReadOptions) -> ReadResult {
let max_items = options.max_items.clamp(1, MAX_READ_MAX_ITEMS);
let max_bytes = options.max_bytes.max(1);
let mut entries = Vec::new();
let mut bytes = 0usize;
let mut truncated = false;
let selected: Vec<&ReferenceEntry> = match selector {
ReadSelector::Id(id) => self
.index
.iter()
.filter(|entry| entry.id.as_str() == id)
.collect(),
ReadSelector::EntryRange([start, end]) => self
.index
.iter()
.filter(|entry| entry.entry_range[0] >= start && entry.entry_range[0] <= end)
.collect(),
};
for entry in selected {
if entry.use_policy.is_some_and(|p| !p.observation_readable) {
continue;
}
if entries.len() >= max_items {
truncated = true;
break;
}
if entry.kind == ReferenceKind::Tool {
if !options.include_tools {
continue;
}
if let Some(actual) = entry.tool_part {
if !options.tool_part.matches(actual) {
continue;
}
}
}
let Some(item) = self
.entries
.iter()
.find(|captured| captured.dto.order == entry.entry_range[0])
.map(|entry| &entry.item)
else {
continue;
};
let remaining = max_bytes.saturating_sub(bytes);
if remaining == 0 {
truncated = true;
break;
}
let rendered = render_item(item, entry, options.detail, usize::MAX);
truncated |= rendered.len() > remaining;
let text = truncate_bytes(&rendered, remaining);
bytes = bytes.saturating_add(text.len());
entries.push(ReadEntry {
id: entry.id.clone(),
origin: entry.origin.clone(),
kind: entry.kind,
tool_part: entry.tool_part,
tool_name: entry.tool_name.clone(),
entry_range: entry.entry_range,
label: entry.label.clone(),
text,
});
if truncated {
break;
}
}
ReadResult { entries, truncated }
}
/// Resolves the committed tool-call input for a provider call id. This is
/// used by explicit-memory receipts only after the containing run is durable,
/// so a model-authored claim can cite its own committed tool invocation
/// without inventing a SessionEntryRef.
pub fn evidence_for_tool_call(&self, call_id: &str) -> Option<SessionEntryEvidence> {
let captured = self.entries.iter().find(|entry| {
matches!(
&entry.item,
Item::ToolCall {
call_id: stored_call_id,
..
} if stored_call_id == call_id
)
})?;
let reference = self.index.iter().find(|entry| {
entry.entry_range[0] == captured.dto.order
&& entry.kind == ReferenceKind::Tool
&& entry.tool_part == Some(ToolPart::Input)
})?;
self.evidence_for(reference.id.as_str())
}
pub fn evidence_for(&self, id: &str) -> Option<SessionEntryEvidence> {
let entry = self.index.iter().find(|entry| entry.id.as_str() == id)?;
if entry.use_policy.is_some_and(|p| !p.memory_evidence) {
return None;
}
let captured = self
.entries
.iter()
.find(|captured| captured.dto.order == entry.entry_range[0])?;
let excerpt = render_item(&captured.item, entry, ReadDetail::Compact, 2 * 1024);
Some(SessionEntryEvidence {
segment_id: captured.dto.source.segment_id.clone(),
entry_ref: entry.id.clone(),
origin: entry.origin.clone(),
entry_range: [captured.dto.source.entry_index; 2],
kind: entry.kind,
tool_part: entry.tool_part,
label: entry.label.clone(),
summary: entry.summary.clone(),
excerpt,
})
}
}
pub(crate) fn public_provenance(origin: &HistoryOrigin) -> SessionEntryProvenance {
match origin {
HistoryOrigin::HumanInput { .. } | HistoryOrigin::LocalHumanInput => {
SessionEntryProvenance::HumanInput
}
HistoryOrigin::WorkerInput { .. } => SessionEntryProvenance::WorkerInput,
HistoryOrigin::FlowInstruction { .. } => SessionEntryProvenance::FlowInstruction,
HistoryOrigin::BackendInstruction { .. } | HistoryOrigin::Provided { .. } => {
SessionEntryProvenance::BackendInstruction
}
HistoryOrigin::ModelOutput { .. } => SessionEntryProvenance::ModelOutput,
HistoryOrigin::ToolOutput { .. } => SessionEntryProvenance::ToolOutput,
HistoryOrigin::CompactionSummary
| HistoryOrigin::DerivedSummary
| HistoryOrigin::JournalContext => SessionEntryProvenance::DerivedSummary,
HistoryOrigin::LegacyUnknown => SessionEntryProvenance::LegacyUnknown,
}
}
fn message_reference_kind(
origin: &SessionEntryProvenance,
provider_role: &Role,
use_policy: Option<protocol::SessionEntryUsePolicy>,
) -> Option<ReferenceKind> {
if use_policy.is_some() {
return match provider_role {
Role::System => Some(ReferenceKind::System),
Role::User => Some(ReferenceKind::User),
Role::Assistant => Some(ReferenceKind::Assistant),
};
}
match origin {
SessionEntryProvenance::HumanInput | SessionEntryProvenance::WorkerInput => {
Some(ReferenceKind::User)
}
SessionEntryProvenance::ModelOutput => Some(ReferenceKind::Assistant),
SessionEntryProvenance::ToolOutput => Some(ReferenceKind::Tool),
SessionEntryProvenance::LegacyUnknown => match provider_role {
Role::User => Some(ReferenceKind::User),
Role::Assistant => Some(ReferenceKind::Assistant),
Role::System => None,
},
// Flow/backend/system content remains out of the observation surface
// even when represented with a provider user/system role.
SessionEntryProvenance::FlowInstruction
| SessionEntryProvenance::BackendInstruction
| SessionEntryProvenance::DerivedSummary => None,
}
}
fn render_item(
item: &Item,
entry: &ReferenceEntry,
detail: ReadDetail,
max_bytes: usize,
) -> String {
let text = match item {
Item::Message { role, content, .. } => {
let text = content
.iter()
.map(|p| p.as_text())
.collect::<Vec<_>>()
.join("");
format!("[{} {:?}] {text}", entry.id, role)
}
Item::ToolCall {
name, arguments, ..
} => match detail {
ReadDetail::Compact => format!("[{} ToolInput {name}] (arguments omitted)", entry.id),
ReadDetail::Full => format!("[{} ToolInput {name}]\narguments: {arguments}", entry.id),
},
Item::ToolResult {
summary,
content,
attachments: _,
is_error,
..
} => {
let attachment_line = if entry.attachment_count == 0 {
String::new()
} else {
format!("\nattachments: {} image(s)", entry.attachment_count)
};
match detail {
ReadDetail::Compact => format!(
"[{} ToolOutput{}]\nsummary: {summary}\ncontent: (omitted){attachment_line}",
entry.id,
if *is_error { " error" } else { "" },
),
ReadDetail::Full => format!(
"[{} ToolOutput{}]\nsummary: {summary}\ncontent: {}{attachment_line}",
entry.id,
if *is_error { " error" } else { "" },
content.as_deref().unwrap_or_default(),
),
}
}
Item::AssistantResponseBoundary { .. } => {
format!("[{} Assistant response boundary omitted]", entry.id)
}
Item::Reasoning { .. } => format!("[{} Reasoning omitted]", entry.id),
};
truncate_bytes(&text, max_bytes)
}
fn truncate_chars(text: &str, max_chars: usize) -> String {
if max_chars == 0 {
return "… [truncated]".to_string();
}
if text.chars().count() <= max_chars {
return text.to_string();
}
let mut out = text.chars().take(max_chars).collect::<String>();
out.push_str("… [truncated]");
out
}
fn truncate_bytes(text: &str, max_bytes: usize) -> String {
if text.len() <= max_bytes {
return text.to_owned();
}
let marker = "… [truncated]";
let mut end = max_bytes.saturating_sub(marker.len());
while !text.is_char_boundary(end) {
end -= 1;
}
let mut out = text[..end].to_owned();
if max_bytes >= marker.len() {
out.push_str(marker);
}
out
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,425 @@
//! Host capture and pure canonical slicing. No Job state or live-reader fallback lives here.
use super::*;
use crate::committed_replay::CommittedPublicReplay;
use crate::public_material::{
PublicMaterialDescriptor, PublicMaterialError, PublicMaterialLimits, PublicMaterialUse,
};
use crate::retention::SessionRetentionStore;
use crate::{SegmentId, SessionId};
use std::path::Path;
/// Continuation in canonical Journal order, not an observation/Memory view cursor.
/// A Host must reconcile the descriptor at this position with its saved checkpoint.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RecordingPosition {
pub canonical_order: u64,
pub byte_offset: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RecordingCaptureSlice {
pub capture: CaptureDto,
pub next: Option<RecordingPosition>,
}
/// Inspect recording admission descriptors without constructing a body-bearing DTO.
///
/// The Host must authorize the source and hold the **same** Session publication/erasure
/// guard from before reading its persisted active pointer until this function returns
/// (including the callback). Pass that pointer's Segment, not a filename/mtime-derived
/// selector. This function verifies the guard and stored Session identities but does
/// not authorize the caller or establish that the supplied guard fences this root.
/// No lock is recursively acquired, no active pointer is reread, and no replay handle
/// or body-read capability is exposed to `inspect`.
///
/// The common committed recording replay fixes per-file ends before decoding. It
/// retains only descriptors/locations, decoding one record at a time and streaming
/// canonical bodies into their digests. `replay_limits` bound record allocation,
/// aggregate storage extent, entries and Segments; there is no full-capture text/JSON
/// allocation or `CaptureLimits` ceiling. The callback receives only descriptors in
/// canonical order and may return a Host-specific mapping (including a `Result`). It
/// runs once on success and not at all on replay/identity/resource errors.
///
/// This catalog is for Host admission, **not** a descriptor-only Job input. Construct
/// the Job's bounded body input with the guarded slice API at its own authorized
/// boundary; after saving it, execute only that immutable value. Neither descriptors
/// nor a completed callback authorize delayed adoption after erasure.
///
/// ```no_run
/// # use std::path::Path;
/// # use session_store::{SessionId, SegmentId};
/// # use session_store::retention::SessionRetentionGuard;
/// # use session_store::public_material::PublicMaterialError;
/// # fn admit(root: &Path, session: SessionId, active: SegmentId,
/// # guard: &SessionRetentionGuard) -> Result<Vec<u64>, PublicMaterialError> {
/// // `active` was resolved from persisted Host metadata while holding `guard`.
/// session_store::capture::recording_catalog_worker_guarded(
/// root, session, active, Default::default(), guard,
/// |descriptors| descriptors.iter().map(|d| d.total_bytes).collect(),
/// )
/// # }
/// ```
pub fn recording_catalog_worker_guarded<T>(
session_root: &Path,
expected_session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
guard: &crate::retention::SessionRetentionGuard,
inspect: impl FnOnce(&[PublicMaterialDescriptor]) -> T,
) -> Result<T, PublicMaterialError> {
if guard.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
let replay = CommittedPublicReplay::open_worker_guarded(
session_root,
active_segment,
replay_limits,
None,
PublicMaterialUse::Recording,
)?;
if replay.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
Ok(inspect(&replay.into_recording_descriptors()))
}
/// FsStore-layout counterpart of [`recording_catalog_worker_guarded`].
///
/// `sessions_root` is the directory containing per-Session directories. Invoke this
/// inside the Host's `with_recording_boundary` callback, passing **its existing guard
/// and persisted active Segment**. That same external retention control store must
/// fence publishers, pointer changes, archive moves and erasure. Do not reacquire the
/// guard or use a separately opened legacy public reader inside the boundary.
/// All descriptor-only, allocation and callback guarantees of the Worker variant
/// apply; this function does not construct an unbounded transient `CaptureDto`.
pub fn recording_catalog_fs_guarded<T>(
sessions_root: &Path,
session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
guard: &crate::retention::SessionRetentionGuard,
inspect: impl FnOnce(&[PublicMaterialDescriptor]) -> T,
) -> Result<T, PublicMaterialError> {
if guard.session_id() != session {
return Err(PublicMaterialError::Inconsistent);
}
let replay = CommittedPublicReplay::open_fs_guarded(
sessions_root,
session,
active_segment,
replay_limits,
)?;
Ok(inspect(&replay.into_recording_descriptors()))
}
/// Capture the selected original lineage, including typed Host run boundaries.
/// The caller must hold the Session publication/erasure guard while selecting the
/// active pointer and through this call. Runtime publishers use this same lock
/// for appends and Segment creation. No lock is recursively acquired here.
pub fn capture_recording_worker_guarded(
session_root: &Path,
expected_session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
capture_limits: CaptureLimits,
guard: &crate::retention::SessionRetentionGuard,
) -> Result<CaptureDto, PublicMaterialError> {
if guard.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
let replay = CommittedPublicReplay::open_worker_guarded(
session_root,
active_segment,
replay_limits,
None,
PublicMaterialUse::Recording,
)?;
if replay.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
replay.into_recording_dto(capture_limits)
}
/// Standalone convenience entry; Hosts resolving a mutable active pointer should
/// instead hold the fence while resolving it and call the guarded entry.
pub fn capture_recording_worker(
session_root: &Path,
expected_session: SessionId,
active_segment: SegmentId,
retention: &SessionRetentionStore,
replay_limits: PublicMaterialLimits,
capture_limits: CaptureLimits,
) -> Result<CaptureDto, PublicMaterialError> {
let guard = retention
.lock_live(expected_session)
.map_err(|error| match error {
crate::StoreError::SessionErased(_) => PublicMaterialError::Erased,
_ => PublicMaterialError::Unavailable,
})?;
capture_recording_worker_guarded(
session_root,
expected_session,
active_segment,
replay_limits,
capture_limits,
&guard,
)
}
/// Bounded construction for one Job input: descriptor replay is bounded separately;
/// only at most the requested body budget is materialized from the fixed files.
pub fn capture_recording_slice_worker_guarded(
session_root: &Path,
expected_session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
start: RecordingPosition,
max_entries: usize,
max_serialized_bytes: usize,
guard: &crate::retention::SessionRetentionGuard,
) -> Result<RecordingCaptureSlice, PublicMaterialError> {
if guard.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
let replay = CommittedPublicReplay::open_worker_guarded(
session_root,
active_segment,
replay_limits,
None,
PublicMaterialUse::Recording,
)?;
if replay.session_id() != expected_session {
return Err(PublicMaterialError::Inconsistent);
}
replay.recording_slice(start, max_entries, max_serialized_bytes)
}
/// The standalone FsStore layout uses the same committed replay and fixed values.
pub fn capture_recording_fs_guarded(
sessions_root: &Path,
session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
capture_limits: CaptureLimits,
guard: &crate::retention::SessionRetentionGuard,
) -> Result<CaptureDto, PublicMaterialError> {
if guard.session_id() != session {
return Err(PublicMaterialError::Inconsistent);
}
CommittedPublicReplay::open_fs_guarded(sessions_root, session, active_segment, replay_limits)?
.into_recording_dto(capture_limits)
}
/// Bounded standalone capture; the caller holds its FsStore retention fence.
pub fn capture_recording_slice_fs_guarded(
sessions_root: &Path,
session: SessionId,
active_segment: SegmentId,
replay_limits: PublicMaterialLimits,
start: RecordingPosition,
max_entries: usize,
max_serialized_bytes: usize,
guard: &crate::retention::SessionRetentionGuard,
) -> Result<RecordingCaptureSlice, PublicMaterialError> {
if guard.session_id() != session {
return Err(PublicMaterialError::Inconsistent);
}
CommittedPublicReplay::open_fs_guarded(sessions_root, session, active_segment, replay_limits)?
.recording_slice(start, max_entries, max_serialized_bytes)
}
impl CaptureDto {
/// Select the oldest canonical prefix fitting both budgets, including JSON
/// escaping, metadata and continuation. Only immutable owned bytes are read.
/// A later Job can recapture at `next`; never combine live pages into this value.
pub fn recording_slice(
&self,
start: RecordingPosition,
max_entries: usize,
max_serialized_bytes: usize,
) -> Result<RecordingCaptureSlice, CaptureError> {
if max_entries == 0 {
return Err(CaptureError::BudgetTooSmall);
}
if start.canonical_order > self.view_entry_count {
return Err(CaptureError::InvalidRange);
}
let mut result = RecordingCaptureSlice {
capture: CaptureDto {
version: self.version,
session_id: self.session_id.clone(),
segment_id: self.segment_id.clone(),
view_entry_count: self.view_entry_count,
entries: Vec::new(),
},
next: (start.canonical_order < self.view_entry_count).then_some(start),
};
if body::json_size(&result, max_serialized_bytes).is_err() {
return Err(CaptureError::BudgetTooSmall);
}
if start.canonical_order == self.view_entry_count {
if start.byte_offset != 0 {
return Err(CaptureError::InvalidRange);
}
result.capture.validate(CaptureLimits {
max_bytes: max_serialized_bytes,
max_text_bytes: max_serialized_bytes,
max_entries,
..Default::default()
})?;
return Ok(result);
}
let first = self
.entries
.iter()
.position(|entry| entry.order == start.canonical_order)
.ok_or(CaptureError::InvalidRange)?;
let mut expected_order = start.canonical_order;
for entry in self.entries.iter().skip(first).take(max_entries) {
if entry.order != expected_order {
return Err(CaptureError::InvalidRange);
}
let CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
boundary,
} = &entry.body
else {
return Err(CaptureError::Invalid("not a recording capture"));
};
if !entry.policy.recording_material {
return Err(CaptureError::PolicyDenied);
}
let offset = if entry.order == start.canonical_order {
start.byte_offset
} else {
*byte_offset
};
let local = offset
.checked_sub(*byte_offset)
.and_then(|o| usize::try_from(o).ok())
.ok_or(CaptureError::InvalidRange)?;
if local > text.len()
|| !text.is_char_boundary(local)
|| offset >= descriptor.total_bytes
{
return Err(CaptureError::InvalidRange);
}
let item = CaptureEntryDto {
entry_id: entry.entry_id.clone(),
order: entry.order,
source: entry.source.clone(),
policy: entry.policy,
body: CaptureBody::CanonicalPublic {
descriptor: descriptor.clone(),
byte_offset: offset,
text: String::new(),
boundary: *boundary,
},
};
result.capture.entries.push(item);
// At most budget bytes are considered before allocating a trial fragment.
let mut low = 0;
let mut high = text.len().saturating_sub(local).min(max_serialized_bytes);
let prior_next = result.next;
while low < high {
let mid = low + (high - low).div_ceil(2);
let mut end = local + mid;
while !text.is_char_boundary(end) {
end -= 1;
}
if let CaptureBody::CanonicalPublic { text: fragment, .. } =
&mut result.capture.entries.last_mut().unwrap().body
{
*fragment = text[local..end].to_owned();
}
let next_offset = offset + (end - local) as u64;
result.next = if next_offset < descriptor.total_bytes {
Some(RecordingPosition {
canonical_order: entry.order,
byte_offset: next_offset,
})
} else {
(entry.order + 1 < self.view_entry_count).then_some(RecordingPosition {
canonical_order: entry.order + 1,
byte_offset: 0,
})
};
if body::json_size(&result, max_serialized_bytes).is_ok() {
low = mid;
} else {
high = mid - 1;
}
}
let mut end = local + low;
while !text.is_char_boundary(end) {
end -= 1;
}
if end == local {
result.capture.entries.pop();
result.next = prior_next;
if result.capture.entries.is_empty() {
return Err(CaptureError::BudgetTooSmall);
}
break;
}
if let CaptureBody::CanonicalPublic { text: fragment, .. } =
&mut result.capture.entries.last_mut().unwrap().body
{
*fragment = text[local..end].to_owned();
}
let next_offset = offset + (end - local) as u64;
result.next = if next_offset < descriptor.total_bytes {
Some(RecordingPosition {
canonical_order: entry.order,
byte_offset: next_offset,
})
} else {
(entry.order + 1 < self.view_entry_count).then_some(RecordingPosition {
canonical_order: entry.order + 1,
byte_offset: 0,
})
};
if next_offset < descriptor.total_bytes {
break;
}
expected_order = entry.order + 1;
}
body::json_size(&result, max_serialized_bytes).map_err(|_| CaptureError::BudgetTooSmall)?;
result.capture.validate(CaptureLimits {
max_bytes: max_serialized_bytes,
max_text_bytes: max_serialized_bytes,
max_entries,
..Default::default()
})?;
Ok(result)
}
/// Resolve canonical text already in the value. No original storage is consulted.
pub fn recording_text(
&self,
order: u64,
) -> Result<(&crate::public_material::PublicMaterialDescriptor, u64, &str), CaptureError> {
let entry = self
.entries
.iter()
.find(|entry| entry.order == order)
.ok_or(CaptureError::UnknownReference)?;
if !entry.policy.recording_material {
return Err(CaptureError::PolicyDenied);
}
match &entry.body {
CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
..
} => Ok((descriptor, *byte_offset, text)),
_ => Err(CaptureError::Invalid("not canonical recording text")),
}
}
}
File diff suppressed because it is too large Load Diff
+23
View File
@@ -68,6 +68,24 @@ impl FsStore {
) -> Result<Self, StoreError> {
let root = root.into();
fs::create_dir_all(&root)?;
for entry in fs::read_dir(&root)? {
let entry = entry?;
if entry.file_type()?.is_dir()
&& let Some(session_id) = entry
.file_name()
.to_str()
.and_then(|name| name.parse().ok())
{
let _guard = match retention.lock_live(session_id) {
Ok(guard) => guard,
// Interrupted erasure may leave bytes behind. Never read or
// upgrade them, and do not block unrelated live Sessions.
Err(StoreError::SessionErased(_)) => continue,
Err(error) => return Err(error),
};
crate::worker_session_store::migrate_fs_session(&entry.path(), session_id)?;
}
}
Ok(Self {
root,
append_lock: Arc::new(Mutex::new(())),
@@ -343,6 +361,7 @@ impl Store for FsStore {
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.log_path(session_id, segment_id), &line)
@@ -430,10 +449,14 @@ impl Store for FsStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.retention().lock_live(session_id)?;
let path = self.log_path(session_id, segment_id);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
crate::worker_session_store::migrate_fs_session(parent, session_id)?;
}
let mut content = String::new();
for entry in entries {
+204
View File
@@ -67,8 +67,14 @@ pub enum LoggedSessionHistoryOrigin {
ToolOutput {
worker: LoggedWorkerSubject,
},
/// Descriptive producer identity; use policies live in separate metadata.
Provided {
producer: String,
},
/// Host-produced opening context summary; not auto-read or TaskStore material.
CompactionSummary,
/// Frozen Host Journal reference, never independent observation evidence.
JournalContext,
DerivedSummary,
LegacyUnknown,
}
@@ -79,19 +85,32 @@ pub struct LoggedHistoryDerivation {
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct LoggedSessionHistoryMetadata {
pub entry_id: LoggedSessionHistoryEntryId,
pub origin: LoggedSessionHistoryOrigin,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub derivation: Option<LoggedHistoryDerivation>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub input_metadata: Option<crate::HistoryInputMetadata>,
}
impl LoggedSessionHistoryMetadata {
pub(crate) fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
if matches!(self.origin, LoggedSessionHistoryOrigin::Provided { .. })
&& self.input_metadata.is_none()
{
return Err(crate::InputMetadataError::MissingMetadata);
}
Ok(())
}
pub fn legacy_unknown() -> Self {
Self {
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
input_metadata: None,
}
}
}
@@ -99,20 +118,146 @@ impl LoggedSessionHistoryMetadata {
/// Persisted item and metadata are one value so transforms cannot reorder or
/// truncate one without the other.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(try_from = "HistoryEntryWire")]
pub struct LoggedHistoryEntry {
pub item: LoggedItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct HistoryEntryWire {
item: LoggedItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<HistoryEntryWire> for LoggedHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: HistoryEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedHistoryEntry {
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
if let Some(input) = &self.metadata.input_metadata {
match &self.item {
LoggedItem::Message {
role: crate::LoggedRole::System,
content,
} => {
let [crate::LoggedContentPart::Text { text }] = content.as_slice() else {
return Err(crate::InputMetadataError::InconsistentRecord);
};
if text.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
// Other logical roles may carry use policy, but cannot masquerade
// as the System-only public/payload extension contract.
_ if input.feature_payload().is_some() || input.public_item().is_some() => {
return Err(crate::InputMetadataError::InconsistentRecord);
}
_ => {}
}
}
Ok(())
}
}
/// Typed system-item history record. The typed system event remains available
/// to client replay while its model-visible projection carries the same stable
/// metadata used by live history.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(try_from = "SystemEntryWire")]
pub struct LoggedSystemHistoryEntry {
pub item: crate::SystemItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct SystemEntryWire {
item: crate::SystemItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<SystemEntryWire> for LoggedSystemHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: SystemEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedSystemHistoryEntry {
pub fn from_input(
origin: LoggedSessionHistoryOrigin,
derivation: Option<LoggedHistoryDerivation>,
input: crate::SystemInputEnvelope,
) -> Self {
let (body, input_metadata) = input.into_parts();
Self {
item: crate::SystemItem::Input { body },
metadata: LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId::new(),
origin,
derivation,
input_metadata: Some(input_metadata),
},
}
}
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
match (&self.item, &self.metadata.input_metadata) {
(crate::SystemItem::Input { body }, Some(_)) => {
if body.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
(crate::SystemItem::Input { .. }, None) => {
return Err(crate::InputMetadataError::MissingMetadata);
}
(_, Some(_)) => return Err(crate::InputMetadataError::InconsistentRecord),
(_, None) => {}
}
Ok(())
}
pub fn to_history_entry(&self) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: self.item.to_history_item().into(),
metadata: self.metadata.clone(),
}
}
pub fn public_item(&self) -> crate::PublicSystemItem {
if matches!(self.item, crate::SystemItem::Input { .. }) {
return self
.metadata
.input_metadata
.as_ref()
.and_then(crate::HistoryInputMetadata::public_item)
.cloned()
.unwrap_or_else(|| self.item.public_item());
}
self.item.public_item()
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -130,6 +275,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -148,6 +294,64 @@ mod tests {
);
}
#[test]
fn external_system_records_reject_missing_policy_role_mismatch_and_overlimit_body() {
let valid = serde_json::json!({
"item": { "kind": "input", "body": "context" },
"metadata": {
"entry_id": "stable-entry", "origin": { "kind": "provided", "producer": "owner" },
"input_metadata": { "usage": {
"recording_material": "exclude", "memory_evidence": "exclude",
"observation": "hidden", "compaction": "reference_input"
} }
}
});
let restored: LoggedSystemHistoryEntry = serde_json::from_value(valid.clone()).unwrap();
let flat = serde_json::to_value(restored.to_history_entry()).unwrap();
assert!(serde_json::from_value::<LoggedHistoryEntry>(flat.clone()).is_ok());
for generic in [true, false] {
let wire = if generic { &valid } else { &flat };
let decode = |wire| {
if generic {
serde_json::from_value::<LoggedSystemHistoryEntry>(wire).is_err()
} else {
serde_json::from_value::<LoggedHistoryEntry>(wire).is_err()
}
};
let mut missing = wire.clone();
missing["metadata"]
.as_object_mut()
.unwrap()
.remove("input_metadata");
assert!(decode(missing));
let mut absent_policy = wire.clone();
absent_policy["metadata"]["input_metadata"]
.as_object_mut()
.unwrap()
.remove("usage");
assert!(decode(absent_policy));
let mut invalid_role = wire.clone();
invalid_role["item"] =
serde_json::to_value(LoggedItem::from(agen::Item::user_message("not system")))
.unwrap();
invalid_role["metadata"]["input_metadata"]["public"] = serde_json::json!({
"kind": "context", "body": "public system projection", "presentation": {"kind": "context"}
});
assert!(decode(invalid_role));
let mut oversized = wire.clone();
if generic {
oversized["item"]["body"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
} else {
oversized["item"]["content"][0]["text"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
}
assert!(decode(oversized));
}
}
#[test]
fn annotated_segment_start_is_restore_visible_without_projecting_metadata() {
let session_id = uuid::Uuid::now_v7();
+563
View File
@@ -0,0 +1,563 @@
//! Validated, Feature-neutral history-input policy and System extensions.
//!
//! The enclosing history annotation owns identity, origin and derivation. None of
//! those descriptive facts selects a use policy here. Model text, independent use
//! policies, private Feature data and an explicit public projection travel as one
//! value. Restoring this value does not load a Feature or invoke domain behavior.
//!
//! Payload validation bounds the decoded value; enclosing log readers must also
//! enforce their record allocation limit before decoding untrusted persisted input.
use agen::Item;
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::PublicSystemItem;
pub const MAX_FEATURE_PAYLOAD_BYTES: usize = 128 * 1024;
pub const MAX_SYSTEM_INPUT_BYTES: usize = 4 * 1024 * 1024;
pub const MAX_SYSTEM_PUBLIC_BYTES: usize = 256 * 1024;
const MAX_SCHEMA_BYTES: usize = 128;
const MAX_JSON_DEPTH: usize = 16;
const MAX_JSON_NODES: usize = 8192;
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum InputMetadataError {
#[error("history input payload schema is invalid")]
InvalidSchema,
#[error("history input payload version must be nonzero")]
InvalidVersion,
#[error("system input resource limit exceeded")]
ResourceLimit,
#[error("history input payload could not be serialized")]
Serialization,
#[error("history input payload schema or version is unsupported by this consumer")]
UnsupportedPayload,
#[error("history input payload does not match the owner's typed contract")]
InvalidTypedPayload,
#[error("provided input is missing explicit metadata")]
MissingMetadata,
#[error("input metadata does not match its logical record")]
InconsistentRecord,
}
/// One private payload owned by a Feature's versioned typed decoder.
/// Unknown valid schemas/versions remain data, never execution instructions.
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(try_from = "PayloadWire")]
pub struct FeatureInputPayload {
schema: String,
version: u32,
value: Value,
}
impl std::fmt::Debug for FeatureInputPayload {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("FeatureInputPayload")
.field("schema", &self.schema)
.field("version", &self.version)
.finish_non_exhaustive()
}
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct PayloadWire {
schema: String,
version: u32,
value: Value,
}
impl TryFrom<PayloadWire> for FeatureInputPayload {
type Error = InputMetadataError;
fn try_from(wire: PayloadWire) -> Result<Self, Self::Error> {
if wire.schema.is_empty()
|| wire.schema.len() > MAX_SCHEMA_BYTES
|| !wire.schema.as_bytes()[0].is_ascii_lowercase()
|| !wire.schema.bytes().all(|byte| {
byte.is_ascii_lowercase() || byte.is_ascii_digit() || b"-_.:/".contains(&byte)
})
{
return Err(InputMetadataError::InvalidSchema);
}
if wire.version == 0 {
return Err(InputMetadataError::InvalidVersion);
}
validate_json(&wire.value, MAX_FEATURE_PAYLOAD_BYTES)?;
Ok(Self {
schema: wire.schema,
version: wire.version,
value: wire.value,
})
}
}
impl FeatureInputPayload {
pub fn encode<T: Serialize>(
schema: impl Into<String>,
version: u32,
value: &T,
) -> Result<Self, InputMetadataError> {
PayloadWire {
schema: schema.into(),
version,
value: serde_json::to_value(value).map_err(|_| InputMetadataError::Serialization)?,
}
.try_into()
}
pub fn schema(&self) -> &str {
&self.schema
}
pub fn version(&self) -> u32 {
self.version
}
/// The owner explicitly selects its contract before decoding. A valid unknown
/// payload can be retained and round-tripped without calling this operation.
pub fn decode<T: DeserializeOwned>(
&self,
schema: &str,
version: u32,
) -> Result<T, InputMetadataError> {
if self.schema != schema || self.version != version {
return Err(InputMetadataError::UnsupportedPayload);
}
serde_json::from_value(self.value.clone())
.map_err(|_| InputMetadataError::InvalidTypedPayload)
}
}
/// Eligibility is supplied independently for recording and Memory evidence.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputEvidenceUse {
Include,
Exclude,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputObservationUse {
Readable,
Hidden,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputCompactionUse {
/// Ordinary history eligible for the selected retained tail.
Retain,
/// A full opening summary consumed separately and superseded by compaction.
OpeningSummary,
/// Full historical context for the compactor, not an accumulating retained tail.
ReferenceInput,
}
/// No permissive defaults: malformed/unknown policies cannot turn into eligibility.
/// Public presentation is deliberately not one of these use decisions.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct HistoryInputUsePolicy {
pub recording_material: InputEvidenceUse,
pub memory_evidence: InputEvidenceUse,
pub observation: InputObservationUse,
pub compaction: InputCompactionUse,
}
/// Use policy is independent of logical role. Non-System derived Tool context
/// carries policy only; private Feature payload and public presentation belong
/// to the System envelope and are rejected on other roles at the history boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "MetadataWire")]
pub struct HistoryInputMetadata {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct MetadataWire {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
impl TryFrom<MetadataWire> for HistoryInputMetadata {
type Error = InputMetadataError;
fn try_from(wire: MetadataWire) -> Result<Self, Self::Error> {
if let Some(public) = &wire.public {
// Kind is a descriptive label, not a Feature dispatch or authority key.
if public.kind.is_empty()
|| public.kind.len() > MAX_SCHEMA_BYTES
|| public.kind.chars().any(char::is_control)
{
return Err(InputMetadataError::InvalidSchema);
}
let value =
serde_json::to_value(public).map_err(|_| InputMetadataError::Serialization)?;
validate_json(&value, MAX_SYSTEM_PUBLIC_BYTES)?;
}
Ok(Self {
usage: wire.usage,
feature_payload: wire.feature_payload,
public: wire.public,
})
}
}
impl HistoryInputMetadata {
pub fn new(
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
) -> Result<Self, InputMetadataError> {
MetadataWire {
usage,
feature_payload,
public,
}
.try_into()
}
pub fn usage(&self) -> HistoryInputUsePolicy {
self.usage
}
pub fn feature_payload(&self) -> Option<&FeatureInputPayload> {
self.feature_payload.as_ref()
}
pub fn public_item(&self) -> Option<&PublicSystemItem> {
self.public.as_ref()
}
pub fn public_use_policy(&self) -> protocol::SessionEntryUsePolicy {
protocol::SessionEntryUsePolicy {
observation_readable: self.usage.observation == InputObservationUse::Readable,
memory_evidence: self.usage.memory_evidence == InputEvidenceUse::Include,
}
}
pub fn public_system_use_policy(&self) -> protocol::SessionEntryUsePolicy {
if self.public.is_some() {
self.public_use_policy()
} else {
protocol::SessionEntryUsePolicy {
observation_readable: false,
memory_evidence: false,
}
}
}
pub fn public_projection(&self) -> PublicSystemItem {
self.public.clone().unwrap_or_else(|| PublicSystemItem {
kind: "input".into(),
body: String::new(),
presentation: crate::PublicSystemPresentation::Ignored,
})
}
}
/// System role is invariant; provenance and use policies are attached data.
/// An absent public projection never implicitly publishes the model body.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "InputWire")]
pub struct SystemInputEnvelope {
body: String,
metadata: HistoryInputMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct InputWire {
body: String,
metadata: HistoryInputMetadata,
}
impl TryFrom<InputWire> for SystemInputEnvelope {
type Error = InputMetadataError;
fn try_from(wire: InputWire) -> Result<Self, Self::Error> {
if wire.body.len() > MAX_SYSTEM_INPUT_BYTES {
return Err(InputMetadataError::ResourceLimit);
}
Ok(Self {
body: wire.body,
metadata: wire.metadata,
})
}
}
impl SystemInputEnvelope {
pub fn new(
body: impl Into<String>,
metadata: HistoryInputMetadata,
) -> Result<Self, InputMetadataError> {
InputWire {
body: body.into(),
metadata,
}
.try_into()
}
pub fn body(&self) -> &str {
&self.body
}
pub fn metadata(&self) -> &HistoryInputMetadata {
&self.metadata
}
pub fn to_history_item(&self) -> Item {
Item::system_message(self.body.clone())
}
pub fn into_parts(self) -> (String, HistoryInputMetadata) {
(self.body, self.metadata)
}
}
fn validate_json(value: &Value, max_bytes: usize) -> Result<(), InputMetadataError> {
fn walk(value: &Value, depth: usize, remaining: &mut usize) -> Result<(), InputMetadataError> {
if depth > MAX_JSON_DEPTH || *remaining == 0 {
return Err(InputMetadataError::ResourceLimit);
}
*remaining -= 1;
match value {
Value::Array(values) => {
for value in values {
walk(value, depth + 1, remaining)?;
}
}
Value::Object(values) => {
for value in values.values() {
walk(value, depth + 1, remaining)?;
}
}
_ => {}
}
Ok(())
}
let mut remaining = MAX_JSON_NODES;
walk(value, 0, &mut remaining)?;
let bytes = serde_json::to_vec(value).map_err(|_| InputMetadataError::Serialization)?;
if bytes.len() > max_bytes {
return Err(InputMetadataError::ResourceLimit);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{LoggedItem, LoggedRole, PublicSystemPresentation};
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct OwnerPayload {
source_digest: String,
omitted: bool,
}
fn reference_policy() -> HistoryInputUsePolicy {
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
}
}
#[test]
fn unknown_valid_payload_round_trips_without_domain_decode_or_public_exposure() {
let payload = FeatureInputPayload::encode(
"future-owner:context",
99,
&OwnerPayload {
source_digest: "private-digest".into(),
omitted: true,
},
)
.unwrap();
let envelope = SystemInputEnvelope::new(
"private model context",
HistoryInputMetadata::new(reference_policy(), Some(payload), None).unwrap(),
)
.unwrap();
let restored: SystemInputEnvelope =
serde_json::from_slice(&serde_json::to_vec(&envelope).unwrap()).unwrap();
assert_eq!(restored.body(), "private model context");
assert!(restored.metadata().public_item().is_none());
let saved_payload = restored.metadata().feature_payload().unwrap();
assert_eq!(saved_payload.schema(), "future-owner:context");
assert_eq!(saved_payload.version(), 99);
assert_eq!(
saved_payload.decode::<OwnerPayload>("known-owner:context", 1),
Err(InputMetadataError::UnsupportedPayload)
);
assert_eq!(
serde_json::to_value(&restored).unwrap(),
serde_json::to_value(&envelope).unwrap()
);
assert!(!format!("{saved_payload:?}").contains("private-digest"));
}
#[test]
fn owner_decodes_typed_payload_only_for_its_explicit_contract() {
let input = OwnerPayload {
source_digest: "rev".into(),
omitted: false,
};
let payload = FeatureInputPayload::encode("owner:context", 1, &input).unwrap();
assert_eq!(
payload.decode::<OwnerPayload>("owner:context", 1).unwrap(),
input
);
let invalid = FeatureInputPayload::encode(
"owner:context",
1,
&serde_json::json!({ "omitted": "not a bool" }),
)
.unwrap();
assert_eq!(
invalid.decode::<OwnerPayload>("owner:context", 1),
Err(InputMetadataError::InvalidTypedPayload)
);
}
#[test]
fn malformed_or_oversized_payload_is_rejected_at_construction_and_restore() {
for (schema, version, value) in [
("", 1, Value::Null),
("invalid schema", 1, Value::Null),
("owner:context", 0, Value::Null),
(
"owner:context",
1,
Value::String("x".repeat(MAX_FEATURE_PAYLOAD_BYTES)),
),
(
"owner:context",
1,
Value::Array(vec![Value::Null; MAX_JSON_NODES]),
),
(
"owner:context",
1,
(0..=MAX_JSON_DEPTH).fold(Value::Null, |inner, _| Value::Array(vec![inner])),
),
] {
assert!(FeatureInputPayload::encode(schema, version, &value).is_err());
assert!(
serde_json::from_value::<FeatureInputPayload>(
serde_json::json!({ "schema": schema, "version": version, "value": value })
)
.is_err()
);
}
}
#[test]
fn policy_is_required_and_unknown_or_malformed_fields_fail_closed() {
let valid = serde_json::to_value(reference_policy()).unwrap();
for field in [
"recording_material",
"memory_evidence",
"observation",
"compaction",
] {
let mut absent = valid.clone();
absent.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<HistoryInputUsePolicy>(absent).is_err());
let mut unknown = valid.clone();
unknown[field] = Value::String("future-allow".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(unknown).is_err());
}
let mut extra = valid;
extra["feature_name"] = Value::String("infer-policy-from-me".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(extra).is_err());
}
#[test]
fn feature_name_and_model_body_do_not_select_policies_or_public_projection() {
for (schema, body) in [
("owner:journal", "past context"),
("unrelated:skill", "do not extract this; legitimate wording"),
] {
let public = PublicSystemItem {
kind: "context-label".into(),
body: "explicit public text".into(),
presentation: PublicSystemPresentation::Context,
};
let metadata = HistoryInputMetadata::new(
reference_policy(),
Some(
FeatureInputPayload::encode(
schema,
1,
&OwnerPayload {
source_digest: "private".into(),
omitted: false,
},
)
.unwrap(),
),
Some(public),
)
.unwrap();
let input = SystemInputEnvelope::new(body, metadata).unwrap();
assert_eq!(input.metadata().usage(), reference_policy());
let history: LoggedItem = input.to_history_item().into();
assert!(matches!(
history,
LoggedItem::Message {
role: LoggedRole::System,
..
}
));
assert_eq!(input.to_history_item().as_text(), Some(body));
let public = serde_json::to_string(input.metadata().public_item().unwrap()).unwrap();
assert!(!public.contains("private"));
assert!(!public.contains(schema));
assert!(!public.contains(body));
}
}
#[test]
fn use_policies_remain_independent_of_each_other_and_of_public_display() {
let usage = HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Include,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Readable,
compaction: InputCompactionUse::Retain,
};
let metadata = HistoryInputMetadata::new(usage, None, None).unwrap();
let restored: HistoryInputMetadata =
serde_json::from_value(serde_json::to_value(&metadata).unwrap()).unwrap();
assert_eq!(restored.usage(), usage);
assert!(restored.public_item().is_none());
assert!(restored.feature_payload().is_none());
}
#[test]
fn model_body_and_public_projection_have_independent_resource_limits() {
let metadata = HistoryInputMetadata::new(reference_policy(), None, None).unwrap();
assert!(
SystemInputEnvelope::new("x".repeat(MAX_SYSTEM_INPUT_BYTES + 1), metadata).is_err()
);
let public = PublicSystemItem {
kind: "context".into(),
body: "x".repeat(MAX_SYSTEM_PUBLIC_BYTES),
presentation: PublicSystemPresentation::Context,
};
assert!(HistoryInputMetadata::new(reference_policy(), None, Some(public)).is_err());
let invalid = serde_json::json!({ "body": "small", "metadata": { "public": null, "feature_payload": null } });
assert!(serde_json::from_value::<SystemInputEnvelope>(invalid).is_err());
}
}
+413 -21
View File
@@ -1,12 +1,15 @@
//! Versioned decoder for Session schemas that predate canonical annotated history.
//! Versioned decoder for Session schemas v1-5, including frozen Feature wire.
//!
//! These types are intentionally private to `session-store`. Current writers,
//! replay, and public projections use [`crate::LogEntry`] exclusively; only the
//! Worker Session schema migration is allowed to deserialize these shapes.
//! explicit Session schema migration is allowed to deserialize these shapes.
use agen::llm_client::types::RequestConfig;
use protocol::Segment;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
#[cfg(test)]
mod tests;
use crate::{
LogEntry, LoggedHistoryEntry, LoggedItem, LoggedSessionHistoryEntryId,
@@ -14,6 +17,212 @@ use crate::{
SegmentOrigin, SessionExtension, SessionId, SystemItem,
};
// Frozen v1-4 Feature wire. These types are deliberately not exported and are
// never used by LogEntry/SystemItem's current deserializers. Payload structs
// retain the owner's v1 shape rather than erasing typed fields into arbitrary JSON.
#[derive(Debug, Serialize, Deserialize)]
struct JournalStartupPayload {
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: crate::PromptRenderProvenance,
}
#[derive(Debug, Serialize, Deserialize)]
struct ResidentSummaryPayload {
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SubjectBehaviorPayload {
#[serde(default)]
behavior_md: String,
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SkillActivationPayload {
name: String,
}
#[derive(Debug, Serialize, Deserialize)]
struct InvocationResultPayload {
result: protocol::FeatureInvocationResult,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyFeatureSystemItem {
JournalStartup {
body: String,
#[serde(flatten)]
payload: JournalStartupPayload,
},
ResidentSummaryRefresh {
body: String,
#[serde(flatten)]
payload: ResidentSummaryPayload,
},
SubjectBehaviorRefresh {
body: String,
#[serde(flatten)]
payload: SubjectBehaviorPayload,
},
SkillActivation {
body: String,
#[serde(flatten)]
payload: SkillActivationPayload,
},
FeatureInvocationResult {
body: String,
#[serde(flatten)]
payload: InvocationResultPayload,
},
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacySystemItem {
Feature(LegacyFeatureSystemItem),
// Includes already-migrated Input records for interrupted migration retry.
Core(SystemItem),
}
impl LegacySystemItem {
fn into_entry(
self,
mut metadata: LoggedSessionHistoryMetadata,
) -> Result<LoggedSystemHistoryEntry, crate::InputMetadataError> {
use crate::InputCompactionUse::{ReferenceInput, Retain};
let item = match self {
Self::Core(item) => item,
Self::Feature(feature) => {
metadata.validate_input_metadata()?;
if metadata.input_metadata.is_some() {
return Err(crate::InputMetadataError::InconsistentRecord);
}
let (body, input_metadata) = match feature {
LegacyFeatureSystemItem::JournalStartup { body, payload } => {
migrated_feature_input(
body,
"journal_startup",
"journal:startup",
payload,
ReferenceInput,
)?
}
LegacyFeatureSystemItem::ResidentSummaryRefresh { body, payload } => {
migrated_feature_input(
body,
"resident_summary_refresh",
"memory:resident-summary",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SubjectBehaviorRefresh { body, payload } => {
migrated_feature_input(
body,
"subject_behavior_refresh",
"subject:behavior",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SkillActivation { body, payload } => {
migrated_feature_input(
body,
"skill_activation",
"skill:activation",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::FeatureInvocationResult { body, payload } => {
migrated_feature_input(
body,
"feature_invocation_result",
"feature:invocation-result",
payload,
Retain,
)?
}
};
// Identity/origin/derivation are retained, never re-inferred.
metadata.input_metadata = Some(input_metadata);
SystemItem::Input { body }
}
};
let entry = LoggedSystemHistoryEntry { item, metadata };
entry.validate_input_metadata()?;
Ok(entry)
}
}
fn migrated_feature_input<T: Serialize>(
body: String,
label: &str,
schema: &str,
payload: T,
compaction: crate::InputCompactionUse,
) -> Result<(String, crate::HistoryInputMetadata), crate::InputMetadataError> {
let metadata = crate::HistoryInputMetadata::new(
migration_policy(compaction),
Some(crate::FeatureInputPayload::encode(schema, 1, &payload)?),
Some(crate::PublicSystemItem {
kind: label.into(),
body: body.clone(),
presentation: crate::PublicSystemPresentation::Context,
}),
)?;
Ok((body, metadata))
}
/// Annotated old wire used by v3/v4, and by already-converted files when an
/// older marker survived an interrupted migration. No ordinary decoder uses it.
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyAnnotatedLogEntry {
AnnotatedSystemItem {
ts: u64,
entry: LegacySystemHistoryEntry,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct LegacySystemHistoryEntry {
item: LegacySystemItem,
metadata: LoggedSessionHistoryMetadata,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacyAnnotatedOrCurrent {
Annotated(LegacyAnnotatedLogEntry),
Current(LogEntry),
}
impl LegacyAnnotatedOrCurrent {
fn into_current(self) -> Result<LogEntry, crate::InputMetadataError> {
match self {
Self::Annotated(LegacyAnnotatedLogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
}) => Ok(LogEntry::AnnotatedSystemItem {
ts,
entry: entry.item.into_entry(entry.metadata)?,
extensions,
}),
Self::Current(entry) => Ok(entry),
}
}
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyHistoryLogEntry {
@@ -44,7 +253,9 @@ enum LegacyHistoryLogEntry {
},
SystemItem {
ts: u64,
item: SystemItem,
item: LegacySystemItem,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
@@ -54,7 +265,7 @@ enum LegacyHistoryLogEntry {
#[serde(untagged)]
enum LegacySessionLogEntryV1 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
/// Schema v2 retained the v1 history shapes while adding non-history records.
@@ -64,7 +275,7 @@ enum LegacySessionLogEntryV1 {
#[serde(untagged)]
enum LegacySessionLogEntryV2 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
pub(crate) fn decode_entry(
@@ -74,28 +285,207 @@ pub(crate) fn decode_entry(
segment_id: SegmentId,
line_index: usize,
) -> Result<LogEntry, serde_json::Error> {
let mut wire: serde_json::Value = serde_json::from_str(line)?;
migrate_legacy_metadata_field(&mut wire)?;
let entry = match schema_version {
1 => match serde_json::from_str::<LegacySessionLogEntryV1>(line)? {
1 => match serde_json::from_value::<LegacySessionLogEntryV1>(wire)? {
LegacySessionLogEntryV1::History(entry) => Entry::History(entry),
LegacySessionLogEntryV1::Current(entry) => Entry::Current(entry),
},
2 => match serde_json::from_str::<LegacySessionLogEntryV2>(line)? {
2 => match serde_json::from_value::<LegacySessionLogEntryV2>(wire)? {
LegacySessionLogEntryV2::History(entry) => Entry::History(entry),
LegacySessionLogEntryV2::Current(entry) => Entry::Current(entry),
},
3 | 4 => Entry::Current(serde_json::from_value(wire)?),
5 => return serde_json::from_value(wire),
_ => unreachable!("legacy decoder called for unsupported schema {schema_version}"),
};
Ok(match entry {
match entry {
Entry::History(entry) => {
canonicalize_history_entry(session_id, segment_id, line_index, entry)
}
Entry::Current(entry) => entry,
})
Entry::Current(entry) => entry.into_current(),
}
.map_err(serde::de::Error::custom)
}
// The v5 System-only metadata field was widened to role-neutral input metadata
// in v6. Touch only known history slots, never arbitrary Feature payload objects.
// Both keys are an ambiguous source, not permission to discard either value.
fn migrate_legacy_metadata_field(record: &mut serde_json::Value) -> Result<(), serde_json::Error> {
fn entry(value: &mut serde_json::Value) -> Result<(), serde_json::Error> {
if let Some(metadata) = value
.get_mut("metadata")
.and_then(serde_json::Value::as_object_mut)
{
if let Some(old) = metadata.remove("system_input") {
if metadata.contains_key("input_metadata") {
return Err(serde::de::Error::custom(
"ambiguous legacy input metadata fields",
));
}
metadata.insert("input_metadata".into(), old);
}
}
Ok(())
}
let kind = record
.get("kind")
.and_then(serde_json::Value::as_str)
.unwrap_or_default()
.to_owned();
if matches!(
kind.as_str(),
"annotated_system_item"
| "annotated_assistant_item"
| "annotated_tool_result"
| "tool_result_corrected"
) {
if let Some(value) = record.get_mut("entry") {
entry(value)?;
}
}
if let Some(history) = record
.get_mut("history")
.and_then(serde_json::Value::as_array_mut)
{
for value in history {
entry(value)?;
}
}
Ok(())
}
/// Schema 3 used DerivedSummary for Host compaction seeds. Only the trusted
/// creation boundary may identify one; arbitrary text and later seed items do not.
/// Include already-upgraded entries so an interrupted multi-file migration resumes.
pub(crate) fn compaction_summary_ids<'a>(
entries: impl Iterator<Item = &'a LogEntry>,
) -> std::collections::HashSet<LoggedSessionHistoryEntryId> {
entries
.filter_map(|entry| {
let LogEntry::AnnotatedSegmentStart {
compacted_from: Some(_),
forked_from: None,
history,
..
} = entry
else {
return None;
};
let first = history.first()?;
(matches!(
first.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
) && matches!(
first.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) && agen::Item::from(first.item.clone())
.as_text()
.is_some_and(|text| text.starts_with("[Compacted context summary]\n\n")))
.then(|| first.metadata.entry_id.clone())
})
.collect()
}
pub(crate) fn migrate_compaction_origins(
entries: &mut [LogEntry],
summaries: &std::collections::HashSet<LoggedSessionHistoryEntryId>,
) {
// Only inherited seed copies can share these identities. Appended input is
// not a construction boundary and must never be reclassified from its text.
for entry in entries {
if let LogEntry::AnnotatedSegmentStart { history, .. } = entry {
for entry in history {
if summaries.contains(&entry.metadata.entry_id)
&& entry.metadata.origin == LoggedSessionHistoryOrigin::DerivedSummary
{
entry.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
}
}
}
}
}
/// Explicit legacy-to-current boundary. No Feature inference belongs in ordinary
/// readers/restoration. Invalid or oversized payloads fail migration; the generic
/// payload/body/public limits are deliberately not relaxed for legacy data.
pub(crate) fn migrate_system_inputs(entry: &mut LogEntry) -> Result<(), crate::InputMetadataError> {
entry.validate_input_metadata()?;
match entry {
// Feature wire has already been converted by the explicit old-schema
// decoder. Ordinary LogEntry can contain only generic Input/core events.
LogEntry::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
LogEntry::AnnotatedSegmentStart { history, .. }
| LogEntry::AnnotatedUserInput { history, .. } => {
for entry in history {
migrate_flattened_system_input(entry)?;
}
}
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. }
| LogEntry::ToolResultCorrected { entry, .. } => {
migrate_flattened_system_input(entry)?;
}
_ => {}
}
entry.validate_input_metadata()
}
fn migration_policy(compaction: crate::InputCompactionUse) -> crate::HistoryInputUsePolicy {
crate::HistoryInputUsePolicy {
recording_material: crate::InputEvidenceUse::Exclude,
memory_evidence: crate::InputEvidenceUse::Exclude,
observation: crate::InputObservationUse::Hidden,
compaction,
}
}
fn migrate_flattened_system_input(
entry: &mut LoggedHistoryEntry,
) -> Result<(), crate::InputMetadataError> {
// Check before supplying legacy policy: Provided without metadata is a
// malformed NEW record, not permission to synthesize a producer policy.
entry.validate_input_metadata()?;
if entry.metadata.input_metadata.is_none()
&& (matches!(
entry.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) || matches!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
))
{
let compaction = match entry.metadata.origin {
LoggedSessionHistoryOrigin::JournalContext => crate::InputCompactionUse::ReferenceInput,
LoggedSessionHistoryOrigin::CompactionSummary => {
crate::InputCompactionUse::OpeningSummary
}
_ => crate::InputCompactionUse::Retain,
};
// Legacy derived context was excluded from independent evidence. Preserve
// that established decision explicitly, including synthetic Tool roles;
// never guess from Tool names or body text. Payload/public stay absent.
entry.metadata.input_metadata = Some(crate::HistoryInputMetadata::new(
migration_policy(compaction),
None,
None,
)?);
}
entry.validate_input_metadata()
}
enum Entry {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
fn legacy_metadata(
@@ -108,6 +498,7 @@ fn legacy_metadata(
identity.extend_from_slice(&(line_index as u64).to_be_bytes());
identity.extend_from_slice(&(item_index as u64).to_be_bytes());
LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(format!(
"l-{}",
base64::Engine::encode(&base64::engine::general_purpose::URL_SAFE_NO_PAD, identity)
@@ -122,8 +513,8 @@ fn canonicalize_history_entry(
segment_id: SegmentId,
line_index: usize,
entry: LegacyHistoryLogEntry,
) -> LogEntry {
match entry {
) -> Result<LogEntry, crate::InputMetadataError> {
Ok(match entry {
LegacyHistoryLogEntry::SegmentStart {
ts,
session_id,
@@ -177,13 +568,14 @@ fn canonicalize_history_entry(
metadata: legacy_metadata(segment_id, line_index, 0),
},
},
LegacyHistoryLogEntry::SystemItem { ts, item } => LogEntry::AnnotatedSystemItem {
LegacyHistoryLogEntry::SystemItem {
ts,
entry: LoggedSystemHistoryEntry {
item,
metadata: legacy_metadata(segment_id, line_index, 0),
},
extensions: Vec::new(),
item,
extensions,
} => LogEntry::AnnotatedSystemItem {
ts,
entry: item.into_entry(legacy_metadata(segment_id, line_index, 0))?,
extensions,
},
}
})
}
@@ -0,0 +1,105 @@
use super::*;
mod v5;
fn seed(
origin: LoggedSessionHistoryOrigin,
compacted: bool,
role: crate::LoggedRole,
text: &str,
) -> LogEntry {
LogEntry::AnnotatedSegmentStart {
ts: 1,
session_id: SessionId::now_v7(),
system_prompt: None,
config: Default::default(),
forked_from: None,
compacted_from: compacted.then(|| SegmentOrigin {
segment_id: SegmentId::now_v7(),
at_turn_index: 0,
}),
history: vec![LoggedHistoryEntry {
item: LoggedItem::Message {
role,
content: vec![crate::LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("summary".into()),
origin,
derivation: Some(crate::LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source".into())],
}),
},
}],
}
}
#[test]
fn only_trusted_compaction_seed_is_migrated_and_retries_update_inherited_copies() {
let text = "[Compacted context summary]\n\nsummary";
for (compacted, role, text) in [
(false, crate::LoggedRole::System, text),
(true, crate::LoggedRole::User, text),
(
true,
crate::LoggedRole::System,
"[Session TaskStore snapshot] tasks",
),
] {
let entry = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
compacted,
role,
text,
);
assert!(compaction_summary_ids([&entry].into_iter()).is_empty());
}
for origin in [
LoggedSessionHistoryOrigin::DerivedSummary,
LoggedSessionHistoryOrigin::CompactionSummary,
] {
let creation = seed(origin, true, crate::LoggedRole::System, text);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
text,
);
let LogEntry::AnnotatedSegmentStart { history, .. } = &inherited else {
panic!()
};
let mut pasted = history[0].clone();
pasted.metadata.entry_id = LoggedSessionHistoryEntryId("pasted".into());
let appended = LogEntry::AnnotatedUserInput {
ts: 2,
history: vec![pasted.clone()],
segments: vec![],
extensions: vec![],
};
let mut entries = vec![creation, inherited, appended];
let ids = compaction_summary_ids(entries.iter());
migrate_compaction_origins(&mut entries, &ids);
for entry in &entries[..2] {
let LogEntry::AnnotatedSegmentStart { history, .. } = entry else {
panic!()
};
assert_eq!(
history[0].metadata.origin,
LoggedSessionHistoryOrigin::CompactionSummary
);
assert_eq!(history[0].metadata.entry_id.0, "summary");
assert_eq!(
history[0].metadata.derivation.as_ref().unwrap().sources[0].0,
"source"
);
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[2] else {
panic!()
};
assert_eq!(history, &[pasted]);
let once = serde_json::to_vec(&entries).unwrap();
migrate_compaction_origins(&mut entries, &ids);
assert_eq!(serde_json::to_vec(&entries).unwrap(), once);
}
}
@@ -0,0 +1,3 @@
{"kind":"system_item","ts":1,"item":{"kind":"skill_activation","name":"legacy-skill","body":"legacy skill body"}}
{"kind":"user_input","ts":2,"segments":[{"kind":"text","content":"legacy user body"}],"extensions":[{"domain":"old","payload":{"kept":true}}]}
{"kind":"assistant_item","ts":3,"item":{"kind":"message","role":"assistant","content":[{"kind":"text","text":"legacy reply"}]}}
@@ -0,0 +1,8 @@
{"kind":"annotated_system_item","ts":101,"entry":{"item":{"kind":"journal_startup","body":"journal body\nunchanged","snapshot":{"entries":[{"text":"private snapshot"}]},"snapshot_sha256":"snapshot-digest","prompt_provenance":{"workspace_id":"workspace","source_digest":"source","projection_digest":"projection","logical_name":"journal_startup"}},"metadata":{"entry_id":"journal-id","origin":{"kind":"legacy_unknown"},"derivation":{"sources":["source-id"]}}},"extensions":[{"domain":"migration-fixture","payload":{"kept":true}}]}
{"kind":"annotated_system_item","ts":102,"entry":{"item":{"kind":"resident_summary_refresh","body":"resident body"},"metadata":{"entry_id":"resident-id","origin":{"kind":"derived_summary"}}}}
{"kind":"annotated_system_item","ts":103,"entry":{"item":{"kind":"subject_behavior_refresh","body":"rendered subject body","behavior_md":"exact behavior","prompt_provenance":{"source_digest":"subject-source","projection_digest":"subject-projection","logical_name":"subject_behavior"}},"metadata":{"entry_id":"subject-id","origin":{"kind":"backend_instruction","operation_id":"op"}}}}
{"kind":"annotated_system_item","ts":104,"entry":{"item":{"kind":"skill_activation","body":"skill body","name":"example-skill"},"metadata":{"entry_id":"skill-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":105,"entry":{"item":{"kind":"feature_invocation_result","body":"result body","result":{"invocation_id":"invoke-id","identity":"builtin:test/prepare","status":"outcome_unknown","message":"may have committed","context":null}},"metadata":{"entry_id":"result-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_user_input","ts":106,"segments":[],"history":[{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Journal] prefix is not Feature evidence"}]},"metadata":{"entry_id":"flat-journal","origin":{"kind":"journal_context"},"derivation":{"sources":["journal-id"]}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"summary body"}]},"metadata":{"entry_id":"flat-summary","origin":{"kind":"compaction_summary"}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Skill: example] never infer Feature"}]},"metadata":{"entry_id":"flat-other","origin":{"kind":"legacy_unknown"}}}]}
{"kind":"annotated_system_item","ts":107,"entry":{"item":{"kind":"notification","message":"raw message","body":"wrapped message"},"metadata":{"entry_id":"core-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":108,"entry":{"item":{"kind":"input","body":"already generic body"},"metadata":{"entry_id":"generic-id","origin":{"kind":"provided","producer":"future-owner"},"derivation":{"sources":["source-id"]},"system_input":{"usage":{"recording_material":"include","memory_evidence":"exclude","observation":"readable","compaction":"retain"},"feature_payload":{"schema":"future:unknown","version":99,"value":{"opaque":"retained"}},"public":null}}}}
@@ -0,0 +1,5 @@
{"kind":"annotated_system_item","ts":201,"entry":{"item":{"kind":"input","body":"frozen v5 reference"},"metadata":{"entry_id":"reference-v5","origin":{"kind":"provided","producer":"owner-v5"},"system_input":{"usage":{"recording_material":"exclude","memory_evidence":"exclude","observation":"hidden","compaction":"reference_input"},"feature_payload":{"schema":"future:owner","version":8,"value":{"system_input":"opaque owner field must not be renamed"}},"public":null}}}}
{"kind":"annotated_assistant_item","ts":202,"entry":{"item":{"kind":"tool_call","call_id":"original-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"original-call-entry","origin":{"kind":"model_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_tool_result","ts":203,"entry":{"item":{"kind":"tool_result","call_id":"original-call","summary":"original result"},"metadata":{"entry_id":"original-result-entry","origin":{"kind":"tool_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_assistant_item","ts":204,"entry":{"item":{"kind":"tool_call","call_id":"derived-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"derived-call-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
{"kind":"annotated_tool_result","ts":205,"entry":{"item":{"kind":"tool_result","call_id":"derived-call","summary":"derived retrieval result"},"metadata":{"entry_id":"derived-result-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
@@ -0,0 +1,709 @@
//! Fixed old wire bytes, not fixtures generated by the current serializer.
use super::*;
use crate::{FsStore, InputCompactionUse, Store, WorkerSessionStore};
use serde_json::{Value, json};
use std::{fs, path::Path};
const V4: &str = include_str!("fixtures/v4-system-inputs.jsonl");
const V1: &str = include_str!("fixtures/v1-system-inputs.jsonl");
fn decode_fixture(version: u32, source: &str, segment: SegmentId) -> Vec<LogEntry> {
source
.lines()
.enumerate()
.map(|(index, line)| decode_entry(version, line, SessionId::nil(), segment, index).unwrap())
.collect()
}
fn encode(entries: &[LogEntry]) -> Vec<u8> {
let mut bytes = Vec::new();
for entry in entries {
serde_json::to_writer(&mut bytes, entry).unwrap();
bytes.push(b'\n');
}
bytes
}
fn migrate(entries: &mut [LogEntry]) {
for entry in entries {
migrate_system_inputs(entry).unwrap();
entry.validate_input_metadata().unwrap();
}
}
fn write_worker(root: &Path, version: u32, source: &[u8]) -> (SessionId, SegmentId) {
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
fs::create_dir_all(root.join("segments")).unwrap();
fs::write(
root.join("session.json"),
serde_json::to_vec(&json!({
"schema_version": version, "session_id": session,
}))
.unwrap(),
)
.unwrap();
fs::write(
root.join("segments").join(format!("{segment}.jsonl")),
source,
)
.unwrap();
(session, segment)
}
fn assert_marker(root: &Path, version: u32) {
let manifest: Value =
serde_json::from_slice(&fs::read(root.join("session.json")).unwrap()).unwrap();
assert_eq!(manifest["schema_version"], version);
}
#[test]
fn v4_fixed_wire_preserves_identity_body_payload_public_and_independent_policy() {
let mut entries = decode_fixture(4, V4, SegmentId::nil());
let before = entries.clone();
migrate(&mut entries);
let schemas = [
"journal:startup",
"memory:resident-summary",
"subject:behavior",
"skill:activation",
"feature:invocation-result",
];
let old_wire: Vec<Value> = V4
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
for index in 0..5 {
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = &entries[index]
else {
panic!()
};
let old = &old_wire[index];
let old_item = &old["entry"]["item"];
let old_metadata: LoggedSessionHistoryMetadata =
serde_json::from_value(old["entry"]["metadata"].clone()).unwrap();
let old_ext: Vec<crate::SessionExtension> = old
.get("extensions")
.map(|value| serde_json::from_value(value.clone()).unwrap())
.unwrap_or_default();
assert_eq!(*ts, old["ts"].as_u64().unwrap());
assert_eq!(extensions, &old_ext);
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
old_item["body"].as_str().unwrap()
);
assert_eq!(entry.metadata.entry_id, old_metadata.entry_id);
assert_eq!(entry.metadata.origin, old_metadata.origin);
assert_eq!(entry.metadata.derivation, old_metadata.derivation);
let metadata = entry.metadata.input_metadata.as_ref().unwrap();
let public = metadata.public_item().unwrap();
assert_eq!(public.kind, old_item["kind"].as_str().unwrap());
assert_eq!(public.body, old_item["body"].as_str().unwrap());
assert_eq!(
public.presentation,
crate::PublicSystemPresentation::Context
);
assert_eq!(
metadata.usage(),
migration_policy(if index == 0 {
InputCompactionUse::ReferenceInput
} else {
InputCompactionUse::Retain
})
);
let payload = metadata.feature_payload().unwrap();
assert_eq!(payload.schema(), schemas[index]);
assert_eq!(payload.version(), 1);
let value: Value = payload.decode(schemas[index], 1).unwrap();
// Independently require each frozen owner's typed v1 payload contract.
match index {
0 => {
let _: JournalStartupPayload = payload.decode(schemas[index], 1).unwrap();
}
1 => {
let _: ResidentSummaryPayload = payload.decode(schemas[index], 1).unwrap();
}
2 => {
let _: SubjectBehaviorPayload = payload.decode(schemas[index], 1).unwrap();
}
3 => {
let _: SkillActivationPayload = payload.decode(schemas[index], 1).unwrap();
}
4 => {
let _: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
}
_ => unreachable!(),
}
let mut expected = old_item.clone();
expected.as_object_mut().unwrap().remove("kind");
expected.as_object_mut().unwrap().remove("body");
// Optional provenance is an explicit Option in the v1 payload contract.
if index == 1 {
expected["prompt_provenance"] = Value::Null;
}
if index == 4 {
let result: protocol::FeatureInvocationResult =
serde_json::from_value(old_item["result"].clone()).unwrap();
let typed: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
assert_eq!(typed.result, result);
// Preserve the protocol's typed serialization contract, including
// its canonical representation of optional result context.
expected["result"] = serde_json::to_value(result).unwrap();
}
assert_eq!(value, expected);
assert!(value.get("body").is_none());
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[5] else {
panic!()
};
let LogEntry::AnnotatedUserInput { history: old, .. } = &before[5] else {
panic!()
};
for (index, compaction) in [
InputCompactionUse::ReferenceInput,
InputCompactionUse::OpeningSummary,
InputCompactionUse::Retain,
]
.into_iter()
.enumerate()
{
assert_eq!(history[index].item, old[index].item);
assert_eq!(
history[index].metadata.entry_id,
old[index].metadata.entry_id
);
assert_eq!(history[index].metadata.origin, old[index].metadata.origin);
assert_eq!(
history[index].metadata.derivation,
old[index].metadata.derivation
);
let metadata = history[index].metadata.input_metadata.as_ref().unwrap();
assert_eq!(metadata.usage(), migration_policy(compaction));
assert!(metadata.public_item().is_none());
assert!(metadata.feature_payload().is_none());
}
// Core notification is unchanged; existing generic Input and unknown valid
// owner payload are retained without decoding/reclassifying a Feature.
assert_eq!(encode(&entries[6..]), encode(&before[6..]));
let once = encode(&entries);
migrate(&mut entries);
assert_eq!(encode(&entries), once);
}
#[test]
fn v1_through_v4_worker_migration_reopen_and_mixed_file_retry() {
for version in 1..=4 {
let source = if version <= 2 { V1 } else { V4 };
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), version, source.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
// Simulate an interrupted publication: v5 records precede legacy records
// in one file, and a fully new file coexists with an untouched old file.
let mut converted = decode_fixture(version, source, segment);
migrate(&mut converted);
let mut mixed = encode(&converted[..1]);
mixed.extend_from_slice(source.split_once('\n').unwrap().1.as_bytes());
fs::write(&path, &mixed).unwrap();
let new_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let new_bytes = encode(&converted);
fs::write(&new_path, &new_bytes).unwrap();
let old_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(3)));
fs::write(&old_path, source).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
WorkerSessionStore::validate_migration(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), mixed);
assert_eq!(fs::read(&old_path).unwrap(), source.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let store = WorkerSessionStore::new(root.path()).unwrap();
assert_marker(root.path(), 6);
let restored = store.read_all(session, segment).unwrap();
assert_eq!(encode(&restored), new_bytes);
assert_eq!(fs::read(&new_path).unwrap(), new_bytes);
let once = fs::read(&path).unwrap();
let old_once = fs::read(&old_path).unwrap();
drop(store);
let reopened = WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(encode(&reopened.read_all(session, segment).unwrap()), once);
assert_eq!(fs::read(&path).unwrap(), once);
assert_eq!(fs::read(&old_path).unwrap(), old_once);
}
}
#[test]
fn all_segments_preflight_keeps_old_bytes_on_decode_or_conversion_failure() {
let mut oversized: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
oversized["entry"]["item"]["snapshot"] =
json!({"large": "x".repeat(crate::input_metadata::MAX_FEATURE_PAYLOAD_BYTES + 1)});
let mut unknown: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
unknown["entry"]["item"]["kind"] = json!("unsupported_future_feature");
let mut malformed = unknown.clone();
malformed["entry"]["item"] = json!({"kind":"journal_startup", "body":"missing snapshot"});
let mut missing_metadata: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing_metadata["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let mut missing_input = missing_metadata.clone();
missing_input["entry"]["metadata"]["origin"] = json!({"kind":"legacy_unknown"});
let mut invalid_policy: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
invalid_policy["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("unknown");
let mut invalid_payload = invalid_policy.clone();
invalid_payload["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("retain");
invalid_payload["entry"]["metadata"]["system_input"]["feature_payload"]["version"] = json!(0);
let mut flat_provided: Value = serde_json::from_str(V4.lines().nth(5).unwrap()).unwrap();
flat_provided["history"][0]["metadata"]["origin"] =
json!({"kind":"provided", "producer":"owner"});
for bad in [
"{broken-json}\n".to_owned(),
format!("{}\n", oversized),
format!("{}\n", unknown),
format!("{}\n", malformed),
format!("{}\n", missing_metadata),
format!("{}\n", invalid_policy),
format!("{}\n", invalid_payload),
format!("{}\n", flat_provided),
format!("{}\n", missing_input),
] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 4, V4.as_bytes());
let first = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&later, &bad).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::validate_migration(root.path()).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read(&later).unwrap(), bad.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn unsupported_schema_preserves_all_old_bytes() {
for version in [0, 7, 99] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V4.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), V4.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn standalone_store_uses_same_v1_through_v4_migration_and_reopens() {
// None represents the existing unmarked annotated standalone layout (v3).
for version in [None, Some(1), Some(2), Some(3), Some(4)] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let source = if version.is_some_and(|v| v <= 2) {
V1
} else {
V4
};
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, source).unwrap();
if let Some(version) = version {
fs::write(dir.join("session-schema.json"), version.to_string()).unwrap();
}
let store = FsStore::new(root.path()).unwrap();
let entries = store.read_all(session, segment).unwrap();
for entry in entries {
entry.validate_input_metadata().unwrap();
}
assert_eq!(
fs::read_to_string(dir.join("session-schema.json"))
.unwrap()
.trim(),
"6"
);
let once = fs::read(&path).unwrap();
drop(store);
FsStore::new(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), once);
}
}
#[test]
fn compaction_origin_upgrade_is_retained_only_for_pre_v4_sources() {
for version in [3, 4] {
let root = tempfile::tempdir().unwrap();
let creation = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
true,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let (session, segment) =
write_worker(root.path(), version, &encode(&[creation, inherited]));
let store = WorkerSessionStore::new(root.path()).unwrap();
for record in store.read_all(session, segment).unwrap() {
let LogEntry::AnnotatedSegmentStart { history, .. } = record else {
panic!()
};
let metadata = &history[0].metadata;
assert_eq!(metadata.entry_id.0, "summary");
assert_eq!(metadata.derivation.as_ref().unwrap().sources[0].0, "source");
assert_eq!(
metadata.origin,
if version == 3 {
LoggedSessionHistoryOrigin::CompactionSummary
} else {
LoggedSessionHistoryOrigin::DerivedSummary
}
);
assert_eq!(
metadata.input_metadata.as_ref().unwrap().usage().compaction,
if version == 3 {
InputCompactionUse::OpeningSummary
} else {
InputCompactionUse::Retain
}
);
}
}
}
#[test]
fn standalone_preflight_and_unsupported_marker_leave_sources_unchanged() {
for version in [4, 99] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let first = dir.join(format!("{}.jsonl", SegmentId::from_u128(1)));
let later = dir.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&first, V4).unwrap();
fs::write(&later, "{broken-json}\n").unwrap();
let marker = version.to_string();
fs::write(dir.join("session-schema.json"), &marker).unwrap();
assert!(FsStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read_to_string(&later).unwrap(), "{broken-json}\n");
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
marker
);
}
}
#[test]
fn current_v5_rejects_all_legacy_feature_kinds_without_rewriting_sources() {
for line in V4.lines().take(5) {
// Current serde, writable open and read-only restore share the same
// current wire contract. None may dispatch to the old Feature decoder.
assert!(serde_json::from_str::<LogEntry>(line).is_err());
let bytes = format!("{line}\n");
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), 6, bytes.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
let readonly = WorkerSessionStore::open_read_only(root.path()).unwrap();
assert!(readonly.read_all_read_only(session, segment).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let standalone = tempfile::tempdir().unwrap();
let dir = standalone.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, &bytes).unwrap();
fs::write(dir.join("session-schema.json"), "5").unwrap();
assert!(FsStore::new(standalone.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
"5"
);
}
}
#[test]
fn v1_v2_feature_wire_uses_the_same_typed_payloads_and_stable_legacy_identity() {
let segment = SegmentId::from_u128(42);
for version in [1, 2] {
for (index, line) in V4.lines().take(5).enumerate() {
let annotated: Value = serde_json::from_str(line).unwrap();
let wire = json!({
"kind": "system_item", "ts": annotated["ts"],
"item": annotated["entry"]["item"],
"extensions": [{"domain":"old-owner", "payload":{"kept":true}}],
})
.to_string();
let entry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
let retry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
assert_eq!(encode(&[entry.clone()]), encode(&[retry]));
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = entry
else {
panic!()
};
assert_eq!(ts, annotated["ts"].as_u64().unwrap());
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
annotated["entry"]["item"]["body"].as_str().unwrap()
);
assert_eq!(
entry.metadata.entry_id,
legacy_metadata(segment, index, 0).entry_id
);
assert_eq!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::LegacyUnknown
);
assert!(entry.metadata.derivation.is_none());
assert_eq!(extensions[0].domain, "old-owner");
assert_eq!(extensions[0].payload, json!({"kept":true}));
let LogEntry::AnnotatedSystemItem { entry: v4, .. } =
decode_entry(4, line, SessionId::nil(), segment, index).unwrap()
else {
panic!()
};
assert_eq!(entry.metadata.input_metadata, v4.metadata.input_metadata);
}
}
}
#[test]
fn migration_rejects_malformed_typed_feature_fields_instead_of_retaining_unchecked_json() {
let mut items: Vec<Value> = V4
.lines()
.take(5)
.map(|line| serde_json::from_str::<Value>(line).unwrap()["entry"]["item"].clone())
.collect();
items[0]["snapshot_sha256"] = json!(17);
items[1]["prompt_provenance"] = json!({"logical_name":"missing digests"});
items[2]["behavior_md"] = json!(["not a string"]);
items[3]["name"] = json!({"not":"a name"});
items[4]["result"]["status"] = json!("unsupported_status");
for version in 1..=4 {
for item in &items {
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"malformed-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).is_err());
}
}
}
#[test]
fn frozen_behavior_counter_is_accepted_only_at_the_old_schema_boundary() {
for version in 1..=4 {
let item = json!({
"kind":"subject_behavior_refresh", "body":"exact archived body",
"behavior_revision":7,
});
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"behavior-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(serde_json::from_str::<LogEntry>(&wire).is_err());
let LogEntry::AnnotatedSystemItem { entry, .. } =
decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).unwrap()
else {
panic!()
};
assert_eq!(entry.item.history_text(), "exact archived body");
let payload = entry
.metadata
.input_metadata
.as_ref()
.unwrap()
.feature_payload()
.unwrap();
let typed: SubjectBehaviorPayload = payload.decode("subject:behavior", 1).unwrap();
assert_eq!(typed.behavior_md, "");
assert!(typed.prompt_provenance.is_none());
assert_eq!(
serde_json::to_value(typed).unwrap(),
json!({
"behavior_md":"", "prompt_provenance":null,
})
);
}
}
#[test]
fn current_schema_rejects_generic_input_without_metadata_instead_of_migrating_it() {
let mut missing: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let bytes = format!("{}\n", missing);
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 6, bytes.as_bytes());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(
fs::read(
root.path()
.join("segments")
.join(format!("{segment}.jsonl"))
)
.unwrap(),
bytes.as_bytes()
);
assert_marker(root.path(), 6);
}
const V5: &str = include_str!("fixtures/v5-role-metadata.jsonl");
#[test]
fn v5_metadata_and_derived_tool_context_upgrade_without_changing_roles_or_original_evidence() {
for standalone in [false, true] {
let root = tempfile::tempdir().unwrap();
let (session, segment) = if standalone {
let session = SessionId::now_v7();
let segment = SegmentId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
fs::write(dir.join("session-schema.json"), b"5").unwrap();
fs::write(dir.join(format!("{segment}.jsonl")), V5).unwrap();
(session, segment)
} else {
write_worker(root.path(), 5, V5.as_bytes())
};
let entries = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
let originals: Vec<Value> = V5
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
assert_eq!(entries.len(), originals.len());
for (index, (entry, old)) in entries.iter().zip(originals).enumerate() {
let wire = serde_json::to_value(entry).unwrap();
assert_eq!(wire["ts"], old["ts"]);
assert_eq!(wire["entry"]["item"], old["entry"]["item"]);
for field in ["entry_id", "origin", "derivation"] {
assert_eq!(
wire["entry"]["metadata"][field],
old["entry"]["metadata"][field]
);
}
let metadata = &wire["entry"]["metadata"];
if index == 0 {
assert_eq!(
metadata["input_metadata"],
old["entry"]["metadata"]["system_input"]
);
assert_eq!(
metadata["input_metadata"]["feature_payload"]["value"]["system_input"],
"opaque owner field must not be renamed"
);
} else if index < 3 {
assert!(
metadata.get("input_metadata").is_none(),
"ordinary model Tool originals must not acquire exclusion policy"
);
} else {
assert_eq!(
metadata["input_metadata"]["usage"],
json!({
"recording_material":"exclude", "memory_evidence":"exclude", "observation":"hidden", "compaction":"retain"
})
);
}
}
let before = encode(&entries);
let reopened = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
assert_eq!(encode(&reopened), before);
}
}
#[test]
fn legacy_metadata_alias_conflict_fails_preflight_without_dropping_either_value() {
let mut bad: Value = serde_json::from_str(V5.lines().next().unwrap()).unwrap();
bad["entry"]["metadata"]["input_metadata"] = bad["entry"]["metadata"]["system_input"].clone();
for version in [4, 5] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V5.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let bad_bytes = format!("{bad}\n");
fs::write(&later, &bad_bytes).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(path).unwrap(), V5.as_bytes());
assert_eq!(fs::read(later).unwrap(), bad_bytes.as_bytes());
assert_marker(root.path(), version);
}
}
+9 -1
View File
@@ -38,9 +38,12 @@ pub(crate) fn read_without_atime_bounded(path: &Path, max_bytes: u64) -> io::Res
}
}
pub mod capture;
mod committed_replay;
pub mod event_trace;
pub mod fs_store;
pub mod history;
pub mod input_metadata;
mod legacy_session_log;
pub mod logged_item;
mod paste_artifact;
@@ -65,6 +68,10 @@ pub use history::{
LoggedSessionHistoryMetadata, LoggedSessionHistoryOrigin, LoggedSystemHistoryEntry,
LoggedWorkerSubject,
};
pub use input_metadata::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputMetadataError, InputObservationUse, SystemInputEnvelope,
};
pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged};
pub use paste_artifact::PasteArtifactLimits;
pub use public_index::{
@@ -98,7 +105,8 @@ pub use segment::{
pub use segment_log::{LogEntry, RestoredState, SegmentOrigin, SessionExtension, collect_state};
pub use store::{Store, StoreError};
pub use system_item::{
PromptRenderProvenance, SystemItem, SystemReminder, SystemReminderSource, render_worker_event,
PromptRenderProvenance, PublicSystemItem, PublicSystemPresentation, SystemItem, SystemReminder,
SystemReminderSource, render_worker_event,
};
pub use uploaded_file::{
DEFAULT_MAX_FILES_PER_SUBMISSION, DEFAULT_MAX_SESSION_ARTIFACT_BYTES,
+7 -1
View File
@@ -1296,6 +1296,7 @@ mod tests {
LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.to_owned()),
origin,
derivation: None,
@@ -1537,6 +1538,7 @@ mod tests {
prompt_provenance: None,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("system-item".into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction {
operation_id: None,
@@ -1602,6 +1604,7 @@ mod tests {
segment_id(20),
"completed",
LogEntry::RunCompleted {
entry_id: None,
ts: 4,
interrupted: false,
result: EngineResult::Finished,
@@ -1640,7 +1643,10 @@ mod tests {
(
segment_id(24),
"abandoned",
LogEntry::PausedTurnAbandoned { ts: 4 },
LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 4,
},
),
];
for (segment_id, text, outcome) in outcomes {
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+76 -21
View File
@@ -27,7 +27,7 @@ use thiserror::Error;
use crate::{
LogEntry, LoggedContentPart, LoggedHistoryEntry, LoggedItem, LoggedRole,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError, SystemItem,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError,
WorkerAggregateStore, WorkerSessionStore, WorkerStoreError,
};
@@ -1165,7 +1165,7 @@ fn capture_tool_correction(
let LogEntry::ToolResultCorrected { entry, .. } = record else {
return false;
};
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
// We read backwards: the first correction seen is the latest one.
corrections
.entry(entry.metadata.entry_id.0.clone())
@@ -1600,12 +1600,12 @@ pub(crate) fn project_session_snapshot_for_segment(
} => extend_history(&mut entries, history, Some(segments), *ts),
LogEntry::AnnotatedAssistantItem { ts, entry }
| LogEntry::AnnotatedToolResult { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
entries.push(history_entry(entry, *ts, data));
}
}
LogEntry::ToolResultCorrected { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
if let Some(existing) = entries
.iter_mut()
.find(|existing| existing.entry_id == entry.metadata.entry_id.0)
@@ -1618,11 +1618,16 @@ pub(crate) fn project_session_snapshot_for_segment(
}
}
LogEntry::AnnotatedSystemItem { ts, entry, .. } => entries.push(system_entry(
&entry.item,
&entry.public_item(),
entry.metadata.entry_id.0.clone(),
*ts,
provenance(&entry.metadata.origin),
derivation_ids(entry),
entry
.metadata
.input_metadata
.as_ref()
.map(crate::HistoryInputMetadata::public_system_use_policy),
)),
LogEntry::RunYielded {
ts,
@@ -1705,6 +1710,7 @@ pub(crate) fn project_session_snapshot_for_segment(
interrupted,
result,
active_run_turn_count: persisted_count,
..
} => {
if *interrupted && matches!(result, agen::EngineResult::Yielded) {
let count = (*persisted_count)
@@ -1774,7 +1780,7 @@ fn extend_history(
segments: input_segments.cloned().unwrap_or_default(),
}
} else {
let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) else {
let Some(data) = project_history_item(entry) else {
continue;
};
data
@@ -1789,6 +1795,19 @@ fn history_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: entry.metadata.input_metadata.as_ref().map(|input| {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
input.public_system_use_policy()
} else {
input.public_use_policy()
}
}),
entry_id: entry.metadata.entry_id.0.clone(),
timestamp,
provenance: provenance(&entry.metadata.origin),
@@ -1832,6 +1851,7 @@ fn run_transition_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: persisted_entry_id
.map(|entry_id| entry_id.0.clone())
.unwrap_or_else(|| legacy_entry_id(session_key, segment_id, log_index, 0)),
@@ -1851,6 +1871,7 @@ fn legacy_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: legacy_entry_id(session_key, segment_id, log_index, item_index),
timestamp,
provenance: SessionEntryProvenance::LegacyUnknown,
@@ -1883,12 +1904,12 @@ fn provenance(origin: &LoggedSessionHistoryOrigin) -> SessionEntryProvenance {
LoggedSessionHistoryOrigin::FlowInstruction { .. } => {
SessionEntryProvenance::FlowInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. } => {
SessionEntryProvenance::BackendInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. }
| LoggedSessionHistoryOrigin::Provided { .. } => SessionEntryProvenance::BackendInstruction,
LoggedSessionHistoryOrigin::ModelOutput { .. } => SessionEntryProvenance::ModelOutput,
LoggedSessionHistoryOrigin::ToolOutput { .. } => SessionEntryProvenance::ToolOutput,
LoggedSessionHistoryOrigin::CompactionSummary
| LoggedSessionHistoryOrigin::JournalContext
| LoggedSessionHistoryOrigin::DerivedSummary => SessionEntryProvenance::DerivedSummary,
LoggedSessionHistoryOrigin::LegacyUnknown => SessionEntryProvenance::LegacyUnknown,
}
@@ -1912,6 +1933,35 @@ pub fn session_tool_attachment_id(
URL_SAFE_NO_PAD.encode(hash.finalize())
}
fn project_history_item(entry: &LoggedHistoryEntry) -> Option<SessionSnapshotEntryData> {
if let Some(input) = &entry.metadata.input_metadata {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
let public = input.public_projection();
return Some(SessionSnapshotEntryData::SystemItem {
item_kind: public.kind.clone(),
content: public.body.clone(),
data: Some(
serde_json::to_value(public).expect("public system input is serializable"),
),
});
}
let usage = input.usage();
if usage.recording_material == crate::InputEvidenceUse::Exclude
&& usage.memory_evidence == crate::InputEvidenceUse::Exclude
&& usage.observation == crate::InputObservationUse::Hidden
{
return None;
}
}
project_item(&entry.metadata.entry_id.0, &entry.item)
}
fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntryData> {
match item {
LoggedItem::Message { role, content } => {
@@ -1984,30 +2034,26 @@ fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntr
}
fn system_entry(
item: &SystemItem,
item: &crate::PublicSystemItem,
entry_id: String,
timestamp: u64,
provenance: SessionEntryProvenance,
derived_from: Vec<String>,
use_policy: Option<protocol::SessionEntryUsePolicy>,
) -> SessionSnapshotEntry {
let mut data = serde_json::to_value(item).ok();
if let Some(serde_json::Value::Object(object)) = data.as_mut() {
object.remove("prompt_provenance");
}
let item_kind = data
.as_ref()
.and_then(|value| value.get("kind"))
.and_then(serde_json::Value::as_str)
.unwrap_or("system_item")
.to_owned();
let public = item.clone();
let data = Some(serde_json::to_value(&public).expect("public system input is serializable"));
let item_kind = public.kind;
let content = public.body;
SessionSnapshotEntry {
use_policy,
entry_id,
timestamp,
provenance,
derived_from,
data: SessionSnapshotEntryData::SystemItem {
item_kind,
content: item.history_text(),
content,
data,
},
}
@@ -2037,6 +2083,7 @@ mod tests {
fn complete(ts: u64) -> LogEntry {
LogEntry::RunCompleted {
entry_id: None,
ts,
interrupted: false,
result: agen::EngineResult::Finished,
@@ -2288,6 +2335,7 @@ mod tests {
content: vec![LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(entry_id.into()),
origin,
derivation: None,
@@ -3586,6 +3634,7 @@ mod tests {
.map(|item| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -3640,6 +3689,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id,
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -3709,6 +3759,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::FlowInstruction {
selector: "builtin:coder-review".into(),
@@ -3728,6 +3779,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id.clone(),
origin: origin.clone(),
derivation: Some(LoggedHistoryDerivation {
@@ -3758,6 +3810,7 @@ mod tests {
fn annotated_projection_preserves_identity_and_provenance() {
let session_id = crate::new_session_id();
let metadata = LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::ModelOutput {
worker: LoggedWorkerSubject {
@@ -4320,6 +4373,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("ancestor-image".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4414,6 +4468,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("tool-images".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -103,6 +103,7 @@ fn finish(
stats.insert(
anchor.clone(),
SessionSnapshotEntry {
use_policy: None,
entry_id: format!("run-stats:{anchor}"),
timestamp: ended_at,
provenance: SessionEntryProvenance::LegacyUnknown,
+11 -1
View File
@@ -46,6 +46,13 @@ pub struct SessionRetentionStore {
/// Do not recursively acquire the same Session lock inside a guarded callback.
pub struct SessionRetentionGuard {
_file: File,
session_id: SessionId,
}
impl SessionRetentionGuard {
pub fn session_id(&self) -> SessionId {
self.session_id
}
}
impl SessionRetentionStore {
@@ -71,7 +78,10 @@ impl SessionRetentionStore {
}
let file = options.open(self.root.join(format!("{session}.lock")))?;
FileExt::lock_exclusive(&file)?;
Ok(SessionRetentionGuard { _file: file })
Ok(SessionRetentionGuard {
_file: file,
session_id: session,
})
}
fn read(&self, session: SessionId) -> Result<RetentionState, StoreError> {
+1
View File
@@ -329,6 +329,7 @@ pub fn save_run_completed(
session_id,
segment_id,
LogEntry::RunCompleted {
entry_id: None,
ts: segment_log::now_millis(),
interrupted,
result,
+43 -6
View File
@@ -122,6 +122,8 @@ pub enum LogEntry {
/// Replay restores both interruption state and any resumable logical-run
/// turn budget.
RunCompleted {
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
ts: u64,
interrupted: bool,
result: EngineResult,
@@ -194,7 +196,11 @@ pub enum LogEntry {
/// A paused interrupted turn was explicitly abandoned without calling
/// `run()` or `resume()` again. Replay clears the interrupted marker so
/// the restored Worker is idle and future user input starts a normal new turn.
PausedTurnAbandoned { ts: u64 },
PausedTurnAbandoned {
ts: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
},
/// `RequestConfig` changed.
ConfigChanged { ts: u64, config: RequestConfig },
@@ -248,12 +254,31 @@ pub struct SegmentOrigin {
}
impl LogEntry {
/// Validate generic System records before a Store publishes any bytes.
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
match self {
Self::AnnotatedSegmentStart { history, .. }
| Self::AnnotatedUserInput { history, .. } => {
for entry in history {
entry.validate_input_metadata()?;
}
}
Self::AnnotatedAssistantItem { entry, .. }
| Self::AnnotatedToolResult { entry, .. }
| Self::ToolResultCorrected { entry, .. } => entry.validate_input_metadata()?,
Self::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
_ => {}
}
Ok(())
}
/// Allocate the durable public identity for a newly committed Run
/// transition. Calling this repeatedly is idempotent so the exact value
/// written to storage is also the value published to live subscribers.
pub fn ensure_session_entry_id(&mut self) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
Self::RunCompleted { entry_id, .. }
| Self::PausedTurnAbandoned { entry_id, .. }
| Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
@@ -275,7 +300,9 @@ impl LogEntry {
record_start: u64,
) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
Self::RunCompleted { entry_id, .. }
| Self::PausedTurnAbandoned { entry_id, .. }
| Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
@@ -527,6 +554,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -586,6 +614,7 @@ mod tests {
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 3200,
interrupted: false,
result: EngineResult::Finished,
@@ -893,12 +922,16 @@ mod tests {
compacted_from: None,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 100,
interrupted: true,
result: EngineResult::Paused,
active_run_turn_count: Some(1),
},
LogEntry::PausedTurnAbandoned { ts: 200 },
LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 200,
},
]);
assert!(!state.last_run_interrupted);
assert_eq!(state.active_run_turn_count, None);
@@ -1056,6 +1089,7 @@ mod tests {
turn_count: 2,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 300,
interrupted: true,
result: EngineResult::LimitReached,
@@ -1069,13 +1103,16 @@ mod tests {
#[test]
fn paused_turn_abandoned_entry_round_trip_via_json() {
let entry = LogEntry::PausedTurnAbandoned { ts: 12345 };
let entry = LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 12345,
};
let json = serde_json::to_string(&entry).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["kind"], "paused_turn_abandoned");
let decoded: LogEntry = serde_json::from_str(&json).unwrap();
match decoded {
LogEntry::PausedTurnAbandoned { ts } => assert_eq!(ts, 12345),
LogEntry::PausedTurnAbandoned { ts, .. } => assert_eq!(ts, 12345),
other => panic!("expected PausedTurnAbandoned, got {other:?}"),
}
}
+3
View File
@@ -27,6 +27,9 @@ pub enum StoreError {
#[error("serialization error: {0}")]
Serde(#[from] serde_json::Error),
#[error("invalid history input metadata: {0}")]
InvalidInputMetadata(#[from] crate::InputMetadataError),
#[error("segment not found: {0}")]
NotFound(SegmentId),
+92 -103
View File
@@ -86,20 +86,16 @@ pub struct PromptRenderProvenance {
/// `notify_wrapper` template) re-shape the live rendering on a later
/// resume.
///
/// New variants get added here as fresh injection kinds come online
/// (e.g. `TaskReminder`). The `kind` JSON tag is the snake_case form of
/// the variant name.
/// Feature-owned inputs use `Input` plus explicit annotated metadata; adding a
/// Feature must not extend this enum or ordinary replay. Only core events have
/// dedicated variants. Pre-v5 Feature kinds exist exclusively in the migration
/// decoder, not in this current wire contract.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum SystemItem {
/// One-time derived Journal reference context. The bounded Host snapshot is
/// retained as JSON independently of its rendered body and prompt projection.
JournalStartup {
body: String,
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: PromptRenderProvenance,
},
/// Generic System body; private metadata and public presentation are supplied
/// independently by the producer in the same annotated history record.
Input { body: String },
/// Free-form notification sent in by an external caller via
/// `Method::Notify`. `message` is the raw caller-supplied text;
/// `body` is the wrapped LLM-context form (Worker renders it via
@@ -122,14 +118,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Result of an explicitly selected chat Feature invocation. The typed
/// result is retained for clients; `body` is the exact bounded context made
/// visible to the Worker model.
FeatureInvocationResult {
result: protocol::FeatureInvocationResult,
body: String,
},
/// `@<path>` file reference resolution. `body` is the rendered
/// LLM-context text (`[File: <path>]\n…` for regular files,
/// `[Dir: <path>]\n…` for directory listings, possibly with a
@@ -137,10 +125,6 @@ pub enum SystemItem {
/// byte-identical to what was sent.
FileAttachment { path: String, body: String },
/// Explicit Agent Skill activation. `body` is the exact LLM-facing
/// Skill text committed before any subsequent LLM request can use it.
SkillActivation { name: String, body: String },
/// Historical persisted Knowledge reference resolution. Knowledge is no
/// longer active, so restored sessions ignore this item instead of replaying
/// archived record text into model context.
@@ -164,29 +148,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Restore-boundary refresh of the current resident Memory projection.
/// The exact rendered body is retained so replay is byte-identical; a
/// refresh with no ready summary explicitly supersedes older resident
/// context without deleting or rewriting history.
ResidentSummaryRefresh {
body: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// A Host-fetched change to user-managed Subject behavior. Each changed
/// body is appended rather than rewriting prior prompt/session content.
SubjectBehaviorRefresh {
body: String,
/// Exact user-managed behavior acknowledged after durable commit. Old
/// logs omit this value; an empty value is not an acknowledgement of a
/// nonempty behavior and does not replace their stored rendered body.
#[serde(default)]
behavior_md: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Synthetic note inserted after an interrupted turn before the next
/// user input. `body` is the exact LLM-context text explaining that the
/// previous turn was cut short.
@@ -197,24 +158,67 @@ pub enum SystemItem {
},
}
/// Allowlisted presentation data shared by live events and retained snapshots.
/// Private Feature payloads and render provenance never cross this boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct PublicSystemItem {
pub kind: String,
pub body: String,
pub presentation: PublicSystemPresentation,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PublicSystemPresentation {
Notification { message: String },
WorkerEvent { event: WorkerEvent },
Context,
TaskReminder,
Ignored,
}
impl SystemItem {
pub fn public_item(&self) -> PublicSystemItem {
let presentation = match self {
Self::Notification { message, .. } => PublicSystemPresentation::Notification {
message: message.clone(),
},
Self::WorkerEvent { event, .. } => PublicSystemPresentation::WorkerEvent {
event: event.clone(),
},
Self::TaskReminder { .. } => PublicSystemPresentation::TaskReminder,
Self::Input { .. }
| Self::LegacyKnowledgeIgnored { .. }
| Self::LegacyIgnored { .. } => PublicSystemPresentation::Ignored,
Self::FileAttachment { .. } | Self::Interrupt { .. } => {
PublicSystemPresentation::Context
}
};
PublicSystemItem {
kind: self.kind_label().to_owned(),
body: if matches!(presentation, PublicSystemPresentation::Ignored) {
String::new()
} else {
self.history_text()
},
presentation,
}
}
/// Free-text body the LLM sees inside its `role:system` message
/// for this item. Returns the variant's stored `body` verbatim.
pub fn history_text(&self) -> String {
match self {
SystemItem::JournalStartup { body, .. } => body.clone(),
SystemItem::Input { body } => body.clone(),
SystemItem::Notification { body, .. } => body.clone(),
SystemItem::WorkerEvent { body, .. } => body.clone(),
SystemItem::FeatureInvocationResult { body, .. } => body.clone(),
SystemItem::FileAttachment { body, .. } => body.clone(),
SystemItem::SkillActivation { body, .. } => body.clone(),
SystemItem::LegacyKnowledgeIgnored { .. } => String::new(),
SystemItem::LegacyIgnored { slug } => {
format!("Ignored legacy procedure item: /{slug}")
}
SystemItem::TaskReminder { body, .. } => body.clone(),
SystemItem::ResidentSummaryRefresh { body, .. } => body.clone(),
SystemItem::SubjectBehaviorRefresh { body, .. } => body.clone(),
SystemItem::Interrupt { body, .. } => body.clone(),
}
}
@@ -229,17 +233,13 @@ impl SystemItem {
/// wire — keep flexible.
pub fn kind_label(&self) -> &'static str {
match self {
SystemItem::JournalStartup { .. } => "journal_startup",
SystemItem::Input { .. } => "input",
SystemItem::Notification { .. } => "notification",
SystemItem::WorkerEvent { .. } => "worker_event",
SystemItem::FeatureInvocationResult { .. } => "feature_invocation_result",
SystemItem::FileAttachment { .. } => "file_attachment",
SystemItem::SkillActivation { .. } => "skill_activation",
SystemItem::LegacyKnowledgeIgnored { .. } => "legacy_knowledge_ignored",
SystemItem::LegacyIgnored { .. } => "legacy_ignored",
SystemItem::TaskReminder { .. } => "task_reminder",
SystemItem::ResidentSummaryRefresh { .. } => "resident_summary_refresh",
SystemItem::SubjectBehaviorRefresh { .. } => "subject_behavior_refresh",
SystemItem::Interrupt { .. } => "interrupt",
}
}
@@ -297,33 +297,45 @@ mod tests {
}
#[test]
fn subject_behavior_refresh_round_trips_with_exact_body() {
let item = SystemItem::SubjectBehaviorRefresh {
body: "current Subject context".to_string(),
behavior_md: "exact behavior".to_string(),
prompt_provenance: None,
};
let raw = serde_json::to_string(&item).unwrap();
let parsed: SystemItem = serde_json::from_str(&raw).unwrap();
assert_eq!(parsed.history_text(), "current Subject context");
assert!(
matches!(parsed, SystemItem::SubjectBehaviorRefresh { behavior_md, .. } if behavior_md == "exact behavior")
);
fn current_system_item_rejects_frozen_feature_kinds() {
// Even well-formed old payloads cannot enter normal v5 decode/replay.
for line in include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl")
.lines()
.take(5)
{
let record: serde_json::Value = serde_json::from_str(line).unwrap();
assert!(serde_json::from_value::<SystemItem>(record["entry"]["item"].clone()).is_err());
assert!(
serde_json::from_value::<crate::LoggedSystemHistoryEntry>(record["entry"].clone())
.is_err()
);
assert!(serde_json::from_str::<crate::LogEntry>(line).is_err());
}
}
#[test]
fn frozen_subject_behavior_log_reads_old_counter_without_republishing_it() {
let old = r#"{"kind":"subject_behavior_refresh","body":"exact archived body","behavior_revision":7}"#;
let item: SystemItem = serde_json::from_str(old).unwrap();
assert_eq!(item.history_text(), "exact archived body");
assert_eq!(
serde_json::to_value(item).unwrap(),
serde_json::json!({
"kind": "subject_behavior_refresh",
"body": "exact archived body",
"behavior_md": ""
})
);
fn public_system_input_decoder_rejects_private_fields_and_unknown_presentations() {
let valid = serde_json::json!({
"kind": "new-feature-label",
"body": "explicit public content",
"presentation": { "kind": "context" },
});
let public: PublicSystemItem = serde_json::from_value(valid.clone()).unwrap();
assert_eq!(public.body, "explicit public content");
for private_field in [
"payload",
"prompt_provenance",
"policy",
"snapshot",
"behavior_md",
] {
let mut malformed = valid.clone();
malformed[private_field] = serde_json::json!({ "private": true });
assert!(serde_json::from_value::<PublicSystemItem>(malformed).is_err());
}
let mut unknown = valid;
unknown["presentation"] = serde_json::json!({ "kind": "execute_feature_payload" });
assert!(serde_json::from_value::<PublicSystemItem>(unknown).is_err());
}
#[test]
@@ -419,29 +431,6 @@ mod tests {
}
}
#[test]
fn invocation_result_retains_typed_outcome_and_exact_context() {
let result = protocol::FeatureInvocationResult {
invocation_id: "stable-invoke".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
status: protocol::FeatureInvocationStatus::OutcomeUnknown,
message: "operation may have committed".into(),
context: None,
};
let body = "Outcome unknown; do not retry automatically";
let item = SystemItem::FeatureInvocationResult {
result: result.clone(),
body: body.into(),
};
let restored: SystemItem =
serde_json::from_str(&serde_json::to_string(&item).unwrap()).unwrap();
assert_eq!(restored.history_text(), body);
assert_eq!(restored.kind_label(), "feature_invocation_result");
assert!(
matches!(restored, SystemItem::FeatureInvocationResult { result: saved, .. } if saved == result)
);
}
#[test]
fn round_trip_worker_event() {
let item = SystemItem::WorkerEvent {
+21 -43
View File
@@ -210,18 +210,6 @@ impl WorkerMetadata {
}
}
fn replace_active_session(metadata: &mut WorkerMetadata, active: Option<WorkerActiveSegmentRef>) {
if let Some(session_id) = active.as_ref().map(|active| active.session_id)
&& metadata
.subjektiv_session_attribution
.as_ref()
.is_some_and(|state| state.session_id() != session_id)
{
metadata.subjektiv_session_attribution = None;
}
metadata.active = active;
}
/// Sync persistence backend for Worker metadata.
pub trait WorkerMetadataStore: Send + Sync {
/// Create or replace metadata for its `worker_name` key.
@@ -297,7 +285,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -317,7 +305,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_id: Option<String>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -334,7 +322,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_root) = workspace_root {
metadata.workspace_root = Some(workspace_root);
@@ -342,18 +330,6 @@ pub trait WorkerMetadataStore: Send + Sync {
})
}
/// Set the Host-owned Subjektiv Session attribution lifecycle state while
/// preserving the active Session pointer and every unrelated field.
fn set_subjektiv_session_attribution(
&self,
worker_name: &str,
state: SubjektivSessionAttributionState,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.subjektiv_session_attribution = Some(state);
})
}
/// Set spawned-child registry state while preserving active pointer and manifest snapshot.
fn set_spawned_children(
&self,
@@ -1087,13 +1063,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
});
})
.unwrap();
store
.set_active(
@@ -1117,7 +1093,7 @@ mod tests {
}
#[test]
fn new_active_session_clears_prior_session_attribution_state() {
fn generic_active_update_does_not_interpret_host_attribution() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let old_session_id = crate::new_session_id();
@@ -1129,13 +1105,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
});
})
.unwrap();
store
@@ -1151,7 +1127,9 @@ mod tests {
restored.active,
Some(WorkerActiveSegmentRef::pending_segment(new_session_id))
);
assert!(restored.subjektiv_session_attribution.is_none());
assert!(
matches!(restored.subjektiv_session_attribution, Some(SubjektivSessionAttributionState::Confirmed { session_id, .. }) if session_id == old_session_id)
);
}
#[test]
+284 -23
View File
@@ -36,7 +36,10 @@ use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::SystemTime;
const SESSION_SCHEMA_VERSION: u32 = 3;
const SESSION_SCHEMA_VERSION: u32 = 6;
const SYSTEM_INPUT_SESSION_SCHEMA_VERSION: u32 = 5;
const COMPACTION_SESSION_SCHEMA_VERSION: u32 = 4;
const ANNOTATED_SESSION_SCHEMA_VERSION: u32 = 3;
const PREVIOUS_SESSION_SCHEMA_VERSION: u32 = 2;
const LEGACY_SESSION_SCHEMA_VERSION: u32 = 1;
const SESSION_FILE: &str = "session.json";
@@ -203,7 +206,15 @@ impl WorkerSessionStore {
/// Open the Session store rooted at `<worker-aggregate>/session`.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
Self::new_with_migration_writer(root.into(), atomic_write_bytes)
}
// Keep fault injection at the storage publication boundary, without global
// flags or a public alternate restore API. Production always uses atomic writes.
fn new_with_migration_writer(
root: PathBuf,
write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<Self, StoreError> {
fs::create_dir_all(root.join(SEGMENTS_DIR))?;
let session_id = match fs::read(root.join(SESSION_FILE)) {
Ok(bytes) => {
@@ -212,11 +223,16 @@ impl WorkerSessionStore {
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_logs(&root)?;
}
PREVIOUS_SESSION_SCHEMA_VERSION | LEGACY_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_v3(
&root,
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths(&root)?,
manifest.session_id,
manifest.schema_version,
write_segment,
)?;
manifest.schema_version = SESSION_SCHEMA_VERSION;
atomic_write_json(&root.join(SESSION_FILE), &manifest)?;
@@ -243,6 +259,33 @@ impl WorkerSessionStore {
})
}
/// Validate an existing Session's conversion without publishing any bytes.
/// Hosts can isolate unreadable archives before admitting a staged migration;
/// the versioned decoder and conversion checks remain owned by this store.
pub fn validate_migration(root: &Path) -> Result<(), StoreError> {
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(root.join(SESSION_FILE))?)?;
match manifest.schema_version {
SESSION_SCHEMA_VERSION => validate_canonical_segment_logs(root),
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => migrate_segment_logs_to_current(
segment_log_paths(root)?,
manifest.session_id,
manifest.schema_version,
|_, _| Ok(()),
),
version => Err(StoreError::Corrupt {
line: 0,
message: format!(
"unsupported Worker Session schema version {version}, expected {SESSION_SCHEMA_VERSION}"
),
}),
}
}
/// Open an already-retained Session without creating directories or migrating
/// persisted data. Observation paths must never mutate retained state.
pub fn open_read_only(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
@@ -739,6 +782,7 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_log_entry(&self.log_path(segment_id), entry)
@@ -797,6 +841,9 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let mut content = Vec::new();
@@ -1107,7 +1154,12 @@ fn open_retained_file(path: &Path) -> std::io::Result<File> {
}
fn segment_log_paths(root: &Path) -> Result<Vec<(SegmentId, PathBuf)>, StoreError> {
let segments = root.join(SEGMENTS_DIR);
segment_log_paths_in_directory(&root.join(SEGMENTS_DIR))
}
fn segment_log_paths_in_directory(
segments: &Path,
) -> Result<Vec<(SegmentId, PathBuf)>, StoreError> {
if !segments.exists() {
return Ok(Vec::new());
}
@@ -1150,21 +1202,23 @@ fn segment_log_paths(root: &Path) -> Result<Vec<(SegmentId, PathBuf)>, StoreErro
Ok(paths)
}
fn migrate_segment_logs_to_v3(
root: &Path,
fn migrate_segment_logs_to_current(
paths: Vec<(SegmentId, PathBuf)>,
session_id: SessionId,
source_schema_version: u32,
mut write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<(), StoreError> {
struct MigrationPlan {
path: PathBuf,
source: Vec<u8>,
entries: Vec<LogEntry>,
output: Vec<u8>,
}
// Phase 1 is strictly read-only. Every segment must parse and canonicalize
// successfully before the first authoritative byte is replaced.
let mut plans = Vec::new();
for (segment_id, path) in segment_log_paths(root)? {
for (segment_id, path) in paths {
let source = fs::read(&path)?;
let canonical = parse_legacy_jsonl(source_schema_version, session_id, segment_id, &source)
.map_err(|error| StoreError::Corrupt {
@@ -1174,17 +1228,32 @@ fn migrate_segment_logs_to_v3(
path.display()
),
})?;
let mut output = Vec::new();
for entry in canonical {
serde_json::to_writer(&mut output, &entry)?;
output.push(b'\n');
}
plans.push(MigrationPlan {
path,
source,
output,
entries: canonical,
output: Vec::new(),
});
}
// Resolve trusted creation boundaries first, then update inherited copies by
// logical identity too (fork/rewind seeds retain the same entry id).
let summaries = if source_schema_version < COMPACTION_SESSION_SCHEMA_VERSION {
crate::legacy_session_log::compaction_summary_ids(
plans.iter().flat_map(|plan| plan.entries.iter()),
)
} else {
// v4 already made the explicit origin migration. Preserve its origins.
Default::default()
};
for plan in &mut plans {
crate::legacy_session_log::migrate_compaction_origins(&mut plan.entries, &summaries);
for mut entry in std::mem::take(&mut plan.entries) {
crate::legacy_session_log::migrate_system_inputs(&mut entry)?;
entry.validate_input_metadata()?;
serde_json::to_writer(&mut plan.output, &entry)?;
plan.output.push(b'\n');
}
}
// Fence the complete preflight snapshot before starting phase 2. Session
// open is the exclusive restore boundary; this additionally fails closed
@@ -1202,14 +1271,56 @@ fn migrate_segment_logs_to_v3(
}
for plan in plans {
atomic_write_bytes(&plan.path, &plan.output)?;
write_segment(&plan.path, &plan.output)?;
}
Ok(())
}
/// Upgrade the standalone per-Session layout using the same persisted format
/// migration as Worker aggregates. Called only at writable Host store startup.
pub(crate) fn migrate_fs_session(root: &Path, session_id: SessionId) -> Result<(), StoreError> {
let marker = root.join("session-schema.json");
let version: u32 = match fs::read(&marker) {
Ok(bytes) => serde_json::from_slice(&bytes)?,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
ANNOTATED_SESSION_SCHEMA_VERSION
}
Err(error) => return Err(error.into()),
};
match version {
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_paths(segment_log_paths_in_directory(root)?)
}
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths_in_directory(root)?,
session_id,
version,
atomic_write_bytes,
)?;
atomic_write_json(&marker, &SESSION_SCHEMA_VERSION)
}
_ => Err(StoreError::Corrupt {
line: 0,
message: format!("unsupported standalone Session schema {version}"),
}),
}
}
fn validate_canonical_segment_logs(root: &Path) -> Result<(), StoreError> {
for (_, path) in segment_log_paths(root)? {
let _: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
validate_canonical_segment_paths(segment_log_paths(root)?)
}
fn validate_canonical_segment_paths(paths: Vec<(SegmentId, PathBuf)>) -> Result<(), StoreError> {
for (_, path) in paths {
let entries: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
for entry in entries {
entry.validate_input_metadata()?;
}
}
Ok(())
}
@@ -1223,17 +1334,26 @@ fn parse_legacy_jsonl(
let text = std::str::from_utf8(bytes).map_err(|error| {
serde_json::Error::io(std::io::Error::new(std::io::ErrorKind::InvalidData, error))
})?;
text.lines()
let mut offset = 0_u64;
text.split_inclusive('\n')
.enumerate()
.filter(|(_, line)| !line.trim().is_empty())
.map(|(line_index, line)| {
crate::legacy_session_log::decode_entry(
.filter_map(|(line_index, line)| {
let start = offset;
offset += line.len() as u64;
(!line.trim().is_empty()).then_some((line_index, line, start))
})
.map(|(line_index, line, start)| {
let mut entry = crate::legacy_session_log::decode_entry(
schema_version,
line,
session_id,
segment_id,
line_index,
)
)?;
// Byte offsets change during migration. Persist identities previously
// synthesized from the OLD offsets before rewriting any record.
entry.ensure_legacy_session_entry_id(session_id, segment_id, start);
Ok(entry)
})
.collect()
}
@@ -1380,6 +1500,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -1967,6 +2088,146 @@ mod tests {
);
}
#[test]
fn v5_migration_retries_after_second_file_publication_fails() {
fn identity_and_body(entries: &[LogEntry]) -> Vec<serde_json::Value> {
entries
.iter()
.flat_map(|record| match record {
LogEntry::AnnotatedSystemItem { entry, .. } => vec![entry.to_history_entry()],
LogEntry::AnnotatedUserInput { history, .. } => history.clone(),
_ => panic!("unexpected record in fixed migration fixture"),
})
.map(|entry| {
serde_json::json!({
"entry_id": entry.metadata.entry_id,
"origin": entry.metadata.origin,
"derivation": entry.metadata.derivation,
"item": entry.item,
})
})
.collect()
}
const SOURCE: &str =
include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl");
let root = tempfile::tempdir().unwrap();
let session_id = new_session_id();
let first_id = SegmentId::from_u128(1);
let second_id = SegmentId::from_u128(2);
fs::create_dir_all(root.path().join(SEGMENTS_DIR)).unwrap();
let marker_path = root.path().join(SESSION_FILE);
atomic_write_json(
&marker_path,
&SessionManifest {
schema_version: COMPACTION_SESSION_SCHEMA_VERSION,
session_id,
},
)
.unwrap();
let old_marker = fs::read(&marker_path).unwrap();
let first_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{first_id}.jsonl"));
let second_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{second_id}.jsonl"));
let second_source = SOURCE.replace("-id", "-second-id");
fs::write(&first_path, SOURCE).unwrap();
fs::write(&second_path, &second_source).unwrap();
let old_first = fs::read(&first_path).unwrap();
let old_second = fs::read(&second_path).unwrap();
let first_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, first_id, &old_first).unwrap());
let second_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, second_id, &old_second).unwrap());
let mut writes = 0;
let mut first_published = None;
let failure = WorkerSessionStore::new_with_migration_writer(
root.path().to_path_buf(),
|path, bytes| {
writes += 1;
match writes {
1 => {
assert_eq!(path, first_path.as_path());
// Both files are still old when the first publication starts.
assert_eq!(fs::read(&first_path).unwrap(), old_first);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
atomic_write_bytes(path, bytes)?;
first_published = Some(bytes.to_vec());
Ok(())
}
2 => {
assert_eq!(path, second_path.as_path());
assert_eq!(
fs::read(&first_path).unwrap(),
*first_published.as_ref().unwrap()
);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
// Fail only this atomic-write boundary, after complete
// preflight and a real successful first-file publication.
Err(std::io::Error::other("injected second migration write failure").into())
}
_ => panic!("publication continued after its write failure"),
}
},
);
let error = match failure {
Ok(_) => panic!("the injected publication failure must reject store open"),
Err(error) => error,
};
assert!(
error
.to_string()
.contains("injected second migration write failure")
);
assert_eq!(writes, 2);
let first_published = first_published.unwrap();
assert_ne!(first_published, old_first);
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
assert_eq!(fs::read(&marker_path).unwrap(), old_marker);
let updated_first: Vec<LogEntry> = parse_jsonl(&first_published).unwrap();
for entry in &updated_first {
entry.validate_input_metadata().unwrap();
}
assert!(matches!(
&updated_first[0],
LogEntry::AnnotatedSystemItem {
entry: crate::LoggedSystemHistoryEntry {
item: crate::SystemItem::Input { .. },
..
},
..
}
));
assert_eq!(identity_and_body(&updated_first), first_identity);
// Retry via the public production constructor, without any injection or
// hand-built mixed fixture. The old marker drives the same migration.
let store = WorkerSessionStore::new(root.path()).unwrap();
let first = store.read_all(session_id, first_id).unwrap();
let second = store.read_all(session_id, second_id).unwrap();
assert_eq!(identity_and_body(&first), first_identity);
assert_eq!(identity_and_body(&second), second_identity);
for entry in first.iter().chain(second.iter()) {
entry.validate_input_metadata().unwrap();
}
assert_eq!(fs::read(&first_path).unwrap(), first_published);
let second_published = fs::read(&second_path).unwrap();
assert_ne!(second_published, old_second);
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(&marker_path).unwrap()).unwrap();
assert_eq!(manifest.schema_version, SESSION_SCHEMA_VERSION);
drop(store);
WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), second_published);
}
#[test]
fn current_jsonl_requires_annotations_across_append_rewrite_and_reopen() {
let root = tempfile::tempdir().unwrap();
@@ -0,0 +1,711 @@
//! Public consumer boundary: real capture adapters, checked DTO codec and pure readers.
//! No Worker, persistence, live Session provider, or substituted capture implementation.
use agen::{HistoryEntry, Item};
use protocol::{
SessionEntryProvenance, SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData,
};
use session_store::capture::*;
use session_store::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputObservationUse, LoggedHistoryDerivation, LoggedSessionHistoryEntryId,
LoggedSessionHistoryMetadata, LoggedSessionHistoryOrigin, PublicSystemItem,
PublicSystemPresentation,
};
fn history(
id: &str,
item: Item,
input_metadata: Option<HistoryInputMetadata>,
) -> HistoryEntry<LoggedSessionHistoryMetadata> {
HistoryEntry::new(
item,
LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::LocalHumanInput,
derivation: Some(LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("earlier-entry".into())],
}),
input_metadata,
},
)
}
fn usage(recording: bool, memory: bool, observation: bool) -> HistoryInputUsePolicy {
HistoryInputUsePolicy {
recording_material: if recording {
InputEvidenceUse::Include
} else {
InputEvidenceUse::Exclude
},
memory_evidence: if memory {
InputEvidenceUse::Include
} else {
InputEvidenceUse::Exclude
},
observation: if observation {
InputObservationUse::Readable
} else {
InputObservationUse::Hidden
},
compaction: InputCompactionUse::Retain,
}
}
fn reference(capture: &SessionCapture, id: &str) -> ScopedSessionEntryRef {
capture
.scoped_ref(&SessionEntryRef::parse(id).unwrap())
.unwrap()
}
#[test]
fn public_consumer_roundtrip_retains_body_original_source_order_origin_and_independent_policy() {
let entries = vec![
history(
"recording",
Item::user_message("record-only"),
Some(HistoryInputMetadata::new(usage(true, false, false), None, None).unwrap()),
),
history(
"memory",
Item::user_message("memory-only"),
Some(HistoryInputMetadata::new(usage(false, true, false), None, None).unwrap()),
),
history(
"observation",
Item::user_message("observation-only"),
Some(HistoryInputMetadata::new(usage(false, false, true), None, None).unwrap()),
),
];
let capture = SessionCapture::from_complete_history("synthetic-session", "selected", entries);
let mut dto = capture.to_dto();
// The checked boundary also accepts entries retained from different original segments.
dto.entries[0].source.segment_id = "original-a".into();
dto.entries[0].source.entry_index = 80;
dto.entries[1].source.segment_id = "original-b".into();
dto.entries[1].source.entry_index = 3;
let capture = SessionCapture::try_from_dto(dto.clone(), CaptureLimits::default()).unwrap();
let encoded = capture.encode(CaptureLimits::default()).unwrap();
let restored = SessionCapture::decode(&encoded, CaptureLimits::default()).unwrap();
assert_eq!(restored.to_dto(), dto);
assert_eq!(restored.capture_id(), capture.capture_id());
for (id, allowed) in [
("Erecording", CapturePurpose::Recording),
("Ememory", CapturePurpose::MemoryEvidence),
("Eobservation", CapturePurpose::Observation),
] {
let reference = reference(&capture, id);
assert_eq!(
restored.resolve_ref(&reference, allowed).unwrap(),
capture.resolve_ref(&reference, allowed).unwrap()
);
for purpose in [
CapturePurpose::Recording,
CapturePurpose::MemoryEvidence,
CapturePurpose::Observation,
] {
assert_eq!(
restored.resolve_ref(&reference, purpose).is_ok(),
purpose == allowed
);
}
}
assert_eq!(restored.overview().len(), 1);
assert_eq!(restored.search(&SearchOptions::default()).len(), 1);
assert!(restored.evidence_for("Erecording").is_none());
let evidence = restored.evidence_for("Ememory").unwrap();
assert_eq!(evidence.segment_id, "original-b");
assert_eq!(evidence.entry_range, [3, 3]);
assert!(evidence.excerpt.contains("memory-only"));
let recorded = restored
.slice(CapturePurpose::Recording, None, 10, 16_000)
.unwrap();
assert_eq!(recorded.slice().items.len(), 1);
assert_eq!(recorded.slice().items[0].text, "record-only");
for (purpose, expected) in [
(CapturePurpose::MemoryEvidence, "memory-only"),
(CapturePurpose::Observation, "observation-only"),
] {
let slice = restored.slice(purpose, None, 10, 16_000).unwrap();
assert_eq!(slice.slice().items.len(), 1);
assert_eq!(slice.slice().items[0].text, expected);
assert!(
restored
.read_raw(
&reference(&restored, "Erecording"),
purpose,
RawField::MessagePart(0),
0,
100
)
.is_err()
);
}
}
#[test]
fn common_data_drops_reasoning_raw_system_and_private_payload_even_when_unindexed() {
let payload = FeatureInputPayload::encode(
"capture-test",
1,
&serde_json::json!({"secret": "PRIVATE-PAYLOAD"}),
)
.unwrap();
let public = PublicSystemItem {
kind: "notice".into(),
body: "public notice".into(),
presentation: PublicSystemPresentation::Ignored,
};
let system =
HistoryInputMetadata::new(usage(true, true, true), Some(payload.clone()), Some(public))
.unwrap();
let private = HistoryInputMetadata::new(usage(true, true, true), Some(payload), None).unwrap();
let capture = SessionCapture::from_complete_history(
"s",
"segment",
vec![
history("raw", Item::system_message("RAW-SYSTEM-PROMPT"), None),
history("reasoning", Item::reasoning("HIDDEN-REASONING"), None),
history(
"private",
Item::system_message("PRIVATE-SYSTEM-BODY"),
Some(private),
),
history(
"public",
Item::system_message("MODEL-SYSTEM-BODY"),
Some(system),
),
history("ordinary", Item::user_message("ordinary"), None),
],
);
let encoded = String::from_utf8(capture.encode(CaptureLimits::default()).unwrap()).unwrap();
let retained_debug = format!("{capture:?}");
for secret in [
"RAW-SYSTEM-PROMPT",
"HIDDEN-REASONING",
"PRIVATE-SYSTEM-BODY",
"MODEL-SYSTEM-BODY",
"PRIVATE-PAYLOAD",
] {
assert!(!encoded.contains(secret));
assert!(!retained_debug.contains(secret));
}
assert_eq!(capture.to_dto().entries.len(), 2);
assert!(encoded.contains("public notice"));
// Filtered entries do not renumber the public source positions.
assert_eq!(capture.to_dto().entries[0].order, 3);
assert_eq!(
capture.view_order_for_ref(&SessionEntryRef::parse("Eordinary").unwrap()),
Some(4)
);
}
#[test]
fn protocol_projection_does_not_invent_recording_permission() {
let capture = SessionCapture::from_session_snapshot(
"s",
"seg",
SessionSnapshot {
pending_submissions: Default::default(),
entries: vec![SessionSnapshotEntry {
entry_id: "projected".into(),
timestamp: 0,
provenance: SessionEntryProvenance::HumanInput,
derived_from: vec!["source".into()],
use_policy: Some(protocol::SessionEntryUsePolicy {
observation_readable: true,
memory_evidence: false,
}),
data: SessionSnapshotEntryData::Message {
role: protocol::SessionMessageRole::User,
content: vec![protocol::SessionContentPart::Text {
text: "projected".into(),
}],
},
}],
},
);
let reference = reference(&capture, "Eprojected");
assert!(matches!(
capture.resolve_ref(&reference, CapturePurpose::Recording),
Err(CaptureError::PolicyDenied)
));
assert!(
capture
.resolve_ref(&reference, CapturePurpose::Observation)
.is_ok()
);
assert!(capture.evidence_for("Eprojected").is_none());
}
#[test]
fn malformed_dtos_and_oversized_representations_are_rejected() {
let capture = SessionCapture::new("s", "seg", vec![Item::user_message("body")]);
let original = serde_json::to_value(capture.to_dto()).unwrap();
let mut invalids = Vec::new();
let mut value = original.clone();
value["version"] = 999.into();
invalids.push(value);
let mut value = original.clone();
value["unexpected"] = true.into();
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["policy"]["recording_material"] = "include".into();
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["policy"]
.as_object_mut()
.unwrap()
.remove("memory_evidence");
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["body"] = serde_json::json!({"kind":"reasoning","text":"secret"});
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["body"]["feature_payload"] = serde_json::json!({"private":true});
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["body"]["role"] = "system".into();
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["source"]["unknown"] = true.into();
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["source"]["history_origin"] =
serde_json::json!({"kind":"legacy_unknown","unknown":true});
invalids.push(value);
let mut value = original.clone();
value["entries"][0]["entry_id"] = "bad/id".into();
invalids.push(value);
let mut value = original.clone();
value["entries"]
.as_array_mut()
.unwrap()
.push(original["entries"][0].clone());
invalids.push(value);
for invalid in invalids {
let bytes = serde_json::to_vec(&invalid).unwrap();
assert!(
SessionCapture::decode(&bytes, CaptureLimits::default()).is_err(),
"accepted {invalid}"
);
}
let bytes = capture.encode(CaptureLimits::default()).unwrap();
assert!(
SessionCapture::decode(
&bytes,
CaptureLimits {
max_bytes: bytes.len() - 1,
..CaptureLimits::default()
}
)
.is_err()
);
assert!(
capture
.encode(CaptureLimits {
max_entries: 0,
..CaptureLimits::default()
})
.is_err()
);
assert!(
capture
.encode(CaptureLimits {
max_text_bytes: 3,
..CaptureLimits::default()
})
.is_err()
);
let mut reversed = capture.to_dto();
reversed.view_entry_count = 2;
let mut second = reversed.entries[0].clone();
second.entry_id = "second".into();
second.source.entry_index = 4;
second.order = 0;
reversed.entries[0].order = 1;
reversed.entries.push(second);
assert!(SessionCapture::try_from_dto(reversed, CaptureLimits::default()).is_err());
let mut duplicate_source = capture.to_dto();
duplicate_source.view_entry_count = 2;
let mut second = duplicate_source.entries[0].clone();
second.entry_id = "distinct-id".into();
second.order = 1;
duplicate_source.entries.push(second);
assert!(matches!(
SessionCapture::try_from_dto(duplicate_source, CaptureLimits::default()),
Err(CaptureError::Invalid("duplicate source position"))
));
}
#[test]
fn raw_utf8_tool_input_and_result_are_lossless_without_preview_markers() {
let arguments = "\"quoted\"\\path\n日本語🙂".repeat(300);
let result = "result\n\t\\\"λ🙂".repeat(400);
let capture = SessionCapture::new(
"s",
"seg",
vec![
Item::tool_call("call", "tool", arguments.clone()),
Item::tool_result_item("call", "summary", Some(result.clone()), false),
],
);
for (id, field, expected) in [
("E00000000", RawField::ToolArguments, arguments),
("E00000001", RawField::ToolContent, result),
] {
let reference = reference(&capture, id);
let mut offset = 0;
let mut actual = String::new();
loop {
let part = capture
.read_raw(&reference, CapturePurpose::Observation, field, offset, 17)
.unwrap();
assert!(part.text.len() <= 17);
assert_eq!(part.offset, offset);
actual.push_str(&part.text);
match part.next_offset {
Some(next) => {
assert!(next > offset);
offset = next;
}
None => break,
}
}
assert_eq!(actual, expected);
}
let unicode = SessionCapture::new("s", "seg", vec![Item::tool_call("call", "tool", "🙂")]);
let reference = reference(&unicode, "E00000000");
assert!(matches!(
unicode.read_raw(
&reference,
CapturePurpose::Observation,
RawField::ToolArguments,
1,
8
),
Err(CaptureError::InvalidRange)
));
assert!(matches!(
unicode.read_raw(
&reference,
CapturePurpose::Observation,
RawField::ToolArguments,
0,
3
),
Err(CaptureError::BudgetTooSmall)
));
assert!(matches!(
unicode.read_raw(
&reference,
CapturePurpose::Observation,
RawField::ToolArguments,
5,
8
),
Err(CaptureError::InvalidRange)
));
}
#[test]
fn escaped_json_slice_budget_includes_metadata_and_bound_continuation() {
let body = "\"\\\n\t🙂日本語".repeat(500);
let capture = SessionCapture::new(
"synthetic-session",
"segment",
vec![Item::tool_call("call", "tool", body.clone())],
);
let other_session = SessionCapture::new(
"other-session",
"segment",
vec![Item::tool_call("call", "tool", body.clone())],
);
let changed = SessionCapture::new(
"synthetic-session",
"segment",
vec![Item::tool_call("call", "tool", "changed")],
);
let mut cursor = None;
let mut actual = String::new();
let budget = 1400;
loop {
let part = capture
.slice(CapturePurpose::Recording, cursor.as_ref(), 1, budget)
.unwrap();
assert!(part.encoded_len() <= budget);
assert_eq!(part.encoded_len(), part.as_json().len());
assert_eq!(serde_json::to_vec(part.slice()).unwrap(), part.as_json());
assert_eq!(part.slice().items.len(), 1);
let item = &part.slice().items[0];
assert_eq!(item.offset, actual.len());
actual.push_str(&item.text);
cursor = part.slice().next_cursor.clone();
if let Some(cursor) = &cursor {
assert!(matches!(
other_session.slice(CapturePurpose::Recording, Some(cursor), 1, budget),
Err(CaptureError::ForeignReference)
));
assert!(matches!(
changed.slice(CapturePurpose::Recording, Some(cursor), 1, budget),
Err(CaptureError::ForeignReference)
));
assert!(matches!(
capture.slice(CapturePurpose::Observation, Some(cursor), 1, budget),
Err(CaptureError::ForeignReference)
));
} else {
break;
}
}
assert_eq!(actual, body);
assert!(matches!(
capture.slice(CapturePurpose::Recording, None, 1, 8),
Err(CaptureError::BudgetTooSmall)
));
assert!(matches!(
capture.slice(CapturePurpose::Recording, None, 0, budget),
Err(CaptureError::BudgetTooSmall)
));
}
#[test]
fn snapshot_immutability_and_scoped_operations_reject_foreign_or_unknown_refs_and_bad_ranges() {
let mut source = vec![
Item::user_message("before"),
Item::assistant_message("answer"),
];
let capture = SessionCapture::new("s", "seg", source.clone());
let fixed_bytes = capture.encode(CaptureLimits::default()).unwrap();
let first = reference(&capture, "E00000000");
let second = reference(&capture, "E00000001");
source.push(Item::user_message("appended"));
source[0] = Item::user_message("compacted");
source.clear();
assert_eq!(
capture.encode(CaptureLimits::default()).unwrap(),
fixed_bytes
);
assert_eq!(
capture.clone().encode(CaptureLimits::default()).unwrap(),
fixed_bytes
);
let foreign = SessionCapture::new(
"different",
"seg",
vec![
Item::user_message("before"),
Item::assistant_message("answer"),
],
);
assert!(matches!(
foreign.resolve_ref(&first, CapturePurpose::Observation),
Err(CaptureError::ForeignReference)
));
assert!(matches!(
capture.checked_range(&second, &first, CapturePurpose::Observation),
Err(CaptureError::InvalidRange)
));
assert_eq!(
capture
.checked_range(&first, &second, CapturePurpose::Observation)
.unwrap()
.len(),
2
);
let unknown = SessionEntryRef::parse("E99999999").unwrap();
assert_eq!(capture.view_order_for_ref(&unknown), None);
assert!(matches!(
capture.scoped_ref(&unknown),
Err(CaptureError::UnknownReference)
));
assert!(
capture
.search_checked(&SearchOptions {
from: Some(unknown.clone()),
..Default::default()
})
.is_err()
);
assert!(
capture
.search(&SearchOptions {
through: Some(unknown),
..Default::default()
})
.is_empty()
);
assert!(
capture
.read_checked(ReadSelector::EntryRange([1, 0]), ReadOptions::default())
.is_err()
);
assert!(
capture
.read_checked(ReadSelector::EntryRange([0, 99]), ReadOptions::default())
.is_err()
);
}
#[test]
fn slices_preserve_empty_and_long_result_fields_under_item_and_metadata_budgets() {
let summary = "summary\n\"🙂".repeat(100);
let content = "content\t\\日本語".repeat(200);
let capture = SessionCapture::new(
"s",
"seg",
vec![
Item::tool_result_item("first", summary.clone(), Some(content.clone()), true),
Item::tool_result_item("empty", "", Some(String::new()), false),
],
);
let mut cursor = None;
let mut fields: Vec<(String, RawField, String)> = Vec::new();
loop {
let part = capture
.slice(CapturePurpose::Recording, cursor.as_ref(), 2, 1500)
.unwrap();
assert!(part.slice().items.len() <= 2);
assert!(serde_json::to_vec(part.slice()).unwrap().len() <= 1500);
for item in &part.slice().items {
let id = item.reference.entry_ref.to_string();
if let Some((last_id, last_field, text)) = fields.last_mut() {
if *last_id == id && *last_field == item.field {
assert_eq!(text.len(), item.offset);
text.push_str(&item.text);
continue;
}
}
assert_eq!(item.offset, 0);
fields.push((id, item.field, item.text.clone()));
}
cursor = part.slice().next_cursor.clone();
if cursor.is_none() {
break;
}
}
assert_eq!(
fields,
vec![
("E00000000".into(), RawField::ToolSummary, summary),
("E00000000".into(), RawField::ToolContent, content),
("E00000001".into(), RawField::ToolSummary, String::new()),
("E00000001".into(), RawField::ToolContent, String::new()),
]
);
let metadata_heavy = SessionCapture::new(
"s",
"seg",
vec![Item::tool_call("call", "name".repeat(200), "x")],
);
assert!(matches!(
metadata_heavy.slice(CapturePurpose::Recording, None, 1, 700),
Err(CaptureError::BudgetTooSmall)
));
}
#[test]
fn evidence_uses_original_segment_positions_after_roundtrip_and_view_selection() {
let full = SessionCapture::new(
"s",
"selected",
vec![
Item::user_message("first"),
Item::assistant_message("second"),
],
);
let mut dto = full.to_dto();
dto.entries[0].source.segment_id = "original-a".into();
dto.entries[0].source.entry_index = 80;
dto.entries[1].source.segment_id = "original-b".into();
dto.entries[1].source.entry_index = 3;
let encoded = dto.encode(CaptureLimits::default()).unwrap();
let restored = SessionCapture::decode(&encoded, CaptureLimits::default()).unwrap();
for (id, segment, index, order) in [
("E00000000", "original-a", 80, 0),
("E00000001", "original-b", 3, 1),
] {
let evidence = restored.evidence_for(id).unwrap();
assert_eq!(evidence.segment_id, segment);
assert_eq!(evidence.entry_range, [index; 2]);
let read = restored.read(ReadSelector::Id(id), ReadOptions::default());
assert_eq!(read.entries[0].entry_range, [order; 2]);
}
let selected = restored.select_order_range(1..2).unwrap();
assert_eq!(selected.overview()[0].entry_range, [0, 0]);
let evidence = selected.evidence_for("E00000001").unwrap();
assert_eq!(evidence.segment_id, "original-b");
assert_eq!(evidence.entry_range, [3, 3]);
let roundtrip = SessionCapture::decode(
&selected.encode(CaptureLimits::default()).unwrap(),
CaptureLimits::default(),
)
.unwrap();
assert_eq!(roundtrip.to_dto(), selected.to_dto());
assert_eq!(
roundtrip.evidence_for("E00000001").unwrap().entry_range,
[3, 3]
);
}
#[test]
fn fixed_history_selection_preserves_original_coordinates_and_checks_private_position_bounds() {
let history = vec![
history("old", Item::user_message("old"), None),
history("hidden", Item::reasoning("PRIVATE"), None),
history("selected", Item::user_message("selected"), None),
history("private-tail", Item::system_message("PRIVATE"), None),
];
let full = SessionCapture::from_complete_history("s", "seg", history);
let suffix = full.select_order_range(1..4).unwrap();
let original = full
.entries()
.find(|entry| entry.entry_id == "selected")
.unwrap();
let selected = suffix.entries().next().unwrap();
assert_eq!(selected.entry_id, original.entry_id);
assert_eq!(selected.source, original.source);
assert_eq!(selected.body, original.body);
assert_eq!(selected.policy, original.policy);
assert!(suffix.evidence_for("Eold").is_none());
assert_eq!(suffix.search(&SearchOptions::default()).len(), 1);
assert_eq!(
suffix.view_order_for_ref(&SessionEntryRef::parse("Eselected").unwrap()),
Some(1)
);
assert_eq!(selected.source.entry_index, 2);
assert_eq!(selected.order, 1);
assert_eq!(suffix.to_dto().view_entry_count, 3);
assert_eq!(
suffix.evidence_for("Eselected").unwrap().entry_range,
[2, 2]
);
let nested = suffix.select_order_range(1..2).unwrap();
assert_eq!(nested.entries().next().unwrap().order, 0);
assert_eq!(nested.entries().next().unwrap().source, original.source);
let restored = SessionCapture::decode(
&suffix.encode(CaptureLimits::default()).unwrap(),
CaptureLimits::default(),
)
.unwrap();
assert_eq!(restored.to_dto(), suffix.to_dto());
assert!(full.select_order_range(4..5).is_err());
let reversed = std::ops::Range { start: 3, end: 2 };
assert!(full.select_order_range(reversed).is_err());
let hidden_only = full.select_order_range(3..4).unwrap();
assert_eq!(hidden_only.entries().len(), 0);
assert_eq!(hidden_only.to_dto().view_entry_count, 1);
let mut invalid = full.to_dto();
invalid.view_entry_count = 2; // the selected entry's view order is outside this extent
assert!(SessionCapture::try_from_dto(invalid, CaptureLimits::default()).is_err());
assert_eq!(
full.retain_entries(|entry| entry.entry_id == "selected")
.entries()
.next()
.unwrap()
.source,
original.source
);
assert_eq!(full.to_dto().view_entry_count, 4);
}
+179 -2
View File
@@ -11,6 +11,7 @@ fn annotated(item: Item) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -62,6 +63,7 @@ fn round_trip_write_and_read() {
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 3200,
interrupted: false,
result: EngineResult::Finished,
@@ -157,9 +159,10 @@ fn create_segment_replacement_keeps_open_old_log_and_commits_complete_new_log()
assert_eq!(reopened.read_entry_count(sid, segid).unwrap(), new.len());
assert_eq!(
std::fs::read_dir(path.parent().unwrap()).unwrap().count(),
1,
"successful replacement must not leave temporary files"
2,
"successful replacement leaves only the segment and Session schema marker"
);
assert!(path.parent().unwrap().join("session-schema.json").is_file());
}
#[test]
@@ -419,3 +422,177 @@ fn lookup_session_of_finds_owning_session() {
assert_eq!(store.lookup_session_of(segid).unwrap(), Some(sid));
}
// Storage/projection component test: both production layouts, temporary originals,
// no Feature, model, runtime or Host authority is substituted into this boundary.
#[test]
fn generic_system_append_and_fork_preserve_metadata_without_public_payload_leak() {
use session_store::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputObservationUse, LoggedHistoryDerivation, LoggedSystemHistoryEntry,
PublicSystemItem, PublicSystemPresentation, SystemInputEnvelope, WorkerSessionStore,
};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let fork = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
store
.create_segment(sid, segment, &[nil_session_start(1, sid)])
.unwrap();
let entry = LoggedSystemHistoryEntry::from_input(
LoggedSessionHistoryOrigin::Provided {
producer: "new-feature".into(),
},
Some(LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source-entry".into())],
}),
SystemInputEnvelope::new(
"private model body",
HistoryInputMetadata::new(
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
},
Some(
FeatureInputPayload::encode(
"new-feature:context",
101,
&serde_json::json!({"private": "secret-detail"}),
)
.unwrap(),
),
Some(PublicSystemItem {
kind: "context".into(),
body: "explicit public body".into(),
presentation: PublicSystemPresentation::Context,
}),
)
.unwrap(),
)
.unwrap(),
);
store
.append(
sid,
segment,
&LogEntry::AnnotatedSystemItem {
ts: 2,
entry: entry.clone(),
extensions: vec![],
},
)
.unwrap();
let saved = store.read_all(sid, segment).unwrap();
let LogEntry::AnnotatedSystemItem {
entry: restored, ..
} = &saved[1]
else {
panic!("missing system input")
};
assert_eq!(restored.to_history_entry(), entry.to_history_entry());
assert_eq!(
collect_state(&saved).history[0].as_text(),
Some("private model body")
);
let mut fork_seed = nil_session_start(3, sid);
if let LogEntry::AnnotatedSegmentStart {
history,
forked_from,
..
} = &mut fork_seed
{
history.push(restored.to_history_entry());
*forked_from = Some(session_store::SegmentOrigin {
segment_id: segment,
at_turn_index: 0,
});
}
store.create_segment(sid, fork, &[fork_seed]).unwrap();
let fork_saved = store.read_all(sid, fork).unwrap();
let LogEntry::AnnotatedSegmentStart { history, .. } = &fork_saved[0] else {
unreachable!()
};
assert_eq!(history, &[entry.to_history_entry()]);
let original = session_store::public_snapshot::project_session_snapshot(sid, &saved);
let inherited = session_store::public_snapshot::project_session_snapshot(sid, &fork_saved);
let public = serde_json::to_string(&original.entries).unwrap();
assert!(public.contains("explicit public body"));
for private in ["private model body", "secret-detail", "new-feature:context"] {
assert!(!public.contains(private), "private field leaked: {private}");
}
assert_eq!(original.entries.len(), 1);
assert_eq!(inherited.entries.len(), 1);
assert_eq!(original.entries[0].entry_id, inherited.entries[0].entry_id);
assert_eq!(
original.entries[0].derived_from,
inherited.entries[0].derived_from
);
assert_eq!(
original.entries[0].provenance,
inherited.entries[0].provenance
);
assert_eq!(
serde_json::to_value(&original.entries[0].data).unwrap(),
serde_json::to_value(&inherited.entries[0].data).unwrap()
);
}
}
#[test]
fn invalid_generic_system_input_rejects_append_create_and_external_json_without_changing_originals()
{
use session_store::{StoreError, SystemItem, WorkerSessionStore};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
let seed = nil_session_start(1, sid);
store
.create_segment(sid, segment, std::slice::from_ref(&seed))
.unwrap();
let path = if worker_layout {
dir.path().join("segments").join(format!("{segment}.jsonl"))
} else {
dir.path()
.join(sid.to_string())
.join(format!("{segment}.jsonl"))
};
let before = std::fs::read(&path).unwrap();
let invalid = LogEntry::AnnotatedSystemItem {
ts: 2,
extensions: vec![],
entry: session_store::LoggedSystemHistoryEntry {
item: SystemItem::Input {
body: "no policy".into(),
},
metadata: LoggedSessionHistoryMetadata::legacy_unknown(),
},
};
assert!(matches!(
store.append(sid, segment, &invalid),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(matches!(
store.create_segment(sid, segment, &[seed, invalid.clone()]),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(
serde_json::from_value::<LogEntry>(serde_json::to_value(invalid).unwrap()).is_err()
);
assert_eq!(std::fs::read(path).unwrap(), before);
assert_eq!(store.read_all(sid, segment).unwrap().len(), 1);
}
}
@@ -24,6 +24,7 @@ fn annotated(items: &[Item]) -> Vec<session_store::LoggedHistoryEntry> {
.map(|item| session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
+1 -1
View File
@@ -23,7 +23,7 @@ serde.workspace = true
serde_json.workspace = true
session-store.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["rt", "sync", "time"] }
tokio = { workspace = true, features = ["rt", "sync", "time", "net", "io-util", "io-std", "signal"] }
uuid = { workspace = true, features = ["v7"] }
worker.workspace = true
+2 -1
View File
@@ -4,7 +4,8 @@ A Host that starts existing Workers in the same process. Assembles interactive W
## Reference index
- [`host`] — bootstrap and confirmed shutdown.
- [`host`] — bootstrap and confirmed shutdown, with or without a TUI.
- [`unix`] — optional operator transport; disconnect does not release execution ownership.
- [`launch`] — Profile, cwd, and Subject validation before launch.
- [`store`] — Worker records and leases.
- [`jobs`] — Job execution and independent result and cleanup lifetimes.
+42 -23
View File
@@ -23,10 +23,7 @@ use session_store::{
CombinedStore, FsStore, FsWorkerStore, WorkerActiveSegmentRef, WorkerMetadataStore,
};
use thiserror::Error;
use worker::bootstrap::{
WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout, bash_output_dir_for_worker_id,
};
use worker::controller::WorkerControllerTransport;
use worker::bootstrap::{WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout};
use worker::ipc::protocol_session::{
WorkerProtocolSessionStreams, dispatch_worker_protocol_method, live_log_entry_event,
subscribe_worker_protocol_session,
@@ -216,7 +213,6 @@ impl StandaloneHost {
WorkerFilesystemAuthority::local(launch.cwd.clone(), launch.cwd.clone());
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
bootstrap_manifest,
@@ -224,11 +220,7 @@ impl StandaloneHost {
launch.prompt_catalog,
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerControllerTransport::InProcess,
WorkerBootstrapLayout::Direct { runtime_base },
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
@@ -428,7 +420,6 @@ impl StandaloneHost {
);
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
manifest,
@@ -436,11 +427,7 @@ impl StandaloneHost {
worker::PromptCatalogSource::builtins_only(),
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerControllerTransport::InProcess,
WorkerBootstrapLayout::Direct { runtime_base },
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
@@ -530,10 +517,7 @@ impl StandaloneHost {
/// Working events, committed session entries, alert snapshots, and the
/// initial history snapshot are merged behind the client boundary.
pub fn connect(&self) -> Client<InProcessSocket> {
let streams = subscribe_worker_protocol_session(&self.handle);
let (socket, peer) = InProcessSocket::pair();
tokio::spawn(run_protocol_session(self.handle.clone(), streams, peer));
Client::new(socket)
Client::new(connect_protocol(self.handle.clone()))
}
/// Logical Subject disconnection, not Worker shutdown or completion of its represented work.
@@ -575,6 +559,35 @@ impl StandaloneHost {
}
}
/// Run without a TUI until an admitted protocol Shutdown, controller exit, or process signal.
/// Client EOF only disconnects that client. All exits confirm the same production shutdown
/// and lease release as the in-process UI; an unconfirmed shutdown retains the lease.
pub async fn serve_unix(
self,
endpoint: crate::unix::UnixEndpoint,
) -> Result<(), crate::HeadlessError> {
let mut adapter = crate::unix::UnixAdapter::start(endpoint, self.handle.clone());
let signal = async {
let mut terminate =
tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())?;
tokio::select! {
result = tokio::signal::ctrl_c() => result,
_ = terminate.recv() => Ok(()),
}
};
let result = tokio::select! {
_ = self.handle.protocol_closed() => Ok(()),
result = signal => result,
result = adapter.ended() => result,
};
// Reclaim the public entry and every connection before releasing execution ownership.
let transport = adapter.shutdown().await;
self.shutdown().await?;
result?;
transport?;
Ok(())
}
pub fn with_shutdown_timeout(mut self, shutdown_timeout: Duration) -> Self {
self.shutdown_timeout = shutdown_timeout;
self
@@ -657,9 +670,8 @@ impl Drop for StandaloneHost {
}
}
// The in-process channel is owned by the local operator's Host, not an
// arbitrary socket or Backend account. decode_method discards wire provenance;
// only this boundary can stamp accepted local human input.
// Both in-process views and credential-checked Unix clients are admitted by this operator's
// Host. decode_method discards wire provenance; only this boundary stamps local human input.
fn authorize_local_input(method: protocol::Method) -> protocol::Method {
use protocol::{AuthenticatedInputSource, Method};
match method {
@@ -702,6 +714,13 @@ fn authorize_local_input(method: protocol::Method) -> protocol::Method {
}
}
pub(crate) fn connect_protocol(handle: worker::WorkerHandle) -> InProcessSocket {
let streams = subscribe_worker_protocol_session(&handle);
let (socket, peer) = InProcessSocket::pair();
tokio::spawn(run_protocol_session(handle, streams, peer));
socket
}
async fn run_protocol_session(
handle: worker::WorkerHandle,
streams: WorkerProtocolSessionStreams,
+529 -20
View File
@@ -25,7 +25,46 @@ use thiserror::Error;
#[path = "job_store/legacy_migrations.rs"]
mod legacy_migrations;
const SCHEMA_VERSION: i64 = 4;
const SCHEMA_VERSION: i64 = 5;
/// Availability of the immutable input body, independent of execution outcome.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JobInputDisposition {
Available,
Reclaimed,
Erased,
Invalidated,
}
impl JobInputDisposition {
pub fn as_str(self) -> &'static str {
match self {
Self::Available => "available",
Self::Reclaimed => "reclaimed",
Self::Erased => "erased",
Self::Invalidated => "invalidated",
}
}
}
/// Host-authorized retirement; erasure and legacy invalidation do not change outcomes.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum JobInputRetirementReason {
Completed,
ExplicitErasure,
LegacyInvalidation,
}
impl JobInputRetirementReason {
pub fn disposition(self) -> JobInputDisposition {
match self {
Self::Completed => JobInputDisposition::Reclaimed,
Self::ExplicitErasure => JobInputDisposition::Erased,
Self::LegacyInvalidation => JobInputDisposition::Invalidated,
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
@@ -67,7 +106,11 @@ impl JobAttemptState {
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct JobSnapshot {
/// Receipt metadata. After retirement `input` is null; use the verified loader to execute.
pub request: JobRequest,
pub input_digest: String,
pub intent_fingerprint: String,
pub input_disposition: JobInputDisposition,
/// Durable Host-issued domain delegation, not model input or Profile identity.
pub domain_grant: Option<Value>,
pub state: JobState,
@@ -77,14 +120,27 @@ pub struct JobSnapshot {
}
impl JobSnapshot {
/// Load only the original available body, checking both content and immutable intent.
pub fn load_executable_request(&self) -> Result<&JobRequest, JobStoreError> {
if self.input_disposition != JobInputDisposition::Available {
return Err(JobStoreError::InvalidTransition("Job input is retired"));
}
self.request.validate()?;
if self.request.input_digest()? != self.input_digest
|| self.request.fingerprint()? != self.intent_fingerprint
{
return Err(JobStoreError::Corrupt(
"immutable input or intent fingerprint mismatch".into(),
));
}
Ok(&self.request)
}
/// Shared Feature-facing read contract; no Workspace or execution identity leaks.
pub fn outcome(&self) -> job::JobOutcome {
job::JobOutcome {
job_id: self.request.job_id.clone(),
input_digest: self
.request
.input_digest()
.expect("validated immutable Job input"),
input_digest: self.input_digest.clone(),
attempt_id: self.attempt.attempt_id.clone(),
attempt: self.attempt.number,
state: match self.state {
@@ -204,9 +260,23 @@ impl JobStore {
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
SCHEMA_VERSION => {}
4 | SCHEMA_VERSION => {}
other => return Err(JobStoreError::UnsupportedSchema(other)),
}
if version < 5 {
tx.execute_batch(INPUT_RETENTION_SCHEMA)?;
let has_legacy_archive: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_schema WHERE type='table' AND name='legacy_job_archive')", [], |r| r.get(0))?;
if has_legacy_archive {
tx.execute_batch("DROP TRIGGER legacy_job_archive_immutable;
CREATE TRIGGER legacy_job_archive_immutable BEFORE UPDATE OF job_id,fingerprint,attempts_json ON legacy_job_archive
BEGIN SELECT RAISE(ABORT, 'immutable legacy Job archive'); END;
CREATE TRIGGER legacy_job_archive_input_retirement BEFORE UPDATE OF request_json ON legacy_job_archive
WHEN NOT EXISTS (SELECT 1 FROM job_intents WHERE job_id=OLD.job_id AND input_disposition<>'available')
OR (json_remove(NEW.request_json,'$.input') = json_remove(OLD.request_json,'$.input') AND json_type(NEW.request_json,'$.input')='null') IS NOT TRUE
BEGIN SELECT RAISE(ABORT, 'immutable legacy Job input'); END;")?;
}
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
// An accepted result is the only evidence of success. Reserved attempts
// were never started and remain pending; cancelled attempts never resume.
tx.execute_batch(
@@ -249,10 +319,7 @@ impl JobStore {
[&request.job_id],
|row| row.get(0),
)?;
if stored != fingerprint
|| snapshot.request != request
|| snapshot.domain_grant != grant
{
if stored != fingerprint || snapshot.domain_grant != grant {
return Err(JobStoreError::IntentConflict(request.job_id));
}
tx.commit()?;
@@ -334,6 +401,7 @@ impl JobStore {
.conn
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let previous = require_snapshot(&tx, job_id)?;
previous.load_executable_request()?;
if previous.state != JobState::Failed || previous.attempt.state != JobAttemptState::Failed {
return Err(JobStoreError::InvalidTransition(
"retry requires known failure",
@@ -354,6 +422,81 @@ impl JobStore {
Ok(snapshot)
}
/// Historical snapshots for one domain Session, including terminal and retired Jobs.
pub(crate) fn list_session_snapshots(
&self,
session_id: &str,
after: Option<&str>,
limit: usize,
) -> Result<Vec<JobSnapshot>, JobStoreError> {
let mut stmt = self.conn.prepare("SELECT i.job_id FROM job_intents i JOIN job_domain_grants g ON g.job_id=i.job_id WHERE json_extract(g.grant_json,'$.session_id')=?1 AND (?2 IS NULL OR i.job_id>?2) ORDER BY i.job_id LIMIT ?3")?;
let ids = stmt
.query_map(
params![session_id, after, limit.clamp(1, 100) as i64],
|r| r.get::<_, String>(0),
)?
.collect::<rusqlite::Result<Vec<_>>>()?;
ids.into_iter()
.map(|id| require_snapshot(&self.conn, &id))
.collect()
}
/// The Host must fence domain mutation and delivery before erasure/invalidation.
/// This atomically prevents dispatch/retry/result acceptance, without rewriting outcome.
pub(crate) fn reclaim_input(
&mut self,
job_id: &str,
expected_input_digest: &str,
reason: JobInputRetirementReason,
) -> Result<JobSnapshot, JobStoreError> {
let tx = self
.conn
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, job_id)?;
if snapshot.input_digest != expected_input_digest {
return Err(JobStoreError::DigestMismatch);
}
let desired = reason.disposition();
if snapshot.input_disposition == desired {
tx.commit()?;
reclaim_retired_input_storage(&self.conn)?;
return Ok(snapshot);
}
if reason == JobInputRetirementReason::Completed
&& (snapshot.state != JobState::Completed
|| snapshot.attempt.state != JobAttemptState::Completed
|| snapshot.result.is_none())
{
return Err(JobStoreError::InvalidTransition(
"reclaim requires known completed result",
));
}
if snapshot.input_disposition != JobInputDisposition::Available
&& reason != JobInputRetirementReason::ExplicitErasure
{
return Err(JobStoreError::InvalidTransition(
"retired input cannot be restored or reclassified",
));
}
if reason == JobInputRetirementReason::Completed {
snapshot.load_executable_request()?;
}
let mut metadata = snapshot.request.clone();
metadata.input = Value::Null;
tx.execute(
"UPDATE job_intents SET request_json=?2, input_disposition=?3 WHERE job_id=?1",
params![job_id, serde_json::to_string(&metadata)?, desired.as_str()],
)?;
let has_legacy_archive: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_schema WHERE type='table' AND name='legacy_job_archive')", [], |r| r.get(0))?;
if has_legacy_archive {
tx.execute("UPDATE legacy_job_archive SET request_json=json_set(request_json,'$.input',json('null')) WHERE job_id=?1", [job_id])?;
}
let snapshot = require_snapshot(&tx, job_id)?;
tx.commit()?;
reclaim_retired_input_storage(&self.conn)?;
Ok(snapshot)
}
pub(crate) fn get(&self, job_id: &str) -> Result<JobSnapshot, JobStoreError> {
require_snapshot(&self.conn, job_id)
}
@@ -383,7 +526,7 @@ impl JobStore {
let mut statement = self.conn.prepare(
"SELECT i.job_id FROM job_intents i JOIN job_attempts a
ON a.job_id = i.job_id AND a.number = i.current_attempt
WHERE i.state = 'pending' AND a.state = 'reserved' ORDER BY i.rowid",
WHERE i.state = 'pending' AND i.input_disposition='available' AND a.state = 'reserved' ORDER BY i.rowid",
)?;
let ids = statement
.query_map([], |row| row.get::<_, String>(0))?
@@ -406,6 +549,7 @@ impl JobStore {
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, job_id)?;
check_attempt(&snapshot, attempt_id)?;
snapshot.load_executable_request()?;
if snapshot.state != JobState::Pending
|| snapshot.attempt.state != JobAttemptState::Reserved
{
@@ -447,7 +591,7 @@ impl JobStore {
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, &bound.job_id)?;
check_attempt(&snapshot, &bound.attempt_id)?;
if snapshot.request.input_digest()? != bound.input_digest {
if snapshot.input_digest != bound.input_digest {
return Err(JobStoreError::DigestMismatch);
}
let (digest, encoded) =
@@ -466,6 +610,7 @@ impl JobStore {
tx.commit()?;
return Ok(snapshot);
}
snapshot.load_executable_request()?;
if snapshot.state != JobState::Pending
|| snapshot.attempt.state != JobAttemptState::Dispatched
{
@@ -649,7 +794,7 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
let row = conn
.query_row(
"SELECT i.request_json, i.state, i.acknowledged, a.attempt_id, a.number,
a.state, a.failure, a.result_json FROM job_intents i JOIN job_attempts a
a.state, a.failure, a.result_json, a.input_digest, i.fingerprint, i.input_disposition FROM job_intents i JOIN job_attempts a
ON a.job_id = i.job_id AND a.number = i.current_attempt WHERE i.job_id = ?1",
[job_id],
|row| {
@@ -662,12 +807,27 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
row.get::<_, String>(5)?,
row.get::<_, Option<String>>(6)?,
row.get::<_, Option<String>>(7)?,
row.get::<_, String>(8)?,
row.get::<_, String>(9)?,
row.get::<_, String>(10)?,
))
},
)
.optional()?;
row.map(
|(request, state, acknowledged, attempt_id, number, attempt_state, failure, result)| {
|(
request,
state,
acknowledged,
attempt_id,
number,
attempt_state,
failure,
result,
input_digest,
intent_fingerprint,
disposition,
)| {
let state = match state.as_str() {
"pending" => JobState::Pending,
"completed" => JobState::Completed,
@@ -693,8 +853,17 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
)));
}
};
let request_value: Value = serde_json::from_str(&request)?;
if request_value.get("input").is_none() {
return Err(JobStoreError::Corrupt(
"stored Job input field is missing".into(),
));
}
Ok(JobSnapshot {
request: serde_json::from_str(&request)?,
request: serde_json::from_value(request_value)?,
input_digest,
intent_fingerprint,
input_disposition: serde_json::from_value(Value::String(disposition))?,
domain_grant: conn
.query_row(
"SELECT grant_json FROM job_domain_grants WHERE job_id = ?1",
@@ -719,6 +888,37 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
.transpose()
}
// Retirement is already durable before this maintenance step. Failure must be
// retried with the same Job/digest/reason; it must never restore input. VACUUM
// removes historical free-page copies, including stores written before secure_delete.
fn reclaim_retired_input_storage(conn: &Connection) -> Result<(), JobStoreError> {
conn.execute_batch("PRAGMA secure_delete=ON; VACUUM;")?;
let busy: i64 = conn.query_row("PRAGMA wal_checkpoint(TRUNCATE)", [], |r| r.get(0))?;
if busy != 0 {
return Err(JobStoreError::InvalidTransition(
"input retired; SQLite body cleanup blocked by a reader, retry retirement",
));
}
Ok(())
}
const INPUT_RETENTION_SCHEMA: &str = "
ALTER TABLE job_intents ADD COLUMN input_disposition TEXT NOT NULL DEFAULT 'available'
CHECK (input_disposition IN ('available','reclaimed','erased','invalidated'));
DROP TRIGGER job_intent_immutable;
CREATE TRIGGER job_intent_immutable BEFORE UPDATE OF
job_id, fingerprint, serialization_key, max_concurrent ON job_intents
BEGIN SELECT RAISE(ABORT, 'immutable Job intent'); END;
CREATE TRIGGER job_input_retirement BEFORE UPDATE OF request_json, input_disposition ON job_intents
WHEN (
((OLD.input_disposition = 'available' AND NEW.input_disposition IN ('reclaimed','erased','invalidated'))
OR (OLD.input_disposition IN ('reclaimed','invalidated') AND NEW.input_disposition = 'erased'))
AND json_remove(NEW.request_json,'$.input') = json_remove(OLD.request_json,'$.input')
AND json_type(NEW.request_json,'$.input') = 'null'
) IS NOT TRUE
BEGIN SELECT RAISE(ABORT, 'immutable Job input retirement'); END;
";
const RECONCILIATION_SCHEMA: &str = "
DROP INDEX job_serialization_singleflight;
CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key)
@@ -885,7 +1085,11 @@ mod tests {
Some("unconfirmed"),
)
.unwrap();
store.conn.execute_batch("DROP TRIGGER job_reconciliation_immutable; DROP TRIGGER job_reconciliation_retained; DROP TABLE job_domain_reconciliations; DROP INDEX job_serialization_singleflight; CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key) WHERE serialization_key IS NOT NULL AND state IN ('pending','unknown'); PRAGMA user_version=3;").unwrap();
store.conn.execute_batch("DROP TRIGGER job_input_retirement; DROP TRIGGER job_intent_immutable;
ALTER TABLE job_intents DROP COLUMN input_disposition;
CREATE TRIGGER job_intent_immutable BEFORE UPDATE OF job_id,request_json,fingerprint,serialization_key,max_concurrent ON job_intents
BEGIN SELECT RAISE(ABORT, 'immutable Job intent'); END;
DROP TRIGGER job_reconciliation_immutable; DROP TRIGGER job_reconciliation_retained; DROP TABLE job_domain_reconciliations; DROP INDEX job_serialization_singleflight; CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key) WHERE serialization_key IS NOT NULL AND state IN ('pending','unknown'); PRAGMA user_version=3;").unwrap();
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(reopened.get("first").unwrap(), unknown);
@@ -911,10 +1115,7 @@ mod tests {
JobResultSubmission {
job_id: snapshot.request.job_id.clone(),
attempt_id: snapshot.attempt.attempt_id.clone(),
input_digest: snapshot
.request
.input_digest()
.expect("validated immutable Job input"),
input_digest: snapshot.input_digest.clone(),
result,
}
}
@@ -926,6 +1127,314 @@ mod tests {
.unwrap()
}
#[test]
fn input_reclaim_reopens_as_receipt_and_exact_replay_never_rehydrates() {
let (dir, mut store) = fixture();
let original = request("retained");
let grant = Some(json!({"session_id":"session-a", "consumer":"fixture"}));
let reserved = store
.reserve_granted(original.clone(), grant.clone())
.unwrap();
let running = store
.dispatch("retained", &reserved.attempt.attempt_id, 8)
.unwrap();
let result = submission(&running, json!({"saved":true}));
let completed = store.accept(&result).unwrap();
store.ack("retained").unwrap();
let retired = store
.reclaim_input(
"retained",
&completed.input_digest,
JobInputRetirementReason::Completed,
)
.unwrap();
assert_eq!(retired.input_disposition, JobInputDisposition::Reclaimed);
assert!(retired.request.input.is_null());
assert!(retired.load_executable_request().is_err());
assert_eq!(
retired.outcome().input_digest,
original.input_digest().unwrap()
);
assert_eq!(retired.result, completed.result);
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(
reopened
.reserve_granted(original.clone(), grant.clone())
.unwrap(),
retired
);
assert_eq!(reopened.accept(&result).unwrap(), retired);
assert_eq!(
reopened
.list_session_snapshots("session-a", None, 1)
.unwrap(),
vec![retired.clone()]
);
assert!(
reopened
.list_session_snapshots("session-b", None, 100)
.unwrap()
.is_empty()
);
assert!(
reopened
.list_session_snapshots("session-a", Some("retained"), 100)
.unwrap()
.is_empty()
);
let mut changed = original.clone();
changed.input = json!({"changed":true});
assert!(matches!(
reopened.reserve_granted(changed, grant),
Err(JobStoreError::IntentConflict(_))
));
assert!(reopened.retry("retained").is_err());
assert!(
reopened
.dispatch("retained", &retired.attempt.attempt_id, 8)
.is_err()
);
assert!(reopened.conn.execute("UPDATE job_intents SET request_json=?1,input_disposition='available' WHERE job_id='retained'", [serde_json::to_string(&original).unwrap()]).is_err());
let erased = reopened
.reclaim_input(
"retained",
&retired.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert_eq!(erased.input_disposition, JobInputDisposition::Erased);
assert_eq!(erased.state, JobState::Completed);
assert!(erased.acknowledged);
assert_eq!(
reopened
.reserve_granted(original, erased.domain_grant.clone())
.unwrap(),
erased
);
}
#[test]
fn unfinished_unknown_and_failed_inputs_are_kept_until_explicit_invalidation() {
let (_dir, mut store) = fixture();
for (id, state) in [
("pending", None),
("unknown", Some(JobAttemptState::Unknown)),
("failed", Some(JobAttemptState::Failed)),
] {
let original = request(id);
let snapshot = store.reserve(original.clone()).unwrap();
if let Some(state) = state {
store.dispatch(id, &snapshot.attempt.attempt_id, 8).unwrap();
store
.finish(
id,
&snapshot.attempt.attempt_id,
state,
Some("not complete"),
)
.unwrap();
}
let before = store.get(id).unwrap();
assert!(
store
.reclaim_input(
id,
&before.input_digest,
JobInputRetirementReason::Completed
)
.is_err()
);
assert_eq!(store.get(id).unwrap(), before);
assert_eq!(before.load_executable_request().unwrap(), &original);
assert!(
store
.reclaim_input(id, "wrong", JobInputRetirementReason::ExplicitErasure)
.is_err()
);
let retired = store
.reclaim_input(
id,
&before.input_digest,
JobInputRetirementReason::LegacyInvalidation,
)
.unwrap();
assert_eq!(retired.state, before.state);
assert_eq!(retired.attempt, before.attempt);
assert_eq!(retired.input_disposition, JobInputDisposition::Invalidated);
assert!(store.retry(id).is_err());
assert!(store.dispatch(id, &before.attempt.attempt_id, 8).is_err());
assert_eq!(store.reserve(original).unwrap(), retired);
}
}
#[test]
fn erased_dispatched_input_refuses_delayed_result_without_rewriting_unknown() {
let (dir, mut store) = fixture();
let running = dispatched(&mut store, request("erase-running"));
let result = submission(&running, json!({"late":true}));
let erased = store
.reclaim_input(
"erase-running",
&running.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert!(store.accept(&result).is_err());
assert_eq!(erased.state, JobState::Pending);
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
let recovered = reopened.get("erase-running").unwrap();
assert_eq!(recovered.state, JobState::Unknown);
assert_eq!(recovered.input_disposition, JobInputDisposition::Erased);
assert_eq!(recovered.input_digest, running.input_digest);
assert!(reopened.accept(&result).is_err());
assert!(reopened.retry("erase-running").is_err());
}
#[test]
fn available_input_loader_refuses_tamper_missing_body_and_envelope_changes() {
for mutation in [
"json_set(request_json,'$.input',json('{\"tampered\":true}'))",
"json_remove(request_json,'$.input')",
"json_set(request_json,'$.instruction','changed')",
] {
let (_dir, mut store) = fixture();
store.reserve(request("tamper")).unwrap();
// Simulate external corruption bypassing SQLite's normal write guard.
store
.conn
.execute_batch("DROP TRIGGER job_input_retirement")
.unwrap();
store
.conn
.execute_batch(&format!("UPDATE job_intents SET request_json={mutation}"))
.unwrap();
match store.get("tamper") {
Ok(snapshot) => {
assert!(snapshot.load_executable_request().is_err());
assert!(
store
.dispatch("tamper", &snapshot.attempt.attempt_id, 8)
.is_err()
);
}
Err(JobStoreError::Json(_) | JobStoreError::Corrupt(_)) => {}
other => panic!("unexpected corrupt input read: {other:?}"),
}
}
}
#[test]
fn input_retirement_cleanup_failure_is_durable_and_exact_retry_finishes_without_rehydration() {
let (dir, mut store) = fixture();
let mut original = request("physical-erasure");
let private_body = "private-input-retirement-original-bytes".repeat(1000);
original.input = json!({"body":private_body});
let reserved = store.reserve(original.clone()).unwrap();
// A reader retaining the old SQLite snapshot prevents truncation of the old WAL.
let reader = Connection::open(dir.path().join("jobs.sqlite3")).unwrap();
reader
.execute_batch("BEGIN; SELECT request_json FROM job_intents;")
.unwrap();
store.conn.busy_timeout(std::time::Duration::ZERO).unwrap();
assert!(
store
.reclaim_input(
"physical-erasure",
&reserved.input_digest,
JobInputRetirementReason::ExplicitErasure
)
.is_err()
);
let retired = store.get("physical-erasure").unwrap();
assert_eq!(retired.input_disposition, JobInputDisposition::Erased);
assert!(retired.load_executable_request().is_err());
reader.execute_batch("ROLLBACK").unwrap();
drop(reader);
let replay = store
.reclaim_input(
"physical-erasure",
&reserved.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert_eq!(replay, retired);
assert_eq!(store.reserve(original).unwrap(), retired);
let needle = b"private-input-retirement-original-bytes";
for entry in std::fs::read_dir(dir.path()).unwrap() {
let path = entry.unwrap().path();
if path.is_file() {
let bytes = std::fs::read(&path).unwrap();
assert!(
!bytes.windows(needle.len()).any(|bytes| bytes == needle),
"retired body remained in {}",
path.display()
);
}
}
}
#[test]
fn missing_original_null_input_is_not_treated_as_available_null() {
let (_dir, mut store) = fixture();
let mut original = request("null-original");
original.input = Value::Null;
let reserved = store.reserve(original).unwrap();
assert!(reserved.load_executable_request().is_ok());
store.conn.execute_batch("DROP TRIGGER job_input_retirement; UPDATE job_intents SET request_json=json_remove(request_json,'$.input')").unwrap();
assert!(store.get("null-original").is_err());
}
#[test]
fn input_retirement_migration_preserves_receipts_and_rolls_back_failed_ddl() {
let (dir, mut store) = fixture();
let original = store.reserve(request("upgrade")).unwrap();
store.conn.execute_batch("DROP TRIGGER job_input_retirement; DROP TRIGGER job_intent_immutable;
ALTER TABLE job_intents DROP COLUMN input_disposition;
CREATE TRIGGER job_intent_immutable BEFORE UPDATE OF job_id,request_json,fingerprint,serialization_key,max_concurrent ON job_intents
BEGIN SELECT RAISE(ABORT, 'immutable Job intent'); END;
PRAGMA user_version=4;
CREATE TRIGGER job_input_retirement BEFORE UPDATE ON job_attempts BEGIN SELECT RAISE(ABORT,'collision'); END;").unwrap();
drop(store);
let path = dir.path().join("jobs.sqlite3");
assert!(JobStore::open(&path).is_err());
let conn = Connection::open(&path).unwrap();
let version: i64 = conn
.pragma_query_value(None, "user_version", |r| r.get(0))
.unwrap();
assert_eq!(version, 4);
let present: bool = conn.query_row("SELECT EXISTS(SELECT 1 FROM pragma_table_info('job_intents') WHERE name='input_disposition')",[],|r|r.get(0)).unwrap();
assert!(!present);
conn.execute_batch("DROP TRIGGER job_input_retirement")
.unwrap();
drop(conn);
let mut reopened = JobStore::open(&path).unwrap();
assert_eq!(reopened.get("upgrade").unwrap(), original);
assert_eq!(
reopened
.get("upgrade")
.unwrap()
.load_executable_request()
.unwrap(),
&request("upgrade")
);
let retired = reopened
.reclaim_input(
"upgrade",
&original.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert_eq!(retired.state, JobState::Pending);
assert!(
reopened
.conn
.execute_batch("UPDATE job_intents SET input_disposition='available'")
.is_err()
);
}
#[test]
fn database_configuration_schema_and_identity_constraints() {
let (_dir, mut store) = fixture();
@@ -143,6 +143,42 @@ mod tests {
(original, receipt, grant)
}
#[test]
fn legacy_archive_input_erasure_preserves_fingerprints_and_attempt_receipts() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("jobs.sqlite3");
seed(&path, 2);
let mut store = JobStore::open(&path).unwrap();
let before: (String, String) = store
.conn
.query_row(
"SELECT fingerprint,attempts_json FROM legacy_job_archive WHERE job_id='done'",
[],
|r| Ok((r.get(0)?, r.get(1)?)),
)
.unwrap();
let snapshot = store.get("done").unwrap();
store
.reclaim_input(
"done",
&snapshot.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
let (body, fingerprint, attempts): (String,String,String) = store.conn.query_row("SELECT request_json,fingerprint,attempts_json FROM legacy_job_archive WHERE job_id='done'",[],|r|Ok((r.get(0)?,r.get(1)?,r.get(2)?))).unwrap();
assert!(serde_json::from_str::<Value>(&body).unwrap()["input"].is_null());
assert_eq!((fingerprint, attempts), before);
assert!(store.conn.execute_batch("UPDATE legacy_job_archive SET request_json=json_set(request_json,'$.input','restored') WHERE job_id='done'").is_err());
drop(store);
let mut reopened = JobStore::open(&path).unwrap();
let receipt = reopened
.reserve_granted(request("done"), snapshot.domain_grant)
.unwrap();
assert_eq!(receipt.state, JobState::Completed);
assert_eq!(receipt.input_disposition, JobInputDisposition::Erased);
assert!(receipt.load_executable_request().is_err());
}
#[test]
fn legacy_versions_preserve_intent_receipts_attempts_grants_and_replay() {
for version in [1, 2] {
+113 -48
View File
@@ -26,7 +26,10 @@ use job::{JobRequest, JobResultSubmission};
use tokio::sync::watch;
use crate::job_store::JobStore;
pub use crate::job_store::{JobAttempt, JobAttemptState, JobSnapshot, JobState, JobStoreError};
pub use crate::job_store::{
JobAttempt, JobAttemptState, JobInputDisposition, JobInputRetirementReason, JobSnapshot,
JobState, JobStoreError,
};
#[derive(Debug, thiserror::Error)]
pub enum JobServiceError {
@@ -40,6 +43,8 @@ pub enum JobServiceError {
Capacity,
#[error("standalone Job cleanup failed: {0}")]
Cleanup(String),
#[error("standalone Job shutdown is incomplete; execution resources are still owned")]
ShutdownIncomplete,
}
pub(crate) trait JobAttemptFence: Send + Sync {
@@ -173,6 +178,36 @@ impl StandaloneJobs {
Ok(self.with_store(|store| store.reserve(request))?)
}
/// Enumerate historical domain snapshots for a Session, including retired input receipts.
pub fn list_session_snapshots(
&self,
session_id: &str,
after: Option<&str>,
limit: usize,
) -> Result<Vec<JobSnapshot>, JobServiceError> {
self.with_store(|store| store.list_session_snapshots(session_id, after, limit))
}
/// Retire body only. The caller fences domain effects/delivered snapshots before erasure.
/// Normal reclamation also requires that execution cleanup has finished.
pub fn reclaim_input(
&self,
job_id: &str,
expected_input_digest: &str,
reason: JobInputRetirementReason,
) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if reason == JobInputRetirementReason::Completed
&& executions
.jobs
.get(job_id)
.is_some_and(|execution| !execution.task.is_finished())
{
return Err(JobServiceError::Capacity);
}
self.with_store(|store| store.reclaim_input(job_id, expected_input_digest, reason))
}
pub fn get(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
Ok(self.with_store(|store| store.get(job_id))?)
}
@@ -274,6 +309,12 @@ impl StandaloneJobs {
job_id: &str,
client: Option<Box<dyn LlmClient>>,
) -> Result<(), JobServiceError> {
// Preparation owns copies of the immutable input and domain adapter too. Exclude
// owner shutdown before creating them, not only before publishing the task.
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
let client = client.or_else(|| {
self.model_client
.lock()
@@ -282,16 +323,14 @@ impl StandaloneJobs {
.map(|c| c.clone_boxed())
});
let snapshot = self.get(job_id)?;
snapshot.load_executable_request()?;
let timeout = Duration::from_secs(u64::from(snapshot.request.limits.timeout_seconds));
let deadline = tokio::time::Instant::now() + timeout;
let fence: Arc<dyn JobAttemptFence> = Arc::new(BoundAttemptFence {
store: self.store.clone(),
job_id: job_id.into(),
attempt_id: snapshot.attempt.attempt_id.clone(),
input_digest: snapshot
.request
.input_digest()
.map_err(JobStoreError::from)?,
input_digest: snapshot.input_digest.clone(),
deadline,
});
let grant = match (
@@ -311,10 +350,6 @@ impl StandaloneJobs {
let prepared =
worker::job::prepare_job_with_grant(&snapshot.request, &self.cwd, client, grant)
.map_err(|error| JobServiceError::Configuration(error.to_string()))?;
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
executions.jobs.retain(|_, run| {
!(run.task.is_finished() && matches!(run.complete.borrow().as_ref(), Some(Ok(()))))
});
@@ -367,10 +402,7 @@ impl StandaloneJobs {
store: store.clone(),
job_id: id.clone(),
attempt_id: attempt_id.clone(),
input_digest: snapshot
.request
.input_digest()
.map_err(JobStoreError::from)?,
input_digest: snapshot.input_digest.clone(),
deadline,
});
let (cancel, mut cancellation) = watch::channel(false);
@@ -446,23 +478,25 @@ impl StandaloneJobs {
.clone()
.expect("completion");
result.map_err(JobServiceError::Cleanup)?;
let task = {
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions
.jobs
.get(job_id)
.is_some_and(|run| run.attempt_id == attempt)
{
executions.jobs.remove(job_id)
} else {
None
}
};
if let Some(execution) = task {
execution
.task
.await
.map_err(|error| JobServiceError::Cleanup(error.to_string()))??;
// Completion is published immediately before the task returns. Do not remove
// its ownership record until the runtime confirms that return (and future drop).
while self
.executions
.lock()
.expect("Job executions poisoned")
.jobs
.get(job_id)
.is_some_and(|run| run.attempt_id == attempt && !run.task.is_finished())
{
tokio::task::yield_now().await;
}
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions
.jobs
.get(job_id)
.is_some_and(|run| run.attempt_id == attempt)
{
executions.jobs.remove(job_id);
}
}
self.get(job_id)
@@ -476,33 +510,63 @@ impl StandaloneJobs {
}
}
pub(crate) async fn shutdown(&self) -> Result<(), JobServiceError> {
/// Initiate owner shutdown without blocking a synchronous caller. Success confirms all
/// local executions have returned, including request/history and Feature cleanup. An
/// incomplete/error result retains the service resources and must retain the Host lease.
pub(crate) fn try_shutdown(&self) -> Result<(), JobServiceError> {
self.close();
let tasks = {
let mut executions = self.executions.lock().expect("Job executions poisoned");
std::mem::take(&mut executions.jobs)
};
let mut error = None;
for execution in tasks.into_values() {
match execution.task.await {
Ok(Ok(())) => {}
Ok(Err(failure)) => {
error = Some(failure);
}
Err(failure) => {
error = Some(JobServiceError::Cleanup(failure.to_string()));
let mut executions = self.executions.lock().expect("Job executions poisoned");
for execution in executions.jobs.values() {
if !execution.task.is_finished() {
return Err(JobServiceError::ShutdownIncomplete);
}
match execution.complete.borrow().as_ref() {
Some(Ok(())) => {}
Some(Err(error)) => return Err(JobServiceError::Cleanup(error.clone())),
None => {
return Err(JobServiceError::Cleanup(
"Internal Worker ended without cleanup confirmation".into(),
));
}
}
}
// Close SQLite before the Host releases its lease. Cloned handles may
// outlive the Host but cannot mutate or retain the live connection.
executions.jobs.clear();
// Retained external handles share these slots; they cannot retain a live store,
// provider or model client after successful owner shutdown.
self.store.lock().expect("Job store poisoned").take();
self.domain.lock().expect("Job domain poisoned").take();
self.model_client
.lock()
.expect("Job model transport poisoned")
.take();
error.map_or(Ok(()), Err)
Ok(())
}
pub(crate) async fn shutdown(&self) -> Result<(), JobServiceError> {
self.close();
let completions: Vec<_> = self
.executions
.lock()
.expect("Job executions poisoned")
.jobs
.values()
.map(|execution| execution.complete.clone())
.collect();
// Never take execution ownership out of the shared service across an await: a
// cancelled shutdown future or a concurrent close must still see the barrier.
for mut completion in completions {
completion
.wait_for(Option::is_some)
.await
.map_err(|error| JobServiceError::Cleanup(error.to_string()))?;
}
loop {
match self.try_shutdown() {
// Only the task's final return follows its completion notification.
Err(JobServiceError::ShutdownIncomplete) => tokio::task::yield_now().await,
result => return result,
}
}
}
}
@@ -574,7 +638,8 @@ impl JobAttemptFence for BoundAttemptFence {
if snapshot.state != JobState::Pending
|| snapshot.attempt.state != JobAttemptState::Dispatched
|| snapshot.attempt.attempt_id != self.attempt_id
|| snapshot.request.input_digest().map_err(|e| e.to_string())? != self.input_digest
|| snapshot.input_digest != self.input_digest
|| snapshot.input_disposition != JobInputDisposition::Available
{
return Err("Job domain grant is not bound to the current live attempt".into());
}
+101
View File
@@ -468,3 +468,104 @@ async fn cleanup_cannot_reclassify_success_or_durable_cancellation_and_stale_han
assert!(matches!(operation, Err(JobServiceError::Closed)));
}
}
#[tokio::test]
async fn cancelled_shutdown_future_keeps_execution_barrier_and_unknown_input() {
// Actual generic runner + SQLite; only the provider stream is scripted/blocked.
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("jobs.sqlite3");
let jobs = StandaloneJobs::open(&path, temp.path().into()).unwrap();
let reserved = jobs.request(request("cancel-shutdown-wait")).unwrap();
let model = DomainTestClient {
stall: true,
..Default::default()
};
jobs.start_with_model_client(&reserved.request.job_id, model.clone())
.await
.unwrap();
await_model(&model).await;
{
let shutdown = jobs.shutdown();
tokio::pin!(shutdown);
assert!(futures::poll!(shutdown.as_mut()).is_pending());
// Drop the caller's waiter while cancellation/cleanup is still pending.
}
assert!(matches!(
jobs.try_shutdown(),
Err(JobServiceError::ShutdownIncomplete)
));
assert!(jobs.store.lock().unwrap().is_some());
assert!(matches!(
jobs.request(request("after-shutdown")),
Err(JobServiceError::Closed)
));
tokio::time::timeout(Duration::from_secs(5), jobs.shutdown())
.await
.unwrap()
.unwrap();
assert!(jobs.store.lock().unwrap().is_none());
let reopened = StandaloneJobs::open(&path, temp.path().into()).unwrap();
let unknown = reopened.get(&reserved.request.job_id).unwrap();
assert_eq!(unknown.attempt.state, JobAttemptState::Unknown);
assert_eq!(unknown.input_disposition, JobInputDisposition::Available);
assert_eq!(unknown.request, reserved.request);
assert_eq!(unknown.input_digest, reserved.input_digest);
assert!(reopened.retry(&unknown.request.job_id).is_err());
reopened.shutdown().await.unwrap();
}
#[tokio::test]
async fn input_retirement_service_fences_live_attempt_and_preserves_session_receipts() {
// Production service + SQLite + attempt/result fences, without a model or Worker fixture.
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("jobs.sqlite3");
let jobs = StandaloneJobs::open(&path, temp.path().to_path_buf()).unwrap();
let original = request("input-retirement");
let grant = json!({"session_id":"session-a"});
let reserved = jobs
.request_granted(original.clone(), grant.clone())
.unwrap();
jobs.with_store(|store| store.dispatch("input-retirement", &reserved.attempt.attempt_id, 8))
.unwrap();
let fence = BoundAttemptFence {
store: jobs.store.clone(),
job_id: "input-retirement".into(),
attempt_id: reserved.attempt.attempt_id.clone(),
input_digest: reserved.input_digest.clone(),
deadline: tokio::time::Instant::now() + Duration::from_secs(10),
};
assert!(fence.ensure_live().is_ok());
assert!(
jobs.reclaim_input(
"input-retirement",
&reserved.input_digest,
JobInputRetirementReason::Completed
)
.is_err()
);
let erased = jobs
.reclaim_input(
"input-retirement",
&reserved.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.unwrap();
assert!(fence.ensure_live().is_err());
assert!(
sink(&jobs, "input-retirement", &reserved.attempt.attempt_id)
.submit(result("input-retirement", &reserved.attempt.attempt_id))
.is_err()
);
assert!(jobs.start("input-retirement").await.is_err());
assert_eq!(
jobs.list_session_snapshots("session-a", None, 100).unwrap(),
vec![erased.clone()]
);
assert!(
jobs.list_session_snapshots("session-b", None, 100)
.unwrap()
.is_empty()
);
assert_eq!(jobs.request_granted(original, grant).unwrap(), erased);
jobs.shutdown().await.unwrap();
}
+38
View File
@@ -7,6 +7,44 @@ pub mod launch;
pub mod store;
pub mod subjektiv;
mod subjektiv_sessions;
pub mod unix;
pub use unix::{UnixEndpoint, relay_unix_client, run_unix_client};
/// Restore reads the saved cwd and Profile, not a newly resolved launch recipe.
pub enum HeadlessLaunch {
Fresh(ResolvedStandaloneLaunch),
Restore {
state_dir: std::path::PathBuf,
worker_id: WorkerId,
},
}
#[derive(Debug, thiserror::Error)]
pub enum HeadlessError {
#[error(transparent)]
Transport(#[from] std::io::Error),
#[error(transparent)]
Startup(#[from] StandaloneStartupError),
#[error(transparent)]
Shutdown(#[from] StandaloneShutdownError),
}
/// Reserve the optional public transport before creating or restoring execution state.
pub async fn run_headless(
launch: HeadlessLaunch,
endpoint: std::path::PathBuf,
) -> Result<(), HeadlessError> {
let endpoint = UnixEndpoint::bind(endpoint)?;
let host = match launch {
HeadlessLaunch::Fresh(launch) => StandaloneHost::start(launch).await?,
HeadlessLaunch::Restore {
state_dir,
worker_id,
} => StandaloneHost::restore(state_dir, worker_id).await?,
};
host.serve_unix(endpoint).await
}
pub use host::{
StandaloneHost, StandaloneShutdownError, StandaloneStartupError,
+49 -1
View File
@@ -54,6 +54,8 @@ pub enum SubjectError {
Journal(#[from] subjektiv::journal::JournalError),
#[error(transparent)]
Storage(#[from] feature_storage::FeatureStorageError),
#[error("local Subject Job shutdown: {0}")]
JobShutdown(#[from] crate::jobs::JobServiceError),
#[error("local Subject storage metadata: {0}")]
Json(#[from] serde_json::Error),
#[error("local Subject storage permission or identity is invalid: {0}")]
@@ -338,8 +340,16 @@ impl SubjectConnection {
}
}
pub fn close(&mut self) -> Result<(), SubjectError> {
self.host.jobs.lock().expect("Subject Jobs poisoned").take();
self.host.live.store(false, Ordering::Release);
{
let mut jobs = self.host.jobs.lock().expect("Subject Jobs poisoned");
if let Some(service) = jobs.as_ref() {
// Signal cancellation, but retain the handle and exclusive lease until
// the generic owner confirms local request/history/resource teardown.
service.try_shutdown()?;
}
jobs.take();
}
self.host.catalog.manager.shutdown()?;
if let Some(file) = &self.lease {
file.unlock()?;
@@ -840,6 +850,44 @@ impl SubjektivHost for LocalSubjectHost {
.journal_startup_snapshot(&reader, predecessor.as_deref())
.map_err(journal::journal_error)
}
fn journal_compaction_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn journal_compaction_context(
&self,
context: &SubjektivHostContext,
) -> Result<subjektiv::journal::startup::JournalStartupSnapshot, SubjektivHostError> {
self.body(context)?;
let attribution = self
.catalog
.store
.session_attribution(&context.session_id)
.map_err(domain_error)?
.filter(|attr| {
attr.subject_id == self.binding.subject_id
&& attr.runtime_id == self.binding.scope_id
&& attr.worker_id == self.worker_id.to_string()
})
.ok_or_else(|| {
SubjektivHostError::InvalidArgument(
"foreign or unattributed compaction Session".into(),
)
})?;
let host = subjektiv::journal::JournalHostContext::validated_session(
&self.catalog.store,
attribution,
)
.map_err(journal::journal_error)?;
if host.writer_binding_id() != Some(self.writer.binding_id.as_str()) {
return Err(SubjektivHostError::InvalidArgument(
"compaction writer lifetime mismatch".into(),
));
}
self.catalog
.store
.journal_compaction_snapshot(&host)
.map_err(journal::journal_error)
}
fn journal_recording_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
+498 -231
View File
@@ -5,9 +5,12 @@
//! recovery authority; shutdown does not delete either original Sessions or pending requests.
use super::*;
use crate::jobs::{JobAttemptState, JobSnapshot, JobState};
use crate::jobs::{
JobAttemptState, JobInputDisposition, JobInputRetirementReason, JobSnapshot, JobState,
};
use session_store::capture::{CaptureDto, CaptureLimits, RecordingPosition};
use subjektiv::api::JobAttemptBinding;
use subjektiv::journal::recording::JournalMaterial;
use subjektiv::journal::recording::JournalInput;
use subjektiv::journal::transport::*;
use subjektiv::journal::*;
use worker::subjektiv::{JournalJobFinalizer, JournalJobHost, JournalJobToolsFeature};
@@ -16,7 +19,6 @@ use worker::subjektiv::{JournalJobFinalizer, JournalJobHost, JournalJobToolsFeat
mod tests;
pub(super) const PURPOSE: &str = "subjektiv_journal";
const INPUT_BUDGET: usize = 48 * 1024;
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
@@ -27,6 +29,7 @@ struct LocalJournalGrant {
session_id: String,
journal_id: String,
ranges: Vec<JournalSourceRange>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
origins: Vec<SourceOrigin>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
@@ -35,11 +38,6 @@ struct SourceOrigin {
pointer: JournalSourcePointer,
origin: session_store::public_material::PublicMaterialOrigin,
}
impl LocalJournalGrant {
fn input(&self) -> serde_json::Value {
serde_json::json!({"selected_ranges": self.ranges, "origins": self.origins})
}
}
impl StandaloneSubjects {
/// Explicit operator erasure, separate from ordinary Worker removal. Requires the
@@ -84,32 +82,29 @@ impl StandaloneSubjects {
self.state_root.clone(),
)
.map_err(|error| SubjectError::InvalidScope(error.to_string()))?;
for snapshot in jobs
.pending()
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?
.into_iter()
.chain(
jobs.unknown_granted()
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?,
)
{
if snapshot.domain_grant.as_ref().is_some_and(|grant| {
grant.get("session_id").and_then(|v| v.as_str()) == Some(session_id)
}) {
let mut after = None;
loop {
let page = jobs
.list_session_snapshots(session_id, after.as_deref(), 100)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
if page.is_empty() {
break;
}
for snapshot in &page {
jobs.cancel(&snapshot.request.job_id)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
jobs.reclaim_input(
&snapshot.request.job_id,
&snapshot.input_digest,
JobInputRetirementReason::ExplicitErasure,
)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
if snapshot.state == JobState::Completed && !snapshot.acknowledged {
jobs.acknowledge(&snapshot.request.job_id)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
}
}
}
for snapshot in jobs
.unacknowledged_results()
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?
{
if snapshot.domain_grant.as_ref().is_some_and(|grant| {
grant.get("session_id").and_then(|v| v.as_str()) == Some(session_id)
}) {
jobs.acknowledge(&snapshot.request.job_id)
.map_err(|e| SubjectError::InvalidScope(e.to_string()))?;
}
after = page.last().map(|job| job.request.job_id.clone());
}
session_store::FsStore::new_with_retention(
path,
@@ -590,17 +585,28 @@ impl LocalSubjectHost {
self.ensure_live()?;
let jobs = self.jobs()?;
for saved in jobs.unacknowledged_results().map_err(service_error)? {
if self.journal_grant(&saved).is_ok() {
if self.journal_grant_scope(&saved).is_ok() {
self.complete_journal_job(&jobs, &saved)?;
}
}
for pending in jobs.pending().map_err(service_error)? {
if self.journal_grant(&pending).is_ok() {
if pending.request.purpose != PURPOSE {
continue;
}
self.journal_grant_scope(&pending)?;
if pending.request.input.get("capture").is_some() {
self.journal_grant(&pending)?;
return Ok(Some((jobs, pending.request.job_id)));
}
// Old descriptor jobs never acquire new execution authority. Their immutable
// identity/input is retired, not upgraded in place; saved checkpoint wins.
let replacement = self.replace_legacy_journal_job(&jobs, &pending)?;
if let Some(id) = replacement {
return Ok(Some((jobs, id)));
}
}
for unknown in jobs.unknown_granted().map_err(service_error)? {
let Ok(grant) = self.journal_grant(&unknown) else {
let Ok(grant) = self.journal_grant_scope(&unknown) else {
continue;
};
let context = authority(self, &grant.session_id)?;
@@ -685,32 +691,96 @@ impl LocalSubjectHost {
}
}
}
let materials = self.select_journal_materials(&context, &journal.id)?;
if materials.is_empty() {
let mut retained = None;
// Saved body snapshots always win over descriptor migration. In particular,
// a failed continuation must not recapture its original text even if an
// invalidated legacy grant still bounds a larger unprocessed suffix.
for legacy_pass in [false, true] {
let mut after_job = None;
loop {
let page = jobs
.list_session_snapshots(
&journal.source_session_id,
after_job.as_deref(),
100,
)
.map_err(service_error)?;
if page.is_empty() {
break;
}
for saved in &page {
if saved.request.purpose != PURPOSE {
continue;
}
let legacy = saved.input_disposition
== JobInputDisposition::Invalidated
|| (saved.input_disposition == JobInputDisposition::Available
&& saved.request.input.get("capture").is_none());
if legacy != legacy_pass
|| (!legacy
&& saved.input_disposition != JobInputDisposition::Available)
{
continue;
}
let saved_grant = self.journal_grant_scope(saved)?;
if saved_grant.journal_id != journal.id {
continue;
}
if legacy {
if let Some(id) = self.replace_legacy_journal_job(&jobs, saved)? {
return Ok(Some((jobs, id)));
}
continue;
}
self.journal_grant(saved)?;
let original = JournalInput::decode(&saved.request.input)
.map_err(journal_error)?;
let continuation = self.build_journal_input(
&context,
&journal.id,
Some(&original.capture),
)?;
if !continuation.selected_ranges.is_empty() {
retained = Some(continuation);
break;
}
if saved.state == JobState::Completed && saved.acknowledged {
// Recover the acknowledgement-before-reclamation gap without
// turning a retained receipt into an executable empty input.
jobs.reclaim_input(
&saved.request.job_id,
&saved.input_digest,
JobInputRetirementReason::Completed,
)
.map_err(service_error)?;
}
}
if retained.is_some() {
break;
}
after_job = page.last().map(|job| job.request.job_id.clone());
}
if retained.is_some() {
break;
}
}
let input = match retained {
Some(input) => input,
None => self.build_journal_input(&context, &journal.id, None)?,
};
if input.selected_ranges.is_empty() {
continue;
}
let ranges = materials
.into_iter()
.map(|material| material.range)
.collect::<Vec<_>>();
let origins = self.journal_origins(&ranges)?;
let grant = LocalJournalGrant {
writer_binding_id: Some(self.writer.binding_id.clone()),
binding: self.binding.clone(),
session_id: journal.source_session_id,
journal_id: journal.id.clone(),
ranges,
origins,
};
let request = job::JobRequest {
job_id: format!("subjektiv-journal:{}", uuid::Uuid::now_v7()),
purpose: PURPOSE.into(), input_ref: format!("subjektiv://{}/journal/{}", self.binding.subject_id, journal.id),
input: grant.input(),
instruction: "ReadJournalState first. Input descriptors are the frozen canonical public ranges authorized for this Job, not already read material. Read every selected source byte with bounded ReadJournalText before recording; then read initial/current position and open text in bounded pages. Reconcile saved operation receipts before new creates. Save sequential Create/Rewrite/Fix and UpdateJournalPosition operations, then CheckpointJournal backed by saved record/no-change receipts for all selected input. Do not fix an ongoing event just because this Job/turn ended. SubmitJobResult with result: {} only after the entire selected prefix is reflected. Never infer tool success or user intent not evidenced by public input.".into(),
profile: "builtin:subjektiv-journal-recording".into(),
serialization_key: Some(format!("subjektiv-journal:{}", journal.id)),
limits: job::JobLimits { timeout_seconds: 600, ..Default::default() },
ranges: input.selected_ranges.clone(),
origins: vec![],
};
let request = self.journal_request(&journal.id, &input)?;
let saved = jobs
.request_granted(
request,
@@ -727,6 +797,138 @@ impl LocalSubjectHost {
}
}
fn replace_legacy_journal_job(
&self,
jobs: &crate::jobs::StandaloneJobs,
old: &JobSnapshot,
) -> Result<Option<String>, SubjektivHostError> {
let grant = self.journal_grant_scope(old)?;
if old.input_disposition == JobInputDisposition::Available {
old.load_executable_request()
.map_err(|e| SubjektivHostError::InvalidArgument(e.to_string()))?;
if old.request.input
!= serde_json::json!({"selected_ranges":grant.ranges,"origins":grant.origins})
{
return Err(SubjektivHostError::InvalidArgument(
"invalid legacy Journal descriptor input".into(),
));
}
} else if old.input_disposition != JobInputDisposition::Invalidated {
return Err(SubjektivHostError::InvalidArgument(
"legacy Journal input is unavailable".into(),
));
}
let context = authority(self, &grant.session_id)?;
if let Some(active) = self
.catalog
.store
.active_journal_generation(&context, &grant.journal_id)
.map_err(journal_error)?
{
if active.job.job_id != old.request.job_id {
return Err(SubjektivHostError::InvalidArgument(
"another generation owns legacy migration".into(),
));
}
self.catalog
.store
.close_journal_generation(&context, &active.id, &active.job)
.map_err(journal_error)?;
}
jobs.cancel(&old.request.job_id).map_err(service_error)?;
let current = self
.catalog
.store
.read_journal(&context, &grant.journal_id)
.map_err(journal_error)?;
let end = grant
.ranges
.last()
.ok_or_else(|| SubjektivHostError::InvalidArgument("empty legacy selection".into()))?;
let end_position = if end.bytes.end == end.source.total_bytes {
(end.source.order + 1, 0)
} else {
(end.source.order, end.bytes.end)
};
let replacement = if (current.process_order, current.process_byte) >= end_position {
None
} else {
let mut capture = self.recording_capture(&grant.session_id)?;
capture
.entries
.retain(|entry| grant.ranges.iter().any(|r| r.source.order == entry.order));
for entry in &mut capture.entries {
let range = grant
.ranges
.iter()
.find(|r| r.source.order == entry.order)
.unwrap();
let session_store::capture::CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
..
} = &mut entry.body
else {
unreachable!()
};
if source_descriptor(&grant.session_id, descriptor) != range.source
|| !grant.origins.iter().any(|origin| {
origin.pointer == range.source.pointer && origin.origin == descriptor.origin
})
|| !text.is_char_boundary(range.bytes.start as usize)
|| !text.is_char_boundary(range.bytes.end as usize)
{
return Err(SubjektivHostError::InvalidArgument(
"legacy original source identity/range changed".into(),
));
}
*text = text[range.bytes.start as usize..range.bytes.end as usize].to_owned();
*byte_offset = range.bytes.start;
}
let input = self.build_journal_input(&context, &grant.journal_id, Some(&capture))?;
let request = self.journal_request(&grant.journal_id, &input)?;
let next_grant = LocalJournalGrant {
ranges: input.selected_ranges.clone(),
origins: vec![],
..grant
};
Some(
jobs.request_granted(
request,
serde_json::to_value(next_grant)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?,
)
.map_err(service_error)?
.request
.job_id,
)
};
jobs.reclaim_input(
&old.request.job_id,
&old.input_digest,
JobInputRetirementReason::LegacyInvalidation,
)
.map_err(service_error)?;
Ok(replacement)
}
fn journal_request(
&self,
journal_id: &str,
input: &JournalInput,
) -> Result<job::JobRequest, SubjektivHostError> {
Ok(job::JobRequest {
job_id: format!("subjektiv-journal:{}", uuid::Uuid::now_v7()), purpose: PURPOSE.into(),
input_ref: format!("subjektiv://{}/journal/{journal_id}", self.binding.subject_id),
input: serde_json::to_value(input).map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?,
instruction: "Input contains the frozen canonical public source bodies, provenance and byte ranges. Use only this saved input, never re-fetch Source. ReadJournalState first and read initial/current Journal position/open text in bounded pages if needed. Reconcile saved operation receipts before new creates. Save sequential Create/Rewrite/Fix and UpdateJournalPosition operations, then CheckpointJournal backed by saved record/no-change receipts for all selected input. Do not fix an ongoing event just because this Job/turn ended. SubmitJobResult with result: {} only after the entire selected prefix is reflected. Never infer tool success or user intent not evidenced by public input.".into(),
profile: "builtin:subjektiv-journal-recording".into(),
serialization_key: Some(format!("subjektiv-journal:{journal_id}")),
limits: job::JobLimits { timeout_seconds: 600, ..Default::default() },
})
}
fn pin_journal_session(&self, session: &str) -> Result<(), SubjektivHostError> {
let path = crate::subjektiv_sessions::authorized_sessions_path(
&self.catalog.state_root,
@@ -869,11 +1071,16 @@ impl LocalSubjectHost {
Ok(())
}
fn material_reader(
fn with_recording_boundary<T>(
&self,
session: &str,
) -> Result<session_store::public_material::SessionPublicMaterialReader, SubjektivHostError>
{
read: impl FnOnce(
&Path,
session_store::SessionId,
session_store::SegmentId,
&session_store::retention::SessionRetentionGuard,
) -> Result<T, SubjektivHostError>,
) -> Result<T, SubjektivHostError> {
let sessions = crate::subjektiv_sessions::authorized_sessions_path(
&self.catalog.state_root,
&self.binding.scope_id,
@@ -906,6 +1113,11 @@ impl LocalSubjectHost {
.ok_or_else(|| SubjektivHostError::InvalidArgument("invalid Session storage".into()))?
.join("worker")
.join(&record.storage_key);
let session_id = session
.parse()
.map_err(|_| SubjektivHostError::InvalidArgument("invalid Session".into()))?;
let retention = crate::store::session_retention(&self.catalog.state_root);
let guard = retention.lock_live(session_id).map_err(source_error)?;
// This is the persisted active selector, not the newest filename/mtime or a model claim.
let metadata =
session_store::WorkerAggregateStore::open_read_only(metadata_root, &record.storage_key)
@@ -933,154 +1145,203 @@ impl LocalSubjectHost {
"no attested active lineage for the retained Session".into(),
)
})?;
session_store::public_material::SessionPublicMaterialReader::open_fs_with_retention(
&sessions,
session
.parse()
.map_err(|_| SubjektivHostError::InvalidArgument("invalid Session".into()))?,
active,
session_store::public_material::PublicMaterialLimits {
max_segments: 256,
max_entries: 100_000,
max_record_bytes: 64 * 1024 * 1024,
},
crate::store::session_retention(&self.catalog.state_root),
)
.map_err(|error| SubjektivHostError::Unavailable(error.to_string()))
read(&sessions, session_id, active, &guard)
}
fn recording_capture(&self, session: &str) -> Result<CaptureDto, SubjektivHostError> {
self.with_recording_boundary(session, |sessions, session_id, active, guard| {
session_store::capture::capture_recording_fs_guarded(
sessions,
session_id,
active,
session_store::public_material::PublicMaterialLimits::default(),
CaptureLimits {
max_bytes: 128 * 1024 * 1024,
max_entries: 100_000,
max_text_bytes: 64 * 1024 * 1024,
..Default::default()
},
guard,
)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))
})
}
fn journal_sources(
&self,
session: &str,
) -> Result<Vec<JournalSourceRange>, SubjektivHostError> {
let reader = self.material_reader(session)?;
Ok(reader
.descriptors()
.map(|descriptor| JournalSourceRange {
source: source_descriptor(session, descriptor),
bytes: JournalByteRange {
start: 0,
end: descriptor.total_bytes,
self.with_recording_boundary(session, |sessions, session_id, active, guard| {
session_store::capture::recording_catalog_fs_guarded(
sessions,
session_id,
active,
session_store::public_material::PublicMaterialLimits::default(),
guard,
|descriptors| {
descriptors
.iter()
.map(|descriptor| JournalSourceRange {
source: source_descriptor(session, descriptor),
bytes: JournalByteRange {
start: 0,
end: descriptor.total_bytes,
},
})
.collect()
},
})
.collect())
}
fn source_reader(
&self,
source: &JournalSource,
) -> Result<session_store::public_material::SessionPublicMaterialReader, SubjektivHostError>
{
let sessions = crate::subjektiv_sessions::authorized_sessions_path(
&self.catalog.state_root,
&self.binding.scope_id,
&self.binding.subject_id,
&self.catalog.store,
&source.session_id,
)?;
// Frozen, already admitted original pointers do not follow the Worker's later active
// Session/Segment. This reads that original lineage and still checks descriptor identity,
// canonical order and digest below; the shared service enforces the frozen byte range.
session_store::public_material::SessionPublicMaterialReader::open_fs_with_retention(
&sessions,
source
.session_id
.parse()
.map_err(|_| SubjektivHostError::InvalidArgument("invalid Session".into()))?,
source.pointer.segment_id,
session_store::public_material::PublicMaterialLimits {
max_segments: 256,
max_entries: 100_000,
max_record_bytes: 64 * 1024 * 1024,
},
crate::store::session_retention(&self.catalog.state_root),
)
.map_err(|error| SubjektivHostError::Unavailable(error.to_string()))
}
fn journal_origins(
&self,
ranges: &[JournalSourceRange],
) -> Result<Vec<SourceOrigin>, SubjektivHostError> {
if ranges.is_empty() {
return Err(SubjektivHostError::InvalidArgument(
"empty material selection".into(),
));
}
ranges
.iter()
.map(|range| {
let reader = self.source_reader(&range.source)?;
reader
.descriptors()
.find(|descriptor| {
source_descriptor(&range.source.session_id, descriptor) == range.source
})
.map(|descriptor| SourceOrigin {
pointer: range.source.pointer.clone(),
origin: descriptor.origin.clone(),
})
.ok_or_else(|| {
SubjektivHostError::InvalidArgument(
"source provenance/identity changed".into(),
)
})
})
.collect()
}
fn journal_source_window(
&self,
source: &JournalSource,
offset: u64,
max_bytes: usize,
) -> Result<String, SubjektivHostError> {
let reader = self.source_reader(source)?;
let descriptor = reader.descriptors().find(|descriptor| source_descriptor(&source.session_id, descriptor) == *source)
.ok_or_else(|| SubjektivHostError::InvalidArgument("canonical source identity/order/digest changed or is outside the authorized Session".into()))?;
reader
.read_content(descriptor, offset, max_bytes)
.map(|window| window.content)
.map_err(|error| SubjektivHostError::Unavailable(error.to_string()))
}
fn journal_source_material(
&self,
source: &JournalSource,
start: u64,
limit: usize,
) -> JournalResult<JournalMaterial> {
let text = self
.journal_source_window(source, start, limit.max(4))
.map_err(|error| JournalError::Conflict(error.to_string()))?;
Ok(JournalMaterial {
range: JournalSourceRange {
source: source.clone(),
bytes: JournalByteRange {
start,
end: start + text.len() as u64,
},
},
text,
)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))
})
}
fn select_journal_materials(
fn build_journal_input(
&self,
context: &JournalHostContext,
id: &str,
) -> Result<Vec<JournalMaterial>, SubjektivHostError> {
subjektiv::journal::recording::select_materials_bounded(
&self.catalog.store,
context,
id,
INPUT_BUDGET,
|source, start, limit| self.journal_source_material(source, start, limit),
)
saved: Option<&CaptureDto>,
) -> Result<JournalInput, SubjektivHostError> {
let session = self
.catalog
.store
.read_journal(context, id)
.map_err(journal_error)?
.source_session_id;
JournalInput::build(&self.catalog.store, context, id, |order, offset, budget| {
let capture = match saved {
Some(capture) => capture.clone(),
None => {
let capture = self
.with_recording_boundary(&session, |sessions, session_id, active, guard| {
session_store::capture::capture_recording_slice_fs_guarded(
sessions,
session_id,
active,
session_store::public_material::PublicMaterialLimits::default(),
RecordingPosition {
canonical_order: order,
byte_offset: offset,
},
MAX_JOURNAL_RANGES,
budget,
guard,
)
.map(|slice| slice.capture)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))
})
.map_err(|e| JournalError::Conflict(e.to_string()))?;
// Admission and capture are separate saves. An append between them
// may be in replay but has not yet been accepted for this Journal.
return self.accepted_capture_prefix(context, id, capture);
}
};
let end = capture
.entries
.last()
.map(|entry| {
capture
.recording_text(entry.order)
.map(|(descriptor, start, text)| {
if start + text.len() as u64 == descriptor.total_bytes {
RecordingPosition {
canonical_order: entry.order + 1,
byte_offset: 0,
}
} else {
RecordingPosition {
canonical_order: entry.order,
byte_offset: start + text.len() as u64,
}
}
})
})
.transpose()
.map_err(|e| JournalError::Conflict(e.to_string()))?;
if end.is_none_or(|end| (order, offset) >= (end.canonical_order, end.byte_offset)) {
return Ok(CaptureDto {
entries: vec![],
..capture
});
}
capture
.recording_slice(
RecordingPosition {
canonical_order: order,
byte_offset: offset,
},
MAX_JOURNAL_RANGES,
budget,
)
.map(|slice| slice.capture)
.map_err(|e| JournalError::Conflict(e.to_string()))
})
.map_err(journal_error)
}
fn journal_grant(
fn accepted_capture_prefix(
&self,
context: &JournalHostContext,
id: &str,
mut capture: CaptureDto,
) -> JournalResult<CaptureDto> {
let mut selected = Vec::new();
for mut entry in capture.entries {
let status = match self
.catalog
.store
.read_journal_source(context, id, entry.order)
{
Ok(status) => status,
Err(JournalError::NotFound(_)) => break,
Err(error) => return Err(error),
};
let session_store::capture::CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
..
} = &mut entry.body
else {
return Err(JournalError::InvalidInput("non-recording capture".into()));
};
if source_descriptor(&capture.session_id, descriptor) != status.source {
return Err(JournalError::Conflict(
"captured source differs from admitted identity".into(),
));
}
let Some(accepted) = status
.accepted
.iter()
.find(|range| range.start <= *byte_offset && *byte_offset < range.end)
else {
break;
};
let end = status
.unavailable
.iter()
.filter(|range| range.end > *byte_offset)
.fold(accepted.end, |end, range| {
end.min(range.start.max(*byte_offset))
});
let mut length = text.len().min(end.saturating_sub(*byte_offset) as usize);
while !text.is_char_boundary(length) {
length -= 1;
}
if length == 0 {
break;
}
text.truncate(length);
let partial = *byte_offset + (text.len() as u64) < descriptor.total_bytes;
selected.push(entry);
if partial {
break;
}
}
capture.entries = selected;
Ok(capture)
}
fn journal_grant_scope(
&self,
snapshot: &JobSnapshot,
) -> Result<LocalJournalGrant, SubjektivHostError> {
@@ -1097,7 +1358,6 @@ impl LocalSubjectHost {
|| snapshot.request.purpose != PURPOSE
|| snapshot.request.serialization_key.as_deref()
!= Some(&format!("subjektiv-journal:{}", grant.journal_id))
|| snapshot.request.input != grant.input()
{
return Err(SubjektivHostError::InvalidArgument(
"Job input/grant does not match this Journal capability".into(),
@@ -1135,12 +1395,33 @@ impl LocalSubjectHost {
Ok(grant)
}
fn journal_grant(
&self,
snapshot: &JobSnapshot,
) -> Result<LocalJournalGrant, SubjektivHostError> {
snapshot
.load_executable_request()
.map_err(|e| SubjektivHostError::InvalidArgument(e.to_string()))?;
let grant = self.journal_grant_scope(snapshot)?;
let input = JournalInput::decode(&snapshot.request.input).map_err(journal_error)?;
if input.subject_id != self.binding.subject_id
|| input.journal_id != grant.journal_id
|| input.capture.session_id != grant.session_id
|| input.selected_ranges != grant.ranges
{
return Err(SubjektivHostError::InvalidArgument(
"saved Journal body identity/ranges differ from grant".into(),
));
}
Ok(grant)
}
fn complete_journal_job(
&self,
jobs: &crate::jobs::StandaloneJobs,
snapshot: &JobSnapshot,
) -> Result<(), SubjektivHostError> {
let grant = self.journal_grant(snapshot)?;
let grant = self.journal_grant_scope(snapshot)?;
let context = authority(self, &grant.session_id)?;
let completion: JournalJobCompletion =
serde_json::from_value(snapshot.result.clone().ok_or_else(|| {
@@ -1174,6 +1455,12 @@ impl LocalSubjectHost {
}
jobs.acknowledge(&snapshot.request.job_id)
.map_err(service_error)?;
jobs.reclaim_input(
&snapshot.request.job_id,
&snapshot.input_digest,
JobInputRetirementReason::Completed,
)
.map_err(service_error)?;
if let Err(error) = self.prune_retained_workers() {
// Pruning is independent of a successful checkpoint/result. Keep the originals
// and continue draining other saved prefixes; a retention error is not model failure.
@@ -1192,47 +1479,18 @@ pub(super) fn grant(
.journal_grant(snapshot)
.map_err(|error| error.to_string())?;
let context = authority(&host, &grant.session_id).map_err(|error| error.to_string())?;
// Re-fetch selected identities before opening mutation authority. Only descriptors are
// retained in Job storage; the model must fetch actual source bytes with bounded tools.
for range in &grant.ranges {
let length = usize::try_from(
range
.bytes
.end
.checked_sub(range.bytes.start)
.ok_or("invalid source range")?,
)
.map_err(|_| "source range overflow")?;
if length > INPUT_BUDGET {
return Err("source range exceeds frozen input budget".into());
}
let text = host
.journal_source_window(&range.source, range.bytes.start, length.max(4))
.map_err(|error| error.to_string())?;
if text.len() != length {
return Err("frozen Journal source byte range changed".into());
}
}
if host
.journal_origins(&grant.ranges)
.map_err(|error| error.to_string())?
!= grant.origins
{
return Err(
"frozen Journal provenance does not match the authorized original Session".into(),
);
}
let input = JournalInput::decode(&snapshot.request.input).map_err(|e| e.to_string())?;
let input_digest = snapshot.input_digest.clone();
let generation = host
.catalog
.store
.prepare_journal_generation(
.prepare_journal_input(
&context,
&grant.journal_id,
&input,
JobAttemptBinding {
job_id: snapshot.request.job_id.clone(),
attempt_id: snapshot.attempt.attempt_id.clone(),
},
grant.ranges.clone(),
)
.map_err(|error| error.to_string())?;
let job_context = JournalJobContext::validated(
@@ -1242,12 +1500,17 @@ pub(super) fn grant(
generation.job.clone(),
)
.map_err(|error| error.to_string())?;
input
.validate_generation(&generation)
.map_err(|e| e.to_string())?;
let adapter = Arc::new(LocalJournalHost {
host,
grant,
context,
job_context,
#[cfg(test)]
generation,
input_digest,
fence,
});
Ok(worker::job::JobFeatureGrant {
@@ -1263,7 +1526,9 @@ struct LocalJournalHost {
grant: LocalJournalGrant,
context: JournalHostContext,
job_context: JournalJobContext,
#[cfg(test)]
generation: JournalGeneration,
input_digest: String,
fence: Arc<dyn crate::jobs::JobAttemptFence>,
}
impl LocalJournalHost {
@@ -1296,18 +1561,20 @@ impl JournalJobHost for LocalJournalHost {
request: JournalJobRequest,
) -> Result<JournalJobResponse, SubjektivHostError> {
let context = self.live()?;
subjektiv::journal::service::execute_bounded(
// The generic runner has supplied this exact immutable input before it can
// invoke this bound tool. Preparation alone is not evidence of delivery.
subjektiv::journal::service::attest_input_delivery(
&self.host.catalog.store,
&context,
&self.input_digest,
&self.grant.ranges,
)
.map_err(journal_error)?;
subjektiv::journal::service::execute_snapshot(
&self.host.catalog.store,
&context,
&self.input_digest,
request,
|source, start, limit| {
if source.session_id != self.generation.initial_journal.source_session_id {
return Err(JournalError::PermissionDenied(
"foreign original Session".into(),
));
}
self.host.journal_source_material(source, start, limit)
},
)
.map_err(journal_error)
}
File diff suppressed because it is too large Load Diff
@@ -215,6 +215,7 @@ async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_a
}
protocol::Event::TextDelta { text } if text.contains(PARTIAL) => break,
protocol::Event::RunEnd { .. } => panic!("dialog ended before cancellation"),
protocol::Event::SessionEntryCommitted { entry } if matches!(entry.data, protocol::SessionSnapshotEntryData::RunError { .. }) => panic!("dialog failed before cancellation: {entry:?}"),
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
@@ -294,6 +295,32 @@ async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_a
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
for history in raw.iter().filter_map(|record| match record {
session_store::LogEntry::AnnotatedSegmentStart { history, compacted_from: Some(_), .. } => Some(history),
_ => None,
}) {
let mut calls = std::collections::HashSet::new();
let mut results = 0;
for entry in history {
let is_tool = matches!(entry.item, session_store::LoggedItem::ToolCall { .. } | session_store::LoggedItem::ToolResult { .. });
let Some(metadata) = entry.metadata.input_metadata.as_ref().filter(|_| is_tool) else { continue; };
assert_eq!(metadata.usage().recording_material, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().memory_evidence, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().observation, session_store::InputObservationUse::Hidden);
assert!(metadata.public_item().is_none());
assert!(entry.metadata.derivation.is_some());
match &entry.item {
session_store::LoggedItem::ToolCall { call_id, .. } => { calls.insert(call_id.clone()); }
session_store::LoggedItem::ToolResult { call_id, .. } => {
assert!(calls.contains(call_id), "derived Tool result must follow its real logical call");
results += 1;
}
_ => unreachable!(),
}
}
assert!(!calls.is_empty());
assert!(results >= calls.len());
}
assert_eq!(
raw.iter()
.filter(|entry| matches!(
@@ -0,0 +1,170 @@
//! Real LocalSubjectHost/Journal storage -> PreparedWorker -> compactor Request -> durable seed.
//! Only external LLM responses and the fixture's Job execution fence are substituted. The
//! in-flight generation is retained while its real adapter updates storage during generation.
//! Temporary storage and the parent fixture own cleanup; ordinary standalone lib tests select
//! this suite. Predetermined summaries attest transport and persistence, not model understanding.
use super::*;
use agen::llm_client::{
ClientError, LlmClient, Request,
event::{Event, ResponseStatus, StatusEvent},
types::Item,
};
use futures::Stream;
use session_store::{CombinedStore, FsWorkerStore, LogEntry};
use std::pin::Pin;
const BEFORE: &str = "FROZEN_POSITION: B partially implemented; tests not run; result unconfirmed";
const AFTER: &str = "NEXT_POSITION: background Journal changed during compactor generation";
const COMPLEMENT: &str =
"Unrecorded request: keep the user's constraint and finish the remaining checks.";
#[derive(Clone)]
struct MockClient {
adapter: Arc<LocalJournalHost>,
requests: Arc<Mutex<Vec<Request>>>,
updated: Arc<AtomicBool>,
}
#[async_trait::async_trait]
impl LlmClient for MockClient {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(
&self,
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>, ClientError> {
assert!(
request
.tools
.iter()
.any(|tool| tool.name == "write_summary")
);
let first = self.requests.lock().unwrap().is_empty();
self.requests.lock().unwrap().push(request);
let events = if first {
assert!(!self.updated.swap(true, Ordering::SeqCst));
let state = self.adapter.state().unwrap();
save(
&self.adapter,
"position-during-compaction",
&state.current_journal.state_token,
JournalAction::UpdatePosition {
current_position: AFTER.into(),
},
vec![],
);
vec![
Event::tool_use_start(0, "summary", "write_summary"),
Event::tool_input_delta(0, serde_json::json!({ "text": COMPLEMENT }).to_string()),
Event::tool_use_stop(0),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
} else {
vec![
Event::text_block_start(0),
Event::text_delta(0, "summary saved"),
Event::text_block_stop(0, None),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
};
Ok(Box::pin(futures::stream::iter(events.into_iter().map(Ok))))
}
}
#[tokio::test]
async fn local_host_compaction_freezes_pending_journal_through_request_and_durable_seed_during_update()
{
tokio::time::timeout(std::time::Duration::from_secs(30), async {
let fixture = Fixture::new("Keep the user's constraint; work remains unfinished and unconfirmed.");
let (adapter, _, job) = fixture.adapter();
let created = save(
&adapter,
"record-partial-work",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "A corrected to B".into(),
body: "B partially implemented; tests not run; result unconfirmed".into(),
ongoing: "Finish remaining checks".into(),
state: JournalEntryState::Open,
source_end: None,
correction_refs: vec![],
},
adapter.generation.selected_ranges.clone(),
);
save(&adapter, "position-before-compaction", &created.state_token,
JournalAction::UpdatePosition { current_position: BEFORE.into() }, vec![]);
let before = adapter.state().unwrap();
let expected = fixture.host().journal_compaction_context(&fixture.context).unwrap();
let journal = &expected.journals[0];
assert_eq!(journal.source_session_id, fixture.context.session_id);
assert_eq!(journal.current_position, BEFORE);
assert_eq!(journal.entries[0].state, JournalEntryState::Open);
assert!(!before.selected_ranges.is_empty());
assert_eq!(journal.process_order, before.current_journal.process_order);
assert_eq!(journal.process_byte, before.current_journal.process_byte);
assert_eq!(journal.process_order, 0);
assert_eq!(journal.process_byte, 0, "saved open records do not advance processing checkpoint");
let expected_json = serde_json::to_string(&expected).unwrap();
let adapter = Arc::new(adapter);
let model = MockClient { adapter: adapter.clone(), requests: Arc::default(), updated: Arc::new(AtomicBool::new(false)) };
let workers = StandaloneWorkerStore::open_existing(&fixture.host().catalog.state_root).unwrap();
let worker_id = fixture._allocation.as_ref().unwrap().worker_id();
let store = CombinedStore::new(fixture.sessions.clone(), FsWorkerStore::new(workers.worker_metadata_dir(worker_id)).unwrap());
let mut manifest = manifest::WorkerManifest::from_toml(&format!("[worker]\nname='{}'\n[model]\nscheme='anthropic'\nmodel_id='scripted'\n[engine]\n[scope]\nallow=[]", fixture.context.worker_id)).unwrap();
manifest.feature.subjektiv.profile.enabled = true;
manifest.feature.subjektiv.profile.extraction.enabled = false;
let mut prepared = worker::WorkerBootstrap::new(
manifest,
store,
worker::PromptCatalogSource::builtins_only(),
worker::WorkerWorkspaceContext::no_workspace(),
worker::WorkerFilesystemAuthority::None,
worker::WorkerBootstrapLayout::Direct { runtime_base: fixture._temp.path().join("runtime") },
).with_model_client(model.clone()).with_subjektiv_host(fixture.host().clone()).prepare_restored("journal-test").await.unwrap();
assert_eq!(prepared.worker().session_id().to_string(), fixture.context.session_id);
let source_segment = prepared.worker().segment_id();
let original = fixture.sessions.read_all(prepared.worker().session_id(), source_segment).unwrap();
let generation = fixture.host().catalog.store.active_journal_generation(&adapter.context, &adapter.grant.journal_id).unwrap().unwrap();
assert!(!fixture.host().session_fully_reflected(&fixture.context.session_id).unwrap());
prepared.worker_mut().compact(0).await.unwrap();
assert!(model.updated.load(Ordering::SeqCst));
let worker = prepared.worker();
assert_ne!(worker.segment_id(), source_segment);
let seed = worker.store().read_all(worker.session_id(), worker.segment_id()).unwrap().into_iter().find_map(|entry| match entry { LogEntry::AnnotatedSegmentStart { history, .. } => Some(history), _ => None }).expect("durable compacted seed");
let journal_items: Vec<_> = seed.iter().filter(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::ReferenceInput)).collect();
assert_eq!(journal_items.len(), 1);
let journal_item = Item::from(journal_items[0].item.clone());
let body = journal_item.as_text().unwrap();
assert!(body.contains(&expected_json), "durable seed contains the full selected JSON and processing checkpoint");
assert!(body.contains(BEFORE));
assert!(!body.contains(AFTER));
assert!(seed.iter().any(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::OpeningSummary) && Item::from(entry.item.clone()).as_text().is_some_and(|text| text.contains(COMPLEMENT))));
let requests = model.requests.lock().unwrap();
let input = requests[0].items.iter().find_map(|item| match item {
Item::Message { role: agen::llm_client::types::Role::User, .. } => item.as_text().filter(|text| text.contains("## Conversation overview/index")),
_ => None,
}).expect("actual compactor user request");
assert!(input.ends_with(body), "exact same frozen reference goes to actual compactor and seed");
assert_eq!(input.matches(&expected_json).count(), 1);
assert!(!input.contains(AFTER));
drop(requests);
let latest = fixture.host().journal_compaction_context(&fixture.context).unwrap();
assert_eq!(latest.journals[0].current_position, AFTER);
assert_ne!(latest.journals[0].state_token, expected.journals[0].state_token);
assert_eq!(latest.journals[0].process_order, expected.journals[0].process_order);
assert_eq!(latest.journals[0].process_byte, expected.journals[0].process_byte);
assert_eq!(latest.journals[0].entries, expected.journals[0].entries);
assert_eq!(fixture.host().catalog.store.active_journal_generation(&adapter.context, &adapter.grant.journal_id).unwrap(), Some(generation));
assert_eq!(fixture.host().jobs().unwrap().get(&job.request.job_id).unwrap().state, job.state);
let unchanged: Vec<_> = fixture.sessions.read_all(worker.session_id(), source_segment).unwrap().into_iter().filter(|entry| !matches!(entry, LogEntry::Extension { domain, .. } if domain == "metrics")).collect();
assert_eq!(serde_json::to_value(unchanged).unwrap(), serde_json::to_value(original).unwrap());
assert!(!fixture.host().session_fully_reflected(&fixture.context.session_id).unwrap());
}).await.expect("compaction must finish without recording Job completion");
}
@@ -79,7 +79,7 @@ async fn recorded_idle_body_handoff_preserves_open_after_source_deletion_and_see
let sessions = session_store::FsStore::new_with_retention(workers.sessions_dir(second.worker_id()), crate::store::session_retention(&state)).unwrap();
let pointer = session_store::FsWorkerStore::new(workers.worker_metadata_dir(second.worker_id())).unwrap().read_by_name(&second.record().storage_key).unwrap().unwrap().active.unwrap();
let entries = sessions.read_all(session, pointer.segment_id.unwrap()).unwrap();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if matches!(entry.item, session_store::SystemItem::JournalStartup { .. }))).count();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if entry.metadata.input_metadata.as_ref().and_then(|input| input.feature_payload()).is_some_and(|payload| payload.schema() == "journal:startup"))).count();
assert_eq!(journal_items, 1);
assert_eq!(catalog.store.subject(&subject.id).unwrap().unwrap().behavior_md, "BEHAVIOR_ORIGINAL_UNCHANGED");
second.shutdown().await.unwrap();
@@ -1621,6 +1621,7 @@ allow = []
let history = |id: &str, item: LoggedItem| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction { operation_id: None },
derivation: None,
@@ -1693,6 +1694,7 @@ allow = []
// A valid JSON record without its final newline is not committed.
log.write_all(
&serde_json::to_vec(&LogEntry::RunCompleted {
entry_id: None,
ts: 4,
interrupted: false,
result: agen::EngineResult::Finished,
+296
View File
@@ -0,0 +1,296 @@
//! Optional operator transport owned by the Standalone Host, never by a Worker.
//!
//! The endpoint's existing parent must be private to the current Unix user. Binding never
//! removes an existing entry, including stale sockets. Recovery is an explicit operator action.
//! Peers must have that same uid; admitted methods still cross the normal local protocol
//! authorization boundary. Disconnect only drops a view, not the Host or its execution lease.
use std::io;
use std::os::unix::fs::{FileTypeExt, MetadataExt, PermissionsExt};
use std::path::{Path, PathBuf};
use client::transport::Socket;
use protocol::stream::{JsonLineWriter, decode_event, decode_method};
use tokio::io::{AsyncBufReadExt, BufReader};
use tokio::net::{UnixListener, UnixStream};
use tokio::sync::oneshot;
use tokio::task::{JoinHandle, JoinSet};
/// A bound endpoint reservation. Drop removes only the exact socket created by this bind.
/// It can be reserved before starting a Host, so bind failures do not allocate Workers.
pub struct UnixEndpoint {
listener: UnixListener,
path: PathBuf,
device: u64,
inode: u64,
}
impl UnixEndpoint {
pub fn bind(path: impl AsRef<Path>) -> io::Result<Self> {
let path = path.as_ref();
let name = path.file_name().ok_or_else(|| {
io::Error::new(io::ErrorKind::InvalidInput, "endpoint requires a filename")
})?;
let parent = std::fs::canonicalize(
path.parent()
.filter(|p| !p.as_os_str().is_empty())
.unwrap_or(Path::new(".")),
)?;
let metadata = std::fs::metadata(&parent)?;
// SAFETY: geteuid has no preconditions and does not mutate process identity.
let uid = unsafe { libc::geteuid() };
if !metadata.is_dir() || metadata.uid() != uid || metadata.mode() & 0o077 != 0 {
return Err(io::Error::new(
io::ErrorKind::PermissionDenied,
"endpoint parent must be a current-user-owned private directory (mode 0700)",
));
}
let path = parent.join(name);
let listener = UnixListener::bind(&path)?;
let metadata = std::fs::symlink_metadata(&path)?;
let endpoint = Self {
listener,
path,
device: metadata.dev(),
inode: metadata.ino(),
};
std::fs::set_permissions(&endpoint.path, std::fs::Permissions::from_mode(0o600))?;
Ok(endpoint)
}
pub fn path(&self) -> &Path {
&self.path
}
}
impl Drop for UnixEndpoint {
fn drop(&mut self) {
if let Ok(current) = std::fs::symlink_metadata(&self.path)
&& current.file_type().is_socket()
&& current.dev() == self.device
&& current.ino() == self.inode
{
let _ = std::fs::remove_file(&self.path);
}
}
}
pub(crate) struct UnixAdapter {
task: Option<JoinHandle<io::Result<()>>>,
stop: Option<oneshot::Sender<()>>,
}
impl UnixAdapter {
pub(crate) fn start(endpoint: UnixEndpoint, handle: worker::WorkerHandle) -> Self {
let (stop, mut stopped) = oneshot::channel();
let task = tokio::spawn(async move {
let mut connections = JoinSet::new();
let result = loop {
tokio::select! {
_ = &mut stopped => break Ok(()),
accepted = endpoint.listener.accept(), if connections.len() < 64 => {
match accepted {
Ok((stream, _)) => {
// The filesystem gate protects connection establishment; credentials
// also fence open connections to the Host's operator identity.
let allowed = stream.peer_cred().is_ok_and(|cred| {
// SAFETY: geteuid has no preconditions.
cred.uid() == unsafe { libc::geteuid() }
});
if allowed {
let socket = crate::host::connect_protocol(handle.clone());
connections.spawn(bridge(stream, socket));
}
}
Err(error) => break Err(error),
}
}
_ = connections.join_next(), if !connections.is_empty() => {}
}
};
connections.shutdown().await;
// Keep the bound endpoint alive until every connection has been reclaimed.
drop(endpoint);
result
});
Self {
task: Some(task),
stop: Some(stop),
}
}
pub(crate) async fn ended(&mut self) -> io::Result<()> {
let result = self.task.as_mut().expect("adapter task").await;
self.task.take();
result.map_err(io::Error::other)?
}
pub(crate) async fn shutdown(mut self) -> io::Result<()> {
if let Some(stop) = self.stop.take() {
let _ = stop.send(());
}
if self.task.is_some() {
self.ended().await?;
}
Ok(())
}
}
impl Drop for UnixAdapter {
fn drop(&mut self) {
if let Some(stop) = self.stop.take() {
let _ = stop.send(());
}
// Sending stop lets the task join its connection set and drop its endpoint.
// A Host dropped without confirmed shutdown keeps the production execution lease.
}
}
async fn bridge(
stream: UnixStream,
mut socket: client::transport::in_process::Socket,
) -> io::Result<()> {
let (reader, writer) = stream.into_split();
let mut reader = BufReader::new(reader).lines();
let mut writer = JsonLineWriter::new(writer);
loop {
tokio::select! {
line = reader.next_line() => {
let Some(message) = line? else { return Ok(()); };
if message.trim().is_empty() { continue; }
socket.send(message).await.map_err(io::Error::other)?;
}
event = socket.next() => {
let Some(message) = event.map_err(io::Error::other)? else { return Ok(()); };
let event = decode_event(&message).map_err(io::Error::other)?;
writer.write_event(&event).await?;
}
}
}
}
/// Connect a JSONL operator client. EOF disconnects without issuing Shutdown.
/// Input is decoded as Worker Methods and output contains Worker Events only.
pub async fn run_unix_client(endpoint: PathBuf) -> io::Result<()> {
relay_unix_client(endpoint, tokio::io::stdin(), tokio::io::stdout()).await
}
/// The CLI's stream relay, also usable by embedders that own their input/output streams.
pub async fn relay_unix_client<R, W>(endpoint: PathBuf, input: R, output: W) -> io::Result<()>
where
R: tokio::io::AsyncRead + Unpin,
W: tokio::io::AsyncWrite + Unpin,
{
let stream = UnixStream::connect(endpoint).await?;
let (reader, writer) = stream.into_split();
let mut input = BufReader::new(input).lines();
let mut to_host = JsonLineWriter::new(writer);
let mut from_host = BufReader::new(reader).lines();
let mut output = JsonLineWriter::new(output);
loop {
tokio::select! {
line = input.next_line() => {
let Some(line) = line? else { return Ok(()); };
if line.trim().is_empty() { continue; }
let method = decode_method(&line).map_err(io::Error::other)?;
to_host.write(&method).await?;
}
line = from_host.next_line() => {
let Some(line) = line? else { return Ok(()); };
let event = decode_event(&line).map_err(io::Error::other)?;
output.write_event(&event).await?;
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn endpoint_requires_private_parent_and_is_owner_only() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("host.sock");
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o755)).unwrap();
let error = UnixEndpoint::bind(&path)
.err()
.expect("public parent rejected");
assert_eq!(error.kind(), io::ErrorKind::PermissionDenied);
assert!(!path.exists());
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let endpoint = UnixEndpoint::bind(&path).unwrap();
assert_eq!(
std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
0o600
);
let client = UnixStream::connect(&path).await.unwrap();
let (server, _) = endpoint.listener.accept().await.unwrap();
assert_eq!(
client.peer_cred().unwrap().uid(),
server.peer_cred().unwrap().uid()
);
drop(endpoint);
assert!(!path.exists());
}
#[tokio::test]
async fn bind_failure_precedes_worker_state_and_startup_failure_releases_only_its_endpoint() {
let directory = tempfile::tempdir().unwrap();
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let path = directory.path().join("host.sock");
let owner = UnixEndpoint::bind(&path).unwrap();
let state_dir = directory.path().join("must-not-be-opened");
let error = crate::run_headless(
crate::HeadlessLaunch::Restore {
state_dir: state_dir.clone(),
worker_id: protocol::WorkerId::now_v7(),
},
path.clone(),
)
.await
.unwrap_err();
assert!(matches!(error, crate::HeadlessError::Transport(_)));
assert!(!state_dir.exists());
assert!(path.exists());
drop(owner);
let error = crate::run_headless(
crate::HeadlessLaunch::Restore {
state_dir,
worker_id: protocol::WorkerId::now_v7(),
},
path.clone(),
)
.await
.unwrap_err();
assert!(matches!(error, crate::HeadlessError::Startup(_)));
assert!(
!path.exists(),
"a failed Host startup must release its own bind"
);
}
#[tokio::test]
async fn bind_failure_and_old_endpoint_drop_preserve_another_owner() {
let directory = tempfile::tempdir().unwrap();
std::fs::set_permissions(directory.path(), std::fs::Permissions::from_mode(0o700)).unwrap();
let path = directory.path().join("host.sock");
let first = UnixEndpoint::bind(&path).unwrap();
let first_inode = std::fs::metadata(&path).unwrap().ino();
assert_eq!(
UnixEndpoint::bind(&path).err().unwrap().kind(),
io::ErrorKind::AddrInUse
);
assert_eq!(std::fs::metadata(&path).unwrap().ino(), first_inode);
std::fs::rename(&path, directory.path().join("retained.sock")).unwrap();
let second = UnixEndpoint::bind(&path).unwrap();
let second_inode = std::fs::metadata(&path).unwrap().ino();
assert_ne!(first_inode, second_inode);
drop(first);
assert_eq!(std::fs::metadata(&path).unwrap().ino(), second_inode);
let _client = UnixStream::connect(&path).await.unwrap();
let _connection = second.listener.accept().await.unwrap();
drop(second);
assert!(!path.exists());
}
}
+233
View File
@@ -608,6 +608,239 @@ async fn standalone_metadata_fails_closed_on_incomplete_or_newer_records() -> Te
Ok(())
}
#[tokio::test]
async fn headless_unix_client_uses_production_history_lease_and_local_authority() -> TestResult {
use protocol::stream::{JsonLineReader, encode_method};
use std::os::unix::fs::PermissionsExt;
use tokio::io::AsyncWriteExt;
let temp = tempfile::tempdir()?;
// Shorten only the owned endpoint's components, preserving the harness TMPDIR
// even when it is checkout-local. No process-global cwd or external /tmp fallback.
let socket_dir = tempfile::Builder::new()
.prefix("")
.rand_bytes(4)
.tempdir()?;
std::fs::set_permissions(socket_dir.path(), std::fs::Permissions::from_mode(0o700))?;
let endpoint_path = socket_dir.path().join("s");
let endpoint = standalone::UnixEndpoint::bind(&endpoint_path)?;
let state_dir = temp.path().join("state");
let launch = StandaloneLaunchConfig::new(
temp.path(),
&state_dir,
manifest::ProfileSelector::Default,
"headless-operator",
)
.resolve()?;
let model = ScriptedClient::new(vec![vec![
LlmEvent::text_block_start(0),
LlmEvent::text_delta(0, "headless production response"),
LlmEvent::text_block_stop(0, Some(StopReason::EndTurn)),
]]);
let inspection = model.clone();
let host = StandaloneHost::start_with_model_client(launch, model).await?;
let worker_id = host.worker_id();
let session_id = host.record().active_session_id;
let profile = host.record().manifest.clone();
let serve = tokio::spawn(host.serve_unix(endpoint));
// Exercise the real external client's stdin/stdout relay over the actual Unix adapter.
let (mut input, relay_input) = tokio::io::duplex(64 * 1024);
let (relay_output, output) = tokio::io::duplex(64 * 1024);
let relay = tokio::spawn(standalone::relay_unix_client(
endpoint_path.clone(),
relay_input,
relay_output,
));
let mut events = JsonLineReader::new(output);
let snapshot = tokio::time::timeout(Duration::from_secs(10), events.next::<Event>())
.await??
.expect("snapshot");
assert!(matches!(snapshot, Event::Snapshot { .. }));
let request = Method::SubmitTracked {
submission_request_id: protocol::new_submission_request_id(),
input: vec![protocol::Segment::text("headless production input")],
source: protocol::AuthenticatedInputSource::Backend {
operation_id: "forged-backend-source".into(),
},
};
let wire = encode_method(&request)?;
println!("headless input: {wire}");
input.write_all(wire.as_bytes()).await?;
input.write_all(b"\n").await?;
let mut saw_user = false;
let mut saw_response = false;
let mut saw_state = false;
tokio::time::timeout(Duration::from_secs(20), async {
loop {
let event = events.next::<Event>().await?.expect("live event");
match &event {
Event::UserMessage { segments, .. } => {
saw_user |= format!("{segments:?}").contains("headless production input");
}
Event::TextDelta { text } => {
saw_response |= text == "headless production response";
}
Event::WorkerState { .. } => saw_state = true,
Event::RunEnd { .. } => break,
_ => {}
}
println!(
"headless output: {}",
protocol::stream::encode_event(&event)?
);
}
Ok::<_, Box<dyn std::error::Error>>(())
})
.await??;
assert!(saw_user && saw_response && saw_state);
assert_eq!(inspection.requests().len(), 1);
assert!(matches!(
StandaloneHost::restore_with_model_client(
state_dir.clone(),
worker_id,
ScriptedClient::new(vec![])
)
.await,
Err(StandaloneStartupError::WorkerActive)
));
// EOF is a disconnect, not a shutdown; another view can reconnect to the same history.
drop(input);
tokio::time::timeout(Duration::from_secs(10), relay).await???;
assert!(!serve.is_finished());
let mut malformed = tokio::net::UnixStream::connect(&endpoint_path).await?;
malformed.write_all(b"not-json\n").await?;
let mut discarded = Vec::new();
use tokio::io::AsyncReadExt;
tokio::time::timeout(
Duration::from_secs(10),
malformed.read_to_end(&mut discarded),
)
.await??;
assert!(
!serve.is_finished(),
"invalid client input must not stop the Host"
);
let mut socket_client =
Client::new(client::transport::unix_socket::Socket::connect(&endpoint_path).await?);
let snapshot = tokio::time::timeout(Duration::from_secs(10), async {
loop {
let event = socket_client.next_event().await?.expect("reconnect event");
if matches!(event, Event::Snapshot { .. }) {
break Ok::<_, Box<dyn std::error::Error>>(event);
}
}
})
.await??;
let snapshot_text = format!("{snapshot:?}");
assert!(
snapshot_text.contains("headless production input"),
"{snapshot_text}"
);
assert!(
snapshot_text.contains("headless production response"),
"{snapshot_text}"
);
socket_client
.send(&Method::Shutdown {
command: protocol::WorkerCommandEnvelope::new(0),
})
.await?;
tokio::time::timeout(Duration::from_secs(10), async {
loop {
if let Some(Event::CommandAcknowledged { acknowledgement }) =
socket_client.next_event().await?
{
assert_eq!(
acknowledgement.disposition,
protocol::WorkerCommandDisposition::StaleCommandId
);
break;
}
}
Ok::<_, Box<dyn std::error::Error>>(())
})
.await??;
assert!(
!serve.is_finished(),
"stale Shutdown must not bypass Controller admission"
);
socket_client
.send(&Method::Shutdown {
command: protocol::WorkerCommandEnvelope::new(100),
})
.await?;
tokio::time::timeout(Duration::from_secs(15), serve).await???;
assert!(!endpoint_path.exists());
let store = StandaloneWorkerStore::open(&state_dir)?;
let saved = store.load(worker_id)?;
assert_eq!(saved.status, StandaloneWorkerStatus::Stopped);
use session_store::Store;
let history = session_store::FsStore::new(store.sessions_dir(worker_id))?.read_all(
saved.active_session_id,
saved.active_segment_id.expect("active segment"),
)?;
let origins = history
.iter()
.filter_map(|entry| match entry {
session_store::LogEntry::AnnotatedUserInput { history, .. } => Some(history),
_ => None,
})
.flatten()
.map(|entry| &entry.metadata.origin)
.collect::<Vec<_>>();
assert!(origins.contains(&&session_store::LoggedSessionHistoryOrigin::LocalHumanInput));
assert!(!format!("{origins:?}").contains("forged-backend-source"));
// This is the same restoration path called by the TUI, not a headless catalog/import.
let restored = StandaloneHost::restore_with_model_client(
state_dir.clone(),
worker_id,
ScriptedClient::new(vec![]),
)
.await?;
assert_eq!(restored.worker_id(), worker_id);
assert_eq!(restored.record().active_session_id, session_id);
assert_eq!(
serde_json::to_value(&restored.record().manifest)?,
serde_json::to_value(&profile)?
);
assert_eq!(
restored.record().cwd.canonical_path,
temp.path().canonicalize()?
);
let mut in_process = restored.connect();
let snapshot = tokio::time::timeout(Duration::from_secs(10), async {
loop {
let event = in_process
.next_event()
.await?
.expect("in-process restored event");
if matches!(event, Event::Snapshot { .. }) {
break Ok::<_, Box<dyn std::error::Error>>(event);
}
}
})
.await??;
assert!(format!("{snapshot:?}").contains("headless production response"));
assert!(
!endpoint_path.exists(),
"in-process restore must not recreate the adapter"
);
let runtime = state_dir
.join(worker_id.to_string())
.join("runtime")
.join(format!("standalone-{worker_id}"));
assert!(!runtime.join("sock").exists());
assert!(!runtime.join("pid").exists());
restored.shutdown().await?;
println!(
"verdict: input/state/response/disconnect/explicit shutdown, shared production restore and concurrent lease exclusion passed for {worker_id}"
);
Ok(())
}
async fn wait_for_run_end(client: &mut Client<InProcessSocket>) -> TestResult {
tokio::time::timeout(Duration::from_secs(10), async {
loop {
+21 -31
View File
@@ -479,38 +479,28 @@ fn journal_response(request: &Request) -> Vec<LlmEvent> {
}),
);
}
for (index, range) in state["selected_ranges"]
.as_array()
.unwrap()
let input: Value = request
.items
.iter()
.enumerate()
{
let mut offset = range["bytes"]["start"].as_u64().unwrap();
let mut token = Value::Null;
loop {
let id = format!("journal-source-{index}-{offset}");
match result(&id) {
Some(page) => match page["value"]["next_byte"].as_u64() {
Some(next) => {
assert!(next > offset);
offset = next;
token = page["value"]["content_token"].clone();
}
None => break,
},
None => {
return tool(
&id,
"ReadJournalText",
json!({
"target":{"target":"source","pointer":range["source"]["pointer"]},
"offset":offset,"limit":subjektiv::journal::MAX_JOURNAL_READ_BYTES,"content_token":token
}),
);
}
}
}
}
.filter_map(Item::as_text)
.find_map(|text| {
text.lines()
.find_map(|line| line.strip_prefix("input_json: "))
})
.map(|json| serde_json::from_str(json).unwrap())
.expect("saved direct Journal input");
let input = subjektiv::journal::recording::JournalInput::decode(&input).unwrap();
assert_eq!(
serde_json::to_value(&input.selected_ranges).unwrap(),
state["selected_ranges"]
);
assert!(
input
.capture
.entries
.iter()
.all(|entry| input.capture.recording_text(entry.order).is_ok())
);
let (name, action) = if state["current_open"].is_null() {
(
"CreateJournalEntry",
+2
View File
@@ -15,6 +15,8 @@ serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
sha2.workspace = true
server-api.workspace = true
session-store.workspace = true
job.workspace = true
thiserror.workspace = true
uuid = { workspace = true, features = ["v7"] }
+8
View File
@@ -1299,6 +1299,14 @@ pub type OperationResult<T> = std::result::Result<T, OperationError>;
pub fn subjektiv_store_error(error: crate::SubjektivError) -> OperationError {
match error {
crate::SubjektivError::AuthorizationDenied(reason) => {
OperationError::PermissionDenied(reason)
}
crate::SubjektivError::SubjectPermissionsConflict(_)
| crate::SubjektivError::PermissionOperationConflict(_)
| crate::SubjektivError::WorkerExecutionGrantConflict(_) => {
OperationError::Conflict(error.to_string())
}
crate::SubjektivError::SubjectScopeMismatch { .. } => {
OperationError::PermissionDenied(format!("subject_scope_mismatch: {error}"))
}
+100
View File
@@ -67,6 +67,27 @@ impl SubjektivStore {
})
}
/// Absence proof for failed body creation, including bindings never attached to a Session
/// and released lifetimes. The Host must hold its lifecycle fence across this observation
/// and recovery; absence alone does not fence a delayed Runtime create.
pub fn journal_body_is_unpublished(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row(
"SELECT NOT EXISTS(SELECT 1 FROM journal_writer_bindings
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)
AND NOT EXISTS(SELECT 1 FROM subject_session_attributions
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)",
params![subject, runtime, worker],
|row| row.get(0),
)?)
})
}
pub fn journal_worker_was_released(
&self,
subject: &str,
@@ -203,3 +224,82 @@ impl SubjektivStore {
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn unpublished_body_rejects_orphan_and_released_bindings_and_attribution_only() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _workspace, store) = crate::tests::open_store(temp.path(), "absence-tests");
let subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
let other_subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
// No Session was attached: an orphan binding is still published body evidence.
let binding = store
.claim_journal_writer(&subject.id, "runtime", "worker")
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
for (subject_id, runtime, worker) in [
(other_subject.id.as_str(), "runtime", "worker"),
(subject.id.as_str(), "other-runtime", "worker"),
(subject.id.as_str(), "runtime", "other-worker"),
] {
assert!(
store
.journal_body_is_unpublished(subject_id, runtime, worker)
.unwrap()
);
}
store.release_journal_writer(&binding).unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
store
.record_session_attribution(
SubjectSessionAttribution::new(
&subject.id,
"runtime",
"attribution-only",
"session",
)
.unwrap(),
)
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "other-runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&other_subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "never-published")
.unwrap()
);
}
}
+181
View File
@@ -0,0 +1,181 @@
//! Bounded saved-Journal browser projections. No history fetch, reconciliation or execution.
use super::store::{check_host, entry, fit_page, journal, summary};
use super::*;
use rusqlite::params;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct EntryCursor {
scope: String,
subject: String,
journal: Option<String>,
query: Option<String>,
correcting: Option<String>,
created_at: String,
id: String,
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
impl SubjektivStore {
pub fn journal_entry_summary(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
) -> JournalResult<JournalEntrySummary> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
summary(tx, entry(tx, journal_id, entry_id)?)
})
}
/// Correction targets are capped at sixteen by the writer contract, and may be in
/// another Journal of this Subject. Include actual Journal IDs rather than guessing.
pub fn journal_correction_targets(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
) -> JournalResult<Vec<JournalEntrySummary>> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
entry(tx, journal_id, entry_id)?;
let mut stmt = tx.prepare(
"SELECT e.journal_id,e.id FROM journal_corrections c
JOIN journal_entries e ON e.subject_id=c.subject_id AND e.id=c.target_id
WHERE c.subject_id=?1 AND c.entry_id=?2 ORDER BY e.created_at,e.id LIMIT 16",
)?;
let rows = stmt
.query_map(params![host.subject_id, entry_id], |r| {
Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?))
})?
.collect::<Result<Vec<_>, _>>()?;
rows.into_iter()
.map(|(journal_id, id)| summary(tx, entry(tx, &journal_id, &id)?))
.collect()
})
}
/// Recent entries use immutable creation time and ID as a stable descending keyset.
/// Search is literal substring matching (not SQL LIKE wildcards), bounded to 256 bytes.
/// A correcting selector filters saved correction edges, not inferred text or intervals.
pub fn browse_journal_entries(
&self,
host: &JournalHostContext,
journal_id: Option<&str>,
query: Option<&str>,
correcting: Option<&str>,
after: Option<&str>,
limit: usize,
) -> JournalResult<JournalPage<JournalEntrySummary>> {
if !(1..=MAX_JOURNAL_PAGE).contains(&limit) {
return Err(invalid("page limit must be 1..32"));
}
if query.is_some_and(|q| q.len() > 256 || q.chars().any(char::is_control)) {
return Err(invalid(
"search query must be at most 256 bytes without controls",
));
}
if correcting.is_some() && journal_id.is_none() {
return Err(invalid("correction lookup requires a Journal"));
}
let cursor: Option<EntryCursor> = after
.map(|raw| {
if raw.len() > 4096 {
return Err(invalid("entry cursor exceeds byte limit"));
}
let cursor: EntryCursor =
serde_json::from_str(raw).map_err(|_| invalid("invalid entry cursor"))?;
if cursor.scope != host.scope_id
|| cursor.subject != host.subject_id
|| cursor.journal.as_deref() != journal_id
|| cursor.query.as_deref() != query
|| cursor.correcting.as_deref() != correcting
{
return Err(invalid(
"entry cursor belongs to another scope, Subject or filter",
));
}
Ok(cursor)
})
.transpose()?;
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
if let Some(id) = journal_id {
journal(tx, host, id, false)?;
}
if let Some(id) = correcting {
entry(tx, journal_id.unwrap(), id)?;
}
let mut stmt = tx.prepare(
"SELECT e.journal_id,e.id,e.created_at FROM journal_entries e
WHERE e.subject_id=?1 AND (?4 IS NOT NULL OR ?2 IS NULL OR e.journal_id=?2)
AND (?3 IS NULL OR instr(lower(e.title),lower(?3))>0
OR instr(lower(e.body),lower(?3))>0 OR instr(lower(e.ongoing),lower(?3))>0)
AND (?4 IS NULL OR EXISTS(SELECT 1 FROM journal_corrections c
WHERE c.subject_id=?1 AND c.entry_id=e.id AND c.target_id=?4))
AND (?5 IS NULL OR (e.created_at,e.id)<(?5,?6))
ORDER BY e.created_at DESC,e.id DESC LIMIT ?7",
)?;
let rows = stmt
.query_map(
params![
host.subject_id,
journal_id,
query,
correcting,
cursor.as_ref().map(|c| &c.created_at),
cursor.as_ref().map(|c| &c.id),
limit + 1
],
|r| {
Ok((
r.get::<_, String>(0)?,
r.get::<_, String>(1)?,
r.get::<_, String>(2)?,
))
},
)?
.collect::<Result<Vec<_>, _>>()?;
let more = rows.len() > limit;
let mut items = Vec::new();
for (journal_id_value, id, created_at) in rows.into_iter().take(limit) {
let next = serde_json::to_string(&EntryCursor {
scope: host.scope_id.clone(),
subject: host.subject_id.clone(),
journal: journal_id.map(str::to_owned),
query: query.map(str::to_owned),
correcting: correcting.map(str::to_owned),
created_at,
id: id.clone(),
})?;
items.push((next, summary(tx, entry(tx, &journal_id_value, &id)?)?));
}
fit_page(items, more)
})
}
/// True only for admitted source bytes which the durable checkpoint has not reflected.
/// False does not claim that all original Session activity was admitted or recorded.
pub fn journal_has_unreflected_sources(
&self,
host: &JournalHostContext,
journal_id: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
Ok(tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_sources s, json_each(s.accepted_json) b
WHERE s.journal_id=?1 AND json_extract(b.value,'$.end')>s.reflected_bytes)",
[journal_id],
|r| r.get(0),
)?)
})
}
}
+1
View File
@@ -40,6 +40,7 @@
pub mod binding;
pub use binding::JournalWriterBinding;
pub(crate) use binding::migrate as migrate_binding;
pub mod browser;
mod lifecycle;
pub mod memory_links;
pub(crate) use lifecycle::migrate as migrate_lifecycle;
+213
View File
@@ -5,8 +5,221 @@
//! original canonical identity while streaming and return only the requested bounded fragment.
use super::*;
use session_store::capture::{CaptureDto, CaptureLimits};
use sha2::{Digest, Sha256};
/// A Job owns only this bounded original-body prefix. The initial content condition binds
/// the separate generation snapshot (whose potentially large Journal text remains paged).
/// Neither possessing these bytes nor delivering them is a processing checkpoint.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalInput {
pub version: u32,
pub subject_id: String,
pub journal_id: String,
pub initial_state_token: String,
pub initial_open_id: Option<String>,
pub previous_fixed_end: Option<JournalSourcePointer>,
pub selected_ranges: Vec<JournalSourceRange>,
pub capture: CaptureDto,
}
impl JournalInput {
/// Capture callback runs once for this immutable Job, never once per page. Fitting uses
/// only that captured value and measures the complete serialized input, including the
/// initial Journal condition and ranges duplicated beside the capture's descriptors.
pub fn build(
store: &SubjektivStore,
host: &JournalHostContext,
journal_id: &str,
capture_once: impl FnOnce(u64, u64, usize) -> JournalResult<CaptureDto>,
) -> JournalResult<Self> {
let (journal, open, previous_fixed_end) = store.database.try_with_connection(|tx| {
super::store::check_host(store, tx, host, true)?;
Ok::<_, JournalError>((
super::store::journal(tx, host, journal_id, true)?,
super::store::open(tx, journal_id)?,
super::store::previous_fixed(tx, journal_id)?.map(|p| p.0),
))
})?;
let mut input = Self {
version: 1,
subject_id: journal.subject_id.clone(),
journal_id: journal_id.into(),
initial_state_token: journal.state_token,
initial_open_id: open.map(|e| e.id),
previous_fixed_end,
selected_ranges: Vec::new(),
capture: CaptureDto {
version: 1,
session_id: journal.source_session_id,
segment_id: "pending".into(),
view_entry_count: 0,
entries: Vec::new(),
},
};
let envelope = serde_json::to_vec(&input)?.len();
// Capture once at a fixed storage boundary. Any adjustment for the Journal envelope
// below operates on this value only, never by making another live capture request.
let budget = job::MAX_JOB_INPUT_BYTES.saturating_sub(envelope);
input.capture = capture_once(journal.process_order, journal.process_byte, budget)?;
input.selected_ranges = capture_ranges(&input.capture)?;
if input.selected_ranges.is_empty() {
return Ok(input);
}
input.fit()?;
let value = serde_json::to_value(&input)?;
Self::decode(&value)
}
fn fit(&mut self) -> JournalResult<()> {
let original = self.capture.clone();
let first = self
.selected_ranges
.first()
.ok_or_else(|| invalid("empty recording selection"))?;
let start = session_store::capture::RecordingPosition {
canonical_order: first.source.order,
byte_offset: first.bytes.start,
};
let mut budget = serde_json::to_vec(&original)?.len();
loop {
let bytes = serde_json::to_vec(self)?.len();
if bytes <= job::MAX_JOB_INPUT_BYTES {
return Ok(());
}
budget = budget
.checked_sub(bytes - job::MAX_JOB_INPUT_BYTES)
.ok_or_else(|| invalid("Journal envelope cannot fit source bytes"))?;
self.capture = original
.recording_slice(start, MAX_JOURNAL_RANGES, budget)
.map_err(|e| invalid(&e.to_string()))?
.capture;
self.selected_ranges = capture_ranges(&self.capture)?;
if self.selected_ranges.is_empty() {
return Err(invalid("Journal envelope cannot fit one source scalar"));
}
}
}
pub fn decode(value: &serde_json::Value) -> JournalResult<Self> {
if serde_json::to_vec(value)?.len() > job::MAX_JOB_INPUT_BYTES {
return Err(invalid(
"Journal input exceeds whole serialized Job input limit",
));
}
let input: Self = serde_json::from_value(value.clone())?;
input
.capture
.validate(CaptureLimits {
max_bytes: job::MAX_JOB_INPUT_BYTES,
max_entries: MAX_JOURNAL_RANGES,
max_text_bytes: job::MAX_JOB_INPUT_BYTES,
..CaptureLimits::default()
})
.map_err(|e| invalid(&e.to_string()))?;
if input.version != 1
|| input.subject_id.is_empty()
|| input.journal_id.is_empty()
|| input.initial_state_token.is_empty()
|| input.selected_ranges.is_empty()
|| input.selected_ranges.len() > MAX_JOURNAL_RANGES
|| input
.selected_ranges
.iter()
.any(|r| r.source.session_id != input.capture.session_id)
{
return Err(invalid("invalid Journal input identity or selection"));
}
if capture_ranges(&input.capture)? != input.selected_ranges {
return Err(conflict(
"Journal body snapshot does not match selected canonical ranges",
));
}
Ok(input)
}
pub fn validate_generation(&self, generation: &JournalGeneration) -> JournalResult<()> {
if generation.journal_id != self.journal_id
|| generation.initial_journal.subject_id != self.subject_id
|| generation.initial_journal.source_session_id != self.capture.session_id
|| generation.initial_journal.state_token != self.initial_state_token
|| generation.initial_open.as_ref().map(|e| &e.id) != self.initial_open_id.as_ref()
|| generation.previous_fixed_end != self.previous_fixed_end
|| generation.selected_ranges != self.selected_ranges
{
return Err(conflict(
"saved Journal input differs from its initial generation snapshot",
));
}
Ok(())
}
}
/// Converts the shared recording projection into Journal's coverage language without
/// interpreting model text or constructing a second capture representation.
pub fn capture_ranges(capture: &CaptureDto) -> JournalResult<Vec<JournalSourceRange>> {
use session_store::capture::CaptureBody;
let mut ranges = Vec::new();
for entry in &capture.entries {
let CaptureBody::CanonicalPublic {
descriptor,
byte_offset,
text,
..
} = &entry.body
else {
return Err(invalid(
"Journal requires canonical original recording capture",
));
};
if !entry.policy.recording_material
|| text.is_empty()
|| descriptor.identity.segment_id.to_string() != entry.source.segment_id
|| descriptor.identity.entry_ref != format!("E{}", entry.entry_id)
|| descriptor.canonical_order != entry.order
|| byte_offset
.checked_add(text.len() as u64)
.is_none_or(|end| end > descriptor.total_bytes)
{
return Err(conflict(
"recording body identity, policy or range mismatch",
));
}
let source = JournalSource {
session_id: capture.session_id.clone(),
pointer: JournalSourcePointer {
segment_id: descriptor.identity.segment_id,
entry_ref: descriptor.identity.entry_ref.clone(),
},
order: descriptor.canonical_order,
content_sha256: descriptor.sha256.clone(),
total_bytes: descriptor.total_bytes,
};
if *byte_offset == 0 && text.len() as u64 == descriptor.total_bytes {
verify_canonical_text(&source, text)?;
}
ranges.push(JournalSourceRange {
source,
bytes: JournalByteRange {
start: *byte_offset,
end: *byte_offset + text.len() as u64,
},
});
}
for pair in ranges.windows(2) {
if pair[0].source.order.checked_add(1) != Some(pair[1].source.order)
|| pair[0].bytes.end != pair[0].source.total_bytes
|| pair[1].bytes.start != 0
{
return Err(conflict(
"recording capture is not an oldest contiguous prefix",
));
}
}
Ok(ranges)
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalMaterial {
+141 -10
View File
@@ -66,8 +66,8 @@ fn text_page(
})
}
/// Adapter for Host captures bounded by their storage contract. Production source reads should
/// use [`execute_bounded`] to avoid allocating an entire source for one small read.
/// Legacy adapter for Host captures bounded by their storage contract.
/// Production Hosts use [`attest_input_delivery`] and [`execute_snapshot`] instead.
pub fn execute(
store: &SubjektivStore,
context: &JournalJobContext,
@@ -101,23 +101,149 @@ pub fn execute(
})
}
/// Reads are delivered through this service before mutation authority is enabled. The durable
/// delivery ledger stores byte ranges only: Jobs may retain immutable descriptors without
/// accumulating a permanent second copy of the original Session in their request ledgers.
/// The Host verifies original provenance/digest/length and returns a bounded UTF-8 fragment.
pub(crate) fn migrate_input_delivery(
tx: &rusqlite::Transaction<'_>,
) -> feature_storage::Result<()> {
tx.execute_batch(
"CREATE TABLE journal_generation_input_deliveries (
generation_id TEXT PRIMARY KEY NOT NULL REFERENCES journal_generations(id) ON DELETE CASCADE,
input_digest TEXT NOT NULL CHECK(length(input_digest)=71 AND substr(input_digest,1,7)='sha256:' AND substr(input_digest,8) NOT GLOB '*[^0-9a-f]*')
);
CREATE TRIGGER journal_generation_input_delivery_no_update
BEFORE UPDATE ON journal_generation_input_deliveries
BEGIN SELECT RAISE(ABORT, 'Journal input delivery bindings are immutable'); END;",
)?;
Ok(())
}
fn validate_input_digest(input_digest: &str) -> JournalResult<()> {
// Preserve JobRequest::input_digest() verbatim, including its algorithm prefix.
if input_digest.strip_prefix("sha256:").is_none_or(|hex| {
hex.len() != 64
|| !hex
.bytes()
.all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b))
}) {
return Err(JournalError::InvalidInput(
"immutable Job input digest must be sha256: followed by 64 lowercase hex digits".into(),
));
}
Ok(())
}
/// Trusted Host-only delivery evidence, never a model tool or a claim of understanding.
/// Before calling, the Host must validate the saved, immutable body-bearing input against the
/// bound Job's input digest, its Session/provenance/ranges, and the actual dispatch plan that
/// supplies that exact input. The service cannot verify bytes it does not own. This transaction
/// stores only the digest bound to the generation's immutable selection; it neither copies the
/// body nor changes records, position, reflection or checkpoint. Exact replay is allowed only
/// while the same generation and writer binding remain active; changed digests/ranges conflict.
pub fn attest_input_delivery(
store: &SubjektivStore,
context: &JournalJobContext,
input_digest: &str,
selected_ranges: &[JournalSourceRange],
) -> JournalResult<()> {
validate_input_digest(input_digest)?;
store.database.try_transaction(|tx| {
super::lifecycle::reject_erased(tx, &context.host)?;
let generation = super::store::job(store, tx, context, true)?;
if generation.selected_ranges != selected_ranges {
return Err(JournalError::Conflict(
"immutable Job input selection differs from generation".into(),
));
}
let saved: Option<String> = tx.query_row(
"SELECT input_digest FROM journal_generation_input_deliveries WHERE generation_id=?1",
[&generation.id], |row| row.get(0),
).optional()?;
if let Some(saved) = saved {
if saved != input_digest {
return Err(JournalError::Conflict(
"generation is bound to a different immutable Job input digest".into(),
));
}
} else {
tx.execute(
"INSERT INTO journal_generation_input_deliveries(generation_id,input_digest) VALUES(?1,?2)",
params![generation.id, input_digest],
)?;
}
Ok(())
})
}
fn require_input_delivery(
tx: &rusqlite::Connection,
context: &JournalJobContext,
input_digest: &str,
) -> JournalResult<()> {
validate_input_digest(input_digest)?;
let saved: Option<String> = tx
.query_row(
"SELECT input_digest FROM journal_generation_input_deliveries WHERE generation_id=?1",
[&context.generation_id],
|row| row.get(0),
)
.optional()?;
if saved.as_deref() != Some(input_digest) {
return Err(JournalError::Conflict(
"trusted Host delivery of this immutable Job input is required".into(),
));
}
Ok(())
}
/// Production sequential execution over Journal state and the Host-attested immutable input.
/// The digest is taken from the bound saved Job by the Host, not from model arguments. Every
/// request requires the matching delivery binding and an active generation/writer. Source text
/// is already in immutable input: Source targets fail explicitly, with no Session reader or
/// fallback. Receipt-backed record, position and checkpoint constraints remain unchanged.
pub fn execute_snapshot(
store: &SubjektivStore,
context: &JournalJobContext,
input_digest: &str,
request: JournalJobRequest,
) -> JournalResult<JournalJobResponse> {
execute_inner(store, context, request, Some(input_digest), |_, _, _| {
Err(JournalError::PermissionDenied(
"Source reads are unavailable for immutable Journal input".into(),
))
})
}
/// Legacy/test source-read adapter. Production Hosts must use [`execute_snapshot`].
/// This ledger attests bounded Source responses only and cannot satisfy the input-digest gate.
pub fn execute_bounded(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
read_source: impl FnMut(&JournalSource, u64, usize) -> JournalResult<recording::JournalMaterial>,
) -> JournalResult<JournalJobResponse> {
execute_inner(store, context, request, None, read_source)
}
fn execute_inner(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
input_digest: Option<&str>,
mut read_source: impl FnMut(&JournalSource, u64, usize) -> JournalResult<recording::JournalMaterial>,
) -> JournalResult<JournalJobResponse> {
let generation = store
.database
.try_with_connection(|tx| super::store::job(store, tx, context, true))?;
let generation = store.database.try_with_connection(|tx| {
super::lifecycle::reject_erased(tx, &context.host)?;
let generation = super::store::job(store, tx, context, true)?;
if let Some(input_digest) = input_digest {
require_input_delivery(tx, context, input_digest)?;
}
Ok::<_, JournalError>(generation)
})?;
let mut delivered = None;
let response = match request {
JournalJobRequest::Execute { operation } => {
require_delivery(store, context, &generation.selected_ranges)?;
if input_digest.is_none() {
require_delivery(store, context, &generation.selected_ranges)?;
}
let receipt = if matches!(operation.action, JournalAction::Checkpoint { .. }) {
crate::api::checkpoint_journal(store, context, operation)?
} else {
@@ -197,6 +323,11 @@ pub fn execute_bounded(
JournalJobRequest::ReadText { read } => {
let text = match &read.target {
JournalTextTarget::Source { pointer } => {
if input_digest.is_some() {
return Err(JournalError::PermissionDenied(
"Source target is unavailable: raw source bodies are in immutable Job input".into(),
));
}
let range = generation
.selected_ranges
.iter()
+29 -4
View File
@@ -1,5 +1,9 @@
//! Read-only, bounded Journal context for a new body Session. Selection and text projection
//! share one storage transaction; no source history, Job, admission or checkpoint is consulted.
//! Read-only, bounded Journal context at body startup and compaction boundaries.
//! Selection, text projection, references and processing checkpoints share one storage
//! transaction. Reads neither consult source history/Job liveness nor mutate recording.
//! Startup prefers the Host-retained predecessor; compaction requires a live body writer
//! and prefers its current Session, without waiting for Idle or recording completion.
//! Both supplement only from the same Subject under the same independent budgets.
//! This is derived reference material, never a new observation or an execution capability.
use super::*;
@@ -71,11 +75,32 @@ impl SubjektivStore {
&self,
host: &JournalHostContext,
previous_session: Option<&str>,
) -> JournalResult<JournalStartupSnapshot> {
self.context_snapshot(host, previous_session, false)
}
/// Read the saved current-Session Journal without reconciling or advancing it.
/// The active writer check fences released body lifetimes, not non-Idle execution.
pub fn journal_compaction_snapshot(
&self,
host: &JournalHostContext,
) -> JournalResult<JournalStartupSnapshot> {
let session = host.attribution.as_ref().ok_or_else(|| {
JournalError::PermissionDenied("compaction requires a Host-bound body Session".into())
})?;
self.context_snapshot(host, Some(&session.session_id), true)
}
fn context_snapshot(
&self,
host: &JournalHostContext,
preferred_session: Option<&str>,
require_current_writer: bool,
) -> JournalResult<JournalStartupSnapshot> {
self.database.try_transaction(|tx| {
super::store::check_host(self, tx, host, false)?;
super::store::check_host(self, tx, host, require_current_writer)?;
let mut ids = Vec::new();
if let Some(session) = previous_session {
if let Some(session) = preferred_session {
let previous: Option<(String, String)> = tx.query_row(
"SELECT id,subject_id FROM journals WHERE source_session_id=?1",
[session], |r| Ok((r.get(0)?, r.get(1)?)),
+60 -3
View File
@@ -70,7 +70,10 @@ fn page_limit(limit: usize) -> JournalResult<usize> {
}
Ok(limit)
}
fn fit_page<T: Serialize>(rows: Vec<(String, T)>, mut more: bool) -> JournalResult<JournalPage<T>> {
pub(super) fn fit_page<T: Serialize>(
rows: Vec<(String, T)>,
mut more: bool,
) -> JournalResult<JournalPage<T>> {
let mut page = JournalPage {
items: Vec::new(),
next: None,
@@ -162,7 +165,7 @@ pub(super) fn journal(
}
Ok(j)
}
fn entry(tx: &Connection, journal_id: &str, id: &str) -> JournalResult<JournalEntry> {
pub(super) fn entry(tx: &Connection, journal_id: &str, id: &str) -> JournalResult<JournalEntry> {
let row = tx.query_row("SELECT id, journal_id, position, state, title, body, ongoing, source_end_json, created_at, updated_at, fixed_at FROM journal_entries WHERE journal_id=?1 AND id=?2", params![journal_id, id], |r| Ok((r.get::<_,String>(0)?,r.get::<_,String>(1)?,r.get::<_,u64>(2)?,r.get::<_,String>(3)?,r.get::<_,String>(4)?,r.get::<_,String>(5)?,r.get::<_,String>(6)?,r.get::<_,Option<String>>(7)?,r.get::<_,String>(8)?,r.get::<_,String>(9)?,r.get::<_,Option<String>>(10)?))).optional()?.ok_or_else(|| JournalError::NotFound(id.into()))?;
let mut stmt = tx.prepare(
"SELECT target_id FROM journal_corrections WHERE entry_id=?1 ORDER BY target_id",
@@ -199,7 +202,7 @@ pub(super) fn open(tx: &Connection, journal_id: &str) -> JournalResult<Option<Jo
.optional()?;
id.map(|id| entry(tx, journal_id, &id)).transpose()
}
fn previous_fixed(
pub(super) fn previous_fixed(
tx: &Connection,
journal_id: &str,
) -> JournalResult<Option<(JournalSourcePointer, u64)>> {
@@ -566,6 +569,60 @@ impl SubjektivStore {
Ok(snapshot)
})
}
/// Rebind a saved body input to a new attempt without resampling its initial state.
/// The first binding freezes the full initial Journal snapshot; retries reuse it and
/// retain operation receipts. Different Jobs cannot borrow that snapshot as authority.
pub fn prepare_journal_input(
&self,
host: &JournalHostContext,
input: &super::recording::JournalInput,
binding: JobAttemptBinding,
) -> JournalResult<JournalGeneration> {
label(&binding.job_id)?;
label(&binding.attempt_id)?;
self.database.try_transaction(|tx| {
check_host(self, tx, host, false)?;
check_session_host(self, tx, host)?;
let current = journal(tx, host, &input.journal_id, true)?;
let existing: Option<String> = tx.query_row(
"SELECT snapshot_json FROM journal_generations WHERE journal_id=?1 AND job_id=?2 AND attempt_id=?3",
params![input.journal_id, binding.job_id, binding.attempt_id], |r| r.get(0)).optional()?;
if let Some(raw) = existing {
let g = serde_json::from_str(&raw)?;
input.validate_generation(&g)?;
return Ok(g);
}
// Retained receipt lookup does not reacquire writer authority. Only a new
// generation requires the current active Subject/Session writer lifetime.
check_host(self, tx, host, true)?;
if tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_generations WHERE journal_id=?1 AND active=1)", [&input.journal_id], |r| r.get::<_, bool>(0))? {
return Err(conflict("another active generation owns this Journal"));
}
let prior: Option<String> = tx.query_row(
"SELECT snapshot_json FROM journal_generations WHERE journal_id=?1 AND job_id=?2 ORDER BY rowid LIMIT 1",
params![input.journal_id, binding.job_id], |r| r.get(0)).optional()?;
let mut g = if let Some(raw) = prior {
serde_json::from_str::<JournalGeneration>(&raw)?
} else {
JournalGeneration { id: String::new(), journal_id: input.journal_id.clone(), job: binding.clone(),
selected_ranges: input.selected_ranges.clone(), initial_journal: current.clone(),
initial_open: open(tx, &input.journal_id)?, previous_fixed_end: previous_fixed(tx, &input.journal_id)?.map(|p| p.0) }
};
input.validate_generation(&g)?;
check_ranges(tx, &current, &input.selected_ranges, true)?;
let end = prefix(&g.initial_journal, &input.selected_ranges)?;
let cursor = (current.process_order, current.process_byte);
if cursor < (g.initial_journal.process_order, g.initial_journal.process_byte) || cursor > end {
return Err(conflict("saved input no longer owns the current processing cursor"));
}
g.id = issued_id("journal-generation");
g.job = binding;
tx.execute("INSERT INTO journal_generations(id,journal_id,job_id,attempt_id,request_json,snapshot_json,active) VALUES(?1,?2,?3,?4,?5,?6,1)",
params![g.id,g.journal_id,g.job.job_id,g.job.attempt_id,serde_json::to_string(&g.selected_ranges)?,serde_json::to_string(&g)?])?;
Ok(g)
})
}
pub(super) fn validate_journal_job(&self, ctx: &JournalJobContext) -> JournalResult<()> {
self.database.try_with_connection(|tx| {
job(self, tx, ctx, true)?;
File diff suppressed because it is too large Load Diff
+14 -5
View File
@@ -1,8 +1,10 @@
//! Bounded, Host-bound Journal recording and sequential Job transport.
//!
//! Subject, Journal and generation authority never come from these model inputs.
//! Hosts validate live execution and public source bytes before selection; reading a
//! preview must not select the enclosing entry. Operation retries retain the entire
//! Hosts validate live execution and public source bytes before selection, save body-bearing
//! immutable Job input, and attest its exact digest and dispatch plan through the service.
//! Model requests cannot attest input delivery; Source targets are rejected by snapshot execution.
//! Initial/current Journal reads remain separate from input delivery. Operation retries retain the entire
//! JournalOperation, including its original expected_state and covered_ranges.
use serde::{Deserialize, Serialize};
@@ -21,6 +23,12 @@ pub enum JournalRecordingReason {
WorkerRestored,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalCompactionRequest {
pub session_id: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalRecordingRequest {
@@ -44,8 +52,8 @@ pub struct JournalJobState {
pub current_journal: JournalMetadata,
pub current_open: Option<JournalEntrySummary>,
pub previous_fixed_end: Option<JournalSourcePointer>,
/// Frozen permission to fetch this prefix, not evidence of delivery or reflection.
/// The sequential service gates mutations on its durable Source-read delivery ledger.
/// Frozen source selection delivered in immutable Job input, not evidence of reflection.
/// Production execution requires the trusted Host's matching input-digest attestation.
pub selected_ranges: Vec<JournalSourceRange>,
}
@@ -77,7 +85,8 @@ pub enum JournalTextTarget {
OpenOngoing {
snapshot: JournalSnapshot,
},
/// The Host restricts this pointer to this Job's frozen public source input.
/// Legacy source-read adapter only. Production snapshot execution rejects this target;
/// raw source bodies must already be present in the immutable Job input.
Source {
pointer: JournalSourcePointer,
},
+1 -1
View File
@@ -929,7 +929,7 @@ mod tests {
let (_manager, _workspace, store) = open_store(&root, "workspace-a");
let migrated_subject = store.subject(&subject_id).unwrap().unwrap();
assert!(migrated_subject.behavior_md.is_empty());
assert!(migrated_subject.behavior_md.is_empty());
assert!(store.subject_permissions(&subject_id).unwrap().is_none());
let lifecycle_attribution = attribution(
&subject_id,
"runtime-1",
+21
View File
@@ -4,6 +4,7 @@ pub mod api;
pub mod job;
pub mod journal;
mod legacy_migrations;
pub mod permissions;
mod schema;
use legacy_migrations::{
add_candidate_decision_receipts, add_subject_behavior, add_subject_session_attribution,
@@ -82,6 +83,16 @@ pub enum SubjektivError {
},
#[error("subject `{subject_id}` behavior content changed")]
SubjectBehaviorConflict { subject_id: String },
#[error("subject `{0}` permission source changed")]
SubjectPermissionsConflict(String),
#[error("permission operation `{0}` was reused with different input")]
PermissionOperationConflict(String),
#[error("authorization denied: {0}")]
AuthorizationDenied(String),
#[error("Worker execution grant binding conflicts for `{0}`")]
WorkerExecutionGrantConflict(String),
#[error("Subject `{subject_id}` has more than {limit} retained Worker execution grants")]
WorkerExecutionGrantListOverflow { subject_id: String, limit: usize },
#[error("surface generation conflict: {0}")]
SurfaceGenerationConflict(String),
#[error("memory `{memory_id}` cannot transition from {from:?} to {to:?}")]
@@ -623,6 +634,16 @@ static MIGRATIONS: &[FeatureMigration] = &[
"add immutable candidate Journal context links",
journal::memory_links::migrate,
),
FeatureMigration::new(
12,
"add explicit Subject permission sources and immutable child execution grants",
permissions::migrate,
),
FeatureMigration::new(
13,
"bind Journal generation delivery to immutable Job input digest",
journal::service::migrate_input_delivery,
),
];
pub const REGISTRATION: FeatureRegistration =
+288
View File
@@ -0,0 +1,288 @@
//! Host-administered permission sources and immutable child grants, separate from behavior.
//!
//! The Host authenticates administrators before source writes and binds verified Worker/Profile
//! identities before grant writes. These records are not a Worker registry. The Host alone
//! authenticates root execution ownership against its canonical Backend singleton and checks
//! every live lineage edge; Journal bindings and retained grants cannot establish ownership.
//! Replays return historical evidence without restoring a source or granting current authority.
use super::*;
use server_api::authorization::{
PermissionResource, ScopedPermission, SubjectPermissions, SubjectPermissionsUpdate,
WorkerExecutionGrant, validate_authorization_identifier, validate_scoped_permissions,
};
use sha2::{Digest, Sha256};
pub const MAX_SUBJECT_WORKER_EXECUTION_GRANTS: usize = 1_000;
pub(super) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("permissions.sql"))?;
Ok(())
}
impl SubjektivStore {
/// Unset sources are `None` (deny), including migrated Subjects. This is an admin read,
/// not an effective-authorization decision; retired Subjects retain their source evidence.
pub fn subject_permissions(&self, subject_id: &str) -> Result<Option<SubjectPermissions>> {
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
source_in_connection(connection, subject_id)
})
}
/// Host-admin-only replacement. Every replacement issues a fresh change ID, including
/// identical contents. Revocation is an explicit empty replacement. A scope-wide operation
/// ID binds Subject, CAS input, and both permission vectors; result and receipt commit together.
pub fn set_subject_permissions(
&self,
subject_id: &str,
update: &SubjectPermissionsUpdate,
) -> Result<SubjectPermissions> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(&update.operation_id)
.map_err(SubjektivError::InvalidRecord)?;
if let Some(change) = &update.expected_change_id {
validate_authorization_identifier(change).map_err(SubjektivError::InvalidRecord)?;
}
validate_scope(self.scope_id(), &update.execution)?;
validate_scope(self.scope_id(), &update.delegation)?;
let fingerprint: String = Sha256::digest(serde_json::to_vec(&(subject_id, update))?)
.iter()
.map(|byte| format!("{byte:02x}"))
.collect();
self.database.try_transaction(|tx| {
require_subject(tx, subject_id)?;
let receipt: Option<(String, String)> = tx.query_row(
"SELECT request_fingerprint, change_id FROM subject_permission_receipts WHERE operation_id=?1",
[&update.operation_id], |row| Ok((row.get(0)?, row.get(1)?)),
).optional()?;
if let Some((saved, change_id)) = receipt {
if saved != fingerprint {
return Err(SubjektivError::PermissionOperationConflict(update.operation_id.clone()));
}
let raw: String = tx.query_row(
"SELECT record_json FROM subject_permission_changes WHERE subject_id=?1 AND change_id=?2",
params![subject_id, change_id], |row| row.get(0),
)?;
return Ok(serde_json::from_str(&raw)?);
}
require_active_subject(tx, subject_id)?;
let current = source_in_connection(tx, subject_id)?;
if current.as_ref().map(|source| &source.change_id) != update.expected_change_id.as_ref() {
return Err(SubjektivError::SubjectPermissionsConflict(subject_id.into()));
}
let source = SubjectPermissions {
change_id: issued_id("permissions"),
execution: update.execution.clone(),
delegation: update.delegation.clone(),
};
tx.execute("INSERT INTO subject_permission_changes(subject_id,change_id,record_json) VALUES(?1,?2,?3)",
params![subject_id, source.change_id, serde_json::to_string(&source)?])?;
tx.execute("INSERT INTO subject_permissions(subject_id,change_id) VALUES(?1,?2) ON CONFLICT(subject_id) DO UPDATE SET change_id=excluded.change_id",
params![subject_id, source.change_id])?;
tx.execute("INSERT INTO subject_permission_receipts(operation_id,subject_id,request_fingerprint,change_id) VALUES(?1,?2,?3,?4)",
params![update.operation_id, subject_id, fingerprint, source.change_id])?;
Ok(source)
})
}
/// Binds child grants only. Exact retries recover an immutable binding even after revocation;
/// that success is not authorization to execute or redelegate. New bindings require the
/// current source, an active Subject and explicit delegation containment. The Host must
/// authenticate root singleton ownership and every live lineage edge before calling;
/// a missing retained parent is not ownership evidence. Profile resolution, provider scope,
/// Worker creation and caller authentication also remain Host duties.
pub fn bind_worker_execution_grant(&self, grant: &WorkerExecutionGrant) -> Result<()> {
validate_grant(self.scope_id(), grant)?;
self.database.try_transaction(|tx| {
if let Some(existing) = grant_in_connection(tx, &grant.runtime_id, &grant.worker_id)? {
return if existing == *grant { Ok(()) } else {
Err(SubjektivError::WorkerExecutionGrantConflict(format!("{}/{}", grant.runtime_id, grant.worker_id)))
};
}
require_active_subject(tx, &grant.subject_id)?;
let source = source_in_connection(tx, &grant.subject_id)?
.ok_or_else(|| denied("Subject permissions are unset"))?;
if source.change_id != grant.source_change_id {
return Err(denied("Subject permission source is no longer current"));
}
require_covered(&source.delegation, &grant.permissions)?;
require_covered(&source.delegation, &grant.redelegation)?;
// A child grant cannot turn a body (including a released historical body) into a
// delegated child or attach a Worker historically attributed to a different Subject.
let invalid_target: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![grant.runtime_id, grant.worker_id, grant.subject_id], |row| row.get(0),
)?;
if invalid_target { return Err(denied("target is a body or belongs to another Subject")); }
let origin = require_retained_origin(tx, grant)?;
if let Some(origin) = origin {
require_covered(&origin.redelegation, &grant.permissions)?;
require_covered(&origin.redelegation, &grant.redelegation)?;
}
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![grant.runtime_id, grant.worker_id, grant.subject_id, grant.source_change_id, serde_json::to_string(grant)?])?;
Ok(())
})
}
/// Returns this Subject's retained grants, including stale grants needed for cancellation
/// and reporting. Never an ownership list or effective-authorization decision. Overflow is
/// explicit rather than silently omitting Workers from a cancellation/reporting pass.
pub fn list_subject_worker_execution_grants(
&self,
subject_id: &str,
) -> Result<Vec<WorkerExecutionGrant>> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
let mut statement = connection.prepare(
"SELECT record_json FROM worker_execution_grants WHERE subject_id=?1 ORDER BY runtime_id,worker_id LIMIT ?2",
)?;
let mut rows = statement.query(params![subject_id, MAX_SUBJECT_WORKER_EXECUTION_GRANTS + 1])?;
let mut grants = Vec::new();
while let Some(row) = rows.next()? {
if grants.len() == MAX_SUBJECT_WORKER_EXECUTION_GRANTS {
return Err(SubjektivError::WorkerExecutionGrantListOverflow {
subject_id: subject_id.into(),
limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS,
});
}
grants.push(serde_json::from_str(&row.get::<_, String>(0)?)?);
}
Ok(grants)
})
}
/// Returns retained binding evidence, not current authorization. The Host must compare the
/// active Subject, source change, current origin ownership, Profile and provider constraints.
pub fn worker_execution_grant(
&self,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
validate_authorization_identifier(runtime).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(worker).map_err(SubjektivError::InvalidRecord)?;
self.database
.try_with_connection(|connection| grant_in_connection(connection, runtime, worker))
}
}
fn source_in_connection(
connection: &Connection,
subject: &str,
) -> Result<Option<SubjectPermissions>> {
let raw: Option<String> = connection.query_row(
"SELECT c.record_json FROM subject_permissions p JOIN subject_permission_changes c ON c.subject_id=p.subject_id AND c.change_id=p.change_id WHERE p.subject_id=?1",
[subject], |row| row.get(0),
).optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn grant_in_connection(
connection: &Connection,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
let raw: Option<String> = connection
.query_row(
"SELECT record_json FROM worker_execution_grants WHERE runtime_id=?1 AND worker_id=?2",
params![runtime, worker],
|row| row.get(0),
)
.optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn validate_scope(scope: &str, permissions: &[ScopedPermission]) -> Result<()> {
validate_scoped_permissions(permissions).map_err(SubjektivError::InvalidRecord)?;
for permission in permissions {
if let PermissionResource::Workspace { workspace_id } = &permission.resource
&& workspace_id != scope
{
return Err(denied("Workspace permission is outside this store scope"));
}
}
Ok(())
}
fn validate_grant(scope: &str, grant: &WorkerExecutionGrant) -> Result<()> {
for value in [
&grant.subject_id,
&grant.origin_runtime_id,
&grant.origin_worker_id,
&grant.source_change_id,
&grant.runtime_id,
&grant.worker_id,
&grant.profile,
&grant.profile_digest,
] {
validate_authorization_identifier(value).map_err(SubjektivError::InvalidRecord)?;
}
if grant.origin_runtime_id == grant.runtime_id && grant.origin_worker_id == grant.worker_id {
return Err(denied("a child grant cannot originate from itself"));
}
validate_scope(scope, &grant.permissions)?;
validate_scope(scope, &grant.redelegation)
}
fn require_covered(ceiling: &[ScopedPermission], requested: &[ScopedPermission]) -> Result<()> {
if requested
.iter()
.all(|permission| ceiling.iter().any(|allowed| allowed.covers(permission)))
{
Ok(())
} else {
Err(denied(
"requested permission or redelegation exceeds its explicit ceiling",
))
}
}
fn require_retained_origin(
tx: &Transaction<'_>,
grant: &WorkerExecutionGrant,
) -> Result<Option<WorkerExecutionGrant>> {
let mut runtime = grant.origin_runtime_id.clone();
let mut worker = grant.origin_worker_id.clone();
let mut direct_origin = None;
let mut visited = HashSet::from([(grant.runtime_id.clone(), grant.worker_id.clone())]);
// Validate retained lineage only. Absence terminates at a Host-authenticated root;
// neither Journal active state nor absence of a grant proves current singleton ownership.
for _ in 0..64 {
if !visited.insert((runtime.clone(), worker.clone())) {
return Err(denied("cyclic grant origin"));
}
let foreign_attribution: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![runtime, worker, grant.subject_id], |row| row.get(0),
)?;
if foreign_attribution {
return Err(denied("grant origin is attributed to another Subject"));
}
let Some(origin) = grant_in_connection(tx, &runtime, &worker)? else {
return Ok(direct_origin);
};
if origin.subject_id != grant.subject_id
|| origin.source_change_id != grant.source_change_id
{
return Err(denied(
"grant origin belongs to another Subject or permission source",
));
}
if direct_origin.is_none() {
direct_origin = Some(origin.clone());
}
runtime = origin.origin_runtime_id;
worker = origin.origin_worker_id;
}
Err(denied("grant origin exceeds maximum delegation depth"))
}
fn denied(reason: &str) -> SubjektivError {
SubjektivError::AuthorizationDenied(reason.into())
}
#[cfg(test)]
mod tests;
+45
View File
@@ -0,0 +1,45 @@
CREATE TABLE subject_permission_changes (
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL CHECK (length(change_id) > 0),
record_json TEXT NOT NULL,
PRIMARY KEY (subject_id, change_id)
);
CREATE TABLE subject_permissions (
subject_id TEXT PRIMARY KEY REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE subject_permission_receipts (
operation_id TEXT PRIMARY KEY CHECK (length(operation_id) > 0),
subject_id TEXT NOT NULL,
request_fingerprint TEXT NOT NULL,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE worker_execution_grants (
runtime_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
source_change_id TEXT NOT NULL,
record_json TEXT NOT NULL,
PRIMARY KEY (runtime_id, worker_id),
FOREIGN KEY (subject_id, source_change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TRIGGER subject_permission_changes_no_update BEFORE UPDATE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are immutable');
END;
CREATE TRIGGER subject_permission_changes_no_delete BEFORE DELETE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are retained');
END;
CREATE TRIGGER subject_permission_receipts_no_update BEFORE UPDATE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are immutable');
END;
CREATE TRIGGER subject_permission_receipts_no_delete BEFORE DELETE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are retained');
END;
CREATE TRIGGER worker_execution_grants_no_update BEFORE UPDATE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are immutable');
END;
CREATE TRIGGER worker_execution_grants_no_delete BEFORE DELETE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are retained');
END;
+656
View File
@@ -0,0 +1,656 @@
//! Real SQLite component tests through store APIs; Host authentication, canonical singleton
//! ownership and live lineage checks are outside this suite. Root origins are Host-attested inputs;
//! Journal fixtures exercise defensive attribution only, never execution authority.
use super::*;
use crate::tests::{open_store, role};
use feature_storage::FeatureStorage;
use std::path::Path;
fn permission(operation: &str, path: &str, recursive: bool) -> ScopedPermission {
ScopedPermission {
operation: operation.into(),
resource: PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
},
}
}
fn update(
operation: &str,
expected: Option<&str>,
delegation: Vec<ScopedPermission>,
) -> SubjectPermissionsUpdate {
SubjectPermissionsUpdate {
operation_id: operation.into(),
expected_change_id: expected.map(Into::into),
execution: Vec::new(),
delegation,
}
}
fn grant(subject: &str, source: &str, worker: &str) -> WorkerExecutionGrant {
WorkerExecutionGrant {
subject_id: subject.into(),
origin_runtime_id: "runtime".into(),
origin_worker_id: "body".into(),
source_change_id: source.into(),
runtime_id: "runtime".into(),
worker_id: worker.into(),
profile: "builtin:coder".into(),
profile_digest: "resolved-profile-sha256".into(),
permissions: vec![permission("file.read", "src", false)],
redelegation: Vec::new(),
}
}
fn schema(store: &SubjektivStore) -> Vec<(String, String)> {
store.database.try_with_connection::<_, SubjektivError>(|connection| {
let mut statement = connection.prepare("SELECT name,sql FROM sqlite_master WHERE name NOT LIKE 'sqlite_%' AND sql IS NOT NULL ORDER BY name")?;
Ok(statement.query_map([], |row| Ok((row.get(0)?, row.get(1)?)))?.collect::<std::result::Result<_, _>>()?)
}).unwrap()
}
fn open_version(
root: &Path,
version: usize,
) -> (FeatureStorage, ScopedFeatureStorage, SubjektivStore) {
let manager = FeatureStorage::new(root);
let scope = manager.scope("workspace-a").unwrap();
let registration = scope
.register(FeatureRegistration::new(
SUBJEKTIV_FEATURE_ID,
&MIGRATIONS[..version],
))
.unwrap();
let store = SubjektivStore::open(&scope, &registration).unwrap();
(manager, scope, store)
}
#[test]
fn replacements_are_cas_fenced_replayable_and_never_restore_old_permissions() {
let temp = tempfile::tempdir().unwrap();
let subject;
let first;
let first_update = update("allow", None, vec![permission("file.read", ".", true)]);
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
subject = store.create_subject(role()).unwrap().id;
assert!(store.subject_permissions(&subject).unwrap().is_none());
first = store
.set_subject_permissions(&subject, &first_update)
.unwrap();
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert!(matches!(
store.set_subject_permissions(&subject, &update("stale", None, Vec::new())),
Err(SubjektivError::SubjectPermissionsConflict(_))
));
assert!(matches!(
store.set_subject_permissions(&subject, &update("allow", None, Vec::new())),
Err(SubjektivError::PermissionOperationConflict(_))
));
let other = store.create_subject(role()).unwrap();
assert!(matches!(
store.set_subject_permissions(&other.id, &first_update),
Err(SubjektivError::PermissionOperationConflict(_))
));
let same = store
.set_subject_permissions(
&subject,
&update("same", Some(&first.change_id), first.delegation.clone()),
)
.unwrap();
assert_ne!(same.change_id, first.change_id);
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&same.change_id), Vec::new()),
)
.unwrap();
assert!(revoked.delegation.is_empty());
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(revoked));
}
let (_reopened_manager, _reopened_scope, reopened) = open_store(temp.path(), "workspace-a");
assert!(
reopened
.subject_permissions(&subject)
.unwrap()
.unwrap()
.delegation
.is_empty()
);
assert_eq!(
reopened
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
}
#[test]
fn failed_receipt_insert_rolls_back_source_history_and_projection() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let first = store
.set_subject_permissions(&subject, &update("first", None, Vec::new()))
.unwrap();
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_permission_receipt BEFORE INSERT ON subject_permission_receipts BEGIN SELECT RAISE(ABORT,'injected receipt failure'); END;")?;
Ok(())
}).unwrap();
let replacement = update(
"second",
Some(&first.change_id),
vec![permission("file.read", ".", true)],
);
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_err()
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(first));
store.database.try_transaction::<_, SubjektivError>(|tx| {
let counts: (u32, u32) = tx.query_row("SELECT (SELECT count(*) FROM subject_permission_changes),(SELECT count(*) FROM subject_permission_receipts)", [], |r| Ok((r.get(0)?, r.get(1)?)))?;
assert_eq!(counts, (1, 1));
tx.execute_batch("DROP TRIGGER fail_permission_receipt")?;
Ok(())
}).unwrap();
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_ok()
);
}
#[test]
fn scope_and_validation_rejections_do_not_issue_permission_sources() {
let temp = tempfile::tempdir().unwrap();
let (_a_manager, _a_scope, a) = open_store(&temp.path().join("a"), "workspace-a");
let (_b_manager, _b_scope, b) = open_store(&temp.path().join("b"), "workspace-b");
let subject = a.create_subject(role()).unwrap().id;
assert!(
b.set_subject_permissions(&subject, &update("cross-subject", None, Vec::new()))
.is_err()
);
let cross_workspace = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "workspace-b".into(),
},
};
assert!(
a.set_subject_permissions(
&subject,
&update("cross-workspace", None, vec![cross_workspace])
)
.is_err()
);
for (operation, path) in [
("file.*", "src"),
("file..read", "src"),
("file.read", "../src"),
("file.read", "src/../escape"),
] {
assert!(
a.set_subject_permissions(
&subject,
&update("invalid", None, vec![permission(operation, path, true)])
)
.is_err()
);
}
assert!(a.subject_permissions(&subject).unwrap().is_none());
}
#[test]
fn child_grants_are_immutable_bounded_and_do_not_create_subject_ownership() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", "src", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&child).unwrap();
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child.clone())
);
assert!(
store
.journal_body_is_unpublished(&subject, "runtime", "child")
.unwrap()
);
let mut changed = child.clone();
changed.profile_digest = "changed-profile".into();
assert!(matches!(
store.bind_worker_execution_grant(&changed),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
for (worker, requested) in [
("write", permission("file.write", "src", false)),
("escape", permission("file.read", ".", true)),
("prefix", permission("file.read", "src-other", false)),
] {
let mut candidate = grant(&subject, &source.change_id, worker);
candidate.permissions = vec![requested];
assert!(store.bind_worker_execution_grant(&candidate).is_err());
assert!(
store
.worker_execution_grant("runtime", worker)
.unwrap()
.is_none()
);
}
let mut self_origin = child.clone();
self_origin.worker_id = "body".into();
assert!(store.bind_worker_execution_grant(&self_origin).is_err());
let other = store.create_subject(role()).unwrap();
store
.claim_journal_writer(&other.id, "runtime", "other-body")
.unwrap();
let mut invalid = grant(&subject, &source.change_id, "other-body");
assert!(store.bind_worker_execution_grant(&invalid).is_err());
invalid.worker_id = "forged-child".into();
invalid.origin_worker_id = "other-body".into();
assert!(store.bind_worker_execution_grant(&invalid).is_err());
}
#[test]
fn redelegation_requires_current_source_and_explicit_retained_parent_ceiling() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = vec![permission("file.read", "src", true)];
store.bind_worker_execution_grant(&child).unwrap();
let mut descendant = grant(&subject, &source.change_id, "grandchild");
descendant.origin_worker_id = "child".into();
store.bind_worker_execution_grant(&descendant).unwrap();
let mut excessive = descendant.clone();
excessive.worker_id = "excessive".into();
excessive.redelegation = vec![permission("file.read", ".", true)];
assert!(store.bind_worker_execution_grant(&excessive).is_err());
let mut nondelegator = descendant.clone();
nondelegator.origin_worker_id = "grandchild".into();
nondelegator.worker_id = "great-grandchild".into();
assert!(store.bind_worker_execution_grant(&nondelegator).is_err());
let mut stale = descendant.clone();
stale.worker_id = "after-source-replacement".into();
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
assert_ne!(revoked.change_id, source.change_id);
stale.origin_worker_id = "new-body".into();
assert!(store.bind_worker_execution_grant(&stale).is_err());
// Exact retries/readback recover evidence but cannot revive the source or origin.
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
store.retire_subject(&subject).unwrap();
stale.source_change_id = revoked.change_id;
assert!(store.bind_worker_execution_grant(&stale).is_err());
}
#[test]
fn journal_state_neither_authenticates_roots_nor_bypasses_retained_child_ceilings() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
// The Host has authenticated root ownership outside this store. No Journal record is
// required; changing Journal writer state cannot supply or invalidate that authentication.
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "without-journal"))
.unwrap();
let journal = store
.claim_journal_writer(&subject, "runtime", "body")
.unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-active-journal"))
.unwrap();
store.release_journal_writer(&journal).unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-released-journal"))
.unwrap();
// An active Journal binding must never upgrade a child's empty redelegation ceiling.
store
.claim_journal_writer(&subject, "runtime", "without-journal")
.unwrap();
let mut descendant = grant(&subject, &source.change_id, "descendant");
descendant.origin_worker_id = "without-journal".into();
assert!(matches!(
store.bind_worker_execution_grant(&descendant),
Err(SubjektivError::AuthorizationDenied(_))
));
assert!(
store
.worker_execution_grant("runtime", "descendant")
.unwrap()
.is_none()
);
}
#[test]
fn retained_origins_reject_cycles_foreign_subjects_and_stale_parent_sources() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = source.delegation.clone();
store.bind_worker_execution_grant(&child).unwrap();
// Closing a retained chain back through its previously unbound root must not create a cycle.
let mut cycle = grant(&subject, &source.change_id, "body");
cycle.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&cycle),
Err(SubjektivError::AuthorizationDenied(_))
));
let other = store.create_subject(role()).unwrap().id;
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let mut foreign = grant(&other, &other_source.change_id, "foreign-child");
foreign.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&foreign),
Err(SubjektivError::AuthorizationDenied(_))
));
let current = store
.set_subject_permissions(
&subject,
&update("replace", Some(&source.change_id), source.delegation),
)
.unwrap();
let mut stale_parent = grant(&subject, &current.change_id, "stale-parent-child");
stale_parent.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&stale_parent),
Err(SubjektivError::AuthorizationDenied(_))
));
}
#[test]
fn failed_grant_insert_leaves_no_binding_and_can_retry_same_grant() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_grant BEFORE INSERT ON worker_execution_grants BEGIN SELECT RAISE(ABORT,'injected grant failure'); END;")?;
Ok(())
}).unwrap();
assert!(store.bind_worker_execution_grant(&child).is_err());
assert!(
store
.worker_execution_grant("runtime", "child")
.unwrap()
.is_none()
);
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("DROP TRIGGER fail_grant")?;
Ok(())
})
.unwrap();
store.bind_worker_execution_grant(&child).unwrap();
for sql in [
"UPDATE worker_execution_grants SET source_change_id='forged'",
"DELETE FROM worker_execution_grants",
"UPDATE subject_permission_changes SET record_json='{}'",
"DELETE FROM subject_permission_receipts",
] {
assert!(
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch(sql)?;
Ok(())
})
.is_err(),
"{sql}"
);
}
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
}
#[test]
fn current_schema_migration_preserves_subject_and_matches_new_database() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("migrated");
let original;
{
let (_old_manager, _old_scope, old) = open_version(&root, 11);
original = old
.create_subject_with_behavior(role(), "Keep behavior separate".into())
.unwrap();
}
let (_migrated_manager, _migrated_scope, migrated) = open_store(&root, "workspace-a");
assert_eq!(
migrated.subject(&original.id).unwrap(),
Some(original.clone())
);
assert!(
migrated
.subject_permissions(&original.id)
.unwrap()
.is_none()
);
let (_fresh_manager, _fresh_scope, fresh) =
open_store(&temp.path().join("fresh"), "workspace-a");
assert_eq!(schema(&migrated), schema(&fresh));
let source = migrated
.set_subject_permissions(&original.id, &update("new-source", None, Vec::new()))
.unwrap();
assert!(!source.change_id.is_empty());
assert_eq!(migrated.subject(&original.id).unwrap(), Some(original));
}
#[test]
fn unset_or_execution_only_source_cannot_issue_child_grants_and_saved_grants_survive_reopen() {
let temp = tempfile::tempdir().unwrap();
let saved;
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
assert!(
store
.bind_worker_execution_grant(&grant(&subject, "unset", "child"))
.is_err()
);
let mut execution_only = update("execution", None, Vec::new());
execution_only.execution = vec![permission("file.read", ".", true)];
let source = store
.set_subject_permissions(&subject, &execution_only)
.unwrap();
assert!(
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "child"))
.is_err()
);
let source = store
.set_subject_permissions(
&subject,
&update(
"delegation",
Some(&source.change_id),
execution_only.execution,
),
)
.unwrap();
saved = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&saved).unwrap();
}
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(saved.clone())
);
store.bind_worker_execution_grant(&saved).unwrap();
let mut conflicting = saved;
conflicting.profile = "builtin:default".into();
assert!(matches!(
store.bind_worker_execution_grant(&conflicting),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
}
#[test]
fn subject_grant_listing_is_scoped_retained_ordered_and_fails_on_overflow() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let other = store.create_subject(role()).unwrap().id;
assert!(
store
.list_subject_worker_execution_grants(&subject)
.unwrap()
.is_empty()
);
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let other_grant = grant(&other, &other_source.change_id, "other-child");
store.bind_worker_execution_grant(&other_grant).unwrap();
for index in (0..MAX_SUBJECT_WORKER_EXECUTION_GRANTS).rev() {
store
.bind_worker_execution_grant(&grant(
&subject,
&source.change_id,
&format!("child-{index:04}"),
))
.unwrap();
}
store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
store.retire_subject(&subject).unwrap();
let listed = store
.list_subject_worker_execution_grants(&subject)
.unwrap();
assert_eq!(listed.len(), MAX_SUBJECT_WORKER_EXECUTION_GRANTS);
assert!(
listed
.iter()
.all(|grant| grant.subject_id == subject && grant.source_change_id == source.change_id)
);
assert!(
listed
.windows(2)
.all(|pair| (&pair[0].runtime_id, &pair[0].worker_id)
< (&pair[1].runtime_id, &pair[1].worker_id))
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant.clone()]
);
// Seed one more retained row directly: the retired Subject cannot issue new grants.
// Listing must report overflow rather than returning a seemingly complete cancellation set.
let overflow = grant(&subject, &source.change_id, "overflow");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![overflow.runtime_id, overflow.worker_id, overflow.subject_id, overflow.source_change_id, serde_json::to_string(&overflow)?])?;
Ok(())
}).unwrap();
assert!(
matches!(store.list_subject_worker_execution_grants(&subject), Err(SubjektivError::WorkerExecutionGrantListOverflow { subject_id, limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS }) if subject_id == subject)
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant]
);
let (_foreign_manager, _foreign_scope, foreign) =
open_store(&temp.path().join("foreign"), "workspace-b");
assert!(matches!(
foreign.list_subject_worker_execution_grants(&subject),
Err(SubjektivError::SubjectNotFound(_))
));
}
#[test]
fn failed_schema_migration_rolls_back_new_tables_and_preserves_old_data() {
let temp = tempfile::tempdir().unwrap();
let original;
{
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
original = old.create_subject(role()).unwrap();
old.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TABLE subject_permissions (collision TEXT)")?;
Ok(())
})
.unwrap();
}
{
let manager = FeatureStorage::new(temp.path());
let scope = manager.scope("workspace-a").unwrap();
let registration = SubjektivStore::register(&scope).unwrap();
assert!(SubjektivStore::open(&scope, &registration).is_err());
}
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
assert_eq!(old.database.schema_version().unwrap(), 11);
assert_eq!(old.subject(&original.id).unwrap(), Some(original));
old.database.try_with_connection::<_, SubjektivError>(|connection| {
let created: bool = connection.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE name='subject_permission_changes')", [], |row| row.get(0))?;
assert!(!created);
Ok(())
}).unwrap();
}
+2 -1
View File
@@ -23,7 +23,8 @@ thiserror = { workspace = true }
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tracing = { workspace = true }
workdir = { workspace = true }
uuid = { workspace = true, features = ["v7"] }
[dev-dependencies]
filetime = "0.2.27"
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "test-util"] }
+246 -232
View File
@@ -1,24 +1,26 @@
//! Bash Tool using WorkdirSession command capability.
//!
//! The selected session owns cwd, process launch, bounded output, timeout, and cancellation.
//! Workdir owns launch and byte transport; Bash owns bounded preview and retained evidence.
//! Bash is not a sandbox; the Runtime process and OS user define the execution boundary.
//! Per-execution handles and nonces prevent cancellation of another command. Dropping a future
//! does not confirm termination; active registrations remain until cleanup requests cancellation
//! and observes terminal output.
//! does not confirm termination. Registrations remain during bounded cancellation/drain cleanup;
//! an unconfirmed provider outcome leaves partial evidence, never a fabricated terminal fence.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use crate::bash_output::{BashOutputMetadata, BashOutputView, BashOutputWriter};
use agen::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use async_trait::async_trait;
use schemars::JsonSchema;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
use workdir::{
CommandHandle, CommandOutputRequest, CommandRequest, WorkdirPath, WorkdirSessionHandle,
WorkdirSessionRouter,
};
mod cleanup;
const DEFAULT_TIMEOUT_SECS: u64 = 120;
const MAX_TIMEOUT_SECS: u64 = 600;
const INLINE_BYTE_BUDGET: usize = 12 * 1024;
@@ -28,16 +30,18 @@ struct BashParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
target_workdir: Option<String>,
/// Workdir-relative directory in which to execute the command.
/// Directory in which to execute the command.
// Keep provider path-coordinate documentation out of the model-facing schema.
#[schemars(with = "String")]
cwd: WorkdirPath,
command: String,
#[serde(default)]
timeout: Option<u64>,
}
pub(crate) struct BashTool {
pub struct BashTool {
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
state: Arc<Mutex<BashExecutionState>>,
}
@@ -63,6 +67,8 @@ struct CommandGuard {
execution_id: String,
execution_nonce: u64,
handle: Option<CommandHandle>,
cursor: u64,
retained: Option<BashOutputWriter>,
}
impl Drop for CommandGuard {
@@ -70,46 +76,69 @@ impl Drop for CommandGuard {
let Some(handle) = self.handle.take() else {
return;
};
let workdir = self.session.clone();
let state = Arc::clone(&self.state);
let execution_id = self.execution_id.clone();
let execution_nonce = self.execution_nonce;
// A dropped provider future is not terminal confirmation. Keep the live
// execution registered until cleanup has both requested cancellation and
// observed terminal command output, so cancellation/session teardown
// cannot race with an apparently empty registry.
tokio::spawn(async move {
let _ = workdir.cancel_command(handle.clone()).await;
let _ = workdir
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
.await;
let mut state = state.lock().unwrap();
if state
.active
.get(&execution_id)
.is_some_and(|active| active.execution_nonce == execution_nonce)
{
state.active.remove(&execution_id);
state.cancellation_requested.remove(&execution_id);
}
});
let cleanup = cleanup::Cleanup {
session: self.session.clone(),
state: Arc::clone(&self.state),
execution_id: self.execution_id.clone(),
execution_nonce: self.execution_nonce,
handle,
cursor: self.cursor,
retained: self
.retained
.take()
.expect("started command owns output writer"),
};
tokio::spawn(cleanup.run());
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
// Output pages are replayable only with the exact same command/cursor/limit.
// Retry read transport failures, never command start, and cap both attempts and
// backoff. The caller's deadline bounds a disconnected or unresponsive provider.
async fn command_page(
session: &workdir::WorkdirSessionHandle,
request: CommandOutputRequest,
) -> Result<workdir::CommandOutput, workdir::WorkdirError> {
for attempt in 0..3 {
match session.command_output(request.clone()).await {
Err(error)
if attempt < 2
&& matches!(
error.classification_source(),
workdir::WorkdirError::Transport(_)
| workdir::WorkdirError::Unavailable(_)
| workdir::WorkdirError::OutcomeUnknown(_)
) =>
{
tokio::time::sleep(std::time::Duration::from_millis(100 << attempt)).await;
}
result => return result,
}
}
unreachable!("bounded retry loop returns its last result")
}
impl BashTool {
pub fn input_schema() -> serde_json::Value {
serde_json::to_value(schemars::schema_for!(BashParams)).expect("Bash schema serialization")
}
pub fn new(router: Arc<WorkdirSessionRouter>, output: BashOutputView) -> Self {
Self {
router,
output,
state: Arc::new(Mutex::new(BashExecutionState::default())),
}
}
pub async fn execute_result(
&self,
input_json: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
) -> Result<BashResult, ToolError> {
let params: BashParams = serde_json::from_str(input_json)
.map_err(|error| ToolError::InvalidArgument(format!("invalid Bash input: {error}")))?;
let retained = self.output.begin().map_err(ToolError::ExecutionFailed)?;
let selected = crate::routing::resolve_session(
&self.router,
params.target_workdir.as_deref(),
@@ -134,14 +163,15 @@ impl Tool for BashTool {
execution_id: execution_id.clone(),
execution_nonce,
handle: None,
cursor: 0,
retained: Some(retained),
};
let handle = session
.start_command(CommandRequest {
command: params.command,
timeout_secs,
output_limit: INLINE_BYTE_BUDGET,
transport_byte_budget: self.output.byte_budget(),
cwd: params.cwd,
spill_dir: Some(self.output_dir.clone()),
tool_call_id: Some(call_id.clone()),
})
.await
@@ -167,15 +197,101 @@ impl Tool for BashTool {
.await
.map_err(crate::ToolsError::from)?;
}
let output = session
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
let mut cursor = 0;
let mut stdout_preview = Vec::new();
let mut stderr_preview = Vec::new();
let mut seen = [0_u64; 2];
let mut empty_polls = 0;
let deadline =
tokio::time::Instant::now() + std::time::Duration::from_secs(timeout_secs + 30);
let terminal = loop {
let page = tokio::time::timeout_at(
deadline,
command_page(
&session,
CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
),
)
.await
.map_err(|_| {
ToolError::ExecutionFailed(
"command byte transport deadline exceeded; terminal outcome unknown".into(),
)
})?
.map_err(crate::ToolsError::from)?;
let mut next = cursor;
for chunk in &page.chunks {
if chunk.sequence != next || chunk.bytes.is_empty() {
return Err(ToolError::ExecutionFailed(
"command byte transport sequence gap".into(),
));
}
next = next
.checked_add(chunk.bytes.len() as u64)
.filter(|n| *n <= self.output.byte_budget())
.ok_or_else(|| {
ToolError::ExecutionFailed(
"command byte transport exceeds delivery budget".into(),
)
})?;
let index = if chunk.stream == workdir::CommandStream::Stdout {
0
} else {
1
};
if chunk.offset != seen[index] {
return Err(ToolError::ExecutionFailed(
"command byte transport offset gap".into(),
));
}
seen[index] += chunk.bytes.len() as u64;
guard
.retained
.as_mut()
.unwrap()
.append_chunk(chunk)
.map_err(ToolError::ExecutionFailed)?;
let available =
INLINE_BYTE_BUDGET.saturating_sub(stdout_preview.len() + stderr_preview.len());
let preview = if index == 0 {
&mut stdout_preview
} else {
&mut stderr_preview
};
preview.extend_from_slice(&chunk.bytes[..available.min(chunk.bytes.len())]);
}
if page.next_cursor != next {
return Err(ToolError::ExecutionFailed(
"command byte transport cursor does not fence the page".into(),
));
}
if page.chunks.is_empty() && page.terminal.is_none() {
empty_polls += 1;
if empty_polls > 32 {
return Err(ToolError::ExecutionFailed(
"command byte transport made no progress after bounded polls".into(),
));
}
} else {
empty_polls = 0;
}
cursor = page.next_cursor;
guard.cursor = cursor;
if let Some(terminal) = page.terminal {
break terminal;
}
};
let metadata = guard
.retained
.as_mut()
.unwrap()
.finish_transport(&terminal)
.map_err(ToolError::ExecutionFailed)?;
let cancellation_requested = {
let mut state = self.state.lock().unwrap();
let owns_registration = state
@@ -193,41 +309,84 @@ impl Tool for BashTool {
};
guard.handle = None;
let timed_out = output.timed_out;
let summary = if cancellation_requested {
format!("$ {cmd_summary} (cancelled)")
} else if output.timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match output.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
let content = if output.content.is_empty() {
None
} else if output.truncated {
let notice = match output.output_path {
Some(path) => format!(
"[showing bounded WorkdirSession command output; full output saved to {}]",
path.display()
),
None => "[showing bounded WorkdirSession command output; additional output was truncated]"
.to_owned(),
let timed_out = terminal.status == workdir::CommandStatus::TimedOut;
let mut summary =
if cancellation_requested || terminal.status == workdir::CommandStatus::Cancelled {
format!("$ {cmd_summary} (cancelled)")
} else if timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match terminal.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
Some(format!("{notice}\n{}", output.content))
} else {
Some(output.content)
};
let output = ToolOutput {
if matches!(
terminal.output_collection,
workdir::CommandOutputCollection::Incomplete { .. }
) {
summary.push_str(" (output incomplete)");
}
Ok(BashResult {
summary,
content,
attachments: Vec::new(),
};
if cancellation_requested {
stdout_preview: String::from_utf8_lossy(&stdout_preview).into_owned(),
stderr_preview: String::from_utf8_lossy(&stderr_preview).into_owned(),
preview_omitted: terminal.stdout_bytes.saturating_add(terminal.stderr_bytes)
> (stdout_preview.len() + stderr_preview.len()) as u64,
metadata,
cancelled: cancellation_requested
|| terminal.status == workdir::CommandStatus::Cancelled,
})
}
}
#[derive(Debug, Serialize)]
pub struct BashResult {
pub summary: String,
pub stdout_preview: String,
pub stderr_preview: String,
pub preview_omitted: bool,
pub metadata: BashOutputMetadata,
pub cancelled: bool,
}
impl BashResult {
pub fn stdout_entry(&self) -> String {
format!("/bash-output/{}/stdout", self.metadata.id)
}
pub fn stderr_entry(&self) -> String {
format!("/bash-output/{}/stderr", self.metadata.id)
}
pub fn order_entry(&self) -> String {
format!("/bash-output/{}/order", self.metadata.id)
}
pub fn tool_output(&self) -> ToolOutput {
let content = serde_json::json!({
"stdout_preview": self.stdout_preview, "stderr_preview": self.stderr_preview,
"preview_omitted": self.preview_omitted, "output": self.metadata,
"stdout_entry": self.stdout_entry(), "stderr_entry": self.stderr_entry(), "order_entry": self.order_entry(),
});
ToolOutput {
summary: self.summary.clone(),
content: Some(content.to_string()),
attachments: vec![],
}
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
&self,
input: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
let result = self.execute_result(input, ctx).await?;
let output = result.tool_output();
if result.cancelled {
Err(ToolError::Cancelled(output))
} else if timed_out {
} else if result.metadata.terminal.status == Some(workdir::CommandStatus::TimedOut) {
Err(ToolError::Interrupted(output))
} else {
Ok(output)
@@ -288,164 +447,19 @@ fn truncate_for_summary(command: &str) -> String {
summary
}
pub fn bash_tool(session: WorkdirSessionHandle, output_dir: PathBuf) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output_dir)
pub fn bash_tool(session: WorkdirSessionHandle, output: BashOutputView) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output)
}
pub(crate) fn routed_bash_tool(
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
) -> ToolDefinition {
Arc::new(move || {
let schema = schemars::schema_for!(BashParams);
let meta = ToolMeta::new("Bash")
.description("Execute a shell command in one selected Workdir. Process start, bounded inline output, full-output spill, timeout and cancellation are owned by that WorkdirSession provider. This is not a sandbox.")
.input_schema(serde_json::to_value(schema).expect("Bash schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(BashTool {
router: router.clone(),
output_dir: output_dir.clone(),
state: Arc::new(Mutex::new(BashExecutionState::default())),
});
.description("Execute a shell command in one selected Workdir. Workdir owns launch, lossless bounded byte transport, timeout and cancel; Bash owns preview, quota-limited ephemeral output and read-only virtual references. This is not a sandbox.")
.input_schema(BashTool::input_schema());
let tool: Arc<dyn Tool> = Arc::new(BashTool::new(router.clone(), output.clone()));
(meta, tool)
})
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use tempfile::TempDir;
use workdir::{LocalWorkdirSession, WorkdirSessionHandle};
use super::bash_tool;
use crate::{grep::grep_tool, read::read_tool, tracker::Tracker};
fn session_with_output_scope(root: &TempDir, output: &TempDir) -> WorkdirSessionHandle {
let scope = Scope::from_config(&ScopeConfig {
allow: vec![
ScopeRule {
target: root.path().to_path_buf(),
permission: Permission::Write,
recursive: true,
symlink_policy: Default::default(),
},
ScopeRule {
target: output.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
},
],
deny: Vec::new(),
})
.unwrap();
Arc::new(LocalWorkdirSession::new(scope, root.path().to_path_buf()))
}
#[tokio::test]
async fn long_output_is_spilled_and_available_to_read_and_grep() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session.clone(), output.path().to_path_buf())();
let command = "i=0; while [ $i -lt 2000 ]; do printf 'line-%04d\\n' \"$i\"; i=$((i+1)); done; printf 'FINAL-NEEDLE\\n'";
let result = bash
.execute(
&serde_json::json!({ "command": command, "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
let rendered = result.content.expect("bounded Bash output");
let artifact = std::fs::read_dir(output.path())
.unwrap()
.next()
.expect("artifact entry")
.unwrap()
.path();
assert!(rendered.contains("full output saved to"));
assert!(rendered.contains(&artifact.display().to_string()));
let retained = std::fs::read_to_string(&artifact).unwrap();
assert!(retained.starts_with("line-0000\n"));
assert!(retained.ends_with("FINAL-NEEDLE\n"));
assert_eq!(retained.lines().count(), 2001);
let (_, read) = read_tool(session.clone(), Tracker::new())();
let read_result = read
.execute(
&serde_json::json!({
"file_path": artifact,
"offset": 2000,
"limit": 1,
})
.to_string(),
Default::default(),
)
.await
.unwrap();
assert!(
read_result
.content
.expect("Read content")
.contains("FINAL-NEEDLE")
);
let (_, grep) = grep_tool(session)();
let grep_result = grep
.execute(
&serde_json::json!({
"pattern": "FINAL-NEEDLE",
"path": artifact,
"output_mode": "content",
})
.to_string(),
Default::default(),
)
.await
.unwrap();
let grep_content = grep_result.content.expect("Grep content");
assert!(
grep_content.contains("FINAL-NEEDLE"),
"unexpected Grep content: {grep_content:?}"
);
}
#[tokio::test]
async fn missing_cwd_is_rejected_before_command_start() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let error = bash
.execute(
&serde_json::json!({ "command": "printf hidden" }).to_string(),
Default::default(),
)
.await
.unwrap_err();
assert!(error.to_string().contains("missing field `cwd`"));
}
#[tokio::test]
async fn short_output_does_not_leave_a_spill_artifact() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let result = bash
.execute(
&serde_json::json!({ "command": "printf short", "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
assert_eq!(result.content.as_deref(), Some("short"));
assert_eq!(std::fs::read_dir(output.path()).unwrap().count(), 0);
}
}
+121
View File
@@ -0,0 +1,121 @@
//! Finite Drop cleanup. A cancel acknowledgement is not termination evidence; only a
//! validated terminal fence confirms the process result. No provider text is logged.
use super::*;
use crate::bash_output::{
CleanupCancel, CleanupDiagnostics, CleanupDrain, CleanupFailure, CleanupProviderFailure,
CleanupRegistration,
};
use std::time::Duration;
const CLEANUP_TIMEOUT: Duration = Duration::from_secs(10);
// Reserve time for independent terminal observation even if cancel never responds.
const CANCEL_TIMEOUT: Duration = Duration::from_secs(2);
pub(super) struct Cleanup {
pub session: WorkdirSessionHandle,
pub state: Arc<Mutex<BashExecutionState>>,
pub execution_id: String,
pub execution_nonce: u64,
pub handle: CommandHandle,
pub cursor: u64,
pub retained: BashOutputWriter,
}
impl Cleanup {
pub async fn run(mut self) {
let deadline = tokio::time::Instant::now() + CLEANUP_TIMEOUT;
let cancel = match tokio::time::timeout(
CANCEL_TIMEOUT,
self.session.cancel_command(self.handle.clone()),
)
.await
{
Ok(Ok(())) => CleanupCancel::Acknowledged,
Ok(Err(error)) => CleanupCancel::Failed {
reason: CleanupProviderFailure::from(&error),
},
Err(_) => CleanupCancel::DeadlineExceeded,
};
let drain = match tokio::time::timeout_at(deadline, self.drain()).await {
Ok(Ok(())) => CleanupDrain::TerminalConfirmed,
Ok(Err(reason)) => CleanupDrain::Failed { reason },
Err(_) => CleanupDrain::DeadlineExceeded,
};
let deadline_expired = matches!(drain, CleanupDrain::DeadlineExceeded);
// Finalize evidence before exposing an empty registry. Keep the lock through
// publication so an older cleanup can never remove a newer generation.
let mut state = self.state.lock().unwrap();
let registration = if state
.active
.get(&self.execution_id)
.is_some_and(|active| active.execution_nonce == self.execution_nonce)
{
state.active.remove(&self.execution_id);
state.cancellation_requested.remove(&self.execution_id);
CleanupRegistration::Released
} else {
CleanupRegistration::Superseded
};
let diagnostics = CleanupDiagnostics {
execution_nonce: self.execution_nonce,
cancel,
drain,
deadline_expired,
registration,
};
tracing::info!(
output_id = %self.retained.id(),
cleanup = ?diagnostics,
"Bash Drop cleanup finished; cancellation and terminal observation are independent"
);
self.retained.finish_cleanup(diagnostics);
drop(self.retained);
}
async fn drain(&mut self) -> Result<(), CleanupFailure> {
loop {
let page = command_page(
&self.session,
CommandOutputRequest {
handle: self.handle.clone(),
cursor: self.cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
)
.await
.map_err(|error| CleanupFailure::Provider {
reason: CleanupProviderFailure::from(&error),
})?;
let mut next = self.cursor;
for chunk in &page.chunks {
if chunk.sequence != next {
return Err(CleanupFailure::SequenceGap);
}
next = next
.checked_add(chunk.bytes.len() as u64)
.filter(|next| *next <= crate::bash_output::COMMAND_BYTES as u64)
.ok_or(CleanupFailure::DeliveryBudgetExceeded)?;
self.retained
.append_chunk(chunk)
.map_err(|_| CleanupFailure::RetainedChunkInvalid)?;
}
if page.next_cursor != next {
return Err(CleanupFailure::CursorGap);
}
self.cursor = next;
if let Some(terminal) = page.terminal {
self.retained
.finish_transport(&terminal)
.map_err(|_| CleanupFailure::TerminalMetadataInvalid)?;
return Ok(());
}
// An immediately-ready provider must not starve the cleanup deadline.
tokio::task::yield_now().await;
}
}
}
#[cfg(test)]
mod tests;
+397
View File
@@ -0,0 +1,397 @@
//! Component tests enter through the production CommandGuard Drop, cleanup, page
//! validation and retained store. Only the Workdir provider is replaced; paused time
//! and a cancel barrier make faults/deadlines deterministic without OS timing.
use super::*;
use std::sync::atomic::{AtomicBool, Ordering};
use workdir::*;
const PRIVATE: &str = "SECRET command /private/host/path output password=poison";
#[derive(Clone, Copy, Debug)]
enum Fault {
Cancel,
CancelHang,
Drain,
TerminalTimeout,
Sequence,
Cursor,
RetainedChunk,
TerminalMetadata,
CollectionIncomplete,
Complete,
}
#[derive(Debug)]
struct Provider {
local: LocalWorkdirSession,
fault: Fault,
cancelling: tokio::sync::Notify,
allow_drain: AtomicBool,
}
#[async_trait]
impl WorkdirSession for Provider {
fn workdir(&self) -> &Workdir {
self.local.workdir()
}
fn capabilities(&self) -> WorkdirSessionCapabilities {
WorkdirSessionCapabilities::ALL
}
async fn stat(&self, _: StatRequest) -> Result<StatResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn read(&self, _: ReadRequest) -> Result<ReadResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn write(&self, _: WriteRequest) -> Result<WriteResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn edit(&self, _: EditRequest) -> Result<EditResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn list(&self, _: ListRequest) -> Result<ListResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn glob(&self, _: GlobRequest) -> Result<GlobResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn grep(&self, _: GrepRequest) -> Result<GrepResult, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn start_command(&self, _: CommandRequest) -> Result<CommandHandle, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn command_status(&self, _: CommandHandle) -> Result<CommandStatus, WorkdirError> {
Err(WorkdirError::OperationFailed)
}
async fn command_output(
&self,
request: CommandOutputRequest,
) -> Result<CommandOutput, WorkdirError> {
assert_eq!(request.cursor, 3);
while !self.allow_drain.load(Ordering::Acquire) {
tokio::task::yield_now().await;
}
match self.fault {
Fault::Drain => {
return Err(WorkdirError::Transport(PRIVATE.into())
.with_failure_reason(WorkdirFailureReason::HttpTransfer));
}
Fault::TerminalTimeout => return std::future::pending().await,
_ => {}
}
let mut chunk = CommandByteChunk {
sequence: 3,
offset: 3,
stream: CommandStream::Stdout,
bytes: b"tail".to_vec(),
};
if matches!(self.fault, Fault::Sequence) {
chunk.sequence = 4;
}
if matches!(self.fault, Fault::RetainedChunk) {
chunk.offset = 42;
}
let mut terminal = terminal();
if matches!(self.fault, Fault::TerminalMetadata) {
terminal.stdout_delivered_bytes = 6;
}
if matches!(self.fault, Fault::CollectionIncomplete) {
terminal.output_collection = CommandOutputCollection::Incomplete {
failure: CommandFailure {
stage: CommandFailureStage::StdoutDrain,
reason: CommandIoErrorKind::Other,
os_code: None,
},
};
}
Ok(CommandOutput {
chunks: vec![chunk],
next_cursor: if matches!(self.fault, Fault::Cursor) {
8
} else {
7
},
terminal: Some(terminal),
})
}
async fn cancel_command(&self, _: CommandHandle) -> Result<(), WorkdirError> {
self.cancelling.notify_one();
match self.fault {
Fault::Cancel => Err(WorkdirError::Unavailable(PRIVATE.into())),
Fault::CancelHang => std::future::pending().await,
_ => Ok(()),
}
}
async fn close(&self) -> Result<(), WorkdirError> {
Ok(())
}
}
fn terminal() -> CommandTerminal {
CommandTerminal {
status: CommandStatus::Completed,
exit_code: Some(0),
process_failure: None,
output_collection: CommandOutputCollection::Complete,
stdout_bytes: 7,
stderr_bytes: 0,
stdout_delivered_bytes: 7,
stderr_delivered_bytes: 0,
truncation_reason: None,
}
}
struct Fixture {
_root: tempfile::TempDir,
provider: Arc<Provider>,
state: Arc<Mutex<BashExecutionState>>,
view: BashOutputView,
id: String,
}
impl Fixture {
fn start(fault: Fault) -> (Self, CommandGuard) {
let base =
std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../target/cleanup-fixtures");
std::fs::create_dir_all(&base).unwrap();
let root = tempfile::tempdir_in(base).unwrap();
let provider = Arc::new(Provider {
local: LocalWorkdirSession::new(
manifest::Scope::writable(root.path()).unwrap(),
root.path().into(),
),
fault,
cancelling: Default::default(),
allow_drain: AtomicBool::new(false),
});
let view = BashOutputView::new();
let mut retained = view.begin().unwrap();
retained.append(CommandStream::Stdout, b"pre");
let id = retained.id();
let state = Arc::new(Mutex::new(BashExecutionState::default()));
let handle = CommandHandle(PRIVATE.into());
state.lock().unwrap().active.insert(
PRIVATE.into(),
ActiveCommand {
call_id: PRIVATE.into(),
execution_nonce: 1,
session: provider.clone(),
handle: handle.clone(),
},
);
state
.lock()
.unwrap()
.cancellation_requested
.insert(PRIVATE.into());
let guard = CommandGuard {
session: provider.clone(),
state: state.clone(),
execution_id: PRIVATE.into(),
execution_nonce: 1,
handle: Some(handle),
cursor: 3,
retained: Some(retained),
};
(
Self {
_root: root,
provider,
state,
view,
id,
},
guard,
)
}
async fn finish(&self) -> BashOutputMetadata {
self.provider.allow_drain.store(true, Ordering::Release);
// The provider's read retry backoff and cleanup deadlines are virtual time.
for _ in 0..4 {
tokio::task::yield_now().await;
tokio::time::advance(Duration::from_secs(3)).await;
}
for _ in 0..1000 {
if let Some(metadata) = self.view.metadata(&self.id) {
return metadata;
}
tokio::task::yield_now().await;
}
panic!("finite cleanup did not publish partial evidence");
}
fn assert_evidence(&self, m: &BashOutputMetadata, terminal: bool, bytes: &[u8]) {
assert_eq!(
self.view
.read(&self.id, CommandStream::Stdout, 0, 64)
.unwrap(),
bytes
);
assert_eq!(m.stdout_saved, bytes.len());
assert_eq!(m.stdout_delivered, bytes.len() as u64);
assert_eq!(m.stdout_seen, bytes.len() as u64);
assert_eq!(
m.terminal.status,
terminal.then_some(CommandStatus::Completed)
);
assert_eq!(m.terminal.exit_code, terminal.then_some(0));
if !terminal {
assert!(m.terminal.output_collection.is_none());
assert!(m.terminal.process_failure.is_none());
assert!(!m.terminal.transport_complete);
}
let diagnostics = m.cleanup.as_ref().unwrap();
assert_eq!(diagnostics.execution_nonce, 1);
let json = serde_json::to_string(diagnostics).unwrap();
let debug = format!("{diagnostics:?}");
for forbidden in [
PRIVATE, "SECRET", "/private", "password", "poison", "tail", "pre",
] {
assert!(!json.contains(forbidden), "diagnostics leaked data: {json}");
assert!(
!debug.contains(forbidden),
"tracing payload leaked data: {debug}"
);
}
}
}
#[tokio::test(start_paused = true)]
async fn cancel_failure_does_not_preclude_confirmed_terminal() {
let (f, guard) = Fixture::start(Fault::Cancel);
drop(guard);
f.provider.cancelling.notified().await;
assert!(f.state.lock().unwrap().active.contains_key(PRIVATE));
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(m.terminal.transport_complete);
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::Failed { .. }
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(!m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
assert!(f.state.lock().unwrap().cancellation_requested.is_empty());
}
#[tokio::test(start_paused = true)]
async fn acknowledged_cancel_drain_faults_keep_distinct_reasons_and_partial_bytes() {
for (fault, reason, bytes) in [
(Fault::Drain, "provider", b"pre".as_slice()),
(Fault::Sequence, "sequence_gap", b"pre"),
(Fault::Cursor, "cursor_gap", b"pretail"),
(Fault::RetainedChunk, "retained_chunk_invalid", b"pre"),
(
Fault::TerminalMetadata,
"terminal_metadata_invalid",
b"pretail",
),
] {
let (f, guard) = Fixture::start(fault);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, false, bytes);
let diagnostic = serde_json::to_value(m.cleanup.as_ref().unwrap()).unwrap();
assert_eq!(diagnostic["cancel"]["outcome"], "acknowledged");
assert_eq!(diagnostic["drain"]["outcome"], "failed");
assert_eq!(diagnostic["drain"]["reason"]["kind"], reason);
assert_eq!(diagnostic["registration"], "released");
assert_eq!(diagnostic["deadline_expired"], false);
assert!(f.state.lock().unwrap().active.is_empty());
}
}
#[tokio::test(start_paused = true)]
async fn terminal_deadline_never_claims_stop_or_leaves_active_collector() {
let (f, guard) = Fixture::start(Fault::TerminalTimeout);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, false, b"pre");
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::Acknowledged
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::DeadlineExceeded
));
assert!(m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
}
#[tokio::test(start_paused = true)]
async fn cancel_deadline_still_attempts_independent_terminal_observation() {
let (f, guard) = Fixture::start(Fault::CancelHang);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(matches!(
m.cleanup.as_ref().unwrap().cancel,
CleanupCancel::DeadlineExceeded
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(!m.cleanup.as_ref().unwrap().deadline_expired);
assert!(f.state.lock().unwrap().active.is_empty());
}
#[tokio::test(start_paused = true)]
async fn old_cleanup_preserves_new_execution_and_cancellation_registration() {
let (f, guard) = Fixture::start(Fault::Complete);
drop(guard);
f.provider.cancelling.notified().await;
f.state
.lock()
.unwrap()
.active
.get_mut(PRIVATE)
.unwrap()
.execution_nonce = 2;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(matches!(
m.cleanup.as_ref().unwrap().registration,
CleanupRegistration::Superseded
));
assert_eq!(f.state.lock().unwrap().active[PRIVATE].execution_nonce, 2);
assert!(
f.state
.lock()
.unwrap()
.cancellation_requested
.contains(PRIVATE)
);
}
#[tokio::test(start_paused = true)]
async fn terminal_confirmation_is_independent_of_output_collection_completeness() {
let (f, guard) = Fixture::start(Fault::CollectionIncomplete);
drop(guard);
f.provider.cancelling.notified().await;
let m = f.finish().await;
f.assert_evidence(&m, true, b"pretail");
assert!(!m.terminal.transport_complete);
assert!(matches!(
m.terminal.output_collection,
Some(CommandOutputCollection::Incomplete { .. })
));
assert!(matches!(
m.cleanup.as_ref().unwrap().drain,
CleanupDrain::TerminalConfirmed
));
assert!(f.state.lock().unwrap().active.is_empty());
}
+725
View File
@@ -0,0 +1,725 @@
//! Bash-owned bounded output, independent of Workdir transport and Console subscribers.
//!
//! Output is ephemeral evidence, not a filesystem grant. Every lookup checks the live view;
//! children can see their own subtree and ancestors can retain evidence after child revocation.
//! Root teardown clears all bytes. A new root (including restore) has a new incarnation and
//! never reconstructs grants from history. Expiry is measured from admission, enforced on
//! lookup and by a weak reaper once commands run in Tokio. Parent and all descendants share
//! one pool: admission reserves a whole command budget, evicts the oldest terminal evidence
//! first, and refuses rather than evicting a live collector. Revoking a child denies access
//! but leaves ancestor evidence until pool eviction/expiry; root teardown clears the pool.
use std::collections::VecDeque;
use std::sync::{
Arc, Mutex, Weak,
atomic::{AtomicBool, Ordering},
};
use std::time::{Duration, Instant};
use serde::Serialize;
use workdir::{CommandStatus, CommandStream};
pub(crate) const COMMAND_BYTES: usize = 4 * 1024 * 1024;
const STORE_BYTES: usize = 32 * 1024 * 1024;
const STORE_COUNT: usize = 64;
const ORDER_COUNT: usize = 4096;
const RETENTION: Duration = Duration::from_secs(3600);
#[derive(Clone)]
pub struct BashOutputView {
store: Arc<Mutex<Store>>,
lineage: Vec<String>,
active: Arc<AtomicBool>,
authority: Arc<dyn Fn() -> bool + Send + Sync>,
}
struct Store {
records: VecDeque<Arc<Mutex<Record>>>,
reserved: usize,
retention: Duration,
reaper_started: bool,
_shutdown: tokio::sync::watch::Sender<()>,
}
struct Record {
id: String,
owner: Vec<String>,
created: Instant,
stdout: Vec<u8>,
stderr: Vec<u8>,
order: Vec<OutputRange>,
stdout_seen: u64,
stderr_seen: u64,
truncated: bool,
limit: Option<RetentionLimit>,
transport_counts: Option<(u64, u64)>,
transport_truncation: Option<workdir::CommandOutputTruncation>,
terminal: Option<OutputTerminal>,
cleanup: Option<CleanupDiagnostics>,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputRange {
pub stream: CommandStream,
pub start: u64,
pub end: u64,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputTerminal {
pub status: Option<CommandStatus>,
pub exit_code: Option<i32>,
pub process_failure: Option<workdir::CommandFailure>,
pub output_collection: Option<workdir::CommandOutputCollection>,
/// True only for confirmed EOF collection (quota is reported separately). A terminal
/// fence or exit zero alone does not confirm the unobserved process tail.
pub transport_complete: bool,
}
#[derive(Clone, Copy, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum RetentionLimit {
ByteQuota,
OrderQuota,
}
#[derive(Clone, Debug, Serialize)]
pub struct BashOutputMetadata {
pub id: String,
pub stdout_saved: usize,
pub stderr_saved: usize,
pub stdout_seen: u64,
pub stderr_seen: u64,
pub stdout_delivered: u64,
pub stderr_delivered: u64,
pub retention_limit: Option<RetentionLimit>,
pub transport_truncation: Option<workdir::CommandOutputTruncation>,
pub storage_truncated: bool,
pub terminal: OutputTerminal,
pub cleanup: Option<CleanupDiagnostics>,
pub text_encoding: &'static str,
pub ordering: &'static str,
}
/// Drop cleanup observations, not assertions that cancellation stopped a process.
/// Correlate with the enclosing output UUID and this host-assigned execution generation;
/// caller identifiers, provider handles, commands and error text are deliberately excluded.
#[derive(Clone, Debug, Serialize)]
pub struct CleanupDiagnostics {
pub execution_nonce: u64,
pub cancel: CleanupCancel,
pub drain: CleanupDrain,
pub deadline_expired: bool,
pub registration: CleanupRegistration,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "outcome", rename_all = "snake_case")]
pub enum CleanupCancel {
Acknowledged,
Failed { reason: CleanupProviderFailure },
DeadlineExceeded,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "outcome", rename_all = "snake_case")]
pub enum CleanupDrain {
TerminalConfirmed,
Failed { reason: CleanupFailure },
DeadlineExceeded,
}
#[derive(Clone, Debug, Serialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum CleanupFailure {
Provider { reason: CleanupProviderFailure },
SequenceGap,
CursorGap,
DeliveryBudgetExceeded,
RetainedChunkInvalid,
TerminalMetadataInvalid,
}
#[derive(Clone, Debug, Serialize)]
pub struct CleanupProviderFailure {
pub failure_reason: Option<workdir::WorkdirFailureReason>,
pub denial_reason: Option<workdir::WorkdirDenialReason>,
}
impl From<&workdir::WorkdirError> for CleanupProviderFailure {
fn from(error: &workdir::WorkdirError) -> Self {
Self {
failure_reason: error.failure_reason(),
denial_reason: error.denial_reason(),
}
}
}
#[derive(Clone, Copy, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum CleanupRegistration {
Released,
Superseded,
}
/// A bounded reservation acquired before command start. Drop terminalizes partial evidence.
pub struct BashOutputWriter {
record: Arc<Mutex<Record>>,
}
impl Default for BashOutputView {
fn default() -> Self {
Self::new()
}
}
impl BashOutputView {
pub fn new() -> Self {
let (shutdown, _) = tokio::sync::watch::channel(());
let store = Arc::new(Mutex::new(Store {
records: VecDeque::new(),
reserved: 0,
retention: RETENTION,
reaper_started: false,
_shutdown: shutdown,
}));
let view = Self {
store,
lineage: vec![uuid::Uuid::now_v7().to_string()],
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(|| true),
};
view.start_reaper();
view
}
fn start_reaper(&self) {
let Ok(runtime) = tokio::runtime::Handle::try_current() else {
return;
};
let mut store = self.store.lock().unwrap();
if store.reaper_started {
return;
}
let receiver = store._shutdown.subscribe();
store.reaper_started = true;
let weak = Arc::downgrade(&self.store);
runtime.spawn(reap(weak, receiver));
}
pub fn byte_budget(&self) -> u64 {
COMMAND_BYTES as u64
}
pub fn child(&self, authority: Arc<dyn Fn() -> bool + Send + Sync>) -> Self {
let parent = self.clone();
let mut lineage = self.lineage.clone();
lineage.push(uuid::Uuid::now_v7().to_string());
Self {
store: self.store.clone(),
lineage,
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(move || parent.is_active() && authority()),
}
}
pub fn is_active(&self) -> bool {
self.active.load(Ordering::Acquire) && (self.authority)()
}
pub fn is_root(&self) -> bool {
self.lineage.len() == 1
}
pub fn revoke(&self) {
self.active.store(false, Ordering::Release);
if self.lineage.len() == 1 {
let mut store = self.store.lock().unwrap();
store.records.clear();
store.reserved = 0;
}
}
pub fn begin(&self) -> Result<BashOutputWriter, String> {
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
// A host may construct the Feature before entering Tokio. Command admission
// happens inside the runtime, so idle expiry must start here as well.
self.start_reaper();
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
store.expire(Instant::now());
while store.records.len() >= STORE_COUNT || store.reserved + COMMAND_BYTES > STORE_BYTES {
let Some(index) = store
.records
.iter()
.position(|r| r.lock().unwrap().terminal.is_some())
else {
return Err("Bash output capacity is occupied by live commands".into());
};
store.records.remove(index);
store.reserved -= COMMAND_BYTES;
}
let record = Arc::new(Mutex::new(Record {
id: uuid::Uuid::now_v7().to_string(),
owner: self.lineage.clone(),
created: Instant::now(),
stdout: Vec::new(),
stderr: Vec::new(),
order: Vec::new(),
stdout_seen: 0,
stderr_seen: 0,
truncated: false,
limit: None,
transport_counts: None,
transport_truncation: None,
terminal: None,
cleanup: None,
}));
store.records.push_back(record.clone());
store.reserved += COMMAND_BYTES;
Ok(BashOutputWriter { record })
}
fn record(&self, id: &str) -> Option<Arc<Mutex<Record>>> {
if !self.is_active() {
return None;
}
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return None;
}
store.expire(Instant::now());
store
.records
.iter()
.find(|r| {
let r = r.lock().unwrap();
r.id == id && r.owner.starts_with(&self.lineage) && r.terminal.is_some()
})
.cloned()
}
pub fn metadata(&self, id: &str) -> Option<BashOutputMetadata> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.metadata())
}
/// Copies at most the caller's bounded range; never publishes output bodies in descriptors.
pub fn read(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Vec<u8>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let start = offset.min(bytes.len());
Some(bytes[start..start.saturating_add(limit.min(64 * 1024)).min(bytes.len())].to_vec())
}
/// Scan the bounded raw source without cloning it, decoding only the requested line window.
/// A long single line uses the common byte reader instead of allocating an oversized text page.
pub fn line_window(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Result<(String, bool), String>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let mut lines = bytes.split_inclusive(|b| *b == b'\n').skip(offset);
let mut window = Vec::new();
for line in lines.by_ref().take(limit.min(2000)) {
if window.len() + line.len() > 64 * 1024 {
return Some(Err(
"text source window exceeds 64 KiB; use the common byte reader".into(),
));
}
window.extend_from_slice(line);
}
let more = lines.next().is_some();
Some(Ok((String::from_utf8_lossy(&window).into_owned(), more)))
}
pub fn order(&self, id: &str) -> Option<Vec<OutputRange>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.order.clone())
}
}
impl Record {
fn metadata(&self) -> BashOutputMetadata {
BashOutputMetadata {
id: self.id.clone(),
stdout_saved: self.stdout.len(),
stderr_saved: self.stderr.len(),
stdout_seen: self.transport_counts.map_or(self.stdout_seen, |c| c.0),
stderr_seen: self.transport_counts.map_or(self.stderr_seen, |c| c.1),
stdout_delivered: self.stdout_seen,
stderr_delivered: self.stderr_seen,
retention_limit: self.limit,
transport_truncation: self.transport_truncation.clone(),
storage_truncated: self.truncated || self.transport_truncation.is_some(),
terminal: self.terminal.clone().expect("finalized record"),
cleanup: self.cleanup.clone(),
text_encoding: "utf8_lossy",
ordering: "provider_read_observation",
}
}
}
impl Store {
fn expire(&mut self, now: Instant) {
let before = self.records.len();
self.records
.retain(|r| now.duration_since(r.lock().unwrap().created) < self.retention);
self.reserved -= (before - self.records.len()) * COMMAND_BYTES;
}
}
async fn reap(weak: Weak<Mutex<Store>>, mut shutdown: tokio::sync::watch::Receiver<()>) {
loop {
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(30)) => {},
_ = shutdown.changed() => return,
}
let Some(store) = weak.upgrade() else {
return;
};
store.lock().unwrap().expire(Instant::now());
}
}
impl BashOutputWriter {
pub(crate) fn id(&self) -> String {
self.record.lock().unwrap().id.clone()
}
pub(crate) fn finish_cleanup(&mut self, diagnostics: CleanupDiagnostics) {
self.record.lock().unwrap().cleanup = Some(diagnostics);
}
pub fn append_chunk(&mut self, chunk: &workdir::CommandByteChunk) -> Result<(), String> {
let expected = {
let r = self.record.lock().unwrap();
match chunk.stream {
CommandStream::Stdout => r.stdout_seen,
CommandStream::Stderr => r.stderr_seen,
}
};
if chunk.offset != expected || chunk.bytes.is_empty() {
return Err("command byte transport stream gap or empty chunk".into());
}
self.append(chunk.stream, &chunk.bytes);
Ok(())
}
pub fn append(&mut self, stream: CommandStream, bytes: &[u8]) {
let mut r = self.record.lock().unwrap();
let seen = match stream {
CommandStream::Stdout => &mut r.stdout_seen,
CommandStream::Stderr => &mut r.stderr_seen,
};
let start = *seen;
*seen = seen.saturating_add(bytes.len() as u64);
let available = COMMAND_BYTES.saturating_sub(r.stdout.len() + r.stderr.len());
let n = if r.truncated || r.order.len() >= ORDER_COUNT {
0
} else {
available.min(bytes.len())
};
if n < bytes.len() {
r.truncated = true;
if r.limit.is_none() {
r.limit = Some(if r.order.len() >= ORDER_COUNT {
RetentionLimit::OrderQuota
} else {
RetentionLimit::ByteQuota
});
}
}
if n > 0 {
match stream {
CommandStream::Stdout => r.stdout.extend_from_slice(&bytes[..n]),
CommandStream::Stderr => r.stderr.extend_from_slice(&bytes[..n]),
}
if let Some(last) = r
.order
.last_mut()
.filter(|last| last.stream == stream && last.end == start)
{
last.end += n as u64;
} else {
r.order.push(OutputRange {
stream,
start,
end: start + n as u64,
});
}
}
}
pub fn finish_transport(
&mut self,
terminal: &workdir::CommandTerminal,
) -> Result<BashOutputMetadata, String> {
let mut r = self.record.lock().unwrap();
if !terminal.metadata_valid()
|| terminal.stdout_delivered_bytes != r.stdout_seen
|| terminal.stderr_delivered_bytes != r.stderr_seen
{
return Err("command terminal counts do not fence delivered bytes".into());
}
r.transport_counts = Some((terminal.stdout_bytes, terminal.stderr_bytes));
r.transport_truncation = terminal.truncation_reason.clone();
r.terminal = Some(OutputTerminal {
status: Some(terminal.status),
exit_code: terminal.exit_code,
transport_complete: matches!(
terminal.output_collection,
workdir::CommandOutputCollection::Complete
),
process_failure: terminal.process_failure,
output_collection: Some(terminal.output_collection),
});
Ok(r.metadata())
}
pub fn finish(&mut self, terminal: OutputTerminal) -> BashOutputMetadata {
let mut r = self.record.lock().unwrap();
r.terminal = Some(terminal);
r.metadata()
}
}
impl Drop for BashOutputWriter {
fn drop(&mut self) {
let mut r = self.record.lock().unwrap();
if r.terminal.is_none() {
r.terminal = Some(OutputTerminal {
status: None,
exit_code: None,
transport_complete: false,
process_failure: None,
output_collection: None,
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn finish(view: &BashOutputView, text: &[u8]) -> String {
let mut writer = view.begin().unwrap();
writer.append(CommandStream::Stdout, text);
writer
.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
process_failure: None,
output_collection: Some(workdir::CommandOutputCollection::Complete),
})
.id
}
#[test]
fn revoked_child_is_denied_while_parent_keeps_evidence_and_sibling_is_denied() {
let root = BashOutputView::new();
let lease = Arc::new(AtomicBool::new(true));
let child = root.child({
let lease = lease.clone();
Arc::new(move || lease.load(Ordering::Acquire))
});
let sibling = root.child(Arc::new(|| true));
let id = finish(&child, b"child");
assert_eq!(
child.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
assert!(sibling.metadata(&id).is_none());
assert!(BashOutputView::new().metadata(&id).is_none());
lease.store(false, Ordering::Release);
assert!(child.metadata(&id).is_none());
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
root.revoke();
let restored = BashOutputView::new();
assert!(root.metadata(&id).is_none());
assert!(restored.metadata(&id).is_none());
let new_id = finish(&restored, b"new incarnation");
assert_ne!(new_id, id);
assert!(child.metadata(&new_id).is_none());
assert!(root.begin().is_err());
}
#[test]
fn parent_and_descendants_share_admission_without_evicting_live_evidence() {
let root = BashOutputView::new();
let child = root.child(Arc::new(|| true));
let sibling = root.child(Arc::new(|| true));
let mut writers = (0..STORE_BYTES / COMMAND_BYTES)
.map(|index| {
if index % 2 == 0 {
root.begin()
} else {
child.begin()
}
.unwrap()
})
.collect::<Vec<_>>();
assert!(sibling.begin().is_err());
let mut first = writers.remove(0);
first.append(CommandStream::Stdout, b"finished");
let metadata = first.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
process_failure: None,
output_collection: Some(workdir::CommandOutputCollection::Complete),
});
assert!(root.metadata(&metadata.id).is_some());
let _replacement = sibling.begin().unwrap();
assert!(root.metadata(&metadata.id).is_none());
assert!(root.begin().is_err());
assert_eq!(writers.len(), STORE_BYTES / COMMAND_BYTES - 1);
}
#[test]
fn quota_and_forced_drop_are_distinct_from_preview_omission() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, &vec![b'x'; COMMAND_BYTES + 1]);
writer.append(CommandStream::Stderr, b"tail");
let id = writer.record.lock().unwrap().id.clone();
drop(writer);
let m = root.metadata(&id).unwrap();
assert_eq!(m.stdout_saved, COMMAND_BYTES);
assert_eq!(m.stdout_seen, COMMAND_BYTES as u64 + 1);
assert_eq!(m.stderr_seen, 4);
assert!(m.storage_truncated);
assert!(!m.terminal.transport_complete);
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, usize::MAX)
.unwrap()
.len(),
64 * 1024
);
}
#[test]
fn expiry_and_count_eviction_never_reuse_old_references() {
let root = BashOutputView::new();
let id = finish(&root, b"old");
root.store.lock().unwrap().retention = Duration::ZERO;
assert!(root.metadata(&id).is_none());
root.store.lock().unwrap().retention = RETENTION;
let first = finish(&root, b"first");
for _ in 0..STORE_COUNT {
finish(&root, b"later");
}
assert!(root.metadata(&first).is_none());
assert!(root.store.lock().unwrap().reserved <= STORE_BYTES);
}
#[tokio::test(start_paused = true)]
async fn expiry_reclaims_bytes_without_another_read_and_reaper_ends_with_owner() {
// Exercise host construction outside a runtime, followed by real async admission.
let root = std::thread::spawn(BashOutputView::new).join().unwrap();
finish(&root, b"expires without lookup");
let mut closed = root.store.lock().unwrap()._shutdown.subscribe();
tokio::task::yield_now().await;
// The store uses monotonic wall time for publication and Tokio time only for scheduling.
// Age the record deterministically rather than delaying this test in real time.
root.store.lock().unwrap().records[0]
.lock()
.unwrap()
.created = Instant::now() - RETENTION;
tokio::time::advance(Duration::from_secs(31)).await;
tokio::task::yield_now().await;
assert_eq!(root.store.lock().unwrap().reserved, 0);
drop(root);
assert!(closed.changed().await.is_err());
}
#[test]
fn terminal_counts_must_fence_the_received_prefix_before_marking_evidence_complete() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, b"abc");
let mut terminal = serde_json::json!({"status":"completed","exit_code":0,"process_failure":null,"output_collection":{"state":"complete"},"stdout_bytes":9,"stderr_bytes":0,"stdout_delivered_bytes":4,"stderr_delivered_bytes":0,"truncation_reason":"byte_budget"});
assert!(
writer
.finish_transport(&serde_json::from_value(terminal.clone()).unwrap())
.is_err()
);
terminal["stdout_delivered_bytes"] = serde_json::json!(3);
let metadata = writer
.finish_transport(&serde_json::from_value(terminal).unwrap())
.unwrap();
assert_eq!(metadata.stdout_saved, 3);
assert_eq!(metadata.stdout_delivered, 3);
assert_eq!(metadata.stdout_seen, 9);
assert!(metadata.storage_truncated);
assert!(metadata.retention_limit.is_none());
assert_eq!(
metadata.transport_truncation,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
}
#[test]
fn incomplete_collection_with_quota_retains_both_causes_without_claiming_complete_output() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, b"abc");
let terminal: workdir::CommandTerminal = serde_json::from_value(serde_json::json!({
"status":"completed", "exit_code":0, "process_failure":null,
"output_collection":{"state":"incomplete","failure":{"stage":"stdout_drain","reason":"other","os_code":null}},
"stdout_bytes":9, "stderr_bytes":0, "stdout_delivered_bytes":3, "stderr_delivered_bytes":0,
"truncation_reason":"byte_budget"
})).unwrap();
let metadata = writer.finish_transport(&terminal).unwrap();
assert!(!metadata.terminal.transport_complete);
assert_eq!(metadata.terminal.exit_code, Some(0));
assert_eq!(
metadata.terminal.output_collection,
Some(terminal.output_collection)
);
assert_eq!(
metadata.transport_truncation,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
assert_eq!(metadata.stdout_seen, 9);
assert_eq!(metadata.stdout_saved, 3);
assert!(metadata.storage_truncated);
assert!(metadata.retention_limit.is_none());
assert_eq!(
root.read(&metadata.id, CommandStream::Stdout, 0, 100)
.unwrap(),
b"abc"
);
assert_eq!(
serde_json::to_value(root.metadata(&metadata.id).unwrap()).unwrap(),
serde_json::to_value(metadata).unwrap()
);
}
}
+68
View File
@@ -403,3 +403,71 @@ fn check_pattern(pattern: &str) -> Result<(), ToolError> {
#[cfg(test)]
#[path = "checkout_tests.rs"]
mod tests;
/// Read a bounded native text window through the same checked target.
/// The returned one-shot action records prior-Read only after the caller's presentation succeeds.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_text(
router: Arc<WorkdirSessionRouter>,
tracker: Tracker,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: usize,
limit: usize,
max_bytes: usize,
) -> Result<(workdir::ReadResult, Option<Vec<u8>>, impl FnOnce()), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let tracker = tracker.scoped_attachment(&selected.alias, selected.generation);
let target = FileTarget {
session: selected.session.clone(),
path: path.clone(),
validator: Some(validator),
};
let (result, validator) = target.read(offset, limit, max_bytes).await?;
let hash = result.content_hash.clone();
let total_lines = result.total_lines;
Ok((result, validator, move || {
tracker.record_workdir_observation(&path, hash, total_lines);
}))
}
/// Binary-safe checked byte window. This does not establish a text prior-Read observation.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_bytes(
router: Arc<WorkdirSessionRouter>,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: u64,
max_bytes: usize,
) -> Result<(workdir::ReadBytesResult, Vec<u8>), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let result = selected
.session
.checkout_execute(workdir::CheckoutRequest {
target: path.clone(),
validator,
operation: workdir::CheckoutOperation::ReadBytes { offset, max_bytes },
})
.await
.map_err(crate::file_target::checked_error)?;
match result.output {
workdir::CheckoutOutput::ReadBytes(read)
if read.path == path && read.offset == offset && read.bytes.len() <= max_bytes =>
{
Ok((read, result.observation.validator))
}
_ => Err(crate::file_target::readonly_result_error("ReadBytes")),
}
}
+1 -1
View File
@@ -21,7 +21,7 @@ pub(crate) struct EditParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
pub target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
/// Existing file in which to replace text.
pub file_path: String,
#[serde(flatten)]
pub edit: fs_operation::text::EditArgs,
+4 -1
View File
@@ -147,7 +147,9 @@ impl FileTarget {
pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
use workdir::WorkdirError;
let code = match error.classification_source() {
WorkdirError::DenialContext { .. } => unreachable!("diagnostic context unwrapped"),
WorkdirError::DenialContext { .. } | WorkdirError::FailureContext { .. } => {
unreachable!("diagnostic context unwrapped")
}
WorkdirError::Conflict(_) | WorkdirError::NotFound(_) => "checkout_stale",
WorkdirError::Denied { .. }
| WorkdirError::OutOfScope { .. }
@@ -165,6 +167,7 @@ pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
| WorkdirError::IsDirectory { .. }
| WorkdirError::InvalidGlob { .. }
| WorkdirError::InvalidRegex { .. } => "checkout_invalid",
WorkdirError::NotDispatched => "checkout_not_dispatched",
WorkdirError::Unavailable { .. } | WorkdirError::SessionClosed => "checkout_unavailable",
// IO/transport failures may arrive after effects have begun. Do not infer
// no-effects from their text or strip explicit OutcomeUnknown evidence.
+2 -2
View File
@@ -17,7 +17,7 @@ pub(crate) struct GlobParams {
pub target_workdir: Option<String>,
/// Glob pattern, for example `**/*.rs` or `src/**/test_*.py`.
pub pattern: String,
/// Logical Workdir-relative directory. Defaults to the Workdir root.
/// Directory to search. Defaults to `.`.
#[serde(default)]
pub path: Option<String>,
}
@@ -106,7 +106,7 @@ pub(crate) fn routed_glob_tool(router: Arc<WorkdirSessionRouter>) -> ToolDefinit
Arc::new(move || {
let schema = schemars::schema_for!(GlobParams);
let meta = ToolMeta::new("Glob")
.description("Find files matching a glob pattern inside the selected Workdir. Results are sorted and capped at 1000 entries. Paths are Workdir-relative.")
.description("Find files matching a glob pattern inside the selected Workdir. Results are sorted and capped at 1000 entries.")
.input_schema(serde_json::to_value(schema).expect("Glob schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(GlobTool {
router: router.clone(),
+2 -2
View File
@@ -27,7 +27,7 @@ pub(crate) struct GrepParams {
#[serde(default)]
target_workdir: Option<String>,
pattern: String,
/// Workdir-relative path, or an absolute path covered by readable scope. Defaults to the Workdir root.
/// File or directory to search. Defaults to `.`.
#[serde(default)]
pub path: Option<String>,
#[serde(default)]
@@ -160,7 +160,7 @@ pub(crate) fn routed_grep_tool(router: Arc<WorkdirSessionRouter>) -> ToolDefinit
Arc::new(move || {
let schema = schemars::schema_for!(GrepParams);
let meta = ToolMeta::new("Grep")
.description("Search a file or directory in the selected Workdir with a regex. Content results group lines by file; `>` marks matching lines and unmarked lines are context. Directory traversal executes inside the WorkdirSession provider. Results are bounded and Workdir-relative.")
.description("Search a file or directory in the selected Workdir with a regex. Content results group lines by file; `>` marks matching lines and unmarked lines are context. Directory traversal executes inside the WorkdirSession provider. Results are bounded.")
.input_schema(serde_json::to_value(schema).expect("Grep schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(GrepTool {
router: router.clone(),
+10 -9
View File
@@ -5,6 +5,7 @@ pub mod error;
pub mod tracker;
mod bash;
pub mod bash_output;
mod edit;
mod file_target;
mod glob;
@@ -16,7 +17,7 @@ pub mod view_image;
mod web;
mod write;
pub use bash::bash_tool;
pub use bash::{BashResult, BashTool, bash_tool};
pub use checkout::{CheckoutToolOutput, execute_checkout_tool};
pub use edit::edit_tool;
pub use error::ToolsError;
@@ -35,7 +36,7 @@ pub use write::write_tool;
pub fn core_builtin_tools(
session: workdir::WorkdirSessionHandle,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -58,7 +59,7 @@ pub fn core_builtin_tools(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -71,7 +72,7 @@ pub fn core_builtin_tools(
pub fn routed_builtin_tools(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
vec![
read::routed_read_tool(router.clone(), tracker.clone()),
@@ -79,14 +80,14 @@ pub fn routed_builtin_tools(
edit::routed_edit_tool(router.clone(), tracker),
glob::routed_glob_tool(router.clone()),
grep::routed_grep_tool(router.clone()),
bash::routed_bash_tool(router, bash_output_dir),
bash::routed_bash_tool(router, bash_output),
]
}
pub fn routed_builtin_tools_for_current_capabilities(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -108,7 +109,7 @@ pub fn routed_builtin_tools_for_current_capabilities(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -121,7 +122,7 @@ pub fn read_only_builtin_tools(
workdir::WorkdirSessionCapabilities::READ_ONLY,
"read-only tool projection requires a read-only Workdir session"
);
core_builtin_tools(session, Tracker::new(), std::path::PathBuf::new())
core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
}
pub fn read_only_routed_builtin_tools(
@@ -173,7 +174,7 @@ mod workdir_tool_tests {
SharedScope::new(Scope::writable(dir.path()).unwrap()),
WorkdirSessionCapabilities::READ_ONLY,
));
let names = core_builtin_tools(session, Tracker::new(), dir.path().join("output"))
let names = core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
.into_iter()
.map(|definition| definition().0.name)
.collect::<Vec<_>>();
+2 -2
View File
@@ -13,7 +13,7 @@ use workdir::{WorkdirPath, WorkdirSessionHandle, WorkdirSessionRouter};
const DESCRIPTION: &str = "Read a text file from a Workdir attachment selected by its Worker-local alias. \
Supports offset/limit for large files. Returns line-numbered output (1-based). \
Directories cannot be read. The file must be read before Write or Edit can \
modify it. Paths are Workdir-relative unless an absolute path is explicitly readable. \
modify it. \
When exactly one Workdir is attached, target_workdir may be omitted.";
const DEFAULT_LIMIT: usize = 2000;
@@ -24,7 +24,7 @@ pub(crate) struct ReadParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
pub target_workdir: Option<String>,
/// Workdir-relative path, or an absolute path covered by readable scope.
/// Text file to read; directories are not supported.
pub file_path: String,
#[serde(flatten)]
pub read: fs_operation::text::LineReadArgs,
+2 -2
View File
@@ -20,7 +20,7 @@ pub const MAX_IMAGE_BYTES: usize = 10 * 1024 * 1024;
const IMAGE_READ_CHUNK_BYTES: usize = 1024 * 1024;
const DESCRIPTION: &str = "Attach an image from the selected Workdir attachment to the next model request. \
The path must be logical and Workdir-relative. Supported formats: PNG, JPEG, GIF, and WebP. \
Supported formats: PNG, JPEG, GIF, and WebP. \
When exactly one Workdir is attached, target_workdir may be omitted.";
#[derive(Debug, Deserialize, schemars::JsonSchema)]
@@ -28,7 +28,7 @@ struct ViewImageParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
/// Image file to attach to the next model request.
path: String,
}
+1 -1
View File
@@ -20,7 +20,7 @@ pub(crate) struct WriteParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
pub target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
/// File to create or overwrite.
pub file_path: String,
#[serde(flatten)]
pub write: fs_operation::text::WriteArgs,
+19 -27
View File
@@ -3,7 +3,7 @@
use std::sync::Arc;
use agen::tool::{Tool, ToolDefinition};
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use manifest::Scope;
use serde_json::json;
use tempfile::TempDir;
use tools::{Tracker, core_builtin_tools};
@@ -28,31 +28,23 @@ impl Registry {
}
}
fn setup() -> (TempDir, TempDir, Registry) {
fn setup() -> (TempDir, Registry) {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let base = Scope::writable(dir.path()).unwrap();
let mut config = ScopeConfig {
allow: base.allow_rules(),
deny: base.deny_rules(),
};
config.allow.push(ScopeRule {
target: spill.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
});
let scope = Scope::from_config(&config).unwrap();
let scope = Scope::writable(dir.path()).unwrap();
let fs: WorkdirSessionHandle =
std::sync::Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let tracker = Tracker::new();
let reg = Registry::new(core_builtin_tools(fs, tracker, spill.path().to_path_buf()));
(dir, spill, reg)
let reg = Registry::new(core_builtin_tools(
fs,
tracker,
tools::bash_output::BashOutputView::new(),
));
(dir, reg)
}
#[tokio::test]
async fn unicode_path_and_content() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("日本語ファイル.txt");
let content = "こんにちは 🦀 世界\nabc\n";
@@ -87,7 +79,7 @@ async fn unicode_path_and_content() {
async fn symlink_to_outside_scope_is_rejected_for_write() {
use std::os::unix::fs::symlink;
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let outside = TempDir::new().unwrap();
let outside_target = outside.path().join("secret.txt");
std::fs::write(&outside_target, "secret").unwrap();
@@ -151,7 +143,7 @@ async fn symlink_to_outside_scope_is_rejected_for_write() {
async fn broken_symlink_reports_target_and_repair_hint() {
use std::os::unix::fs::symlink;
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let link = dir.path().join("external-project");
let target = dir.path().join("missing-target");
symlink(&target, &link).unwrap();
@@ -173,7 +165,7 @@ async fn broken_symlink_reports_target_and_repair_hint() {
#[tokio::test]
async fn empty_file_read_and_edit() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("empty.txt");
std::fs::write(&file, "").unwrap();
@@ -207,7 +199,7 @@ async fn empty_file_read_and_edit() {
#[tokio::test]
async fn very_long_single_line() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("long.txt");
let big: String = "x".repeat(1024 * 1024); // 1 MiB, no newlines
std::fs::write(&file, &big).unwrap();
@@ -226,7 +218,7 @@ async fn very_long_single_line() {
#[tokio::test]
async fn absolute_path_requires_matching_read_scope() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let outside = tempfile::tempdir().unwrap();
let outside_file = outside.path().join("outside.txt");
std::fs::write(&outside_file, "secret").unwrap();
@@ -247,7 +239,7 @@ async fn absolute_path_requires_matching_read_scope() {
#[tokio::test]
async fn directory_target_is_rejected_for_read() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let read = reg.get("Read");
let err = read
.execute(&json!({ "file_path": "." }).to_string(), Default::default())
@@ -258,7 +250,7 @@ async fn directory_target_is_rejected_for_read() {
#[tokio::test]
async fn deeply_nested_new_file_is_created() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let deep = dir.path().join("a/b/c/d/e/deep.txt");
let write = reg.get("Write");
write
@@ -277,7 +269,7 @@ async fn deeply_nested_new_file_is_created() {
#[tokio::test]
async fn replace_preserves_unicode() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("u.txt");
std::fs::write(&file, "🦀 rust 🦀\n").unwrap();
@@ -306,7 +298,7 @@ async fn replace_preserves_unicode() {
#[tokio::test]
async fn grep_handles_unicode_pattern() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("u.txt");
std::fs::write(&file, "English\n日本語\nрусский\n").unwrap();
File diff suppressed because it is too large Load Diff
+36 -20
View File
@@ -3013,7 +3013,8 @@ impl App {
/// `[File: …]` parsing path: each kind maps directly to a typed
/// block (`Block::Notify`, `Block::WorkerEvent`, …).
fn apply_system_item(&mut self, value: &serde_json::Value) {
let Ok(item) = serde_json::from_value::<session_store::SystemItem>(value.clone()) else {
let Ok(item) = serde_json::from_value::<session_store::PublicSystemItem>(value.clone())
else {
// Unknown / forward-compat shape: fall back to rendering the
// raw text payload (if any) as a generic system message.
if let Some(text) = value.get("body").and_then(|b| b.as_str()) {
@@ -3024,29 +3025,23 @@ impl App {
}
return;
};
match item {
session_store::SystemItem::Notification { message, .. } => {
let body = item.body;
match item.presentation {
session_store::PublicSystemPresentation::Notification { message } => {
self.blocks.push(Block::Notify { message });
}
session_store::SystemItem::WorkerEvent { event, .. } => {
session_store::PublicSystemPresentation::WorkerEvent { event } => {
self.blocks.push(Block::WorkerEvent { event });
}
session_store::SystemItem::FeatureInvocationResult { body, .. }
| session_store::SystemItem::FileAttachment { body, .. }
| session_store::SystemItem::SkillActivation { body, .. }
| session_store::SystemItem::JournalStartup { body, .. }
| session_store::SystemItem::ResidentSummaryRefresh { body, .. }
| session_store::SystemItem::SubjectBehaviorRefresh { body, .. }
| session_store::SystemItem::Interrupt { body, .. } => {
session_store::PublicSystemPresentation::Context => {
self.task_store.apply_system_message_text(&body);
self.blocks.push(Block::SystemMessage { text: body });
}
session_store::SystemItem::TaskReminder { body, .. } => {
session_store::PublicSystemPresentation::TaskReminder => {
self.task_store.apply_system_message_text(&body);
self.blocks.push(Block::TaskReminder { text: body });
}
session_store::SystemItem::LegacyIgnored { .. } => {}
session_store::SystemItem::LegacyKnowledgeIgnored { .. } => {}
session_store::PublicSystemPresentation::Ignored => {}
}
}
@@ -3572,6 +3567,7 @@ mod completion_flow_tests {
session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4329,6 +4325,7 @@ mod completion_flow_tests {
}));
app.handle_worker_event(Event::SessionEntryCommitted {
entry: protocol::SessionSnapshotEntry {
use_policy: None,
entry_id: "run-failure".into(),
timestamp: 2,
provenance: protocol::SessionEntryProvenance::LegacyUnknown,
@@ -4450,12 +4447,12 @@ mod completion_flow_tests {
}
#[test]
fn live_legacy_workflow_system_item_is_ignored() {
fn retired_system_input_is_ignored() {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "workflow",
"slug": "build",
"body": "[Workflow /build]\nRun the build",
"kind": "retired_input",
"body": "",
"presentation": { "kind": "ignored" },
});
app.handle_worker_event(Event::SystemItem {
entry_id: None,
@@ -4861,12 +4858,30 @@ mod completion_flow_tests {
);
}
#[test]
fn feature_label_does_not_select_system_input_presentation() {
for label in ["journal_startup", "new-unregistered-feature"] {
let mut app = App::new("test".into());
app.handle_worker_event(Event::SystemItem {
entry_id: None,
item: serde_json::json!({
"kind": label,
"body": "explicit public content",
"presentation": { "kind": "context" },
}),
});
assert!(
matches!(app.blocks.as_slice(), [Block::SystemMessage { text }] if text == "explicit public content")
);
}
}
#[test]
fn live_system_item_notification_appends_notify_block() {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "notification",
"message": "hi",
"presentation": { "kind": "notification", "message": "hi" },
"body": "[Notification] hi",
});
app.handle_worker_event(Event::SystemItem {
@@ -4884,7 +4899,7 @@ mod completion_flow_tests {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "worker_event",
"event": { "kind": "turn_ended", "worker_name": "child" },
"presentation": { "kind": "worker_event", "event": { "kind": "turn_ended", "worker_name": "child" } },
"body": "[Notification] worker `child` finished a turn",
});
app.handle_worker_event(Event::SystemItem {
@@ -5285,6 +5300,7 @@ mod completion_flow_tests {
item: serde_json::json!({
"kind": "task_reminder",
"body": snapshot,
"presentation": { "kind": "task_reminder" },
}),
});
+1
View File
@@ -162,6 +162,7 @@ fn request_from_selection(selection: Selection) -> BackendCreateWorkerRequest {
ticket_assignment: None,
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
}
}
+2 -1
View File
@@ -20,8 +20,9 @@ serde_json.workspace = true
sha2.workspace = true
tempfile.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tokio = { workspace = true, features = ["io-util", "process", "rt", "sync", "time"] }
[dev-dependencies]
axum.workspace = true
serde_json.workspace = true
tempfile.workspace = true
+10 -6
View File
@@ -6,7 +6,7 @@
//! hashes alone do not accept same-content inode replacements. Callers own Read line numbering
//! and per-attachment read history.
//!
//! Create paths are checkout-root-relative and must be strict descendants of the observed target
//! Create paths use the target's session coordinates and must be strict descendants of the observed target
//! directory. Existing destinations are never overwritten; results return the updated observation
//! of the retained target directory, not the created file. Read results also return the validator
//! of the state actually read, never a replacement from a later independent stat.
@@ -18,7 +18,8 @@
//!
//! Search delegates to shared List/Glob/Grep engines. Scope rules form unions within layers and
//! intersections across layers, provider authority, and output roots. Wrappers transform coordinates
//! once; returned paths and Grep content are relative to the final session's logical root.
//! once; returned paths and Grep content are relative to the composed output root (the caller's
//! cwd by default).
//! Paths are not reverse-parsed from rendered filenames; non-UTF-8 names are not lossily aliased.
//! Unsupported providers reject search rather than fall back to broader trait searches. Search is
//! live traversal, not an atomic snapshot of all descendants or exclusion of external editors.
@@ -48,6 +49,10 @@ pub struct CheckoutRequest {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(tag = "operation", rename_all = "snake_case", deny_unknown_fields)]
pub enum CheckoutOperation {
ReadBytes {
offset: u64,
max_bytes: usize,
},
Read {
offset: usize,
limit: usize,
@@ -71,7 +76,7 @@ pub enum CheckoutOperation {
impl CheckoutOperation {
pub fn capability(&self) -> WorkdirSessionCapability {
match self {
Self::Read { .. } => WorkdirSessionCapability::Read,
Self::Read { .. } | Self::ReadBytes { .. } => WorkdirSessionCapability::Read,
Self::Edit { .. } => WorkdirSessionCapability::Edit,
Self::Write { .. } | Self::Create { .. } => WorkdirSessionCapability::Write,
}
@@ -86,6 +91,7 @@ impl CheckoutOperation {
)]
pub enum CheckoutOutput {
Read(ReadResult),
ReadBytes(crate::ReadBytesResult),
Write(WriteResult),
Edit(EditResult),
}
@@ -551,15 +557,13 @@ mod tests {
));
}
#[tokio::test]
async fn checkout_hardlink_save_isolates_alias_and_rejects_symlinks_special_files() {
async fn checkout_hardlink_save_isolates_alias_and_rejects_symlinks() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join("a"), "original").unwrap();
std::fs::hard_link(root.path().join("a"), root.path().join("alias")).unwrap();
std::os::unix::fs::symlink("a", root.path().join("link")).unwrap();
let _socket = std::os::unix::net::UnixListener::bind(root.path().join("socket")).unwrap();
let session = session(root.path());
assert!(session.checkout_observe(path("link")).await.is_err());
assert!(session.checkout_observe(path("socket")).await.is_err());
let read = checked_read(&session, session.checkout_observe(path("a")).await.unwrap()).await;
session
.checkout_execute(CheckoutRequest {
@@ -0,0 +1,43 @@
//! Bounded collection for tests whose subject is command routing/lifecycle,
//! rather than the byte-page protocol itself.
use crate::*;
#[derive(Debug)]
pub(crate) struct CollectedCommand {
pub content: String,
pub status: CommandStatus,
pub exit_code: Option<i32>,
pub timed_out: bool,
}
#[async_trait::async_trait]
pub(crate) trait CollectCommand: WorkdirSession {
async fn collect_command_output(
&self,
mut request: CommandOutputRequest,
) -> Result<CollectedCommand, WorkdirError> {
let mut bytes = Vec::new();
request.wait = true;
loop {
let page = self.command_output(request.clone()).await?;
for chunk in page.chunks {
bytes.extend(chunk.bytes);
assert!(
bytes.len() <= 16 * 1024 * 1024,
"fixture output exceeded bounded test collection"
);
}
request.cursor = page.next_cursor;
if let Some(terminal) = page.terminal {
return Ok(CollectedCommand {
content: String::from_utf8_lossy(&bytes).into_owned(),
status: terminal.status,
exit_code: terminal.exit_code,
timed_out: terminal.status == CommandStatus::TimedOut,
});
}
}
}
}
impl<T: WorkdirSession + ?Sized> CollectCommand for T {}
+68
View File
@@ -0,0 +1,68 @@
//! Model-facing session coordinates, separate from provider paths and authority.
use crate::{WorkdirPath, WorkdirSessionCapabilities, WorkdirToolScopeRule};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
/// A snapshot of the environment actually used by a session's operations.
///
/// Cwd is a logical Workdir-root coordinate. Scope targets use that root or
/// explicit absolute provider paths. Scope is an additional allow ceiling, not
/// a replacement for provider checks. Providers
/// may retain further restrictions (including deny rules); this projection
/// never authorizes an operation.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct WorkdirSessionEnvironment {
#[serde(deserialize_with = "deserialize_cwd")]
pub cwd: WorkdirPath,
pub scope: Option<Vec<WorkdirToolScopeRule>>,
pub capabilities: WorkdirSessionCapabilities,
pub restrictions: Vec<String>,
}
fn deserialize_cwd<'de, D: serde::Deserializer<'de>>(
deserializer: D,
) -> Result<WorkdirPath, D::Error> {
let path = String::deserialize(deserializer)?;
WorkdirPath::new(path).map_err(serde::de::Error::custom)
}
impl WorkdirSessionEnvironment {
pub fn root(capabilities: WorkdirSessionCapabilities) -> Self {
Self {
cwd: WorkdirPath::root(),
scope: None,
capabilities,
restrictions: Vec::new(),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn wire_cwd_is_normalized_and_cannot_claim_an_absolute_or_parent_base() {
let mut wire = serde_json::to_value(WorkdirSessionEnvironment::root(
WorkdirSessionCapabilities::READ_ONLY,
))
.unwrap();
for invalid in [
"/absolute",
"../parent",
"nested/../parent",
"nested\\other",
] {
wire["cwd"] = invalid.into();
assert!(
serde_json::from_value::<WorkdirSessionEnvironment>(wire.clone()).is_err(),
"{invalid}"
);
}
wire["cwd"] = "./nested//child".into();
let decoded: WorkdirSessionEnvironment = serde_json::from_value(wire).unwrap();
assert_eq!(decoded.cwd.as_str(), "nested/child");
}
}
+175
View File
@@ -0,0 +1,175 @@
//! Real Local/scoped/read-only/router sessions across loopback HTTP. Only the
//! HTTP authentication boundary is a fixture; coordinate and scope checks are
//! production implementations. Runs in the normal http-client library suite.
#![cfg(all(feature = "http-client", target_os = "linux"))]
use crate::http::*;
use crate::*;
use axum::{
Json, Router,
extract::State,
routing::{delete, post},
};
use std::sync::Arc;
struct Server(tokio::task::JoinHandle<()>);
impl Drop for Server {
fn drop(&mut self) {
self.0.abort();
}
}
async fn open(State(session): State<WorkdirSessionHandle>) -> Json<OpenWorkdirSessionResponse> {
Json(OpenWorkdirSessionResponse {
session_id: WorkdirSessionId::new("environment-session").unwrap(),
workdir_id: session.workdir().id().clone(),
capabilities: session.capabilities(),
environment: session.environment(),
})
}
async fn operation(
State(session): State<WorkdirSessionHandle>,
Json(request): Json<WorkdirSessionOperationRequest>,
) -> Result<
Json<WorkdirSessionOperationResult>,
(axum::http::StatusCode, Json<WorkdirTransportError>),
> {
dispatch_workdir_session_operation(session.as_ref(), request.operation)
.await
.map(Json)
.map_err(|error| {
(
axum::http::StatusCode::FORBIDDEN,
Json(WorkdirTransportError::from_workdir_error(&error)),
)
})
}
#[tokio::test]
async fn remote_environment_matches_scoped_readonly_operations_without_rebasing_twice() {
let root = tempfile::tempdir().unwrap();
std::fs::create_dir(root.path().join("nested")).unwrap();
std::fs::write(root.path().join("nested/file.txt"), "nested marker").unwrap();
std::fs::write(root.path().join("file.txt"), "root decoy").unwrap();
let broker = WorkdirToolBroker::new(Arc::new(LocalWorkdirSession::new(
manifest::Scope::writable(root.path()).unwrap(),
root.path().to_path_buf(),
)));
let lease = broker
.scope(WorkdirToolScope {
cwd: WorkdirPath::new("nested").unwrap(),
rules: vec![WorkdirToolScopeRule {
target: WorkdirPath::new("nested").unwrap(),
permission: WorkdirToolScopePermission::Write,
recursive: true,
symlink_policy: Default::default(),
}],
command: true,
})
.await
.unwrap();
let session: WorkdirSessionHandle = Arc::new(ReadOnlyWorkdirSession::new(lease.tool_session()));
let expected = session.environment();
let app = Router::new()
.route("/v1/working-directories/{id}/sessions", post(open))
.route("/v1/workdir-sessions/{id}/operations", post(operation))
.route(
"/v1/workdir-sessions/{id}",
delete(|| async { axum::http::StatusCode::NO_CONTENT }),
)
.with_state(session.clone());
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let _server = Server(tokio::spawn(async move {
axum::serve(listener, app).await.unwrap();
}));
let remote: WorkdirSessionHandle = Arc::new(
RemoteWorkdirSession::open(
reqwest::Client::builder().no_proxy().build().unwrap(),
format!("http://{address}").parse().unwrap(),
"fixture",
session.workdir().id().clone(),
OpenWorkdirSessionRequest::default(),
)
.await
.unwrap(),
);
let router = WorkdirSessionRouter::new();
router
.attach(WorkdirAttachmentAlias::new("remote").unwrap(), remote)
.unwrap();
let selected = router.resolve(Some("remote")).unwrap().session;
assert_eq!(selected.environment(), expected);
assert_eq!(expected.cwd, WorkdirPath::new("nested").unwrap());
assert!(
!expected
.capabilities
.supports(WorkdirSessionCapability::Command)
);
assert!(
expected
.scope
.unwrap()
.iter()
.all(|rule| rule.permission == WorkdirToolScopePermission::Read)
);
let found = selected
.glob(GlobRequest {
path: WorkdirPath::root(),
pattern: "*.txt".into(),
limit: 10,
})
.await
.unwrap();
assert_eq!(found.paths, vec![WorkdirPath::new("file.txt").unwrap()]);
let path = found.paths[0].clone();
let read = selected
.read(ReadRequest {
path: path.clone(),
offset: 0,
limit: 10,
max_bytes: 100,
})
.await
.unwrap();
assert_eq!(read.path, path);
assert_eq!(read.bytes, b"nested marker");
assert!(
selected
.write(WriteRequest {
path,
content: b"denied".to_vec(),
expected_hash: Some(read.content_hash)
})
.await
.is_err()
);
assert!(
selected
.read(ReadRequest {
path: WorkdirPath::new_scoped(root.path().join("file.txt").to_str().unwrap())
.unwrap(),
offset: 0,
limit: 10,
max_bytes: 100
})
.await
.is_err()
);
assert!(
selected
.start_command(CommandRequest {
command: "touch escaped".into(),
cwd: WorkdirPath::root(),
timeout_secs: 5,
transport_byte_budget: 100,
tool_call_id: None
})
.await
.is_err()
);
assert!(!root.path().join("nested/escaped").exists());
router.close_all().await.unwrap();
lease.close().await.unwrap();
broker.close().await.unwrap();
}

Some files were not shown because too many files have changed in this diff Show More