91 Commits
Author SHA1 Message Date
Hare 5b12c3e085 fix: make mobile sidebar swipes track touch gestures 2026-10-27 06:49:28 +09:00
Hare 623c31615d fix: assign unique event refs to merge integration comments 2026-10-27 06:15:48 +09:00
Hare af719e387d fix: align reviewer guidance with Bash-enabled scope 2026-10-27 06:15:48 +09:00
Hare d12bc2ea0b fix: deduplicate live and committed task updates 2026-10-26 10:13:36 +09:00
Hare a1df3742b1 fix: integrate passkey results with current dogfooding shell
Preserve authenticated home/account navigation and guarded mobile sidebar swipes. Keep both browser task selections. Complete the new account fixture contract and synchronize lifecycle probes on visible registered-key results, without changing their success or security assertions.
2026-10-26 09:50:13 +09:00
Hare d8818b776b feat: open and close mobile sidebar with guarded swipes 2026-10-26 09:39:35 +09:00
Hare 40c36ccb1d fix: restore authenticated home and account navigation 2026-10-26 09:39:35 +09:00
Hare 80c151d084 fix: fence recovered passkey results against cross-tab changes 2026-10-26 07:52:06 +09:00
Hare c6a8759e7b fix: fence activation and setup against delayed cross-tab events
Capture the shared authentication stamp as well as generation before dispatch and across response/confirmation/application awaits. Do not let stale setup failure reconciliation overwrite a newer tab's stamp. Add no-event-yet logout and account-change regressions for activation and setup, including prompt, pre-complete and post-dispatch confirmation boundaries.

Addresses MR AC4_CROSS_TAB_ACTIVATION. Validation: activation21/setup40 narrow tests; Web720 unit+625 component, check0/0/build, activation and hosting browser suites; root check/fmt/diff; both explicit real Backend browser suites including native CTAP2. Existing Server exclusions and 15 identical baseline naming findings remain unchanged. No verifier, binding, migration or rendering contract weakened.
2026-10-26 07:34:04 +09:00
Hare f17b5212c7 fix(auth): confirm passkey operations and preserve safe legacy migration
Fence displayed identity and native prompt generations, classify secret-free outcomes, recover atomic addition receipts without restoring old credentials, and expose truthful own-key verification metadata. Preserve required verifier and enrollment authority boundaries. Include real-store, production browser, native CTAP2 and visible responsive regressions.

Validation: root check/fmt; API 115; passkey security 25; Web unit 712/component 605; auth/entry/activation/hosting browsers; product auth E2E 8 and two explicit browser suites. Server suite: 1262 lib+12 CLI, 3 ignored and 2 environment-specific exclusions. State-name checker retains 15 byte-identical baseline findings; no new exceptions. Physical keys/mobile and hosted CI remain unverified.
2026-10-26 07:18:23 +09:00
Hare b284934e34 test: add captured Codex worker restore fixture 2026-10-24 12:57:28 +09:00
Hare 2d33321fab fix: validate live fixture snapshots and shutdown boundaries 2026-10-24 12:57:28 +09:00
Hare 0975471986 fix: preserve diagnostic context in backend test assertions 2026-10-24 12:51:28 +09:00
Hare 214d6e21dd fix: assert classified workdir session closure in tests 2026-10-24 12:40:37 +09:00
Hare fca5c23345 fix: reject unrestorable and opaque fixture records 2026-10-24 12:39:22 +09:00
Hare f9b1500ba2 fix: align panel tests with current ticket case states 2026-10-24 12:33:31 +09:00
Hare 931bf3b27d test: add isolated saved worker restore scenarios 2026-10-24 12:33:31 +09:00
Hare 6bb671be21 test: add typed stopped worker fixture transport 2026-10-24 12:19:43 +09:00
Hare b8f2fa479c test: add isolated backend fixture seeding 2026-10-24 12:15:10 +09:00
Hare 1639be9841 fix: allow reviewers to read related tickets 2026-10-23 03:22:45 +09:00
Hare aa0fb2f787 fix: continue workdir output reads after long-poll expiry 2026-10-21 04:48:43 +09:00
Hare 1b6d4dbfd8 fix: preserve workdir failure diagnostics across transport 2026-10-21 01:22:04 +09:00
Hare 4ef7b03651 fix: stop background session checks and loading flicker 2026-10-20 22:35:38 +09:00
Hare af615ab177 fix: retain archive and worker restore failure diagnostics 2026-10-20 22:00:58 +09:00
Hare 338e39bcdb fix: defer restored workdir access until use 2026-10-20 22:00:58 +09:00
Hare 3cfe04be3a fix: preserve saved feature configuration during worker restore 2026-10-20 22:00:39 +09:00
Hare 70d153b365 fix: isolate unreadable archives from runtime startup 2026-10-20 17:39:38 +09:00
Hare ce93849948 chore: merge develop into hare/develop 2026-10-20 15:39:09 +09:00
Hare 3c499719fe feat: default Coder ticket work to review flow 2026-10-20 15:38:50 +09:00
Hare 9e1d3df7ad fix: remove overbroad Worker bypass instructions 2026-10-20 15:38:50 +09:00
Hare 18bac16a23 fix: require replay-safe control lifecycle for Subject delegation 2026-10-20 08:12:26 +09:00
Hare 9005856f77 fix: replay accepted delegation without reserving active Workdirs 2026-10-20 07:24:51 +09:00
Hare e39372a52a Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants 2026-10-20 06:38:12 +09:00
Hare 223a74a9e2 fix: retain byte command transport and explicit authorization fixtures 2026-10-20 06:38:12 +09:00
Hare f38d9a16ea Merge remote-tracking branch 'origin/develop' into ticket/t-768-subject-grants
# Conflicts:
#	crates/worker-runtime/src/worker_backend.rs
#	openapi/server-api.json
#	web/workspace/src/lib/generated/companion-api.ts
#	web/workspace/src/lib/generated/repository-api.ts
#	web/workspace/src/lib/generated/runtime-api.ts
#	web/workspace/src/lib/generated/ticket-api.ts
#	web/workspace/src/lib/generated/worker-launch-api.ts
2026-10-20 06:24:11 +09:00
Hare b135fa5e99 feat: enforce explicit Subject permissions and delegated Worker grants 2026-10-20 06:22:58 +09:00
Hare 07b8b6d41b Merge commit 'b8e548b92d3c785c482dbb418517cc5a2cf9e7e0' into refactor/T-765-session-feature-boundary 2026-10-20 05:34:22 +09:00
Hare 8017c55d82 fix: retain annotated compaction history and honor non-System input policy 2026-10-20 05:34:22 +09:00
Hare b8e548b92d chore: integrate reviewed browser session navigation 2026-10-20 05:31:32 +09:00
Hare 90e38551cf Merge commit '00ad02d04728a8e32a82a4e72747fefcf8a82d4e' into refactor/T-765-session-feature-boundary
# Conflicts:
#	crates/worker/src/feature/builtin.rs
2026-10-20 05:25:16 +09:00
Hare 3e8164e757 fix: fence hosting setup verification against superseded sessions 2026-10-20 04:53:43 +09:00
Hare 3b3381900f fix: preserve role-neutral input policy across compaction and replay 2026-10-20 04:45:50 +09:00
Hare 00ad02d047 chore: integrate bounded shared text page refusal 2026-10-20 03:55:13 +09:00
Hare a774925dec fix: reject byte-truncated line windows before advancing search 2026-10-20 03:53:03 +09:00
Hare 8f759911ab fix: emit Session use policy type in browser bindings 2026-10-20 03:52:55 +09:00
Hare daf1f6ed50 fix: route retained Session migration through Runtime staging 2026-10-20 03:52:55 +09:00
Hare 86602240fa refactor: bind Subject metadata and prompt confirmation to Host lifecycle 2026-10-20 03:52:55 +09:00
Hare bfeee487fd feat(auth): separate anonymous entry from expired session recovery 2026-10-20 03:52:48 +09:00
Hare f9830fe7dd test: reject Bash search pages that would skip an unread long line 2026-10-20 03:40:36 +09:00
Hare de96c5b577 chore: update shared file contracts and validation fixtures
# Conflicts:
#	crates/worker/src/checkout_http_tests.rs
2026-10-20 03:26:56 +09:00
Hare 6e729ed06c refactor: isolate Feature System input migration and compaction policy 2026-10-20 03:21:22 +09:00
Hare f38b2a52b1 fix: record native prior Read only after result validation 2026-10-20 03:21:14 +09:00
Hare dd5384df3b feat: expose retained Bash streams through common WIP file contracts 2026-10-20 03:13:46 +09:00
Hare 6576850b7a chore: connect shared file contracts for Bash validation 2026-10-20 03:03:53 +09:00
Hare 4555aace28 feat: capture bounded Bash output over replayable byte transport 2026-10-20 03:01:22 +09:00
Hare 47b32ef6ab fix: bind capability changes to Object freshness and update contract consumers 2026-10-20 02:53:21 +09:00
Hare bb5c50f9ab feat: persist System input policies through public consumers 2026-10-20 02:44:28 +09:00
Hare 32a409c15d feat: share virtual file contracts and checked binary byte ranges 2026-10-20 02:18:49 +09:00
Hare 19514a5d6a feat: define bounded Feature-owned system input envelopes 2026-10-20 02:14:52 +09:00
Hare 887184796e docs: distinguish commit completion from Feature authority 2026-10-20 02:05:02 +09:00
Hare cd0eb5eb00 fix: allowlist public system input projections 2026-10-20 02:02:28 +09:00
Hare e8dfb3e68d refactor: notify system input producers after durable commit 2026-10-20 02:02:28 +09:00
Hare 434ce34da7 refactor: bind checkout target lifetimes to Object observations 2026-10-20 02:01:22 +09:00
Hare bfb51b7ca9 fix: bound encoded common text results including escape and record overhead 2026-10-20 01:48:23 +09:00
Hare 38d9069632 feat: publish stable root file contracts and target-owned dispatch 2026-10-20 01:46:30 +09:00
Hare e5749b9be0 test: integrate Drive connection and lifetime coverage 2026-10-20 00:37:32 +09:00
Hare 387b5ff7df docs: record Drive connection coverage and fault evidence 2026-10-20 00:35:11 +09:00
Hare ec6c5beda0 test: connect Drive publication to client and resource lifetimes 2026-10-20 00:34:50 +09:00
Hare 8e40fae6f4 fix: recover unpublished Subject creation and preserve auth rejections 2026-10-19 18:56:24 +09:00
Hare c961f244ab fix: preserve feature metadata for concise Journal startup context 2026-10-19 18:45:32 +09:00
Hare cea0225a6e fix: migrate legacy session history and retained archives 2026-10-19 13:52:43 +09:00
Hare b3189d8e85 fix: preserve worker retention failure diagnostics 2026-10-19 13:52:43 +09:00
Hare 24bc45a31e chore: merge develop into hare/develop 2026-10-19 01:31:23 +09:00
Hare b02360fd75 fix: restore orchestrator target push requirement 2026-10-19 01:31:01 +09:00
Hare 41c9d2d5c7 docs: record restored parent regression evidence and cost 2026-10-19 00:02:02 +09:00
Hare cc66bd5052 test: connect restored Runtime parent tools to child session 2026-10-19 00:02:02 +09:00
Hare d1408d3a2f fix: project runtime list subjects for Worker control tools 2026-10-19 00:02:02 +09:00
Hare 19122fb5ea fix: use public Worker control entry in regression design 2026-10-19 00:02:02 +09:00
Hare 1fb216e6ae fix: refresh Ticket regression design evidence 2026-10-19 00:02:02 +09:00
Hare 4b14f7e6b3 docs: design prioritized regression coverage 2026-10-19 00:02:02 +09:00
Hare 67767eeee7 test: integrate Ticket public regression coverage 2026-10-18 02:35:16 +09:00
Hare 1bfb4ce119 fix: align Ticket regression design with Host retry identity 2026-10-18 02:10:15 +09:00
Hare 651820dc6d test: connect Ticket public regressions to real Backend 2026-10-18 02:10:15 +09:00
Hare 75b1557d63 feat: browse Subject Journals and persistent Memory links 2026-10-18 02:03:51 +09:00
Hare 23ff518209 fix: clarify Ticket regression fixture boundaries 2026-10-18 01:26:52 +09:00
Hare fab9e0f4b5 docs: design focused Ticket regression coverage 2026-10-18 01:26:52 +09:00
Hare 5ee42b8db8 docs: record Journal compaction validation and evaluation limits 2026-10-17 12:29:47 +09:00
Hare daf438701f docs: explain shared Journal snapshot boundary guarantees 2026-10-17 12:25:24 +09:00
Hare 23ee401287 fix: disambiguate historical Journal retention instructions 2026-10-17 12:25:24 +09:00
Hare 8f598da931 test: verify frozen Journal compaction across Hosts and restores 2026-10-17 12:25:24 +09:00
Hare 956aeed6b2 feat: freeze Host Journal context across compaction 2026-10-17 12:19:37 +09:00
350 changed files with 59713 additions and 5469 deletions
Generated
+9
View File
@@ -5266,6 +5266,7 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"tracing",
"uuid",
"workdir",
]
@@ -6705,15 +6706,23 @@ dependencies = [
name = "yoi-e2e"
version = "0.0.0"
dependencies = [
"chrono",
"config-source",
"libc",
"manifest",
"protocol",
"reqwest",
"serde",
"serde_json",
"server-api",
"session-store",
"sha2 0.11.0",
"tempfile",
"tungstenite 0.29.0",
"webauthn-authenticator-rs",
"webauthn-rs-proto",
"worker-runtime",
"yoi-workspace-server",
]
[[package]]
+1
View File
@@ -908,6 +908,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let response = create_backend_worker_with_client(&target, &create, &api)
+1
View File
@@ -567,6 +567,7 @@ impl BackendWorkspaceProductClient {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
};
let workspace_id = self.workspace_id.clone();
let response: BrowserCreateWorkerResponse = self.generated(move |client| async move {
+30
View File
@@ -200,6 +200,7 @@ fn run_read_with_limits(
total_lines,
content_hash: content_hasher.finalize().into(),
truncated: end < total_lines || byte_truncated,
byte_truncated,
})
}
@@ -885,6 +886,34 @@ mod bounded_read_tests {
assert!(error.to_string().contains("exceeds provider limit 4096"));
}
#[test]
fn bounded_read_distinguishes_utf8_byte_cut_from_complete_line_continuation() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
fs::write(root.join("text"), "雪雪needle\nnext").unwrap();
for (offset, max_bytes, bytes, more, cut) in [
(0, 5, "雪", true, true),
(0, 64, "雪雪needle\n", true, false),
(1, 64, "next", false, false),
] {
let result = run_read_bounded(
&root,
ReadRequest {
path: FsPath::new("text").unwrap(),
offset,
limit: 1,
max_bytes,
},
&RootAccess(root.clone()),
BoundedReadLimits::new(1024, 64).unwrap(),
)
.unwrap();
assert_eq!(result.bytes, bytes.as_bytes());
assert_eq!(result.truncated, more);
assert_eq!(result.byte_truncated, cut);
}
}
#[test]
fn bounded_read_caps_retained_response_independently_of_request() {
let temp = tempfile::tempdir().unwrap();
@@ -907,6 +936,7 @@ mod bounded_read_tests {
assert_eq!(result.bytes, b"alpha");
assert_eq!(result.total_lines, 3);
assert!(result.truncated);
assert!(result.byte_truncated);
}
}
+4
View File
@@ -204,7 +204,11 @@ pub struct ReadResult {
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub total_lines: usize,
pub content_hash: ContentHash,
/// More source lines remain, or the selected window exceeded the response byte budget.
pub truncated: bool,
/// The selected line window was cut by its byte budget, possibly within a line.
/// Line-based cursors must not treat such a window as completely scanned.
pub byte_truncated: bool,
}
/// Binary-safe byte range read with an optional whole-file version fence.
+17 -1
View File
@@ -1747,7 +1747,23 @@ fn ticket_event(
) -> Result<(), MergeRequestError> {
let ticket = &mr.ticket_ids[0];
let n:i64=t.query_row("SELECT COALESCE(MAX(event_index),-1)+1 FROM typed_ticket_events WHERE workspace_id=?1 AND ticket_id=?2",params![mr.workspace_id,ticket],|r|r.get(0))?;
t.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,author,at,from_state,to_state,heading,body)VALUES(?1,?2,?3,'comment',?4,?5,NULL,NULL,'Merge Request integrated',?6)",params![mr.workspace_id,ticket,n,format!("worker:{}:{}",e.merged_by.runtime_id,e.merged_by.worker_id),e.created_at.to_rfc3339(),format!("Merge Request `{}` integrated approved source ref `{}`; Ticket remains in progress until guarded Ticket completion.",mr.merge_request_id,e.approved_source_ref)])?;
t.execute(
"INSERT INTO typed_ticket_events(
workspace_id,ticket_id,event_index,kind,author,at,from_state,to_state,heading,body,event_ref
) VALUES(?1,?2,?3,'comment',?4,?5,NULL,NULL,'Merge Request integrated',?6,?7)",
params![
mr.workspace_id,
ticket,
n,
format!("worker:{}:{}", e.merged_by.runtime_id, e.merged_by.worker_id),
e.created_at.to_rfc3339(),
format!(
"Merge Request `{}` integrated approved source ref `{}`; Ticket remains in progress until guarded Ticket completion.",
mr.merge_request_id, e.approved_source_ref
),
format!("{ticket}:{n}"),
],
)?;
for (k, v) in [
("implementation_assignment_id", a),
("merge_request_id", &mr.merge_request_id),
+116 -5
View File
@@ -53,8 +53,8 @@ fn fixture() -> (tempfile::TempDir, MergeRequestStore) {
let d = tempfile::tempdir().unwrap();
let p = d.path().join("db");
let c = Connection::open(&p).unwrap();
c.execute_batch("CREATE TABLE workspaces(workspace_id TEXT PRIMARY KEY);CREATE TABLE repositories(workspace_id TEXT,repository_id TEXT,PRIMARY KEY(workspace_id,repository_id));CREATE TABLE ticket_current_worker_assignments(workspace_id TEXT,ticket_id TEXT,assignment_id TEXT,runtime_id TEXT,worker_id TEXT,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_tickets(workspace_id TEXT,ticket_id TEXT,workflow_state TEXT,workflow_state_explicit INTEGER,updated_at TEXT,title TEXT NOT NULL,body TEXT NOT NULL,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_ticket_targets(workspace_id TEXT,ticket_id TEXT,ordinal INTEGER,repository_key TEXT,ref_selector TEXT,access TEXT);CREATE TABLE typed_ticket_events(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,kind TEXT,author TEXT,at TEXT,from_state TEXT,to_state TEXT,heading TEXT,body TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index));CREATE TABLE typed_ticket_event_attributes(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,key TEXT,value TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index,key));INSERT INTO workspaces VALUES('W');INSERT INTO repositories VALUES('W','R');INSERT INTO repositories VALUES('W','R2');INSERT INTO ticket_current_worker_assignments VALUES('W','T','A','runtime','coder','t');INSERT INTO typed_tickets VALUES('W','T','inprogress',1,'t','Title','Requirements');CREATE VIEW ticket_active_worker_assignments AS SELECT current.* FROM ticket_current_worker_assignments current JOIN typed_tickets ticket ON ticket.workspace_id=current.workspace_id AND ticket.ticket_id=current.ticket_id WHERE ticket.workflow_state NOT IN ('done','closed');").unwrap();
c.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body) VALUES('W','T',-1,'comment','Requirements')",[]).unwrap();
c.execute_batch("CREATE TABLE workspaces(workspace_id TEXT PRIMARY KEY);CREATE TABLE repositories(workspace_id TEXT,repository_id TEXT,PRIMARY KEY(workspace_id,repository_id));CREATE TABLE ticket_current_worker_assignments(workspace_id TEXT,ticket_id TEXT,assignment_id TEXT,runtime_id TEXT,worker_id TEXT,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_tickets(workspace_id TEXT,ticket_id TEXT,workflow_state TEXT,workflow_state_explicit INTEGER,updated_at TEXT,title TEXT NOT NULL,body TEXT NOT NULL,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_ticket_targets(workspace_id TEXT,ticket_id TEXT,ordinal INTEGER,repository_key TEXT,ref_selector TEXT,access TEXT);CREATE TABLE typed_ticket_events(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,kind TEXT,author TEXT,at TEXT,from_state TEXT,to_state TEXT,heading TEXT,body TEXT,event_ref TEXT NOT NULL DEFAULT '',PRIMARY KEY(workspace_id,ticket_id,event_index),UNIQUE(workspace_id,ticket_id,event_ref));CREATE TABLE typed_ticket_event_attributes(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,key TEXT,value TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index,key));INSERT INTO workspaces VALUES('W');INSERT INTO repositories VALUES('W','R');INSERT INTO repositories VALUES('W','R2');INSERT INTO ticket_current_worker_assignments VALUES('W','T','A','runtime','coder','t');INSERT INTO typed_tickets VALUES('W','T','inprogress',1,'t','Title','Requirements');CREATE VIEW ticket_active_worker_assignments AS SELECT current.* FROM ticket_current_worker_assignments current JOIN typed_tickets ticket ON ticket.workspace_id=current.workspace_id AND ticket.ticket_id=current.ticket_id WHERE ticket.workflow_state NOT IN ('done','closed');").unwrap();
c.execute("INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body,event_ref) VALUES('W','T',-1,'comment','Requirements','T:-1')",[]).unwrap();
drop(c);
let a = Assignments(Arc::new(Mutex::new(CurrentAssignment {
assignment_id: "A".into(),
@@ -457,6 +457,18 @@ fn integration_preserves_selectors_ticket_state_and_assignment_and_replays_its_m
.unwrap();
assert_eq!(replayed, merged);
let connection = Connection::open(d.path().join("db")).unwrap();
let ticket_refs = connection
.prepare("SELECT event_ref FROM typed_ticket_events WHERE event_index >= 0")
.unwrap()
.query_map([], |row| row.get::<_, String>(0))
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
ticket_refs,
vec!["T:0"],
"replay must not append another post"
);
let (state, active): (String, bool) = connection.query_row(
"SELECT workflow_state, EXISTS(SELECT 1 FROM ticket_active_worker_assignments WHERE workspace_id='W' AND ticket_id='T') FROM typed_tickets WHERE workspace_id='W' AND ticket_id='T'",
[],
@@ -477,6 +489,87 @@ fn integration_preserves_selectors_ticket_state_and_assignment_and_replays_its_m
assert!(!json.contains(banned), "{banned} in {json}")
}
}
#[test]
fn integration_post_failure_rolls_back_and_same_operation_recovers_with_legacy_empty_ref() {
let (dir, store) = fixture();
open(&store);
let review = approve(&store, "source", "approved");
request(&store, "source", "pending");
let completion = CompleteMergeRequest {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
operation_id: "recover-integration".into(),
approval_event_id: review.event_id,
current_subject_ref: "source".into(),
target_ref_before: "before".into(),
target_ref_after: "after".into(),
strategy: MergeStrategy::FastForward,
resolution: ConflictResolution::None,
auth: auth(),
now: at(5),
};
let connection = Connection::open(dir.path().join("db")).unwrap();
// Retain a comment written before integration populated event_ref.
connection.execute_batch(
"INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,body,event_ref)
VALUES('W','T',0,'comment','Legacy integration','');
CREATE TRIGGER reject_integration_post BEFORE INSERT ON typed_ticket_events
WHEN NEW.heading='Merge Request integrated'
BEGIN SELECT RAISE(ABORT,'injected integration post failure'); END;",
).unwrap();
let before = store.get_by_id("W", "MR").unwrap();
let error = store.complete(completion.clone()).unwrap_err();
assert!(
error
.to_string()
.contains("injected integration post failure")
);
let failed = store.get_by_id("W", "MR").unwrap();
assert_eq!(failed.state, MergeRequestState::Open);
assert_eq!(failed.thread, before.thread);
assert!(store.authorize_review_submission("MR", "pending").is_ok());
assert_eq!(
connection
.query_row(
"SELECT COUNT(*) FROM typed_ticket_event_attributes",
[],
|row| row.get::<_, i64>(0),
)
.unwrap(),
0
);
connection
.execute_batch("DROP TRIGGER reject_integration_post")
.unwrap();
let merged = store.complete(completion.clone()).unwrap();
assert_eq!(store.complete(completion).unwrap(), merged);
assert_eq!(
store.get_by_id("W", "MR").unwrap().state,
MergeRequestState::Merged
);
assert!(store.authorize_review_submission("MR", "pending").is_err());
let posts = connection
.prepare("SELECT event_ref,body FROM typed_ticket_events WHERE event_index >= 0 ORDER BY event_index")
.unwrap()
.query_map([], |row| Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?)))
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
posts.len(),
2,
"retry and replay must append exactly one post"
);
assert_eq!(posts[0], (String::new(), "Legacy integration".into()));
assert_eq!(posts[1].0, "T:1");
assert!(posts[1].1.contains("`MR`"));
assert_eq!(connection.query_row(
"SELECT value FROM typed_ticket_event_attributes WHERE event_index=1 AND key='operation_id'",
[], |row| row.get::<_, String>(0),
).unwrap(), "recover-integration");
}
#[test]
fn source_move_cancels_submission_and_old_approval_is_reusable_when_source_returns() {
let (_d, s) = fixture();
@@ -955,7 +1048,7 @@ fn ticket_rescope_requires_fresh_review_before_merge_and_remains_recoverable() {
let connection = Connection::open(dir.path().join("db")).unwrap();
connection
.execute_batch(
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL)",
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL,'T:1')",
)
.unwrap();
connection
@@ -1082,7 +1175,7 @@ fn ticket_rescope_after_integration_accepts_fresh_requirement_attestation() {
let connection = Connection::open(dir.path().join("db")).unwrap();
connection
.execute_batch(
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL)",
"UPDATE typed_tickets SET title='Rescoped' WHERE workspace_id='W' AND ticket_id='T'; INSERT INTO typed_ticket_events VALUES('W','T',1,'item_edit','user','t2',NULL,NULL,NULL,NULL,'T:1')",
)
.unwrap();
connection
@@ -1307,6 +1400,24 @@ fn partial_and_full_integration_retain_ticket_state_and_active_assignment() {
assert_eq!(second_request.merged_result().unwrap(), &second_merge);
assert_eq!(first_request.integration_approval().unwrap(), &first);
assert_eq!(second_request.integration_approval().unwrap(), &second);
let posts = connection
.prepare(
"SELECT event.event_ref, attribute.value
FROM typed_ticket_events event
JOIN typed_ticket_event_attributes attribute USING(workspace_id,ticket_id,event_index)
WHERE attribute.key='merge_request_id' ORDER BY event.event_index",
)
.unwrap()
.query_map([], |row| {
Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?))
})
.unwrap()
.collect::<Result<Vec<_>, _>>()
.unwrap();
assert_eq!(
posts,
vec![("T:0".into(), "MR".into()), ("T:1".into(), "MR-2".into())]
);
}
#[test]
@@ -1324,7 +1435,7 @@ fn historical_ticket_completion_payload_remains_readable_and_unchanged_on_migrat
}"#;
connection.execute_batch(
"UPDATE typed_tickets SET workflow_state='done' WHERE workspace_id='W' AND ticket_id='T';
INSERT INTO typed_ticket_events VALUES('W','T',0,'state_changed','worker:runtime:coder','2026-07-26T12:00:07Z','inprogress','done','Ticket requirements completed','historical body');"
INSERT INTO typed_ticket_events VALUES('W','T',0,'state_changed','worker:runtime:coder','2026-07-26T12:00:07Z','inprogress','done','Ticket requirements completed','historical body','T:0');"
).unwrap();
connection
.execute(
+19 -7
View File
@@ -408,7 +408,7 @@ pub enum Method {
/// Transport is fire-and-forget; receivers must tolerate out-of-order
/// delivery (e.g. `TurnEnded` arriving after `ShutDown` for the same
/// child Worker).
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
@@ -982,6 +982,17 @@ pub enum SessionEntryProvenance {
LegacyUnknown,
}
/// Public use decisions contain no Feature identity or private payload. Absence
/// denotes the older projection contract, not permission to expose System input.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(deny_unknown_fields)]
pub struct SessionEntryUsePolicy {
pub observation_readable: bool,
pub memory_evidence: bool,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -994,6 +1005,8 @@ pub struct SessionSnapshotEntry {
pub provenance: SessionEntryProvenance,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub derived_from: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub use_policy: Option<SessionEntryUsePolicy>,
#[serde(flatten)]
pub data: SessionSnapshotEntryData,
}
@@ -1180,12 +1193,10 @@ pub enum Event {
},
/// One agent-injected system item committed to history.
///
/// Carries the JSON form of `session_store::SystemItem`. Covers
/// `Method::Notify` echoes, child-Worker lifecycle events from
/// `Method::WorkerEvent`, `@<path>` resolution payloads, and any future
/// Clients dispatch on the `kind` tag for typed rendering instead
/// of parsing free-text prefixes like `[Notification] …` or
/// `[File: …]`.
/// Carries the allowlisted JSON form of `session_store::PublicSystemItem`,
/// not the storage record. Its descriptive `kind` is independent from the
/// typed `presentation` used for notification, reminder, or context rendering.
/// Private Feature payloads and prompt provenance are never event data.
///
/// One event per `LogEntry::AnnotatedSystemItem` commit. Disk-side and
/// wire-side are 1:1.
@@ -2621,6 +2632,7 @@ mod tests {
session: SessionSnapshot {
pending_submissions: PendingSubmissionsSnapshot::default(),
entries: vec![SessionSnapshotEntry {
use_policy: None,
entry_id: "entry-1".into(),
timestamp: 1,
provenance: SessionEntryProvenance::HumanInput,
+7 -6
View File
@@ -17,12 +17,12 @@ use crate::{
PasteArtifactRef, PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission,
ReasoningConfig, RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason,
ScopeRule, Segment, SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment, SubmissionDisposition,
SymlinkPolicy, ToolResultDisposition, TurnResult, UploadedFileAvailability, UploadedFileRef,
WorkerBusyState, WorkerCommandAcknowledgement, WorkerCommandDisposition, WorkerCommandEnvelope,
WorkerCommandKind, WorkerEvent, WorkerMaintenanceState, WorkerRunState, WorkerState,
WorkerStateSnapshot, WorkerStatus,
SessionEntryUsePolicy, SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole,
SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment,
SubmissionDisposition, SymlinkPolicy, ToolResultDisposition, TurnResult,
UploadedFileAvailability, UploadedFileRef, WorkerBusyState, WorkerCommandAcknowledgement,
WorkerCommandDisposition, WorkerCommandEnvelope, WorkerCommandKind, WorkerEvent,
WorkerMaintenanceState, WorkerRunState, WorkerState, WorkerStateSnapshot, WorkerStatus,
subscription::{
EventSubscriptionSelector, MAX_CORRELATION_ID_BYTES, MAX_REJECTION_MESSAGE_BYTES,
MAX_RESOURCE_ID_BYTES, MAX_SUBSCRIPTION_COLLECTION_ITEMS,
@@ -117,6 +117,7 @@ pub fn generated_protocol_types() -> String {
push_decl::<InFlightCompaction>(&cfg, &mut output);
push_decl::<InFlightSnapshot>(&cfg, &mut output);
push_decl::<SessionEntryProvenance>(&cfg, &mut output);
push_decl::<SessionEntryUsePolicy>(&cfg, &mut output);
push_decl::<SessionMessageRole>(&cfg, &mut output);
push_decl::<SessionContentPart>(&cfg, &mut output);
push_decl::<SessionToolAttachment>(&cfg, &mut output);
+57 -1
View File
@@ -518,9 +518,10 @@ pub struct WorkerRestorePreparation {
pub workspace_api: Option<WorkspaceApiRef>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workdir_attachments: Option<Vec<LogicalWorkdirAttachment>>,
/// Retained in saved Restore intents; Restore does not apply Repository access.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access: Vec<WorkingDirectoryRepositoryAccessRequest>,
/// Read-only references resolved by Workspace only after this operation owns admission.
/// Retained references from saved resource preparation, not Restore prerequisites.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access_workdirs: Vec<RepositoryAccessWorkdirReference>,
}
@@ -1140,8 +1141,36 @@ pub struct SubjectReleaseResponse {
pub state: SubjectReleaseState,
}
/// Irreversibly fence an unpublished failed Subject create. Exact retries must retain
/// all four fields; this is distinct from releasing an existing Subject Controller.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryRequest {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
/// Durable admission-fence proof plus freshly confirmed execution absence.
/// An error (including a lost response) is never evidence of absence.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectCreateRecoveryResponse {
pub workspace_id: String,
pub worker_id: WorkerId,
pub create_fingerprint: String,
pub operation_id: String,
}
#[api(reqwest, axum)]
pub trait RuntimeApi {
#[post("/v1/subject-create-recovery", status = 200, error_status = 400)]
async fn recover_subject_create(
&self,
#[body] request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError>;
#[post("/v1/subject-release", status = 200, error_status = 400)]
async fn subject_release(
&self,
@@ -1675,6 +1704,18 @@ mod tests {
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn recover_subject_create(
&self,
request: SubjectCreateRecoveryRequest,
) -> Result<SubjectCreateRecoveryResponse, RuntimeApiError> {
Ok(SubjectCreateRecoveryResponse {
workspace_id: request.workspace_id,
worker_id: request.worker_id,
create_fingerprint: request.create_fingerprint,
operation_id: request.operation_id,
})
}
async fn subject_release(
&self,
_request: SubjectReleaseRequest,
@@ -1820,6 +1861,21 @@ mod tests {
.build()
.unwrap();
let recovery = SubjectCreateRecoveryRequest {
workspace_id: "workspace-a".into(),
worker_id: WorkerId::now_v7(),
create_fingerprint: "failed-create-fingerprint".into(),
operation_id: "recovery-1".into(),
};
let response = client
.recover_subject_create(recovery.clone())
.await
.unwrap();
assert_eq!(response.workspace_id, recovery.workspace_id);
assert_eq!(response.worker_id, recovery.worker_id);
assert_eq!(response.create_fingerprint, recovery.create_fingerprint);
assert_eq!(response.operation_id, recovery.operation_id);
let summary = client.runtime_summary().await.unwrap();
assert_eq!(summary.runtime.display_name.as_deref(), Some("round-trip"));
let workers = client
+475
View File
@@ -0,0 +1,475 @@
//! Neutral explicit-allow authorization contracts. Resource containment never crosses kinds.
//!
//! # Trusted Feature boundary
//!
//! A Profile requests execution configuration, not authority. The Worker asks the
//! Host for each enabled Feature's exact activation name before initialization,
//! then trusted Feature code asks for each protected operation and its semantic
//! resource before access or effects. `ticket` on [`PermissionResource::Feature`]
//! activates Ticket; it implies neither `ticket.create` nor any future operation.
//! `ticket.create` on a Workspace permits creation there but cannot activate the
//! Feature. There are no wildcard permissions or implicit dotted-name inheritance.
//!
//! Identity is absent from the authorization request intentionally: the Host
//! authenticates the bound Runtime/Worker and resolves current source permissions,
//! body ownership and any per-Worker execution grant. A shared client, Profile,
//! remembered receipt or caller-supplied Subject claim is not authority. Features
//! depend on this contract, never on a particular Subject policy implementation.
//!
//! Supported cooperative boundaries are Feature activation, ordinary Worker
//! delegation, Ticket creation (Tool/native WIP/typed service), Workdir filesystem
//! read/write and whole-attachment command execution, and scoped result notify.
//! Workdir providers still enforce resolved paths, traversal, capability, Ticket
//! and provider ceilings. The returned scope is for provider narrowing, not a
//! transferable credential. Scoped checkout/overlap operations are refused until
//! their complete access semantics are adapted. Other Feature operations retain
//! their existing owner checks; they do not automatically acquire this operation
//! policy. Missing calls are trusted Feature bugs, not interception by a universal
//! Tool executor or protection against malicious Feature code. Connection tests
//! must exercise each exposed surface, assert a genuine permitted effect and
//! assert zero downstream calls on denial, including revocation and provider denial.
//!
//! # Delegation callers and recovery
//!
//! Callers such as a chat Feature supply explicit operation/resource selections
//! and retain the same public operation ID and full request after an unknown
//! result. Normal Profile/Worker allocation and control/resource grants are reused;
//! there is no Subject Worker registry. Effective grants bind immutable Profile
//! resolution, the reserved Worker and source assignment. Permission replacement
//! invalidates existing children rather than refreshing them. Body replacement,
//! retirement and restore must recheck current authority; historical result lookup
//! is not new execution permission. Cancellation acceptance is distinct from
//! terminal confirmation and does not undo completed effects.
//!
//! A child receives no Subject body identity, singleton, whole Memory or public
//! conversation-post authority. `worker.result` is explicit notify authority to a
//! named origin Worker, not arbitrary control or publication. Chat delivery and
//! conversation correlation belong to the caller integration, not this contract.
//! Unset existing Subject policies deny new Subject execution/delegation; ordinary
//! non-Subject Workers explicitly retain existing Backend/provider contracts.
//! Standalone non-Workspace activation compatibility grants no Workspace operation.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
pub const MAX_SCOPED_PERMISSIONS: usize = 256;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PermissionResource {
Feature,
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
// Serde's internally tagged unit variants ignore extra fields even with deny_unknown_fields.
// A struct-shaped wire variant preserves the public unit variant while rejecting those claims.
impl<'de> Deserialize<'de> for PermissionResource {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
#[derive(Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
enum Wire {
Feature {},
Workspace {
workspace_id: String,
},
Workdir {
working_directory_id: String,
path: String,
recursive: bool,
},
Worker {
runtime_id: String,
worker_id: String,
},
}
Ok(match Wire::deserialize(deserializer)? {
Wire::Feature {} => Self::Feature,
Wire::Workspace { workspace_id } => Self::Workspace { workspace_id },
Wire::Workdir {
working_directory_id,
path,
recursive,
} => Self::Workdir {
working_directory_id,
path,
recursive,
},
Wire::Worker {
runtime_id,
worker_id,
} => Self::Worker {
runtime_id,
worker_id,
},
})
}
}
impl PermissionResource {
pub fn validate(&self) -> Result<(), String> {
match self {
Self::Feature => Ok(()),
Self::Workspace { workspace_id } => validate_authorization_identifier(workspace_id),
Self::Workdir {
working_directory_id,
path,
..
} => {
validate_authorization_identifier(working_directory_id)?;
validate_relative_permission_path(path)
}
Self::Worker {
runtime_id,
worker_id,
} => {
validate_authorization_identifier(runtime_id)?;
validate_authorization_identifier(worker_id)
}
}
}
pub fn covers(&self, other: &Self) -> bool {
if self.validate().is_err() || other.validate().is_err() {
return false;
}
match (self, other) {
(Self::Feature, Self::Feature) => true,
(Self::Workspace { workspace_id: a }, Self::Workspace { workspace_id: b }) => a == b,
(
Self::Worker {
runtime_id: ar,
worker_id: aw,
},
Self::Worker {
runtime_id: br,
worker_id: bw,
},
) => ar == br && aw == bw,
(
Self::Workdir {
working_directory_id: a,
path: ap,
recursive: ar,
},
Self::Workdir {
working_directory_id: b,
path: bp,
recursive: br,
},
) => {
a == b
&& if *ar {
ap == "."
|| ap == bp
|| bp
.strip_prefix(ap)
.is_some_and(|tail| tail.starts_with('/'))
} else {
ap == bp && !br
}
}
_ => false,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ScopedPermission {
pub operation: String,
pub resource: PermissionResource,
}
impl ScopedPermission {
pub fn validate(&self) -> Result<(), String> {
validate_operation_name(&self.operation)?;
self.resource.validate()?;
if matches!(self.resource, PermissionResource::Feature) && self.operation.contains('.') {
return Err("Feature resources authorize activation names only, not operations".into());
}
Ok(())
}
pub fn covers(&self, other: &Self) -> bool {
self.validate().is_ok()
&& other.validate().is_ok()
&& self.operation == other.operation
&& self.resource.covers(&other.resource)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissions {
pub change_id: String,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsUpdate {
pub operation_id: String,
pub expected_change_id: Option<String>,
pub execution: Vec<ScopedPermission>,
pub delegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerExecutionGrant {
pub subject_id: String,
pub origin_runtime_id: String,
pub origin_worker_id: String,
pub source_change_id: String,
pub runtime_id: String,
pub worker_id: String,
pub profile: String,
pub profile_digest: String,
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerDelegationRequest {
pub permissions: Vec<ScopedPermission>,
pub redelegation: Vec<ScopedPermission>,
}
impl WorkerDelegationRequest {
pub fn validate(&self) -> Result<(), String> {
validate_scoped_permissions(&self.permissions)?;
validate_scoped_permissions(&self.redelegation)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationRequest {
pub operation: String,
pub resource: PermissionResource,
}
impl OperationAuthorizationRequest {
pub fn validate(&self) -> Result<(), String> {
ScopedPermission {
operation: self.operation.clone(),
resource: self.resource.clone(),
}
.validate()
}
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct OperationAuthorizationResponse {
/// Matching granted scope for provider-resolved checks, not a credential.
/// `None` identifies the explicit non-Subject legacy authorization path.
#[serde(default)]
pub scope: Option<PermissionResource>,
}
pub fn validate_operation_name(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| !value.split('.').all(|part| {
!part.is_empty()
&& part
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'_' || b == b'-')
})
{
return Err(
"permission operation must be an exact, nonempty dot-separated name (no wildcards)"
.into(),
);
}
Ok(())
}
pub fn validate_scoped_permissions(permissions: &[ScopedPermission]) -> Result<(), String> {
if permissions.len() > MAX_SCOPED_PERMISSIONS {
return Err("too many scoped permissions".into());
}
for permission in permissions {
permission.validate()?;
}
Ok(())
}
pub fn validate_authorization_identifier(value: &str) -> Result<(), String> {
if value.is_empty()
|| value.len() > 256
|| value.trim() != value
|| value.chars().any(|c| c.is_control() || c == '*')
{
return Err("authorization identity must be nonempty, bounded and exact".into());
}
Ok(())
}
/// Paths are already-normalized lexical paths, not host filesystem paths. The Host must
/// still enforce symlink/provider boundaries; lexical containment grants no filesystem access.
pub fn validate_relative_permission_path(path: &str) -> Result<(), String> {
if path == "." {
return Ok(());
}
if path.is_empty()
|| path.len() > 4096
|| path.contains(['\\', ':'])
|| path.chars().any(char::is_control)
|| path
.split('/')
.any(|part| part.is_empty() || part == "." || part == "..")
{
return Err("permission path must be a normalized relative path without escape".into());
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn workdir(path: &str, recursive: bool) -> PermissionResource {
PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
}
}
#[test]
fn lexical_containment_rejects_escape_and_respects_component_and_recursion_boundaries() {
for path in [
"",
"/src",
"../src",
"src/../secret",
"src/./a",
"src//a",
"src/",
"C:/src",
"src\\a",
] {
assert!(workdir(path, true).validate().is_err(), "{path}");
}
let root = workdir(".", true);
let src = workdir("src", true);
assert!(root.covers(&src));
assert!(src.covers(&workdir("src/a", true)));
assert!(!src.covers(&workdir("src-other", false)));
assert!(!workdir("src", false).covers(&src));
assert!(workdir("src", false).covers(&workdir("src", false)));
assert!(!workdir("src", false).covers(&workdir("src/a", false)));
assert!(!src.covers(&PermissionResource::Workdir {
working_directory_id: "other".into(),
path: "src".into(),
recursive: false
}));
}
#[test]
fn operation_hierarchy_never_implies_inheritance_or_feature_activation() {
for name in [
"",
"*",
"ticket.*",
".ticket",
"ticket.",
"ticket..read",
"ticket read",
] {
assert!(validate_operation_name(name).is_err(), "{name}");
}
let activation = ScopedPermission {
operation: "ticket".into(),
resource: PermissionResource::Feature,
};
let create = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "ws".into(),
},
};
let parent_name = ScopedPermission {
operation: "ticket".into(),
resource: create.resource.clone(),
};
assert!(!parent_name.covers(&create));
assert!(!create.covers(&parent_name));
assert!(!activation.covers(&create));
assert!(!create.covers(&activation));
assert!(
ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Feature
}
.validate()
.is_err()
);
assert!(
!PermissionResource::Workspace {
workspace_id: "ws".into()
}
.covers(&workdir(".", true))
);
}
#[test]
fn wire_contract_rejects_self_claimed_identity_and_unknown_fields() {
let request = serde_json::json!({"permissions":[],"redelegation":[],"subject_id":"forged"});
assert!(serde_json::from_value::<WorkerDelegationRequest>(request).is_err());
let resource: PermissionResource =
serde_json::from_value(serde_json::json!({"kind":"feature"})).unwrap();
assert_eq!(resource, PermissionResource::Feature);
assert!(
serde_json::from_value::<PermissionResource>(
serde_json::json!({"kind":"feature","workspace_id":"ws"})
)
.is_err()
);
}
#[test]
fn authorization_response_carries_matching_scope_and_defaults_legacy_absence() {
let legacy: OperationAuthorizationResponse =
serde_json::from_value(serde_json::json!({})).unwrap();
assert_eq!(legacy.scope, None);
assert_eq!(
serde_json::to_value(legacy).unwrap(),
serde_json::json!({"scope":null})
);
let granted = OperationAuthorizationResponse {
scope: Some(workdir("src", true)),
};
let wire = serde_json::to_value(&granted).unwrap();
assert_eq!(
wire,
serde_json::json!({"scope":{"kind":"workdir","working_directory_id":"wd","path":"src","recursive":true}})
);
assert_eq!(
serde_json::from_value::<OperationAuthorizationResponse>(wire).unwrap(),
granted
);
}
}
+10
View File
@@ -134,6 +134,14 @@ pub enum DriveEntryKind {
File,
}
/// Current caller's advisory publication ceiling, never a grant or execution credential.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveCapabilities {
pub write: bool,
}
/// Metadata only: content is returned through bounded text or binary routes.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -491,6 +499,7 @@ impl api_macros::HttpRequestError for DriveApiError {
impl api_macros::openapi::OpenApiSchema for DriveApiError {}
impl api_macros::openapi::OpenApiSchema for DriveEntry {}
impl api_macros::openapi::OpenApiSchema for DriveCapabilities {}
impl api_macros::openapi::OpenApiSchema for DriveEntryQuery {}
impl api_macros::openapi::OpenApiSchema for DriveListQuery {}
impl api_macros::openapi::OpenApiSchema for DriveSearchQuery {}
@@ -518,6 +527,7 @@ pub fn drive_api_typescript() -> String {
DriveEntryRef::decl(&config),
DriveEntryKind::decl(&config),
DriveEntry::decl(&config),
DriveCapabilities::decl(&config),
DriveEntryQuery::decl(&config),
DriveListQuery::decl(&config),
DriveSearchQuery::decl(&config),
+249 -4
View File
@@ -6,6 +6,7 @@ use api_macros::api;
pub use api_macros::axum as server_support;
pub use api_macros::reqwest as client_support;
pub use api_macros::{ApiContract, BinaryBody, HttpMethod, TransportMetadata, WebSocketOperation};
pub mod authorization;
mod drive;
mod notification;
pub use notification::*;
@@ -23,6 +24,34 @@ use webauthn_rs_proto::{
};
pub use workspace_config::*;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsResponse {
pub permissions: Option<authorization::SubjectPermissions>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionsMutationResponse {
pub permissions: authorization::SubjectPermissions,
/// New operations are already fenced. These are cancellation requests and
/// Backend command cleanup evidence, not proof that a Runtime turn stopped.
pub interruptions: Vec<SubjectPermissionInterruption>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjectPermissionInterruption {
pub runtime_id: String,
pub worker_id: String,
pub cancellation_accepted: bool,
pub backend_command_sessions_closed: bool,
pub diagnostics: Vec<String>,
}
#[allow(dead_code)]
#[derive(JsonSchema)]
struct ServerJsonSafeU64(#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))] u64);
@@ -103,6 +132,8 @@ pub struct WorkdirOperationApiError {
pub workdir_log: Option<WorkdirOperationLogContext>,
#[serde(skip)]
pub denial_reason: Option<workdir::WorkdirDenialReason>,
#[serde(skip)]
pub failure_reason: Option<workdir::WorkdirFailureReason>,
}
/// Internal operational correlation. Producers supply fixed-length ID fingerprints and
@@ -128,6 +159,7 @@ impl WorkdirOperationApiError {
status: error.status,
workdir_log: None,
denial_reason: None,
failure_reason: None,
}
}
@@ -141,6 +173,7 @@ impl WorkdirOperationApiError {
status,
workdir_log: None,
denial_reason: error.denial_reason,
failure_reason: error.failure_reason,
}
}
}
@@ -168,6 +201,11 @@ macro_rules! impl_openapi_schema {
}
impl_openapi_schema!(
SubjectPermissionsResponse,
SubjectPermissionsMutationResponse,
SubjectPermissionInterruption,
authorization::SubjectPermissions,
authorization::SubjectPermissionsUpdate,
AuthBootstrapUserRequest,
AuthPublicConfig,
HostingSettings,
@@ -187,6 +225,10 @@ impl_openapi_schema!(
PasskeyLoginOptionsResponse,
PasskeyLoginCompleteRequest,
PasskeyAdditionOptionsRequest,
PasskeySummary,
PasskeyListResponse,
PasskeyAdditionResultRequest,
PasskeyAdditionResultResponse,
PasskeyRegistrationOptionsRequest,
PasskeyRegistrationOptionsResponse,
PasskeyRegistrationCompleteRequest,
@@ -610,6 +652,33 @@ pub trait ServerApi {
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthPasskeyRegistrationComplete, RepositoryApiError>;
#[get(
"/api/auth/passkeys",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_list(
&self,
#[extension] context: ServerRequestContext,
) -> Result<PasskeyListResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/result",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_result(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyAdditionResultRequest,
) -> Result<PasskeyAdditionResultResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/options",
status = 200,
@@ -1298,6 +1367,51 @@ pub trait ServerApi {
#[body] request: MemoryConsolidateStagingRequest,
) -> Result<MemoryConsolidationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/authorization/check",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
openapi = false,
)]
async fn operation_authorize(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: authorization::OperationAuthorizationRequest,
) -> Result<authorization::OperationAuthorizationResponse, RepositoryApiError>;
#[get(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_get(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
) -> Result<SubjectPermissionsResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/permissions",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_permissions_set(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
#[body] request: authorization::SubjectPermissionsUpdate,
) -> Result<SubjectPermissionsMutationResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects",
status = 201,
@@ -2762,6 +2876,12 @@ pub trait ServerApi {
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveEntry, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/capabilities", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_capabilities(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<DriveCapabilities, DriveApiError>;
#[get("/api/w/{workspace_id}/drive/metadata", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,503], bearer_auth = true, browser_auth = true)]
async fn drive_metadata(
&self,
@@ -4025,6 +4145,19 @@ impl RequestActor {
#[serde(deny_unknown_fields)]
pub struct WhoamiResponse {
pub actor: Option<RequestActor>,
pub session: BrowserSessionState,
/// Non-secret comparison evidence, never an authentication credential.
pub authentication_reference: Option<String>,
}
/// Browser navigation evidence, not an authorization grant or a generic 401 classification.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum BrowserSessionState {
Anonymous {},
Authenticated { expires_at: Option<String> },
Expired {},
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -4042,6 +4175,8 @@ pub struct AuthBootstrapUserRequest {
#[serde(deny_unknown_fields)]
pub struct AuthUserResponse {
pub user: AuthenticatedUser,
/// Comparison evidence for the exact newly issued session, not a credential.
pub authentication_reference: String,
}
/// Starts new-account registration only; an existing handle is never interpreted as key addition.
@@ -4058,16 +4193,64 @@ pub struct PasskeyRegistrationOptionsRequest {
pub browser_origin: Option<String>,
}
/// Adds a credential to the current authenticated actor; target identity is never public input.
/// Adds to the current actor. Expected identity/binding fields only fence stale UI;
/// they never select an owner or authorize an addition.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyAdditionOptionsRequest {
pub expected_user_id: String,
pub expected_account_id: String,
pub expected_authentication_reference: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub browser_origin: Option<String>,
}
/// Safe account metadata. Neither the WebAuthn credential ID nor key material is exposed.
/// Normal-login verification is observed verifier success, not inferred discoverability.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeySummary {
pub key_reference: String,
pub created_at: String,
pub last_used_at: Option<String>,
pub normal_login_verified_at: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyListResponse {
pub user: AuthenticatedUser,
pub keys: Vec<PasskeySummary>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyAdditionResultRequest {
pub challenge_id: String,
pub expected_user_id: String,
pub expected_account_id: String,
/// Original options authentication reference; the current same-owner session may differ.
pub expected_authentication_reference: String,
}
/// Unknown is not proof that registration failed: completion may still be in flight.
/// Saved proves the credential commit, not session issuance or current session health.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "status", rename_all = "snake_case", deny_unknown_fields)]
pub enum PasskeyAdditionResultResponse {
Unknown {},
Saved {
user: AuthenticatedUser,
key: PasskeySummary,
},
}
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -10428,6 +10611,10 @@ pub struct CreateWorkspaceWorkerRequest {
/// Backend idempotency key used only for authenticated Worker-owned spawn/control.
#[serde(default)]
pub control_operation_id: Option<String>,
/// Requested least-authority child permissions. Host derives the origin; this
/// input cannot assert Subject identity or create a body/singleton attachment.
#[serde(default)]
pub execution_grant: Option<authorization::WorkerDelegationRequest>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -11533,10 +11720,15 @@ pub fn auth_api_typescript() -> String {
ActorAuthMethod::decl(&config),
AuthenticatedUser::decl(&config),
RequestActor::decl(&config),
BrowserSessionState::decl(&config),
WhoamiResponse::decl(&config),
AuthBootstrapUserRequest::decl(&config),
AuthUserResponse::decl(&config),
PasskeyAdditionOptionsRequest::decl(&config),
PasskeySummary::decl(&config),
PasskeyListResponse::decl(&config),
PasskeyAdditionResultRequest::decl(&config),
PasskeyAdditionResultResponse::decl(&config),
PasskeyRegistrationOptionsRequest::decl(&config),
PasskeyRegistrationOptionsResponse::decl(&config),
PasskeyRegistrationCompleteRequest::decl(&config),
@@ -12929,6 +13121,7 @@ mod tests {
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
})
.unwrap();
assert_eq!(
@@ -12943,6 +13136,7 @@ mod tests {
"workdir_attachments": [],
"feature_connections": {},
"control_operation_id": null,
"execution_grant": null,
})
);
}
@@ -13961,7 +14155,9 @@ mod tests {
"handle": "hare",
"display_name": "Hare",
"auth_method": "browser_session"
}
},
"session": { "kind": "authenticated", "expires_at": "2026-10-20T00:00:00Z" },
"authentication_reference": "synthetic-session-reference"
});
let decoded = serde_json::from_value::<WhoamiResponse>(whoami.clone())
.expect("server whoami fixture should match shared DTO");
@@ -13983,7 +14179,8 @@ mod tests {
"account_id": "account-1",
"handle": "hare",
"display_name": "Hare"
}
},
"authentication_reference": "synthetic-session-reference"
});
let decoded = serde_json::from_value::<AuthUserResponse>(auth_user.clone())
.expect("server auth-user fixture should match shared DTO");
@@ -14143,11 +14340,36 @@ mod tests {
let unexpected_field = serde_json::json!({
"actor": null,
"session": { "kind": "anonymous" },
"access_token": "must-not-be-accepted"
});
assert!(serde_json::from_value::<WhoamiResponse>(unexpected_field).is_err());
}
#[test]
fn whoami_session_json_distinguishes_anonymous_expired_and_token_auth() {
for session in [
serde_json::json!({"kind":"anonymous"}),
serde_json::json!({"kind":"expired"}),
serde_json::json!({"kind":"authenticated", "expires_at":null}),
] {
let value = serde_json::json!({"actor":null, "session":session, "authentication_reference": null});
let response: WhoamiResponse = serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(response).unwrap(), value);
}
for session in [
serde_json::json!({"kind":"unknown"}),
serde_json::json!({"kind":"expired", "expires_at":null}),
] {
assert!(
serde_json::from_value::<WhoamiResponse>(
serde_json::json!({"actor":null, "session":session})
)
.is_err()
);
}
}
#[test]
fn device_login_debug_output_redacts_secret_material() {
let start = DeviceLoginStartResponse {
@@ -14532,6 +14754,7 @@ mod openapi_artifact_tests {
// identity, or a one-use Reviewer capability. They remain generated Rust client/Axum
// operations, but must not appear as unauthenticated operations in the public OpenAPI.
const SIGNED_INTERNAL: &[&str] = &[
"operation_authorize",
"current_worker_workspace_config_get",
"current_worker_workspace_config_attach",
"current_worker_workspace_config_observe",
@@ -14668,10 +14891,28 @@ mod openapi_artifact_tests {
);
}
assert!(
serde_json::from_value::<PasskeyAdditionOptionsRequest>(serde_json::json!({})).is_ok()
serde_json::from_value::<PasskeyAdditionOptionsRequest>(serde_json::json!({})).is_err()
);
}
#[test]
fn passkey_addition_requires_expected_owner_and_binding() {
let expected = serde_json::json!({
"expected_user_id": "user-a", "expected_account_id": "account-a",
"expected_authentication_reference": "reference"
});
assert!(serde_json::from_value::<PasskeyAdditionOptionsRequest>(expected.clone()).is_ok());
for field in [
"expected_user_id",
"expected_account_id",
"expected_authentication_reference",
] {
let mut missing = expected.clone();
missing.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<PasskeyAdditionOptionsRequest>(missing).is_err());
}
}
#[test]
fn auth_workspace_catalog_and_typed_header_contract_is_strict() {
let document = canonical_openapi_document().expect("canonical OpenAPI contract must build");
@@ -14917,10 +15158,12 @@ mod workdir_error_diagnostic_tests {
#[test]
fn workdir_internal_diagnostics_never_enter_public_error_body_or_schema() {
let mut error = WorkdirOperationApiError::provider(workdir::http::WorkdirTransportError {
failure_reason: None,
code: workdir::http::WorkdirTransportErrorCode::Denied,
message: "Workdir operation was denied".to_string(),
denial_reason: Some(workdir::WorkdirDenialReason::OsPermissionDenied),
});
error.failure_reason = Some(workdir::WorkdirFailureReason::HttpTimeout);
error.workdir_log = Some(WorkdirOperationLogContext {
operation: "write",
workspace_id_hash: "internal-workspace-fingerprint".to_string(),
@@ -14934,10 +15177,12 @@ mod workdir_error_diagnostic_tests {
assert_eq!(api_macros::HttpError::status_code(&error), 403);
let decoded: WorkdirOperationApiError = serde_json::from_value(wire).unwrap();
assert!(decoded.denial_reason.is_none());
assert!(decoded.failure_reason.is_none());
assert!(decoded.workdir_log.is_none());
let schema = serde_json::to_value(schemars::schema_for!(WorkdirOperationApiError)).unwrap();
assert!(schema["properties"].get("workdir_log").is_none());
assert!(schema["properties"].get("denial_reason").is_none());
assert!(schema["properties"].get("failure_reason").is_none());
}
}
+23
View File
@@ -68,6 +68,24 @@ impl FsStore {
) -> Result<Self, StoreError> {
let root = root.into();
fs::create_dir_all(&root)?;
for entry in fs::read_dir(&root)? {
let entry = entry?;
if entry.file_type()?.is_dir()
&& let Some(session_id) = entry
.file_name()
.to_str()
.and_then(|name| name.parse().ok())
{
let _guard = match retention.lock_live(session_id) {
Ok(guard) => guard,
// Interrupted erasure may leave bytes behind. Never read or
// upgrade them, and do not block unrelated live Sessions.
Err(StoreError::SessionErased(_)) => continue,
Err(error) => return Err(error),
};
crate::worker_session_store::migrate_fs_session(&entry.path(), session_id)?;
}
}
Ok(Self {
root,
append_lock: Arc::new(Mutex::new(())),
@@ -343,6 +361,7 @@ impl Store for FsStore {
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.log_path(session_id, segment_id), &line)
@@ -430,10 +449,14 @@ impl Store for FsStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.retention().lock_live(session_id)?;
let path = self.log_path(session_id, segment_id);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
crate::worker_session_store::migrate_fs_session(parent, session_id)?;
}
let mut content = String::new();
for entry in entries {
+204
View File
@@ -67,8 +67,14 @@ pub enum LoggedSessionHistoryOrigin {
ToolOutput {
worker: LoggedWorkerSubject,
},
/// Descriptive producer identity; use policies live in separate metadata.
Provided {
producer: String,
},
/// Host-produced opening context summary; not auto-read or TaskStore material.
CompactionSummary,
/// Frozen Host Journal reference, never independent observation evidence.
JournalContext,
DerivedSummary,
LegacyUnknown,
}
@@ -79,19 +85,32 @@ pub struct LoggedHistoryDerivation {
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct LoggedSessionHistoryMetadata {
pub entry_id: LoggedSessionHistoryEntryId,
pub origin: LoggedSessionHistoryOrigin,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub derivation: Option<LoggedHistoryDerivation>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub input_metadata: Option<crate::HistoryInputMetadata>,
}
impl LoggedSessionHistoryMetadata {
pub(crate) fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
if matches!(self.origin, LoggedSessionHistoryOrigin::Provided { .. })
&& self.input_metadata.is_none()
{
return Err(crate::InputMetadataError::MissingMetadata);
}
Ok(())
}
pub fn legacy_unknown() -> Self {
Self {
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
input_metadata: None,
}
}
}
@@ -99,20 +118,146 @@ impl LoggedSessionHistoryMetadata {
/// Persisted item and metadata are one value so transforms cannot reorder or
/// truncate one without the other.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(try_from = "HistoryEntryWire")]
pub struct LoggedHistoryEntry {
pub item: LoggedItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct HistoryEntryWire {
item: LoggedItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<HistoryEntryWire> for LoggedHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: HistoryEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedHistoryEntry {
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
if let Some(input) = &self.metadata.input_metadata {
match &self.item {
LoggedItem::Message {
role: crate::LoggedRole::System,
content,
} => {
let [crate::LoggedContentPart::Text { text }] = content.as_slice() else {
return Err(crate::InputMetadataError::InconsistentRecord);
};
if text.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
// Other logical roles may carry use policy, but cannot masquerade
// as the System-only public/payload extension contract.
_ if input.feature_payload().is_some() || input.public_item().is_some() => {
return Err(crate::InputMetadataError::InconsistentRecord);
}
_ => {}
}
}
Ok(())
}
}
/// Typed system-item history record. The typed system event remains available
/// to client replay while its model-visible projection carries the same stable
/// metadata used by live history.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(try_from = "SystemEntryWire")]
pub struct LoggedSystemHistoryEntry {
pub item: crate::SystemItem,
pub metadata: LoggedSessionHistoryMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct SystemEntryWire {
item: crate::SystemItem,
metadata: LoggedSessionHistoryMetadata,
}
impl TryFrom<SystemEntryWire> for LoggedSystemHistoryEntry {
type Error = crate::InputMetadataError;
fn try_from(wire: SystemEntryWire) -> Result<Self, Self::Error> {
let entry = Self {
item: wire.item,
metadata: wire.metadata,
};
entry.validate_input_metadata()?;
Ok(entry)
}
}
impl LoggedSystemHistoryEntry {
pub fn from_input(
origin: LoggedSessionHistoryOrigin,
derivation: Option<LoggedHistoryDerivation>,
input: crate::SystemInputEnvelope,
) -> Self {
let (body, input_metadata) = input.into_parts();
Self {
item: crate::SystemItem::Input { body },
metadata: LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId::new(),
origin,
derivation,
input_metadata: Some(input_metadata),
},
}
}
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
self.metadata.validate_input_metadata()?;
match (&self.item, &self.metadata.input_metadata) {
(crate::SystemItem::Input { body }, Some(_)) => {
if body.len() > crate::input_metadata::MAX_SYSTEM_INPUT_BYTES {
return Err(crate::InputMetadataError::ResourceLimit);
}
}
(crate::SystemItem::Input { .. }, None) => {
return Err(crate::InputMetadataError::MissingMetadata);
}
(_, Some(_)) => return Err(crate::InputMetadataError::InconsistentRecord),
(_, None) => {}
}
Ok(())
}
pub fn to_history_entry(&self) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: self.item.to_history_item().into(),
metadata: self.metadata.clone(),
}
}
pub fn public_item(&self) -> crate::PublicSystemItem {
if matches!(self.item, crate::SystemItem::Input { .. }) {
return self
.metadata
.input_metadata
.as_ref()
.and_then(crate::HistoryInputMetadata::public_item)
.cloned()
.unwrap_or_else(|| self.item.public_item());
}
self.item.public_item()
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -130,6 +275,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -148,6 +294,64 @@ mod tests {
);
}
#[test]
fn external_system_records_reject_missing_policy_role_mismatch_and_overlimit_body() {
let valid = serde_json::json!({
"item": { "kind": "input", "body": "context" },
"metadata": {
"entry_id": "stable-entry", "origin": { "kind": "provided", "producer": "owner" },
"input_metadata": { "usage": {
"recording_material": "exclude", "memory_evidence": "exclude",
"observation": "hidden", "compaction": "reference_input"
} }
}
});
let restored: LoggedSystemHistoryEntry = serde_json::from_value(valid.clone()).unwrap();
let flat = serde_json::to_value(restored.to_history_entry()).unwrap();
assert!(serde_json::from_value::<LoggedHistoryEntry>(flat.clone()).is_ok());
for generic in [true, false] {
let wire = if generic { &valid } else { &flat };
let decode = |wire| {
if generic {
serde_json::from_value::<LoggedSystemHistoryEntry>(wire).is_err()
} else {
serde_json::from_value::<LoggedHistoryEntry>(wire).is_err()
}
};
let mut missing = wire.clone();
missing["metadata"]
.as_object_mut()
.unwrap()
.remove("input_metadata");
assert!(decode(missing));
let mut absent_policy = wire.clone();
absent_policy["metadata"]["input_metadata"]
.as_object_mut()
.unwrap()
.remove("usage");
assert!(decode(absent_policy));
let mut invalid_role = wire.clone();
invalid_role["item"] =
serde_json::to_value(LoggedItem::from(agen::Item::user_message("not system")))
.unwrap();
invalid_role["metadata"]["input_metadata"]["public"] = serde_json::json!({
"kind": "context", "body": "public system projection", "presentation": {"kind": "context"}
});
assert!(decode(invalid_role));
let mut oversized = wire.clone();
if generic {
oversized["item"]["body"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
} else {
oversized["item"]["content"][0]["text"] = "x"
.repeat(crate::input_metadata::MAX_SYSTEM_INPUT_BYTES + 1)
.into();
}
assert!(decode(oversized));
}
}
#[test]
fn annotated_segment_start_is_restore_visible_without_projecting_metadata() {
let session_id = uuid::Uuid::now_v7();
+563
View File
@@ -0,0 +1,563 @@
//! Validated, Feature-neutral history-input policy and System extensions.
//!
//! The enclosing history annotation owns identity, origin and derivation. None of
//! those descriptive facts selects a use policy here. Model text, independent use
//! policies, private Feature data and an explicit public projection travel as one
//! value. Restoring this value does not load a Feature or invoke domain behavior.
//!
//! Payload validation bounds the decoded value; enclosing log readers must also
//! enforce their record allocation limit before decoding untrusted persisted input.
use agen::Item;
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::PublicSystemItem;
pub const MAX_FEATURE_PAYLOAD_BYTES: usize = 128 * 1024;
pub const MAX_SYSTEM_INPUT_BYTES: usize = 4 * 1024 * 1024;
pub const MAX_SYSTEM_PUBLIC_BYTES: usize = 256 * 1024;
const MAX_SCHEMA_BYTES: usize = 128;
const MAX_JSON_DEPTH: usize = 16;
const MAX_JSON_NODES: usize = 8192;
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum InputMetadataError {
#[error("history input payload schema is invalid")]
InvalidSchema,
#[error("history input payload version must be nonzero")]
InvalidVersion,
#[error("system input resource limit exceeded")]
ResourceLimit,
#[error("history input payload could not be serialized")]
Serialization,
#[error("history input payload schema or version is unsupported by this consumer")]
UnsupportedPayload,
#[error("history input payload does not match the owner's typed contract")]
InvalidTypedPayload,
#[error("provided input is missing explicit metadata")]
MissingMetadata,
#[error("input metadata does not match its logical record")]
InconsistentRecord,
}
/// One private payload owned by a Feature's versioned typed decoder.
/// Unknown valid schemas/versions remain data, never execution instructions.
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(try_from = "PayloadWire")]
pub struct FeatureInputPayload {
schema: String,
version: u32,
value: Value,
}
impl std::fmt::Debug for FeatureInputPayload {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("FeatureInputPayload")
.field("schema", &self.schema)
.field("version", &self.version)
.finish_non_exhaustive()
}
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct PayloadWire {
schema: String,
version: u32,
value: Value,
}
impl TryFrom<PayloadWire> for FeatureInputPayload {
type Error = InputMetadataError;
fn try_from(wire: PayloadWire) -> Result<Self, Self::Error> {
if wire.schema.is_empty()
|| wire.schema.len() > MAX_SCHEMA_BYTES
|| !wire.schema.as_bytes()[0].is_ascii_lowercase()
|| !wire.schema.bytes().all(|byte| {
byte.is_ascii_lowercase() || byte.is_ascii_digit() || b"-_.:/".contains(&byte)
})
{
return Err(InputMetadataError::InvalidSchema);
}
if wire.version == 0 {
return Err(InputMetadataError::InvalidVersion);
}
validate_json(&wire.value, MAX_FEATURE_PAYLOAD_BYTES)?;
Ok(Self {
schema: wire.schema,
version: wire.version,
value: wire.value,
})
}
}
impl FeatureInputPayload {
pub fn encode<T: Serialize>(
schema: impl Into<String>,
version: u32,
value: &T,
) -> Result<Self, InputMetadataError> {
PayloadWire {
schema: schema.into(),
version,
value: serde_json::to_value(value).map_err(|_| InputMetadataError::Serialization)?,
}
.try_into()
}
pub fn schema(&self) -> &str {
&self.schema
}
pub fn version(&self) -> u32 {
self.version
}
/// The owner explicitly selects its contract before decoding. A valid unknown
/// payload can be retained and round-tripped without calling this operation.
pub fn decode<T: DeserializeOwned>(
&self,
schema: &str,
version: u32,
) -> Result<T, InputMetadataError> {
if self.schema != schema || self.version != version {
return Err(InputMetadataError::UnsupportedPayload);
}
serde_json::from_value(self.value.clone())
.map_err(|_| InputMetadataError::InvalidTypedPayload)
}
}
/// Eligibility is supplied independently for recording and Memory evidence.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputEvidenceUse {
Include,
Exclude,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputObservationUse {
Readable,
Hidden,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum InputCompactionUse {
/// Ordinary history eligible for the selected retained tail.
Retain,
/// A full opening summary consumed separately and superseded by compaction.
OpeningSummary,
/// Full historical context for the compactor, not an accumulating retained tail.
ReferenceInput,
}
/// No permissive defaults: malformed/unknown policies cannot turn into eligibility.
/// Public presentation is deliberately not one of these use decisions.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct HistoryInputUsePolicy {
pub recording_material: InputEvidenceUse,
pub memory_evidence: InputEvidenceUse,
pub observation: InputObservationUse,
pub compaction: InputCompactionUse,
}
/// Use policy is independent of logical role. Non-System derived Tool context
/// carries policy only; private Feature payload and public presentation belong
/// to the System envelope and are rejected on other roles at the history boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "MetadataWire")]
pub struct HistoryInputMetadata {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct MetadataWire {
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
}
impl TryFrom<MetadataWire> for HistoryInputMetadata {
type Error = InputMetadataError;
fn try_from(wire: MetadataWire) -> Result<Self, Self::Error> {
if let Some(public) = &wire.public {
// Kind is a descriptive label, not a Feature dispatch or authority key.
if public.kind.is_empty()
|| public.kind.len() > MAX_SCHEMA_BYTES
|| public.kind.chars().any(char::is_control)
{
return Err(InputMetadataError::InvalidSchema);
}
let value =
serde_json::to_value(public).map_err(|_| InputMetadataError::Serialization)?;
validate_json(&value, MAX_SYSTEM_PUBLIC_BYTES)?;
}
Ok(Self {
usage: wire.usage,
feature_payload: wire.feature_payload,
public: wire.public,
})
}
}
impl HistoryInputMetadata {
pub fn new(
usage: HistoryInputUsePolicy,
feature_payload: Option<FeatureInputPayload>,
public: Option<PublicSystemItem>,
) -> Result<Self, InputMetadataError> {
MetadataWire {
usage,
feature_payload,
public,
}
.try_into()
}
pub fn usage(&self) -> HistoryInputUsePolicy {
self.usage
}
pub fn feature_payload(&self) -> Option<&FeatureInputPayload> {
self.feature_payload.as_ref()
}
pub fn public_item(&self) -> Option<&PublicSystemItem> {
self.public.as_ref()
}
pub fn public_use_policy(&self) -> protocol::SessionEntryUsePolicy {
protocol::SessionEntryUsePolicy {
observation_readable: self.usage.observation == InputObservationUse::Readable,
memory_evidence: self.usage.memory_evidence == InputEvidenceUse::Include,
}
}
pub fn public_system_use_policy(&self) -> protocol::SessionEntryUsePolicy {
if self.public.is_some() {
self.public_use_policy()
} else {
protocol::SessionEntryUsePolicy {
observation_readable: false,
memory_evidence: false,
}
}
}
pub fn public_projection(&self) -> PublicSystemItem {
self.public.clone().unwrap_or_else(|| PublicSystemItem {
kind: "input".into(),
body: String::new(),
presentation: crate::PublicSystemPresentation::Ignored,
})
}
}
/// System role is invariant; provenance and use policies are attached data.
/// An absent public projection never implicitly publishes the model body.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(try_from = "InputWire")]
pub struct SystemInputEnvelope {
body: String,
metadata: HistoryInputMetadata,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct InputWire {
body: String,
metadata: HistoryInputMetadata,
}
impl TryFrom<InputWire> for SystemInputEnvelope {
type Error = InputMetadataError;
fn try_from(wire: InputWire) -> Result<Self, Self::Error> {
if wire.body.len() > MAX_SYSTEM_INPUT_BYTES {
return Err(InputMetadataError::ResourceLimit);
}
Ok(Self {
body: wire.body,
metadata: wire.metadata,
})
}
}
impl SystemInputEnvelope {
pub fn new(
body: impl Into<String>,
metadata: HistoryInputMetadata,
) -> Result<Self, InputMetadataError> {
InputWire {
body: body.into(),
metadata,
}
.try_into()
}
pub fn body(&self) -> &str {
&self.body
}
pub fn metadata(&self) -> &HistoryInputMetadata {
&self.metadata
}
pub fn to_history_item(&self) -> Item {
Item::system_message(self.body.clone())
}
pub fn into_parts(self) -> (String, HistoryInputMetadata) {
(self.body, self.metadata)
}
}
fn validate_json(value: &Value, max_bytes: usize) -> Result<(), InputMetadataError> {
fn walk(value: &Value, depth: usize, remaining: &mut usize) -> Result<(), InputMetadataError> {
if depth > MAX_JSON_DEPTH || *remaining == 0 {
return Err(InputMetadataError::ResourceLimit);
}
*remaining -= 1;
match value {
Value::Array(values) => {
for value in values {
walk(value, depth + 1, remaining)?;
}
}
Value::Object(values) => {
for value in values.values() {
walk(value, depth + 1, remaining)?;
}
}
_ => {}
}
Ok(())
}
let mut remaining = MAX_JSON_NODES;
walk(value, 0, &mut remaining)?;
let bytes = serde_json::to_vec(value).map_err(|_| InputMetadataError::Serialization)?;
if bytes.len() > max_bytes {
return Err(InputMetadataError::ResourceLimit);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{LoggedItem, LoggedRole, PublicSystemPresentation};
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct OwnerPayload {
source_digest: String,
omitted: bool,
}
fn reference_policy() -> HistoryInputUsePolicy {
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
}
}
#[test]
fn unknown_valid_payload_round_trips_without_domain_decode_or_public_exposure() {
let payload = FeatureInputPayload::encode(
"future-owner:context",
99,
&OwnerPayload {
source_digest: "private-digest".into(),
omitted: true,
},
)
.unwrap();
let envelope = SystemInputEnvelope::new(
"private model context",
HistoryInputMetadata::new(reference_policy(), Some(payload), None).unwrap(),
)
.unwrap();
let restored: SystemInputEnvelope =
serde_json::from_slice(&serde_json::to_vec(&envelope).unwrap()).unwrap();
assert_eq!(restored.body(), "private model context");
assert!(restored.metadata().public_item().is_none());
let saved_payload = restored.metadata().feature_payload().unwrap();
assert_eq!(saved_payload.schema(), "future-owner:context");
assert_eq!(saved_payload.version(), 99);
assert_eq!(
saved_payload.decode::<OwnerPayload>("known-owner:context", 1),
Err(InputMetadataError::UnsupportedPayload)
);
assert_eq!(
serde_json::to_value(&restored).unwrap(),
serde_json::to_value(&envelope).unwrap()
);
assert!(!format!("{saved_payload:?}").contains("private-digest"));
}
#[test]
fn owner_decodes_typed_payload_only_for_its_explicit_contract() {
let input = OwnerPayload {
source_digest: "rev".into(),
omitted: false,
};
let payload = FeatureInputPayload::encode("owner:context", 1, &input).unwrap();
assert_eq!(
payload.decode::<OwnerPayload>("owner:context", 1).unwrap(),
input
);
let invalid = FeatureInputPayload::encode(
"owner:context",
1,
&serde_json::json!({ "omitted": "not a bool" }),
)
.unwrap();
assert_eq!(
invalid.decode::<OwnerPayload>("owner:context", 1),
Err(InputMetadataError::InvalidTypedPayload)
);
}
#[test]
fn malformed_or_oversized_payload_is_rejected_at_construction_and_restore() {
for (schema, version, value) in [
("", 1, Value::Null),
("invalid schema", 1, Value::Null),
("owner:context", 0, Value::Null),
(
"owner:context",
1,
Value::String("x".repeat(MAX_FEATURE_PAYLOAD_BYTES)),
),
(
"owner:context",
1,
Value::Array(vec![Value::Null; MAX_JSON_NODES]),
),
(
"owner:context",
1,
(0..=MAX_JSON_DEPTH).fold(Value::Null, |inner, _| Value::Array(vec![inner])),
),
] {
assert!(FeatureInputPayload::encode(schema, version, &value).is_err());
assert!(
serde_json::from_value::<FeatureInputPayload>(
serde_json::json!({ "schema": schema, "version": version, "value": value })
)
.is_err()
);
}
}
#[test]
fn policy_is_required_and_unknown_or_malformed_fields_fail_closed() {
let valid = serde_json::to_value(reference_policy()).unwrap();
for field in [
"recording_material",
"memory_evidence",
"observation",
"compaction",
] {
let mut absent = valid.clone();
absent.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<HistoryInputUsePolicy>(absent).is_err());
let mut unknown = valid.clone();
unknown[field] = Value::String("future-allow".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(unknown).is_err());
}
let mut extra = valid;
extra["feature_name"] = Value::String("infer-policy-from-me".into());
assert!(serde_json::from_value::<HistoryInputUsePolicy>(extra).is_err());
}
#[test]
fn feature_name_and_model_body_do_not_select_policies_or_public_projection() {
for (schema, body) in [
("owner:journal", "past context"),
("unrelated:skill", "do not extract this; legitimate wording"),
] {
let public = PublicSystemItem {
kind: "context-label".into(),
body: "explicit public text".into(),
presentation: PublicSystemPresentation::Context,
};
let metadata = HistoryInputMetadata::new(
reference_policy(),
Some(
FeatureInputPayload::encode(
schema,
1,
&OwnerPayload {
source_digest: "private".into(),
omitted: false,
},
)
.unwrap(),
),
Some(public),
)
.unwrap();
let input = SystemInputEnvelope::new(body, metadata).unwrap();
assert_eq!(input.metadata().usage(), reference_policy());
let history: LoggedItem = input.to_history_item().into();
assert!(matches!(
history,
LoggedItem::Message {
role: LoggedRole::System,
..
}
));
assert_eq!(input.to_history_item().as_text(), Some(body));
let public = serde_json::to_string(input.metadata().public_item().unwrap()).unwrap();
assert!(!public.contains("private"));
assert!(!public.contains(schema));
assert!(!public.contains(body));
}
}
#[test]
fn use_policies_remain_independent_of_each_other_and_of_public_display() {
let usage = HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Include,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Readable,
compaction: InputCompactionUse::Retain,
};
let metadata = HistoryInputMetadata::new(usage, None, None).unwrap();
let restored: HistoryInputMetadata =
serde_json::from_value(serde_json::to_value(&metadata).unwrap()).unwrap();
assert_eq!(restored.usage(), usage);
assert!(restored.public_item().is_none());
assert!(restored.feature_payload().is_none());
}
#[test]
fn model_body_and_public_projection_have_independent_resource_limits() {
let metadata = HistoryInputMetadata::new(reference_policy(), None, None).unwrap();
assert!(
SystemInputEnvelope::new("x".repeat(MAX_SYSTEM_INPUT_BYTES + 1), metadata).is_err()
);
let public = PublicSystemItem {
kind: "context".into(),
body: "x".repeat(MAX_SYSTEM_PUBLIC_BYTES),
presentation: PublicSystemPresentation::Context,
};
assert!(HistoryInputMetadata::new(reference_policy(), None, Some(public)).is_err());
let invalid = serde_json::json!({ "body": "small", "metadata": { "public": null, "feature_payload": null } });
assert!(serde_json::from_value::<SystemInputEnvelope>(invalid).is_err());
}
}
+413 -21
View File
@@ -1,12 +1,15 @@
//! Versioned decoder for Session schemas that predate canonical annotated history.
//! Versioned decoder for Session schemas v1-5, including frozen Feature wire.
//!
//! These types are intentionally private to `session-store`. Current writers,
//! replay, and public projections use [`crate::LogEntry`] exclusively; only the
//! Worker Session schema migration is allowed to deserialize these shapes.
//! explicit Session schema migration is allowed to deserialize these shapes.
use agen::llm_client::types::RequestConfig;
use protocol::Segment;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
#[cfg(test)]
mod tests;
use crate::{
LogEntry, LoggedHistoryEntry, LoggedItem, LoggedSessionHistoryEntryId,
@@ -14,6 +17,212 @@ use crate::{
SegmentOrigin, SessionExtension, SessionId, SystemItem,
};
// Frozen v1-4 Feature wire. These types are deliberately not exported and are
// never used by LogEntry/SystemItem's current deserializers. Payload structs
// retain the owner's v1 shape rather than erasing typed fields into arbitrary JSON.
#[derive(Debug, Serialize, Deserialize)]
struct JournalStartupPayload {
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: crate::PromptRenderProvenance,
}
#[derive(Debug, Serialize, Deserialize)]
struct ResidentSummaryPayload {
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SubjectBehaviorPayload {
#[serde(default)]
behavior_md: String,
#[serde(default)]
prompt_provenance: Option<crate::PromptRenderProvenance>,
}
#[derive(Debug, Serialize, Deserialize)]
struct SkillActivationPayload {
name: String,
}
#[derive(Debug, Serialize, Deserialize)]
struct InvocationResultPayload {
result: protocol::FeatureInvocationResult,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyFeatureSystemItem {
JournalStartup {
body: String,
#[serde(flatten)]
payload: JournalStartupPayload,
},
ResidentSummaryRefresh {
body: String,
#[serde(flatten)]
payload: ResidentSummaryPayload,
},
SubjectBehaviorRefresh {
body: String,
#[serde(flatten)]
payload: SubjectBehaviorPayload,
},
SkillActivation {
body: String,
#[serde(flatten)]
payload: SkillActivationPayload,
},
FeatureInvocationResult {
body: String,
#[serde(flatten)]
payload: InvocationResultPayload,
},
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacySystemItem {
Feature(LegacyFeatureSystemItem),
// Includes already-migrated Input records for interrupted migration retry.
Core(SystemItem),
}
impl LegacySystemItem {
fn into_entry(
self,
mut metadata: LoggedSessionHistoryMetadata,
) -> Result<LoggedSystemHistoryEntry, crate::InputMetadataError> {
use crate::InputCompactionUse::{ReferenceInput, Retain};
let item = match self {
Self::Core(item) => item,
Self::Feature(feature) => {
metadata.validate_input_metadata()?;
if metadata.input_metadata.is_some() {
return Err(crate::InputMetadataError::InconsistentRecord);
}
let (body, input_metadata) = match feature {
LegacyFeatureSystemItem::JournalStartup { body, payload } => {
migrated_feature_input(
body,
"journal_startup",
"journal:startup",
payload,
ReferenceInput,
)?
}
LegacyFeatureSystemItem::ResidentSummaryRefresh { body, payload } => {
migrated_feature_input(
body,
"resident_summary_refresh",
"memory:resident-summary",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SubjectBehaviorRefresh { body, payload } => {
migrated_feature_input(
body,
"subject_behavior_refresh",
"subject:behavior",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::SkillActivation { body, payload } => {
migrated_feature_input(
body,
"skill_activation",
"skill:activation",
payload,
Retain,
)?
}
LegacyFeatureSystemItem::FeatureInvocationResult { body, payload } => {
migrated_feature_input(
body,
"feature_invocation_result",
"feature:invocation-result",
payload,
Retain,
)?
}
};
// Identity/origin/derivation are retained, never re-inferred.
metadata.input_metadata = Some(input_metadata);
SystemItem::Input { body }
}
};
let entry = LoggedSystemHistoryEntry { item, metadata };
entry.validate_input_metadata()?;
Ok(entry)
}
}
fn migrated_feature_input<T: Serialize>(
body: String,
label: &str,
schema: &str,
payload: T,
compaction: crate::InputCompactionUse,
) -> Result<(String, crate::HistoryInputMetadata), crate::InputMetadataError> {
let metadata = crate::HistoryInputMetadata::new(
migration_policy(compaction),
Some(crate::FeatureInputPayload::encode(schema, 1, &payload)?),
Some(crate::PublicSystemItem {
kind: label.into(),
body: body.clone(),
presentation: crate::PublicSystemPresentation::Context,
}),
)?;
Ok((body, metadata))
}
/// Annotated old wire used by v3/v4, and by already-converted files when an
/// older marker survived an interrupted migration. No ordinary decoder uses it.
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyAnnotatedLogEntry {
AnnotatedSystemItem {
ts: u64,
entry: LegacySystemHistoryEntry,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct LegacySystemHistoryEntry {
item: LegacySystemItem,
metadata: LoggedSessionHistoryMetadata,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum LegacyAnnotatedOrCurrent {
Annotated(LegacyAnnotatedLogEntry),
Current(LogEntry),
}
impl LegacyAnnotatedOrCurrent {
fn into_current(self) -> Result<LogEntry, crate::InputMetadataError> {
match self {
Self::Annotated(LegacyAnnotatedLogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
}) => Ok(LogEntry::AnnotatedSystemItem {
ts,
entry: entry.item.into_entry(entry.metadata)?,
extensions,
}),
Self::Current(entry) => Ok(entry),
}
}
}
#[derive(Debug, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
enum LegacyHistoryLogEntry {
@@ -44,7 +253,9 @@ enum LegacyHistoryLogEntry {
},
SystemItem {
ts: u64,
item: SystemItem,
item: LegacySystemItem,
#[serde(default)]
extensions: Vec<SessionExtension>,
},
}
@@ -54,7 +265,7 @@ enum LegacyHistoryLogEntry {
#[serde(untagged)]
enum LegacySessionLogEntryV1 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
/// Schema v2 retained the v1 history shapes while adding non-history records.
@@ -64,7 +275,7 @@ enum LegacySessionLogEntryV1 {
#[serde(untagged)]
enum LegacySessionLogEntryV2 {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
pub(crate) fn decode_entry(
@@ -74,28 +285,207 @@ pub(crate) fn decode_entry(
segment_id: SegmentId,
line_index: usize,
) -> Result<LogEntry, serde_json::Error> {
let mut wire: serde_json::Value = serde_json::from_str(line)?;
migrate_legacy_metadata_field(&mut wire)?;
let entry = match schema_version {
1 => match serde_json::from_str::<LegacySessionLogEntryV1>(line)? {
1 => match serde_json::from_value::<LegacySessionLogEntryV1>(wire)? {
LegacySessionLogEntryV1::History(entry) => Entry::History(entry),
LegacySessionLogEntryV1::Current(entry) => Entry::Current(entry),
},
2 => match serde_json::from_str::<LegacySessionLogEntryV2>(line)? {
2 => match serde_json::from_value::<LegacySessionLogEntryV2>(wire)? {
LegacySessionLogEntryV2::History(entry) => Entry::History(entry),
LegacySessionLogEntryV2::Current(entry) => Entry::Current(entry),
},
3 | 4 => Entry::Current(serde_json::from_value(wire)?),
5 => return serde_json::from_value(wire),
_ => unreachable!("legacy decoder called for unsupported schema {schema_version}"),
};
Ok(match entry {
match entry {
Entry::History(entry) => {
canonicalize_history_entry(session_id, segment_id, line_index, entry)
}
Entry::Current(entry) => entry,
})
Entry::Current(entry) => entry.into_current(),
}
.map_err(serde::de::Error::custom)
}
// The v5 System-only metadata field was widened to role-neutral input metadata
// in v6. Touch only known history slots, never arbitrary Feature payload objects.
// Both keys are an ambiguous source, not permission to discard either value.
fn migrate_legacy_metadata_field(record: &mut serde_json::Value) -> Result<(), serde_json::Error> {
fn entry(value: &mut serde_json::Value) -> Result<(), serde_json::Error> {
if let Some(metadata) = value
.get_mut("metadata")
.and_then(serde_json::Value::as_object_mut)
{
if let Some(old) = metadata.remove("system_input") {
if metadata.contains_key("input_metadata") {
return Err(serde::de::Error::custom(
"ambiguous legacy input metadata fields",
));
}
metadata.insert("input_metadata".into(), old);
}
}
Ok(())
}
let kind = record
.get("kind")
.and_then(serde_json::Value::as_str)
.unwrap_or_default()
.to_owned();
if matches!(
kind.as_str(),
"annotated_system_item"
| "annotated_assistant_item"
| "annotated_tool_result"
| "tool_result_corrected"
) {
if let Some(value) = record.get_mut("entry") {
entry(value)?;
}
}
if let Some(history) = record
.get_mut("history")
.and_then(serde_json::Value::as_array_mut)
{
for value in history {
entry(value)?;
}
}
Ok(())
}
/// Schema 3 used DerivedSummary for Host compaction seeds. Only the trusted
/// creation boundary may identify one; arbitrary text and later seed items do not.
/// Include already-upgraded entries so an interrupted multi-file migration resumes.
pub(crate) fn compaction_summary_ids<'a>(
entries: impl Iterator<Item = &'a LogEntry>,
) -> std::collections::HashSet<LoggedSessionHistoryEntryId> {
entries
.filter_map(|entry| {
let LogEntry::AnnotatedSegmentStart {
compacted_from: Some(_),
forked_from: None,
history,
..
} = entry
else {
return None;
};
let first = history.first()?;
(matches!(
first.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
) && matches!(
first.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) && agen::Item::from(first.item.clone())
.as_text()
.is_some_and(|text| text.starts_with("[Compacted context summary]\n\n")))
.then(|| first.metadata.entry_id.clone())
})
.collect()
}
pub(crate) fn migrate_compaction_origins(
entries: &mut [LogEntry],
summaries: &std::collections::HashSet<LoggedSessionHistoryEntryId>,
) {
// Only inherited seed copies can share these identities. Appended input is
// not a construction boundary and must never be reclassified from its text.
for entry in entries {
if let LogEntry::AnnotatedSegmentStart { history, .. } = entry {
for entry in history {
if summaries.contains(&entry.metadata.entry_id)
&& entry.metadata.origin == LoggedSessionHistoryOrigin::DerivedSummary
{
entry.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
}
}
}
}
}
/// Explicit legacy-to-current boundary. No Feature inference belongs in ordinary
/// readers/restoration. Invalid or oversized payloads fail migration; the generic
/// payload/body/public limits are deliberately not relaxed for legacy data.
pub(crate) fn migrate_system_inputs(entry: &mut LogEntry) -> Result<(), crate::InputMetadataError> {
entry.validate_input_metadata()?;
match entry {
// Feature wire has already been converted by the explicit old-schema
// decoder. Ordinary LogEntry can contain only generic Input/core events.
LogEntry::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
LogEntry::AnnotatedSegmentStart { history, .. }
| LogEntry::AnnotatedUserInput { history, .. } => {
for entry in history {
migrate_flattened_system_input(entry)?;
}
}
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. }
| LogEntry::ToolResultCorrected { entry, .. } => {
migrate_flattened_system_input(entry)?;
}
_ => {}
}
entry.validate_input_metadata()
}
fn migration_policy(compaction: crate::InputCompactionUse) -> crate::HistoryInputUsePolicy {
crate::HistoryInputUsePolicy {
recording_material: crate::InputEvidenceUse::Exclude,
memory_evidence: crate::InputEvidenceUse::Exclude,
observation: crate::InputObservationUse::Hidden,
compaction,
}
}
fn migrate_flattened_system_input(
entry: &mut LoggedHistoryEntry,
) -> Result<(), crate::InputMetadataError> {
// Check before supplying legacy policy: Provided without metadata is a
// malformed NEW record, not permission to synthesize a producer policy.
entry.validate_input_metadata()?;
if entry.metadata.input_metadata.is_none()
&& (matches!(
entry.item,
LoggedItem::Message {
role: crate::LoggedRole::System,
..
}
) || matches!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary
| LoggedSessionHistoryOrigin::CompactionSummary
))
{
let compaction = match entry.metadata.origin {
LoggedSessionHistoryOrigin::JournalContext => crate::InputCompactionUse::ReferenceInput,
LoggedSessionHistoryOrigin::CompactionSummary => {
crate::InputCompactionUse::OpeningSummary
}
_ => crate::InputCompactionUse::Retain,
};
// Legacy derived context was excluded from independent evidence. Preserve
// that established decision explicitly, including synthetic Tool roles;
// never guess from Tool names or body text. Payload/public stay absent.
entry.metadata.input_metadata = Some(crate::HistoryInputMetadata::new(
migration_policy(compaction),
None,
None,
)?);
}
entry.validate_input_metadata()
}
enum Entry {
History(LegacyHistoryLogEntry),
Current(LogEntry),
Current(LegacyAnnotatedOrCurrent),
}
fn legacy_metadata(
@@ -108,6 +498,7 @@ fn legacy_metadata(
identity.extend_from_slice(&(line_index as u64).to_be_bytes());
identity.extend_from_slice(&(item_index as u64).to_be_bytes());
LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(format!(
"l-{}",
base64::Engine::encode(&base64::engine::general_purpose::URL_SAFE_NO_PAD, identity)
@@ -122,8 +513,8 @@ fn canonicalize_history_entry(
segment_id: SegmentId,
line_index: usize,
entry: LegacyHistoryLogEntry,
) -> LogEntry {
match entry {
) -> Result<LogEntry, crate::InputMetadataError> {
Ok(match entry {
LegacyHistoryLogEntry::SegmentStart {
ts,
session_id,
@@ -177,13 +568,14 @@ fn canonicalize_history_entry(
metadata: legacy_metadata(segment_id, line_index, 0),
},
},
LegacyHistoryLogEntry::SystemItem { ts, item } => LogEntry::AnnotatedSystemItem {
LegacyHistoryLogEntry::SystemItem {
ts,
entry: LoggedSystemHistoryEntry {
item,
metadata: legacy_metadata(segment_id, line_index, 0),
},
extensions: Vec::new(),
item,
extensions,
} => LogEntry::AnnotatedSystemItem {
ts,
entry: item.into_entry(legacy_metadata(segment_id, line_index, 0))?,
extensions,
},
}
})
}
@@ -0,0 +1,105 @@
use super::*;
mod v5;
fn seed(
origin: LoggedSessionHistoryOrigin,
compacted: bool,
role: crate::LoggedRole,
text: &str,
) -> LogEntry {
LogEntry::AnnotatedSegmentStart {
ts: 1,
session_id: SessionId::now_v7(),
system_prompt: None,
config: Default::default(),
forked_from: None,
compacted_from: compacted.then(|| SegmentOrigin {
segment_id: SegmentId::now_v7(),
at_turn_index: 0,
}),
history: vec![LoggedHistoryEntry {
item: LoggedItem::Message {
role,
content: vec![crate::LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("summary".into()),
origin,
derivation: Some(crate::LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source".into())],
}),
},
}],
}
}
#[test]
fn only_trusted_compaction_seed_is_migrated_and_retries_update_inherited_copies() {
let text = "[Compacted context summary]\n\nsummary";
for (compacted, role, text) in [
(false, crate::LoggedRole::System, text),
(true, crate::LoggedRole::User, text),
(
true,
crate::LoggedRole::System,
"[Session TaskStore snapshot] tasks",
),
] {
let entry = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
compacted,
role,
text,
);
assert!(compaction_summary_ids([&entry].into_iter()).is_empty());
}
for origin in [
LoggedSessionHistoryOrigin::DerivedSummary,
LoggedSessionHistoryOrigin::CompactionSummary,
] {
let creation = seed(origin, true, crate::LoggedRole::System, text);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
text,
);
let LogEntry::AnnotatedSegmentStart { history, .. } = &inherited else {
panic!()
};
let mut pasted = history[0].clone();
pasted.metadata.entry_id = LoggedSessionHistoryEntryId("pasted".into());
let appended = LogEntry::AnnotatedUserInput {
ts: 2,
history: vec![pasted.clone()],
segments: vec![],
extensions: vec![],
};
let mut entries = vec![creation, inherited, appended];
let ids = compaction_summary_ids(entries.iter());
migrate_compaction_origins(&mut entries, &ids);
for entry in &entries[..2] {
let LogEntry::AnnotatedSegmentStart { history, .. } = entry else {
panic!()
};
assert_eq!(
history[0].metadata.origin,
LoggedSessionHistoryOrigin::CompactionSummary
);
assert_eq!(history[0].metadata.entry_id.0, "summary");
assert_eq!(
history[0].metadata.derivation.as_ref().unwrap().sources[0].0,
"source"
);
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[2] else {
panic!()
};
assert_eq!(history, &[pasted]);
let once = serde_json::to_vec(&entries).unwrap();
migrate_compaction_origins(&mut entries, &ids);
assert_eq!(serde_json::to_vec(&entries).unwrap(), once);
}
}
@@ -0,0 +1,3 @@
{"kind":"system_item","ts":1,"item":{"kind":"skill_activation","name":"legacy-skill","body":"legacy skill body"}}
{"kind":"user_input","ts":2,"segments":[{"kind":"text","content":"legacy user body"}],"extensions":[{"domain":"old","payload":{"kept":true}}]}
{"kind":"assistant_item","ts":3,"item":{"kind":"message","role":"assistant","content":[{"kind":"text","text":"legacy reply"}]}}
@@ -0,0 +1,8 @@
{"kind":"annotated_system_item","ts":101,"entry":{"item":{"kind":"journal_startup","body":"journal body\nunchanged","snapshot":{"entries":[{"text":"private snapshot"}]},"snapshot_sha256":"snapshot-digest","prompt_provenance":{"workspace_id":"workspace","source_digest":"source","projection_digest":"projection","logical_name":"journal_startup"}},"metadata":{"entry_id":"journal-id","origin":{"kind":"legacy_unknown"},"derivation":{"sources":["source-id"]}}},"extensions":[{"domain":"migration-fixture","payload":{"kept":true}}]}
{"kind":"annotated_system_item","ts":102,"entry":{"item":{"kind":"resident_summary_refresh","body":"resident body"},"metadata":{"entry_id":"resident-id","origin":{"kind":"derived_summary"}}}}
{"kind":"annotated_system_item","ts":103,"entry":{"item":{"kind":"subject_behavior_refresh","body":"rendered subject body","behavior_md":"exact behavior","prompt_provenance":{"source_digest":"subject-source","projection_digest":"subject-projection","logical_name":"subject_behavior"}},"metadata":{"entry_id":"subject-id","origin":{"kind":"backend_instruction","operation_id":"op"}}}}
{"kind":"annotated_system_item","ts":104,"entry":{"item":{"kind":"skill_activation","body":"skill body","name":"example-skill"},"metadata":{"entry_id":"skill-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":105,"entry":{"item":{"kind":"feature_invocation_result","body":"result body","result":{"invocation_id":"invoke-id","identity":"builtin:test/prepare","status":"outcome_unknown","message":"may have committed","context":null}},"metadata":{"entry_id":"result-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_user_input","ts":106,"segments":[],"history":[{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Journal] prefix is not Feature evidence"}]},"metadata":{"entry_id":"flat-journal","origin":{"kind":"journal_context"},"derivation":{"sources":["journal-id"]}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"summary body"}]},"metadata":{"entry_id":"flat-summary","origin":{"kind":"compaction_summary"}}},{"item":{"kind":"message","role":"system","content":[{"kind":"text","text":"[Skill: example] never infer Feature"}]},"metadata":{"entry_id":"flat-other","origin":{"kind":"legacy_unknown"}}}]}
{"kind":"annotated_system_item","ts":107,"entry":{"item":{"kind":"notification","message":"raw message","body":"wrapped message"},"metadata":{"entry_id":"core-id","origin":{"kind":"legacy_unknown"}}}}
{"kind":"annotated_system_item","ts":108,"entry":{"item":{"kind":"input","body":"already generic body"},"metadata":{"entry_id":"generic-id","origin":{"kind":"provided","producer":"future-owner"},"derivation":{"sources":["source-id"]},"system_input":{"usage":{"recording_material":"include","memory_evidence":"exclude","observation":"readable","compaction":"retain"},"feature_payload":{"schema":"future:unknown","version":99,"value":{"opaque":"retained"}},"public":null}}}}
@@ -0,0 +1,5 @@
{"kind":"annotated_system_item","ts":201,"entry":{"item":{"kind":"input","body":"frozen v5 reference"},"metadata":{"entry_id":"reference-v5","origin":{"kind":"provided","producer":"owner-v5"},"system_input":{"usage":{"recording_material":"exclude","memory_evidence":"exclude","observation":"hidden","compaction":"reference_input"},"feature_payload":{"schema":"future:owner","version":8,"value":{"system_input":"opaque owner field must not be renamed"}},"public":null}}}}
{"kind":"annotated_assistant_item","ts":202,"entry":{"item":{"kind":"tool_call","call_id":"original-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"original-call-entry","origin":{"kind":"model_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_tool_result","ts":203,"entry":{"item":{"kind":"tool_result","call_id":"original-call","summary":"original result"},"metadata":{"entry_id":"original-result-entry","origin":{"kind":"tool_output","worker":{"worker_id":"body"}}}}}
{"kind":"annotated_assistant_item","ts":204,"entry":{"item":{"kind":"tool_call","call_id":"derived-call","name":"ArbitraryTool","arguments":"{}"},"metadata":{"entry_id":"derived-call-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
{"kind":"annotated_tool_result","ts":205,"entry":{"item":{"kind":"tool_result","call_id":"derived-call","summary":"derived retrieval result"},"metadata":{"entry_id":"derived-result-entry","origin":{"kind":"derived_summary"},"derivation":{"sources":["original-call-entry","original-result-entry"]}}}}
@@ -0,0 +1,709 @@
//! Fixed old wire bytes, not fixtures generated by the current serializer.
use super::*;
use crate::{FsStore, InputCompactionUse, Store, WorkerSessionStore};
use serde_json::{Value, json};
use std::{fs, path::Path};
const V4: &str = include_str!("fixtures/v4-system-inputs.jsonl");
const V1: &str = include_str!("fixtures/v1-system-inputs.jsonl");
fn decode_fixture(version: u32, source: &str, segment: SegmentId) -> Vec<LogEntry> {
source
.lines()
.enumerate()
.map(|(index, line)| decode_entry(version, line, SessionId::nil(), segment, index).unwrap())
.collect()
}
fn encode(entries: &[LogEntry]) -> Vec<u8> {
let mut bytes = Vec::new();
for entry in entries {
serde_json::to_writer(&mut bytes, entry).unwrap();
bytes.push(b'\n');
}
bytes
}
fn migrate(entries: &mut [LogEntry]) {
for entry in entries {
migrate_system_inputs(entry).unwrap();
entry.validate_input_metadata().unwrap();
}
}
fn write_worker(root: &Path, version: u32, source: &[u8]) -> (SessionId, SegmentId) {
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
fs::create_dir_all(root.join("segments")).unwrap();
fs::write(
root.join("session.json"),
serde_json::to_vec(&json!({
"schema_version": version, "session_id": session,
}))
.unwrap(),
)
.unwrap();
fs::write(
root.join("segments").join(format!("{segment}.jsonl")),
source,
)
.unwrap();
(session, segment)
}
fn assert_marker(root: &Path, version: u32) {
let manifest: Value =
serde_json::from_slice(&fs::read(root.join("session.json")).unwrap()).unwrap();
assert_eq!(manifest["schema_version"], version);
}
#[test]
fn v4_fixed_wire_preserves_identity_body_payload_public_and_independent_policy() {
let mut entries = decode_fixture(4, V4, SegmentId::nil());
let before = entries.clone();
migrate(&mut entries);
let schemas = [
"journal:startup",
"memory:resident-summary",
"subject:behavior",
"skill:activation",
"feature:invocation-result",
];
let old_wire: Vec<Value> = V4
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
for index in 0..5 {
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = &entries[index]
else {
panic!()
};
let old = &old_wire[index];
let old_item = &old["entry"]["item"];
let old_metadata: LoggedSessionHistoryMetadata =
serde_json::from_value(old["entry"]["metadata"].clone()).unwrap();
let old_ext: Vec<crate::SessionExtension> = old
.get("extensions")
.map(|value| serde_json::from_value(value.clone()).unwrap())
.unwrap_or_default();
assert_eq!(*ts, old["ts"].as_u64().unwrap());
assert_eq!(extensions, &old_ext);
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
old_item["body"].as_str().unwrap()
);
assert_eq!(entry.metadata.entry_id, old_metadata.entry_id);
assert_eq!(entry.metadata.origin, old_metadata.origin);
assert_eq!(entry.metadata.derivation, old_metadata.derivation);
let metadata = entry.metadata.input_metadata.as_ref().unwrap();
let public = metadata.public_item().unwrap();
assert_eq!(public.kind, old_item["kind"].as_str().unwrap());
assert_eq!(public.body, old_item["body"].as_str().unwrap());
assert_eq!(
public.presentation,
crate::PublicSystemPresentation::Context
);
assert_eq!(
metadata.usage(),
migration_policy(if index == 0 {
InputCompactionUse::ReferenceInput
} else {
InputCompactionUse::Retain
})
);
let payload = metadata.feature_payload().unwrap();
assert_eq!(payload.schema(), schemas[index]);
assert_eq!(payload.version(), 1);
let value: Value = payload.decode(schemas[index], 1).unwrap();
// Independently require each frozen owner's typed v1 payload contract.
match index {
0 => {
let _: JournalStartupPayload = payload.decode(schemas[index], 1).unwrap();
}
1 => {
let _: ResidentSummaryPayload = payload.decode(schemas[index], 1).unwrap();
}
2 => {
let _: SubjectBehaviorPayload = payload.decode(schemas[index], 1).unwrap();
}
3 => {
let _: SkillActivationPayload = payload.decode(schemas[index], 1).unwrap();
}
4 => {
let _: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
}
_ => unreachable!(),
}
let mut expected = old_item.clone();
expected.as_object_mut().unwrap().remove("kind");
expected.as_object_mut().unwrap().remove("body");
// Optional provenance is an explicit Option in the v1 payload contract.
if index == 1 {
expected["prompt_provenance"] = Value::Null;
}
if index == 4 {
let result: protocol::FeatureInvocationResult =
serde_json::from_value(old_item["result"].clone()).unwrap();
let typed: InvocationResultPayload = payload.decode(schemas[index], 1).unwrap();
assert_eq!(typed.result, result);
// Preserve the protocol's typed serialization contract, including
// its canonical representation of optional result context.
expected["result"] = serde_json::to_value(result).unwrap();
}
assert_eq!(value, expected);
assert!(value.get("body").is_none());
}
let LogEntry::AnnotatedUserInput { history, .. } = &entries[5] else {
panic!()
};
let LogEntry::AnnotatedUserInput { history: old, .. } = &before[5] else {
panic!()
};
for (index, compaction) in [
InputCompactionUse::ReferenceInput,
InputCompactionUse::OpeningSummary,
InputCompactionUse::Retain,
]
.into_iter()
.enumerate()
{
assert_eq!(history[index].item, old[index].item);
assert_eq!(
history[index].metadata.entry_id,
old[index].metadata.entry_id
);
assert_eq!(history[index].metadata.origin, old[index].metadata.origin);
assert_eq!(
history[index].metadata.derivation,
old[index].metadata.derivation
);
let metadata = history[index].metadata.input_metadata.as_ref().unwrap();
assert_eq!(metadata.usage(), migration_policy(compaction));
assert!(metadata.public_item().is_none());
assert!(metadata.feature_payload().is_none());
}
// Core notification is unchanged; existing generic Input and unknown valid
// owner payload are retained without decoding/reclassifying a Feature.
assert_eq!(encode(&entries[6..]), encode(&before[6..]));
let once = encode(&entries);
migrate(&mut entries);
assert_eq!(encode(&entries), once);
}
#[test]
fn v1_through_v4_worker_migration_reopen_and_mixed_file_retry() {
for version in 1..=4 {
let source = if version <= 2 { V1 } else { V4 };
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), version, source.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
// Simulate an interrupted publication: v5 records precede legacy records
// in one file, and a fully new file coexists with an untouched old file.
let mut converted = decode_fixture(version, source, segment);
migrate(&mut converted);
let mut mixed = encode(&converted[..1]);
mixed.extend_from_slice(source.split_once('\n').unwrap().1.as_bytes());
fs::write(&path, &mixed).unwrap();
let new_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let new_bytes = encode(&converted);
fs::write(&new_path, &new_bytes).unwrap();
let old_path = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(3)));
fs::write(&old_path, source).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
WorkerSessionStore::validate_migration(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), mixed);
assert_eq!(fs::read(&old_path).unwrap(), source.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let store = WorkerSessionStore::new(root.path()).unwrap();
assert_marker(root.path(), 6);
let restored = store.read_all(session, segment).unwrap();
assert_eq!(encode(&restored), new_bytes);
assert_eq!(fs::read(&new_path).unwrap(), new_bytes);
let once = fs::read(&path).unwrap();
let old_once = fs::read(&old_path).unwrap();
drop(store);
let reopened = WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(encode(&reopened.read_all(session, segment).unwrap()), once);
assert_eq!(fs::read(&path).unwrap(), once);
assert_eq!(fs::read(&old_path).unwrap(), old_once);
}
}
#[test]
fn all_segments_preflight_keeps_old_bytes_on_decode_or_conversion_failure() {
let mut oversized: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
oversized["entry"]["item"]["snapshot"] =
json!({"large": "x".repeat(crate::input_metadata::MAX_FEATURE_PAYLOAD_BYTES + 1)});
let mut unknown: Value = serde_json::from_str(V4.lines().next().unwrap()).unwrap();
unknown["entry"]["item"]["kind"] = json!("unsupported_future_feature");
let mut malformed = unknown.clone();
malformed["entry"]["item"] = json!({"kind":"journal_startup", "body":"missing snapshot"});
let mut missing_metadata: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing_metadata["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let mut missing_input = missing_metadata.clone();
missing_input["entry"]["metadata"]["origin"] = json!({"kind":"legacy_unknown"});
let mut invalid_policy: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
invalid_policy["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("unknown");
let mut invalid_payload = invalid_policy.clone();
invalid_payload["entry"]["metadata"]["system_input"]["usage"]["compaction"] = json!("retain");
invalid_payload["entry"]["metadata"]["system_input"]["feature_payload"]["version"] = json!(0);
let mut flat_provided: Value = serde_json::from_str(V4.lines().nth(5).unwrap()).unwrap();
flat_provided["history"][0]["metadata"]["origin"] =
json!({"kind":"provided", "producer":"owner"});
for bad in [
"{broken-json}\n".to_owned(),
format!("{}\n", oversized),
format!("{}\n", unknown),
format!("{}\n", malformed),
format!("{}\n", missing_metadata),
format!("{}\n", invalid_policy),
format!("{}\n", invalid_payload),
format!("{}\n", flat_provided),
format!("{}\n", missing_input),
] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 4, V4.as_bytes());
let first = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&later, &bad).unwrap();
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::validate_migration(root.path()).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read(&later).unwrap(), bad.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn unsupported_schema_preserves_all_old_bytes() {
for version in [0, 7, 99] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V4.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), V4.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
}
}
#[test]
fn standalone_store_uses_same_v1_through_v4_migration_and_reopens() {
// None represents the existing unmarked annotated standalone layout (v3).
for version in [None, Some(1), Some(2), Some(3), Some(4)] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let segment = SegmentId::from_u128(1);
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let source = if version.is_some_and(|v| v <= 2) {
V1
} else {
V4
};
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, source).unwrap();
if let Some(version) = version {
fs::write(dir.join("session-schema.json"), version.to_string()).unwrap();
}
let store = FsStore::new(root.path()).unwrap();
let entries = store.read_all(session, segment).unwrap();
for entry in entries {
entry.validate_input_metadata().unwrap();
}
assert_eq!(
fs::read_to_string(dir.join("session-schema.json"))
.unwrap()
.trim(),
"6"
);
let once = fs::read(&path).unwrap();
drop(store);
FsStore::new(root.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), once);
}
}
#[test]
fn compaction_origin_upgrade_is_retained_only_for_pre_v4_sources() {
for version in [3, 4] {
let root = tempfile::tempdir().unwrap();
let creation = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
true,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let inherited = seed(
LoggedSessionHistoryOrigin::DerivedSummary,
false,
crate::LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
let (session, segment) =
write_worker(root.path(), version, &encode(&[creation, inherited]));
let store = WorkerSessionStore::new(root.path()).unwrap();
for record in store.read_all(session, segment).unwrap() {
let LogEntry::AnnotatedSegmentStart { history, .. } = record else {
panic!()
};
let metadata = &history[0].metadata;
assert_eq!(metadata.entry_id.0, "summary");
assert_eq!(metadata.derivation.as_ref().unwrap().sources[0].0, "source");
assert_eq!(
metadata.origin,
if version == 3 {
LoggedSessionHistoryOrigin::CompactionSummary
} else {
LoggedSessionHistoryOrigin::DerivedSummary
}
);
assert_eq!(
metadata.input_metadata.as_ref().unwrap().usage().compaction,
if version == 3 {
InputCompactionUse::OpeningSummary
} else {
InputCompactionUse::Retain
}
);
}
}
}
#[test]
fn standalone_preflight_and_unsupported_marker_leave_sources_unchanged() {
for version in [4, 99] {
let root = tempfile::tempdir().unwrap();
let session = SessionId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let first = dir.join(format!("{}.jsonl", SegmentId::from_u128(1)));
let later = dir.join(format!("{}.jsonl", SegmentId::from_u128(2)));
fs::write(&first, V4).unwrap();
fs::write(&later, "{broken-json}\n").unwrap();
let marker = version.to_string();
fs::write(dir.join("session-schema.json"), &marker).unwrap();
assert!(FsStore::new(root.path()).is_err());
assert_eq!(fs::read(&first).unwrap(), V4.as_bytes());
assert_eq!(fs::read_to_string(&later).unwrap(), "{broken-json}\n");
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
marker
);
}
}
#[test]
fn current_v5_rejects_all_legacy_feature_kinds_without_rewriting_sources() {
for line in V4.lines().take(5) {
// Current serde, writable open and read-only restore share the same
// current wire contract. None may dispatch to the old Feature decoder.
assert!(serde_json::from_str::<LogEntry>(line).is_err());
let bytes = format!("{line}\n");
let root = tempfile::tempdir().unwrap();
let (session, segment) = write_worker(root.path(), 6, bytes.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let marker = fs::read(root.path().join("session.json")).unwrap();
let readonly = WorkerSessionStore::open_read_only(root.path()).unwrap();
assert!(readonly.read_all_read_only(session, segment).is_err());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(fs::read(root.path().join("session.json")).unwrap(), marker);
let standalone = tempfile::tempdir().unwrap();
let dir = standalone.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
let path = dir.join(format!("{segment}.jsonl"));
fs::write(&path, &bytes).unwrap();
fs::write(dir.join("session-schema.json"), "5").unwrap();
assert!(FsStore::new(standalone.path()).is_err());
assert_eq!(fs::read(&path).unwrap(), bytes.as_bytes());
assert_eq!(
fs::read_to_string(dir.join("session-schema.json")).unwrap(),
"5"
);
}
}
#[test]
fn v1_v2_feature_wire_uses_the_same_typed_payloads_and_stable_legacy_identity() {
let segment = SegmentId::from_u128(42);
for version in [1, 2] {
for (index, line) in V4.lines().take(5).enumerate() {
let annotated: Value = serde_json::from_str(line).unwrap();
let wire = json!({
"kind": "system_item", "ts": annotated["ts"],
"item": annotated["entry"]["item"],
"extensions": [{"domain":"old-owner", "payload":{"kept":true}}],
})
.to_string();
let entry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
let retry = decode_entry(version, &wire, SessionId::nil(), segment, index).unwrap();
assert_eq!(encode(&[entry.clone()]), encode(&[retry]));
let LogEntry::AnnotatedSystemItem {
ts,
entry,
extensions,
} = entry
else {
panic!()
};
assert_eq!(ts, annotated["ts"].as_u64().unwrap());
assert!(matches!(entry.item, SystemItem::Input { .. }));
assert_eq!(
entry.item.history_text(),
annotated["entry"]["item"]["body"].as_str().unwrap()
);
assert_eq!(
entry.metadata.entry_id,
legacy_metadata(segment, index, 0).entry_id
);
assert_eq!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::LegacyUnknown
);
assert!(entry.metadata.derivation.is_none());
assert_eq!(extensions[0].domain, "old-owner");
assert_eq!(extensions[0].payload, json!({"kept":true}));
let LogEntry::AnnotatedSystemItem { entry: v4, .. } =
decode_entry(4, line, SessionId::nil(), segment, index).unwrap()
else {
panic!()
};
assert_eq!(entry.metadata.input_metadata, v4.metadata.input_metadata);
}
}
}
#[test]
fn migration_rejects_malformed_typed_feature_fields_instead_of_retaining_unchecked_json() {
let mut items: Vec<Value> = V4
.lines()
.take(5)
.map(|line| serde_json::from_str::<Value>(line).unwrap()["entry"]["item"].clone())
.collect();
items[0]["snapshot_sha256"] = json!(17);
items[1]["prompt_provenance"] = json!({"logical_name":"missing digests"});
items[2]["behavior_md"] = json!(["not a string"]);
items[3]["name"] = json!({"not":"a name"});
items[4]["result"]["status"] = json!("unsupported_status");
for version in 1..=4 {
for item in &items {
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"malformed-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).is_err());
}
}
}
#[test]
fn frozen_behavior_counter_is_accepted_only_at_the_old_schema_boundary() {
for version in 1..=4 {
let item = json!({
"kind":"subject_behavior_refresh", "body":"exact archived body",
"behavior_revision":7,
});
let wire = if version <= 2 {
json!({"kind":"system_item", "ts":1, "item":item})
} else {
json!({"kind":"annotated_system_item", "ts":1, "entry":{
"item":item, "metadata":{
"entry_id":"behavior-id", "origin":{"kind":"legacy_unknown"},
},
}})
}
.to_string();
assert!(serde_json::from_str::<LogEntry>(&wire).is_err());
let LogEntry::AnnotatedSystemItem { entry, .. } =
decode_entry(version, &wire, SessionId::nil(), SegmentId::nil(), 0).unwrap()
else {
panic!()
};
assert_eq!(entry.item.history_text(), "exact archived body");
let payload = entry
.metadata
.input_metadata
.as_ref()
.unwrap()
.feature_payload()
.unwrap();
let typed: SubjectBehaviorPayload = payload.decode("subject:behavior", 1).unwrap();
assert_eq!(typed.behavior_md, "");
assert!(typed.prompt_provenance.is_none());
assert_eq!(
serde_json::to_value(typed).unwrap(),
json!({
"behavior_md":"", "prompt_provenance":null,
})
);
}
}
#[test]
fn current_schema_rejects_generic_input_without_metadata_instead_of_migrating_it() {
let mut missing: Value = serde_json::from_str(V4.lines().last().unwrap()).unwrap();
missing["entry"]["metadata"]
.as_object_mut()
.unwrap()
.remove("system_input");
let bytes = format!("{}\n", missing);
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), 6, bytes.as_bytes());
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(
fs::read(
root.path()
.join("segments")
.join(format!("{segment}.jsonl"))
)
.unwrap(),
bytes.as_bytes()
);
assert_marker(root.path(), 6);
}
const V5: &str = include_str!("fixtures/v5-role-metadata.jsonl");
#[test]
fn v5_metadata_and_derived_tool_context_upgrade_without_changing_roles_or_original_evidence() {
for standalone in [false, true] {
let root = tempfile::tempdir().unwrap();
let (session, segment) = if standalone {
let session = SessionId::now_v7();
let segment = SegmentId::now_v7();
let dir = root.path().join(session.to_string());
fs::create_dir_all(&dir).unwrap();
fs::write(dir.join("session-schema.json"), b"5").unwrap();
fs::write(dir.join(format!("{segment}.jsonl")), V5).unwrap();
(session, segment)
} else {
write_worker(root.path(), 5, V5.as_bytes())
};
let entries = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
let originals: Vec<Value> = V5
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
assert_eq!(entries.len(), originals.len());
for (index, (entry, old)) in entries.iter().zip(originals).enumerate() {
let wire = serde_json::to_value(entry).unwrap();
assert_eq!(wire["ts"], old["ts"]);
assert_eq!(wire["entry"]["item"], old["entry"]["item"]);
for field in ["entry_id", "origin", "derivation"] {
assert_eq!(
wire["entry"]["metadata"][field],
old["entry"]["metadata"][field]
);
}
let metadata = &wire["entry"]["metadata"];
if index == 0 {
assert_eq!(
metadata["input_metadata"],
old["entry"]["metadata"]["system_input"]
);
assert_eq!(
metadata["input_metadata"]["feature_payload"]["value"]["system_input"],
"opaque owner field must not be renamed"
);
} else if index < 3 {
assert!(
metadata.get("input_metadata").is_none(),
"ordinary model Tool originals must not acquire exclusion policy"
);
} else {
assert_eq!(
metadata["input_metadata"]["usage"],
json!({
"recording_material":"exclude", "memory_evidence":"exclude", "observation":"hidden", "compaction":"retain"
})
);
}
}
let before = encode(&entries);
let reopened = if standalone {
FsStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
} else {
WorkerSessionStore::new(root.path())
.unwrap()
.read_all(session, segment)
.unwrap()
};
assert_eq!(encode(&reopened), before);
}
}
#[test]
fn legacy_metadata_alias_conflict_fails_preflight_without_dropping_either_value() {
let mut bad: Value = serde_json::from_str(V5.lines().next().unwrap()).unwrap();
bad["entry"]["metadata"]["input_metadata"] = bad["entry"]["metadata"]["system_input"].clone();
for version in [4, 5] {
let root = tempfile::tempdir().unwrap();
let (_, segment) = write_worker(root.path(), version, V5.as_bytes());
let path = root
.path()
.join("segments")
.join(format!("{segment}.jsonl"));
let later = root
.path()
.join("segments")
.join(format!("{}.jsonl", SegmentId::from_u128(2)));
let bad_bytes = format!("{bad}\n");
fs::write(&later, &bad_bytes).unwrap();
assert!(WorkerSessionStore::new(root.path()).is_err());
assert_eq!(fs::read(path).unwrap(), V5.as_bytes());
assert_eq!(fs::read(later).unwrap(), bad_bytes.as_bytes());
assert_marker(root.path(), version);
}
}
+7 -1
View File
@@ -41,6 +41,7 @@ pub(crate) fn read_without_atime_bounded(path: &Path, max_bytes: u64) -> io::Res
pub mod event_trace;
pub mod fs_store;
pub mod history;
pub mod input_metadata;
mod legacy_session_log;
pub mod logged_item;
mod paste_artifact;
@@ -65,6 +66,10 @@ pub use history::{
LoggedSessionHistoryMetadata, LoggedSessionHistoryOrigin, LoggedSystemHistoryEntry,
LoggedWorkerSubject,
};
pub use input_metadata::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputMetadataError, InputObservationUse, SystemInputEnvelope,
};
pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged};
pub use paste_artifact::PasteArtifactLimits;
pub use public_index::{
@@ -98,7 +103,8 @@ pub use segment::{
pub use segment_log::{LogEntry, RestoredState, SegmentOrigin, SessionExtension, collect_state};
pub use store::{Store, StoreError};
pub use system_item::{
PromptRenderProvenance, SystemItem, SystemReminder, SystemReminderSource, render_worker_event,
PromptRenderProvenance, PublicSystemItem, PublicSystemPresentation, SystemItem, SystemReminder,
SystemReminderSource, render_worker_event,
};
pub use uploaded_file::{
DEFAULT_MAX_FILES_PER_SUBMISSION, DEFAULT_MAX_SESSION_ARTIFACT_BYTES,
+7 -1
View File
@@ -1296,6 +1296,7 @@ mod tests {
LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.to_owned()),
origin,
derivation: None,
@@ -1537,6 +1538,7 @@ mod tests {
prompt_provenance: None,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("system-item".into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction {
operation_id: None,
@@ -1602,6 +1604,7 @@ mod tests {
segment_id(20),
"completed",
LogEntry::RunCompleted {
entry_id: None,
ts: 4,
interrupted: false,
result: EngineResult::Finished,
@@ -1640,7 +1643,10 @@ mod tests {
(
segment_id(24),
"abandoned",
LogEntry::PausedTurnAbandoned { ts: 4 },
LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 4,
},
),
];
for (segment_id, text, outcome) in outcomes {
+114 -28
View File
@@ -148,15 +148,24 @@ struct SegmentHeader {
file_len: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PublicMaterialUse {
Recording,
MemoryEvidence,
}
/// A fixed material catalog. Opening a new catalog observes newly appended
/// records; reading a body through this catalog verifies its saved digest and
/// byte length again. Missing/corrected/deleted sources are explicit failures.
/// Evidence eligibility never publishes private model text or Feature payload:
/// System inputs also require an explicit public projection for every public use.
pub struct SessionPublicMaterialReader {
session_id: SessionId,
active_segment_id: SegmentId,
layout: Layout,
limits: PublicMaterialLimits,
materials: Vec<Material>,
purpose: PublicMaterialUse,
retention: Option<crate::retention::SessionRetentionStore>,
}
@@ -180,7 +189,13 @@ impl SessionPublicMaterialReader {
active_segment_id: SegmentId,
limits: PublicMaterialLimits,
) -> Result<Self, PublicMaterialError> {
Self::open_worker_guarded(session_root, active_segment_id, limits, None)
Self::open_worker_guarded(
session_root,
active_segment_id,
limits,
None,
PublicMaterialUse::Recording,
)
}
/// Uses the Host's aggregate-external control root for the whole scan and every body read.
@@ -191,7 +206,30 @@ impl SessionPublicMaterialReader {
limits: PublicMaterialLimits,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, PublicMaterialError> {
Self::open_worker_guarded(session_root, active_segment_id, limits, Some(retention))
Self::open_worker_for_use(
session_root,
active_segment_id,
limits,
retention,
PublicMaterialUse::Recording,
)
}
/// Selects the evidence purpose independently of public display and observation.
pub fn open_worker_for_use(
session_root: &Path,
active_segment_id: SegmentId,
limits: PublicMaterialLimits,
retention: crate::retention::SessionRetentionStore,
purpose: PublicMaterialUse,
) -> Result<Self, PublicMaterialError> {
Self::open_worker_guarded(
session_root,
active_segment_id,
limits,
Some(retention),
purpose,
)
}
fn open_worker_guarded(
@@ -199,6 +237,7 @@ impl SessionPublicMaterialReader {
active_segment_id: SegmentId,
limits: PublicMaterialLimits,
retention: Option<crate::retention::SessionRetentionStore>,
purpose: PublicMaterialUse,
) -> Result<Self, PublicMaterialError> {
let store = WorkerSessionStore::open_read_only(session_root)
.map_err(|_| PublicMaterialError::Unavailable)?;
@@ -212,6 +251,7 @@ impl SessionPublicMaterialReader {
Layout::Worker(session_root.into()),
limits,
retention,
purpose,
)
}
@@ -237,6 +277,25 @@ impl SessionPublicMaterialReader {
active_segment_id: SegmentId,
limits: PublicMaterialLimits,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, PublicMaterialError> {
Self::open_fs_for_use(
sessions_root,
session_id,
active_segment_id,
limits,
retention,
PublicMaterialUse::Recording,
)
}
/// Uses retained originals with the independently selected evidence policy.
pub fn open_fs_for_use(
sessions_root: &Path,
session_id: SessionId,
active_segment_id: SegmentId,
limits: PublicMaterialLimits,
retention: crate::retention::SessionRetentionStore,
purpose: PublicMaterialUse,
) -> Result<Self, PublicMaterialError> {
Self::open(
session_id,
@@ -244,6 +303,7 @@ impl SessionPublicMaterialReader {
Layout::Fs(sessions_root.join(session_id.to_string())),
limits,
Some(retention),
purpose,
)
}
@@ -263,6 +323,7 @@ impl SessionPublicMaterialReader {
layout: Layout,
limits: PublicMaterialLimits,
retention: Option<crate::retention::SessionRetentionStore>,
purpose: PublicMaterialUse,
) -> Result<Self, PublicMaterialError> {
let _guard = retention
.as_ref()
@@ -310,11 +371,10 @@ impl SessionPublicMaterialReader {
let source_ids: Option<Vec<String>> = if compact {
let summaries: Vec<_> = history
.iter()
.filter(|e| {
matches!(
e.metadata.origin,
LoggedSessionHistoryOrigin::CompactionSummary
)
.filter(|entry| {
entry.metadata.input_metadata.as_ref().is_some_and(|input| {
input.usage().compaction == crate::InputCompactionUse::OpeningSummary
})
})
.collect();
if summaries.len() != 1 {
@@ -435,23 +495,19 @@ impl SessionPublicMaterialReader {
} => turn = *total_turn_count,
_ => {}
}
let system_history;
let entries: &[LoggedHistoryEntry] = match &record {
LogEntry::AnnotatedSegmentStart { history, .. }
| LogEntry::AnnotatedUserInput { history, .. } => history,
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. }
| LogEntry::ToolResultCorrected { entry, .. } => std::slice::from_ref(entry),
LogEntry::AnnotatedSystemItem { entry, .. } => {
system_history = entry.to_history_entry();
std::slice::from_ref(&system_history)
}
_ => &[],
};
if let LogEntry::AnnotatedSystemItem { entry, .. } = &record {
charge(&mut examined, limits.max_entries)?;
if entry.metadata.entry_id.0.is_empty()
|| !history_seen.insert(entry.metadata.entry_id.0.clone())
{
return Err(PublicMaterialError::Inconsistent);
}
history_ids.push(entry.metadata.entry_id.0.clone());
}
for (history_index, entry) in entries.iter().enumerate() {
charge(&mut examined, limits.max_entries)?;
let id = &entry.metadata.entry_id.0;
@@ -468,7 +524,7 @@ impl SessionPublicMaterialReader {
if in_run && ai_materialized(&entry.item) {
safe = true;
}
let Some((sha256, total_bytes)) = canonical_metadata(entry)? else {
let Some((sha256, total_bytes)) = canonical_metadata(entry, purpose)? else {
continue;
};
let location = Location {
@@ -621,6 +677,7 @@ impl SessionPublicMaterialReader {
layout,
limits,
materials,
purpose,
retention: retention.clone(),
})
}
@@ -660,6 +717,7 @@ impl SessionPublicMaterialReader {
let Some((_, record)) = reader.next()? else {
return Err(PublicMaterialError::Missing);
};
let system_history;
let (body, origin) = if let Some(boundary) = run_boundary(&record) {
if boundary_reference(
&record,
@@ -683,12 +741,16 @@ impl SessionPublicMaterialReader {
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. }
| LogEntry::ToolResultCorrected { entry, .. } => Some(entry),
LogEntry::AnnotatedSystemItem { entry, .. } => {
system_history = entry.to_history_entry();
Some(&system_history)
}
_ => None,
}
.filter(|e| format!("E{}", e.metadata.entry_id.0) == descriptor.identity.entry_ref)
.ok_or(PublicMaterialError::Changed)?;
(
canonical_body(entry).ok_or(PublicMaterialError::Changed)?,
canonical_body(entry, self.purpose).ok_or(PublicMaterialError::Changed)?,
PublicMaterialOrigin::History {
origin: entry.metadata.origin.clone(),
},
@@ -765,7 +827,7 @@ enum CanonicalBody<'a> {
Message {
origin: &'a LoggedSessionHistoryOrigin,
role: LoggedRole,
content: &'a [crate::LoggedContentPart],
content: std::borrow::Cow<'a, [crate::LoggedContentPart]>,
},
ToolCall {
origin: &'a LoggedSessionHistoryOrigin,
@@ -814,7 +876,9 @@ fn boundary_reference(
offset: u64,
) -> String {
let id = match record {
LogEntry::RunYielded { entry_id, .. }
LogEntry::RunCompleted { entry_id, .. }
| LogEntry::PausedTurnAbandoned { entry_id, .. }
| LogEntry::RunYielded { entry_id, .. }
| LogEntry::RunResumed { entry_id, .. }
| LogEntry::RunCancelled { entry_id, .. }
| LogEntry::RunErrored { entry_id, .. } => entry_id.as_ref(),
@@ -826,12 +890,33 @@ fn boundary_reference(
.unwrap_or_else(|| format!("El-run-{session}-{segment}-{offset}"))
}
fn canonical_body(entry: &LoggedHistoryEntry) -> Option<CanonicalBody<'_>> {
if matches!(
entry.metadata.origin,
LoggedSessionHistoryOrigin::DerivedSummary | LoggedSessionHistoryOrigin::CompactionSummary
) {
return None;
fn canonical_body(
entry: &LoggedHistoryEntry,
purpose: PublicMaterialUse,
) -> Option<CanonicalBody<'_>> {
if let Some(input) = &entry.metadata.input_metadata {
let usage = input.usage();
let allowed = match purpose {
PublicMaterialUse::Recording => usage.recording_material,
PublicMaterialUse::MemoryEvidence => usage.memory_evidence,
};
if allowed == crate::InputEvidenceUse::Exclude {
return None;
}
if let LoggedItem::Message {
role: LoggedRole::System,
..
} = &entry.item
{
let public = input.public_item()?;
return Some(CanonicalBody::Message {
origin: &entry.metadata.origin,
role: LoggedRole::System,
content: std::borrow::Cow::Owned(vec![crate::LoggedContentPart::Text {
text: public.body.clone(),
}]),
});
}
}
match &entry.item {
LoggedItem::Message {
@@ -843,7 +928,7 @@ fn canonical_body(entry: &LoggedHistoryEntry) -> Option<CanonicalBody<'_>> {
LoggedItem::Message { role, content } => Some(CanonicalBody::Message {
origin: &entry.metadata.origin,
role: *role,
content,
content: std::borrow::Cow::Borrowed(content),
}),
LoggedItem::ToolCall {
call_id,
@@ -876,8 +961,9 @@ fn canonical_body(entry: &LoggedHistoryEntry) -> Option<CanonicalBody<'_>> {
fn canonical_metadata(
entry: &LoggedHistoryEntry,
purpose: PublicMaterialUse,
) -> Result<Option<(String, u64)>, PublicMaterialError> {
let Some(body) = canonical_body(entry) else {
let Some(body) = canonical_body(entry, purpose) else {
return Ok(None);
};
let mut sink = BodySink::new(0, 0);
+413 -17
View File
@@ -9,6 +9,7 @@ fn entry(id: &str, item: LoggedItem) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::LocalHumanInput,
derivation: None,
@@ -24,6 +25,27 @@ fn message(id: &str, role: LoggedRole, text: &str) -> LoggedHistoryEntry {
},
)
}
fn system_context(
id: &str,
text: &str,
compaction: crate::InputCompactionUse,
) -> LoggedHistoryEntry {
let mut entry = message(id, LoggedRole::System, text);
entry.metadata.input_metadata = Some(
crate::HistoryInputMetadata::new(
crate::HistoryInputUsePolicy {
recording_material: crate::InputEvidenceUse::Exclude,
memory_evidence: crate::InputEvidenceUse::Exclude,
observation: crate::InputObservationUse::Hidden,
compaction,
},
None,
None,
)
.unwrap(),
);
entry
}
fn seed(session: SessionId, history: Vec<LoggedHistoryEntry>) -> LogEntry {
LogEntry::AnnotatedSegmentStart {
ts: 1,
@@ -244,6 +266,7 @@ fn late_run_boundary_is_new_material_without_changing_previously_supplied_bodies
),
(
LogEntry::RunCompleted {
entry_id: None,
ts: 5,
interrupted: false,
result: agen::EngineResult::Finished,
@@ -256,6 +279,7 @@ fn late_run_boundary_is_new_material_without_changing_previously_supplied_bodies
),
(
LogEntry::RunCompleted {
entry_id: None,
ts: 5,
interrupted: true,
result: agen::EngineResult::LimitReached,
@@ -345,7 +369,11 @@ fn yield_resume_cancel_keep_original_boundary_identity_across_compaction() {
.descriptors()
.cloned()
.collect();
let mut summary = message("summary", LoggedRole::User, "must not replace originals");
let mut summary = system_context(
"summary",
"must not replace originals",
crate::InputCompactionUse::OpeningSummary,
);
summary.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
summary.metadata.derivation = Some(crate::LoggedHistoryDerivation {
sources: vec![
@@ -541,8 +569,20 @@ fn reasoning_proves_rollback_safety_but_private_and_derived_items_are_not_materi
let store = FsStore::new(temp.path()).unwrap();
let session = crate::new_session_id();
let segment = crate::new_segment_id();
let mut derived = message("derived", LoggedRole::User, "summary");
let mut derived = system_context("derived", "summary", crate::InputCompactionUse::Retain);
derived.metadata.origin = LoggedSessionHistoryOrigin::DerivedSummary;
let mut journal = system_context(
"journal",
"frozen Journal reference",
crate::InputCompactionUse::ReferenceInput,
);
journal.metadata.origin = LoggedSessionHistoryOrigin::JournalContext;
let mut summary = system_context(
"compaction-summary",
"complementary summary",
crate::InputCompactionUse::OpeningSummary,
);
summary.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
let reasoning = entry(
"reasoning",
LoggedItem::Reasoning {
@@ -559,23 +599,33 @@ fn reasoning_proves_rollback_safety_but_private_and_derived_items_are_not_materi
&[
seed(
session,
vec![derived, message("system", LoggedRole::System, "secret")],
vec![
derived,
journal,
summary,
message("system", LoggedRole::System, "secret"),
],
),
LogEntry::AnnotatedSystemItem {
ts: 1,
entry: crate::LoggedSystemHistoryEntry {
item: crate::SystemItem::JournalStartup {
item: crate::SystemItem::Input {
body: "previous Journal must not become new recording material".into(),
snapshot: serde_json::json!({"journals": []}),
snapshot_sha256: "retained-digest".into(),
prompt_provenance: crate::PromptRenderProvenance {
workspace_id: None,
source_digest: "source".into(),
projection_digest: "projection".into(),
logical_name: "internal.subjektiv_journal_startup".into(),
},
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: Some(
crate::HistoryInputMetadata::new(
crate::HistoryInputUsePolicy {
recording_material: crate::InputEvidenceUse::Exclude,
memory_evidence: crate::InputEvidenceUse::Exclude,
observation: crate::InputObservationUse::Hidden,
compaction: crate::InputCompactionUse::ReferenceInput,
},
None,
None,
)
.unwrap(),
),
entry_id: LoggedSessionHistoryEntryId("startup-journal".into()),
origin: LoggedSessionHistoryOrigin::DerivedSummary,
derivation: None,
@@ -649,6 +699,156 @@ fn fork_clips_parent_suffix_and_maps_inherited_copy_to_original_segment() {
);
}
#[test]
fn schema_v3_compaction_migrates_persistently_and_strict_reader_keeps_original_ids() {
for standalone in [false, true] {
let temp = tempfile::tempdir().unwrap();
let session = crate::new_session_id();
let root = crate::new_segment_id();
let compact = crate::new_segment_id();
let mut summary = message(
"summary",
LoggedRole::System,
"[Compacted context summary]\n\nold summary",
);
summary.metadata.origin = LoggedSessionHistoryOrigin::DerivedSummary;
summary.metadata.derivation = Some(crate::LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("u".into())],
});
let mut start = seed(session, vec![summary.clone()]);
if let LogEntry::AnnotatedSegmentStart { compacted_from, .. } = &mut start {
*compacted_from = Some(SegmentOrigin {
segment_id: root,
at_turn_index: 0,
});
}
let records = [
start,
output("new", "new"),
cancelled(),
LogEntry::RunCompleted {
ts: 6,
entry_id: None,
interrupted: false,
result: agen::EngineResult::Finished,
active_run_turn_count: None,
},
LogEntry::PausedTurnAbandoned {
ts: 7,
entry_id: None,
},
];
let session_root = if standalone {
temp.path().join(session.to_string())
} else {
temp.path().to_path_buf()
};
let paths = if standalone {
session_root.clone()
} else {
session_root.join("segments")
};
std::fs::create_dir_all(&paths).unwrap();
for (id, entries) in [
(
root,
vec![seed(
session,
vec![message("u", LoggedRole::User, "request")],
)],
),
(compact, records.to_vec()),
] {
let bytes: String = entries
.iter()
.map(|entry| format!("{}\n", serde_json::to_string(entry).unwrap()))
.collect();
std::fs::write(paths.join(format!("{id}.jsonl")), bytes).unwrap();
}
if !standalone {
std::fs::write(
session_root.join("session.json"),
serde_json::to_vec(
&serde_json::json!({"schema_version": 3, "session_id": session}),
)
.unwrap(),
)
.unwrap();
}
let open = || {
if standalone {
SessionPublicMaterialReader::open_fs(
temp.path(),
session,
compact,
Default::default(),
)
} else {
SessionPublicMaterialReader::open_worker(&session_root, compact, Default::default())
}
};
assert!(
open().is_err(),
"strict reader must not reinterpret old provenance"
);
let migrate = || {
if standalone {
FsStore::new(temp.path()).unwrap();
} else {
WorkerSessionStore::new(&session_root).unwrap();
}
};
migrate();
let reader = open().unwrap();
assert_eq!(&ids(&reader)[..2], ["Eu", "Enew"]);
let path = paths.join(format!("{compact}.jsonl"));
let bytes = std::fs::read(&path).unwrap();
let entries: Vec<LogEntry> = String::from_utf8(bytes.clone())
.unwrap()
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
let LogEntry::AnnotatedSegmentStart { history, .. } = &entries[0] else {
panic!()
};
summary.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
summary.metadata.input_metadata = Some(
crate::HistoryInputMetadata::new(
crate::HistoryInputUsePolicy {
recording_material: crate::InputEvidenceUse::Exclude,
memory_evidence: crate::InputEvidenceUse::Exclude,
observation: crate::InputObservationUse::Hidden,
compaction: crate::InputCompactionUse::OpeningSummary,
},
None,
None,
)
.unwrap(),
);
assert_eq!(history, &[summary]);
// Run-transition ids synthesized from byte offsets must be frozen before
// the changed summary record shifts every subsequent record offset.
let mut old_offset = 0;
for (index, original) in records.iter().enumerate() {
if index >= 2 {
let mut expected = original.clone();
expected.ensure_legacy_session_entry_id(session, compact, old_offset);
assert_eq!(
serde_json::to_value(&entries[index]).unwrap(),
serde_json::to_value(expected).unwrap()
);
assert_eq!(
ids(&reader)[index],
format!("El-run-{session}-{compact}-{old_offset}")
);
}
old_offset += serde_json::to_vec(original).unwrap().len() as u64 + 1;
}
migrate();
assert_eq!(std::fs::read(&path).unwrap(), bytes);
}
}
#[test]
fn midturn_compaction_uses_exact_source_ids_and_preserves_discarded_originals() {
let temp = tempfile::tempdir().unwrap();
@@ -673,8 +873,14 @@ fn midturn_compaction_uses_exact_source_ids_and_preserves_discarded_originals()
],
)
.unwrap();
let mut summary = message("summary", LoggedRole::User, "never use as original");
summary.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
let mut summary = system_context(
"summary",
"never use as original",
crate::InputCompactionUse::OpeningSummary,
);
summary.metadata.origin = LoggedSessionHistoryOrigin::Provided {
producer: "renamed-summary-owner".into(),
};
summary.metadata.derivation = Some(crate::LoggedHistoryDerivation {
sources: vec![
LoggedSessionHistoryEntryId("u".into()),
@@ -878,7 +1084,11 @@ fn zero_turn_fork_from_seed_and_consecutive_compactions_preserve_original_identi
for index in 0..3 {
let compact = crate::new_segment_id();
let summary_id = format!("summary-{index}");
let mut summary = message(&summary_id, LoggedRole::User, "not original");
let mut summary = system_context(
&summary_id,
"not original",
crate::InputCompactionUse::OpeningSummary,
);
summary.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
summary.metadata.derivation = Some(crate::LoggedHistoryDerivation {
sources: source_ids
@@ -1026,7 +1236,11 @@ fn correction_preserves_original_identity_across_compaction_without_summary_fall
],
)
.unwrap();
let mut summary = message("summary", LoggedRole::User, "not original");
let mut summary = system_context(
"summary",
"not original",
crate::InputCompactionUse::OpeningSummary,
);
summary.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
summary.metadata.derivation = Some(crate::LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("result".into())],
@@ -1074,7 +1288,11 @@ fn compaction_without_exact_derivation_or_with_invalid_turn_is_rejected() {
),
] {
let segment = crate::new_segment_id();
let mut summary = message("summary", LoggedRole::User, "not original");
let mut summary = system_context(
"summary",
"not original",
crate::InputCompactionUse::OpeningSummary,
);
summary.metadata.origin = LoggedSessionHistoryOrigin::CompactionSummary;
summary.metadata.derivation = derivation;
let mut start = seed(session, vec![summary]);
@@ -1091,3 +1309,181 @@ fn compaction_without_exact_derivation_or_with_invalid_turn_is_rejected() {
));
}
}
#[test]
fn generic_system_material_uses_explicit_evidence_purpose_not_producer_or_body() {
use crate::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputObservationUse, LoggedSystemHistoryEntry, SystemInputEnvelope,
};
for worker_layout in [false, true] {
let temp = tempfile::tempdir().unwrap();
let session = crate::new_session_id();
let segment = crate::new_segment_id();
let fork = crate::new_segment_id();
let make =
|producer: &str, recording_material, memory_evidence, body: &str, public: bool| {
LoggedSystemHistoryEntry::from_input(
LoggedSessionHistoryOrigin::Provided {
producer: producer.into(),
},
None,
SystemInputEnvelope::new(
format!("private model text: {body}"),
HistoryInputMetadata::new(
HistoryInputUsePolicy {
recording_material,
memory_evidence,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::Retain,
},
Some(
FeatureInputPayload::encode(
"unknown:future",
99,
&serde_json::json!({"private": "not-public"}),
)
.unwrap(),
),
public.then(|| crate::PublicSystemItem {
kind: "context".into(),
body: body.into(),
presentation: crate::PublicSystemPresentation::Context,
}),
)
.unwrap(),
)
.unwrap(),
)
};
let inputs = [
make(
"same-owner",
InputEvidenceUse::Include,
InputEvidenceUse::Exclude,
"do not extract this; original material",
true,
),
make(
"same-owner",
InputEvidenceUse::Exclude,
InputEvidenceUse::Include,
"memory-only original",
true,
),
make(
"renamed-owner",
InputEvidenceUse::Exclude,
InputEvidenceUse::Exclude,
"reference without warning text",
true,
),
make(
"private-only",
InputEvidenceUse::Include,
InputEvidenceUse::Include,
"must not publish without projection",
false,
),
];
let root = if worker_layout {
temp.path().join("worker")
} else {
temp.path().join(session.to_string())
};
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(&root).unwrap())
} else {
Box::new(FsStore::new(temp.path()).unwrap())
};
store
.create_segment(session, segment, &[seed(session, vec![])])
.unwrap();
for entry in &inputs {
store
.append(
session,
segment,
&LogEntry::AnnotatedSystemItem {
ts: 2,
entry: entry.clone(),
extensions: vec![],
},
)
.unwrap();
}
store
.append(
session,
segment,
&LogEntry::TurnEnd {
ts: 3,
turn_count: 1,
},
)
.unwrap();
let mut fork_seed = seed(
session,
inputs
.iter()
.map(LoggedSystemHistoryEntry::to_history_entry)
.collect(),
);
if let LogEntry::AnnotatedSegmentStart { forked_from, .. } = &mut fork_seed {
*forked_from = Some(crate::SegmentOrigin {
segment_id: segment,
at_turn_index: 1,
});
}
store.create_segment(session, fork, &[fork_seed]).unwrap();
let open = |active, purpose| {
if worker_layout {
SessionPublicMaterialReader::open_worker_for_use(
&root,
active,
Default::default(),
crate::retention::fs_retention(temp.path()),
purpose,
)
.unwrap()
} else {
SessionPublicMaterialReader::open_fs_for_use(
temp.path(),
session,
active,
Default::default(),
crate::retention::fs_retention(temp.path()),
purpose,
)
.unwrap()
}
};
for (purpose, selected, expected_text) in [
(
PublicMaterialUse::Recording,
0,
"do not extract this; original material",
),
(PublicMaterialUse::MemoryEvidence, 1, "memory-only original"),
] {
let original = open(segment, purpose);
let inherited = open(fork, purpose);
assert_eq!(
ids(&original),
vec![format!("E{}", inputs[selected].metadata.entry_id.0)]
);
assert_eq!(ids(&inherited), ids(&original));
let descriptor = inherited.descriptors().next().unwrap();
let body = inherited
.read_content(descriptor, 0, 16 * 1024)
.unwrap()
.content;
let value: serde_json::Value = serde_json::from_str(&body).unwrap();
assert_eq!(value["role"], "system");
assert_eq!(value["content"][0]["text"], expected_text);
assert!(!body.contains("not-public"));
assert!(!body.contains("private model text"));
assert!(!body.contains("unknown:future"));
}
}
}
+76 -21
View File
@@ -27,7 +27,7 @@ use thiserror::Error;
use crate::{
LogEntry, LoggedContentPart, LoggedHistoryEntry, LoggedItem, LoggedRole,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError, SystemItem,
LoggedSessionHistoryOrigin, SegmentId, SegmentOrigin, SessionId, StoreError,
WorkerAggregateStore, WorkerSessionStore, WorkerStoreError,
};
@@ -1165,7 +1165,7 @@ fn capture_tool_correction(
let LogEntry::ToolResultCorrected { entry, .. } = record else {
return false;
};
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
// We read backwards: the first correction seen is the latest one.
corrections
.entry(entry.metadata.entry_id.0.clone())
@@ -1600,12 +1600,12 @@ pub(crate) fn project_session_snapshot_for_segment(
} => extend_history(&mut entries, history, Some(segments), *ts),
LogEntry::AnnotatedAssistantItem { ts, entry }
| LogEntry::AnnotatedToolResult { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
entries.push(history_entry(entry, *ts, data));
}
}
LogEntry::ToolResultCorrected { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(data) = project_history_item(entry) {
if let Some(existing) = entries
.iter_mut()
.find(|existing| existing.entry_id == entry.metadata.entry_id.0)
@@ -1618,11 +1618,16 @@ pub(crate) fn project_session_snapshot_for_segment(
}
}
LogEntry::AnnotatedSystemItem { ts, entry, .. } => entries.push(system_entry(
&entry.item,
&entry.public_item(),
entry.metadata.entry_id.0.clone(),
*ts,
provenance(&entry.metadata.origin),
derivation_ids(entry),
entry
.metadata
.input_metadata
.as_ref()
.map(crate::HistoryInputMetadata::public_system_use_policy),
)),
LogEntry::RunYielded {
ts,
@@ -1705,6 +1710,7 @@ pub(crate) fn project_session_snapshot_for_segment(
interrupted,
result,
active_run_turn_count: persisted_count,
..
} => {
if *interrupted && matches!(result, agen::EngineResult::Yielded) {
let count = (*persisted_count)
@@ -1774,7 +1780,7 @@ fn extend_history(
segments: input_segments.cloned().unwrap_or_default(),
}
} else {
let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) else {
let Some(data) = project_history_item(entry) else {
continue;
};
data
@@ -1789,6 +1795,19 @@ fn history_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: entry.metadata.input_metadata.as_ref().map(|input| {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
input.public_system_use_policy()
} else {
input.public_use_policy()
}
}),
entry_id: entry.metadata.entry_id.0.clone(),
timestamp,
provenance: provenance(&entry.metadata.origin),
@@ -1832,6 +1851,7 @@ fn run_transition_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: persisted_entry_id
.map(|entry_id| entry_id.0.clone())
.unwrap_or_else(|| legacy_entry_id(session_key, segment_id, log_index, 0)),
@@ -1851,6 +1871,7 @@ fn legacy_entry(
data: SessionSnapshotEntryData,
) -> SessionSnapshotEntry {
SessionSnapshotEntry {
use_policy: None,
entry_id: legacy_entry_id(session_key, segment_id, log_index, item_index),
timestamp,
provenance: SessionEntryProvenance::LegacyUnknown,
@@ -1883,12 +1904,12 @@ fn provenance(origin: &LoggedSessionHistoryOrigin) -> SessionEntryProvenance {
LoggedSessionHistoryOrigin::FlowInstruction { .. } => {
SessionEntryProvenance::FlowInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. } => {
SessionEntryProvenance::BackendInstruction
}
LoggedSessionHistoryOrigin::BackendInstruction { .. }
| LoggedSessionHistoryOrigin::Provided { .. } => SessionEntryProvenance::BackendInstruction,
LoggedSessionHistoryOrigin::ModelOutput { .. } => SessionEntryProvenance::ModelOutput,
LoggedSessionHistoryOrigin::ToolOutput { .. } => SessionEntryProvenance::ToolOutput,
LoggedSessionHistoryOrigin::CompactionSummary
| LoggedSessionHistoryOrigin::JournalContext
| LoggedSessionHistoryOrigin::DerivedSummary => SessionEntryProvenance::DerivedSummary,
LoggedSessionHistoryOrigin::LegacyUnknown => SessionEntryProvenance::LegacyUnknown,
}
@@ -1912,6 +1933,35 @@ pub fn session_tool_attachment_id(
URL_SAFE_NO_PAD.encode(hash.finalize())
}
fn project_history_item(entry: &LoggedHistoryEntry) -> Option<SessionSnapshotEntryData> {
if let Some(input) = &entry.metadata.input_metadata {
if matches!(
entry.item,
LoggedItem::Message {
role: LoggedRole::System,
..
}
) {
let public = input.public_projection();
return Some(SessionSnapshotEntryData::SystemItem {
item_kind: public.kind.clone(),
content: public.body.clone(),
data: Some(
serde_json::to_value(public).expect("public system input is serializable"),
),
});
}
let usage = input.usage();
if usage.recording_material == crate::InputEvidenceUse::Exclude
&& usage.memory_evidence == crate::InputEvidenceUse::Exclude
&& usage.observation == crate::InputObservationUse::Hidden
{
return None;
}
}
project_item(&entry.metadata.entry_id.0, &entry.item)
}
fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntryData> {
match item {
LoggedItem::Message { role, content } => {
@@ -1984,30 +2034,26 @@ fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntr
}
fn system_entry(
item: &SystemItem,
item: &crate::PublicSystemItem,
entry_id: String,
timestamp: u64,
provenance: SessionEntryProvenance,
derived_from: Vec<String>,
use_policy: Option<protocol::SessionEntryUsePolicy>,
) -> SessionSnapshotEntry {
let mut data = serde_json::to_value(item).ok();
if let Some(serde_json::Value::Object(object)) = data.as_mut() {
object.remove("prompt_provenance");
}
let item_kind = data
.as_ref()
.and_then(|value| value.get("kind"))
.and_then(serde_json::Value::as_str)
.unwrap_or("system_item")
.to_owned();
let public = item.clone();
let data = Some(serde_json::to_value(&public).expect("public system input is serializable"));
let item_kind = public.kind;
let content = public.body;
SessionSnapshotEntry {
use_policy,
entry_id,
timestamp,
provenance,
derived_from,
data: SessionSnapshotEntryData::SystemItem {
item_kind,
content: item.history_text(),
content,
data,
},
}
@@ -2037,6 +2083,7 @@ mod tests {
fn complete(ts: u64) -> LogEntry {
LogEntry::RunCompleted {
entry_id: None,
ts,
interrupted: false,
result: agen::EngineResult::Finished,
@@ -2288,6 +2335,7 @@ mod tests {
content: vec![LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(entry_id.into()),
origin,
derivation: None,
@@ -3586,6 +3634,7 @@ mod tests {
.map(|item| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -3640,6 +3689,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id,
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".into(),
@@ -3709,6 +3759,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::FlowInstruction {
selector: "builtin:coder-review".into(),
@@ -3728,6 +3779,7 @@ mod tests {
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: user_entry_id.clone(),
origin: origin.clone(),
derivation: Some(LoggedHistoryDerivation {
@@ -3758,6 +3810,7 @@ mod tests {
fn annotated_projection_preserves_identity_and_provenance() {
let session_id = crate::new_session_id();
let metadata = LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::ModelOutput {
worker: LoggedWorkerSubject {
@@ -4320,6 +4373,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("ancestor-image".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4414,6 +4468,7 @@ mod tests {
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("tool-images".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -103,6 +103,7 @@ fn finish(
stats.insert(
anchor.clone(),
SessionSnapshotEntry {
use_policy: None,
entry_id: format!("run-stats:{anchor}"),
timestamp: ended_at,
provenance: SessionEntryProvenance::LegacyUnknown,
+1
View File
@@ -329,6 +329,7 @@ pub fn save_run_completed(
session_id,
segment_id,
LogEntry::RunCompleted {
entry_id: None,
ts: segment_log::now_millis(),
interrupted,
result,
+43 -6
View File
@@ -122,6 +122,8 @@ pub enum LogEntry {
/// Replay restores both interruption state and any resumable logical-run
/// turn budget.
RunCompleted {
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
ts: u64,
interrupted: bool,
result: EngineResult,
@@ -194,7 +196,11 @@ pub enum LogEntry {
/// A paused interrupted turn was explicitly abandoned without calling
/// `run()` or `resume()` again. Replay clears the interrupted marker so
/// the restored Worker is idle and future user input starts a normal new turn.
PausedTurnAbandoned { ts: u64 },
PausedTurnAbandoned {
ts: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
},
/// `RequestConfig` changed.
ConfigChanged { ts: u64, config: RequestConfig },
@@ -248,12 +254,31 @@ pub struct SegmentOrigin {
}
impl LogEntry {
/// Validate generic System records before a Store publishes any bytes.
pub fn validate_input_metadata(&self) -> Result<(), crate::InputMetadataError> {
match self {
Self::AnnotatedSegmentStart { history, .. }
| Self::AnnotatedUserInput { history, .. } => {
for entry in history {
entry.validate_input_metadata()?;
}
}
Self::AnnotatedAssistantItem { entry, .. }
| Self::AnnotatedToolResult { entry, .. }
| Self::ToolResultCorrected { entry, .. } => entry.validate_input_metadata()?,
Self::AnnotatedSystemItem { entry, .. } => entry.validate_input_metadata()?,
_ => {}
}
Ok(())
}
/// Allocate the durable public identity for a newly committed Run
/// transition. Calling this repeatedly is idempotent so the exact value
/// written to storage is also the value published to live subscribers.
pub fn ensure_session_entry_id(&mut self) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
Self::RunCompleted { entry_id, .. }
| Self::PausedTurnAbandoned { entry_id, .. }
| Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
@@ -275,7 +300,9 @@ impl LogEntry {
record_start: u64,
) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
Self::RunCompleted { entry_id, .. }
| Self::PausedTurnAbandoned { entry_id, .. }
| Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
@@ -527,6 +554,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -586,6 +614,7 @@ mod tests {
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 3200,
interrupted: false,
result: EngineResult::Finished,
@@ -893,12 +922,16 @@ mod tests {
compacted_from: None,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 100,
interrupted: true,
result: EngineResult::Paused,
active_run_turn_count: Some(1),
},
LogEntry::PausedTurnAbandoned { ts: 200 },
LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 200,
},
]);
assert!(!state.last_run_interrupted);
assert_eq!(state.active_run_turn_count, None);
@@ -1056,6 +1089,7 @@ mod tests {
turn_count: 2,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 300,
interrupted: true,
result: EngineResult::LimitReached,
@@ -1069,13 +1103,16 @@ mod tests {
#[test]
fn paused_turn_abandoned_entry_round_trip_via_json() {
let entry = LogEntry::PausedTurnAbandoned { ts: 12345 };
let entry = LogEntry::PausedTurnAbandoned {
entry_id: None,
ts: 12345,
};
let json = serde_json::to_string(&entry).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["kind"], "paused_turn_abandoned");
let decoded: LogEntry = serde_json::from_str(&json).unwrap();
match decoded {
LogEntry::PausedTurnAbandoned { ts } => assert_eq!(ts, 12345),
LogEntry::PausedTurnAbandoned { ts, .. } => assert_eq!(ts, 12345),
other => panic!("expected PausedTurnAbandoned, got {other:?}"),
}
}
+3
View File
@@ -27,6 +27,9 @@ pub enum StoreError {
#[error("serialization error: {0}")]
Serde(#[from] serde_json::Error),
#[error("invalid history input metadata: {0}")]
InvalidInputMetadata(#[from] crate::InputMetadataError),
#[error("segment not found: {0}")]
NotFound(SegmentId),
+92 -103
View File
@@ -86,20 +86,16 @@ pub struct PromptRenderProvenance {
/// `notify_wrapper` template) re-shape the live rendering on a later
/// resume.
///
/// New variants get added here as fresh injection kinds come online
/// (e.g. `TaskReminder`). The `kind` JSON tag is the snake_case form of
/// the variant name.
/// Feature-owned inputs use `Input` plus explicit annotated metadata; adding a
/// Feature must not extend this enum or ordinary replay. Only core events have
/// dedicated variants. Pre-v5 Feature kinds exist exclusively in the migration
/// decoder, not in this current wire contract.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum SystemItem {
/// One-time derived Journal reference context. The bounded Host snapshot is
/// retained as JSON independently of its rendered body and prompt projection.
JournalStartup {
body: String,
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: PromptRenderProvenance,
},
/// Generic System body; private metadata and public presentation are supplied
/// independently by the producer in the same annotated history record.
Input { body: String },
/// Free-form notification sent in by an external caller via
/// `Method::Notify`. `message` is the raw caller-supplied text;
/// `body` is the wrapped LLM-context form (Worker renders it via
@@ -122,14 +118,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Result of an explicitly selected chat Feature invocation. The typed
/// result is retained for clients; `body` is the exact bounded context made
/// visible to the Worker model.
FeatureInvocationResult {
result: protocol::FeatureInvocationResult,
body: String,
},
/// `@<path>` file reference resolution. `body` is the rendered
/// LLM-context text (`[File: <path>]\n…` for regular files,
/// `[Dir: <path>]\n…` for directory listings, possibly with a
@@ -137,10 +125,6 @@ pub enum SystemItem {
/// byte-identical to what was sent.
FileAttachment { path: String, body: String },
/// Explicit Agent Skill activation. `body` is the exact LLM-facing
/// Skill text committed before any subsequent LLM request can use it.
SkillActivation { name: String, body: String },
/// Historical persisted Knowledge reference resolution. Knowledge is no
/// longer active, so restored sessions ignore this item instead of replaying
/// archived record text into model context.
@@ -164,29 +148,6 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Restore-boundary refresh of the current resident Memory projection.
/// The exact rendered body is retained so replay is byte-identical; a
/// refresh with no ready summary explicitly supersedes older resident
/// context without deleting or rewriting history.
ResidentSummaryRefresh {
body: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// A Host-fetched change to user-managed Subject behavior. Each changed
/// body is appended rather than rewriting prior prompt/session content.
SubjectBehaviorRefresh {
body: String,
/// Exact user-managed behavior acknowledged after durable commit. Old
/// logs omit this value; an empty value is not an acknowledgement of a
/// nonempty behavior and does not replace their stored rendered body.
#[serde(default)]
behavior_md: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Synthetic note inserted after an interrupted turn before the next
/// user input. `body` is the exact LLM-context text explaining that the
/// previous turn was cut short.
@@ -197,24 +158,67 @@ pub enum SystemItem {
},
}
/// Allowlisted presentation data shared by live events and retained snapshots.
/// Private Feature payloads and render provenance never cross this boundary.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct PublicSystemItem {
pub kind: String,
pub body: String,
pub presentation: PublicSystemPresentation,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum PublicSystemPresentation {
Notification { message: String },
WorkerEvent { event: WorkerEvent },
Context,
TaskReminder,
Ignored,
}
impl SystemItem {
pub fn public_item(&self) -> PublicSystemItem {
let presentation = match self {
Self::Notification { message, .. } => PublicSystemPresentation::Notification {
message: message.clone(),
},
Self::WorkerEvent { event, .. } => PublicSystemPresentation::WorkerEvent {
event: event.clone(),
},
Self::TaskReminder { .. } => PublicSystemPresentation::TaskReminder,
Self::Input { .. }
| Self::LegacyKnowledgeIgnored { .. }
| Self::LegacyIgnored { .. } => PublicSystemPresentation::Ignored,
Self::FileAttachment { .. } | Self::Interrupt { .. } => {
PublicSystemPresentation::Context
}
};
PublicSystemItem {
kind: self.kind_label().to_owned(),
body: if matches!(presentation, PublicSystemPresentation::Ignored) {
String::new()
} else {
self.history_text()
},
presentation,
}
}
/// Free-text body the LLM sees inside its `role:system` message
/// for this item. Returns the variant's stored `body` verbatim.
pub fn history_text(&self) -> String {
match self {
SystemItem::JournalStartup { body, .. } => body.clone(),
SystemItem::Input { body } => body.clone(),
SystemItem::Notification { body, .. } => body.clone(),
SystemItem::WorkerEvent { body, .. } => body.clone(),
SystemItem::FeatureInvocationResult { body, .. } => body.clone(),
SystemItem::FileAttachment { body, .. } => body.clone(),
SystemItem::SkillActivation { body, .. } => body.clone(),
SystemItem::LegacyKnowledgeIgnored { .. } => String::new(),
SystemItem::LegacyIgnored { slug } => {
format!("Ignored legacy procedure item: /{slug}")
}
SystemItem::TaskReminder { body, .. } => body.clone(),
SystemItem::ResidentSummaryRefresh { body, .. } => body.clone(),
SystemItem::SubjectBehaviorRefresh { body, .. } => body.clone(),
SystemItem::Interrupt { body, .. } => body.clone(),
}
}
@@ -229,17 +233,13 @@ impl SystemItem {
/// wire — keep flexible.
pub fn kind_label(&self) -> &'static str {
match self {
SystemItem::JournalStartup { .. } => "journal_startup",
SystemItem::Input { .. } => "input",
SystemItem::Notification { .. } => "notification",
SystemItem::WorkerEvent { .. } => "worker_event",
SystemItem::FeatureInvocationResult { .. } => "feature_invocation_result",
SystemItem::FileAttachment { .. } => "file_attachment",
SystemItem::SkillActivation { .. } => "skill_activation",
SystemItem::LegacyKnowledgeIgnored { .. } => "legacy_knowledge_ignored",
SystemItem::LegacyIgnored { .. } => "legacy_ignored",
SystemItem::TaskReminder { .. } => "task_reminder",
SystemItem::ResidentSummaryRefresh { .. } => "resident_summary_refresh",
SystemItem::SubjectBehaviorRefresh { .. } => "subject_behavior_refresh",
SystemItem::Interrupt { .. } => "interrupt",
}
}
@@ -297,33 +297,45 @@ mod tests {
}
#[test]
fn subject_behavior_refresh_round_trips_with_exact_body() {
let item = SystemItem::SubjectBehaviorRefresh {
body: "current Subject context".to_string(),
behavior_md: "exact behavior".to_string(),
prompt_provenance: None,
};
let raw = serde_json::to_string(&item).unwrap();
let parsed: SystemItem = serde_json::from_str(&raw).unwrap();
assert_eq!(parsed.history_text(), "current Subject context");
assert!(
matches!(parsed, SystemItem::SubjectBehaviorRefresh { behavior_md, .. } if behavior_md == "exact behavior")
);
fn current_system_item_rejects_frozen_feature_kinds() {
// Even well-formed old payloads cannot enter normal v5 decode/replay.
for line in include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl")
.lines()
.take(5)
{
let record: serde_json::Value = serde_json::from_str(line).unwrap();
assert!(serde_json::from_value::<SystemItem>(record["entry"]["item"].clone()).is_err());
assert!(
serde_json::from_value::<crate::LoggedSystemHistoryEntry>(record["entry"].clone())
.is_err()
);
assert!(serde_json::from_str::<crate::LogEntry>(line).is_err());
}
}
#[test]
fn frozen_subject_behavior_log_reads_old_counter_without_republishing_it() {
let old = r#"{"kind":"subject_behavior_refresh","body":"exact archived body","behavior_revision":7}"#;
let item: SystemItem = serde_json::from_str(old).unwrap();
assert_eq!(item.history_text(), "exact archived body");
assert_eq!(
serde_json::to_value(item).unwrap(),
serde_json::json!({
"kind": "subject_behavior_refresh",
"body": "exact archived body",
"behavior_md": ""
})
);
fn public_system_input_decoder_rejects_private_fields_and_unknown_presentations() {
let valid = serde_json::json!({
"kind": "new-feature-label",
"body": "explicit public content",
"presentation": { "kind": "context" },
});
let public: PublicSystemItem = serde_json::from_value(valid.clone()).unwrap();
assert_eq!(public.body, "explicit public content");
for private_field in [
"payload",
"prompt_provenance",
"policy",
"snapshot",
"behavior_md",
] {
let mut malformed = valid.clone();
malformed[private_field] = serde_json::json!({ "private": true });
assert!(serde_json::from_value::<PublicSystemItem>(malformed).is_err());
}
let mut unknown = valid;
unknown["presentation"] = serde_json::json!({ "kind": "execute_feature_payload" });
assert!(serde_json::from_value::<PublicSystemItem>(unknown).is_err());
}
#[test]
@@ -419,29 +431,6 @@ mod tests {
}
}
#[test]
fn invocation_result_retains_typed_outcome_and_exact_context() {
let result = protocol::FeatureInvocationResult {
invocation_id: "stable-invoke".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
status: protocol::FeatureInvocationStatus::OutcomeUnknown,
message: "operation may have committed".into(),
context: None,
};
let body = "Outcome unknown; do not retry automatically";
let item = SystemItem::FeatureInvocationResult {
result: result.clone(),
body: body.into(),
};
let restored: SystemItem =
serde_json::from_str(&serde_json::to_string(&item).unwrap()).unwrap();
assert_eq!(restored.history_text(), body);
assert_eq!(restored.kind_label(), "feature_invocation_result");
assert!(
matches!(restored, SystemItem::FeatureInvocationResult { result: saved, .. } if saved == result)
);
}
#[test]
fn round_trip_worker_event() {
let item = SystemItem::WorkerEvent {
+21 -43
View File
@@ -210,18 +210,6 @@ impl WorkerMetadata {
}
}
fn replace_active_session(metadata: &mut WorkerMetadata, active: Option<WorkerActiveSegmentRef>) {
if let Some(session_id) = active.as_ref().map(|active| active.session_id)
&& metadata
.subjektiv_session_attribution
.as_ref()
.is_some_and(|state| state.session_id() != session_id)
{
metadata.subjektiv_session_attribution = None;
}
metadata.active = active;
}
/// Sync persistence backend for Worker metadata.
pub trait WorkerMetadataStore: Send + Sync {
/// Create or replace metadata for its `worker_name` key.
@@ -297,7 +285,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -317,7 +305,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_id: Option<String>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -334,7 +322,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
replace_active_session(metadata, active);
metadata.active = active;
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_root) = workspace_root {
metadata.workspace_root = Some(workspace_root);
@@ -342,18 +330,6 @@ pub trait WorkerMetadataStore: Send + Sync {
})
}
/// Set the Host-owned Subjektiv Session attribution lifecycle state while
/// preserving the active Session pointer and every unrelated field.
fn set_subjektiv_session_attribution(
&self,
worker_name: &str,
state: SubjektivSessionAttributionState,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.subjektiv_session_attribution = Some(state);
})
}
/// Set spawned-child registry state while preserving active pointer and manifest snapshot.
fn set_spawned_children(
&self,
@@ -1087,13 +1063,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
});
})
.unwrap();
store
.set_active(
@@ -1117,7 +1093,7 @@ mod tests {
}
#[test]
fn new_active_session_clears_prior_session_attribution_state() {
fn generic_active_update_does_not_interpret_host_attribution() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let old_session_id = crate::new_session_id();
@@ -1129,13 +1105,13 @@ mod tests {
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
},
)
.update_by_name("subject-worker", |metadata| {
metadata.subjektiv_session_attribution =
Some(SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
});
})
.unwrap();
store
@@ -1151,7 +1127,9 @@ mod tests {
restored.active,
Some(WorkerActiveSegmentRef::pending_segment(new_session_id))
);
assert!(restored.subjektiv_session_attribution.is_none());
assert!(
matches!(restored.subjektiv_session_attribution, Some(SubjektivSessionAttributionState::Confirmed { session_id, .. }) if session_id == old_session_id)
);
}
#[test]
+284 -23
View File
@@ -36,7 +36,10 @@ use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::SystemTime;
const SESSION_SCHEMA_VERSION: u32 = 3;
const SESSION_SCHEMA_VERSION: u32 = 6;
const SYSTEM_INPUT_SESSION_SCHEMA_VERSION: u32 = 5;
const COMPACTION_SESSION_SCHEMA_VERSION: u32 = 4;
const ANNOTATED_SESSION_SCHEMA_VERSION: u32 = 3;
const PREVIOUS_SESSION_SCHEMA_VERSION: u32 = 2;
const LEGACY_SESSION_SCHEMA_VERSION: u32 = 1;
const SESSION_FILE: &str = "session.json";
@@ -203,7 +206,15 @@ impl WorkerSessionStore {
/// Open the Session store rooted at `<worker-aggregate>/session`.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
Self::new_with_migration_writer(root.into(), atomic_write_bytes)
}
// Keep fault injection at the storage publication boundary, without global
// flags or a public alternate restore API. Production always uses atomic writes.
fn new_with_migration_writer(
root: PathBuf,
write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<Self, StoreError> {
fs::create_dir_all(root.join(SEGMENTS_DIR))?;
let session_id = match fs::read(root.join(SESSION_FILE)) {
Ok(bytes) => {
@@ -212,11 +223,16 @@ impl WorkerSessionStore {
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_logs(&root)?;
}
PREVIOUS_SESSION_SCHEMA_VERSION | LEGACY_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_v3(
&root,
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths(&root)?,
manifest.session_id,
manifest.schema_version,
write_segment,
)?;
manifest.schema_version = SESSION_SCHEMA_VERSION;
atomic_write_json(&root.join(SESSION_FILE), &manifest)?;
@@ -243,6 +259,33 @@ impl WorkerSessionStore {
})
}
/// Validate an existing Session's conversion without publishing any bytes.
/// Hosts can isolate unreadable archives before admitting a staged migration;
/// the versioned decoder and conversion checks remain owned by this store.
pub fn validate_migration(root: &Path) -> Result<(), StoreError> {
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(root.join(SESSION_FILE))?)?;
match manifest.schema_version {
SESSION_SCHEMA_VERSION => validate_canonical_segment_logs(root),
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => migrate_segment_logs_to_current(
segment_log_paths(root)?,
manifest.session_id,
manifest.schema_version,
|_, _| Ok(()),
),
version => Err(StoreError::Corrupt {
line: 0,
message: format!(
"unsupported Worker Session schema version {version}, expected {SESSION_SCHEMA_VERSION}"
),
}),
}
}
/// Open an already-retained Session without creating directories or migrating
/// persisted data. Observation paths must never mutate retained state.
pub fn open_read_only(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
@@ -739,6 +782,7 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
entry.validate_input_metadata()?;
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_log_entry(&self.log_path(segment_id), entry)
@@ -797,6 +841,9 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
for entry in entries {
entry.validate_input_metadata()?;
}
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let mut content = Vec::new();
@@ -1107,7 +1154,12 @@ fn open_retained_file(path: &Path) -> std::io::Result<File> {
}
fn segment_log_paths(root: &Path) -> Result<Vec<(SegmentId, PathBuf)>, StoreError> {
let segments = root.join(SEGMENTS_DIR);
segment_log_paths_in_directory(&root.join(SEGMENTS_DIR))
}
fn segment_log_paths_in_directory(
segments: &Path,
) -> Result<Vec<(SegmentId, PathBuf)>, StoreError> {
if !segments.exists() {
return Ok(Vec::new());
}
@@ -1150,21 +1202,23 @@ fn segment_log_paths(root: &Path) -> Result<Vec<(SegmentId, PathBuf)>, StoreErro
Ok(paths)
}
fn migrate_segment_logs_to_v3(
root: &Path,
fn migrate_segment_logs_to_current(
paths: Vec<(SegmentId, PathBuf)>,
session_id: SessionId,
source_schema_version: u32,
mut write_segment: impl FnMut(&Path, &[u8]) -> Result<(), StoreError>,
) -> Result<(), StoreError> {
struct MigrationPlan {
path: PathBuf,
source: Vec<u8>,
entries: Vec<LogEntry>,
output: Vec<u8>,
}
// Phase 1 is strictly read-only. Every segment must parse and canonicalize
// successfully before the first authoritative byte is replaced.
let mut plans = Vec::new();
for (segment_id, path) in segment_log_paths(root)? {
for (segment_id, path) in paths {
let source = fs::read(&path)?;
let canonical = parse_legacy_jsonl(source_schema_version, session_id, segment_id, &source)
.map_err(|error| StoreError::Corrupt {
@@ -1174,17 +1228,32 @@ fn migrate_segment_logs_to_v3(
path.display()
),
})?;
let mut output = Vec::new();
for entry in canonical {
serde_json::to_writer(&mut output, &entry)?;
output.push(b'\n');
}
plans.push(MigrationPlan {
path,
source,
output,
entries: canonical,
output: Vec::new(),
});
}
// Resolve trusted creation boundaries first, then update inherited copies by
// logical identity too (fork/rewind seeds retain the same entry id).
let summaries = if source_schema_version < COMPACTION_SESSION_SCHEMA_VERSION {
crate::legacy_session_log::compaction_summary_ids(
plans.iter().flat_map(|plan| plan.entries.iter()),
)
} else {
// v4 already made the explicit origin migration. Preserve its origins.
Default::default()
};
for plan in &mut plans {
crate::legacy_session_log::migrate_compaction_origins(&mut plan.entries, &summaries);
for mut entry in std::mem::take(&mut plan.entries) {
crate::legacy_session_log::migrate_system_inputs(&mut entry)?;
entry.validate_input_metadata()?;
serde_json::to_writer(&mut plan.output, &entry)?;
plan.output.push(b'\n');
}
}
// Fence the complete preflight snapshot before starting phase 2. Session
// open is the exclusive restore boundary; this additionally fails closed
@@ -1202,14 +1271,56 @@ fn migrate_segment_logs_to_v3(
}
for plan in plans {
atomic_write_bytes(&plan.path, &plan.output)?;
write_segment(&plan.path, &plan.output)?;
}
Ok(())
}
/// Upgrade the standalone per-Session layout using the same persisted format
/// migration as Worker aggregates. Called only at writable Host store startup.
pub(crate) fn migrate_fs_session(root: &Path, session_id: SessionId) -> Result<(), StoreError> {
let marker = root.join("session-schema.json");
let version: u32 = match fs::read(&marker) {
Ok(bytes) => serde_json::from_slice(&bytes)?,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
ANNOTATED_SESSION_SCHEMA_VERSION
}
Err(error) => return Err(error.into()),
};
match version {
SESSION_SCHEMA_VERSION => {
validate_canonical_segment_paths(segment_log_paths_in_directory(root)?)
}
LEGACY_SESSION_SCHEMA_VERSION
| PREVIOUS_SESSION_SCHEMA_VERSION
| ANNOTATED_SESSION_SCHEMA_VERSION
| COMPACTION_SESSION_SCHEMA_VERSION
| SYSTEM_INPUT_SESSION_SCHEMA_VERSION => {
migrate_segment_logs_to_current(
segment_log_paths_in_directory(root)?,
session_id,
version,
atomic_write_bytes,
)?;
atomic_write_json(&marker, &SESSION_SCHEMA_VERSION)
}
_ => Err(StoreError::Corrupt {
line: 0,
message: format!("unsupported standalone Session schema {version}"),
}),
}
}
fn validate_canonical_segment_logs(root: &Path) -> Result<(), StoreError> {
for (_, path) in segment_log_paths(root)? {
let _: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
validate_canonical_segment_paths(segment_log_paths(root)?)
}
fn validate_canonical_segment_paths(paths: Vec<(SegmentId, PathBuf)>) -> Result<(), StoreError> {
for (_, path) in paths {
let entries: Vec<LogEntry> = parse_jsonl(&fs::read(&path)?)?;
for entry in entries {
entry.validate_input_metadata()?;
}
}
Ok(())
}
@@ -1223,17 +1334,26 @@ fn parse_legacy_jsonl(
let text = std::str::from_utf8(bytes).map_err(|error| {
serde_json::Error::io(std::io::Error::new(std::io::ErrorKind::InvalidData, error))
})?;
text.lines()
let mut offset = 0_u64;
text.split_inclusive('\n')
.enumerate()
.filter(|(_, line)| !line.trim().is_empty())
.map(|(line_index, line)| {
crate::legacy_session_log::decode_entry(
.filter_map(|(line_index, line)| {
let start = offset;
offset += line.len() as u64;
(!line.trim().is_empty()).then_some((line_index, line, start))
})
.map(|(line_index, line, start)| {
let mut entry = crate::legacy_session_log::decode_entry(
schema_version,
line,
session_id,
segment_id,
line_index,
)
)?;
// Byte offsets change during migration. Persist identities previously
// synthesized from the OLD offsets before rewriting any record.
entry.ensure_legacy_session_entry_id(session_id, segment_id, start);
Ok(entry)
})
.collect()
}
@@ -1380,6 +1500,7 @@ mod tests {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -1967,6 +2088,146 @@ mod tests {
);
}
#[test]
fn v5_migration_retries_after_second_file_publication_fails() {
fn identity_and_body(entries: &[LogEntry]) -> Vec<serde_json::Value> {
entries
.iter()
.flat_map(|record| match record {
LogEntry::AnnotatedSystemItem { entry, .. } => vec![entry.to_history_entry()],
LogEntry::AnnotatedUserInput { history, .. } => history.clone(),
_ => panic!("unexpected record in fixed migration fixture"),
})
.map(|entry| {
serde_json::json!({
"entry_id": entry.metadata.entry_id,
"origin": entry.metadata.origin,
"derivation": entry.metadata.derivation,
"item": entry.item,
})
})
.collect()
}
const SOURCE: &str =
include_str!("legacy_session_log/tests/fixtures/v4-system-inputs.jsonl");
let root = tempfile::tempdir().unwrap();
let session_id = new_session_id();
let first_id = SegmentId::from_u128(1);
let second_id = SegmentId::from_u128(2);
fs::create_dir_all(root.path().join(SEGMENTS_DIR)).unwrap();
let marker_path = root.path().join(SESSION_FILE);
atomic_write_json(
&marker_path,
&SessionManifest {
schema_version: COMPACTION_SESSION_SCHEMA_VERSION,
session_id,
},
)
.unwrap();
let old_marker = fs::read(&marker_path).unwrap();
let first_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{first_id}.jsonl"));
let second_path = root
.path()
.join(SEGMENTS_DIR)
.join(format!("{second_id}.jsonl"));
let second_source = SOURCE.replace("-id", "-second-id");
fs::write(&first_path, SOURCE).unwrap();
fs::write(&second_path, &second_source).unwrap();
let old_first = fs::read(&first_path).unwrap();
let old_second = fs::read(&second_path).unwrap();
let first_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, first_id, &old_first).unwrap());
let second_identity =
identity_and_body(&parse_legacy_jsonl(4, session_id, second_id, &old_second).unwrap());
let mut writes = 0;
let mut first_published = None;
let failure = WorkerSessionStore::new_with_migration_writer(
root.path().to_path_buf(),
|path, bytes| {
writes += 1;
match writes {
1 => {
assert_eq!(path, first_path.as_path());
// Both files are still old when the first publication starts.
assert_eq!(fs::read(&first_path).unwrap(), old_first);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
atomic_write_bytes(path, bytes)?;
first_published = Some(bytes.to_vec());
Ok(())
}
2 => {
assert_eq!(path, second_path.as_path());
assert_eq!(
fs::read(&first_path).unwrap(),
*first_published.as_ref().unwrap()
);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
// Fail only this atomic-write boundary, after complete
// preflight and a real successful first-file publication.
Err(std::io::Error::other("injected second migration write failure").into())
}
_ => panic!("publication continued after its write failure"),
}
},
);
let error = match failure {
Ok(_) => panic!("the injected publication failure must reject store open"),
Err(error) => error,
};
assert!(
error
.to_string()
.contains("injected second migration write failure")
);
assert_eq!(writes, 2);
let first_published = first_published.unwrap();
assert_ne!(first_published, old_first);
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), old_second);
assert_eq!(fs::read(&marker_path).unwrap(), old_marker);
let updated_first: Vec<LogEntry> = parse_jsonl(&first_published).unwrap();
for entry in &updated_first {
entry.validate_input_metadata().unwrap();
}
assert!(matches!(
&updated_first[0],
LogEntry::AnnotatedSystemItem {
entry: crate::LoggedSystemHistoryEntry {
item: crate::SystemItem::Input { .. },
..
},
..
}
));
assert_eq!(identity_and_body(&updated_first), first_identity);
// Retry via the public production constructor, without any injection or
// hand-built mixed fixture. The old marker drives the same migration.
let store = WorkerSessionStore::new(root.path()).unwrap();
let first = store.read_all(session_id, first_id).unwrap();
let second = store.read_all(session_id, second_id).unwrap();
assert_eq!(identity_and_body(&first), first_identity);
assert_eq!(identity_and_body(&second), second_identity);
for entry in first.iter().chain(second.iter()) {
entry.validate_input_metadata().unwrap();
}
assert_eq!(fs::read(&first_path).unwrap(), first_published);
let second_published = fs::read(&second_path).unwrap();
assert_ne!(second_published, old_second);
let manifest: SessionManifest =
serde_json::from_slice(&fs::read(&marker_path).unwrap()).unwrap();
assert_eq!(manifest.schema_version, SESSION_SCHEMA_VERSION);
drop(store);
WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(fs::read(&first_path).unwrap(), first_published);
assert_eq!(fs::read(&second_path).unwrap(), second_published);
}
#[test]
fn current_jsonl_requires_annotations_across_append_rewrite_and_reopen() {
let root = tempfile::tempdir().unwrap();
+179 -2
View File
@@ -11,6 +11,7 @@ fn annotated(item: Item) -> LoggedHistoryEntry {
LoggedHistoryEntry {
item: LoggedItem::from(item),
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId::new(),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -62,6 +63,7 @@ fn round_trip_write_and_read() {
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: None,
ts: 3200,
interrupted: false,
result: EngineResult::Finished,
@@ -157,9 +159,10 @@ fn create_segment_replacement_keeps_open_old_log_and_commits_complete_new_log()
assert_eq!(reopened.read_entry_count(sid, segid).unwrap(), new.len());
assert_eq!(
std::fs::read_dir(path.parent().unwrap()).unwrap().count(),
1,
"successful replacement must not leave temporary files"
2,
"successful replacement leaves only the segment and Session schema marker"
);
assert!(path.parent().unwrap().join("session-schema.json").is_file());
}
#[test]
@@ -419,3 +422,177 @@ fn lookup_session_of_finds_owning_session() {
assert_eq!(store.lookup_session_of(segid).unwrap(), Some(sid));
}
// Storage/projection component test: both production layouts, temporary originals,
// no Feature, model, runtime or Host authority is substituted into this boundary.
#[test]
fn generic_system_append_and_fork_preserve_metadata_without_public_payload_leak() {
use session_store::{
FeatureInputPayload, HistoryInputMetadata, HistoryInputUsePolicy, InputCompactionUse,
InputEvidenceUse, InputObservationUse, LoggedHistoryDerivation, LoggedSystemHistoryEntry,
PublicSystemItem, PublicSystemPresentation, SystemInputEnvelope, WorkerSessionStore,
};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let fork = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
store
.create_segment(sid, segment, &[nil_session_start(1, sid)])
.unwrap();
let entry = LoggedSystemHistoryEntry::from_input(
LoggedSessionHistoryOrigin::Provided {
producer: "new-feature".into(),
},
Some(LoggedHistoryDerivation {
sources: vec![LoggedSessionHistoryEntryId("source-entry".into())],
}),
SystemInputEnvelope::new(
"private model body",
HistoryInputMetadata::new(
HistoryInputUsePolicy {
recording_material: InputEvidenceUse::Exclude,
memory_evidence: InputEvidenceUse::Exclude,
observation: InputObservationUse::Hidden,
compaction: InputCompactionUse::ReferenceInput,
},
Some(
FeatureInputPayload::encode(
"new-feature:context",
101,
&serde_json::json!({"private": "secret-detail"}),
)
.unwrap(),
),
Some(PublicSystemItem {
kind: "context".into(),
body: "explicit public body".into(),
presentation: PublicSystemPresentation::Context,
}),
)
.unwrap(),
)
.unwrap(),
);
store
.append(
sid,
segment,
&LogEntry::AnnotatedSystemItem {
ts: 2,
entry: entry.clone(),
extensions: vec![],
},
)
.unwrap();
let saved = store.read_all(sid, segment).unwrap();
let LogEntry::AnnotatedSystemItem {
entry: restored, ..
} = &saved[1]
else {
panic!("missing system input")
};
assert_eq!(restored.to_history_entry(), entry.to_history_entry());
assert_eq!(
collect_state(&saved).history[0].as_text(),
Some("private model body")
);
let mut fork_seed = nil_session_start(3, sid);
if let LogEntry::AnnotatedSegmentStart {
history,
forked_from,
..
} = &mut fork_seed
{
history.push(restored.to_history_entry());
*forked_from = Some(session_store::SegmentOrigin {
segment_id: segment,
at_turn_index: 0,
});
}
store.create_segment(sid, fork, &[fork_seed]).unwrap();
let fork_saved = store.read_all(sid, fork).unwrap();
let LogEntry::AnnotatedSegmentStart { history, .. } = &fork_saved[0] else {
unreachable!()
};
assert_eq!(history, &[entry.to_history_entry()]);
let original = session_store::public_snapshot::project_session_snapshot(sid, &saved);
let inherited = session_store::public_snapshot::project_session_snapshot(sid, &fork_saved);
let public = serde_json::to_string(&original.entries).unwrap();
assert!(public.contains("explicit public body"));
for private in ["private model body", "secret-detail", "new-feature:context"] {
assert!(!public.contains(private), "private field leaked: {private}");
}
assert_eq!(original.entries.len(), 1);
assert_eq!(inherited.entries.len(), 1);
assert_eq!(original.entries[0].entry_id, inherited.entries[0].entry_id);
assert_eq!(
original.entries[0].derived_from,
inherited.entries[0].derived_from
);
assert_eq!(
original.entries[0].provenance,
inherited.entries[0].provenance
);
assert_eq!(
serde_json::to_value(&original.entries[0].data).unwrap(),
serde_json::to_value(&inherited.entries[0].data).unwrap()
);
}
}
#[test]
fn invalid_generic_system_input_rejects_append_create_and_external_json_without_changing_originals()
{
use session_store::{StoreError, SystemItem, WorkerSessionStore};
for worker_layout in [false, true] {
let dir = tempfile::tempdir().unwrap();
let sid = new_session_id();
let segment = new_segment_id();
let store: Box<dyn Store> = if worker_layout {
Box::new(WorkerSessionStore::new(dir.path()).unwrap())
} else {
Box::new(FsStore::new(dir.path()).unwrap())
};
let seed = nil_session_start(1, sid);
store
.create_segment(sid, segment, std::slice::from_ref(&seed))
.unwrap();
let path = if worker_layout {
dir.path().join("segments").join(format!("{segment}.jsonl"))
} else {
dir.path()
.join(sid.to_string())
.join(format!("{segment}.jsonl"))
};
let before = std::fs::read(&path).unwrap();
let invalid = LogEntry::AnnotatedSystemItem {
ts: 2,
extensions: vec![],
entry: session_store::LoggedSystemHistoryEntry {
item: SystemItem::Input {
body: "no policy".into(),
},
metadata: LoggedSessionHistoryMetadata::legacy_unknown(),
},
};
assert!(matches!(
store.append(sid, segment, &invalid),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(matches!(
store.create_segment(sid, segment, &[seed, invalid.clone()]),
Err(StoreError::InvalidInputMetadata(_))
));
assert!(
serde_json::from_value::<LogEntry>(serde_json::to_value(invalid).unwrap()).is_err()
);
assert_eq!(std::fs::read(path).unwrap(), before);
assert_eq!(store.read_all(sid, segment).unwrap().len(), 1);
}
}
@@ -24,6 +24,7 @@ fn annotated(items: &[Item]) -> Vec<session_store::LoggedHistoryEntry> {
.map(|item| session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
+3 -13
View File
@@ -23,9 +23,7 @@ use session_store::{
CombinedStore, FsStore, FsWorkerStore, WorkerActiveSegmentRef, WorkerMetadataStore,
};
use thiserror::Error;
use worker::bootstrap::{
WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout, bash_output_dir_for_worker_id,
};
use worker::bootstrap::{WorkerBootstrap, WorkerBootstrapError, WorkerBootstrapLayout};
use worker::controller::WorkerControllerTransport;
use worker::ipc::protocol_session::{
WorkerProtocolSessionStreams, dispatch_worker_protocol_method, live_log_entry_event,
@@ -216,7 +214,6 @@ impl StandaloneHost {
WorkerFilesystemAuthority::local(launch.cwd.clone(), launch.cwd.clone());
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
bootstrap_manifest,
@@ -224,10 +221,7 @@ impl StandaloneHost {
launch.prompt_catalog,
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerBootstrapLayout::Direct { runtime_base },
WorkerControllerTransport::InProcess,
);
if let Some(subject) = &subject {
@@ -428,7 +422,6 @@ impl StandaloneHost {
);
let workspace_context = WorkerWorkspaceContext::local_filesystem(None);
let runtime_base = store.runtime_dir(worker_id);
let bash_output_dir = bash_output_dir_for_worker_id(worker_id);
let mut bootstrap = WorkerBootstrap::new(
manifest,
@@ -436,10 +429,7 @@ impl StandaloneHost {
worker::PromptCatalogSource::builtins_only(),
workspace_context,
filesystem_authority,
WorkerBootstrapLayout::Direct {
runtime_base,
bash_output_dir,
},
WorkerBootstrapLayout::Direct { runtime_base },
WorkerControllerTransport::InProcess,
);
if let Some(subject) = &subject {
+38
View File
@@ -840,6 +840,44 @@ impl SubjektivHost for LocalSubjectHost {
.journal_startup_snapshot(&reader, predecessor.as_deref())
.map_err(journal::journal_error)
}
fn journal_compaction_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn journal_compaction_context(
&self,
context: &SubjektivHostContext,
) -> Result<subjektiv::journal::startup::JournalStartupSnapshot, SubjektivHostError> {
self.body(context)?;
let attribution = self
.catalog
.store
.session_attribution(&context.session_id)
.map_err(domain_error)?
.filter(|attr| {
attr.subject_id == self.binding.subject_id
&& attr.runtime_id == self.binding.scope_id
&& attr.worker_id == self.worker_id.to_string()
})
.ok_or_else(|| {
SubjektivHostError::InvalidArgument(
"foreign or unattributed compaction Session".into(),
)
})?;
let host = subjektiv::journal::JournalHostContext::validated_session(
&self.catalog.store,
attribution,
)
.map_err(journal::journal_error)?;
if host.writer_binding_id() != Some(self.writer.binding_id.as_str()) {
return Err(SubjektivHostError::InvalidArgument(
"compaction writer lifetime mismatch".into(),
));
}
self.catalog
.store
.journal_compaction_snapshot(&host)
.map_err(journal::journal_error)
}
fn journal_recording_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
@@ -8,6 +8,7 @@ use session_store::{Store, WorkerMetadataStore};
use std::sync::atomic::AtomicBool;
mod compaction;
mod connected_compaction;
mod handoff;
mod memory_links;
@@ -211,6 +212,7 @@ fn message(id: &str, text: &str) -> session_store::LoggedHistoryEntry {
content: vec![session_store::LoggedContentPart::Text { text: text.into() }],
},
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId(id.into()),
origin: session_store::LoggedSessionHistoryOrigin::LocalHumanInput,
derivation: None,
@@ -290,6 +292,65 @@ fn save(
receipt
}
#[test]
fn compaction_host_reads_pending_recording_without_advancing_or_accepting_foreign_sessions() {
let fixture = Fixture::new("partially performed; not confirmed");
let connection = worker::subjektiv::SubjektivHostConnection::new(
fixture.host().clone(),
fixture.context.clone(),
)
.unwrap();
assert!(connection.journal_compaction_granted());
assert!(
connection
.journal_compaction_context()
.unwrap()
.journals
.is_empty()
);
let (adapter, _, _) = fixture.adapter();
let before = adapter.state().unwrap();
let snapshot = connection.journal_compaction_context().unwrap();
assert_eq!(
snapshot.journals[0].source_session_id,
fixture.context.session_id
);
assert_eq!(
snapshot.journals[0].process_order,
before.current_journal.process_order
);
assert_eq!(
snapshot.journals[0].process_byte,
before.current_journal.process_byte
);
assert_eq!(adapter.state().unwrap(), before);
assert!(
fixture
.host()
.catalog
.store
.active_journal_generation(&adapter.context, &adapter.grant.journal_id)
.unwrap()
.is_some()
);
for context in [
SubjektivHostContext {
worker_id: "foreign-worker".into(),
..fixture.context.clone()
},
SubjektivHostContext {
session_id: session_store::SessionId::now_v7().to_string(),
..fixture.context.clone()
},
] {
assert!(fixture.host().journal_compaction_context(&context).is_err());
}
fixture.host().live.store(false, Ordering::Release);
assert!(!connection.journal_compaction_granted());
assert!(connection.journal_compaction_context().is_err());
fixture.host().live.store(true, Ordering::Release);
}
#[test]
fn duplicate_admission_reserves_one_job_and_excludes_private_prompt() {
let fixture = Fixture::new("short request");
@@ -215,6 +215,7 @@ async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_a
}
protocol::Event::TextDelta { text } if text.contains(PARTIAL) => break,
protocol::Event::RunEnd { .. } => panic!("dialog ended before cancellation"),
protocol::Event::SessionEntryCommitted { entry } if matches!(entry.data, protocol::SessionSnapshotEntryData::RunError { .. }) => panic!("dialog failed before cancellation: {entry:?}"),
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
@@ -294,6 +295,32 @@ async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_a
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
for history in raw.iter().filter_map(|record| match record {
session_store::LogEntry::AnnotatedSegmentStart { history, compacted_from: Some(_), .. } => Some(history),
_ => None,
}) {
let mut calls = std::collections::HashSet::new();
let mut results = 0;
for entry in history {
let is_tool = matches!(entry.item, session_store::LoggedItem::ToolCall { .. } | session_store::LoggedItem::ToolResult { .. });
let Some(metadata) = entry.metadata.input_metadata.as_ref().filter(|_| is_tool) else { continue; };
assert_eq!(metadata.usage().recording_material, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().memory_evidence, session_store::InputEvidenceUse::Exclude);
assert_eq!(metadata.usage().observation, session_store::InputObservationUse::Hidden);
assert!(metadata.public_item().is_none());
assert!(entry.metadata.derivation.is_some());
match &entry.item {
session_store::LoggedItem::ToolCall { call_id, .. } => { calls.insert(call_id.clone()); }
session_store::LoggedItem::ToolResult { call_id, .. } => {
assert!(calls.contains(call_id), "derived Tool result must follow its real logical call");
results += 1;
}
_ => unreachable!(),
}
}
assert!(!calls.is_empty());
assert!(results >= calls.len());
}
assert_eq!(
raw.iter()
.filter(|entry| matches!(
@@ -0,0 +1,171 @@
//! Real LocalSubjectHost/Journal storage -> PreparedWorker -> compactor Request -> durable seed.
//! Only external LLM responses and the fixture's Job execution fence are substituted. The
//! in-flight generation is retained while its real adapter updates storage during generation.
//! Temporary storage and the parent fixture own cleanup; ordinary standalone lib tests select
//! this suite. Predetermined summaries attest transport and persistence, not model understanding.
use super::*;
use agen::llm_client::{
ClientError, LlmClient, Request,
event::{Event, ResponseStatus, StatusEvent},
types::Item,
};
use futures::Stream;
use session_store::{CombinedStore, FsWorkerStore, LogEntry};
use std::pin::Pin;
const BEFORE: &str = "FROZEN_POSITION: B partially implemented; tests not run; result unconfirmed";
const AFTER: &str = "NEXT_POSITION: background Journal changed during compactor generation";
const COMPLEMENT: &str =
"Unrecorded request: keep the user's constraint and finish the remaining checks.";
#[derive(Clone)]
struct MockClient {
adapter: Arc<LocalJournalHost>,
requests: Arc<Mutex<Vec<Request>>>,
updated: Arc<AtomicBool>,
}
#[async_trait::async_trait]
impl LlmClient for MockClient {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(
&self,
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>, ClientError> {
assert!(
request
.tools
.iter()
.any(|tool| tool.name == "write_summary")
);
let first = self.requests.lock().unwrap().is_empty();
self.requests.lock().unwrap().push(request);
let events = if first {
assert!(!self.updated.swap(true, Ordering::SeqCst));
let state = self.adapter.state().unwrap();
save(
&self.adapter,
"position-during-compaction",
&state.current_journal.state_token,
JournalAction::UpdatePosition {
current_position: AFTER.into(),
},
vec![],
);
vec![
Event::tool_use_start(0, "summary", "write_summary"),
Event::tool_input_delta(0, serde_json::json!({ "text": COMPLEMENT }).to_string()),
Event::tool_use_stop(0),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
} else {
vec![
Event::text_block_start(0),
Event::text_delta(0, "summary saved"),
Event::text_block_stop(0, None),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
};
Ok(Box::pin(futures::stream::iter(events.into_iter().map(Ok))))
}
}
#[tokio::test]
async fn local_host_compaction_freezes_pending_journal_through_request_and_durable_seed_during_update()
{
tokio::time::timeout(std::time::Duration::from_secs(30), async {
let fixture = Fixture::new("Keep the user's constraint; work remains unfinished and unconfirmed.");
let (adapter, _, job) = fixture.adapter();
let created = save(
&adapter,
"record-partial-work",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "A corrected to B".into(),
body: "B partially implemented; tests not run; result unconfirmed".into(),
ongoing: "Finish remaining checks".into(),
state: JournalEntryState::Open,
source_end: None,
correction_refs: vec![],
},
adapter.generation.selected_ranges.clone(),
);
save(&adapter, "position-before-compaction", &created.state_token,
JournalAction::UpdatePosition { current_position: BEFORE.into() }, vec![]);
let before = adapter.state().unwrap();
let expected = fixture.host().journal_compaction_context(&fixture.context).unwrap();
let journal = &expected.journals[0];
assert_eq!(journal.source_session_id, fixture.context.session_id);
assert_eq!(journal.current_position, BEFORE);
assert_eq!(journal.entries[0].state, JournalEntryState::Open);
assert!(!before.selected_ranges.is_empty());
assert_eq!(journal.process_order, before.current_journal.process_order);
assert_eq!(journal.process_byte, before.current_journal.process_byte);
assert_eq!(journal.process_order, 0);
assert_eq!(journal.process_byte, 0, "saved open records do not advance processing checkpoint");
let expected_json = serde_json::to_string(&expected).unwrap();
let adapter = Arc::new(adapter);
let model = MockClient { adapter: adapter.clone(), requests: Arc::default(), updated: Arc::new(AtomicBool::new(false)) };
let workers = StandaloneWorkerStore::open_existing(&fixture.host().catalog.state_root).unwrap();
let worker_id = fixture._allocation.as_ref().unwrap().worker_id();
let store = CombinedStore::new(fixture.sessions.clone(), FsWorkerStore::new(workers.worker_metadata_dir(worker_id)).unwrap());
let mut manifest = manifest::WorkerManifest::from_toml(&format!("[worker]\nname='{}'\n[model]\nscheme='anthropic'\nmodel_id='scripted'\n[engine]\n[scope]\nallow=[]", fixture.context.worker_id)).unwrap();
manifest.feature.subjektiv.profile.enabled = true;
manifest.feature.subjektiv.profile.extraction.enabled = false;
let mut prepared = worker::WorkerBootstrap::new(
manifest,
store,
worker::PromptCatalogSource::builtins_only(),
worker::WorkerWorkspaceContext::no_workspace(),
worker::WorkerFilesystemAuthority::None,
worker::WorkerBootstrapLayout::Direct { runtime_base: fixture._temp.path().join("runtime") },
worker::WorkerControllerTransport::InProcess,
).with_model_client(model.clone()).with_subjektiv_host(fixture.host().clone()).prepare_restored("journal-test").await.unwrap();
assert_eq!(prepared.worker().session_id().to_string(), fixture.context.session_id);
let source_segment = prepared.worker().segment_id();
let original = fixture.sessions.read_all(prepared.worker().session_id(), source_segment).unwrap();
let generation = fixture.host().catalog.store.active_journal_generation(&adapter.context, &adapter.grant.journal_id).unwrap().unwrap();
assert!(!fixture.host().session_fully_reflected(&fixture.context.session_id).unwrap());
prepared.worker_mut().compact(0).await.unwrap();
assert!(model.updated.load(Ordering::SeqCst));
let worker = prepared.worker();
assert_ne!(worker.segment_id(), source_segment);
let seed = worker.store().read_all(worker.session_id(), worker.segment_id()).unwrap().into_iter().find_map(|entry| match entry { LogEntry::AnnotatedSegmentStart { history, .. } => Some(history), _ => None }).expect("durable compacted seed");
let journal_items: Vec<_> = seed.iter().filter(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::ReferenceInput)).collect();
assert_eq!(journal_items.len(), 1);
let journal_item = Item::from(journal_items[0].item.clone());
let body = journal_item.as_text().unwrap();
assert!(body.contains(&expected_json), "durable seed contains the full selected JSON and processing checkpoint");
assert!(body.contains(BEFORE));
assert!(!body.contains(AFTER));
assert!(seed.iter().any(|entry| entry.metadata.input_metadata.as_ref().is_some_and(|input| input.usage().compaction == session_store::InputCompactionUse::OpeningSummary) && Item::from(entry.item.clone()).as_text().is_some_and(|text| text.contains(COMPLEMENT))));
let requests = model.requests.lock().unwrap();
let input = requests[0].items.iter().find_map(|item| match item {
Item::Message { role: agen::llm_client::types::Role::User, .. } => item.as_text().filter(|text| text.contains("## Conversation overview/index")),
_ => None,
}).expect("actual compactor user request");
assert!(input.ends_with(body), "exact same frozen reference goes to actual compactor and seed");
assert_eq!(input.matches(&expected_json).count(), 1);
assert!(!input.contains(AFTER));
drop(requests);
let latest = fixture.host().journal_compaction_context(&fixture.context).unwrap();
assert_eq!(latest.journals[0].current_position, AFTER);
assert_ne!(latest.journals[0].state_token, expected.journals[0].state_token);
assert_eq!(latest.journals[0].process_order, expected.journals[0].process_order);
assert_eq!(latest.journals[0].process_byte, expected.journals[0].process_byte);
assert_eq!(latest.journals[0].entries, expected.journals[0].entries);
assert_eq!(fixture.host().catalog.store.active_journal_generation(&adapter.context, &adapter.grant.journal_id).unwrap(), Some(generation));
assert_eq!(fixture.host().jobs().unwrap().get(&job.request.job_id).unwrap().state, job.state);
let unchanged: Vec<_> = fixture.sessions.read_all(worker.session_id(), source_segment).unwrap().into_iter().filter(|entry| !matches!(entry, LogEntry::Extension { domain, .. } if domain == "metrics")).collect();
assert_eq!(serde_json::to_value(unchanged).unwrap(), serde_json::to_value(original).unwrap());
assert!(!fixture.host().session_fully_reflected(&fixture.context.session_id).unwrap());
}).await.expect("compaction must finish without recording Job completion");
}
@@ -79,7 +79,7 @@ async fn recorded_idle_body_handoff_preserves_open_after_source_deletion_and_see
let sessions = session_store::FsStore::new_with_retention(workers.sessions_dir(second.worker_id()), crate::store::session_retention(&state)).unwrap();
let pointer = session_store::FsWorkerStore::new(workers.worker_metadata_dir(second.worker_id())).unwrap().read_by_name(&second.record().storage_key).unwrap().unwrap().active.unwrap();
let entries = sessions.read_all(session, pointer.segment_id.unwrap()).unwrap();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if matches!(entry.item, session_store::SystemItem::JournalStartup { .. }))).count();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if entry.metadata.input_metadata.as_ref().and_then(|input| input.feature_payload()).is_some_and(|payload| payload.schema() == "journal:startup"))).count();
assert_eq!(journal_items, 1);
assert_eq!(catalog.store.subject(&subject.id).unwrap().unwrap().behavior_md, "BEHAVIOR_ORIGINAL_UNCHANGED");
second.shutdown().await.unwrap();
@@ -1621,6 +1621,7 @@ allow = []
let history = |id: &str, item: LoggedItem| LoggedHistoryEntry {
item,
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::BackendInstruction { operation_id: None },
derivation: None,
@@ -1693,6 +1694,7 @@ allow = []
// A valid JSON record without its final newline is not committed.
log.write_all(
&serde_json::to_vec(&LogEntry::RunCompleted {
entry_id: None,
ts: 4,
interrupted: false,
result: agen::EngineResult::Finished,
+8
View File
@@ -1299,6 +1299,14 @@ pub type OperationResult<T> = std::result::Result<T, OperationError>;
pub fn subjektiv_store_error(error: crate::SubjektivError) -> OperationError {
match error {
crate::SubjektivError::AuthorizationDenied(reason) => {
OperationError::PermissionDenied(reason)
}
crate::SubjektivError::SubjectPermissionsConflict(_)
| crate::SubjektivError::PermissionOperationConflict(_)
| crate::SubjektivError::WorkerExecutionGrantConflict(_) => {
OperationError::Conflict(error.to_string())
}
crate::SubjektivError::SubjectScopeMismatch { .. } => {
OperationError::PermissionDenied(format!("subject_scope_mismatch: {error}"))
}
+100
View File
@@ -67,6 +67,27 @@ impl SubjektivStore {
})
}
/// Absence proof for failed body creation, including bindings never attached to a Session
/// and released lifetimes. The Host must hold its lifecycle fence across this observation
/// and recovery; absence alone does not fence a delayed Runtime create.
pub fn journal_body_is_unpublished(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row(
"SELECT NOT EXISTS(SELECT 1 FROM journal_writer_bindings
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)
AND NOT EXISTS(SELECT 1 FROM subject_session_attributions
WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)",
params![subject, runtime, worker],
|row| row.get(0),
)?)
})
}
pub fn journal_worker_was_released(
&self,
subject: &str,
@@ -203,3 +224,82 @@ impl SubjektivStore {
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn unpublished_body_rejects_orphan_and_released_bindings_and_attribution_only() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _workspace, store) = crate::tests::open_store(temp.path(), "absence-tests");
let subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
let other_subject = store
.create_subject(crate::SubjectRole::new("companion").unwrap())
.unwrap();
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
// No Session was attached: an orphan binding is still published body evidence.
let binding = store
.claim_journal_writer(&subject.id, "runtime", "worker")
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
for (subject_id, runtime, worker) in [
(other_subject.id.as_str(), "runtime", "worker"),
(subject.id.as_str(), "other-runtime", "worker"),
(subject.id.as_str(), "runtime", "other-worker"),
] {
assert!(
store
.journal_body_is_unpublished(subject_id, runtime, worker)
.unwrap()
);
}
store.release_journal_writer(&binding).unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "worker")
.unwrap()
);
store
.record_session_attribution(
SubjectSessionAttribution::new(
&subject.id,
"runtime",
"attribution-only",
"session",
)
.unwrap(),
)
.unwrap();
assert!(
!store
.journal_body_is_unpublished(&subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "other-runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&other_subject.id, "runtime", "attribution-only")
.unwrap()
);
assert!(
store
.journal_body_is_unpublished(&subject.id, "runtime", "never-published")
.unwrap()
);
}
}
+181
View File
@@ -0,0 +1,181 @@
//! Bounded saved-Journal browser projections. No history fetch, reconciliation or execution.
use super::store::{check_host, entry, fit_page, journal, summary};
use super::*;
use rusqlite::params;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct EntryCursor {
scope: String,
subject: String,
journal: Option<String>,
query: Option<String>,
correcting: Option<String>,
created_at: String,
id: String,
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
impl SubjektivStore {
pub fn journal_entry_summary(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
) -> JournalResult<JournalEntrySummary> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
summary(tx, entry(tx, journal_id, entry_id)?)
})
}
/// Correction targets are capped at sixteen by the writer contract, and may be in
/// another Journal of this Subject. Include actual Journal IDs rather than guessing.
pub fn journal_correction_targets(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
) -> JournalResult<Vec<JournalEntrySummary>> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
entry(tx, journal_id, entry_id)?;
let mut stmt = tx.prepare(
"SELECT e.journal_id,e.id FROM journal_corrections c
JOIN journal_entries e ON e.subject_id=c.subject_id AND e.id=c.target_id
WHERE c.subject_id=?1 AND c.entry_id=?2 ORDER BY e.created_at,e.id LIMIT 16",
)?;
let rows = stmt
.query_map(params![host.subject_id, entry_id], |r| {
Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?))
})?
.collect::<Result<Vec<_>, _>>()?;
rows.into_iter()
.map(|(journal_id, id)| summary(tx, entry(tx, &journal_id, &id)?))
.collect()
})
}
/// Recent entries use immutable creation time and ID as a stable descending keyset.
/// Search is literal substring matching (not SQL LIKE wildcards), bounded to 256 bytes.
/// A correcting selector filters saved correction edges, not inferred text or intervals.
pub fn browse_journal_entries(
&self,
host: &JournalHostContext,
journal_id: Option<&str>,
query: Option<&str>,
correcting: Option<&str>,
after: Option<&str>,
limit: usize,
) -> JournalResult<JournalPage<JournalEntrySummary>> {
if !(1..=MAX_JOURNAL_PAGE).contains(&limit) {
return Err(invalid("page limit must be 1..32"));
}
if query.is_some_and(|q| q.len() > 256 || q.chars().any(char::is_control)) {
return Err(invalid(
"search query must be at most 256 bytes without controls",
));
}
if correcting.is_some() && journal_id.is_none() {
return Err(invalid("correction lookup requires a Journal"));
}
let cursor: Option<EntryCursor> = after
.map(|raw| {
if raw.len() > 4096 {
return Err(invalid("entry cursor exceeds byte limit"));
}
let cursor: EntryCursor =
serde_json::from_str(raw).map_err(|_| invalid("invalid entry cursor"))?;
if cursor.scope != host.scope_id
|| cursor.subject != host.subject_id
|| cursor.journal.as_deref() != journal_id
|| cursor.query.as_deref() != query
|| cursor.correcting.as_deref() != correcting
{
return Err(invalid(
"entry cursor belongs to another scope, Subject or filter",
));
}
Ok(cursor)
})
.transpose()?;
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
if let Some(id) = journal_id {
journal(tx, host, id, false)?;
}
if let Some(id) = correcting {
entry(tx, journal_id.unwrap(), id)?;
}
let mut stmt = tx.prepare(
"SELECT e.journal_id,e.id,e.created_at FROM journal_entries e
WHERE e.subject_id=?1 AND (?4 IS NOT NULL OR ?2 IS NULL OR e.journal_id=?2)
AND (?3 IS NULL OR instr(lower(e.title),lower(?3))>0
OR instr(lower(e.body),lower(?3))>0 OR instr(lower(e.ongoing),lower(?3))>0)
AND (?4 IS NULL OR EXISTS(SELECT 1 FROM journal_corrections c
WHERE c.subject_id=?1 AND c.entry_id=e.id AND c.target_id=?4))
AND (?5 IS NULL OR (e.created_at,e.id)<(?5,?6))
ORDER BY e.created_at DESC,e.id DESC LIMIT ?7",
)?;
let rows = stmt
.query_map(
params![
host.subject_id,
journal_id,
query,
correcting,
cursor.as_ref().map(|c| &c.created_at),
cursor.as_ref().map(|c| &c.id),
limit + 1
],
|r| {
Ok((
r.get::<_, String>(0)?,
r.get::<_, String>(1)?,
r.get::<_, String>(2)?,
))
},
)?
.collect::<Result<Vec<_>, _>>()?;
let more = rows.len() > limit;
let mut items = Vec::new();
for (journal_id_value, id, created_at) in rows.into_iter().take(limit) {
let next = serde_json::to_string(&EntryCursor {
scope: host.scope_id.clone(),
subject: host.subject_id.clone(),
journal: journal_id.map(str::to_owned),
query: query.map(str::to_owned),
correcting: correcting.map(str::to_owned),
created_at,
id: id.clone(),
})?;
items.push((next, summary(tx, entry(tx, &journal_id_value, &id)?)?));
}
fit_page(items, more)
})
}
/// True only for admitted source bytes which the durable checkpoint has not reflected.
/// False does not claim that all original Session activity was admitted or recorded.
pub fn journal_has_unreflected_sources(
&self,
host: &JournalHostContext,
journal_id: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
check_host(self, tx, host, false)?;
journal(tx, host, journal_id, false)?;
Ok(tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_sources s, json_each(s.accepted_json) b
WHERE s.journal_id=?1 AND json_extract(b.value,'$.end')>s.reflected_bytes)",
[journal_id],
|r| r.get(0),
)?)
})
}
}
+1
View File
@@ -40,6 +40,7 @@
pub mod binding;
pub use binding::JournalWriterBinding;
pub(crate) use binding::migrate as migrate_binding;
pub mod browser;
mod lifecycle;
pub mod memory_links;
pub(crate) use lifecycle::migrate as migrate_lifecycle;
+29 -4
View File
@@ -1,5 +1,9 @@
//! Read-only, bounded Journal context for a new body Session. Selection and text projection
//! share one storage transaction; no source history, Job, admission or checkpoint is consulted.
//! Read-only, bounded Journal context at body startup and compaction boundaries.
//! Selection, text projection, references and processing checkpoints share one storage
//! transaction. Reads neither consult source history/Job liveness nor mutate recording.
//! Startup prefers the Host-retained predecessor; compaction requires a live body writer
//! and prefers its current Session, without waiting for Idle or recording completion.
//! Both supplement only from the same Subject under the same independent budgets.
//! This is derived reference material, never a new observation or an execution capability.
use super::*;
@@ -71,11 +75,32 @@ impl SubjektivStore {
&self,
host: &JournalHostContext,
previous_session: Option<&str>,
) -> JournalResult<JournalStartupSnapshot> {
self.context_snapshot(host, previous_session, false)
}
/// Read the saved current-Session Journal without reconciling or advancing it.
/// The active writer check fences released body lifetimes, not non-Idle execution.
pub fn journal_compaction_snapshot(
&self,
host: &JournalHostContext,
) -> JournalResult<JournalStartupSnapshot> {
let session = host.attribution.as_ref().ok_or_else(|| {
JournalError::PermissionDenied("compaction requires a Host-bound body Session".into())
})?;
self.context_snapshot(host, Some(&session.session_id), true)
}
fn context_snapshot(
&self,
host: &JournalHostContext,
preferred_session: Option<&str>,
require_current_writer: bool,
) -> JournalResult<JournalStartupSnapshot> {
self.database.try_transaction(|tx| {
super::store::check_host(self, tx, host, false)?;
super::store::check_host(self, tx, host, require_current_writer)?;
let mut ids = Vec::new();
if let Some(session) = previous_session {
if let Some(session) = preferred_session {
let previous: Option<(String, String)> = tx.query_row(
"SELECT id,subject_id FROM journals WHERE source_session_id=?1",
[session], |r| Ok((r.get(0)?, r.get(1)?)),
+5 -2
View File
@@ -70,7 +70,10 @@ fn page_limit(limit: usize) -> JournalResult<usize> {
}
Ok(limit)
}
fn fit_page<T: Serialize>(rows: Vec<(String, T)>, mut more: bool) -> JournalResult<JournalPage<T>> {
pub(super) fn fit_page<T: Serialize>(
rows: Vec<(String, T)>,
mut more: bool,
) -> JournalResult<JournalPage<T>> {
let mut page = JournalPage {
items: Vec::new(),
next: None,
@@ -162,7 +165,7 @@ pub(super) fn journal(
}
Ok(j)
}
fn entry(tx: &Connection, journal_id: &str, id: &str) -> JournalResult<JournalEntry> {
pub(super) fn entry(tx: &Connection, journal_id: &str, id: &str) -> JournalResult<JournalEntry> {
let row = tx.query_row("SELECT id, journal_id, position, state, title, body, ongoing, source_end_json, created_at, updated_at, fixed_at FROM journal_entries WHERE journal_id=?1 AND id=?2", params![journal_id, id], |r| Ok((r.get::<_,String>(0)?,r.get::<_,String>(1)?,r.get::<_,u64>(2)?,r.get::<_,String>(3)?,r.get::<_,String>(4)?,r.get::<_,String>(5)?,r.get::<_,String>(6)?,r.get::<_,Option<String>>(7)?,r.get::<_,String>(8)?,r.get::<_,String>(9)?,r.get::<_,Option<String>>(10)?))).optional()?.ok_or_else(|| JournalError::NotFound(id.into()))?;
let mut stmt = tx.prepare(
"SELECT target_id FROM journal_corrections WHERE entry_id=?1 ORDER BY target_id",
+100
View File
@@ -100,6 +100,10 @@ fn released_binding_preserves_open_and_receipts_but_never_revives_old_session_wr
f.store().release_journal_writer(&binding).unwrap();
f.store().release_journal_writer(&binding).unwrap();
f.reopen();
assert!(matches!(
f.store().journal_compaction_snapshot(&f.host),
Err(JournalError::PermissionDenied(_))
));
let old = f
.store()
.query_journal_operation(&f.host, &f.journal_id, "create")
@@ -409,6 +413,102 @@ fn release_and_late_admission_are_serialized_by_the_persisted_binding_check() {
manager.shutdown().unwrap();
}
#[test]
fn compaction_snapshot_prioritizes_current_session_without_waiting_or_mutating_recording() {
let f = Fixture::new();
let (generation, context) = f.generation("in-flight", f.ranges.clone());
f.execute(
&context,
"open",
open_action(&"日本語\\\"\\n".repeat(6000)),
f.ranges.clone(),
);
f.execute(
&context,
"position",
JournalAction::UpdatePosition {
current_position: "partial activity; checkpoint still behind".into(),
},
vec![],
);
let mut latest = String::new();
for n in 0..5 {
let session = format!("supplement-{n}");
let host = session_host(f.store(), &f.host.subject_id, &session);
f.store()
.accept_journal_sources(&host, "recent", &[source_range(&session, 0, 10)])
.unwrap();
latest = session;
}
let reader = JournalHostContext::validated_reader(f.store(), &f.host.subject_id).unwrap();
let startup = f
.store()
.journal_startup_snapshot(&reader, Some(&latest))
.unwrap();
assert_eq!(startup.journals[0].source_session_id, latest);
let before = f.state();
let snapshot = f.store().journal_compaction_snapshot(&f.host).unwrap();
assert_eq!(snapshot.journals[0].source_session_id, "source-session");
assert_eq!(snapshot.journals[0].process_order, before.0.process_order);
assert_eq!(snapshot.journals[0].process_byte, before.0.process_byte);
assert_eq!(
snapshot.journals[0].entries[0].state,
JournalEntryState::Open
);
assert!(snapshot.journals[0].entries[0].omitted);
assert!(snapshot.omitted);
assert_eq!(snapshot.journals.len(), 3);
assert!(serde_json::to_vec(&snapshot).unwrap().len() <= startup::STARTUP_MAX_BYTES);
assert_eq!(
f.store().journal_compaction_snapshot(&f.host).unwrap(),
snapshot
);
assert_eq!(f.state(), before);
assert_eq!(
f.store()
.active_journal_generation(&f.host, &f.journal_id)
.unwrap(),
Some(generation)
);
assert!(matches!(
f.store().journal_compaction_snapshot(&reader),
Err(JournalError::PermissionDenied(_))
));
f.execute(
&context,
"new-position",
JournalAction::UpdatePosition {
current_position: "background recording progressed after snapshot".into(),
},
vec![],
);
let refreshed = f.store().journal_compaction_snapshot(&f.host).unwrap();
assert_ne!(
refreshed.journals[0].state_token,
snapshot.journals[0].state_token
);
assert_ne!(
refreshed.journals[0].current_position,
snapshot.journals[0].current_position
);
let empty_host = session_host(f.store(), &f.host.subject_id, "not-yet-admitted");
assert!(
!f.store()
.journal_compaction_snapshot(&empty_host)
.unwrap()
.journals
.is_empty()
);
let other = f
.store()
.create_subject_with_behavior(SubjectRole::new("other").unwrap(), "unchanged".into())
.unwrap();
let other_host = session_host(f.store(), &other.id, "other-session");
let empty = f.store().journal_compaction_snapshot(&other_host).unwrap();
assert!(empty.journals.is_empty());
assert!(!empty.omitted);
}
#[test]
fn startup_snapshot_distinguishes_none_is_subject_local_and_bounds_utf8_at_storage() {
let f = Fixture::new();
@@ -21,6 +21,12 @@ pub enum JournalRecordingReason {
WorkerRestored,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalCompactionRequest {
pub session_id: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalRecordingRequest {
+1 -1
View File
@@ -929,7 +929,7 @@ mod tests {
let (_manager, _workspace, store) = open_store(&root, "workspace-a");
let migrated_subject = store.subject(&subject_id).unwrap().unwrap();
assert!(migrated_subject.behavior_md.is_empty());
assert!(migrated_subject.behavior_md.is_empty());
assert!(store.subject_permissions(&subject_id).unwrap().is_none());
let lifecycle_attribution = attribution(
&subject_id,
"runtime-1",
+16
View File
@@ -4,6 +4,7 @@ pub mod api;
pub mod job;
pub mod journal;
mod legacy_migrations;
pub mod permissions;
mod schema;
use legacy_migrations::{
add_candidate_decision_receipts, add_subject_behavior, add_subject_session_attribution,
@@ -82,6 +83,16 @@ pub enum SubjektivError {
},
#[error("subject `{subject_id}` behavior content changed")]
SubjectBehaviorConflict { subject_id: String },
#[error("subject `{0}` permission source changed")]
SubjectPermissionsConflict(String),
#[error("permission operation `{0}` was reused with different input")]
PermissionOperationConflict(String),
#[error("authorization denied: {0}")]
AuthorizationDenied(String),
#[error("Worker execution grant binding conflicts for `{0}`")]
WorkerExecutionGrantConflict(String),
#[error("Subject `{subject_id}` has more than {limit} retained Worker execution grants")]
WorkerExecutionGrantListOverflow { subject_id: String, limit: usize },
#[error("surface generation conflict: {0}")]
SurfaceGenerationConflict(String),
#[error("memory `{memory_id}` cannot transition from {from:?} to {to:?}")]
@@ -623,6 +634,11 @@ static MIGRATIONS: &[FeatureMigration] = &[
"add immutable candidate Journal context links",
journal::memory_links::migrate,
),
FeatureMigration::new(
12,
"add explicit Subject permission sources and immutable child execution grants",
permissions::migrate,
),
];
pub const REGISTRATION: FeatureRegistration =
+288
View File
@@ -0,0 +1,288 @@
//! Host-administered permission sources and immutable child grants, separate from behavior.
//!
//! The Host authenticates administrators before source writes and binds verified Worker/Profile
//! identities before grant writes. These records are not a Worker registry. The Host alone
//! authenticates root execution ownership against its canonical Backend singleton and checks
//! every live lineage edge; Journal bindings and retained grants cannot establish ownership.
//! Replays return historical evidence without restoring a source or granting current authority.
use super::*;
use server_api::authorization::{
PermissionResource, ScopedPermission, SubjectPermissions, SubjectPermissionsUpdate,
WorkerExecutionGrant, validate_authorization_identifier, validate_scoped_permissions,
};
use sha2::{Digest, Sha256};
pub const MAX_SUBJECT_WORKER_EXECUTION_GRANTS: usize = 1_000;
pub(super) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("permissions.sql"))?;
Ok(())
}
impl SubjektivStore {
/// Unset sources are `None` (deny), including migrated Subjects. This is an admin read,
/// not an effective-authorization decision; retired Subjects retain their source evidence.
pub fn subject_permissions(&self, subject_id: &str) -> Result<Option<SubjectPermissions>> {
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
source_in_connection(connection, subject_id)
})
}
/// Host-admin-only replacement. Every replacement issues a fresh change ID, including
/// identical contents. Revocation is an explicit empty replacement. A scope-wide operation
/// ID binds Subject, CAS input, and both permission vectors; result and receipt commit together.
pub fn set_subject_permissions(
&self,
subject_id: &str,
update: &SubjectPermissionsUpdate,
) -> Result<SubjectPermissions> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(&update.operation_id)
.map_err(SubjektivError::InvalidRecord)?;
if let Some(change) = &update.expected_change_id {
validate_authorization_identifier(change).map_err(SubjektivError::InvalidRecord)?;
}
validate_scope(self.scope_id(), &update.execution)?;
validate_scope(self.scope_id(), &update.delegation)?;
let fingerprint: String = Sha256::digest(serde_json::to_vec(&(subject_id, update))?)
.iter()
.map(|byte| format!("{byte:02x}"))
.collect();
self.database.try_transaction(|tx| {
require_subject(tx, subject_id)?;
let receipt: Option<(String, String)> = tx.query_row(
"SELECT request_fingerprint, change_id FROM subject_permission_receipts WHERE operation_id=?1",
[&update.operation_id], |row| Ok((row.get(0)?, row.get(1)?)),
).optional()?;
if let Some((saved, change_id)) = receipt {
if saved != fingerprint {
return Err(SubjektivError::PermissionOperationConflict(update.operation_id.clone()));
}
let raw: String = tx.query_row(
"SELECT record_json FROM subject_permission_changes WHERE subject_id=?1 AND change_id=?2",
params![subject_id, change_id], |row| row.get(0),
)?;
return Ok(serde_json::from_str(&raw)?);
}
require_active_subject(tx, subject_id)?;
let current = source_in_connection(tx, subject_id)?;
if current.as_ref().map(|source| &source.change_id) != update.expected_change_id.as_ref() {
return Err(SubjektivError::SubjectPermissionsConflict(subject_id.into()));
}
let source = SubjectPermissions {
change_id: issued_id("permissions"),
execution: update.execution.clone(),
delegation: update.delegation.clone(),
};
tx.execute("INSERT INTO subject_permission_changes(subject_id,change_id,record_json) VALUES(?1,?2,?3)",
params![subject_id, source.change_id, serde_json::to_string(&source)?])?;
tx.execute("INSERT INTO subject_permissions(subject_id,change_id) VALUES(?1,?2) ON CONFLICT(subject_id) DO UPDATE SET change_id=excluded.change_id",
params![subject_id, source.change_id])?;
tx.execute("INSERT INTO subject_permission_receipts(operation_id,subject_id,request_fingerprint,change_id) VALUES(?1,?2,?3,?4)",
params![update.operation_id, subject_id, fingerprint, source.change_id])?;
Ok(source)
})
}
/// Binds child grants only. Exact retries recover an immutable binding even after revocation;
/// that success is not authorization to execute or redelegate. New bindings require the
/// current source, an active Subject and explicit delegation containment. The Host must
/// authenticate root singleton ownership and every live lineage edge before calling;
/// a missing retained parent is not ownership evidence. Profile resolution, provider scope,
/// Worker creation and caller authentication also remain Host duties.
pub fn bind_worker_execution_grant(&self, grant: &WorkerExecutionGrant) -> Result<()> {
validate_grant(self.scope_id(), grant)?;
self.database.try_transaction(|tx| {
if let Some(existing) = grant_in_connection(tx, &grant.runtime_id, &grant.worker_id)? {
return if existing == *grant { Ok(()) } else {
Err(SubjektivError::WorkerExecutionGrantConflict(format!("{}/{}", grant.runtime_id, grant.worker_id)))
};
}
require_active_subject(tx, &grant.subject_id)?;
let source = source_in_connection(tx, &grant.subject_id)?
.ok_or_else(|| denied("Subject permissions are unset"))?;
if source.change_id != grant.source_change_id {
return Err(denied("Subject permission source is no longer current"));
}
require_covered(&source.delegation, &grant.permissions)?;
require_covered(&source.delegation, &grant.redelegation)?;
// A child grant cannot turn a body (including a released historical body) into a
// delegated child or attach a Worker historically attributed to a different Subject.
let invalid_target: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![grant.runtime_id, grant.worker_id, grant.subject_id], |row| row.get(0),
)?;
if invalid_target { return Err(denied("target is a body or belongs to another Subject")); }
let origin = require_retained_origin(tx, grant)?;
if let Some(origin) = origin {
require_covered(&origin.redelegation, &grant.permissions)?;
require_covered(&origin.redelegation, &grant.redelegation)?;
}
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![grant.runtime_id, grant.worker_id, grant.subject_id, grant.source_change_id, serde_json::to_string(grant)?])?;
Ok(())
})
}
/// Returns this Subject's retained grants, including stale grants needed for cancellation
/// and reporting. Never an ownership list or effective-authorization decision. Overflow is
/// explicit rather than silently omitting Workers from a cancellation/reporting pass.
pub fn list_subject_worker_execution_grants(
&self,
subject_id: &str,
) -> Result<Vec<WorkerExecutionGrant>> {
validate_authorization_identifier(subject_id).map_err(SubjektivError::InvalidRecord)?;
self.database.try_with_connection(|connection| {
require_subject_in_connection(connection, subject_id)?;
let mut statement = connection.prepare(
"SELECT record_json FROM worker_execution_grants WHERE subject_id=?1 ORDER BY runtime_id,worker_id LIMIT ?2",
)?;
let mut rows = statement.query(params![subject_id, MAX_SUBJECT_WORKER_EXECUTION_GRANTS + 1])?;
let mut grants = Vec::new();
while let Some(row) = rows.next()? {
if grants.len() == MAX_SUBJECT_WORKER_EXECUTION_GRANTS {
return Err(SubjektivError::WorkerExecutionGrantListOverflow {
subject_id: subject_id.into(),
limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS,
});
}
grants.push(serde_json::from_str(&row.get::<_, String>(0)?)?);
}
Ok(grants)
})
}
/// Returns retained binding evidence, not current authorization. The Host must compare the
/// active Subject, source change, current origin ownership, Profile and provider constraints.
pub fn worker_execution_grant(
&self,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
validate_authorization_identifier(runtime).map_err(SubjektivError::InvalidRecord)?;
validate_authorization_identifier(worker).map_err(SubjektivError::InvalidRecord)?;
self.database
.try_with_connection(|connection| grant_in_connection(connection, runtime, worker))
}
}
fn source_in_connection(
connection: &Connection,
subject: &str,
) -> Result<Option<SubjectPermissions>> {
let raw: Option<String> = connection.query_row(
"SELECT c.record_json FROM subject_permissions p JOIN subject_permission_changes c ON c.subject_id=p.subject_id AND c.change_id=p.change_id WHERE p.subject_id=?1",
[subject], |row| row.get(0),
).optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn grant_in_connection(
connection: &Connection,
runtime: &str,
worker: &str,
) -> Result<Option<WorkerExecutionGrant>> {
let raw: Option<String> = connection
.query_row(
"SELECT record_json FROM worker_execution_grants WHERE runtime_id=?1 AND worker_id=?2",
params![runtime, worker],
|row| row.get(0),
)
.optional()?;
raw.map(|raw| serde_json::from_str(&raw).map_err(Into::into))
.transpose()
}
fn validate_scope(scope: &str, permissions: &[ScopedPermission]) -> Result<()> {
validate_scoped_permissions(permissions).map_err(SubjektivError::InvalidRecord)?;
for permission in permissions {
if let PermissionResource::Workspace { workspace_id } = &permission.resource
&& workspace_id != scope
{
return Err(denied("Workspace permission is outside this store scope"));
}
}
Ok(())
}
fn validate_grant(scope: &str, grant: &WorkerExecutionGrant) -> Result<()> {
for value in [
&grant.subject_id,
&grant.origin_runtime_id,
&grant.origin_worker_id,
&grant.source_change_id,
&grant.runtime_id,
&grant.worker_id,
&grant.profile,
&grant.profile_digest,
] {
validate_authorization_identifier(value).map_err(SubjektivError::InvalidRecord)?;
}
if grant.origin_runtime_id == grant.runtime_id && grant.origin_worker_id == grant.worker_id {
return Err(denied("a child grant cannot originate from itself"));
}
validate_scope(scope, &grant.permissions)?;
validate_scope(scope, &grant.redelegation)
}
fn require_covered(ceiling: &[ScopedPermission], requested: &[ScopedPermission]) -> Result<()> {
if requested
.iter()
.all(|permission| ceiling.iter().any(|allowed| allowed.covers(permission)))
{
Ok(())
} else {
Err(denied(
"requested permission or redelegation exceeds its explicit ceiling",
))
}
}
fn require_retained_origin(
tx: &Transaction<'_>,
grant: &WorkerExecutionGrant,
) -> Result<Option<WorkerExecutionGrant>> {
let mut runtime = grant.origin_runtime_id.clone();
let mut worker = grant.origin_worker_id.clone();
let mut direct_origin = None;
let mut visited = HashSet::from([(grant.runtime_id.clone(), grant.worker_id.clone())]);
// Validate retained lineage only. Absence terminates at a Host-authenticated root;
// neither Journal active state nor absence of a grant proves current singleton ownership.
for _ in 0..64 {
if !visited.insert((runtime.clone(), worker.clone())) {
return Err(denied("cyclic grant origin"));
}
let foreign_attribution: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3) OR EXISTS(SELECT 1 FROM subject_session_attributions WHERE runtime_id=?1 AND worker_id=?2 AND subject_id<>?3)",
params![runtime, worker, grant.subject_id], |row| row.get(0),
)?;
if foreign_attribution {
return Err(denied("grant origin is attributed to another Subject"));
}
let Some(origin) = grant_in_connection(tx, &runtime, &worker)? else {
return Ok(direct_origin);
};
if origin.subject_id != grant.subject_id
|| origin.source_change_id != grant.source_change_id
{
return Err(denied(
"grant origin belongs to another Subject or permission source",
));
}
if direct_origin.is_none() {
direct_origin = Some(origin.clone());
}
runtime = origin.origin_runtime_id;
worker = origin.origin_worker_id;
}
Err(denied("grant origin exceeds maximum delegation depth"))
}
fn denied(reason: &str) -> SubjektivError {
SubjektivError::AuthorizationDenied(reason.into())
}
#[cfg(test)]
mod tests;
+45
View File
@@ -0,0 +1,45 @@
CREATE TABLE subject_permission_changes (
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL CHECK (length(change_id) > 0),
record_json TEXT NOT NULL,
PRIMARY KEY (subject_id, change_id)
);
CREATE TABLE subject_permissions (
subject_id TEXT PRIMARY KEY REFERENCES subjects(subject_id) ON DELETE RESTRICT,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE subject_permission_receipts (
operation_id TEXT PRIMARY KEY CHECK (length(operation_id) > 0),
subject_id TEXT NOT NULL,
request_fingerprint TEXT NOT NULL,
change_id TEXT NOT NULL,
FOREIGN KEY (subject_id, change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TABLE worker_execution_grants (
runtime_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
source_change_id TEXT NOT NULL,
record_json TEXT NOT NULL,
PRIMARY KEY (runtime_id, worker_id),
FOREIGN KEY (subject_id, source_change_id) REFERENCES subject_permission_changes(subject_id, change_id) ON DELETE RESTRICT
);
CREATE TRIGGER subject_permission_changes_no_update BEFORE UPDATE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are immutable');
END;
CREATE TRIGGER subject_permission_changes_no_delete BEFORE DELETE ON subject_permission_changes BEGIN
SELECT RAISE(ABORT, 'Subject permission changes are retained');
END;
CREATE TRIGGER subject_permission_receipts_no_update BEFORE UPDATE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are immutable');
END;
CREATE TRIGGER subject_permission_receipts_no_delete BEFORE DELETE ON subject_permission_receipts BEGIN
SELECT RAISE(ABORT, 'Subject permission receipts are retained');
END;
CREATE TRIGGER worker_execution_grants_no_update BEFORE UPDATE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are immutable');
END;
CREATE TRIGGER worker_execution_grants_no_delete BEFORE DELETE ON worker_execution_grants BEGIN
SELECT RAISE(ABORT, 'Worker execution grants are retained');
END;
+656
View File
@@ -0,0 +1,656 @@
//! Real SQLite component tests through store APIs; Host authentication, canonical singleton
//! ownership and live lineage checks are outside this suite. Root origins are Host-attested inputs;
//! Journal fixtures exercise defensive attribution only, never execution authority.
use super::*;
use crate::tests::{open_store, role};
use feature_storage::FeatureStorage;
use std::path::Path;
fn permission(operation: &str, path: &str, recursive: bool) -> ScopedPermission {
ScopedPermission {
operation: operation.into(),
resource: PermissionResource::Workdir {
working_directory_id: "wd".into(),
path: path.into(),
recursive,
},
}
}
fn update(
operation: &str,
expected: Option<&str>,
delegation: Vec<ScopedPermission>,
) -> SubjectPermissionsUpdate {
SubjectPermissionsUpdate {
operation_id: operation.into(),
expected_change_id: expected.map(Into::into),
execution: Vec::new(),
delegation,
}
}
fn grant(subject: &str, source: &str, worker: &str) -> WorkerExecutionGrant {
WorkerExecutionGrant {
subject_id: subject.into(),
origin_runtime_id: "runtime".into(),
origin_worker_id: "body".into(),
source_change_id: source.into(),
runtime_id: "runtime".into(),
worker_id: worker.into(),
profile: "builtin:coder".into(),
profile_digest: "resolved-profile-sha256".into(),
permissions: vec![permission("file.read", "src", false)],
redelegation: Vec::new(),
}
}
fn schema(store: &SubjektivStore) -> Vec<(String, String)> {
store.database.try_with_connection::<_, SubjektivError>(|connection| {
let mut statement = connection.prepare("SELECT name,sql FROM sqlite_master WHERE name NOT LIKE 'sqlite_%' AND sql IS NOT NULL ORDER BY name")?;
Ok(statement.query_map([], |row| Ok((row.get(0)?, row.get(1)?)))?.collect::<std::result::Result<_, _>>()?)
}).unwrap()
}
fn open_version(
root: &Path,
version: usize,
) -> (FeatureStorage, ScopedFeatureStorage, SubjektivStore) {
let manager = FeatureStorage::new(root);
let scope = manager.scope("workspace-a").unwrap();
let registration = scope
.register(FeatureRegistration::new(
SUBJEKTIV_FEATURE_ID,
&MIGRATIONS[..version],
))
.unwrap();
let store = SubjektivStore::open(&scope, &registration).unwrap();
(manager, scope, store)
}
#[test]
fn replacements_are_cas_fenced_replayable_and_never_restore_old_permissions() {
let temp = tempfile::tempdir().unwrap();
let subject;
let first;
let first_update = update("allow", None, vec![permission("file.read", ".", true)]);
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
subject = store.create_subject(role()).unwrap().id;
assert!(store.subject_permissions(&subject).unwrap().is_none());
first = store
.set_subject_permissions(&subject, &first_update)
.unwrap();
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert!(matches!(
store.set_subject_permissions(&subject, &update("stale", None, Vec::new())),
Err(SubjektivError::SubjectPermissionsConflict(_))
));
assert!(matches!(
store.set_subject_permissions(&subject, &update("allow", None, Vec::new())),
Err(SubjektivError::PermissionOperationConflict(_))
));
let other = store.create_subject(role()).unwrap();
assert!(matches!(
store.set_subject_permissions(&other.id, &first_update),
Err(SubjektivError::PermissionOperationConflict(_))
));
let same = store
.set_subject_permissions(
&subject,
&update("same", Some(&first.change_id), first.delegation.clone()),
)
.unwrap();
assert_ne!(same.change_id, first.change_id);
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&same.change_id), Vec::new()),
)
.unwrap();
assert!(revoked.delegation.is_empty());
assert_eq!(
store
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(revoked));
}
let (_reopened_manager, _reopened_scope, reopened) = open_store(temp.path(), "workspace-a");
assert!(
reopened
.subject_permissions(&subject)
.unwrap()
.unwrap()
.delegation
.is_empty()
);
assert_eq!(
reopened
.set_subject_permissions(&subject, &first_update)
.unwrap(),
first
);
}
#[test]
fn failed_receipt_insert_rolls_back_source_history_and_projection() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let first = store
.set_subject_permissions(&subject, &update("first", None, Vec::new()))
.unwrap();
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_permission_receipt BEFORE INSERT ON subject_permission_receipts BEGIN SELECT RAISE(ABORT,'injected receipt failure'); END;")?;
Ok(())
}).unwrap();
let replacement = update(
"second",
Some(&first.change_id),
vec![permission("file.read", ".", true)],
);
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_err()
);
assert_eq!(store.subject_permissions(&subject).unwrap(), Some(first));
store.database.try_transaction::<_, SubjektivError>(|tx| {
let counts: (u32, u32) = tx.query_row("SELECT (SELECT count(*) FROM subject_permission_changes),(SELECT count(*) FROM subject_permission_receipts)", [], |r| Ok((r.get(0)?, r.get(1)?)))?;
assert_eq!(counts, (1, 1));
tx.execute_batch("DROP TRIGGER fail_permission_receipt")?;
Ok(())
}).unwrap();
assert!(
store
.set_subject_permissions(&subject, &replacement)
.is_ok()
);
}
#[test]
fn scope_and_validation_rejections_do_not_issue_permission_sources() {
let temp = tempfile::tempdir().unwrap();
let (_a_manager, _a_scope, a) = open_store(&temp.path().join("a"), "workspace-a");
let (_b_manager, _b_scope, b) = open_store(&temp.path().join("b"), "workspace-b");
let subject = a.create_subject(role()).unwrap().id;
assert!(
b.set_subject_permissions(&subject, &update("cross-subject", None, Vec::new()))
.is_err()
);
let cross_workspace = ScopedPermission {
operation: "ticket.create".into(),
resource: PermissionResource::Workspace {
workspace_id: "workspace-b".into(),
},
};
assert!(
a.set_subject_permissions(
&subject,
&update("cross-workspace", None, vec![cross_workspace])
)
.is_err()
);
for (operation, path) in [
("file.*", "src"),
("file..read", "src"),
("file.read", "../src"),
("file.read", "src/../escape"),
] {
assert!(
a.set_subject_permissions(
&subject,
&update("invalid", None, vec![permission(operation, path, true)])
)
.is_err()
);
}
assert!(a.subject_permissions(&subject).unwrap().is_none());
}
#[test]
fn child_grants_are_immutable_bounded_and_do_not_create_subject_ownership() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", "src", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&child).unwrap();
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child.clone())
);
assert!(
store
.journal_body_is_unpublished(&subject, "runtime", "child")
.unwrap()
);
let mut changed = child.clone();
changed.profile_digest = "changed-profile".into();
assert!(matches!(
store.bind_worker_execution_grant(&changed),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
for (worker, requested) in [
("write", permission("file.write", "src", false)),
("escape", permission("file.read", ".", true)),
("prefix", permission("file.read", "src-other", false)),
] {
let mut candidate = grant(&subject, &source.change_id, worker);
candidate.permissions = vec![requested];
assert!(store.bind_worker_execution_grant(&candidate).is_err());
assert!(
store
.worker_execution_grant("runtime", worker)
.unwrap()
.is_none()
);
}
let mut self_origin = child.clone();
self_origin.worker_id = "body".into();
assert!(store.bind_worker_execution_grant(&self_origin).is_err());
let other = store.create_subject(role()).unwrap();
store
.claim_journal_writer(&other.id, "runtime", "other-body")
.unwrap();
let mut invalid = grant(&subject, &source.change_id, "other-body");
assert!(store.bind_worker_execution_grant(&invalid).is_err());
invalid.worker_id = "forged-child".into();
invalid.origin_worker_id = "other-body".into();
assert!(store.bind_worker_execution_grant(&invalid).is_err());
}
#[test]
fn redelegation_requires_current_source_and_explicit_retained_parent_ceiling() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = vec![permission("file.read", "src", true)];
store.bind_worker_execution_grant(&child).unwrap();
let mut descendant = grant(&subject, &source.change_id, "grandchild");
descendant.origin_worker_id = "child".into();
store.bind_worker_execution_grant(&descendant).unwrap();
let mut excessive = descendant.clone();
excessive.worker_id = "excessive".into();
excessive.redelegation = vec![permission("file.read", ".", true)];
assert!(store.bind_worker_execution_grant(&excessive).is_err());
let mut nondelegator = descendant.clone();
nondelegator.origin_worker_id = "grandchild".into();
nondelegator.worker_id = "great-grandchild".into();
assert!(store.bind_worker_execution_grant(&nondelegator).is_err());
let mut stale = descendant.clone();
stale.worker_id = "after-source-replacement".into();
let revoked = store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
assert_ne!(revoked.change_id, source.change_id);
stale.origin_worker_id = "new-body".into();
assert!(store.bind_worker_execution_grant(&stale).is_err());
// Exact retries/readback recover evidence but cannot revive the source or origin.
store.bind_worker_execution_grant(&child).unwrap();
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
store.retire_subject(&subject).unwrap();
stale.source_change_id = revoked.change_id;
assert!(store.bind_worker_execution_grant(&stale).is_err());
}
#[test]
fn journal_state_neither_authenticates_roots_nor_bypasses_retained_child_ceilings() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
// The Host has authenticated root ownership outside this store. No Journal record is
// required; changing Journal writer state cannot supply or invalidate that authentication.
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "without-journal"))
.unwrap();
let journal = store
.claim_journal_writer(&subject, "runtime", "body")
.unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-active-journal"))
.unwrap();
store.release_journal_writer(&journal).unwrap();
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "with-released-journal"))
.unwrap();
// An active Journal binding must never upgrade a child's empty redelegation ceiling.
store
.claim_journal_writer(&subject, "runtime", "without-journal")
.unwrap();
let mut descendant = grant(&subject, &source.change_id, "descendant");
descendant.origin_worker_id = "without-journal".into();
assert!(matches!(
store.bind_worker_execution_grant(&descendant),
Err(SubjektivError::AuthorizationDenied(_))
));
assert!(
store
.worker_execution_grant("runtime", "descendant")
.unwrap()
.is_none()
);
}
#[test]
fn retained_origins_reject_cycles_foreign_subjects_and_stale_parent_sources() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let mut child = grant(&subject, &source.change_id, "child");
child.redelegation = source.delegation.clone();
store.bind_worker_execution_grant(&child).unwrap();
// Closing a retained chain back through its previously unbound root must not create a cycle.
let mut cycle = grant(&subject, &source.change_id, "body");
cycle.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&cycle),
Err(SubjektivError::AuthorizationDenied(_))
));
let other = store.create_subject(role()).unwrap().id;
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let mut foreign = grant(&other, &other_source.change_id, "foreign-child");
foreign.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&foreign),
Err(SubjektivError::AuthorizationDenied(_))
));
let current = store
.set_subject_permissions(
&subject,
&update("replace", Some(&source.change_id), source.delegation),
)
.unwrap();
let mut stale_parent = grant(&subject, &current.change_id, "stale-parent-child");
stale_parent.origin_worker_id = "child".into();
assert!(matches!(
store.bind_worker_execution_grant(&stale_parent),
Err(SubjektivError::AuthorizationDenied(_))
));
}
#[test]
fn failed_grant_insert_leaves_no_binding_and_can_retry_same_grant() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let child = grant(&subject, &source.change_id, "child");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TRIGGER fail_grant BEFORE INSERT ON worker_execution_grants BEGIN SELECT RAISE(ABORT,'injected grant failure'); END;")?;
Ok(())
}).unwrap();
assert!(store.bind_worker_execution_grant(&child).is_err());
assert!(
store
.worker_execution_grant("runtime", "child")
.unwrap()
.is_none()
);
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("DROP TRIGGER fail_grant")?;
Ok(())
})
.unwrap();
store.bind_worker_execution_grant(&child).unwrap();
for sql in [
"UPDATE worker_execution_grants SET source_change_id='forged'",
"DELETE FROM worker_execution_grants",
"UPDATE subject_permission_changes SET record_json='{}'",
"DELETE FROM subject_permission_receipts",
] {
assert!(
store
.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch(sql)?;
Ok(())
})
.is_err(),
"{sql}"
);
}
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(child)
);
}
#[test]
fn current_schema_migration_preserves_subject_and_matches_new_database() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("migrated");
let original;
{
let (_old_manager, _old_scope, old) = open_version(&root, 11);
original = old
.create_subject_with_behavior(role(), "Keep behavior separate".into())
.unwrap();
}
let (_migrated_manager, _migrated_scope, migrated) = open_store(&root, "workspace-a");
assert_eq!(
migrated.subject(&original.id).unwrap(),
Some(original.clone())
);
assert!(
migrated
.subject_permissions(&original.id)
.unwrap()
.is_none()
);
let (_fresh_manager, _fresh_scope, fresh) =
open_store(&temp.path().join("fresh"), "workspace-a");
assert_eq!(schema(&migrated), schema(&fresh));
let source = migrated
.set_subject_permissions(&original.id, &update("new-source", None, Vec::new()))
.unwrap();
assert!(!source.change_id.is_empty());
assert_eq!(migrated.subject(&original.id).unwrap(), Some(original));
}
#[test]
fn unset_or_execution_only_source_cannot_issue_child_grants_and_saved_grants_survive_reopen() {
let temp = tempfile::tempdir().unwrap();
let saved;
{
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
assert!(
store
.bind_worker_execution_grant(&grant(&subject, "unset", "child"))
.is_err()
);
let mut execution_only = update("execution", None, Vec::new());
execution_only.execution = vec![permission("file.read", ".", true)];
let source = store
.set_subject_permissions(&subject, &execution_only)
.unwrap();
assert!(
store
.bind_worker_execution_grant(&grant(&subject, &source.change_id, "child"))
.is_err()
);
let source = store
.set_subject_permissions(
&subject,
&update(
"delegation",
Some(&source.change_id),
execution_only.execution,
),
)
.unwrap();
saved = grant(&subject, &source.change_id, "child");
store.bind_worker_execution_grant(&saved).unwrap();
}
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
assert_eq!(
store.worker_execution_grant("runtime", "child").unwrap(),
Some(saved.clone())
);
store.bind_worker_execution_grant(&saved).unwrap();
let mut conflicting = saved;
conflicting.profile = "builtin:default".into();
assert!(matches!(
store.bind_worker_execution_grant(&conflicting),
Err(SubjektivError::WorkerExecutionGrantConflict(_))
));
}
#[test]
fn subject_grant_listing_is_scoped_retained_ordered_and_fails_on_overflow() {
let temp = tempfile::tempdir().unwrap();
let (_manager, _scope, store) = open_store(temp.path(), "workspace-a");
let subject = store.create_subject(role()).unwrap().id;
let other = store.create_subject(role()).unwrap().id;
assert!(
store
.list_subject_worker_execution_grants(&subject)
.unwrap()
.is_empty()
);
let source = store
.set_subject_permissions(
&subject,
&update("allow", None, vec![permission("file.read", ".", true)]),
)
.unwrap();
let other_source = store
.set_subject_permissions(&other, &update("other", None, source.delegation.clone()))
.unwrap();
let other_grant = grant(&other, &other_source.change_id, "other-child");
store.bind_worker_execution_grant(&other_grant).unwrap();
for index in (0..MAX_SUBJECT_WORKER_EXECUTION_GRANTS).rev() {
store
.bind_worker_execution_grant(&grant(
&subject,
&source.change_id,
&format!("child-{index:04}"),
))
.unwrap();
}
store
.set_subject_permissions(
&subject,
&update("revoke", Some(&source.change_id), Vec::new()),
)
.unwrap();
store.retire_subject(&subject).unwrap();
let listed = store
.list_subject_worker_execution_grants(&subject)
.unwrap();
assert_eq!(listed.len(), MAX_SUBJECT_WORKER_EXECUTION_GRANTS);
assert!(
listed
.iter()
.all(|grant| grant.subject_id == subject && grant.source_change_id == source.change_id)
);
assert!(
listed
.windows(2)
.all(|pair| (&pair[0].runtime_id, &pair[0].worker_id)
< (&pair[1].runtime_id, &pair[1].worker_id))
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant.clone()]
);
// Seed one more retained row directly: the retired Subject cannot issue new grants.
// Listing must report overflow rather than returning a seemingly complete cancellation set.
let overflow = grant(&subject, &source.change_id, "overflow");
store.database.try_transaction::<_, SubjektivError>(|tx| {
tx.execute("INSERT INTO worker_execution_grants(runtime_id,worker_id,subject_id,source_change_id,record_json) VALUES(?1,?2,?3,?4,?5)",
params![overflow.runtime_id, overflow.worker_id, overflow.subject_id, overflow.source_change_id, serde_json::to_string(&overflow)?])?;
Ok(())
}).unwrap();
assert!(
matches!(store.list_subject_worker_execution_grants(&subject), Err(SubjektivError::WorkerExecutionGrantListOverflow { subject_id, limit: MAX_SUBJECT_WORKER_EXECUTION_GRANTS }) if subject_id == subject)
);
assert_eq!(
store.list_subject_worker_execution_grants(&other).unwrap(),
vec![other_grant]
);
let (_foreign_manager, _foreign_scope, foreign) =
open_store(&temp.path().join("foreign"), "workspace-b");
assert!(matches!(
foreign.list_subject_worker_execution_grants(&subject),
Err(SubjektivError::SubjectNotFound(_))
));
}
#[test]
fn failed_schema_migration_rolls_back_new_tables_and_preserves_old_data() {
let temp = tempfile::tempdir().unwrap();
let original;
{
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
original = old.create_subject(role()).unwrap();
old.database
.try_transaction::<_, SubjektivError>(|tx| {
tx.execute_batch("CREATE TABLE subject_permissions (collision TEXT)")?;
Ok(())
})
.unwrap();
}
{
let manager = FeatureStorage::new(temp.path());
let scope = manager.scope("workspace-a").unwrap();
let registration = SubjektivStore::register(&scope).unwrap();
assert!(SubjektivStore::open(&scope, &registration).is_err());
}
let (_old_manager, _old_scope, old) = open_version(temp.path(), 11);
assert_eq!(old.database.schema_version().unwrap(), 11);
assert_eq!(old.subject(&original.id).unwrap(), Some(original));
old.database.try_with_connection::<_, SubjektivError>(|connection| {
let created: bool = connection.query_row("SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE name='subject_permission_changes')", [], |row| row.get(0))?;
assert!(!created);
Ok(())
}).unwrap();
}
+2 -1
View File
@@ -23,7 +23,8 @@ thiserror = { workspace = true }
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tracing = { workspace = true }
workdir = { workspace = true }
uuid = { workspace = true, features = ["v7"] }
[dev-dependencies]
filetime = "0.2.27"
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "test-util"] }
+277 -212
View File
@@ -1,19 +1,19 @@
//! Bash Tool using WorkdirSession command capability.
//!
//! The selected session owns cwd, process launch, bounded output, timeout, and cancellation.
//! Workdir owns launch and byte transport; Bash owns bounded preview and retained evidence.
//! Bash is not a sandbox; the Runtime process and OS user define the execution boundary.
//! Per-execution handles and nonces prevent cancellation of another command. Dropping a future
//! does not confirm termination; active registrations remain until cleanup requests cancellation
//! and observes terminal output.
//! does not confirm termination. Registrations remain during bounded cancellation/drain cleanup;
//! an unconfirmed provider outcome leaves partial evidence, never a fabricated terminal fence.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use crate::bash_output::{BashOutputMetadata, BashOutputView, BashOutputWriter};
use agen::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use async_trait::async_trait;
use schemars::JsonSchema;
use serde::Deserialize;
use serde::{Deserialize, Serialize};
use workdir::{
CommandHandle, CommandOutputRequest, CommandRequest, WorkdirPath, WorkdirSessionHandle,
WorkdirSessionRouter,
@@ -35,9 +35,9 @@ struct BashParams {
timeout: Option<u64>,
}
pub(crate) struct BashTool {
pub struct BashTool {
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
state: Arc<Mutex<BashExecutionState>>,
}
@@ -63,6 +63,8 @@ struct CommandGuard {
execution_id: String,
execution_nonce: u64,
handle: Option<CommandHandle>,
cursor: u64,
retained: Option<BashOutputWriter>,
}
impl Drop for CommandGuard {
@@ -74,20 +76,66 @@ impl Drop for CommandGuard {
let state = Arc::clone(&self.state);
let execution_id = self.execution_id.clone();
let execution_nonce = self.execution_nonce;
let mut retained = self.retained.take();
let cursor = self.cursor;
// A dropped provider future is not terminal confirmation. Keep the live
// execution registered until cleanup has both requested cancellation and
// observed terminal command output, so cancellation/session teardown
// cannot race with an apparently empty registry.
tokio::spawn(async move {
let _ = workdir.cancel_command(handle.clone()).await;
let _ = workdir
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
.await;
let result = tokio::time::timeout(std::time::Duration::from_secs(10), async {
let _ = workdir.cancel_command(handle.clone()).await;
let mut cursor = cursor;
loop {
let page = command_page(
&workdir,
CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
)
.await?;
if let Some(writer) = &mut retained {
let mut next = cursor;
for chunk in &page.chunks {
if chunk.sequence != next {
return Err(workdir::WorkdirError::Unavailable(
"command cleanup sequence gap".into(),
));
}
writer
.append_chunk(chunk)
.map_err(workdir::WorkdirError::Unavailable)?;
next = next.checked_add(chunk.bytes.len() as u64).ok_or_else(|| {
workdir::WorkdirError::Unavailable(
"command cleanup cursor overflow".into(),
)
})?;
}
if page.next_cursor != next {
return Err(workdir::WorkdirError::Unavailable(
"command cleanup cursor gap".into(),
));
}
}
cursor = page.next_cursor;
if let Some(terminal) = page.terminal {
if let Some(writer) = &mut retained {
writer
.finish_transport(&terminal)
.map_err(workdir::WorkdirError::Unavailable)?;
}
return Ok::<(), workdir::WorkdirError>(());
}
}
})
.await;
// Bounded cleanup can end with an unknown provider outcome. The writer then
// terminalizes partial evidence on drop; retaining a dead collector in `active`
// would leak registrations and falsely suggest it could still collect output.
let _ = result;
let mut state = state.lock().unwrap();
if state
.active
@@ -101,15 +149,53 @@ impl Drop for CommandGuard {
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
// Output pages are replayable only with the exact same command/cursor/limit.
// Retry read transport failures, never command start, and cap both attempts and
// backoff. The caller's deadline bounds a disconnected or unresponsive provider.
async fn command_page(
session: &workdir::WorkdirSessionHandle,
request: CommandOutputRequest,
) -> Result<workdir::CommandOutput, workdir::WorkdirError> {
for attempt in 0..3 {
match session.command_output(request.clone()).await {
Err(error)
if attempt < 2
&& matches!(
error.classification_source(),
workdir::WorkdirError::Transport(_)
| workdir::WorkdirError::Unavailable(_)
| workdir::WorkdirError::OutcomeUnknown(_)
) =>
{
tokio::time::sleep(std::time::Duration::from_millis(100 << attempt)).await;
}
result => return result,
}
}
unreachable!("bounded retry loop returns its last result")
}
impl BashTool {
pub fn input_schema() -> serde_json::Value {
serde_json::to_value(schemars::schema_for!(BashParams)).expect("Bash schema serialization")
}
pub fn new(router: Arc<WorkdirSessionRouter>, output: BashOutputView) -> Self {
Self {
router,
output,
state: Arc::new(Mutex::new(BashExecutionState::default())),
}
}
pub async fn execute_result(
&self,
input_json: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
) -> Result<BashResult, ToolError> {
let params: BashParams = serde_json::from_str(input_json)
.map_err(|error| ToolError::InvalidArgument(format!("invalid Bash input: {error}")))?;
let retained = self.output.begin().map_err(ToolError::ExecutionFailed)?;
let selected = crate::routing::resolve_session(
&self.router,
params.target_workdir.as_deref(),
@@ -134,14 +220,15 @@ impl Tool for BashTool {
execution_id: execution_id.clone(),
execution_nonce,
handle: None,
cursor: 0,
retained: Some(retained),
};
let handle = session
.start_command(CommandRequest {
command: params.command,
timeout_secs,
output_limit: INLINE_BYTE_BUDGET,
transport_byte_budget: self.output.byte_budget(),
cwd: params.cwd,
spill_dir: Some(self.output_dir.clone()),
tool_call_id: Some(call_id.clone()),
})
.await
@@ -167,15 +254,101 @@ impl Tool for BashTool {
.await
.map_err(crate::ToolsError::from)?;
}
let output = session
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: INLINE_BYTE_BUDGET,
wait: true,
})
let mut cursor = 0;
let mut stdout_preview = Vec::new();
let mut stderr_preview = Vec::new();
let mut seen = [0_u64; 2];
let mut empty_polls = 0;
let deadline =
tokio::time::Instant::now() + std::time::Duration::from_secs(timeout_secs + 30);
let terminal = loop {
let page = tokio::time::timeout_at(
deadline,
command_page(
&session,
CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
},
),
)
.await
.map_err(|_| {
ToolError::ExecutionFailed(
"command byte transport deadline exceeded; terminal outcome unknown".into(),
)
})?
.map_err(crate::ToolsError::from)?;
let mut next = cursor;
for chunk in &page.chunks {
if chunk.sequence != next || chunk.bytes.is_empty() {
return Err(ToolError::ExecutionFailed(
"command byte transport sequence gap".into(),
));
}
next = next
.checked_add(chunk.bytes.len() as u64)
.filter(|n| *n <= self.output.byte_budget())
.ok_or_else(|| {
ToolError::ExecutionFailed(
"command byte transport exceeds delivery budget".into(),
)
})?;
let index = if chunk.stream == workdir::CommandStream::Stdout {
0
} else {
1
};
if chunk.offset != seen[index] {
return Err(ToolError::ExecutionFailed(
"command byte transport offset gap".into(),
));
}
seen[index] += chunk.bytes.len() as u64;
guard
.retained
.as_mut()
.unwrap()
.append_chunk(chunk)
.map_err(ToolError::ExecutionFailed)?;
let available =
INLINE_BYTE_BUDGET.saturating_sub(stdout_preview.len() + stderr_preview.len());
let preview = if index == 0 {
&mut stdout_preview
} else {
&mut stderr_preview
};
preview.extend_from_slice(&chunk.bytes[..available.min(chunk.bytes.len())]);
}
if page.next_cursor != next {
return Err(ToolError::ExecutionFailed(
"command byte transport cursor does not fence the page".into(),
));
}
if page.chunks.is_empty() && page.terminal.is_none() {
empty_polls += 1;
if empty_polls > 32 {
return Err(ToolError::ExecutionFailed(
"command byte transport made no progress after bounded polls".into(),
));
}
} else {
empty_polls = 0;
}
cursor = page.next_cursor;
guard.cursor = cursor;
if let Some(terminal) = page.terminal {
break terminal;
}
};
let metadata = guard
.retained
.as_mut()
.unwrap()
.finish_transport(&terminal)
.map_err(ToolError::ExecutionFailed)?;
let cancellation_requested = {
let mut state = self.state.lock().unwrap();
let owns_registration = state
@@ -193,41 +366,78 @@ impl Tool for BashTool {
};
guard.handle = None;
let timed_out = output.timed_out;
let summary = if cancellation_requested {
format!("$ {cmd_summary} (cancelled)")
} else if output.timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match output.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
let content = if output.content.is_empty() {
None
} else if output.truncated {
let notice = match output.output_path {
Some(path) => format!(
"[showing bounded WorkdirSession command output; full output saved to {}]",
path.display()
),
None => "[showing bounded WorkdirSession command output; additional output was truncated]"
.to_owned(),
let timed_out = terminal.status == workdir::CommandStatus::TimedOut;
let summary =
if cancellation_requested || terminal.status == workdir::CommandStatus::Cancelled {
format!("$ {cmd_summary} (cancelled)")
} else if timed_out {
format!("$ {cmd_summary} (timed out after {timeout_secs}s)")
} else {
match terminal.exit_code {
Some(0) => format!("$ {cmd_summary}"),
Some(code) => format!("$ {cmd_summary} (exit {code})"),
None => format!("$ {cmd_summary} (terminated)"),
}
};
Some(format!("{notice}\n{}", output.content))
} else {
Some(output.content)
};
let output = ToolOutput {
Ok(BashResult {
summary,
content,
attachments: Vec::new(),
};
if cancellation_requested {
stdout_preview: String::from_utf8_lossy(&stdout_preview).into_owned(),
stderr_preview: String::from_utf8_lossy(&stderr_preview).into_owned(),
preview_omitted: terminal.stdout_bytes.saturating_add(terminal.stderr_bytes)
> (stdout_preview.len() + stderr_preview.len()) as u64,
metadata,
cancelled: cancellation_requested
|| terminal.status == workdir::CommandStatus::Cancelled,
})
}
}
#[derive(Debug, Serialize)]
pub struct BashResult {
pub summary: String,
pub stdout_preview: String,
pub stderr_preview: String,
pub preview_omitted: bool,
pub metadata: BashOutputMetadata,
pub cancelled: bool,
}
impl BashResult {
pub fn stdout_entry(&self) -> String {
format!("/bash-output/{}/stdout", self.metadata.id)
}
pub fn stderr_entry(&self) -> String {
format!("/bash-output/{}/stderr", self.metadata.id)
}
pub fn order_entry(&self) -> String {
format!("/bash-output/{}/order", self.metadata.id)
}
pub fn tool_output(&self) -> ToolOutput {
let content = serde_json::json!({
"stdout_preview": self.stdout_preview, "stderr_preview": self.stderr_preview,
"preview_omitted": self.preview_omitted, "output": self.metadata,
"stdout_entry": self.stdout_entry(), "stderr_entry": self.stderr_entry(), "order_entry": self.order_entry(),
});
ToolOutput {
summary: self.summary.clone(),
content: Some(content.to_string()),
attachments: vec![],
}
}
}
#[async_trait]
impl Tool for BashTool {
async fn execute(
&self,
input: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
let result = self.execute_result(input, ctx).await?;
let output = result.tool_output();
if result.cancelled {
Err(ToolError::Cancelled(output))
} else if timed_out {
} else if result.metadata.terminal.status == Some(workdir::CommandStatus::TimedOut) {
Err(ToolError::Interrupted(output))
} else {
Ok(output)
@@ -288,164 +498,19 @@ fn truncate_for_summary(command: &str) -> String {
summary
}
pub fn bash_tool(session: WorkdirSessionHandle, output_dir: PathBuf) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output_dir)
pub fn bash_tool(session: WorkdirSessionHandle, output: BashOutputView) -> ToolDefinition {
routed_bash_tool(crate::routing::singleton_router(session), output)
}
pub(crate) fn routed_bash_tool(
router: Arc<WorkdirSessionRouter>,
output_dir: PathBuf,
output: BashOutputView,
) -> ToolDefinition {
Arc::new(move || {
let schema = schemars::schema_for!(BashParams);
let meta = ToolMeta::new("Bash")
.description("Execute a shell command in one selected Workdir. Process start, bounded inline output, full-output spill, timeout and cancellation are owned by that WorkdirSession provider. This is not a sandbox.")
.input_schema(serde_json::to_value(schema).expect("Bash schema serialization"));
let tool: Arc<dyn Tool> = Arc::new(BashTool {
router: router.clone(),
output_dir: output_dir.clone(),
state: Arc::new(Mutex::new(BashExecutionState::default())),
});
.description("Execute a shell command in one selected Workdir. Workdir owns launch, lossless bounded byte transport, timeout and cancel; Bash owns preview, quota-limited ephemeral output and read-only virtual references. This is not a sandbox.")
.input_schema(BashTool::input_schema());
let tool: Arc<dyn Tool> = Arc::new(BashTool::new(router.clone(), output.clone()));
(meta, tool)
})
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use tempfile::TempDir;
use workdir::{LocalWorkdirSession, WorkdirSessionHandle};
use super::bash_tool;
use crate::{grep::grep_tool, read::read_tool, tracker::Tracker};
fn session_with_output_scope(root: &TempDir, output: &TempDir) -> WorkdirSessionHandle {
let scope = Scope::from_config(&ScopeConfig {
allow: vec![
ScopeRule {
target: root.path().to_path_buf(),
permission: Permission::Write,
recursive: true,
symlink_policy: Default::default(),
},
ScopeRule {
target: output.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
},
],
deny: Vec::new(),
})
.unwrap();
Arc::new(LocalWorkdirSession::new(scope, root.path().to_path_buf()))
}
#[tokio::test]
async fn long_output_is_spilled_and_available_to_read_and_grep() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session.clone(), output.path().to_path_buf())();
let command = "i=0; while [ $i -lt 2000 ]; do printf 'line-%04d\\n' \"$i\"; i=$((i+1)); done; printf 'FINAL-NEEDLE\\n'";
let result = bash
.execute(
&serde_json::json!({ "command": command, "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
let rendered = result.content.expect("bounded Bash output");
let artifact = std::fs::read_dir(output.path())
.unwrap()
.next()
.expect("artifact entry")
.unwrap()
.path();
assert!(rendered.contains("full output saved to"));
assert!(rendered.contains(&artifact.display().to_string()));
let retained = std::fs::read_to_string(&artifact).unwrap();
assert!(retained.starts_with("line-0000\n"));
assert!(retained.ends_with("FINAL-NEEDLE\n"));
assert_eq!(retained.lines().count(), 2001);
let (_, read) = read_tool(session.clone(), Tracker::new())();
let read_result = read
.execute(
&serde_json::json!({
"file_path": artifact,
"offset": 2000,
"limit": 1,
})
.to_string(),
Default::default(),
)
.await
.unwrap();
assert!(
read_result
.content
.expect("Read content")
.contains("FINAL-NEEDLE")
);
let (_, grep) = grep_tool(session)();
let grep_result = grep
.execute(
&serde_json::json!({
"pattern": "FINAL-NEEDLE",
"path": artifact,
"output_mode": "content",
})
.to_string(),
Default::default(),
)
.await
.unwrap();
let grep_content = grep_result.content.expect("Grep content");
assert!(
grep_content.contains("FINAL-NEEDLE"),
"unexpected Grep content: {grep_content:?}"
);
}
#[tokio::test]
async fn missing_cwd_is_rejected_before_command_start() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let error = bash
.execute(
&serde_json::json!({ "command": "printf hidden" }).to_string(),
Default::default(),
)
.await
.unwrap_err();
assert!(error.to_string().contains("missing field `cwd`"));
}
#[tokio::test]
async fn short_output_does_not_leave_a_spill_artifact() {
let root = TempDir::new().unwrap();
let output = TempDir::new().unwrap();
let session = session_with_output_scope(&root, &output);
let (_, bash) = bash_tool(session, output.path().to_path_buf())();
let result = bash
.execute(
&serde_json::json!({ "command": "printf short", "cwd": "." }).to_string(),
Default::default(),
)
.await
.unwrap();
assert_eq!(result.content.as_deref(), Some("short"));
assert_eq!(std::fs::read_dir(output.path()).unwrap().count(), 0);
}
}
+605
View File
@@ -0,0 +1,605 @@
//! Bash-owned bounded output, independent of Workdir transport and Console subscribers.
//!
//! Output is ephemeral evidence, not a filesystem grant. Every lookup checks the live view;
//! children can see their own subtree and ancestors can retain evidence after child revocation.
//! Root teardown clears all bytes. A new root (including restore) has a new incarnation and
//! never reconstructs grants from history. Expiry is measured from admission, enforced on
//! lookup and by a weak reaper once commands run in Tokio. Parent and all descendants share
//! one pool: admission reserves a whole command budget, evicts the oldest terminal evidence
//! first, and refuses rather than evicting a live collector. Revoking a child denies access
//! but leaves ancestor evidence until pool eviction/expiry; root teardown clears the pool.
use std::collections::VecDeque;
use std::sync::{
Arc, Mutex, Weak,
atomic::{AtomicBool, Ordering},
};
use std::time::{Duration, Instant};
use serde::Serialize;
use workdir::{CommandStatus, CommandStream};
const COMMAND_BYTES: usize = 4 * 1024 * 1024;
const STORE_BYTES: usize = 32 * 1024 * 1024;
const STORE_COUNT: usize = 64;
const ORDER_COUNT: usize = 4096;
const RETENTION: Duration = Duration::from_secs(3600);
#[derive(Clone)]
pub struct BashOutputView {
store: Arc<Mutex<Store>>,
lineage: Vec<String>,
active: Arc<AtomicBool>,
authority: Arc<dyn Fn() -> bool + Send + Sync>,
}
struct Store {
records: VecDeque<Arc<Mutex<Record>>>,
reserved: usize,
retention: Duration,
reaper_started: bool,
_shutdown: tokio::sync::watch::Sender<()>,
}
struct Record {
id: String,
owner: Vec<String>,
created: Instant,
stdout: Vec<u8>,
stderr: Vec<u8>,
order: Vec<OutputRange>,
stdout_seen: u64,
stderr_seen: u64,
truncated: bool,
limit: Option<RetentionLimit>,
transport_counts: Option<(u64, u64)>,
transport_truncation: Option<workdir::CommandOutputTruncation>,
terminal: Option<OutputTerminal>,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputRange {
pub stream: CommandStream,
pub start: u64,
pub end: u64,
}
#[derive(Clone, Debug, Serialize)]
pub struct OutputTerminal {
pub status: Option<CommandStatus>,
pub exit_code: Option<i32>,
/// False on a forcibly dropped collector or failed transport: received bytes are retained,
/// but they are not a claim about the unobserved process tail.
pub transport_complete: bool,
}
#[derive(Clone, Copy, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum RetentionLimit {
ByteQuota,
OrderQuota,
}
#[derive(Clone, Debug, Serialize)]
pub struct BashOutputMetadata {
pub id: String,
pub stdout_saved: usize,
pub stderr_saved: usize,
pub stdout_seen: u64,
pub stderr_seen: u64,
pub stdout_delivered: u64,
pub stderr_delivered: u64,
pub retention_limit: Option<RetentionLimit>,
pub transport_truncation: Option<workdir::CommandOutputTruncation>,
pub storage_truncated: bool,
pub terminal: OutputTerminal,
pub text_encoding: &'static str,
pub ordering: &'static str,
}
/// A bounded reservation acquired before command start. Drop terminalizes partial evidence.
pub struct BashOutputWriter {
record: Arc<Mutex<Record>>,
}
impl Default for BashOutputView {
fn default() -> Self {
Self::new()
}
}
impl BashOutputView {
pub fn new() -> Self {
let (shutdown, _) = tokio::sync::watch::channel(());
let store = Arc::new(Mutex::new(Store {
records: VecDeque::new(),
reserved: 0,
retention: RETENTION,
reaper_started: false,
_shutdown: shutdown,
}));
let view = Self {
store,
lineage: vec![uuid::Uuid::now_v7().to_string()],
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(|| true),
};
view.start_reaper();
view
}
fn start_reaper(&self) {
let Ok(runtime) = tokio::runtime::Handle::try_current() else {
return;
};
let mut store = self.store.lock().unwrap();
if store.reaper_started {
return;
}
let receiver = store._shutdown.subscribe();
store.reaper_started = true;
let weak = Arc::downgrade(&self.store);
runtime.spawn(reap(weak, receiver));
}
pub fn byte_budget(&self) -> u64 {
COMMAND_BYTES as u64
}
pub fn child(&self, authority: Arc<dyn Fn() -> bool + Send + Sync>) -> Self {
let parent = self.clone();
let mut lineage = self.lineage.clone();
lineage.push(uuid::Uuid::now_v7().to_string());
Self {
store: self.store.clone(),
lineage,
active: Arc::new(AtomicBool::new(true)),
authority: Arc::new(move || parent.is_active() && authority()),
}
}
pub fn is_active(&self) -> bool {
self.active.load(Ordering::Acquire) && (self.authority)()
}
pub fn is_root(&self) -> bool {
self.lineage.len() == 1
}
pub fn revoke(&self) {
self.active.store(false, Ordering::Release);
if self.lineage.len() == 1 {
let mut store = self.store.lock().unwrap();
store.records.clear();
store.reserved = 0;
}
}
pub fn begin(&self) -> Result<BashOutputWriter, String> {
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
// A host may construct the Feature before entering Tokio. Command admission
// happens inside the runtime, so idle expiry must start here as well.
self.start_reaper();
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return Err("Bash output authority revoked".into());
}
store.expire(Instant::now());
while store.records.len() >= STORE_COUNT || store.reserved + COMMAND_BYTES > STORE_BYTES {
let Some(index) = store
.records
.iter()
.position(|r| r.lock().unwrap().terminal.is_some())
else {
return Err("Bash output capacity is occupied by live commands".into());
};
store.records.remove(index);
store.reserved -= COMMAND_BYTES;
}
let record = Arc::new(Mutex::new(Record {
id: uuid::Uuid::now_v7().to_string(),
owner: self.lineage.clone(),
created: Instant::now(),
stdout: Vec::new(),
stderr: Vec::new(),
order: Vec::new(),
stdout_seen: 0,
stderr_seen: 0,
truncated: false,
limit: None,
transport_counts: None,
transport_truncation: None,
terminal: None,
}));
store.records.push_back(record.clone());
store.reserved += COMMAND_BYTES;
Ok(BashOutputWriter { record })
}
fn record(&self, id: &str) -> Option<Arc<Mutex<Record>>> {
if !self.is_active() {
return None;
}
let mut store = self.store.lock().unwrap();
if !self.is_active() {
return None;
}
store.expire(Instant::now());
store
.records
.iter()
.find(|r| {
let r = r.lock().unwrap();
r.id == id && r.owner.starts_with(&self.lineage) && r.terminal.is_some()
})
.cloned()
}
pub fn metadata(&self, id: &str) -> Option<BashOutputMetadata> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.metadata())
}
/// Copies at most the caller's bounded range; never publishes output bodies in descriptors.
pub fn read(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Vec<u8>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let start = offset.min(bytes.len());
Some(bytes[start..start.saturating_add(limit.min(64 * 1024)).min(bytes.len())].to_vec())
}
/// Scan the bounded raw source without cloning it, decoding only the requested line window.
/// A long single line uses the common byte reader instead of allocating an oversized text page.
pub fn line_window(
&self,
id: &str,
stream: CommandStream,
offset: usize,
limit: usize,
) -> Option<Result<(String, bool), String>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
let bytes = match stream {
CommandStream::Stdout => &r.stdout,
CommandStream::Stderr => &r.stderr,
};
let mut lines = bytes.split_inclusive(|b| *b == b'\n').skip(offset);
let mut window = Vec::new();
for line in lines.by_ref().take(limit.min(2000)) {
if window.len() + line.len() > 64 * 1024 {
return Some(Err(
"text source window exceeds 64 KiB; use the common byte reader".into(),
));
}
window.extend_from_slice(line);
}
let more = lines.next().is_some();
Some(Ok((String::from_utf8_lossy(&window).into_owned(), more)))
}
pub fn order(&self, id: &str) -> Option<Vec<OutputRange>> {
let record = self.record(id)?;
let r = record.lock().unwrap();
Some(r.order.clone())
}
}
impl Record {
fn metadata(&self) -> BashOutputMetadata {
BashOutputMetadata {
id: self.id.clone(),
stdout_saved: self.stdout.len(),
stderr_saved: self.stderr.len(),
stdout_seen: self.transport_counts.map_or(self.stdout_seen, |c| c.0),
stderr_seen: self.transport_counts.map_or(self.stderr_seen, |c| c.1),
stdout_delivered: self.stdout_seen,
stderr_delivered: self.stderr_seen,
retention_limit: self.limit,
transport_truncation: self.transport_truncation.clone(),
storage_truncated: self.truncated || self.transport_truncation.is_some(),
terminal: self.terminal.clone().expect("finalized record"),
text_encoding: "utf8_lossy",
ordering: "provider_read_observation",
}
}
}
impl Store {
fn expire(&mut self, now: Instant) {
let before = self.records.len();
self.records
.retain(|r| now.duration_since(r.lock().unwrap().created) < self.retention);
self.reserved -= (before - self.records.len()) * COMMAND_BYTES;
}
}
async fn reap(weak: Weak<Mutex<Store>>, mut shutdown: tokio::sync::watch::Receiver<()>) {
loop {
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(30)) => {},
_ = shutdown.changed() => return,
}
let Some(store) = weak.upgrade() else {
return;
};
store.lock().unwrap().expire(Instant::now());
}
}
impl BashOutputWriter {
pub fn append_chunk(&mut self, chunk: &workdir::CommandByteChunk) -> Result<(), String> {
let expected = {
let r = self.record.lock().unwrap();
match chunk.stream {
CommandStream::Stdout => r.stdout_seen,
CommandStream::Stderr => r.stderr_seen,
}
};
if chunk.offset != expected || chunk.bytes.is_empty() {
return Err("command byte transport stream gap or empty chunk".into());
}
self.append(chunk.stream, &chunk.bytes);
Ok(())
}
pub fn append(&mut self, stream: CommandStream, bytes: &[u8]) {
let mut r = self.record.lock().unwrap();
let seen = match stream {
CommandStream::Stdout => &mut r.stdout_seen,
CommandStream::Stderr => &mut r.stderr_seen,
};
let start = *seen;
*seen = seen.saturating_add(bytes.len() as u64);
let available = COMMAND_BYTES.saturating_sub(r.stdout.len() + r.stderr.len());
let n = if r.truncated || r.order.len() >= ORDER_COUNT {
0
} else {
available.min(bytes.len())
};
if n < bytes.len() {
r.truncated = true;
if r.limit.is_none() {
r.limit = Some(if r.order.len() >= ORDER_COUNT {
RetentionLimit::OrderQuota
} else {
RetentionLimit::ByteQuota
});
}
}
if n > 0 {
match stream {
CommandStream::Stdout => r.stdout.extend_from_slice(&bytes[..n]),
CommandStream::Stderr => r.stderr.extend_from_slice(&bytes[..n]),
}
if let Some(last) = r
.order
.last_mut()
.filter(|last| last.stream == stream && last.end == start)
{
last.end += n as u64;
} else {
r.order.push(OutputRange {
stream,
start,
end: start + n as u64,
});
}
}
}
pub fn finish_transport(
&mut self,
terminal: &workdir::CommandTerminal,
) -> Result<BashOutputMetadata, String> {
let mut r = self.record.lock().unwrap();
if terminal.stdout_delivered_bytes != r.stdout_seen
|| terminal.stderr_delivered_bytes != r.stderr_seen
|| terminal.stdout_bytes < r.stdout_seen
|| terminal.stderr_bytes < r.stderr_seen
|| ((terminal.stdout_bytes != r.stdout_seen || terminal.stderr_bytes != r.stderr_seen)
!= terminal.truncation_reason.is_some())
{
return Err("command terminal counts do not fence delivered bytes".into());
}
r.transport_counts = Some((terminal.stdout_bytes, terminal.stderr_bytes));
r.transport_truncation = terminal.truncation_reason.clone();
r.terminal = Some(OutputTerminal {
status: Some(terminal.status),
exit_code: terminal.exit_code,
transport_complete: true,
});
Ok(r.metadata())
}
pub fn finish(&mut self, terminal: OutputTerminal) -> BashOutputMetadata {
let mut r = self.record.lock().unwrap();
r.terminal = Some(terminal);
r.metadata()
}
}
impl Drop for BashOutputWriter {
fn drop(&mut self) {
let mut r = self.record.lock().unwrap();
if r.terminal.is_none() {
r.terminal = Some(OutputTerminal {
status: None,
exit_code: None,
transport_complete: false,
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn finish(view: &BashOutputView, text: &[u8]) -> String {
let mut writer = view.begin().unwrap();
writer.append(CommandStream::Stdout, text);
writer
.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
})
.id
}
#[test]
fn revoked_child_is_denied_while_parent_keeps_evidence_and_sibling_is_denied() {
let root = BashOutputView::new();
let lease = Arc::new(AtomicBool::new(true));
let child = root.child({
let lease = lease.clone();
Arc::new(move || lease.load(Ordering::Acquire))
});
let sibling = root.child(Arc::new(|| true));
let id = finish(&child, b"child");
assert_eq!(
child.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
assert!(sibling.metadata(&id).is_none());
assert!(BashOutputView::new().metadata(&id).is_none());
lease.store(false, Ordering::Release);
assert!(child.metadata(&id).is_none());
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, 20).unwrap(),
b"child"
);
root.revoke();
let restored = BashOutputView::new();
assert!(root.metadata(&id).is_none());
assert!(restored.metadata(&id).is_none());
let new_id = finish(&restored, b"new incarnation");
assert_ne!(new_id, id);
assert!(child.metadata(&new_id).is_none());
assert!(root.begin().is_err());
}
#[test]
fn parent_and_descendants_share_admission_without_evicting_live_evidence() {
let root = BashOutputView::new();
let child = root.child(Arc::new(|| true));
let sibling = root.child(Arc::new(|| true));
let mut writers = (0..STORE_BYTES / COMMAND_BYTES)
.map(|index| {
if index % 2 == 0 {
root.begin()
} else {
child.begin()
}
.unwrap()
})
.collect::<Vec<_>>();
assert!(sibling.begin().is_err());
let mut first = writers.remove(0);
first.append(CommandStream::Stdout, b"finished");
let metadata = first.finish(OutputTerminal {
status: Some(CommandStatus::Completed),
exit_code: Some(0),
transport_complete: true,
});
assert!(root.metadata(&metadata.id).is_some());
let _replacement = sibling.begin().unwrap();
assert!(root.metadata(&metadata.id).is_none());
assert!(root.begin().is_err());
assert_eq!(writers.len(), STORE_BYTES / COMMAND_BYTES - 1);
}
#[test]
fn quota_and_forced_drop_are_distinct_from_preview_omission() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, &vec![b'x'; COMMAND_BYTES + 1]);
writer.append(CommandStream::Stderr, b"tail");
let id = writer.record.lock().unwrap().id.clone();
drop(writer);
let m = root.metadata(&id).unwrap();
assert_eq!(m.stdout_saved, COMMAND_BYTES);
assert_eq!(m.stdout_seen, COMMAND_BYTES as u64 + 1);
assert_eq!(m.stderr_seen, 4);
assert!(m.storage_truncated);
assert!(!m.terminal.transport_complete);
assert_eq!(
root.read(&id, CommandStream::Stdout, 0, usize::MAX)
.unwrap()
.len(),
64 * 1024
);
}
#[test]
fn expiry_and_count_eviction_never_reuse_old_references() {
let root = BashOutputView::new();
let id = finish(&root, b"old");
root.store.lock().unwrap().retention = Duration::ZERO;
assert!(root.metadata(&id).is_none());
root.store.lock().unwrap().retention = RETENTION;
let first = finish(&root, b"first");
for _ in 0..STORE_COUNT {
finish(&root, b"later");
}
assert!(root.metadata(&first).is_none());
assert!(root.store.lock().unwrap().reserved <= STORE_BYTES);
}
#[tokio::test(start_paused = true)]
async fn expiry_reclaims_bytes_without_another_read_and_reaper_ends_with_owner() {
// Exercise host construction outside a runtime, followed by real async admission.
let root = std::thread::spawn(BashOutputView::new).join().unwrap();
finish(&root, b"expires without lookup");
let mut closed = root.store.lock().unwrap()._shutdown.subscribe();
tokio::task::yield_now().await;
// The store uses monotonic wall time for publication and Tokio time only for scheduling.
// Age the record deterministically rather than delaying this test in real time.
root.store.lock().unwrap().records[0]
.lock()
.unwrap()
.created = Instant::now() - RETENTION;
tokio::time::advance(Duration::from_secs(31)).await;
tokio::task::yield_now().await;
assert_eq!(root.store.lock().unwrap().reserved, 0);
drop(root);
assert!(closed.changed().await.is_err());
}
#[test]
fn terminal_counts_must_fence_the_received_prefix_before_marking_evidence_complete() {
let root = BashOutputView::new();
let mut writer = root.begin().unwrap();
writer.append(CommandStream::Stdout, b"abc");
let mut terminal = serde_json::json!({"status":"completed","exit_code":0,"stdout_bytes":9,"stderr_bytes":0,"stdout_delivered_bytes":4,"stderr_delivered_bytes":0,"truncation_reason":"byte_budget"});
assert!(
writer
.finish_transport(&serde_json::from_value(terminal.clone()).unwrap())
.is_err()
);
terminal["stdout_delivered_bytes"] = serde_json::json!(3);
let metadata = writer
.finish_transport(&serde_json::from_value(terminal).unwrap())
.unwrap();
assert_eq!(metadata.stdout_saved, 3);
assert_eq!(metadata.stdout_delivered, 3);
assert_eq!(metadata.stdout_seen, 9);
assert!(metadata.storage_truncated);
assert!(metadata.retention_limit.is_none());
assert_eq!(
metadata.transport_truncation,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
}
}
+68
View File
@@ -403,3 +403,71 @@ fn check_pattern(pattern: &str) -> Result<(), ToolError> {
#[cfg(test)]
#[path = "checkout_tests.rs"]
mod tests;
/// Read a bounded native text window through the same checked target.
/// The returned one-shot action records prior-Read only after the caller's presentation succeeds.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_text(
router: Arc<WorkdirSessionRouter>,
tracker: Tracker,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: usize,
limit: usize,
max_bytes: usize,
) -> Result<(workdir::ReadResult, Option<Vec<u8>>, impl FnOnce()), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let tracker = tracker.scoped_attachment(&selected.alias, selected.generation);
let target = FileTarget {
session: selected.session.clone(),
path: path.clone(),
validator: Some(validator),
};
let (result, validator) = target.read(offset, limit, max_bytes).await?;
let hash = result.content_hash.clone();
let total_lines = result.total_lines;
Ok((result, validator, move || {
tracker.record_workdir_observation(&path, hash, total_lines);
}))
}
/// Binary-safe checked byte window. This does not establish a text prior-Read observation.
#[allow(clippy::too_many_arguments)]
pub async fn read_checkout_bytes(
router: Arc<WorkdirSessionRouter>,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
offset: u64,
max_bytes: usize,
) -> Result<(workdir::ReadBytesResult, Vec<u8>), ToolError> {
let selected =
crate::routing::resolve_session(&router, Some(alias), WorkdirSessionCapability::Read)?;
if selected.generation != generation {
return Err(stale_checkout("checkout connection changed"));
}
let result = selected
.session
.checkout_execute(workdir::CheckoutRequest {
target: path.clone(),
validator,
operation: workdir::CheckoutOperation::ReadBytes { offset, max_bytes },
})
.await
.map_err(crate::file_target::checked_error)?;
match result.output {
workdir::CheckoutOutput::ReadBytes(read)
if read.path == path && read.offset == offset && read.bytes.len() <= max_bytes =>
{
Ok((read, result.observation.validator))
}
_ => Err(crate::file_target::readonly_result_error("ReadBytes")),
}
}
+3 -1
View File
@@ -147,7 +147,9 @@ impl FileTarget {
pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
use workdir::WorkdirError;
let code = match error.classification_source() {
WorkdirError::DenialContext { .. } => unreachable!("diagnostic context unwrapped"),
WorkdirError::DenialContext { .. } | WorkdirError::FailureContext { .. } => {
unreachable!("diagnostic context unwrapped")
}
WorkdirError::Conflict(_) | WorkdirError::NotFound(_) => "checkout_stale",
WorkdirError::Denied { .. }
| WorkdirError::OutOfScope { .. }
+10 -9
View File
@@ -5,6 +5,7 @@ pub mod error;
pub mod tracker;
mod bash;
pub mod bash_output;
mod edit;
mod file_target;
mod glob;
@@ -16,7 +17,7 @@ pub mod view_image;
mod web;
mod write;
pub use bash::bash_tool;
pub use bash::{BashResult, BashTool, bash_tool};
pub use checkout::{CheckoutToolOutput, execute_checkout_tool};
pub use edit::edit_tool;
pub use error::ToolsError;
@@ -35,7 +36,7 @@ pub use write::write_tool;
pub fn core_builtin_tools(
session: workdir::WorkdirSessionHandle,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -58,7 +59,7 @@ pub fn core_builtin_tools(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -71,7 +72,7 @@ pub fn core_builtin_tools(
pub fn routed_builtin_tools(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
vec![
read::routed_read_tool(router.clone(), tracker.clone()),
@@ -79,14 +80,14 @@ pub fn routed_builtin_tools(
edit::routed_edit_tool(router.clone(), tracker),
glob::routed_glob_tool(router.clone()),
grep::routed_grep_tool(router.clone()),
bash::routed_bash_tool(router, bash_output_dir),
bash::routed_bash_tool(router, bash_output),
]
}
pub fn routed_builtin_tools_for_current_capabilities(
router: std::sync::Arc<workdir::WorkdirSessionRouter>,
tracker: Tracker,
bash_output_dir: std::path::PathBuf,
bash_output: bash_output::BashOutputView,
) -> Vec<agen::tool::ToolDefinition> {
use workdir::WorkdirSessionCapability;
@@ -108,7 +109,7 @@ pub fn routed_builtin_tools_for_current_capabilities(
tools.push(grep::routed_grep_tool(router.clone()));
}
if capabilities.supports(WorkdirSessionCapability::Command) {
tools.push(bash::routed_bash_tool(router, bash_output_dir));
tools.push(bash::routed_bash_tool(router, bash_output));
}
tools
}
@@ -121,7 +122,7 @@ pub fn read_only_builtin_tools(
workdir::WorkdirSessionCapabilities::READ_ONLY,
"read-only tool projection requires a read-only Workdir session"
);
core_builtin_tools(session, Tracker::new(), std::path::PathBuf::new())
core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
}
pub fn read_only_routed_builtin_tools(
@@ -173,7 +174,7 @@ mod workdir_tool_tests {
SharedScope::new(Scope::writable(dir.path()).unwrap()),
WorkdirSessionCapabilities::READ_ONLY,
));
let names = core_builtin_tools(session, Tracker::new(), dir.path().join("output"))
let names = core_builtin_tools(session, Tracker::new(), bash_output::BashOutputView::new())
.into_iter()
.map(|definition| definition().0.name)
.collect::<Vec<_>>();
+19 -27
View File
@@ -3,7 +3,7 @@
use std::sync::Arc;
use agen::tool::{Tool, ToolDefinition};
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use manifest::Scope;
use serde_json::json;
use tempfile::TempDir;
use tools::{Tracker, core_builtin_tools};
@@ -28,31 +28,23 @@ impl Registry {
}
}
fn setup() -> (TempDir, TempDir, Registry) {
fn setup() -> (TempDir, Registry) {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let base = Scope::writable(dir.path()).unwrap();
let mut config = ScopeConfig {
allow: base.allow_rules(),
deny: base.deny_rules(),
};
config.allow.push(ScopeRule {
target: spill.path().to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
});
let scope = Scope::from_config(&config).unwrap();
let scope = Scope::writable(dir.path()).unwrap();
let fs: WorkdirSessionHandle =
std::sync::Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let tracker = Tracker::new();
let reg = Registry::new(core_builtin_tools(fs, tracker, spill.path().to_path_buf()));
(dir, spill, reg)
let reg = Registry::new(core_builtin_tools(
fs,
tracker,
tools::bash_output::BashOutputView::new(),
));
(dir, reg)
}
#[tokio::test]
async fn unicode_path_and_content() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("日本語ファイル.txt");
let content = "こんにちは 🦀 世界\nabc\n";
@@ -87,7 +79,7 @@ async fn unicode_path_and_content() {
async fn symlink_to_outside_scope_is_rejected_for_write() {
use std::os::unix::fs::symlink;
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let outside = TempDir::new().unwrap();
let outside_target = outside.path().join("secret.txt");
std::fs::write(&outside_target, "secret").unwrap();
@@ -151,7 +143,7 @@ async fn symlink_to_outside_scope_is_rejected_for_write() {
async fn broken_symlink_reports_target_and_repair_hint() {
use std::os::unix::fs::symlink;
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let link = dir.path().join("external-project");
let target = dir.path().join("missing-target");
symlink(&target, &link).unwrap();
@@ -173,7 +165,7 @@ async fn broken_symlink_reports_target_and_repair_hint() {
#[tokio::test]
async fn empty_file_read_and_edit() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("empty.txt");
std::fs::write(&file, "").unwrap();
@@ -207,7 +199,7 @@ async fn empty_file_read_and_edit() {
#[tokio::test]
async fn very_long_single_line() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("long.txt");
let big: String = "x".repeat(1024 * 1024); // 1 MiB, no newlines
std::fs::write(&file, &big).unwrap();
@@ -226,7 +218,7 @@ async fn very_long_single_line() {
#[tokio::test]
async fn absolute_path_requires_matching_read_scope() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let outside = tempfile::tempdir().unwrap();
let outside_file = outside.path().join("outside.txt");
std::fs::write(&outside_file, "secret").unwrap();
@@ -247,7 +239,7 @@ async fn absolute_path_requires_matching_read_scope() {
#[tokio::test]
async fn directory_target_is_rejected_for_read() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let read = reg.get("Read");
let err = read
.execute(&json!({ "file_path": "." }).to_string(), Default::default())
@@ -258,7 +250,7 @@ async fn directory_target_is_rejected_for_read() {
#[tokio::test]
async fn deeply_nested_new_file_is_created() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let deep = dir.path().join("a/b/c/d/e/deep.txt");
let write = reg.get("Write");
write
@@ -277,7 +269,7 @@ async fn deeply_nested_new_file_is_created() {
#[tokio::test]
async fn replace_preserves_unicode() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("u.txt");
std::fs::write(&file, "🦀 rust 🦀\n").unwrap();
@@ -306,7 +298,7 @@ async fn replace_preserves_unicode() {
#[tokio::test]
async fn grep_handles_unicode_pattern() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("u.txt");
std::fs::write(&file, "English\n日本語\nрусский\n").unwrap();
+350 -153
View File
@@ -11,7 +11,7 @@ use agen::tool::{
Tool, ToolDefinition, ToolError, ToolExecutionContext, ToolExecutionHandle,
ToolExecutionTerminal, ToolMeta,
};
use manifest::{Permission, Scope, ScopeConfig, ScopeRule};
use manifest::Scope;
use serde_json::json;
use tempfile::TempDir;
use tools::{
@@ -23,21 +23,6 @@ use workdir::{
WorkdirToolScopeRule,
};
fn scope_with_spill(workspace: &Path, spill: &Path) -> Scope {
let base = Scope::writable(workspace).unwrap();
let mut config = ScopeConfig {
allow: base.allow_rules(),
deny: base.deny_rules(),
};
config.allow.push(ScopeRule {
target: spill.to_path_buf(),
permission: Permission::Read,
recursive: true,
symlink_policy: Default::default(),
});
Scope::from_config(&config).unwrap()
}
struct Registry {
entries: Vec<(ToolMeta, Arc<dyn Tool>)>,
}
@@ -61,39 +46,38 @@ impl Registry {
}
}
fn setup() -> (TempDir, TempDir, Registry) {
fn setup() -> (TempDir, Registry) {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let fs: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let tracker = Tracker::new();
let reg = Registry::new(core_builtin_tools(fs, tracker, spill.path().to_path_buf()));
(dir, spill, reg)
let reg = Registry::new(core_builtin_tools(
fs,
tracker,
tools::bash_output::BashOutputView::new(),
));
(dir, reg)
}
fn setup_routed() -> (
TempDir,
TempDir,
TempDir,
Arc<WorkdirSessionRouter>,
Registry,
) {
fn setup_routed() -> (TempDir, TempDir, Arc<WorkdirSessionRouter>, Registry) {
let left = TempDir::new().unwrap();
let right = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let router = Arc::new(WorkdirSessionRouter::new());
for (alias, dir) in [("left", &left), ("right", &right)] {
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
router
.attach(WorkdirAttachmentAlias::new(alias).unwrap(), session)
.unwrap();
}
let definitions =
routed_builtin_tools(router.clone(), Tracker::new(), spill.path().to_path_buf());
(left, right, spill, router, Registry::new(definitions))
let definitions = routed_builtin_tools(
router.clone(),
Tracker::new(),
tools::bash_output::BashOutputView::new(),
);
(left, right, router, Registry::new(definitions))
}
fn valid_gif_with_len(target_len: usize) -> Vec<u8> {
@@ -137,7 +121,7 @@ async fn call_err(tool: &Arc<dyn Tool>, input: serde_json::Value) -> agen::tool:
#[test]
fn core_builtin_tools_registers_full_set() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let mut names = reg.names();
names.sort();
assert_eq!(names, vec!["Bash", "Edit", "Glob", "Grep", "Read", "Write"]);
@@ -145,7 +129,7 @@ fn core_builtin_tools_registers_full_set() {
#[test]
fn meta_has_description_and_schema() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
for (meta, _) in &reg.entries {
assert!(
!meta.description.is_empty(),
@@ -163,7 +147,7 @@ fn meta_has_description_and_schema() {
#[test]
fn routed_tool_schemas_share_optional_target_workdir() {
let (_left, _right, _spill, router, mut reg) = setup_routed();
let (_left, _right, router, mut reg) = setup_routed();
reg.entries.push(routed_view_image_tool(router)());
for (meta, _) in &reg.entries {
let target = &meta.input_schema["properties"]["target_workdir"];
@@ -174,7 +158,7 @@ fn routed_tool_schemas_share_optional_target_workdir() {
#[tokio::test]
async fn routed_tools_require_alias_only_when_attachment_set_is_ambiguous() {
let (_left, _right, _spill, router, reg) = setup_routed();
let (_left, _right, router, reg) = setup_routed();
let error = call_err(&reg.get("Read"), json!({ "file_path": "same.txt" })).await;
let message = error.to_string();
assert!(message.contains("target_workdir_required"), "{message}");
@@ -199,7 +183,7 @@ async fn empty_routed_surface_returns_no_attachment_error() {
let reg = Registry::new(routed_builtin_tools(
router,
Tracker::new(),
Path::new("unused").to_path_buf(),
tools::bash_output::BashOutputView::new(),
));
let message = call_err(&reg.get("Read"), json!({ "file_path": "file.txt" }))
.await
@@ -209,7 +193,7 @@ async fn empty_routed_surface_returns_no_attachment_error() {
#[tokio::test]
async fn target_alias_routes_each_tool_and_isolates_read_before_edit() {
let (left, right, _spill, router, mut reg) = setup_routed();
let (left, right, router, mut reg) = setup_routed();
std::fs::write(left.path().join("same.txt"), "left value\n").unwrap();
std::fs::write(right.path().join("same.txt"), "right value\n").unwrap();
std::fs::write(left.path().join("only-left.log"), "LEFT-NEEDLE\n").unwrap();
@@ -282,7 +266,7 @@ async fn target_alias_routes_each_tool_and_isolates_read_before_edit() {
)
.await;
assert_eq!(
std::fs::canonicalize(bash.content.unwrap().trim()).unwrap(),
std::fs::canonicalize(serde_json::from_str::<serde_json::Value>(&bash.content.unwrap()).unwrap()["stdout_preview"].as_str().unwrap().trim()).unwrap(),
std::fs::canonicalize(right.path()).unwrap()
);
@@ -304,13 +288,12 @@ async fn target_alias_routes_each_tool_and_isolates_read_before_edit() {
async fn selected_attachment_capability_is_checked_without_fallback() {
let left = TempDir::new().unwrap();
let right = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let router = Arc::new(WorkdirSessionRouter::new());
router
.attach(
WorkdirAttachmentAlias::new("writable").unwrap(),
Arc::new(LocalWorkdirSession::new(
scope_with_spill(left.path(), spill.path()),
Scope::writable(left.path()).unwrap(),
left.path().to_path_buf(),
)),
)
@@ -322,7 +305,7 @@ async fn selected_attachment_capability_is_checked_without_fallback() {
workdir::Workdir::new("readonly"),
right.path().to_path_buf(),
right.path().to_path_buf(),
manifest::SharedScope::new(scope_with_spill(right.path(), spill.path())),
manifest::SharedScope::new(Scope::writable(right.path()).unwrap()),
workdir::WorkdirSessionCapabilities::READ_ONLY,
)),
)
@@ -330,7 +313,7 @@ async fn selected_attachment_capability_is_checked_without_fallback() {
let reg = Registry::new(routed_builtin_tools(
router.clone(),
Tracker::new(),
spill.path().to_path_buf(),
tools::bash_output::BashOutputView::new(),
));
let message = call_err(
@@ -352,7 +335,7 @@ async fn selected_attachment_capability_is_checked_without_fallback() {
#[tokio::test]
async fn routed_tools_observe_detach_after_registration() {
let (_left, _right, _spill, router, reg) = setup_routed();
let (_left, _right, router, reg) = setup_routed();
let alias = WorkdirAttachmentAlias::new("left").unwrap();
router.detach(&alias).await.unwrap();
@@ -372,7 +355,6 @@ async fn routed_tools_observe_detach_after_registration() {
async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
let first = TempDir::new().unwrap();
let second = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
std::fs::write(first.path().join("same.txt"), "same bytes\n").unwrap();
std::fs::write(second.path().join("same.txt"), "same bytes\n").unwrap();
@@ -382,7 +364,7 @@ async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
.attach(
alias.clone(),
Arc::new(LocalWorkdirSession::new(
scope_with_spill(first.path(), spill.path()),
Scope::writable(first.path()).unwrap(),
first.path().to_path_buf(),
)),
)
@@ -390,7 +372,7 @@ async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
let reg = Registry::new(routed_builtin_tools(
router.clone(),
Tracker::new(),
spill.path().to_path_buf(),
tools::bash_output::BashOutputView::new(),
));
call(
&reg.get("Read"),
@@ -403,7 +385,7 @@ async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
.attach(
alias.clone(),
Arc::new(LocalWorkdirSession::new(
scope_with_spill(second.path(), spill.path()),
Scope::writable(second.path()).unwrap(),
second.path().to_path_buf(),
)),
)
@@ -432,6 +414,8 @@ async fn detach_and_reattach_same_alias_requires_a_fresh_read() {
struct ExternalValidatingSession {
inner: LocalWorkdirSession,
truncate_first_binary_response: bool,
command_response_losses: std::sync::atomic::AtomicUsize,
lose_terminal_only: bool,
mutate_after_first_binary_response: Option<std::path::PathBuf>,
}
@@ -550,7 +534,24 @@ impl workdir::WorkdirSession for ExternalValidatingSession {
&self,
request: workdir::CommandOutputRequest,
) -> Result<workdir::CommandOutput, workdir::WorkdirError> {
self.inner.command_output(request).await
let output = self.inner.command_output(request).await?;
if (!self.lose_terminal_only || output.terminal.is_some())
&& self
.command_response_losses
.fetch_update(
std::sync::atomic::Ordering::AcqRel,
std::sync::atomic::Ordering::Acquire,
|remaining| remaining.checked_sub(1),
)
.is_ok()
{
// The production provider has committed a page, but its response is lost.
return Err(
workdir::WorkdirError::Transport("injected lost response".into())
.with_failure_reason(workdir::WorkdirFailureReason::HttpTimeout),
);
}
Ok(output)
}
async fn cancel_command(
@@ -568,11 +569,12 @@ impl workdir::WorkdirSession for ExternalValidatingSession {
#[tokio::test]
async fn external_view_image_uses_a_protocol_bounded_request() {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle = Arc::new(ExternalValidatingSession {
inner: LocalWorkdirSession::new(scope, dir.path().to_path_buf()),
truncate_first_binary_response: false,
command_response_losses: std::sync::atomic::AtomicUsize::new(0),
lose_terminal_only: false,
mutate_after_first_binary_response: None,
});
let png = include_bytes!("fixtures/view-image/valid.png");
@@ -590,13 +592,14 @@ async fn external_view_image_uses_a_protocol_bounded_request() {
#[tokio::test]
async fn external_view_image_rejects_truncated_and_cross_version_chunks() {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let image_path = dir.path().join("image.png");
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
std::fs::write(&image_path, include_bytes!("fixtures/view-image/valid.png")).unwrap();
let truncated: WorkdirSessionHandle = Arc::new(ExternalValidatingSession {
inner: LocalWorkdirSession::new(scope.clone(), dir.path().to_path_buf()),
truncate_first_binary_response: true,
command_response_losses: std::sync::atomic::AtomicUsize::new(0),
lose_terminal_only: false,
mutate_after_first_binary_response: None,
});
let (_meta, tool) = view_image_tool(truncated)();
@@ -613,6 +616,8 @@ async fn external_view_image_rejects_truncated_and_cross_version_chunks() {
let changing: WorkdirSessionHandle = Arc::new(ExternalValidatingSession {
inner: LocalWorkdirSession::new(scope, dir.path().to_path_buf()),
truncate_first_binary_response: false,
command_response_losses: std::sync::atomic::AtomicUsize::new(0),
lose_terminal_only: false,
mutate_after_first_binary_response: Some(image_path),
});
let (_meta, tool) = view_image_tool(changing)();
@@ -626,8 +631,7 @@ async fn external_view_image_rejects_truncated_and_cross_version_chunks() {
async fn view_image_enforces_the_ten_mib_boundary_without_partial_attachments() {
const LIMIT: usize = 10 * 1024 * 1024;
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let (_meta, tool) = view_image_tool(session)();
@@ -650,8 +654,7 @@ async fn view_image_enforces_the_ten_mib_boundary_without_partial_attachments()
#[tokio::test]
async fn view_image_attaches_valid_supported_image_fixtures() {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let (_meta, tool) = view_image_tool(session)();
@@ -726,8 +729,7 @@ async fn delegated_view_image_preserves_caller_relative_path_identity() {
#[tokio::test]
async fn view_image_reads_scoped_bytes_into_durable_tool_detail() {
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let session: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let png = include_bytes!("fixtures/view-image/valid.png");
@@ -752,7 +754,7 @@ async fn view_image_reads_scoped_bytes_into_durable_tool_detail() {
#[tokio::test]
async fn read_then_edit_then_read_roundtrip() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("a.txt");
std::fs::write(&file, "hello world\n").unwrap();
let p = "a.txt";
@@ -784,7 +786,7 @@ async fn read_then_edit_then_read_roundtrip() {
#[tokio::test]
async fn write_then_grep_finds_content() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let write = reg.get("Write");
let grep = reg.get("Grep");
@@ -813,7 +815,7 @@ async fn write_then_grep_finds_content() {
#[tokio::test]
async fn glob_finds_written_files() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let write = reg.get("Write");
let glob = reg.get("Glob");
@@ -837,7 +839,7 @@ async fn glob_finds_written_files() {
#[tokio::test]
async fn absolute_path_is_rejected() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
let outside = TempDir::new().unwrap();
let write = reg.get("Write");
@@ -859,7 +861,7 @@ async fn absolute_path_is_rejected() {
#[tokio::test]
async fn write_to_existing_without_read_fails() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("exists.txt");
std::fs::write(&file, "preexisting").unwrap();
@@ -880,7 +882,7 @@ async fn write_to_existing_without_read_fails() {
async fn shared_workdir_across_tools() {
// The key invariant: all builtin tools share the same WorkdirSession instance,
// so read-history set by Read is visible to Edit and Write.
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("shared.txt");
std::fs::write(&file, "one\n").unwrap();
@@ -907,7 +909,7 @@ async fn shared_workdir_across_tools() {
#[tokio::test]
async fn edit_requires_read_across_tools() {
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let file = dir.path().join("a.txt");
std::fs::write(&file, "foo\n").unwrap();
@@ -928,7 +930,7 @@ async fn edit_requires_read_across_tools() {
#[tokio::test]
async fn deterministic_tool_order_is_registration_order() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
// Registration order from core_builtin_tools(): Read, Write, Edit, Glob, Grep, Bash
let names: Vec<&str> = reg.entries.iter().map(|(m, _)| m.name.as_str()).collect();
assert_eq!(
@@ -940,7 +942,7 @@ async fn deterministic_tool_order_is_registration_order() {
// Regression: tool name capitalization matches Claude Code reference
#[test]
fn tool_names_match_reference_spec() {
let (_dir, _spill, reg) = setup();
let (_dir, reg) = setup();
for expected in ["Read", "Write", "Edit", "Glob", "Grep", "Bash"] {
assert!(
reg.entries.iter().any(|(m, _)| m.name == expected),
@@ -953,15 +955,14 @@ fn tool_names_match_reference_spec() {
async fn tracker_recent_files_tracks_read_write_edit() {
// Build a fresh registry that shares a tracker we can query afterwards.
let dir = TempDir::new().unwrap();
let spill = TempDir::new().unwrap();
let scope = scope_with_spill(dir.path(), spill.path());
let scope = Scope::writable(dir.path()).unwrap();
let fs: WorkdirSessionHandle =
Arc::new(LocalWorkdirSession::new(scope, dir.path().to_path_buf()));
let tracker = Tracker::new();
let reg = Registry::new(core_builtin_tools(
fs,
tracker.clone(),
spill.path().to_path_buf(),
tools::bash_output::BashOutputView::new(),
));
let a = dir.path().join("a.txt");
@@ -1003,118 +1004,314 @@ async fn tracker_recent_files_tracks_read_write_edit() {
async fn bash_inherits_workdir_cwd() {
// The Bash tool starts at the WorkdirSession's pwd. Without any `cd`, its
// `pwd` should canonicalize to the workspace root we set up.
let (dir, _spill, reg) = setup();
let (dir, reg) = setup();
let bash = reg.get("Bash");
let out = call(&bash, json!({ "command": "pwd", "cwd": "." })).await;
let body = out.content.unwrap();
let actual = std::fs::canonicalize(body.trim()).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&body).unwrap();
let actual = std::fs::canonicalize(parsed["stdout_preview"].as_str().unwrap().trim()).unwrap();
let expected = std::fs::canonicalize(dir.path()).unwrap();
assert_eq!(actual, expected);
}
fn output_fixture() -> (
TempDir,
WorkdirSessionHandle,
tools::bash_output::BashOutputView,
Arc<dyn Tool>,
) {
let root = TempDir::new().unwrap();
let session: WorkdirSessionHandle = Arc::new(LocalWorkdirSession::new(
Scope::writable(root.path()).unwrap(),
root.path().into(),
));
let view = tools::bash_output::BashOutputView::new();
let (_, bash) = tools::bash_tool(session.clone(), view.clone())();
(root, session, view, bash)
}
#[tokio::test]
async fn bash_provider_output_exposes_readable_retained_path() {
let (_dir, spill, reg) = setup();
let bash = reg.get("Bash");
async fn bash_output_is_retained_without_a_filesystem_grant_or_telemetry_subscriber() {
let (root, _session, view, bash) = output_fixture();
let out = call(
&bash,
json!({ "command": "printf 'x%.0s' {1..20480}", "cwd": "." }),
json!({ "command": "printf 'x%.0s' {1..20480}; printf '\\377€' >&2; exit 7", "cwd": "." }),
)
.await;
let body = out.content.unwrap();
assert!(body.contains("bounded WorkdirSession command output"));
assert!(body.contains("full output saved to"));
assert!(body.contains(spill.path().to_str().unwrap()));
let artifact = std::fs::read_dir(spill.path())
.unwrap()
.next()
.expect("retained output")
.unwrap()
.path();
assert_eq!(std::fs::metadata(artifact).unwrap().len(), 20_480);
let body: serde_json::Value = serde_json::from_str(&out.content.unwrap()).unwrap();
assert_eq!(body["output"]["terminal"]["exit_code"], 7);
assert_eq!(body["preview_omitted"], true);
assert_eq!(body["output"]["storage_truncated"], false);
let id = body["output"]["id"].as_str().unwrap();
assert_eq!(
view.read(id, workdir::CommandStream::Stdout, 0, 65536)
.unwrap(),
vec![b'x'; 20480]
);
assert_eq!(
view.read(id, workdir::CommandStream::Stderr, 0, 65536)
.unwrap(),
[vec![255], "€".as_bytes().to_vec()].concat()
);
assert!(
body["stdout_entry"]
.as_str()
.unwrap()
.starts_with("/bash-output/")
);
assert!(!body.to_string().contains(root.path().to_str().unwrap()));
assert_eq!(std::fs::read_dir(root.path()).unwrap().count(), 0);
}
async fn await_command_output(
events: &mut tokio::sync::broadcast::Receiver<workdir::CommandEvent>,
) {
tokio::time::timeout(std::time::Duration::from_secs(5), async {
loop {
if matches!(
events.recv().await.unwrap(),
workdir::CommandEvent::Output { .. }
) {
break;
}
}
})
.await
.expect("command output readiness");
}
async fn await_command_terminal(
events: &mut tokio::sync::broadcast::Receiver<workdir::CommandEvent>,
) {
tokio::time::timeout(std::time::Duration::from_secs(5), async {
loop {
if let workdir::CommandEvent::Terminal { status, .. } = events.recv().await.unwrap() {
assert_eq!(status, workdir::CommandStatus::Cancelled);
break;
}
}
})
.await
.expect("command terminal fence");
}
#[tokio::test]
async fn bash_cancellation_returns_bounded_progress_as_terminal_output() {
let (dir, _spill, reg) = setup();
let marker = dir.path().join("must-not-run-after-cancel");
let command = format!(
"printf 'before\\n'; printf 'err-before\\n' >&2; sleep 1; touch {}; printf 'after\\n'",
marker.display()
);
let input = serde_json::to_string(&json!({ "command": command, "cwd": "." })).unwrap();
let (_root, session, _view, bash) = output_fixture();
let mut events = session.subscribe_command_events().unwrap();
let context = ToolExecutionContext::new("call-heavy", "attempt-heavy", 0);
let bash = reg.get("Bash");
let executing = bash.clone();
let execution_context = context.clone();
let execution = tokio::spawn(async move { executing.execute(&input, execution_context).await });
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
bash.cancel_execution(&context)
.await
.expect("signal exact execution cancellation");
let error = tokio::time::timeout(std::time::Duration::from_secs(2), execution)
.await
.expect("cancelled Bash should terminate inside the Engine grace budget")
.expect("Bash task join");
let ToolError::Cancelled(output) = error.expect_err("cancelled command is non-success") else {
panic!("expected typed cancellation result");
let execution = tokio::spawn(async move {
executing.execute(r#"{"command":"printf 'before'; printf 'err-before' >&2; while :; do :; done","cwd":"."}"#, execution_context).await
});
await_command_output(&mut events).await;
bash.cancel_execution(&context).await.unwrap();
let ToolError::Cancelled(output) =
tokio::time::timeout(std::time::Duration::from_secs(5), execution)
.await
.unwrap()
.unwrap()
.unwrap_err()
else {
panic!("expected cancellation");
};
let content = output.content.expect("bounded progress output");
assert!(
content.contains("before"),
"missing pre-cancel stdout: {content}"
);
assert!(
content.contains("err-before"),
"missing pre-cancel stderr: {content}"
);
assert!(
!content.contains("after"),
"post-cancel output leaked: {content}"
);
assert!(content.len() <= 16 * 1024, "output must remain bounded");
tokio::time::sleep(std::time::Duration::from_millis(1_100)).await;
assert!(
!marker.exists(),
"the cancelled command continued executing after terminal confirmation"
);
let body: serde_json::Value = serde_json::from_str(&output.content.unwrap()).unwrap();
assert_eq!(body["stdout_preview"], "before");
assert_eq!(body["stderr_preview"], "err-before");
assert_eq!(body["output"]["terminal"]["status"], "cancelled");
assert_eq!(body["output"]["terminal"]["transport_complete"], true);
}
#[tokio::test]
async fn bash_force_close_cleanup_stops_command_and_keeps_session_reusable() {
let (dir, _spill, reg) = setup();
let marker = dir.path().join("must-not-survive-force-close");
let command = format!("sleep 1; touch {}", marker.display());
let input = serde_json::to_string(&json!({ "command": command, "cwd": "." })).unwrap();
let bash = reg.get("Bash");
let (_root, session, _view, bash) = output_fixture();
let mut events = session.subscribe_command_events().unwrap();
let context = ToolExecutionContext::new("call-force", "attempt-force", 0);
let (handle, terminal) = ToolExecutionHandle::start(bash.clone(), input, context);
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
let (handle, terminal) = ToolExecutionHandle::start(
bash.clone(),
r#"{"command":"printf ready; while :; do :; done","cwd":"."}"#.into(),
context,
);
await_command_output(&mut events).await;
handle.force_close();
assert!(matches!(
terminal.await,
ToolExecutionTerminal::OutcomeUnknown
));
tokio::time::sleep(std::time::Duration::from_millis(1_100)).await;
assert!(
!marker.exists(),
"CommandGuard cleanup allowed a force-closed command to continue"
);
let output = bash
.execute(
r#"{"command":"printf 'reused'","cwd":"."}"#,
Default::default(),
)
.await
.expect("workdir session remains reusable after cleanup");
assert_eq!(output.content.as_deref(), Some("reused"));
await_command_terminal(&mut events).await;
let output = call(&bash, json!({"command":"printf reused", "cwd":"."})).await;
let body: serde_json::Value = serde_json::from_str(&output.content.unwrap()).unwrap();
assert_eq!(body["stdout_preview"], "reused");
}
// Sanity: unused Path import guard
const _: fn() -> &'static Path = || Path::new("/");
#[tokio::test]
async fn parent_and_sibling_broker_commands_keep_output_ownership_after_revoke() {
let (root, source, parent_view, parent_bash) = output_fixture();
let broker = WorkdirToolBroker::new(source);
let mut leases = Vec::new();
let mut views = Vec::new();
let mut commands = Vec::new();
for name in ["left", "right"] {
std::fs::create_dir(root.path().join(name)).unwrap();
let lease = Arc::new(
broker
.scope(WorkdirToolScope {
rules: vec![WorkdirToolScopeRule {
target: WorkdirPath::new(name).unwrap(),
permission: WorkdirToolScopePermission::Write,
recursive: true,
symlink_policy: Default::default(),
}],
cwd: WorkdirPath::new(name).unwrap(),
command: true,
})
.await
.unwrap(),
);
let authority = lease.clone();
let view = parent_view.child(Arc::new(move || authority.is_active()));
let (_, bash) = tools::bash_tool(lease.tool_session(), view.clone())();
leases.push(lease);
views.push(view);
commands.push(bash);
}
let left_input =
json!({"command":"printf 'left-%.0s' {1..10000}; printf left-err >&2", "cwd":"."});
let right_input =
json!({"command":"printf 'right-%.0s' {1..10000}; printf right-err >&2", "cwd":"."});
let parent_input = json!({"command":"printf parent; printf parent-err >&2", "cwd":"."});
let (left, right, parent) = tokio::join!(
call(&commands[0], left_input),
call(&commands[1], right_input),
call(&parent_bash, parent_input)
);
let outputs = [left, right, parent]
.map(|out| serde_json::from_str::<serde_json::Value>(&out.content.unwrap()).unwrap());
for (index, label) in ["left", "right", "parent"].iter().enumerate() {
let id = outputs[index]["output"]["id"].as_str().unwrap();
let bytes = parent_view
.read(id, workdir::CommandStream::Stderr, 0, 65536)
.unwrap();
assert_eq!(bytes, format!("{label}-err").as_bytes());
for (child_index, view) in views.iter().enumerate() {
assert_eq!(view.metadata(id).is_some(), child_index == index);
}
}
let left_id = outputs[0]["output"]["id"].as_str().unwrap();
leases[0].close().await.unwrap();
assert!(views[0].metadata(left_id).is_none());
assert!(parent_view.metadata(left_id).is_some());
assert!(
views[1]
.metadata(outputs[1]["output"]["id"].as_str().unwrap())
.is_some()
);
leases[1].close().await.unwrap();
broker.close().await.unwrap();
parent_view.revoke();
assert!(parent_view.metadata(left_id).is_none());
}
#[tokio::test]
async fn utf8_quota_boundary_preserves_raw_prefix_and_reports_saved_vs_seen() {
let (_root, _session, view, bash) = output_fixture();
let out = call(
&bash,
json!({"command":"printf '%4194303s' ''; printf '€'", "cwd":"."}),
)
.await;
let body: serde_json::Value = serde_json::from_str(&out.content.unwrap()).unwrap();
let id = body["output"]["id"].as_str().unwrap();
let saved = body["output"]["stdout_saved"].as_u64().unwrap() as usize;
assert_eq!(saved, 4 * 1024 * 1024);
assert_eq!(body["output"]["stdout_seen"], 4 * 1024 * 1024 + 2);
assert_eq!(body["output"]["stderr_seen"], 0);
assert_eq!(body["output"]["storage_truncated"], true);
assert_eq!(
view.read(id, workdir::CommandStream::Stdout, saved - 1, 10)
.unwrap(),
vec![0xe2]
);
assert!(
view.line_window(id, workdir::CommandStream::Stdout, 0, 1)
.unwrap()
.is_err()
);
assert!(out.summary.starts_with("$ "));
}
#[tokio::test]
async fn timeout_terminal_fences_retained_stdout_and_stderr() {
let (_root, _session, view, bash) = output_fixture();
let error = tokio::time::timeout(std::time::Duration::from_secs(5), bash.execute(r#"{"command":"printf before; printf err-before >&2; while :; do :; done","cwd":".","timeout":1}"#, Default::default())).await.unwrap().unwrap_err();
let ToolError::Interrupted(out) = error else {
panic!("expected timeout output");
};
let body: serde_json::Value = serde_json::from_str(&out.content.unwrap()).unwrap();
let id = body["output"]["id"].as_str().unwrap();
assert_eq!(body["output"]["terminal"]["status"], "timed_out");
assert_eq!(body["output"]["terminal"]["transport_complete"], true);
assert_eq!(
view.read(id, workdir::CommandStream::Stdout, 0, 65536)
.unwrap(),
b"before"
);
assert_eq!(
view.read(id, workdir::CommandStream::Stderr, 0, 65536)
.unwrap(),
b"err-before"
);
}
// Integration: real process pipes and provider replay into the production Bash collector.
// Only response delivery is fault-injected; Runtime's HTTP suite covers actual wire routing.
#[tokio::test]
async fn lost_page_and_terminal_responses_replay_without_restarting_or_double_appending() {
for lose_terminal_only in [false, true] {
let dir = TempDir::new().unwrap();
let session = Arc::new(ExternalValidatingSession {
inner: LocalWorkdirSession::new(
Scope::writable(dir.path()).unwrap(),
dir.path().into(),
),
truncate_first_binary_response: false,
mutate_after_first_binary_response: None,
command_response_losses: std::sync::atomic::AtomicUsize::new(1),
lose_terminal_only,
});
let view = tools::bash_output::BashOutputView::new();
let definitions = core_builtin_tools(session.clone(), Tracker::new(), view.clone());
let bash = Registry::new(definitions).get("Bash");
let out = call(&bash, json!({"cwd":".","command":"printf once >> effect.txt; printf 'x%.0s' {1..200000}; printf err >&2; exit 3"})).await;
let body: serde_json::Value = serde_json::from_str(&out.content.unwrap()).unwrap();
assert_eq!(
std::fs::read(dir.path().join("effect.txt")).unwrap(),
b"once"
);
assert_eq!(
session
.command_response_losses
.load(std::sync::atomic::Ordering::Acquire),
0
);
assert_eq!(body["output"]["stdout_saved"], 200000);
assert_eq!(body["output"]["stdout_delivered"], 200000);
assert_eq!(body["output"]["stderr_saved"], 3);
assert_eq!(body["output"]["terminal"]["exit_code"], 3);
assert_eq!(body["output"]["terminal"]["transport_complete"], true);
assert_eq!(body["output"]["storage_truncated"], false);
let id = body["output"]["id"].as_str().unwrap();
for offset in (0..200000).step_by(65536) {
assert_eq!(
view.read(id, workdir::CommandStream::Stdout, offset, 65536)
.unwrap(),
vec![b'x'; (200000 - offset).min(65536)]
);
}
use workdir::WorkdirSession as _;
session.close().await.unwrap();
}
}
+36 -20
View File
@@ -3013,7 +3013,8 @@ impl App {
/// `[File: …]` parsing path: each kind maps directly to a typed
/// block (`Block::Notify`, `Block::WorkerEvent`, …).
fn apply_system_item(&mut self, value: &serde_json::Value) {
let Ok(item) = serde_json::from_value::<session_store::SystemItem>(value.clone()) else {
let Ok(item) = serde_json::from_value::<session_store::PublicSystemItem>(value.clone())
else {
// Unknown / forward-compat shape: fall back to rendering the
// raw text payload (if any) as a generic system message.
if let Some(text) = value.get("body").and_then(|b| b.as_str()) {
@@ -3024,29 +3025,23 @@ impl App {
}
return;
};
match item {
session_store::SystemItem::Notification { message, .. } => {
let body = item.body;
match item.presentation {
session_store::PublicSystemPresentation::Notification { message } => {
self.blocks.push(Block::Notify { message });
}
session_store::SystemItem::WorkerEvent { event, .. } => {
session_store::PublicSystemPresentation::WorkerEvent { event } => {
self.blocks.push(Block::WorkerEvent { event });
}
session_store::SystemItem::FeatureInvocationResult { body, .. }
| session_store::SystemItem::FileAttachment { body, .. }
| session_store::SystemItem::SkillActivation { body, .. }
| session_store::SystemItem::JournalStartup { body, .. }
| session_store::SystemItem::ResidentSummaryRefresh { body, .. }
| session_store::SystemItem::SubjectBehaviorRefresh { body, .. }
| session_store::SystemItem::Interrupt { body, .. } => {
session_store::PublicSystemPresentation::Context => {
self.task_store.apply_system_message_text(&body);
self.blocks.push(Block::SystemMessage { text: body });
}
session_store::SystemItem::TaskReminder { body, .. } => {
session_store::PublicSystemPresentation::TaskReminder => {
self.task_store.apply_system_message_text(&body);
self.blocks.push(Block::TaskReminder { text: body });
}
session_store::SystemItem::LegacyIgnored { .. } => {}
session_store::SystemItem::LegacyKnowledgeIgnored { .. } => {}
session_store::PublicSystemPresentation::Ignored => {}
}
}
@@ -3572,6 +3567,7 @@ mod completion_flow_tests {
session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::from(item),
metadata: session_store::LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: session_store::LoggedSessionHistoryEntryId::new(),
origin: session_store::LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
@@ -4329,6 +4325,7 @@ mod completion_flow_tests {
}));
app.handle_worker_event(Event::SessionEntryCommitted {
entry: protocol::SessionSnapshotEntry {
use_policy: None,
entry_id: "run-failure".into(),
timestamp: 2,
provenance: protocol::SessionEntryProvenance::LegacyUnknown,
@@ -4450,12 +4447,12 @@ mod completion_flow_tests {
}
#[test]
fn live_legacy_workflow_system_item_is_ignored() {
fn retired_system_input_is_ignored() {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "workflow",
"slug": "build",
"body": "[Workflow /build]\nRun the build",
"kind": "retired_input",
"body": "",
"presentation": { "kind": "ignored" },
});
app.handle_worker_event(Event::SystemItem {
entry_id: None,
@@ -4861,12 +4858,30 @@ mod completion_flow_tests {
);
}
#[test]
fn feature_label_does_not_select_system_input_presentation() {
for label in ["journal_startup", "new-unregistered-feature"] {
let mut app = App::new("test".into());
app.handle_worker_event(Event::SystemItem {
entry_id: None,
item: serde_json::json!({
"kind": label,
"body": "explicit public content",
"presentation": { "kind": "context" },
}),
});
assert!(
matches!(app.blocks.as_slice(), [Block::SystemMessage { text }] if text == "explicit public content")
);
}
}
#[test]
fn live_system_item_notification_appends_notify_block() {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "notification",
"message": "hi",
"presentation": { "kind": "notification", "message": "hi" },
"body": "[Notification] hi",
});
app.handle_worker_event(Event::SystemItem {
@@ -4884,7 +4899,7 @@ mod completion_flow_tests {
let mut app = App::new("test".into());
let item = serde_json::json!({
"kind": "worker_event",
"event": { "kind": "turn_ended", "worker_name": "child" },
"presentation": { "kind": "worker_event", "event": { "kind": "turn_ended", "worker_name": "child" } },
"body": "[Notification] worker `child` finished a turn",
});
app.handle_worker_event(Event::SystemItem {
@@ -5285,6 +5300,7 @@ mod completion_flow_tests {
item: serde_json::json!({
"kind": "task_reminder",
"body": snapshot,
"presentation": { "kind": "task_reminder" },
}),
});
+1
View File
@@ -162,6 +162,7 @@ fn request_from_selection(selection: Selection) -> BackendCreateWorkerRequest {
ticket_assignment: None,
feature_connections: Default::default(),
control_operation_id: None,
execution_grant: None,
}
}
+1 -1
View File
@@ -20,7 +20,7 @@ serde_json.workspace = true
sha2.workspace = true
tempfile.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["process", "rt", "sync", "time"] }
tokio = { workspace = true, features = ["io-util", "process", "rt", "sync", "time"] }
[dev-dependencies]
serde_json.workspace = true
+6 -1
View File
@@ -48,6 +48,10 @@ pub struct CheckoutRequest {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(tag = "operation", rename_all = "snake_case", deny_unknown_fields)]
pub enum CheckoutOperation {
ReadBytes {
offset: u64,
max_bytes: usize,
},
Read {
offset: usize,
limit: usize,
@@ -71,7 +75,7 @@ pub enum CheckoutOperation {
impl CheckoutOperation {
pub fn capability(&self) -> WorkdirSessionCapability {
match self {
Self::Read { .. } => WorkdirSessionCapability::Read,
Self::Read { .. } | Self::ReadBytes { .. } => WorkdirSessionCapability::Read,
Self::Edit { .. } => WorkdirSessionCapability::Edit,
Self::Write { .. } | Self::Create { .. } => WorkdirSessionCapability::Write,
}
@@ -86,6 +90,7 @@ impl CheckoutOperation {
)]
pub enum CheckoutOutput {
Read(ReadResult),
ReadBytes(crate::ReadBytesResult),
Write(WriteResult),
Edit(EditResult),
}
@@ -0,0 +1,43 @@
//! Bounded collection for tests whose subject is command routing/lifecycle,
//! rather than the byte-page protocol itself.
use crate::*;
#[derive(Debug)]
pub(crate) struct CollectedCommand {
pub content: String,
pub status: CommandStatus,
pub exit_code: Option<i32>,
pub timed_out: bool,
}
#[async_trait::async_trait]
pub(crate) trait CollectCommand: WorkdirSession {
async fn collect_command_output(
&self,
mut request: CommandOutputRequest,
) -> Result<CollectedCommand, WorkdirError> {
let mut bytes = Vec::new();
request.wait = true;
loop {
let page = self.command_output(request.clone()).await?;
for chunk in page.chunks {
bytes.extend(chunk.bytes);
assert!(
bytes.len() <= 16 * 1024 * 1024,
"fixture output exceeded bounded test collection"
);
}
request.cursor = page.next_cursor;
if let Some(terminal) = page.terminal {
return Ok(CollectedCommand {
content: String::from_utf8_lossy(&bytes).into_owned(),
status: terminal.status,
exit_code: terminal.exit_code,
timed_out: terminal.status == CommandStatus::TimedOut,
});
}
}
}
}
impl<T: WorkdirSession + ?Sized> CollectCommand for T {}
+154 -38
View File
@@ -9,7 +9,8 @@
//! COMMAND remains separate explicit authority. General session capability intersections may form
//! subsets independently of this grant hierarchy; the codec never implicitly expands nonhierarchical
//! grants. Commands run with the shared host's OS permissions, not a sandbox confining processes
//! to a directory. Output is bounded and inline; provider-local spill paths are not exposed.
//! to a directory. Output uses bounded byte pages with cursor acknowledgements;
//! providers never receive caller-selected output paths.
use std::path::{Component, Path};
@@ -18,7 +19,7 @@ use serde::{Deserialize, Serialize};
use crate::http::{WorkdirSessionOperation, WorkdirSessionOperationResult, WorkdirTransportError};
use crate::{BoundedReadLimits, WorkdirId, WorkdirPath, WorkdirSessionCapabilities};
pub const EXTERNAL_WORKDIR_PROVIDER_PROTOCOL_VERSION: u16 = 2;
pub const EXTERNAL_WORKDIR_PROVIDER_PROTOCOL_VERSION: u16 = 3;
/// Maximum serialized Backend-to-provider command frame size.
pub const MAX_EXTERNAL_SERVER_FRAME_BYTES: usize = 1024 * 1024;
/// Maximum serialized provider-to-Backend result frame size. Read bytes are
@@ -33,9 +34,9 @@ const MAX_EXTERNAL_GREP_CONTEXT: usize = 100;
const MAX_EXTERNAL_SCOPE_RULES: usize = 64;
pub const MAX_EXTERNAL_WRITE_BYTES: usize = 512 * 1024;
pub const MAX_EXTERNAL_COMMAND_BYTES: usize = 64 * 1024;
pub const MAX_EXTERNAL_COMMAND_OUTPUT_BYTES: usize = 512 * 1024;
pub const MAX_EXTERNAL_COMMAND_OUTPUT_BYTES: usize = crate::MAX_COMMAND_READ_BYTES;
pub const MAX_EXTERNAL_COMMAND_TIMEOUT_SECS: u64 = 600;
const MAX_EXTERNAL_COMMAND_CURSOR: usize = 1_000_000;
const MAX_EXTERNAL_COMMAND_CURSOR: u64 = crate::MAX_COMMAND_TRANSPORT_BYTES;
fn valid_command_handle(handle: &crate::CommandHandle) -> bool {
!handle.0.is_empty()
@@ -185,6 +186,11 @@ fn validate_external_operation(operation: &WorkdirSessionOperation) -> Result<()
&& !request.validator.is_empty()
&& request.validator.len() <= 256
&& match &request.operation {
crate::CheckoutOperation::ReadBytes { offset, max_bytes } => {
*offset <= 9_007_199_254_740_991
&& *max_bytes > 0
&& *max_bytes <= BoundedReadLimits::EXTERNAL_DEFAULT.max_response_bytes
}
crate::CheckoutOperation::Read {
limit, max_bytes, ..
} => {
@@ -267,11 +273,10 @@ fn validate_external_operation(operation: &WorkdirSessionOperation) -> Result<()
}
WorkdirSessionOperation::CommandStart(request) => {
!request.command.is_empty()
&& request.validate().is_ok()
&& request.command.len() <= MAX_EXTERNAL_COMMAND_BYTES
&& (1..=MAX_EXTERNAL_COMMAND_TIMEOUT_SECS).contains(&request.timeout_secs)
&& (1..=MAX_EXTERNAL_COMMAND_OUTPUT_BYTES).contains(&request.output_limit)
&& is_root_relative(&request.cwd)
&& request.spill_dir.is_none()
&& request.tool_call_id.as_ref().is_none_or(|tool_call_id| {
!tool_call_id.is_empty()
&& tool_call_id.len() <= MAX_PROTOCOL_ID_BYTES
@@ -375,6 +380,13 @@ fn validate_external_result(result: &WorkdirSessionOperationResult) -> Result<()
WorkdirSessionOperationResult::CheckoutExecute(result) => {
valid_checkout_observation(&result.observation)
&& match &result.output {
crate::CheckoutOutput::ReadBytes(read) => {
read.path == result.observation.path
&& validate_external_result(&WorkdirSessionOperationResult::ReadBytes(
read.clone(),
))
.is_ok()
}
crate::CheckoutOutput::Read(read) => {
read.path == result.observation.path
&& validate_external_result(&WorkdirSessionOperationResult::Read(
@@ -450,11 +462,19 @@ fn validate_external_result(result: &WorkdirSessionOperationResult) -> Result<()
WorkdirSessionOperationResult::CommandStart(handle) => valid_command_handle(handle),
WorkdirSessionOperationResult::CommandStatus(_) => true,
WorkdirSessionOperationResult::CommandOutput(output) => {
output.content.len() <= MAX_EXTERNAL_COMMAND_OUTPUT_BYTES
output.transport_bounds_valid()
&& output.chunks.len() <= MAX_EXTERNAL_COMMAND_OUTPUT_BYTES
&& output
.next_cursor
.is_none_or(|cursor| cursor <= MAX_EXTERNAL_COMMAND_CURSOR)
&& output.output_path.is_none()
.chunks
.iter()
.map(|chunk| chunk.bytes.len())
.sum::<usize>()
<= MAX_EXTERNAL_COMMAND_OUTPUT_BYTES
&& output.next_cursor <= MAX_EXTERNAL_COMMAND_CURSOR
&& output
.terminal
.as_ref()
.is_none_or(|terminal| terminal.status != crate::CommandStatus::Running)
}
WorkdirSessionOperationResult::CommandCancel => true,
};
@@ -505,6 +525,8 @@ pub struct ExternalWorkdirOperationError {
code: crate::http::WorkdirTransportErrorCode,
#[serde(default, skip_serializing_if = "Option::is_none")]
denial_reason: Option<crate::WorkdirDenialReason>,
#[serde(default, skip_serializing_if = "Option::is_none")]
failure_reason: Option<crate::WorkdirFailureReason>,
}
impl ExternalWorkdirOperationError {
@@ -514,11 +536,13 @@ impl ExternalWorkdirOperationError {
Self {
code: error.code,
denial_reason: error.denial_reason,
failure_reason: error.failure_reason,
}
}
pub fn into_transport_error(self) -> WorkdirTransportError {
WorkdirTransportError {
failure_reason: self.failure_reason,
code: self.code,
denial_reason: self.denial_reason,
message: format!("External Workdir provider reported {}", self.code.as_str()),
@@ -742,6 +766,7 @@ mod tests {
] {
let external =
ExternalWorkdirOperationError::from_transport_error(WorkdirTransportError {
failure_reason: None,
code: Code::Denied,
message: "/home/operator/private provider-secret".repeat(4096),
denial_reason: reason,
@@ -768,6 +793,7 @@ mod tests {
#[test]
fn provider_failures_are_closed_and_drop_provider_authored_diagnostics() {
let error = ExternalWorkdirOperationError::from_transport_error(WorkdirTransportError {
failure_reason: None,
code: crate::http::WorkdirTransportErrorCode::Internal,
denial_reason: None,
message: "/home/operator/private provider-secret\ncontrol".to_string(),
@@ -801,11 +827,16 @@ mod tests {
#[test]
fn unsupported_protocol_versions_and_invalid_read_limits_fail_closed() {
let future = r#"{
"version": 3,
"message": {"kind": "heartbeat", "generation": 1, "sequence": 1}
}"#;
assert!(serde_json::from_str::<ExternalWorkdirServerFrame>(future).is_err());
for version in [
EXTERNAL_WORKDIR_PROVIDER_PROTOCOL_VERSION - 1,
EXTERNAL_WORKDIR_PROVIDER_PROTOCOL_VERSION + 1,
] {
let unsupported = serde_json::json!({
"version": version,
"message": {"kind": "heartbeat", "generation": 1, "sequence": 1}
});
assert!(serde_json::from_value::<ExternalWorkdirServerFrame>(unsupported).is_err());
}
let invalid_limits = r#"{
"max_source_bytes": 1024,
@@ -825,32 +856,35 @@ mod tests {
assert!(ExternalWorkdirOperation::try_from(absolute).is_err());
let command = WorkdirSessionOperation::CommandStart(CommandRequest {
command: "cat /home/operator/private/session.log".to_string(),
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf unsafe".to_string(),
timeout_secs: 1,
output_limit: 1024,
cwd: WorkdirPath::root(),
spill_dir: Some("/home/operator/private/output".into()),
cwd: serde_json::from_value(serde_json::json!("/absolute")).unwrap(),
tool_call_id: None,
});
assert!(ExternalWorkdirOperation::try_from(command).is_err());
let command = WorkdirSessionOperation::CommandStart(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf bounded".to_string(),
timeout_secs: 30,
output_limit: 12 * 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: Some("tool-call-1".to_string()),
});
assert!(ExternalWorkdirOperation::try_from(command).is_ok());
assert!(
ExternalWorkdirOperation::try_from(WorkdirSessionOperation::CommandStart(
CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "x".repeat(MAX_EXTERNAL_COMMAND_BYTES + 1),
timeout_secs: 30,
output_limit: 12 * 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
}
))
@@ -887,13 +921,17 @@ mod tests {
assert!(
ExternalWorkdirOperationResult::try_from(WorkdirSessionOperationResult::CommandOutput(
crate::CommandOutput {
status: crate::CommandStatus::Completed,
exit_code: Some(0),
timed_out: false,
content: "bounded".to_string(),
next_cursor: None,
truncated: false,
output_path: None,
chunks: vec![],
next_cursor: 0,
terminal: Some(crate::CommandTerminal {
status: crate::CommandStatus::Completed,
exit_code: Some(0),
stdout_bytes: 0,
stderr_bytes: 0,
stdout_delivered_bytes: 0,
stderr_delivered_bytes: 0,
truncation_reason: None,
})
}
))
.is_ok()
@@ -901,13 +939,14 @@ mod tests {
assert!(
ExternalWorkdirOperationResult::try_from(WorkdirSessionOperationResult::CommandOutput(
crate::CommandOutput {
status: crate::CommandStatus::Completed,
exit_code: Some(0),
timed_out: false,
content: "bounded".to_string(),
next_cursor: None,
truncated: true,
output_path: Some("/provider/private/output".into()),
chunks: vec![crate::CommandByteChunk {
sequence: 0,
stream: crate::CommandStream::Stdout,
offset: 0,
bytes: vec![0; MAX_EXTERNAL_COMMAND_OUTPUT_BYTES + 1]
}],
next_cursor: 0,
terminal: None
}
))
.is_err()
@@ -958,7 +997,7 @@ mod tests {
#[test]
fn protocol_rejects_unknown_nested_fields_and_unbounded_requests() {
let unknown = r#"{
"version": 2,
"version": 3,
"message": {
"kind": "heartbeat",
"generation": 1,
@@ -1012,4 +1051,81 @@ mod tests {
assert!(ExternalWorkdirOperationId::new("x".repeat(129)).is_err());
assert!(ExternalWorkdirOperationId::new("operation-1").is_ok());
}
#[test]
fn command_budget_and_page_metadata_obey_external_hard_bounds() {
fn request(budget: u64) -> WorkdirSessionOperation {
WorkdirSessionOperation::CommandStart(CommandRequest {
command: "true".into(),
timeout_secs: 1,
cwd: WorkdirPath::root(),
transport_byte_budget: budget,
tool_call_id: None,
})
}
for budget in [0, crate::MAX_COMMAND_TRANSPORT_BYTES] {
let operation = ExternalWorkdirOperation::try_from(request(budget)).unwrap();
let wire = serde_json::to_value(operation).unwrap();
assert!(serde_json::from_value::<ExternalWorkdirOperation>(wire).is_ok());
}
assert!(
ExternalWorkdirOperation::try_from(request(crate::MAX_COMMAND_TRANSPORT_BYTES + 1))
.is_err()
);
fn page() -> crate::CommandOutput {
crate::CommandOutput {
chunks: vec![crate::CommandByteChunk {
sequence: 0,
stream: crate::CommandStream::Stdout,
offset: 0,
bytes: b"abc".to_vec(),
}],
next_cursor: 3,
terminal: Some(crate::CommandTerminal {
status: crate::CommandStatus::Completed,
exit_code: Some(0),
stdout_bytes: 100 * 1024 * 1024,
stderr_bytes: 7,
stdout_delivered_bytes: 3,
stderr_delivered_bytes: 0,
truncation_reason: Some(crate::CommandOutputTruncation::ByteBudget),
}),
}
}
let accepted = ExternalWorkdirOperationResult::try_from(
WorkdirSessionOperationResult::CommandOutput(page()),
)
.unwrap();
assert!(
serde_json::from_value::<ExternalWorkdirOperationResult>(
serde_json::to_value(accepted).unwrap()
)
.is_ok()
);
for mutate in [
|p: &mut crate::CommandOutput| p.next_cursor = crate::MAX_COMMAND_TRANSPORT_BYTES + 1,
|p: &mut crate::CommandOutput| p.chunks[0].offset = crate::MAX_COMMAND_TRANSPORT_BYTES,
|p: &mut crate::CommandOutput| p.chunks[0].sequence = u64::MAX,
|p: &mut crate::CommandOutput| p.terminal.as_mut().unwrap().stdout_delivered_bytes = 4,
|p: &mut crate::CommandOutput| p.terminal.as_mut().unwrap().stdout_bytes = 2,
|p: &mut crate::CommandOutput| p.terminal.as_mut().unwrap().truncation_reason = None,
] {
let mut invalid = page();
mutate(&mut invalid);
let wire = serde_json::to_value(WorkdirSessionOperationResult::CommandOutput(invalid))
.unwrap();
assert!(serde_json::from_value::<ExternalWorkdirOperationResult>(wire).is_err());
}
assert!(
ExternalWorkdirOperation::try_from(WorkdirSessionOperation::CommandOutput(
crate::CommandOutputRequest {
handle: crate::CommandHandle("command-1".into()),
cursor: crate::MAX_COMMAND_TRANSPORT_BYTES + 1,
limit: 1,
wait: false,
}
))
.is_err()
);
}
}
+13 -7
View File
@@ -1,6 +1,9 @@
#[cfg(test)]
use crate::command_test_support::CollectCommand as _;
use crate::http::{
WorkdirSessionOperation, WorkdirSessionOperationResult, dispatch_workdir_session_operation,
};
use crate::{
BoundedReadLimits, CommandOutputRequest, CommandRequest, EditRequest, ExternalWorkdirRoot,
GlobRequest, GrepOutputMode, GrepRequest, LocalWorkdirSession, ReadRequest, Workdir,
@@ -59,11 +62,12 @@ async fn external_local_provider_is_strictly_read_only() {
let command = WorkdirSession::start_command(
&session,
CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "echo changed > item.txt".to_string(),
timeout_secs: 1,
output_limit: 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
},
)
@@ -145,11 +149,12 @@ async fn external_local_provider_read_write_is_filesystem_only() {
let command = WorkdirSession::start_command(
&session,
CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "touch command-ran".to_string(),
timeout_secs: 1,
output_limit: 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
},
)
@@ -193,17 +198,18 @@ async fn external_command_permission_is_explicit_and_independent_from_file_acces
let handle = session
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf command-ok".to_string(),
timeout_secs: 5,
output_limit: 1024,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
})
.await
.unwrap();
let output = session
.command_output(CommandOutputRequest {
.collect_command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 1024,
+169 -18
View File
@@ -275,6 +275,8 @@ pub struct WorkdirTransportError {
/// Internal provider diagnostics. Public response boundaries may omit this.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub denial_reason: Option<crate::WorkdirDenialReason>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub failure_reason: Option<crate::WorkdirFailureReason>,
}
impl WorkdirTransportError {
@@ -282,6 +284,11 @@ impl WorkdirTransportError {
pub fn from_workdir_error(error: &WorkdirError) -> Self {
use WorkdirTransportErrorCode as Code;
let (code, message) = match error {
WorkdirError::FailureContext { reason, source } => {
let mut transport = Self::from_workdir_error(source);
transport.failure_reason = Some(*reason);
return transport;
}
WorkdirError::DenialContext { reason, source } => {
let mut transport = Self::from_workdir_error(source);
transport.denial_reason = Some(*reason);
@@ -301,6 +308,7 @@ impl WorkdirTransportError {
code: Code::Unsupported,
message: format!("Workdir capability {capability:?} is not available"),
denial_reason: None,
failure_reason: None,
};
}
WorkdirError::UnsupportedOperation(_) => {
@@ -359,6 +367,7 @@ impl WorkdirTransportError {
code,
message: message.to_string(),
denial_reason: error.denial_reason(),
failure_reason: error.failure_reason(),
}
}
@@ -408,6 +417,11 @@ impl WorkdirTransportError {
Code::Transport => WorkdirError::Transport(self.message),
Code::Internal => WorkdirError::OperationFailed,
};
let error = if let Some(reason) = self.failure_reason {
error.with_failure_reason(reason)
} else {
error
};
// Preserve any supplied reason independently of the public classification.
// Without a reason, retain the existing classification's deliberate fallback.
if let Some(reason) = self.denial_reason {
@@ -557,9 +571,10 @@ mod client {
operation: WorkdirSessionOperation,
) -> Result<WorkdirSessionOperationResult, WorkdirError> {
if self.closed.load(Ordering::Acquire) {
return Err(WorkdirError::Unavailable(
"Workdir session is closed".to_string(),
));
return Err(
WorkdirError::Unavailable("Workdir session is closed".to_string())
.with_failure_reason(crate::WorkdirFailureReason::SessionClosed),
);
}
let url = endpoint(
&self.base_url,
@@ -693,6 +708,12 @@ mod client {
let mutation = request.operation.capability() != crate::WorkdirSessionCapability::Read;
let expected_path = request.target.clone();
let expected_output = request.operation.capability();
let byte_range = match &request.operation {
crate::CheckoutOperation::ReadBytes { offset, max_bytes } => {
Some((*offset, *max_bytes))
}
_ => None,
};
match self
.operate(WorkdirSessionOperation::CheckoutExecute(request))
.await
@@ -702,11 +723,30 @@ mod client {
&& !result.observation.validator.is_empty()
&& result.observation.validator.len() <= 256
&& match (&result.output, expected_output) {
(
crate::CheckoutOutput::ReadBytes(read),
crate::WorkdirSessionCapability::Read,
) => byte_range.is_some_and(|(offset, max)| {
read.path == expected_path
&& read.offset == offset
&& read.bytes.len() <= max
&& read
.offset
.checked_add(read.bytes.len() as u64)
.is_some_and(|end| {
end <= read.total_bytes
|| (read.bytes.is_empty() && read.eof)
})
&& read.eof
== (read.offset.saturating_add(read.bytes.len() as u64)
>= read.total_bytes)
}),
(
crate::CheckoutOutput::Read(read),
crate::WorkdirSessionCapability::Read,
) => {
read.path == expected_path
byte_range.is_none()
&& read.path == expected_path
&& read.bytes.len()
<= crate::BoundedReadLimits::EXTERNAL_DEFAULT
.max_response_bytes
@@ -742,6 +782,10 @@ mod client {
let outcome = WorkdirError::OutcomeUnknown(
"checkout transport failed; effects may have occurred".into(),
);
let outcome = match error.failure_reason() {
Some(reason) => outcome.with_failure_reason(reason),
None => outcome,
};
Err(match error.denial_reason() {
Some(reason) => WorkdirError::DenialContext {
reason,
@@ -822,6 +866,7 @@ mod client {
&self,
request: CommandRequest,
) -> Result<CommandHandle, WorkdirError> {
request.validate()?;
match self
.operate(WorkdirSessionOperation::CommandStart(request))
.await?
@@ -848,15 +893,46 @@ mod client {
&self,
request: CommandOutputRequest,
) -> Result<CommandOutput, WorkdirError> {
let wait = request.wait;
if !(1..=crate::MAX_COMMAND_READ_BYTES).contains(&request.limit) {
return Err(WorkdirError::InvalidArgument(
"command byte limit is outside transport bounds".into(),
));
}
loop {
let started = tokio::time::Instant::now();
match self
.operate(WorkdirSessionOperation::CommandOutput(request.clone()))
.await?
{
WorkdirSessionOperationResult::CommandOutput(result)
if wait && result.status == CommandStatus::Running => {}
WorkdirSessionOperationResult::CommandOutput(result) => return Ok(result),
WorkdirSessionOperationResult::CommandOutput(result) => {
let size = result
.chunks
.iter()
.try_fold(0usize, |size, chunk| size.checked_add(chunk.bytes.len()));
if !result.transport_bounds_valid()
|| size.is_none_or(|size| {
size > request.limit
|| request.cursor.checked_add(size as u64)
!= Some(result.next_cursor)
})
{
return Err(Self::mismatch("bounded command byte page"));
}
if request.wait && result.chunks.is_empty() && result.terminal.is_none() {
// A normal long-poll expiry is not a transport failure.
// Continue at the same cursor, without spending the caller's
// error retry budget or acknowledging any bytes. Pace peers
// that return empty pages immediately; a real long poll has
// already consumed this delay. The caller owns cancellation
// and the overall deadline.
tokio::time::sleep_until(
started + std::time::Duration::from_millis(100),
)
.await;
continue;
}
return Ok(result);
}
_ => return Err(Self::mismatch("command_output")),
}
}
@@ -917,24 +993,64 @@ mod client {
response: reqwest::Response,
) -> Result<T, WorkdirError> {
if response.status().is_success() {
response.json().await.map_err(http_unavailable)
let bytes = bounded_response_bytes(response).await?;
serde_json::from_slice(&bytes).map_err(|_| {
WorkdirError::Transport("Runtime Workdir response was invalid".into())
.with_failure_reason(crate::WorkdirFailureReason::HttpResponseInvalid)
})
} else {
Err(decode_error(response).await)
}
}
async fn decode_error(response: reqwest::Response) -> WorkdirError {
response
.json::<WorkdirTransportError>()
.await
.map(WorkdirTransportError::into_workdir_error)
.unwrap_or_else(|_| {
WorkdirError::Transport("Runtime Workdir error response was invalid".to_string())
})
async fn bounded_response_bytes(
mut response: reqwest::Response,
) -> Result<Vec<u8>, WorkdirError> {
const MAX_RESPONSE_BYTES: usize = crate::external::MAX_EXTERNAL_PROVIDER_FRAME_BYTES;
if response
.content_length()
.is_some_and(|size| size > MAX_RESPONSE_BYTES as u64)
{
return Err(WorkdirError::Transport(
"Runtime Workdir response exceeded transport bound".into(),
)
.with_failure_reason(crate::WorkdirFailureReason::HttpResponseTooLarge));
}
let mut bytes = Vec::new();
while let Some(chunk) = response.chunk().await.map_err(http_unavailable)? {
if chunk.len() > MAX_RESPONSE_BYTES.saturating_sub(bytes.len()) {
return Err(WorkdirError::Transport(
"Runtime Workdir response exceeded transport bound".into(),
)
.with_failure_reason(crate::WorkdirFailureReason::HttpResponseTooLarge));
}
bytes.extend_from_slice(&chunk);
}
Ok(bytes)
}
fn http_unavailable(_error: reqwest::Error) -> WorkdirError {
async fn decode_error(response: reqwest::Response) -> WorkdirError {
match bounded_response_bytes(response).await {
Ok(bytes) => serde_json::from_slice::<WorkdirTransportError>(&bytes)
.map(WorkdirTransportError::into_workdir_error)
.unwrap_or_else(|_| {
WorkdirError::Transport("Runtime Workdir error response was invalid".into())
.with_failure_reason(crate::WorkdirFailureReason::HttpResponseInvalid)
}),
Err(error) => error,
}
}
fn http_unavailable(error: reqwest::Error) -> WorkdirError {
let reason = if error.is_timeout() {
crate::WorkdirFailureReason::HttpTimeout
} else if error.is_connect() {
crate::WorkdirFailureReason::HttpConnect
} else {
crate::WorkdirFailureReason::HttpTransfer
};
WorkdirError::Transport("Runtime Workdir HTTP request failed".to_string())
.with_failure_reason(reason)
}
#[cfg(test)]
@@ -942,6 +1058,40 @@ mod client {
use super::*;
use std::sync::Mutex;
#[tokio::test]
async fn http_failure_reasons_do_not_copy_urls_or_credentials() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let client = Client::builder().no_proxy().build().unwrap();
// A listening socket that never replies makes a real HTTP timeout.
let error = client
.get(format!("http://{address}/private?token=secret"))
.timeout(std::time::Duration::from_millis(25))
.send()
.await
.unwrap_err();
assert!(error.is_timeout());
let error = http_unavailable(error);
assert_eq!(
error.failure_reason(),
Some(crate::WorkdirFailureReason::HttpTimeout)
);
assert!(!format!("{error:?}").contains("secret"));
assert!(!format!("{error:?}").contains("/private"));
drop(listener);
let error = client
.get(format!("http://{address}/private?token=secret"))
.timeout(std::time::Duration::from_secs(2))
.send()
.await
.unwrap_err();
assert!(error.is_connect());
assert_eq!(
http_unavailable(error).failure_reason(),
Some(crate::WorkdirFailureReason::HttpConnect)
);
}
#[derive(Debug, Default)]
struct RejectAuthorization {
requests: Arc<Mutex<Vec<(String, String, Vec<u8>)>>>,
@@ -1118,6 +1268,7 @@ mod tests {
(WorkdirTransportErrorCode::Internal, 500, "operation failed"),
] {
let transport = WorkdirTransportError {
failure_reason: None,
code,
denial_reason: None,
message: if code == WorkdirTransportErrorCode::Conflict {
+58 -1
View File
@@ -401,8 +401,44 @@ impl From<&str> for WorkdirDenial {
}
}
/// Bounded operational diagnostics, never provider text, paths or credentials.
/// Reasons describe an observation, not retry permission or proof of no effects.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum WorkdirFailureReason {
SessionClosed,
ProviderUnavailable,
CommandTerminalMissing,
HttpTimeout,
HttpConnect,
HttpTransfer,
HttpResponseInvalid,
HttpResponseTooLarge,
}
impl WorkdirFailureReason {
pub const fn as_str(self) -> &'static str {
match self {
Self::SessionClosed => "session_closed",
Self::ProviderUnavailable => "provider_unavailable",
Self::CommandTerminalMissing => "command_terminal_missing",
Self::HttpTimeout => "http_timeout",
Self::HttpConnect => "http_connect",
Self::HttpTransfer => "http_transfer",
Self::HttpResponseInvalid => "http_response_invalid",
Self::HttpResponseTooLarge => "http_response_too_large",
}
}
}
#[derive(Debug, thiserror::Error)]
pub enum WorkdirError {
#[error("{source}")]
FailureContext {
reason: WorkdirFailureReason,
#[source]
source: Box<WorkdirError>,
},
#[error("Workdir operation denied: {0}")]
Denied(WorkdirDenial),
@@ -521,12 +557,29 @@ impl WorkdirError {
/// and `denial_reason`: enrichment must not alter behavior or retry policy.
pub fn classification_source(&self) -> &Self {
let mut error = self;
while let Self::DenialContext { source, .. } = error {
while let Self::DenialContext { source, .. } | Self::FailureContext { source, .. } = error {
error = source;
}
error
}
pub fn with_failure_reason(self, reason: WorkdirFailureReason) -> Self {
Self::FailureContext {
reason,
source: Box::new(self),
}
}
pub fn failure_reason(&self) -> Option<WorkdirFailureReason> {
match self {
Self::FailureContext { reason, .. } => Some(*reason),
Self::DenialContext { source, .. } => source.failure_reason(),
Self::SessionClosed => Some(WorkdirFailureReason::SessionClosed),
Self::Unavailable(_) => Some(WorkdirFailureReason::ProviderUnavailable),
_ => None,
}
}
pub fn denied(reason: WorkdirDenialReason, message: impl Into<String>) -> Self {
Self::Denied(WorkdirDenial {
reason: Some(reason),
@@ -540,6 +593,7 @@ impl WorkdirError {
match self {
Self::Denied(denial) => denial.reason,
Self::DenialContext { reason, .. } => Some(*reason),
Self::FailureContext { source, .. } => source.denial_reason(),
Self::OutOfScope(_) => Some(Reason::PathOutOfScope),
Self::SymlinkOutOfScope { .. } => Some(Reason::SymlinkTargetOutOfScope),
Self::ReadOnly(_) => Some(Reason::PathReadOnly),
@@ -618,3 +672,6 @@ mod capability_tests {
assert_eq!(bits(WorkdirSessionCapabilities::ALL), 63);
}
}
#[cfg(test)]
mod command_test_support;
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,378 @@
//! Captured process bytes use a bounded queue, never the lossy Console channel.
//! Pipe backpressure bounds unread process output without temporary-file growth.
//! A replay page survives a lost HTTP response until its next cursor is sent.
use super::*;
use tokio::io::{AsyncRead, AsyncReadExt};
use tokio::sync::mpsc;
const QUEUE_CHUNKS: usize = 8;
pub(super) const MAX_COMMANDS: usize = 64;
#[derive(Debug)]
enum Message {
Bytes(crate::CommandByteChunk),
}
#[derive(Debug)]
pub(super) struct LocalCommand {
pub task: JoinHandle<()>,
pub cancel: watch::Sender<bool>,
pub status: watch::Receiver<Option<crate::CommandTerminal>>,
pub transport: Arc<Mutex<Transport>>,
}
#[derive(Debug)]
pub(super) struct Transport {
receiver: mpsc::Receiver<Message>,
pending: Option<crate::CommandByteChunk>,
cursor: u64,
replay: Option<(u64, CommandOutput)>,
terminal: Option<crate::CommandTerminal>,
terminal_status: watch::Receiver<Option<crate::CommandTerminal>>,
}
impl Transport {
pub async fn read(
&mut self,
request: &CommandOutputRequest,
) -> Result<CommandOutput, WorkdirError> {
if !(1..=crate::MAX_COMMAND_READ_BYTES).contains(&request.limit) {
return Err(WorkdirError::Conflict(
"command byte limit is outside transport bounds".into(),
));
}
if let Some((cursor, page)) = &self.replay {
if request.cursor == *cursor {
if page
.chunks
.iter()
.map(|chunk| chunk.bytes.len())
.sum::<usize>()
> request.limit
{
return Err(WorkdirError::Conflict(
"retry byte limit is smaller than retained page".into(),
));
}
return Ok(page.clone());
}
if request.cursor != page.next_cursor {
return Err(WorkdirError::Conflict(
"command cursor is not the current page acknowledgement".into(),
));
}
self.cursor = page.next_cursor;
self.replay = None;
}
if request.cursor != self.cursor {
return Err(WorkdirError::Conflict(
"command cursor was acknowledged or is ahead of output".into(),
));
}
let mut page = CommandOutput {
chunks: Vec::new(),
next_cursor: self.cursor,
terminal: self.terminal.clone(),
};
let mut remaining = request.limit;
loop {
let message = if let Some(chunk) = self.pending.take() {
Some(Message::Bytes(chunk))
} else if page.chunks.is_empty() && page.terminal.is_none() && request.wait {
self.receiver.recv().await
} else {
match self.receiver.try_recv() {
Ok(message) => Some(message),
Err(mpsc::error::TryRecvError::Empty) => break,
Err(mpsc::error::TryRecvError::Disconnected) => None,
}
};
match message {
Some(Message::Bytes(mut chunk)) => {
let count = remaining.min(chunk.bytes.len());
if count < chunk.bytes.len() {
let tail = chunk.bytes.split_off(count);
self.pending = Some(crate::CommandByteChunk {
sequence: chunk.sequence + count as u64,
stream: chunk.stream,
offset: chunk.offset + count as u64,
bytes: tail,
});
}
page.next_cursor += count as u64;
remaining -= count;
page.chunks.push(chunk);
if remaining == 0 {
break;
}
}
None if page.terminal.is_some() => break,
None => {
let terminal = self.terminal_status.borrow().clone().ok_or_else(|| {
WorkdirError::Unavailable(
"command transport ended before terminal fence".into(),
)
.with_failure_reason(crate::WorkdirFailureReason::CommandTerminalMissing)
})?;
self.terminal = Some(terminal.clone());
page.terminal = Some(terminal);
break;
}
}
}
// An empty poll must not become a permanently replayed empty page.
if !page.chunks.is_empty() || page.terminal.is_some() {
self.replay = Some((self.cursor, page.clone()));
}
Ok(page)
}
pub fn terminal_delivered(&self) -> bool {
self.replay
.as_ref()
.is_some_and(|(_, page)| page.terminal.is_some())
}
pub fn close(&mut self) {
self.receiver.close();
}
}
pub(super) fn start(
cwd: PathBuf,
request: CommandRequest,
command_id: String,
telemetry: CommandTelemetry,
environment: BTreeMap<String, String>,
resources: Vec<Arc<dyn WorkdirSessionResource>>,
) -> LocalCommand {
let (sender, receiver) = mpsc::channel(QUEUE_CHUNKS);
let (cancel, cancel_rx) = watch::channel(false);
let (status_tx, status) = watch::channel(None);
let transport = Arc::new(Mutex::new(Transport {
receiver,
pending: None,
cursor: 0,
replay: None,
terminal: None,
terminal_status: status.clone(),
}));
let task = tokio::spawn(async move {
let _resources = resources;
let terminal = execute(
cwd,
request,
&command_id,
&telemetry,
environment,
cancel_rx,
&sender,
)
.await;
telemetry.terminal(&command_id, terminal.status, terminal.exit_code);
// Closing the byte queue fences this already-published terminal, without
// requiring spare queue capacity or a receiver to release task resources.
let _ = status_tx.send(Some(terminal));
});
LocalCommand {
task,
cancel,
status,
transport,
}
}
#[derive(Default)]
struct ByteAccounting {
stdout: u64,
stderr: u64,
stdout_delivered: u64,
stderr_delivered: u64,
}
impl ByteAccounting {
fn terminal(&self, status: CommandStatus, exit_code: Option<i32>) -> crate::CommandTerminal {
crate::CommandTerminal {
status,
exit_code,
stdout_bytes: self.stdout,
stderr_bytes: self.stderr,
stdout_delivered_bytes: self.stdout_delivered,
stderr_delivered_bytes: self.stderr_delivered,
truncation_reason: (self.stdout > self.stdout_delivered
|| self.stderr > self.stderr_delivered)
.then_some(crate::CommandOutputTruncation::ByteBudget),
}
}
}
async fn pump<R: AsyncRead + Unpin>(
mut stdout: R,
mut stderr: impl AsyncRead + Unpin,
sender: &mpsc::Sender<Message>,
telemetry: &CommandTelemetry,
command_id: &str,
mut remaining: u64,
accounting: &mut ByteAccounting,
) -> std::io::Result<()> {
let mut stdout_done = false;
let mut stderr_done = false;
let mut stdout_decoder = CommandOutputDecoder::default();
let mut stderr_decoder = CommandOutputDecoder::default();
let mut sequence = 0;
let mut stdout_buffer = [0; COMMAND_EVENT_CHUNK_BYTES];
let mut stderr_buffer = [0; COMMAND_EVENT_CHUNK_BYTES];
while !stdout_done || !stderr_done {
// After quota, drain locally even if nobody acknowledges the queue.
// Before quota, reserve first to retain bounded pipe backpressure.
let permit = if remaining > 0 {
let Ok(permit) = sender.reserve().await else {
return Ok(());
};
Some(permit)
} else {
None
};
let (stream, result) = tokio::select! {
result = stdout.read(&mut stdout_buffer), if !stdout_done => (CommandStream::Stdout, result),
result = stderr.read(&mut stderr_buffer), if !stderr_done => (CommandStream::Stderr, result),
};
let buffer = match stream {
CommandStream::Stdout => &stdout_buffer,
CommandStream::Stderr => &stderr_buffer,
};
let count = result?;
let decoder = match stream {
CommandStream::Stdout => &mut stdout_decoder,
CommandStream::Stderr => &mut stderr_decoder,
};
if count == 0 {
match stream {
CommandStream::Stdout => stdout_done = true,
CommandStream::Stderr => stderr_done = true,
}
publish_decoded_output(decoder, telemetry, command_id, stream, true);
continue;
}
let (seen, delivered) = match stream {
CommandStream::Stdout => (&mut accounting.stdout, &mut accounting.stdout_delivered),
CommandStream::Stderr => (&mut accounting.stderr, &mut accounting.stderr_delivered),
};
*seen = seen.saturating_add(count as u64);
let captured = remaining.min(count as u64) as usize;
if captured > 0 {
let offset = *delivered;
decoder.read_offset += captured as u64;
decoder.pending.extend_from_slice(&buffer[..captured]);
publish_decoded_output(decoder, telemetry, command_id, stream, false);
permit
.expect("reserved while budget remains")
.send(Message::Bytes(crate::CommandByteChunk {
sequence,
stream,
offset,
bytes: buffer[..captured].to_vec(),
}));
*delivered += captured as u64;
sequence += captured as u64;
remaining -= captured as u64;
}
}
Ok(())
}
async fn execute(
cwd: PathBuf,
request: CommandRequest,
command_id: &str,
telemetry: &CommandTelemetry,
environment: BTreeMap<String, String>,
mut cancel: watch::Receiver<bool>,
sender: &mpsc::Sender<Message>,
) -> crate::CommandTerminal {
telemetry.started(command_id, request.tool_call_id);
let mut command = Command::new("bash");
command
.arg("-c")
.arg(request.command)
.current_dir(cwd)
.envs(environment)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.kill_on_drop(true);
#[cfg(unix)]
command.as_std_mut().process_group(0);
let mut accounting = ByteAccounting::default();
let Ok(mut child) = command.spawn() else {
return accounting.terminal(CommandStatus::Failed, None);
};
let mut process_group = CommandProcessGroupGuard::new(child.id());
let stdout = child.stdout.take().expect("piped stdout");
let stderr = child.stderr.take().expect("piped stderr");
let (status, exit_code) = {
let output = pump(
stdout,
stderr,
sender,
telemetry,
command_id,
request.transport_byte_budget,
&mut accounting,
);
tokio::pin!(output);
let mut drained = false;
let timeout = tokio::time::sleep(Duration::from_secs(request.timeout_secs.max(1)));
tokio::pin!(timeout);
let terminal = loop {
tokio::select! {
result = &mut output, if !drained => {
drained = true;
if result.is_err() || sender.is_closed() {
process_group.terminate();
let _ = child.start_kill();
let _ = child.wait().await;
break (CommandStatus::Failed, None);
}
}
exit = child.wait() => {
process_group.terminate();
break match exit {
Ok(exit) => (if exit.success() { CommandStatus::Completed } else { CommandStatus::Failed }, exit.code()),
Err(_) => (CommandStatus::Failed, None),
};
}
_ = &mut timeout => {
process_group.terminate();
let _ = child.start_kill();
let code = child.wait().await.ok().and_then(|exit| exit.code());
break (CommandStatus::TimedOut, code);
}
_ = sender.closed() => {
process_group.terminate();
let _ = child.start_kill();
let _ = child.wait().await;
break (CommandStatus::Cancelled, None);
}
changed = cancel.changed() => {
if changed.is_err() || *cancel.borrow() {
process_group.terminate();
let _ = child.start_kill();
let code = child.wait().await.ok().and_then(|exit| exit.code());
break (CommandStatus::Cancelled, code);
}
}
}
};
// Even after kill, preserve the bounded pipe tail before publishing terminal.
if !drained {
let _ = output.await;
}
terminal
};
accounting.terminal(status, exit_code)
}
#[cfg(test)]
mod tests;
@@ -0,0 +1,522 @@
//! Process/pipe integration through the production local provider. Temporary roots
//! and provider tasks are RAII-owned; byte pulls and bounded timeouts synchronize tests.
use super::*;
use tempfile::TempDir;
#[tokio::test]
async fn missing_terminal_is_diagnosed_without_changing_error_classification() {
let (sender, receiver) = mpsc::channel(1);
let (_status, terminal_status) = watch::channel(None);
drop(sender);
let mut transport = Transport {
receiver,
pending: None,
cursor: 0,
replay: None,
terminal: None,
terminal_status,
};
let error = transport
.read(&CommandOutputRequest {
handle: CommandHandle("diagnostic-test".into()),
cursor: 0,
limit: 16,
wait: true,
})
.await
.unwrap_err();
assert_eq!(
error.failure_reason(),
Some(crate::WorkdirFailureReason::CommandTerminalMissing)
);
assert!(matches!(
error.classification_source(),
WorkdirError::Unavailable(_)
));
}
struct Harness {
session: LocalWorkdirSession,
_root: TempDir,
}
impl Harness {
fn new() -> Self {
let root = TempDir::new().unwrap();
let session = LocalWorkdirSession::new(
Scope::writable(root.path()).unwrap(),
root.path().to_path_buf(),
);
Self {
session,
_root: root,
}
}
async fn start(&self, command: &str, timeout_secs: u64) -> CommandHandle {
self.start_budget(command, timeout_secs, crate::MAX_COMMAND_TRANSPORT_BYTES)
.await
}
fn request(command: &str, timeout_secs: u64, budget: u64) -> CommandRequest {
CommandRequest {
transport_byte_budget: budget,
command: command.into(),
timeout_secs,
cwd: WorkdirPath::root(),
tool_call_id: None,
}
}
async fn start_budget(&self, command: &str, timeout_secs: u64, budget: u64) -> CommandHandle {
self.session
.start_command(Self::request(command, timeout_secs, budget))
.await
.unwrap()
}
async fn page(&self, handle: &CommandHandle, cursor: u64, limit: usize) -> CommandOutput {
tokio::time::timeout(
Duration::from_secs(10),
self.session.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor,
limit,
wait: true,
}),
)
.await
.expect("bounded command pull")
.unwrap()
}
async fn collect(
&self,
handle: &CommandHandle,
mut cursor: u64,
) -> (Vec<crate::CommandByteChunk>, crate::CommandTerminal) {
let mut chunks = Vec::new();
loop {
let page = self
.page(handle, cursor, crate::MAX_COMMAND_READ_BYTES)
.await;
assert!(
page.chunks
.iter()
.map(|chunk| chunk.bytes.len())
.sum::<usize>()
<= crate::MAX_COMMAND_READ_BYTES
);
cursor = page.next_cursor;
chunks.extend(page.chunks);
assert!(chunks.iter().map(|chunk| chunk.bytes.len()).sum::<usize>() <= 8 * 1024 * 1024);
if let Some(terminal) = page.terminal {
return (chunks, terminal);
}
}
}
}
fn stream_bytes(chunks: &[crate::CommandByteChunk], stream: CommandStream) -> Vec<u8> {
let mut bytes = Vec::new();
for chunk in chunks.iter().filter(|chunk| chunk.stream == stream) {
assert_eq!(chunk.offset, bytes.len() as u64);
bytes.extend_from_slice(&chunk.bytes);
}
bytes
}
#[tokio::test]
async fn high_single_line_output_survives_absent_and_lagged_console_subscribers() {
for lagged in [false, true] {
let harness = Harness::new();
let _lagged_receiver = lagged.then(|| harness.session.subscribe_command_events().unwrap());
let handle = harness.start("head -c 3145728 /dev/zero | tr '\\0' x; head -c 65536 /dev/zero | tr '\\0' e >&2; exit 7", 20).await;
let (chunks, terminal) = harness.collect(&handle, 0).await;
assert_eq!(terminal.status, CommandStatus::Failed);
assert_eq!(terminal.exit_code, Some(7));
assert_eq!(
stream_bytes(&chunks, CommandStream::Stdout),
vec![b'x'; 3145728]
);
assert_eq!(
stream_bytes(&chunks, CommandStream::Stderr),
vec![b'e'; 65536]
);
let mut cursor = 0;
for chunk in chunks {
assert_eq!(chunk.sequence, cursor);
cursor += chunk.bytes.len() as u64;
}
harness.session.close().await.unwrap();
}
}
#[tokio::test]
async fn one_byte_pages_preserve_raw_utf8_invalid_bytes_and_replay_until_acknowledged() {
let harness = Harness::new();
let handle = harness
.start(
"printf '\\342\\202\\254\\377\\000'; printf '\\346\\227\\245' >&2",
5,
)
.await;
let mut chunks = Vec::new();
let mut cursor = 0;
loop {
let page = harness.page(&handle, cursor, 1).await;
assert_eq!(harness.page(&handle, cursor, 1).await, page);
cursor = page.next_cursor;
chunks.extend(page.chunks);
if let Some(terminal) = page.terminal {
assert_eq!(terminal.status, CommandStatus::Completed);
break;
}
}
assert_eq!(
stream_bytes(&chunks, CommandStream::Stdout),
b"\xe2\x82\xac\xff\0"
);
assert_eq!(
stream_bytes(&chunks, CommandStream::Stderr),
b"\xe6\x97\xa5"
);
assert!(
harness
.session
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 1,
wait: false
})
.await
.is_err()
);
}
#[tokio::test]
async fn cancellation_while_backpressured_drains_bytes_before_terminal_fence() {
let harness = Harness::new();
let handle = harness.start("printf ready; exec yes x", 30).await;
let first = harness.page(&handle, 0, 1).await;
harness
.session
.cancel_command(handle.clone())
.await
.unwrap();
let (mut chunks, terminal) = harness.collect(&handle, first.next_cursor).await;
let mut all = first.chunks;
all.append(&mut chunks);
assert!(stream_bytes(&all, CommandStream::Stdout).starts_with(b"ready"));
assert_eq!(terminal.status, CommandStatus::Cancelled);
let final_page = harness
.page(
&handle,
all.iter().map(|chunk| chunk.bytes.len() as u64).sum(),
1,
)
.await;
assert!(final_page.chunks.is_empty());
assert_eq!(final_page.terminal, Some(terminal));
}
#[tokio::test]
async fn timeout_and_session_close_finalize_without_an_output_subscriber() {
let harness = Harness::new();
let handle = harness
.start("printf ready; exec tail -f /dev/null", 1)
.await;
let (chunks, terminal) = harness.collect(&handle, 0).await;
assert_eq!(stream_bytes(&chunks, CommandStream::Stdout), b"ready");
assert_eq!(terminal.status, CommandStatus::TimedOut);
let handle = harness.start("exec yes x", 30).await;
// Receiving one byte establishes actual process output, without a timing sleep.
harness.page(&handle, 0, 1).await;
tokio::time::timeout(Duration::from_secs(5), harness.session.close())
.await
.unwrap()
.unwrap();
assert!(harness.session.command_status(handle).await.is_err());
}
#[tokio::test]
async fn interleaved_streams_keep_independent_offsets_and_bounded_read_validation() {
let harness = Harness::new();
let handle = harness
.start(
"for ((i=0;i<100;i++)); do printf 'out%s;' \"$i\"; printf 'err%s;' \"$i\" >&2; done",
5,
)
.await;
for limit in [0, crate::MAX_COMMAND_READ_BYTES + 1] {
assert!(
harness
.session
.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor: 0,
limit,
wait: false
})
.await
.is_err()
);
}
let (chunks, terminal) = harness.collect(&handle, 0).await;
let expected = |prefix| {
(0..100)
.map(|i| format!("{prefix}{i};"))
.collect::<String>()
.into_bytes()
};
assert_eq!(
stream_bytes(&chunks, CommandStream::Stdout),
expected("out")
);
assert_eq!(
stream_bytes(&chunks, CommandStream::Stderr),
expected("err")
);
assert_eq!(terminal.status, CommandStatus::Completed);
}
#[tokio::test]
async fn byte_budget_zero_exact_partial_and_shared_keep_seen_and_delivered_distinct() {
let harness = Harness::new();
for (command, budget, stdout_seen, stderr_seen, exit_code) in [
("printf abc; printf defg >&2", 0, 3, 4, 0),
("printf abc", 3, 3, 0, 0),
("printf abcdefg", 3, 7, 0, 0),
("printf abcdefg; exit 7", 3, 7, 0, 7),
(
"head -c 32768 /dev/zero; head -c 32768 /dev/zero >&2",
12345,
32768,
32768,
0,
),
] {
let handle = harness.start_budget(command, 5, budget).await;
let (chunks, terminal) = harness.collect(&handle, 0).await;
let stdout = stream_bytes(&chunks, CommandStream::Stdout);
let stderr = stream_bytes(&chunks, CommandStream::Stderr);
let mut cursor = 0;
for chunk in &chunks {
assert_eq!(chunk.sequence, cursor);
cursor += chunk.bytes.len() as u64;
}
assert_eq!(cursor, budget.min(stdout_seen + stderr_seen));
assert_eq!(terminal.stdout_bytes, stdout_seen);
assert_eq!(terminal.stderr_bytes, stderr_seen);
assert_eq!(terminal.stdout_delivered_bytes, stdout.len() as u64);
assert_eq!(terminal.stderr_delivered_bytes, stderr.len() as u64);
assert_eq!(terminal.exit_code, Some(exit_code));
assert_eq!(
terminal.status,
if exit_code == 0 {
CommandStatus::Completed
} else {
CommandStatus::Failed
}
);
assert_eq!(
terminal.truncation_reason,
(cursor < stdout_seen + stderr_seen)
.then_some(crate::CommandOutputTruncation::ByteBudget)
);
}
}
#[tokio::test]
async fn exhausted_budget_finishes_without_receiver_and_keeps_timeout_cancel_distinct() {
let harness = Harness::new();
for budget in [0, 1, (QUEUE_CHUNKS * COMMAND_EVENT_CHUNK_BYTES) as u64] {
let handle = harness
.start_budget("head -c 1048576 /dev/zero; printf error >&2", 5, budget)
.await;
let mut status = harness.session.inner.commands.lock().await[&handle.0]
.status
.clone();
let terminal = tokio::time::timeout(
Duration::from_secs(5),
status.wait_for(|value| value.is_some()),
)
.await
.unwrap()
.unwrap()
.clone()
.unwrap();
assert_eq!(terminal.status, CommandStatus::Completed);
assert_eq!(terminal.stdout_bytes, 1048576);
assert_eq!(terminal.stderr_bytes, 5);
assert_eq!(
terminal.stdout_delivered_bytes + terminal.stderr_delivered_bytes,
budget
);
// Full capture queue cannot retain the process task or its resources.
{
let mut commands = harness.session.inner.commands.lock().await;
tokio::time::timeout(
Duration::from_secs(5),
&mut commands.get_mut(&handle.0).unwrap().task,
)
.await
.unwrap()
.unwrap();
}
// Only now acknowledge the captured queue and terminal fence.
let (_, delivered_terminal) = harness.collect(&handle, 0).await;
assert_eq!(delivered_terminal, terminal);
}
for (cancel, expected) in [
(false, CommandStatus::TimedOut),
(true, CommandStatus::Cancelled),
] {
let handle = harness
.start_budget(
"printf ready-extra; head -c 32768 /dev/zero; exec tail -f /dev/null",
1,
5,
)
.await;
let first = harness.page(&handle, 0, 5).await;
assert_eq!(first.next_cursor, 5);
if cancel {
harness
.session
.cancel_command(handle.clone())
.await
.unwrap();
}
let (rest, terminal) = harness.collect(&handle, first.next_cursor).await;
assert!(rest.is_empty());
assert_eq!(terminal.status, expected);
assert_eq!(terminal.stdout_delivered_bytes, 5);
assert!(terminal.stdout_bytes >= 11);
assert_eq!(
terminal.truncation_reason,
Some(crate::CommandOutputTruncation::ByteBudget)
);
if !cancel {
assert_eq!(terminal.stdout_bytes, 32779);
assert_eq!(
terminal.truncation_reason,
Some(crate::CommandOutputTruncation::ByteBudget)
);
}
}
}
#[tokio::test]
async fn budget_replay_preserves_unique_bytes_and_rejects_future_and_old_cursors() {
let harness = Harness::new();
let handle = harness.start_budget("printf abcdefghij", 5, 7).await;
let first = harness.page(&handle, 0, 2).await;
for _ in 0..3 {
assert_eq!(harness.page(&handle, 0, 2).await, first);
}
assert!(
harness
.session
.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor: 8,
limit: 2,
wait: false,
})
.await
.is_err()
);
let (rest, terminal) = harness.collect(&handle, first.next_cursor).await;
let mut all = first.chunks;
all.extend(rest);
assert_eq!(stream_bytes(&all, CommandStream::Stdout), b"abcdefg");
assert_eq!(terminal.stdout_bytes, 10);
assert_eq!(terminal.stdout_delivered_bytes, 7);
assert!(
harness
.session
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 2,
wait: false,
})
.await
.is_err()
);
}
#[tokio::test]
async fn invalid_budget_is_rejected_before_process_start() {
let harness = Harness::new();
assert!(matches!(
harness
.session
.start_command(Harness::request(
"touch must-not-exist",
5,
crate::MAX_COMMAND_TRANSPORT_BYTES + 1,
))
.await,
Err(WorkdirError::InvalidArgument(_))
));
assert!(!harness._root.path().join("must-not-exist").exists());
assert!(harness.session.inner.commands.lock().await.is_empty());
}
#[tokio::test]
async fn slow_receiver_backpressures_before_quota_but_full_queue_does_not_block_discard() {
use std::future::Future;
use std::task::Poll;
let telemetry = CommandTelemetry::new();
let input = vec![b'x'; 1024 * 1024];
for budget in [
crate::MAX_COMMAND_TRANSPORT_BYTES,
(QUEUE_CHUNKS * COMMAND_EVENT_CHUNK_BYTES) as u64,
] {
let (sender, mut receiver) = mpsc::channel(QUEUE_CHUNKS);
let mut accounting = ByteAccounting::default();
{
let output = pump(
input.as_slice(),
&b""[..],
&sender,
&telemetry,
"test",
budget,
&mut accounting,
);
tokio::pin!(output);
let polled = std::future::poll_fn(|cx| Poll::Ready(output.as_mut().poll(cx))).await;
if budget == crate::MAX_COMMAND_TRANSPORT_BYTES {
assert!(polled.is_pending(), "a slow receiver must stop pipe reads");
} else {
assert!(
matches!(polled, Poll::Ready(Ok(()))),
"discard must not reserve queue capacity"
);
}
}
assert_eq!(receiver.len(), QUEUE_CHUNKS);
assert_eq!(
accounting.stdout_delivered,
(QUEUE_CHUNKS * COMMAND_EVENT_CHUNK_BYTES) as u64
);
assert_eq!(
accounting.stdout,
if budget == crate::MAX_COMMAND_TRANSPORT_BYTES {
accounting.stdout_delivered
} else {
input.len() as u64
}
);
let mut queued_bytes = 0;
while let Ok(Message::Bytes(chunk)) = receiver.try_recv() {
queued_bytes += chunk.bytes.len();
}
assert_eq!(queued_bytes, QUEUE_CHUNKS * COMMAND_EVENT_CHUNK_BYTES);
}
}
+162 -19
View File
@@ -1,5 +1,3 @@
use std::path::PathBuf;
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
@@ -8,18 +6,18 @@ use serde::{Deserialize, Serialize};
pub struct CommandHandle(pub String);
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CommandRequest {
pub command: String,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub timeout_secs: u64,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub output_limit: usize,
/// Aggregate unique-byte capture allowance, independent of consumer storage.
/// Providers drain and count excess bytes without delivering their payload.
#[schemars(range(min = 0, max = 67_108_864_u64))]
pub transport_byte_budget: u64,
/// Workdir-relative command directory. Providers validate it against the
/// active session before process start.
pub cwd: fs_operation::FsPath,
/// Provider-local directory where complete output is retained when the
/// inline result exceeds `output_limit`.
pub spill_dir: Option<PathBuf>,
/// Optional caller-owned correlation id. Bash supplies its tool-call id so
/// user-facing command telemetry can update the corresponding Console row
/// without exposing provider/session handles.
@@ -27,12 +25,23 @@ pub struct CommandRequest {
pub tool_call_id: Option<String>,
}
impl CommandRequest {
pub fn validate(&self) -> Result<(), crate::WorkdirError> {
if self.transport_byte_budget > MAX_COMMAND_TRANSPORT_BYTES {
return Err(crate::WorkdirError::InvalidArgument(
"command transport byte budget exceeds provider maximum".into(),
));
}
Ok(())
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct CommandOutputRequest {
pub handle: CommandHandle,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub cursor: usize,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
#[schemars(range(min = 0, max = 67_108_864_u64))]
pub cursor: u64,
#[schemars(range(min = 1, max = 65_536_usize))]
pub limit: usize,
pub wait: bool,
}
@@ -107,16 +116,150 @@ pub enum CommandEvent {
},
}
/// Sequence is the merged transport byte position. Offset addresses raw bytes
/// within the named stream; neither ordering nor offsets depend on UTF-8 decoding.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct CommandOutput {
pub struct CommandByteChunk {
pub sequence: u64,
pub stream: CommandStream,
pub offset: u64,
pub bytes: Vec<u8>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum CommandOutputTruncation {
ByteBudget,
}
/// Raw totals include discarded pipe bytes. Delivered totals count only prefixes
/// admitted to the transport queue, not page reads or retries by a consumer.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct CommandTerminal {
pub status: CommandStatus,
#[schemars(range(min = -2147483648_i32, max = 2147483647_i32))]
pub exit_code: Option<i32>,
pub timed_out: bool,
pub content: String,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub next_cursor: Option<usize>,
pub truncated: bool,
/// Complete output retained by the provider when `truncated` is true.
pub output_path: Option<PathBuf>,
pub stdout_bytes: u64,
pub stderr_bytes: u64,
pub stdout_delivered_bytes: u64,
pub stderr_delivered_bytes: u64,
pub truncation_reason: Option<CommandOutputTruncation>,
}
/// Sending next_cursor acknowledges this page. Only the current page may be
/// retried; acknowledged bytes expire. Terminal fences all captured bytes,
/// including on timeout/cancel. Consumers must pull while the process runs.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CommandOutput {
pub chunks: Vec<CommandByteChunk>,
pub next_cursor: u64,
pub terminal: Option<CommandTerminal>,
}
impl CommandOutput {
pub(crate) fn transport_bounds_valid(&self) -> bool {
let mut end = None;
let mut stream_ends = [None, None];
for chunk in &self.chunks {
let stream_index = match chunk.stream {
CommandStream::Stdout => 0,
CommandStream::Stderr => 1,
};
if chunk.bytes.is_empty()
|| chunk.offset > chunk.sequence
|| stream_ends[stream_index].is_some_and(|end| chunk.offset != end)
|| end.is_some_and(|end| chunk.sequence != end)
|| chunk
.sequence
.checked_add(chunk.bytes.len() as u64)
.is_none_or(|end| end > MAX_COMMAND_TRANSPORT_BYTES)
|| chunk
.offset
.checked_add(chunk.bytes.len() as u64)
.is_none_or(|end| end > MAX_COMMAND_TRANSPORT_BYTES)
{
return false;
}
end = chunk.sequence.checked_add(chunk.bytes.len() as u64);
stream_ends[stream_index] = chunk.offset.checked_add(chunk.bytes.len() as u64);
}
self.next_cursor <= MAX_COMMAND_TRANSPORT_BYTES
&& end.is_none_or(|end| end == self.next_cursor)
&& self.terminal.as_ref().is_none_or(|terminal| {
let truncated = terminal.stdout_bytes > terminal.stdout_delivered_bytes
|| terminal.stderr_bytes > terminal.stderr_delivered_bytes;
stream_ends[0].is_none_or(|end| end == terminal.stdout_delivered_bytes)
&& stream_ends[1].is_none_or(|end| end == terminal.stderr_delivered_bytes)
&& terminal.status != CommandStatus::Running
&& terminal.stdout_delivered_bytes <= terminal.stdout_bytes
&& terminal.stderr_delivered_bytes <= terminal.stderr_bytes
&& terminal
.stdout_delivered_bytes
.checked_add(terminal.stderr_delivered_bytes)
== Some(self.next_cursor)
&& terminal.truncation_reason
== truncated.then_some(CommandOutputTruncation::ByteBudget)
})
}
}
pub const MAX_COMMAND_TRANSPORT_BYTES: u64 = 64 * 1024 * 1024;
pub const MAX_COMMAND_READ_BYTES: usize = 64 * 1024;
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn command_wire_rejects_legacy_output_host_paths() {
let command = serde_json::json!({
"command": "printf bytes", "timeout_secs": 1, "cwd": "", "transport_byte_budget": 0
});
assert!(serde_json::from_value::<CommandRequest>(command.clone()).is_ok());
for field in ["spill_dir", "output_limit"] {
let mut legacy = command.clone();
legacy[field] = serde_json::json!("/host/private");
assert!(serde_json::from_value::<CommandRequest>(legacy).is_err());
}
let output = serde_json::json!({"chunks": [], "next_cursor": 0, "terminal": null, "output_path": "/host/private"});
assert!(serde_json::from_value::<CommandOutput>(output).is_err());
}
#[test]
fn command_budget_wire_requires_field_and_checks_provider_hard_bound() {
let wire = serde_json::json!({"command": "true", "timeout_secs": 1, "cwd": ""});
assert!(serde_json::from_value::<CommandRequest>(wire.clone()).is_err());
for budget in [
0,
MAX_COMMAND_TRANSPORT_BYTES,
MAX_COMMAND_TRANSPORT_BYTES + 1,
] {
let mut value = wire.clone();
value["transport_byte_budget"] = serde_json::json!(budget);
let request: CommandRequest = serde_json::from_value(value).unwrap();
assert_eq!(
request.validate().is_ok(),
budget <= MAX_COMMAND_TRANSPORT_BYTES
);
}
let mut negative = wire;
negative["transport_byte_budget"] = serde_json::json!(-1);
assert!(serde_json::from_value::<CommandRequest>(negative).is_err());
assert_eq!(
serde_json::to_value(CommandOutputTruncation::ByteBudget).unwrap(),
"byte_budget"
);
let schema = serde_json::to_value(schemars::schema_for!(CommandRequest)).unwrap();
assert_eq!(
schema["properties"]["transport_byte_budget"]["maximum"],
MAX_COMMAND_TRANSPORT_BYTES
);
assert!(
schema["required"]
.as_array()
.unwrap()
.contains(&serde_json::json!("transport_byte_budget"))
);
}
}
+13 -8
View File
@@ -341,7 +341,9 @@ impl WorkdirSession for RoutedWorkdirSession {
let _active = self.state.enter()?;
let handle = request.handle.clone();
let output = self.state.session.command_output(request).await?;
self.state.mark_terminal(&handle, output.status);
if let Some(terminal) = &output.terminal {
self.state.mark_terminal(&handle, terminal.status);
}
Ok(output)
}
@@ -789,11 +791,12 @@ mod tests {
.unwrap();
let handle = routed
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf done".to_string(),
timeout_secs: 5,
output_limit: 1024,
cwd: crate::WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
})
.await
@@ -823,11 +826,12 @@ mod tests {
let routed = router.attach(alias, Arc::clone(&underlying)).unwrap();
let handle = routed
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "sleep 30".to_string(),
timeout_secs: 60,
output_limit: 1024,
cwd: crate::WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
})
.await
@@ -838,7 +842,7 @@ mod tests {
assert!(router.is_empty());
assert!(matches!(
underlying.command_status(handle).await,
Err(WorkdirError::Unavailable(_))
Err(ref error) if matches!(error.classification_source(), WorkdirError::Unavailable(_))
));
}
@@ -850,11 +854,12 @@ mod tests {
let routed = router.attach(alias.clone(), session("wd", &dir)).unwrap();
let handle = routed
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "sleep 30".to_string(),
timeout_secs: 60,
output_limit: 1024,
cwd: crate::WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
})
.await
+294 -42
View File
@@ -10,6 +10,9 @@
//! provider-side before name enumeration, descent, or content reads. Returned coordinates are not
//! prefixed with cwd again.
#[cfg(test)]
use crate::command_test_support::CollectCommand as _;
use std::collections::{BTreeMap, HashMap, HashSet};
use std::path::Path;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
@@ -124,6 +127,7 @@ impl WorkdirToolBroker {
next_lease_id: AtomicU64::new(1),
close_lock: Arc::new(tokio::sync::Mutex::new(())),
owned_commands: Arc::new(Mutex::new(HashSet::new())),
command_reads: Mutex::new(HashMap::new()),
pending_command_events: Arc::new(Mutex::new(HashMap::new())),
starting_tool_calls: Arc::new(Mutex::new(HashSet::new())),
forwarded_starts: Arc::new(Mutex::new(HashSet::new())),
@@ -230,25 +234,48 @@ impl WorkdirScopeLease {
.source
.cancel_command(handle.clone())
.await;
let terminal = self
let read_cursor = self
.broker
.authority
.source
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 1,
wait: true,
})
.await;
.command_reads
.lock()
.expect("command cursor mutex poisoned")
.get(&command_id)
.cloned()
.expect("owned command cursor");
let mut read_cursor = read_cursor.lock().await;
let mut cursor = *read_cursor;
let terminal = loop {
let output = self
.broker
.authority
.source
.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: crate::MAX_COMMAND_READ_BYTES,
wait: true,
})
.await;
match output {
Ok(output) => {
cursor = output.next_cursor;
*read_cursor = cursor;
if let Some(terminal) = output.terminal {
break Ok(terminal);
}
}
Err(error) => break Err(error),
}
};
match (cancel, terminal) {
(_, Ok(output)) => {
self.broker.authority.publish_terminal_if_missing(
&command_id,
output.status,
output.exit_code,
output.next_cursor.unwrap_or(output.content.len()) as u64,
&output.content,
cursor,
"",
);
self.broker
.authority
@@ -582,7 +609,12 @@ struct ScopedWorkdirSession {
scope_lock: tokio::sync::Mutex<()>,
next_lease_id: AtomicU64,
close_lock: Arc<tokio::sync::Mutex<()>>,
// Live cleanup responsibility ends at terminal delivery. Read ownership is
// retained separately so a lost terminal response can still be replayed.
owned_commands: Arc<Mutex<HashSet<String>>>,
// Starting commands prunes completed entries at the bounded replay ceiling;
// current scope validity is still checked before every read or retry.
command_reads: Mutex<HashMap<String, Arc<tokio::sync::Mutex<u64>>>>,
pending_command_events: Arc<Mutex<HashMap<String, Vec<CommandEvent>>>>,
starting_tool_calls: Arc<Mutex<HashSet<String>>>,
forwarded_starts: Arc<Mutex<HashSet<String>>>,
@@ -700,6 +732,11 @@ impl ScopedWorkdirSession {
.lock()
.expect("scoped command set mutex poisoned")
.contains(&handle.0)
|| self
.command_reads
.lock()
.expect("command cursor mutex poisoned")
.contains_key(&handle.0)
{
Ok(())
} else {
@@ -1013,6 +1050,7 @@ impl ScopedWorkdirSession {
next_lease_id: AtomicU64::new(1),
close_lock: close_lock.clone(),
owned_commands,
command_reads: Mutex::new(HashMap::new()),
pending_command_events,
starting_tool_calls,
forwarded_starts,
@@ -1222,6 +1260,9 @@ impl WorkdirSession for ScopedWorkdirSession {
.intersection(WorkdirSessionCapabilities::READ_ONLY);
}
if let crate::CheckoutOutput::Read(read) = &mut result.output {
read.path = result_path.clone();
}
if let crate::CheckoutOutput::ReadBytes(read) = &mut result.output {
read.path = result_path;
}
Ok(result)
@@ -1407,6 +1448,16 @@ impl WorkdirSession for ScopedWorkdirSession {
.lock()
.expect("scoped command set mutex poisoned");
owned.insert(handle.0.clone());
{
let mut reads = self
.command_reads
.lock()
.expect("command cursor mutex poisoned");
if reads.len() >= 64 {
reads.retain(|id, _| owned.contains(id));
}
reads.insert(handle.0.clone(), Arc::new(tokio::sync::Mutex::new(0)));
}
if let Some(tool_call_id) = &tool_call_id {
self.starting_tool_calls
.lock()
@@ -1457,14 +1508,28 @@ impl WorkdirSession for ScopedWorkdirSession {
) -> Result<CommandOutput, WorkdirError> {
self.ensure_owned_command(&request.handle)?;
let command_id = request.handle.0.clone();
let read_cursor = self
.command_reads
.lock()
.expect("command cursor mutex poisoned")
.get(&command_id)
.cloned();
let mut read_cursor = match &read_cursor {
Some(cursor) => Some(cursor.lock().await),
None => None,
};
self.ensure_owned_command(&request.handle)?;
let output = self.source.command_output(request).await?;
if !matches!(output.status, CommandStatus::Running) {
if let Some(cursor) = &mut read_cursor {
**cursor = output.next_cursor;
}
if let Some(terminal) = &output.terminal {
self.publish_terminal_if_missing(
&command_id,
output.status,
output.exit_code,
output.next_cursor.unwrap_or(output.content.len()) as u64,
&output.content,
terminal.status,
terminal.exit_code,
output.next_cursor,
"",
);
self.owned_commands
.lock()
@@ -2080,20 +2145,21 @@ mod tests {
session: &WorkdirSessionHandle,
command: impl Into<String>,
tool_call_id: impl Into<String>,
) -> CommandOutput {
) -> crate::command_test_support::CollectedCommand {
let handle = session
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: command.into(),
timeout_secs: 5,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some(tool_call_id.into()),
})
.await
.unwrap();
session
.command_output(CommandOutputRequest {
.collect_command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 1024,
@@ -2112,11 +2178,12 @@ mod tests {
.expect("delegation wrapper must preserve command observation");
let handle = parent
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf ready; sleep 0.2; printf done".into(),
timeout_secs: 5,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some("tool-delegated".into()),
})
.await
@@ -2139,7 +2206,7 @@ mod tests {
assert_eq!(snapshots[0].stdout.content, "ready");
let output = parent
.command_output(CommandOutputRequest {
.collect_command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 1024,
@@ -2179,11 +2246,12 @@ mod tests {
);
let error = child
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "pwd".into(),
timeout_secs: 5,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: None,
})
.await
@@ -2211,11 +2279,12 @@ mod tests {
let handle = first
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "pwd; sleep 0.2".into(),
timeout_secs: 5,
output_limit: 4096,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some("first-command".into()),
})
.await
@@ -2239,7 +2308,7 @@ mod tests {
));
let output = first
.command_output(CommandOutputRequest {
.collect_command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 4096,
@@ -2316,11 +2385,12 @@ mod tests {
assert!(matches!(
child
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf denied".into(),
timeout_secs: 5,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some("read-only-command".into()),
})
.await,
@@ -2622,11 +2692,12 @@ mod tests {
assert!(matches!(
child
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf revoked".into(),
timeout_secs: 5,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some("revoked-child-command".into()),
})
.await,
@@ -2724,11 +2795,12 @@ mod tests {
for index in 0..MAX_SCOPED_COMMANDS {
child
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "sleep 30".into(),
timeout_secs: 60,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some(format!("command-{index}")),
})
.await
@@ -2737,11 +2809,12 @@ mod tests {
let error = child
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "sleep 30".into(),
timeout_secs: 60,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some("command-over-limit".into()),
})
.await
@@ -2771,11 +2844,12 @@ mod tests {
let command = tokio::spawn(async move {
command_child
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "sleep 30".into(),
timeout_secs: 60,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some("racing-command".into()),
})
.await
@@ -2817,11 +2891,12 @@ mod tests {
let mut events = child.subscribe_command_events().unwrap();
let handle = child
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "sleep 30; printf leaked > marker".into(),
timeout_secs: 60,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some("owned-command".into()),
})
.await
@@ -3012,11 +3087,12 @@ mod tests {
assert!(matches!(
parent
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf closed".into(),
timeout_secs: 5,
output_limit: 1024,
cwd: FsPath::root(),
spill_dir: None,
tool_call_id: Some("closed-parent-command".into()),
})
.await,
@@ -3028,4 +3104,180 @@ mod tests {
"child result after parent close: {child_result:?}"
);
}
#[tokio::test]
async fn scoped_byte_cursor_survives_partial_ack_then_revocation_and_terminal_retry() {
let root = TempDir::new().unwrap();
fs::create_dir(root.path().join("child")).unwrap();
let parent = session(root.path());
let child = parent
.scope(request("child", WorkdirToolScopePermission::Write))
.await
.unwrap();
let tool = child.tool_session();
let handle = tool
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf ready; exec yes x".into(),
timeout_secs: 10,
cwd: FsPath::root(),
tool_call_id: None,
})
.await
.unwrap();
let mut cursor = 0;
for _ in 0..2 {
let page = tool
.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: 1,
wait: true,
})
.await
.unwrap();
cursor = page.next_cursor;
}
tokio::time::timeout(std::time::Duration::from_secs(5), child.close())
.await
.unwrap()
.unwrap();
assert!(
tool.command_output(CommandOutputRequest {
handle,
cursor,
limit: 1,
wait: false
})
.await
.is_err()
);
let handle = parent
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf parent".into(),
timeout_secs: 5,
cwd: FsPath::root(),
tool_call_id: None,
})
.await
.unwrap();
let mut cursor = 0;
loop {
let request = CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: 1024,
wait: true,
};
let page = parent.command_output(request.clone()).await.unwrap();
assert_eq!(parent.command_output(request).await.unwrap(), page);
cursor = page.next_cursor;
if page.terminal.is_some() {
break;
}
}
}
#[tokio::test]
async fn scoped_terminal_response_loss_replays_without_duplicate_capture_and_rejects_invalid_cursors()
{
tokio::time::timeout(std::time::Duration::from_secs(5), async {
let root = TempDir::new().unwrap();
fs::create_dir(root.path().join("child")).unwrap();
let parent = session(root.path());
let child = parent
.scope(request("child", WorkdirToolScopePermission::Write))
.await
.unwrap();
let tool = child.tool_session();
let handle = tool
.start_command(CommandRequest {
transport_byte_budget: crate::MAX_COMMAND_TRANSPORT_BYTES,
command: "printf abcdef".into(),
timeout_secs: 5,
cwd: FsPath::root(),
tool_call_id: None,
})
.await
.unwrap();
let mut cursor = 0;
let mut captured = Vec::new();
let terminal_request = loop {
let request = CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: 2,
wait: true,
};
// Simulate losing each produced HTTP response, including terminal:
// append only the retried page, never both copies.
let lost = tool.command_output(request.clone()).await.unwrap();
let replay = tool.command_output(request.clone()).await.unwrap();
assert_eq!(replay, lost);
for chunk in &replay.chunks {
captured.extend_from_slice(&chunk.bytes);
}
cursor = replay.next_cursor;
if let Some(terminal) = replay.terminal {
assert_eq!(terminal.status, CommandStatus::Completed);
break request;
}
};
assert_eq!(captured, b"abcdef");
assert!(
!child
.broker
.authority
.owned_commands
.lock()
.unwrap()
.contains(&handle.0)
);
assert!(
child
.broker
.authority
.command_reads
.lock()
.unwrap()
.contains_key(&handle.0)
);
assert_eq!(
tool.command_status(handle.clone()).await.unwrap(),
CommandStatus::Completed
);
for invalid in [0, cursor + 1] {
assert!(
tool.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor: invalid,
limit: 2,
wait: false
})
.await
.is_err()
);
}
// Invalid cursor requests do not destroy the retained terminal page.
assert!(
tool.command_output(terminal_request)
.await
.unwrap()
.terminal
.is_some()
);
child.close().await.unwrap();
assert!(
tool.command_output(CommandOutputRequest {
handle,
cursor,
limit: 2,
wait: false
})
.await
.is_err()
);
})
.await
.expect("bounded scoped terminal replay");
}
}
@@ -103,11 +103,12 @@ impl WorkdirSession for CommandFailureSession {
}
fn command() -> CommandRequest {
CommandRequest {
transport_byte_budget: workdir::MAX_COMMAND_TRANSPORT_BYTES,
command: "fake command, never executed".into(),
timeout_secs: 1,
output_limit: 100,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
}
}
+3 -2
View File
@@ -167,11 +167,12 @@ async fn read_only_and_scoped_capability_refusals_do_not_collapse() {
assert_reason(
child
.start_command(workdir::CommandRequest {
transport_byte_budget: workdir::MAX_COMMAND_TRANSPORT_BYTES,
command: "must-not-run".into(),
timeout_secs: 1,
output_limit: 1,
cwd: WorkdirPath::root(),
spill_dir: None,
tool_call_id: None,
})
.await
@@ -0,0 +1,85 @@
use workdir::external::ExternalWorkdirOperationError;
use workdir::http::{WorkdirTransportError, WorkdirTransportErrorCode as Code};
use workdir::{WorkdirError, WorkdirFailureReason as Reason};
#[test]
fn failure_reasons_survive_internal_transports_without_provider_text() {
for reason in [
Reason::SessionClosed,
Reason::ProviderUnavailable,
Reason::CommandTerminalMissing,
Reason::HttpTimeout,
Reason::HttpConnect,
Reason::HttpTransfer,
Reason::HttpResponseInvalid,
Reason::HttpResponseTooLarge,
] {
let source = WorkdirError::Unavailable("secret-token /private/path".into())
.with_failure_reason(reason);
let transport = WorkdirTransportError::from_workdir_error(&source);
let wire = serde_json::to_string(&transport).unwrap();
assert!(!wire.contains("secret-token"));
assert!(!wire.contains("/private/path"));
assert_eq!(transport.code, Code::Unavailable);
assert_eq!(transport.code.http_status(), 503);
assert_eq!(transport.message, "Workdir session is unavailable");
assert_eq!(serde_json::to_value(reason).unwrap(), reason.as_str());
let decoded: WorkdirTransportError = serde_json::from_str(&wire).unwrap();
let external = ExternalWorkdirOperationError::from_transport_error(decoded);
let external_wire = serde_json::to_string(&external).unwrap();
let external: ExternalWorkdirOperationError = serde_json::from_str(&external_wire).unwrap();
let error = external.into_transport_error().into_workdir_error();
assert_eq!(error.failure_reason(), Some(reason));
assert!(matches!(
error.classification_source(),
WorkdirError::Unavailable(_)
));
assert_eq!(
WorkdirTransportError::from_workdir_error(&error).failure_reason,
Some(reason)
);
}
}
#[test]
fn legacy_errors_and_nested_context_preserve_classification() {
let wire = r#"{"code":"unavailable","message":"Workdir session is unavailable"}"#;
let old: WorkdirTransportError = serde_json::from_str(wire).unwrap();
assert!(old.failure_reason.is_none());
assert!(matches!(
old.into_workdir_error(),
WorkdirError::Unavailable(_)
));
let closed = WorkdirTransportError::from_workdir_error(&WorkdirError::SessionClosed);
assert_eq!(closed.failure_reason, Some(Reason::SessionClosed));
// Reason-looking text never becomes a typed diagnosis.
let generic = WorkdirError::Unavailable("http_timeout".into());
assert_eq!(generic.failure_reason(), Some(Reason::ProviderUnavailable));
let error = WorkdirError::DenialContext {
reason: workdir::WorkdirDenialReason::OsPermissionDenied,
source: Box::new(
WorkdirError::OutcomeUnknown("do not retry".into())
.with_failure_reason(Reason::HttpTransfer),
),
};
let error = WorkdirTransportError::from_workdir_error(&error).into_workdir_error();
assert!(matches!(
error.classification_source(),
WorkdirError::OutcomeUnknown(_)
));
assert_eq!(
error.denial_reason(),
Some(workdir::WorkdirDenialReason::OsPermissionDenied)
);
assert_eq!(error.failure_reason(), Some(Reason::HttpTransfer));
}
#[test]
fn provider_cannot_inject_free_text_failure_reason() {
assert!(
serde_json::from_str::<WorkdirTransportError>(
r#"{"code":"unavailable","message":"x","failure_reason":"secret /path"}"#
)
.is_err()
);
}
+3
View File
@@ -19,6 +19,8 @@ required-features = ["ws-server", "fs-store"]
[features]
default = ["ws-server", "fs-store"]
fs-store = ["dep:fs4"]
# Offline fixture tooling only; never enabled by the product defaults.
test-support = ["fs-store", "ws-server"]
http-server = ["dep:axum", "dep:tower", "dep:reqwest"]
ws-server = ["http-server", "axum/ws", "dep:futures", "tokio/sync"]
@@ -56,6 +58,7 @@ server-api = { path = "../server-api" }
workdir.workspace = true
[dev-dependencies]
workdir = { workspace = true, features = ["http-client"] }
futures.workspace = true
agen.workspace = true
serial_test = "3.4.0"
+31
View File
@@ -8,3 +8,34 @@ Manages Worker creation, stopping, and Restore, execution connections within the
- [`execution`] — execution backends and operation-specific results.
- [`working_directory`] — repository materialization, session resources, and safe deletion.
- [`workspace_issuer`] — Workspace signature verification and replay prevention.
## Offline stopped-Worker fixtures
The nondefault `test-support` feature exposes `test_support::StoppedWorkerTemplate`.
It is intentionally limited to current-schema, stopped, attachment-free artificial
conversations with one Segment, text history, and no pending execution/activation.
It does not add a product endpoint or change create/restore authorization.
1. With the source Runtime offline, call `StoppedWorkerTemplate::extract(worker_dir)`.
The source must have its existing Runtime owner lock and identity. Extraction is
read-only; the returned serde template excludes source auth, saved Manifest,
request/config references, run directories, and Runtime operation history.
2. Initialize **new** Runtime identity and Workspace trust in a fresh destination
using normal setup. Preserve the template's Runtime and Workspace IDs.
3. Call `template.install(&options, request, &resolved_manifest)` before starting
the Runtime. Supply a fresh typed `CreateWorkerRequest` (including config/profile
references and trusted memory settings) and `manifest::WorkerManifest` resolved
for that environment. Its Worker name must equal `template.worker_name()`.
4. Start normally and use the ordinary explicit restore operation. Installation
itself creates no execution and leaves the Worker stopped with a fresh guard.
The template preserves the original Session manifest/JSONL bytes, including stable
entry IDs, timestamps, and settled submission receipts. It rejects resumable queues
and Feature invocation history. A saved system prompt is required for restore;
checkpoint receipts must match the Worker's typed receipt schema. Preserved records
must use canonical writer field shapes: unknown/duplicate fields (including nested
fields), discarded optional fields, and opaque Feature history payloads are rejected,
not stripped or rewritten. Portability/secret-marker checks do not replace
review of the supplied artificial conversation or its independently verified hashes.
The installer refuses any destination content other than initialized `auth.toml`;
interrupted publication is fail-closed, not an overwrite/recovery facility.
+19
View File
@@ -432,6 +432,9 @@ pub struct WorkerExecutionRestoreRequest {
pub previous_workdir_attachments: Vec<crate::catalog::WorkingDirectoryAttachmentStatus>,
pub logical_workdir_attachments: Vec<crate::catalog::LogicalWorkdirAttachment>,
pub workdir_attachments: BTreeMap<workdir::WorkdirAttachmentAlias, WorkingDirectoryBinding>,
/// Process-owned sessions that open providers only on a Workdir operation.
pub deferred_workdir_sessions:
BTreeMap<workdir::WorkdirAttachmentAlias, workdir::WorkdirSessionHandle>,
pub config_bundle: Option<ConfigBundle>,
}
@@ -567,6 +570,15 @@ impl fmt::Debug for WorkerProtocolTransport {
pub trait WorkerExecutionBackend: Send + Sync + 'static {
fn backend_id(&self) -> &str;
/// Confirm actual absence under the backend Worker operation lock. Runtime has
/// already durably fenced admission; unsupported or uncertain backends must fail.
fn recover_subject_create(
&self,
_request: &runtime_api::SubjectCreateRecoveryRequest,
) -> Result<(), String> {
Err("execution backend cannot confirm Subject create execution absence".into())
}
/// Must reserve the actual Controller, not infer Idle from a catalog snapshot.
fn subject_release(
&self,
@@ -839,6 +851,13 @@ impl WorkerExecutionBackendRef {
self.backend.fetch_workspace_config(request)
}
pub(crate) fn recover_subject_create(
&self,
request: &runtime_api::SubjectCreateRecoveryRequest,
) -> Result<(), String> {
self.backend.recover_subject_create(request)
}
pub(crate) fn subject_release(
&self,
request: &runtime_api::SubjectReleaseRequest,
+417 -35
View File
@@ -39,8 +39,14 @@ use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicU8, AtomicU64, Ordering};
const SCHEMA_VERSION: u32 = 10;
#[cfg(feature = "test-support")]
#[path = "test_support.rs"]
pub mod test_support;
const SCHEMA_VERSION: u32 = 12;
// Frozen source schemas: do not move their meaning when adding a migration.
const SOURCE_SCHEMA_VERSION: u32 = 11;
const EXECUTION_SCHEMA_VERSION: u32 = 10;
const AGGREGATE_SCHEMA_VERSION: u32 = 9;
const PREVIOUS_SCHEMA_VERSION: u32 = 8;
const LEGACY_SCHEMA_VERSION: u32 = 7;
@@ -479,6 +485,8 @@ pub(crate) struct OpenedFsRuntimeStore {
#[derive(Clone, Debug)]
pub(crate) struct PersistedRuntimeState {
pub(crate) subject_create_recovery_fences:
BTreeMap<WorkerId, crate::runtime::SubjectCreateRecoveryFence>,
pub(crate) subject_releases: Vec<(
runtime_api::SubjectReleaseRequest,
runtime_api::SubjectReleaseState,
@@ -640,8 +648,8 @@ fn plan_runtime_store_migration(
format!("Runtime store schema version {schema_version} is out of range"),
)
})?;
let staging = migration_sibling(root, "schema-v10-staging")?;
let backup = migration_sibling(root, "pre-schema-v10-backup")?;
let staging = migration_sibling(root, "schema-v12-staging")?;
let backup = migration_sibling(root, "pre-schema-v12-backup")?;
if staging.exists() || backup.exists() {
return Err(runtime_store_corrupt(
root,
@@ -653,18 +661,32 @@ fn plan_runtime_store_migration(
));
}
if current_schema_version == SCHEMA_VERSION {
// Runtime and Session schemas evolve independently. Probe only through
// Session Store's non-mutating current-schema admission boundary; any
// conversion still runs on the existing owner-locked staging tree.
let migration_required = crate::retention::sessions_require_migration(root)?;
let excluded_ephemeral_paths = if migration_required {
runtime_tree_exclusions(root)?
} else {
Vec::new()
};
let archived_orphan_worker_count = if migration_required {
orphaned_worker_directories(root)?.len()
} else {
0
};
let plan = FsRuntimeStoreMigrationPlan {
current_schema_version,
target_schema_version: SCHEMA_VERSION,
migration_required: false,
migration_required,
worker_count: 0,
archived_orphan_worker_count: 0,
archived_orphan_worker_count,
migrated_worker_aggregate_count: 0,
migrated_diagnostic_worker_ref_count: 0,
cleared_diagnostic_worker_ref_count: 0,
mapping_digest: legacy_worker_identity_mapping_digest(&[]),
mappings: Vec::new(),
excluded_ephemeral_paths: Vec::new(),
excluded_ephemeral_paths,
};
return Ok((plan, Vec::new()));
}
@@ -674,11 +696,13 @@ fn plan_runtime_store_migration(
| LEGACY_SCHEMA_VERSION
| PREVIOUS_SCHEMA_VERSION
| AGGREGATE_SCHEMA_VERSION
| EXECUTION_SCHEMA_VERSION
| SOURCE_SCHEMA_VERSION
) {
return Err(runtime_store_corrupt(
&runtime_path,
format!(
"unsupported Runtime store schema version {schema_version}; expected {OLDEST_SCHEMA_VERSION}, {LEGACY_SCHEMA_VERSION}, {PREVIOUS_SCHEMA_VERSION}, {AGGREGATE_SCHEMA_VERSION}, or {SCHEMA_VERSION}"
"unsupported Runtime store schema version {schema_version}; expected {OLDEST_SCHEMA_VERSION}, {LEGACY_SCHEMA_VERSION}, {PREVIOUS_SCHEMA_VERSION}, {AGGREGATE_SCHEMA_VERSION}, {EXECUTION_SCHEMA_VERSION}, {SOURCE_SCHEMA_VERSION}, or {SCHEMA_VERSION}"
),
));
}
@@ -1008,6 +1032,28 @@ fn migrate_worker_record(
mapping: Option<&LegacyWorkerIdentityMapping>,
identity_path: &Path,
) -> Result<WorkerAggregateRecord, RuntimeError> {
if matches!(
source_schema_version,
EXECUTION_SCHEMA_VERSION | SOURCE_SCHEMA_VERSION
) {
// Schemas 10 and 11 already use the current Worker aggregate layout.
// Only advance its version; never regenerate Restore admission identity.
if identity_document
.get("schema_version")
.and_then(serde_json::Value::as_u64)
!= Some(u64::from(source_schema_version))
{
return Err(runtime_store_corrupt(
identity_path,
"Worker schema does not match Runtime migration source".into(),
));
}
identity_document["schema_version"] = serde_json::Value::from(SCHEMA_VERSION);
let aggregate: WorkerAggregateRecord = serde_json::from_value(identity_document)
.map_err(|error| runtime_store_corrupt(identity_path, error.to_string()))?;
aggregate.clone().validate(identity_path)?;
return Ok(aggregate);
}
if source_schema_version == AGGREGATE_SCHEMA_VERSION {
migrate_v9_restore_guard(&mut identity_document, identity_path)?;
remove_legacy_job_input_counter(&mut identity_document);
@@ -1386,11 +1432,18 @@ fn migrate_runtime_document(
"schema_version".to_string(),
serde_json::Value::from(SCHEMA_VERSION),
);
object.remove("workers");
// This legacy cache is not Runtime lifecycle or restore authority. Bundles
// are reacquired from Workspace content authority, never renumbered/rehash-pinned here.
object.remove("config_bundles");
object.remove("next_worker_sequence");
// Intermediate schema-11 writers may already have saved irreversible fences.
// Default only a missing field; malformed or existing fences must not be erased.
object
.entry("subject_create_recovery_fences")
.or_insert_with(|| serde_json::json!({}));
if source_schema_version != SOURCE_SCHEMA_VERSION {
object.remove("workers");
// This legacy cache is not Runtime lifecycle or restore authority. Bundles
// are reacquired from Workspace content authority, never renumbered/rehash-pinned here.
object.remove("config_bundles");
object.remove("next_worker_sequence");
}
let snapshot: RuntimeSnapshot = serde_json::from_value(document.clone()).map_err(|error| {
runtime_store_corrupt(
runtime_path,
@@ -1499,14 +1552,21 @@ fn remove_runtime_migration_directory(path: &Path) -> Result<(), RuntimeError> {
}
fn recover_runtime_store_migration(root: &Path) -> Result<(), RuntimeError> {
for suffix in ["schema-v9-staging", "pre-schema-v9-backup"] {
for suffix in [
"schema-v9-staging",
"pre-schema-v9-backup",
"schema-v10-staging",
"pre-schema-v10-backup",
"schema-v11-staging",
"pre-schema-v11-backup",
] {
let earlier = migration_sibling(root, suffix)?;
if earlier.exists() {
return Err(runtime_store_corrupt(root, "unfinished schema-v9 migration requires explicit recovery before schema-v10 startup".into()));
return Err(runtime_store_corrupt(root, "unfinished earlier Runtime migration requires explicit recovery before schema-v12 startup".into()));
}
}
let staging = migration_sibling(root, "schema-v10-staging")?;
let backup = migration_sibling(root, "pre-schema-v10-backup")?;
let staging = migration_sibling(root, "schema-v12-staging")?;
let backup = migration_sibling(root, "pre-schema-v12-backup")?;
let root_exists = migration_directory_exists(root)?;
let staging_exists = migration_directory_exists(&staging)?;
let backup_exists = migration_directory_exists(&backup)?;
@@ -1708,8 +1768,8 @@ fn migrate_runtime_store(
if !plan.migration_required {
return Ok(plan);
}
let staging = migration_sibling(root, "schema-v10-staging")?;
let backup = migration_sibling(root, "pre-schema-v10-backup")?;
let staging = migration_sibling(root, "schema-v12-staging")?;
let backup = migration_sibling(root, "pre-schema-v12-backup")?;
if staging.exists() || backup.exists() {
return Err(runtime_store_corrupt(
root,
@@ -1766,7 +1826,9 @@ fn migrate_runtime_store(
Ok(plan)
}
fn archive_orphaned_worker_directories(root: &Path) -> Result<usize, RuntimeError> {
fn orphaned_worker_directories(
root: &Path,
) -> Result<Vec<(std::ffi::OsString, PathBuf)>, RuntimeError> {
let workers_dir = root.join(WORKERS_DIR);
let mut orphaned = fs::read_dir(&workers_dir)
.map_err(|source| runtime_io_error("read Worker records", &workers_dir, source))?
@@ -1779,6 +1841,11 @@ fn archive_orphaned_worker_directories(root: &Path) -> Result<usize, RuntimeErro
})
.collect::<Vec<_>>();
orphaned.sort_by(|(left, _), (right, _)| left.cmp(right));
Ok(orphaned)
}
fn archive_orphaned_worker_directories(root: &Path) -> Result<usize, RuntimeError> {
let orphaned = orphaned_worker_directories(root)?;
if orphaned.is_empty() {
return Ok(0);
}
@@ -1813,6 +1880,21 @@ fn migrate_runtime_store_in_place(
if !plan.migration_required {
return Ok(plan);
}
if plan.current_schema_version == SCHEMA_VERSION {
// Session-only migration: do not reserialize Runtime/Worker identity,
// Restore guards, recovery fences, or retained execution receipts.
// As in a Runtime schema migration, retain orphan trees outside the
// Worker catalog before validating the completed staging tree.
let archived = archive_orphaned_worker_directories(root)?;
if archived != plan.archived_orphan_worker_count {
return Err(runtime_store_corrupt(
root,
"orphan count changed during Session migration".into(),
));
}
crate::retention::migrate_staged_sessions(root)?;
return Ok(plan);
}
let runtime_path = root.join(RUNTIME_FILE);
let bytes =
fs::read(&runtime_path).map_err(|error| runtime_io_error("read", &runtime_path, error))?;
@@ -1884,7 +1966,10 @@ fn migrate_runtime_store_in_place(
));
}
}
if let Some(metadata) = metadata {
// Schema 11 metadata was validated above but needs no conversion.
if let Some(metadata) = metadata
&& plan.current_schema_version != SOURCE_SCHEMA_VERSION
{
atomic_write_json(
&migrated_dir.join(WORKER_METADATA_FILE),
&metadata,
@@ -1903,6 +1988,9 @@ fn migrate_runtime_store_in_place(
),
));
}
// A Runtime schema bump says nothing about the retained Session schema.
// Session Store owns versioned conversion, including v4 -> generic v5.
crate::retention::migrate_staged_sessions(root)?;
let (document, _) = migrate_runtime_document(
document,
plan.current_schema_version,
@@ -1919,6 +2007,8 @@ fn migrate_runtime_store_in_place(
#[derive(Clone, Debug, Serialize, Deserialize)]
struct RuntimeSnapshot {
#[serde(default)]
subject_create_recovery_fences: BTreeMap<WorkerId, crate::runtime::SubjectCreateRecoveryFence>,
#[serde(default)]
subject_releases: Vec<(
runtime_api::SubjectReleaseRequest,
@@ -1955,6 +2045,7 @@ fn record_worker_load_diagnostic(
impl RuntimeSnapshot {
fn from_persisted(state: &PersistedRuntimeState) -> Self {
Self {
subject_create_recovery_fences: state.subject_create_recovery_fences.clone(),
subject_releases: state.subject_releases.clone(),
schema_version: SCHEMA_VERSION,
display_name: state.display_name.clone(),
@@ -1978,6 +2069,33 @@ impl RuntimeSnapshot {
),
});
}
for (worker_id, fence) in &self.subject_create_recovery_fences {
let request = &fence.request;
let response_matches = fence.response.as_ref().is_none_or(|response| {
response.worker_id == request.worker_id
&& response.workspace_id == request.workspace_id
&& response.create_fingerprint == request.create_fingerprint
&& response.operation_id == request.operation_id
});
if *worker_id != request.worker_id
|| request.workspace_id.trim().is_empty()
|| request.create_fingerprint.trim().is_empty()
|| request.create_fingerprint.len() > 512
|| request.operation_id.trim().is_empty()
|| request.operation_id.len() > 512
|| self
.workspace_owners
.get(&request.workspace_id)
.is_none_or(|owner| owner.trim().is_empty())
|| !response_matches
{
return Err(RuntimeError::StoreCorrupt {
operation: "read Subject create recovery fence",
path: path.to_path_buf(),
message: "invalid recovery identity, receipt or Workspace owner".into(),
});
}
}
if self.backend != RuntimeBackendKind::FsStore {
return Err(RuntimeError::StoreCorrupt {
operation: "read runtime snapshot",
@@ -1993,6 +2111,7 @@ impl RuntimeSnapshot {
workers: BTreeMap<WorkerId, PersistedWorkerRecord>,
) -> PersistedRuntimeState {
PersistedRuntimeState {
subject_create_recovery_fences: self.subject_create_recovery_fences,
subject_releases: self.subject_releases,
display_name: self.display_name,
status: self.status,
@@ -2683,6 +2802,262 @@ mod tests {
document["schema_version"].as_u64().unwrap()
}
#[test]
fn schema_v11_migration_preserves_identity_history_and_recovery_fences_on_reopen() {
// Cover the released schema (no field) and intermediate writers with
// either a pending fence or a completed recovery receipt.
for fence_state in ["absent", "pending", "completed"] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let worker_id = WorkerId::now_v7();
let fenced_worker_id = WorkerId::now_v7();
let worker_dir = root.join(WORKERS_DIR).join(worker_id.to_string());
fs::create_dir_all(worker_dir.join("session")).unwrap();
let runtime_path = root.join(RUNTIME_FILE);
let worker_path = worker_dir.join(WORKER_FILE);
let metadata_path = worker_dir.join(WORKER_METADATA_FILE);
let history_path = worker_dir.join("session/history.json");
let history = b"[{\"role\":\"user\",\"content\":\"retained history\"}]\n";
fs::write(&history_path, history).unwrap();
let metadata = serde_json::to_vec(&serde_json::json!({
"worker_name": runtime_worker_name(worker_id),
"workspace_id": "workspace-a",
"retained_extension": {"history": "must not be rewritten"}
}))
.unwrap();
fs::write(&metadata_path, &metadata).unwrap();
let mut runtime = serde_json::json!({
"schema_version": 11,
"display_name": "schema-11 Runtime",
"backend": "fs_store",
"status": "running",
"next_diagnostic_id": 2,
"config_bundles": {},
"workspace_owners": {"workspace-a": "owner-a"},
"subject_releases": [],
"diagnostics": [{
"id": 1, "worker_ref": {"worker_id": worker_id},
"severity": "warning", "code": "retained_history",
"message": "preserve diagnostic identity"
}]
});
let recovery_identity = serde_json::json!({
"workspace_id": "workspace-a", "worker_id": fenced_worker_id,
"create_fingerprint": "failed-create", "operation_id": "recover-a"
});
if fence_state != "absent" {
runtime["subject_create_recovery_fences"] = serde_json::json!({
fenced_worker_id.to_string(): {
"request": recovery_identity,
"response": if fence_state == "completed" {
recovery_identity.clone()
} else {
serde_json::Value::Null
}
}
});
}
atomic_write_json(&runtime_path, &runtime, "write schema-11 fixture").unwrap();
let runtime_before = fs::read(&runtime_path).unwrap();
let mut request = schema_v7_worker_document(worker_id)["request"].clone();
request["workspace_api"] = serde_json::json!({
"workspace_id": "workspace-a", "base_url": "https://workspace.invalid"
});
// Serialize defaults as a schema-11 writer would, without running
// a migration or generating a fresh Restore guard for the fixture.
let request: CreateWorkerRequest = serde_json::from_value(request).unwrap();
let worker: WorkerAggregateRecord = serde_json::from_value(serde_json::json!({
"schema_version": 11,
"restore_guard": {
"observation_token": "retained-restore-token",
"active_request_id": null, "owners": {}
},
"worker_ref": {"worker_id": worker_id}, "worker_id": worker_id,
"profile": request.profile,
"display_name": null,
"profile_source": request.profile_source.reference(),
"config_bundle": null,
"created_at_ms": 1234,
"status": "running",
"last_finished_submission_request_id": "finished-submission",
"workspace_id": "workspace-a",
"execution_state": {"state": "available", "execution": {
"request": request, "binding": {}, "restore_intent": "automatic"
}}
}))
.unwrap();
let mut expected_worker = serde_json::to_value(&worker).unwrap();
atomic_write_json(&worker_path, &worker, "write schema-11 Worker").unwrap();
let worker_before = fs::read(&worker_path).unwrap();
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-test");
let plan = FsRuntimeStore::migration_plan(&options).unwrap();
assert_eq!(plan.current_schema_version, 11);
assert_eq!(plan.target_schema_version, 12);
assert!(plan.migration_required);
assert_eq!(plan.worker_count, 1);
assert_eq!(plan.migrated_worker_aggregate_count, 0);
assert!(plan.mappings.is_empty());
assert_eq!(fs::read(&runtime_path).unwrap(), runtime_before);
assert_eq!(fs::read(&worker_path).unwrap(), worker_before);
// The direct reader still rejects a previous schema: migration is
// explicit, not permissive deserialization of a newer format.
let snapshot: RuntimeSnapshot = serde_json::from_value(runtime.clone()).unwrap();
assert!(snapshot.validate(&runtime_path).is_err());
let opened = FsRuntimeStore::open_or_create(root.clone(), "runtime-test").unwrap();
let state = opened.state.as_ref().unwrap();
let persisted_worker = state.workers.get(&worker_id).unwrap();
expected_worker["schema_version"] = serde_json::json!(12);
assert_eq!(
serde_json::to_value(WorkerAggregateRecord::from_persisted(persisted_worker))
.unwrap(),
expected_worker
);
runtime["schema_version"] = serde_json::json!(12);
if fence_state == "absent" {
runtime["subject_create_recovery_fences"] = serde_json::json!({});
}
assert_eq!(
serde_json::to_value(&state.subject_create_recovery_fences).unwrap(),
runtime["subject_create_recovery_fences"]
);
assert_eq!(
read_json::<serde_json::Value>(&runtime_path, "read migrated Runtime").unwrap(),
runtime
);
assert_eq!(
read_json::<serde_json::Value>(&worker_path, "read migrated Worker").unwrap(),
expected_worker
);
// Exercise the normal writer too: it must retain intermediate fences.
opened.store.write_runtime_snapshot(state).unwrap();
drop(opened);
let runtime_once = fs::read(&runtime_path).unwrap();
let worker_once = fs::read(&worker_path).unwrap();
let reopened = FsRuntimeStore::open_or_create(root.clone(), "runtime-test").unwrap();
assert_eq!(
serde_json::to_value(
&reopened
.state
.as_ref()
.unwrap()
.subject_create_recovery_fences
)
.unwrap(),
runtime["subject_create_recovery_fences"]
);
drop(reopened);
let plan = FsRuntimeStore::migrate(&options).unwrap();
assert!(!plan.migration_required);
assert_eq!(plan.current_schema_version, 12);
assert_eq!(plan.target_schema_version, 12);
assert!(
!FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
assert_eq!(fs::read(&runtime_path).unwrap(), runtime_once);
assert_eq!(fs::read(&worker_path).unwrap(), worker_once);
assert_eq!(fs::read(&metadata_path).unwrap(), metadata);
assert_eq!(fs::read(&history_path).unwrap(), history);
assert!(
!migration_sibling(&root, "schema-v12-staging")
.unwrap()
.exists()
);
assert!(
!migration_sibling(&root, "pre-schema-v12-backup")
.unwrap()
.exists()
);
}
}
#[test]
fn schema_v11_migration_rejects_invalid_fences_without_replacing_them() {
for fences in [
serde_json::Value::Null,
serde_json::json!({"invalid-worker": {}}),
] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
write_empty_schema_v8_store(&root);
let path = root.join(RUNTIME_FILE);
let mut runtime: serde_json::Value = read_json(&path, "read fixture").unwrap();
runtime["schema_version"] = serde_json::json!(11);
runtime["subject_create_recovery_fences"] = fences;
atomic_write_json(&path, &runtime, "write invalid fence fixture").unwrap();
let before = fs::read(&path).unwrap();
assert!(FsRuntimeStore::open_or_create(root, "runtime-test").is_err());
assert_eq!(fs::read(&path).unwrap(), before);
}
}
#[test]
fn schema_v10_startup_migrates_stopped_sessions_atomically() {
for corrupt in [false, true] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
write_empty_schema_v8_store(&root);
let mut runtime: serde_json::Value =
read_json(&root.join(RUNTIME_FILE), "test").unwrap();
runtime["schema_version"] = serde_json::json!(10);
atomic_write_json(&root.join(RUNTIME_FILE), &runtime, "test").unwrap();
let worker_id = WorkerId::now_v7();
let aggregate = root.join(WORKERS_DIR).join(worker_id.to_string());
fs::create_dir_all(&aggregate).unwrap();
let record = migrate_worker_record(
schema_v7_worker_document(worker_id),
7,
None,
&aggregate.join(WORKER_FILE),
)
.unwrap();
let mut record = serde_json::to_value(record).unwrap();
record["schema_version"] = serde_json::json!(10);
atomic_write_json(&aggregate.join(WORKER_FILE), &record, "test").unwrap();
let session = aggregate.join("session");
fs::create_dir_all(session.join("segments")).unwrap();
atomic_write_json(&session.join("session.json"), &serde_json::json!({"schema_version":3,"session_id":session_store::new_session_id()}), "test").unwrap();
if corrupt {
fs::write(
session
.join("segments")
.join(format!("{}.jsonl", session_store::new_segment_id())),
b"invalid\n",
)
.unwrap();
}
let before = fs::read(session.join("session.json")).unwrap();
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-test");
assert!(
FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
assert_eq!(fs::read(session.join("session.json")).unwrap(), before);
let result = FsRuntimeStore::open_or_create(root.clone(), "runtime-test");
if corrupt {
assert!(result.is_err());
assert_eq!(stored_schema_version(&root), 10);
assert_eq!(fs::read(session.join("session.json")).unwrap(), before);
let record: serde_json::Value =
read_json(&aggregate.join(WORKER_FILE), "test").unwrap();
assert_eq!(record["schema_version"], 10);
} else {
drop(result.unwrap());
assert_eq!(stored_schema_version(&root), u64::from(SCHEMA_VERSION));
session_store::WorkerSessionStore::open_read_only(&session)
.expect("startup migration must admit the current Session schema");
let once = fs::read(session.join("session.json")).unwrap();
drop(FsRuntimeStore::open_or_create(root.clone(), "runtime-test").unwrap());
assert_eq!(fs::read(session.join("session.json")).unwrap(), once);
}
}
}
#[test]
fn auth_initialized_directory_without_snapshot_is_a_fresh_store() {
let parent = tempfile::tempdir().unwrap();
@@ -3062,7 +3437,7 @@ mod tests {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime-store");
write_empty_schema_v8_store(&root);
let staging = migration_sibling(&root, "schema-v10-staging").unwrap();
let staging = migration_sibling(&root, "schema-v12-staging").unwrap();
fs::create_dir_all(&staging).unwrap();
fs::write(staging.join("partial"), b"partial").unwrap();
let store = FsRuntimeStore::open_or_create(root.clone(), "test-runtime").unwrap();
@@ -3077,7 +3452,7 @@ mod tests {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime-store");
write_empty_schema_v8_store(&root);
let backup = migration_sibling(&root, "pre-schema-v10-backup").unwrap();
let backup = migration_sibling(&root, "pre-schema-v12-backup").unwrap();
fs::rename(&root, &backup).unwrap();
let store = FsRuntimeStore::open_or_create(root.clone(), "test-runtime").unwrap();
@@ -3091,10 +3466,10 @@ mod tests {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime-store");
write_empty_schema_v8_store(&root);
let staging = migration_sibling(&root, "schema-v10-staging").unwrap();
let staging = migration_sibling(&root, "schema-v12-staging").unwrap();
copy_runtime_tree(&root, &root, &staging).unwrap();
migrate_runtime_store_in_place(&staging, "test-runtime").unwrap();
let backup = migration_sibling(&root, "pre-schema-v10-backup").unwrap();
let backup = migration_sibling(&root, "pre-schema-v12-backup").unwrap();
fs::rename(&root, &backup).unwrap();
let store = FsRuntimeStore::open_or_create(root.clone(), "test-runtime").unwrap();
@@ -3109,10 +3484,10 @@ mod tests {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime-store");
write_empty_schema_v8_store(&root);
let staging = migration_sibling(&root, "schema-v10-staging").unwrap();
let staging = migration_sibling(&root, "schema-v12-staging").unwrap();
fs::create_dir_all(&staging).unwrap();
fs::write(staging.join(RUNTIME_FILE), b"{\"schema_version\":8}").unwrap();
let backup = migration_sibling(&root, "pre-schema-v10-backup").unwrap();
let backup = migration_sibling(&root, "pre-schema-v12-backup").unwrap();
fs::rename(&root, &backup).unwrap();
let store = FsRuntimeStore::open_or_create(root.clone(), "test-runtime").unwrap();
@@ -3129,7 +3504,7 @@ mod tests {
let root = temp.path().join("runtime-store");
write_empty_schema_v8_store(&root);
migrate_runtime_store_in_place(&root, "test-runtime").unwrap();
let backup = migration_sibling(&root, "pre-schema-v10-backup").unwrap();
let backup = migration_sibling(&root, "pre-schema-v12-backup").unwrap();
write_empty_schema_v8_store(&backup);
let store = FsRuntimeStore::open_or_create(root.clone(), "test-runtime").unwrap();
@@ -3139,14 +3514,21 @@ mod tests {
#[test]
fn earlier_migration_backup_cannot_be_mistaken_for_a_fresh_runtime() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime-store");
let backup = migration_sibling(&root, "pre-schema-v9-backup").unwrap();
write_empty_schema_v8_store(&backup);
let error = FsRuntimeStore::open_or_create(root.clone(), "test-runtime").unwrap_err();
assert!(error.to_string().contains("explicit recovery"));
assert!(!root.exists());
assert!(backup.join(RUNTIME_FILE).exists());
for suffix in [
"pre-schema-v9-backup",
"pre-schema-v10-backup",
"schema-v11-staging",
"pre-schema-v11-backup",
] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime-store");
let backup = migration_sibling(&root, suffix).unwrap();
write_empty_schema_v8_store(&backup);
let error = FsRuntimeStore::open_or_create(root.clone(), "test-runtime").unwrap_err();
assert!(error.to_string().contains("explicit recovery"));
assert!(!root.exists());
assert!(backup.join(RUNTIME_FILE).exists());
}
}
#[test]
+45 -9
View File
@@ -67,7 +67,7 @@ use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use tokio::net::TcpListener;
use workdir::{
CommandOutput, CommandStatus, WorkdirSessionHandle, dispatch_workdir_session_operation,
CommandOutput, WorkdirSessionHandle, dispatch_workdir_session_operation,
http::{
OpenWorkdirSessionRequest, OpenWorkdirSessionResponse, WorkdirSessionId,
WorkdirSessionOperation, WorkdirSessionOperationRequest, WorkdirSessionOperationResult,
@@ -914,13 +914,9 @@ async fn run_workdir_session_operation(
{
Ok(result) => result?,
Err(_) => CommandOutput {
exit_code: None,
timed_out: false,
status: CommandStatus::Running,
content: String::new(),
next_cursor: Some(cursor),
truncated: false,
output_path: None,
chunks: Vec::new(),
next_cursor: cursor,
terminal: None,
},
};
WorkdirSessionOperationResult::CommandOutput(output)
@@ -2102,6 +2098,9 @@ fn required_runtime_permission(method: &Method, path: &str) -> Option<&'static s
{
return Some("workers:input");
}
if path == "/v1/subject-create-recovery" && *method == Method::POST {
return Some("workers:delete");
}
if path.ends_with("/stop")
|| path.ends_with("/cancel")
|| (path == "/v1/subject-release" && *method == Method::POST)
@@ -2147,6 +2146,7 @@ impl RuntimeHttpWorkdirError {
Self {
status,
payload: WorkdirTransportError {
failure_reason: None,
denial_reason: None,
code,
message: message.into(),
@@ -2215,6 +2215,16 @@ impl From<workdir::WorkdirError> for RuntimeHttpWorkdirError {
impl IntoResponse for RuntimeHttpWorkdirError {
fn into_response(self) -> Response {
if let Some(reason) = self.payload.failure_reason {
tracing::warn!(
target: "worker_runtime::workdir",
event = "workdir_provider_failure",
status = self.status.as_u16(),
code = self.payload.code.as_str(),
failure_reason = reason.as_str(),
"Workdir provider operation failed"
);
}
(self.status, Json(self.payload)).into_response()
}
}
@@ -2423,6 +2433,7 @@ pub enum RuntimeHttpServerError {
#[cfg(test)]
mod tests {
mod command_transport_tests;
#[cfg(feature = "fs-store")]
mod session_material_tests;
@@ -2973,6 +2984,19 @@ mod tests {
}
}
#[test]
fn subject_create_recovery_requires_worker_delete_permission() {
assert_eq!(
required_runtime_permission(&Method::POST, "/v1/subject-create-recovery"),
Some("workers:delete")
);
// Existing ordinary release authorization is unchanged.
assert_eq!(
required_runtime_permission(&Method::POST, "/v1/subject-release"),
Some("workers:stop")
);
}
#[test]
fn retention_routes_require_worker_delete_permission() {
assert_eq!(
@@ -3182,9 +3206,13 @@ mod tests {
.await
.expect_err("close must be terminal");
assert!(matches!(
closed_error,
closed_error.classification_source(),
workdir::WorkdirError::Unavailable(_)
));
assert_eq!(
closed_error.failure_reason(),
Some(workdir::WorkdirFailureReason::SessionClosed)
);
assert!(
state
.workdir_sessions
@@ -3197,6 +3225,14 @@ mod tests {
struct AcceptingBackend;
impl WorkerExecutionBackend for AcceptingBackend {
fn recover_subject_create(
&self,
_request: &runtime_api::SubjectCreateRecoveryRequest,
) -> Result<(), String> {
// This test backend owns no real executions.
Ok(())
}
fn backend_id(&self) -> &str {
"http-test"
}
@@ -406,6 +406,18 @@ impl runtime_api::RuntimeApi for RuntimeManagementApi {
}
}
async fn recover_subject_create(
&self,
request: runtime_api::SubjectCreateRecoveryRequest,
) -> Result<runtime_api::SubjectCreateRecoveryResponse, runtime_api::RuntimeApiError> {
let scope = material_scope(&self.state, &request.workspace_id)?;
self.state
.runtime
.recover_subject_create_scoped(&scope, request)
.map_err(RuntimeHttpRestError::runtime)
.map_err(api_error)
}
async fn subject_release(
&self,
request: runtime_api::SubjectReleaseRequest,
@@ -0,0 +1,510 @@
//! Loopback integration: production RemoteWorkdirSession, signed HTTP middleware,
//! command operation handler, local process pipes, and JSON DTOs. Only opening the
//! pre-seeded session is a fixture; no Runtime Worker/materializer is claimed here.
use super::*;
use std::time::Duration;
use workdir::http::{
RemoteWorkdirSession, WorkdirHttpAuthorization, WorkdirHttpAuthorizationFuture,
};
use workdir::{CommandOutputRequest, CommandRequest, CommandStream, WorkdirSession};
struct SignedAuthorization {
identity: RuntimeIdentityMaterial,
trust: WorkspaceIssuerTrustRecord,
}
impl std::fmt::Debug for SignedAuthorization {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("SignedAuthorization")
}
}
impl WorkdirHttpAuthorization for SignedAuthorization {
fn bearer_token(
&self,
method: String,
path: String,
body: Vec<u8>,
) -> WorkdirHttpAuthorizationFuture {
let claims = signed_claims(&self.trust, &method, &path, &body);
let token =
issue_workspace_capability_token(&self.identity.signing_key().unwrap(), &claims)
.unwrap();
Box::pin(async move { Ok(token) })
}
}
struct Harness {
server: tokio::task::JoinHandle<()>,
session: WorkdirSessionHandle,
_root: tempfile::TempDir,
remote: RemoteWorkdirSession,
client: reqwest::Client,
operation_url: url::Url,
authorization: Arc<SignedAuthorization>,
}
impl Drop for Harness {
fn drop(&mut self) {
self.server.abort();
}
}
impl Harness {
async fn raw_operation(&self, operation: serde_json::Value) -> reqwest::Response {
let body = serde_json::to_vec(&serde_json::json!({"operation": operation})).unwrap();
let token = self
.authorization
.bearer_token(
"POST".into(),
self.operation_url.path().into(),
body.clone(),
)
.await
.unwrap();
self.client
.post(self.operation_url.clone())
.bearer_auth(token)
.header("content-type", "application/json")
.body(body)
.send()
.await
.unwrap()
}
async fn new() -> Self {
let root = tempfile::tempdir().unwrap();
let session: WorkdirSessionHandle = Arc::new(LocalWorkdirSession::materialized_bound(
Workdir::new("wd-transport"),
root.path().to_path_buf(),
root.path().to_path_buf(),
SharedScope::new(Scope::writable(root.path()).unwrap()),
WorkdirSessionCapabilities::ALL,
));
let identity = RuntimeIdentityMaterial::generate("transport-key").unwrap();
let trust = workspace_trust(&identity);
let state = RuntimeHttpState {
runtime: Runtime::new_memory(),
local_token: None,
workspace_auth: Some(Arc::new(workspace_test_auth(vec![trust.clone()]))),
workdir_sessions: Arc::new(Mutex::new(HashMap::from([(
"session-transport".into(),
RuntimeHttpWorkdirSession {
owner: RuntimeWorkspaceScope::new("workspace-a", "https://backend.test"),
session: session.clone(),
},
)]))),
ssh_keyscan_program: Arc::new(PathBuf::from("ssh-keyscan")),
};
let app = Router::new()
.route(
"/v1/working-directories/{id}/sessions",
post(|| async {
Json(OpenWorkdirSessionResponse {
session_id: WorkdirSessionId::new("session-transport").unwrap(),
workdir_id: Workdir::new("wd-transport").id().clone(),
capabilities: WorkdirSessionCapabilities::ALL,
})
}),
)
.route(
"/v1/workdir-sessions/{session_id}/operations",
post(run_workdir_session_operation),
)
.route(
"/v1/workdir-sessions/{session_id}",
delete(close_workdir_session),
)
.with_state(state.clone())
.layer(middleware::from_fn_with_state(state, require_runtime_auth));
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let base: url::Url = format!("http://{}", listener.local_addr().unwrap())
.parse()
.unwrap();
let server = tokio::spawn(async move {
axum::serve(listener, app).await.unwrap();
});
let mut headers = reqwest::header::HeaderMap::new();
headers.insert(
RUNTIME_WORKSPACE_SCOPE_HEADER,
"workspace-a".parse().unwrap(),
);
let client = reqwest::Client::builder()
.no_proxy()
.default_headers(headers)
.build()
.unwrap();
let authorization = Arc::new(SignedAuthorization { identity, trust });
let operation_url = base
.join("/v1/workdir-sessions/session-transport/operations")
.unwrap();
let remote = RemoteWorkdirSession::open_with_authorization(
client.clone(),
base,
authorization.clone(),
Workdir::new("wd-transport").id().clone(),
OpenWorkdirSessionRequest::default(),
)
.await
.unwrap();
Self {
server,
session,
_root: root,
remote,
client,
operation_url,
authorization,
}
}
}
#[tokio::test]
async fn real_http_waiting_command_survives_long_poll_expiry() {
tokio::time::timeout(Duration::from_secs(45), async {
let harness = Harness::new().await;
let handle = harness
.remote
.start_command(CommandRequest {
transport_byte_budget: 1024,
command: "printf ready; while [ ! -e release ]; do sleep 0.05; done; printf after; printf err >&2; exit 7".into(),
timeout_secs: 40,
cwd: WorkdirPath::root(),
tool_call_id: None,
})
.await
.unwrap();
let mut request = CommandOutputRequest {
handle,
cursor: 0,
limit: 1024,
wait: true,
};
let ready = harness.remote.command_output(request.clone()).await.unwrap();
assert_eq!(ready.chunks.iter().flat_map(|c| c.bytes.iter().copied()).collect::<Vec<_>>(), b"ready");
assert!(ready.terminal.is_none());
request.cursor = ready.next_cursor;
// Keep the command silent until the production HTTP handler has expired
// its 20-second long poll. The client must keep waiting, not fail or
// expose an empty page as completed output.
let waiting = harness.remote.command_output(request.clone());
tokio::pin!(waiting);
tokio::select! {
result = &mut waiting => panic!("waiting read ended before release: {result:?}"),
_ = tokio::time::sleep(Duration::from_secs(21)) => {}
}
std::fs::write(harness._root.path().join("release"), b"").unwrap();
let mut page = waiting.await.unwrap();
let mut stdout = Vec::new();
let mut stderr = Vec::new();
let terminal = loop {
// The retry cursor still replays the exact nonempty/terminal page.
assert_eq!(harness.remote.command_output(request.clone()).await.unwrap(), page);
for chunk in &page.chunks {
assert_eq!(chunk.sequence, request.cursor);
request.cursor += chunk.bytes.len() as u64;
match chunk.stream {
CommandStream::Stdout => stdout.extend_from_slice(&chunk.bytes),
CommandStream::Stderr => stderr.extend_from_slice(&chunk.bytes),
}
}
assert_eq!(page.next_cursor, request.cursor);
if let Some(terminal) = page.terminal {
break terminal;
}
page = harness.remote.command_output(request.clone()).await.unwrap();
};
assert_eq!(stdout, b"after");
assert_eq!(stderr, b"err");
assert_eq!(terminal.status, workdir::CommandStatus::Failed);
assert_eq!(terminal.exit_code, Some(7));
harness.remote.close().await.unwrap();
})
.await
.expect("bounded long-poll continuation over HTTP");
}
#[tokio::test]
async fn real_http_closed_session_retains_failure_reason() {
let harness = Harness::new().await;
let handle = harness
.remote
.start_command(CommandRequest {
transport_byte_budget: 1024,
command: "printf done".into(),
timeout_secs: 5,
cwd: WorkdirPath::root(),
tool_call_id: None,
})
.await
.unwrap();
// Keep the HTTP session identity registered, but close the underlying provider.
harness.session.close().await.unwrap();
let error = harness
.remote
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 16,
wait: true,
})
.await
.unwrap_err();
assert_eq!(
error.failure_reason(),
Some(workdir::WorkdirFailureReason::SessionClosed)
);
assert!(matches!(
error.classification_source(),
workdir::WorkdirError::Unavailable(_)
));
}
#[tokio::test]
async fn real_http_command_pages_preserve_binary_streams_replay_and_nonzero_exit() {
tokio::time::timeout(Duration::from_secs(20), async {
let harness = Harness::new().await;
let handle = harness.remote.start_command(CommandRequest {
transport_byte_budget: workdir::MAX_COMMAND_TRANSPORT_BYTES,
command: "head -c 2097152 /dev/zero | tr '\\0' x; printf '\\342\\202\\254\\377\\000'; printf 'stderr\\346\\227\\245' >&2; exit 9".into(),
timeout_secs: 15, cwd: WorkdirPath::root(), tool_call_id: Some("http-tool-call".into()),
}).await.unwrap();
let mut cursor = 0;
let mut stdout = Vec::new();
let mut stderr = Vec::new();
let terminal = loop {
let request = CommandOutputRequest { handle: handle.clone(), cursor, limit: 16384, wait: true };
let page = harness.remote.command_output(request.clone()).await.unwrap();
assert_eq!(harness.remote.command_output(request).await.unwrap(), page);
assert!(page.chunks.iter().map(|chunk| chunk.bytes.len()).sum::<usize>() <= 16384);
for chunk in page.chunks {
assert_eq!(chunk.sequence, cursor);
cursor += chunk.bytes.len() as u64;
let target = match chunk.stream { CommandStream::Stdout => &mut stdout, CommandStream::Stderr => &mut stderr };
assert_eq!(chunk.offset, target.len() as u64);
target.extend(chunk.bytes);
}
assert_eq!(cursor, page.next_cursor);
if let Some(terminal) = page.terminal { break terminal; }
};
let mut expected = vec![b'x'; 2097152];
expected.extend_from_slice(b"\xe2\x82\xac\xff\0");
assert_eq!(stdout, expected);
assert_eq!(stderr, b"stderr\xe6\x97\xa5");
assert_eq!(terminal.status, workdir::CommandStatus::Failed);
assert_eq!(terminal.exit_code, Some(9));
assert!(harness.remote.command_output(CommandOutputRequest { handle, cursor: 0, limit: 1, wait: false }).await.is_err());
harness.remote.close().await.unwrap();
assert!(harness.session.command_snapshot().is_empty());
}).await.expect("bounded HTTP command transport");
}
#[tokio::test]
async fn real_http_cancel_drains_tail_before_terminal_and_timeout_is_distinct() {
tokio::time::timeout(Duration::from_secs(15), async {
let harness = Harness::new().await;
for (command, timeout, cancel, expected) in [
(
"printf ready; exec yes x",
10,
true,
workdir::CommandStatus::Cancelled,
),
(
"printf ready; exec tail -f /dev/null",
1,
false,
workdir::CommandStatus::TimedOut,
),
] {
let handle = harness
.remote
.start_command(CommandRequest {
transport_byte_budget: workdir::MAX_COMMAND_TRANSPORT_BYTES,
command: command.into(),
timeout_secs: timeout,
cwd: WorkdirPath::root(),
tool_call_id: None,
})
.await
.unwrap();
let mut cursor = 0;
let first = harness
.remote
.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: 5,
wait: true,
})
.await
.unwrap();
assert_eq!(
first
.chunks
.iter()
.flat_map(|chunk| chunk.bytes.iter().copied())
.collect::<Vec<_>>(),
b"ready"
);
cursor = first.next_cursor;
if cancel {
harness.remote.cancel_command(handle.clone()).await.unwrap();
}
loop {
let page = harness
.remote
.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: workdir::MAX_COMMAND_READ_BYTES,
wait: true,
})
.await
.unwrap();
cursor = page.next_cursor;
if let Some(terminal) = page.terminal {
assert_eq!(terminal.status, expected);
break;
}
}
}
harness.remote.close().await.unwrap();
})
.await
.expect("bounded cancellation and timeout over HTTP");
}
#[tokio::test]
async fn real_http_command_budget_bounds_unique_payload_and_reports_larger_seen_totals() {
tokio::time::timeout(Duration::from_secs(20), async {
let harness = Harness::new().await;
for budget in [0, 12345] {
let handle = harness
.remote
.start_command(CommandRequest {
command: "head -c 1048576 /dev/zero; head -c 65536 /dev/zero >&2; exit 9"
.into(),
timeout_secs: 10,
transport_byte_budget: budget,
cwd: WorkdirPath::root(),
tool_call_id: None,
})
.await
.unwrap();
let mut cursor = 0;
let mut delivered = [0_u64; 2];
let terminal = loop {
let request = CommandOutputRequest {
handle: handle.clone(),
cursor,
limit: 997,
wait: true,
};
let page = harness
.remote
.command_output(request.clone())
.await
.unwrap();
assert_eq!(harness.remote.command_output(request).await.unwrap(), page);
for chunk in page.chunks {
assert_eq!(chunk.sequence, cursor);
cursor += chunk.bytes.len() as u64;
let index = match chunk.stream {
CommandStream::Stdout => 0,
CommandStream::Stderr => 1,
};
assert_eq!(chunk.offset, delivered[index]);
delivered[index] += chunk.bytes.len() as u64;
}
assert_eq!(cursor, page.next_cursor);
assert!(
cursor <= budget,
"RemoteWorkdirSession must not receive bytes beyond budget"
);
if let Some(terminal) = page.terminal {
break terminal;
}
};
assert_eq!(cursor, budget);
assert_eq!(terminal.stdout_bytes, 1048576);
assert_eq!(terminal.stderr_bytes, 65536);
assert_eq!(terminal.stdout_delivered_bytes, delivered[0]);
assert_eq!(terminal.stderr_delivered_bytes, delivered[1]);
assert_eq!(
terminal.truncation_reason,
Some(workdir::CommandOutputTruncation::ByteBudget)
);
assert_eq!(terminal.status, workdir::CommandStatus::Failed);
assert_eq!(terminal.exit_code, Some(9));
assert!(
harness
.remote
.command_output(CommandOutputRequest {
handle: handle.clone(),
cursor: budget + 1,
limit: 1,
wait: false,
})
.await
.is_err()
);
if budget > 0 {
assert!(
harness
.remote
.command_output(CommandOutputRequest {
handle,
cursor: 0,
limit: 1,
wait: false,
})
.await
.is_err()
);
}
}
harness.remote.close().await.unwrap();
})
.await
.expect("bounded HTTP byte-budget transport");
}
#[tokio::test]
async fn real_http_rejects_invalid_or_legacy_budget_requests_before_process_start() {
tokio::time::timeout(Duration::from_secs(10), async {
let harness = Harness::new().await;
let request = CommandRequest {
command: "touch must-not-exist".into(),
timeout_secs: 1,
transport_byte_budget: workdir::MAX_COMMAND_TRANSPORT_BYTES + 1,
cwd: WorkdirPath::root(),
tool_call_id: None,
};
let oversize =
serde_json::to_value(WorkdirSessionOperation::CommandStart(request)).unwrap();
assert_eq!(
harness.raw_operation(oversize.clone()).await.status(),
reqwest::StatusCode::BAD_REQUEST
);
let mut missing = oversize.clone();
// CommandStart is externally tagged by operation / request.
let object = missing.as_object_mut().unwrap();
let request = object.get_mut("request").unwrap().as_object_mut().unwrap();
request.remove("transport_byte_budget");
assert_eq!(
harness.raw_operation(missing).await.status(),
reqwest::StatusCode::BAD_REQUEST
);
let mut legacy = oversize;
let request = legacy["request"].as_object_mut().unwrap();
request.insert("transport_byte_budget".into(), serde_json::json!(0));
request.insert("spill_dir".into(), serde_json::json!("/private/host"));
assert_eq!(
harness.raw_operation(legacy).await.status(),
reqwest::StatusCode::BAD_REQUEST
);
assert!(!harness._root.path().join("must-not-exist").exists());
harness.remote.close().await.unwrap();
})
.await
.expect("bounded HTTP budget admission");
}
@@ -35,6 +35,7 @@ fn message(id: &str, role: LoggedRole, text: &str) -> LoggedHistoryEntry {
content: vec![LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.into()),
origin: LoggedSessionHistoryOrigin::LocalHumanInput,
derivation: None,
@@ -1086,6 +1087,54 @@ async fn canonical_http_rejects_replay_tampered_body_and_workspace_mismatch() {
assert_eq!(unsigned.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn subject_create_recovery_signed_http_requires_delete_and_matching_workspace() {
let f = Fixture::new();
let identity = RuntimeIdentityMaterial::generate("recovery-key").unwrap();
let trust = workspace_trust(&identity);
let app = runtime_http_router_with_workspace_auth(
f.runtime.clone(),
None,
workspace_test_auth(vec![trust.clone()]),
);
let path = "/v1/subject-create-recovery";
let mut request = runtime_api::SubjectCreateRecoveryRequest {
workspace_id: f.scope.workspace_id.clone(),
worker_id: WorkerId::now_v7(),
create_fingerprint: "unpublished-create".into(),
operation_id: "recovery-1".into(),
};
for operation in ["workers:read", "workers:stop", "workers:create"] {
let body = serde_json::to_vec(&request).unwrap();
let mut claims = signed_claims(&trust, "POST", path, &body);
claims.operation = operation.into();
let response = signed_request(app.clone(), &identity, &claims, body, "workspace-a").await;
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
}
request.workspace_id = "other-workspace".into();
let body = serde_json::to_vec(&request).unwrap();
let claims = signed_claims(&trust, "POST", path, &body);
let response = signed_request(app.clone(), &identity, &claims, body, "workspace-a").await;
assert_eq!(response.status(), StatusCode::FORBIDDEN);
request.workspace_id = f.scope.workspace_id.clone();
for _ in 0..2 {
let body = serde_json::to_vec(&request).unwrap();
let claims = signed_claims(&trust, "POST", path, &body);
assert_eq!(claims.operation, "workers:delete");
let response = signed_request(app.clone(), &identity, &claims, body, "workspace-a").await;
assert_eq!(response.status(), StatusCode::OK);
let bytes = axum::body::to_bytes(response.into_body(), 16384)
.await
.unwrap();
let receipt: runtime_api::SubjectCreateRecoveryResponse =
serde_json::from_slice(&bytes).unwrap();
assert_eq!(receipt.worker_id, request.worker_id);
assert_eq!(receipt.workspace_id, request.workspace_id);
assert_eq!(receipt.create_fingerprint, request.create_fingerprint);
assert_eq!(receipt.operation_id, request.operation_id);
}
}
#[tokio::test]
async fn explicit_material_erasure_requires_delete_capability_and_matching_scope() {
let f = Fixture::new();
+2
View File
@@ -32,6 +32,8 @@ pub mod workspace_issuer;
#[cfg(feature = "http-server")]
pub mod workspace_request;
#[cfg(feature = "test-support")]
pub use fs_store::test_support;
#[cfg(feature = "fs-store")]
pub use fs_store::{FsRuntimeStore, FsRuntimeStoreOptions};
pub use management::RuntimeOptions;
+737 -16
View File
@@ -640,7 +640,14 @@ impl FsWorkerRetentionProvider {
) -> Result<(WorkerSessionArchiveManifest, PathBuf), RuntimeError> {
let (manifest, session_root) = self.session_archive_identity(archive_id)?;
let archive_dir = self.archive_dir(archive_id)?;
let manifest = validate_existing_archive(&archive_dir, &manifest)?;
let manifest = validate_existing_archive(&archive_dir, &manifest, false)?;
session_store::WorkerSessionStore::open_read_only(&session_root).map_err(|_| {
RuntimeError::StoreCorrupt {
operation: "read Worker Session archive",
path: session_root.clone(),
message: "archive Session is unavailable in the current schema".into(),
}
})?;
Ok((manifest, session_root))
}
@@ -653,13 +660,7 @@ impl FsWorkerRetentionProvider {
&archive_dir.join("manifest.json"),
"read Worker Session archive",
)?;
if manifest.schema_version != ARCHIVE_SCHEMA_VERSION || manifest.archive_id != archive_id {
return Err(RuntimeError::StoreCorrupt {
operation: "read Worker Session archive",
path: archive_dir.join("manifest.json"),
message: "archive manifest identity mismatch".to_string(),
});
}
validate_archive_identity(&archive_dir, &manifest)?;
Ok((manifest, archive_dir.join("session")))
}
@@ -1108,7 +1109,7 @@ fn commit_session_archive(
};
let archive_dir = provider.archive_dir(archive_id)?;
if archive_dir.exists() {
return validate_existing_archive(&archive_dir, &manifest);
return validate_existing_archive(&archive_dir, &manifest, true);
}
let parent = archive_dir
.parent()
@@ -1150,7 +1151,7 @@ fn commit_session_archive(
source,
})?;
sync_directory(parent, "archive Worker Session")?;
validate_existing_archive(&archive_dir, &manifest)
validate_existing_archive(&archive_dir, &manifest, true)
})();
if result.is_err() {
let _ = fs::remove_dir_all(&staging);
@@ -1158,17 +1159,334 @@ fn commit_session_archive(
result
}
// The public archive manifest is an immutable ingestion receipt also retained by
// Backend. Schema migration replaces the payload, not that receipt. This local
// storage attestation binds the new bytes to the exact verified ingestion receipt.
// It is published only with the entire staged Runtime tree, never by a reader.
#[derive(Serialize, Deserialize)]
struct MigratedArchiveStorage {
schema_version: u32,
ingestion: WorkerSessionArchiveManifest,
checksum_sha256: String,
bytes: u64,
file_count: u64,
}
const ARCHIVE_STORAGE_FILE: &str = "session-storage.json";
fn archive_storage(
archive_dir: &Path,
ingestion: &WorkerSessionArchiveManifest,
) -> Result<Option<MigratedArchiveStorage>, RuntimeError> {
let path = archive_dir.join(ARCHIVE_STORAGE_FILE);
match fs::read(&path) {
Ok(bytes) => {
let storage: MigratedArchiveStorage =
serde_json::from_slice(&bytes).map_err(|error| RuntimeError::StoreCorrupt {
operation: "read migrated archive storage",
path: path.clone(),
message: error.to_string(),
})?;
if storage.schema_version != 1 || &storage.ingestion != ingestion {
return Err(RuntimeError::StoreCorrupt {
operation: "read migrated archive storage",
path,
message: "archive storage attestation does not match ingestion receipt".into(),
});
}
Ok(Some(storage))
}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(None),
Err(source) => Err(RuntimeError::StoreIo {
operation: "read migrated archive storage",
path,
source,
}),
}
}
// Share discovery and erasure fencing between Runtime preflight and staging.
// Unreadable archives are retained but excluded before any conversion writes;
// one historical archive must not block the live Runtime's startup. Session
// Store owns conversion admission, including the current resource limits.
fn for_each_live_session(
root: &Path,
mut visit: impl FnMut(&Path, bool) -> Result<(), RuntimeError>,
) -> Result<(), RuntimeError> {
for directory in ["workers", "orphaned-workers", "archives/workers"] {
let directory = root.join(directory);
let entries = match fs::read_dir(&directory) {
Ok(entries) => entries,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue,
Err(source) => {
return Err(RuntimeError::StoreIo {
operation: "enumerate Session migration",
path: directory,
source,
});
}
};
for entry in entries {
let entry = entry.map_err(|source| RuntimeError::StoreIo {
operation: "enumerate Session migration",
path: directory.clone(),
source,
})?;
let aggregate = entry.path();
let session = aggregate.join("session");
let is_archive = directory == root.join("archives/workers");
let identity = (|| {
if !session
.join("session.json")
.try_exists()
.map_err(|source| RuntimeError::StoreIo {
operation: "inspect Session migration",
path: session.clone(),
source,
})?
{
return Ok(None);
}
let identity: CanonicalSessionManifest = read_json(
&session.join("session.json"),
"read Session migration identity",
)?;
identity.session_id.parse().map(Some).map_err(|_| {
RuntimeError::InvalidRequest("Session migration identity is invalid".into())
})
})();
let session_id = match identity {
Ok(Some(session_id)) => session_id,
Ok(None) => continue,
Err(error) if is_archive => {
warn_unavailable_archive(
&aggregate,
ArchiveMigrationFailure::runtime("session_identity", &error),
);
continue;
}
Err(error) => return Err(error),
};
match runtime_session_retention(root).ensure_live(session_id) {
Ok(()) => {}
// Retention-ledger failures are not archive-local. Never bypass
// erasure authority while isolating an unreadable archive.
Err(session_store::StoreError::SessionErased(_)) => continue,
Err(error) => return Err(retention_error(error)),
}
if is_archive {
if let Err(failure) = validate_archive_migration(&aggregate) {
warn_unavailable_archive(&aggregate, failure);
continue;
}
}
// Publication failures still abort the outer staging transaction.
visit(&aggregate, is_archive)?;
}
}
Ok(())
}
struct ArchiveMigrationFailure {
stage: &'static str,
reason: &'static str,
}
impl ArchiveMigrationFailure {
fn runtime(stage: &'static str, error: &RuntimeError) -> Self {
let reason = match error {
RuntimeError::StoreIo { source, .. } => safe_io_reason(source),
RuntimeError::StoreCorrupt { message, .. } => match message.as_str() {
"archive manifest identity mismatch" => "archive_identity_mismatch",
"archive id collision or manifest mismatch" => "archive_manifest_mismatch",
"archive checksum or content summary mismatch" => "archive_checksum_mismatch",
"archive storage attestation does not match ingestion receipt" => {
"archive_storage_attestation_mismatch"
}
_ => "invalid_stored_data",
},
RuntimeError::InvalidRequest(_) => "invalid_identity",
_ => "archive_validation_failed",
};
Self { stage, reason }
}
fn conversion(error: session_store::StoreError) -> Self {
use session_store::StoreError;
let reason = match error {
StoreError::Io(error) => safe_io_reason(&error),
StoreError::Serde(_) => "invalid_session_manifest",
StoreError::Corrupt { message, .. }
if message.starts_with("unsupported Worker Session schema version ") =>
{
"unsupported_session_schema"
}
// The legacy decoder wraps InputMetadataError in a path-bearing Corrupt
// message. Recognize only its fixed cause; never publish that wrapper.
StoreError::Corrupt { message, .. }
if message.ends_with(&format!(
": {}",
session_store::InputMetadataError::ResourceLimit
)) =>
{
"input_resource_limit_exceeded"
}
StoreError::Corrupt { .. } => "invalid_session_log",
StoreError::InvalidInputMetadata(session_store::InputMetadataError::ResourceLimit) => {
"input_resource_limit_exceeded"
}
StoreError::InvalidInputMetadata(_) => "invalid_input_metadata",
StoreError::ReadLimitExceeded => "read_limit_exceeded",
_ => "session_conversion_failed",
};
Self {
stage: "session_conversion",
reason,
}
}
}
fn safe_io_reason(error: &std::io::Error) -> &'static str {
match error.kind() {
std::io::ErrorKind::NotFound => "not_found",
std::io::ErrorKind::PermissionDenied => "permission_denied",
std::io::ErrorKind::InvalidData => "invalid_data",
_ => "io_error",
}
}
fn validate_archive_migration(aggregate: &Path) -> Result<(), ArchiveMigrationFailure> {
let session = aggregate.join("session");
// Current archives need no conversion. Their payload integrity is checked
// on access, not by scanning every retained byte during Runtime startup.
if session_store::WorkerSessionStore::open_read_only(&session).is_ok() {
return Ok(());
}
let manifest: WorkerSessionArchiveManifest = read_json(
&aggregate.join("manifest.json"),
"read archive before Session migration",
)
.map_err(|error| ArchiveMigrationFailure::runtime("archive_manifest", &error))?;
validate_existing_archive(aggregate, &manifest, false)
.map_err(|error| ArchiveMigrationFailure::runtime("archive_integrity", &error))?;
session_store::WorkerSessionStore::validate_migration(&session)
.map_err(ArchiveMigrationFailure::conversion)
}
fn warn_unavailable_archive(aggregate: &Path, failure: ArchiveMigrationFailure) {
let archive_id = aggregate.file_name().unwrap_or_default().to_string_lossy();
tracing::warn!(
archive_id = %bounded_diagnostic_id(&archive_id),
code = "session_archive_unavailable",
stage = failure.stage,
reason = failure.reason,
"Retained Session archive could not be admitted for migration; original bytes retained"
);
}
/// Read-only admission probe. Only convertible archives request a staged pass;
/// unreadable originals remain available for archive-local diagnostics/recovery.
/// Never create a writable Session store or rewrite archive attestations here.
pub(crate) fn sessions_require_migration(root: &Path) -> Result<bool, RuntimeError> {
let mut required = false;
for_each_live_session(root, |aggregate, _| {
required |=
session_store::WorkerSessionStore::open_read_only(aggregate.join("session")).is_err();
Ok(())
})?;
Ok(required)
}
/// Called only on the private Runtime migration staging tree under its owner
/// lock. Failure discards the whole tree, including manifests and attestations.
pub(crate) fn migrate_staged_sessions(root: &Path) -> Result<(), RuntimeError> {
for_each_live_session(root, |aggregate, is_archive| {
let session = aggregate.join("session");
// Current Sessions need no writable admission, and their storage
// attestations must remain byte-for-byte unchanged.
if session_store::WorkerSessionStore::open_read_only(&session).is_ok() {
return Ok(());
}
let archive = if is_archive {
let manifest: WorkerSessionArchiveManifest = read_json(
&aggregate.join("manifest.json"),
"read archive before Session migration",
)?;
Some(validate_existing_archive(aggregate, &manifest, false)?)
} else {
None
};
session_store::WorkerSessionStore::new(&session).map_err(|error| {
RuntimeError::StoreCorrupt {
operation: "migrate Session schema",
path: session.clone(),
message: error.to_string(),
}
})?;
if let Some(ingestion) = archive {
let (checksum_sha256, bytes, file_count) = checksum_files(
&collect_files(&session, "migrate archive storage")?,
"migrate archive storage",
)?;
atomic_write_json(
&aggregate.join(ARCHIVE_STORAGE_FILE),
&MigratedArchiveStorage {
schema_version: 1,
ingestion,
checksum_sha256,
bytes,
file_count,
},
"commit migrated archive storage",
)?;
}
Ok(())
})
}
fn validate_archive_identity(
archive_dir: &Path,
manifest: &WorkerSessionArchiveManifest,
) -> Result<(), RuntimeError> {
if manifest.schema_version != ARCHIVE_SCHEMA_VERSION
|| archive_dir.file_name().and_then(|name| name.to_str()) != Some(&manifest.archive_id)
{
return Err(RuntimeError::StoreCorrupt {
operation: "read Worker Session archive",
path: archive_dir.join("manifest.json"),
message: "archive manifest identity mismatch".into(),
});
}
Ok(())
}
fn validate_existing_archive(
archive_dir: &Path,
expected: &WorkerSessionArchiveManifest,
require_current_source: bool,
) -> Result<WorkerSessionArchiveManifest, RuntimeError> {
let existing: WorkerSessionArchiveManifest = read_json(
&archive_dir.join("manifest.json"),
"verify Worker Session archive",
)?;
validate_archive_identity(archive_dir, &existing)?;
let storage = archive_storage(archive_dir, &existing)?;
let mut comparable_expected = expected.clone();
comparable_expected.archived_at_unix_seconds = existing.archived_at_unix_seconds;
if existing != comparable_expected {
let mut current = existing.clone();
if let Some(storage) = &storage {
current.content_checksum_sha256 = storage.checksum_sha256.clone();
current.content_bytes = storage.bytes;
current.content_file_count = storage.file_count;
}
// Reads use the original receipt; a retry with a migrated live source uses
// the current storage summary. All other identity/policy fields must agree.
let expected_manifest = if require_current_source {
&current
} else {
&existing
};
if expected_manifest != &comparable_expected {
return Err(RuntimeError::StoreCorrupt {
operation: "verify Worker Session archive",
path: archive_dir.join("manifest.json"),
@@ -1180,9 +1498,9 @@ fn validate_existing_archive(
"verify Worker Session archive",
)?;
let (checksum, bytes, count) = checksum_files(&files, "verify Worker Session archive")?;
if checksum != existing.content_checksum_sha256
|| bytes != existing.content_bytes
|| count != existing.content_file_count
if checksum != current.content_checksum_sha256
|| bytes != current.content_bytes
|| count != current.content_file_count
{
return Err(RuntimeError::StoreCorrupt {
operation: "verify Worker Session archive",
@@ -1605,12 +1923,16 @@ fn temporary_path(path: &Path) -> PathBuf {
path.with_file_name(format!(".{name}.tmp-{}-{sequence}", std::process::id()))
}
#[cfg(all(test, feature = "fs-store"))]
#[path = "retention/runtime_v5_tests.rs"]
mod runtime_v5_tests;
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{Arc, Barrier};
fn write_json(path: &Path, value: &impl Serialize) {
pub(super) fn write_json(path: &Path, value: &impl Serialize) {
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, serde_json::to_vec_pretty(value).unwrap()).unwrap();
}
@@ -1638,7 +1960,7 @@ mod tests {
fs::write(worker.join("runs/attempt-a/worker.sock"), b"not retained").unwrap();
}
fn request(
pub(super) fn request(
worker_id: WorkerId,
disposition: SessionDisposition,
) -> WorkerRetentionExecutionRequest {
@@ -1660,6 +1982,405 @@ mod tests {
}
}
fn old_compacted_source(root: &Path, worker_id: WorkerId) -> session_store::SegmentId {
use session_store::{
LogEntry, LoggedContentPart, LoggedHistoryEntry, LoggedItem, LoggedRole,
LoggedSessionHistoryEntryId, LoggedSessionHistoryMetadata, LoggedSessionHistoryOrigin,
Store,
};
let worker = root.join("workers").join(worker_id.to_string());
write_json(
&worker.join("worker.json"),
&serde_json::json!({"workspace_id": "workspace-a"}),
);
let store = session_store::WorkerSessionStore::new(worker.join("session")).unwrap();
let session_id = session_store::new_session_id();
let parent = session_store::new_segment_id();
let compact = session_store::new_segment_id();
let original = LoggedHistoryEntry {
item: LoggedItem::Message {
role: LoggedRole::User,
content: vec![LoggedContentPart::Text {
text: "request".into(),
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("original".into()),
origin: LoggedSessionHistoryOrigin::LocalHumanInput,
derivation: None,
},
};
let summary = LoggedHistoryEntry {
item: LoggedItem::Message {
role: LoggedRole::System,
content: vec![LoggedContentPart::Text {
text: "[Compacted context summary]\n\nsummary".into(),
}],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId("summary".into()),
origin: LoggedSessionHistoryOrigin::DerivedSummary,
derivation: Some(session_store::LoggedHistoryDerivation {
sources: vec![original.metadata.entry_id.clone()],
}),
},
};
for (id, history, compacted_from) in [
(parent, vec![original], None),
(
compact,
vec![summary],
Some(session_store::SegmentOrigin {
segment_id: parent,
at_turn_index: 0,
}),
),
] {
store
.create_segment(
session_id,
id,
&[LogEntry::AnnotatedSegmentStart {
ts: 1,
session_id,
system_prompt: None,
config: Default::default(),
history,
forked_from: None,
compacted_from,
}],
)
.unwrap();
}
let mut metadata = session_store::WorkerMetadata::new(
worker_id.to_string(),
Some(session_store::WorkerActiveSegmentRef::active_segment(
session_id, compact,
)),
);
metadata.workspace_id = Some("workspace-a".into());
write_json(&worker.join("metadata.json"), &metadata);
write_json(
&worker.join("session/session.json"),
&serde_json::json!({"schema_version": 3, "session_id": session_id}),
);
compact
}
#[test]
fn session_migration_repairs_failed_archive_and_preserves_ingestion_receipt_on_retry() {
let temp = tempfile::tempdir().unwrap();
let worker = WorkerId::now_v7();
let compact = old_compacted_source(temp.path(), worker);
let provider = FsWorkerRetentionProvider::new(temp.path());
let request = request(worker, SessionDisposition::Archive);
assert!(
provider
.execute(&request)
.unwrap_err()
.to_string()
.contains("Archived canonical source is unreadable")
);
let archive_dir = provider.archive_dir("archive-a").unwrap();
let original_manifest = fs::read(archive_dir.join("manifest.json")).unwrap();
// Production invokes this only in the private tree before atomic switch.
migrate_staged_sessions(temp.path()).unwrap();
assert_eq!(
fs::read(archive_dir.join("manifest.json")).unwrap(),
original_manifest
);
let (manifest, session_root) = provider.session_archive("archive-a").unwrap();
let reader = session_store::public_material::SessionPublicMaterialReader::open_worker(
&session_root,
compact,
Default::default(),
)
.unwrap();
assert_eq!(reader.descriptors().len(), 1);
let once = fs::read(archive_dir.join(ARCHIVE_STORAGE_FILE)).unwrap();
migrate_staged_sessions(temp.path()).unwrap();
assert_eq!(
fs::read(archive_dir.join(ARCHIVE_STORAGE_FILE)).unwrap(),
once
);
let result = provider.execute(&request).unwrap();
assert!(result.source_removed);
assert_eq!(result.archive.as_ref(), Some(&manifest));
assert_eq!(provider.execute(&request).unwrap(), result);
assert!(!provider.worker_dir(worker).exists());
// Both historical receipt and rewritten payload remain verified.
fs::write(
session_root
.join("segments")
.join(format!("{compact}.jsonl")),
b"tampered",
)
.unwrap();
assert!(provider.session_archive("archive-a").is_err());
}
#[test]
fn session_migration_updates_published_archive_without_changing_completed_receipt() {
let temp = tempfile::tempdir().unwrap();
let worker = WorkerId::now_v7();
let compact = old_compacted_source(temp.path(), worker);
let provider = FsWorkerRetentionProvider::new(temp.path());
let path = provider
.worker_dir(worker)
.join("session/segments")
.join(format!("{compact}.jsonl"));
let old = fs::read_to_string(&path).unwrap();
fs::write(&path, old.replace("derived_summary", "compaction_summary")).unwrap();
let request = request(worker, SessionDisposition::Archive);
// Frozen schema-v3 completed-operation fixture: the old Runtime had
// already accepted this payload and removed its live source.
let archive =
commit_session_archive(&provider, &request, &provider.worker_dir(worker)).unwrap();
let completed = WorkerRetentionExecutionResult {
operation_id: request.operation_id.clone(),
input_fingerprint: request.input_fingerprint.clone(),
worker_id: worker,
session_disposition: SessionDisposition::Archive,
diagnostics_disposition: DiagnosticsDisposition::Purge,
archive: Some(archive),
source_removed: true,
diagnostics_retained: false,
};
write_json(
&provider.operation_path(&request.operation_id).unwrap(),
&RetentionOperationReceipt {
schema_version: OPERATION_SCHEMA_VERSION,
request: request.clone(),
result: completed.clone(),
},
);
fs::remove_dir_all(provider.worker_dir(worker)).unwrap();
let receipt = fs::read(provider.operation_path(&request.operation_id).unwrap()).unwrap();
migrate_staged_sessions(temp.path()).unwrap();
let (manifest, root) = provider.session_archive("archive-a").unwrap();
assert_eq!(completed.archive.as_ref(), Some(&manifest));
assert_eq!(
fs::read(provider.operation_path(&request.operation_id).unwrap()).unwrap(),
receipt
);
assert_eq!(provider.execute(&request).unwrap(), completed);
assert!(session_store::WorkerSessionStore::open_read_only(&root).is_ok());
let storage_path = provider
.archive_dir("archive-a")
.unwrap()
.join(ARCHIVE_STORAGE_FILE);
let mut storage: MigratedArchiveStorage = read_json(&storage_path, "test").unwrap();
storage.ingestion.archive_id = "another-archive".into();
write_json(&storage_path, &storage);
assert!(provider.session_archive("archive-a").is_err());
}
#[test]
fn session_migration_preserves_corrupt_archive_without_blocking_other_sessions() {
let temp = tempfile::tempdir().unwrap();
let worker = WorkerId::now_v7();
let compact = old_compacted_source(temp.path(), worker);
let provider = FsWorkerRetentionProvider::new(temp.path());
let request = request(worker, SessionDisposition::Archive);
assert!(provider.execute(&request).is_err());
let archive = provider.archive_dir("archive-a").unwrap();
let path = archive
.join("session/segments")
.join(format!("{compact}.jsonl"));
fs::write(&path, b"corrupt\n").unwrap();
migrate_staged_sessions(temp.path()).unwrap();
assert_eq!(fs::read(&path).unwrap(), b"corrupt\n");
assert!(!archive.join(ARCHIVE_STORAGE_FILE).exists());
}
#[cfg(feature = "fs-store")]
#[test]
fn archive_admission_logs_safe_causes_and_startup_preserves_original_bytes() {
// Exercise FsRuntimeStore startup, real archive admission and Session conversion;
// only the tracing sink is substituted, and all storage is temporary.
#[derive(Clone)]
struct Capture(Arc<std::sync::Mutex<Vec<u8>>>);
impl std::io::Write for Capture {
fn write(&mut self, bytes: &[u8]) -> std::io::Result<usize> {
self.0.lock().unwrap().extend_from_slice(bytes);
Ok(bytes.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
use crate::fs_store::FsRuntimeStore;
const PRIVATE: &str = "private-session-content-token=/private/host/session";
for (case, stage, reason) in [
("identity_json", "session_identity", "invalid_stored_data"),
("identity_value", "session_identity", "invalid_identity"),
("manifest_json", "archive_manifest", "invalid_stored_data"),
(
"manifest_identity",
"archive_integrity",
"archive_identity_mismatch",
),
("storage_json", "archive_integrity", "invalid_stored_data"),
("checksum", "archive_integrity", "archive_checksum_mismatch"),
("schema", "session_conversion", "unsupported_session_schema"),
("log", "session_conversion", "invalid_session_log"),
(
"resource_limit",
"session_conversion",
"input_resource_limit_exceeded",
),
] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("private-runtime-root");
drop(
crate::Runtime::with_fs_store(
crate::fs_store::FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a"),
)
.unwrap(),
);
let worker = WorkerId::now_v7();
let segment = old_compacted_source(&root, worker);
let provider = FsWorkerRetentionProvider::new(&root);
let worker_dir = provider.worker_dir(worker);
let log = worker_dir
.join("session/segments")
.join(format!("{segment}.jsonl"));
if case == "log" {
fs::write(&log, format!("{{\"type\":\"{PRIVATE}\"}}\n")).unwrap();
} else if case == "schema" {
let path = worker_dir.join("session/session.json");
let mut manifest: serde_json::Value = read_json(&path, "test").unwrap();
manifest["schema_version"] = serde_json::json!(999);
write_json(&path, &manifest);
} else if case == "resource_limit" {
let mut bytes = fs::read_to_string(&log).unwrap();
for line in include_str!("retention/fixtures/system-input-v4.jsonl").lines() {
let mut entry: serde_json::Value = serde_json::from_str(line).unwrap();
if let Some(item) = entry.pointer_mut("/entry/item")
&& item.get("snapshot").is_some()
{
item["snapshot"] = serde_json::json!({
"private": PRIVATE.repeat(128 * 1024 / PRIVATE.len() + 1)
});
}
bytes.push_str(&serde_json::to_string(&entry).unwrap());
bytes.push('\n');
}
fs::write(&log, bytes).unwrap();
}
commit_session_archive(
&provider,
&request(worker, SessionDisposition::Archive),
&worker_dir,
)
.unwrap();
fs::remove_dir_all(worker_dir).unwrap();
let archive = provider.archive_dir("archive-a").unwrap();
match case {
"identity_json" => write_json(
&archive.join("session/session.json"),
&serde_json::json!({"session_id": [PRIVATE]}),
),
"identity_value" => write_json(
&archive.join("session/session.json"),
&serde_json::json!({"session_id": PRIVATE}),
),
"manifest_json" => write_json(
&archive.join("manifest.json"),
&serde_json::json!({"schema_version": PRIVATE}),
),
"manifest_identity" => {
let path = archive.join("manifest.json");
let mut manifest: WorkerSessionArchiveManifest =
read_json(&path, "test").unwrap();
manifest.archive_id = PRIVATE.into();
write_json(&path, &manifest);
}
"storage_json" => write_json(
&archive.join(ARCHIVE_STORAGE_FILE),
&serde_json::json!({"schema_version": PRIVATE}),
),
"checksum" => fs::write(
archive
.join("session/segments")
.join(format!("{segment}.jsonl")),
PRIVATE,
)
.unwrap(),
_ => {}
}
let snapshot = || {
collect_files(&archive, "test")
.unwrap()
.into_iter()
.map(|(relative, path)| (relative, fs::read(path).unwrap()))
.collect::<Vec<_>>()
};
let before = snapshot();
for _ in 0..2 {
let output = Arc::new(std::sync::Mutex::new(Vec::new()));
let writer = Capture(output.clone());
let subscriber = tracing_subscriber::fmt()
.with_ansi(false)
.without_time()
.with_writer(move || writer.clone())
.finish();
tracing::subscriber::with_default(subscriber, || {
tracing::callsite::rebuild_interest_cache();
drop(FsRuntimeStore::open_or_create(root.clone(), "runtime-a").unwrap());
});
let log = String::from_utf8(output.lock().unwrap().clone()).unwrap();
assert!(log.contains("session_archive_unavailable"), "{case}: {log}");
assert!(log.contains("archive_id=archive-a"), "{case}: {log}");
assert!(log.contains(&format!("stage=\"{stage}\"")), "{case}: {log}");
assert!(
log.contains(&format!("reason=\"{reason}\"")),
"{case}: {log}"
);
assert!(log.contains("original bytes retained"), "{case}: {log}");
assert!(!log.contains("private-session-content"), "{case}: {log}");
assert!(!log.contains("/private"), "{case}: {log}");
assert!(!log.contains("private-runtime-root"), "{case}: {log}");
assert_eq!(snapshot(), before, "{case}");
assert!(!archive.join(ARCHIVE_STORAGE_FILE).exists() || case == "storage_json");
}
}
}
#[test]
fn session_migration_does_not_read_or_upgrade_erased_originals() {
let temp = tempfile::tempdir().unwrap();
let worker = WorkerId::now_v7();
let compact = old_compacted_source(temp.path(), worker);
let session = temp
.path()
.join("workers")
.join(worker.to_string())
.join("session");
let manifest = fs::read(session.join("session.json")).unwrap();
let identity: CanonicalSessionManifest = serde_json::from_slice(&manifest).unwrap();
let id = identity.session_id.parse().unwrap();
let retention = runtime_session_retention(temp.path());
retention
.delete(
id,
session_store::retention::SessionDeletionIntent::ErasePermanently,
|| Ok(()),
)
.unwrap();
let path = session.join("segments").join(format!("{compact}.jsonl"));
fs::write(&path, b"unreadable fenced content\n").unwrap();
migrate_staged_sessions(temp.path()).unwrap();
assert_eq!(fs::read(session.join("session.json")).unwrap(), manifest);
assert_eq!(fs::read(path).unwrap(), b"unreadable fenced content\n");
assert!(matches!(
retention.ensure_live(id),
Err(session_store::StoreError::SessionErased(_))
));
}
#[test]
fn archive_is_verified_before_source_removal_and_retry_converges() {
let temp = tempfile::tempdir().unwrap();
@@ -0,0 +1,2 @@
{"kind":"annotated_system_item","ts":101,"entry":{"item":{"kind":"journal_startup","body":"journal body\nunchanged","snapshot":{"entries":[{"text":"private snapshot"}]},"snapshot_sha256":"snapshot-digest","prompt_provenance":{"workspace_id":"workspace","source_digest":"source","projection_digest":"projection","logical_name":"journal_startup"}},"metadata":{"entry_id":"journal-id","origin":{"kind":"legacy_unknown"},"derivation":{"sources":["source-id"]}}},"extensions":[{"domain":"migration-fixture","payload":{"kept":true}}]}
{"kind":"annotated_system_item","ts":103,"entry":{"item":{"kind":"subject_behavior_refresh","body":"rendered subject body","behavior_md":"exact behavior","prompt_provenance":{"source_digest":"subject-source","projection_digest":"subject-projection","logical_name":"subject_behavior"}},"metadata":{"entry_id":"subject-id","origin":{"kind":"backend_instruction","operation_id":"op"}}}}
@@ -0,0 +1,774 @@
//! Runtime integration, not a second implementation of Session migration.
//! The Feature records are frozen v4 wire copied from the Session Store fixture;
//! only the empty segment envelope is built with current APIs.
use super::tests::{request, write_json};
use super::*;
use crate::fs_store::{FsRuntimeStore, FsRuntimeStoreOptions};
use serde_json::{Value, json};
use session_store::{LogEntry, SegmentId, SessionId, Store, WorkerSessionStore};
use std::collections::BTreeMap;
const V4_SYSTEM_INPUTS: &str = include_str!("fixtures/system-input-v4.jsonl");
fn write_v4_session(aggregate: &Path) -> (SessionId, SegmentId) {
let root = aggregate.join("session");
let session = session_store::new_session_id();
let segment = session_store::new_segment_id();
let store = WorkerSessionStore::new(&root).unwrap();
store
.create_segment(
session,
segment,
&[LogEntry::AnnotatedSegmentStart {
ts: 1,
session_id: session,
system_prompt: None,
config: Default::default(),
history: vec![],
forked_from: None,
compacted_from: None,
}],
)
.unwrap();
let path = root.join("segments").join(format!("{segment}.jsonl"));
let mut bytes = fs::read(&path).unwrap();
bytes.extend_from_slice(V4_SYSTEM_INPUTS.as_bytes());
fs::write(path, bytes).unwrap();
write_json(
&root.join("session.json"),
&json!({
"schema_version": 4, "session_id": session,
}),
);
(session, segment)
}
fn stopped_worker(root: &Path, schema: u32) -> (WorkerId, SessionId, SegmentId) {
let worker = WorkerId::now_v7();
let aggregate = root.join("workers").join(worker.to_string());
// Serialize request defaults just as a production aggregate writer does.
let request: crate::catalog::CreateWorkerRequest = serde_json::from_value(json!({
"worker_id": worker,
"workspace_api": {"workspace_id": "workspace-a", "base_url": "https://workspace.invalid"},
"create_fingerprint": "runtime-v5-fixture",
"profile": {"kind": "builtin", "value": "builtin:coder"},
"profile_source": {"kind": "workspace_config", "archive": {
"id": "archive-1", "digest": "sha256:archive", "size_bytes": 0,
"source_graph": {"source_count": 0, "total_source_bytes": 0,
"entrypoints": {}, "import_count": 0}
}}
}))
.unwrap();
write_json(
&aggregate.join("worker.json"),
&json!({
"schema_version": schema,
"worker_ref": {"worker_id": worker}, "worker_id": worker,
"profile": request.profile,
"profile_source": request.profile_source.reference(),
"display_name": null, "config_bundle": null,
"created_at_ms": 1234, "status": "stopped", "workspace_id": "workspace-a",
"last_finished_submission_request_id": "retained-submission",
"restore_guard": {"observation_token": "retained-token",
"active_request_id": null, "owners": {}},
"execution_state": {"state": "available", "execution": {
"request": request, "binding": {}, "restore_intent": "explicit"
}}
}),
);
let (session, segment) = write_v4_session(&aggregate);
let mut metadata = session_store::WorkerMetadata::new(
format!("worker-runtime-{worker}"),
Some(session_store::WorkerActiveSegmentRef::active_segment(
session, segment,
)),
);
metadata.workspace_id = Some("workspace-a".into());
write_json(&aggregate.join("metadata.json"), &metadata);
(worker, session, segment)
}
fn runtime_fixture(root: &Path, schema: u32) -> (WorkerId, SessionId, SegmentId) {
fs::create_dir_all(root.join("workers")).unwrap();
let fenced_worker = WorkerId::now_v7();
write_json(
&root.join("runtime.json"),
&json!({
"schema_version": schema, "display_name": "retained Runtime",
"backend": "fs_store", "status": "running", "next_diagnostic_id": 1,
"config_bundles": {}, "workspace_owners": {"workspace-a": "owner-a"},
"diagnostics": [], "subject_releases": [],
"subject_create_recovery_fences": {fenced_worker.to_string(): {
"request": {"workspace_id": "workspace-a", "worker_id": fenced_worker,
"create_fingerprint": "retained-create", "operation_id": "retained-recovery"},
"response": null
}}
}),
);
stopped_worker(root, schema)
}
// Include every file, not just schema markers: failures must preserve original
// history, metadata, archive receipts, checksums and retention evidence together.
fn tree_bytes(root: &Path) -> BTreeMap<PathBuf, Vec<u8>> {
fn walk(root: &Path, at: &Path, result: &mut BTreeMap<PathBuf, Vec<u8>>) {
for entry in fs::read_dir(at).unwrap() {
let entry = entry.unwrap();
if entry.file_type().unwrap().is_dir() {
walk(root, &entry.path(), result);
} else {
result.insert(
entry.path().strip_prefix(root).unwrap().to_path_buf(),
fs::read(entry.path()).unwrap(),
);
}
}
}
let mut result = BTreeMap::new();
walk(root, root, &mut result);
result
}
fn assert_generic(root: &Path, session: SessionId, segment: SegmentId) {
use session_store::{InputCompactionUse, InputEvidenceUse, InputObservationUse};
let store = WorkerSessionStore::open_read_only(root).unwrap();
let entries = store.read_all_read_only(session, segment).unwrap();
assert_eq!(entries.len(), 3);
let originals: Vec<Value> = V4_SYSTEM_INPUTS
.lines()
.map(|line| serde_json::from_str(line).unwrap())
.collect();
for (log, old) in entries[1..].iter().zip(originals) {
let LogEntry::AnnotatedSystemItem { entry, .. } = log else {
panic!("lost System record: {log:?}");
};
let current = serde_json::to_value(log).unwrap();
assert_eq!(current["ts"], old["ts"]);
for field in ["entry_id", "origin", "derivation"] {
assert_eq!(
current["entry"]["metadata"][field],
old["entry"]["metadata"][field]
);
}
assert_eq!(current["entry"]["item"]["kind"], "input");
assert_eq!(
entry.item.history_text(),
old["entry"]["item"]["body"].as_str().unwrap()
);
let input = entry.metadata.input_metadata.as_ref().unwrap();
let usage = input.usage();
assert_eq!(usage.recording_material, InputEvidenceUse::Exclude);
assert_eq!(usage.memory_evidence, InputEvidenceUse::Exclude);
assert_eq!(usage.observation, InputObservationUse::Hidden);
let kind = old["entry"]["item"]["kind"].as_str().unwrap();
let schema = if kind == "journal_startup" {
assert_eq!(usage.compaction, InputCompactionUse::ReferenceInput);
assert_eq!(current["extensions"], old["extensions"]);
"journal:startup"
} else {
assert_eq!(usage.compaction, InputCompactionUse::Retain);
"subject:behavior"
};
let payload = input.feature_payload().unwrap();
assert_eq!(payload.schema(), schema);
assert_eq!(payload.version(), 1);
let mut expected_payload = old["entry"]["item"].clone();
expected_payload.as_object_mut().unwrap().remove("kind");
expected_payload.as_object_mut().unwrap().remove("body");
assert_eq!(
payload.decode::<Value>(schema, 1).unwrap(),
expected_payload
);
let public = serde_json::to_value(input.public_item().unwrap()).unwrap();
assert_eq!(public["kind"], kind);
assert_eq!(public["body"], old["entry"]["item"]["body"]);
assert_eq!(public["presentation"], json!({"kind": "context"}));
}
}
fn completed_v4_archive(
root: &Path,
schema: u32,
) -> (
SessionId,
SegmentId,
WorkerRetentionExecutionRequest,
WorkerRetentionExecutionResult,
) {
let (worker, session, segment) = stopped_worker(root, schema);
let provider = FsWorkerRetentionProvider::new(root);
let request = request(worker, SessionDisposition::Archive);
// Frozen old-runtime receipt: ingestion and removal completed before the
// current strict decoder existed. Do not ask the v5 reader to mint it.
let manifest =
commit_session_archive(&provider, &request, &provider.worker_dir(worker)).unwrap();
let completed = WorkerRetentionExecutionResult {
operation_id: request.operation_id.clone(),
input_fingerprint: request.input_fingerprint.clone(),
worker_id: worker,
session_disposition: SessionDisposition::Archive,
diagnostics_disposition: DiagnosticsDisposition::Purge,
archive: Some(manifest),
source_removed: true,
diagnostics_retained: false,
};
write_json(
&provider.operation_path(&request.operation_id).unwrap(),
&RetentionOperationReceipt {
schema_version: OPERATION_SCHEMA_VERSION,
request: request.clone(),
result: completed.clone(),
},
);
fs::remove_dir_all(provider.worker_dir(worker)).unwrap();
(session, segment, request, completed)
}
#[test]
fn runtime_v11_and_current_route_v4_stopped_orphan_archive_through_staging() {
for runtime_schema in [11, 12] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let (worker, session, segment) = runtime_fixture(&root, runtime_schema);
let orphan = root
.join("orphaned-workers")
.join(WorkerId::now_v7().to_string());
let (orphan_session, orphan_segment) = write_v4_session(&orphan);
// Include an unregistered tree still under workers, not just an
// already-archived orphan. Both Runtime source schemas must fence it
// out of the live catalog without discarding its Session.
let loose_id = WorkerId::now_v7();
let loose = root.join("workers").join(loose_id.to_string());
let (loose_session, loose_segment) = write_v4_session(&loose);
let retained_loose = root.join("orphaned-workers").join(loose_id.to_string());
let (archived_session, archived_segment, request, completed) =
completed_v4_archive(&root, runtime_schema);
let provider = FsWorkerRetentionProvider::new(&root);
let archive = provider.archive_dir("archive-a").unwrap();
// Also cover an archive which already has a local storage attestation
// from an earlier schema migration. Keep ingestion bytes immutable;
// harmless marker whitespace makes current storage differ from them.
if runtime_schema == 12 {
let marker = archive.join("session/session.json");
let mut bytes = fs::read(&marker).unwrap();
bytes.push(b'\n');
fs::write(marker, bytes).unwrap();
let ingestion = completed.archive.clone().unwrap();
let (checksum_sha256, bytes, file_count) = checksum_files(
&collect_files(&archive.join("session"), "test").unwrap(),
"test",
)
.unwrap();
write_json(
&archive.join(ARCHIVE_STORAGE_FILE),
&MigratedArchiveStorage {
schema_version: 1,
ingestion,
checksum_sha256,
bytes,
file_count,
},
);
}
let storage_before = fs::read(archive.join(ARCHIVE_STORAGE_FILE)).ok();
let manifest_before = fs::read(archive.join("manifest.json")).unwrap();
let receipt_path = provider.operation_path(&request.operation_id).unwrap();
let receipt_before = fs::read(&receipt_path).unwrap();
let stopped = root.join("workers").join(worker.to_string());
assert!(WorkerSessionStore::open_read_only(stopped.join("session")).is_err());
// Current decode itself must not be made permissive to fix routing.
for line in V4_SYSTEM_INPUTS.lines() {
assert!(serde_json::from_str::<LogEntry>(line).is_err());
}
let before = tree_bytes(&root);
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
let plan = FsRuntimeStore::migration_plan(&options).unwrap();
assert!(plan.migration_required);
assert_eq!(plan.archived_orphan_worker_count, 1);
assert_eq!(plan.current_schema_version, runtime_schema);
assert_eq!(tree_bytes(&root), before, "preflight mutated source");
// Exercise normal startup, not just the private migration helper.
let opened = FsRuntimeStore::open_or_create(root.clone(), "runtime-a").unwrap();
assert!(
matches!(
FsRuntimeStore::migrate(&options),
Err(RuntimeError::RuntimeStoreAlreadyOpen { .. })
),
"owner lock not fenced"
);
drop(opened);
assert_generic(&stopped.join("session"), session, segment);
assert_generic(&orphan.join("session"), orphan_session, orphan_segment);
assert!(!loose.exists());
assert_generic(
&retained_loose.join("session"),
loose_session,
loose_segment,
);
assert_generic(&archive.join("session"), archived_session, archived_segment);
assert_eq!(
fs::read(archive.join("manifest.json")).unwrap(),
manifest_before
);
assert_eq!(fs::read(&receipt_path).unwrap(), receipt_before);
let (manifest, archive_session) = provider.session_archive("archive-a").unwrap();
assert_eq!(completed.archive.as_ref(), Some(&manifest));
assert_eq!(provider.execute(&request).unwrap(), completed);
if let Some(storage_before) = storage_before {
assert_ne!(
fs::read(archive.join(ARCHIVE_STORAGE_FILE)).unwrap(),
storage_before
);
}
let storage: MigratedArchiveStorage =
read_json(&archive.join(ARCHIVE_STORAGE_FILE), "test").unwrap();
assert_eq!(storage.ingestion, manifest);
let (checksum, bytes, count) =
checksum_files(&collect_files(&archive_session, "test").unwrap(), "test").unwrap();
assert_eq!(
(storage.checksum_sha256, storage.bytes, storage.file_count),
(checksum, bytes, count)
);
if runtime_schema == 12 {
for name in [
PathBuf::from("runtime.json"),
PathBuf::from(format!("workers/{worker}/worker.json")),
PathBuf::from(format!("workers/{worker}/metadata.json")),
] {
assert_eq!(fs::read(root.join(&name)).unwrap(), before[&name]);
}
}
let once = tree_bytes(&root);
assert!(
!FsRuntimeStore::migrate(&options)
.unwrap()
.migration_required
);
drop(FsRuntimeStore::open_or_create(root.clone(), "runtime-a").unwrap());
assert_eq!(tree_bytes(&root), once, "retry rewrote migrated evidence");
assert!(!temp.path().join(".runtime.schema-v12-staging").exists());
assert!(!temp.path().join(".runtime.pre-schema-v12-backup").exists());
// session-storage, not the immutable receipt's old checksum, attests
// current bytes. Tampering with those bytes must remain detectable.
let path = archive_session
.join("segments")
.join(format!("{archived_segment}.jsonl"));
fs::write(path, b"tampered\n").unwrap();
assert!(provider.session_archive("archive-a").is_err());
}
}
#[test]
fn corrupt_archive_does_not_block_startup_or_healthy_session_migration_and_repair_converges() {
for runtime_schema in [11, 12] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let (worker, session, segment) = runtime_fixture(&root, runtime_schema);
let (archived_session, archived_segment, request, completed) =
completed_v4_archive(&root, runtime_schema);
let provider = FsWorkerRetentionProvider::new(&root);
let archive = provider.archive_dir("archive-a").unwrap();
let path = archive
.join("session/segments")
.join(format!("{archived_segment}.jsonl"));
let original = fs::read(&path).unwrap();
fs::write(&path, b"corrupt archive\n").unwrap();
let (healthy_worker, healthy_session, healthy_segment) =
stopped_worker(&root, runtime_schema);
let mut healthy_request =
super::tests::request(healthy_worker, SessionDisposition::Archive);
healthy_request.operation_id = "healthy-operation".into();
healthy_request.archive_id = Some("healthy-archive".into());
let healthy_manifest = commit_session_archive(
&provider,
&healthy_request,
&provider.worker_dir(healthy_worker),
)
.unwrap();
fs::remove_dir_all(provider.worker_dir(healthy_worker)).unwrap();
let before = tree_bytes(&archive);
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
assert!(
FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
drop(FsRuntimeStore::open_or_create(root.clone(), "runtime-a").unwrap());
assert_eq!(tree_bytes(&archive), before);
assert!(provider.session_archive("archive-a").is_err());
let (manifest, healthy_root) = provider.session_archive("healthy-archive").unwrap();
assert_eq!(manifest, healthy_manifest);
assert_generic(&healthy_root, healthy_session, healthy_segment);
assert!(!archive.join(ARCHIVE_STORAGE_FILE).exists());
assert_generic(
&root
.join("workers")
.join(worker.to_string())
.join("session"),
session,
segment,
);
assert!(
!FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
drop(FsRuntimeStore::open_or_create(root.clone(), "runtime-a").unwrap());
assert_eq!(tree_bytes(&archive), before);
fs::write(&path, original).unwrap();
FsRuntimeStore::migrate(&options).unwrap();
assert_generic(
&root
.join("workers")
.join(worker.to_string())
.join("session"),
session,
segment,
);
assert_generic(&archive.join("session"), archived_session, archived_segment);
assert_eq!(provider.execute(&request).unwrap(), completed);
}
}
#[test]
fn unmigratable_archive_preserves_evidence_across_startup_and_reopen() {
for (field, value) in [
("snapshot_sha256", json!(42)),
("behavior_md", json!([])),
("snapshot", json!({"large": "x".repeat(128 * 1024 + 1)})),
("body", json!("x".repeat(256 * 1024 + 1))),
] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let (live_worker, live_session, live_segment) = runtime_fixture(&root, 12);
let (worker, _, segment) = stopped_worker(&root, 12);
let provider = FsWorkerRetentionProvider::new(&root);
let path = provider
.worker_dir(worker)
.join("session/segments")
.join(format!("{segment}.jsonl"));
let source = fs::read_to_string(&path).unwrap();
let mut replaced = false;
let mut altered = String::new();
for line in source.lines() {
let mut entry: Value = serde_json::from_str(line).unwrap();
if let Some(item) = entry.pointer_mut("/entry/item")
&& item.get(field).is_some()
{
item[field] = value.clone();
replaced = true;
}
altered.push_str(&serde_json::to_string(&entry).unwrap());
altered.push('\n');
}
assert!(replaced);
fs::write(&path, altered).unwrap();
let request = request(worker, SessionDisposition::Archive);
let manifest =
commit_session_archive(&provider, &request, &provider.worker_dir(worker)).unwrap();
fs::remove_dir_all(provider.worker_dir(worker)).unwrap();
let archive = provider.archive_dir("archive-a").unwrap();
validate_existing_archive(&archive, &manifest, false).unwrap();
let before = tree_bytes(&archive);
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
for _ in 0..2 {
let runtime = crate::Runtime::with_fs_store(options.clone()).unwrap();
assert_eq!(runtime.summary().unwrap().worker_count, 1);
runtime.bind_runtime_identity("runtime-a").unwrap();
let search = runtime_api::SessionPublicSearchRequest {
workspace_id: "workspace-a".into(),
source: runtime_api::SessionPublicSource::Archived {
archive_id: "archive-a".into(),
},
expected_session_id: manifest.source_session_id.clone(),
expected_generation: None,
query: None,
kind: None,
tool_name: None,
tool_part: runtime_api::SessionPublicToolPart::Both,
scan_cursor: None,
limit: 10,
max_scan_bytes: 1024 * 1024,
max_segments: 10,
max_entries: 100,
};
let availability = runtime
.session_public_search_scoped(
&crate::runtime::RuntimeWorkspaceScope::new("workspace-a", "owner-a"),
&search,
)
.unwrap();
assert!(matches!(
availability,
runtime_api::SessionPublicSearchAvailability::Unavailable {
reason: runtime_api::SessionPublicUnavailableReason::ArchiveIncomplete,
..
}
));
drop(runtime);
assert_eq!(tree_bytes(&archive), before);
validate_existing_archive(&archive, &manifest, false).unwrap();
assert!(matches!(
provider.session_archive("archive-a"),
Err(RuntimeError::StoreCorrupt { .. })
));
assert!(
!FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
assert_generic(
&provider.worker_dir(live_worker).join("session"),
live_session,
live_segment,
);
}
}
}
#[test]
fn unreadable_archive_metadata_does_not_block_archive_only_runtime_startup() {
for (path, field, value) in [
("session/session.json", None, Value::Null),
("manifest.json", None, Value::Null),
(ARCHIVE_STORAGE_FILE, None, Value::Null),
("session/session.json", Some("schema_version"), json!(999)),
("manifest.json", Some("schema_version"), json!(999)),
(
"manifest.json",
Some("archive_id"),
json!("another-archive"),
),
] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let (worker, _, _) = runtime_fixture(&root, 12);
completed_v4_archive(&root, 12);
let provider = FsWorkerRetentionProvider::new(&root);
fs::remove_dir_all(provider.worker_dir(worker)).unwrap();
let archive = provider.archive_dir("archive-a").unwrap();
if let Some(field) = field {
let mut record: Value = read_json(&archive.join(path), "test").unwrap();
record[field] = value;
write_json(&archive.join(path), &record);
} else {
fs::write(archive.join(path), b"{broken\n").unwrap();
}
let before = tree_bytes(&archive);
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
for _ in 0..2 {
assert!(
!FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
let runtime = crate::Runtime::with_fs_store(options.clone()).unwrap();
assert_eq!(runtime.summary().unwrap().worker_count, 0);
drop(runtime);
assert!(provider.session_archive("archive-a").is_err());
assert_eq!(tree_bytes(&archive), before);
}
}
}
#[test]
fn archive_isolation_does_not_bypass_corrupt_erasure_authority() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
runtime_fixture(&root, 12);
let (session, _, _, _) = completed_v4_archive(&root, 12);
let ledger = root.join("retention/session-sources");
fs::create_dir_all(&ledger).unwrap();
fs::write(ledger.join(format!("{session}.json")), b"{broken\n").unwrap();
let before = tree_bytes(&root);
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
assert!(FsRuntimeStore::migration_plan(&options).is_err());
assert!(crate::Runtime::with_fs_store(options).is_err());
assert_eq!(tree_bytes(&root), before);
}
#[test]
fn runtime_v5_probe_and_staging_skip_erased_originals() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let (worker, live_session, live_segment) = runtime_fixture(&root, 12);
let erased = root
.join("orphaned-workers")
.join(WorkerId::now_v7().to_string());
let (session, segment) = write_v4_session(&erased);
runtime_session_retention(&root)
.delete(
session,
session_store::retention::SessionDeletionIntent::ErasePermanently,
|| Ok(()),
)
.unwrap();
fs::write(
erased
.join("session/segments")
.join(format!("{segment}.jsonl")),
b"fenced malformed JSON\n",
)
.unwrap();
let before = tree_bytes(&erased);
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
assert!(
FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
FsRuntimeStore::migrate(&options).unwrap();
assert_generic(
&root
.join("workers")
.join(worker.to_string())
.join("session"),
live_session,
live_segment,
);
assert_eq!(tree_bytes(&erased), before);
assert!(
!FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
assert!(matches!(
runtime_session_retention(&root).ensure_live(session),
Err(session_store::StoreError::SessionErased(_))
));
}
#[test]
fn runtime_v5_rejects_broken_or_future_source_without_rewriting_evidence() {
for corruption in ["broken-json", "future-schema"] {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let (worker, session, segment) = runtime_fixture(&root, 12);
let session_root = root
.join("workers")
.join(worker.to_string())
.join("session");
if corruption == "broken-json" {
fs::write(
session_root
.join("segments")
.join(format!("{segment}.jsonl")),
b"{not valid JSON\n",
)
.unwrap();
} else {
write_json(
&session_root.join("session.json"),
&json!({
"schema_version": 999, "session_id": session,
}),
);
}
let before = tree_bytes(&root);
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
assert!(
FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
assert_eq!(tree_bytes(&root), before);
assert!(FsRuntimeStore::migrate(&options).is_err());
assert_eq!(tree_bytes(&root), before);
assert!(!temp.path().join(".runtime.schema-v12-staging").exists());
assert!(!temp.path().join(".runtime.pre-schema-v12-backup").exists());
}
}
#[cfg(unix)]
#[test]
fn current_runtime_session_migration_still_rejects_live_worker_sockets() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let (worker, _, _) = runtime_fixture(&root, 12);
let before = tree_bytes(&root);
let runs = root.join("workers").join(worker.to_string()).join("runs");
fs::create_dir_all(runs.join("1")).unwrap();
let listener = std::os::unix::net::UnixListener::bind(runs.join("1/worker.sock")).unwrap();
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
let error = FsRuntimeStore::migrate(&options).unwrap_err();
assert!(error.to_string().contains("active Worker socket"));
drop(listener);
fs::remove_dir_all(runs).unwrap();
assert_eq!(tree_bytes(&root), before);
assert!(!temp.path().join(".runtime.schema-v12-staging").exists());
}
#[test]
fn current_runtime_upgrades_v5_role_metadata_through_staging_without_reserializing_owner_evidence()
{
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
let (worker, session, segment) = runtime_fixture(&root, 12);
let aggregate = root.join("workers").join(worker.to_string());
let path = aggregate
.join("session/segments")
.join(format!("{segment}.jsonl"));
let old = fs::read_to_string(&path).unwrap();
let header = old.lines().next().unwrap();
let source = include_str!(
"../../../session-store/src/legacy_session_log/tests/fixtures/v5-role-metadata.jsonl"
);
fs::write(&path, format!("{header}\n{source}")).unwrap();
write_json(
&aggregate.join("session/session.json"),
&json!({"schema_version":5, "session_id": session}),
);
let owner_before = fs::read(root.join("runtime.json")).unwrap();
let worker_before = fs::read(aggregate.join("worker.json")).unwrap();
let metadata_before = fs::read(aggregate.join("metadata.json")).unwrap();
assert!(WorkerSessionStore::open_read_only(aggregate.join("session")).is_err());
let options = FsRuntimeStoreOptions::new(&root).with_runtime_id("runtime-a");
assert!(
FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
FsRuntimeStore::migrate(&options).unwrap();
let store = WorkerSessionStore::open_read_only(aggregate.join("session")).unwrap();
let entries = store.read_all_read_only(session, segment).unwrap();
assert_eq!(entries.len(), 6);
for (current, old) in entries[1..].iter().zip(source.lines()) {
let current = serde_json::to_value(current).unwrap();
let old: Value = serde_json::from_str(old).unwrap();
assert_eq!(current["ts"], old["ts"]);
assert_eq!(current["entry"]["item"], old["entry"]["item"]);
for field in ["entry_id", "origin", "derivation"] {
assert_eq!(
current["entry"]["metadata"][field],
old["entry"]["metadata"][field]
);
}
if current["entry"]["metadata"]["origin"]["kind"] == "derived_summary" {
assert_eq!(
current["entry"]["metadata"]["input_metadata"]["usage"]["recording_material"],
"exclude"
);
assert_eq!(
current["entry"]["metadata"]["input_metadata"]["usage"]["memory_evidence"],
"exclude"
);
}
}
assert_eq!(fs::read(root.join("runtime.json")).unwrap(), owner_before);
assert_eq!(
fs::read(aggregate.join("worker.json")).unwrap(),
worker_before
);
assert_eq!(
fs::read(aggregate.join("metadata.json")).unwrap(),
metadata_before
);
let once = tree_bytes(&aggregate.join("session"));
assert!(
!FsRuntimeStore::migration_plan(&options)
.unwrap()
.migration_required
);
FsRuntimeStore::migrate(&options).unwrap();
assert_eq!(tree_bytes(&aggregate.join("session")), once);
}
File diff suppressed because it is too large Load Diff
@@ -398,6 +398,7 @@ mod tests {
content: vec![LoggedContentPart::Text { text: text.into() }],
},
metadata: LoggedSessionHistoryMetadata {
input_metadata: None,
entry_id: LoggedSessionHistoryEntryId(id.to_string()),
origin: LoggedSessionHistoryOrigin::HumanInput {
account_id: "account-1".to_string(),
+859
View File
@@ -0,0 +1,859 @@
//! Offline support for the approved, attachment-free stopped Worker fixture.
//!
//! Enabled only by `test-support`. This is not an import API: extraction accepts a
//! current-schema, settled, single-Segment aggregate; installation accepts only a
//! fresh Runtime directory containing its newly initialized `auth.toml`. No
//! Runtime or execution backend is constructed. Original Session bytes are never
//! rewritten. The caller must separately review/attest the artificial conversation;
//! structural checks and secret-marker checks cannot prove arbitrary prose safe.
use super::*;
use crate::catalog::ProfileSourceArchiveSource;
use crate::workspace_issuer::{
WorkspaceIssuerTrustRecord, WorkspaceIssuerTrustState, validate_workspace_issuer_trust_records,
};
use session_store::{
SegmentId, SessionId, WorkerActiveSegmentRef, WorkerMetadata, WorkerSessionStore,
};
#[cfg(test)]
#[path = "test_support_tests.rs"]
mod tests;
const TEMPLATE_VERSION: u32 = 1;
const MAX_SESSION_BYTES: usize = 8 * 1024 * 1024;
/// Portable fixture, intentionally without Debug (Session material is private).
/// Fields are private and every deserialized template is validated on install.
/// Serialize this value to retain the fixture; do not serialize its source aggregate.
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct StoppedWorkerTemplate {
template_version: u32,
runtime_schema_version: u32,
runtime_id: String,
workspace_id: String,
worker_id: WorkerId,
created_at_ms: Option<u64>,
session_id: SessionId,
segment_id: SegmentId,
session_manifest: Vec<u8>,
segment_jsonl: Vec<u8>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct SessionManifest {
schema_version: u32,
session_id: SessionId,
}
// Read-only projection: never retain source key material in a template.
#[derive(Deserialize)]
struct AuthIdentity {
identity_id: String,
}
#[derive(Deserialize)]
struct SourceAuth {
identity: AuthIdentity,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct InitializedAuth {
identity: crate::auth::RuntimeIdentityMaterial,
workspace_issuers: Vec<WorkspaceIssuerTrustRecord>,
#[serde(default)]
archived_legacy_issuers: Vec<String>,
}
#[derive(Deserialize)]
struct GuardState {
active_request_id: Option<String>,
owners: BTreeMap<String, GuardOwner>,
}
#[derive(Deserialize)]
struct GuardOwner {
receipt: Option<serde::de::IgnoredAny>,
}
fn reject(message: &'static str) -> RuntimeError {
RuntimeError::InvalidRequest(format!("stopped Worker fixture: {message}"))
}
fn io<T>(result: std::io::Result<T>) -> Result<T, RuntimeError> {
result.map_err(|_| reject("filesystem operation failed"))
}
fn decode<T: serde::de::DeserializeOwned>(bytes: &[u8]) -> Result<T, RuntimeError> {
serde_json::from_slice(bytes).map_err(|_| reject("invalid typed document"))
}
fn no_symlink_components(path: &Path) -> Result<(), RuntimeError> {
let mut current = PathBuf::new();
for part in path.components() {
if matches!(part, std::path::Component::ParentDir) {
return Err(reject("parent traversal is unsupported"));
}
current.push(part);
let metadata = io(fs::symlink_metadata(&current))?;
if metadata.file_type().is_symlink() {
return Err(reject("symlinks are unsupported"));
}
}
Ok(())
}
fn check_tree(path: &Path, budget: &mut usize) -> Result<(), RuntimeError> {
if *budget == 0 {
return Err(reject("file count exceeds fixture limit"));
}
*budget -= 1;
let metadata = io(fs::symlink_metadata(path))?;
if metadata.is_dir() {
for entry in io(fs::read_dir(path))? {
check_tree(&io(entry)?.path(), budget)?;
}
} else if !metadata.is_file() {
return Err(reject("symlinks and special files are unsupported"));
}
Ok(())
}
fn regular_file(path: &Path) -> Result<(), RuntimeError> {
no_symlink_components(path)?;
if !io(fs::symlink_metadata(path))?.is_file() {
return Err(reject("expected a regular file"));
}
Ok(())
}
fn bytes(path: &Path) -> Result<Vec<u8>, RuntimeError> {
regular_file(path)?;
let mut options = OpenOptions::new();
options.read(true);
#[cfg(target_os = "linux")]
{
use std::os::unix::fs::OpenOptionsExt;
options.custom_flags(libc::O_NOATIME | libc::O_NOFOLLOW);
}
let file = io(options.open(path))?;
let mut result = Vec::new();
io(file
.take(MAX_SESSION_BYTES as u64 + 1)
.read_to_end(&mut result))?;
if result.len() > MAX_SESSION_BYTES {
return Err(reject("file exceeds fixture limit"));
}
Ok(result)
}
fn text(path: &Path) -> Result<String, RuntimeError> {
String::from_utf8(bytes(path)?).map_err(|_| reject("invalid UTF-8 document"))
}
fn portable_id(id: &str) -> bool {
!id.is_empty()
&& id.len() <= 256
&& id
.bytes()
.all(|c| c.is_ascii_alphanumeric() || b"-_".contains(&c))
}
impl StoppedWorkerTemplate {
pub fn runtime_id(&self) -> &str {
&self.runtime_id
}
pub fn workspace_id(&self) -> &str {
&self.workspace_id
}
pub fn worker_id(&self) -> WorkerId {
self.worker_id
}
pub fn worker_name(&self) -> String {
runtime_worker_name(self.worker_id)
}
pub fn session_id(&self) -> SessionId {
self.session_id
}
pub fn segment_id(&self) -> SegmentId {
self.segment_id
}
/// Extract without migration, rewriting metadata, or copying credentials,
/// resolved manifests, execution-operation receipts, or run directories.
/// The source Runtime must be offline; its existing owner lock is held for
/// the entire read. The path must be `<runtime>/workers/<WorkerId>`.
pub fn extract(worker_dir: &Path) -> Result<Self, RuntimeError> {
no_symlink_components(worker_dir)?;
let workers = worker_dir
.parent()
.ok_or_else(|| reject("invalid aggregate path"))?;
if workers.file_name().and_then(|s| s.to_str()) != Some(WORKERS_DIR) {
return Err(reject("invalid aggregate path"));
}
let root = workers
.parent()
.ok_or_else(|| reject("invalid Runtime path"))?;
// Require the existing lock, rather than creating a file in the source.
let lock_path = root.with_file_name(format!(
".{}.runtime-owner.lock",
root.file_name()
.and_then(|s| s.to_str())
.ok_or_else(|| reject("invalid Runtime path"))?
));
regular_file(&lock_path)?;
let lock = io(OpenOptions::new().read(true).open(lock_path))?;
if !io(FileExt::try_lock_shared(&lock))? {
return Err(reject("source Runtime is not offline"));
}
let _lock = RuntimeStoreOwnerLock { file: lock };
check_tree(worker_dir, &mut 256)?;
let snapshot: RuntimeSnapshot = decode(&bytes(&root.join(RUNTIME_FILE))?)?;
snapshot
.validate(&root.join(RUNTIME_FILE))
.map_err(|_| reject("unsupported Runtime schema"))?;
if !snapshot.subject_create_recovery_fences.is_empty()
|| !snapshot.subject_releases.is_empty()
{
return Err(reject("source has lifecycle operations"));
}
let record: WorkerAggregateRecord = decode(&bytes(&worker_dir.join(WORKER_FILE))?)?;
let record = record
.validate(&worker_dir.join(WORKER_FILE))
.map_err(|_| reject("invalid Worker schema or identity"))?;
let guard: GuardState = decode(
&serde_json::to_vec(&record.restore_guard)
.map_err(|_| reject("invalid restore guard"))?,
)?;
if record.status != WorkerStatus::Stopped
|| guard.active_request_id.is_some()
|| guard.owners.values().any(|owner| owner.receipt.is_none())
{
return Err(reject("Worker is not settled and stopped"));
}
let PersistedWorkerExecutionState::Available(execution) = &record.execution_state else {
return Err(reject("pending or unavailable execution metadata"));
};
// A stopped aggregate may retain the empty historical binding marker.
// It is deliberately not copied to the installed Worker.
if execution.restore_intent != WorkerRestoreIntent::Explicit {
return Err(reject("Worker requires explicit restore"));
}
if !record.workdir_attachments.is_empty() || !record.logical_workdir_attachments.is_empty()
{
return Err(reject("attachments are unsupported"));
}
check_request(&execution.request, false)?;
let workspace_id = record
.workspace_id
.ok_or_else(|| reject("missing Workspace identity"))?;
let metadata: WorkerMetadata = decode(&bytes(&worker_dir.join(WORKER_METADATA_FILE))?)?;
crate::worker_backend::validate_fixture_restore_metadata(&metadata, &execution.request)
.map_err(|_| reject("invalid source restore Manifest or settings"))?;
if metadata.worker_name != runtime_worker_name(record.worker_id)
|| metadata.workspace_id.as_deref() != Some(&workspace_id)
|| !metadata.spawned_children.is_empty()
|| !metadata.reclaimed_children.is_empty()
|| !metadata.peers.is_empty()
|| metadata.subjektiv_session_attribution.is_some()
{
return Err(reject("unsupported or inconsistent Worker metadata"));
}
let active = metadata
.active
.ok_or_else(|| reject("missing active Session"))?;
let segment_id = active
.segment_id
.ok_or_else(|| reject("unmaterialized Session"))?;
let source_auth: SourceAuth = toml::from_str(&text(&root.join("auth.toml"))?)
.map_err(|_| reject("missing Runtime identity"))?;
let session = worker_dir.join("session");
let fixture = Self {
template_version: TEMPLATE_VERSION,
runtime_schema_version: SCHEMA_VERSION,
runtime_id: source_auth.identity.identity_id,
workspace_id,
worker_id: record.worker_id,
created_at_ms: record.created_at_ms,
session_id: active.session_id,
segment_id,
session_manifest: bytes(&session.join("session.json"))?,
segment_jsonl: bytes(&session.join("segments").join(format!("{segment_id}.jsonl")))?,
};
fixture.validate_shape()?;
fixture.validate_session(&session)?;
Ok(fixture)
}
/// Install after normal identity/trust initialization, before Runtime start.
/// `request` and `manifest` supply NEW backend/config/profile references and
/// freshly resolved policy, not a launch. The saved source Manifest is never
/// reused. `manifest.worker.name` must equal `worker_name()`.
/// Only `auth.toml` may already exist. The owner lock fences live Runtimes;
/// staged files are synced and published without overwriting any Worker.
/// An interrupted publication fails closed as an incomplete Runtime layout.
pub fn install(
&self,
options: &FsRuntimeStoreOptions,
request: CreateWorkerRequest,
manifest: &manifest::WorkerManifest,
) -> Result<(), RuntimeError> {
self.validate_shape()?;
check_request(&request, true)?;
if manifest.worker.name != self.worker_name() {
return Err(reject("resolved Manifest Worker name mismatch"));
}
let mut metadata = WorkerMetadata::new(
self.worker_name(),
Some(WorkerActiveSegmentRef {
session_id: self.session_id,
segment_id: Some(self.segment_id),
}),
)
.with_workspace_id(&self.workspace_id);
metadata.resolved_manifest_snapshot = Some(
manifest::write_persisted_worker_manifest_snapshot(manifest)
.map_err(|_| reject("invalid resolved Manifest"))?,
);
crate::worker_backend::validate_fixture_restore_metadata(&metadata, &request)
.map_err(|_| reject("resolved Manifest does not match fresh request settings"))?;
if options.runtime_id != self.runtime_id
|| request.worker_id != self.worker_id
|| request
.workspace_api
.as_ref()
.map(|api| api.workspace_id.as_str())
!= Some(&self.workspace_id)
{
return Err(reject("installation identity mismatch"));
}
no_symlink_components(&options.root)?;
check_tree(&options.root, &mut 256)?;
// Validate the lock location before the ordinary owner-lock opener.
let lock_path = options.root.with_file_name(format!(
".{}.runtime-owner.lock",
options
.root
.file_name()
.and_then(|s| s.to_str())
.ok_or_else(|| reject("invalid Runtime path"))?
));
if fs::symlink_metadata(&lock_path).is_ok() {
regular_file(&lock_path)?;
}
let lock = acquire_runtime_store_owner_lock(&options.root)?;
validate_fresh_runtime_store_layout(&options.root)
.map_err(|_| reject("destination is not fresh; refusing overwrite"))?;
let auth: InitializedAuth = toml::from_str(&text(&options.root.join("auth.toml"))?)
.map_err(|_| reject("initialize Runtime identity and trust before install"))?;
if auth.identity.identity_id != self.runtime_id
|| !auth.archived_legacy_issuers.is_empty()
|| auth.identity.signing_key().is_err()
{
return Err(reject("invalid destination Runtime identity"));
}
validate_workspace_issuer_trust_records(&auth.workspace_issuers)
.map_err(|_| reject("invalid destination Workspace trust"))?;
let api = request
.workspace_api
.as_ref()
.expect("validated Workspace binding");
if !auth.workspace_issuers.iter().any(|trust| {
trust.workspace_id == self.workspace_id
&& trust.backend_url == api.base_url
&& trust.state == WorkspaceIssuerTrustState::Active
}) {
return Err(reject("request does not match initialized Workspace trust"));
}
let stage = options
.root
.join(format!(".fixture-{}", uuid::Uuid::now_v7()));
io(fs::create_dir(&stage))?;
let store = FsRuntimeStore {
root: stage.clone(),
_owner_lock: Some(lock),
worker_write_fault: Arc::new(AtomicU8::new(AtomicWriteFault::None as u8)),
};
let result = (|| {
let worker = PersistedWorkerRecord {
restore_guard: crate::runtime::RestoreGuard::default(),
worker_ref: WorkerRef::new(self.worker_id),
worker_id: self.worker_id,
profile: request.profile.clone(),
display_name: request.display_name.clone(),
profile_source: request.profile_source.reference(),
config_bundle: request.config_bundle.clone(),
created_at_ms: self.created_at_ms,
status: WorkerStatus::Stopped,
last_finished_submission_request_id: None,
workspace_id: Some(self.workspace_id.clone()),
workdir_attachments: Vec::new(),
logical_workdir_attachments: Vec::new(),
execution_state: PersistedWorkerExecutionState::Available(
PersistedWorkerExecution {
request,
binding: None,
restore_intent: WorkerRestoreIntent::Explicit,
},
),
};
WorkerAggregateRecord::from_persisted(&worker)
.validate(&store.worker_dir(&self.worker_id).join(WORKER_FILE))
.map_err(|_| reject("invalid installation request"))?;
store.write_worker_record(&worker)?;
let worker_dir = store.worker_dir(&self.worker_id);
atomic_write_json(
&worker_dir.join(WORKER_METADATA_FILE),
&metadata,
"write fixture metadata",
)?;
let session = worker_dir.join("session");
io(fs::create_dir_all(session.join("segments")))?;
write_new(&session.join("session.json"), &self.session_manifest)?;
write_new(
&session
.join("segments")
.join(format!("{}.jsonl", self.segment_id)),
&self.segment_jsonl,
)?;
self.validate_session(&session)?;
store.write_runtime_snapshot(&PersistedRuntimeState {
subject_create_recovery_fences: BTreeMap::new(),
subject_releases: Vec::new(),
// Match normal store initialization: this is catalog availability,
// not a process or Worker execution start.
display_name: options.display_name.clone(),
status: RuntimeStatus::Running,
next_diagnostic_id: 1,
workers: BTreeMap::new(),
workspace_owners: BTreeMap::new(),
diagnostics: Vec::new(),
})?;
let loaded = store.load_runtime_state()?;
if loaded.workers.len() != 1 || !loaded.diagnostics.is_empty() {
return Err(reject("staged aggregate failed ordinary store roundtrip"));
}
sync_runtime_tree(&stage)?;
// No destination existed when checked under the owner lock. Hard-link
// the snapshot last with create-new semantics; never replace a file.
// Reserve the destination catalog with create-new semantics. Even
// an empty preexisting workers directory must never be replaced.
let destination_workers = options.root.join(WORKERS_DIR);
io(fs::create_dir(&destination_workers))?;
io(fs::rename(
store.worker_dir(&self.worker_id),
destination_workers.join(self.worker_id.to_string()),
))?;
sync_directory(&destination_workers, "publish stopped Worker aggregate")?;
io(fs::hard_link(
stage.join(RUNTIME_FILE),
options.root.join(RUNTIME_FILE),
))?;
sync_directory(&options.root, "publish stopped fixture")?;
Ok(())
})();
// Only remove our private staging directory, never existing destination data.
let cleanup = fs::remove_dir_all(&stage);
result?;
io(cleanup)?;
sync_directory(&options.root, "finish stopped fixture")
}
fn validate_shape(&self) -> Result<(), RuntimeError> {
if self.template_version != TEMPLATE_VERSION
|| self.runtime_schema_version != SCHEMA_VERSION
{
return Err(reject("unsupported template schema"));
}
if !portable_id(&self.runtime_id)
|| !portable_id(&self.workspace_id)
|| self.session_id.get_version_num() != 7
|| self.segment_id.get_version_num() != 7
|| self.session_manifest.len() > MAX_SESSION_BYTES
|| self.segment_jsonl.len() > MAX_SESSION_BYTES
{
return Err(reject("invalid portable identity or size"));
}
let manifest: SessionManifest = decode(&self.session_manifest)?;
if manifest.session_id != self.session_id
|| manifest.schema_version == 0
|| !self.segment_jsonl.ends_with(b"\n")
{
return Err(reject("Session identity or committed boundary mismatch"));
}
// Read-only JSON traversal, never mutation/reconstruction of Session bytes.
// Reject known secret/host markers even in otherwise valid typed records.
for line in self
.segment_jsonl
.split(|b| *b == b'\n')
.filter(|line| !line.is_empty())
{
let _: PortableJson = decode(line)?;
validate_preserved_record(line)?;
}
Ok(())
}
fn validate_history_entry(
&self,
entry: &session_store::LoggedHistoryEntry,
ids: &mut std::collections::BTreeSet<String>,
) -> Result<(), RuntimeError> {
if !portable_id(&entry.metadata.entry_id.0)
|| !ids.insert(entry.metadata.entry_id.0.clone())
{
return Err(reject("duplicate or invalid entry identity"));
}
if entry
.metadata
.input_metadata
.as_ref()
.is_some_and(|metadata| {
metadata.feature_payload().is_some() || metadata.public_item().is_some()
})
{
// These are opaque owner-specific payloads, not part of the narrow
// artificial conversation contract. Never preserve unchecked JSON.
return Err(reject("Feature history payloads are unsupported"));
}
if !matches!(
entry.item,
session_store::LoggedItem::Message { .. }
| session_store::LoggedItem::AssistantResponseBoundary { .. }
) {
return Err(reject("only text conversation records are supported"));
}
if let session_store::LoggedSessionHistoryOrigin::ModelOutput { worker } =
&entry.metadata.origin
{
// Session producer IDs identify the original execution, not the
// Workspace catalog Worker. Preserve them; do not rebind/rewrite them.
if !portable_id(&worker.worker_id)
|| worker
.workspace_id
.as_ref()
.is_some_and(|id| id != &self.workspace_id)
|| worker
.runtime_id
.as_ref()
.is_some_and(|id| id != &self.runtime_id)
{
return Err(reject("Session origin identity mismatch"));
}
}
Ok(())
}
fn validate_session(&self, root: &Path) -> Result<(), RuntimeError> {
for entry in io(fs::read_dir(root))? {
let entry = io(entry)?;
match entry.file_name().to_str() {
Some("session.json" | "segments") => {}
Some("artifacts") => check_empty_artifacts(&entry.path())?,
_ => return Err(reject("unsupported Session files")),
}
}
let entries = io(fs::read_dir(root.join("segments")))?.collect::<Result<Vec<_>, _>>();
let entries = io(entries)?;
if entries.len() != 1
|| entries[0].file_name() != format!("{}.jsonl", self.segment_id).as_str()
{
return Err(reject("only the fixed single-Segment fixture is supported"));
}
let store = WorkerSessionStore::open_read_only(root)
.map_err(|_| reject("unsupported Session schema"))?;
let logs = store
.read_all_read_only_bounded(self.session_id, self.segment_id, MAX_SESSION_BYTES as u64)
.map_err(|_| reject("invalid canonical Session records"))?
.0;
if !matches!(logs.first(), Some(session_store::LogEntry::AnnotatedSegmentStart {
session_id, system_prompt: Some(_), forked_from: None, compacted_from: None, ..
}) if *session_id == self.session_id)
{
return Err(reject(
"missing system prompt, invalid Segment seed, or unsupported lineage",
));
}
let mut ids = std::collections::BTreeSet::new();
for (index, log) in logs.iter().enumerate() {
use session_store::LogEntry;
match log {
LogEntry::AnnotatedSegmentStart { history, .. }
if index == 0 && history.is_empty() => {}
LogEntry::AnnotatedUserInput {
history,
extensions,
..
} => {
for extension in extensions {
validate_settled_activation(extension)?;
}
for entry in history {
self.validate_history_entry(entry, &mut ids)?;
}
}
LogEntry::AnnotatedAssistantItem { entry, .. } => {
self.validate_history_entry(entry, &mut ids)?
}
LogEntry::RunCompleted {
entry_id: Some(id), ..
} => {
if !portable_id(&id.0) || !ids.insert(id.0.clone()) {
return Err(reject("duplicate or invalid entry identity"));
}
}
LogEntry::Invoke { .. } | LogEntry::LlmUsage { .. } | LogEntry::TurnEnd { .. } => {}
_ => {
return Err(reject(
"unsupported conversation record or pending operation",
));
}
}
}
let state = session_store::collect_state(&logs);
if state.last_run_interrupted || state.active_run_turn_count.is_some() {
return Err(reject("Session has pending execution"));
}
for segments in &state.user_segments {
if segments
.iter()
.any(|segment| !matches!(segment, protocol::Segment::Text { .. }))
{
return Err(reject("Session attachments are unsupported"));
}
}
Ok(())
}
}
// Compare *fields*, not re-encoded bytes or scalar values (e.g. f32 JSON
// normalization). Serde's permissive/untagged/flattened compatibility readers
// can drop unknown data without surfacing it to an ignored-field callback. A
// retained field must survive the owner's typed projection at every depth.
// This intentionally accepts only canonical writer shapes: even an explicit
// empty optional field omitted by that writer is rejected, not repaired.
// The projection is validation-only; installation always writes original bytes.
fn validate_preserved_record(bytes: &[u8]) -> Result<(), RuntimeError> {
let original: serde_json::Value = decode(bytes)?;
let record: session_store::LogEntry = decode(bytes)?;
let projection =
serde_json::to_value(&record).map_err(|_| reject("invalid typed Session record"))?;
require_preserved_fields(&original, &projection)
}
fn require_preserved_fields(
original: &serde_json::Value,
projection: &serde_json::Value,
) -> Result<(), RuntimeError> {
use serde_json::Value;
match (original, projection) {
(Value::Object(original), Value::Object(projection)) => {
for (key, value) in original {
let projected = projection
.get(key)
.ok_or_else(|| reject("unknown or noncanonical Session field"))?;
require_preserved_fields(value, projected)?;
}
}
(Value::Array(original), Value::Array(projection))
if original.len() == projection.len() =>
{
for (value, projected) in original.iter().zip(projection) {
require_preserved_fields(value, projected)?;
}
}
(Value::Null, Value::Null)
| (Value::Bool(_), Value::Bool(_))
| (Value::Number(_), Value::Number(_))
| (Value::String(_), Value::String(_)) => {}
_ => return Err(reject("noncanonical Session field shape")),
}
Ok(())
}
// Exact receipt wire fields owned by worker::PendingActivationState. Keep these
// fixture-only mirrors strict; do not change the production compatibility reader.
#[derive(Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct SubmissionReceipt {
submission_request_id: String,
source_namespace: String,
submission_id: String,
payload_digest: String,
disposition: protocol::SubmissionDisposition,
}
#[derive(Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct NotificationReceipt {
notification_request_id: String,
source_namespace: String,
payload_digest: String,
}
// The approved conversation has committed submission checkpoints in its original
// Session records. Keep their receipt bytes, but reject any resumable queue or
// Feature invocation history. This projection confers no execution authority.
fn validate_settled_activation(
extension: &session_store::SessionExtension,
) -> Result<(), RuntimeError> {
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Checkpoint {
#[serde(rename = "next_activation_sequence")]
_next_activation_sequence: u64,
activating: Option<serde::de::IgnoredAny>,
activating_notification: Vec<serde::de::IgnoredAny>,
pending: Vec<serde::de::IgnoredAny>,
pending_notifications: Vec<serde::de::IgnoredAny>,
#[serde(rename = "receipts")]
_receipts: Vec<SubmissionReceipt>,
#[serde(rename = "notification_receipts")]
_notification_receipts: Vec<NotificationReceipt>,
invocation_receipts: Vec<serde::de::IgnoredAny>,
invocation_recovery_blocked: bool,
}
if extension.domain != "worker.pending_activations.v1" {
return Err(reject("unsupported Session extension"));
}
let checkpoint: Checkpoint = serde_json::from_value(extension.payload.clone())
.map_err(|_| reject("invalid Session activation checkpoint"))?;
if checkpoint.activating.is_some()
|| !checkpoint.activating_notification.is_empty()
|| !checkpoint.pending.is_empty()
|| !checkpoint.pending_notifications.is_empty()
|| !checkpoint.invocation_receipts.is_empty()
|| checkpoint.invocation_recovery_blocked
{
return Err(reject(
"Session has pending activation or invocation history",
));
}
Ok(())
}
fn check_request(request: &CreateWorkerRequest, install: bool) -> Result<(), RuntimeError> {
if !request.workdir_attachment_requests.is_empty()
|| !request.workdir_attachments.is_empty()
|| request.backend_job.is_some()
|| request.subjektiv_attached
|| request.worker_observation_enabled
|| !request.worker_observation_grants.is_empty()
|| (install && request.initial_input.is_some())
{
return Err(reject(
"unsupported capabilities, attachments, or initial input",
));
}
if install
&& (request.create_fingerprint.trim().is_empty()
|| request.config_bundle.is_none()
|| !matches!(
request.profile_source,
ProfileSourceArchiveSource::WorkspaceConfig { .. }
))
{
return Err(reject(
"fresh config/profile references and create fingerprint are required",
));
}
Ok(())
}
fn check_empty_artifacts(root: &Path) -> Result<(), RuntimeError> {
for entry in io(fs::read_dir(root))? {
let entry = io(entry)?;
let metadata = io(entry.metadata())?;
if metadata.is_dir() {
check_empty_artifacts(&entry.path())?;
} else if entry.file_name() != ".aggregate.lock" || metadata.len() != 0 {
return Err(reject("Session artifacts are unsupported"));
}
}
Ok(())
}
fn write_new(path: &Path, bytes: &[u8]) -> Result<(), RuntimeError> {
let mut file = io(OpenOptions::new().write(true).create_new(true).open(path))?;
io(file.write_all(bytes))?;
io(file.sync_all())
}
// A streaming validation visitor, not a generic document available to callers.
struct PortableJson;
impl<'de> Deserialize<'de> for PortableJson {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
struct Visitor;
impl<'de> serde::de::Visitor<'de> for Visitor {
type Value = PortableJson;
fn expecting(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("portable fixture JSON")
}
fn visit_bool<E: serde::de::Error>(self, _: bool) -> Result<PortableJson, E> {
Ok(PortableJson)
}
fn visit_i64<E: serde::de::Error>(self, _: i64) -> Result<PortableJson, E> {
Ok(PortableJson)
}
fn visit_u64<E: serde::de::Error>(self, _: u64) -> Result<PortableJson, E> {
Ok(PortableJson)
}
fn visit_f64<E: serde::de::Error>(self, _: f64) -> Result<PortableJson, E> {
Ok(PortableJson)
}
fn visit_unit<E: serde::de::Error>(self) -> Result<PortableJson, E> {
Ok(PortableJson)
}
fn visit_str<E: serde::de::Error>(self, value: &str) -> Result<PortableJson, E> {
let lower = value.to_ascii_lowercase();
if [
"://",
"private key",
"bearer ",
"sk-",
"api_key",
"access_token",
"refresh_token",
"private_key",
"authorization",
]
.iter()
.any(|marker| lower.contains(marker))
|| value
.split_whitespace()
.any(|word| word.starts_with('/') || word.contains(":\\"))
{
return Err(E::custom("nonportable or sensitive fixture material"));
}
Ok(PortableJson)
}
fn visit_seq<A: serde::de::SeqAccess<'de>>(
self,
mut seq: A,
) -> Result<PortableJson, A::Error> {
while seq.next_element::<PortableJson>()?.is_some() {}
Ok(PortableJson)
}
fn visit_map<A: serde::de::MapAccess<'de>>(
self,
mut map: A,
) -> Result<PortableJson, A::Error> {
let mut keys = std::collections::BTreeSet::new();
while let Some(key) = map.next_key::<String>()? {
// A duplicate key can hide discarded data even inside an
// otherwise opaque extension payload. Never retain it.
if !keys.insert(key.clone()) {
return Err(serde::de::Error::custom("duplicate fixture JSON field"));
}
serde::de::Visitor::visit_str::<A::Error>(Visitor, &key)?;
map.next_value::<PortableJson>()?;
}
Ok(PortableJson)
}
}
deserializer.deserialize_any(Visitor)
}
}
@@ -0,0 +1,885 @@
use super::*;
use crate::auth::RuntimeIdentityMaterial;
use crate::catalog::{ProfileSelector, WorkspaceApiRef};
use crate::profile_archive::ProfileSourceGraphSummary;
use session_store::{LogEntry, Store};
use sha2::{Digest, Sha256};
const RUNTIME: &str = "fixture-runtime";
const WORKSPACE: &str = "fixture-workspace";
const NEW_URL: &str = "http://127.0.0.1:34567";
#[derive(Serialize)]
#[serde(untagged)]
enum ReceiptFixture<T> {
Valid(T),
Malformed(u8),
}
#[derive(Default, Serialize)]
struct CheckpointFixture {
next_activation_sequence: u64,
activating: Option<u8>,
activating_notification: Vec<u8>,
pending: Vec<u8>,
pending_notifications: Vec<u8>,
receipts: Vec<ReceiptFixture<SubmissionReceipt>>,
notification_receipts: Vec<ReceiptFixture<NotificationReceipt>>,
invocation_receipts: Vec<u8>,
invocation_recovery_blocked: bool,
}
fn settled_checkpoint() -> CheckpointFixture {
CheckpointFixture {
next_activation_sequence: 2,
receipts: vec![ReceiptFixture::Valid(SubmissionReceipt {
submission_request_id: "completed-submission-request".into(),
source_namespace: "fixture-source".into(),
submission_id: "completed-submission".into(),
payload_digest: "fixture-payload-digest".into(),
disposition: protocol::SubmissionDisposition::Started,
})],
notification_receipts: vec![ReceiptFixture::Valid(NotificationReceipt {
notification_request_id: "completed-notification-request".into(),
source_namespace: "fixture-source".into(),
payload_digest: "fixture-notification-digest".into(),
})],
..Default::default()
}
}
fn checkpoint_extension(checkpoint: &CheckpointFixture) -> session_store::SessionExtension {
session_store::SessionExtension::new(
"worker.pending_activations.v1",
serde_json::to_value(checkpoint).unwrap(),
)
}
fn worker_id() -> WorkerId {
"01a1657d-b762-7791-87ae-a49f28cadbce".parse().unwrap()
}
fn resolved() -> manifest::WorkerManifest {
let mut config = manifest::WorkerManifestConfig::resolution_defaults();
config.worker.name = Some(runtime_worker_name(worker_id()));
manifest::WorkerManifest::try_from(config).unwrap()
}
fn request() -> CreateWorkerRequest {
CreateWorkerRequest {
worker_id: worker_id(),
create_fingerprint: "new-create-fingerprint".into(),
profile: ProfileSelector::Builtin("default".into()),
display_name: Some("fixture".into()),
profile_source: ProfileSourceArchiveSource::WorkspaceConfig {
archive: ProfileSourceArchiveRef {
id: "new-profile".into(),
digest: "new-profile-digest".into(),
size_bytes: 1,
source_graph: ProfileSourceGraphSummary {
source_count: 1,
total_source_bytes: 1,
entrypoints: BTreeMap::from([("builtin:default".into(), "default.dcl".into())]),
import_count: 0,
},
},
},
config_bundle: Some(ConfigBundleRef {
id: "new-config".into(),
digest: "new-config-digest".into(),
}),
workspace_api: Some(WorkspaceApiRef {
workspace_id: WORKSPACE.into(),
base_url: NEW_URL.into(),
}),
initial_input: None,
workdir_attachment_requests: Vec::new(),
workdir_attachments: Vec::new(),
worker_observation_enabled: false,
worker_observation_grants: Vec::new(),
memory_settings: Some(manifest::WorkspaceMemorySettingsSnapshot {
workspace_id: WORKSPACE.into(),
language: "en".into(),
}),
subjektiv_attached: false,
backend_job: None,
}
}
fn initialize(root: &Path, runtime_id: &str, workspace_id: &str, url: &str) {
#[derive(Serialize)]
struct Auth {
identity: RuntimeIdentityMaterial,
workspace_issuers: Vec<WorkspaceIssuerTrustRecord>,
}
let issuer = RuntimeIdentityMaterial::generate("fixture-issuer").unwrap();
let digest = Sha256::digest(crate::auth::decode_public_key(&issuer.public_key).unwrap());
let fingerprint = format!(
"sha256:{}",
digest
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>()
);
let trust = WorkspaceIssuerTrustRecord::from_bundle(
server_api::WorkspacePublicIdentityBundle {
workspace_id: workspace_id.into(),
backend_url: url.into(),
key_id: "fixture-issuer".into(),
algorithm: "ed25519".into(),
public_key: issuer.public_key,
public_key_fingerprint: fingerprint,
},
"fixture-trust".into(),
100,
)
.unwrap();
fs::create_dir_all(root).unwrap();
fs::write(
root.join("auth.toml"),
toml::to_string(&Auth {
identity: RuntimeIdentityMaterial::generate(runtime_id).unwrap(),
workspace_issuers: vec![trust],
})
.unwrap(),
)
.unwrap();
}
fn record(request: CreateWorkerRequest) -> PersistedWorkerRecord {
PersistedWorkerRecord {
restore_guard: crate::runtime::RestoreGuard::default(),
worker_ref: WorkerRef::new(request.worker_id),
worker_id: request.worker_id,
profile: request.profile.clone(),
display_name: request.display_name.clone(),
profile_source: request.profile_source.reference(),
config_bundle: request.config_bundle.clone(),
created_at_ms: Some(123456),
status: WorkerStatus::Stopped,
last_finished_submission_request_id: Some("old-submission-operation".into()),
workspace_id: Some(WORKSPACE.into()),
workdir_attachments: vec![],
logical_workdir_attachments: vec![],
execution_state: PersistedWorkerExecutionState::Available(PersistedWorkerExecution {
request,
binding: Some(PersistedWorkerExecutionBinding {}),
restore_intent: WorkerRestoreIntent::Explicit,
}),
}
}
fn source() -> (tempfile::TempDir, PathBuf, StoppedWorkerTemplate) {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
initialize(&root, RUNTIME, WORKSPACE, "http://127.0.0.1:23456");
drop(
crate::Runtime::with_fs_store(FsRuntimeStoreOptions::new(&root).with_runtime_id(RUNTIME))
.unwrap(),
);
let opened = FsRuntimeStore::open_or_create(root.clone(), RUNTIME).unwrap();
let mut old = request();
old.workspace_api.as_mut().unwrap().base_url = "http://127.0.0.1:23456".into();
old.config_bundle.as_mut().unwrap().id = "old-config".into();
opened.store.write_worker_record(&record(old)).unwrap();
let worker = opened.store.worker_dir(&worker_id());
let session_id = "01a1657d-b800-7777-8888-111111111111".parse().unwrap();
let segment_id = "01a1657d-b818-77c1-bd73-176172a4bd82".parse().unwrap();
let mut metadata = WorkerMetadata::new(
runtime_worker_name(worker_id()),
Some(WorkerActiveSegmentRef {
session_id,
segment_id: Some(segment_id),
}),
)
.with_workspace_id(WORKSPACE)
.with_workspace_root(PathBuf::from("/old/host/path"));
metadata.resolved_manifest_snapshot =
Some(manifest::write_persisted_worker_manifest_snapshot(&resolved()).unwrap());
atomic_write_json(
&worker.join(WORKER_METADATA_FILE),
&metadata,
"test metadata",
)
.unwrap();
let sessions = WorkerSessionStore::new(worker.join("session")).unwrap();
let history = session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::Message {
role: session_store::LoggedRole::Assistant,
content: vec![session_store::LoggedContentPart::Text {
text: "artificial fixture reply".into(),
}],
},
metadata: session_store::LoggedSessionHistoryMetadata {
entry_id: session_store::LoggedSessionHistoryEntryId("fixed-entry-id".into()),
origin: session_store::LoggedSessionHistoryOrigin::ModelOutput {
worker: session_store::LoggedWorkerSubject {
workspace_id: Some(WORKSPACE.into()),
runtime_id: Some(RUNTIME.into()),
worker_id: worker_id().to_string(),
},
},
derivation: None,
input_metadata: None,
},
};
sessions
.create_segment(
session_id,
segment_id,
&[
LogEntry::AnnotatedSegmentStart {
ts: 123,
session_id,
system_prompt: Some("Harmless artificial fixture instructions.".into()),
config: Default::default(),
history: vec![],
forked_from: None,
compacted_from: None,
},
LogEntry::Invoke {
ts: 124,
trigger: protocol::InvokeKind::UserSend,
},
LogEntry::AnnotatedUserInput {
ts: 124,
segments: vec![protocol::Segment::text(
"Harmless artificial fixture input.",
)],
extensions: vec![checkpoint_extension(&settled_checkpoint())],
history: vec![session_store::LoggedHistoryEntry {
item: session_store::LoggedItem::Message {
role: session_store::LoggedRole::User,
content: vec![session_store::LoggedContentPart::Text {
text: "Harmless artificial fixture input.".into(),
}],
},
metadata: session_store::LoggedSessionHistoryMetadata {
entry_id: session_store::LoggedSessionHistoryEntryId(
"fixed-user-id".into(),
),
origin: session_store::LoggedSessionHistoryOrigin::BackendInstruction {
operation_id: None,
},
derivation: None,
input_metadata: None,
},
}],
},
LogEntry::AnnotatedAssistantItem {
ts: 125,
entry: history,
},
LogEntry::LlmUsage {
ts: 125,
history_len: 2,
input_total_tokens: 1,
cache_read_tokens: 0,
cache_write_tokens: 0,
output_tokens: 1,
},
LogEntry::TurnEnd {
ts: 126,
turn_count: 1,
},
LogEntry::RunCompleted {
entry_id: Some(session_store::LoggedSessionHistoryEntryId(
"fixed-terminal-id".into(),
)),
ts: 126,
interrupted: false,
result: agen::EngineResult::Finished,
active_run_turn_count: None,
},
],
)
.unwrap();
fs::create_dir_all(worker.join("runs/old-run")).unwrap();
fs::write(worker.join("runs/old-run/private.log"), "not exported").unwrap();
drop(opened);
let template = StoppedWorkerTemplate::extract(&worker).unwrap();
(temp, worker, template)
}
fn destination(template: &StoppedWorkerTemplate) -> (tempfile::TempDir, FsRuntimeStoreOptions) {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("runtime");
initialize(
&root,
template.runtime_id(),
template.workspace_id(),
NEW_URL,
);
(
temp,
FsRuntimeStoreOptions::new(root).with_runtime_id(template.runtime_id()),
)
}
fn assert_fresh(options: &FsRuntimeStoreOptions) {
assert_eq!(fs::read_dir(&options.root).unwrap().count(), 1);
assert!(options.root.join("auth.toml").is_file());
}
#[test]
fn extraction_install_roundtrip_preserves_bytes_and_is_inactive() {
let (_source, worker, template) = source();
let original_worker = fs::read(worker.join(WORKER_FILE)).unwrap();
let original_metadata = fs::read(worker.join(WORKER_METADATA_FILE)).unwrap();
let exported = serde_json::to_vec(&template).unwrap();
let exported_text = String::from_utf8(exported.clone()).unwrap();
for forbidden in [
"old-config",
"old-submission-operation",
"restore_guard",
"private_key",
"http://",
"/old/host",
"private.log",
"resolved_manifest_snapshot",
] {
assert!(!exported_text.contains(forbidden));
}
let template: StoppedWorkerTemplate = serde_json::from_slice(&exported).unwrap();
let (_target, options) = destination(&template);
let auth_before = fs::read(options.root.join("auth.toml")).unwrap();
template.install(&options, request(), &resolved()).unwrap();
assert_eq!(
fs::read(options.root.join("auth.toml")).unwrap(),
auth_before
);
assert_eq!(fs::read(worker.join(WORKER_FILE)).unwrap(), original_worker);
assert_eq!(
fs::read(worker.join(WORKER_METADATA_FILE)).unwrap(),
original_metadata
);
let installed = options
.root
.join(WORKERS_DIR)
.join(template.worker_id().to_string());
assert_eq!(
fs::read(installed.join("session/session.json")).unwrap(),
template.session_manifest
);
assert_eq!(
fs::read(installed.join(format!("session/segments/{}.jsonl", template.segment_id())))
.unwrap(),
template.segment_jsonl
);
assert!(!installed.join("runs").exists());
let metadata: WorkerMetadata =
decode(&fs::read(installed.join(WORKER_METADATA_FILE)).unwrap()).unwrap();
assert_eq!(metadata.active.unwrap().session_id, template.session_id());
assert!(metadata.workspace_root.is_none() && metadata.resolved_manifest_snapshot.is_some());
let opened = FsRuntimeStore::open_or_create(options.root.clone(), RUNTIME).unwrap();
let state = opened.state.unwrap();
assert!(state.diagnostics.is_empty());
let saved = state.workers.get(&worker_id()).unwrap();
assert_eq!(saved.status, WorkerStatus::Stopped);
assert!(saved.last_finished_submission_request_id.is_none());
let PersistedWorkerExecutionState::Available(execution) = &saved.execution_state else {
panic!("not available")
};
assert_eq!(execution.restore_intent, WorkerRestoreIntent::Explicit);
assert!(execution.binding.is_none());
assert_eq!(execution.request, request());
// Fresh restore guard, no operation owner/receipt history.
assert_ne!(
serde_json::to_vec(&saved.restore_guard).unwrap(),
serde_json::to_vec(
&decode::<WorkerAggregateRecord>(&original_worker)
.unwrap()
.restore_guard
)
.unwrap()
);
let guard: GuardState = decode(&serde_json::to_vec(&saved.restore_guard).unwrap()).unwrap();
assert!(guard.active_request_id.is_none() && guard.owners.is_empty());
drop(opened.store);
struct NoExecution;
impl crate::execution::WorkerExecutionBackend for NoExecution {
fn backend_id(&self) -> &str {
"fixture-no-execution"
}
fn spawn_worker(
&self,
_: crate::execution::WorkerExecutionSpawnRequest,
) -> crate::execution::WorkerExecutionSpawnResult {
panic!("fixture executed")
}
fn dispatch_input(
&self,
_: &WorkerRef,
_: crate::interaction::WorkerInput,
) -> crate::execution::WorkerExecutionResult {
panic!("fixture submitted")
}
fn restore_worker(
&self,
_: crate::execution::WorkerExecutionRestoreRequest,
) -> crate::execution::WorkerExecutionSpawnResult {
panic!("fixture auto-restored")
}
}
let runtime =
crate::Runtime::with_fs_store_and_execution_backend(options.clone(), Arc::new(NoExecution))
.unwrap();
assert_eq!(runtime.list_workers().unwrap().len(), 1);
assert_eq!(runtime.summary().unwrap().active_worker_count, 0);
assert_eq!(
runtime
.worker_detail(&WorkerRef::new(worker_id()))
.unwrap()
.status,
WorkerStatus::Stopped
);
drop(runtime);
let runtime = crate::Runtime::with_fs_store(options).unwrap();
assert_eq!(
runtime.list_workers().unwrap()[0].status,
WorkerStatus::Stopped
);
}
#[test]
fn install_rejects_schema_identity_input_and_overwrite() {
let (_source, _, template) = source();
for change in [
"template",
"runtime-schema",
"session-schema",
"session-id",
"segment-id",
"runtime-id",
"workspace-id",
"worker-id",
"origin-id",
"duplicate-id",
"truncated",
] {
let (_target, options) = destination(&template);
let mut invalid = template.clone();
match change {
"template" => invalid.template_version += 1,
"runtime-schema" => invalid.runtime_schema_version += 1,
"session-schema" => {
invalid.session_manifest = format!(
"{{\"schema_version\":999,\"session_id\":\"{}\"}}",
invalid.session_id
)
.into_bytes()
}
"session-id" => invalid.session_id = uuid::Uuid::now_v7(),
"segment-id" => invalid.segment_id = uuid::Uuid::nil(), // no external attestation: valid new Segment ID is otherwise portable
"runtime-id" => invalid.runtime_id = "other-runtime".into(),
"workspace-id" => invalid.workspace_id = "other-workspace".into(),
"worker-id" => invalid.worker_id = WorkerId::now_v7(),
"origin-id" => {
invalid.segment_jsonl = String::from_utf8(invalid.segment_jsonl)
.unwrap()
.replace(RUNTIME, "other-runtime")
.into_bytes()
}
"duplicate-id" => {
let last = invalid
.segment_jsonl
.split_inclusive(|b| *b == b'\n')
.last()
.unwrap()
.to_vec();
invalid.segment_jsonl.extend(last);
}
"truncated" => {
invalid.segment_jsonl.pop();
}
_ => unreachable!(),
}
assert!(
invalid.install(&options, request(), &resolved()).is_err(),
"accepted {change}"
);
assert_fresh(&options);
}
let (_target, options) = destination(&template);
let mut launch = request();
launch.initial_input = Some(crate::interaction::WorkerInput::user("do not execute"));
assert!(template.install(&options, launch, &resolved()).is_err());
assert_fresh(&options);
template.install(&options, request(), &resolved()).unwrap();
let worker_path = options
.root
.join(WORKERS_DIR)
.join(worker_id().to_string())
.join(WORKER_FILE);
let before = fs::read(&worker_path).unwrap();
assert!(template.install(&options, request(), &resolved()).is_err());
assert_eq!(fs::read(worker_path).unwrap(), before);
}
#[test]
fn rejects_uninitialized_untrusted_live_and_partial_destination() {
let (_source, _, template) = source();
let (_target, options) = destination(&template);
let held = acquire_runtime_store_owner_lock(&options.root).unwrap();
assert!(template.install(&options, request(), &resolved()).is_err());
assert_fresh(&options);
drop(held);
initialize(&options.root, "wrong-runtime", WORKSPACE, NEW_URL);
assert!(template.install(&options, request(), &resolved()).is_err());
initialize(&options.root, RUNTIME, WORKSPACE, "http://127.0.0.1:23456");
assert!(template.install(&options, request(), &resolved()).is_err());
fs::remove_file(options.root.join("auth.toml")).unwrap();
assert!(template.install(&options, request(), &resolved()).is_err());
initialize(&options.root, RUNTIME, WORKSPACE, NEW_URL);
fs::create_dir(options.root.join(WORKERS_DIR)).unwrap();
assert!(template.install(&options, request(), &resolved()).is_err());
}
#[test]
fn extraction_rejects_nonstopped_pending_schema_and_metadata_mismatch() {
for change in [
"running",
"pending",
"guard-pending",
"attachment",
"schema",
"worker-id",
"metadata-workspace",
"metadata-session",
"artifact",
"secret",
"url",
"path",
] {
let (_source, worker, _) = source();
let mut aggregate: WorkerAggregateRecord =
decode(&fs::read(worker.join(WORKER_FILE)).unwrap()).unwrap();
match change {
"running" => aggregate.status = WorkerStatus::Running,
"pending" => {
aggregate.execution_state = WorkerExecutionStateRecord::ReconciliationRequired(
PersistedWorkerLifecycleOperation::Restore(PersistedWorkerRestoreOperation {
operation_id: WorkerLifecycleOperationId::new(),
mode: PersistedWorkerRestoreMode::Explicit,
request: request(),
binding: None,
last_settled_status: WorkerStatus::Stopped,
}),
)
}
"guard-pending" => aggregate.restore_guard = serde_json::from_str(r#"{"observation_token":"old-token","active_request_id":"pending-restore","owners":{}}"#).unwrap(),
"attachment" => aggregate.logical_workdir_attachments.push(LogicalWorkdirAttachment {
alias: workdir::WorkdirAttachmentAlias::new("source").unwrap(),
working_directory_id: "old-workdir".into(),
capabilities: workdir::WorkdirSessionCapabilities::READ_ONLY,
}),
"schema" => aggregate.schema_version += 1,
"worker-id" => aggregate.worker_id = WorkerId::now_v7(),
"metadata-workspace" | "metadata-session" => {
let mut metadata: WorkerMetadata =
decode(&fs::read(worker.join(WORKER_METADATA_FILE)).unwrap()).unwrap();
if change == "metadata-workspace" {
metadata.workspace_id = Some("wrong-workspace".into());
} else {
metadata.active.as_mut().unwrap().session_id = uuid::Uuid::now_v7();
}
atomic_write_json(
&worker.join(WORKER_METADATA_FILE),
&metadata,
"test mismatch",
)
.unwrap();
}
"artifact" => {
fs::create_dir_all(worker.join("session/artifacts/paste")).unwrap();
fs::write(
worker.join("session/artifacts/paste/payload"),
"unsupported",
)
.unwrap();
}
"secret" | "url" | "path" => {
let segment = fs::read_dir(worker.join("session/segments"))
.unwrap()
.next()
.unwrap()
.unwrap()
.path();
let replacement = match change {
"secret" => "Bearer secret",
"url" => "https://old-host.invalid",
_ => "/old/host/path",
};
let text = fs::read_to_string(&segment)
.unwrap()
.replace("artificial fixture reply", replacement);
fs::write(segment, text).unwrap();
}
_ => unreachable!(),
}
atomic_write_json(&worker.join(WORKER_FILE), &aggregate, "test aggregate").unwrap();
assert!(
StoppedWorkerTemplate::extract(&worker).is_err(),
"accepted {change}"
);
}
let (_source, worker, _) = source();
let held =
acquire_runtime_store_owner_lock(worker.parent().unwrap().parent().unwrap()).unwrap();
assert!(StoppedWorkerTemplate::extract(&worker).is_err());
drop(held);
}
#[test]
fn fresh_manifest_and_memory_settings_are_required() {
let (_source, _, template) = source();
let (_target, options) = destination(&template);
let mut manifest = resolved();
manifest.worker.name = "wrong-worker".into();
assert!(template.install(&options, request(), &manifest).is_err());
let mut binding = request();
binding.memory_settings = None;
assert!(template.install(&options, binding, &resolved()).is_err());
let mut binding = request();
binding.memory_settings.as_mut().unwrap().workspace_id = "other-workspace".into();
assert!(template.install(&options, binding, &resolved()).is_err());
assert_fresh(&options);
}
#[test]
fn settled_checkpoints_preserve_receipts_but_refuse_pending_work() {
let mut checkpoint = settled_checkpoint();
assert!(validate_settled_activation(&checkpoint_extension(&checkpoint)).is_ok());
checkpoint.pending.push(1);
assert!(validate_settled_activation(&checkpoint_extension(&checkpoint)).is_err());
checkpoint.pending.clear();
checkpoint.activating = Some(1);
assert!(validate_settled_activation(&checkpoint_extension(&checkpoint)).is_err());
checkpoint.activating = None;
checkpoint.invocation_receipts.push(1);
assert!(validate_settled_activation(&checkpoint_extension(&checkpoint)).is_err());
checkpoint.invocation_receipts.clear();
checkpoint.invocation_recovery_blocked = true;
assert!(validate_settled_activation(&checkpoint_extension(&checkpoint)).is_err());
}
// Deliberately corrupt only test-owned synthetic source files, never the lab.
fn assert_rejected_session_bytes(worker: &Path, template: &StoppedWorkerTemplate, bytes: Vec<u8>) {
let path = worker.join(format!("session/segments/{}.jsonl", template.segment_id()));
fs::write(path, &bytes).unwrap();
let extraction_error = StoppedWorkerTemplate::extract(worker)
.err()
.expect("invalid source accepted");
assert!(
!extraction_error
.to_string()
.contains("synthetic-discarded-material")
);
let (_target, options) = destination(template);
let mut invalid = template.clone();
invalid.segment_jsonl = bytes;
let installation_error = invalid
.install(&options, request(), &resolved())
.unwrap_err();
assert!(
!installation_error
.to_string()
.contains("synthetic-discarded-material")
);
assert_fresh(&options);
}
fn synthetic_logs(template: &StoppedWorkerTemplate) -> Vec<LogEntry> {
template
.segment_jsonl
.split(|byte| *byte == b'\n')
.filter(|line| !line.is_empty())
.map(|line| serde_json::from_slice(line).unwrap())
.collect()
}
fn encode_synthetic_logs(logs: &[LogEntry]) -> Vec<u8> {
let mut bytes = Vec::new();
for log in logs {
serde_json::to_writer(&mut bytes, log).unwrap();
bytes.push(b'\n');
}
bytes
}
#[tokio::test]
async fn missing_system_prompt_matches_worker_restore_rejection() {
let (_source, worker_dir, template) = source();
let mut logs = synthetic_logs(&template);
assert!(session_store::collect_state(&logs).system_prompt.is_some());
let LogEntry::AnnotatedSegmentStart { system_prompt, .. } = &mut logs[0] else {
panic!("missing seed")
};
*system_prompt = None;
assert_rejected_session_bytes(&worker_dir, &template, encode_synthetic_logs(&logs));
// The owning Worker constructor rejects this before model-client creation,
// allocation, or execution. Do not start a controller or submit any input.
let store = session_store::CombinedStore::new(
WorkerSessionStore::open_read_only(worker_dir.join("session")).unwrap(),
session_store::WorkerAggregateStore::open_read_only(&worker_dir, template.worker_name())
.unwrap(),
);
let restored = worker::Worker::restore_from_manifest(
template.session_id(),
template.segment_id(),
resolved(),
store,
worker::PromptCatalogSource::builtins_only(),
)
.await;
assert!(matches!(
restored,
Err(worker::WorkerError::SegmentSystemPromptMissing { .. })
));
}
#[test]
fn malformed_submission_and_notification_receipts_fail_extraction_and_install() {
let (_source, worker, template) = source();
for notification in [false, true] {
let mut checkpoint = settled_checkpoint();
if notification {
checkpoint.notification_receipts =
vec![ReceiptFixture::Malformed(1), ReceiptFixture::Malformed(2)];
} else {
checkpoint.receipts = vec![ReceiptFixture::Malformed(1), ReceiptFixture::Malformed(2)];
}
let mut logs = synthetic_logs(&template);
for log in &mut logs {
if let LogEntry::AnnotatedUserInput { extensions, .. } = log {
*extensions = vec![checkpoint_extension(&checkpoint)];
}
}
assert_rejected_session_bytes(&worker, &template, encode_synthetic_logs(&logs));
}
}
#[test]
fn unknown_fields_at_every_retained_record_depth_are_rejected_without_secret_diagnostics() {
let (_source, worker, template) = source();
let original = String::from_utf8(template.segment_jsonl.clone()).unwrap();
// These values are harmless synthetic markers, not lab/private material.
// The compatibility decoder normally discards many of these fields.
let locations = [
r#""kind":"annotated_segment_start""#,
r#""kind":"invoke""#,
r#""kind":"annotated_user_input""#,
r#""kind":"annotated_assistant_item""#,
r#""kind":"llm_usage""#,
r#""kind":"turn_end""#,
r#""kind":"run_completed""#,
r#""max_tokens":null"#,
r#""kind":"message","role":"assistant""#,
r#""kind":"text","text":"artificial fixture reply""#,
r#""kind":"text","content":"Harmless artificial fixture input.""#,
r#""entry_id":"fixed-entry-id""#,
r#""kind":"model_output""#,
r#""worker_id":"01a1657d-b762-7791-87ae-a49f28cadbce""#,
r#""domain":"worker.pending_activations.v1""#,
r#""next_activation_sequence":2"#,
r#""submission_request_id":"completed-submission-request""#,
r#""notification_request_id":"completed-notification-request""#,
];
for location in locations {
assert!(original.contains(location));
for extra in [
r#", "reasoning":{"text":"synthetic-discarded-material"}"#,
r#", "encrypted_content":"synthetic-discarded-material""#,
] {
let altered = original.replacen(location, &format!("{location}{extra}"), 1);
assert_rejected_session_bytes(&worker, &template, altered.into_bytes());
}
}
let duplicate = original.replacen(
r#""role":"assistant""#,
r#""role":"assistant","role":"assistant""#,
1,
);
assert_rejected_session_bytes(&worker, &template, duplicate.into_bytes());
}
#[test]
fn opaque_feature_payloads_cannot_bypass_strict_session_validation() {
let (_source, worker, template) = source();
let mut logs = synthetic_logs(&template);
for log in &mut logs {
if let LogEntry::AnnotatedUserInput { history, .. } = log {
if let session_store::LoggedItem::Message { role, .. } = &mut history[0].item {
*role = session_store::LoggedRole::System;
}
history[0].metadata.input_metadata = Some(
session_store::HistoryInputMetadata::new(
session_store::HistoryInputUsePolicy {
recording_material: session_store::InputEvidenceUse::Exclude,
memory_evidence: session_store::InputEvidenceUse::Exclude,
observation: session_store::InputObservationUse::Hidden,
compaction: session_store::InputCompactionUse::Retain,
},
Some(
session_store::FeatureInputPayload::encode(
"fixture.test",
1,
&BTreeMap::from([("reasoning", "harmless synthetic payload")]),
)
.unwrap(),
),
None,
)
.unwrap(),
);
}
}
assert_rejected_session_bytes(&worker, &template, encode_synthetic_logs(&logs));
}
#[cfg(unix)]
#[test]
fn refuses_source_and_destination_symlinks() {
let (_source, worker, template) = source();
let (_target, options) = destination(&template);
std::os::unix::fs::symlink(worker.join(WORKER_FILE), options.root.join("linked")).unwrap();
assert!(template.install(&options, request(), &resolved()).is_err());
fs::remove_file(options.root.join("linked")).unwrap();
std::os::unix::fs::symlink(worker.join(WORKER_FILE), worker.join("runs/link")).unwrap();
assert!(StoppedWorkerTemplate::extract(&worker).is_err());
}
/// Opt-in only: source path supplied explicitly; assertions never display content.
#[test]
#[ignore = "requires the approved artificial lab aggregate via YOI_STOPPED_FIXTURE_SOURCE"]
fn approved_lab_extract_install() {
let path = PathBuf::from(
std::env::var_os("YOI_STOPPED_FIXTURE_SOURCE").expect("explicit source required"),
);
let template = StoppedWorkerTemplate::extract(&path).unwrap();
let (_target, options) = destination(&template);
let mut bind = request();
bind.worker_id = template.worker_id();
bind.workspace_api.as_mut().unwrap().workspace_id = template.workspace_id().into();
bind.memory_settings.as_mut().unwrap().workspace_id = template.workspace_id().into();
let mut manifest = resolved();
manifest.worker.name = template.worker_name();
template.install(&options, bind, &manifest).unwrap();
let installed = options
.root
.join(WORKERS_DIR)
.join(template.worker_id().to_string());
for relative in [
"session/session.json".to_string(),
format!("session/segments/{}.jsonl", template.segment_id()),
] {
assert!(
fs::read(path.join(&relative)).unwrap() == fs::read(installed.join(relative)).unwrap()
);
}
let runtime = crate::Runtime::with_fs_store(options).unwrap();
assert_eq!(runtime.list_workers().unwrap().len(), 1);
assert_eq!(runtime.summary().unwrap().active_worker_count, 0);
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,256 @@
//! Restore retains attachment identity without opening a provider. Failed opens
//! are retryable at operation time; closing an unopened session never opens it.
use std::sync::Arc;
use async_trait::async_trait;
use tokio::sync::{Mutex, broadcast};
use workdir::*;
type OpenSession = dyn Fn() -> Result<WorkdirSessionHandle, String> + Send + Sync;
pub(super) struct DeferredWorkdirSession {
workdir: Workdir,
capabilities: WorkdirSessionCapabilities,
open: Arc<OpenSession>,
state: Mutex<State>,
events: broadcast::Sender<CommandEvent>,
}
#[derive(Default)]
struct State {
session: Option<WorkdirSessionHandle>,
event_bridge: Option<tokio::task::JoinHandle<()>>,
closed: bool,
}
impl Drop for State {
fn drop(&mut self) {
if let Some(task) = &self.event_bridge {
task.abort();
}
}
}
impl std::fmt::Debug for DeferredWorkdirSession {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DeferredWorkdirSession")
.field("workdir", &self.workdir)
.finish_non_exhaustive()
}
}
impl DeferredWorkdirSession {
pub(super) fn handle(
workdir_id: &str,
capabilities: WorkdirSessionCapabilities,
open: impl Fn() -> Result<WorkdirSessionHandle, String> + Send + Sync + 'static,
) -> WorkdirSessionHandle {
Arc::new(Self {
workdir: Workdir::new(workdir_id),
capabilities,
open: Arc::new(open),
state: Mutex::new(State::default()),
events: broadcast::channel(1024).0,
})
}
async fn session(&self) -> Result<WorkdirSessionHandle, WorkdirError> {
let mut state = self.state.lock().await;
if state.closed {
return Err(WorkdirError::SessionClosed);
}
if let Some(session) = &state.session {
return Ok(session.clone());
}
let open = self.open.clone();
let session = tokio::task::spawn_blocking(move || open())
.await
.map_err(|_| WorkdirError::Unavailable("Workdir session opener failed".into()))?
.map_err(WorkdirError::Unavailable)?;
if session.workdir().id() != self.workdir.id() {
session.close().await?;
return Err(WorkdirError::Unavailable(
"Workdir session identity mismatch".into(),
));
}
if let Some(mut source) = session.subscribe_command_events() {
let events = self.events.clone();
state.event_bridge = Some(tokio::spawn(async move {
loop {
match source.recv().await {
Ok(event) => {
let _ = events.send(event);
}
Err(broadcast::error::RecvError::Lagged(_)) => continue,
Err(broadcast::error::RecvError::Closed) => break,
}
}
}));
}
state.session = Some(session.clone());
Ok(session)
}
}
macro_rules! impl_deferred_session {
($( $method:ident($request:ident: $request_type:ty) -> $result:ty; )*) => {
#[async_trait]
impl WorkdirSession for DeferredWorkdirSession {
fn workdir(&self) -> &Workdir {
&self.workdir
}
fn capabilities(&self) -> WorkdirSessionCapabilities {
self.capabilities
}
$(
async fn $method(&self, $request: $request_type) -> Result<$result, WorkdirError> {
self.session().await?.$method($request).await
}
)*
fn subscribe_command_events(&self) -> Option<broadcast::Receiver<CommandEvent>> {
Some(self.events.subscribe())
}
fn command_snapshot(&self) -> Vec<CommandSnapshot> {
self.state.try_lock().ok().and_then(|state| {
state.session.as_ref().map(|session| session.command_snapshot())
}).unwrap_or_default()
}
async fn close(&self) -> Result<(), WorkdirError> {
let mut state = self.state.lock().await;
state.closed = true;
if let Some(session) = &state.session {
session.close().await?;
}
if let Some(task) = state.event_bridge.take() {
task.abort();
let _ = task.await;
}
Ok(())
}
}
};
}
impl_deferred_session! {
authorize_scope_path(request: WorkdirScopeAuthorizationRequest) -> ();
scope_rules_overlap(request: WorkdirScopeOverlapRequest) -> bool;
checkout_search(request: CheckoutSearchRequest) -> CheckoutSearchResult;
checkout_observe(path: WorkdirPath) -> CheckoutObservation;
checkout_execute(request: CheckoutRequest) -> CheckoutResult;
stat(request: StatRequest) -> StatResult;
read(request: ReadRequest) -> ReadResult;
read_bytes(request: ReadBytesRequest) -> ReadBytesResult;
write(request: WriteRequest) -> WriteResult;
edit(request: EditRequest) -> EditResult;
list(request: ListRequest) -> ListResult;
glob(request: GlobRequest) -> GlobResult;
grep(request: GrepRequest) -> GrepResult;
start_command(request: CommandRequest) -> CommandHandle;
command_status(handle: CommandHandle) -> CommandStatus;
command_output(request: CommandOutputRequest) -> CommandOutput;
cancel_command(handle: CommandHandle) -> ();
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering};
fn read_request() -> ReadRequest {
ReadRequest {
path: WorkdirPath::new("README.md").unwrap(),
offset: 0,
limit: 10,
max_bytes: 1024,
}
}
#[tokio::test]
async fn unopened_session_close_does_not_connect_and_is_terminal() {
let attempts = Arc::new(AtomicUsize::new(0));
let observed = attempts.clone();
let session = DeferredWorkdirSession::handle(
"offline",
WorkdirSessionCapabilities::READ_ONLY,
move || {
observed.fetch_add(1, Ordering::SeqCst);
Err("offline".into())
},
);
assert_eq!(session.workdir().id().as_str(), "offline");
assert!(session.subscribe_command_events().is_some());
session.close().await.unwrap();
session.close().await.unwrap();
assert!(matches!(
session.read(read_request()).await,
Err(WorkdirError::SessionClosed)
));
assert_eq!(attempts.load(Ordering::SeqCst), 0);
}
#[tokio::test]
async fn failed_open_retries_on_use_and_keeps_read_only_capabilities() {
let temp = tempfile::tempdir().unwrap();
std::fs::write(temp.path().join("README.md"), "restored").unwrap();
let root = temp.path().to_path_buf();
let attempts = Arc::new(AtomicUsize::new(0));
let observed = attempts.clone();
let session = DeferredWorkdirSession::handle(
"saved",
WorkdirSessionCapabilities::READ_ONLY,
move || {
if observed.fetch_add(1, Ordering::SeqCst) == 0 {
return Err("offline".into());
}
let scope = super::super::runtime_attachment_scope(
&root,
WorkdirSessionCapabilities::READ_ONLY,
)?;
Ok(super::super::runtime_local_workdir_session(
"saved",
&root,
&root,
scope,
WorkdirSessionCapabilities::READ_ONLY,
Default::default(),
Vec::new(),
))
},
);
assert_eq!(attempts.load(Ordering::SeqCst), 0);
assert!(matches!(
session.read(read_request()).await,
Err(WorkdirError::Unavailable(_))
));
assert_eq!(
session.read(read_request()).await.unwrap().bytes,
b"restored"
);
assert_eq!(
session.read(read_request()).await.unwrap().bytes,
b"restored"
);
assert_eq!(attempts.load(Ordering::SeqCst), 2);
assert!(
session
.write(WriteRequest {
path: WorkdirPath::new("README.md").unwrap(),
content: b"forbidden".to_vec(),
expected_hash: None
})
.await
.is_err()
);
assert_eq!(
std::fs::read(temp.path().join("README.md")).unwrap(),
b"restored"
);
session.close().await.unwrap();
}
}

Some files were not shown because too many files have changed in this diff Show More