38 Commits
Author SHA1 Message Date
Hare adfa422973 chore: merge develop into hare/develop 2026-10-17 12:06:05 +09:00
Hare 10d200d9ba chore: align Journal references with integrated develop 2026-10-17 03:52:11 +09:00
Hare 7a5228b8e6 fix: reconcile Memory Journal links with handoff fencing 2026-10-17 03:52:11 +09:00
Hare 0eabdf3a1f chore: integrate Journal-gated Subject handoff 2026-10-17 03:47:11 +09:00
Hare b1808b7c4d fix: unblock MR follow-up and simplify orchestration guidance 2026-10-17 03:35:58 +09:00
Hare 6645ea0a47 fix: observe original presence through saved legacy Journal links 2026-10-17 00:10:35 +09:00
Hare b544e53bd7 feat: connect Memory provenance to durable Journal context 2026-10-16 23:54:54 +09:00
Hare 04a4e59100 feat: fence Subject handoff on recorded Journal and seed new bodies once 2026-10-16 20:51:08 +09:00
Hare 52e914eebc feat: offer ID-free passkey login and explicit legacy migration 2026-10-16 19:01:43 +09:00
Hare 9c12fb4372 feat: verify discoverable passkeys against their saved owners 2026-10-16 19:01:43 +09:00
Hare 3c18916118 test: wait for complete isolated server address records 2026-10-16 19:01:43 +09:00
Hare be1c030350 chore: merge develop into hare/develop 2026-10-16 15:08:40 +09:00
Hare a7faf2cea6 docs: record test optimization results and runner validation 2026-10-16 15:04:52 +09:00
Hare 5cc5ece997 feat: add bounded parallel runner for the normal test suite 2026-10-16 15:04:52 +09:00
Hare ffaaf2ecb0 test: isolate local rules and reduce repeated fixture setup 2026-10-16 15:04:52 +09:00
Hare 6667985883 fix: preserve test completion and fixture invariants under concurrency 2026-10-16 15:04:42 +09:00
Hare 22dcd8a29f fix: make reverse session reads linear for large records 2026-10-16 15:04:42 +09:00
Hare a2816a053d fix: isolate Workspace notifications from public entry 2026-10-16 11:30:00 +09:00
Hare d74660d1d1 fix: retain workspace creation state across session rechecks 2026-10-16 11:08:27 +09:00
Hare 24c6f772f7 fix: serve entry shells independently of workspace catalog 2026-10-16 10:48:51 +09:00
Hare a060ffada8 feat: add public entry and authenticated workspace dashboard 2026-10-16 10:48:51 +09:00
Hare b8b12442a0 feat: separate login registration and own account flows 2026-10-16 09:43:57 +09:00
Hare d22899d39e feat: activate provisional users with scoped passkey enrollment 2026-10-16 08:48:35 +09:00
Hare af008f802d feat: explicit hosting administrators and persistent self-registration policy 2026-10-16 07:26:21 +09:00
Hare 7042ec7d50 chore: merge develop into hare/develop 2026-10-16 05:52:36 +09:00
Hare a2cec5afd1 fix: preserve console alert order and improve alert colors 2026-10-16 05:28:17 +09:00
Hare b5a307811b fix: reconcile erased Archive removals and exercise live compaction recording
Reconcile immutable Runtime removal input under a confirmed permanent erase fence. A confirmed source_removed receipt is audited without republishing erased archives; unknown receipt plus attested remaining aggregate permits an audited stale plan and proof-bound purge replacement. Unknown receipt and inventory remain unresolved. Distinguish domain tombstones from scoped Runtime physical erasure acknowledgements.

Erase diagnostic originals and interrupted staging bound to the same archived Session/removal operation, validating Workspace/Worker/fingerprint and saved receipts without rewriting their historical retained flags. Finalize Backend removal only after physical acknowledgement, including previous Retain disposition.

Connect public standalone protocol, real Worker request-threshold compaction twice within one turn, cancellation and real pre-AI input rollback, original canonical readers, existing Jobs and sequential Journal tools. Gate generation while compaction completes; verify all durable originals were served, summary/rolled-back input exclusion, exact final cursor, no extra input for successor drain, and one ongoing open entry. External model streams/usage are scripted, not semantic evaluation.

Validation: Backend1112lib+11bin (existing ignore1), Runtime429lib+38bin, standalone72lib+27integration (existing model ignore1), runtime-api10, root check/doc/fmt/diff/naming, Server/Runtime builds, isolated worker_environment2 passed. Earlier broad suites remain recorded on the prior source. Fixtures and injected boundaries are not arbitrary crash/remote-transport-loss coverage; worker_environment only verifies startup/connection/cleanup.

Acceptance clarification: T-734 assignment answer event22 explicitly defers real-model semantic condition6 to user use, excluding it from this completion scope. It remains unexecuted and unevaluated, not passed. Technical review findings still require a fresh independent verdict on this source.
2026-10-16 04:58:53 +09:00
Hare 5bd49760e5 fix: fence Journal originals before publication and reconcile removal intent
Protect attributed originals before the first public commit; distinguish pending Session material from missing committed sources. Gate removal admission by the Journal Feature and resolve Subject ownership from durable singleton authority. Resume explicit erasure through signed delete transport and proof-bound purge plans without creating a Journal Job or archive.

Validation: changed-crate suites, root check, doc/fmt/diff/naming checks, Server/Runtime builds and worker_environment isolation suite passed. Public standalone scripted dialog connects Host, automatic Job, sequential tools and drain while dialog continues. Backend public embedded WorkerRemove tests connect removal forwarding, durable Jobs, retention, Runtime and SQLite; Session/attribution/profile fixture setup bypasses creation/auth routes. Runtime signed generated-client tests separately exercise request authentication.

Limits: real-model semantic acceptance condition 6 remains unevaluated without a new explicit model/credential/budget grant. Conditions 1/7/8/11 do not have complete product E2E coverage: worker_environment only verifies isolated startup/connection/cleanup. Compaction tests connect local hooks/reader/store rather than the full public dialog-to-Journal chain. Erasure racing a previously prepared Archive removal plan fails closed with conflict; automatic stale-plan replacement is not implemented.
2026-10-16 03:59:45 +09:00
Hare 9debf43645 Merge remote-tracking branch 'origin/develop' into work/T-734-journal-recording 2026-10-16 02:19:24 +09:00
Hare 81af1f085c feat: record confirmed public Session history through durable Journal Jobs 2026-10-16 02:19:24 +09:00
Hare b67aa2de7d fix: simplify Ticket reporting and speed up browser lists
Remove model-supplied assignment and retry identities from implementation reports; retain Host idempotency, authentication and notification evidence.

Add a SQL-only browser projection and decouple auxiliary page loads. Preserve authoritative MR evidence queries. Include regression tests and component benchmarks, documenting the remaining no-match search cost.
2026-10-16 01:58:15 +09:00
Hare 9f1cbbd385 fix: inherit parent profile for default subworkers 2026-10-16 01:57:58 +09:00
Hare 8a3d9d67d2 fix: guard deferred state when leaving workspace routes 2026-10-16 01:57:58 +09:00
Hare 6218c27fb6 fix: separate account registration from authenticated passkey addition
Bind enrollment purpose and allocated/current account to verifier state. Reject existing-handle registration, derive additions from live transport authentication, recheck revocation/expiry in the insert-only transaction, and consume challenges atomically across connections. Derive stable account WebAuthn handles without rewriting legacy credentials; reject implicit-user login and unsafe legacy ceremonies.

Align account callers, DTO/OpenAPI/generated artifacts and migration/recovery rationale. Preserve credential-less legacy identities rather than allowing anonymous reclaim.

Validation: 14 actual-verifier passkey tests; full workspace-server 1086 library + 11 binary passed (Drive ignored case explicitly selected separately and passed); server-api default 99 and TypeScript feature 108 passed; isolated auth/Runtime E2E 1+2 passed; Web 645 + component 394 passed; root cargo check, docs/doctests, Server/Runtime builds, fmt, naming and generated consistency passed. Software authenticator only; hosted CI/hardware login not claimed.
2026-10-16 00:55:16 +09:00
Hare b744efd104 fix: align isolated runtime harness with current trust contract
Keep Workspace trust enrollment and owner-authenticated registration, then verify each Runtime connection with a fresh signed request and match its binding ID. Remove obsolete workspace_trust_id input and cached connection_state assertions.

Validation: cargo test -p yoi-e2e --features e2e --test worker_environment --test auth_passkeys -- --nocapture (2 Runtime harness tests and 1 Passkey product E2E passed); root cargo check.
2026-10-16 00:35:23 +09:00
Hare 18384482f6 feat: integrate durable Journal storage APIs 2026-10-15 20:51:51 +09:00
Hare c2df3041f9 fix: pin Journal source pagination to saved coverage content 2026-10-14 22:18:02 +09:00
Hare aabc502f30 feat: add durable subject Journal operations and bounded reads 2026-10-14 21:57:35 +09:00
293 changed files with 61359 additions and 2305 deletions
Generated
+5
View File
@@ -6548,6 +6548,7 @@ dependencies = [
"session-metrics",
"session-store",
"sha2 0.11.0",
"subjektiv",
"tempfile",
"thiserror 2.0.18",
"ticket",
@@ -6708,8 +6709,11 @@ dependencies = [
"reqwest",
"serde",
"serde_json",
"server-api",
"sha2 0.11.0",
"tempfile",
"webauthn-authenticator-rs",
"webauthn-rs-proto",
]
[[package]]
@@ -6770,6 +6774,7 @@ dependencies = [
"uuid",
"webauthn-authenticator-rs",
"webauthn-rs",
"webauthn-rs-proto",
"wip-protocol",
"workdir",
"worker",
+10
View File
@@ -91,6 +91,16 @@ debug = "line-tables-only"
[profile.dev.package."*"]
debug = false
# Keep the real SQLite engine in schema/transaction tests without paying for
# an unoptimized C interpreter on every statement. Rust test assertions stay enabled.
[profile.test.package.libsqlite3-sys]
opt-level = 2
# Schema/authority integration tests execute substantial parsing and validation.
# Optimize that code without disabling test/debug assertions or overflow checks.
[profile.test.package.yoi-workspace-server]
opt-level = 1
[workspace.dependencies]
# Internal crates
client = { path = "crates/client" }
+42 -9
View File
@@ -1,13 +1,38 @@
//! Compile the real public macro through a renamed dependency in an isolated consumer.
//! Only Cargo artifacts are shared; Cargo checks the fresh consumer on every invocation.
use std::{fs, path::PathBuf, process::Command};
struct Fixture(PathBuf);
impl Fixture {
fn new() -> Self {
for attempt in 0..1000 {
let path = std::env::temp_dir().join(format!(
"api-macros-renamed-dependency-{}-{attempt}",
std::process::id()
));
match fs::create_dir(&path) {
Ok(()) => return Self(path),
Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue,
Err(error) => panic!("create renamed dependency fixture: {error}"),
}
}
panic!("could not allocate a renamed dependency fixture directory");
}
}
impl Drop for Fixture {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
#[test]
fn macro_expansion_resolves_a_renamed_support_dependency() {
let fixture = std::env::temp_dir().join(format!(
"api-macros-renamed-dependency-{}",
std::process::id()
));
let _ = fs::remove_dir_all(&fixture);
fs::create_dir_all(fixture.join("src")).expect("create fixture source directory");
let fixture = Fixture::new();
let fixture = &fixture.0;
fs::create_dir(fixture.join("src")).expect("create fixture source directory");
let package_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let package_path = package_path
@@ -49,13 +74,21 @@ pub fn operation_count() -> usize {
)
.expect("write fixture source");
// Keep nested Cargo checks off the outer build's target lock. Cargo owns cache
// invalidation and locking between concurrent consumers, not a saved
// test-success marker.
let executable = std::env::current_exe().expect("current test executable");
let target_dir = executable
.parent()
.and_then(|deps| deps.parent())
.expect("test executable has a profile directory")
.join("compile-tests/api-macros-renamed");
let output = Command::new(env!("CARGO"))
.args(["check", "--quiet", "--offline"])
.current_dir(&fixture)
.env("CARGO_TARGET_DIR", fixture.join("target"))
.current_dir(fixture)
.env("CARGO_TARGET_DIR", target_dir)
.output()
.expect("run cargo check for renamed dependency fixture");
let _ = fs::remove_dir_all(&fixture);
assert!(
output.status.success(),
+11 -1
View File
@@ -81,6 +81,16 @@ pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
resolved_path: "profiles/job.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:subjektiv-journal-recording"),
path: "profiles/subjektiv-journal-recording.dcdl",
source: include_str!("../../../resources/profiles/subjektiv-journal-recording.dcdl"),
description: "Journal recording policy with Host-bound sequential Job tools.",
imports: &[BuiltinProfileImport {
specifier: "./job.dcdl",
resolved_path: "profiles/job.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:ticket-worker"),
path: "profiles/ticket-worker.dcdl",
@@ -123,7 +133,7 @@ pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
selector: Some("builtin:orchestrator"),
path: "profiles/orchestrator.dcdl",
source: include_str!("../../../resources/profiles/orchestrator.dcdl"),
description: "Intent-led Ticket orchestration with separate guarded MR integration authority.",
description: "Outcome-led orchestration, selection of candidate changes, and risk-proportionate validation.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
+2 -1
View File
@@ -1746,7 +1746,7 @@ mod tests {
}
#[test]
fn builtin_orchestrator_keeps_cleanup_tool_providers_enabled() {
fn builtin_orchestrator_enables_reviewer_handoff_and_cleanup_without_self_review() {
let tmp = TempDir::new().unwrap();
let resolved = ProfileResolver::new()
.with_workspace_base(tmp.path())
@@ -1757,6 +1757,7 @@ mod tests {
.unwrap();
let feature = resolved.manifest.feature;
assert!(feature.sub_worker.enabled);
assert!(feature.worker.enabled);
assert!(!feature.worker.direct_spawn);
assert!(feature.manage_workdir.enabled);
+6 -6
View File
@@ -826,7 +826,7 @@ impl MergeRequestStore {
));
}
let mr = self.get_for_operation(&i.auth.workspace_id, &i.merge_request_id, &i.ticket_id)?;
self.assigned(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
if mr.state == MergeRequestState::Closed
|| (mr.state == MergeRequestState::Open && mr.selector_from.is_none())
{
@@ -1035,7 +1035,7 @@ impl MergeRequestStore {
) -> Result<ReviewRevokedEvent, MergeRequestError> {
bounded_body("revocation reason", &i.reason)?;
let mr = self.get_for_operation(&i.auth.workspace_id, &i.merge_request_id, &i.ticket_id)?;
self.assigned(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
let r = mr
.thread
.iter()
@@ -1152,7 +1152,7 @@ impl MergeRequestStore {
"operation fingerprint mismatch".into(),
));
}
self.completion_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
self.current_assignment_auth(&i.auth, &i.ticket_id, &mr.repository_id)?;
if mr.state != MergeRequestState::Open {
return Err(MergeRequestError::Conflict(
"Merge Request is not open".into(),
@@ -1186,7 +1186,7 @@ impl MergeRequestStore {
));
}
// Case openness is not an integration phase. Exact source approval,
// current Ticket content and active assignment fencing above remain
// current Ticket content and current assignment fencing above remain
// the independent MR integration guards.
Ok(())
}
@@ -1229,7 +1229,7 @@ impl MergeRequestStore {
}
let assignment_is_current: bool = transaction.query_row(
"SELECT EXISTS(
SELECT 1 FROM ticket_active_worker_assignments
SELECT 1 FROM ticket_current_worker_assignments
WHERE workspace_id=?1 AND ticket_id=?2 AND assignment_id=?3
)",
params![i.auth.workspace_id, i.ticket_id, i.auth.assignment_id],
@@ -1548,7 +1548,7 @@ impl MergeRequestStore {
}
Ok(())
}
fn completion_auth(
fn current_assignment_auth(
&self,
a: &MergeRequestAuth,
t: &str,
+64 -11
View File
@@ -1,3 +1,6 @@
//! Check the shipped templates against the real PDK and WIT for the guest target.
//! Each check owns its source and lockfile; Cargo alone shares and validates artifacts.
use std::fs;
use std::path::Path;
use std::process::Command;
@@ -135,36 +138,86 @@ fn embedded_service_template_cargo_checks_for_wasm_target() {
fn cargo_check_template(template: &str) {
let crate_dir = Path::new(env!("CARGO_MANIFEST_DIR"));
let template_dir = crate_dir.join(format!("../../resources/plugin/templates/{template}"));
let manifest_path = template_dir.join("Cargo.toml");
let lock_path = template_dir.join("Cargo.lock");
let _ = fs::remove_file(&lock_path);
let templates_dir = crate_dir.join("../../resources/plugin/templates");
let template_dir = templates_dir.join(template);
let target_dir = tempfile::tempdir().expect("temporary cargo target dir");
let cargo = std::env::var("CARGO").unwrap_or_else(|_| "cargo".to_string());
let output = Command::new(cargo)
// A sibling fixture preserves the shipped relative PDK and WIT paths verbatim.
// Never create or remove a lockfile in the shipped template: another invocation
// may be checking it at the same time, or a developer may own that lockfile.
let fixture = tempfile::Builder::new()
.prefix(".cargo-check-")
.tempdir_in(&templates_dir)
.expect("temporary template fixture");
copy_template_tree(&template_dir, fixture.path(), true);
// Both templates share the PDK dependencies. A separate persistent target
// reuses those artifacts without sharing source or the outer Cargo build lock.
// Cargo fingerprints and its target lock handle changes and concurrent checks.
let executable = std::env::current_exe().expect("current test executable");
let target_dir = executable
.parent()
.and_then(|deps| deps.parent())
.expect("test executable has a profile directory")
.join("compile-tests/plugin-pdk-templates");
let output = Command::new(env!("CARGO"))
.arg("check")
.arg("--manifest-path")
.arg(&manifest_path)
.arg(fixture.path().join("Cargo.toml"))
.arg("--target")
.arg("wasm32-unknown-unknown")
.arg("--offline")
.arg("--target-dir")
.arg(target_dir.path())
.arg(target_dir)
.current_dir(fixture.path())
.env("CARGO_TERM_COLOR", "never")
.output()
.expect("spawn cargo check for embedded template");
let _ = fs::remove_file(&lock_path);
assert!(
output.status.success(),
"template cargo check failed\nstatus: {}\nstdout:\n{}\nstderr:\n{}",
"template {template} cargo check failed\nstatus: {}\nstdout:\n{}\nstderr:\n{}",
output.status,
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
}
fn copy_template_tree(source: &Path, destination: &Path, is_root: bool) {
let metadata = fs::symlink_metadata(source).expect("read template directory metadata");
assert!(
metadata.is_dir() && !metadata.file_type().is_symlink(),
"template input must be a real directory: {}",
source.display()
);
for entry in fs::read_dir(source).expect("read template directory") {
let entry = entry.expect("read template entry");
let path = entry.path();
let kind = entry.file_type().expect("read template entry type");
assert!(
!kind.is_symlink() && (kind.is_dir() || kind.is_file()),
"unsupported template input (including symlinks): {}",
path.display()
);
let name = entry.file_name();
// Preserve all compiler inputs, including build scripts and nested assets.
// Only omit the root build output and lockfile: each isolated consumer must
// resolve its own lockfile, as the original unlocked template check did.
if is_root
&& ((kind.is_dir() && name == "target") || (kind.is_file() && name == "Cargo.lock"))
{
continue;
}
let copied = destination.join(name);
if kind.is_dir() {
fs::create_dir(&copied).expect("create template fixture directory");
copy_template_tree(&path, &copied, false);
} else {
fs::copy(&path, &copied).expect("copy unmodified template input");
}
}
}
#[test]
fn pdk_runtime_dependencies_are_guest_side_only() {
let cargo: Value = toml::from_str(PDK_CARGO).expect("PDK Cargo.toml parses");
+54
View File
@@ -1497,6 +1497,11 @@ pub enum Event {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct Alert {
/// Stable only within the emitting live Worker; not durable alert history.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub position: Option<AlertPosition>,
pub level: AlertLevel,
pub source: AlertSource,
pub message: String,
@@ -1504,6 +1509,18 @@ pub struct Alert {
pub timestamp_ms: i64,
}
/// Display anchor captured at emission, resolved when pending text is committed.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum AlertPosition {
Start,
AfterEntry { entry_id: String },
AfterToolCall { call_id: String },
AfterInFlight { block_id: u64 },
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -1704,6 +1721,9 @@ pub struct InFlightCompaction {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct InFlightSnapshot {
/// Live-Worker identities parallel to the blocks actually included below.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub block_ids: Vec<u64>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub blocks: Vec<InFlightBlock>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
@@ -2721,6 +2741,7 @@ mod tests {
},
state: WorkerStatus::Running.into(),
in_flight: InFlightSnapshot {
block_ids: Vec::new(),
blocks: vec![
InFlightBlock::Text {
text: "hel".into(),
@@ -3048,9 +3069,42 @@ mod tests {
}
}
#[test]
fn alert_ordering_metadata_is_optional_and_uses_tagged_wire_anchors() {
let legacy = serde_json::json!({ "level": "warn", "source": "worker", "message": "old", "timestamp_ms": 42 });
let alert: Alert = serde_json::from_value(legacy.clone()).unwrap();
assert!(alert.id.is_none());
assert!(alert.position.is_none());
assert_eq!(serde_json::to_value(alert).unwrap(), legacy);
for position in [
serde_json::json!({ "kind": "start" }),
serde_json::json!({ "kind": "after_entry", "entry_id": "entry-1" }),
serde_json::json!({ "kind": "after_tool_call", "call_id": "call-1" }),
serde_json::json!({ "kind": "after_in_flight", "block_id": 7 }),
] {
let mut wire = legacy.clone();
wire["id"] = serde_json::json!("alert-1");
wire["position"] = position;
let alert: Alert = serde_json::from_value(wire.clone()).unwrap();
assert_eq!(serde_json::to_value(alert).unwrap(), wire);
}
let old_snapshot: InFlightSnapshot = serde_json::from_value(serde_json::json!({})).unwrap();
assert!(old_snapshot.block_ids.is_empty());
assert_eq!(
serde_json::to_value(old_snapshot).unwrap(),
serde_json::json!({})
);
let wire =
serde_json::json!({"blocks": [{"kind": "text", "text": "pending"}], "block_ids": [7]});
let snapshot: InFlightSnapshot = serde_json::from_value(wire.clone()).unwrap();
assert_eq!(serde_json::to_value(snapshot).unwrap(), wire);
}
#[test]
fn event_alert_format() {
let event = Event::Alert(Alert {
id: None,
position: None,
level: AlertLevel::Warn,
source: AlertSource::Compactor,
message: "compaction failed".into(),
+14 -13
View File
@@ -4,19 +4,19 @@ use serde_json::{Map, Value};
use ts_rs::{Config, TS};
use crate::{
Alert, AlertLevel, AlertSource, CommandEvent, CommandSnapshot, CommandStatus, CommandStream,
CommandStreamSlice, CompactionLifecycle, CompactionLifecycleState, CompactionPhase,
CompactionTrigger, CompletionContext, CompletionEntry, CompletionKind, ContextTokenSource,
ContextUsage, ErrorCode, Event, FeatureInvocation, FeatureInvocationDescriptor,
FeatureInvocationIdentity, FeatureInvocationResult, FeatureInvocationStatus,
FeatureInvocationSyntax, Greeting, InFlightBlock, InFlightCompaction, InFlightSnapshot,
InFlightToolCallState, InternalWorkerKind, InternalWorkerRef, InternalWorkerSnapshot,
InvocationArgumentDescriptor, InvocationArgumentType, InvocationArgumentValue,
InvocationClientAdapter, InvocationCompletion, InvocationValue, InvokeKind, MemoryWorkerEvent,
Method, PasteArtifactAvailability, PasteArtifactMediaType, PasteArtifactRef,
PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission, ReasoningConfig,
RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason, ScopeRule, Segment,
SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
Alert, AlertLevel, AlertPosition, AlertSource, CommandEvent, CommandSnapshot, CommandStatus,
CommandStream, CommandStreamSlice, CompactionLifecycle, CompactionLifecycleState,
CompactionPhase, CompactionTrigger, CompletionContext, CompletionEntry, CompletionKind,
ContextTokenSource, ContextUsage, ErrorCode, Event, FeatureInvocation,
FeatureInvocationDescriptor, FeatureInvocationIdentity, FeatureInvocationResult,
FeatureInvocationStatus, FeatureInvocationSyntax, Greeting, InFlightBlock, InFlightCompaction,
InFlightSnapshot, InFlightToolCallState, InternalWorkerKind, InternalWorkerRef,
InternalWorkerSnapshot, InvocationArgumentDescriptor, InvocationArgumentType,
InvocationArgumentValue, InvocationClientAdapter, InvocationCompletion, InvocationValue,
InvokeKind, MemoryWorkerEvent, Method, PasteArtifactAvailability, PasteArtifactMediaType,
PasteArtifactRef, PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission,
ReasoningConfig, RewindSummary, RunFailureKind, RunResult, RunResumeSource, RunYieldReason,
ScopeRule, Segment, SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment, SubmissionDisposition,
SymlinkPolicy, ToolResultDisposition, TurnResult, UploadedFileAvailability, UploadedFileRef,
@@ -65,6 +65,7 @@ pub fn generated_protocol_types() -> String {
);
push_decl::<AlertLevel>(&cfg, &mut output);
push_decl::<AlertPosition>(&cfg, &mut output);
push_decl::<AlertSource>(&cfg, &mut output);
push_decl::<CompletionKind>(&cfg, &mut output);
push_decl::<CompletionContext>(&cfg, &mut output);
+262 -2
View File
@@ -425,6 +425,8 @@ pub struct BackendJobExecutionBinding {
pub input_digest: Option<String>,
#[serde(default)]
pub subjektiv_consolidation: bool,
#[serde(default)]
pub subjektiv_journal: bool,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
@@ -712,6 +714,8 @@ pub struct WorkerSessionArchiveManifest {
pub source_runtime_id: String,
pub source_worker_id: WorkerId,
pub source_session_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source_active_segment_id: Option<String>,
pub segment_ids: Vec<String>,
pub source_created_at: String,
pub removed_at: String,
@@ -787,6 +791,118 @@ pub enum SessionPublicSource {
Archived { archive_id: String },
}
/// Canonical Session metadata, distinct from Memory's evidence/public-index projection.
/// `origin_json` is the exact serialized saved origin; it is not model-authored evidence.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionMaterialDescriptor {
pub segment_id: String,
pub entry_ref: String,
pub canonical_order: u64,
pub origin_json: String,
pub sha256: String,
pub total_bytes: u64,
}
/// Explicit Host erasure, distinct from ordinary retention policy or pin release.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialEraseRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
}
/// Acknowledges the permanent fence and physical erasure of Session originals,
/// including archived diagnostics bound to the same removal operations. Historical
/// retention receipts remain immutable; their old retained flags are not current availability.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialEraseResponse {
pub session_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialPinRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub pin_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialPinResponse {
pub session_id: String,
pub pin_id: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialCatalogRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub expected_catalog_sha256: Option<String>,
pub offset: u64,
pub limit: usize,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialCatalogPage {
pub session_id: String,
pub active_segment_id: String,
pub catalog_sha256: String,
pub descriptors: Vec<SessionMaterialDescriptor>,
pub next_offset: Option<u64>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionMaterialReadRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
/// Compared with a freshly resolved saved descriptor, never adopted as evidence.
pub expected_descriptor: SessionMaterialDescriptor,
pub byte_offset: u64,
pub max_content_bytes: usize,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SessionMaterialReadPage {
pub descriptor: SessionMaterialDescriptor,
pub content: String,
pub next_offset: u64,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionMaterialCatalogAvailability {
/// The saved active pointer is pending and storage proves no Session material
/// has ever been published. Missing/corrupt materialized sources are not empty.
NotMaterialized {
session_id: String,
},
Page {
page: SessionMaterialCatalogPage,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionMaterialReadAvailability {
Page {
page: SessionMaterialReadPage,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionPublicEntryKind {
@@ -988,8 +1104,50 @@ pub enum WorkerSessionAvailability {
},
}
/// Host-owned Subject release operation. A reservation has no timeout: uncertainty retains
/// occupancy and mutation exclusion until the same operation is explicitly reconciled.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectReleaseRequest {
pub workspace_id: String,
pub worker_id: WorkerId,
pub binding_id: String,
pub operation_id: String,
pub action: SubjectReleaseAction,
}
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum SubjectReleaseAction {
Reserve,
Disconnect,
Cancel,
}
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum SubjectReleaseState {
Reserved,
Disconnected,
Cancelled,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct SubjectReleaseResponse {
pub operation_id: String,
pub binding_id: String,
pub state: SubjectReleaseState,
}
#[api(reqwest, axum)]
pub trait RuntimeApi {
#[post("/v1/subject-release", status = 200, error_status = 400)]
async fn subject_release(
&self,
#[body] request: SubjectReleaseRequest,
) -> Result<SubjectReleaseResponse, RuntimeApiError>;
#[get("/v1/ping", status = 200, error_status = 400)]
async fn ping(
&self,
@@ -1029,6 +1187,52 @@ pub trait RuntimeApi {
#[query] request: WorkerSessionHistoryRequest,
) -> Result<WorkerSessionHistoryAvailability, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/erase",
status = 200,
error_status = 400
)]
async fn session_material_erase(
&self,
#[body] request: SessionMaterialEraseRequest,
) -> Result<SessionMaterialEraseResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/pin",
status = 200,
error_status = 400
)]
async fn session_material_pin(
&self,
#[body] request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/retention/release",
status = 200,
error_status = 400
)]
async fn session_material_release(
&self,
#[body] request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError>;
#[post(
"/v1/session-public/material/catalog",
status = 200,
error_status = 400
)]
async fn session_material_catalog(
&self,
#[body] request: SessionMaterialCatalogRequest,
) -> Result<SessionMaterialCatalogAvailability, RuntimeApiError>;
#[post("/v1/session-public/material/read", status = 200, error_status = 400)]
async fn session_material_read(
&self,
#[body] request: SessionMaterialReadRequest,
) -> Result<SessionMaterialReadAvailability, RuntimeApiError>;
#[post("/v1/session-public/search", status = 200, error_status = 400)]
async fn session_public_search(
&self,
@@ -1277,6 +1481,22 @@ pub const REMAINING_RUNTIME_ROUTES: &[RemainingRuntimeRoute] = &[
mod tests {
use super::*;
#[test]
fn backend_job_journal_grant_defaults_off_and_does_not_enable_memory_grant() {
let legacy = serde_json::json!({"job_id":"job", "attempt_id":"attempt", "subjektiv_consolidation":true});
let binding: BackendJobExecutionBinding = serde_json::from_value(legacy).unwrap();
assert!(binding.subjektiv_consolidation);
assert!(!binding.subjektiv_journal);
let journal =
serde_json::json!({"job_id":"job", "attempt_id":"attempt", "subjektiv_journal":true});
let binding: BackendJobExecutionBinding = serde_json::from_value(journal.clone()).unwrap();
assert!(binding.subjektiv_journal);
assert!(!binding.subjektiv_consolidation);
let roundtrip: BackendJobExecutionBinding =
serde_json::from_value(serde_json::to_value(&binding).unwrap()).unwrap();
assert_eq!(roundtrip, binding);
}
#[test]
fn restore_request_is_mandatory_and_conflict_is_not_an_operation_state() {
for invalid in [
@@ -1442,6 +1662,47 @@ mod tests {
})
}
async fn session_material_erase(
&self,
_request: SessionMaterialEraseRequest,
) -> Result<SessionMaterialEraseResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_pin(
&self,
_request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn subject_release(
&self,
_request: SubjectReleaseRequest,
) -> Result<SubjectReleaseResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_release(
&self,
_request: SessionMaterialPinRequest,
) -> Result<SessionMaterialPinResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_catalog(
&self,
_request: SessionMaterialCatalogRequest,
) -> Result<SessionMaterialCatalogAvailability, RuntimeApiError> {
Err(test_error(501))
}
async fn session_material_read(
&self,
_request: SessionMaterialReadRequest,
) -> Result<SessionMaterialReadAvailability, RuntimeApiError> {
Err(test_error(501))
}
async fn session_public_search(
&self,
_request: SessionPublicSearchRequest,
@@ -1603,9 +1864,8 @@ mod tests {
}
#[test]
fn contract_inventory_is_complete_and_unique() {
fn contract_inventory_has_unique_routes_under_api_base_path() {
let operations = RuntimeApiMetadata::OPERATIONS;
assert_eq!(operations.len(), 20);
let mut routes = operations
.iter()
.map(|operation| (format!("{:?}", operation.method), operation.path))
+269 -2
View File
@@ -9,6 +9,8 @@ pub use api_macros::{ApiContract, BinaryBody, HttpMethod, TransportMetadata, Web
mod drive;
mod notification;
pub use notification::*;
mod ticket_browse;
pub use ticket_browse::*;
pub mod repository_openapi_typescript;
pub use drive::*;
mod workspace_config;
@@ -168,6 +170,11 @@ macro_rules! impl_openapi_schema {
impl_openapi_schema!(
AuthBootstrapUserRequest,
AuthPublicConfig,
HostingSettings,
HostingAccess,
ProvisionalAccountRequest,
ActivationCodeResponse,
ActivationOptionsRequest,
AuthUserResponse,
DeviceLoginApproveRequest,
DeviceLoginApproveResponse,
@@ -179,6 +186,7 @@ impl_openapi_schema!(
PasskeyLoginOptionsRequest,
PasskeyLoginOptionsResponse,
PasskeyLoginCompleteRequest,
PasskeyAdditionOptionsRequest,
PasskeyRegistrationOptionsRequest,
PasskeyRegistrationOptionsResponse,
PasskeyRegistrationCompleteRequest,
@@ -376,6 +384,8 @@ impl_openapi_schema!(
TicketAssignmentDiagnosticsPage,
TicketQueryRequest,
TicketQueryResponse,
TicketBrowseQueryRequest,
TicketBrowseQueryResponse,
TicketQueueResponse,
TicketRecord,
TicketRecordRef,
@@ -510,6 +520,60 @@ pub trait ServerApi {
#[get("/api/auth/config", status = 200, error_status = 400)]
async fn auth_config(&self) -> Result<AuthPublicConfig, RepositoryApiError>;
#[get("/api/auth/registration-policy", status = 200, error_status = 400)]
async fn auth_registration_policy(&self) -> Result<HostingSettings, RepositoryApiError>;
#[get("/api/hosting/access", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_access(
&self,
#[extension] context: ServerRequestContext,
) -> Result<HostingAccess, RepositoryApiError>;
#[get("/api/hosting/settings", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_settings(
&self,
#[extension] context: ServerRequestContext,
) -> Result<HostingSettings, RepositoryApiError>;
#[put("/api/hosting/settings", status = 200, error_status = 403, additional_error_statuses = [400, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_settings_update(
&self,
#[extension] context: ServerRequestContext,
#[body] request: HostingSettings,
) -> Result<HostingSettings, RepositoryApiError>;
#[post("/api/hosting/accounts", status = 200, error_status = 400, additional_error_statuses = [403, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_account_create(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ProvisionalAccountRequest,
) -> Result<ActivationCodeResponse, RepositoryApiError>;
#[post("/api/hosting/accounts/activation-code", status = 200, error_status = 400, additional_error_statuses = [403, 500], bearer_auth = true, browser_auth = true)]
async fn hosting_activation_code_reissue(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ProvisionalAccountRequest,
) -> Result<ActivationCodeResponse, RepositoryApiError>;
#[post("/api/auth/activation/options", status = 200, error_status = 400, additional_error_statuses = [500])]
async fn auth_activation_options(
&self,
#[extension] context: ServerRequestContext,
#[body] request: ActivationOptionsRequest,
) -> Result<PasskeyRegistrationOptionsResponse, RepositoryApiError>;
#[post(
"/api/auth/activation/complete",
responses = [(status = 200, body = AuthUserResponse, headers = [("set-cookie", SetCookieHeader)])],
error_status = 400,
additional_error_statuses = [500]
)]
async fn auth_activation_complete(
&self,
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthActivationComplete, RepositoryApiError>;
#[post(
"/api/auth/bootstrap-user",
status = 200,
@@ -546,6 +610,36 @@ pub trait ServerApi {
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthPasskeyRegistrationComplete, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/options",
status = 200,
error_status = 400,
additional_error_statuses = [403, 500, 502],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_options(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyAdditionOptionsRequest,
) -> Result<PasskeyRegistrationOptionsResponse, RepositoryApiError>;
#[post(
"/api/auth/passkeys/addition/complete",
responses = [
(status = 200, body = AuthUserResponse, headers = [("set-cookie", SetCookieHeader)])
],
error_status = 400,
additional_error_statuses = [403, 500],
bearer_auth = true,
browser_auth = true
)]
async fn auth_passkey_addition_complete(
&self,
#[extension] context: ServerRequestContext,
#[body] request: PasskeyRegistrationCompleteRequest,
) -> Result<server_api_responses::AuthPasskeyAdditionComplete, RepositoryApiError>;
#[post(
"/api/auth/passkeys/login/options",
status = 200,
@@ -3033,6 +3127,14 @@ pub trait ServerApi {
#[body] request: TicketTagsSetRequest,
) -> Result<WorkspaceTicketTagList, RepositoryApiError>;
/// Lightweight browser list; deliberately excludes live MR evidence and external observations.
#[post("/api/w/{workspace_id}/tickets/browse", status = 200, error_status = 400, additional_error_statuses = [401, 403, 404, 500], bearer_auth = true, browser_auth = true)]
async fn ticket_browse(
&self,
#[path] workspace_id: String,
#[body] request: TicketBrowseQueryRequest,
) -> Result<TicketBrowseQueryResponse, RepositoryApiError>;
#[post("/api/w/{workspace_id}/tickets/query", status = 200, error_status = 400, additional_error_statuses = [401, 403, 404, 500], bearer_auth = true, browser_auth = true)]
async fn ticket_query(
&self,
@@ -3803,6 +3905,76 @@ pub struct AuthPublicConfig {
pub cookie_name: String,
}
/// The only public registration policy. No administrator identities or host secrets are exposed.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct HostingSettings {
pub self_registration_enabled: bool,
}
/// Hosting authority for the current authenticated human, not Workspace permissions.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct HostingAccess {
pub is_admin: bool,
}
/// A hosting administrator reserves the same normalized ID used by passkey login.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ProvisionalAccountRequest {
pub handle: String,
}
/// The only response allowed to disclose an activation code. Clients must keep it
/// out of URLs, persistent browser storage and diagnostics. A lost response is
/// recovered by reissuing, which invalidates the previous code and its ceremonies.
#[derive(Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ActivationCodeResponse {
pub user: AuthenticatedUser,
pub code: String,
pub expires_at: String,
}
impl std::fmt::Debug for ActivationCodeResponse {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("ActivationCodeResponse")
.field("user", &self.user)
.field("code", &"[REDACTED]")
.field("expires_at", &self.expires_at)
.finish()
}
}
/// Proof for a pending account's initial passkey ceremony, never a bearer token
/// or browser session. No caller-selected account ID or enrollment purpose is accepted.
#[derive(Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct ActivationOptionsRequest {
pub handle: String,
pub code: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub browser_origin: Option<String>,
}
impl std::fmt::Debug for ActivationOptionsRequest {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("ActivationOptionsRequest")
.field("handle", &self.handle)
.field("code", &"[REDACTED]")
.field("browser_origin", &self.browser_origin)
.finish()
}
}
/// Authentication method that established the current request actor.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
@@ -3872,6 +4044,7 @@ pub struct AuthUserResponse {
pub user: AuthenticatedUser,
}
/// Starts new-account registration only; an existing handle is never interpreted as key addition.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -3885,6 +4058,16 @@ pub struct PasskeyRegistrationOptionsRequest {
pub browser_origin: Option<String>,
}
/// Adds a credential to the current authenticated actor; target identity is never public input.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct PasskeyAdditionOptionsRequest {
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub browser_origin: Option<String>,
}
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -3905,6 +4088,11 @@ pub struct PasskeyRegistrationCompleteRequest {
pub credential: RegisterPublicKeyCredential,
}
/// Normal discoverable login omits the handle: the authenticator selects a credential.
/// An explicit handle selects legacy identified-account login, for non-resident keys or
/// keys registered with an old random WebAuthn user handle. This never adds a key or
/// falls back to another identity. Authenticate first, then enroll an RK from Account.
/// Existing challenge states are short-lived and are not migrated to the new ceremony.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -8058,9 +8246,8 @@ pub enum TicketImplementationResultKind {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct TicketImplementationResultRequest {
/// Host-owned replay identity; model-facing tools derive it from the Tool call.
pub operation_key: String,
pub assignment_id: String,
pub orchestrator_assignment_id: String,
pub kind: TicketImplementationResultKind,
pub body: String,
pub merge_request: Option<MergeRequestReviewSubject>,
@@ -11338,12 +11525,18 @@ pub fn auth_api_typescript() -> String {
let config = ts_rs::Config::default();
let declarations = [
AuthPublicConfig::decl(&config),
HostingSettings::decl(&config),
HostingAccess::decl(&config),
ProvisionalAccountRequest::decl(&config),
ActivationCodeResponse::decl(&config),
ActivationOptionsRequest::decl(&config),
ActorAuthMethod::decl(&config),
AuthenticatedUser::decl(&config),
RequestActor::decl(&config),
WhoamiResponse::decl(&config),
AuthBootstrapUserRequest::decl(&config),
AuthUserResponse::decl(&config),
PasskeyAdditionOptionsRequest::decl(&config),
PasskeyRegistrationOptionsRequest::decl(&config),
PasskeyRegistrationOptionsResponse::decl(&config),
PasskeyRegistrationCompleteRequest::decl(&config),
@@ -12061,6 +12254,16 @@ mod tests {
HttpMethod::Post,
"/api/auth/passkeys/registration/options",
),
(
"auth_passkey_addition_options",
HttpMethod::Post,
"/api/auth/passkeys/addition/options",
),
(
"auth_passkey_addition_complete",
HttpMethod::Post,
"/api/auth/passkeys/addition/complete",
),
(
"auth_passkey_login_options",
HttpMethod::Post,
@@ -14312,6 +14515,9 @@ mod openapi_artifact_tests {
const PUBLIC_UNAUTHENTICATED: &[&str] = &[
"health",
"auth_config",
"auth_registration_policy",
"auth_activation_options",
"auth_activation_complete",
"auth_bootstrap_user",
"auth_passkey_registration_options",
"auth_passkey_registration_complete",
@@ -14452,6 +14658,20 @@ mod openapi_artifact_tests {
}
}
#[test]
fn passkey_addition_rejects_caller_selected_target_fields() {
for field in ["handle", "user_id", "account_id"] {
let payload = serde_json::json!({ (field): "other-user" });
assert!(
serde_json::from_value::<PasskeyAdditionOptionsRequest>(payload).is_err(),
"addition must reject caller-selected {field}"
);
}
assert!(
serde_json::from_value::<PasskeyAdditionOptionsRequest>(serde_json::json!({})).is_ok()
);
}
#[test]
fn auth_workspace_catalog_and_typed_header_contract_is_strict() {
let document = canonical_openapi_document().expect("canonical OpenAPI contract must build");
@@ -14465,6 +14685,8 @@ mod openapi_artifact_tests {
("/api/auth/bootstrap-user", "post"),
("/api/auth/passkeys/registration/options", "post"),
("/api/auth/passkeys/registration/complete", "post"),
("/api/auth/passkeys/addition/options", "post"),
("/api/auth/passkeys/addition/complete", "post"),
("/api/auth/passkeys/login/options", "post"),
("/api/auth/passkeys/login/complete", "post"),
("/api/auth/logout", "post"),
@@ -14503,6 +14725,7 @@ mod openapi_artifact_tests {
}
for path in [
"/api/auth/passkeys/registration/complete",
"/api/auth/passkeys/addition/complete",
"/api/auth/passkeys/login/complete",
"/api/auth/logout",
] {
@@ -14717,3 +14940,47 @@ mod workdir_error_diagnostic_tests {
assert!(schema["properties"].get("denial_reason").is_none());
}
}
#[cfg(test)]
mod activation_contract_tests {
use super::*;
#[test]
fn activation_inputs_reject_caller_selected_authority_and_debug_redacts_codes() {
for field in [
"account_id",
"user_id",
"purpose",
"is_admin",
"authentication",
] {
let mut body = serde_json::json!({"handle":"new-user","code":"activation-secret"});
body[field] = serde_json::json!("caller-controlled");
assert!(serde_json::from_value::<ActivationOptionsRequest>(body).is_err());
let mut body = serde_json::json!({"handle":"new-user"});
body[field] = serde_json::json!("caller-controlled");
assert!(serde_json::from_value::<ProvisionalAccountRequest>(body).is_err());
}
let request = ActivationOptionsRequest {
handle: "new-user".into(),
code: "activation-secret".into(),
browser_origin: None,
};
assert!(!format!("{request:?}").contains(&request.code));
let response = ActivationCodeResponse {
user: AuthenticatedUser {
user_id: "u".into(),
account_id: "a".into(),
handle: "new-user".into(),
display_name: "new-user".into(),
},
code: "activation-secret".into(),
expires_at: "2030-01-01T00:00:00Z".into(),
};
assert!(!format!("{response:?}").contains(&response.code));
assert_eq!(
serde_json::to_value(&response).unwrap()["code"],
"activation-secret"
);
}
}
@@ -459,6 +459,13 @@ fn validate_ticket_browser_operations(
request: Some("TicketTagSetRequest"),
successes: &[("200", "Array_of_WorkspaceTag")],
},
OperationContract {
path: "/api/w/{workspace_id}/tickets/browse",
method: "post",
operation_id: "ticket_browse",
request: Some("TicketBrowseQueryRequest"),
successes: &[("200", "TicketBrowseQueryResponse")],
},
OperationContract {
path: "/api/w/{workspace_id}/tickets/query",
method: "post",
+65
View File
@@ -0,0 +1,65 @@
//! Lightweight Tickets browser contract, separate from authoritative evidence queries.
use crate::{BrowserTicketState, QueryPage};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryRequest {
pub query: Option<String>,
#[serde(default)]
#[cfg_attr(feature = "typescript", ts(type = "Array<BrowserTicketState>"))]
pub states: Vec<ticket::TicketState>,
/// Every supplied active tag ID must match (AND).
#[serde(default)]
pub tag_ids: Vec<String>,
#[cfg_attr(
feature = "typescript",
ts(type = "TicketAssignmentPrincipalSnapshot | null")
)]
pub assigned_principal: Option<ticket::TicketAssignmentPrincipalSnapshot>,
#[cfg_attr(
feature = "typescript",
ts(type = "TicketAssignmentTrackingStatus | null")
)]
pub handling_status: Option<ticket::TicketAssignmentTrackingStatus>,
/// Exact current closed-case reason; historical reasons do not match this filter.
#[cfg_attr(feature = "typescript", ts(type = "TicketClosureReason | null"))]
pub closure_reason: Option<ticket::TicketClosureReason>,
/// Only updated_desc is supported; defaults to updated_desc.
pub sort: Option<String>,
#[schemars(range(min = 0, max = 1000))]
pub limit: Option<usize>,
/// Browse-only cursor bound to the Workspace, canonical filters, and sort.
pub cursor: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryItem {
pub id: String,
pub resource_key: String,
pub title: String,
pub state: BrowserTicketState,
pub updated_at: Option<String>,
#[cfg_attr(feature = "typescript", ts(type = "Array<WorkspaceTag>"))]
pub tags: Vec<ticket::WorkspaceTag>,
#[cfg_attr(feature = "typescript", ts(type = "TicketClosureJudgment | null"))]
pub current_closure: Option<ticket::TicketClosureJudgment>,
#[cfg_attr(
feature = "typescript",
ts(type = "Array<TicketAssignmentTrackingProjection>")
)]
pub assignment_tracking: Vec<ticket::TicketAssignmentTrackingProjection>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct TicketBrowseQueryResponse {
pub items: Vec<TicketBrowseQueryItem>,
pub page: QueryPage,
pub record_authority: String,
}
+81 -6
View File
@@ -48,17 +48,30 @@ pub struct FsStore {
/// `write_all` may have extended the file, so rollback is safe only while
/// no sibling writer can append behind it.
append_lock: Arc<Mutex<()>>,
retention: crate::retention::SessionRetentionStore,
}
impl FsStore {
/// Create a new `FsStore` rooted at the given directory.
/// Creates the directory if it does not exist.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
let retention = crate::retention::fs_retention(&root);
Self::new_with_retention(root, retention)
}
/// Hosts that relocate Worker originals must keep control state outside the relocated
/// aggregate and pass the same store to readers and deletion/transfer operations.
pub fn new_with_retention(
root: impl Into<PathBuf>,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, StoreError> {
let root = root.into();
fs::create_dir_all(&root)?;
Ok(Self {
root,
append_lock: Arc::new(Mutex::new(())),
retention,
})
}
@@ -98,12 +111,46 @@ impl FsStore {
/// Returns `Ok(true)` when a Session directory was removed and `Ok(false)`
/// when it was already absent.
pub fn delete_session(&self, session_id: SessionId) -> Result<bool, StoreError> {
let session_dir = self.session_dir(session_id);
match fs::remove_dir_all(&session_dir) {
Ok(()) => Ok(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
Err(error) => Err(error.into()),
}
self.delete_session_with_intent(
session_id,
crate::retention::SessionDeletionIntent::Ordinary,
)
}
pub fn retention(&self) -> crate::retention::SessionRetentionStore {
self.retention.clone()
}
pub fn pin_session(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().pin(session_id, pin, || {
Ok(self.session_dir(session_id).is_dir())
})
}
pub fn release_session_pin(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().release(session_id, pin)
}
pub fn delete_session_with_intent(
&self,
session_id: SessionId,
intent: crate::retention::SessionDeletionIntent,
) -> Result<bool, StoreError> {
self.retention().delete(session_id, intent, || {
match fs::remove_dir_all(self.session_dir(session_id)) {
Ok(()) => Ok(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
Err(error) => Err(error.into()),
}
})
}
fn session_dir(&self, session_id: SessionId) -> PathBuf {
@@ -282,12 +329,21 @@ impl FsStore {
}
impl Store for FsStore {
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention().retain_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.log_path(session_id, segment_id), &line)
}
@@ -374,6 +430,7 @@ impl Store for FsStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let path = self.log_path(session_id, segment_id);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
@@ -423,6 +480,7 @@ impl Store for FsStore {
content: &str,
limits: PasteArtifactLimits,
) -> Result<PasteArtifactRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -451,6 +509,7 @@ impl Store for FsStore {
content: &[u8],
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -474,6 +533,7 @@ impl Store for FsStore {
context: &UploadedFileUploadContext,
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -510,6 +570,7 @@ impl Store for FsStore {
reference: &UploadedFileRef,
source_entry_id: &str,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -537,6 +598,7 @@ impl Store for FsStore {
reference: &UploadedFileRef,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -550,6 +612,7 @@ impl Store for FsStore {
artifact_id: &str,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -563,6 +626,7 @@ impl Store for FsStore {
artifact_id: &str,
source_entry_id: &str,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -579,6 +643,7 @@ impl Store for FsStore {
session_id: SessionId,
live_owner_ids: &[String],
) -> Result<u64, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -591,6 +656,7 @@ impl Store for FsStore {
session_id: SessionId,
artifact_id: &str,
) -> Result<bool, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -599,6 +665,7 @@ impl Store for FsStore {
}
fn delete_uncommitted_uploaded_files(&self, session_id: SessionId) -> Result<u64, StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let _guard = self
.append_lock
.lock()
@@ -631,6 +698,13 @@ impl Store for FsStore {
source_session_id: SessionId,
target_session_id: SessionId,
) -> Result<u64, StoreError> {
let mut sessions = vec![source_session_id, target_session_id];
sessions.sort();
sessions.dedup();
let _retention = sessions
.into_iter()
.map(|id| self.retention().lock_live(id))
.collect::<Result<Vec<_>, _>>()?;
let _guard = self
.append_lock
.lock()
@@ -647,6 +721,7 @@ impl Store for FsStore {
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError> {
let _retention = self.retention().lock_live(session_id)?;
let line = serde_json::to_string(entry)?;
self.append_line(&self.trace_path(session_id, segment_id), &line)
}
+2
View File
@@ -45,7 +45,9 @@ mod legacy_session_log;
pub mod logged_item;
mod paste_artifact;
pub mod public_index;
pub mod public_material;
pub mod public_snapshot;
pub mod retention;
pub mod segment;
pub mod segment_log;
pub mod store;
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+398
View File
@@ -0,0 +1,398 @@
//! Durable, domain-neutral Session retention pins and permanent erasure fences.
//!
//! Hosts choose a control root whose lifetime outlasts the physical Session.
//! Every deletion path must execute its physical removal inside `delete`, and
//! publishers must hold `lock_live` while creating or appending source data.
//! A check followed by unlocked removal is not a retention guarantee.
//!
//! Pins have explicit owners and no timer/expiry: domain code releases its pin
//! only after its durable consumer no longer needs originals. A permanent erase
//! persists its fence before physical removal, invalidates pins, and rejects
//! late pins/publication even after reopening the control store. Retrying an
//! interrupted physical erase uses the same intent and is safe.
use std::collections::BTreeSet;
use std::fs::{self, File, OpenOptions};
use std::path::{Path, PathBuf};
use fs4::fs_std::FileExt;
use serde::{Deserialize, Serialize};
use crate::{SessionId, StoreError};
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(transparent)]
pub struct SessionRetentionPin(pub uuid::Uuid);
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SessionDeletionIntent {
Ordinary,
ErasePermanently,
}
#[derive(Debug, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct RetentionState {
erased: bool,
pins: BTreeSet<SessionRetentionPin>,
}
#[derive(Debug, Clone)]
pub struct SessionRetentionStore {
root: PathBuf,
}
/// A process-independent lock; dropping it releases the filesystem lock.
/// Do not recursively acquire the same Session lock inside a guarded callback.
pub struct SessionRetentionGuard {
_file: File,
}
impl SessionRetentionStore {
pub fn new(control_root: impl Into<PathBuf>) -> Self {
Self {
root: control_root.into(),
}
}
fn state_path(&self, session: SessionId) -> PathBuf {
self.root.join(format!("{session}.json"))
}
fn lock(&self, session: SessionId) -> Result<SessionRetentionGuard, StoreError> {
fs::create_dir_all(&self.root)?;
let mut options = OpenOptions::new();
options.read(true).write(true).create(true).truncate(false);
#[cfg(any(target_os = "linux", target_os = "android"))]
{
use std::os::unix::fs::OpenOptionsExt;
// Control roots are Host-owned; never follow a replaced lock file.
options.custom_flags(libc::O_NOFOLLOW);
}
let file = options.open(self.root.join(format!("{session}.lock")))?;
FileExt::lock_exclusive(&file)?;
Ok(SessionRetentionGuard { _file: file })
}
fn read(&self, session: SessionId) -> Result<RetentionState, StoreError> {
match fs::read(self.state_path(session)) {
Ok(bytes) => Ok(serde_json::from_slice(&bytes)?),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
Ok(RetentionState::default())
}
Err(error) => Err(error.into()),
}
}
fn save(&self, session: SessionId, state: &RetentionState) -> Result<(), StoreError> {
crate::worker_session_store::atomic_write_bytes(
&self.state_path(session),
&serde_json::to_vec(state)?,
)
}
/// Observational fence check; unlike `lock_live`, never creates storage.
/// Hosts still fence result adoption in their own durable consumer: a read
/// finishing before erasure is not authority for a delayed domain write.
pub fn ensure_live(&self, session: SessionId) -> Result<(), StoreError> {
if self.read(session)?.erased {
return Err(StoreError::SessionErased(session));
}
Ok(())
}
pub fn lock_live(&self, session: SessionId) -> Result<SessionRetentionGuard, StoreError> {
let guard = self.lock(session)?;
if self.read(session)?.erased {
return Err(StoreError::SessionErased(session));
}
Ok(guard)
}
/// Protect a Host-owned Session before its first source publication. This is
/// retention intent, not proof that any original material exists. The owner must
/// establish it before accepting input and use the same control store for writes
/// and deletion. Source-serving APIs must still verify actual material separately.
pub fn retain_before_publication(
&self,
session: SessionId,
pin: SessionRetentionPin,
) -> Result<(), StoreError> {
let _guard = self.lock_live(session)?;
let mut state = self.read(session)?;
if state.pins.insert(pin) {
self.save(session, &state)?;
}
Ok(())
}
/// Call `source_exists` under the same lock used by deletion. Replaying the
/// same pin is idempotent; missing source never becomes a successful lease.
pub fn pin(
&self,
session: SessionId,
pin: SessionRetentionPin,
source_exists: impl FnOnce() -> Result<bool, StoreError>,
) -> Result<(), StoreError> {
let _guard = self.lock_live(session)?;
if !source_exists()? {
return Err(StoreError::SessionSourceMissing(session));
}
let mut state = self.read(session)?;
if state.pins.insert(pin) {
self.save(session, &state)?;
}
Ok(())
}
pub fn release(&self, session: SessionId, pin: SessionRetentionPin) -> Result<(), StoreError> {
let _guard = self.lock(session)?;
let mut state = self.read(session)?;
if state.pins.remove(&pin) {
self.save(session, &state)?;
}
Ok(())
}
pub fn pins(&self, session: SessionId) -> Result<Vec<SessionRetentionPin>, StoreError> {
let _guard = self.lock(session)?;
Ok(self.read(session)?.pins.into_iter().collect())
}
/// Execute physical removal while fencing concurrent pins and publishers.
/// An erasure fence is retained even if removal fails or the source is
/// already absent. Ordinary deletion after erasure is rejected: recovery
/// must preserve the caller's explicit erase intent.
pub fn delete<T>(
&self,
session: SessionId,
intent: SessionDeletionIntent,
remove: impl FnOnce() -> Result<T, StoreError>,
) -> Result<T, StoreError> {
let _guard = self.lock(session)?;
let mut state = self.read(session)?;
match intent {
SessionDeletionIntent::Ordinary => {
if state.erased {
return Err(StoreError::SessionErased(session));
}
if !state.pins.is_empty() {
return Err(StoreError::SessionPinned(session));
}
}
SessionDeletionIntent::ErasePermanently => {
if !state.erased || !state.pins.is_empty() {
state.erased = true;
state.pins.clear();
self.save(session, &state)?;
}
}
}
remove()
}
}
/// FsStore's control state is outside all per-Session source directories.
pub(crate) fn fs_retention(root: &Path) -> SessionRetentionStore {
SessionRetentionStore::new(root.join(".retention"))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn publication_intent_survives_reopen_without_attesting_missing_originals() {
let temp = tempfile::tempdir().unwrap();
let source = temp.path().join("source");
let controls = temp.path().join("controls");
let session = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let retention = SessionRetentionStore::new(&controls);
retention.retain_before_publication(session, pin).unwrap();
let reopened = SessionRetentionStore::new(&controls);
assert_eq!(reopened.pins(session).unwrap(), [pin]);
// An early owner is not a successful source-availability receipt.
assert!(
matches!(reopened.pin(session, pin, || Ok(source.is_file())), Err(StoreError::SessionSourceMissing(id)) if id == session)
);
{
let _publication = reopened.lock_live(session).unwrap();
fs::write(&source, "committed before notification").unwrap();
}
assert!(
matches!(reopened.delete(session, SessionDeletionIntent::Ordinary, || { fs::remove_file(&source)?; Ok(()) }), Err(StoreError::SessionPinned(id)) if id == session)
);
assert_eq!(
fs::read_to_string(&source).unwrap(),
"committed before notification"
);
reopened
.delete(session, SessionDeletionIntent::ErasePermanently, || {
fs::remove_file(&source)?;
Ok(())
})
.unwrap();
assert!(
matches!(retention.retain_before_publication(session, pin), Err(StoreError::SessionErased(id)) if id == session)
);
}
#[test]
fn pin_survives_reopen_blocks_delete_and_releases_idempotently() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let store = SessionRetentionStore::new(temp.path());
store.pin(id, pin, || Ok(true)).unwrap();
store.pin(id, pin, || Ok(true)).unwrap();
let reopened = SessionRetentionStore::new(temp.path());
assert_eq!(reopened.pins(id).unwrap(), [pin]);
assert!(matches!(
reopened.delete(id, SessionDeletionIntent::Ordinary, || Ok(())),
Err(StoreError::SessionPinned(_))
));
reopened.release(id, pin).unwrap();
reopened.release(id, pin).unwrap();
assert_eq!(
reopened
.delete(id, SessionDeletionIntent::Ordinary, || Ok(7))
.unwrap(),
7
);
}
#[test]
fn erase_failure_persists_fence_invalidates_pin_and_rejects_late_publication() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let store = SessionRetentionStore::new(temp.path());
store.pin(id, pin, || Ok(true)).unwrap();
assert!(
store
.delete(
id,
SessionDeletionIntent::ErasePermanently,
|| Err::<(), _>(std::io::Error::other("injected removal failure").into())
)
.is_err()
);
let reopened = SessionRetentionStore::new(temp.path());
assert!(reopened.pins(id).unwrap().is_empty());
assert!(matches!(
reopened.pin(id, pin, || Ok(true)),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.lock_live(id),
Err(StoreError::SessionErased(_))
));
reopened
.delete(id, SessionDeletionIntent::ErasePermanently, || Ok(()))
.unwrap();
reopened.release(id, pin).unwrap();
assert!(matches!(
reopened.lock_live(id),
Err(StoreError::SessionErased(_))
));
}
#[test]
fn fs_normal_delete_obeys_pin_and_permanent_erase_fences_all_source_writes() {
use crate::{FsStore, LogEntry, Store};
let temp = tempfile::tempdir().unwrap();
let store = FsStore::new(temp.path()).unwrap();
let session = crate::new_session_id();
let segment = crate::new_segment_id();
let entry = LogEntry::TurnEnd {
ts: 1,
turn_count: 1,
};
store
.create_segment(session, segment, &[entry.clone()])
.unwrap();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
store.pin_session(session, pin).unwrap();
assert!(matches!(
store.delete_session(session),
Err(StoreError::SessionPinned(_))
));
assert!(store.exists(session, segment).unwrap());
store
.delete_session_with_intent(session, SessionDeletionIntent::ErasePermanently)
.unwrap();
let reopened = FsStore::new(temp.path()).unwrap();
assert!(!reopened.exists(session, segment).unwrap());
assert!(matches!(
reopened.pin_session(session, pin),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.append(session, segment, &entry),
Err(StoreError::SessionErased(_))
));
assert!(matches!(
reopened.create_segment(session, segment, &[entry]),
Err(StoreError::SessionErased(_))
));
assert!(
!reopened
.delete_session_with_intent(session, SessionDeletionIntent::ErasePermanently)
.unwrap()
);
assert!(!reopened.exists(session, segment).unwrap());
}
#[test]
fn independent_handles_serialize_delete_with_late_pin() {
let temp = tempfile::tempdir().unwrap();
let source = temp.path().join("source");
fs::create_dir(&source).unwrap();
let control = temp.path().join("control");
let session = crate::new_session_id();
let pin = SessionRetentionPin(uuid::Uuid::now_v7());
let (entered_tx, entered_rx) = std::sync::mpsc::channel();
let (release_tx, release_rx) = std::sync::mpsc::channel();
let control2 = control.clone();
let source2 = source.clone();
let delete = std::thread::spawn(move || {
SessionRetentionStore::new(control2).delete(
session,
SessionDeletionIntent::Ordinary,
|| {
entered_tx.send(()).unwrap();
release_rx
.recv_timeout(std::time::Duration::from_secs(10))
.unwrap();
fs::remove_dir(&source2)?;
Ok(())
},
)
});
entered_rx
.recv_timeout(std::time::Duration::from_secs(10))
.unwrap();
let late = std::thread::spawn(move || {
SessionRetentionStore::new(control).pin(session, pin, || Ok(source.is_dir()))
});
release_tx.send(()).unwrap();
delete.join().unwrap().unwrap();
assert!(matches!(
late.join().unwrap(),
Err(StoreError::SessionSourceMissing(_))
));
}
#[test]
fn missing_source_cannot_acquire_pin() {
let temp = tempfile::tempdir().unwrap();
let id = crate::new_session_id();
let store = SessionRetentionStore::new(temp.path());
assert!(matches!(
store.pin(id, SessionRetentionPin(uuid::Uuid::now_v7()), || Ok(false)),
Err(StoreError::SessionSourceMissing(_))
));
assert!(store.pins(id).unwrap().is_empty());
}
}
+23
View File
@@ -33,6 +33,18 @@ pub enum StoreError {
#[error("log corrupted at line {line}: {message}")]
Corrupt { line: usize, message: String },
#[error("session source is missing: {0}")]
SessionSourceMissing(SessionId),
#[error("session original material is retained by a durable consumer: {0}")]
SessionPinned(SessionId),
#[error("session was permanently erased: {0}")]
SessionErased(SessionId),
#[error("session source retention is unsupported by this store")]
SessionRetentionUnsupported,
#[error("retained observation read limit exceeded")]
ReadLimitExceeded,
@@ -80,6 +92,17 @@ pub enum StoreError {
/// take `(SessionId, SegmentId)` so segments can be physically grouped
/// per Session on disk (or per session_id in a DB).
pub trait Store: Send + Sync {
/// Establish durable retention intent before the Host accepts public input.
/// This does not attest source availability; it must use the same control store
/// as publishers and physical deletion. Unsupported stores fail closed.
fn retain_session_before_publication(
&self,
_session_id: SessionId,
_pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
Err(StoreError::SessionRetentionUnsupported)
}
/// Append a single log entry to the segment log.
///
/// One committed line per successful call. Implementations must not expose
+10
View File
@@ -92,6 +92,14 @@ pub struct PromptRenderProvenance {
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum SystemItem {
/// One-time derived Journal reference context. The bounded Host snapshot is
/// retained as JSON independently of its rendered body and prompt projection.
JournalStartup {
body: String,
snapshot: serde_json::Value,
snapshot_sha256: String,
prompt_provenance: PromptRenderProvenance,
},
/// Free-form notification sent in by an external caller via
/// `Method::Notify`. `message` is the raw caller-supplied text;
/// `body` is the wrapped LLM-context form (Worker renders it via
@@ -194,6 +202,7 @@ impl SystemItem {
/// for this item. Returns the variant's stored `body` verbatim.
pub fn history_text(&self) -> String {
match self {
SystemItem::JournalStartup { body, .. } => body.clone(),
SystemItem::Notification { body, .. } => body.clone(),
SystemItem::WorkerEvent { body, .. } => body.clone(),
SystemItem::FeatureInvocationResult { body, .. } => body.clone(),
@@ -220,6 +229,7 @@ impl SystemItem {
/// wire — keep flexible.
pub fn kind_label(&self) -> &'static str {
match self {
SystemItem::JournalStartup { .. } => "journal_startup",
SystemItem::Notification { .. } => "notification",
SystemItem::WorkerEvent { .. } => "worker_event",
SystemItem::FeatureInvocationResult { .. } => "feature_invocation_result",
@@ -806,6 +806,15 @@ where
S: crate::Store,
W: Send + Sync,
{
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), crate::StoreError> {
self.session_store
.retain_session_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
+245 -22
View File
@@ -48,6 +48,7 @@ pub struct WorkerSessionStore {
root: PathBuf,
session_id: Arc<Mutex<Option<SessionId>>>,
append_lock: Arc<Mutex<()>>,
retention: Option<crate::retention::SessionRetentionStore>,
}
#[derive(Debug, Serialize, Deserialize)]
@@ -72,6 +73,9 @@ pub(crate) struct RetainedSegmentReader {
read_end: u64,
buffer_start: u64,
buffer: Vec<u8>,
// Newest-to-oldest fragments of a record spanning multiple read chunks.
// Keep them separate until its boundary is found to avoid repeated prepend copies.
record_suffix: Vec<Vec<u8>>,
}
impl RetainedSegmentReader {
@@ -82,13 +86,19 @@ impl RetainedSegmentReader {
let mut read_bytes = 0_u64;
loop {
if !self.buffer.is_empty() {
if self.buffer.last() != Some(&b'\n') {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment cursor is not at a record boundary".to_string(),
});
}
let prior_newline = self.buffer[..self.buffer.len() - 1]
let search_end = if self.record_suffix.is_empty() {
if self.buffer.last() != Some(&b'\n') {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment cursor is not at a record boundary"
.to_string(),
});
}
self.buffer.len() - 1
} else {
self.buffer.len()
};
let prior_newline = self.buffer[..search_end]
.iter()
.rposition(|byte| *byte == b'\n');
if let Some(newline) =
@@ -100,9 +110,13 @@ impl RetainedSegmentReader {
newline + 1
};
let record_start = self.buffer_start + record_start_in_buffer as u64;
let record_end = self.buffer_start + self.buffer.len() as u64;
let record =
self.buffer[record_start_in_buffer..self.buffer.len() - 1].to_vec();
let record_end = self.read_end;
let mut record = Vec::with_capacity((record_end - record_start) as usize);
record.extend_from_slice(&self.buffer[record_start_in_buffer..]);
for fragment in self.record_suffix.drain(..).rev() {
record.extend_from_slice(&fragment);
}
record.pop(); // The final newline was checked before collecting fragments.
self.buffer.truncate(record_start_in_buffer);
self.read_end = record_start;
if record.is_empty() {
@@ -126,24 +140,17 @@ impl RetainedSegmentReader {
read_bytes,
));
}
// This chunk has no earlier boundary. Never scan or copy it again
// until the complete record can be assembled once.
self.record_suffix.push(std::mem::take(&mut self.buffer));
} else if self.read_end == 0 {
return Ok((None, read_bytes));
}
if self.buffer_start == 0 && !self.buffer.is_empty() {
return Err(StoreError::Corrupt {
line: 0,
message: "retained Segment log does not end at a record boundary".to_string(),
});
}
if read_bytes >= max_additional_bytes {
return Err(StoreError::ReadLimitExceeded);
}
let end = if self.buffer.is_empty() {
self.read_end
} else {
self.buffer_start
};
let end = self.buffer_start;
let available = max_additional_bytes - read_bytes;
let chunk_len = end
.min(RETAINED_BACKWARD_READ_CHUNK_BYTES as u64)
@@ -156,7 +163,6 @@ impl RetainedSegmentReader {
let mut chunk = vec![0_u8; chunk_len as usize];
self.file.read_exact(&mut chunk)?;
read_bytes += chunk_len;
chunk.extend_from_slice(&self.buffer);
self.buffer = chunk;
self.buffer_start = start;
}
@@ -164,6 +170,37 @@ impl RetainedSegmentReader {
}
impl WorkerSessionStore {
/// Publishes through the same external control store used by Runtime retention.
/// Existing manifest migration is fenced too; future writes keep the guard until their
/// physical publication finishes. The control root must outlive this aggregate.
pub fn new_with_retention(
root: impl Into<PathBuf>,
retention: crate::retention::SessionRetentionStore,
) -> Result<Self, StoreError> {
let root = root.into();
let existing = match fs::read(root.join(SESSION_FILE)) {
Ok(bytes) => Some(serde_json::from_slice::<SessionManifest>(&bytes)?.session_id),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => return Err(error.into()),
};
let _guard = existing
.map(|session| retention.lock_live(session))
.transpose()?;
let mut store = Self::new(root)?;
store.retention = Some(retention);
Ok(store)
}
fn lock_publication(
&self,
session: SessionId,
) -> Result<Option<crate::retention::SessionRetentionGuard>, StoreError> {
self.retention
.as_ref()
.map(|store| store.lock_live(session))
.transpose()
}
/// Open the Session store rooted at `<worker-aggregate>/session`.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, StoreError> {
let root = root.into();
@@ -202,6 +239,7 @@ impl WorkerSessionStore {
root,
session_id: Arc::new(Mutex::new(session_id)),
append_lock: Arc::new(Mutex::new(())),
retention: None,
})
}
@@ -224,6 +262,7 @@ impl WorkerSessionStore {
root,
session_id: Arc::new(Mutex::new(Some(manifest.session_id))),
append_lock: Arc::new(Mutex::new(())),
retention: None,
})
}
@@ -456,6 +495,7 @@ impl WorkerSessionStore {
read_end: before_offset,
buffer_start: before_offset,
buffer: Vec::new(),
record_suffix: Vec::new(),
})
}
@@ -682,12 +722,24 @@ impl WorkerSessionStore {
}
impl Store for WorkerSessionStore {
fn retain_session_before_publication(
&self,
session_id: SessionId,
pin: crate::retention::SessionRetentionPin,
) -> Result<(), StoreError> {
self.retention
.as_ref()
.ok_or(StoreError::SessionRetentionUnsupported)?
.retain_before_publication(session_id, pin)
}
fn append(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_log_entry(&self.log_path(segment_id), entry)
}
@@ -710,6 +762,11 @@ impl Store for WorkerSessionStore {
}
fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError> {
// A pending Worker can have metadata before its atomic initial seed.
// Absence is queryable without materializing or binding this store.
if self.session_id()?.is_none() {
return Ok(Vec::new());
}
self.ensure_session(session_id, false)?;
let mut segments: Vec<SegmentId> = Vec::new();
for entry in fs::read_dir(self.root.join(SEGMENTS_DIR))? {
@@ -740,6 +797,7 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let mut content = Vec::new();
for entry in entries {
@@ -751,6 +809,9 @@ impl Store for WorkerSessionStore {
}
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
if self.session_id()?.is_none() {
return Ok(false);
}
self.ensure_session(session_id, false)?;
Ok(self.log_path(segment_id).exists())
}
@@ -788,6 +849,7 @@ impl Store for WorkerSessionStore {
content: &str,
limits: PasteArtifactLimits,
) -> Result<PasteArtifactRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -818,6 +880,7 @@ impl Store for WorkerSessionStore {
content: &[u8],
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -842,6 +905,7 @@ impl Store for WorkerSessionStore {
context: &UploadedFileUploadContext,
limits: UploadedFileLimits,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
let _guard = self
.append_lock
@@ -881,6 +945,7 @@ impl Store for WorkerSessionStore {
reference: &UploadedFileRef,
source_entry_id: &str,
) -> Result<UploadedFileRef, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -909,6 +974,7 @@ impl Store for WorkerSessionStore {
reference: &UploadedFileRef,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -923,6 +989,7 @@ impl Store for WorkerSessionStore {
artifact_id: &str,
owner_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -937,6 +1004,7 @@ impl Store for WorkerSessionStore {
artifact_id: &str,
source_entry_id: &str,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -954,6 +1022,7 @@ impl Store for WorkerSessionStore {
session_id: SessionId,
live_owner_ids: &[String],
) -> Result<u64, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -967,6 +1036,7 @@ impl Store for WorkerSessionStore {
session_id: SessionId,
artifact_id: &str,
) -> Result<bool, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -976,6 +1046,7 @@ impl Store for WorkerSessionStore {
}
fn delete_uncommitted_uploaded_files(&self, session_id: SessionId) -> Result<u64, StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, false)?;
let _guard = self
.append_lock
@@ -1010,6 +1081,7 @@ impl Store for WorkerSessionStore {
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError> {
let _retention = self.lock_publication(session_id)?;
self.ensure_session(session_id, true)?;
self.append_line(&self.trace_path(segment_id), &serde_json::to_string(entry)?)
}
@@ -1315,6 +1387,139 @@ mod tests {
}
}
fn retained_reader(bytes: &[u8]) -> RetainedSegmentReader {
let mut file = tempfile::tempfile().unwrap();
file.write_all(bytes).unwrap();
RetainedSegmentReader {
file,
session_id: new_session_id(),
segment_id: new_segment_id(),
read_end: bytes.len() as u64,
buffer_start: bytes.len() as u64,
buffer: Vec::new(),
record_suffix: Vec::new(),
}
}
fn checkpoint_line(text: String) -> Vec<u8> {
let mut bytes = serde_json::to_vec(&LogEntry::InputSegmentsCheckpoint {
ts: 1,
user_segments: vec![vec![protocol::Segment::Text { content: text }]],
})
.unwrap();
bytes.push(b'\n');
bytes
}
#[test]
fn retained_reader_preserves_records_and_offsets_across_chunk_boundaries() {
let chunk = RETAINED_BACKWARD_READ_CHUNK_BYTES;
let overhead = checkpoint_line(String::new()).len();
for line_size in [chunk - 1, chunk, chunk + 1, chunk * 3] {
let lines = [
checkpoint_line("最初\nの行".into()),
checkpoint_line("x".repeat(line_size - overhead)),
checkpoint_line("最後".into()),
];
let bytes = lines.concat();
let mut reader = retained_reader(&bytes);
let mut end = bytes.len() as u64;
let mut total_read = 0;
for line in lines.iter().rev() {
let (record, read) = reader.previous_record(u64::MAX).unwrap();
total_read += read;
let record = record.unwrap();
assert_eq!(record.end_offset, end);
end -= line.len() as u64;
assert_eq!(record.start_offset, end);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
line[..line.len() - 1]
);
}
assert_eq!(total_read, bytes.len() as u64);
let (record, read) = reader.previous_record(0).unwrap();
assert!(record.is_none());
assert_eq!(read, 0);
}
}
#[test]
fn retained_reader_resumes_after_budget_exhaustion_and_serves_buffered_records() {
let first = checkpoint_line("first".into());
let last = checkpoint_line("大きな行".repeat(RETAINED_BACKWARD_READ_CHUNK_BYTES));
let mut reader = retained_reader(&[first.clone(), last.clone()].concat());
let budget = RETAINED_BACKWARD_READ_CHUNK_BYTES as u64;
assert!(matches!(
reader.previous_record(budget),
Err(StoreError::ReadLimitExceeded)
));
assert!(matches!(
reader.previous_record(0),
Err(StoreError::ReadLimitExceeded)
));
let (record, read) = reader.previous_record(u64::MAX).unwrap();
let record = record.unwrap();
assert_eq!(record.start_offset, first.len() as u64);
assert_eq!(record.end_offset, (first.len() + last.len()) as u64);
assert_eq!(read + budget, record.end_offset);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
last[..last.len() - 1]
);
let (record, read) = reader.previous_record(0).unwrap();
assert_eq!(record.unwrap().start_offset, 0);
assert_eq!(read, 0);
assert!(reader.previous_record(0).unwrap().0.is_none());
}
#[test]
fn retained_reader_requires_the_prior_boundary_byte_at_exact_chunk_edges() {
let chunk = RETAINED_BACKWARD_READ_CHUNK_BYTES;
let overhead = checkpoint_line(String::new()).len();
for size in [chunk - 1, chunk, chunk + 1] {
let first = checkpoint_line("first".into());
let last = checkpoint_line("x".repeat(size - overhead));
let bytes = [first.clone(), last.clone()].concat();
let mut reader = retained_reader(&bytes);
// The final record alone is insufficient: its preceding newline
// must also be read before its start can be established.
assert!(matches!(
reader.previous_record(size as u64),
Err(StoreError::ReadLimitExceeded)
));
let (record, read) = reader.previous_record(1).unwrap();
let record = record.unwrap();
assert_eq!(read, 1);
assert_eq!(record.start_offset, first.len() as u64);
assert_eq!(record.end_offset, bytes.len() as u64);
assert_eq!(
serde_json::to_vec(&record.entry).unwrap(),
last[..last.len() - 1]
);
let (record, read) = reader.previous_record(u64::MAX).unwrap();
assert_eq!(record.unwrap().start_offset, 0);
assert_eq!(read, (first.len() - 1) as u64);
}
}
#[test]
fn retained_reader_rejects_empty_malformed_and_unterminated_records() {
for bytes in [
b"\n".as_slice(),
b"{bad json}\n",
b"{\"kind\":\"input_segments_checkpoint\"}",
] {
assert!(retained_reader(bytes).previous_record(u64::MAX).is_err());
}
let mut bytes = checkpoint_line("valid".into());
bytes.push(b'\n');
assert!(matches!(
retained_reader(&bytes).previous_record(u64::MAX),
Err(StoreError::Corrupt { .. })
));
}
#[test]
fn canonical_combined_store_supports_attachment_uploads() {
let root = tempfile::tempdir().unwrap();
@@ -1906,6 +2111,24 @@ mod tests {
assert!(matches!(error, StoreError::Corrupt { .. }));
}
#[test]
fn pending_session_probes_are_empty_without_binding_and_foreign_materialized_session_is_rejected()
{
let root = tempfile::tempdir().unwrap();
let session_id = new_session_id();
let segment_id = new_segment_id();
let store = WorkerSessionStore::new(root.path()).unwrap();
assert!(!store.exists(session_id, segment_id).unwrap());
assert!(store.list_segments(session_id).unwrap().is_empty());
assert_eq!(store.session_id().unwrap(), None);
let reopened = WorkerSessionStore::new(root.path()).unwrap();
assert_eq!(reopened.session_id().unwrap(), None);
store.create_segment(session_id, segment_id, &[]).unwrap();
let foreign = new_session_id();
assert!(store.exists(foreign, segment_id).is_err());
assert!(store.list_segments(foreign).is_err());
}
#[test]
fn reopen_preserves_session_and_segment_ids() {
let root = tempfile::tempdir().unwrap();
+208 -17
View File
@@ -57,6 +57,15 @@ pub struct StandaloneHost {
lease: Option<StandaloneWorkerLease>,
jobs: crate::jobs::StandaloneJobs,
subject: Option<crate::subjektiv::SubjectConnection>,
subject_release: Option<worker::SubjectReleaseReservation>,
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum StandaloneSubjectDisconnectError {
#[error("Subject release requires an actual Idle Controller without pending work: {0}")]
Controller(String),
#[error("Subject release remains incomplete; retained binding/lease must be reconciled: {0}")]
Incomplete(String),
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
@@ -89,6 +98,10 @@ pub enum StandaloneStartupError {
FeatureComposition,
#[error("the in-process Worker controller could not start")]
Controller,
#[error(
"mandatory initial context remains unconfirmed; restore retained Worker {worker_id} to retry this Session"
)]
InitialContextPending { worker_id: WorkerId },
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
@@ -171,7 +184,7 @@ impl StandaloneHost {
let jobs = match crate::jobs::StandaloneJobs::open(&jobs_path, launch.cwd.clone()) {
Ok(jobs) => jobs,
Err(_) => {
let _ = store.abandon_allocation(allocation);
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(StandaloneStartupError::StateStore);
}
};
@@ -179,8 +192,11 @@ impl StandaloneHost {
if let Some(client) = &model_client {
jobs.bind_model_client(client.clone_boxed());
}
if let Some(subject) = &subject {
subject.host.bind_jobs(jobs.clone())?;
if let Some(connection) = &subject
&& let Err(error) = connection.host.bind_jobs(jobs.clone())
{
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(error);
}
// WorkerId is the stable identity. The current Worker store remains
// name-keyed, so keep its derived storage key separate from the
@@ -192,7 +208,7 @@ impl StandaloneHost {
let (backing_store, worker_store) = match backing_store(&store, worker_id) {
Ok(stores) => stores,
Err(error) => {
let _ = store.abandon_allocation(allocation);
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(error);
}
};
@@ -220,12 +236,61 @@ impl StandaloneHost {
if let Some(model_client) = model_client {
bootstrap = bootstrap.with_model_client(model_client);
}
let prepared = match bootstrap.prepare().await {
Ok(prepared) => prepared,
Err(error) => {
cancel_unpublished_start(&store, &mut subject, allocation)?;
return Err(classify_startup_error(error));
}
};
// Reserve a recoverable identity before mandatory initial-context persistence.
// A failed/unknown seed must be reopened as this Session, never abandoned and
// replaced by a newly fetched context under an invisible ghost binding.
let pending_record = if subject.is_some() {
match store.commit_created_connected(
&allocation,
manifest.clone(),
storage_key.clone(),
prepared.worker().session_id(),
None,
subject.as_ref().map(|s| s.binding.clone()),
) {
Ok(record) => Some(record),
Err(_) => {
if let Some(subject) = &mut subject {
subject.retain();
}
allocation.into_lease().retain();
return Err(StandaloneStartupError::StateStore);
}
}
} else {
None
};
if let Some(subject) = &mut subject {
subject.arm();
}
let started = match bootstrap.start().await {
let started = match prepared.start().await {
Ok(started) => started,
Err(error) => {
if let Some(record) = &pending_record {
let cleanup = jobs.shutdown().await;
let saved = store.mark_stopped(
record,
record.active_session_id,
record.active_segment_id,
StandaloneShutdownReason::StartupFailed,
);
let closed = cleanup.is_ok()
&& saved.is_ok()
&& subject.as_mut().is_some_and(|s| s.close().is_ok());
if closed {
let _ = allocation.into_lease().release();
} else {
allocation.into_lease().retain();
}
return Err(StandaloneStartupError::InitialContextPending { worker_id });
}
let _ = store.abandon_allocation(allocation);
return Err(classify_startup_error(error));
}
@@ -234,22 +299,35 @@ impl StandaloneHost {
Ok(active) => active,
Err(error) => {
stop_started_worker(started).await;
let _ = store.abandon_allocation(allocation);
if pending_record.is_some() {
allocation.into_lease().retain();
} else {
let _ = store.abandon_allocation(allocation);
}
return Err(error);
}
};
let record = match store.commit_created_connected(
&allocation,
manifest,
storage_key,
active.session_id,
active.segment_id,
subject.as_ref().map(|s| s.binding.clone()),
) {
let committed = if let Some(record) = &pending_record {
store.update_active_pointer(record, active.session_id, active.segment_id)
} else {
store.commit_created_connected(
&allocation,
manifest,
storage_key,
active.session_id,
active.segment_id,
None,
)
};
let record = match committed {
Ok(record) => record,
Err(_) => {
stop_started_worker(started).await;
let _ = store.abandon_allocation(allocation);
if pending_record.is_some() {
allocation.into_lease().retain();
} else {
let _ = store.abandon_allocation(allocation);
}
return Err(StandaloneStartupError::StateStore);
}
};
@@ -427,6 +505,7 @@ impl StandaloneHost {
lease: Some(lease),
jobs,
subject,
subject_release: None,
}
}
@@ -457,6 +536,45 @@ impl StandaloneHost {
Client::new(socket)
}
/// Logical Subject disconnection, not Worker shutdown or completion of its represented work.
/// A successful release keeps the old Controller fenced; a fresh Worker/Session may then
/// acquire the Subject. Failures before durable revocation allow normal recovery and retry.
pub async fn disconnect_subject(&mut self) -> Result<(), StandaloneSubjectDisconnectError> {
let Some(subject) = &mut self.subject else {
return Ok(());
};
let mut reservation = match self.subject_release.take() {
Some(reservation) => reservation,
None => self
.handle
.reserve_subject_release()
.await
.map_err(StandaloneSubjectDisconnectError::Controller)?,
};
let release = match subject.release_for_handoff(&self.record, &mut reservation) {
Ok(()) => match self.jobs.shutdown().await {
Ok(()) => subject.close().map_err(|error| error.to_string()),
Err(error) => Err(error.to_string()),
},
Err(error) => Err(error),
};
match release {
Ok(()) => {
self.subject_release = Some(reservation);
self.subject = None;
Ok(())
}
Err(error) => {
// If revocation committed, keep the actual Controller reserved even if closing
// storage/unlocking failed. Dropping a reservation must not undo revocation.
if subject.writer_released_or_unknown() {
self.subject_release = Some(reservation);
}
Err(StandaloneSubjectDisconnectError::Incomplete(error))
}
}
}
pub fn with_shutdown_timeout(mut self, shutdown_timeout: Duration) -> Self {
self.shutdown_timeout = shutdown_timeout;
self
@@ -619,6 +737,11 @@ async fn run_protocol_session(
event = events.recv() => {
match event {
Ok(event) => {
let Ok(event) = handle.alerter.freshen_event(event) else {
// The retained anchor was evicted; reconnect must obtain
// a paired session/Alert snapshot rather than a stale row.
return;
};
if !send_protocol_event(&peer, event).await {
return;
}
@@ -690,12 +813,32 @@ async fn send_protocol_event(peer: &InProcessPeer, event: Event) -> bool {
peer.send(message).await.is_ok()
}
fn cancel_unpublished_start(
store: &StandaloneWorkerStore,
subject: &mut Option<crate::subjektiv::SubjectConnection>,
allocation: crate::store::StandaloneWorkerAllocation,
) -> Result<(), StandaloneStartupError> {
if let Some(connection) = subject {
if connection.abort_unpublished_start().is_err() {
connection.retain();
allocation.into_lease().retain();
return Err(StandaloneStartupError::StateStore);
}
}
store
.abandon_allocation(allocation)
.map_err(classify_store_startup_error)
}
fn backing_store(
store: &StandaloneWorkerStore,
worker_id: WorkerId,
) -> Result<(StandaloneBackingStore, FsWorkerStore), StandaloneStartupError> {
let session_store = FsStore::new(store.sessions_dir(worker_id))
.map_err(|_| StandaloneStartupError::StateStore)?;
let session_store = FsStore::new_with_retention(
store.sessions_dir(worker_id),
crate::store::session_retention(store.root()),
)
.map_err(|_| StandaloneStartupError::StateStore)?;
let worker_store = FsWorkerStore::new(store.worker_metadata_dir(worker_id))
.map_err(|_| StandaloneStartupError::StateStore)?;
Ok((
@@ -808,3 +951,51 @@ mod local_input_tests {
}
}
}
#[cfg(test)]
mod source_retention_tests {
//! Host backing-store assembly uses the production publisher and an isolated external
//! erasure fence. No controller/model is substituted for the source authority under test.
use super::*;
use session_store::{LogEntry, Store, StoreError};
#[test]
fn backing_publisher_and_reopen_use_the_host_wide_erasure_fence() {
let temp = tempfile::tempdir().unwrap();
let workers = StandaloneWorkerStore::open(temp.path().join("state")).unwrap();
let allocation = workers
.allocate(temp.path(), StaleLeasePolicy::Reject)
.unwrap();
let worker = allocation.worker_id();
let (backing, _) = backing_store(&workers, worker).unwrap();
let session = session_store::SessionId::now_v7();
let segment = uuid::Uuid::now_v7();
let entry = LogEntry::TurnEnd {
ts: 1,
turn_count: 1,
};
backing
.create_segment(session, segment, &[entry.clone()])
.unwrap();
let retention = crate::store::session_retention(workers.root());
assert!(
retention
.delete(
session,
session_store::retention::SessionDeletionIntent::ErasePermanently,
|| Err::<(), StoreError>(
std::io::Error::other("injected physical erase failure").into()
)
)
.is_err()
);
assert!(
matches!(backing.append(session, segment, &entry), Err(StoreError::SessionErased(id)) if id == session)
);
let (reopened, _) = backing_store(&workers, worker).unwrap();
assert!(
matches!(reopened.append(session, segment, &entry), Err(StoreError::SessionErased(id)) if id == session)
);
assert!(!workers.sessions_dir(worker).join(".retention").exists());
}
}
+172 -3
View File
@@ -25,7 +25,7 @@ use thiserror::Error;
#[path = "job_store/legacy_migrations.rs"]
mod legacy_migrations;
const SCHEMA_VERSION: i64 = 3;
const SCHEMA_VERSION: i64 = 4;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
@@ -189,6 +189,7 @@ impl JobStore {
0 => {
tx.execute_batch(SCHEMA)?;
tx.execute_batch(GRANT_SCHEMA)?;
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
1 | 2 => {
@@ -196,6 +197,11 @@ impl JobStore {
tx.execute_batch(GRANT_SCHEMA)?;
}
legacy_migrations::migrate_legacy_jobs(&tx)?;
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
3 => {
tx.execute_batch(RECONCILIATION_SCHEMA)?;
tx.pragma_update(None, "user_version", SCHEMA_VERSION)?;
}
SCHEMA_VERSION => {}
@@ -278,6 +284,49 @@ impl JobStore {
Ok(snapshot)
}
/// Domain Host attests that this fenced attempt's only effects are reconciled in its
/// authoritative durable consumer. This releases serialization, not the original Unknown
/// history, and cannot authorize retry or change its result. The service checks cleanup.
pub(crate) fn reconcile_granted_attempt(
&mut self,
job_id: &str,
attempt_id: &str,
domain_receipt: &str,
) -> Result<(), JobStoreError> {
if domain_receipt.is_empty()
|| domain_receipt.len() > 256
|| domain_receipt.chars().any(char::is_control)
{
return Err(JobStoreError::InvalidTransition(
"invalid domain reconciliation receipt",
));
}
let tx = self
.conn
.transaction_with_behavior(TransactionBehavior::Immediate)?;
let snapshot = require_snapshot(&tx, job_id)?;
check_attempt(&snapshot, attempt_id)?;
if snapshot.domain_grant.is_none()
|| snapshot.attempt.state != JobAttemptState::Unknown
|| !matches!(snapshot.state, JobState::Unknown | JobState::Cancelled)
{
return Err(JobStoreError::InvalidTransition(
"domain reconciliation requires a fenced unknown granted attempt",
));
}
let old: Option<String> = tx.query_row("SELECT domain_receipt FROM job_domain_reconciliations WHERE job_id=?1 AND attempt_id=?2",
params![job_id,attempt_id], |row| row.get(0)).optional()?;
if old.as_deref().is_some_and(|old| old != domain_receipt) {
return Err(JobStoreError::InvalidTransition(
"domain reconciliation receipt changed",
));
}
tx.execute("INSERT OR IGNORE INTO job_domain_reconciliations(job_id,attempt_id,domain_receipt) VALUES(?1,?2,?3)",
params![job_id,attempt_id,domain_receipt])?;
tx.commit()?;
Ok(())
}
/// Only definitive failure can create a new immutable attempt. Unknown
/// outcomes require caller reconciliation, never an automatic replay.
pub(crate) fn retry(&mut self, job_id: &str) -> Result<JobSnapshot, JobStoreError> {
@@ -319,6 +368,16 @@ impl JobStore {
.collect()
}
pub(crate) fn unknown_granted(&self) -> Result<Vec<JobSnapshot>, JobStoreError> {
let mut statement = self.conn.prepare("SELECT i.job_id FROM job_intents i JOIN job_attempts a ON a.job_id=i.job_id AND a.number=i.current_attempt JOIN job_domain_grants g ON g.job_id=i.job_id WHERE a.state='unknown' AND NOT EXISTS (SELECT 1 FROM job_domain_reconciliations r WHERE r.job_id=a.job_id AND r.attempt_id=a.attempt_id) ORDER BY i.rowid")?;
let ids = statement
.query_map([], |row| row.get::<_, String>(0))?
.collect::<rusqlite::Result<Vec<_>>>()?;
ids.into_iter()
.map(|id| require_snapshot(&self.conn, &id))
.collect()
}
/// Only unstarted current attempts are eligible for service startup/resume.
pub(crate) fn pending(&self) -> Result<Vec<JobSnapshot>, JobStoreError> {
let mut statement = self.conn.prepare(
@@ -527,8 +586,10 @@ fn check_serialization(
if let Some(key) = key {
let conflicting: bool = conn.query_row(
"SELECT EXISTS(SELECT 1 FROM job_intents i WHERE serialization_key = ?1
AND job_id <> ?2 AND (state IN ('pending', 'unknown') OR EXISTS
(SELECT 1 FROM job_attempts a WHERE a.job_id = i.job_id AND a.state = 'unknown')))",
AND job_id <> ?2 AND (state = 'pending' OR EXISTS
(SELECT 1 FROM job_attempts a WHERE a.job_id = i.job_id AND a.state = 'unknown'
AND NOT EXISTS (SELECT 1 FROM job_domain_reconciliations r
WHERE r.job_id=a.job_id AND r.attempt_id=a.attempt_id))))",
params![key, job_id],
|row| row.get(0),
)?;
@@ -658,6 +719,22 @@ fn read_snapshot(conn: &Connection, job_id: &str) -> Result<Option<JobSnapshot>,
.transpose()
}
const RECONCILIATION_SCHEMA: &str = "
DROP INDEX job_serialization_singleflight;
CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key)
WHERE serialization_key IS NOT NULL AND state = 'pending';
CREATE TABLE job_domain_reconciliations (
job_id TEXT NOT NULL REFERENCES job_intents(job_id),
attempt_id TEXT NOT NULL,
domain_receipt TEXT NOT NULL,
PRIMARY KEY (job_id, attempt_id)
);
CREATE TRIGGER job_reconciliation_immutable BEFORE UPDATE ON job_domain_reconciliations
BEGIN SELECT RAISE(ABORT, 'immutable domain reconciliation'); END;
CREATE TRIGGER job_reconciliation_retained BEFORE DELETE ON job_domain_reconciliations
BEGIN SELECT RAISE(ABORT, 'retained domain reconciliation'); END;
";
const GRANT_SCHEMA: &str = "
CREATE TABLE job_domain_grants (
job_id TEXT PRIMARY KEY NOT NULL REFERENCES job_intents(job_id),
@@ -732,6 +809,98 @@ mod tests {
}
}
#[test]
fn granted_unknown_reconciliation_releases_serialization_but_preserves_history() {
let (dir, mut store) = fixture();
let mut first = request("first");
first.serialization_key = Some("journal".into());
let reserved = store
.reserve_granted(first, Some(json!({"domain":"journal"})))
.unwrap();
store
.dispatch("first", &reserved.attempt.attempt_id, 8)
.unwrap();
let unknown = store
.finish(
"first",
&reserved.attempt.attempt_id,
JobAttemptState::Unknown,
Some("response lost"),
)
.unwrap();
let mut second = request("second");
second.serialization_key = Some("journal".into());
assert!(matches!(
store.reserve(second.clone()),
Err(JobStoreError::SerializationConflict(_))
));
assert!(
store
.reconcile_granted_attempt("first", "foreign-attempt", "domain-fenced")
.is_err()
);
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
assert!(
store
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "different-proof")
.is_err()
);
assert_eq!(store.get("first").unwrap(), unknown);
store.reserve(second).unwrap();
assert!(store.retry("first").is_err());
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(reopened.get("first").unwrap(), unknown);
assert!(reopened.unknown_granted().unwrap().is_empty());
assert!(
reopened
.conn
.execute("DELETE FROM job_domain_reconciliations", [])
.is_err()
);
assert!(reopened.retry("first").is_err());
}
#[test]
fn schema_three_migration_preserves_unknown_grant_and_its_serialization_fence() {
let (dir, mut store) = fixture();
let mut first = request("first");
first.serialization_key = Some("journal".into());
let reserved = store
.reserve_granted(first, Some(json!({"domain":"journal"})))
.unwrap();
store
.dispatch("first", &reserved.attempt.attempt_id, 8)
.unwrap();
let unknown = store
.finish(
"first",
&reserved.attempt.attempt_id,
JobAttemptState::Unknown,
Some("unconfirmed"),
)
.unwrap();
store.conn.execute_batch("DROP TRIGGER job_reconciliation_immutable; DROP TRIGGER job_reconciliation_retained; DROP TABLE job_domain_reconciliations; DROP INDEX job_serialization_singleflight; CREATE UNIQUE INDEX job_serialization_singleflight ON job_intents(serialization_key) WHERE serialization_key IS NOT NULL AND state IN ('pending','unknown'); PRAGMA user_version=3;").unwrap();
drop(store);
let mut reopened = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
assert_eq!(reopened.get("first").unwrap(), unknown);
let mut next = request("next");
next.serialization_key = Some("journal".into());
assert!(matches!(
reopened.reserve(next.clone()),
Err(JobStoreError::SerializationConflict(_))
));
reopened
.reconcile_granted_attempt("first", &unknown.attempt.attempt_id, "domain-fenced")
.unwrap();
reopened.reserve(next).unwrap();
}
fn fixture() -> (TempDir, JobStore) {
let dir = tempfile::tempdir().unwrap();
let store = JobStore::open(&dir.path().join("jobs.sqlite3")).unwrap();
+54
View File
@@ -66,6 +66,15 @@ struct Executions {
jobs: HashMap<String, Execution>,
}
pub(crate) struct QuiescentJobClosure<'a> {
executions: std::sync::MutexGuard<'a, Executions>,
}
impl QuiescentJobClosure<'_> {
pub(crate) fn commit(&mut self) {
self.executions.closed = true;
}
}
/// Cloneable Feature-facing access to a Host-owned service. Cloning this handle
/// neither extends the Host's authority nor permits opening/recovering its store.
#[derive(Clone)]
@@ -79,6 +88,27 @@ pub struct StandaloneJobs {
}
impl StandaloneJobs {
/// Excludes request/start/result acknowledgement while a Subject Host checks its final
/// saved obligations and releases ownership. Accepted results alone are insufficient:
/// every spawned execution must have actually finished successful cleanup.
pub(crate) fn reserve_quiescent_closure(
&self,
) -> Result<QuiescentJobClosure<'_>, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
if executions.jobs.values().any(|execution| {
!execution.task.is_finished()
|| !matches!(execution.complete.borrow().as_ref(), Some(Ok(())))
}) {
return Err(JobServiceError::Cleanup(
"delegated Job execution/cleanup remains running or unconfirmed".into(),
));
}
Ok(QuiescentJobClosure { executions })
}
pub(crate) fn open(path: &Path, cwd: PathBuf) -> Result<Self, JobServiceError> {
Ok(Self {
store: Arc::new(Mutex::new(Some(JobStore::open(path)?))),
@@ -157,6 +187,30 @@ impl StandaloneJobs {
self.with_store(|store| store.unacknowledged())
}
pub(crate) fn unknown_granted(&self) -> Result<Vec<JobSnapshot>, JobServiceError> {
self.with_store(|store| store.unknown_granted())
}
/// Release only an Unknown attempt's serialization reservation after the bound domain
/// has reconciled its receipts and durably fenced further effects. Never a generic retry.
pub(crate) fn reconcile_granted_attempt(
&self,
job_id: &str,
attempt_id: &str,
domain_receipt: &str,
) -> Result<(), JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
if executions.jobs.contains_key(job_id) {
return Err(JobServiceError::Cleanup(
"domain reconciliation requires observed execution cleanup".into(),
));
}
self.with_store(|store| store.reconcile_granted_attempt(job_id, attempt_id, domain_receipt))
}
/// Explicit new attempt, bounded by immutable intent limits. Unknown
/// outcomes cannot be retried without domain reconciliation/new intent.
pub fn retry(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
+4 -1
View File
@@ -8,7 +8,10 @@ pub mod store;
pub mod subjektiv;
mod subjektiv_sessions;
pub use host::{StandaloneHost, StandaloneShutdownError, StandaloneStartupError};
pub use host::{
StandaloneHost, StandaloneShutdownError, StandaloneStartupError,
StandaloneSubjectDisconnectError,
};
pub use launch::{ResolvedStandaloneLaunch, StandaloneLaunchConfig, StandaloneLaunchError};
pub use protocol::WorkerId;
pub use store::{
+26 -1
View File
@@ -119,6 +119,16 @@ pub enum StaleLeasePolicy {
Recover,
}
/// Source control outlives Worker relocation and physical Session pruning. Always derive this
/// from the Host state root, never from an individual live or retained Worker directory.
pub(crate) fn session_retention(
state_root: &Path,
) -> session_store::retention::SessionRetentionStore {
session_store::retention::SessionRetentionStore::new(
state_root.join("retention/session-sources"),
)
}
#[derive(Debug, Clone)]
pub struct StandaloneWorkerStore {
root: PathBuf,
@@ -359,7 +369,19 @@ impl StandaloneWorkerStore {
}
pub fn delete(&self, id: WorkerId) -> Result<(), StandaloneStoreError> {
let record = self.load(id)?;
let record = match self.load(id) {
Err(StandaloneStoreError::WorkerNotFound(_)) => {
let retained_root = self.root.join("subjektiv").join("retained-workers");
let retained =
Self::open_existing(retained_root).and_then(|store| store.load(id))?;
if retained.status == StandaloneWorkerStatus::Stopped && retained.subject.is_some()
{
return Ok(());
}
return Err(StandaloneStoreError::InvalidRecord(id));
}
result => result?,
};
if record.status != StandaloneWorkerStatus::Stopped {
return Err(StandaloneStoreError::DeleteActive(id));
}
@@ -374,6 +396,7 @@ impl StandaloneWorkerStore {
LeaseLiveness::Unknown => StandaloneStoreError::LeaseLivenessUnknown(id),
});
}
crate::subjektiv::prepare_worker_deletion(&self.root, &record)?;
fs::remove_dir_all(self.worker_dir(id)).map_err(StandaloneStoreError::Io)?;
sync_directory(&self.root)
}
@@ -793,6 +816,8 @@ pub enum StandaloneStoreError {
LeaseOwnershipLost,
#[error("standalone Worker {0} must be stopped before deletion")]
DeleteActive(WorkerId),
#[error("Journal deletion handoff/retention is unconfirmed: {0}")]
JournalRetention(String),
#[error("standalone Worker {0} metadata changed since it was read")]
MetadataChanged(WorkerId),
#[error("system clock is before the Unix epoch or out of range")]
+187 -10
View File
@@ -5,9 +5,11 @@
//! non-private nodes; delegates schema/migration to `subjektiv` and connection policy to `feature-storage`.
//!
//! The Host holds each selected Subject's exclusive kernel lease through body execution and
//! confirmed delegated Job cleanup. Process death releases kernel locks; uncertain in-process
//! cleanup retains leases. Restore validates saved Subject/storage bindings and reacquires the
//! same Subject. Launches without a selected Subject do not implicitly connect this store.
//! confirmed delegated Job cleanup. Kernel lock release never clears durable body ownership:
//! shutdown/process death permits only the same body to restore, not a replacement to start.
//! Explicit disconnection requires actual Controller quiescence and final Journal completion;
//! storage then revokes that writer lifetime before the lease is released. Uncertain cleanup
//! retains leases. Launches without a selected Subject do not implicitly connect this store.
//!
//! The selected Subject's Job store is shared across Workers; the generic Jobs service owns execution
//! and cleanup. Consolidation grants allow only immutable candidate batches and limited operations,
@@ -15,6 +17,9 @@
//! Unknown attempts are not rerun; delivery failures for accepted results do not trigger model
//! reexecution. Memory application and surface repair use shared-domain CAS/receipts.
mod journal;
pub(crate) use journal::prepare_worker_deletion;
use std::fs::{self, File, OpenOptions};
use std::io::Write;
use std::path::{Path, PathBuf};
@@ -46,6 +51,8 @@ pub enum SubjectError {
#[error(transparent)]
Domain(#[from] subjektiv::SubjektivError),
#[error(transparent)]
Journal(#[from] subjektiv::journal::JournalError),
#[error(transparent)]
Storage(#[from] feature_storage::FeatureStorageError),
#[error("local Subject storage metadata: {0}")]
Json(#[from] serde_json::Error),
@@ -220,7 +227,51 @@ impl StandaloneSubjects {
scope_id: self.scope_id.clone(),
subject_id: subject_id.to_string(),
};
let host = Arc::new(LocalSubjectHost {
if self.store.journal_worker_was_released(
subject_id,
&self.scope_id,
&worker_id.to_string(),
)? {
return Err(SubjectError::InvalidScope("this body's Subject binding was released; start a new Worker/Session rather than restore its old authority".into()));
}
// Before writer lifetimes existed, a saved body record was the only durable
// attachment evidence. Never interpret its stopped status as a release on upgrade.
let workers = crate::store::StandaloneWorkerStore::open_existing(&self.state_root)
.map_err(|error| SubjectError::InvalidScope(error.to_string()))?;
for entry in fs::read_dir(&self.state_root)? {
let entry = entry?;
let Ok(prior_id) = entry
.file_name()
.to_string_lossy()
.parse::<protocol::WorkerId>()
else {
continue;
};
if prior_id == worker_id {
continue;
}
let prior = match workers.load(prior_id) {
Ok(prior) => prior,
Err(crate::store::StandaloneStoreError::WorkerNotFound(_)) => continue,
Err(error) => return Err(SubjectError::InvalidScope(error.to_string())),
};
if prior.subject.as_ref() == Some(&binding)
&& !self.store.journal_worker_was_released(
subject_id,
&self.scope_id,
&prior_id.to_string(),
)?
{
return Err(SubjectError::InvalidScope("saved Subject body has no explicit release; restore it and complete the Idle/Journal handoff, rather than treating inactive storage as unoccupied".into()));
}
}
let writer =
self.store
.claim_journal_writer(subject_id, &self.scope_id, &worker_id.to_string())?;
let host = Arc::new_cyclic(|weak| LocalSubjectHost {
writer,
self_weak: weak.clone(),
journal_dispatch: Mutex::new(false),
language: self.memory_language.clone(),
catalog: self,
binding: binding.clone(),
@@ -247,6 +298,37 @@ pub(crate) struct SubjectConnection {
armed: bool,
}
impl SubjectConnection {
/// Construction failed before any Controller or mandatory startup context was published.
/// This is cancellation of an unpublished reservation, not Idle inferred from a stopped body.
pub(crate) fn abort_unpublished_start(&mut self) -> Result<(), SubjectError> {
self.host
.catalog
.store
.release_journal_writer(&self.host.writer)?;
self.close()
}
pub(crate) fn writer_released_or_unknown(&self) -> bool {
self.host
.catalog
.store
.journal_writer_active(&self.host.writer)
.map(|active| !active)
.unwrap_or(true)
}
pub(crate) fn release_for_handoff(
&mut self,
record: &crate::store::StandaloneWorkerRecord,
reservation: &mut worker::SubjectReleaseReservation,
) -> Result<(), String> {
if let Err(error) = self.host.release_for_handoff(record, reservation) {
self.host.wake_journals();
return Err(error.to_string());
}
reservation.commit_disconnect()
}
pub fn arm(&mut self) {
self.armed = true;
}
@@ -277,6 +359,9 @@ impl Drop for SubjectConnection {
}
pub(crate) struct LocalSubjectHost {
writer: subjektiv::journal::JournalWriterBinding,
self_weak: std::sync::Weak<Self>,
journal_dispatch: Mutex<bool>,
catalog: StandaloneSubjects,
binding: StandaloneSubjectBinding,
worker_id: protocol::WorkerId,
@@ -293,6 +378,16 @@ impl LocalSubjectHost {
"local Subject Host is closed".into(),
));
}
if !self
.catalog
.store
.journal_writer_active(&self.writer)
.map_err(journal::journal_error)?
{
return Err(SubjektivHostError::Unavailable(
"Subject writer binding was released".into(),
));
}
Ok(())
}
fn body(&self, context: &SubjektivHostContext) -> Result<(), SubjektivHostError> {
@@ -332,6 +427,9 @@ impl LocalSubjectHost {
"Session does not belong to this local Subject/Worker scope".into(),
));
}
store
.bind_journal_session(&self.writer, record.clone())
.map_err(journal::journal_error)?;
return Ok(record);
}
if !create {
@@ -348,9 +446,13 @@ impl LocalSubjectHost {
&context.session_id,
)
.map_err(domain_error)?;
store
let record = store
.record_session_attribution(record)
.map_err(domain_error)
.map_err(domain_error)?;
store
.bind_journal_session(&self.writer, record.clone())
.map_err(journal::journal_error)?;
Ok(record)
}
}
@@ -551,10 +653,11 @@ impl LocalSubjectHost {
.and_then(|v| serde_json::from_value::<LocalConsolidationGrant>(v.clone()).ok())
.is_some_and(|g| g.binding == self.binding)
{
schedule_job(jobs.clone(), job.request.job_id);
schedule_job(jobs.clone(), job.request.job_id, self.self_weak.clone());
}
}
}
self.recover_journals();
}
fn validate_evidence(
&self,
@@ -690,6 +793,69 @@ impl LocalSubjectHost {
}
}
impl SubjektivHost for LocalSubjectHost {
fn memory_journal(
&self,
context: &SubjektivHostContext,
request: subjektiv::journal::memory_links::MemoryJournalRequest,
) -> Result<subjektiv::journal::memory_links::MemoryJournalResponse, SubjektivHostError> {
self.body(context)?;
let reader = subjektiv::journal::JournalHostContext::validated_reader(
&self.catalog.store,
&self.binding.subject_id,
)
.map_err(journal::journal_error)?;
self.catalog
.store
.execute_memory_journal_with_source_presence(&reader, request, |session| {
crate::subjektiv_sessions::original_session_presence(
&self.catalog.state_root,
&self.binding.scope_id,
&self.binding.subject_id,
&self.catalog.store,
session,
)
})
.map_err(journal::journal_error)
}
fn journal_startup_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn journal_startup_context(
&self,
context: &SubjektivHostContext,
) -> Result<subjektiv::journal::startup::JournalStartupSnapshot, SubjektivHostError> {
self.body(context)?;
let reader = subjektiv::journal::JournalHostContext::validated_reader(
&self.catalog.store,
&self.binding.subject_id,
)
.map_err(journal::journal_error)?;
let predecessor = self
.catalog
.store
.journal_predecessor_session(&self.writer)
.map_err(journal::journal_error)?;
self.catalog
.store
.journal_startup_snapshot(&reader, predecessor.as_deref())
.map_err(journal::journal_error)
}
fn journal_recording_granted(&self) -> bool {
self.live.load(Ordering::Acquire)
}
fn request_journal_recording(
&self,
context: &SubjektivHostContext,
_reason: subjektiv::journal::transport::JournalRecordingReason,
) -> Result<subjektiv::journal::transport::JournalRecordingResponse, SubjektivHostError> {
let admitted = self.admit_journal(context)?;
self.wake_journals();
Ok(if admitted {
subjektiv::journal::transport::JournalRecordingResponse::Accepted
} else {
subjektiv::journal::transport::JournalRecordingResponse::NoNewSources
})
}
fn settings(&self) -> SubjektivHostSettings {
SubjektivHostSettings {
language: self.language.clone(),
@@ -788,6 +954,7 @@ impl SubjektivHost for LocalSubjectHost {
create_if_missing: bool,
) -> Result<SubjektivRecordSessionResponse, SubjektivHostError> {
let attribution = self.attribution(context, create_if_missing)?;
self.reserve_journal_session(context)?;
Ok(SubjektivRecordSessionResponse {
subject_id: attribution.subject_id,
session_id: attribution.session_id,
@@ -811,7 +978,7 @@ impl SubjektivHost for LocalSubjectHost {
.is_some_and(|g| g.binding == self.binding)
})
{
schedule_job(jobs, pending.request.job_id);
schedule_job(jobs, pending.request.job_id, self.self_weak.clone());
let (count, bytes) = self
.catalog
.store
@@ -874,7 +1041,7 @@ impl SubjektivHost for LocalSubjectHost {
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?,
)
.map_err(|e| SubjektivHostError::Unavailable(e.to_string()))?;
schedule_job(jobs, snapshot.request.job_id);
schedule_job(jobs, snapshot.request.job_id, self.self_weak.clone());
Ok(memory::backend::MemoryConsolidationOutput { status:"started".into(),summary:"Requested a durable generic standalone consolidation Job; processing stops on Host shutdown".into(),candidate_count:count,total_bytes:bytes })
}
}
@@ -894,6 +1061,9 @@ impl crate::jobs::JobDomainProvider for LocalDomainProvider {
) -> Result<worker::job::JobFeatureGrant, String> {
let host = self.0.upgrade().ok_or("Subject Host unavailable")?;
host.ensure_live().map_err(|e| e.to_string())?;
if snapshot.request.purpose == journal::PURPOSE {
return journal::grant(host, snapshot, fence);
}
let grant: LocalConsolidationGrant = serde_json::from_value(
snapshot
.domain_grant
@@ -1097,7 +1267,11 @@ impl worker::job::JobResultFinalizer for LocalConsolidationFinalizer {
}
}
fn schedule_job(jobs: crate::jobs::StandaloneJobs, id: String) {
fn schedule_job(
jobs: crate::jobs::StandaloneJobs,
id: String,
host: std::sync::Weak<LocalSubjectHost>,
) {
// Scheduling/consumer acknowledgement has no Internal Worker ownership.
// The generic service fences and retains every execution until cleanup.
tokio::spawn(async move {
@@ -1114,6 +1288,9 @@ fn schedule_job(jobs: crate::jobs::StandaloneJobs, id: String) {
"standalone Subject Job {id} cleanup/acknowledgement is unconfirmed: {error}"
),
}
if let Some(host) = host.upgrade() {
host.wake_journals();
}
});
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,436 @@
//! Public standalone protocol through real mid-turn compaction and Journal Jobs.
//! Only external model streams are scripted. No history, attribution, hook or Job
//! result is injected; cancellation and rollback use the controller's public commands.
//! Reported model usage and a low configured request threshold force two rewrites.
//! The test attests durable public originals, not unfinished display deltas or meaning.
use super::*;
use agen::llm_client::{
ClientError, LlmClient, Request,
event::{Event, ResponseStatus, StatusEvent, StopReason, UsageEvent},
types::Item,
};
use futures::{Stream, StreamExt};
use std::pin::Pin;
use std::sync::atomic::AtomicUsize;
type ResponseStream = Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>;
const SUMMARY: &str = "DERIVED_COMPACTION_SENTINEL_NOT_AN_ORIGINAL";
const ROLLED_BACK: &str = "ROLLED_BACK_INPUT_MUST_NOT_REACH_JOURNAL";
const PARTIAL: &str = "public partial answer before cancellation";
#[derive(Clone, Default)]
struct CompactingDialog {
shared: DialogAndJournal,
dialog_calls: Arc<AtomicUsize>,
compactions: Arc<AtomicUsize>,
rollback_entered: Arc<tokio::sync::Notify>,
partial_stream_paused: Arc<tokio::sync::Notify>,
}
fn events(events: Vec<Event>) -> ResponseStream {
Box::pin(futures::stream::iter(events.into_iter().map(Ok)))
}
fn done() -> ResponseStream {
events(vec![
Event::text_block_start(0),
Event::text_delta(0, "done"),
Event::text_block_stop(0, Some(StopReason::EndTurn)),
])
}
#[async_trait::async_trait]
impl LlmClient for CompactingDialog {
async fn stream(&self, request: Request) -> Result<ResponseStream, ClientError> {
if request
.tools
.iter()
.any(|tool| tool.name == "write_summary")
{
if request.items.iter().any(|item| matches!(item, Item::ToolResult { call_id, .. } if call_id == "compact-summary")) {
return Ok(done());
}
let count = self.compactions.fetch_add(1, Ordering::SeqCst);
assert!(count < 2, "unexpected additional compaction");
return Ok(events(vec![
Event::tool_use_start(0, "compact-summary", "write_summary"),
Event::tool_input_delta(0, serde_json::json!({"text": SUMMARY}).to_string()),
Event::tool_use_stop(0),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]));
}
if request.tools.iter().any(|tool| {
matches!(
tool.name.as_str(),
"ReadJournalState" | "SubmitMemorySurface" | "SubmitJobResult"
)
}) {
return self.shared.stream(request).await;
}
match self.dialog_calls.fetch_add(1, Ordering::SeqCst) {
step @ (0 | 1) => {
assert!(request.tools.iter().any(|tool| tool.name == "TaskCreate"));
Ok(events(vec![
Event::tool_use_start(0, format!("public-task-{step}"), "TaskCreate"),
Event::tool_input_delta(
0,
serde_json::json!({
"subject": format!("original activity {step}"),
"description": format!("original task result before compaction {step}")
})
.to_string(),
),
Event::tool_use_stop(0),
Event::Usage(UsageEvent {
input_tokens: Some(100_000),
output_tokens: Some(10),
total_tokens: Some(100_010),
..Default::default()
}),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]))
}
2 => {
let paused = self.partial_stream_paused.clone();
Ok(Box::pin(
events(vec![
Event::text_block_start(0),
Event::text_delta(0, PARTIAL),
Event::text_block_stop(0, None),
])
.chain(futures::stream::once(async move {
paused.notify_one();
std::future::pending::<Result<Event, ClientError>>().await
})),
))
}
3 => {
// No AI event: cancelling this request must roll back the submitted input.
self.rollback_entered.notify_one();
std::future::pending::<()>().await;
unreachable!()
}
other => panic!("unexpected dialog request {other}"),
}
}
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
}
async fn cancel_and_wait(
client: &mut client::Client<client::transport::in_process::Socket>,
sequence: u64,
) {
client
.send(&protocol::Method::Cancel {
command: protocol::WorkerCommandEnvelope::new(sequence),
})
.await
.unwrap();
tokio::time::timeout(std::time::Duration::from_secs(20), async {
loop {
match client.next_event().await.unwrap().unwrap() {
protocol::Event::RunEnd { result } => {
assert_ne!(result, protocol::RunResult::Finished);
return;
}
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
}
})
.await
.unwrap();
}
#[tokio::test]
async fn public_mid_turn_compactions_preserve_originals_and_drain_after_cancel_and_rollback() {
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("compaction-journal", None).unwrap();
let model = CompactingDialog {
shared: DialogAndJournal {
journal: JournalScript {
gate_first: true,
..Default::default()
},
..Default::default()
},
..Default::default()
};
let mut launch = crate::StandaloneLaunchConfig::new(
temp.path(),
&state,
manifest::ProfileSelector::source_named(
manifest::ProfileRegistrySource::Builtin,
"standalone-subjektiv",
),
"public-compaction-journal",
)
.with_subject(subject.id.clone())
.resolve()
.unwrap();
launch.profile.manifest.compaction = Some(manifest::CompactionConfig {
threshold: None,
request_threshold: Some(50_000),
retained_tokens: 0,
..Default::default()
});
let mut host = crate::StandaloneHost::start_with_model_client(launch, model.clone())
.await
.unwrap();
let session = host.record().active_session_id;
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
let source_store = session_store::FsStore::new_with_retention(
workers.sessions_dir(host.record().worker_id),
crate::store::session_retention(&state),
)
.unwrap();
let mut client = host.connect();
client
.send(&protocol::Method::submit_text(
protocol::new_submission_request_id(),
"original long turn survives compaction",
))
.await
.unwrap();
let mut completed_compactions = 0;
loop {
match client.next_event().await.unwrap().unwrap() {
protocol::Event::CompactionProgress {
compaction: Some(progress),
} if progress.phase == protocol::CompactionPhase::Committing => {
assert_eq!(
progress.trigger,
protocol::CompactionTrigger::RequestThreshold
);
completed_compactions += 1;
}
protocol::Event::TextDelta { text } if text.contains(PARTIAL) => break,
protocol::Event::RunEnd { .. } => panic!("dialog ended before cancellation"),
protocol::Event::Error { code, message } => panic!("{code:?}: {message}"),
_ => {}
}
}
assert_eq!(model.compactions.load(Ordering::SeqCst), 2);
assert_eq!(completed_compactions, 2);
model.shared.journal.entered.notified().await;
let context = JournalHostContext::validated_session(
&catalog.store,
catalog
.store
.session_attribution(&session.to_string())
.unwrap()
.unwrap(),
)
.unwrap();
let reader = JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap();
let journal = catalog
.store
.list_journals(&reader, None, 1)
.unwrap()
.items
.pop()
.unwrap();
let first = catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap();
assert!(
catalog
.store
.list_journal_entries(&reader, &journal.id, None, 10)
.unwrap()
.items
.is_empty(),
"compaction must finish before generation is released"
);
model.partial_stream_paused.notified().await;
cancel_and_wait(&mut client, 1).await;
client
.send(&protocol::Method::submit_text(
protocol::new_submission_request_id(),
ROLLED_BACK,
))
.await
.unwrap();
model.rollback_entered.notified().await;
let before_rollback: Vec<_> = source_store
.list_segments(session)
.unwrap()
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
assert!(
serde_json::to_string(&before_rollback)
.unwrap()
.contains(ROLLED_BACK),
"the submitted input must reach the actual durable rollback boundary"
);
cancel_and_wait(&mut client, 2).await;
assert_eq!(
catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap(),
first,
"compaction/cancellation must not alter the running generation's frozen input"
);
assert!(matches!(source_store.delete_session(session),
Err(session_store::StoreError::SessionPinned(id)) if id == session));
let segments = source_store.list_segments(session).unwrap();
assert_eq!(segments.len(), 3);
let raw: Vec<_> = segments
.iter()
.flat_map(|segment| source_store.read_all(session, *segment).unwrap())
.collect();
assert_eq!(
raw.iter()
.filter(|entry| matches!(
entry,
session_store::LogEntry::AnnotatedSegmentStart {
compacted_from: Some(_),
..
}
))
.count(),
2
);
assert!(
!serde_json::to_string(&raw).unwrap().contains(ROLLED_BACK),
"pre-AI cancellation must roll back the speculative input"
);
assert!(
serde_json::to_string(&raw).unwrap().contains(SUMMARY),
"real rewrite did not persist its summary"
);
let metadata =
session_store::FsWorkerStore::new(workers.worker_metadata_dir(host.record().worker_id))
.unwrap()
.read_by_name(&host.record().storage_key)
.unwrap()
.unwrap();
let originals =
session_store::public_material::SessionPublicMaterialReader::open_fs_with_retention(
&workers.sessions_dir(host.record().worker_id),
session,
metadata.active.unwrap().segment_id.unwrap(),
session_store::public_material::PublicMaterialLimits::default(),
crate::store::session_retention(&state),
)
.unwrap();
let final_original = originals.descriptors().last().unwrap();
let expected_cursor = (final_original.canonical_order + 1, 0);
let canonical: Vec<_> = originals
.descriptors()
.map(|descriptor| {
let page = originals
.read_content(descriptor, 0, MAX_JOURNAL_READ_BYTES)
.unwrap();
assert_eq!(page.next_offset, descriptor.total_bytes);
page.content
})
.collect();
assert!(!canonical.join("\n").contains(ROLLED_BACK));
assert!(!canonical.join("\n").contains(SUMMARY));
// Resume the existing runner, then its one durable successor, with no more input.
let jobs = host.jobs();
model.shared.journal.release.notify_one();
jobs.wait(&first.job.job_id).await.unwrap();
model.shared.journal.entered.notified().await;
let second = catalog
.store
.active_journal_generation(&context, &journal.id)
.unwrap()
.unwrap();
assert_ne!(first.id, second.id);
model.shared.journal.release.notify_one();
jobs.wait(&second.job.job_id).await.unwrap();
assert!(!catalog.store.journal_retention_required(&context).unwrap());
let (saved, open) = catalog
.store
.journal_resume_state(&context, &journal.id)
.unwrap();
let open = open.unwrap();
assert!(!open.ongoing.is_empty());
let entries = catalog
.store
.list_journal_entries(&reader, &journal.id, None, 10)
.unwrap();
assert_eq!(
entries.items.len(),
1,
"compaction/turn boundaries must not fix or duplicate the open entry"
);
assert_eq!(entries.items[0].id, open.id);
let supplied = model
.shared
.journal
.source_reads
.lock()
.unwrap()
.values()
.cloned()
.collect::<Vec<_>>()
.join("\n");
for original in [
"original long turn survives compaction",
"original activity 0",
"original activity 1",
] {
assert!(
supplied.contains(original),
"Journal did not receive {original}"
);
}
for original in canonical {
assert!(
supplied.contains(&original),
"a durable public original was skipped"
);
}
assert!(
!supplied.contains(SUMMARY),
"derived summary substituted for original input"
);
assert!(
!supplied.contains(ROLLED_BACK),
"cancelled input was admitted after rollback"
);
assert_eq!((saved.process_order, saved.process_byte), expected_cursor);
let position = catalog
.store
.read_journal_text(
&reader,
&journal.id,
None,
JournalTextField::CurrentPosition,
None,
0,
MAX_JOURNAL_READ_BYTES,
)
.unwrap();
assert!(!position.text.is_empty());
loop {
match host.disconnect_subject().await {
Ok(()) => break,
Err(crate::StandaloneSubjectDisconnectError::Incomplete(message)) if message.contains("dispatcher") || message.contains("cleanup") => tokio::task::yield_now().await,
Err(error) => panic!("recorded compaction/cancellation lineage must remain releasable while work is unfinished: {error}"),
}
}
host.shutdown().await.unwrap();
})
.await
.expect("bounded public compaction/Journal integration");
}
@@ -0,0 +1,228 @@
//! Real standalone Subject/Controller/Journal Job/initial input connection. Only external
//! model streams are scripted. The scenario checks persisted data and ordering, not whether
//! a real model understands the correction or can autonomously continue the work.
use super::*;
use crate::{StandaloneHost, StandaloneLaunchConfig, StandaloneSubjectDisconnectError};
use agen::llm_client::types::Item;
fn launch(root: &Path, state: &Path, subject: &str, name: &str) -> crate::ResolvedStandaloneLaunch {
StandaloneLaunchConfig::new(
root,
state,
manifest::ProfileSelector::source_named(
manifest::ProfileRegistrySource::Builtin,
"standalone-subjektiv",
),
name,
)
.with_subject(subject.into())
.resolve()
.unwrap()
}
#[tokio::test]
async fn recorded_idle_body_handoff_preserves_open_after_source_deletion_and_seeds_next_body_once()
{
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("continuity", Some("BEHAVIOR_ORIGINAL_UNCHANGED")).unwrap();
let model = DialogAndJournal { journal: JournalScript { gate_first: true, ..Default::default() }, ..Default::default() };
let mut first = StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "first"), model.clone()).await.unwrap();
let old_worker = first.worker_id();
let old_session = first.record().active_session_id;
let old_writer = catalog.store.claim_journal_writer(&subject.id, &catalog.scope_id, &old_worker.to_string()).unwrap();
submit_dialog(&first, "A was proposed, user corrected its constraint; B selected because A is unusable. One step performed; tests not performed; external outcome Unknown. Work is unfinished.").await;
model.journal.entered.notified().await;
assert!(matches!(first.disconnect_subject().await, Err(StandaloneSubjectDisconnectError::Incomplete(_))));
assert!(catalog.store.journal_writer_active(&old_writer).unwrap());
assert!(StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "premature"), DialogAndJournal::default()).await.is_err());
let context = JournalHostContext::validated_session(&catalog.store, catalog.store.session_attribution(&old_session.to_string()).unwrap().unwrap()).unwrap();
let id = catalog.store.journal_for_session(&context).unwrap().unwrap();
let generation = catalog.store.active_journal_generation(&context, &id).unwrap().unwrap();
model.journal.release.notify_one();
first.jobs().wait(&generation.job.job_id).await.unwrap();
// Observe the authoritative gate after cleanup; yielding is cooperative scheduling,
// never elapsed time used as proof of recording completion.
loop {
match first.disconnect_subject().await {
Ok(()) => break,
Err(StandaloneSubjectDisconnectError::Incomplete(message)) if message.contains("dispatcher") || message.contains("cleanup") => tokio::task::yield_now().await,
Err(error) => panic!("unexpected handoff blocker: {error}"),
}
}
assert!(!catalog.store.journal_writer_active(&old_writer).unwrap());
assert!(catalog.store.accept_journal_sources(&context, "late old activity", &[generation.selected_ranges[0].clone()]).is_err());
first.shutdown().await.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
workers.delete(old_worker).unwrap();
assert!(!workers.sessions_dir(old_worker).exists());
let reader = JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap();
let historical = catalog.store.journal_startup_snapshot(&reader, Some(&old_session.to_string())).unwrap();
assert_eq!(historical.journals[0].entries[0].state, JournalEntryState::Open);
assert!(historical.journals[0].entries[0].ongoing.contains("unknown"));
assert!(catalog.store.list_memories(&subject.id).unwrap().is_empty());
let second_model = DialogAndJournal::default();
let second = StandaloneHost::start_with_model_client(launch(temp.path(), &state, &subject.id, "second"), second_model.clone()).await.unwrap();
submit_dialog(&second, "Continue B; describe what is still unperformed and unconfirmed.").await;
let requests = second_model.dialog_requests.lock().unwrap();
let initial = &requests[0];
let context_index = initial.items.iter().position(|item| item.as_text().is_some_and(|text| text.contains("Proposed A; user corrected it to B"))).expect("saved Journal visible to first model request");
let user_index = initial.items.iter().position(|item| matches!(item, Item::Message { role: agen::llm_client::types::Role::User, .. })).unwrap();
assert!(context_index < user_index);
assert_eq!(initial.items.iter().filter(|item| item.as_text().is_some_and(|text| text.contains("Proposed A; user corrected it to B"))).count(), 1);
drop(requests);
let session = second.record().active_session_id;
let sessions = session_store::FsStore::new_with_retention(workers.sessions_dir(second.worker_id()), crate::store::session_retention(&state)).unwrap();
let pointer = session_store::FsWorkerStore::new(workers.worker_metadata_dir(second.worker_id())).unwrap().read_by_name(&second.record().storage_key).unwrap().unwrap().active.unwrap();
let entries = sessions.read_all(session, pointer.segment_id.unwrap()).unwrap();
let journal_items = entries.iter().filter(|entry| matches!(entry, session_store::LogEntry::AnnotatedSystemItem { entry, .. } if matches!(entry.item, session_store::SystemItem::JournalStartup { .. }))).count();
assert_eq!(journal_items, 1);
assert_eq!(catalog.store.subject(&subject.id).unwrap().unwrap().behavior_md, "BEHAVIOR_ORIGINAL_UNCHANGED");
second.shutdown().await.unwrap();
}).await.expect("bounded complete Subject handoff connection");
}
#[tokio::test]
async fn saved_public_history_without_admission_notification_blocks_release_until_actual_recording()
{
tokio::time::timeout(std::time::Duration::from_secs(60), async {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("saved-before-notified", None).unwrap();
let model = DialogAndJournal {
journal: JournalScript {
gate_first: true,
..Default::default()
},
..Default::default()
};
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "gap"),
model.clone(),
)
.await
.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
let record = host.record().clone();
let writer = catalog
.store
.claim_journal_writer(
&subject.id,
&catalog.scope_id,
&host.worker_id().to_string(),
)
.unwrap();
let sessions = session_store::FsStore::new_with_retention(
workers.sessions_dir(host.worker_id()),
crate::store::session_retention(&state),
)
.unwrap();
// The production saved-history boundary, with its admission callback deliberately
// omitted. The real Host release must discover these bytes, not trust Job status.
sessions
.append(
record.active_session_id,
record.active_segment_id.unwrap(),
&session_store::LogEntry::AnnotatedUserInput {
ts: 3,
segments: vec![],
extensions: vec![],
history: vec![message(
"missed-admission",
"A corrected to B, partially performed, outcome Unknown",
)],
},
)
.unwrap();
assert!(host.disconnect_subject().await.is_err());
assert!(catalog.store.journal_writer_active(&writer).unwrap());
model.journal.entered.notified().await;
let context = JournalHostContext::validated_session(
&catalog.store,
catalog
.store
.session_attribution(&record.active_session_id.to_string())
.unwrap()
.unwrap(),
)
.unwrap();
let id = catalog
.store
.journal_for_session(&context)
.unwrap()
.unwrap();
let generation = catalog
.store
.active_journal_generation(&context, &id)
.unwrap()
.unwrap();
model.journal.release.notify_one();
host.jobs().wait(&generation.job.job_id).await.unwrap();
loop {
match host.disconnect_subject().await {
Ok(()) => break,
Err(StandaloneSubjectDisconnectError::Incomplete(message))
if message.contains("dispatcher") || message.contains("cleanup") =>
{
tokio::task::yield_now().await
}
Err(error) => {
panic!("unexpected release blocker after saved-history recording: {error}")
}
}
}
assert!(!catalog.store.journal_writer_active(&writer).unwrap());
let snapshot = catalog
.store
.journal_startup_snapshot(
&JournalHostContext::validated_reader(&catalog.store, &subject.id).unwrap(),
None,
)
.unwrap();
assert!(!snapshot.journals[0].current_position.is_empty());
assert_eq!(
snapshot.journals[0].entries[0].state,
JournalEntryState::Open
);
host.shutdown().await.unwrap();
})
.await
.expect("saved-history/notification-gap handoff is bounded");
}
#[tokio::test]
async fn shutdown_and_normal_deletion_cannot_release_subject_occupancy() {
let temp = tempfile::tempdir().unwrap();
let state = temp.path().join("state");
let catalog = StandaloneSubjects::open(&state).unwrap();
let subject = catalog.create("retained-owner", None).unwrap();
let host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "retained"),
DialogAndJournal::default(),
)
.await
.unwrap();
let worker = host.worker_id();
host.shutdown().await.unwrap();
let workers = StandaloneWorkerStore::open_existing(&state).unwrap();
assert!(workers.delete(worker).is_err());
assert!(
StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "not-an-owner"),
DialogAndJournal::default()
)
.await
.is_err()
);
let mut restored =
StandaloneHost::restore_with_model_client(state, worker, DialogAndJournal::default())
.await
.unwrap();
restored.disconnect_subject().await.unwrap();
restored.shutdown().await.unwrap();
}
@@ -0,0 +1,596 @@
//! Connected standalone Host -> Subjektiv Memory/Journal -> real FsStore coverage.
//! Candidate and Journal decisions are explicit test inputs; no model, Runtime or live DB
//! is used. The parent Journal fixture owns isolated temporary storage and binding cleanup.
//! Selected by the ordinary standalone lib tests, with no additional feature or service.
use super::*;
use memory::{
backend::MemoryStageCandidateOperation,
extract::{CandidateKind, ExtractedCandidate, StagingEvidence},
schema::{EvidenceKind, EvidenceOriginKind, SourceEvidenceRef, SourceRef},
};
use subjektiv::{
CandidateDecision, CandidateDecisionRequest, MemoryChangeTarget, MemoryDraft, MemoryRecord,
StagingResolution, SubjectStagingRecord,
journal::memory_links::{
JournalTargetAvailability, MemoryJournalLinks, MemoryJournalRequest, MemoryJournalResponse,
MemoryJournalSelector, SourceConnection, UnconnectedReason,
},
};
fn adopt_original(fixture: &Fixture) -> (MemoryRecord, SubjectStagingRecord, StagingResolution) {
let evidence_index = session_store::public_index::read_fs_session_evidence_index(
fixture.sessions.root_dir(),
fixture.context.session_id.parse().unwrap(),
session_store::SessionPublicIndexLimits {
max_bytes: 8 * 1024 * 1024,
max_entries: 20_000,
max_segments: 64,
},
)
.unwrap();
let range = *evidence_index
.ranges
.get(&(fixture.segment.to_string(), "Efirst".into()))
.unwrap()
.first()
.unwrap();
let staged = fixture
.host()
.stage_candidate(
&fixture.context,
MemoryStageCandidateOperation {
source: SourceRef {
segment_id: fixture.segment.to_string(),
range,
},
extract_run_id: "explicit-test-extraction-input".into(),
candidate: ExtractedCandidate {
kind: CandidateKind::Preference,
claim: "Keep correction history".into(),
why_useful: "Preserves the reason for later decisions".into(),
staleness: None,
evidence_ids: vec!["Efirst".into()],
},
evidence: vec![StagingEvidence {
id: "Efirst".into(),
kind: EvidenceKind::new(EvidenceKind::MESSAGE),
entry_range: Some(range),
origin: None,
excerpt: None,
summary: None,
}],
source_refs: vec![SourceEvidenceRef {
session_id: Some(fixture.context.session_id.clone()),
segment_id: Some(fixture.segment.to_string()),
entry_range: Some(range),
evidence_id: Some("Efirst".into()),
..Default::default()
}],
},
)
.unwrap();
let store = &fixture.host().catalog.store;
let subject = &fixture.connection.binding.subject_id;
let candidate = store
.staging_candidate(subject, &staged.staging_id)
.unwrap()
.unwrap();
let receipt = store
.decide_candidate(
subject,
CandidateDecisionRequest {
request_id: "explicit-test-adoption-input".into(),
candidate_id: staged.staging_id,
reason: "Adopt the committed original evidence, not the Journal summary".into(),
decision: CandidateDecision::Apply {
target: MemoryChangeTarget::Create,
draft: MemoryDraft::active(
CandidateKind::Preference,
"Keep correction history",
"Preserve earlier decisions and their corrections",
"Preserves the reason for later decisions",
"Adopt original HumanInput",
),
},
},
)
.unwrap();
(receipt.memory.unwrap(), candidate, receipt.resolution)
}
fn create_entry(adapter: &LocalJournalHost, state: JournalEntryState) -> JournalReceipt {
save(
adapter,
"record-original-context",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "A proposal was corrected".into(),
body: "Proposal A changed to B; only part was performed, outcome unconfirmed.".into(),
ongoing: if state == JournalEntryState::Open {
"Remaining work is not complete".into()
} else {
String::new()
},
state,
source_end: (state == JournalEntryState::Fixed).then(|| {
adapter
.generation
.selected_ranges
.last()
.unwrap()
.source
.pointer
.clone()
}),
correction_refs: vec![],
},
adapter.generation.selected_ranges.clone(),
)
}
fn assert_originals_unchanged(
fixture: &Fixture,
memory: &MemoryRecord,
candidate: &SubjectStagingRecord,
resolution: &StagingResolution,
) {
let catalog = StandaloneSubjects::open_existing(&fixture.host().catalog.state_root).unwrap();
let store = &catalog.store;
let subject = &fixture.connection.binding.subject_id;
assert_eq!(
serde_json::to_value(store.memory(subject, &memory.id).unwrap().unwrap()).unwrap(),
serde_json::to_value(memory).unwrap(),
);
assert_eq!(
serde_json::to_value(
store
.staging_candidate(subject, &candidate.id)
.unwrap()
.unwrap()
)
.unwrap(),
serde_json::to_value(candidate).unwrap(),
);
assert_eq!(
serde_json::to_value(
store
.staging_resolution(subject, &candidate.id)
.unwrap()
.unwrap()
)
.unwrap(),
serde_json::to_value(resolution).unwrap(),
);
assert_eq!(store.list_memories(subject).unwrap().len(), 1);
}
fn selector(memory: &MemoryRecord, candidate: &SubjectStagingRecord) -> MemoryJournalSelector {
MemoryJournalSelector {
memory_id: memory.id.clone(),
change_id: memory.change_id.clone(),
candidate_id: candidate.id.clone(),
}
}
fn links(
fixture: &Fixture,
selector: &MemoryJournalSelector,
reconcile: bool,
) -> MemoryJournalLinks {
let request = if reconcile {
MemoryJournalRequest::Reconcile {
selector: selector.clone(),
}
} else {
MemoryJournalRequest::Links {
selector: selector.clone(),
}
};
let MemoryJournalResponse::Links(links) = fixture
.host()
.memory_journal(&fixture.context, request)
.unwrap()
else {
panic!("expected Memory Journal links")
};
links
}
fn read_request(
selector: &MemoryJournalSelector,
journal: &str,
entry: &str,
) -> MemoryJournalRequest {
MemoryJournalRequest::Read {
selector: selector.clone(),
source_index: 0,
journal_id: journal.into(),
entry_id: entry.into(),
expected_token: None,
byte_offset: 0,
max_bytes: MAX_JOURNAL_READ_BYTES,
}
}
fn assert_link(links: &MemoryJournalLinks, journal: &str, entry: &str, reconsideration: bool) {
assert_eq!(links.sources.len(), 1);
assert_eq!(links.sources[0].source_index, 0);
assert_eq!(
links.sources[0].connection,
SourceConnection::Linked {
journal_id: journal.into(),
entry_id: entry.into(),
target: JournalTargetAvailability::Available,
reconsideration,
},
);
}
#[test]
fn saved_link_reads_after_original_worker_and_session_deletion_from_replacement_host() {
let mut fixture = Fixture::new("Please preserve the reasons for my corrected decisions");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
assert_eq!(
candidate.evidence[0].origin.as_ref().unwrap().kind,
EvidenceOriginKind::HumanInput
);
assert_eq!(
candidate.source_refs[0].origin,
candidate.evidence[0].origin
);
assert!(
candidate.evidence[0]
.excerpt
.as_ref()
.unwrap()
.contains("corrected decisions")
);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Fixed);
let entry = recorded.entry_id.clone().unwrap();
let journal = adapter.grant.journal_id.clone();
let connected = links(&fixture, &selector, true);
assert_link(&connected, &journal, &entry, false);
let request = read_request(&selector, &journal, &entry);
let before = fixture
.host()
.memory_journal(&fixture.context, request.clone())
.unwrap();
let positioned = save(
&adapter,
"position-original",
&recorded.state_token,
JournalAction::UpdatePosition {
current_position: "Recorded correction; remaining outcome unconfirmed".into(),
},
vec![],
);
save(
&adapter,
"checkpoint-original",
&positioned.state_token,
JournalAction::Checkpoint {
record_operation_ids: vec![recorded.request_id],
},
adapter.generation.selected_ranges.clone(),
);
adapter.finish().unwrap();
fixture
.host()
.catalog
.store
.close_journal_generation(
&adapter.context,
&adapter.generation.id,
&adapter.generation.job,
)
.unwrap();
let state = fixture.host().catalog.state_root.clone();
let subject = fixture.connection.binding.subject_id.clone();
let original_session = fixture.context.session_id.clone();
// This Host/storage suite has no Controller. Complete the explicit persisted
// writer-release boundary before deletion; the sibling handoff suite exercises
// the production Idle reservation + Host lifecycle that authorizes that release.
fixture
.host()
.catalog
.store
.release_journal_writer(&fixture.host().writer)
.unwrap();
let (worker, workers) = fixture.stop_worker();
workers.delete(worker).unwrap();
assert!(matches!(
workers.load(worker),
Err(crate::store::StandaloneStoreError::WorkerNotFound(_))
));
assert!(!fixture.sessions.root_dir().join(&original_session).exists());
assert!(
!retained_workers_root(&state)
.join(worker.to_string())
.exists()
);
let replacement_worker = protocol::WorkerId::now_v7();
let mut replacement = StandaloneSubjects::open_existing(&state)
.unwrap()
.connect(&subject, replacement_worker)
.unwrap();
let replacement_context = SubjektivHostContext {
worker_id: replacement_worker.to_string(),
session_id: session_store::SessionId::now_v7().to_string(),
};
let MemoryJournalResponse::Links(saved) = replacement
.host
.memory_journal(
&replacement_context,
MemoryJournalRequest::Links {
selector: selector.clone(),
},
)
.unwrap()
else {
panic!("expected saved links")
};
assert_link(&saved, &journal, &entry, false);
let after = replacement
.host
.memory_journal(&replacement_context, request)
.unwrap();
let MemoryJournalResponse::Read(read) = &after else {
panic!("expected saved Journal text")
};
assert!(
read.page
.as_ref()
.unwrap()
.text
.contains("outcome unconfirmed")
);
assert_eq!(
read.source.original_source,
subjektiv::journal::memory_links::OriginalSourceAvailability::Missing
);
// Reading the saved edge does not depend on a live Session fetch or a replacement Job.
let MemoryJournalResponse::Read(before) = before else {
panic!("expected Journal text")
};
assert_eq!(before.page, read.page);
assert!(
replacement.host.jobs().is_err(),
"the reader has no model/Job executor"
);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
replacement.close().unwrap();
}
#[test]
fn late_journal_fix_connects_existing_memory_without_using_checkpoint_as_event_end() {
let fixture = Fixture::new("Keep the reasons behind corrected decisions");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
assert_eq!(
links(&fixture, &selector, false).sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::NotReconciled
}
);
assert_eq!(
links(&fixture, &selector, true).sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::SourceNotAdmitted
}
);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Open);
let entry = recorded.entry_id.clone().unwrap();
let positioned = save(
&adapter,
"position-open",
&recorded.state_token,
JournalAction::UpdatePosition {
current_position: "Work remains open; no fixed endpoint yet".into(),
},
vec![],
);
// Checkpoint a proper prefix, leaving source bytes for the later Fix.
// A fully checkpointed generation cannot admit another activity write.
let mut reflected = adapter.generation.selected_ranges.clone();
reflected.last_mut().unwrap().bytes.end -= 1;
let checkpoint = save(
&adapter,
"checkpoint-open",
&positioned.state_token,
JournalAction::Checkpoint {
record_operation_ids: vec![recorded.request_id],
},
reflected.clone(),
);
let unconnected = links(&fixture, &selector, true);
assert_eq!(
unconnected.sources[0].connection,
SourceConnection::Unconnected {
reason: UnconnectedReason::JournalNotFixed
}
);
assert_eq!(links(&fixture, &selector, true), unconnected);
let state = adapter.state().unwrap();
let open = state.current_open.unwrap();
assert_eq!(open.id, entry);
assert_eq!(open.state, JournalEntryState::Open);
assert!(open.source_end.is_none());
let last = reflected.last().unwrap();
assert_eq!(state.current_journal.process_order, last.source.order);
assert_eq!(state.current_journal.process_byte, last.bytes.end);
save(
&adapter,
"fix-later",
&checkpoint.state_token,
JournalAction::Fix {
entry_id: entry.clone(),
source_end: adapter
.generation
.selected_ranges
.last()
.unwrap()
.source
.pointer
.clone(),
},
adapter.generation.selected_ranges.clone(),
);
// A plain saved-state read must not silently perform reconciliation.
assert_eq!(links(&fixture, &selector, false), unconnected);
let connected = links(&fixture, &selector, true);
assert_link(&connected, &adapter.grant.journal_id, &entry, false);
assert_eq!(links(&fixture, &selector, true), connected);
assert_eq!(links(&fixture, &selector, false), connected);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
#[test]
fn host_rejects_foreign_subject_worker_context_and_closed_connection_for_all_link_operations() {
let mut fixture = Fixture::new("Keep correction history");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
let (adapter, _, _) = fixture.adapter();
let recorded = create_entry(&adapter, JournalEntryState::Fixed);
let journal = adapter.grant.journal_id.clone();
let entry = recorded.entry_id.unwrap();
assert_link(&links(&fixture, &selector, true), &journal, &entry, false);
let requests = [
MemoryJournalRequest::Reconcile {
selector: selector.clone(),
},
MemoryJournalRequest::Links {
selector: selector.clone(),
},
read_request(&selector, &journal, &entry),
MemoryJournalRequest::Related {
journal_id: journal,
entry_id: entry,
after: None,
limit: 32,
},
];
let catalog = StandaloneSubjects::open_existing(&fixture.host().catalog.state_root).unwrap();
let foreign_subject = catalog.create("different-reader", None).unwrap();
let foreign_worker = protocol::WorkerId::now_v7();
let mut foreign = catalog
.connect(&foreign_subject.id, foreign_worker)
.unwrap();
let foreign_context = SubjektivHostContext {
worker_id: foreign_worker.to_string(),
session_id: session_store::SessionId::now_v7().to_string(),
};
let wrong_worker = SubjektivHostContext {
worker_id: foreign_worker.to_string(),
..fixture.context.clone()
};
for request in &requests {
assert!(
foreign
.host
.memory_journal(&foreign_context, request.clone())
.is_err()
);
assert!(
fixture
.host()
.memory_journal(&wrong_worker, request.clone())
.is_err()
);
}
foreign.close().unwrap();
fixture.connection.close().unwrap();
for request in requests {
assert!(
fixture
.host()
.memory_journal(&fixture.context, request)
.is_err()
);
}
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
#[test]
fn journal_correction_finds_related_memory_without_revising_or_retargeting_its_evidence() {
let fixture = Fixture::new("Keep correction history");
let (memory, candidate, resolution) = adopt_original(&fixture);
let selector = selector(&memory, &candidate);
fixture.append("correction", "The earlier result was not confirmed");
let (adapter, _, _) = fixture.adapter();
let ranges = &adapter.generation.selected_ranges;
assert_eq!(ranges.len(), 2);
let original = save(
&adapter,
"original-event",
&adapter.generation.initial_journal.state_token,
JournalAction::Create {
title: "Original context".into(),
body: "A change was proposed; execution remained uncertain".into(),
ongoing: String::new(),
state: JournalEntryState::Fixed,
source_end: Some(ranges[0].source.pointer.clone()),
correction_refs: vec![],
},
vec![ranges[0].clone()],
);
let entry = original.entry_id.unwrap();
let journal = &adapter.grant.journal_id;
assert_link(&links(&fixture, &selector, true), journal, &entry, false);
let corrected = save(
&adapter,
"correction-event",
&original.state_token,
JournalAction::Create {
title: "Correction".into(),
body: "The earlier result was unconfirmed; reconsider its associated Memory".into(),
ongoing: String::new(),
state: JournalEntryState::Fixed,
source_end: Some(ranges[1].source.pointer.clone()),
correction_refs: vec![entry.clone()],
},
vec![ranges[1].clone()],
);
assert_link(&links(&fixture, &selector, false), journal, &entry, true);
let correction_entry = corrected.entry_id.unwrap();
for target in [&entry, &correction_entry] {
let MemoryJournalResponse::Related(related) = fixture
.host()
.memory_journal(
&fixture.context,
MemoryJournalRequest::Related {
journal_id: journal.clone(),
entry_id: target.clone(),
after: None,
limit: 32,
},
)
.unwrap()
else {
panic!("expected related Memory")
};
assert_eq!(related.items.len(), 1);
assert_eq!(related.items[0].selector, selector);
assert_eq!(related.items[0].source_index, 0);
assert!(related.items[0].reconsideration);
assert!(related.next.is_none());
}
let MemoryJournalResponse::Read(read) = fixture
.host()
.memory_journal(&fixture.context, read_request(&selector, journal, &entry))
.unwrap()
else {
panic!("expected original context")
};
assert!(
read.page
.unwrap()
.text
.contains("execution remained uncertain")
);
assert_originals_unchanged(&fixture, &memory, &candidate, &resolution);
}
+12 -6
View File
@@ -37,9 +37,8 @@ fn subject_kernel_lease_is_not_worker_or_profile_scoped() {
let catalog = StandaloneSubjects::open(temp.path().join("state")).unwrap();
let a = catalog.create("a", None).unwrap();
let b = catalog.create("b", None).unwrap();
let mut first = catalog
.connect(&a.id, protocol::WorkerId::now_v7())
.unwrap();
let first_id = protocol::WorkerId::now_v7();
let mut first = catalog.connect(&a.id, first_id).unwrap();
assert!(matches!(
StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
@@ -51,11 +50,18 @@ fn subject_kernel_lease_is_not_worker_or_profile_scoped() {
.connect(&b.id, protocol::WorkerId::now_v7())
.unwrap();
first.close().unwrap();
let mut replacement = StandaloneSubjects::open_existing(&temp.path().join("state"))
assert!(
StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&a.id, protocol::WorkerId::now_v7())
.is_err(),
"kernel lease release is not durable body release"
);
let mut resumed = StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&a.id, protocol::WorkerId::now_v7())
.connect(&a.id, first_id)
.unwrap();
replacement.close().unwrap();
resumed.close().unwrap();
second.close().unwrap();
}
+104 -3
View File
@@ -155,6 +155,81 @@ pub(crate) fn authorized_index(
)
}
pub(crate) fn authorized_sessions_path(
state_root: &Path,
scope_id: &str,
subject_id: &str,
store: &SubjektivStore,
session_id: &str,
) -> Result<PathBuf, SubjektivHostError> {
let backend = Backend {
state_root,
scope_id,
subject_id,
store,
reader: session_store::public_index::read_fs_session_public_index,
};
(|| {
backend.check_scope()?;
required_selector(session_id)?;
let attribution = backend.attribution(session_id)?;
Ok(backend.authorize(attribution)?.sessions_path)
})()
.map_err(|error: Failure| SubjektivHostError::Unavailable(error.message))
}
/// Checks only saved ownership and filesystem metadata, never Session bytes or a Worker restore.
/// Inaccessible/corrupt storage stays unknown rather than being reported as absent.
pub(crate) fn original_session_presence(
state_root: &Path,
scope_id: &str,
subject_id: &str,
store: &SubjektivStore,
session_id: &str,
) -> Option<bool> {
let attribution = store.session_attribution(session_id).ok()??;
if store.scope_id() != scope_id
|| attribution.subject_id != subject_id
|| attribution.runtime_id != scope_id
{
return None;
}
let worker: WorkerId = attribution.worker_id.parse().ok()?;
let session: SessionId = session_id.parse().ok()?;
let active = StandaloneWorkerStore::open_existing(state_root).ok()?;
let (workers, record) = match active.load(worker) {
Ok(record) => (active, record),
Err(StandaloneStoreError::WorkerNotFound(_)) => {
let root = state_root.join("subjektiv").join("retained-workers");
match std::fs::metadata(&root) {
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Some(false),
Err(_) => return None,
Ok(_) => {}
}
let retained = StandaloneWorkerStore::open_existing(root).ok()?;
match retained.load(worker) {
Ok(record) => (retained, record),
Err(StandaloneStoreError::WorkerNotFound(_)) => return Some(false),
Err(_) => return None,
}
}
Err(_) => return None,
};
if record.worker_id != worker
|| !record
.subject
.as_ref()
.is_some_and(|binding| binding.scope_id == scope_id && binding.subject_id == subject_id)
{
return None;
}
match std::fs::metadata(workers.sessions_dir(worker).join(session.to_string())) {
Ok(metadata) if metadata.is_dir() => Some(true),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Some(false),
_ => None,
}
}
fn authorized_index_with_reader(
state_root: &Path,
scope_id: &str,
@@ -221,13 +296,28 @@ impl Backend<'_> {
"Attributed standalone Worker state is missing",
));
}
let workers = StandaloneWorkerStore::open_existing(self.state_root).map_err(|_| {
let mut workers = StandaloneWorkerStore::open_existing(self.state_root).map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RuntimeUnavailable,
"Standalone Worker storage is inaccessible",
)
})?;
let record = workers.load(id).map_err(|error| match error {
let record = match workers.load(id) {
Err(StandaloneStoreError::WorkerNotFound(_)) => {
workers = StandaloneWorkerStore::open_existing(
self.state_root.join("subjektiv").join("retained-workers"),
)
.map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Original Session is not retained",
)
})?;
workers.load(id)
}
result => result,
}
.map_err(|error| match error {
StandaloneStoreError::WorkerNotFound(_) => failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Attributed standalone Worker record is missing",
@@ -262,6 +352,13 @@ impl Backend<'_> {
"Attributed Session identity is invalid",
)
})?;
let retention = crate::store::session_retention(self.state_root);
let _guard = retention.lock_live(id).map_err(|_| {
failure(
SubjektivSessionDiagnosticCode::RetentionMissing,
"Original Session is erased or its retention fence is unavailable",
)
})?;
let index = (self.reader)(&source.sessions_path, id, limits).map_err(index_failure)?;
if index.session_id != source.attribution.session_id {
return Err(failure(
@@ -1539,7 +1636,11 @@ allow = []
)
};
let sessions = f.index_path.parent().unwrap();
let fs_store = session_store::FsStore::new(sessions).unwrap();
let fs_store = session_store::FsStore::new_with_retention(
sessions,
crate::store::session_retention(&f.temp.path().join("workers")),
)
.unwrap();
let segment: SessionId = f.segment.parse().unwrap();
fs_store
.create_segment(
+151 -6
View File
@@ -1,5 +1,7 @@
//! T-709: real standalone Hosts and test-owned processes, with scripted models.
//! These tests do not create a Workspace or contact a model provider.
//! Integration of real standalone Subject lifecycle, dialog, Journal and Memory services.
//! Only model streams are scripted; temporary storage and test-owned subprocesses isolate
//! persistence and kernel leases. Run with `cargo test -p standalone --test subjektiv`.
//! No Workspace or live model provider is used; this is not a continuity meaning evaluation.
use std::collections::{BTreeMap, VecDeque};
use std::io::{Read, Write};
@@ -354,6 +356,17 @@ impl LlmClient for ScriptedClient {
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<LlmEvent, ClientError>> + Send>>, ClientError>
{
// Journal is a separate scripted model boundary. Successful production recording
// is required before these Memory scenarios may hand the Subject to a new body.
if request
.tools
.iter()
.any(|tool| tool.name == "ReadJournalState")
{
return Ok(Box::pin(stream::iter(
journal_response(&request).into_iter().map(Ok),
)));
}
// Lifecycle-triggered consolidation uses a separate Internal Worker. It
// must not consume the dialog script or silently become a live request.
let job = request
@@ -411,6 +424,134 @@ impl LlmClient for ScriptedClient {
}
}
// The outer model script is not a continuity meaning evaluation. Persistence, complete
// source delivery and the handoff gate remain the production implementations.
fn journal_response(request: &Request) -> Vec<LlmEvent> {
let result = |id: &str| -> Option<Value> {
request.items.iter().find_map(|item| match item {
Item::ToolResult {
call_id,
content,
is_error,
summary,
..
} if call_id == id => {
assert!(!is_error, "{id}: {summary}; {content:?}");
Some(
content
.as_deref()
.map(|v| serde_json::from_str(v).unwrap())
.unwrap_or(Value::Null),
)
}
_ => None,
})
};
if result("journal-submit").is_some() {
return prose();
}
if result("journal-checkpoint").is_some() {
return tool("journal-submit", "SubmitJobResult", json!({"result":{}}));
}
let Some(state) = result("journal-state") else {
return tool("journal-state", "ReadJournalState", json!({}));
};
let state = &state["value"];
let generation = state["generation_id"].as_str().unwrap();
if let Some(position) = result("journal-position") {
return tool(
"journal-checkpoint",
"CheckpointJournal",
json!({
"request_id":format!("checkpoint:{generation}"), "expected_state":position["value"]["state_token"],
"action":{"action":"checkpoint","record_operation_ids":[format!("record:{generation}")]},
"covered_ranges":state["selected_ranges"]
}),
);
}
if let Some(record) = result("journal-record") {
return tool(
"journal-position",
"UpdateJournalPosition",
json!({
"request_id":format!("position:{generation}"), "expected_state":record["value"]["state_token"],
"action":{"action":"update_position","current_position":"Memory scenario recorded; represented work may remain unfinished"}, "covered_ranges":[]
}),
);
}
for (index, range) in state["selected_ranges"]
.as_array()
.unwrap()
.iter()
.enumerate()
{
let mut offset = range["bytes"]["start"].as_u64().unwrap();
let mut token = Value::Null;
loop {
let id = format!("journal-source-{index}-{offset}");
match result(&id) {
Some(page) => match page["value"]["next_byte"].as_u64() {
Some(next) => {
assert!(next > offset);
offset = next;
token = page["value"]["content_token"].clone();
}
None => break,
},
None => {
return tool(
&id,
"ReadJournalText",
json!({
"target":{"target":"source","pointer":range["source"]["pointer"]},
"offset":offset,"limit":subjektiv::journal::MAX_JOURNAL_READ_BYTES,"content_token":token
}),
);
}
}
}
}
let (name, action) = if state["current_open"].is_null() {
(
"CreateJournalEntry",
json!({"action":"create","title":"Memory conversation","body":"Conversation recorded by the scripted model fixture","ongoing":"Unfinished work is not declared complete","state":"open","source_end":null,"correction_refs":[]}),
)
} else {
(
"RewriteJournalEntry",
json!({"action":"rewrite","entry_id":state["current_open"]["id"],"title":"Memory conversation","body":"Further conversation recorded by the scripted model fixture","ongoing":"Unfinished work is not declared complete"}),
)
};
tool(
"journal-record",
name,
json!({
"request_id":format!("record:{generation}"), "expected_state":state["current_journal"]["state_token"],
"action":action,"covered_ranges":state["selected_ranges"]
}),
)
}
async fn disconnect_recorded(host: &mut StandaloneHost) {
let mut last_error = None;
let completed = tokio::time::timeout(Duration::from_secs(30), async {
loop {
match host.disconnect_subject().await {
Ok(()) => return,
Err(error) => {
last_error = Some(error.to_string());
tokio::task::yield_now().await;
}
}
}
})
.await;
assert!(
completed.is_ok(),
"actual Idle/Journal completion is required before Memory scenario handoff: {last_error:?}"
);
}
fn prose() -> Vec<LlmEvent> {
vec![
LlmEvent::text_block_start(0),
@@ -750,7 +891,7 @@ async fn explicit_remember_is_committed_staging_and_sessions_follow_subject_acro
tool("query-restored", "SubjektivMemoryQuery", json!({})),
prose(),
]);
let restored =
let mut restored =
StandaloneHost::restore_with_model_client(state.clone(), worker_id, restored_model.clone())
.await
.unwrap();
@@ -761,6 +902,7 @@ async fn explicit_remember_is_committed_staging_and_sessions_follow_subject_acro
assert!(results["recall"].to_string().contains(claim));
assert_eq!(results["query-restored"]["items"], json!([]));
assert!(format!("{:?}", restored_model.requests()[0].items).contains("Remember this lesson"));
disconnect_recorded(&mut restored).await;
restored.shutdown().await.unwrap();
let second_model = ScriptedClient::new(vec![
@@ -980,7 +1122,7 @@ async fn kernel_subject_lease_excludes_other_processes_and_recovers_after_abnorm
let independent = catalog.create("independent", None).unwrap();
drop(catalog);
let ready = temp.path().join("ready");
let parent = StandaloneHost::start_with_model_client(
let mut parent = StandaloneHost::start_with_model_client(
launch(&parent_cwd, &state, &subject.id, "parent-owner"),
ScriptedClient::new(Vec::new()),
)
@@ -1002,6 +1144,7 @@ async fn kernel_subject_lease_excludes_other_processes_and_recovers_after_abnorm
.await
.unwrap();
independent_host.shutdown().await.unwrap();
disconnect_recorded(&mut parent).await;
parent.shutdown().await.unwrap();
let mut owner = OwnedChild(
@@ -1070,7 +1213,7 @@ async fn automatic_generic_consolidation_applies_candidate_publishes_surface_and
drop(catalog);
let claim = "Validate the immutable result before reporting success";
let model = ScriptedClient::consolidating(remember_script(claim), false);
let host = StandaloneHost::start_with_model_client(
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "consolidation-body"),
model.clone(),
)
@@ -1145,6 +1288,7 @@ async fn automatic_generic_consolidation_applies_candidate_publishes_surface_and
.exists(),
"selected Subject owns the shared generic Job ledger"
);
disconnect_recorded(&mut host).await;
host.shutdown().await.unwrap();
let memory = inspect_store(&state, |store| {
assert!(
@@ -1224,7 +1368,7 @@ async fn transport_loss_after_candidate_apply_preserves_memory_and_later_job_onl
drop(catalog);
let claim = "Candidate receipts survive transport failure";
let losing = ScriptedClient::consolidating(remember_script(claim), true);
let host = StandaloneHost::start_with_model_client(
let mut host = StandaloneHost::start_with_model_client(
launch(temp.path(), &state, &subject.id, "losing-body"),
losing.clone(),
)
@@ -1249,6 +1393,7 @@ async fn transport_loss_after_candidate_apply_preserves_memory_and_later_job_onl
failed.result.is_none(),
"partial apply is not an accepted Job result"
);
disconnect_recorded(&mut host).await;
host.shutdown().await.unwrap();
let original = inspect_store(&state, |store| {
assert!(
+4 -1
View File
@@ -1,7 +1,9 @@
# subjektiv
Shared Host domain implementation for per-Subject Memory, change history, candidate dispositions, and derived surfaces.
Shared Host domain implementation for per-Subject Memory, change history, candidate dispositions, derived surfaces, and Session activity Journals.
`feature-storage` manages physical SQLite storage; the calling Host handles authentication and execution authority.
Journal storage does not generate prose, launch Jobs, or fetch Session history.
Hosts retain body ownership across inactivity, reserve actual execution quiescence, and reconcile final committed history before releasing a Journal writer lifetime. [`journal::binding`] fences new mutations inside storage transactions; historical attribution and exact receipt recovery cannot renew a released lifetime. [`journal::startup`] selects bounded reference context independently of source retention and long-term Memory.
## Main entry points
@@ -9,3 +11,4 @@ Shared Host domain implementation for per-Subject Memory, change history, candid
- [`MemoryRecord`], [`SurfaceSnapshot`]: change history and derived context
- [`api`]: shared Host operations and bounded responses
- [`job`]: consolidation selection rules and result validation
- [`journal`]: individually committed activity operations, source coverage, retry receipts, and bounded reading
+4 -2
View File
@@ -11,8 +11,10 @@
mod context;
mod operation;
pub use context::{HostOperationContext, JobAttemptBinding};
pub use operation::execute;
/// Shared Journal domain operations; no model tools or Host-specific transport.
pub use crate::journal;
pub use context::{HostOperationContext, JobAttemptBinding, JournalHostContext, JournalJobContext};
pub use operation::{checkpoint_journal, execute, execute_journal};
use serde::{Deserialize, Serialize};
use std::collections::HashSet;
+2 -1
View File
@@ -1,8 +1,9 @@
//! Trusted Host operation context, never deserialized from model input.
use super::{OperationError, OperationResult, subjektiv_store_error};
pub use crate::journal::{JournalHostContext, JournalJobContext};
use crate::{SubjectSessionAttribution, SubjektivStore};
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct JobAttemptBinding {
pub job_id: String,
pub attempt_id: String,
+18
View File
@@ -1,4 +1,22 @@
use super::*;
/// Commits one Journal effect with its exact retry response, using Journal-only authority.
pub fn execute_journal(
store: &crate::SubjektivStore,
context: &JournalJobContext,
operation: journal::JournalOperation,
) -> journal::JournalResult<journal::JournalReceipt> {
store.execute_journal(context, operation)
}
/// Advances only selected input backed by already committed Journal effects.
pub fn checkpoint_journal(
store: &crate::SubjektivStore,
context: &JournalJobContext,
operation: journal::JournalOperation,
) -> journal::JournalResult<journal::JournalReceipt> {
store.checkpoint_journal(context, operation)
}
use server_api::{
SubjektivMemoryBackendOperation as Op, SubjektivMemoryBackendResponse as Response,
};
+205
View File
@@ -0,0 +1,205 @@
//! Durable Journal writer lifetime, issued only from the Host's body binding. Kernel leases
//! and Runtime liveness do not end ownership. The immutable binding ID fences retained contexts
//! even when the same Worker later reconnects. Hosts serialize release with body lifecycle and
//! reconcile final committed history before calling release; this store checks saved effects.
use super::*;
use rusqlite::{Connection, OptionalExtension, params};
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalWriterBinding {
pub subject_id: String,
pub runtime_id: String,
pub worker_id: String,
pub binding_id: String,
}
pub(crate) fn migrate(tx: &Transaction<'_>) -> Result<(), FeatureStorageError> {
tx.execute_batch("CREATE TABLE journal_writer_bindings (
binding_id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
runtime_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
active INTEGER NOT NULL CHECK(active IN (0,1))
);
CREATE UNIQUE INDEX journal_one_body_binding ON journal_writer_bindings(subject_id) WHERE active=1;
CREATE TABLE journal_session_bindings (
session_id TEXT PRIMARY KEY REFERENCES subject_session_attributions(session_id) ON DELETE RESTRICT,
binding_id TEXT NOT NULL REFERENCES journal_writer_bindings(binding_id) ON DELETE RESTRICT
);
CREATE INDEX journal_body_history ON journal_writer_bindings(subject_id,active);
CREATE INDEX journal_body_sessions ON journal_session_bindings(binding_id);
CREATE INDEX journal_recent_subject ON journals(subject_id,updated_at DESC,id DESC);")?;
Ok(())
}
fn denied() -> JournalError {
JournalError::PermissionDenied("Journal writer binding is absent, released or replaced".into())
}
pub(super) fn check_writer(tx: &Connection, host: &JournalHostContext) -> JournalResult<()> {
let attr = host.attribution.as_ref().ok_or_else(denied)?;
let binding = host.writer_binding_id.as_ref().ok_or_else(denied)?;
if !tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings b JOIN journal_session_bindings s ON s.binding_id=b.binding_id WHERE b.binding_id=?1 AND b.subject_id=?2 AND b.runtime_id=?3 AND b.worker_id=?4 AND b.active=1 AND s.session_id=?5)", params![binding,host.subject_id,attr.runtime_id,attr.worker_id,attr.session_id], |r| r.get::<_,bool>(0))? {
return Err(denied());
}
Ok(())
}
pub(super) fn require_saved_position(tx: &Connection, generation: &str) -> JournalResult<()> {
let saved: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_operations p WHERE p.generation_id=?1 AND json_extract(p.request_json,'$.action.action')='update_position' AND p.rowid > COALESCE((SELECT MAX(r.rowid) FROM journal_operations r WHERE r.generation_id=?1 AND json_extract(r.request_json,'$.action.action') IN ('create','rewrite','fix','confirm_unchanged')),0))",[generation],|r|r.get(0))?;
if !saved {
return Err(JournalError::Conflict("Journal current position must be saved after the generation's activity records before completion".into()));
}
Ok(())
}
impl SubjektivStore {
/// The last released body's most recently updated Journal Session, independent of source
/// retention. Session attachment order may differ from the order of recorded activity.
pub fn journal_predecessor_session(
&self,
binding: &JournalWriterBinding,
) -> JournalResult<Option<String>> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT j.source_session_id FROM journals j JOIN journal_session_bindings s ON s.session_id=j.source_session_id WHERE j.subject_id=?1 AND s.binding_id=(SELECT binding_id FROM journal_writer_bindings WHERE subject_id=?1 AND active=0 ORDER BY rowid DESC LIMIT 1) ORDER BY j.updated_at DESC,j.id DESC LIMIT 1", [&binding.subject_id], |r| r.get(0)).optional()?)
})
}
pub fn journal_worker_was_released(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3 AND active=0)", params![subject,runtime,worker], |r| r.get(0))?)
})
}
pub fn journal_writer_active(&self, binding: &JournalWriterBinding) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE binding_id=?1 AND subject_id=?2 AND runtime_id=?3 AND worker_id=?4 AND active=1)", params![binding.binding_id,binding.subject_id,binding.runtime_id,binding.worker_id], |r| r.get(0))?)
})
}
pub fn journal_session_writer_matches(
&self,
binding: &JournalWriterBinding,
session: &str,
) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
Ok(tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_session_bindings WHERE binding_id=?1 AND session_id=?2)", params![binding.binding_id,session], |r| r.get(0))?)
})
}
/// Called under the existing Host Subject lifecycle authority. A stopped or unavailable
/// owner remains the owner; only the exact same body may resume an unreleased binding.
pub fn claim_journal_writer(
&self,
subject: &str,
runtime: &str,
worker: &str,
) -> JournalResult<JournalWriterBinding> {
for identifier in [subject, runtime, worker] {
if identifier.is_empty()
|| identifier.len() > 128
|| identifier.trim() != identifier
|| identifier.chars().any(char::is_control)
{
return Err(JournalError::InvalidInput(
"invalid Journal binding identity".into(),
));
}
}
self.database.try_transaction(|tx| {
let state: Option<String> = tx.query_row("SELECT state FROM subjects WHERE subject_id=?1", [subject], |r| r.get(0)).optional()?;
if state.as_deref() != Some("active") { return Err(JournalError::PermissionDenied("Subject is absent or retired".into())); }
let current: Option<JournalWriterBinding> = tx.query_row("SELECT subject_id,runtime_id,worker_id,binding_id FROM journal_writer_bindings WHERE subject_id=?1 AND active=1", [subject], |r| Ok(JournalWriterBinding { subject_id:r.get(0)?,runtime_id:r.get(1)?,worker_id:r.get(2)?,binding_id:r.get(3)? })).optional()?;
if let Some(current) = current {
if current.runtime_id != runtime || current.worker_id != worker {
return Err(JournalError::Conflict("Subject is still occupied; explicit Idle and Journal-complete release required".into()));
}
return Ok(current);
}
// Upgraded T-734 data can contain a removed body's durable recording handoff.
// Retention is not permission for a different body to occupy that Subject.
let legacy_busy: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journals j JOIN subject_session_attributions a ON a.session_id=j.source_session_id WHERE j.subject_id=?1 AND (a.runtime_id<>?2 OR a.worker_id<>?3) AND NOT EXISTS(SELECT 1 FROM journal_session_bindings b WHERE b.session_id=j.source_session_id) AND (EXISTS(SELECT 1 FROM journal_generations g WHERE g.journal_id=j.id AND g.active=1) OR EXISTS(SELECT 1 FROM journal_sources s,json_each(s.accepted_json) r WHERE s.journal_id=j.id AND json_extract(r.value,'$.end')>s.reflected_bytes)))", params![subject,runtime,worker], |r| r.get(0))?;
if legacy_busy { return Err(JournalError::Conflict("previous body's retained Journal handoff is not complete".into())); }
let binding = JournalWriterBinding { subject_id:subject.into(),runtime_id:runtime.into(),worker_id:worker.into(),binding_id:crate::issued_id("journal-binding") };
tx.execute("INSERT INTO journal_writer_bindings(binding_id,subject_id,runtime_id,worker_id,active) VALUES(?1,?2,?3,?4,1)", params![binding.binding_id,subject,runtime,worker])?;
Ok(binding)
})
}
/// Binds a new Session once. Reconnecting a Worker under a later binding cannot acquire
/// mutation authority for its old Session. Attribution and binding are Host-only input.
pub fn bind_journal_session(
&self,
binding: &JournalWriterBinding,
attribution: SubjectSessionAttribution,
) -> JournalResult<JournalHostContext> {
let mut host = JournalHostContext::validated_session(self, attribution)?;
self.database.try_transaction(|tx| {
let attr = host.attribution.as_ref().expect("validated Session");
if binding.subject_id != attr.subject_id || binding.runtime_id != attr.runtime_id || binding.worker_id != attr.worker_id {
return Err(denied());
}
tx.execute("INSERT OR IGNORE INTO journal_session_bindings(session_id,binding_id) SELECT ?1,binding_id FROM journal_writer_bindings WHERE binding_id=?2 AND subject_id=?3 AND runtime_id=?4 AND worker_id=?5 AND active=1",params![attr.session_id,binding.binding_id,attr.subject_id,attr.runtime_id,attr.worker_id])?;
host.writer_binding_id = Some(binding.binding_id.clone());
check_writer(tx,&host)?;
Ok(host.clone())
})
}
/// Destructive release is distinct from a continuity handoff: every attributed Session
/// of this body must already have a permanent erasure tombstone. This also records a
/// released lifetime for pre-binding bodies, without granting them new mutation authority.
pub fn release_erased_journal_body(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_transaction(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
let remaining: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM subject_session_attributions a WHERE a.subject_id=?1 AND a.runtime_id=?2 AND a.worker_id=?3 AND NOT EXISTS(SELECT 1 FROM journal_erased_sessions e WHERE e.session_id=a.session_id AND e.subject_id=a.subject_id))", params![host.subject_id,attr.runtime_id,attr.worker_id], |r| r.get(0))?;
if remaining {
return Ok(false);
}
let existing: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_writer_bindings WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3)", params![host.subject_id,attr.runtime_id,attr.worker_id], |r| r.get(0))?;
tx.execute("UPDATE journal_writer_bindings SET active=0 WHERE subject_id=?1 AND runtime_id=?2 AND worker_id=?3",params![host.subject_id,attr.runtime_id,attr.worker_id])?;
if !existing {
tx.execute("INSERT INTO journal_writer_bindings(binding_id,subject_id,runtime_id,worker_id,active) VALUES(?1,?2,?3,?4,0)", params![crate::issued_id("journal-erased-body"),host.subject_id,attr.runtime_id,attr.worker_id])?;
}
Ok(true)
})
}
/// The Host must hold its lifecycle fence, confirm Idle, reconcile all committed original
/// Sessions (including the save/notification gap), and exclude queued/running/Unknown Jobs.
/// This transaction rejects retained writers, pending source bytes and active generations.
/// It never fixes an open entry or declares the represented work complete.
pub fn release_journal_writer(&self, binding: &JournalWriterBinding) -> JournalResult<()> {
self.database.try_transaction(|tx| {
let active: Option<bool> = tx.query_row("SELECT active FROM journal_writer_bindings WHERE binding_id=?1 AND subject_id=?2 AND runtime_id=?3 AND worker_id=?4",params![binding.binding_id,binding.subject_id,binding.runtime_id,binding.worker_id],|r|r.get(0)).optional()?;
let Some(active) = active else { return Err(denied()); };
if !active { return Ok(()); }
let unbound: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM subject_session_attributions a WHERE a.subject_id=?1 AND a.runtime_id=?2 AND a.worker_id=?3 AND NOT EXISTS(SELECT 1 FROM journal_session_bindings s WHERE s.session_id=a.session_id) AND NOT EXISTS(SELECT 1 FROM journal_erased_sessions e WHERE e.session_id=a.session_id AND e.subject_id=a.subject_id))",params![binding.subject_id,binding.runtime_id,binding.worker_id],|r|r.get(0))?;
if unbound { return Err(JournalError::Conflict("body has legacy Sessions whose committed history has not been reconciled under its writer binding".into())); }
let busy: bool = tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_generations g JOIN journals j ON j.id=g.journal_id JOIN journal_session_bindings s ON s.session_id=j.source_session_id WHERE s.binding_id=?1 AND g.active=1)",[&binding.binding_id],|r|r.get(0))?;
if busy { return Err(JournalError::Conflict("Journal generation remains active or its result is unconfirmed".into())); }
let mut stmt = tx.prepare("SELECT s.accepted_json,s.reflected_bytes,s.unavailable_json FROM journal_sources s JOIN journals j ON j.id=s.journal_id JOIN journal_session_bindings b ON b.session_id=j.source_session_id WHERE b.binding_id=?1")?;
let mut rows = stmt.query([&binding.binding_id])?;
while let Some(row) = rows.next()? {
let accepted: Vec<JournalByteRange> = serde_json::from_str(&row.get::<_,String>(0)?)?;
let reflected: u64 = row.get(1)?;
if accepted.iter().any(|r| r.end > reflected) {
return Err(JournalError::Conflict("Journal activity/position/checkpoint has unreflected source bytes".into()));
}
}
let mut checkpoints = tx.prepare("SELECT p.generation_id FROM journal_operations p JOIN journals j ON j.id=p.journal_id JOIN journal_session_bindings b ON b.session_id=j.source_session_id WHERE b.binding_id=?1 AND p.rowid=(SELECT MAX(c.rowid) FROM journal_operations c WHERE c.journal_id=j.id AND json_extract(c.request_json,'$.action.action')='checkpoint')")?;
let generations = checkpoints.query_map([&binding.binding_id], |r| r.get::<_,String>(0))?;
for generation in generations { require_saved_position(tx, &generation?)?; }
tx.execute("UPDATE journal_writer_bindings SET active=0 WHERE binding_id=?1 AND active=1",[&binding.binding_id])?;
Ok(())
})
}
}
+153
View File
@@ -0,0 +1,153 @@
//! Journal-specific retention and explicit erasure. Hosts reconcile the final committed public
//! Session before consulting retention; absence from the source ledger is not proof of no work.
//! Ordinary retention never deletes Journal text. Explicit erasure leaves a content-free Session
//! tombstone in the same transaction as removal, fencing delayed admission and Job operations.
use super::*;
use crate::now;
use rusqlite::{Connection, OptionalExtension, params};
pub(crate) fn migrate(tx: &Transaction<'_>) -> Result<(), FeatureStorageError> {
tx.execute_batch(
"ALTER TABLE journal_generations ADD COLUMN completion_json TEXT;
CREATE TABLE journal_generation_deliveries (
generation_id TEXT NOT NULL REFERENCES journal_generations(id) ON DELETE CASCADE,
source_order INTEGER NOT NULL,
ranges_json TEXT NOT NULL,
PRIMARY KEY(generation_id, source_order)
);
CREATE TABLE journal_erased_sessions (
session_id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL,
request_id TEXT NOT NULL UNIQUE,
erased_at TEXT NOT NULL
);
DROP TRIGGER journal_operations_no_delete;
CREATE TRIGGER journal_operations_no_delete BEFORE DELETE ON journal_operations
WHEN NOT EXISTS (
SELECT 1 FROM journals j JOIN journal_erased_sessions e
ON e.session_id=j.source_session_id AND e.subject_id=j.subject_id
WHERE j.id=OLD.journal_id
)
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are retained'); END;",
)?;
Ok(())
}
pub(super) fn reject_erased(tx: &Connection, host: &JournalHostContext) -> JournalResult<()> {
let session = host
.attribution
.as_ref()
.ok_or_else(|| JournalError::PermissionDenied("Session authority required".into()))?;
if tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1)",
[&session.session_id],
|row| row.get::<_, bool>(0),
)? {
return Err(JournalError::PermissionDenied(
"Session was explicitly erased".into(),
));
}
Ok(())
}
impl SubjektivStore {
/// Reads the content-free erasure fence under the original Session attribution.
/// Hosts use this before source reconciliation so ordinary removal of an erased Session
/// neither fetches erased material nor forces a new recording generation.
pub fn journal_session_erased(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
Ok(tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1 AND subject_id=?2)",
params![attr.session_id, host.subject_id],
|row| row.get(0),
)?)
})
}
/// Finds this Host's immutable source Session without scanning unrelated Journals.
pub fn journal_for_session(&self, host: &JournalHostContext) -> JournalResult<Option<String>> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
reject_erased(tx, host)?;
let attr = host
.attribution
.as_ref()
.expect("checked Session authority");
Ok(tx
.query_row(
"SELECT id FROM journals WHERE source_session_id=?1 AND subject_id=?2",
params![attr.session_id, host.subject_id],
|row| row.get(0),
)
.optional()?)
})
}
/// True while any admitted bytes still need reflection, including unavailable bytes.
/// A Host must first reconcile storage against admission under its deletion/append fence.
pub fn journal_retention_required(&self, host: &JournalHostContext) -> JournalResult<bool> {
self.database.try_with_connection(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
if tx.query_row("SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE session_id=?1)", [&attr.session_id], |r| r.get::<_,bool>(0))? {
return Ok(false);
}
let mut stmt = tx.prepare("SELECT s.accepted_json,s.reflected_bytes FROM journal_sources s JOIN journals j ON j.id=s.journal_id WHERE j.source_session_id=?1 AND j.subject_id=?2 ORDER BY s.source_order")?;
let mut rows = stmt.query(params![attr.session_id,host.subject_id])?;
while let Some(row) = rows.next()? {
let accepted: Vec<JournalByteRange> = serde_json::from_str(&row.get::<_, String>(0)?)?;
let reflected: u64 = row.get(1)?;
if accepted.iter().any(|range| range.end > reflected) {
return Ok(true);
}
}
Ok(false)
})
}
/// Explicit Host-authorized complete erasure, not an ordinary Worker removal policy.
/// No generation is forced. The tombstone remains even if no Journal was admitted yet.
/// Exact retries converge; a request ID cannot be reused to erase a different Session.
pub fn erase_journal_session(
&self,
host: &JournalHostContext,
request_id: &str,
) -> JournalResult<()> {
if request_id.is_empty()
|| request_id.len() > 128
|| request_id.chars().any(char::is_control)
{
return Err(JournalError::InvalidInput(
"invalid erasure request identifier".into(),
));
}
self.database.try_transaction(|tx| {
super::store::check_session_host(self, tx, host)?;
let attr = host.attribution.as_ref().expect("checked Session authority");
let previous: Option<(String,String)> = tx.query_row("SELECT session_id,subject_id FROM journal_erased_sessions WHERE request_id=?1", [request_id], |r| Ok((r.get(0)?,r.get(1)?))).optional()?;
if previous.as_ref().is_some_and(|(session,subject)| session!=&attr.session_id || subject!=&host.subject_id) {
return Err(JournalError::Conflict("erasure request belongs to a different Session".into()));
}
let erased_by: Option<String> = tx.query_row("SELECT request_id FROM journal_erased_sessions WHERE session_id=?1", [&attr.session_id], |row| row.get(0)).optional()?;
if erased_by.as_deref().is_some_and(|key| key != request_id) {
return Err(JournalError::Conflict("Session already erased by a different request".into()));
}
tx.execute("INSERT OR IGNORE INTO journal_erased_sessions(session_id,subject_id,request_id,erased_at) VALUES(?1,?2,?3,?4)", params![attr.session_id,host.subject_id,request_id,now()])?;
let journal: Option<String> = tx.query_row("SELECT id FROM journals WHERE source_session_id=?1 AND subject_id=?2",params![attr.session_id,host.subject_id],|r|r.get(0)).optional()?;
if let Some(id) = journal {
// The erasure marker is the only condition permitting receipt deletion. Normal
// record/checkpoint/retention paths retain their immutable receipts.
tx.execute("DELETE FROM journal_operations WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_generations WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_corrections WHERE entry_id IN (SELECT id FROM journal_entries WHERE journal_id=?1) OR target_id IN (SELECT id FROM journal_entries WHERE journal_id=?1)",[&id])?;
tx.execute("DELETE FROM journal_entries WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journal_sources WHERE journal_id=?1",[&id])?;
tx.execute("DELETE FROM journals WHERE id=?1",[&id])?;
}
Ok(())
})
}
}
@@ -0,0 +1,668 @@
//! Additive, immutable provenance-to-context links, not additional evidence for a Memory claim.
//!
//! Reconciliation reads only saved candidate anchors and the Host-attested Journal source ledger.
//! It never fetches history, starts Workers, changes extraction/consolidation, fixes an open event,
//! or mutates Memory. A stable evidence reference identifies a public entry, not its legacy numeric
//! display range. Missing stable identity stays unconnected; numeric ranges are not guessed to be
//! canonical order. Saved edges remain usable without the original runtime and cannot be retargeted.
//! Journal content is never copied here. Explicit erasure retains edges and exposes the tombstone.
use super::{
JournalError, JournalHostContext, JournalResult, JournalSourcePointer, JournalTextField,
JournalTextPage, MAX_JOURNAL_PAGE, MAX_JOURNAL_RESPONSE_BYTES,
};
use crate::{MAX_STAGING_ANCHORS, SubjectStagingRecord, SubjektivStore};
use memory::schema::SourceEvidenceRef;
use rusqlite::{Connection, OptionalExtension, Transaction, params};
use serde::{Deserialize, Serialize};
#[cfg(test)]
mod tests;
pub(crate) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("schema.sql"))?;
Ok(())
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalSelector {
pub memory_id: String,
pub change_id: String,
pub candidate_id: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case", deny_unknown_fields)]
pub enum MemoryJournalRequest {
Reconcile {
selector: MemoryJournalSelector,
},
/// Reads saved link state only; does not implicitly retry reconciliation.
Links {
selector: MemoryJournalSelector,
},
/// Continues reconciliation over immutable source indices, at most MAX_STAGING_ANCHORS per call.
ReconcilePage {
selector: MemoryJournalSelector,
source_offset: usize,
limit: usize,
},
/// Reads another saved source page without reconciliation.
LinksPage {
selector: MemoryJournalSelector,
source_offset: usize,
limit: usize,
},
Read {
selector: MemoryJournalSelector,
source_index: usize,
journal_id: String,
entry_id: String,
expected_token: Option<String>,
byte_offset: usize,
max_bytes: usize,
},
Related {
journal_id: String,
entry_id: String,
after: Option<RelatedMemoryCursor>,
limit: usize,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(
tag = "kind",
content = "value",
rename_all = "snake_case",
deny_unknown_fields
)]
pub enum MemoryJournalResponse {
Links(MemoryJournalLinks),
Read(MemoryJournalRead),
Related(RelatedMemories),
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum OriginalSourceAvailability {
/// No saved Host attestation. None of these values imply a live history fetch.
Unknown,
Missing,
Available,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalTargetAvailability {
Available,
Unavailable,
Erased,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum UnconnectedReason {
NotReconciled,
InsufficientSource,
SourceNotAdmitted,
AmbiguousSource,
JournalNotFixed,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "state", rename_all = "snake_case", deny_unknown_fields)]
pub enum SourceConnection {
Unconnected {
reason: UnconnectedReason,
},
Linked {
journal_id: String,
entry_id: String,
target: JournalTargetAvailability,
/// A saved correction invites reconsideration, never an automatic Memory edit.
reconsideration: bool,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalSourceLink {
pub source_index: usize,
pub original_source: OriginalSourceAvailability,
pub connection: SourceConnection,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalLinks {
pub selector: MemoryJournalSelector,
pub sources: Vec<MemoryJournalSourceLink>,
pub total_sources: usize,
pub next_source_index: Option<usize>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MemoryJournalRead {
pub source: MemoryJournalSourceLink,
/// Absent for an unavailable or explicitly erased target; never falls back to another entry.
pub page: Option<JournalTextPage>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemoryCursor {
pub memory_id: String,
pub change_id: String,
pub candidate_id: String,
pub source_index: usize,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemory {
pub selector: MemoryJournalSelector,
pub source_index: usize,
pub reconsideration: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct RelatedMemories {
pub items: Vec<RelatedMemory>,
pub next: Option<RelatedMemoryCursor>,
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
fn identifier(value: &str) -> JournalResult<()> {
if value.is_empty() || value.len() > 256 || value.chars().any(char::is_control) {
return Err(invalid("invalid Memory/Journal selector identifier"));
}
Ok(())
}
fn selector_valid(selector: &MemoryJournalSelector) -> JournalResult<()> {
identifier(&selector.memory_id)?;
identifier(&selector.change_id)?;
identifier(&selector.candidate_id)
}
fn candidate(
store: &SubjektivStore,
tx: &Connection,
host: &JournalHostContext,
selector: &MemoryJournalSelector,
) -> JournalResult<SubjectStagingRecord> {
super::store::check_host(store, tx, host, false)?;
selector_valid(selector)?;
// The immutable change-candidate edge, not the current Memory projection or a supplied
// candidate ID alone, authorizes this provenance selector.
let raw: Option<String> = tx
.query_row(
"SELECT s.record_json FROM staging_records s JOIN memory_change_candidates c
ON c.subject_id=s.subject_id AND c.candidate_id=s.candidate_id
WHERE c.subject_id=?1 AND c.memory_id=?2 AND c.change_id=?3 AND c.candidate_id=?4",
params![
host.subject_id,
selector.memory_id,
selector.change_id,
selector.candidate_id
],
|r| r.get(0),
)
.optional()?;
let record: SubjectStagingRecord = serde_json::from_str(
&raw.ok_or_else(|| JournalError::NotFound("Memory change candidate membership".into()))?,
)?;
Ok(record)
}
#[derive(Debug)]
struct SavedLink {
reason: String,
journal_id: Option<String>,
entry_id: Option<String>,
session_id: Option<String>,
order: Option<u64>,
}
fn saved(
tx: &Connection,
host: &JournalHostContext,
candidate_id: &str,
index: usize,
) -> JournalResult<Option<SavedLink>> {
Ok(tx.query_row(
"SELECT reason,journal_id,entry_id,source_session_id,source_order FROM memory_journal_links
WHERE subject_id=?1 AND candidate_id=?2 AND source_index=?3",
params![host.subject_id, candidate_id, index],
|r| Ok(SavedLink { reason:r.get(0)?,journal_id:r.get(1)?,entry_id:r.get(2)?,session_id:r.get(3)?,order:r.get(4)? }),
).optional()?)
}
fn original_availability(
tx: &Connection,
journal: &str,
order: u64,
) -> JournalResult<OriginalSourceAvailability> {
let unavailable: Option<String> = tx
.query_row(
"SELECT unavailable_json FROM journal_sources WHERE journal_id=?1 AND source_order=?2",
params![journal, order],
|r| r.get(0),
)
.optional()?;
Ok(match unavailable {
None => OriginalSourceAvailability::Unknown,
Some(raw) => {
let ranges: Vec<super::JournalByteRange> = serde_json::from_str(&raw)?;
if ranges.is_empty() {
OriginalSourceAvailability::Available
} else {
OriginalSourceAvailability::Missing
}
}
})
}
fn project(
tx: &Connection,
host: &JournalHostContext,
candidate_id: &str,
index: usize,
) -> JournalResult<MemoryJournalSourceLink> {
let Some(link) = saved(tx, host, candidate_id, index)? else {
return Ok(MemoryJournalSourceLink {
source_index: index,
original_source: OriginalSourceAvailability::Unknown,
connection: SourceConnection::Unconnected {
reason: UnconnectedReason::NotReconciled,
},
});
};
if link.reason != "linked" {
let reason = match link.reason.as_str() {
"insufficient_source" => UnconnectedReason::InsufficientSource,
"source_not_admitted" => UnconnectedReason::SourceNotAdmitted,
"ambiguous_source" => UnconnectedReason::AmbiguousSource,
"journal_not_fixed" => UnconnectedReason::JournalNotFixed,
_ => return Err(invalid("invalid saved connection reason")),
};
return Ok(MemoryJournalSourceLink {
source_index: index,
original_source: OriginalSourceAvailability::Unknown,
connection: SourceConnection::Unconnected { reason },
});
}
let journal_id = link
.journal_id
.ok_or_else(|| invalid("saved link lacks Journal"))?;
let entry_id = link
.entry_id
.ok_or_else(|| invalid("saved link lacks entry"))?;
let session = link
.session_id
.ok_or_else(|| invalid("saved link lacks Session"))?;
let order = link
.order
.ok_or_else(|| invalid("saved link lacks source order"))?;
let erased: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_erased_sessions WHERE subject_id=?1 AND session_id=?2)",
params![host.subject_id, session], |r| r.get(0),
)?;
let exists: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_entries e JOIN journals j ON j.id=e.journal_id
WHERE e.subject_id=?1 AND e.journal_id=?2 AND e.id=?3 AND j.source_session_id=?4)",
params![host.subject_id, journal_id, entry_id, session],
|r| r.get(0),
)?;
let reconsideration = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_corrections WHERE subject_id=?1 AND target_id=?2)",
params![host.subject_id, entry_id],
|r| r.get(0),
)?;
Ok(MemoryJournalSourceLink {
source_index: index,
original_source: original_availability(tx, &journal_id, order)?,
connection: SourceConnection::Linked {
journal_id,
entry_id,
target: if erased {
JournalTargetAvailability::Erased
} else if exists {
JournalTargetAvailability::Available
} else {
JournalTargetAvailability::Unavailable
},
reconsideration,
},
})
}
struct Resolved {
journal: String,
entry: String,
session: String,
order: u64,
}
fn resolve(
tx: &Connection,
host: &JournalHostContext,
source: &SourceEvidenceRef,
) -> JournalResult<Result<Resolved, &'static str>> {
let Some(evidence) = source.evidence_id.as_deref() else {
return Ok(Err("insufficient_source"));
};
if !evidence.starts_with('E')
|| evidence.len() < 2
|| evidence.len() > 129
|| !evidence[1..]
.bytes()
.all(|c| c.is_ascii_alphanumeric() || c == b'-')
|| (source.session_id.is_none() && source.segment_id.is_none())
{
return Ok(Err("insufficient_source"));
}
// Segment identity may be omitted only when an immutable Session attribution disambiguates it.
// A candidate's primary legacy source range is not substituted for an individual anchor.
let segment = match source.segment_id.as_deref() {
Some(raw) => match uuid::Uuid::parse_str(raw) {
Ok(id) => Some(id),
Err(_) => return Ok(Err("insufficient_source")),
},
None => None,
};
let mut stmt = tx.prepare(
"SELECT j.id,j.source_session_id,s.source_order,s.pointer_json FROM journal_sources s
JOIN journals j ON j.id=s.journal_id
JOIN subject_session_attributions a ON a.subject_id=j.subject_id AND a.session_id=j.source_session_id
WHERE j.subject_id=?1 AND s.entry_ref=?2
AND (?3 IS NULL OR j.source_session_id=?3)
AND (?4 IS NULL OR json_extract(s.pointer_json,'$.segment_id')=?4)
LIMIT 2",
)?;
let found = stmt
.query_map(
params![
host.subject_id,
evidence,
source.session_id,
segment.map(|s| s.to_string())
],
|r| {
Ok((
r.get::<_, String>(0)?,
r.get::<_, String>(1)?,
r.get::<_, u64>(2)?,
r.get::<_, String>(3)?,
))
},
)?
.collect::<Result<Vec<_>, _>>()?;
if found.is_empty() {
return Ok(Err("source_not_admitted"));
}
if found.len() != 1 {
return Ok(Err("ambiguous_source"));
}
let (journal, session, order, pointer) = &found[0];
let pointer: JournalSourcePointer = serde_json::from_str(pointer)?;
if pointer.entry_ref != evidence || segment.is_some_and(|s| s != pointer.segment_id) {
return Ok(Err("ambiguous_source"));
}
// A fixed endpoint covers (previous_fixed_end, source_end]. The Session start is the first
// interval's lower boundary. Processing checkpoints and open entries cannot supply an end.
let mut stmt = tx.prepare(
"SELECT e.id FROM journal_entries e WHERE e.subject_id=?1 AND e.journal_id=?2
AND e.state='fixed' AND e.source_order>=?3
AND NOT EXISTS(SELECT 1 FROM journal_entries p WHERE p.journal_id=e.journal_id
AND p.position<e.position AND p.state='fixed' AND p.source_order>=?3)
ORDER BY e.position LIMIT 2",
)?;
let entries = stmt
.query_map(params![host.subject_id, journal, order], |r| {
r.get::<_, String>(0)
})?
.collect::<Result<Vec<_>, _>>()?;
match entries.as_slice() {
[entry] => Ok(Ok(Resolved {
journal: journal.clone(),
entry: entry.clone(),
session: session.clone(),
order: *order,
})),
[] => Ok(Err("journal_not_fixed")),
_ => Ok(Err("ambiguous_source")),
}
}
fn reconcile(
tx: &Connection,
host: &JournalHostContext,
record: &SubjectStagingRecord,
indices: std::ops::Range<usize>,
) -> JournalResult<()> {
for index in indices {
let source = &record.source_refs[index];
if saved(tx, host, &record.id, index)?.is_some_and(|s| s.reason == "linked") {
continue;
}
let (reason, resolved) = match resolve(tx, host, source)? {
Ok(link) => ("linked", Some(link)),
Err(reason) => (reason, None),
};
tx.execute(
"INSERT INTO memory_journal_links(subject_id,candidate_id,source_index,reason,
journal_id,entry_id,source_session_id,source_order) VALUES(?1,?2,?3,?4,?5,?6,?7,?8)
ON CONFLICT(subject_id,candidate_id,source_index) DO UPDATE SET
reason=excluded.reason,journal_id=excluded.journal_id,entry_id=excluded.entry_id,
source_session_id=excluded.source_session_id,source_order=excluded.source_order",
params![
host.subject_id,
record.id,
index,
reason,
resolved.as_ref().map(|r| r.journal.as_str()),
resolved.as_ref().map(|r| r.entry.as_str()),
resolved.as_ref().map(|r| r.session.as_str()),
resolved.as_ref().map(|r| r.order)
],
)?;
}
Ok(())
}
impl SubjektivStore {
/// Scoped Memory provenance bridge. Reconcile is additive and idempotent, so a lost response
/// needs no operation ledger: retry the same selector and reread saved edges. Reads never reconcile.
pub fn execute_memory_journal(
&self,
host: &JournalHostContext,
request: MemoryJournalRequest,
) -> JournalResult<MemoryJournalResponse> {
let response = match request {
MemoryJournalRequest::Reconcile { selector } => {
self.memory_journal_links(host, selector, true, 0, MAX_STAGING_ANCHORS)?
}
MemoryJournalRequest::Links { selector } => {
self.memory_journal_links(host, selector, false, 0, MAX_STAGING_ANCHORS)?
}
MemoryJournalRequest::ReconcilePage {
selector,
source_offset,
limit,
} => self.memory_journal_links(host, selector, true, source_offset, limit)?,
MemoryJournalRequest::LinksPage {
selector,
source_offset,
limit,
} => self.memory_journal_links(host, selector, false, source_offset, limit)?,
MemoryJournalRequest::Read {
selector,
source_index,
journal_id,
entry_id,
expected_token,
byte_offset,
max_bytes,
} => {
identifier(&journal_id)?;
identifier(&entry_id)?;
if !(4..=super::MAX_JOURNAL_READ_BYTES).contains(&max_bytes) {
return Err(invalid("text page must be 4..16384 bytes"));
}
if let Some(token) = expected_token.as_deref() {
identifier(token)?;
}
let mut source = self.database.try_with_connection(|tx| {
let record = candidate(self, tx, host, &selector)?;
if source_index >= record.source_refs.len() {
return Err(invalid("source index outside candidate anchors"));
}
let link = project(tx, host, &selector.candidate_id, source_index)?;
match &link.connection {
SourceConnection::Linked {
journal_id: saved_journal,
entry_id: saved_entry,
..
} if saved_journal == &journal_id && saved_entry == &entry_id => Ok(link),
_ => Err(JournalError::PermissionDenied(
"read requires the exact saved Journal link".into(),
)),
}
})?;
let page = match &source.connection {
SourceConnection::Linked {
target: JournalTargetAvailability::Available,
..
} => {
match self.read_journal_text(
host,
&journal_id,
Some(&entry_id),
JournalTextField::Body,
expected_token.as_deref(),
byte_offset,
max_bytes,
) {
Ok(page) => Some(page),
// Erasure between the saved-edge check and shared text read must not
// fall back, expose cached text, or pretend the link never existed.
Err(JournalError::NotFound(_)) => {
source = self.database.try_with_connection(|tx| {
project(tx, host, &selector.candidate_id, source_index)
})?;
None
}
Err(error) => return Err(error),
}
}
_ => None,
};
MemoryJournalResponse::Read(MemoryJournalRead { source, page })
}
MemoryJournalRequest::Related {
journal_id,
entry_id,
after,
limit,
} => self.related_memories(host, &journal_id, &entry_id, after.as_ref(), limit)?,
};
if serde_json::to_string_pretty(&response)?.len() > MAX_JOURNAL_RESPONSE_BYTES {
return Err(invalid("Memory Journal response exceeds byte budget"));
}
Ok(response)
}
fn memory_journal_links(
&self,
host: &JournalHostContext,
selector: MemoryJournalSelector,
do_reconcile: bool,
source_offset: usize,
limit: usize,
) -> JournalResult<MemoryJournalResponse> {
if !(1..=MAX_STAGING_ANCHORS).contains(&limit) {
return Err(invalid("source page exceeds staging anchor bound"));
}
self.database.try_transaction(|tx| {
let record = candidate(self, tx, host, &selector)?;
let total_sources = record.source_refs.len();
if source_offset > total_sources {
return Err(invalid("source offset outside immutable candidate anchors"));
}
let end = source_offset.saturating_add(limit).min(total_sources);
if do_reconcile {
reconcile(tx, host, &record, source_offset..end)?;
}
let sources = (source_offset..end)
.map(|index| project(tx, host, &selector.candidate_id, index))
.collect::<JournalResult<Vec<_>>>()?;
Ok(MemoryJournalResponse::Links(MemoryJournalLinks {
selector,
sources,
total_sources,
next_source_index: (end < total_sources).then_some(end),
}))
})
}
fn related_memories(
&self,
host: &JournalHostContext,
journal_id: &str,
entry_id: &str,
after: Option<&RelatedMemoryCursor>,
limit: usize,
) -> JournalResult<MemoryJournalResponse> {
identifier(journal_id)?;
identifier(entry_id)?;
if !(1..=MAX_JOURNAL_PAGE).contains(&limit) {
return Err(invalid("related page must be 1..32"));
}
if let Some(cursor) = after {
identifier(&cursor.memory_id)?;
identifier(&cursor.change_id)?;
identifier(&cursor.candidate_id)?;
if i64::try_from(cursor.source_index).is_err() {
return Err(invalid("invalid related source index"));
}
}
self.database.try_with_connection(|tx| {
super::store::check_host(self,tx,host,false)?;
// Saved target identity authorizes reverse lookup after erasure as well as before it.
let known:bool=tx.query_row(
"SELECT EXISTS(SELECT 1 FROM journal_entries WHERE subject_id=?1 AND journal_id=?2 AND id=?3)
OR EXISTS(SELECT 1 FROM memory_journal_links WHERE subject_id=?1 AND journal_id=?2 AND entry_id=?3)",
params![host.subject_id,journal_id,entry_id],|r|r.get(0))?;
if !known { return Err(JournalError::NotFound("Journal entry".into())); }
let mut stmt=tx.prepare(
"SELECT m.memory_id,m.change_id,l.candidate_id,l.source_index,
EXISTS(SELECT 1 FROM journal_corrections c WHERE c.subject_id=l.subject_id AND c.target_id=l.entry_id)
FROM memory_journal_links l JOIN memory_change_candidates m
ON m.subject_id=l.subject_id AND m.candidate_id=l.candidate_id
WHERE l.subject_id=?1 AND l.reason='linked'
AND ((l.journal_id=?2 AND l.entry_id=?3)
OR EXISTS(SELECT 1 FROM journal_corrections c WHERE c.subject_id=?1 AND c.entry_id=?3 AND c.target_id=l.entry_id))
AND (?4 IS NULL OR (m.memory_id,m.change_id,l.candidate_id,l.source_index)>(?4,?5,?6,?7))
ORDER BY m.memory_id,m.change_id,l.candidate_id,l.source_index LIMIT ?8")?;
let mut items=stmt.query_map(params![host.subject_id,journal_id,entry_id,
after.map(|c|c.memory_id.as_str()),after.map(|c|c.change_id.as_str()),
after.map(|c|c.candidate_id.as_str()),after.map(|c|c.source_index),limit+1],
|r|Ok(RelatedMemory {selector:MemoryJournalSelector {memory_id:r.get(0)?,change_id:r.get(1)?,candidate_id:r.get(2)?},
source_index:r.get(3)?,reconsideration:r.get(4)?}))?.collect::<Result<Vec<_>,_>>()?;
let more=items.len()>limit;
items.truncate(limit);
let next=if more {items.last().map(|last|RelatedMemoryCursor {
memory_id:last.selector.memory_id.clone(),change_id:last.selector.change_id.clone(),
candidate_id:last.selector.candidate_id.clone(),source_index:last.source_index })} else {None};
Ok(MemoryJournalResponse::Related(RelatedMemories {items,next}))
})
}
}
@@ -0,0 +1,33 @@
-- References outlive explicit Journal erasure. Do not add target-row foreign keys:
-- lifecycle erasure deletes Journal content but retains its Session tombstone.
CREATE TABLE memory_journal_links (
subject_id TEXT NOT NULL,
candidate_id TEXT NOT NULL,
source_index INTEGER NOT NULL CHECK(source_index >= 0),
reason TEXT NOT NULL CHECK(reason IN (
'insufficient_source', 'source_not_admitted', 'ambiguous_source',
'journal_not_fixed', 'linked'
)),
journal_id TEXT,
entry_id TEXT,
source_session_id TEXT,
source_order INTEGER,
PRIMARY KEY(subject_id, candidate_id, source_index),
FOREIGN KEY(subject_id, candidate_id)
REFERENCES staging_records(subject_id, candidate_id) ON DELETE RESTRICT,
CHECK((reason = 'linked' AND journal_id IS NOT NULL AND entry_id IS NOT NULL
AND source_session_id IS NOT NULL AND source_order IS NOT NULL AND source_order >= 0)
OR (reason <> 'linked' AND journal_id IS NULL AND entry_id IS NULL
AND source_session_id IS NULL AND source_order IS NULL))
);
CREATE INDEX memory_journal_links_reverse
ON memory_journal_links(subject_id, journal_id, entry_id, candidate_id, source_index);
CREATE TRIGGER memory_journal_links_identity BEFORE UPDATE ON memory_journal_links
WHEN OLD.subject_id <> NEW.subject_id OR OLD.candidate_id <> NEW.candidate_id
OR OLD.source_index <> NEW.source_index
BEGIN SELECT RAISE(ABORT, 'Journal source reference identity is immutable'); END;
CREATE TRIGGER memory_journal_links_connected_immutable BEFORE UPDATE ON memory_journal_links
WHEN OLD.reason = 'linked'
BEGIN SELECT RAISE(ABORT, 'connected Journal references are immutable'); END;
CREATE TRIGGER memory_journal_links_retained BEFORE DELETE ON memory_journal_links
BEGIN SELECT RAISE(ABORT, 'Journal references are retained'); END;
File diff suppressed because it is too large Load Diff
+414
View File
@@ -0,0 +1,414 @@
//! Durable activity records owned by Subjektiv, independent of Session retention.
//!
//! Hosts attest committed public source identity and canonical Session order, not models. Byte
//! ranges are half-open in the Host's canonical UTF-8 public-entry serialization (including
//! framing, so even an empty message has a nonzero length). Compaction copies must retain the
//! original pointer/order/digest; derived summaries are not new sources. Rewind replacement is
//! an identity conflict, never an implicit reset. This module does not fetch or retain history.
//!
//! Each record operation and its exact retry receipt commit together. A separate checkpoint
//! advances only a contiguous, selected prefix backed by saved record/explicit no-change
//! receipts. It cannot use a position update as evidence. Stopping between these boundaries
//! leaves saved records and unreflected input; read receipts before retrying, even across reopen
//! or replacement Jobs. No whole-Job result submission is required. The Host checks live Job
//! execution leases before issuing contexts, and durably closes a generation on cancellation.
//!
//! Selected ranges describe concrete public bytes supplied to generation, not a preview's
//! enclosing history range or a permission to claim unobserved bytes. The Host verifies canonical
//! serialization, SHA-256 identity and UTF-8 slice boundaries before admission/selection. This
//! store validates identities, byte bounds and coverage but cannot attest text it does not fetch.
//! Availability is the latest Host attestation; readers must not equate it with a live fetch.
//! Public text and metadata reads are paged. Full generation/resume snapshots are Host-only
//! construction material bounded by per-field storage caps (three text fields plus current position),
//! not model-facing reads. A restored snapshot never restores an external Job lease.
//!
//! A T-734 caller saves `Create`/`Rewrite`/`Fix` under stable request IDs, carries each returned
//! `state_token` into the next operation, optionally saves `UpdatePosition`, then submits
//! `Checkpoint { record_operation_ids }` for just the reflected prefix. After a lost response it
//! queries the original receipt, status and coverage pages before submitting any new create.
//! On restart it reads the active generation and current resume state, verifies the external Job
//! binding, and continues missing operations; cancellation closes the generation before replacement.
//! A replacement may checkpoint old saved record receipts only for identical selected source bytes.
//! Tokens incorporate prior saved content conditions as well as current content, preventing A→B→A
//! edits from resurrecting an old operation premise after intervening saved effects.
//! Source-ledger pagination uses an opaque, scope/Subject/Journal-bound content cursor: late gap
//! admission, accepted-fragment extension, reflection, or availability changes reject continuation.
//! The fingerprint and bounded projection share one transaction; hashing streams bounded rows.
//! Hosts resuming at the processing cursor can use a bounded source point lookup instead of
//! traversing already reflected pages. Receipt/entry keysets remain append-stable.
pub mod binding;
pub use binding::JournalWriterBinding;
pub(crate) use binding::migrate as migrate_binding;
mod lifecycle;
pub mod memory_links;
pub(crate) use lifecycle::migrate as migrate_lifecycle;
pub mod recording;
pub mod service;
mod source_status;
pub mod startup;
mod store;
#[cfg(test)]
mod tests;
pub mod transport;
use crate::{SubjectSessionAttribution, SubjektivStore, api::JobAttemptBinding};
use feature_storage::FeatureStorageError;
use rusqlite::Transaction;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
pub type JournalId = String;
pub type JournalEntryId = String;
pub type JournalResult<T> = Result<T, JournalError>;
pub const MAX_JOURNAL_TEXT_BYTES: usize = 1024 * 1024;
pub const MAX_JOURNAL_RANGES: usize = 64;
pub const MAX_JOURNAL_PAGE: usize = 32;
pub const MAX_JOURNAL_READ_BYTES: usize = 16 * 1024;
/// Whole pretty-serialized public page budget, including escaped text and metadata.
pub const MAX_JOURNAL_RESPONSE_BYTES: usize = 64 * 1024;
/// Stable owner of Backend source retention, shared by attribution-time protection
/// and later source admission/release. This identifier confers no read authority.
/// Local standalone storage uses its scope-exclusive Session UUID as the owner.
pub fn source_retention_pin(workspace: &str, subject: &str, session: &str) -> Uuid {
use sha2::{Digest, Sha256};
let digest = Sha256::digest(
serde_json::to_vec(&(
"subjektiv-journal-retention-v1",
workspace,
subject,
session,
))
.expect("string tuple serialization"),
);
let mut bytes = [0; 16];
bytes.copy_from_slice(&digest[..16]);
Uuid::from_bytes(bytes)
}
#[derive(Debug, thiserror::Error)]
pub enum JournalError {
#[error("journal permission denied: {0}")]
PermissionDenied(String),
#[error("journal conflict: {0}")]
Conflict(String),
#[error("invalid journal input: {0}")]
InvalidInput(String),
#[error("journal record not found: {0}")]
NotFound(String),
#[error(transparent)]
Storage(#[from] FeatureStorageError),
#[error(transparent)]
Json(#[from] serde_json::Error),
#[error(transparent)]
Subject(#[from] crate::SubjektivError),
}
impl From<rusqlite::Error> for JournalError {
fn from(e: rusqlite::Error) -> Self {
Self::Storage(e.into())
}
}
/// Host-issued in-process authority, never a model request DTO. Reader contexts need no live Session.
#[derive(Debug, Clone)]
pub struct JournalHostContext {
scope_id: String,
subject_id: String,
attribution: Option<SubjectSessionAttribution>,
writer_binding_id: Option<String>,
}
impl JournalHostContext {
/// Evidence for a retained Job grant, not permission to mint or renew a binding.
pub fn writer_binding_id(&self) -> Option<&str> {
self.writer_binding_id.as_deref()
}
pub fn validated_reader(store: &SubjektivStore, subject_id: &str) -> JournalResult<Self> {
store
.subject(subject_id)?
.ok_or_else(|| JournalError::NotFound(subject_id.into()))?;
Ok(Self {
scope_id: store.scope_id().into(),
subject_id: subject_id.into(),
attribution: None,
writer_binding_id: None,
})
}
/// External authentication, public-history validation and execution authority are Host duties.
pub fn validated_session(
store: &SubjektivStore,
attribution: SubjectSessionAttribution,
) -> JournalResult<Self> {
if store.session_attribution(&attribution.session_id)?.as_ref() != Some(&attribution) {
return Err(JournalError::PermissionDenied(
"immutable Session attribution mismatch".into(),
));
}
let mut context = Self::validated_reader(store, &attribution.subject_id)?;
use rusqlite::OptionalExtension;
context.writer_binding_id = store.database.try_with_connection(|tx| {
Ok::<_, JournalError>(
tx.query_row(
"SELECT binding_id FROM journal_session_bindings WHERE session_id=?1",
[&attribution.session_id],
|r| r.get(0),
)
.optional()?,
)
})?;
context.attribution = Some(attribution);
Ok(context)
}
}
/// A restricted Journal generation, not Memory consolidation authority.
#[derive(Debug, Clone)]
pub struct JournalJobContext {
host: JournalHostContext,
generation_id: String,
binding: JobAttemptBinding,
}
impl JournalJobContext {
pub fn validated(
store: &SubjektivStore,
host: &JournalHostContext,
generation_id: &str,
binding: JobAttemptBinding,
) -> JournalResult<Self> {
let context = Self {
host: host.clone(),
generation_id: generation_id.into(),
binding,
};
store.validate_journal_job(&context)?;
Ok(context)
}
/// Reconstructs a retained binding for exact completion/receipt recovery. Hosts still check
/// the external attempt lease. New record/checkpoint operations independently require an
/// active generation inside their transaction, so this never revives cancelled authority.
pub fn validated_recovery(
store: &SubjektivStore,
host: &JournalHostContext,
generation_id: &str,
binding: JobAttemptBinding,
) -> JournalResult<Self> {
let context = Self {
host: host.clone(),
generation_id: generation_id.into(),
binding,
};
store.database.try_with_connection(|tx| {
self::store::job(store, tx, &context, false)?;
Ok::<_, JournalError>(())
})?;
Ok(context)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSourcePointer {
pub segment_id: Uuid,
/// Serialized public SessionEntryRef, including its `E` prefix.
pub entry_ref: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSource {
pub session_id: String,
pub pointer: JournalSourcePointer,
/// Host-attested canonical public entry order across Segments, starting at zero.
pub order: u64,
pub content_sha256: String,
pub total_bytes: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalByteRange {
pub start: u64,
pub end: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalSourceRange {
pub source: JournalSource,
pub bytes: JournalByteRange,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalEntryState {
Open,
Fixed,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Journal {
pub id: JournalId,
pub subject_id: String,
pub source_session_id: String,
pub title: String,
pub current_position: String,
/// Processing cursor is independent of every fixed event endpoint.
pub process_order: u64,
pub process_byte: u64,
/// Content condition advanced by committed operations, never by acceptance notifications.
pub state_token: String,
pub created_at: String,
pub updated_at: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalMetadata {
pub id: JournalId,
pub subject_id: String,
pub source_session_id: String,
pub title: String,
pub current_position_bytes: usize,
pub process_order: u64,
pub process_byte: u64,
pub state_token: String,
pub created_at: String,
pub updated_at: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalEntry {
pub id: JournalEntryId,
pub journal_id: JournalId,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub body: String,
pub ongoing: String,
pub source_end: Option<JournalSourcePointer>,
pub created_at: String,
pub updated_at: String,
pub fixed_at: Option<String>,
pub correction_refs: Vec<JournalEntryId>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalGeneration {
pub id: String,
pub journal_id: JournalId,
pub job: JobAttemptBinding,
pub selected_ranges: Vec<JournalSourceRange>,
pub initial_journal: Journal,
pub initial_open: Option<JournalEntry>,
pub previous_fixed_end: Option<JournalSourcePointer>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalAction {
Create {
title: String,
body: String,
ongoing: String,
state: JournalEntryState,
source_end: Option<JournalSourcePointer>,
correction_refs: Vec<JournalEntryId>,
},
Rewrite {
entry_id: JournalEntryId,
title: String,
body: String,
ongoing: String,
},
Fix {
entry_id: JournalEntryId,
source_end: JournalSourcePointer,
},
UpdatePosition {
current_position: String,
},
/// Explicitly records the caller's checked conclusion that no entry change is needed.
ConfirmUnchanged,
/// Links durable record/no-change receipts to actual reflected input; IDs are Journal scoped.
Checkpoint {
record_operation_ids: Vec<String>,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalOperation {
pub request_id: String,
pub expected_state: String,
pub action: JournalAction,
pub covered_ranges: Vec<JournalSourceRange>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalReceipt {
pub request_id: String,
pub generation_id: String,
pub entry_id: Option<JournalEntryId>,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalSavedOperation {
pub operation: JournalOperation,
pub receipt: JournalReceipt,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalOperationKind {
Create,
Rewrite,
Fix,
UpdatePosition,
ConfirmUnchanged,
Checkpoint,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalOperationStatus {
pub receipt: JournalReceipt,
pub kind: JournalOperationKind,
pub covered_range_count: usize,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalSourceStatus {
pub source: JournalSource,
pub accepted: Vec<JournalByteRange>,
pub unavailable: Vec<JournalByteRange>,
pub reflected_bytes: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalPage<T> {
pub items: Vec<T>,
pub next: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalEntrySummary {
pub id: JournalEntryId,
pub journal_id: JournalId,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub source_end: Option<JournalSourcePointer>,
/// Exclusive start; None means Session start. The end is inclusive.
pub previous_fixed_end: Option<JournalSourcePointer>,
pub source_available: bool,
pub correction_refs: Vec<JournalEntryId>,
pub content_token: String,
pub created_at: String,
pub updated_at: String,
pub fixed_at: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalTextField {
Body,
Ongoing,
CurrentPosition,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalTextPage {
pub text: String,
pub next_byte: Option<usize>,
pub content_token: String,
}
pub(super) fn migrate(tx: &Transaction<'_>) -> feature_storage::Result<()> {
tx.execute_batch(include_str!("schema.sql"))?;
Ok(())
}
+194
View File
@@ -0,0 +1,194 @@
//! Host-side construction of bounded Journal generation input.
//!
//! Admission and byte identity belong to the Session Host. Selection neither starts Jobs nor
//! treats previews, failed fetches, or missing bytes as reflected. Production readers verify
//! original canonical identity while streaming and return only the requested bounded fragment.
use super::*;
use sha2::{Digest, Sha256};
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalMaterial {
pub range: JournalSourceRange,
pub text: String,
}
/// Adapter for Host captures already bounded by their storage read contract. Filesystem/Runtime
/// callers should use [`select_materials_bounded`] rather than allocating a whole large entry.
pub fn select_materials(
store: &SubjektivStore,
host: &JournalHostContext,
journal_id: &str,
max_bytes: usize,
mut read: impl FnMut(&JournalSource) -> JournalResult<String>,
) -> JournalResult<Vec<JournalMaterial>> {
select_materials_bounded(
store,
host,
journal_id,
max_bytes,
|source, start, limit| {
let text = read(source)?;
verify_canonical_text(source, &text)?;
let start = usize::try_from(start).map_err(|_| invalid("source offset overflow"))?;
if !text.is_char_boundary(start) {
return Err(conflict("source offset is not UTF-8 aligned"));
}
let mut end = start.saturating_add(limit).min(text.len());
while !text.is_char_boundary(end) {
end -= 1;
}
Ok(JournalMaterial {
range: JournalSourceRange {
source: source.clone(),
bytes: JournalByteRange {
start: start as u64,
end: end as u64,
},
},
text: text[start..end].into(),
})
},
)
}
fn invalid(message: &str) -> JournalError {
JournalError::InvalidInput(message.into())
}
fn conflict(message: &str) -> JournalError {
JournalError::Conflict(message.into())
}
/// Selects the oldest contiguous input under a whole serialized materials-array budget.
/// The Host callback verifies original public provenance/digest/length while streaming, and
/// returns at most `limit` canonical UTF-8 bytes at exactly `start`. Its descriptor must equal
/// the admitted source. This in-process attestation must never be deserialized from model input.
pub fn select_materials_bounded(
store: &SubjektivStore,
host: &JournalHostContext,
journal_id: &str,
max_bytes: usize,
mut read: impl FnMut(&JournalSource, u64, usize) -> JournalResult<JournalMaterial>,
) -> JournalResult<Vec<JournalMaterial>> {
let journal = store.read_journal(host, journal_id)?;
let mut order = journal.process_order;
let mut offset = journal.process_byte;
let mut materials = Vec::new();
let mut remaining = max_bytes.saturating_sub(2);
while materials.len() < MAX_JOURNAL_RANGES {
let status = match store.read_journal_source(host, journal_id, order) {
Ok(status) => status,
Err(JournalError::NotFound(_)) => break,
Err(error) => return Err(error),
};
if offset == status.source.total_bytes {
order = order
.checked_add(1)
.ok_or_else(|| invalid("source order exhausted"))?;
offset = 0;
continue;
}
let Some(accepted) = status
.accepted
.iter()
.find(|r| r.start <= offset && offset < r.end)
else {
break;
};
let mut end = accepted.end;
for unavailable in &status.unavailable {
if unavailable.start <= offset && offset < unavailable.end {
end = offset;
break;
}
if offset < unavailable.start {
end = end.min(unavailable.start);
}
}
if end == offset {
break;
}
let empty = JournalMaterial {
range: JournalSourceRange {
source: status.source.clone(),
bytes: JournalByteRange {
start: offset,
end: offset,
},
},
text: String::new(),
};
let envelope = serde_json::to_vec(&empty)?.len() + usize::from(!materials.is_empty());
// Reserve decimal endpoint growth and enough room for one UTF-8 scalar.
let allowance = remaining.saturating_sub(envelope.saturating_add(20));
if allowance < 4 {
if materials.is_empty() {
return Err(invalid(
"input budget cannot fit source metadata and a UTF-8 scalar",
));
}
break;
}
let limit = allowance.min((end - offset).min(usize::MAX as u64) as usize);
let mut material = read(&status.source, offset, limit)?;
if material.range.source != status.source
|| material.range.bytes.start != offset
|| material.range.bytes.end != offset.saturating_add(material.text.len() as u64)
|| material.range.bytes.end > end
|| material.text.is_empty()
|| material.text.len() > limit
{
return Err(conflict(
"Host source chunk does not match the requested canonical byte range",
));
}
let mut allowance = remaining - envelope;
let mut selected_bytes = 0;
for ch in material.text.chars() {
let cost = match ch {
'"' | '\\' | '\u{8}' | '\u{c}' | '\n' | '\r' | '\t' => 2,
ch if ch < '\u{20}' => 6,
ch => ch.len_utf8(),
};
if cost.saturating_add(20) > allowance {
break;
}
allowance -= cost;
selected_bytes += ch.len_utf8();
}
if selected_bytes == 0 {
if materials.is_empty() {
return Err(invalid("input budget cannot fit one escaped UTF-8 scalar"));
}
break;
}
material.text.truncate(selected_bytes);
material.range.bytes.end = offset + selected_bytes as u64;
let cost = serde_json::to_vec(&material)?.len() + usize::from(!materials.is_empty());
if cost > remaining {
return Err(invalid("generation input budget accounting failed"));
}
remaining -= cost;
offset = material.range.bytes.end;
materials.push(material);
if offset < end {
break;
}
}
Ok(materials)
}
/// Verifies the full canonical public role/tool framing of an already bounded Host capture.
pub fn verify_canonical_text(source: &JournalSource, text: &str) -> JournalResult<()> {
if text.len() as u64 != source.total_bytes
|| Sha256::digest(text.as_bytes())
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>()
!= source.content_sha256
{
return Err(conflict("canonical public source bytes changed"));
}
Ok(())
}
+88
View File
@@ -0,0 +1,88 @@
CREATE TABLE journals (
id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL REFERENCES subjects(subject_id) ON DELETE RESTRICT,
source_session_id TEXT NOT NULL UNIQUE,
title TEXT NOT NULL,
current_position TEXT NOT NULL,
process_order INTEGER NOT NULL DEFAULT 0 CHECK (process_order >= 0),
process_byte INTEGER NOT NULL DEFAULT 0 CHECK (process_byte >= 0),
state_token TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(subject_id, id)
);
CREATE TABLE journal_entries (
id TEXT PRIMARY KEY,
subject_id TEXT NOT NULL,
journal_id TEXT NOT NULL,
position INTEGER NOT NULL CHECK (position >= 0),
state TEXT NOT NULL CHECK (state IN ('open', 'fixed')),
title TEXT NOT NULL,
body TEXT NOT NULL,
ongoing TEXT NOT NULL,
source_end_json TEXT,
source_order INTEGER,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
fixed_at TEXT,
UNIQUE(journal_id, position),
UNIQUE(subject_id, id),
FOREIGN KEY(subject_id, journal_id) REFERENCES journals(subject_id, id) ON DELETE RESTRICT,
CHECK ((state = 'open' AND source_end_json IS NULL AND source_order IS NULL AND fixed_at IS NULL)
OR (state = 'fixed' AND source_end_json IS NOT NULL AND source_order IS NOT NULL AND source_order >= 0 AND fixed_at IS NOT NULL AND ongoing = ''))
);
CREATE UNIQUE INDEX journal_one_open ON journal_entries(journal_id) WHERE state = 'open';
CREATE TRIGGER journal_append_after_open BEFORE INSERT ON journal_entries
WHEN EXISTS (SELECT 1 FROM journal_entries WHERE journal_id = NEW.journal_id AND state = 'open')
BEGIN SELECT RAISE(ABORT, 'journal open must be the tail'); END;
CREATE TRIGGER journal_fixed_immutable BEFORE UPDATE ON journal_entries WHEN OLD.state = 'fixed'
BEGIN SELECT RAISE(ABORT, 'journal fixed entries are immutable'); END;
CREATE TRIGGER journal_entry_identity BEFORE UPDATE ON journal_entries
WHEN OLD.id <> NEW.id OR OLD.subject_id <> NEW.subject_id OR OLD.journal_id <> NEW.journal_id
OR OLD.position <> NEW.position OR OLD.created_at <> NEW.created_at
BEGIN SELECT RAISE(ABORT, 'journal entry identity is immutable'); END;
CREATE TABLE journal_corrections (
subject_id TEXT NOT NULL,
entry_id TEXT NOT NULL,
target_id TEXT NOT NULL,
PRIMARY KEY(entry_id, target_id),
FOREIGN KEY(subject_id, entry_id) REFERENCES journal_entries(subject_id, id) ON DELETE RESTRICT,
FOREIGN KEY(subject_id, target_id) REFERENCES journal_entries(subject_id, id) ON DELETE RESTRICT,
CHECK(entry_id <> target_id)
);
CREATE TABLE journal_sources (
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
source_order INTEGER NOT NULL CHECK(source_order >= 0),
pointer_json TEXT NOT NULL,
entry_ref TEXT NOT NULL,
source_json TEXT NOT NULL,
accepted_json TEXT NOT NULL,
unavailable_json TEXT NOT NULL,
reflected_bytes INTEGER NOT NULL DEFAULT 0 CHECK(reflected_bytes >= 0),
PRIMARY KEY(journal_id, source_order),
UNIQUE(journal_id, pointer_json),
UNIQUE(journal_id, entry_ref)
);
CREATE TABLE journal_generations (
id TEXT PRIMARY KEY,
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
job_id TEXT NOT NULL,
attempt_id TEXT NOT NULL,
request_json TEXT NOT NULL,
snapshot_json TEXT NOT NULL,
active INTEGER NOT NULL CHECK(active IN (0, 1)),
UNIQUE(journal_id, job_id, attempt_id)
);
CREATE UNIQUE INDEX journal_one_generation ON journal_generations(journal_id) WHERE active = 1;
CREATE TABLE journal_operations (
journal_id TEXT NOT NULL REFERENCES journals(id) ON DELETE RESTRICT,
request_id TEXT NOT NULL,
generation_id TEXT NOT NULL REFERENCES journal_generations(id) ON DELETE RESTRICT,
request_json TEXT NOT NULL,
receipt_json TEXT NOT NULL,
PRIMARY KEY(journal_id, request_id)
);
CREATE TRIGGER journal_operations_no_update BEFORE UPDATE ON journal_operations
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are immutable'); END;
CREATE TRIGGER journal_operations_no_delete BEFORE DELETE ON journal_operations
BEGIN SELECT RAISE(ABORT, 'journal operation receipts are retained'); END;
+424
View File
@@ -0,0 +1,424 @@
//! Shared sequential Job service. Backend and standalone authenticate and check the live attempt
//! before each call; the store additionally fences closed generations inside mutation transactions.
//! Job completion attests a saved processing checkpoint, not model prose or fixed event state.
use super::transport::*;
use super::*;
use rusqlite::{OptionalExtension, params};
use sha2::{Digest, Sha256};
fn metadata(j: &Journal) -> JournalMetadata {
JournalMetadata {
id: j.id.clone(),
subject_id: j.subject_id.clone(),
source_session_id: j.source_session_id.clone(),
title: j.title.clone(),
current_position_bytes: j.current_position.len(),
process_order: j.process_order,
process_byte: j.process_byte,
state_token: j.state_token.clone(),
created_at: j.created_at.clone(),
updated_at: j.updated_at.clone(),
}
}
fn text_page(
text: &str,
read: &JournalTextRead,
condition: &[u8],
maximum_end: usize,
) -> JournalResult<JournalTextPage> {
if !(1..=MAX_JOURNAL_READ_BYTES).contains(&read.limit)
|| read.offset > maximum_end
|| !text.is_char_boundary(read.offset)
{
return Err(JournalError::InvalidInput(
"invalid Journal UTF-8 read range".into(),
));
}
let token: String = Sha256::digest(condition)
.iter()
.map(|b| format!("{b:02x}"))
.collect();
if read
.content_token
.as_deref()
.is_some_and(|expected| expected != token)
|| (read.offset > 0 && read.content_token.is_none())
{
return Err(JournalError::Conflict(
"Journal text changed or continuation condition missing".into(),
));
}
let mut end = read.offset.saturating_add(read.limit).min(maximum_end);
while !text.is_char_boundary(end) {
end -= 1;
}
if end == read.offset && end < maximum_end {
return Err(JournalError::InvalidInput(
"read limit cannot fit one UTF-8 scalar".into(),
));
}
Ok(JournalTextPage {
text: text[read.offset..end].into(),
next_byte: (end < maximum_end).then_some(end),
content_token: token,
})
}
/// Adapter for Host captures bounded by their storage contract. Production source reads should
/// use [`execute_bounded`] to avoid allocating an entire source for one small read.
pub fn execute(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
mut read_source: impl FnMut(&JournalSource) -> JournalResult<String>,
) -> JournalResult<JournalJobResponse> {
execute_bounded(store, context, request, |source, start, limit| {
let text = read_source(source)?;
super::recording::verify_canonical_text(source, &text)?;
let start = usize::try_from(start)
.map_err(|_| JournalError::InvalidInput("source offset overflow".into()))?;
if !text.is_char_boundary(start) {
return Err(JournalError::InvalidInput(
"source offset is not UTF-8 aligned".into(),
));
}
let mut end = start.saturating_add(limit).min(text.len());
while !text.is_char_boundary(end) {
end -= 1;
}
Ok(recording::JournalMaterial {
range: JournalSourceRange {
source: source.clone(),
bytes: JournalByteRange {
start: start as u64,
end: end as u64,
},
},
text: text[start..end].into(),
})
})
}
/// Reads are delivered through this service before mutation authority is enabled. The durable
/// delivery ledger stores byte ranges only: Jobs may retain immutable descriptors without
/// accumulating a permanent second copy of the original Session in their request ledgers.
/// The Host verifies original provenance/digest/length and returns a bounded UTF-8 fragment.
pub fn execute_bounded(
store: &SubjektivStore,
context: &JournalJobContext,
request: JournalJobRequest,
mut read_source: impl FnMut(&JournalSource, u64, usize) -> JournalResult<recording::JournalMaterial>,
) -> JournalResult<JournalJobResponse> {
let generation = store
.database
.try_with_connection(|tx| super::store::job(store, tx, context, true))?;
let mut delivered = None;
let response = match request {
JournalJobRequest::Execute { operation } => {
require_delivery(store, context, &generation.selected_ranges)?;
let receipt = if matches!(operation.action, JournalAction::Checkpoint { .. }) {
crate::api::checkpoint_journal(store, context, operation)?
} else {
crate::api::execute_journal(store, context, operation)?
};
JournalJobResponse::Saved(receipt)
}
JournalJobRequest::ReadState => {
let state = store.database.try_with_connection(|tx| {
let generation = super::store::job(store, tx, context, true)?;
let current =
super::store::journal(tx, &context.host, &generation.journal_id, true)?;
let current_open = super::store::open(tx, &generation.journal_id)?;
Ok::<_, JournalError>(JournalJobState {
generation_id: generation.id,
initial_journal: metadata(&generation.initial_journal),
initial_open: generation
.initial_open
.map(|e| super::store::summary(tx, e))
.transpose()?,
current_journal: metadata(&current),
current_open: current_open
.map(|e| super::store::summary(tx, e))
.transpose()?,
previous_fixed_end: generation.previous_fixed_end,
selected_ranges: generation.selected_ranges,
})
})?;
JournalJobResponse::State(state)
}
JournalJobRequest::QueryOperation { request_id } => {
let status = match store.read_journal_operation_status(
&context.host,
&generation.journal_id,
&request_id,
) {
Ok(status) => Some(status),
Err(JournalError::NotFound(_)) => None,
Err(error) => return Err(error),
};
JournalJobResponse::Operation(status)
}
JournalJobRequest::ReadOperationCoverage {
request_id,
offset,
limit,
} => JournalJobResponse::Coverage(store.read_journal_operation_coverage(
&context.host,
&generation.journal_id,
&request_id,
offset,
limit,
)?),
JournalJobRequest::ListOperations { after, limit } => {
let receipts = store.list_journal_operations(
&context.host,
&generation.journal_id,
after.as_deref(),
limit,
)?;
let items = receipts
.items
.iter()
.map(|r| {
store.read_journal_operation_status(
&context.host,
&generation.journal_id,
&r.request_id,
)
})
.collect::<JournalResult<Vec<_>>>()?;
JournalJobResponse::Operations(JournalPage {
items,
next: receipts.next,
})
}
JournalJobRequest::ReadText { read } => {
let text = match &read.target {
JournalTextTarget::Source { pointer } => {
let range = generation
.selected_ranges
.iter()
.find(|r| {
&r.source.pointer == pointer
&& r.bytes.start <= read.offset as u64
&& read.offset as u64 <= r.bytes.end
})
.ok_or_else(|| {
JournalError::PermissionDenied(
"source bytes are outside this generation".into(),
)
})?;
if !(1..=MAX_JOURNAL_READ_BYTES).contains(&read.limit) {
return Err(JournalError::InvalidInput(
"invalid source read limit".into(),
));
}
let condition =
serde_json::to_vec(&(&generation.id, &range.source, range.bytes))?;
let token: String = Sha256::digest(&condition)
.iter()
.map(|b| format!("{b:02x}"))
.collect();
if read
.content_token
.as_deref()
.is_some_and(|expected| expected != token)
|| (read.offset as u64 != range.bytes.start && read.content_token.is_none())
{
return Err(JournalError::Conflict(
"source continuation condition missing or changed".into(),
));
}
let mut material =
read_source(&range.source, read.offset as u64, read.limit.max(4))?;
if material.range.source != range.source
|| material.range.bytes.start != read.offset as u64
|| material.range.bytes.end
!= (read.offset as u64).saturating_add(material.text.len() as u64)
|| material.range.bytes.end > range.source.total_bytes
|| material.text.len() > read.limit.max(4)
{
return Err(JournalError::Conflict(
"source fragment identity or byte range changed".into(),
));
}
let mut length = material
.text
.len()
.min(read.limit)
.min((range.bytes.end - read.offset as u64) as usize);
while !material.text.is_char_boundary(length) {
length -= 1;
}
if length == 0 && (read.offset as u64) < range.bytes.end {
return Err(JournalError::InvalidInput(
"read limit cannot fit a source UTF-8 scalar".into(),
));
}
material.text.truncate(length);
let end = read.offset + length;
if length > 0 {
delivered = Some((
range.source.order,
JournalByteRange {
start: read.offset as u64,
end: end as u64,
},
));
}
JournalTextPage {
text: material.text,
next_byte: ((end as u64) < range.bytes.end).then_some(end),
content_token: token,
}
}
target => {
let (journal, open) = match target {
JournalTextTarget::Position { snapshot }
| JournalTextTarget::OpenBody { snapshot }
| JournalTextTarget::OpenOngoing { snapshot } => match snapshot {
JournalSnapshot::Initial => (
generation.initial_journal.clone(),
generation.initial_open.clone(),
),
JournalSnapshot::Current => {
store.journal_resume_state(&context.host, &generation.journal_id)?
}
},
JournalTextTarget::Source { .. } => unreachable!(),
};
let text = match target {
JournalTextTarget::Position { .. } => journal.current_position.clone(),
JournalTextTarget::OpenBody { .. } => open
.as_ref()
.ok_or_else(|| JournalError::NotFound("no open entry".into()))?
.body
.clone(),
JournalTextTarget::OpenOngoing { .. } => open
.as_ref()
.ok_or_else(|| JournalError::NotFound("no open entry".into()))?
.ongoing
.clone(),
JournalTextTarget::Source { .. } => unreachable!(),
};
let condition = serde_json::to_vec(&(
&generation.id,
&read.target,
&journal.state_token,
&text,
))?;
text_page(&text, &read, &condition, text.len())?
}
};
JournalJobResponse::Text(text)
}
};
if serde_json::to_vec(&response)?.len() > MAX_JOURNAL_RESPONSE_BYTES {
return Err(JournalError::InvalidInput(
"Journal response exceeds bounded tool budget".into(),
));
}
if let Some((order, range)) = delivered {
store.database.try_transaction(|tx| {
// A response that cannot fit the tool budget or loses its live generation must
// not attest bytes. Lost responses can safely reread the identical source page.
super::store::job(store, tx, context, true)?;
let saved: Option<String> = tx.query_row(
"SELECT ranges_json FROM journal_generation_deliveries WHERE generation_id=?1 AND source_order=?2",
params![context.generation_id, order], |row| row.get(0),
).optional()?;
let mut ranges: Vec<JournalByteRange> = saved.map(|raw| serde_json::from_str(&raw)).transpose()?.unwrap_or_default();
ranges.push(range);
tx.execute(
"INSERT INTO journal_generation_deliveries(generation_id,source_order,ranges_json) VALUES(?1,?2,?3) ON CONFLICT(generation_id,source_order) DO UPDATE SET ranges_json=excluded.ranges_json",
params![context.generation_id, order, serde_json::to_string(&super::store::merge(ranges)?)?],
)?;
Ok::<_, JournalError>(())
})?;
}
Ok(response)
}
fn require_delivery(
store: &SubjektivStore,
context: &JournalJobContext,
selected: &[JournalSourceRange],
) -> JournalResult<()> {
store.database.try_with_connection(|tx| {
super::store::job(store, tx, context, true)?;
for range in selected {
let saved: Option<String> = tx.query_row(
"SELECT ranges_json FROM journal_generation_deliveries WHERE generation_id=?1 AND source_order=?2",
params![context.generation_id, range.source.order], |row| row.get(0),
).optional()?;
let delivered: Vec<JournalByteRange> = saved.map(|raw| serde_json::from_str(&raw)).transpose()?.unwrap_or_default();
if !delivered.iter().any(|bytes| bytes.start <= range.bytes.start && range.bytes.end <= bytes.end) {
return Err(JournalError::Conflict(
"read every selected source byte with ReadJournalText before recording or checkpointing".into(),
));
}
}
Ok(())
})
}
/// Atomically saves completion and fences new operations. Exact replay returns the same saved
/// completion even after later generations advance the Journal. Cancelled incomplete generations
/// cannot manufacture completion, and neither fixing an entry nor updating position can substitute
/// for the receipt-backed processing cursor.
pub fn finish(
store: &SubjektivStore,
context: &JournalJobContext,
) -> JournalResult<JournalJobCompletion> {
store.database.try_transaction(|tx| {
let generation = super::store::job(store, tx, context, false)?;
let saved: Option<String> = tx
.query_row(
"SELECT completion_json FROM journal_generations WHERE id=?1",
[&generation.id],
|row| row.get(0),
)
.optional()?
.flatten();
if let Some(saved) = saved {
return Ok(serde_json::from_str(&saved)?);
}
super::store::job(store, tx, context, true)?;
super::binding::require_saved_position(tx, &generation.id)?;
let last = generation
.selected_ranges
.last()
.ok_or_else(|| JournalError::InvalidInput("empty generation selection".into()))?;
let end = if last.bytes.end == last.source.total_bytes {
(
last.source
.order
.checked_add(1)
.ok_or_else(|| JournalError::InvalidInput("source order exhausted".into()))?,
0,
)
} else {
(last.source.order, last.bytes.end)
};
let journal = super::store::journal(tx, &context.host, &generation.journal_id, true)?;
if (journal.process_order, journal.process_byte) != end {
return Err(JournalError::Conflict(
"generation has unreflected selected bytes; save a receipt-backed checkpoint first"
.into(),
));
}
let completion = JournalJobCompletion {
generation_id: generation.id.clone(),
state_token: journal.state_token,
process_order: journal.process_order,
process_byte: journal.process_byte,
};
tx.execute(
"UPDATE journal_generations SET active=0,completion_json=?2 WHERE id=?1",
params![generation.id, serde_json::to_string(&completion)?],
)?;
Ok(completion)
})
}
@@ -0,0 +1,93 @@
//! Optional Host metadata observations for original material, independent of saved Journal reading.
//!
//! Presence checks must not fetch original history or restore a Worker. A failed/unauthorized
//! metadata lookup is unknown, not missing, and cannot prevent a saved Journal body from being read.
use super::memory_links::{
MemoryJournalRequest, MemoryJournalResponse, OriginalSourceAvailability,
};
use super::{JournalHostContext, JournalResult, SubjektivStore};
use rusqlite::{OptionalExtension, params};
impl SubjektivStore {
/// Executes the saved-reference operation first, then annotates original-source absence using
/// Host-owned metadata only. `Some(false)` is confirmed absence; `None` is an unavailable lookup.
/// Presence alone cannot attest that a particular historical entry is still readable.
pub fn execute_memory_journal_with_source_presence(
&self,
host: &JournalHostContext,
request: MemoryJournalRequest,
mut presence: impl FnMut(&str) -> Option<bool>,
) -> JournalResult<MemoryJournalResponse> {
let selector = match &request {
MemoryJournalRequest::Reconcile { selector }
| MemoryJournalRequest::Links { selector }
| MemoryJournalRequest::ReconcilePage { selector, .. }
| MemoryJournalRequest::LinksPage { selector, .. }
| MemoryJournalRequest::Read { selector, .. } => Some(selector.clone()),
MemoryJournalRequest::Related { .. } => None,
};
let mut response = self.execute_memory_journal(host, request)?;
let Some(selector) = selector else {
return Ok(response);
};
// Membership and scoped reader authority were checked by the operation. Candidate JSON is
// immutable; never infer source ownership from its presence without checking attribution.
let candidate = self.staging_candidate(&host.subject_id, &selector.candidate_id)?;
let mut observations = std::collections::HashMap::new();
let mut annotate =
|link: &mut super::memory_links::MemoryJournalSourceLink| -> JournalResult<()> {
let source = candidate
.as_ref()
.and_then(|c| c.source_refs.get(link.source_index));
// Legacy anchors may lack a Session ID. A connected edge already resolved it
// from immutable attribution and stable Segment/entry identity; reuse that saved
// identity rather than fetching history or modifying the original candidate.
let saved_session: Option<String> = self.database.try_with_connection(|tx| {
Ok::<_, super::JournalError>(tx.query_row(
"SELECT source_session_id FROM memory_journal_links
WHERE subject_id=?1 AND candidate_id=?2 AND source_index=?3 AND reason='linked'",
params![host.subject_id, selector.candidate_id, link.source_index],
|row| row.get(0),
).optional()?)
})?;
let session = saved_session
.as_deref()
.or_else(|| source.and_then(|s| s.session_id.as_deref()));
let observed = if let Some(session) = session {
if let Some(value) = observations.get(session) {
*value
} else {
let owned = self
.session_attribution(session)?
.is_some_and(|a| a.subject_id == host.subject_id);
let value = if owned { presence(session) } else { None };
observations.insert(session.to_owned(), value);
value
}
} else {
None
};
link.original_source = match observed {
Some(false) => OriginalSourceAvailability::Missing,
// Preserve explicit saved missing-byte attestations. Directory/registry presence
// does not prove the referenced entry survived rewind or storage corruption.
_ if link.original_source == OriginalSourceAvailability::Missing => {
OriginalSourceAvailability::Missing
}
_ => OriginalSourceAvailability::Unknown,
};
Ok(())
};
match &mut response {
MemoryJournalResponse::Links(page) => {
for source in &mut page.sources {
annotate(source)?;
}
}
MemoryJournalResponse::Read(read) => annotate(&mut read.source)?,
MemoryJournalResponse::Related(_) => {}
}
Ok(response)
}
}
+126
View File
@@ -0,0 +1,126 @@
//! Read-only, bounded Journal context for a new body Session. Selection and text projection
//! share one storage transaction; no source history, Job, admission or checkpoint is consulted.
//! This is derived reference material, never a new observation or an execution capability.
use super::*;
use rusqlite::{Connection, OptionalExtension, params};
pub const STARTUP_MAX_BYTES: usize = 32 * 1024;
const JOURNALS: usize = 3;
const ENTRIES: usize = 3;
const TEXT_BYTES: usize = 2048;
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalStartupSnapshot {
pub subject_id: String,
pub journals: Vec<StartupJournal>,
pub omitted: bool,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct StartupJournal {
pub journal_id: String,
pub source_session_id: String,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
pub current_position: String,
pub entries: Vec<StartupEntry>,
pub omitted: bool,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct StartupEntry {
pub entry_id: String,
pub position: u64,
pub state: JournalEntryState,
pub title: String,
pub body: String,
pub ongoing: String,
pub source_end: Option<JournalSourcePointer>,
pub correction_refs: Vec<String>,
pub omitted: bool,
}
// Read a bounded UTF-8 prefix at SQLite, not a full megabyte field followed by truncation.
fn text(tx: &Connection, table: &str, id: &str, field: &str) -> JournalResult<(String, bool)> {
let (mut bytes, length): (Vec<u8>, usize) = tx.query_row(
&format!("SELECT COALESCE(substr(CAST({field} AS BLOB),1,?2),X''),length(CAST({field} AS BLOB)) FROM {table} WHERE id=?1"),
params![id, TEXT_BYTES],
|r| Ok((r.get(0)?, r.get(1)?)),
)?;
let length_read = bytes.len();
match std::str::from_utf8(&bytes) {
Ok(_) => {}
Err(error) if error.error_len().is_none() => bytes.truncate(error.valid_up_to()),
Err(_) => {
return Err(JournalError::Conflict(
"stored Journal text is not UTF-8".into(),
));
}
}
let omitted = bytes.len() < length || length_read < length;
Ok((String::from_utf8(bytes).expect("validated UTF-8"), omitted))
}
impl SubjektivStore {
/// `previous_session` comes from the Host's retained predecessor binding, not model input.
/// Recent Subject-local Journals supplement that predecessor under independent fixed budgets.
pub fn journal_startup_snapshot(
&self,
host: &JournalHostContext,
previous_session: Option<&str>,
) -> JournalResult<JournalStartupSnapshot> {
self.database.try_transaction(|tx| {
super::store::check_host(self, tx, host, false)?;
let mut ids = Vec::new();
if let Some(session) = previous_session {
let previous: Option<(String, String)> = tx.query_row(
"SELECT id,subject_id FROM journals WHERE source_session_id=?1",
[session], |r| Ok((r.get(0)?, r.get(1)?)),
).optional()?;
if let Some((id, subject)) = previous {
if subject != host.subject_id { return Err(JournalError::PermissionDenied("predecessor Journal belongs to another Subject".into())); }
ids.push(id);
}
}
let mut query = tx.prepare("SELECT id FROM journals WHERE subject_id=?1 ORDER BY updated_at DESC,id DESC LIMIT ?2")?;
let recent = query.query_map(params![host.subject_id, JOURNALS + 1], |r| r.get::<_, String>(0))?;
for id in recent { let id = id?; if !ids.contains(&id) { ids.push(id); } }
let omitted = ids.len() > JOURNALS;
ids.truncate(JOURNALS);
let mut snapshot = JournalStartupSnapshot { subject_id: host.subject_id.clone(), journals: Vec::new(), omitted };
for id in ids {
let (session, token, order, byte) = tx.query_row(
"SELECT source_session_id,state_token,process_order,process_byte FROM journals WHERE id=?1 AND subject_id=?2",
params![id, host.subject_id], |r| Ok((r.get(0)?,r.get(1)?,r.get(2)?,r.get(3)?)),
)?;
let (current_position, mut omitted) = text(tx, "journals", &id, "current_position")?;
let mut stmt = tx.prepare("SELECT id,position,state,source_end_json FROM journal_entries WHERE journal_id=?1 ORDER BY position DESC LIMIT ?2")?;
let rows = stmt.query_map(params![id, ENTRIES + 1], |r| Ok((r.get::<_,String>(0)?,r.get::<_,u64>(1)?,r.get::<_,String>(2)?,r.get::<_,Option<String>>(3)?)))?.collect::<Result<Vec<_>,_>>()?;
omitted |= rows.len() > ENTRIES;
let mut entries = Vec::new();
for (entry_id, position, state, source_end) in rows.into_iter().take(ENTRIES) {
let (title, title_cut) = text(tx, "journal_entries", &entry_id, "title")?;
let (body, body_cut) = text(tx, "journal_entries", &entry_id, "body")?;
let (ongoing, ongoing_cut) = text(tx, "journal_entries", &entry_id, "ongoing")?;
let mut refs = tx.prepare("SELECT target_id FROM journal_corrections WHERE entry_id=?1 ORDER BY target_id LIMIT 17")?;
let mut correction_refs = refs.query_map([&entry_id], |r| r.get::<_,String>(0))?.collect::<Result<Vec<_>,_>>()?;
let omitted = title_cut || body_cut || ongoing_cut || correction_refs.len() > 16;
correction_refs.truncate(16);
entries.push(StartupEntry { entry_id, position, state: match state.as_str() { "open" => JournalEntryState::Open, "fixed" => JournalEntryState::Fixed, _ => return Err(JournalError::Conflict("invalid stored Journal entry state".into())) }, title, body, ongoing, source_end: source_end.map(|s| serde_json::from_str(&s)).transpose()?, correction_refs, omitted });
}
snapshot.journals.push(StartupJournal { journal_id: id, source_session_id: session, state_token: token, process_order: order, process_byte: byte, current_position, entries, omitted });
}
// JSON escaping also consumes the independent output budget. Drop least-priority
// entries/Journals rather than returning an over-budget or silently partial document.
while serde_json::to_vec(&snapshot)?.len() > STARTUP_MAX_BYTES {
snapshot.omitted = true;
let journal = snapshot.journals.last_mut().ok_or_else(|| JournalError::InvalidInput("startup metadata exceeds budget".into()))?;
if journal.entries.pop().is_some() { journal.omitted = true; } else { snapshot.journals.pop(); }
}
Ok(snapshot)
})
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+129
View File
@@ -0,0 +1,129 @@
//! Bounded, Host-bound Journal recording and sequential Job transport.
//!
//! Subject, Journal and generation authority never come from these model inputs.
//! Hosts validate live execution and public source bytes before selection; reading a
//! preview must not select the enclosing entry. Operation retries retain the entire
//! JournalOperation, including its original expected_state and covered_ranges.
use serde::{Deserialize, Serialize};
use super::{
JournalEntrySummary, JournalMetadata, JournalOperation, JournalOperationStatus, JournalPage,
JournalReceipt, JournalSourcePointer, JournalSourceRange, JournalTextPage,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalRecordingReason {
RunCommitted,
TaskCompleted,
BeforeCompact,
WorkerRestored,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalRecordingRequest {
pub reason: JournalRecordingReason,
}
/// Admission is durable Host evidence, not proof that generation finished.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "status", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalRecordingResponse {
NoNewSources,
Accepted,
}
/// No unbounded initial/current body or position is included in model state.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalJobState {
pub generation_id: String,
pub initial_journal: JournalMetadata,
pub initial_open: Option<JournalEntrySummary>,
pub current_journal: JournalMetadata,
pub current_open: Option<JournalEntrySummary>,
pub previous_fixed_end: Option<JournalSourcePointer>,
/// Frozen permission to fetch this prefix, not evidence of delivery or reflection.
/// The sequential service gates mutations on its durable Source-read delivery ledger.
pub selected_ranges: Vec<JournalSourceRange>,
}
/// Host-verified saved progress only; not an aggregate record submission.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct JournalJobCompletion {
pub generation_id: String,
pub state_token: String,
pub process_order: u64,
pub process_byte: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum JournalSnapshot {
Initial,
Current,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "target", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalTextTarget {
Position {
snapshot: JournalSnapshot,
},
OpenBody {
snapshot: JournalSnapshot,
},
OpenOngoing {
snapshot: JournalSnapshot,
},
/// The Host restricts this pointer to this Job's frozen public source input.
Source {
pointer: JournalSourcePointer,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct JournalTextRead {
pub target: JournalTextTarget,
pub offset: usize,
pub limit: usize,
/// Required continuation condition; None only for the first chunk.
pub content_token: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum JournalJobRequest {
Execute {
operation: JournalOperation,
},
ReadState,
QueryOperation {
request_id: String,
},
ReadOperationCoverage {
request_id: String,
offset: usize,
limit: usize,
},
ListOperations {
after: Option<String>,
limit: usize,
},
ReadText {
read: JournalTextRead,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "result", content = "value", rename_all = "snake_case")]
pub enum JournalJobResponse {
Saved(JournalReceipt),
State(JournalJobState),
Operation(Option<JournalOperationStatus>),
Coverage(JournalPage<JournalSourceRange>),
Operations(JournalPage<JournalOperationStatus>),
Text(JournalTextPage),
}
+26
View File
@@ -953,6 +953,32 @@ mod tests {
store.session_attribution("session-1").unwrap().unwrap(),
stored_attribution
);
let binding = store
.claim_journal_writer(&subject_id, "runtime-1", "worker-1")
.unwrap();
let host = store
.bind_journal_session(&binding, stored_attribution.clone())
.unwrap();
let range = crate::journal::JournalSourceRange {
source: crate::journal::JournalSource {
session_id: stored_attribution.session_id.clone(),
pointer: crate::journal::JournalSourcePointer {
segment_id: Uuid::now_v7(),
entry_ref: "Eafter-v1-migration".into(),
},
order: 0,
content_sha256: "ab".repeat(32),
total_bytes: 8,
},
bytes: crate::journal::JournalByteRange { start: 0, end: 8 },
};
let journal_id = store
.accept_journal_sources(&host, "After oldest supported migration", &[range])
.unwrap();
assert_eq!(
store.read_journal(&host, &journal_id).unwrap().subject_id,
subject_id
);
}
#[test]
+21
View File
@@ -2,6 +2,7 @@
pub mod api;
pub mod job;
pub mod journal;
mod legacy_migrations;
mod schema;
use legacy_migrations::{
@@ -602,6 +603,26 @@ static MIGRATIONS: &[FeatureMigration] = &[
"replace counters with immutable changes and content conditions",
schema::migrate,
),
FeatureMigration::new(
8,
"add durable journal operations and processing ranges",
journal::migrate,
),
FeatureMigration::new(
9,
"fence erased journal Sessions and permit explicit content erasure",
journal::migrate_lifecycle,
),
FeatureMigration::new(
10,
"fence journal writers by explicit body binding lifetime",
journal::migrate_binding,
),
FeatureMigration::new(
11,
"add immutable candidate Journal context links",
journal::memory_links::migrate,
),
];
pub const REGISTRATION: FeatureRegistration =
+44 -2
View File
@@ -360,10 +360,52 @@ mod tests {
}
#[tokio::test]
async fn bounded_binary_read_accepts_arbitrary_bytes_at_caller_limit_and_rejects_larger_files()
{
async fn bounded_binary_read_accepts_caller_limits_above_the_image_tool_limit() {
// The caller's limit is a reader policy, not a LocalWorkdir storage contract.
// Supply immutable chunks here; the separate component case exercises the
// real provider across a chunk boundary without repeatedly hashing 16 MiB.
let dir = TempDir::new().unwrap();
let bytes = vec![0xff; 16 * 1024 * 1024];
let hash = Sha256::digest(&bytes).into();
let responses = bytes
.chunks(IMAGE_READ_CHUNK_BYTES)
.enumerate()
.map(|(index, bytes)| {
chunk(
bytes,
(index * IMAGE_READ_CHUNK_BYTES) as u64,
16 * 1024 * 1024,
hash,
)
})
.collect();
let session = binary_session(&dir, responses);
let path = WorkdirPath::new("artifact.bin").unwrap();
assert_eq!(
read_workdir_bytes(&session, path.clone(), bytes.len())
.await
.unwrap(),
bytes
);
let session = binary_session(
&dir,
vec![chunk(
&bytes[..IMAGE_READ_CHUNK_BYTES],
0,
bytes.len() as u64,
hash,
)],
);
assert!(matches!(
read_workdir_bytes(&session, path, bytes.len() - 1).await,
Err(WorkdirError::InvalidArgument(_))
));
}
#[tokio::test]
async fn bounded_binary_read_reassembles_provider_chunks_and_enforces_the_caller_limit() {
let dir = TempDir::new().unwrap();
let bytes = vec![0xff; IMAGE_READ_CHUNK_BYTES + 1];
std::fs::write(dir.path().join("artifact.bin"), &bytes).unwrap();
let session =
LocalWorkdirSession::new(Scope::writable(dir.path()).unwrap(), dir.path().into());
+4
View File
@@ -3034,6 +3034,7 @@ impl App {
session_store::SystemItem::FeatureInvocationResult { body, .. }
| session_store::SystemItem::FileAttachment { body, .. }
| session_store::SystemItem::SkillActivation { body, .. }
| session_store::SystemItem::JournalStartup { body, .. }
| session_store::SystemItem::ResidentSummaryRefresh { body, .. }
| session_store::SystemItem::SubjectBehaviorRefresh { body, .. }
| session_store::SystemItem::Interrupt { body, .. } => {
@@ -4319,6 +4320,8 @@ mod completion_flow_tests {
let mut app = App::new("test".into());
let message = "mid-run compaction failed: summary unavailable";
app.handle_worker_event(Event::Alert(protocol::Alert {
id: None,
position: None,
level: AlertLevel::Error,
source: AlertSource::Compactor,
message: message.into(),
@@ -4402,6 +4405,7 @@ mod completion_flow_tests {
},
state: test_worker_state(WorkerStatus::Running),
in_flight: InFlightSnapshot {
block_ids: Vec::new(),
blocks: vec![
InFlightBlock::Thinking {
text: "why".into(),
+2 -1
View File
@@ -23,6 +23,7 @@ http-server = ["dep:axum", "dep:tower", "dep:reqwest"]
ws-server = ["http-server", "axum/ws", "dep:futures", "tokio/sync"]
[dependencies]
agen.workspace = true
async-trait.workspace = true
base64.workspace = true
axum = { workspace = true, optional = true }
@@ -59,5 +60,5 @@ futures.workspace = true
agen.workspace = true
serial_test = "3.4.0"
tempfile.workspace = true
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "test-util"] }
tokio-tungstenite.workspace = true
+15
View File
@@ -567,6 +567,14 @@ impl fmt::Debug for WorkerProtocolTransport {
pub trait WorkerExecutionBackend: Send + Sync + 'static {
fn backend_id(&self) -> &str;
/// Must reserve the actual Controller, not infer Idle from a catalog snapshot.
fn subject_release(
&self,
_request: &runtime_api::SubjectReleaseRequest,
) -> Result<runtime_api::SubjectReleaseResponse, String> {
Err("execution backend cannot reserve the actual Subject Controller".into())
}
fn fetch_workspace_config(
&self,
_request: WorkspaceConfigFetchRequest,
@@ -831,6 +839,13 @@ impl WorkerExecutionBackendRef {
self.backend.fetch_workspace_config(request)
}
pub(crate) fn subject_release(
&self,
request: &runtime_api::SubjectReleaseRequest,
) -> Result<runtime_api::SubjectReleaseResponse, String> {
self.backend.subject_release(request)
}
pub(crate) fn worker_session(
&self,
request: WorkerSessionObservationRequest,
+23 -5
View File
@@ -256,15 +256,22 @@ impl FsRuntimeStore {
}
pub(crate) fn delete_worker_record(&self, worker_id: &WorkerId) -> Result<(), RuntimeError> {
self.delete_worker_record_with_intent(
worker_id,
session_store::retention::SessionDeletionIntent::Ordinary,
)
}
pub(crate) fn delete_worker_record_with_intent(
&self,
worker_id: &WorkerId,
intent: session_store::retention::SessionDeletionIntent,
) -> Result<(), RuntimeError> {
let worker_dir = self.worker_dir(worker_id);
if !worker_dir.exists() {
return Ok(());
}
fs::remove_dir_all(&worker_dir).map_err(|source| RuntimeError::StoreIo {
operation: "delete worker store",
path: worker_dir,
source,
})
crate::retention::delete_worker_aggregate(&self.root, &worker_dir, intent)
}
pub(crate) fn load_runtime_state(&self) -> Result<PersistedRuntimeState, RuntimeError> {
@@ -472,6 +479,10 @@ pub(crate) struct OpenedFsRuntimeStore {
#[derive(Clone, Debug)]
pub(crate) struct PersistedRuntimeState {
pub(crate) subject_releases: Vec<(
runtime_api::SubjectReleaseRequest,
runtime_api::SubjectReleaseState,
)>,
pub(crate) display_name: Option<String>,
pub(crate) status: RuntimeStatus,
pub(crate) next_diagnostic_id: u64,
@@ -1908,6 +1919,11 @@ fn migrate_runtime_store_in_place(
#[derive(Clone, Debug, Serialize, Deserialize)]
struct RuntimeSnapshot {
#[serde(default)]
subject_releases: Vec<(
runtime_api::SubjectReleaseRequest,
runtime_api::SubjectReleaseState,
)>,
schema_version: u32,
display_name: Option<String>,
backend: RuntimeBackendKind,
@@ -1939,6 +1955,7 @@ fn record_worker_load_diagnostic(
impl RuntimeSnapshot {
fn from_persisted(state: &PersistedRuntimeState) -> Self {
Self {
subject_releases: state.subject_releases.clone(),
schema_version: SCHEMA_VERSION,
display_name: state.display_name.clone(),
backend: RuntimeBackendKind::FsStore,
@@ -1976,6 +1993,7 @@ impl RuntimeSnapshot {
workers: BTreeMap<WorkerId, PersistedWorkerRecord>,
) -> PersistedRuntimeState {
PersistedRuntimeState {
subject_releases: self.subject_releases,
display_name: self.display_name,
status: self.status,
next_diagnostic_id: self.next_diagnostic_id,
+8 -1
View File
@@ -2102,7 +2102,10 @@ fn required_runtime_permission(method: &Method, path: &str) -> Option<&'static s
{
return Some("workers:input");
}
if path.ends_with("/stop") || path.ends_with("/cancel") {
if path.ends_with("/stop")
|| path.ends_with("/cancel")
|| (path == "/v1/subject-release" && *method == Method::POST)
{
return Some("workers:stop");
}
if path == runtime_api::RUNTIME_ROUTE_PROTOCOL_WS {
@@ -2420,6 +2423,9 @@ pub enum RuntimeHttpServerError {
#[cfg(test)]
mod tests {
#[cfg(feature = "fs-store")]
mod session_material_tests;
use super::*;
use crate::auth::RuntimeIdentityMaterial;
use crate::catalog::{ConfigBundleRef, ProfileSelector, WorkspaceApiRef};
@@ -4821,6 +4827,7 @@ mod ws_tests {
},
state: protocol::WorkerStateSnapshot::initial(),
in_flight: protocol::InFlightSnapshot {
block_ids: Vec::new(),
blocks: Vec::new(),
commands: Vec::new(),
compaction: None,
@@ -35,6 +35,86 @@ fn required_auth() -> Result<Extension<RuntimeAuthContext>, runtime_api::Runtime
})
}
fn material_scope(
state: &RuntimeHttpState,
workspace_id: &str,
) -> Result<RuntimeWorkspaceScope, runtime_api::RuntimeApiError> {
let scope = auth_workspace_scope(state, auth_extension().as_ref())
.map_err(api_error)?
.ok_or_else(|| {
runtime_api::RuntimeApiError::new(
403,
"runtime_session_material_scope_required",
"Canonical Session material requires a Workspace-scoped capability",
)
})?;
if scope.workspace_id != workspace_id {
return Err(runtime_api::RuntimeApiError::new(
403,
"runtime_session_material_scope_mismatch",
"Canonical Session Workspace does not match the authenticated capability",
));
}
Ok(scope)
}
fn material_pin(
state: &RuntimeHttpState,
request: runtime_api::SessionMaterialPinRequest,
release: bool,
) -> Result<runtime_api::SessionMaterialPinResponse, runtime_api::RuntimeApiError> {
let scope = material_scope(state, &request.workspace_id)?;
#[cfg(feature = "fs-store")]
{
let pin = request
.pin_id
.parse::<uuid::Uuid>()
.map(session_store::retention::SessionRetentionPin)
.map_err(|_| {
runtime_api::RuntimeApiError::new(
400,
"invalid_session_retention_pin",
"Invalid Session retention pin identity",
)
})?;
if release {
state.runtime.release_session_public_source_pin_scoped(
&scope,
&request.source,
&request.expected_session_id,
pin,
)
} else {
state.runtime.pin_session_public_source_scoped(
&scope,
&request.source,
&request.expected_session_id,
pin,
)
}
.map_err(|_| {
runtime_api::RuntimeApiError::new(
409,
"session_retention_rejected",
"Session retention operation could not be verified",
)
})?;
Ok(runtime_api::SessionMaterialPinResponse {
session_id: request.expected_session_id,
pin_id: request.pin_id,
})
}
#[cfg(not(feature = "fs-store"))]
{
let _ = (scope, release);
Err(runtime_api::RuntimeApiError::new(
503,
"session_retention_unavailable",
"Session retention requires filesystem storage",
))
}
}
#[derive(Clone)]
struct RuntimeManagementApi {
state: RuntimeHttpState,
@@ -289,6 +369,115 @@ impl runtime_api::RuntimeApi for RuntimeManagementApi {
.map_err(api_error)
}
async fn session_material_erase(
&self,
request: runtime_api::SessionMaterialEraseRequest,
) -> Result<runtime_api::SessionMaterialEraseResponse, runtime_api::RuntimeApiError> {
let scope = material_scope(&self.state, &request.workspace_id)?;
#[cfg(feature = "fs-store")]
{
self.state
.runtime
.delete_session_public_source_scoped(
&scope,
&request.source,
&request.expected_session_id,
session_store::retention::SessionDeletionIntent::ErasePermanently,
)
.map_err(|_| {
runtime_api::RuntimeApiError::new(
409,
"session_erasure_incomplete",
"Session erasure could not complete; retain erasure intent and retry",
)
})?;
Ok(runtime_api::SessionMaterialEraseResponse {
session_id: request.expected_session_id,
})
}
#[cfg(not(feature = "fs-store"))]
{
let _ = scope;
Err(runtime_api::RuntimeApiError::new(
503,
"session_retention_unavailable",
"Session erasure requires filesystem storage",
))
}
}
async fn subject_release(
&self,
request: runtime_api::SubjectReleaseRequest,
) -> Result<runtime_api::SubjectReleaseResponse, runtime_api::RuntimeApiError> {
let scope = material_scope(&self.state, &request.workspace_id)?;
self.state
.runtime
.subject_release_scoped(&scope, request)
.map_err(RuntimeHttpRestError::runtime)
.map_err(api_error)
}
async fn session_material_pin(
&self,
request: runtime_api::SessionMaterialPinRequest,
) -> Result<runtime_api::SessionMaterialPinResponse, runtime_api::RuntimeApiError> {
material_pin(&self.state, request, false)
}
async fn session_material_release(
&self,
request: runtime_api::SessionMaterialPinRequest,
) -> Result<runtime_api::SessionMaterialPinResponse, runtime_api::RuntimeApiError> {
material_pin(&self.state, request, true)
}
async fn session_material_catalog(
&self,
request: runtime_api::SessionMaterialCatalogRequest,
) -> Result<runtime_api::SessionMaterialCatalogAvailability, runtime_api::RuntimeApiError> {
let scope = material_scope(&self.state, &request.workspace_id)?;
#[cfg(feature = "fs-store")]
{
Ok(self
.state
.runtime
.session_material_catalog_scoped(&scope, &request))
}
#[cfg(not(feature = "fs-store"))]
{
let _ = scope;
Ok(
runtime_api::SessionMaterialCatalogAvailability::Unavailable {
reason: runtime_api::SessionPublicUnavailableReason::StorageUnavailable,
message: "Canonical Session storage unavailable".into(),
},
)
}
}
async fn session_material_read(
&self,
request: runtime_api::SessionMaterialReadRequest,
) -> Result<runtime_api::SessionMaterialReadAvailability, runtime_api::RuntimeApiError> {
let scope = material_scope(&self.state, &request.workspace_id)?;
#[cfg(feature = "fs-store")]
{
Ok(self
.state
.runtime
.session_material_read_scoped(&scope, &request))
}
#[cfg(not(feature = "fs-store"))]
{
let _ = scope;
Ok(runtime_api::SessionMaterialReadAvailability::Unavailable {
reason: runtime_api::SessionPublicUnavailableReason::StorageUnavailable,
message: "Canonical Session storage unavailable".into(),
})
}
}
async fn session_public_search(
&self,
request: runtime_api::SessionPublicSearchRequest,
File diff suppressed because it is too large Load Diff
+3
View File
@@ -20,7 +20,10 @@ pub mod resource;
pub mod retention;
mod runtime;
#[cfg(feature = "fs-store")]
mod session_material;
#[cfg(feature = "fs-store")]
mod session_public;
mod session_retention;
pub mod ssh_host_key_probe;
pub mod worker_backend;
pub mod worker_source;
+410 -46
View File
@@ -133,6 +133,9 @@ pub struct WorkerSessionArchiveManifest {
pub source_runtime_id: String,
pub source_worker_id: WorkerId,
pub source_session_id: String,
/// Older archives have no attested active lineage and cannot serve canonical scans.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source_active_segment_id: Option<session_store::SegmentId>,
pub segment_ids: Vec<String>,
pub source_created_at: String,
pub removed_at: String,
@@ -173,6 +176,285 @@ pub(crate) trait WorkerRetentionProvider: Send + Sync {
) -> Result<WorkerRetentionExecutionResult, RuntimeError>;
}
pub use crate::session_retention::runtime_session_retention;
/// Prove the legitimate pre-publication state, rather than interpreting a missing
/// source as empty. A committed active pointer or any source file defeats this proof.
pub(crate) fn unmaterialized_session(
worker_dir: &Path,
workspace_id: &str,
) -> Result<Option<session_store::SessionId>, RuntimeError> {
let metadata_path = worker_dir.join("metadata.json");
let bytes = match fs::read(&metadata_path) {
Ok(bytes) => bytes,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
Err(source) => {
return Err(RuntimeError::StoreIo {
operation: "inspect pending Session",
path: metadata_path,
source,
});
}
};
let metadata: session_store::WorkerMetadata = serde_json::from_slice(&bytes)
.map_err(|_| RuntimeError::InvalidRequest("Pending Session metadata is invalid".into()))?;
if metadata.workspace_id.as_deref() != Some(workspace_id) {
return Ok(None);
}
let Some(active) = metadata.active.filter(|active| active.segment_id.is_none()) else {
return Ok(None);
};
let root = worker_dir.join("session");
if collect_files(&root, "inspect pending Session")?.is_empty() {
Ok(Some(active.session_id))
} else {
Ok(None)
}
}
pub(crate) fn aggregate_session_id(
worker_dir: &Path,
) -> Result<Option<session_store::SessionId>, RuntimeError> {
let session_path = worker_dir.join("session/session.json");
let saved = match fs::read(&session_path) {
Ok(bytes) => {
let manifest: CanonicalSessionManifest =
serde_json::from_slice(&bytes).map_err(|_| {
RuntimeError::InvalidRequest("Retained Session manifest is invalid".into())
})?;
Some(
manifest
.session_id
.parse::<session_store::SessionId>()
.map_err(|_| {
RuntimeError::InvalidRequest("Retained Session identity is invalid".into())
})?,
)
}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(source) => {
return Err(RuntimeError::StoreIo {
operation: "read retained Session identity",
path: session_path,
source,
});
}
};
let metadata_path = worker_dir.join("metadata.json");
let active = match fs::read(&metadata_path) {
Ok(bytes) => {
let metadata: session_store::WorkerMetadata =
serde_json::from_slice(&bytes).map_err(|_| {
RuntimeError::InvalidRequest("Retained Worker metadata is invalid".into())
})?;
metadata.active.map(|active| active.session_id)
}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(source) => {
return Err(RuntimeError::StoreIo {
operation: "read retained Session pointer",
path: metadata_path,
source,
});
}
};
if let (Some(saved), Some(active)) = (saved, active) {
if saved != active {
return Err(RuntimeError::InvalidRequest(
"Retained Session manifest and active pointer disagree".into(),
));
}
}
if saved.is_none() && active.is_none() {
let session_dir = worker_dir.join("session");
if session_dir.is_dir()
&& !collect_files(&session_dir, "inspect uninitialized Session")?.is_empty()
{
return Err(RuntimeError::InvalidRequest(
"Session material has no retained identity".into(),
));
}
}
Ok(saved.or(active))
}
/// Called only while the common Session deletion lock is held. Include staged
/// archives from interrupted transfers: erasure must not leave a second original.
pub(crate) fn remove_archived_session_copies(
runtime_root: &Path,
session: session_store::SessionId,
) -> Result<(), session_store::StoreError> {
let archives = runtime_root.join("archives/workers");
let entries = match fs::read_dir(archives) {
Ok(entries) => entries,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()),
Err(error) => return Err(error.into()),
};
for entry in entries {
let entry = entry?;
if !entry.file_type()?.is_dir() {
continue;
}
let session_root = entry.path().join("session");
let owner = match fs::read(entry.path().join("manifest.json")) {
Ok(bytes) => Some(serde_json::from_slice::<WorkerSessionArchiveManifest>(
&bytes,
)?),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => return Err(error.into()),
};
if let Some(owner) = owner {
if owner.source_session_id == session.to_string() {
erase_archived_diagnostics(runtime_root, &owner)?;
match fs::remove_dir_all(&session_root) {
Ok(()) => (),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => (),
Err(error) => return Err(error.into()),
}
}
continue;
}
let bytes = match fs::read(session_root.join("session.json")) {
Ok(bytes) => bytes,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue,
Err(error) => return Err(error.into()),
};
let manifest: CanonicalSessionManifest = serde_json::from_slice(&bytes)?;
if manifest.session_id == session.to_string() {
fs::remove_dir_all(&session_root)?;
}
}
Ok(())
}
/// Erasure includes diagnostic originals of the same archived removal operation.
/// The immutable removal receipt is historical evidence and is never rewritten.
/// Staging names are bound to that exact operation; unrelated archives are untouched.
fn erase_archived_diagnostics(
runtime_root: &Path,
archive: &WorkerSessionArchiveManifest,
) -> Result<(), session_store::StoreError> {
let invalid = || {
session_store::StoreError::Io(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"Archived diagnostics erasure identity mismatch",
))
};
if archive.schema_version != ARCHIVE_SCHEMA_VERSION {
return Err(invalid());
}
validate_id("operation_id", &archive.operation_id).map_err(|_| invalid())?;
let receipt_path = runtime_root
.join("retention/operations")
.join(format!("{}.json", archive.operation_id));
if let Some(receipt) = read_operation_receipt(&receipt_path).map_err(|_| invalid())? {
if receipt.request.workspace_id != archive.workspace_id
|| receipt.request.source_runtime_id != archive.source_runtime_id
|| receipt.request.worker_id != archive.source_worker_id
|| receipt.request.operation_id != archive.operation_id
|| receipt.request.input_fingerprint != archive.input_fingerprint
|| receipt.request.archive_id.as_ref() != Some(&archive.archive_id)
|| receipt.result.archive.as_ref() != Some(archive)
{
return Err(invalid());
}
}
let parent = runtime_root.join("archives/diagnostics");
let entries = match fs::read_dir(&parent) {
Ok(entries) => entries,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()),
Err(error) => return Err(error.into()),
};
let staging_prefix = format!(".{}.tmp-", archive.operation_id);
for entry in entries {
let entry = entry?;
let name = entry.file_name();
let Some(name) = name.to_str() else {
continue;
};
let staged = name.strip_prefix(&staging_prefix).is_some_and(|suffix| {
suffix.split_once('-').is_some_and(|(pid, sequence)| {
!pid.is_empty()
&& !sequence.is_empty()
&& pid.bytes().all(|b| b.is_ascii_digit())
&& sequence.bytes().all(|b| b.is_ascii_digit())
})
});
if name != archive.operation_id && !staged {
continue;
}
if !entry.file_type()?.is_dir() {
return Err(invalid());
}
let manifest = match fs::read(entry.path().join("manifest.json")) {
Ok(bytes) => Some(serde_json::from_slice::<DiagnosticsArchiveManifest>(
&bytes,
)?),
Err(error) if error.kind() == std::io::ErrorKind::NotFound && staged => None,
Err(error) => return Err(error.into()),
};
if let Some(manifest) = manifest {
if manifest.schema_version != ARCHIVE_SCHEMA_VERSION
|| manifest.workspace_id != archive.workspace_id
|| manifest.source_runtime_id != archive.source_runtime_id
|| manifest.source_worker_id != archive.source_worker_id
|| manifest.operation_id != archive.operation_id
|| manifest.input_fingerprint != archive.input_fingerprint
{
return Err(invalid());
}
}
fs::remove_dir_all(entry.path())?;
}
sync_directory(&parent, "erase archived Worker diagnostics").map_err(|_| invalid())?;
Ok(())
}
pub(crate) fn retention_error(error: session_store::StoreError) -> RuntimeError {
RuntimeError::InvalidRequest(format!("Session retention rejected operation: {error}"))
}
/// Direct Worker deletion cannot bypass pins. Purge policy is not erase intent.
pub(crate) fn delete_worker_aggregate(
runtime_root: &Path,
worker_dir: &Path,
intent: session_store::retention::SessionDeletionIntent,
) -> Result<(), RuntimeError> {
let session = aggregate_session_id(worker_dir)?;
let retention = runtime_session_retention(runtime_root);
// Ordinary policy never creates erase authority. A pre-existing permanent
// tombstone is an irrevocable decision whose physical cleanup may be resumed.
let intent = match session
.map(|session| retention.ensure_live(session))
.transpose()
{
Err(session_store::StoreError::SessionErased(_)) => {
session_store::retention::SessionDeletionIntent::ErasePermanently
}
Err(error) => return Err(retention_error(error)),
Ok(_) => intent,
};
let remove = || -> Result<(), session_store::StoreError> {
if intent == session_store::retention::SessionDeletionIntent::ErasePermanently {
if let Some(session) = session {
remove_archived_session_copies(runtime_root, session)?;
}
}
match fs::remove_dir_all(worker_dir) {
Ok(()) => Ok(()),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(error) => Err(error.into()),
}
};
if let Some(session) = session {
runtime_session_retention(runtime_root)
.delete(session, intent, remove)
.map_err(retention_error)
} else {
remove().map_err(retention_error)
}
}
/// Filesystem provider for the canonical Runtime Worker aggregate.
#[derive(Clone, Debug)]
pub(crate) struct FsWorkerRetentionProvider {
@@ -355,6 +637,16 @@ impl FsWorkerRetentionProvider {
pub(crate) fn session_archive(
&self,
archive_id: &str,
) -> Result<(WorkerSessionArchiveManifest, PathBuf), RuntimeError> {
let (manifest, session_root) = self.session_archive_identity(archive_id)?;
let archive_dir = self.archive_dir(archive_id)?;
let manifest = validate_existing_archive(&archive_dir, &manifest)?;
Ok((manifest, session_root))
}
pub(crate) fn session_archive_identity(
&self,
archive_id: &str,
) -> Result<(WorkerSessionArchiveManifest, PathBuf), RuntimeError> {
let archive_dir = self.archive_dir(archive_id)?;
let manifest: WorkerSessionArchiveManifest = read_json(
@@ -368,7 +660,6 @@ impl FsWorkerRetentionProvider {
message: "archive manifest identity mismatch".to_string(),
});
}
let manifest = validate_existing_archive(&archive_dir, &manifest)?;
Ok((manifest, archive_dir.join("session")))
}
@@ -401,33 +692,36 @@ impl WorkerRetentionProvider for FsWorkerRetentionProvider {
return Err(RuntimeError::WorkerNotFound { worker_id });
}
let session_dir = worker_dir.join("session");
let (session_id, segment_ids, session_bytes) = if session_dir.is_dir() {
let manifest: CanonicalSessionManifest = read_json(
&session_dir.join("session.json"),
"inventory Worker retention",
)?;
let files = collect_files(&session_dir, "inventory Worker retention")?;
let mut segment_ids = BTreeSet::new();
let mut bytes = 0_u64;
for (relative, path) in files {
bytes = bytes.saturating_add(file_len(&path, "inventory Worker retention")?);
if let Some(name) = relative.file_name().and_then(|value| value.to_str()) {
let segment = name
.strip_suffix(".trace.jsonl")
.or_else(|| name.strip_suffix(".jsonl"));
if let Some(segment) = segment {
segment_ids.insert(segment.to_string());
let (session_id, segment_ids, session_bytes) =
if unmaterialized_session(&worker_dir, workspace_id)?.is_some() {
(None, Vec::new(), 0)
} else if session_dir.is_dir() {
let manifest: CanonicalSessionManifest = read_json(
&session_dir.join("session.json"),
"inventory Worker retention",
)?;
let files = collect_files(&session_dir, "inventory Worker retention")?;
let mut segment_ids = BTreeSet::new();
let mut bytes = 0_u64;
for (relative, path) in files {
bytes = bytes.saturating_add(file_len(&path, "inventory Worker retention")?);
if let Some(name) = relative.file_name().and_then(|value| value.to_str()) {
let segment = name
.strip_suffix(".trace.jsonl")
.or_else(|| name.strip_suffix(".jsonl"));
if let Some(segment) = segment {
segment_ids.insert(segment.to_string());
}
}
}
}
(
Some(manifest.session_id),
segment_ids.into_iter().collect(),
bytes,
)
} else {
(None, Vec::new(), 0)
};
(
Some(manifest.session_id),
segment_ids.into_iter().collect(),
bytes,
)
} else {
(None, Vec::new(), 0)
};
let diagnostics_bytes = diagnostics_files(&worker_dir, "inventory Worker retention")?
.into_iter()
.try_fold(0_u64, |total, path| {
@@ -503,12 +797,46 @@ impl WorkerRetentionProvider for FsWorkerRetentionProvider {
worker_id: request.worker_id,
});
}
// Transfer keeps the same Session pin/fence namespace. Hold its lock
// across verified archive publication and aggregate removal. A policy
// purge instead uses ordinary deletion and must reject outstanding pins.
let retention = runtime_session_retention(&self.runtime_root);
let transfer_guard = if request.session_disposition == SessionDisposition::Archive {
aggregate_session_id(&worker_dir)?
.map(|session| retention.lock_live(session))
.transpose()
.map_err(retention_error)?
} else {
None
};
let archive = match request.session_disposition {
SessionDisposition::Archive => {
Some(commit_session_archive(self, request, &worker_dir)?)
}
SessionDisposition::Purge => None,
};
if let Some(manifest) = &archive {
if let Some(active_segment) = manifest.source_active_segment_id {
// We already hold the Session transfer lock. Do not reacquire it
// through the external-retention constructor during verification.
let reader =
session_store::public_material::SessionPublicMaterialReader::open_worker(
&self.archive_dir(&manifest.archive_id)?.join("session"),
active_segment,
Default::default(),
)
.map_err(|error| {
RuntimeError::InvalidRequest(format!(
"Archived canonical source is unreadable: {error}"
))
})?;
if reader.session_id().to_string() != manifest.source_session_id {
return Err(RuntimeError::InvalidRequest(
"Archived canonical source identity mismatch".into(),
));
}
}
}
let diagnostics_retained = match request.diagnostics_disposition {
DiagnosticsDisposition::Purge => false,
DiagnosticsDisposition::Retain => {
@@ -540,11 +868,23 @@ impl WorkerRetentionProvider for FsWorkerRetentionProvider {
"commit pending Worker retention receipt",
)?;
fs::remove_dir_all(&worker_dir).map_err(|source| RuntimeError::StoreIo {
operation: "remove retained Worker aggregate",
path: worker_dir.clone(),
source,
})?;
match request.session_disposition {
SessionDisposition::Archive => {
// commit_session_archive already compared all original bytes.
// The transfer lock remains held until removal has completed.
fs::remove_dir_all(&worker_dir).map_err(|source| RuntimeError::StoreIo {
operation: "remove retained Worker aggregate",
path: worker_dir.clone(),
source,
})?;
}
SessionDisposition::Purge => delete_worker_aggregate(
&self.runtime_root,
&worker_dir,
session_store::retention::SessionDeletionIntent::Ordinary,
)?,
}
drop(transfer_guard);
sync_directory(
worker_dir.parent().unwrap_or(&self.runtime_root),
"remove retained Worker aggregate",
@@ -703,14 +1043,14 @@ fn commit_session_archive(
RuntimeError::InvalidRequest("archive disposition requires archive_id".to_string())
})?;
let session_dir = worker_dir.join("session");
if !session_dir.is_dir() {
return Err(RuntimeError::StoreMissing {
operation: "archive Worker Session",
path: session_dir,
});
}
let session: CanonicalSessionManifest =
read_json(&session_dir.join("session.json"), "archive Worker Session")?;
let pending = unmaterialized_session(worker_dir, &request.workspace_id)?;
let session: CanonicalSessionManifest = if let Some(session) = pending {
CanonicalSessionManifest {
session_id: session.to_string(),
}
} else {
read_json(&session_dir.join("session.json"), "archive Worker Session")?
};
let source_files = collect_files(&session_dir, "archive Worker Session")?;
let (checksum, bytes, count) = checksum_files(&source_files, "archive Worker Session")?;
let mut segment_ids = BTreeSet::new();
@@ -730,6 +1070,25 @@ fn commit_session_archive(
workspace_id: request.workspace_id.clone(),
source_runtime_id: request.source_runtime_id.clone(),
source_worker_id: request.worker_id,
source_active_segment_id: if worker_dir.join("metadata.json").is_file() {
let metadata: session_store::WorkerMetadata = read_json(
&worker_dir.join("metadata.json"),
"archive active Session pointer",
)?;
let active = metadata.active.ok_or_else(|| {
RuntimeError::InvalidRequest("Worker has no active Session pointer".into())
})?;
if active.session_id.to_string() != session.session_id
|| metadata.workspace_id.as_deref() != Some(&request.workspace_id)
{
return Err(RuntimeError::InvalidRequest(
"Archive Session pointer scope mismatch".into(),
));
}
active.segment_id
} else {
None
},
source_session_id: session.session_id,
segment_ids: segment_ids.into_iter().collect(),
source_created_at: request.source_created_at.clone(),
@@ -770,6 +1129,13 @@ fn commit_session_archive(
path: staging.clone(),
source,
})?;
// Persist source ownership before copying any original bytes. An
// interrupted staging tree remains attributable to explicit erasure.
atomic_write_json(
&staging.join("manifest.json"),
&manifest,
"archive Worker Session",
)?;
let target_session = staging.join("session");
fs::create_dir(&target_session).map_err(|source| RuntimeError::StoreIo {
operation: "archive Worker Session",
@@ -777,11 +1143,6 @@ fn commit_session_archive(
source,
})?;
copy_files(&source_files, &target_session, "archive Worker Session")?;
atomic_write_json(
&staging.join("manifest.json"),
&manifest,
"archive Worker Session",
)?;
sync_tree(&staging, "archive Worker Session")?;
fs::rename(&staging, &archive_dir).map_err(|source| RuntimeError::StoreIo {
operation: "archive Worker Session",
@@ -1264,7 +1625,7 @@ mod tests {
);
write_json(
&worker.join("session/session.json"),
&serde_json::json!({"schema_version": 1, "session_id": "session-a"}),
&serde_json::json!({"schema_version": 1, "session_id": "00000000-0000-4000-8000-000000000001"}),
);
fs::create_dir_all(worker.join("session/segments")).unwrap();
fs::write(worker.join("session/segments/segment-a.jsonl"), b"one\n").unwrap();
@@ -1310,7 +1671,10 @@ mod tests {
let first = provider.execute(&request).unwrap();
assert!(first.source_removed);
let archive = first.archive.as_ref().unwrap();
assert_eq!(archive.source_session_id, "session-a");
assert_eq!(
archive.source_session_id,
"00000000-0000-4000-8000-000000000001"
);
assert_eq!(archive.segment_ids, vec!["segment-a"]);
assert!(!temp.path().join(format!("workers/{worker_id}")).exists());
assert!(
+589 -6
View File
@@ -356,6 +356,109 @@ impl Drop for WorkerOperationLease {
}
impl Runtime {
/// Reserves the actual Controller while the Host reconciles final saved Journal material.
/// A Runtime restart loses proof of a reservation, never reconstructs it from Worker status.
pub fn subject_release_scoped(
&self,
scope: &RuntimeWorkspaceScope,
request: runtime_api::SubjectReleaseRequest,
) -> Result<runtime_api::SubjectReleaseResponse, RuntimeError> {
use runtime_api::SubjectReleaseAction as Action;
if request.workspace_id != scope.workspace_id
|| request.binding_id.trim().is_empty()
|| request.operation_id.trim().is_empty()
|| request.binding_id.len() > 512
|| request.operation_id.len() > 512
{
return Err(RuntimeError::InvalidRequest(
"invalid Subject release identity".into(),
));
}
let worker_ref = WorkerRef::new(request.worker_id);
let lease = self.worker_operation_lock(request.worker_id)?;
let _operation = lease.lock().map_err(|_| RuntimeError::StatePoisoned)?;
self.ensure_worker_in_workspace(scope, &worker_ref)?;
let backend = {
let state = self.lock()?;
state.ensure_running()?;
let worker = state.worker(&worker_ref)?;
if worker.has_pending_lifecycle_operation() {
return Err(RuntimeError::InvalidRequest(
"Subject lifecycle is unresolved".into(),
));
}
if let Some((saved, _)) = state
.subject_release_operations
.get(&(request.worker_id, request.operation_id.clone()))
{
if saved.operation_id != request.operation_id
|| saved.binding_id != request.binding_id
|| saved.workspace_id != request.workspace_id
{
return Err(RuntimeError::InvalidRequest(
"different Subject release is retained".into(),
));
}
} else if request.action != Action::Reserve {
return Err(RuntimeError::InvalidRequest(
"Subject reservation is absent or lost; retain occupancy".into(),
));
} else {
state.ensure_subject_not_reserved(&worker_ref, false)?;
}
state.execution_backend.clone().ok_or_else(|| {
RuntimeError::InvalidRequest("actual Controller reservation is unavailable".into())
})?
};
let key = (request.worker_id, request.operation_id.clone());
{
let mut state = self.lock()?;
if let Some((_, saved)) = state.subject_release_operations.get(&key) {
if *saved == runtime_api::SubjectReleaseState::Cancelled {
if request.action != Action::Cancel {
return Err(RuntimeError::InvalidRequest(
"cancelled release cannot be rearmed; use a new operation".into(),
));
}
return Ok(runtime_api::SubjectReleaseResponse {
operation_id: request.operation_id,
binding_id: request.binding_id,
state: *saved,
});
}
} else {
// Persist admission exclusion before any Controller ACK, under the Worker op
// lock. Restart retains this fence without manufacturing disconnect proof.
state.subject_release_operations.insert(
key.clone(),
(request.clone(), runtime_api::SubjectReleaseState::Reserved),
);
}
state.persist_runtime_snapshot()?;
}
let response = backend
.subject_release(&request)
.map_err(RuntimeError::InvalidRequest)?;
if response.binding_id != request.binding_id
|| response.operation_id != request.operation_id
{
return Err(RuntimeError::InvalidRequest(
"Subject release acknowledgment mismatch".into(),
));
}
let mut state = self.lock()?;
let before = state
.subject_release_operations
.insert(key.clone(), (request.clone(), response.state));
if let Err(error) = state.persist_runtime_snapshot() {
if let Some(before) = before {
state.subject_release_operations.insert(key, before);
}
return Err(error);
}
Ok(response)
}
fn reserve_workdir_use<'a>(
&self,
ids: impl IntoIterator<Item = &'a str>,
@@ -1961,6 +2064,8 @@ impl Runtime {
let _operation_guard = operation_lock
.lock()
.map_err(|_| RuntimeError::StatePoisoned)?;
self.lock()?
.ensure_subject_not_reserved(worker_ref, false)?;
if let Some(scope) = scope {
let mut state = self.lock()?;
if !state
@@ -2332,6 +2437,8 @@ impl Runtime {
mode: WorkerRestoreMode,
preparation_authorized: bool,
) -> Result<RuntimeWorkerRestoreResult, RuntimeError> {
self.lock()?
.ensure_subject_not_reserved(worker_ref, false)?;
let owner = {
let state = self.lock()?;
let worker = state.worker(worker_ref)?;
@@ -2586,6 +2693,8 @@ impl Runtime {
worker_ref: &WorkerRef,
mode: WorkerRestoreMode,
) -> Result<RuntimeWorkerRestoreResult, RuntimeError> {
self.lock()?
.ensure_subject_not_reserved(worker_ref, false)?;
let fresh_operation_id = {
let state = self.lock()?;
let worker = state.worker(worker_ref)?;
@@ -3007,6 +3116,7 @@ impl Runtime {
fn ensure_worker_operation_settled(&self, worker_ref: &WorkerRef) -> Result<(), RuntimeError> {
let state = self.lock()?;
state.ensure_subject_not_reserved(worker_ref, false)?;
let worker = state.worker(worker_ref)?;
if let Some(operation) = worker.pending_lifecycle_operation_name() {
return Err(RuntimeError::WorkerExecutionUnavailable {
@@ -3803,6 +3913,7 @@ impl Runtime {
self.ensure_worker_in_workspace(scope, worker_ref)?;
}
self.lock()?.ensure_subject_not_reserved(worker_ref, true)?;
let (backend, pending_stop, pending_restore, status) = {
let state = self.lock()?;
state.ensure_running()?;
@@ -3932,7 +4043,11 @@ impl Runtime {
scope: &RuntimeWorkspaceScope,
worker_ref: &WorkerRef,
) -> Result<WorkerDeleteResult, RuntimeError> {
self.delete_worker_with_scope(worker_ref, Some(scope))
self.delete_worker_with_scope(
worker_ref,
Some(scope),
session_store::retention::SessionDeletionIntent::Ordinary,
)
}
/// Delete a non-running Worker from Runtime state and persisted Worker storage.
@@ -3940,13 +4055,30 @@ impl Runtime {
&self,
worker_ref: &WorkerRef,
) -> Result<WorkerDeleteResult, RuntimeError> {
self.delete_worker_with_scope(worker_ref, None)
self.delete_worker_with_scope(
worker_ref,
None,
session_store::retention::SessionDeletionIntent::Ordinary,
)
}
/// A trusted Host must choose explicit erase intent separately from policy
/// Purge. This also permits finishing aggregate removal after source erasure.
#[cfg(feature = "fs-store")]
pub fn delete_worker_with_intent_scoped(
&self,
scope: &RuntimeWorkspaceScope,
worker_ref: &WorkerRef,
intent: session_store::retention::SessionDeletionIntent,
) -> Result<WorkerDeleteResult, RuntimeError> {
self.delete_worker_with_scope(worker_ref, Some(scope), intent)
}
fn delete_worker_with_scope(
&self,
worker_ref: &WorkerRef,
scope: Option<&RuntimeWorkspaceScope>,
intent: session_store::retention::SessionDeletionIntent,
) -> Result<WorkerDeleteResult, RuntimeError> {
let operation_lock = self.worker_operation_lock(worker_ref.worker_id)?;
let _operation_guard = operation_lock
@@ -3959,6 +4091,7 @@ impl Runtime {
let state = self.lock()?;
state.ensure_running()?;
state.ensure_worker_ref(worker_ref)?;
state.ensure_subject_not_reserved(worker_ref, true)?;
let worker = state.worker(worker_ref)?;
if let Some(operation) = worker.pending_lifecycle_operation_name() {
return Err(RuntimeError::InvalidRequest(format!(
@@ -4002,7 +4135,17 @@ impl Runtime {
worker_ref.worker_id
)));
}
if state.delete_worker_record(&worker_ref.worker_id).is_err() {
#[cfg(feature = "fs-store")]
let deletion = match state.fs_store() {
Some(store) => store.delete_worker_record_with_intent(&worker_ref.worker_id, intent),
None => Ok(()),
};
#[cfg(not(feature = "fs-store"))]
let deletion = {
let _ = intent;
state.delete_worker_record(&worker_ref.worker_id)
};
if deletion.is_err() {
state.record_worker_delete_persistence_failure(worker_ref);
let _diagnostic_persistence = state.persist_runtime_snapshot();
return Err(RuntimeError::WorkerDeletePersistenceFailed {
@@ -4239,7 +4382,10 @@ impl Runtime {
.workers
.values()
.filter(|worker| {
worker.restore_guard.active_request_id.is_none()
state
.ensure_subject_not_reserved(&worker.worker_ref, false)
.is_ok()
&& worker.restore_guard.active_request_id.is_none()
&& worker.pending_restore.is_none()
&& worker.pending_stop.is_none()
&& worker.execution_bound
@@ -4282,7 +4428,12 @@ impl Runtime {
state
.workers
.values()
.filter(|worker| worker.pending_stop.is_none())
.filter(|worker| {
worker.pending_stop.is_none()
&& state
.ensure_subject_not_reserved(&worker.worker_ref, false)
.is_ok()
})
.filter_map(|worker| {
worker
.pending_restore
@@ -4773,6 +4924,277 @@ impl Runtime {
)
}
/// Retain only an authenticated source whose saved Session identity matches.
/// The existence check and pin publication share the physical deletion lock.
#[cfg(feature = "fs-store")]
pub fn pin_session_public_source_scoped(
&self,
scope: &RuntimeWorkspaceScope,
source: &runtime_api::SessionPublicSource,
expected_session_id: &str,
pin: session_store::retention::SessionRetentionPin,
) -> Result<(), RuntimeError> {
let (root, _) = self.resolve_session_public_source(scope, source)?;
let session = expected_session_id
.parse::<session_store::SessionId>()
.map_err(|_| RuntimeError::InvalidRequest("Invalid Session identity".into()))?;
self.session_retention_store()?
.pin(session, pin, || {
let store = match session_store::WorkerSessionStore::open_read_only(&root) {
Ok(store) => store,
Err(session_store::StoreError::Io(error))
if error.kind() == std::io::ErrorKind::NotFound =>
{
return Ok(false);
}
Err(error) => return Err(error),
};
Ok(store.session_id()? == Some(session))
})
.map_err(crate::retention::retention_error)
}
#[cfg(feature = "fs-store")]
pub fn release_session_public_source_pin_scoped(
&self,
scope: &RuntimeWorkspaceScope,
source: &runtime_api::SessionPublicSource,
expected_session_id: &str,
pin: session_store::retention::SessionRetentionPin,
) -> Result<(), RuntimeError> {
let (root, archive) =
self.resolve_session_public_source_with_validation(scope, source, false)?;
let session = expected_session_id
.parse::<session_store::SessionId>()
.map_err(|_| RuntimeError::InvalidRequest("Invalid Session identity".into()))?;
let owned_session = match archive {
Some(manifest) => manifest.source_session_id.parse().ok(),
None => {
crate::retention::aggregate_session_id(root.parent().ok_or_else(|| {
RuntimeError::InvalidRequest("Session root unavailable".into())
})?)?
}
};
if owned_session != Some(session) {
return Err(RuntimeError::InvalidRequest(
"Session source identity mismatch".into(),
));
}
self.session_retention_store()?
.release(session, pin)
.map_err(crate::retention::retention_error)
}
/// Physical source deletion is separate from Worker registration/lifecycle.
/// Only this explicit intent may invalidate pins and persist an erase fence;
/// a retention policy's Purge remains ordinary deletion.
#[cfg(feature = "fs-store")]
pub fn delete_session_public_source_scoped(
&self,
scope: &RuntimeWorkspaceScope,
source: &runtime_api::SessionPublicSource,
expected_session_id: &str,
intent: session_store::retention::SessionDeletionIntent,
) -> Result<(), RuntimeError> {
let (root, archive) =
self.resolve_session_public_source_with_validation(scope, source, false)?;
let owner = match source {
runtime_api::SessionPublicSource::Retained { worker_id } => *worker_id,
runtime_api::SessionPublicSource::Archived { .. } => {
archive
.as_ref()
.ok_or_else(|| {
RuntimeError::InvalidRequest("Archive identity unavailable".into())
})?
.source_worker_id
}
};
let operation_lock = self.worker_operation_lock(owner)?;
let _operation_guard = operation_lock
.lock()
.map_err(|_| RuntimeError::StatePoisoned)?;
let state = self.lock()?;
if let Some(worker) = state.workers.get(&owner) {
if worker.workspace_id.as_deref() != Some(scope.workspace_id.as_str())
|| worker.status != WorkerStatus::Stopped
|| worker.pending_lifecycle_operation_name().is_some()
{
return Err(RuntimeError::InvalidRequest(
"Session deletion requires a stopped, settled source Worker".into(),
));
}
}
let store = state.fs_store().ok_or_else(|| {
RuntimeError::InvalidRequest("Session deletion requires filesystem storage".into())
})?;
let session = expected_session_id
.parse::<session_store::SessionId>()
.map_err(|_| RuntimeError::InvalidRequest("Invalid Session identity".into()))?;
let source_session = match archive.as_ref() {
Some(manifest) => manifest
.source_session_id
.parse::<session_store::SessionId>()
.ok(),
None => {
crate::retention::aggregate_session_id(root.parent().ok_or_else(|| {
RuntimeError::InvalidRequest("Source root unavailable".into())
})?)?
}
};
if source_session != Some(session) {
return Err(RuntimeError::InvalidRequest(
"Session source identity mismatch".into(),
));
}
crate::retention::runtime_session_retention(store.runtime_dir())
.delete(session, intent, || {
let saved = match archive.as_ref() {
Some(manifest) => manifest
.source_session_id
.parse::<session_store::SessionId>()
.ok(),
None => crate::retention::aggregate_session_id(
root.parent()
.ok_or(session_store::StoreError::SessionSourceMissing(session))?,
)
.map_err(|error| {
session_store::StoreError::Io(std::io::Error::other(error.to_string()))
})?,
};
if saved != Some(session) {
return Err(session_store::StoreError::SessionSourceMissing(session));
}
if intent == session_store::retention::SessionDeletionIntent::ErasePermanently {
crate::retention::remove_archived_session_copies(store.runtime_dir(), session)?;
let original = store
.runtime_dir()
.join("workers")
.join(owner.to_string())
.join("session");
if original != root
&& original.is_dir()
&& session_store::WorkerSessionStore::open_read_only(&original)?
.session_id()?
== Some(session)
{
std::fs::remove_dir_all(original)?;
}
}
match std::fs::remove_dir_all(&root) {
Ok(()) => Ok(()),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(error) => Err(error.into()),
}
})
.map_err(crate::retention::retention_error)
}
#[cfg(feature = "fs-store")]
pub fn session_material_catalog_scoped(
&self,
scope: &RuntimeWorkspaceScope,
request: &runtime_api::SessionMaterialCatalogRequest,
) -> runtime_api::SessionMaterialCatalogAvailability {
crate::session_material::catalog(self, scope, request)
}
#[cfg(feature = "fs-store")]
pub fn session_material_read_scoped(
&self,
scope: &RuntimeWorkspaceScope,
request: &runtime_api::SessionMaterialReadRequest,
) -> runtime_api::SessionMaterialReadAvailability {
crate::session_material::read(self, scope, request)
}
#[cfg(feature = "fs-store")]
fn session_retention_store(
&self,
) -> Result<session_store::retention::SessionRetentionStore, RuntimeError> {
let state = self.lock()?;
let store = state.fs_store().ok_or_else(|| {
RuntimeError::InvalidRequest("Session retention requires an fs-backed Runtime".into())
})?;
Ok(crate::retention::runtime_session_retention(
store.runtime_dir(),
))
}
/// Prove an attributed but not yet materialized Session without treating a
/// missing committed log as empty. The source/Workspace binding is resolved first.
#[cfg(feature = "fs-store")]
pub(crate) fn session_not_materialized_scoped(
&self,
scope: &RuntimeWorkspaceScope,
source: &runtime_api::SessionPublicSource,
expected_session_id: &str,
) -> Result<bool, RuntimeError> {
let (root, archive) = self.resolve_session_public_source(scope, source)?;
if archive.is_some() {
return Ok(false);
}
let session = expected_session_id
.parse::<session_store::SessionId>()
.map_err(|_| RuntimeError::InvalidRequest("Invalid Session identity".into()))?;
let retention = self.session_retention_store()?;
let _guard = retention
.lock_live(session)
.map_err(crate::retention::retention_error)?;
Ok(crate::retention::unmaterialized_session(
root.parent()
.ok_or_else(|| RuntimeError::InvalidRequest("Session root unavailable".into()))?,
&scope.workspace_id,
)? == Some(session))
}
/// Resolve the Backend-selected source without exposing local storage paths.
/// The saved active pointer, never a caller-provided Segment, selects lineage.
#[cfg(feature = "fs-store")]
pub fn session_public_material_reader_scoped(
&self,
scope: &RuntimeWorkspaceScope,
source: &runtime_api::SessionPublicSource,
expected_session_id: &str,
) -> Result<
session_store::public_material::SessionPublicMaterialReader,
crate::worker_backend::SessionMaterialSourceError,
> {
use crate::worker_backend::SessionMaterialSourceError::Source;
use session_store::public_material::{PublicMaterialLimits, SessionPublicMaterialReader};
let (session_root, archive) = self
.resolve_session_public_source(scope, source)
.map_err(|error| Source(error.to_string()))?;
let reader = match (source, archive) {
(runtime_api::SessionPublicSource::Retained { worker_id }, None) => {
crate::worker_backend::open_session_public_material_reader(
session_root.parent().ok_or("Session root is unavailable")?,
&scope.workspace_id,
&WorkerRef::new(*worker_id),
expected_session_id,
)?
}
(runtime_api::SessionPublicSource::Archived { .. }, Some(manifest)) => {
if manifest.source_session_id != expected_session_id {
return Err(Source("Archive Session identity mismatch".into()));
}
SessionPublicMaterialReader::open_worker_with_retention(
&session_root,
manifest
.source_active_segment_id
.ok_or("Archive has no canonical active Segment")?,
PublicMaterialLimits::default(),
self.session_retention_store()
.map_err(|error| Source(error.to_string()))?,
)?
}
_ => return Err(Source("Session source is unavailable".into())),
};
if reader.session_id().to_string() != expected_session_id {
return Err(Source("Session identity mismatch".into()));
}
Ok(reader)
}
/// Search the committed public projection of one Backend-selected retained
/// Session or verified archive without restoring a Worker.
#[cfg(feature = "fs-store")]
@@ -4844,6 +5266,22 @@ impl Runtime {
Option<crate::retention::WorkerSessionArchiveManifest>,
),
RuntimeError,
> {
self.resolve_session_public_source_with_validation(scope, source, true)
}
#[cfg(feature = "fs-store")]
fn resolve_session_public_source_with_validation(
&self,
scope: &RuntimeWorkspaceScope,
source: &runtime_api::SessionPublicSource,
verify_archive: bool,
) -> Result<
(
std::path::PathBuf,
Option<crate::retention::WorkerSessionArchiveManifest>,
),
RuntimeError,
> {
let state = self.lock()?;
let store = state.fs_store().ok_or_else(|| {
@@ -4876,7 +5314,11 @@ impl Runtime {
)
})?;
let provider = FsWorkerRetentionProvider::new(store.runtime_dir());
let (manifest, session_root) = provider.session_archive(archive_id)?;
let (manifest, session_root) = if verify_archive {
provider.session_archive(archive_id)?
} else {
provider.session_archive_identity(archive_id)?
};
if manifest.workspace_id != scope.workspace_id
|| manifest.source_runtime_id != runtime_id
{
@@ -4921,6 +5363,10 @@ impl Runtime {
&self,
request: &WorkerRetentionExecutionRequest,
) -> Result<WorkerRetentionExecutionResult, RuntimeError> {
let operation_lock = self.worker_operation_lock(request.worker_id)?;
let _operation_guard = operation_lock
.lock()
.map_err(|_| RuntimeError::StatePoisoned)?;
let mut state = self.lock()?;
let runtime_id = state.runtime_identity.clone().ok_or_else(|| {
RuntimeError::InvalidRequest(
@@ -5036,6 +5482,13 @@ struct RuntimeState {
#[cfg(feature = "fs-store")]
next_diagnostic_id: u64,
workers: BTreeMap<WorkerId, WorkerRecord>,
subject_release_operations: BTreeMap<
(WorkerId, String),
(
runtime_api::SubjectReleaseRequest,
runtime_api::SubjectReleaseState,
),
>,
workdir_operations: BTreeMap<String, WorkdirOperation>,
workspace_owners: BTreeMap<String, String>,
config_bundles: BTreeMap<String, ConfigBundle>,
@@ -5066,6 +5519,7 @@ impl RuntimeState {
#[cfg(feature = "fs-store")]
next_diagnostic_id: 1,
workers: BTreeMap::new(),
subject_release_operations: BTreeMap::new(),
workdir_operations: BTreeMap::new(),
workspace_owners: BTreeMap::new(),
config_bundles: BTreeMap::new(),
@@ -5097,6 +5551,7 @@ impl RuntimeState {
#[cfg(feature = "fs-store")]
next_diagnostic_id: 1,
workers: BTreeMap::new(),
subject_release_operations: BTreeMap::new(),
workdir_operations: BTreeMap::new(),
workspace_owners: BTreeMap::new(),
config_bundles: BTreeMap::new(),
@@ -5242,6 +5697,16 @@ impl RuntimeState {
workspace_backend_resource_clients: BTreeMap::new(),
next_diagnostic_id,
workers,
subject_release_operations: persisted
.subject_releases
.into_iter()
.map(|(request, state)| {
(
(request.worker_id, request.operation_id.clone()),
(request, state),
)
})
.collect(),
workdir_operations: BTreeMap::new(),
config_bundles: BTreeMap::new(),
workspace_config_latest: BTreeMap::new(),
@@ -5262,6 +5727,7 @@ impl RuntimeState {
#[cfg(feature = "fs-store")]
fn persisted_state(&self) -> PersistedRuntimeState {
PersistedRuntimeState {
subject_releases: self.subject_release_operations.values().cloned().collect(),
display_name: self.display_name.clone(),
status: self.status,
next_diagnostic_id: self.next_diagnostic_id,
@@ -5405,6 +5871,28 @@ impl RuntimeState {
Ok(())
}
fn ensure_subject_not_reserved(
&self,
worker: &WorkerRef,
allow_disconnected: bool,
) -> Result<(), RuntimeError> {
use runtime_api::SubjectReleaseState as State;
if self
.subject_release_operations
.iter()
.any(|((id, _), (_, state))| {
*id == worker.worker_id
&& (*state == State::Reserved
|| (!allow_disconnected && *state == State::Disconnected))
})
{
return Err(RuntimeError::InvalidRequest(
"Subject Controller is reserved for release or disconnected".into(),
));
}
Ok(())
}
fn ensure_worker_ref(&self, worker_ref: &WorkerRef) -> Result<(), RuntimeError> {
if !self.workers.contains_key(&worker_ref.worker_id) {
return Err(RuntimeError::WorkerNotFound {
@@ -7302,6 +7790,7 @@ mod tests {
attempt_id: "attempt-1".into(),
input_digest: Some(format!("sha256:{}", "a".repeat(64))),
subjektiv_consolidation: false,
subjektiv_journal: false,
});
let owner = scope("local", "server-job-test");
let created = runtime
@@ -7351,6 +7840,7 @@ mod tests {
attempt_id: "attempt-1".into(),
input_digest: None,
subjektiv_consolidation: true,
subjektiv_journal: false,
});
assert!(validate_create_worker_request(&request).is_err());
request.workspace_api = Some(WorkspaceApiRef {
@@ -7877,6 +8367,21 @@ mod tests {
}
impl WorkerExecutionBackend for TestExecutionBackend {
fn subject_release(
&self,
request: &runtime_api::SubjectReleaseRequest,
) -> Result<runtime_api::SubjectReleaseResponse, String> {
use runtime_api::{SubjectReleaseAction as A, SubjectReleaseState as S};
Ok(runtime_api::SubjectReleaseResponse {
operation_id: request.operation_id.clone(),
binding_id: request.binding_id.clone(),
state: match request.action {
A::Reserve => S::Reserved,
A::Cancel => S::Cancelled,
A::Disconnect => S::Disconnected,
},
})
}
fn cleanup_working_directory(
&self,
_id: &str,
@@ -11818,6 +12323,7 @@ mod tests {
},
state: protocol::WorkerStatus::Running.into(),
in_flight: protocol::InFlightSnapshot {
block_ids: Vec::new(),
blocks: Vec::new(),
commands: Vec::new(),
compaction: None,
@@ -12776,6 +13282,83 @@ mod tests {
let _ = std::fs::remove_dir_all(root);
}
// Persistence component test; real Controller reservation has separate coverage.
#[cfg(feature = "fs-store")]
#[test]
fn subject_release_fence_survives_reopen_without_automatic_or_explicit_restore() {
let root = tempfile::tempdir().unwrap();
let options = crate::fs_store::FsRuntimeStoreOptions::new(root.path())
.with_runtime_id("test-runtime");
let scoped = scope("workspace-a", "server-a");
let runtime = Runtime::with_fs_store_and_execution_backend(
options.clone(),
Arc::new(TestExecutionBackend::default()),
)
.unwrap();
runtime.store_config_bundle(test_bundle()).unwrap();
let worker = runtime
.create_worker_scoped(
&scoped,
scoped_task_request("release durable", "workspace-a"),
)
.unwrap();
let mut request = runtime_api::SubjectReleaseRequest {
workspace_id: "workspace-a".into(),
worker_id: worker.worker_id,
binding_id: "writer-1".into(),
operation_id: "release-1".into(),
action: runtime_api::SubjectReleaseAction::Reserve,
};
runtime
.subject_release_scoped(&scoped, request.clone())
.unwrap();
drop(runtime);
let backend = Arc::new(TestExecutionBackend::default());
let reopened =
Runtime::with_fs_store_and_execution_backend(options.clone(), backend.clone()).unwrap();
assert_eq!(*backend.restore_count.lock().unwrap(), 0);
assert!(
reopened
.send_input(&worker.worker_ref, WorkerInput::user("must remain fenced"))
.is_err()
);
assert!(
reopened
.restore_worker(
&worker.worker_ref,
reopened.test_restore_request(&worker.worker_ref)
)
.is_err()
);
request.action = runtime_api::SubjectReleaseAction::Cancel;
assert_eq!(
reopened
.subject_release_scoped(&scoped, request.clone())
.unwrap()
.state,
runtime_api::SubjectReleaseState::Cancelled
);
request.action = runtime_api::SubjectReleaseAction::Reserve;
assert!(
reopened
.subject_release_scoped(&scoped, request.clone())
.is_err(),
"cancelled identity must never rearm"
);
drop(reopened);
let reopened = Runtime::with_fs_store_and_execution_backend(
options,
Arc::new(TestExecutionBackend::default()),
)
.unwrap();
assert!(
reopened
.subject_release_scoped(&scoped, request.clone())
.is_err(),
"terminal cancellation survives restart"
);
}
#[cfg(feature = "fs-store")]
#[test]
fn fs_store_restores_active_worker_executions() {
@@ -0,0 +1,166 @@
//! Bounded wire projection of canonical public material, never Memory previews.
use runtime_api::*;
use session_store::public_material::{
PublicMaterialDescriptor, PublicMaterialError, SessionPublicMaterialReader,
};
use sha2::{Digest, Sha256};
use crate::{Runtime, RuntimeWorkspaceScope, worker_backend::SessionMaterialSourceError};
fn descriptor(
value: &PublicMaterialDescriptor,
) -> Result<SessionMaterialDescriptor, PublicMaterialError> {
let result = SessionMaterialDescriptor {
segment_id: value.identity.segment_id.to_string(),
entry_ref: value.identity.entry_ref.clone(),
canonical_order: value.canonical_order,
origin_json: serde_json::to_string(&value.origin)?,
sha256: value.sha256.clone(),
total_bytes: value.total_bytes,
};
if serde_json::to_vec(&result)?.len() > 16 * 1024 {
return Err(PublicMaterialError::ResourceLimit);
}
Ok(result)
}
fn catalog_digest(reader: &SessionPublicMaterialReader) -> Result<String, PublicMaterialError> {
let mut digest = Sha256::new();
digest.update(reader.session_id().to_string());
digest.update(reader.active_segment_id().to_string());
for value in reader.descriptors() {
digest.update(serde_json::to_vec(&descriptor(value)?)?);
digest.update(b"\n");
}
Ok(digest
.finalize()
.iter()
.map(|byte| format!("{byte:02x}"))
.collect())
}
fn unavailable(error: SessionMaterialSourceError) -> SessionPublicUnavailableReason {
use PublicMaterialError as E;
use SessionPublicUnavailableReason as R;
match error {
SessionMaterialSourceError::Material(E::ResourceLimit) => R::ResourceLimit,
SessionMaterialSourceError::Material(E::Changed | E::InvalidOffset) => R::InvalidCursor,
SessionMaterialSourceError::Material(E::Missing | E::Erased) => R::RetentionMissing,
SessionMaterialSourceError::Material(E::Inconsistent | E::Record(_)) => R::CorruptLog,
_ => R::StorageUnavailable,
}
}
pub(crate) fn catalog(
runtime: &Runtime,
scope: &RuntimeWorkspaceScope,
request: &SessionMaterialCatalogRequest,
) -> SessionMaterialCatalogAvailability {
if scope.workspace_id == request.workspace_id
&& (1..=100).contains(&request.limit)
&& request.offset == 0
&& request.expected_catalog_sha256.is_none()
&& runtime
.session_not_materialized_scoped(scope, &request.source, &request.expected_session_id)
.is_ok_and(|empty| empty)
{
return SessionMaterialCatalogAvailability::NotMaterialized {
session_id: request.expected_session_id.clone(),
};
}
let result = (|| -> Result<_, SessionMaterialSourceError> {
if scope.workspace_id != request.workspace_id {
return Err("Session Workspace scope mismatch".into());
}
if request.limit == 0 || request.limit > 100 {
return Err(PublicMaterialError::ResourceLimit.into());
}
let reader = runtime.session_public_material_reader_scoped(
scope,
&request.source,
&request.expected_session_id,
)?;
let digest = catalog_digest(&reader)?;
if request
.expected_catalog_sha256
.as_ref()
.is_some_and(|expected| expected != &digest)
{
return Err(PublicMaterialError::Changed.into());
}
let offset =
usize::try_from(request.offset).map_err(|_| PublicMaterialError::InvalidOffset)?;
let count = reader.descriptors().len();
if offset > count {
return Err(PublicMaterialError::InvalidOffset.into());
}
let descriptors = reader
.descriptors()
.skip(offset)
.take(request.limit)
.map(descriptor)
.collect::<Result<Vec<_>, _>>()?;
let end = offset + descriptors.len();
Ok(SessionMaterialCatalogPage {
session_id: reader.session_id().to_string(),
active_segment_id: reader.active_segment_id().to_string(),
catalog_sha256: digest,
descriptors,
next_offset: (end < count).then_some(end as u64),
})
})();
match result {
Ok(page) => SessionMaterialCatalogAvailability::Page { page },
Err(error) => SessionMaterialCatalogAvailability::Unavailable {
reason: unavailable(error),
message: "Canonical Session catalog unavailable".into(),
},
}
}
pub(crate) fn read(
runtime: &Runtime,
scope: &RuntimeWorkspaceScope,
request: &SessionMaterialReadRequest,
) -> SessionMaterialReadAvailability {
let result = (|| -> Result<_, SessionMaterialSourceError> {
if scope.workspace_id != request.workspace_id {
return Err("Session Workspace scope mismatch".into());
}
if !(4..=16 * 1024).contains(&request.max_content_bytes) {
return Err(PublicMaterialError::ResourceLimit.into());
}
let reader = runtime.session_public_material_reader_scoped(
scope,
&request.source,
&request.expected_session_id,
)?;
// Resolve saved identity/order/digest here; no caller-supplied descriptor is adopted.
// New tail entries may arrive during a Job without changing its frozen originals.
let selected = reader
.descriptors()
.find(|value| {
value.identity.segment_id.to_string() == request.expected_descriptor.segment_id
&& value.identity.entry_ref == request.expected_descriptor.entry_ref
})
.ok_or(PublicMaterialError::Missing)?;
if descriptor(selected)? != request.expected_descriptor {
return Err(PublicMaterialError::Changed.into());
}
let content =
reader.read_content(selected, request.byte_offset, request.max_content_bytes)?;
Ok(SessionMaterialReadPage {
descriptor: descriptor(selected)?,
content: content.content,
next_offset: content.next_offset,
})
})();
match result {
Ok(page) => SessionMaterialReadAvailability::Page { page },
Err(error) => SessionMaterialReadAvailability::Unavailable {
reason: unavailable(error),
message: "Canonical Session material unavailable".into(),
},
}
}
@@ -364,6 +364,7 @@ fn api_archive_manifest(manifest: &DomainArchiveManifest) -> WorkerSessionArchiv
source_runtime_id: manifest.source_runtime_id.clone(),
source_worker_id: manifest.source_worker_id,
source_session_id: manifest.source_session_id.clone(),
source_active_segment_id: manifest.source_active_segment_id.map(|id| id.to_string()),
segment_ids: manifest.segment_ids.clone(),
source_created_at: manifest.source_created_at.clone(),
removed_at: manifest.removed_at.clone(),
@@ -0,0 +1,9 @@
//! One control namespace for live Worker sources and their verified archives.
//! It outlives aggregate removal and is also used by publishers and readers.
use session_store::retention::SessionRetentionStore;
use std::path::Path;
pub fn runtime_session_retention(runtime_root: &Path) -> SessionRetentionStore {
SessionRetentionStore::new(runtime_root.join("retention/session-sources"))
}
File diff suppressed because it is too large Load Diff
+6 -4
View File
@@ -1816,10 +1816,12 @@ mod tests {
}
#[test]
fn runtime_owned_workspace_client_has_no_fixed_request_timeout() {
// This must exceed reqwest::blocking::Client's default 30-second timeout.
// A short delay only checks our Option field, not the effective HTTP deadline.
let (base_url, server) = delayed_workspace_response(Duration::from_secs(31));
fn runtime_owned_workspace_client_returns_successful_response_without_request_timeout() {
// Effective deadline clearing is exercised with a short inherited deadline in
// workspace_request::tests::blocking_workspace_client_clears_initial_request_timeout.
let (base_url, server) = binary_loopback(
b"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: 11\r\nConnection: close\r\n\r\n{\"ok\":true}".to_vec(),
);
let client =
RuntimeOwnedWorkspaceClient::new("workspace-a", base_url, "runtime-a", "worker-a");
assert_eq!(client.request_timeout, None);
@@ -1223,6 +1223,7 @@ pub(super) mod tests {
}
fn sized_fixture(
repo: &Path,
size: u64,
) -> (
tempfile::TempDir,
@@ -1230,7 +1231,9 @@ pub(super) mod tests {
WorkingDirectoryBinding,
BodyWatch,
) {
let (runtime, materializer, binding) = fixture();
let runtime = tempfile::tempdir().unwrap();
let materializer = RuntimeGitMaterializer::new(runtime.path());
let binding = materializer.create(&request(repo)).unwrap();
fs::write(binding.root.join(".git/info/exclude"), "target/\n").unwrap();
fs::create_dir(binding.root.join("target")).unwrap();
// Entry count remains constant, including .git and many ignored files.
@@ -1256,10 +1259,14 @@ pub(super) mod tests {
#[test]
fn actual_cleanup_and_restart_retry_never_open_bodies_when_only_file_sizes_grow() {
use io_observation::observe;
// Vary only file sizes, not Git object names/layout. Separate commits
// can have different object-prefix directory counts as timestamps change.
// Each case still owns an independent materialized checkout and Runtime.
let repo = create_clean_repo();
for failure in [None, Some(0), Some(10)] {
let mut small_counts = None;
for size in [256, 128 * 1024 * 1024] {
let (runtime, materializer, binding, mut watch) = sized_fixture(size);
let (runtime, materializer, binding, mut watch) = sized_fixture(repo.path(), size);
let id = &binding.working_directory.id;
let (first, first_io) =
observe(failure, || materializer.cleanup_working_directory(id));
+80 -13
View File
@@ -262,12 +262,14 @@ impl RuntimeWorkspaceRequestClient {
request: RuntimeWorkspaceRequest,
) -> Result<RuntimeWorkspaceResponse, RuntimeWorkspaceRequestError> {
let client = self.clone();
std::thread::spawn(move || client.execute_blocking_inner(request, true))
.join()
.map_err(|_| RuntimeWorkspaceRequestError::Transport {
message: "Workspace request thread panicked".to_string(),
timeout: false,
})?
std::thread::spawn(move || {
client.execute_blocking_inner(request, true, reqwest::blocking::Client::builder())
})
.join()
.map_err(|_| RuntimeWorkspaceRequestError::Transport {
message: "Workspace request thread panicked".to_string(),
timeout: false,
})?
}
/// Binary requests must retain their exact target and never follow redirects.
@@ -276,18 +278,21 @@ impl RuntimeWorkspaceRequestClient {
request: RuntimeWorkspaceRequest,
) -> Result<RuntimeWorkspaceResponse, RuntimeWorkspaceRequestError> {
let client = self.clone();
std::thread::spawn(move || client.execute_blocking_inner(request, false))
.join()
.map_err(|_| RuntimeWorkspaceRequestError::Transport {
message: "Workspace binary request thread panicked".to_string(),
timeout: false,
})?
std::thread::spawn(move || {
client.execute_blocking_inner(request, false, reqwest::blocking::Client::builder())
})
.join()
.map_err(|_| RuntimeWorkspaceRequestError::Transport {
message: "Workspace binary request thread panicked".to_string(),
timeout: false,
})?
}
fn execute_blocking_inner(
&self,
request: RuntimeWorkspaceRequest,
follow_redirects: bool,
client_builder: reqwest::blocking::ClientBuilder,
) -> Result<RuntimeWorkspaceResponse, RuntimeWorkspaceRequestError> {
let prepared = self.prepare(&request)?;
if !follow_redirects {
@@ -304,7 +309,7 @@ impl RuntimeWorkspaceRequestClient {
}
// The blocking client defaults to 30 seconds. Apply None explicitly so
// provider-owned command waits are not cut short by an HTTP deadline.
let mut client_builder = reqwest::blocking::Client::builder().timeout(request.timeout);
let mut client_builder = client_builder.timeout(request.timeout);
if !follow_redirects {
client_builder = client_builder.redirect(reqwest::redirect::Policy::none());
}
@@ -425,6 +430,68 @@ mod tests {
use super::*;
use crate::auth::decode_runtime_request_source_claims;
// Component boundary: production request preparation, timeout policy and HTTP execution;
// only the initial client deadline and remote Workspace are supplied by the fixture.
#[tokio::test]
async fn blocking_workspace_client_clears_initial_request_timeout() {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
tokio::time::timeout(Duration::from_secs(5), async {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.unwrap();
let address = listener.local_addr().unwrap();
let initial_timeout = Duration::from_millis(20);
let mut request = tokio::task::spawn_blocking(move || {
let builder = reqwest::blocking::Client::builder()
.no_proxy()
.timeout(initial_timeout);
RuntimeWorkspaceRequestClient::new("workspace-a", format!("http://{address}"), "runtime-a")
.execute_blocking_inner(
RuntimeWorkspaceRequest {
method: reqwest::Method::GET,
path_and_query: "/api/test".into(),
body: Vec::new(),
headers: HeaderMap::new(),
permission: "test".into(),
worker_id: None,
timeout: None,
max_response_bytes: 1024,
},
true,
builder,
)
});
let receive_and_cross_deadline = async {
let (mut stream, _) = listener.accept().await.unwrap();
let mut headers = Vec::new();
while !headers.ends_with(b"\r\n\r\n") {
headers.push(stream.read_u8().await.unwrap());
assert!(headers.len() <= 8192, "unbounded request headers");
}
// Deliberately cross the inherited deadline after request arrival.
// This timer supplies the fault stimulus, not a completion signal.
tokio::time::sleep(initial_timeout * 2).await;
stream
};
let mut stream = tokio::select! {
result = &mut request => panic!("request ended before response: {result:?}"),
stream = receive_and_cross_deadline => stream,
};
stream
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 11\r\nConnection: close\r\n\r\n{\"ok\":true}")
.await
.unwrap();
drop(stream);
let response = request.await.unwrap().unwrap();
assert_eq!(response.status, reqwest::StatusCode::OK);
assert_eq!(response.body, br#"{"ok":true}"#);
})
.await
.expect("loopback request did not complete");
}
#[test]
fn route_issues_workspace_scoped_request_proof() {
let identity = RuntimeIdentityMaterial::generate("runtime-a").unwrap();
+1
View File
@@ -43,6 +43,7 @@ libc = { workspace = true }
schemars = { workspace = true }
ticket = { workspace = true }
memory = { workspace = true }
subjektiv = { workspace = true }
uuid = { workspace = true, features = ["v7"] }
session-metrics = { workspace = true }
arc-swap = "1.9.1"
+32
View File
@@ -13,9 +13,16 @@ pub struct BackendJobExecutionBinding {
pub input_digest: Option<String>,
#[serde(default)]
pub subjektiv_consolidation: bool,
/// Trusted Host grant, independent of reusable Profile policy and Job purpose.
#[serde(default)]
pub subjektiv_journal: bool,
}
impl BackendJobExecutionBinding {
/// Subject maintenance Jobs are not the Subject's main conversation Session.
pub(crate) fn has_subjektiv_job_grant(&self) -> bool {
self.subjektiv_consolidation || self.subjektiv_journal
}
pub fn validate(&self) -> Result<(), &'static str> {
if [&self.job_id, &self.attempt_id]
.into_iter()
@@ -24,6 +31,31 @@ impl BackendJobExecutionBinding {
{
return Err("Backend Job binding contains an invalid identity or input digest");
}
if self.subjektiv_consolidation && self.subjektiv_journal {
return Err("Backend Job cannot combine Memory consolidation and Journal grants");
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn omitted_journal_grant_is_disabled_and_mixed_subject_grants_are_rejected() {
let mut binding: BackendJobExecutionBinding = serde_json::from_value(serde_json::json!({
"job_id":"job", "attempt_id":"attempt", "subjektiv_consolidation":true,
}))
.unwrap();
assert!(!binding.subjektiv_journal);
binding.subjektiv_journal = true;
assert!(binding.validate().is_err());
assert!(
serde_json::from_value::<BackendJobExecutionBinding>(serde_json::json!({
"job_id":"job", "attempt_id":"attempt", "purpose":"journal"
}))
.is_err(),
"purpose is not a grant field"
);
}
}
+182 -6
View File
@@ -78,7 +78,45 @@ impl WorkerHandle {
self.method_tx.same_channel(&other.method_tx)
}
/// Ask the actual Controller to acknowledge quiescence. No catalog status,
/// profile, or model protocol message grants this Host lifecycle capability.
pub async fn reserve_subject_release(
&self,
) -> Result<crate::SubjectReleaseReservation, String> {
if self.protocol_is_closed() {
return Err("Controller is unavailable".into());
}
if self.shared_state.snapshot().state != WorkerState::Idle {
return Err("Controller is not Idle".into());
}
let control = self
.shared_state
.subject_release
.get()
.ok_or("Controller release authority unavailable")?;
tokio::select! {
result = control.request() => result,
_ = self.method_tx.closed() => Err("Controller closed before release acknowledgement".into()),
}
}
pub async fn send(&self, method: Method) -> Result<(), mpsc::error::SendError<Method>> {
// The lease covers enqueue itself. The Controller's exclusive barrier
// cannot overlook an accepted send between its queue check and ack.
let _admission = if release_fenced_method(&method) {
match self
.shared_state
.subject_release
.get()
.map(|control| control.authority.enter())
.transpose()
{
Ok(lease) => lease,
Err(_) => return Err(mpsc::error::SendError(method)),
}
} else {
None
};
// Reject known-busy sends before channel dispatch, including maintenance
// that cannot receive methods until it finishes. The controller checks
// again at admission to cover an Idle -> busy race after this snapshot.
@@ -290,6 +328,49 @@ impl WorkerHandle {
}
}
fn answer_idle_release<St: Store + Clone>(
control: &Arc<crate::subject_release::ReleaseControl>,
shared_state: &WorkerSharedState,
methods: &mpsc::Receiver<Method>,
staged: bool,
pending: &PendingSubmissionHandle<St>,
notifications: &NotifyBuffer,
attention: bool,
) {
control.respond(|| {
// Read ingress and durable pending state only after acquiring exclusivity.
// Actor-local staged/deferred/attention flags cannot change during this call.
if shared_state.snapshot().state != WorkerState::Idle
|| staged
|| attention
|| !methods.is_empty()
|| methods.is_closed()
|| !notifications.is_empty()
|| pending.has_unfinished_admission()
{
return Err("Controller has pending, staged, busy, or unavailable work".into());
}
Ok(())
});
}
fn release_fenced_method(method: &Method) -> bool {
!matches!(
method,
Method::Shutdown { .. }
| Method::ListWorkers
| Method::ListPendingSubmissions
| Method::ListCompletions { .. }
)
}
async fn release_requested(shared_state: &WorkerSharedState) {
match shared_state.subject_release.get() {
Some(control) => control.notified().await,
None => std::future::pending::<()>().await,
}
}
fn command_admission_disposition(
admission: WorkerCommandAdmission,
) -> Result<(), WorkerCommandDisposition> {
@@ -802,6 +883,10 @@ impl WorkerController {
C: LlmClient + Clone + 'static,
St: Store + WorkerMetadataStore + Clone + Send + Sync + 'static,
{
worker
.validate_subjektiv_body_publication()
.map_err(std::io::Error::other)?;
// === 1. Initialization (channels / RuntimeDir / worker-immutable
// snapshots / SpawnedWorkerRegistry / alerter attach /
// bash-output scope) ===
@@ -911,13 +996,13 @@ impl WorkerController {
// the actual registered set instead of a hand-maintained mirror.
worker.engine().tool_server_handle().flush_pending();
// Runtime-owned Workers persist their current model-visible head before
// Fresh Workers persist their current model-visible head before
// controller exposure. Feature installation above must happen first so a
// fresh subject Worker captures the current resident surface in its durable
// prompt. Restored Workers defer generic Feature restore notification until
// the next model-visible context materialization; only an installed
// Feature may then request a typed durable history append.
if runtime_managed && worker.needs_initial_session_head_materialization() {
if worker.needs_initial_session_head_materialization() {
worker
.materialize_durable_session_head()
.await
@@ -958,6 +1043,13 @@ impl WorkerController {
let artifact_store: Arc<dyn Store> = Arc::new(worker.store().clone());
let session_id = worker.session_id();
let pending_activations = worker.pending_activation_state();
let release_control =
crate::subject_release::ReleaseControl::new(worker.mutation_authority());
shared_state
.subject_release
.set(release_control)
.ok()
.expect("one Controller release authority");
let handle = WorkerHandle {
method_tx,
working_event_tx: working_event_tx.clone(),
@@ -1439,7 +1531,19 @@ where
let worker_enabled = feature_config.worker.enabled;
let sub_worker_enabled = feature_config.sub_worker.enabled;
let mut feature_registry = FeatureRegistryBuilder::new();
let journal_grant = backend_job
.as_ref()
.filter(|binding| binding.subjektiv_journal)
.map(|binding| {
crate::subjektiv::BackendJournalJobHost::new(worker.workspace_client_handle(), binding)
.map(|host| host.into_grant())
})
.transpose()?;
let (mut feature_registry, journal_finalizer) = if let Some(grant) = journal_grant {
(grant.features, grant.finalizer)
} else {
(FeatureRegistryBuilder::new(), None)
};
feature_registry
.add_module(crate::feature::builtin::chat_invocation::AttachmentInvocationFeature);
if let Some(binding) = backend_job.as_ref() {
@@ -1461,6 +1565,13 @@ where
)?;
result_feature = result_feature.with_surface(surface);
}
if let Some(finalizer) = journal_finalizer {
result_feature = result_feature.with_finalizer(
finalizer,
spawner_manifest.clone(),
worker.llm_client_handle(),
)?;
}
feature_registry.add_module(result_feature);
}
let memory_profile = &worker.manifest().feature.memory.profile;
@@ -1542,6 +1653,27 @@ where
{
feature_registry.add_module(subjektiv_sessions);
}
if ordinary_subjektiv_features_enabled
&& lifecycle_enabled
&& let Some(host) = subjektiv_host.as_ref()
&& host.host.journal_startup_granted()
{
worker.install_initial_context_source(Arc::new(
crate::feature::builtin::journal::JournalStartupContextSource::new(
host.clone(),
worker.prompts(),
),
));
}
if ordinary_subjektiv_features_enabled
&& lifecycle_enabled
&& let Some(host) = subjektiv_host.as_ref()
&& host.host.journal_recording_granted()
{
feature_registry.add_module(crate::subjektiv::JournalRecordingFeature::new(
host.clone(),
)?);
}
if let Some(memory_lifecycle) =
crate::feature::builtin::memory_lifecycle::MemoryLifecycleFeature::from_resolved_config(
lifecycle_enabled,
@@ -2204,6 +2336,12 @@ async fn controller_loop<C, St>(
C: LlmClient + Clone + 'static,
St: Store + WorkerMetadataStore + Clone + 'static,
{
let release_control = shared_state
.subject_release
.get()
.expect("Controller release authority installed")
.clone();
let _release_lifetime = crate::subject_release::ControllerLifetime(release_control.clone());
// Hold an optional external attach server alive for the controller lifetime.
// In-process runtimes retain and drive the WorkerHandle directly.
let _socket_server = socket_server;
@@ -2424,6 +2562,13 @@ async fn controller_loop<C, St>(
break 'controller;
}
},
_ = release_control.notified() => {
answer_idle_release(&release_control, &shared_state, &method_rx,
pending.is_some() || !deferred_methods.is_empty(),
&pending_submissions, &notify_buffer,
worker.has_pending_compaction_cleanup() || worker.has_failed_segment_activation());
continue;
}
_ = notify_buffer.notified() => continue,
_ = tokio::time::sleep(remaining) => {
match prepare_pending_run(
@@ -2458,10 +2603,32 @@ async fn controller_loop<C, St>(
break 'controller;
}
},
_ = release_control.notified() => {
answer_idle_release(&release_control, &shared_state, &method_rx,
pending.is_some() || !deferred_methods.is_empty(),
&pending_submissions, &notify_buffer,
worker.has_pending_compaction_cleanup() || worker.has_failed_segment_activation());
continue;
}
_ = notify_buffer.notified() => continue,
}
};
let _method_admission = if release_fenced_method(&method) {
match release_control.authority.enter() {
Ok(lease) => Some(lease),
Err(error) => {
let _ = working_event_tx.send(Event::Error {
code: ErrorCode::InvalidRequest,
message: error.to_string(),
});
continue;
}
}
} else {
None
};
if shared_state.snapshot().state != WorkerState::Idle
&& reject_idle_only_submission(&method, &working_event_tx)
{
@@ -2857,6 +3024,7 @@ async fn controller_loop<C, St>(
let mut compact = Box::pin(worker.manual_compact_with_cancel(cancel_rx));
loop {
tokio::select! {
_ = release_control.notified() => { release_control.reject_busy(); }
result = &mut compact => break result,
method = method_rx.recv() => {
match method {
@@ -3114,6 +3282,7 @@ async fn controller_loop<C, St>(
}
}
release_control.close();
// Close method admission before any fallible child/session cleanup. Existing
// senders then fail instead of accepting work that this execution can no
// longer process.
@@ -3258,6 +3427,9 @@ where
// Running only after processing the commit fence. This makes the
// Running snapshot contract deterministic even for immediate clients.
biased;
_ = release_requested(shared_state) => {
if let Some(control) = shared_state.subject_release.get() { control.reject_busy(); }
}
committed = async {
input_commit
.as_mut()
@@ -3878,6 +4050,10 @@ fn worker_error_code(e: &WorkerError) -> ErrorCode {
}
}
#[cfg(test)]
#[path = "controller_subject_release_tests.rs"]
mod subject_release_tests;
#[cfg(test)]
mod tests {
use super::*;
@@ -4223,7 +4399,7 @@ mod tests {
}
#[test]
fn runtime_session_head_is_materialized_after_feature_installation_before_exposure() {
fn fresh_session_head_is_materialized_after_feature_installation_before_exposure() {
let production = include_str!("controller.rs")
.split_once("#[cfg(test)]\nmod tests")
.map(|(production, _)| production)
@@ -4242,8 +4418,8 @@ mod tests {
.find("let handle = WorkerHandle")
.expect("controller handle exposure");
let materialize_gate = startup[..materialize]
.rfind("if runtime_managed && worker.needs_initial_session_head_materialization() {")
.expect("fresh runtime-managed materialization gate");
.rfind("if worker.needs_initial_session_head_materialization() {")
.expect("fresh materialization gate");
assert!(feature_install < materialize_gate);
assert!(materialize_gate < materialize);
@@ -0,0 +1,474 @@
//! Production Direct bootstrap, Controller admission, pending persistence and FsStore.
//! Only the provider is replaced by an explicitly released async boundary; no live Host,
//! Journal policy, socket or model continuation semantics are claimed.
use super::*;
use crate::{
PreparedWorker, WorkerBootstrap, WorkerBootstrapLayout, WorkerFilesystemAuthority,
WorkerManifest, WorkerWorkspaceContext,
};
use session_store::{CombinedStore, FsStore, FsWorkerStore};
use tempfile::TempDir;
#[derive(Clone)]
struct HeldProvider {
started: mpsc::UnboundedSender<()>,
finish: Arc<tokio::sync::Notify>,
}
#[async_trait::async_trait]
impl LlmClient for HeldProvider {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(
&self,
_: agen::llm_client::Request,
) -> Result<
std::pin::Pin<
Box<
dyn futures_util::Stream<
Item = Result<
agen::llm_client::event::Event,
agen::llm_client::ClientError,
>,
> + Send,
>,
>,
agen::llm_client::ClientError,
> {
self.started.send(()).unwrap();
self.finish.notified().await;
Ok(Box::pin(futures_util::stream::iter([Ok(
agen::llm_client::event::Event::Status(agen::llm_client::event::StatusEvent {
status: agen::llm_client::event::ResponseStatus::Completed,
}),
)])))
}
}
async fn prepared(
temp: &TempDir,
client: HeldProvider,
) -> PreparedWorker<Box<dyn LlmClient>, CombinedStore<FsStore, FsWorkerStore>> {
let mut manifest = WorkerManifest::from_toml(&format!("[worker]\nname='release-{}'\n[model]\nscheme='anthropic'\nmodel_id='scripted'\n[engine]\n[scope]\nallow=[]", uuid::Uuid::now_v7())).unwrap();
manifest.compaction = Some(manifest::CompactionConfig {
retained_tokens: 0,
..Default::default()
});
WorkerBootstrap::new(
manifest,
CombinedStore::new(
FsStore::new(temp.path().join("sessions")).unwrap(),
FsWorkerStore::new(temp.path().join("workers")).unwrap(),
),
crate::PromptCatalogSource::builtins_only(),
WorkerWorkspaceContext::no_workspace(),
WorkerFilesystemAuthority::None,
WorkerBootstrapLayout::Direct {
runtime_base: temp.path().join("runtime"),
bash_output_dir: temp.path().join("bash"),
},
WorkerControllerTransport::InProcess,
)
.with_model_client(client)
.prepare()
.await
.unwrap()
}
async fn reserve(handle: &WorkerHandle) -> Result<crate::SubjectReleaseReservation, String> {
tokio::time::timeout(Duration::from_secs(5), handle.reserve_subject_release())
.await
.unwrap()
}
async fn shutdown(controller: crate::BootstrappedWorker) {
controller
.handle
.send(Method::Shutdown {
command: WorkerCommandEnvelope::new(1),
})
.await
.unwrap();
tokio::time::timeout(Duration::from_secs(5), controller.shutdown)
.await
.unwrap()
.unwrap();
controller.controller_task.await.unwrap();
}
fn extension() -> LogEntry {
LogEntry::Extension {
ts: 1,
domain: "release-test".into(),
payload: serde_json::json!({}),
}
}
#[tokio::test]
async fn reservation_fences_ingress_fallback_and_history_drop_recovers_commit_stays_fenced() {
let temp = TempDir::new().unwrap();
let _sandbox =
crate::runtime::worker_allocation::test_util::RuntimeDirSandbox::new(temp.path());
let (started, mut starts) = mpsc::unbounded_channel();
let prepared = prepared(
&temp,
HeldProvider {
started,
finish: Arc::default(),
},
)
.await;
let writer = prepared.worker().log_writer_handle();
let pending = prepared.worker().pending_submission_handle();
let buffer = prepared.worker().notify_buffer_handle();
let capture = prepared.worker().committed_session_capture_handle();
let extensions = prepared.worker().session_extension_handle();
let fallback = durable_parent_notification_target(pending.clone(), buffer.clone());
let controller = prepared.start().await.unwrap();
let before = controller.handle.committed_entries().len();
let expected = capture.capture().unwrap().location();
let guard = reserve(&controller.handle).await.unwrap();
assert!(
controller
.handle
.send(Method::submit_text(
protocol::new_submission_request_id(),
"blocked"
))
.await
.is_err()
);
assert!(
controller
.handle
.send(Method::Notify {
notification_request_id: protocol::new_submission_request_id(),
message: "blocked".into()
})
.await
.is_err()
);
assert!(
pending
.accept_notification("blocked".into(), "blocked".into())
.is_err()
);
fallback.notify("child-session".into(), "must not be admitted".into());
assert!(buffer.is_empty());
assert!(!pending.has_unfinished_admission());
assert!(writer.append_entry(extension()).is_err());
assert!(
extensions
.append_if_current(&expected, "release-test", serde_json::json!({}))
.is_err()
);
assert_eq!(controller.handle.committed_entries().len(), before);
assert!(starts.try_recv().is_err());
drop(guard);
writer.append_entry(extension()).unwrap();
let mut guard = reserve(&controller.handle).await.unwrap();
guard.commit_disconnect().unwrap();
guard.commit_disconnect().unwrap();
drop(guard);
assert!(writer.append_entry(extension()).is_err());
assert!(
pending
.accept_notification("after-commit".into(), "blocked".into())
.is_err()
);
assert!(
controller
.handle
.send(Method::Notify {
notification_request_id: protocol::new_submission_request_id(),
message: "blocked".into()
})
.await
.is_err()
);
assert!(reserve(&controller.handle).await.is_err());
let _ = controller.handle.snapshot_event();
controller
.handle
.send(Method::ListPendingSubmissions)
.await
.unwrap();
let handle = controller.handle.clone();
shutdown(controller).await;
assert!(handle.protocol_is_closed());
assert!(reserve(&handle).await.is_err());
}
#[tokio::test]
async fn accepted_input_cannot_be_overtaken_by_release_and_running_request_is_rejected() {
let temp = TempDir::new().unwrap();
let _sandbox =
crate::runtime::worker_allocation::test_util::RuntimeDirSandbox::new(temp.path());
let (started, mut starts) = mpsc::unbounded_channel();
let finish = Arc::new(tokio::sync::Notify::new());
let controller = prepared(
&temp,
HeldProvider {
started,
finish: finish.clone(),
},
)
.await
.start()
.await
.unwrap();
let mut events = controller.handle.subscribe();
// Enqueue succeeds before submitting the separate internal release request.
// Either the queue or the held provider is observed; neither is quiescence.
controller
.handle
.send(Method::submit_text(
protocol::new_submission_request_id(),
"accepted first",
))
.await
.unwrap();
assert!(reserve(&controller.handle).await.is_err());
tokio::time::timeout(Duration::from_secs(5), starts.recv())
.await
.unwrap()
.unwrap();
assert!(reserve(&controller.handle).await.is_err());
finish.notify_one();
tokio::time::timeout(Duration::from_secs(5), async {
loop { if matches!(events.recv().await.unwrap(), Event::WorkerState { snapshot } if snapshot.state == WorkerState::Idle) { break; } }
}).await.unwrap();
let guard = reserve(&controller.handle).await.unwrap();
drop(guard);
shutdown(controller).await;
}
#[tokio::test]
async fn staged_notification_fallback_blocks_release_until_the_actual_turn_finishes() {
let temp = TempDir::new().unwrap();
let _sandbox =
crate::runtime::worker_allocation::test_util::RuntimeDirSandbox::new(temp.path());
let (started, mut starts) = mpsc::unbounded_channel();
let finish = Arc::new(tokio::sync::Notify::new());
let prepared = prepared(
&temp,
HeldProvider {
started,
finish: finish.clone(),
},
)
.await;
let pending = prepared.worker().pending_submission_handle();
let buffer = prepared.worker().notify_buffer_handle();
let fallback = durable_parent_notification_target(pending.clone(), buffer);
let controller = prepared.start().await.unwrap();
let mut events = controller.handle.subscribe();
fallback.notify("child".into(), "durable accepted notification".into());
assert!(reserve(&controller.handle).await.is_err());
tokio::time::timeout(Duration::from_secs(5), starts.recv())
.await
.unwrap()
.unwrap();
finish.notify_one();
tokio::time::timeout(Duration::from_secs(5), async {
loop { if matches!(events.recv().await.unwrap(), Event::WorkerState { snapshot } if snapshot.state == WorkerState::Idle) { break; } }
}).await.unwrap();
let guard = reserve(&controller.handle).await.unwrap();
drop(guard);
shutdown(controller).await;
}
#[derive(Clone)]
struct HeldRewrite {
started: mpsc::UnboundedSender<()>,
finish: Arc<tokio::sync::Notify>,
}
impl crate::feature::FeatureModule for HeldRewrite {
fn descriptor(&self) -> crate::feature::FeatureDescriptor {
crate::feature::FeatureDescriptor::builtin("held-rewrite", "Held rewrite").with_hook(
crate::feature::HookDeclaration::new(
"hold",
crate::feature::FeatureHookPoint::BeforeSessionRewrite,
),
)
}
fn install(
&self,
context: &mut crate::feature::FeatureInstallContext<'_>,
) -> Result<(), crate::feature::FeatureInstallError> {
context.hooks().add_before_session_rewrite(
"hold",
crate::hook::HookExecutionPolicy::fail_closed(),
self.clone(),
)?;
Ok(())
}
}
#[async_trait::async_trait]
impl crate::hook::Hook<crate::hook::BeforeSessionRewrite> for HeldRewrite {
async fn call(
&self,
context: &crate::hook::BeforeSessionRewriteContext,
) -> Result<crate::hook::BeforeSessionRewriteAction, crate::hook::HookError> {
if context.kind == crate::hook::SessionRewriteKind::Compact {
self.started.send(()).unwrap();
self.finish.notified().await;
return Ok(crate::hook::BeforeSessionRewriteAction::Deny(
"stop before compactor boundary".into(),
));
}
Ok(crate::hook::BeforeSessionRewriteAction::Continue)
}
}
#[tokio::test]
async fn compact_maintenance_rejects_release_until_the_real_rewrite_boundary_exits() {
let temp = TempDir::new().unwrap();
let _sandbox =
crate::runtime::worker_allocation::test_util::RuntimeDirSandbox::new(temp.path());
let (started, mut starts) = mpsc::unbounded_channel();
let finish = Arc::new(tokio::sync::Notify::new());
let mut prepared = prepared(
&temp,
HeldProvider {
started,
finish: finish.clone(),
},
)
.await;
let (rewrite_started, mut rewrites) = mpsc::unbounded_channel();
let rewrite_finish = Arc::new(tokio::sync::Notify::new());
let mut registry = FeatureRegistryBuilder::new();
registry.add_module(HeldRewrite {
started: rewrite_started,
finish: rewrite_finish.clone(),
});
assert!(
!prepared
.worker_mut()
.install_features(registry)
.has_errors()
);
let controller = prepared.start().await.unwrap();
let mut events = controller.handle.subscribe();
controller
.handle
.send(Method::submit_text(
protocol::new_submission_request_id(),
"history to compact ".repeat(128),
))
.await
.unwrap();
tokio::time::timeout(Duration::from_secs(5), starts.recv())
.await
.unwrap()
.unwrap();
finish.notify_one();
tokio::time::timeout(Duration::from_secs(5), async {
loop { if matches!(events.recv().await.unwrap(), Event::WorkerState { snapshot } if snapshot.state == WorkerState::Idle) { break; } }
}).await.unwrap();
controller
.handle
.send(Method::Compact {
command: WorkerCommandEnvelope::new(1),
})
.await
.unwrap();
tokio::time::timeout(Duration::from_secs(5), rewrites.recv())
.await
.unwrap()
.unwrap();
assert!(reserve(&controller.handle).await.is_err());
assert!(matches!(
controller.handle.shared_state.snapshot().state,
WorkerState::Busy(WorkerBusyState::Maintenance(_))
));
rewrite_finish.notify_one();
tokio::time::timeout(Duration::from_secs(5), async {
loop { if matches!(events.recv().await.unwrap(), Event::WorkerState { snapshot } if snapshot.state == WorkerState::Idle) { break; } }
}).await.unwrap();
let guard = reserve(&controller.handle).await.unwrap();
drop(guard);
controller
.handle
.send(Method::Shutdown {
command: WorkerCommandEnvelope::new(2),
})
.await
.unwrap();
tokio::time::timeout(Duration::from_secs(5), controller.shutdown)
.await
.unwrap()
.unwrap();
controller.controller_task.await.unwrap();
}
#[tokio::test]
async fn queued_or_actor_staged_work_cannot_pass_the_exclusive_ack_boundary() {
let temp = TempDir::new().unwrap();
let _sandbox =
crate::runtime::worker_allocation::test_util::RuntimeDirSandbox::new(temp.path());
let (started, _) = mpsc::unbounded_channel();
let mut prepared = prepared(
&temp,
HeldProvider {
started,
finish: Arc::default(),
},
)
.await;
prepared
.worker_mut()
.materialize_durable_session_head()
.await
.unwrap();
let pending = prepared.worker().pending_submission_handle();
let buffer = prepared.worker().notify_buffer_handle();
let shared = WorkerSharedState::new(
"queue-boundary".into(),
prepared.worker().segment_id(),
String::new(),
build_greeting(prepared.worker()),
);
let control =
crate::subject_release::ReleaseControl::new(prepared.worker().mutation_authority());
for condition in [
"queued",
"staged",
"attention",
"closed",
"durable-notification",
] {
let (sender, receiver) = mpsc::channel(1);
if condition == "queued" {
sender
.send(Method::submit_text(
protocol::new_submission_request_id(),
"already enqueued",
))
.await
.unwrap();
}
if condition == "durable-notification" {
pending
.accept_notification("already-admitted".into(), "pending".into())
.unwrap();
}
if condition == "closed" {
drop(sender);
}
let request = tokio::spawn({
let control = control.clone();
async move { control.request().await }
});
control.notified().await;
answer_idle_release(
&control,
&shared,
&receiver,
condition == "staged",
&pending,
&buffer,
condition == "attention",
);
assert!(request.await.unwrap().is_err(), "{condition}");
// A failed request must not leave admission fenced for recovery.
assert!(prepared.worker().mutation_authority().enter().is_ok());
}
}
+4
View File
@@ -1532,6 +1532,8 @@ mod tests {
let mut writer = JsonLineWriter::new(writer_half);
writer
.write(&Event::Alert(Alert {
id: None,
position: None,
level: AlertLevel::Warn,
source: AlertSource::Worker,
message: "connect-time alert".into(),
@@ -1763,6 +1765,8 @@ mod tests {
let mut writer = JsonLineWriter::new(stream);
writer
.write(&Event::Alert(Alert {
id: None,
position: None,
level: AlertLevel::Warn,
source: AlertSource::Worker,
message: "replayed alert".into(),
+1
View File
@@ -8,6 +8,7 @@ pub mod backend_job_result;
pub mod chat_invocation;
pub mod drive;
pub mod flow_transition;
pub mod journal;
pub mod manage_workdir;
pub mod manage_worker;
pub mod memory;
@@ -55,10 +55,18 @@ struct SealedResult {
completed: Value,
}
#[derive(Clone)]
struct ResultFinalizer {
finalizer: Arc<dyn crate::job::JobResultFinalizer>,
manifest: manifest::WorkerManifest,
client: Box<dyn agen::llm_client::LlmClient>,
}
#[derive(Clone)]
struct SubmitResultTool {
client: Arc<dyn WorkspaceClient>,
surface: Option<SubjektivSurfaceLifecycleFeature>,
finalizer: Option<ResultFinalizer>,
submission: Arc<tokio::sync::Mutex<ResultSubmissionState>>,
active: Arc<Mutex<HashMap<String, BackgroundTaskCancellation>>>,
}
@@ -80,8 +88,7 @@ impl Drop for ActiveResultGuard {
fn cancelled_output() -> ToolError {
ToolError::Cancelled(ToolOutput {
summary: "Job result submission cancelled; committed candidate decisions are preserved"
.into(),
summary: "Job result submission cancelled; saved domain operations are preserved".into(),
content: None,
attachments: Vec::new(),
})
@@ -157,6 +164,17 @@ impl Tool for SubmitResultTool {
submission.reusable_surface = Some(outcome.clone());
object.insert("surface".into(), outcome);
}
if let Some(finalizer) = &self.finalizer {
result = finalizer
.finalizer
.finalize(
result,
finalizer.manifest.clone(),
finalizer.client.clone_boxed(),
cancellation.clone(),
)
.await?;
}
result
};
if cancellation.is_cancelled() {
@@ -238,6 +256,7 @@ impl Tool for SubmitResultTool {
fn definition(
client: Arc<dyn WorkspaceClient>,
surface: Option<SubjektivSurfaceLifecycleFeature>,
finalizer: Option<ResultFinalizer>,
) -> ToolDefinition {
let submission = Arc::new(tokio::sync::Mutex::new(ResultSubmissionState::default()));
let active = Arc::new(Mutex::new(HashMap::new()));
@@ -255,6 +274,7 @@ fn definition(
Arc::new(SubmitResultTool {
client: client.clone(),
surface: surface.clone(),
finalizer: finalizer.clone(),
submission: submission.clone(),
active: active.clone(),
}) as Arc<dyn Tool>,
@@ -266,6 +286,7 @@ fn definition(
pub struct BackendJobResultFeature {
client: Arc<dyn WorkspaceClient>,
surface: Option<SubjektivSurfaceLifecycleFeature>,
finalizer: Option<ResultFinalizer>,
}
impl BackendJobResultFeature {
@@ -273,11 +294,28 @@ impl BackendJobResultFeature {
Self {
client,
surface: None,
finalizer: None,
}
}
}
impl BackendJobResultFeature {
pub(crate) fn with_finalizer(
mut self,
finalizer: Arc<dyn crate::job::JobResultFinalizer>,
manifest: manifest::WorkerManifest,
client: Box<dyn agen::llm_client::LlmClient>,
) -> std::io::Result<Self> {
finalizer
.validate_profile(&manifest)
.map_err(std::io::Error::other)?;
self.finalizer = Some(ResultFinalizer {
finalizer,
manifest,
client,
});
Ok(self)
}
pub(crate) fn with_surface(mut self, surface: SubjektivSurfaceLifecycleFeature) -> Self {
self.surface = Some(surface);
self
@@ -294,7 +332,11 @@ impl FeatureModule for BackendJobResultFeature {
fn install(&self, context: &mut FeatureInstallContext<'_>) -> Result<(), FeatureInstallError> {
context.tools().register(ToolContribution::new(
TOOL_NAME,
definition(self.client.clone(), self.surface.clone()),
definition(
self.client.clone(),
self.surface.clone(),
self.finalizer.clone(),
),
))
}
}
@@ -364,7 +406,7 @@ mod tests {
#[tokio::test]
async fn cancellation_before_execution_never_submits_a_result() {
let client = Arc::new(RecordingClient::default());
let (_, tool) = definition(client.clone(), None)();
let (_, tool) = definition(client.clone(), None, None)();
let context = ToolExecutionContext::new("result-call", "batch", 0);
tool.cancel_execution(&context).await.unwrap();
let outcome = tool.execute(r#"{"result":{"valid":true}}"#, context).await;
@@ -0,0 +1,690 @@
//! Journal admission and attenuated sequential Job tools, independent of Memory extraction.
//!
//! Hooks request Host reconciliation, never infer source ranges from in-memory history.
//! Hosts retain exact operation inputs/receipts and validate live Job authority on every
//! operation. Successful operations are never rolled back by Job finalization.
use crate::feature::{
FeatureDescriptor, FeatureHookPoint, FeatureInstallContext, FeatureInstallError, FeatureModule,
HookDeclaration, ToolContribution, ToolDeclaration,
};
use crate::hook::{
BeforeSessionRewrite, BeforeSessionRewriteAction, BeforeSessionRewriteContext, Hook, HookError,
HookErrorCategory, HookExecutionPolicy, HookPostToolAction, PostToolCall, RunCommitted,
RunCommittedContext, SessionRewriteKind, ToolResultSummary, WorkerRestored,
WorkerRestoredContext,
};
use crate::subjektiv::{SubjektivHostConnection, SubjektivHostError};
use agen::tool::{Tool, ToolDefinition, ToolError, ToolExecutionContext, ToolMeta, ToolOutput};
use async_trait::async_trait;
use serde::Deserialize;
use serde_json::{Value, json};
use std::sync::Arc;
use subjektiv::journal::transport::*;
use subjektiv::journal::{
JournalAction, JournalOperation, MAX_JOURNAL_PAGE, MAX_JOURNAL_READ_BYTES,
MAX_JOURNAL_RESPONSE_BYTES,
};
/// Subjektiv owns startup projection; Core only commits its typed initial items.
pub(crate) struct JournalStartupContextSource {
host: SubjektivHostConnection,
prompts: Arc<arc_swap::ArcSwap<crate::prompt::catalog::PromptCatalog>>,
}
impl JournalStartupContextSource {
pub(crate) fn new(
host: SubjektivHostConnection,
prompts: Arc<arc_swap::ArcSwap<crate::prompt::catalog::PromptCatalog>>,
) -> Self {
Self { host, prompts }
}
}
#[async_trait]
impl crate::worker::InitialContextSource for JournalStartupContextSource {
async fn load(&self) -> Result<Vec<session_store::SystemItem>, String> {
use sha2::{Digest, Sha256};
let snapshot = self
.host
.journal_startup_context()
.map_err(|e| e.to_string())?;
// Hash the same canonical JSON value retained by SystemItem, so replay
// can verify provenance without depending on domain struct field order.
let snapshot = serde_json::to_value(snapshot).map_err(|e| e.to_string())?;
let snapshot_json = serde_json::to_string(&snapshot).map_err(|e| e.to_string())?;
if snapshot_json.len() > subjektiv::journal::startup::STARTUP_MAX_BYTES {
return Err("Host Journal startup snapshot exceeds independent byte budget".into());
}
let prompts = self.prompts.load();
let body = prompts
.subjektiv_journal_startup(&snapshot_json)
.map_err(|e| e.to_string())?;
if body.len() > subjektiv::journal::startup::STARTUP_MAX_BYTES + 4096 {
return Err("Journal startup prompt exceeds rendered byte budget".into());
}
let projection = prompts.projection();
Ok(vec![session_store::SystemItem::JournalStartup {
body,
snapshot,
snapshot_sha256: Sha256::digest(snapshot_json.as_bytes())
.iter()
.map(|byte| format!("{byte:02x}"))
.collect(),
prompt_provenance: session_store::PromptRenderProvenance {
workspace_id: self.host.host.workspace_id(),
source_digest: projection.source_digest.clone(),
projection_digest: projection.catalog_digest.clone(),
logical_name: "internal.subjektiv_journal_startup".into(),
},
}])
}
}
#[derive(Clone)]
pub struct JournalRecordingFeature {
host: SubjektivHostConnection,
}
impl JournalRecordingFeature {
pub fn new(host: SubjektivHostConnection) -> std::io::Result<Self> {
if !host.host.journal_recording_granted() {
return Err(std::io::Error::new(
std::io::ErrorKind::PermissionDenied,
"Journal recording requires an explicit Host grant",
));
}
Ok(Self { host })
}
fn request(&self, reason: JournalRecordingReason) -> Result<(), HookError> {
self.host
.host
.request_journal_recording(&self.host.context, reason)
.map(|_| ())
.map_err(|_| {
HookError::new(
HookErrorCategory::Dependency,
"Journal admission unconfirmed; Host must reconcile saved public history",
)
})
}
}
impl FeatureModule for JournalRecordingFeature {
fn descriptor(&self) -> FeatureDescriptor {
FeatureDescriptor::builtin("journal-recording", "Journal recording admission")
.with_hook(HookDeclaration::new(
"record-task",
FeatureHookPoint::PostToolCall,
))
.with_hook(HookDeclaration::new(
"record-run",
FeatureHookPoint::RunCommitted,
))
.with_hook(HookDeclaration::new(
"record-compact",
FeatureHookPoint::BeforeSessionRewrite,
))
.with_hook(HookDeclaration::new(
"record-restore",
FeatureHookPoint::WorkerRestored,
))
}
fn install(&self, context: &mut FeatureInstallContext<'_>) -> Result<(), FeatureInstallError> {
context.hooks().add_post_tool_call(
"record-task",
HookExecutionPolicy::new(crate::hook::HookFailurePolicy::AttentionRequired, 30_000),
self.clone(),
)?;
context.hooks().add_run_committed(
"record-run",
HookExecutionPolicy::new(crate::hook::HookFailurePolicy::AttentionRequired, 30_000),
self.clone(),
)?;
context.hooks().add_before_session_rewrite(
"record-compact",
HookExecutionPolicy::fail_closed(),
self.clone(),
)?;
context.hooks().add_worker_restored(
"record-restore",
HookExecutionPolicy::new(crate::hook::HookFailurePolicy::AttentionRequired, 30_000),
self.clone(),
)?;
Ok(())
}
}
#[async_trait]
impl Hook<PostToolCall> for JournalRecordingFeature {
async fn call(&self, result: &ToolResultSummary) -> Result<HookPostToolAction, HookError> {
if result.tool_name == "TaskUpdate"
&& !result.is_error
&& result.arguments.get("status").and_then(Value::as_str) == Some("completed")
{
self.request(JournalRecordingReason::TaskCompleted)?;
}
Ok(HookPostToolAction::Continue)
}
}
#[async_trait]
impl Hook<RunCommitted> for JournalRecordingFeature {
async fn call(&self, _: &RunCommittedContext) -> Result<(), HookError> {
self.request(JournalRecordingReason::RunCommitted)
}
}
#[async_trait]
impl Hook<BeforeSessionRewrite> for JournalRecordingFeature {
async fn call(
&self,
context: &BeforeSessionRewriteContext,
) -> Result<BeforeSessionRewriteAction, HookError> {
if context.kind == SessionRewriteKind::Compact {
// Only admission/retention is awaited, not Journal generation.
self.request(JournalRecordingReason::BeforeCompact)?;
}
Ok(BeforeSessionRewriteAction::Continue)
}
}
#[async_trait]
impl Hook<WorkerRestored> for JournalRecordingFeature {
async fn call(&self, _: &WorkerRestoredContext) -> Result<(), HookError> {
self.request(JournalRecordingReason::WorkerRestored)
}
}
/// A distinct Host capability, bound to one Journal/generation and frozen public input.
/// Profile names and main-Worker recording grants cannot construct this authority.
/// Implementations must preserve exact request_id/full input and reject stale leases.
pub trait JournalJobHost: Send + Sync + 'static {
fn execute(&self, request: JournalJobRequest)
-> Result<JournalJobResponse, SubjektivHostError>;
/// Verify saved checkpoints and remaining input; model prose is not completion evidence.
fn finish(&self) -> Result<JournalJobCompletion, SubjektivHostError>;
}
#[derive(Clone)]
pub struct JournalJobToolsFeature {
host: Arc<dyn JournalJobHost>,
serial: Arc<tokio::sync::Mutex<()>>,
}
impl JournalJobToolsFeature {
pub fn new(host: Arc<dyn JournalJobHost>) -> Self {
Self {
host,
serial: Arc::default(),
}
}
}
const TOOLS: &[(&str, &str)] = &[
(
"CreateJournalEntry",
"Save a new open or fixed event. Existing open must first be fixed; fixed needs source_end. Retain exact request_id/full input on retry; carry returned state_token into the next new operation.",
),
(
"RewriteJournalEntry",
"Rewrite only an open entry's title/body/ongoing. Never rewrite fixed history. Retain exact request_id/full input on retry.",
),
(
"FixJournalEntry",
"Fix an open event with an allowed original source endpoint. Task/Job/turn/compaction completion is NOT evidence that an event ended.",
),
(
"UpdateJournalPosition",
"Save a short current-position memo. This alone is never evidence for checkpointing source bytes.",
),
(
"ConfirmJournalUnchanged",
"Save an explicit checked no-entry-change conclusion for actually read source bytes. This receipt can support a checkpoint.",
),
(
"CheckpointJournal",
"Advance only the contiguous actually reflected source prefix backed by saved record/no-change request IDs. Never infer coverage from final prose or position updates.",
),
(
"ReadJournalState",
"Read bounded initial/current Journal metadata and immutable selected_ranges descriptors, not source bodies or proof of delivery. Fetch every selected Source range with ReadJournalText before any record, position, no-change or checkpoint mutation.",
),
(
"QueryJournalOperation",
"Reconcile a saved request_id after unknown outcome before any new create. Absence is not evidence of an execution failure.",
),
(
"ReadJournalOperationCoverage",
"Read a bounded page of exact saved source ranges for a request_id. Reconcile old receipts against selected bytes before reusing them in a replacement Job checkpoint; a metadata count alone is not coverage evidence.",
),
(
"ListJournalOperations",
"Page saved operation receipts for recovery without replaying record creation.",
),
(
"ReadJournalText",
"Read a bounded UTF-8 chunk of Host-bound initial/current position/open text or source text addressed by a frozen Source descriptor. Job input contains no raw source bodies: fetch every selected Source byte range, continuing with next_byte and content_token, before any mutation. Only successful Source delivery counts; failed fetches and previews do not. Do not mix later input into this generation.",
),
];
impl FeatureModule for JournalJobToolsFeature {
fn descriptor(&self) -> FeatureDescriptor {
TOOLS.iter().fold(
FeatureDescriptor::builtin("journal-job-tools", "Journal Job tools")
.with_description("Descriptor-only Journal Jobs fetch immutable selected Source bytes through bounded tools before sequential mutations; selection metadata is not delivery evidence."),
|descriptor, (name, description)| {
descriptor.with_tool(ToolDeclaration::new(*name, *description))
},
)
}
fn install(&self, context: &mut FeatureInstallContext<'_>) -> Result<(), FeatureInstallError> {
for &(name, description) in TOOLS {
let state = self.clone();
let definition: ToolDefinition = Arc::new(move || {
(
ToolMeta::new(name)
.description(description)
.input_schema(tool_schema(name)),
Arc::new(JournalTool {
name,
state: state.clone(),
}) as Arc<dyn Tool>,
)
});
context
.tools()
.register(ToolContribution::new(name, definition))?;
}
Ok(())
}
}
struct JournalTool {
name: &'static str,
state: JournalJobToolsFeature,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Empty {}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Query {
request_id: String,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Coverage {
request_id: String,
offset: usize,
limit: usize,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct List {
after: Option<String>,
limit: usize,
}
fn parse<T: serde::de::DeserializeOwned>(input: &str) -> Result<T, ToolError> {
serde_json::from_str(input).map_err(|error| ToolError::InvalidArgument(error.to_string()))
}
#[async_trait]
impl Tool for JournalTool {
async fn execute(&self, input: &str, _: ToolExecutionContext) -> Result<ToolOutput, ToolError> {
let request = match self.name {
"ReadJournalState" => {
let _: Empty = parse(input)?;
JournalJobRequest::ReadState
}
"QueryJournalOperation" => {
let query: Query = parse(input)?;
JournalJobRequest::QueryOperation {
request_id: query.request_id,
}
}
"ReadJournalOperationCoverage" => {
let coverage: Coverage = parse(input)?;
if !(1..=MAX_JOURNAL_PAGE).contains(&coverage.limit) {
return Err(ToolError::InvalidArgument(
"limit outside Journal page budget".into(),
));
}
JournalJobRequest::ReadOperationCoverage {
request_id: coverage.request_id,
offset: coverage.offset,
limit: coverage.limit,
}
}
"ListJournalOperations" => {
let list: List = parse(input)?;
if !(1..=MAX_JOURNAL_PAGE).contains(&list.limit) {
return Err(ToolError::InvalidArgument(
"limit outside Journal page budget".into(),
));
}
JournalJobRequest::ListOperations {
after: list.after,
limit: list.limit,
}
}
"ReadJournalText" => {
let read: JournalTextRead = parse(input)?;
// A selected Source range may begin at a nonzero byte. Only the
// bound Host knows that first-chunk offset and enforces its token
// condition; position/open reads always start at zero.
if !(1..=MAX_JOURNAL_READ_BYTES).contains(&read.limit)
|| (read.offset > 0
&& read.content_token.is_none()
&& !matches!(&read.target, JournalTextTarget::Source { .. }))
{
return Err(ToolError::InvalidArgument(
"invalid bounded text read or missing continuation content_token".into(),
));
}
JournalJobRequest::ReadText { read }
}
_ => {
let operation: JournalOperation = parse(input)?;
let correct = matches!(
(self.name, &operation.action),
("CreateJournalEntry", JournalAction::Create { .. })
| ("RewriteJournalEntry", JournalAction::Rewrite { .. })
| ("FixJournalEntry", JournalAction::Fix { .. })
| (
"UpdateJournalPosition",
JournalAction::UpdatePosition { .. }
)
| ("ConfirmJournalUnchanged", JournalAction::ConfirmUnchanged)
| ("CheckpointJournal", JournalAction::Checkpoint { .. })
);
if !correct {
return Err(ToolError::InvalidArgument(
"action does not match Journal tool".into(),
));
}
JournalJobRequest::Execute { operation }
}
};
let _serial = self.state.serial.lock().await;
let response = self.state.host.execute(request)?;
let content = serde_json::to_string(&response)
.map_err(|error| ToolError::ExecutionFailed(error.to_string()))?;
if content.len() > MAX_JOURNAL_RESPONSE_BYTES {
return Err(ToolError::ExecutionFailed(
"Journal response exceeds bounded output; reconcile saved operation before retry"
.into(),
));
}
Ok(ToolOutput {
summary: format!("{}: Host response", self.name),
content: Some(content),
attachments: vec![],
})
}
}
/// Checks Host-saved state at the existing Job result boundary, not a whole-Job write.
pub struct JournalJobFinalizer {
host: Arc<dyn JournalJobHost>,
}
impl JournalJobFinalizer {
pub fn new(host: Arc<dyn JournalJobHost>) -> Self {
Self { host }
}
}
#[async_trait]
impl crate::job::JobResultFinalizer for JournalJobFinalizer {
fn validate_profile(&self, manifest: &manifest::WorkerManifest) -> Result<(), String> {
// Host-selected source bytes must reach the model intact, not be silently
// truncated by the generic Engine output limiter after domain delivery.
if TOOLS.iter().any(|(name, _)| {
manifest.engine.tool_output.limit_for(name) < MAX_JOURNAL_RESPONSE_BYTES
}) {
return Err(
"Journal tool output limits must preserve the complete bounded Host response"
.into(),
);
}
Ok(())
}
async fn finalize(
&self,
result: Value,
_: manifest::WorkerManifest,
_: Box<dyn agen::llm_client::LlmClient>,
cancellation: crate::feature::background::BackgroundTaskCancellation,
) -> Result<Value, ToolError> {
if cancellation.is_cancelled() {
return Err(ToolError::ExecutionFailed("Journal Job cancelled".into()));
}
if result != json!({}) {
return Err(ToolError::InvalidArgument(
"Submit empty result {}; Journal changes must already be saved by sequential tools"
.into(),
));
}
let completion = self.host.finish()?;
serde_json::to_value(completion)
.map_err(|error| ToolError::ExecutionFailed(error.to_string()))
}
}
fn object(properties: Value, required: &[&str]) -> Value {
json!({"type":"object", "properties":properties, "required":required, "additionalProperties":false})
}
fn tool_schema(name: &str) -> Value {
let string = json!({"type":"string"});
let pointer = object(
json!({"segment_id":{"type":"string","format":"uuid"},"entry_ref":string}),
&["segment_id", "entry_ref"],
);
let range = object(
json!({"source":object(json!({"session_id":string,"pointer":pointer,"order":{"type":"integer","minimum":0},"content_sha256":string,"total_bytes":{"type":"integer","minimum":1}}), &["session_id","pointer","order","content_sha256","total_bytes"]),"bytes":object(json!({"start":{"type":"integer","minimum":0},"end":{"type":"integer","minimum":1}}), &["start","end"])}),
&["source", "bytes"],
);
match name {
"ReadJournalState" => object(json!({}), &[]),
"QueryJournalOperation" => object(json!({"request_id":string}), &["request_id"]),
"ReadJournalOperationCoverage" => object(
json!({"request_id":string,"offset":{"type":"integer","minimum":0},"limit":{"type":"integer","minimum":1,"maximum":MAX_JOURNAL_PAGE}}),
&["request_id", "offset", "limit"],
),
"ListJournalOperations" => object(
json!({"after":{"type":["string","null"]},"limit":{"type":"integer","minimum":1,"maximum":MAX_JOURNAL_PAGE}}),
&["limit"],
),
"ReadJournalText" => {
let snapshot = json!({"type":"string","enum":["initial","current"]});
let targets = ["position", "open_body", "open_ongoing"].map(|target| {
object(
json!({"target":{"const":target},"snapshot":snapshot}),
&["target", "snapshot"],
)
});
object(
json!({"target":{"oneOf":[targets[0],targets[1],targets[2],object(json!({"target":{"const":"source"},"pointer":pointer}), &["target","pointer"])]},"offset":{"type":"integer","minimum":0},"limit":{"type":"integer","minimum":1,"maximum":MAX_JOURNAL_READ_BYTES},"content_token":{"type":["string","null"]}}),
&["target", "offset", "limit"],
)
}
_ => {
let strings = json!({"type":"array","items":{"type":"string"}});
let action = match name {
"CreateJournalEntry" => object(
json!({"action":{"const":"create"},"title":string,"body":string,"ongoing":string,"state":{"enum":["open","fixed"]},"source_end":{"anyOf":[pointer,{"type":"null"}]},"correction_refs":strings}),
&[
"action",
"title",
"body",
"ongoing",
"state",
"correction_refs",
],
),
"RewriteJournalEntry" => object(
json!({"action":{"const":"rewrite"},"entry_id":string,"title":string,"body":string,"ongoing":string}),
&["action", "entry_id", "title", "body", "ongoing"],
),
"FixJournalEntry" => object(
json!({"action":{"const":"fix"},"entry_id":string,"source_end":pointer}),
&["action", "entry_id", "source_end"],
),
"UpdateJournalPosition" => object(
json!({"action":{"const":"update_position"},"current_position":string}),
&["action", "current_position"],
),
"ConfirmJournalUnchanged" => {
object(json!({"action":{"const":"confirm_unchanged"}}), &["action"])
}
"CheckpointJournal" => object(
json!({"action":{"const":"checkpoint"},"record_operation_ids":strings}),
&["action", "record_operation_ids"],
),
_ => unreachable!(),
};
object(
json!({"request_id":string,"expected_state":string,"action":action,"covered_ranges":{"type":"array","items":range,"maxItems":64}}),
&["request_id", "expected_state", "action", "covered_ranges"],
)
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
use subjektiv::journal::JournalReceipt;
#[derive(Default)]
struct Host {
requests: Mutex<Vec<JournalJobRequest>>,
}
impl JournalJobHost for Host {
fn execute(
&self,
request: JournalJobRequest,
) -> Result<JournalJobResponse, SubjektivHostError> {
self.requests.lock().unwrap().push(request.clone());
match request {
JournalJobRequest::Execute { operation } => {
Ok(JournalJobResponse::Saved(JournalReceipt {
request_id: operation.request_id,
generation_id: "generation".into(),
entry_id: None,
state_token: "saved-token".into(),
process_order: 0,
process_byte: 0,
}))
}
_ => Ok(JournalJobResponse::Operation(None)),
}
}
fn finish(&self) -> Result<JournalJobCompletion, SubjektivHostError> {
Ok(JournalJobCompletion {
generation_id: "generation".into(),
state_token: "saved-token".into(),
process_order: 1,
process_byte: 0,
})
}
}
async fn call(
host: Arc<Host>,
name: &'static str,
input: Value,
) -> Result<ToolOutput, ToolError> {
JournalTool {
name,
state: JournalJobToolsFeature::new(host),
}
.execute(
&input.to_string(),
ToolExecutionContext::new("call", "batch", 0),
)
.await
}
#[tokio::test]
async fn operation_retry_keeps_complete_original_input_and_exposes_saved_token() {
let host = Arc::new(Host::default());
let input = json!({"request_id":"stable-id","expected_state":"old-token", "action":{"action":"confirm_unchanged"},"covered_ranges":[]});
for _ in 0..2 {
let output = call(host.clone(), "ConfirmJournalUnchanged", input.clone())
.await
.unwrap();
let output: Value = serde_json::from_str(output.content.as_ref().unwrap()).unwrap();
assert_eq!(output["value"]["state_token"], "saved-token");
}
let requests = host.requests.lock().unwrap();
assert_eq!(requests[0], requests[1]);
let JournalJobRequest::Execute { operation } = &requests[0] else {
panic!("expected operation")
};
assert_eq!(serde_json::to_value(operation).unwrap(), input);
}
#[tokio::test]
async fn first_source_chunk_can_start_at_nonzero_selected_offset_without_a_token() {
let host = Arc::new(Host::default());
let input = json!({
"target":{"target":"source","pointer":{"segment_id":uuid::Uuid::nil(),"entry_ref":"Eselected"}},
"offset":32,"limit":16,"content_token":null
});
let schema = tool_schema("ReadJournalText");
assert!(jsonschema::validator_for(&schema).unwrap().is_valid(&input));
call(host.clone(), "ReadJournalText", input.clone())
.await
.unwrap();
let requests = host.requests.lock().unwrap();
let JournalJobRequest::ReadText { read } = &requests[0] else {
panic!("expected Source read")
};
assert_eq!(serde_json::to_value(read).unwrap(), input);
// Source membership, first offset and later token conditions remain Host checks.
}
#[tokio::test]
async fn model_cannot_select_subject_or_cross_tool_action_or_unbounded_read() {
let host = Arc::new(Host::default());
for (name, input) in [
("ReadJournalState", json!({"subject_id":"foreign"})),
(
"FixJournalEntry",
json!({"request_id":"id","expected_state":"state", "action":{"action":"confirm_unchanged"},"covered_ranges":[]}),
),
(
"ReadJournalText",
json!({"target":{"target":"position","snapshot":"current"},"offset":0,"limit":MAX_JOURNAL_READ_BYTES+1}),
),
(
"ReadJournalText",
json!({"target":{"target":"position","snapshot":"current"},"offset":1,"limit":10}),
),
(
"ReadJournalOperationCoverage",
json!({"request_id":"old-record","offset":0,"limit":MAX_JOURNAL_PAGE+1}),
),
(
"ReadJournalOperationCoverage",
json!({"request_id":"old-record","offset":0,"limit":1,"journal_id":"foreign"}),
),
("ListJournalOperations", json!({"limit":MAX_JOURNAL_PAGE+1})),
] {
assert!(call(host.clone(), name, input).await.is_err(), "{name}");
}
assert!(host.requests.lock().unwrap().is_empty());
}
#[test]
fn truncating_profile_output_budget_is_rejected_before_source_delivery() {
use crate::job::JobResultFinalizer;
let finalizer = JournalJobFinalizer::new(Arc::new(Host::default()));
let mut manifest = manifest::WorkerManifest::from_toml("[worker]\nname='journal-budget'\n[model]\nscheme='anthropic'\nmodel_id='scripted'\n[engine]\n[scope]\nallow=[]").unwrap();
assert!(finalizer.validate_profile(&manifest).is_ok());
manifest
.engine
.tool_output
.per_tool
.insert("ReadJournalText".into(), MAX_JOURNAL_RESPONSE_BYTES - 1);
assert!(finalizer.validate_profile(&manifest).is_err());
}
#[test]
fn model_schemas_compile_and_constrain_action_and_scope() {
for &(name, _) in TOOLS {
let schema = tool_schema(name);
let validator = jsonschema::validator_for(&schema).unwrap();
assert!(
!validator.is_valid(&json!({"subject_id":"foreign"})),
"{name}"
);
}
let schema = tool_schema("CheckpointJournal");
let validator = jsonschema::validator_for(&schema).unwrap();
assert!(validator.is_valid(&json!({"request_id":"checkpoint", "expected_state":"saved-token", "action":{"action":"checkpoint","record_operation_ids":["saved-record"]},"covered_ranges":[]})));
assert!(!validator.is_valid(&json!({"request_id":"checkpoint", "expected_state":"saved-token", "action":{"action":"confirm_unchanged"},"covered_ranges":[]})));
}
}
@@ -1108,7 +1108,7 @@ pub fn description(n: &str) -> Option<&'static str> {
"Open one repository-scoped Merge Request linked to the Ticket; reuse an existing Merge Request for that repository and advance only its selector_from with a normal non-force push.",
),
"CompleteMergeRequest" => Some(
"Record Orchestrator-owned integration for the explicitly addressed Merge Request using current trusted exact-source approval and provider evidence. This records an already-applied integration, not a Ticket conclusion or assignment release; a review-not-required Ticket request does not bypass MR integration guards.",
"Record an exact-source MR integration receipt using current trusted exact-source approval and provider evidence. This optional recording workflow describes an already-applied integration; it does not update a branch, select which Coder changes to adopt, conclude a Ticket, or release an assignment. Do not record partial or adapted adoption as full integration of the reviewed source.",
),
"ReviewMergeRequest" => Some(
"Submit the injected trusted Reviewer capability result for its captured exact source ref; source movement cancels it, while target-only movement does not. The capability authenticates the attempt, not the verdict: independent natural-language judgment and validation remain the Reviewer's responsibility.",
@@ -91,8 +91,9 @@ struct AnswerInput {
#[serde(deny_unknown_fields)]
struct ImplementationResultInput {
ticket: String,
#[serde(flatten)]
request: server_api::TicketImplementationResultRequest,
kind: server_api::TicketImplementationResultKind,
body: String,
merge_request: Option<server_api::MergeRequestReviewSubject>,
}
#[derive(Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
@@ -157,7 +158,7 @@ pub(super) fn description(name: &str) -> &'static str {
"Append continuing input to the observed current waiting/inprogress assignment with assignment_id, expected_last_event_id, stable operation_key and bounded answer body. Query current tracking first. Answers continue the same row without automatic inprogress; done requires explicit new Assign/reissue instead. Backend authenticates/fences source, current assignment and event CAS, then saves input/audit/exact receipt atomically and notifies committed input. Exact retry retains full input/key; stale/cleared/replaced conflicts require rereading, and unknown outcome must not be retried as a fresh operation. No model actor/role claims, new assignment, case closure or Worker start."
}
"TicketImplementationResult" => {
"Save an explicit handoff, result or blocker for the current implementation assignment and its original Orchestrator delegation. Read ShowTicket first for assignment identities. Backend authenticates the implementation Worker and fences both assignments; this is not AssignmentAnswer or a handling update. Optional MR/source identifies retained evidence, never approval. Retry an unknown response using exactly the same operation_key and full input; changed input with the same key is rejected. Result and durable notification source commit together. No case closure, new Assign, Worker stop or integration occurs."
"Report facts as the authenticated active Workspace Worker for this Ticket: an explicit handoff, result or blocker with kind, bounded body and optional retained MR/source evidence. No current or pinned implementation assignment or Orchestrator delegation match is required; do not supply assignment identities or actor claims. The Host owns transport retry identity: retries of the same tool invocation and full input are idempotent; a new invocation is a new report. If the outcome is uncertain, reconcile saved evidence rather than blindly submitting a new invocation. This is not AssignmentAnswer, a handling update or approval. Result and durable notification source commit together. No case closure, new Assign, Worker stop or integration occurs."
}
"TicketImplementationResultQuery" => {
"Read one saved implementation result by Ticket and result_id from a result notification/receipt. Historical evidence is not a new request or approval; reread current delegation and MR before acting."
@@ -196,7 +197,11 @@ struct PublicAssignmentTool {
}
#[async_trait]
impl Tool for PublicAssignmentTool {
async fn execute(&self, input: &str, _: ToolExecutionContext) -> Result<ToolOutput, ToolError> {
async fn execute(
&self,
input: &str,
ctx: ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
self.client
.workspace_id()
.filter(|id| !id.trim().is_empty())
@@ -282,31 +287,34 @@ impl Tool for PublicAssignmentTool {
let input: ImplementationResultInput =
serde_json::from_str(input).map_err(parse)?;
identity("ticket", &input.ticket)?;
for (name, value) in [
("assignment_id", &input.request.assignment_id),
(
"orchestrator_assignment_id",
&input.request.orchestrator_assignment_id,
),
("operation_key", &input.request.operation_key),
] {
identity(name, value)?;
}
if input.request.body.trim().is_empty()
|| input.request.body.len() > 16384
|| input.request.body.contains('\0')
// Retry identity belongs to the Host, never model input. Missing trusted
// context must fail closed rather than minting a replacement identity.
identity("tool call_id", &ctx.call_id)?;
let operation_key = format!(
"ticket-implementation-result:{}:{}",
input.ticket, ctx.call_id
);
identity("operation_key", &operation_key)?;
if input.body.trim().is_empty()
|| input.body.len() > 16384
|| input.body.contains('\0')
{
return Err(ToolError::InvalidArgument(
"Result body must be nonblank, NUL-free and at most 16384 bytes".into(),
));
}
if let Some(mr) = &input.request.merge_request {
if let Some(mr) = &input.merge_request {
identity("merge_request_id", &mr.merge_request_id)?;
identity("subject_ref", &mr.subject_ref)?;
}
WorkspaceServerOperation::TicketImplementationResult {
ticket_id: input.ticket,
request: input.request,
request: server_api::TicketImplementationResultRequest {
operation_key,
kind: input.kind,
body: input.body,
merge_request: input.merge_request,
},
}
}
PublicAssignmentKind::ImplementationResultQuery => {
@@ -59,13 +59,13 @@ fn cases() -> Vec<(PublicAssignmentKind, Value, Value)> {
),
(
PublicAssignmentKind::ImplementationResult,
json!({"ticket":"T-743","assignment_id":"implementation","orchestrator_assignment_id":"delegation","operation_key":"result-1","kind":"result","body":"Research without an MR","merge_request":null}),
json!({"result_id":"result-id","ticket_id":"T-743","created_at":"now","request":{"assignment_id":"implementation","orchestrator_assignment_id":"delegation","operation_key":"result-1","kind":"result","body":"Research without an MR","merge_request":null}}),
json!({"ticket":"T-743","kind":"result","body":"Research without an MR","merge_request":null}),
json!({"result_id":"result-id","ticket_id":"T-743","created_at":"now","request":{"operation_key":"result-1","kind":"result","body":"Research without an MR","merge_request":null}}),
),
(
PublicAssignmentKind::ImplementationResultQuery,
json!({"ticket":"T-743","result_id":"result-id"}),
json!({"result_id":"result-id","ticket_id":"T-743","created_at":"now","request":{"assignment_id":"implementation","orchestrator_assignment_id":"delegation","operation_key":"result-1","kind":"result","body":"Research without an MR","merge_request":null}}),
json!({"result_id":"result-id","ticket_id":"T-743","created_at":"now","request":{"operation_key":"result-1","kind":"result","body":"Research without an MR","merge_request":null}}),
),
(
PublicAssignmentKind::Answer,
@@ -175,6 +175,203 @@ async fn assign_reissue_and_clear_forward_observed_identity_and_exact_retry_with
}
}
}
#[tokio::test]
async fn implementation_result_schema_exposes_only_report_facts_and_rejects_forged_fields() {
let (kind, input, response) = cases().remove(4);
let client = transport(200, response);
let (meta, tool) = definition(client.clone(), kind)();
let properties = meta.input_schema["properties"].as_object().unwrap();
assert_eq!(properties.len(), 4);
for field in ["ticket", "kind", "body", "merge_request"] {
assert!(properties.contains_key(field), "missing {field}");
}
assert_eq!(meta.input_schema["additionalProperties"], false);
let schema = jsonschema::validator_for(&meta.input_schema).unwrap();
let mut minimal = input.clone();
minimal.as_object_mut().unwrap().remove("merge_request");
assert!(schema.is_valid(&minimal));
assert!(serde_json::from_value::<ImplementationResultInput>(minimal).is_ok());
for field in [
"operation_key",
"assignment_id",
"orchestrator_assignment_id",
"result_id",
"call_id",
"request",
"actor",
"runtime_id",
"worker_id",
] {
let mut forged = input.clone();
forged[field] = json!("forged");
assert!(!schema.is_valid(&forged), "schema accepts {field}");
assert!(serde_json::from_value::<ImplementationResultInput>(forged.clone()).is_err());
assert!(matches!(
tool.execute(&forged.to_string(), ToolExecutionContext::direct())
.await,
Err(ToolError::InvalidArgument(_))
));
}
assert!(client.requests.lock().unwrap().is_empty());
let query = PublicAssignmentKind::ImplementationResultQuery.schema();
assert!(query["properties"].get("result_id").is_some());
assert!(
query["required"]
.as_array()
.unwrap()
.contains(&json!("result_id"))
);
}
#[tokio::test]
async fn implementation_result_transport_key_uses_only_trusted_invocation_identity() {
let (kind, input, response) = cases().remove(4);
let client = transport(200, response);
let (_, tool) = definition(client.clone(), kind)();
for ctx in [
ToolExecutionContext::new("call-a", "batch-a", 0),
ToolExecutionContext::new("call-a", "batch-b", 7),
ToolExecutionContext::new("call-b", "batch-a", 0),
] {
tool.execute(&input.to_string(), ctx).await.unwrap();
}
let requests = client.requests.lock().unwrap();
assert_eq!(requests.len(), 3);
assert_eq!(requests[0], requests[1]);
assert_ne!(requests[0], requests[2]);
for (request, call_id) in requests.iter().zip(["call-a", "call-a", "call-b"]) {
assert_eq!(request.method, WorkspaceRequestMethod::Post);
assert!(
request
.path
.ends_with("/tickets/T-743/implementation-results")
);
let body: Value = serde_json::from_str(request.body.as_deref().unwrap()).unwrap();
let mut expected = input.clone();
expected.as_object_mut().unwrap().remove("ticket");
expected["operation_key"] = json!(format!("ticket-implementation-result:T-743:{call_id}"));
assert_eq!(body, expected);
assert!(body.get("assignment_id").is_none());
assert!(body.get("orchestrator_assignment_id").is_none());
}
}
#[tokio::test]
async fn implementation_result_missing_or_invalid_context_fails_before_dispatch() {
let (kind, input, response) = cases().remove(4);
let client = transport(200, response);
let (_, tool) = definition(client.clone(), kind)();
for call_id in [
String::new(),
" \t".into(),
"call\nforged".into(),
"x".repeat(1025),
] {
let error = tool
.execute(
&input.to_string(),
ToolExecutionContext::new(call_id, "batch", 0),
)
.await
.unwrap_err();
assert!(matches!(error, ToolError::InvalidArgument(_)));
assert!(error.to_string().contains("tool call_id"));
}
// A valid call ID must also fit the bounded transport key after namespacing.
assert!(matches!(
tool.execute(
&input.to_string(),
ToolExecutionContext::new("x".repeat(1024), "batch", 0),
)
.await,
Err(ToolError::InvalidArgument(_))
));
assert!(client.requests.lock().unwrap().is_empty());
}
#[tokio::test]
async fn implementation_result_validates_report_body_ticket_kind_and_optional_mr() {
let (kind, input, response) = cases().remove(4);
let client = transport(200, response);
let (meta, tool) = definition(client.clone(), kind)();
let schema = jsonschema::validator_for(&meta.input_schema).unwrap();
let mut invalid_inputs = vec![];
for body in [
String::new(),
" \n".into(),
"x".repeat(16385),
"text\0hidden".into(),
"é".repeat(8193),
] {
let mut invalid = input.clone();
invalid["body"] = json!(body);
invalid_inputs.push(invalid);
}
for value in [String::new(), " \t".into(), "x\n".into(), "x".repeat(1025)] {
let mut invalid = input.clone();
invalid["ticket"] = json!(value);
invalid_inputs.push(invalid);
for field in ["merge_request_id", "subject_ref"] {
let mut invalid = input.clone();
invalid["merge_request"] = json!({"merge_request_id":"MR-1","subject_ref":"source"});
invalid["merge_request"][field] = json!(value);
invalid_inputs.push(invalid);
}
}
for value in [json!("done"), json!(null), json!(42)] {
let mut invalid = input.clone();
invalid["kind"] = value;
assert!(!schema.is_valid(&invalid));
invalid_inputs.push(invalid);
}
for mr in [
json!({"merge_request_id":"MR-1"}),
json!({"merge_request_id":"MR-1","subject_ref":"source","operation_key":"forged"}),
] {
let mut invalid = input.clone();
invalid["merge_request"] = mr;
assert!(!schema.is_valid(&invalid));
invalid_inputs.push(invalid);
}
for invalid in invalid_inputs {
assert!(matches!(
tool.execute(&invalid.to_string(), ToolExecutionContext::direct())
.await,
Err(ToolError::InvalidArgument(_))
));
}
assert!(client.requests.lock().unwrap().is_empty());
for (index, report_kind) in ["handoff", "result", "blocker"].into_iter().enumerate() {
let mut valid = input.clone();
valid["kind"] = json!(report_kind);
valid["body"] = json!("é".repeat(8192));
if index == 0 {
valid.as_object_mut().unwrap().remove("merge_request");
} else if index == 2 {
valid["merge_request"] = json!({"merge_request_id":"MR-1","subject_ref":"source"});
}
assert!(schema.is_valid(&valid));
tool.execute(
&valid.to_string(),
ToolExecutionContext::new(format!("call-{index}"), "batch", index),
)
.await
.unwrap();
}
assert_eq!(client.requests.lock().unwrap().len(), 3);
}
#[test]
fn implementation_result_guidance_keeps_retry_identity_host_owned() {
let description = description("TicketImplementationResult");
assert!(description.contains("authenticated active Workspace Worker"));
assert!(description.contains("Host owns transport retry identity"));
assert!(description.contains("a new invocation is a new report"));
assert!(description.contains("rather than blindly submitting"));
assert!(!description.contains("operation_key"));
assert!(!description.contains("fences both assignments"));
}
#[tokio::test]
async fn continuing_answer_forwards_same_assignment_event_cas_and_saved_receipt() {
let (kind, input, response) = cases().pop().unwrap();
@@ -511,6 +708,105 @@ async fn invalid_answer_body_and_assignment_identities_fail_before_dispatch() {
assert!(client.requests.lock().unwrap().is_empty());
}
#[tokio::test]
async fn implementation_result_wip_and_native_dispatch_preserve_trusted_call_context() {
let (kind, input, response) = cases().remove(4);
let client = transport(200, response);
let (_, direct_tool) = definition(client.clone(), kind)();
direct_tool
.execute(
&input.to_string(),
ToolExecutionContext::new("call-a", "native", 0),
)
.await
.unwrap();
let tools = native_ticket_tools(client.clone(), TicketFeatureAccess::workflow()).unwrap();
let handler = TicketItemWipHandler {
tools: operation_map(
tools
.into_iter()
.filter(|t| t.projection.surface == NativeTicketSurface::Item)
.collect(),
),
permissions: None,
ticket_reference: "T-743".into(),
observations: Arc::new(Mutex::new(TicketObservationState::default())),
};
let args: BTreeMap<_, _> = input
.as_object()
.unwrap()
.iter()
.filter(|(key, _)| key.as_str() != "ticket")
.map(|(key, value)| (key.clone(), json_to_wip(value).unwrap()))
.collect();
for call_id in ["call-a", "call-b"] {
handler
.call(
"implementation_result",
&args,
WipCallContext {
execution: ToolExecutionContext::new(call_id, "wip", 3),
security_context: "worker".into(),
},
)
.await
.unwrap_or_else(|_| panic!("WIP implementation_result rejected {call_id}"));
}
{
let requests = client.requests.lock().unwrap();
assert_eq!(requests.len(), 3);
assert_eq!(requests[0], requests[1]);
assert_ne!(requests[1], requests[2]);
let body: Value = serde_json::from_str(requests[2].body.as_deref().unwrap()).unwrap();
assert_eq!(
body["operation_key"],
"ticket-implementation-result:T-743:call-b"
);
}
assert!(matches!(
handler
.call(
"implementation_result",
&args,
WipCallContext {
execution: ToolExecutionContext::new("", "wip", 0),
security_context: "worker".into(),
},
)
.await,
Err(WipOperationError::Protocol(ProtocolError {
code: ProtocolErrorCode::InvalidArguments,
..
}))
));
for field in [
"operation_key",
"assignment_id",
"orchestrator_assignment_id",
"call_id",
] {
let mut forged = args.clone();
forged.insert(field.into(), WipValue::String("forged".into()));
assert!(matches!(
handler
.call(
"implementation_result",
&forged,
WipCallContext {
execution: ToolExecutionContext::new("call-c", "wip", 0),
security_context: "worker".into(),
},
)
.await,
Err(WipOperationError::Protocol(ProtocolError {
code: ProtocolErrorCode::InvalidArguments,
..
}))
));
}
assert_eq!(client.requests.lock().unwrap().len(), 3);
}
#[tokio::test]
async fn assignment_wip_binds_ticket_and_preserves_normal_permission_and_cas_inputs() {
for (kind, mut input, response) in cases() {
+4 -1
View File
@@ -405,12 +405,15 @@ pub struct ToolCallSummary {
pub arguments: Value,
}
/// Information passed to `PostToolCall` hooks.
/// Information passed to `PostToolCall` hooks after the terminal result's
/// annotation/storage callback succeeds. Failure to persist prevents this callback.
pub struct ToolResultSummary {
/// Provider-assigned tool call id this result corresponds to.
pub call_id: String,
/// Registered tool name.
pub tool_name: String,
/// Original call arguments, read-only and never an authorization grant.
pub arguments: Value,
/// Whether the tool reported an error.
pub is_error: bool,
/// Tool output (`summary` always present, `content` may be `None`).
+142 -14
View File
@@ -1,11 +1,11 @@
use std::{
collections::HashSet,
sync::{Arc, Mutex, MutexGuard},
sync::{Arc, Mutex, MutexGuard, Weak},
};
use protocol::{
CommandEvent, CommandSnapshot, CommandStatus, CommandStream, CommandStreamSlice, Event,
InFlightBlock, InFlightSnapshot, InFlightToolCallState,
AlertPosition, CommandEvent, CommandSnapshot, CommandStatus, CommandStream, CommandStreamSlice,
Event, InFlightBlock, InFlightSnapshot, InFlightToolCallState,
};
use session_store::{LoggedContentPart, LoggedItem};
use tokio::sync::broadcast;
@@ -38,6 +38,7 @@ enum TrackedBlock {
block_id: InFlightBlockId,
text: String,
finished: bool,
alert_positions: Vec<Weak<Mutex<AlertPosition>>>,
},
Thinking {
block_id: InFlightBlockId,
@@ -78,6 +79,7 @@ impl InFlightEvents {
block_id,
text: String::new(),
finished: false,
alert_positions: Vec::new(),
});
block_id
}
@@ -213,10 +215,11 @@ impl InFlightEvents {
pub(crate) fn clear_for_committed_item_then<R>(
&self,
item: &LoggedItem,
entry_id: Option<&str>,
f: impl FnOnce() -> R,
) -> R {
let mut inner = self.lock();
inner.clear_for_committed_item(item);
inner.clear_for_committed_item(item, entry_id);
f()
}
@@ -378,7 +381,7 @@ impl InFlightInner {
}
}
fn clear_for_committed_item(&mut self, item: &LoggedItem) {
fn clear_for_committed_item(&mut self, item: &LoggedItem, entry_id: Option<&str>) {
match item {
LoggedItem::Message { role, content }
if matches!(role, session_store::LoggedRole::Assistant) =>
@@ -391,7 +394,7 @@ impl InFlightInner {
})
.collect::<String>();
if !text.is_empty() {
self.remove_first_text_matching(&text);
self.remove_first_text_matching(&text, entry_id);
}
}
LoggedItem::Reasoning {
@@ -420,13 +423,55 @@ impl InFlightInner {
}
}
pub(crate) fn capture_alert_position(&mut self) -> Option<Arc<Mutex<AlertPosition>>> {
// Prune evicted alerts even when their streamed block has not committed yet.
for block in &mut self.blocks {
if let TrackedBlock::Text {
alert_positions, ..
} = block
{
alert_positions.retain(|position| position.strong_count() > 0);
}
}
for block in self.blocks.iter_mut().rev() {
match block {
TrackedBlock::Text {
block_id,
text,
alert_positions,
..
} if !text.is_empty() => {
let position = Arc::new(Mutex::new(AlertPosition::AfterInFlight {
block_id: block_id.0,
}));
alert_positions.push(Arc::downgrade(&position));
return Some(position);
}
TrackedBlock::ToolCall { id, .. } => {
return Some(Arc::new(Mutex::new(AlertPosition::AfterToolCall {
call_id: id.clone(),
})));
}
// Hidden reasoning and empty text are not conversation rows.
_ => {}
}
}
None
}
fn snapshot(&self) -> InFlightSnapshot {
let (block_ids, blocks) = self
.blocks
.iter()
.filter_map(|block| {
block
.to_snapshot_block()
.map(|snapshot| (block.block_id().0, snapshot))
})
.unzip();
InFlightSnapshot {
blocks: self
.blocks
.iter()
.filter_map(TrackedBlock::to_snapshot_block)
.collect(),
block_ids,
blocks,
commands: self.commands.clone(),
compaction: self.compaction.clone(),
}
@@ -444,11 +489,25 @@ impl InFlightInner {
}
}
fn remove_first_text_matching(&mut self, committed: &str) -> bool {
fn remove_first_text_matching(&mut self, committed: &str, entry_id: Option<&str>) -> bool {
if let Some(index) = self.blocks.iter().position(|block| match block {
TrackedBlock::Text { text, .. } => text == committed,
_ => false,
}) {
if let (
TrackedBlock::Text {
alert_positions, ..
},
Some(entry_id),
) = (&self.blocks[index], entry_id)
{
for position in alert_positions.iter().filter_map(Weak::upgrade) {
*position.lock().expect("alert position mutex poisoned") =
AlertPosition::AfterEntry {
entry_id: entry_id.to_owned(),
};
}
}
self.blocks.remove(index);
true
} else {
@@ -559,6 +618,72 @@ mod tests {
}
}
#[test]
fn alert_watchers_are_pruned_and_removed_with_their_text_block() {
let (tx, _) = broadcast::channel(16);
let in_flight = InFlightEvents::new(tx);
let text = in_flight.start_text_block();
in_flight.text_delta(text, "same".into());
let first = in_flight.snapshot_guard().capture_alert_position().unwrap();
drop(first);
let second = in_flight.snapshot_guard().capture_alert_position().unwrap();
{
let guard = in_flight.snapshot_guard();
let TrackedBlock::Text {
alert_positions, ..
} = &guard.blocks[0]
else {
panic!("expected text")
};
assert_eq!(alert_positions.len(), 1);
}
let later = in_flight.start_text_block();
in_flight.text_delta(later, "same".into());
let third = in_flight.snapshot_guard().capture_alert_position().unwrap();
let item = LoggedItem::Message {
role: session_store::LoggedRole::Assistant,
content: vec![LoggedContentPart::Text {
text: "same".into(),
}],
};
in_flight.clear_for_committed_item_then(&item, Some("first-entry"), || ());
assert_eq!(
*second.lock().unwrap(),
AlertPosition::AfterEntry {
entry_id: "first-entry".into()
}
);
assert_eq!(
*third.lock().unwrap(),
AlertPosition::AfterInFlight { block_id: later.0 }
);
in_flight.clear();
assert!(in_flight.snapshot_guard().blocks.is_empty());
assert_eq!(
*third.lock().unwrap(),
AlertPosition::AfterInFlight { block_id: later.0 }
);
}
#[test]
fn snapshot_ids_are_parallel_to_emitted_blocks_not_filtered_placeholders() {
let (tx, _) = broadcast::channel(16);
let in_flight = InFlightEvents::new(tx);
in_flight.start_text_block();
let hidden = in_flight.thinking_start();
in_flight.thinking_done(hidden, String::new());
let visible = in_flight.start_text_block();
in_flight.text_delta(visible, "shown".into());
let tool = in_flight.tool_call_start("call-1".into(), "Read".into());
let snapshot = snapshot_from_guard(&in_flight.snapshot_guard());
assert_eq!(snapshot.blocks.len(), 2);
assert_eq!(snapshot.block_ids, vec![visible.0, tool.0]);
assert_eq!(
snapshot.block_ids,
snapshot_from_guard(&in_flight.snapshot_guard()).block_ids
);
}
#[test]
fn snapshot_boundary_does_not_duplicate_or_gap_delta_sent_after_subscribe() {
let (working_event_tx, _) = broadcast::channel(16);
@@ -621,7 +746,7 @@ mod tests {
// This mirrors Worker::append_entry ordering: clear in-flight first,
// then publish the finalized AssistantItem. Identity-aware clients
// reconcile this durable commit with the preceding streamed block.
in_flight_for_commit.clear_for_committed_item_then(&assistant_item, || {
in_flight_for_commit.clear_for_committed_item_then(&assistant_item, None, || {
sink_for_commit.publish(assistant_entry);
});
committed_tx.send(()).unwrap();
@@ -670,7 +795,7 @@ mod tests {
entry: crate::session_history::test_logged_history_entry(assistant_item.clone()),
};
in_flight.clear_for_committed_item_then(&assistant_item, || {
in_flight.clear_for_committed_item_then(&assistant_item, None, || {
sink.publish(assistant_entry);
});
@@ -698,6 +823,7 @@ mod tests {
text: "done".into(),
}],
},
None,
|| (),
);
@@ -723,6 +849,7 @@ mod tests {
encrypted_content: Some("opaque".into()),
signature: None,
},
None,
|| (),
);
@@ -746,6 +873,7 @@ mod tests {
encrypted_content: Some("opaque".into()),
signature: None,
},
None,
|| (),
);
+380 -8
View File
@@ -6,7 +6,7 @@
//!
//! Each alert is broadcast on the shared `Event` channel and
//! also appended to an in-memory buffer so that clients connecting
//! after the fact still see everything emitted during the session.
//! after the fact can replay the bounded recent prefix from the same live Worker.
use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
@@ -14,7 +14,9 @@ use std::time::{SystemTime, UNIX_EPOCH};
use tokio::sync::broadcast;
use protocol::{Alert, AlertLevel, AlertSource, Event};
use protocol::{Alert, AlertLevel, AlertPosition, AlertSource, Event};
use crate::{in_flight::InFlightEvents, segment_log_sink::SegmentLogSink};
/// Upper bound on buffered alerts. When exceeded, the oldest
/// entries are discarded so a long-running session cannot leak
@@ -27,9 +29,53 @@ pub struct Alerter {
inner: Arc<Inner>,
}
/// A queued identified Alert is no longer retained by this live Worker.
/// Transports must disconnect/resnapshot instead of forwarding its stale anchor.
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
#[error("alert is no longer retained; reconnect for a fresh snapshot")]
pub struct AlertReplayRequired;
struct Inner {
working_event_tx: broadcast::Sender<Event>,
buffer: Mutex<VecDeque<Alert>>,
buffer: Mutex<VecDeque<BufferedAlert>>,
position_source: Mutex<Option<PositionSource>>,
}
#[derive(Clone)]
struct PositionSource {
sink: SegmentLogSink,
in_flight: Option<InFlightEvents>,
}
#[derive(Clone)]
struct BufferedAlert {
alert: Alert,
position: Option<Arc<Mutex<AlertPosition>>>,
}
impl BufferedAlert {
fn snapshot(&self) -> Alert {
let mut alert = self.alert.clone();
alert.position = self.position.as_ref().map(|position| {
position
.lock()
.expect("alert position mutex poisoned")
.clone()
});
alert
}
}
impl PositionSource {
fn capture(&self) -> Arc<Mutex<AlertPosition>> {
// Same order as commit/snapshot: in-flight -> sink. Never hold the
// alerter buffer lock while acquiring either of these locks.
let mut in_flight = self.in_flight.as_ref().map(InFlightEvents::snapshot_guard);
in_flight
.as_mut()
.and_then(|guard| guard.capture_alert_position())
.unwrap_or_else(|| Arc::new(Mutex::new(self.sink.alert_position())))
}
}
impl Alerter {
@@ -38,10 +84,23 @@ impl Alerter {
inner: Arc::new(Inner {
working_event_tx,
buffer: Mutex::new(VecDeque::with_capacity(MAX_BUFFERED_ALERTS)),
position_source: Mutex::new(None),
}),
}
}
pub(crate) fn attach_position_source(
&self,
sink: SegmentLogSink,
in_flight: Option<InFlightEvents>,
) {
*self
.inner
.position_source
.lock()
.expect("alert source mutex poisoned") = Some(PositionSource { sink, in_flight });
}
/// Record and broadcast an alert.
///
/// The broadcast may have no subscribers (e.g. during Worker
@@ -55,35 +114,88 @@ impl Alerter {
/// exactly once: older ones from the snapshot, newer ones from
/// the freshly-subscribed receiver.
pub fn alert(&self, level: AlertLevel, source: AlertSource, message: String) {
self.alert_at(level, source, message, now_ms());
}
fn alert_at(&self, level: AlertLevel, source: AlertSource, message: String, timestamp_ms: i64) {
let source_handle = self
.inner
.position_source
.lock()
.expect("alert source mutex poisoned")
.clone();
let position = source_handle.map(|source| source.capture());
let alert = Alert {
id: Some(uuid::Uuid::now_v7().to_string()),
position: None,
level,
source,
message,
timestamp_ms: now_ms(),
timestamp_ms,
};
if let Ok(mut buf) = self.inner.buffer.lock() {
if buf.len() >= MAX_BUFFERED_ALERTS {
buf.pop_front();
}
buf.push_back(alert.clone());
let _ = self.inner.working_event_tx.send(Event::Alert(alert));
let buffered = BufferedAlert { alert, position };
let event = Event::Alert(buffered.snapshot());
buf.push_back(buffered);
let _ = self.inner.working_event_tx.send(event);
}
}
/// Refresh a queued Alert immediately before transport forwarding.
///
/// The independent log lane may have already delivered the text commit, so
/// the broadcast's copied in-flight anchor is not authoritative anymore.
/// Missing identified alerts require reconnect/resnapshot; legacy alerts
/// without IDs and non-Alert events pass through unchanged.
pub fn freshen_event(&self, event: Event) -> Result<Event, AlertReplayRequired> {
let Event::Alert(ref alert) = event else {
return Ok(event);
};
let Some(id) = alert.id.as_ref() else {
return Ok(event);
};
let buffer = self
.inner
.buffer
.lock()
.expect("alerter buffer mutex poisoned");
buffer
.iter()
.find(|buffered| buffered.alert.id.as_ref() == Some(id))
.map(|buffered| Event::Alert(buffered.snapshot()))
.ok_or(AlertReplayRequired)
}
/// Subscribe and atomically snapshot the current buffer.
///
/// The returned snapshot contains alerts emitted before
/// this call; the receiver will deliver alerts emitted
/// after. An alert cannot appear in both.
pub fn subscribe_with_snapshot(&self) -> (Vec<Alert>, broadcast::Receiver<Event>) {
let (snapshot, rx) = self.subscribe_with_deferred_snapshot();
(snapshot(), rx)
}
// Resolve positions after the session snapshot has been captured. Otherwise
// a text commit between the two snapshots can leave only its old block ID
// in replay, although that block is already absent from the session snapshot.
pub(crate) fn subscribe_with_deferred_snapshot(
&self,
) -> (impl FnOnce() -> Vec<Alert>, broadcast::Receiver<Event>) {
let buf = self
.inner
.buffer
.lock()
.expect("alerter buffer mutex poisoned");
let rx = self.inner.working_event_tx.subscribe();
let snapshot: Vec<Alert> = buf.iter().cloned().collect();
(snapshot, rx)
let snapshot: Vec<BufferedAlert> = buf.iter().cloned().collect();
(
move || snapshot.iter().map(BufferedAlert::snapshot).collect(),
rx,
)
}
}
@@ -98,6 +210,266 @@ fn now_ms() -> i64 {
mod tests {
use super::*;
fn positioned_alerter() -> (Alerter, SegmentLogSink, InFlightEvents) {
let (tx, _) = broadcast::channel(1024);
let in_flight = InFlightEvents::new(tx.clone());
let sink = SegmentLogSink::new();
let alerter = Alerter::new(tx);
alerter.attach_position_source(sink.clone(), Some(in_flight.clone()));
(alerter, sink, in_flight)
}
#[test]
fn queued_alert_uses_committed_anchor_when_log_lane_was_delivered_first() {
let (alerter, sink, in_flight) = positioned_alerter();
let text = in_flight.start_text_block();
in_flight.text_delta(text, "finished text".into());
in_flight.text_done(text, "finished text".into());
let (_, mut events) = alerter.subscribe_with_snapshot();
let (_, mut log_entries) = sink.subscribe_with_snapshot();
alerter.alert(AlertLevel::Warn, AlertSource::Worker, "queued".into());
let queued = events.try_recv().unwrap();
let Event::Alert(ref original) = queued else {
panic!("expected alert")
};
assert!(matches!(
original.position,
Some(AlertPosition::AfterInFlight { .. })
));
let original_id = original.id.clone();
let item: session_store::LoggedItem = agen::Item::assistant_message("finished text").into();
let entry = crate::session_history::test_logged_history_entry(item.clone());
let entry_id = entry.metadata.entry_id.0.clone();
in_flight.clear_for_committed_item_then(&item, Some(&entry_id), || {
sink.publish(session_store::LogEntry::AnnotatedAssistantItem { ts: 42, entry });
});
let committed =
crate::ipc::protocol_session::live_log_entry_event(log_entries.try_recv().unwrap())
.unwrap();
assert!(
matches!(committed, Event::SessionEntryCommitted { entry } if entry.entry_id == entry_id)
);
let Event::Alert(forwarded) = alerter.freshen_event(queued).unwrap() else {
panic!("expected alert")
};
assert_eq!(forwarded.id, original_id);
assert_eq!(
forwarded.position,
Some(AlertPosition::AfterEntry { entry_id })
);
let replay = alerter.subscribe_with_snapshot().0;
assert_eq!(
serde_json::to_value(forwarded).unwrap(),
serde_json::to_value(&replay[0]).unwrap()
);
}
#[test]
fn evicted_or_unknown_alert_ids_require_replay_but_legacy_events_pass_through() {
let (alerter, _, _) = positioned_alerter();
let (_, mut events) = alerter.subscribe_with_snapshot();
alerter.alert(AlertLevel::Warn, AlertSource::Worker, "old".into());
let queued = events.try_recv().unwrap();
for _ in 0..MAX_BUFFERED_ALERTS {
alerter.alert(AlertLevel::Warn, AlertSource::Worker, "later".into());
}
assert!(matches!(
alerter.freshen_event(queued),
Err(AlertReplayRequired)
));
let unknown = Event::Alert(Alert {
id: Some("not-retained".into()),
position: Some(AlertPosition::AfterInFlight { block_id: 1 }),
level: AlertLevel::Warn,
source: AlertSource::Worker,
message: "unknown".into(),
timestamp_ms: 42,
});
assert!(matches!(
alerter.freshen_event(unknown),
Err(AlertReplayRequired)
));
let legacy = Event::Alert(Alert {
id: None,
position: None,
level: AlertLevel::Warn,
source: AlertSource::Worker,
message: "legacy".into(),
timestamp_ms: 42,
});
let expected = serde_json::to_value(&legacy).unwrap();
assert_eq!(
serde_json::to_value(alerter.freshen_event(legacy).unwrap()).unwrap(),
expected
);
assert!(matches!(
alerter.freshen_event(Event::Shutdown),
Ok(Event::Shutdown)
));
}
#[test]
fn rowless_run_transitions_preserve_visible_alert_anchor_live_and_on_replay() {
use session_store::LogEntry;
let (alerter, sink, _) = positioned_alerter();
let entry = crate::session_history::test_logged_history_entry(
agen::Item::assistant_message("visible"),
);
let expected = Some(AlertPosition::AfterEntry {
entry_id: entry.metadata.entry_id.0.clone(),
});
sink.publish(LogEntry::AnnotatedAssistantItem { ts: 1, entry });
let (_, mut events) = alerter.subscribe_with_snapshot();
let transitions = [
LogEntry::RunCancelled {
ts: 2,
entry_id: None,
},
LogEntry::RunYielded {
ts: 3,
entry_id: None,
reason: protocol::RunYieldReason::Compaction,
active_run_turn_count: 1,
},
LogEntry::RunResumed {
ts: 4,
entry_id: None,
source: protocol::RunResumeSource::Compaction,
active_run_turn_count: 1,
},
];
for mut transition in transitions {
transition.ensure_session_entry_id();
sink.publish(transition);
alerter.alert(
AlertLevel::Warn,
AlertSource::Worker,
"after transition".into(),
);
let Event::Alert(live) = alerter.freshen_event(events.try_recv().unwrap()).unwrap()
else {
panic!("expected alert")
};
assert_eq!(live.position, expected);
}
let replay = alerter.subscribe_with_snapshot().0;
assert_eq!(replay.len(), 3);
assert!(replay.iter().all(|alert| alert.position == expected));
let (entries, _) = sink.subscribe_with_snapshot();
// Rebuilding the sink cache from retained entries must use the same rule.
sink.replace_silent(entries);
alerter.alert(
AlertLevel::Warn,
AlertSource::Worker,
"after cache rebuild".into(),
);
assert_eq!(
alerter.subscribe_with_snapshot().0.last().unwrap().position,
expected
);
}
#[test]
fn identical_alerts_in_same_millisecond_keep_distinct_ids_across_replay() {
let (alerter, _, _) = positioned_alerter();
let (_, mut rx) = alerter.subscribe_with_snapshot();
for _ in 0..2 {
alerter.alert_at(AlertLevel::Warn, AlertSource::Worker, "same".into(), 42);
}
let (snapshot, _) = alerter.subscribe_with_snapshot();
assert_ne!(snapshot[0].id, snapshot[1].id);
for expected in &snapshot {
let Event::Alert(live) = rx.try_recv().unwrap() else {
panic!("expected alert")
};
assert!(live.id.is_some());
assert_eq!(live.id, expected.id);
assert_eq!(live.timestamp_ms, 42);
assert_eq!(live.position, Some(AlertPosition::Start));
assert_eq!(live.position, expected.position);
}
let (replay, _) = alerter.subscribe_with_snapshot();
assert_eq!(
serde_json::to_value(snapshot).unwrap(),
serde_json::to_value(replay).unwrap()
);
}
#[test]
fn tool_call_anchor_survives_commit_and_new_text() {
let (alerter, sink, in_flight) = positioned_alerter();
in_flight.tool_call_start("call-1".into(), "Read".into());
alerter.alert(AlertLevel::Warn, AlertSource::Worker, "tool alert".into());
let item: session_store::LoggedItem = agen::Item::tool_call("call-1", "Read", "{}").into();
let entry = session_store::LogEntry::AnnotatedAssistantItem {
ts: 1,
entry: crate::session_history::test_logged_history_entry(item.clone()),
};
in_flight.clear_for_committed_item_then(&item, Some("entry-tool"), || sink.publish(entry));
alerter.alert(AlertLevel::Warn, AlertSource::Worker, "after commit".into());
let text = in_flight.start_text_block();
in_flight.text_delta(text, "next".into());
let (snapshot, _) = alerter.subscribe_with_snapshot();
for alert in snapshot {
assert_eq!(
alert.position,
Some(AlertPosition::AfterToolCall {
call_id: "call-1".into()
})
);
}
}
#[test]
fn eviction_drops_position_ownership_and_clear_does_not_retarget_old_alerts() {
let (alerter, _, in_flight) = positioned_alerter();
let text = in_flight.start_text_block();
in_flight.text_delta(text, "discarded".into());
alerter.alert(AlertLevel::Warn, AlertSource::Worker, "old".into());
let weak = {
let buffer = alerter.inner.buffer.lock().unwrap();
Arc::downgrade(buffer[0].position.as_ref().unwrap())
};
for _ in 0..MAX_BUFFERED_ALERTS {
alerter.alert(AlertLevel::Warn, AlertSource::Worker, "new".into());
}
assert!(weak.upgrade().is_none());
let (before, _) = alerter.subscribe_with_snapshot();
in_flight.clear();
let next = in_flight.start_text_block();
in_flight.text_delta(next, "discarded".into());
let item = session_store::LoggedItem::Message {
role: session_store::LoggedRole::Assistant,
content: vec![session_store::LoggedContentPart::Text {
text: "discarded".into(),
}],
};
in_flight.clear_for_committed_item_then(&item, Some("unrelated-entry"), || ());
let (after, _) = alerter.subscribe_with_snapshot();
assert_eq!(before.len(), MAX_BUFFERED_ALERTS);
assert_eq!(
serde_json::to_value(before).unwrap(),
serde_json::to_value(after).unwrap()
);
}
#[test]
fn snapshot_does_not_acquire_in_flight_lock_under_buffer_lock() {
let (alerter, _, in_flight) = positioned_alerter();
alerter.alert(AlertLevel::Warn, AlertSource::Worker, "before".into());
let guard = in_flight.snapshot_guard();
let cloned = alerter.clone();
let (tx, rx) = std::sync::mpsc::channel();
let thread = std::thread::spawn(move || {
let (snapshot, _) = cloned.subscribe_with_snapshot();
tx.send(snapshot).unwrap();
});
let result = rx.recv_timeout(std::time::Duration::from_secs(5));
drop(guard);
thread.join().unwrap();
assert_eq!(result.unwrap().len(), 1);
}
#[test]
fn alert_broadcasts_to_existing_subscriber() {
let (tx, _keep) = broadcast::channel::<Event>(8);
+1
View File
@@ -747,6 +747,7 @@ impl Interceptor<SessionHistoryMetadata> for WorkerInterceptor {
let summary = ToolResultSummary {
call_id: info.result.tool_use_id.clone(),
tool_name: info.call.name.clone(),
arguments: info.call.input.clone(),
is_error: info.result.is_error,
output: ToolOutput {
summary: info.result.summary.clone(),
+19
View File
@@ -76,9 +76,16 @@ impl PendingNotify {
pub struct NotifyBuffer {
inner: Arc<Mutex<VecDeque<PendingNotify>>>,
wake: Arc<tokio::sync::Notify>,
admission: Arc<std::sync::OnceLock<Arc<crate::subject_release::MutationAuthority>>>,
}
impl NotifyBuffer {
pub(crate) fn bind_mutation_authority(
&self,
authority: Arc<crate::subject_release::MutationAuthority>,
) {
let _ = self.admission.set(authority);
}
pub fn new() -> Self {
Self::default()
}
@@ -113,6 +120,18 @@ impl NotifyBuffer {
}
fn push_entry(&self, entry: PendingNotify) {
let _admission = match self
.admission
.get()
.map(|authority| authority.enter())
.transpose()
{
Ok(lease) => lease,
Err(error) => {
tracing::warn!(%error, "notification rejected by Subject release fence");
return;
}
};
self.inner
.lock()
.expect("notify buffer poisoned")
+2 -2
View File
@@ -30,12 +30,12 @@ fn subscribe_before_snapshot<Subscription, Snapshot>(
pub fn subscribe_worker_protocol_session(handle: &WorkerHandle) -> WorkerProtocolSessionStreams {
let ((snapshot_event, log_entries), (alert_snapshot, events)) = subscribe_before_snapshot(
|| handle.alerter.subscribe_with_snapshot(),
|| handle.alerter.subscribe_with_deferred_snapshot(),
|| handle.snapshot_event_with_entry_subscription(),
);
WorkerProtocolSessionStreams {
snapshot_event,
alert_snapshot,
alert_snapshot: alert_snapshot(),
log_entries,
events,
}
+5
View File
@@ -154,6 +154,11 @@ async fn handle_connection(stream: tokio::net::UnixStream, handle: WorkerHandle)
event = streams.events.recv() => {
match event {
Ok(event) => {
let Ok(event) = handle.alerter.freshen_event(event) else {
// The queued Alert was evicted; reconnect must rebuild
// the bounded prefix rather than forward a stale anchor.
break;
};
if writer.write_event(&event).await.is_err() {
break;
}
+157
View File
@@ -102,6 +102,163 @@ fn request(profile: &str) -> ::job::JobRequest {
limits: Default::default(),
}
}
#[derive(Default)]
struct JournalHostFixture(Mutex<Vec<subjektiv::journal::JournalOperation>>);
impl crate::subjektiv::JournalJobHost for JournalHostFixture {
fn execute(
&self,
request: subjektiv::journal::transport::JournalJobRequest,
) -> Result<
subjektiv::journal::transport::JournalJobResponse,
crate::subjektiv::SubjektivHostError,
> {
use subjektiv::journal::{JournalReceipt, transport::*};
let JournalJobRequest::Execute { operation } = request else {
panic!("script only writes")
};
let mut saved = self.0.lock().unwrap();
assert_eq!(operation.expected_state, format!("state-{}", saved.len()));
let request_id = operation.request_id.clone();
saved.push(operation);
Ok(JournalJobResponse::Saved(JournalReceipt {
request_id,
generation_id: "generation".into(),
entry_id: None,
state_token: format!("state-{}", saved.len()),
process_order: 0,
process_byte: 0,
}))
}
fn finish(
&self,
) -> Result<
subjektiv::journal::transport::JournalJobCompletion,
crate::subjektiv::SubjektivHostError,
> {
let saved = self.0.lock().unwrap();
assert!(
matches!(saved.last().map(|op| &op.action), Some(subjektiv::journal::JournalAction::Checkpoint { record_operation_ids }) if record_operation_ids == &["record".to_string()])
);
Ok(subjektiv::journal::transport::JournalJobCompletion {
generation_id: "generation".into(),
state_token: format!("state-{}", saved.len()),
process_order: 0,
process_byte: 0,
})
}
}
// Real built-in registry/Profile/Prompt resolution, Job runner, Feature tools and result finalizer.
// Only external model transport and the domain Host are replaced; store rules belong to Subjektiv.
#[tokio::test]
async fn builtin_journal_profile_installs_only_bound_tools_and_finalizes_saved_sequential_progress()
{
use crate::subjektiv::{JournalJobFinalizer, JournalJobToolsFeature};
let root = tempfile::tempdir().unwrap();
let registry = ProfileDiscovery::with_sources(None, None)
.discover()
.unwrap();
let host = Arc::new(JournalHostFixture::default());
let client = ScriptedClient::new(vec![
call(
"ConfirmJournalUnchanged",
json!({"request_id":"record","expected_state":"state-0","action":{"action":"confirm_unchanged"},"covered_ranges":[]}),
),
call(
"UpdateJournalPosition",
json!({"request_id":"position","expected_state":"state-1","action":{"action":"update_position","current_position":"Still in progress; no confirmed outcome."},"covered_ranges":[]}),
),
call(
"CheckpointJournal",
json!({"request_id":"checkpoint","expected_state":"state-2","action":{"action":"checkpoint","record_operation_ids":["record"]},"covered_ranges":[]}),
),
call(
SUBMIT_JOB_RESULT_TOOL,
json!({"result":{"fabricated_aggregate":true}}),
),
call(SUBMIT_JOB_RESULT_TOOL, json!({"result":{}})),
prose(),
]);
// This runner/Feature fixture has no selected sources; descriptor delivery gates
// for nonempty ranges are owned and tested by the Subjektiv service.
let mut journal_request = request("builtin:subjektiv-journal-recording");
journal_request.input = json!({"selected_ranges":[],"origins":[]});
journal_request.instruction = "Read immutable selected_ranges through ReadJournalText Source before recording; descriptors are not source bodies.".into();
let job = prepare_job_from_registry_with_grant(
&journal_request,
root.path(),
Some(Box::new(client.clone())),
&registry,
JobFeatureGrant {
features: FeatureRegistryBuilder::new()
.with_module(JournalJobToolsFeature::new(host.clone())),
satisfied_requirements: vec![],
finalizer: Some(Arc::new(JournalJobFinalizer::new(host.clone()))),
},
)
.unwrap();
assert!(!job.manifest.feature.subjektiv.profile.enabled);
assert!(job.manifest.compaction.is_none());
assert!(job.manifest.scope.allow.is_empty());
assert!(job.manifest.delegation_scope.allow.is_empty());
assert_eq!(
job.manifest.engine.instruction,
"internal.subjektiv_journal_system"
);
let sink = Arc::new(RecordingSink::default());
job.run("attempt".into(), sink.clone(), |_| {})
.await
.unwrap();
assert_eq!(host.0.lock().unwrap().len(), 3);
let accepted = sink.0.lock().unwrap();
assert_eq!(accepted.len(), 1);
assert_eq!(accepted[0].result["state_token"], "state-3");
let sent = client.requests.lock().unwrap();
assert!(!sent.is_empty());
for request in sent.iter() {
let names = request
.tools
.iter()
.map(|tool| tool.name.as_str())
.collect::<Vec<_>>();
for name in [
"CreateJournalEntry",
"RewriteJournalEntry",
"FixJournalEntry",
"ReadJournalText",
SUBMIT_JOB_RESULT_TOOL,
] {
assert!(names.contains(&name), "missing {name}");
}
for forbidden in [
"TaskCreate",
"SubjektivMemoryRemember",
"SubjektivSessionRead",
"Bash",
"Write",
"JournalRecordResult",
] {
assert!(!names.contains(&forbidden), "unexpected {forbidden}");
}
let prompt = request.system_prompt.as_deref().unwrap();
for contract in [
"OutcomeUnknown",
"selected_ranges",
"origins",
"ReadJournalState",
"ReadJournalText",
"Source",
"content_token",
"Conflict",
] {
assert!(
prompt.contains(contract),
"missing Journal input contract {contract}"
);
}
}
}
fn artifact(instruction: &str, tokens: u32, turns: u32) -> Value {
json!({"model":{"scheme":"anthropic", "model_id":"scripted-model", "auth":{"kind":"none"}},
"engine":{"instruction":instruction, "language":"English", "max_tokens":tokens, "max_turns":turns}})
+2
View File
@@ -25,6 +25,8 @@ mod session_capture;
mod session_history;
pub mod shared_state;
mod shutdown_after_idle;
mod subject_release;
pub use subject_release::SubjectReleaseReservation;
pub mod skill;
pub mod spawn;
pub mod subjektiv;
+24 -70
View File
@@ -102,6 +102,7 @@ pub enum WorkerPrompt {
ResidentMemorySummarySection,
ResidentMemoryRestoreSection,
SubjektivResidentContext,
SubjektivJournalStartup,
WorkerOrchestrationGuidanceSection,
SubWorkerSpawnToolDescription,
}
@@ -126,6 +127,7 @@ impl WorkerPrompt {
Self::AgentsMdSection => "internal.agents_md_section",
Self::ResidentMemorySummarySection => "internal.resident_memory_summary_section",
Self::ResidentMemoryRestoreSection => "internal.resident_memory_restore_section",
Self::SubjektivJournalStartup => "internal.subjektiv_journal_startup",
Self::SubjektivResidentContext => "internal.subjektiv_resident_context",
Self::WorkerOrchestrationGuidanceSection => {
"internal.worker_orchestration_guidance_section"
@@ -151,6 +153,7 @@ impl WorkerPrompt {
WorkerPrompt::ResidentMemorySummarySection,
WorkerPrompt::ResidentMemoryRestoreSection,
WorkerPrompt::SubjektivResidentContext,
WorkerPrompt::SubjektivJournalStartup,
WorkerPrompt::WorkerOrchestrationGuidanceSection,
WorkerPrompt::SubWorkerSpawnToolDescription,
];
@@ -540,6 +543,14 @@ impl PromptCatalog {
},
)
}
pub fn subjektiv_journal_startup(&self, snapshot_json: &str) -> Result<String, CatalogError> {
self.render(
WorkerPrompt::SubjektivJournalStartup,
minijinja::context! {
snapshot_json => snapshot_json,
},
)
}
pub fn subjektiv_resident_context(
&self,
behavior_md: &str,
@@ -791,58 +802,17 @@ mod tests {
}
#[test]
fn orchestrator_prompt_uses_normal_branch_integration_before_escalation() {
fn merge_request_prompts_keep_recording_and_reviewer_contracts() {
let catalog = PromptCatalog::builtins_only().unwrap();
let prompt = &catalog.projection.templates["role.orchestrator"];
assert!(prompt.contains("`merge_from`"));
assert!(prompt.contains("`merge_to`"));
assert!(prompt.contains("normal source-control operations"));
assert!(prompt.contains("switch to `merge_to`"));
assert!(prompt.contains("concrete source-control or provider failure"));
assert!(prompt.contains("does not update the branch itself"));
assert!(!prompt.contains("use the Ticket repository `origin` transport"));
}
#[test]
fn merge_request_prompts_use_selector_refs_and_verb_first_operations() {
let catalog = PromptCatalog::builtins_only().unwrap();
let templates = &catalog.projection.templates;
let common = &templates["common.merge_request"];
let coder = &templates["role.coder"];
let orchestrator = &templates["role.orchestrator"];
let reviewer = &templates["role.reviewer"];
let combined = format!("{common}\n{coder}\n{orchestrator}\n{reviewer}");
for operation in [
"OpenMergeRequest",
"ShowMergeRequest",
"ReviewMergeRequest",
"CheckMergeRequestReadiness",
"CompleteMergeRequest",
"TicketCaseUpdate",
] {
assert!(combined.contains(operation), "missing {operation}");
}
for stale_operation in [
"MergeRequestOpen",
"MergeRequestShow",
"MergeRequestReview",
"MergeRequestReadinessCheck",
"MergeRequestComplete",
] {
assert!(
!combined.contains(stale_operation),
"stale operation {stale_operation} remains in prompt authority"
);
}
let common = &catalog.projection.templates["common.merge_request"];
let reviewer = &catalog.projection.templates["role.reviewer"];
assert!(common.contains("selector_from"));
assert!(common.contains("selector_to"));
assert!(common.contains("normal non-force push"));
assert!(common.contains("Moving only the target ref does not invalidate approval"));
assert!(common.contains("take precedence over stale Memory"));
assert!(coder.contains("Never invent an operation to append another source candidate"));
assert!(orchestrator.contains("Target-only movement preserves source approval"));
assert!(reviewer.contains("target-only movement does not invalidate approval"));
assert!(common.contains("target-only movement preserves approval"));
assert!(reviewer.contains("complete linked MR/result snapshot"));
assert!(reviewer.contains("Submit `ReviewMergeRequest` once"));
assert!(reviewer.contains("without modifying implementation files"));
}
#[test]
@@ -1076,28 +1046,12 @@ mod tests {
.unwrap();
assert!(rendered.contains("This policy governs naming only"));
assert!(rendered.contains("Worker cleanup is a separate lifecycle decision"));
assert!(rendered.contains("not an automatic consequence of a Ticket state"));
assert!(rendered.contains("perform one cleanup pass before ending the orchestration turn"));
assert!(rendered.contains("Never predeclare `delete_on_completion`"));
assert!(rendered.contains("call `WorkerStop`"));
assert!(rendered.contains("call `WorkerRemove`"));
assert!(rendered.contains("only then call `WorkdirDelete`"));
assert!(rendered.contains("must have no unfinished Ticket work"));
assert!(rendered.contains("does not require unassignment before cleanup"));
assert!(rendered.contains("`UnfinishedWork` (`unfinished_work` on the wire)"));
assert!(rendered.contains("requires explicit work end or reassignment"));
assert!(rendered.contains("stop the Worker, remove it, then delete the Workdir"));
assert!(rendered.contains("releases attachments but preserves the Workdir"));
assert!(
rendered.contains("Retained terminal responsibility does not require unassignment")
);
assert!(rendered.contains("idle or stopped status does not end unfinished work"));
for stale_policy in [
"no current Ticket assignment",
"unassign it through the available orchestration authority",
"`CurrentAssignment`",
] {
assert!(
!rendered.contains(stale_policy),
"stale cleanup policy: {stale_policy}"
);
}
assert!(!rendered.contains("{% include"));
}
}
+27 -14
View File
@@ -409,7 +409,19 @@ mod tests {
}
#[test]
fn effective_code_instructions_preserve_independent_review_before_merge() {
fn effective_orchestrator_instructions_separate_adoption_from_optional_mr_receipts() {
let (without_mr, without_mr_tools) =
ticket_workflow_fixture("role.orchestrator", MergeRequestFeatureConfig::default());
assert!(without_mr.contains("Select, adapt, or reject Coder changes"));
assert!(
without_mr.contains("Use Tickets and Merge Requests by default, adapting the workflow when the request calls for a different approach.")
);
assert!(
!without_mr_tools
.iter()
.any(|meta| meta.name == "CompleteMergeRequest")
);
let (prompt, tools) = ticket_workflow_fixture(
"role.orchestrator",
MergeRequestFeatureConfig {
@@ -419,16 +431,15 @@ mod tests {
..Default::default()
},
);
assert!(prompt.contains("Trusted Reviewer authority"));
assert!(
prompt.contains(
"Target integration and `CompleteMergeRequest` are Orchestrator authority"
)
);
assert!(
prompt.contains("does not bypass the guarded review requirements for MR integration")
);
assert!(prompt.contains("not a universal Ticket pipeline"));
for policy in [
"Select, adapt, or reject Coder changes",
"validate the combined result",
"Use Tickets and Merge Requests by default, adapting the workflow when the request calls for a different approach.",
"its approval requirement is not a general Git adoption gate",
"Partial or adapted adoption is not full MR integration",
] {
assert!(prompt.contains(policy), "missing {policy}");
}
assert!(!tools.iter().any(|meta| meta.name == "ReviewMergeRequest"));
let complete = tools
.iter()
@@ -453,7 +464,10 @@ mod tests {
for role in ["role.ticket_worker", "role.coder", "role.orchestrator"] {
let (prompt, tools) =
ticket_workflow_fixture(role, MergeRequestFeatureConfig::default());
assert!(prompt.contains("return without conclusion"), "{role}");
assert!(
prompt.contains("return without concluding the Ticket"),
"{role}"
);
assert!(
prompt.contains(
"a turn ending or a result report is not itself a completion decision"
@@ -534,9 +548,8 @@ mod tests {
| "TicketClose"
)));
if role == "role.orchestrator" {
assert!(prompt.contains("observed current `orchestrator_assignment_id`"));
assert!(prompt.contains(
"generic `WorkerSpawn` Ticket starts and existing-Worker assignment"
"`SpawnTicketWorker` using the current `orchestrator_assignment_id`"
));
}
}
+129 -10
View File
@@ -24,6 +24,7 @@
use std::sync::{Arc, Mutex as StdMutex};
use protocol::{AlertPosition, SessionSnapshotEntryData};
use session_store::LogEntry;
use tokio::sync::broadcast;
@@ -46,7 +47,7 @@ pub struct SegmentLogSink {
struct SinkInner {
/// Full session log mirror in commit order. Reset on session swap
/// (compaction / fork) via [`SegmentLogSink::reset_with_initial`].
mirror: StdMutex<Vec<LogEntry>>,
mirror: StdMutex<Mirror>,
/// Broadcast channel for live entry updates. The same `Sender`
/// survives session swaps so existing subscribers keep their
/// receiver — they observe the swap as a freshly broadcast
@@ -54,6 +55,46 @@ struct SinkInner {
session_entry_tx: broadcast::Sender<LogEntry>,
}
struct Mirror {
entries: Vec<LogEntry>,
alert_position: AlertPosition,
}
impl Mirror {
fn new(entries: Vec<LogEntry>) -> Self {
let alert_position = latest_alert_position(&entries).unwrap_or(AlertPosition::Start);
Self {
entries,
alert_position,
}
}
}
// Use the public projection so private reasoning/response boundaries cannot become
// anchors. Results/accounting update existing rows, and non-error Run lifecycle
// transitions have no conversation row of their own.
fn latest_alert_position(entries: &[LogEntry]) -> Option<AlertPosition> {
session_store::public_snapshot::project_current_session_snapshot(entries)
.entries
.into_iter()
.rev()
.find_map(|entry| match entry.data {
SessionSnapshotEntryData::ToolCall { call_id, .. } => {
Some(AlertPosition::AfterToolCall { call_id })
}
SessionSnapshotEntryData::ToolResult { .. }
| SessionSnapshotEntryData::Invoke { .. }
| SessionSnapshotEntryData::Usage { .. }
| SessionSnapshotEntryData::RunStats { .. }
| SessionSnapshotEntryData::RunYielded { .. }
| SessionSnapshotEntryData::RunResumed { .. }
| SessionSnapshotEntryData::RunCancelled => None,
_ => Some(AlertPosition::AfterEntry {
entry_id: entry.entry_id,
}),
})
}
impl SegmentLogSink {
/// Create a fresh sink with an empty mirror. Used before any entry
/// has been written (deferred SegmentStart) or as a placeholder in
@@ -62,7 +103,7 @@ impl SegmentLogSink {
let (session_entry_tx, _) = broadcast::channel(BROADCAST_CAPACITY);
Self {
inner: Arc::new(SinkInner {
mirror: StdMutex::new(Vec::new()),
mirror: StdMutex::new(Mirror::new(Vec::new())),
session_entry_tx,
}),
}
@@ -75,7 +116,7 @@ impl SegmentLogSink {
let (session_entry_tx, _) = broadcast::channel(BROADCAST_CAPACITY);
Self {
inner: Arc::new(SinkInner {
mirror: StdMutex::new(entries),
mirror: StdMutex::new(Mirror::new(entries)),
session_entry_tx,
}),
}
@@ -103,7 +144,10 @@ impl SegmentLogSink {
.mirror
.lock()
.expect("session log mirror mutex poisoned");
mirror.push(entry.clone());
if let Some(position) = latest_alert_position(std::slice::from_ref(&entry)) {
mirror.alert_position = position;
}
mirror.entries.push(entry.clone());
if Self::is_live_relevant(&entry) {
// SendError means there are zero subscribers; harmless. The
// mirror lock is held across `send` so subscribers cannot
@@ -147,8 +191,7 @@ impl SegmentLogSink {
.mirror
.lock()
.expect("session log mirror mutex poisoned");
mirror.clear();
mirror.push(initial.clone());
*mirror = Mirror::new(vec![initial.clone()]);
let _ = self.inner.session_entry_tx.send(initial);
}
@@ -164,7 +207,7 @@ impl SegmentLogSink {
.mirror
.lock()
.expect("session log mirror mutex poisoned");
*mirror = entries;
*mirror = Mirror::new(entries);
if let Some(initial) = first {
let _ = self.inner.session_entry_tx.send(initial);
}
@@ -181,7 +224,7 @@ impl SegmentLogSink {
.mirror
.lock()
.expect("session log mirror mutex poisoned");
*mirror = entries;
*mirror = Mirror::new(entries);
}
/// Truncate the mirror without broadcasting.
@@ -191,7 +234,9 @@ impl SegmentLogSink {
.mirror
.lock()
.expect("session log mirror mutex poisoned");
mirror.truncate(entries_len);
mirror.entries.truncate(entries_len);
mirror.alert_position =
latest_alert_position(&mirror.entries).unwrap_or(AlertPosition::Start);
}
/// Atomically read the current mirror and subscribe to subsequent
@@ -203,17 +248,27 @@ impl SegmentLogSink {
.mirror
.lock()
.expect("session log mirror mutex poisoned");
let snapshot = mirror.clone();
let snapshot = mirror.entries.clone();
let rx = self.inner.session_entry_tx.subscribe();
(snapshot, rx)
}
pub(crate) fn alert_position(&self) -> AlertPosition {
self.inner
.mirror
.lock()
.expect("session log mirror mutex poisoned")
.alert_position
.clone()
}
/// Current entry count. Useful for tests / diagnostics.
pub fn len(&self) -> usize {
self.inner
.mirror
.lock()
.expect("session log mirror mutex poisoned")
.entries
.len()
}
@@ -267,6 +322,70 @@ mod tests {
}
}
#[test]
fn alert_anchor_tracks_visible_rows_not_reasoning_results_or_accounting() {
let sink = SegmentLogSink::new();
assert_eq!(sink.alert_position(), AlertPosition::Start);
sink.publish(session_start());
assert_eq!(sink.alert_position(), AlertPosition::Start);
let user = user_input("hello");
sink.publish(user.clone());
let expected = latest_alert_position(&[user.clone()]).unwrap();
assert_eq!(sink.alert_position(), expected);
sink.publish(turn_end(1));
sink.publish(LogEntry::Invoke {
ts: 2,
trigger: protocol::InvokeKind::UserSend,
});
sink.publish(LogEntry::AnnotatedAssistantItem {
ts: 3,
entry: crate::session_history::test_logged_history_entry(
session_store::LoggedItem::Reasoning {
text: "private".into(),
summary: vec![],
encrypted_content: None,
signature: None,
},
),
});
assert_eq!(sink.alert_position(), expected);
let tool = LogEntry::AnnotatedAssistantItem {
ts: 4,
entry: crate::session_history::test_logged_history_entry(agen::Item::tool_call(
"call-1", "Read", "{}",
)),
};
sink.publish(tool.clone());
assert_eq!(
sink.alert_position(),
AlertPosition::AfterToolCall {
call_id: "call-1".into()
}
);
sink.publish(user_input("newer row"));
let newer = sink.alert_position();
sink.publish(LogEntry::AnnotatedToolResult {
ts: 5,
entry: crate::session_history::test_logged_history_entry(agen::Item::tool_result(
"call-1", "done",
)),
});
assert_eq!(sink.alert_position(), newer);
sink.replace_silent(vec![session_start(), user.clone(), tool]);
assert_eq!(
sink.alert_position(),
AlertPosition::AfterToolCall {
call_id: "call-1".into()
}
);
sink.truncate_silent(2);
assert_eq!(sink.alert_position(), expected);
sink.reset_with_initial(session_start());
assert_eq!(sink.alert_position(), AlertPosition::Start);
sink.reset_with_initial_entries(vec![session_start(), user]);
assert_eq!(sink.alert_position(), expected);
}
#[test]
fn publish_then_subscribe_returns_history_in_snapshot() {
let sink = SegmentLogSink::new();
+2
View File
@@ -50,6 +50,7 @@ pub struct WorkerSharedState {
fs_view: OnceLock<WorkerFsView>,
feature_invocations: RwLock<crate::feature::FeatureInvocationRegistry>,
flow_transition_enabled: AtomicBool,
pub(crate) subject_release: OnceLock<std::sync::Arc<crate::subject_release::ReleaseControl>>,
}
impl WorkerSharedState {
@@ -69,6 +70,7 @@ impl WorkerSharedState {
fs_view: OnceLock::new(),
feature_invocations: RwLock::new(crate::feature::FeatureInvocationRegistry::default()),
flow_transition_enabled: AtomicBool::new(false),
subject_release: OnceLock::new(),
}
}
+1
View File
@@ -88,6 +88,7 @@ mod tests {
ToolResultSummary {
call_id: "tool-1".to_string(),
tool_name: name.to_string(),
arguments: serde_json::json!({}),
is_error,
output: ToolOutput {
summary: "result".to_string(),
+2
View File
@@ -131,6 +131,8 @@ mod tests {
listener,
vec![
Event::Alert(Alert {
id: None,
position: None,
level: AlertLevel::Warn,
source: AlertSource::Worker,
message: "replayed alert".into(),

Some files were not shown because too many files have changed in this diff Show More