107 Commits
Author SHA1 Message Date
Hare b0d8c7b786 chore: merge develop into hare/develop 2026-10-07 18:03:33 +09:00
Hare cfacf04260 fix: prevent subject startup deadlocks and diagnose auth rejection 2026-10-07 17:56:48 +09:00
Hare f086cf3375 feat(profiles): seal builtin import provenance and expose read-only editor imports
Retain the evaluated JSON launch entrypoint, seal observed builtin source bytes and digests as verified provenance, and fail on missing/corrupt dependencies without Runtime builtin reevaluation. Cover real HTTP commits and launch resolution, including worker mode and Workspace Config feature composition.

Expose read-only builtin import completions/diagnostics in the editor and rebuild wasm32 release assets with wasm-bindgen 0.2.117. Document public paths, ordinary composition, limits, resave semantics, and the lazy-cycle incident.

Validation: worker-runtime 344 library + 34 integration tests; workspace-server 609 library + 11 binary tests plus final builtin HTTP/launch tests (2); archive tests (16); native WASM tests (2); real-WASM/web tests (37), editor component tests (10); root cargo check, cargo fmt --all -- --check and git diff --check HEAD pass.
2026-10-07 03:38:49 +09:00
Hare 4343be86c1 feat(config-source): add confined read-only builtin Decodal imports
Share the explicit embedded Profile catalog with native and WASM evaluators. Preserve ordinary Decodal values, isolate builtin relative imports, reject reserved Workspace mutations, and capture the exact loaded builtin closure. Guard lazy import cycles and dependency depth before evaluation recurses.

Validation: config-source 48 tests, builtin-source 2 tests, manifest 185 tests; root cargo check, formatting and diff checks pass. Existing contract version/fingerprint stay unchanged; archive provenance binds builtin content instead of invalidating stored Workspace metadata.
2026-10-07 03:35:59 +09:00
Hare a64707ee69 feat: integrate value-based Workspace Profiles 2026-10-07 03:04:07 +09:00
Hare e24fbabe54 fix: align Profile authoring with compaction and reasoning contracts (T-710)
Resolve MR seq9 supported-compaction-fields and numeric-extraction-reasoning findings. Cover all canonical CompactionConfigPartial settings, token aliases and ratio helpers; accept string effort and signed integer extraction budgets without materializing editor defaults. Audit the remaining Profile field groups against their real serde contracts.

Share an actual-provider value corpus across native, generated WASM, and shared save/projection/verified Runtime archive consumption. Prove inline/import/patch exact values and effective Manifest settings; gate canonical compaction field coverage against serialization of the actual partial type. Keep prior scope union and worker/feature diagnostics fixes intact.

Validation: root cargo check; config-source40; unfiltered server644+11; Runtime340+17+17; API typescript77; Deno39; deno task check0/0; wasm32 release regeneration byte-identical with wasm-bindgen0.2.117/Binaryen132; cargo fmt/diff checks. Logs target/t710-contract-{check,config-source,server,api,runtime}.log. Reviewer seq9 independently confirmed seq3/5/7 resolution and correct main/relative-cwd routing. No target integration/ref mutations.
2026-10-07 02:51:29 +09:00
Hare 61f0469986 fix: preserve string and table Profile scope authoring (T-710)
Resolve MR sequence 5 profile_authoring_rejects_valid_string_scope_intents. Decodal 0.4 has no union constraint: retain an opaque scope range with non-materialized table completion hints so string intents remain valid. Keep shared Backend semantic validation authoritative for scope forms/types/unknown keys without changing persisted values or weakening worker/feature editor diagnostics.

Prove actual-provider native/WASM scalar/table, inline/import/patch, nested intent/deny_write/symlink_policy completion and evaluation invariance. Add real save/projection/Runtime scope equivalence and invalid-scope save non-loss coverage. Regenerated WASM artifact is byte-identical; the changed shape is transported via the fingerprint-bound Backend schema metadata.

Validation: root cargo check; config-source39; value_profiles4; unfiltered server643+11; worker-runtime340+17+17; server-api --features typescript77; wasm32 release/wasm-bindgen0.2.117/Binaryen132 -Oz; Deno39; deno task check0/0; fmt/diff checks. Logs target/t710-scope-{check,server,server-api,runtime}.log. No target/ref integration changes.
2026-10-07 02:25:16 +09:00
Hare 728ea3d915 feat: integrate standalone subjektiv support 2026-10-07 02:04:30 +09:00
Hare 24ae6288a5 feat(subjektiv): share domain and add explicit standalone Subject hosts 2026-10-07 01:52:06 +09:00
Hare 3558e4aae1 fix: diagnose nested partial Profile values in the editor (T-710)
Resolve MR sequence 3 profile_nested_editor_diagnostics_missing with a separate, non-materializing authoring constraint pass. Preserve omission and the persisted evaluation schema, and map nested/imported failures to supplied virtual-source spans. Exercise actual Backend schema in native and regenerated WASM tests for empty/partial values, worker and feature types, unknown fields, imports and composition.

Validation: root cargo check; config-source 38; config-source-wasm native/doc; unfiltered yoi-workspace-server 642 lib + 11 bin; worker-runtime 340 lib + 17 per binary; server-api --features typescript 77; focused Deno 39; deno task check; wasm32 release build, wasm-bindgen 0.2.117 + Binaryen 132 -Oz regeneration; cargo fmt/diff checks. Earlier checker failure did not recur: Reviewer independently passed full source suite and exact original baseline checker filter, recorded in MR sequence 3.
2026-10-07 00:36:16 +09:00
Hare ed3bcf35bd fix(worker): remove Wip prefix from gateway tool names (T-712)
Expose Discover, Inspect, and Call in WIP mode and update schema descriptions and current documentation. Preserve execution, argument schemas, internal types, and ordinary Tools mode. Add a registered-tool discovery/inspection/call regression.

Validation: cargo test -p worker --lib wip::tests:: (43 passed); cargo test -p worker (827 unit + 110 integration passed); root cargo check; cargo fmt --all -- --check; git diff --check HEAD.
2026-10-07 00:31:33 +09:00
Hare 58462d5844 feat: register Workspace Profiles as Decodal values (T-710)
Project the evaluated Workspace Profile value into a revision-bound JSON archive without source-path registration or Decodal re-evaluation. Preserve partial-value omissions and explicit overrides, validate before save, and reject legacy registrations with a documented explicit migration.

Fix array-element schema completion and provide fingerprint-bound authoring shapes independently of materialization defaults. Carry metadata through REST, regenerate TypeScript/OpenAPI/WASM artifacts, and exercise save/projection/Runtime consumption and persisted Manifest restore boundaries.

Validation: root cargo check; cargo test -p config-source; cargo test -p worker-runtime; cargo test -p server-api --features typescript; focused profile_settings/value_profiles/archive tests; 39 web config-source/profile API tests; deno task check; cargo fmt --all -- --check; git diff --check HEAD.

Full workspace-server test: 641 passed, one pre-existing ticket_checker_suppresses_stale_and_clean_results_and_diagnoses_invalid_output assertion failure. The same filter fails at base 3c424fd17d in a separately archived source tree and isolated target directory. Remaining server crate tests pass with only that filter skipped (641 lib + 11 bin); unrelated checker code is unchanged.
2026-10-07 00:14:51 +09:00
Hare 13055d7ee1 feat: execute durable standalone Jobs through Host-owned Internal Workers (T-708) 2026-10-07 00:08:01 +09:00
Hare 5c95859d39 chore: merge develop into hare/develop 2026-10-06 20:45:19 +09:00
Hare 3c424fd17d fix: budget subjektiv batches for compact verified Job results (T-704) 2026-10-06 19:58:11 +09:00
Hare d12cdc9106 feat: caller-selected Backend Jobs and bounded subjektiv consolidation (T-704) 2026-10-06 19:45:37 +09:00
Hare 6f9621ca2e fix: retain Restore recovery intent and retire stale caller conflicts (T-701) 2026-10-06 18:15:31 +09:00
Hare fdce12a47b feat: guarded Worker Restore from shared Web actions (T-701) 2026-10-06 17:45:54 +09:00
Hare 1a6d391f83 refactor: integrate registry-bound worker operations 2026-10-06 14:58:49 +09:00
Hare 849a6b4c1c fix(server): route embedded WorkerRemove through registry-bound reception 2026-10-06 14:50:31 +09:00
Hare 7d8e595270 docs(report): record command output readback limitation 2026-10-06 14:24:23 +09:00
Hare 4892a82a28 refactor(server): unify registry-bound workspace worker operations 2026-10-06 14:24:23 +09:00
Hare 4f38f7d5b0 refactor(worker): share file contracts across Tools and WIP providers
Flatten shared typed Tool arguments and derive WIP parameters and checkout allowlists from core schemas. Workspace settings uses shared read/write/edit processing within its captured-read and canonical tree-CAS commit path. Identical settings edits now reject before read or commit; provider response contracts and unknown-outcome handling stay intact.

T-707 validation: root cargo check; fs-operation (58); tools (74 lib, 39 integration, 1 doctest); worker (802 lib, 109 integration); workdir --features http-client (125); yoi-workspace-server --lib workspace_config (19); cargo fmt --all -- --check; git diff --check HEAD.
2026-10-06 14:18:34 +09:00
Hare 4511d4f8ac refactor(fs-operation): extract provider-neutral text operations
Share typed read/edit/write arguments, Operation schemas, decoding, pure outcomes, byte bounds, replacement policy and numbered rendering. Keep filesystem CAS and descriptor-protected mutation within the provider.

T-707: cargo test -p fs-operation passes (58 tests); overflow, Unicode, limits and local checked-save regressions are covered.
2026-10-06 14:18:34 +09:00
Hare 4ceabef40e docs: require reading test design rules before editing tests 2026-10-06 13:31:37 +09:00
Hare 3029e77c7a fix: remove tests that pin mutable profile defaults 2026-10-06 13:31:37 +09:00
Hare b96bd110c6 refactor(worker-runtime): route by WorkerRef and reconcile owned execution cleanup 2026-10-06 09:13:46 +09:00
Hare 0e2a6f2860 fix(worker): bind protocol lifetime to controller endpoints and join startup observers
T-700 prerequisite: expose real method-channel closure and endpoint identity without introducing WorkerRef routing tokens or catalog liveness mirrors. Delay command observer creation until fallible head/recovery/runtime writes finish; abort and join it on socket startup failure.

Validation: cargo test -p worker (909 tests including integration targets); focused controller startup ordering regression; git diff --check HEAD. Full Runtime/Server validation and MR publication follow after the dependent T-700 implementation is clean.
2026-10-06 08:26:09 +09:00
Hare 2c023ac2be fix(workspace-config): fence user editors from runtime source proofs
Resolve MR finding WC-AUTH-001: reject authored source-tree and Profile editor requests from Runtime/Worker proofs in both scoped and single-Workspace middleware. Keep legitimate UI editing and independent Prompt/Skill/config-bundle consumption contracts. Add signed-router authority-state matrix and authenticated UI/WIP positive regressions.
2026-10-06 07:14:31 +09:00
Hare 0a471242cb feat(workspace-config): project grant-bound logical settings through WIP 2026-10-06 06:56:51 +09:00
Hare 7a96c42130 fix(checkouts): preserve scoped descriptor ignore rules
T-699 rev20, MR 01a10d66-db02-70c2-b152-197aca92d6b7 seq3 native_glob_ignore_regression. Load authorized root/nested ignore sources through FsAccessPolicy descriptor opens, retain only bounded ancestor matchers, preserve source precedence/negation/anchors and prune ignored directories before enumeration. Expanded parity also exposed Grep source-class precedence and ignored-directory rescue errors; use the shared matcher without relaxing Grep's existing ignore-open/Git-marker provider contract. No ambient global/linked-worktree administration pointer traversal or unsafe WalkBuilder ignore loading.

Add 7 fs boundary tests, 4 actual Local/scoped provider parity tests and 1 native WIP search/link regression. Current corrected source passes cargo check --workspace --all-targets; cargo test -p fs-operation -p workdir -p tools -p worker (fs37 Workdir125 Tools72+10+29+1doc Worker780+109integration); cargo test -p worker-runtime -p yoi-workspace-server (Runtime279+17+17 Backend576+11); fmt/diff. Logs target/validation/T-699/rereview-{check,core,hosts}.log. Earlier intermediate parity/WIP assertion failures retained, not passing evidence. No target/ref integration, RO source changes, environment update or completion.
2026-10-06 04:24:35 +09:00
Hare b952e56bc2 test: use required Bash cwd in compatibility regressions 2026-10-06 03:44:03 +09:00
Hare ae19426298 fix: preserve sanitized dangling-symlink read diagnostics 2026-10-06 03:42:06 +09:00
Hare 603a814b80 Merge commit '5670c20a6a50077985168269dfc15a6776c66ba6' into work/T-699-checkout-projection 2026-10-06 03:37:07 +09:00
Hare 149853bb0e feat: project attached checkouts with scoped native AI operations 2026-10-06 03:37:07 +09:00
Hare 5670c20a6a fix: integrate develop and preserve invocation notification recovery 2026-10-05 22:25:32 +09:00
Hare c847c2ffad fix: cancel pending uploads when composer chips are removed 2026-10-05 20:55:17 +09:00
Hare 9812e68b4d fix: synchronize native picker interception in Feature browser tests 2026-10-05 20:14:19 +09:00
Hare 9146331450 fix: echo REST completion nonce and argument context 2026-10-05 20:14:19 +09:00
Hare 1a45542842 fix: preserve invocation recovery across rewind and revalidate replay authority 2026-10-05 20:14:19 +09:00
Hare 3deef49897 fix: reconcile T-697 generated contracts with latest develop 2026-10-05 19:35:04 +09:00
Hare c57e236768 fix: update builtin Sol and Luna model defaults 2026-10-05 19:15:02 +09:00
Hare f5c6b0aeb1 fix: deduplicate console entries and contain worker rendering errors 2026-10-05 19:13:15 +09:00
Hare 31f7847d1f fix: preserve multiple positional invocation arguments and update API schema 2026-10-05 19:07:23 +09:00
Hare fa1169c556 docs(report): record reviewer severity schema mismatch 2026-10-05 19:01:13 +09:00
Hare 6feed0f22f fix(wip): page complete workdir catalogs and fence full connection sets 2026-10-05 19:01:13 +09:00
Hare 6972928612 feat: integrate persistent Subject behavior context 2026-10-05 18:48:12 +09:00
Hare a3c3bda3fa feat: add enlarged previews for console images 2026-10-05 18:45:23 +09:00
Hare f7783b3d2a feat: add declarative Feature chat invocations and typed composers 2026-10-05 18:44:41 +09:00
Hare 0fe61df25d fix: integrate serialized orchestrator attention 2026-10-05 18:10:12 +09:00
Hare 01156f4c78 docs: report SubWorker cwd attachment-root ambiguity 2026-10-05 18:10:01 +09:00
Hare 42e4ce5550 feat: publish Repository and Workdir WIP objects
Use Host root namespace ownership and disjoint Feature operation contributions. Separate read-only workdir_catalog from manage_workdir, encode all existing identities canonically, publish safe bounded inventory and lifetime-bound attachment Objects, and reuse normal lifecycle/session router/lock authority.

Contextual interfaces resolve the current permission/capability intersection; object and interface validators reject stale cached access. Cover fresh workspace WIP-only discovery/create/attach/read/detach, readonly and unsupported targets, collision/revocation/alias reuse, active cleanup exclusion and terminal unknown outcomes without retries.

Validation: WIP 40 tests; worker 730 unit + 96 integration tests serial; manifest 182 tests; workspace-root cargo check; cargo fmt --all -- --check; git diff --check HEAD.
2026-10-05 18:10:01 +09:00
Hare bbfbf92264 feat: bind Workdir attachments to durable connection lifetimes
Add caller-scoped bounded connection inventory and conditional detach on the existing ledger/session lifecycle. Preserve lifetime identity during attenuation and compensation, migrate persisted rows, and bind self operations to verified Runtime/Worker proof.

Validation: workspace-root cargo check; Backend 561 unit + 11 CLI tests; server-api 62 default + 69 typescript tests. Canonical OpenAPI and generated TypeScript source digests regenerated; browser declarations unchanged.
2026-10-05 18:10:01 +09:00
Hare 0a8f1f3b1b fix(workspace-server): drain Restore rechecks after reserved no-op checks
T-703 review follow-up: retire only the exact pre-Restore accepted ID, preserve pending deliveries, and retain an explicit deferred Restore recheck when another caller owns the reservation. Atomically release/take deferred work at completion and drain it without holding a Runtime lock.

Regression fixes the independently identified 3174142b interleaving: pause the real reservation before fingerprint comparison and after a no-op match, accept an actual fixture Restore, and verify exactly one new send after reservation completion. Also verify unknown delivery ID/content survive Restore.

Validation: cargo check; cargo test -p yoi-workspace-server (566 lib +11 bin); cargo test -p worker --lib notification_ (14); cargo fmt --all -- --check; git diff --check HEAD. Previous full Worker serial 703 lib+96 integration remains applicable (Worker unchanged in this fix). Tool schema rejection/Reviewer one-call barrier recorded in docs/report.
2026-10-05 18:01:10 +09:00
Hare 3174142bd1 fix(workspace-server): serialize explicit orchestrator backlog attention
T-703: snapshots establish baselines; only observed Running -> Idle rechecks backlog. Scope reservation and success/pending records to the exact Orchestrator; preserve payload and request ID across bounded retries and Restore races. Drop the launch lock before attention delivery.

Regression evidence: Barrier/parallel start, reconnect snapshots, retry payload identity, Restore/replacement; real Runtime adapter + Worker Controller + canonical Session fixture proves start/Restore send/receipt/commit counts and post-acceptance response-loss dedupe. Worker FsStore regression proves same-ID retry after restore versus legitimate new-ID same-body notification.

Validation: cargo check; cargo test -p yoi-workspace-server (564 lib + 11 bin); cargo test -p worker -- --test-threads=1 (703 lib + 96 integration); cargo fmt --all -- --check; git diff --check HEAD. Default parallel worker test reproduced existing pending_restore_reuses_persisted_session_and_recovers_unknown_attribution ScopeLock NotFound; isolated test and full serial suite pass.
2026-10-05 17:43:11 +09:00
Hare 5e4902db2b Merge latest develop after WIP namespace integration
Normal merge of develop@5a4c823c preserves integrated T-698 controller/native projections alongside the T-692/T-693 reconciliation. Target branch remains untouched.

Validation after this target movement: root cargo check; full worker(serial) and workspace-server tests; cargo fmt --all -- --check; git diff --check HEAD. Earlier reconciliation passed session-store, server-api generated/OpenAPI, full Web and production browser-shell checks.
2026-10-05 17:34:47 +09:00
Hare a4af26e8fd Merge develop into T-693 and reconcile Subject behavior with Memory information
Normal merge of develop@b0bce50b retains T-692 state hierarchy, diagnostics disclosures, ready-empty and candidate-only regressions alongside T-693 behavior/CAS/route fences. ResidentContext load and refresh retain the five-second timeout from 685b8aa6, with a seven-request regression.

Validation: root cargo check; full worker(serial), session-store, workspace-server and server-api(typescript) suites; frontend check with zero diagnostics; 464 Web tests and 171 component tests (19 Memory); production Memory browser-shell test/build; fmt/diff/OpenAPI checks.
2026-10-05 17:31:17 +09:00
Hare 5a4c823c40 feat: integrate host-owned WIP namespaces 2026-10-05 17:17:27 +09:00
Hare d3a1788445 fix: pin WIP cancellation to the active operation provider
Resolve WIP_CANCEL_ROUTING from MR 01a10ac8-c63a-7e33-bc27-07a52f87abe9. Retain the dispatched handler instance and selected operation per active execution; forward cancellation only to that contributor, not unrelated providers or shared-handler aliases. Keep cancellation terminal confirmation and failure classification unchanged.

Validation: contributed_cancellation tests (2 passed), all WIP tests (23 passed), root cargo check, cargo test -p worker -- --test-threads=1 (713 unit and 96 integration passed), cargo fmt --all -- --check, and git diff --check HEAD. Static/dynamic pending-call regressions cover unrelated cancel errors, shared handler duplication, exact dynamic handler identity, unknown IDs, terminal cleanup and cancelled audit.
2026-10-05 17:08:39 +09:00
Hare b0bce50bb4 chore: integrate Subject Memory information improvements 2026-10-05 17:08:32 +09:00
Hare 06480b9c7c Fix Subject behavior editor isolation across reused routes
Reset editor state for each loader identity and bind saves to their captured Subject and Workspace. Fence late success, error, and finally updates using the loader generation, including navigation back to the same Subject.

Validation: 14 Memory component tests (5 new route/workspace/in-flight isolation cases); frontend check with zero diagnostics; production Memory browser shell test; cargo check; cargo fmt --all -- --check; git diff --check HEAD. Restored implementation full Rust and frontend suites were independently rerun before this Web-only fix.
2026-10-05 16:59:13 +09:00
Hare b981e60231 test(web): document Subject Memory information decisions 2026-10-05 16:58:35 +09:00
Hare e17e81a9aa fix: enforce WIP namespace ownership and preserve collection identity
Restrict compatibility mounts to single tool routes, reject foreign dynamic Object owners, and prefer explicit Objects over synthetic namespaces during discovery. Document subsequent provider registration and keep reference resolution independent from enabled management operations.

Validation: cargo test -p worker --lib wip::tests (21 passed); root cargo check passed; cargo test -p worker -- --test-threads=1 passed (711 unit, 96 integration tests); cargo fmt --all -- --check and git diff --check HEAD passed. Initial parallel worker suite hit ScopeLock NotFound in pending_restore_reuses_persisted_session_and_recovers_unknown_attribution; the isolated test and full serial suite both passed.
2026-10-05 16:53:39 +09:00
Hare 8891236db3 Add persistent Subject behavior context 2026-10-05 16:27:23 +09:00
Hare 0c644c13e5 fix: preserve failed reviewer handoff evidence 2026-10-05 15:43:06 +09:00
Hare bc7b2f894c feat: expose WIP objects from host namespaces 2026-10-05 15:33:58 +09:00
Hare a00a0575f6 chore: merge develop into hare/develop 2026-10-05 14:06:04 +09:00
Hare 90c74df879 fix: enable indentation keys in Decodal editor 2026-10-05 14:05:45 +09:00
Hare b489a59057 fix: handle configuration schema bodies and editor load failures 2026-10-05 14:05:45 +09:00
Hare 4cf6c130cd fix(web): distinguish empty resident context 2026-10-05 13:59:39 +09:00
Hare fbb09e6597 feat(web): clarify Subject Memory status 2026-10-05 13:56:33 +09:00
Hare cc97987482 fix: preserve ambiguous legacy session attribution 2026-10-05 12:50:47 +09:00
Hare d2a0b75045 fix: finalize subjektiv attribution at session lifecycle 2026-10-05 12:25:20 +09:00
Hare fec85dd38d fix: simplify console mode controls with a compact footer toggle 2026-10-05 01:58:22 +09:00
Hare 685b8aa65f fix: prevent subject worker launch stalls 2026-10-05 01:22:58 +09:00
Hare bd79b93c1d chore: merge develop into hare/develop 2026-10-04 23:28:46 +09:00
Hare e741643717 fix: accept submission completion state in worker API parser 2026-10-04 23:27:50 +09:00
Hare 74a00f9edc fix: recover from provider image size rejections 2026-10-04 23:17:43 +09:00
Hare c6d2ad1f4c fix(test): format Subject browser evidence 2026-10-04 22:52:37 +09:00
Hare c8caa5518a fix(test): use shared Subject singleton key 2026-10-04 22:42:44 +09:00
Hare 3a9def7b80 Merge remote-tracking branch 'origin/develop' into work/T-689-web-subject-create 2026-10-04 22:35:43 +09:00
Hare 1c5966c535 feat(web): add Subject creation to Memory index 2026-10-04 22:35:21 +09:00
Hare 08881fa27a chore: merge develop into hare/develop 2026-10-04 21:03:16 +09:00
Hare 8a2a82d13c fix: close mobile sidebar after navigation 2026-10-04 21:03:16 +09:00
Hare b0d4973717 fix: align console metadata and details controls 2026-10-04 21:03:16 +09:00
Hare 728f03952e fix: recover job cleanup across runtime restart 2026-10-04 15:03:04 +09:00
Hare c7ae825a02 fix: persist worker run completion fence 2026-10-04 14:08:44 +09:00
Hare 8ad52e36c1 test: update worker state fixtures 2026-10-04 13:49:12 +09:00
Hare d8f3e46126 Merge remote-tracking branch 'origin/develop' into work/T-690-backend-job-worker-cleanup
# Conflicts:
#	web/workspace/src/lib/generated/protocol-validator.ts
2026-10-04 13:44:01 +09:00
Hare 20109b88df fix: fence cleanup on tracked run completion 2026-10-04 13:40:10 +09:00
Hare 773907462b Resolve Console images across session lineage 2026-10-04 13:14:36 +09:00
Hare 1fbf93c8c1 fix: fence job cleanup on run teardown 2026-10-04 13:12:39 +09:00
Hare 2eb9418a4e Separate Console image bodies from session snapshots 2026-10-04 12:56:40 +09:00
Hare d3eb7bc66e feat: clean up terminal backend job workers 2026-10-04 12:45:19 +09:00
Hare f7b722246d feat: configure subject attachment on worker launch 2026-10-04 08:40:30 +09:00
Hare 1bbc48dad0 feat(worker): project merge requests as native WIP objects 2026-10-04 03:10:42 +09:00
Hare 983c20c3f7 fix(worker): preserve Ticket WIP outcome semantics 2026-10-04 02:38:55 +09:00
Hare 31ee6b3c6a feat(worker): project tickets as native WIP objects 2026-10-04 02:22:45 +09:00
Hare 28c9d752e3 fix(worker): preserve Objective WIP authority semantics 2026-10-04 01:49:16 +09:00
Hare e5f255b6fd feat(worker): project objectives as native WIP objects 2026-10-04 01:28:45 +09:00
Hare e2181e9052 fix(worker): preserve typed native WIP contracts 2026-10-04 00:46:51 +09:00
Hare e44637519e fix(worker): forward native WIP arguments by descriptor name 2026-10-04 00:31:49 +09:00
Hare 4a05aad508 feat: add explicit Worker WIP mode 2026-10-04 00:26:15 +09:00
Hare 85a8944a6c fix: integrate feature-owned subjektiv restore refresh 2026-10-03 23:49:38 +09:00
Hare 8399d4f79c fix: integrate sidebar workspace alerts 2026-10-03 23:43:27 +09:00
Hare eea873e7dc fix: keep subjektiv restore refresh feature-owned 2026-10-03 23:37:23 +09:00
Hare bf918d1a11 fix: route sidebar failures through workspace alerts 2026-10-03 23:35:43 +09:00
395 changed files with 111930 additions and 15609 deletions
+2
View File
@@ -19,6 +19,8 @@ LLM に投げる context はappend-onlyが基本であり、またその永続
## 検証
テストを追加・変更する前に、必ず[テスト設計規則](docs/development/rust-testing-strategy.md)を読み、その規則に従うこと。
開発中は、変更した契約を証明する最小の target / filter から実行する。
```sh
Generated
+287 -3
View File
@@ -64,6 +64,7 @@ dependencies = [
"dotenv",
"eventsource-stream",
"futures",
"imagesize",
"reqwest",
"schemars",
"serde",
@@ -88,6 +89,20 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "ahash"
version = "0.8.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
dependencies = [
"cfg-if",
"getrandom 0.3.4",
"once_cell",
"serde",
"version_check",
"zerocopy",
]
[[package]]
name = "aho-corasick"
version = "1.1.4"
@@ -438,7 +453,16 @@ version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0700ddab506f33b20a03b13996eccd309a48e5ff77d0d95926aa0210fb4e95f1"
dependencies = [
"bit-vec",
"bit-vec 0.6.3",
]
[[package]]
name = "bit-set"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3"
dependencies = [
"bit-vec 0.8.0",
]
[[package]]
@@ -447,6 +471,12 @@ version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "349f9b6a179ed607305526ca489b34ad0a41aed5f7980fa90eb03160b69598fb"
[[package]]
name = "bit-vec"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
[[package]]
name = "bitflags"
version = "1.3.2"
@@ -496,6 +526,12 @@ dependencies = [
"cipher",
]
[[package]]
name = "borrow-or-share"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc0b364ead1874514c8c2855ab558056ebfeb775653e7ae45ff72f28f8f3166c"
[[package]]
name = "bstr"
version = "1.12.1"
@@ -507,6 +543,13 @@ dependencies = [
"serde",
]
[[package]]
name = "builtin-source"
version = "0.1.0"
dependencies = [
"sha2 0.11.0",
]
[[package]]
name = "bumpalo"
version = "3.20.2"
@@ -730,6 +773,7 @@ dependencies = [
name = "config-source"
version = "0.1.0"
dependencies = [
"builtin-source",
"decodal",
"decodal-language-service",
"decodal-language-tools",
@@ -1249,6 +1293,15 @@ dependencies = [
"zeroize",
]
[[package]]
name = "email_address"
version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e079f19b08ca6239f47f8ba8509c11cf3ea30095831f7fed61441475edd8c449"
dependencies = [
"serde",
]
[[package]]
name = "encoding_rs"
version = "0.8.35"
@@ -1330,16 +1383,37 @@ version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b95f7c0680e4142284cf8b22c14a476e87d61b004a3a0861872b32ef7ead40a2"
dependencies = [
"bit-set",
"bit-set 0.5.3",
"regex",
]
[[package]]
name = "fancy-regex"
version = "0.16.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "998b056554fbe42e03ae0e152895cd1a7e1002aec800fdc6635d20270260c46f"
dependencies = [
"bit-set 0.8.0",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "fastrand"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be"
[[package]]
name = "feature-storage"
version = "0.1.0"
dependencies = [
"rusqlite",
"tempfile",
"thiserror 2.0.18",
"uuid",
]
[[package]]
name = "ff"
version = "0.13.1"
@@ -1418,6 +1492,17 @@ dependencies = [
"thiserror 2.0.18",
]
[[package]]
name = "fluent-uri"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1918b65d96df47d3591bed19c5cca17e3fa5d0707318e4b5ef2eae01764df7e5"
dependencies = [
"borrow-or-share",
"ref-cast",
"serde",
]
[[package]]
name = "fnv"
version = "1.0.7"
@@ -1460,6 +1545,16 @@ dependencies = [
"percent-encoding",
]
[[package]]
name = "fraction"
version = "0.15.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e076045bb43dac435333ed5f04caf35c7463631d0dae2deb2638d94dd0a5b872"
dependencies = [
"lazy_static",
"num",
]
[[package]]
name = "fs-operation"
version = "0.1.0"
@@ -2112,6 +2207,12 @@ dependencies = [
"winapi-util",
]
[[package]]
name = "imagesize"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09e54e57b4c48b40f7aec75635392b12b3421fa26fe8b4332e63138ed278459c"
[[package]]
name = "include_dir"
version = "0.7.4"
@@ -2256,6 +2357,17 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "job"
version = "0.1.0"
dependencies = [
"manifest",
"serde",
"serde_json",
"sha2 0.11.0",
"thiserror 2.0.18",
]
[[package]]
name = "jobserver"
version = "0.1.34"
@@ -2278,6 +2390,32 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "jsonschema"
version = "0.33.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d46662859bc5f60a145b75f4632fbadc84e829e45df6c5de74cfc8e05acb96b5"
dependencies = [
"ahash",
"base64 0.22.1",
"bytecount",
"email_address",
"fancy-regex 0.16.2",
"fraction",
"idna",
"itoa",
"num-cmp",
"num-traits",
"once_cell",
"percent-encoding",
"referencing",
"regex",
"regex-syntax",
"serde",
"serde_json",
"uuid-simd",
]
[[package]]
name = "kasuari"
version = "0.4.12"
@@ -2468,6 +2606,7 @@ version = "0.1.0"
dependencies = [
"agen",
"arc-swap",
"builtin-source",
"decodal",
"protocol",
"secrets",
@@ -2730,6 +2869,20 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "num"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35bd024e8b2ff75562e5f34e7f4905839deb4b22955ef5e73d2fea1b9813cb23"
dependencies = [
"num-bigint",
"num-complex",
"num-integer",
"num-iter",
"num-rational",
"num-traits",
]
[[package]]
name = "num-bigint"
version = "0.4.8"
@@ -2756,6 +2909,21 @@ dependencies = [
"zeroize",
]
[[package]]
name = "num-cmp"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63335b2e2c34fae2fb0aa2cecfd9f0832a1e24b3b32ecec612c3426d46dc8aaa"
[[package]]
name = "num-complex"
version = "0.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495"
dependencies = [
"num-traits",
]
[[package]]
name = "num-conv"
version = "0.2.1"
@@ -2792,6 +2960,17 @@ dependencies = [
"num-traits",
]
[[package]]
name = "num-rational"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f83d14da390562dca69fc84082e73e548e1ad308d24accdedd2720017cb37824"
dependencies = [
"num-bigint",
"num-integer",
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.19"
@@ -2925,6 +3104,12 @@ dependencies = [
"num-traits",
]
[[package]]
name = "outref"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a80800c0488c3a21695ea981a54918fbb37abf04f4d0720c453632255e2ff0e"
[[package]]
name = "p256"
version = "0.13.2"
@@ -3335,6 +3520,7 @@ dependencies = [
"serde",
"serde_json",
"sha2 0.11.0",
"thiserror 2.0.18",
"tokio",
"ts-rs",
"uuid",
@@ -3628,6 +3814,20 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "referencing"
version = "0.33.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e9c261f7ce75418b3beadfb3f0eb1299fe8eb9640deba45ffa2cb783098697d"
dependencies = [
"ahash",
"fluent-uri",
"once_cell",
"parking_lot",
"percent-encoding",
"serde_json",
]
[[package]]
name = "regex"
version = "1.12.3"
@@ -4453,13 +4653,20 @@ dependencies = [
"agen",
"async-trait",
"client",
"feature-storage",
"fs4",
"futures",
"job",
"libc",
"manifest",
"memory",
"protocol",
"rusqlite",
"serde",
"serde_json",
"server-api",
"session-store",
"subjektiv",
"tempfile",
"thiserror 2.0.18",
"tokio",
@@ -4536,6 +4743,23 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "subjektiv"
version = "0.1.0"
dependencies = [
"chrono",
"feature-storage",
"manifest",
"memory",
"rusqlite",
"serde",
"serde_json",
"server-api",
"tempfile",
"thiserror 2.0.18",
"uuid",
]
[[package]]
name = "subtle"
version = "2.6.1"
@@ -4674,7 +4898,7 @@ dependencies = [
"anyhow",
"base64 0.22.1",
"bitflags 2.11.0",
"fancy-regex",
"fancy-regex 0.11.0",
"filedescriptor",
"finl_unicode",
"fixedbitset",
@@ -5205,6 +5429,7 @@ dependencies = [
"toml",
"unicode-width",
"uuid",
"worker-runtime",
]
[[package]]
@@ -5409,6 +5634,17 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "uuid-simd"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23b082222b4f6619906941c17eb2297fff4c2fb96cb60164170522942a200bd8"
dependencies = [
"outref",
"uuid",
"vsimd",
]
[[package]]
name = "valuable"
version = "0.1.1"
@@ -5427,6 +5663,12 @@ version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "vsimd"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c3082ca00d5a5ef149bb8b555a72ae84c9c59f7250f013ac822ac2e49b19c64"
[[package]]
name = "vtparse"
version = "0.6.2"
@@ -6063,6 +6305,36 @@ dependencies = [
"memchr",
]
[[package]]
name = "wip-client"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73a82416411af69b7eb4ad0ed9a954df062769d3a03b3a1c59b0b4bf617f3534"
dependencies = [
"wip-http",
"wip-protocol",
]
[[package]]
name = "wip-http"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ec0d32d1e56e52fb9ce925db808354a3432b3ebbd112fab8c443c9b865c58c6"
dependencies = [
"base64 0.22.1",
"http",
"serde",
"serde_json",
"url",
"wip-protocol",
]
[[package]]
name = "wip-protocol"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "275ffaebf544a077de7578fd3674c062a098a72c5a904cf66e934f76bd9efbab"
[[package]]
name = "wiremock"
version = "0.6.5"
@@ -6200,6 +6472,8 @@ dependencies = [
"agen",
"arc-swap",
"async-trait",
"axum",
"base64 0.22.1",
"chrono",
"clap",
"client",
@@ -6210,7 +6484,10 @@ dependencies = [
"fs4",
"futures",
"futures-util",
"http",
"include_dir",
"job",
"jsonschema",
"libc",
"manifest",
"mcp",
@@ -6237,6 +6514,9 @@ dependencies = [
"tracing",
"tungstenite 0.28.0",
"uuid",
"wip-client",
"wip-http",
"wip-protocol",
"workdir",
]
@@ -6344,6 +6624,7 @@ dependencies = [
"server-api",
"session-analytics",
"session-store",
"standalone",
"tempfile",
"ticket",
"tokio",
@@ -6388,8 +6669,10 @@ dependencies = [
"axum",
"chrono",
"config-source",
"feature-storage",
"flow",
"futures",
"job",
"manifest",
"memory",
"merge-request",
@@ -6406,6 +6689,7 @@ dependencies = [
"session-store",
"sha2 0.11.0",
"ssh-key",
"subjektiv",
"tempfile",
"thiserror 2.0.18",
"ticket",
+17
View File
@@ -8,6 +8,7 @@ members = [
"crates/api-macros-impl",
"crates/session-store",
"crates/standalone",
"crates/job",
"crates/secrets",
"crates/manifest",
"crates/mcp",
@@ -24,11 +25,14 @@ members = [
"crates/tools",
"crates/fs-operation",
"crates/flow",
"crates/builtin-source",
"crates/config-source",
"crates/config-source-wasm",
"crates/workdir",
"crates/tui",
"crates/memory",
"crates/feature-storage",
"crates/subjektiv",
"crates/ticket",
"crates/merge-request",
"crates/project-record",
@@ -43,6 +47,7 @@ default-members = [
"crates/api-macros-impl",
"crates/session-store",
"crates/standalone",
"crates/job",
"crates/secrets",
"crates/manifest",
"crates/mcp",
@@ -59,11 +64,14 @@ default-members = [
"crates/tools",
"crates/fs-operation",
"crates/flow",
"crates/builtin-source",
"crates/config-source",
"crates/config-source-wasm",
"crates/workdir",
"crates/tui",
"crates/memory",
"crates/feature-storage",
"crates/subjektiv",
"crates/ticket",
"crates/merge-request",
"crates/project-record",
@@ -73,6 +81,7 @@ default-members = [
[workspace.package]
edition = "2024"
license = "MIT"
rust-version = "1.88"
[profile.dev]
debug = "line-tables-only"
@@ -90,6 +99,8 @@ manifest = { path = "crates/manifest" }
mcp = { path = "crates/mcp" }
lint-common = { path = "crates/lint-common" }
memory = { path = "crates/memory" }
feature-storage = { path = "crates/feature-storage" }
subjektiv = { path = "crates/subjektiv" }
merge-request = { path = "crates/merge-request" }
ticket = { path = "crates/ticket" }
project-record = { path = "crates/project-record" }
@@ -102,10 +113,12 @@ session-metrics = { path = "crates/session-metrics" }
session-analytics = { path = "crates/session-analytics" }
session-store = { path = "crates/session-store" }
standalone = { path = "crates/standalone" }
job = { path = "crates/job" }
runtime-api = { path = "crates/runtime-api" }
server-api = { path = "crates/server-api" }
secrets = { path = "crates/secrets" }
tools = { path = "crates/tools" }
builtin-source = { path = "crates/builtin-source" }
config-source = { path = "crates/config-source" }
fs-operation = { path = "crates/fs-operation" }
workdir = { path = "crates/workdir" }
@@ -124,6 +137,7 @@ decodal-language-service = "0.4.0"
decodal-language-tools = "0.4.0"
fs4 = "0.13"
futures = "0.3"
http = "1"
libc = "0.2"
schemars = "1.2"
serde = "1.0"
@@ -144,5 +158,8 @@ tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
url = "2.5"
uuid = "1.23"
wip-client = "0.1.0"
wip-http = "0.1.0"
wip-protocol = "0.1.0"
zeroize = "1"
webauthn-rs = { version = "0.5.2", features = ["danger-allow-state-serialisation", "danger-credential-internals"] }
+1
View File
@@ -32,6 +32,7 @@ reqwest = { version = "0.13", default-features = false, features = ["stream", "j
eventsource-stream = "0.2"
zstd = "0.13"
base64 = "0.22.1"
imagesize = "0.14"
chrono = { version = "0.4", default-features = false, features = ["serde", "clock"], optional = true }
agen-macros = { workspace = true }
+45 -1
View File
@@ -626,6 +626,10 @@ pub struct Engine<C: LlmClient, S: EngineState = Mutable, A: Send + Sync = ()> {
/// engine history. An error rejects the item and aborts the turn, allowing
/// upper layers to make durable storage the commit gate.
history_append_cbs: Vec<Box<dyn Fn(&Item) -> Result<(), String> + Send + Sync>>,
/// Durable host commit gate for image-result corrections. Runs before the
/// correction becomes live or a replacement request can be sent.
image_rejection_handler:
Option<Box<dyn Fn(&HistoryEntry<A>, &Item) -> Result<(), String> + Send + Sync>>,
/// Request configuration (max_tokens, temperature, etc.)
request_config: RequestConfig,
/// Cancel notification channel (for interrupting execution)
@@ -971,6 +975,16 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
self.history_append_cbs.push(Box::new(callback));
}
/// Set the host's durable commit gate for correcting a rejected ViewImage
/// result. The original annotation/identity must be preserved. A failed
/// commit leaves both history and the outgoing request untouched.
pub fn set_image_rejection_handler(
&mut self,
handler: impl Fn(&HistoryEntry<A>, &Item) -> Result<(), String> + Send + Sync + 'static,
) {
self.image_rejection_handler = Some(Box::new(handler));
}
fn emit_history_append(&self, item: &Item) -> Result<(), EngineError> {
for cb in &self.history_append_cbs {
cb(item).map_err(EngineError::HistoryAppend)?;
@@ -3205,7 +3219,34 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
"Sending request to LLM"
);
let stream = self.open_stream_with_retry(request, turn, llm_call).await?;
let mut request = request;
let stream = loop {
match self
.open_stream_with_retry(request.clone(), turn, llm_call)
.await
{
Ok(stream) => break stream,
Err(EngineError::Client(error)) if error.is_image_size_rejection() => {
let Some((history_index, request_index)) =
crate::image_recovery::largest_candidate(history, &request)
else {
return Err(EngineError::Client(error));
};
let entry = &history.entries()[history_index];
let replacement = crate::image_recovery::rejected_result(&entry.item);
if let Some(commit) = &self.image_rejection_handler {
commit(entry, &replacement).map_err(EngineError::HistoryAppend)?;
}
history.entries_mut()[history_index].item = replacement.clone();
// Rebuild the provider request from the corrected projection,
// retaining prior prune/interceptor decisions. No tool runs
// or assistant output have occurred for this failed request.
request.items[request_index] = replacement;
self.emit_warning(crate::image_recovery::REJECTION_MESSAGE);
}
Err(error) => return Err(error),
}
};
self.tool_call_collector.begin_response();
let mut stream = ResponseStreamPump::start(stream);
let mut early_tools: Option<EarlyToolExecutionBatch> = None;
@@ -3567,6 +3608,7 @@ impl<C: LlmClient, A: Send + Sync> Engine<C, Mutable, A> {
warning_cbs: Vec::new(),
tool_result_cbs: Vec::new(),
history_append_cbs: Vec::new(),
image_rejection_handler: None,
request_config: RequestConfig::default(),
cancel_tx,
cancel_rx,
@@ -3856,6 +3898,7 @@ impl<C: LlmClient, A: Send + Sync> Engine<C, Mutable, A> {
warning_cbs: self.warning_cbs,
tool_result_cbs: self.tool_result_cbs,
history_append_cbs: self.history_append_cbs,
image_rejection_handler: self.image_rejection_handler,
request_config: self.request_config,
cancel_tx: self.cancel_tx,
@@ -4040,6 +4083,7 @@ impl<C: LlmClient, A: Send + Sync> Engine<C, Locked, A> {
warning_cbs: self.warning_cbs,
tool_result_cbs: self.tool_result_cbs,
history_append_cbs: self.history_append_cbs,
image_rejection_handler: self.image_rejection_handler,
request_config: self.request_config,
cancel_tx: self.cancel_tx,
+106
View File
@@ -0,0 +1,106 @@
//! Recovery is driven by provider rejection, never a locally chosen image limit.
use crate::{History, Item, Role, ToolResultDisposition, llm_client::Request, tool::Attachment};
pub(crate) const REJECTION_MESSAGE: &str = "ViewImage image omitted from model input after the provider rejected image size. This is a recovery candidate, not proof that this particular image caused the rejection. Resize or crop the image and retry ViewImage; do not read the unchanged image again.";
/// Keep the most recent response's tool results eligible, including early tool
/// results interleaved with that response's later text/calls. Empty boundaries
/// from older versions' failed requests are not evidence of accepted input.
pub(crate) fn accepted_input_end<'a>(items: impl Iterator<Item = &'a Item>) -> Option<usize> {
let mut response_start = None;
let mut accepted_prefix = None;
for (index, item) in items.enumerate() {
match item {
Item::AssistantResponseBoundary { .. } => response_start = Some(index),
Item::Message {
role: Role::Assistant,
..
}
| Item::ToolCall { .. }
| Item::Reasoning { .. } => {
accepted_prefix = Some(response_start.unwrap_or(index));
}
Item::Message {
role: Role::User, ..
} => response_start = None,
_ => {}
}
}
accepted_prefix
}
/// Select only a ViewImage result actually sent in the rejected request. The
/// tool contract returns exactly one image, so each correction removes one
/// image and preserves the tool call/result pair. Other tools and user uploads
/// are deliberately not rewritten by this recovery policy.
pub(crate) fn largest_candidate<A>(
history: &History<A>,
request: &Request,
) -> Option<(usize, usize)> {
let start = accepted_input_end(history.items()).unwrap_or(0);
let mut largest = None;
for (index, entry) in history.entries().iter().enumerate().skip(start) {
let Item::ToolResult {
call_id,
attachments,
..
} = &entry.item
else {
continue;
};
let [Attachment::Image(image)] = attachments.as_slice() else {
continue;
};
let from_view_image = history.items().any(|item| {
matches!(
item,
Item::ToolCall { call_id: id, name, .. } if id == call_id && name == "ViewImage"
)
});
if !from_view_image {
continue;
}
let sent = request.items.iter().position(|item| {
matches!(
item,
Item::ToolResult { call_id: id, attachments: sent, .. }
if id == call_id && sent == attachments
)
});
let Some(request_index) = sent else { continue };
// Header inspection only; no raster allocation or size threshold.
// Unknown dimensions sort by bytes as a deterministic fallback.
let pixels = imagesize::blob_size(image.data())
.ok()
.map(|size| (size.width as u64).saturating_mul(size.height as u64))
.unwrap_or(0);
let rank = (pixels, image.data().len(), index);
if largest
.as_ref()
.is_none_or(|(previous, _)| rank > *previous)
{
largest = Some((rank, (index, request_index)));
}
}
largest.map(|(_, candidate)| candidate)
}
pub(crate) fn rejected_result(item: &Item) -> Item {
let mut item = item.clone();
if let Item::ToolResult {
summary,
content,
attachments,
disposition,
is_error,
..
} = &mut item
{
*summary = REJECTION_MESSAGE.to_owned();
*content = None;
attachments.clear();
*disposition = ToolResultDisposition::Error;
*is_error = true;
}
item
}
+1
View File
@@ -3,6 +3,7 @@
mod engine;
mod handler;
mod history;
mod image_recovery;
mod message;
pub(crate) mod callback;
+29
View File
@@ -90,6 +90,35 @@ impl ClientError {
}
}
/// A provider-declared image size rejection, not a generic invalid input.
/// Some providers (including OpenAI) expose only `invalid_value` here, so
/// narrowly recognize their image-size diagnostic without hardcoded limits.
pub fn is_image_size_rejection(&self) -> bool {
let Self::Api {
status: Some(400 | 413 | 422),
code,
message,
..
} = self
else {
return false;
};
if matches!(
code.as_deref(),
Some("image_too_large" | "image_size_exceeded")
) {
return true;
}
let message = message.to_ascii_lowercase();
message.contains("image")
&& ((message.contains("patches")
&& message.contains("exceed")
&& message.contains("limit"))
|| message.contains("image is too large")
|| message.contains("image too large")
|| (message.contains("image dimensions") && message.contains("exceed")))
}
pub fn retry_after(&self) -> Option<Duration> {
match self {
ClientError::Api { retry_after, .. } => *retry_after,
+6 -7
View File
@@ -136,14 +136,13 @@ pub fn project(items: &mut [Item], indices: &[usize]) -> usize {
/// Remove binary attachments after the first model request that could consume them.
///
/// Attachments on tool results following the latest model output are fresh outputs for the
/// immediate follow-up request and remain visible. Once a later assistant/reasoning/tool-call
/// item exists, the model has already had an opportunity to inspect the attachment. Subsequent
/// requests keep the durable tool summary and can call the originating tool again when the image
/// is still needed. Only the request-context clone is modified.
/// immediate follow-up request and remain visible. With early dispatch, later
/// text/calls in the same response have not consumed its tool results: use the
/// confirmed response boundary, not that response's last individual item.
/// Subsequent requests keep the durable tool summary and can call the originating
/// tool again when the image is still needed. Only the request-context clone is modified.
pub fn project_consumed_attachments(items: &mut [Item]) -> usize {
let latest_model_output = items.iter().rposition(|item| {
item.is_assistant_message() || item.is_reasoning() || item.is_tool_call()
});
let latest_model_output = crate::image_recovery::accepted_input_end(items.iter());
let mut count = 0;
for (index, item) in items.iter_mut().enumerate() {
let is_fresh = latest_model_output.is_some_and(|boundary| index > boundary);
+350
View File
@@ -0,0 +1,350 @@
use agen::llm_client::event::{Event, ResponseStatus, StatusEvent};
use agen::llm_client::{ClientError, LlmClient, Request, ResponseStream};
use agen::tool::{Attachment, ImageAttachment};
use agen::{Engine, EngineRunExit, History, Item, ToolResultDisposition};
use async_trait::async_trait;
use std::sync::{Arc, Mutex};
fn rejection() -> ClientError {
ClientError::Api { status: Some(400), code: Some("invalid_value".into()), message: "The image you provided requires 52150 patches after processing, exceeding the limit of 30000. Please resize the image and try again.".into(), retry_after: None }
}
fn image_result(id: &str, width: u32, height: u32, padding: usize) -> Item {
// PNG header: dimension inspection never decodes/allocates the pixel plane.
let mut bytes = b"\x89PNG\r\n\x1a\n\0\0\0\rIHDR".to_vec();
bytes.extend(width.to_be_bytes());
bytes.extend(height.to_be_bytes());
bytes.resize(40 + padding, 0);
Item::ToolResult {
id: None,
call_id: id.into(),
summary: "Attached image".into(),
content: None,
disposition: ToolResultDisposition::Success,
is_error: false,
attachments: vec![Attachment::Image(ImageAttachment::new("image/png", bytes))],
}
}
fn history() -> History {
History::from_items(vec![
Item::user_message("inspect"),
Item::tool_call("small", "ViewImage", "{}"),
Item::tool_call("large", "ViewImage", "{}"),
image_result("small", 10, 10, 2000),
image_result("large", 1600, 33349, 0),
])
}
fn images(request: &Request) -> Vec<String> {
request
.items
.iter()
.filter_map(|item| match item {
Item::ToolResult {
call_id,
attachments,
..
} if !attachments.is_empty() => Some(call_id.clone()),
_ => None,
})
.collect()
}
#[derive(Clone)]
struct Client {
requests: Arc<Mutex<Vec<Request>>>,
failures: usize,
other_error: bool,
}
impl Client {
fn new(failures: usize) -> Self {
Self {
requests: Arc::default(),
failures,
other_error: false,
}
}
}
#[async_trait]
impl LlmClient for Client {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(&self, request: Request) -> Result<ResponseStream, ClientError> {
let mut requests = self.requests.lock().unwrap();
requests.push(request);
if requests.len() <= self.failures {
return Err(rejection());
}
if self.other_error {
return Err(ClientError::Api {
status: Some(400),
code: Some("invalid_value".into()),
message: "invalid tool arguments".into(),
retry_after: None,
});
}
Ok(Box::pin(futures::stream::iter(vec![
Ok(Event::text_block_start(0)),
Ok(Event::text_delta(0, "resize and retry")),
Ok(Event::text_block_stop(0, None)),
Ok(Event::Status(StatusEvent {
status: ResponseStatus::Completed,
})),
])))
}
}
#[tokio::test]
async fn removes_largest_pixels_not_largest_file_and_keeps_pairing() {
let client = Client::new(1);
let observer = client.clone();
let mut history = history();
let mut engine = Engine::new(client);
let committed = Arc::new(Mutex::new(Vec::new()));
let commits = committed.clone();
engine.set_image_rejection_handler(move |original, replacement| {
commits
.lock()
.unwrap()
.push((original.item.clone(), replacement.clone()));
Ok(())
});
let output = engine.run(&mut history, "continue").await;
assert!(
matches!(output.result, EngineRunExit::Finished),
"{:?}",
output.result
);
let requests = observer.requests.lock().unwrap();
assert_eq!(requests.len(), 2);
assert_eq!(images(&requests[0]), ["small", "large"]);
assert_eq!(images(&requests[1]), ["small"]);
assert!(requests[1].items.iter().any(|item| matches!(item, Item::ToolResult { call_id, is_error: true, disposition: ToolResultDisposition::Error, summary, .. } if call_id == "large" && summary.contains("retry ViewImage"))));
assert_eq!(committed.lock().unwrap().len(), 1);
assert_eq!(
history
.items()
.filter(|item| matches!(item, Item::ToolResult {call_id, ..} if call_id == "large"))
.count(),
1
);
}
#[tokio::test]
async fn multiple_rejections_remove_one_image_at_a_time_and_stop_at_exhaustion() {
let client = Client::new(99);
let observer = client.clone();
let mut history = history();
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
let requests = observer.requests.lock().unwrap();
assert_eq!(requests.len(), 3);
assert_eq!(images(&requests[1]), ["small"]);
assert!(images(&requests[2]).is_empty());
}
#[tokio::test]
async fn unrelated_error_stops_recovery_without_removing_another_image() {
let mut client = Client::new(1);
client.other_error = true;
let observer = client.clone();
let mut history = history();
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
assert_eq!(observer.requests.lock().unwrap().len(), 2);
assert!(history.items().any(|item| matches!(item, Item::ToolResult {call_id, attachments, ..} if call_id == "small" && !attachments.is_empty())));
}
#[tokio::test]
async fn persistence_failure_does_not_change_history_or_retry() {
let client = Client::new(99);
let observer = client.clone();
let mut history = history();
let original = history.items_cloned();
let mut engine = Engine::new(client);
engine.set_image_rejection_handler(|_, _| Err("disk full".into()));
let out = engine.run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
assert_eq!(
&history.items_cloned()[..original.len()],
original.as_slice()
);
assert_eq!(observer.requests.lock().unwrap().len(), 1);
}
#[tokio::test]
async fn previously_accepted_images_are_not_recovery_candidates() {
let client = Client::new(99);
let observer = client.clone();
let mut history = history();
history.push(Item::assistant_message("I saw those images"));
history.push(Item::tool_call("new", "ViewImage", "{}"));
history.push(image_result("new", 1, 1, 0));
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
let requests = observer.requests.lock().unwrap();
assert_eq!(requests.len(), 2);
// Already-consumed images are projected out, but their durable records
// must not be rewritten as failures.
assert!(images(&requests[1]).is_empty());
assert_eq!(history.items().filter(|item| matches!(item, Item::ToolResult {call_id, is_error: false, attachments, ..} if (call_id == "small" || call_id == "large") && !attachments.is_empty())).count(), 2);
}
#[tokio::test]
async fn legacy_empty_failed_request_boundaries_do_not_hide_candidates() {
let client = Client::new(1);
let observer = client.clone();
let mut history = history();
history.push(Item::assistant_response_boundary("failed-1"));
history.push(Item::assistant_response_boundary("failed-2"));
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Finished));
assert_eq!(images(&observer.requests.lock().unwrap()[1]), ["small"]);
}
#[tokio::test]
async fn early_results_remain_candidates_despite_later_text_in_same_response() {
let client = Client::new(1);
let observer = client.clone();
let mut history = History::from_items(vec![
Item::user_message("inspect"),
Item::assistant_response_boundary("response-1"),
Item::tool_call("image", "ViewImage", "{}"),
image_result("image", 99, 99, 0),
Item::assistant_message("waiting for image"),
]);
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(
matches!(out.result, EngineRunExit::Finished),
"{:?}; requests: {:?}",
out.result,
observer.requests.lock().unwrap()
);
assert!(images(&observer.requests.lock().unwrap()[1]).is_empty());
}
#[derive(Clone)]
struct CountingTool {
image: bool,
count: Arc<std::sync::atomic::AtomicUsize>,
}
#[async_trait]
impl agen::tool::Tool for CountingTool {
async fn execute(
&self,
_: &str,
_: agen::tool::ToolExecutionContext,
) -> Result<agen::tool::ToolOutput, agen::tool::ToolError> {
self.count.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
let attachments = if self.image {
let Item::ToolResult { attachments, .. } = image_result("image", 1600, 33349, 0) else {
unreachable!()
};
attachments
} else {
Vec::new()
};
Ok(agen::tool::ToolOutput {
summary: "executed once".into(),
content: None,
attachments,
})
}
}
#[derive(Clone)]
struct ToolClient(Client);
#[async_trait]
impl LlmClient for ToolClient {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
fn tool_call_completion_support(&self) -> agen::llm_client::ToolCallCompletionSupport {
agen::llm_client::ToolCallCompletionSupport::PerBlock
}
async fn stream(&self, request: Request) -> Result<ResponseStream, ClientError> {
if self.0.requests.lock().unwrap().is_empty() {
self.0.requests.lock().unwrap().push(request);
return Ok(Box::pin(futures::stream::iter(
vec![
Event::tool_use_start(0, "image", "ViewImage"),
Event::tool_input_delta(0, "{}"),
Event::tool_use_stop(0),
Event::tool_use_start(1, "side-effect", "Bash"),
Event::tool_input_delta(1, "{}"),
Event::tool_use_stop(1),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
.into_iter()
.map(Ok),
)));
}
self.0.stream(request).await
}
}
#[tokio::test]
async fn fresh_image_recovery_does_not_repeat_tools_in_either_dispatch_mode() {
for mode in [
agen::ToolCallDispatchMode::AfterResponse,
agen::ToolCallDispatchMode::OnToolCallComplete,
] {
let client = Client::new(2);
let mut engine = Engine::new(ToolClient(client.clone()));
engine.set_tool_call_dispatch_mode(mode);
let mut counts = Vec::new();
for (name, image) in [("ViewImage", true), ("Bash", false)] {
let tool = CountingTool {
image,
count: Arc::default(),
};
counts.push(tool.count.clone());
engine.register_tool(Arc::new(move || {
(
agen::tool::ToolMeta::new(name)
.description("test")
.input_schema(serde_json::json!({"type":"object"})),
Arc::new(tool.clone()),
)
}));
}
let mut history = History::new();
let out = engine.run(&mut history, "inspect").await;
assert!(
matches!(out.result, EngineRunExit::Finished),
"{:?}",
out.result
);
assert!(
counts
.iter()
.all(|count| count.load(std::sync::atomic::Ordering::SeqCst) == 1)
);
let requests = client.requests.lock().unwrap();
assert_eq!(requests.len(), 3);
assert_eq!(images(&requests[1]), ["image"]);
assert!(images(&requests[2]).is_empty());
}
}
#[test]
fn image_error_classifier_is_not_a_generic_400_retry() {
assert!(rejection().is_image_size_rejection());
for (status, message) in [
(400, "invalid image format"),
(400, "image URL not found"),
(400, "context window exceeds token limit"),
(401, "image is too large"),
(500, "image is too large"),
] {
assert!(
!ClientError::Api {
status: Some(status),
code: Some("invalid_value".into()),
message: message.into(),
retry_after: None
}
.is_image_size_rejection()
);
}
}
+9
View File
@@ -0,0 +1,9 @@
[package]
name = "builtin-source"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish = false
[dependencies]
sha2.workspace = true
+214
View File
@@ -0,0 +1,214 @@
//! Explicit compile-time public Decodal source catalog, shared by native and WASM hosts.
use sha2::{Digest, Sha256};
use std::collections::BTreeMap;
pub const BUILTIN_PROFILE_CATALOG_ID: &str = "builtin-profiles-v2";
pub const BUILTIN_DEFAULT_PROFILE: &str = "builtin:default";
pub const BUILTIN_STANDALONE_PROFILE: &str = "builtin:standalone";
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BuiltinProfileImport {
pub specifier: &'static str,
pub resolved_path: &'static str,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BuiltinProfileResource {
pub selector: Option<&'static str>,
pub path: &'static str,
pub source: &'static str,
pub description: &'static str,
pub imports: &'static [BuiltinProfileImport],
}
const BASE_PATH: &str = "profiles/base.dcdl";
pub const DEFAULT_PATH: &str = "profiles/default.dcdl";
const BASE_IMPORT: &[BuiltinProfileImport] = &[BuiltinProfileImport {
specifier: "./base.dcdl",
resolved_path: BASE_PATH,
}];
const DEFAULT_IMPORT: &[BuiltinProfileImport] = &[BuiltinProfileImport {
specifier: "./default.dcdl",
resolved_path: DEFAULT_PATH,
}];
const NO_IMPORTS: &[BuiltinProfileImport] = &[];
pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
BuiltinProfileResource {
selector: None,
path: BASE_PATH,
source: include_str!("../../../resources/profiles/base.dcdl"),
description: "Shared built-in Profile defaults.",
imports: NO_IMPORTS,
},
BuiltinProfileResource {
selector: Some(BUILTIN_DEFAULT_PROFILE),
path: DEFAULT_PATH,
source: include_str!("../../../resources/profiles/default.dcdl"),
description: "Default Yoi coding profile.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some(BUILTIN_STANDALONE_PROFILE),
path: "profiles/standalone.dcdl",
source: include_str!("../../../resources/profiles/standalone.dcdl"),
description: "Standalone Yoi coding profile.",
imports: DEFAULT_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:standalone-subjektiv"),
path: "profiles/standalone-subjektiv.dcdl",
source: include_str!("../../../resources/profiles/standalone-subjektiv.dcdl"),
description: "Standalone with an explicitly connected local Subject.",
imports: &[BuiltinProfileImport {
specifier: "./standalone.dcdl",
resolved_path: "profiles/standalone.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:standalone-subjektiv-consolidation"),
path: "profiles/standalone-subjektiv-consolidation.dcdl",
source: include_str!("../../../resources/profiles/standalone-subjektiv-consolidation.dcdl"),
description: "Delegated local Subject Job policy.",
imports: &[BuiltinProfileImport {
specifier: "./job.dcdl",
resolved_path: "profiles/job.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:coder"),
path: "profiles/coder.dcdl",
source: include_str!("../../../resources/profiles/coder.dcdl"),
description: "Ticket implementation with direct Reviewer SubWorkers.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:companion"),
path: "profiles/companion.dcdl",
source: include_str!("../../../resources/profiles/companion.dcdl"),
description: "General assistance with Workspace tools.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:intake"),
path: "profiles/intake.dcdl",
source: include_str!("../../../resources/profiles/intake.dcdl"),
description: "Read-only intake and planning.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:reviewer"),
path: "profiles/reviewer.dcdl",
source: include_str!("../../../resources/profiles/reviewer.dcdl"),
description: "Independent review of a published Merge Request source.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:orchestrator"),
path: "profiles/orchestrator.dcdl",
source: include_str!("../../../resources/profiles/orchestrator.dcdl"),
description: "Workspace orchestration and Worker control.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:job"),
path: "profiles/job.dcdl",
source: include_str!("../../../resources/profiles/job.dcdl"),
description: "Result-only caller-selected Job policy.",
imports: NO_IMPORTS,
},
BuiltinProfileResource {
selector: Some("builtin:backend-job"),
path: "profiles/backend-job.dcdl",
source: include_str!("../../../resources/profiles/backend-job.dcdl"),
description: "Backend-owned bounded Job Worker.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:memory-consolidation"),
path: "profiles/memory-consolidation.dcdl",
source: include_str!("../../../resources/profiles/memory-consolidation.dcdl"),
description: "Internal Memory consolidation service.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:subjektiv-memory-consolidation"),
path: "profiles/subjektiv-memory-consolidation.dcdl",
source: include_str!("../../../resources/profiles/subjektiv-memory-consolidation.dcdl"),
description: "Delegated subject Memory consolidation service.",
imports: BASE_IMPORT,
},
];
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BuiltinProfileCatalogSnapshot {
pub id: &'static str,
pub sources: BTreeMap<String, String>,
pub entrypoints: BTreeMap<String, String>,
pub imports: BTreeMap<String, String>,
}
impl BuiltinProfileCatalogSnapshot {
pub fn digest(&self) -> String {
let mut hasher = Sha256::new();
hasher.update(self.id.as_bytes());
for (path, source) in &self.sources {
hasher.update((path.len() as u64).to_le_bytes());
hasher.update(path.as_bytes());
hasher.update((source.len() as u64).to_le_bytes());
hasher.update(source.as_bytes());
}
for (selector, path) in &self.entrypoints {
hasher.update((selector.len() as u64).to_le_bytes());
hasher.update(selector.as_bytes());
hasher.update((path.len() as u64).to_le_bytes());
hasher.update(path.as_bytes());
}
for (request, resolved_path) in &self.imports {
hasher.update((request.len() as u64).to_le_bytes());
hasher.update(request.as_bytes());
hasher.update((resolved_path.len() as u64).to_le_bytes());
hasher.update(resolved_path.as_bytes());
}
format!(
"sha256:{}",
hasher
.finalize()
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>()
)
}
}
pub fn builtin_profile_catalog_snapshot() -> BuiltinProfileCatalogSnapshot {
let mut sources = BTreeMap::new();
let mut entrypoints = BTreeMap::new();
let mut imports = BTreeMap::new();
for resource in BUILTIN_PROFILE_RESOURCES {
sources.insert(resource.path.to_owned(), resource.source.to_owned());
for import in resource.imports {
imports.insert(
format!("{}\0{}", resource.path, import.specifier),
import.resolved_path.to_owned(),
);
}
if let Some(selector) = resource.selector {
entrypoints.insert(selector.to_owned(), resource.path.to_owned());
}
}
BuiltinProfileCatalogSnapshot {
id: BUILTIN_PROFILE_CATALOG_ID,
sources,
entrypoints,
imports,
}
}
pub fn builtin_profile_entrypoints() -> impl Iterator<Item = &'static BuiltinProfileResource> {
BUILTIN_PROFILE_RESOURCES
.iter()
.filter(|resource| resource.selector.is_some())
}
+41
View File
@@ -0,0 +1,41 @@
use builtin_source::{BUILTIN_PROFILE_RESOURCES, builtin_profile_catalog_snapshot};
use std::collections::BTreeSet;
#[test]
fn public_catalog_is_closed_and_explicit() {
let catalog = builtin_profile_catalog_snapshot();
assert_eq!(catalog.sources.len(), BUILTIN_PROFILE_RESOURCES.len());
assert!(catalog.digest().starts_with("sha256:"));
for resource in BUILTIN_PROFILE_RESOURCES {
assert!(resource.path.starts_with("profiles/"));
assert!(resource.path.ends_with(".dcdl"));
assert_eq!(catalog.sources[resource.path], resource.source);
for import in resource.imports {
assert!(catalog.sources.contains_key(import.resolved_path));
assert!(
resource
.source
.contains(&format!("import {:?}", import.specifier))
);
}
}
assert!(!catalog.sources.contains_key("prompts/default.md"));
}
#[test]
fn catalog_import_metadata_has_no_cycles() {
fn visit(path: &str, visiting: &mut BTreeSet<String>) {
assert!(visiting.insert(path.to_string()), "catalog cycle at {path}");
let resource = BUILTIN_PROFILE_RESOURCES
.iter()
.find(|resource| resource.path == path)
.unwrap();
for import in resource.imports {
visit(import.resolved_path, visiting);
}
visiting.remove(path);
}
for resource in BUILTIN_PROFILE_RESOURCES {
visit(resource.path, &mut BTreeSet::new());
}
}
+15 -1
View File
@@ -10,6 +10,7 @@ pub use server_api::{
WorkerLaunchProfileCandidate as BackendWorkerLaunchProfileCandidate,
WorkerLaunchRuntimeOption as BackendWorkerLaunchRuntimeOption,
WorkerOperationState as BackendWorkerOperationState,
WorkerRestoreRequest as BackendWorkerRestoreRequest,
WorkerRestoreResponse as BackendWorkerRestoreResponse,
WorkerRestoreResult as BackendWorkerRestoreResult,
WorkerRestoreState as BackendWorkerRestoreState, WorkerSummary as BackendWorkerSummary,
@@ -259,6 +260,15 @@ impl fmt::Display for BackendRuntimeClientError {
}
}
impl BackendRuntimeClientError {
/// A stale intent is not a rejected Restore operation and must not be retried
/// with a silently refreshed token.
pub fn is_restore_observation_conflict(&self) -> bool {
matches!(self, Self::ContractApi(server_api::client_support::ClientError::Public { status, error })
if *status == reqwest::StatusCode::CONFLICT && error.error == "restore_observation_conflict")
}
}
impl std::error::Error for BackendRuntimeClientError {}
impl From<server_api::ServerApiClientError> for BackendRuntimeClientError {
@@ -502,6 +512,7 @@ pub async fn list_backend_stopped_workers(
pub async fn restore_backend_worker(
target: &BackendRuntimeTarget,
request: BackendWorkerRestoreRequest,
) -> Result<BackendWorkerRestoreResponse, BackendRuntimeClientError> {
validate_target(target)?;
let api = BackendApiClient::from_stored_token(&target.base_url)?;
@@ -514,6 +525,7 @@ pub async fn restore_backend_worker(
ticket_id: None,
assignment_operation_id: None,
},
request,
)
.await
.map_err(Into::into)
@@ -819,7 +831,8 @@ mod tests {
"profiles": [{
"id": "builtin:default",
"label": "Default",
"description": ""
"description": "",
"feature_connections": { "subjektiv": false }
}],
"repositories": [],
"working_directories": [],
@@ -883,6 +896,7 @@ mod tests {
ticket_assignment: None,
initial_submit: Vec::new(),
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
};
+6 -6
View File
@@ -26,12 +26,12 @@ pub use backend_runtime::{
BackendRuntimeListTarget, BackendRuntimeSummary, BackendRuntimeTarget,
BackendWorkerImplementationSummary, BackendWorkerLaunchOptions,
BackendWorkerLaunchProfileCandidate, BackendWorkerLaunchRuntimeOption,
BackendWorkerLaunchTarget, BackendWorkerOperationState, BackendWorkerRestoreResponse,
BackendWorkerRestoreResult, BackendWorkerRestoreState, BackendWorkerSummary,
BackendWorkerWorkspaceSummary, BackendWorkingDirectorySource, BackendWorkingDirectorySummary,
connect_backend_runtime, create_backend_worker, get_backend_worker_launch_options,
list_backend_stopped_workers, list_backend_workers, observe_backend_worker_session,
restore_backend_worker,
BackendWorkerLaunchTarget, BackendWorkerOperationState, BackendWorkerRestoreRequest,
BackendWorkerRestoreResponse, BackendWorkerRestoreResult, BackendWorkerRestoreState,
BackendWorkerSummary, BackendWorkerWorkspaceSummary, BackendWorkingDirectorySource,
BackendWorkingDirectorySummary, connect_backend_runtime, create_backend_worker,
get_backend_worker_launch_options, list_backend_stopped_workers, list_backend_workers,
observe_backend_worker_session, restore_backend_worker,
};
pub use backend_workspace::{
BackendWorkspace, BackendWorkspaceCatalogTarget, BackendWorkspaceClientError,
+1
View File
@@ -532,6 +532,7 @@ impl BackendWorkspaceProductClient {
content: format!("Please handle intake for Ticket {ticket_id}."),
}],
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
};
let workspace_id = self.workspace_id.clone();
+133 -22
View File
@@ -98,31 +98,47 @@ pub fn complete_current(
let snapshot = session
.as_ref()
.ok_or_else(|| JsValue::from_str("config source snapshot is not initialized"))?;
let utf8_byte_offset = utf16_to_utf8_offset(&source, utf16_offset)?;
let result = session_environment(snapshot.clone())
.complete_config(&entrypoint, &source, utf8_byte_offset, explicit)
.map_err(|error| JsValue::from_str(&format!("{error:?}")))?;
let result = result
.map(|result| {
Ok::<WasmCompletionResult, JsValue>(WasmCompletionResult {
from: utf8_to_utf16_offset(&source, result.from)?,
items: result
.items
.into_iter()
.map(|item| WasmCompletionItem {
label: item.label,
kind: format!("{:?}", item.kind).to_lowercase(),
detail: item.detail,
priority: item.priority,
})
.collect(),
})
})
.transpose()?;
encode(result)
encode(complete_snapshot(
snapshot,
&entrypoint,
&source,
utf16_offset,
explicit,
)?)
})
}
fn complete_snapshot(
snapshot: &ConfigTreeSnapshot,
entrypoint: &VirtualPath,
source: &str,
utf16_offset: usize,
explicit: bool,
) -> Result<Option<WasmCompletionResult>, JsValue> {
let utf8_byte_offset = utf16_to_utf8_offset(source, utf16_offset)?;
let result = session_environment(snapshot.clone())
.complete_config(entrypoint, source, utf8_byte_offset, explicit)
.map_err(|error| JsValue::from_str(&format!("{error:?}")))?;
let result = result
.map(|result| {
Ok::<WasmCompletionResult, JsValue>(WasmCompletionResult {
from: utf8_to_utf16_offset(source, result.from)?,
items: result
.items
.into_iter()
.map(|item| WasmCompletionItem {
label: item.label,
kind: format!("{:?}", item.kind).to_lowercase(),
detail: item.detail,
priority: item.priority,
})
.collect(),
})
})
.transpose()?;
Ok(result)
}
#[wasm_bindgen]
pub fn evaluate_snapshot(snapshot: JsValue, contract: JsValue) -> Result<JsValue, JsValue> {
let snapshot: ConfigTreeSnapshot = decode(snapshot)?;
@@ -207,3 +223,98 @@ fn js_error(error: impl std::fmt::Display) -> JsValue {
#[allow(dead_code)]
fn _assert_serializable(_: EvaluationResult) {}
#[cfg(test)]
mod tests {
use super::*;
use config_source::{ConfigContentType, ConfigEntry};
fn snapshot() -> ConfigTreeSnapshot {
ConfigTreeSnapshot::from_entries(
7,
[
ConfigEntry::new(
VirtualPath::parse("main.dcdl").unwrap(),
ConfigContentType::Decodal,
"{}",
)
.unwrap(),
// A same-suffix workspace file must not be an unknown builtin fallback.
ConfigEntry::new(
VirtualPath::parse("profiles/missing.dcdl").unwrap(),
ConfigContentType::Decodal,
"{}",
)
.unwrap(),
],
)
.unwrap()
}
#[test]
fn wasm_completion_boundary_preserves_read_only_builtin_detail() {
let snapshot = snapshot();
let source = "let 名 = 1; import \"$builtin/profiles/comp\"";
let cursor = source.encode_utf16().count() - 1;
let result = complete_snapshot(
&snapshot,
&VirtualPath::parse("main.dcdl").unwrap(),
source,
cursor,
true,
)
.unwrap()
.unwrap();
assert_eq!(result.from, "let 名 = 1; import \"".encode_utf16().count());
let item = result
.items
.iter()
.find(|item| item.label == "$builtin/profiles/companion.dcdl")
.unwrap();
assert_eq!(item.kind, "file");
assert_eq!(
item.detail.as_deref(),
Some("read-only builtin Decodal source")
);
assert_eq!(snapshot.entries.len(), 2);
assert!(
snapshot
.entries
.keys()
.all(|path| !path.as_str().starts_with("$builtin/"))
);
}
#[test]
fn wasm_analysis_boundary_reports_unknown_builtin_without_workspace_fallback() {
let snapshot = snapshot();
let path = VirtualPath::parse("main.dcdl").unwrap();
let diagnostics = session_environment(snapshot.clone())
.analyze(&path, Some("import \"$builtin/profiles/missing.dcdl\""));
assert!(!diagnostics.is_empty());
let diagnostic = diagnostics
.iter()
.find(|diagnostic| {
diagnostic
.message
.contains("unknown or non-public read-only builtin source")
})
.unwrap();
assert_eq!(diagnostic.path, path);
assert_eq!(diagnostic.revision, 7);
assert_eq!(diagnostic.tree_digest, snapshot.digest);
assert!(
diagnostic
.message
.contains("$builtin/profiles/missing.dcdl")
);
assert!(
session_environment(snapshot)
.analyze(
&VirtualPath::parse("main.dcdl").unwrap(),
Some("import \"./profiles/missing.dcdl\"")
)
.is_empty()
);
}
}
+1
View File
@@ -6,6 +6,7 @@ license.workspace = true
publish = false
[dependencies]
builtin-source.workspace = true
decodal.workspace = true
decodal-language-service.workspace = true
decodal-language-tools.workspace = true
File diff suppressed because it is too large Load Diff
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "feature-storage"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish = false
[dependencies]
rusqlite.workspace = true
thiserror.workspace = true
uuid = { workspace = true, features = ["v7"] }
[dev-dependencies]
tempfile.workspace = true
+28
View File
@@ -0,0 +1,28 @@
# feature-storage
Neutral Host-owned SQLite lifecycle manager for trusted Features. Owns placement,
WAL/connection settings, transaction serialization, migrations, snapshots,
restore/delete fencing, and shutdown. Features receive a managed
`FeatureDatabase` after registration. Worker/model input never selects a database
path or receives SQL access.
Use `FeatureStorage::new(trusted_root)`, `register(FeatureRegistration)`, and
`scope(host_issued_scope_id).open(&registration)`. `ScopedFeatureStorage` and
`FeatureDatabase::scope_id()` are neutral storage isolation APIs: a local Host's
persisted random scope does not create a Workspace record or imply Backend
membership. Existing placement `<root>/<scope>/features/<feature>.sqlite` and
snapshot `workspace=` keys remain unchanged for physical compatibility.
`WorkspaceFeatureStorage`, `workspace()`, `workspace_id()`, and
`for_server_database()` are Backend compatibility aliases/helpers. They do not
introduce a Server dependency. The Host is responsible for state-root privacy,
symlink/path validation, and external execution leases before constructing this
trusted manager.
`configure_connection(&rusqlite::Connection)` is reusable by other Host stores
(such as JobStore). It sets the five-second busy timeout, foreign keys ON, WAL,
and synchronous FULL before migrations/transactions. The Feature manager uses
this same helper; WAL lock-contention retries stay within the busy timeout.
Dependencies: `rusqlite` (existing workspace backup support), `thiserror`, `uuid`;
tests use `tempfile`.
File diff suppressed because it is too large Load Diff
+1 -3
View File
@@ -13,9 +13,7 @@ ignore = "0.4.25"
libc = "0.2"
schemars.workspace = true
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
sha2 = "0.10.9"
tempfile.workspace = true
thiserror.workspace = true
[dev-dependencies]
serde_json.workspace = true
+518
View File
@@ -0,0 +1,518 @@
//! Identity-fenced access adapter for the existing read/edit/write engine.
//!
//! Descriptor-rooted resolution is Linux-only and symlinks/mount crossings fail
//! closed. The final compare and atomic rename are NOT a kernel CAS against an
//! arbitrary external editor: external editors must cooperate with provider
//! serialization. A hostile rename in that last syscall gap is not excluded.
use crate::{AtomicWriteMode, FsAccessPolicy};
use std::fs::{File, Metadata};
use std::io::{Seek, SeekFrom};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
/// Identity and state, deliberately not just a content digest. atime is omitted
/// so reading does not invalidate an observation. No host paths are encoded.
pub fn identity_validator(metadata: &Metadata) -> std::io::Result<Vec<u8>> {
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
let mut bytes = b"yoi-checkout-v1".to_vec();
for value in [
metadata.dev(),
metadata.ino(),
metadata.len(),
metadata.mode() as u64,
metadata.mtime() as u64,
metadata.mtime_nsec() as u64,
metadata.ctime() as u64,
metadata.ctime_nsec() as u64,
] {
bytes.extend_from_slice(&value.to_le_bytes());
}
Ok(bytes)
}
#[cfg(not(unix))]
{
let _ = metadata;
Err(std::io::Error::new(
std::io::ErrorKind::Unsupported,
"identity validators unavailable",
))
}
}
/// One pinned target and its save parent. Retained through the shared operation.
pub struct CheckedTarget<'a> {
policy: &'a dyn FsAccessPolicy,
root: File,
relative: PathBuf,
logical: PathBuf,
file: File,
parent: Option<File>,
validator: Vec<u8>,
effects_possible: AtomicBool,
}
impl<'a> CheckedTarget<'a> {
pub fn pin(
root: &File,
relative: &Path,
logical: &Path,
policy: &'a dyn FsAccessPolicy,
write: bool,
) -> std::io::Result<Self> {
policy.check_cancelled()?;
if !(if write {
policy.is_writable_paths(logical, logical)
} else {
policy.is_readable_paths(logical, logical)
}) {
return Err(std::io::Error::new(
std::io::ErrorKind::PermissionDenied,
"checkout target denied",
));
}
let parent = if relative.as_os_str().is_empty() {
None
} else {
Some(crate::open_beneath_no_symlinks_at(
root,
relative.parent().unwrap_or(Path::new("")),
)?)
};
let file = if let Some(parent) = &parent {
crate::open_beneath_no_symlinks_at(parent, Path::new(relative.file_name().unwrap()))?
} else {
crate::open_beneath_no_symlinks_at(root, Path::new(""))?
};
let metadata = file.metadata()?;
if !metadata.is_file() && !metadata.is_dir() {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"checkout requires regular file or directory",
));
}
let result = Self {
policy,
root: root.try_clone()?,
relative: relative.to_path_buf(),
logical: logical.to_path_buf(),
file,
parent,
validator: identity_validator(&metadata)?,
effects_possible: AtomicBool::new(false),
};
result.confirm()?;
Ok(result)
}
pub fn validator(&self) -> &[u8] {
&self.validator
}
pub fn metadata(&self) -> std::io::Result<Metadata> {
self.file.metadata()
}
pub fn effects_possible(&self) -> bool {
self.effects_possible.load(Ordering::Acquire)
}
pub fn confirm(&self) -> std::io::Result<()> {
self.policy.check_cancelled()?;
if let Some(parent) = &self.parent {
let named_parent = crate::open_beneath_no_symlinks_at(
&self.root,
self.relative.parent().unwrap_or(Path::new("")),
)?;
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
let pinned_metadata = parent.metadata()?;
let named_metadata = named_parent.metadata()?;
if (pinned_metadata.dev(), pinned_metadata.ino())
!= (named_metadata.dev(), named_metadata.ino())
{
return Err(std::io::Error::new(
std::io::ErrorKind::WouldBlock,
"checkout parent identity changed",
));
}
}
}
let named = crate::open_beneath_no_symlinks_at(&self.root, &self.relative)?;
if identity_validator(&named.metadata()?)? != self.validator
|| identity_validator(&self.file.metadata()?)? != self.validator
{
return Err(std::io::Error::new(
std::io::ErrorKind::WouldBlock,
"checkout observation changed",
));
}
Ok(())
}
/// Refresh a directory observation after Create, retaining the original
/// pinned target/root/parent handles. A replacement named directory cannot
/// supply the validator for effects performed in the original directory.
pub fn refresh_directory_observation(&mut self) -> std::io::Result<()> {
let metadata = self.file.metadata()?;
if !metadata.is_dir() {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"Create target must be directory",
));
}
self.validator = identity_validator(&metadata)?;
self.confirm()
}
/// Adapter for no-clobber creation beneath a pinned directory. Both target
/// and destination authorization remain in the provider's normal policy.
pub fn create_access<'b>(&'b self, suffix: PathBuf, logical: PathBuf) -> CheckedCreate<'b, 'a> {
CheckedCreate {
target: self,
suffix,
logical,
}
}
}
impl FsAccessPolicy for CheckedTarget<'_> {
fn is_readable(&self, p: &Path) -> bool {
self.policy.is_readable(p)
}
fn is_writable(&self, p: &Path) -> bool {
self.policy.is_writable(p)
}
fn is_readable_paths(&self, p: &Path, r: &Path) -> bool {
p == self.logical && r == p && self.policy.is_readable_paths(p, r)
}
fn is_writable_paths(&self, p: &Path, r: &Path) -> bool {
p == self.logical && r == p && self.policy.is_writable_paths(p, r)
}
fn check_cancelled(&self) -> std::io::Result<()> {
self.policy.check_cancelled()
}
fn resolve_access_path(&self, p: &Path) -> std::io::Result<PathBuf> {
Ok(p.to_path_buf())
}
fn read_metadata(&self, _: &Path, _: &Path) -> std::io::Result<Metadata> {
self.file.metadata()
}
fn open_read_file(&self, _: &Path, _: &Path) -> std::io::Result<File> {
self.confirm()?;
let mut file = self.file.try_clone()?;
file.seek(SeekFrom::Start(0))?;
Ok(file)
}
fn max_write_bytes(&self) -> Option<usize> {
Some(
self.policy
.max_write_bytes()
.unwrap_or(8 * 1024 * 1024)
.min(8 * 1024 * 1024),
)
}
fn max_edit_replacements(&self) -> Option<usize> {
Some(
self.policy
.max_edit_replacements()
.unwrap_or(10_000)
.min(10_000),
)
}
fn atomic_write_file(
&self,
_: &Path,
_: &Path,
content: &[u8],
mode: AtomicWriteMode,
) -> std::io::Result<()> {
self.confirm()?;
let parent = self.parent.as_ref().ok_or_else(|| {
std::io::Error::new(std::io::ErrorKind::InvalidInput, "cannot save root")
})?;
crate::atomic_write_beneath_no_symlinks_at(
parent,
Path::new(self.relative.file_name().unwrap()),
content,
mode,
|| {
self.confirm()?;
// From here rename/encoding/post-observation failures have an unknown outcome.
self.effects_possible.store(true, Ordering::Release);
Ok(())
},
)
}
}
pub struct CheckedCreate<'b, 'a> {
target: &'b CheckedTarget<'a>,
suffix: PathBuf,
logical: PathBuf,
}
impl FsAccessPolicy for CheckedCreate<'_, '_> {
fn is_readable(&self, p: &Path) -> bool {
self.target.policy.is_readable(p)
}
fn is_writable(&self, p: &Path) -> bool {
p == self.logical && self.target.policy.is_writable_paths(p, p)
}
fn resolve_access_path(&self, p: &Path) -> std::io::Result<PathBuf> {
Ok(p.to_path_buf())
}
fn check_cancelled(&self) -> std::io::Result<()> {
self.target.policy.check_cancelled()
}
fn max_write_bytes(&self) -> Option<usize> {
self.target.max_write_bytes()
}
fn read_metadata(&self, _: &Path, _: &Path) -> std::io::Result<Metadata> {
match crate::open_beneath_no_symlinks_at(&self.target.file, &self.suffix) {
Ok(_) => Err(std::io::Error::new(
std::io::ErrorKind::AlreadyExists,
"Create destination exists",
)),
Err(e) => Err(e),
}
}
fn atomic_write_file(
&self,
_: &Path,
_: &Path,
content: &[u8],
mode: AtomicWriteMode,
) -> std::io::Result<()> {
self.target.confirm()?;
if mode != AtomicWriteMode::CreateNew || !self.target.file.metadata()?.is_dir() {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"Create requires directory",
));
}
// Creating parents must not turn a narrow destination grant into
// permission to populate ungranted intermediate directories.
let mut parent = self.target.logical.clone();
let mut components = self.suffix.components().peekable();
while let Some(component) = components.next() {
parent.push(component.as_os_str());
if components.peek().is_some()
&& !self.target.policy.is_writable_paths(&parent, &parent)
{
return Err(std::io::Error::new(
std::io::ErrorKind::PermissionDenied,
"Create parent denied",
));
}
}
// Nested mkdir may have effects before failure; never auto-retry.
self.target.effects_possible.store(true, Ordering::Release);
crate::atomic_write_beneath_no_symlinks_at(
&self.target.file,
&self.suffix,
content,
mode,
|| self.target.policy.check_cancelled(),
)
}
}
#[cfg(all(test, target_os = "linux"))]
mod tests {
use super::*;
use crate::{FsError, FsPath, ReadRequest, WriteRequest};
use sha2::{Digest, Sha256};
use std::sync::atomic::AtomicUsize;
struct Access {
path: PathBuf,
counter: AtomicUsize,
trigger: usize,
replacement: bool,
}
impl FsAccessPolicy for Access {
fn is_readable(&self, _: &Path) -> bool {
true
}
fn is_writable(&self, _: &Path) -> bool {
true
}
fn check_cancelled(&self) -> std::io::Result<()> {
if self.counter.fetch_add(1, Ordering::SeqCst) + 1 == self.trigger {
if self.replacement {
std::fs::rename(&self.path, self.path.with_extension("old"))?;
std::fs::write(&self.path, "replacement")?;
} else {
std::fs::write(&self.path, "updated")?;
}
}
Ok(())
}
}
#[test]
fn checked_save_boundary_rejects_name_substitution_without_mutating_replacement() {
let root = tempfile::tempdir().unwrap();
let path = root.path().join("a");
std::fs::write(&path, "original").unwrap();
let policy = Access {
path: path.clone(),
counter: AtomicUsize::new(0),
trigger: 4,
replacement: true,
};
let handle = crate::open_root_no_symlinks(root.path()).unwrap();
let checked = CheckedTarget::pin(&handle, Path::new("a"), &path, &policy, true).unwrap();
policy.counter.store(0, Ordering::SeqCst);
let error = crate::run_write(
root.path(),
WriteRequest {
path: FsPath::new("a").unwrap(),
content: b"worker".to_vec(),
expected_hash: Some(Sha256::digest(b"original").into()),
},
&checked,
)
.unwrap_err();
assert!(
matches!(error, FsError::Io { source, .. } if source.kind() == std::io::ErrorKind::WouldBlock)
);
assert!(!checked.effects_possible());
assert_eq!(std::fs::read_to_string(&path).unwrap(), "replacement");
assert_eq!(
std::fs::read_to_string(path.with_extension("old")).unwrap(),
"original"
);
}
#[test]
fn checked_read_midread_name_replacement_never_pairs_old_bytes_with_new_validator() {
let root = tempfile::tempdir().unwrap();
let path = root.path().join("a");
std::fs::write(&path, "original").unwrap();
let policy = Access {
path: path.clone(),
counter: AtomicUsize::new(0),
trigger: 3,
replacement: true,
};
let handle = crate::open_root_no_symlinks(root.path()).unwrap();
let checked = CheckedTarget::pin(&handle, Path::new("a"), &path, &policy, false).unwrap();
let validator = checked.validator().to_vec();
policy.counter.store(0, Ordering::SeqCst);
let result = crate::run_read(
root.path(),
ReadRequest {
path: FsPath::new("a").unwrap(),
offset: 0,
limit: 20,
max_bytes: 1024,
},
&checked,
);
match result {
Err(FsError::Conflict(_)) => {}
Ok(read) => {
// Same pinned inode supplies bytes and hash. Execute's final
// named-target check must reject, not refresh to replacement.
assert_eq!(read.bytes, b"original");
let expected_hash: crate::ContentHash = Sha256::digest(b"original").into();
assert_eq!(read.content_hash, expected_hash);
assert_eq!(
checked.confirm().unwrap_err().kind(),
std::io::ErrorKind::WouldBlock
);
}
other => panic!("unexpected Read result: {other:?}"),
}
assert_eq!(checked.validator(), validator);
assert_ne!(
identity_validator(&std::fs::metadata(&path).unwrap()).unwrap(),
validator
);
assert_eq!(std::fs::read_to_string(&path).unwrap(), "replacement");
}
#[test]
fn create_post_parent_observation_retains_bound_directory_and_rejects_substitution() {
let root = tempfile::tempdir().unwrap();
let directory = root.path().join("d");
std::fs::create_dir(&directory).unwrap();
let policy = Access {
path: directory.clone(),
counter: AtomicUsize::new(0),
trigger: usize::MAX,
replacement: false,
};
let handle = crate::open_root_no_symlinks(root.path()).unwrap();
let mut checked =
CheckedTarget::pin(&handle, Path::new("d"), &directory, &policy, true).unwrap();
let initial = checked.validator().to_vec();
let create = checked.create_access(PathBuf::from("new"), directory.join("new"));
crate::run_write(
root.path(),
WriteRequest {
path: FsPath::new("d/new").unwrap(),
content: b"new".to_vec(),
expected_hash: None,
},
&create,
)
.unwrap();
checked.refresh_directory_observation().unwrap();
assert_ne!(initial, checked.validator());
assert_eq!(
checked.validator(),
identity_validator(&std::fs::metadata(&directory).unwrap()).unwrap()
);
std::fs::rename(&directory, root.path().join("old")).unwrap();
std::fs::create_dir(&directory).unwrap();
assert_eq!(
checked.refresh_directory_observation().unwrap_err().kind(),
std::io::ErrorKind::WouldBlock
);
assert!(checked.effects_possible());
}
#[test]
fn checked_parent_swap_is_rejected_even_with_a_hardlinked_same_inode_target() {
let root = tempfile::tempdir().unwrap();
let directory = root.path().join("d");
std::fs::create_dir(&directory).unwrap();
let path = directory.join("a");
std::fs::write(&path, "original").unwrap();
let policy = Access {
path: path.clone(),
counter: AtomicUsize::new(0),
trigger: usize::MAX,
replacement: false,
};
let handle = crate::open_root_no_symlinks(root.path()).unwrap();
let checked = CheckedTarget::pin(&handle, Path::new("d/a"), &path, &policy, true).unwrap();
std::fs::rename(&directory, root.path().join("old")).unwrap();
std::fs::create_dir(&directory).unwrap();
std::fs::hard_link(root.path().join("old/a"), &path).unwrap();
assert_eq!(
checked.confirm().unwrap_err().kind(),
std::io::ErrorKind::WouldBlock
);
assert!(!checked.effects_possible());
}
#[test]
fn shared_text_read_rejects_mid_read_metadata_change() {
let root = tempfile::tempdir().unwrap();
let path = root.path().join("a");
std::fs::write(&path, "original").unwrap();
let policy = Access {
path,
counter: AtomicUsize::new(0),
trigger: 2,
replacement: false,
};
let error = crate::run_read(
root.path(),
ReadRequest {
path: FsPath::new("a").unwrap(),
offset: 0,
limit: 20,
max_bytes: 1024,
},
&policy,
)
.unwrap_err();
assert!(matches!(error, FsError::Conflict(_)));
}
}
@@ -0,0 +1,294 @@
//! Ignore rules loaded only through the provider's authorized descriptor opens.
//! Never use GitignoreBuilder::add/build_global: those reopen host pathnames.
use crate::{FsAccessPolicy, FsError};
use ignore::gitignore::{Gitignore, GitignoreBuilder};
use std::io::{BufRead, BufReader, Read};
use std::path::{Path, PathBuf};
pub(crate) const MAX_IGNORE_SOURCE_BYTES: u64 = 8 * 1024 * 1024;
const MAX_IGNORE_FILE_BYTES: u64 = 1024 * 1024;
const MAX_IGNORE_LINE_BYTES: u64 = 16 * 1024;
pub(crate) struct SourceBoundedReader<'a, R> {
pub inner: R,
pub remaining: &'a mut u64,
pub access: &'a dyn FsAccessPolicy,
}
impl<R: Read> Read for SourceBoundedReader<'_, R> {
fn read(&mut self, buffer: &mut [u8]) -> std::io::Result<usize> {
self.access.check_cancelled()?;
if buffer.is_empty() {
return Ok(0);
}
if *self.remaining == 0 {
let mut probe = [0_u8; 1];
return match self.inner.read(&mut probe)? {
0 => Ok(0),
_ => Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"search source exceeded provider byte limit",
)),
};
}
let limit = usize::try_from(*self.remaining)
.unwrap_or(usize::MAX)
.min(buffer.len());
let read = self.inner.read(&mut buffer[..limit])?;
*self.remaining = (*self.remaining).saturating_sub(read as u64);
Ok(read)
}
}
/// Shared Glob/Grep grammar and provider authorization. Glob, like WalkBuilder,
/// ignores unavailable configuration files; Grep retains its existing explicit
/// error on a rejected ignore-file open. Cancellation/bounds always fail closed.
pub(crate) fn add_checked_ignore_file(
builder: &mut GitignoreBuilder,
path: &Path,
remaining: &mut u64,
access: &dyn FsAccessPolicy,
skip_unavailable: bool,
) -> Result<bool, FsError> {
access.check_cancelled().map_err(|e| FsError::io(path, e))?;
let resolved = match access.resolve_access_path(path) {
Ok(path) => path,
Err(error) if skip_unavailable && error.kind() != std::io::ErrorKind::Interrupted => {
return Ok(false);
}
Err(error) => return Err(FsError::io(path, error)),
};
if !access.is_readable_paths(path, &resolved) {
return Ok(false);
}
let file = match access.open_read_file(path, &resolved) {
Ok(file) => file,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(false),
Err(error) if skip_unavailable && error.kind() != std::io::ErrorKind::Interrupted => {
return Ok(false);
}
Err(error) => return Err(FsError::io(path, error)),
};
let metadata = file.metadata().map_err(|e| FsError::io(path, e))?;
if !metadata.is_file() {
return Ok(false);
}
if metadata.len() > MAX_IGNORE_FILE_BYTES {
return Err(FsError::InvalidArgument(
"ignore file exceeds provider limit 1 MiB".into(),
));
}
let bounded = SourceBoundedReader {
inner: file,
remaining,
access,
};
// Take also fences files that grow after metadata. Each line is bounded
// before allocation, and every underlying read checks cancellation.
let mut reader = BufReader::new(bounded.take(MAX_IGNORE_FILE_BYTES + 1));
let mut line = Vec::new();
let mut bytes = 0_u64;
let mut added = false;
loop {
line.clear();
// BufRead::read_line/read_until retry Interrupted internally. Explicit
// fill_buf prevents a cooperative cancellation from becoming a retry
// loop, and caps allocation before copying even a growing long line.
loop {
access.check_cancelled().map_err(|e| FsError::io(path, e))?;
let buffer = reader.fill_buf().map_err(|e| FsError::io(path, e))?;
if buffer.is_empty() {
break;
}
let count = buffer
.iter()
.position(|byte| *byte == b'\n')
.map_or(buffer.len(), |index| index + 1);
if line.len().saturating_add(count) as u64 > MAX_IGNORE_LINE_BYTES {
return Err(FsError::InvalidArgument(
"ignore line exceeds provider limit 16 KiB".into(),
));
}
let finished = buffer[count - 1] == b'\n';
line.extend_from_slice(&buffer[..count]);
reader.consume(count);
if finished {
break;
}
}
if line.is_empty() {
break;
}
bytes = bytes.saturating_add(line.len() as u64);
if bytes > MAX_IGNORE_FILE_BYTES {
return Err(FsError::InvalidArgument(
"ignore file exceeds provider limit 1 MiB".into(),
));
}
let line = std::str::from_utf8(&line)
.map_err(|_| FsError::InvalidArgument("ignore file must be UTF-8".into()))?
.trim_end_matches(['\r', '\n']);
// One invalid glob must not discard other valid rules.
if builder.add_line(Some(path.to_path_buf()), line).is_ok() {
added = true;
}
}
Ok(added)
}
struct DirectoryRules {
path: PathBuf,
ignore: Gitignore,
git_ignore: Gitignore,
exclude: Gitignore,
has_git: bool,
}
/// Request-local ancestor stack, not a file-tree cache. Only the current
/// directory's ancestry is retained; source bytes and depth bound its matchers.
pub(crate) struct DescriptorIgnores {
directories: Vec<DirectoryRules>,
remaining: u64,
require_git: bool,
skip_unavailable: bool,
}
impl DescriptorIgnores {
pub fn new(root: &Path, base: &Path, access: &dyn FsAccessPolicy) -> Result<Self, FsError> {
Self::with_options(root, base, access, true, true)
}
pub fn grep(root: &Path, base: &Path, access: &dyn FsAccessPolicy) -> Result<Self, FsError> {
// Preserve the existing descriptor Grep provider contract: gitignore
// sources apply without a .git marker and rejected opens fail closed.
Self::with_options(root, base, access, false, false)
}
fn with_options(
root: &Path,
base: &Path,
access: &dyn FsAccessPolicy,
require_git: bool,
skip_unavailable: bool,
) -> Result<Self, FsError> {
let relative = base.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("glob base is outside its provider root".into())
})?;
if relative.components().count() >= 128 {
return Err(FsError::InvalidArgument(
"ignore ancestry exceeds provider depth limit 128".into(),
));
}
let mut this = Self {
directories: Vec::new(),
remaining: MAX_IGNORE_SOURCE_BYTES,
require_git,
skip_unavailable,
};
let mut directory = root.to_path_buf();
for component in relative.components() {
this.load_directory(&directory, access)?;
directory.push(component);
}
Ok(this)
}
pub fn load_directory(
&mut self,
directory: &Path,
access: &dyn FsAccessPolicy,
) -> Result<(), FsError> {
self.directories
.retain(|rules| directory.starts_with(&rules.path));
if self
.directories
.last()
.is_some_and(|rules| rules.path == directory)
{
return Ok(());
}
if self.directories.len() >= 128 {
return Err(FsError::InvalidArgument(
"ignore ancestry exceeds provider depth limit 128".into(),
));
}
access
.check_cancelled()
.map_err(|e| FsError::io(directory, e))?;
let marker = directory.join(".git");
let git_kind = access
.resolve_access_path(&marker)
.ok()
.and_then(|resolved| {
if !access.is_readable_paths(&marker, &resolved) {
return None;
}
access
.read_metadata(&marker, &resolved)
.ok()
.map(|metadata| metadata.file_type())
});
let has_git = git_kind.is_some_and(|kind| kind.is_dir() || kind.is_file());
let mut load = |name: &str| -> Result<Gitignore, FsError> {
let mut builder = GitignoreBuilder::new(directory);
add_checked_ignore_file(
&mut builder,
&directory.join(name),
&mut self.remaining,
access,
self.skip_unavailable,
)?;
builder.build().map_err(|_| {
FsError::InvalidArgument("ignore patterns could not be compiled".into())
})
};
let ignore = load(".ignore")?;
let git_ignore = load(".gitignore")?;
// A linked-worktree .git file activates repository rules but is not a
// directory. Never chase its arbitrary host gitdir/commondir pointers.
let exclude = if git_kind.is_some_and(|kind| kind.is_dir()) {
load(".git/info/exclude")?
} else {
Gitignore::empty()
};
self.directories.push(DirectoryRules {
path: directory.to_path_buf(),
ignore,
git_ignore,
exclude,
has_git,
});
Ok(())
}
pub fn is_ignored(&mut self, path: &Path, is_dir: bool) -> bool {
self.directories
.retain(|rules| path.starts_with(&rules.path));
// .ignore has priority over all .gitignore/exclude rules. Within each
// class the nearest directory's last matching rule wins. Match entries,
// not any parents: ignored directories are pruned by the walker, while
// an explicitly selected ignored search root remains traversable.
for rules in self.directories.iter().rev() {
let matched = rules.ignore.matched(path, is_dir);
if !matched.is_none() {
return matched.is_ignore();
}
}
if self.require_git && !self.directories.iter().any(|rules| rules.has_git) {
return false;
}
let mut exclude = None;
for rules in self.directories.iter().rev() {
let matched = rules.git_ignore.matched(path, is_dir);
if !matched.is_none() {
return matched.is_ignore();
}
if exclude.is_none() {
let matched = rules.exclude.matched(path, is_dir);
if !matched.is_none() {
exclude = Some(matched.is_ignore());
}
}
if rules.has_git {
break;
}
}
exclude.unwrap_or(false)
}
}
+45 -39
View File
@@ -44,34 +44,29 @@ pub fn run_glob(
.as_ref()
.map(|traversal| traversal.path())
.unwrap_or(base);
let mut walker = WalkBuilder::new(walker_root);
walker.hidden(false).follow_links(false);
if traversal.is_some() {
// Descriptor providers must never let WalkBuilder open pathname-based
// ignore files, which may be symlinked outside the pinned root. Glob's
// explicit pattern remains the complete selection contract.
walker
.git_ignore(false)
.git_global(false)
.git_exclude(false)
.ignore(false)
.parents(false);
let search_relative = base.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("glob base is outside its provider root".to_string())
})?;
let search_relative = search_relative.to_path_buf();
let filter_root = walker_root.to_path_buf();
walker.filter_entry(move |entry| {
entry
.path()
.strip_prefix(&filter_root)
.is_ok_and(|relative| {
relative.starts_with(&search_relative) || search_relative.starts_with(relative)
})
});
}
let logical_walk_root = traversal
.as_ref()
.and_then(|t| t.logical_root())
.unwrap_or(root);
let mut descriptor_ignores = traversal
.as_ref()
.map(|_| crate::descriptor_ignore::DescriptorIgnores::new(root, logical_walk_root, access))
.transpose()?;
let mut visited = 0_usize;
for entry in walker.build().flatten() {
let mut walk = if traversal.is_some() {
// Configuration is read via authorized provider opens, never by
// WalkBuilder's host-path ignore discovery.
crate::walk::Walk::descriptor(logical_walk_root, access)
} else {
crate::walk::Walk::Plain(
WalkBuilder::new(walker_root)
.hidden(false)
.follow_links(false)
.build(),
)
};
while let Some(entry) = walk.next() {
let entry = entry?;
access.check_cancelled().map_err(|source| FsError::Io {
path: PathBuf::from(request.path.as_str()),
source,
@@ -83,21 +78,27 @@ pub fn run_glob(
crate::MAX_TRAVERSAL_ENTRIES
)));
}
let walked_path = entry.path();
let path = if traversal.is_some() {
let relative = walked_path.strip_prefix(walker_root).map_err(|_| {
FsError::InvalidArgument(
"descriptor traversal returned a path outside its root".to_string(),
)
})?;
root.join(relative)
} else {
walked_path.to_path_buf()
};
let path = entry.path;
if path.strip_prefix(root).unwrap_or(&path).to_str().is_none() {
continue;
}
let resolved = match access.resolve_access_path(&path) {
Ok(resolved) if access.is_readable_paths(&path, &resolved) => resolved,
_ => continue,
};
if let Some(ignores) = descriptor_ignores.as_mut() {
let is_dir = entry.kind.is_some_and(|kind| kind.is_dir());
if path != logical_walk_root && ignores.is_ignored(&path, is_dir) {
if is_dir {
walk.skip_current_dir();
}
continue;
}
if is_dir {
ignores.load_directory(&path, access)?;
continue;
}
}
let Ok(file) = access.open_read_file(&path, &resolved) else {
continue;
};
@@ -111,7 +112,12 @@ pub fn run_glob(
let logical = path.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("provider returned a path outside its root".to_string())
})?;
let result_path = FsPath::new(logical.to_string_lossy())?;
let Some(logical) = logical.to_str() else {
continue;
};
let Ok(result_path) = FsPath::new(logical) else {
continue;
};
let retained = result_path.as_str().len().saturating_add(16);
if retained_path_bytes.saturating_add(retained) > crate::MAX_RESULT_PATH_BYTES {
provider_truncated = true;
+326
View File
@@ -0,0 +1,326 @@
#![cfg(target_os = "linux")]
use crate::*;
use std::path::{Path, PathBuf};
use std::sync::{
Mutex,
atomic::{AtomicUsize, Ordering},
};
struct Access {
root: PathBuf,
denied: Option<PathBuf>,
opened: Mutex<Vec<PathBuf>>,
enumerated: Mutex<Vec<PathBuf>>,
checks: AtomicUsize,
cancel_at: usize,
}
impl Access {
fn new(root: &Path) -> Self {
Self {
root: root.into(),
denied: None,
opened: Mutex::new(Vec::new()),
enumerated: Mutex::new(Vec::new()),
checks: AtomicUsize::new(0),
cancel_at: usize::MAX,
}
}
}
impl FsAccessPolicy for Access {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.root) && self.denied.as_ref().is_none_or(|denied| path != denied)
}
fn is_writable(&self, _: &Path) -> bool {
false
}
fn resolve_access_path(&self, path: &Path) -> std::io::Result<PathBuf> {
Ok(path.into())
}
fn check_cancelled(&self) -> std::io::Result<()> {
if self.checks.fetch_add(1, Ordering::Relaxed) >= self.cancel_at {
Err(std::io::ErrorKind::Interrupted.into())
} else {
Ok(())
}
}
fn open_read_file(&self, logical: &Path, resolved: &Path) -> std::io::Result<std::fs::File> {
assert!(
self.is_readable(logical),
"denied ignore/source file opened"
);
self.opened.lock().unwrap().push(logical.into());
open_beneath_no_symlinks(&self.root, resolved)
}
fn read_metadata(&self, _: &Path, resolved: &Path) -> std::io::Result<std::fs::Metadata> {
open_beneath_no_symlinks(&self.root, resolved)?.metadata()
}
fn open_traversal_root(
&self,
_: &Path,
resolved: &Path,
) -> std::io::Result<Option<FsTraversalRoot>> {
use std::os::fd::AsRawFd;
let directory = open_beneath_no_symlinks(&self.root, resolved)?;
let walk = PathBuf::from(format!("/proc/self/fd/{}", directory.as_raw_fd()));
Ok(Some(FsTraversalRoot::new_at(
walk,
directory,
resolved.into(),
)))
}
fn open_read_dir(&self, logical: &Path, resolved: &Path) -> std::io::Result<std::fs::ReadDir> {
use std::os::fd::AsRawFd;
self.enumerated.lock().unwrap().push(logical.into());
let directory = open_beneath_no_symlinks(&self.root, resolved)?;
std::fs::read_dir(format!("/proc/self/fd/{}", directory.as_raw_fd()))
}
}
fn glob(root: &Path, access: &dyn FsAccessPolicy) -> Result<GlobResult, FsError> {
run_glob(
root,
root,
GlobRequest {
path: FsPath::root(),
pattern: "**/*.txt".into(),
limit: 100,
},
access,
)
}
#[test]
fn descriptor_glob_prunes_ignored_directories_before_enumeration_and_loading_rules() {
let root = tempfile::tempdir().unwrap();
std::fs::create_dir_all(root.path().join("ignored/deep")).unwrap();
std::fs::write(root.path().join(".ignore"), "ignored/\n").unwrap();
// This rejected directory must never even attempt to load ignore files.
std::fs::File::create(root.path().join("ignored/.ignore"))
.unwrap()
.set_len(2 * 1024 * 1024)
.unwrap();
std::fs::write(root.path().join("ignored/deep/secret.txt"), "secret").unwrap();
std::fs::write(root.path().join("visible.txt"), "visible").unwrap();
let access = Access::new(root.path());
assert_eq!(
glob(root.path(), &access).unwrap().paths,
vec![FsPath::new("visible.txt").unwrap()]
);
assert_eq!(
*access.enumerated.lock().unwrap(),
vec![root.path().to_path_buf()]
);
assert!(
!access
.opened
.lock()
.unwrap()
.iter()
.any(|path| path.starts_with(root.path().join("ignored")))
);
}
#[test]
fn descriptor_glob_denied_and_symlinked_ignore_files_cannot_change_selection() {
let root = tempfile::tempdir().unwrap();
let outside = tempfile::tempdir().unwrap();
std::fs::write(outside.path().join("rules"), "visible.txt\n").unwrap();
std::fs::create_dir(root.path().join(".git")).unwrap();
std::os::unix::fs::symlink(outside.path().join("rules"), root.path().join(".gitignore"))
.unwrap();
std::fs::write(root.path().join(".ignore"), "visible.txt\n").unwrap();
std::fs::write(root.path().join("visible.txt"), "visible").unwrap();
let mut access = Access::new(root.path());
access.denied = Some(root.path().join(".ignore"));
assert_eq!(
glob(root.path(), &access).unwrap().paths,
vec![FsPath::new("visible.txt").unwrap()]
);
assert!(
!access
.opened
.lock()
.unwrap()
.contains(&root.path().join(".ignore"))
);
}
#[test]
fn descriptor_ignore_loading_checks_cancellation_without_retrying_interrupted() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join(".ignore"), "ignored.txt\n".repeat(1000)).unwrap();
let mut access = Access::new(root.path());
// First check is before resolution, second at the first fill_buf.
access.cancel_at = 1;
let error = crate::descriptor_ignore::add_checked_ignore_file(
&mut ignore::gitignore::GitignoreBuilder::new(root.path()),
&root.path().join(".ignore"),
&mut 1024_000,
&access,
true,
)
.unwrap_err();
assert!(
matches!(error, FsError::Io { source, .. } if source.kind() == std::io::ErrorKind::Interrupted)
);
assert_eq!(access.checks.load(Ordering::Relaxed), 2);
}
#[test]
fn descriptor_ignore_loading_bounds_file_line_and_aggregate_bytes() {
let root = tempfile::tempdir().unwrap();
let path = root.path().join(".ignore");
let access = Access::new(root.path());
std::fs::File::create(&path)
.unwrap()
.set_len(1024 * 1024 + 1)
.unwrap();
assert!(matches!(
glob(root.path(), &access),
Err(FsError::InvalidArgument(_))
));
std::fs::write(&path, "x".repeat(16 * 1024 + 1)).unwrap();
assert!(matches!(
glob(root.path(), &access),
Err(FsError::InvalidArgument(_))
));
std::fs::write(&path, "ignored.txt\n").unwrap();
let error = crate::descriptor_ignore::add_checked_ignore_file(
&mut ignore::gitignore::GitignoreBuilder::new(root.path()),
&path,
&mut 3,
&access,
true,
)
.unwrap_err();
assert!(
matches!(error, FsError::Io { source, .. } if source.kind() == std::io::ErrorKind::InvalidData)
);
}
#[test]
fn descriptor_ignore_ancestry_is_bounded_without_enumerating_a_tree() {
let root = tempfile::tempdir().unwrap();
let access = Access::new(root.path());
let base = (0..128).fold(root.path().to_path_buf(), |path, _| path.join("d"));
assert!(matches!(
crate::descriptor_ignore::DescriptorIgnores::new(root.path(), &base, &access),
Err(FsError::InvalidArgument(_))
));
assert!(access.opened.lock().unwrap().is_empty());
}
struct Plain(PathBuf);
impl FsAccessPolicy for Plain {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.0)
}
fn is_writable(&self, _: &Path) -> bool {
false
}
}
#[test]
fn descriptor_glob_matches_plain_git_markers_exclude_and_nested_repository_boundaries() {
let root = tempfile::tempdir().unwrap();
std::fs::create_dir_all(root.path().join("nested/.git/info")).unwrap();
std::fs::create_dir_all(root.path().join(".git/info")).unwrap();
std::fs::write(
root.path().join(".gitignore"),
"*.outer.txt\n!include.txt\n",
)
.unwrap();
std::fs::write(root.path().join(".ignore"), "always.txt\n").unwrap();
std::fs::write(
root.path().join(".git/info/exclude"),
"excluded.txt\ninclude.txt\n",
)
.unwrap();
std::fs::write(
root.path().join("nested/.git/info/exclude"),
"/excluded.txt\n",
)
.unwrap();
for name in [
"a.outer.txt",
"excluded.txt",
"include.txt",
"always.txt",
"nested/a.outer.txt",
"nested/excluded.txt",
"nested/always.txt",
] {
std::fs::write(root.path().join(name), "data").unwrap();
}
let native = glob(root.path(), &Access::new(root.path())).unwrap();
let ordinary = glob(root.path(), &Plain(root.path().into())).unwrap();
assert_eq!(native, ordinary);
assert_eq!(
native.paths,
vec![
FsPath::new("include.txt").unwrap(),
FsPath::new("nested/a.outer.txt").unwrap()
]
);
}
#[test]
fn descriptor_glob_requires_authorized_git_marker_for_gitignore_not_for_ignore() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join(".gitignore"), "gitignored.txt\n").unwrap();
std::fs::write(root.path().join(".ignore"), "ignored.txt\n").unwrap();
for name in ["gitignored.txt", "ignored.txt", "visible.txt"] {
std::fs::write(root.path().join(name), "data").unwrap();
}
let native = glob(root.path(), &Access::new(root.path())).unwrap();
// Ambient ancestors may themselves be repositories (e.g. the harness's
// TMPDIR). Their markers/ignore files are outside this provider root and
// cannot activate Git rules or be inspected by descriptor search.
assert_eq!(native.paths.len(), 2);
std::fs::create_dir(root.path().join(".git")).unwrap();
let native = glob(root.path(), &Access::new(root.path())).unwrap();
assert_eq!(
native,
glob(root.path(), &Plain(root.path().into())).unwrap()
);
assert_eq!(native.paths, vec![FsPath::new("visible.txt").unwrap()]);
let mut denied = Access::new(root.path());
denied.denied = Some(root.path().join(".git"));
assert_eq!(glob(root.path(), &denied).unwrap().paths.len(), 2);
std::fs::remove_dir(root.path().join(".git")).unwrap();
let outside = tempfile::tempdir().unwrap();
std::fs::write(outside.path().join("exclude"), "visible.txt\n").unwrap();
std::fs::write(
root.path().join(".git"),
format!("gitdir: {}\n", outside.path().display()),
)
.unwrap();
// Real managed Workdirs use a .git file. Activate root .gitignore without
// failing on .git/info/exclude or following external administration paths.
let access = Access::new(root.path());
assert_eq!(
glob(root.path(), &access).unwrap().paths,
vec![FsPath::new("visible.txt").unwrap()]
);
std::fs::write(root.path().join("visible.txt"), "needle\n").unwrap();
let grep = run_grep(
root.path(),
root.path().into(),
GrepRequest {
path: FsPath::root(),
pattern: "needle".into(),
glob: None,
file_type: None,
case_insensitive: false,
before_context: 0,
after_context: 0,
multiline: false,
output_mode: GrepOutputMode::FilesWithMatches,
offset: 0,
limit: 100,
},
&access,
)
.unwrap();
assert_eq!(grep.paths, vec![FsPath::new("visible.txt").unwrap()]);
assert!(
!access
.opened
.lock()
.unwrap()
.iter()
.any(|path| path.starts_with(outside.path()))
);
}
+34
View File
@@ -4,10 +4,19 @@
//! transport, or LLM Tool implementations. Paths are logical and root-relative;
//! providers supply host roots and access policy.
mod checked;
pub use checked::{CheckedTarget, identity_validator};
mod descriptor_ignore;
mod glob;
#[cfg(test)]
mod ignore_tests;
mod local;
mod operation;
mod search;
pub mod text;
#[cfg(test)]
mod traversal_tests;
mod walk;
use std::path::{Path, PathBuf};
@@ -272,6 +281,7 @@ pub fn atomic_write_beneath_no_symlinks_at(
#[derive(Debug)]
pub struct FsTraversalRoot {
path: PathBuf,
logical_root: Option<PathBuf>,
_directory: std::fs::File,
}
@@ -279,20 +289,44 @@ impl FsTraversalRoot {
pub fn new(path: PathBuf, directory: std::fs::File) -> Self {
Self {
path,
logical_root: None,
_directory: directory,
}
}
/// Map walked paths to this logical anchor rather than the provider root.
pub fn new_at(path: PathBuf, directory: std::fs::File, logical_root: PathBuf) -> Self {
Self {
path,
logical_root: Some(logical_root),
_directory: directory,
}
}
pub fn logical_root(&self) -> Option<&Path> {
self.logical_root.as_deref()
}
pub fn path(&self) -> &Path {
&self.path
}
}
/// Owned predicate used before walker descent. Paths are provider-logical;
/// true directories require enumeration authority, other entries require read.
pub type FsTraversalFilter = std::sync::Arc<dyn Fn(&Path, bool) -> bool + Send + Sync>;
/// Provider-owned access policy used by local filesystem operations.
pub trait FsAccessPolicy: Send + Sync {
fn is_readable(&self, path: &Path) -> bool;
fn is_writable(&self, path: &Path) -> bool;
fn can_enumerate_directory(&self, logical: &Path, resolved: &Path) -> bool {
self.is_readable_paths(logical, resolved)
}
/// Owned because the shared walker requires a 'static entry predicate.
fn traversal_filter(&self) -> Option<FsTraversalFilter> {
None
}
/// Cooperatively stop provider-owned filesystem work. Implementations may
/// use this for per-operation cancellation and deadlines.
fn check_cancelled(&self) -> std::io::Result<()> {
+285 -66
View File
@@ -162,6 +162,19 @@ fn run_read_with_limits(
}
}
#[cfg(unix)]
if crate::identity_validator(&metadata).map_err(|e| map_io(&logical, e))?
!= crate::identity_validator(
&reader
.get_ref()
.metadata()
.map_err(|e| map_io(&logical, e))?,
)
.map_err(|e| map_io(&logical, e))?
{
return Err(FsError::Conflict(logical.as_str().to_string()));
}
let total_lines =
current_line.saturating_add(usize::from(last_byte.is_some_and(|byte| byte != b'\n')));
if request.offset > total_lines && request.offset != 0 {
@@ -176,15 +189,7 @@ fn run_read_with_limits(
.min(total_lines);
let byte_truncated = selected_bytes_seen > response_limit;
if byte_truncated {
let mut byte_end = response_limit.min(selected.len());
while byte_end > 0 {
match std::str::from_utf8(&selected[..byte_end]) {
Ok(_) => break,
Err(error) if error.error_len().is_none() => byte_end -= 1,
Err(_) => break,
}
}
selected.truncate(byte_end);
selected.truncate(crate::text::bounded_utf8_len(&selected, response_limit));
}
Ok(ReadResult {
path: logical,
@@ -402,55 +407,33 @@ pub fn run_edit(
let content = String::from_utf8(bytes).map_err(|_| {
FsError::InvalidArgument(format!("{} is not valid UTF-8", logical.as_str()))
})?;
let occurrences = content.matches(&request.old_string).count();
if occurrences == 0 {
return Err(FsError::InvalidArgument(
"old_string was not found".to_string(),
));
}
if !request.replace_all && occurrences != 1 {
return Err(FsError::InvalidArgument(format!(
"old_string matched {occurrences} times; set replace_all=true or provide a unique string"
)));
}
let replacement_count = if request.replace_all { occurrences } else { 1 };
if let Some(limit) = access.max_edit_replacements()
&& replacement_count > limit
{
return Err(FsError::InvalidArgument(format!(
"{} exceeds provider Edit replacement limit {limit}",
logical.as_str()
)));
}
let removed_bytes = request
.old_string
.len()
.checked_mul(replacement_count)
.ok_or_else(|| FsError::InvalidArgument("edited content size overflowed".to_string()))?;
let added_bytes = request
.new_string
.len()
.checked_mul(replacement_count)
.ok_or_else(|| FsError::InvalidArgument("edited content size overflowed".to_string()))?;
let edited_len = content
.len()
.checked_sub(removed_bytes)
.and_then(|size| size.checked_add(added_bytes))
.ok_or_else(|| FsError::InvalidArgument("edited content size overflowed".to_string()))?;
enforce_write_bound(&logical, edited_len, access)?;
let edited = if request.replace_all {
content.replace(&request.old_string, &request.new_string)
} else {
content.replacen(&request.old_string, &request.new_string, 1)
let args = crate::text::EditArgs {
old_string: request.old_string,
new_string: request.new_string,
replace_all: request.replace_all,
};
debug_assert_eq!(edited.len(), edited_len);
let edited = crate::text::edit(
&content,
&args,
crate::text::TextLimits {
max_input_bytes: access.max_write_bytes(),
max_output_bytes: access.max_write_bytes(),
max_replacements: access.max_edit_replacements(),
},
)
.map_err(|error| map_text_error(&logical, error))?;
access
.atomic_write_file(&path, &target, edited.as_bytes(), AtomicWriteMode::Replace)
.atomic_write_file(
&path,
&target,
edited.content.as_bytes(),
AtomicWriteMode::Replace,
)
.map_err(|error| map_io(&logical, error))?;
Ok(EditResult {
replacements: if request.replace_all { occurrences } else { 1 },
bytes_written: edited.len(),
content_hash: hash_bytes(edited.as_bytes()),
replacements: edited.replacements,
bytes_written: edited.bytes_written,
content_hash: hash_bytes(edited.content.as_bytes()),
})
}
@@ -473,6 +456,14 @@ pub fn run_list(
if !metadata.is_dir() {
return Err(FsError::NotDirectory(PathBuf::from(logical.as_str())));
}
if !access.can_enumerate_directory(&logical_base, &path) {
return Ok(ListResult {
entries: Vec::new(),
total_entries: 0,
total_bytes: 0,
truncated: false,
});
}
let mut entries = Vec::new();
let mut retained_path_bytes = 0_usize;
let mut provider_truncated = false;
@@ -490,6 +481,9 @@ pub fn run_list(
)));
}
let entry = entry.map_err(|error| map_io(&logical, error))?;
if entry.file_name().to_str().is_none() {
continue;
}
let logical_absolute = logical_base.join(entry.file_name());
let resolved = match access.resolve_access_path(&logical_absolute) {
Ok(resolved) => resolved,
@@ -513,7 +507,12 @@ pub fn run_list(
let relative = logical_absolute.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("provider returned a path outside its root".to_string())
})?;
let result_path = FsPath::new(relative.to_string_lossy())?;
let Some(relative) = relative.to_str() else {
continue;
};
let Ok(result_path) = FsPath::new(relative) else {
continue;
};
let retained = result_path.as_str().len().saturating_add(64);
if retained_path_bytes.saturating_add(retained) > crate::MAX_RESULT_PATH_BYTES {
provider_truncated = true;
@@ -550,15 +549,37 @@ fn enforce_write_bound(
bytes: usize,
access: &dyn FsAccessPolicy,
) -> Result<(), FsError> {
if let Some(limit) = access.max_write_bytes()
&& bytes > limit
{
return Err(FsError::InvalidArgument(format!(
"{} exceeds provider write limit {limit}",
crate::text::check_output_bytes(
bytes,
crate::text::TextLimits {
max_output_bytes: access.max_write_bytes(),
..Default::default()
},
)
.map_err(|error| map_text_error(logical, error))
}
fn map_text_error(logical: &FsPath, error: crate::text::TextError) -> FsError {
use crate::text::TextError;
let message = match error {
TextError::InputTooLarge { limit, .. } | TextError::OutputTooLarge { limit, .. } => {
format!("{} exceeds provider write limit {limit}", logical.as_str())
}
TextError::ReplacementLimitExceeded { limit, .. } => format!(
"{} exceeds provider Edit replacement limit {limit}",
logical.as_str()
)));
}
Ok(())
),
TextError::Decode(error) => format!("invalid arguments: {error}"),
TextError::EmptyOldString => "old_string must not be empty".to_string(),
TextError::IdenticalStrings => "old_string and new_string are identical".to_string(),
TextError::NotFound => "old_string was not found".to_string(),
TextError::MultipleMatches { occurrences } => format!(
"old_string matched {occurrences} times; set replace_all=true or provide a unique string"
),
TextError::Overflow => "edited content size overflowed".to_string(),
};
FsError::InvalidArgument(message)
}
fn read_mutation_preimage(
@@ -610,9 +631,21 @@ fn require_access(
write: bool,
allow_symlink_directory: bool,
) -> Result<PathBuf, FsError> {
let resolved = access
.resolve_access_path(path)
.map_err(|error| map_io(logical, error))?;
let resolved = access.resolve_access_path(path).map_err(|error| {
// A dangling direct link has no resolved identity, but still needs the
// existing sanitized repair diagnostic rather than an ordinary missing-file error.
if error.kind() == std::io::ErrorKind::NotFound
&& direct_symlink(path).is_some_and(|info| !info.target_exists)
{
FsError::BrokenSymlink {
path: PathBuf::from(logical.as_str()),
link: PathBuf::from(logical.as_str()),
target: PathBuf::from("<provider-internal target>"),
}
} else {
map_io(logical, error)
}
})?;
let symlink = (resolved != path).then(|| direct_symlink(path)).flatten();
if let Some(info) = symlink.as_ref()
&& !info.target_exists
@@ -773,3 +806,189 @@ mod bounded_read_tests {
assert!(result.truncated);
}
}
#[cfg(test)]
mod text_policy_tests {
use super::*;
struct Access {
root: PathBuf,
bytes: Option<usize>,
replacements: Option<usize>,
}
impl FsAccessPolicy for Access {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.root)
}
fn is_writable(&self, path: &Path) -> bool {
path.starts_with(&self.root)
}
fn max_write_bytes(&self) -> Option<usize> {
self.bytes
}
fn max_edit_replacements(&self) -> Option<usize> {
self.replacements
}
}
#[test]
fn edit_policy_errors_preserve_the_preimage_and_provider_messages() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
let source = "a a";
fs::write(root.join("text.txt"), source).unwrap();
let access = Access {
root: root.clone(),
bytes: Some(4),
replacements: Some(1),
};
for (old, new, all, expected_message) in [
("", "x", false, "old_string must not be empty"),
("a", "a", false, "old_string and new_string are identical"),
(
"missing",
"x",
false,
"text.txt exceeds provider write limit 4",
),
("b", "x", false, "old_string was not found"),
(
"a",
"x",
false,
"old_string matched 2 times; set replace_all=true or provide a unique string",
),
(
"a",
"x",
true,
"text.txt exceeds provider Edit replacement limit 1",
),
(
"a a",
"😀x",
false,
"text.txt exceeds provider write limit 4",
),
] {
let error = run_edit(
&root,
EditRequest {
path: FsPath::new("text.txt").unwrap(),
old_string: old.into(),
new_string: new.into(),
replace_all: all,
expected_hash: hash_bytes(source.as_bytes()),
},
&access,
)
.unwrap_err();
assert!(
matches!(error, FsError::InvalidArgument(message) if message == expected_message)
);
assert_eq!(fs::read_to_string(root.join("text.txt")).unwrap(), source);
}
let access = Access {
replacements: None,
..access
};
let error = run_edit(
&root,
EditRequest {
path: FsPath::new("text.txt").unwrap(),
old_string: "a".into(),
new_string: "xxx".into(),
replace_all: true,
expected_hash: hash_bytes(source.as_bytes()),
},
&access,
)
.unwrap_err();
assert!(
matches!(error, FsError::InvalidArgument(message) if message == "text.txt exceeds provider write limit 4")
);
assert_eq!(fs::read_to_string(root.join("text.txt")).unwrap(), source);
}
#[test]
fn edit_keeps_cas_before_content_policy_and_hashes_the_core_result() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
fs::write(root.join("text.txt"), "é😀").unwrap();
let access = Access {
root: root.clone(),
bytes: None,
replacements: None,
};
let error = run_edit(
&root,
EditRequest {
path: FsPath::new("text.txt").unwrap(),
old_string: String::new(),
new_string: "x".into(),
replace_all: false,
expected_hash: [0; 32],
},
&access,
)
.unwrap_err();
assert!(matches!(error, FsError::Conflict(_)));
let result = run_edit(
&root,
EditRequest {
path: FsPath::new("text.txt").unwrap(),
old_string: "é".into(),
new_string: "界".into(),
replace_all: false,
expected_hash: hash_bytes("é😀".as_bytes()),
},
&access,
)
.unwrap();
assert_eq!(result.replacements, 1);
assert_eq!(result.bytes_written, 7);
assert_eq!(result.content_hash, hash_bytes("界😀".as_bytes()));
assert_eq!(fs::read_to_string(root.join("text.txt")).unwrap(), "界😀");
}
#[test]
fn local_write_remains_binary_safe_and_maps_shared_bounds() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
let access = Access {
root: root.clone(),
bytes: Some(2),
replacements: None,
};
let result = run_write(
&root,
WriteRequest {
path: FsPath::new("binary").unwrap(),
content: vec![0xff, 0],
expected_hash: None,
},
&access,
)
.unwrap();
assert_eq!(result.bytes_written, 2);
assert_eq!(fs::read(root.join("binary")).unwrap(), vec![0xff, 0]);
let error = run_write(
&root,
WriteRequest {
path: FsPath::new("large").unwrap(),
content: vec![0xff, 0, 1],
expected_hash: None,
},
&access,
)
.unwrap_err();
assert!(
matches!(error, FsError::InvalidArgument(message) if message == "large exceeds provider write limit 2")
);
assert!(!root.join("large").exists());
}
}
+4
View File
@@ -353,6 +353,10 @@ pub struct GrepRequest {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct GrepResult {
/// Exact typed provider paths represented in this returned report, not all
/// candidate/matched source files. Older providers may omit this field.
#[serde(default)]
pub paths: Vec<FsPath>,
/// Provider-rendered bounded grep report. Keeping rendering here avoids
/// transferring candidate files across a remote provider boundary.
pub output: String,
+87 -216
View File
@@ -1,50 +1,20 @@
use std::collections::{BTreeMap, HashSet};
use std::collections::BTreeMap;
use std::fmt::Write as _;
use std::io::{BufRead, BufReader, Read};
use std::io::Read;
use std::path::{Path, PathBuf};
use crate::descriptor_ignore::{DescriptorIgnores, SourceBoundedReader};
use crate::FsAccessPolicy;
use grep_regex::RegexMatcherBuilder;
use grep_searcher::sinks::UTF8 as UTF8Sink;
use grep_searcher::{BinaryDetection, Searcher, SearcherBuilder, Sink, SinkContext, SinkMatch};
use ignore::WalkBuilder;
use ignore::gitignore::{Gitignore, GitignoreBuilder};
use ignore::overrides::{Override, OverrideBuilder};
use ignore::types::{Types, TypesBuilder};
use crate::{FsError, GrepOutputMode, GrepRequest, GrepResult, direct_symlink};
struct SourceBoundedReader<'a, R> {
inner: R,
remaining: &'a mut u64,
access: &'a dyn FsAccessPolicy,
}
impl<R: Read> Read for SourceBoundedReader<'_, R> {
fn read(&mut self, buffer: &mut [u8]) -> std::io::Result<usize> {
self.access.check_cancelled()?;
if buffer.is_empty() {
return Ok(0);
}
if *self.remaining == 0 {
let mut probe = [0_u8; 1];
return match self.inner.read(&mut probe)? {
0 => Ok(0),
_ => Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"grep source exceeded provider byte limit",
)),
};
}
let limit = usize::try_from(*self.remaining)
.unwrap_or(usize::MAX)
.min(buffer.len());
let read = self.inner.read(&mut buffer[..limit])?;
*self.remaining = (*self.remaining).saturating_sub(read as u64);
Ok(read)
}
}
struct ContentLine {
path: PathBuf,
line_number: Option<u64>,
@@ -79,6 +49,28 @@ fn reserve_report_bytes(report: &mut GrepReport, bytes: usize) -> bool {
impl GrepReport {
fn into_result(self, root: &Path) -> GrepResult {
// Build Object paths from retained provider identities, never by parsing
// rendered text. Offset, limit and truncation already selected this set.
let paths = match self.mode {
GrepOutputMode::FilesWithMatches => self
.files
.iter()
.filter_map(|path| logical_path(root, path))
.collect(),
GrepOutputMode::Count => self
.counts
.iter()
.filter_map(|(path, _)| logical_path(root, path))
.collect(),
GrepOutputMode::Content => self
.lines
.iter()
.map(|line| line.path.as_path())
.collect::<std::collections::BTreeSet<_>>()
.into_iter()
.filter_map(|path| logical_path(root, path))
.collect(),
};
let (match_count, matched_files) = match self.mode {
GrepOutputMode::FilesWithMatches => (self.files.len(), self.files.len()),
GrepOutputMode::Count => (
@@ -116,6 +108,7 @@ impl GrepReport {
}
}
GrepResult {
paths,
output,
match_count,
matched_files,
@@ -166,11 +159,15 @@ fn render_content_lines(root: &Path, lines: &[ContentLine], show_line_numbers: b
output
}
fn logical_path(root: &Path, path: &Path) -> Option<crate::FsPath> {
crate::FsPath::new_scoped(path.strip_prefix(root).unwrap_or(path).to_str()?).ok()
}
fn logical_display(root: &Path, path: &Path) -> String {
path.strip_prefix(root)
.unwrap_or(path)
.to_string_lossy()
.replace('\\', "/")
logical_path(root, path)
.expect("only representable provider paths enter a report")
.as_str()
.to_string()
}
const DEFAULT_HEAD_LIMIT: usize = 250;
@@ -207,119 +204,6 @@ fn direct_file_selected(path: &Path, overrides: Option<&Override>, types: Option
&& !types.is_some_and(|filter| filter.matched(path, false).is_ignore())
}
#[derive(Default)]
struct DescriptorIgnoreMatchers {
loaded_directories: HashSet<PathBuf>,
by_directory: BTreeMap<PathBuf, Gitignore>,
}
impl DescriptorIgnoreMatchers {
fn load_directory(
&mut self,
directory: &Path,
explicit_base: &Path,
source_bytes_remaining: &mut u64,
access: &dyn FsAccessPolicy,
) -> Result<(), FsError> {
if !self.loaded_directories.insert(directory.to_path_buf()) {
return Ok(());
}
let mut builder = GitignoreBuilder::new(directory);
let mut has_patterns = false;
for name in [".gitignore", ".ignore"] {
let path = directory.join(name);
let resolved = access
.resolve_access_path(&path)
.map_err(|error| FsError::io(&path, error))?;
if !access.is_readable_paths(&path, &resolved) {
continue;
}
let file = match access.open_read_file(&path, &resolved) {
Ok(file) => file,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue,
Err(error) => return Err(FsError::io(&path, error)),
};
if !file
.metadata()
.map_err(|error| FsError::io(&path, error))?
.is_file()
{
continue;
}
let bounded = SourceBoundedReader {
inner: file,
remaining: source_bytes_remaining,
access,
};
let mut reader = BufReader::new(bounded);
let mut line = String::new();
let mut line_number = 0_u64;
loop {
line.clear();
let read = reader
.read_line(&mut line)
.map_err(|error| FsError::io(&path, error))?;
if read == 0 {
break;
}
line_number = line_number.saturating_add(1);
if line.len() > 1024 * 1024 {
return Err(FsError::InvalidArgument(format!(
"ignore pattern line {line_number} exceeds provider limit"
)));
}
let line = line.trim_end_matches(['\r', '\n']);
// Match the ignore crate's partial-error behavior: one invalid
// pattern does not discard the remaining valid lines.
if builder.add_line(Some(path.clone()), line).is_ok() {
has_patterns = true;
}
}
}
if !has_patterns {
return Ok(());
}
let matcher = builder.build().map_err(|_| {
FsError::InvalidArgument("ignore patterns could not be compiled".to_string())
})?;
// An explicitly selected search root is traversed even when a strict
// ancestor ignores that directory, matching path-backed Grep behavior.
if directory != explicit_base
&& explicit_base.starts_with(directory)
&& matcher
.matched_path_or_any_parents(explicit_base, true)
.is_ignore()
{
return Ok(());
}
self.by_directory.insert(directory.to_path_buf(), matcher);
Ok(())
}
fn is_ignored(&self, root: &Path, path: &Path) -> bool {
let mut directories = path
.parent()
.into_iter()
.flat_map(Path::ancestors)
.take_while(|directory| directory.starts_with(root))
.collect::<Vec<_>>();
directories.reverse();
let mut ignored = false;
for directory in directories {
let Some(matcher) = self.by_directory.get(directory) else {
continue;
};
let matched = matcher.matched_path_or_any_parents(path, false);
if matched.is_ignore() {
ignored = true;
} else if matched.is_whitelist() {
ignored = false;
}
}
ignored
}
}
struct GrepParams {
pattern: String,
path: Option<PathBuf>,
@@ -464,6 +348,9 @@ pub fn run_grep(
retained_bytes: 0,
truncated: false,
};
if head_limit == 0 {
return Ok(report.into_result(root));
}
let mut matching_files_seen = 0;
let mut matches_seen = 0;
@@ -474,6 +361,7 @@ pub fn run_grep(
&mut searcher,
&matcher,
&base,
root,
mode,
&mut report,
&mut matching_files_seen,
@@ -494,6 +382,10 @@ pub fn run_grep(
.as_ref()
.map(|traversal| traversal.path())
.unwrap_or(&base);
let logical_walk_root = traversal
.as_ref()
.and_then(|t| t.logical_root())
.unwrap_or(root);
let mut walker = WalkBuilder::new(walker_root);
if traversal.is_some() {
walker
@@ -506,19 +398,6 @@ pub fn run_grep(
.ignore(false)
.parents(false)
.follow_links(false);
let search_relative = base.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("grep base is outside its provider root".to_string())
})?;
let search_relative = search_relative.to_path_buf();
let filter_root = walker_root.to_path_buf();
walker.filter_entry(move |entry| {
entry
.path()
.strip_prefix(&filter_root)
.is_ok_and(|relative| {
relative.starts_with(&search_relative) || search_relative.starts_with(relative)
})
});
} else {
walker
.hidden(true)
@@ -538,32 +417,17 @@ pub fn run_grep(
let mut visited = 0_usize;
let mut source_bytes_remaining = crate::MAX_GREP_SOURCE_BYTES;
let mut descriptor_ignores = DescriptorIgnoreMatchers::default();
if traversal.is_some() {
let mut directory = root.to_path_buf();
descriptor_ignores.load_directory(
&directory,
&base,
&mut source_bytes_remaining,
access,
)?;
for component in base
.strip_prefix(root)
.map_err(|_| {
FsError::InvalidArgument("grep base is outside its provider root".to_string())
})?
.components()
{
directory.push(component);
descriptor_ignores.load_directory(
&directory,
&base,
&mut source_bytes_remaining,
access,
)?;
}
}
for entry in walker.build().flatten() {
let mut descriptor_ignores = traversal
.as_ref()
.map(|_| DescriptorIgnores::grep(root, logical_walk_root, access))
.transpose()?;
let mut walk = if traversal.is_some() {
crate::walk::Walk::descriptor(logical_walk_root, access)
} else {
crate::walk::Walk::Plain(walker.build())
};
while let Some(entry) = walk.next() {
let entry = entry?;
access
.check_cancelled()
.map_err(|error| FsError::io(&base, error))?;
@@ -574,39 +438,41 @@ pub fn run_grep(
crate::MAX_TRAVERSAL_ENTRIES
)));
}
let walked_path = entry.path();
let path = if traversal.is_some() {
let relative = walked_path.strip_prefix(walker_root).map_err(|_| {
FsError::InvalidArgument(
"descriptor traversal returned a path outside its root".to_string(),
)
})?;
root.join(relative)
} else {
walked_path.to_path_buf()
};
let file_type = entry.file_type();
if traversal.is_some() && file_type.as_ref().is_some_and(|kind| kind.is_dir()) {
descriptor_ignores.load_directory(&path, &base, &mut source_bytes_remaining, access)?;
let path = entry.path;
if logical_path(root, &path).is_none() {
continue;
}
if !file_type.map(|kind| kind.is_file()).unwrap_or(false) {
continue;
}
if traversal.is_some() {
let file_type = entry.kind;
if let Some(ignores) = descriptor_ignores.as_mut() {
let relative = path.strip_prefix(&base).map_err(|_| {
FsError::InvalidArgument(
"descriptor traversal returned a path outside its search base".to_string(),
)
})?;
if relative
.components()
.any(|component| component.as_os_str().to_string_lossy().starts_with('.'))
|| descriptor_ignores.is_ignored(root, &path)
|| !direct_file_selected(&path, overrides.as_ref(), types.as_ref())
let is_dir = file_type.is_some_and(|kind| kind.is_dir());
if path != logical_walk_root
&& (relative.components().any(|component| {
component
.as_os_str()
.to_str()
.is_some_and(|name| name.starts_with('.'))
}) || ignores.is_ignored(&path, is_dir))
{
if is_dir {
walk.skip_current_dir();
}
continue;
}
if is_dir {
ignores.load_directory(&path, access)?;
continue;
}
if !direct_file_selected(&path, overrides.as_ref(), types.as_ref()) {
continue;
}
}
if !file_type.map(|kind| kind.is_file()).unwrap_or(false) {
continue;
}
let readable = access
.resolve_access_path(&path)
@@ -618,6 +484,7 @@ pub fn run_grep(
&mut searcher,
&matcher,
&path,
root,
mode,
&mut report,
&mut matching_files_seen,
@@ -639,6 +506,7 @@ fn scan_path(
searcher: &mut Searcher,
matcher: &grep_regex::RegexMatcher,
path: &Path,
root: &Path,
mode: GrepOutputMode,
report: &mut GrepReport,
matching_files_seen: &mut usize,
@@ -648,6 +516,9 @@ fn scan_path(
source_bytes_remaining: &mut u64,
access: &dyn FsAccessPolicy,
) -> Result<bool, FsError> {
if logical_path(root, path).is_none() {
return Ok(false);
}
let resolved = access
.resolve_access_path(path)
.map_err(|error| FsError::io(path, error))?;
@@ -678,7 +549,7 @@ fn scan_path(
return Ok(false);
}
if *matching_files_seen >= offset {
if !reserve_report_bytes(report, path.to_string_lossy().len().saturating_add(1)) {
if !reserve_report_bytes(report, path.as_os_str().len().saturating_add(1)) {
return Ok(true);
}
report.files.push(path.to_path_buf());
@@ -695,7 +566,7 @@ fn scan_path(
return Ok(false);
}
if *matching_files_seen >= offset {
if !reserve_report_bytes(report, path.to_string_lossy().len().saturating_add(32)) {
if !reserve_report_bytes(report, path.as_os_str().len().saturating_add(32)) {
return Ok(true);
}
report.counts.push((path.to_path_buf(), count));
@@ -780,7 +651,7 @@ impl ContentSink<'_> {
fn reserve(&mut self, content_bytes: usize) -> bool {
let retained = self
.path
.to_string_lossy()
.as_os_str()
.len()
.saturating_add(content_bytes)
.saturating_add(64);
+900
View File
@@ -0,0 +1,900 @@
//! Pure text-operation policy shared by filesystem providers and operation frontends.
//!
//! This module owns no paths, descriptors, version fences, or I/O. Limits count
//! UTF-8 bytes, not characters, and default to unlimited.
use schemars::JsonSchema;
use serde::{Deserialize, de::DeserializeOwned};
use thiserror::Error;
/// The shared text Operation definitions, independent of paths and frontends.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TextOperation {
Read,
ReadLines,
Edit,
Write,
}
impl TextOperation {
/// Resolve a lower-case Operation name. A frontend chooses whether its
/// `read` Operation accepts whole-text or line-range arguments.
pub fn from_name(name: &str, line_read: bool) -> Option<Self> {
match name {
"read" if line_read => Some(Self::ReadLines),
"read" => Some(Self::Read),
"edit" => Some(Self::Edit),
"write" => Some(Self::Write),
_ => None,
}
}
pub fn argument_schema(self) -> schemars::Schema {
match self {
Self::Read => schemars::schema_for!(ReadArgs),
Self::ReadLines => schemars::schema_for!(LineReadArgs),
Self::Edit => schemars::schema_for!(EditArgs),
Self::Write => schemars::schema_for!(WriteArgs),
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Deserialize, JsonSchema)]
pub struct EditArgs {
/// String to replace. Must be nonempty and unique unless replace_all is true.
pub old_string: String,
/// Replacement text. Must differ from old_string; may be empty to delete it.
pub new_string: String,
/// Replace every non-overlapping occurrence. Defaults to false.
#[serde(default)]
pub replace_all: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, Deserialize, JsonSchema)]
pub struct WriteArgs {
/// Full text to write.
pub content: String,
}
/// Whole-text reads accept no operation-specific arguments.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct ReadArgs {}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, JsonSchema)]
pub struct LineReadArgs {
/// Zero-based line offset. Omitted or null means zero.
#[serde(default)]
pub offset: Option<usize>,
/// Maximum lines to return. Omitted or null uses the frontend's default.
#[serde(default)]
pub limit: Option<usize>,
}
impl LineReadArgs {
pub fn range(&self, default_limit: usize) -> (usize, usize) {
(
self.offset.unwrap_or(0),
self.limit.unwrap_or(default_limit).max(1),
)
}
}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TextLimits {
pub max_input_bytes: Option<usize>,
pub max_output_bytes: Option<usize>,
pub max_replacements: Option<usize>,
}
#[derive(Debug, Error)]
pub enum TextError {
#[error("invalid arguments: {0}")]
Decode(#[source] serde_json::Error),
#[error("old_string must not be empty")]
EmptyOldString,
#[error("old_string and new_string are identical")]
IdenticalStrings,
#[error("old_string was not found")]
NotFound,
#[error(
"old_string matched {occurrences} times; set replace_all=true or provide a unique string"
)]
MultipleMatches { occurrences: usize },
#[error("input size {bytes} exceeds limit {limit}")]
InputTooLarge { bytes: usize, limit: usize },
#[error("output size {bytes} exceeds limit {limit}")]
OutputTooLarge { bytes: usize, limit: usize },
#[error("replacement count {replacements} exceeds limit {limit}")]
ReplacementLimitExceeded { replacements: usize, limit: usize },
#[error("edited content size overflowed")]
Overflow,
}
pub fn decode<T: DeserializeOwned>(value: serde_json::Value) -> Result<T, TextError> {
serde_json::from_value(value).map_err(TextError::Decode)
}
/// Decode raw Tool JSON directly, retaining duplicate-field rejection rather
/// than losing that information through an intermediate JSON object.
pub fn decode_json<T: DeserializeOwned>(input: &str) -> Result<T, TextError> {
serde_json::from_str(input).map_err(TextError::Decode)
}
impl EditArgs {
/// Validate argument-only policy before a caller acquires a preimage.
pub fn validate(&self, limits: TextLimits) -> Result<(), TextError> {
if self.old_string.is_empty() {
return Err(TextError::EmptyOldString);
}
if self.old_string == self.new_string {
return Err(TextError::IdenticalStrings);
}
check_input_bytes(self.old_string.len(), limits)?;
check_input_bytes(self.new_string.len(), limits)?;
Ok(())
}
}
/// A numbered, bounded text response with provider line metadata.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Rendered {
pub body: String,
pub line_count: usize,
pub total_lines: usize,
pub truncated: bool,
}
const RENDERED_TRUNCATION_MARKER: &str = "\n[truncated at rendered output byte limit]\n";
/// Render selected text with one-based line numbers and a hard UTF-8 byte bound.
///
/// As in the Tool renderer, room for the truncation marker is always reserved,
/// and a partially emitted numbered line counts as a line. Tiny bounds emit
/// only the marker prefix that fits. Line numbers saturate rather than wrap.
pub fn render_numbered(
text: &str,
start_line: usize,
total_lines: usize,
truncated: bool,
max_bytes: usize,
) -> Rendered {
let budget = max_bytes.saturating_sub(RENDERED_TRUNCATION_MARKER.len());
let mut body = String::with_capacity(text.len().min(max_bytes));
let mut line_count = 0usize;
let mut output_truncated = false;
for (index, line) in text.lines().enumerate() {
// Build only the small numeric prefix, not an unbounded copy of a line.
let number = start_line.saturating_add(index).saturating_add(1);
let prefix = format!("{number:>6}\t");
let numbered_len = prefix.len().saturating_add(line.len()).saturating_add(1);
let mut remaining = budget.saturating_sub(body.len());
if remaining > 0 {
line_count = line_count.saturating_add(1);
}
let line_truncated = numbered_len > remaining;
for fragment in [prefix.as_str(), line, "\n"] {
let mut end = remaining.min(fragment.len());
while !fragment.is_char_boundary(end) {
end -= 1;
}
body.push_str(&fragment[..end]);
remaining -= end;
// Do not skip a character that failed to fit and then emit later
// fragments; the response must be a prefix of the numbered line.
if end < fragment.len() {
break;
}
}
if line_truncated {
let marker_len = max_bytes
.saturating_sub(body.len())
.min(RENDERED_TRUNCATION_MARKER.len());
body.push_str(&RENDERED_TRUNCATION_MARKER[..marker_len]);
output_truncated = true;
break;
}
}
Rendered {
body,
line_count,
total_lines,
truncated: start_line > 0 || truncated || output_truncated,
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct EditOutcome {
pub content: String,
pub replacements: usize,
pub bytes_written: usize,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct TextOutcome {
pub content: String,
pub bytes: usize,
}
pub fn edit(original: &str, args: &EditArgs, limits: TextLimits) -> Result<EditOutcome, TextError> {
args.validate(limits)?;
check_input_bytes(original.len(), limits)?;
let occurrences = original.matches(&args.old_string).count();
if occurrences == 0 {
return Err(TextError::NotFound);
}
if !args.replace_all && occurrences != 1 {
return Err(TextError::MultipleMatches { occurrences });
}
let replacements = occurrences;
if let Some(limit) = limits.max_replacements
&& replacements > limit
{
return Err(TextError::ReplacementLimitExceeded {
replacements,
limit,
});
}
// All arithmetic and output policy are checked before allocating the result.
let bytes_written = edited_len(
original.len(),
args.old_string.len(),
args.new_string.len(),
replacements,
)?;
check_output_bytes(bytes_written, limits)?;
let mut content = String::with_capacity(bytes_written);
let mut cursor = 0;
for (start, matched) in original.match_indices(&args.old_string) {
content.push_str(&original[cursor..start]);
content.push_str(&args.new_string);
cursor = start + matched.len();
}
content.push_str(&original[cursor..]);
debug_assert_eq!(content.len(), bytes_written);
Ok(EditOutcome {
content,
replacements,
bytes_written,
})
}
fn edited_len(
original: usize,
old: usize,
new: usize,
replacements: usize,
) -> Result<usize, TextError> {
let removed = old.checked_mul(replacements).ok_or(TextError::Overflow)?;
let added = new.checked_mul(replacements).ok_or(TextError::Overflow)?;
let bytes = original
.checked_sub(removed)
.and_then(|bytes| bytes.checked_add(added))
.ok_or(TextError::Overflow)?;
// String/Vec capacity is bounded by isize::MAX even on platforms where usize
// arithmetic still fits. Reject that case before String::with_capacity.
if bytes > isize::MAX as usize {
return Err(TextError::Overflow);
}
Ok(bytes)
}
pub fn write(content: String, limits: TextLimits) -> Result<TextOutcome, TextError> {
text_outcome(content, limits)
}
pub fn read(content: String, limits: TextLimits) -> Result<TextOutcome, TextError> {
text_outcome(content, limits)
}
fn text_outcome(content: String, limits: TextLimits) -> Result<TextOutcome, TextError> {
let bytes = content.len();
check_input_bytes(bytes, limits)?;
check_output_bytes(bytes, limits)?;
Ok(TextOutcome { content, bytes })
}
pub(crate) fn check_input_bytes(bytes: usize, limits: TextLimits) -> Result<(), TextError> {
if let Some(limit) = limits.max_input_bytes
&& bytes > limit
{
return Err(TextError::InputTooLarge { bytes, limit });
}
Ok(())
}
pub(crate) fn check_output_bytes(bytes: usize, limits: TextLimits) -> Result<(), TextError> {
if let Some(limit) = limits.max_output_bytes
&& bytes > limit
{
return Err(TextError::OutputTooLarge { bytes, limit });
}
Ok(())
}
/// Keep valid UTF-8 prefixes intact without changing byte-read behavior on
/// genuinely invalid UTF-8. Providers must remain able to return binary data.
pub(crate) fn bounded_utf8_len(bytes: &[u8], max_bytes: usize) -> usize {
let mut end = max_bytes.min(bytes.len());
while end > 0 {
match std::str::from_utf8(&bytes[..end]) {
Ok(_) => break,
Err(error) if error.error_len().is_none() => end -= 1,
Err(_) => break,
}
}
end
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn args(old: &str, new: &str, replace_all: bool) -> EditArgs {
EditArgs {
old_string: old.into(),
new_string: new.into(),
replace_all,
}
}
#[test]
fn decode_uses_shared_argument_defaults_and_types() {
let edit = decode::<EditArgs>(json!({"old_string": "a", "new_string": "b"})).unwrap();
assert!(!edit.replace_all);
assert!(
decode::<EditArgs>(json!({"old_string": "a", "new_string": "b", "replace_all": true}))
.unwrap()
.replace_all
);
assert_eq!(
decode::<WriteArgs>(json!({"content": "😀"}))
.unwrap()
.content,
"😀"
);
assert_eq!(decode::<ReadArgs>(json!({})).unwrap(), ReadArgs {});
for value in [
json!({"old_string": "a"}),
json!({"old_string": 2, "new_string": "b"}),
json!({"old_string": "a", "new_string": "b", "replace_all": null}),
] {
assert!(matches!(
decode::<EditArgs>(value),
Err(TextError::Decode(_))
));
}
assert!(matches!(
decode::<WriteArgs>(json!({})),
Err(TextError::Decode(_))
));
assert!(matches!(
decode::<ReadArgs>(json!({"offset": 0})),
Err(TextError::Decode(_))
));
assert!(matches!(
decode::<ReadArgs>(json!(null)),
Err(TextError::Decode(_))
));
}
#[test]
fn line_range_preserves_null_defaults_and_minimum_one() {
for value in [json!({}), json!({"offset": null, "limit": null})] {
assert_eq!(
decode::<LineReadArgs>(value).unwrap().range(2000),
(0, 2000)
);
}
assert_eq!(
decode::<LineReadArgs>(json!({"offset": 7, "limit": 0}))
.unwrap()
.range(99),
(7, 1)
);
assert_eq!(LineReadArgs::default().range(0), (0, 1));
assert_eq!(
LineReadArgs {
offset: Some(usize::MAX),
limit: Some(usize::MAX)
}
.range(1),
(usize::MAX, usize::MAX)
);
for value in [
json!({"offset": -1}),
json!({"limit": "2"}),
json!({"limit": 1.5}),
] {
assert!(matches!(
decode::<LineReadArgs>(value),
Err(TextError::Decode(_))
));
}
}
#[test]
fn schema_is_the_argument_contract() {
let edit = serde_json::to_value(schemars::schema_for!(EditArgs)).unwrap();
assert_eq!(edit["required"], json!(["old_string", "new_string"]));
assert_eq!(edit["properties"]["replace_all"]["default"], json!(false));
let read = serde_json::to_value(schemars::schema_for!(ReadArgs)).unwrap();
assert_eq!(read["additionalProperties"], json!(false));
assert!(
read.get("properties")
.is_none_or(|properties| properties == &json!({}))
);
let lines = serde_json::to_value(schemars::schema_for!(LineReadArgs)).unwrap();
assert_eq!(
lines["properties"]["offset"]["type"],
json!(["integer", "null"])
);
assert_eq!(lines["properties"]["limit"]["default"], json!(null));
}
#[test]
fn edit_rejects_empty_identical_missing_and_ambiguous_matches() {
assert!(matches!(
args("", "x", true).validate(TextLimits::default()),
Err(TextError::EmptyOldString)
));
assert!(matches!(
edit("a", &args("a", "a", false), TextLimits::default()),
Err(TextError::IdenticalStrings)
));
assert!(matches!(
edit("a", &args("b", "c", false), TextLimits::default()),
Err(TextError::NotFound)
));
assert!(matches!(
edit("aa", &args("a", "b", false), TextLimits::default()),
Err(TextError::MultipleMatches { occurrences: 2 })
));
assert!(matches!(
edit("", &args("a", "", false), TextLimits::default()),
Err(TextError::NotFound)
));
}
#[test]
fn edit_handles_unique_all_deletion_literal_and_nonoverlapping_matches() {
for (original, old, new, all, expected, replacements) in [
(
"before old after",
"old",
"new",
false,
"before new after",
1,
),
("aaa", "aa", "x", true, "xa", 1),
("a-a-a", "a", "", true, "--", 3),
("[a].* [a].*", "[a].*", "$1", true, "$1 $1", 2),
("x", "x", "", false, "", 1),
] {
let result = edit(original, &args(old, new, all), TextLimits::default()).unwrap();
assert_eq!(result.content, expected);
assert_eq!(result.replacements, replacements);
assert_eq!(result.bytes_written, expected.len());
}
}
#[test]
fn edit_preserves_unicode_and_counts_bytes() {
let result = edit("é😀\né😀", &args("é😀", "界", true), TextLimits::default()).unwrap();
assert_eq!(result.content, "界\n界");
assert_eq!(result.replacements, 2);
assert_eq!(result.bytes_written, 7);
// Matching is literal, without Unicode normalization.
assert!(matches!(
edit("e\u{301}", &args("é", "x", false), TextLimits::default()),
Err(TextError::NotFound)
));
}
#[test]
fn edit_input_limits_cover_arguments_and_preimage_with_inclusive_boundaries() {
let limits = TextLimits {
max_input_bytes: Some(4),
..Default::default()
};
args("😀", "a", false).validate(limits).unwrap();
assert!(matches!(
args("😀x", "a", false).validate(limits),
Err(TextError::InputTooLarge { bytes: 5, limit: 4 })
));
assert!(matches!(
args("a", "😀x", false).validate(limits),
Err(TextError::InputTooLarge { bytes: 5, limit: 4 })
));
assert!(matches!(
edit("aaaaa", &args("a", "b", true), limits),
Err(TextError::InputTooLarge { bytes: 5, limit: 4 })
));
assert_eq!(
edit("aaaa", &args("a", "b", true), limits).unwrap().content,
"bbbb"
);
}
#[test]
fn edit_output_and_replacement_limits_apply_before_constructing_result() {
let args = args("a", "😀", true);
let limits = TextLimits {
max_output_bytes: Some(7),
..Default::default()
};
assert!(matches!(
edit("aa", &args, limits),
Err(TextError::OutputTooLarge { bytes: 8, limit: 7 })
));
assert_eq!(
edit(
"aa",
&args,
TextLimits {
max_output_bytes: Some(8),
..limits
}
)
.unwrap()
.bytes_written,
8
);
assert!(matches!(
edit(
"aa",
&args,
TextLimits {
max_replacements: Some(1),
..Default::default()
}
),
Err(TextError::ReplacementLimitExceeded {
replacements: 2,
limit: 1
})
));
assert!(matches!(
edit(
"a",
&args,
TextLimits {
max_replacements: Some(0),
..Default::default()
}
),
Err(TextError::ReplacementLimitExceeded {
replacements: 1,
limit: 0
})
));
assert_eq!(
edit(
"aa",
&args,
TextLimits {
max_replacements: Some(2),
..Default::default()
}
)
.unwrap()
.replacements,
2
);
assert_eq!(
edit(
"a",
&EditArgs {
old_string: "a".into(),
new_string: String::new(),
replace_all: false
},
TextLimits {
max_output_bytes: Some(0),
..Default::default()
}
)
.unwrap()
.content,
""
);
}
#[test]
fn raw_json_decode_preserves_duplicate_argument_rejection() {
for input in [
r#"{"old_string":"a","old_string":"b","new_string":"c"}"#,
r#"{"old_string":"a","new_string":"b","replace_all":false,"replace_all":true}"#,
] {
assert!(matches!(
decode_json::<EditArgs>(input),
Err(TextError::Decode(_))
));
}
assert!(matches!(
decode_json::<WriteArgs>(r#"{"content":"a","content":"b"}"#),
Err(TextError::Decode(_))
));
}
#[test]
fn size_arithmetic_rejects_every_overflow_before_preallocation() {
// Exercise impossible-to-allocate sizes through the exact helper edit
// invokes before allocating; no giant strings need to be constructed.
for (original, old, new, count) in [
(usize::MAX, usize::MAX, 0, 2), // removed multiplication
(2, 1, usize::MAX, 2), // added multiplication
(1, 2, 0, 1), // subtraction
(usize::MAX, 1, 2, 1), // addition
(1, 1, isize::MAX as usize + 1, 1), // allocation capacity
] {
assert!(matches!(
edited_len(original, old, new, count),
Err(TextError::Overflow)
));
}
assert_eq!(
edited_len(1, 1, isize::MAX as usize, 1).unwrap(),
isize::MAX as usize
);
assert_eq!(edited_len(8, 2, 1, 3).unwrap(), 5);
}
#[test]
fn read_and_write_preserve_whole_text_and_enforce_both_byte_limits() {
for operation in [read, write] {
assert_eq!(
operation(String::new(), TextLimits::default()).unwrap(),
TextOutcome {
content: String::new(),
bytes: 0
}
);
let content = "é\r\n😀\n";
let limits = TextLimits {
max_input_bytes: Some(content.len()),
max_output_bytes: Some(content.len()),
max_replacements: Some(0),
};
let result = operation(content.into(), limits).unwrap();
assert_eq!(result.content, content);
assert_eq!(result.bytes, content.len());
assert!(matches!(
operation(
"😀".into(),
TextLimits {
max_input_bytes: Some(3),
..Default::default()
}
),
Err(TextError::InputTooLarge { bytes: 4, limit: 3 })
));
assert!(matches!(
operation(
"😀".into(),
TextLimits {
max_output_bytes: Some(3),
..Default::default()
}
),
Err(TextError::OutputTooLarge { bytes: 4, limit: 3 })
));
assert!(matches!(
operation(
"a".into(),
TextLimits {
max_output_bytes: Some(0),
..Default::default()
}
),
Err(TextError::OutputTooLarge { bytes: 1, limit: 0 })
));
}
}
#[test]
fn utf8_byte_bounds_preserve_binary_compatibility() {
for (limit, expected) in [
(0, 0),
(1, 1),
(2, 1),
(3, 1),
(4, 1),
(5, 5),
(usize::MAX, 6),
] {
assert_eq!(bounded_utf8_len("a😀z".as_bytes(), limit), expected);
}
assert_eq!(bounded_utf8_len(&[0xff, 0xfe, 0x00], 2), 2);
}
#[test]
fn operation_names_and_schemas_are_shared_authority() {
assert_eq!(
TextOperation::from_name("read", false),
Some(TextOperation::Read)
);
assert_eq!(
TextOperation::from_name("read", true),
Some(TextOperation::ReadLines)
);
for line_read in [false, true] {
assert_eq!(
TextOperation::from_name("edit", line_read),
Some(TextOperation::Edit)
);
assert_eq!(
TextOperation::from_name("write", line_read),
Some(TextOperation::Write)
);
for name in ["Read", "WRITE", "read_lines", "stat", "", " read"] {
assert_eq!(TextOperation::from_name(name, line_read), None);
}
}
for (operation, expected) in [
(TextOperation::Read, schemars::schema_for!(ReadArgs)),
(
TextOperation::ReadLines,
schemars::schema_for!(LineReadArgs),
),
(TextOperation::Edit, schemars::schema_for!(EditArgs)),
(TextOperation::Write, schemars::schema_for!(WriteArgs)),
] {
assert_eq!(operation.argument_schema(), expected);
}
let schema = serde_json::to_value(TextOperation::Edit.argument_schema()).unwrap();
assert!(
schema["properties"]["new_string"]["description"]
.as_str()
.unwrap()
.contains("Must differ")
);
}
#[test]
fn renderer_preserves_numbering_lines_and_provider_metadata() {
let result = render_numbered("alpha\r\n\nβeta\n", 0, 3, false, 1024);
assert_eq!(result.body, " 1\talpha\n 2\t\n 3\tβeta\n");
assert_eq!(result.line_count, 3);
assert_eq!(result.total_lines, 3);
assert!(!result.truncated);
let result = render_numbered("last", 9, 10, false, 1024);
assert_eq!(result.body, " 10\tlast\n");
assert!(result.truncated);
assert!(render_numbered("last", 0, 10, true, 1024).truncated);
assert_eq!(
render_numbered("", 0, 0, false, 0),
Rendered {
body: String::new(),
line_count: 0,
total_lines: 0,
truncated: false,
}
);
assert!(render_numbered("", 1, 1, false, 0).truncated);
}
#[test]
fn renderer_reserves_marker_and_counts_partial_lines_as_before() {
let marker = RENDERED_TRUNCATION_MARKER;
let result = render_numbered("a\nb", 0, 2, false, marker.len() + 9);
assert_eq!(result.body, format!(" 1\ta\n{marker}"));
assert_eq!(result.line_count, 1);
assert_eq!(result.total_lines, 2);
assert!(result.truncated);
let result = render_numbered("a", 0, 1, false, marker.len() + 1);
assert_eq!(result.body, format!(" {marker}"));
assert_eq!(result.line_count, 1);
assert!(result.truncated);
let result = render_numbered("a", 0, 1, false, marker.len() + 9);
assert_eq!(result.body, " 1\ta\n");
assert_eq!(result.line_count, 1);
assert!(!result.truncated);
}
#[test]
fn renderer_tiny_bounds_unicode_and_large_offsets_never_overflow() {
let marker = RENDERED_TRUNCATION_MARKER;
for max_bytes in 0..=marker.len() {
let result = render_numbered("😀", 0, 1, false, max_bytes);
assert_eq!(result.body, marker[..max_bytes]);
assert_eq!(result.line_count, 0);
assert!(result.truncated);
}
for extra in 1..=4 {
let result = render_numbered("😀😀", 0, 1, false, marker.len() + 7 + extra);
let expected = if extra < 4 {
" 1\t"
} else {
" 1\t😀"
};
assert_eq!(result.body, format!("{expected}{marker}"));
assert_eq!(result.line_count, 1);
assert!(result.body.len() <= marker.len() + 7 + extra);
assert!(result.truncated);
}
let result = render_numbered("a\nb", usize::MAX, usize::MAX, false, usize::MAX);
assert_eq!(
result.body,
format!("{}\ta\n{}\tb\n", usize::MAX, usize::MAX)
);
assert_eq!(result.line_count, 2);
assert_eq!(result.total_lines, usize::MAX);
assert!(result.truncated);
}
#[test]
fn renderer_matches_original_behavior_for_ordinary_bounds() {
fn original(
text: &str,
start_line: usize,
total_lines: usize,
truncated: bool,
max_bytes: usize,
) -> Rendered {
let marker = RENDERED_TRUNCATION_MARKER;
let budget = max_bytes - marker.len();
let mut body = String::with_capacity(text.len().min(max_bytes));
let mut line_count = 0;
let mut output_truncated = false;
for (index, line) in text.lines().enumerate() {
let numbered = format!("{:>6}\t{}\n", start_line + index + 1, line);
let remaining = budget.saturating_sub(body.len());
if remaining > 0 {
line_count += 1;
}
if numbered.len() > remaining {
let mut end = remaining;
while !numbered.is_char_boundary(end) {
end -= 1;
}
body.push_str(&numbered[..end]);
body.push_str(marker);
output_truncated = true;
break;
}
body.push_str(&numbered);
}
Rendered {
body,
line_count,
total_lines,
truncated: start_line > 0 || truncated || output_truncated,
}
}
for text in ["", "a", "\n", "😀é界\nsecond\n", "first\r\n\r\nlast"] {
for start in [0, 1, 999_999] {
for bound in RENDERED_TRUNCATION_MARKER.len()..=160 {
let result = render_numbered(text, start, 99, false, bound);
assert_eq!(result, original(text, start, 99, false, bound));
assert!(result.body.len() <= bound);
}
}
}
}
#[test]
fn errors_keep_actionable_messages_and_decode_source() {
assert_eq!(
TextError::EmptyOldString.to_string(),
"old_string must not be empty"
);
assert_eq!(
TextError::IdenticalStrings.to_string(),
"old_string and new_string are identical"
);
assert_eq!(TextError::NotFound.to_string(), "old_string was not found");
assert_eq!(
TextError::MultipleMatches { occurrences: 3 }.to_string(),
"old_string matched 3 times; set replace_all=true or provide a unique string"
);
assert_eq!(
TextError::Overflow.to_string(),
"edited content size overflowed"
);
let error = decode::<ReadArgs>(json!({"unknown": true})).unwrap_err();
assert!(std::error::Error::source(&error).is_some());
}
}
+108
View File
@@ -0,0 +1,108 @@
#![cfg(target_os = "linux")]
use crate::*;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
struct Filtered {
root: PathBuf,
seen: Arc<Mutex<Vec<PathBuf>>>,
}
impl FsAccessPolicy for Filtered {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.root) && !path.starts_with(self.root.join("search/denied"))
}
fn is_writable(&self, _: &Path) -> bool {
false
}
fn open_traversal_root(
&self,
_: &Path,
resolved: &Path,
) -> std::io::Result<Option<FsTraversalRoot>> {
use std::os::fd::AsRawFd;
let file = std::fs::File::open(resolved)?;
let walk = PathBuf::from(format!("/proc/self/fd/{}", file.as_raw_fd()));
Ok(Some(FsTraversalRoot::new_at(
walk,
file,
resolved.to_path_buf(),
)))
}
fn traversal_filter(&self) -> Option<FsTraversalFilter> {
let denied = self.root.join("search/denied");
let seen = self.seen.clone();
Some(Arc::new(move |path, _| {
seen.lock().unwrap().push(path.to_path_buf());
!path.starts_with(&denied)
}))
}
fn open_read_file(&self, _: &Path, resolved: &Path) -> std::io::Result<std::fs::File> {
assert!(
!resolved.starts_with(self.root.join("search/denied")),
"denied source opened"
);
std::fs::File::open(resolved)
}
}
#[test]
fn descriptor_search_anchor_and_predicate_prune_before_denied_directory_descent() {
let root = tempfile::tempdir().unwrap();
std::fs::create_dir_all(root.path().join("search/denied/deep")).unwrap();
std::fs::write(root.path().join("search/allowed.txt"), "needle allowed").unwrap();
std::fs::write(
root.path().join("search/denied/deep/secret.txt"),
"needle secret",
)
.unwrap();
std::fs::write(root.path().join("outside.txt"), "needle outside").unwrap();
let policy = Filtered {
root: root.path().to_path_buf(),
seen: Arc::new(Mutex::new(Vec::new())),
};
let glob = run_glob(
root.path(),
&root.path().join("search"),
GlobRequest {
path: FsPath::new("search").unwrap(),
pattern: "**/*.txt".into(),
limit: 100,
},
&policy,
)
.unwrap();
assert_eq!(glob.paths, vec![FsPath::new("search/allowed.txt").unwrap()]);
let grep = run_grep(
root.path(),
root.path().join("search"),
GrepRequest {
path: FsPath::new("search").unwrap(),
pattern: "needle".into(),
glob: None,
file_type: None,
case_insensitive: false,
before_context: 0,
after_context: 0,
multiline: false,
output_mode: GrepOutputMode::Content,
offset: 0,
limit: 100,
},
&policy,
)
.unwrap();
assert_eq!(grep.paths, vec![FsPath::new("search/allowed.txt").unwrap()]);
let seen = policy.seen.lock().unwrap();
assert!(
seen.iter()
.any(|path| path == &root.path().join("search/denied"))
);
assert!(
!seen
.iter()
.any(|path| path.starts_with(root.path().join("search/denied/deep")))
);
assert!(
!seen
.iter()
.any(|path| path == &root.path().join("outside.txt"))
);
}
+233
View File
@@ -0,0 +1,233 @@
//! Bounded descriptor-provider walk shared by Glob and Grep. Unlike a
//! WalkBuilder predicate, this iterator counts even rejected entries and checks
//! cancellation while scanning them. Denied directories are never enumerated.
use crate::{FsAccessPolicy, FsError, FsTraversalFilter};
use std::path::{Path, PathBuf};
pub(crate) struct Entry {
pub path: PathBuf,
pub kind: Option<std::fs::FileType>,
}
pub(crate) enum Walk<'a> {
Plain(ignore::Walk),
Descriptor(DescriptorWalk<'a>),
}
impl<'a> Walk<'a> {
/// A rejected directory has been yielded but not enumerated yet. Keep
/// ignore pruning inside the same bounded descriptor walk as scope pruning.
pub fn skip_current_dir(&mut self) {
if let Self::Descriptor(walk) = self {
walk.pending = None;
}
}
pub fn descriptor(root: &Path, access: &'a dyn FsAccessPolicy) -> Self {
Self::Descriptor(DescriptorWalk {
access,
filter: access.traversal_filter(),
pending: Some((root.to_path_buf(), 0)),
root: Some(root.to_path_buf()),
stack: Vec::new(),
visited: 0,
failed: false,
})
}
}
impl Iterator for Walk<'_> {
type Item = Result<Entry, FsError>;
fn next(&mut self) -> Option<Self::Item> {
match self {
Self::Plain(walk) => loop {
match walk.next()? {
Ok(entry) => {
return Some(Ok(Entry {
path: entry.path().to_path_buf(),
kind: entry.file_type(),
}));
}
Err(_) => continue,
}
},
Self::Descriptor(walk) => walk.next(),
}
}
}
pub(crate) struct DescriptorWalk<'a> {
access: &'a dyn FsAccessPolicy,
filter: Option<FsTraversalFilter>,
root: Option<PathBuf>,
pending: Option<(PathBuf, usize)>,
stack: Vec<(PathBuf, usize, std::fs::ReadDir)>,
visited: usize,
failed: bool,
}
impl DescriptorWalk<'_> {
fn next_entry(&mut self) -> Result<Option<Entry>, FsError> {
if let Some(root) = self.root.take() {
self.access
.check_cancelled()
.map_err(|e| FsError::io(&root, e))?;
let resolved = self
.access
.resolve_access_path(&root)
.map_err(|e| FsError::io(&root, e))?;
let metadata = self
.access
.read_metadata(&root, &resolved)
.map_err(|e| FsError::io(&root, e))?;
return Ok(Some(Entry {
path: root,
kind: Some(metadata.file_type()),
}));
}
loop {
if let Some((path, depth)) = self.pending.take() {
self.access
.check_cancelled()
.map_err(|e| FsError::io(&path, e))?;
let selected = self
.filter
.as_ref()
.is_none_or(|filter| filter(&path, true));
let resolved = self
.access
.resolve_access_path(&path)
.map_err(|e| FsError::io(&path, e))?;
if selected && self.access.can_enumerate_directory(&path, &resolved) {
if depth >= 128 {
return Err(FsError::InvalidArgument(
"descriptor traversal depth exceeds provider limit 128".into(),
));
}
let entries = self
.access
.open_read_dir(&path, &resolved)
.map_err(|e| FsError::io(&path, e))?;
self.stack.push((path, depth, entries));
}
}
let Some((parent, depth, entries)) = self.stack.last_mut() else {
return Ok(None);
};
self.access
.check_cancelled()
.map_err(|e| FsError::io(&*parent, e))?;
let Some(entry) = entries.next() else {
self.stack.pop();
continue;
};
self.visited = self.visited.saturating_add(1);
if self.visited > crate::MAX_TRAVERSAL_ENTRIES {
return Err(FsError::InvalidArgument(
"descriptor traversal exceeds provider entry limit".into(),
));
}
let entry = entry.map_err(|e| FsError::io(&*parent, e))?;
if entry.file_name().to_str().is_none() {
continue;
}
let path = parent.join(entry.file_name());
let kind = entry.file_type().map_err(|e| FsError::io(&path, e))?;
let selected = self
.filter
.as_ref()
.is_none_or(|filter| filter(&path, kind.is_dir()));
if !selected {
continue;
}
let resolved = match self.access.resolve_access_path(&path) {
Ok(path) => path,
Err(_) => continue,
};
if !self.access.is_readable_paths(&path, &resolved) {
continue;
}
if kind.is_dir() {
self.pending = Some((path.clone(), depth.saturating_add(1)));
}
return Ok(Some(Entry {
path,
kind: Some(kind),
}));
}
}
}
impl Iterator for DescriptorWalk<'_> {
type Item = Result<Entry, FsError>;
fn next(&mut self) -> Option<Self::Item> {
if self.failed {
return None;
}
match self.next_entry() {
Ok(entry) => entry.map(Ok),
Err(error) => {
self.failed = true;
Some(Err(error))
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{
Arc,
atomic::{AtomicUsize, Ordering},
};
struct RejectFiles {
calls: AtomicUsize,
cancel_at: usize,
}
impl FsAccessPolicy for RejectFiles {
fn is_readable(&self, _: &Path) -> bool {
true
}
fn is_writable(&self, _: &Path) -> bool {
false
}
fn traversal_filter(&self) -> Option<FsTraversalFilter> {
Some(Arc::new(|_, directory| directory))
}
fn check_cancelled(&self) -> std::io::Result<()> {
if self.calls.fetch_add(1, Ordering::Relaxed) >= self.cancel_at {
Err(std::io::Error::from(std::io::ErrorKind::Interrupted))
} else {
Ok(())
}
}
}
#[test]
fn descriptor_walk_counts_rejected_entries_toward_the_entry_budget() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join("rejected"), "must not be opened").unwrap();
let policy = RejectFiles {
calls: AtomicUsize::new(0),
cancel_at: usize::MAX,
};
let Walk::Descriptor(mut walk) = Walk::descriptor(root.path(), &policy) else {
panic!()
};
assert!(walk.next().unwrap().is_ok());
walk.visited = crate::MAX_TRAVERSAL_ENTRIES;
assert!(matches!(
walk.next().unwrap(),
Err(FsError::InvalidArgument(_))
));
}
#[test]
fn descriptor_walk_checks_cancellation_while_scanning_only_rejected_entries() {
let root = tempfile::tempdir().unwrap();
for index in 0..20 {
std::fs::write(root.path().join(index.to_string()), "must not be opened").unwrap();
}
let policy = RejectFiles {
calls: AtomicUsize::new(0),
cancel_at: 4,
};
let mut walk = Walk::descriptor(root.path(), &policy);
assert!(walk.next().unwrap().is_ok());
assert!(
matches!(walk.next().unwrap(), Err(FsError::Io { source, .. }) if source.kind() == std::io::ErrorKind::Interrupted)
);
}
}
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "job"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish = false
[dependencies]
manifest.workspace = true
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
sha2.workspace = true
thiserror.workspace = true
+336
View File
@@ -0,0 +1,336 @@
//! Transport-neutral immutable Job intent and structured-result contract.
//!
//! Hosts own execution, authority, persistence and lifecycle state. This crate
//! deliberately contains no runner, state machine, Runtime or domain grants.
use serde::{Deserialize, Serialize};
use serde_json::Value;
use sha2::{Digest, Sha256};
pub const DEFAULT_MAX_CONCURRENT_JOBS: u16 = 8;
pub const ABSOLUTE_MAX_CONCURRENT_JOBS: u16 = 64;
pub const DEFAULT_JOB_TIMEOUT_SECONDS: u32 = 120;
pub const MAX_JOB_TIMEOUT_SECONDS: u32 = 600;
pub const DEFAULT_MAX_RESULT_BYTES: u32 = 16 * 1024;
pub const ABSOLUTE_MAX_RESULT_BYTES: u32 = 64 * 1024;
pub const DEFAULT_MAX_ATTEMPTS: u8 = 2;
pub const ABSOLUTE_MAX_ATTEMPTS: u8 = 3;
pub const MAX_JOB_INPUT_BYTES: usize = 128 * 1024;
pub const MAX_JOB_INSTRUCTION_BYTES: usize = 16 * 1024;
pub const MAX_JOB_PURPOSE_BYTES: usize = 120;
pub const MAX_JOB_REFERENCE_BYTES: usize = 512;
pub const MAX_JOB_FAILURE_BYTES: usize = 1024;
pub const MAX_JOB_DELIVERY_BYTES: usize = 16 * 1024;
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum JobError {
#[error("{0}")]
InvalidInput(String),
}
pub type Result<T> = std::result::Result<T, JobError>;
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct JobLimits {
#[serde(default = "default_max_concurrent_jobs")]
pub max_concurrent_jobs: u16,
#[serde(default = "default_job_timeout_seconds")]
pub timeout_seconds: u32,
#[serde(default = "default_max_result_bytes")]
pub max_result_bytes: u32,
#[serde(default = "default_max_attempts")]
pub max_attempts: u8,
}
impl Default for JobLimits {
fn default() -> Self {
Self {
max_concurrent_jobs: DEFAULT_MAX_CONCURRENT_JOBS,
timeout_seconds: DEFAULT_JOB_TIMEOUT_SECONDS,
max_result_bytes: DEFAULT_MAX_RESULT_BYTES,
max_attempts: DEFAULT_MAX_ATTEMPTS,
}
}
}
impl JobLimits {
pub fn validate(&self) -> Result<()> {
if self.max_concurrent_jobs == 0 || self.max_concurrent_jobs > ABSOLUTE_MAX_CONCURRENT_JOBS
{
return Err(JobError::InvalidInput(format!(
"Job max_concurrent_jobs must be in 1..={ABSOLUTE_MAX_CONCURRENT_JOBS}"
)));
}
if self.timeout_seconds == 0 || self.timeout_seconds > MAX_JOB_TIMEOUT_SECONDS {
return Err(JobError::InvalidInput(format!(
"Job timeout_seconds must be in 1..={MAX_JOB_TIMEOUT_SECONDS}"
)));
}
if self.max_result_bytes == 0 || self.max_result_bytes > ABSOLUTE_MAX_RESULT_BYTES {
return Err(JobError::InvalidInput(format!(
"Job max_result_bytes must be in 1..={ABSOLUTE_MAX_RESULT_BYTES}"
)));
}
if self.max_attempts == 0 || self.max_attempts > ABSOLUTE_MAX_ATTEMPTS {
return Err(JobError::InvalidInput(format!(
"Job max_attempts must be in 1..={ABSOLUTE_MAX_ATTEMPTS}"
)));
}
Ok(())
}
}
const fn default_max_concurrent_jobs() -> u16 {
DEFAULT_MAX_CONCURRENT_JOBS
}
const fn default_job_timeout_seconds() -> u32 {
DEFAULT_JOB_TIMEOUT_SECONDS
}
const fn default_max_result_bytes() -> u32 {
DEFAULT_MAX_RESULT_BYTES
}
const fn default_max_attempts() -> u8 {
DEFAULT_MAX_ATTEMPTS
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct JobRequest {
/// Stable caller-selected idempotency key within the host's namespace.
pub job_id: String,
pub purpose: String,
/// Revision fenced by the structured result.
pub input_revision: String,
pub input_ref: String,
/// Bounded immutable snapshot; must not contain credentials.
pub input: Value,
pub instruction: String,
/// Manifest registry selector, never raw source or a filesystem path.
pub profile: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub serialization_key: Option<String>,
#[serde(default)]
pub limits: JobLimits,
}
impl JobRequest {
pub fn validate(&self) -> Result<()> {
self.validate_fields()?;
self.validate_input()?;
self.limits.validate()
}
/// Request metadata validation, separable so adapters can retain their
/// domain-validation ordering before validating input and limits.
pub fn validate_fields(&self) -> Result<()> {
validate_identifier("job_id", &self.job_id, 256)?;
validate_identifier("purpose", &self.purpose, MAX_JOB_PURPOSE_BYTES)?;
validate_identifier("input_revision", &self.input_revision, 256)?;
validate_bounded_text("input_ref", &self.input_ref, MAX_JOB_REFERENCE_BYTES)?;
validate_bounded_text("instruction", &self.instruction, MAX_JOB_INSTRUCTION_BYTES)?;
self.profile_selector()?;
if let Some(key) = &self.serialization_key {
validate_identifier("serialization_key", key, MAX_JOB_REFERENCE_BYTES)?;
}
Ok(())
}
pub fn validate_input(&self) -> Result<()> {
let input = serde_json::to_vec(&self.input)
.map_err(|error| JobError::InvalidInput(format!("serialize Job input: {error}")))?;
if input.len() > MAX_JOB_INPUT_BYTES {
return Err(JobError::InvalidInput(format!(
"Job input exceeds {MAX_JOB_INPUT_BYTES} bytes"
)));
}
Ok(())
}
/// Syntax only; registry existence and resolution are host authority.
pub fn profile_selector(&self) -> Result<manifest::ProfileSelector> {
profile_selector(&self.profile)
}
pub fn fingerprint(&self) -> Result<String> {
self.validate()?;
fingerprint(self)
}
pub fn worker_input(&self, attempt_id: &str) -> Result<String> {
worker_input(self, attempt_id)
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct JobResultSubmission {
pub job_id: String,
pub attempt_id: String,
pub input_revision: String,
pub result: Value,
}
/// Client-facing lifecycle vocabulary, not a runner or transition policy.
/// A variant's presence does not imply every host supports that operation.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum JobState {
Pending,
Completed,
Failed,
Unknown,
Cancelled,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum JobAttemptState {
Reserved,
Dispatching,
Dispatched,
Completed,
Failed,
Unknown,
Cancelled,
}
/// Transport-neutral read snapshot. Job status/result and the corresponding
/// attempt identity/status remain distinct; host identities, delivery and
/// resource cleanup are deliberately not part of the model outcome.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct JobOutcome {
pub job_id: String,
pub input_revision: String,
pub attempt_id: String,
pub attempt: u8,
pub state: JobState,
pub attempt_state: JobAttemptState,
pub result: Option<Value>,
pub failure_category: Option<String>,
pub failure_detail: Option<String>,
}
pub fn profile_selector(profile: &str) -> Result<manifest::ProfileSelector> {
use manifest::ProfileSelector;
validate_identifier("profile", profile, 256)?;
let selector = ProfileSelector::parse_cli(profile);
let valid = match &selector {
ProfileSelector::Default => true,
ProfileSelector::Path { .. } => false,
ProfileSelector::Named { source, name } => {
!name.is_empty()
&& !name.starts_with('.')
&& !name.contains(['/', '\\', ':'])
&& name
.chars()
.all(|ch| ch.is_ascii_alphanumeric() || matches!(ch, '-' | '_' | '.'))
&& ![".toml", ".json", ".dcdl", ".nix"]
.iter()
.any(|suffix| name.ends_with(suffix))
&& (source.is_some() || name != "inherit")
}
};
if !valid {
return Err(JobError::InvalidInput(
"Job profile must be a manifest registry selector, not raw source or a path".into(),
));
}
Ok(selector)
}
/// Hash the full serialized host intent, not just its neutral projection.
/// Callers must validate the complete intent first. JSON field ordering and
/// omission rules are intentionally preserved for durable replay compatibility.
pub fn fingerprint<T: Serialize + ?Sized>(intent: &T) -> Result<String> {
let encoded = serde_json::to_vec(intent)
.map_err(|error| JobError::InvalidInput(format!("serialize Job intent: {error}")))?;
Ok(sha256(&encoded))
}
pub fn worker_input(request: &JobRequest, attempt_id: &str) -> Result<String> {
worker_input_with_label(request, attempt_id, "Job")
}
/// Adapter-specific capability label; only the host, never the model, selects it.
/// Keeps an existing host's prompt envelope byte-for-byte compatible.
pub fn worker_input_with_label(
request: &JobRequest,
attempt_id: &str,
label: &str,
) -> Result<String> {
let input = serde_json::to_string(&request.input)
.map_err(|error| JobError::InvalidInput(format!("serialize Job input: {error}")))?;
Ok(format!(
"{}\n\n{label} envelope (immutable):\njob_id: {}\nattempt_id: {}\ninput_revision: {}\ninput_ref: {}\ninput_json: {}\n\nReturn success only through the structured {label} result capability. Final prose and Worker Idle/Stopped state are not result authority.",
request.instruction,
request.job_id,
attempt_id,
request.input_revision,
request.input_ref,
input
))
}
pub fn attempt_id(job_id: &str, attempt: u8) -> String {
format!("{job_id}:attempt:{attempt}")
}
/// Bound the encoded JSON before hashing. Returns (digest, exact JSON).
pub fn result_digest(result: &Value, max_bytes: u32) -> Result<(String, String)> {
let encoded = serde_json::to_string(result)
.map_err(|error| JobError::InvalidInput(format!("serialize Job result: {error}")))?;
if encoded.len() > max_bytes as usize {
return Err(JobError::InvalidInput(format!(
"Job result exceeds {max_bytes} bytes"
)));
}
Ok((sha256(encoded.as_bytes()), encoded))
}
pub fn bounded_failure_detail(detail: &str) -> String {
if detail.len() <= MAX_JOB_FAILURE_BYTES {
return detail.to_string();
}
let mut end = MAX_JOB_FAILURE_BYTES;
while !detail.is_char_boundary(end) {
end -= 1;
}
detail[..end].to_string()
}
pub fn validate_identifier(field: &str, value: &str, max_bytes: usize) -> Result<()> {
validate_bounded_text(field, value, max_bytes)?;
if value.chars().any(char::is_control) {
return Err(JobError::InvalidInput(format!(
"Job {field} must not contain control characters"
)));
}
Ok(())
}
pub fn validate_bounded_text(field: &str, value: &str, max_bytes: usize) -> Result<()> {
if value.is_empty() || value.trim() != value {
return Err(JobError::InvalidInput(format!(
"Job {field} must be non-empty and trimmed"
)));
}
if value.len() > max_bytes {
return Err(JobError::InvalidInput(format!(
"Job {field} exceeds {max_bytes} bytes"
)));
}
Ok(())
}
pub fn sha256(bytes: &[u8]) -> String {
let digest = Sha256::digest(bytes);
let mut output = String::with_capacity(7 + digest.len() * 2);
output.push_str("sha256:");
for byte in digest {
use std::fmt::Write as _;
let _ = write!(output, "{byte:02x}");
}
output
}
+308
View File
@@ -0,0 +1,308 @@
use job::*;
use serde_json::json;
fn request() -> JobRequest {
JobRequest {
job_id: "check:r7".into(),
purpose: "snapshot_check".into(),
input_revision: "7".into(),
input_ref: "snapshot://7".into(),
input: json!({"title": "Check me"}),
instruction: "Check this immutable snapshot.".into(),
profile: "project:custom-check".into(),
serialization_key: None,
limits: JobLimits::default(),
}
}
#[test]
fn shared_lifecycle_enums_round_trip_all_host_states() {
for (state, name) in [
(JobState::Pending, "pending"),
(JobState::Completed, "completed"),
(JobState::Failed, "failed"),
(JobState::Unknown, "unknown"),
(JobState::Cancelled, "cancelled"),
] {
assert_eq!(serde_json::to_value(state).unwrap(), json!(name));
assert_eq!(
serde_json::from_value::<JobState>(json!(name)).unwrap(),
state
);
}
for (state, name) in [
(JobAttemptState::Reserved, "reserved"),
(JobAttemptState::Dispatching, "dispatching"),
(JobAttemptState::Dispatched, "dispatched"),
(JobAttemptState::Completed, "completed"),
(JobAttemptState::Failed, "failed"),
(JobAttemptState::Unknown, "unknown"),
(JobAttemptState::Cancelled, "cancelled"),
] {
assert_eq!(serde_json::to_value(state).unwrap(), json!(name));
assert_eq!(
serde_json::from_value::<JobAttemptState>(json!(name)).unwrap(),
state
);
}
assert!(serde_json::from_value::<JobState>(json!("running")).is_err());
assert!(serde_json::from_value::<JobAttemptState>(json!("idle")).is_err());
}
#[test]
fn outcome_is_a_neutral_read_snapshot_with_optional_result_and_failure() {
let mut outcome = JobOutcome {
job_id: "job-1".into(),
input_revision: "7".into(),
attempt_id: "job-1:attempt:2".into(),
attempt: 2,
state: JobState::Pending,
attempt_state: JobAttemptState::Dispatching,
result: None,
failure_category: None,
failure_detail: None,
};
let encoded = serde_json::to_value(&outcome).unwrap();
assert_eq!(
encoded,
json!({
"job_id": "job-1", "input_revision": "7", "attempt_id": "job-1:attempt:2",
"attempt": 2, "state": "pending", "attempt_state": "dispatching",
"result": null, "failure_category": null, "failure_detail": null
})
);
assert_eq!(
serde_json::from_value::<JobOutcome>(encoded.clone()).unwrap(),
outcome
);
let mut with_host_identity = encoded;
with_host_identity["workspace_id"] = json!("workspace-a");
assert!(serde_json::from_value::<JobOutcome>(with_host_identity).is_err());
for (state, attempt_state, result, category, detail) in [
(
JobState::Completed,
JobAttemptState::Completed,
Some(json!({"ok": true})),
None,
None,
),
(
JobState::Failed,
JobAttemptState::Failed,
None,
Some("execution"),
Some("known failure"),
),
(
JobState::Unknown,
JobAttemptState::Unknown,
None,
Some("interrupted"),
Some("requires reconciliation"),
),
(
JobState::Cancelled,
JobAttemptState::Cancelled,
None,
Some("cancelled"),
Some("caller cancelled"),
),
] {
outcome.state = state;
outcome.attempt_state = attempt_state;
outcome.result = result;
outcome.failure_category = category.map(str::to_owned);
outcome.failure_detail = detail.map(str::to_owned);
assert_eq!(
serde_json::from_str::<JobOutcome>(&serde_json::to_string(&outcome).unwrap()).unwrap(),
outcome
);
}
}
#[test]
fn neutral_request_and_result_have_strict_round_trip_contracts() {
let request = request();
let encoded = serde_json::to_value(&request).unwrap();
assert_eq!(encoded.as_object().unwrap().len(), 8);
assert!(encoded.get("serialization_key").is_none());
assert_eq!(
serde_json::from_value::<JobRequest>(encoded.clone()).unwrap(),
request
);
let mut with_grant = encoded.clone();
with_grant["grants"] = json!({});
assert!(serde_json::from_value::<JobRequest>(with_grant).is_err());
let mut with_provenance = encoded;
with_provenance["source_worker"] = json!({});
assert!(serde_json::from_value::<JobRequest>(with_provenance).is_err());
let submission = JobResultSubmission {
job_id: request.job_id,
attempt_id: "check:r7:attempt:1".into(),
input_revision: request.input_revision,
result: json!({"ok": true}),
};
let mut encoded = serde_json::to_value(&submission).unwrap();
assert_eq!(
serde_json::from_value::<JobResultSubmission>(encoded.clone()).unwrap(),
submission
);
encoded["authority"] = json!("profile");
assert!(serde_json::from_value::<JobResultSubmission>(encoded).is_err());
assert_eq!(
serde_json::from_str::<JobLimits>("{}").unwrap(),
JobLimits::default()
);
assert!(serde_json::from_str::<JobLimits>(r#"{"turns":999}"#).is_err());
}
#[test]
fn fingerprint_covers_every_immutable_field_and_preserves_replay() {
let request = request();
let digest = request.fingerprint().unwrap();
let decoded: JobRequest =
serde_json::from_str(&serde_json::to_string(&request).unwrap()).unwrap();
assert_eq!(digest, decoded.fingerprint().unwrap());
assert_eq!(digest, fingerprint(&request).unwrap());
for mutate in [
|r: &mut JobRequest| r.job_id.push('x'),
|r: &mut JobRequest| r.purpose.push('x'),
|r: &mut JobRequest| r.input_revision.push('x'),
|r: &mut JobRequest| r.input_ref.push('x'),
|r: &mut JobRequest| r.input = json!({"title": "changed"}),
|r: &mut JobRequest| r.instruction.push('x'),
|r: &mut JobRequest| r.profile = "builtin:backend-job".into(),
|r: &mut JobRequest| r.serialization_key = Some("resource:7".into()),
|r: &mut JobRequest| r.limits.max_attempts = 3,
] {
let mut changed = request.clone();
mutate(&mut changed);
assert_ne!(digest, changed.fingerprint().unwrap());
}
}
#[test]
fn registry_profiles_are_not_authority_or_a_fixed_allowlist() {
for profile in [
"default",
"custom",
"project:custom",
"user:custom",
"builtin:future-job",
] {
profile_selector(profile).unwrap();
}
for profile in [
"",
" inherit",
"inherit",
"path:custom",
"./a.toml",
"/tmp/a",
"project:../a",
"project:a.toml",
"builtin:",
"unknown:a",
"project:a:b",
"C:\\custom",
"{\"worker\":{}}",
] {
assert!(profile_selector(profile).is_err(), "accepted {profile}");
}
}
#[test]
fn validation_enforces_utf8_bytes_and_metadata_bounds() {
let mut r = request();
r.job_id = "x".repeat(256);
r.input_revision = "x".repeat(256);
r.purpose = "x".repeat(MAX_JOB_PURPOSE_BYTES);
r.input_ref = "x".repeat(MAX_JOB_REFERENCE_BYTES);
r.instruction = "x".repeat(MAX_JOB_INSTRUCTION_BYTES);
r.serialization_key = Some("x".repeat(MAX_JOB_REFERENCE_BYTES));
r.input = json!("x".repeat(MAX_JOB_INPUT_BYTES - 2));
r.validate().unwrap();
for mutate in [
|r: &mut JobRequest| r.job_id.push('x'),
|r: &mut JobRequest| r.input_revision.push('x'),
|r: &mut JobRequest| r.purpose.push('x'),
|r: &mut JobRequest| r.input_ref.push('x'),
|r: &mut JobRequest| r.instruction.push('x'),
|r: &mut JobRequest| r.serialization_key.as_mut().unwrap().push('x'),
|r: &mut JobRequest| r.input = json!("x".repeat(MAX_JOB_INPUT_BYTES - 1)),
] {
let mut changed = r.clone();
mutate(&mut changed);
assert!(changed.validate().is_err());
assert!(changed.fingerprint().is_err());
}
assert!(validate_identifier("id", "a\nb", 256).is_err());
assert!(validate_bounded_text("text", " untrimmed", 256).is_err());
assert!(validate_bounded_text("text", "", 256).is_err());
assert!(validate_bounded_text("text", "界", 2).is_err());
validate_bounded_text("text", "a\nb", 256).unwrap();
}
#[test]
fn all_limit_dimensions_reject_zero_and_above_absolute_bounds() {
let maximum = JobLimits {
max_concurrent_jobs: ABSOLUTE_MAX_CONCURRENT_JOBS,
timeout_seconds: MAX_JOB_TIMEOUT_SECONDS,
max_result_bytes: ABSOLUTE_MAX_RESULT_BYTES,
max_attempts: ABSOLUTE_MAX_ATTEMPTS,
};
maximum.validate().unwrap();
for mutate in [
|l: &mut JobLimits| l.max_concurrent_jobs = 0,
|l: &mut JobLimits| l.max_concurrent_jobs += 1,
|l: &mut JobLimits| l.timeout_seconds = 0,
|l: &mut JobLimits| l.timeout_seconds += 1,
|l: &mut JobLimits| l.max_result_bytes = 0,
|l: &mut JobLimits| l.max_result_bytes += 1,
|l: &mut JobLimits| l.max_attempts = 0,
|l: &mut JobLimits| l.max_attempts += 1,
] {
let mut changed = maximum.clone();
mutate(&mut changed);
assert!(changed.validate().is_err());
}
}
#[test]
fn result_digest_and_failure_detail_bound_exact_utf8_encoding() {
let result = json!({"ok": true});
let (digest, encoded) = result_digest(&result, 11).unwrap();
assert_eq!(encoded, r#"{"ok":true}"#);
assert_eq!(digest, sha256(encoded.as_bytes()));
assert!(result_digest(&result, 10).is_err());
let unicode = json!("界");
assert!(result_digest(&unicode, 4).is_err());
result_digest(&unicode, 5).unwrap();
let detail = "界".repeat(MAX_JOB_FAILURE_BYTES);
let bounded = bounded_failure_detail(&detail);
assert_eq!(bounded.len(), MAX_JOB_FAILURE_BYTES - 1);
assert!(detail.starts_with(&bounded));
assert_eq!(bounded_failure_detail("known failure"), "known failure");
}
#[test]
fn worker_input_is_revision_and_attempt_bound_not_final_prose_authority() {
let request = request();
let attempt = attempt_id(&request.job_id, 2);
assert_eq!(attempt, "check:r7:attempt:2");
let input = request.worker_input(&attempt).unwrap();
assert!(input.starts_with(&request.instruction));
for field in [
"Job envelope (immutable):",
"job_id: check:r7",
"attempt_id: check:r7:attempt:2",
"input_revision: 7",
"input_ref: snapshot://7",
r#"input_json: {"title":"Check me"}"#,
"structured Job result capability",
"Final prose and Worker Idle/Stopped state are not result authority.",
] {
assert!(input.contains(field), "missing {field}");
}
}
+2
View File
@@ -3,8 +3,10 @@ name = "manifest"
version = "0.1.0"
edition.workspace = true
license.workspace = true
rust-version.workspace = true
[dependencies]
builtin-source.workspace = true
arc-swap = "1"
agen = { workspace = true }
decodal.workspace = true
+2
View File
@@ -25,6 +25,8 @@ Does not own:
Profiles are reusable recipes; resolved Manifests are runtime contracts. Keep runtime-bound fields such as Worker name, concrete delegated scope, sockets, session pointers, and raw secrets out of reusable Profiles.
`feature.workdir_catalog.enabled` defaults to `true` for read-only Repository/Workdir/attachment reference discovery in WIP mode. It is independent of `feature.manage_workdir` (which defaults disabled), registers no normal Tools, and does not grant filesystem, attachment, or Workspace authority. Permissions remain independent. Standalone Tools-mode profiles may retain this default; only WIP installation consumes the flag. Set it to `false` to opt out.
## See also
- [`../../docs/design/profiles-manifests-prompts.md`](../../docs/design/profiles-manifests-prompts.md)
+1 -178
View File
@@ -2,186 +2,10 @@ use std::collections::BTreeMap;
use decodal::{Data, Engine, ImportLoader, LoadedImport};
use serde_json::{Map, Number, Value};
use sha2::{Digest, Sha256};
use crate::profile::ProfileError;
pub const BUILTIN_PROFILE_CATALOG_ID: &str = "builtin-profiles-v2";
pub const BUILTIN_DEFAULT_PROFILE: &str = "builtin:default";
pub const BUILTIN_STANDALONE_PROFILE: &str = "builtin:standalone";
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BuiltinProfileImport {
pub specifier: &'static str,
pub resolved_path: &'static str,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BuiltinProfileResource {
pub selector: Option<&'static str>,
pub path: &'static str,
pub source: &'static str,
pub description: &'static str,
pub imports: &'static [BuiltinProfileImport],
}
const BASE_PATH: &str = "profiles/base.dcdl";
const DEFAULT_PATH: &str = "profiles/default.dcdl";
const BASE_IMPORT: &[BuiltinProfileImport] = &[BuiltinProfileImport {
specifier: "./base.dcdl",
resolved_path: BASE_PATH,
}];
const DEFAULT_IMPORT: &[BuiltinProfileImport] = &[BuiltinProfileImport {
specifier: "./default.dcdl",
resolved_path: DEFAULT_PATH,
}];
const NO_IMPORTS: &[BuiltinProfileImport] = &[];
pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
BuiltinProfileResource {
selector: None,
path: BASE_PATH,
source: include_str!("../../../resources/profiles/base.dcdl"),
description: "Shared built-in Profile defaults.",
imports: NO_IMPORTS,
},
BuiltinProfileResource {
selector: Some(BUILTIN_DEFAULT_PROFILE),
path: DEFAULT_PATH,
source: include_str!("../../../resources/profiles/default.dcdl"),
description: "Default Yoi coding profile.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some(BUILTIN_STANDALONE_PROFILE),
path: "profiles/standalone.dcdl",
source: include_str!("../../../resources/profiles/standalone.dcdl"),
description: "Standalone Yoi coding profile.",
imports: DEFAULT_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:coder"),
path: "profiles/coder.dcdl",
source: include_str!("../../../resources/profiles/coder.dcdl"),
description: "Ticket implementation with direct Reviewer SubWorkers.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:companion"),
path: "profiles/companion.dcdl",
source: include_str!("../../../resources/profiles/companion.dcdl"),
description: "General assistance with Workspace tools.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:intake"),
path: "profiles/intake.dcdl",
source: include_str!("../../../resources/profiles/intake.dcdl"),
description: "Read-only intake and planning.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:reviewer"),
path: "profiles/reviewer.dcdl",
source: include_str!("../../../resources/profiles/reviewer.dcdl"),
description: "Independent review of a published Merge Request source.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:orchestrator"),
path: "profiles/orchestrator.dcdl",
source: include_str!("../../../resources/profiles/orchestrator.dcdl"),
description: "Workspace orchestration and Worker control.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:backend-job"),
path: "profiles/backend-job.dcdl",
source: include_str!("../../../resources/profiles/backend-job.dcdl"),
description: "Backend-owned bounded Job Worker.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:memory-consolidation"),
path: "profiles/memory-consolidation.dcdl",
source: include_str!("../../../resources/profiles/memory-consolidation.dcdl"),
description: "Internal Memory consolidation service.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:subjektiv-memory-consolidation"),
path: "profiles/subjektiv-memory-consolidation.dcdl",
source: include_str!("../../../resources/profiles/subjektiv-memory-consolidation.dcdl"),
description: "Delegated subject Memory consolidation service.",
imports: BASE_IMPORT,
},
];
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BuiltinProfileCatalogSnapshot {
pub id: &'static str,
pub sources: BTreeMap<String, String>,
pub entrypoints: BTreeMap<String, String>,
pub imports: BTreeMap<String, String>,
}
impl BuiltinProfileCatalogSnapshot {
pub fn digest(&self) -> String {
let mut hasher = Sha256::new();
hasher.update(self.id.as_bytes());
for (path, source) in &self.sources {
hasher.update((path.len() as u64).to_le_bytes());
hasher.update(path.as_bytes());
hasher.update((source.len() as u64).to_le_bytes());
hasher.update(source.as_bytes());
}
for (selector, path) in &self.entrypoints {
hasher.update((selector.len() as u64).to_le_bytes());
hasher.update(selector.as_bytes());
hasher.update((path.len() as u64).to_le_bytes());
hasher.update(path.as_bytes());
}
for (request, resolved_path) in &self.imports {
hasher.update((request.len() as u64).to_le_bytes());
hasher.update(request.as_bytes());
hasher.update((resolved_path.len() as u64).to_le_bytes());
hasher.update(resolved_path.as_bytes());
}
format!("sha256:{:x}", hasher.finalize())
}
}
pub fn builtin_profile_catalog_snapshot() -> BuiltinProfileCatalogSnapshot {
let mut sources = BTreeMap::new();
let mut entrypoints = BTreeMap::new();
let mut imports = BTreeMap::new();
for resource in BUILTIN_PROFILE_RESOURCES {
sources.insert(resource.path.to_owned(), resource.source.to_owned());
for import in resource.imports {
imports.insert(
format!("{}\0{}", resource.path, import.specifier),
import.resolved_path.to_owned(),
);
}
if let Some(selector) = resource.selector {
entrypoints.insert(selector.to_owned(), resource.path.to_owned());
}
}
BuiltinProfileCatalogSnapshot {
id: BUILTIN_PROFILE_CATALOG_ID,
sources,
entrypoints,
imports,
}
}
pub fn builtin_profile_entrypoints() -> impl Iterator<Item = &'static BuiltinProfileResource> {
BUILTIN_PROFILE_RESOURCES
.iter()
.filter(|resource| resource.selector.is_some())
}
pub use builtin_source::*;
pub(crate) fn resolve_builtin_profile_artifact(
selector: &str,
@@ -337,7 +161,6 @@ mod tests {
.expect("evaluate built-in standalone")
.expect("standalone exists");
assert_eq!(value["slug"], "standalone");
assert_eq!(value["model"]["ref"], "codex-oauth/gpt-5.6-sol");
assert_eq!(value["scope"]["intent"], "workspace_write");
assert_eq!(value["scope"]["symlink_policy"], "logical");
assert_eq!(value["delegation_scope"]["intent"], "workspace_write");
+306 -5
View File
@@ -51,8 +51,9 @@ pub struct WorkerManifestConfig {
/// is disabled; `Some` requires `default_action` during final resolve.
#[serde(default)]
pub permissions: Option<PermissionConfigPartial>,
/// Explicit built-in feature/tool-surface enablement. Absent flags resolve
/// disabled after cascade merge.
/// Built-in feature/tool-surface enablement. Absent tool flags resolve
/// disabled after cascade merge; the read-only WIP workdir catalog defaults
/// enabled without granting authority.
#[serde(default)]
pub feature: FeatureConfigPartial,
/// Explicit Model Context Protocol provider declarations. Config parsing
@@ -95,6 +96,10 @@ pub struct FeatureConfigPartial {
#[serde(default)]
pub manage_workdir: Option<FeatureFlagConfigPartial>,
#[serde(default)]
pub workdir_catalog: Option<FeatureFlagConfigPartial>,
#[serde(default)]
pub workspace_config: Option<WorkspaceConfigFeatureConfigPartial>,
#[serde(default)]
pub ticket: Option<TicketFeatureConfigPartial>,
#[serde(default)]
pub merge_request: Option<MergeRequestFeatureConfigPartial>,
@@ -136,6 +141,16 @@ impl FeatureConfigPartial {
other.manage_workdir,
FeatureFlagConfigPartial::merge,
),
workdir_catalog: merge_option(
self.workdir_catalog,
other.workdir_catalog,
FeatureFlagConfigPartial::merge,
),
workspace_config: merge_option(
self.workspace_config,
other.workspace_config,
WorkspaceConfigFeatureConfigPartial::merge,
),
ticket: merge_option(self.ticket, other.ticket, TicketFeatureConfigPartial::merge),
merge_request: merge_option(
self.merge_request,
@@ -165,6 +180,38 @@ impl FeatureFlagConfigPartial {
}
}
/// Activation only: no Workspace/grant/source/permission fields are accepted.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigFeatureConfigPartial {
#[serde(default)]
pub enabled: Option<bool>,
}
impl WorkspaceConfigFeatureConfigPartial {
fn merge(self, other: Self) -> Self {
Self {
enabled: other.enabled.or(self.enabled),
}
}
}
impl From<WorkspaceConfigFeatureConfigPartial> for FeatureFlagConfig {
fn from(value: WorkspaceConfigFeatureConfigPartial) -> Self {
Self {
enabled: value.enabled.unwrap_or_default(),
}
}
}
impl From<FeatureFlagConfig> for WorkspaceConfigFeatureConfigPartial {
fn from(value: FeatureFlagConfig) -> Self {
Self {
enabled: Some(value.enabled),
}
}
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct WorkerFeatureConfigPartial {
#[serde(default)]
@@ -241,6 +288,8 @@ pub struct SubjektivFeatureConfigPartial {
#[serde(default)]
pub enabled: Option<bool>,
#[serde(default)]
pub consolidation_tools: Option<bool>,
#[serde(default)]
pub extraction: Option<MemoryExtractionProfileConfigPartial>,
}
@@ -248,6 +297,7 @@ impl SubjektivFeatureConfigPartial {
fn merge(self, other: Self) -> Self {
Self {
enabled: other.enabled.or(self.enabled),
consolidation_tools: other.consolidation_tools.or(self.consolidation_tools),
extraction: merge_option(
self.extraction,
other.extraction,
@@ -388,6 +438,16 @@ impl From<FeatureConfigPartial> for FeatureConfig {
.manage_workdir
.map(FeatureFlagConfig::from)
.unwrap_or_default(),
workdir_catalog: FeatureFlagConfig {
enabled: value
.workdir_catalog
.and_then(|flag| flag.enabled)
.unwrap_or(defaults::WORKDIR_CATALOG_ENABLED),
},
workspace_config: value
.workspace_config
.map(FeatureFlagConfig::from)
.unwrap_or_default(),
ticket: value
.ticket
.map(TicketFeatureConfig::from)
@@ -487,6 +547,7 @@ impl From<SubjektivFeatureConfigPartial> for ResolvedSubjektivFeatureConfig {
Self {
profile: SubjektivFeatureProfileConfig {
enabled: value.enabled.unwrap_or_default(),
consolidation_tools: value.consolidation_tools.unwrap_or_default(),
extraction: MemoryExtractionProfileConfig {
enabled: extraction.enabled.unwrap_or(true),
model: extraction.model,
@@ -506,6 +567,7 @@ impl From<ResolvedSubjektivFeatureConfig> for SubjektivFeatureConfigPartial {
fn from(value: ResolvedSubjektivFeatureConfig) -> Self {
Self {
enabled: Some(value.profile.enabled),
consolidation_tools: Some(value.profile.consolidation_tools),
extraction: Some(MemoryExtractionProfileConfigPartial {
enabled: Some(value.profile.extraction.enabled),
model: value.profile.extraction.model,
@@ -579,6 +641,8 @@ impl From<FeatureConfig> for FeatureConfigPartial {
workspace_worker_discovery: Some(value.workspace_worker_discovery.into()),
objective: Some(value.objective.into()),
manage_workdir: Some(value.manage_workdir.into()),
workdir_catalog: Some(value.workdir_catalog.into()),
workspace_config: Some(value.workspace_config.into()),
ticket: Some(value.ticket.into()),
merge_request: Some(value.merge_request.into()),
orchestration: Some(value.orchestration.into()),
@@ -590,6 +654,8 @@ impl From<FeatureConfig> for FeatureConfigPartial {
pub struct WorkerMetaConfig {
#[serde(default)]
pub name: Option<String>,
#[serde(default)]
pub mode: Option<crate::WorkerMode>,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
@@ -764,7 +830,8 @@ impl WorkerManifestConfig {
/// Base config populated with the in-code per-field defaults listed in
/// [`crate::defaults`]. This is not a selectable Profile and does not
/// enable a launch capability surface. Profile and one-file Manifest
/// enable launch authority or normal Tools. The read-only WIP workdir
/// catalog defaults enabled independently. Profile and one-file Manifest
/// resolvers start from this layer so every per-field default lives at
/// exactly one call site (the `defaults` module).
///
@@ -925,6 +992,7 @@ impl WorkerMetaConfig {
fn merge(self, upper: Self) -> Self {
Self {
name: upper.name.or(self.name),
mode: upper.mode.or(self.mode),
}
}
}
@@ -1356,7 +1424,10 @@ impl TryFrom<WorkerManifestConfig> for WorkerManifest {
validate_mcp_config(&cfg.mcp)?;
Ok(WorkerManifest {
worker: WorkerMeta { name },
worker: WorkerMeta {
name,
mode: cfg.worker.mode.unwrap_or_default(),
},
model: cfg.model,
engine,
scope: cfg.scope,
@@ -1391,6 +1462,7 @@ mod tests {
WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("test".into()),
mode: None,
},
model: ModelManifest {
scheme: Some(SchemeKind::Anthropic),
@@ -1498,10 +1570,31 @@ mod tests {
fn resolve_minimal_succeeds() {
let manifest: WorkerManifest = minimal_valid().try_into().unwrap();
assert_eq!(manifest.worker.name, "test");
assert_eq!(manifest.worker.mode, crate::WorkerMode::Tools);
assert_eq!(manifest.model.scheme, Some(SchemeKind::Anthropic));
assert!(manifest.permissions.is_none());
}
#[test]
fn worker_mode_is_explicit_opt_in_and_persists_in_snapshots() {
let mut configured = minimal_valid();
configured.worker.mode = Some(crate::WorkerMode::Wip);
let manifest: WorkerManifest = configured.try_into().unwrap();
assert_eq!(manifest.worker.mode, crate::WorkerMode::Wip);
let snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
let restored = crate::read_persisted_worker_manifest_snapshot(snapshot.clone()).unwrap();
assert_eq!(restored.worker.mode, crate::WorkerMode::Wip);
let mut legacy = snapshot;
legacy["manifest"]["worker"]
.as_object_mut()
.unwrap()
.remove("mode");
let restored = crate::read_persisted_worker_manifest_snapshot(legacy).unwrap();
assert_eq!(restored.worker.mode, crate::WorkerMode::Tools);
}
#[test]
fn resolve_mcp_stdio_config_preserves_explicit_policy() {
let mut cfg = minimal_valid();
@@ -1784,6 +1877,7 @@ mod tests {
let lower = WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("lower".into()),
mode: None,
},
model: ModelManifest {
model_id: Some("lower-model".into()),
@@ -1794,6 +1888,7 @@ mod tests {
let upper = WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("upper".into()),
mode: None,
},
..Default::default()
};
@@ -2202,7 +2297,7 @@ worker_max_turns = 7
}
#[test]
fn feature_flags_default_disabled_in_resolved_manifest() {
fn tool_feature_flags_default_disabled_except_read_only_workdir_catalog() {
let manifest: WorkerManifest = minimal_valid().try_into().unwrap();
assert!(!manifest.feature.task.enabled);
assert!(!manifest.feature.memory.profile.enabled);
@@ -2210,10 +2305,213 @@ worker_max_turns = 7
assert!(!manifest.feature.sub_worker.enabled);
assert!(!manifest.feature.objective.enabled);
assert!(!manifest.feature.manage_workdir.enabled);
assert!(manifest.feature.workdir_catalog.enabled);
assert!(!manifest.feature.workspace_config.enabled);
assert!(!manifest.feature.ticket.enabled);
assert!(!manifest.feature.merge_request.any());
}
#[test]
fn workspace_config_defaults_disabled_for_partial_and_resolved_configs() {
assert!(!FeatureConfig::default().workspace_config.enabled);
assert!(FeatureConfigPartial::default().workspace_config.is_none());
for source in ["", "[feature]\n", "[feature.workspace_config]\n"] {
let partial = WorkerManifestConfig::from_toml(source).unwrap();
assert!(
!FeatureConfig::from(partial.feature)
.workspace_config
.enabled
);
}
for source in ["{}", r#"{"workspace_config": {}}"#] {
let feature: FeatureConfig = serde_json::from_str(source).unwrap();
assert!(!feature.workspace_config.enabled);
}
}
#[test]
fn workspace_config_enabled_only_merges_and_roundtrips_snapshots() {
for enabled in [false, true] {
let lower = WorkerManifestConfig::from_toml(&format!(
"[feature.workspace_config]\nenabled = {enabled}\n"
))
.unwrap();
let empty = WorkerManifestConfig::from_toml("[feature.workspace_config]\n").unwrap();
let manifest: WorkerManifest = minimal_valid()
.merge(lower.clone())
.merge(empty)
.try_into()
.unwrap();
assert_eq!(manifest.feature.workspace_config.enabled, enabled);
assert!(!manifest.feature.manage_workdir.enabled);
let partial = FeatureConfigPartial::from(manifest.feature.clone());
assert_eq!(
partial.workspace_config.as_ref().unwrap().enabled,
Some(enabled)
);
let serialized = toml::to_string(&partial).unwrap();
let restored: FeatureConfigPartial = toml::from_str(&serialized).unwrap();
assert_eq!(FeatureConfig::from(restored), manifest.feature);
let json = serde_json::to_value(&manifest.feature).unwrap();
assert_eq!(
json["workspace_config"],
serde_json::json!({"enabled": enabled})
);
assert_eq!(
serde_json::from_value::<FeatureConfig>(json).unwrap(),
manifest.feature
);
let snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
assert_eq!(
crate::read_persisted_worker_manifest_snapshot(snapshot.clone())
.unwrap()
.feature
.workspace_config
.enabled,
enabled
);
let mut old_snapshot = snapshot;
old_snapshot["manifest"]["feature"]
.as_object_mut()
.unwrap()
.remove("workspace_config");
assert!(
!crate::read_persisted_worker_manifest_snapshot(old_snapshot)
.unwrap()
.feature
.workspace_config
.enabled
);
let upper = WorkerManifestConfig::from_toml(&format!(
"[feature.workspace_config]\nenabled = {}\n",
!enabled
))
.unwrap();
let manifest: WorkerManifest = minimal_valid()
.merge(lower)
.merge(upper)
.try_into()
.unwrap();
assert_eq!(manifest.feature.workspace_config.enabled, !enabled);
}
}
#[test]
fn workspace_config_rejects_manifest_authority_fields() {
for (field, value) in [
("grant_id", "\"grant-1\""),
("workspace_id", "\"workspace-1\""),
("access", "\"read_write\""),
("config_root", "\"/host/config\""),
("source", "\"config\""),
] {
let source = format!("[feature.workspace_config]\nenabled = true\n{field} = {value}\n");
assert!(WorkerManifestConfig::from_toml(&source).is_err(), "{field}");
assert!(
toml::from_str::<FeatureConfig>(&format!(
"[workspace_config]\nenabled = true\n{field} = {value}\n"
))
.is_err(),
"{field}"
);
}
}
#[test]
fn workdir_catalog_defaults_agree_for_partial_and_resolved_configs() {
assert!(FeatureConfig::default().workdir_catalog.enabled);
assert!(FeatureConfigPartial::default().workdir_catalog.is_none());
for source in ["", "[feature]\n", "[feature.workdir_catalog]\n"] {
let partial = WorkerManifestConfig::from_toml(source).unwrap();
let feature = FeatureConfig::from(partial.feature);
assert!(feature.workdir_catalog.enabled, "{source}");
let value: toml::Value = toml::from_str(source).unwrap();
let resolved: FeatureConfig = serde_json::from_value(
serde_json::to_value(
value
.get("feature")
.cloned()
.unwrap_or_else(|| toml::Value::Table(Default::default())),
)
.unwrap(),
)
.unwrap();
assert_eq!(feature, resolved);
}
for source in ["{}", r#"{"workdir_catalog": {}}"#] {
let feature: FeatureConfig = serde_json::from_str(source).unwrap();
assert!(feature.workdir_catalog.enabled);
assert!(!feature.manage_workdir.enabled);
}
}
#[test]
fn workdir_catalog_is_independent_and_merges_partial_layers() {
for catalog in [false, true] {
for manage in [false, true] {
let source = format!(
"[feature.workdir_catalog]\nenabled = {catalog}\n\
[feature.manage_workdir]\nenabled = {manage}\n"
);
let lower = WorkerManifestConfig::from_toml(&source).unwrap();
let empty = WorkerManifestConfig::from_toml("[feature.workdir_catalog]").unwrap();
let cfg = minimal_valid().merge(lower.clone()).merge(empty);
let manifest: WorkerManifest = cfg.try_into().unwrap();
assert_eq!(manifest.feature.workdir_catalog.enabled, catalog);
assert_eq!(manifest.feature.manage_workdir.enabled, manage);
let override_source =
format!("[feature.workdir_catalog]\nenabled = {}\n", !catalog);
let upper = WorkerManifestConfig::from_toml(&override_source).unwrap();
let manifest: WorkerManifest = minimal_valid()
.merge(lower)
.merge(upper)
.try_into()
.unwrap();
assert_eq!(manifest.feature.workdir_catalog.enabled, !catalog);
assert_eq!(manifest.feature.manage_workdir.enabled, manage);
}
}
}
#[test]
fn workdir_catalog_roundtrips_config_and_persisted_snapshots() {
for enabled in [false, true] {
let mut cfg = minimal_valid();
cfg.feature.workdir_catalog = Some(FeatureFlagConfigPartial {
enabled: Some(enabled),
});
let manifest: WorkerManifest = cfg.try_into().unwrap();
let partial = FeatureConfigPartial::from(manifest.feature.clone());
assert_eq!(
partial.workdir_catalog.as_ref().unwrap().enabled,
Some(enabled)
);
let serialized = toml::to_string(&partial).unwrap();
let partial: FeatureConfigPartial = toml::from_str(&serialized).unwrap();
assert_eq!(FeatureConfig::from(partial), manifest.feature);
let serialized = serde_json::to_value(&manifest.feature).unwrap();
assert_eq!(serialized["workdir_catalog"]["enabled"], enabled);
let feature: FeatureConfig = serde_json::from_value(serialized).unwrap();
assert_eq!(feature, manifest.feature);
let snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
let restored =
crate::read_persisted_worker_manifest_snapshot(snapshot.clone()).unwrap();
assert_eq!(restored.feature.workdir_catalog.enabled, enabled);
assert!(!restored.feature.manage_workdir.enabled);
let mut legacy = snapshot;
legacy["manifest"]["feature"]
.as_object_mut()
.unwrap()
.remove("workdir_catalog");
let restored = crate::read_persisted_worker_manifest_snapshot(legacy).unwrap();
assert!(restored.feature.workdir_catalog.enabled);
assert!(!restored.feature.manage_workdir.enabled);
}
}
#[test]
fn from_toml_parses_explicit_feature_flags() {
let cfg = WorkerManifestConfig::from_toml(
@@ -2248,6 +2546,7 @@ enabled = false
.merge(WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("feature-test".into()),
mode: None,
},
model: ModelManifest {
scheme: Some(SchemeKind::Anthropic),
@@ -2351,6 +2650,7 @@ enabled = true
.merge(WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("feature-merge-test".into()),
mode: None,
},
model: ModelManifest {
scheme: Some(SchemeKind::Anthropic),
@@ -2428,6 +2728,7 @@ permission = "write"
let overlay = WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("x".into()),
mode: None,
},
model: ModelManifest {
scheme: Some(SchemeKind::Anthropic),
+4
View File
@@ -6,6 +6,10 @@
//! `pub const`. Both paths read from this module, so changing a
//! default requires editing exactly one line.
/// Read-only Repository/Workdir/attachment reference discovery in WIP mode.
/// This enables a reference provider, not normal Tools or any authority grant.
pub const WORKDIR_CATALOG_ENABLED: bool = true;
/// Byte-size cap applied to any tool's `content` output when no
/// per-tool override is set. See [`crate::ToolOutputLimits`].
pub const TOOL_OUTPUT_MAX_BYTES: usize = 64 * 1024;
+59 -2
View File
@@ -65,8 +65,9 @@ pub struct WorkerManifest {
/// permission layer is disabled and tool calls run as before.
#[serde(default)]
pub permissions: Option<ToolPermissionConfig>,
/// Explicit built-in feature/tool-surface enablement. Omitted feature flags
/// resolve disabled so Profile authors choose the exposed built-in surfaces.
/// Built-in feature/tool-surface enablement. Omitted tool features resolve
/// disabled; the read-only WIP workdir catalog defaults enabled without
/// granting authority or registering normal Tools.
#[serde(default)]
pub feature: FeatureConfig,
/// Explicit external Model Context Protocol provider configuration. This
@@ -131,6 +132,19 @@ pub struct FeatureConfig {
pub objective: FeatureFlagConfig,
#[serde(default)]
pub manage_workdir: FeatureFlagConfig,
/// Read-only Repository/Workdir/attachment reference discovery. Defaults
/// enabled, is installed only in WIP mode, and exposes no normal Tools.
/// Independent of `manage_workdir`; permissions and host authority remain
/// separate, and discovering a reference grants no attachment or file access.
#[serde(
default = "default_workdir_catalog",
deserialize_with = "deserialize_workdir_catalog"
)]
pub workdir_catalog: FeatureFlagConfig,
/// Opt-in WIP-only logical Workspace configuration feature. This flag is
/// not an access grant; Backend-owned Worker grants remain authoritative.
#[serde(default, deserialize_with = "deserialize_workspace_config")]
pub workspace_config: FeatureFlagConfig,
#[serde(default)]
pub ticket: TicketFeatureConfig,
#[serde(default)]
@@ -139,6 +153,29 @@ pub struct FeatureConfig {
pub orchestration: FeatureFlagConfig,
}
fn default_workdir_catalog() -> FeatureFlagConfig {
FeatureFlagConfig {
enabled: defaults::WORKDIR_CATALOG_ENABLED,
}
}
fn deserialize_workdir_catalog<'de, D>(deserializer: D) -> Result<FeatureFlagConfig, D::Error>
where
D: serde::Deserializer<'de>,
{
let flag = config::FeatureFlagConfigPartial::deserialize(deserializer)?;
Ok(FeatureFlagConfig {
enabled: flag.enabled.unwrap_or(defaults::WORKDIR_CATALOG_ENABLED),
})
}
fn deserialize_workspace_config<'de, D>(deserializer: D) -> Result<FeatureFlagConfig, D::Error>
where
D: serde::Deserializer<'de>,
{
Ok(config::WorkspaceConfigFeatureConfigPartial::deserialize(deserializer)?.into())
}
impl Default for FeatureConfig {
fn default() -> Self {
Self {
@@ -153,6 +190,8 @@ impl Default for FeatureConfig {
workspace_worker_discovery: FeatureFlagConfig::disabled(),
objective: FeatureFlagConfig::disabled(),
manage_workdir: FeatureFlagConfig::disabled(),
workdir_catalog: default_workdir_catalog(),
workspace_config: FeatureFlagConfig::disabled(),
ticket: TicketFeatureConfig::default(),
merge_request: MergeRequestFeatureConfig::default(),
orchestration: FeatureFlagConfig::disabled(),
@@ -363,6 +402,8 @@ impl ResolvedMemoryFeatureConfig {
#[serde(default, deny_unknown_fields)]
pub struct SubjektivFeatureProfileConfig {
pub enabled: bool,
/// Explicit consolidation tool policy; never grants Job or subject authority.
pub consolidation_tools: bool,
pub extraction: MemoryExtractionProfileConfig,
}
@@ -370,6 +411,7 @@ impl Default for SubjektivFeatureProfileConfig {
fn default() -> Self {
Self {
enabled: false,
consolidation_tools: false,
extraction: MemoryExtractionProfileConfig::default(),
}
}
@@ -670,6 +712,21 @@ pub struct WorkspaceMemorySettingsSnapshot {
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WorkerMeta {
pub name: String,
/// Model-facing capability surface. Existing manifests default to ordinary tools.
#[serde(default)]
pub mode: WorkerMode,
}
/// Model-facing Worker capability surface.
///
/// WIP is explicit opt-in: it replaces individual model-visible tools with the
/// discover/inspect/call Worldspace client while retaining the same host authority.
#[derive(Debug, Clone, Copy, Default, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum WorkerMode {
#[default]
Tools,
Wip,
}
/// When the engine may begin executing provider-confirmed tool calls.
+100 -108
View File
@@ -181,13 +181,17 @@ pub fn validate_profile_execution_target(
if feature.flow.enabled {
requirements.insert(WorkspaceAuthorityRequirement::Flow);
}
// workdir_catalog is a WIP-only reference provider, not Workspace authority
// or normal Tools. It must not prevent standalone Tools-mode launches.
if feature.manage_workdir.enabled {
requirements.insert(WorkspaceAuthorityRequirement::ManageWorkdir);
}
if feature.memory.profile.enabled || feature.memory.profile.staging_tools {
requirements.insert(WorkspaceAuthorityRequirement::Memory);
}
if feature.subjektiv.profile.enabled {
// Subject policy is Host-neutral. Only an already bound Backend snapshot
// requires Workspace authority; a local Host supplies its own live capability.
if feature.subjektiv.workspace_settings.is_some() {
requirements.insert(WorkspaceAuthorityRequirement::Subjektiv);
}
if feature.merge_request.show
@@ -626,6 +630,7 @@ fn resolve_profile_value(
let config = WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some(worker_name),
mode: profile.worker.and_then(|worker| worker.mode),
},
model: profile.model.unwrap_or_default(),
engine: profile.engine.unwrap_or_default(),
@@ -668,6 +673,8 @@ struct ProfileConfig {
#[serde(default)]
description: Option<String>,
#[serde(default)]
worker: Option<ProfileWorkerConfig>,
#[serde(default)]
model: Option<ModelManifest>,
#[serde(default)]
engine: Option<EngineManifestConfig>,
@@ -691,6 +698,13 @@ struct ProfileConfig {
skills: Option<SkillsConfig>,
}
#[derive(Debug, Default, Deserialize)]
#[serde(deny_unknown_fields)]
struct ProfileWorkerConfig {
#[serde(default)]
mode: Option<crate::WorkerMode>,
}
#[derive(Debug, Clone, Deserialize)]
#[serde(untagged)]
enum ProfileScopeConfig {
@@ -864,8 +878,15 @@ fn reject_manifest_shaped_profile(value: &serde_json::Value) -> Result<(), Profi
)));
}
}
if map.contains_key("worker") {
return Err(ProfileError::InvalidProfile("field `worker` is runtime-bound and is not allowed in reusable Profiles; pass the Worker name via CLI/TUI runtime inputs".into()));
if let Some(worker) = map.get("worker") {
let Some(worker) = worker.as_object() else {
return Err(ProfileError::InvalidProfile(
"field `worker` must be an object containing only reusable `mode` policy".into(),
));
};
if worker.contains_key("name") {
return Err(ProfileError::InvalidProfile("field `worker.name` is runtime-bound and is not allowed in reusable Profiles; pass the Worker name via CLI/TUI runtime inputs".into()));
}
}
if let Some(scope) = map.get("scope").and_then(|v| v.as_object()) {
for key in ["allow", "deny"] {
@@ -1238,22 +1259,23 @@ mod tests {
)
.unwrap();
assert_eq!(resolved.manifest.worker.name, "role-worker");
let compaction = resolved.manifest.compaction.as_ref().unwrap();
assert_eq!(compaction.worker_max_turns, Some(100), "{}", entry.name);
assert_eq!(
compaction.worker_context_max_tokens, 100_000,
"{}",
entry.name
);
assert_eq!(
compaction
.model
.as_ref()
.and_then(|model| model.ref_.as_deref()),
Some("codex-oauth/gpt-6-luna"),
"{}",
entry.name
);
if matches!(
entry.name.as_str(),
"job" | "standalone-subjektiv-consolidation"
) {
assert!(resolved.manifest.compaction.is_none());
assert!(resolved.manifest.scope.allow.is_empty());
assert!(resolved.manifest.delegation_scope.allow.is_empty());
assert_eq!(
resolved.manifest.engine.instruction,
if entry.name == "job" {
"internal.job_system"
} else {
"internal.subjektiv_memory_consolidation_system"
}
);
continue;
}
}
}
@@ -1315,90 +1337,6 @@ mod tests {
assert!(!legacy_consolidator.feature.subjektiv.profile.enabled);
}
#[test]
fn builtin_profiles_pin_role_models_and_reasoning() {
use crate::model::{ReasoningControl, ReasoningEffort};
let tmp = TempDir::new().unwrap();
let resolve = |name: &str| {
ProfileResolver::new()
.with_workspace_base(tmp.path())
.resolve(
&ProfileSelector::source_named(ProfileRegistrySource::Builtin, name),
ProfileResolveOptions::with_worker_name(format!("{name}-worker")),
)
.unwrap()
};
for name in [
"default",
"standalone",
"intake",
"orchestrator",
"companion",
"coder",
"reviewer",
"memory-consolidation",
"subjektiv-memory-consolidation",
] {
let role = resolve(name);
assert_eq!(
role.manifest
.compaction
.as_ref()
.map(|compaction| compaction.prune_enabled),
Some(false),
"{name} must keep pruning disabled for the experiment"
);
}
let companion = resolve("companion");
assert_eq!(
companion.manifest.model.ref_.as_deref(),
Some("codex-oauth/gpt-6-astra")
);
assert_eq!(
companion.manifest.engine.reasoning,
Some(ReasoningControl::Effort(ReasoningEffort::High))
);
for name in ["coder", "reviewer"] {
let role = resolve(name);
assert_eq!(
role.manifest.model.ref_.as_deref(),
Some("codex-oauth/gpt-5.6-sol")
);
assert_eq!(
role.manifest.engine.reasoning,
Some(ReasoningControl::Effort(ReasoningEffort::High))
);
let extraction = &role.manifest.feature.subjektiv.profile.extraction;
assert_eq!(
extraction
.model
.as_ref()
.and_then(|model| model.ref_.as_deref()),
Some("codex-oauth/gpt-5.6-luna")
);
assert_eq!(
extraction.reasoning,
Some(ReasoningControl::Effort(ReasoningEffort::Medium))
);
}
for name in ["memory-consolidation", "subjektiv-memory-consolidation"] {
let consolidation = resolve(name);
assert_eq!(
consolidation.manifest.model.ref_.as_deref(),
Some("codex-oauth/gpt-5.6-luna")
);
assert_eq!(
consolidation.manifest.engine.reasoning,
Some(ReasoningControl::Effort(ReasoningEffort::Medium))
);
}
}
#[test]
fn subjektiv_memory_consolidation_profile_is_lifecycle_and_capability_isolated() {
let tmp = TempDir::new().unwrap();
@@ -1543,10 +1481,6 @@ mod tests {
..
} if name == "standalone" && provenance.starts_with("profiles/standalone.dcdl#sha256:")
));
assert_eq!(
resolved.manifest.model.ref_.as_deref(),
Some("codex-oauth/gpt-5.6-sol")
);
assert!(resolved.manifest.feature.task.enabled);
assert!(resolved.manifest.feature.web.enabled);
assert!(resolved.manifest.feature.image.enabled);
@@ -1569,6 +1503,59 @@ mod tests {
assert!(!resolved.manifest.feature.manage_workdir.enabled);
}
#[test]
fn workdir_catalog_does_not_require_workspace_authority_or_grant_scope() {
let tmp = TempDir::new().unwrap();
for catalog in [None, Some(false), Some(true)] {
let mut artifact = serde_json::json!({
"worker": { "mode": "tools" },
"feature": { "manage_workdir": { "enabled": false } },
"permissions": { "default_action": "deny" }
});
if let Some(enabled) = catalog {
artifact["feature"]["workdir_catalog"] = serde_json::json!({ "enabled": enabled });
}
let resolved = resolve_profile_artifact_value(
artifact,
ProfileSource::Archive {
archive_id: "catalog-test".into(),
source: "test".into(),
},
tmp.path(),
"catalog-worker",
)
.unwrap();
let manifest = resolved.manifest;
assert_eq!(
manifest.feature.workdir_catalog.enabled,
catalog.unwrap_or(true)
);
assert!(!manifest.feature.manage_workdir.enabled);
assert_eq!(manifest.worker.mode, crate::WorkerMode::Tools);
assert!(manifest.scope.allow.is_empty());
assert!(manifest.delegation_scope.allow.is_empty());
assert_eq!(
manifest.permissions.as_ref().unwrap().default_action,
crate::ToolPermissionAction::Deny
);
validate_profile_execution_target(&manifest, ProfileExecutionTarget::Standalone)
.unwrap();
let mut manage_manifest = manifest;
manage_manifest.feature.manage_workdir.enabled = true;
let error = validate_profile_execution_target(
&manage_manifest,
ProfileExecutionTarget::Standalone,
)
.unwrap_err();
assert!(matches!(
error,
ProfileError::UnsupportedExecutionTarget { requirements, .. }
if requirements == vec![WorkspaceAuthorityRequirement::ManageWorkdir]
));
}
}
#[test]
fn standalone_rejects_profiles_that_require_workspace_authority() {
let tmp = TempDir::new().unwrap();
@@ -1590,7 +1577,10 @@ mod tests {
panic!("unexpected error: {error}");
};
assert_eq!(target, ProfileExecutionTarget::Standalone);
assert!(requirements.contains(&WorkspaceAuthorityRequirement::Subjektiv));
assert!(
!requirements.contains(&WorkspaceAuthorityRequirement::Subjektiv),
"subject policy alone is no longer Backend authority"
);
assert!(!requirements.contains(&WorkspaceAuthorityRequirement::Memory));
assert!(requirements.contains(&WorkspaceAuthorityRequirement::MergeRequest));
assert!(requirements.contains(&WorkspaceAuthorityRequirement::Ticket));
@@ -1691,6 +1681,7 @@ mod tests {
r#"
slug = "coder"
scope = "workspace_read"
worker = { mode = "wip" }
[model]
scheme = "anthropic"
@@ -1710,6 +1701,7 @@ reasoning = "high"
)
.unwrap();
assert_eq!(resolved.manifest.worker.name, "runtime-worker");
assert_eq!(resolved.manifest.worker.mode, crate::WorkerMode::Wip);
assert_eq!(resolved.manifest.model.scheme, Some(SchemeKind::Anthropic));
assert_eq!(
resolved.manifest.engine.reasoning,
+1
View File
@@ -14,6 +14,7 @@ json-schema = ["dep:schemars"]
schemars = { workspace = true, optional = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
thiserror = { workspace = true }
sha2.workspace = true
tokio = { workspace = true, features = ["io-util"], optional = true }
ts-rs = { version = "12.0.1", optional = true }
+949
View File
@@ -0,0 +1,949 @@
//! Declarative chat Feature invocation contract.
//!
//! The input spelling is deliberately bounded and unambiguous:
//! `/name(arg=value, "positional value")`. Parentheses terminate arguments,
//! so following prose is never consumed. Strings use JSON-style quoting and
//! escaping. Unquoted values are limited to booleans, integers, enum-like words,
//! and paths without comma/parenthesis/whitespace. This module parses input for
//! clients and validates the structured payload again at the host boundary.
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, BTreeSet};
use std::fmt;
/// UTF-8 byte bound shared by input validation and durable recovery keys.
pub const MAX_FEATURE_INVOCATION_ID_BYTES: usize = 128;
/// Installed descriptors must fit a public completion entry without truncation.
pub const MAX_FEATURE_INVOCATION_DESCRIPTOR_BYTES: usize = 32 * 1024;
/// Source-qualified identity of one invocation contribution, for example
/// `builtin:attachments/attach`. Display names and aliases are never authority.
#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct FeatureInvocationIdentity(pub String);
impl fmt::Display for FeatureInvocationIdentity {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str(&self.0)
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum FeatureInvocationSyntax {
/// `/name(value, key=value)` with JSON-style quoted strings and `)` as the
/// explicit boundary before following natural language.
Parenthesized,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum InvocationArgumentType {
String,
Integer,
Boolean,
Enum {
values: Vec<String>,
},
/// A Worker-readable path. This does not grant filesystem authority.
WorkerFile,
/// A resource selected and staged by the client. It must never be resolved
/// as a path on the Worker host.
ClientFile,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum InvocationCompletion {
None,
Static {
values: Vec<String>,
},
WorkerFile,
ClientFile,
/// Host-owned provider selected by source-qualified id. Metadata lookup is
/// side-effect free; implementations never cross this wire boundary.
Provider {
provider: String,
},
}
impl Default for InvocationCompletion {
fn default() -> Self {
Self::None
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct InvocationArgumentDescriptor {
pub name: String,
/// Zero-based positional index. `None` means named-only. A positional
/// argument may also be supplied by name; this makes chip editing stable.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub position: Option<u16>,
#[serde(default)]
pub required: bool,
pub value_type: InvocationArgumentType,
#[serde(default)]
pub completion: InvocationCompletion,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum InvocationClientAdapter {
/// The client stages a local file and replaces the draft invocation with an
/// `UploadedFile` chip. Local paths are not sent to or persisted by Worker.
Attachment,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct FeatureInvocationDescriptor {
pub identity: FeatureInvocationIdentity,
pub name: String,
#[serde(default)]
pub aliases: Vec<String>,
pub display_name: String,
pub description: String,
pub syntax: FeatureInvocationSyntax,
#[serde(default)]
pub arguments: Vec<InvocationArgumentDescriptor>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub client_adapter: Option<InvocationClientAdapter>,
}
impl FeatureInvocationDescriptor {
pub fn usage(&self) -> String {
let mut usage = format!("/{}(", self.name);
for (index, argument) in self.arguments.iter().enumerate() {
if index > 0 {
usage.push_str(", ");
}
if argument.position.is_none() {
usage.push_str(&argument.name);
usage.push('=');
}
let marker = match &argument.value_type {
InvocationArgumentType::String => "text",
InvocationArgumentType::Integer => "integer",
InvocationArgumentType::Boolean => "true|false",
InvocationArgumentType::Enum { .. } => "choice",
InvocationArgumentType::WorkerFile => "worker-path",
InvocationArgumentType::ClientFile => "local-file",
};
if argument.required {
usage.push('<');
usage.push_str(marker);
usage.push('>');
} else {
usage.push('[');
usage.push_str(marker);
usage.push(']');
}
}
usage.push(')');
usage
}
pub fn validate(&self) -> Result<(), InvocationValidationError> {
const MAX_PUBLIC_NAMES: usize = 16;
const MAX_ARGUMENTS: usize = 32;
const MAX_METADATA_BYTES: usize = 4096;
validate_invocation_name(&self.name)?;
if self.aliases.len() > MAX_PUBLIC_NAMES || self.arguments.len() > MAX_ARGUMENTS {
return Err(InvocationValidationError::InvalidDescriptor(
"invocation descriptor exceeds collection limits".into(),
));
}
if self.display_name.len() > MAX_METADATA_BYTES
|| self.description.len() > MAX_METADATA_BYTES
{
return Err(InvocationValidationError::InvalidDescriptor(
"invocation metadata exceeds byte limits".into(),
));
}
if self.identity.0.trim().is_empty()
|| !self.identity.0.contains(':')
|| self.identity.0.len() > 256
{
return Err(InvocationValidationError::InvalidDescriptor(
"invocation identity must be source-qualified".into(),
));
}
let mut names = BTreeSet::new();
let mut positions = BTreeSet::new();
for name in std::iter::once(&self.name).chain(self.aliases.iter()) {
validate_invocation_name(name)?;
if !names.insert(name) {
return Err(InvocationValidationError::InvalidDescriptor(format!(
"duplicate invocation name or alias `{name}`"
)));
}
}
let mut argument_names = BTreeSet::new();
for argument in &self.arguments {
validate_argument_name(&argument.name)?;
if !argument_names.insert(argument.name.as_str()) {
return Err(InvocationValidationError::InvalidDescriptor(format!(
"duplicate argument `{}`",
argument.name
)));
}
if let Some(position) = argument.position
&& !positions.insert(position)
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"duplicate positional index `{position}`"
)));
}
if argument
.description
.as_ref()
.is_some_and(|value| value.len() > MAX_METADATA_BYTES)
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"argument `{}` description exceeds byte limits",
argument.name
)));
}
if let InvocationCompletion::Provider { provider } = &argument.completion
&& (provider.trim().is_empty() || provider.len() > 256)
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"argument `{}` has an invalid completion provider",
argument.name
)));
}
if matches!(&argument.value_type, InvocationArgumentType::Enum { values }
if values.len() > 256 || values.iter().any(|value| value.len() > MAX_METADATA_BYTES))
|| matches!(&argument.completion, InvocationCompletion::Static { values }
if values.len() > 256 || values.iter().any(|value| value.len() > MAX_METADATA_BYTES))
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"argument `{}` completion values exceed limits",
argument.name
)));
}
if matches!(&argument.value_type, InvocationArgumentType::Enum { values } if values.is_empty())
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"enum argument `{}` has no values",
argument.name
)));
}
}
for (expected, actual) in positions.iter().copied().enumerate() {
if actual as usize != expected {
return Err(InvocationValidationError::InvalidDescriptor(
"positional argument indices must be contiguous from zero".into(),
));
}
}
if serde_json::to_vec(self).map_or(true, |encoded| {
encoded.len() > MAX_FEATURE_INVOCATION_DESCRIPTOR_BYTES
}) {
return Err(InvocationValidationError::InvalidDescriptor(
"invocation descriptor exceeds the public completion byte limit".into(),
));
}
Ok(())
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", content = "value", rename_all = "snake_case")]
pub enum InvocationValue {
String(String),
Integer(i32),
Boolean(bool),
}
impl InvocationValue {
pub fn display_input(&self) -> String {
match self {
Self::String(value) => quote_invocation_string(value),
Self::Integer(value) => value.to_string(),
Self::Boolean(value) => value.to_string(),
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct InvocationArgumentValue {
pub name: String,
pub value: InvocationValue,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct FeatureInvocation {
/// Stable client-generated id retained across queueing and transport retry.
pub invocation_id: String,
pub identity: FeatureInvocationIdentity,
/// Selected public name, retained for exact draft/history restoration. Host
/// resolution is always by `identity`.
pub name: String,
pub arguments: Vec<InvocationArgumentValue>,
}
impl FeatureInvocation {
pub fn display_input(&self) -> String {
let mut input = format!("/{}(", self.name);
for (index, argument) in self.arguments.iter().enumerate() {
if index > 0 {
input.push_str(", ");
}
input.push_str(&argument.name);
input.push('=');
input.push_str(&argument.value.display_input());
}
input.push(')');
input
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum FeatureInvocationStatus {
Succeeded,
Failed,
OutcomeUnknown,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct FeatureInvocationResult {
pub invocation_id: String,
pub identity: FeatureInvocationIdentity,
pub status: FeatureInvocationStatus,
pub message: String,
/// Bounded model-visible context produced by the handler. Credentials and
/// executable implementation details must not be included.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ParsedFeatureInvocation {
pub invocation: FeatureInvocation,
/// UTF-8 byte offset immediately after the closing parenthesis.
pub end: usize,
}
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum InvocationValidationError {
#[error("invalid invocation descriptor: {0}")]
InvalidDescriptor(String),
#[error("incomplete invocation at byte {0}")]
Incomplete(usize),
#[error("unknown argument `{0}`")]
UnknownArgument(String),
#[error("duplicate argument `{0}`")]
DuplicateArgument(String),
#[error("missing required argument `{0}`")]
MissingRequired(String),
#[error("too many positional arguments")]
TooManyPositional,
#[error("invalid value for `{argument}`: {message}")]
InvalidValue { argument: String, message: String },
#[error("invocation identity does not match descriptor")]
IdentityMismatch,
#[error("invocation name does not match descriptor")]
NameMismatch,
#[error("invalid invocation syntax at byte {offset}: {message}")]
Syntax { offset: usize, message: String },
}
pub fn validate_feature_invocation(
descriptor: &FeatureInvocationDescriptor,
invocation: &FeatureInvocation,
) -> Result<(), InvocationValidationError> {
descriptor.validate()?;
if invocation.identity != descriptor.identity {
return Err(InvocationValidationError::IdentityMismatch);
}
if invocation.name != descriptor.name && !descriptor.aliases.contains(&invocation.name) {
return Err(InvocationValidationError::NameMismatch);
}
if invocation.invocation_id.trim().is_empty()
|| invocation.invocation_id.len() > MAX_FEATURE_INVOCATION_ID_BYTES
{
return Err(InvocationValidationError::InvalidValue {
argument: "invocation_id".into(),
message: "must be nonempty and at most 128 UTF-8 bytes".into(),
});
}
if invocation.arguments.len() > 32 {
return Err(InvocationValidationError::InvalidValue {
argument: "arguments".into(),
message: "too many arguments".into(),
});
}
let mut values = BTreeMap::new();
for argument in &invocation.arguments {
if values
.insert(argument.name.as_str(), &argument.value)
.is_some()
{
return Err(InvocationValidationError::DuplicateArgument(
argument.name.clone(),
));
}
}
for name in values.keys() {
if !descriptor
.arguments
.iter()
.any(|argument| argument.name == **name)
{
return Err(InvocationValidationError::UnknownArgument(
(*name).to_string(),
));
}
}
for argument in &descriptor.arguments {
match values.get(argument.name.as_str()) {
Some(value) => validate_value(argument, value)?,
None if argument.required => {
return Err(InvocationValidationError::MissingRequired(
argument.name.clone(),
));
}
None => {}
}
}
Ok(())
}
/// Parse one complete parenthesized invocation starting at `start`. Callers
/// decide when input becomes a typed chip; parsing alone never executes it.
pub fn parse_feature_invocation(
input: &str,
start: usize,
descriptor: &FeatureInvocationDescriptor,
invocation_id: impl Into<String>,
) -> Result<ParsedFeatureInvocation, InvocationValidationError> {
descriptor.validate()?;
if !input.is_char_boundary(start) || input.as_bytes().get(start) != Some(&b'/') {
return Err(InvocationValidationError::Syntax {
offset: start,
message: "expected `/`".into(),
});
}
let mut cursor = start + 1;
let name_start = cursor;
while let Some(byte) = input.as_bytes().get(cursor) {
if byte.is_ascii_alphanumeric() || matches!(*byte, b'-' | b'_') {
cursor += 1;
} else {
break;
}
}
let name = &input[name_start..cursor];
if name != descriptor.name && !descriptor.aliases.iter().any(|alias| alias == name) {
return Err(InvocationValidationError::NameMismatch);
}
skip_ws(input, &mut cursor);
if input.as_bytes().get(cursor) != Some(&b'(') {
return Err(InvocationValidationError::Incomplete(cursor));
}
cursor += 1;
let positional = descriptor
.arguments
.iter()
.filter_map(|argument| argument.position.map(|position| (position, argument)))
.collect::<BTreeMap<_, _>>();
let mut next_position = 0u16;
let mut parsed = Vec::<InvocationArgumentValue>::new();
loop {
skip_ws(input, &mut cursor);
match input.as_bytes().get(cursor) {
Some(b')') => {
cursor += 1;
break;
}
None => return Err(InvocationValidationError::Incomplete(cursor)),
_ => {}
}
let token_start = cursor;
let possible_name = parse_identifier(input, &mut cursor);
let after_name = cursor;
skip_ws(input, &mut cursor);
let (argument, raw_value) = if !possible_name.is_empty()
&& input.as_bytes().get(cursor) == Some(&b'=')
{
cursor += 1;
skip_ws(input, &mut cursor);
let argument = descriptor
.arguments
.iter()
.find(|argument| argument.name == possible_name)
.ok_or_else(|| InvocationValidationError::UnknownArgument(possible_name.into()))?;
(argument, parse_raw_value(input, &mut cursor)?)
} else {
cursor = token_start;
let argument = positional
.get(&next_position)
.copied()
.ok_or(InvocationValidationError::TooManyPositional)?;
next_position += 1;
(argument, parse_raw_value(input, &mut cursor)?)
};
if parsed.iter().any(|value| value.name == argument.name) {
return Err(InvocationValidationError::DuplicateArgument(
argument.name.clone(),
));
}
let value = coerce_value(argument, raw_value)?;
parsed.push(InvocationArgumentValue {
name: argument.name.clone(),
value,
});
skip_ws(input, &mut cursor);
match input.as_bytes().get(cursor) {
Some(b',') => cursor += 1,
Some(b')') => {}
None => return Err(InvocationValidationError::Incomplete(cursor)),
_ => {
return Err(InvocationValidationError::Syntax {
offset: after_name.max(cursor),
message: "expected `,` or `)`".into(),
});
}
}
}
let invocation = FeatureInvocation {
invocation_id: invocation_id.into(),
identity: descriptor.identity.clone(),
name: name.to_string(),
arguments: parsed,
};
validate_feature_invocation(descriptor, &invocation)?;
Ok(ParsedFeatureInvocation {
invocation,
end: cursor,
})
}
#[derive(Debug)]
enum RawValue {
Quoted(String),
Bare(String),
}
fn parse_raw_value(input: &str, cursor: &mut usize) -> Result<RawValue, InvocationValidationError> {
if input.as_bytes().get(*cursor) == Some(&b'"') {
let start = *cursor;
*cursor += 1;
let mut escaped = false;
loop {
let Some(ch) = input[*cursor..].chars().next() else {
return Err(InvocationValidationError::Incomplete(*cursor));
};
*cursor += ch.len_utf8();
if ch == '"' && !escaped {
let value =
serde_json::from_str::<String>(&input[start..*cursor]).map_err(|error| {
InvocationValidationError::Syntax {
offset: start,
message: error.to_string(),
}
})?;
return Ok(RawValue::Quoted(value));
}
escaped = ch == '\\' && !escaped;
}
}
let start = *cursor;
while let Some(ch) = input[*cursor..].chars().next() {
if ch == ',' || ch == ')' || ch.is_whitespace() {
break;
}
*cursor += ch.len_utf8();
}
if *cursor == start {
return Err(InvocationValidationError::Syntax {
offset: start,
message: "expected argument value".into(),
});
}
Ok(RawValue::Bare(input[start..*cursor].to_string()))
}
fn coerce_value(
descriptor: &InvocationArgumentDescriptor,
raw: RawValue,
) -> Result<InvocationValue, InvocationValidationError> {
let text = match raw {
RawValue::Quoted(value) | RawValue::Bare(value) => value,
};
let invalid = |message: &str| InvocationValidationError::InvalidValue {
argument: descriptor.name.clone(),
message: message.into(),
};
match &descriptor.value_type {
InvocationArgumentType::String
| InvocationArgumentType::WorkerFile
| InvocationArgumentType::ClientFile => Ok(InvocationValue::String(text)),
InvocationArgumentType::Integer => {
let digits = text.strip_prefix('-').unwrap_or(&text);
if digits.is_empty() || !digits.bytes().all(|byte| byte.is_ascii_digit()) {
return Err(invalid("expected a decimal integer"));
}
text.parse::<i32>()
.map(InvocationValue::Integer)
.map_err(|_| invalid("expected a signed 32-bit integer"))
}
InvocationArgumentType::Boolean => match text.as_str() {
"true" => Ok(InvocationValue::Boolean(true)),
"false" => Ok(InvocationValue::Boolean(false)),
_ => Err(invalid("expected `true` or `false`")),
},
InvocationArgumentType::Enum { values } => {
if values.contains(&text) {
Ok(InvocationValue::String(text))
} else {
Err(invalid("value is not in the declared enum"))
}
}
}
}
fn validate_value(
descriptor: &InvocationArgumentDescriptor,
value: &InvocationValue,
) -> Result<(), InvocationValidationError> {
let valid = match (&descriptor.value_type, value) {
(
InvocationArgumentType::String
| InvocationArgumentType::WorkerFile
| InvocationArgumentType::ClientFile,
InvocationValue::String(value),
) => value.len() <= 16 * 1024,
(InvocationArgumentType::Integer, InvocationValue::Integer(_)) => true,
(InvocationArgumentType::Boolean, InvocationValue::Boolean(_)) => true,
(InvocationArgumentType::Enum { values }, InvocationValue::String(value)) => {
values.contains(value)
}
_ => false,
};
if valid {
Ok(())
} else {
Err(InvocationValidationError::InvalidValue {
argument: descriptor.name.clone(),
message: "structured value does not match the declared type".into(),
})
}
}
fn validate_invocation_name(name: &str) -> Result<(), InvocationValidationError> {
if name.is_empty()
|| !name
.bytes()
.all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'-')
|| !name.as_bytes()[0].is_ascii_lowercase()
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"invalid invocation name `{name}`"
)));
}
Ok(())
}
fn validate_argument_name(name: &str) -> Result<(), InvocationValidationError> {
if name.is_empty()
|| !name
.bytes()
.all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'_')
|| !name.as_bytes()[0].is_ascii_lowercase()
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"invalid argument name `{name}`"
)));
}
Ok(())
}
fn parse_identifier<'a>(input: &'a str, cursor: &mut usize) -> &'a str {
let start = *cursor;
while let Some(byte) = input.as_bytes().get(*cursor) {
if byte.is_ascii_lowercase() || byte.is_ascii_digit() || *byte == b'_' {
*cursor += 1;
} else {
break;
}
}
&input[start..*cursor]
}
fn skip_ws(input: &str, cursor: &mut usize) {
while let Some(ch) = input[*cursor..].chars().next() {
if !ch.is_whitespace() {
break;
}
*cursor += ch.len_utf8();
}
}
pub fn quote_invocation_string(value: &str) -> String {
// Keep the spelling identical to Web's JSON.stringify, including control
// characters and JSON unicode escapes accepted by both common parsers.
serde_json::to_string(value).expect("serializing a string cannot fail")
}
#[cfg(test)]
mod tests {
use super::*;
fn descriptor() -> FeatureInvocationDescriptor {
FeatureInvocationDescriptor {
identity: FeatureInvocationIdentity("builtin:test/run".into()),
name: "run".into(),
aliases: vec!["execute".into()],
display_name: "Run".into(),
description: "test invocation".into(),
syntax: FeatureInvocationSyntax::Parenthesized,
arguments: vec![
InvocationArgumentDescriptor {
name: "path".into(),
position: Some(0),
required: true,
value_type: InvocationArgumentType::String,
completion: InvocationCompletion::WorkerFile,
description: None,
},
InvocationArgumentDescriptor {
name: "mode".into(),
position: None,
required: false,
value_type: InvocationArgumentType::Enum {
values: vec!["fast".into(), "safe".into()],
},
completion: InvocationCompletion::Static {
values: vec!["fast".into(), "safe".into()],
},
description: None,
},
],
client_adapter: None,
}
}
#[test]
fn parses_unicode_quoted_paths_and_stops_before_natural_language() {
let input = "/run(\"資料/my file / x\", mode=safe) この後は本文";
let parsed = parse_feature_invocation(input, 0, &descriptor(), "invoke-1").unwrap();
assert_eq!(&input[parsed.end..], " この後は本文");
assert_eq!(
parsed.invocation.arguments,
vec![
InvocationArgumentValue {
name: "path".into(),
value: InvocationValue::String("資料/my file / x".into()),
},
InvocationArgumentValue {
name: "mode".into(),
value: InvocationValue::String("safe".into()),
},
]
);
}
#[test]
fn reports_incomplete_unknown_duplicate_missing_and_type_errors() {
assert!(matches!(
parse_feature_invocation("/run(\"x\"", 0, &descriptor(), "1"),
Err(InvocationValidationError::Incomplete(_))
));
assert!(matches!(
parse_feature_invocation("/run(path=x, nope=y)", 0, &descriptor(), "1"),
Err(InvocationValidationError::UnknownArgument(name)) if name == "nope"
));
assert!(matches!(
parse_feature_invocation("/run(path=x, path=y)", 0, &descriptor(), "1"),
Err(InvocationValidationError::DuplicateArgument(name)) if name == "path"
));
assert!(matches!(
parse_feature_invocation("/run(mode=fast)", 0, &descriptor(), "1"),
Err(InvocationValidationError::MissingRequired(name)) if name == "path"
));
assert!(matches!(
parse_feature_invocation("/run(x, mode=unknown)", 0, &descriptor(), "1"),
Err(InvocationValidationError::InvalidValue { argument, .. }) if argument == "mode"
));
}
#[test]
fn descriptors_cannot_install_with_undiscoverable_oversized_metadata() {
let mut descriptor = descriptor();
descriptor.arguments[0].completion = InvocationCompletion::Static {
values: vec!["a".repeat(1024); 40],
};
assert!(
matches!(descriptor.validate(), Err(InvocationValidationError::InvalidDescriptor(message)) if message.contains("completion byte limit"))
);
}
#[test]
fn client_and_recovery_invocation_key_bound_is_shared() {
let descriptor = descriptor();
assert!(
parse_feature_invocation(
"/run(x)",
0,
&descriptor,
"a".repeat(MAX_FEATURE_INVOCATION_ID_BYTES)
)
.is_ok()
);
assert!(
parse_feature_invocation(
"/run(x)",
0,
&descriptor,
"a".repeat(MAX_FEATURE_INVOCATION_ID_BYTES + 1)
)
.is_err()
);
assert!(parse_feature_invocation("/run(x)", 0, &descriptor, "界".repeat(43)).is_err());
}
#[test]
fn structured_payload_validation_does_not_trust_client_identity_or_types() {
let descriptor = descriptor();
let original = parse_feature_invocation("/run(path=x)", 0, &descriptor, "stable")
.unwrap()
.invocation;
let mut changed = original.clone();
changed.identity = FeatureInvocationIdentity("builtin:other/run".into());
assert_eq!(
validate_feature_invocation(&descriptor, &changed),
Err(InvocationValidationError::IdentityMismatch)
);
changed = original.clone();
changed.name = "unregistered".into();
assert_eq!(
validate_feature_invocation(&descriptor, &changed),
Err(InvocationValidationError::NameMismatch)
);
changed = original.clone();
changed.arguments[0].value = InvocationValue::Boolean(true);
assert!(matches!(
validate_feature_invocation(&descriptor, &changed),
Err(InvocationValidationError::InvalidValue { .. })
));
changed = original.clone();
changed.arguments.push(changed.arguments[0].clone());
assert!(matches!(
validate_feature_invocation(&descriptor, &changed),
Err(InvocationValidationError::DuplicateArgument(_))
));
changed = original;
changed.invocation_id.clear();
assert!(validate_feature_invocation(&descriptor, &changed).is_err());
}
#[test]
fn integer_boolean_and_utf8_offsets_use_the_common_typed_contract() {
let mut descriptor = descriptor();
descriptor.arguments[0].value_type = InvocationArgumentType::Integer;
descriptor.arguments[1].value_type = InvocationArgumentType::Boolean;
let parsed = parse_feature_invocation(
"本文 /execute(-2147483648, mode=true) 後",
"本文 ".len(),
&descriptor,
"typed",
)
.unwrap();
assert_eq!(
parsed.invocation.arguments[0].value,
InvocationValue::Integer(i32::MIN)
);
assert_eq!(
parsed.invocation.arguments[1].value,
InvocationValue::Boolean(true)
);
for invalid in ["/run(2147483648)", "/run(1.0)", "/run(1, mode=yes)"] {
assert!(parse_feature_invocation(invalid, 0, &descriptor, "typed").is_err());
}
assert!(parse_feature_invocation("本文 /run(1)", 1, &descriptor, "typed").is_err());
assert!(parse_feature_invocation("/run(1)", usize::MAX, &descriptor, "typed").is_err());
}
#[test]
fn json_escapes_and_multiple_invocations_have_explicit_boundaries() {
let input = r#"/run("\u8cc7\u6599\/a\b\f", mode=safe) prose /execute("second") end"#;
let first = parse_feature_invocation(input, 0, &descriptor(), "first").unwrap();
assert_eq!(
first.invocation.arguments[0].value,
InvocationValue::String("資料/a\u{8}\u{c}".into())
);
let next = input[first.end..].find("/execute").unwrap() + first.end;
let second = parse_feature_invocation(input, next, &descriptor(), "second").unwrap();
assert_eq!(&input[second.end..], " end");
assert!(matches!(
parse_feature_invocation(r#"/run("\uD800")"#, 0, &descriptor(), "bad"),
Err(InvocationValidationError::Syntax { .. })
));
let value = "controls \u{0}\u{8}\u{c}";
let spelling = format!("/run({})", quote_invocation_string(value));
assert_eq!(
parse_feature_invocation(&spelling, 0, &descriptor(), "control")
.unwrap()
.invocation
.arguments[0]
.value,
InvocationValue::String(value.into())
);
}
#[test]
fn typed_segment_round_trips_without_flattening_identity_or_arguments() {
let parsed =
parse_feature_invocation("/run(\"x y\", mode=fast)", 0, &descriptor(), "1").unwrap();
let segment = crate::Segment::FeatureInvoke {
invocation: parsed.invocation,
};
let json = serde_json::to_string(&segment).unwrap();
assert!(json.contains("\"kind\":\"feature_invoke\""));
assert!(json.contains("builtin:test/run"));
assert_eq!(
serde_json::from_str::<crate::Segment>(&json).unwrap(),
segment
);
}
#[test]
fn quoting_round_trips_spaces_unicode_slashes_and_escapes() {
let value = "a b/界\\\"\n";
let input = format!("/run({})", quote_invocation_string(value));
let parsed = parse_feature_invocation(&input, 0, &descriptor(), "1").unwrap();
assert_eq!(
parsed.invocation.arguments[0].value,
InvocationValue::String(value.into())
);
}
}
+174 -13
View File
@@ -1,4 +1,5 @@
pub mod identity;
pub mod invocation;
#[cfg(feature = "stream")]
pub mod stream;
pub mod subscription;
@@ -10,6 +11,14 @@ use std::path::PathBuf;
use serde::{Deserialize, Serialize};
pub use identity::{WorkerId, WorkerIdParseError};
pub use invocation::{
FeatureInvocation, FeatureInvocationDescriptor, FeatureInvocationIdentity,
FeatureInvocationResult, FeatureInvocationStatus, FeatureInvocationSyntax,
InvocationArgumentDescriptor, InvocationArgumentType, InvocationArgumentValue,
InvocationClientAdapter, InvocationCompletion, InvocationValidationError, InvocationValue,
ParsedFeatureInvocation, parse_feature_invocation, quote_invocation_string,
validate_feature_invocation,
};
/// Allocate an opaque idempotency key for one client Submit request.
pub fn new_submission_request_id() -> String {
@@ -41,6 +50,9 @@ pub enum AuthenticatedInputSource {
/// Assigned whenever a serialized tracked method crosses an untrusted
/// protocol boundary. Receivers must handle it exactly like public input.
UntrustedWire,
/// Stamped only by the operator-owned standalone in-process transport.
/// This is not a Backend Account or authority accepted from wire fields.
LocalOperator,
Account {
account_id: String,
},
@@ -62,10 +74,22 @@ impl Default for AuthenticatedInputSource {
}
}
// Tracked methods may carry serialized transport provenance, but wire values
// never grant authority. Keep this field recognized under fail-closed decoding
// while replacing it with UntrustedWire, including malformed/forged values.
fn ignore_wire_input_source<'de, D>(deserializer: D) -> Result<AuthenticatedInputSource, D::Error>
where
D: serde::Deserializer<'de>,
{
let _ = serde::de::IgnoredAny::deserialize(deserializer)?;
Ok(AuthenticatedInputSource::UntrustedWire)
}
impl AuthenticatedInputSource {
pub fn namespace(&self) -> String {
match self {
Self::UntrustedWire => "untrusted-wire".into(),
Self::LocalOperator => "local-operator".into(),
Self::Account { account_id } => format!("account:{account_id}"),
Self::Worker {
runtime_id,
@@ -175,6 +199,11 @@ pub struct WorkerStateSnapshot {
schemars(range(min = 0, max = 9_007_199_254_740_991_u64))
)]
pub last_command_id: u64,
/// Most recent Submit request whose controller Run has returned and whose
/// post-Run state transition has been published. This is a teardown fence,
/// not a successful-result claim; cancelled and failed Runs may also finish.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub last_finished_submission_request_id: Option<String>,
pub state: WorkerState,
}
@@ -182,6 +211,7 @@ impl WorkerStateSnapshot {
pub fn initial() -> Self {
Self {
last_command_id: 0,
last_finished_submission_request_id: None,
state: WorkerState::Idle,
}
}
@@ -209,6 +239,7 @@ impl From<WorkerStatus> for WorkerStateSnapshot {
};
Self {
last_command_id: 0,
last_finished_submission_request_id: None,
state,
}
}
@@ -217,7 +248,12 @@ impl From<WorkerStatus> for WorkerStateSnapshot {
#[derive(Debug, Clone, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "method", content = "params", rename_all = "snake_case")]
#[serde(
tag = "method",
content = "params",
rename_all = "snake_case",
deny_unknown_fields
)]
pub enum Method {
/// Durably accept typed input for immediate activation or the session FIFO.
/// Running or Paused Workers queue Submit for a later turn; use Notify for
@@ -237,7 +273,7 @@ pub enum Method {
SubmitTracked {
submission_request_id: String,
input: Vec<Segment>,
#[serde(skip_deserializing, default)]
#[serde(default, deserialize_with = "ignore_wire_input_source")]
source: AuthenticatedInputSource,
},
/// Agent-controlled input: accept only at an Idle controller boundary, never
@@ -247,7 +283,7 @@ pub enum Method {
SubmitIfIdle {
submission_request_id: String,
input: Vec<Segment>,
#[serde(skip_deserializing, default)]
#[serde(default, deserialize_with = "ignore_wire_input_source")]
source: AuthenticatedInputSource,
},
/// Human-readable text injected into the target Worker's LLM context as a
@@ -265,7 +301,7 @@ pub enum Method {
NotifyTracked {
notification_request_id: String,
message: String,
#[serde(skip_deserializing, default)]
#[serde(default, deserialize_with = "ignore_wire_input_source")]
source: AuthenticatedInputSource,
},
/// Typed lifecycle report from a child Worker to its direct parent.
@@ -329,6 +365,12 @@ pub enum Method {
ListCompletions {
kind: CompletionKind,
prefix: String,
/// Client query generation. Echoed verbatim so identical-prefix ABA
/// responses cannot cross edits, target switches, or permission changes.
#[serde(default, skip_serializing_if = "Option::is_none")]
request_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
context: Option<CompletionContext>,
},
/// List Workers visible to this Worker from durable Worker state and the spawned-child
/// registry. This is not a host-wide Worker universe query.
@@ -602,6 +644,10 @@ pub enum Segment {
/// `[Dir: <path>]` listings; the flattened user text keeps the literal
/// `@<path>` placeholder either way.
FileRef { path: String },
/// Explicitly selected, source-qualified Feature invocation. The host never
/// promotes free text to this variant and validates it against the installed
/// declaration before execution.
FeatureInvoke { invocation: FeatureInvocation },
/// Source-qualified request for the host to start a Flow before committing
/// this user input. Runtime resolves it through Workspace authority and
/// replaces it with the entered state's instructions before Worker-side
@@ -670,6 +716,9 @@ impl Segment {
out.push('@');
out.push_str(path);
}
Segment::FeatureInvoke { invocation } => {
out.push_str(&invocation.display_input());
}
Segment::Flow { selector } => {
out.push_str("[Flow: ");
out.push_str(selector);
@@ -1037,10 +1086,11 @@ pub enum SessionContentPart {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct SessionToolAttachment {
/// Stable, bounded identity derived from the owning Session entry, attachment
/// position, and immutable image bytes. It is a locator, not a bearer secret.
pub attachment_id: String,
pub media_type: String,
/// Base64-encoded durable attachment body. Public snapshots preserve the
/// committed multimodal value instead of replacing it with placeholder text.
pub data_base64: String,
pub byte_len: u64,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -1340,6 +1390,12 @@ pub enum Event {
/// wired up.
Completions {
kind: CompletionKind,
#[serde(default)]
prefix: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
request_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
context: Option<CompletionContext>,
entries: Vec<CompletionEntry>,
},
/// Reply to `Method::ListRewindTargets`. Clients should only open a picker
@@ -1452,25 +1508,42 @@ pub enum AlertSource {
/// Kind of completion requested by `Method::ListCompletions`.
///
/// Mirrors the completion prefix sigil: `@` → `File`.
/// `File` is the existing `@` path lane. `Feature` lists installed chat
/// invocations. `FeatureArgument` resolves a declared argument using the
/// source-qualified [`CompletionContext`].
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum CompletionKind {
File,
Feature,
FeatureArgument,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct CompletionContext {
pub invocation: FeatureInvocationIdentity,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub argument: Option<String>,
}
/// One completion candidate for a prefix query.
///
/// `value` is a path (file kind).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct CompletionEntry {
pub value: String,
#[serde(default)]
pub is_dir: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub usage: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub invocation: Option<FeatureInvocationDescriptor>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -1925,6 +1998,7 @@ mod tests {
fn worker_state_snapshot_wire_shape_has_one_authoritative_state() {
let snapshot = WorkerStateSnapshot {
last_command_id: 7,
last_finished_submission_request_id: Some("request-7".to_string()),
state: WorkerState::Busy(WorkerBusyState::Run(WorkerRunState::Running)),
};
let value = serde_json::to_value(&snapshot).unwrap();
@@ -1932,6 +2006,7 @@ mod tests {
value,
serde_json::json!({
"last_command_id": 7,
"last_finished_submission_request_id": "request-7",
"state": {
"kind": "busy",
"state": { "kind": "run", "state": "running" }
@@ -2104,6 +2179,49 @@ mod tests {
);
}
#[test]
fn local_operator_source_cannot_be_granted_by_serialized_input() {
let method = Method::SubmitTracked {
input: vec![Segment::text("forged local operator")],
submission_request_id: "local-forged".into(),
source: AuthenticatedInputSource::LocalOperator,
};
let decoded: Method =
serde_json::from_str(&serde_json::to_string(&method).unwrap()).unwrap();
assert!(matches!(
decoded,
Method::SubmitTracked {
source: AuthenticatedInputSource::UntrustedWire,
..
}
));
assert_ne!(
AuthenticatedInputSource::LocalOperator.namespace(),
AuthenticatedInputSource::UntrustedWire.namespace()
);
}
#[test]
fn notify_rejects_typed_intent_instead_of_silently_discarding_it() {
for method in ["notify", "notify_tracked"] {
for key in ["input", "segments", "invocation"] {
let mut params =
serde_json::json!({"notification_request_id":"request", "message":"advisory"});
params[key] = serde_json::json!([{ "kind":"feature_invoke", "invocation": { "identity":"builtin:test/run" } }]);
assert!(
serde_json::from_value::<Method>(
serde_json::json!({"method":method,"params":params})
)
.is_err()
);
}
}
// Text is still text: no decoding or Host scan promotes it to an invoke.
assert!(
matches!(serde_json::from_value::<Method>(serde_json::json!({"method":"notify", "params":{"notification_request_id":"text","message":"/run()"}})).unwrap(), Method::Notify { message, .. } if message == "/run()")
);
}
#[test]
fn segment_unknown_variant_decodes_as_unknown() {
// A future client sends a segment kind this Worker has never heard of.
@@ -2342,6 +2460,8 @@ mod tests {
let method = Method::ListCompletions {
kind: CompletionKind::File,
prefix: "src/".into(),
request_id: None,
context: None,
};
let json = serde_json::to_string(&method).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
@@ -2351,7 +2471,7 @@ mod tests {
let decoded: Method = serde_json::from_str(&json).unwrap();
match decoded {
Method::ListCompletions { kind, prefix } => {
Method::ListCompletions { kind, prefix, .. } => {
assert_eq!(kind, CompletionKind::File);
assert_eq!(prefix, "src/");
}
@@ -2359,13 +2479,52 @@ mod tests {
}
}
#[test]
fn completion_generation_round_trips_for_identical_query_correlation() {
let context = CompletionContext {
invocation: FeatureInvocationIdentity("builtin:test/run".into()),
argument: Some("mode".into()),
};
for request_id in ["query-1", "query-2"] {
let method = Method::ListCompletions {
kind: CompletionKind::FeatureArgument,
prefix: "s".into(),
request_id: Some(request_id.into()),
context: Some(context.clone()),
};
let decoded: Method =
serde_json::from_str(&serde_json::to_string(&method).unwrap()).unwrap();
assert!(
matches!(decoded, Method::ListCompletions { request_id: Some(id), context: Some(restored), .. } if id == request_id && restored == context)
);
let event = Event::Completions {
kind: CompletionKind::FeatureArgument,
prefix: "s".into(),
request_id: Some(request_id.into()),
context: Some(context.clone()),
entries: Vec::new(),
};
let decoded: Event =
serde_json::from_str(&serde_json::to_string(&event).unwrap()).unwrap();
assert!(
matches!(decoded, Event::Completions { request_id: Some(id), context: Some(restored), .. } if id == request_id && restored == context)
);
}
}
#[test]
fn event_completions_format_and_default_is_dir() {
let event = Event::Completions {
kind: CompletionKind::File,
prefix: "src/".into(),
request_id: None,
context: None,
entries: vec![CompletionEntry {
value: "src/main.rs".into(),
is_dir: false,
description: None,
usage: None,
invocation: None,
}],
};
let json = serde_json::to_string(&event).unwrap();
@@ -2379,7 +2538,7 @@ mod tests {
r#"{"event":"completions","data":{"kind":"file","entries":[{"value":"main.rs"}]}}"#;
let decoded: Event = serde_json::from_str(inbound).unwrap();
match decoded {
Event::Completions { kind, entries } => {
Event::Completions { kind, entries, .. } => {
assert_eq!(kind, CompletionKind::File);
assert_eq!(entries.len(), 1);
assert_eq!(entries[0].value, "main.rs");
@@ -2648,6 +2807,7 @@ mod tests {
let event = Event::WorkerState {
snapshot: WorkerStateSnapshot {
last_command_id: 9,
last_finished_submission_request_id: None,
state: WorkerState::Busy(WorkerBusyState::Run(WorkerRunState::Running)),
},
};
@@ -2669,6 +2829,7 @@ mod tests {
snapshot: WorkerStateSnapshot {
last_command_id: 9,
state: WorkerState::Busy(WorkerBusyState::Run(WorkerRunState::Running)),
..
}
}
));
+10
View File
@@ -651,6 +651,10 @@ impl SubscriptionWorkerJob {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct SubscriptionWorker {
/// Opaque Runtime-owned Restore observation identity, never a display-state hash.
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub restore_observation_token: Option<String>,
pub worker_id: SubscriptionWorkerId,
/// Set by the Workspace Server when projecting a Runtime-owned Worker to clients.
/// Runtime producers leave this unset because the connection identifies the Runtime.
@@ -689,6 +693,9 @@ pub struct SubscriptionWorker {
impl SubscriptionWorker {
pub fn validate(&self) -> Result<(), SubscriptionValidationError> {
self.worker_id.validate()?;
if let Some(token) = &self.restore_observation_token {
validate_identifier("restore_observation_token", token, MAX_RESOURCE_ID_BYTES)?;
}
if let Some(runtime_id) = &self.runtime_id {
validate_identifier("runtime_id", runtime_id, MAX_RESOURCE_ID_BYTES)?;
}
@@ -962,6 +969,7 @@ mod tests {
fn worker(value: &str) -> SubscriptionWorker {
SubscriptionWorker {
restore_observation_token: None,
worker_id: worker_id(value),
runtime_id: None,
resource_key: None,
@@ -1197,6 +1205,8 @@ mod tests {
method: crate::Method::ListCompletions {
kind: crate::CompletionKind::File,
prefix: "src/".to_string(),
request_id: None,
context: None,
},
},
));
+30 -13
View File
@@ -6,20 +6,23 @@ use ts_rs::{Config, TS};
use crate::{
Alert, AlertLevel, AlertSource, CommandEvent, CommandSnapshot, CommandStatus, CommandStream,
CommandStreamSlice, CompactionLifecycle, CompactionLifecycleState, CompactionPhase,
CompactionTrigger, CompletionEntry, CompletionKind, ContextTokenSource, ContextUsage,
ErrorCode, Event, Greeting, InFlightBlock, InFlightCompaction, InFlightSnapshot,
CompactionTrigger, CompletionContext, CompletionEntry, CompletionKind, ContextTokenSource,
ContextUsage, ErrorCode, Event, FeatureInvocation, FeatureInvocationDescriptor,
FeatureInvocationIdentity, FeatureInvocationResult, FeatureInvocationStatus,
FeatureInvocationSyntax, Greeting, InFlightBlock, InFlightCompaction, InFlightSnapshot,
InFlightToolCallState, InternalWorkerKind, InternalWorkerRef, InternalWorkerSnapshot,
InvokeKind, MemoryWorkerEvent, Method, PasteArtifactAvailability, PasteArtifactMediaType,
PasteArtifactRef, PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission,
ReasoningConfig, RewindSummary, RewindTarget, RewindTargetId, RunFailureKind, RunResult,
RunResumeSource, RunYieldReason, ScopeRule, Segment, SessionContentPart,
SessionConversationTurn, SessionEntryProvenance, SessionHistoryLineageBoundary,
SessionHistoryPage, SessionMessageRole, SessionSnapshot, SessionSnapshotEntry,
SessionSnapshotEntryData, SessionToolAttachment, SubmissionDisposition, SymlinkPolicy,
ToolResultDisposition, TurnResult, UploadedFileAvailability, UploadedFileRef, WorkerBusyState,
WorkerCommandAcknowledgement, WorkerCommandDisposition, WorkerCommandEnvelope,
WorkerCommandKind, WorkerEvent, WorkerMaintenanceState, WorkerRunState, WorkerState,
WorkerStateSnapshot, WorkerStatus,
InvocationArgumentDescriptor, InvocationArgumentType, InvocationArgumentValue,
InvocationClientAdapter, InvocationCompletion, InvocationValue, InvokeKind, MemoryWorkerEvent,
Method, PasteArtifactAvailability, PasteArtifactMediaType, PasteArtifactRef,
PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission, ReasoningConfig,
RewindSummary, RewindTarget, RewindTargetId, RunFailureKind, RunResult, RunResumeSource,
RunYieldReason, ScopeRule, Segment, SessionContentPart, SessionConversationTurn,
SessionEntryProvenance, SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole,
SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment,
SubmissionDisposition, SymlinkPolicy, ToolResultDisposition, TurnResult,
UploadedFileAvailability, UploadedFileRef, WorkerBusyState, WorkerCommandAcknowledgement,
WorkerCommandDisposition, WorkerCommandEnvelope, WorkerCommandKind, WorkerEvent,
WorkerMaintenanceState, WorkerRunState, WorkerState, WorkerStateSnapshot, WorkerStatus,
subscription::{
EventSubscriptionSelector, MAX_CORRELATION_ID_BYTES, MAX_REJECTION_MESSAGE_BYTES,
MAX_RESOURCE_ID_BYTES, MAX_SUBSCRIPTION_COLLECTION_ITEMS,
@@ -64,6 +67,19 @@ pub fn generated_protocol_types() -> String {
push_decl::<AlertLevel>(&cfg, &mut output);
push_decl::<AlertSource>(&cfg, &mut output);
push_decl::<CompletionKind>(&cfg, &mut output);
push_decl::<CompletionContext>(&cfg, &mut output);
push_decl::<FeatureInvocationIdentity>(&cfg, &mut output);
push_decl::<FeatureInvocationSyntax>(&cfg, &mut output);
push_decl::<InvocationArgumentType>(&cfg, &mut output);
push_decl::<InvocationCompletion>(&cfg, &mut output);
push_decl::<InvocationArgumentDescriptor>(&cfg, &mut output);
push_decl::<InvocationClientAdapter>(&cfg, &mut output);
push_decl::<FeatureInvocationDescriptor>(&cfg, &mut output);
push_decl::<InvocationValue>(&cfg, &mut output);
push_decl::<InvocationArgumentValue>(&cfg, &mut output);
push_decl::<FeatureInvocation>(&cfg, &mut output);
push_decl::<FeatureInvocationStatus>(&cfg, &mut output);
push_decl::<FeatureInvocationResult>(&cfg, &mut output);
push_decl::<WorkerStatus>(&cfg, &mut output);
push_decl::<WorkerCommandEnvelope>(&cfg, &mut output);
push_decl::<WorkerCommandKind>(&cfg, &mut output);
@@ -230,6 +246,7 @@ fn rust_serialized_subscription_frame_fixtures() -> Vec<SubscriptionFrame> {
let external_workdir_id = SubscriptionWorkdirId::new("fixture-external-workdir")
.expect("fixture external Workdir id must be valid");
let worker = SubscriptionWorker {
restore_observation_token: Some("fixture-restore-observation".into()),
worker_id: worker_id.clone(),
runtime_id: Some("fixture-runtime".to_string()),
resource_key: Some("fixture-worker-resource".to_string()),
@@ -378,6 +378,7 @@ function validateWorkers(workers: SubscriptionWorker[]): boolean {
}
function validateWorker(worker: SubscriptionWorker): boolean {
if (worker.restore_observation_token != null && !validateIdentifier(worker.restore_observation_token, MAX_RESOURCE_ID_BYTES)) return false;
if (!validateIdentifier(worker.worker_id, MAX_RESOURCE_ID_BYTES)) return false;
if (worker.runtime_id != null && !validateIdentifier(worker.runtime_id, MAX_RESOURCE_ID_BYTES)) {
return false;
+161 -5
View File
@@ -420,6 +420,18 @@ pub struct WorkerInput {
pub segments: Option<Vec<Segment>>,
}
/// Host-authored Job/attempt identity, independent of Profile selection.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct BackendJobExecutionBinding {
pub job_id: String,
pub attempt_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub input_revision: Option<String>,
#[serde(default)]
pub subjektiv_consolidation: bool,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
pub struct CreateWorkerRequest {
pub worker_id: WorkerId,
@@ -448,10 +460,14 @@ pub struct CreateWorkerRequest {
/// never activates subjektiv for an ordinary Workspace Worker.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub subjektiv_attached: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub backend_job: Option<BackendJobExecutionBinding>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
pub struct WorkerSummary {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub restore_observation_token: Option<String>,
pub worker_ref: WorkerRef,
pub worker_id: WorkerId,
pub status: WorkerStatus,
@@ -484,6 +500,62 @@ pub struct WorkerResponse {
pub worker: WorkerDetail,
}
/// Explicit observed Restore intent. Preparation is authored by the Workspace
/// Backend, not browser/Tool JSON, and applied only after Runtime admission.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct WorkerRestoreRequest {
pub expected_observation_token: String,
pub request_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub preparation: Option<WorkerRestorePreparation>,
}
#[derive(Clone, Debug, Default, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct WorkerRestorePreparation {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workspace_api: Option<WorkspaceApiRef>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workdir_attachments: Option<Vec<LogicalWorkdirAttachment>>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access: Vec<WorkingDirectoryRepositoryAccessRequest>,
/// Read-only references resolved by Workspace only after this operation owns admission.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access_workdirs: Vec<RepositoryAccessWorkdirReference>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct RepositoryAccessWorkdirReference {
pub runtime_id: String,
pub working_directory_id: String,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct WorkerRestoreCoordinationRequest {
pub expected_observation_token: String,
pub request_id: String,
/// None means owner/result recovery only; it can never admit a new intent.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub preparation: Option<WorkerRestorePreparation>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
pub struct WorkerRestoreCoordinationResponse {
pub result: Option<WorkerRestoreResponse>,
/// Original read-only preparation, only when the existing operation still needs it.
pub preparation: Option<WorkerRestorePreparation>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct WorkingDirectoryRepositoryAccessRequest {
pub working_directory_id: String,
pub materialization: RepositoryMaterializationContext,
}
/// Explicit restore outcome returned for every reachable Runtime restore attempt,
/// including preflight rejection and uncertain post-side-effect reconciliation.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
@@ -567,12 +639,20 @@ pub struct CompletionRequest {
pub kind: CompletionKind,
#[serde(default)]
pub prefix: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context: Option<protocol::CompletionContext>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub request_id: Option<String>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
pub struct CompletionResponse {
pub kind: CompletionKind,
pub prefix: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context: Option<protocol::CompletionContext>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub request_id: Option<String>,
pub entries: Vec<CompletionEntry>,
}
@@ -992,11 +1072,18 @@ pub trait RuntimeApi {
#[body] request: WorkerLifecycleRequest,
) -> Result<WorkerLifecycleResponse, RuntimeApiError>;
#[post("/v1/workers/{worker_id}/restore", status = 200, error_status = 400)]
#[post("/v1/workers/{worker_id}/restore/coordinate", status = 200, error_status = 400, additional_error_statuses = [409])]
async fn coordinate_worker_restore(
&self,
#[path] worker_id: String,
#[body] request: WorkerRestoreCoordinationRequest,
) -> Result<WorkerRestoreCoordinationResponse, RuntimeApiError>;
#[post("/v1/workers/{worker_id}/restore", status = 200, error_status = 400, additional_error_statuses = [409])]
async fn restore_worker(
&self,
#[path] worker_id: String,
#[body] request: EmptyObjectRequest,
#[body] request: WorkerRestoreRequest,
) -> Result<WorkerRestoreResponse, RuntimeApiError>;
#[post(
@@ -1085,6 +1172,8 @@ pub const RUNTIME_ROUTE_WORKER_PROTOCOL_WS: &str = "/v1/workers/{worker_id}/prot
pub const RUNTIME_ROUTE_WORKER_ATTACHMENTS: &str = "/v1/workers/{worker_id}/attachments";
pub const RUNTIME_ROUTE_WORKER_ATTACHMENT: &str =
"/v1/workers/{worker_id}/attachments/{artifact_id}";
pub const RUNTIME_ROUTE_WORKER_SESSION_ATTACHMENT: &str =
"/v1/workers/{worker_id}/sessions/{session_id}/attachments/{attachment_id}";
/// Intentionally out-of-contract Runtime routes. This inventory keeps manual paths visible rather
/// than letting them be mistaken for management-contract omissions.
@@ -1189,12 +1278,72 @@ pub const REMAINING_RUNTIME_ROUTES: &[RemainingRuntimeRoute] = &[
path: RUNTIME_ROUTE_WORKER_ATTACHMENT,
reason: "Binary attachment transport",
},
RemainingRuntimeRoute {
method: "GET",
path: RUNTIME_ROUTE_WORKER_SESSION_ATTACHMENT,
reason: "Session image attachment transport",
},
];
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn restore_request_is_mandatory_and_conflict_is_not_an_operation_state() {
for invalid in [
serde_json::json!({}),
serde_json::json!({"request_id":"r"}),
serde_json::json!({"expected_observation_token":"t"}),
] {
assert!(serde_json::from_value::<WorkerRestoreRequest>(invalid).is_err());
}
let request: WorkerRestoreRequest = serde_json::from_value(
serde_json::json!({"expected_observation_token":"t", "request_id":"r"}),
)
.unwrap();
assert!(request.preparation.is_none());
for state in [
"accepted",
"rejected",
"rolled_back",
"reconciliation_required",
] {
assert!(serde_json::from_value::<WorkerRestoreState>(serde_json::json!(state)).is_ok());
}
assert!(
serde_json::from_value::<WorkerRestoreState>(serde_json::json!(
"restore_observation_conflict"
))
.is_err()
);
let conflict = RuntimeApiError::new(
409,
"restore_observation_conflict",
"refresh before a new intent",
);
assert_eq!(conflict.status(), 409);
assert_eq!(
serde_json::to_value(conflict).unwrap()["error"]["code"],
"restore_observation_conflict"
);
}
#[test]
fn feature_argument_completion_wire_preserves_source_and_argument() {
let json = serde_json::json!({"kind":"feature_argument", "prefix":"資料/", "context": {"invocation":"builtin:test/prepare", "argument":"path"}});
let request: CompletionRequest = serde_json::from_value(json.clone()).unwrap();
assert_eq!(request.kind, CompletionKind::FeatureArgument);
assert_eq!(
request.context.as_ref().unwrap().argument.as_deref(),
Some("path")
);
assert_eq!(serde_json::to_value(request).unwrap(), json);
let legacy: CompletionRequest =
serde_json::from_value(serde_json::json!({"kind":"file"})).unwrap();
assert!(legacy.context.is_none());
}
#[derive(Clone)]
struct RoundTripService;
@@ -1358,10 +1507,17 @@ mod tests {
) -> Result<WorkerLifecycleResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn coordinate_worker_restore(
&self,
_worker_id: String,
_request: WorkerRestoreCoordinationRequest,
) -> Result<WorkerRestoreCoordinationResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn restore_worker(
&self,
_worker_id: String,
_request: EmptyObjectRequest,
_request: WorkerRestoreRequest,
) -> Result<WorkerRestoreResponse, RuntimeApiError> {
Err(test_error(501))
}
@@ -1461,7 +1617,7 @@ mod tests {
#[test]
fn contract_inventory_is_complete_and_unique() {
let operations = RuntimeApiMetadata::OPERATIONS;
assert_eq!(operations.len(), 19);
assert_eq!(operations.len(), 20);
let mut routes = operations
.iter()
.map(|operation| (format!("{:?}", operation.method), operation.path))
@@ -1478,7 +1634,7 @@ mod tests {
#[test]
fn remaining_route_inventory_does_not_overlap_contract() {
assert_eq!(REMAINING_RUNTIME_ROUTES.len(), 20);
assert_eq!(REMAINING_RUNTIME_ROUTES.len(), 21);
for remaining in REMAINING_RUNTIME_ROUTES {
assert!(!RuntimeApiMetadata::OPERATIONS.iter().any(|operation| {
format!("{:?}", operation.method).eq_ignore_ascii_case(remaining.method)
+518 -28
View File
@@ -11,6 +11,7 @@ pub use api_macros::axum as server_support;
pub use api_macros::reqwest as client_support;
pub use api_macros::{ApiContract, BinaryBody, HttpMethod, TransportMetadata, WebSocketOperation};
pub mod repository_openapi_typescript;
mod workspace_config;
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
@@ -18,6 +19,7 @@ use webauthn_rs_proto::{
CreationChallengeResponse, PublicKeyCredential, RegisterPublicKeyCredential,
RequestChallengeResponse,
};
pub use workspace_config::*;
#[allow(dead_code)]
#[derive(JsonSchema)]
@@ -190,10 +192,32 @@ impl_openapi_schema!(
WorkingDirectoryListResponse,
WorkingDirectoryDetailResponse,
WorkingDirectoryCreateResponse,
CurrentWorkerWorkdirCatalogQuery,
CurrentWorkerWorkdirCatalogResponse,
CurrentWorkerWorkdirAttachRequest,
CurrentWorkerWorkdirAttachmentResponse,
CurrentWorkerWorkdirAttachmentItem,
CurrentWorkerWorkdirAttachmentListQuery,
CurrentWorkerWorkdirAttachmentListResponse,
CurrentWorkerWorkdirDetachQuery,
CurrentWorkerWorkdirOperationRequest,
CurrentWorkerWorkdirOperationResponse,
WorkspaceConfigAccess,
WorkspaceConfigNodeKind,
WorkspaceConfigFailureClassification,
WorkspaceConfigAttachRequest,
WorkspaceConfigAttachment,
WorkspaceConfigCurrentResponse,
WorkspaceConfigObserveRequest,
WorkspaceConfigNode,
WorkspaceConfigObserveResponse,
WorkspaceConfigReadRequest,
WorkspaceConfigReadResponse,
WorkspaceConfigCommitRequest,
WorkspaceConfigCommitResponse,
WorkspaceConfigApiError,
WorkspaceConfigGrantCreateRequest,
WorkspaceConfigGrantResponse,
ExternalWorkdirGrantCreateRequest,
ExternalWorkdirGrantResponse,
WorkspaceResponse,
@@ -237,6 +261,7 @@ impl_openapi_schema!(
MemoryConsolidateStagingRequest,
MemoryConsolidationResponse,
SubjektivSubjectCreateRequest,
SubjektivSubjectBehaviorUpdateRequest,
SubjektivSubjectListQuery,
SubjektivSubjectListResponse,
SubjektivSubjectResponse,
@@ -249,7 +274,6 @@ impl_openapi_schema!(
SubjektivMemoryQueryResponse,
SubjektivMemoryReadResponse,
SubjektivMemoryListRevisionsResponse,
SubjektivSubjectWorkerStartRequest,
SubjektivStageCandidateRequest,
SubjektivStageCandidateResponse,
SubjektivRecordSessionRequest,
@@ -365,6 +389,7 @@ impl_openapi_schema!(
WorkspaceRuntimeListResponse,
RuntimeWorkerListResponse,
WorkerRestoreResponse,
WorkerRestoreRequest,
WorkerControlSubject,
WorkerControlRecord,
WorkerControlListResponse,
@@ -1164,6 +1189,20 @@ pub trait ServerApi {
#[path] workspace_id: String,
#[path] subject_id: String,
) -> Result<SubjektivSubjectResponse, RepositoryApiError>;
#[patch(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/behavior",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_behavior_update(
&self,
#[path] workspace_id: String,
#[path] subject_id: String,
#[body] request: SubjektivSubjectBehaviorUpdateRequest,
) -> Result<SubjektivSubjectResponse, RepositoryApiError>;
#[get(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/surface",
status = 200,
@@ -1234,21 +1273,6 @@ pub trait ServerApi {
#[path] workspace_id: String,
#[path] subject_id: String,
) -> Result<SubjektivSubjectResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/subjects/{subject_id}/worker",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 409, 500, 502, 503],
bearer_auth = true,
browser_auth = true,
)]
async fn subjektiv_subject_worker_start(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] subject_id: String,
#[body] request: SubjektivSubjectWorkerStartRequest,
) -> Result<BrowserCreateWorkerResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/subjektiv/staging",
status = 200,
@@ -1529,6 +1553,7 @@ pub trait ServerApi {
#[path] workspace_id: String,
#[path] runtime_id: String,
#[path] worker_id: String,
#[body] request: WorkerRestoreRequest,
) -> Result<WorkerRestoreResponse, RepositoryApiError>;
#[get(
"/api/w/{workspace_id}/worker-observation/sessions",
@@ -2054,8 +2079,10 @@ pub trait ServerApi {
)]
async fn runtime_worker_restore_alias(
&self,
#[extension] context: ServerRequestContext,
#[path] runtime_id: String,
#[path] worker_id: String,
#[body] request: WorkerRestoreRequest,
) -> Result<WorkerRestoreResponse, RepositoryApiError>;
#[post(
@@ -2068,10 +2095,12 @@ pub trait ServerApi {
)]
async fn runtime_worker_restore(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] runtime_id: String,
#[path] worker_id: String,
#[query] query: RestoreTicketAssignmentQuery,
#[body] request: WorkerRestoreRequest,
) -> Result<WorkerRestoreResponse, RepositoryApiError>;
#[put(
@@ -2084,6 +2113,7 @@ pub trait ServerApi {
)]
async fn runtime_worker_pin(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] runtime_id: String,
#[path] worker_id: String,
@@ -2099,6 +2129,7 @@ pub trait ServerApi {
)]
async fn runtime_worker_unpin(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] runtime_id: String,
#[path] worker_id: String,
@@ -2128,6 +2159,7 @@ pub trait ServerApi {
)]
async fn runtime_cleanup_execute(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] runtime_id: String,
#[body] request: ExecuteRuntimeCleanupRequest,
@@ -2143,6 +2175,7 @@ pub trait ServerApi {
)]
async fn runtime_worker_input_alias(
&self,
#[extension] context: ServerRequestContext,
#[path] runtime_id: String,
#[path] worker_id: String,
#[body] request: RuntimeWorkerInputRequest,
@@ -2158,6 +2191,7 @@ pub trait ServerApi {
)]
async fn runtime_worker_input(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] runtime_id: String,
#[path] worker_id: String,
@@ -2274,6 +2308,7 @@ pub trait ServerApi {
)]
async fn runtime_worker_stop_alias(
&self,
#[extension] context: ServerRequestContext,
#[path] runtime_id: String,
#[path] worker_id: String,
#[body] request: RuntimeWorkerLifecycleRequest,
@@ -2289,6 +2324,7 @@ pub trait ServerApi {
)]
async fn runtime_worker_stop(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] runtime_id: String,
#[path] worker_id: String,
@@ -2305,6 +2341,7 @@ pub trait ServerApi {
)]
async fn runtime_worker_cancel_alias(
&self,
#[extension] context: ServerRequestContext,
#[path] runtime_id: String,
#[path] worker_id: String,
#[body] request: RuntimeWorkerLifecycleRequest,
@@ -2320,6 +2357,7 @@ pub trait ServerApi {
)]
async fn runtime_worker_cancel(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] runtime_id: String,
#[path] worker_id: String,
@@ -2565,6 +2603,81 @@ pub trait ServerApi {
#[path] working_directory_id: String,
#[body] request: WorkingDirectoryRemovalRequest,
) -> Result<WorkingDirectoryRemovalResponse, RepositoryApiError>;
#[get("/api/w/{workspace_id}/workers/self/workspace-config", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,500,503], openapi = false)]
async fn current_worker_workspace_config_get(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
) -> Result<WorkspaceConfigCurrentResponse, WorkspaceConfigApiError>;
#[post("/api/w/{workspace_id}/workers/self/workspace-config", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,500,503], openapi = false)]
async fn current_worker_workspace_config_attach(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: WorkspaceConfigAttachRequest,
) -> Result<WorkspaceConfigAttachment, WorkspaceConfigApiError>;
#[post("/api/w/{workspace_id}/workers/self/workspace-config/observe", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,500,503], openapi = false)]
async fn current_worker_workspace_config_observe(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: WorkspaceConfigObserveRequest,
) -> Result<WorkspaceConfigObserveResponse, WorkspaceConfigApiError>;
#[post("/api/w/{workspace_id}/workers/self/workspace-config/read", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,500,503], openapi = false)]
async fn current_worker_workspace_config_read(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: WorkspaceConfigReadRequest,
) -> Result<WorkspaceConfigReadResponse, WorkspaceConfigApiError>;
#[post("/api/w/{workspace_id}/workers/self/workspace-config/commit", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,413,500,503], openapi = false)]
async fn current_worker_workspace_config_commit(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[body] request: WorkspaceConfigCommitRequest,
) -> Result<WorkspaceConfigCommitResponse, WorkspaceConfigApiError>;
#[post("/api/w/{workspace_id}/workspace-config-grants", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,500], bearer_auth = true, browser_auth = true)]
async fn workspace_config_grant_create(
&self,
#[extension] actor: RequestActor,
#[path] workspace_id: String,
#[body] request: WorkspaceConfigGrantCreateRequest,
) -> Result<WorkspaceConfigGrantResponse, WorkspaceConfigApiError>;
#[delete("/api/w/{workspace_id}/workspace-config-grants/{grant_id}", status = 200, error_status = 400, additional_error_statuses = [401,403,404,409,500], bearer_auth = true, browser_auth = true)]
async fn workspace_config_grant_revoke(
&self,
#[extension] actor: RequestActor,
#[path] workspace_id: String,
#[path] grant_id: String,
) -> Result<WorkspaceConfigGrantResponse, WorkspaceConfigApiError>;
#[get(
"/api/w/{workspace_id}/workers/self/workdir-catalog",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 500],
openapi = false,
)]
async fn current_worker_workdir_catalog(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[query] query: CurrentWorkerWorkdirCatalogQuery,
) -> Result<CurrentWorkerWorkdirCatalogResponse, RepositoryApiError>;
#[get(
"/api/w/{workspace_id}/workers/self/workdir-attachments",
status = 200,
error_status = 400,
additional_error_statuses = [401, 403, 404, 500],
openapi = false,
)]
async fn current_worker_workdir_attachment_list(
&self,
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[query] query: CurrentWorkerWorkdirAttachmentListQuery,
) -> Result<CurrentWorkerWorkdirAttachmentListResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/workers/self/workdir-attachments",
status = 200,
@@ -2590,6 +2703,7 @@ pub trait ServerApi {
#[extension] context: ServerRequestContext,
#[path] workspace_id: String,
#[path] alias: String,
#[query] query: CurrentWorkerWorkdirDetachQuery,
) -> Result<CurrentWorkerWorkdirAttachmentResponse, RepositoryApiError>;
#[post(
"/api/w/{workspace_id}/workers/self/workdir-session/operations",
@@ -4382,6 +4496,10 @@ pub struct ConfigSchemaContribution {
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional))]
pub projection_validator: Option<ConfigProjectionValidator>,
/// Completion-only shape for partial values; never materialized as defaults.
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional))]
pub authoring_source: Option<String>,
pub source_digest: String,
}
@@ -4595,9 +4713,20 @@ pub enum SubjektivSubjectState {
#[serde(deny_unknown_fields)]
pub struct SubjektivSubjectCreateRequest {
pub role: String,
#[serde(default)]
pub behavior_md: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct SubjektivSubjectBehaviorUpdateRequest {
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub expected_behavior_revision: u64,
pub behavior_md: String,
}
pub const SUBJEKTIV_BROWSER_MAX_LIST_LIMIT: usize = 100;
pub const SUBJEKTIV_MAX_BEHAVIOR_BYTES: usize = 16 * 1024;
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
@@ -4625,6 +4754,9 @@ pub struct SubjektivSubjectListResponse {
pub struct SubjektivSubjectResponse {
pub id: String,
pub role: String,
pub behavior_md: String,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub behavior_revision: u64,
pub state: SubjektivSubjectState,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub store_revision: u64,
@@ -4780,12 +4912,6 @@ pub struct SubjektivMemoryRevisionsQuery {
pub cursor: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct SubjektivSubjectWorkerStartRequest {
pub worker: CreateWorkspaceWorkerRequest,
}
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct SubjektivStageCandidateRequest {
@@ -4804,6 +4930,11 @@ pub struct SubjektivStageCandidateResponse {
#[serde(deny_unknown_fields)]
pub struct SubjektivRecordSessionRequest {
pub session_id: String,
/// Creation is allowed only while finalizing a newly generated Session or
/// recovering that exact lifecycle's durable incomplete state. Restoring a
/// legacy Session uses verification-only mode so current ownership cannot
/// invent historical attribution.
pub create_if_missing: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -5403,6 +5534,9 @@ pub struct SubjektivSurfacePrepareRequest {}
#[serde(deny_unknown_fields)]
pub struct SubjektivSurfacePrepareResponse {
pub generation_id: String,
/// Existing ready surface at this exact store revision, if no rebuild is needed.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub current_snapshot_id: Option<String>,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub store_revision: u64,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
@@ -5459,10 +5593,22 @@ pub struct SubjektivSurfaceFailureResponse {
pub status: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct SubjektivResidentContextOutput {
/// Exact user-managed document. Empty means explicitly unset.
pub behavior_md: String,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub behavior_revision: u64,
/// Independently-fresh generated Memory projection.
pub memory_surface: memory::backend::MemoryResidentSummaryOutput,
}
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
#[serde(tag = "operation", content = "input", rename_all = "snake_case")]
pub enum SubjektivMemoryBackendOperation {
ResidentSummary(memory::backend::MemoryResidentSummaryOperation),
ResidentContext(memory::backend::MemoryResidentSummaryOperation),
Query(SubjektivMemoryQueryRequest),
Read(SubjektivMemoryReadRequest),
ListRevisions(SubjektivMemoryListRevisionsRequest),
@@ -5635,6 +5781,7 @@ pub struct SubjektivMemoryStageExplicitResponse {
#[serde(tag = "result", content = "data", rename_all = "snake_case")]
pub enum SubjektivMemoryBackendResponse {
ResidentSummary(memory::backend::MemoryResidentSummaryOutput),
ResidentContext(SubjektivResidentContextOutput),
Query(SubjektivMemoryQueryResponse),
Read(SubjektivMemoryReadResponse),
ListRevisions(SubjektivMemoryListRevisionsResponse),
@@ -6265,6 +6412,7 @@ pub struct Diagnostic {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum WorkingDirectoryMaterializerKind {
LogicalWorkspaceConfig,
#[default]
RuntimeGitClone,
ClientHostedExternal,
@@ -6404,6 +6552,11 @@ pub struct RuntimeWorkingDirectorySummary {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum WorkingDirectorySource {
WorkspaceConfig {
access: WorkspaceConfigAccess,
content_path: String,
purpose: String,
},
Repository {
repository_key: String,
},
@@ -6771,6 +6924,69 @@ pub struct CurrentWorkerWorkdirAttachRequest {
pub working_directory_id: String,
}
/// Stable Workdir-ID keyset paging over the authoritative inventory, not a Runtime snapshot.
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CurrentWorkerWorkdirCatalogQuery {
/// Defaults to 50; valid range is 1..=100 scanned registry records.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub limit: Option<u32>,
/// Exclusive stable Workdir ID. An empty policy-filtered page can still have a next cursor.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub cursor: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CurrentWorkerWorkdirCatalogResponse {
pub workspace_id: String,
pub items: Vec<WorkingDirectorySummary>,
pub next_cursor: Option<String>,
/// Opaque digest of the complete authoritative inventory and its occupancy.
pub revision: String,
}
/// Bounded, alias-ordered paging over the current Worker's active attachments.
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CurrentWorkerWorkdirAttachmentListQuery {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub limit: Option<u32>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub offset: Option<u32>,
/// Exact caller-scoped connection lookup, applied before paging; 1..=128 bytes.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub connection_id: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CurrentWorkerWorkdirAttachmentItem {
pub alias: String,
pub working_directory_id: String,
pub capabilities: workdir::WorkdirSessionCapabilities,
/// Opaque durable identity of this attachment lifetime, not a Workdir ID.
pub connection_id: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CurrentWorkerWorkdirAttachmentListResponse {
pub workspace_id: String,
pub items: Vec<CurrentWorkerWorkdirAttachmentItem>,
pub next_offset: Option<u32>,
/// Opaque digest of this caller's complete active connection set, independent of query paging/filtering.
pub revision: String,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CurrentWorkerWorkdirDetachQuery {
/// When omitted, retain the normal Tool's unconditional detach contract.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub expected_connection_id: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct CurrentWorkerWorkdirAttachmentResponse {
@@ -8250,7 +8466,7 @@ pub struct RuntimeWorkerInputRequest {
pub kind: Option<String>,
pub content: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub segments: Option<Vec<serde_json::Value>>,
pub segments: Option<Vec<protocol::Segment>>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -8285,9 +8501,13 @@ pub struct RuntimeWorkerLifecycleResult {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct RuntimeWorkerCompletionsRequest {
pub kind: serde_json::Value,
pub kind: protocol::CompletionKind,
#[serde(default)]
pub prefix: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context: Option<protocol::CompletionContext>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub request_id: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema)]
@@ -8295,9 +8515,13 @@ pub struct RuntimeWorkerCompletionsRequest {
pub struct RuntimeWorkerCompletionsResult {
pub runtime_id: String,
pub worker_id: String,
pub kind: serde_json::Value,
pub kind: protocol::CompletionKind,
pub prefix: String,
pub entries: Vec<serde_json::Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context: Option<protocol::CompletionContext>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub request_id: Option<String>,
pub entries: Vec<protocol::CompletionEntry>,
#[serde(default)]
pub diagnostics: Vec<Diagnostic>,
}
@@ -9381,6 +9605,10 @@ pub struct RuntimeWorkerWorkdirAttachmentSummary {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
pub struct WorkerSummary {
/// Opaque Runtime-owned observation fence used by explicit Restore intent.
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub restore_observation_token: Option<String>,
pub runtime_id: String,
pub worker_id: String,
pub resource_key: String,
@@ -9489,6 +9717,14 @@ pub struct WorkerLaunchProfileCandidate {
pub id: String,
pub label: String,
pub description: String,
pub feature_connections: WorkerLaunchFeatureConnectionOptions,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerLaunchFeatureConnectionOptions {
pub subjektiv: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
@@ -9520,6 +9756,22 @@ pub struct CreateWorkspaceWorkerTicketAssignmentRequest {
pub operation_id: String,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceWorkerFeatureConnectionsRequest {
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub subjektiv: Option<SubjektivWorkerConnectionRequest>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct SubjektivWorkerConnectionRequest {
pub subject_id: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -9537,6 +9789,9 @@ pub struct CreateWorkspaceWorkerRequest {
pub initial_submit: Vec<protocol::Segment>,
#[serde(default)]
pub workdir_attachments: Vec<BrowserWorkerWorkingDirectorySelection>,
/// Explicit product Feature connections. Backend validation turns these into
/// trusted host attachments; Runtime and Profile inputs cannot self-attest them.
pub feature_connections: WorkspaceWorkerFeatureConnectionsRequest,
/// Backend idempotency key used only for authenticated Worker-owned spawn/control.
#[serde(default)]
pub control_operation_id: Option<String>,
@@ -9590,6 +9845,16 @@ pub enum WorkerRestoreState {
ReconciliationRequired,
}
/// One explicit Restore intent. Retries retain both fields; conflicts require a
/// fresh observation and a deliberate new intent, not automatic token replacement.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkerRestoreRequest {
pub expected_observation_token: String,
pub request_id: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
@@ -9851,7 +10116,11 @@ pub fn memory_api_typescript() -> String {
// `ts-rs` solely for this read-only projection.
let subjektiv_browser_declarations = r#"export type SubjektivSubjectState = "active" | "retired";
export type SubjektivSubjectResponse = { id: string, role: string, state: SubjektivSubjectState, store_revision: number, created_at: string, updated_at: string, current_worker?: | import("./worker-launch-api").WorkerLaunchWorkerSummary | null, };
export type SubjektivSubjectCreateRequest = { role: string, behavior_md?: string, };
export type SubjektivSubjectBehaviorUpdateRequest = { expected_behavior_revision: number, behavior_md: string, };
export type SubjektivSubjectResponse = { id: string, role: string, behavior_md: string, behavior_revision: number, state: SubjektivSubjectState, store_revision: number, created_at: string, updated_at: string, current_worker?: | import("./worker-launch-api").WorkerLaunchWorkerSummary | null, };
export type SubjektivSubjectListResponse = { limit: number, items: Array<SubjektivSubjectResponse>, next_cursor?: string | null, has_more: boolean, };
@@ -9884,8 +10153,11 @@ export type SubjektivMemoryListRevisionsResponse = { memory_id: string, current_
let limits = format!(
"export const MEMORY_API_LIMITS = {{\n maxResponseBytes: {MEMORY_API_MAX_RESPONSE_BYTES},\n maxDocumentBytes: {MEMORY_API_MAX_DOCUMENT_BYTES},\n maxCollectionItems: {MEMORY_API_MAX_COLLECTION_ITEMS},\n maxStringBytes: {MEMORY_API_MAX_STRING_BYTES},\n maxIdentifierBytes: {MEMORY_API_MAX_IDENTIFIER_BYTES},\n}} as const;"
);
let subjektiv_limits = format!(
"export const SUBJEKTIV_API_LIMITS = {{\n maxBehaviorBytes: {SUBJEKTIV_MAX_BEHAVIOR_BYTES},\n}} as const;"
);
format!(
"// Generated from server-api. Do not edit by hand.\n// Regenerate: cargo run -q -p server-api --features typescript --example generate_memory_api_types > web/workspace/src/lib/generated/memory-api.ts\n\n{limits}\n\n{}\n\n{subjektiv_browser_declarations}\n",
"// Generated from server-api. Do not edit by hand.\n// Regenerate: cargo run -q -p server-api --features typescript --example generate_memory_api_types > web/workspace/src/lib/generated/memory-api.ts\n\n{limits}\n\n{subjektiv_limits}\n\n{}\n\n{subjektiv_browser_declarations}\n",
declarations
.into_iter()
.map(|declaration| format!("export {declaration}"))
@@ -10560,6 +10832,22 @@ pub fn legacy_catalog_typescript() -> String {
ToolchainContract::decl(&config),
WorkspaceConfigTreeResponse::decl(&config),
ConfigCommitRequest::decl(&config),
WorkspaceConfigAccess::decl(&config),
WorkspaceConfigNodeKind::decl(&config),
WorkspaceConfigFailureClassification::decl(&config),
WorkspaceConfigAttachRequest::decl(&config),
WorkspaceConfigAttachment::decl(&config),
WorkspaceConfigCurrentResponse::decl(&config),
WorkspaceConfigObserveRequest::decl(&config),
WorkspaceConfigNode::decl(&config),
WorkspaceConfigObserveResponse::decl(&config),
WorkspaceConfigReadRequest::decl(&config),
WorkspaceConfigReadResponse::decl(&config),
WorkspaceConfigCommitRequest::decl(&config),
WorkspaceConfigCommitResponse::decl(&config),
WorkspaceConfigApiError::decl(&config),
WorkspaceConfigGrantCreateRequest::decl(&config),
WorkspaceConfigGrantResponse::decl(&config),
ProfileSettingsResponse::decl(&config),
WorkspaceProfileSummary::decl(&config),
WorkspaceProfileSourceSummary::decl(&config),
@@ -10703,11 +10991,14 @@ pub fn worker_launch_api_typescript() -> String {
RuntimeWorkerWorkdirAttachmentSummary::decl(&config),
WorkerLaunchWorkerSummary::decl(&config),
WorkerLaunchRuntimeOption::decl(&config),
WorkerLaunchFeatureConnectionOptions::decl(&config),
WorkerLaunchProfileCandidate::decl(&config),
WorkingDirectoryRepositoryOption::decl(&config),
WorkerLaunchOptionsResponse::decl(&config),
BrowserWorkerWorkingDirectorySelection::decl(&config),
CreateWorkspaceWorkerTicketAssignmentRequest::decl(&config),
SubjektivWorkerConnectionRequest::decl(&config),
WorkspaceWorkerFeatureConnectionsRequest::decl(&config),
CreateWorkspaceWorkerRequest::decl(&config),
BrowserCreateWorkerResponse::decl(&config),
BrowserWorkspaceOrchestratorResponse::decl(&config),
@@ -11428,6 +11719,28 @@ mod tests {
);
}
#[test]
fn worker_restore_request_requires_observation_and_request_identity() {
let valid = serde_json::json!({"expected_observation_token":"opaque-generation", "request_id":"restore-request"});
let request: WorkerRestoreRequest = serde_json::from_value(valid.clone()).unwrap();
assert_eq!(serde_json::to_value(request).unwrap(), valid);
for invalid in [
serde_json::json!({}),
serde_json::json!({"request_id":"restore-request"}),
serde_json::json!({"expected_observation_token":"opaque-generation"}),
serde_json::json!({"expected_observation_token":"opaque-generation", "request_id":"restore-request", "command_id":701}),
] {
assert!(serde_json::from_value::<WorkerRestoreRequest>(invalid).is_err());
}
let contract = canonical_openapi_document().unwrap().to_json().unwrap();
let contract: serde_json::Value = serde_json::from_str(&contract).unwrap();
let restore = &contract["paths"]["/api/w/{workspace_id}/runtimes/{runtime_id}/workers/{worker_id}/restore"]
["post"];
assert_eq!(restore["requestBody"]["required"], true);
assert!(restore["responses"]["409"].is_object());
assert!(contract["components"]["schemas"]["WorkerRestoreRequest"].is_object());
}
#[test]
fn worker_restore_result_rejects_unknown_fields() {
let result = serde_json::from_value::<WorkerRestoreResult>(serde_json::json!({
@@ -11748,6 +12061,7 @@ mod tests {
ticket_assignment: None,
initial_submit: Vec::new(),
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
})
.unwrap();
@@ -11761,6 +12075,7 @@ mod tests {
"ticket_assignment": null,
"initial_submit": [],
"workdir_attachments": [],
"feature_connections": {},
"control_operation_id": null,
})
);
@@ -11913,6 +12228,135 @@ mod tests {
);
}
#[test]
fn current_worker_attachment_contract_keeps_normal_tool_response_closed() {
let lookup = CurrentWorkerWorkdirAttachmentListQuery {
limit: Some(1),
offset: None,
connection_id: Some("opaque-connection".to_string()),
};
let value = serde_json::json!({"limit": 1, "connection_id": "opaque-connection"});
assert_eq!(serde_json::to_value(&lookup).unwrap(), value);
assert_eq!(
serde_json::from_value::<CurrentWorkerWorkdirAttachmentListQuery>(value).unwrap(),
lookup
);
assert_eq!(
serde_json::to_value(CurrentWorkerWorkdirAttachmentListQuery::default()).unwrap(),
serde_json::json!({})
);
assert!(
serde_json::from_value::<CurrentWorkerWorkdirAttachmentListQuery>(
serde_json::json!({"connection_id": 123})
)
.is_err()
);
assert!(
serde_json::from_value::<CurrentWorkerWorkdirAttachmentListQuery>(
serde_json::json!({"expected_connection_id": "wrong-field"})
)
.is_err()
);
let query: CurrentWorkerWorkdirDetachQuery =
serde_json::from_value(serde_json::json!({})).unwrap();
assert_eq!(query.expected_connection_id, None);
assert_eq!(serde_json::to_value(query).unwrap(), serde_json::json!({}));
assert!(
serde_json::from_value::<CurrentWorkerWorkdirDetachQuery>(
serde_json::json!({"connection_id":"wrong-field"})
)
.is_err()
);
let response = CurrentWorkerWorkdirAttachmentResponse {
workspace_id: "workspace".to_string(),
alias: "checkout".to_string(),
working_directory_id: "workdir".to_string(),
capabilities: workdir::WorkdirSessionCapabilities::EMPTY,
attached: false,
};
let value = serde_json::to_value(&response).unwrap();
assert_eq!(value.as_object().unwrap().len(), 5);
assert!(value.get("connection_id").is_none());
assert_eq!(
serde_json::from_value::<CurrentWorkerWorkdirAttachmentResponse>(value).unwrap(),
response
);
let list = CurrentWorkerWorkdirAttachmentListResponse {
workspace_id: "workspace".to_string(),
revision: "opaque-revision".to_string(),
next_offset: Some(1),
items: vec![CurrentWorkerWorkdirAttachmentItem {
alias: "checkout".to_string(),
working_directory_id: "workdir".to_string(),
capabilities: workdir::WorkdirSessionCapabilities::READ_ONLY,
connection_id: "opaque".to_string(),
}],
};
assert_eq!(
serde_json::from_value::<CurrentWorkerWorkdirAttachmentListResponse>(
serde_json::to_value(&list).unwrap()
)
.unwrap(),
list
);
}
#[test]
fn current_worker_workdir_catalog_contract_is_bounded_and_revision_is_required() {
let query = CurrentWorkerWorkdirCatalogQuery {
limit: Some(100),
cursor: Some("stable-id".to_string()),
};
assert_eq!(
serde_json::from_value::<CurrentWorkerWorkdirCatalogQuery>(
serde_json::to_value(&query).unwrap()
)
.unwrap(),
query
);
assert_eq!(
serde_json::to_value(CurrentWorkerWorkdirCatalogQuery::default()).unwrap(),
serde_json::json!({})
);
assert!(
serde_json::from_value::<CurrentWorkerWorkdirCatalogQuery>(
serde_json::json!({"offset": 0})
)
.is_err()
);
assert!(
serde_json::from_value::<CurrentWorkerWorkdirCatalogQuery>(
serde_json::json!({"limit": -1})
)
.is_err()
);
let response = CurrentWorkerWorkdirCatalogResponse {
workspace_id: "workspace".to_string(),
items: Vec::new(),
next_cursor: Some("scanned-legacy-row".to_string()),
revision: "opaque".to_string(),
};
assert_eq!(
serde_json::from_value::<CurrentWorkerWorkdirCatalogResponse>(
serde_json::to_value(&response).unwrap()
)
.unwrap(),
response
);
assert!(
serde_json::from_value::<CurrentWorkerWorkdirCatalogResponse>(
serde_json::json!({"workspace_id": "workspace", "items": [], "next_cursor": null})
)
.is_err()
);
assert!(
serde_json::from_value::<CurrentWorkerWorkdirAttachmentListResponse>(
serde_json::json!({"workspace_id": "workspace", "items": [], "next_offset": null})
)
.is_err()
);
}
#[test]
fn workspace_and_repository_response_shapes_round_trip() {
let workspace = serde_json::json!({
@@ -12644,6 +13088,45 @@ mod tests {
assert!(serde_json::from_value::<CompanionTranscriptItem>(private_item).is_err());
}
#[test]
fn structured_invocation_input_round_trips_through_typed_api() {
let json = serde_json::json!({"kind":"user", "content":"", "segments":[{"kind":"feature_invoke", "invocation":{
"invocation_id":"stable-invoke", "identity":"builtin:test/prepare", "name":"prepare",
"arguments":[{"name":"path", "value":{"kind":"string","value":"資料/a b"}}]
}}]});
let request: RuntimeWorkerInputRequest = serde_json::from_value(json.clone()).unwrap();
assert!(
matches!(&request.segments.as_ref().unwrap()[0], protocol::Segment::FeatureInvoke { invocation }
if invocation.identity.0 == "builtin:test/prepare" && invocation.invocation_id == "stable-invoke")
);
assert_eq!(serde_json::to_value(request).unwrap(), json);
let mut invalid = json;
invalid["segments"][0]["invocation"]["arguments"][0]["value"] =
serde_json::json!({"kind":"integer","value":2147483648_i64});
assert!(serde_json::from_value::<RuntimeWorkerInputRequest>(invalid).is_err());
}
#[test]
fn feature_completion_contract_is_typed_and_preserves_argument_context() {
let json = serde_json::json!({"kind":"feature_argument", "prefix":"資料/", "context":{"invocation":"builtin:test/prepare","argument":"path"}});
let request: RuntimeWorkerCompletionsRequest =
serde_json::from_value(json.clone()).unwrap();
assert_eq!(request.kind, protocol::CompletionKind::FeatureArgument);
assert_eq!(serde_json::to_value(request).unwrap(), json);
assert!(
serde_json::from_value::<RuntimeWorkerCompletionsRequest>(
serde_json::json!({"kind":"unknown"})
)
.is_err()
);
assert!(
serde_json::from_value::<RuntimeWorkerCompletionsRequest>(
serde_json::json!({"kind":"feature_argument", "context":{"invocation":123}})
)
.is_err()
);
}
#[test]
fn generated_companion_api_contract_is_current() {
let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
@@ -12878,6 +13361,13 @@ mod openapi_artifact_tests {
// identity, or a one-use Reviewer capability. They remain generated Rust client/Axum
// operations, but must not appear as unauthenticated operations in the public OpenAPI.
const SIGNED_INTERNAL: &[&str] = &[
"current_worker_workspace_config_get",
"current_worker_workspace_config_attach",
"current_worker_workspace_config_observe",
"current_worker_workspace_config_read",
"current_worker_workspace_config_commit",
"current_worker_workdir_catalog",
"current_worker_workdir_attachment_list",
"current_worker_workdir_attach",
"current_worker_workdir_detach",
"current_worker_workdir_operation",
@@ -872,14 +872,14 @@ fn validate_runtime_operations(
"/api/runtimes/{runtime_id}/workers/{worker_id}/restore",
"post",
"runtime_worker_restore_alias",
None,
Some("WorkerRestoreRequest"),
&[("200", "WorkerRestoreResponse")],
),
runtime_operation(
"/api/w/{workspace_id}/runtimes/{runtime_id}/workers/{worker_id}/restore",
"post",
"runtime_worker_restore",
None,
Some("WorkerRestoreRequest"),
&[("200", "WorkerRestoreResponse")],
),
runtime_operation(
@@ -1227,8 +1227,16 @@ fn json_content_schema<'a>(
}
fn schema_reference<'a>(value: &'a Value, context: &str) -> Result<&'a str, GenerationError> {
const ALLOWED_REFERENCE_SIBLINGS: &[&str] =
&["$ref", "$comment", "deprecated", "description", "title"];
// `default` is a JSON Schema annotation, not a constraint or a replacement
// for the referenced type. Schemars emits it for defaulted enum fields.
const ALLOWED_REFERENCE_SIBLINGS: &[&str] = &[
"$ref",
"$comment",
"default",
"deprecated",
"description",
"title",
];
let schema = object(value, context)?;
if let Some(keyword) = schema
.keys()
@@ -2023,6 +2031,27 @@ mod tests {
assert!(error.to_string().contains("ambiguous anyOf"));
}
#[test]
fn reference_default_annotations_do_not_replace_the_referenced_type() {
let value = serde_json::json!({"$ref":"#/components/schemas/InvocationCompletion", "default":{"kind":"none"}});
assert_eq!(
schema_reference(&value, "completion").unwrap(),
"#/components/schemas/InvocationCompletion"
);
let output = generate_runtime_typescript(
&crate::canonical_openapi_document()
.unwrap()
.to_json()
.unwrap(),
)
.unwrap();
assert!(output.contains("completion?: InvocationCompletion"));
assert!(output.contains("export type FeatureInvocation ="));
assert!(output.contains(
"export type CompletionKind = \"file\" | \"feature\" | \"feature_argument\""
));
}
#[test]
fn generator_rejects_lossy_reference_siblings() {
let mut document: Value = serde_json::from_str(OPENAPI).unwrap();
+284
View File
@@ -0,0 +1,284 @@
//! Grant-bound logical Workspace configuration transport. No host filesystem contract.
use super::{ConfigCommitRequest, ConfigContentType};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum WorkspaceConfigAccess {
ReadOnly,
ReadWrite,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum WorkspaceConfigNodeKind {
Directory,
File,
Missing,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum WorkspaceConfigFailureClassification {
NotCommitted,
Unknown,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigAttachRequest {
#[serde(default)]
pub alias: Option<String>,
#[serde(default)]
pub access: Option<WorkspaceConfigAccess>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigAttachment {
pub workspace_id: String,
pub connection_id: String,
pub alias: String,
pub working_directory_id: String,
pub access: WorkspaceConfigAccess,
pub name: String,
pub purpose: String,
pub content_path: String,
pub already_attached: bool,
}
/// Named transport wrapper required by the generated API; JSON remains attachment-or-null.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(transparent)]
pub struct WorkspaceConfigCurrentResponse(pub Option<WorkspaceConfigAttachment>);
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigObserveRequest {
pub connection_id: String,
pub paths: Vec<String>,
pub depth: u32,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigNode {
pub path: String,
pub kind: WorkspaceConfigNodeKind,
pub validator: String,
pub digest: Option<String>,
pub content_type: Option<ConfigContentType>,
pub operations: Vec<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigObserveResponse {
pub connection_id: String,
pub validator: String,
#[cfg_attr(feature = "typescript", ts(type = "number"))]
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub revision: u64,
pub digest: String,
/// Canonical commit entrypoints from this exact metadata snapshot.
pub entrypoints: Vec<String>,
pub nodes: Vec<WorkspaceConfigNode>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigReadRequest {
pub connection_id: String,
pub path: String,
pub validator: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigReadResponse {
pub path: String,
pub content: String,
pub content_type: ConfigContentType,
pub digest: String,
pub validator: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigCommitRequest {
pub connection_id: String,
pub validator: String,
pub request: ConfigCommitRequest,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigCommitResponse {
pub validator: String,
#[cfg_attr(feature = "typescript", ts(type = "number"))]
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub revision: u64,
pub digest: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigApiError {
pub status: u16,
pub code: String,
pub message: String,
pub classification: WorkspaceConfigFailureClassification,
}
impl api_macros::HttpError for WorkspaceConfigApiError {
fn status_code(&self) -> u16 {
self.status
}
}
impl api_macros::HttpRequestError for WorkspaceConfigApiError {
fn from_request_rejection(status: u16, _message: String) -> Self {
Self {
status,
code: "invalid_request".into(),
message: "Invalid configuration request".into(),
classification: WorkspaceConfigFailureClassification::NotCommitted,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigGrantCreateRequest {
pub runtime_id: String,
pub worker_id: String,
pub access: WorkspaceConfigAccess,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigGrantResponse {
pub grant_id: String,
pub workspace_id: String,
pub runtime_id: String,
pub worker_id: String,
pub working_directory_id: String,
pub access: WorkspaceConfigAccess,
pub revoked: bool,
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{ApiContract, HttpMethod, ServerApiMetadata};
#[test]
fn workspace_config_observation_is_metadata_only_and_has_same_snapshot_cas_inputs() {
let value = serde_json::json!({"connection_id":"c", "validator":"v", "revision":7, "digest":"d", "entrypoints":["main.dcdl"], "nodes":[{"path":"", "kind":"directory", "validator":"v", "digest":null, "content_type":null, "operations":["apply_changes"]}]});
let response: WorkspaceConfigObserveResponse =
serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(response).unwrap(), value);
for field in ["revision", "digest", "entrypoints"] {
let mut missing = value.clone();
missing.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<WorkspaceConfigObserveResponse>(missing).is_err());
}
let mut with_body = value;
with_body["nodes"][0]["content"] = serde_json::json!("must never appear here");
assert!(serde_json::from_value::<WorkspaceConfigObserveResponse>(with_body).is_err());
assert!(
serde_json::from_value::<WorkspaceConfigGrantCreateRequest>(
serde_json::json!({"runtime_id":"r","worker_id":"w","access":"read_write_command"})
)
.is_err()
);
assert!(
serde_json::from_value::<WorkspaceConfigAttachRequest>(
serde_json::json!({"grant_id":"invented"})
)
.is_err()
);
assert_eq!(
serde_json::to_value(WorkspaceConfigCurrentResponse(None)).unwrap(),
serde_json::Value::Null
);
}
#[test]
fn workspace_config_failure_classification_roundtrips_and_rejections_are_pre_effects() {
for classification in [
WorkspaceConfigFailureClassification::NotCommitted,
WorkspaceConfigFailureClassification::Unknown,
] {
let error = WorkspaceConfigApiError {
status: 500,
code: "unavailable".into(),
message: "safe".into(),
classification,
};
assert_eq!(
serde_json::from_value::<WorkspaceConfigApiError>(
serde_json::to_value(&error).unwrap()
)
.unwrap(),
error
);
assert_eq!(api_macros::HttpError::status_code(&error), 500);
}
let error =
<WorkspaceConfigApiError as api_macros::HttpRequestError>::from_request_rejection(
413,
"/private/secret".into(),
);
assert_eq!(
error.classification,
WorkspaceConfigFailureClassification::NotCommitted
);
assert!(!error.message.contains("private"));
}
#[test]
fn workspace_config_generated_routes_are_exact_and_grants_are_separate() {
for (name, method, suffix) in [
("get", HttpMethod::Get, ""),
("attach", HttpMethod::Post, ""),
("observe", HttpMethod::Post, "/observe"),
("read", HttpMethod::Post, "/read"),
("commit", HttpMethod::Post, "/commit"),
] {
let name = format!("current_worker_workspace_config_{name}");
let operation = ServerApiMetadata::OPERATIONS
.iter()
.find(|operation| operation.operation_id == name)
.unwrap();
assert_eq!(operation.method, method);
assert_eq!(
operation.path,
format!("/api/w/{{workspace_id}}/workers/self/workspace-config{suffix}")
);
}
for (name, method, path) in [
(
"workspace_config_grant_create",
HttpMethod::Post,
"/api/w/{workspace_id}/workspace-config-grants",
),
(
"workspace_config_grant_revoke",
HttpMethod::Delete,
"/api/w/{workspace_id}/workspace-config-grants/{grant_id}",
),
] {
let operation = ServerApiMetadata::OPERATIONS
.iter()
.find(|operation| operation.operation_id == name)
.unwrap();
assert_eq!(operation.method, method);
assert_eq!(operation.path, path);
}
}
}
+6 -3
View File
@@ -211,7 +211,7 @@ impl FsStore {
}
}
fn parse_jsonl(
pub(crate) fn parse_jsonl(
content: &[u8],
session_id: SessionId,
segment_id: SegmentId,
@@ -383,8 +383,11 @@ impl Store for FsStore {
content.push_str(&serde_json::to_string(entry)?);
content.push('\n');
}
fs::write(&path, content.as_bytes())?;
Ok(())
let _guard = self
.append_lock
.lock()
.map_err(|_| std::io::Error::other("session store append lock was poisoned"))?;
crate::worker_session_store::atomic_write_bytes(&path, content.as_bytes())
}
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
+2
View File
@@ -40,6 +40,8 @@ pub struct LoggedWorkerSubject {
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum LoggedSessionHistoryOrigin {
/// Human input verified by the standalone Host, without a Backend Account.
LocalHumanInput,
HumanInput {
account_id: String,
},
+9 -6
View File
@@ -110,9 +110,12 @@ pub use public_snapshot::{
DEFAULT_RETAINED_HISTORY_MAX_ENTRIES, DEFAULT_RETAINED_HISTORY_MAX_RESPONSE_BYTES,
DEFAULT_RETAINED_HISTORY_MAX_SCAN_BYTES, DEFAULT_RETAINED_HISTORY_MAX_SEGMENTS,
DEFAULT_RETAINED_HISTORY_PAGE_TURNS, DEFAULT_RETAINED_SNAPSHOT_MAX_BYTES,
MAX_RETAINED_HISTORY_PAGE_TURNS, RetainedHistoryReadError, RetainedHistoryReadLimits,
RetainedSessionIdentity, RetainedSessionSnapshot, RetainedSnapshotReadError,
project_session_snapshot, read_retained_session_history_page, read_retained_session_snapshot,
MAX_RETAINED_HISTORY_PAGE_TURNS, RetainedAttachmentReadError, RetainedHistoryReadError,
RetainedHistoryReadLimits, RetainedSessionAttachment, RetainedSessionIdentity,
RetainedSessionSnapshot, RetainedSnapshotReadError, project_session_snapshot,
read_retained_session_attachment, read_retained_session_history_page,
read_retained_session_snapshot, read_session_attachment_from_entries,
session_tool_attachment_id,
};
pub use segment::{
SegmentStartState, append_entry, append_system_item, classify_logged_history_entry,
@@ -132,9 +135,9 @@ pub use uploaded_file::{
UploadedFileUploadContext,
};
pub use worker_metadata::{
CombinedStore, FsWorkerStore, WorkerActiveSegmentRef, WorkerAggregateStore, WorkerMetadata,
WorkerMetadataStore, WorkerPeer, WorkerReclaimedChild, WorkerSpawnedChild,
WorkerSpawnedScopeRule, WorkerStoreError, validate_worker_name,
CombinedStore, FsWorkerStore, SubjektivSessionAttributionState, WorkerActiveSegmentRef,
WorkerAggregateStore, WorkerMetadata, WorkerMetadataStore, WorkerPeer, WorkerReclaimedChild,
WorkerSpawnedChild, WorkerSpawnedScopeRule, WorkerStoreError, validate_worker_name,
};
pub use worker_session_store::WorkerSessionStore;
+168 -3
View File
@@ -208,6 +208,172 @@ pub fn read_session_public_index(
let segment_ids = store
.list_segments_read_only(session_id)
.map_err(map_store_error)?;
project_public_index_records(session_id, segment_ids, limits, |segment_id, remaining| {
store
.read_all_read_only_bounded(session_id, segment_id, remaining)
.map_err(map_store_error)
})
}
/// Strict committed public projection for the standalone FsStore layout.
/// Uses the same lineage, commit filtering and entry projection as retained
/// WorkerSessionStore observation. No writes, migrations, trace/artifact reads,
/// access-time updates or directory creation occur here.
pub fn read_fs_session_public_index(
sessions_root: &Path,
session_id: SessionId,
limits: SessionPublicIndexLimits,
) -> Result<SessionPublicIndex, SessionPublicIndexReadError> {
read_fs_session_evidence_index(sessions_root, session_id, limits).map(|value| value.index)
}
/// Producer-compatible numeric hints for stable public evidence references.
/// Both committed annotated-history and filtered public-snapshot capture are
/// valid producers. No hidden history content is exposed by this metadata.
pub struct FsSessionEvidenceIndex {
pub index: SessionPublicIndex,
pub ranges: HashMap<(String, String), Vec<[u64; 2]>>,
pub has_open_run: bool,
}
pub fn read_fs_session_evidence_index(
sessions_root: &Path,
session_id: SessionId,
limits: SessionPublicIndexLimits,
) -> Result<FsSessionEvidenceIndex, SessionPublicIndexReadError> {
if limits.max_bytes == 0 || limits.max_segments == 0 || limits.max_entries == 0 {
return Err(SessionPublicIndexReadError::ResourceLimit);
}
let session_dir = sessions_root.join(session_id.to_string());
// IDs are typed and every storage component is observationally validated;
// a known Session ID never authorizes following arbitrary local symlinks.
let mut prefix = std::path::PathBuf::new();
for part in session_dir.components() {
prefix.push(part);
let metadata =
std::fs::symlink_metadata(&prefix).map_err(|_| SessionPublicIndexReadError::Missing)?;
if metadata.file_type().is_symlink() {
return Err(SessionPublicIndexReadError::Storage);
}
}
let mut segment_ids = Vec::new();
for entry in
std::fs::read_dir(&session_dir).map_err(|_| SessionPublicIndexReadError::Missing)?
{
let entry = entry.map_err(|_| SessionPublicIndexReadError::Storage)?;
let path = entry.path();
if path.extension().and_then(|v| v.to_str()) != Some("jsonl") {
continue;
}
let Some(id) = path
.file_stem()
.and_then(|v| v.to_str())
.and_then(|v| v.parse::<SegmentId>().ok())
else {
continue;
};
if !entry
.file_type()
.map_err(|_| SessionPublicIndexReadError::Storage)?
.is_file()
{
return Err(SessionPublicIndexReadError::Storage);
}
segment_ids.push(id);
if segment_ids.len() > limits.max_segments {
return Err(SessionPublicIndexReadError::ResourceLimit);
}
}
segment_ids.sort();
if segment_ids.is_empty() {
return Err(SessionPublicIndexReadError::Missing);
}
let mut ranges = HashMap::<(String, String), Vec<[u64; 2]>>::new();
let mut has_open_run = false;
let index =
project_public_index_records(session_id, segment_ids, limits, |segment_id, remaining| {
let path = session_dir.join(format!("{segment_id}.jsonl"));
let bytes = crate::read_without_atime_bounded(&path, remaining)
.map_err(|_| SessionPublicIndexReadError::Storage)?;
if bytes.len() as u64 > remaining {
return Err(SessionPublicIndexReadError::ResourceLimit);
}
let entries = crate::FsStore::parse_jsonl(&bytes, session_id, segment_id)
.map_err(map_store_error)?;
let mut open = false;
for entry in &entries {
match entry {
LogEntry::Invoke { .. } | LogEntry::RunResumed { .. } => open = true,
LogEntry::RunCompleted { .. }
| LogEntry::RunYielded { .. }
| LogEntry::RunCancelled { .. }
| LogEntry::RunErrored { .. }
| LogEntry::PausedTurnAbandoned { .. } => open = false,
_ => {}
}
}
has_open_run |= open;
let committed = committed_conversation_records(&entries);
let state = crate::segment_log::collect_state(&committed);
for (position, entry) in state.annotated_history.iter().enumerate() {
ranges
.entry((
segment_id.to_string(),
format!("E{}", entry.metadata.entry_id.0),
))
.or_default()
.push([position as u64, position as u64]);
}
let snapshot = crate::public_snapshot::project_session_snapshot_for_segment(
session_id,
Some(segment_id),
&committed,
);
let mut position = 0_u64;
for entry in snapshot.entries {
if matches!(
entry.data,
SessionSnapshotEntryData::UserInput { .. }
| SessionSnapshotEntryData::Message { .. }
| SessionSnapshotEntryData::ToolCall { .. }
| SessionSnapshotEntryData::ToolResult { .. }
) {
ranges
.entry((segment_id.to_string(), format!("E{}", entry.entry_id)))
.or_default()
.push([position, position]);
position += 1;
}
}
Ok((entries, bytes.len() as u64))
})?;
// Intersect with the strict public projection, never turn knowledge of a
// hidden/reasoning history identity into a public evidence reference.
let public = index
.segments
.iter()
.flat_map(|seg| {
seg.entries
.iter()
.map(|entry| (seg.segment_id.clone(), entry.entry_ref.clone()))
})
.collect::<HashSet<_>>();
ranges.retain(|key, _| public.contains(key));
Ok(FsSessionEvidenceIndex {
index,
ranges,
has_open_run,
})
}
fn project_public_index_records(
session_id: SessionId,
segment_ids: Vec<SegmentId>,
limits: SessionPublicIndexLimits,
mut read_segment: impl FnMut(
SegmentId,
u64,
) -> Result<(Vec<LogEntry>, u64), SessionPublicIndexReadError>,
) -> Result<SessionPublicIndex, SessionPublicIndexReadError> {
if segment_ids.len() > limits.max_segments {
return Err(SessionPublicIndexReadError::ResourceLimit);
}
@@ -224,9 +390,7 @@ pub fn read_session_public_index(
.max_bytes
.checked_sub(scanned_bytes)
.ok_or(SessionPublicIndexReadError::ResourceLimit)?;
let (entries, bytes) = store
.read_all_read_only_bounded(session_id, segment_id, remaining)
.map_err(map_store_error)?;
let (entries, bytes) = read_segment(segment_id, remaining)?;
scanned_bytes = scanned_bytes
.checked_add(bytes)
.ok_or(SessionPublicIndexReadError::ResourceLimit)?;
@@ -1071,6 +1235,7 @@ fn persisted_entry_units(entries: &[LogEntry]) -> usize {
| LogEntry::AnnotatedUserInput { history, .. } => history.len(),
LogEntry::AnnotatedAssistantItem { .. }
| LogEntry::AnnotatedToolResult { .. }
| LogEntry::ToolResultCorrected { .. }
| LogEntry::AnnotatedSystemItem { .. } => 1,
_ => 0,
};
+754 -42
View File
@@ -1,7 +1,4 @@
use base64::{
Engine as _,
engine::general_purpose::{STANDARD as BASE64, URL_SAFE_NO_PAD},
};
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use protocol::{
Segment, SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole, SessionSnapshot,
@@ -10,7 +7,7 @@ use protocol::{
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use std::collections::HashSet;
use std::collections::{HashMap, HashSet};
use std::path::Path;
use thiserror::Error;
@@ -54,6 +51,32 @@ pub enum RetainedSnapshotReadError {
SnapshotTooLarge,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RetainedSessionAttachment {
pub media_type: String,
pub data: Vec<u8>,
}
#[derive(Debug, Error, PartialEq, Eq)]
pub enum RetainedAttachmentReadError {
#[error("retained worker state is unavailable")]
RetentionMissing,
#[error("retained worker state has no active session pointer")]
ActivePointerMissing,
#[error("retained session identity does not match")]
SessionMismatch,
#[error("session attachment was not found")]
NotFound,
#[error("retained session requires migration")]
MigrationRequired,
#[error("retained session log is corrupt")]
CorruptLog,
#[error("retained session storage is unavailable")]
StorageUnavailable,
#[error("retained session attachment exceeds a resource limit")]
ResourceLimit,
}
pub const DEFAULT_RETAINED_HISTORY_PAGE_TURNS: usize = 5;
pub const MAX_RETAINED_HISTORY_PAGE_TURNS: usize = 5;
pub const DEFAULT_RETAINED_HISTORY_MAX_SCAN_BYTES: u64 = 64 * 1024 * 1024;
@@ -116,6 +139,17 @@ struct RetainedHistoryCursor {
before_offset: u64,
record_end_offset: Option<u64>,
seed_entry_index: Option<usize>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
corrections: Vec<ToolCorrectionPosition>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
struct ToolCorrectionPosition {
entry_id: String,
segment_id: SegmentId,
start: u64,
end: u64,
}
#[derive(Debug)]
@@ -175,7 +209,9 @@ pub fn read_retained_session_snapshot(
.ok_or(RetainedSnapshotReadError::ActivePointerMissing)?;
let store = WorkerSessionStore::open_read_only(aggregate_root.join("session"))
.map_err(map_store_error)?;
if store.segment_log_len(segment_id).map_err(map_store_error)? > max_bytes {
if store.segment_log_len(segment_id).map_err(map_store_error)?
> DEFAULT_RETAINED_HISTORY_MAX_SCAN_BYTES
{
return Err(RetainedSnapshotReadError::SnapshotTooLarge);
}
let entries = store
@@ -199,6 +235,153 @@ pub fn read_retained_session_snapshot(
})
}
/// Resolve one immutable image body from the active retained Session. The
/// caller supplies only public identities; storage paths never cross this API.
pub fn read_retained_session_attachment(
aggregate_root: &Path,
worker_name: &str,
expected_session_id: &str,
attachment_id: &str,
max_scan_bytes: u64,
max_attachment_bytes: u64,
) -> Result<RetainedSessionAttachment, RetainedAttachmentReadError> {
let aggregate = WorkerAggregateStore::open_read_only(aggregate_root, worker_name)
.map_err(map_attachment_worker_store_error)?;
let metadata = aggregate
.read_read_only()
.map_err(map_attachment_worker_store_error)?
.ok_or(RetainedAttachmentReadError::RetentionMissing)?;
let active = metadata
.active
.ok_or(RetainedAttachmentReadError::ActivePointerMissing)?;
if active.session_id.to_string() != expected_session_id {
return Err(RetainedAttachmentReadError::SessionMismatch);
}
let segment_id = active
.segment_id
.ok_or(RetainedAttachmentReadError::ActivePointerMissing)?;
let store = WorkerSessionStore::open_read_only(aggregate_root.join("session"))
.map_err(map_attachment_store_error)?;
let limits = RetainedHistoryReadLimits {
max_scan_bytes,
..RetainedHistoryReadLimits::default()
};
let (lineage, _, mut scanned_bytes, mut scanned_entries) =
load_adopted_lineage(&store, active.session_id, segment_id, limits)
.map_err(map_history_attachment_error)?;
for segment in &lineage {
let adopted_end = locate_adopted_segment_end(
&store,
active.session_id,
segment,
limits,
&mut scanned_bytes,
&mut scanned_entries,
)
.map_err(map_history_attachment_error)?;
let mut reader = store
.open_retained_segment_reader(active.session_id, segment.segment_id, adopted_end)
.map_err(map_attachment_store_error)?;
loop {
let remaining = limits.max_scan_bytes.saturating_sub(scanned_bytes);
let (record, bytes) = reader
.previous_record(remaining)
.map_err(map_attachment_store_error)?;
scanned_bytes = scanned_bytes
.checked_add(bytes)
.ok_or(RetainedAttachmentReadError::ResourceLimit)?;
let Some(record) = record else {
break;
};
scanned_entries = scanned_entries
.checked_add(persisted_entry_units(std::slice::from_ref(&record.entry)))
.ok_or(RetainedAttachmentReadError::ResourceLimit)?;
if scanned_entries > limits.max_entries {
return Err(RetainedAttachmentReadError::ResourceLimit);
}
if let Some(result) =
session_attachment_from_record(&record.entry, attachment_id, max_attachment_bytes)
{
return result;
}
}
}
Err(RetainedAttachmentReadError::NotFound)
}
pub fn read_session_attachment_from_entries(
entries: &[LogEntry],
attachment_id: &str,
max_attachment_bytes: u64,
) -> Result<RetainedSessionAttachment, RetainedAttachmentReadError> {
for record in entries {
if let Some(result) =
session_attachment_from_record(record, attachment_id, max_attachment_bytes)
{
return result;
}
}
Err(RetainedAttachmentReadError::NotFound)
}
fn session_attachment_from_record(
record: &LogEntry,
attachment_id: &str,
max_attachment_bytes: u64,
) -> Option<Result<RetainedSessionAttachment, RetainedAttachmentReadError>> {
let mut found = None;
visit_record_history(record, &mut |entry| {
if found.is_none() {
found = attachment_from_entry(entry, attachment_id, max_attachment_bytes);
}
});
found
}
fn visit_record_history(record: &LogEntry, visit: &mut impl FnMut(&LoggedHistoryEntry)) {
match record {
LogEntry::AnnotatedSegmentStart { history, .. }
| LogEntry::AnnotatedUserInput { history, .. } => {
for entry in history {
visit(entry);
}
}
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. } => visit(entry),
_ => {}
}
}
fn attachment_from_entry(
entry: &LoggedHistoryEntry,
requested_id: &str,
max_attachment_bytes: u64,
) -> Option<Result<RetainedSessionAttachment, RetainedAttachmentReadError>> {
let LoggedItem::ToolResult { attachments, .. } = &entry.item else {
return None;
};
for (attachment_index, attachment) in attachments.iter().enumerate() {
let crate::logged_item::LoggedAttachment::Image { mime_type, data } = attachment;
if session_tool_attachment_id(
&entry.metadata.entry_id.0,
attachment_index,
mime_type,
data,
) != requested_id
{
continue;
}
if data.len() as u64 > max_attachment_bytes {
return Some(Err(RetainedAttachmentReadError::ResourceLimit));
}
return Some(Ok(RetainedSessionAttachment {
media_type: mime_type.clone(),
data: data.clone(),
}));
}
None
}
/// Read one bounded backward page from the retained active Segment's adopted
/// lineage without creating, migrating, restoring, or rewriting persisted state.
///
@@ -273,6 +456,7 @@ pub fn read_retained_session_history_page(
limits,
&mut scanned_bytes,
&mut scanned_entries,
&mut Vec::new(),
)?;
Some(SessionHistoryLineageBoundary {
lineage_id: lineage_identity(active.session_id, &lineage[1..]),
@@ -284,6 +468,10 @@ pub fn read_retained_session_history_page(
} else {
None
};
let mut correction_positions = cursor
.as_ref()
.map(|c| c.corrections.clone())
.unwrap_or_default();
let mut turns = read_history_turns_backward(
&store,
active.session_id,
@@ -293,11 +481,23 @@ pub fn read_retained_session_history_page(
limits,
&mut scanned_bytes,
&mut scanned_entries,
&mut correction_positions,
)?;
let has_more = turns.len() > page_limit;
if has_more {
turns.truncate(page_limit);
}
// Carry only corrections whose original results are still on older pages.
let returned_ids: HashSet<_> = turns
.iter()
.flat_map(|turn| {
turn.turn
.entries
.iter()
.map(|entry| entry.entry_id.as_str())
})
.collect();
correction_positions.retain(|position| !returned_ids.contains(position.entry_id.as_str()));
let next_cursor = if has_more {
let oldest = turns
.last()
@@ -313,6 +513,7 @@ pub fn read_retained_session_history_page(
before_offset: oldest.position.before_offset,
record_end_offset: oldest.position.record_end_offset,
seed_entry_index: oldest.position.seed_entry_index,
corrections: correction_positions,
})?)
} else {
None
@@ -530,6 +731,7 @@ fn persisted_entry_units(entries: &[LogEntry]) -> usize {
| LogEntry::AnnotatedUserInput { history, .. } => history.len(),
LogEntry::AnnotatedAssistantItem { .. }
| LogEntry::AnnotatedToolResult { .. }
| LogEntry::ToolResultCorrected { .. }
| LogEntry::AnnotatedSystemItem { .. } => 1,
_ => 0,
};
@@ -642,10 +844,56 @@ fn read_history_turns_backward(
limits: RetainedHistoryReadLimits,
scanned_bytes: &mut u64,
scanned_entries: &mut usize,
correction_positions: &mut Vec<ToolCorrectionPosition>,
) -> Result<Vec<PositionedHistoryTurn>, RetainedHistoryReadError> {
let mut turns = Vec::new();
let mut pending_entries = Vec::new();
let mut seen_entries = HashSet::new();
let mut corrections = HashMap::<String, SessionSnapshotEntryData>::new();
// Carry only bounded log positions, never correction text or image bodies,
// across pages. Re-read and validate the referenced records from adopted
// lineage rather than rescanning an arbitrarily large newer suffix.
for position in correction_positions.iter() {
let segment = lineage
.iter()
.find(|s| s.segment_id == position.segment_id)
.ok_or(RetainedHistoryReadError::InvalidCursor)?;
let adopted_end = locate_adopted_segment_end(
store,
session_id,
segment,
limits,
scanned_bytes,
scanned_entries,
)?;
if position.start >= position.end || position.end > adopted_end {
return Err(RetainedHistoryReadError::InvalidCursor);
}
let record = store
.read_log_record_range_read_only_bounded(
session_id,
position.segment_id,
position.start,
position.end,
limits.max_scan_bytes.saturating_sub(*scanned_bytes),
)
.map_err(map_history_cursor_store_error)?;
*scanned_bytes = scanned_bytes
.checked_add(record.end_offset - record.start_offset)
.ok_or(RetainedHistoryReadError::ResourceLimit)?;
*scanned_entries = scanned_entries
.checked_add(persisted_entry_units(std::slice::from_ref(&record.entry)))
.ok_or(RetainedHistoryReadError::ResourceLimit)?;
if *scanned_entries > limits.max_entries {
return Err(RetainedHistoryReadError::ResourceLimit);
}
if !matches!(&record.entry, LogEntry::ToolResultCorrected { entry, .. } if entry.metadata.entry_id.0 == position.entry_id)
|| !capture_tool_correction(&record.entry, &mut corrections)
{
return Err(RetainedHistoryReadError::InvalidCursor);
}
}
let (start_segment_index, start_position, cursor_turn_id) = match start {
Some((position, turn_id)) => {
@@ -706,32 +954,6 @@ fn read_history_turns_backward(
for (segment_index, segment) in lineage.iter().enumerate().skip(start_segment_index) {
let starting_here = segment_index == start_segment_index;
if starting_here
&& start_position
.as_ref()
.and_then(|position| position.seed_entry_index)
.is_some()
{
let seed_index = start_position
.as_ref()
.and_then(|position| position.seed_entry_index)
.expect("checked seed cursor");
if collect_history_entries_backward(
&segment.seed_entries[..seed_index],
segment.segment_id,
0,
None,
true,
turn_limit,
&mut seen_entries,
&mut pending_entries,
&mut turns,
) {
return Ok(turns);
}
continue;
}
let adopted_end = locate_adopted_segment_end(
store,
session_id,
@@ -751,6 +973,32 @@ fn read_history_turns_backward(
if before_offset > adopted_end {
return Err(RetainedHistoryReadError::InvalidCursor);
}
if starting_here
&& start_position
.as_ref()
.and_then(|position| position.seed_entry_index)
.is_some()
{
let seed_index = start_position
.as_ref()
.and_then(|position| position.seed_entry_index)
.expect("checked seed cursor");
if collect_history_entries_backward(
&corrected_history_entries(&segment.seed_entries[..seed_index], &corrections),
segment.segment_id,
0,
None,
true,
turn_limit,
&mut seen_entries,
&mut pending_entries,
&mut turns,
) {
return Ok(turns);
}
continue;
}
let mut reader = store
.open_retained_segment_reader(session_id, segment.segment_id, before_offset)
.map_err(map_history_store_error)?;
@@ -777,7 +1025,25 @@ fn read_history_turns_backward(
}
continue;
}
let entries = project_history_record(session_id, segment.segment_id, &record.entry);
if capture_tool_correction(&record.entry, &mut corrections) {
let LogEntry::ToolResultCorrected { entry, .. } = &record.entry else {
unreachable!()
};
let position = ToolCorrectionPosition {
entry_id: entry.metadata.entry_id.0.clone(),
segment_id: segment.segment_id,
start: record.start_offset,
end: record.end_offset,
};
if !correction_positions.contains(&position) {
correction_positions.push(position);
}
continue;
}
let entries = corrected_history_entries(
&project_history_record(session_id, segment.segment_id, &record.entry),
&corrections,
);
if collect_history_entries_backward(
&entries,
segment.segment_id,
@@ -795,7 +1061,7 @@ fn read_history_turns_backward(
if segment_index + 1 == lineage.len()
&& collect_history_entries_backward(
&segment.seed_entries,
&corrected_history_entries(&segment.seed_entries, &corrections),
segment.segment_id,
0,
None,
@@ -812,6 +1078,38 @@ fn read_history_turns_backward(
Ok(turns)
}
fn capture_tool_correction(
record: &LogEntry,
corrections: &mut HashMap<String, SessionSnapshotEntryData>,
) -> bool {
let LogEntry::ToolResultCorrected { entry, .. } = record else {
return false;
};
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
// We read backwards: the first correction seen is the latest one.
corrections
.entry(entry.metadata.entry_id.0.clone())
.or_insert(data);
}
true
}
fn corrected_history_entries(
entries: &[SessionSnapshotEntry],
corrections: &HashMap<String, SessionSnapshotEntryData>,
) -> Vec<SessionSnapshotEntry> {
entries
.iter()
.cloned()
.map(|mut entry| {
if let Some(data) = corrections.get(&entry.entry_id) {
entry.data = data.clone();
}
entry
})
.collect()
}
#[allow(clippy::too_many_arguments)]
fn collect_history_entries_backward(
entries: &[SessionSnapshotEntry],
@@ -880,9 +1178,13 @@ fn is_user_entry(entry: &SessionSnapshotEntry) -> bool {
fn encode_history_cursor(
cursor: &RetainedHistoryCursor,
) -> Result<String, RetainedHistoryReadError> {
serde_json::to_vec(cursor)
let encoded = serde_json::to_vec(cursor)
.map(|bytes| URL_SAFE_NO_PAD.encode(bytes))
.map_err(|_| RetainedHistoryReadError::CorruptLog)
.map_err(|_| RetainedHistoryReadError::CorruptLog)?;
if encoded.len() > 4096 {
return Err(RetainedHistoryReadError::ResourceLimit);
}
Ok(encoded)
}
fn decode_history_cursor(cursor: &str) -> Result<RetainedHistoryCursor, RetainedHistoryReadError> {
@@ -935,6 +1237,54 @@ fn map_history_store_error(error: StoreError) -> RetainedHistoryReadError {
}
}
fn map_history_attachment_error(error: RetainedHistoryReadError) -> RetainedAttachmentReadError {
match error {
RetainedHistoryReadError::RetentionMissing => RetainedAttachmentReadError::RetentionMissing,
RetainedHistoryReadError::ActivePointerMissing => {
RetainedAttachmentReadError::ActivePointerMissing
}
RetainedHistoryReadError::MigrationRequired => {
RetainedAttachmentReadError::MigrationRequired
}
RetainedHistoryReadError::CorruptLog | RetainedHistoryReadError::InvalidCursor => {
RetainedAttachmentReadError::CorruptLog
}
RetainedHistoryReadError::StorageUnavailable => {
RetainedAttachmentReadError::StorageUnavailable
}
RetainedHistoryReadError::ResourceLimit => RetainedAttachmentReadError::ResourceLimit,
}
}
fn map_attachment_worker_store_error(error: WorkerStoreError) -> RetainedAttachmentReadError {
match error {
WorkerStoreError::Io(error) if error.kind() == std::io::ErrorKind::NotFound => {
RetainedAttachmentReadError::RetentionMissing
}
WorkerStoreError::Serde(_) | WorkerStoreError::InvalidWorkerName(_) => {
RetainedAttachmentReadError::CorruptLog
}
WorkerStoreError::Io(_) => RetainedAttachmentReadError::StorageUnavailable,
}
}
fn map_attachment_store_error(error: StoreError) -> RetainedAttachmentReadError {
match error {
StoreError::Io(error) if error.kind() == std::io::ErrorKind::NotFound => {
RetainedAttachmentReadError::RetentionMissing
}
StoreError::Corrupt { message, .. } if message.contains("requires migration") => {
RetainedAttachmentReadError::MigrationRequired
}
StoreError::ReadLimitExceeded => RetainedAttachmentReadError::ResourceLimit,
StoreError::Io(_) => RetainedAttachmentReadError::StorageUnavailable,
StoreError::Serde(_) | StoreError::Corrupt { .. } | StoreError::NotFound(_) => {
RetainedAttachmentReadError::CorruptLog
}
_ => RetainedAttachmentReadError::CorruptLog,
}
}
fn map_worker_store_error(error: WorkerStoreError) -> RetainedSnapshotReadError {
match error {
WorkerStoreError::Io(error) if error.kind() == std::io::ErrorKind::NotFound => {
@@ -1028,10 +1378,23 @@ pub(crate) fn project_session_snapshot_for_segment(
} => extend_history(&mut entries, history, Some(segments), *ts),
LogEntry::AnnotatedAssistantItem { ts, entry }
| LogEntry::AnnotatedToolResult { ts, entry } => {
if let Some(data) = project_item(&entry.item) {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
entries.push(history_entry(entry, *ts, data));
}
}
LogEntry::ToolResultCorrected { ts, entry } => {
if let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) {
if let Some(existing) = entries
.iter_mut()
.find(|existing| existing.entry_id == entry.metadata.entry_id.0)
{
// The original result keeps its place/time in the conversation.
existing.data = data;
} else {
entries.push(history_entry(entry, *ts, data));
}
}
}
LogEntry::AnnotatedSystemItem { ts, entry, .. } => entries.push(system_entry(
&entry.item,
entry.metadata.entry_id.0.clone(),
@@ -1188,7 +1551,7 @@ fn extend_history(
segments: input_segments.cloned().unwrap_or_default(),
}
} else {
let Some(data) = project_item(&entry.item) else {
let Some(data) = project_item(&entry.metadata.entry_id.0, &entry.item) else {
continue;
};
data
@@ -1291,7 +1654,8 @@ fn legacy_entry_id(
fn provenance(origin: &LoggedSessionHistoryOrigin) -> SessionEntryProvenance {
match origin {
LoggedSessionHistoryOrigin::HumanInput { .. } => SessionEntryProvenance::HumanInput,
LoggedSessionHistoryOrigin::HumanInput { .. }
| LoggedSessionHistoryOrigin::LocalHumanInput => SessionEntryProvenance::HumanInput,
LoggedSessionHistoryOrigin::WorkerInput { .. } => SessionEntryProvenance::WorkerInput,
LoggedSessionHistoryOrigin::FlowInstruction { .. } => {
SessionEntryProvenance::FlowInstruction
@@ -1306,7 +1670,25 @@ fn provenance(origin: &LoggedSessionHistoryOrigin) -> SessionEntryProvenance {
}
}
fn project_item(item: &LoggedItem) -> Option<SessionSnapshotEntryData> {
pub fn session_tool_attachment_id(
entry_id: &str,
attachment_index: usize,
media_type: &str,
data: &[u8],
) -> String {
let mut hash = Sha256::new();
hash.update(b"yoi-session-tool-attachment-v1\0");
hash.update((entry_id.len() as u64).to_be_bytes());
hash.update(entry_id.as_bytes());
hash.update((attachment_index as u64).to_be_bytes());
hash.update((media_type.len() as u64).to_be_bytes());
hash.update(media_type.as_bytes());
hash.update((data.len() as u64).to_be_bytes());
hash.update(data);
URL_SAFE_NO_PAD.encode(hash.finalize())
}
fn project_item(entry_id: &str, item: &LoggedItem) -> Option<SessionSnapshotEntryData> {
match item {
LoggedItem::Message { role, content } => {
let role = match role {
@@ -1355,11 +1737,18 @@ fn project_item(item: &LoggedItem) -> Option<SessionSnapshotEntryData> {
is_error: *is_error,
attachments: attachments
.iter()
.map(|attachment| match attachment {
.enumerate()
.map(|(attachment_index, attachment)| match attachment {
crate::logged_item::LoggedAttachment::Image { mime_type, data } => {
SessionToolAttachment {
attachment_id: session_tool_attachment_id(
entry_id,
attachment_index,
mime_type,
data,
),
media_type: mime_type.clone(),
data_base64: BASE64.encode(data),
byte_len: data.len() as u64,
}
}
})
@@ -1537,6 +1926,129 @@ mod tests {
.collect()
}
#[test]
fn image_correction_keeps_original_position_across_history_pages() {
let session_id = crate::new_session_id();
let segment_id = crate::new_segment_id();
let mut image = assistant_message(1, "image");
image.item = agen::Item::tool_result_item_with_disposition_and_attachments(
"image-call",
"original image",
None,
agen::ToolResultDisposition::Success,
vec![agen::tool::Attachment::Image(
agen::tool::ImageAttachment::new("image/png", vec![1_u8, 2, 3]),
)],
)
.into();
let mut corrected = image.clone();
corrected.item = agen::Item::tool_result_item_with_disposition_and_attachments(
"image-call",
"Resize or crop and retry ViewImage",
None,
agen::ToolResultDisposition::Error,
Vec::new(),
)
.into();
let mut log = vec![segment_start(session_id, Vec::new(), None, None)];
append_turn(&mut log, 1);
log.push(LogEntry::AnnotatedToolResult {
ts: 15,
entry: image.clone(),
});
append_turn(&mut log, 2);
append_turn(&mut log, 3);
log.push(LogEntry::ToolResultCorrected {
ts: 35,
entry: corrected,
});
let (_root, aggregate_root) = persist_history_fixture(
"corrected-image",
session_id,
segment_id,
vec![(segment_id, log.clone())],
);
let first = read_retained_session_history_page(
&aggregate_root,
"corrected-image",
None,
Some(1),
RetainedHistoryReadLimits::default(),
)
.unwrap();
assert_eq!(turn_ids(&first), ["user-3"]);
assert!(
!first.turns[0]
.entries
.iter()
.any(|entry| entry.entry_id == image.metadata.entry_id.0)
);
let second = read_retained_session_history_page(
&aggregate_root,
"corrected-image",
first.next_cursor.as_deref(),
Some(1),
RetainedHistoryReadLimits::default(),
)
.unwrap();
assert_eq!(turn_ids(&second), ["user-2"]);
let third = read_retained_session_history_page(
&aggregate_root,
"corrected-image",
second.next_cursor.as_deref(),
Some(1),
RetainedHistoryReadLimits::default(),
)
.unwrap();
assert_eq!(turn_ids(&third), ["user-1"]);
let entry = third.turns[0]
.entries
.iter()
.find(|entry| entry.entry_id == image.metadata.entry_id.0)
.unwrap();
assert_eq!(entry.timestamp, 15);
assert!(
matches!(&entry.data, SessionSnapshotEntryData::ToolResult { is_error: true, attachments, .. } if attachments.is_empty())
);
assert_eq!(
*entry,
*project_session_snapshot(session_id, &log)
.entries
.iter()
.find(|entry| entry.entry_id == image.metadata.entry_id.0)
.unwrap()
);
let restored = crate::collect_state(&log);
assert_eq!(
restored
.annotated_history
.iter()
.filter(|entry| entry.metadata.entry_id == image.metadata.entry_id)
.count(),
1
);
assert!(
matches!(&restored.annotated_history.iter().find(|entry| entry.metadata.entry_id == image.metadata.entry_id).unwrap().item, LoggedItem::ToolResult { is_error: true, attachments, .. } if attachments.is_empty())
);
// Cursor pointers are only hints: the referenced record is read under
// current lineage authority and must actually be a correction record.
let mut cursor = decode_history_cursor(first.next_cursor.as_deref().unwrap()).unwrap();
cursor.corrections[0].segment_id = crate::new_segment_id();
let forged = encode_history_cursor(&cursor).unwrap();
assert_eq!(
read_retained_session_history_page(
&aggregate_root,
"corrected-image",
Some(&forged),
Some(1),
RetainedHistoryReadLimits::default()
)
.unwrap_err(),
RetainedHistoryReadError::InvalidCursor
);
}
#[test]
fn retained_history_pages_twelve_turns_as_five_five_two_across_compaction() {
let worker_name = "worker-history-12";
@@ -3307,6 +3819,206 @@ mod tests {
));
}
#[test]
fn retained_attachment_resolves_from_an_adopted_compaction_ancestor() {
let worker_name = "worker-ancestor-image";
let session_id = crate::new_session_id();
let source_segment = crate::new_segment_id();
let active_segment = crate::new_segment_id();
let image_entry = LoggedHistoryEntry {
item: LoggedItem::ToolResult {
call_id: "call-ancestor-image".into(),
summary: "ancestor screenshot".into(),
content: None,
attachments: vec![crate::logged_item::LoggedAttachment::Image {
mime_type: "image/png".into(),
data: vec![7, 8, 9],
}],
disposition: agen::tool::ToolResultDisposition::Success,
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId("ancestor-image".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
},
};
let mut source_log = vec![segment_start(session_id, Vec::new(), None, None)];
source_log.push(LogEntry::AnnotatedUserInput {
ts: 2,
segments: vec![Segment::Text {
content: "show image".into(),
}],
history: vec![user_message(1)],
extensions: Vec::new(),
});
source_log.push(LogEntry::AnnotatedToolResult {
ts: 3,
entry: image_entry,
});
source_log.push(LogEntry::TurnEnd {
ts: 4,
turn_count: 1,
});
let active_log = vec![segment_start(
session_id,
Vec::new(),
None,
Some(SegmentOrigin {
segment_id: source_segment,
at_turn_index: 1,
}),
)];
let (_root, aggregate_root) = persist_history_fixture(
worker_name,
session_id,
active_segment,
vec![(source_segment, source_log), (active_segment, active_log)],
);
let page = read_retained_session_history_page(
&aggregate_root,
worker_name,
None,
None,
RetainedHistoryReadLimits::default(),
)
.unwrap();
let attachment = page.turns[0]
.entries
.iter()
.find_map(|entry| match &entry.data {
SessionSnapshotEntryData::ToolResult { attachments, .. } => attachments.first(),
_ => None,
})
.expect("ancestor attachment reference");
let body = read_retained_session_attachment(
&aggregate_root,
worker_name,
&session_id.to_string(),
&attachment.attachment_id,
DEFAULT_RETAINED_HISTORY_MAX_SCAN_BYTES,
10 * 1024 * 1024,
)
.unwrap();
assert_eq!(body.media_type, "image/png");
assert_eq!(body.data, vec![7, 8, 9]);
}
#[test]
fn large_legacy_inline_images_project_as_bounded_refs_and_fetch_from_retained_storage() {
let worker_name = "worker-images";
let session_id = crate::new_session_id();
let segment_id = crate::new_segment_id();
// Approximate the decoded byte sizes of the four W-296 screenshots.
let sizes = [3_475_194_usize, 3_459_051, 5_735_340, 5_703_924];
let attachments = sizes
.iter()
.enumerate()
.map(
|(index, size)| crate::logged_item::LoggedAttachment::Image {
mime_type: "image/png".into(),
data: vec![index as u8 + 1; *size],
},
)
.collect();
let tool_result = LoggedHistoryEntry {
item: LoggedItem::ToolResult {
call_id: "call-images".into(),
summary: "four screenshots".into(),
content: None,
attachments,
disposition: agen::tool::ToolResultDisposition::Success,
is_error: false,
},
metadata: LoggedSessionHistoryMetadata {
entry_id: LoggedSessionHistoryEntryId("tool-images".into()),
origin: LoggedSessionHistoryOrigin::LegacyUnknown,
derivation: None,
},
};
let live = project_current_session_snapshot(&[LogEntry::AnnotatedToolResult {
ts: 2,
entry: tool_result.clone(),
}]);
let log = vec![LogEntry::AnnotatedSegmentStart {
ts: 1,
session_id,
system_prompt: None,
config: RequestConfig::default(),
history: vec![tool_result],
forked_from: None,
compacted_from: None,
}];
let (_root, aggregate_root) =
persist_history_fixture(worker_name, session_id, segment_id, vec![(segment_id, log)]);
let retained = read_retained_session_snapshot(
&aggregate_root,
worker_name,
DEFAULT_RETAINED_SNAPSHOT_MAX_BYTES,
)
.unwrap();
let SessionSnapshotEntryData::ToolResult { attachments, .. } =
&retained.snapshot.entries[0].data
else {
panic!("expected tool result");
};
assert_eq!(attachments.len(), 4);
let SessionSnapshotEntryData::ToolResult {
attachments: live_attachments,
..
} = &live.entries[0].data
else {
panic!("expected live tool result");
};
assert_eq!(live_attachments, attachments);
let snapshot_json = serde_json::to_vec(&retained.snapshot).unwrap();
assert!(snapshot_json.len() < 16 * 1024);
assert!(!String::from_utf8_lossy(&snapshot_json).contains("data_base64"));
for (index, attachment) in attachments.iter().enumerate() {
assert_eq!(attachment.byte_len, sizes[index] as u64);
assert_eq!(attachment.attachment_id.len(), 43);
let body = read_retained_session_attachment(
&aggregate_root,
worker_name,
&session_id.to_string(),
&attachment.attachment_id,
DEFAULT_RETAINED_HISTORY_MAX_SCAN_BYTES,
10 * 1024 * 1024,
)
.unwrap();
assert_eq!(body.media_type, "image/png");
assert_eq!(body.data.len(), sizes[index]);
assert!(body.data.iter().all(|byte| *byte == index as u8 + 1));
}
assert_eq!(
read_retained_session_attachment(
&aggregate_root,
worker_name,
&crate::new_session_id().to_string(),
&attachments[0].attachment_id,
DEFAULT_RETAINED_HISTORY_MAX_SCAN_BYTES,
10 * 1024 * 1024,
)
.unwrap_err(),
RetainedAttachmentReadError::SessionMismatch
);
assert_eq!(
read_retained_session_attachment(
&aggregate_root,
worker_name,
&session_id.to_string(),
"../../metadata.json",
DEFAULT_RETAINED_HISTORY_MAX_SCAN_BYTES,
10 * 1024 * 1024,
)
.unwrap_err(),
RetainedAttachmentReadError::NotFound
);
}
#[test]
fn retained_reads_do_not_create_missing_aggregate() {
let root = tempfile::tempdir().unwrap();
+13
View File
@@ -107,6 +107,9 @@ pub enum LogEntry {
/// Canonical tool output and metadata committed as one journal record.
AnnotatedToolResult { ts: u64, entry: LoggedHistoryEntry },
/// Append-only correction of a provider-rejected image result. Retains the
/// original stable identity; replay replaces rather than appends the item.
ToolResultCorrected { ts: u64, entry: LoggedHistoryEntry },
/// Canonical typed system event and model-visible metadata committed
/// together.
@@ -399,6 +402,16 @@ pub fn collect_state(entries: &[LogEntry]) -> RestoredState {
state.annotated_history.push(entry.clone());
state.history.push(Item::from(entry.item.clone()));
}
LogEntry::ToolResultCorrected { entry, .. } => {
if let Some(index) = state
.annotated_history
.iter()
.position(|existing| existing.metadata.entry_id == entry.metadata.entry_id)
{
state.annotated_history[index] = entry.clone();
state.history[index] = Item::from(entry.item.clone());
}
}
LogEntry::AnnotatedSystemItem {
entry, extensions, ..
} => {
+8 -1
View File
@@ -110,7 +110,14 @@ pub trait Store: Send + Sync {
/// segment ID without its session ID (e.g. legacy `--session <UUID>`).
fn lookup_session_of(&self, segment_id: SegmentId) -> Result<Option<SessionId>, StoreError>;
/// Create a new segment within `session_id`, with initial entries.
/// Create or atomically replace a segment within `session_id` with all entries.
///
/// Readers and crash recovery must observe either the previous complete log
/// or the complete replacement, never a truncated prefix or partial record.
/// Failure before replacement leaves the old log intact; an error after the
/// atomic commit (for example a durability sync failure) may leave the complete
/// replacement. Filesystem implementations must sync replacement bytes before
/// rename and the containing directory afterward before reporting success.
fn create_segment(
&self,
session_id: SessionId,
+63
View File
@@ -123,6 +123,14 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Result of an explicitly selected chat Feature invocation. The typed
/// result is retained for clients; `body` is the exact bounded context made
/// visible to the Worker model.
FeatureInvocationResult {
result: protocol::FeatureInvocationResult,
body: String,
},
/// `@<path>` file reference resolution. `body` is the rendered
/// LLM-context text (`[File: <path>]\n…` for regular files,
/// `[Dir: <path>]\n…` for directory listings, possibly with a
@@ -167,6 +175,15 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// A Host-fetched change to user-managed Subject behavior. Each changed
/// revision is appended rather than rewriting prior prompt/session content.
SubjectBehaviorRefresh {
body: String,
behavior_revision: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Synthetic note inserted after an interrupted turn before the next
/// user input. `body` is the exact LLM-context text explaining that the
/// previous turn was cut short.
@@ -184,6 +201,7 @@ impl SystemItem {
match self {
SystemItem::Notification { body, .. } => body.clone(),
SystemItem::WorkerEvent { body, .. } => body.clone(),
SystemItem::FeatureInvocationResult { body, .. } => body.clone(),
SystemItem::FileAttachment { body, .. } => body.clone(),
SystemItem::SkillActivation { body, .. } => body.clone(),
SystemItem::LegacyKnowledgeIgnored { .. } => String::new(),
@@ -192,6 +210,7 @@ impl SystemItem {
}
SystemItem::TaskReminder { body, .. } => body.clone(),
SystemItem::ResidentSummaryRefresh { body, .. } => body.clone(),
SystemItem::SubjectBehaviorRefresh { body, .. } => body.clone(),
SystemItem::Interrupt { body, .. } => body.clone(),
}
}
@@ -208,12 +227,14 @@ impl SystemItem {
match self {
SystemItem::Notification { .. } => "notification",
SystemItem::WorkerEvent { .. } => "worker_event",
SystemItem::FeatureInvocationResult { .. } => "feature_invocation_result",
SystemItem::FileAttachment { .. } => "file_attachment",
SystemItem::SkillActivation { .. } => "skill_activation",
SystemItem::LegacyKnowledgeIgnored { .. } => "legacy_knowledge_ignored",
SystemItem::LegacyIgnored { .. } => "legacy_ignored",
SystemItem::TaskReminder { .. } => "task_reminder",
SystemItem::ResidentSummaryRefresh { .. } => "resident_summary_refresh",
SystemItem::SubjectBehaviorRefresh { .. } => "subject_behavior_refresh",
SystemItem::Interrupt { .. } => "interrupt",
}
}
@@ -270,6 +291,25 @@ mod tests {
));
}
#[test]
fn subject_behavior_refresh_round_trips_with_revision_and_exact_body() {
let item = SystemItem::SubjectBehaviorRefresh {
body: "current Subject context".to_string(),
behavior_revision: 7,
prompt_provenance: None,
};
let raw = serde_json::to_string(&item).unwrap();
let parsed: SystemItem = serde_json::from_str(&raw).unwrap();
assert_eq!(parsed.history_text(), "current Subject context");
assert!(matches!(
parsed,
SystemItem::SubjectBehaviorRefresh {
behavior_revision: 7,
..
}
));
}
#[test]
fn notification_history_text_returns_stored_body() {
let item = SystemItem::Notification {
@@ -363,6 +403,29 @@ mod tests {
}
}
#[test]
fn invocation_result_retains_typed_outcome_and_exact_context() {
let result = protocol::FeatureInvocationResult {
invocation_id: "stable-invoke".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
status: protocol::FeatureInvocationStatus::OutcomeUnknown,
message: "operation may have committed".into(),
context: None,
};
let body = "Outcome unknown; do not retry automatically";
let item = SystemItem::FeatureInvocationResult {
result: result.clone(),
body: body.into(),
};
let restored: SystemItem =
serde_json::from_str(&serde_json::to_string(&item).unwrap()).unwrap();
assert_eq!(restored.history_text(), body);
assert_eq!(restored.kind_label(), "feature_invocation_result");
assert!(
matches!(restored, SystemItem::FeatureInvocationResult { result: saved, .. } if saved == result)
);
}
#[test]
fn round_trip_worker_event() {
let item = SystemItem::WorkerEvent {
+152 -3
View File
@@ -112,6 +112,47 @@ pub struct WorkerPeer {
pub worker_name: String,
}
/// Durable completion state for the Host-owned Subjektiv Session attribution
/// lifecycle. The Host writes `OutcomeUnknown` before sending the request, so a
/// process interruption or lost response remains explicitly recoverable by the
/// next Worker create/restore lifecycle instead of leaking into ordinary Runs.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "state", rename_all = "snake_case", deny_unknown_fields)]
pub enum SubjektivSessionAttributionState {
/// A lifecycle-created Session sent a create-if-missing request but the
/// Host could not determine whether the Backend committed it.
OutcomeUnknown {
session_id: SessionId,
diagnostic: String,
},
/// A legacy restored Session has no local lifecycle evidence. Recovery may
/// verify an immutable Backend record but must never create one from current
/// ownership.
LegacyUnknown {
session_id: SessionId,
diagnostic: String,
},
Failed {
session_id: SessionId,
diagnostic: String,
},
Confirmed {
session_id: SessionId,
subject_id: String,
},
}
impl SubjektivSessionAttributionState {
pub fn session_id(&self) -> SessionId {
match self {
Self::OutcomeUnknown { session_id, .. }
| Self::LegacyUnknown { session_id, .. }
| Self::Failed { session_id, .. }
| Self::Confirmed { session_id, .. } => *session_id,
}
}
}
/// Persistent metadata for a Worker name.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct WorkerMetadata {
@@ -133,6 +174,10 @@ pub struct WorkerMetadata {
pub peers: Vec<WorkerPeer>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub resolved_manifest_snapshot: Option<serde_json::Value>,
/// Host-owned completion evidence for Session-to-Subject attribution. Its
/// absence is also the legacy representation and never grants visibility.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub subjektiv_session_attribution: Option<SubjektivSessionAttributionState>,
}
impl WorkerMetadata {
@@ -147,6 +192,7 @@ impl WorkerMetadata {
reclaimed_children: Vec::new(),
peers: Vec::new(),
resolved_manifest_snapshot: None,
subjektiv_session_attribution: None,
}
}
@@ -161,6 +207,18 @@ impl WorkerMetadata {
}
}
fn replace_active_session(metadata: &mut WorkerMetadata, active: Option<WorkerActiveSegmentRef>) {
if let Some(session_id) = active.as_ref().map(|active| active.session_id)
&& metadata
.subjektiv_session_attribution
.as_ref()
.is_some_and(|state| state.session_id() != session_id)
{
metadata.subjektiv_session_attribution = None;
}
metadata.active = active;
}
/// Sync persistence backend for Worker metadata.
pub trait WorkerMetadataStore: Send + Sync {
/// Create or replace metadata for its `worker_name` key.
@@ -236,7 +294,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.active = active;
replace_active_session(metadata, active);
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -256,7 +314,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_id: Option<String>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.active = active;
replace_active_session(metadata, active);
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -273,7 +331,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.active = active;
replace_active_session(metadata, active);
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_root) = workspace_root {
metadata.workspace_root = Some(workspace_root);
@@ -281,6 +339,18 @@ pub trait WorkerMetadataStore: Send + Sync {
})
}
/// Set the Host-owned Subjektiv Session attribution lifecycle state while
/// preserving the active Session pointer and every unrelated field.
fn set_subjektiv_session_attribution(
&self,
worker_name: &str,
state: SubjektivSessionAttributionState,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.subjektiv_session_attribution = Some(state);
})
}
/// Set spawned-child registry state while preserving active pointer and manifest snapshot.
fn set_spawned_children(
&self,
@@ -993,6 +1063,85 @@ mod tests {
assert_eq!(restored, metadata);
}
#[test]
fn subjektiv_attribution_state_roundtrips_and_active_updates_preserve_it() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let session_id = crate::new_session_id();
store
.write(&WorkerMetadata::new(
"subject-worker",
Some(WorkerActiveSegmentRef::pending_segment(session_id)),
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
},
)
.unwrap();
store
.set_active(
"subject-worker",
Some(WorkerActiveSegmentRef::active_segment(
session_id,
crate::new_segment_id(),
)),
None,
)
.unwrap();
let restored = store.read_by_name("subject-worker").unwrap().unwrap();
assert!(matches!(
restored.subjektiv_session_attribution,
Some(SubjektivSessionAttributionState::OutcomeUnknown {
session_id: stored_session_id,
ref diagnostic,
}) if stored_session_id == session_id && diagnostic == "request interrupted"
));
}
#[test]
fn new_active_session_clears_prior_session_attribution_state() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let old_session_id = crate::new_session_id();
let new_session_id = crate::new_session_id();
store
.write(&WorkerMetadata::new(
"subject-worker",
Some(WorkerActiveSegmentRef::pending_segment(old_session_id)),
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
},
)
.unwrap();
store
.set_active(
"subject-worker",
Some(WorkerActiveSegmentRef::pending_segment(new_session_id)),
None,
)
.unwrap();
let restored = store.read_by_name("subject-worker").unwrap().unwrap();
assert_eq!(
restored.active,
Some(WorkerActiveSegmentRef::pending_segment(new_session_id))
);
assert!(restored.subjektiv_session_attribution.is_none());
}
#[test]
fn spawned_scope_rule_defaults_resolved_and_roundtrips_logical_policy() {
let legacy: WorkerSpawnedScopeRule = serde_json::from_value(serde_json::json!({
@@ -1166,7 +1166,7 @@ fn atomic_write_json<T: Serialize>(path: &Path, value: &T) -> Result<(), StoreEr
atomic_write_bytes(path, &bytes)
}
fn atomic_write_bytes(path: &Path, bytes: &[u8]) -> Result<(), StoreError> {
pub(crate) fn atomic_write_bytes(path: &Path, bytes: &[u8]) -> Result<(), StoreError> {
let parent = path
.parent()
.ok_or_else(|| std::io::Error::other("Worker Session path has no parent"))?;
@@ -115,6 +115,100 @@ fn create_segment_writes_all_entries() {
assert_eq!(state.session_id, Some(sid));
}
#[test]
fn create_segment_replacement_keeps_open_old_log_and_commits_complete_new_log() {
use std::io::Read;
let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let sid = new_session_id();
let segid = new_segment_id();
let old = vec![
nil_session_start(1, sid),
LogEntry::TurnEnd {
ts: 2,
turn_count: 1,
},
];
store.create_segment(sid, segid, &old).unwrap();
let path = dir
.path()
.join(sid.to_string())
.join(format!("{segid}.jsonl"));
let old_bytes = std::fs::read(&path).unwrap();
let mut old_reader = std::fs::File::open(&path).unwrap();
let new = vec![
nil_session_start(1, sid),
LogEntry::Extension {
ts: 3,
domain: "recovery-checkpoint".into(),
payload: serde_json::json!({"receipt": "retained"}),
},
];
store.create_segment(sid, segid, &new).unwrap();
let mut retained_inode = Vec::new();
old_reader.read_to_end(&mut retained_inode).unwrap();
// An in-place fs::write would truncate the inode held by this reader.
assert_eq!(retained_inode, old_bytes);
let reopened = FsStore::new(dir.path()).unwrap();
assert_eq!(
serde_json::to_value(reopened.read_all(sid, segid).unwrap()).unwrap(),
serde_json::to_value(&new).unwrap()
);
assert_eq!(reopened.read_entry_count(sid, segid).unwrap(), new.len());
assert_eq!(
std::fs::read_dir(path.parent().unwrap()).unwrap().count(),
1,
"successful replacement must not leave temporary files"
);
}
#[test]
fn create_segment_atomic_replacement_readers_only_observe_complete_old_or_new() {
let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let sid = new_session_id();
let segid = new_segment_id();
let old = vec![
nil_session_start(1, sid),
LogEntry::Extension {
ts: 2,
domain: "old-evidence".into(),
payload: serde_json::json!({"result": "old"}),
},
];
let new = vec![
nil_session_start(1, sid),
LogEntry::Extension {
ts: 3,
domain: "new-checkpoint".into(),
payload: serde_json::json!({"result": "new"}),
},
];
store.create_segment(sid, segid, &old).unwrap();
let old_json = serde_json::to_value(&old).unwrap();
let new_json = serde_json::to_value(&new).unwrap();
let barrier = std::sync::Arc::new(std::sync::Barrier::new(2));
let reader_store = store.clone();
let reader_barrier = barrier.clone();
let reader = std::thread::spawn(move || {
reader_barrier.wait();
for _ in 0..1000 {
let observed =
serde_json::to_value(reader_store.read_all(sid, segid).unwrap()).unwrap();
assert!(
observed == old_json || observed == new_json,
"replacement must never expose a partial log: {observed}"
);
}
});
barrier.wait();
for _ in 0..50 {
store.create_segment(sid, segid, &new).unwrap();
store.create_segment(sid, segid, &old).unwrap();
}
reader.join().unwrap();
}
#[test]
fn list_sessions_and_segments() {
let dir = tempfile::tempdir().unwrap();
@@ -606,6 +606,70 @@ fn rewound_fork_preserves_uploaded_file_segments_in_snapshot() {
)));
}
#[test]
fn feature_invocations_survive_replay_snapshot_and_rewind_without_reparsing() {
let (_dir, store) = make_store();
let config = RequestConfig::default();
let (sid, segid) = session_store::create_segment(
&store,
SegmentStartState {
system_prompt: Some("System prompt"),
config: &config,
history: Vec::new(),
user_segments: Vec::new(),
},
)
.unwrap();
let segments = vec![
Segment::Text {
content: "before ".into(),
},
Segment::FeatureInvoke {
invocation: protocol::FeatureInvocation {
invocation_id: "stable-request-1".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
name: "prepare".into(),
arguments: vec![protocol::InvocationArgumentValue {
name: "path".into(),
value: protocol::InvocationValue::String("資料/a b".into()),
}],
},
},
Segment::Text {
content: " after /prepare(unselected)".into(),
},
];
session_store::save_user_input(
&store,
sid,
segid,
segments.clone(),
annotated(&[Item::user_message(Segment::flatten_to_text(&segments))]),
)
.unwrap();
session_store::save_turn_end(&store, sid, segid, 1).unwrap();
let entries = store.read_all(sid, segid).unwrap();
assert_eq!(
collect_state(&entries).user_segments,
vec![segments.clone()]
);
let fork_id = session_store::fork_at(&store, sid, segid, 1).unwrap();
let fork_entries = store.read_all(sid, fork_id).unwrap();
assert_eq!(
collect_state(&fork_entries).user_segments,
vec![segments.clone()]
);
let snapshot = session_store::public_snapshot::project_session_snapshot(sid, &fork_entries);
assert!(snapshot.entries.iter().any(|entry| matches!(
&entry.data, SessionSnapshotEntryData::UserInput { segments: restored } if restored == &segments
)));
// Original history remains append-only and retains the same exact typed intent.
assert_eq!(
serde_json::to_value(store.read_all(sid, segid).unwrap()).unwrap(),
serde_json::to_value(entries).unwrap()
);
}
#[tokio::test]
async fn session_config_changed_logged() {
let (_dir, store) = make_store();
+8
View File
@@ -9,7 +9,15 @@ license.workspace = true
agen.workspace = true
client.workspace = true
fs4.workspace = true
feature-storage.workspace = true
subjektiv.workspace = true
memory.workspace = true
server-api.workspace = true
libc.workspace = true
async-trait.workspace = true
manifest.workspace = true
job.workspace = true
rusqlite.workspace = true
protocol.workspace = true
serde.workspace = true
serde_json.workspace = true
+253 -1
View File
@@ -43,12 +43,16 @@ pub struct StandaloneHost {
worker_store: FsWorkerStore,
record: StandaloneWorkerRecord,
lease: Option<StandaloneWorkerLease>,
jobs: crate::jobs::StandaloneJobs,
subject: Option<crate::subjektiv::SubjectConnection>,
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum StandaloneStartupError {
#[error("the standalone state store could not be opened or validated")]
StateStore,
#[error("local Subject connection rejected: {0}")]
Subject(String),
#[error("the standalone Worker is already active")]
WorkerActive,
#[error("the standalone Worker lease cannot be observed safely; recovery is rejected")]
@@ -77,6 +81,8 @@ pub enum StandaloneStartupError {
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
pub enum StandaloneShutdownError {
#[error("standalone Job execution cleanup or terminal persistence could not be confirmed")]
JobCleanup,
#[error("the standalone Worker did not stop before the shutdown deadline")]
DeadlineExceeded,
#[error("the standalone Worker shutdown confirmation was lost")]
@@ -104,13 +110,66 @@ impl StandaloneHost {
launch: ResolvedStandaloneLaunch,
model_client: Option<Box<dyn LlmClient>>,
) -> Result<Self, StandaloneStartupError> {
if launch.subject_id.is_some()
&& (!launch.profile.manifest.feature.subjektiv.profile.enabled
|| launch
.profile
.manifest
.feature
.subjektiv
.profile
.consolidation_tools
|| launch
.profile
.manifest
.feature
.subjektiv
.workspace_settings
.is_some())
{
return Err(StandaloneStartupError::Subject(
"incompatible local Subject policy".into(),
));
}
let store =
StandaloneWorkerStore::open(&launch.state_dir).map_err(classify_store_startup_error)?;
let allocation = store
.allocate(&launch.cwd, StaleLeasePolicy::Reject)
.map_err(classify_store_startup_error)?;
let worker_id = allocation.worker_id();
let mut subject = match launch
.subject_id
.as_deref()
.map(|id| {
crate::subjektiv::StandaloneSubjects::open_existing(&launch.state_dir)?
.connect(id, worker_id)
})
.transpose()
{
Ok(subject) => subject,
Err(error) => {
let _ = store.abandon_allocation(allocation);
return Err(StandaloneStartupError::Subject(error.to_string()));
}
};
let jobs_path = subject
.as_ref()
.map(|s| s.jobs_path.clone())
.unwrap_or_else(|| store.jobs_path(worker_id));
let jobs = match crate::jobs::StandaloneJobs::open(&jobs_path, launch.cwd.clone()) {
Ok(jobs) => jobs,
Err(_) => {
let _ = store.abandon_allocation(allocation);
return Err(StandaloneStartupError::StateStore);
}
};
if let Some(client) = &model_client {
jobs.bind_model_client(client.clone_boxed());
}
if let Some(subject) = &subject {
subject.host.bind_jobs(jobs.clone())?;
}
// WorkerId is the stable identity. The current Worker store remains
// name-keyed, so keep its derived storage key separate from the
// user-facing profile name.
@@ -143,9 +202,15 @@ impl StandaloneHost {
},
WorkerControllerTransport::InProcess,
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
}
if let Some(model_client) = model_client {
bootstrap = bootstrap.with_model_client(model_client);
}
if let Some(subject) = &mut subject {
subject.arm();
}
let started = match bootstrap.start().await {
Ok(started) => started,
Err(error) => {
@@ -161,12 +226,13 @@ impl StandaloneHost {
return Err(error);
}
};
let record = match store.commit_created(
let record = match store.commit_created_connected(
&allocation,
manifest,
storage_key,
active.session_id,
active.segment_id,
subject.as_ref().map(|s| s.binding.clone()),
) {
Ok(record) => record,
Err(_) => {
@@ -181,6 +247,8 @@ impl StandaloneHost {
worker_store,
record,
allocation.into_lease(),
jobs,
subject,
))
}
@@ -216,6 +284,50 @@ impl StandaloneHost {
let lease = store
.acquire_lease(worker_id, StaleLeasePolicy::Recover)
.map_err(classify_store_startup_error)?;
let mut subject = record
.subject
.as_ref()
.map(|binding| {
if !record.manifest.feature.subjektiv.profile.enabled
|| record
.manifest
.feature
.subjektiv
.profile
.consolidation_tools
|| record
.manifest
.feature
.subjektiv
.workspace_settings
.is_some()
{
return Err(crate::subjektiv::SubjectError::InvalidScope(
"persisted Subject policy changed".into(),
));
}
let catalog = crate::subjektiv::StandaloneSubjects::open_existing(store.root())?;
if catalog.scope_id() != binding.scope_id {
return Err(crate::subjektiv::SubjectError::InvalidScope(
"persisted local storage scope changed".into(),
));
}
catalog.connect(&binding.subject_id, worker_id)
})
.transpose()
.map_err(|e| StandaloneStartupError::Subject(e.to_string()))?;
let jobs_path = subject
.as_ref()
.map(|s| s.jobs_path.clone())
.unwrap_or_else(|| store.jobs_path(worker_id));
let jobs = crate::jobs::StandaloneJobs::open(&jobs_path, record.cwd.canonical_path.clone())
.map_err(|_| StandaloneStartupError::StateStore)?;
if let Some(client) = &model_client {
jobs.bind_model_client(client.clone_boxed());
}
if let Some(subject) = &subject {
subject.host.bind_jobs(jobs.clone())?;
}
let (backing_store, worker_store) = backing_store(&store, worker_id)?;
let storage_key = record.storage_key.clone();
let mut manifest = record.manifest.clone();
@@ -240,6 +352,9 @@ impl StandaloneHost {
},
WorkerControllerTransport::InProcess,
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
}
if let Some(model_client) = model_client {
bootstrap = bootstrap.with_model_client(model_client);
}
@@ -247,6 +362,9 @@ impl StandaloneHost {
.prepare_restored(&storage_key)
.await
.map_err(classify_startup_error)?;
if let Some(subject) = &mut subject {
subject.arm();
}
let started = prepared.start().await.map_err(classify_startup_error)?;
let active = match active_pointer(&worker_store, &storage_key) {
Ok(active) => active,
@@ -270,6 +388,8 @@ impl StandaloneHost {
worker_store,
record,
lease,
jobs,
subject,
))
}
@@ -279,7 +399,12 @@ impl StandaloneHost {
worker_store: FsWorkerStore,
record: StandaloneWorkerRecord,
lease: StandaloneWorkerLease,
jobs: crate::jobs::StandaloneJobs,
subject: Option<crate::subjektiv::SubjectConnection>,
) -> Self {
if let Some(connection) = &subject {
connection.host.recover_jobs();
}
Self {
handle: started.handle,
shutdown: Some(started.shutdown),
@@ -288,9 +413,17 @@ impl StandaloneHost {
worker_store,
record,
lease: Some(lease),
jobs,
subject,
}
}
/// Explicit Host execution capability for local Features, separate from the
/// interactive Worker protocol and its model-created SubWorkers.
pub fn jobs(&self) -> crate::jobs::StandaloneJobs {
self.jobs.clone()
}
#[must_use]
pub fn worker_id(&self) -> WorkerId {
self.record.worker_id
@@ -318,6 +451,7 @@ impl StandaloneHost {
}
pub async fn shutdown(mut self) -> Result<(), StandaloneShutdownError> {
let jobs_result = self.jobs.shutdown().await;
let command = protocol::WorkerCommandEnvelope::new(u64::MAX);
let _ = self.handle.send(Method::Shutdown { command }).await;
let Some(shutdown) = self.shutdown.take() else {
@@ -335,6 +469,10 @@ impl StandaloneHost {
return Err(StandaloneShutdownError::DeadlineExceeded);
}
}
if jobs_result.is_err() {
self.retain_lease();
return Err(StandaloneShutdownError::JobCleanup);
}
let active = match active_pointer(&self.worker_store, &self.record.storage_key) {
Ok(active) => active,
Err(_) => {
@@ -355,6 +493,12 @@ impl StandaloneHost {
self.retain_lease();
return Err(StandaloneShutdownError::StateStore);
}
if let Some(subject) = &mut self.subject {
if subject.close().is_err() {
self.retain_lease();
return Err(StandaloneShutdownError::StateStore);
}
}
if let Some(lease) = self.lease.take() {
lease
.release()
@@ -364,12 +508,70 @@ impl StandaloneHost {
}
fn retain_lease(&mut self) {
if let Some(subject) = &mut self.subject {
subject.retain();
}
if let Some(lease) = self.lease.take() {
lease.retain();
}
}
}
impl Drop for StandaloneHost {
fn drop(&mut self) {
self.jobs.close();
// A dropped Host cannot synchronously confirm either controller or Job
// cleanup. Keep its lease until explicit shutdown or process recovery,
// preventing another Host from reopening a still-live execution store.
self.retain_lease();
}
}
// The in-process channel is owned by the local operator's Host, not an
// arbitrary socket or Backend account. decode_method discards wire provenance;
// only this boundary can stamp accepted local human input.
fn authorize_local_input(method: protocol::Method) -> protocol::Method {
use protocol::{AuthenticatedInputSource, Method};
match method {
Method::Submit {
submission_request_id,
input,
}
| Method::SubmitTracked {
submission_request_id,
input,
..
} => Method::SubmitTracked {
submission_request_id,
input,
source: AuthenticatedInputSource::LocalOperator,
},
Method::SubmitIfIdle {
submission_request_id,
input,
..
} => Method::SubmitIfIdle {
submission_request_id,
input,
source: AuthenticatedInputSource::LocalOperator,
},
Method::Notify {
notification_request_id,
message,
}
| Method::NotifyTracked {
notification_request_id,
message,
..
} => Method::NotifyTracked {
notification_request_id,
message,
source: AuthenticatedInputSource::LocalOperator,
},
method => method,
}
}
async fn run_protocol_session(
handle: worker::WorkerHandle,
streams: WorkerProtocolSessionStreams,
@@ -395,6 +597,7 @@ async fn run_protocol_session(
let Ok(method) = decode_method(&message) else {
return;
};
let method = authorize_local_input(method);
if let Some(event) = dispatch_worker_protocol_method(&handle, method).await
&& !send_protocol_event(&peer, event).await
{
@@ -535,6 +738,9 @@ fn classify_startup_error(error: WorkerBootstrapError) -> StandaloneStartupError
WorkerBootstrapError::Worker(WorkerError::Provider(_)) => {
StandaloneStartupError::ModelProvider
}
WorkerBootstrapError::Worker(WorkerError::SubjektivSessionAttribution {
message, ..
}) => StandaloneStartupError::Subject(message),
WorkerBootstrapError::Worker(_) => StandaloneStartupError::WorkerConfiguration,
WorkerBootstrapError::Controller { source, .. }
if source.kind() == std::io::ErrorKind::Other =>
@@ -544,3 +750,49 @@ fn classify_startup_error(error: WorkerBootstrapError) -> StandaloneStartupError
WorkerBootstrapError::Controller { .. } => StandaloneStartupError::Controller,
}
}
#[cfg(test)]
mod local_input_tests {
use super::*;
use protocol::{AuthenticatedInputSource, Method};
#[test]
fn local_transport_stamps_operator_without_backend_account() {
let source = AuthenticatedInputSource::Account {
account_id: "forged-account".into(),
};
for method in [
Method::submit_text("plain", "hello"),
Method::SubmitTracked {
submission_request_id: "tracked".into(),
input: vec![],
source: source.clone(),
},
Method::SubmitIfIdle {
submission_request_id: "idle".into(),
input: vec![],
source: source.clone(),
},
Method::Notify {
notification_request_id: "plain-notify".into(),
message: "hello".into(),
},
Method::NotifyTracked {
notification_request_id: "tracked-notify".into(),
message: "hello".into(),
source,
},
] {
let decoded: Method =
serde_json::from_str(&serde_json::to_string(&method).unwrap()).unwrap();
match authorize_local_input(decoded) {
Method::SubmitTracked { source, .. }
| Method::SubmitIfIdle { source, .. }
| Method::NotifyTracked { source, .. } => {
assert_eq!(source, AuthenticatedInputSource::LocalOperator)
}
other => panic!("unexpected local input {other:?}"),
}
}
}
}
File diff suppressed because it is too large Load Diff
+508
View File
@@ -0,0 +1,508 @@
//! Host-provided local Jobs. No Runtime registry, Workspace client or dialog-session state.
#[cfg(test)]
mod tests;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use agen::llm_client::LlmClient;
use job::{JobRequest, JobResultSubmission};
use tokio::sync::watch;
use crate::job_store::JobStore;
pub use crate::job_store::{JobAttempt, JobAttemptState, JobSnapshot, JobState, JobStoreError};
#[derive(Debug, thiserror::Error)]
pub enum JobServiceError {
#[error(transparent)]
Store(#[from] JobStoreError),
#[error("standalone Job execution requirements are unavailable: {0}")]
Configuration(String),
#[error("the standalone Job Host is shutting down")]
Closed,
#[error("standalone Job execution capacity is exhausted")]
Capacity,
#[error("standalone Job cleanup failed: {0}")]
Cleanup(String),
}
pub(crate) trait JobAttemptFence: Send + Sync {
fn ensure_live(&self) -> Result<(), String>;
}
pub(crate) trait JobDomainProvider: Send + Sync {
fn grant(
&self,
snapshot: &JobSnapshot,
fence: Arc<dyn JobAttemptFence>,
) -> Result<worker::job::JobFeatureGrant, String>;
}
struct Execution {
cancel: watch::Sender<bool>,
complete: watch::Receiver<Option<Result<(), String>>>,
attempt_id: String,
resource_key: Option<String>,
max_concurrent: u16,
task: tokio::task::JoinHandle<Result<(), JobServiceError>>,
}
struct Executions {
closed: bool,
jobs: HashMap<String, Execution>,
}
/// Cloneable Feature-facing access to a Host-owned service. Cloning this handle
/// neither extends the Host's authority nor permits opening/recovering its store.
#[derive(Clone)]
pub struct StandaloneJobs {
store: Arc<Mutex<Option<JobStore>>>,
executions: Arc<Mutex<Executions>>,
cwd: PathBuf,
capacity: u16,
domain: Arc<Mutex<Option<Arc<dyn JobDomainProvider>>>>,
model_client: Arc<Mutex<Option<Box<dyn LlmClient>>>>,
}
impl StandaloneJobs {
pub(crate) fn open(path: &Path, cwd: PathBuf) -> Result<Self, JobServiceError> {
Ok(Self {
store: Arc::new(Mutex::new(Some(JobStore::open(path)?))),
executions: Arc::new(Mutex::new(Executions {
closed: false,
jobs: HashMap::new(),
})),
cwd,
capacity: job::DEFAULT_MAX_CONCURRENT_JOBS,
domain: Arc::default(),
model_client: Arc::default(),
})
}
pub(crate) fn bind_model_client(&self, client: Box<dyn LlmClient>) {
*self
.model_client
.lock()
.expect("Job model transport poisoned") = Some(client);
}
pub(crate) fn bind_domain(
&self,
provider: Arc<dyn JobDomainProvider>,
) -> Result<(), JobServiceError> {
let mut domain = self.domain.lock().expect("Job domain poisoned");
if domain.is_some() {
return Err(JobServiceError::Configuration(
"Job domain already bound".into(),
));
}
*domain = Some(provider);
Ok(())
}
pub(crate) fn request_granted(
&self,
request: JobRequest,
grant: serde_json::Value,
) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
self.with_store(|store| store.reserve_granted(request, Some(grant)))
}
fn with_store<T>(
&self,
operation: impl FnOnce(&mut JobStore) -> Result<T, JobStoreError>,
) -> Result<T, JobServiceError> {
let mut slot = self.store.lock().expect("Job store poisoned");
Ok(operation(slot.as_mut().ok_or(JobServiceError::Closed)?)?)
}
/// Persist intent without starting a model. Exact resend returns the same
/// current attempt, including after acknowledgement or cancellation.
pub fn request(&self, request: JobRequest) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
Ok(self.with_store(|store| store.reserve(request))?)
}
pub fn get(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
Ok(self.with_store(|store| store.get(job_id))?)
}
/// Pending attempts survive shutdown. The consumer chooses when to resume
/// them; opening a Host never implicitly replays unknown side effects.
pub fn pending(&self) -> Result<Vec<JobSnapshot>, JobServiceError> {
Ok(self.with_store(|store| store.pending())?)
}
pub fn unacknowledged_results(&self) -> Result<Vec<JobSnapshot>, JobServiceError> {
self.with_store(|store| store.unacknowledged())
}
/// Explicit new attempt, bounded by immutable intent limits. Unknown
/// outcomes cannot be retried without domain reconciliation/new intent.
pub fn retry(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
if executions
.jobs
.get(job_id)
.is_some_and(|run| !run.task.is_finished())
{
return Err(JobServiceError::Capacity);
}
Ok(self.with_store(|store| store.retry(job_id))?)
}
/// Successful model processing and consumer postprocessing have independent
/// lifetimes. A failed delivery simply leaves this acknowledgement pending.
pub fn acknowledge(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
Ok(self.with_store(|store| store.ack(job_id))?)
}
pub fn cancel(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
let snapshot = self.with_store(|store| store.cancel(job_id))?;
if let Some(execution) = executions.jobs.get(job_id) {
execution.cancel.send_replace(true);
}
Ok(snapshot)
}
pub async fn start(&self, job_id: &str) -> Result<(), JobServiceError> {
self.start_with_optional_model_client(job_id, None).await
}
/// Injectable model transport for embedders and scripted execution tests.
/// The requested registry Profile still resolves/validates normally; the
/// transport grants no filesystem, Feature or result-submission authority.
pub async fn start_with_model_client<C>(
&self,
job_id: &str,
client: C,
) -> Result<(), JobServiceError>
where
C: LlmClient + 'static,
{
self.start_with_optional_model_client(job_id, Some(Box::new(client)))
.await
}
async fn start_with_optional_model_client(
&self,
job_id: &str,
client: Option<Box<dyn LlmClient>>,
) -> Result<(), JobServiceError> {
let client = client.or_else(|| {
self.model_client
.lock()
.expect("Job model transport poisoned")
.as_ref()
.map(|c| c.clone_boxed())
});
let snapshot = self.get(job_id)?;
let timeout = Duration::from_secs(u64::from(snapshot.request.limits.timeout_seconds));
let deadline = tokio::time::Instant::now() + timeout;
let fence: Arc<dyn JobAttemptFence> = Arc::new(BoundAttemptFence {
store: self.store.clone(),
job_id: job_id.into(),
attempt_id: snapshot.attempt.attempt_id.clone(),
input_revision: snapshot.request.input_revision.clone(),
deadline,
});
let grant = match (
&snapshot.domain_grant,
self.domain.lock().expect("Job domain poisoned").as_ref(),
) {
(Some(_), Some(provider)) => provider
.grant(&snapshot, fence)
.map_err(JobServiceError::Configuration)?,
(Some(_), None) => {
return Err(JobServiceError::Configuration(
"persisted Job requires an unavailable explicit domain capability".into(),
));
}
(None, _) => worker::job::JobFeatureGrant::default(),
};
let prepared =
worker::job::prepare_job_with_grant(&snapshot.request, &self.cwd, client, grant)
.map_err(|error| JobServiceError::Configuration(error.to_string()))?;
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
executions.jobs.retain(|_, run| {
!(run.task.is_finished() && matches!(run.complete.borrow().as_ref(), Some(Ok(()))))
});
// Count live resources, not only result states: acceptance can precede
// the Internal Worker's cleanup and must not free a concurrency slot.
let live = || {
executions
.jobs
.values()
.filter(|run| !run.task.is_finished())
};
let count = live().count();
let active_limit = live()
.map(|run| run.max_concurrent)
.min()
.unwrap_or(self.capacity);
if count >= usize::from(self.capacity.min(active_limit))
|| count >= usize::from(snapshot.request.limits.max_concurrent_jobs)
|| executions
.jobs
.get(job_id)
.is_some_and(|run| !run.task.is_finished())
{
return Err(JobServiceError::Capacity);
}
let resource_key = snapshot.request.serialization_key.clone();
if let Some(key) = &resource_key {
if executions
.jobs
.values()
.any(|run| !run.task.is_finished() && run.resource_key.as_ref() == Some(key))
{
return Err(JobServiceError::Capacity);
}
}
// Retain errors from finished tasks until a caller observes them rather
// than silently swallowing store/cleanup failures.
if let Some(previous) = executions.jobs.get(job_id) {
if !previous.task.is_finished() {
return Err(JobServiceError::Capacity);
}
}
let snapshot = self.with_store(|store| {
store.dispatch(job_id, &snapshot.attempt.attempt_id, self.capacity)
})?;
let store = self.store.clone();
let id = job_id.to_string();
let attempt_id = snapshot.attempt.attempt_id.clone();
let sink = Arc::new(BoundResultSink {
store: store.clone(),
job_id: id.clone(),
attempt_id: attempt_id.clone(),
input_revision: snapshot.request.input_revision.clone(),
deadline,
});
let (cancel, mut cancellation) = watch::channel(false);
let (complete_tx, complete) = watch::channel(None);
let execution_attempt = attempt_id.clone();
let task = tokio::spawn(async move {
let outcome: Result<(), JobServiceError> = async {
let (sender_tx, mut sender_rx) = tokio::sync::oneshot::channel();
let run = prepared.run(attempt_id.clone(), sink, move |sender| { let _ = sender_tx.send(sender); });
tokio::pin!(run);
let reason = tokio::select! {
result = &mut run => {
let (state, detail) = match result {
Ok(_) => (JobAttemptState::Failed, "Worker finished without an accepted structured result".to_string()),
Err(error @ worker::job::JobExecutionError::ResultUnconfirmed(_)) => (JobAttemptState::Unknown, error.to_string()),
Err(error) => (JobAttemptState::Failed, error.to_string()),
};
finish_if_pending(&store, &id, &attempt_id, state, &detail)?;
return Ok(());
}
_ = tokio::time::sleep(timeout) => "timeout",
_ = cancellation.wait_for(|cancelled| *cancelled) => "Host cancellation/shutdown",
};
// Fence timeout/shutdown before cancelling so a late tool call can
// never turn an interrupted unaccepted execution into success.
let fence_result = finish_if_pending(&store, &id, &attempt_id, JobAttemptState::Unknown, reason);
// Cancellation is routed through Engine; keep the future alive to
// allow normal Feature cleanup. Hard stop is a fenced unknown
// outcome, never an automatic retry. Result acceptance wins races.
if let Ok(sender) = sender_rx.try_recv() {
let _ = sender.try_send(());
} else if let Ok(Ok(sender)) = tokio::time::timeout(Duration::from_secs(1), &mut sender_rx).await {
let _ = sender.try_send(());
}
let cleanup = tokio::time::timeout(Duration::from_secs(2), &mut run).await;
fence_result?;
if cleanup.is_err() { return Err(JobServiceError::Cleanup("Internal Worker cancellation deadline exceeded; outcome fenced unknown".into())); }
Ok(())
}.await;
complete_tx.send_replace(Some(
outcome.as_ref().map(|_| ()).map_err(ToString::to_string),
));
outcome
});
executions.jobs.insert(
job_id.to_string(),
Execution {
cancel,
complete,
task,
attempt_id: execution_attempt,
resource_key,
max_concurrent: snapshot.request.limits.max_concurrent_jobs,
},
);
Ok(())
}
/// Observe execution/cleanup, not success: read the durable result separately.
pub async fn wait(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
let execution = self
.executions
.lock()
.expect("Job executions poisoned")
.jobs
.get(job_id)
.map(|run| (run.attempt_id.clone(), run.complete.clone()));
if let Some((attempt, mut completion)) = execution {
let result = completion
.wait_for(Option::is_some)
.await
.map_err(|error| JobServiceError::Cleanup(error.to_string()))?
.clone()
.expect("completion");
result.map_err(JobServiceError::Cleanup)?;
let task = {
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions
.jobs
.get(job_id)
.is_some_and(|run| run.attempt_id == attempt)
{
executions.jobs.remove(job_id)
} else {
None
}
};
if let Some(execution) = task {
execution
.task
.await
.map_err(|error| JobServiceError::Cleanup(error.to_string()))??;
}
}
self.get(job_id)
}
pub(crate) fn close(&self) {
let mut executions = self.executions.lock().expect("Job executions poisoned");
executions.closed = true;
for execution in executions.jobs.values() {
execution.cancel.send_replace(true);
}
}
pub(crate) async fn shutdown(&self) -> Result<(), JobServiceError> {
self.close();
let tasks = {
let mut executions = self.executions.lock().expect("Job executions poisoned");
std::mem::take(&mut executions.jobs)
};
let mut error = None;
for execution in tasks.into_values() {
match execution.task.await {
Ok(Ok(())) => {}
Ok(Err(failure)) => {
error = Some(failure);
}
Err(failure) => {
error = Some(JobServiceError::Cleanup(failure.to_string()));
}
}
}
// Close SQLite before the Host releases its lease. Cloned handles may
// outlive the Host but cannot mutate or retain the live connection.
self.store.lock().expect("Job store poisoned").take();
self.domain.lock().expect("Job domain poisoned").take();
self.model_client
.lock()
.expect("Job model transport poisoned")
.take();
error.map_or(Ok(()), Err)
}
}
fn finish_if_pending(
store: &Mutex<Option<JobStore>>,
id: &str,
attempt: &str,
state: JobAttemptState,
detail: &str,
) -> Result<(), JobServiceError> {
let mut slot = store.lock().expect("Job store poisoned");
let store = slot.as_mut().ok_or(JobServiceError::Closed)?;
let snapshot = store.get(id)?;
if snapshot.state == JobState::Pending {
store.finish(id, attempt, state, Some(detail))?;
}
Ok(())
}
struct BoundResultSink {
store: Arc<Mutex<Option<JobStore>>>,
job_id: String,
attempt_id: String,
input_revision: String,
deadline: tokio::time::Instant,
}
impl worker::job::JobResultSink for BoundResultSink {
fn submit(&self, submission: JobResultSubmission) -> Result<(), String> {
if submission.job_id != self.job_id
|| submission.attempt_id != self.attempt_id
|| submission.input_revision != self.input_revision
{
return Err("Job result capability binding mismatch".into());
}
let mut slot = self
.store
.lock()
.map_err(|_| "Job store poisoned".to_string())?;
let store = slot.as_mut().ok_or_else(|| "Job Host closed".to_string())?;
let snapshot = store.get(&self.job_id).map_err(|error| error.to_string())?;
if tokio::time::Instant::now() >= self.deadline && snapshot.state != JobState::Completed {
return Err("Job result deadline expired".into());
}
store
.accept(&submission)
.map(|_| ())
.map_err(|error| error.to_string())
}
}
struct BoundAttemptFence {
store: Arc<Mutex<Option<JobStore>>>,
job_id: String,
attempt_id: String,
input_revision: String,
deadline: tokio::time::Instant,
}
impl JobAttemptFence for BoundAttemptFence {
fn ensure_live(&self) -> Result<(), String> {
if tokio::time::Instant::now() >= self.deadline {
return Err("Job domain deadline expired".into());
}
let slot = self.store.lock().map_err(|_| "Job store poisoned")?;
let snapshot = slot
.as_ref()
.ok_or("Job Host closed")?
.get(&self.job_id)
.map_err(|e| e.to_string())?;
if snapshot.state != JobState::Pending
|| snapshot.attempt.state != JobAttemptState::Dispatched
|| snapshot.attempt.attempt_id != self.attempt_id
|| snapshot.request.input_revision != self.input_revision
{
return Err("Job domain grant is not bound to the current live attempt".into());
}
Ok(())
}
}
+471
View File
@@ -0,0 +1,471 @@
use super::*;
use agen::llm_client::{ClientError, Request, event::Event};
use async_trait::async_trait;
use futures::Stream;
use serde_json::{Value, json};
use std::collections::VecDeque;
use std::pin::Pin;
use worker::job::JobResultSink;
fn request(id: &str) -> JobRequest {
JobRequest {
job_id: id.into(),
purpose: "test".into(),
input_revision: "r1".into(),
input_ref: "fixture:input".into(),
input: serde_json::json!({"data":42}),
instruction: "Return a structured result".into(),
profile: "builtin:job".into(),
serialization_key: None,
limits: job::JobLimits::default(),
}
}
fn sink(jobs: &StandaloneJobs, id: &str, attempt: &str) -> BoundResultSink {
BoundResultSink {
store: jobs.store.clone(),
job_id: id.into(),
attempt_id: attempt.into(),
input_revision: "r1".into(),
deadline: tokio::time::Instant::now() + Duration::from_secs(10),
}
}
fn result(id: &str, attempt: &str) -> JobResultSubmission {
JobResultSubmission {
job_id: id.into(),
attempt_id: attempt.into(),
input_revision: "r1".into(),
result: serde_json::json!({"valid":true}),
}
}
#[derive(Clone, Default)]
struct DomainTestClient {
scripts: Arc<Mutex<VecDeque<Vec<Event>>>>,
requests: Arc<Mutex<Vec<Request>>>,
entered: Arc<tokio::sync::Notify>,
stall: bool,
failure_release: Option<Arc<tokio::sync::Notify>>,
}
impl DomainTestClient {
fn success() -> Self {
Self {
scripts: Arc::new(Mutex::new(VecDeque::from([
vec![
Event::tool_use_start(0, "result-1", "SubmitJobResult"),
Event::tool_input_delta(0, r#"{"result":{"valid":true}}"#),
Event::tool_use_stop(0),
],
vec![
Event::text_block_start(0),
Event::text_delta(0, "done"),
Event::text_block_stop(0, None),
],
]))),
..Default::default()
}
}
}
#[async_trait]
impl LlmClient for DomainTestClient {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(
&self,
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>, ClientError> {
self.requests.lock().unwrap().push(request);
self.entered.notify_one();
if let Some(release) = &self.failure_release {
release.notified().await;
return Err(ClientError::Config(
"known scripted provider failure".into(),
));
}
if self.stall {
return Ok(Box::pin(futures::stream::pending()));
}
let events = self
.scripts
.lock()
.unwrap()
.pop_front()
.expect("unexpected model turn");
Ok(Box::pin(futures::stream::iter(events.into_iter().map(Ok))))
}
}
type CapturedDomainAttempt = (JobSnapshot, Arc<dyn JobAttemptFence>);
#[derive(Default)]
struct RecordingDomainProvider(Mutex<Vec<CapturedDomainAttempt>>);
fn domain_grant(id: &str) -> Value {
json!({"consumer":"test-domain", "job_id":id, "input_revision":"r1"})
}
fn domain_request(id: &str) -> JobRequest {
let mut intent = request(id);
intent.profile = "builtin:standalone-subjektiv-consolidation".into();
intent
}
impl JobDomainProvider for RecordingDomainProvider {
fn grant(
&self,
snapshot: &JobSnapshot,
fence: Arc<dyn JobAttemptFence>,
) -> Result<worker::job::JobFeatureGrant, String> {
if snapshot.domain_grant != Some(domain_grant(&snapshot.request.job_id)) {
return Err("foreign or mismatched immutable domain grant".into());
}
self.0.lock().unwrap().push((snapshot.clone(), fence));
Ok(worker::job::JobFeatureGrant {
satisfied_requirements: vec!["feature.subjektiv"],
..Default::default()
})
}
}
async fn await_model(client: &DomainTestClient) {
tokio::time::timeout(Duration::from_secs(2), client.entered.notified())
.await
.unwrap();
}
async fn await_job(jobs: &StandaloneJobs, id: &str) -> JobSnapshot {
tokio::time::timeout(Duration::from_secs(3), jobs.wait(id))
.await
.unwrap()
.unwrap()
}
#[tokio::test]
async fn pending_domain_grants_survive_service_reopen_and_remain_immutable() {
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("jobs.sqlite3");
let jobs = StandaloneJobs::open(&path, temp.path().to_path_buf()).unwrap();
let intent = domain_request("granted");
let grant = domain_grant("granted");
let reserved = jobs.request_granted(intent.clone(), grant.clone()).unwrap();
let root = jobs.request(request("root")).unwrap();
assert_eq!(reserved.domain_grant, Some(grant.clone()));
assert!(root.domain_grant.is_none());
assert_eq!(
jobs.request_granted(intent.clone(), grant.clone()).unwrap(),
reserved
);
assert!(matches!(
jobs.request(intent.clone()),
Err(JobServiceError::Store(JobStoreError::IntentConflict(_)))
));
assert!(matches!(
jobs.request_granted(intent.clone(), domain_grant("foreign")),
Err(JobServiceError::Store(JobStoreError::IntentConflict(_)))
));
assert!(matches!(
jobs.request_granted(request("root"), grant.clone()),
Err(JobServiceError::Store(JobStoreError::IntentConflict(_)))
));
assert_eq!(jobs.pending().unwrap().len(), 2);
jobs.shutdown().await.unwrap();
let reopened = StandaloneJobs::open(&path, temp.path().to_path_buf()).unwrap();
let pending = reopened.pending().unwrap();
assert_eq!(pending.len(), 2);
assert!(pending.contains(&reserved));
assert!(pending.contains(&root));
assert_eq!(reopened.get("granted").unwrap(), reserved);
assert_eq!(
reopened
.request_granted(intent.clone(), grant.clone())
.unwrap(),
reserved
);
assert!(matches!(
reopened.request_granted(intent, json!({"different":true})),
Err(JobServiceError::Store(JobStoreError::IntentConflict(_)))
));
let provider = Arc::new(RecordingDomainProvider::default());
reopened.bind_domain(provider.clone()).unwrap();
reopened
.start_with_model_client("granted", DomainTestClient::success())
.await
.unwrap();
let completed = await_job(&reopened, "granted").await;
assert_eq!(completed.state, JobState::Completed);
assert_eq!(completed.domain_grant, Some(grant));
assert_eq!(completed.result, Some(json!({"valid":true})));
let captured = provider.0.lock().unwrap()[0].0.clone();
assert_eq!(
captured, reserved,
"provider receives the persisted immutable intent and grant"
);
reopened.shutdown().await.unwrap();
}
#[tokio::test]
async fn persisted_grant_requires_provider_and_missing_grant_cannot_be_derived_from_profile() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let granted = jobs
.request_granted(domain_request("granted"), domain_grant("granted"))
.unwrap();
let client = DomainTestClient::success();
let error = jobs
.start_with_model_client("granted", client.clone())
.await
.unwrap_err();
assert!(matches!(error, JobServiceError::Configuration(_)));
assert!(
error
.to_string()
.contains("unavailable explicit domain capability"),
"{error}"
);
assert_eq!(jobs.get("granted").unwrap(), granted);
assert!(client.requests.lock().unwrap().is_empty());
let provider = Arc::new(RecordingDomainProvider::default());
jobs.bind_domain(provider.clone()).unwrap();
assert!(matches!(
jobs.bind_domain(provider.clone()),
Err(JobServiceError::Configuration(_))
));
let ungranted = jobs.request(domain_request("ungranted")).unwrap();
let error = jobs
.start_with_model_client("ungranted", client.clone())
.await
.unwrap_err();
assert!(matches!(error, JobServiceError::Configuration(_)));
assert!(error.to_string().contains("feature.subjektiv"), "{error}");
assert_eq!(jobs.get("ungranted").unwrap(), ungranted);
assert!(
provider.0.lock().unwrap().is_empty(),
"bound provider is not a profile-derived grant"
);
assert!(client.requests.lock().unwrap().is_empty());
jobs.shutdown().await.unwrap();
}
#[tokio::test]
async fn foreign_job_revision_and_consumer_grants_are_refused_before_model_dispatch() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let provider = Arc::new(RecordingDomainProvider::default());
jobs.bind_domain(provider.clone()).unwrap();
for (id, grant) in [
("foreign-job", domain_grant("another-job")),
(
"foreign-revision",
json!({"consumer":"test-domain","job_id":"foreign-revision","input_revision":"r0"}),
),
(
"foreign-consumer",
json!({"consumer":"another-domain","job_id":"foreign-consumer","input_revision":"r1"}),
),
] {
let reserved = jobs.request_granted(domain_request(id), grant).unwrap();
let client = DomainTestClient::success();
let error = jobs
.start_with_model_client(id, client.clone())
.await
.unwrap_err();
assert!(matches!(error, JobServiceError::Configuration(_)));
assert!(
error.to_string().contains("foreign or mismatched"),
"{error}"
);
assert_eq!(jobs.get(id).unwrap(), reserved);
assert!(client.requests.lock().unwrap().is_empty());
}
assert!(provider.0.lock().unwrap().is_empty());
jobs.shutdown().await.unwrap();
}
#[tokio::test]
async fn domain_fence_is_live_only_for_current_dispatched_attempt_and_revoked_on_cancel() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let provider = Arc::new(RecordingDomainProvider::default());
jobs.bind_domain(provider.clone()).unwrap();
let reserved = jobs
.request_granted(domain_request("fenced"), domain_grant("fenced"))
.unwrap();
let failure_release = Arc::new(tokio::sync::Notify::new());
let first = DomainTestClient {
failure_release: Some(failure_release.clone()),
..Default::default()
};
jobs.start_with_model_client("fenced", first.clone())
.await
.unwrap();
await_model(&first).await;
let old_fence = provider.0.lock().unwrap()[0].1.clone();
old_fence.ensure_live().unwrap();
failure_release.notify_one();
let failed = await_job(&jobs, "fenced").await;
assert_eq!(failed.state, JobState::Failed);
assert!(failed.result.is_none());
assert!(old_fence.ensure_live().is_err());
// Only known failure permits a retry; cancellation remains terminal.
let retried = jobs.retry("fenced").unwrap();
assert_ne!(retried.attempt.attempt_id, reserved.attempt.attempt_id);
assert_eq!(retried.domain_grant, reserved.domain_grant);
assert!(old_fence.ensure_live().is_err());
let second = DomainTestClient {
stall: true,
..Default::default()
};
jobs.start_with_model_client("fenced", second.clone())
.await
.unwrap();
await_model(&second).await;
let current_fence = provider.0.lock().unwrap()[1].1.clone();
current_fence.ensure_live().unwrap();
assert!(
old_fence.ensure_live().is_err(),
"a new attempt must not revive a stale capability"
);
let cancelled = jobs.cancel("fenced").unwrap();
assert_eq!(cancelled.state, JobState::Cancelled);
assert!(
current_fence.ensure_live().is_err(),
"durable cancellation revokes current domain operations immediately"
);
assert!(old_fence.ensure_live().is_err());
let cleaned = await_job(&jobs, "fenced").await;
assert_eq!(cleaned.state, JobState::Cancelled);
assert!(cleaned.result.is_none());
assert!(
jobs.retry("fenced").is_err(),
"cancelled intent is not a known retryable failure"
);
jobs.shutdown().await.unwrap();
assert!(old_fence.ensure_live().is_err());
assert!(current_fence.ensure_live().is_err());
}
#[tokio::test]
async fn bound_domain_does_not_change_root_generic_result_only_execution() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let provider = Arc::new(RecordingDomainProvider::default());
jobs.bind_domain(provider.clone()).unwrap();
let root = jobs.request(request("root")).unwrap();
assert!(root.domain_grant.is_none());
let client = DomainTestClient::success();
jobs.start_with_model_client("root", client.clone())
.await
.unwrap();
let completed = await_job(&jobs, "root").await;
assert_eq!(completed.state, JobState::Completed);
assert_eq!(completed.result, Some(json!({"valid":true})));
assert!(completed.domain_grant.is_none());
assert!(provider.0.lock().unwrap().is_empty());
let requests = client.requests.lock().unwrap();
assert_eq!(requests.len(), 2);
for sent in requests.iter() {
assert_eq!(
sent.tools
.iter()
.map(|tool| tool.name.as_str())
.collect::<Vec<_>>(),
["SubmitJobResult"]
);
assert_eq!(sent.config.max_tokens, Some(worker::job::MAX_JOB_TOKENS));
}
drop(requests);
jobs.shutdown().await.unwrap();
}
#[tokio::test]
async fn host_result_capability_cannot_cross_job_attempt_revision_or_deadline() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let first = jobs.request(request("first")).unwrap();
let second = jobs.request(request("second")).unwrap();
for snapshot in [&first, &second] {
jobs.with_store(|store| {
store.dispatch(&snapshot.request.job_id, &snapshot.attempt.attempt_id, 8)
})
.unwrap();
}
let bound = sink(&jobs, "first", &first.attempt.attempt_id);
assert!(
bound
.submit(result("second", &second.attempt.attempt_id))
.is_err()
);
assert!(bound.submit(result("first", "first:attempt:2")).is_err());
let mut stale = result("first", &first.attempt.attempt_id);
stale.input_revision = "r0".into();
assert!(bound.submit(stale).is_err());
let mut expired = sink(&jobs, "first", &first.attempt.attempt_id);
expired.deadline = tokio::time::Instant::now() - Duration::from_secs(1);
assert!(
expired
.submit(result("first", &first.attempt.attempt_id))
.is_err()
);
assert!(jobs.get("first").unwrap().result.is_none());
bound
.submit(result("first", &first.attempt.attempt_id))
.unwrap();
// Accepted exact replay is still allowed after the deadline, never a new result.
expired
.submit(result("first", &first.attempt.attempt_id))
.unwrap();
let mut changed = result("first", &first.attempt.attempt_id);
changed.result = serde_json::json!({"valid":false});
assert!(expired.submit(changed).is_err());
assert_eq!(jobs.get("second").unwrap().state, JobState::Pending);
jobs.shutdown().await.unwrap();
assert!(
bound
.submit(result("first", &first.attempt.attempt_id))
.is_err()
);
}
#[tokio::test]
async fn cleanup_cannot_reclassify_success_or_durable_cancellation_and_stale_handles_are_closed() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let cancelled = jobs.request(request("cancelled")).unwrap();
jobs.cancel("cancelled").unwrap();
finish_if_pending(
&jobs.store,
"cancelled",
&cancelled.attempt.attempt_id,
JobAttemptState::Unknown,
"cleanup",
)
.unwrap();
assert_eq!(jobs.get("cancelled").unwrap().state, JobState::Cancelled);
let accepted = jobs.request(request("accepted")).unwrap();
jobs.with_store(|store| store.dispatch("accepted", &accepted.attempt.attempt_id, 8))
.unwrap();
sink(&jobs, "accepted", &accepted.attempt.attempt_id)
.submit(result("accepted", &accepted.attempt.attempt_id))
.unwrap();
finish_if_pending(
&jobs.store,
"accepted",
&accepted.attempt.attempt_id,
JobAttemptState::Failed,
"model failed later",
)
.unwrap();
assert_eq!(jobs.get("accepted").unwrap().state, JobState::Completed);
jobs.shutdown().await.unwrap();
assert!(jobs.store.lock().unwrap().is_none());
for operation in [
jobs.cancel("cancelled"),
jobs.acknowledge("accepted"),
jobs.retry("cancelled"),
jobs.request(request("new")),
] {
assert!(matches!(operation, Err(JobServiceError::Closed)));
}
}
+32
View File
@@ -15,6 +15,7 @@ pub struct StandaloneLaunchConfig {
pub state_dir: PathBuf,
pub profile: ProfileSelector,
pub worker_name: String,
pub subject_id: Option<String>,
}
pub struct ResolvedStandaloneLaunch {
@@ -22,6 +23,7 @@ pub struct ResolvedStandaloneLaunch {
pub state_dir: PathBuf,
pub profile: ResolvedProfile,
pub prompt_catalog: PromptCatalogSource,
pub subject_id: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
@@ -32,6 +34,10 @@ pub enum StandaloneLaunchError {
PathProfileUnsupported,
#[error("the standalone profile could not be resolved")]
ProfileResolutionFailed,
#[error(
"local Subject selection requires an enabled, non-consolidator subjektiv Profile and canonical Subject id"
)]
InvalidSubjectPolicy,
}
impl StandaloneLaunchConfig {
@@ -46,9 +52,15 @@ impl StandaloneLaunchConfig {
state_dir: state_dir.into(),
profile,
worker_name: worker_name.into(),
subject_id: None,
}
}
pub fn with_subject(mut self, subject_id: String) -> Self {
self.subject_id = Some(subject_id);
self
}
/// Resolve only built-in/XDG profile authority and bind standalone scope
/// to the canonical process cwd. Repository-local profile discovery is
/// deliberately not part of this path.
@@ -74,12 +86,32 @@ impl StandaloneLaunchConfig {
.map_err(|_| StandaloneLaunchError::ProfileResolutionFailed)?;
validate_profile_execution_target(&profile.manifest, ProfileExecutionTarget::Standalone)
.map_err(|_| StandaloneLaunchError::ProfileResolutionFailed)?;
if let Some(id) = &self.subject_id {
if !crate::subjektiv::valid_subject_id(id)
|| !profile.manifest.feature.subjektiv.profile.enabled
|| profile
.manifest
.feature
.subjektiv
.profile
.consolidation_tools
|| profile
.manifest
.feature
.subjektiv
.workspace_settings
.is_some()
{
return Err(StandaloneLaunchError::InvalidSubjectPolicy);
}
}
Ok(ResolvedStandaloneLaunch {
cwd,
state_dir: self.state_dir,
profile,
prompt_catalog: PromptCatalogSource::builtins_only(),
subject_id: self.subject_id,
})
}
}
+4
View File
@@ -5,8 +5,12 @@
//! Server, HTTP, WebSocket, subprocess Worker, or alternative execution path.
pub mod host;
mod job_store;
pub mod jobs;
pub mod launch;
pub mod store;
pub mod subjektiv;
mod subjektiv_sessions;
pub use host::{StandaloneHost, StandaloneShutdownError, StandaloneStartupError};
pub use launch::{ResolvedStandaloneLaunch, StandaloneLaunchConfig, StandaloneLaunchError};
+38
View File
@@ -87,6 +87,9 @@ pub struct StandaloneWorkerRecord {
pub storage_key: String,
pub cwd: StandaloneCwdIdentity,
pub manifest: WorkerManifest,
/// Original explicit local Subject/storage binding; never inferred from policy.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub subject: Option<crate::subjektiv::StandaloneSubjectBinding>,
pub active_session_id: SessionId,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub active_segment_id: Option<SegmentId>,
@@ -127,6 +130,16 @@ impl StandaloneWorkerStore {
Ok(Self { root })
}
/// Observational constructor: never creates a missing root or repairs state.
pub fn open_existing(root: impl Into<PathBuf>) -> Result<Self, StandaloneStoreError> {
let root = root.into();
let metadata = fs::symlink_metadata(&root).map_err(StandaloneStoreError::Io)?;
if !metadata.is_dir() || metadata.file_type().is_symlink() {
return Err(StandaloneStoreError::NotDirectory);
}
Ok(Self { root })
}
#[must_use]
pub fn root(&self) -> &Path {
&self.root
@@ -158,6 +171,25 @@ impl StandaloneWorkerStore {
storage_key: String,
active_session_id: SessionId,
active_segment_id: Option<SegmentId>,
) -> Result<StandaloneWorkerRecord, StandaloneStoreError> {
self.commit_created_connected(
allocation,
manifest,
storage_key,
active_session_id,
active_segment_id,
None,
)
}
pub(crate) fn commit_created_connected(
&self,
allocation: &StandaloneWorkerAllocation,
manifest: WorkerManifest,
storage_key: String,
active_session_id: SessionId,
active_segment_id: Option<SegmentId>,
subject: Option<crate::subjektiv::StandaloneSubjectBinding>,
) -> Result<StandaloneWorkerRecord, StandaloneStoreError> {
let now = now_unix_ms()?;
let record = StandaloneWorkerRecord {
@@ -168,6 +200,7 @@ impl StandaloneWorkerStore {
storage_key,
cwd: allocation.cwd.clone(),
manifest,
subject,
active_session_id,
active_segment_id,
status: StandaloneWorkerStatus::Active,
@@ -352,6 +385,10 @@ impl StandaloneWorkerStore {
}
#[must_use]
pub(crate) fn jobs_path(&self, id: WorkerId) -> PathBuf {
self.worker_dir(id).join("jobs.sqlite3")
}
pub(crate) fn runtime_dir(&self, id: WorkerId) -> PathBuf {
self.worker_dir(id).join("runtime")
}
@@ -789,6 +826,7 @@ permission = "write"
inode: None,
},
manifest,
subject: None,
active_session_id: "01a05782-d5dd-78f1-b9cd-ce37535bdb9e".parse().unwrap(),
active_segment_id: None,
status: StandaloneWorkerStatus::Stopped,
File diff suppressed because it is too large Load Diff
+130
View File
@@ -0,0 +1,130 @@
use super::*;
#[test]
fn catalog_reopen_preserves_random_scope_subject_and_behavior() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("state");
let catalog = StandaloneSubjects::open(&root).unwrap();
let subject = catalog
.create(
"continuing-owner",
Some("Keep user behavior separate from Memory."),
)
.unwrap();
let scope = catalog.scope_id().to_owned();
assert!(uuid::Uuid::parse_str(&scope).is_ok());
assert!(uuid::Uuid::parse_str(subject.id.strip_prefix("subject-").unwrap()).is_ok());
assert_ne!(scope, subject.id);
drop(catalog);
let reopened = StandaloneSubjects::open_existing(&root).unwrap();
assert_eq!(reopened.scope_id(), scope);
assert_eq!(reopened.list().unwrap(), vec![subject]);
}
#[test]
fn implicit_open_does_not_create_subject_storage() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("missing");
assert!(StandaloneSubjects::open_existing(&root).is_err());
assert!(!root.exists());
assert!(StandaloneSubjects::open_existing(temp.path()).is_err());
assert!(!temp.path().join("subjektiv").exists());
}
#[test]
fn subject_kernel_lease_is_not_worker_or_profile_scoped() {
let temp = tempfile::tempdir().unwrap();
let catalog = StandaloneSubjects::open(temp.path().join("state")).unwrap();
let a = catalog.create("a", None).unwrap();
let b = catalog.create("b", None).unwrap();
let mut first = catalog
.connect(&a.id, protocol::WorkerId::now_v7())
.unwrap();
assert!(matches!(
StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&a.id, protocol::WorkerId::now_v7()),
Err(SubjectError::Active(_))
));
let mut second = StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&b.id, protocol::WorkerId::now_v7())
.unwrap();
first.close().unwrap();
let mut replacement = StandaloneSubjects::open_existing(&temp.path().join("state"))
.unwrap()
.connect(&a.id, protocol::WorkerId::now_v7())
.unwrap();
replacement.close().unwrap();
second.close().unwrap();
}
#[test]
fn closed_capabilities_and_foreign_worker_context_are_rejected() {
let temp = tempfile::tempdir().unwrap();
let catalog = StandaloneSubjects::open(temp.path().join("state")).unwrap();
let subject = catalog.create("owner", None).unwrap();
let id = protocol::WorkerId::now_v7();
let mut connection = catalog.connect(&subject.id, id).unwrap();
let port = connection.host.clone();
let foreign = SubjektivHostContext {
worker_id: protocol::WorkerId::now_v7().to_string(),
session_id: session_store::SessionId::now_v7().to_string(),
};
let read = SubjektivMemoryBackendOperation::ResidentContext(
memory::backend::MemoryResidentSummaryOperation {},
);
assert!(port.memory(&foreign, read.clone()).is_err());
let owner = SubjektivHostContext {
worker_id: format!("standalone-{id}"),
session_id: session_store::SessionId::now_v7().to_string(),
};
assert!(port.memory(&owner, read.clone()).is_ok());
assert!(
port.record_session(&owner, false).is_err(),
"unattributed Session is not read authority"
);
connection.close().unwrap();
assert!(port.memory(&owner, read).is_err());
}
#[cfg(unix)]
#[test]
fn scope_rejects_symlinks_and_non_private_roots() {
use std::os::unix::fs::{PermissionsExt, symlink};
let temp = tempfile::tempdir().unwrap();
let alias = temp.path().join("alias");
let real = temp.path().join("real");
fs::create_dir(&real).unwrap();
fs::set_permissions(&real, fs::Permissions::from_mode(0o700)).unwrap();
symlink(&real, &alias).unwrap();
assert!(StandaloneSubjects::open(&alias).is_err());
let public = temp.path().join("public");
fs::create_dir(&public).unwrap();
fs::set_permissions(&public, fs::Permissions::from_mode(0o755)).unwrap();
assert!(StandaloneSubjects::open(&public).is_err());
let catalog = StandaloneSubjects::open(&real).unwrap();
drop(catalog);
let scope = real.join("subjektiv/scope.json");
let original = fs::read(&scope).unwrap();
fs::remove_file(&scope).unwrap();
let foreign = temp.path().join("foreign.json");
fs::write(&foreign, &original).unwrap();
symlink(&foreign, &scope).unwrap();
assert!(StandaloneSubjects::open_existing(&real).is_err());
assert_eq!(fs::read(&foreign).unwrap(), original);
}
#[test]
fn different_local_roots_do_not_authorize_shared_subject_ids() {
let temp = tempfile::tempdir().unwrap();
let first = StandaloneSubjects::open(temp.path().join("first")).unwrap();
let subject = first.create("owner", None).unwrap();
let other = StandaloneSubjects::open(temp.path().join("other")).unwrap();
assert_ne!(first.scope_id(), other.scope_id());
assert!(
other
.connect(&subject.id, protocol::WorkerId::now_v7())
.is_err()
);
}
File diff suppressed because it is too large Load Diff
+478
View File
@@ -0,0 +1,478 @@
use std::collections::VecDeque;
use std::pin::Pin;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use agen::llm_client::{
LlmClient,
error::ClientError,
event::{Event as LlmEvent, StopReason},
types::Request,
};
use async_trait::async_trait;
use futures::{Stream, stream};
use job::{JobLimits, JobRequest};
use standalone::jobs::{JobServiceError, JobState};
use standalone::{StandaloneHost, StandaloneLaunchConfig};
#[derive(Clone)]
struct ScriptedClient {
responses: Arc<Mutex<VecDeque<Vec<LlmEvent>>>>,
requests: Arc<Mutex<Vec<Request>>>,
started: Arc<tokio::sync::Notify>,
stall: bool,
fail: bool,
}
impl ScriptedClient {
fn new(responses: Vec<Vec<LlmEvent>>) -> Self {
Self {
responses: Arc::new(Mutex::new(responses.into())),
requests: Arc::new(Mutex::new(Vec::new())),
started: Arc::new(tokio::sync::Notify::new()),
stall: false,
fail: false,
}
}
fn stall() -> Self {
Self {
stall: true,
..Self::new(Vec::new())
}
}
fn requests(&self) -> Vec<Request> {
self.requests.lock().unwrap().clone()
}
}
#[async_trait]
impl LlmClient for ScriptedClient {
async fn stream(
&self,
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<LlmEvent, ClientError>> + Send>>, ClientError>
{
self.requests.lock().unwrap().push(request);
self.started.notify_one();
if self.fail {
return Err(ClientError::Config("scripted model failure".into()));
}
if self.stall {
return Ok(Box::pin(stream::pending()));
}
let events = self
.responses
.lock()
.unwrap()
.pop_front()
.expect("scripted response");
Ok(Box::pin(stream::iter(events.into_iter().map(Ok))))
}
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
}
fn prose() -> Vec<LlmEvent> {
vec![
LlmEvent::text_block_start(0),
LlmEvent::text_delta(0, "done"),
LlmEvent::text_block_stop(0, Some(StopReason::EndTurn)),
]
}
fn success() -> ScriptedClient {
ScriptedClient::new(vec![
vec![
LlmEvent::tool_use_start(0, "result-1", "SubmitJobResult"),
LlmEvent::tool_input_delta(0, r#"{"result":{"valid":true}}"#),
LlmEvent::tool_use_stop(0),
],
prose(),
])
}
fn request(id: &str) -> JobRequest {
JobRequest {
job_id: id.into(),
purpose: "generic_test".into(),
input_revision: "r1".into(),
input_ref: "test:immutable".into(),
input: serde_json::json!({"test_input":42}),
instruction: "Return a structured validity result".into(),
profile: "builtin:job".into(),
serialization_key: None,
limits: JobLimits::default(),
}
}
async fn host(temp: &tempfile::TempDir) -> (StandaloneHost, ScriptedClient) {
let launch = StandaloneLaunchConfig::new(
temp.path(),
temp.path().join("state"),
manifest::ProfileSelector::Default,
"job-host",
)
.resolve()
.unwrap();
let client = ScriptedClient::new(Vec::new());
let host = StandaloneHost::start_with_model_client(launch, client.clone())
.await
.unwrap();
(host, client)
}
async fn wait_started(client: &ScriptedClient) {
tokio::time::timeout(Duration::from_secs(10), client.started.notified())
.await
.unwrap();
}
#[tokio::test]
async fn host_job_runs_without_dialog_and_result_survives_restore_and_ack() {
let temp = tempfile::tempdir().unwrap();
let (host, dialog) = host(&temp).await;
let id = host.worker_id();
let jobs = host.jobs();
let intent = request("success");
let reserved = jobs.request(intent.clone()).unwrap();
assert_eq!(jobs.request(intent.clone()).unwrap(), reserved);
let model = success();
jobs.start_with_model_client("success", model.clone())
.await
.unwrap();
let completed = jobs.wait("success").await.unwrap();
assert_eq!(completed.state, JobState::Completed);
assert_eq!(completed.result, Some(serde_json::json!({"valid":true})));
assert!(!completed.acknowledged);
let neutral = completed.outcome();
assert_eq!(neutral.state, job::JobState::Completed);
assert_eq!(neutral.result, completed.result);
assert_eq!(neutral.attempt_state, job::JobAttemptState::Completed);
assert!(
dialog.requests().is_empty(),
"Job never runs on the interactive model/session"
);
assert_eq!(
model.requests()[0]
.tools
.iter()
.map(|tool| tool.name.as_str())
.collect::<Vec<_>>(),
["SubmitJobResult"]
);
assert_eq!(jobs.request(intent.clone()).unwrap(), completed);
assert!(
jobs.start_with_model_client("success", success())
.await
.is_err()
);
assert!(jobs.retry("success").is_err());
let db = temp
.path()
.join("state")
.join(id.to_string())
.join("jobs.sqlite3");
assert!(db.exists());
host.shutdown().await.unwrap();
assert!(matches!(
jobs.request(request("after-shutdown")),
Err(JobServiceError::Closed)
));
assert!(matches!(
jobs.cancel("success"),
Err(JobServiceError::Closed)
));
assert!(matches!(
jobs.acknowledge("success"),
Err(JobServiceError::Closed)
));
assert!(matches!(
jobs.start_with_model_client("success", success()).await,
Err(JobServiceError::Closed)
));
let restored = StandaloneHost::restore_with_model_client(
temp.path().join("state"),
id,
ScriptedClient::new(Vec::new()),
)
.await
.unwrap();
let restored_jobs = restored.jobs();
assert_eq!(restored_jobs.get("success").unwrap(), completed);
assert!(restored_jobs.acknowledge("success").unwrap().acknowledged);
restored.shutdown().await.unwrap();
}
#[tokio::test]
async fn live_resource_limits_cancellation_and_wait_are_host_owned() {
let temp = tempfile::tempdir().unwrap();
let (host, _) = host(&temp).await;
let jobs = host.jobs();
let mut first = request("first");
first.limits.max_concurrent_jobs = 1;
jobs.request(first).unwrap();
jobs.request(request("second")).unwrap();
let stalled = ScriptedClient::stall();
jobs.start_with_model_client("first", stalled.clone())
.await
.unwrap();
wait_started(&stalled).await;
// Waiting must not remove the Host-owned task/cancellation capability.
let waiting_jobs = jobs.clone();
let waiter = tokio::spawn(async move { waiting_jobs.wait("first").await });
assert!(matches!(
jobs.start_with_model_client("second", success()).await,
Err(JobServiceError::Capacity)
));
jobs.cancel("first").unwrap();
let cancelled = tokio::time::timeout(Duration::from_secs(10), waiter)
.await
.unwrap()
.unwrap()
.unwrap();
assert_eq!(cancelled.state, JobState::Cancelled);
assert!(jobs.retry("first").is_err());
jobs.start_with_model_client("second", success())
.await
.unwrap();
assert_eq!(
jobs.wait("second").await.unwrap().state,
JobState::Completed
);
host.shutdown().await.unwrap();
}
#[tokio::test]
async fn prose_failure_requires_explicit_bounded_new_attempt_and_exact_replay_does_not_retry() {
let temp = tempfile::tempdir().unwrap();
let (host, _) = host(&temp).await;
let jobs = host.jobs();
let intent = request("prose");
jobs.request(intent.clone()).unwrap();
jobs.start_with_model_client("prose", ScriptedClient::new(vec![prose()]))
.await
.unwrap();
let failed = jobs.wait("prose").await.unwrap();
assert_eq!(failed.state, JobState::Failed);
assert!(failed.result.is_none());
assert_eq!(jobs.request(intent).unwrap(), failed);
let retry = jobs.retry("prose").unwrap();
assert_eq!(retry.attempt.number, 2);
assert_ne!(retry.attempt.attempt_id, failed.attempt.attempt_id);
jobs.start_with_model_client("prose", ScriptedClient::new(vec![prose()]))
.await
.unwrap();
assert_eq!(jobs.wait("prose").await.unwrap().state, JobState::Failed);
assert!(jobs.retry("prose").is_err());
host.shutdown().await.unwrap();
}
#[tokio::test]
async fn timeout_is_unknown_not_prose_success_or_automatic_retry() {
let temp = tempfile::tempdir().unwrap();
let (host, _) = host(&temp).await;
let jobs = host.jobs();
let mut intent = request("timeout");
intent.limits.timeout_seconds = 1;
jobs.request(intent).unwrap();
let stalled = ScriptedClient::stall();
jobs.start_with_model_client("timeout", stalled.clone())
.await
.unwrap();
wait_started(&stalled).await;
let outcome = tokio::time::timeout(Duration::from_secs(10), jobs.wait("timeout"))
.await
.unwrap()
.unwrap();
assert_eq!(outcome.state, JobState::Unknown);
assert!(outcome.result.is_none());
assert!(jobs.retry("timeout").is_err());
host.shutdown().await.unwrap();
}
#[tokio::test]
async fn shutdown_keeps_reserved_results_and_fences_running_attempts_on_reopen() {
let temp = tempfile::tempdir().unwrap();
let (host, _) = host(&temp).await;
let id = host.worker_id();
let jobs = host.jobs();
jobs.request(request("reserved")).unwrap();
jobs.request(request("cancelled")).unwrap();
jobs.cancel("cancelled").unwrap();
jobs.request(request("running")).unwrap();
let stalled = ScriptedClient::stall();
jobs.start_with_model_client("running", stalled.clone())
.await
.unwrap();
wait_started(&stalled).await;
host.shutdown().await.unwrap();
assert!(matches!(jobs.get("running"), Err(JobServiceError::Closed)));
let restored = StandaloneHost::restore_with_model_client(
temp.path().join("state"),
id,
ScriptedClient::new(Vec::new()),
)
.await
.unwrap();
let jobs = restored.jobs();
assert_eq!(
jobs.pending()
.unwrap()
.iter()
.map(|snapshot| snapshot.request.job_id.as_str())
.collect::<Vec<_>>(),
["reserved"]
);
assert_eq!(jobs.get("cancelled").unwrap().state, JobState::Cancelled);
assert_eq!(jobs.get("running").unwrap().state, JobState::Unknown);
assert!(jobs.retry("running").is_err());
jobs.start_with_model_client("reserved", success())
.await
.unwrap();
assert_eq!(
jobs.wait("reserved").await.unwrap().state,
JobState::Completed
);
restored.shutdown().await.unwrap();
}
#[tokio::test]
async fn profile_requirements_are_rejected_without_model_execution_or_implicit_fallback() {
let temp = tempfile::tempdir().unwrap();
let (host, _) = host(&temp).await;
let jobs = host.jobs();
for profile in [
"builtin:coder",
"builtin:backend-job",
"builtin:does-not-exist",
] {
let mut intent = request(profile);
intent.profile = profile.into();
jobs.request(intent).unwrap();
let model = success();
assert!(matches!(
jobs.start_with_model_client(profile, model.clone()).await,
Err(JobServiceError::Configuration(_))
));
assert!(model.requests().is_empty());
assert_eq!(jobs.get(profile).unwrap().state, JobState::Pending);
}
host.shutdown().await.unwrap();
}
// Only this test harness starts a subprocess: production Jobs remain in-process.
// The child exits deliberately without destructors to prove actual process-loss
// recovery, rather than treating a hand-edited SQLite fixture as that evidence.
#[tokio::test]
#[ignore = "entry point for process_loss_recovers_durable_jobs_without_replaying_unknown"]
async fn process_job_fixture() {
let root = std::path::PathBuf::from(
std::env::var_os("YOI_JOB_PROCESS_FIXTURE_DIR").expect("fixture root"),
);
let launch = StandaloneLaunchConfig::new(
&root,
root.join("state"),
manifest::ProfileSelector::Default,
"job-process-fixture",
)
.resolve()
.unwrap();
let host = StandaloneHost::start_with_model_client(launch, ScriptedClient::new(Vec::new()))
.await
.unwrap();
std::fs::write(root.join("worker-id"), host.worker_id().to_string()).unwrap();
let jobs = host.jobs();
jobs.request(request("reserved")).unwrap();
jobs.request(request("cancelled")).unwrap();
jobs.cancel("cancelled").unwrap();
jobs.request(request("accepted")).unwrap();
jobs.start_with_model_client("accepted", success())
.await
.unwrap();
assert_eq!(
jobs.wait("accepted").await.unwrap().state,
JobState::Completed
);
jobs.request(request("interrupted")).unwrap();
let stalled = ScriptedClient::stall();
jobs.start_with_model_client("interrupted", stalled.clone())
.await
.unwrap();
wait_started(&stalled).await;
// Test-owned process only, never the operating Yoi Worker or dogfood daemon.
std::process::exit(23);
}
#[tokio::test]
async fn process_loss_recovers_durable_jobs_without_replaying_unknown() {
let temp = tempfile::tempdir().unwrap();
let output = std::process::Command::new(std::env::current_exe().unwrap())
.args(["--exact", "process_job_fixture", "--ignored", "--nocapture"])
.env("YOI_JOB_PROCESS_FIXTURE_DIR", temp.path())
.output()
.unwrap();
assert_eq!(
output.status.code(),
Some(23),
"fixture failed: {}\n{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
let worker_id = std::fs::read_to_string(temp.path().join("worker-id"))
.unwrap()
.parse()
.unwrap();
let host = StandaloneHost::restore_with_model_client(
temp.path().join("state"),
worker_id,
ScriptedClient::new(Vec::new()),
)
.await
.unwrap();
let jobs = host.jobs();
assert_eq!(jobs.get("interrupted").unwrap().state, JobState::Unknown);
assert!(jobs.retry("interrupted").is_err());
let completed = jobs.get("accepted").unwrap();
assert_eq!(completed.state, JobState::Completed);
assert_eq!(completed.result, Some(serde_json::json!({"valid":true})));
assert!(!completed.acknowledged);
assert_eq!(jobs.get("cancelled").unwrap().state, JobState::Cancelled);
assert_eq!(jobs.pending().unwrap().len(), 1);
jobs.start_with_model_client("reserved", success())
.await
.unwrap();
assert_eq!(
jobs.wait("reserved").await.unwrap().state,
JobState::Completed
);
host.shutdown().await.unwrap();
}
#[tokio::test]
async fn model_failure_is_durable_known_failure_with_explicit_successful_retry() {
let temp = tempfile::tempdir().unwrap();
let (host, _) = host(&temp).await;
let jobs = host.jobs();
jobs.request(request("model-failure")).unwrap();
let client = ScriptedClient {
fail: true,
..ScriptedClient::new(Vec::new())
};
jobs.start_with_model_client("model-failure", client)
.await
.unwrap();
let failure = jobs.wait("model-failure").await.unwrap();
assert_eq!(failure.state, JobState::Failed);
assert!(failure.result.is_none());
assert!(
failure
.attempt
.failure
.unwrap()
.contains("scripted model failure")
);
let next = jobs.retry("model-failure").unwrap();
assert_eq!(next.attempt.number, 2);
jobs.start_with_model_client("model-failure", success())
.await
.unwrap();
assert_eq!(
jobs.wait("model-failure").await.unwrap().state,
JobState::Completed
);
host.shutdown().await.unwrap();
}
File diff suppressed because it is too large Load Diff
+21
View File
@@ -0,0 +1,21 @@
[package]
name = "subjektiv"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish = false
[dependencies]
feature-storage = { path = "../feature-storage" }
chrono = { version = "0.4", default-features = false, features = ["clock"] }
memory.workspace = true
manifest.workspace = true
rusqlite.workspace = true
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
server-api.workspace = true
thiserror.workspace = true
uuid = { workspace = true, features = ["v7"] }
[dev-dependencies]
tempfile.workspace = true
+76
View File
@@ -0,0 +1,76 @@
# subjektiv
Shared Subject-scoped Memory domain and model-facing operations. This crate has
no Workspace Server dependency. `server-api` is the existing transport-neutral
request/response contract crate, not an authentication or Server implementation.
## Storage scope
The trusted Host creates `feature_storage::FeatureStorage`, selects
`FeatureStorage::scope(scope_id)`, registers `REGISTRATION` (or uses
`SubjektivStore::register`), and opens `SubjektivStore::open`. A trusted managed
`FeatureDatabase` can also be opened with `SubjektivStore::from_database`.
`scope_id` identifies the physical storage sharing boundary. Backend uses its
existing Workspace ID. Standalone uses a persisted random local storage-scope
identity under its operator-managed state directory, **not a fake Workspace**.
Subject IDs are independently issued random domain identities. An ID is a lookup
key, never an authorization credential.
Existing SQL schema and migrations are retained. In particular,
`store_scope.workspace_id` still stores the physical storage scope, under its
historical column name. Existing Backend placement and snapshot manifest keys
are unchanged. Optional historical `EvidenceOrigin.workspace_id` values continue
to be checked against the physical scope; local provenance need not invent a
Workspace value. `workspace_id()` is a Backend compatibility alias for
`scope_id()`.
## Host operation boundary
`api::execute(store, &HostOperationContext, operation)` is the common dispatcher
used by Backend and local Hosts. It owns query/cursor rules, bounded body and
nested provenance projections, immutable revision reads, proposal validation,
explicit staging/receipt rules, resident context, candidate decisions and
surface generation/publication/failure projections.
The Host context is not serializable or model input. Trusted Rust Hosts construct
it only after validating their external execution authority:
- `HostOperationContext::validated_body` binds one Subject and optionally an
existing immutable `SubjectSessionAttribution`. Explicit staging requires the
request Session to match that attribution.
- `HostOperationContext::validated_consolidation` binds one Subject, an optional
immutable candidate batch, and an optional `JobAttemptBinding`. Attenuated Jobs
must supply `Some(batch)`, including an empty batch; `None` is legacy non-Job
Host consolidation authority. Body-only operations are unavailable to
consolidation contexts, and candidate reads/decisions cannot escape the batch.
These constructors validate store/domain bindings, not external authentication.
Backend retains authenticated Runtime/Worker identity, current singleton and
live Job attempt checks, candidate grant checks, and public Session validation.
Local Hosts retain exclusive Subject execution leases, generic Job admission,
Worker/Session ownership, and committed public Session evidence validation.
Before explicit staging, the Host resolves evidence using its committed public
Session projection; the common API does not inspect Session files, hidden
reasoning, system prompts, or arbitrary paths. Lower-level repository/projection
functions are trusted Rust APIs, not model capabilities.
## Common consolidation Job rules
`job::bounded_candidate_batch(ids, max_result_bytes)` selects the exact T-704
ordered-prefix budget (100 candidates maximum, 2,048-byte envelope, twice each
JSON-escaped ID size plus 128 bytes per disposition).
`job::consolidation_required(count, bytes, force, availability)` owns the shared
trigger thresholds and empty-ready/surface-retry rules.
`job::validate_result(store, subject_id, batch, job_id, attempt_id, value)` checks
the strict result shape, exact admitted Subject/batch, durable dispositions, and
the persisted terminal surface outcome from that attempt. After validation,
`job::with_dispositions(store, subject_id, batch, value)` adds the deterministic
compact disposition projection. Hosts retain Job result binding/admission; these
helpers never run or replay a Job. The store's `prepare_job_surface_generation`,
`require_job_surface_generation`, and `validate_job_surface_outcome` are public
trusted-domain methods for both Backend and local Hosts.
Dependencies: `feature-storage`, `chrono`, `memory`, `manifest`, `rusqlite`,
`serde`, `serde_json`, `server-api`, `thiserror`, `uuid`; tests use `tempfile`.
File diff suppressed because it is too large Load Diff
+155
View File
@@ -0,0 +1,155 @@
//! Trusted Host operation context, never deserialized from model input.
use super::{OperationError, OperationResult, subjektiv_store_error};
use crate::{SubjectSessionAttribution, SubjektivStore};
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct JobAttemptBinding {
pub job_id: String,
pub attempt_id: String,
}
#[derive(Debug, Clone)]
enum Authority {
Body {
attribution: Option<SubjectSessionAttribution>,
},
Consolidation {
candidate_ids: Option<Vec<String>>,
job: Option<JobAttemptBinding>,
},
}
/// A capability issued by trusted Rust Host code after authentication/lease,
/// delegation, and committed Session evidence checks. Not a request DTO.
/// Constructors check domain binding, not external identity or execution leases.
/// Backend must retain its authenticated Worker and active Job grant checks;
/// standalone must hold the selected Subject lease and attenuate Job batches.
#[derive(Debug, Clone)]
pub struct HostOperationContext {
scope_id: String,
subject_id: String,
authority: Authority,
}
impl HostOperationContext {
/// The Host has validated body execution authority. Attribution, when supplied,
/// must already be immutable history in this store. Before StageExplicit the
/// Host must resolve/validate all evidence against committed public Session
/// entries; neither this constructor nor the operation reads Session files.
pub fn validated_body(
store: &SubjektivStore,
subject_id: &str,
attribution: Option<SubjectSessionAttribution>,
) -> OperationResult<Self> {
Self::require_subject(store, subject_id)?;
if let Some(record) = &attribution {
if record.subject_id != subject_id
|| store
.session_attribution(&record.session_id)
.map_err(subjektiv_store_error)?
.as_ref()
!= Some(record)
{
return Err(OperationError::PermissionDenied(
"Session attribution does not match the Host subject".into(),
));
}
}
Ok(Self {
scope_id: store.scope_id().into(),
subject_id: subject_id.into(),
authority: Authority::Body { attribution },
})
}
/// The Host has validated consolidation authority and (for a Job) its active
/// attempt. `Some(candidate_ids)` is the immutable admitted batch; no candidate
/// outside it is accessible. `None` retains Backend's non-Job consolidation
/// authority, and must not be issued to an attenuated Job.
pub fn validated_consolidation(
store: &SubjektivStore,
subject_id: &str,
candidate_ids: Option<Vec<String>>,
job: Option<JobAttemptBinding>,
) -> OperationResult<Self> {
Self::require_subject(store, subject_id)?;
if let Some(ids) = &candidate_ids {
for id in ids {
if store
.staging_candidate(subject_id, id)
.map_err(subjektiv_store_error)?
.is_none()
{
return Err(OperationError::InvalidInput(format!(
"candidate_not_found: {id}"
)));
}
}
}
if job.as_ref().is_some_and(|binding| {
binding.job_id.trim().is_empty() || binding.attempt_id.trim().is_empty()
}) {
return Err(OperationError::InvalidInput(
"Job attempt binding must not be empty".into(),
));
}
Ok(Self {
scope_id: store.scope_id().into(),
subject_id: subject_id.into(),
authority: Authority::Consolidation { candidate_ids, job },
})
}
fn require_subject(store: &SubjektivStore, id: &str) -> OperationResult<()> {
store
.subject(id)
.map_err(subjektiv_store_error)?
.ok_or_else(|| OperationError::SubjectNotFound(id.into()))?;
Ok(())
}
pub fn subject_id(&self) -> &str {
&self.subject_id
}
pub(super) fn require_store(&self, store: &SubjektivStore) -> OperationResult<()> {
if self.scope_id != store.scope_id() {
return Err(OperationError::PermissionDenied(
"Host operation context belongs to another storage scope".into(),
));
}
Ok(())
}
pub(super) fn body(&self) -> OperationResult<Option<&SubjectSessionAttribution>> {
match &self.authority {
Authority::Body { attribution } => Ok(attribution.as_ref()),
_ => Err(OperationError::PermissionDenied(
"operation requires Subject body authority".into(),
)),
}
}
pub(super) fn consolidation(
&self,
) -> OperationResult<(Option<&[String]>, Option<&JobAttemptBinding>)> {
match &self.authority {
Authority::Consolidation { candidate_ids, job } => {
Ok((candidate_ids.as_deref(), job.as_ref()))
}
_ => Err(OperationError::PermissionDenied(
"operation requires consolidation authority".into(),
)),
}
}
pub(super) fn candidate(&self, id: &str) -> OperationResult<()> {
let (ids, _) = self.consolidation()?;
if ids.is_some_and(|ids| !ids.iter().any(|allowed| allowed == id)) {
return Err(OperationError::PermissionDenied(
"candidate is outside the admitted Job batch".into(),
));
}
Ok(())
}
}
+255
View File
@@ -0,0 +1,255 @@
use super::*;
use server_api::{
SubjektivMemoryBackendOperation as Op, SubjektivMemoryBackendResponse as Response,
};
/// Executes the same domain operations and bounded projections for every Host.
/// The caller authenticates and validates evidence before issuing `context`.
pub fn execute(
store: &crate::SubjektivStore,
context: &HostOperationContext,
operation: Op,
) -> OperationResult<Response> {
context.require_store(store)?;
let subject_id = context.subject_id();
Ok(match operation {
Op::ResidentSummary(_) => {
context.body()?;
Response::ResidentSummary(resident_summary_output(
store
.resident_surface(subject_id)
.map_err(subjektiv_store_error)?,
))
}
Op::ResidentContext(_) => {
context.body()?;
let resident = store
.resident_context(subject_id)
.map_err(subjektiv_store_error)?;
Response::ResidentContext(server_api::SubjektivResidentContextOutput {
behavior_md: resident.subject.behavior_md,
behavior_revision: resident.subject.behavior_revision,
memory_surface: resident_summary_output(resident.surface),
})
}
Op::Query(input) => Response::Query(subjektiv_memory_query(store, subject_id, input)?),
Op::Read(input) => Response::Read(subjektiv_memory_read(store, subject_id, input)?),
Op::ListRevisions(input) => {
Response::ListRevisions(subjektiv_memory_list_revisions(store, subject_id, input)?)
}
Op::ValidateProposal(input) => {
context.body()?;
Response::ProposalValidated(subjektiv_memory_validate_proposal(
store, subject_id, input,
)?)
}
Op::ReceiptStatus(input) => {
context.body()?;
validate_subjektiv_receipt_id(&input.receipt_id)?;
let candidate = store
.staging_candidate(subject_id, &input.receipt_id)
.map_err(subjektiv_store_error)?;
Response::ReceiptStatus(server_api::SubjektivMemoryReceiptStatusResponse {
receipt_id: input.receipt_id,
status: if candidate.is_some() {
server_api::SubjektivMemoryReceiptStatus::Staged
} else {
server_api::SubjektivMemoryReceiptStatus::Missing
},
candidate_id: candidate.map(|record| record.id),
})
}
Op::StageExplicit(input) => {
let attribution = context.body()?.ok_or_else(|| {
OperationError::PermissionDenied(
"explicit staging requires validated Session attribution".into(),
)
})?;
if input.session_id != attribution.session_id {
return Err(OperationError::PermissionDenied(
"explicit staging Session does not match Host attribution".into(),
));
}
Response::Staged(subjektiv_memory_stage_explicit(
store,
input,
attribution.clone(),
)?)
}
Op::ListCandidates(input) => {
let (ids, _) = context.consolidation()?;
let limit = bounded_subjektiv_limit(input.limit, SUBJEKTIV_QUERY_DEFAULT_LIMIT)?;
let mut candidates = if let Some(ids) = ids {
let mut pending = Vec::new();
for id in ids {
if store
.staging_resolution(subject_id, id)
.map_err(subjektiv_store_error)?
.is_none()
{
if let Some(record) = store
.staging_candidate(subject_id, id)
.map_err(subjektiv_store_error)?
{
pending.push(record);
}
}
}
pending
} else {
store
.pending_staging_candidates(subject_id, limit.saturating_add(1))
.map_err(subjektiv_store_error)?
};
let has_more = candidates.len() > limit;
candidates.truncate(limit);
Response::Candidates(server_api::SubjektivMemoryCandidateListResponse {
items: candidates
.into_iter()
.map(subjektiv_candidate_summary)
.collect(),
has_more,
})
}
Op::ReadCandidate(input) => {
context.candidate(&input.candidate_id)?;
let candidate = store
.staging_candidate(subject_id, &input.candidate_id)
.map_err(subjektiv_store_error)?
.ok_or_else(|| {
OperationError::InvalidInput(format!(
"candidate_not_found: pending candidate `{}` was not found",
input.candidate_id
))
})?;
if store
.staging_resolution(subject_id, &input.candidate_id)
.map_err(subjektiv_store_error)?
.is_some()
{
return Err(OperationError::Conflict(format!(
"candidate_resolved: candidate `{}` is already resolved",
input.candidate_id
)));
}
Response::Candidate(subjektiv_candidate(candidate))
}
Op::DecideCandidate(input) => {
context.candidate(&input.candidate_id)?;
let receipt = store
.decide_candidate(subject_id, subjektiv_candidate_decision(input)?)
.map_err(subjektiv_store_error)?;
Response::CandidateDecided(subjektiv_candidate_decision_response(receipt))
}
Op::PrepareSurface(_) => {
let (ids, job) = context.consolidation()?;
if let Some(ids) = ids {
for id in ids {
if store
.staging_resolution(subject_id, id)
.map_err(subjektiv_store_error)?
.is_none()
{
return Err(OperationError::InvalidInput(format!(
"surface generation requires a durable disposition for batch candidate `{id}`"
)));
}
}
}
let generation = store
.prepare_job_surface_generation(
subject_id,
job.map(|b| (b.job_id.as_str(), b.attempt_id.as_str())),
)
.map_err(subjektiv_store_error)?;
let resident = store
.resident_surface(subject_id)
.map_err(subjektiv_store_error)?;
let mut response = subjektiv_surface_prepare_response(generation);
if resident.availability == crate::SurfaceAvailability::Ready {
response.current_snapshot_id = resident
.snapshot
.filter(|s| s.built_from_store_revision == response.store_revision)
.map(|s| s.id);
}
Response::SurfacePrepared(response)
}
Op::PublishSurface(input) => {
let (_, job) = context.consolidation()?;
if let Some(job) = job {
store
.require_job_surface_generation(
subject_id,
&input.generation_id,
&job.job_id,
&job.attempt_id,
)
.map_err(subjektiv_store_error)?;
}
let points = input
.points
.into_iter()
.map(|point| crate::SurfacePoint {
body_md: point.body_md,
memory_refs: point
.memory_refs
.into_iter()
.map(|r| crate::MemoryRevisionRef {
memory_id: r.memory_id,
revision: r.revision,
})
.collect(),
})
.collect();
let snapshot = store
.publish_surface_generation(subject_id, &input.generation_id, points)
.map_err(subjektiv_store_error)?;
Response::SurfacePublished(server_api::SubjektivSurfacePublishResponse {
snapshot_id: snapshot.id,
built_from_store_revision: snapshot.built_from_store_revision,
empty: snapshot.body_md.is_empty(),
})
}
Op::FailSurface(input) => {
let (_, job) = context.consolidation()?;
if let Some(job) = job {
store
.require_job_surface_generation(
subject_id,
&input.generation_id,
&job.job_id,
&job.attempt_id,
)
.map_err(subjektiv_store_error)?;
}
let revision = store
.fail_surface_generation(subject_id, &input.generation_id, &input.reason_code)
.map_err(subjektiv_store_error)?;
let status = if let Some(job) = job {
store
.validate_job_surface_outcome(
subject_id,
&input.generation_id,
revision,
"failed",
None,
Some(&input.reason_code),
&job.job_id,
&job.attempt_id,
)
.map_err(|e| {
OperationError::Conflict(format!(
"surface failure was not recorded for this attempt: {e}"
))
})?;
"failed_confirmed"
} else {
"failed"
};
Response::SurfaceFailed(server_api::SubjektivSurfaceFailureResponse {
store_revision: revision,
status: status.into(),
})
}
})
}
+343
View File
@@ -0,0 +1,343 @@
use super::*;
use crate::{MemoryDraft, SubjectRole, SubjectSessionAttribution, SubjektivStore};
use memory::extract::CandidateKind;
use server_api::{
SubjektivMemoryBackendOperation as Op, SubjektivMemoryBackendResponse as Response,
};
fn open(root: &std::path::Path, scope: &str) -> (feature_storage::FeatureStorage, SubjektivStore) {
let manager = feature_storage::FeatureStorage::new(root);
let scope = manager.scope(scope).unwrap();
let registration = SubjektivStore::register(&scope).unwrap();
let store = SubjektivStore::open(&scope, &registration).unwrap();
(manager, store)
}
fn query(cursor: Option<String>) -> Op {
Op::Query(server_api::SubjektivMemoryQueryRequest {
query: None,
kinds: None,
states: None,
limit: Some(1),
cursor,
})
}
fn read(id: &str, revision: Option<u64>, offset: Option<usize>, byte_offset: Option<usize>) -> Op {
Op::Read(server_api::SubjektivMemoryReadRequest {
memory_id: id.into(),
revision,
offset,
byte_offset,
limit: Some(1),
evidence_cursor: None,
})
}
#[test]
fn neutral_scope_reopen_and_context_attenuation() {
let temp = tempfile::tempdir().unwrap();
let random_scope = uuid::Uuid::now_v7().to_string();
let (manager, store) = open(temp.path(), &random_scope);
assert_eq!(store.scope_id(), random_scope);
let subject = store
.create_subject(SubjectRole::new("companion").unwrap())
.unwrap();
let body = HostOperationContext::validated_body(&store, &subject.id, None).unwrap();
let consolidation =
HostOperationContext::validated_consolidation(&store, &subject.id, Some(vec![]), None)
.unwrap();
assert!(
execute(
&store,
&body,
Op::ListCandidates(server_api::SubjektivMemoryCandidateListRequest { limit: None })
)
.is_err()
);
assert!(
execute(
&store,
&consolidation,
Op::ResidentSummary(memory::backend::MemoryResidentSummaryOperation {})
)
.is_err()
);
let attribution =
SubjectSessionAttribution::new(&subject.id, "local-host", "worker", "session").unwrap();
assert!(
HostOperationContext::validated_body(&store, &subject.id, Some(attribution.clone()))
.is_err()
);
store
.record_session_attribution(attribution.clone())
.unwrap();
HostOperationContext::validated_body(&store, &subject.id, Some(attribution)).unwrap();
let (_, foreign) = open(&temp.path().join("foreign"), "foreign-scope");
assert!(matches!(
execute(&foreign, &body, query(None)),
Err(OperationError::PermissionDenied(_))
));
manager.shutdown().unwrap();
drop(store);
let (_manager, reopened) = open(temp.path(), &random_scope);
assert_eq!(reopened.subject(&subject.id).unwrap().unwrap(), subject);
assert!(reopened.session_attribution("session").unwrap().is_some());
}
#[test]
fn shared_query_cursor_and_utf8_fixed_revision_projection() {
let temp = tempfile::tempdir().unwrap();
let (_manager, store) = open(temp.path(), "local-random-scope");
let subject = store
.create_subject(SubjectRole::new("companion").unwrap())
.unwrap();
let context = HostOperationContext::validated_body(&store, &subject.id, None).unwrap();
let original = "日本語\\\"".repeat(10_000);
let first = store
.create_memory(
&subject.id,
MemoryDraft::active(
CandidateKind::Lesson,
"first",
&original,
"reason",
"initial",
),
)
.unwrap();
store
.create_memory(
&subject.id,
MemoryDraft::active(
CandidateKind::Decision,
"second",
"second body",
"reason",
"initial",
),
)
.unwrap();
let Response::Query(page) = execute(&store, &context, query(None)).unwrap() else {
panic!()
};
assert!(page.has_more);
assert_eq!(page.items.len(), 1);
let cursor = page.next_cursor.unwrap();
store
.create_memory(
&subject.id,
MemoryDraft::active(
CandidateKind::Constraint,
"third",
"third body",
"reason",
"initial",
),
)
.unwrap();
assert!(matches!(
execute(&store, &context, query(Some(cursor))),
Err(OperationError::Conflict(_))
));
assert!(execute(&store, &context, read(&first.id, None, Some(1), None)).is_err());
let mut offset = None;
let mut byte_offset = None;
let mut reconstructed = String::new();
loop {
let Response::Read(page) = execute(
&store,
&context,
read(&first.id, Some(1), offset, byte_offset),
)
.unwrap() else {
panic!()
};
assert!(
serde_json::to_vec_pretty(&page).unwrap().len()
<= server_api::SUBJEKTIV_MEMORY_READ_MAX_TOOL_CONTENT_BYTES
);
reconstructed.push_str(&page.body_md);
if !page.body_truncated {
break;
}
assert!(!page.body_md.is_empty());
offset = page.body_next_offset;
byte_offset = page.body_next_byte_offset;
}
assert_eq!(reconstructed, original);
}
#[test]
fn shared_surface_empty_snapshot_is_ready_not_missing() {
let temp = tempfile::tempdir().unwrap();
let (_manager, store) = open(temp.path(), "local-random-scope");
let subject = store
.create_subject(SubjectRole::new("companion").unwrap())
.unwrap();
let consolidation = HostOperationContext::validated_consolidation(
&store,
&subject.id,
Some(vec![]),
Some(JobAttemptBinding {
job_id: "job".into(),
attempt_id: "attempt".into(),
}),
)
.unwrap();
let prepared: server_api::SubjektivSurfacePrepareRequest = serde_json::from_str("{}").unwrap();
let Response::SurfacePrepared(prepared) =
execute(&store, &consolidation, Op::PrepareSurface(prepared)).unwrap()
else {
panic!()
};
let Response::SurfacePublished(published) = execute(
&store,
&consolidation,
Op::PublishSurface(server_api::SubjektivSurfacePublishRequest {
generation_id: prepared.generation_id,
points: vec![],
}),
)
.unwrap() else {
panic!()
};
assert!(published.empty);
let body = HostOperationContext::validated_body(&store, &subject.id, None).unwrap();
let Response::ResidentSummary(resident) = execute(
&store,
&body,
Op::ResidentSummary(memory::backend::MemoryResidentSummaryOperation {}),
)
.unwrap() else {
panic!()
};
assert_eq!(
resident.availability,
memory::backend::MemoryResidentSummaryAvailability::Ready
);
assert_eq!(resident.content, None);
}
#[test]
fn explicit_receipts_candidate_batch_and_decision_retries_share_logic() {
let temp = tempfile::tempdir().unwrap();
let (_manager, store) = open(temp.path(), "random-local-scope");
let subject = store
.create_subject(SubjectRole::new("companion").unwrap())
.unwrap();
let attribution =
SubjectSessionAttribution::new(&subject.id, "host", "worker", "session").unwrap();
store
.record_session_attribution(attribution.clone())
.unwrap();
let context =
HostOperationContext::validated_body(&store, &subject.id, Some(attribution)).unwrap();
// Host-resolved public Session evidence with no invented Workspace identity.
let input = server_api::SubjektivMemoryStageExplicitRequest {
receipt_id: "receipt-1".into(),
session_id: "session".into(),
kind: CandidateKind::Lesson,
claim: "Preserve correction history".into(),
why_useful: "Allows auditing".into(),
staleness: None,
proposal: None,
evidence: vec![memory::extract::StagingEvidence {
id: "e1".into(),
entry_range: Some([1, 1]),
kind: memory::schema::EvidenceKind::new("model_output"),
origin: None,
excerpt: Some("Use immutable revisions".into()),
summary: None,
}],
source_refs: vec![memory::schema::SourceEvidenceRef {
session_id: Some("session".into()),
segment_id: Some("segment".into()),
entry_range: Some([1, 1]),
evidence_id: Some("e1".into()),
..Default::default()
}],
};
let mut foreign_session = input.clone();
foreign_session.session_id = "foreign".into();
assert!(matches!(
execute(&store, &context, Op::StageExplicit(foreign_session)),
Err(OperationError::PermissionDenied(_))
));
execute(&store, &context, Op::StageExplicit(input.clone())).unwrap();
execute(&store, &context, Op::StageExplicit(input)).unwrap();
let Response::ReceiptStatus(receipt) = execute(
&store,
&context,
Op::ReceiptStatus(server_api::SubjektivMemoryReceiptStatusRequest {
receipt_id: "receipt-1".into(),
}),
)
.unwrap() else {
panic!()
};
assert_eq!(
receipt.status,
server_api::SubjektivMemoryReceiptStatus::Staged
);
let candidate_id = receipt.candidate_id.unwrap();
let no_batch =
HostOperationContext::validated_consolidation(&store, &subject.id, Some(vec![]), None)
.unwrap();
assert!(matches!(
execute(
&store,
&no_batch,
Op::ReadCandidate(server_api::SubjektivMemoryCandidateReadRequest {
candidate_id: candidate_id.clone()
})
),
Err(OperationError::PermissionDenied(_))
));
let batch = HostOperationContext::validated_consolidation(
&store,
&subject.id,
Some(vec![candidate_id.clone()]),
None,
)
.unwrap();
assert!(
execute(
&store,
&batch,
Op::PrepareSurface(serde_json::from_str("{}").unwrap())
)
.is_err()
);
let decision = server_api::SubjektivMemoryCandidateDecisionRequest {
request_id: "decision-1".into(),
candidate_id,
reason: "adopt lesson".into(),
decision: server_api::SubjektivMemoryCandidateDecision::Apply {
target: server_api::SubjektivMemoryApplyTarget::Create,
memory: server_api::SubjektivMemoryDraft {
kind: CandidateKind::Lesson,
state: server_api::SubjektivMemoryState::Active,
claim: "Preserve correction history".into(),
body_md: "Use immutable revisions".into(),
why_useful: "Allows auditing".into(),
staleness: None,
derived_from: vec![],
change_reason: "adopt".into(),
},
},
};
let first = execute(&store, &batch, Op::DecideCandidate(decision.clone())).unwrap();
let retry = execute(&store, &batch, Op::DecideCandidate(decision)).unwrap();
assert_eq!(
serde_json::to_value(first).unwrap(),
serde_json::to_value(retry).unwrap()
);
assert_eq!(store.list_memories(&subject.id).unwrap().len(), 1);
execute(
&store,
&batch,
Op::PrepareSurface(serde_json::from_str("{}").unwrap()),
)
.unwrap();
}
+153
View File
@@ -0,0 +1,153 @@
//! Common consolidation Job policy and verified result projections.
//!
//! Hosts authenticate/admit Jobs and bind their immutable batch and attempt.
//! These helpers do not dispatch, replay, or grant Job execution authority.
use serde::Deserialize;
use crate::api::{OperationError, OperationResult};
use crate::{SubjektivStore, SurfaceAvailability};
pub const CANDIDATE_BATCH_LIMIT: usize = 100;
pub const CONSOLIDATION_THRESHOLD_FILES: usize = 5;
pub const CONSOLIDATION_THRESHOLD_BYTES: u64 = 50_000;
/// Selects an ordered prefix using the T-704 complete-result budget. Reserve
/// 2,048 bytes for the Subject/Host surface envelope, then twice the JSON-escaped
/// ID size plus 128 bytes per candidate for its compact verified disposition.
pub fn bounded_candidate_batch(
ids: impl IntoIterator<Item = String>,
max_result_bytes: u32,
) -> OperationResult<Vec<String>> {
let mut estimated_bytes = 2_048_usize;
let mut selected = Vec::new();
for id in ids.into_iter().take(CANDIDATE_BATCH_LIMIT) {
let id_bytes = serde_json::to_vec(&id)
.map_err(|e| OperationError::InvalidInput(e.to_string()))?
.len();
let additional = 2 * id_bytes + 128;
if estimated_bytes + additional > max_result_bytes as usize {
break;
}
estimated_bytes += additional;
selected.push(id);
}
Ok(selected)
}
/// Empty ready Subjects need no work, even when forced. Missing/stale/failed
/// surfaces bypass candidate thresholds; confirmed Memory is never discarded.
pub fn consolidation_required(
candidate_count: usize,
total_bytes: u64,
force: bool,
surface_availability: SurfaceAvailability,
) -> bool {
let surface_needs_generation = surface_availability != SurfaceAvailability::Ready;
if candidate_count == 0 {
return surface_needs_generation;
}
force
|| surface_needs_generation
|| candidate_count >= CONSOLIDATION_THRESHOLD_FILES
|| total_bytes >= CONSOLIDATION_THRESHOLD_BYTES
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct ConsolidationResult {
subject_id: String,
candidate_ids: Vec<String>,
surface: SurfaceResult,
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct SurfaceResult {
availability: String,
generation_id: String,
store_revision: u64,
#[serde(default)]
snapshot_id: Option<String>,
#[serde(default)]
reason_code: Option<String>,
}
/// Verifies the exact Subject/batch, every immutable disposition, and the actual
/// terminal surface outcome begun by this Job attempt. Model claims alone never
/// establish completion. Same admission semantics as the Backend T-704 adapter.
pub fn validate_result(
store: &SubjektivStore,
subject_id: &str,
batch: &[String],
job_id: &str,
attempt_id: &str,
value: &serde_json::Value,
) -> OperationResult<()> {
let result: ConsolidationResult = serde_json::from_value(value.clone())
.map_err(|e| OperationError::InvalidInput(format!("invalid consolidation result: {e}")))?;
if result.subject_id != subject_id || result.candidate_ids != batch {
return Err(OperationError::InvalidInput(
"consolidation result does not match the immutable Subject/batch grant".into(),
));
}
for candidate_id in batch {
if store
.staging_resolution(subject_id, candidate_id)
.map_err(|e| OperationError::Store(e.to_string()))?
.is_none()
{
return Err(OperationError::InvalidInput(format!(
"consolidation candidate `{candidate_id}` is unresolved"
)));
}
}
store
.validate_job_surface_outcome(
subject_id,
&result.surface.generation_id,
result.surface.store_revision,
&result.surface.availability,
result.surface.snapshot_id.as_deref(),
result.surface.reason_code.as_deref(),
job_id,
attempt_id,
)
.map_err(|e| {
OperationError::InvalidInput(format!("consolidation surface result mismatch: {e}"))
})
}
/// Enriches a previously verified result with compact durable dispositions. Full
/// reasons, affected revisions and provenance remain in the receipt store. This
/// deterministic projection neither applies candidates nor creates a new ledger.
pub fn with_dispositions(
store: &SubjektivStore,
subject_id: &str,
batch: &[String],
value: &serde_json::Value,
) -> OperationResult<serde_json::Value> {
let mut dispositions = Vec::new();
for id in batch {
let resolution = store
.staging_resolution(subject_id, id)
.map_err(|e| OperationError::Store(e.to_string()))?
.ok_or_else(|| {
OperationError::InvalidInput(format!("candidate `{id}` has no durable disposition"))
})?;
dispositions.push(serde_json::json!({"candidate_id": id, "action": resolution.action}));
}
let mut enriched = value.clone();
enriched
.as_object_mut()
.ok_or_else(|| {
OperationError::InvalidInput("consolidation result must be an object".into())
})?
.insert(
"candidate_dispositions".into(),
serde_json::Value::Array(dispositions),
);
Ok(enriched)
}
#[cfg(test)]
mod tests;
+183
View File
@@ -0,0 +1,183 @@
use super::*;
use crate::{
CandidateDecision, CandidateDecisionRequest, StagingResolutionAction, SubjectRole,
SubjectStagingRecord,
};
use memory::extract::{CandidateKind, ExtractedCandidate, StagingRecord};
use memory::schema::SourceRef;
#[test]
fn escaped_ids_use_exact_t704_complete_result_budget() {
const MAX_RESULT_BYTES: u32 = 16 * 1024;
let ids: Vec<_> = (0..100)
.map(|i| format!("{i:03}{}", "\\\"".repeat(126)))
.collect();
let batch = bounded_candidate_batch(ids.clone(), MAX_RESULT_BYTES).unwrap();
assert!(!batch.is_empty());
assert!(batch.len() < ids.len());
assert_eq!(batch, ids[..batch.len()]);
let dispositions: Vec<_> = batch
.iter()
.map(|id| serde_json::json!({"candidate_id": id, "action": "already_covered"}))
.collect();
let result = serde_json::json!({
"subject_id": "\\\"".repeat(128), "candidate_ids": batch, "candidate_dispositions": dispositions,
"surface": { "availability": "failed", "generation_id": "g".repeat(100), "store_revision": u64::MAX, "reason_code": "\\\"".repeat(128) }
});
assert!(serde_json::to_vec(&result).unwrap().len() <= MAX_RESULT_BYTES as usize);
assert!(
bounded_candidate_batch(["one".into()], 2048)
.unwrap()
.is_empty()
);
assert_eq!(
bounded_candidate_batch((0..200).map(|i| i.to_string()), u32::MAX)
.unwrap()
.len(),
CANDIDATE_BATCH_LIMIT
);
}
#[test]
fn trigger_policy_preserves_empty_ready_and_surface_retry_rules() {
for availability in [
SurfaceAvailability::Ungenerated,
SurfaceAvailability::Stale,
SurfaceAvailability::Failed,
] {
assert!(consolidation_required(0, 0, false, availability));
assert!(consolidation_required(1, 1, false, availability));
}
assert!(!consolidation_required(
0,
0,
true,
SurfaceAvailability::Ready
));
assert!(!consolidation_required(
4,
49_999,
false,
SurfaceAvailability::Ready
));
assert!(consolidation_required(
5,
0,
false,
SurfaceAvailability::Ready
));
assert!(consolidation_required(
1,
50_000,
false,
SurfaceAvailability::Ready
));
assert!(consolidation_required(
1,
1,
true,
SurfaceAvailability::Ready
));
}
#[test]
fn result_checks_durable_dispositions_and_exact_job_surface_outcome() {
let temp = tempfile::tempdir().unwrap();
let manager = feature_storage::FeatureStorage::new(temp.path());
let scope = manager.scope("local-random-scope").unwrap();
let registration = SubjektivStore::register(&scope).unwrap();
let store = SubjektivStore::open(&scope, &registration).unwrap();
let subject = store
.create_subject(SubjectRole::new("companion").unwrap())
.unwrap();
let candidate = SubjectStagingRecord::attach(
&subject.id,
StagingRecord::from_candidate(
"candidate",
"dedup",
SourceRef {
segment_id: "segment".into(),
range: [1, 1],
},
ExtractedCandidate {
kind: CandidateKind::Lesson,
claim: "claim".into(),
why_useful: "reason".into(),
staleness: None,
evidence_ids: vec![],
},
vec![],
vec![],
),
);
store.stage_candidate(candidate).unwrap();
let batch = vec!["candidate".to_string()];
let generation = store
.prepare_job_surface_generation(&subject.id, Some(("job", "attempt")))
.unwrap();
let snapshot = store
.publish_surface_generation(&subject.id, &generation.id, vec![])
.unwrap();
let ready = serde_json::json!({"subject_id": subject.id, "candidate_ids": batch,
"surface": {"availability": "ready", "generation_id": generation.id, "store_revision": snapshot.built_from_store_revision, "snapshot_id": snapshot.id}});
assert!(
validate_result(&store, &subject.id, &batch, "job", "attempt", &ready)
.unwrap_err()
.to_string()
.contains("unresolved")
);
assert!(with_dispositions(&store, &subject.id, &batch, &ready).is_err());
store
.decide_candidate(
&subject.id,
CandidateDecisionRequest {
request_id: "decision".into(),
candidate_id: "candidate".into(),
reason: "invalid".into(),
decision: CandidateDecision::Close {
action: StagingResolutionAction::Invalid,
affected_memory: vec![],
},
},
)
.unwrap();
validate_result(&store, &subject.id, &batch, "job", "attempt", &ready).unwrap();
let enriched = with_dispositions(&store, &subject.id, &batch, &ready).unwrap();
assert_eq!(
enriched["candidate_dispositions"],
serde_json::json!([{"candidate_id": "candidate", "action": "invalid"}])
);
assert_eq!(
enriched,
with_dispositions(&store, &subject.id, &batch, &ready).unwrap()
);
assert!(validate_result(&store, &subject.id, &batch, "job", "other-attempt", &ready).is_err());
assert!(validate_result(&store, &subject.id, &[], "job", "attempt", &ready).is_err());
// A current ready snapshot survives a redundant editor failure. Advance
// Memory first so this attempt genuinely needs a new surface and records failure.
store
.create_memory(
&subject.id,
crate::MemoryDraft::active(
CandidateKind::Lesson,
"new lesson",
"new body",
"reason",
"change",
),
)
.unwrap();
let failure = store
.prepare_job_surface_generation(&subject.id, Some(("job", "attempt")))
.unwrap();
let revision = store
.fail_surface_generation(&subject.id, &failure.id, "model_failure")
.unwrap();
let failed = serde_json::json!({"subject_id": subject.id, "candidate_ids": batch,
"surface": {"availability": "failed", "generation_id": failure.id, "store_revision": revision, "reason_code": "model_failure"}});
validate_result(&store, &subject.id, &batch, "job", "attempt", &failed).unwrap();
assert!(validate_result(&store, &subject.id, &batch, "job", "attempt", &ready).is_err());
let mut false_reason = failed;
false_reason["surface"]["reason_code"] = serde_json::json!("invented");
assert!(validate_result(&store, &subject.id, &batch, "job", "attempt", &false_reason).is_err());
}
File diff suppressed because it is too large Load Diff
+19 -3
View File
@@ -1545,11 +1545,15 @@ fn model_ticket_reference(
) -> Result<String, ToolError> {
let ticket = backend
.show(TicketIdOrSlug::Id(reference.to_string()))
.map_err(|error| backend_error(tool_name, error))?;
.map_err(|error| {
ToolError::Internal(format!(
"{tool_name} outcome unknown after mutation while resolving Ticket reference: {error}"
))
})?;
match ticket.meta.resource_key {
Some(resource_key) if is_canonical_ticket_resource_key(&resource_key) => Ok(resource_key),
Some(_) => Err(ToolError::ExecutionFailed(format!(
"{tool_name} failed: required Ticket key is unavailable"
Some(_) => Err(ToolError::Internal(format!(
"{tool_name} outcome unknown after mutation: required Ticket key is unavailable"
))),
None => Ok(ticket.meta.id),
}
@@ -1998,6 +2002,18 @@ mod tests {
.expect("tool exists")
}
#[test]
fn post_mutation_reference_failures_are_outcome_unknown() {
let temp = TempDir::new().unwrap();
let backend = TicketToolBackend::new(backend(&temp));
let error = model_ticket_reference(&backend, "missing", "TicketClose").unwrap_err();
assert!(matches!(
error,
ToolError::Internal(message)
if message.contains("outcome unknown after mutation")
));
}
#[test]
fn ticket_tool_name_partitions_are_explicit() {
assert_eq!(
+53
View File
@@ -24,6 +24,59 @@ Does not own:
A tool implementation must assume model input is untrusted. Permission policy, scope checks, output bounding, and redaction are part of the safety boundary, not optional UI behavior.
## Checkout-native processing
`execute_checkout_tool` (also exported through `tools::checkout`) accepts the
original `WorkdirSessionRouter`, shared `Tracker`, route-bound alias/generation,
logical target and provider validator. It does not create an attachment or router.
The Host owns native publication and permission identity; provider scope and
capabilities remain authoritative. `Create` uses the `Write` capability/identity.
| Operation | Arguments (only these keys accepted) | Target |
| --- | --- | --- |
| Read | `offset?`, `limit?` | Existing file; same numbered lines, defaults and byte bound as Tool Read |
| Edit | `old_string`, `new_string`, `replace_all?` | Existing file; prior read/hash and provider validator required |
| Write | `content` | Existing file only; prior read/hash and provider validator required |
| Create | `path`, `content` | Bound parent directory; relative destination strictly below it; create-new, missing parents allowed |
| Glob | `pattern`, `path?` | Bound directory; optional path relative beneath it |
| Grep | `pattern`, `path?`, `glob?`, `type?`, `case_insensitive?`, `-B?`, `-A?`, `-C?`, `multiline?`, `output_mode?`, `head_limit?`, `offset?` | Bound directory; same provider search/options/rendering as Tool Grep |
Native arguments cannot override `file_path`, `target_workdir`, generation or
validator. Absolute/parent-traversing directory-relative paths are rejected.
Checkout file calls share Tool processing through an optional checked `FileTarget`
and use `checkout_execute`, never a separate Worker-side filesystem engine.
Read records the provider's full content hash **and total line count**, including
partial reads, in the original alias/generation namespace. Normal and native
observations/mutations interoperate. Only successful mutations update tracking.
Glob/Grep share normal request defaults/options and rendering through `SearchTarget`.
Only native dispatch uses `CheckoutSearchRequest::new` and `checkout_search`.
The constructor's empty scope layers/root output are refined by provider wrappers;
Tools does not synthesize authority or parse/rebase rendered filenames. Ordinary
Tool dispatch remains `session.glob`/`session.grep`, with its existing error mapping.
`CheckoutToolOutput.paths` contains typed checkout-root-relative search paths
(from `GlobResult.paths` / `GrepResult.paths`) and the created path for Create.
Rendered Grep text is never parsed to construct links. The optional validator is
the provider's returned observation validator, unchanged. Create's result
observation describes the bound parent target, while returned paths and tracker
updates describe the destination. Search checks the bound observation before and
after the bounded checkout provider search, but does **not** guarantee an atomic tree snapshot
or prevent external edits to descendants. File validator/hash checks are inside
the provider's checked execution boundary. Checked native provider conflicts and
native generation/search fences use `ToolError::StructuredConflict` with code
`checkout_stale`; ordinary Tool conflicts retain `ExecutionFailed`. Checked-only
known provider refusals use `checkout_denied` (scope/capability/read-only policy),
`checkout_invalid` (invalid arguments/paths/target kind/search syntax), or
`checkout_unavailable` (closed session/unavailable capacity). A disappeared bound
target is `checkout_stale`. These codes preserve known no-effect rejection for the
Host; ordinary Tool error mappings are unchanged. Ambiguous transport/IO and
operation failures remain `ExecutionFailed`, as does explicit provider
`OutcomeUnknown`; a failure after mutation must not
be automatically retried. A malformed post-mutation provider result is likewise
reported explicitly as outcome unknown. Explicit mismatched Read/Glob/Grep results
are readonly failures classified as `checkout_unavailable`, not stale or unknown.
## See also
- [`../../docs/design/tool-permissions-scope.md`](../../docs/design/tool-permissions-scope.md)
+258
View File
@@ -0,0 +1,258 @@
//! Native checkout entrypoint using the same processing as ordinary filesystem Tools.
//!
//! The Host binds alias, generation, path and validator from the checkout route; none
//! can be overridden by Operation arguments. File reads/mutations dispatch through
//! the provider's checked execution boundary. Search shares normal request and
//! rendering processing but dispatches through checkout_search so provider scope
//! layers and output rebasing are preserved; checks bracket the bound directory
//! observation, not an atomic snapshot of the searched tree.
use std::sync::Arc;
use agen::tool::{ToolError, ToolExecutionContext, ToolOutput};
use serde::de::DeserializeOwned;
use serde_json::Value;
use workdir::{WorkdirPath, WorkdirSessionCapability, WorkdirSessionRouter};
use crate::{ToolsError, Tracker, file_target::FileTarget};
pub struct CheckoutToolOutput {
pub output: ToolOutput,
pub paths: Vec<WorkdirPath>,
pub validator: Option<Vec<u8>>,
}
/// Execute an exact named native Operation on a route-bound checkout target.
/// `Create.path` and search `path` are relative to the bound directory; returned
/// paths are typed checkout-root-relative provider paths, never parsed output.
#[allow(clippy::too_many_arguments)]
pub async fn execute_checkout_tool(
router: Arc<WorkdirSessionRouter>,
tracker: Tracker,
alias: &str,
generation: u64,
path: WorkdirPath,
validator: Vec<u8>,
tool_name: &str,
arguments: Value,
ctx: ToolExecutionContext,
) -> Result<CheckoutToolOutput, ToolError> {
// Native file arguments come from the same definitions used by ordinary
// Tools and both WIP projections. Route fields are intentionally not in them.
let file_schema = fs_operation::text::TextOperation::from_name(&tool_name.to_lowercase(), true)
.map(|operation| operation.argument_schema());
let file_fields: Vec<&str> = file_schema
.as_ref()
.and_then(|schema| schema.as_value()["properties"].as_object())
.map(|properties| properties.keys().map(String::as_str).collect())
.unwrap_or_default();
let (capability, allowed): (_, &[&str]) = match tool_name {
"Read" => (WorkdirSessionCapability::Read, &file_fields),
"Edit" => (WorkdirSessionCapability::Edit, &file_fields),
"Write" => (WorkdirSessionCapability::Write, &file_fields),
"Create" => (WorkdirSessionCapability::Write, &["path", "content"]),
"Glob" => (WorkdirSessionCapability::Glob, &["pattern", "path"]),
"Grep" => (
WorkdirSessionCapability::Grep,
&[
"pattern",
"path",
"glob",
"type",
"case_insensitive",
"-B",
"-A",
"-C",
"multiline",
"output_mode",
"head_limit",
"offset",
],
),
_ => {
return Err(ToolError::InvalidArgument(format!(
"unknown checkout Operation: {tool_name}"
)));
}
};
let mut arguments = arguments
.as_object()
.cloned()
.ok_or_else(|| ToolError::InvalidArgument("checkout arguments must be an object".into()))?;
if let Some(key) = arguments
.keys()
.find(|key| !allowed.contains(&key.as_str()))
{
return Err(ToolError::InvalidArgument(format!(
"unknown or route override checkout argument: {key}"
)));
}
// Do not rebuild a singleton router: this is the original router identity and
// tracker namespace shared with normal Tool calls and its detach/reuse fence.
let selected = crate::routing::resolve_session(&router, Some(alias), capability)?;
if selected.generation != generation {
return Err(stale_checkout(
"checkout attachment generation changed; resolve the checkout again",
));
}
// Checkout paths are always logical and cannot use Tool-mode absolute scope.
let path = WorkdirPath::new(path.as_str()).map_err(ToolsError::from)?;
let tracker = tracker.scoped_attachment(&selected.alias, selected.generation);
let target = FileTarget {
session: selected.session.clone(),
path: path.clone(),
validator: Some(validator.clone()),
};
match tool_name {
"Read" | "Edit" | "Write" => {
arguments.insert("file_path".into(), Value::String(path.to_string()));
let arguments = Value::Object(arguments);
match tool_name {
"Read" => {
crate::read::execute_read(target, tracker, decode(arguments, tool_name)?).await
}
"Edit" => {
crate::edit::execute_edit(target, tracker, decode(arguments, tool_name)?, ctx)
.await
}
_ => {
let params: crate::write::WriteParams = decode(arguments, tool_name)?;
crate::write::execute_write(target, tracker, params.write.content, None, ctx)
.await
}
}
}
"Create" => {
#[derive(serde::Deserialize)]
struct CreateParams {
path: String,
content: String,
}
let params: CreateParams = decode(Value::Object(arguments), tool_name)?;
let destination = relative_target(&path, &params.path, true)?;
crate::write::execute_write(target, tracker, params.content, Some(destination), ctx)
.await
}
"Glob" | "Grep" => {
let before = selected
.session
.checkout_observe(path.clone())
.await
.map_err(crate::file_target::checked_error)?;
if before.validator != validator {
return Err(stale_search());
}
let mut result = if tool_name == "Glob" {
let params: crate::glob::GlobParams = decode(Value::Object(arguments), tool_name)?;
check_pattern(&params.pattern)?;
let search_path =
relative_target(&path, params.path.as_deref().unwrap_or("."), false)?;
crate::glob::execute_glob(
crate::search_target::SearchTarget::checkout(selected.session.clone()),
search_path,
params.pattern,
)
.await?
} else {
let params: crate::grep::GrepParams = decode(Value::Object(arguments), tool_name)?;
if let Some(glob) = &params.glob {
check_pattern(glob)?;
}
let search_path =
relative_target(&path, params.path.as_deref().unwrap_or("."), false)?;
crate::grep::execute_grep(
crate::search_target::SearchTarget::checkout(selected.session.clone()),
search_path,
params,
)
.await?
};
// Reject rather than emit a link/rendered path outside the binding.
// Provider scope is still authoritative for both traversal and content.
for result_path in &result.paths {
let logical = WorkdirPath::new(result_path.as_str()).map_err(ToolsError::from)?;
if !is_beneath(&path, &logical) {
return Err(crate::file_target::readonly_result_error(
"search paths escaped bound directory",
));
}
}
let after = selected
.session
.checkout_observe(path)
.await
.map_err(crate::file_target::checked_error)?;
if after.validator != before.validator {
return Err(stale_search());
}
result.validator = Some(after.validator);
Ok(result)
}
_ => unreachable!("Operation names checked above"),
}
}
fn decode<T: DeserializeOwned>(arguments: Value, name: &str) -> Result<T, ToolError> {
crate::error::decode_file_value(arguments, name)
}
fn stale_search() -> ToolError {
stale_checkout("checkout directory changed; resolve it again before searching")
}
pub(crate) fn stale_checkout(message: impl Into<String>) -> ToolError {
ToolError::StructuredConflict {
code: "checkout_stale".into(),
message: message.into(),
}
}
fn is_beneath(base: &WorkdirPath, path: &WorkdirPath) -> bool {
base == &WorkdirPath::root()
|| std::path::Path::new(path.as_str()).starts_with(std::path::Path::new(base.as_str()))
}
fn relative_target(
base: &WorkdirPath,
relative: &str,
strict: bool,
) -> Result<WorkdirPath, ToolError> {
if relative.is_empty()
|| relative.starts_with('/')
|| relative.contains('\\')
|| relative.split('/').any(|part| part == "..")
{
return Err(ToolError::InvalidArgument(
"path must be relative beneath the bound checkout directory, without parent traversal"
.into(),
));
}
let relative = WorkdirPath::new(relative).map_err(ToolsError::from)?;
let joined = if relative == WorkdirPath::root() {
base.clone()
} else if base == &WorkdirPath::root() {
relative
} else {
WorkdirPath::new(&format!("{}/{}", base.as_str(), relative.as_str()))
.map_err(ToolsError::from)?
};
if !is_beneath(base, &joined) || (strict && joined == *base) {
return Err(ToolError::InvalidArgument(
"destination must be strictly beneath the bound checkout directory".into(),
));
}
Ok(joined)
}
fn check_pattern(pattern: &str) -> Result<(), ToolError> {
// Patterns filter descendants, not an alternate filesystem starting point.
if pattern.starts_with('/') || pattern.split('/').any(|part| part == "..") {
return Err(ToolError::InvalidArgument(
"search glob must stay beneath the bound checkout directory".into(),
));
}
Ok(())
}
#[cfg(test)]
#[path = "checkout_tests.rs"]
mod tests;
+562
View File
@@ -0,0 +1,562 @@
use super::*;
use manifest::Scope;
use serde_json::json;
use tempfile::TempDir;
use workdir::{LocalWorkdirSession, WorkdirAttachmentAlias};
struct Fixture {
dir: TempDir,
router: Arc<WorkdirSessionRouter>,
tracker: Tracker,
}
impl Fixture {
fn new() -> Self {
let dir = TempDir::new().unwrap();
let router = Arc::new(WorkdirSessionRouter::new());
let fixture = Self {
dir,
router,
tracker: Tracker::new(),
};
fixture.attach("main");
fixture
}
fn attach(&self, alias: &str) {
self.router
.attach(
WorkdirAttachmentAlias::new(alias).unwrap(),
Arc::new(LocalWorkdirSession::new(
Scope::writable(self.dir.path()).unwrap(),
self.dir.path().to_path_buf(),
)),
)
.unwrap();
}
async fn observation(&self, alias: &str, path: &str) -> (u64, Vec<u8>) {
let selected = self.router.resolve(Some(alias)).unwrap();
let observation = selected
.session
.checkout_observe(WorkdirPath::new(path).unwrap())
.await
.unwrap();
(selected.generation, observation.validator)
}
async fn native(
&self,
alias: &str,
path: &str,
name: &str,
arguments: Value,
) -> Result<CheckoutToolOutput, ToolError> {
let (generation, validator) = self.observation(alias, path).await;
execute_checkout_tool(
self.router.clone(),
self.tracker.clone(),
alias,
generation,
WorkdirPath::new(path).unwrap(),
validator,
name,
arguments,
Default::default(),
)
.await
}
async fn normal(&self, name: &str, arguments: Value) -> Result<ToolOutput, ToolError> {
let definitions = crate::routed_builtin_tools(
self.router.clone(),
self.tracker.clone(),
Default::default(),
);
for definition in definitions {
let (meta, tool) = definition();
if meta.name == name {
return tool
.execute(&arguments.to_string(), Default::default())
.await;
}
}
panic!("missing Tool {name}")
}
}
fn assert_checkout_stale(error: ToolError) {
assert_checkout_code(error, "checkout_stale");
}
fn assert_checkout_code(error: ToolError, expected: &str) {
match error {
ToolError::StructuredConflict { code, .. } => assert_eq!(code, expected),
other => panic!("expected {expected} structured conflict, got {other:?}"),
}
}
#[test]
fn checkout_known_refusals_have_typed_codes_without_changing_normal_tools() {
use workdir::{WorkdirError, WorkdirSessionCapability};
let cases = [
(WorkdirError::Denied("denied".into()), "checkout_denied"),
(
WorkdirError::OutOfScope("outside".into()),
"checkout_denied",
),
(
WorkdirError::ReadOnly("read only".into()),
"checkout_denied",
),
(
WorkdirError::Unsupported(WorkdirSessionCapability::Write),
"checkout_denied",
),
(
WorkdirError::InvalidArgument("bad argument".into()),
"checkout_invalid",
),
(
WorkdirError::InvalidPath("bad path".into()),
"checkout_invalid",
),
(
WorkdirError::Unavailable("budget exhausted".into()),
"checkout_unavailable",
),
(WorkdirError::NotFound("missing".into()), "checkout_stale"),
(WorkdirError::SessionClosed, "checkout_unavailable"),
(
WorkdirError::UnsupportedOperation("unsupported".into()),
"checkout_denied",
),
(
WorkdirError::UnknownCommand("missing command".into()),
"checkout_invalid",
),
(
WorkdirError::RelativePath("relative".into()),
"checkout_invalid",
),
(
WorkdirError::IsDirectory("directory".into()),
"checkout_invalid",
),
(
WorkdirError::InvalidGlob("bad glob".into()),
"checkout_invalid",
),
(
WorkdirError::InvalidRegex("bad regex".into()),
"checkout_invalid",
),
];
for (error, code) in cases {
assert_checkout_code(crate::file_target::checked_error(error), code);
}
assert!(matches!(
ToolError::from(ToolsError::from(WorkdirError::Unavailable(
"unavailable".into()
))),
ToolError::ExecutionFailed(_)
));
assert!(matches!(
ToolError::from(ToolsError::from(WorkdirError::Unsupported(
WorkdirSessionCapability::Write
))),
ToolError::InvalidArgument(_)
));
}
#[test]
fn checkout_conflict_classification_preserves_normal_tools_and_unknown_outcomes() {
assert_checkout_stale(crate::file_target::checked_error(
workdir::WorkdirError::Conflict("a.txt".into()),
));
assert_checkout_stale(stale_search());
assert!(matches!(
ToolError::from(ToolsError::from(workdir::WorkdirError::Conflict(
"a.txt".into()
))),
ToolError::ExecutionFailed(_)
));
assert!(matches!(
crate::file_target::checked_error(workdir::WorkdirError::OutcomeUnknown(
"may have committed".into()
)),
ToolError::ExecutionFailed(_)
));
}
#[test]
fn checkout_ambiguous_transport_and_operation_failures_remain_unknown() {
use workdir::WorkdirError;
// Even refusal-looking text in an ambiguous transport error is not proof
// that the provider rejected the request before any effects.
for error in [
WorkdirError::Transport("denied (ambiguous response)".into()),
WorkdirError::OperationFailed,
] {
assert!(matches!(
crate::file_target::checked_error(error),
ToolError::ExecutionFailed(_)
));
}
}
#[test]
fn checkout_mismatched_readonly_results_are_unavailable_not_stale_or_unknown() {
use workdir::{CheckoutSearchResult, GlobResult, GrepResult};
let glob = CheckoutSearchResult::Glob(GlobResult {
paths: Vec::new(),
truncated: false,
});
let grep = CheckoutSearchResult::Grep(GrepResult {
output: String::new(),
match_count: 0,
matched_files: 0,
truncated: false,
paths: Vec::new(),
});
assert_checkout_code(
crate::search_target::grep_result(glob).err().unwrap(),
"checkout_unavailable",
);
assert_checkout_code(
crate::search_target::glob_result(grep).err().unwrap(),
"checkout_unavailable",
);
assert_checkout_code(
crate::file_target::readonly_result_error("Read"),
"checkout_unavailable",
);
}
#[test]
fn checkout_relative_path_boundaries() {
let base = WorkdirPath::new("src").unwrap();
assert_eq!(
relative_target(&base, "nested/a.txt", true)
.unwrap()
.as_str(),
"src/nested/a.txt"
);
assert_eq!(relative_target(&base, ".", false).unwrap(), base);
assert_eq!(
relative_target(&WorkdirPath::root(), "a.txt", true)
.unwrap()
.as_str(),
"a.txt"
);
for bad in [
"",
"..",
"../outside",
"nested/../../outside",
"/absolute",
"a\\..\\outside",
] {
assert!(
relative_target(&base, bad, false).is_err(),
"accepted {bad}"
);
}
assert!(relative_target(&base, ".", true).is_err());
assert!(!is_beneath(
&base,
&WorkdirPath::new("src-other/a.txt").unwrap()
));
assert!(check_pattern("../*").is_err());
assert!(check_pattern("/tmp/*").is_err());
assert!(check_pattern("**/*.rs").is_ok());
}
#[tokio::test]
async fn checkout_read_and_normal_edit_share_hash_and_line_count() {
let fixture = Fixture::new();
std::fs::write(fixture.dir.path().join("a.txt"), "alpha\nbeta\ngamma\n").unwrap();
let read = fixture
.native("main", "a.txt", "Read", json!({"offset": 1, "limit": 1}))
.await
.unwrap();
assert!(read.output.summary.contains("[2..2] of 3"));
assert_eq!(read.output.content.as_deref(), Some(" 2\tbeta\n"));
assert!(read.validator.is_some());
let selected = fixture.router.resolve(Some("main")).unwrap();
let tracker = fixture
.tracker
.scoped_attachment(&selected.alias, selected.generation);
assert_eq!(
tracker.observed_workdir_line_count(&WorkdirPath::new("a.txt").unwrap()),
Some(3)
);
fixture.normal("Edit", json!({"target_workdir": "main", "file_path": "a.txt", "old_string": "beta", "new_string": "changed"})).await.unwrap();
let write = fixture
.native("main", "a.txt", "Write", json!({"content": "new\n"}))
.await
.unwrap();
assert!(write.output.summary.contains("Overwrote"));
assert_eq!(fixture.tracker.change_stat().deleted, 4);
let reread = fixture
.native("main", "a.txt", "Read", json!({}))
.await
.unwrap();
assert_eq!(reread.output.content.as_deref(), Some(" 1\tnew\n"));
}
#[tokio::test]
async fn checkout_normal_read_then_native_edit_uniqueness_and_replace_all() {
let fixture = Fixture::new();
std::fs::write(fixture.dir.path().join("a.txt"), "x x x\n").unwrap();
let before_read = fixture
.native(
"main",
"a.txt",
"Edit",
json!({"old_string": "x", "new_string": "y"}),
)
.await
.err()
.unwrap();
assert!(before_read.to_string().contains("has not been read"));
fixture
.normal(
"Read",
json!({"target_workdir": "main", "file_path": "a.txt"}),
)
.await
.unwrap();
let not_unique = fixture
.native(
"main",
"a.txt",
"Edit",
json!({"old_string": "x", "new_string": "y"}),
)
.await
.err()
.unwrap();
assert_checkout_code(not_unique, "checkout_invalid");
assert_eq!(fixture.tracker.change_stat(), crate::ChangeStat::default());
let edit = fixture
.native(
"main",
"a.txt",
"Edit",
json!({"old_string": "x", "new_string": "y", "replace_all": true}),
)
.await
.unwrap();
assert!(edit.output.summary.contains("3 replacements"));
assert_eq!(
std::fs::read_to_string(fixture.dir.path().join("a.txt")).unwrap(),
"y y y\n"
);
fixture.normal("Write", json!({"target_workdir": "main", "file_path": "a.txt", "content": "normal after native"})).await.unwrap();
}
#[tokio::test]
async fn checkout_alias_and_generation_fences() {
let fixture = Fixture::new();
fixture.attach("other");
std::fs::write(fixture.dir.path().join("a.txt"), "same\n").unwrap();
fixture
.native("main", "a.txt", "Read", json!({}))
.await
.unwrap();
let wrong_alias = fixture
.native("other", "a.txt", "Write", json!({"content": "not allowed"}))
.await
.err()
.unwrap();
assert!(wrong_alias.to_string().contains("has not been read"));
let (old_generation, old_validator) = fixture.observation("main", "a.txt").await;
fixture
.router
.detach(&WorkdirAttachmentAlias::new("main").unwrap())
.await
.unwrap();
fixture.attach("main");
let error = execute_checkout_tool(
fixture.router.clone(),
fixture.tracker.clone(),
"main",
old_generation,
WorkdirPath::new("a.txt").unwrap(),
old_validator,
"Read",
json!({}),
Default::default(),
)
.await
.err()
.unwrap();
assert!(error.to_string().contains("generation changed"));
assert_checkout_stale(error);
let fresh_generation_no_read = fixture
.native("main", "a.txt", "Write", json!({"content": "not allowed"}))
.await
.err()
.unwrap();
assert!(
fresh_generation_no_read
.to_string()
.contains("has not been read")
);
}
#[tokio::test]
async fn checkout_stale_validator_and_hash_are_independent() {
let fixture = Fixture::new();
std::fs::write(fixture.dir.path().join("a.txt"), "old\n").unwrap();
fixture
.native("main", "a.txt", "Read", json!({}))
.await
.unwrap();
let (generation, validator) = fixture.observation("main", "a.txt").await;
std::fs::write(fixture.dir.path().join("a.txt"), "externally changed\n").unwrap();
let stale = execute_checkout_tool(
fixture.router.clone(),
fixture.tracker.clone(),
"main",
generation,
WorkdirPath::new("a.txt").unwrap(),
validator,
"Write",
json!({"content": "bad"}),
Default::default(),
)
.await
.err()
.unwrap();
assert_checkout_stale(stale);
let hash = fixture
.native("main", "a.txt", "Write", json!({"content": "bad"}))
.await
.err()
.unwrap();
assert_checkout_stale(hash);
assert_eq!(
std::fs::read_to_string(fixture.dir.path().join("a.txt")).unwrap(),
"externally changed\n"
);
assert_eq!(fixture.tracker.change_stat(), crate::ChangeStat::default());
}
#[tokio::test]
async fn checkout_create_new_missing_parents_never_overwrites() {
let fixture = Fixture::new();
let created = fixture
.native(
"main",
".",
"Create",
json!({"path": "nested/deep/a.txt", "content": "new\n"}),
)
.await
.unwrap();
assert!(created.output.summary.contains("Created"));
assert_eq!(
created.paths,
vec![WorkdirPath::new("nested/deep/a.txt").unwrap()]
);
let duplicate = fixture
.native(
"main",
".",
"Create",
json!({"path": "nested/deep/a.txt", "content": "overwrite"}),
)
.await;
assert!(duplicate.is_err());
assert_eq!(
std::fs::read_to_string(fixture.dir.path().join("nested/deep/a.txt")).unwrap(),
"new\n"
);
// Successful creation updates the same tracker as normal Tool Write.
fixture.normal("Write", json!({"target_workdir": "main", "file_path": "nested/deep/a.txt", "content": "changed"})).await.unwrap();
}
#[tokio::test]
async fn checkout_rejects_argument_overrides_and_escapes() {
let fixture = Fixture::new();
std::fs::write(fixture.dir.path().join("a.txt"), "safe").unwrap();
for (name, arguments) in [
("Read", json!({"file_path": "other"})),
("Read", json!({"target_workdir": "other"})),
("Write", json!({"content": "bad", "validator": []})),
("Create", json!({"path": "../outside", "content": "bad"})),
("Glob", json!({"pattern": "*", "path": "/absolute"})),
("Grep", json!({"pattern": "safe", "path": "../outside"})),
("Other", json!({})),
] {
let path = if ["Create", "Glob", "Grep"].contains(&name) {
"."
} else {
"a.txt"
};
let error = fixture
.native("main", path, name, arguments)
.await
.err()
.unwrap();
assert!(
matches!(error, ToolError::InvalidArgument(_)),
"{name}: {error}"
);
}
}
#[tokio::test]
async fn checkout_search_typed_paths_and_normal_rendering() {
let fixture = Fixture::new();
std::fs::create_dir_all(fixture.dir.path().join("src/nested")).unwrap();
std::fs::write(
fixture.dir.path().join("src/nested/a.txt"),
"context\nhit\nafter\n",
)
.unwrap();
std::fs::write(fixture.dir.path().join("outside.txt"), "hit\n").unwrap();
let glob = fixture
.native(
"main",
"src",
"Glob",
json!({"pattern": "**/*.txt", "path": "nested"}),
)
.await
.unwrap();
assert_eq!(
glob.paths,
vec![WorkdirPath::new("src/nested/a.txt").unwrap()]
);
let normal = fixture
.normal(
"Glob",
json!({"target_workdir": "main", "pattern": "**/*.txt", "path": "src/nested"}),
)
.await
.unwrap();
assert_eq!(glob.output.summary, normal.summary);
assert_eq!(glob.output.content, normal.content);
for mode in ["files_with_matches", "content", "count"] {
let grep = fixture
.native(
"main",
"src",
"Grep",
json!({"pattern": "hit", "output_mode": mode, "-C": 1}),
)
.await
.unwrap();
assert_eq!(
grep.paths,
vec![WorkdirPath::new("src/nested/a.txt").unwrap()]
);
let normal = fixture.normal("Grep", json!({"target_workdir": "main", "path": "src", "pattern": "hit", "output_mode": mode, "-C": 1})).await.unwrap();
assert_eq!(grep.output.summary, normal.summary);
assert_eq!(grep.output.content, normal.content);
assert!(!grep.output.content.unwrap().contains("outside.txt"));
}
}
+83 -56
View File
@@ -9,7 +9,7 @@ use serde::Deserialize;
use crate::error::ToolsError;
use crate::tracker::Tracker;
use workdir::{EditRequest, WorkdirPath, WorkdirSessionHandle, WorkdirSessionRouter};
use workdir::{WorkdirPath, WorkdirSessionHandle, WorkdirSessionRouter};
const DESCRIPTION: &str = "Replace a substring in an existing file in the selected Workdir attachment. By default \
`old_string` must be unique in the file; set `replace_all: true` to replace \
@@ -23,13 +23,8 @@ pub(crate) struct EditParams {
pub target_workdir: Option<String>,
/// Logical path relative to the bound Workdir root.
pub file_path: String,
/// String to replace. Must be unique in the file unless `replace_all` is true.
pub old_string: String,
/// Replacement string. Must differ from `old_string`.
pub new_string: String,
/// Replace all occurrences. Defaults to false.
#[serde(default)]
pub replace_all: bool,
#[serde(flatten)]
pub edit: fs_operation::text::EditArgs,
}
pub(crate) struct EditTool {
@@ -44,8 +39,7 @@ impl Tool for EditTool {
input_json: &str,
ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
let params: EditParams = serde_json::from_str(input_json)
.map_err(|e| ToolError::InvalidArgument(format!("invalid Edit input: {e}")))?;
let params: EditParams = crate::error::decode_file_input(input_json, "Edit")?;
let selected = crate::routing::resolve_session(
&self.router,
@@ -56,56 +50,68 @@ impl Tool for EditTool {
.tracker
.scoped_attachment(&selected.alias, selected.generation);
let path = WorkdirPath::new(&params.file_path).map_err(ToolsError::from)?;
tracing::debug!(path = %path, replace_all = params.replace_all, "Edit");
Ok(execute_edit(
crate::file_target::FileTarget {
session: selected.session,
path,
validator: None,
},
tracker,
params,
ctx,
)
.await?
.output)
}
}
if params.old_string.is_empty() {
return Err(ToolError::InvalidArgument(
"old_string must not be empty".into(),
));
}
if params.old_string == params.new_string {
return Err(ToolError::InvalidArgument(
"old_string and new_string are identical".into(),
));
}
let mutation_key = PathBuf::from(path.as_str());
let _mutation_permit = tracker.acquire_mutation(&mutation_key, &ctx).await;
let expected_hash = tracker.expected_workdir_hash(&path)?;
let result = selected
.session
.edit(EditRequest {
path: path.clone(),
old_string: params.old_string.clone(),
new_string: params.new_string.clone(),
replace_all: params.replace_all,
expected_hash,
})
.await
.map_err(ToolsError::from)?;
let replacements = result.replacements;
tracker.record_workdir_edit(
&path,
result.content_hash,
replacements,
params.new_string.lines().count(),
params.old_string.lines().count(),
);
let summary = format!(
"Edited {} ({} replacement{})",
path,
replacements,
if replacements == 1 { "" } else { "s" }
);
let preview = make_preview(&params.new_string, &params.new_string);
Ok(ToolOutput {
pub(crate) async fn execute_edit(
target: crate::file_target::FileTarget,
tracker: Tracker,
params: EditParams,
ctx: agen::tool::ToolExecutionContext,
) -> Result<crate::checkout::CheckoutToolOutput, ToolError> {
let path = &target.path;
tracing::debug!(path = %path, replace_all = params.edit.replace_all, "Edit");
params
.edit
.validate(Default::default())
.map_err(crate::error::text_error)?;
let mutation_key = PathBuf::from(path.as_str());
let _mutation_permit = tracker.acquire_mutation(&mutation_key, &ctx).await;
let expected_hash = tracker.expected_workdir_hash(path)?;
let (result, validator) = target
.edit(
params.edit.old_string.clone(),
params.edit.new_string.clone(),
params.edit.replace_all,
expected_hash,
)
.await?;
let replacements = result.replacements;
tracker.record_workdir_edit(
path,
result.content_hash,
replacements,
params.edit.new_string.lines().count(),
params.edit.old_string.lines().count(),
);
let summary = format!(
"Edited {} ({} replacement{})",
path,
replacements,
if replacements == 1 { "" } else { "s" }
);
let preview = make_preview(&params.edit.new_string, &params.edit.new_string);
Ok(crate::checkout::CheckoutToolOutput {
output: ToolOutput {
summary,
content: Some(preview),
attachments: Vec::new(),
})
}
},
paths: Vec::new(),
validator,
})
}
/// Build a small line-numbered snippet centered on the first occurrence of
@@ -182,6 +188,27 @@ mod tests {
.unwrap();
}
#[tokio::test]
async fn edit_shared_validation_rejects_identical_text_before_read_or_mutation() {
let (dir, fs, tracker) = setup();
let file = dir.path().join("a.txt");
std::fs::write(&file, "same").unwrap();
let (_, tool) = edit_tool(fs, tracker.clone())();
let error = tool
.execute(
&serde_json::json!({"file_path":"a.txt","old_string":"same","new_string":"same"})
.to_string(),
Default::default(),
)
.await
.unwrap_err();
assert!(
matches!(error, ToolError::InvalidArgument(message) if message == "old_string and new_string are identical")
);
assert_eq!(std::fs::read_to_string(file).unwrap(), "same");
assert_eq!(tracker.change_stat(), crate::ChangeStat::default());
}
#[tokio::test]
async fn edit_unique_replacement() {
let (dir, fs, tracker) = setup();
+58 -1
View File
@@ -44,7 +44,8 @@ impl From<ToolsError> for ToolError {
| workdir::WorkdirError::Unavailable(_)
| workdir::WorkdirError::OperationFailed
| workdir::WorkdirError::Transport(_)
| workdir::WorkdirError::Conflict(_),
| workdir::WorkdirError::Conflict(_)
| workdir::WorkdirError::OutcomeUnknown(_),
) => ToolError::ExecutionFailed(err.to_string()),
ToolsError::FileSystem(_)
| ToolsError::WorkdirSession(_)
@@ -57,11 +58,67 @@ impl From<ToolsError> for ToolError {
}
}
/// Pure text failures are pre-dispatch argument failures, never unknown outcomes.
pub(crate) fn text_error(error: fs_operation::text::TextError) -> ToolError {
ToolError::InvalidArgument(error.to_string())
}
pub(crate) fn decode_file_input<T: serde::de::DeserializeOwned>(
input: &str,
name: &str,
) -> Result<T, ToolError> {
fs_operation::text::decode_json(input).map_err(|error| decode_error(error, name))
}
pub(crate) fn decode_file_value<T: serde::de::DeserializeOwned>(
value: serde_json::Value,
name: &str,
) -> Result<T, ToolError> {
fs_operation::text::decode(value).map_err(|error| decode_error(error, name))
}
fn decode_error(error: fs_operation::text::TextError, name: &str) -> ToolError {
match error {
fs_operation::text::TextError::Decode(error) => {
ToolError::InvalidArgument(format!("invalid {name} input: {error}"))
}
error => text_error(error),
}
}
#[cfg(test)]
mod tests {
use super::*;
use workdir::http::{WorkdirTransportError, WorkdirTransportErrorCode};
#[test]
fn flattened_file_tool_arguments_keep_raw_json_duplicate_rejection() {
let error = decode_file_input::<crate::edit::EditParams>(
r#"{"file_path":"a.txt","old_string":"a","old_string":"b","new_string":"c"}"#,
"Edit",
)
.unwrap_err();
assert!(
matches!(error, ToolError::InvalidArgument(message) if message.contains("duplicate field `old_string`"))
);
}
#[test]
fn checkout_outcome_unknown_remains_execution_failure() {
let error = ToolError::from(ToolsError::WorkdirSession(
workdir::WorkdirError::OutcomeUnknown(
"mutation may have committed; do not retry automatically".into(),
),
));
match error {
ToolError::ExecutionFailed(message) => {
assert!(message.contains("mutation may have committed"));
assert!(message.contains("do not retry automatically"));
}
other => panic!("outcome unknown must not become invalid input: {other:?}"),
}
}
#[test]
fn local_workdir_content_conflict_is_retryable_execution_failure() {
let error = ToolError::from(ToolsError::WorkdirSession(
+186
View File
@@ -0,0 +1,186 @@
//! Shared provider dispatch for Tool and checkout file processing.
use agen::tool::ToolError;
use workdir::{
CheckoutOperation, CheckoutOutput, CheckoutRequest, ContentHash, EditRequest, EditResult,
ReadRequest, ReadResult, WorkdirPath, WorkdirSessionHandle, WriteRequest, WriteResult,
};
use crate::ToolsError;
pub(crate) struct FileTarget {
pub session: WorkdirSessionHandle,
pub path: WorkdirPath,
pub validator: Option<Vec<u8>>,
}
impl FileTarget {
async fn checked(
&self,
operation: CheckoutOperation,
) -> Result<(CheckoutOutput, Option<Vec<u8>>), ToolError> {
let result = self
.session
.checkout_execute(CheckoutRequest {
target: self.path.clone(),
validator: self
.validator
.clone()
.expect("checked dispatch requires validator"),
operation,
})
.await
.map_err(checked_error)?;
Ok((result.output, Some(result.observation.validator)))
}
pub async fn read(
&self,
offset: usize,
limit: usize,
max_bytes: usize,
) -> Result<(ReadResult, Option<Vec<u8>>), ToolError> {
if self.validator.is_some() {
let (output, validator) = self
.checked(CheckoutOperation::Read {
offset,
limit,
max_bytes,
})
.await?;
match output {
CheckoutOutput::Read(result) => Ok((result, validator)),
_ => Err(readonly_result_error("Read")),
}
} else {
let result = self
.session
.read(ReadRequest {
path: self.path.clone(),
offset,
limit,
max_bytes,
})
.await
.map_err(ToolsError::from)?;
Ok((result, None))
}
}
pub async fn edit(
&self,
old_string: String,
new_string: String,
replace_all: bool,
expected_hash: ContentHash,
) -> Result<(EditResult, Option<Vec<u8>>), ToolError> {
if self.validator.is_some() {
let (output, validator) = self
.checked(CheckoutOperation::Edit {
old_string,
new_string,
replace_all,
expected_hash,
})
.await?;
match output {
CheckoutOutput::Edit(result) => Ok((result, validator)),
_ => Err(unknown_result("Edit")),
}
} else {
let result = self
.session
.edit(EditRequest {
path: self.path.clone(),
old_string,
new_string,
replace_all,
expected_hash,
})
.await
.map_err(ToolsError::from)?;
Ok((result, None))
}
}
pub async fn write(
&self,
content: Vec<u8>,
expected_hash: Option<ContentHash>,
create_path: Option<WorkdirPath>,
) -> Result<(WriteResult, Option<Vec<u8>>), ToolError> {
if self.validator.is_some() {
let operation = match create_path {
Some(path) => CheckoutOperation::Create { path, content },
None => CheckoutOperation::Write {
content,
expected_hash: expected_hash
.expect("existing checkout Write requires prior read"),
},
};
let (output, validator) = self.checked(operation).await?;
match output {
CheckoutOutput::Write(result) => Ok((result, validator)),
_ => Err(unknown_result("Write/Create")),
}
} else {
let result = self
.session
.write(WriteRequest {
path: self.path.clone(),
content,
expected_hash,
})
.await
.map_err(ToolsError::from)?;
Ok((result, None))
}
}
}
pub(crate) fn checked_error(error: workdir::WorkdirError) -> ToolError {
use workdir::WorkdirError;
let code = match &error {
WorkdirError::Conflict(_) | WorkdirError::NotFound(_) => "checkout_stale",
WorkdirError::Denied { .. }
| WorkdirError::OutOfScope { .. }
| WorkdirError::ReadOnly { .. }
| WorkdirError::Unsupported { .. }
| WorkdirError::UnsupportedOperation { .. }
| WorkdirError::SymlinkOutOfScope { .. }
| WorkdirError::SymlinkDirectoryNotTraversed { .. } => "checkout_denied",
WorkdirError::InvalidArgument { .. }
| WorkdirError::InvalidPath { .. }
| WorkdirError::UnknownCommand { .. }
| WorkdirError::RelativePath { .. }
| WorkdirError::BrokenSymlink { .. }
| WorkdirError::SymlinkTargetIsDirectory { .. }
| WorkdirError::IsDirectory { .. }
| WorkdirError::InvalidGlob { .. }
| WorkdirError::InvalidRegex { .. } => "checkout_invalid",
WorkdirError::Unavailable { .. } | WorkdirError::SessionClosed => "checkout_unavailable",
// IO/transport failures may arrive after effects have begun. Do not infer
// no-effects from their text or strip explicit OutcomeUnknown evidence.
WorkdirError::Io { .. }
| WorkdirError::Transport { .. }
| WorkdirError::OutcomeUnknown { .. }
| WorkdirError::OperationFailed => {
return ToolsError::from(error).into();
}
};
ToolError::StructuredConflict {
code: code.into(),
message: error.to_string(),
}
}
pub(crate) fn readonly_result_error(operation: &str) -> ToolError {
checked_error(workdir::WorkdirError::Unavailable(format!(
"provider returned an invalid readonly checkout {operation} result"
)))
}
fn unknown_result(operation: &str) -> ToolError {
ToolError::ExecutionFailed(format!(
"Outcome unknown: provider returned wrong checkout {operation} result after mutation; do not automatically retry"
))
}
+52 -37
View File
@@ -11,15 +11,15 @@ use crate::ToolsError;
const RESULT_LIMIT: usize = 1000;
#[derive(Debug, Deserialize, JsonSchema)]
struct GlobParams {
pub(crate) struct GlobParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
target_workdir: Option<String>,
pub target_workdir: Option<String>,
/// Glob pattern, for example `**/*.rs` or `src/**/test_*.py`.
pattern: String,
pub pattern: String,
/// Logical Workdir-relative directory. Defaults to the Workdir root.
#[serde(default)]
path: Option<String>,
pub path: Option<String>,
}
struct GlobTool {
@@ -44,42 +44,57 @@ impl Tool for GlobTool {
Some(path) => WorkdirPath::new(&path).map_err(ToolsError::from)?,
None => WorkdirPath::root(),
};
let pattern = params.pattern;
tracing::debug!(%pattern, %path, "Glob");
let result = selected
.session
.glob(GlobRequest {
pattern: pattern.clone(),
path,
limit: RESULT_LIMIT,
})
.await
.map_err(ToolsError::from)?;
let mut body = result
.paths
.iter()
.map(ToString::to_string)
.collect::<Vec<_>>()
.join("\n");
if !body.is_empty() {
body.push('\n');
}
let summary = if result.paths.is_empty() {
format!("No files found matching {pattern}")
} else if result.truncated {
format!(
"Found {}+ files matching {pattern} (truncated to {RESULT_LIMIT})",
result.paths.len()
)
} else {
format!("Found {} file(s) matching {pattern}", result.paths.len())
};
Ok(ToolOutput {
Ok(execute_glob(
crate::search_target::SearchTarget::tool(selected.session),
path,
params.pattern,
)
.await?
.output)
}
}
pub(crate) async fn execute_glob(
target: crate::search_target::SearchTarget,
path: WorkdirPath,
pattern: String,
) -> Result<crate::checkout::CheckoutToolOutput, ToolError> {
tracing::debug!(%pattern, %path, "Glob");
let result = target
.glob(GlobRequest {
pattern: pattern.clone(),
path,
limit: RESULT_LIMIT,
})
.await?;
let mut body = result
.paths
.iter()
.map(ToString::to_string)
.collect::<Vec<_>>()
.join("\n");
if !body.is_empty() {
body.push('\n');
}
let summary = if result.paths.is_empty() {
format!("No files found matching {pattern}")
} else if result.truncated {
format!(
"Found {}+ files matching {pattern} (truncated to {RESULT_LIMIT})",
result.paths.len()
)
} else {
format!("Found {} file(s) matching {pattern}", result.paths.len())
};
Ok(crate::checkout::CheckoutToolOutput {
output: ToolOutput {
summary,
content: (!body.is_empty()).then_some(body),
attachments: Vec::new(),
})
}
},
paths: result.paths,
validator: None,
})
}
pub fn glob_tool(session: WorkdirSessionHandle) -> ToolDefinition {
+75 -59
View File
@@ -22,16 +22,16 @@ enum OutputMode {
}
#[derive(Debug, Deserialize, JsonSchema)]
struct GrepParams {
pub(crate) struct GrepParams {
/// Worker-local alias of the Workdir attachment to use.
#[serde(default)]
target_workdir: Option<String>,
pattern: String,
/// Workdir-relative path, or an absolute path covered by readable scope. Defaults to the Workdir root.
#[serde(default)]
path: Option<String>,
pub path: Option<String>,
#[serde(default)]
glob: Option<String>,
pub glob: Option<String>,
#[serde(default, rename = "type")]
file_type: Option<String>,
#[serde(default)]
@@ -70,69 +70,85 @@ impl Tool for GrepTool {
params.target_workdir.as_deref(),
workdir::WorkdirSessionCapability::Grep,
)?;
let path = match params.path {
Some(path) => WorkdirPath::new_scoped(&path).map_err(ToolsError::from)?,
let path = match params.path.as_deref() {
Some(path) => WorkdirPath::new_scoped(path).map_err(ToolsError::from)?,
None => WorkdirPath::root(),
};
let mode = match params.output_mode.unwrap_or_default() {
OutputMode::FilesWithMatches => GrepOutputMode::FilesWithMatches,
OutputMode::Content => GrepOutputMode::Content,
OutputMode::Count => GrepOutputMode::Count,
};
let (before_context, after_context) = params
.context
.map(|context| (context, context))
.unwrap_or((params.before.unwrap_or(0), params.after.unwrap_or(0)));
let head_limit = params.head_limit.unwrap_or(DEFAULT_HEAD_LIMIT);
let result = selected
.session
.grep(GrepRequest {
pattern: params.pattern,
path,
glob: params.glob,
file_type: params.file_type,
case_insensitive: params.case_insensitive,
before_context,
after_context,
multiline: params.multiline,
output_mode: mode,
limit: head_limit,
offset: params.offset.unwrap_or(0),
})
.await
.map_err(ToolsError::from)?;
Ok(execute_grep(
crate::search_target::SearchTarget::tool(selected.session),
path,
params,
)
.await?
.output)
}
}
let summary = if result.match_count == 0 {
match mode {
GrepOutputMode::Content => "No matches".to_owned(),
_ => "No files matched".to_owned(),
pub(crate) async fn execute_grep(
target: crate::search_target::SearchTarget,
path: WorkdirPath,
params: GrepParams,
) -> Result<crate::checkout::CheckoutToolOutput, ToolError> {
let mode = match params.output_mode.unwrap_or_default() {
OutputMode::FilesWithMatches => GrepOutputMode::FilesWithMatches,
OutputMode::Content => GrepOutputMode::Content,
OutputMode::Count => GrepOutputMode::Count,
};
let (before_context, after_context) = params
.context
.map(|context| (context, context))
.unwrap_or((params.before.unwrap_or(0), params.after.unwrap_or(0)));
let head_limit = params.head_limit.unwrap_or(DEFAULT_HEAD_LIMIT);
let result = target
.grep(GrepRequest {
pattern: params.pattern,
path,
glob: params.glob,
file_type: params.file_type,
case_insensitive: params.case_insensitive,
before_context,
after_context,
multiline: params.multiline,
output_mode: mode,
limit: head_limit,
offset: params.offset.unwrap_or(0),
})
.await?;
let summary = if result.match_count == 0 {
match mode {
GrepOutputMode::Content => "No matches".to_owned(),
_ => "No files matched".to_owned(),
}
} else {
match mode {
GrepOutputMode::FilesWithMatches => {
format!("Found matches in {} file(s)", result.matched_files)
}
} else {
match mode {
GrepOutputMode::FilesWithMatches => {
format!("Found matches in {} file(s)", result.matched_files)
}
GrepOutputMode::Count => format!(
"Found matches in {} file(s), {} total line(s)",
result.matched_files, result.match_count
),
GrepOutputMode::Content => format!(
"{} matching line(s) in {} file(s)",
result.match_count, result.matched_files
),
}
};
let summary = if result.truncated {
format!("{summary} (truncated at {head_limit})")
} else {
summary
};
Ok(ToolOutput {
GrepOutputMode::Count => format!(
"Found matches in {} file(s), {} total line(s)",
result.matched_files, result.match_count
),
GrepOutputMode::Content => format!(
"{} matching line(s) in {} file(s)",
result.match_count, result.matched_files
),
}
};
let summary = if result.truncated {
format!("{summary} (truncated at {head_limit})")
} else {
summary
};
Ok(crate::checkout::CheckoutToolOutput {
output: ToolOutput {
summary,
content: (!result.output.is_empty()).then_some(result.output),
attachments: Vec::new(),
})
}
},
paths: result.paths,
validator: None,
})
}
pub fn grep_tool(session: WorkdirSessionHandle) -> ToolDefinition {
+4
View File
@@ -9,20 +9,24 @@
//! and command capability, while the Runtime process and OS user remain the
//! trusted execution boundary.
pub mod checkout;
pub mod error;
pub mod tracker;
mod bash;
mod edit;
mod file_target;
mod glob;
mod grep;
mod read;
mod routing;
mod search_target;
mod view_image;
mod web;
mod write;
pub use bash::bash_tool;
pub use checkout::{CheckoutToolOutput, execute_checkout_tool};
pub use edit::edit_tool;
pub use error::ToolsError;
pub use glob::glob_tool;
+93 -96
View File
@@ -8,7 +8,7 @@ use serde::Deserialize;
use crate::error::ToolsError;
use crate::tracker::Tracker;
use workdir::{ReadRequest, WorkdirPath, WorkdirSessionHandle, WorkdirSessionRouter};
use workdir::{WorkdirPath, WorkdirSessionHandle, WorkdirSessionRouter};
const DESCRIPTION: &str = "Read a text file from a Workdir attachment selected by its Worker-local alias. \
Supports offset/limit for large files. Returns line-numbered output (1-based). \
@@ -26,12 +26,8 @@ pub(crate) struct ReadParams {
pub target_workdir: Option<String>,
/// Workdir-relative path, or an absolute path covered by readable scope.
pub file_path: String,
/// 0-based line offset from the start. Defaults to 0.
#[serde(default)]
pub offset: Option<usize>,
/// Maximum number of lines to return. Defaults to 2000.
#[serde(default)]
pub limit: Option<usize>,
#[serde(flatten)]
pub read: fs_operation::text::LineReadArgs,
}
pub(crate) struct ReadTool {
@@ -46,8 +42,7 @@ impl Tool for ReadTool {
input_json: &str,
_ctx: agen::tool::ToolExecutionContext,
) -> Result<ToolOutput, ToolError> {
let params: ReadParams = serde_json::from_str(input_json)
.map_err(|e| ToolError::InvalidArgument(format!("invalid Read input: {e}")))?;
let params: ReadParams = crate::error::decode_file_input(input_json, "Read")?;
let selected = crate::routing::resolve_session(
&self.router,
params.target_workdir.as_deref(),
@@ -56,58 +51,65 @@ impl Tool for ReadTool {
let tracker = self
.tracker
.scoped_attachment(&selected.alias, selected.generation);
let offset = params.offset.unwrap_or(0);
let limit = params.limit.unwrap_or(DEFAULT_LIMIT).max(1);
let path = WorkdirPath::new_scoped(&params.file_path).map_err(ToolsError::from)?;
tracing::debug!(path = %path, offset, limit, "Read");
let result = selected
.session
.read(ReadRequest {
path: path.clone(),
offset,
limit,
max_bytes: PROVIDER_BYTE_LIMIT,
})
.await
.map_err(ToolsError::from)?;
tracker.record_workdir_hash(&path, result.content_hash);
let text = String::from_utf8_lossy(&result.bytes).into_owned();
let rendered = render_provider_read(
&text,
result.start_line,
result.total_lines,
result.truncated,
);
let summary = if rendered.truncated {
format!(
"Read {} line(s) [{}..{}] of {} from {}",
rendered.line_count,
offset + 1,
offset + rendered.line_count,
rendered.total_lines,
path
)
} else {
format!("Read {} line(s) from {}", rendered.line_count, path)
};
Ok(ToolOutput {
summary,
content: Some(rendered.body),
attachments: Vec::new(),
})
Ok(execute_read(
crate::file_target::FileTarget {
session: selected.session,
path: WorkdirPath::new_scoped(&params.file_path).map_err(ToolsError::from)?,
validator: None,
},
tracker,
params,
)
.await?
.output)
}
}
struct Rendered {
body: String,
line_count: usize,
total_lines: usize,
truncated: bool,
pub(crate) async fn execute_read(
target: crate::file_target::FileTarget,
tracker: Tracker,
params: ReadParams,
) -> Result<crate::checkout::CheckoutToolOutput, ToolError> {
let (offset, limit) = params.read.range(DEFAULT_LIMIT);
let path = &target.path;
tracing::debug!(path = %path, offset, limit, "Read");
let (result, validator) = target.read(offset, limit, PROVIDER_BYTE_LIMIT).await?;
tracker.record_workdir_observation(path, result.content_hash, result.total_lines);
// Source/response bounds were enforced by the streaming provider. Lossy
// UTF-8 conversion is the existing Tool presentation contract; rendering
// below independently bounds its possible byte expansion.
let text = fs_operation::text::read(
String::from_utf8_lossy(&result.bytes).into_owned(),
Default::default(),
)
.map_err(crate::error::text_error)?;
let rendered = render_provider_read(
&text.content,
result.start_line,
result.total_lines,
result.truncated,
);
let summary = if rendered.truncated {
format!(
"Read {} line(s) [{}..{}] of {} from {}",
rendered.line_count,
offset.saturating_add(1),
offset.saturating_add(rendered.line_count),
rendered.total_lines,
path
)
} else {
format!("Read {} line(s) from {}", rendered.line_count, path)
};
Ok(crate::checkout::CheckoutToolOutput {
output: ToolOutput {
summary,
content: Some(rendered.body),
attachments: Vec::new(),
},
paths: Vec::new(),
validator,
})
}
fn render_provider_read(
@@ -115,45 +117,14 @@ fn render_provider_read(
start_line: usize,
total_lines: usize,
truncated: bool,
) -> Rendered {
use std::fmt::Write as _;
let lines = text.lines().collect::<Vec<_>>();
let mut body = String::with_capacity(text.len().saturating_add(lines.len() * 8));
for (index, line) in lines.iter().enumerate() {
let _ = writeln!(&mut body, "{:>6}\t{}", start_line + index + 1, line);
}
Rendered {
body,
line_count: lines.len(),
) -> fs_operation::text::Rendered {
fs_operation::text::render_numbered(
text,
start_line,
total_lines,
truncated: start_line > 0 || truncated,
}
}
/// Format a slice of lines from `text` with `cat -n` style 1-based line
/// numbers. Pure function — no I/O, no history touching.
#[cfg(test)]
fn render_numbered(text: &str, offset: usize, limit: usize) -> Rendered {
let all_lines: Vec<&str> = text.lines().collect();
let total_lines = all_lines.len();
let start = offset.min(total_lines);
let end = start.saturating_add(limit).min(total_lines);
let slice = &all_lines[start..end];
let line_count = slice.len();
use std::fmt::Write as _;
let mut body = String::with_capacity(text.len().saturating_add(line_count * 8));
for (i, line) in slice.iter().enumerate() {
let lineno = start + i + 1;
let _ = writeln!(&mut body, "{:>6}\t{}", lineno, line);
}
Rendered {
body,
line_count,
total_lines,
truncated: start > 0 || end < total_lines,
}
truncated,
PROVIDER_BYTE_LIMIT,
)
}
/// Factory for the `Read` tool bound to one compatibility session.
@@ -195,6 +166,32 @@ mod tests {
(dir, session, Tracker::new())
}
#[test]
fn read_numbering_and_unicode_expansion_keep_rendered_output_bounded() {
for text in [
"\n".repeat(PROVIDER_BYTE_LIMIT),
"界".repeat(PROVIDER_BYTE_LIMIT),
"\u{1}".repeat(PROVIDER_BYTE_LIMIT),
] {
let rendered = render_provider_read(&text, 0, text.lines().count(), false);
assert!(rendered.body.len() <= PROVIDER_BYTE_LIMIT);
assert!(serde_json::to_vec(&rendered.body).unwrap().len() < 2 * 1024 * 1024);
}
let rendered = render_provider_read(
&"\n".repeat(PROVIDER_BYTE_LIMIT),
0,
PROVIDER_BYTE_LIMIT,
false,
);
assert!(rendered.truncated);
assert!(rendered.line_count < PROVIDER_BYTE_LIMIT);
assert!(
rendered
.body
.contains("truncated at rendered output byte limit")
);
}
#[tokio::test]
async fn read_tool_basic_records_history() {
let (dir, fs, tracker) = setup();
+81
View File
@@ -0,0 +1,81 @@
//! Provider dispatch shared by native and ordinary search processing.
//! Request construction/defaults and rendering remain in Glob/Grep; this only
//! selects the provider surface and verifies the returned readonly result kind.
use agen::tool::ToolError;
use workdir::{
CheckoutSearchOperation, CheckoutSearchRequest, CheckoutSearchResult, GlobRequest, GlobResult,
GrepRequest, GrepResult, WorkdirSessionHandle,
};
use crate::{ToolsError, file_target::checked_error};
pub(crate) struct SearchTarget {
session: WorkdirSessionHandle,
checkout: bool,
}
impl SearchTarget {
pub fn tool(session: WorkdirSessionHandle) -> Self {
Self {
session,
checkout: false,
}
}
pub fn checkout(session: WorkdirSessionHandle) -> Self {
Self {
session,
checkout: true,
}
}
pub async fn glob(&self, request: GlobRequest) -> Result<GlobResult, ToolError> {
if self.checkout {
let result = self
.session
.checkout_search(CheckoutSearchRequest::new(CheckoutSearchOperation::Glob(
request,
)))
.await
.map_err(checked_error)?;
glob_result(result)
} else {
self.session
.glob(request)
.await
.map_err(|error| ToolsError::from(error).into())
}
}
pub async fn grep(&self, request: GrepRequest) -> Result<GrepResult, ToolError> {
if self.checkout {
let result = self
.session
.checkout_search(CheckoutSearchRequest::new(CheckoutSearchOperation::Grep(
request,
)))
.await
.map_err(checked_error)?;
grep_result(result)
} else {
self.session
.grep(request)
.await
.map_err(|error| ToolsError::from(error).into())
}
}
}
pub(crate) fn glob_result(result: CheckoutSearchResult) -> Result<GlobResult, ToolError> {
match result {
CheckoutSearchResult::Glob(result) => Ok(result),
_ => Err(crate::file_target::readonly_result_error("Glob")),
}
}
pub(crate) fn grep_result(result: CheckoutSearchResult) -> Result<GrepResult, ToolError> {
match result {
CheckoutSearchResult::Grep(result) => Ok(result),
_ => Err(crate::file_target::readonly_result_error("Grep")),
}
}

Some files were not shown because too many files have changed in this diff Show More