264 Commits
Author SHA1 Message Date
Hare e8828cd1b5 feat: improve Drive file browsing and actions 2026-10-09 18:39:27 +09:00
Hare 73ac13e802 fix: restore compact Workdirs table and icon actions 2026-10-09 18:21:25 +09:00
Hare e7b1aa2a6a chore: merge develop into hare/develop 2026-10-09 18:03:58 +09:00
Hare c550abb11b fix: preserve Workdir operation errors in API logs 2026-10-09 18:03:37 +09:00
Hare caac1c18e0 fix: remove legacy ticket conclusion reason limits 2026-10-09 18:03:37 +09:00
Hare 5831190012 fix: integrate safe Workdir removal recovery
Merge approved T-730 source b1dc8f9f into develop. Resolve the test-registration conflict by preserving both Drive and Workdir-removal tests. Integration root check, Runtime/Server/Worker suites, real Web Drive adapter, generated Workdir contract, Web check/unit/components/build/browser, fmt and diff checks passed.
2026-10-09 17:43:19 +09:00
Hare b1dc8f9f1c fix: guard mount-safe Workdir cleanup and current-condition recovery 2026-10-09 17:20:51 +09:00
Hare 4d266420fc fix: retain concrete Workdir observation failure causes 2026-10-09 17:02:38 +09:00
Hare b02eb7bee3 feat: integrate Workspace Web Drive management
Merge approved T-724 source e4daa4aa into current develop a91eec29 without conflicts. Root check, Server714+11 and explicit real SQLite/LocalFileSystem/Web HTTP adapter, API TypeScript91 and generated contracts, Web check0/0/unit569/component305/build/browser9/responsive1 all passed. Fixture-browser results are distinct from real storage/auth tests; no live deployment update.
2026-10-09 16:59:19 +09:00
Hare ed50847afe fix: fence removal while permitting safe Workdir reconnection 2026-10-09 16:51:59 +09:00
Hare ef642b34be fix: explain safe WorkdirDelete recovery in tool output 2026-10-09 16:41:43 +09:00
Hare e4daa4aaa7 feat(workspace): add shared Drive browsing editing and Worker grants 2026-10-09 16:40:50 +09:00
Hare c889c9a68d fix: show Workdir removal causes and refresh ordinary retries 2026-10-09 16:35:26 +09:00
Hare a91eec295f fix: align API logging tests with worker assignment names 2026-10-09 16:28:14 +09:00
Hare 4337f415f9 fix: revalidate durable Workdir removal on ordinary retry 2026-10-09 16:26:01 +09:00
Hare 34f9e70ce8 chore: merge develop into hare/develop 2026-10-09 16:25:49 +09:00
Hare e6aa56ff6f feat: integrate authorized Worker Drive operations
Merge approved T-723 source e94af40d into develop3a2994cd. Resolve overlapping configuration WIP test migration by retaining source typed result assertions and target live Tree/Inspect publication checks. Runtime implementation merged without conflicts. Root check; full Manifest, Tools, Worker935+110, Runtime349+34, Server710+11, FS and Workdir suites; Workdir HTTP; API83/TypeScript91; WIP0.2.0 resolution and fmt/diff checks passed.
2026-10-09 16:11:07 +09:00
Hare 3a2994cdd7 feat: integrate bounded checkout discovery
Merge approved T-714 source b8faa83c into develop c79506b6. Preserve the target's already-integrated public Tree/Inspect/Invoke configuration fixture migration in the overlapping test helper; runtime implementation merged without conflicts. Root check, complete FS/Workdir/Tools/Worker/Server/API suites, HTTP Workdir suite, WIP 0.2.0 resolution, formatting and diff checks passed.
2026-10-09 15:45:41 +09:00
Hare e94af40dab fix(worker): reconcile invalid Drive completions and default pages
Accept the Backend shared default page maxima (list 200/search 128), not an unsent 100 limit. Classify semantically invalid completions before receipt reconciliation; query the original identity once and accept only validated committed results, never resend or upgrade revisions.

Regressions fail on 61afdb77 and pass after the fix: real API 201-entry default Tools/native WIP pagination; mismatched completion ID/invalid entry metadata recovery and unresolved/invalid receipt matrices. Root cargo check; Worker 920 unit +110 integration; Server 709 lib +11 binary; Drive unit18/API18; fmt and diff checks passed.
2026-10-09 15:36:06 +09:00
Hare b8faa83c47 fix: preserve unknown mutation outcomes after connection loss
Address POST_COMMIT_CONNECTION_FENCE from independent review of af655ae6. Post-success publication must not claim pre-effect rejection if Write/Edit/Create may have committed before attachment loss. Keep read-only validator mismatch and lazy entry mapping unchanged.

Add deterministic real checked provider/Tools commit -> detach -> production publication -> WIP Invoke/Client tests for all three mutations, asserting committed effects, OutcomeUnknown audit and exactly one dispatch. Narrow race9, Worker903+110, root check, dependent Server config19/proxy1, SDK resolution and fmt/diff checks passed.
2026-10-09 15:30:21 +09:00
Hare c79506b60f chore(ticket): preserve integrated authentication and WIP test fixes
# Conflicts:
#	crates/workspace-server/src/server_workspace_config_wip_tests.rs
2026-10-09 03:10:39 +09:00
Hare e90e2ab391 fix(ticket): validate configuration adapters through structured WIP 2026-10-09 03:04:47 +09:00
Hare 61afdb7773 feat: expose authorized Worker Drive tools and native WIP entries
Use T-722 identity-bound API for paged discovery, bounded text read/create/write/edit, folders, relocate/delete, revision-fixed images, Workdir artifact publication, and mutation receipts. Keep IDs as Workspace-bound decimal strings, observed CAS preconditions and Client validators hidden from model arguments, and never substitute revisions or blindly replay uncertain effects.

Activate via host-bound Feature/Profile/schema and centralized instructions. Publish only the Drive entrance as indexable; resolve selected stable IDs directly with coherent self-scoped WIP publication. Internal SubWorkers and Reviewer children deny undelegated Drive access at feature and client boundaries; no child grant or local storage authority is synthesized.

T-723 validation: root cargo check; manifest 190; tools 78 unit +39 integration +1 doctest; Worker 918 unit +110 integration; Runtime 348 unit +34 binary; Server 708 library +11 binary including 17 real DB+LocalFileSystem signed-API adapter tests. Native image capture/replay, multichunk revision bounds, Workdir import, concurrent CAS, current grants, response-loss reconciliation and dependent config WIP0.2 regressions pass. WIP crates.io 0.2.0 resolution verifier, fmt and diff checks pass.
2026-10-09 03:01:41 +09:00
Hare 8affa46bce feat: add bounded Workspace binary transport and native WIP attachments
Keep JSON and raw-byte authority paths distinct. Runtime signs exact method/query/worker/body; binary requests are bounded, reject target normalization and redirects, and sanitize transport errors. Native attachments accompany the validated WIP value and exact validator through existing ToolOutput capture/replay. Reuse scoped Workdir binary reads with complete SHA256 verification.

T-723 validation: binary transport/reviewer denial, native Invoke image capture and history replay, and bounded Workdir image helpers pass. Complete changed-crate suites and root cargo check passed on the combined Ticket implementation.
2026-10-09 03:01:14 +09:00
Hare cb8784c108 fix: integrate safe runtime proof diagnostics 2026-10-09 03:00:40 +09:00
Hare e67d75615d chore(ticket): incorporate integrated Drive and structured WIP contracts
# Conflicts:
#	openapi/server-api.json
#	web/workspace/src/lib/generated/companion-api.ts
#	web/workspace/src/lib/generated/repository-api.ts
#	web/workspace/src/lib/generated/runtime-api.ts
#	web/workspace/src/lib/generated/ticket-api.ts
#	web/workspace/src/lib/generated/worker-launch-api.ts
2026-10-09 02:54:09 +09:00
Hare 7bb4096373 fix: retain safe runtime proof rejection diagnostics
T-729: distinguish issued_in_future from expired with exact checked times and overflow-safe deltas; retain request-binding mismatch reasons and proof verification progress. Correlate claimed Runtime/Worker/JTI via fixed-size fingerprints and server-generated rejection IDs, bound request metadata, omit query/raw claims/error/body/header/key material. Preserve allow/deny ordering and generic 401 responses; document diagnostic limits and T-728 policy ownership. Reported same-machine incident remains unconfirmed.

Validation: worker-runtime auth filter 3 passed; worker-runtime crate 345 lib + 17 per binary passed; Server auth logging 3 passed (17 rejection cases on both routers plus bounded metadata and generic repository401); workspace_config_integration 12 passed; Server crate 692 lib + 11 bin passed; root cargo check; cargo fmt --all -- --check; rustfmt included config tests --check; git diff --check HEAD. Existing no-default Runtime feature-gate breakage remains documented and out of scope.
2026-10-09 02:53:23 +09:00
Hare 5244308c0d feat(ticket): align Web actions and instructions with intent-led work 2026-10-09 02:51:22 +09:00
Hare 717ff2555f fix: migrate server config test helpers to WIP object API
T-729 validation prerequisite approved in Ticket seq7. Preserve Backend authority, attachment, mutation, stale observation and unknown-outcome assertions using public Tree, path-centered Inspect and structured InterfaceReference Invoke. No production API expansion.

Validation: cargo test -p yoi-workspace-server --lib workspace_config_integration (12 passed); git diff --check HEAD.
2026-10-09 02:47:56 +09:00
Hare d4ed6192ea fix: align ticket web consumers with generic worker assignments 2026-10-09 02:36:59 +09:00
Hare ea1badc0be chore: merge develop into hare/develop
# Conflicts:
#	crates/api-macros/src/axum.rs
2026-10-09 02:28:08 +09:00
Hare f1287f27e7 fix: restore console run accounting from persisted logs 2026-10-09 02:27:27 +09:00
Hare af655ae632 feat: limit checkout indexing to entrances and expose bounded typed discovery
Content directories always have no indexable children; direct authorized paths remain resolvable. Add List using provider Read/enumeration authority and separate List permission, dirs-first lexical live continuation, bounded retention/path bytes, scoped result coordinates and External DTO validation. List/Glob/Grep return Descriptor-typed entries without eager acquisition. Bind content Interfaces coherently to the live entrance scope.

Validation: root cargo check; FS 58 unit+6 pagination, Workdir 125+5 (default and http-client), Tools 84 unit+39 integration+1 doctest, Worker 901 unit+110 integration, Server 691 unit+11 CLI, Server API 83 tests; exact crates.io WIP 0.2.0 resolution; format/diff checks. Focused model/local/remote/provider/race/scope checks passed. Commands and semantics in docs/development/wip-integration-validation.md.

Publication/review paused: continuation ShowTicket returned HTTP 401. No Backend bypass, source push, MR approval, target integration or live environment update is claimed.
2026-10-09 02:22:52 +09:00
Hare 6445999120 fix: migrate dependent server WIP tests to public object tools
Dependent test compilation exposed removed discover, opaque Interface strings and private call usage. Select structured references from actual path Inspect and invoke through the public WIP 0.2.0 API.

Validation: complete Workspace Server suite (691 unit + 11 CLI), Server API 83 tests including generated artifact freshness, root cargo check and SDK resolution verification passed.
2026-10-09 02:22:18 +09:00
Hare 37a13e3566 fix: resolve target-qualified ancestor interfaces through live publications
Add a provider-owned routing hint without granting authority. Fetch checks provider ownership and re-resolves one coherent target/scope publication; default self/shared scope behavior remains unchanged.

Validation: 7 ancestor regression tests, 901 Worker unit and 110 integration tests, root cargo check, formatting and diff checks passed.
2026-10-09 02:22:18 +09:00
Hare 32c2227e8e fix: integrate guarded SSE image recovery 2026-10-09 01:51:43 +09:00
Hare edf5368e09 fix: (agen) keep malformed SSE errors diagnostic-only
Resolve MR finding raw_error_data_authorizes_image_correction. Unparseable error frames retain bounded raw_data diagnostics under a neutral base message; echoed image wording cannot authorize attachment removal. Parser-to-Engine regressions assert one request, unchanged history/images, and retained raw data for malformed numeric message and non-JSON error payloads.

Validation passed: focused unparseable_stream_error_data test; all 18 image_recovery_test tests; worker image_recovery_tests (2 route matrices); root cargo check; full cargo test -p agen, agen --features codex, and worker; cargo fmt --all -- --check; git diff --check HEAD.
2026-10-09 01:46:06 +09:00
Hare f9d4c97f1e feat: integrate WIP 0.2.0 object-centered tools 2026-10-09 01:34:42 +09:00
Hare 5480421fcc fix: (agen) recover pre-output SSE image size rejections
Handle OpenAI Responses error and response.failed diagnostics without broadening status-less HTTP retry authority. Reuse the durable ViewImage correction gate and replace the stream pump only before output/tool admission; partial streams retain terminal cleanup and are never replayed.

Regression coverage: parser-to-engine correction, pairing/largest candidate, exhaustion/no image, unrelated invalid_value, partial text/reasoning and exactly-once side effects, commit failure, filesystem persistence before retry, cancellation, restore/reseed.

Validation: cargo test -p agen --test image_recovery_test (17 passed); cargo test -p worker --lib image_recovery_tests (2 route-matrix tests passed); root cargo check; cargo test -p agen; cargo test -p agen --features codex; cargo test -p worker; cargo fmt --all -- --check; git diff --check HEAD. All passed.
2026-10-09 01:33:41 +09:00
Hare c1b3b95c5d test: wip validate scope replacement and target compatibility 2026-10-09 01:13:15 +09:00
Hare 23cf6c4f88 fix: wip bind live ancestor scopes and adopt 0.2.0 2026-10-09 01:00:12 +09:00
Hare 85c9b218dd feat(wip): integrate scoped SDK and object-centered tools
Vendor the reproducible 1cbe03b4 Client/HTTP/Protocol/Text View snapshot with Cargo checksums and credential-free Nix dependency closure. Migrate native and compatibility publications to structured references and scope_ref, preserve target-first preconditions and frozen call descriptors, and expose Tree/Inspect/Invoke with complete edge coverage and official signatures.

Validation: worker 888 unit + 110 integration tests; pinned upstream Client63, HTTP39, Protocol23, TextView20 tests/doctests; root cargo check --locked --offline; format/diff checks; 43-file vendor checksum verification; nix build --no-link .#yoi.cargoDeps. Details and provider migration seams: docs/development/wip-integration-validation.md.
2026-10-09 00:12:38 +09:00
Hare b5ea162f83 feat(drive): expose authorized typed Workspace Drive API
T-722: reuse T-721 storage/CAS/receipts with clone-bound transaction authorization against attached Server authority. Add migration 84 audited Worker read_only/read_write grants and removal/deletion fencing, member-facing ordinary access, owner-controlled grant settings, Workspace-bound decimal entry references, bounded text/binary chunks, streamed HTTP receive/download with integrity and safe headers, typed errors and generated Rust/OpenAPI/TypeScript contracts.

Validation: root cargo check; server-api 83 default / 91 typescript tests; FeatureStorage 16; Drive 31 unit + 12 integration; Workspace Server 691 library + 11 binary tests (--test-threads=4), including 9 real DB/LocalFileSystem HTTP tests for two Workers/non-owner usage, revoke during reception, concurrent CAS, response-loss receipts, chunk expiry and safe binary media. cargo fmt --all -- --check and git diff --check HEAD passed. Existing OpenAPI schemas/routes semantically unchanged; dependent TS digest projections regenerated.
2026-10-09 00:02:09 +09:00
Hare 9da79ab9f0 feat(api): support typed binary success response bodies
Preserve byte-exact BinaryBody responses in generated Axum/Reqwest adapters, declared response headers, metadata and inline binary OpenAPI schemas. Request and JSON/error behavior stay unchanged. Validated all-feature api-macros/api-macros-impl suites including 5 binary response tests, existing HTTP adapters, OpenAPI and compile-fail contracts.
2026-10-09 00:02:09 +09:00
Hare 60ec815237 chore: merge approved Ticket Worker launch into develop 2026-10-08 23:06:49 +09:00
Hare 3637824c8c fix(storage): persist metadata ancestor links before SQLite admission
Address MR finding drive-metadata-directory-durability. Sync every configured and resolved metadata directory ancestor before initializing/migrating SQLite; re-sync existing links on retry and propagate failures without admitting/caching a usable database.

Validation: metadata ordering/failure/retry tests; FeatureStorage16, Drive29unit+12integration, Subjektiv36, Server649lib+11bin (threads4); root cargo check; fmt/diff checks. strace of first-use CRUD confirms fsync(metadata/ws/features), fsync(metadata/ws), fsync(metadata) before DB initialization. No physical power-cut claim.
2026-10-08 21:53:34 +09:00
Hare bb1b05ed20 fix(ticket): preserve uncertain launch claims on local replay rejection 2026-10-08 21:53:28 +09:00
Hare 633522278b feat(drive): add durable Workspace document storage and lifecycle
Use object_store 0.13.2 immutable local blobs and FeatureStorage SQLite trees, revision CAS, same-transaction receipts, bounded reads/search, staged/orphan GC and retryable deletion. Attach Server authority to serialize cached Drive operations with deletion reservation. Document fsync compensation, Linux guarantees and offline DB-plus-blob restore.

Validation: root cargo check; workspace-drive 29 unit + 12 integration tests; feature-storage 13 tests; workspace-server 649 library + 11 binary tests with test-threads=4; cargo fmt --all -- --check; git diff --check HEAD. Default-parallel Server run hit the existing ticket_checker_burst timeout assertion (648 passed); the isolated filter and bounded-parallel full suite passed. Fault tests are not physical power-cut evidence.
2026-10-08 21:39:51 +09:00
Hare 761de69e18 feat(ticket): generalize Worker launch and durable resource claims 2026-10-08 21:36:33 +09:00
Hare e5ea7544d4 Merge branch 'develop' into hare/develop 2026-10-08 20:46:36 +09:00
Hare 1a095315ee chore: merge approved optional workspace creation into develop 2026-10-08 19:58:04 +09:00
Hare 148da0b3d9 test(workspace): preserve member-facing access without repositories
Apply T-717 item6/thread7 user decision: settings stay owner-controlled, ordinary non-owner viewing remains permitted under existing authentication. Add real HTTP assertions for non-owner workspace reads and absent management permissions; existing owner-only repository/deletion mutations and anonymous rejection remain exercised. Replace obsolete pending-policy documentation; no production authorization policy change.

Validation: targeted repository-free HTTP test, root cargo check, yoi-workspace-server 639 lib + 11 bin tests, canonical OpenAPI freshness, cargo fmt and baseline/HEAD diff checks pass. Rust tests/documentation only; rendered UI and previously reviewed browser/visual evidence are unchanged.
2026-10-08 19:41:41 +09:00
Hare df390f02e9 feat(ticket): decouple audited lifecycle decisions from MR integration
T-718: unify state, completion, close and reopen with explicit CAS and operation receipts; keep MR integration proofs independent. Queue only the explicitly selected Ticket and fence target edits against active Worker grants. Align REST, Worker/WIP, evidence projection and generated contracts.

Validation: root cargo check; full ticket, merge-request, client, worker, yoi-workspace-server and server-api tests; server-api typescript feature; OpenAPI and six generated API checkers; Svelte check and 36 narrow Web tests; fmt and diff checks.
2026-10-08 19:12:39 +09:00
Hare f8a9555626 docs(workspace): specify optional creation and replay semantics
Document current-key replay, saved receipt compatibility, explicit entry points, zero-Repository lifecycle guards, and the unresolved viewing-policy decision. Normalize visual review evidence EOF.
2026-10-08 18:35:50 +09:00
Hare 7919616db5 feat(workspace): allow creation without an initial repository
Keep bootstrap, signing identity, config and resource-key allocation atomic with optional Repository intent. Preserve repository-present fingerprints for saved receipts; replay reads current authority at the initial key and never recreates missing assets.

Provide explicit repository-free CLI creation, optional TUI/Web creation inputs, nullable generated contracts, and valid empty catalog/Worker paths. Retain existing authenticated non-owner viewing policy pending the T-717 authorization decision.

Validation: root cargo check; server tests 639 lib + 11 bin; server-api 70 default / 77 typescript; client 52 lib + 2 integration; tui 337; yoi 11 lib + 120 bin. OpenAPI/TS freshness, cargo fmt and diff checks pass. Web check/build, 18 model tests, 268 component tests and Chromium 8 width/theme combinations pass; before/after visual evidence is documented in web/workspace/test/visual-review. Full Web suite has one independently reproduced baseline Ticket source-policy failure.
2026-10-08 18:34:44 +09:00
Hare e939cd4e4b fix(engine): keep tool truncation notices in tool results
Stop emitting a user-facing Engine warning for ordinary output limiting while preserving structured size diagnostics and the model-visible truncation marker. Keep the Worker warning bridge for genuine warnings.

Regression coverage proves bounded ASCII/UTF-8 results, per-tool limits, attachments, dispositions and commit/model agreement, plus absence of live/buffered truncation alerts and preservation of other Engine alerts.

Validation: cargo check; cargo test -p agen; cargo test -p worker; targeted callback test with codex; cargo fmt --all -- --check; git diff --check HEAD.
2026-10-08 17:33:15 +09:00
Hare b715b1ee37 fix(web): label current Ticket approval from requirement attestation
T-716 MR seq3 CURRENT_REQUIREMENT_APPROVAL_USES_INTEGRATION_PROOF: derive the requirement approval label from review_after_rescope rather than approved_current_subject. Preserve separate persisted integration/source approval display. Strengthen the Web regression for approved integration with stale current requirements and recovery by exact current attestation; require the Ticket route to render that tested status projection.

Validation: 28 focused Deno decoder/status/resource/route-reuse tests; deno task check (0 errors/warnings); root cargo check; cargo fmt --all -- --check; git diff --check HEAD. Rust contracts unchanged from validated source14fea520.
2026-10-08 16:34:37 +09:00
Hare 14fea520df merge(develop): retain active-work guards with immutable Ticket evidence
T-716: incorporate T-715 (b6de908e) into the existing Ticket source branch only. Resolve MR read/readiness conflicts by keeping persisted merged-result evaluation and active assignment selection for open source observation. Preserve responsibility records and terminal work releases. Update temporary API regression fixtures to distinguish retained identity from active work.

Narrow validation: merge-request store 21 tests; Workspace Server Ticket evidence API 5 tests, including done/merged remote evidence without active assignment and unavailable open source with retained released responsibility.
2026-10-08 16:20:38 +09:00
Hare 5424280348 fix(ticket): project immutable results and typed source observations
T-716: merged evidence uses recorded result approval; open source reads use Runtime provider authority and preserve unavailability. Current requirements use the shared exact snapshot evaluator. Remove primary-MR/report SQL heuristics; continue bounded query scans through nonmatching candidates. Align Worker/Web projections and explain lifecycle versus integration/attestation. Synchronize checker test on delivered inputs, not pre-delivery attempt state.

Validation: root cargo check; cargo test -p merge-request (30), -p server-api (70), -p worker (949), -p yoi-workspace-server (620 lib + 11 binary); server-api typescript feature check; 32 focused Deno tests; deno task check (0 errors/warnings); cargo fmt --all -- --check; git diff --check HEAD; canonical OpenAPI/generated Ticket artifact checks.
2026-10-08 15:44:27 +09:00
Hare 4d0f2afc1e fix(merge-request): share immutable integration and requirement evidence
T-716: validate recorded source/approval/request identity, chronology and revocation independently of mutable selectors. Reuse exact revision and complete linked-source attestation in CompleteTicket. Assignment and lifecycle mutation are unchanged.\n\nValidation: cargo test -p merge-request (9 evidence and 21 store tests).
2026-10-08 15:44:26 +09:00
Hare b6de908e9f fix(ticket): separate retained responsibility from unfinished work
T-715: end assignment work identities atomically on done/close without deleting responsibility. Reopen never reactivates old authority. Schema 82 preserves legacy closed assignments, historical removal plans and Worker attribution; active work drives admission, auth/routing, retention and cleanup while independent lifecycle/attachment fences remain.

Validation: root cargo check; cargo test -p yoi-workspace-server (630 lib + 11 bin), -p merge-request (20), -p worker (947); worker-runtime restore/removal filters (2); Deno ticket responsibility/runtime UI contracts (12); cargo fmt --all -- --check; git diff --check HEAD.
2026-10-08 15:35:16 +09:00
Hare e723afe5f5 fix: authorize restore coordination with worker input capability 2026-10-08 15:33:38 +09:00
Hare 56ab8c22ba fix: preserve generated API error details in logs 2026-10-08 15:33:38 +09:00
Hare b0d8c7b786 chore: merge develop into hare/develop 2026-10-07 18:03:33 +09:00
Hare cfacf04260 fix: prevent subject startup deadlocks and diagnose auth rejection 2026-10-07 17:56:48 +09:00
Hare f086cf3375 feat(profiles): seal builtin import provenance and expose read-only editor imports
Retain the evaluated JSON launch entrypoint, seal observed builtin source bytes and digests as verified provenance, and fail on missing/corrupt dependencies without Runtime builtin reevaluation. Cover real HTTP commits and launch resolution, including worker mode and Workspace Config feature composition.

Expose read-only builtin import completions/diagnostics in the editor and rebuild wasm32 release assets with wasm-bindgen 0.2.117. Document public paths, ordinary composition, limits, resave semantics, and the lazy-cycle incident.

Validation: worker-runtime 344 library + 34 integration tests; workspace-server 609 library + 11 binary tests plus final builtin HTTP/launch tests (2); archive tests (16); native WASM tests (2); real-WASM/web tests (37), editor component tests (10); root cargo check, cargo fmt --all -- --check and git diff --check HEAD pass.
2026-10-07 03:38:49 +09:00
Hare 4343be86c1 feat(config-source): add confined read-only builtin Decodal imports
Share the explicit embedded Profile catalog with native and WASM evaluators. Preserve ordinary Decodal values, isolate builtin relative imports, reject reserved Workspace mutations, and capture the exact loaded builtin closure. Guard lazy import cycles and dependency depth before evaluation recurses.

Validation: config-source 48 tests, builtin-source 2 tests, manifest 185 tests; root cargo check, formatting and diff checks pass. Existing contract version/fingerprint stay unchanged; archive provenance binds builtin content instead of invalidating stored Workspace metadata.
2026-10-07 03:35:59 +09:00
Hare a64707ee69 feat: integrate value-based Workspace Profiles 2026-10-07 03:04:07 +09:00
Hare e24fbabe54 fix: align Profile authoring with compaction and reasoning contracts (T-710)
Resolve MR seq9 supported-compaction-fields and numeric-extraction-reasoning findings. Cover all canonical CompactionConfigPartial settings, token aliases and ratio helpers; accept string effort and signed integer extraction budgets without materializing editor defaults. Audit the remaining Profile field groups against their real serde contracts.

Share an actual-provider value corpus across native, generated WASM, and shared save/projection/verified Runtime archive consumption. Prove inline/import/patch exact values and effective Manifest settings; gate canonical compaction field coverage against serialization of the actual partial type. Keep prior scope union and worker/feature diagnostics fixes intact.

Validation: root cargo check; config-source40; unfiltered server644+11; Runtime340+17+17; API typescript77; Deno39; deno task check0/0; wasm32 release regeneration byte-identical with wasm-bindgen0.2.117/Binaryen132; cargo fmt/diff checks. Logs target/t710-contract-{check,config-source,server,api,runtime}.log. Reviewer seq9 independently confirmed seq3/5/7 resolution and correct main/relative-cwd routing. No target integration/ref mutations.
2026-10-07 02:51:29 +09:00
Hare 61f0469986 fix: preserve string and table Profile scope authoring (T-710)
Resolve MR sequence 5 profile_authoring_rejects_valid_string_scope_intents. Decodal 0.4 has no union constraint: retain an opaque scope range with non-materialized table completion hints so string intents remain valid. Keep shared Backend semantic validation authoritative for scope forms/types/unknown keys without changing persisted values or weakening worker/feature editor diagnostics.

Prove actual-provider native/WASM scalar/table, inline/import/patch, nested intent/deny_write/symlink_policy completion and evaluation invariance. Add real save/projection/Runtime scope equivalence and invalid-scope save non-loss coverage. Regenerated WASM artifact is byte-identical; the changed shape is transported via the fingerprint-bound Backend schema metadata.

Validation: root cargo check; config-source39; value_profiles4; unfiltered server643+11; worker-runtime340+17+17; server-api --features typescript77; wasm32 release/wasm-bindgen0.2.117/Binaryen132 -Oz; Deno39; deno task check0/0; fmt/diff checks. Logs target/t710-scope-{check,server,server-api,runtime}.log. No target/ref integration changes.
2026-10-07 02:25:16 +09:00
Hare 728ea3d915 feat: integrate standalone subjektiv support 2026-10-07 02:04:30 +09:00
Hare 24ae6288a5 feat(subjektiv): share domain and add explicit standalone Subject hosts 2026-10-07 01:52:06 +09:00
Hare 3558e4aae1 fix: diagnose nested partial Profile values in the editor (T-710)
Resolve MR sequence 3 profile_nested_editor_diagnostics_missing with a separate, non-materializing authoring constraint pass. Preserve omission and the persisted evaluation schema, and map nested/imported failures to supplied virtual-source spans. Exercise actual Backend schema in native and regenerated WASM tests for empty/partial values, worker and feature types, unknown fields, imports and composition.

Validation: root cargo check; config-source 38; config-source-wasm native/doc; unfiltered yoi-workspace-server 642 lib + 11 bin; worker-runtime 340 lib + 17 per binary; server-api --features typescript 77; focused Deno 39; deno task check; wasm32 release build, wasm-bindgen 0.2.117 + Binaryen 132 -Oz regeneration; cargo fmt/diff checks. Earlier checker failure did not recur: Reviewer independently passed full source suite and exact original baseline checker filter, recorded in MR sequence 3.
2026-10-07 00:36:16 +09:00
Hare ed3bcf35bd fix(worker): remove Wip prefix from gateway tool names (T-712)
Expose Discover, Inspect, and Call in WIP mode and update schema descriptions and current documentation. Preserve execution, argument schemas, internal types, and ordinary Tools mode. Add a registered-tool discovery/inspection/call regression.

Validation: cargo test -p worker --lib wip::tests:: (43 passed); cargo test -p worker (827 unit + 110 integration passed); root cargo check; cargo fmt --all -- --check; git diff --check HEAD.
2026-10-07 00:31:33 +09:00
Hare 58462d5844 feat: register Workspace Profiles as Decodal values (T-710)
Project the evaluated Workspace Profile value into a revision-bound JSON archive without source-path registration or Decodal re-evaluation. Preserve partial-value omissions and explicit overrides, validate before save, and reject legacy registrations with a documented explicit migration.

Fix array-element schema completion and provide fingerprint-bound authoring shapes independently of materialization defaults. Carry metadata through REST, regenerate TypeScript/OpenAPI/WASM artifacts, and exercise save/projection/Runtime consumption and persisted Manifest restore boundaries.

Validation: root cargo check; cargo test -p config-source; cargo test -p worker-runtime; cargo test -p server-api --features typescript; focused profile_settings/value_profiles/archive tests; 39 web config-source/profile API tests; deno task check; cargo fmt --all -- --check; git diff --check HEAD.

Full workspace-server test: 641 passed, one pre-existing ticket_checker_suppresses_stale_and_clean_results_and_diagnoses_invalid_output assertion failure. The same filter fails at base 3c424fd17d in a separately archived source tree and isolated target directory. Remaining server crate tests pass with only that filter skipped (641 lib + 11 bin); unrelated checker code is unchanged.
2026-10-07 00:14:51 +09:00
Hare 13055d7ee1 feat: execute durable standalone Jobs through Host-owned Internal Workers (T-708) 2026-10-07 00:08:01 +09:00
Hare 5c95859d39 chore: merge develop into hare/develop 2026-10-06 20:45:19 +09:00
Hare 3c424fd17d fix: budget subjektiv batches for compact verified Job results (T-704) 2026-10-06 19:58:11 +09:00
Hare d12cdc9106 feat: caller-selected Backend Jobs and bounded subjektiv consolidation (T-704) 2026-10-06 19:45:37 +09:00
Hare 6f9621ca2e fix: retain Restore recovery intent and retire stale caller conflicts (T-701) 2026-10-06 18:15:31 +09:00
Hare fdce12a47b feat: guarded Worker Restore from shared Web actions (T-701) 2026-10-06 17:45:54 +09:00
Hare 1a6d391f83 refactor: integrate registry-bound worker operations 2026-10-06 14:58:49 +09:00
Hare 849a6b4c1c fix(server): route embedded WorkerRemove through registry-bound reception 2026-10-06 14:50:31 +09:00
Hare 7d8e595270 docs(report): record command output readback limitation 2026-10-06 14:24:23 +09:00
Hare 4892a82a28 refactor(server): unify registry-bound workspace worker operations 2026-10-06 14:24:23 +09:00
Hare 4f38f7d5b0 refactor(worker): share file contracts across Tools and WIP providers
Flatten shared typed Tool arguments and derive WIP parameters and checkout allowlists from core schemas. Workspace settings uses shared read/write/edit processing within its captured-read and canonical tree-CAS commit path. Identical settings edits now reject before read or commit; provider response contracts and unknown-outcome handling stay intact.

T-707 validation: root cargo check; fs-operation (58); tools (74 lib, 39 integration, 1 doctest); worker (802 lib, 109 integration); workdir --features http-client (125); yoi-workspace-server --lib workspace_config (19); cargo fmt --all -- --check; git diff --check HEAD.
2026-10-06 14:18:34 +09:00
Hare 4511d4f8ac refactor(fs-operation): extract provider-neutral text operations
Share typed read/edit/write arguments, Operation schemas, decoding, pure outcomes, byte bounds, replacement policy and numbered rendering. Keep filesystem CAS and descriptor-protected mutation within the provider.

T-707: cargo test -p fs-operation passes (58 tests); overflow, Unicode, limits and local checked-save regressions are covered.
2026-10-06 14:18:34 +09:00
Hare 4ceabef40e docs: require reading test design rules before editing tests 2026-10-06 13:31:37 +09:00
Hare 3029e77c7a fix: remove tests that pin mutable profile defaults 2026-10-06 13:31:37 +09:00
Hare b96bd110c6 refactor(worker-runtime): route by WorkerRef and reconcile owned execution cleanup 2026-10-06 09:13:46 +09:00
Hare 0e2a6f2860 fix(worker): bind protocol lifetime to controller endpoints and join startup observers
T-700 prerequisite: expose real method-channel closure and endpoint identity without introducing WorkerRef routing tokens or catalog liveness mirrors. Delay command observer creation until fallible head/recovery/runtime writes finish; abort and join it on socket startup failure.

Validation: cargo test -p worker (909 tests including integration targets); focused controller startup ordering regression; git diff --check HEAD. Full Runtime/Server validation and MR publication follow after the dependent T-700 implementation is clean.
2026-10-06 08:26:09 +09:00
Hare 2c023ac2be fix(workspace-config): fence user editors from runtime source proofs
Resolve MR finding WC-AUTH-001: reject authored source-tree and Profile editor requests from Runtime/Worker proofs in both scoped and single-Workspace middleware. Keep legitimate UI editing and independent Prompt/Skill/config-bundle consumption contracts. Add signed-router authority-state matrix and authenticated UI/WIP positive regressions.
2026-10-06 07:14:31 +09:00
Hare 0a471242cb feat(workspace-config): project grant-bound logical settings through WIP 2026-10-06 06:56:51 +09:00
Hare 7a96c42130 fix(checkouts): preserve scoped descriptor ignore rules
T-699 rev20, MR 01a10d66-db02-70c2-b152-197aca92d6b7 seq3 native_glob_ignore_regression. Load authorized root/nested ignore sources through FsAccessPolicy descriptor opens, retain only bounded ancestor matchers, preserve source precedence/negation/anchors and prune ignored directories before enumeration. Expanded parity also exposed Grep source-class precedence and ignored-directory rescue errors; use the shared matcher without relaxing Grep's existing ignore-open/Git-marker provider contract. No ambient global/linked-worktree administration pointer traversal or unsafe WalkBuilder ignore loading.

Add 7 fs boundary tests, 4 actual Local/scoped provider parity tests and 1 native WIP search/link regression. Current corrected source passes cargo check --workspace --all-targets; cargo test -p fs-operation -p workdir -p tools -p worker (fs37 Workdir125 Tools72+10+29+1doc Worker780+109integration); cargo test -p worker-runtime -p yoi-workspace-server (Runtime279+17+17 Backend576+11); fmt/diff. Logs target/validation/T-699/rereview-{check,core,hosts}.log. Earlier intermediate parity/WIP assertion failures retained, not passing evidence. No target/ref integration, RO source changes, environment update or completion.
2026-10-06 04:24:35 +09:00
Hare b952e56bc2 test: use required Bash cwd in compatibility regressions 2026-10-06 03:44:03 +09:00
Hare ae19426298 fix: preserve sanitized dangling-symlink read diagnostics 2026-10-06 03:42:06 +09:00
Hare 603a814b80 Merge commit '5670c20a6a50077985168269dfc15a6776c66ba6' into work/T-699-checkout-projection 2026-10-06 03:37:07 +09:00
Hare 149853bb0e feat: project attached checkouts with scoped native AI operations 2026-10-06 03:37:07 +09:00
Hare 5670c20a6a fix: integrate develop and preserve invocation notification recovery 2026-10-05 22:25:32 +09:00
Hare c847c2ffad fix: cancel pending uploads when composer chips are removed 2026-10-05 20:55:17 +09:00
Hare 9812e68b4d fix: synchronize native picker interception in Feature browser tests 2026-10-05 20:14:19 +09:00
Hare 9146331450 fix: echo REST completion nonce and argument context 2026-10-05 20:14:19 +09:00
Hare 1a45542842 fix: preserve invocation recovery across rewind and revalidate replay authority 2026-10-05 20:14:19 +09:00
Hare 3deef49897 fix: reconcile T-697 generated contracts with latest develop 2026-10-05 19:35:04 +09:00
Hare c57e236768 fix: update builtin Sol and Luna model defaults 2026-10-05 19:15:02 +09:00
Hare f5c6b0aeb1 fix: deduplicate console entries and contain worker rendering errors 2026-10-05 19:13:15 +09:00
Hare 31f7847d1f fix: preserve multiple positional invocation arguments and update API schema 2026-10-05 19:07:23 +09:00
Hare fa1169c556 docs(report): record reviewer severity schema mismatch 2026-10-05 19:01:13 +09:00
Hare 6feed0f22f fix(wip): page complete workdir catalogs and fence full connection sets 2026-10-05 19:01:13 +09:00
Hare 6972928612 feat: integrate persistent Subject behavior context 2026-10-05 18:48:12 +09:00
Hare a3c3bda3fa feat: add enlarged previews for console images 2026-10-05 18:45:23 +09:00
Hare f7783b3d2a feat: add declarative Feature chat invocations and typed composers 2026-10-05 18:44:41 +09:00
Hare 0fe61df25d fix: integrate serialized orchestrator attention 2026-10-05 18:10:12 +09:00
Hare 01156f4c78 docs: report SubWorker cwd attachment-root ambiguity 2026-10-05 18:10:01 +09:00
Hare 42e4ce5550 feat: publish Repository and Workdir WIP objects
Use Host root namespace ownership and disjoint Feature operation contributions. Separate read-only workdir_catalog from manage_workdir, encode all existing identities canonically, publish safe bounded inventory and lifetime-bound attachment Objects, and reuse normal lifecycle/session router/lock authority.

Contextual interfaces resolve the current permission/capability intersection; object and interface validators reject stale cached access. Cover fresh workspace WIP-only discovery/create/attach/read/detach, readonly and unsupported targets, collision/revocation/alias reuse, active cleanup exclusion and terminal unknown outcomes without retries.

Validation: WIP 40 tests; worker 730 unit + 96 integration tests serial; manifest 182 tests; workspace-root cargo check; cargo fmt --all -- --check; git diff --check HEAD.
2026-10-05 18:10:01 +09:00
Hare bbfbf92264 feat: bind Workdir attachments to durable connection lifetimes
Add caller-scoped bounded connection inventory and conditional detach on the existing ledger/session lifecycle. Preserve lifetime identity during attenuation and compensation, migrate persisted rows, and bind self operations to verified Runtime/Worker proof.

Validation: workspace-root cargo check; Backend 561 unit + 11 CLI tests; server-api 62 default + 69 typescript tests. Canonical OpenAPI and generated TypeScript source digests regenerated; browser declarations unchanged.
2026-10-05 18:10:01 +09:00
Hare 0a8f1f3b1b fix(workspace-server): drain Restore rechecks after reserved no-op checks
T-703 review follow-up: retire only the exact pre-Restore accepted ID, preserve pending deliveries, and retain an explicit deferred Restore recheck when another caller owns the reservation. Atomically release/take deferred work at completion and drain it without holding a Runtime lock.

Regression fixes the independently identified 3174142b interleaving: pause the real reservation before fingerprint comparison and after a no-op match, accept an actual fixture Restore, and verify exactly one new send after reservation completion. Also verify unknown delivery ID/content survive Restore.

Validation: cargo check; cargo test -p yoi-workspace-server (566 lib +11 bin); cargo test -p worker --lib notification_ (14); cargo fmt --all -- --check; git diff --check HEAD. Previous full Worker serial 703 lib+96 integration remains applicable (Worker unchanged in this fix). Tool schema rejection/Reviewer one-call barrier recorded in docs/report.
2026-10-05 18:01:10 +09:00
Hare 3174142bd1 fix(workspace-server): serialize explicit orchestrator backlog attention
T-703: snapshots establish baselines; only observed Running -> Idle rechecks backlog. Scope reservation and success/pending records to the exact Orchestrator; preserve payload and request ID across bounded retries and Restore races. Drop the launch lock before attention delivery.

Regression evidence: Barrier/parallel start, reconnect snapshots, retry payload identity, Restore/replacement; real Runtime adapter + Worker Controller + canonical Session fixture proves start/Restore send/receipt/commit counts and post-acceptance response-loss dedupe. Worker FsStore regression proves same-ID retry after restore versus legitimate new-ID same-body notification.

Validation: cargo check; cargo test -p yoi-workspace-server (564 lib + 11 bin); cargo test -p worker -- --test-threads=1 (703 lib + 96 integration); cargo fmt --all -- --check; git diff --check HEAD. Default parallel worker test reproduced existing pending_restore_reuses_persisted_session_and_recovers_unknown_attribution ScopeLock NotFound; isolated test and full serial suite pass.
2026-10-05 17:43:11 +09:00
Hare 5e4902db2b Merge latest develop after WIP namespace integration
Normal merge of develop@5a4c823c preserves integrated T-698 controller/native projections alongside the T-692/T-693 reconciliation. Target branch remains untouched.

Validation after this target movement: root cargo check; full worker(serial) and workspace-server tests; cargo fmt --all -- --check; git diff --check HEAD. Earlier reconciliation passed session-store, server-api generated/OpenAPI, full Web and production browser-shell checks.
2026-10-05 17:34:47 +09:00
Hare a4af26e8fd Merge develop into T-693 and reconcile Subject behavior with Memory information
Normal merge of develop@b0bce50b retains T-692 state hierarchy, diagnostics disclosures, ready-empty and candidate-only regressions alongside T-693 behavior/CAS/route fences. ResidentContext load and refresh retain the five-second timeout from 685b8aa6, with a seven-request regression.

Validation: root cargo check; full worker(serial), session-store, workspace-server and server-api(typescript) suites; frontend check with zero diagnostics; 464 Web tests and 171 component tests (19 Memory); production Memory browser-shell test/build; fmt/diff/OpenAPI checks.
2026-10-05 17:31:17 +09:00
Hare 5a4c823c40 feat: integrate host-owned WIP namespaces 2026-10-05 17:17:27 +09:00
Hare d3a1788445 fix: pin WIP cancellation to the active operation provider
Resolve WIP_CANCEL_ROUTING from MR 01a10ac8-c63a-7e33-bc27-07a52f87abe9. Retain the dispatched handler instance and selected operation per active execution; forward cancellation only to that contributor, not unrelated providers or shared-handler aliases. Keep cancellation terminal confirmation and failure classification unchanged.

Validation: contributed_cancellation tests (2 passed), all WIP tests (23 passed), root cargo check, cargo test -p worker -- --test-threads=1 (713 unit and 96 integration passed), cargo fmt --all -- --check, and git diff --check HEAD. Static/dynamic pending-call regressions cover unrelated cancel errors, shared handler duplication, exact dynamic handler identity, unknown IDs, terminal cleanup and cancelled audit.
2026-10-05 17:08:39 +09:00
Hare b0bce50bb4 chore: integrate Subject Memory information improvements 2026-10-05 17:08:32 +09:00
Hare 06480b9c7c Fix Subject behavior editor isolation across reused routes
Reset editor state for each loader identity and bind saves to their captured Subject and Workspace. Fence late success, error, and finally updates using the loader generation, including navigation back to the same Subject.

Validation: 14 Memory component tests (5 new route/workspace/in-flight isolation cases); frontend check with zero diagnostics; production Memory browser shell test; cargo check; cargo fmt --all -- --check; git diff --check HEAD. Restored implementation full Rust and frontend suites were independently rerun before this Web-only fix.
2026-10-05 16:59:13 +09:00
Hare b981e60231 test(web): document Subject Memory information decisions 2026-10-05 16:58:35 +09:00
Hare e17e81a9aa fix: enforce WIP namespace ownership and preserve collection identity
Restrict compatibility mounts to single tool routes, reject foreign dynamic Object owners, and prefer explicit Objects over synthetic namespaces during discovery. Document subsequent provider registration and keep reference resolution independent from enabled management operations.

Validation: cargo test -p worker --lib wip::tests (21 passed); root cargo check passed; cargo test -p worker -- --test-threads=1 passed (711 unit, 96 integration tests); cargo fmt --all -- --check and git diff --check HEAD passed. Initial parallel worker suite hit ScopeLock NotFound in pending_restore_reuses_persisted_session_and_recovers_unknown_attribution; the isolated test and full serial suite both passed.
2026-10-05 16:53:39 +09:00
Hare 8891236db3 Add persistent Subject behavior context 2026-10-05 16:27:23 +09:00
Hare 0c644c13e5 fix: preserve failed reviewer handoff evidence 2026-10-05 15:43:06 +09:00
Hare bc7b2f894c feat: expose WIP objects from host namespaces 2026-10-05 15:33:58 +09:00
Hare a00a0575f6 chore: merge develop into hare/develop 2026-10-05 14:06:04 +09:00
Hare 90c74df879 fix: enable indentation keys in Decodal editor 2026-10-05 14:05:45 +09:00
Hare b489a59057 fix: handle configuration schema bodies and editor load failures 2026-10-05 14:05:45 +09:00
Hare 4cf6c130cd fix(web): distinguish empty resident context 2026-10-05 13:59:39 +09:00
Hare fbb09e6597 feat(web): clarify Subject Memory status 2026-10-05 13:56:33 +09:00
Hare cc97987482 fix: preserve ambiguous legacy session attribution 2026-10-05 12:50:47 +09:00
Hare d2a0b75045 fix: finalize subjektiv attribution at session lifecycle 2026-10-05 12:25:20 +09:00
Hare fec85dd38d fix: simplify console mode controls with a compact footer toggle 2026-10-05 01:58:22 +09:00
Hare 685b8aa65f fix: prevent subject worker launch stalls 2026-10-05 01:22:58 +09:00
Hare bd79b93c1d chore: merge develop into hare/develop 2026-10-04 23:28:46 +09:00
Hare e741643717 fix: accept submission completion state in worker API parser 2026-10-04 23:27:50 +09:00
Hare 74a00f9edc fix: recover from provider image size rejections 2026-10-04 23:17:43 +09:00
Hare c6d2ad1f4c fix(test): format Subject browser evidence 2026-10-04 22:52:37 +09:00
Hare c8caa5518a fix(test): use shared Subject singleton key 2026-10-04 22:42:44 +09:00
Hare 3a9def7b80 Merge remote-tracking branch 'origin/develop' into work/T-689-web-subject-create 2026-10-04 22:35:43 +09:00
Hare 1c5966c535 feat(web): add Subject creation to Memory index 2026-10-04 22:35:21 +09:00
Hare 08881fa27a chore: merge develop into hare/develop 2026-10-04 21:03:16 +09:00
Hare 8a2a82d13c fix: close mobile sidebar after navigation 2026-10-04 21:03:16 +09:00
Hare b0d4973717 fix: align console metadata and details controls 2026-10-04 21:03:16 +09:00
Hare 728f03952e fix: recover job cleanup across runtime restart 2026-10-04 15:03:04 +09:00
Hare c7ae825a02 fix: persist worker run completion fence 2026-10-04 14:08:44 +09:00
Hare 8ad52e36c1 test: update worker state fixtures 2026-10-04 13:49:12 +09:00
Hare d8f3e46126 Merge remote-tracking branch 'origin/develop' into work/T-690-backend-job-worker-cleanup
# Conflicts:
#	web/workspace/src/lib/generated/protocol-validator.ts
2026-10-04 13:44:01 +09:00
Hare 20109b88df fix: fence cleanup on tracked run completion 2026-10-04 13:40:10 +09:00
Hare 773907462b Resolve Console images across session lineage 2026-10-04 13:14:36 +09:00
Hare 1fbf93c8c1 fix: fence job cleanup on run teardown 2026-10-04 13:12:39 +09:00
Hare 2eb9418a4e Separate Console image bodies from session snapshots 2026-10-04 12:56:40 +09:00
Hare d3eb7bc66e feat: clean up terminal backend job workers 2026-10-04 12:45:19 +09:00
Hare f7b722246d feat: configure subject attachment on worker launch 2026-10-04 08:40:30 +09:00
Hare 1bbc48dad0 feat(worker): project merge requests as native WIP objects 2026-10-04 03:10:42 +09:00
Hare 983c20c3f7 fix(worker): preserve Ticket WIP outcome semantics 2026-10-04 02:38:55 +09:00
Hare 31ee6b3c6a feat(worker): project tickets as native WIP objects 2026-10-04 02:22:45 +09:00
Hare 28c9d752e3 fix(worker): preserve Objective WIP authority semantics 2026-10-04 01:49:16 +09:00
Hare e5f255b6fd feat(worker): project objectives as native WIP objects 2026-10-04 01:28:45 +09:00
Hare e2181e9052 fix(worker): preserve typed native WIP contracts 2026-10-04 00:46:51 +09:00
Hare e44637519e fix(worker): forward native WIP arguments by descriptor name 2026-10-04 00:31:49 +09:00
Hare 4a05aad508 feat: add explicit Worker WIP mode 2026-10-04 00:26:15 +09:00
Hare 85a8944a6c fix: integrate feature-owned subjektiv restore refresh 2026-10-03 23:49:38 +09:00
Hare 8399d4f79c fix: integrate sidebar workspace alerts 2026-10-03 23:43:27 +09:00
Hare eea873e7dc fix: keep subjektiv restore refresh feature-owned 2026-10-03 23:37:23 +09:00
Hare bf918d1a11 fix: route sidebar failures through workspace alerts 2026-10-03 23:35:43 +09:00
Hare 29ed19fc8c Merge branch 'fix/compact-result-warning' into hare/develop 2026-10-03 21:57:18 +09:00
Hare 75fc7c921b fix: warn instead of failing on oversized compact results 2026-10-03 21:56:58 +09:00
Hare 3875c8e675 fix: preserve system item identity in live events 2026-10-03 21:44:36 +09:00
Hare 00c7e223b1 fix: preserve workdir link columns during schema migration 2026-10-03 21:01:55 +09:00
Hare f637667384 chore: merge develop into hare/develop 2026-10-03 19:14:17 +09:00
Hare 13bce155aa fix subject authority and bounded memory browsing 2026-10-03 01:18:24 +09:00
Hare f4966b78ea feat: cut over Memory to subjektiv product surfaces 2026-10-03 00:06:47 +09:00
Hare 07fcc249fe fix: resume exact reads within large segments 2026-10-02 18:11:51 +09:00
Hare ec7a6f59c8 fix: complete subject session paging 2026-10-02 18:00:30 +09:00
Hare c491cf8a48 fix: make subject session reads resumable 2026-10-02 17:36:10 +09:00
Hare ce841cfee0 Merge remote-tracking branch 'origin/develop' into work/T-673-subjektiv-session-history 2026-10-02 16:52:02 +09:00
Hare ee6aa9de41 feat: discover subject session history 2026-10-02 16:41:29 +09:00
Hare 3ae9b1c6d1 test(workspace-server): update greeting fixture 2026-10-02 16:36:06 +09:00
Hare 07cbff9a5f fix(console): keep live context metadata race-free 2026-10-02 16:25:43 +09:00
Hare 3d37f056ad fix: synchronize live worker context metadata 2026-10-02 16:04:55 +09:00
Hare cf3cf4b766 feat: expose worker model and context metadata 2026-10-02 15:29:45 +09:00
Hare 43e3b42055 feat: integrate bounded subject memory surfaces 2026-10-02 15:11:12 +09:00
Hare dfe837655a fix: preserve resident surface startup states 2026-10-02 15:01:09 +09:00
Hare aad200258c fix: integrate explicit Workspace repository key requirement 2026-10-02 14:43:57 +09:00
Hare 7b13303724 fix: preserve repository key for backend validation 2026-10-02 14:34:51 +09:00
Hare e662e2f465 fix: refresh memory surfaces on restore 2026-10-02 14:32:37 +09:00
Hare 138a8f4cec fix: stabilize persisted run transition identities 2026-10-02 14:31:04 +09:00
Hare 013f909b9b fix: require explicit Workspace repository key 2026-10-02 14:27:52 +09:00
Hare 504ea00ae3 fix: harden memory surface generation 2026-10-02 14:01:27 +09:00
Hare 860252c22d fix: report actual worker execution unavailability 2026-10-02 14:00:04 +09:00
Hare 539163acf3 feat: generate bounded subject memory surfaces 2026-10-02 13:25:40 +09:00
Hare 2d28c05447 chore: use 6-luna and raise compaction turn limit 2026-10-02 11:52:39 +09:00
Hare b74f1b495d Merge current develop into T-678 2026-10-02 11:31:52 +09:00
Hare 16fa3f14c9 Scope Worker resource keys by Runtime 2026-10-02 11:31:46 +09:00
Hare c631310192 Scope durable Worker identity by Runtime 2026-10-02 11:13:38 +09:00
Hare ba98a6afe6 fix: harden subject memory consolidation 2026-10-02 11:13:11 +09:00
Hare 046e606a38 feat: connect subject memory consolidation 2026-10-02 10:46:12 +09:00
Hare 2c329072a2 Replace Worker list revisions with ordered subscriptions 2026-10-02 10:41:02 +09:00
Hare 8f8c2676b4 fix: keep memory reads globally bounded 2026-10-02 09:52:32 +09:00
Hare 860c34b8da fix: bound memory read continuations 2026-10-02 09:26:53 +09:00
Hare f50a0caea9 fix: harden subject memory delivery 2026-10-02 09:05:11 +09:00
Hare 07f2b0e8a3 Merge remote-tracking branch 'origin/develop' into work/T-669-subjektiv-memory-tools 2026-10-02 07:39:25 +09:00
Hare b391392246 feat: add subject-scoped memory tools 2026-10-02 07:39:04 +09:00
Hare a0460f107d chore: upgrade frontend dependencies to SvelteKit 3 2026-10-02 07:36:25 +09:00
Hare 6d7eac940c fix(web): restore in-flight blocks through live projection 2026-10-02 07:18:58 +09:00
Hare 1f26a7f2bb fix: fence subjektiv completion side effects 2026-10-02 06:33:30 +09:00
Hare fede0fb51b chore: refresh subjektiv API artifacts 2026-10-02 06:18:02 +09:00
Hare 6e293d57cd test: cover subjektiv lifecycle boundaries 2026-10-02 06:18:02 +09:00
Hare 9a7bfba507 Merge remote-tracking branch 'origin/develop' into work/T-668-subjektiv-memory-lifecycle 2026-10-02 06:00:49 +09:00
Hare 6091fafcac feat: connect subjektiv extraction lifecycle 2026-10-02 06:00:31 +09:00
Hare d10783c0d6 fix: honor compaction cancellation and replay identity 2026-10-02 05:38:42 +09:00
Hare 88338229a1 fix: deduplicate compaction terminal failures 2026-10-02 05:30:17 +09:00
Hare d0ea9f598d fix: include idle-only input in runtime API contract 2026-10-02 03:33:30 +09:00
Hare d5f49cbf00 fix: restrict worker send input to idle admission 2026-10-02 02:39:10 +09:00
Hare 4ac448d66c fix: separate console pending groups and align row typography 2026-10-02 01:55:54 +09:00
Hare 17b7bdc769 chore: merge develop into hare/develop 2026-10-02 01:02:37 +09:00
Hare 62b0f87477 feat: integrate subjektiv revisioned memory store 2026-10-01 17:23:49 +09:00
Hare 1e59a07ea8 feat: integrate durable run transitions 2026-10-01 17:17:44 +09:00
Hare 014b948990 Add subjektiv revisioned memory store 2026-10-01 17:16:25 +09:00
Hare e43410f5e3 Fence lifecycle capture on cancelled runs 2026-10-01 17:12:02 +09:00
Hare 64ed50f6c0 Treat intentional cancellation as a non-failure terminal 2026-10-01 17:04:05 +09:00
Hare 0ec2b3ad86 Persist logical run yield and resume transitions 2026-10-01 16:44:11 +09:00
Hare 2f75176648 Fence feature operations during teardown 2026-10-01 16:37:46 +09:00
Hare 7797a65d54 Add server-managed feature SQLite storage 2026-10-01 16:27:30 +09:00
Hare f5d8c94713 fix: bind singleton retries to caller intent 2026-10-01 15:57:05 +09:00
Hare f351996d2c fix: harden singleton retries and authority 2026-10-01 15:46:18 +09:00
Hare 1afddc148d feat: enforce arbitrary worker singleton ownership 2026-10-01 15:27:05 +09:00
Hare d066f3963f fix: separate workspace settings display and editing 2026-10-01 15:14:57 +09:00
Hare 8e0dad98e8 fix: replace home navigation with work dashboard 2026-10-01 14:25:21 +09:00
Hare f742362b03 fix: capture ticket edit preimages atomically 2026-10-01 14:06:33 +09:00
Hare d101b4bacb fix: queue ticket checks with exact snapshots 2026-10-01 13:52:57 +09:00
Hare 696e6a51e7 fix: rebuild workspace home around daily navigation 2026-10-01 13:52:16 +09:00
Hare b9aaff4052 fix: synchronize sidebar pin and main layout animations 2026-10-01 13:36:19 +09:00
Hare f99da27891 fix: tighten sidebar footer spacing and inset divider 2026-10-01 13:30:22 +09:00
Hare d3c3f6f4d5 fix: exclude temporary Vitest configs from dev watcher 2026-10-01 13:23:32 +09:00
Hare 9579f3e245 feat: animate main layout and separate sidebar controls 2026-10-01 13:12:13 +09:00
Hare ab8b4ab1ef feat: check saved ticket requirements asynchronously 2026-10-01 13:06:57 +09:00
Hare 101b021c5f fix: anchor sidebar pin button at bottom left 2026-10-01 12:54:09 +09:00
Hare ee83fca95b fix: keep sidebar stationary when changing pin mode 2026-10-01 12:43:22 +09:00
Hare 5c98dbb208 feat: add pinned and hover sidebar modes 2026-10-01 12:36:39 +09:00
Hare b061ab68de Merge remote-tracking branch 'origin/work/T-674-backend-job-runner' into develop 2026-10-01 12:33:46 +09:00
Hare 63951398f0 fix durable job notification receipts 2026-10-01 12:26:30 +09:00
Hare 027a346e9d fix tracked job dispatch and delivery recovery 2026-10-01 11:58:49 +09:00
Hare 3ee1698957 feat: slide sidebar when expanding and folding 2026-10-01 11:48:50 +09:00
Hare a07d70b107 feat: persist sidebar fold preference across logins 2026-10-01 11:42:49 +09:00
Hare 213ed76061 fix backend job execution fencing and delivery 2026-10-01 11:18:14 +09:00
Hare eb4c7959a7 chore: replace GPT-6 Sol with GPT-6.1 Sol 2026-10-01 11:16:45 +09:00
Hare b5ae7dcd64 feat: animate console jump to latest button 2026-10-01 11:07:50 +09:00
Hare 1bc1f54ea0 feat: add console jump to latest icon 2026-10-01 10:59:13 +09:00
Hare 2ddf547308 fix: remove composer footer error messages 2026-10-01 10:09:52 +09:00
Hare ca838e4acf feat: add durable backend job runner 2026-10-01 10:06:55 +09:00
Hare 6b2f40a2a2 fix: isolate legacy external grants from workdir inventory 2026-09-30 14:42:29 +09:00
Hare 13f01b2495 style: apply workspace rustfmt 2026-09-30 14:28:42 +09:00
Hare 44d6d740be feat: default tool calls to early dispatch 2026-09-30 14:28:42 +09:00
Hare 3d87c3860a fix: remove overridden turn boundary line height 2026-09-30 14:16:58 +09:00
Hare 8627d45085 fix: reconcile console history before grouping activity 2026-09-30 14:16:58 +09:00
Hare cbf1b83599 fix: constrain external grep ignore ancestor matching 2026-09-30 13:58:39 +09:00
Hare 699a3c051c fix: restore centered turn navigation 2026-09-30 13:50:21 +09:00
Hare 5f8ae508fc fix: top-align turn bars and constrain history boundary 2026-09-30 13:45:45 +09:00
731 changed files with 211538 additions and 14210 deletions
+2
View File
@@ -19,6 +19,8 @@ LLM に投げる context はappend-onlyが基本であり、またその永続
## 検証
テストを追加・変更する前に、必ず[テスト設計規則](docs/development/rust-testing-strategy.md)を読み、その規則に従うこと。
開発中は、変更した契約を証明する最小の target / filter から実行する。
```sh
Generated
+353 -3
View File
@@ -64,6 +64,7 @@ dependencies = [
"dotenv",
"eventsource-stream",
"futures",
"imagesize",
"reqwest",
"schemars",
"serde",
@@ -88,6 +89,20 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "ahash"
version = "0.8.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
dependencies = [
"cfg-if",
"getrandom 0.3.4",
"once_cell",
"serde",
"version_check",
"zerocopy",
]
[[package]]
name = "aho-corasick"
version = "1.1.4"
@@ -183,6 +198,7 @@ dependencies = [
"serde_json",
"serde_urlencoded",
"tokio",
"tower",
"trybuild",
"url",
]
@@ -438,7 +454,16 @@ version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0700ddab506f33b20a03b13996eccd309a48e5ff77d0d95926aa0210fb4e95f1"
dependencies = [
"bit-vec",
"bit-vec 0.6.3",
]
[[package]]
name = "bit-set"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3"
dependencies = [
"bit-vec 0.8.0",
]
[[package]]
@@ -447,6 +472,12 @@ version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "349f9b6a179ed607305526ca489b34ad0a41aed5f7980fa90eb03160b69598fb"
[[package]]
name = "bit-vec"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
[[package]]
name = "bitflags"
version = "1.3.2"
@@ -496,6 +527,12 @@ dependencies = [
"cipher",
]
[[package]]
name = "borrow-or-share"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc0b364ead1874514c8c2855ab558056ebfeb775653e7ae45ff72f28f8f3166c"
[[package]]
name = "bstr"
version = "1.12.1"
@@ -507,6 +544,13 @@ dependencies = [
"serde",
]
[[package]]
name = "builtin-source"
version = "0.1.0"
dependencies = [
"sha2 0.11.0",
]
[[package]]
name = "bumpalo"
version = "3.20.2"
@@ -730,6 +774,7 @@ dependencies = [
name = "config-source"
version = "0.1.0"
dependencies = [
"builtin-source",
"decodal",
"decodal-language-service",
"decodal-language-tools",
@@ -1249,6 +1294,15 @@ dependencies = [
"zeroize",
]
[[package]]
name = "email_address"
version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e079f19b08ca6239f47f8ba8509c11cf3ea30095831f7fed61441475edd8c449"
dependencies = [
"serde",
]
[[package]]
name = "encoding_rs"
version = "0.8.35"
@@ -1330,16 +1384,37 @@ version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b95f7c0680e4142284cf8b22c14a476e87d61b004a3a0861872b32ef7ead40a2"
dependencies = [
"bit-set",
"bit-set 0.5.3",
"regex",
]
[[package]]
name = "fancy-regex"
version = "0.16.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "998b056554fbe42e03ae0e152895cd1a7e1002aec800fdc6635d20270260c46f"
dependencies = [
"bit-set 0.8.0",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "fastrand"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be"
[[package]]
name = "feature-storage"
version = "0.1.0"
dependencies = [
"rusqlite",
"tempfile",
"thiserror 2.0.18",
"uuid",
]
[[package]]
name = "ff"
version = "0.13.1"
@@ -1418,6 +1493,17 @@ dependencies = [
"thiserror 2.0.18",
]
[[package]]
name = "fluent-uri"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1918b65d96df47d3591bed19c5cca17e3fa5d0707318e4b5ef2eae01764df7e5"
dependencies = [
"borrow-or-share",
"ref-cast",
"serde",
]
[[package]]
name = "fnv"
version = "1.0.7"
@@ -1460,6 +1546,16 @@ dependencies = [
"percent-encoding",
]
[[package]]
name = "fraction"
version = "0.15.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e076045bb43dac435333ed5f04caf35c7463631d0dae2deb2638d94dd0a5b872"
dependencies = [
"lazy_static",
"num",
]
[[package]]
name = "fs-operation"
version = "0.1.0"
@@ -1871,6 +1967,12 @@ version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
[[package]]
name = "humantime"
version = "2.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "15cdd26707701c53297e2fa6afb323d55fbc1d0810c3aec078ae3ef0424c3c15"
[[package]]
name = "hybrid-array"
version = "0.4.10"
@@ -2112,6 +2214,12 @@ dependencies = [
"winapi-util",
]
[[package]]
name = "imagesize"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09e54e57b4c48b40f7aec75635392b12b3421fa26fe8b4332e63138ed278459c"
[[package]]
name = "include_dir"
version = "0.7.4"
@@ -2256,6 +2364,17 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "job"
version = "0.1.0"
dependencies = [
"manifest",
"serde",
"serde_json",
"sha2 0.11.0",
"thiserror 2.0.18",
]
[[package]]
name = "jobserver"
version = "0.1.34"
@@ -2278,6 +2397,32 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "jsonschema"
version = "0.33.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d46662859bc5f60a145b75f4632fbadc84e829e45df6c5de74cfc8e05acb96b5"
dependencies = [
"ahash",
"base64 0.22.1",
"bytecount",
"email_address",
"fancy-regex 0.16.2",
"fraction",
"idna",
"itoa",
"num-cmp",
"num-traits",
"once_cell",
"percent-encoding",
"referencing",
"regex",
"regex-syntax",
"serde",
"serde_json",
"uuid-simd",
]
[[package]]
name = "kasuari"
version = "0.4.12"
@@ -2468,6 +2613,7 @@ version = "0.1.0"
dependencies = [
"agen",
"arc-swap",
"builtin-source",
"decodal",
"protocol",
"secrets",
@@ -2730,6 +2876,20 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "num"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35bd024e8b2ff75562e5f34e7f4905839deb4b22955ef5e73d2fea1b9813cb23"
dependencies = [
"num-bigint",
"num-complex",
"num-integer",
"num-iter",
"num-rational",
"num-traits",
]
[[package]]
name = "num-bigint"
version = "0.4.8"
@@ -2756,6 +2916,21 @@ dependencies = [
"zeroize",
]
[[package]]
name = "num-cmp"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63335b2e2c34fae2fb0aa2cecfd9f0832a1e24b3b32ecec612c3426d46dc8aaa"
[[package]]
name = "num-complex"
version = "0.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495"
dependencies = [
"num-traits",
]
[[package]]
name = "num-conv"
version = "0.2.1"
@@ -2792,6 +2967,17 @@ dependencies = [
"num-traits",
]
[[package]]
name = "num-rational"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f83d14da390562dca69fc84082e73e548e1ad308d24accdedd2720017cb37824"
dependencies = [
"num-bigint",
"num-integer",
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.19"
@@ -2845,6 +3031,32 @@ dependencies = [
"url",
]
[[package]]
name = "object_store"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "622acbc9100d3c10e2ee15804b0caa40e55c933d5aa53814cd520805b7958a49"
dependencies = [
"async-trait",
"bytes",
"chrono",
"futures-channel",
"futures-core",
"futures-util",
"http",
"humantime",
"itertools",
"parking_lot",
"percent-encoding",
"thiserror 2.0.18",
"tokio",
"tracing",
"url",
"walkdir",
"wasm-bindgen-futures",
"web-time",
]
[[package]]
name = "oid-registry"
version = "0.7.1"
@@ -2925,6 +3137,12 @@ dependencies = [
"num-traits",
]
[[package]]
name = "outref"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a80800c0488c3a21695ea981a54918fbb37abf04f4d0720c453632255e2ff0e"
[[package]]
name = "p256"
version = "0.13.2"
@@ -3335,6 +3553,7 @@ dependencies = [
"serde",
"serde_json",
"sha2 0.11.0",
"thiserror 2.0.18",
"tokio",
"ts-rs",
"uuid",
@@ -3628,6 +3847,20 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "referencing"
version = "0.33.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e9c261f7ce75418b3beadfb3f0eb1299fe8eb9640deba45ffa2cb783098697d"
dependencies = [
"ahash",
"fluent-uri",
"once_cell",
"parking_lot",
"percent-encoding",
"serde_json",
]
[[package]]
name = "regex"
version = "1.12.3"
@@ -4453,13 +4686,20 @@ dependencies = [
"agen",
"async-trait",
"client",
"feature-storage",
"fs4",
"futures",
"job",
"libc",
"manifest",
"memory",
"protocol",
"rusqlite",
"serde",
"serde_json",
"server-api",
"session-store",
"subjektiv",
"tempfile",
"thiserror 2.0.18",
"tokio",
@@ -4536,6 +4776,23 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "subjektiv"
version = "0.1.0"
dependencies = [
"chrono",
"feature-storage",
"manifest",
"memory",
"rusqlite",
"serde",
"serde_json",
"server-api",
"tempfile",
"thiserror 2.0.18",
"uuid",
]
[[package]]
name = "subtle"
version = "2.6.1"
@@ -4674,7 +4931,7 @@ dependencies = [
"anyhow",
"base64 0.22.1",
"bitflags 2.11.0",
"fancy-regex",
"fancy-regex 0.11.0",
"filedescriptor",
"finl_unicode",
"fixedbitset",
@@ -5205,6 +5462,7 @@ dependencies = [
"toml",
"unicode-width",
"uuid",
"worker-runtime",
]
[[package]]
@@ -5409,6 +5667,17 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "uuid-simd"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23b082222b4f6619906941c17eb2297fff4c2fb96cb60164170522942a200bd8"
dependencies = [
"outref",
"uuid",
"vsimd",
]
[[package]]
name = "valuable"
version = "0.1.1"
@@ -5427,6 +5696,12 @@ version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "vsimd"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c3082ca00d5a5ef149bb8b555a72ae84c9c59f7250f013ac822ac2e49b19c64"
[[package]]
name = "vtparse"
version = "0.6.2"
@@ -6063,6 +6338,45 @@ dependencies = [
"memchr",
]
[[package]]
name = "wip-client"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "def7dc8d056477bf8cb7c12f2dc70cabdb3eee3527cf95ba6131a8a44519ed7b"
dependencies = [
"wip-http",
"wip-protocol",
]
[[package]]
name = "wip-http"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da1f6a0023c0c599cbdc5571e9b131c8bb6b113ef04c20467d77634d66b44a5a"
dependencies = [
"base64 0.22.1",
"http",
"serde",
"serde_json",
"url",
"wip-protocol",
]
[[package]]
name = "wip-protocol"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a23ee0fe02badd2ef77847652cff241c6d868c8f162a911475500831ff348fe"
[[package]]
name = "wip-text-view"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c95fd8e3fba673de44084c2f89c11b50d744c6db73726c50e04975c974d2759c"
dependencies = [
"wip-protocol",
]
[[package]]
name = "wiremock"
version = "0.6.5"
@@ -6200,6 +6514,8 @@ dependencies = [
"agen",
"arc-swap",
"async-trait",
"axum",
"base64 0.22.1",
"chrono",
"clap",
"client",
@@ -6210,7 +6526,10 @@ dependencies = [
"fs4",
"futures",
"futures-util",
"http",
"include_dir",
"job",
"jsonschema",
"libc",
"manifest",
"mcp",
@@ -6237,6 +6556,10 @@ dependencies = [
"tracing",
"tungstenite 0.28.0",
"uuid",
"wip-client",
"wip-http",
"wip-protocol",
"wip-text-view",
"workdir",
]
@@ -6252,6 +6575,7 @@ dependencies = [
"flow",
"fs4",
"futures",
"libc",
"manifest",
"protocol",
"reqwest",
@@ -6279,6 +6603,24 @@ dependencies = [
"zeroize",
]
[[package]]
name = "workspace-drive"
version = "0.1.0"
dependencies = [
"bytes",
"feature-storage",
"futures",
"object_store",
"rusqlite",
"serde",
"serde_json",
"sha2 0.11.0",
"tempfile",
"thiserror 2.0.18",
"tokio",
"uuid",
]
[[package]]
name = "writeable"
version = "0.6.3"
@@ -6344,6 +6686,7 @@ dependencies = [
"server-api",
"session-analytics",
"session-store",
"standalone",
"tempfile",
"ticket",
"tokio",
@@ -6384,12 +6727,15 @@ name = "yoi-workspace-server"
version = "0.1.0"
dependencies = [
"agen",
"api-macros",
"async-trait",
"axum",
"chrono",
"config-source",
"feature-storage",
"flow",
"futures",
"job",
"manifest",
"memory",
"merge-request",
@@ -6403,8 +6749,10 @@ dependencies = [
"serde_json",
"serde_yaml",
"server-api",
"session-store",
"sha2 0.11.0",
"ssh-key",
"subjektiv",
"tempfile",
"thiserror 2.0.18",
"ticket",
@@ -6420,9 +6768,11 @@ dependencies = [
"uuid",
"webauthn-authenticator-rs",
"webauthn-rs",
"wip-protocol",
"workdir",
"worker",
"worker-runtime",
"workspace-drive",
"yoi",
"zeroize",
]
+28
View File
@@ -8,6 +8,7 @@ members = [
"crates/api-macros-impl",
"crates/session-store",
"crates/standalone",
"crates/job",
"crates/secrets",
"crates/manifest",
"crates/mcp",
@@ -24,11 +25,15 @@ members = [
"crates/tools",
"crates/fs-operation",
"crates/flow",
"crates/builtin-source",
"crates/config-source",
"crates/config-source-wasm",
"crates/workdir",
"crates/tui",
"crates/memory",
"crates/feature-storage",
"crates/workspace-drive",
"crates/subjektiv",
"crates/ticket",
"crates/merge-request",
"crates/project-record",
@@ -43,6 +48,7 @@ default-members = [
"crates/api-macros-impl",
"crates/session-store",
"crates/standalone",
"crates/job",
"crates/secrets",
"crates/manifest",
"crates/mcp",
@@ -59,11 +65,15 @@ default-members = [
"crates/tools",
"crates/fs-operation",
"crates/flow",
"crates/builtin-source",
"crates/config-source",
"crates/config-source-wasm",
"crates/workdir",
"crates/tui",
"crates/memory",
"crates/feature-storage",
"crates/workspace-drive",
"crates/subjektiv",
"crates/ticket",
"crates/merge-request",
"crates/project-record",
@@ -73,6 +83,7 @@ default-members = [
[workspace.package]
edition = "2024"
license = "MIT"
rust-version = "1.88"
[profile.dev]
debug = "line-tables-only"
@@ -90,6 +101,9 @@ manifest = { path = "crates/manifest" }
mcp = { path = "crates/mcp" }
lint-common = { path = "crates/lint-common" }
memory = { path = "crates/memory" }
feature-storage = { path = "crates/feature-storage" }
workspace-drive = { path = "crates/workspace-drive" }
subjektiv = { path = "crates/subjektiv" }
merge-request = { path = "crates/merge-request" }
ticket = { path = "crates/ticket" }
project-record = { path = "crates/project-record" }
@@ -102,10 +116,12 @@ session-metrics = { path = "crates/session-metrics" }
session-analytics = { path = "crates/session-analytics" }
session-store = { path = "crates/session-store" }
standalone = { path = "crates/standalone" }
job = { path = "crates/job" }
runtime-api = { path = "crates/runtime-api" }
server-api = { path = "crates/server-api" }
secrets = { path = "crates/secrets" }
tools = { path = "crates/tools" }
builtin-source = { path = "crates/builtin-source" }
config-source = { path = "crates/config-source" }
fs-operation = { path = "crates/fs-operation" }
workdir = { path = "crates/workdir" }
@@ -124,6 +140,7 @@ decodal-language-service = "0.4.0"
decodal-language-tools = "0.4.0"
fs4 = "0.13"
futures = "0.3"
http = "1"
libc = "0.2"
schemars = "1.2"
serde = "1.0"
@@ -144,5 +161,16 @@ tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
url = "2.5"
uuid = "1.23"
# Released structured scope/lifetime contract and canonical Text View, one registry source.
wip-client = "=0.2.0"
wip-http = "=0.2.0"
wip-protocol = "=0.2.0"
wip-text-view = "=0.2.0"
zeroize = "1"
webauthn-rs = { version = "0.5.2", features = ["danger-allow-state-serialisation", "danger-credential-internals"] }
[workspace.metadata.wip-upstream]
repository = "https://gitea.hareworks.net/Hare/wip-rs"
version = "0.2.0"
reference-revision = "6086f3c5ef10aa1464ce9c824750d7f667217bfd"
source = "crates.io"
+1
View File
@@ -32,6 +32,7 @@ reqwest = { version = "0.13", default-features = false, features = ["stream", "j
eventsource-stream = "0.2"
zstd = "0.13"
base64 = "0.22.1"
imagesize = "0.14"
chrono = { version = "0.4", default-features = false, features = ["serde", "clock"], optional = true }
agen-macros = { workspace = true }
+106 -11
View File
@@ -83,9 +83,9 @@ pub enum ToolRegistryError {
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum ToolCallDispatchMode {
/// Wait for the complete assistant response before starting any tools.
#[default]
AfterResponse,
/// Start calls at trustworthy provider per-call completion boundaries.
#[default]
OnToolCallComplete,
}
@@ -626,6 +626,10 @@ pub struct Engine<C: LlmClient, S: EngineState = Mutable, A: Send + Sync = ()> {
/// engine history. An error rejects the item and aborts the turn, allowing
/// upper layers to make durable storage the commit gate.
history_append_cbs: Vec<Box<dyn Fn(&Item) -> Result<(), String> + Send + Sync>>,
/// Durable host commit gate for image-result corrections. Runs before the
/// correction becomes live or a replacement request can be sent.
image_rejection_handler:
Option<Box<dyn Fn(&HistoryEntry<A>, &Item) -> Result<(), String> + Send + Sync>>,
/// Request configuration (max_tokens, temperature, etc.)
request_config: RequestConfig,
/// Cancel notification channel (for interrupting execution)
@@ -933,7 +937,8 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
///
/// The callback is invoked with a short human-readable message
/// whenever the Engine encounters a condition that should be
/// surfaced to a human (e.g. tool output byte-cap truncation).
/// surfaced to a human (e.g. a provider-rejected image). Tool output
/// byte-cap truncation is reported in the tool result, not this channel.
/// This channel is separate from `tracing::warn!`, which remains
/// in place for developer logs.
pub fn on_warning(&mut self, callback: impl Fn(&str) + Send + Sync + 'static) {
@@ -971,6 +976,16 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
self.history_append_cbs.push(Box::new(callback));
}
/// Set the host's durable commit gate for correcting a rejected ViewImage
/// result. The original annotation/identity must be preserved. A failed
/// commit leaves both history and the outgoing request untouched.
pub fn set_image_rejection_handler(
&mut self,
handler: impl Fn(&HistoryEntry<A>, &Item) -> Result<(), String> + Send + Sync + 'static,
) {
self.image_rejection_handler = Some(Box::new(handler));
}
fn emit_history_append(&self, item: &Item) -> Result<(), EngineError> {
for cb in &self.history_append_cbs {
cb(item).map_err(EngineError::HistoryAppend)?;
@@ -2365,13 +2380,9 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
limit_bytes = limit,
"Tool output exceeded byte limit and was truncated"
);
self.emit_warning(&format!(
"tool `{}` output truncated from {} to {} bytes (limit {})",
tool_call.name,
before,
content.len(),
limit
));
// Truncation is a tool-result constraint, not an actionable
// engine warning. The content marker tells the model to refine
// its query; keep diagnostics in tracing, not user alerts.
}
}
@@ -3188,6 +3199,55 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
}
}
/// Correct one candidate actually sent in the rejected request. Commit the
/// durable correction before changing either in-memory projection or retrying.
fn recover_rejected_image(
&self,
history: &mut History<A>,
request: &mut Request,
) -> Result<bool, EngineError> {
let Some((history_index, request_index)) =
crate::image_recovery::largest_candidate(history, request)
else {
return Ok(false);
};
let entry = &history.entries()[history_index];
let replacement = crate::image_recovery::rejected_result(&entry.item);
if let Some(commit) = &self.image_rejection_handler {
commit(entry, &replacement).map_err(EngineError::HistoryAppend)?;
}
history.entries_mut()[history_index].item = replacement.clone();
// Retain the original request's prune/interceptor decisions.
request.items[request_index] = replacement;
self.emit_warning(crate::image_recovery::REJECTION_MESSAGE);
Ok(true)
}
/// Opening a corrected request can itself yield an HTTP image rejection.
/// Each successful correction removes one candidate, bounding this loop.
async fn open_stream_with_image_recovery(
&mut self,
request: &mut Request,
turn: usize,
llm_call: usize,
history: &mut History<A>,
) -> Result<ResponseStream, EngineError> {
loop {
match self
.open_stream_with_retry(request.clone(), turn, llm_call)
.await
{
Ok(stream) => return Ok(stream),
Err(EngineError::Client(error)) if error.is_image_size_rejection() => {
if !self.recover_rejected_image(history, request)? {
return Err(EngineError::Client(error));
}
}
Err(error) => return Err(error),
}
}
}
/// Open a stream, dispatch all events to the timeline, handle cancellation.
async fn stream_response(
&mut self,
@@ -3205,11 +3265,15 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
"Sending request to LLM"
);
let stream = self.open_stream_with_retry(request, turn, llm_call).await?;
let mut request = request;
let stream = self
.open_stream_with_image_recovery(&mut request, turn, llm_call, history)
.await?;
self.tool_call_collector.begin_response();
let mut stream = ResponseStreamPump::start(stream);
let mut early_tools: Option<EarlyToolExecutionBatch> = None;
let mut response_completed = false;
let mut response_progress = false;
let mut event_count: usize = 0;
loop {
@@ -3247,6 +3311,35 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
json!({}),
);
}
self.emit_stream_event(turn, llm_call, &event);
// Only pre-output provider rejection can safely replay the
// original request. Even an empty block start closes this
// window, before collectors or early tool admission run.
// Unknown events are conservatively not replayable either.
match &event {
Event::Ping(_) | Event::Usage(_) | Event::Error(_) => {}
Event::Status(StatusEvent {
status: crate::llm_client::event::ResponseStatus::Started,
}) => {}
_ => response_progress = true,
}
if let Event::Error(error) = &event {
if !response_progress && error.is_image_size_rejection() {
self.timeline.flush_usage();
if self.recover_rejected_image(history, &mut request)? {
// Drop the old pump (including queued failed status)
// before opening the corrected request. No collector or
// tool state exists for this pre-output attempt.
drop(stream);
stream = ResponseStreamPump::start(self.open_stream_with_image_recovery(
&mut request, turn, llm_call, history,
).await?);
self.tool_call_collector.begin_response();
event_count = 0;
continue;
}
}
}
if matches!(
&event,
Event::Status(StatusEvent {
@@ -3255,7 +3348,6 @@ impl<C: LlmClient, S: EngineState, A: Send + Sync> Engine<C, S, A> {
) {
response_completed = true;
}
self.emit_stream_event(turn, llm_call, &event);
self.timeline.dispatch(&event);
if early_dispatch {
let collected = self.tool_call_collector.take_collected_with_index();
@@ -3567,6 +3659,7 @@ impl<C: LlmClient, A: Send + Sync> Engine<C, Mutable, A> {
warning_cbs: Vec::new(),
tool_result_cbs: Vec::new(),
history_append_cbs: Vec::new(),
image_rejection_handler: None,
request_config: RequestConfig::default(),
cancel_tx,
cancel_rx,
@@ -3856,6 +3949,7 @@ impl<C: LlmClient, A: Send + Sync> Engine<C, Mutable, A> {
warning_cbs: self.warning_cbs,
tool_result_cbs: self.tool_result_cbs,
history_append_cbs: self.history_append_cbs,
image_rejection_handler: self.image_rejection_handler,
request_config: self.request_config,
cancel_tx: self.cancel_tx,
@@ -4040,6 +4134,7 @@ impl<C: LlmClient, A: Send + Sync> Engine<C, Locked, A> {
warning_cbs: self.warning_cbs,
tool_result_cbs: self.tool_result_cbs,
history_append_cbs: self.history_append_cbs,
image_rejection_handler: self.image_rejection_handler,
request_config: self.request_config,
cancel_tx: self.cancel_tx,
+106
View File
@@ -0,0 +1,106 @@
//! Recovery is driven by provider rejection, never a locally chosen image limit.
use crate::{History, Item, Role, ToolResultDisposition, llm_client::Request, tool::Attachment};
pub(crate) const REJECTION_MESSAGE: &str = "ViewImage image omitted from model input after the provider rejected image size. This is a recovery candidate, not proof that this particular image caused the rejection. Resize or crop the image and retry ViewImage; do not read the unchanged image again.";
/// Keep the most recent response's tool results eligible, including early tool
/// results interleaved with that response's later text/calls. Empty boundaries
/// from older versions' failed requests are not evidence of accepted input.
pub(crate) fn accepted_input_end<'a>(items: impl Iterator<Item = &'a Item>) -> Option<usize> {
let mut response_start = None;
let mut accepted_prefix = None;
for (index, item) in items.enumerate() {
match item {
Item::AssistantResponseBoundary { .. } => response_start = Some(index),
Item::Message {
role: Role::Assistant,
..
}
| Item::ToolCall { .. }
| Item::Reasoning { .. } => {
accepted_prefix = Some(response_start.unwrap_or(index));
}
Item::Message {
role: Role::User, ..
} => response_start = None,
_ => {}
}
}
accepted_prefix
}
/// Select only a ViewImage result actually sent in the rejected request. The
/// tool contract returns exactly one image, so each correction removes one
/// image and preserves the tool call/result pair. Other tools and user uploads
/// are deliberately not rewritten by this recovery policy.
pub(crate) fn largest_candidate<A>(
history: &History<A>,
request: &Request,
) -> Option<(usize, usize)> {
let start = accepted_input_end(history.items()).unwrap_or(0);
let mut largest = None;
for (index, entry) in history.entries().iter().enumerate().skip(start) {
let Item::ToolResult {
call_id,
attachments,
..
} = &entry.item
else {
continue;
};
let [Attachment::Image(image)] = attachments.as_slice() else {
continue;
};
let from_view_image = history.items().any(|item| {
matches!(
item,
Item::ToolCall { call_id: id, name, .. } if id == call_id && name == "ViewImage"
)
});
if !from_view_image {
continue;
}
let sent = request.items.iter().position(|item| {
matches!(
item,
Item::ToolResult { call_id: id, attachments: sent, .. }
if id == call_id && sent == attachments
)
});
let Some(request_index) = sent else { continue };
// Header inspection only; no raster allocation or size threshold.
// Unknown dimensions sort by bytes as a deterministic fallback.
let pixels = imagesize::blob_size(image.data())
.ok()
.map(|size| (size.width as u64).saturating_mul(size.height as u64))
.unwrap_or(0);
let rank = (pixels, image.data().len(), index);
if largest
.as_ref()
.is_none_or(|(previous, _)| rank > *previous)
{
largest = Some((rank, (index, request_index)));
}
}
largest.map(|(_, candidate)| candidate)
}
pub(crate) fn rejected_result(item: &Item) -> Item {
let mut item = item.clone();
if let Item::ToolResult {
summary,
content,
attachments,
disposition,
is_error,
..
} = &mut item
{
*summary = REJECTION_MESSAGE.to_owned();
*content = None;
attachments.clear();
*disposition = ToolResultDisposition::Error;
*is_error = true;
}
item
}
+1
View File
@@ -3,6 +3,7 @@
mod engine;
mod handler;
mod history;
mod image_recovery;
mod message;
pub(crate) mod callback;
+39
View File
@@ -90,6 +90,22 @@ impl ClientError {
}
}
/// A provider-declared image size rejection, not a generic invalid input.
/// Some providers (including OpenAI) expose only `invalid_value` here, so
/// narrowly recognize their image-size diagnostic without hardcoded limits.
pub fn is_image_size_rejection(&self) -> bool {
let Self::Api {
status: Some(400 | 413 | 422),
code,
message,
..
} = self
else {
return false;
};
is_image_size_diagnostic(code.as_deref(), message)
}
pub fn retry_after(&self) -> Option<Duration> {
match self {
ClientError::Api { retry_after, .. } => *retry_after,
@@ -98,6 +114,29 @@ impl ClientError {
}
}
/// Shared diagnostic policy for HTTP rejections and provider Error events.
/// The caller must establish the rejection path and whether replay is safe;
/// this predicate alone never authorizes a retry.
pub(crate) fn is_image_size_diagnostic(code: Option<&str>, message: &str) -> bool {
if matches!(code, Some("image_too_large" | "image_size_exceeded")) {
return true;
}
// OpenAI Responses appends structured diagnostic details for observation.
// Extra fields (including echoed input) are not the error message itself.
let message = message
.split(" | diagnostic=")
.next()
.unwrap_or(message)
.to_ascii_lowercase();
message.contains("image")
&& ((message.contains("patches")
&& message.contains("exceed")
&& message.contains("limit"))
|| message.contains("image is too large")
|| message.contains("image too large")
|| (message.contains("image dimensions") && message.contains("exceed")))
}
/// transient な失敗としてリトライ対象になるかを判定する。
///
/// 対象:
+9
View File
@@ -109,6 +109,15 @@ pub struct ErrorEvent {
pub message: String,
}
impl ErrorEvent {
/// A provider Error event has no HTTP status. Classify its diagnostic
/// explicitly, rather than making every status-less ClientError eligible.
/// The engine must separately establish that no response output was seen.
pub(crate) fn is_image_size_rejection(&self) -> bool {
super::error::is_image_size_diagnostic(self.code.as_deref(), &self.message)
}
}
/// 未対応 SSE イベントの観測用メタイベント。
///
/// `data_preview` は provider から受け取った raw SSE data の bounded preview、
@@ -753,17 +753,31 @@ pub(crate) fn parse_sse(
}
"error" => {
let ev = from_json::<TopLevelErrorEnvelope>(data).unwrap_or_else(|_| {
TopLevelErrorEnvelope {
// Responses documents flat error frames (code/message/param),
// while providers also send nested error envelopes. Normalize both
// structurally so classification never scans arbitrary raw JSON.
let ev = from_json::<TopLevelErrorEnvelope>(data)
.or_else(|_| {
from_json::<TopLevelError>(data).map(|error| TopLevelErrorEnvelope {
error,
extra: BTreeMap::new(),
})
})
.unwrap_or_else(|_| TopLevelErrorEnvelope {
error: TopLevelError {
message: Some(data.to_string()),
// Raw data is observation, not a provider diagnostic that
// can authorize request recovery. Keep it out of the base
// message even when it contains image-size wording.
message: Some("OpenAI Responses error with unparseable data".to_string()),
error_type: None,
code: None,
extra: BTreeMap::new(),
extra: BTreeMap::from([(
"raw_data".to_string(),
Value::String(data.to_string()),
)]),
},
extra: BTreeMap::new(),
}
});
});
let (code, message) = top_level_error_diagnostic(ev);
Ok(vec![Event::Error(ErrorEvent { code, message })])
}
+6 -7
View File
@@ -136,14 +136,13 @@ pub fn project(items: &mut [Item], indices: &[usize]) -> usize {
/// Remove binary attachments after the first model request that could consume them.
///
/// Attachments on tool results following the latest model output are fresh outputs for the
/// immediate follow-up request and remain visible. Once a later assistant/reasoning/tool-call
/// item exists, the model has already had an opportunity to inspect the attachment. Subsequent
/// requests keep the durable tool summary and can call the originating tool again when the image
/// is still needed. Only the request-context clone is modified.
/// immediate follow-up request and remain visible. With early dispatch, later
/// text/calls in the same response have not consumed its tool results: use the
/// confirmed response boundary, not that response's last individual item.
/// Subsequent requests keep the durable tool summary and can call the originating
/// tool again when the image is still needed. Only the request-context clone is modified.
pub fn project_consumed_attachments(items: &mut [Item]) -> usize {
let latest_model_output = items.iter().rposition(|item| {
item.is_assistant_message() || item.is_reasoning() || item.is_tool_call()
});
let latest_model_output = crate::image_recovery::accepted_input_end(items.iter());
let mut count = 0;
for (index, item) in items.iter_mut().enumerate() {
let is_fresh = latest_model_output.is_some_and(|boundary| index > boundary);
+3
View File
@@ -30,6 +30,9 @@ pub enum ToolError {
/// Internal error
#[error("Internal error: {0}")]
Internal(String),
/// Machine-readable domain conflict that a tool adapter may project as structured output.
#[error("{code}: {message}")]
StructuredConflict { code: String, message: String },
/// Cooperative cancellation completed with bounded terminal output.
#[error("Tool execution cancelled")]
Cancelled(ToolOutput),
+176
View File
@@ -16,6 +16,182 @@ use agen::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use async_trait::async_trait;
use common::MockLlmClient;
/// A per-run tracing sink: no global subscriber or output-body logging.
#[derive(Clone)]
struct DiagnosticLog(Arc<Mutex<Vec<u8>>>);
impl std::io::Write for DiagnosticLog {
fn write(&mut self, bytes: &[u8]) -> std::io::Result<usize> {
self.0.lock().unwrap().extend_from_slice(bytes);
Ok(bytes.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
#[tokio::test]
async fn output_limits_keep_partial_results_without_warning_callbacks() {
use agen::tool::{Attachment, ImageAttachment, ToolOutputLimits, ToolResult};
use agen::{History, Item, ToolResultDisposition};
use tracing::instrument::WithSubscriber;
let limit = 256;
let cases = [
("below", "small result".to_string(), limit),
("exact", "x".repeat(limit), limit),
("Grep", "matched line\n".repeat(200), limit),
("Read", "あいうえお".repeat(200), 120),
];
let mut events = Vec::new();
for (index, name) in cases.iter().map(|c| c.0).chain(["erroring"]).enumerate() {
events.extend([
Event::tool_use_start(index, name, name),
Event::tool_input_delta(index, "{}"),
Event::tool_use_stop(index),
]);
}
events.push(Event::Status(ClientStatusEvent {
status: ResponseStatus::Completed,
}));
let client = MockLlmClient::with_responses(vec![
events,
vec![Event::Status(ClientStatusEvent {
status: ResponseStatus::Completed,
})],
]);
let probe = client.clone();
let mut engine = Engine::new(client);
engine.set_tool_call_dispatch_mode(agen::ToolCallDispatchMode::AfterResponse);
engine.set_tool_output_limits(Some(ToolOutputLimits {
default_max_bytes: limit,
per_tool: [("Read".to_string(), 120)].into(),
}));
let attachments = vec![Attachment::Image(ImageAttachment::new(
"image/png",
b"\x89PNG\r\n\x1a\n\0\0\0\rIHDR\0\0\0\x01\0\0\0\x01".to_vec(),
))];
for (name, content, _) in &cases {
engine.register_tool(fixed_tool(
name,
ToolOutput {
summary: format!("result from {name}"),
content: Some(content.clone()),
attachments: attachments.clone(),
},
));
}
engine.register_tool(erroring_tool("erroring", "execution failed"));
let warnings = Arc::new(Mutex::new(Vec::new()));
let sink = warnings.clone();
engine.on_warning(move |warning| sink.lock().unwrap().push(warning.to_owned()));
let published = Arc::new(Mutex::new(Vec::<ToolResult>::new()));
let sink = published.clone();
engine.on_tool_result(move |result| sink.lock().unwrap().push(result.clone()));
let committed = Arc::new(Mutex::new(Vec::new()));
let sink = committed.clone();
engine.on_history_append(move |item| {
if matches!(item, Item::ToolResult { .. }) {
sink.lock().unwrap().push(item.clone());
}
Ok(())
});
let log = DiagnosticLog(Arc::new(Mutex::new(Vec::new())));
let writer = log.clone();
let subscriber = tracing_subscriber::fmt()
.without_time()
.with_ansi(false)
.with_max_level(tracing::Level::WARN)
.with_writer(move || writer.clone())
.finish();
let mut history = History::new();
let run = engine
.run(&mut history, "run tools")
.with_subscriber(subscriber)
.await;
assert!(matches!(run.result, agen::EngineRunExit::Finished));
assert!(warnings.lock().unwrap().is_empty());
let requests = probe.requests();
assert_eq!(requests.len(), 2);
let saved: Vec<_> = history
.iter()
.filter_map(|entry| matches!(entry.item, Item::ToolResult { .. }).then_some(&entry.item))
.collect();
assert_eq!(saved.len(), cases.len() + 1);
let committed = committed.lock().unwrap();
let published = published.lock().unwrap();
let logs = String::from_utf8(log.0.lock().unwrap().clone()).unwrap();
for item in saved {
let Item::ToolResult {
call_id,
summary,
content,
disposition,
is_error,
attachments: saved_attachments,
..
} = item
else {
unreachable!()
};
assert!(committed.iter().any(|candidate| candidate == item));
assert!(requests[1].items.iter().any(|candidate| candidate == item));
let result = published
.iter()
.find(|result| result.tool_use_id == *call_id)
.unwrap();
assert_eq!(&result.summary, summary);
assert_eq!(&result.content, content);
assert_eq!(&result.disposition, disposition);
assert_eq!(&result.is_error, is_error);
assert_eq!(&result.attachments, saved_attachments);
if call_id == "erroring" {
assert_eq!(*disposition, ToolResultDisposition::Error);
assert!(*is_error);
assert!(content.is_none());
continue;
}
assert_eq!(*disposition, ToolResultDisposition::Success);
assert!(!*is_error);
assert_eq!(*summary, format!("result from {call_id}"));
assert_eq!(saved_attachments, &attachments);
let (_, original, cap) = cases.iter().find(|c| c.0 == call_id).unwrap();
let content = content.as_ref().unwrap();
assert!(content.len() <= *cap);
if original.len() <= *cap {
assert_eq!(content, original);
} else {
let (body, marker) = content.split_once("\n\n[truncated: ").unwrap();
assert!(original.starts_with(body));
assert_eq!(
marker,
format!(
"{} bytes dropped, refine your query]",
original.len() - body.len()
)
);
let diagnostic = logs
.lines()
.find(|line| line.contains(&format!("tool={call_id}")))
.unwrap();
assert!(diagnostic.contains(&format!("before_bytes={}", original.len())));
assert!(diagnostic.contains(&format!("after_bytes={}", content.len())));
assert!(diagnostic.contains(&format!("limit_bytes={cap}")));
assert!(!diagnostic.contains(body));
}
}
assert_eq!(
logs.lines()
.filter(|line| line.contains("Tool output exceeded byte limit"))
.count(),
2
);
}
#[derive(Clone)]
struct FailOnceClient {
calls: Arc<AtomicUsize>,
+738
View File
@@ -0,0 +1,738 @@
use agen::llm_client::event::{Event, ResponseStatus, StatusEvent};
use agen::llm_client::scheme::{Scheme, openai_responses::OpenAIResponsesScheme};
use agen::llm_client::{ClientError, LlmClient, Request, ResponseStream};
use agen::tool::{Attachment, ImageAttachment};
use agen::{Engine, EngineRunExit, History, Item, ToolResultDisposition};
use async_trait::async_trait;
use futures::StreamExt;
use serde_json::{Value, json};
use std::sync::{Arc, Mutex};
fn rejection() -> ClientError {
ClientError::Api { status: Some(400), code: Some("invalid_value".into()), message: "The image you provided requires 52150 patches after processing, exceeding the limit of 30000. Please resize the image and try again.".into(), retry_after: None }
}
fn image_result(id: &str, width: u32, height: u32, padding: usize) -> Item {
// PNG header: dimension inspection never decodes/allocates the pixel plane.
let mut bytes = b"\x89PNG\r\n\x1a\n\0\0\0\rIHDR".to_vec();
bytes.extend(width.to_be_bytes());
bytes.extend(height.to_be_bytes());
bytes.resize(40 + padding, 0);
Item::ToolResult {
id: None,
call_id: id.into(),
summary: "Attached image".into(),
content: None,
disposition: ToolResultDisposition::Success,
is_error: false,
attachments: vec![Attachment::Image(ImageAttachment::new("image/png", bytes))],
}
}
fn history() -> History {
History::from_items(vec![
Item::user_message("inspect"),
Item::tool_call("small", "ViewImage", "{}"),
Item::tool_call("large", "ViewImage", "{}"),
image_result("small", 10, 10, 2000),
image_result("large", 1600, 33349, 0),
])
}
fn images(request: &Request) -> Vec<String> {
request
.items
.iter()
.filter_map(|item| match item {
Item::ToolResult {
call_id,
attachments,
..
} if !attachments.is_empty() => Some(call_id.clone()),
_ => None,
})
.collect()
}
#[derive(Clone, Copy)]
enum RejectionPath {
Http,
SseError,
SseNestedError,
ResponseFailed,
}
const STREAM_PATHS: [RejectionPath; 3] = [
RejectionPath::SseError,
RejectionPath::SseNestedError,
RejectionPath::ResponseFailed,
];
#[derive(Clone)]
struct Client {
requests: Arc<Mutex<Vec<Request>>>,
failures: usize,
other_error: bool,
path: RejectionPath,
prefix: Vec<(&'static str, Value)>,
wait_before_error: Option<Arc<tokio::sync::Notify>>,
rejection_data: Option<&'static str>,
}
impl Client {
fn new(failures: usize) -> Self {
Self {
requests: Arc::default(),
failures,
other_error: false,
path: RejectionPath::Http,
prefix: Vec::new(),
wait_before_error: None,
rejection_data: None,
}
}
fn streamed(failures: usize, path: RejectionPath) -> Self {
Self {
path,
..Self::new(failures)
}
}
fn reject(&self, error: ClientError) -> Result<ResponseStream, ClientError> {
if matches!(self.path, RejectionPath::Http) {
return Err(error);
}
let ClientError::Api { code, message, .. } = error else {
unreachable!()
};
let error = json!({"type":"invalid_request_error", "code":code, "message":message, "param":"input"});
let (kind, data) = match self.path {
RejectionPath::SseError => (
"error",
json!({"type":"error", "code":code, "message":message, "param":"input", "error_type":"invalid_request_error"}),
),
RejectionPath::SseNestedError => ("error", json!({"type":"error", "error":error})),
RejectionPath::ResponseFailed => (
"response.failed",
json!({"type":"response.failed", "response":{"id":"rejected", "error":error}}),
),
RejectionPath::Http => unreachable!(),
};
let scheme = OpenAIResponsesScheme::new();
let mut state = Default::default();
let mut events = scheme.parse_sse("response.created", r#"{"response":{}}"#, &mut state)?;
for (kind, data) in &self.prefix {
events.extend(scheme.parse_sse(kind, &data.to_string(), &mut state)?);
}
let encoded = data.to_string();
events.extend(scheme.parse_sse(
kind,
self.rejection_data.unwrap_or(&encoded),
&mut state,
)?);
let wait = self.wait_before_error.clone();
Ok(Box::pin(futures::stream::iter(events).then(move |event| {
let wait = wait.clone();
async move {
if matches!(event, Event::Error(_))
&& let Some(wait) = wait
{
wait.notified().await;
}
Ok(event)
}
})))
}
}
#[async_trait]
impl LlmClient for Client {
fn tool_call_completion_support(&self) -> agen::llm_client::ToolCallCompletionSupport {
agen::llm_client::ToolCallCompletionSupport::PerBlock
}
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(&self, request: Request) -> Result<ResponseStream, ClientError> {
let mut requests = self.requests.lock().unwrap();
requests.push(request);
if requests.len() <= self.failures {
return self.reject(rejection());
}
if self.other_error {
return self.reject(ClientError::Api {
status: Some(400),
code: Some("invalid_value".into()),
message: "invalid tool arguments".into(),
retry_after: None,
});
}
Ok(Box::pin(futures::stream::iter(vec![
Ok(Event::text_block_start(0)),
Ok(Event::text_delta(0, "resize and retry")),
Ok(Event::text_block_stop(0, None)),
Ok(Event::Status(StatusEvent {
status: ResponseStatus::Completed,
})),
])))
}
}
#[tokio::test]
async fn removes_largest_pixels_not_largest_file_and_keeps_pairing() {
let client = Client::new(1);
let observer = client.clone();
let mut history = history();
let mut engine = Engine::new(client);
let committed = Arc::new(Mutex::new(Vec::new()));
let commits = committed.clone();
engine.set_image_rejection_handler(move |original, replacement| {
commits
.lock()
.unwrap()
.push((original.item.clone(), replacement.clone()));
Ok(())
});
let output = engine.run(&mut history, "continue").await;
assert!(
matches!(output.result, EngineRunExit::Finished),
"{:?}",
output.result
);
let requests = observer.requests.lock().unwrap();
assert_eq!(requests.len(), 2);
assert_eq!(images(&requests[0]), ["small", "large"]);
assert_eq!(images(&requests[1]), ["small"]);
assert!(requests[1].items.iter().any(|item| matches!(item, Item::ToolResult { call_id, is_error: true, disposition: ToolResultDisposition::Error, summary, .. } if call_id == "large" && summary.contains("retry ViewImage"))));
assert_eq!(committed.lock().unwrap().len(), 1);
assert_eq!(
history
.items()
.filter(|item| matches!(item, Item::ToolResult {call_id, ..} if call_id == "large"))
.count(),
1
);
}
#[tokio::test]
async fn multiple_rejections_remove_one_image_at_a_time_and_stop_at_exhaustion() {
let client = Client::new(99);
let observer = client.clone();
let mut history = history();
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
let requests = observer.requests.lock().unwrap();
assert_eq!(requests.len(), 3);
assert_eq!(images(&requests[1]), ["small"]);
assert!(images(&requests[2]).is_empty());
}
#[tokio::test]
async fn unrelated_error_stops_recovery_without_removing_another_image() {
let mut client = Client::new(1);
client.other_error = true;
let observer = client.clone();
let mut history = history();
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
assert_eq!(observer.requests.lock().unwrap().len(), 2);
assert!(history.items().any(|item| matches!(item, Item::ToolResult {call_id, attachments, ..} if call_id == "small" && !attachments.is_empty())));
}
#[tokio::test]
async fn persistence_failure_does_not_change_history_or_retry() {
let client = Client::new(99);
let observer = client.clone();
let mut history = history();
let original = history.items_cloned();
let mut engine = Engine::new(client);
engine.set_image_rejection_handler(|_, _| Err("disk full".into()));
let out = engine.run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
assert_eq!(
&history.items_cloned()[..original.len()],
original.as_slice()
);
assert_eq!(observer.requests.lock().unwrap().len(), 1);
}
#[tokio::test]
async fn previously_accepted_images_are_not_recovery_candidates() {
let client = Client::new(99);
let observer = client.clone();
let mut history = history();
history.push(Item::assistant_message("I saw those images"));
history.push(Item::tool_call("new", "ViewImage", "{}"));
history.push(image_result("new", 1, 1, 0));
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
let requests = observer.requests.lock().unwrap();
assert_eq!(requests.len(), 2);
// Already-consumed images are projected out, but their durable records
// must not be rewritten as failures.
assert!(images(&requests[1]).is_empty());
assert_eq!(history.items().filter(|item| matches!(item, Item::ToolResult {call_id, is_error: false, attachments, ..} if (call_id == "small" || call_id == "large") && !attachments.is_empty())).count(), 2);
}
#[tokio::test]
async fn legacy_empty_failed_request_boundaries_do_not_hide_candidates() {
let client = Client::new(1);
let observer = client.clone();
let mut history = history();
history.push(Item::assistant_response_boundary("failed-1"));
history.push(Item::assistant_response_boundary("failed-2"));
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Finished));
assert_eq!(images(&observer.requests.lock().unwrap()[1]), ["small"]);
}
#[tokio::test]
async fn early_results_remain_candidates_despite_later_text_in_same_response() {
let client = Client::new(1);
let observer = client.clone();
let mut history = History::from_items(vec![
Item::user_message("inspect"),
Item::assistant_response_boundary("response-1"),
Item::tool_call("image", "ViewImage", "{}"),
image_result("image", 99, 99, 0),
Item::assistant_message("waiting for image"),
]);
let out = Engine::new(client).run(&mut history, "continue").await;
assert!(
matches!(out.result, EngineRunExit::Finished),
"{:?}; requests: {:?}",
out.result,
observer.requests.lock().unwrap()
);
assert!(images(&observer.requests.lock().unwrap()[1]).is_empty());
}
#[derive(Clone)]
struct CountingTool {
image: bool,
count: Arc<std::sync::atomic::AtomicUsize>,
executed: Option<Arc<tokio::sync::Notify>>,
}
#[async_trait]
impl agen::tool::Tool for CountingTool {
async fn execute(
&self,
_: &str,
_: agen::tool::ToolExecutionContext,
) -> Result<agen::tool::ToolOutput, agen::tool::ToolError> {
self.count.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
if let Some(executed) = &self.executed {
executed.notify_one();
}
let attachments = if self.image {
let Item::ToolResult { attachments, .. } = image_result("image", 1600, 33349, 0) else {
unreachable!()
};
attachments
} else {
Vec::new()
};
Ok(agen::tool::ToolOutput {
summary: "executed once".into(),
content: None,
attachments,
})
}
}
#[derive(Clone)]
struct ToolClient(Client);
#[async_trait]
impl LlmClient for ToolClient {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
fn tool_call_completion_support(&self) -> agen::llm_client::ToolCallCompletionSupport {
agen::llm_client::ToolCallCompletionSupport::PerBlock
}
async fn stream(&self, request: Request) -> Result<ResponseStream, ClientError> {
if self.0.requests.lock().unwrap().is_empty() {
self.0.requests.lock().unwrap().push(request);
return Ok(Box::pin(futures::stream::iter(
vec![
Event::tool_use_start(0, "image", "ViewImage"),
Event::tool_input_delta(0, "{}"),
Event::tool_use_stop(0),
Event::tool_use_start(1, "side-effect", "Bash"),
Event::tool_input_delta(1, "{}"),
Event::tool_use_stop(1),
Event::Status(StatusEvent {
status: ResponseStatus::Completed,
}),
]
.into_iter()
.map(Ok),
)));
}
self.0.stream(request).await
}
}
#[tokio::test]
async fn fresh_image_recovery_does_not_repeat_tools_in_either_dispatch_mode() {
for mode in [
agen::ToolCallDispatchMode::AfterResponse,
agen::ToolCallDispatchMode::OnToolCallComplete,
] {
let client = Client::new(2);
let mut engine = Engine::new(ToolClient(client.clone()));
engine.set_tool_call_dispatch_mode(mode);
let mut counts = Vec::new();
for (name, image) in [("ViewImage", true), ("Bash", false)] {
let tool = CountingTool {
image,
count: Arc::default(),
executed: None,
};
counts.push(tool.count.clone());
engine.register_tool(Arc::new(move || {
(
agen::tool::ToolMeta::new(name)
.description("test")
.input_schema(serde_json::json!({"type":"object"})),
Arc::new(tool.clone()),
)
}));
}
let mut history = History::new();
let out = engine.run(&mut history, "inspect").await;
assert!(
matches!(out.result, EngineRunExit::Finished),
"{:?}",
out.result
);
assert!(
counts
.iter()
.all(|count| count.load(std::sync::atomic::Ordering::SeqCst) == 1)
);
let requests = client.requests.lock().unwrap();
assert_eq!(requests.len(), 3);
assert_eq!(images(&requests[1]), ["image"]);
assert!(images(&requests[2]).is_empty());
}
}
#[test]
fn image_error_classifier_is_not_a_generic_400_retry() {
assert!(rejection().is_image_size_rejection());
for (status, message) in [
(400, "invalid image format"),
(400, "image URL not found"),
(400, "context window exceeds token limit"),
(401, "image is too large"),
(500, "image is too large"),
] {
assert!(
!ClientError::Api {
status: Some(status),
code: Some("invalid_value".into()),
message: message.into(),
retry_after: None
}
.is_image_size_rejection()
);
}
}
#[tokio::test]
async fn streamed_image_rejections_correct_largest_candidate_before_next_request() {
for path in STREAM_PATHS {
let client = Client::streamed(1, path);
let mut history = history();
let mut engine = Engine::new(client.clone());
let committed = Arc::new(Mutex::new(Vec::new()));
let commits = committed.clone();
engine.set_image_rejection_handler(move |original, replacement| {
commits
.lock()
.unwrap()
.push((original.item.clone(), replacement.clone()));
Ok(())
});
let out = engine.run(&mut history, "continue").await;
assert!(
matches!(out.result, EngineRunExit::Finished),
"{:?}",
out.result
);
let requests = client.requests.lock().unwrap();
assert_eq!(requests.len(), 2);
assert_eq!(images(&requests[0]), ["small", "large"]);
assert_eq!(images(&requests[1]), ["small"]);
assert!(requests[1].items.iter().any(|item| matches!(item, Item::ToolResult { call_id, is_error: true, disposition: ToolResultDisposition::Error, summary, .. } if call_id == "large" && summary.contains("Resize or crop") && summary.contains("retry ViewImage"))));
assert!(requests[1].items.iter().any(|item| matches!(item, Item::ToolCall { call_id, name, .. } if call_id == "large" && name == "ViewImage")));
assert_eq!(committed.lock().unwrap().len(), 1);
assert_eq!(
history
.items()
.filter(
|item| matches!(item, Item::ToolResult { call_id, .. } if call_id == "large")
)
.count(),
1
);
}
}
#[tokio::test]
async fn streamed_rejections_stop_when_images_are_absent_or_candidates_exhausted() {
for path in STREAM_PATHS {
for (mut history, expected_requests) in [(History::new(), 1), (history(), 3)] {
let client = Client::streamed(99, path);
let out = Engine::new(client.clone())
.run(&mut history, "continue")
.await;
assert!(
matches!(out.result, EngineRunExit::Interrupted(_)),
"{:?}",
out.result
);
let requests = client.requests.lock().unwrap();
assert_eq!(requests.len(), expected_requests);
assert!(images(requests.last().unwrap()).is_empty());
}
}
}
#[tokio::test]
async fn unrelated_streamed_invalid_value_preserves_images_and_diagnostic() {
for path in STREAM_PATHS {
let mut client = Client::streamed(0, path);
client.other_error = true;
let mut history = history();
let original = history.items_cloned();
let out = Engine::new(client.clone())
.run(&mut history, "continue")
.await;
let EngineRunExit::Interrupted(reason) = out.result else {
panic!("{:?}", out.result)
};
let reason = format!("{reason:?}");
assert!(
reason.contains("invalid_value")
&& reason.contains("invalid tool arguments")
&& reason.contains("diagnostic=")
&& reason.contains("input"),
"{reason}"
);
assert_eq!(client.requests.lock().unwrap().len(), 1);
assert_eq!(
&history.items_cloned()[..original.len()],
original.as_slice()
);
}
}
#[tokio::test]
async fn streamed_correction_commit_failure_does_not_mutate_or_resend() {
for path in STREAM_PATHS {
let client = Client::streamed(99, path);
let mut history = history();
let original = history.items_cloned();
let mut engine = Engine::new(client.clone());
engine.set_image_rejection_handler(|_, _| Err("disk full".into()));
let out = engine.run(&mut history, "continue").await;
assert!(matches!(out.result, EngineRunExit::Interrupted(_)));
assert_eq!(
&history.items_cloned()[..original.len()],
original.as_slice()
);
assert_eq!(client.requests.lock().unwrap().len(), 1);
}
}
#[tokio::test]
async fn streamed_rejection_after_output_never_replays_or_corrects_input() {
for path in STREAM_PATHS {
// Start alone closes the replay window, as do partial/completed text,
// reasoning and unrecognized events. None may change accepted input.
for prefix in [
vec![(
"response.content_part.added",
json!({"output_index":0,"content_index":0,"part":{"type":"output_text"}}),
)],
vec![(
"response.output_text.delta",
json!({"output_index":0,"content_index":0,"delta":"partial"}),
)],
vec![
(
"response.output_text.delta",
json!({"output_index":0,"content_index":0,"delta":"completed"}),
),
(
"response.content_part.done",
json!({"output_index":0,"content_index":0,"part":{"type":"output_text"}}),
),
],
vec![(
"response.reasoning_text.delta",
json!({"output_index":0,"content_index":0,"delta":"thinking"}),
)],
vec![("future.unknown", json!({"value":"unknown semantics"}))],
] {
let mut client = Client::streamed(1, path);
client.prefix = prefix;
let mut history = history();
let original = history.items_cloned();
let out = Engine::new(client.clone())
.run(&mut history, "continue")
.await;
assert!(
matches!(out.result, EngineRunExit::Interrupted(_)),
"{:?}",
out.result
);
assert_eq!(client.requests.lock().unwrap().len(), 1);
assert_eq!(
&history.items_cloned()[..original.len()],
original.as_slice()
);
}
}
}
#[test]
fn statusless_api_errors_are_not_image_retry_authority() {
let ClientError::Api { code, message, .. } = rejection() else {
unreachable!()
};
assert!(
!ClientError::Api {
status: None,
code,
message,
retry_after: None
}
.is_image_size_rejection()
);
}
#[tokio::test]
async fn streamed_rejection_after_tool_completion_never_reexecutes_side_effects() {
for path in STREAM_PATHS {
for mode in [
agen::ToolCallDispatchMode::AfterResponse,
agen::ToolCallDispatchMode::OnToolCallComplete,
] {
let executed = Arc::new(tokio::sync::Notify::new());
let mut client = Client::streamed(1, path);
client.prefix = vec![
(
"response.output_item.added",
json!({"output_index":0,"item":{"type":"function_call","call_id":"side-effect","name":"Bash","arguments":"{}"}}),
),
(
"response.output_item.done",
json!({"output_index":0,"item":{"type":"function_call","call_id":"side-effect","name":"Bash","arguments":"{}"}}),
),
];
if mode == agen::ToolCallDispatchMode::OnToolCallComplete {
client.wait_before_error = Some(executed.clone());
}
let tool = CountingTool {
image: false,
count: Arc::default(),
executed: Some(executed),
};
let count = tool.count.clone();
let mut engine = Engine::new(client.clone());
engine.set_tool_call_dispatch_mode(mode);
engine.register_tool(Arc::new(move || {
(
agen::tool::ToolMeta::new("Bash")
.description("side effect")
.input_schema(json!({"type":"object"})),
Arc::new(tool.clone()),
)
}));
let mut history = history();
let original = history.items_cloned();
let out = tokio::time::timeout(
std::time::Duration::from_secs(5),
engine.run(&mut history, "continue"),
)
.await
.expect("tool admission/error cleanup must terminate");
assert!(
matches!(out.result, EngineRunExit::Interrupted(_)),
"{:?}",
out.result
);
assert_eq!(client.requests.lock().unwrap().len(), 1);
assert_eq!(
&history.items_cloned()[..original.len()],
original.as_slice()
);
if mode == agen::ToolCallDispatchMode::OnToolCallComplete {
assert_eq!(count.load(std::sync::atomic::Ordering::SeqCst), 1);
assert_eq!(history.items().filter(|item| matches!(item, Item::ToolCall { call_id, .. } if call_id == "side-effect")).count(), 1);
assert_eq!(history.items().filter(|item| matches!(item, Item::ToolResult { call_id, .. } if call_id == "side-effect")).count(), 1);
} else {
assert_eq!(count.load(std::sync::atomic::Ordering::SeqCst), 0);
assert!(!history.items().any(|item| matches!(item, Item::ToolCall { call_id, .. } | Item::ToolResult { call_id, .. } if call_id == "side-effect")));
}
}
}
}
#[test]
fn streamed_error_extra_fields_do_not_turn_unrelated_message_into_image_rejection() {
let scheme = OpenAIResponsesScheme::new();
let mut state = Default::default();
let events = scheme.parse_sse("error", r#"{"type":"error","code":"invalid_value","message":"invalid tool arguments","param":"input","input":"image too large"}"#, &mut state).unwrap();
let Event::Error(error) = &events[0] else {
panic!("{events:?}")
};
assert_eq!(error.code.as_deref(), Some("invalid_value"));
assert!(
error
.message
.starts_with("invalid tool arguments | diagnostic=")
);
// HTTP uses the same diagnostic policy, but still requires status authority.
assert!(
!ClientError::Api {
status: Some(400),
code: error.code.clone(),
message: error.message.clone(),
retry_after: None
}
.is_image_size_rejection()
);
}
#[tokio::test]
async fn unparseable_stream_error_data_preserves_images_and_terminates_with_diagnostic() {
for data in [
r#"{"type":"error","code":"invalid_value","message":123,"input":"image too large"}"#,
"unparseable provider error with echoed input: image too large",
] {
let mut client = Client::streamed(1, RejectionPath::SseError);
client.rejection_data = Some(data);
let mut history = history();
let original = history.items_cloned();
let out = Engine::new(client.clone())
.run(&mut history, "continue")
.await;
let EngineRunExit::Interrupted(agen::RunInterruptionReason::Unexpected(
agen::EngineError::Client(ClientError::Api { code, message, .. }),
)) = out.result
else {
panic!(
"malformed provider error must terminate, got {:?}",
out.result
);
};
assert_eq!(code, None);
let (base, diagnostic) = message
.split_once(" | diagnostic=")
.expect("raw data remains diagnostic-only");
assert!(!base.contains("image"));
let diagnostic: Value = serde_json::from_str(diagnostic).unwrap();
assert_eq!(diagnostic["error_extra"]["raw_data"], data);
assert_eq!(client.requests.lock().unwrap().len(), 1);
assert_eq!(
&history.items_cloned()[..original.len()],
original.as_slice()
);
}
}
+10 -8
View File
@@ -20,8 +20,8 @@ use agen::tool::{
ToolResultDisposition,
};
use agen::{
Engine, EngineError, EngineRunExit, History, Item, RunInterruptionReason, ToolCallDispatchMode,
ToolExecutionPolicy,
Engine, EngineConfig, EngineError, EngineRunExit, History, Item, RunInterruptionReason,
ToolCallDispatchMode, ToolExecutionPolicy,
};
use async_trait::async_trait;
@@ -465,12 +465,13 @@ impl Tool for BarrierTool {
// =============================================================================
#[tokio::test]
async fn default_dispatch_waits_for_response_completion() {
async fn explicit_after_response_dispatch_waits_for_response_completion() {
let (client, tx) = ControlledStreamClient::new(ToolCallCompletionSupport::PerBlock);
let tool = BarrierTool::new("barrier_default");
let probe = tool.clone();
let mut engine = Engine::new(client);
engine.set_max_turns(Some(1));
engine.set_tool_call_dispatch_mode(ToolCallDispatchMode::AfterResponse);
engine.register_tool(tool.definition());
let (observed_tx, mut observed_rx) = tokio::sync::mpsc::unbounded_channel();
@@ -528,7 +529,6 @@ async fn turn_end_policy_incompatibility_visibly_falls_back_before_side_effects(
let tool = BarrierTool::new("turn_end_fallback");
let probe = tool.clone();
let mut engine = Engine::new(client);
engine.set_tool_call_dispatch_mode(ToolCallDispatchMode::OnToolCallComplete);
engine.set_interceptor(TurnEndPause);
engine.register_tool(tool.definition());
let (warning_tx, mut warning_rx) = tokio::sync::mpsc::unbounded_channel();
@@ -762,7 +762,11 @@ async fn blocking_pre_tool_policy_keeps_receiving_stream_and_services_cancel() {
}
#[tokio::test]
async fn per_block_dispatch_starts_siblings_without_waiting_for_stream_or_prior_result() {
async fn default_per_block_dispatch_starts_siblings_without_waiting_for_stream_or_prior_result() {
assert_eq!(
EngineConfig::default().tool_call_dispatch,
ToolCallDispatchMode::OnToolCallComplete
);
let (client, tx) = ControlledStreamClient::new(ToolCallCompletionSupport::PerBlock);
let first = BarrierTool::new("barrier_a");
let second = BarrierTool::new("barrier_b");
@@ -770,7 +774,6 @@ async fn per_block_dispatch_starts_siblings_without_waiting_for_stream_or_prior_
let second_probe = second.clone();
let mut engine = Engine::new(client);
engine.set_max_turns(Some(1));
engine.set_tool_call_dispatch_mode(ToolCallDispatchMode::OnToolCallComplete);
engine.register_tool(first.definition());
engine.register_tool(second.definition());
@@ -1641,13 +1644,12 @@ async fn clean_eof_without_provider_completion_after_early_start_stops_without_c
}
#[tokio::test]
async fn unsupported_provider_visibly_defers_opt_in_dispatch() {
async fn unsupported_provider_visibly_defers_default_dispatch() {
let (client, tx) = ControlledStreamClient::new(ToolCallCompletionSupport::ResponseComplete);
let tool = BarrierTool::new("barrier_fallback");
let probe = tool.clone();
let mut engine = Engine::new(client);
engine.set_max_turns(Some(1));
engine.set_tool_call_dispatch_mode(ToolCallDispatchMode::OnToolCallComplete);
engine.register_tool(tool.definition());
let warnings = Arc::new(Mutex::new(Vec::new()));
let warning_probe = warnings.clone();
@@ -80,33 +80,38 @@ async fn assert_response_complete_scheme_accepts_clean_eof<S: Scheme>(
ToolCallCompletionSupport::ResponseComplete
);
for mode in [
ToolCallDispatchMode::AfterResponse,
ToolCallDispatchMode::OnToolCallComplete,
for (label, mode) in [
("default on_tool_call_complete fallback", None),
(
"explicit after_response",
Some(ToolCallDispatchMode::AfterResponse),
),
] {
let client = SchemeReplayClient::new(scheme.clone(), vec![frames.clone()]);
let probe = client.clone();
let mut engine = Engine::new(client);
engine.set_tool_call_dispatch_mode(mode);
if let Some(mode) = mode {
engine.set_tool_call_dispatch_mode(mode);
}
let mut history = History::new();
let output = engine.run(&mut history, "hello").await;
assert!(
matches!(output.result, EngineRunExit::Finished),
"{mode:?} should accept the response-complete scheme's clean EOF, got {:?}",
"{label} should accept the response-complete scheme's clean EOF, got {:?}",
output.result
);
assert_eq!(
probe.stream_count(),
1,
"{mode:?} must not open a continuation request after the adapter's normal EOF"
"{label} must not open a continuation request after the adapter's normal EOF"
);
}
}
#[tokio::test]
async fn openai_chat_clean_eof_completes_in_default_and_opt_in_fallback_modes() {
async fn openai_chat_clean_eof_completes_in_default_and_explicit_after_response_modes() {
assert_response_complete_scheme_accepts_clean_eof(
OpenAIScheme::new(),
vec![
@@ -124,7 +129,7 @@ async fn openai_chat_clean_eof_completes_in_default_and_opt_in_fallback_modes()
}
#[tokio::test]
async fn gemini_clean_eof_completes_in_default_and_opt_in_fallback_modes() {
async fn gemini_clean_eof_completes_in_default_and_explicit_after_response_modes() {
assert_response_complete_scheme_accepts_clean_eof(
GeminiScheme::new(),
vec![
+35 -7
View File
@@ -1534,6 +1534,13 @@ fn validate_named_body_type(ty: &Type, kind: &str) -> syn::Result<()> {
Ok(())
}
fn is_binary_body_type(ty: &Type) -> bool {
matches!(ty, Type::Path(path)
if path.qself.is_none()
&& path.path.segments.last().is_some_and(|last|
last.ident == "BinaryBody" && matches!(last.arguments, PathArguments::None)))
}
fn validate_binary_body_type(ty: &Type) -> syn::Result<()> {
let Type::Path(path) = ty else {
return Err(syn::Error::new_spanned(
@@ -1738,6 +1745,9 @@ fn reqwest_adapter_tokens(
}
});
let body_parser = match response.body.as_ref() {
Some(ty) if is_binary_body_type(ty) => quote! {
let body = __response.decode_binary();
},
Some(ty) => quote! {
let body = __response
.decode_json::<#ty>(#api_crate::reqwest::DecodeKind::Success)
@@ -1775,7 +1785,11 @@ fn reqwest_adapter_tokens(
.alternate_status
.map(|alternate| quote!(__actual == #response_status || __actual == #alternate))
.unwrap_or_else(|| quote!(__actual == #response_status));
let success = if let Some(response) = operation.response_body.as_ref() {
let success = if operation.response_body.as_ref().is_some_and(is_binary_body_type) {
quote! {
return ::core::result::Result::Ok(__response.decode_binary());
}
} else if let Some(response) = operation.response_body.as_ref() {
quote! {
return __response
.decode_json::<#response>(#api_crate::reqwest::DecodeKind::Success)
@@ -2077,7 +2091,7 @@ fn axum_adapter_tokens(
status,
rejection.body_text(),
);
return #api_crate::axum::json_response(
return #api_crate::axum::error_response(
#api_crate::axum::status(status),
error,
);
@@ -2111,7 +2125,7 @@ fn axum_adapter_tokens(
status,
rejection.body_text(),
);
return #api_crate::axum::json_response(
return #api_crate::axum::error_response(
#api_crate::axum::status(status),
error,
);
@@ -2133,7 +2147,9 @@ fn axum_adapter_tokens(
let variant_ident = format_ident!("Status{}", status);
let body_pattern = response.body.as_ref().map(|_| quote!(body,));
let header_fields = response.headers.iter().map(|header| &header.field_ident);
let base_response = if response.body.is_some() {
let base_response = if response.body.as_ref().is_some_and(is_binary_body_type) {
quote!(#api_crate::axum::binary_response(#api_crate::axum::status(#status), body))
} else if response.body.is_some() {
quote!(#api_crate::axum::json_response(#api_crate::axum::status(#status), body))
} else {
quote!(#api_crate::axum::empty_response(#api_crate::axum::status(#status)))
@@ -2170,14 +2186,21 @@ fn axum_adapter_tokens(
}
}
} else if let Some(alternate_status) = operation.alternate_status {
let response_encoder = if operation.response_body.as_ref().is_some_and(is_binary_body_type) {
quote!(#api_crate::axum::binary_response)
} else {
quote!(#api_crate::axum::json_response)
};
quote!({
let status = #api_crate::HttpSuccess::status_code(&value);
if status == #response_status || status == #alternate_status {
#api_crate::axum::json_response(#api_crate::axum::status(status), value)
#response_encoder(#api_crate::axum::status(status), value)
} else {
#api_crate::axum::empty_response(#api_crate::axum::status(500))
}
})
} else if operation.response_body.as_ref().is_some_and(is_binary_body_type) {
quote!(#api_crate::axum::binary_response(#api_crate::axum::status(#response_status), value))
} else if operation.response_body.is_some() {
quote!(#api_crate::axum::json_response(#api_crate::axum::status(#response_status), value))
} else {
@@ -2188,7 +2211,7 @@ fn axum_adapter_tokens(
let error = match operation.error_status {
Some(_) => quote!({
let status = #api_crate::HttpError::status_code(&error);
#api_crate::axum::json_response(#api_crate::axum::status(status), error)
#api_crate::axum::error_response(#api_crate::axum::status(status), error)
}),
None => quote!({ let _ = error; #api_crate::axum::empty_response(#api_crate::axum::framework::StatusCode::INTERNAL_SERVER_ERROR) }),
};
@@ -2323,6 +2346,9 @@ fn openapi_adapter_tokens(
"Alternate successful response"
};
let body = match response.body.as_ref() {
Some(ty) if is_binary_body_type(ty) => quote! {
operation.binary_response(#status, #description)?;
},
Some(ty) => quote! {
operation.response::<#ty>(#status, "application/json", #description)?;
},
@@ -2620,7 +2646,9 @@ fn metadata_tokens(
};
let success_responses = operation.success_responses.iter().map(|response| {
let status = response.status;
let wire_kind = if response.body.is_some() {
let wire_kind = if response.body.as_ref().is_some_and(is_binary_body_type) {
quote!(#api_crate::WireKind::Binary)
} else if response.body.is_some() {
quote!(#api_crate::WireKind::Json)
} else {
quote!(#api_crate::WireKind::Empty)
+1
View File
@@ -35,4 +35,5 @@ oas3 = { version = "0.22", default-features = false }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
tokio = { workspace = true, features = ["net"] }
tower = { workspace = true, features = ["util"] }
trybuild = "1"
+30
View File
@@ -77,6 +77,36 @@ pub fn json_response<T: Serialize>(status: StatusCode, value: T) -> Response {
(status, Json(value)).into_response()
}
/// Serialize a public error and retain its typed value for outer application middleware.
///
/// The extension is server-local (`Arc<T>`); it is not another wire field. This lets observers
/// inspect the declared error without consuming the response body or requiring `T: Clone`.
/// Applications still choose which public fields are safe to log; the adapter does not log them.
pub fn error_response<T: Serialize + Send + Sync + 'static>(
status: StatusCode,
error: T,
) -> Response {
let mut response = json_response(status, &error);
// JSON encoding failures are plain-text responses, including when the requested status is 500.
if response.status() == status
&& response.headers().get(axum::http::header::CONTENT_TYPE)
== Some(&axum::http::HeaderValue::from_static("application/json"))
{
response.extensions_mut().insert(std::sync::Arc::new(error));
}
response
}
/// Emit an exact binary response with the contract status and octet-stream content type.
pub fn binary_response(status: StatusCode, value: crate::BinaryBody) -> Response {
(
status,
[(axum::http::header::CONTENT_TYPE, "application/octet-stream")],
value.into_bytes(),
)
.into_response()
}
/// Emit an empty response with the contract status.
pub fn empty_response(status: StatusCode) -> Response {
status.into_response()
+6 -3
View File
@@ -56,6 +56,9 @@
//! `#[header("x-request-id")]`. Exactly one JSON or binary body is allowed. JSON request, response,
//! and error bodies must be named Rust types; tuples, references, arrays, and other anonymous
//! structural types are rejected. Binary request bodies use [`BinaryBody`] and remain byte-exact.
//! A named success response whose final type identifier is `BinaryBody` is likewise byte-exact,
//! uses `application/octet-stream`, and may include declared response headers. Generated clients
//! collect binary responses under the same configured response byte limit as JSON responses.
//! `openapi = false` explicitly excludes an operation whose wire body cannot yet satisfy the
//! strict OpenAPI schema boundary; Reqwest and Axum adapters are still generated.
//!
@@ -96,7 +99,7 @@
//! deterministic `Operation<utf8-hex>` fallback. The `ApiContract::OPERATIONS` inventory is
//! sorted by operation ID, so its ordering is independent of source method order.
//!
//! Empty, JSON, and bounded binary request bodies are supported. [`WireKind`] reserves explicit
//! Empty, JSON, and bounded binary request and success response bodies are supported. [`WireKind`] reserves explicit
//! variants for other future transport work; accepting one requires a deliberate macro and adapter
//! change rather than silently treating it as JSON.
//!
@@ -148,7 +151,7 @@ pub mod openapi;
#[cfg(feature = "reqwest")]
pub mod reqwest;
/// Framework-neutral owned bytes used by explicit `#[binary]` request bodies.
/// Framework-neutral owned bytes used by explicit `#[binary]` requests and binary success responses.
///
/// The wrapper preserves the exact byte sequence while keeping payload content out of `Debug`
/// output. Its `bytes::Bytes` storage lets generated Axum adapters transfer an extracted body
@@ -314,7 +317,7 @@ pub trait Operation {
type Parameters;
/// Typed JSON or binary request body, or [`NoBody`].
type RequestBody;
/// JSON success response body type, generated declared-response result, or [`NoBody`].
/// JSON or binary success body type, generated declared-response result, or [`NoBody`].
type ResponseBody;
/// JSON public error body type, or [`NoBody`].
type ErrorBody;
+19
View File
@@ -419,6 +419,25 @@ impl OpenApiOperation<'_> {
)
}
/// Describe an octet-stream response inline, without registering a JSON schema component.
pub fn binary_response(
&mut self,
status: u16,
description: &'static str,
) -> Result<(), OpenApiError> {
self.insert_response(
status,
json!({
"description": description,
"content": {
"application/octet-stream": {
"schema": { "type": "string", "format": "binary" }
}
},
}),
)
}
pub fn empty_response(
&mut self,
status: u16,
+5
View File
@@ -516,6 +516,11 @@ impl ReceivedResponse {
serde_json::from_slice(&self.bytes).map_err(|_| ClientFailure::Decode { kind })
}
/// Move the exact response bytes, already collected under the configured limit, into a body.
pub fn decode_binary(self) -> crate::BinaryBody {
self.bytes.into()
}
pub fn require_empty(&self) -> Result<(), ClientFailure> {
if self.bytes.is_empty() {
Ok(())
+331
View File
@@ -0,0 +1,331 @@
#![cfg(all(feature = "reqwest", feature = "axum"))]
use api_macros::{
BinaryBody, Operation, api,
axum::framework::{Body, Response, Router, get},
reqwest::{ClientError, ClientFailure},
};
#[cfg(feature = "openapi")]
use api_macros::{
WireKind,
openapi::{OpenApiInfo, OpenApiSchema},
};
#[cfg(feature = "openapi")]
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
#[cfg(feature = "openapi")]
use serde_json::json;
const BYTES: &[u8] = b"\0\xff\x80\r\n{\"not\":\"a JSON response\"}";
const ETAG: &str = "\"binary-v1\"";
const DISPOSITION: &str = "attachment; filename=fixture.bin";
#[derive(Debug, Deserialize, Serialize)]
#[cfg_attr(feature = "openapi", derive(JsonSchema))]
pub struct PublicError {
message: String,
}
#[cfg(feature = "openapi")]
impl OpenApiSchema for PublicError {}
impl api_macros::HttpError for PublicError {
fn status_code(&self) -> u16 {
404
}
}
#[cfg_attr(feature = "openapi", api(reqwest, axum, openapi))]
#[cfg_attr(not(feature = "openapi"), api(reqwest, axum))]
pub trait DownloadApi {
#[get("/plain/{mode}", error_status = 404)]
async fn plain(&self, mode: String) -> Result<api_macros::BinaryBody, PublicError>;
#[get(
"/download/{mode}",
error_status = 404,
responses = [
(status = 200, body = BinaryBody, headers = [
("x-byte-length", u32), ("content-disposition", String), ("etag", String)
]),
(status = 304, headers = [("etag", String)])
]
)]
async fn download(&self, mode: String)
-> Result<download_api_responses::Download, PublicError>;
}
struct DownloadService;
impl DownloadApi for DownloadService {
async fn plain(&self, mode: String) -> Result<BinaryBody, PublicError> {
if mode == "missing" {
return Err(PublicError {
message: "not found".to_owned(),
});
}
Ok(if mode == "empty" {
b"".as_slice()
} else {
BYTES
}
.into())
}
async fn download(
&self,
mode: String,
) -> Result<download_api_responses::Download, PublicError> {
if mode == "cached" {
return Ok(download_api_responses::Download::Status304 {
header_etag: ETAG.to_owned(),
});
}
let body = self.plain(mode).await?;
Ok(download_api_responses::Download::Status200 {
header_x_byte_length: body.len() as u32,
header_content_disposition: DISPOSITION.to_owned(),
header_etag: ETAG.to_owned(),
body,
})
}
}
struct Server {
base_url: String,
task: tokio::task::JoinHandle<()>,
}
impl Server {
async fn start(router: Router) -> Self {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let base_url = format!("http://{}/", listener.local_addr().unwrap());
let task = tokio::spawn(async move {
api_macros::axum::framework::serve(listener, router)
.await
.unwrap();
});
Self { base_url, task }
}
fn client(&self, limit: usize) -> DownloadApiClient {
DownloadApiClient::builder(&self.base_url)
.unwrap()
.response_body_limit(limit)
.build()
.unwrap()
}
}
impl Drop for Server {
fn drop(&mut self) {
self.task.abort();
}
}
#[tokio::test]
async fn binary_success_preserves_exact_bytes_content_type_and_declared_headers() {
let server = Server::start(DownloadApiAxum::router(DownloadService)).await;
let client = server.client(BYTES.len());
for (mode, expected) in [("full", BYTES), ("empty", b"".as_slice())] {
let response = reqwest::get(format!("{}plain/{mode}", server.base_url))
.await
.unwrap();
assert_eq!(response.status().as_u16(), 200);
assert_eq!(
response.headers()["content-type"],
"application/octet-stream"
);
assert_eq!(response.bytes().await.unwrap().as_ref(), expected);
assert_eq!(
client.plain(mode.to_owned()).await.unwrap().as_ref(),
expected
);
let response = client.download(mode.to_owned()).await.unwrap();
match response {
download_api_responses::Download::Status200 {
body,
header_x_byte_length,
header_content_disposition,
header_etag,
} => {
assert_eq!(body.as_ref(), expected);
assert_eq!(header_x_byte_length as usize, expected.len());
assert_eq!(header_content_disposition, DISPOSITION);
assert_eq!(header_etag, ETAG);
}
other => panic!("expected binary success, received {other:?}"),
}
}
assert!(
matches!(client.download("cached".to_owned()).await.unwrap(),
download_api_responses::Download::Status304 { header_etag } if header_etag == ETAG)
);
}
#[cfg(feature = "openapi")]
#[test]
fn binary_success_metadata_and_openapi_use_inline_binary_schema_with_headers() {
let document = download_api_openapi(OpenApiInfo {
title: "Download fixture",
version: "1",
source_digest: "fixture",
})
.unwrap();
let value = document.as_value();
for (path, metadata) in [
("/plain/{mode}", download_api_operations::Plain::METADATA),
(
"/download/{mode}",
download_api_operations::Download::METADATA,
),
] {
assert_eq!(
metadata.success_responses[0].body.wire_kind,
WireKind::Binary
);
assert_eq!(metadata.error_responses[0].body.wire_kind, WireKind::Json);
let responses = &value["paths"][path]["get"]["responses"];
assert_eq!(
responses["200"]["content"],
json!({
"application/octet-stream": { "schema": { "type": "string", "format": "binary" } }
})
);
assert_eq!(
responses["404"]["content"]["application/json"]["schema"]["$ref"],
"#/components/schemas/PublicError"
);
}
let metadata = download_api_operations::Download::METADATA;
assert_eq!(
metadata.success_responses[1].body.wire_kind,
WireKind::Empty
);
let responses = &value["paths"]["/download/{mode}"]["get"]["responses"];
for header in metadata.success_responses[0].headers {
assert!(responses["200"]["headers"][header.wire_name]["schema"].is_object());
}
assert!(responses["304"].get("content").is_none());
assert!(responses["304"]["headers"]["etag"]["schema"].is_object());
assert!(
value["components"]["schemas"]
.as_object()
.unwrap()
.keys()
.all(|name| !name.contains("BinaryBody"))
);
}
#[tokio::test]
async fn binary_success_keeps_json_public_errors_and_rejects_unexpected_statuses() {
let server = Server::start(DownloadApiAxum::router(DownloadService)).await;
let client = server.client(1024);
for result in [
client.plain("missing".to_owned()).await.map(|_| ()),
client.download("missing".to_owned()).await.map(|_| ()),
] {
match result.unwrap_err() {
ClientError::Public { status, error } => {
assert_eq!(status.as_u16(), 404);
assert_eq!(error.message, "not found");
}
other => panic!("expected JSON public error, received {other:?}"),
}
}
for (status, expected) in [
(
201,
ClientFailure::UnexpectedStatus {
expected: 200,
actual: 201,
},
),
(404, ClientFailure::ErrorResponseDecode { status: 404 }),
] {
let server = Server::start(Router::new().route(
download_api_operations::Plain::METADATA.path,
get(move || async move {
Response::builder()
.status(status)
.body(Body::from(BYTES))
.unwrap()
}),
))
.await;
assert!(matches!(server.client(1024).plain("full".to_owned()).await,
Err(ClientError::Failure(actual)) if actual == expected));
}
}
#[tokio::test]
async fn binary_declared_headers_reject_missing_invalid_and_duplicate_values() {
for (values, expected) in [
(
vec![],
ClientFailure::MissingResponseHeader { name: "etag" },
),
(
vec![b"\xff".as_slice()],
ClientFailure::InvalidResponseHeader { name: "etag" },
),
(
vec![b"one".as_slice(), b"two".as_slice()],
ClientFailure::DuplicateResponseHeader { name: "etag" },
),
] {
let server = Server::start(Router::new().route(
download_api_operations::Download::METADATA.path,
get(move || {
let values = values.clone();
async move {
let mut response = Response::builder()
.status(200)
.header("content-type", "application/octet-stream")
.header("x-byte-length", BYTES.len())
.header("content-disposition", DISPOSITION);
for value in values {
response = response.header("etag", value);
}
response.body(Body::from(BYTES)).unwrap()
}
}),
))
.await;
assert!(
matches!(server.client(1024).download("full".to_owned()).await,
Err(ClientError::Failure(actual)) if actual == expected)
);
}
}
#[tokio::test]
async fn binary_responses_enforce_the_same_limit_with_and_without_content_length() {
for chunked in [false, true] {
let server = Server::start(Router::new().route(
download_api_operations::Plain::METADATA.path,
get(move || async move {
if chunked {
let chunks = [BYTES[..3].to_vec(), BYTES[3..].to_vec()];
let stream = futures::stream::iter(chunks.map(Ok::<_, std::io::Error>));
Response::builder()
.status(200)
.body(Body::from_stream(stream))
.unwrap()
} else {
api_macros::axum::binary_response(api_macros::axum::status(200), BYTES.into())
}
}),
))
.await;
assert_eq!(
server
.client(BYTES.len())
.plain("full".to_owned())
.await
.unwrap()
.as_ref(),
BYTES
);
assert!(
matches!(server.client(BYTES.len() - 1).plain("full".to_owned()).await,
Err(ClientError::Failure(ClientFailure::ResponseTooLarge { limit })) if limit == BYTES.len() - 1)
);
}
}
+176
View File
@@ -0,0 +1,176 @@
#![cfg(feature = "axum")]
use api_macros::api;
use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode},
};
use serde::{Deserialize, Serialize};
use std::sync::Arc;
use tower::ServiceExt;
// No Clone requirement: retaining an error must not require duplicating its typed value.
#[derive(Debug, Serialize)]
pub struct PublicError {
message: String,
#[serde(skip)]
status: u16,
}
impl api_macros::HttpError for PublicError {
fn status_code(&self) -> u16 {
self.status
}
}
impl api_macros::HttpRequestError for PublicError {
fn from_request_rejection(status: u16, message: String) -> Self {
Self { status, message }
}
}
#[derive(Deserialize)]
pub struct Input {
fail: bool,
}
#[derive(Serialize)]
pub struct Output {
message: String,
}
#[api(axum)]
pub trait ErrorMetadataApi {
#[post(
"/json",
status = 200,
error_status = 400,
normalize_body_errors = true
)]
async fn json(&self, #[body] input: Input) -> Result<Output, PublicError>;
#[post(
"/binary",
status = 200,
error_status = 400,
normalize_body_errors = true
)]
async fn binary(&self, #[binary] input: api_macros::BinaryBody) -> Result<Output, PublicError>;
}
pub struct Service;
impl ErrorMetadataApi for Service {
async fn json(&self, input: Input) -> Result<Output, PublicError> {
if input.fail {
Err(PublicError {
status: 502,
message: "upstream rejected operation".to_string(),
})
} else {
Ok(Output {
message: "ok".to_string(),
})
}
}
async fn binary(&self, _input: api_macros::BinaryBody) -> Result<Output, PublicError> {
Ok(Output {
message: "ok".to_string(),
})
}
}
#[tokio::test]
async fn generated_service_and_body_errors_retain_typed_metadata_without_changing_wire_body() {
let app = ErrorMetadataApiAxum::router(Service).layer(axum::extract::DefaultBodyLimit::max(32));
for (path, content_type, input, status) in [
(
"/json",
"application/json",
"{\"fail\":true}".to_string(),
StatusCode::BAD_GATEWAY,
),
(
"/json",
"application/json",
"{".to_string(),
StatusCode::BAD_REQUEST,
),
(
"/binary",
"application/octet-stream",
"x".repeat(33),
StatusCode::PAYLOAD_TOO_LARGE,
),
] {
let response = app
.clone()
.oneshot(
Request::builder()
.method("POST")
.uri(path)
.header("content-type", content_type)
.body(Body::from(input))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), status);
assert_eq!(response.headers()["content-type"], "application/json");
let error = response
.extensions()
.get::<Arc<PublicError>>()
.expect("typed error retained")
.clone();
assert_eq!(error.status, status.as_u16());
assert!(!error.message.is_empty());
let body = to_bytes(response.into_body(), usize::MAX).await.unwrap();
assert_eq!(
serde_json::from_slice::<serde_json::Value>(&body).unwrap(),
serde_json::json!({"message": error.message})
);
}
let response = app
.oneshot(
Request::builder()
.method("POST")
.uri("/json")
.header("content-type", "application/json")
.body(Body::from("{\"fail\":false}"))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
assert!(response.extensions().get::<Arc<PublicError>>().is_none());
assert_eq!(
to_bytes(response.into_body(), usize::MAX)
.await
.unwrap()
.as_ref(),
b"{\"message\":\"ok\"}"
);
}
#[test]
fn error_encoding_failure_does_not_retain_misleading_original_metadata() {
struct UnencodableError;
impl Serialize for UnencodableError {
fn serialize<S: serde::Serializer>(&self, _serializer: S) -> Result<S::Ok, S::Error> {
Err(serde::ser::Error::custom("cannot encode fixture error"))
}
}
for status in [StatusCode::BAD_GATEWAY, StatusCode::INTERNAL_SERVER_ERROR] {
let response = api_macros::axum::error_response(status, UnencodableError);
assert_eq!(response.status(), status);
assert_eq!(
response.headers()["content-type"],
"text/plain; charset=utf-8"
);
assert!(
response
.extensions()
.get::<Arc<UnencodableError>>()
.is_none()
);
}
}
+9
View File
@@ -0,0 +1,9 @@
[package]
name = "builtin-source"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish = false
[dependencies]
sha2.workspace = true
+269
View File
@@ -0,0 +1,269 @@
//! Explicit compile-time public Decodal source catalog, shared by native and WASM hosts.
use sha2::{Digest, Sha256};
use std::collections::BTreeMap;
pub const BUILTIN_PROFILE_CATALOG_ID: &str = "builtin-profiles-v2";
pub const BUILTIN_DEFAULT_PROFILE: &str = "builtin:default";
pub const BUILTIN_STANDALONE_PROFILE: &str = "builtin:standalone";
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BuiltinProfileImport {
pub specifier: &'static str,
pub resolved_path: &'static str,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BuiltinProfileResource {
pub selector: Option<&'static str>,
pub path: &'static str,
pub source: &'static str,
pub description: &'static str,
pub imports: &'static [BuiltinProfileImport],
}
const BASE_PATH: &str = "profiles/base.dcdl";
pub const DEFAULT_PATH: &str = "profiles/default.dcdl";
const BASE_IMPORT: &[BuiltinProfileImport] = &[BuiltinProfileImport {
specifier: "./base.dcdl",
resolved_path: BASE_PATH,
}];
const DEFAULT_IMPORT: &[BuiltinProfileImport] = &[BuiltinProfileImport {
specifier: "./default.dcdl",
resolved_path: DEFAULT_PATH,
}];
const NO_IMPORTS: &[BuiltinProfileImport] = &[];
pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
BuiltinProfileResource {
selector: None,
path: BASE_PATH,
source: include_str!("../../../resources/profiles/base.dcdl"),
description: "Shared built-in Profile defaults.",
imports: NO_IMPORTS,
},
BuiltinProfileResource {
selector: Some(BUILTIN_DEFAULT_PROFILE),
path: DEFAULT_PATH,
source: include_str!("../../../resources/profiles/default.dcdl"),
description: "Default Yoi coding profile.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some(BUILTIN_STANDALONE_PROFILE),
path: "profiles/standalone.dcdl",
source: include_str!("../../../resources/profiles/standalone.dcdl"),
description: "Standalone Yoi coding profile.",
imports: DEFAULT_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:standalone-subjektiv"),
path: "profiles/standalone-subjektiv.dcdl",
source: include_str!("../../../resources/profiles/standalone-subjektiv.dcdl"),
description: "Standalone with an explicitly connected local Subject.",
imports: &[BuiltinProfileImport {
specifier: "./standalone.dcdl",
resolved_path: "profiles/standalone.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:standalone-subjektiv-consolidation"),
path: "profiles/standalone-subjektiv-consolidation.dcdl",
source: include_str!("../../../resources/profiles/standalone-subjektiv-consolidation.dcdl"),
description: "Delegated local Subject Job policy.",
imports: &[BuiltinProfileImport {
specifier: "./job.dcdl",
resolved_path: "profiles/job.dcdl",
}],
},
BuiltinProfileResource {
selector: Some("builtin:ticket-worker"),
path: "profiles/ticket-worker.dcdl",
source: include_str!("../../../resources/profiles/ticket-worker.dcdl"),
description: "General intent-led Ticket work with optional result recording and authorized conclusion decisions.",
imports: &[BuiltinProfileImport {
specifier: "./default.dcdl",
resolved_path: DEFAULT_PATH,
}],
},
BuiltinProfileResource {
selector: Some("builtin:coder"),
path: "profiles/coder.dcdl",
source: include_str!("../../../resources/profiles/coder.dcdl"),
description: "Code implementation with optional Git/MR review recipe; no universal Ticket conclusion gate.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:companion"),
path: "profiles/companion.dcdl",
source: include_str!("../../../resources/profiles/companion.dcdl"),
description: "General assistance with Workspace tools.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:intake"),
path: "profiles/intake.dcdl",
source: include_str!("../../../resources/profiles/intake.dcdl"),
description: "Read-only intake and planning.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:reviewer"),
path: "profiles/reviewer.dcdl",
source: include_str!("../../../resources/profiles/reviewer.dcdl"),
description: "Independent review of a published Merge Request source.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:orchestrator"),
path: "profiles/orchestrator.dcdl",
source: include_str!("../../../resources/profiles/orchestrator.dcdl"),
description: "Intent-led Ticket orchestration with separate guarded MR integration authority.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:job"),
path: "profiles/job.dcdl",
source: include_str!("../../../resources/profiles/job.dcdl"),
description: "Result-only caller-selected Job policy.",
imports: NO_IMPORTS,
},
BuiltinProfileResource {
selector: Some("builtin:backend-job"),
path: "profiles/backend-job.dcdl",
source: include_str!("../../../resources/profiles/backend-job.dcdl"),
description: "Backend-owned bounded Job Worker.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:memory-consolidation"),
path: "profiles/memory-consolidation.dcdl",
source: include_str!("../../../resources/profiles/memory-consolidation.dcdl"),
description: "Internal Memory consolidation service.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:subjektiv-memory-consolidation"),
path: "profiles/subjektiv-memory-consolidation.dcdl",
source: include_str!("../../../resources/profiles/subjektiv-memory-consolidation.dcdl"),
description: "Delegated subject Memory consolidation service.",
imports: BASE_IMPORT,
},
];
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BuiltinProfileCatalogSnapshot {
pub id: &'static str,
pub sources: BTreeMap<String, String>,
pub entrypoints: BTreeMap<String, String>,
pub imports: BTreeMap<String, String>,
}
impl BuiltinProfileCatalogSnapshot {
pub fn digest(&self) -> String {
let mut hasher = Sha256::new();
hasher.update(self.id.as_bytes());
for (path, source) in &self.sources {
hasher.update((path.len() as u64).to_le_bytes());
hasher.update(path.as_bytes());
hasher.update((source.len() as u64).to_le_bytes());
hasher.update(source.as_bytes());
}
for (selector, path) in &self.entrypoints {
hasher.update((selector.len() as u64).to_le_bytes());
hasher.update(selector.as_bytes());
hasher.update((path.len() as u64).to_le_bytes());
hasher.update(path.as_bytes());
}
for (request, resolved_path) in &self.imports {
hasher.update((request.len() as u64).to_le_bytes());
hasher.update(request.as_bytes());
hasher.update((resolved_path.len() as u64).to_le_bytes());
hasher.update(resolved_path.as_bytes());
}
format!(
"sha256:{}",
hasher
.finalize()
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>()
)
}
}
pub fn builtin_profile_catalog_snapshot() -> BuiltinProfileCatalogSnapshot {
let mut sources = BTreeMap::new();
let mut entrypoints = BTreeMap::new();
let mut imports = BTreeMap::new();
for resource in BUILTIN_PROFILE_RESOURCES {
sources.insert(resource.path.to_owned(), resource.source.to_owned());
for import in resource.imports {
imports.insert(
format!("{}\0{}", resource.path, import.specifier),
import.resolved_path.to_owned(),
);
}
if let Some(selector) = resource.selector {
entrypoints.insert(selector.to_owned(), resource.path.to_owned());
}
}
BuiltinProfileCatalogSnapshot {
id: BUILTIN_PROFILE_CATALOG_ID,
sources,
entrypoints,
imports,
}
}
pub fn builtin_profile_entrypoints() -> impl Iterator<Item = &'static BuiltinProfileResource> {
BUILTIN_PROFILE_RESOURCES
.iter()
.filter(|resource| resource.selector.is_some())
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::BTreeSet;
#[test]
fn ticket_worker_catalog_import_graph_includes_default_and_base() {
// This snapshot supplies the Runtime profile source archive. Follow its
// declared edges rather than resolving against every catalog source.
let catalog = builtin_profile_catalog_snapshot();
let entrypoint = catalog.entrypoints["builtin:ticket-worker"].clone();
assert_eq!(
catalog
.imports
.get(&format!("{entrypoint}\0./default.dcdl")),
Some(&DEFAULT_PATH.to_owned())
);
assert_eq!(
catalog.imports.get(&format!("{DEFAULT_PATH}\0./base.dcdl")),
Some(&BASE_PATH.to_owned())
);
let mut pending = vec![entrypoint.clone()];
let mut reachable = BTreeSet::new();
while let Some(path) = pending.pop() {
if !reachable.insert(path.clone()) {
continue;
}
assert!(catalog.sources.contains_key(&path), "missing source {path}");
let prefix = format!("{path}\0");
pending.extend(
catalog
.imports
.iter()
.filter(|(request, _)| request.starts_with(&prefix))
.map(|(_, resolved)| resolved.clone()),
);
}
assert_eq!(
reachable,
BTreeSet::from([entrypoint, DEFAULT_PATH.to_owned(), BASE_PATH.to_owned()])
);
}
}
+41
View File
@@ -0,0 +1,41 @@
use builtin_source::{BUILTIN_PROFILE_RESOURCES, builtin_profile_catalog_snapshot};
use std::collections::BTreeSet;
#[test]
fn public_catalog_is_closed_and_explicit() {
let catalog = builtin_profile_catalog_snapshot();
assert_eq!(catalog.sources.len(), BUILTIN_PROFILE_RESOURCES.len());
assert!(catalog.digest().starts_with("sha256:"));
for resource in BUILTIN_PROFILE_RESOURCES {
assert!(resource.path.starts_with("profiles/"));
assert!(resource.path.ends_with(".dcdl"));
assert_eq!(catalog.sources[resource.path], resource.source);
for import in resource.imports {
assert!(catalog.sources.contains_key(import.resolved_path));
assert!(
resource
.source
.contains(&format!("import {:?}", import.specifier))
);
}
}
assert!(!catalog.sources.contains_key("prompts/default.md"));
}
#[test]
fn catalog_import_metadata_has_no_cycles() {
fn visit(path: &str, visiting: &mut BTreeSet<String>) {
assert!(visiting.insert(path.to_string()), "catalog cycle at {path}");
let resource = BUILTIN_PROFILE_RESOURCES
.iter()
.find(|resource| resource.path == path)
.unwrap();
for import in resource.imports {
visit(import.resolved_path, visiting);
}
visiting.remove(path);
}
for resource in BUILTIN_PROFILE_RESOURCES {
visit(resource.path, &mut BTreeSet::new());
}
}
+17 -2
View File
@@ -10,6 +10,7 @@ pub use server_api::{
WorkerLaunchProfileCandidate as BackendWorkerLaunchProfileCandidate,
WorkerLaunchRuntimeOption as BackendWorkerLaunchRuntimeOption,
WorkerOperationState as BackendWorkerOperationState,
WorkerRestoreRequest as BackendWorkerRestoreRequest,
WorkerRestoreResponse as BackendWorkerRestoreResponse,
WorkerRestoreResult as BackendWorkerRestoreResult,
WorkerRestoreState as BackendWorkerRestoreState, WorkerSummary as BackendWorkerSummary,
@@ -259,6 +260,15 @@ impl fmt::Display for BackendRuntimeClientError {
}
}
impl BackendRuntimeClientError {
/// A stale intent is not a rejected Restore operation and must not be retried
/// with a silently refreshed token.
pub fn is_restore_observation_conflict(&self) -> bool {
matches!(self, Self::ContractApi(server_api::client_support::ClientError::Public { status, error })
if *status == reqwest::StatusCode::CONFLICT && error.error == "restore_observation_conflict")
}
}
impl std::error::Error for BackendRuntimeClientError {}
impl From<server_api::ServerApiClientError> for BackendRuntimeClientError {
@@ -502,6 +512,7 @@ pub async fn list_backend_stopped_workers(
pub async fn restore_backend_worker(
target: &BackendRuntimeTarget,
request: BackendWorkerRestoreRequest,
) -> Result<BackendWorkerRestoreResponse, BackendRuntimeClientError> {
validate_target(target)?;
let api = BackendApiClient::from_stored_token(&target.base_url)?;
@@ -514,6 +525,7 @@ pub async fn restore_backend_worker(
ticket_id: None,
assignment_operation_id: None,
},
request,
)
.await
.map_err(Into::into)
@@ -811,7 +823,7 @@ mod tests {
"display_name": "Embedded",
"built_in": true,
"worker_creation_available": true,
"working_directory_required": false,
"supports_workdir_attachments": false,
"status": "online",
"diagnostics": []
}],
@@ -819,7 +831,8 @@ mod tests {
"profiles": [{
"id": "builtin:default",
"label": "Default",
"description": ""
"description": "",
"feature_connections": { "subjektiv": false }
}],
"repositories": [],
"working_directories": [],
@@ -878,10 +891,12 @@ mod tests {
let create = BackendCreateWorkerRequest {
runtime_id: "embedded".to_string(),
display_name: "Coder one".to_string(),
singleton_key: None,
profile: Some("builtin:coder".to_string()),
ticket_assignment: None,
initial_submit: Vec::new(),
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
};
+2 -2
View File
@@ -434,11 +434,11 @@ mod tests {
let request = CreateBackendWorkspaceRequest {
operation_key: "workspace-create-1".to_string(),
display_name: "Alpha".to_string(),
repository: CreateBackendWorkspaceRepository {
repository: Some(CreateBackendWorkspaceRepository {
repository_key: "main".to_string(),
uri: "/srv/repos/alpha".to_string(),
default_ref: Some("develop".to_string()),
},
}),
};
let retry = request.clone();
+6 -6
View File
@@ -26,12 +26,12 @@ pub use backend_runtime::{
BackendRuntimeListTarget, BackendRuntimeSummary, BackendRuntimeTarget,
BackendWorkerImplementationSummary, BackendWorkerLaunchOptions,
BackendWorkerLaunchProfileCandidate, BackendWorkerLaunchRuntimeOption,
BackendWorkerLaunchTarget, BackendWorkerOperationState, BackendWorkerRestoreResponse,
BackendWorkerRestoreResult, BackendWorkerRestoreState, BackendWorkerSummary,
BackendWorkerWorkspaceSummary, BackendWorkingDirectorySource, BackendWorkingDirectorySummary,
connect_backend_runtime, create_backend_worker, get_backend_worker_launch_options,
list_backend_stopped_workers, list_backend_workers, observe_backend_worker_session,
restore_backend_worker,
BackendWorkerLaunchTarget, BackendWorkerOperationState, BackendWorkerRestoreRequest,
BackendWorkerRestoreResponse, BackendWorkerRestoreResult, BackendWorkerRestoreState,
BackendWorkerSummary, BackendWorkerWorkspaceSummary, BackendWorkingDirectorySource,
BackendWorkingDirectorySummary, connect_backend_runtime, create_backend_worker,
get_backend_worker_launch_options, list_backend_stopped_workers, list_backend_workers,
observe_backend_worker_session, restore_backend_worker,
};
pub use backend_workspace::{
BackendWorkspace, BackendWorkspaceCatalogTarget, BackendWorkspaceClientError,
+323 -8
View File
@@ -3,8 +3,11 @@ use server_api::{
CreateWorkspaceWorkerRequest, ListResponse, MemoryDocumentResponse, MemoryStagingListResponse,
ObjectiveCreateRequest, ObjectiveDetail, ObjectiveEditRequest, ObjectiveLinkTicketRequest,
ObjectiveStateRequest, ObjectiveSummary, RevokeRuntimeTrustKeyRequest,
RuntimeTrustKeyRevealResponse, WorkerLaunchOptionsResponse, WorkspaceRuntimeDetail,
WorkspaceRuntimeResource,
RuntimeTrustKeyRevealResponse, SubjektivMemoryDetailQuery, SubjektivMemoryListQuery,
SubjektivMemoryListRevisionsResponse, SubjektivMemoryQueryResponse,
SubjektivMemoryReadResponse, SubjektivMemoryRevisionsQuery, SubjektivResidentSurfaceResponse,
SubjektivSubjectListQuery, SubjektivSubjectListResponse, SubjektivSubjectResponse,
WorkerLaunchOptionsResponse, WorkspaceRuntimeDetail, WorkspaceRuntimeResource,
};
use ticket::{
MarkdownText, NewOrchestrationPlanRecord, NewTicket, NewTicketEvent, NewTicketRelation,
@@ -152,14 +155,40 @@ impl BackendWorkspaceProductClient {
})
}
fn decision_from_state_change(
&self,
id: &TicketIdOrSlug,
change: &TicketStateChange,
) -> Result<ticket::TicketStateUpdate, BackendWorkspaceClientError> {
let current = self.show_ticket(id)?;
let parse = |state: &str| {
ticket::TicketWorkflowState::parse(state).ok_or_else(|| {
BackendWorkspaceClientError::InvalidTarget(format!("invalid Ticket state {state}"))
})
};
Ok(ticket::TicketStateUpdate {
operation_key: uuid::Uuid::now_v7().to_string(),
expected_item_revision: ticket::ticket_item_revision(&current),
expected_state: parse(&change.from)?,
state: parse(&change.to)?,
reason: if change.body.as_str().is_empty() {
change.reason.clone()
} else {
format!("{}\n\n{}", change.reason, change.body.as_str())
},
references: change.references.clone(),
author: change.author.clone(),
})
}
pub fn set_ticket_workflow_state(
&self,
id: &TicketIdOrSlug,
change: &TicketStateChange,
) -> Result<(), BackendWorkspaceClientError> {
let change = self.decision_from_state_change(id, change)?;
let workspace_id = self.workspace_id.clone();
let id = ticket_reference(id);
let change = change.clone();
self.generated(move |client| async move {
client
.ticket_workflow_state_set(
@@ -176,15 +205,23 @@ impl BackendWorkspaceProductClient {
id: &TicketIdOrSlug,
resolution: &MarkdownText,
) -> Result<(), BackendWorkspaceClientError> {
let current = self.show_ticket(id)?;
let request = ticket::TicketCompletion {
operation_key: uuid::Uuid::now_v7().to_string(),
expected_item_revision: ticket::ticket_item_revision(&current),
expected_state: current.meta.workflow_state,
reason: resolution.as_str().to_string(),
references: Vec::new(),
author: None,
};
let workspace_id = self.workspace_id.clone();
let id = ticket_reference(id);
let resolution = resolution.clone();
self.generated(move |client| async move {
client
.ticket_close_record(
workspace_id,
id,
server_api::TicketCloseRecordRequest(resolution),
server_api::TicketCloseRecordRequest(request),
)
.await
})
@@ -410,6 +447,97 @@ impl BackendWorkspaceProductClient {
crate::backend_workspace::memory_staging_list_blocking(&self.api, &self.workspace_id, limit)
}
pub fn list_subjektiv_subjects(
&self,
limit: usize,
) -> Result<SubjektivSubjectListResponse, BackendWorkspaceClientError> {
let workspace_id = self.workspace_id.clone();
self.generated(move |client| async move {
client
.subjektiv_subject_list(
workspace_id,
SubjektivSubjectListQuery {
limit: Some(limit),
cursor: None,
},
)
.await
})
}
pub fn subjektiv_subject(
&self,
subject_id: &str,
) -> Result<SubjektivSubjectResponse, BackendWorkspaceClientError> {
let workspace_id = self.workspace_id.clone();
let subject_id = subject_id.to_string();
self.generated(move |client| async move {
client.subjektiv_subject_get(workspace_id, subject_id).await
})
}
pub fn subjektiv_resident_surface(
&self,
subject_id: &str,
) -> Result<SubjektivResidentSurfaceResponse, BackendWorkspaceClientError> {
let workspace_id = self.workspace_id.clone();
let subject_id = subject_id.to_string();
self.generated(move |client| async move {
client
.subjektiv_resident_surface(workspace_id, subject_id)
.await
})
}
pub fn list_subjektiv_memories(
&self,
subject_id: &str,
query: &SubjektivMemoryListQuery,
) -> Result<SubjektivMemoryQueryResponse, BackendWorkspaceClientError> {
let workspace_id = self.workspace_id.clone();
let subject_id = subject_id.to_string();
let query = query.clone();
self.generated(move |client| async move {
client
.subjektiv_memory_list(workspace_id, subject_id, query)
.await
})
}
pub fn subjektiv_memory(
&self,
subject_id: &str,
memory_id: &str,
query: &SubjektivMemoryDetailQuery,
) -> Result<SubjektivMemoryReadResponse, BackendWorkspaceClientError> {
let workspace_id = self.workspace_id.clone();
let subject_id = subject_id.to_string();
let memory_id = memory_id.to_string();
let query = query.clone();
self.generated(move |client| async move {
client
.subjektiv_memory_detail(workspace_id, subject_id, memory_id, query)
.await
})
}
pub fn list_subjektiv_memory_revisions(
&self,
subject_id: &str,
memory_id: &str,
query: &SubjektivMemoryRevisionsQuery,
) -> Result<SubjektivMemoryListRevisionsResponse, BackendWorkspaceClientError> {
let workspace_id = self.workspace_id.clone();
let subject_id = subject_id.to_string();
let memory_id = memory_id.to_string();
let query = query.clone();
self.generated(move |client| async move {
client
.subjektiv_memory_revisions(workspace_id, subject_id, memory_id, query)
.await
})
}
pub fn launch_ticket_intake(
&self,
ticket_id: &str,
@@ -421,7 +549,7 @@ impl BackendWorkspaceProductClient {
let runtime = options
.runtimes
.iter()
.find(|runtime| runtime.worker_creation_available && !runtime.working_directory_required)
.find(|runtime| runtime.worker_creation_available)
.ok_or_else(|| {
BackendWorkspaceClientError::InvalidTarget(
"Backend has no spawn-capable Runtime that supports a Workdir-less Intake Worker"
@@ -431,12 +559,14 @@ impl BackendWorkspaceProductClient {
let request = CreateWorkspaceWorkerRequest {
runtime_id: runtime.runtime_id.clone(),
display_name: format!("intake-{ticket_id}"),
singleton_key: None,
profile: Some("builtin:intake".to_string()),
ticket_assignment: None,
initial_submit: vec![protocol::Segment::Text {
content: format!("Please handle intake for Ticket {ticket_id}."),
}],
workdir_attachments: Vec::new(),
feature_connections: Default::default(),
control_operation_id: None,
};
let workspace_id = self.workspace_id.clone();
@@ -537,6 +667,9 @@ impl TicketBackend for BackendWorkspaceProductClient {
id: TicketIdOrSlug,
change: TicketStateChange,
) -> ticket::Result<()> {
let change = self
.decision_from_state_change(&id, &change)
.map_err(ticket_client_error)?;
let workspace_id = self.workspace_id.clone();
let id = ticket_reference(&id);
self.generated(move |client| async move {
@@ -576,6 +709,14 @@ impl TicketBackend for BackendWorkspaceProductClient {
field: &str,
change: TicketStateChange,
) -> ticket::Result<()> {
if field != "state" {
return Err(TicketError::Conflict(
"unsupported Ticket state field".into(),
));
}
let change = self
.decision_from_state_change(&id, &change)
.map_err(ticket_client_error)?;
let workspace_id = self.workspace_id.clone();
let id = ticket_reference(&id);
let field = field.to_string();
@@ -601,6 +742,26 @@ impl TicketBackend for BackendWorkspaceProductClient {
.map_err(ticket_client_error)
}
fn update_state(
&self,
ticket: &str,
request: ticket::TicketStateUpdate,
) -> ticket::Result<Ticket> {
let workspace_id = self.workspace_id.clone();
let id = ticket.to_owned();
self.generated(move |client| async move {
client
.ticket_state_update(
workspace_id,
id,
server_api::TicketStateUpdateRequest(request),
)
.await
.map(|response| response.0)
})
.map_err(ticket_client_error)
}
fn mark_ready(&self, id: TicketIdOrSlug, request: TicketMarkReady) -> ticket::Result<Ticket> {
let workspace_id = self.workspace_id.clone();
let id = ticket_reference(&id);
@@ -807,8 +968,12 @@ mod tests {
}
fn response_sequence_server(
responses: Vec<(&'static str, &'static str)>,
responses: Vec<(&str, &str)>,
) -> (String, mpsc::Receiver<String>, thread::JoinHandle<()>) {
let responses = responses
.into_iter()
.map(|(status, body)| (status.to_owned(), body.to_owned()))
.collect::<Vec<_>>();
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let address = listener.local_addr().unwrap();
let (sender, receiver) = mpsc::channel();
@@ -831,6 +996,117 @@ mod tests {
(format!("http://{address}"), receiver, handle)
}
#[test]
fn legacy_state_adapter_uses_snapshot_revision_without_overriding_stale_expected_state() {
let temp = tempfile::tempdir().unwrap();
let backend =
ticket::SqliteTicketBackend::open(temp.path().join("fixture.db"), "workspace-a")
.unwrap();
let reference = backend.create(NewTicket::new("Adapter snapshot")).unwrap();
let snapshot = backend.show(reference.id.clone().into()).unwrap();
let body = serde_json::to_string(&snapshot).unwrap();
let (base_url, requests, handle) =
response_sequence_server(vec![("200 OK", &body), ("204 No Content", "")]);
let client =
BackendWorkspaceProductClient::new_with_access_token(base_url, "workspace-a", "token")
.unwrap();
client
.set_ticket_workflow_state(
&reference.id.clone().into(),
&TicketStateChange::new("ready", "done", "Judgment", "Results in the thread"),
)
.unwrap();
assert!(requests.recv().unwrap().starts_with("GET "));
let request = requests.recv().unwrap();
let body: serde_json::Value =
serde_json::from_str(request.split("\r\n\r\n").nth(1).unwrap()).unwrap();
assert_eq!(
body["expected_item_revision"],
ticket::ticket_item_revision(&snapshot)
);
assert_eq!(body["expected_state"], "ready");
assert_eq!(body["state"], "done");
assert!(!body["operation_key"].as_str().unwrap().is_empty());
handle.join().unwrap();
}
#[test]
fn legacy_close_adapter_forwards_snapshot_cas_and_resolution_as_judgment() {
let temp = tempfile::tempdir().unwrap();
let backend =
ticket::SqliteTicketBackend::open(temp.path().join("fixture.db"), "workspace-a")
.unwrap();
let reference = backend
.create(NewTicket::new("Close adapter snapshot"))
.unwrap();
let snapshot = backend.show(reference.id.clone().into()).unwrap();
let body = serde_json::to_string(&snapshot).unwrap();
let (base_url, requests, handle) =
response_sequence_server(vec![("200 OK", &body), ("204 No Content", "")]);
let client =
BackendWorkspaceProductClient::new_with_access_token(base_url, "workspace-a", "token")
.unwrap();
client
.close_ticket(
&reference.id.into(),
&MarkdownText::new("The request was withdrawn"),
)
.unwrap();
assert!(requests.recv().unwrap().starts_with("GET "));
let request = requests.recv().unwrap();
let body: serde_json::Value =
serde_json::from_str(request.split("\r\n\r\n").nth(1).unwrap()).unwrap();
assert_eq!(
body["expected_item_revision"],
ticket::ticket_item_revision(&snapshot)
);
assert_eq!(body["expected_state"], "planning");
assert_eq!(body["reason"], "The request was withdrawn");
assert!(!body["operation_key"].as_str().unwrap().is_empty());
handle.join().unwrap();
}
#[test]
fn ticket_backend_completion_forwards_cas_judgment_and_backend_conflicts() {
let (base_url, request, handle) = one_response_server(
"409 Conflict",
r#"{"error":"ticket_conflict","message":"stale revision","diagnostics":[]}"#,
);
let client = BackendWorkspaceProductClient::new_with_access_token(
base_url,
"workspace-a",
"test-token",
)
.unwrap();
let result = TicketBackend::complete(
&client,
"T-718",
ticket::TicketCompletion {
operation_key: "research-done".into(),
expected_item_revision: "ticket:1".into(),
expected_state: ticket::TicketWorkflowState::Planning,
reason: "Research findings recorded".into(),
references: Vec::new(),
author: None,
},
);
assert!(result.is_err());
let request = request.recv().unwrap();
assert!(
request.starts_with("POST /api/w/workspace-a/tickets/T-718/state-update "),
"{request}"
);
let body: serde_json::Value =
serde_json::from_str(request.split("\r\n\r\n").nth(1).unwrap()).unwrap();
assert_eq!(body["state"], "done");
assert_eq!(body["operation_key"], "research-done");
assert_eq!(body["expected_item_revision"], "ticket:1");
assert_eq!(body["expected_state"], "planning");
assert_eq!(body["reason"], "Research findings recorded");
assert!(body.get("merge_request_ids").is_none());
handle.join().unwrap();
}
#[test]
fn memory_document_uses_shared_workspace_scoped_response() {
let body = r##"{"body_md":"# Memory\\n","created_at":"2026-09-01T00:00:00Z","updated_at":"2026-09-02T00:00:00Z","bytes":10,"record_source":"workspace-sqlite"}"##;
@@ -916,6 +1192,45 @@ mod tests {
handle.join().unwrap();
}
#[test]
fn subjektiv_memory_list_encodes_typed_filters_as_bounded_query_scalars() {
let (base_url, request, handle) =
one_response_server("200 OK", r#"{"items":[],"has_more":false}"#);
let client = BackendWorkspaceProductClient::new_with_access_token(
base_url,
"workspace-a",
"test-backend-token",
)
.unwrap();
let response = client
.list_subjektiv_memories(
"subject/a",
&SubjektivMemoryListQuery {
query: Some("stable cursor".to_string()),
kinds: None,
states: Some(vec![
server_api::SubjektivMemoryState::Active,
server_api::SubjektivMemoryState::Resolved,
]),
limit: Some(25),
cursor: None,
},
)
.unwrap();
assert!(response.items.is_empty());
let request = request.recv().unwrap();
assert!(
request.starts_with("GET /api/w/workspace-a/subjektiv/subjects/subject%2Fa/memories?")
);
assert!(request.contains("query=stable+cursor"));
assert!(request.contains("states=active%2Cresolved"));
assert!(request.contains("limit=25"));
assert!(request.contains("authorization: Bearer test-backend-token\r\n"));
handle.join().unwrap();
}
#[test]
fn objective_list_uses_workspace_scoped_backend_route() {
let body = r#"{"workspace_id":"workspace-a","limit":1000,"items":[],"invalid_records":[{"label":"broken.md","reason":"invalid frontmatter"}],"record_authority":"sqlite"}"#;
@@ -1025,7 +1340,7 @@ mod tests {
let (base_url, requests, handle) = response_sequence_server(vec![
(
"200 OK",
r#"{"workspace_id":"workspace-a","runtimes":[{"runtime_id":"embedded","display_name":"Embedded","built_in":true,"worker_creation_available":true,"working_directory_required":false,"status":"connected","diagnostics":[]}],"default_profile":null,"profiles":[],"repositories":[],"working_directories":[],"diagnostics":[]}"#,
r#"{"workspace_id":"workspace-a","runtimes":[{"runtime_id":"embedded","display_name":"Embedded","built_in":true,"worker_creation_available":true,"supports_workdir_attachments":false,"status":"connected","diagnostics":[]}],"default_profile":null,"profiles":[],"repositories":[],"working_directories":[],"diagnostics":[]}"#,
),
(
"200 OK",
+2 -2
View File
@@ -8,11 +8,11 @@ fn workspace_creation_request_preserves_operation_key_for_retry() {
let request = CreateBackendWorkspaceRequest {
operation_key: "workspace-create-1".to_string(),
display_name: "Alpha".to_string(),
repository: CreateBackendWorkspaceRepository {
repository: Some(CreateBackendWorkspaceRepository {
repository_key: "main".to_string(),
uri: "/srv/repos/alpha".to_string(),
default_ref: Some("develop".to_string()),
},
}),
};
assert_eq!(request.clone(), request);
+133 -22
View File
@@ -98,31 +98,47 @@ pub fn complete_current(
let snapshot = session
.as_ref()
.ok_or_else(|| JsValue::from_str("config source snapshot is not initialized"))?;
let utf8_byte_offset = utf16_to_utf8_offset(&source, utf16_offset)?;
let result = session_environment(snapshot.clone())
.complete_config(&entrypoint, &source, utf8_byte_offset, explicit)
.map_err(|error| JsValue::from_str(&format!("{error:?}")))?;
let result = result
.map(|result| {
Ok::<WasmCompletionResult, JsValue>(WasmCompletionResult {
from: utf8_to_utf16_offset(&source, result.from)?,
items: result
.items
.into_iter()
.map(|item| WasmCompletionItem {
label: item.label,
kind: format!("{:?}", item.kind).to_lowercase(),
detail: item.detail,
priority: item.priority,
})
.collect(),
})
})
.transpose()?;
encode(result)
encode(complete_snapshot(
snapshot,
&entrypoint,
&source,
utf16_offset,
explicit,
)?)
})
}
fn complete_snapshot(
snapshot: &ConfigTreeSnapshot,
entrypoint: &VirtualPath,
source: &str,
utf16_offset: usize,
explicit: bool,
) -> Result<Option<WasmCompletionResult>, JsValue> {
let utf8_byte_offset = utf16_to_utf8_offset(source, utf16_offset)?;
let result = session_environment(snapshot.clone())
.complete_config(entrypoint, source, utf8_byte_offset, explicit)
.map_err(|error| JsValue::from_str(&format!("{error:?}")))?;
let result = result
.map(|result| {
Ok::<WasmCompletionResult, JsValue>(WasmCompletionResult {
from: utf8_to_utf16_offset(source, result.from)?,
items: result
.items
.into_iter()
.map(|item| WasmCompletionItem {
label: item.label,
kind: format!("{:?}", item.kind).to_lowercase(),
detail: item.detail,
priority: item.priority,
})
.collect(),
})
})
.transpose()?;
Ok(result)
}
#[wasm_bindgen]
pub fn evaluate_snapshot(snapshot: JsValue, contract: JsValue) -> Result<JsValue, JsValue> {
let snapshot: ConfigTreeSnapshot = decode(snapshot)?;
@@ -207,3 +223,98 @@ fn js_error(error: impl std::fmt::Display) -> JsValue {
#[allow(dead_code)]
fn _assert_serializable(_: EvaluationResult) {}
#[cfg(test)]
mod tests {
use super::*;
use config_source::{ConfigContentType, ConfigEntry};
fn snapshot() -> ConfigTreeSnapshot {
ConfigTreeSnapshot::from_entries(
7,
[
ConfigEntry::new(
VirtualPath::parse("main.dcdl").unwrap(),
ConfigContentType::Decodal,
"{}",
)
.unwrap(),
// A same-suffix workspace file must not be an unknown builtin fallback.
ConfigEntry::new(
VirtualPath::parse("profiles/missing.dcdl").unwrap(),
ConfigContentType::Decodal,
"{}",
)
.unwrap(),
],
)
.unwrap()
}
#[test]
fn wasm_completion_boundary_preserves_read_only_builtin_detail() {
let snapshot = snapshot();
let source = "let 名 = 1; import \"$builtin/profiles/comp\"";
let cursor = source.encode_utf16().count() - 1;
let result = complete_snapshot(
&snapshot,
&VirtualPath::parse("main.dcdl").unwrap(),
source,
cursor,
true,
)
.unwrap()
.unwrap();
assert_eq!(result.from, "let 名 = 1; import \"".encode_utf16().count());
let item = result
.items
.iter()
.find(|item| item.label == "$builtin/profiles/companion.dcdl")
.unwrap();
assert_eq!(item.kind, "file");
assert_eq!(
item.detail.as_deref(),
Some("read-only builtin Decodal source")
);
assert_eq!(snapshot.entries.len(), 2);
assert!(
snapshot
.entries
.keys()
.all(|path| !path.as_str().starts_with("$builtin/"))
);
}
#[test]
fn wasm_analysis_boundary_reports_unknown_builtin_without_workspace_fallback() {
let snapshot = snapshot();
let path = VirtualPath::parse("main.dcdl").unwrap();
let diagnostics = session_environment(snapshot.clone())
.analyze(&path, Some("import \"$builtin/profiles/missing.dcdl\""));
assert!(!diagnostics.is_empty());
let diagnostic = diagnostics
.iter()
.find(|diagnostic| {
diagnostic
.message
.contains("unknown or non-public read-only builtin source")
})
.unwrap();
assert_eq!(diagnostic.path, path);
assert_eq!(diagnostic.revision, 7);
assert_eq!(diagnostic.tree_digest, snapshot.digest);
assert!(
diagnostic
.message
.contains("$builtin/profiles/missing.dcdl")
);
assert!(
session_environment(snapshot)
.analyze(
&VirtualPath::parse("main.dcdl").unwrap(),
Some("import \"./profiles/missing.dcdl\"")
)
.is_empty()
);
}
}
+1
View File
@@ -6,6 +6,7 @@ license.workspace = true
publish = false
[dependencies]
builtin-source.workspace = true
decodal.workspace = true
decodal-language-service.workspace = true
decodal-language-tools.workspace = true
File diff suppressed because it is too large Load Diff
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "feature-storage"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish = false
[dependencies]
rusqlite.workspace = true
thiserror.workspace = true
uuid = { workspace = true, features = ["v7"] }
[dev-dependencies]
tempfile.workspace = true
+28
View File
@@ -0,0 +1,28 @@
# feature-storage
Neutral Host-owned SQLite lifecycle manager for trusted Features. Owns placement,
WAL/connection settings, transaction serialization, migrations, snapshots,
restore/delete fencing, and shutdown. Features receive a managed
`FeatureDatabase` after registration. Worker/model input never selects a database
path or receives SQL access.
Use `FeatureStorage::new(trusted_root)`, `register(FeatureRegistration)`, and
`scope(host_issued_scope_id).open(&registration)`. `ScopedFeatureStorage` and
`FeatureDatabase::scope_id()` are neutral storage isolation APIs: a local Host's
persisted random scope does not create a Workspace record or imply Backend
membership. Existing placement `<root>/<scope>/features/<feature>.sqlite` and
snapshot `workspace=` keys remain unchanged for physical compatibility.
`WorkspaceFeatureStorage`, `workspace()`, `workspace_id()`, and
`for_server_database()` are Backend compatibility aliases/helpers. They do not
introduce a Server dependency. The Host is responsible for state-root privacy,
symlink/path validation, and external execution leases before constructing this
trusted manager.
`configure_connection(&rusqlite::Connection)` is reusable by other Host stores
(such as JobStore). It sets the five-second busy timeout, foreign keys ON, WAL,
and synchronous FULL before migrations/transactions. The Feature manager uses
this same helper; WAL lock-contention retries stay within the busy timeout.
Dependencies: `rusqlite` (existing workspace backup support), `thiserror`, `uuid`;
tests use `tempfile`.
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -24,7 +24,7 @@ pub fn builtin_flow_source(slug: &str) -> Option<BuiltinFlowSource> {
match slug {
CODER_REVIEW_FLOW_SLUG => Some(BuiltinFlowSource {
slug: CODER_REVIEW_FLOW_SLUG,
revision: 4,
revision: 5,
path: "builtin/flows/coder-review.dcdl",
content: CODER_REVIEW_FLOW_SOURCE,
}),
@@ -35,7 +35,7 @@ pub fn builtin_flow_source(slug: &str) -> Option<BuiltinFlowSource> {
pub fn builtin_flow_sources() -> &'static [BuiltinFlowSource] {
const SOURCES: &[BuiltinFlowSource] = &[BuiltinFlowSource {
slug: CODER_REVIEW_FLOW_SLUG,
revision: 4,
revision: 5,
path: "builtin/flows/coder-review.dcdl",
content: CODER_REVIEW_FLOW_SOURCE,
}];
+1 -3
View File
@@ -13,9 +13,7 @@ ignore = "0.4.25"
libc = "0.2"
schemars.workspace = true
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
sha2 = "0.10.9"
tempfile.workspace = true
thiserror.workspace = true
[dev-dependencies]
serde_json.workspace = true
+518
View File
@@ -0,0 +1,518 @@
//! Identity-fenced access adapter for the existing read/edit/write engine.
//!
//! Descriptor-rooted resolution is Linux-only and symlinks/mount crossings fail
//! closed. The final compare and atomic rename are NOT a kernel CAS against an
//! arbitrary external editor: external editors must cooperate with provider
//! serialization. A hostile rename in that last syscall gap is not excluded.
use crate::{AtomicWriteMode, FsAccessPolicy};
use std::fs::{File, Metadata};
use std::io::{Seek, SeekFrom};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
/// Identity and state, deliberately not just a content digest. atime is omitted
/// so reading does not invalidate an observation. No host paths are encoded.
pub fn identity_validator(metadata: &Metadata) -> std::io::Result<Vec<u8>> {
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
let mut bytes = b"yoi-checkout-v1".to_vec();
for value in [
metadata.dev(),
metadata.ino(),
metadata.len(),
metadata.mode() as u64,
metadata.mtime() as u64,
metadata.mtime_nsec() as u64,
metadata.ctime() as u64,
metadata.ctime_nsec() as u64,
] {
bytes.extend_from_slice(&value.to_le_bytes());
}
Ok(bytes)
}
#[cfg(not(unix))]
{
let _ = metadata;
Err(std::io::Error::new(
std::io::ErrorKind::Unsupported,
"identity validators unavailable",
))
}
}
/// One pinned target and its save parent. Retained through the shared operation.
pub struct CheckedTarget<'a> {
policy: &'a dyn FsAccessPolicy,
root: File,
relative: PathBuf,
logical: PathBuf,
file: File,
parent: Option<File>,
validator: Vec<u8>,
effects_possible: AtomicBool,
}
impl<'a> CheckedTarget<'a> {
pub fn pin(
root: &File,
relative: &Path,
logical: &Path,
policy: &'a dyn FsAccessPolicy,
write: bool,
) -> std::io::Result<Self> {
policy.check_cancelled()?;
if !(if write {
policy.is_writable_paths(logical, logical)
} else {
policy.is_readable_paths(logical, logical)
}) {
return Err(std::io::Error::new(
std::io::ErrorKind::PermissionDenied,
"checkout target denied",
));
}
let parent = if relative.as_os_str().is_empty() {
None
} else {
Some(crate::open_beneath_no_symlinks_at(
root,
relative.parent().unwrap_or(Path::new("")),
)?)
};
let file = if let Some(parent) = &parent {
crate::open_beneath_no_symlinks_at(parent, Path::new(relative.file_name().unwrap()))?
} else {
crate::open_beneath_no_symlinks_at(root, Path::new(""))?
};
let metadata = file.metadata()?;
if !metadata.is_file() && !metadata.is_dir() {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"checkout requires regular file or directory",
));
}
let result = Self {
policy,
root: root.try_clone()?,
relative: relative.to_path_buf(),
logical: logical.to_path_buf(),
file,
parent,
validator: identity_validator(&metadata)?,
effects_possible: AtomicBool::new(false),
};
result.confirm()?;
Ok(result)
}
pub fn validator(&self) -> &[u8] {
&self.validator
}
pub fn metadata(&self) -> std::io::Result<Metadata> {
self.file.metadata()
}
pub fn effects_possible(&self) -> bool {
self.effects_possible.load(Ordering::Acquire)
}
pub fn confirm(&self) -> std::io::Result<()> {
self.policy.check_cancelled()?;
if let Some(parent) = &self.parent {
let named_parent = crate::open_beneath_no_symlinks_at(
&self.root,
self.relative.parent().unwrap_or(Path::new("")),
)?;
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
let pinned_metadata = parent.metadata()?;
let named_metadata = named_parent.metadata()?;
if (pinned_metadata.dev(), pinned_metadata.ino())
!= (named_metadata.dev(), named_metadata.ino())
{
return Err(std::io::Error::new(
std::io::ErrorKind::WouldBlock,
"checkout parent identity changed",
));
}
}
}
let named = crate::open_beneath_no_symlinks_at(&self.root, &self.relative)?;
if identity_validator(&named.metadata()?)? != self.validator
|| identity_validator(&self.file.metadata()?)? != self.validator
{
return Err(std::io::Error::new(
std::io::ErrorKind::WouldBlock,
"checkout observation changed",
));
}
Ok(())
}
/// Refresh a directory observation after Create, retaining the original
/// pinned target/root/parent handles. A replacement named directory cannot
/// supply the validator for effects performed in the original directory.
pub fn refresh_directory_observation(&mut self) -> std::io::Result<()> {
let metadata = self.file.metadata()?;
if !metadata.is_dir() {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"Create target must be directory",
));
}
self.validator = identity_validator(&metadata)?;
self.confirm()
}
/// Adapter for no-clobber creation beneath a pinned directory. Both target
/// and destination authorization remain in the provider's normal policy.
pub fn create_access<'b>(&'b self, suffix: PathBuf, logical: PathBuf) -> CheckedCreate<'b, 'a> {
CheckedCreate {
target: self,
suffix,
logical,
}
}
}
impl FsAccessPolicy for CheckedTarget<'_> {
fn is_readable(&self, p: &Path) -> bool {
self.policy.is_readable(p)
}
fn is_writable(&self, p: &Path) -> bool {
self.policy.is_writable(p)
}
fn is_readable_paths(&self, p: &Path, r: &Path) -> bool {
p == self.logical && r == p && self.policy.is_readable_paths(p, r)
}
fn is_writable_paths(&self, p: &Path, r: &Path) -> bool {
p == self.logical && r == p && self.policy.is_writable_paths(p, r)
}
fn check_cancelled(&self) -> std::io::Result<()> {
self.policy.check_cancelled()
}
fn resolve_access_path(&self, p: &Path) -> std::io::Result<PathBuf> {
Ok(p.to_path_buf())
}
fn read_metadata(&self, _: &Path, _: &Path) -> std::io::Result<Metadata> {
self.file.metadata()
}
fn open_read_file(&self, _: &Path, _: &Path) -> std::io::Result<File> {
self.confirm()?;
let mut file = self.file.try_clone()?;
file.seek(SeekFrom::Start(0))?;
Ok(file)
}
fn max_write_bytes(&self) -> Option<usize> {
Some(
self.policy
.max_write_bytes()
.unwrap_or(8 * 1024 * 1024)
.min(8 * 1024 * 1024),
)
}
fn max_edit_replacements(&self) -> Option<usize> {
Some(
self.policy
.max_edit_replacements()
.unwrap_or(10_000)
.min(10_000),
)
}
fn atomic_write_file(
&self,
_: &Path,
_: &Path,
content: &[u8],
mode: AtomicWriteMode,
) -> std::io::Result<()> {
self.confirm()?;
let parent = self.parent.as_ref().ok_or_else(|| {
std::io::Error::new(std::io::ErrorKind::InvalidInput, "cannot save root")
})?;
crate::atomic_write_beneath_no_symlinks_at(
parent,
Path::new(self.relative.file_name().unwrap()),
content,
mode,
|| {
self.confirm()?;
// From here rename/encoding/post-observation failures have an unknown outcome.
self.effects_possible.store(true, Ordering::Release);
Ok(())
},
)
}
}
pub struct CheckedCreate<'b, 'a> {
target: &'b CheckedTarget<'a>,
suffix: PathBuf,
logical: PathBuf,
}
impl FsAccessPolicy for CheckedCreate<'_, '_> {
fn is_readable(&self, p: &Path) -> bool {
self.target.policy.is_readable(p)
}
fn is_writable(&self, p: &Path) -> bool {
p == self.logical && self.target.policy.is_writable_paths(p, p)
}
fn resolve_access_path(&self, p: &Path) -> std::io::Result<PathBuf> {
Ok(p.to_path_buf())
}
fn check_cancelled(&self) -> std::io::Result<()> {
self.target.policy.check_cancelled()
}
fn max_write_bytes(&self) -> Option<usize> {
self.target.max_write_bytes()
}
fn read_metadata(&self, _: &Path, _: &Path) -> std::io::Result<Metadata> {
match crate::open_beneath_no_symlinks_at(&self.target.file, &self.suffix) {
Ok(_) => Err(std::io::Error::new(
std::io::ErrorKind::AlreadyExists,
"Create destination exists",
)),
Err(e) => Err(e),
}
}
fn atomic_write_file(
&self,
_: &Path,
_: &Path,
content: &[u8],
mode: AtomicWriteMode,
) -> std::io::Result<()> {
self.target.confirm()?;
if mode != AtomicWriteMode::CreateNew || !self.target.file.metadata()?.is_dir() {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"Create requires directory",
));
}
// Creating parents must not turn a narrow destination grant into
// permission to populate ungranted intermediate directories.
let mut parent = self.target.logical.clone();
let mut components = self.suffix.components().peekable();
while let Some(component) = components.next() {
parent.push(component.as_os_str());
if components.peek().is_some()
&& !self.target.policy.is_writable_paths(&parent, &parent)
{
return Err(std::io::Error::new(
std::io::ErrorKind::PermissionDenied,
"Create parent denied",
));
}
}
// Nested mkdir may have effects before failure; never auto-retry.
self.target.effects_possible.store(true, Ordering::Release);
crate::atomic_write_beneath_no_symlinks_at(
&self.target.file,
&self.suffix,
content,
mode,
|| self.target.policy.check_cancelled(),
)
}
}
#[cfg(all(test, target_os = "linux"))]
mod tests {
use super::*;
use crate::{FsError, FsPath, ReadRequest, WriteRequest};
use sha2::{Digest, Sha256};
use std::sync::atomic::AtomicUsize;
struct Access {
path: PathBuf,
counter: AtomicUsize,
trigger: usize,
replacement: bool,
}
impl FsAccessPolicy for Access {
fn is_readable(&self, _: &Path) -> bool {
true
}
fn is_writable(&self, _: &Path) -> bool {
true
}
fn check_cancelled(&self) -> std::io::Result<()> {
if self.counter.fetch_add(1, Ordering::SeqCst) + 1 == self.trigger {
if self.replacement {
std::fs::rename(&self.path, self.path.with_extension("old"))?;
std::fs::write(&self.path, "replacement")?;
} else {
std::fs::write(&self.path, "updated")?;
}
}
Ok(())
}
}
#[test]
fn checked_save_boundary_rejects_name_substitution_without_mutating_replacement() {
let root = tempfile::tempdir().unwrap();
let path = root.path().join("a");
std::fs::write(&path, "original").unwrap();
let policy = Access {
path: path.clone(),
counter: AtomicUsize::new(0),
trigger: 4,
replacement: true,
};
let handle = crate::open_root_no_symlinks(root.path()).unwrap();
let checked = CheckedTarget::pin(&handle, Path::new("a"), &path, &policy, true).unwrap();
policy.counter.store(0, Ordering::SeqCst);
let error = crate::run_write(
root.path(),
WriteRequest {
path: FsPath::new("a").unwrap(),
content: b"worker".to_vec(),
expected_hash: Some(Sha256::digest(b"original").into()),
},
&checked,
)
.unwrap_err();
assert!(
matches!(error, FsError::Io { source, .. } if source.kind() == std::io::ErrorKind::WouldBlock)
);
assert!(!checked.effects_possible());
assert_eq!(std::fs::read_to_string(&path).unwrap(), "replacement");
assert_eq!(
std::fs::read_to_string(path.with_extension("old")).unwrap(),
"original"
);
}
#[test]
fn checked_read_midread_name_replacement_never_pairs_old_bytes_with_new_validator() {
let root = tempfile::tempdir().unwrap();
let path = root.path().join("a");
std::fs::write(&path, "original").unwrap();
let policy = Access {
path: path.clone(),
counter: AtomicUsize::new(0),
trigger: 3,
replacement: true,
};
let handle = crate::open_root_no_symlinks(root.path()).unwrap();
let checked = CheckedTarget::pin(&handle, Path::new("a"), &path, &policy, false).unwrap();
let validator = checked.validator().to_vec();
policy.counter.store(0, Ordering::SeqCst);
let result = crate::run_read(
root.path(),
ReadRequest {
path: FsPath::new("a").unwrap(),
offset: 0,
limit: 20,
max_bytes: 1024,
},
&checked,
);
match result {
Err(FsError::Conflict(_)) => {}
Ok(read) => {
// Same pinned inode supplies bytes and hash. Execute's final
// named-target check must reject, not refresh to replacement.
assert_eq!(read.bytes, b"original");
let expected_hash: crate::ContentHash = Sha256::digest(b"original").into();
assert_eq!(read.content_hash, expected_hash);
assert_eq!(
checked.confirm().unwrap_err().kind(),
std::io::ErrorKind::WouldBlock
);
}
other => panic!("unexpected Read result: {other:?}"),
}
assert_eq!(checked.validator(), validator);
assert_ne!(
identity_validator(&std::fs::metadata(&path).unwrap()).unwrap(),
validator
);
assert_eq!(std::fs::read_to_string(&path).unwrap(), "replacement");
}
#[test]
fn create_post_parent_observation_retains_bound_directory_and_rejects_substitution() {
let root = tempfile::tempdir().unwrap();
let directory = root.path().join("d");
std::fs::create_dir(&directory).unwrap();
let policy = Access {
path: directory.clone(),
counter: AtomicUsize::new(0),
trigger: usize::MAX,
replacement: false,
};
let handle = crate::open_root_no_symlinks(root.path()).unwrap();
let mut checked =
CheckedTarget::pin(&handle, Path::new("d"), &directory, &policy, true).unwrap();
let initial = checked.validator().to_vec();
let create = checked.create_access(PathBuf::from("new"), directory.join("new"));
crate::run_write(
root.path(),
WriteRequest {
path: FsPath::new("d/new").unwrap(),
content: b"new".to_vec(),
expected_hash: None,
},
&create,
)
.unwrap();
checked.refresh_directory_observation().unwrap();
assert_ne!(initial, checked.validator());
assert_eq!(
checked.validator(),
identity_validator(&std::fs::metadata(&directory).unwrap()).unwrap()
);
std::fs::rename(&directory, root.path().join("old")).unwrap();
std::fs::create_dir(&directory).unwrap();
assert_eq!(
checked.refresh_directory_observation().unwrap_err().kind(),
std::io::ErrorKind::WouldBlock
);
assert!(checked.effects_possible());
}
#[test]
fn checked_parent_swap_is_rejected_even_with_a_hardlinked_same_inode_target() {
let root = tempfile::tempdir().unwrap();
let directory = root.path().join("d");
std::fs::create_dir(&directory).unwrap();
let path = directory.join("a");
std::fs::write(&path, "original").unwrap();
let policy = Access {
path: path.clone(),
counter: AtomicUsize::new(0),
trigger: usize::MAX,
replacement: false,
};
let handle = crate::open_root_no_symlinks(root.path()).unwrap();
let checked = CheckedTarget::pin(&handle, Path::new("d/a"), &path, &policy, true).unwrap();
std::fs::rename(&directory, root.path().join("old")).unwrap();
std::fs::create_dir(&directory).unwrap();
std::fs::hard_link(root.path().join("old/a"), &path).unwrap();
assert_eq!(
checked.confirm().unwrap_err().kind(),
std::io::ErrorKind::WouldBlock
);
assert!(!checked.effects_possible());
}
#[test]
fn shared_text_read_rejects_mid_read_metadata_change() {
let root = tempfile::tempdir().unwrap();
let path = root.path().join("a");
std::fs::write(&path, "original").unwrap();
let policy = Access {
path,
counter: AtomicUsize::new(0),
trigger: 2,
replacement: false,
};
let error = crate::run_read(
root.path(),
ReadRequest {
path: FsPath::new("a").unwrap(),
offset: 0,
limit: 20,
max_bytes: 1024,
},
&policy,
)
.unwrap_err();
assert!(matches!(error, FsError::Conflict(_)));
}
}
@@ -0,0 +1,294 @@
//! Ignore rules loaded only through the provider's authorized descriptor opens.
//! Never use GitignoreBuilder::add/build_global: those reopen host pathnames.
use crate::{FsAccessPolicy, FsError};
use ignore::gitignore::{Gitignore, GitignoreBuilder};
use std::io::{BufRead, BufReader, Read};
use std::path::{Path, PathBuf};
pub(crate) const MAX_IGNORE_SOURCE_BYTES: u64 = 8 * 1024 * 1024;
const MAX_IGNORE_FILE_BYTES: u64 = 1024 * 1024;
const MAX_IGNORE_LINE_BYTES: u64 = 16 * 1024;
pub(crate) struct SourceBoundedReader<'a, R> {
pub inner: R,
pub remaining: &'a mut u64,
pub access: &'a dyn FsAccessPolicy,
}
impl<R: Read> Read for SourceBoundedReader<'_, R> {
fn read(&mut self, buffer: &mut [u8]) -> std::io::Result<usize> {
self.access.check_cancelled()?;
if buffer.is_empty() {
return Ok(0);
}
if *self.remaining == 0 {
let mut probe = [0_u8; 1];
return match self.inner.read(&mut probe)? {
0 => Ok(0),
_ => Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"search source exceeded provider byte limit",
)),
};
}
let limit = usize::try_from(*self.remaining)
.unwrap_or(usize::MAX)
.min(buffer.len());
let read = self.inner.read(&mut buffer[..limit])?;
*self.remaining = (*self.remaining).saturating_sub(read as u64);
Ok(read)
}
}
/// Shared Glob/Grep grammar and provider authorization. Glob, like WalkBuilder,
/// ignores unavailable configuration files; Grep retains its existing explicit
/// error on a rejected ignore-file open. Cancellation/bounds always fail closed.
pub(crate) fn add_checked_ignore_file(
builder: &mut GitignoreBuilder,
path: &Path,
remaining: &mut u64,
access: &dyn FsAccessPolicy,
skip_unavailable: bool,
) -> Result<bool, FsError> {
access.check_cancelled().map_err(|e| FsError::io(path, e))?;
let resolved = match access.resolve_access_path(path) {
Ok(path) => path,
Err(error) if skip_unavailable && error.kind() != std::io::ErrorKind::Interrupted => {
return Ok(false);
}
Err(error) => return Err(FsError::io(path, error)),
};
if !access.is_readable_paths(path, &resolved) {
return Ok(false);
}
let file = match access.open_read_file(path, &resolved) {
Ok(file) => file,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(false),
Err(error) if skip_unavailable && error.kind() != std::io::ErrorKind::Interrupted => {
return Ok(false);
}
Err(error) => return Err(FsError::io(path, error)),
};
let metadata = file.metadata().map_err(|e| FsError::io(path, e))?;
if !metadata.is_file() {
return Ok(false);
}
if metadata.len() > MAX_IGNORE_FILE_BYTES {
return Err(FsError::InvalidArgument(
"ignore file exceeds provider limit 1 MiB".into(),
));
}
let bounded = SourceBoundedReader {
inner: file,
remaining,
access,
};
// Take also fences files that grow after metadata. Each line is bounded
// before allocation, and every underlying read checks cancellation.
let mut reader = BufReader::new(bounded.take(MAX_IGNORE_FILE_BYTES + 1));
let mut line = Vec::new();
let mut bytes = 0_u64;
let mut added = false;
loop {
line.clear();
// BufRead::read_line/read_until retry Interrupted internally. Explicit
// fill_buf prevents a cooperative cancellation from becoming a retry
// loop, and caps allocation before copying even a growing long line.
loop {
access.check_cancelled().map_err(|e| FsError::io(path, e))?;
let buffer = reader.fill_buf().map_err(|e| FsError::io(path, e))?;
if buffer.is_empty() {
break;
}
let count = buffer
.iter()
.position(|byte| *byte == b'\n')
.map_or(buffer.len(), |index| index + 1);
if line.len().saturating_add(count) as u64 > MAX_IGNORE_LINE_BYTES {
return Err(FsError::InvalidArgument(
"ignore line exceeds provider limit 16 KiB".into(),
));
}
let finished = buffer[count - 1] == b'\n';
line.extend_from_slice(&buffer[..count]);
reader.consume(count);
if finished {
break;
}
}
if line.is_empty() {
break;
}
bytes = bytes.saturating_add(line.len() as u64);
if bytes > MAX_IGNORE_FILE_BYTES {
return Err(FsError::InvalidArgument(
"ignore file exceeds provider limit 1 MiB".into(),
));
}
let line = std::str::from_utf8(&line)
.map_err(|_| FsError::InvalidArgument("ignore file must be UTF-8".into()))?
.trim_end_matches(['\r', '\n']);
// One invalid glob must not discard other valid rules.
if builder.add_line(Some(path.to_path_buf()), line).is_ok() {
added = true;
}
}
Ok(added)
}
struct DirectoryRules {
path: PathBuf,
ignore: Gitignore,
git_ignore: Gitignore,
exclude: Gitignore,
has_git: bool,
}
/// Request-local ancestor stack, not a file-tree cache. Only the current
/// directory's ancestry is retained; source bytes and depth bound its matchers.
pub(crate) struct DescriptorIgnores {
directories: Vec<DirectoryRules>,
remaining: u64,
require_git: bool,
skip_unavailable: bool,
}
impl DescriptorIgnores {
pub fn new(root: &Path, base: &Path, access: &dyn FsAccessPolicy) -> Result<Self, FsError> {
Self::with_options(root, base, access, true, true)
}
pub fn grep(root: &Path, base: &Path, access: &dyn FsAccessPolicy) -> Result<Self, FsError> {
// Preserve the existing descriptor Grep provider contract: gitignore
// sources apply without a .git marker and rejected opens fail closed.
Self::with_options(root, base, access, false, false)
}
fn with_options(
root: &Path,
base: &Path,
access: &dyn FsAccessPolicy,
require_git: bool,
skip_unavailable: bool,
) -> Result<Self, FsError> {
let relative = base.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("glob base is outside its provider root".into())
})?;
if relative.components().count() >= 128 {
return Err(FsError::InvalidArgument(
"ignore ancestry exceeds provider depth limit 128".into(),
));
}
let mut this = Self {
directories: Vec::new(),
remaining: MAX_IGNORE_SOURCE_BYTES,
require_git,
skip_unavailable,
};
let mut directory = root.to_path_buf();
for component in relative.components() {
this.load_directory(&directory, access)?;
directory.push(component);
}
Ok(this)
}
pub fn load_directory(
&mut self,
directory: &Path,
access: &dyn FsAccessPolicy,
) -> Result<(), FsError> {
self.directories
.retain(|rules| directory.starts_with(&rules.path));
if self
.directories
.last()
.is_some_and(|rules| rules.path == directory)
{
return Ok(());
}
if self.directories.len() >= 128 {
return Err(FsError::InvalidArgument(
"ignore ancestry exceeds provider depth limit 128".into(),
));
}
access
.check_cancelled()
.map_err(|e| FsError::io(directory, e))?;
let marker = directory.join(".git");
let git_kind = access
.resolve_access_path(&marker)
.ok()
.and_then(|resolved| {
if !access.is_readable_paths(&marker, &resolved) {
return None;
}
access
.read_metadata(&marker, &resolved)
.ok()
.map(|metadata| metadata.file_type())
});
let has_git = git_kind.is_some_and(|kind| kind.is_dir() || kind.is_file());
let mut load = |name: &str| -> Result<Gitignore, FsError> {
let mut builder = GitignoreBuilder::new(directory);
add_checked_ignore_file(
&mut builder,
&directory.join(name),
&mut self.remaining,
access,
self.skip_unavailable,
)?;
builder.build().map_err(|_| {
FsError::InvalidArgument("ignore patterns could not be compiled".into())
})
};
let ignore = load(".ignore")?;
let git_ignore = load(".gitignore")?;
// A linked-worktree .git file activates repository rules but is not a
// directory. Never chase its arbitrary host gitdir/commondir pointers.
let exclude = if git_kind.is_some_and(|kind| kind.is_dir()) {
load(".git/info/exclude")?
} else {
Gitignore::empty()
};
self.directories.push(DirectoryRules {
path: directory.to_path_buf(),
ignore,
git_ignore,
exclude,
has_git,
});
Ok(())
}
pub fn is_ignored(&mut self, path: &Path, is_dir: bool) -> bool {
self.directories
.retain(|rules| path.starts_with(&rules.path));
// .ignore has priority over all .gitignore/exclude rules. Within each
// class the nearest directory's last matching rule wins. Match entries,
// not any parents: ignored directories are pruned by the walker, while
// an explicitly selected ignored search root remains traversable.
for rules in self.directories.iter().rev() {
let matched = rules.ignore.matched(path, is_dir);
if !matched.is_none() {
return matched.is_ignore();
}
}
if self.require_git && !self.directories.iter().any(|rules| rules.has_git) {
return false;
}
let mut exclude = None;
for rules in self.directories.iter().rev() {
let matched = rules.git_ignore.matched(path, is_dir);
if !matched.is_none() {
return matched.is_ignore();
}
if exclude.is_none() {
let matched = rules.exclude.matched(path, is_dir);
if !matched.is_none() {
exclude = Some(matched.is_ignore());
}
}
if rules.has_git {
break;
}
}
exclude.unwrap_or(false)
}
}
+45 -39
View File
@@ -44,34 +44,29 @@ pub fn run_glob(
.as_ref()
.map(|traversal| traversal.path())
.unwrap_or(base);
let mut walker = WalkBuilder::new(walker_root);
walker.hidden(false).follow_links(false);
if traversal.is_some() {
// Descriptor providers must never let WalkBuilder open pathname-based
// ignore files, which may be symlinked outside the pinned root. Glob's
// explicit pattern remains the complete selection contract.
walker
.git_ignore(false)
.git_global(false)
.git_exclude(false)
.ignore(false)
.parents(false);
let search_relative = base.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("glob base is outside its provider root".to_string())
})?;
let search_relative = search_relative.to_path_buf();
let filter_root = walker_root.to_path_buf();
walker.filter_entry(move |entry| {
entry
.path()
.strip_prefix(&filter_root)
.is_ok_and(|relative| {
relative.starts_with(&search_relative) || search_relative.starts_with(relative)
})
});
}
let logical_walk_root = traversal
.as_ref()
.and_then(|t| t.logical_root())
.unwrap_or(root);
let mut descriptor_ignores = traversal
.as_ref()
.map(|_| crate::descriptor_ignore::DescriptorIgnores::new(root, logical_walk_root, access))
.transpose()?;
let mut visited = 0_usize;
for entry in walker.build().flatten() {
let mut walk = if traversal.is_some() {
// Configuration is read via authorized provider opens, never by
// WalkBuilder's host-path ignore discovery.
crate::walk::Walk::descriptor(logical_walk_root, access)
} else {
crate::walk::Walk::Plain(
WalkBuilder::new(walker_root)
.hidden(false)
.follow_links(false)
.build(),
)
};
while let Some(entry) = walk.next() {
let entry = entry?;
access.check_cancelled().map_err(|source| FsError::Io {
path: PathBuf::from(request.path.as_str()),
source,
@@ -83,21 +78,27 @@ pub fn run_glob(
crate::MAX_TRAVERSAL_ENTRIES
)));
}
let walked_path = entry.path();
let path = if traversal.is_some() {
let relative = walked_path.strip_prefix(walker_root).map_err(|_| {
FsError::InvalidArgument(
"descriptor traversal returned a path outside its root".to_string(),
)
})?;
root.join(relative)
} else {
walked_path.to_path_buf()
};
let path = entry.path;
if path.strip_prefix(root).unwrap_or(&path).to_str().is_none() {
continue;
}
let resolved = match access.resolve_access_path(&path) {
Ok(resolved) if access.is_readable_paths(&path, &resolved) => resolved,
_ => continue,
};
if let Some(ignores) = descriptor_ignores.as_mut() {
let is_dir = entry.kind.is_some_and(|kind| kind.is_dir());
if path != logical_walk_root && ignores.is_ignored(&path, is_dir) {
if is_dir {
walk.skip_current_dir();
}
continue;
}
if is_dir {
ignores.load_directory(&path, access)?;
continue;
}
}
let Ok(file) = access.open_read_file(&path, &resolved) else {
continue;
};
@@ -111,7 +112,12 @@ pub fn run_glob(
let logical = path.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("provider returned a path outside its root".to_string())
})?;
let result_path = FsPath::new(logical.to_string_lossy())?;
let Some(logical) = logical.to_str() else {
continue;
};
let Ok(result_path) = FsPath::new(logical) else {
continue;
};
let retained = result_path.as_str().len().saturating_add(16);
if retained_path_bytes.saturating_add(retained) > crate::MAX_RESULT_PATH_BYTES {
provider_truncated = true;
+326
View File
@@ -0,0 +1,326 @@
#![cfg(target_os = "linux")]
use crate::*;
use std::path::{Path, PathBuf};
use std::sync::{
Mutex,
atomic::{AtomicUsize, Ordering},
};
struct Access {
root: PathBuf,
denied: Option<PathBuf>,
opened: Mutex<Vec<PathBuf>>,
enumerated: Mutex<Vec<PathBuf>>,
checks: AtomicUsize,
cancel_at: usize,
}
impl Access {
fn new(root: &Path) -> Self {
Self {
root: root.into(),
denied: None,
opened: Mutex::new(Vec::new()),
enumerated: Mutex::new(Vec::new()),
checks: AtomicUsize::new(0),
cancel_at: usize::MAX,
}
}
}
impl FsAccessPolicy for Access {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.root) && self.denied.as_ref().is_none_or(|denied| path != denied)
}
fn is_writable(&self, _: &Path) -> bool {
false
}
fn resolve_access_path(&self, path: &Path) -> std::io::Result<PathBuf> {
Ok(path.into())
}
fn check_cancelled(&self) -> std::io::Result<()> {
if self.checks.fetch_add(1, Ordering::Relaxed) >= self.cancel_at {
Err(std::io::ErrorKind::Interrupted.into())
} else {
Ok(())
}
}
fn open_read_file(&self, logical: &Path, resolved: &Path) -> std::io::Result<std::fs::File> {
assert!(
self.is_readable(logical),
"denied ignore/source file opened"
);
self.opened.lock().unwrap().push(logical.into());
open_beneath_no_symlinks(&self.root, resolved)
}
fn read_metadata(&self, _: &Path, resolved: &Path) -> std::io::Result<std::fs::Metadata> {
open_beneath_no_symlinks(&self.root, resolved)?.metadata()
}
fn open_traversal_root(
&self,
_: &Path,
resolved: &Path,
) -> std::io::Result<Option<FsTraversalRoot>> {
use std::os::fd::AsRawFd;
let directory = open_beneath_no_symlinks(&self.root, resolved)?;
let walk = PathBuf::from(format!("/proc/self/fd/{}", directory.as_raw_fd()));
Ok(Some(FsTraversalRoot::new_at(
walk,
directory,
resolved.into(),
)))
}
fn open_read_dir(&self, logical: &Path, resolved: &Path) -> std::io::Result<std::fs::ReadDir> {
use std::os::fd::AsRawFd;
self.enumerated.lock().unwrap().push(logical.into());
let directory = open_beneath_no_symlinks(&self.root, resolved)?;
std::fs::read_dir(format!("/proc/self/fd/{}", directory.as_raw_fd()))
}
}
fn glob(root: &Path, access: &dyn FsAccessPolicy) -> Result<GlobResult, FsError> {
run_glob(
root,
root,
GlobRequest {
path: FsPath::root(),
pattern: "**/*.txt".into(),
limit: 100,
},
access,
)
}
#[test]
fn descriptor_glob_prunes_ignored_directories_before_enumeration_and_loading_rules() {
let root = tempfile::tempdir().unwrap();
std::fs::create_dir_all(root.path().join("ignored/deep")).unwrap();
std::fs::write(root.path().join(".ignore"), "ignored/\n").unwrap();
// This rejected directory must never even attempt to load ignore files.
std::fs::File::create(root.path().join("ignored/.ignore"))
.unwrap()
.set_len(2 * 1024 * 1024)
.unwrap();
std::fs::write(root.path().join("ignored/deep/secret.txt"), "secret").unwrap();
std::fs::write(root.path().join("visible.txt"), "visible").unwrap();
let access = Access::new(root.path());
assert_eq!(
glob(root.path(), &access).unwrap().paths,
vec![FsPath::new("visible.txt").unwrap()]
);
assert_eq!(
*access.enumerated.lock().unwrap(),
vec![root.path().to_path_buf()]
);
assert!(
!access
.opened
.lock()
.unwrap()
.iter()
.any(|path| path.starts_with(root.path().join("ignored")))
);
}
#[test]
fn descriptor_glob_denied_and_symlinked_ignore_files_cannot_change_selection() {
let root = tempfile::tempdir().unwrap();
let outside = tempfile::tempdir().unwrap();
std::fs::write(outside.path().join("rules"), "visible.txt\n").unwrap();
std::fs::create_dir(root.path().join(".git")).unwrap();
std::os::unix::fs::symlink(outside.path().join("rules"), root.path().join(".gitignore"))
.unwrap();
std::fs::write(root.path().join(".ignore"), "visible.txt\n").unwrap();
std::fs::write(root.path().join("visible.txt"), "visible").unwrap();
let mut access = Access::new(root.path());
access.denied = Some(root.path().join(".ignore"));
assert_eq!(
glob(root.path(), &access).unwrap().paths,
vec![FsPath::new("visible.txt").unwrap()]
);
assert!(
!access
.opened
.lock()
.unwrap()
.contains(&root.path().join(".ignore"))
);
}
#[test]
fn descriptor_ignore_loading_checks_cancellation_without_retrying_interrupted() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join(".ignore"), "ignored.txt\n".repeat(1000)).unwrap();
let mut access = Access::new(root.path());
// First check is before resolution, second at the first fill_buf.
access.cancel_at = 1;
let error = crate::descriptor_ignore::add_checked_ignore_file(
&mut ignore::gitignore::GitignoreBuilder::new(root.path()),
&root.path().join(".ignore"),
&mut 1024_000,
&access,
true,
)
.unwrap_err();
assert!(
matches!(error, FsError::Io { source, .. } if source.kind() == std::io::ErrorKind::Interrupted)
);
assert_eq!(access.checks.load(Ordering::Relaxed), 2);
}
#[test]
fn descriptor_ignore_loading_bounds_file_line_and_aggregate_bytes() {
let root = tempfile::tempdir().unwrap();
let path = root.path().join(".ignore");
let access = Access::new(root.path());
std::fs::File::create(&path)
.unwrap()
.set_len(1024 * 1024 + 1)
.unwrap();
assert!(matches!(
glob(root.path(), &access),
Err(FsError::InvalidArgument(_))
));
std::fs::write(&path, "x".repeat(16 * 1024 + 1)).unwrap();
assert!(matches!(
glob(root.path(), &access),
Err(FsError::InvalidArgument(_))
));
std::fs::write(&path, "ignored.txt\n").unwrap();
let error = crate::descriptor_ignore::add_checked_ignore_file(
&mut ignore::gitignore::GitignoreBuilder::new(root.path()),
&path,
&mut 3,
&access,
true,
)
.unwrap_err();
assert!(
matches!(error, FsError::Io { source, .. } if source.kind() == std::io::ErrorKind::InvalidData)
);
}
#[test]
fn descriptor_ignore_ancestry_is_bounded_without_enumerating_a_tree() {
let root = tempfile::tempdir().unwrap();
let access = Access::new(root.path());
let base = (0..128).fold(root.path().to_path_buf(), |path, _| path.join("d"));
assert!(matches!(
crate::descriptor_ignore::DescriptorIgnores::new(root.path(), &base, &access),
Err(FsError::InvalidArgument(_))
));
assert!(access.opened.lock().unwrap().is_empty());
}
struct Plain(PathBuf);
impl FsAccessPolicy for Plain {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.0)
}
fn is_writable(&self, _: &Path) -> bool {
false
}
}
#[test]
fn descriptor_glob_matches_plain_git_markers_exclude_and_nested_repository_boundaries() {
let root = tempfile::tempdir().unwrap();
std::fs::create_dir_all(root.path().join("nested/.git/info")).unwrap();
std::fs::create_dir_all(root.path().join(".git/info")).unwrap();
std::fs::write(
root.path().join(".gitignore"),
"*.outer.txt\n!include.txt\n",
)
.unwrap();
std::fs::write(root.path().join(".ignore"), "always.txt\n").unwrap();
std::fs::write(
root.path().join(".git/info/exclude"),
"excluded.txt\ninclude.txt\n",
)
.unwrap();
std::fs::write(
root.path().join("nested/.git/info/exclude"),
"/excluded.txt\n",
)
.unwrap();
for name in [
"a.outer.txt",
"excluded.txt",
"include.txt",
"always.txt",
"nested/a.outer.txt",
"nested/excluded.txt",
"nested/always.txt",
] {
std::fs::write(root.path().join(name), "data").unwrap();
}
let native = glob(root.path(), &Access::new(root.path())).unwrap();
let ordinary = glob(root.path(), &Plain(root.path().into())).unwrap();
assert_eq!(native, ordinary);
assert_eq!(
native.paths,
vec![
FsPath::new("include.txt").unwrap(),
FsPath::new("nested/a.outer.txt").unwrap()
]
);
}
#[test]
fn descriptor_glob_requires_authorized_git_marker_for_gitignore_not_for_ignore() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join(".gitignore"), "gitignored.txt\n").unwrap();
std::fs::write(root.path().join(".ignore"), "ignored.txt\n").unwrap();
for name in ["gitignored.txt", "ignored.txt", "visible.txt"] {
std::fs::write(root.path().join(name), "data").unwrap();
}
let native = glob(root.path(), &Access::new(root.path())).unwrap();
// Ambient ancestors may themselves be repositories (e.g. the harness's
// TMPDIR). Their markers/ignore files are outside this provider root and
// cannot activate Git rules or be inspected by descriptor search.
assert_eq!(native.paths.len(), 2);
std::fs::create_dir(root.path().join(".git")).unwrap();
let native = glob(root.path(), &Access::new(root.path())).unwrap();
assert_eq!(
native,
glob(root.path(), &Plain(root.path().into())).unwrap()
);
assert_eq!(native.paths, vec![FsPath::new("visible.txt").unwrap()]);
let mut denied = Access::new(root.path());
denied.denied = Some(root.path().join(".git"));
assert_eq!(glob(root.path(), &denied).unwrap().paths.len(), 2);
std::fs::remove_dir(root.path().join(".git")).unwrap();
let outside = tempfile::tempdir().unwrap();
std::fs::write(outside.path().join("exclude"), "visible.txt\n").unwrap();
std::fs::write(
root.path().join(".git"),
format!("gitdir: {}\n", outside.path().display()),
)
.unwrap();
// Real managed Workdirs use a .git file. Activate root .gitignore without
// failing on .git/info/exclude or following external administration paths.
let access = Access::new(root.path());
assert_eq!(
glob(root.path(), &access).unwrap().paths,
vec![FsPath::new("visible.txt").unwrap()]
);
std::fs::write(root.path().join("visible.txt"), "needle\n").unwrap();
let grep = run_grep(
root.path(),
root.path().into(),
GrepRequest {
path: FsPath::root(),
pattern: "needle".into(),
glob: None,
file_type: None,
case_insensitive: false,
before_context: 0,
after_context: 0,
multiline: false,
output_mode: GrepOutputMode::FilesWithMatches,
offset: 0,
limit: 100,
},
&access,
)
.unwrap();
assert_eq!(grep.paths, vec![FsPath::new("visible.txt").unwrap()]);
assert!(
!access
.opened
.lock()
.unwrap()
.iter()
.any(|path| path.starts_with(outside.path()))
);
}
+36
View File
@@ -4,10 +4,19 @@
//! transport, or LLM Tool implementations. Paths are logical and root-relative;
//! providers supply host roots and access policy.
mod checked;
pub use checked::{CheckedTarget, identity_validator};
mod descriptor_ignore;
mod glob;
#[cfg(test)]
mod ignore_tests;
mod local;
mod operation;
mod search;
pub mod text;
#[cfg(test)]
mod traversal_tests;
mod walk;
use std::path::{Path, PathBuf};
@@ -272,6 +281,7 @@ pub fn atomic_write_beneath_no_symlinks_at(
#[derive(Debug)]
pub struct FsTraversalRoot {
path: PathBuf,
logical_root: Option<PathBuf>,
_directory: std::fs::File,
}
@@ -279,20 +289,44 @@ impl FsTraversalRoot {
pub fn new(path: PathBuf, directory: std::fs::File) -> Self {
Self {
path,
logical_root: None,
_directory: directory,
}
}
/// Map walked paths to this logical anchor rather than the provider root.
pub fn new_at(path: PathBuf, directory: std::fs::File, logical_root: PathBuf) -> Self {
Self {
path,
logical_root: Some(logical_root),
_directory: directory,
}
}
pub fn logical_root(&self) -> Option<&Path> {
self.logical_root.as_deref()
}
pub fn path(&self) -> &Path {
&self.path
}
}
/// Owned predicate used before walker descent. Paths are provider-logical;
/// true directories require enumeration authority, other entries require read.
pub type FsTraversalFilter = std::sync::Arc<dyn Fn(&Path, bool) -> bool + Send + Sync>;
/// Provider-owned access policy used by local filesystem operations.
pub trait FsAccessPolicy: Send + Sync {
fn is_readable(&self, path: &Path) -> bool;
fn is_writable(&self, path: &Path) -> bool;
fn can_enumerate_directory(&self, logical: &Path, resolved: &Path) -> bool {
self.is_readable_paths(logical, resolved)
}
/// Owned because the shared walker requires a 'static entry predicate.
fn traversal_filter(&self) -> Option<FsTraversalFilter> {
None
}
/// Cooperatively stop provider-owned filesystem work. Implementations may
/// use this for per-operation cancellation and deadlines.
fn check_cancelled(&self) -> std::io::Result<()> {
@@ -907,6 +941,7 @@ mod tests {
let list_error = run_list(
&root,
ListRequest {
after: None,
path: FsPath::new("selected-dir").unwrap(),
limit: 10,
},
@@ -974,6 +1009,7 @@ mod tests {
let listed = run_list(
&root,
ListRequest {
after: None,
path: FsPath::new("notes").unwrap(),
limit: 10,
},
+398 -87
View File
@@ -1,3 +1,5 @@
use std::cmp::Ordering;
use std::collections::BinaryHeap;
use std::ffi::OsString;
use std::fs;
use std::io::{BufReader, Read};
@@ -7,9 +9,9 @@ use sha2::{Digest, Sha256};
use crate::{
AtomicWriteMode, BoundedReadLimits, ContentHash, EditRequest, EditResult, EntryKind,
FsAccessPolicy, FsError, FsPath, ListEntry, ListRequest, ListResult, ReadBytesRequest,
ReadBytesResult, ReadRequest, ReadResult, StatRequest, StatResult, WriteRequest, WriteResult,
direct_symlink,
FsAccessPolicy, FsError, FsPath, ListCursor, ListEntry, ListRequest, ListResult,
ReadBytesRequest, ReadBytesResult, ReadRequest, ReadResult, StatRequest, StatResult,
WriteRequest, WriteResult, direct_symlink,
};
const READ_BUFFER_BYTES: usize = 16 * 1024;
@@ -162,6 +164,19 @@ fn run_read_with_limits(
}
}
#[cfg(unix)]
if crate::identity_validator(&metadata).map_err(|e| map_io(&logical, e))?
!= crate::identity_validator(
&reader
.get_ref()
.metadata()
.map_err(|e| map_io(&logical, e))?,
)
.map_err(|e| map_io(&logical, e))?
{
return Err(FsError::Conflict(logical.as_str().to_string()));
}
let total_lines =
current_line.saturating_add(usize::from(last_byte.is_some_and(|byte| byte != b'\n')));
if request.offset > total_lines && request.offset != 0 {
@@ -176,15 +191,7 @@ fn run_read_with_limits(
.min(total_lines);
let byte_truncated = selected_bytes_seen > response_limit;
if byte_truncated {
let mut byte_end = response_limit.min(selected.len());
while byte_end > 0 {
match std::str::from_utf8(&selected[..byte_end]) {
Ok(_) => break,
Err(error) if error.error_len().is_none() => byte_end -= 1,
Err(_) => break,
}
}
selected.truncate(byte_end);
selected.truncate(crate::text::bounded_utf8_len(&selected, response_limit));
}
Ok(ReadResult {
path: logical,
@@ -402,64 +409,85 @@ pub fn run_edit(
let content = String::from_utf8(bytes).map_err(|_| {
FsError::InvalidArgument(format!("{} is not valid UTF-8", logical.as_str()))
})?;
let occurrences = content.matches(&request.old_string).count();
if occurrences == 0 {
return Err(FsError::InvalidArgument(
"old_string was not found".to_string(),
));
}
if !request.replace_all && occurrences != 1 {
return Err(FsError::InvalidArgument(format!(
"old_string matched {occurrences} times; set replace_all=true or provide a unique string"
)));
}
let replacement_count = if request.replace_all { occurrences } else { 1 };
if let Some(limit) = access.max_edit_replacements()
&& replacement_count > limit
{
return Err(FsError::InvalidArgument(format!(
"{} exceeds provider Edit replacement limit {limit}",
logical.as_str()
)));
}
let removed_bytes = request
.old_string
.len()
.checked_mul(replacement_count)
.ok_or_else(|| FsError::InvalidArgument("edited content size overflowed".to_string()))?;
let added_bytes = request
.new_string
.len()
.checked_mul(replacement_count)
.ok_or_else(|| FsError::InvalidArgument("edited content size overflowed".to_string()))?;
let edited_len = content
.len()
.checked_sub(removed_bytes)
.and_then(|size| size.checked_add(added_bytes))
.ok_or_else(|| FsError::InvalidArgument("edited content size overflowed".to_string()))?;
enforce_write_bound(&logical, edited_len, access)?;
let edited = if request.replace_all {
content.replace(&request.old_string, &request.new_string)
} else {
content.replacen(&request.old_string, &request.new_string, 1)
let args = crate::text::EditArgs {
old_string: request.old_string,
new_string: request.new_string,
replace_all: request.replace_all,
};
debug_assert_eq!(edited.len(), edited_len);
let edited = crate::text::edit(
&content,
&args,
crate::text::TextLimits {
max_input_bytes: access.max_write_bytes(),
max_output_bytes: access.max_write_bytes(),
max_replacements: access.max_edit_replacements(),
},
)
.map_err(|error| map_text_error(&logical, error))?;
access
.atomic_write_file(&path, &target, edited.as_bytes(), AtomicWriteMode::Replace)
.atomic_write_file(
&path,
&target,
edited.content.as_bytes(),
AtomicWriteMode::Replace,
)
.map_err(|error| map_io(&logical, error))?;
Ok(EditResult {
replacements: if request.replace_all { occurrences } else { 1 },
bytes_written: edited.len(),
content_hash: hash_bytes(edited.as_bytes()),
replacements: edited.replacements,
bytes_written: edited.bytes_written,
content_hash: hash_bytes(edited.content.as_bytes()),
})
}
fn list_order(
left_kind: EntryKind,
left: &FsPath,
right_kind: EntryKind,
right: &FsPath,
) -> Ordering {
(left_kind != EntryKind::Directory)
.cmp(&(right_kind != EntryKind::Directory))
.then_with(|| left.as_str().cmp(right.as_str()))
}
struct RetainedListEntry(ListEntry);
impl PartialEq for RetainedListEntry {
fn eq(&self, other: &Self) -> bool {
self.cmp(other) == Ordering::Equal
}
}
impl Eq for RetainedListEntry {}
impl Ord for RetainedListEntry {
fn cmp(&self, other: &Self) -> Ordering {
list_order(self.0.kind, &self.0.path, other.0.kind, &other.0.path)
}
}
impl PartialOrd for RetainedListEntry {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
fn list_entry_bytes(entry: &ListEntry) -> usize {
entry.path.as_str().len().saturating_add(64)
}
pub fn run_list(
root: &Path,
request: ListRequest,
access: &dyn FsAccessPolicy,
) -> Result<ListResult, FsError> {
let logical = request.path;
if let Some(after) = &request.after {
if after.path.is_root()
|| Path::new(after.path.as_str()).parent() != Some(Path::new(logical.as_str()))
{
return Err(FsError::InvalidArgument(
"List cursor must name a direct child of the listed directory".into(),
));
}
}
access
.check_cancelled()
.map_err(|error| map_io(&logical, error))?;
@@ -473,9 +501,21 @@ pub fn run_list(
if !metadata.is_dir() {
return Err(FsError::NotDirectory(PathBuf::from(logical.as_str())));
}
let mut entries = Vec::new();
if !access.can_enumerate_directory(&logical_base, &path) {
return Ok(ListResult {
entries: Vec::new(),
total_entries: 0,
total_bytes: 0,
truncated: false,
next_after: None,
});
}
let mut entries: BinaryHeap<RetainedListEntry> = BinaryHeap::new();
let mut retained_limit = request.limit.saturating_add(1);
let mut retained_path_bytes = 0_usize;
let mut provider_truncated = false;
let mut total_entries = 0usize;
let mut total_bytes = 0u64;
let mut remaining_entries = 0usize;
let read_dir = access
.open_read_dir(&logical_base, &path)
.map_err(|error| map_io(&logical, error))?;
@@ -490,6 +530,9 @@ pub fn run_list(
)));
}
let entry = entry.map_err(|error| map_io(&logical, error))?;
if entry.file_name().to_str().is_none() {
continue;
}
let logical_absolute = logical_base.join(entry.file_name());
let resolved = match access.resolve_access_path(&logical_absolute) {
Ok(resolved) => resolved,
@@ -513,35 +556,83 @@ pub fn run_list(
let relative = logical_absolute.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("provider returned a path outside its root".to_string())
})?;
let result_path = FsPath::new(relative.to_string_lossy())?;
let retained = result_path.as_str().len().saturating_add(64);
if retained_path_bytes.saturating_add(retained) > crate::MAX_RESULT_PATH_BYTES {
provider_truncated = true;
break;
let Some(relative) = relative.to_str() else {
continue;
};
let Ok(result_path) = FsPath::new(relative) else {
continue;
};
total_entries += 1;
total_bytes = total_bytes.saturating_add(metadata.len());
if request.after.as_ref().is_some_and(|after| {
list_order(kind, &result_path, after.kind, &after.path) != Ordering::Greater
}) {
continue;
}
retained_path_bytes = retained_path_bytes.saturating_add(retained);
entries.push(ListEntry {
remaining_entries += 1;
let candidate = RetainedListEntry(ListEntry {
path: result_path,
kind,
size: metadata.len(),
});
if entries.len() == retained_limit {
if candidate >= *entries.peek().expect("nonempty List retention") {
continue;
}
retained_path_bytes -= list_entry_bytes(&entries.pop().unwrap().0);
}
retained_path_bytes = retained_path_bytes.saturating_add(list_entry_bytes(&candidate.0));
entries.push(candidate);
// Evict greatest keys, never stop scanning: enumeration order is
// unspecified and a later entry may precede every retained key.
while retained_path_bytes > crate::MAX_RESULT_PATH_BYTES {
retained_path_bytes -= list_entry_bytes(&entries.pop().unwrap().0);
// Never grow cardinality again after a byte-bound eviction: a
// later short path must not leap over a discarded earlier key.
retained_limit = entries.len();
}
if retained_limit == 0 {
return Err(FsError::InvalidArgument(
"List entry exceeds provider response bound".into(),
));
}
}
entries.sort_by(|left, right| {
let left_dir = left.kind == EntryKind::Directory;
let right_dir = right.kind == EntryKind::Directory;
right_dir
.cmp(&left_dir)
.then_with(|| left.path.as_str().cmp(right.path.as_str()))
});
let total_entries = entries.len();
let total_bytes = entries.iter().map(|entry| entry.size).sum();
let truncated = provider_truncated || entries.len() > request.limit;
let mut entries: Vec<_> = entries
.into_sorted_vec()
.into_iter()
.map(|entry| entry.0)
.collect();
entries.truncate(request.limit);
// Continuation duplicates the last path on the wire; include it in the
// aggregate provider response bound too.
let mut response_bytes: usize = entries.iter().map(list_entry_bytes).sum();
while remaining_entries > entries.len()
&& entries.last().is_some_and(|last| {
response_bytes.saturating_add(list_entry_bytes(last)) > crate::MAX_RESULT_PATH_BYTES
})
{
response_bytes -= list_entry_bytes(&entries.pop().unwrap());
}
if request.limit > 0 && remaining_entries > 0 && entries.is_empty() {
return Err(FsError::InvalidArgument(
"List entry exceeds provider response bound".into(),
));
}
let truncated = remaining_entries > entries.len();
let next_after = if truncated {
entries.last().map(|entry| ListCursor {
kind: entry.kind,
path: entry.path.clone(),
})
} else {
None
};
Ok(ListResult {
entries,
total_entries,
total_bytes,
truncated,
next_after,
})
}
@@ -550,15 +641,37 @@ fn enforce_write_bound(
bytes: usize,
access: &dyn FsAccessPolicy,
) -> Result<(), FsError> {
if let Some(limit) = access.max_write_bytes()
&& bytes > limit
{
return Err(FsError::InvalidArgument(format!(
"{} exceeds provider write limit {limit}",
crate::text::check_output_bytes(
bytes,
crate::text::TextLimits {
max_output_bytes: access.max_write_bytes(),
..Default::default()
},
)
.map_err(|error| map_text_error(logical, error))
}
fn map_text_error(logical: &FsPath, error: crate::text::TextError) -> FsError {
use crate::text::TextError;
let message = match error {
TextError::InputTooLarge { limit, .. } | TextError::OutputTooLarge { limit, .. } => {
format!("{} exceeds provider write limit {limit}", logical.as_str())
}
TextError::ReplacementLimitExceeded { limit, .. } => format!(
"{} exceeds provider Edit replacement limit {limit}",
logical.as_str()
)));
}
Ok(())
),
TextError::Decode(error) => format!("invalid arguments: {error}"),
TextError::EmptyOldString => "old_string must not be empty".to_string(),
TextError::IdenticalStrings => "old_string and new_string are identical".to_string(),
TextError::NotFound => "old_string was not found".to_string(),
TextError::MultipleMatches { occurrences } => format!(
"old_string matched {occurrences} times; set replace_all=true or provide a unique string"
),
TextError::Overflow => "edited content size overflowed".to_string(),
};
FsError::InvalidArgument(message)
}
fn read_mutation_preimage(
@@ -610,9 +723,21 @@ fn require_access(
write: bool,
allow_symlink_directory: bool,
) -> Result<PathBuf, FsError> {
let resolved = access
.resolve_access_path(path)
.map_err(|error| map_io(logical, error))?;
let resolved = access.resolve_access_path(path).map_err(|error| {
// A dangling direct link has no resolved identity, but still needs the
// existing sanitized repair diagnostic rather than an ordinary missing-file error.
if error.kind() == std::io::ErrorKind::NotFound
&& direct_symlink(path).is_some_and(|info| !info.target_exists)
{
FsError::BrokenSymlink {
path: PathBuf::from(logical.as_str()),
link: PathBuf::from(logical.as_str()),
target: PathBuf::from("<provider-internal target>"),
}
} else {
map_io(logical, error)
}
})?;
let symlink = (resolved != path).then(|| direct_symlink(path)).flatten();
if let Some(info) = symlink.as_ref()
&& !info.target_exists
@@ -773,3 +898,189 @@ mod bounded_read_tests {
assert!(result.truncated);
}
}
#[cfg(test)]
mod text_policy_tests {
use super::*;
struct Access {
root: PathBuf,
bytes: Option<usize>,
replacements: Option<usize>,
}
impl FsAccessPolicy for Access {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.root)
}
fn is_writable(&self, path: &Path) -> bool {
path.starts_with(&self.root)
}
fn max_write_bytes(&self) -> Option<usize> {
self.bytes
}
fn max_edit_replacements(&self) -> Option<usize> {
self.replacements
}
}
#[test]
fn edit_policy_errors_preserve_the_preimage_and_provider_messages() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
let source = "a a";
fs::write(root.join("text.txt"), source).unwrap();
let access = Access {
root: root.clone(),
bytes: Some(4),
replacements: Some(1),
};
for (old, new, all, expected_message) in [
("", "x", false, "old_string must not be empty"),
("a", "a", false, "old_string and new_string are identical"),
(
"missing",
"x",
false,
"text.txt exceeds provider write limit 4",
),
("b", "x", false, "old_string was not found"),
(
"a",
"x",
false,
"old_string matched 2 times; set replace_all=true or provide a unique string",
),
(
"a",
"x",
true,
"text.txt exceeds provider Edit replacement limit 1",
),
(
"a a",
"😀x",
false,
"text.txt exceeds provider write limit 4",
),
] {
let error = run_edit(
&root,
EditRequest {
path: FsPath::new("text.txt").unwrap(),
old_string: old.into(),
new_string: new.into(),
replace_all: all,
expected_hash: hash_bytes(source.as_bytes()),
},
&access,
)
.unwrap_err();
assert!(
matches!(error, FsError::InvalidArgument(message) if message == expected_message)
);
assert_eq!(fs::read_to_string(root.join("text.txt")).unwrap(), source);
}
let access = Access {
replacements: None,
..access
};
let error = run_edit(
&root,
EditRequest {
path: FsPath::new("text.txt").unwrap(),
old_string: "a".into(),
new_string: "xxx".into(),
replace_all: true,
expected_hash: hash_bytes(source.as_bytes()),
},
&access,
)
.unwrap_err();
assert!(
matches!(error, FsError::InvalidArgument(message) if message == "text.txt exceeds provider write limit 4")
);
assert_eq!(fs::read_to_string(root.join("text.txt")).unwrap(), source);
}
#[test]
fn edit_keeps_cas_before_content_policy_and_hashes_the_core_result() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
fs::write(root.join("text.txt"), "é😀").unwrap();
let access = Access {
root: root.clone(),
bytes: None,
replacements: None,
};
let error = run_edit(
&root,
EditRequest {
path: FsPath::new("text.txt").unwrap(),
old_string: String::new(),
new_string: "x".into(),
replace_all: false,
expected_hash: [0; 32],
},
&access,
)
.unwrap_err();
assert!(matches!(error, FsError::Conflict(_)));
let result = run_edit(
&root,
EditRequest {
path: FsPath::new("text.txt").unwrap(),
old_string: "é".into(),
new_string: "界".into(),
replace_all: false,
expected_hash: hash_bytes("é😀".as_bytes()),
},
&access,
)
.unwrap();
assert_eq!(result.replacements, 1);
assert_eq!(result.bytes_written, 7);
assert_eq!(result.content_hash, hash_bytes("界😀".as_bytes()));
assert_eq!(fs::read_to_string(root.join("text.txt")).unwrap(), "界😀");
}
#[test]
fn local_write_remains_binary_safe_and_maps_shared_bounds() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().canonicalize().unwrap();
let access = Access {
root: root.clone(),
bytes: Some(2),
replacements: None,
};
let result = run_write(
&root,
WriteRequest {
path: FsPath::new("binary").unwrap(),
content: vec![0xff, 0],
expected_hash: None,
},
&access,
)
.unwrap();
assert_eq!(result.bytes_written, 2);
assert_eq!(fs::read(root.join("binary")).unwrap(), vec![0xff, 0]);
let error = run_write(
&root,
WriteRequest {
path: FsPath::new("large").unwrap(),
content: vec![0xff, 0, 1],
expected_hash: None,
},
&access,
)
.unwrap_err();
assert!(
matches!(error, FsError::InvalidArgument(message) if message == "large exceeds provider write limit 2")
);
assert!(!root.join("large").exists());
}
}
+25
View File
@@ -278,6 +278,22 @@ pub struct ListRequest {
pub path: FsPath,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub limit: usize,
/// Exclusive live continuation in the same coordinates as returned entries.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub after: Option<ListCursor>,
}
/// A List ordering key: directories first, then lexical path (all other kinds
/// share one ordering group). The cursor need not still exist. Each page scans
/// the live directory: insertions before the key are not revisited, insertions
/// after it may appear, and removals disappear. Renames or kind changes can move
/// an entry across the key and cause omission or repetition; no snapshot or
/// cross-page consistency is promised. Scoped wrappers translate coordinates.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct ListCursor {
pub kind: EntryKind,
pub path: FsPath,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
@@ -293,6 +309,11 @@ pub struct ListEntry {
#[serde(deny_unknown_fields)]
pub struct ListResult {
pub entries: Vec<ListEntry>,
/// Last returned key when more entries follow. A zero-limit request has no
/// continuation even when truncated. Totals describe all visible entries
/// in this scan, including those at or before `after`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub next_after: Option<ListCursor>,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_usize))]
pub total_entries: usize,
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
@@ -353,6 +374,10 @@ pub struct GrepRequest {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct GrepResult {
/// Exact typed provider paths represented in this returned report, not all
/// candidate/matched source files. Older providers may omit this field.
#[serde(default)]
pub paths: Vec<FsPath>,
/// Provider-rendered bounded grep report. Keeping rendering here avoids
/// transferring candidate files across a remote provider boundary.
pub output: String,
+87 -215
View File
@@ -1,50 +1,20 @@
use std::collections::{BTreeMap, HashSet};
use std::collections::BTreeMap;
use std::fmt::Write as _;
use std::io::{BufRead, BufReader, Read};
use std::io::Read;
use std::path::{Path, PathBuf};
use crate::descriptor_ignore::{DescriptorIgnores, SourceBoundedReader};
use crate::FsAccessPolicy;
use grep_regex::RegexMatcherBuilder;
use grep_searcher::sinks::UTF8 as UTF8Sink;
use grep_searcher::{BinaryDetection, Searcher, SearcherBuilder, Sink, SinkContext, SinkMatch};
use ignore::WalkBuilder;
use ignore::gitignore::{Gitignore, GitignoreBuilder};
use ignore::overrides::{Override, OverrideBuilder};
use ignore::types::{Types, TypesBuilder};
use crate::{FsError, GrepOutputMode, GrepRequest, GrepResult, direct_symlink};
struct SourceBoundedReader<'a, R> {
inner: R,
remaining: &'a mut u64,
access: &'a dyn FsAccessPolicy,
}
impl<R: Read> Read for SourceBoundedReader<'_, R> {
fn read(&mut self, buffer: &mut [u8]) -> std::io::Result<usize> {
self.access.check_cancelled()?;
if buffer.is_empty() {
return Ok(0);
}
if *self.remaining == 0 {
let mut probe = [0_u8; 1];
return match self.inner.read(&mut probe)? {
0 => Ok(0),
_ => Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"grep source exceeded provider byte limit",
)),
};
}
let limit = usize::try_from(*self.remaining)
.unwrap_or(usize::MAX)
.min(buffer.len());
let read = self.inner.read(&mut buffer[..limit])?;
*self.remaining = (*self.remaining).saturating_sub(read as u64);
Ok(read)
}
}
struct ContentLine {
path: PathBuf,
line_number: Option<u64>,
@@ -79,6 +49,28 @@ fn reserve_report_bytes(report: &mut GrepReport, bytes: usize) -> bool {
impl GrepReport {
fn into_result(self, root: &Path) -> GrepResult {
// Build Object paths from retained provider identities, never by parsing
// rendered text. Offset, limit and truncation already selected this set.
let paths = match self.mode {
GrepOutputMode::FilesWithMatches => self
.files
.iter()
.filter_map(|path| logical_path(root, path))
.collect(),
GrepOutputMode::Count => self
.counts
.iter()
.filter_map(|(path, _)| logical_path(root, path))
.collect(),
GrepOutputMode::Content => self
.lines
.iter()
.map(|line| line.path.as_path())
.collect::<std::collections::BTreeSet<_>>()
.into_iter()
.filter_map(|path| logical_path(root, path))
.collect(),
};
let (match_count, matched_files) = match self.mode {
GrepOutputMode::FilesWithMatches => (self.files.len(), self.files.len()),
GrepOutputMode::Count => (
@@ -116,6 +108,7 @@ impl GrepReport {
}
}
GrepResult {
paths,
output,
match_count,
matched_files,
@@ -166,11 +159,15 @@ fn render_content_lines(root: &Path, lines: &[ContentLine], show_line_numbers: b
output
}
fn logical_path(root: &Path, path: &Path) -> Option<crate::FsPath> {
crate::FsPath::new_scoped(path.strip_prefix(root).unwrap_or(path).to_str()?).ok()
}
fn logical_display(root: &Path, path: &Path) -> String {
path.strip_prefix(root)
.unwrap_or(path)
.to_string_lossy()
.replace('\\', "/")
logical_path(root, path)
.expect("only representable provider paths enter a report")
.as_str()
.to_string()
}
const DEFAULT_HEAD_LIMIT: usize = 250;
@@ -207,118 +204,6 @@ fn direct_file_selected(path: &Path, overrides: Option<&Override>, types: Option
&& !types.is_some_and(|filter| filter.matched(path, false).is_ignore())
}
#[derive(Default)]
struct DescriptorIgnoreMatchers {
loaded_directories: HashSet<PathBuf>,
by_directory: BTreeMap<PathBuf, Gitignore>,
}
impl DescriptorIgnoreMatchers {
fn load_directory(
&mut self,
directory: &Path,
explicit_base: &Path,
source_bytes_remaining: &mut u64,
access: &dyn FsAccessPolicy,
) -> Result<(), FsError> {
if !self.loaded_directories.insert(directory.to_path_buf()) {
return Ok(());
}
let mut builder = GitignoreBuilder::new(directory);
let mut has_patterns = false;
for name in [".gitignore", ".ignore"] {
let path = directory.join(name);
let resolved = access
.resolve_access_path(&path)
.map_err(|error| FsError::io(&path, error))?;
if !access.is_readable_paths(&path, &resolved) {
continue;
}
let file = match access.open_read_file(&path, &resolved) {
Ok(file) => file,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue,
Err(error) => return Err(FsError::io(&path, error)),
};
if !file
.metadata()
.map_err(|error| FsError::io(&path, error))?
.is_file()
{
continue;
}
let bounded = SourceBoundedReader {
inner: file,
remaining: source_bytes_remaining,
access,
};
let mut reader = BufReader::new(bounded);
let mut line = String::new();
let mut line_number = 0_u64;
loop {
line.clear();
let read = reader
.read_line(&mut line)
.map_err(|error| FsError::io(&path, error))?;
if read == 0 {
break;
}
line_number = line_number.saturating_add(1);
if line.len() > 1024 * 1024 {
return Err(FsError::InvalidArgument(format!(
"ignore pattern line {line_number} exceeds provider limit"
)));
}
let line = line.trim_end_matches(['\r', '\n']);
// Match the ignore crate's partial-error behavior: one invalid
// pattern does not discard the remaining valid lines.
if builder.add_line(Some(path.clone()), line).is_ok() {
has_patterns = true;
}
}
}
if !has_patterns {
return Ok(());
}
let matcher = builder.build().map_err(|_| {
FsError::InvalidArgument("ignore patterns could not be compiled".to_string())
})?;
// An explicitly selected search root is traversed even when an ancestor
// ignores that directory, matching path-backed Grep behavior.
if directory != explicit_base
&& matcher
.matched_path_or_any_parents(explicit_base, true)
.is_ignore()
{
return Ok(());
}
self.by_directory.insert(directory.to_path_buf(), matcher);
Ok(())
}
fn is_ignored(&self, root: &Path, path: &Path) -> bool {
let mut directories = path
.parent()
.into_iter()
.flat_map(Path::ancestors)
.take_while(|directory| directory.starts_with(root))
.collect::<Vec<_>>();
directories.reverse();
let mut ignored = false;
for directory in directories {
let Some(matcher) = self.by_directory.get(directory) else {
continue;
};
let matched = matcher.matched_path_or_any_parents(path, false);
if matched.is_ignore() {
ignored = true;
} else if matched.is_whitelist() {
ignored = false;
}
}
ignored
}
}
struct GrepParams {
pattern: String,
path: Option<PathBuf>,
@@ -463,6 +348,9 @@ pub fn run_grep(
retained_bytes: 0,
truncated: false,
};
if head_limit == 0 {
return Ok(report.into_result(root));
}
let mut matching_files_seen = 0;
let mut matches_seen = 0;
@@ -473,6 +361,7 @@ pub fn run_grep(
&mut searcher,
&matcher,
&base,
root,
mode,
&mut report,
&mut matching_files_seen,
@@ -493,6 +382,10 @@ pub fn run_grep(
.as_ref()
.map(|traversal| traversal.path())
.unwrap_or(&base);
let logical_walk_root = traversal
.as_ref()
.and_then(|t| t.logical_root())
.unwrap_or(root);
let mut walker = WalkBuilder::new(walker_root);
if traversal.is_some() {
walker
@@ -505,19 +398,6 @@ pub fn run_grep(
.ignore(false)
.parents(false)
.follow_links(false);
let search_relative = base.strip_prefix(root).map_err(|_| {
FsError::InvalidArgument("grep base is outside its provider root".to_string())
})?;
let search_relative = search_relative.to_path_buf();
let filter_root = walker_root.to_path_buf();
walker.filter_entry(move |entry| {
entry
.path()
.strip_prefix(&filter_root)
.is_ok_and(|relative| {
relative.starts_with(&search_relative) || search_relative.starts_with(relative)
})
});
} else {
walker
.hidden(true)
@@ -537,32 +417,17 @@ pub fn run_grep(
let mut visited = 0_usize;
let mut source_bytes_remaining = crate::MAX_GREP_SOURCE_BYTES;
let mut descriptor_ignores = DescriptorIgnoreMatchers::default();
if traversal.is_some() {
let mut directory = root.to_path_buf();
descriptor_ignores.load_directory(
&directory,
&base,
&mut source_bytes_remaining,
access,
)?;
for component in base
.strip_prefix(root)
.map_err(|_| {
FsError::InvalidArgument("grep base is outside its provider root".to_string())
})?
.components()
{
directory.push(component);
descriptor_ignores.load_directory(
&directory,
&base,
&mut source_bytes_remaining,
access,
)?;
}
}
for entry in walker.build().flatten() {
let mut descriptor_ignores = traversal
.as_ref()
.map(|_| DescriptorIgnores::grep(root, logical_walk_root, access))
.transpose()?;
let mut walk = if traversal.is_some() {
crate::walk::Walk::descriptor(logical_walk_root, access)
} else {
crate::walk::Walk::Plain(walker.build())
};
while let Some(entry) = walk.next() {
let entry = entry?;
access
.check_cancelled()
.map_err(|error| FsError::io(&base, error))?;
@@ -573,39 +438,41 @@ pub fn run_grep(
crate::MAX_TRAVERSAL_ENTRIES
)));
}
let walked_path = entry.path();
let path = if traversal.is_some() {
let relative = walked_path.strip_prefix(walker_root).map_err(|_| {
FsError::InvalidArgument(
"descriptor traversal returned a path outside its root".to_string(),
)
})?;
root.join(relative)
} else {
walked_path.to_path_buf()
};
let file_type = entry.file_type();
if traversal.is_some() && file_type.as_ref().is_some_and(|kind| kind.is_dir()) {
descriptor_ignores.load_directory(&path, &base, &mut source_bytes_remaining, access)?;
let path = entry.path;
if logical_path(root, &path).is_none() {
continue;
}
if !file_type.map(|kind| kind.is_file()).unwrap_or(false) {
continue;
}
if traversal.is_some() {
let file_type = entry.kind;
if let Some(ignores) = descriptor_ignores.as_mut() {
let relative = path.strip_prefix(&base).map_err(|_| {
FsError::InvalidArgument(
"descriptor traversal returned a path outside its search base".to_string(),
)
})?;
if relative
.components()
.any(|component| component.as_os_str().to_string_lossy().starts_with('.'))
|| descriptor_ignores.is_ignored(root, &path)
|| !direct_file_selected(&path, overrides.as_ref(), types.as_ref())
let is_dir = file_type.is_some_and(|kind| kind.is_dir());
if path != logical_walk_root
&& (relative.components().any(|component| {
component
.as_os_str()
.to_str()
.is_some_and(|name| name.starts_with('.'))
}) || ignores.is_ignored(&path, is_dir))
{
if is_dir {
walk.skip_current_dir();
}
continue;
}
if is_dir {
ignores.load_directory(&path, access)?;
continue;
}
if !direct_file_selected(&path, overrides.as_ref(), types.as_ref()) {
continue;
}
}
if !file_type.map(|kind| kind.is_file()).unwrap_or(false) {
continue;
}
let readable = access
.resolve_access_path(&path)
@@ -617,6 +484,7 @@ pub fn run_grep(
&mut searcher,
&matcher,
&path,
root,
mode,
&mut report,
&mut matching_files_seen,
@@ -638,6 +506,7 @@ fn scan_path(
searcher: &mut Searcher,
matcher: &grep_regex::RegexMatcher,
path: &Path,
root: &Path,
mode: GrepOutputMode,
report: &mut GrepReport,
matching_files_seen: &mut usize,
@@ -647,6 +516,9 @@ fn scan_path(
source_bytes_remaining: &mut u64,
access: &dyn FsAccessPolicy,
) -> Result<bool, FsError> {
if logical_path(root, path).is_none() {
return Ok(false);
}
let resolved = access
.resolve_access_path(path)
.map_err(|error| FsError::io(path, error))?;
@@ -677,7 +549,7 @@ fn scan_path(
return Ok(false);
}
if *matching_files_seen >= offset {
if !reserve_report_bytes(report, path.to_string_lossy().len().saturating_add(1)) {
if !reserve_report_bytes(report, path.as_os_str().len().saturating_add(1)) {
return Ok(true);
}
report.files.push(path.to_path_buf());
@@ -694,7 +566,7 @@ fn scan_path(
return Ok(false);
}
if *matching_files_seen >= offset {
if !reserve_report_bytes(report, path.to_string_lossy().len().saturating_add(32)) {
if !reserve_report_bytes(report, path.as_os_str().len().saturating_add(32)) {
return Ok(true);
}
report.counts.push((path.to_path_buf(), count));
@@ -779,7 +651,7 @@ impl ContentSink<'_> {
fn reserve(&mut self, content_bytes: usize) -> bool {
let retained = self
.path
.to_string_lossy()
.as_os_str()
.len()
.saturating_add(content_bytes)
.saturating_add(64);
+900
View File
@@ -0,0 +1,900 @@
//! Pure text-operation policy shared by filesystem providers and operation frontends.
//!
//! This module owns no paths, descriptors, version fences, or I/O. Limits count
//! UTF-8 bytes, not characters, and default to unlimited.
use schemars::JsonSchema;
use serde::{Deserialize, de::DeserializeOwned};
use thiserror::Error;
/// The shared text Operation definitions, independent of paths and frontends.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TextOperation {
Read,
ReadLines,
Edit,
Write,
}
impl TextOperation {
/// Resolve a lower-case Operation name. A frontend chooses whether its
/// `read` Operation accepts whole-text or line-range arguments.
pub fn from_name(name: &str, line_read: bool) -> Option<Self> {
match name {
"read" if line_read => Some(Self::ReadLines),
"read" => Some(Self::Read),
"edit" => Some(Self::Edit),
"write" => Some(Self::Write),
_ => None,
}
}
pub fn argument_schema(self) -> schemars::Schema {
match self {
Self::Read => schemars::schema_for!(ReadArgs),
Self::ReadLines => schemars::schema_for!(LineReadArgs),
Self::Edit => schemars::schema_for!(EditArgs),
Self::Write => schemars::schema_for!(WriteArgs),
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Deserialize, JsonSchema)]
pub struct EditArgs {
/// String to replace. Must be nonempty and unique unless replace_all is true.
pub old_string: String,
/// Replacement text. Must differ from old_string; may be empty to delete it.
pub new_string: String,
/// Replace every non-overlapping occurrence. Defaults to false.
#[serde(default)]
pub replace_all: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, Deserialize, JsonSchema)]
pub struct WriteArgs {
/// Full text to write.
pub content: String,
}
/// Whole-text reads accept no operation-specific arguments.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct ReadArgs {}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, JsonSchema)]
pub struct LineReadArgs {
/// Zero-based line offset. Omitted or null means zero.
#[serde(default)]
pub offset: Option<usize>,
/// Maximum lines to return. Omitted or null uses the frontend's default.
#[serde(default)]
pub limit: Option<usize>,
}
impl LineReadArgs {
pub fn range(&self, default_limit: usize) -> (usize, usize) {
(
self.offset.unwrap_or(0),
self.limit.unwrap_or(default_limit).max(1),
)
}
}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TextLimits {
pub max_input_bytes: Option<usize>,
pub max_output_bytes: Option<usize>,
pub max_replacements: Option<usize>,
}
#[derive(Debug, Error)]
pub enum TextError {
#[error("invalid arguments: {0}")]
Decode(#[source] serde_json::Error),
#[error("old_string must not be empty")]
EmptyOldString,
#[error("old_string and new_string are identical")]
IdenticalStrings,
#[error("old_string was not found")]
NotFound,
#[error(
"old_string matched {occurrences} times; set replace_all=true or provide a unique string"
)]
MultipleMatches { occurrences: usize },
#[error("input size {bytes} exceeds limit {limit}")]
InputTooLarge { bytes: usize, limit: usize },
#[error("output size {bytes} exceeds limit {limit}")]
OutputTooLarge { bytes: usize, limit: usize },
#[error("replacement count {replacements} exceeds limit {limit}")]
ReplacementLimitExceeded { replacements: usize, limit: usize },
#[error("edited content size overflowed")]
Overflow,
}
pub fn decode<T: DeserializeOwned>(value: serde_json::Value) -> Result<T, TextError> {
serde_json::from_value(value).map_err(TextError::Decode)
}
/// Decode raw Tool JSON directly, retaining duplicate-field rejection rather
/// than losing that information through an intermediate JSON object.
pub fn decode_json<T: DeserializeOwned>(input: &str) -> Result<T, TextError> {
serde_json::from_str(input).map_err(TextError::Decode)
}
impl EditArgs {
/// Validate argument-only policy before a caller acquires a preimage.
pub fn validate(&self, limits: TextLimits) -> Result<(), TextError> {
if self.old_string.is_empty() {
return Err(TextError::EmptyOldString);
}
if self.old_string == self.new_string {
return Err(TextError::IdenticalStrings);
}
check_input_bytes(self.old_string.len(), limits)?;
check_input_bytes(self.new_string.len(), limits)?;
Ok(())
}
}
/// A numbered, bounded text response with provider line metadata.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Rendered {
pub body: String,
pub line_count: usize,
pub total_lines: usize,
pub truncated: bool,
}
const RENDERED_TRUNCATION_MARKER: &str = "\n[truncated at rendered output byte limit]\n";
/// Render selected text with one-based line numbers and a hard UTF-8 byte bound.
///
/// As in the Tool renderer, room for the truncation marker is always reserved,
/// and a partially emitted numbered line counts as a line. Tiny bounds emit
/// only the marker prefix that fits. Line numbers saturate rather than wrap.
pub fn render_numbered(
text: &str,
start_line: usize,
total_lines: usize,
truncated: bool,
max_bytes: usize,
) -> Rendered {
let budget = max_bytes.saturating_sub(RENDERED_TRUNCATION_MARKER.len());
let mut body = String::with_capacity(text.len().min(max_bytes));
let mut line_count = 0usize;
let mut output_truncated = false;
for (index, line) in text.lines().enumerate() {
// Build only the small numeric prefix, not an unbounded copy of a line.
let number = start_line.saturating_add(index).saturating_add(1);
let prefix = format!("{number:>6}\t");
let numbered_len = prefix.len().saturating_add(line.len()).saturating_add(1);
let mut remaining = budget.saturating_sub(body.len());
if remaining > 0 {
line_count = line_count.saturating_add(1);
}
let line_truncated = numbered_len > remaining;
for fragment in [prefix.as_str(), line, "\n"] {
let mut end = remaining.min(fragment.len());
while !fragment.is_char_boundary(end) {
end -= 1;
}
body.push_str(&fragment[..end]);
remaining -= end;
// Do not skip a character that failed to fit and then emit later
// fragments; the response must be a prefix of the numbered line.
if end < fragment.len() {
break;
}
}
if line_truncated {
let marker_len = max_bytes
.saturating_sub(body.len())
.min(RENDERED_TRUNCATION_MARKER.len());
body.push_str(&RENDERED_TRUNCATION_MARKER[..marker_len]);
output_truncated = true;
break;
}
}
Rendered {
body,
line_count,
total_lines,
truncated: start_line > 0 || truncated || output_truncated,
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct EditOutcome {
pub content: String,
pub replacements: usize,
pub bytes_written: usize,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct TextOutcome {
pub content: String,
pub bytes: usize,
}
pub fn edit(original: &str, args: &EditArgs, limits: TextLimits) -> Result<EditOutcome, TextError> {
args.validate(limits)?;
check_input_bytes(original.len(), limits)?;
let occurrences = original.matches(&args.old_string).count();
if occurrences == 0 {
return Err(TextError::NotFound);
}
if !args.replace_all && occurrences != 1 {
return Err(TextError::MultipleMatches { occurrences });
}
let replacements = occurrences;
if let Some(limit) = limits.max_replacements
&& replacements > limit
{
return Err(TextError::ReplacementLimitExceeded {
replacements,
limit,
});
}
// All arithmetic and output policy are checked before allocating the result.
let bytes_written = edited_len(
original.len(),
args.old_string.len(),
args.new_string.len(),
replacements,
)?;
check_output_bytes(bytes_written, limits)?;
let mut content = String::with_capacity(bytes_written);
let mut cursor = 0;
for (start, matched) in original.match_indices(&args.old_string) {
content.push_str(&original[cursor..start]);
content.push_str(&args.new_string);
cursor = start + matched.len();
}
content.push_str(&original[cursor..]);
debug_assert_eq!(content.len(), bytes_written);
Ok(EditOutcome {
content,
replacements,
bytes_written,
})
}
fn edited_len(
original: usize,
old: usize,
new: usize,
replacements: usize,
) -> Result<usize, TextError> {
let removed = old.checked_mul(replacements).ok_or(TextError::Overflow)?;
let added = new.checked_mul(replacements).ok_or(TextError::Overflow)?;
let bytes = original
.checked_sub(removed)
.and_then(|bytes| bytes.checked_add(added))
.ok_or(TextError::Overflow)?;
// String/Vec capacity is bounded by isize::MAX even on platforms where usize
// arithmetic still fits. Reject that case before String::with_capacity.
if bytes > isize::MAX as usize {
return Err(TextError::Overflow);
}
Ok(bytes)
}
pub fn write(content: String, limits: TextLimits) -> Result<TextOutcome, TextError> {
text_outcome(content, limits)
}
pub fn read(content: String, limits: TextLimits) -> Result<TextOutcome, TextError> {
text_outcome(content, limits)
}
fn text_outcome(content: String, limits: TextLimits) -> Result<TextOutcome, TextError> {
let bytes = content.len();
check_input_bytes(bytes, limits)?;
check_output_bytes(bytes, limits)?;
Ok(TextOutcome { content, bytes })
}
pub(crate) fn check_input_bytes(bytes: usize, limits: TextLimits) -> Result<(), TextError> {
if let Some(limit) = limits.max_input_bytes
&& bytes > limit
{
return Err(TextError::InputTooLarge { bytes, limit });
}
Ok(())
}
pub(crate) fn check_output_bytes(bytes: usize, limits: TextLimits) -> Result<(), TextError> {
if let Some(limit) = limits.max_output_bytes
&& bytes > limit
{
return Err(TextError::OutputTooLarge { bytes, limit });
}
Ok(())
}
/// Keep valid UTF-8 prefixes intact without changing byte-read behavior on
/// genuinely invalid UTF-8. Providers must remain able to return binary data.
pub(crate) fn bounded_utf8_len(bytes: &[u8], max_bytes: usize) -> usize {
let mut end = max_bytes.min(bytes.len());
while end > 0 {
match std::str::from_utf8(&bytes[..end]) {
Ok(_) => break,
Err(error) if error.error_len().is_none() => end -= 1,
Err(_) => break,
}
}
end
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn args(old: &str, new: &str, replace_all: bool) -> EditArgs {
EditArgs {
old_string: old.into(),
new_string: new.into(),
replace_all,
}
}
#[test]
fn decode_uses_shared_argument_defaults_and_types() {
let edit = decode::<EditArgs>(json!({"old_string": "a", "new_string": "b"})).unwrap();
assert!(!edit.replace_all);
assert!(
decode::<EditArgs>(json!({"old_string": "a", "new_string": "b", "replace_all": true}))
.unwrap()
.replace_all
);
assert_eq!(
decode::<WriteArgs>(json!({"content": "😀"}))
.unwrap()
.content,
"😀"
);
assert_eq!(decode::<ReadArgs>(json!({})).unwrap(), ReadArgs {});
for value in [
json!({"old_string": "a"}),
json!({"old_string": 2, "new_string": "b"}),
json!({"old_string": "a", "new_string": "b", "replace_all": null}),
] {
assert!(matches!(
decode::<EditArgs>(value),
Err(TextError::Decode(_))
));
}
assert!(matches!(
decode::<WriteArgs>(json!({})),
Err(TextError::Decode(_))
));
assert!(matches!(
decode::<ReadArgs>(json!({"offset": 0})),
Err(TextError::Decode(_))
));
assert!(matches!(
decode::<ReadArgs>(json!(null)),
Err(TextError::Decode(_))
));
}
#[test]
fn line_range_preserves_null_defaults_and_minimum_one() {
for value in [json!({}), json!({"offset": null, "limit": null})] {
assert_eq!(
decode::<LineReadArgs>(value).unwrap().range(2000),
(0, 2000)
);
}
assert_eq!(
decode::<LineReadArgs>(json!({"offset": 7, "limit": 0}))
.unwrap()
.range(99),
(7, 1)
);
assert_eq!(LineReadArgs::default().range(0), (0, 1));
assert_eq!(
LineReadArgs {
offset: Some(usize::MAX),
limit: Some(usize::MAX)
}
.range(1),
(usize::MAX, usize::MAX)
);
for value in [
json!({"offset": -1}),
json!({"limit": "2"}),
json!({"limit": 1.5}),
] {
assert!(matches!(
decode::<LineReadArgs>(value),
Err(TextError::Decode(_))
));
}
}
#[test]
fn schema_is_the_argument_contract() {
let edit = serde_json::to_value(schemars::schema_for!(EditArgs)).unwrap();
assert_eq!(edit["required"], json!(["old_string", "new_string"]));
assert_eq!(edit["properties"]["replace_all"]["default"], json!(false));
let read = serde_json::to_value(schemars::schema_for!(ReadArgs)).unwrap();
assert_eq!(read["additionalProperties"], json!(false));
assert!(
read.get("properties")
.is_none_or(|properties| properties == &json!({}))
);
let lines = serde_json::to_value(schemars::schema_for!(LineReadArgs)).unwrap();
assert_eq!(
lines["properties"]["offset"]["type"],
json!(["integer", "null"])
);
assert_eq!(lines["properties"]["limit"]["default"], json!(null));
}
#[test]
fn edit_rejects_empty_identical_missing_and_ambiguous_matches() {
assert!(matches!(
args("", "x", true).validate(TextLimits::default()),
Err(TextError::EmptyOldString)
));
assert!(matches!(
edit("a", &args("a", "a", false), TextLimits::default()),
Err(TextError::IdenticalStrings)
));
assert!(matches!(
edit("a", &args("b", "c", false), TextLimits::default()),
Err(TextError::NotFound)
));
assert!(matches!(
edit("aa", &args("a", "b", false), TextLimits::default()),
Err(TextError::MultipleMatches { occurrences: 2 })
));
assert!(matches!(
edit("", &args("a", "", false), TextLimits::default()),
Err(TextError::NotFound)
));
}
#[test]
fn edit_handles_unique_all_deletion_literal_and_nonoverlapping_matches() {
for (original, old, new, all, expected, replacements) in [
(
"before old after",
"old",
"new",
false,
"before new after",
1,
),
("aaa", "aa", "x", true, "xa", 1),
("a-a-a", "a", "", true, "--", 3),
("[a].* [a].*", "[a].*", "$1", true, "$1 $1", 2),
("x", "x", "", false, "", 1),
] {
let result = edit(original, &args(old, new, all), TextLimits::default()).unwrap();
assert_eq!(result.content, expected);
assert_eq!(result.replacements, replacements);
assert_eq!(result.bytes_written, expected.len());
}
}
#[test]
fn edit_preserves_unicode_and_counts_bytes() {
let result = edit("é😀\né😀", &args("é😀", "界", true), TextLimits::default()).unwrap();
assert_eq!(result.content, "界\n界");
assert_eq!(result.replacements, 2);
assert_eq!(result.bytes_written, 7);
// Matching is literal, without Unicode normalization.
assert!(matches!(
edit("e\u{301}", &args("é", "x", false), TextLimits::default()),
Err(TextError::NotFound)
));
}
#[test]
fn edit_input_limits_cover_arguments_and_preimage_with_inclusive_boundaries() {
let limits = TextLimits {
max_input_bytes: Some(4),
..Default::default()
};
args("😀", "a", false).validate(limits).unwrap();
assert!(matches!(
args("😀x", "a", false).validate(limits),
Err(TextError::InputTooLarge { bytes: 5, limit: 4 })
));
assert!(matches!(
args("a", "😀x", false).validate(limits),
Err(TextError::InputTooLarge { bytes: 5, limit: 4 })
));
assert!(matches!(
edit("aaaaa", &args("a", "b", true), limits),
Err(TextError::InputTooLarge { bytes: 5, limit: 4 })
));
assert_eq!(
edit("aaaa", &args("a", "b", true), limits).unwrap().content,
"bbbb"
);
}
#[test]
fn edit_output_and_replacement_limits_apply_before_constructing_result() {
let args = args("a", "😀", true);
let limits = TextLimits {
max_output_bytes: Some(7),
..Default::default()
};
assert!(matches!(
edit("aa", &args, limits),
Err(TextError::OutputTooLarge { bytes: 8, limit: 7 })
));
assert_eq!(
edit(
"aa",
&args,
TextLimits {
max_output_bytes: Some(8),
..limits
}
)
.unwrap()
.bytes_written,
8
);
assert!(matches!(
edit(
"aa",
&args,
TextLimits {
max_replacements: Some(1),
..Default::default()
}
),
Err(TextError::ReplacementLimitExceeded {
replacements: 2,
limit: 1
})
));
assert!(matches!(
edit(
"a",
&args,
TextLimits {
max_replacements: Some(0),
..Default::default()
}
),
Err(TextError::ReplacementLimitExceeded {
replacements: 1,
limit: 0
})
));
assert_eq!(
edit(
"aa",
&args,
TextLimits {
max_replacements: Some(2),
..Default::default()
}
)
.unwrap()
.replacements,
2
);
assert_eq!(
edit(
"a",
&EditArgs {
old_string: "a".into(),
new_string: String::new(),
replace_all: false
},
TextLimits {
max_output_bytes: Some(0),
..Default::default()
}
)
.unwrap()
.content,
""
);
}
#[test]
fn raw_json_decode_preserves_duplicate_argument_rejection() {
for input in [
r#"{"old_string":"a","old_string":"b","new_string":"c"}"#,
r#"{"old_string":"a","new_string":"b","replace_all":false,"replace_all":true}"#,
] {
assert!(matches!(
decode_json::<EditArgs>(input),
Err(TextError::Decode(_))
));
}
assert!(matches!(
decode_json::<WriteArgs>(r#"{"content":"a","content":"b"}"#),
Err(TextError::Decode(_))
));
}
#[test]
fn size_arithmetic_rejects_every_overflow_before_preallocation() {
// Exercise impossible-to-allocate sizes through the exact helper edit
// invokes before allocating; no giant strings need to be constructed.
for (original, old, new, count) in [
(usize::MAX, usize::MAX, 0, 2), // removed multiplication
(2, 1, usize::MAX, 2), // added multiplication
(1, 2, 0, 1), // subtraction
(usize::MAX, 1, 2, 1), // addition
(1, 1, isize::MAX as usize + 1, 1), // allocation capacity
] {
assert!(matches!(
edited_len(original, old, new, count),
Err(TextError::Overflow)
));
}
assert_eq!(
edited_len(1, 1, isize::MAX as usize, 1).unwrap(),
isize::MAX as usize
);
assert_eq!(edited_len(8, 2, 1, 3).unwrap(), 5);
}
#[test]
fn read_and_write_preserve_whole_text_and_enforce_both_byte_limits() {
for operation in [read, write] {
assert_eq!(
operation(String::new(), TextLimits::default()).unwrap(),
TextOutcome {
content: String::new(),
bytes: 0
}
);
let content = "é\r\n😀\n";
let limits = TextLimits {
max_input_bytes: Some(content.len()),
max_output_bytes: Some(content.len()),
max_replacements: Some(0),
};
let result = operation(content.into(), limits).unwrap();
assert_eq!(result.content, content);
assert_eq!(result.bytes, content.len());
assert!(matches!(
operation(
"😀".into(),
TextLimits {
max_input_bytes: Some(3),
..Default::default()
}
),
Err(TextError::InputTooLarge { bytes: 4, limit: 3 })
));
assert!(matches!(
operation(
"😀".into(),
TextLimits {
max_output_bytes: Some(3),
..Default::default()
}
),
Err(TextError::OutputTooLarge { bytes: 4, limit: 3 })
));
assert!(matches!(
operation(
"a".into(),
TextLimits {
max_output_bytes: Some(0),
..Default::default()
}
),
Err(TextError::OutputTooLarge { bytes: 1, limit: 0 })
));
}
}
#[test]
fn utf8_byte_bounds_preserve_binary_compatibility() {
for (limit, expected) in [
(0, 0),
(1, 1),
(2, 1),
(3, 1),
(4, 1),
(5, 5),
(usize::MAX, 6),
] {
assert_eq!(bounded_utf8_len("a😀z".as_bytes(), limit), expected);
}
assert_eq!(bounded_utf8_len(&[0xff, 0xfe, 0x00], 2), 2);
}
#[test]
fn operation_names_and_schemas_are_shared_authority() {
assert_eq!(
TextOperation::from_name("read", false),
Some(TextOperation::Read)
);
assert_eq!(
TextOperation::from_name("read", true),
Some(TextOperation::ReadLines)
);
for line_read in [false, true] {
assert_eq!(
TextOperation::from_name("edit", line_read),
Some(TextOperation::Edit)
);
assert_eq!(
TextOperation::from_name("write", line_read),
Some(TextOperation::Write)
);
for name in ["Read", "WRITE", "read_lines", "stat", "", " read"] {
assert_eq!(TextOperation::from_name(name, line_read), None);
}
}
for (operation, expected) in [
(TextOperation::Read, schemars::schema_for!(ReadArgs)),
(
TextOperation::ReadLines,
schemars::schema_for!(LineReadArgs),
),
(TextOperation::Edit, schemars::schema_for!(EditArgs)),
(TextOperation::Write, schemars::schema_for!(WriteArgs)),
] {
assert_eq!(operation.argument_schema(), expected);
}
let schema = serde_json::to_value(TextOperation::Edit.argument_schema()).unwrap();
assert!(
schema["properties"]["new_string"]["description"]
.as_str()
.unwrap()
.contains("Must differ")
);
}
#[test]
fn renderer_preserves_numbering_lines_and_provider_metadata() {
let result = render_numbered("alpha\r\n\nβeta\n", 0, 3, false, 1024);
assert_eq!(result.body, " 1\talpha\n 2\t\n 3\tβeta\n");
assert_eq!(result.line_count, 3);
assert_eq!(result.total_lines, 3);
assert!(!result.truncated);
let result = render_numbered("last", 9, 10, false, 1024);
assert_eq!(result.body, " 10\tlast\n");
assert!(result.truncated);
assert!(render_numbered("last", 0, 10, true, 1024).truncated);
assert_eq!(
render_numbered("", 0, 0, false, 0),
Rendered {
body: String::new(),
line_count: 0,
total_lines: 0,
truncated: false,
}
);
assert!(render_numbered("", 1, 1, false, 0).truncated);
}
#[test]
fn renderer_reserves_marker_and_counts_partial_lines_as_before() {
let marker = RENDERED_TRUNCATION_MARKER;
let result = render_numbered("a\nb", 0, 2, false, marker.len() + 9);
assert_eq!(result.body, format!(" 1\ta\n{marker}"));
assert_eq!(result.line_count, 1);
assert_eq!(result.total_lines, 2);
assert!(result.truncated);
let result = render_numbered("a", 0, 1, false, marker.len() + 1);
assert_eq!(result.body, format!(" {marker}"));
assert_eq!(result.line_count, 1);
assert!(result.truncated);
let result = render_numbered("a", 0, 1, false, marker.len() + 9);
assert_eq!(result.body, " 1\ta\n");
assert_eq!(result.line_count, 1);
assert!(!result.truncated);
}
#[test]
fn renderer_tiny_bounds_unicode_and_large_offsets_never_overflow() {
let marker = RENDERED_TRUNCATION_MARKER;
for max_bytes in 0..=marker.len() {
let result = render_numbered("😀", 0, 1, false, max_bytes);
assert_eq!(result.body, marker[..max_bytes]);
assert_eq!(result.line_count, 0);
assert!(result.truncated);
}
for extra in 1..=4 {
let result = render_numbered("😀😀", 0, 1, false, marker.len() + 7 + extra);
let expected = if extra < 4 {
" 1\t"
} else {
" 1\t😀"
};
assert_eq!(result.body, format!("{expected}{marker}"));
assert_eq!(result.line_count, 1);
assert!(result.body.len() <= marker.len() + 7 + extra);
assert!(result.truncated);
}
let result = render_numbered("a\nb", usize::MAX, usize::MAX, false, usize::MAX);
assert_eq!(
result.body,
format!("{}\ta\n{}\tb\n", usize::MAX, usize::MAX)
);
assert_eq!(result.line_count, 2);
assert_eq!(result.total_lines, usize::MAX);
assert!(result.truncated);
}
#[test]
fn renderer_matches_original_behavior_for_ordinary_bounds() {
fn original(
text: &str,
start_line: usize,
total_lines: usize,
truncated: bool,
max_bytes: usize,
) -> Rendered {
let marker = RENDERED_TRUNCATION_MARKER;
let budget = max_bytes - marker.len();
let mut body = String::with_capacity(text.len().min(max_bytes));
let mut line_count = 0;
let mut output_truncated = false;
for (index, line) in text.lines().enumerate() {
let numbered = format!("{:>6}\t{}\n", start_line + index + 1, line);
let remaining = budget.saturating_sub(body.len());
if remaining > 0 {
line_count += 1;
}
if numbered.len() > remaining {
let mut end = remaining;
while !numbered.is_char_boundary(end) {
end -= 1;
}
body.push_str(&numbered[..end]);
body.push_str(marker);
output_truncated = true;
break;
}
body.push_str(&numbered);
}
Rendered {
body,
line_count,
total_lines,
truncated: start_line > 0 || truncated || output_truncated,
}
}
for text in ["", "a", "\n", "😀é界\nsecond\n", "first\r\n\r\nlast"] {
for start in [0, 1, 999_999] {
for bound in RENDERED_TRUNCATION_MARKER.len()..=160 {
let result = render_numbered(text, start, 99, false, bound);
assert_eq!(result, original(text, start, 99, false, bound));
assert!(result.body.len() <= bound);
}
}
}
}
#[test]
fn errors_keep_actionable_messages_and_decode_source() {
assert_eq!(
TextError::EmptyOldString.to_string(),
"old_string must not be empty"
);
assert_eq!(
TextError::IdenticalStrings.to_string(),
"old_string and new_string are identical"
);
assert_eq!(TextError::NotFound.to_string(), "old_string was not found");
assert_eq!(
TextError::MultipleMatches { occurrences: 3 }.to_string(),
"old_string matched 3 times; set replace_all=true or provide a unique string"
);
assert_eq!(
TextError::Overflow.to_string(),
"edited content size overflowed"
);
let error = decode::<ReadArgs>(json!({"unknown": true})).unwrap_err();
assert!(std::error::Error::source(&error).is_some());
}
}
+108
View File
@@ -0,0 +1,108 @@
#![cfg(target_os = "linux")]
use crate::*;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
struct Filtered {
root: PathBuf,
seen: Arc<Mutex<Vec<PathBuf>>>,
}
impl FsAccessPolicy for Filtered {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.root) && !path.starts_with(self.root.join("search/denied"))
}
fn is_writable(&self, _: &Path) -> bool {
false
}
fn open_traversal_root(
&self,
_: &Path,
resolved: &Path,
) -> std::io::Result<Option<FsTraversalRoot>> {
use std::os::fd::AsRawFd;
let file = std::fs::File::open(resolved)?;
let walk = PathBuf::from(format!("/proc/self/fd/{}", file.as_raw_fd()));
Ok(Some(FsTraversalRoot::new_at(
walk,
file,
resolved.to_path_buf(),
)))
}
fn traversal_filter(&self) -> Option<FsTraversalFilter> {
let denied = self.root.join("search/denied");
let seen = self.seen.clone();
Some(Arc::new(move |path, _| {
seen.lock().unwrap().push(path.to_path_buf());
!path.starts_with(&denied)
}))
}
fn open_read_file(&self, _: &Path, resolved: &Path) -> std::io::Result<std::fs::File> {
assert!(
!resolved.starts_with(self.root.join("search/denied")),
"denied source opened"
);
std::fs::File::open(resolved)
}
}
#[test]
fn descriptor_search_anchor_and_predicate_prune_before_denied_directory_descent() {
let root = tempfile::tempdir().unwrap();
std::fs::create_dir_all(root.path().join("search/denied/deep")).unwrap();
std::fs::write(root.path().join("search/allowed.txt"), "needle allowed").unwrap();
std::fs::write(
root.path().join("search/denied/deep/secret.txt"),
"needle secret",
)
.unwrap();
std::fs::write(root.path().join("outside.txt"), "needle outside").unwrap();
let policy = Filtered {
root: root.path().to_path_buf(),
seen: Arc::new(Mutex::new(Vec::new())),
};
let glob = run_glob(
root.path(),
&root.path().join("search"),
GlobRequest {
path: FsPath::new("search").unwrap(),
pattern: "**/*.txt".into(),
limit: 100,
},
&policy,
)
.unwrap();
assert_eq!(glob.paths, vec![FsPath::new("search/allowed.txt").unwrap()]);
let grep = run_grep(
root.path(),
root.path().join("search"),
GrepRequest {
path: FsPath::new("search").unwrap(),
pattern: "needle".into(),
glob: None,
file_type: None,
case_insensitive: false,
before_context: 0,
after_context: 0,
multiline: false,
output_mode: GrepOutputMode::Content,
offset: 0,
limit: 100,
},
&policy,
)
.unwrap();
assert_eq!(grep.paths, vec![FsPath::new("search/allowed.txt").unwrap()]);
let seen = policy.seen.lock().unwrap();
assert!(
seen.iter()
.any(|path| path == &root.path().join("search/denied"))
);
assert!(
!seen
.iter()
.any(|path| path.starts_with(root.path().join("search/denied/deep")))
);
assert!(
!seen
.iter()
.any(|path| path == &root.path().join("outside.txt"))
);
}
+233
View File
@@ -0,0 +1,233 @@
//! Bounded descriptor-provider walk shared by Glob and Grep. Unlike a
//! WalkBuilder predicate, this iterator counts even rejected entries and checks
//! cancellation while scanning them. Denied directories are never enumerated.
use crate::{FsAccessPolicy, FsError, FsTraversalFilter};
use std::path::{Path, PathBuf};
pub(crate) struct Entry {
pub path: PathBuf,
pub kind: Option<std::fs::FileType>,
}
pub(crate) enum Walk<'a> {
Plain(ignore::Walk),
Descriptor(DescriptorWalk<'a>),
}
impl<'a> Walk<'a> {
/// A rejected directory has been yielded but not enumerated yet. Keep
/// ignore pruning inside the same bounded descriptor walk as scope pruning.
pub fn skip_current_dir(&mut self) {
if let Self::Descriptor(walk) = self {
walk.pending = None;
}
}
pub fn descriptor(root: &Path, access: &'a dyn FsAccessPolicy) -> Self {
Self::Descriptor(DescriptorWalk {
access,
filter: access.traversal_filter(),
pending: Some((root.to_path_buf(), 0)),
root: Some(root.to_path_buf()),
stack: Vec::new(),
visited: 0,
failed: false,
})
}
}
impl Iterator for Walk<'_> {
type Item = Result<Entry, FsError>;
fn next(&mut self) -> Option<Self::Item> {
match self {
Self::Plain(walk) => loop {
match walk.next()? {
Ok(entry) => {
return Some(Ok(Entry {
path: entry.path().to_path_buf(),
kind: entry.file_type(),
}));
}
Err(_) => continue,
}
},
Self::Descriptor(walk) => walk.next(),
}
}
}
pub(crate) struct DescriptorWalk<'a> {
access: &'a dyn FsAccessPolicy,
filter: Option<FsTraversalFilter>,
root: Option<PathBuf>,
pending: Option<(PathBuf, usize)>,
stack: Vec<(PathBuf, usize, std::fs::ReadDir)>,
visited: usize,
failed: bool,
}
impl DescriptorWalk<'_> {
fn next_entry(&mut self) -> Result<Option<Entry>, FsError> {
if let Some(root) = self.root.take() {
self.access
.check_cancelled()
.map_err(|e| FsError::io(&root, e))?;
let resolved = self
.access
.resolve_access_path(&root)
.map_err(|e| FsError::io(&root, e))?;
let metadata = self
.access
.read_metadata(&root, &resolved)
.map_err(|e| FsError::io(&root, e))?;
return Ok(Some(Entry {
path: root,
kind: Some(metadata.file_type()),
}));
}
loop {
if let Some((path, depth)) = self.pending.take() {
self.access
.check_cancelled()
.map_err(|e| FsError::io(&path, e))?;
let selected = self
.filter
.as_ref()
.is_none_or(|filter| filter(&path, true));
let resolved = self
.access
.resolve_access_path(&path)
.map_err(|e| FsError::io(&path, e))?;
if selected && self.access.can_enumerate_directory(&path, &resolved) {
if depth >= 128 {
return Err(FsError::InvalidArgument(
"descriptor traversal depth exceeds provider limit 128".into(),
));
}
let entries = self
.access
.open_read_dir(&path, &resolved)
.map_err(|e| FsError::io(&path, e))?;
self.stack.push((path, depth, entries));
}
}
let Some((parent, depth, entries)) = self.stack.last_mut() else {
return Ok(None);
};
self.access
.check_cancelled()
.map_err(|e| FsError::io(&*parent, e))?;
let Some(entry) = entries.next() else {
self.stack.pop();
continue;
};
self.visited = self.visited.saturating_add(1);
if self.visited > crate::MAX_TRAVERSAL_ENTRIES {
return Err(FsError::InvalidArgument(
"descriptor traversal exceeds provider entry limit".into(),
));
}
let entry = entry.map_err(|e| FsError::io(&*parent, e))?;
if entry.file_name().to_str().is_none() {
continue;
}
let path = parent.join(entry.file_name());
let kind = entry.file_type().map_err(|e| FsError::io(&path, e))?;
let selected = self
.filter
.as_ref()
.is_none_or(|filter| filter(&path, kind.is_dir()));
if !selected {
continue;
}
let resolved = match self.access.resolve_access_path(&path) {
Ok(path) => path,
Err(_) => continue,
};
if !self.access.is_readable_paths(&path, &resolved) {
continue;
}
if kind.is_dir() {
self.pending = Some((path.clone(), depth.saturating_add(1)));
}
return Ok(Some(Entry {
path,
kind: Some(kind),
}));
}
}
}
impl Iterator for DescriptorWalk<'_> {
type Item = Result<Entry, FsError>;
fn next(&mut self) -> Option<Self::Item> {
if self.failed {
return None;
}
match self.next_entry() {
Ok(entry) => entry.map(Ok),
Err(error) => {
self.failed = true;
Some(Err(error))
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{
Arc,
atomic::{AtomicUsize, Ordering},
};
struct RejectFiles {
calls: AtomicUsize,
cancel_at: usize,
}
impl FsAccessPolicy for RejectFiles {
fn is_readable(&self, _: &Path) -> bool {
true
}
fn is_writable(&self, _: &Path) -> bool {
false
}
fn traversal_filter(&self) -> Option<FsTraversalFilter> {
Some(Arc::new(|_, directory| directory))
}
fn check_cancelled(&self) -> std::io::Result<()> {
if self.calls.fetch_add(1, Ordering::Relaxed) >= self.cancel_at {
Err(std::io::Error::from(std::io::ErrorKind::Interrupted))
} else {
Ok(())
}
}
}
#[test]
fn descriptor_walk_counts_rejected_entries_toward_the_entry_budget() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join("rejected"), "must not be opened").unwrap();
let policy = RejectFiles {
calls: AtomicUsize::new(0),
cancel_at: usize::MAX,
};
let Walk::Descriptor(mut walk) = Walk::descriptor(root.path(), &policy) else {
panic!()
};
assert!(walk.next().unwrap().is_ok());
walk.visited = crate::MAX_TRAVERSAL_ENTRIES;
assert!(matches!(
walk.next().unwrap(),
Err(FsError::InvalidArgument(_))
));
}
#[test]
fn descriptor_walk_checks_cancellation_while_scanning_only_rejected_entries() {
let root = tempfile::tempdir().unwrap();
for index in 0..20 {
std::fs::write(root.path().join(index.to_string()), "must not be opened").unwrap();
}
let policy = RejectFiles {
calls: AtomicUsize::new(0),
cancel_at: 4,
};
let mut walk = Walk::descriptor(root.path(), &policy);
assert!(walk.next().unwrap().is_ok());
assert!(
matches!(walk.next().unwrap(), Err(FsError::Io { source, .. }) if source.kind() == std::io::ErrorKind::Interrupted)
);
}
}
@@ -0,0 +1,248 @@
use fs_operation::{
EntryKind, FsAccessPolicy, FsError, FsPath, ListCursor, ListRequest, ListResult,
MAX_RESULT_PATH_BYTES, run_list,
};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicUsize, Ordering};
struct Access {
root: PathBuf,
checks: AtomicUsize,
cancel_at: usize,
enumerate: bool,
}
impl Access {
fn new(root: &Path) -> Self {
Self {
root: root.to_path_buf(),
checks: AtomicUsize::new(0),
cancel_at: usize::MAX,
enumerate: true,
}
}
}
impl FsAccessPolicy for Access {
fn is_readable(&self, path: &Path) -> bool {
path.starts_with(&self.root) && path != self.root.join("denied")
}
fn is_writable(&self, _: &Path) -> bool {
false
}
fn can_enumerate_directory(&self, _: &Path, _: &Path) -> bool {
self.enumerate
}
fn check_cancelled(&self) -> std::io::Result<()> {
if self.checks.fetch_add(1, Ordering::Relaxed) >= self.cancel_at {
return Err(std::io::Error::new(
std::io::ErrorKind::Interrupted,
"cancelled",
));
}
Ok(())
}
}
fn path(value: &str) -> FsPath {
FsPath::new(value).unwrap()
}
fn page(root: &Path, limit: usize, after: Option<ListCursor>) -> ListResult {
run_list(
root,
ListRequest {
path: FsPath::root(),
limit,
after,
},
&Access::new(root),
)
.unwrap()
}
fn names(result: &ListResult) -> Vec<&str> {
result
.entries
.iter()
.map(|entry| entry.path.as_str())
.collect()
}
#[test]
fn list_pages_order_directories_first_then_paths_without_duplicates() {
let root = tempfile::tempdir().unwrap();
for name in ["z", "b", "a"] {
std::fs::write(root.path().join(name), b"xx").unwrap();
}
for name in ["z-dir", "a-dir"] {
std::fs::create_dir(root.path().join(name)).unwrap();
}
let full = page(root.path(), usize::MAX, None);
assert_eq!(names(&full), ["a-dir", "z-dir", "a", "b", "z"]);
let mut after = None;
let mut entries = Vec::new();
loop {
let result = page(root.path(), 2, after);
assert!(result.entries.len() <= 2);
assert_eq!(result.total_entries, 5);
assert_eq!(result.total_bytes, full.total_bytes);
assert_eq!(result.truncated, result.next_after.is_some());
after = result.next_after;
entries.extend(result.entries);
if after.is_none() {
break;
}
}
assert_eq!(entries, full.entries);
let zero = page(root.path(), 0, None);
assert!(zero.entries.is_empty());
assert!(zero.truncated);
assert!(zero.next_after.is_none());
assert_eq!(zero.total_entries, 5);
let exhausted = page(
root.path(),
2,
Some(ListCursor {
kind: EntryKind::File,
path: path("z"),
}),
);
assert!(exhausted.entries.is_empty());
assert!(!exhausted.truncated);
assert!(exhausted.next_after.is_none());
assert_eq!(exhausted.total_entries, 5);
}
#[test]
fn list_live_cursor_survives_removal_and_only_observes_new_keys_after_it() {
let root = tempfile::tempdir().unwrap();
for name in ["b", "d"] {
std::fs::write(root.path().join(name), b"x").unwrap();
}
let first = page(root.path(), 1, None);
assert_eq!(names(&first), ["b"]);
std::fs::remove_file(root.path().join("b")).unwrap();
for name in ["a", "c"] {
std::fs::write(root.path().join(name), b"xx").unwrap();
}
// All directories sort before a file cursor, even when created later.
std::fs::create_dir(root.path().join("z-directory")).unwrap();
let second = page(root.path(), 10, first.next_after);
assert_eq!(names(&second), ["c", "d"]);
assert_eq!(second.total_entries, 4);
assert!(!second.truncated);
assert!(second.next_after.is_none());
std::fs::remove_file(root.path().join("d")).unwrap();
std::fs::rename(root.path().join("a"), root.path().join("e")).unwrap();
let moved = page(
root.path(),
10,
Some(ListCursor {
kind: EntryKind::File,
path: path("d"),
}),
);
assert_eq!(names(&moved), ["e"]);
}
#[test]
fn list_path_byte_bound_returns_a_continuation_instead_of_losing_the_tail() {
let root = tempfile::tempdir().unwrap();
let long_count = MAX_RESULT_PATH_BYTES / (240 + 64) + 10;
// A shorter late key must not leap over a longer evicted earlier key.
let count = long_count + 50;
let expected: Vec<_> = (0..count)
.map(|n| format!("{n:05}{}", "x".repeat(if n < long_count { 235 } else { 1 })))
.collect();
for name in expected.iter().rev() {
std::fs::write(root.path().join(name), b"x").unwrap();
}
let first = page(root.path(), usize::MAX, None);
assert!(first.truncated);
assert_eq!(first.total_entries, count);
assert_eq!(first.total_bytes, count as u64);
let path_bytes: usize = first
.entries
.iter()
.map(|entry| entry.path.as_str().len() + 64)
.sum::<usize>()
+ first.next_after.as_ref().unwrap().path.as_str().len()
+ 64;
assert!(path_bytes <= MAX_RESULT_PATH_BYTES);
let next = page(root.path(), usize::MAX, first.next_after.clone());
assert!(!next.truncated);
let got: Vec<_> = names(&first).into_iter().chain(names(&next)).collect();
assert_eq!(got, expected.iter().map(String::as_str).collect::<Vec<_>>());
let small = page(root.path(), 1, None);
assert_eq!(names(&small), [expected[0].as_str()]);
assert_eq!(small.total_entries, count);
}
#[test]
fn list_keeps_ignored_and_hidden_entries_but_respects_read_and_enumeration_authority() {
let root = tempfile::tempdir().unwrap();
std::fs::write(root.path().join(".gitignore"), "ignored\n").unwrap();
for name in ["ignored", ".hidden", "denied"] {
std::fs::write(root.path().join(name), b"x").unwrap();
}
let first = page(root.path(), 2, None);
assert_eq!(names(&first), [".gitignore", ".hidden"]);
let second = page(root.path(), 2, first.next_after);
assert_eq!(names(&second), ["ignored"]);
assert_eq!(second.total_entries, 3);
let mut access = Access::new(root.path());
access.enumerate = false;
let result = run_list(
root.path(),
ListRequest {
path: FsPath::root(),
limit: 1,
after: None,
},
&access,
)
.unwrap();
assert!(result.entries.is_empty());
assert!(!result.truncated);
assert!(result.next_after.is_none());
}
#[test]
fn list_cancellation_is_checked_even_after_the_page_is_full() {
let root = tempfile::tempdir().unwrap();
for name in ["a", "b", "c"] {
std::fs::write(root.path().join(name), b"x").unwrap();
}
let mut access = Access::new(root.path());
access.cancel_at = 3;
let error = run_list(
root.path(),
ListRequest {
path: FsPath::root(),
limit: 1,
after: None,
},
&access,
)
.unwrap_err();
assert!(
matches!(error, FsError::Io { source, .. } if source.kind() == std::io::ErrorKind::Interrupted)
);
}
#[test]
fn list_rejects_cursor_outside_the_requested_directory() {
let root = tempfile::tempdir().unwrap();
for value in ["", "other/a"] {
let error = run_list(
root.path(),
ListRequest {
path: FsPath::root(),
limit: 1,
after: Some(ListCursor {
kind: EntryKind::File,
path: path(value),
}),
},
&Access::new(root.path()),
)
.unwrap_err();
assert!(matches!(error, FsError::InvalidArgument(_)));
}
}
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "job"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish = false
[dependencies]
manifest.workspace = true
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
sha2.workspace = true
thiserror.workspace = true
+336
View File
@@ -0,0 +1,336 @@
//! Transport-neutral immutable Job intent and structured-result contract.
//!
//! Hosts own execution, authority, persistence and lifecycle state. This crate
//! deliberately contains no runner, state machine, Runtime or domain grants.
use serde::{Deserialize, Serialize};
use serde_json::Value;
use sha2::{Digest, Sha256};
pub const DEFAULT_MAX_CONCURRENT_JOBS: u16 = 8;
pub const ABSOLUTE_MAX_CONCURRENT_JOBS: u16 = 64;
pub const DEFAULT_JOB_TIMEOUT_SECONDS: u32 = 120;
pub const MAX_JOB_TIMEOUT_SECONDS: u32 = 600;
pub const DEFAULT_MAX_RESULT_BYTES: u32 = 16 * 1024;
pub const ABSOLUTE_MAX_RESULT_BYTES: u32 = 64 * 1024;
pub const DEFAULT_MAX_ATTEMPTS: u8 = 2;
pub const ABSOLUTE_MAX_ATTEMPTS: u8 = 3;
pub const MAX_JOB_INPUT_BYTES: usize = 128 * 1024;
pub const MAX_JOB_INSTRUCTION_BYTES: usize = 16 * 1024;
pub const MAX_JOB_PURPOSE_BYTES: usize = 120;
pub const MAX_JOB_REFERENCE_BYTES: usize = 512;
pub const MAX_JOB_FAILURE_BYTES: usize = 1024;
pub const MAX_JOB_DELIVERY_BYTES: usize = 16 * 1024;
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum JobError {
#[error("{0}")]
InvalidInput(String),
}
pub type Result<T> = std::result::Result<T, JobError>;
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct JobLimits {
#[serde(default = "default_max_concurrent_jobs")]
pub max_concurrent_jobs: u16,
#[serde(default = "default_job_timeout_seconds")]
pub timeout_seconds: u32,
#[serde(default = "default_max_result_bytes")]
pub max_result_bytes: u32,
#[serde(default = "default_max_attempts")]
pub max_attempts: u8,
}
impl Default for JobLimits {
fn default() -> Self {
Self {
max_concurrent_jobs: DEFAULT_MAX_CONCURRENT_JOBS,
timeout_seconds: DEFAULT_JOB_TIMEOUT_SECONDS,
max_result_bytes: DEFAULT_MAX_RESULT_BYTES,
max_attempts: DEFAULT_MAX_ATTEMPTS,
}
}
}
impl JobLimits {
pub fn validate(&self) -> Result<()> {
if self.max_concurrent_jobs == 0 || self.max_concurrent_jobs > ABSOLUTE_MAX_CONCURRENT_JOBS
{
return Err(JobError::InvalidInput(format!(
"Job max_concurrent_jobs must be in 1..={ABSOLUTE_MAX_CONCURRENT_JOBS}"
)));
}
if self.timeout_seconds == 0 || self.timeout_seconds > MAX_JOB_TIMEOUT_SECONDS {
return Err(JobError::InvalidInput(format!(
"Job timeout_seconds must be in 1..={MAX_JOB_TIMEOUT_SECONDS}"
)));
}
if self.max_result_bytes == 0 || self.max_result_bytes > ABSOLUTE_MAX_RESULT_BYTES {
return Err(JobError::InvalidInput(format!(
"Job max_result_bytes must be in 1..={ABSOLUTE_MAX_RESULT_BYTES}"
)));
}
if self.max_attempts == 0 || self.max_attempts > ABSOLUTE_MAX_ATTEMPTS {
return Err(JobError::InvalidInput(format!(
"Job max_attempts must be in 1..={ABSOLUTE_MAX_ATTEMPTS}"
)));
}
Ok(())
}
}
const fn default_max_concurrent_jobs() -> u16 {
DEFAULT_MAX_CONCURRENT_JOBS
}
const fn default_job_timeout_seconds() -> u32 {
DEFAULT_JOB_TIMEOUT_SECONDS
}
const fn default_max_result_bytes() -> u32 {
DEFAULT_MAX_RESULT_BYTES
}
const fn default_max_attempts() -> u8 {
DEFAULT_MAX_ATTEMPTS
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct JobRequest {
/// Stable caller-selected idempotency key within the host's namespace.
pub job_id: String,
pub purpose: String,
/// Revision fenced by the structured result.
pub input_revision: String,
pub input_ref: String,
/// Bounded immutable snapshot; must not contain credentials.
pub input: Value,
pub instruction: String,
/// Manifest registry selector, never raw source or a filesystem path.
pub profile: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub serialization_key: Option<String>,
#[serde(default)]
pub limits: JobLimits,
}
impl JobRequest {
pub fn validate(&self) -> Result<()> {
self.validate_fields()?;
self.validate_input()?;
self.limits.validate()
}
/// Request metadata validation, separable so adapters can retain their
/// domain-validation ordering before validating input and limits.
pub fn validate_fields(&self) -> Result<()> {
validate_identifier("job_id", &self.job_id, 256)?;
validate_identifier("purpose", &self.purpose, MAX_JOB_PURPOSE_BYTES)?;
validate_identifier("input_revision", &self.input_revision, 256)?;
validate_bounded_text("input_ref", &self.input_ref, MAX_JOB_REFERENCE_BYTES)?;
validate_bounded_text("instruction", &self.instruction, MAX_JOB_INSTRUCTION_BYTES)?;
self.profile_selector()?;
if let Some(key) = &self.serialization_key {
validate_identifier("serialization_key", key, MAX_JOB_REFERENCE_BYTES)?;
}
Ok(())
}
pub fn validate_input(&self) -> Result<()> {
let input = serde_json::to_vec(&self.input)
.map_err(|error| JobError::InvalidInput(format!("serialize Job input: {error}")))?;
if input.len() > MAX_JOB_INPUT_BYTES {
return Err(JobError::InvalidInput(format!(
"Job input exceeds {MAX_JOB_INPUT_BYTES} bytes"
)));
}
Ok(())
}
/// Syntax only; registry existence and resolution are host authority.
pub fn profile_selector(&self) -> Result<manifest::ProfileSelector> {
profile_selector(&self.profile)
}
pub fn fingerprint(&self) -> Result<String> {
self.validate()?;
fingerprint(self)
}
pub fn worker_input(&self, attempt_id: &str) -> Result<String> {
worker_input(self, attempt_id)
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct JobResultSubmission {
pub job_id: String,
pub attempt_id: String,
pub input_revision: String,
pub result: Value,
}
/// Client-facing lifecycle vocabulary, not a runner or transition policy.
/// A variant's presence does not imply every host supports that operation.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum JobState {
Pending,
Completed,
Failed,
Unknown,
Cancelled,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum JobAttemptState {
Reserved,
Dispatching,
Dispatched,
Completed,
Failed,
Unknown,
Cancelled,
}
/// Transport-neutral read snapshot. Job status/result and the corresponding
/// attempt identity/status remain distinct; host identities, delivery and
/// resource cleanup are deliberately not part of the model outcome.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct JobOutcome {
pub job_id: String,
pub input_revision: String,
pub attempt_id: String,
pub attempt: u8,
pub state: JobState,
pub attempt_state: JobAttemptState,
pub result: Option<Value>,
pub failure_category: Option<String>,
pub failure_detail: Option<String>,
}
pub fn profile_selector(profile: &str) -> Result<manifest::ProfileSelector> {
use manifest::ProfileSelector;
validate_identifier("profile", profile, 256)?;
let selector = ProfileSelector::parse_cli(profile);
let valid = match &selector {
ProfileSelector::Default => true,
ProfileSelector::Path { .. } => false,
ProfileSelector::Named { source, name } => {
!name.is_empty()
&& !name.starts_with('.')
&& !name.contains(['/', '\\', ':'])
&& name
.chars()
.all(|ch| ch.is_ascii_alphanumeric() || matches!(ch, '-' | '_' | '.'))
&& ![".toml", ".json", ".dcdl", ".nix"]
.iter()
.any(|suffix| name.ends_with(suffix))
&& (source.is_some() || name != "inherit")
}
};
if !valid {
return Err(JobError::InvalidInput(
"Job profile must be a manifest registry selector, not raw source or a path".into(),
));
}
Ok(selector)
}
/// Hash the full serialized host intent, not just its neutral projection.
/// Callers must validate the complete intent first. JSON field ordering and
/// omission rules are intentionally preserved for durable replay compatibility.
pub fn fingerprint<T: Serialize + ?Sized>(intent: &T) -> Result<String> {
let encoded = serde_json::to_vec(intent)
.map_err(|error| JobError::InvalidInput(format!("serialize Job intent: {error}")))?;
Ok(sha256(&encoded))
}
pub fn worker_input(request: &JobRequest, attempt_id: &str) -> Result<String> {
worker_input_with_label(request, attempt_id, "Job")
}
/// Adapter-specific capability label; only the host, never the model, selects it.
/// Keeps an existing host's prompt envelope byte-for-byte compatible.
pub fn worker_input_with_label(
request: &JobRequest,
attempt_id: &str,
label: &str,
) -> Result<String> {
let input = serde_json::to_string(&request.input)
.map_err(|error| JobError::InvalidInput(format!("serialize Job input: {error}")))?;
Ok(format!(
"{}\n\n{label} envelope (immutable):\njob_id: {}\nattempt_id: {}\ninput_revision: {}\ninput_ref: {}\ninput_json: {}\n\nReturn success only through the structured {label} result capability. Final prose and Worker Idle/Stopped state are not result authority.",
request.instruction,
request.job_id,
attempt_id,
request.input_revision,
request.input_ref,
input
))
}
pub fn attempt_id(job_id: &str, attempt: u8) -> String {
format!("{job_id}:attempt:{attempt}")
}
/// Bound the encoded JSON before hashing. Returns (digest, exact JSON).
pub fn result_digest(result: &Value, max_bytes: u32) -> Result<(String, String)> {
let encoded = serde_json::to_string(result)
.map_err(|error| JobError::InvalidInput(format!("serialize Job result: {error}")))?;
if encoded.len() > max_bytes as usize {
return Err(JobError::InvalidInput(format!(
"Job result exceeds {max_bytes} bytes"
)));
}
Ok((sha256(encoded.as_bytes()), encoded))
}
pub fn bounded_failure_detail(detail: &str) -> String {
if detail.len() <= MAX_JOB_FAILURE_BYTES {
return detail.to_string();
}
let mut end = MAX_JOB_FAILURE_BYTES;
while !detail.is_char_boundary(end) {
end -= 1;
}
detail[..end].to_string()
}
pub fn validate_identifier(field: &str, value: &str, max_bytes: usize) -> Result<()> {
validate_bounded_text(field, value, max_bytes)?;
if value.chars().any(char::is_control) {
return Err(JobError::InvalidInput(format!(
"Job {field} must not contain control characters"
)));
}
Ok(())
}
pub fn validate_bounded_text(field: &str, value: &str, max_bytes: usize) -> Result<()> {
if value.is_empty() || value.trim() != value {
return Err(JobError::InvalidInput(format!(
"Job {field} must be non-empty and trimmed"
)));
}
if value.len() > max_bytes {
return Err(JobError::InvalidInput(format!(
"Job {field} exceeds {max_bytes} bytes"
)));
}
Ok(())
}
pub fn sha256(bytes: &[u8]) -> String {
let digest = Sha256::digest(bytes);
let mut output = String::with_capacity(7 + digest.len() * 2);
output.push_str("sha256:");
for byte in digest {
use std::fmt::Write as _;
let _ = write!(output, "{byte:02x}");
}
output
}
+308
View File
@@ -0,0 +1,308 @@
use job::*;
use serde_json::json;
fn request() -> JobRequest {
JobRequest {
job_id: "check:r7".into(),
purpose: "snapshot_check".into(),
input_revision: "7".into(),
input_ref: "snapshot://7".into(),
input: json!({"title": "Check me"}),
instruction: "Check this immutable snapshot.".into(),
profile: "project:custom-check".into(),
serialization_key: None,
limits: JobLimits::default(),
}
}
#[test]
fn shared_lifecycle_enums_round_trip_all_host_states() {
for (state, name) in [
(JobState::Pending, "pending"),
(JobState::Completed, "completed"),
(JobState::Failed, "failed"),
(JobState::Unknown, "unknown"),
(JobState::Cancelled, "cancelled"),
] {
assert_eq!(serde_json::to_value(state).unwrap(), json!(name));
assert_eq!(
serde_json::from_value::<JobState>(json!(name)).unwrap(),
state
);
}
for (state, name) in [
(JobAttemptState::Reserved, "reserved"),
(JobAttemptState::Dispatching, "dispatching"),
(JobAttemptState::Dispatched, "dispatched"),
(JobAttemptState::Completed, "completed"),
(JobAttemptState::Failed, "failed"),
(JobAttemptState::Unknown, "unknown"),
(JobAttemptState::Cancelled, "cancelled"),
] {
assert_eq!(serde_json::to_value(state).unwrap(), json!(name));
assert_eq!(
serde_json::from_value::<JobAttemptState>(json!(name)).unwrap(),
state
);
}
assert!(serde_json::from_value::<JobState>(json!("running")).is_err());
assert!(serde_json::from_value::<JobAttemptState>(json!("idle")).is_err());
}
#[test]
fn outcome_is_a_neutral_read_snapshot_with_optional_result_and_failure() {
let mut outcome = JobOutcome {
job_id: "job-1".into(),
input_revision: "7".into(),
attempt_id: "job-1:attempt:2".into(),
attempt: 2,
state: JobState::Pending,
attempt_state: JobAttemptState::Dispatching,
result: None,
failure_category: None,
failure_detail: None,
};
let encoded = serde_json::to_value(&outcome).unwrap();
assert_eq!(
encoded,
json!({
"job_id": "job-1", "input_revision": "7", "attempt_id": "job-1:attempt:2",
"attempt": 2, "state": "pending", "attempt_state": "dispatching",
"result": null, "failure_category": null, "failure_detail": null
})
);
assert_eq!(
serde_json::from_value::<JobOutcome>(encoded.clone()).unwrap(),
outcome
);
let mut with_host_identity = encoded;
with_host_identity["workspace_id"] = json!("workspace-a");
assert!(serde_json::from_value::<JobOutcome>(with_host_identity).is_err());
for (state, attempt_state, result, category, detail) in [
(
JobState::Completed,
JobAttemptState::Completed,
Some(json!({"ok": true})),
None,
None,
),
(
JobState::Failed,
JobAttemptState::Failed,
None,
Some("execution"),
Some("known failure"),
),
(
JobState::Unknown,
JobAttemptState::Unknown,
None,
Some("interrupted"),
Some("requires reconciliation"),
),
(
JobState::Cancelled,
JobAttemptState::Cancelled,
None,
Some("cancelled"),
Some("caller cancelled"),
),
] {
outcome.state = state;
outcome.attempt_state = attempt_state;
outcome.result = result;
outcome.failure_category = category.map(str::to_owned);
outcome.failure_detail = detail.map(str::to_owned);
assert_eq!(
serde_json::from_str::<JobOutcome>(&serde_json::to_string(&outcome).unwrap()).unwrap(),
outcome
);
}
}
#[test]
fn neutral_request_and_result_have_strict_round_trip_contracts() {
let request = request();
let encoded = serde_json::to_value(&request).unwrap();
assert_eq!(encoded.as_object().unwrap().len(), 8);
assert!(encoded.get("serialization_key").is_none());
assert_eq!(
serde_json::from_value::<JobRequest>(encoded.clone()).unwrap(),
request
);
let mut with_grant = encoded.clone();
with_grant["grants"] = json!({});
assert!(serde_json::from_value::<JobRequest>(with_grant).is_err());
let mut with_provenance = encoded;
with_provenance["source_worker"] = json!({});
assert!(serde_json::from_value::<JobRequest>(with_provenance).is_err());
let submission = JobResultSubmission {
job_id: request.job_id,
attempt_id: "check:r7:attempt:1".into(),
input_revision: request.input_revision,
result: json!({"ok": true}),
};
let mut encoded = serde_json::to_value(&submission).unwrap();
assert_eq!(
serde_json::from_value::<JobResultSubmission>(encoded.clone()).unwrap(),
submission
);
encoded["authority"] = json!("profile");
assert!(serde_json::from_value::<JobResultSubmission>(encoded).is_err());
assert_eq!(
serde_json::from_str::<JobLimits>("{}").unwrap(),
JobLimits::default()
);
assert!(serde_json::from_str::<JobLimits>(r#"{"turns":999}"#).is_err());
}
#[test]
fn fingerprint_covers_every_immutable_field_and_preserves_replay() {
let request = request();
let digest = request.fingerprint().unwrap();
let decoded: JobRequest =
serde_json::from_str(&serde_json::to_string(&request).unwrap()).unwrap();
assert_eq!(digest, decoded.fingerprint().unwrap());
assert_eq!(digest, fingerprint(&request).unwrap());
for mutate in [
|r: &mut JobRequest| r.job_id.push('x'),
|r: &mut JobRequest| r.purpose.push('x'),
|r: &mut JobRequest| r.input_revision.push('x'),
|r: &mut JobRequest| r.input_ref.push('x'),
|r: &mut JobRequest| r.input = json!({"title": "changed"}),
|r: &mut JobRequest| r.instruction.push('x'),
|r: &mut JobRequest| r.profile = "builtin:backend-job".into(),
|r: &mut JobRequest| r.serialization_key = Some("resource:7".into()),
|r: &mut JobRequest| r.limits.max_attempts = 3,
] {
let mut changed = request.clone();
mutate(&mut changed);
assert_ne!(digest, changed.fingerprint().unwrap());
}
}
#[test]
fn registry_profiles_are_not_authority_or_a_fixed_allowlist() {
for profile in [
"default",
"custom",
"project:custom",
"user:custom",
"builtin:future-job",
] {
profile_selector(profile).unwrap();
}
for profile in [
"",
" inherit",
"inherit",
"path:custom",
"./a.toml",
"/tmp/a",
"project:../a",
"project:a.toml",
"builtin:",
"unknown:a",
"project:a:b",
"C:\\custom",
"{\"worker\":{}}",
] {
assert!(profile_selector(profile).is_err(), "accepted {profile}");
}
}
#[test]
fn validation_enforces_utf8_bytes_and_metadata_bounds() {
let mut r = request();
r.job_id = "x".repeat(256);
r.input_revision = "x".repeat(256);
r.purpose = "x".repeat(MAX_JOB_PURPOSE_BYTES);
r.input_ref = "x".repeat(MAX_JOB_REFERENCE_BYTES);
r.instruction = "x".repeat(MAX_JOB_INSTRUCTION_BYTES);
r.serialization_key = Some("x".repeat(MAX_JOB_REFERENCE_BYTES));
r.input = json!("x".repeat(MAX_JOB_INPUT_BYTES - 2));
r.validate().unwrap();
for mutate in [
|r: &mut JobRequest| r.job_id.push('x'),
|r: &mut JobRequest| r.input_revision.push('x'),
|r: &mut JobRequest| r.purpose.push('x'),
|r: &mut JobRequest| r.input_ref.push('x'),
|r: &mut JobRequest| r.instruction.push('x'),
|r: &mut JobRequest| r.serialization_key.as_mut().unwrap().push('x'),
|r: &mut JobRequest| r.input = json!("x".repeat(MAX_JOB_INPUT_BYTES - 1)),
] {
let mut changed = r.clone();
mutate(&mut changed);
assert!(changed.validate().is_err());
assert!(changed.fingerprint().is_err());
}
assert!(validate_identifier("id", "a\nb", 256).is_err());
assert!(validate_bounded_text("text", " untrimmed", 256).is_err());
assert!(validate_bounded_text("text", "", 256).is_err());
assert!(validate_bounded_text("text", "界", 2).is_err());
validate_bounded_text("text", "a\nb", 256).unwrap();
}
#[test]
fn all_limit_dimensions_reject_zero_and_above_absolute_bounds() {
let maximum = JobLimits {
max_concurrent_jobs: ABSOLUTE_MAX_CONCURRENT_JOBS,
timeout_seconds: MAX_JOB_TIMEOUT_SECONDS,
max_result_bytes: ABSOLUTE_MAX_RESULT_BYTES,
max_attempts: ABSOLUTE_MAX_ATTEMPTS,
};
maximum.validate().unwrap();
for mutate in [
|l: &mut JobLimits| l.max_concurrent_jobs = 0,
|l: &mut JobLimits| l.max_concurrent_jobs += 1,
|l: &mut JobLimits| l.timeout_seconds = 0,
|l: &mut JobLimits| l.timeout_seconds += 1,
|l: &mut JobLimits| l.max_result_bytes = 0,
|l: &mut JobLimits| l.max_result_bytes += 1,
|l: &mut JobLimits| l.max_attempts = 0,
|l: &mut JobLimits| l.max_attempts += 1,
] {
let mut changed = maximum.clone();
mutate(&mut changed);
assert!(changed.validate().is_err());
}
}
#[test]
fn result_digest_and_failure_detail_bound_exact_utf8_encoding() {
let result = json!({"ok": true});
let (digest, encoded) = result_digest(&result, 11).unwrap();
assert_eq!(encoded, r#"{"ok":true}"#);
assert_eq!(digest, sha256(encoded.as_bytes()));
assert!(result_digest(&result, 10).is_err());
let unicode = json!("界");
assert!(result_digest(&unicode, 4).is_err());
result_digest(&unicode, 5).unwrap();
let detail = "界".repeat(MAX_JOB_FAILURE_BYTES);
let bounded = bounded_failure_detail(&detail);
assert_eq!(bounded.len(), MAX_JOB_FAILURE_BYTES - 1);
assert!(detail.starts_with(&bounded));
assert_eq!(bounded_failure_detail("known failure"), "known failure");
}
#[test]
fn worker_input_is_revision_and_attempt_bound_not_final_prose_authority() {
let request = request();
let attempt = attempt_id(&request.job_id, 2);
assert_eq!(attempt, "check:r7:attempt:2");
let input = request.worker_input(&attempt).unwrap();
assert!(input.starts_with(&request.instruction));
for field in [
"Job envelope (immutable):",
"job_id: check:r7",
"attempt_id: check:r7:attempt:2",
"input_revision: 7",
"input_ref: snapshot://7",
r#"input_json: {"title":"Check me"}"#,
"structured Job result capability",
"Final prose and Worker Idle/Stopped state are not result authority.",
] {
assert!(input.contains(field), "missing {field}");
}
}
+2
View File
@@ -3,8 +3,10 @@ name = "manifest"
version = "0.1.0"
edition.workspace = true
license.workspace = true
rust-version.workspace = true
[dependencies]
builtin-source.workspace = true
arc-swap = "1"
agen = { workspace = true }
decodal.workspace = true
+2
View File
@@ -25,6 +25,8 @@ Does not own:
Profiles are reusable recipes; resolved Manifests are runtime contracts. Keep runtime-bound fields such as Worker name, concrete delegated scope, sockets, session pointers, and raw secrets out of reusable Profiles.
`feature.workdir_catalog.enabled` defaults to `true` for read-only Repository/Workdir/attachment reference discovery in WIP mode. It is independent of `feature.manage_workdir` (which defaults disabled), registers no normal Tools, and does not grant filesystem, attachment, or Workspace authority. Permissions remain independent. Standalone Tools-mode profiles may retain this default; only WIP installation consumes the flag. Set it to `false` to opt out.
## See also
- [`../../docs/design/profiles-manifests-prompts.md`](../../docs/design/profiles-manifests-prompts.md)
+1 -164
View File
@@ -2,172 +2,10 @@ use std::collections::BTreeMap;
use decodal::{Data, Engine, ImportLoader, LoadedImport};
use serde_json::{Map, Number, Value};
use sha2::{Digest, Sha256};
use crate::profile::ProfileError;
pub const BUILTIN_PROFILE_CATALOG_ID: &str = "builtin-profiles-v2";
pub const BUILTIN_DEFAULT_PROFILE: &str = "builtin:default";
pub const BUILTIN_STANDALONE_PROFILE: &str = "builtin:standalone";
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BuiltinProfileImport {
pub specifier: &'static str,
pub resolved_path: &'static str,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BuiltinProfileResource {
pub selector: Option<&'static str>,
pub path: &'static str,
pub source: &'static str,
pub description: &'static str,
pub imports: &'static [BuiltinProfileImport],
}
const BASE_PATH: &str = "profiles/base.dcdl";
const DEFAULT_PATH: &str = "profiles/default.dcdl";
const BASE_IMPORT: &[BuiltinProfileImport] = &[BuiltinProfileImport {
specifier: "./base.dcdl",
resolved_path: BASE_PATH,
}];
const DEFAULT_IMPORT: &[BuiltinProfileImport] = &[BuiltinProfileImport {
specifier: "./default.dcdl",
resolved_path: DEFAULT_PATH,
}];
const NO_IMPORTS: &[BuiltinProfileImport] = &[];
pub const BUILTIN_PROFILE_RESOURCES: &[BuiltinProfileResource] = &[
BuiltinProfileResource {
selector: None,
path: BASE_PATH,
source: include_str!("../../../resources/profiles/base.dcdl"),
description: "Shared built-in Profile defaults.",
imports: NO_IMPORTS,
},
BuiltinProfileResource {
selector: Some(BUILTIN_DEFAULT_PROFILE),
path: DEFAULT_PATH,
source: include_str!("../../../resources/profiles/default.dcdl"),
description: "Default Yoi coding profile.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some(BUILTIN_STANDALONE_PROFILE),
path: "profiles/standalone.dcdl",
source: include_str!("../../../resources/profiles/standalone.dcdl"),
description: "Standalone Yoi coding profile.",
imports: DEFAULT_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:coder"),
path: "profiles/coder.dcdl",
source: include_str!("../../../resources/profiles/coder.dcdl"),
description: "Ticket implementation with direct Reviewer SubWorkers.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:companion"),
path: "profiles/companion.dcdl",
source: include_str!("../../../resources/profiles/companion.dcdl"),
description: "General assistance with Workspace tools.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:intake"),
path: "profiles/intake.dcdl",
source: include_str!("../../../resources/profiles/intake.dcdl"),
description: "Read-only intake and planning.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:reviewer"),
path: "profiles/reviewer.dcdl",
source: include_str!("../../../resources/profiles/reviewer.dcdl"),
description: "Independent review of a published Merge Request source.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:orchestrator"),
path: "profiles/orchestrator.dcdl",
source: include_str!("../../../resources/profiles/orchestrator.dcdl"),
description: "Workspace orchestration and Worker control.",
imports: BASE_IMPORT,
},
BuiltinProfileResource {
selector: Some("builtin:memory-consolidation"),
path: "profiles/memory-consolidation.dcdl",
source: include_str!("../../../resources/profiles/memory-consolidation.dcdl"),
description: "Internal Memory consolidation service.",
imports: BASE_IMPORT,
},
];
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BuiltinProfileCatalogSnapshot {
pub id: &'static str,
pub sources: BTreeMap<String, String>,
pub entrypoints: BTreeMap<String, String>,
pub imports: BTreeMap<String, String>,
}
impl BuiltinProfileCatalogSnapshot {
pub fn digest(&self) -> String {
let mut hasher = Sha256::new();
hasher.update(self.id.as_bytes());
for (path, source) in &self.sources {
hasher.update((path.len() as u64).to_le_bytes());
hasher.update(path.as_bytes());
hasher.update((source.len() as u64).to_le_bytes());
hasher.update(source.as_bytes());
}
for (selector, path) in &self.entrypoints {
hasher.update((selector.len() as u64).to_le_bytes());
hasher.update(selector.as_bytes());
hasher.update((path.len() as u64).to_le_bytes());
hasher.update(path.as_bytes());
}
for (request, resolved_path) in &self.imports {
hasher.update((request.len() as u64).to_le_bytes());
hasher.update(request.as_bytes());
hasher.update((resolved_path.len() as u64).to_le_bytes());
hasher.update(resolved_path.as_bytes());
}
format!("sha256:{:x}", hasher.finalize())
}
}
pub fn builtin_profile_catalog_snapshot() -> BuiltinProfileCatalogSnapshot {
let mut sources = BTreeMap::new();
let mut entrypoints = BTreeMap::new();
let mut imports = BTreeMap::new();
for resource in BUILTIN_PROFILE_RESOURCES {
sources.insert(resource.path.to_owned(), resource.source.to_owned());
for import in resource.imports {
imports.insert(
format!("{}\0{}", resource.path, import.specifier),
import.resolved_path.to_owned(),
);
}
if let Some(selector) = resource.selector {
entrypoints.insert(selector.to_owned(), resource.path.to_owned());
}
}
BuiltinProfileCatalogSnapshot {
id: BUILTIN_PROFILE_CATALOG_ID,
sources,
entrypoints,
imports,
}
}
pub fn builtin_profile_entrypoints() -> impl Iterator<Item = &'static BuiltinProfileResource> {
BUILTIN_PROFILE_RESOURCES
.iter()
.filter(|resource| resource.selector.is_some())
}
pub use builtin_source::*;
pub(crate) fn resolve_builtin_profile_artifact(
selector: &str,
@@ -323,7 +161,6 @@ mod tests {
.expect("evaluate built-in standalone")
.expect("standalone exists");
assert_eq!(value["slug"], "standalone");
assert_eq!(value["model"]["ref"], "codex-oauth/gpt-5.6-sol");
assert_eq!(value["scope"]["intent"], "workspace_write");
assert_eq!(value["scope"]["symlink_policy"], "logical");
assert_eq!(value["delegation_scope"]["intent"], "workspace_write");
+559 -18
View File
@@ -19,9 +19,10 @@ use crate::{
CompactionConfig, EngineManifest, FeatureConfig, FeatureFlagConfig, FileUploadLimits,
McpConfig, McpEnvValue, McpStdioCwdPolicy, MemoryConsolidationProfileConfig,
MemoryExtractionProfileConfig, MemoryFeatureProfileConfig, MemoryResidentProfileConfig,
MergeRequestFeatureConfig, ResolvedMemoryFeatureConfig, ScopeConfig, SessionConfig,
SkillsConfig, TicketFeatureConfig, ToolOutputLimits, ToolPermissionConfig, ToolPermissionRule,
WebConfig, WorkerFeatureConfig, WorkerManifest, WorkerMeta,
MergeRequestFeatureConfig, ResolvedMemoryFeatureConfig, ResolvedSubjektivFeatureConfig,
ScopeConfig, SessionConfig, SkillsConfig, SubjektivFeatureProfileConfig, TicketFeatureConfig,
ToolOutputLimits, ToolPermissionConfig, ToolPermissionRule, WebConfig, WorkerFeatureConfig,
WorkerManifest, WorkerMeta,
};
/// Partial-form Worker manifest. Every field is optional; one or more
@@ -50,8 +51,9 @@ pub struct WorkerManifestConfig {
/// is disabled; `Some` requires `default_action` during final resolve.
#[serde(default)]
pub permissions: Option<PermissionConfigPartial>,
/// Explicit built-in feature/tool-surface enablement. Absent flags resolve
/// disabled after cascade merge.
/// Built-in feature/tool-surface enablement. Absent tool flags resolve
/// disabled after cascade merge; the read-only WIP workdir catalog defaults
/// enabled without granting authority.
#[serde(default)]
pub feature: FeatureConfigPartial,
/// Explicit Model Context Protocol provider declarations. Config parsing
@@ -76,6 +78,8 @@ pub struct FeatureConfigPartial {
#[serde(default)]
pub memory: Option<MemoryFeatureConfigPartial>,
#[serde(default)]
pub subjektiv: Option<SubjektivFeatureConfigPartial>,
#[serde(default)]
pub web: Option<FeatureFlagConfigPartial>,
#[serde(default)]
pub image: Option<FeatureFlagConfigPartial>,
@@ -92,6 +96,12 @@ pub struct FeatureConfigPartial {
#[serde(default)]
pub manage_workdir: Option<FeatureFlagConfigPartial>,
#[serde(default)]
pub workdir_catalog: Option<FeatureFlagConfigPartial>,
#[serde(default)]
pub workspace_config: Option<ActivationFeatureConfigPartial>,
#[serde(default)]
pub drive: Option<ActivationFeatureConfigPartial>,
#[serde(default)]
pub ticket: Option<TicketFeatureConfigPartial>,
#[serde(default)]
pub merge_request: Option<MergeRequestFeatureConfigPartial>,
@@ -104,6 +114,11 @@ impl FeatureConfigPartial {
Self {
task: merge_option(self.task, other.task, FeatureFlagConfigPartial::merge),
memory: merge_option(self.memory, other.memory, MemoryFeatureConfigPartial::merge),
subjektiv: merge_option(
self.subjektiv,
other.subjektiv,
SubjektivFeatureConfigPartial::merge,
),
web: merge_option(self.web, other.web, FeatureFlagConfigPartial::merge),
image: merge_option(self.image, other.image, FeatureFlagConfigPartial::merge),
sub_worker: merge_option(
@@ -128,6 +143,21 @@ impl FeatureConfigPartial {
other.manage_workdir,
FeatureFlagConfigPartial::merge,
),
workdir_catalog: merge_option(
self.workdir_catalog,
other.workdir_catalog,
FeatureFlagConfigPartial::merge,
),
workspace_config: merge_option(
self.workspace_config,
other.workspace_config,
ActivationFeatureConfigPartial::merge,
),
drive: merge_option(
self.drive,
other.drive,
ActivationFeatureConfigPartial::merge,
),
ticket: merge_option(self.ticket, other.ticket, TicketFeatureConfigPartial::merge),
merge_request: merge_option(
self.merge_request,
@@ -157,6 +187,38 @@ impl FeatureFlagConfigPartial {
}
}
/// Activation only: no Workspace/grant/source/permission fields are accepted.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct ActivationFeatureConfigPartial {
#[serde(default)]
pub enabled: Option<bool>,
}
impl ActivationFeatureConfigPartial {
fn merge(self, other: Self) -> Self {
Self {
enabled: other.enabled.or(self.enabled),
}
}
}
impl From<ActivationFeatureConfigPartial> for FeatureFlagConfig {
fn from(value: ActivationFeatureConfigPartial) -> Self {
Self {
enabled: value.enabled.unwrap_or_default(),
}
}
}
impl From<FeatureFlagConfig> for ActivationFeatureConfigPartial {
fn from(value: FeatureFlagConfig) -> Self {
Self {
enabled: Some(value.enabled),
}
}
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct WorkerFeatureConfigPartial {
#[serde(default)]
@@ -227,6 +289,31 @@ pub struct MemoryConsolidationProfileConfigPartial {
pub request_enabled: Option<bool>,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SubjektivFeatureConfigPartial {
#[serde(default)]
pub enabled: Option<bool>,
#[serde(default)]
pub consolidation_tools: Option<bool>,
#[serde(default)]
pub extraction: Option<MemoryExtractionProfileConfigPartial>,
}
impl SubjektivFeatureConfigPartial {
fn merge(self, other: Self) -> Self {
Self {
enabled: other.enabled.or(self.enabled),
consolidation_tools: other.consolidation_tools.or(self.consolidation_tools),
extraction: merge_option(
self.extraction,
other.extraction,
MemoryExtractionProfileConfigPartial::merge,
),
}
}
}
impl MemoryFeatureConfigPartial {
fn merge(self, other: Self) -> Self {
Self {
@@ -331,6 +418,10 @@ impl From<FeatureConfigPartial> for FeatureConfig {
.memory
.map(ResolvedMemoryFeatureConfig::from)
.unwrap_or_default(),
subjektiv: value
.subjektiv
.map(ResolvedSubjektivFeatureConfig::from)
.unwrap_or_default(),
web: value.web.map(FeatureFlagConfig::from).unwrap_or_default(),
image: value.image.map(FeatureFlagConfig::from).unwrap_or_default(),
sub_worker: value
@@ -354,6 +445,17 @@ impl From<FeatureConfigPartial> for FeatureConfig {
.manage_workdir
.map(FeatureFlagConfig::from)
.unwrap_or_default(),
workdir_catalog: FeatureFlagConfig {
enabled: value
.workdir_catalog
.and_then(|flag| flag.enabled)
.unwrap_or(defaults::WORKDIR_CATALOG_ENABLED),
},
workspace_config: value
.workspace_config
.map(FeatureFlagConfig::from)
.unwrap_or_default(),
drive: value.drive.map(FeatureFlagConfig::from).unwrap_or_default(),
ticket: value
.ticket
.map(TicketFeatureConfig::from)
@@ -447,6 +549,44 @@ impl From<ResolvedMemoryFeatureConfig> for MemoryFeatureConfigPartial {
}
}
impl From<SubjektivFeatureConfigPartial> for ResolvedSubjektivFeatureConfig {
fn from(value: SubjektivFeatureConfigPartial) -> Self {
let extraction = value.extraction.unwrap_or_default();
Self {
profile: SubjektivFeatureProfileConfig {
enabled: value.enabled.unwrap_or_default(),
consolidation_tools: value.consolidation_tools.unwrap_or_default(),
extraction: MemoryExtractionProfileConfig {
enabled: extraction.enabled.unwrap_or(true),
model: extraction.model,
reasoning: extraction.reasoning,
threshold: extraction.threshold.or(Some(50_000)),
worker_max_turns: extraction
.worker_max_turns
.or(defaults::MEMORY_EXTRACT_WORKER_MAX_TURNS),
},
},
workspace_settings: None,
}
}
}
impl From<ResolvedSubjektivFeatureConfig> for SubjektivFeatureConfigPartial {
fn from(value: ResolvedSubjektivFeatureConfig) -> Self {
Self {
enabled: Some(value.profile.enabled),
consolidation_tools: Some(value.profile.consolidation_tools),
extraction: Some(MemoryExtractionProfileConfigPartial {
enabled: Some(value.profile.extraction.enabled),
model: value.profile.extraction.model,
reasoning: value.profile.extraction.reasoning,
threshold: value.profile.extraction.threshold,
worker_max_turns: value.profile.extraction.worker_max_turns,
}),
}
}
}
impl From<TicketFeatureConfigPartial> for TicketFeatureConfig {
fn from(value: TicketFeatureConfigPartial) -> Self {
Self {
@@ -500,6 +640,7 @@ impl From<FeatureConfig> for FeatureConfigPartial {
Self {
task: Some(value.task.into()),
memory: Some(value.memory.into()),
subjektiv: Some(value.subjektiv.into()),
web: Some(value.web.into()),
image: Some(value.image.into()),
sub_worker: Some(value.sub_worker.into()),
@@ -508,6 +649,9 @@ impl From<FeatureConfig> for FeatureConfigPartial {
workspace_worker_discovery: Some(value.workspace_worker_discovery.into()),
objective: Some(value.objective.into()),
manage_workdir: Some(value.manage_workdir.into()),
workdir_catalog: Some(value.workdir_catalog.into()),
workspace_config: Some(value.workspace_config.into()),
drive: Some(value.drive.into()),
ticket: Some(value.ticket.into()),
merge_request: Some(value.merge_request.into()),
orchestration: Some(value.orchestration.into()),
@@ -519,6 +663,8 @@ impl From<FeatureConfig> for FeatureConfigPartial {
pub struct WorkerMetaConfig {
#[serde(default)]
pub name: Option<String>,
#[serde(default)]
pub mode: Option<crate::WorkerMode>,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
@@ -693,7 +839,8 @@ impl WorkerManifestConfig {
/// Base config populated with the in-code per-field defaults listed in
/// [`crate::defaults`]. This is not a selectable Profile and does not
/// enable a launch capability surface. Profile and one-file Manifest
/// enable launch authority or normal Tools. The read-only WIP workdir
/// catalog defaults enabled independently. Profile and one-file Manifest
/// resolvers start from this layer so every per-field default lives at
/// exactly one call site (the `defaults` module).
///
@@ -854,6 +1001,7 @@ impl WorkerMetaConfig {
fn merge(self, upper: Self) -> Self {
Self {
name: upper.name.or(self.name),
mode: upper.mode.or(self.mode),
}
}
}
@@ -1285,7 +1433,10 @@ impl TryFrom<WorkerManifestConfig> for WorkerManifest {
validate_mcp_config(&cfg.mcp)?;
Ok(WorkerManifest {
worker: WorkerMeta { name },
worker: WorkerMeta {
name,
mode: cfg.worker.mode.unwrap_or_default(),
},
model: cfg.model,
engine,
scope: cfg.scope,
@@ -1320,6 +1471,7 @@ mod tests {
WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("test".into()),
mode: None,
},
model: ModelManifest {
scheme: Some(SchemeKind::Anthropic),
@@ -1348,35 +1500,78 @@ mod tests {
}
#[test]
fn tool_call_dispatch_defaults_merges_and_round_trips() {
fn tool_call_dispatch_default_merge_and_snapshots() {
assert_eq!(
EngineManifestConfig::default().tool_call_dispatch,
None,
"partial configs must preserve omission until final resolution"
);
let manifest: WorkerManifest = minimal_valid().try_into().unwrap();
assert_eq!(
manifest.engine.tool_call_dispatch,
crate::ToolCallDispatchMode::AfterResponse
crate::ToolCallDispatchMode::OnToolCallComplete
);
let lower = WorkerManifestConfig {
engine: EngineManifestConfig {
tool_call_dispatch: Some(crate::ToolCallDispatchMode::AfterResponse),
..Default::default()
},
..Default::default()
};
let merged = lower.clone().merge(WorkerManifestConfig::default());
assert_eq!(
merged.engine.tool_call_dispatch,
Some(crate::ToolCallDispatchMode::AfterResponse),
"an omitted upper profile must not replace a lower explicit value"
);
let overridden = lower.merge(WorkerManifestConfig {
engine: EngineManifestConfig {
tool_call_dispatch: Some(crate::ToolCallDispatchMode::OnToolCallComplete),
..Default::default()
},
..Default::default()
};
let upper = WorkerManifestConfig::default();
});
assert_eq!(
lower.merge(upper).engine.tool_call_dispatch,
overridden.engine.tool_call_dispatch,
Some(crate::ToolCallDispatchMode::OnToolCallComplete)
);
let mut configured = minimal_valid();
configured.engine.tool_call_dispatch =
let mut explicit_early = minimal_valid();
explicit_early.engine.tool_call_dispatch =
Some(crate::ToolCallDispatchMode::OnToolCallComplete);
let explicit_early: WorkerManifest = explicit_early.try_into().unwrap();
assert_eq!(
explicit_early.engine.tool_call_dispatch,
crate::ToolCallDispatchMode::OnToolCallComplete
);
let mut configured = minimal_valid();
configured.engine.tool_call_dispatch = Some(crate::ToolCallDispatchMode::AfterResponse);
let manifest: WorkerManifest = configured.try_into().unwrap();
let snapshot = serde_json::to_value(&manifest).unwrap();
let restored: WorkerManifest = serde_json::from_value(snapshot).unwrap();
assert_eq!(
manifest.engine.tool_call_dispatch,
crate::ToolCallDispatchMode::AfterResponse
);
let snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
let restored = crate::read_persisted_worker_manifest_snapshot(snapshot.clone()).unwrap();
assert_eq!(
restored.engine.tool_call_dispatch,
crate::ToolCallDispatchMode::OnToolCallComplete
crate::ToolCallDispatchMode::AfterResponse,
"an explicitly persisted after_response value must survive restore"
);
let mut field_missing = snapshot;
field_missing["manifest"]["engine"]
.as_object_mut()
.unwrap()
.remove("tool_call_dispatch");
let restored = crate::read_persisted_worker_manifest_snapshot(field_missing).unwrap();
assert_eq!(
restored.engine.tool_call_dispatch,
crate::ToolCallDispatchMode::OnToolCallComplete,
"snapshots without the field must use the new default"
);
}
@@ -1384,10 +1579,31 @@ mod tests {
fn resolve_minimal_succeeds() {
let manifest: WorkerManifest = minimal_valid().try_into().unwrap();
assert_eq!(manifest.worker.name, "test");
assert_eq!(manifest.worker.mode, crate::WorkerMode::Tools);
assert_eq!(manifest.model.scheme, Some(SchemeKind::Anthropic));
assert!(manifest.permissions.is_none());
}
#[test]
fn worker_mode_is_explicit_opt_in_and_persists_in_snapshots() {
let mut configured = minimal_valid();
configured.worker.mode = Some(crate::WorkerMode::Wip);
let manifest: WorkerManifest = configured.try_into().unwrap();
assert_eq!(manifest.worker.mode, crate::WorkerMode::Wip);
let snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
let restored = crate::read_persisted_worker_manifest_snapshot(snapshot.clone()).unwrap();
assert_eq!(restored.worker.mode, crate::WorkerMode::Wip);
let mut legacy = snapshot;
legacy["manifest"]["worker"]
.as_object_mut()
.unwrap()
.remove("mode");
let restored = crate::read_persisted_worker_manifest_snapshot(legacy).unwrap();
assert_eq!(restored.worker.mode, crate::WorkerMode::Tools);
}
#[test]
fn resolve_mcp_stdio_config_preserves_explicit_policy() {
let mut cfg = minimal_valid();
@@ -1670,6 +1886,7 @@ mod tests {
let lower = WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("lower".into()),
mode: None,
},
model: ModelManifest {
model_id: Some("lower-model".into()),
@@ -1680,6 +1897,7 @@ mod tests {
let upper = WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("upper".into()),
mode: None,
},
..Default::default()
};
@@ -2088,7 +2306,7 @@ worker_max_turns = 7
}
#[test]
fn feature_flags_default_disabled_in_resolved_manifest() {
fn tool_feature_flags_default_disabled_except_read_only_workdir_catalog() {
let manifest: WorkerManifest = minimal_valid().try_into().unwrap();
assert!(!manifest.feature.task.enabled);
assert!(!manifest.feature.memory.profile.enabled);
@@ -2096,10 +2314,300 @@ worker_max_turns = 7
assert!(!manifest.feature.sub_worker.enabled);
assert!(!manifest.feature.objective.enabled);
assert!(!manifest.feature.manage_workdir.enabled);
assert!(manifest.feature.workdir_catalog.enabled);
assert!(!manifest.feature.workspace_config.enabled);
assert!(!manifest.feature.ticket.enabled);
assert!(!manifest.feature.merge_request.any());
}
#[test]
fn drive_defaults_disabled_in_partial_resolved_and_missing_snapshot_fields() {
for source in ["", "[feature]\n", "[feature.drive]\n"] {
let partial = WorkerManifestConfig::from_toml(source).unwrap();
let manifest: WorkerManifest = minimal_valid().merge(partial).try_into().unwrap();
assert!(!manifest.feature.drive.enabled, "{source}");
}
for source in ["{}", r#"{"drive": {}}"#] {
let feature: FeatureConfig = serde_json::from_str(source).unwrap();
assert!(!feature.drive.enabled);
}
let manifest: WorkerManifest = minimal_valid().try_into().unwrap();
let mut snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
snapshot["manifest"]["feature"]
.as_object_mut()
.unwrap()
.remove("drive");
let restored = crate::read_persisted_worker_manifest_snapshot(snapshot).unwrap();
assert!(!restored.feature.drive.enabled);
}
#[test]
fn drive_activation_merges_and_roundtrips_config_and_snapshots() {
for enabled in [false, true] {
let lower =
WorkerManifestConfig::from_toml(&format!("[feature.drive]\nenabled = {enabled}\n"))
.unwrap();
let empty = WorkerManifestConfig::from_toml("[feature.drive]\n").unwrap();
let manifest: WorkerManifest = minimal_valid()
.merge(lower.clone())
.merge(empty)
.try_into()
.unwrap();
assert_eq!(manifest.feature.drive.enabled, enabled);
assert!(!manifest.feature.workspace_config.enabled);
assert!(!manifest.feature.manage_workdir.enabled);
let partial = FeatureConfigPartial::from(manifest.feature.clone());
assert_eq!(partial.drive.as_ref().unwrap().enabled, Some(enabled));
let restored: FeatureConfigPartial =
toml::from_str(&toml::to_string(&partial).unwrap()).unwrap();
assert_eq!(FeatureConfig::from(restored), manifest.feature);
let json = serde_json::to_value(&manifest.feature).unwrap();
assert_eq!(json["drive"], serde_json::json!({"enabled": enabled}));
assert_eq!(
serde_json::from_value::<FeatureConfig>(json).unwrap(),
manifest.feature
);
let snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
let restored = crate::read_persisted_worker_manifest_snapshot(snapshot).unwrap();
assert_eq!(restored.feature.drive.enabled, enabled);
let upper = WorkerManifestConfig::from_toml(&format!(
"[feature.drive]\nenabled = {}\n",
!enabled
))
.unwrap();
let overridden: WorkerManifest = minimal_valid()
.merge(lower)
.merge(upper)
.try_into()
.unwrap();
assert_eq!(overridden.feature.drive.enabled, !enabled);
}
}
#[test]
fn drive_activation_rejects_authority_and_unknown_fields() {
for (field, value) in [
("workspace_id", "\"workspace-1\""),
("grant_id", "\"grant-1\""),
("access", "\"read_write\""),
("host_path", "\"/host/drive\""),
("source", "\"drive\""),
("writable", "true"),
("enabeld", "true"),
] {
let source = format!("[feature.drive]\nenabled = true\n{field} = {value}\n");
assert!(WorkerManifestConfig::from_toml(&source).is_err(), "{field}");
assert!(
toml::from_str::<FeatureConfig>(&format!(
"[drive]\nenabled = true\n{field} = {value}\n"
))
.is_err(),
"{field}"
);
}
}
#[test]
fn workspace_config_defaults_disabled_for_partial_and_resolved_configs() {
assert!(!FeatureConfig::default().workspace_config.enabled);
assert!(FeatureConfigPartial::default().workspace_config.is_none());
for source in ["", "[feature]\n", "[feature.workspace_config]\n"] {
let partial = WorkerManifestConfig::from_toml(source).unwrap();
assert!(
!FeatureConfig::from(partial.feature)
.workspace_config
.enabled
);
}
for source in ["{}", r#"{"workspace_config": {}}"#] {
let feature: FeatureConfig = serde_json::from_str(source).unwrap();
assert!(!feature.workspace_config.enabled);
}
}
#[test]
fn workspace_config_enabled_only_merges_and_roundtrips_snapshots() {
for enabled in [false, true] {
let lower = WorkerManifestConfig::from_toml(&format!(
"[feature.workspace_config]\nenabled = {enabled}\n"
))
.unwrap();
let empty = WorkerManifestConfig::from_toml("[feature.workspace_config]\n").unwrap();
let manifest: WorkerManifest = minimal_valid()
.merge(lower.clone())
.merge(empty)
.try_into()
.unwrap();
assert_eq!(manifest.feature.workspace_config.enabled, enabled);
assert!(!manifest.feature.manage_workdir.enabled);
let partial = FeatureConfigPartial::from(manifest.feature.clone());
assert_eq!(
partial.workspace_config.as_ref().unwrap().enabled,
Some(enabled)
);
let serialized = toml::to_string(&partial).unwrap();
let restored: FeatureConfigPartial = toml::from_str(&serialized).unwrap();
assert_eq!(FeatureConfig::from(restored), manifest.feature);
let json = serde_json::to_value(&manifest.feature).unwrap();
assert_eq!(
json["workspace_config"],
serde_json::json!({"enabled": enabled})
);
assert_eq!(
serde_json::from_value::<FeatureConfig>(json).unwrap(),
manifest.feature
);
let snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
assert_eq!(
crate::read_persisted_worker_manifest_snapshot(snapshot.clone())
.unwrap()
.feature
.workspace_config
.enabled,
enabled
);
let mut old_snapshot = snapshot;
old_snapshot["manifest"]["feature"]
.as_object_mut()
.unwrap()
.remove("workspace_config");
assert!(
!crate::read_persisted_worker_manifest_snapshot(old_snapshot)
.unwrap()
.feature
.workspace_config
.enabled
);
let upper = WorkerManifestConfig::from_toml(&format!(
"[feature.workspace_config]\nenabled = {}\n",
!enabled
))
.unwrap();
let manifest: WorkerManifest = minimal_valid()
.merge(lower)
.merge(upper)
.try_into()
.unwrap();
assert_eq!(manifest.feature.workspace_config.enabled, !enabled);
}
}
#[test]
fn workspace_config_rejects_manifest_authority_fields() {
for (field, value) in [
("grant_id", "\"grant-1\""),
("workspace_id", "\"workspace-1\""),
("access", "\"read_write\""),
("config_root", "\"/host/config\""),
("source", "\"config\""),
] {
let source = format!("[feature.workspace_config]\nenabled = true\n{field} = {value}\n");
assert!(WorkerManifestConfig::from_toml(&source).is_err(), "{field}");
assert!(
toml::from_str::<FeatureConfig>(&format!(
"[workspace_config]\nenabled = true\n{field} = {value}\n"
))
.is_err(),
"{field}"
);
}
}
#[test]
fn workdir_catalog_defaults_agree_for_partial_and_resolved_configs() {
assert!(FeatureConfig::default().workdir_catalog.enabled);
assert!(FeatureConfigPartial::default().workdir_catalog.is_none());
for source in ["", "[feature]\n", "[feature.workdir_catalog]\n"] {
let partial = WorkerManifestConfig::from_toml(source).unwrap();
let feature = FeatureConfig::from(partial.feature);
assert!(feature.workdir_catalog.enabled, "{source}");
let value: toml::Value = toml::from_str(source).unwrap();
let resolved: FeatureConfig = serde_json::from_value(
serde_json::to_value(
value
.get("feature")
.cloned()
.unwrap_or_else(|| toml::Value::Table(Default::default())),
)
.unwrap(),
)
.unwrap();
assert_eq!(feature, resolved);
}
for source in ["{}", r#"{"workdir_catalog": {}}"#] {
let feature: FeatureConfig = serde_json::from_str(source).unwrap();
assert!(feature.workdir_catalog.enabled);
assert!(!feature.manage_workdir.enabled);
}
}
#[test]
fn workdir_catalog_is_independent_and_merges_partial_layers() {
for catalog in [false, true] {
for manage in [false, true] {
let source = format!(
"[feature.workdir_catalog]\nenabled = {catalog}\n\
[feature.manage_workdir]\nenabled = {manage}\n"
);
let lower = WorkerManifestConfig::from_toml(&source).unwrap();
let empty = WorkerManifestConfig::from_toml("[feature.workdir_catalog]").unwrap();
let cfg = minimal_valid().merge(lower.clone()).merge(empty);
let manifest: WorkerManifest = cfg.try_into().unwrap();
assert_eq!(manifest.feature.workdir_catalog.enabled, catalog);
assert_eq!(manifest.feature.manage_workdir.enabled, manage);
let override_source =
format!("[feature.workdir_catalog]\nenabled = {}\n", !catalog);
let upper = WorkerManifestConfig::from_toml(&override_source).unwrap();
let manifest: WorkerManifest = minimal_valid()
.merge(lower)
.merge(upper)
.try_into()
.unwrap();
assert_eq!(manifest.feature.workdir_catalog.enabled, !catalog);
assert_eq!(manifest.feature.manage_workdir.enabled, manage);
}
}
}
#[test]
fn workdir_catalog_roundtrips_config_and_persisted_snapshots() {
for enabled in [false, true] {
let mut cfg = minimal_valid();
cfg.feature.workdir_catalog = Some(FeatureFlagConfigPartial {
enabled: Some(enabled),
});
let manifest: WorkerManifest = cfg.try_into().unwrap();
let partial = FeatureConfigPartial::from(manifest.feature.clone());
assert_eq!(
partial.workdir_catalog.as_ref().unwrap().enabled,
Some(enabled)
);
let serialized = toml::to_string(&partial).unwrap();
let partial: FeatureConfigPartial = toml::from_str(&serialized).unwrap();
assert_eq!(FeatureConfig::from(partial), manifest.feature);
let serialized = serde_json::to_value(&manifest.feature).unwrap();
assert_eq!(serialized["workdir_catalog"]["enabled"], enabled);
let feature: FeatureConfig = serde_json::from_value(serialized).unwrap();
assert_eq!(feature, manifest.feature);
let snapshot = crate::write_persisted_worker_manifest_snapshot(&manifest).unwrap();
let restored =
crate::read_persisted_worker_manifest_snapshot(snapshot.clone()).unwrap();
assert_eq!(restored.feature.workdir_catalog.enabled, enabled);
assert!(!restored.feature.manage_workdir.enabled);
let mut legacy = snapshot;
legacy["manifest"]["feature"]
.as_object_mut()
.unwrap()
.remove("workdir_catalog");
let restored = crate::read_persisted_worker_manifest_snapshot(legacy).unwrap();
assert!(restored.feature.workdir_catalog.enabled);
assert!(!restored.feature.manage_workdir.enabled);
}
}
#[test]
fn from_toml_parses_explicit_feature_flags() {
let cfg = WorkerManifestConfig::from_toml(
@@ -2134,6 +2642,7 @@ enabled = false
.merge(WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("feature-test".into()),
mode: None,
},
model: ModelManifest {
scheme: Some(SchemeKind::Anthropic),
@@ -2237,6 +2746,7 @@ enabled = true
.merge(WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("feature-merge-test".into()),
mode: None,
},
model: ModelManifest {
scheme: Some(SchemeKind::Anthropic),
@@ -2314,6 +2824,7 @@ permission = "write"
let overlay = WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some("x".into()),
mode: None,
},
model: ModelManifest {
scheme: Some(SchemeKind::Anthropic),
@@ -2447,6 +2958,36 @@ directories = [".claude/skills", ".cursor/skills"]
);
}
#[test]
fn subjektiv_config_is_explicit_and_roundtrips_without_subject_identity() {
let config = WorkerManifestConfig::from_toml(
r#"
[worker]
name = "subject-worker"
[feature.subjektiv]
enabled = true
[feature.subjektiv.extraction]
enabled = true
threshold = 321
worker_max_turns = 7
"#,
)
.unwrap();
let partial = config.feature.subjektiv.unwrap();
assert_eq!(partial.enabled, Some(true));
assert_eq!(partial.extraction.as_ref().unwrap().threshold, Some(321));
let resolved = ResolvedSubjektivFeatureConfig::from(partial);
assert!(resolved.profile.enabled);
assert_eq!(resolved.profile.extraction.threshold, Some(321));
assert_eq!(resolved.profile.extraction.worker_max_turns, Some(7));
let encoded = serde_json::to_value(SubjektivFeatureConfigPartial::from(resolved)).unwrap();
assert!(encoded.get("subject_id").is_none());
assert_eq!(encoded["enabled"], true);
}
#[test]
fn merge_preserves_ref() {
let lower = WorkerManifestConfig {
+5 -1
View File
@@ -6,6 +6,10 @@
//! `pub const`. Both paths read from this module, so changing a
//! default requires editing exactly one line.
/// Read-only Repository/Workdir/attachment reference discovery in WIP mode.
/// This enables a reference provider, not normal Tools or any authority grant.
pub const WORKDIR_CATALOG_ENABLED: bool = true;
/// Byte-size cap applied to any tool's `content` output when no
/// per-tool override is set. See [`crate::ToolOutputLimits`].
pub const TOOL_OUTPUT_MAX_BYTES: usize = 64 * 1024;
@@ -88,7 +92,7 @@ pub const COMPACT_SUMMARY_TARGET_TOKENS: u64 = 2_000;
/// See [`crate::CompactionConfig::summary_max_tokens`].
pub const COMPACT_SUMMARY_MAX_TOKENS: u64 = 4_000;
/// Dry-run cap for the compacted session's initial request context.
/// Warning threshold for the assembled compacted history. Compaction continues.
/// See [`crate::CompactionConfig::result_context_max_tokens`].
pub const COMPACT_RESULT_CONTEXT_MAX_TOKENS: u64 = 60_000;
+179 -7
View File
@@ -65,8 +65,9 @@ pub struct WorkerManifest {
/// permission layer is disabled and tool calls run as before.
#[serde(default)]
pub permissions: Option<ToolPermissionConfig>,
/// Explicit built-in feature/tool-surface enablement. Omitted feature flags
/// resolve disabled so Profile authors choose the exposed built-in surfaces.
/// Built-in feature/tool-surface enablement. Omitted tool features resolve
/// disabled; the read-only WIP workdir catalog defaults enabled without
/// granting authority or registering normal Tools.
#[serde(default)]
pub feature: FeatureConfig,
/// Explicit external Model Context Protocol provider configuration. This
@@ -108,6 +109,11 @@ pub struct FeatureConfig {
pub task: FeatureFlagConfig,
#[serde(default)]
pub memory: ResolvedMemoryFeatureConfig,
/// Subject-scoped Memory extraction. Subject identity is resolved by the
/// Workspace host from the Worker's keyed singleton lease, never from this
/// model-visible manifest.
#[serde(default)]
pub subjektiv: ResolvedSubjektivFeatureConfig,
#[serde(default)]
pub web: FeatureFlagConfig,
#[serde(default)]
@@ -126,6 +132,23 @@ pub struct FeatureConfig {
pub objective: FeatureFlagConfig,
#[serde(default)]
pub manage_workdir: FeatureFlagConfig,
/// Read-only Repository/Workdir/attachment reference discovery. Defaults
/// enabled, is installed only in WIP mode, and exposes no normal Tools.
/// Independent of `manage_workdir`; permissions and host authority remain
/// separate, and discovering a reference grants no attachment or file access.
#[serde(
default = "default_workdir_catalog",
deserialize_with = "deserialize_workdir_catalog"
)]
pub workdir_catalog: FeatureFlagConfig,
/// Opt-in WIP-only logical Workspace configuration feature. This flag is
/// not an access grant; Backend-owned Worker grants remain authoritative.
#[serde(default, deserialize_with = "deserialize_activation_feature")]
pub workspace_config: FeatureFlagConfig,
/// Opt-in Drive tools and native WIP surface. Activation grants no Drive
/// access: Backend-owned Worker grants are checked on every operation.
#[serde(default, deserialize_with = "deserialize_activation_feature")]
pub drive: FeatureFlagConfig,
#[serde(default)]
pub ticket: TicketFeatureConfig,
#[serde(default)]
@@ -134,11 +157,35 @@ pub struct FeatureConfig {
pub orchestration: FeatureFlagConfig,
}
fn default_workdir_catalog() -> FeatureFlagConfig {
FeatureFlagConfig {
enabled: defaults::WORKDIR_CATALOG_ENABLED,
}
}
fn deserialize_workdir_catalog<'de, D>(deserializer: D) -> Result<FeatureFlagConfig, D::Error>
where
D: serde::Deserializer<'de>,
{
let flag = config::FeatureFlagConfigPartial::deserialize(deserializer)?;
Ok(FeatureFlagConfig {
enabled: flag.enabled.unwrap_or(defaults::WORKDIR_CATALOG_ENABLED),
})
}
fn deserialize_activation_feature<'de, D>(deserializer: D) -> Result<FeatureFlagConfig, D::Error>
where
D: serde::Deserializer<'de>,
{
Ok(config::ActivationFeatureConfigPartial::deserialize(deserializer)?.into())
}
impl Default for FeatureConfig {
fn default() -> Self {
Self {
task: FeatureFlagConfig::disabled(),
memory: ResolvedMemoryFeatureConfig::default(),
subjektiv: ResolvedSubjektivFeatureConfig::default(),
web: FeatureFlagConfig::disabled(),
image: FeatureFlagConfig::disabled(),
sub_worker: FeatureFlagConfig::disabled(),
@@ -147,6 +194,9 @@ impl Default for FeatureConfig {
workspace_worker_discovery: FeatureFlagConfig::disabled(),
objective: FeatureFlagConfig::disabled(),
manage_workdir: FeatureFlagConfig::disabled(),
workdir_catalog: default_workdir_catalog(),
workspace_config: FeatureFlagConfig::disabled(),
drive: FeatureFlagConfig::disabled(),
ticket: TicketFeatureConfig::default(),
merge_request: MergeRequestFeatureConfig::default(),
orchestration: FeatureFlagConfig::disabled(),
@@ -350,6 +400,88 @@ impl ResolvedMemoryFeatureConfig {
}
}
/// Profile-owned subjektiv activation and extraction policy. The subject id is
/// deliberately absent: the Workspace host binds it from `subjektiv:<id>`
/// singleton ownership for each operation.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(default, deny_unknown_fields)]
pub struct SubjektivFeatureProfileConfig {
pub enabled: bool,
/// Explicit consolidation tool policy; never grants Job or subject authority.
pub consolidation_tools: bool,
pub extraction: MemoryExtractionProfileConfig,
}
impl Default for SubjektivFeatureProfileConfig {
fn default() -> Self {
Self {
enabled: false,
consolidation_tools: false,
extraction: MemoryExtractionProfileConfig::default(),
}
}
}
/// Immutable subjektiv execution configuration persisted in a resolved Worker
/// Manifest. Workspace settings provide only trusted extraction language and
/// are not subject identity.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Default)]
#[serde(default, deny_unknown_fields)]
pub struct ResolvedSubjektivFeatureConfig {
pub profile: SubjektivFeatureProfileConfig,
#[serde(skip_serializing_if = "Option::is_none")]
pub workspace_settings: Option<WorkspaceMemorySettingsSnapshot>,
}
impl ResolvedSubjektivFeatureConfig {
pub fn enabled(&self) -> bool {
self.profile.enabled
}
/// Returns whether the host attached this profile policy to trusted subject
/// authority. Profile enablement without a settings snapshot is inert.
pub fn execution_enabled(&self) -> bool {
self.profile.enabled && self.workspace_settings.is_some()
}
pub fn bind_workspace_settings(
&mut self,
settings: WorkspaceMemorySettingsSnapshot,
) -> Result<(), &'static str> {
if !self.profile.enabled {
if self.workspace_settings.is_some() {
return Err("disabled subjektiv feature must not carry Workspace settings");
}
return Ok(());
}
if self.workspace_settings.is_some() {
return Err("subjektiv Workspace settings are already bound");
}
self.workspace_settings = Some(settings);
Ok(())
}
pub fn workspace_settings(&self) -> Option<WorkspaceMemorySettingsSnapshot> {
self.workspace_settings.clone()
}
pub fn validate_execution(&self) -> Result<(), &'static str> {
// `profile.enabled` is reusable policy. Only a host-bound settings
// snapshot activates subject execution; an ordinary Worker or Internal
// SubWorker with the same Profile remains inert.
if !self.profile.enabled && self.workspace_settings.is_some() {
return Err("disabled subjektiv feature must not carry Workspace settings");
}
if let Some(settings) = &self.workspace_settings
&& (settings.settings_revision == 0
|| !is_normalized_workspace_memory_language(&settings.language))
{
return Err("subjektiv Workspace settings snapshot metadata is invalid");
}
Ok(())
}
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, Default)]
pub struct TicketFeatureConfig {
#[serde(default)]
@@ -585,6 +717,21 @@ pub struct WorkspaceMemorySettingsSnapshot {
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WorkerMeta {
pub name: String,
/// Model-facing capability surface. Existing manifests default to ordinary tools.
#[serde(default)]
pub mode: WorkerMode,
}
/// Model-facing Worker capability surface.
///
/// WIP is explicit opt-in: it replaces individual model-visible tools with the
/// discover/inspect/call Worldspace client while retaining the same host authority.
#[derive(Debug, Clone, Copy, Default, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum WorkerMode {
#[default]
Tools,
Wip,
}
/// When the engine may begin executing provider-confirmed tool calls.
@@ -594,10 +741,10 @@ pub struct WorkerMeta {
#[derive(Debug, Clone, Copy, Default, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum ToolCallDispatchMode {
/// Preserve the historical behavior: wait for the whole response stream.
#[default]
/// Wait for the whole response stream before starting any tools.
AfterResponse,
/// Start each call once the provider emits a trustworthy per-call completion.
#[default]
OnToolCallComplete,
}
@@ -628,7 +775,7 @@ pub struct EngineManifest {
pub stop_sequences: Vec<String>,
#[serde(default)]
pub reasoning: Option<ReasoningControl>,
/// Tool-call start timing. Omitted manifests preserve `after_response`.
/// Tool-call start timing. Omitted manifests use `on_tool_call_complete`.
#[serde(default)]
pub tool_call_dispatch: ToolCallDispatchMode,
/// Byte-size caps applied to tool `content` before it reaches the
@@ -880,7 +1027,9 @@ pub struct CompactionConfig {
)]
pub auto_read_budget_tokens: u64,
/// Dry-run cap for the compacted session's initial request context.
/// Warning threshold for the assembled compacted history estimate, including
/// retained items and auto-read files. Exceeding it does not abort compaction;
/// request-time context safety checks still apply. `0` disables this warning.
#[serde(default = "default_result_context_max_tokens")]
pub result_context_max_tokens: u64,
@@ -963,7 +1112,9 @@ impl Default for CompactionConfig {
impl WorkerManifest {
pub fn requires_persisted_execution_snapshot(&self) -> bool {
self.profile.is_some() || self.feature.memory.workspace_settings.is_some()
self.profile.is_some()
|| self.feature.memory.workspace_settings.is_some()
|| self.feature.subjektiv.workspace_settings.is_some()
}
/// Parse a manifest from a TOML string.
@@ -1638,6 +1789,27 @@ model_id = "claude-sonnet-4-20250514"
assert!(disabled.bind_workspace_settings(snapshot).is_err());
}
#[test]
fn resolved_subjektiv_policy_is_inert_until_trusted_attachment() {
let snapshot = WorkspaceMemorySettingsSnapshot {
workspace_id: "workspace-1".to_string(),
settings_revision: 1,
language: "English".to_string(),
};
let mut enabled = ResolvedSubjektivFeatureConfig::default();
enabled.profile.enabled = true;
assert!(enabled.validate_execution().is_ok());
assert!(!enabled.execution_enabled());
enabled.bind_workspace_settings(snapshot.clone()).unwrap();
assert!(enabled.validate_execution().is_ok());
assert!(enabled.execution_enabled());
let mut disabled = ResolvedSubjektivFeatureConfig::default();
disabled.workspace_settings = Some(snapshot.clone());
assert!(disabled.validate_execution().is_err());
assert!(disabled.bind_workspace_settings(snapshot).is_err());
}
#[test]
fn current_manifest_rejects_legacy_top_level_memory_authority() {
let toml = format!("{MINIMAL_REQUIRED}\n[memory]\nlanguage = \"Japanese\"\n");
+25 -2
View File
@@ -570,10 +570,16 @@ mod tests {
}
#[test]
fn builtin_codex_catalog_includes_gpt_6_sol_and_luna() {
fn builtin_codex_catalog_replaces_gpt_6_sol_with_6_1_and_keeps_astra_and_luna() {
let models = load_builtin_models().unwrap();
let providers = load_builtin_providers().unwrap();
assert!(
!models
.iter()
.any(|model| model.provider == "codex-oauth" && model.id == "gpt-6-sol")
);
for model_id in ["gpt-6-sol", "gpt-6-luna"] {
for model_id in ["gpt-6.1-sol", "gpt-6-astra", "gpt-6-luna"] {
let model = models
.iter()
.find(|model| model.provider == "codex-oauth" && model.id == model_id)
@@ -581,6 +587,23 @@ mod tests {
assert_eq!(model.context_window, Some(1_050_000));
assert_eq!(model.max_context_window, Some(272_000));
assert!(model.capability.is_some());
let config = resolve_with_catalogs(
&ModelManifest {
ref_: Some(format!("codex-oauth/{model_id}")),
context_window: Some(1_050_000),
..Default::default()
},
&providers,
&models,
)
.unwrap();
assert_eq!(config.model_id, model_id);
assert_eq!(config.scheme, SchemeKind::OpenaiResponses);
assert_eq!(config.auth, AuthRef::CodexOAuth);
assert_eq!(config.context_window, 272_000);
assert_eq!(config.max_context_window, Some(272_000));
assert_eq!(config.capability, model.capability);
}
}
+287 -65
View File
@@ -144,6 +144,7 @@ pub enum WorkspaceAuthorityRequirement {
Flow,
ManageWorkdir,
Memory,
Subjektiv,
MergeRequest,
Objective,
Orchestration,
@@ -157,6 +158,7 @@ impl fmt::Display for WorkspaceAuthorityRequirement {
Self::Flow => formatter.write_str("feature.flow"),
Self::ManageWorkdir => formatter.write_str("feature.manage_workdir"),
Self::Memory => formatter.write_str("feature.memory"),
Self::Subjektiv => formatter.write_str("feature.subjektiv"),
Self::MergeRequest => formatter.write_str("feature.merge_request"),
Self::Objective => formatter.write_str("feature.objective"),
Self::Orchestration => formatter.write_str("feature.orchestration"),
@@ -179,12 +181,19 @@ pub fn validate_profile_execution_target(
if feature.flow.enabled {
requirements.insert(WorkspaceAuthorityRequirement::Flow);
}
// workdir_catalog is a WIP-only reference provider, not Workspace authority
// or normal Tools. It must not prevent standalone Tools-mode launches.
if feature.manage_workdir.enabled {
requirements.insert(WorkspaceAuthorityRequirement::ManageWorkdir);
}
if feature.memory.profile.enabled || feature.memory.profile.staging_tools {
requirements.insert(WorkspaceAuthorityRequirement::Memory);
}
// Subject policy is Host-neutral. Only an already bound Backend snapshot
// requires Workspace authority; a local Host supplies its own live capability.
if feature.subjektiv.workspace_settings.is_some() {
requirements.insert(WorkspaceAuthorityRequirement::Subjektiv);
}
if feature.merge_request.show
|| feature.merge_request.open
|| feature.merge_request.review
@@ -621,6 +630,7 @@ fn resolve_profile_value(
let config = WorkerManifestConfig {
worker: WorkerMetaConfig {
name: Some(worker_name),
mode: profile.worker.and_then(|worker| worker.mode),
},
model: profile.model.unwrap_or_default(),
engine: profile.engine.unwrap_or_default(),
@@ -663,6 +673,8 @@ struct ProfileConfig {
#[serde(default)]
description: Option<String>,
#[serde(default)]
worker: Option<ProfileWorkerConfig>,
#[serde(default)]
model: Option<ModelManifest>,
#[serde(default)]
engine: Option<EngineManifestConfig>,
@@ -686,6 +698,13 @@ struct ProfileConfig {
skills: Option<SkillsConfig>,
}
#[derive(Debug, Default, Deserialize)]
#[serde(deny_unknown_fields)]
struct ProfileWorkerConfig {
#[serde(default)]
mode: Option<crate::WorkerMode>,
}
#[derive(Debug, Clone, Deserialize)]
#[serde(untagged)]
enum ProfileScopeConfig {
@@ -859,8 +878,15 @@ fn reject_manifest_shaped_profile(value: &serde_json::Value) -> Result<(), Profi
)));
}
}
if map.contains_key("worker") {
return Err(ProfileError::InvalidProfile("field `worker` is runtime-bound and is not allowed in reusable Profiles; pass the Worker name via CLI/TUI runtime inputs".into()));
if let Some(worker) = map.get("worker") {
let Some(worker) = worker.as_object() else {
return Err(ProfileError::InvalidProfile(
"field `worker` must be an object containing only reusable `mode` policy".into(),
));
};
if worker.contains_key("name") {
return Err(ProfileError::InvalidProfile("field `worker.name` is runtime-bound and is not allowed in reusable Profiles; pass the Worker name via CLI/TUI runtime inputs".into()));
}
}
if let Some(scope) = map.get("scope").and_then(|v| v.as_object()) {
for key in ["allow", "deny"] {
@@ -1233,13 +1259,128 @@ mod tests {
)
.unwrap();
assert_eq!(resolved.manifest.worker.name, "role-worker");
if matches!(
entry.name.as_str(),
"job" | "standalone-subjektiv-consolidation"
) {
assert!(resolved.manifest.compaction.is_none());
assert!(resolved.manifest.scope.allow.is_empty());
assert!(resolved.manifest.delegation_scope.allow.is_empty());
assert_eq!(
resolved.manifest.engine.instruction,
if entry.name == "job" {
"internal.job_system"
} else {
"internal.subjektiv_memory_consolidation_system"
}
);
continue;
}
}
}
#[test]
fn builtin_profiles_pin_role_models_and_reasoning() {
use crate::model::{ReasoningControl, ReasoningEffort};
fn builtin_drive_activation_is_workspace_role_specific_and_excludes_jobs() {
// Production DCDL imports are the policy boundary: changing the shared
// base must not accidentally activate Drive in a read-only Job/Reviewer.
let tmp = TempDir::new().unwrap();
let registry = ProfileDiscovery::with_sources(None, None)
.discover()
.unwrap();
for entry in registry.entries() {
let resolved = ProfileResolver::new()
.with_workspace_base(tmp.path())
.resolve(
&ProfileSelector::source_named(ProfileRegistrySource::Builtin, &entry.name),
ProfileResolveOptions::with_worker_name("drive-profile-worker"),
)
.unwrap();
let expected = matches!(
entry.name.as_str(),
"default" | "ticket-worker" | "coder" | "companion" | "intake" | "orchestrator"
);
assert_eq!(
resolved.manifest.feature.drive.enabled, expected,
"{}",
entry.name
);
}
}
#[test]
fn ticket_worker_profile_enables_work_reporting_without_authoring_or_integration() {
let tmp = TempDir::new().unwrap();
let registry = ProfileDiscovery::with_sources(None, None)
.discover()
.unwrap();
let selector = ProfileSelector::parse_cli("builtin:ticket-worker");
assert_eq!(
registry.select(&selector).unwrap().qualified_name(),
"builtin:ticket-worker"
);
let manifest = ProfileResolver::new()
.with_workspace_base(tmp.path())
.resolve(
&selector,
ProfileResolveOptions::with_worker_name("ticket-worker"),
)
.unwrap()
.manifest;
assert_eq!(manifest.engine.instruction, "role.ticket_worker");
// enabled supplies ShowTicket; thread supplies TicketComment,
// TicketWorkflowState and CompleteTicket without broad workflow access.
let feature = &manifest.feature;
assert!(feature.ticket.enabled);
assert!(feature.ticket.thread);
assert!(!feature.ticket.authoring);
assert!(!feature.ticket.intake);
assert!(!feature.ticket.workflow);
assert!(!feature.merge_request.show);
assert!(!feature.merge_request.open);
assert!(!feature.merge_request.review);
assert!(!feature.merge_request.readiness_check);
assert!(!feature.merge_request.complete);
assert!(!feature.orchestration.enabled);
}
#[test]
fn ticket_worker_profile_inherits_default_without_flow_or_workdir_management() {
let tmp = TempDir::new().unwrap();
let resolve = |selector| {
ProfileResolver::new()
.with_workspace_base(tmp.path())
.resolve(
&ProfileSelector::parse_cli(selector),
ProfileResolveOptions::with_worker_name("ticket-worker"),
)
.unwrap()
.manifest
};
let default = resolve("builtin:default");
let worker = resolve("builtin:ticket-worker");
assert_eq!(worker.model, default.model);
assert_eq!(worker.scope.allow, default.scope.allow);
assert_eq!(worker.scope.deny, default.scope.deny);
assert_eq!(
worker.delegation_scope.allow,
default.delegation_scope.allow
);
assert_eq!(worker.delegation_scope.deny, default.delegation_scope.deny);
assert!(worker.feature.task.enabled);
assert!(worker.feature.web.enabled);
assert!(worker.feature.image.enabled);
assert!(worker.feature.sub_worker.enabled);
assert!(!worker.feature.flow.enabled);
assert!(!worker.feature.manage_workdir.enabled);
assert!(!worker.feature.worker.enabled);
assert!(!worker.feature.memory.profile.enabled);
assert!(!worker.feature.subjektiv.profile.enabled);
}
#[test]
fn builtin_workspace_profiles_cut_over_to_subjektiv_without_legacy_memory() {
let tmp = TempDir::new().unwrap();
let resolve = |name: &str| {
ProfileResolver::new()
@@ -1249,71 +1390,97 @@ mod tests {
ProfileResolveOptions::with_worker_name(format!("{name}-worker")),
)
.unwrap()
.manifest
};
for name in [
"default",
"standalone",
"intake",
"orchestrator",
"companion",
"coder",
"reviewer",
"memory-consolidation",
] {
let role = resolve(name);
assert_eq!(
role.manifest
.compaction
.as_ref()
.map(|compaction| compaction.prune_enabled),
Some(false),
"{name} must keep pruning disabled for the experiment"
for name in ["coder", "companion", "intake", "reviewer", "orchestrator"] {
let manifest = resolve(name);
assert!(
manifest.feature.subjektiv.profile.enabled,
"{name} must enable subjektiv"
);
assert!(
manifest.feature.subjektiv.profile.extraction.enabled,
"{name} must enable subject extraction"
);
assert!(
!manifest.feature.memory.profile.enabled,
"{name} must not enable legacy Workspace Memory"
);
}
let companion = resolve("companion");
assert_eq!(
companion.manifest.model.ref_.as_deref(),
Some("codex-oauth/gpt-6-astra")
);
assert_eq!(
companion.manifest.engine.reasoning,
Some(ReasoningControl::Effort(ReasoningEffort::High))
);
for name in ["coder", "reviewer"] {
let role = resolve(name);
assert_eq!(
role.manifest.model.ref_.as_deref(),
Some("codex-oauth/gpt-5.6-sol")
);
assert_eq!(
role.manifest.engine.reasoning,
Some(ReasoningControl::Effort(ReasoningEffort::High))
);
let extraction = &role.manifest.feature.memory.profile.extraction;
assert_eq!(
extraction
.model
.as_ref()
.and_then(|model| model.ref_.as_deref()),
Some("codex-oauth/gpt-5.6-luna")
);
assert_eq!(
extraction.reasoning,
Some(ReasoningControl::Effort(ReasoningEffort::Medium))
);
for name in ["default", "standalone", "backend-job"] {
let manifest = resolve(name);
assert!(!manifest.feature.memory.profile.enabled, "{name}");
assert!(!manifest.feature.subjektiv.profile.enabled, "{name}");
}
let consolidation = resolve("memory-consolidation");
assert_eq!(
consolidation.manifest.model.ref_.as_deref(),
Some("codex-oauth/gpt-5.6-luna")
let legacy_consolidator = resolve("memory-consolidation");
assert!(legacy_consolidator.feature.memory.profile.enabled);
assert!(legacy_consolidator.feature.memory.profile.staging_tools);
assert!(
legacy_consolidator
.feature
.memory
.profile
.extraction
.enabled
);
assert!(
legacy_consolidator
.feature
.memory
.profile
.consolidation
.request_enabled
);
assert!(!legacy_consolidator.feature.subjektiv.profile.enabled);
}
#[test]
fn subjektiv_memory_consolidation_profile_is_lifecycle_and_capability_isolated() {
let tmp = TempDir::new().unwrap();
let resolved = ProfileResolver::new()
.with_workspace_base(tmp.path())
.resolve(
&ProfileSelector::source_named(
ProfileRegistrySource::Builtin,
"subjektiv-memory-consolidation",
),
ProfileResolveOptions::with_worker_name("subject-consolidator"),
)
.unwrap();
let manifest = resolved.manifest;
let feature = &manifest.feature;
assert!(!feature.memory.profile.enabled);
assert!(!feature.memory.profile.staging_tools);
assert!(feature.subjektiv.profile.enabled);
assert!(!feature.subjektiv.profile.extraction.enabled);
assert!(!feature.task.enabled);
assert!(!feature.web.enabled);
assert!(!feature.image.enabled);
assert!(!feature.sub_worker.enabled);
assert!(!feature.flow.enabled);
assert!(!feature.worker.enabled);
assert!(!feature.worker.direct_spawn);
assert!(!feature.workspace_worker_discovery.enabled);
assert!(!feature.objective.enabled);
assert!(!feature.manage_workdir.enabled);
assert!(!feature.ticket.enabled);
assert!(!feature.ticket.authoring);
assert!(!feature.ticket.thread);
assert!(!feature.ticket.intake);
assert!(!feature.ticket.workflow);
assert!(!feature.merge_request.show);
assert!(!feature.merge_request.open);
assert!(!feature.merge_request.review);
assert!(!feature.merge_request.readiness_check);
assert!(!feature.merge_request.complete);
assert!(!feature.orchestration.enabled);
assert_eq!(
consolidation.manifest.engine.reasoning,
Some(ReasoningControl::Effort(ReasoningEffort::Medium))
manifest.web.as_ref().and_then(|web| web.enabled),
Some(false)
);
}
@@ -1414,10 +1581,6 @@ mod tests {
..
} if name == "standalone" && provenance.starts_with("profiles/standalone.dcdl#sha256:")
));
assert_eq!(
resolved.manifest.model.ref_.as_deref(),
Some("codex-oauth/gpt-5.6-sol")
);
assert!(resolved.manifest.feature.task.enabled);
assert!(resolved.manifest.feature.web.enabled);
assert!(resolved.manifest.feature.image.enabled);
@@ -1440,6 +1603,59 @@ mod tests {
assert!(!resolved.manifest.feature.manage_workdir.enabled);
}
#[test]
fn workdir_catalog_does_not_require_workspace_authority_or_grant_scope() {
let tmp = TempDir::new().unwrap();
for catalog in [None, Some(false), Some(true)] {
let mut artifact = serde_json::json!({
"worker": { "mode": "tools" },
"feature": { "manage_workdir": { "enabled": false } },
"permissions": { "default_action": "deny" }
});
if let Some(enabled) = catalog {
artifact["feature"]["workdir_catalog"] = serde_json::json!({ "enabled": enabled });
}
let resolved = resolve_profile_artifact_value(
artifact,
ProfileSource::Archive {
archive_id: "catalog-test".into(),
source: "test".into(),
},
tmp.path(),
"catalog-worker",
)
.unwrap();
let manifest = resolved.manifest;
assert_eq!(
manifest.feature.workdir_catalog.enabled,
catalog.unwrap_or(true)
);
assert!(!manifest.feature.manage_workdir.enabled);
assert_eq!(manifest.worker.mode, crate::WorkerMode::Tools);
assert!(manifest.scope.allow.is_empty());
assert!(manifest.delegation_scope.allow.is_empty());
assert_eq!(
manifest.permissions.as_ref().unwrap().default_action,
crate::ToolPermissionAction::Deny
);
validate_profile_execution_target(&manifest, ProfileExecutionTarget::Standalone)
.unwrap();
let mut manage_manifest = manifest;
manage_manifest.feature.manage_workdir.enabled = true;
let error = validate_profile_execution_target(
&manage_manifest,
ProfileExecutionTarget::Standalone,
)
.unwrap_err();
assert!(matches!(
error,
ProfileError::UnsupportedExecutionTarget { requirements, .. }
if requirements == vec![WorkspaceAuthorityRequirement::ManageWorkdir]
));
}
}
#[test]
fn standalone_rejects_profiles_that_require_workspace_authority() {
let tmp = TempDir::new().unwrap();
@@ -1461,7 +1677,11 @@ mod tests {
panic!("unexpected error: {error}");
};
assert_eq!(target, ProfileExecutionTarget::Standalone);
assert!(requirements.contains(&WorkspaceAuthorityRequirement::Memory));
assert!(
!requirements.contains(&WorkspaceAuthorityRequirement::Subjektiv),
"subject policy alone is no longer Backend authority"
);
assert!(!requirements.contains(&WorkspaceAuthorityRequirement::Memory));
assert!(requirements.contains(&WorkspaceAuthorityRequirement::MergeRequest));
assert!(requirements.contains(&WorkspaceAuthorityRequirement::Ticket));
assert!(!diagnostic.contains(tmp.path().to_string_lossy().as_ref()));
@@ -1561,6 +1781,7 @@ mod tests {
r#"
slug = "coder"
scope = "workspace_read"
worker = { mode = "wip" }
[model]
scheme = "anthropic"
@@ -1580,6 +1801,7 @@ reasoning = "high"
)
.unwrap();
assert_eq!(resolved.manifest.worker.name, "runtime-worker");
assert_eq!(resolved.manifest.worker.mode, crate::WorkerMode::Wip);
assert_eq!(resolved.manifest.model.scheme, Some(SchemeKind::Anthropic));
assert_eq!(
resolved.manifest.engine.reasoning,
+18
View File
@@ -41,6 +41,7 @@ pub enum MemoryBackendHttpResponse {
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum MemoryBackendOperationResult {
ToolOutput(MemoryToolOutput),
ResidentSummary(MemoryResidentSummaryOutput),
Acknowledged(MemoryBackendAckOutput),
StagingWritten(MemoryStagingWriteOutput),
}
@@ -52,6 +53,23 @@ pub struct MemoryToolOutput {
pub content: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum MemoryResidentSummaryAvailability {
Ready,
Ungenerated,
Stale,
Failed,
Unavailable,
}
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct MemoryResidentSummaryOutput {
pub availability: MemoryResidentSummaryAvailability,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub content: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct MemoryQueryOperation {
#[serde(default, skip_serializing_if = "Option::is_none")]
+237 -288
View File
@@ -236,10 +236,244 @@ pub struct MergeRequest {
#[serde(default)]
pub thread: Vec<MergeRequestThreadEvent>,
}
/// Bounded failures from pure evaluation of persisted review and merge evidence.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum MergeRequestEvidenceError {
NotMerged,
MergeResultMissing,
ApprovalMissing,
ApprovalRevoked,
ApprovalNotApproved,
ApprovalSourceMismatch,
ApprovalTimelineMismatch,
ReviewRequestMissing,
ReviewRequestMismatch,
ApprovalNotEffective,
ItemRevisionMismatch,
SourceSnapshotMismatch,
RequirementApprovalMissing,
}
impl MergeRequestEvidenceError {
/// Stable machine-readable diagnostic code, separate from the human message.
pub fn code(self) -> &'static str {
match self {
Self::NotMerged => "not_merged",
Self::MergeResultMissing => "merge_result_missing",
Self::ApprovalMissing => "approval_missing",
Self::ApprovalRevoked => "approval_revoked",
Self::ApprovalNotApproved => "approval_not_approved",
Self::ApprovalSourceMismatch => "approval_source_mismatch",
Self::ApprovalTimelineMismatch => "approval_timeline_mismatch",
Self::ReviewRequestMissing => "review_request_missing",
Self::ReviewRequestMismatch => "review_request_mismatch",
Self::ApprovalNotEffective => "approval_not_effective",
Self::ItemRevisionMismatch => "item_revision_mismatch",
Self::SourceSnapshotMismatch => "source_snapshot_mismatch",
Self::RequirementApprovalMissing => "requirement_approval_missing",
}
}
pub fn as_str(self) -> &'static str {
match self {
Self::NotMerged => "Merge Request has no merged result",
Self::MergeResultMissing => "merged Merge Request has no MergeResult",
Self::ApprovalMissing => "integration approval is missing",
Self::ApprovalRevoked => "Reviewer approval is revoked",
Self::ApprovalNotApproved => "Reviewer decision is not an approval",
Self::ApprovalSourceMismatch => "Reviewer approval does not match the exact source ref",
Self::ApprovalTimelineMismatch => {
"integration approval does not predate the merged result"
}
Self::ReviewRequestMissing => "linked ReviewRequested evidence is missing",
Self::ReviewRequestMismatch => {
"Reviewer approval does not match linked ReviewRequested evidence"
}
Self::ApprovalNotEffective => "Reviewer approval is not the effective review",
Self::ItemRevisionMismatch => {
"Reviewer approval does not attest the current Ticket revision"
}
Self::SourceSnapshotMismatch => {
"Reviewer approval does not attest the exact linked source set"
}
Self::RequirementApprovalMissing => {
"no effective Reviewer approval attests the current Ticket revision and exact linked source set"
}
}
}
}
impl std::fmt::Display for MergeRequestEvidenceError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str(self.as_str())
}
}
impl std::error::Error for MergeRequestEvidenceError {}
impl MergeRequest {
pub fn effective_review(&self, subject: &str) -> Option<&ReviewEvent> {
self.thread.iter().rev().find_map(|e|match e{MergeRequestThreadEvent::Review(v)if v.subject_ref==subject&&!self.thread.iter().any(|x|matches!(x,MergeRequestThreadEvent::ReviewRevoked(r)if r.review_event_id==v.event_id))=>Some(v),_=>None})
}
/// Returns the recorded merged result, without resolving selectors or mutating state.
pub fn merged_result(&self) -> Result<&MergeEvent, MergeRequestEvidenceError> {
if self.state != MergeRequestState::Merged {
return Err(MergeRequestEvidenceError::NotMerged);
}
self.thread
.iter()
.rev()
.find_map(|event| match event {
MergeRequestThreadEvent::Merge(event) => Some(event),
_ => None,
})
.ok_or(MergeRequestEvidenceError::MergeResultMissing)
}
/// Validates the specific approval recorded by the merged result. A later
/// requirement review does not replace this integration evidence.
pub fn integration_approval(&self) -> Result<&ReviewEvent, MergeRequestEvidenceError> {
let merge = self.merged_result()?;
let review = self
.thread
.iter()
.find_map(|event| match event {
MergeRequestThreadEvent::Review(review)
if review.event_id == merge.approval_event_id =>
{
Some(review)
}
_ => None,
})
.ok_or(MergeRequestEvidenceError::ApprovalMissing)?;
if merge.approved_source_ref.trim().is_empty()
|| review.subject_ref != merge.approved_source_ref
{
return Err(MergeRequestEvidenceError::ApprovalSourceMismatch);
}
if review.sequence >= merge.sequence {
return Err(MergeRequestEvidenceError::ApprovalTimelineMismatch);
}
self.approval_evidence(review)?;
Ok(review)
}
fn approval_evidence(&self, review: &ReviewEvent) -> Result<(), MergeRequestEvidenceError> {
if self.thread.iter().any(|event| {
matches!(event, MergeRequestThreadEvent::ReviewRevoked(revoked)
if revoked.review_event_id == review.event_id)
}) {
return Err(MergeRequestEvidenceError::ApprovalRevoked);
}
if review.decision != ReviewDecision::Approve {
return Err(MergeRequestEvidenceError::ApprovalNotApproved);
}
let requested = self
.thread
.iter()
.find_map(|event| match event {
MergeRequestThreadEvent::ReviewRequested(requested)
if requested.event_id == review.request_event_id =>
{
Some(requested)
}
_ => None,
})
.ok_or(MergeRequestEvidenceError::ReviewRequestMissing)?;
if requested.subject_ref != review.subject_ref
|| requested.ticket_item_revision != review.ticket_item_revision
|| requested.ticket_merge_request_subjects != review.ticket_merge_request_subjects
|| requested.reviewer != review.reviewer
|| requested.sequence >= review.sequence
{
return Err(MergeRequestEvidenceError::ReviewRequestMismatch);
}
Ok(())
}
}
/// Finds an effective approval of the authoritative current Ticket revision and
/// exact linked source snapshot. Callers supply all linked requests and resolved
/// source refs (recorded merged refs for completion); this function does not read
/// authority, resolve selectors, require merged state, or mutate lifecycle state.
/// An explicit event id validates that approval only; `None` discovers any valid
/// attestation. Snapshot ordering is immaterial, but duplicate ids are rejected.
pub fn requirement_approval<'a>(
requests: &'a [MergeRequest],
item_revision: &str,
subjects: &[MergeRequestReviewSubject],
event_id: Option<&str>,
) -> Result<&'a ReviewEvent, MergeRequestEvidenceError> {
if requests.len() != subjects.len()
|| !same_source_snapshot(subjects, subjects)
|| requests.iter().any(|request| {
requests
.iter()
.filter(|other| other.merge_request_id == request.merge_request_id)
.count()
!= 1
|| !subjects
.iter()
.any(|subject| subject.merge_request_id == request.merge_request_id)
})
{
return Err(MergeRequestEvidenceError::SourceSnapshotMismatch);
}
for request in requests {
for event in request.thread.iter().rev() {
let MergeRequestThreadEvent::Review(review) = event else {
continue;
};
if event_id.is_some_and(|id| id != review.event_id) {
continue;
}
let evidence = (|| {
request.approval_evidence(review)?;
if !request
.effective_review(&review.subject_ref)
.is_some_and(|effective| effective.event_id == review.event_id)
{
return Err(MergeRequestEvidenceError::ApprovalNotEffective);
}
if review.ticket_item_revision != item_revision || item_revision.trim().is_empty() {
return Err(MergeRequestEvidenceError::ItemRevisionMismatch);
}
if !same_source_snapshot(&review.ticket_merge_request_subjects, subjects) {
return Err(MergeRequestEvidenceError::SourceSnapshotMismatch);
}
if !subjects.iter().any(|subject| {
subject.merge_request_id == request.merge_request_id
&& subject.subject_ref == review.subject_ref
}) {
return Err(MergeRequestEvidenceError::ApprovalSourceMismatch);
}
Ok(review)
})();
if evidence.is_ok() || event_id.is_some() {
return evidence;
}
}
}
Err(MergeRequestEvidenceError::RequirementApprovalMissing)
}
fn same_source_snapshot(
left: &[MergeRequestReviewSubject],
right: &[MergeRequestReviewSubject],
) -> bool {
!left.is_empty()
&& left.len() == right.len()
&& left.iter().all(|subject| {
!subject.merge_request_id.trim().is_empty()
&& !subject.subject_ref.trim().is_empty()
&& left
.iter()
.filter(|other| other.merge_request_id == subject.merge_request_id)
.count()
== 1
&& right.iter().filter(|other| *other == subject).count() == 1
})
}
#[derive(Clone, Debug, Default)]
@@ -364,16 +598,8 @@ pub struct CompleteMergeRequest {
pub auth: MergeRequestAuth,
pub now: DateTime<Utc>,
}
#[derive(Debug, Clone)]
pub struct CompleteTicket {
pub ticket_id: String,
pub operation_id: String,
pub item_revision: String,
pub merge_request_ids: Vec<String>,
pub requirement_approval_event_id: String,
pub auth: MergeRequestAuth,
pub now: DateTime<Utc>,
}
/// Historical MR-owned Ticket completion payload, retained for reading existing events.
/// Canonical Ticket completion is owned by the Ticket domain.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, schemars::JsonSchema)]
pub struct TicketCompletionEvent {
pub operation_id: String,
@@ -979,7 +1205,7 @@ impl MergeRequestStore {
}
let assignment_is_current: bool = transaction.query_row(
"SELECT EXISTS(
SELECT 1 FROM ticket_current_worker_assignments
SELECT 1 FROM ticket_active_worker_assignments
WHERE workspace_id=?1 AND ticket_id=?2 AND assignment_id=?3
)",
params![i.auth.workspace_id, i.ticket_id, i.auth.assignment_id],
@@ -1098,227 +1324,6 @@ impl MergeRequestStore {
Ok(event)
}
pub fn complete_ticket(
&self,
mut input: CompleteTicket,
) -> Result<TicketCompletionEvent, MergeRequestError> {
nonempty("operation_id", &input.operation_id)?;
nonempty("item_revision", &input.item_revision)?;
nonempty(
"requirement_approval_event_id",
&input.requirement_approval_event_id,
)?;
if input.merge_request_ids.is_empty() {
return Err(MergeRequestError::NotReady(
"Ticket completion requires at least one linked Merge Request result".into(),
));
}
input.merge_request_ids.sort();
if input
.merge_request_ids
.windows(2)
.any(|ids| ids[0] == ids[1])
{
return Err(MergeRequestError::Validation(
"merge_request_ids must not contain duplicates".into(),
));
}
let mut connection = self.lock()?;
let transaction = connection.transaction()?;
let existing: Option<String> = transaction
.query_row(
"SELECT payload.value
FROM typed_ticket_event_attributes operation
JOIN typed_ticket_event_attributes payload
ON payload.workspace_id=operation.workspace_id
AND payload.ticket_id=operation.ticket_id
AND payload.event_index=operation.event_index
AND payload.key='ticket_completion'
WHERE operation.workspace_id=?1 AND operation.ticket_id=?2
AND operation.key='operation_id' AND operation.value=?3",
params![input.auth.workspace_id, input.ticket_id, input.operation_id],
|row| row.get(0),
)
.optional()?;
if let Some(payload) = existing {
let event: TicketCompletionEvent = json(&payload)?;
if event.item_revision == input.item_revision
&& event.merge_request_ids == input.merge_request_ids
&& event.requirement_approval_event_id == input.requirement_approval_event_id
&& event.completed_by == input.auth.actor()
{
return Ok(event);
}
return Err(MergeRequestError::Conflict(
"Ticket completion operation fingerprint mismatch".into(),
));
}
let current_revision =
current_ticket_revision(&transaction, &input.auth.workspace_id, &input.ticket_id)?
.ok_or(MergeRequestError::NotFound)?;
if current_revision != input.item_revision {
return Err(MergeRequestError::Conflict(format!(
"Ticket item revision changed from `{}` to `{current_revision}`; repeat requirement review before completion",
input.item_revision
)));
}
let state: String = transaction.query_row(
"SELECT workflow_state FROM typed_tickets WHERE workspace_id=?1 AND ticket_id=?2",
params![input.auth.workspace_id, input.ticket_id],
|row| row.get(0),
)?;
if state != "inprogress" {
return Err(MergeRequestError::Conflict(
"Ticket must be inprogress".into(),
));
}
let assignment_is_current: bool = transaction.query_row(
"SELECT EXISTS(
SELECT 1 FROM ticket_current_worker_assignments
WHERE workspace_id=?1 AND ticket_id=?2 AND assignment_id=?3
)",
params![
input.auth.workspace_id,
input.ticket_id,
input.auth.assignment_id
],
|row| row.get(0),
)?;
if !assignment_is_current {
return Err(MergeRequestError::Unauthorized(
"Ticket completion assignment changed before commit".into(),
));
}
let actual_ids =
linked_merge_request_ids(&transaction, &input.auth.workspace_id, &input.ticket_id)?;
if actual_ids != input.merge_request_ids {
return Err(MergeRequestError::Conflict(
"linked Merge Request set changed; refresh Ticket completion evidence".into(),
));
}
let mut requests = Vec::with_capacity(actual_ids.len());
let mut merged_subjects = Vec::with_capacity(actual_ids.len());
for merge_request_id in &actual_ids {
let request = load_mr(&transaction, &input.auth.workspace_id, merge_request_id)?
.ok_or(MergeRequestError::NotFound)?;
if request.state != MergeRequestState::Merged {
return Err(MergeRequestError::NotReady(format!(
"Merge Request `{merge_request_id}` has no merged result"
)));
}
let merge = request
.thread
.iter()
.rev()
.find_map(|event| match event {
MergeRequestThreadEvent::Merge(event) => Some(event),
_ => None,
})
.ok_or_else(|| {
MergeRequestError::Corrupt(format!(
"merged Merge Request `{merge_request_id}` has no MergeResult"
))
})?;
let review = request
.thread
.iter()
.find_map(|event| match event {
MergeRequestThreadEvent::Review(review)
if review.event_id == merge.approval_event_id
&& review.subject_ref == merge.approved_source_ref =>
{
Some(review)
}
_ => None,
})
.filter(|review| {
!request.thread.iter().any(|event| {
matches!(
event,
MergeRequestThreadEvent::ReviewRevoked(revoked)
if revoked.review_event_id == review.event_id
)
})
})
.ok_or_else(|| {
MergeRequestError::NotReady(format!(
"Merge Request `{merge_request_id}` integration approval is missing or revoked"
))
})?;
if review.decision != ReviewDecision::Approve {
return Err(MergeRequestError::NotReady(format!(
"Merge Request `{merge_request_id}` lacks an approved integration result"
)));
}
merged_subjects.push(MergeRequestReviewSubject {
merge_request_id: merge_request_id.clone(),
subject_ref: merge.approved_source_ref.clone(),
});
requests.push(request);
}
let has_requirement_attestation = requests.iter().any(|request| {
request.thread.iter().rev().any(|event| {
let MergeRequestThreadEvent::Review(review) = event else {
return false;
};
review.event_id == input.requirement_approval_event_id
&& review.decision == ReviewDecision::Approve
&& review.ticket_item_revision == input.item_revision
&& review.ticket_merge_request_subjects == merged_subjects
&& request
.effective_review(&review.subject_ref)
.is_some_and(|effective| effective.event_id == review.event_id)
})
});
if !has_requirement_attestation {
return Err(MergeRequestError::NotReady(
"no effective Reviewer approval attests the current Ticket revision and exact merged source set"
.into(),
));
}
let updated = transaction.execute(
"UPDATE typed_tickets
SET workflow_state='done',workflow_state_explicit=1,updated_at=?3
WHERE workspace_id=?1 AND ticket_id=?2 AND workflow_state='inprogress'",
params![
input.auth.workspace_id,
input.ticket_id,
input.now.to_rfc3339()
],
)?;
if updated != 1 {
return Err(MergeRequestError::Conflict(
"Ticket state changed during completion".into(),
));
}
let released = transaction.execute(
"DELETE FROM ticket_current_worker_assignments
WHERE workspace_id=?1 AND ticket_id=?2 AND assignment_id=?3",
params![
input.auth.workspace_id,
input.ticket_id,
input.auth.assignment_id
],
)?;
if released != 1 {
return Err(MergeRequestError::Unauthorized(
"Ticket completion assignment changed while releasing it".into(),
));
}
let event = TicketCompletionEvent {
operation_id: input.operation_id,
ticket_id: input.ticket_id.clone(),
item_revision: input.item_revision,
merge_request_ids: input.merge_request_ids,
requirement_approval_event_id: input.requirement_approval_event_id,
completed_by: input.auth.actor(),
created_at: input.now,
};
ticket_completion_event(&transaction, &input.auth.workspace_id, &event)?;
transaction.commit()?;
Ok(event)
}
pub fn repair_selector_from(
&self,
i: RepairSelectorFrom,
@@ -1764,62 +1769,6 @@ fn ticket_event(
Ok(())
}
fn ticket_completion_event(
transaction: &Transaction<'_>,
workspace_id: &str,
event: &TicketCompletionEvent,
) -> Result<(), MergeRequestError> {
let event_index: i64 = transaction.query_row(
"SELECT COALESCE(MAX(event_index),-1)+1 FROM typed_ticket_events WHERE workspace_id=?1 AND ticket_id=?2",
params![workspace_id, event.ticket_id],
|row| row.get(0),
)?;
transaction.execute(
"INSERT INTO typed_ticket_events(workspace_id,ticket_id,event_index,kind,author,at,from_state,to_state,heading,body)
VALUES(?1,?2,?3,'state_changed',?4,?5,'inprogress','done','Ticket requirements completed',?6)",
params![
workspace_id,
event.ticket_id,
event_index,
format!(
"worker:{}:{}",
event.completed_by.runtime_id, event.completed_by.worker_id
),
event.created_at.to_rfc3339(),
format!(
"Current Ticket revision `{}` was approved by Review `{}` and completed with Merge Requests: {}.",
event.item_revision,
event.requirement_approval_event_id,
event.merge_request_ids.join(", ")
)
],
)?;
for (key, value) in [
("operation_id", event.operation_id.clone()),
("item_revision", event.item_revision.clone()),
(
"requirement_approval_event_id",
event.requirement_approval_event_id.clone(),
),
(
"merge_request_ids",
serde_json::to_string(&event.merge_request_ids)
.map_err(|error| MergeRequestError::Operation(error.to_string()))?,
),
(
"ticket_completion",
serde_json::to_string(event)
.map_err(|error| MergeRequestError::Operation(error.to_string()))?,
),
] {
transaction.execute(
"INSERT INTO typed_ticket_event_attributes VALUES(?1,?2,?3,?4,?5)",
params![workspace_id, event.ticket_id, event_index, key, value],
)?;
}
Ok(())
}
pub fn migrate(c: &Connection) -> Result<(), MergeRequestError> {
match schema_state(c)? {
SchemaState::Fresh => fresh(c),
+459
View File
@@ -0,0 +1,459 @@
use chrono::{TimeZone, Utc};
use merge_request::*;
fn actor() -> WorkerIdentity {
WorkerIdentity {
runtime_id: "runtime".into(),
worker_id: "reviewer".into(),
}
}
fn subjects() -> Vec<MergeRequestReviewSubject> {
vec![
MergeRequestReviewSubject {
merge_request_id: "MR-1".into(),
subject_ref: "source-one".into(),
},
MergeRequestReviewSubject {
merge_request_id: "MR-2".into(),
subject_ref: "source-two".into(),
},
]
}
fn add_review(
request: &mut MergeRequest,
event_id: &str,
revision: &str,
snapshot: &[MergeRequestReviewSubject],
decision: ReviewDecision,
) {
let subject = snapshot
.iter()
.find(|subject| subject.merge_request_id == request.merge_request_id)
.unwrap();
let sequence = request.thread.len() as u64 + 1;
let requested = ReviewRequestedEvent {
event_id: format!("request-{event_id}"),
sequence,
subject_ref: subject.subject_ref.clone(),
ticket_item_revision: revision.into(),
ticket_merge_request_subjects: snapshot.to_vec(),
requested_by: actor(),
reviewer: actor(),
created_at: request.created_at,
};
let review = ReviewEvent {
event_id: event_id.into(),
sequence: sequence + 1,
request_event_id: requested.event_id.clone(),
subject_ref: requested.subject_ref.clone(),
ticket_item_revision: revision.into(),
ticket_merge_request_subjects: snapshot.to_vec(),
decision,
body: String::new(),
findings: vec![],
reviewer: actor(),
created_at: request.created_at,
};
request
.thread
.push(MergeRequestThreadEvent::ReviewRequested(requested));
request.thread.push(MergeRequestThreadEvent::Review(review));
}
fn merged_requests() -> Vec<MergeRequest> {
let now = Utc.with_ymd_and_hms(2026, 10, 8, 12, 0, 0).unwrap();
let snapshot = subjects();
snapshot
.iter()
.map(|subject| {
let mut request = MergeRequest {
workspace_id: "W".into(),
merge_request_id: subject.merge_request_id.clone(),
repository_id: subject.merge_request_id.clone(),
state: MergeRequestState::Merged,
selector_from: Some("work/source".into()),
selector_to: "main".into(),
ticket_ids: vec!["T".into()],
created_at: now,
updated_at: now,
thread: vec![],
};
let approval_id = format!("integration-{}", subject.merge_request_id);
add_review(
&mut request,
&approval_id,
"revision-one",
&snapshot,
ReviewDecision::Approve,
);
request
.thread
.push(MergeRequestThreadEvent::Merge(MergeEvent {
event_id: format!("merge-{}", subject.merge_request_id),
sequence: 3,
operation_id: format!("operation-{}", subject.merge_request_id),
approval_event_id: approval_id,
approved_source_ref: subject.subject_ref.clone(),
target_ref_before: "target-before".into(),
target_ref_after: "target-after".into(),
strategy: MergeStrategy::FastForward,
resolution: ConflictResolution::None,
merged_by: actor(),
created_at: now,
}));
request
})
.collect()
}
#[test]
fn postmerge_latest_approval_attests_the_current_revision_without_replacing_integration() {
let mut requests = merged_requests();
let snapshot = subjects();
assert_eq!(
requirement_approval(
&requests,
"revision-two",
&snapshot,
Some("integration-MR-1")
),
Err(MergeRequestEvidenceError::ItemRevisionMismatch)
);
add_review(
&mut requests[0],
"postmerge",
"revision-two",
&snapshot,
ReviewDecision::Approve,
);
assert_eq!(
requests[0].integration_approval().unwrap().event_id,
"integration-MR-1"
);
assert_eq!(
requirement_approval(&requests, "revision-two", &snapshot, None)
.unwrap()
.event_id,
"postmerge"
);
assert_eq!(
requirement_approval(&requests, "revision-two", &snapshot, Some("postmerge"))
.unwrap()
.event_id,
"postmerge"
);
assert_eq!(
requirement_approval(
&requests,
"revision-one",
&snapshot,
Some("integration-MR-1")
),
Err(MergeRequestEvidenceError::ApprovalNotEffective)
);
assert_eq!(
requirement_approval(&requests, "revision-two", &snapshot, Some("unknown")),
Err(MergeRequestEvidenceError::RequirementApprovalMissing)
);
}
#[test]
fn later_request_changes_prevents_older_requirement_approval_from_being_effective() {
let mut requests = merged_requests();
let snapshot = subjects();
add_review(
&mut requests[0],
"approved",
"revision-two",
&snapshot,
ReviewDecision::Approve,
);
add_review(
&mut requests[0],
"changes",
"revision-two",
&snapshot,
ReviewDecision::RequestChanges,
);
assert_eq!(
requirement_approval(&requests, "revision-two", &snapshot, None),
Err(MergeRequestEvidenceError::RequirementApprovalMissing)
);
assert_eq!(
requirement_approval(&requests, "revision-two", &snapshot, Some("approved")),
Err(MergeRequestEvidenceError::ApprovalNotEffective)
);
assert_eq!(
requirement_approval(&requests, "revision-two", &snapshot, Some("changes")),
Err(MergeRequestEvidenceError::ApprovalNotApproved)
);
assert!(requests[0].integration_approval().is_ok());
}
#[test]
fn changed_revision_or_any_linked_source_invalidates_a_multiple_mr_attestation() {
let requests = merged_requests();
let original = subjects();
assert!(requirement_approval(&requests, "revision-one", &original, None).is_ok());
assert_eq!(
requirement_approval(
&requests,
"revision-two",
&original,
Some("integration-MR-1")
),
Err(MergeRequestEvidenceError::ItemRevisionMismatch)
);
let mut changed = original.clone();
changed[1].subject_ref = "changed-source-two".into();
assert_eq!(
requirement_approval(
&requests,
"revision-one",
&changed,
Some("integration-MR-1")
),
Err(MergeRequestEvidenceError::SourceSnapshotMismatch)
);
let mut reversed = original.clone();
reversed.reverse();
assert!(requirement_approval(&requests, "revision-one", &reversed, None).is_ok());
// Even after the caller refreshes the linked request collection, an old
// attestation cannot authorize completion against a smaller linked set.
assert_eq!(
requirement_approval(
&requests[..1],
"revision-one",
&original[..1],
Some("integration-MR-1")
),
Err(MergeRequestEvidenceError::SourceSnapshotMismatch)
);
assert_eq!(
requirement_approval(&requests, "revision-one", &original[..1], None),
Err(MergeRequestEvidenceError::SourceSnapshotMismatch)
);
assert_eq!(
requirement_approval(
&requests,
"revision-one",
&[original[0].clone(), original[0].clone()],
None
),
Err(MergeRequestEvidenceError::SourceSnapshotMismatch)
);
}
#[test]
fn refreshed_postmerge_approval_can_attest_a_changed_multiple_mr_set_and_revision() {
let mut requests = merged_requests();
let mut snapshot = subjects();
snapshot.pop();
requests.pop();
assert_eq!(
requirement_approval(&requests, "revision-two", &snapshot, None),
Err(MergeRequestEvidenceError::RequirementApprovalMissing)
);
add_review(
&mut requests[0],
"refreshed-set",
"revision-two",
&snapshot,
ReviewDecision::Approve,
);
assert_eq!(
requirement_approval(&requests, "revision-two", &snapshot, None)
.unwrap()
.event_id,
"refreshed-set"
);
assert_eq!(
requests[0].integration_approval().unwrap().event_id,
"integration-MR-1"
);
}
#[test]
fn requirement_approval_must_bind_its_own_mr_source_and_corresponding_request() {
let original = subjects();
for (case, expected) in [
("missing", MergeRequestEvidenceError::ReviewRequestMissing),
("revision", MergeRequestEvidenceError::ReviewRequestMismatch),
("snapshot", MergeRequestEvidenceError::ReviewRequestMismatch),
("reviewer", MergeRequestEvidenceError::ReviewRequestMismatch),
("source", MergeRequestEvidenceError::ApprovalSourceMismatch),
] {
let mut requests = merged_requests();
for event in &mut requests[0].thread {
match event {
MergeRequestThreadEvent::Review(review) if case == "missing" => {
review.request_event_id = "absent-request".into();
}
MergeRequestThreadEvent::ReviewRequested(requested) => match case {
"revision" => requested.ticket_item_revision = "other-revision".into(),
"snapshot" => {
requested.ticket_merge_request_subjects.pop();
}
"reviewer" => requested.reviewer.worker_id = "other-reviewer".into(),
"source" => requested.subject_ref = "unbound-source".into(),
_ => {}
},
MergeRequestThreadEvent::Review(review) if case == "source" => {
review.subject_ref = "unbound-source".into();
}
_ => {}
}
}
assert_eq!(
requirement_approval(
&requests,
"revision-one",
&original,
Some("integration-MR-1")
),
Err(expected),
"{case}"
);
}
}
#[test]
fn revoked_requirement_approval_is_rejected_and_discovery_uses_another_valid_mr() {
let mut requests = merged_requests();
let created_at = requests[0].created_at;
requests[0]
.thread
.push(MergeRequestThreadEvent::ReviewRevoked(ReviewRevokedEvent {
event_id: "revocation".into(),
sequence: 4,
review_event_id: "integration-MR-1".into(),
subject_ref: "source-one".into(),
reason: String::new(),
revoked_by: actor(),
created_at,
}));
assert_eq!(
requirement_approval(
&requests,
"revision-one",
&subjects(),
Some("integration-MR-1")
),
Err(MergeRequestEvidenceError::ApprovalRevoked)
);
assert_eq!(
requirement_approval(&requests, "revision-one", &subjects(), None)
.unwrap()
.event_id,
"integration-MR-2"
);
}
#[test]
fn integration_approval_must_strictly_predate_the_merge_sequence() {
for approval_sequence in [2, 3, 4] {
let mut requests = merged_requests();
for event in &mut requests[0].thread {
if let MergeRequestThreadEvent::Review(review) = event {
review.sequence = approval_sequence;
}
}
let result = requests[0].integration_approval();
if approval_sequence < 3 {
assert!(result.is_ok());
} else {
assert_eq!(
result,
Err(MergeRequestEvidenceError::ApprovalTimelineMismatch),
"approval sequence {approval_sequence}"
);
}
}
}
#[test]
fn blank_saved_source_is_not_valid_integration_evidence_even_when_the_review_matches() {
for source in ["", " \t\n"] {
let mut requests = merged_requests();
for event in &mut requests[0].thread {
match event {
MergeRequestThreadEvent::ReviewRequested(requested) => {
requested.subject_ref = source.into();
}
MergeRequestThreadEvent::Review(review) => {
review.subject_ref = source.into();
}
MergeRequestThreadEvent::Merge(merge) => {
merge.approved_source_ref = source.into();
}
_ => {}
}
}
assert_eq!(
requests[0].integration_approval(),
Err(MergeRequestEvidenceError::ApprovalSourceMismatch)
);
}
}
#[test]
fn evidence_error_codes_are_stable_machine_diagnostics_separate_from_messages() {
for (error, code) in [
(MergeRequestEvidenceError::NotMerged, "not_merged"),
(
MergeRequestEvidenceError::MergeResultMissing,
"merge_result_missing",
),
(
MergeRequestEvidenceError::ApprovalMissing,
"approval_missing",
),
(
MergeRequestEvidenceError::ApprovalRevoked,
"approval_revoked",
),
(
MergeRequestEvidenceError::ApprovalNotApproved,
"approval_not_approved",
),
(
MergeRequestEvidenceError::ApprovalSourceMismatch,
"approval_source_mismatch",
),
(
MergeRequestEvidenceError::ApprovalTimelineMismatch,
"approval_timeline_mismatch",
),
(
MergeRequestEvidenceError::ReviewRequestMissing,
"review_request_missing",
),
(
MergeRequestEvidenceError::ReviewRequestMismatch,
"review_request_mismatch",
),
(
MergeRequestEvidenceError::ApprovalNotEffective,
"approval_not_effective",
),
(
MergeRequestEvidenceError::ItemRevisionMismatch,
"item_revision_mismatch",
),
(
MergeRequestEvidenceError::SourceSnapshotMismatch,
"source_snapshot_mismatch",
),
(
MergeRequestEvidenceError::RequirementApprovalMissing,
"requirement_approval_missing",
),
] {
assert_eq!(error.code(), code);
assert_ne!(error.code(), error.as_str());
}
}
+275 -134
View File
@@ -39,7 +39,7 @@ fn fixture() -> (tempfile::TempDir, MergeRequestStore) {
let d = tempfile::tempdir().unwrap();
let p = d.path().join("db");
let c = Connection::open(&p).unwrap();
c.execute_batch("CREATE TABLE workspaces(workspace_id TEXT PRIMARY KEY);CREATE TABLE repositories(workspace_id TEXT,repository_id TEXT,PRIMARY KEY(workspace_id,repository_id));CREATE TABLE ticket_current_worker_assignments(workspace_id TEXT,ticket_id TEXT,assignment_id TEXT,runtime_id TEXT,worker_id TEXT,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_tickets(workspace_id TEXT,ticket_id TEXT,workflow_state TEXT,workflow_state_explicit INTEGER,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_ticket_events(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,kind TEXT,author TEXT,at TEXT,from_state TEXT,to_state TEXT,heading TEXT,body TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index));CREATE TABLE typed_ticket_event_attributes(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,key TEXT,value TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index,key));INSERT INTO workspaces VALUES('W');INSERT INTO repositories VALUES('W','R');INSERT INTO repositories VALUES('W','R2');INSERT INTO ticket_current_worker_assignments VALUES('W','T','A','runtime','coder','t');INSERT INTO typed_tickets VALUES('W','T','inprogress',1,'t');").unwrap();
c.execute_batch("CREATE TABLE workspaces(workspace_id TEXT PRIMARY KEY);CREATE TABLE repositories(workspace_id TEXT,repository_id TEXT,PRIMARY KEY(workspace_id,repository_id));CREATE TABLE ticket_current_worker_assignments(workspace_id TEXT,ticket_id TEXT,assignment_id TEXT,runtime_id TEXT,worker_id TEXT,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_tickets(workspace_id TEXT,ticket_id TEXT,workflow_state TEXT,workflow_state_explicit INTEGER,updated_at TEXT,PRIMARY KEY(workspace_id,ticket_id));CREATE TABLE typed_ticket_events(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,kind TEXT,author TEXT,at TEXT,from_state TEXT,to_state TEXT,heading TEXT,body TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index));CREATE TABLE typed_ticket_event_attributes(workspace_id TEXT,ticket_id TEXT,event_index INTEGER,key TEXT,value TEXT,PRIMARY KEY(workspace_id,ticket_id,event_index,key));INSERT INTO workspaces VALUES('W');INSERT INTO repositories VALUES('W','R');INSERT INTO repositories VALUES('W','R2');INSERT INTO ticket_current_worker_assignments VALUES('W','T','A','runtime','coder','t');INSERT INTO typed_tickets VALUES('W','T','inprogress',1,'t');CREATE VIEW ticket_active_worker_assignments AS SELECT current.* FROM ticket_current_worker_assignments current JOIN typed_tickets ticket ON ticket.workspace_id=current.workspace_id AND ticket.ticket_id=current.ticket_id WHERE ticket.workflow_state NOT IN ('done','closed');").unwrap();
drop(c);
let a = Assignments(Arc::new(Mutex::new(CurrentAssignment {
assignment_id: "A".into(),
@@ -166,7 +166,7 @@ fn review_submission_authorization_rejects_invalid_grants_before_side_effects()
}
#[test]
fn selectors_thread_and_completion_have_no_revision_or_commit_api() {
fn integration_preserves_selectors_ticket_state_and_assignment_and_replays_its_merge() {
let (d, s) = fixture();
open(&s);
let review = approve(&s, "opaque-source-ref", "token");
@@ -226,43 +226,14 @@ fn selectors_thread_and_completion_have_no_revision_or_commit_api() {
})
.unwrap();
assert_eq!(replayed, merged);
let completed = s
.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-op".into(),
item_revision: "t".into(),
merge_request_ids: vec!["MR".into()],
requirement_approval_event_id: merged.approval_event_id.clone(),
auth: auth(),
now: at(7),
})
.unwrap();
assert_eq!(completed.merge_request_ids, vec!["MR"]);
let connection = Connection::open(d.path().join("db")).unwrap();
let (state, assignment_exists): (String, bool) = connection
.query_row(
"SELECT workflow_state, EXISTS(
SELECT 1 FROM ticket_current_worker_assignments
WHERE workspace_id='W' AND ticket_id='T'
) FROM typed_tickets WHERE workspace_id='W' AND ticket_id='T'",
[],
|row| Ok((row.get(0)?, row.get(1)?)),
)
.unwrap();
assert_eq!(state, "done");
assert!(!assignment_exists);
let replayed_completion = s
.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-op".into(),
item_revision: "t".into(),
merge_request_ids: vec!["MR".into()],
requirement_approval_event_id: merged.approval_event_id.clone(),
auth: auth(),
now: at(8),
})
.unwrap();
assert_eq!(replayed_completion, completed);
let (state, active): (String, bool) = connection.query_row(
"SELECT workflow_state, EXISTS(SELECT 1 FROM ticket_active_worker_assignments WHERE workspace_id='W' AND ticket_id='T') FROM typed_tickets WHERE workspace_id='W' AND ticket_id='T'",
[],
|row| Ok((row.get(0)?, row.get(1)?)),
).unwrap();
assert_eq!(state, "inprogress");
assert!(active);
let json = serde_json::to_string(&mr).unwrap();
for banned in [
"revision_id",
@@ -864,12 +835,12 @@ fn ticket_rescope_after_integration_accepts_fresh_requirement_attestation() {
let (dir, store) = fixture();
open(&store);
let integration_approval = approve(&store, "subject", "token-integration");
store
let merge = store
.complete(CompleteMergeRequest {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
operation_id: "merge".into(),
approval_event_id: integration_approval.event_id,
approval_event_id: integration_approval.event_id.clone(),
current_subject_ref: "subject".into(),
target_ref_before: "target-before".into(),
target_ref_after: "target-after".into(),
@@ -894,6 +865,26 @@ fn ticket_rescope_after_integration_accepts_fresh_requirement_attestation() {
.unwrap();
drop(connection);
let snapshot = vec![MergeRequestReviewSubject {
merge_request_id: "MR".into(),
subject_ref: merge.approved_source_ref.clone(),
}];
let before_refresh = store.list_for_ticket("W", "T").unwrap();
assert_eq!(
requirement_approval(
&before_refresh,
"T:1",
&snapshot,
Some(&integration_approval.event_id),
),
Err(MergeRequestEvidenceError::ItemRevisionMismatch)
);
assert_eq!(before_refresh[0].merged_result().unwrap(), &merge);
assert_eq!(
before_refresh[0].integration_approval().unwrap(),
&integration_approval
);
store
.register_reviewer_child_session(RegisterReviewerChildSession {
workspace_id: "W".into(),
@@ -920,7 +911,7 @@ fn ticket_rescope_after_integration_accepts_fresh_requirement_attestation() {
now: at(7),
})
.unwrap();
let requirement_approval = store
let requirement_review = store
.submit_review(SubmitMergeRequestReview {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
@@ -932,17 +923,33 @@ fn ticket_rescope_after_integration_accepts_fresh_requirement_attestation() {
now: at(8),
})
.unwrap();
store
.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-complete-post-rescope".into(),
item_revision: "T:1".into(),
merge_request_ids: vec!["MR".into()],
requirement_approval_event_id: requirement_approval.event_id,
auth: auth(),
now: at(9),
})
.unwrap();
let refreshed = store.list_for_ticket("W", "T").unwrap();
assert_eq!(
requirement_approval(
&refreshed,
"T:1",
&snapshot,
Some(&requirement_review.event_id),
)
.unwrap(),
&requirement_review
);
assert_eq!(
requirement_approval(&refreshed, "T:1", &snapshot, None).unwrap(),
&requirement_review
);
assert_eq!(refreshed[0].merged_result().unwrap(), &merge);
assert_eq!(
refreshed[0].integration_approval().unwrap(),
&integration_approval
);
let connection = Connection::open(dir.path().join("db")).unwrap();
let merge_payload: String = connection.query_row(
"SELECT payload_json FROM merge_request_thread_events WHERE workspace_id='W' AND merge_request_id='MR' AND kind='merge'",
[],
|row| row.get(0),
).unwrap();
assert_eq!(merge_payload, serde_json::to_string(&merge).unwrap());
}
#[test]
@@ -978,19 +985,19 @@ fn review_request_rejects_a_snapshot_that_omits_a_linked_merge_request() {
}
#[test]
fn partial_integration_retains_ticket_and_assignment_until_guarded_ticket_completion() {
fn partial_and_full_integration_retain_ticket_state_and_active_assignment() {
let (dir, store) = fixture();
open_for(&store, "MR", "R");
open_for(&store, "MR-2", "R2");
let first = approve_for(&store, "MR", "R", "subject-one", "token-one");
let second = approve_for(&store, "MR-2", "R2", "subject-two", "token-two");
store
let first_merge = store
.complete(CompleteMergeRequest {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
operation_id: "merge-one".into(),
approval_event_id: first.event_id,
approval_event_id: first.event_id.clone(),
current_subject_ref: "subject-one".into(),
target_ref_before: "target-one-before".into(),
target_ref_after: "target-one-after".into(),
@@ -1000,30 +1007,33 @@ fn partial_integration_retains_ticket_and_assignment_until_guarded_ticket_comple
now: at(5),
})
.unwrap();
let connection = Connection::open(dir.path().join("db")).unwrap();
let (state, assigned): (String, bool) = connection
.query_row(
"SELECT workflow_state, EXISTS(SELECT 1 FROM ticket_current_worker_assignments WHERE workspace_id='W' AND ticket_id='T') FROM typed_tickets WHERE workspace_id='W' AND ticket_id='T'",
[],
|row| Ok((row.get(0)?, row.get(1)?)),
)
.unwrap();
assert_eq!(state, "inprogress");
assert!(assigned);
assert!(matches!(
store.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-complete".into(),
item_revision: "t".into(),
merge_request_ids: vec!["MR".into(), "MR-2".into()],
requirement_approval_event_id: second.event_id.clone(),
auth: auth(),
now: at(6),
}),
Err(MergeRequestError::NotReady(_))
));
let partial = store.get_by_id("W", "MR-2").unwrap();
assert_eq!(
partial.integration_approval(),
Err(MergeRequestEvidenceError::NotMerged)
);
store
let connection = Connection::open(dir.path().join("db")).unwrap();
let assert_ticket_unchanged = || {
let (state, assigned, active, state_changes): (String, bool, bool, i64) = connection
.query_row(
"SELECT workflow_state,
EXISTS(SELECT 1 FROM ticket_current_worker_assignments WHERE workspace_id='W' AND ticket_id='T'),
EXISTS(SELECT 1 FROM ticket_active_worker_assignments WHERE workspace_id='W' AND ticket_id='T'),
(SELECT COUNT(*) FROM typed_ticket_events WHERE workspace_id='W' AND ticket_id='T' AND kind='state_changed')
FROM typed_tickets WHERE workspace_id='W' AND ticket_id='T'",
[],
|row| Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?)),
)
.unwrap();
assert_eq!(state, "inprogress");
assert!(assigned);
assert!(active);
assert_eq!(state_changes, 0);
};
assert_ticket_unchanged();
let second_merge = store
.complete(CompleteMergeRequest {
merge_request_id: "MR-2".into(),
ticket_id: "T".into(),
@@ -1038,63 +1048,194 @@ fn partial_integration_retains_ticket_and_assignment_until_guarded_ticket_comple
now: at(7),
})
.unwrap();
assert!(matches!(
store.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-complete".into(),
item_revision: "stale".into(),
merge_request_ids: vec!["MR".into(), "MR-2".into()],
requirement_approval_event_id: second.event_id.clone(),
auth: auth(),
now: at(8),
}),
Err(MergeRequestError::Conflict(_))
));
assert!(matches!(
store.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-complete-missing-attestation".into(),
item_revision: "t".into(),
merge_request_ids: vec!["MR".into(), "MR-2".into()],
requirement_approval_event_id: "missing-review".into(),
auth: auth(),
now: at(8),
}),
Err(MergeRequestError::NotReady(_))
));
assert!(matches!(
store.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-complete".into(),
item_revision: "t".into(),
merge_request_ids: vec!["MR".into()],
requirement_approval_event_id: second.event_id.clone(),
auth: auth(),
now: at(8),
}),
Err(MergeRequestError::Conflict(_))
));
let completed = store
.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-complete".into(),
item_revision: "t".into(),
merge_request_ids: vec!["MR-2".into(), "MR".into()],
requirement_approval_event_id: second.event_id.clone(),
auth: auth(),
now: at(9),
assert_ticket_unchanged();
let requests = store.list_for_ticket("W", "T").unwrap();
let snapshot = requests
.iter()
.map(|request| MergeRequestReviewSubject {
merge_request_id: request.merge_request_id.clone(),
subject_ref: request.merged_result().unwrap().approved_source_ref.clone(),
})
.collect::<Vec<_>>();
assert_eq!(
requirement_approval(&requests, "t", &snapshot, Some(&second.event_id)).unwrap(),
&second
);
let first_request = requests
.iter()
.find(|request| request.merge_request_id == "MR")
.unwrap();
let replay = store
.complete_ticket(CompleteTicket {
ticket_id: "T".into(),
operation_id: "ticket-complete".into(),
item_revision: "t".into(),
merge_request_ids: vec!["MR".into(), "MR-2".into()],
requirement_approval_event_id: second.event_id.clone(),
auth: auth(),
now: at(10),
})
let second_request = requests
.iter()
.find(|request| request.merge_request_id == "MR-2")
.unwrap();
assert_eq!(replay, completed);
assert_eq!(first_request.merged_result().unwrap(), &first_merge);
assert_eq!(second_request.merged_result().unwrap(), &second_merge);
assert_eq!(first_request.integration_approval().unwrap(), &first);
assert_eq!(second_request.integration_approval().unwrap(), &second);
}
#[test]
fn historical_ticket_completion_payload_remains_readable_and_unchanged_on_migration() {
let (dir, _store) = fixture();
let connection = Connection::open(dir.path().join("db")).unwrap();
let payload = r#"{
"operation_id": "historical-operation",
"ticket_id": "T",
"item_revision": "historical-revision",
"merge_request_ids": ["MR-2", "MR"],
"requirement_approval_event_id": "historical-review",
"completed_by": {"runtime_id": "runtime", "worker_id": "coder"},
"created_at": "2026-07-26T12:00:07Z"
}"#;
connection.execute_batch(
"UPDATE typed_tickets SET workflow_state='done' WHERE workspace_id='W' AND ticket_id='T';
INSERT INTO typed_ticket_events VALUES('W','T',0,'state_changed','worker:runtime:coder','2026-07-26T12:00:07Z','inprogress','done','Ticket requirements completed','historical body');"
).unwrap();
connection
.execute(
"INSERT INTO typed_ticket_event_attributes VALUES('W','T',0,'ticket_completion',?1)",
[payload],
)
.unwrap();
migrate(&connection).unwrap();
let (saved_payload, kind, from_state, to_state, heading, body, state): (String, String, String, String, String, String, String) = connection.query_row(
"SELECT attribute.value,event.kind,event.from_state,event.to_state,event.heading,event.body,ticket.workflow_state
FROM typed_ticket_event_attributes attribute
JOIN typed_ticket_events event USING(workspace_id,ticket_id,event_index)
JOIN typed_tickets ticket USING(workspace_id,ticket_id)
WHERE attribute.workspace_id='W' AND attribute.ticket_id='T' AND attribute.key='ticket_completion'",
[],
|row| Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?,row.get(6)?)),
).unwrap();
assert_eq!(saved_payload, payload);
assert_eq!(kind, "state_changed");
assert_eq!(from_state, "inprogress");
assert_eq!(to_state, "done");
assert_eq!(heading, "Ticket requirements completed");
assert_eq!(body, "historical body");
assert_eq!(state, "done");
let historical: TicketCompletionEvent = serde_json::from_str(&saved_payload).unwrap();
assert_eq!(
serde_json::to_value(&historical).unwrap(),
serde_json::from_str::<serde_json::Value>(payload).unwrap()
);
}
#[test]
fn persisted_integration_evidence_rejects_missing_mismatched_revoked_or_late_approval() {
for (case, expected) in [
(
"merge-missing",
MergeRequestEvidenceError::MergeResultMissing,
),
(
"approval-missing",
MergeRequestEvidenceError::ApprovalMissing,
),
("revoked", MergeRequestEvidenceError::ApprovalRevoked),
(
"source-mismatch",
MergeRequestEvidenceError::ApprovalSourceMismatch,
),
(
"not-approved",
MergeRequestEvidenceError::ApprovalNotApproved,
),
(
"request-missing",
MergeRequestEvidenceError::ReviewRequestMissing,
),
(
"request-mismatch",
MergeRequestEvidenceError::ReviewRequestMismatch,
),
(
"approval-after-merge",
MergeRequestEvidenceError::ApprovalTimelineMismatch,
),
] {
let (dir, store) = fixture();
open(&store);
let review = approve(&store, "merged-source", "integration-token");
let merge = store
.complete(CompleteMergeRequest {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
operation_id: "merge".into(),
approval_event_id: review.event_id.clone(),
current_subject_ref: "merged-source".into(),
target_ref_before: "before".into(),
target_ref_after: "after".into(),
strategy: MergeStrategy::FastForward,
resolution: ConflictResolution::None,
auth: auth(),
now: at(5),
})
.unwrap();
let persisted = store.get_by_id("W", "MR").unwrap();
assert_eq!(persisted.merged_result().unwrap(), &merge);
assert_eq!(persisted.integration_approval().unwrap(), &review);
let connection = Connection::open(dir.path().join("db")).unwrap();
match case {
"merge-missing" | "approval-missing" | "request-missing" => {
let kind = match case {
"merge-missing" => "merge",
"approval-missing" => "review",
_ => "review_requested",
};
connection.execute(
"DELETE FROM merge_request_thread_events WHERE workspace_id='W' AND merge_request_id='MR' AND kind=?1",
[kind],
).unwrap();
}
"revoked" => {
store
.revoke_review(RevokeMergeRequestReview {
merge_request_id: "MR".into(),
ticket_id: "T".into(),
review_event_id: review.event_id.clone(),
reason: "invalid integration evidence".into(),
auth: auth(),
now: at(6),
})
.unwrap();
}
"approval-after-merge" => {
let mut altered = review.clone();
altered.sequence = merge.sequence + 1;
connection.execute(
"UPDATE merge_request_thread_events SET payload_json=?1,sequence=?2 WHERE workspace_id='W' AND merge_request_id='MR' AND kind='review'",
rusqlite::params![serde_json::to_string(&altered).unwrap(), altered.sequence],
).unwrap();
}
_ => {
let mut altered = review.clone();
match case {
"source-mismatch" => altered.subject_ref = "different-source".into(),
"not-approved" => altered.decision = ReviewDecision::RequestChanges,
"request-mismatch" => {
altered.ticket_item_revision = "different-revision".into()
}
_ => unreachable!(),
}
connection.execute(
"UPDATE merge_request_thread_events SET payload_json=?1 WHERE workspace_id='W' AND merge_request_id='MR' AND kind='review'",
[serde_json::to_string(&altered).unwrap()],
).unwrap();
}
}
let reloaded = store.get_by_id("W", "MR").unwrap();
assert_eq!(reloaded.integration_approval(), Err(expected), "{case}");
let (state, assigned): (String, bool) = connection.query_row(
"SELECT workflow_state, EXISTS(SELECT 1 FROM ticket_current_worker_assignments WHERE workspace_id='W' AND ticket_id='T') FROM typed_tickets WHERE workspace_id='W' AND ticket_id='T'",
[],
|row| Ok((row.get(0)?, row.get(1)?)),
).unwrap();
assert_eq!(state, "inprogress", "{case}");
assert!(assigned, "{case}");
}
}
+1
View File
@@ -14,6 +14,7 @@ json-schema = ["dep:schemars"]
schemars = { workspace = true, optional = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
thiserror = { workspace = true }
sha2.workspace = true
tokio = { workspace = true, features = ["io-util"], optional = true }
ts-rs = { version = "12.0.1", optional = true }
+949
View File
@@ -0,0 +1,949 @@
//! Declarative chat Feature invocation contract.
//!
//! The input spelling is deliberately bounded and unambiguous:
//! `/name(arg=value, "positional value")`. Parentheses terminate arguments,
//! so following prose is never consumed. Strings use JSON-style quoting and
//! escaping. Unquoted values are limited to booleans, integers, enum-like words,
//! and paths without comma/parenthesis/whitespace. This module parses input for
//! clients and validates the structured payload again at the host boundary.
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, BTreeSet};
use std::fmt;
/// UTF-8 byte bound shared by input validation and durable recovery keys.
pub const MAX_FEATURE_INVOCATION_ID_BYTES: usize = 128;
/// Installed descriptors must fit a public completion entry without truncation.
pub const MAX_FEATURE_INVOCATION_DESCRIPTOR_BYTES: usize = 32 * 1024;
/// Source-qualified identity of one invocation contribution, for example
/// `builtin:attachments/attach`. Display names and aliases are never authority.
#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct FeatureInvocationIdentity(pub String);
impl fmt::Display for FeatureInvocationIdentity {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str(&self.0)
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum FeatureInvocationSyntax {
/// `/name(value, key=value)` with JSON-style quoted strings and `)` as the
/// explicit boundary before following natural language.
Parenthesized,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum InvocationArgumentType {
String,
Integer,
Boolean,
Enum {
values: Vec<String>,
},
/// A Worker-readable path. This does not grant filesystem authority.
WorkerFile,
/// A resource selected and staged by the client. It must never be resolved
/// as a path on the Worker host.
ClientFile,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum InvocationCompletion {
None,
Static {
values: Vec<String>,
},
WorkerFile,
ClientFile,
/// Host-owned provider selected by source-qualified id. Metadata lookup is
/// side-effect free; implementations never cross this wire boundary.
Provider {
provider: String,
},
}
impl Default for InvocationCompletion {
fn default() -> Self {
Self::None
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct InvocationArgumentDescriptor {
pub name: String,
/// Zero-based positional index. `None` means named-only. A positional
/// argument may also be supplied by name; this makes chip editing stable.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub position: Option<u16>,
#[serde(default)]
pub required: bool,
pub value_type: InvocationArgumentType,
#[serde(default)]
pub completion: InvocationCompletion,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum InvocationClientAdapter {
/// The client stages a local file and replaces the draft invocation with an
/// `UploadedFile` chip. Local paths are not sent to or persisted by Worker.
Attachment,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct FeatureInvocationDescriptor {
pub identity: FeatureInvocationIdentity,
pub name: String,
#[serde(default)]
pub aliases: Vec<String>,
pub display_name: String,
pub description: String,
pub syntax: FeatureInvocationSyntax,
#[serde(default)]
pub arguments: Vec<InvocationArgumentDescriptor>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub client_adapter: Option<InvocationClientAdapter>,
}
impl FeatureInvocationDescriptor {
pub fn usage(&self) -> String {
let mut usage = format!("/{}(", self.name);
for (index, argument) in self.arguments.iter().enumerate() {
if index > 0 {
usage.push_str(", ");
}
if argument.position.is_none() {
usage.push_str(&argument.name);
usage.push('=');
}
let marker = match &argument.value_type {
InvocationArgumentType::String => "text",
InvocationArgumentType::Integer => "integer",
InvocationArgumentType::Boolean => "true|false",
InvocationArgumentType::Enum { .. } => "choice",
InvocationArgumentType::WorkerFile => "worker-path",
InvocationArgumentType::ClientFile => "local-file",
};
if argument.required {
usage.push('<');
usage.push_str(marker);
usage.push('>');
} else {
usage.push('[');
usage.push_str(marker);
usage.push(']');
}
}
usage.push(')');
usage
}
pub fn validate(&self) -> Result<(), InvocationValidationError> {
const MAX_PUBLIC_NAMES: usize = 16;
const MAX_ARGUMENTS: usize = 32;
const MAX_METADATA_BYTES: usize = 4096;
validate_invocation_name(&self.name)?;
if self.aliases.len() > MAX_PUBLIC_NAMES || self.arguments.len() > MAX_ARGUMENTS {
return Err(InvocationValidationError::InvalidDescriptor(
"invocation descriptor exceeds collection limits".into(),
));
}
if self.display_name.len() > MAX_METADATA_BYTES
|| self.description.len() > MAX_METADATA_BYTES
{
return Err(InvocationValidationError::InvalidDescriptor(
"invocation metadata exceeds byte limits".into(),
));
}
if self.identity.0.trim().is_empty()
|| !self.identity.0.contains(':')
|| self.identity.0.len() > 256
{
return Err(InvocationValidationError::InvalidDescriptor(
"invocation identity must be source-qualified".into(),
));
}
let mut names = BTreeSet::new();
let mut positions = BTreeSet::new();
for name in std::iter::once(&self.name).chain(self.aliases.iter()) {
validate_invocation_name(name)?;
if !names.insert(name) {
return Err(InvocationValidationError::InvalidDescriptor(format!(
"duplicate invocation name or alias `{name}`"
)));
}
}
let mut argument_names = BTreeSet::new();
for argument in &self.arguments {
validate_argument_name(&argument.name)?;
if !argument_names.insert(argument.name.as_str()) {
return Err(InvocationValidationError::InvalidDescriptor(format!(
"duplicate argument `{}`",
argument.name
)));
}
if let Some(position) = argument.position
&& !positions.insert(position)
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"duplicate positional index `{position}`"
)));
}
if argument
.description
.as_ref()
.is_some_and(|value| value.len() > MAX_METADATA_BYTES)
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"argument `{}` description exceeds byte limits",
argument.name
)));
}
if let InvocationCompletion::Provider { provider } = &argument.completion
&& (provider.trim().is_empty() || provider.len() > 256)
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"argument `{}` has an invalid completion provider",
argument.name
)));
}
if matches!(&argument.value_type, InvocationArgumentType::Enum { values }
if values.len() > 256 || values.iter().any(|value| value.len() > MAX_METADATA_BYTES))
|| matches!(&argument.completion, InvocationCompletion::Static { values }
if values.len() > 256 || values.iter().any(|value| value.len() > MAX_METADATA_BYTES))
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"argument `{}` completion values exceed limits",
argument.name
)));
}
if matches!(&argument.value_type, InvocationArgumentType::Enum { values } if values.is_empty())
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"enum argument `{}` has no values",
argument.name
)));
}
}
for (expected, actual) in positions.iter().copied().enumerate() {
if actual as usize != expected {
return Err(InvocationValidationError::InvalidDescriptor(
"positional argument indices must be contiguous from zero".into(),
));
}
}
if serde_json::to_vec(self).map_or(true, |encoded| {
encoded.len() > MAX_FEATURE_INVOCATION_DESCRIPTOR_BYTES
}) {
return Err(InvocationValidationError::InvalidDescriptor(
"invocation descriptor exceeds the public completion byte limit".into(),
));
}
Ok(())
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", content = "value", rename_all = "snake_case")]
pub enum InvocationValue {
String(String),
Integer(i32),
Boolean(bool),
}
impl InvocationValue {
pub fn display_input(&self) -> String {
match self {
Self::String(value) => quote_invocation_string(value),
Self::Integer(value) => value.to_string(),
Self::Boolean(value) => value.to_string(),
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct InvocationArgumentValue {
pub name: String,
pub value: InvocationValue,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct FeatureInvocation {
/// Stable client-generated id retained across queueing and transport retry.
pub invocation_id: String,
pub identity: FeatureInvocationIdentity,
/// Selected public name, retained for exact draft/history restoration. Host
/// resolution is always by `identity`.
pub name: String,
pub arguments: Vec<InvocationArgumentValue>,
}
impl FeatureInvocation {
pub fn display_input(&self) -> String {
let mut input = format!("/{}(", self.name);
for (index, argument) in self.arguments.iter().enumerate() {
if index > 0 {
input.push_str(", ");
}
input.push_str(&argument.name);
input.push('=');
input.push_str(&argument.value.display_input());
}
input.push(')');
input
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum FeatureInvocationStatus {
Succeeded,
Failed,
OutcomeUnknown,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct FeatureInvocationResult {
pub invocation_id: String,
pub identity: FeatureInvocationIdentity,
pub status: FeatureInvocationStatus,
pub message: String,
/// Bounded model-visible context produced by the handler. Credentials and
/// executable implementation details must not be included.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ParsedFeatureInvocation {
pub invocation: FeatureInvocation,
/// UTF-8 byte offset immediately after the closing parenthesis.
pub end: usize,
}
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum InvocationValidationError {
#[error("invalid invocation descriptor: {0}")]
InvalidDescriptor(String),
#[error("incomplete invocation at byte {0}")]
Incomplete(usize),
#[error("unknown argument `{0}`")]
UnknownArgument(String),
#[error("duplicate argument `{0}`")]
DuplicateArgument(String),
#[error("missing required argument `{0}`")]
MissingRequired(String),
#[error("too many positional arguments")]
TooManyPositional,
#[error("invalid value for `{argument}`: {message}")]
InvalidValue { argument: String, message: String },
#[error("invocation identity does not match descriptor")]
IdentityMismatch,
#[error("invocation name does not match descriptor")]
NameMismatch,
#[error("invalid invocation syntax at byte {offset}: {message}")]
Syntax { offset: usize, message: String },
}
pub fn validate_feature_invocation(
descriptor: &FeatureInvocationDescriptor,
invocation: &FeatureInvocation,
) -> Result<(), InvocationValidationError> {
descriptor.validate()?;
if invocation.identity != descriptor.identity {
return Err(InvocationValidationError::IdentityMismatch);
}
if invocation.name != descriptor.name && !descriptor.aliases.contains(&invocation.name) {
return Err(InvocationValidationError::NameMismatch);
}
if invocation.invocation_id.trim().is_empty()
|| invocation.invocation_id.len() > MAX_FEATURE_INVOCATION_ID_BYTES
{
return Err(InvocationValidationError::InvalidValue {
argument: "invocation_id".into(),
message: "must be nonempty and at most 128 UTF-8 bytes".into(),
});
}
if invocation.arguments.len() > 32 {
return Err(InvocationValidationError::InvalidValue {
argument: "arguments".into(),
message: "too many arguments".into(),
});
}
let mut values = BTreeMap::new();
for argument in &invocation.arguments {
if values
.insert(argument.name.as_str(), &argument.value)
.is_some()
{
return Err(InvocationValidationError::DuplicateArgument(
argument.name.clone(),
));
}
}
for name in values.keys() {
if !descriptor
.arguments
.iter()
.any(|argument| argument.name == **name)
{
return Err(InvocationValidationError::UnknownArgument(
(*name).to_string(),
));
}
}
for argument in &descriptor.arguments {
match values.get(argument.name.as_str()) {
Some(value) => validate_value(argument, value)?,
None if argument.required => {
return Err(InvocationValidationError::MissingRequired(
argument.name.clone(),
));
}
None => {}
}
}
Ok(())
}
/// Parse one complete parenthesized invocation starting at `start`. Callers
/// decide when input becomes a typed chip; parsing alone never executes it.
pub fn parse_feature_invocation(
input: &str,
start: usize,
descriptor: &FeatureInvocationDescriptor,
invocation_id: impl Into<String>,
) -> Result<ParsedFeatureInvocation, InvocationValidationError> {
descriptor.validate()?;
if !input.is_char_boundary(start) || input.as_bytes().get(start) != Some(&b'/') {
return Err(InvocationValidationError::Syntax {
offset: start,
message: "expected `/`".into(),
});
}
let mut cursor = start + 1;
let name_start = cursor;
while let Some(byte) = input.as_bytes().get(cursor) {
if byte.is_ascii_alphanumeric() || matches!(*byte, b'-' | b'_') {
cursor += 1;
} else {
break;
}
}
let name = &input[name_start..cursor];
if name != descriptor.name && !descriptor.aliases.iter().any(|alias| alias == name) {
return Err(InvocationValidationError::NameMismatch);
}
skip_ws(input, &mut cursor);
if input.as_bytes().get(cursor) != Some(&b'(') {
return Err(InvocationValidationError::Incomplete(cursor));
}
cursor += 1;
let positional = descriptor
.arguments
.iter()
.filter_map(|argument| argument.position.map(|position| (position, argument)))
.collect::<BTreeMap<_, _>>();
let mut next_position = 0u16;
let mut parsed = Vec::<InvocationArgumentValue>::new();
loop {
skip_ws(input, &mut cursor);
match input.as_bytes().get(cursor) {
Some(b')') => {
cursor += 1;
break;
}
None => return Err(InvocationValidationError::Incomplete(cursor)),
_ => {}
}
let token_start = cursor;
let possible_name = parse_identifier(input, &mut cursor);
let after_name = cursor;
skip_ws(input, &mut cursor);
let (argument, raw_value) = if !possible_name.is_empty()
&& input.as_bytes().get(cursor) == Some(&b'=')
{
cursor += 1;
skip_ws(input, &mut cursor);
let argument = descriptor
.arguments
.iter()
.find(|argument| argument.name == possible_name)
.ok_or_else(|| InvocationValidationError::UnknownArgument(possible_name.into()))?;
(argument, parse_raw_value(input, &mut cursor)?)
} else {
cursor = token_start;
let argument = positional
.get(&next_position)
.copied()
.ok_or(InvocationValidationError::TooManyPositional)?;
next_position += 1;
(argument, parse_raw_value(input, &mut cursor)?)
};
if parsed.iter().any(|value| value.name == argument.name) {
return Err(InvocationValidationError::DuplicateArgument(
argument.name.clone(),
));
}
let value = coerce_value(argument, raw_value)?;
parsed.push(InvocationArgumentValue {
name: argument.name.clone(),
value,
});
skip_ws(input, &mut cursor);
match input.as_bytes().get(cursor) {
Some(b',') => cursor += 1,
Some(b')') => {}
None => return Err(InvocationValidationError::Incomplete(cursor)),
_ => {
return Err(InvocationValidationError::Syntax {
offset: after_name.max(cursor),
message: "expected `,` or `)`".into(),
});
}
}
}
let invocation = FeatureInvocation {
invocation_id: invocation_id.into(),
identity: descriptor.identity.clone(),
name: name.to_string(),
arguments: parsed,
};
validate_feature_invocation(descriptor, &invocation)?;
Ok(ParsedFeatureInvocation {
invocation,
end: cursor,
})
}
#[derive(Debug)]
enum RawValue {
Quoted(String),
Bare(String),
}
fn parse_raw_value(input: &str, cursor: &mut usize) -> Result<RawValue, InvocationValidationError> {
if input.as_bytes().get(*cursor) == Some(&b'"') {
let start = *cursor;
*cursor += 1;
let mut escaped = false;
loop {
let Some(ch) = input[*cursor..].chars().next() else {
return Err(InvocationValidationError::Incomplete(*cursor));
};
*cursor += ch.len_utf8();
if ch == '"' && !escaped {
let value =
serde_json::from_str::<String>(&input[start..*cursor]).map_err(|error| {
InvocationValidationError::Syntax {
offset: start,
message: error.to_string(),
}
})?;
return Ok(RawValue::Quoted(value));
}
escaped = ch == '\\' && !escaped;
}
}
let start = *cursor;
while let Some(ch) = input[*cursor..].chars().next() {
if ch == ',' || ch == ')' || ch.is_whitespace() {
break;
}
*cursor += ch.len_utf8();
}
if *cursor == start {
return Err(InvocationValidationError::Syntax {
offset: start,
message: "expected argument value".into(),
});
}
Ok(RawValue::Bare(input[start..*cursor].to_string()))
}
fn coerce_value(
descriptor: &InvocationArgumentDescriptor,
raw: RawValue,
) -> Result<InvocationValue, InvocationValidationError> {
let text = match raw {
RawValue::Quoted(value) | RawValue::Bare(value) => value,
};
let invalid = |message: &str| InvocationValidationError::InvalidValue {
argument: descriptor.name.clone(),
message: message.into(),
};
match &descriptor.value_type {
InvocationArgumentType::String
| InvocationArgumentType::WorkerFile
| InvocationArgumentType::ClientFile => Ok(InvocationValue::String(text)),
InvocationArgumentType::Integer => {
let digits = text.strip_prefix('-').unwrap_or(&text);
if digits.is_empty() || !digits.bytes().all(|byte| byte.is_ascii_digit()) {
return Err(invalid("expected a decimal integer"));
}
text.parse::<i32>()
.map(InvocationValue::Integer)
.map_err(|_| invalid("expected a signed 32-bit integer"))
}
InvocationArgumentType::Boolean => match text.as_str() {
"true" => Ok(InvocationValue::Boolean(true)),
"false" => Ok(InvocationValue::Boolean(false)),
_ => Err(invalid("expected `true` or `false`")),
},
InvocationArgumentType::Enum { values } => {
if values.contains(&text) {
Ok(InvocationValue::String(text))
} else {
Err(invalid("value is not in the declared enum"))
}
}
}
}
fn validate_value(
descriptor: &InvocationArgumentDescriptor,
value: &InvocationValue,
) -> Result<(), InvocationValidationError> {
let valid = match (&descriptor.value_type, value) {
(
InvocationArgumentType::String
| InvocationArgumentType::WorkerFile
| InvocationArgumentType::ClientFile,
InvocationValue::String(value),
) => value.len() <= 16 * 1024,
(InvocationArgumentType::Integer, InvocationValue::Integer(_)) => true,
(InvocationArgumentType::Boolean, InvocationValue::Boolean(_)) => true,
(InvocationArgumentType::Enum { values }, InvocationValue::String(value)) => {
values.contains(value)
}
_ => false,
};
if valid {
Ok(())
} else {
Err(InvocationValidationError::InvalidValue {
argument: descriptor.name.clone(),
message: "structured value does not match the declared type".into(),
})
}
}
fn validate_invocation_name(name: &str) -> Result<(), InvocationValidationError> {
if name.is_empty()
|| !name
.bytes()
.all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'-')
|| !name.as_bytes()[0].is_ascii_lowercase()
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"invalid invocation name `{name}`"
)));
}
Ok(())
}
fn validate_argument_name(name: &str) -> Result<(), InvocationValidationError> {
if name.is_empty()
|| !name
.bytes()
.all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'_')
|| !name.as_bytes()[0].is_ascii_lowercase()
{
return Err(InvocationValidationError::InvalidDescriptor(format!(
"invalid argument name `{name}`"
)));
}
Ok(())
}
fn parse_identifier<'a>(input: &'a str, cursor: &mut usize) -> &'a str {
let start = *cursor;
while let Some(byte) = input.as_bytes().get(*cursor) {
if byte.is_ascii_lowercase() || byte.is_ascii_digit() || *byte == b'_' {
*cursor += 1;
} else {
break;
}
}
&input[start..*cursor]
}
fn skip_ws(input: &str, cursor: &mut usize) {
while let Some(ch) = input[*cursor..].chars().next() {
if !ch.is_whitespace() {
break;
}
*cursor += ch.len_utf8();
}
}
pub fn quote_invocation_string(value: &str) -> String {
// Keep the spelling identical to Web's JSON.stringify, including control
// characters and JSON unicode escapes accepted by both common parsers.
serde_json::to_string(value).expect("serializing a string cannot fail")
}
#[cfg(test)]
mod tests {
use super::*;
fn descriptor() -> FeatureInvocationDescriptor {
FeatureInvocationDescriptor {
identity: FeatureInvocationIdentity("builtin:test/run".into()),
name: "run".into(),
aliases: vec!["execute".into()],
display_name: "Run".into(),
description: "test invocation".into(),
syntax: FeatureInvocationSyntax::Parenthesized,
arguments: vec![
InvocationArgumentDescriptor {
name: "path".into(),
position: Some(0),
required: true,
value_type: InvocationArgumentType::String,
completion: InvocationCompletion::WorkerFile,
description: None,
},
InvocationArgumentDescriptor {
name: "mode".into(),
position: None,
required: false,
value_type: InvocationArgumentType::Enum {
values: vec!["fast".into(), "safe".into()],
},
completion: InvocationCompletion::Static {
values: vec!["fast".into(), "safe".into()],
},
description: None,
},
],
client_adapter: None,
}
}
#[test]
fn parses_unicode_quoted_paths_and_stops_before_natural_language() {
let input = "/run(\"資料/my file / x\", mode=safe) この後は本文";
let parsed = parse_feature_invocation(input, 0, &descriptor(), "invoke-1").unwrap();
assert_eq!(&input[parsed.end..], " この後は本文");
assert_eq!(
parsed.invocation.arguments,
vec![
InvocationArgumentValue {
name: "path".into(),
value: InvocationValue::String("資料/my file / x".into()),
},
InvocationArgumentValue {
name: "mode".into(),
value: InvocationValue::String("safe".into()),
},
]
);
}
#[test]
fn reports_incomplete_unknown_duplicate_missing_and_type_errors() {
assert!(matches!(
parse_feature_invocation("/run(\"x\"", 0, &descriptor(), "1"),
Err(InvocationValidationError::Incomplete(_))
));
assert!(matches!(
parse_feature_invocation("/run(path=x, nope=y)", 0, &descriptor(), "1"),
Err(InvocationValidationError::UnknownArgument(name)) if name == "nope"
));
assert!(matches!(
parse_feature_invocation("/run(path=x, path=y)", 0, &descriptor(), "1"),
Err(InvocationValidationError::DuplicateArgument(name)) if name == "path"
));
assert!(matches!(
parse_feature_invocation("/run(mode=fast)", 0, &descriptor(), "1"),
Err(InvocationValidationError::MissingRequired(name)) if name == "path"
));
assert!(matches!(
parse_feature_invocation("/run(x, mode=unknown)", 0, &descriptor(), "1"),
Err(InvocationValidationError::InvalidValue { argument, .. }) if argument == "mode"
));
}
#[test]
fn descriptors_cannot_install_with_undiscoverable_oversized_metadata() {
let mut descriptor = descriptor();
descriptor.arguments[0].completion = InvocationCompletion::Static {
values: vec!["a".repeat(1024); 40],
};
assert!(
matches!(descriptor.validate(), Err(InvocationValidationError::InvalidDescriptor(message)) if message.contains("completion byte limit"))
);
}
#[test]
fn client_and_recovery_invocation_key_bound_is_shared() {
let descriptor = descriptor();
assert!(
parse_feature_invocation(
"/run(x)",
0,
&descriptor,
"a".repeat(MAX_FEATURE_INVOCATION_ID_BYTES)
)
.is_ok()
);
assert!(
parse_feature_invocation(
"/run(x)",
0,
&descriptor,
"a".repeat(MAX_FEATURE_INVOCATION_ID_BYTES + 1)
)
.is_err()
);
assert!(parse_feature_invocation("/run(x)", 0, &descriptor, "界".repeat(43)).is_err());
}
#[test]
fn structured_payload_validation_does_not_trust_client_identity_or_types() {
let descriptor = descriptor();
let original = parse_feature_invocation("/run(path=x)", 0, &descriptor, "stable")
.unwrap()
.invocation;
let mut changed = original.clone();
changed.identity = FeatureInvocationIdentity("builtin:other/run".into());
assert_eq!(
validate_feature_invocation(&descriptor, &changed),
Err(InvocationValidationError::IdentityMismatch)
);
changed = original.clone();
changed.name = "unregistered".into();
assert_eq!(
validate_feature_invocation(&descriptor, &changed),
Err(InvocationValidationError::NameMismatch)
);
changed = original.clone();
changed.arguments[0].value = InvocationValue::Boolean(true);
assert!(matches!(
validate_feature_invocation(&descriptor, &changed),
Err(InvocationValidationError::InvalidValue { .. })
));
changed = original.clone();
changed.arguments.push(changed.arguments[0].clone());
assert!(matches!(
validate_feature_invocation(&descriptor, &changed),
Err(InvocationValidationError::DuplicateArgument(_))
));
changed = original;
changed.invocation_id.clear();
assert!(validate_feature_invocation(&descriptor, &changed).is_err());
}
#[test]
fn integer_boolean_and_utf8_offsets_use_the_common_typed_contract() {
let mut descriptor = descriptor();
descriptor.arguments[0].value_type = InvocationArgumentType::Integer;
descriptor.arguments[1].value_type = InvocationArgumentType::Boolean;
let parsed = parse_feature_invocation(
"本文 /execute(-2147483648, mode=true) 後",
"本文 ".len(),
&descriptor,
"typed",
)
.unwrap();
assert_eq!(
parsed.invocation.arguments[0].value,
InvocationValue::Integer(i32::MIN)
);
assert_eq!(
parsed.invocation.arguments[1].value,
InvocationValue::Boolean(true)
);
for invalid in ["/run(2147483648)", "/run(1.0)", "/run(1, mode=yes)"] {
assert!(parse_feature_invocation(invalid, 0, &descriptor, "typed").is_err());
}
assert!(parse_feature_invocation("本文 /run(1)", 1, &descriptor, "typed").is_err());
assert!(parse_feature_invocation("/run(1)", usize::MAX, &descriptor, "typed").is_err());
}
#[test]
fn json_escapes_and_multiple_invocations_have_explicit_boundaries() {
let input = r#"/run("\u8cc7\u6599\/a\b\f", mode=safe) prose /execute("second") end"#;
let first = parse_feature_invocation(input, 0, &descriptor(), "first").unwrap();
assert_eq!(
first.invocation.arguments[0].value,
InvocationValue::String("資料/a\u{8}\u{c}".into())
);
let next = input[first.end..].find("/execute").unwrap() + first.end;
let second = parse_feature_invocation(input, next, &descriptor(), "second").unwrap();
assert_eq!(&input[second.end..], " end");
assert!(matches!(
parse_feature_invocation(r#"/run("\uD800")"#, 0, &descriptor(), "bad"),
Err(InvocationValidationError::Syntax { .. })
));
let value = "controls \u{0}\u{8}\u{c}";
let spelling = format!("/run({})", quote_invocation_string(value));
assert_eq!(
parse_feature_invocation(&spelling, 0, &descriptor(), "control")
.unwrap()
.invocation
.arguments[0]
.value,
InvocationValue::String(value.into())
);
}
#[test]
fn typed_segment_round_trips_without_flattening_identity_or_arguments() {
let parsed =
parse_feature_invocation("/run(\"x y\", mode=fast)", 0, &descriptor(), "1").unwrap();
let segment = crate::Segment::FeatureInvoke {
invocation: parsed.invocation,
};
let json = serde_json::to_string(&segment).unwrap();
assert!(json.contains("\"kind\":\"feature_invoke\""));
assert!(json.contains("builtin:test/run"));
assert_eq!(
serde_json::from_str::<crate::Segment>(&json).unwrap(),
segment
);
}
#[test]
fn quoting_round_trips_spaces_unicode_slashes_and_escapes() {
let value = "a b/界\\\"\n";
let input = format!("/run({})", quote_invocation_string(value));
let parsed = parse_feature_invocation(&input, 0, &descriptor(), "1").unwrap();
assert_eq!(
parsed.invocation.arguments[0].value,
InvocationValue::String(value.into())
);
}
}
+436 -22
View File
@@ -1,4 +1,5 @@
pub mod identity;
pub mod invocation;
#[cfg(feature = "stream")]
pub mod stream;
pub mod subscription;
@@ -10,6 +11,14 @@ use std::path::PathBuf;
use serde::{Deserialize, Serialize};
pub use identity::{WorkerId, WorkerIdParseError};
pub use invocation::{
FeatureInvocation, FeatureInvocationDescriptor, FeatureInvocationIdentity,
FeatureInvocationResult, FeatureInvocationStatus, FeatureInvocationSyntax,
InvocationArgumentDescriptor, InvocationArgumentType, InvocationArgumentValue,
InvocationClientAdapter, InvocationCompletion, InvocationValidationError, InvocationValue,
ParsedFeatureInvocation, parse_feature_invocation, quote_invocation_string,
validate_feature_invocation,
};
/// Allocate an opaque idempotency key for one client Submit request.
pub fn new_submission_request_id() -> String {
@@ -41,6 +50,9 @@ pub enum AuthenticatedInputSource {
/// Assigned whenever a serialized tracked method crosses an untrusted
/// protocol boundary. Receivers must handle it exactly like public input.
UntrustedWire,
/// Stamped only by the operator-owned standalone in-process transport.
/// This is not a Backend Account or authority accepted from wire fields.
LocalOperator,
Account {
account_id: String,
},
@@ -62,10 +74,22 @@ impl Default for AuthenticatedInputSource {
}
}
// Tracked methods may carry serialized transport provenance, but wire values
// never grant authority. Keep this field recognized under fail-closed decoding
// while replacing it with UntrustedWire, including malformed/forged values.
fn ignore_wire_input_source<'de, D>(deserializer: D) -> Result<AuthenticatedInputSource, D::Error>
where
D: serde::Deserializer<'de>,
{
let _ = serde::de::IgnoredAny::deserialize(deserializer)?;
Ok(AuthenticatedInputSource::UntrustedWire)
}
impl AuthenticatedInputSource {
pub fn namespace(&self) -> String {
match self {
Self::UntrustedWire => "untrusted-wire".into(),
Self::LocalOperator => "local-operator".into(),
Self::Account { account_id } => format!("account:{account_id}"),
Self::Worker {
runtime_id,
@@ -175,6 +199,11 @@ pub struct WorkerStateSnapshot {
schemars(range(min = 0, max = 9_007_199_254_740_991_u64))
)]
pub last_command_id: u64,
/// Most recent Submit request whose controller Run has returned and whose
/// post-Run state transition has been published. This is a teardown fence,
/// not a successful-result claim; cancelled and failed Runs may also finish.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub last_finished_submission_request_id: Option<String>,
pub state: WorkerState,
}
@@ -182,6 +211,7 @@ impl WorkerStateSnapshot {
pub fn initial() -> Self {
Self {
last_command_id: 0,
last_finished_submission_request_id: None,
state: WorkerState::Idle,
}
}
@@ -209,6 +239,7 @@ impl From<WorkerStatus> for WorkerStateSnapshot {
};
Self {
last_command_id: 0,
last_finished_submission_request_id: None,
state,
}
}
@@ -217,7 +248,12 @@ impl From<WorkerStatus> for WorkerStateSnapshot {
#[derive(Debug, Clone, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "method", content = "params", rename_all = "snake_case")]
#[serde(
tag = "method",
content = "params",
rename_all = "snake_case",
deny_unknown_fields
)]
pub enum Method {
/// Durably accept typed input for immediate activation or the session FIFO.
/// Running or Paused Workers queue Submit for a later turn; use Notify for
@@ -237,7 +273,17 @@ pub enum Method {
SubmitTracked {
submission_request_id: String,
input: Vec<Segment>,
#[serde(skip_deserializing, default)]
#[serde(default, deserialize_with = "ignore_wire_input_source")]
source: AuthenticatedInputSource,
},
/// Agent-controlled input: accept only at an Idle controller boundary, never
/// enqueue behind a run. Unlike human Submit, non-Idle admission is rejected.
#[cfg_attr(feature = "typescript", ts(skip))]
#[cfg_attr(feature = "json-schema", schemars(skip))]
SubmitIfIdle {
submission_request_id: String,
input: Vec<Segment>,
#[serde(default, deserialize_with = "ignore_wire_input_source")]
source: AuthenticatedInputSource,
},
/// Human-readable text injected into the target Worker's LLM context as a
@@ -255,7 +301,7 @@ pub enum Method {
NotifyTracked {
notification_request_id: String,
message: String,
#[serde(skip_deserializing, default)]
#[serde(default, deserialize_with = "ignore_wire_input_source")]
source: AuthenticatedInputSource,
},
/// Typed lifecycle report from a child Worker to its direct parent.
@@ -319,6 +365,12 @@ pub enum Method {
ListCompletions {
kind: CompletionKind,
prefix: String,
/// Client query generation. Echoed verbatim so identical-prefix ABA
/// responses cannot cross edits, target switches, or permission changes.
#[serde(default, skip_serializing_if = "Option::is_none")]
request_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
context: Option<CompletionContext>,
},
/// List Workers visible to this Worker from durable Worker state and the spawned-child
/// registry. This is not a host-wide Worker universe query.
@@ -592,6 +644,10 @@ pub enum Segment {
/// `[Dir: <path>]` listings; the flattened user text keeps the literal
/// `@<path>` placeholder either way.
FileRef { path: String },
/// Explicitly selected, source-qualified Feature invocation. The host never
/// promotes free text to this variant and validates it against the installed
/// declaration before execution.
FeatureInvoke { invocation: FeatureInvocation },
/// Source-qualified request for the host to start a Flow before committing
/// this user input. Runtime resolves it through Workspace authority and
/// replaces it with the entered state's instructions before Worker-side
@@ -660,6 +716,9 @@ impl Segment {
out.push('@');
out.push_str(path);
}
Segment::FeatureInvoke { invocation } => {
out.push_str(&invocation.display_input());
}
Segment::Flow { selector } => {
out.push_str("[Flow: ");
out.push_str(selector);
@@ -756,6 +815,10 @@ pub struct InternalWorkerSnapshot {
pub worker: InternalWorkerRef,
pub revision: u64,
pub session: SessionSnapshot,
/// Public execution metadata for this child. Older snapshots and service
/// workers may omit it; clients must never substitute the parent's values.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub greeting: Option<Greeting>,
#[serde(default)]
pub status: WorkerStatus,
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -919,6 +982,32 @@ pub struct SessionSnapshotEntry {
pub data: SessionSnapshotEntryData,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum RunYieldReason {
Compaction,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum RunResumeSource {
Compaction,
Pause,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum RunFailureKind {
Engine,
Compaction,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -952,8 +1041,43 @@ pub enum SessionSnapshotEntryData {
#[cfg_attr(feature = "typescript", ts(type = "unknown"))]
data: Option<serde_json::Value>,
},
/// Durable, non-terminal control return from one logical Run. This is not a
/// completed run and must not be used to infer the current Worker state.
RunYielded {
reason: RunYieldReason,
active_run_turn_count: usize,
},
/// Durable boundary immediately before Engine execution resumes. The source
/// distinguishes automatic compaction continuation from an intentional
/// user-pause resume; it does not prove that later model output occurred.
RunResumed {
source: RunResumeSource,
active_run_turn_count: usize,
},
/// Snapshot-only replay of persisted invocation/usage records. These are
/// accounting metadata, never model context or conversation rows.
Invoke {
trigger: InvokeKind,
},
Usage {
input_tokens: u64,
cache_read_input_tokens: u64,
output_tokens: u64,
},
/// Display-only accounting reconstructed from persisted run/usage records.
RunStats {
elapsed_ms: u64,
requests: u64,
upload_tokens: u64,
output_tokens: u64,
},
/// Durable terminal for an intentional cancellation. Consumers must not
/// present this lifecycle result as an execution failure.
RunCancelled,
RunError {
message: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
failure: Option<RunFailureKind>,
},
}
@@ -979,10 +1103,11 @@ pub enum SessionContentPart {
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct SessionToolAttachment {
/// Stable, bounded identity derived from the owning Session entry, attachment
/// position, and immutable image bytes. It is a locator, not a bearer secret.
pub attachment_id: String,
pub media_type: String,
/// Base64-encoded durable attachment body. Public snapshots preserve the
/// committed multimodal value instead of replacing it with placeholder text.
pub data_base64: String,
pub byte_len: u64,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -1005,6 +1130,18 @@ pub enum Event {
submission_request_id: String,
message: String,
},
/// Durable notification acceptance. The request identity is persisted with
/// the notification receipt before this event is emitted. Repeating the
/// same request id and exact payload returns the same acknowledgement
/// without appending the notification twice.
NotificationAccepted {
notification_request_id: String,
},
/// Correlated notification rejection before durable acceptance.
NotificationRejected {
notification_request_id: String,
message: String,
},
/// Revisioned FIFO replacement following enqueue, activation, cancel, or clear.
PendingSubmissionsChanged {
pending: PendingSubmissionsSnapshot,
@@ -1036,6 +1173,9 @@ pub enum Event {
/// One event per `LogEntry::AnnotatedSystemItem` commit. Disk-side and
/// wire-side are 1:1.
SystemItem {
/// Stable durable history identity. Synthetic compatibility events omit it.
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<String>,
#[cfg_attr(feature = "typescript", ts(type = "unknown"))]
item: serde_json::Value,
},
@@ -1054,6 +1194,9 @@ pub enum Event {
/// following `UserMessage` / `SystemItem` event.
InvokeStart {
kind: InvokeKind,
/// Timestamp of the persisted Invoke, not the client's attach time.
#[serde(default, skip_serializing_if = "Option::is_none")]
timestamp_ms: Option<u64>,
},
/// One AgentTurn boundary opened. An AgentTurn is a maximal run of
/// LLM generation calls whose input messages are identical (i.e.
@@ -1170,11 +1313,24 @@ pub enum Event {
/// actually paid full price to send on this request, which is what
/// the TUI status line accumulates per turn.
Usage {
/// Present for usage delivered from the committed session-log lane.
#[serde(default, skip_serializing_if = "Option::is_none")]
timestamp_ms: Option<u64>,
input_tokens: Option<u64>,
output_tokens: Option<u64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
cache_read_input_tokens: Option<u64>,
},
/// Authoritative current-context occupancy after a non-request boundary.
///
/// Unlike [`Event::Usage`], this is not cumulative request traffic. It lets
/// connected clients converge with reconnect snapshots after run completion,
/// compaction, rewind, and other history rewrites. `None` explicitly means
/// the current value is unavailable and stale occupancy must be cleared.
ContextUsage {
#[serde(default, skip_serializing_if = "Option::is_none")]
usage: Option<ContextUsage>,
},
RunEnd {
result: RunResult,
},
@@ -1257,6 +1413,12 @@ pub enum Event {
/// wired up.
Completions {
kind: CompletionKind,
#[serde(default)]
prefix: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
request_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
context: Option<CompletionContext>,
entries: Vec<CompletionEntry>,
},
/// Reply to `Method::ListRewindTargets`. Clients should only open a picker
@@ -1369,25 +1531,42 @@ pub enum AlertSource {
/// Kind of completion requested by `Method::ListCompletions`.
///
/// Mirrors the completion prefix sigil: `@` → `File`.
/// `File` is the existing `@` path lane. `Feature` lists installed chat
/// invocations. `FeatureArgument` resolves a declared argument using the
/// source-qualified [`CompletionContext`].
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum CompletionKind {
File,
Feature,
FeatureArgument,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct CompletionContext {
pub invocation: FeatureInvocationIdentity,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub argument: Option<String>,
}
/// One completion candidate for a prefix query.
///
/// `value` is a path (file kind).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct CompletionEntry {
pub value: String,
#[serde(default)]
pub is_dir: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub usage: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub invocation: Option<FeatureInvocationDescriptor>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -1601,11 +1780,45 @@ impl InFlightToolCallState {
}
}
/// Worker self-description rendered by the TUI when a session starts empty.
/// Public reasoning configuration resolved for the model client.
///
/// Built once in the Worker controller from the resolved manifest and
/// transmitted alongside `Event::Snapshot` so clients don't need
/// their own view of the manifest.
/// This deliberately exposes only the effective provider control, not the
/// Worker manifest or provider credentials. The tagged shape prevents token
/// budgets from being presented as effort labels.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum ReasoningConfig {
Effort { effort: String },
BudgetTokens { budget_tokens: i32 },
}
/// Provenance of the current-context token count in a [`Greeting`].
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(rename_all = "snake_case")]
pub enum ContextTokenSource {
/// Provider-reported input occupancy from the most recent LLM request.
Measured,
/// Current history projected from the Worker's token accounting records.
Estimated,
}
/// Current prompt-context occupancy, distinct from cumulative run traffic.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct ContextUsage {
pub tokens: u64,
pub source: ContextTokenSource,
}
/// Worker self-description rendered by clients when a session starts empty.
///
/// Snapshot metadata contains only resolved public execution information so
/// clients do not need their own copy of the manifest.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
@@ -1614,14 +1827,22 @@ pub struct Greeting {
pub cwd: String,
pub provider: String,
pub model: String,
/// Effective reasoning control. Absent means unspecified or unavailable.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub reasoning: Option<ReasoningConfig>,
pub scope_summary: String,
pub tools: Vec<String>,
/// Model context window in tokens. Always filled by the Worker greeting.
/// Resolved effective model context window in tokens. Legacy producers use
/// zero when this is unavailable; clients must treat that as unknown.
#[serde(default)]
pub context_window: u64,
/// Estimated current session context tokens at connect time.
/// Legacy numeric mirror retained for mixed-version clients.
#[serde(default)]
pub context_tokens: u64,
/// Typed current-context value. Older snapshots omit it; clients may fall
/// back to `context_tokens` only when `context_window` is known.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context_usage: Option<ContextUsage>,
}
// ---------------------------------------------------------------------------
@@ -1684,6 +1905,9 @@ pub enum RunResult {
Finished,
Paused,
LimitReached,
/// The logical Run ended because the user intentionally cancelled it after
/// observable work. Unlike `RolledBack`, committed history is retained.
Cancelled,
/// The accepted Method::Submit produced no assistant/tool output before
/// user interruption, so the Worker rolled the submit-time turn state back
/// to its pre-submit snapshot. Clients should treat the Worker as Idle and
@@ -1797,6 +2021,7 @@ mod tests {
fn worker_state_snapshot_wire_shape_has_one_authoritative_state() {
let snapshot = WorkerStateSnapshot {
last_command_id: 7,
last_finished_submission_request_id: Some("request-7".to_string()),
state: WorkerState::Busy(WorkerBusyState::Run(WorkerRunState::Running)),
};
let value = serde_json::to_value(&snapshot).unwrap();
@@ -1804,6 +2029,7 @@ mod tests {
value,
serde_json::json!({
"last_command_id": 7,
"last_finished_submission_request_id": "request-7",
"state": {
"kind": "busy",
"state": { "kind": "run", "state": "running" }
@@ -1976,6 +2202,49 @@ mod tests {
);
}
#[test]
fn local_operator_source_cannot_be_granted_by_serialized_input() {
let method = Method::SubmitTracked {
input: vec![Segment::text("forged local operator")],
submission_request_id: "local-forged".into(),
source: AuthenticatedInputSource::LocalOperator,
};
let decoded: Method =
serde_json::from_str(&serde_json::to_string(&method).unwrap()).unwrap();
assert!(matches!(
decoded,
Method::SubmitTracked {
source: AuthenticatedInputSource::UntrustedWire,
..
}
));
assert_ne!(
AuthenticatedInputSource::LocalOperator.namespace(),
AuthenticatedInputSource::UntrustedWire.namespace()
);
}
#[test]
fn notify_rejects_typed_intent_instead_of_silently_discarding_it() {
for method in ["notify", "notify_tracked"] {
for key in ["input", "segments", "invocation"] {
let mut params =
serde_json::json!({"notification_request_id":"request", "message":"advisory"});
params[key] = serde_json::json!([{ "kind":"feature_invoke", "invocation": { "identity":"builtin:test/run" } }]);
assert!(
serde_json::from_value::<Method>(
serde_json::json!({"method":method,"params":params})
)
.is_err()
);
}
}
// Text is still text: no decoding or Host scan promotes it to an invoke.
assert!(
matches!(serde_json::from_value::<Method>(serde_json::json!({"method":"notify", "params":{"notification_request_id":"text","message":"/run()"}})).unwrap(), Method::Notify { message, .. } if message == "/run()")
);
}
#[test]
fn segment_unknown_variant_decodes_as_unknown() {
// A future client sends a segment kind this Worker has never heard of.
@@ -2089,19 +2358,29 @@ mod tests {
InvokeKind::SystemReminder,
InvokeKind::Wakeup,
] {
let event = Event::InvokeStart { kind };
let event = Event::InvokeStart {
kind,
timestamp_ms: Some(1234),
};
let json = serde_json::to_string(&event).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["event"], "invoke_start");
let decoded: Event = serde_json::from_str(&json).unwrap();
match decoded {
Event::InvokeStart { kind: k } => assert_eq!(k, kind),
Event::InvokeStart {
kind: k,
timestamp_ms,
} => {
assert_eq!(k, kind);
assert_eq!(timestamp_ms, Some(1234));
}
other => panic!("expected InvokeStart, got {other:?}"),
}
}
let parsed: serde_json::Value = serde_json::from_str(
&serde_json::to_string(&Event::InvokeStart {
kind: InvokeKind::UserSend,
timestamp_ms: None,
})
.unwrap(),
)
@@ -2214,6 +2493,8 @@ mod tests {
let method = Method::ListCompletions {
kind: CompletionKind::File,
prefix: "src/".into(),
request_id: None,
context: None,
};
let json = serde_json::to_string(&method).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
@@ -2223,7 +2504,7 @@ mod tests {
let decoded: Method = serde_json::from_str(&json).unwrap();
match decoded {
Method::ListCompletions { kind, prefix } => {
Method::ListCompletions { kind, prefix, .. } => {
assert_eq!(kind, CompletionKind::File);
assert_eq!(prefix, "src/");
}
@@ -2231,13 +2512,52 @@ mod tests {
}
}
#[test]
fn completion_generation_round_trips_for_identical_query_correlation() {
let context = CompletionContext {
invocation: FeatureInvocationIdentity("builtin:test/run".into()),
argument: Some("mode".into()),
};
for request_id in ["query-1", "query-2"] {
let method = Method::ListCompletions {
kind: CompletionKind::FeatureArgument,
prefix: "s".into(),
request_id: Some(request_id.into()),
context: Some(context.clone()),
};
let decoded: Method =
serde_json::from_str(&serde_json::to_string(&method).unwrap()).unwrap();
assert!(
matches!(decoded, Method::ListCompletions { request_id: Some(id), context: Some(restored), .. } if id == request_id && restored == context)
);
let event = Event::Completions {
kind: CompletionKind::FeatureArgument,
prefix: "s".into(),
request_id: Some(request_id.into()),
context: Some(context.clone()),
entries: Vec::new(),
};
let decoded: Event =
serde_json::from_str(&serde_json::to_string(&event).unwrap()).unwrap();
assert!(
matches!(decoded, Event::Completions { request_id: Some(id), context: Some(restored), .. } if id == request_id && restored == context)
);
}
}
#[test]
fn event_completions_format_and_default_is_dir() {
let event = Event::Completions {
kind: CompletionKind::File,
prefix: "src/".into(),
request_id: None,
context: None,
entries: vec![CompletionEntry {
value: "src/main.rs".into(),
is_dir: false,
description: None,
usage: None,
invocation: None,
}],
};
let json = serde_json::to_string(&event).unwrap();
@@ -2251,7 +2571,7 @@ mod tests {
r#"{"event":"completions","data":{"kind":"file","entries":[{"value":"main.rs"}]}}"#;
let decoded: Event = serde_json::from_str(inbound).unwrap();
match decoded {
Event::Completions { kind, entries } => {
Event::Completions { kind, entries, .. } => {
assert_eq!(kind, CompletionKind::File);
assert_eq!(entries.len(), 1);
assert_eq!(entries[0].value, "main.rs");
@@ -2285,6 +2605,11 @@ mod tests {
tools: vec!["Read".into()],
context_window: 200_000,
context_tokens: 42_000,
reasoning: Some(ReasoningConfig::BudgetTokens { budget_tokens: -1 }),
context_usage: Some(ContextUsage {
tokens: 42_000,
source: ContextTokenSource::Measured,
}),
},
state: WorkerStatus::Paused.into(),
in_flight: InFlightSnapshot::default(),
@@ -2303,16 +2628,63 @@ mod tests {
assert_eq!(parsed["data"]["greeting"]["tools"][0], "Read");
assert_eq!(parsed["data"]["greeting"]["context_window"], 200_000);
assert_eq!(parsed["data"]["greeting"]["context_tokens"], 42_000);
assert_eq!(
parsed["data"]["greeting"]["reasoning"]["kind"],
"budget_tokens"
);
assert_eq!(parsed["data"]["greeting"]["reasoning"]["budget_tokens"], -1);
assert_eq!(
parsed["data"]["greeting"]["context_usage"]["tokens"],
42_000
);
assert_eq!(
parsed["data"]["greeting"]["context_usage"]["source"],
"measured"
);
assert_eq!(parsed["data"]["state"]["state"]["kind"], "busy");
assert_eq!(parsed["data"]["state"]["state"]["state"]["state"], "paused");
}
#[test]
fn context_usage_event_roundtrips_typed_value_and_unknown() {
let event = Event::ContextUsage {
usage: Some(ContextUsage {
tokens: 25_000,
source: ContextTokenSource::Estimated,
}),
};
let json = serde_json::to_string(&event).unwrap();
assert_eq!(
serde_json::from_str::<serde_json::Value>(&json).unwrap(),
serde_json::json!({
"event": "context_usage",
"data": {
"usage": { "tokens": 25_000, "source": "estimated" }
}
})
);
assert!(matches!(
serde_json::from_str::<Event>(r#"{"event":"context_usage","data":{}}"#).unwrap(),
Event::ContextUsage { usage: None }
));
}
#[test]
fn event_snapshot_in_flight_roundtrip_and_default() {
let inbound = r#"{"event":"snapshot","data":{"session":{"entries":[]},"greeting":{"worker_name":"test","cwd":"/tmp","provider":"p","model":"m","scope_summary":"s","tools":[]},"state":{"last_command_id":0,"state":{"kind":"busy","state":{"kind":"run","state":"running"}}}}}"#;
let decoded: Event = serde_json::from_str(inbound).unwrap();
match decoded {
Event::Snapshot { in_flight, .. } => assert!(in_flight.is_empty()),
Event::Snapshot {
greeting,
in_flight,
..
} => {
assert!(in_flight.is_empty());
assert_eq!(greeting.reasoning, None);
assert_eq!(greeting.context_usage, None);
assert_eq!(greeting.context_window, 0);
assert_eq!(greeting.context_tokens, 0);
}
other => panic!("expected Snapshot, got {other:?}"),
}
@@ -2330,6 +2702,8 @@ mod tests {
tools: Vec::new(),
context_window: 0,
context_tokens: 0,
reasoning: None,
context_usage: None,
},
state: WorkerStatus::Running.into(),
in_flight: InFlightSnapshot {
@@ -2436,17 +2810,29 @@ mod tests {
#[test]
fn event_system_item_roundtrip() {
let event = Event::SystemItem {
entry_id: Some("entry-1".to_string()),
item: serde_json::json!({"kind": "notification", "message": "hello"}),
};
let json = serde_json::to_string(&event).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["event"], "system_item");
assert_eq!(parsed["data"]["entry_id"], "entry-1");
assert_eq!(parsed["data"]["item"]["kind"], "notification");
let decoded: Event = serde_json::from_str(&json).unwrap();
match decoded {
Event::SystemItem { item } => assert_eq!(item["kind"], "notification"),
Event::SystemItem { entry_id, item } => {
assert_eq!(entry_id.as_deref(), Some("entry-1"));
assert_eq!(item["kind"], "notification");
}
other => panic!("expected SystemItem, got {other:?}"),
}
let legacy: Event = serde_json::from_value(serde_json::json!({
"event": "system_item",
"data": { "item": { "kind": "notification", "message": "legacy" } }
}))
.unwrap();
assert!(matches!(legacy, Event::SystemItem { entry_id: None, .. }));
}
#[test]
@@ -2454,6 +2840,7 @@ mod tests {
let event = Event::WorkerState {
snapshot: WorkerStateSnapshot {
last_command_id: 9,
last_finished_submission_request_id: None,
state: WorkerState::Busy(WorkerBusyState::Run(WorkerRunState::Running)),
},
};
@@ -2475,6 +2862,7 @@ mod tests {
snapshot: WorkerStateSnapshot {
last_command_id: 9,
state: WorkerState::Busy(WorkerBusyState::Run(WorkerRunState::Running)),
..
}
}
));
@@ -2809,6 +3197,32 @@ mod tests {
}
}
#[test]
fn notification_receipt_events_roundtrip_with_request_identity() {
for event in [
Event::NotificationAccepted {
notification_request_id: "notification-1".into(),
},
Event::NotificationRejected {
notification_request_id: "notification-1".into(),
message: "invalid notification".into(),
},
] {
let json = serde_json::to_string(&event).unwrap();
let decoded: Event = serde_json::from_str(&json).unwrap();
match decoded {
Event::NotificationAccepted {
notification_request_id,
}
| Event::NotificationRejected {
notification_request_id,
..
} => assert_eq!(notification_request_id, "notification-1"),
other => panic!("expected notification receipt, got {other:?}"),
}
}
}
#[test]
fn event_error_format() {
let event = Event::Error {
+43 -14
View File
@@ -12,7 +12,7 @@ use serde::{Deserialize, Deserializer, Serialize};
use crate::Event as WorkerProtocolEvent;
pub const SUBSCRIPTION_PROTOCOL_VERSION: u16 = 1;
pub const SUBSCRIPTION_PROTOCOL_VERSION: u16 = 2;
/// Browser/runtime wire guardrails. These bounds are generated into the Workspace
/// Browser validator so aggregate limits remain part of the Rust-owned contract.
pub const MAX_SUBSCRIPTION_FRAME_JSON_BYTES: usize = 16 * 1024 * 1024;
@@ -426,7 +426,6 @@ pub enum SubscriptionResponse {
request_id: SubscriptionRequestId,
subscription_id: SubscriptionId,
selector: EventSubscriptionSelector,
snapshot_revision: u64,
snapshot: SubscriptionSnapshot,
},
Unsubscribed {
@@ -512,7 +511,6 @@ pub enum SubscriptionTerminationCode {
pub enum SubscriptionEvent {
Event {
subscription_id: SubscriptionId,
subject_revision: u64,
payload: SubscriptionEventPayload,
},
SubscriptionClosed {
@@ -631,10 +629,32 @@ impl SubscriptionWorkerWorkdirAttachment {
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
#[serde(deny_unknown_fields)]
pub struct SubscriptionWorkerJob {
pub job_id: String,
pub attempt_id: String,
pub purpose: String,
}
impl SubscriptionWorkerJob {
pub fn validate(&self) -> Result<(), SubscriptionValidationError> {
validate_identifier("job_id", &self.job_id, MAX_RESOURCE_ID_BYTES)?;
validate_identifier("attempt_id", &self.attempt_id, MAX_RESOURCE_ID_BYTES)?;
validate_identifier("job_purpose", &self.purpose, MAX_RESOURCE_ID_BYTES)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[cfg_attr(feature = "json-schema", derive(schemars::JsonSchema))]
pub struct SubscriptionWorker {
/// Opaque Runtime-owned Restore observation identity, never a display-state hash.
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "typescript", ts(optional = nullable))]
pub restore_observation_token: Option<String>,
pub worker_id: SubscriptionWorkerId,
/// Set by the Workspace Server when projecting a Runtime-owned Worker to clients.
/// Runtime producers leave this unset because the connection identifies the Runtime.
@@ -648,9 +668,7 @@ pub struct SubscriptionWorker {
/// `Unavailable` preserves catalog membership without claiming that execution stopped.
#[serde(default)]
pub availability: SubscriptionWorkerAvailability,
/// Producer-owned monotonic revision for this Worker subject.
pub subject_revision: u64,
/// Latest revisioned foreground state observed from the Worker. This remains
/// Latest foreground state observed from the Worker. This remains
/// absent until an authoritative Worker snapshot/event has been applied.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub worker_state: Option<crate::WorkerStateSnapshot>,
@@ -664,6 +682,10 @@ pub struct SubscriptionWorker {
pub display_name: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub profile: Option<String>,
/// Backend-owned Job binding. Runtime producers leave this unset; the
/// Workspace projection enriches both snapshots and live updates.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub job: Option<SubscriptionWorkerJob>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub workdir_attachments: Vec<SubscriptionWorkerWorkdirAttachment>,
}
@@ -671,12 +693,18 @@ pub struct SubscriptionWorker {
impl SubscriptionWorker {
pub fn validate(&self) -> Result<(), SubscriptionValidationError> {
self.worker_id.validate()?;
if let Some(token) = &self.restore_observation_token {
validate_identifier("restore_observation_token", token, MAX_RESOURCE_ID_BYTES)?;
}
if let Some(runtime_id) = &self.runtime_id {
validate_identifier("runtime_id", runtime_id, MAX_RESOURCE_ID_BYTES)?;
}
if let Some(resource_key) = &self.resource_key {
validate_identifier("resource_key", resource_key, MAX_RESOURCE_ID_BYTES)?;
}
if let Some(job) = &self.job {
job.validate()?;
}
let mut aliases = HashSet::new();
let mut workdir_ids = HashSet::new();
for attachment in &self.workdir_attachments {
@@ -941,17 +969,18 @@ mod tests {
fn worker(value: &str) -> SubscriptionWorker {
SubscriptionWorker {
restore_observation_token: None,
worker_id: worker_id(value),
runtime_id: None,
resource_key: None,
availability: SubscriptionWorkerAvailability::Observed,
subject_revision: 0,
worker_state: None,
state: SubscriptionWorkerState::Idle,
has_running_internal_workers: false,
workspace_id: Some("workspace-1".to_string()),
display_name: Some(format!("Worker {value}")),
profile: Some("builtin:coder".to_string()),
job: None,
workdir_attachments: Vec::new(),
}
}
@@ -1067,7 +1096,7 @@ mod tests {
assert_eq!(
json,
serde_json::json!({
"protocol_version": 1,
"protocol_version": 2,
"frame": "request",
"message": {
"method": "subscribe_events",
@@ -1093,7 +1122,6 @@ mod tests {
request_id: request_id(),
subscription_id: subscription_id(),
selector: EventSubscriptionSelector::RuntimeWorkers,
snapshot_revision: 7,
snapshot: SubscriptionSnapshot::Workers {
workers: vec![worker("worker-1")],
},
@@ -1132,10 +1160,10 @@ mod tests {
subscription_id: subscription_id(),
},
));
frame.protocol_version = 2;
frame.protocol_version = 1;
assert!(matches!(
frame.validate(),
Err(SubscriptionValidationError::UnsupportedProtocolVersion { actual: 2 })
Err(SubscriptionValidationError::UnsupportedProtocolVersion { actual: 1 })
));
}
@@ -1154,7 +1182,6 @@ mod tests {
let event = SubscriptionEvent::Event {
subscription_id: subscription_id(),
subject_revision: 8,
payload: SubscriptionEventPayload::WorkerRemoved {
worker_id: worker_id("worker-2"),
runtime_id: None,
@@ -1178,6 +1205,8 @@ mod tests {
method: crate::Method::ListCompletions {
kind: crate::CompletionKind::File,
prefix: "src/".to_string(),
request_id: None,
context: None,
},
},
));
@@ -1185,7 +1214,7 @@ mod tests {
assert_eq!(
serde_json::to_value(frame).unwrap(),
serde_json::json!({
"protocol_version": 1,
"protocol_version": 2,
"frame": "worker_protocol",
"message": {
"subscription_id": "subscription-1",
@@ -1227,7 +1256,7 @@ mod tests {
assert_eq!(
serde_json::to_value(frame).unwrap(),
serde_json::json!({
"protocol_version": 1,
"protocol_version": 2,
"frame": "event",
"message": {
"event": "subscription_closed",
+96 -21
View File
@@ -6,18 +6,23 @@ use ts_rs::{Config, TS};
use crate::{
Alert, AlertLevel, AlertSource, CommandEvent, CommandSnapshot, CommandStatus, CommandStream,
CommandStreamSlice, CompactionLifecycle, CompactionLifecycleState, CompactionPhase,
CompactionTrigger, CompletionEntry, CompletionKind, ErrorCode, Event, Greeting, InFlightBlock,
InFlightCompaction, InFlightSnapshot, InFlightToolCallState, InternalWorkerKind,
InternalWorkerRef, InternalWorkerSnapshot, InvokeKind, MemoryWorkerEvent, Method,
PasteArtifactAvailability, PasteArtifactMediaType, PasteArtifactRef, PendingSubmissionSummary,
PendingSubmissionsSnapshot, Permission, RewindSummary, RewindTarget, RewindTargetId, RunResult,
ScopeRule, Segment, SessionContentPart, SessionConversationTurn, SessionEntryProvenance,
SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole, SessionSnapshot,
SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment, SubmissionDisposition,
SymlinkPolicy, ToolResultDisposition, TurnResult, UploadedFileAvailability, UploadedFileRef,
WorkerBusyState, WorkerCommandAcknowledgement, WorkerCommandDisposition, WorkerCommandEnvelope,
WorkerCommandKind, WorkerEvent, WorkerMaintenanceState, WorkerRunState, WorkerState,
WorkerStateSnapshot, WorkerStatus,
CompactionTrigger, CompletionContext, CompletionEntry, CompletionKind, ContextTokenSource,
ContextUsage, ErrorCode, Event, FeatureInvocation, FeatureInvocationDescriptor,
FeatureInvocationIdentity, FeatureInvocationResult, FeatureInvocationStatus,
FeatureInvocationSyntax, Greeting, InFlightBlock, InFlightCompaction, InFlightSnapshot,
InFlightToolCallState, InternalWorkerKind, InternalWorkerRef, InternalWorkerSnapshot,
InvocationArgumentDescriptor, InvocationArgumentType, InvocationArgumentValue,
InvocationClientAdapter, InvocationCompletion, InvocationValue, InvokeKind, MemoryWorkerEvent,
Method, PasteArtifactAvailability, PasteArtifactMediaType, PasteArtifactRef,
PendingSubmissionSummary, PendingSubmissionsSnapshot, Permission, ReasoningConfig,
RewindSummary, RewindTarget, RewindTargetId, RunFailureKind, RunResult, RunResumeSource,
RunYieldReason, ScopeRule, Segment, SessionContentPart, SessionConversationTurn,
SessionEntryProvenance, SessionHistoryLineageBoundary, SessionHistoryPage, SessionMessageRole,
SessionSnapshot, SessionSnapshotEntry, SessionSnapshotEntryData, SessionToolAttachment,
SubmissionDisposition, SymlinkPolicy, ToolResultDisposition, TurnResult,
UploadedFileAvailability, UploadedFileRef, WorkerBusyState, WorkerCommandAcknowledgement,
WorkerCommandDisposition, WorkerCommandEnvelope, WorkerCommandKind, WorkerEvent,
WorkerMaintenanceState, WorkerRunState, WorkerState, WorkerStateSnapshot, WorkerStatus,
subscription::{
EventSubscriptionSelector, MAX_CORRELATION_ID_BYTES, MAX_REJECTION_MESSAGE_BYTES,
MAX_RESOURCE_ID_BYTES, MAX_SUBSCRIPTION_COLLECTION_ITEMS,
@@ -28,7 +33,7 @@ use crate::{
SubscriptionRequest, SubscriptionRequestId, SubscriptionResponse, SubscriptionSnapshot,
SubscriptionTerminationCode, SubscriptionWorkdirId, SubscriptionWorker,
SubscriptionWorkerAvailability, SubscriptionWorkerId, SubscriptionWorkerIds,
SubscriptionWorkerProtocolMethod, SubscriptionWorkerState,
SubscriptionWorkerJob, SubscriptionWorkerProtocolMethod, SubscriptionWorkerState,
SubscriptionWorkerWorkdirAttachment, WorkspaceSubscriptionWorkdir,
},
};
@@ -62,6 +67,19 @@ pub fn generated_protocol_types() -> String {
push_decl::<AlertLevel>(&cfg, &mut output);
push_decl::<AlertSource>(&cfg, &mut output);
push_decl::<CompletionKind>(&cfg, &mut output);
push_decl::<CompletionContext>(&cfg, &mut output);
push_decl::<FeatureInvocationIdentity>(&cfg, &mut output);
push_decl::<FeatureInvocationSyntax>(&cfg, &mut output);
push_decl::<InvocationArgumentType>(&cfg, &mut output);
push_decl::<InvocationCompletion>(&cfg, &mut output);
push_decl::<InvocationArgumentDescriptor>(&cfg, &mut output);
push_decl::<InvocationClientAdapter>(&cfg, &mut output);
push_decl::<FeatureInvocationDescriptor>(&cfg, &mut output);
push_decl::<InvocationValue>(&cfg, &mut output);
push_decl::<InvocationArgumentValue>(&cfg, &mut output);
push_decl::<FeatureInvocation>(&cfg, &mut output);
push_decl::<FeatureInvocationStatus>(&cfg, &mut output);
push_decl::<FeatureInvocationResult>(&cfg, &mut output);
push_decl::<WorkerStatus>(&cfg, &mut output);
push_decl::<WorkerCommandEnvelope>(&cfg, &mut output);
push_decl::<WorkerCommandKind>(&cfg, &mut output);
@@ -103,6 +121,9 @@ pub fn generated_protocol_types() -> String {
push_decl::<SessionMessageRole>(&cfg, &mut output);
push_decl::<SessionContentPart>(&cfg, &mut output);
push_decl::<SessionToolAttachment>(&cfg, &mut output);
push_decl::<RunYieldReason>(&cfg, &mut output);
push_decl::<RunResumeSource>(&cfg, &mut output);
push_decl::<RunFailureKind>(&cfg, &mut output);
push_decl::<SessionSnapshotEntryData>(&cfg, &mut output);
push_decl::<SessionSnapshotEntry>(&cfg, &mut output);
push_decl::<SessionConversationTurn>(&cfg, &mut output);
@@ -112,10 +133,13 @@ pub fn generated_protocol_types() -> String {
push_decl::<PendingSubmissionsSnapshot>(&cfg, &mut output);
push_decl::<SubmissionDisposition>(&cfg, &mut output);
push_decl::<SessionSnapshot>(&cfg, &mut output);
push_decl::<ReasoningConfig>(&cfg, &mut output);
push_decl::<ContextTokenSource>(&cfg, &mut output);
push_decl::<ContextUsage>(&cfg, &mut output);
push_decl::<Greeting>(&cfg, &mut output);
push_decl::<InternalWorkerKind>(&cfg, &mut output);
push_decl::<InternalWorkerRef>(&cfg, &mut output);
push_decl::<InternalWorkerSnapshot>(&cfg, &mut output);
push_decl::<Greeting>(&cfg, &mut output);
push_decl::<Alert>(&cfg, &mut output);
push_decl::<MemoryWorkerEvent>(&cfg, &mut output);
push_decl::<PasteArtifactMediaType>(&cfg, &mut output);
@@ -132,6 +156,7 @@ pub fn generated_protocol_types() -> String {
push_decl::<SubscriptionWorkerAvailability>(&cfg, &mut output);
push_decl::<EventSubscriptionSelector>(&cfg, &mut output);
push_decl::<SubscriptionWorkerWorkdirAttachment>(&cfg, &mut output);
push_decl::<SubscriptionWorkerJob>(&cfg, &mut output);
push_decl::<SubscriptionWorker>(&cfg, &mut output);
push_decl::<WorkspaceSubscriptionWorkdir>(&cfg, &mut output);
push_decl::<SubscriptionSnapshot>(&cfg, &mut output);
@@ -221,17 +246,18 @@ fn rust_serialized_subscription_frame_fixtures() -> Vec<SubscriptionFrame> {
let external_workdir_id = SubscriptionWorkdirId::new("fixture-external-workdir")
.expect("fixture external Workdir id must be valid");
let worker = SubscriptionWorker {
restore_observation_token: Some("fixture-restore-observation".into()),
worker_id: worker_id.clone(),
runtime_id: Some("fixture-runtime".to_string()),
resource_key: Some("fixture-worker-resource".to_string()),
availability: SubscriptionWorkerAvailability::default(),
subject_revision: 1,
worker_state: None,
state: SubscriptionWorkerState::Idle,
has_running_internal_workers: false,
workspace_id: Some("fixture-workspace".to_string()),
display_name: None,
profile: None,
job: None,
workdir_attachments: vec![SubscriptionWorkerWorkdirAttachment {
alias: "external".to_string(),
repository_id: None,
@@ -244,7 +270,6 @@ fn rust_serialized_subscription_frame_fixtures() -> Vec<SubscriptionFrame> {
request_id,
subscription_id: subscription_id.clone(),
selector: EventSubscriptionSelector::WorkspaceWorkers,
snapshot_revision: 1,
snapshot: SubscriptionSnapshot::Workers {
workers: vec![worker],
},
@@ -252,21 +277,71 @@ fn rust_serialized_subscription_frame_fixtures() -> Vec<SubscriptionFrame> {
));
let pending =
SubscriptionFrame::new(SubscriptionFramePayload::Event(SubscriptionEvent::Event {
subscription_id,
subject_revision: 2,
subscription_id: subscription_id.clone(),
payload: SubscriptionEventPayload::WorkerProtocol {
worker_id,
worker_id: worker_id.clone(),
event: Event::PendingSubmissionsChanged {
pending: PendingSubmissionsSnapshot::default(),
},
},
}));
for fixture in [&subscribed, &pending] {
let greeting = Greeting {
worker_name: "fixture-worker".into(),
cwd: "/fixture".into(),
provider: "openai_responses".into(),
model: "gpt-6-astra".into(),
reasoning: Some(ReasoningConfig::Effort {
effort: "high".into(),
}),
scope_summary: "fixture scope".into(),
tools: vec!["Read".into()],
context_window: 272_000,
context_tokens: 142_000,
context_usage: Some(ContextUsage {
tokens: 142_000,
source: ContextTokenSource::Measured,
}),
};
let snapshot =
SubscriptionFrame::new(SubscriptionFramePayload::Event(SubscriptionEvent::Event {
subscription_id,
payload: SubscriptionEventPayload::WorkerProtocol {
worker_id,
event: Event::Snapshot {
session: SessionSnapshot {
pending_submissions: PendingSubmissionsSnapshot::default(),
entries: Vec::new(),
},
greeting: greeting.clone(),
state: WorkerStateSnapshot::initial(),
in_flight: InFlightSnapshot::default(),
internal_workers: vec![InternalWorkerSnapshot {
worker: InternalWorkerRef {
session_id: "fixture-child-session".into(),
name: "fixture-child".into(),
parent_session_id: Some("fixture-parent-session".into()),
kind: InternalWorkerKind::SubWorker,
},
revision: 1,
session: SessionSnapshot {
pending_submissions: PendingSubmissionsSnapshot::default(),
entries: Vec::new(),
},
greeting: Some(greeting),
status: WorkerStatus::Idle,
error: None,
in_flight: InFlightSnapshot::default(),
internal_workers: Vec::new(),
}],
},
},
}));
for fixture in [&subscribed, &pending, &snapshot] {
fixture
.validate()
.expect("Rust-serialized Browser compatibility fixture must validate");
}
vec![subscribed, pending]
vec![subscribed, pending, snapshot]
}
fn close_object_shapes(schema: &mut Value) {
@@ -378,6 +378,7 @@ function validateWorkers(workers: SubscriptionWorker[]): boolean {
}
function validateWorker(worker: SubscriptionWorker): boolean {
if (worker.restore_observation_token != null && !validateIdentifier(worker.restore_observation_token, MAX_RESOURCE_ID_BYTES)) return false;
if (!validateIdentifier(worker.worker_id, MAX_RESOURCE_ID_BYTES)) return false;
if (worker.runtime_id != null && !validateIdentifier(worker.runtime_id, MAX_RESOURCE_ID_BYTES)) {
return false;
+376 -5
View File
@@ -402,6 +402,8 @@ pub struct WorkingDirectoryStatus {
#[serde(rename_all = "snake_case")]
pub enum WorkerInputKind {
User,
/// Agent input accepted only while Idle; never enters the human Submit queue.
UserIfIdle,
Notify,
Compact,
ListRewindTargets,
@@ -418,6 +420,18 @@ pub struct WorkerInput {
pub segments: Option<Vec<Segment>>,
}
/// Host-authored Job/attempt identity, independent of Profile selection.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct BackendJobExecutionBinding {
pub job_id: String,
pub attempt_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub input_revision: Option<String>,
#[serde(default)]
pub subjektiv_consolidation: bool,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
pub struct CreateWorkerRequest {
pub worker_id: WorkerId,
@@ -442,10 +456,18 @@ pub struct CreateWorkerRequest {
pub workspace_api: Option<WorkspaceApiRef>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub memory_settings: Option<manifest::WorkspaceMemorySettingsSnapshot>,
/// Backend-attested subject or consolidation attachment. Profile policy alone
/// never activates subjektiv for an ordinary Workspace Worker.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub subjektiv_attached: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub backend_job: Option<BackendJobExecutionBinding>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
pub struct WorkerSummary {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub restore_observation_token: Option<String>,
pub worker_ref: WorkerRef,
pub worker_id: WorkerId,
pub status: WorkerStatus,
@@ -478,6 +500,62 @@ pub struct WorkerResponse {
pub worker: WorkerDetail,
}
/// Explicit observed Restore intent. Preparation is authored by the Workspace
/// Backend, not browser/Tool JSON, and applied only after Runtime admission.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct WorkerRestoreRequest {
pub expected_observation_token: String,
pub request_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub preparation: Option<WorkerRestorePreparation>,
}
#[derive(Clone, Debug, Default, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct WorkerRestorePreparation {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workspace_api: Option<WorkspaceApiRef>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workdir_attachments: Option<Vec<LogicalWorkdirAttachment>>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access: Vec<WorkingDirectoryRepositoryAccessRequest>,
/// Read-only references resolved by Workspace only after this operation owns admission.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub repository_access_workdirs: Vec<RepositoryAccessWorkdirReference>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct RepositoryAccessWorkdirReference {
pub runtime_id: String,
pub working_directory_id: String,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct WorkerRestoreCoordinationRequest {
pub expected_observation_token: String,
pub request_id: String,
/// None means owner/result recovery only; it can never admit a new intent.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub preparation: Option<WorkerRestorePreparation>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
pub struct WorkerRestoreCoordinationResponse {
pub result: Option<WorkerRestoreResponse>,
/// Original read-only preparation, only when the existing operation still needs it.
pub preparation: Option<WorkerRestorePreparation>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct WorkingDirectoryRepositoryAccessRequest {
pub working_directory_id: String,
pub materialization: RepositoryMaterializationContext,
}
/// Explicit restore outcome returned for every reachable Runtime restore attempt,
/// including preflight rejection and uncertain post-side-effect reconciliation.
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
@@ -561,12 +639,20 @@ pub struct CompletionRequest {
pub kind: CompletionKind,
#[serde(default)]
pub prefix: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context: Option<protocol::CompletionContext>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub request_id: Option<String>,
}
#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
pub struct CompletionResponse {
pub kind: CompletionKind,
pub prefix: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context: Option<protocol::CompletionContext>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub request_id: Option<String>,
pub entries: Vec<CompletionEntry>,
}
@@ -690,6 +776,183 @@ pub enum WorkerSessionHistoryAvailability {
},
}
/// Backend-selected storage source. This type is never exposed as model tool
/// input; Runtime paths remain behind the authenticated Workspace capability.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "storage", rename_all = "snake_case")]
pub enum SessionPublicSource {
Retained { worker_id: WorkerId },
Archived { archive_id: String },
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionPublicEntryKind {
User,
Assistant,
Tool,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionPublicToolPart {
Input,
Output,
Both,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionPublicReadMode {
Compact,
Full,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionPublicLineageKind {
Root,
Fork,
Compact,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionPublicLineage {
pub kind: SessionPublicLineageKind,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub parent_segment_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub at_turn_index: Option<usize>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionPublicSearchRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub expected_generation: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub query: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub kind: Option<SessionPublicEntryKind>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_name: Option<String>,
pub tool_part: SessionPublicToolPart,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub scan_cursor: Option<String>,
pub limit: usize,
pub max_scan_bytes: u64,
pub max_segments: usize,
pub max_entries: usize,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionPublicReadRequest {
pub workspace_id: String,
pub source: SessionPublicSource,
pub expected_session_id: String,
pub expected_generation: Option<String>,
pub segment_id: String,
pub entry_ref: String,
pub mode: SessionPublicReadMode,
pub byte_offset: usize,
pub max_content_bytes: usize,
pub max_scan_bytes: u64,
pub max_segments: usize,
pub max_entries: usize,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionPublicSearchItem {
pub segment_id: String,
pub entry_ref: String,
pub kind: SessionPublicEntryKind,
pub origin: protocol::SessionEntryProvenance,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_name: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_part: Option<SessionPublicToolPart>,
pub compact: String,
pub compact_truncated: bool,
pub lineage: SessionPublicLineage,
/// Opaque Runtime continuation immediately before this result.
pub scan_cursor: String,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionPublicSearchPage {
pub session_id: String,
pub generation: String,
pub items: Vec<SessionPublicSearchItem>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub next_scan_cursor: Option<String>,
pub has_more: bool,
pub scanned_bytes: u64,
pub scanned_segments: usize,
pub scanned_entries: usize,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub archive_manifest: Option<WorkerSessionArchiveManifest>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct SessionPublicReadPage {
pub session_id: String,
pub generation: String,
pub segment_id: String,
pub entry_ref: String,
pub kind: SessionPublicEntryKind,
pub origin: protocol::SessionEntryProvenance,
pub lineage: SessionPublicLineage,
pub mode: SessionPublicReadMode,
pub content: String,
pub next_byte_offset: Option<usize>,
pub has_more: bool,
pub scanned_bytes: u64,
pub scanned_segments: usize,
pub scanned_entries: usize,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub archive_manifest: Option<WorkerSessionArchiveManifest>,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SessionPublicUnavailableReason {
NotFound,
RetentionMissing,
RetentionExpired,
ArchiveIncomplete,
CorruptLog,
MigrationRequired,
StorageUnavailable,
InvalidCursor,
ResourceLimit,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionPublicSearchAvailability {
Page {
page: SessionPublicSearchPage,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "availability", rename_all = "snake_case")]
pub enum SessionPublicReadAvailability {
Page {
page: SessionPublicReadPage,
},
Unavailable {
reason: SessionPublicUnavailableReason,
message: String,
},
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct RetainedSessionIdentity {
pub session_id: String,
@@ -764,6 +1027,18 @@ pub trait RuntimeApi {
#[query] request: WorkerSessionHistoryRequest,
) -> Result<WorkerSessionHistoryAvailability, RuntimeApiError>;
#[post("/v1/session-public/search", status = 200, error_status = 400)]
async fn session_public_search(
&self,
#[body] request: SessionPublicSearchRequest,
) -> Result<SessionPublicSearchAvailability, RuntimeApiError>;
#[post("/v1/session-public/read", status = 200, error_status = 400)]
async fn session_public_read(
&self,
#[body] request: SessionPublicReadRequest,
) -> Result<SessionPublicReadAvailability, RuntimeApiError>;
#[post("/v1/workers", status = 200, error_status = 400)]
async fn create_worker(
&self,
@@ -797,11 +1072,18 @@ pub trait RuntimeApi {
#[body] request: WorkerLifecycleRequest,
) -> Result<WorkerLifecycleResponse, RuntimeApiError>;
#[post("/v1/workers/{worker_id}/restore", status = 200, error_status = 400)]
#[post("/v1/workers/{worker_id}/restore/coordinate", status = 200, error_status = 400, additional_error_statuses = [409])]
async fn coordinate_worker_restore(
&self,
#[path] worker_id: String,
#[body] request: WorkerRestoreCoordinationRequest,
) -> Result<WorkerRestoreCoordinationResponse, RuntimeApiError>;
#[post("/v1/workers/{worker_id}/restore", status = 200, error_status = 400, additional_error_statuses = [409])]
async fn restore_worker(
&self,
#[path] worker_id: String,
#[body] request: EmptyObjectRequest,
#[body] request: WorkerRestoreRequest,
) -> Result<WorkerRestoreResponse, RuntimeApiError>;
#[post(
@@ -890,6 +1172,8 @@ pub const RUNTIME_ROUTE_WORKER_PROTOCOL_WS: &str = "/v1/workers/{worker_id}/prot
pub const RUNTIME_ROUTE_WORKER_ATTACHMENTS: &str = "/v1/workers/{worker_id}/attachments";
pub const RUNTIME_ROUTE_WORKER_ATTACHMENT: &str =
"/v1/workers/{worker_id}/attachments/{artifact_id}";
pub const RUNTIME_ROUTE_WORKER_SESSION_ATTACHMENT: &str =
"/v1/workers/{worker_id}/sessions/{session_id}/attachments/{attachment_id}";
/// Intentionally out-of-contract Runtime routes. This inventory keeps manual paths visible rather
/// than letting them be mistaken for management-contract omissions.
@@ -994,12 +1278,72 @@ pub const REMAINING_RUNTIME_ROUTES: &[RemainingRuntimeRoute] = &[
path: RUNTIME_ROUTE_WORKER_ATTACHMENT,
reason: "Binary attachment transport",
},
RemainingRuntimeRoute {
method: "GET",
path: RUNTIME_ROUTE_WORKER_SESSION_ATTACHMENT,
reason: "Session image attachment transport",
},
];
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn restore_request_is_mandatory_and_conflict_is_not_an_operation_state() {
for invalid in [
serde_json::json!({}),
serde_json::json!({"request_id":"r"}),
serde_json::json!({"expected_observation_token":"t"}),
] {
assert!(serde_json::from_value::<WorkerRestoreRequest>(invalid).is_err());
}
let request: WorkerRestoreRequest = serde_json::from_value(
serde_json::json!({"expected_observation_token":"t", "request_id":"r"}),
)
.unwrap();
assert!(request.preparation.is_none());
for state in [
"accepted",
"rejected",
"rolled_back",
"reconciliation_required",
] {
assert!(serde_json::from_value::<WorkerRestoreState>(serde_json::json!(state)).is_ok());
}
assert!(
serde_json::from_value::<WorkerRestoreState>(serde_json::json!(
"restore_observation_conflict"
))
.is_err()
);
let conflict = RuntimeApiError::new(
409,
"restore_observation_conflict",
"refresh before a new intent",
);
assert_eq!(conflict.status(), 409);
assert_eq!(
serde_json::to_value(conflict).unwrap()["error"]["code"],
"restore_observation_conflict"
);
}
#[test]
fn feature_argument_completion_wire_preserves_source_and_argument() {
let json = serde_json::json!({"kind":"feature_argument", "prefix":"資料/", "context": {"invocation":"builtin:test/prepare", "argument":"path"}});
let request: CompletionRequest = serde_json::from_value(json.clone()).unwrap();
assert_eq!(request.kind, CompletionKind::FeatureArgument);
assert_eq!(
request.context.as_ref().unwrap().argument.as_deref(),
Some("path")
);
assert_eq!(serde_json::to_value(request).unwrap(), json);
let legacy: CompletionRequest =
serde_json::from_value(serde_json::json!({"kind":"file"})).unwrap();
assert!(legacy.context.is_none());
}
#[derive(Clone)]
struct RoundTripService;
@@ -1110,6 +1454,26 @@ mod tests {
})
}
async fn session_public_search(
&self,
_request: SessionPublicSearchRequest,
) -> Result<SessionPublicSearchAvailability, RuntimeApiError> {
Ok(SessionPublicSearchAvailability::Unavailable {
reason: SessionPublicUnavailableReason::StorageUnavailable,
message: "unsupported".to_string(),
})
}
async fn session_public_read(
&self,
_request: SessionPublicReadRequest,
) -> Result<SessionPublicReadAvailability, RuntimeApiError> {
Ok(SessionPublicReadAvailability::Unavailable {
reason: SessionPublicUnavailableReason::StorageUnavailable,
message: "unsupported".to_string(),
})
}
async fn create_worker(
&self,
_request: CreateWorkerRequest,
@@ -1143,10 +1507,17 @@ mod tests {
) -> Result<WorkerLifecycleResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn coordinate_worker_restore(
&self,
_worker_id: String,
_request: WorkerRestoreCoordinationRequest,
) -> Result<WorkerRestoreCoordinationResponse, RuntimeApiError> {
Err(test_error(501))
}
async fn restore_worker(
&self,
_worker_id: String,
_request: EmptyObjectRequest,
_request: WorkerRestoreRequest,
) -> Result<WorkerRestoreResponse, RuntimeApiError> {
Err(test_error(501))
}
@@ -1246,7 +1617,7 @@ mod tests {
#[test]
fn contract_inventory_is_complete_and_unique() {
let operations = RuntimeApiMetadata::OPERATIONS;
assert_eq!(operations.len(), 17);
assert_eq!(operations.len(), 20);
let mut routes = operations
.iter()
.map(|operation| (format!("{:?}", operation.method), operation.path))
@@ -1263,7 +1634,7 @@ mod tests {
#[test]
fn remaining_route_inventory_does_not_overlap_contract() {
assert_eq!(REMAINING_RUNTIME_ROUTES.len(), 20);
assert_eq!(REMAINING_RUNTIME_ROUTES.len(), 21);
for remaining in REMAINING_RUNTIME_ROUTES {
assert!(!RuntimeApiMetadata::OPERATIONS.iter().any(|operation| {
format!("{:?}", operation.method).eq_ignore_ascii_case(remaining.method)
+4
View File
@@ -55,3 +55,7 @@ required-features = ["typescript"]
[[example]]
name = "generate_repository_access_types"
required-features = ["typescript"]
[[example]]
name = "generate_drive_api_types"
required-features = ["typescript"]
@@ -0,0 +1,6 @@
fn main() {
print!(
"{}",
server_api::normalized_typescript(server_api::drive_api_typescript())
);
}
+902
View File
@@ -0,0 +1,902 @@
//! Workspace Drive HTTP contracts. Entries are logical, workspace-bound resources,
//! never host paths. Worker identity and grants are resolved by the service from
//! trusted request context, not supplied by ordinary operation DTOs.
use schemars::JsonSchema;
use serde::{Deserialize, Deserializer, Serialize};
pub const DRIVE_TEXT_MAX_BYTES: usize = 64 * 1024;
pub const DRIVE_CHUNK_MAX_BYTES: u32 = 64 * 1024;
pub const DRIVE_PAGE_MAX_LIMIT: u32 = 200;
pub const DRIVE_SEARCH_MAX_LIMIT: u32 = 128;
pub const DRIVE_FILE_MAX_BYTES: u32 = 16 * 1024 * 1024;
fn decimal<'de, D: Deserializer<'de>>(deserializer: D) -> Result<String, D::Error> {
let value = String::deserialize(deserializer)?;
if !canonical_positive_decimal(&value) {
return Err(serde::de::Error::custom(
"expected a canonical positive decimal string",
));
}
Ok(value)
}
fn optional_decimal<'de, D: Deserializer<'de>>(
deserializer: D,
) -> Result<Option<String>, D::Error> {
let value = Option::<String>::deserialize(deserializer)?;
if value
.as_deref()
.is_some_and(|value| !canonical_positive_decimal(value))
{
return Err(serde::de::Error::custom(
"expected a canonical positive decimal string",
));
}
Ok(value)
}
fn canonical_positive_decimal(value: &str) -> bool {
value
.as_bytes()
.first()
.is_some_and(|first| matches!(first, b'1'..=b'9'))
&& value.bytes().all(|byte| byte.is_ascii_digit())
&& value.parse::<i64>().is_ok_and(|value| value > 0)
}
fn bounded_text<'de, D: Deserializer<'de>>(deserializer: D) -> Result<String, D::Error> {
let value = String::deserialize(deserializer)?;
if value.len() > DRIVE_TEXT_MAX_BYTES {
return Err(serde::de::Error::custom("Drive text exceeds 64 KiB"));
}
Ok(value)
}
fn bounded_chunk<'de, D: Deserializer<'de>>(deserializer: D) -> Result<u32, D::Error> {
let value = u32::deserialize(deserializer)?;
if value == 0 || value > DRIVE_CHUNK_MAX_BYTES {
return Err(serde::de::Error::custom(
"Drive read size must be between 1 and 65536 bytes",
));
}
Ok(value)
}
fn optional_limit<'de, D: Deserializer<'de>>(deserializer: D) -> Result<Option<u32>, D::Error> {
let value = Option::<u32>::deserialize(deserializer)?;
if value.is_some_and(|value| value == 0 || value > DRIVE_PAGE_MAX_LIMIT) {
return Err(serde::de::Error::custom(
"Drive page limit must be between 1 and 200",
));
}
Ok(value)
}
const fn default_read_size() -> u32 {
DRIVE_CHUNK_MAX_BYTES
}
fn optional_search_limit<'de, D: Deserializer<'de>>(
deserializer: D,
) -> Result<Option<u32>, D::Error> {
let value = Option::<u32>::deserialize(deserializer)?;
if value.is_some_and(|value| value == 0 || value > DRIVE_SEARCH_MAX_LIMIT) {
return Err(serde::de::Error::custom(
"Drive search limit must be between 1 and 128",
));
}
Ok(value)
}
/// Numeric node IDs alone are not public references: the referenced Workspace
/// must match the route Workspace before the service resolves the node.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveEntryRef {
pub workspace_id: String,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub node_id: String,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum DriveEntryKind {
Folder,
File,
}
/// Metadata only: content is returned through bounded text or binary routes.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveEntry {
pub entry: DriveEntryRef,
pub parent: Option<DriveEntryRef>,
pub name: String,
pub kind: DriveEntryKind,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub revision: String,
/// File length in bytes; absent for folders.
#[schemars(range(min = 0, max = 16_777_216))]
pub size: Option<u32>,
pub content_type: Option<String>,
pub updated_by: String,
/// Timestamp projected from storage's updated_at_ms; not a creation time.
pub updated_at: String,
/// Stable relative authenticated URL in this same Workspace. No bearer token
/// or cross-Workspace link is embedded in this metadata.
pub latest_url: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveEntryQuery {
pub entry_workspace_id: String,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub id: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveListQuery {
pub entry_workspace_id: String,
/// Folder to list.
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub id: String,
#[serde(default, deserialize_with = "optional_limit")]
#[schemars(range(min = 1, max = 200))]
pub limit: Option<u32>,
/// Opaque service-issued page cursor, not a bare node ID.
#[serde(default)]
pub after: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveSearchQuery {
pub query: String,
/// Search file text as well as names only when explicitly requested.
#[serde(default)]
pub include_text: bool,
#[serde(default, deserialize_with = "optional_search_limit")]
#[schemars(range(min = 1, max = 128))]
pub limit: Option<u32>,
#[serde(default)]
pub after: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveListResponse {
pub entries: Vec<DriveEntry>,
pub next_after: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveReadTextQuery {
pub entry_workspace_id: String,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub id: String,
#[serde(default = "default_read_size", deserialize_with = "bounded_chunk")]
#[schemars(range(min = 1, max = 65536))]
pub max_bytes: u32,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveReadTextResponse {
pub entry: DriveEntry,
#[serde(deserialize_with = "bounded_text")]
pub text: String,
pub truncated: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveReadChunkQuery {
pub entry_workspace_id: String,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub id: String,
/// Bind a multi-chunk read to the same file revision.
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub expected_revision: String,
#[schemars(range(min = 0, max = 16_777_216))]
pub offset: u32,
#[serde(default = "default_read_size", deserialize_with = "bounded_chunk")]
#[schemars(range(min = 1, max = 65536))]
pub length: u32,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveDownloadQuery {
pub entry_workspace_id: String,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub id: String,
#[serde(default, deserialize_with = "optional_decimal")]
pub expected_revision: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveMutationRequest {
pub request_id: String,
pub mutation: DriveMutation,
}
/// JSON can mutate bounded UTF-8 TEXT only. Arbitrary file bytes use drive_upload.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(tag = "operation", rename_all = "snake_case", deny_unknown_fields)]
pub enum DriveMutation {
CreateFolder {
parent: DriveEntryRef,
name: String,
},
CreateText {
parent: DriveEntryRef,
name: String,
#[serde(deserialize_with = "bounded_text")]
text: String,
content_type: String,
},
UpdateText {
id: DriveEntryRef,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
expected_revision: String,
#[serde(deserialize_with = "bounded_text")]
text: String,
content_type: String,
},
Relocate {
id: DriveEntryRef,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
expected_revision: String,
parent: DriveEntryRef,
name: String,
},
Delete {
id: DriveEntryRef,
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
expected_revision: String,
},
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveMutationResponse {
pub request_id: String,
/// A deleted entry has no live metadata.
pub entry: Option<DriveEntry>,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum DriveUploadOperation {
Create,
Update,
}
/// Flat query metadata accompanies the binary body. The service rejects invalid
/// combinations: create requires parent_id/name and forbids id/expected_revision;
/// update requires id/expected_revision and forbids parent_id/name. Both targets
/// are bound by entry_workspace_id; declared size and SHA256 must match the body.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveUploadQuery {
pub operation: DriveUploadOperation,
pub request_id: String,
pub entry_workspace_id: String,
#[serde(default, deserialize_with = "optional_decimal")]
pub parent_id: Option<String>,
#[serde(default)]
pub name: Option<String>,
#[serde(default, deserialize_with = "optional_decimal")]
pub id: Option<String>,
#[serde(default, deserialize_with = "optional_decimal")]
pub expected_revision: Option<String>,
pub content_type: String,
/// Values above the file limit still decode as u32 so the service returns
/// typed Limit rather than a generic request-decoding error.
#[schemars(range(min = 0, max = 16_777_216))]
pub size: u32,
pub sha256: String,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum DriveRequestState {
/// No commit record exists at the observation snapshot. An in-flight request
/// can still commit afterward; this is not proof of a final failed outcome.
Uncommitted,
Committed,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveRequestStatusResponse {
pub request_id: String,
pub state: DriveRequestState,
pub response: Option<DriveMutationResponse>,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum DriveAccess {
ReadOnly,
ReadWrite,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveGrantCreateRequest {
pub runtime_id: String,
pub worker_id: String,
pub access: DriveAccess,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveGrantResponse {
#[serde(deserialize_with = "decimal")]
#[schemars(regex(pattern = "^[1-9][0-9]*$"))]
pub grant_id: String,
pub workspace_id: String,
pub runtime_id: String,
pub worker_id: String,
pub access: DriveAccess,
pub revoked: bool,
pub created_by: String,
pub created_at: String,
pub revoked_by: Option<String>,
pub revoked_at: Option<String>,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveGrantListQuery {
#[serde(default, deserialize_with = "optional_limit")]
#[schemars(range(min = 1, max = 200))]
pub limit: Option<u32>,
/// Canonical grant ID cursor within this Workspace.
#[serde(default, deserialize_with = "optional_decimal")]
pub after: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveGrantListResponse {
pub grants: Vec<DriveGrantResponse>,
pub next_after: Option<String>,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum DriveApiErrorCode {
Denied,
NotFound,
Conflict,
Invalid,
Limit,
StorageUnavailable,
OutcomeUnknown,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum DriveFailureClassification {
NotCommitted,
Unknown,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct DriveApiError {
pub code: DriveApiErrorCode,
pub classification: DriveFailureClassification,
pub message: String,
}
impl DriveApiError {
/// Never include provider errors, paths, SQL, or caller-supplied messages.
pub fn new(code: DriveApiErrorCode) -> Self {
use DriveApiErrorCode::*;
Self {
code,
classification: if code == OutcomeUnknown {
DriveFailureClassification::Unknown
} else {
DriveFailureClassification::NotCommitted
},
message: match code {
Denied => "Drive access denied",
NotFound => "Drive resource not found",
Conflict => "Drive revision or name conflict",
Invalid => "Invalid Drive request",
Limit => "Drive request exceeds a limit",
StorageUnavailable => "Drive storage unavailable",
OutcomeUnknown => "Drive mutation outcome unknown",
}
.into(),
}
}
}
impl api_macros::HttpError for DriveApiError {
fn status_code(&self) -> u16 {
use DriveApiErrorCode::*;
match self.code {
Denied => 403,
NotFound => 404,
Conflict => 409,
Invalid => 400,
Limit => 413,
StorageUnavailable | OutcomeUnknown => 503,
}
}
}
impl api_macros::HttpRequestError for DriveApiError {
fn from_request_rejection(status: u16, _message: String) -> Self {
Self::new(if status == 413 {
DriveApiErrorCode::Limit
} else {
DriveApiErrorCode::Invalid
})
}
}
impl api_macros::openapi::OpenApiSchema for DriveApiError {}
impl api_macros::openapi::OpenApiSchema for DriveEntry {}
impl api_macros::openapi::OpenApiSchema for DriveEntryQuery {}
impl api_macros::openapi::OpenApiSchema for DriveListQuery {}
impl api_macros::openapi::OpenApiSchema for DriveSearchQuery {}
impl api_macros::openapi::OpenApiSchema for DriveListResponse {}
impl api_macros::openapi::OpenApiSchema for DriveReadTextQuery {}
impl api_macros::openapi::OpenApiSchema for DriveReadTextResponse {}
impl api_macros::openapi::OpenApiSchema for DriveReadChunkQuery {}
impl api_macros::openapi::OpenApiSchema for DriveDownloadQuery {}
impl api_macros::openapi::OpenApiSchema for DriveMutationRequest {}
impl api_macros::openapi::OpenApiSchema for DriveMutationResponse {}
impl api_macros::openapi::OpenApiSchema for DriveUploadQuery {}
impl api_macros::openapi::OpenApiSchema for DriveRequestStatusResponse {}
impl api_macros::openapi::OpenApiSchema for DriveGrantCreateRequest {}
impl api_macros::openapi::OpenApiSchema for DriveGrantResponse {}
impl api_macros::openapi::OpenApiSchema for DriveGrantListQuery {}
impl api_macros::openapi::OpenApiSchema for DriveGrantListResponse {}
/// Browser wire declarations are derived from the transport DTOs, not copied
/// from storage models or maintained independently in the Web client.
#[cfg(feature = "typescript")]
pub fn drive_api_typescript() -> String {
use ts_rs::TS;
let config = ts_rs::Config::default();
let declarations = [
DriveEntryRef::decl(&config),
DriveEntryKind::decl(&config),
DriveEntry::decl(&config),
DriveEntryQuery::decl(&config),
DriveListQuery::decl(&config),
DriveSearchQuery::decl(&config),
DriveListResponse::decl(&config),
DriveReadTextQuery::decl(&config),
DriveReadTextResponse::decl(&config),
DriveReadChunkQuery::decl(&config),
DriveDownloadQuery::decl(&config),
DriveMutation::decl(&config),
DriveMutationRequest::decl(&config),
DriveMutationResponse::decl(&config),
DriveUploadOperation::decl(&config),
DriveUploadQuery::decl(&config),
DriveRequestState::decl(&config),
DriveRequestStatusResponse::decl(&config),
DriveAccess::decl(&config),
DriveGrantCreateRequest::decl(&config),
DriveGrantResponse::decl(&config),
DriveGrantListQuery::decl(&config),
DriveGrantListResponse::decl(&config),
DriveApiErrorCode::decl(&config),
DriveFailureClassification::decl(&config),
DriveApiError::decl(&config),
];
format!(
"// Generated from server-api. Do not edit by hand.\n// Regenerate: cargo run -q -p server-api --features typescript --example generate_drive_api_types > web/workspace/src/lib/generated/drive-api.ts\n\n{}\n",
declarations
.into_iter()
.map(|declaration| format!("export {declaration}"))
.collect::<Vec<_>>()
.join("\n\n")
)
}
#[cfg(all(test, feature = "typescript"))]
mod typescript_tests {
#[test]
fn generated_drive_api_contract_is_current() {
let expected = super::drive_api_typescript();
let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
.join("../../web/workspace/src/lib/generated/drive-api.ts");
let actual = std::fs::read_to_string(&path)
.unwrap_or_else(|error| panic!("failed to read {}: {error}", path.display()));
assert_eq!(
normalize(&actual),
normalize(&expected),
"regenerate Drive API TypeScript types with `cargo run -q -p server-api --features typescript --example generate_drive_api_types > web/workspace/src/lib/generated/drive-api.ts` and format the generated file"
);
}
fn normalize(value: &str) -> String {
value
.chars()
.filter_map(|character| match character {
character if character.is_whitespace() => None,
',' => Some(';'),
character => Some(character),
})
.collect::<String>()
.replace("=|", "=")
.replace(";}", "}")
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{ApiContract, HttpMethod, ServerApiMetadata, canonical_openapi_document};
use serde_json::json;
#[test]
fn drive_entry_references_reject_unbound_or_noncanonical_numeric_ids() {
let valid = json!({"workspace_id":"workspace-a", "node_id":"9007199254740993"});
let entry: DriveEntryRef = serde_json::from_value(valid.clone()).unwrap();
assert_eq!(serde_json::to_value(entry).unwrap(), valid);
for invalid in [
json!("7"),
json!(7),
json!({"node_id":"7"}),
json!({"workspace_id":"a", "node_id":7}),
] {
assert!(serde_json::from_value::<DriveEntryRef>(invalid).is_err());
}
for id in [
"",
"0",
"01",
"-1",
"+1",
"1.0",
"1e3",
" 1",
"١",
"9223372036854775808",
"18446744073709551616",
] {
assert!(
serde_json::from_value::<DriveEntryRef>(json!({"workspace_id":"a", "node_id":id}))
.is_err(),
"accepted {id:?}"
);
assert!(
serde_json::from_value::<DriveEntryQuery>(
json!({"entry_workspace_id":"a", "id":id})
)
.is_err()
);
}
assert!(
serde_json::from_value::<DriveEntryRef>(
json!({"workspace_id":"a","node_id":"9223372036854775807"})
)
.is_ok()
);
assert!(serde_json::from_value::<DriveEntryQuery>(json!({"id":"7"})).is_err());
assert!(serde_json::from_value::<DriveListQuery>(json!({"id":"7"})).is_err());
assert!(serde_json::from_value::<DriveReadTextQuery>(json!({"id":"7"})).is_err());
assert!(serde_json::from_value::<DriveDownloadQuery>(json!({"id":"7"})).is_err());
assert!(
serde_json::from_value::<DriveEntryQuery>(
json!({"entry_workspace_id":"a", "id":"7", "worker_id":"spoofed"})
)
.is_err()
);
}
#[test]
fn drive_metadata_exposes_only_stored_update_provenance_and_same_workspace_url() {
let value = json!({
"entry":{"workspace_id":"a","node_id":"7"},
"parent":{"workspace_id":"a","node_id":"1"},
"name":"file.txt","kind":"file","revision":"2","size":5,
"content_type":"text/plain","updated_by":"user-a","updated_at":"2026-10-08T00:00:00Z",
"latest_url":"/api/w/a/drive/download?entry_workspace_id=a&id=7"
});
let entry: DriveEntry = serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(entry).unwrap(), value);
for field in ["sha256", "created_at"] {
let mut fabricated = value.clone();
fabricated[field] = json!("not stored");
assert!(serde_json::from_value::<DriveEntry>(fabricated).is_err());
}
}
#[test]
fn drive_search_text_is_opt_in_and_work_budget_is_bounded() {
let query: DriveSearchQuery = serde_json::from_value(json!({"query":"name"})).unwrap();
assert!(!query.include_text);
let query: DriveSearchQuery =
serde_json::from_value(json!({"query":"text","include_text":true,"limit":128}))
.unwrap();
assert!(query.include_text);
for limit in [0, 129, u32::MAX] {
assert!(
serde_json::from_value::<DriveSearchQuery>(json!({"query":"name","limit":limit}))
.is_err()
);
}
}
#[test]
fn drive_request_status_is_a_commit_snapshot_not_a_fictitious_lifecycle() {
for state in ["uncommitted", "committed"] {
let value = json!({"request_id":"r","state":state,"response":null});
let response: DriveRequestStatusResponse =
serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(response).unwrap(), value);
}
for state in ["pending", "unknown", "not_committed"] {
assert!(
serde_json::from_value::<DriveRequestStatusResponse>(
json!({"request_id":"r","state":state,"response":null})
)
.is_err()
);
}
}
#[test]
fn drive_json_mutations_accept_only_bounded_text_and_workspace_bound_cas() {
let valid = json!({"request_id":"request-a", "mutation":{
"operation":"update_text", "id":{"workspace_id":"a","node_id":"7"},
"expected_revision":"9007199254740993", "text":"hello", "content_type":"text/plain"
}});
let request: DriveMutationRequest = serde_json::from_value(valid.clone()).unwrap();
assert_eq!(serde_json::to_value(request).unwrap(), valid);
let mut invalid = valid.clone();
invalid["mutation"]["bytes"] = json!([0, 255]);
assert!(serde_json::from_value::<DriveMutationRequest>(invalid).is_err());
let mut invalid = valid.clone();
invalid["mutation"]["id"] = json!("7");
assert!(serde_json::from_value::<DriveMutationRequest>(invalid).is_err());
let mut invalid = valid.clone();
invalid["mutation"]["expected_revision"] = json!("01");
assert!(serde_json::from_value::<DriveMutationRequest>(invalid).is_err());
// UTF-8 byte count, not character count, determines the text bound.
let mut boundary = valid;
boundary["mutation"]["text"] = json!("é".repeat(DRIVE_TEXT_MAX_BYTES / 2));
assert!(serde_json::from_value::<DriveMutationRequest>(boundary.clone()).is_ok());
boundary["mutation"]["text"] = json!("é".repeat(DRIVE_TEXT_MAX_BYTES / 2 + 1));
assert!(serde_json::from_value::<DriveMutationRequest>(boundary).is_err());
}
#[test]
fn drive_reads_reject_zero_or_over_64kib_requests() {
for size in [0, 65537, u32::MAX] {
assert!(
serde_json::from_value::<DriveReadTextQuery>(
json!({"entry_workspace_id":"a","id":"7","max_bytes":size})
)
.is_err()
);
assert!(serde_json::from_value::<DriveReadChunkQuery>(json!({"entry_workspace_id":"a","id":"7","expected_revision":"1","offset":0,"length":size})).is_err());
}
let default: DriveReadTextQuery =
serde_json::from_value(json!({"entry_workspace_id":"a","id":"7"})).unwrap();
assert_eq!(default.max_bytes, 65536);
let chunk: DriveReadChunkQuery = serde_json::from_value(json!({"entry_workspace_id":"a","id":"7","expected_revision":"1","offset":0,"length":65536})).unwrap();
assert_eq!(chunk.length, 65536);
}
#[test]
fn drive_upload_requires_binding_declared_integrity_and_request_identity() {
let valid = json!({"operation":"update","request_id":"request-a","entry_workspace_id":"a","id":"7","expected_revision":"2","content_type":"application/octet-stream","size":3,"sha256":"a".repeat(64)});
assert!(serde_json::from_value::<DriveUploadQuery>(valid.clone()).is_ok());
for field in ["entry_workspace_id", "size", "sha256", "request_id"] {
let mut missing = valid.clone();
missing.as_object_mut().unwrap().remove(field);
assert!(
serde_json::from_value::<DriveUploadQuery>(missing).is_err(),
"accepted missing {field}"
);
}
for size in [DRIVE_FILE_MAX_BYTES, DRIVE_FILE_MAX_BYTES + 1, u32::MAX] {
let mut oversized = valid.clone();
oversized["size"] = json!(size);
let query: DriveUploadQuery = serde_json::from_value(oversized).unwrap();
assert_eq!(
query.size, size,
"service must receive oversized declared sizes for typed Limit classification"
);
}
assert!(serde_json::from_value::<DriveReadChunkQuery>(json!({"entry_workspace_id":"a","id":"7","expected_revision":"1","offset":DRIVE_FILE_MAX_BYTES,"length":1})).is_ok());
let mut invalid = valid;
invalid["bytes"] = json!([0, 255]);
assert!(serde_json::from_value::<DriveUploadQuery>(invalid).is_err());
}
#[test]
fn drive_grants_reject_command_access_and_unbounded_pages() {
let grant = json!({"runtime_id":"r","worker_id":"w","access":"read_write"});
assert_eq!(
serde_json::to_value(
serde_json::from_value::<DriveGrantCreateRequest>(grant.clone()).unwrap()
)
.unwrap(),
grant
);
assert!(
serde_json::from_value::<DriveGrantCreateRequest>(
json!({"runtime_id":"r","worker_id":"w","access":"read_write_command"})
)
.is_err()
);
assert!(serde_json::from_value::<DriveGrantCreateRequest>(json!({"runtime_id":"r","worker_id":"w","access":"read_only","workspace_id":"spoofed"})).is_err());
for limit in [0, 201, u32::MAX] {
assert!(serde_json::from_value::<DriveGrantListQuery>(json!({"limit":limit})).is_err());
}
assert!(
serde_json::from_value::<DriveGrantListQuery>(json!({"limit":200,"after":"7"})).is_ok()
);
assert!(serde_json::from_value::<DriveGrantListQuery>(json!({"after":"01"})).is_err());
}
#[test]
fn drive_errors_have_fixed_safe_messages_and_typed_outcome_classification() {
use DriveApiErrorCode::*;
for (code, wire, status, message) in [
(Denied, "denied", 403, "Drive access denied"),
(NotFound, "not_found", 404, "Drive resource not found"),
(Conflict, "conflict", 409, "Drive revision or name conflict"),
(Invalid, "invalid", 400, "Invalid Drive request"),
(Limit, "limit", 413, "Drive request exceeds a limit"),
(
StorageUnavailable,
"storage_unavailable",
503,
"Drive storage unavailable",
),
(
OutcomeUnknown,
"outcome_unknown",
503,
"Drive mutation outcome unknown",
),
] {
let error = DriveApiError::new(code);
assert_eq!(api_macros::HttpError::status_code(&error), status);
assert_eq!(
serde_json::to_value(&error).unwrap(),
json!({"code":wire,"classification":if code == OutcomeUnknown {"unknown"} else {"not_committed"},"message":message})
);
}
for status in [400, 413, 415, 422] {
let error = <DriveApiError as api_macros::HttpRequestError>::from_request_rejection(
status,
"/private/path: SQL failed".into(),
);
assert_eq!(
error,
DriveApiError::new(if status == 413 { Limit } else { Invalid })
);
}
assert!(serde_json::from_value::<DriveApiError>(json!({"code":"invalid","classification":"not_committed","message":"Invalid Drive request","provider_error":"secret"})).is_err());
}
#[test]
fn drive_routes_are_workspace_scoped_authenticated_and_binary_is_not_json() {
let document = canonical_openapi_document().unwrap();
let document: serde_json::Value =
serde_json::from_str(&document.to_json().unwrap()).unwrap();
let schemas = &document["components"]["schemas"];
assert_eq!(
schemas["DriveEntry"]["properties"]["size"]["maximum"],
DRIVE_FILE_MAX_BYTES
);
assert_eq!(
schemas["DriveReadChunkQuery"]["properties"]["offset"]["maximum"],
DRIVE_FILE_MAX_BYTES
);
assert_eq!(
schemas["DriveUploadQuery"]["properties"]["size"]["maximum"],
DRIVE_FILE_MAX_BYTES
);
assert_eq!(
schemas["DriveSearchQuery"]["properties"]["limit"]["maximum"],
DRIVE_SEARCH_MAX_LIMIT
);
for (name, method, suffix, wire_method) in [
("drive_root", HttpMethod::Get, "/root", "get"),
("drive_metadata", HttpMethod::Get, "/metadata", "get"),
("drive_list", HttpMethod::Get, "/list", "get"),
("drive_search", HttpMethod::Get, "/search", "get"),
("drive_read_text", HttpMethod::Get, "/read-text", "get"),
("drive_read_chunk", HttpMethod::Get, "/read-chunk", "get"),
("drive_download", HttpMethod::Get, "/download", "get"),
("drive_mutate", HttpMethod::Post, "/mutate", "post"),
("drive_upload", HttpMethod::Put, "/upload", "put"),
(
"drive_request_status",
HttpMethod::Get,
"/requests/{request_id}",
"get",
),
("drive_grant_create", HttpMethod::Post, "/grants", "post"),
(
"drive_grant_revoke",
HttpMethod::Delete,
"/grants/{grant_id}",
"delete",
),
("drive_grant_list", HttpMethod::Get, "/grants", "get"),
] {
let operation = ServerApiMetadata::OPERATIONS
.iter()
.find(|operation| operation.operation_id == name)
.unwrap();
let path = format!("/api/w/{{workspace_id}}/drive{suffix}");
assert_eq!(operation.method, method);
assert_eq!(operation.path, path);
let documented = &document["paths"][&path][wire_method];
assert_eq!(documented["operationId"], name);
assert_eq!(
documented["security"],
json!([{"bearerAuth":[]},{"browserSession":[]}])
);
// Server extensions must not become caller-controlled parameters.
assert!(
documented["parameters"]
.as_array()
.unwrap()
.iter()
.all(|parameter| parameter["name"] != "context")
);
if matches!(name, "drive_read_chunk" | "drive_download") {
let content = &documented["responses"]["200"]["content"];
assert!(content["application/octet-stream"].is_object());
assert!(content["application/json"].is_null());
}
if name == "drive_download" {
let headers = documented["responses"]["200"]["headers"]
.as_object()
.unwrap();
assert_eq!(headers.len(), 5);
for header in [
"content-disposition",
"cache-control",
"etag",
"x-content-type-options",
"content-security-policy",
] {
assert_eq!(
headers[header]["schema"]["$ref"],
"#/components/schemas/string"
);
}
assert!(!headers.contains_key("content-type"));
assert!(!headers.contains_key("content-length"));
}
if name == "drive_upload" {
let content = &documented["requestBody"]["content"];
assert!(content["application/octet-stream"].is_object());
assert!(content["application/json"].is_null());
}
}
}
}
File diff suppressed because it is too large Load Diff
@@ -872,14 +872,14 @@ fn validate_runtime_operations(
"/api/runtimes/{runtime_id}/workers/{worker_id}/restore",
"post",
"runtime_worker_restore_alias",
None,
Some("WorkerRestoreRequest"),
&[("200", "WorkerRestoreResponse")],
),
runtime_operation(
"/api/w/{workspace_id}/runtimes/{runtime_id}/workers/{worker_id}/restore",
"post",
"runtime_worker_restore",
None,
Some("WorkerRestoreRequest"),
&[("200", "WorkerRestoreResponse")],
),
runtime_operation(
@@ -1227,8 +1227,16 @@ fn json_content_schema<'a>(
}
fn schema_reference<'a>(value: &'a Value, context: &str) -> Result<&'a str, GenerationError> {
const ALLOWED_REFERENCE_SIBLINGS: &[&str] =
&["$ref", "$comment", "deprecated", "description", "title"];
// `default` is a JSON Schema annotation, not a constraint or a replacement
// for the referenced type. Schemars emits it for defaulted enum fields.
const ALLOWED_REFERENCE_SIBLINGS: &[&str] = &[
"$ref",
"$comment",
"default",
"deprecated",
"description",
"title",
];
let schema = object(value, context)?;
if let Some(keyword) = schema
.keys()
@@ -2023,6 +2031,27 @@ mod tests {
assert!(error.to_string().contains("ambiguous anyOf"));
}
#[test]
fn reference_default_annotations_do_not_replace_the_referenced_type() {
let value = serde_json::json!({"$ref":"#/components/schemas/InvocationCompletion", "default":{"kind":"none"}});
assert_eq!(
schema_reference(&value, "completion").unwrap(),
"#/components/schemas/InvocationCompletion"
);
let output = generate_runtime_typescript(
&crate::canonical_openapi_document()
.unwrap()
.to_json()
.unwrap(),
)
.unwrap();
assert!(output.contains("completion?: InvocationCompletion"));
assert!(output.contains("export type FeatureInvocation ="));
assert!(output.contains(
"export type CompletionKind = \"file\" | \"feature\" | \"feature_argument\""
));
}
#[test]
fn generator_rejects_lossy_reference_siblings() {
let mut document: Value = serde_json::from_str(OPENAPI).unwrap();
+284
View File
@@ -0,0 +1,284 @@
//! Grant-bound logical Workspace configuration transport. No host filesystem contract.
use super::{ConfigCommitRequest, ConfigContentType};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum WorkspaceConfigAccess {
ReadOnly,
ReadWrite,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum WorkspaceConfigNodeKind {
Directory,
File,
Missing,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(rename_all = "snake_case")]
pub enum WorkspaceConfigFailureClassification {
NotCommitted,
Unknown,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigAttachRequest {
#[serde(default)]
pub alias: Option<String>,
#[serde(default)]
pub access: Option<WorkspaceConfigAccess>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigAttachment {
pub workspace_id: String,
pub connection_id: String,
pub alias: String,
pub working_directory_id: String,
pub access: WorkspaceConfigAccess,
pub name: String,
pub purpose: String,
pub content_path: String,
pub already_attached: bool,
}
/// Named transport wrapper required by the generated API; JSON remains attachment-or-null.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(transparent)]
pub struct WorkspaceConfigCurrentResponse(pub Option<WorkspaceConfigAttachment>);
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigObserveRequest {
pub connection_id: String,
pub paths: Vec<String>,
pub depth: u32,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigNode {
pub path: String,
pub kind: WorkspaceConfigNodeKind,
pub validator: String,
pub digest: Option<String>,
pub content_type: Option<ConfigContentType>,
pub operations: Vec<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigObserveResponse {
pub connection_id: String,
pub validator: String,
#[cfg_attr(feature = "typescript", ts(type = "number"))]
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub revision: u64,
pub digest: String,
/// Canonical commit entrypoints from this exact metadata snapshot.
pub entrypoints: Vec<String>,
pub nodes: Vec<WorkspaceConfigNode>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigReadRequest {
pub connection_id: String,
pub path: String,
pub validator: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigReadResponse {
pub path: String,
pub content: String,
pub content_type: ConfigContentType,
pub digest: String,
pub validator: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigCommitRequest {
pub connection_id: String,
pub validator: String,
pub request: ConfigCommitRequest,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigCommitResponse {
pub validator: String,
#[cfg_attr(feature = "typescript", ts(type = "number"))]
#[schemars(range(min = 0, max = 9_007_199_254_740_991_u64))]
pub revision: u64,
pub digest: String,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigApiError {
pub status: u16,
pub code: String,
pub message: String,
pub classification: WorkspaceConfigFailureClassification,
}
impl api_macros::HttpError for WorkspaceConfigApiError {
fn status_code(&self) -> u16 {
self.status
}
}
impl api_macros::HttpRequestError for WorkspaceConfigApiError {
fn from_request_rejection(status: u16, _message: String) -> Self {
Self {
status,
code: "invalid_request".into(),
message: "Invalid configuration request".into(),
classification: WorkspaceConfigFailureClassification::NotCommitted,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigGrantCreateRequest {
pub runtime_id: String,
pub worker_id: String,
pub access: WorkspaceConfigAccess,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, JsonSchema)]
#[cfg_attr(feature = "typescript", derive(ts_rs::TS))]
#[serde(deny_unknown_fields)]
pub struct WorkspaceConfigGrantResponse {
pub grant_id: String,
pub workspace_id: String,
pub runtime_id: String,
pub worker_id: String,
pub working_directory_id: String,
pub access: WorkspaceConfigAccess,
pub revoked: bool,
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{ApiContract, HttpMethod, ServerApiMetadata};
#[test]
fn workspace_config_observation_is_metadata_only_and_has_same_snapshot_cas_inputs() {
let value = serde_json::json!({"connection_id":"c", "validator":"v", "revision":7, "digest":"d", "entrypoints":["main.dcdl"], "nodes":[{"path":"", "kind":"directory", "validator":"v", "digest":null, "content_type":null, "operations":["apply_changes"]}]});
let response: WorkspaceConfigObserveResponse =
serde_json::from_value(value.clone()).unwrap();
assert_eq!(serde_json::to_value(response).unwrap(), value);
for field in ["revision", "digest", "entrypoints"] {
let mut missing = value.clone();
missing.as_object_mut().unwrap().remove(field);
assert!(serde_json::from_value::<WorkspaceConfigObserveResponse>(missing).is_err());
}
let mut with_body = value;
with_body["nodes"][0]["content"] = serde_json::json!("must never appear here");
assert!(serde_json::from_value::<WorkspaceConfigObserveResponse>(with_body).is_err());
assert!(
serde_json::from_value::<WorkspaceConfigGrantCreateRequest>(
serde_json::json!({"runtime_id":"r","worker_id":"w","access":"read_write_command"})
)
.is_err()
);
assert!(
serde_json::from_value::<WorkspaceConfigAttachRequest>(
serde_json::json!({"grant_id":"invented"})
)
.is_err()
);
assert_eq!(
serde_json::to_value(WorkspaceConfigCurrentResponse(None)).unwrap(),
serde_json::Value::Null
);
}
#[test]
fn workspace_config_failure_classification_roundtrips_and_rejections_are_pre_effects() {
for classification in [
WorkspaceConfigFailureClassification::NotCommitted,
WorkspaceConfigFailureClassification::Unknown,
] {
let error = WorkspaceConfigApiError {
status: 500,
code: "unavailable".into(),
message: "safe".into(),
classification,
};
assert_eq!(
serde_json::from_value::<WorkspaceConfigApiError>(
serde_json::to_value(&error).unwrap()
)
.unwrap(),
error
);
assert_eq!(api_macros::HttpError::status_code(&error), 500);
}
let error =
<WorkspaceConfigApiError as api_macros::HttpRequestError>::from_request_rejection(
413,
"/private/secret".into(),
);
assert_eq!(
error.classification,
WorkspaceConfigFailureClassification::NotCommitted
);
assert!(!error.message.contains("private"));
}
#[test]
fn workspace_config_generated_routes_are_exact_and_grants_are_separate() {
for (name, method, suffix) in [
("get", HttpMethod::Get, ""),
("attach", HttpMethod::Post, ""),
("observe", HttpMethod::Post, "/observe"),
("read", HttpMethod::Post, "/read"),
("commit", HttpMethod::Post, "/commit"),
] {
let name = format!("current_worker_workspace_config_{name}");
let operation = ServerApiMetadata::OPERATIONS
.iter()
.find(|operation| operation.operation_id == name)
.unwrap();
assert_eq!(operation.method, method);
assert_eq!(
operation.path,
format!("/api/w/{{workspace_id}}/workers/self/workspace-config{suffix}")
);
}
for (name, method, path) in [
(
"workspace_config_grant_create",
HttpMethod::Post,
"/api/w/{workspace_id}/workspace-config-grants",
),
(
"workspace_config_grant_revoke",
HttpMethod::Delete,
"/api/w/{workspace_id}/workspace-config-grants/{grant_id}",
),
] {
let operation = ServerApiMetadata::OPERATIONS
.iter()
.find(|operation| operation.operation_id == name)
.unwrap();
assert_eq!(operation.method, method);
assert_eq!(operation.path, path);
}
}
}
+23 -12
View File
@@ -211,7 +211,11 @@ impl FsStore {
}
}
fn parse_jsonl<T: serde::de::DeserializeOwned>(content: &[u8]) -> Result<Vec<T>, StoreError> {
pub(crate) fn parse_jsonl(
content: &[u8],
session_id: SessionId,
segment_id: SegmentId,
) -> Result<Vec<LogEntry>, StoreError> {
let complete = Self::complete_jsonl_prefix(content);
let content = std::str::from_utf8(complete).map_err(|error| StoreError::Corrupt {
line: complete[..error.valid_up_to()]
@@ -222,15 +226,19 @@ impl FsStore {
message: error.to_string(),
})?;
let mut entries = Vec::new();
for (i, line) in content.lines().enumerate() {
if line.trim().is_empty() {
continue;
let mut record_start = 0_u64;
for (index, record) in content.split_inclusive('\n').enumerate() {
let line = record.strip_suffix('\n').unwrap_or(record);
if !line.trim().is_empty() {
let mut entry: LogEntry =
serde_json::from_str(line).map_err(|error| StoreError::Corrupt {
line: index + 1,
message: error.to_string(),
})?;
entry.ensure_legacy_session_entry_id(session_id, segment_id, record_start);
entries.push(entry);
}
let entry: T = serde_json::from_str(line).map_err(|e| StoreError::Corrupt {
line: i + 1,
message: e.to_string(),
})?;
entries.push(entry);
record_start = record_start.saturating_add(record.len() as u64);
}
Ok(entries)
}
@@ -294,7 +302,7 @@ impl Store for FsStore {
return Err(StoreError::NotFound(segment_id));
}
let content = fs::read(&path)?;
Self::parse_jsonl(&content)
Self::parse_jsonl(&content, session_id, segment_id)
}
fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError> {
@@ -375,8 +383,11 @@ impl Store for FsStore {
content.push_str(&serde_json::to_string(entry)?);
content.push('\n');
}
fs::write(&path, content.as_bytes())?;
Ok(())
let _guard = self
.append_lock
.lock()
.map_err(|_| std::io::Error::other("session store append lock was poisoned"))?;
crate::worker_session_store::atomic_write_bytes(&path, content.as_bytes())
}
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
+6 -1
View File
@@ -4,7 +4,10 @@ use serde::{Deserialize, Serialize};
use crate::LoggedItem;
/// Stable logical identity of one model-visible history entry.
/// Stable logical identity of one durable public Session entry.
///
/// Model-visible history entries and typed Run transitions share this identity
/// space so every projection can reconcile the same committed record.
///
/// This value is generated at the trusted Worker session boundary and copied
/// unchanged across fork, rewind, compaction retention, restore, and reboot.
@@ -37,6 +40,8 @@ pub struct LoggedWorkerSubject {
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum LoggedSessionHistoryOrigin {
/// Human input verified by the standalone Host, without a Backend Account.
LocalHumanInput,
HumanInput {
account_id: String,
},
+22 -7
View File
@@ -75,6 +75,7 @@ pub mod history;
mod legacy_session_log;
pub mod logged_item;
mod paste_artifact;
pub mod public_index;
pub mod public_snapshot;
pub mod segment;
pub mod segment_log;
@@ -95,19 +96,33 @@ pub use history::{
};
pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged};
pub use paste_artifact::PasteArtifactLimits;
pub use public_index::{
DEFAULT_SESSION_PUBLIC_INDEX_MAX_BYTES, DEFAULT_SESSION_PUBLIC_INDEX_MAX_ENTRIES,
DEFAULT_SESSION_PUBLIC_INDEX_MAX_SEGMENTS, SESSION_PUBLIC_INDEX_COMPACT_TEXT_MAX_BYTES,
SessionPublicIndex, SessionPublicIndexEntry, SessionPublicIndexEntryKind,
SessionPublicIndexEntryRead, SessionPublicIndexLimits, SessionPublicIndexLineage,
SessionPublicIndexOriginKind, SessionPublicIndexPage, SessionPublicIndexPageEntry,
SessionPublicIndexReadError, SessionPublicIndexScanPosition, SessionPublicIndexSegment,
SessionPublicIndexToolPart, read_session_public_index, read_session_public_index_entry,
read_session_public_index_page,
};
pub use public_snapshot::{
DEFAULT_RETAINED_HISTORY_MAX_ENTRIES, DEFAULT_RETAINED_HISTORY_MAX_RESPONSE_BYTES,
DEFAULT_RETAINED_HISTORY_MAX_SCAN_BYTES, DEFAULT_RETAINED_HISTORY_MAX_SEGMENTS,
DEFAULT_RETAINED_HISTORY_PAGE_TURNS, DEFAULT_RETAINED_SNAPSHOT_MAX_BYTES,
MAX_RETAINED_HISTORY_PAGE_TURNS, RetainedHistoryReadError, RetainedHistoryReadLimits,
RetainedSessionIdentity, RetainedSessionSnapshot, RetainedSnapshotReadError,
project_session_snapshot, read_retained_session_history_page, read_retained_session_snapshot,
MAX_RETAINED_HISTORY_PAGE_TURNS, RetainedAttachmentReadError, RetainedHistoryReadError,
RetainedHistoryReadLimits, RetainedSessionAttachment, RetainedSessionIdentity,
RetainedSessionSnapshot, RetainedSnapshotReadError, project_session_snapshot,
read_retained_session_attachment, read_retained_session_history_page,
read_retained_session_snapshot, read_session_attachment_from_entries,
session_tool_attachment_id,
};
pub use segment::{
SegmentStartState, append_entry, append_system_item, classify_logged_history_entry,
create_compacted_segment, create_segment, create_segment_with_ids, ensure_head_or_fork, fork,
fork_at, restore, restore_by_segment, save_config_changed, save_delta, save_extension,
save_run_completed, save_run_errored, save_turn_end, save_usage, save_user_input,
save_run_completed, save_run_errored, save_run_resumed, save_run_yielded, save_turn_end,
save_usage, save_user_input,
};
pub use segment_log::{LogEntry, RestoredState, SegmentOrigin, SessionExtension, collect_state};
pub use store::{Store, StoreError};
@@ -120,9 +135,9 @@ pub use uploaded_file::{
UploadedFileUploadContext,
};
pub use worker_metadata::{
CombinedStore, FsWorkerStore, WorkerActiveSegmentRef, WorkerAggregateStore, WorkerMetadata,
WorkerMetadataStore, WorkerPeer, WorkerReclaimedChild, WorkerSpawnedChild,
WorkerSpawnedScopeRule, WorkerStoreError, validate_worker_name,
CombinedStore, FsWorkerStore, SubjektivSessionAttributionState, WorkerActiveSegmentRef,
WorkerAggregateStore, WorkerMetadata, WorkerMetadataStore, WorkerPeer, WorkerReclaimedChild,
WorkerSpawnedChild, WorkerSpawnedScopeRule, WorkerStoreError, validate_worker_name,
};
pub use worker_session_store::WorkerSessionStore;
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,133 @@
//! Display-only accounting; never inserted into model history or written back
//! to the journal. Both snapshot and bounded history paging use this projection.
use std::collections::HashMap;
use protocol::{SessionEntryProvenance, SessionSnapshotEntry, SessionSnapshotEntryData};
use crate::LogEntry;
#[derive(Default)]
struct Activity {
started_at: u64,
requests: u64,
upload: u64,
output: u64,
anchor: Option<String>,
}
pub(super) fn project<'a>(
records: impl IntoIterator<Item = &'a LogEntry>,
) -> HashMap<String, SessionSnapshotEntry> {
let mut active: Option<Activity> = None;
let mut stats = HashMap::new();
for record in records {
match record {
LogEntry::Invoke { ts, .. } => {
active = Some(Activity {
started_at: *ts,
..Activity::default()
})
}
LogEntry::LlmUsage {
input_total_tokens,
cache_read_tokens,
output_tokens,
..
} => {
if let Some(run) = &mut active {
run.requests += 1;
run.upload = run
.upload
.saturating_add(input_total_tokens.saturating_sub(*cache_read_tokens));
run.output = run.output.saturating_add(*output_tokens);
}
}
LogEntry::AnnotatedUserInput { history, .. } => {
if let Some(run) = &mut active {
for entry in history {
if super::project_item(&entry.metadata.entry_id.0, &entry.item).is_some() {
run.anchor = Some(entry.metadata.entry_id.0.clone());
}
}
}
}
LogEntry::AnnotatedAssistantItem { entry, .. }
| LogEntry::AnnotatedToolResult { entry, .. } => {
if let Some(run) = &mut active
&& super::project_item(&entry.metadata.entry_id.0, &entry.item).is_some()
{
run.anchor = Some(entry.metadata.entry_id.0.clone());
}
}
LogEntry::AnnotatedSystemItem { entry, .. } => {
if let Some(run) = &mut active {
run.anchor = Some(entry.metadata.entry_id.0.clone());
}
}
LogEntry::RunCompleted { ts, result, .. }
if !matches!(result, agen::EngineResult::Yielded) =>
{
finish(&active, *ts, &mut stats);
// Pause/resume retains the Invoke's cumulative accounting, just
// like the live TUI. Terminal outcomes cannot leak traffic into
// a later resumed segment whose Invoke is unavailable.
if !matches!(result, agen::EngineResult::Paused) {
active = None;
}
}
LogEntry::RunCancelled { ts, .. } | LogEntry::RunErrored { ts, .. } => {
finish(&active, *ts, &mut stats);
active = None;
}
// Compaction seeds repeat retained entries, not fresh traffic.
// A page traverses adopted lineage in journal order and retains the
// open Invoke across these checkpoints. A standalone new segment
// without that Invoke cannot claim complete accounting.
_ => {}
}
}
stats
}
fn finish(
active: &Option<Activity>,
ended_at: u64,
stats: &mut HashMap<String, SessionSnapshotEntry>,
) {
let Some(run) = active else { return };
let Some(anchor) = &run.anchor else { return };
// No usage records means unavailable measurement, not known zero traffic.
if run.requests == 0 {
return;
}
stats.insert(
anchor.clone(),
SessionSnapshotEntry {
entry_id: format!("run-stats:{anchor}"),
timestamp: ended_at,
provenance: SessionEntryProvenance::LegacyUnknown,
derived_from: vec![anchor.clone()],
data: SessionSnapshotEntryData::RunStats {
elapsed_ms: ended_at.saturating_sub(run.started_at),
requests: run.requests,
upload_tokens: run.upload,
output_tokens: run.output,
},
},
);
}
pub(super) fn append_to(
entries: &mut Vec<SessionSnapshotEntry>,
stats: &HashMap<String, SessionSnapshotEntry>,
) {
let mut with_stats = Vec::with_capacity(entries.len());
for entry in std::mem::take(entries) {
let summary = stats.get(&entry.entry_id);
with_stats.push(entry);
if let Some(summary) = summary {
with_stats.push(summary.clone());
}
}
*entries = with_stats;
}
+48 -2
View File
@@ -10,7 +10,7 @@ use crate::{LoggedHistoryEntry, LoggedSystemHistoryEntry, SegmentId, SessionId};
use agen::EngineResult;
use agen::llm_client::RequestConfig;
use agen::llm_client::types::Item;
use protocol::Segment;
use protocol::{RunFailureKind, RunResumeSource, RunYieldReason, Segment};
/// State snapshot for creating a SegmentStart entry.
pub struct SegmentStartState<'a> {
@@ -332,6 +332,49 @@ pub fn save_run_completed(
)
}
/// Log a non-terminal yield of one logical Run. New writers must use this
/// instead of encoding `EngineResult::Yielded` in `RunCompleted`.
pub fn save_run_yielded(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
reason: RunYieldReason,
active_run_turn_count: usize,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::RunYielded {
ts: segment_log::now_millis(),
entry_id: None,
reason,
active_run_turn_count,
},
)
}
/// Log the boundary immediately before Engine resumes one logical Run.
pub fn save_run_resumed(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
source: RunResumeSource,
active_run_turn_count: usize,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::RunResumed {
ts: segment_log::now_millis(),
entry_id: None,
source,
active_run_turn_count,
},
)
}
/// Log a `RunErrored` entry — `run()` / `resume()` returned `Err(EngineError)`.
///
/// `EngineError` is not `Serialize`, so the caller passes a lossy
@@ -349,8 +392,10 @@ pub fn save_run_errored(
segment_id,
LogEntry::RunErrored {
ts: segment_log::now_millis(),
entry_id: None,
interrupted,
message,
failure: Some(RunFailureKind::Engine),
},
)
}
@@ -526,7 +571,8 @@ pub fn append_entry(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
entry: LogEntry,
mut entry: LogEntry,
) -> Result<(), StoreError> {
entry.ensure_session_entry_id();
store.append(session_id, segment_id, &entry)
}
+205 -7
View File
@@ -11,7 +11,7 @@
use agen::llm_client::types::{Item, RequestConfig};
use agen::{EngineResult, UsageRecord};
use protocol::{InvokeKind, Segment};
use protocol::{InvokeKind, RunFailureKind, RunResumeSource, RunYieldReason, Segment};
use serde::{Deserialize, Serialize};
use crate::history::{LoggedHistoryEntry, LoggedSystemHistoryEntry};
@@ -28,7 +28,7 @@ use crate::logged_item::LoggedItem;
/// `run()`/`resume()` (current callers persist a single TurnEnd at
/// run completion); the fork-point seq for `at_turn_index` is the
/// preceding `Invoke` entry, not the TurnEnd.
/// - `RunCompleted` / `RunErrored` — marks end of a `run()` or `resume()` call
/// - `RunCompleted` / `RunCancelled` / `RunErrored` — terminal run outcomes
/// - `PausedTurnAbandoned` — explicit abandon/cancel of a paused interrupted turn
/// - `ConfigChanged` — `RequestConfig` mutation
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
@@ -86,9 +86,10 @@ pub enum LogEntry {
/// serde tag already occupies `"kind"`.
///
/// Replay marks the run interrupted until a terminal `RunCompleted`,
/// `RunErrored`, or `PausedTurnAbandoned` entry proves how it ended. This
/// makes a process/disk failure between Invoke and its terminal record
/// restore conservatively instead of re-running a dangling tool call.
/// `RunCancelled`, `RunErrored`, or `PausedTurnAbandoned` entry proves how
/// it ended. This makes a process/disk failure between Invoke and its
/// terminal record restore conservatively instead of re-running a dangling
/// tool call.
Invoke { ts: u64, trigger: InvokeKind },
/// Canonical user submission with its exact model-visible entries. Typed
@@ -106,6 +107,9 @@ pub enum LogEntry {
/// Canonical tool output and metadata committed as one journal record.
AnnotatedToolResult { ts: u64, entry: LoggedHistoryEntry },
/// Append-only correction of a provider-rejected image result. Retains the
/// original stable identity; replay replaces rather than appends the item.
ToolResultCorrected { ts: u64, entry: LoggedHistoryEntry },
/// Canonical typed system event and model-visible metadata committed
/// together.
@@ -126,19 +130,61 @@ pub enum LogEntry {
ts: u64,
interrupted: bool,
result: EngineResult,
/// AgentTurns consumed by a paused/yielded logical run. Terminal
/// outcomes persist `None`.
/// AgentTurns consumed by a paused legacy logical run. Terminal
/// outcomes persist `None`. Legacy logs may also contain Yielded here;
/// new yields use `RunYielded` below.
#[serde(default, skip_serializing_if = "Option::is_none")]
active_run_turn_count: Option<usize>,
},
/// The current logical Run returned control so compaction can prepare a new
/// Segment. This is an intermediate transition, not a successful terminal
/// result. Ordering plus Session/Segment lineage correlate it with the
/// preceding Invoke and a later RunResumed or terminal record.
RunYielded {
ts: u64,
/// Stable public identity allocated before the record is committed.
/// `None` is accepted only when reading logs written before this field
/// existed; read paths derive the same legacy identity from the durable
/// record position without rewriting the log.
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
reason: RunYieldReason,
active_run_turn_count: usize,
},
/// Resume preparation has completed and Engine execution is about to begin.
/// A committed record is required before control is handed back to Engine.
/// It deliberately does not assert that a subsequent model call occurred.
RunResumed {
ts: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
source: RunResumeSource,
active_run_turn_count: usize,
},
/// The logical Run ended because the user intentionally cancelled it.
/// This is terminal, but is not an execution failure.
RunCancelled {
ts: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
},
/// `run()` / `resume()` が `EngineError` で終了した。
/// `EngineError` は `Serialize` 不可なので `message` のみ lossy 保持する。
/// Audit-only metadata: replay は `interrupted` のみ反映する。
RunErrored {
ts: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
entry_id: Option<crate::LoggedSessionHistoryEntryId>,
interrupted: bool,
message: String,
/// Older records omit this field. New records distinguish failures in
/// Engine execution from failure to continue after a compaction yield.
#[serde(default, skip_serializing_if = "Option::is_none")]
failure: Option<RunFailureKind>,
},
/// Restores an active logical-run budget at a segment boundary, notably
@@ -207,6 +253,48 @@ pub struct SegmentOrigin {
pub at_turn_index: usize,
}
impl LogEntry {
/// Allocate the durable public identity for a newly committed Run
/// transition. Calling this repeatedly is idempotent so the exact value
/// written to storage is also the value published to live subscribers.
pub fn ensure_session_entry_id(&mut self) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
_ => return,
};
if entry_id.is_none() {
*entry_id = Some(crate::LoggedSessionHistoryEntryId::new());
}
}
/// Supply an in-memory compatibility identity for a legacy Run transition
/// that predates persisted entry IDs. The physical record position is
/// stable within the append-only Segment and distinguishes equal payloads.
/// This helper never writes the synthesized value back to the log.
pub(crate) fn ensure_legacy_session_entry_id(
&mut self,
session_id: crate::SessionId,
segment_id: crate::SegmentId,
record_start: u64,
) {
let entry_id = match self {
Self::RunYielded { entry_id, .. }
| Self::RunResumed { entry_id, .. }
| Self::RunCancelled { entry_id, .. }
| Self::RunErrored { entry_id, .. } => entry_id,
_ => return,
};
if entry_id.is_none() {
*entry_id = Some(crate::LoggedSessionHistoryEntryId(format!(
"l-run-{session_id}-{segment_id}-{record_start}"
)));
}
}
}
/// State collected from log entries.
#[derive(Debug, Clone)]
pub struct RestoredState {
@@ -314,6 +402,16 @@ pub fn collect_state(entries: &[LogEntry]) -> RestoredState {
state.annotated_history.push(entry.clone());
state.history.push(Item::from(entry.item.clone()));
}
LogEntry::ToolResultCorrected { entry, .. } => {
if let Some(index) = state
.annotated_history
.iter()
.position(|existing| existing.metadata.entry_id == entry.metadata.entry_id)
{
state.annotated_history[index] = entry.clone();
state.history[index] = Item::from(entry.item.clone());
}
}
LogEntry::AnnotatedSystemItem {
entry, extensions, ..
} => {
@@ -351,6 +449,24 @@ pub fn collect_state(entries: &[LogEntry]) -> RestoredState {
state.active_run_turn_count = None;
}
}
LogEntry::RunYielded {
active_run_turn_count,
..
}
| LogEntry::RunResumed {
active_run_turn_count,
..
} => {
// Both are resumable boundaries of the same logical Run. A log
// ending at RunResumed proves only that the resume boundary was
// committed, not that Engine produced later output.
state.last_run_interrupted = true;
state.active_run_turn_count = Some(*active_run_turn_count);
}
LogEntry::RunCancelled { .. } => {
state.last_run_interrupted = false;
state.active_run_turn_count = None;
}
LogEntry::RunErrored { interrupted, .. } => {
state.last_run_interrupted = *interrupted;
state.active_run_turn_count = None;
@@ -818,6 +934,88 @@ mod tests {
assert!(state.last_run_interrupted);
}
#[test]
fn replay_new_yield_and_resume_keep_one_logical_run_budget() {
let state = collect_state(&[
LogEntry::Invoke {
ts: 100,
trigger: InvokeKind::UserSend,
},
LogEntry::TurnEnd {
ts: 200,
turn_count: 2,
},
LogEntry::RunYielded {
ts: 300,
entry_id: None,
reason: RunYieldReason::Compaction,
active_run_turn_count: 2,
},
LogEntry::ActiveRunCheckpoint {
ts: 400,
active_turn_count: 2,
total_turn_count: 2,
},
LogEntry::RunResumed {
ts: 500,
entry_id: None,
source: RunResumeSource::Compaction,
active_run_turn_count: 2,
},
]);
assert!(state.last_run_interrupted);
assert_eq!(state.turn_count, 2);
assert_eq!(state.active_run_turn_count, Some(2));
}
#[test]
fn replay_cancelled_run_is_terminal_without_becoming_a_failure() {
let state = collect_state(&[
LogEntry::Invoke {
ts: 100,
trigger: InvokeKind::UserSend,
},
LogEntry::TurnEnd {
ts: 200,
turn_count: 1,
},
LogEntry::RunCancelled {
ts: 300,
entry_id: None,
},
]);
assert!(!state.last_run_interrupted);
assert_eq!(state.active_run_turn_count, None);
assert_eq!(state.turn_count, 1);
}
#[test]
fn legacy_yielded_run_completed_remains_resumable_but_is_not_terminal_success() {
let entry: LogEntry = serde_json::from_value(serde_json::json!({
"kind": "run_completed",
"ts": 300,
"interrupted": true,
"result": "yielded"
}))
.expect("legacy yielded run-completed entry");
let state = collect_state(&[
LogEntry::Invoke {
ts: 100,
trigger: InvokeKind::UserSend,
},
LogEntry::TurnEnd {
ts: 200,
turn_count: 3,
},
entry,
]);
assert!(state.last_run_interrupted);
assert_eq!(state.active_run_turn_count, Some(3));
}
#[test]
fn legacy_interrupted_run_derives_budget_from_invoke_and_turn_end() {
let entry: LogEntry = serde_json::from_value(serde_json::json!({
+8 -1
View File
@@ -110,7 +110,14 @@ pub trait Store: Send + Sync {
/// segment ID without its session ID (e.g. legacy `--session <UUID>`).
fn lookup_session_of(&self, segment_id: SegmentId) -> Result<Option<SessionId>, StoreError>;
/// Create a new segment within `session_id`, with initial entries.
/// Create or atomically replace a segment within `session_id` with all entries.
///
/// Readers and crash recovery must observe either the previous complete log
/// or the complete replacement, never a truncated prefix or partial record.
/// Failure before replacement leaves the old log intact; an error after the
/// atomic commit (for example a durability sync failure) may leave the complete
/// replacement. Filesystem implementations must sync replacement bytes before
/// rename and the containing directory afterward before reporting success.
fn create_segment(
&self,
session_id: SessionId,
+75
View File
@@ -123,6 +123,14 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Result of an explicitly selected chat Feature invocation. The typed
/// result is retained for clients; `body` is the exact bounded context made
/// visible to the Worker model.
FeatureInvocationResult {
result: protocol::FeatureInvocationResult,
body: String,
},
/// `@<path>` file reference resolution. `body` is the rendered
/// LLM-context text (`[File: <path>]\n…` for regular files,
/// `[Dir: <path>]\n…` for directory listings, possibly with a
@@ -157,6 +165,25 @@ pub enum SystemItem {
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Restore-boundary refresh of the current resident Memory projection.
/// The exact rendered body is retained so replay is byte-identical; a
/// refresh with no ready summary explicitly supersedes older resident
/// context without deleting or rewriting history.
ResidentSummaryRefresh {
body: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// A Host-fetched change to user-managed Subject behavior. Each changed
/// revision is appended rather than rewriting prior prompt/session content.
SubjectBehaviorRefresh {
body: String,
behavior_revision: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
prompt_provenance: Option<PromptRenderProvenance>,
},
/// Synthetic note inserted after an interrupted turn before the next
/// user input. `body` is the exact LLM-context text explaining that the
/// previous turn was cut short.
@@ -174,6 +201,7 @@ impl SystemItem {
match self {
SystemItem::Notification { body, .. } => body.clone(),
SystemItem::WorkerEvent { body, .. } => body.clone(),
SystemItem::FeatureInvocationResult { body, .. } => body.clone(),
SystemItem::FileAttachment { body, .. } => body.clone(),
SystemItem::SkillActivation { body, .. } => body.clone(),
SystemItem::LegacyKnowledgeIgnored { .. } => String::new(),
@@ -181,6 +209,8 @@ impl SystemItem {
format!("Ignored legacy procedure item: /{slug}")
}
SystemItem::TaskReminder { body, .. } => body.clone(),
SystemItem::ResidentSummaryRefresh { body, .. } => body.clone(),
SystemItem::SubjectBehaviorRefresh { body, .. } => body.clone(),
SystemItem::Interrupt { body, .. } => body.clone(),
}
}
@@ -197,11 +227,14 @@ impl SystemItem {
match self {
SystemItem::Notification { .. } => "notification",
SystemItem::WorkerEvent { .. } => "worker_event",
SystemItem::FeatureInvocationResult { .. } => "feature_invocation_result",
SystemItem::FileAttachment { .. } => "file_attachment",
SystemItem::SkillActivation { .. } => "skill_activation",
SystemItem::LegacyKnowledgeIgnored { .. } => "legacy_knowledge_ignored",
SystemItem::LegacyIgnored { .. } => "legacy_ignored",
SystemItem::TaskReminder { .. } => "task_reminder",
SystemItem::ResidentSummaryRefresh { .. } => "resident_summary_refresh",
SystemItem::SubjectBehaviorRefresh { .. } => "subject_behavior_refresh",
SystemItem::Interrupt { .. } => "interrupt",
}
}
@@ -258,6 +291,25 @@ mod tests {
));
}
#[test]
fn subject_behavior_refresh_round_trips_with_revision_and_exact_body() {
let item = SystemItem::SubjectBehaviorRefresh {
body: "current Subject context".to_string(),
behavior_revision: 7,
prompt_provenance: None,
};
let raw = serde_json::to_string(&item).unwrap();
let parsed: SystemItem = serde_json::from_str(&raw).unwrap();
assert_eq!(parsed.history_text(), "current Subject context");
assert!(matches!(
parsed,
SystemItem::SubjectBehaviorRefresh {
behavior_revision: 7,
..
}
));
}
#[test]
fn notification_history_text_returns_stored_body() {
let item = SystemItem::Notification {
@@ -351,6 +403,29 @@ mod tests {
}
}
#[test]
fn invocation_result_retains_typed_outcome_and_exact_context() {
let result = protocol::FeatureInvocationResult {
invocation_id: "stable-invoke".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
status: protocol::FeatureInvocationStatus::OutcomeUnknown,
message: "operation may have committed".into(),
context: None,
};
let body = "Outcome unknown; do not retry automatically";
let item = SystemItem::FeatureInvocationResult {
result: result.clone(),
body: body.into(),
};
let restored: SystemItem =
serde_json::from_str(&serde_json::to_string(&item).unwrap()).unwrap();
assert_eq!(restored.history_text(), body);
assert_eq!(restored.kind_label(), "feature_invocation_result");
assert!(
matches!(restored, SystemItem::FeatureInvocationResult { result: saved, .. } if saved == result)
);
}
#[test]
fn round_trip_worker_event() {
let item = SystemItem::WorkerEvent {
+152 -3
View File
@@ -112,6 +112,47 @@ pub struct WorkerPeer {
pub worker_name: String,
}
/// Durable completion state for the Host-owned Subjektiv Session attribution
/// lifecycle. The Host writes `OutcomeUnknown` before sending the request, so a
/// process interruption or lost response remains explicitly recoverable by the
/// next Worker create/restore lifecycle instead of leaking into ordinary Runs.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "state", rename_all = "snake_case", deny_unknown_fields)]
pub enum SubjektivSessionAttributionState {
/// A lifecycle-created Session sent a create-if-missing request but the
/// Host could not determine whether the Backend committed it.
OutcomeUnknown {
session_id: SessionId,
diagnostic: String,
},
/// A legacy restored Session has no local lifecycle evidence. Recovery may
/// verify an immutable Backend record but must never create one from current
/// ownership.
LegacyUnknown {
session_id: SessionId,
diagnostic: String,
},
Failed {
session_id: SessionId,
diagnostic: String,
},
Confirmed {
session_id: SessionId,
subject_id: String,
},
}
impl SubjektivSessionAttributionState {
pub fn session_id(&self) -> SessionId {
match self {
Self::OutcomeUnknown { session_id, .. }
| Self::LegacyUnknown { session_id, .. }
| Self::Failed { session_id, .. }
| Self::Confirmed { session_id, .. } => *session_id,
}
}
}
/// Persistent metadata for a Worker name.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct WorkerMetadata {
@@ -133,6 +174,10 @@ pub struct WorkerMetadata {
pub peers: Vec<WorkerPeer>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub resolved_manifest_snapshot: Option<serde_json::Value>,
/// Host-owned completion evidence for Session-to-Subject attribution. Its
/// absence is also the legacy representation and never grants visibility.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub subjektiv_session_attribution: Option<SubjektivSessionAttributionState>,
}
impl WorkerMetadata {
@@ -147,6 +192,7 @@ impl WorkerMetadata {
reclaimed_children: Vec::new(),
peers: Vec::new(),
resolved_manifest_snapshot: None,
subjektiv_session_attribution: None,
}
}
@@ -161,6 +207,18 @@ impl WorkerMetadata {
}
}
fn replace_active_session(metadata: &mut WorkerMetadata, active: Option<WorkerActiveSegmentRef>) {
if let Some(session_id) = active.as_ref().map(|active| active.session_id)
&& metadata
.subjektiv_session_attribution
.as_ref()
.is_some_and(|state| state.session_id() != session_id)
{
metadata.subjektiv_session_attribution = None;
}
metadata.active = active;
}
/// Sync persistence backend for Worker metadata.
pub trait WorkerMetadataStore: Send + Sync {
/// Create or replace metadata for its `worker_name` key.
@@ -236,7 +294,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.active = active;
replace_active_session(metadata, active);
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -256,7 +314,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_id: Option<String>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.active = active;
replace_active_session(metadata, active);
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_id) = workspace_id {
metadata.workspace_id = Some(workspace_id);
@@ -273,7 +331,7 @@ pub trait WorkerMetadataStore: Send + Sync {
workspace_root: Option<PathBuf>,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.active = active;
replace_active_session(metadata, active);
metadata.resolved_manifest_snapshot = resolved_manifest_snapshot;
if let Some(workspace_root) = workspace_root {
metadata.workspace_root = Some(workspace_root);
@@ -281,6 +339,18 @@ pub trait WorkerMetadataStore: Send + Sync {
})
}
/// Set the Host-owned Subjektiv Session attribution lifecycle state while
/// preserving the active Session pointer and every unrelated field.
fn set_subjektiv_session_attribution(
&self,
worker_name: &str,
state: SubjektivSessionAttributionState,
) -> Result<WorkerMetadata, WorkerStoreError> {
self.update_by_name(worker_name, |metadata| {
metadata.subjektiv_session_attribution = Some(state);
})
}
/// Set spawned-child registry state while preserving active pointer and manifest snapshot.
fn set_spawned_children(
&self,
@@ -993,6 +1063,85 @@ mod tests {
assert_eq!(restored, metadata);
}
#[test]
fn subjektiv_attribution_state_roundtrips_and_active_updates_preserve_it() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let session_id = crate::new_session_id();
store
.write(&WorkerMetadata::new(
"subject-worker",
Some(WorkerActiveSegmentRef::pending_segment(session_id)),
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::OutcomeUnknown {
session_id,
diagnostic: "request interrupted".to_string(),
},
)
.unwrap();
store
.set_active(
"subject-worker",
Some(WorkerActiveSegmentRef::active_segment(
session_id,
crate::new_segment_id(),
)),
None,
)
.unwrap();
let restored = store.read_by_name("subject-worker").unwrap().unwrap();
assert!(matches!(
restored.subjektiv_session_attribution,
Some(SubjektivSessionAttributionState::OutcomeUnknown {
session_id: stored_session_id,
ref diagnostic,
}) if stored_session_id == session_id && diagnostic == "request interrupted"
));
}
#[test]
fn new_active_session_clears_prior_session_attribution_state() {
let tmp = tempfile::TempDir::new().unwrap();
let store = FsWorkerStore::new(tmp.path()).unwrap();
let old_session_id = crate::new_session_id();
let new_session_id = crate::new_session_id();
store
.write(&WorkerMetadata::new(
"subject-worker",
Some(WorkerActiveSegmentRef::pending_segment(old_session_id)),
))
.unwrap();
store
.set_subjektiv_session_attribution(
"subject-worker",
SubjektivSessionAttributionState::Confirmed {
session_id: old_session_id,
subject_id: "subject-1".to_string(),
},
)
.unwrap();
store
.set_active(
"subject-worker",
Some(WorkerActiveSegmentRef::pending_segment(new_session_id)),
None,
)
.unwrap();
let restored = store.read_by_name("subject-worker").unwrap().unwrap();
assert_eq!(
restored.active,
Some(WorkerActiveSegmentRef::pending_segment(new_session_id))
);
assert!(restored.subjektiv_session_attribution.is_none());
}
#[test]
fn spawned_scope_rule_defaults_resolved_and_roundtrips_logical_policy() {
let legacy: WorkerSpawnedScopeRule = serde_json::from_value(serde_json::json!({
@@ -61,6 +61,8 @@ pub(crate) struct RetainedLogRecord {
pub(crate) struct RetainedSegmentReader {
file: File,
session_id: SessionId,
segment_id: SegmentId,
read_end: u64,
buffer_start: u64,
buffer: Vec<u8>,
@@ -103,7 +105,12 @@ impl RetainedSegmentReader {
message: "empty retained Segment log record".to_string(),
});
}
let entry = serde_json::from_slice(&record)?;
let mut entry: LogEntry = serde_json::from_slice(&record)?;
entry.ensure_legacy_session_entry_id(
self.session_id,
self.segment_id,
record_start,
);
return Ok((
Some(RetainedLogRecord {
start_offset: record_start,
@@ -255,7 +262,10 @@ impl WorkerSessionStore {
if byte_len > max_bytes {
return Err(StoreError::ReadLimitExceeded);
}
Ok((parse_jsonl(&bytes)?, byte_len))
Ok((
parse_session_log_jsonl(&bytes, session_id, segment_id)?,
byte_len,
))
}
pub(crate) fn read_first_log_record_read_only_bounded(
@@ -264,6 +274,7 @@ impl WorkerSessionStore {
segment_id: SegmentId,
max_bytes: u64,
) -> Result<(Option<RetainedLogRecord>, u64, u64), StoreError> {
let record_start = 0;
self.validate_retained_session(session_id)?;
let file = open_retained_file(&self.log_path(segment_id)).map_err(|error| {
if error.kind() == std::io::ErrorKind::NotFound {
@@ -289,7 +300,8 @@ impl WorkerSessionStore {
message: "empty retained Segment log record".to_string(),
});
}
let entry = serde_json::from_slice(&record)?;
let mut entry: LogEntry = serde_json::from_slice(&record)?;
entry.ensure_legacy_session_entry_id(session_id, segment_id, record_start);
Ok((
Some(RetainedLogRecord {
start_offset: 0,
@@ -308,6 +320,7 @@ impl WorkerSessionStore {
start_offset: u64,
max_bytes: u64,
) -> Result<(Option<RetainedLogRecord>, u64), StoreError> {
let record_start = start_offset;
self.validate_retained_session(session_id)?;
let mut file = open_retained_file(&self.log_path(segment_id)).map_err(|error| {
if error.kind() == std::io::ErrorKind::NotFound {
@@ -334,7 +347,8 @@ impl WorkerSessionStore {
message: "empty retained Segment log record".to_string(),
});
}
let entry = serde_json::from_slice(&record)?;
let mut entry: LogEntry = serde_json::from_slice(&record)?;
entry.ensure_legacy_session_entry_id(session_id, segment_id, record_start);
Ok((
Some(RetainedLogRecord {
start_offset,
@@ -353,6 +367,7 @@ impl WorkerSessionStore {
end_offset: u64,
max_bytes: u64,
) -> Result<RetainedLogRecord, StoreError> {
let record_start = start_offset;
self.validate_retained_session(session_id)?;
let record_len =
end_offset
@@ -387,7 +402,8 @@ impl WorkerSessionStore {
});
}
record.pop();
let entry = serde_json::from_slice(&record)?;
let mut entry: LogEntry = serde_json::from_slice(&record)?;
entry.ensure_legacy_session_entry_id(session_id, segment_id, record_start);
Ok(RetainedLogRecord {
start_offset,
end_offset,
@@ -429,6 +445,8 @@ impl WorkerSessionStore {
}
Ok(RetainedSegmentReader {
file,
session_id,
segment_id,
read_end: before_offset,
buffer_start: before_offset,
buffer: Vec::new(),
@@ -450,6 +468,39 @@ impl WorkerSessionStore {
Ok(())
}
/// Enumerate canonical Segment logs in ascending identity order without
/// creating or migrating state. Unlike the general [`Store`] listing, this
/// observation boundary rejects malformed names and non-regular files so a
/// persisted Segment cannot be silently omitted from a public index.
pub(crate) fn list_segments_read_only(
&self,
session_id: SessionId,
) -> Result<Vec<SegmentId>, StoreError> {
self.validate_retained_session(session_id)?;
segment_log_paths(&self.root).map(|paths| {
paths
.into_iter()
.map(|(segment_id, _)| segment_id)
.collect()
})
}
pub(crate) fn segment_log_observation(
&self,
segment_id: SegmentId,
) -> Result<(u64, u128), StoreError> {
let metadata = fs::metadata(self.log_path(segment_id))?;
let modified = metadata
.modified()?
.duration_since(SystemTime::UNIX_EPOCH)
.map_err(|_| StoreError::Corrupt {
line: 0,
message: "retained Segment modification time predates the Unix epoch".to_string(),
})?
.as_nanos();
Ok((metadata.len(), modified))
}
pub fn segment_log_len(&self, segment_id: SegmentId) -> Result<u64, StoreError> {
Ok(fs::metadata(self.log_path(segment_id))?.len())
}
@@ -645,7 +696,7 @@ impl Store for WorkerSessionStore {
if !path.exists() {
return Err(StoreError::NotFound(segment_id));
}
parse_jsonl(&fs::read(path)?)
parse_session_log_jsonl(&fs::read(path)?, session_id, segment_id)
}
fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError> {
@@ -1115,7 +1166,7 @@ fn atomic_write_json<T: Serialize>(path: &Path, value: &T) -> Result<(), StoreEr
atomic_write_bytes(path, &bytes)
}
fn atomic_write_bytes(path: &Path, bytes: &[u8]) -> Result<(), StoreError> {
pub(crate) fn atomic_write_bytes(path: &Path, bytes: &[u8]) -> Result<(), StoreError> {
let parent = path
.parent()
.ok_or_else(|| std::io::Error::other("Worker Session path has no parent"))?;
@@ -1154,6 +1205,38 @@ fn complete_jsonl_prefix(content: &[u8]) -> &[u8] {
.unwrap_or(&[])
}
fn parse_session_log_jsonl(
content: &[u8],
session_id: SessionId,
segment_id: SegmentId,
) -> Result<Vec<LogEntry>, StoreError> {
let complete = complete_jsonl_prefix(content);
let content = std::str::from_utf8(complete).map_err(|error| StoreError::Corrupt {
line: complete[..error.valid_up_to()]
.iter()
.filter(|byte| **byte == b'\n')
.count()
+ 1,
message: error.to_string(),
})?;
let mut entries = Vec::new();
let mut record_start = 0_u64;
for (index, record) in content.split_inclusive('\n').enumerate() {
let line = record.strip_suffix('\n').unwrap_or(record);
if !line.trim().is_empty() {
let mut entry: LogEntry =
serde_json::from_str(line).map_err(|error| StoreError::Corrupt {
line: index + 1,
message: error.to_string(),
})?;
entry.ensure_legacy_session_entry_id(session_id, segment_id, record_start);
entries.push(entry);
}
record_start = record_start.saturating_add(record.len() as u64);
}
Ok(entries)
}
fn parse_jsonl<T: serde::de::DeserializeOwned>(content: &[u8]) -> Result<Vec<T>, StoreError> {
let complete = complete_jsonl_prefix(content);
let content = std::str::from_utf8(complete).map_err(|error| StoreError::Corrupt {
@@ -115,6 +115,100 @@ fn create_segment_writes_all_entries() {
assert_eq!(state.session_id, Some(sid));
}
#[test]
fn create_segment_replacement_keeps_open_old_log_and_commits_complete_new_log() {
use std::io::Read;
let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let sid = new_session_id();
let segid = new_segment_id();
let old = vec![
nil_session_start(1, sid),
LogEntry::TurnEnd {
ts: 2,
turn_count: 1,
},
];
store.create_segment(sid, segid, &old).unwrap();
let path = dir
.path()
.join(sid.to_string())
.join(format!("{segid}.jsonl"));
let old_bytes = std::fs::read(&path).unwrap();
let mut old_reader = std::fs::File::open(&path).unwrap();
let new = vec![
nil_session_start(1, sid),
LogEntry::Extension {
ts: 3,
domain: "recovery-checkpoint".into(),
payload: serde_json::json!({"receipt": "retained"}),
},
];
store.create_segment(sid, segid, &new).unwrap();
let mut retained_inode = Vec::new();
old_reader.read_to_end(&mut retained_inode).unwrap();
// An in-place fs::write would truncate the inode held by this reader.
assert_eq!(retained_inode, old_bytes);
let reopened = FsStore::new(dir.path()).unwrap();
assert_eq!(
serde_json::to_value(reopened.read_all(sid, segid).unwrap()).unwrap(),
serde_json::to_value(&new).unwrap()
);
assert_eq!(reopened.read_entry_count(sid, segid).unwrap(), new.len());
assert_eq!(
std::fs::read_dir(path.parent().unwrap()).unwrap().count(),
1,
"successful replacement must not leave temporary files"
);
}
#[test]
fn create_segment_atomic_replacement_readers_only_observe_complete_old_or_new() {
let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let sid = new_session_id();
let segid = new_segment_id();
let old = vec![
nil_session_start(1, sid),
LogEntry::Extension {
ts: 2,
domain: "old-evidence".into(),
payload: serde_json::json!({"result": "old"}),
},
];
let new = vec![
nil_session_start(1, sid),
LogEntry::Extension {
ts: 3,
domain: "new-checkpoint".into(),
payload: serde_json::json!({"result": "new"}),
},
];
store.create_segment(sid, segid, &old).unwrap();
let old_json = serde_json::to_value(&old).unwrap();
let new_json = serde_json::to_value(&new).unwrap();
let barrier = std::sync::Arc::new(std::sync::Barrier::new(2));
let reader_store = store.clone();
let reader_barrier = barrier.clone();
let reader = std::thread::spawn(move || {
reader_barrier.wait();
for _ in 0..1000 {
let observed =
serde_json::to_value(reader_store.read_all(sid, segid).unwrap()).unwrap();
assert!(
observed == old_json || observed == new_json,
"replacement must never expose a partial log: {observed}"
);
}
});
barrier.wait();
for _ in 0..50 {
store.create_segment(sid, segid, &new).unwrap();
store.create_segment(sid, segid, &old).unwrap();
}
reader.join().unwrap();
}
#[test]
fn list_sessions_and_segments() {
let dir = tempfile::tempdir().unwrap();
+80 -5
View File
@@ -179,14 +179,13 @@ async fn run_and_persist(
session_store::save_turn_end(store, session_id, segment_id, worker.turn_count()).unwrap();
match &result {
agen::EngineRunExit::Finished
| agen::EngineRunExit::Paused
| agen::EngineRunExit::Yielded => {
agen::EngineRunExit::Finished | agen::EngineRunExit::Paused => {
let (legacy_result, interrupted) = match &result {
agen::EngineRunExit::Finished => (agen::EngineResult::Finished, false),
agen::EngineRunExit::Paused => (agen::EngineResult::Paused, true),
agen::EngineRunExit::Yielded => (agen::EngineResult::Yielded, true),
agen::EngineRunExit::Interrupted(_) => unreachable!(),
agen::EngineRunExit::Yielded | agen::EngineRunExit::Interrupted(_) => {
unreachable!()
}
};
session_store::save_run_completed(
store,
@@ -198,6 +197,18 @@ async fn run_and_persist(
)
.unwrap();
}
agen::EngineRunExit::Yielded => {
session_store::save_run_yielded(
store,
session_id,
segment_id,
protocol::RunYieldReason::Compaction,
worker
.active_run_turn_count()
.expect("yielded test run retains its turn budget"),
)
.unwrap();
}
agen::EngineRunExit::Interrupted(agen::RunInterruptionReason::LimitReached) => {
session_store::save_run_completed(
store,
@@ -595,6 +606,70 @@ fn rewound_fork_preserves_uploaded_file_segments_in_snapshot() {
)));
}
#[test]
fn feature_invocations_survive_replay_snapshot_and_rewind_without_reparsing() {
let (_dir, store) = make_store();
let config = RequestConfig::default();
let (sid, segid) = session_store::create_segment(
&store,
SegmentStartState {
system_prompt: Some("System prompt"),
config: &config,
history: Vec::new(),
user_segments: Vec::new(),
},
)
.unwrap();
let segments = vec![
Segment::Text {
content: "before ".into(),
},
Segment::FeatureInvoke {
invocation: protocol::FeatureInvocation {
invocation_id: "stable-request-1".into(),
identity: protocol::FeatureInvocationIdentity("builtin:test/prepare".into()),
name: "prepare".into(),
arguments: vec![protocol::InvocationArgumentValue {
name: "path".into(),
value: protocol::InvocationValue::String("資料/a b".into()),
}],
},
},
Segment::Text {
content: " after /prepare(unselected)".into(),
},
];
session_store::save_user_input(
&store,
sid,
segid,
segments.clone(),
annotated(&[Item::user_message(Segment::flatten_to_text(&segments))]),
)
.unwrap();
session_store::save_turn_end(&store, sid, segid, 1).unwrap();
let entries = store.read_all(sid, segid).unwrap();
assert_eq!(
collect_state(&entries).user_segments,
vec![segments.clone()]
);
let fork_id = session_store::fork_at(&store, sid, segid, 1).unwrap();
let fork_entries = store.read_all(sid, fork_id).unwrap();
assert_eq!(
collect_state(&fork_entries).user_segments,
vec![segments.clone()]
);
let snapshot = session_store::public_snapshot::project_session_snapshot(sid, &fork_entries);
assert!(snapshot.entries.iter().any(|entry| matches!(
&entry.data, SessionSnapshotEntryData::UserInput { segments: restored } if restored == &segments
)));
// Original history remains append-only and retains the same exact typed intent.
assert_eq!(
serde_json::to_value(store.read_all(sid, segid).unwrap()).unwrap(),
serde_json::to_value(entries).unwrap()
);
}
#[tokio::test]
async fn session_config_changed_logged() {
let (_dir, store) = make_store();
+8
View File
@@ -9,7 +9,15 @@ license.workspace = true
agen.workspace = true
client.workspace = true
fs4.workspace = true
feature-storage.workspace = true
subjektiv.workspace = true
memory.workspace = true
server-api.workspace = true
libc.workspace = true
async-trait.workspace = true
manifest.workspace = true
job.workspace = true
rusqlite.workspace = true
protocol.workspace = true
serde.workspace = true
serde_json.workspace = true
+253 -1
View File
@@ -43,12 +43,16 @@ pub struct StandaloneHost {
worker_store: FsWorkerStore,
record: StandaloneWorkerRecord,
lease: Option<StandaloneWorkerLease>,
jobs: crate::jobs::StandaloneJobs,
subject: Option<crate::subjektiv::SubjectConnection>,
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum StandaloneStartupError {
#[error("the standalone state store could not be opened or validated")]
StateStore,
#[error("local Subject connection rejected: {0}")]
Subject(String),
#[error("the standalone Worker is already active")]
WorkerActive,
#[error("the standalone Worker lease cannot be observed safely; recovery is rejected")]
@@ -77,6 +81,8 @@ pub enum StandaloneStartupError {
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
pub enum StandaloneShutdownError {
#[error("standalone Job execution cleanup or terminal persistence could not be confirmed")]
JobCleanup,
#[error("the standalone Worker did not stop before the shutdown deadline")]
DeadlineExceeded,
#[error("the standalone Worker shutdown confirmation was lost")]
@@ -104,13 +110,66 @@ impl StandaloneHost {
launch: ResolvedStandaloneLaunch,
model_client: Option<Box<dyn LlmClient>>,
) -> Result<Self, StandaloneStartupError> {
if launch.subject_id.is_some()
&& (!launch.profile.manifest.feature.subjektiv.profile.enabled
|| launch
.profile
.manifest
.feature
.subjektiv
.profile
.consolidation_tools
|| launch
.profile
.manifest
.feature
.subjektiv
.workspace_settings
.is_some())
{
return Err(StandaloneStartupError::Subject(
"incompatible local Subject policy".into(),
));
}
let store =
StandaloneWorkerStore::open(&launch.state_dir).map_err(classify_store_startup_error)?;
let allocation = store
.allocate(&launch.cwd, StaleLeasePolicy::Reject)
.map_err(classify_store_startup_error)?;
let worker_id = allocation.worker_id();
let mut subject = match launch
.subject_id
.as_deref()
.map(|id| {
crate::subjektiv::StandaloneSubjects::open_existing(&launch.state_dir)?
.connect(id, worker_id)
})
.transpose()
{
Ok(subject) => subject,
Err(error) => {
let _ = store.abandon_allocation(allocation);
return Err(StandaloneStartupError::Subject(error.to_string()));
}
};
let jobs_path = subject
.as_ref()
.map(|s| s.jobs_path.clone())
.unwrap_or_else(|| store.jobs_path(worker_id));
let jobs = match crate::jobs::StandaloneJobs::open(&jobs_path, launch.cwd.clone()) {
Ok(jobs) => jobs,
Err(_) => {
let _ = store.abandon_allocation(allocation);
return Err(StandaloneStartupError::StateStore);
}
};
if let Some(client) = &model_client {
jobs.bind_model_client(client.clone_boxed());
}
if let Some(subject) = &subject {
subject.host.bind_jobs(jobs.clone())?;
}
// WorkerId is the stable identity. The current Worker store remains
// name-keyed, so keep its derived storage key separate from the
// user-facing profile name.
@@ -143,9 +202,15 @@ impl StandaloneHost {
},
WorkerControllerTransport::InProcess,
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
}
if let Some(model_client) = model_client {
bootstrap = bootstrap.with_model_client(model_client);
}
if let Some(subject) = &mut subject {
subject.arm();
}
let started = match bootstrap.start().await {
Ok(started) => started,
Err(error) => {
@@ -161,12 +226,13 @@ impl StandaloneHost {
return Err(error);
}
};
let record = match store.commit_created(
let record = match store.commit_created_connected(
&allocation,
manifest,
storage_key,
active.session_id,
active.segment_id,
subject.as_ref().map(|s| s.binding.clone()),
) {
Ok(record) => record,
Err(_) => {
@@ -181,6 +247,8 @@ impl StandaloneHost {
worker_store,
record,
allocation.into_lease(),
jobs,
subject,
))
}
@@ -216,6 +284,50 @@ impl StandaloneHost {
let lease = store
.acquire_lease(worker_id, StaleLeasePolicy::Recover)
.map_err(classify_store_startup_error)?;
let mut subject = record
.subject
.as_ref()
.map(|binding| {
if !record.manifest.feature.subjektiv.profile.enabled
|| record
.manifest
.feature
.subjektiv
.profile
.consolidation_tools
|| record
.manifest
.feature
.subjektiv
.workspace_settings
.is_some()
{
return Err(crate::subjektiv::SubjectError::InvalidScope(
"persisted Subject policy changed".into(),
));
}
let catalog = crate::subjektiv::StandaloneSubjects::open_existing(store.root())?;
if catalog.scope_id() != binding.scope_id {
return Err(crate::subjektiv::SubjectError::InvalidScope(
"persisted local storage scope changed".into(),
));
}
catalog.connect(&binding.subject_id, worker_id)
})
.transpose()
.map_err(|e| StandaloneStartupError::Subject(e.to_string()))?;
let jobs_path = subject
.as_ref()
.map(|s| s.jobs_path.clone())
.unwrap_or_else(|| store.jobs_path(worker_id));
let jobs = crate::jobs::StandaloneJobs::open(&jobs_path, record.cwd.canonical_path.clone())
.map_err(|_| StandaloneStartupError::StateStore)?;
if let Some(client) = &model_client {
jobs.bind_model_client(client.clone_boxed());
}
if let Some(subject) = &subject {
subject.host.bind_jobs(jobs.clone())?;
}
let (backing_store, worker_store) = backing_store(&store, worker_id)?;
let storage_key = record.storage_key.clone();
let mut manifest = record.manifest.clone();
@@ -240,6 +352,9 @@ impl StandaloneHost {
},
WorkerControllerTransport::InProcess,
);
if let Some(subject) = &subject {
bootstrap = bootstrap.with_subjektiv_host(subject.host.clone());
}
if let Some(model_client) = model_client {
bootstrap = bootstrap.with_model_client(model_client);
}
@@ -247,6 +362,9 @@ impl StandaloneHost {
.prepare_restored(&storage_key)
.await
.map_err(classify_startup_error)?;
if let Some(subject) = &mut subject {
subject.arm();
}
let started = prepared.start().await.map_err(classify_startup_error)?;
let active = match active_pointer(&worker_store, &storage_key) {
Ok(active) => active,
@@ -270,6 +388,8 @@ impl StandaloneHost {
worker_store,
record,
lease,
jobs,
subject,
))
}
@@ -279,7 +399,12 @@ impl StandaloneHost {
worker_store: FsWorkerStore,
record: StandaloneWorkerRecord,
lease: StandaloneWorkerLease,
jobs: crate::jobs::StandaloneJobs,
subject: Option<crate::subjektiv::SubjectConnection>,
) -> Self {
if let Some(connection) = &subject {
connection.host.recover_jobs();
}
Self {
handle: started.handle,
shutdown: Some(started.shutdown),
@@ -288,9 +413,17 @@ impl StandaloneHost {
worker_store,
record,
lease: Some(lease),
jobs,
subject,
}
}
/// Explicit Host execution capability for local Features, separate from the
/// interactive Worker protocol and its model-created SubWorkers.
pub fn jobs(&self) -> crate::jobs::StandaloneJobs {
self.jobs.clone()
}
#[must_use]
pub fn worker_id(&self) -> WorkerId {
self.record.worker_id
@@ -318,6 +451,7 @@ impl StandaloneHost {
}
pub async fn shutdown(mut self) -> Result<(), StandaloneShutdownError> {
let jobs_result = self.jobs.shutdown().await;
let command = protocol::WorkerCommandEnvelope::new(u64::MAX);
let _ = self.handle.send(Method::Shutdown { command }).await;
let Some(shutdown) = self.shutdown.take() else {
@@ -335,6 +469,10 @@ impl StandaloneHost {
return Err(StandaloneShutdownError::DeadlineExceeded);
}
}
if jobs_result.is_err() {
self.retain_lease();
return Err(StandaloneShutdownError::JobCleanup);
}
let active = match active_pointer(&self.worker_store, &self.record.storage_key) {
Ok(active) => active,
Err(_) => {
@@ -355,6 +493,12 @@ impl StandaloneHost {
self.retain_lease();
return Err(StandaloneShutdownError::StateStore);
}
if let Some(subject) = &mut self.subject {
if subject.close().is_err() {
self.retain_lease();
return Err(StandaloneShutdownError::StateStore);
}
}
if let Some(lease) = self.lease.take() {
lease
.release()
@@ -364,12 +508,70 @@ impl StandaloneHost {
}
fn retain_lease(&mut self) {
if let Some(subject) = &mut self.subject {
subject.retain();
}
if let Some(lease) = self.lease.take() {
lease.retain();
}
}
}
impl Drop for StandaloneHost {
fn drop(&mut self) {
self.jobs.close();
// A dropped Host cannot synchronously confirm either controller or Job
// cleanup. Keep its lease until explicit shutdown or process recovery,
// preventing another Host from reopening a still-live execution store.
self.retain_lease();
}
}
// The in-process channel is owned by the local operator's Host, not an
// arbitrary socket or Backend account. decode_method discards wire provenance;
// only this boundary can stamp accepted local human input.
fn authorize_local_input(method: protocol::Method) -> protocol::Method {
use protocol::{AuthenticatedInputSource, Method};
match method {
Method::Submit {
submission_request_id,
input,
}
| Method::SubmitTracked {
submission_request_id,
input,
..
} => Method::SubmitTracked {
submission_request_id,
input,
source: AuthenticatedInputSource::LocalOperator,
},
Method::SubmitIfIdle {
submission_request_id,
input,
..
} => Method::SubmitIfIdle {
submission_request_id,
input,
source: AuthenticatedInputSource::LocalOperator,
},
Method::Notify {
notification_request_id,
message,
}
| Method::NotifyTracked {
notification_request_id,
message,
..
} => Method::NotifyTracked {
notification_request_id,
message,
source: AuthenticatedInputSource::LocalOperator,
},
method => method,
}
}
async fn run_protocol_session(
handle: worker::WorkerHandle,
streams: WorkerProtocolSessionStreams,
@@ -395,6 +597,7 @@ async fn run_protocol_session(
let Ok(method) = decode_method(&message) else {
return;
};
let method = authorize_local_input(method);
if let Some(event) = dispatch_worker_protocol_method(&handle, method).await
&& !send_protocol_event(&peer, event).await
{
@@ -535,6 +738,9 @@ fn classify_startup_error(error: WorkerBootstrapError) -> StandaloneStartupError
WorkerBootstrapError::Worker(WorkerError::Provider(_)) => {
StandaloneStartupError::ModelProvider
}
WorkerBootstrapError::Worker(WorkerError::SubjektivSessionAttribution {
message, ..
}) => StandaloneStartupError::Subject(message),
WorkerBootstrapError::Worker(_) => StandaloneStartupError::WorkerConfiguration,
WorkerBootstrapError::Controller { source, .. }
if source.kind() == std::io::ErrorKind::Other =>
@@ -544,3 +750,49 @@ fn classify_startup_error(error: WorkerBootstrapError) -> StandaloneStartupError
WorkerBootstrapError::Controller { .. } => StandaloneStartupError::Controller,
}
}
#[cfg(test)]
mod local_input_tests {
use super::*;
use protocol::{AuthenticatedInputSource, Method};
#[test]
fn local_transport_stamps_operator_without_backend_account() {
let source = AuthenticatedInputSource::Account {
account_id: "forged-account".into(),
};
for method in [
Method::submit_text("plain", "hello"),
Method::SubmitTracked {
submission_request_id: "tracked".into(),
input: vec![],
source: source.clone(),
},
Method::SubmitIfIdle {
submission_request_id: "idle".into(),
input: vec![],
source: source.clone(),
},
Method::Notify {
notification_request_id: "plain-notify".into(),
message: "hello".into(),
},
Method::NotifyTracked {
notification_request_id: "tracked-notify".into(),
message: "hello".into(),
source,
},
] {
let decoded: Method =
serde_json::from_str(&serde_json::to_string(&method).unwrap()).unwrap();
match authorize_local_input(decoded) {
Method::SubmitTracked { source, .. }
| Method::SubmitIfIdle { source, .. }
| Method::NotifyTracked { source, .. } => {
assert_eq!(source, AuthenticatedInputSource::LocalOperator)
}
other => panic!("unexpected local input {other:?}"),
}
}
}
}
File diff suppressed because it is too large Load Diff
+508
View File
@@ -0,0 +1,508 @@
//! Host-provided local Jobs. No Runtime registry, Workspace client or dialog-session state.
#[cfg(test)]
mod tests;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use agen::llm_client::LlmClient;
use job::{JobRequest, JobResultSubmission};
use tokio::sync::watch;
use crate::job_store::JobStore;
pub use crate::job_store::{JobAttempt, JobAttemptState, JobSnapshot, JobState, JobStoreError};
#[derive(Debug, thiserror::Error)]
pub enum JobServiceError {
#[error(transparent)]
Store(#[from] JobStoreError),
#[error("standalone Job execution requirements are unavailable: {0}")]
Configuration(String),
#[error("the standalone Job Host is shutting down")]
Closed,
#[error("standalone Job execution capacity is exhausted")]
Capacity,
#[error("standalone Job cleanup failed: {0}")]
Cleanup(String),
}
pub(crate) trait JobAttemptFence: Send + Sync {
fn ensure_live(&self) -> Result<(), String>;
}
pub(crate) trait JobDomainProvider: Send + Sync {
fn grant(
&self,
snapshot: &JobSnapshot,
fence: Arc<dyn JobAttemptFence>,
) -> Result<worker::job::JobFeatureGrant, String>;
}
struct Execution {
cancel: watch::Sender<bool>,
complete: watch::Receiver<Option<Result<(), String>>>,
attempt_id: String,
resource_key: Option<String>,
max_concurrent: u16,
task: tokio::task::JoinHandle<Result<(), JobServiceError>>,
}
struct Executions {
closed: bool,
jobs: HashMap<String, Execution>,
}
/// Cloneable Feature-facing access to a Host-owned service. Cloning this handle
/// neither extends the Host's authority nor permits opening/recovering its store.
#[derive(Clone)]
pub struct StandaloneJobs {
store: Arc<Mutex<Option<JobStore>>>,
executions: Arc<Mutex<Executions>>,
cwd: PathBuf,
capacity: u16,
domain: Arc<Mutex<Option<Arc<dyn JobDomainProvider>>>>,
model_client: Arc<Mutex<Option<Box<dyn LlmClient>>>>,
}
impl StandaloneJobs {
pub(crate) fn open(path: &Path, cwd: PathBuf) -> Result<Self, JobServiceError> {
Ok(Self {
store: Arc::new(Mutex::new(Some(JobStore::open(path)?))),
executions: Arc::new(Mutex::new(Executions {
closed: false,
jobs: HashMap::new(),
})),
cwd,
capacity: job::DEFAULT_MAX_CONCURRENT_JOBS,
domain: Arc::default(),
model_client: Arc::default(),
})
}
pub(crate) fn bind_model_client(&self, client: Box<dyn LlmClient>) {
*self
.model_client
.lock()
.expect("Job model transport poisoned") = Some(client);
}
pub(crate) fn bind_domain(
&self,
provider: Arc<dyn JobDomainProvider>,
) -> Result<(), JobServiceError> {
let mut domain = self.domain.lock().expect("Job domain poisoned");
if domain.is_some() {
return Err(JobServiceError::Configuration(
"Job domain already bound".into(),
));
}
*domain = Some(provider);
Ok(())
}
pub(crate) fn request_granted(
&self,
request: JobRequest,
grant: serde_json::Value,
) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
self.with_store(|store| store.reserve_granted(request, Some(grant)))
}
fn with_store<T>(
&self,
operation: impl FnOnce(&mut JobStore) -> Result<T, JobStoreError>,
) -> Result<T, JobServiceError> {
let mut slot = self.store.lock().expect("Job store poisoned");
Ok(operation(slot.as_mut().ok_or(JobServiceError::Closed)?)?)
}
/// Persist intent without starting a model. Exact resend returns the same
/// current attempt, including after acknowledgement or cancellation.
pub fn request(&self, request: JobRequest) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
Ok(self.with_store(|store| store.reserve(request))?)
}
pub fn get(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
Ok(self.with_store(|store| store.get(job_id))?)
}
/// Pending attempts survive shutdown. The consumer chooses when to resume
/// them; opening a Host never implicitly replays unknown side effects.
pub fn pending(&self) -> Result<Vec<JobSnapshot>, JobServiceError> {
Ok(self.with_store(|store| store.pending())?)
}
pub fn unacknowledged_results(&self) -> Result<Vec<JobSnapshot>, JobServiceError> {
self.with_store(|store| store.unacknowledged())
}
/// Explicit new attempt, bounded by immutable intent limits. Unknown
/// outcomes cannot be retried without domain reconciliation/new intent.
pub fn retry(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
if executions
.jobs
.get(job_id)
.is_some_and(|run| !run.task.is_finished())
{
return Err(JobServiceError::Capacity);
}
Ok(self.with_store(|store| store.retry(job_id))?)
}
/// Successful model processing and consumer postprocessing have independent
/// lifetimes. A failed delivery simply leaves this acknowledgement pending.
pub fn acknowledge(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
Ok(self.with_store(|store| store.ack(job_id))?)
}
pub fn cancel(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
let executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
let snapshot = self.with_store(|store| store.cancel(job_id))?;
if let Some(execution) = executions.jobs.get(job_id) {
execution.cancel.send_replace(true);
}
Ok(snapshot)
}
pub async fn start(&self, job_id: &str) -> Result<(), JobServiceError> {
self.start_with_optional_model_client(job_id, None).await
}
/// Injectable model transport for embedders and scripted execution tests.
/// The requested registry Profile still resolves/validates normally; the
/// transport grants no filesystem, Feature or result-submission authority.
pub async fn start_with_model_client<C>(
&self,
job_id: &str,
client: C,
) -> Result<(), JobServiceError>
where
C: LlmClient + 'static,
{
self.start_with_optional_model_client(job_id, Some(Box::new(client)))
.await
}
async fn start_with_optional_model_client(
&self,
job_id: &str,
client: Option<Box<dyn LlmClient>>,
) -> Result<(), JobServiceError> {
let client = client.or_else(|| {
self.model_client
.lock()
.expect("Job model transport poisoned")
.as_ref()
.map(|c| c.clone_boxed())
});
let snapshot = self.get(job_id)?;
let timeout = Duration::from_secs(u64::from(snapshot.request.limits.timeout_seconds));
let deadline = tokio::time::Instant::now() + timeout;
let fence: Arc<dyn JobAttemptFence> = Arc::new(BoundAttemptFence {
store: self.store.clone(),
job_id: job_id.into(),
attempt_id: snapshot.attempt.attempt_id.clone(),
input_revision: snapshot.request.input_revision.clone(),
deadline,
});
let grant = match (
&snapshot.domain_grant,
self.domain.lock().expect("Job domain poisoned").as_ref(),
) {
(Some(_), Some(provider)) => provider
.grant(&snapshot, fence)
.map_err(JobServiceError::Configuration)?,
(Some(_), None) => {
return Err(JobServiceError::Configuration(
"persisted Job requires an unavailable explicit domain capability".into(),
));
}
(None, _) => worker::job::JobFeatureGrant::default(),
};
let prepared =
worker::job::prepare_job_with_grant(&snapshot.request, &self.cwd, client, grant)
.map_err(|error| JobServiceError::Configuration(error.to_string()))?;
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions.closed {
return Err(JobServiceError::Closed);
}
executions.jobs.retain(|_, run| {
!(run.task.is_finished() && matches!(run.complete.borrow().as_ref(), Some(Ok(()))))
});
// Count live resources, not only result states: acceptance can precede
// the Internal Worker's cleanup and must not free a concurrency slot.
let live = || {
executions
.jobs
.values()
.filter(|run| !run.task.is_finished())
};
let count = live().count();
let active_limit = live()
.map(|run| run.max_concurrent)
.min()
.unwrap_or(self.capacity);
if count >= usize::from(self.capacity.min(active_limit))
|| count >= usize::from(snapshot.request.limits.max_concurrent_jobs)
|| executions
.jobs
.get(job_id)
.is_some_and(|run| !run.task.is_finished())
{
return Err(JobServiceError::Capacity);
}
let resource_key = snapshot.request.serialization_key.clone();
if let Some(key) = &resource_key {
if executions
.jobs
.values()
.any(|run| !run.task.is_finished() && run.resource_key.as_ref() == Some(key))
{
return Err(JobServiceError::Capacity);
}
}
// Retain errors from finished tasks until a caller observes them rather
// than silently swallowing store/cleanup failures.
if let Some(previous) = executions.jobs.get(job_id) {
if !previous.task.is_finished() {
return Err(JobServiceError::Capacity);
}
}
let snapshot = self.with_store(|store| {
store.dispatch(job_id, &snapshot.attempt.attempt_id, self.capacity)
})?;
let store = self.store.clone();
let id = job_id.to_string();
let attempt_id = snapshot.attempt.attempt_id.clone();
let sink = Arc::new(BoundResultSink {
store: store.clone(),
job_id: id.clone(),
attempt_id: attempt_id.clone(),
input_revision: snapshot.request.input_revision.clone(),
deadline,
});
let (cancel, mut cancellation) = watch::channel(false);
let (complete_tx, complete) = watch::channel(None);
let execution_attempt = attempt_id.clone();
let task = tokio::spawn(async move {
let outcome: Result<(), JobServiceError> = async {
let (sender_tx, mut sender_rx) = tokio::sync::oneshot::channel();
let run = prepared.run(attempt_id.clone(), sink, move |sender| { let _ = sender_tx.send(sender); });
tokio::pin!(run);
let reason = tokio::select! {
result = &mut run => {
let (state, detail) = match result {
Ok(_) => (JobAttemptState::Failed, "Worker finished without an accepted structured result".to_string()),
Err(error @ worker::job::JobExecutionError::ResultUnconfirmed(_)) => (JobAttemptState::Unknown, error.to_string()),
Err(error) => (JobAttemptState::Failed, error.to_string()),
};
finish_if_pending(&store, &id, &attempt_id, state, &detail)?;
return Ok(());
}
_ = tokio::time::sleep(timeout) => "timeout",
_ = cancellation.wait_for(|cancelled| *cancelled) => "Host cancellation/shutdown",
};
// Fence timeout/shutdown before cancelling so a late tool call can
// never turn an interrupted unaccepted execution into success.
let fence_result = finish_if_pending(&store, &id, &attempt_id, JobAttemptState::Unknown, reason);
// Cancellation is routed through Engine; keep the future alive to
// allow normal Feature cleanup. Hard stop is a fenced unknown
// outcome, never an automatic retry. Result acceptance wins races.
if let Ok(sender) = sender_rx.try_recv() {
let _ = sender.try_send(());
} else if let Ok(Ok(sender)) = tokio::time::timeout(Duration::from_secs(1), &mut sender_rx).await {
let _ = sender.try_send(());
}
let cleanup = tokio::time::timeout(Duration::from_secs(2), &mut run).await;
fence_result?;
if cleanup.is_err() { return Err(JobServiceError::Cleanup("Internal Worker cancellation deadline exceeded; outcome fenced unknown".into())); }
Ok(())
}.await;
complete_tx.send_replace(Some(
outcome.as_ref().map(|_| ()).map_err(ToString::to_string),
));
outcome
});
executions.jobs.insert(
job_id.to_string(),
Execution {
cancel,
complete,
task,
attempt_id: execution_attempt,
resource_key,
max_concurrent: snapshot.request.limits.max_concurrent_jobs,
},
);
Ok(())
}
/// Observe execution/cleanup, not success: read the durable result separately.
pub async fn wait(&self, job_id: &str) -> Result<JobSnapshot, JobServiceError> {
let execution = self
.executions
.lock()
.expect("Job executions poisoned")
.jobs
.get(job_id)
.map(|run| (run.attempt_id.clone(), run.complete.clone()));
if let Some((attempt, mut completion)) = execution {
let result = completion
.wait_for(Option::is_some)
.await
.map_err(|error| JobServiceError::Cleanup(error.to_string()))?
.clone()
.expect("completion");
result.map_err(JobServiceError::Cleanup)?;
let task = {
let mut executions = self.executions.lock().expect("Job executions poisoned");
if executions
.jobs
.get(job_id)
.is_some_and(|run| run.attempt_id == attempt)
{
executions.jobs.remove(job_id)
} else {
None
}
};
if let Some(execution) = task {
execution
.task
.await
.map_err(|error| JobServiceError::Cleanup(error.to_string()))??;
}
}
self.get(job_id)
}
pub(crate) fn close(&self) {
let mut executions = self.executions.lock().expect("Job executions poisoned");
executions.closed = true;
for execution in executions.jobs.values() {
execution.cancel.send_replace(true);
}
}
pub(crate) async fn shutdown(&self) -> Result<(), JobServiceError> {
self.close();
let tasks = {
let mut executions = self.executions.lock().expect("Job executions poisoned");
std::mem::take(&mut executions.jobs)
};
let mut error = None;
for execution in tasks.into_values() {
match execution.task.await {
Ok(Ok(())) => {}
Ok(Err(failure)) => {
error = Some(failure);
}
Err(failure) => {
error = Some(JobServiceError::Cleanup(failure.to_string()));
}
}
}
// Close SQLite before the Host releases its lease. Cloned handles may
// outlive the Host but cannot mutate or retain the live connection.
self.store.lock().expect("Job store poisoned").take();
self.domain.lock().expect("Job domain poisoned").take();
self.model_client
.lock()
.expect("Job model transport poisoned")
.take();
error.map_or(Ok(()), Err)
}
}
fn finish_if_pending(
store: &Mutex<Option<JobStore>>,
id: &str,
attempt: &str,
state: JobAttemptState,
detail: &str,
) -> Result<(), JobServiceError> {
let mut slot = store.lock().expect("Job store poisoned");
let store = slot.as_mut().ok_or(JobServiceError::Closed)?;
let snapshot = store.get(id)?;
if snapshot.state == JobState::Pending {
store.finish(id, attempt, state, Some(detail))?;
}
Ok(())
}
struct BoundResultSink {
store: Arc<Mutex<Option<JobStore>>>,
job_id: String,
attempt_id: String,
input_revision: String,
deadline: tokio::time::Instant,
}
impl worker::job::JobResultSink for BoundResultSink {
fn submit(&self, submission: JobResultSubmission) -> Result<(), String> {
if submission.job_id != self.job_id
|| submission.attempt_id != self.attempt_id
|| submission.input_revision != self.input_revision
{
return Err("Job result capability binding mismatch".into());
}
let mut slot = self
.store
.lock()
.map_err(|_| "Job store poisoned".to_string())?;
let store = slot.as_mut().ok_or_else(|| "Job Host closed".to_string())?;
let snapshot = store.get(&self.job_id).map_err(|error| error.to_string())?;
if tokio::time::Instant::now() >= self.deadline && snapshot.state != JobState::Completed {
return Err("Job result deadline expired".into());
}
store
.accept(&submission)
.map(|_| ())
.map_err(|error| error.to_string())
}
}
struct BoundAttemptFence {
store: Arc<Mutex<Option<JobStore>>>,
job_id: String,
attempt_id: String,
input_revision: String,
deadline: tokio::time::Instant,
}
impl JobAttemptFence for BoundAttemptFence {
fn ensure_live(&self) -> Result<(), String> {
if tokio::time::Instant::now() >= self.deadline {
return Err("Job domain deadline expired".into());
}
let slot = self.store.lock().map_err(|_| "Job store poisoned")?;
let snapshot = slot
.as_ref()
.ok_or("Job Host closed")?
.get(&self.job_id)
.map_err(|e| e.to_string())?;
if snapshot.state != JobState::Pending
|| snapshot.attempt.state != JobAttemptState::Dispatched
|| snapshot.attempt.attempt_id != self.attempt_id
|| snapshot.request.input_revision != self.input_revision
{
return Err("Job domain grant is not bound to the current live attempt".into());
}
Ok(())
}
}
+471
View File
@@ -0,0 +1,471 @@
use super::*;
use agen::llm_client::{ClientError, Request, event::Event};
use async_trait::async_trait;
use futures::Stream;
use serde_json::{Value, json};
use std::collections::VecDeque;
use std::pin::Pin;
use worker::job::JobResultSink;
fn request(id: &str) -> JobRequest {
JobRequest {
job_id: id.into(),
purpose: "test".into(),
input_revision: "r1".into(),
input_ref: "fixture:input".into(),
input: serde_json::json!({"data":42}),
instruction: "Return a structured result".into(),
profile: "builtin:job".into(),
serialization_key: None,
limits: job::JobLimits::default(),
}
}
fn sink(jobs: &StandaloneJobs, id: &str, attempt: &str) -> BoundResultSink {
BoundResultSink {
store: jobs.store.clone(),
job_id: id.into(),
attempt_id: attempt.into(),
input_revision: "r1".into(),
deadline: tokio::time::Instant::now() + Duration::from_secs(10),
}
}
fn result(id: &str, attempt: &str) -> JobResultSubmission {
JobResultSubmission {
job_id: id.into(),
attempt_id: attempt.into(),
input_revision: "r1".into(),
result: serde_json::json!({"valid":true}),
}
}
#[derive(Clone, Default)]
struct DomainTestClient {
scripts: Arc<Mutex<VecDeque<Vec<Event>>>>,
requests: Arc<Mutex<Vec<Request>>>,
entered: Arc<tokio::sync::Notify>,
stall: bool,
failure_release: Option<Arc<tokio::sync::Notify>>,
}
impl DomainTestClient {
fn success() -> Self {
Self {
scripts: Arc::new(Mutex::new(VecDeque::from([
vec![
Event::tool_use_start(0, "result-1", "SubmitJobResult"),
Event::tool_input_delta(0, r#"{"result":{"valid":true}}"#),
Event::tool_use_stop(0),
],
vec![
Event::text_block_start(0),
Event::text_delta(0, "done"),
Event::text_block_stop(0, None),
],
]))),
..Default::default()
}
}
}
#[async_trait]
impl LlmClient for DomainTestClient {
fn clone_boxed(&self) -> Box<dyn LlmClient> {
Box::new(self.clone())
}
async fn stream(
&self,
request: Request,
) -> Result<Pin<Box<dyn Stream<Item = Result<Event, ClientError>> + Send>>, ClientError> {
self.requests.lock().unwrap().push(request);
self.entered.notify_one();
if let Some(release) = &self.failure_release {
release.notified().await;
return Err(ClientError::Config(
"known scripted provider failure".into(),
));
}
if self.stall {
return Ok(Box::pin(futures::stream::pending()));
}
let events = self
.scripts
.lock()
.unwrap()
.pop_front()
.expect("unexpected model turn");
Ok(Box::pin(futures::stream::iter(events.into_iter().map(Ok))))
}
}
type CapturedDomainAttempt = (JobSnapshot, Arc<dyn JobAttemptFence>);
#[derive(Default)]
struct RecordingDomainProvider(Mutex<Vec<CapturedDomainAttempt>>);
fn domain_grant(id: &str) -> Value {
json!({"consumer":"test-domain", "job_id":id, "input_revision":"r1"})
}
fn domain_request(id: &str) -> JobRequest {
let mut intent = request(id);
intent.profile = "builtin:standalone-subjektiv-consolidation".into();
intent
}
impl JobDomainProvider for RecordingDomainProvider {
fn grant(
&self,
snapshot: &JobSnapshot,
fence: Arc<dyn JobAttemptFence>,
) -> Result<worker::job::JobFeatureGrant, String> {
if snapshot.domain_grant != Some(domain_grant(&snapshot.request.job_id)) {
return Err("foreign or mismatched immutable domain grant".into());
}
self.0.lock().unwrap().push((snapshot.clone(), fence));
Ok(worker::job::JobFeatureGrant {
satisfied_requirements: vec!["feature.subjektiv"],
..Default::default()
})
}
}
async fn await_model(client: &DomainTestClient) {
tokio::time::timeout(Duration::from_secs(2), client.entered.notified())
.await
.unwrap();
}
async fn await_job(jobs: &StandaloneJobs, id: &str) -> JobSnapshot {
tokio::time::timeout(Duration::from_secs(3), jobs.wait(id))
.await
.unwrap()
.unwrap()
}
#[tokio::test]
async fn pending_domain_grants_survive_service_reopen_and_remain_immutable() {
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("jobs.sqlite3");
let jobs = StandaloneJobs::open(&path, temp.path().to_path_buf()).unwrap();
let intent = domain_request("granted");
let grant = domain_grant("granted");
let reserved = jobs.request_granted(intent.clone(), grant.clone()).unwrap();
let root = jobs.request(request("root")).unwrap();
assert_eq!(reserved.domain_grant, Some(grant.clone()));
assert!(root.domain_grant.is_none());
assert_eq!(
jobs.request_granted(intent.clone(), grant.clone()).unwrap(),
reserved
);
assert!(matches!(
jobs.request(intent.clone()),
Err(JobServiceError::Store(JobStoreError::IntentConflict(_)))
));
assert!(matches!(
jobs.request_granted(intent.clone(), domain_grant("foreign")),
Err(JobServiceError::Store(JobStoreError::IntentConflict(_)))
));
assert!(matches!(
jobs.request_granted(request("root"), grant.clone()),
Err(JobServiceError::Store(JobStoreError::IntentConflict(_)))
));
assert_eq!(jobs.pending().unwrap().len(), 2);
jobs.shutdown().await.unwrap();
let reopened = StandaloneJobs::open(&path, temp.path().to_path_buf()).unwrap();
let pending = reopened.pending().unwrap();
assert_eq!(pending.len(), 2);
assert!(pending.contains(&reserved));
assert!(pending.contains(&root));
assert_eq!(reopened.get("granted").unwrap(), reserved);
assert_eq!(
reopened
.request_granted(intent.clone(), grant.clone())
.unwrap(),
reserved
);
assert!(matches!(
reopened.request_granted(intent, json!({"different":true})),
Err(JobServiceError::Store(JobStoreError::IntentConflict(_)))
));
let provider = Arc::new(RecordingDomainProvider::default());
reopened.bind_domain(provider.clone()).unwrap();
reopened
.start_with_model_client("granted", DomainTestClient::success())
.await
.unwrap();
let completed = await_job(&reopened, "granted").await;
assert_eq!(completed.state, JobState::Completed);
assert_eq!(completed.domain_grant, Some(grant));
assert_eq!(completed.result, Some(json!({"valid":true})));
let captured = provider.0.lock().unwrap()[0].0.clone();
assert_eq!(
captured, reserved,
"provider receives the persisted immutable intent and grant"
);
reopened.shutdown().await.unwrap();
}
#[tokio::test]
async fn persisted_grant_requires_provider_and_missing_grant_cannot_be_derived_from_profile() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let granted = jobs
.request_granted(domain_request("granted"), domain_grant("granted"))
.unwrap();
let client = DomainTestClient::success();
let error = jobs
.start_with_model_client("granted", client.clone())
.await
.unwrap_err();
assert!(matches!(error, JobServiceError::Configuration(_)));
assert!(
error
.to_string()
.contains("unavailable explicit domain capability"),
"{error}"
);
assert_eq!(jobs.get("granted").unwrap(), granted);
assert!(client.requests.lock().unwrap().is_empty());
let provider = Arc::new(RecordingDomainProvider::default());
jobs.bind_domain(provider.clone()).unwrap();
assert!(matches!(
jobs.bind_domain(provider.clone()),
Err(JobServiceError::Configuration(_))
));
let ungranted = jobs.request(domain_request("ungranted")).unwrap();
let error = jobs
.start_with_model_client("ungranted", client.clone())
.await
.unwrap_err();
assert!(matches!(error, JobServiceError::Configuration(_)));
assert!(error.to_string().contains("feature.subjektiv"), "{error}");
assert_eq!(jobs.get("ungranted").unwrap(), ungranted);
assert!(
provider.0.lock().unwrap().is_empty(),
"bound provider is not a profile-derived grant"
);
assert!(client.requests.lock().unwrap().is_empty());
jobs.shutdown().await.unwrap();
}
#[tokio::test]
async fn foreign_job_revision_and_consumer_grants_are_refused_before_model_dispatch() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let provider = Arc::new(RecordingDomainProvider::default());
jobs.bind_domain(provider.clone()).unwrap();
for (id, grant) in [
("foreign-job", domain_grant("another-job")),
(
"foreign-revision",
json!({"consumer":"test-domain","job_id":"foreign-revision","input_revision":"r0"}),
),
(
"foreign-consumer",
json!({"consumer":"another-domain","job_id":"foreign-consumer","input_revision":"r1"}),
),
] {
let reserved = jobs.request_granted(domain_request(id), grant).unwrap();
let client = DomainTestClient::success();
let error = jobs
.start_with_model_client(id, client.clone())
.await
.unwrap_err();
assert!(matches!(error, JobServiceError::Configuration(_)));
assert!(
error.to_string().contains("foreign or mismatched"),
"{error}"
);
assert_eq!(jobs.get(id).unwrap(), reserved);
assert!(client.requests.lock().unwrap().is_empty());
}
assert!(provider.0.lock().unwrap().is_empty());
jobs.shutdown().await.unwrap();
}
#[tokio::test]
async fn domain_fence_is_live_only_for_current_dispatched_attempt_and_revoked_on_cancel() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let provider = Arc::new(RecordingDomainProvider::default());
jobs.bind_domain(provider.clone()).unwrap();
let reserved = jobs
.request_granted(domain_request("fenced"), domain_grant("fenced"))
.unwrap();
let failure_release = Arc::new(tokio::sync::Notify::new());
let first = DomainTestClient {
failure_release: Some(failure_release.clone()),
..Default::default()
};
jobs.start_with_model_client("fenced", first.clone())
.await
.unwrap();
await_model(&first).await;
let old_fence = provider.0.lock().unwrap()[0].1.clone();
old_fence.ensure_live().unwrap();
failure_release.notify_one();
let failed = await_job(&jobs, "fenced").await;
assert_eq!(failed.state, JobState::Failed);
assert!(failed.result.is_none());
assert!(old_fence.ensure_live().is_err());
// Only known failure permits a retry; cancellation remains terminal.
let retried = jobs.retry("fenced").unwrap();
assert_ne!(retried.attempt.attempt_id, reserved.attempt.attempt_id);
assert_eq!(retried.domain_grant, reserved.domain_grant);
assert!(old_fence.ensure_live().is_err());
let second = DomainTestClient {
stall: true,
..Default::default()
};
jobs.start_with_model_client("fenced", second.clone())
.await
.unwrap();
await_model(&second).await;
let current_fence = provider.0.lock().unwrap()[1].1.clone();
current_fence.ensure_live().unwrap();
assert!(
old_fence.ensure_live().is_err(),
"a new attempt must not revive a stale capability"
);
let cancelled = jobs.cancel("fenced").unwrap();
assert_eq!(cancelled.state, JobState::Cancelled);
assert!(
current_fence.ensure_live().is_err(),
"durable cancellation revokes current domain operations immediately"
);
assert!(old_fence.ensure_live().is_err());
let cleaned = await_job(&jobs, "fenced").await;
assert_eq!(cleaned.state, JobState::Cancelled);
assert!(cleaned.result.is_none());
assert!(
jobs.retry("fenced").is_err(),
"cancelled intent is not a known retryable failure"
);
jobs.shutdown().await.unwrap();
assert!(old_fence.ensure_live().is_err());
assert!(current_fence.ensure_live().is_err());
}
#[tokio::test]
async fn bound_domain_does_not_change_root_generic_result_only_execution() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let provider = Arc::new(RecordingDomainProvider::default());
jobs.bind_domain(provider.clone()).unwrap();
let root = jobs.request(request("root")).unwrap();
assert!(root.domain_grant.is_none());
let client = DomainTestClient::success();
jobs.start_with_model_client("root", client.clone())
.await
.unwrap();
let completed = await_job(&jobs, "root").await;
assert_eq!(completed.state, JobState::Completed);
assert_eq!(completed.result, Some(json!({"valid":true})));
assert!(completed.domain_grant.is_none());
assert!(provider.0.lock().unwrap().is_empty());
let requests = client.requests.lock().unwrap();
assert_eq!(requests.len(), 2);
for sent in requests.iter() {
assert_eq!(
sent.tools
.iter()
.map(|tool| tool.name.as_str())
.collect::<Vec<_>>(),
["SubmitJobResult"]
);
assert_eq!(sent.config.max_tokens, Some(worker::job::MAX_JOB_TOKENS));
}
drop(requests);
jobs.shutdown().await.unwrap();
}
#[tokio::test]
async fn host_result_capability_cannot_cross_job_attempt_revision_or_deadline() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let first = jobs.request(request("first")).unwrap();
let second = jobs.request(request("second")).unwrap();
for snapshot in [&first, &second] {
jobs.with_store(|store| {
store.dispatch(&snapshot.request.job_id, &snapshot.attempt.attempt_id, 8)
})
.unwrap();
}
let bound = sink(&jobs, "first", &first.attempt.attempt_id);
assert!(
bound
.submit(result("second", &second.attempt.attempt_id))
.is_err()
);
assert!(bound.submit(result("first", "first:attempt:2")).is_err());
let mut stale = result("first", &first.attempt.attempt_id);
stale.input_revision = "r0".into();
assert!(bound.submit(stale).is_err());
let mut expired = sink(&jobs, "first", &first.attempt.attempt_id);
expired.deadline = tokio::time::Instant::now() - Duration::from_secs(1);
assert!(
expired
.submit(result("first", &first.attempt.attempt_id))
.is_err()
);
assert!(jobs.get("first").unwrap().result.is_none());
bound
.submit(result("first", &first.attempt.attempt_id))
.unwrap();
// Accepted exact replay is still allowed after the deadline, never a new result.
expired
.submit(result("first", &first.attempt.attempt_id))
.unwrap();
let mut changed = result("first", &first.attempt.attempt_id);
changed.result = serde_json::json!({"valid":false});
assert!(expired.submit(changed).is_err());
assert_eq!(jobs.get("second").unwrap().state, JobState::Pending);
jobs.shutdown().await.unwrap();
assert!(
bound
.submit(result("first", &first.attempt.attempt_id))
.is_err()
);
}
#[tokio::test]
async fn cleanup_cannot_reclassify_success_or_durable_cancellation_and_stale_handles_are_closed() {
let temp = tempfile::tempdir().unwrap();
let jobs =
StandaloneJobs::open(&temp.path().join("jobs.sqlite3"), temp.path().to_path_buf()).unwrap();
let cancelled = jobs.request(request("cancelled")).unwrap();
jobs.cancel("cancelled").unwrap();
finish_if_pending(
&jobs.store,
"cancelled",
&cancelled.attempt.attempt_id,
JobAttemptState::Unknown,
"cleanup",
)
.unwrap();
assert_eq!(jobs.get("cancelled").unwrap().state, JobState::Cancelled);
let accepted = jobs.request(request("accepted")).unwrap();
jobs.with_store(|store| store.dispatch("accepted", &accepted.attempt.attempt_id, 8))
.unwrap();
sink(&jobs, "accepted", &accepted.attempt.attempt_id)
.submit(result("accepted", &accepted.attempt.attempt_id))
.unwrap();
finish_if_pending(
&jobs.store,
"accepted",
&accepted.attempt.attempt_id,
JobAttemptState::Failed,
"model failed later",
)
.unwrap();
assert_eq!(jobs.get("accepted").unwrap().state, JobState::Completed);
jobs.shutdown().await.unwrap();
assert!(jobs.store.lock().unwrap().is_none());
for operation in [
jobs.cancel("cancelled"),
jobs.acknowledge("accepted"),
jobs.retry("cancelled"),
jobs.request(request("new")),
] {
assert!(matches!(operation, Err(JobServiceError::Closed)));
}
}
+32
View File
@@ -15,6 +15,7 @@ pub struct StandaloneLaunchConfig {
pub state_dir: PathBuf,
pub profile: ProfileSelector,
pub worker_name: String,
pub subject_id: Option<String>,
}
pub struct ResolvedStandaloneLaunch {
@@ -22,6 +23,7 @@ pub struct ResolvedStandaloneLaunch {
pub state_dir: PathBuf,
pub profile: ResolvedProfile,
pub prompt_catalog: PromptCatalogSource,
pub subject_id: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
@@ -32,6 +34,10 @@ pub enum StandaloneLaunchError {
PathProfileUnsupported,
#[error("the standalone profile could not be resolved")]
ProfileResolutionFailed,
#[error(
"local Subject selection requires an enabled, non-consolidator subjektiv Profile and canonical Subject id"
)]
InvalidSubjectPolicy,
}
impl StandaloneLaunchConfig {
@@ -46,9 +52,15 @@ impl StandaloneLaunchConfig {
state_dir: state_dir.into(),
profile,
worker_name: worker_name.into(),
subject_id: None,
}
}
pub fn with_subject(mut self, subject_id: String) -> Self {
self.subject_id = Some(subject_id);
self
}
/// Resolve only built-in/XDG profile authority and bind standalone scope
/// to the canonical process cwd. Repository-local profile discovery is
/// deliberately not part of this path.
@@ -74,12 +86,32 @@ impl StandaloneLaunchConfig {
.map_err(|_| StandaloneLaunchError::ProfileResolutionFailed)?;
validate_profile_execution_target(&profile.manifest, ProfileExecutionTarget::Standalone)
.map_err(|_| StandaloneLaunchError::ProfileResolutionFailed)?;
if let Some(id) = &self.subject_id {
if !crate::subjektiv::valid_subject_id(id)
|| !profile.manifest.feature.subjektiv.profile.enabled
|| profile
.manifest
.feature
.subjektiv
.profile
.consolidation_tools
|| profile
.manifest
.feature
.subjektiv
.workspace_settings
.is_some()
{
return Err(StandaloneLaunchError::InvalidSubjectPolicy);
}
}
Ok(ResolvedStandaloneLaunch {
cwd,
state_dir: self.state_dir,
profile,
prompt_catalog: PromptCatalogSource::builtins_only(),
subject_id: self.subject_id,
})
}
}

Some files were not shown because too many files have changed in this diff Show More