Hare/yoi
1
1
Fork 0
Code Actions Packages Releases Activity

feat: integrate bounded checkout discovery

Browse Source 
Merge approved T-714 source b8faa83c into develop c79506b6. Preserve the target's already-integrated public Tree/Inspect/Invoke configuration fixture migration in the overlapping test helper; runtime implementation merged without conflicts. Root check, complete FS/Workdir/Tools/Worker/Server/API suites, HTTP Workdir suite, WIP 0.2.0 resolution, formatting and diff checks passed.
This commit is contained in:
Hare
2026-10-09 15:45:41 +09:00
parent c79506b60f b8faa83c47
commit 3a2994cdd7
31 changed files with 2655 additions and 221 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/checkout-wip-projection.md
+93 -21
View File
@@ -14,9 +14,11 @@ is created.
├── workdirs identity/lifecycle
├── workdir-attachments Backend connection lifetimes
└── checkouts current accessible content entrances; list
└── <encoded alias> directory; glob, grep, create_file
├── README.md file; read, edit, write
└── src directory; glob, grep, create_file
└── <encoded alias> directory; list, glob, grep, create_file
# Not indexable; resolve known paths directly or discover via Operations:
/checkouts/<encoded alias>/README.md file; read, edit, write
/checkouts/<encoded alias>/src directory; list, glob, grep, create_file
```
Operations are Interface declarations, not children or `/features/...` routes.
@@ -42,7 +44,7 @@ validates every resolved projection's path/owner/descriptor and every enumerated
child's immediate-parent relationship. Static and dynamic/subtree overlaps are
rejected. Observe resolves only the requested depth; successful observations are
complete through that depth, not partial trees masked as success. Directory
listing and native search use the shared List/Glob/Grep engines through typed
listing Operations and native search use the shared List/Glob/Grep engines through typed
`CheckoutSearchRequest` forwarding. Scoped sessions append intersecting rule
layers, translate their cwd, and require provider-side visibility checks before
entry descent/content reads. Results retain session-logical coordinates, so a
@@ -51,6 +53,21 @@ providers fail closed rather than falling back to a broader source search.
Direct resolution also uses current provider authority, not possession of a
cached Interface.
## Indexable boundary (T-714)
Only `/checkouts` and accessible immediate attachment entrances are indexable.
No directory content List runs during tree/observe, even at deep depth or when a
known deep directory is the observation root. Empty, small and large directories
all have the same boundary. Positive-depth content observations return
`children: []` (no **indexable** children), not an omitted field and not
an assertion that the directory is empty. At depth zero children are omitted;
`observe("/checkouts", 1)` omits entrance children, depth 2 returns [] there.
The Client retains discovered/inspected paths without adding indexable edges.
Known authorized paths remain directly resolvable and operable. Use List/Glob/Grep
for discovery, then Inspect only selected entry coordinates; no successful search
is undone when an entry disappears before Inspect. Bash/COMMAND is unnecessary.
## Names and connection isolation
Aliases use T-697's canonical encoding: nonempty ASCII letters/digits/`-`/`_` are
@@ -66,18 +83,27 @@ existence alone does not expose its content. Following a reference still require
current scope/capability and an accessible filesystem projection.
Objects have no shared inode `ref`: hard-linked names and different attachments
must not be mistaken for one path-independent representation. Interface references
are opaque, path-qualified and include a fresh projection incarnation and live
attachment generation. Validators combine that namespace with the provider's
must not be mistaken for one path-independent representation. Interface
references are structured `(scope, name)`. Each entrance is its own
scope; content Interfaces use that entrance as ancestor scope, irrespective of
indexability. Local names bind target coordinates, projection incarnation and
live attachment generation. The provider's target-qualified-name routing hint
only selects a target; fetch re-resolves one coherent current publication and
checks the exact Interface, Descriptor, validator and scope under the pinned
router connection. It never turns a cached registration or name into authority.
Validators combine that namespace with the provider's
opaque identity/state validator. Restoring a Worker creates a fresh incarnation;
detach/alias reuse changes the router generation. An old Interface, validator or
read-history entry cannot retarget a new connection. No separate connection ledger
is added. The existing router's active-operation exclusion and begin/finish/cancel
detach paths remain authoritative; detach does not remove the Workdir/files.
Collection enumeration rejects connection-set changes between sampling and
publication instead of pairing old items with a new validator. Search-link
publication checks the captured attachment generation, not merely a reused
alias; response-side provider awaits are covered by the operation deadline.
publication instead of pairing old items with a new validator. Result publication
checks the captured attachment generation, not merely a
reused alias. It maps typed provider paths without observing/statting every
result. Provider awaits are covered by the operation deadline. Objects publish
no scope_ref here; legitimate ref-less binding still checks membership,
validators, current provider authority and attachment generation.
## AI Operation mapping
@@ -89,15 +115,20 @@ below are relative to the **bound directory**, not the Worldspace or endpoint.
| Object | Operation | Arguments | Result |
| --- | --- | --- | --- |
| `/checkouts` | `list` | none | bounded `items` containing alias/slug/Workdir/generation/path |
| directory | `glob` | `pattern`, optional `path` | normal bounded Glob text/summary plus same-checkout Object links |
| directory | `grep` | `pattern`; optional `path`, `glob`, `type`, `case_insensitive`, `-A`, `-B`, `-C`, `multiline`, `output_mode`, `head_limit`, `offset` | normal Grep grouped/context/count text and typed provider search targets mapped to Object links |
| directory | `list` | optional `limit` (default 100, 1..1000), `after` (returned `{kind, entry}`) | direct-content records `{kind, size, entry}`, `truncated`, optional `after` |
| directory | `glob` | `pattern`, optional `path` | normal bounded Glob text/summary plus Descriptor-typed same-checkout `entry` coordinates |
| directory | `grep` | `pattern`; optional `path`, `glob`, `type`, `case_insensitive`, `-A`, `-B`, `-C`, `multiline`, `output_mode`, `head_limit`, `offset` | normal Grep grouped/context/count text and typed provider search targets mapped to Descriptor-typed `entry` coordinates |
| file | `read` | optional line `offset`, `limit` | line-numbered text and normal Read summary |
| file | `edit` | `old_string`, `new_string`, optional `replace_all` | replacement count/summary and preview |
| file | `write` | `content` | existing-file save summary; never creation |
| directory | `create_file` | `path`, `content` | create-new summary and created-file link; permitted missing parents are created |
Native operation results use a JSON record with `summary`, nullable `content` and
`items` (`path` is an absolute WIP Object navigation coordinate, not an OS path).
Native operation results declare a typed record with `summary`, optional `content`
and `items`. Each item contains `entry: entry` in its Descriptor (an absolute
Worldspace coordinate, encoded as a string in WIP Value), never a guessed ordinary
string or an OS path. List additionally declares `kind`, byte `size`, `truncated`
and optional `after: {kind: string, entry: entry}`. File operations have empty
items except Create, which returns the new coordinate.
Search links are constructed from **typed provider paths**, not by parsing Grep's
rendered filenames/line delimiters. Provider scope remains authoritative for
search output as well as links. Glob/Grep share normal provider ignore/pattern,
@@ -129,12 +160,47 @@ must not record history; successful mutations update hash/change statistics only
after provider success. Create is strictly beneath the bound parent, creates only
an absent file, and must not overwrite an existing destination or symlink.
Manifest permission identities remain Read/Edit/Write/Glob/Grep; create_file uses
Write. The Host reconstructs the same route-bound input for policy evaluation;
Manifest permission identities are Read/Edit/Write/Glob/Grep plus **List** for
native directory listing; create_file uses Write. List publication uses provider
Read capability and potential List allow policy; execution reconstructs the bound
directory's logical path for the exact List policy. Allowing Glob or COMMAND does
not override a List deny/ask rule, and Interface scope validity grants no file
access. Normal Tools mode is unchanged (no new top-level List tool). The Host reconstructs the same route-bound input for policy evaluation;
provider read/write scope, capabilities and delegated leases are checked again.
`ask` fails closed. Native replacement claims remove `/tools/Read`, `/tools/Edit`,
`/tools/Write`, `/tools/Glob`, `/tools/Grep`, not unrelated Tools.
## Directory List pagination
List uses the existing provider List engine, **not** glob("*") or shell ls. It
includes direct visible empty directories, hidden and ignored names: unlike
Glob/Grep, provider List does not apply search ignore files. Existing Read scope,
readability and can_enumerate_directory checks still gate names; no ancestor or
nonrecursive enumeration permission is inferred from a known descendant.
Order is directories first, then lexical canonical provider path, with every
other kind in the second group. after is the last returned exclusive ordering
key, rebased to a canonical Worldspace entry by the Host. Pass the returned record
unchanged to continue on the same directory; a foreign checkout, non-direct child,
absolute provider path or unknown field is rejected. No cursor stat is required:
the key may have disappeared. This is **live pagination, not a snapshot**. New
keys before the cursor are not revisited; later keys may appear, removals vanish,
and renames/kind changes may omit or repeat an entry. Directory validators fence
each call, not all pages; refresh Inspect after a stale rejection before issuing
a new explicit call. Detach/reuse expires the old Interface and validator.
Each page scans the live directory under provider cancellation/deadline and the
existing traversal-entry ceiling, retaining at most limit+1 ordering candidates
and the aggregate path-byte budget. It never retains the whole unbounded listing.
The next ordering key is explicit whenever truncated; no next key means complete.
Provider resource failures stay explicit failures, not silent truncation or retries.
UTF-8 canonical names are retained verbatim; unrepresentable names are skipped by
the existing provider. Native List may reject visible symlinks under the checked
provider's no-follow policy. If a provider returns symlink/other kinds they remain
classified path coordinates, not a promise of inspectable/operable Objects:
direct checkout resolution only publishes authorized regular files/directories.
## Consistency, safety and resource policy
The provider resolves/pins the entry/parent, compares its opaque validator and
@@ -172,7 +238,7 @@ rediscover/reread. Do not report partially created parents as unchanged.
Host observations are bounded to depth 32 and 1024 nodes with a 30-second total
cooperative deadline. The checkout adapter permits at most 16 concurrent requests
and bounds admission waiting and the complete operation (including result-link
and bounds admission waiting and the complete operation (including bounded result
publication) to 30 seconds each; provider path/depth, source/response,
mutation/search and transport bounds apply too. No initial context expands an
entire tree. Resource-limit errors do not turn incomplete observation into a
@@ -182,8 +248,9 @@ truncation under the existing search contract.
## Typical sequence
1. Use Tree on `/checkouts` with depth 1; Inspect `/checkouts` directly and Invoke `list`.
2. Inspect an entrance or a known deep directory directly; Invoke `glob`/`grep`.
3. Follow a returned file `path`, Inspect that path and Invoke line `read`.
2. Inspect an entrance or known deep directory directly; Invoke `list`/`glob`/`grep`.
For List, pass returned `after` with the next limit to obtain another live page.
3. Follow a returned file `entry`, Inspect that path and Invoke line `read`.
4. Call `edit` or `write`; Client-managed validators protect the exact observed
object while the shared tracker enforces the prior Read.
5. Rediscover after external changes; a stale rejection is not permission to skip
@@ -203,9 +270,14 @@ outside this feature.
Run from the main repository, on Linux with `openat2` and `/proc/self/fd`:
```sh
cargo check --workspace --all-targets
cargo test -p worker --lib checkout_wip
cargo test -p workdir --test list_pagination
cargo check
cargo test -p fs-operation -p workdir -p tools -p worker
cargo test -p worker-runtime -p yoi-workspace-server
# Workdir External/HTTP features and dependent server fixture:
cargo test -p workdir --features http-client
cargo test -p yoi-workspace-server --lib backend_workdir_session_proxy_executes_typed_operations
node scripts/verify-wip-dependencies.mjs
cargo fmt --all -- --check
git diff --check
```
docs/development/wip-integration-validation.md
+92 -4
View File
@@ -115,10 +115,98 @@ permission identity and ToolOutput/attachments.
- `WipMountRegistry::mount_interface` adds a separate static Interface namespace
without replacing Object ownership or flattening same-named operations. It does
not overlay stale registration metadata onto a live subtree provider.
- FS indexable boundaries and directory.list remain provider policy. This change
deliberately does not implement the separate T-714 boundary/list migration.
Relevant transport/commit tests are `checkout_wip_tests.rs`,
`checkout_http_tests.rs`, and `checkout_race_tests.rs`.
- FS indexable boundaries and directory.list are provider policy. T-714 now
implements entrance-only trees and bounded typed discovery (details below).
Transport/commit tests are `checkout_wip_tests.rs`, `checkout_http_tests.rs`,
and `checkout_race_tests.rs`.
- Worldspace state is runtime-only. Historical opaque references in append-only
session/tool history are not migrated into cache or parsed for a guessed scope;
a restored Client starts with fresh observations and direct path Inspect.
## T-714 checkout discovery validation
Based on integrated T-713 at `f9d4c97f`, still using the exact released crates.io
0.2.0 SDK and official renderer. No alternate wire, renderer or upstream version
was introduced. The checkout projection design documents provider live pagination,
its ordering/cursor/byte ceilings, special-entry policy and the distinction between
indexable emptiness and empty filesystem directories.
Focused regressions:
```sh
cargo test -p worker --lib checkout_wip
cargo test -p worker --lib checkout::race_tests
cargo test -p worker --lib checkout_http_tests
cargo test -p worker --lib ancestor_tests
cargo test -p tools --lib checkout_list
cargo test -p fs-operation --test list_pagination
cargo test -p workdir --test list_pagination
```
Host depth-zero omission / positive-depth `children: []`, empty/small/1100-entry
folders, direct deep roots and zero provider content-List calls during observation
are tested independently of Client output. Registered model tools exercise entrance
self-scope and deep target ancestor-scope, paginated List/Glob/Grep typed entries,
lazy selected Inspect with official signatures/path context, and Read without
COMMAND. Acquired paths never become tree edges; a stalled result observation is
never awaited and disappearance only fails a later explicit Inspect. Remote HTTP
checks verify List continuation on the wire with no ordinary List fallback or
response credentials/host-path leakage. Provider tests cover scoped External and
nested cwd/output_root rebasing, nonrecursive enumeration/read authority, ignore
policy, live mutations, byte bounds, cancellation and malicious cursor/result DTOs.
`WipSubtreeProvider::interface_target` is a routing hint for target-qualified local
names under ancestor scopes. The default retains self/shared-scope behavior. Host
checks provider ownership and resolves **one** coherent current target/scope
publication; it never treats the hint as authorization or fetches a scope in an
independent mutable lookup. Focused Host tests reject forged/outside/prefix/missing
hints, ref-less deletion and absent membership while preserving existing ancestor
ref/validator and in-flight snapshots. Actual checkout tests reject sibling/prefix
scopes, supplied refs when current publication has none and stale alias generations;
ordinary content updates do not terminate the entrance Interface lifetime.
Completion validation:
- Root `cargo check` passed.
- `cargo test -p fs-operation -p workdir -p tools -p worker` passed: FS **58 unit +
6 pagination**; Workdir **125 unit + 5 pagination**; Tools **84 unit + 39
integration + 1 doctest**; Worker **901 unit + 110 integration**.
- `cargo test -p workdir --features http-client` passed (**125 + 5**).
- `cargo test -p yoi-workspace-server` passed (**691 unit + 11 CLI**).
- `cargo test -p server-api` passed (**83**, including checked-in OpenAPI/TypeScript
freshness). No generated API artifact changes were needed.
- `node scripts/verify-wip-dependencies.mjs` confirmed one crates.io 0.2.0 Protocol
source and all four exact SDK packages. `cargo fmt --all -- --check` and
`git diff --check HEAD` passed.
The first dependent Server test build exposed predecessor fixtures still using
removed `discover`/opaque strings/private `call`. Those tests now select structured
references from actual path Inspect and use public Tree/Inspect/Invoke; the complete
Server suite above includes configuration integration and the typed Workdir proxy.
Its dev-only Protocol dependency reuses the same locked SDK source; the resolved
package/version set is unchanged.
No full workspace test, full Nix/image build, deployment, target integration or
live dogfood update is claimed. Target movement remains Orchestrator authority.
### Post-success connection-loss review regression
The first independent source review identified `POST_COMMIT_CONNECTION_FENCE`:
lazy result publication had propagated a connection mismatch as a pre-effect
rejection even after successful Write/Edit/Create. The production result
publication boundary now maps post-success connection loss for mutations to
`OutcomeUnknown`, while Read/List/Glob/Grep keep their validator mismatch. Lazy
entry mapping is unchanged; there is still no result-wide acquisition.
`committed_checkout_mutation_losing_connection_is_unknown_not_rejected_or_replayed`
executes real checked Tools/provider mutations, deterministically detaches after
the routed execution guard finishes but before the same production publication
boundary, then passes the error through WIP Invoke/Client. It checks committed
file contents, unknown audit (not rejection), and exactly one dispatch for each
Write/Edit/Create. `readonly_checkout_result_losing_connection_keeps_validator_mismatch`
protects the non-mutating error policy. No sleep or production test hook is used.
After this fix, all **9** checkout race tests and the complete Worker suite
(**903 unit + 110 integration**) passed, as did root Cargo check, formatting,
diff checks and released SDK resolution verification. The earlier **901** Worker
count above describes the pre-review combined-crate run.
docs/report/t-714-ticket-read-auth-blocker.md
+29
View File
@@ -0,0 +1,29 @@
# T-714: Ticket authority read rejected after implementation validation
## Observed boundary
On the continuation turn, `ShowTicket({ id: "T-714", event_limit: 5 })`
returned `Workspace Ticket API request failed with HTTP status 401`.
The same assigned Coder session had successfully read the Ticket and recorded
one shared-seam coordination decision earlier. This report does not diagnose
whether the rejection is authentication expiry, capability invalidation, or
another Backend condition.
## Consequence
The local checkout implementation and validation were complete. Git retained the
assigned `work/t-714-checkout-discovery` branch and all changes; coherent local
commits were authorized by the user. Publication/MR creation and review were
paused because a fresh authoritative Ticket/MR read could not be obtained.
No alternate actor, CLI, Backend storage access or replacement control-plane
path was attempted. No target integration or live dogfood update occurred.
## Improvement / recovery
Provide an actionable typed error distinction between authentication failure and
capability rejection, and a host-owned way to restore valid transport authority
without changing actor identity or bypassing the rejected capability. After that,
reread the Ticket and any linked MR, normally publish only its original source
selector, verify exact provider HEAD, and request an independently bound review.
The local implementation/validation evidence is in commits and
`docs/development/wip-integration-validation.md`; no approval is claimed here.