docs: consolidate design documentation and standardize crate English

This commit is contained in:
2026-10-12 14:58:00 +09:00
parent a633a53274
commit 4efb0ace1f
404 changed files with 4556 additions and 11654 deletions
-199
View File
@@ -1,199 +0,0 @@
# Request-time checkout provider boundary
`workdir::checkout` (also crate-root reexports) exposes `CheckoutObservation`,
`CheckoutRequest`, `CheckoutOperation`, `CheckoutOutput`, and `CheckoutResult`.
`WorkdirSession::checkout_observe(path)` defaults to unsupported; execution also
fails closed by default. Observation is metadata-only, with no recursive tree
walk. Directory discovery/search remains the ordinary bounded List/Glob/Grep
provider engine. GrepResult.paths carries exact typed provider paths appearing
in the returned report, respecting its offset/limit/truncation; filenames are
never parsed from rendered text. List/Glob/Grep skip unrepresentable non-UTF8
paths, preserving valid Unicode exactly without normalization. Native callers supply their own attachment generation/namespace
fence; this provider validator is not a substitute for that fence.
* Read: line `offset`, `limit`, `max_bytes`; same `ReadResult` (bytes, start line,
total lines, full-source content hash, truncation) as normal Tools. Formatting
line numbers and recording the returned hash in caller read-history are caller
responsibilities. The validator is **not** permission to bypass read-before-write.
* Write: replace existing file with `content` and required `expected_hash`.
* Edit: existing UTF-8 file, unique `old_string` unless `replace_all`, required
`expected_hash`; same match/replacement/result-size checks as Tools.
* Create: target must be an observed directory; `path` is **checkout-root
relative**, strictly below target, not target-relative. No existing file can
be overwritten. Intermediate directories and destination must be writable.
* Every result observation describes the bound request target. Create returns
the updated **parent directory** observation from the retained pinned target,
not the created destination. Read returns the read state's observation.
## Shared implementation
`fs_operation::CheckedTarget` is an `FsAccessPolicy` adapter around pinned target,
save-parent, and provider-root descriptors. It calls the ordinary
`run_read_bounded`, `run_edit`, and `run_write` functions, including full-source
hashing, UTF-8/string matching, bounds, and atomic temp-save primitive; there is
no second content-operation engine. Shared text reads now check descriptor
metadata before/after reading. Validator bytes encode device/inode/size/mode/
mtime/ctime with nanoseconds, excluding atime and host paths; same-content inode
replacement is distinguishable. Successful native Read rechecks the named target
and pinned descriptor before returning hash and observation.
Local sessions serialize checked observe/execute and ordinary trait Read/Write/
Edit using one nonqueueing session lock. Expected validator comparison happens
inside that boundary. The adapter retains handles across reading and save,
rechecks named target + save-parent identity at save boundary, and writes a new
temporary inode then atomically renames it, isolating existing hardlink aliases.
External sessions retain the approved descriptor root after pathname rename or
replacement. Checkout resolution rejects symlinks, special files, mount crossings,
absolute/traversal paths and unavailable descriptor primitives, rather than
weakening confinement. This currently requires Linux `openat2` and `/proc/self/fd`.
Ordinary Tools preserve their existing explicit symlink-policy behavior.
Scoped sessions use their existing scope mutex, current validity/source
capability checks, provider authorization and write leases; Create also checks
intermediate paths. Observation capabilities are attenuated per path and live
write leases, never broadened to the whole attachment. ReadOnly removes mutation
capabilities and rejects mutations. Routed operations use existing active-op
accounting and detach fencing. Remote HTTP forwards exact operation/result types
and validates returned operation/path pairing. External DTOs validate checkout
path/validator/content/result bounds. No provider endpoint or host root is added
to the observation.
## Effects and limits
Absent paths during checkout observation return `NotFound` (including a missing
ancestor or a non-directory ancestor); native discovery can return no route.
An execution against a previously observed target that disappeared instead
returns `Conflict`, consistently at pinning, shared-engine, and final-check
boundaries. Conflict means stale validator/hash/name or existing Create destination before
save effects. `WorkdirError::OutcomeUnknown` has a separate sanitized wire code
`outcome_unknown` (HTTP 500), never an automatic retry. Partial parent creation,
possible commit failure, or failed post-save observation is outcome unknown.
Remote lost/malformed/mismatched mutation responses are also outcome unknown.
Create failures conservatively report unknown once parent creation may begin,
even if a particular failure happened without a persisted effect.
Bounds: path <=4096 UTF-8 bytes/128 components, validator <=256 bytes; Read source
64 MiB, retained response <=1 MiB (or stricter provider read limits), Write/Edit
preimage and result <=8 MiB (or stricter provider limit), Edit replacements
<=10,000 (or stricter provider limit). One checked operation per Local session,
with bounded retained handles and no waiting queue. Cancellation/deadline checks
use the existing provider guard, or a 30-second cooperative guard for Local
checkout operations. Blocking kernel/filesystem calls are not preempted; this
is **not** a hard wall-time guarantee for hung storage. Transport request-body
bounds before serde allocation remain the hosting transport's responsibility.
### External-editor limitations (not a magic advisory-lock guarantee)
No advisory `flock` is claimed to exclude arbitrary editors. Provider-session
serialization does not lock other sessions/processes. Metadata checks detect
concurrent reads/edits up to each check, but are not a transactional snapshot
against privileged actors forging metadata. Linux offers no atomic
compare-identity-and-replace-by-name primitive here: an uncooperative external
rename/change between the final validator check and rename can still race the
save. The pinned handles prevent following replacements/symlinks into arbitrary
objects, and deterministic substitution before the save boundary is rejected,
but the final syscall gap is an explicit limitation. Callers must not interpret
this as linearizability against all external filesystem mutations. Likewise,
post-save observation is the then-current named state, not an exclusion of an
external edit after save. Crash durability of parent directories is not claimed.
Host generation reuse, native permission identity, read-history updates, model
Operation publication, transport hosting (Runtime/Backend/External), and durable
Ticket/MR review remain the parent integration's responsibilities.
## Bounded live List pagination
`ListRequest { path, limit, after: Option<ListCursor> }` accepts an exclusive
ordering key `ListCursor { kind: EntryKind, path: FsPath }`. Directories sort
first, then paths sort lexically within the directory/non-directory groups;
File, Symlink and Other share the non-directory group. `ListResult.next_after`
is the last returned key when another page remains. Reuse that key unchanged
with the same directory and output root. Cursor paths must name a direct child
in the result coordinate frame; they are not statted and need not still exist.
The optional fields default to absent and are omitted from JSON when absent.
The shared provider engine retains at most `limit + 1` candidates, additionally
bounded by the 1 MiB aggregate path-response ceiling (including continuation),
while scanning the directory once. It never stops merely because the page is
full: enumeration order cannot determine the retained prefix. `total_entries`
and `total_bytes` describe all visible entries in the current scan, including
keys at/before `after`, not only the page or remaining suffix. A byte-bounded
page may contain fewer than `limit` entries and still provide continuation.
A zero-limit request returns totals and truncation but no continuation.
List uses Read capability and read/enumeration scope, and **does not ignore-filter**
`.gitignore`/`.ignore` entries or hide dotfiles; Glob/Grep keep their own policy.
Cancellation and the 100,000-entry traversal ceiling apply to the complete scan,
including excluded entries. External DTOs retain their existing item/path bounds
and also validate cursor paths and continuation response bytes.
Each page observes a live directory, not a snapshot, and no cross-page
consistency is promised. Insertions after the cursor may appear; earlier keys
are not revisited. Removed entries disappear, including the cursor entry.
Renames or kind changes can move an entry across the cursor and cause omission
or repetition. Even one scan is not transactional against external editors.
## Scoped checkout search boundary
`CheckoutSearchRequest::new(CheckoutSearchOperation::{List, Glob, Grep}(request))`
uses the current session's logical root as its result frame. The typed result is
`CheckoutSearchResult::{List, Glob, Grep}`. The request's internal context fields
are `scope_layers: Vec<Vec<WorkdirToolScopeRule>>` and `output_root: WorkdirPath`.
Rules union within each layer; layers intersect with each other, the current
provider authority, and the output-root boundary. Wrappers contribute their own
scope; caller-supplied layers can only narrow, never replace it. Each wrapper
translates incoming provider coordinates once and forwards the composed output root.
List cursors remain output-root-relative, just like result paths.
Provider-rendered Grep text and every typed result path are already relative to
that final root; wrappers must not prefix cwd onto returned paths again.
Local/External use the same shared `run_list`, `run_glob`, `run_grep` engines.
`SearchAccess` gates metadata/content opens and enumeration. Shared List checks
`can_enumerate_directory`; Glob/Grep use the shared bounded descriptor walk and
owned `traversal_filter` policy to prune before descending/opening directories.
The walk anchors at the actual search directory, not at a broader provider
root. It counts rejected entries too, checks cancellation while scanning them,
and limits descent to 128 levels (bounded retained directory handles). Existing
provider-owned ignore loading, glob/type selection, grep context/offset/limit,
rendering, and source/result limits remain in their shared engines. Unreadable
ignore files are not opened to discover out-of-scope contents.
Descriptor search loads `.ignore`, `.gitignore` and an authorized directory's
`.git/info/exclude` through the same `FsAccessPolicy` opens as other sources.
It preserves anchored rules, negation, nearest-directory precedence within a
source class, `.ignore` precedence over Git rules, and nested repository
boundaries. Ignored directories are pruned before enumeration or loading their
own rules; explicitly selected ignored search roots are still traversable.
Only the current ancestor matchers are retained, never a second tree/store.
Glob requires an authorized `.git` directory/file marker for Git rules. Grep
retains the existing descriptor-provider contract of applying `.gitignore`
without a marker and rejecting unavailable/symlinked ignore opens; Glob skips
unavailable ignore sources like its ordinary walker. Scope-denied sources are
skipped by both before open. Bounds/cancellation failures are never retried.
Configuration discovery cannot expand provider authority. Ambient ancestors
outside the search result/provider root, global Git configuration/ignore files,
and linked-worktree `.git`/`commondir` administration pointers are not followed.
A `.git` file activates repository matching without treating it as a directory
or opening the host path it contains. In-root readable ignore rules remain
active, including inherited rules above a nested base. Descriptor ignore text
must be UTF-8, bounded to 1 MiB/file, 16 KiB/line and 8 MiB aggregate per search;
limit violations fail rather than returning silently incomplete filtering.
Grep's separate candidate-content budget remains 64 MiB. Ordinary unscoped
path-backed Tools keep their original discovery policy.
Scoped ordinary trait List/Glob/Grep use this boundary as well, returning
session-logical coordinates. Unscoped normal Tools continue using their original
trait/provider behavior. A provider without checkout search support returns
UnsupportedOperation; there is no broad traversal fallback. Native/scoped
searches intentionally fail closed on symlink paths, including logical-policy
aliases which normal unscoped Tools may follow. A visible symlink may therefore
cause a native List failure rather than expanding authority. Descriptor support
is currently Linux-only. Search remains request-time traversal, not a snapshot
or transactional exclusion of external editors.
HTTP operation/result variant: `CheckoutSearch`. External DTO validation rejects
absolute/out-of-root paths, empty scope layers, oversized paths/patterns/context/
results and invalid result paths. Generic request ceilings are 16 layers and 256
aggregate rules; External transport further limits aggregate rules to 64. Search
result limits follow the existing shared List/Glob/Grep wire bounds. Runtime and
Backend forwarding outside this crate remain parent integration work.
+12
View File
@@ -0,0 +1,12 @@
# workdir
Provides contracts for persistent Workdir identity and Worker-bound operation sessions. Handles local, HTTP, and External providers through shared scope and capability boundaries. Providers own materialization and deletion; caller Tools do not take over those responsibilities.
## Reference index
- [`WorkdirSession`] — provider operation and close boundaries.
- [`WorkdirSessionRouter`] — aliases, connection generations, and detach coordination.
- [`WorkdirToolBroker`] — scope delegation and write leases for child sessions.
- [`checkout`] — identity-validated operations and search coordinates.
- [`external`] / [`http`] — typed transport, unknown outcomes, and denial reasons.
- [`workspace`] — inventory projection without host information.
+24 -3
View File
@@ -1,6 +1,27 @@
//! Request-time checkout contracts. Validators are opaque provider observations,
//! not content hashes. Callers must separately retain Read's content hash for
//! read-before-mutation. Create destinations are checkout-root relative.
//! Contracts for request-time checkout operations and provider observations.
//!
//! Observations fetch metadata only, not recursive trees or file caches. Validators are opaque
//! identity/protected-metadata values, separate from Read content hashes and caller attachment
//! generations. Write/Edit require both a validator and a previously read content hash;
//! hashes alone do not accept same-content inode replacements. Callers own Read line numbering
//! and per-attachment read history.
//!
//! Create paths are checkout-root-relative and must be strict descendants of the observed target
//! directory. Existing destinations are never overwritten; results return the updated observation
//! of the retained target directory, not the created file. Read results also return the validator
//! of the state actually read, never a replacement from a later independent stat.
//!
//! Missing targets at observation yield NotFound; disappearance of observed targets or validator/hash
//! mismatches yield Conflict. Once parent creation starts, uncertain commit or post-save observation
//! yields OutcomeUnknown, not a no-effect claim. Providers/transports do not automatically retry
//! uncertain mutations.
//!
//! Search delegates to shared List/Glob/Grep engines. Scope rules form unions within layers and
//! intersections across layers, provider authority, and output roots. Wrappers transform coordinates
//! once; returned paths and Grep content are relative to the final session's logical root.
//! Paths are not reverse-parsed from rendered filenames; non-UTF-8 names are not lossily aliased.
//! Unsupported providers reject search rather than fall back to broader trait searches. Search is
//! live traversal, not an atomic snapshot of all descendants or exclusion of external editors.
use crate::{
ContentHash, EditResult, EntryKind, ReadResult, WorkdirError, WorkdirPath,
WorkdirSessionCapabilities, WorkdirSessionCapability, WriteResult,
+11 -5
View File
@@ -1,9 +1,15 @@
//! Versioned websocket DTOs for a client-hosted External Workdir provider.
//! Bounded WebSocket DTOs for client-hosted External providers.
//!
//! Authentication belongs to the websocket upgrade and Backend grant
//! authority. These frames intentionally contain only opaque identities,
//! logical Workdir operations, generation fences, and bounded control data.
//! They must never carry a provider host path, credential, or tunnel token.
//! Upgrade/Backend owns authentication and grant issuance. Frames contain only opaque identities,
//! logical operations, and generations, not host paths, credentials, or tunnel tokens. Reconnection
//! creates a new generation; old handles/results do not enter the new connection's command namespace.
//! Provider free-text errors are not forwarded; only closed codes/reasons are used.
//!
//! External grants follow COMMAND ⇒ WRITE ⇒ READ. WRITE includes READ for previously read hashes;
//! COMMAND remains separate explicit authority. General session capability intersections may form
//! subsets independently of this grant hierarchy; the codec never implicitly expands nonhierarchical
//! grants. Commands run with the shared host's OS permissions, not a sandbox confining processes
//! to a directory. Output is bounded and inline; provider-local spill paths are not exposed.
use std::path::{Component, Path};
+8 -5
View File
@@ -1,9 +1,12 @@
//! HTTP transport contract and client for remote Workdir sessions.
//! HTTP contracts and an optional client for remote Workdir sessions.
//!
//! The protocol keeps filesystem/search/process operations provider-side: one
//! [`WorkdirSessionOperation`] is one bounded HTTP request. The HTTP client is
//! optional so Runtime servers can share these DTOs without depending on a
//! client stack.
//! Keeps filesystem/search/process operations provider-side and pairs typed operations with results.
//! The optional client lets Runtime share DTOs without the client stack. Lost mutation responses,
//! decode failures, or pairing mismatches after dispatch yield OutcomeUnknown; no automatic resend.
//!
//! Denial reasons travel as an optional closed enum; provider free text maps to safe vocabulary.
//! Missing reasons stay unknown. Diagnostic wrappers preserve original classifications such as
//! Conflict/OutcomeUnknown and do not infer meaning from unknown reason strings.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
+14 -13
View File
@@ -1,9 +1,4 @@
//! Persistent Workdir identity and Worker-bound operation sessions.
//!
//! A [`Workdir`] identifies a materialized repository execution context across
//! Worker lifetimes. A [`WorkdirSession`] is the live operation attachment
//! bound to one Worker. Tools consume sessions; they do not own Workdir
//! materialization or cleanup.
#![doc = include_str!("../README.md")]
pub mod checkout;
pub use checkout::{
@@ -187,12 +182,13 @@ impl WorkdirSessionCapabilities {
pub type WriteOutcome = WriteResult;
/// Live, Worker-bound operations for one persistent [`Workdir`].
/// Worker-bound operation session for a persistent [`Workdir`].
///
/// Implementations execute filesystem search and command work on the host
/// that owns the materialization. Structured requests and results never
/// contain the raw materialized root. Closing a session is terminal and does
/// not delete the persistent Workdir or its materialization.
/// Filesystem search and commands run on the host owning materialization.
/// Session closure is terminal but does not delete the Workdir or files.
/// Unsupported checkout/search implementations explicitly reject requests, with no ambient
/// filesystem fallback. Providers own identity validators; callers own read hash history
/// and attachment generations.
#[async_trait]
pub trait WorkdirSession: std::fmt::Debug + Send + Sync {
fn workdir(&self) -> &Workdir;
@@ -294,8 +290,13 @@ pub trait WorkdirSession: std::fmt::Debug + Send + Sync {
pub type WorkdirSessionHandle = Arc<dyn WorkdirSession>;
/// Closed diagnostic vocabulary: no request text, paths or secrets.
/// A reason describes a refusing branch, not an authorization grant.
/// Closed diagnostic vocabulary for denial reasons, excluding paths and free text.
///
/// Assigned by the branch deciding denial, never inferred from error text. Missing reasons from
/// legacy providers remain unknown. Distinguishes PermissionDenied with an OS error code from
/// synthetic PermissionDenied of unknown origin, without asserting the specific OS-level cause.
/// Remote provider reasons are reported data, not authority or independent attestation.
/// `DenialContext` adds diagnostics without changing error classification or retry/cleanup rules.
#[derive(
Clone, Copy, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize, schemars::JsonSchema,
)]
+14 -12
View File
@@ -1,12 +1,15 @@
//! Scope-aware filesystem primitive.
//! Local sessions retaining scope and provider resources.
//!
//! `LocalWorkdirSession` is the write/read gate layered on top of a [`manifest::Scope`]
//! and a Worker's working directory. The scope decides which paths are
//! readable and writable; the cwd is carried alongside for convenience
//! (Glob/Grep default their search base to it).
//! Scope determines read/write bounds; cwd is the reference for searches and operations.
//! Clones share resources, but Tools own per-attachment read history. Checked observe/execute/search
//! and regular trait Read/Write/Edit share a nonqueueing session lock, serializing validator
//! comparison through I/O. This does not exclude other sessions/processes or external editors.
//!
//! `LocalWorkdirSession` is cheap to clone (`Arc` inside). Tool-specific session
//! state, such as read-before-edit tracking, remains owned by the tool layer.
//! Checkout uses Linux openat2 and /proc/self/fd, rejecting symlinks, special files, mount crossings,
//! absolute/traversal paths, and missing primitives. It does not implicitly expand regular Tools'
//! explicit symlink policies. Hashes bounded sources and uses shared fs-operation engines for
//! editing and temporary-file saves. Cooperative deadlines cannot forcibly interrupt kernel I/O,
//! so they provide no hard wall-time guarantee for hung storage.
use std::collections::{BTreeMap, HashMap};
use std::fmt::Debug;
@@ -670,12 +673,11 @@ impl Drop for LocalWorkdirSessionInner {
pub trait WorkdirSessionResource: Debug + Send + Sync {}
impl<T> WorkdirSessionResource for T where T: Debug + Send + Sync {}
/// An atomically validated, descriptor-pinned External Workdir root.
/// External provider root with identity checked and retained before approval.
///
/// Pinning compares the selected path identity with the opened descriptor, so
/// a rename or replacement between approval and open fails closed. The
/// descriptor, rather than the pathname, remains the filesystem authority for
/// the lifetime of every session created from this root.
/// Compares selected-path metadata with the opened descriptor, rejecting renames/replacements
/// during acquisition. Sessions from this root use the descriptor as filesystem authority;
/// replacing the ambient pathname after approval does not transfer authority to another tree.
#[derive(Debug, Clone)]
pub struct ExternalWorkdirRoot {
canonical_path: PathBuf,
+10 -6
View File
@@ -1,9 +1,9 @@
//! Alias-keyed Worker Workdir session routing.
//! Live session routing through Worker-local aliases.
//!
//! The router owns live provider sessions without exposing whether a session is
//! local or transported over HTTP. Attachment aliases are Worker-local stable
//! identities; provider Workdir ids and human display names are deliberately not
//! routing keys.
//! Pins targets by alias and connection generation, not provider Workdir IDs or display names.
//! Reusing an alias on detach/reattach does not direct old observations, read history, or operations
//! to the new connection. Routed operations retain active-operation accounting coordinated with
//! begin/finish/cancel-detach. Detach ends a session connection; it does not delete Workdirs or files.
use std::collections::{BTreeMap, HashSet};
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
@@ -376,7 +376,11 @@ impl WorkdirSession for RoutedWorkdirSession {
}
}
/// Worker-owned map of stable attachment aliases to provider-neutral sessions.
/// Maps Worker-owned aliases to live provider sessions.
///
/// Issues a generation per connection to distinguish alias reuse. Coordinates active operation
/// and command accounting with detach; old connections never retarget new sessions.
/// Detach does not delete the Workdir.
#[derive(Debug, Default)]
pub struct WorkdirSessionRouter {
sessions: RwLock<BTreeMap<WorkdirAttachmentAlias, Arc<RoutedSessionState>>>,
+17
View File
@@ -1,3 +1,15 @@
//! Scope brokers and write leases that attenuate parent session authority.
//!
//! Checks provider capabilities, attachment ceilings, parent scopes, and delegated scopes per operation.
//! Child write leases protect their ranges from conflicting parent/sibling writes; commands require
//! explicit grants and writable scope. Cwd sets coordinate transformations, not wider authority.
//! Rejects operations when the provider cannot compare scopes.
//!
//! Close revokes new operations before awaiting command cleanup; write leases remain held while
//! cleanup is uncertain. Checkout search composes rule layers and output roots, checking visibility
//! provider-side before name enumeration, descent, or content reads. Returned coordinates are not
//! prefixed with cwd again.
use std::collections::{BTreeMap, HashMap, HashSet};
use std::path::Path;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
@@ -64,6 +76,11 @@ pub struct WorkdirToolScope {
pub command: bool,
}
/// Broker mediating operations within parent provider and attachment ceilings.
///
/// Delegation only attenuates scope; cwd and command grants do not expand file authority.
/// Child write leases suppress conflicting writes and remain held if command cleanup on close
/// is uncertain.
#[derive(Clone)]
pub struct WorkdirToolBroker {
authority: Arc<ScopedWorkdirSession>,
+5 -4
View File
@@ -1,8 +1,9 @@
//! Shared Workdir inventory domain contracts.
//! Shared domain projection of Workdir inventory.
//!
//! These types are a neutral lower-level authority shared by Server and Runtime
//! transport projections. They intentionally do not expose provider/session
//! handles, host paths, Runtime URLs, or credentials.
//! Types shared by Server and Runtime expose no provider/session handles, host paths, Runtime URLs,
//! or credentials. A Workdir's inventory presence is separate from content access through a Worker's
//! live attachment. Status and past materialization evidence alone do not grant current scope
//! or cleanup authority.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};