Hare/yoi
1
1
Fork 0
Code Actions Packages Releases Activity

docs: consolidate design documentation and standardize crate English

Browse Source
This commit is contained in:
Hare
2026-10-12 14:58:00 +09:00
parent a633a53274
commit 4efb0ace1f
404 changed files with 4556 additions and 11654 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/tools/README.md
+6 -96
View File
@@ -1,100 +1,10 @@
# tools
## Role
`tools` implements file and command operations through Host-owned `WorkdirSession`s, plus Web fetching and image output. Owns input schemas, output formatting, and change detection after reads, but not Workdir creation or connection lifetimes, Manifest permission definitions, or the Engine execution loop.
`tools` implements built-in tools and shared tool execution helpers used by Workers.
## Main entry points
## Boundaries
Owns:
- built-in filesystem, web, memory, and Worker-management tool implementations where applicable
- bounded tool output formatting
- scope-aware file operation helpers
- tool-facing diagnostics suitable for history/model consumption
Does not own:
- manifest permission policy definition (`manifest`)
- Engine tool-loop semantics (`agen`)
- Worker lifecycle decisions (`worker`)
- UI presentation (`tui`)
## Design notes
A tool implementation must assume model input is untrusted. Permission policy, scope checks, output bounding, and redaction are part of the safety boundary, not optional UI behavior.
## Checkout-native processing
`execute_checkout_tool` (also exported through `tools::checkout`) accepts the
original `WorkdirSessionRouter`, shared `Tracker`, route-bound alias/generation,
logical target and provider validator. It does not create an attachment or router.
The Host owns native publication and permission identity; provider scope and
capabilities remain authoritative. `Create` uses the `Write` capability/identity.
| Operation | Arguments (only these keys accepted) | Target |
| --- | --- | --- |
| Read | `offset?`, `limit?` | Existing file; same numbered lines, defaults and byte bound as Tool Read |
| Edit | `old_string`, `new_string`, `replace_all?` | Existing file; prior read/hash and provider validator required |
| Write | `content` | Existing file only; prior read/hash and provider validator required |
| Create | `path`, `content` | Bound parent directory; relative destination strictly below it; create-new, missing parents allowed |
| List | `limit?` (1..1000, default 100), `after?` (`{kind, path}`) | Bound directory; Read capability; all typed `EntryKind` cursor values accepted, path is a direct child in checkout-root-relative provider coordinates |
| Glob | `pattern`, `path?` | Bound directory; optional path relative beneath it |
| Grep | `pattern`, `path?`, `glob?`, `type?`, `case_insensitive?`, `-B?`, `-A?`, `-C?`, `multiline?`, `output_mode?`, `head_limit?`, `offset?` | Bound directory; same provider search/options/rendering as Tool Grep |
Native arguments cannot override `file_path`, `target_workdir`, generation or
validator. Absolute/parent-traversing directory-relative paths are rejected.
Checkout file calls share Tool processing through an optional checked `FileTarget`
and use `checkout_execute`, never a separate Worker-side filesystem engine.
Read records the provider's full content hash **and total line count**, including
partial reads, in the original alias/generation namespace. Normal and native
observations/mutations interoperate. Only successful mutations update tracking.
Glob/Grep share normal request defaults/options and rendering through `SearchTarget`.
Only native dispatch uses `CheckoutSearchRequest::new` and `checkout_search`.
The constructor's empty scope layers/root output are refined by provider wrappers;
Tools does not synthesize authority or parse/rebase rendered filenames. Ordinary
Tool dispatch remains `session.glob`/`session.grep`, with its existing error mapping.
List dispatches one bounded `checkout_search` page without Bash or entry prefetch.
`CheckoutToolOutput.listing` preserves the typed `ListResult`, including
`next_after`, for Host projection into canonical Worldspace entries and cursors.
Its normal Tool output is a summary only, and `paths` is empty; every other
Operation returns `listing: None`. The Host decodes canonical cursor paths before
calling Tools, mapping the public `{kind, entry}` cursor into `{kind, path}`.
Both returned entries and the continuation must stay directly inside the bound
directory. Cursor kinds retain all typed values (`directory`, `file`, `symlink`,
`other`); ordering groups directories first and all non-directories together,
then compares paths. A truncated page must supply `next_after` matching its last
entry's ordering key; an untruncated page has no continuation. Special typed
listing entries are path coordinates, not checkout observations: direct checkout
observation remains provider File/Directory-only. Pagination is live, not a
cross-page snapshot; the same before/after bound-directory validator checks
used for search apply to List.
`CheckoutToolOutput.paths` contains typed checkout-root-relative search paths
(from `GlobResult.paths` / `GrepResult.paths`) and the created path for Create.
Rendered Grep text is never parsed to construct links. The optional validator is
the provider's returned observation validator, unchanged. Create's result
observation describes the bound parent target, while returned paths and tracker
updates describe the destination. Search checks the bound observation before and
after the bounded checkout provider search, but does **not** guarantee an atomic tree snapshot
or prevent external edits to descendants. File validator/hash checks are inside
the provider's checked execution boundary. Checked native provider conflicts and
native generation/search fences use `ToolError::StructuredConflict` with code
`checkout_stale`; ordinary Tool conflicts retain `ExecutionFailed`. Checked-only
known provider refusals use `checkout_denied` (scope/capability/read-only policy),
`checkout_invalid` (invalid arguments/paths/target kind/search syntax), or
`checkout_unavailable` (closed session/unavailable capacity). A disappeared bound
target is `checkout_stale`. These codes preserve known no-effect rejection for the
Host; ordinary Tool error mappings are unchanged. Ambiguous transport/IO and
operation failures remain `ExecutionFailed`, as does explicit provider
`OutcomeUnknown`; a failure after mutation must not
be automatically retried. A malformed post-mutation provider result is likewise
reported explicitly as outcome unknown. Explicit mismatched Read/Glob/Grep results
are readonly failures classified as `checkout_unavailable`, not stale or unknown.
## See also
- [`../../docs/design/tool-permissions-scope.md`](../../docs/design/tool-permissions-scope.md)
- [`../../docs/design/memory-knowledge.md`](../../docs/design/memory-knowledge.md)
- [`core_builtin_tools`](src/lib.rs) / [`routed_builtin_tools`](src/lib.rs): registration by provider capability and connection resolution.
- [`checkout`](src/checkout.rs): shared handling for route-bound native operations and regular Tools.
- [`tracker`](src/tracker.rs): read hashes and change tracking per connection generation.
- [`view_image`](src/view_image.rs): bounded byte retrieval and image detection.
crates/tools/src/bash.rs
+8
View File
@@ -1,3 +1,11 @@
//! Bash Tool using WorkdirSession command capability.
//!
//! The selected session owns cwd, process launch, bounded output, timeout, and cancellation.
//! Bash is not a sandbox; the Runtime process and OS user define the execution boundary.
//! Per-execution handles and nonces prevent cancellation of another command. Dropping a future
//! does not confirm termination; active registrations remain until cleanup requests cancellation
//! and observes terminal output.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
crates/tools/src/checkout.rs
+32 -7
View File
@@ -1,11 +1,36 @@
//! Native checkout entrypoint using the same processing as ordinary filesystem Tools.
//! Native checkout entry points sharing regular filesystem Tool handling.
//!
//! The Host binds alias, generation, path and validator from the checkout route; none
//! can be overridden by Operation arguments. File reads/mutations dispatch through
//! the provider's checked execution boundary. Search shares normal request and
//! rendering processing but dispatches through checkout_search so provider scope
//! layers and output rebasing are preserved; checks bracket the bound directory
//! observation, not an atomic snapshot of the searched tree.
//! The Host pins aliases, connection generations, paths, and validators from routes; operation
//! arguments cannot override them. The original router and Tracker record regular Tool and native
//! read/change observations under the same connection generation. Checks provider capabilities
//! without creating new routers or attachments. Create uses Write capability and accepts only
//! relative paths below the bound directory.
//!
//! # Checked execution and observation
//!
//! File operations dispatch through checked FileTarget to `checkout_execute`; hash and validator
//! comparisons occur at the provider execution boundary. Partial Read also records the whole-file
//! hash and total line count; tracking updates only after successful mutation. Post-operation
//! validators are returned unchanged from provider responses. Create observes the parent directory,
//! while its created path and tracking target are the new file; these remain distinct.
//!
//! Glob/Grep share regular Tool defaults and formatting, but native operations use `checkout_search`
//! to preserve provider wrapper scope layers and output coordinates. Only the bound directory is
//! checked before/after search; this neither prevents concurrent descendant edits nor guarantees
//! an atomic snapshot of the whole tree. Paths come from typed provider results, not links
//! reconstructed from rendered Grep content.
//!
//! # live directory page
//!
//! List returns one bounded page and a typed continuation without prefetching entries.
//! Entries and cursors are limited to direct children of the bound directory; kind/path ordering
//! is checked. Special entries are path coordinates, not substitutes for checked file/directory
//! observations. Pagination is live, not a cross-page snapshot. The Host owns conversion to
//! Worldspace Entry/cursors; Tools retain checkout-root-relative provider coordinates.
//!
//! file_target classifies known checked-operation rejections into structured codes. Ambiguous
//! transport/IO failures and invalid post-mutation responses are not downgraded to stale/no-execution.
//! Regular Tool error mapping stays unchanged; diagnostic strings are not parsed to classify denial.
use std::sync::Arc;
use agen::tool::{ToolError, ToolExecutionContext, ToolOutput};
crates/tools/src/file_target.rs
+8 -1
View File
@@ -1,4 +1,11 @@
//! Shared provider dispatch for Tool and checkout file processing.
//! File dispatch shared by regular Tools and native checkout.
//!
//! Targets with validators use provider checked execution; others use regular session operations.
//! Post-operation validators come from the same execution response, not a later stat.
//! Only known checked rejections map to `checkout_stale`, `checkout_denied`, `checkout_invalid`,
//! or `checkout_unavailable`. Ambiguous IO/transport failures and explicit OutcomeUnknown do not
//! imply nonexecution. Invalid read-only responses mean retrieval failure; invalid mutation
//! responses mean unknown outcomes.
use agen::tool::ToolError;
use workdir::{
CheckoutOperation, CheckoutOutput, CheckoutRequest, ContentHash, EditRequest, EditResult,
crates/tools/src/lib.rs
+1 -10
View File
@@ -1,13 +1,4 @@
//! Built-in tools for the Yoi LLM agent.
//!
//! Read / Write / Edit / Glob / Grep / Bash operate through a host-owned
//! [`workdir::WorkdirSession`] handle. This crate owns tool schemas, rendering, and
//! read-before-edit tracking; it does not own Workdir identity/materialization or
//! WorkdirSession lifecycle.
//!
//! Bash is intentionally not sandboxed. The WorkdirSession supplies its initial cwd
//! and command capability, while the Runtime process and OS user remain the
//! trusted execution boundary.
#![doc = include_str!("../README.md")]
pub mod checkout;
pub mod error;
crates/tools/src/search_target.rs
+6 -3
View File
@@ -1,6 +1,9 @@
//! Provider dispatch shared by native and ordinary search processing.
//! Request construction/defaults and rendering remain in Glob/Grep; this only
//! selects the provider surface and verifies the returned readonly result kind.
//! Provider dispatch shared by regular search and native checkout search.
//!
//! Glob/Grep own request defaults, input handling, and formatting; this module checks provider
//! routing and read-only response kinds. Only native search uses `checkout_search`; provider
//! wrappers restrict its request scope and output coordinates. Tools neither create scopes
//! nor reinterpret rendered paths.
use agen::tool::ToolError;
use workdir::{
CheckoutSearchOperation, CheckoutSearchRequest, CheckoutSearchResult, GlobRequest, GlobResult,
crates/tools/src/tracker.rs
+12 -42
View File
@@ -1,46 +1,16 @@
//! Worker-lifetime tracker for file operations performed by the builtin
//! file-manipulation tools.
//! Tracker retaining file Tool read hashes, mutation coordination, and recent operation paths.
//!
//! A `Tracker` serves two orthogonal purposes:
//! Read and successful Write/Edit record content hashes and line counts. Before changing existing
//! files, checks against the last observation. This verifies mutation preconditions, not scope
//! authorization, and complements hash/validator checks at the provider execution boundary.
//! Regular Tools and native operations on the same attachment share observations. Aliases and
//! connection generations separate namespaces, preventing stale reads from applying to a different
//! session when an alias is reused after detach.
//!
//! 1. **Read-before-edit policy.** It records a SHA-256 hash of each
//! file's contents at the moment it was observed via `Read` (or
//! mutated via `Write` / `Edit`), and lets `Write` / `Edit` later
//! verify that the file has not been externally modified since then.
//!
//! 2. **Recency of touched files.** It keeps an LRU-ordered list of
//! files that have been touched by any of the tools, so the Worker
//! layer can ask "which files did the agent recently look at?" —
//! used e.g. as a default reference set passed to context compaction.
//!
//! Despite its historic name, the Tracker already watches all three of
//! Read / Write / Edit; the rename away from `ReadTracker` reflects this.
//!
//! # Lifetime
//!
//! A `Tracker` is **Worker-process scoped**: the Worker layer creates a fresh
//! instance at the start of each Worker run (including resume) and discards
//! it when the process exits — it is not persisted, so a resumed
//! conversation starts with an empty read/edit history. The local WorkdirSession
//! scope boundary is likewise Worker-process scoped (derived from the
//! manifest). The two are orthogonal and the Worker wires them together
//! when registering builtin tools.
//!
//! ```no_run
//! # use std::path::PathBuf;
//! # use std::sync::Arc;
//! # use manifest::Scope;
//! # use tools::{Tracker, core_builtin_tools};
//! # use workdir::{LocalWorkdirSession, WorkdirSessionHandle};
//! let scope = Scope::writable("/workspace").unwrap();
//! let session: WorkdirSessionHandle = Arc::new(LocalWorkdirSession::new(
//! scope,
//! PathBuf::from("/workspace"),
//! ));
//! let tracker = Tracker::new(); // session lifetime
//! let bash_outputs = PathBuf::from("/run/yoi/bash-output");
//! let defs = core_builtin_tools(session, tracker, bash_outputs);
//! ```
//! Per-path mutexes coordinate mutations within a Worker only; they do not prevent external process
//! edits. A bounded LRU of recently touched paths supplies reference candidates for compaction, etc.
//! Tracker is transient state of a running Worker, never persisted/restored. Past Reads in a restored
//! conversation do not establish mutation preconditions for a new Tracker.
use std::collections::{HashMap, VecDeque};
use std::path::{Component, Path, PathBuf};
@@ -215,7 +185,7 @@ impl Tracker {
/// subsequent edits see a clean history).
///
/// Also bumps `path` to the front of the recency list. If the list
/// grows past [`RECENCY_CAPACITY`], the oldest entry is evicted.
/// grows past `RECENCY_CAPACITY`, the oldest entry is evicted.
pub fn record(&self, path: &Path, bytes: &[u8]) {
let key = self.key(canonicalize_or_owned(path));
let hash = hash_bytes(bytes);