feat: limit checkout indexing to entrances and expose bounded typed discovery

Content directories always have no indexable children; direct authorized paths remain resolvable. Add List using provider Read/enumeration authority and separate List permission, dirs-first lexical live continuation, bounded retention/path bytes, scoped result coordinates and External DTO validation. List/Glob/Grep return Descriptor-typed entries without eager acquisition. Bind content Interfaces coherently to the live entrance scope.

Validation: root cargo check; FS 58 unit+6 pagination, Workdir 125+5 (default and http-client), Tools 84 unit+39 integration+1 doctest, Worker 901 unit+110 integration, Server 691 unit+11 CLI, Server API 83 tests; exact crates.io WIP 0.2.0 resolution; format/diff checks. Focused model/local/remote/provider/race/scope checks passed. Commands and semantics in docs/development/wip-integration-validation.md.

Publication/review paused: continuation ShowTicket returned HTTP 401. No Backend bypass, source push, MR approval, target integration or live environment update is claimed.

crates/workdir/CHECKOUT_PROVIDER.md

@@ -101,6 +101,37 @@ Host generation reuse, native permission identity, read-history updates, model
 Operation publication, transport hosting (Runtime/Backend/External), and durable
 Ticket/MR review remain the parent integration's responsibilities.
 
+## Bounded live List pagination
+
+`ListRequest { path, limit, after: Option<ListCursor> }` accepts an exclusive
+ordering key `ListCursor { kind: EntryKind, path: FsPath }`. Directories sort
+first, then paths sort lexically within the directory/non-directory groups;
+File, Symlink and Other share the non-directory group. `ListResult.next_after`
+is the last returned key when another page remains. Reuse that key unchanged
+with the same directory and output root. Cursor paths must name a direct child
+in the result coordinate frame; they are not statted and need not still exist.
+The optional fields default to absent and are omitted from JSON when absent.
+
+The shared provider engine retains at most `limit + 1` candidates, additionally
+bounded by the 1 MiB aggregate path-response ceiling (including continuation),
+while scanning the directory once. It never stops merely because the page is
+full: enumeration order cannot determine the retained prefix. `total_entries`
+and `total_bytes` describe all visible entries in the current scan, including
+keys at/before `after`, not only the page or remaining suffix. A byte-bounded
+page may contain fewer than `limit` entries and still provide continuation.
+A zero-limit request returns totals and truncation but no continuation.
+List uses Read capability and read/enumeration scope, and **does not ignore-filter**
+`.gitignore`/`.ignore` entries or hide dotfiles; Glob/Grep keep their own policy.
+Cancellation and the 100,000-entry traversal ceiling apply to the complete scan,
+including excluded entries. External DTOs retain their existing item/path bounds
+and also validate cursor paths and continuation response bytes.
+
+Each page observes a live directory, not a snapshot, and no cross-page
+consistency is promised. Insertions after the cursor may appear; earlier keys
+are not revisited. Removed entries disappear, including the cursor entry.
+Renames or kind changes can move an entry across the cursor and cause omission
+or repetition. Even one scan is not transactional against external editors.
+
 ## Scoped checkout search boundary
 
 `CheckoutSearchRequest::new(CheckoutSearchOperation::{List, Glob, Grep}(request))`
@@ -110,7 +141,8 @@ are `scope_layers: Vec<Vec<WorkdirToolScopeRule>>` and `output_root: WorkdirPath
 Rules union within each layer; layers intersect with each other, the current
 provider authority, and the output-root boundary. Wrappers contribute their own
 scope; caller-supplied layers can only narrow, never replace it. Each wrapper
-translates all incoming coordinates once and forwards the composed output root.
+translates incoming provider coordinates once and forwards the composed output root.
+List cursors remain output-root-relative, just like result paths.
 Provider-rendered Grep text and every typed result path are already relative to
 that final root; wrappers must not prefix cwd onto returned paths again.
 

crates/workdir/src/checkout.rs

@@ -167,6 +167,19 @@ impl CheckoutSearchRequest {
         }
         let valid = match &self.operation {
             CheckoutSearchOperation::List(r) => {
+                if let Some(after) = &r.after {
+                    validate_path(&after.path)?;
+                    if after.path.is_root()
+                        || std::path::Path::new(after.path.as_str()).parent()
+                            != std::path::Path::new(r.path.as_str())
+                                .strip_prefix(self.output_root.as_str())
+                                .ok()
+                    {
+                        return Err(WorkdirError::InvalidArgument(
+                            "List cursor must name a direct child of the listed directory".into(),
+                        ));
+                    }
+                }
                 r.limit <= crate::external::MAX_EXTERNAL_RESULT_ITEMS
             }
             CheckoutSearchOperation::Glob(r) => {
@@ -793,6 +806,7 @@ mod tests {
         for provider in [&local as &dyn WorkdirSession, &external] {
             let listed = provider
                 .list(ListRequest {
+                    after: None,
                     path: WorkdirPath::root(),
                     limit: 100,
                 })

crates/workdir/src/checkout_search_tests.rs

@@ -60,6 +60,7 @@ fn fixture() -> tempfile::TempDir {
 async fn verify_sparse(session: &dyn WorkdirSession) {
     let listed = session
         .list(ListRequest {
+            after: None,
             path: WorkdirPath::root(),
             limit: 100,
         })
@@ -75,6 +76,7 @@ async fn verify_sparse(session: &dyn WorkdirSession) {
     );
     let empty = session
         .list(ListRequest {
+            after: None,
             path: path("denied"),
             limit: 100,
         })
@@ -108,6 +110,7 @@ async fn verify_sparse(session: &dyn WorkdirSession) {
     let CheckoutSearchResult::List(native) = session
         .checkout_search(CheckoutSearchRequest::new(CheckoutSearchOperation::List(
             ListRequest {
+                after: None,
                 path: WorkdirPath::root(),
                 limit: 100,
             },
@@ -161,6 +164,7 @@ async fn checkout_search_nested_sparse_nonrecursive_scopes_rebase_and_exclude_co
             .tool_session()
             .checkout_search(CheckoutSearchRequest::new(CheckoutSearchOperation::List(
                 ListRequest {
+                    after: None,
                     path: WorkdirPath::root(),
                     limit: 100
                 }
@@ -196,6 +200,7 @@ async fn checkout_search_composes_actual_nested_wrappers_and_output_root() {
     let CheckoutSearchResult::List(result) = stacked
         .checkout_search(CheckoutSearchRequest::new(CheckoutSearchOperation::List(
             ListRequest {
+                after: None,
                 path: path("keep"),
                 limit: 100,
             },
@@ -229,6 +234,7 @@ async fn checkout_search_composes_actual_nested_wrappers_and_output_root() {
     let entries = second
         .tool_session()
         .list(ListRequest {
+            after: None,
             path: WorkdirPath::root(),
             limit: 100,
         })
@@ -246,6 +252,7 @@ async fn checkout_search_composes_actual_nested_wrappers_and_output_root() {
         second
             .tool_session()
             .list(ListRequest {
+                after: None,
                 path: path("deep"),
                 limit: 100
             })
@@ -292,6 +299,7 @@ async fn checkout_search_scope_layers_intersect_before_list_and_grep() {
         root.path().to_path_buf(),
     );
     let mut request = CheckoutSearchRequest::new(CheckoutSearchOperation::List(ListRequest {
+        after: None,
         path: path("child"),
         limit: 100,
     }));
@@ -325,6 +333,7 @@ async fn checkout_search_wire_dto_bounds_and_provider_capabilities() {
         root.path().to_path_buf(),
     );
     let request = CheckoutSearchRequest::new(CheckoutSearchOperation::List(ListRequest {
+        after: None,
         path: path("child/keep"),
         limit: 100,
     }));
@@ -795,6 +804,7 @@ async fn checkout_search_preserves_current_provider_scope_and_rejects_symlink_es
     let CheckoutSearchResult::List(result) = source
         .checkout_search(CheckoutSearchRequest {
             operation: CheckoutSearchOperation::List(ListRequest {
+                after: None,
                 path: path("child/denied"),
                 limit: 100,
             }),

crates/workdir/src/external.rs

@@ -155,7 +155,11 @@ fn validate_external_operation(operation: &WorkdirSessionOperation) -> Result<()
                     <= MAX_EXTERNAL_SCOPE_RULES
                 && validate_external_operation(&match &request.operation {
                     crate::CheckoutSearchOperation::List(r) => {
-                        WorkdirSessionOperation::List(r.clone())
+                        // request.validate() checks the output_root-relative
+                        // cursor; validate the provider-coordinate path here.
+                        let mut list = r.clone();
+                        list.after = None;
+                        WorkdirSessionOperation::List(list)
                     }
                     crate::CheckoutSearchOperation::Glob(r) => {
                         WorkdirSessionOperation::Glob(r.clone())
@@ -212,7 +216,14 @@ fn validate_external_operation(operation: &WorkdirSessionOperation) -> Result<()
                     .contains(&request.max_bytes)
         }
         WorkdirSessionOperation::List(request) => {
-            is_root_relative(&request.path) && request.limit <= MAX_EXTERNAL_RESULT_ITEMS
+            is_root_relative(&request.path)
+                && request.limit <= MAX_EXTERNAL_RESULT_ITEMS
+                && request.after.as_ref().is_none_or(|after| {
+                    is_root_relative(&after.path)
+                        && !after.path.is_root()
+                        && Path::new(after.path.as_str()).parent()
+                            == Some(Path::new(request.path.as_str()))
+                })
         }
         WorkdirSessionOperation::Glob(request) => {
             is_root_relative(&request.path)
@@ -395,7 +406,20 @@ fn validate_external_result(result: &WorkdirSessionOperationResult) -> Result<()
                     .entries
                     .iter()
                     .all(|entry| is_root_relative(&entry.path))
-                && result_paths_fit(result.entries.iter().map(|entry| &entry.path))
+                && result.next_after.as_ref().is_none_or(|after| {
+                    is_root_relative(&after.path)
+                        && result.truncated
+                        && result.entries.last().is_some_and(|entry| {
+                            entry.kind == after.kind && entry.path == after.path
+                        })
+                })
+                && result_paths_fit(
+                    result
+                        .entries
+                        .iter()
+                        .map(|entry| &entry.path)
+                        .chain(result.next_after.iter().map(|after| &after.path)),
+                )
         }
         WorkdirSessionOperationResult::Glob(result) => {
             result.paths.len() <= MAX_EXTERNAL_RESULT_ITEMS
@@ -917,6 +941,7 @@ mod tests {
         assert!(serde_json::from_value::<ExternalWorkdirServerFrame>(nested).is_err());
 
         let unbounded = WorkdirSessionOperation::List(ListRequest {
+            after: None,
             path: WorkdirPath::root(),
             limit: MAX_EXTERNAL_RESULT_ITEMS + 1,
         });

crates/workdir/src/lib.rs

@@ -33,9 +33,9 @@ use tokio::sync::broadcast;
 
 pub use fs_operation::{
     BoundedReadLimits, ContentHash, EditRequest, EditResult, EntryKind, FsPath as WorkdirPath,
-    GlobRequest, GlobResult, GrepOutputMode, GrepRequest, GrepResult, ListEntry, ListRequest,
-    ListResult, ReadBytesRequest, ReadBytesResult, ReadRequest, ReadResult, StatRequest,
-    StatResult, WriteRequest, WriteResult,
+    GlobRequest, GlobResult, GrepOutputMode, GrepRequest, GrepResult, ListCursor, ListEntry,
+    ListRequest, ListResult, ReadBytesRequest, ReadBytesResult, ReadRequest, ReadResult,
+    StatRequest, StatResult, WriteRequest, WriteResult,
 };
 pub use http::dispatch_workdir_session_operation;
 pub use local::{

crates/workdir/src/local.rs

@@ -2695,6 +2695,7 @@ mod tests {
         let listed = WorkdirSession::list(
             &workdir,
             ListRequest {
+                after: None,
                 path: WorkdirPath::new("notes").unwrap(),
                 limit: 10,
             },
@@ -3385,6 +3386,7 @@ mod tests {
         let list = WorkdirSession::list(
             &workdir,
             ListRequest {
+                after: None,
                 path: WorkdirPath::new("yoi.local").unwrap(),
                 limit: 10,
             },

crates/workdir/src/scope.rs

@@ -1005,6 +1005,9 @@ impl WorkdirSession for ScopedWorkdirSession {
             .await?;
         self.ensure_read(&path, request.operation.capability())?;
         *request.operation.path_mut() = path;
+        // List cursors stay output_root-relative, just like source results.
+        // Composing output_root below handles their coordinates without stat
+        // of a key that may have disappeared between live pages.
         request.output_root = self.resolve_path(&request.output_root)?;
         for layer in &mut request.scope_layers {
             for rule in layer {
@@ -1873,6 +1876,7 @@ mod tests {
         assert_eq!(
             parent
                 .list(ListRequest {
+                    after: None,
                     path: fs_path("child"),
                     limit: 100
                 })
@@ -1890,6 +1894,7 @@ mod tests {
             child
                 .tool_session()
                 .list(ListRequest {
+                    after: None,
                     path: FsPath::root(),
                     limit: 100
                 })
@@ -1901,6 +1906,7 @@ mod tests {
                 .tool_session()
                 .checkout_search(crate::CheckoutSearchRequest::new(
                     crate::CheckoutSearchOperation::List(ListRequest {
+                        after: None,
                         path: FsPath::root(),
                         limit: 100
                     })

crates/workdir/tests/list_pagination.rs

@@ -0,0 +1,267 @@
+#![cfg(target_os = "linux")]
+use std::sync::Arc;
+use workdir::external::{
+    ExternalWorkdirOperation, ExternalWorkdirOperationResult, MAX_EXTERNAL_RESULT_ITEMS,
+};
+use workdir::http::{WorkdirSessionOperation as Op, WorkdirSessionOperationResult as Output};
+use workdir::*;
+
+fn path(value: &str) -> WorkdirPath {
+    WorkdirPath::new(value).unwrap()
+}
+fn rule(value: &str, recursive: bool) -> WorkdirToolScopeRule {
+    WorkdirToolScopeRule {
+        target: path(value),
+        recursive,
+        permission: WorkdirToolScopePermission::Read,
+        symlink_policy: manifest::SymlinkPolicy::Logical,
+    }
+}
+fn scope(rules: Vec<WorkdirToolScopeRule>, cwd: &str) -> WorkdirToolScope {
+    WorkdirToolScope {
+        rules,
+        cwd: path(cwd),
+        command: false,
+    }
+}
+fn request(value: &str, limit: usize, after: Option<ListCursor>) -> ListRequest {
+    ListRequest {
+        path: path(value),
+        limit,
+        after,
+    }
+}
+fn names(result: &ListResult) -> Vec<&str> {
+    result
+        .entries
+        .iter()
+        .map(|entry| entry.path.as_str())
+        .collect()
+}
+
+#[tokio::test]
+async fn list_scoped_external_pages_rebase_cursor_and_preserve_sparse_authority() {
+    let root = tempfile::tempdir().unwrap();
+    std::fs::create_dir_all(root.path().join("child/deep")).unwrap();
+    for name in ["child/a", "child/z", "child/deep/secret", "outside"] {
+        std::fs::write(root.path().join(name), b"x").unwrap();
+    }
+    let source = LocalWorkdirSession::external_read_only(
+        Workdir::new("pagination"),
+        root.path(),
+        BoundedReadLimits::EXTERNAL_DEFAULT,
+    )
+    .unwrap();
+    let broker = WorkdirToolBroker::new(Arc::new(source));
+    let lease = broker
+        .scope(scope(vec![rule("child", false)], "child"))
+        .await
+        .unwrap();
+    // Two actual wrappers must not duplicate child/ in the continuation.
+    let stacked = WorkdirToolBroker::new(lease.tool_session());
+    let first = stacked.list(request("", 1, None)).await.unwrap();
+    assert_eq!(names(&first), ["deep"]);
+    assert_eq!(first.next_after.as_ref().unwrap().path, path("deep"));
+    assert_eq!(
+        first.next_after.as_ref().unwrap().kind,
+        EntryKind::Directory
+    );
+    assert_eq!(first.total_entries, 3);
+    assert!(
+        stacked
+            .list(request("deep", 1, None))
+            .await
+            .unwrap()
+            .entries
+            .is_empty()
+    );
+    let second = stacked
+        .list(request("", 1, first.next_after))
+        .await
+        .unwrap();
+    assert_eq!(names(&second), ["a"]);
+    std::fs::remove_file(root.path().join("child/a")).unwrap();
+    std::fs::write(root.path().join("child/b"), b"xx").unwrap();
+    let third = stacked
+        .list(request("", 10, second.next_after))
+        .await
+        .unwrap();
+    assert_eq!(names(&third), ["b", "z"]);
+    assert!(!third.truncated);
+    assert!(third.next_after.is_none());
+    assert_eq!(third.total_entries, 3);
+    lease.close().await.unwrap();
+    assert!(matches!(
+        stacked.list(request("", 1, None)).await,
+        Err(WorkdirError::SessionClosed)
+    ));
+}
+
+#[tokio::test]
+async fn list_checkout_search_cursor_uses_output_root_coordinates_across_nested_wrappers() {
+    let root = tempfile::tempdir().unwrap();
+    std::fs::create_dir_all(root.path().join("child/keep")).unwrap();
+    for name in ["a", "b", "c"] {
+        std::fs::write(root.path().join("child/keep").join(name), b"x").unwrap();
+    }
+    let source = LocalWorkdirSession::external_read_only(
+        Workdir::new("rebased-pagination"),
+        root.path(),
+        BoundedReadLimits::EXTERNAL_DEFAULT,
+    )
+    .unwrap();
+    let broker = WorkdirToolBroker::new(Arc::new(source));
+    let lease = broker
+        .scope(scope(vec![rule("child", true)], "child"))
+        .await
+        .unwrap();
+    let stacked = WorkdirToolBroker::new(lease.tool_session());
+    let mut search =
+        CheckoutSearchRequest::new(CheckoutSearchOperation::List(request("keep", 1, None)));
+    search.output_root = path("keep");
+    let CheckoutSearchResult::List(first) = stacked.checkout_search(search.clone()).await.unwrap()
+    else {
+        panic!()
+    };
+    assert_eq!(names(&first), ["a"]);
+    if let CheckoutSearchOperation::List(list) = &mut search.operation {
+        list.after = first.next_after;
+    }
+    let CheckoutSearchResult::List(second) = stacked.checkout_search(search).await.unwrap() else {
+        panic!()
+    };
+    assert_eq!(names(&second), ["b"]);
+    assert_eq!(second.next_after.unwrap().path, path("b"));
+}
+
+#[tokio::test]
+async fn list_read_capability_suffices_without_glob_and_missing_read_is_denied() {
+    let root = tempfile::tempdir().unwrap();
+    std::fs::write(root.path().join("a"), b"x").unwrap();
+    let source = LocalWorkdirSession::materialized(
+        root.path().to_path_buf(),
+        root.path().to_path_buf(),
+        manifest::SharedScope::new(manifest::Scope::writable(root.path()).unwrap()),
+        WorkdirSessionCapabilities::from_capabilities([WorkdirSessionCapability::Read]),
+    );
+    let broker = WorkdirToolBroker::new(Arc::new(source));
+    let lease = broker.scope(scope(vec![rule("", true)], "")).await.unwrap();
+    assert_eq!(
+        names(
+            &lease
+                .tool_session()
+                .list(request("", 1, None))
+                .await
+                .unwrap()
+        ),
+        ["a"]
+    );
+    let denied = LocalWorkdirSession::external_with_capabilities_pinned(
+        Workdir::new("no-read"),
+        ExternalWorkdirRoot::pin(root.path()).unwrap(),
+        BoundedReadLimits::EXTERNAL_DEFAULT,
+        WorkdirSessionCapabilities::COMMAND_ONLY,
+    )
+    .unwrap();
+    assert!(matches!(
+        denied.list(request("", 1, None)).await,
+        Err(WorkdirError::Unsupported(WorkdirSessionCapability::Read))
+    ));
+}
+
+#[tokio::test]
+async fn list_external_wire_preserves_continuation_and_rejects_unbounded_cursor() {
+    let root = tempfile::tempdir().unwrap();
+    std::fs::create_dir(root.path().join("child")).unwrap();
+    for name in ["a", "b"] {
+        std::fs::write(root.path().join("child").join(name), b"x").unwrap();
+    }
+    let source = LocalWorkdirSession::external_read_only(
+        Workdir::new("wire-list"),
+        root.path(),
+        BoundedReadLimits::EXTERNAL_DEFAULT,
+    )
+    .unwrap();
+    let first = source.list(request("child", 1, None)).await.unwrap();
+    let wire = serde_json::to_value(
+        ExternalWorkdirOperationResult::try_from(Output::List(first.clone())).unwrap(),
+    )
+    .unwrap();
+    assert_eq!(wire["result"]["next_after"]["path"], "child/a");
+    let decoded: ExternalWorkdirOperationResult = serde_json::from_value(wire.clone()).unwrap();
+    assert_eq!(decoded.into_inner(), Output::List(first.clone()));
+    let operation =
+        ExternalWorkdirOperation::try_from(Op::List(request("child", 1, first.next_after)))
+            .unwrap();
+    let decoded: ExternalWorkdirOperation =
+        serde_json::from_value(serde_json::to_value(operation).unwrap()).unwrap();
+    let Output::List(next) = dispatch_workdir_session_operation(&source, decoded.into_inner())
+        .await
+        .unwrap()
+    else {
+        panic!()
+    };
+    assert_eq!(names(&next), ["child/b"]);
+    assert!(next.next_after.is_none());
+    for bad_path in ["/host".to_owned(), "x".repeat(4097), "outside/a".to_owned()] {
+        let mut bad = serde_json::to_value(Op::List(request("child", 1, None))).unwrap();
+        bad["request"]["after"] = serde_json::json!({"kind": "file", "path": bad_path});
+        assert!(serde_json::from_value::<ExternalWorkdirOperation>(bad).is_err());
+        let mut bad_result = wire.clone();
+        bad_result["result"]["next_after"]["path"] = serde_json::json!(bad_path);
+        assert!(serde_json::from_value::<ExternalWorkdirOperationResult>(bad_result).is_err());
+    }
+    assert!(
+        ExternalWorkdirOperation::try_from(Op::List(request(
+            "child",
+            MAX_EXTERNAL_RESULT_ITEMS + 1,
+            None
+        )))
+        .is_err()
+    );
+    // Search cursors use result coordinates, not provider path coordinates.
+    let mut search = CheckoutSearchRequest::new(CheckoutSearchOperation::List(request(
+        "child",
+        1,
+        Some(ListCursor {
+            kind: EntryKind::File,
+            path: path("a"),
+        }),
+    )));
+    search.output_root = path("child");
+    let bounded = ExternalWorkdirOperation::try_from(Op::CheckoutSearch(search)).unwrap();
+    let Output::CheckoutSearch(CheckoutSearchResult::List(next)) =
+        dispatch_workdir_session_operation(&source, bounded.into_inner())
+            .await
+            .unwrap()
+    else {
+        panic!()
+    };
+    assert_eq!(names(&next), ["b"]);
+}
+
+#[test]
+fn list_external_result_counts_cursor_toward_aggregate_path_bound() {
+    let entries: Vec<_> = (0..252)
+        .map(|n| ListEntry {
+            path: path(&format!("{n:04}{}", "x".repeat(4092))),
+            kind: EntryKind::File,
+            size: 1,
+        })
+        .collect();
+    let after = ListCursor {
+        kind: EntryKind::File,
+        path: entries.last().unwrap().path.clone(),
+    };
+    let result = ListResult {
+        entries,
+        total_entries: 253,
+        total_bytes: 253,
+        truncated: true,
+        next_after: None,
+    };
+    let mut wire = serde_json::to_value(Output::List(result)).unwrap();
+    assert!(serde_json::from_value::<ExternalWorkdirOperationResult>(wire.clone()).is_ok());
+    wire["result"]["next_after"] = serde_json::to_value(after).unwrap();
+    assert!(serde_json::from_value::<ExternalWorkdirOperationResult>(wire).is_err());
+}