Compare commits

...

66 Commits

Author SHA1 Message Date
3f7de349c3
chore: complete memory-extract-remove-input-cap ticket 2026-05-23 09:14:37 +09:00
1feb560ff9
merge: memory-extract-remove-input-cap 2026-05-23 09:14:15 +09:00
902083bd38
fix: remove memory extract input cap 2026-05-23 09:14:07 +09:00
221b1edd92
chore: complete tui-pod-restore-picker ticket 2026-05-23 09:13:57 +09:00
828004a5e2
merge: tui-pod-restore-picker 2026-05-23 09:13:19 +09:00
b26bc420f1
feat: restore tui sessions by pod 2026-05-23 09:13:06 +09:00
fb7abb1b7c
chore: complete spawned-delegation-scope-reclaim ticket 2026-05-23 08:39:04 +09:00
66996f902b
merge: spawned-delegation-scope-reclaim 2026-05-23 08:38:50 +09:00
d62cd09c4d
fix: reclaim delegated scope from stopped children 2026-05-23 08:38:42 +09:00
a4f03e7688
docs: refine pod visibility and tui restore flow 2026-05-23 08:33:00 +09:00
5ade50dec5
update: tui -rの際のリストの時系列ソート 2026-05-23 08:02:05 +09:00
dbb6cca894
chore: complete tui-streaming-input-loss ticket 2026-05-23 07:16:08 +09:00
7c9abb37ad
merge: tui-streaming-input-loss 2026-05-23 07:15:55 +09:00
0535260c8a
fix: preserve tui input during streaming 2026-05-23 07:15:39 +09:00
c435635e5b
chore: complete tui-context-usage-indicator ticket 2026-05-23 07:15:30 +09:00
c78cd28b27
merge: tui-context-usage-indicator 2026-05-23 07:15:17 +09:00
abe890cda5
feat: show context usage in tui status 2026-05-23 07:15:03 +09:00
fa04b03643
docs: identify tui streaming input loss race 2026-05-23 05:47:59 +09:00
1bebd8b6df
Create tui-parts.md 2026-05-23 05:41:48 +09:00
4dec7f916f
fix: tighten task tool usage guidance 2026-05-23 05:11:48 +09:00
8662ca404f
chore: complete prune-token-budget ticket 2026-05-23 05:00:30 +09:00
5f2efeb75e
merge: prune-token-budget 2026-05-23 05:00:15 +09:00
820dea1873
feat: protect prune tail by token budget 2026-05-23 05:00:06 +09:00
8d5ee0f0b8
chore: complete pod-event-callback-delivery ticket 2026-05-23 04:57:26 +09:00
0d39170bbe
merge: pod-event-callback-delivery 2026-05-23 04:57:10 +09:00
16963d15f2
fix: drain snapshots before pod callbacks 2026-05-23 04:57:03 +09:00
2cea02648f
docs: add memory extract input cap ticket 2026-05-23 04:42:38 +09:00
f55503edc3
docs: add pod event callback delivery ticket 2026-05-23 03:29:01 +09:00
d18e3a0256
docs: add spawned delegation scope reclaim ticket 2026-05-23 03:02:48 +09:00
5ba4be1c9b
refactor: remove legacy plural log entries 2026-05-23 02:03:42 +09:00
90d4c8f5ad
docs: track read pod output log entry bug 2026-05-23 00:53:47 +09:00
61c9719da5
docs: add pod discovery restore tools ticket 2026-05-23 00:09:34 +09:00
73fbdcc025
chore: complete spawned-registry-persist ticket 2026-05-22 23:30:16 +09:00
5fdc46db47
merge: spawned-registry-persist 2026-05-22 23:30:06 +09:00
534c6f4cac
feat: persist spawned pod registry 2026-05-22 23:30:02 +09:00
5540ca3d0e
chore: complete pod-name-resume ticket 2026-05-22 22:57:31 +09:00
edfdca3457
merge: pod-name-resume 2026-05-22 22:57:23 +09:00
bd32f704b4
feat: resume pods by name 2026-05-22 22:57:16 +09:00
e55fc9a834
chore: complete pod-state-write-points ticket 2026-05-22 22:29:23 +09:00
7f6e3b949f
merge: pod-state-write-points 2026-05-22 22:29:12 +09:00
0954a4804a
feat: wire pod metadata lifecycle writes 2026-05-22 22:29:08 +09:00
e9cc4b90dc
chore: complete pod-state-backend ticket 2026-05-22 22:03:36 +09:00
8aca67c97c
style: run cargo fmt 2026-05-22 22:03:27 +09:00
d7eabb18c9
merge: pod-state-backend 2026-05-22 22:03:17 +09:00
b13c2735bb
feat: add pod metadata store backend 2026-05-22 22:03:11 +09:00
67f135fbc6
Merge: live-fork-marker 2026-05-20 06:45:49 +09:00
a728b7045d
chore: 空になった Storage 親見出しを TODO から削除 2026-05-20 06:45:43 +09:00
3eabcb6a6d
ticket: live-fork-marker 完了 2026-05-20 06:45:19 +09:00
ffcba3aa54
chore: auto-fork ロジック二重実装を KNOWN_ISSUES に登録 2026-05-20 06:45:14 +09:00
eb752fb295
ticket: live-fork-marker レビュー (Approve) 2026-05-20 06:44:54 +09:00
ac3ee5fcbe
feat: live auto-fork の marker 形式を確定(seq 比較 + forked_from 記録)
方針: 末尾 entry-count 比較で検知し、元 Segment は immutable のまま
(terminal marker を書き戻さない)。fork lineage は新 Segment の
SegmentStart.forked_from に前向きに記録するため、log だけから辿れる。
過去 fork と対称で、nested fork も marker 位置の調停が不要。

- session-store ensure_head_or_fork に at_turn_index 引数を追加し
  新 Segment へ forked_from を記録
- pod ensure_segment_head の auto-fork も同様に forked_from を記録
  (at_turn_index = writer の現 turn_count)
- fork_at の doc に「元 Segment を mutate しない」invariant を明記
- test: nested past-fork が祖先を不変に保つ / Pod 並行 writer drift で
  auto-fork し forked_from を記録 / 元 Segment に marker が書かれない
2026-05-20 06:42:09 +09:00
46b0e20685
Merge: session-grouping-introduce 2026-05-20 06:29:48 +09:00
6a4ee37be8
ticket: session-grouping-introduce 完了 2026-05-20 06:29:43 +09:00
3f3ead3b71
update: session-grouping review follow-up
- PickerOutcome::Picked から未使用の session_id を除去(pod-cli が lookup_session_of で再解決)
- picker preview が singular AssistantItem も拾うように
- fs_store layout doc に migration(後方互換なし、旧 flat sessions は破棄)を明記
- TaskStore は Session-lifetime、ScopedFs/Tracker は Pod-process lifetime と用語整理
- Pod::session_id / from_manifest_spawned のコメント補強
2026-05-20 06:29:37 +09:00
a6cc05f74c
feat: Session(Segment 群の grouping)を導入
- SessionId 型を新設、各 SegmentStart に session_id を持たせる
- compaction / 内部 fork は同 SessionId を継承、fork() は新 Session を発行
- Store API を (SessionId, SegmentId) ベースに、FsStore layout は
  <root>/<session_id>/<segment_id>.jsonl に
- Store::list_sessions / list_segments(session_id) / lookup_session_of を追加
- restore_by_segment shim を session-store に提供(pod-cli --session で使用)
- SegmentState に SegmentLocation (session_id, segment_id) を保持し ArcSwap で更新
- RestoredState に session_id: Option<SessionId> を追加
- Picker は Session 単位に列挙、leaf segment を解決して resume
2026-05-20 06:17:56 +09:00
e4cda5d3f2
Merge: segment-rename 2026-05-20 05:18:11 +09:00
dd9abfee2e
ticket: segment-rename 完了 2026-05-20 05:18:04 +09:00
d7ff25b6a7
update: 残存 Session 識別子の Segment 化(review follow-up)
レビュー指摘の通り、次の session-grouping-introduce で新 SessionId が
入る前に名称衝突を避けるため取り残しを掃除。

- PodError::Session{Empty,ScopeMissing} → Segment{Empty,ScopeMissing}
- ScopeLockError::SessionConflict → SegmentConflict
- Pod.session_state / SegmentState.set_session_id 系
- source_session_id / prev_session_id / ensure_session_head / short_session
- pod_cli の "Session ID:" 表示
- fs_store の sessions ローカル変数
2026-05-20 05:17:49 +09:00
7577577c9f
update: Session-lifetime/scoped を Pod-lifetime に修正
タスクストア/ファイルトラッカーは compaction を跨いで Pod プロセス寿命まで生きる。
旧 SessionId = Segment の時代の表現を Pod-lifetime に正す。pod_cli の表示も Segment: に。
2026-05-20 05:06:38 +09:00
0d7c37f673
update: SessionId / SessionStart / SessionOrigin 等を Segment 系名称へ
- Type/Function/Variantを Segment* 系へ統一
  - SessionId/SessionStart/SessionOrigin/SessionStartState/SessionState/SessionLogSink/SessionLockInfo
  - new_session_id / session_id / create_session* / list_sessions / lookup_session / update_session / find_by_session
  - protocol Event::SessionRotated → SegmentRotated、CompactDone.new_session_id → new_segment_id
- Module: session_log → segment_log / session → segment (file mv 含む)
  pod 側の session_log_sink → segment_log_sink も同様
- crate 名 (session-store)、CLI flag (--session)、ResumeWithSession (CLI tied) は据え置き
- session-tests/session_metrics_test 等の Store impl も追従
2026-05-20 05:06:04 +09:00
d5c7330659
Merge: entry-hash-abolish 2026-05-20 04:53:52 +09:00
9c1f51b4f0
ticket: entry-hash-abolish 完了 2026-05-20 04:53:47 +09:00
1d8a490504
update: 旧用語コメントの掃除と KNOWN_ISSUES 追記
- 残存していた head_hash / SessionHead 言及コメントを 3 箇所更新
- FsStore::read_entry_count の O(n) 計測コストを KNOWN_ISSUES に登録
2026-05-20 04:53:33 +09:00
6e791d8668
ticket: entry-hash-abolish レビュー (Approve) 2026-05-20 04:49:17 +09:00
d5dff6d17b
update: entry hash chain と session_head mutex を撤廃
- HashedEntry / EntryHash / compute_hash / build_chain 撤去、JSONL は 1 行 1 LogEntry
- SessionOrigin.at_hash → at_turn_index (TurnEnd 由来) に置換
- Pod 側 SessionHead mutex を ArcSwap<SessionId> + AtomicUsize の SessionState に置換
- ensure_head_or_fork は store の entry count と writer の append tally で判定
- session-store から sha2 / hex 依存、pod から parking_lot 依存を削除
2026-05-20 04:31:37 +09:00
35c15923db
ticket: 永続化整理を 8 個に分割
persistence-semantics と pod-persistent-state を実装可能な粒度に分割。
Storage 層 (Phase 1) を entry-hash-abolish / segment-rename /
session-grouping-introduce / live-fork-marker に、Pod 単位永続化
(Phase 2) を pod-state-backend / pod-state-write-points /
pod-name-resume / spawned-registry-persist に切り出した。
2026-05-20 04:07:44 +09:00
102 changed files with 5440 additions and 2667 deletions

5
Cargo.lock generated
View File

@ -2148,6 +2148,7 @@ checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
name = "pod" name = "pod"
version = "0.1.0" version = "0.1.0"
dependencies = [ dependencies = [
"arc-swap",
"async-trait", "async-trait",
"chrono", "chrono",
"clap", "clap",
@ -2160,7 +2161,6 @@ dependencies = [
"manifest", "manifest",
"memory", "memory",
"minijinja", "minijinja",
"parking_lot",
"pod-registry", "pod-registry",
"protocol", "protocol",
"provider", "provider",
@ -2977,12 +2977,10 @@ version = "0.1.0"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"futures", "futures",
"hex",
"llm-worker", "llm-worker",
"protocol", "protocol",
"serde", "serde",
"serde_json", "serde_json",
"sha2 0.11.0",
"tempfile", "tempfile",
"thiserror 2.0.18", "thiserror 2.0.18",
"tokio", "tokio",
@ -3655,6 +3653,7 @@ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
"session-store", "session-store",
"tempfile",
"tokio", "tokio",
"toml", "toml",
"tools", "tools",

View File

@ -31,7 +31,6 @@ llm-worker-macros = { path = "crates/llm-worker-macros", version = "0.2" }
manifest = { path = "crates/manifest" } manifest = { path = "crates/manifest" }
lint-common = { path = "crates/lint-common" } lint-common = { path = "crates/lint-common" }
memory = { path = "crates/memory" } memory = { path = "crates/memory" }
workflow = { path = "crates/workflow" }
pod-registry = { path = "crates/pod-registry" } pod-registry = { path = "crates/pod-registry" }
protocol = { path = "crates/protocol" } protocol = { path = "crates/protocol" }
provider = { path = "crates/provider" } provider = { path = "crates/provider" }

View File

@ -14,3 +14,5 @@ Ticket を切るほどではないが、次に近所を触るときに合わせ
- `crates/tui/src/app.rs:478-485` — bad workflow slug を含む `Method::Run` 送信時、`Event::UserMessage` の早期 broadcast で `turn_index += 1` されターンヘッダだけ残る ("ghost turn header")。次に TUI のターンヘッダ / エラー表示周りを触るときに整理。→ [tickets/pod-input-validate-internalize.md] の review 由来。 - `crates/tui/src/app.rs:478-485` — bad workflow slug を含む `Method::Run` 送信時、`Event::UserMessage` の早期 broadcast で `turn_index += 1` されターンヘッダだけ残る ("ghost turn header")。次に TUI のターンヘッダ / エラー表示周りを触るときに整理。→ [tickets/pod-input-validate-internalize.md] の review 由来。
- `crates/pod/src/controller.rs:944``worker_error_code``PodError::WorkflowResolve(_) => InvalidRequest` が post-commit な resolve エラー (`KnowledgeNotFound` 等) にも適用される。意味論的には妥当方向だが、resolve 系のエラー粒度を分けたくなったタイミングで再評価。 - `crates/pod/src/controller.rs:944``worker_error_code``PodError::WorkflowResolve(_) => InvalidRequest` が post-commit な resolve エラー (`KnowledgeNotFound` 等) にも適用される。意味論的には妥当方向だが、resolve 系のエラー粒度を分けたくなったタイミングで再評価。
- `crates/pod/tests/controller_test.rs``double_run_returns_error` がたまに失敗する flakiness を観測。`pod-interrupt-prep-internalize` 以前から存在する別件。次に controller_test の Run 連投系のタイミングを触るときに併せて原因を切り分け。 - `crates/pod/tests/controller_test.rs``double_run_returns_error` がたまに失敗する flakiness を観測。`pod-interrupt-prep-internalize` 以前から存在する別件。次に controller_test の Run 連投系のタイミングを触るときに併せて原因を切り分け。
- `crates/session-store/src/fs_store.rs:117-122``FsStore::read_entry_count``fs::read_to_string` で全文ロードしてから行数カウントするため O(n)。`ensure_head_or_fork` は run-start でしか呼ばれず現状は許容範囲だが、長期セッションが普通になった時点で `\n` バイト数の cheap count か末尾 seek に置き換える。
- `crates/session-store/src/segment.rs:121` `ensure_head_or_fork` (free fn, test 専用・本番 caller ゼロ) と `crates/pod/src/pod.rs` `Pod::ensure_segment_head` (本番 inline) に live auto-fork の検知 + forked_from 記録が二重実装されている。entry-hash-abolish 以前からの重複で、両方独立にテスト済みだが drift 必至。session-store 側を本番から呼ぶ形に寄せるか free fn を畳むかは要設計判断。Pod state / fork 周辺を次に触るときに統合を検討。

View File

@ -7,8 +7,7 @@
- Pod: 空応答ターン (Submit 後 AI 応答ゼロで Pause/Cancel) を自動巻き戻し → [tickets/pod-empty-turn-rollback.md](tickets/pod-empty-turn-rollback.md) - Pod: 空応答ターン (Submit 後 AI 応答ゼロで Pause/Cancel) を自動巻き戻し → [tickets/pod-empty-turn-rollback.md](tickets/pod-empty-turn-rollback.md)
- Pod: 任意ターンからの Fork複数ターン巻き戻しを汎用化 → [tickets/pod-session-fork.md](tickets/pod-session-fork.md) - Pod: 任意ターンからの Fork複数ターン巻き戻しを汎用化 → [tickets/pod-session-fork.md](tickets/pod-session-fork.md)
- Pod: Inbound PodEvent ハンドリングの重複を統合 → [tickets/pod-inbound-pod-event-dedup.md](tickets/pod-inbound-pod-event-dedup.md) - Pod: Inbound PodEvent ハンドリングの重複を統合 → [tickets/pod-inbound-pod-event-dedup.md](tickets/pod-inbound-pod-event-dedup.md)
- Pod: セッションログをバックエンドにした Pod 単位の永続化 → [tickets/pod-persistent-state.md](tickets/pod-persistent-state.md) - Pod: 過去 Pod の探索と restore ツール → [tickets/pod-discovery-restore-tools.md](tickets/pod-discovery-restore-tools.md)
- 永続化層のセマンティック整理 → [tickets/persistence-semantics.md](tickets/persistence-semantics.md)
- llm-worker のエラー耐性 - llm-worker のエラー耐性
- ストリーム途中失敗時の継続 → [tickets/llm-worker-stream-continuation.md](tickets/llm-worker-stream-continuation.md) - ストリーム途中失敗時の継続 → [tickets/llm-worker-stream-continuation.md](tickets/llm-worker-stream-continuation.md)
- llm-worker: history append を callback 経由の単一経路に閉じる → [tickets/worker-history-append-contract.md](tickets/worker-history-append-contract.md) - llm-worker: history append を callback 経由の単一経路に閉じる → [tickets/worker-history-append-contract.md](tickets/worker-history-append-contract.md)
@ -21,8 +20,6 @@
- ユーザーマニフェストのモデル設定 wizard → [tickets/tui-user-model-setup.md](tickets/tui-user-model-setup.md) - ユーザーマニフェストのモデル設定 wizard → [tickets/tui-user-model-setup.md](tickets/tui-user-model-setup.md)
- spawn 失敗時に Pod の stderr が TUI に表示されない → [tickets/tui-spawn-error-surface.md](tickets/tui-spawn-error-surface.md) - spawn 失敗時に Pod の stderr が TUI に表示されない → [tickets/tui-spawn-error-surface.md](tickets/tui-spawn-error-surface.md)
- 巻き戻されたターンの入力テキストを編集領域に復元 → [tickets/tui-empty-turn-restore.md](tickets/tui-empty-turn-restore.md) - 巻き戻されたターンの入力テキストを編集領域に復元 → [tickets/tui-empty-turn-restore.md](tickets/tui-empty-turn-restore.md)
- セッションコンテキスト長 / ウィンドウ占有率の常時表示 → [tickets/tui-context-usage-indicator.md](tickets/tui-context-usage-indicator.md)
- Prune: 保護境界を turn 数から末尾 token budget に置き換え → [tickets/prune-token-budget.md](tickets/prune-token-budget.md)
- メモリ機構 - メモリ機構
- extract / consolidation 監査ログ → [tickets/memory-audit-log.md](tickets/memory-audit-log.md) - extract / consolidation 監査ログ → [tickets/memory-audit-log.md](tickets/memory-audit-log.md)
- セッション内 Task ツールの注意機構(無アクティビティで `<system-reminder>` ナッジ) → [tickets/session-todo-reminder.md](tickets/session-todo-reminder.md) - セッション内 Task ツールの注意機構(無アクティビティで `<system-reminder>` ナッジ) → [tickets/session-todo-reminder.md](tickets/session-todo-reminder.md)

View File

@ -34,6 +34,9 @@ pub struct SpawnConfig {
/// `Some(id)` のとき `--session <id>` を付与し、当該セッションから /// `Some(id)` のとき `--session <id>` を付与し、当該セッションから
/// resume させる。 /// resume させる。
pub resume_from: Option<Uuid>, pub resume_from: Option<Uuid>,
/// true のとき `--pod <pod_name>` を付与し、pod 側で name-keyed state
/// があれば resume、なければ同名の新規 Pod として起動させる。
pub resume_by_pod_name: bool,
} }
pub struct SpawnReady { pub struct SpawnReady {
@ -111,6 +114,9 @@ where
.stdout(Stdio::null()) .stdout(Stdio::null())
.stderr(Stdio::from(stderr_file)) .stderr(Stdio::from(stderr_file))
.process_group(0); .process_group(0);
if config.resume_by_pod_name {
command.arg("--pod").arg(&config.pod_name);
}
if let Some(id) = config.resume_from { if let Some(id) = config.resume_from {
command.arg("--session").arg(id.to_string()); command.arg("--session").arg(id.to_string());
} }

View File

@ -51,7 +51,7 @@ pub(crate) struct ResponsesRequest {
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub top_p: Option<f32>, pub top_p: Option<f32>,
/// 会話単位の安定キー。ChatGPT backend (codex-oauth) は明示キーが /// 会話単位の安定キー。ChatGPT backend (codex-oauth) は明示キーが
/// 無いとプロンプトキャッシュがほぼ効かない。pod 側は `SessionId` /// 無いとプロンプトキャッシュがほぼ効かない。pod 側は `SegmentId`
/// を渡す。`Request::cache_key` が `None` のときはキー自体を送らない。 /// を渡す。`Request::cache_key` が `None` のときはキー自体を送らない。
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub prompt_cache_key: Option<String>, pub prompt_cache_key: Option<String>,

View File

@ -492,7 +492,7 @@ pub struct Request {
/// 会話単位の安定キー。`prompt_cache_key` として送られる /// 会話単位の安定キー。`prompt_cache_key` として送られる
/// (OpenAI Responses)。ChatGPT backend (codex-oauth) は明示キーが /// (OpenAI Responses)。ChatGPT backend (codex-oauth) は明示キーが
/// 無いと org/project ハッシュ衝突でプロンプトキャッシュが /// 無いと org/project ハッシュ衝突でプロンプトキャッシュが
/// ほぼヒットしないため、pod 側で `SessionId` を渡す運用を想定。 /// ほぼヒットしないため、pod 側で `SegmentId` を渡す運用を想定。
/// `cache_anchor` と違い名前空間キーであり、`prefix anchor` とは /// `cache_anchor` と違い名前空間キーであり、`prefix anchor` とは
/// 別の概念。`cache_anchor` を読まない provider と同じく、 /// 別の概念。`cache_anchor` を読まない provider と同じく、
/// `prompt_cache_key` を持たない provider は無視する。 /// `prompt_cache_key` を持たない provider は無視する。

View File

@ -11,12 +11,23 @@
//! 射影の適用は上位層(`pod::prune_hook` 等)が LLM に送る一時コンテキスト //! 射影の適用は上位層(`pod::prune_hook` 等)が LLM に送る一時コンテキスト
//! に対してだけ行う。Worker の永続履歴は決して変更されない。 //! に対してだけ行う。Worker の永続履歴は決して変更されない。
//! //!
//! `min_savings` 判定や savings 推定もこの crate には置かず、上位層が //! 保護境界は末尾 token budget で決めるが、この crate は usage 履歴を
//! usage 履歴ベースのトークン会計と組み合わせて行う。 //! 所有しない。prefix ごとの token 推定値と savings 推定は上位層から
//! callback で注入される。
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::llm_client::types::Item; use crate::llm_client::types::Item;
use crate::token_counter::{EstimateSource, TokenEstimate};
/// Callback that returns token estimates for every prefix boundary of the
/// supplied request history.
///
/// The returned slice must have `history.len() + 1` entries where entry `i`
/// estimates the token count of `history[..i]`. Returning a malformed vector,
/// or estimates whose source is [`EstimateSource::NoData`], makes prune treat
/// the request as having no candidates.
pub type TokenEstimator = Box<dyn Fn(&[Item]) -> Vec<TokenEstimate> + Send + Sync>;
/// Callback that estimates the token savings for projecting the /// Callback that estimates the token savings for projecting the
/// `ToolResult.content` out of `history[i]` for each `i` in `indices`. /// `ToolResult.content` out of `history[i]` for each `i` in `indices`.
@ -35,16 +46,16 @@ pub type SavingsEstimator = Box<dyn Fn(&[Item], &[usize]) -> u64 + Send + Sync>;
/// ///
/// Worker は LLM リクエストごとに 1 回 prune の評価をし、その結果を /// Worker は LLM リクエストごとに 1 回 prune の評価をし、その結果を
/// observer が登録されていればこの値で通知する。fire/skip の判定 /// observer が登録されていればこの値で通知する。fire/skip の判定
/// 結果と、判定材料になった候補数 / 推定 savings / 境界ターン位置を持つ。 /// 結果と、判定材料になった候補数 / 推定 savings / 保護領域の先頭 index を持つ。
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct PruneEvaluation { pub struct PruneEvaluation {
/// `prunable_indices` の長さ。`Skipped::NoCandidates` の時は 0。 /// `prunable_indices` の長さ。`Skipped::NoCandidates` の時は 0。
pub candidate_count: usize, pub candidate_count: usize,
/// 推定された savings (tokens)。`NoCandidates` の時は 0。 /// 推定された savings (tokens)。`NoCandidates` の時は 0。
pub estimated_savings: u64, pub estimated_savings: u64,
/// `protected_turns` 境界に当たる turn-start アイテムの index。 /// Token budget で保護される suffix の先頭 item index。
/// turn 数が `protected_turns` 以下で境界が決まらない場合は `None`。 /// usage 推定が `NoData` で境界が決まらない場合は `None`。
pub border_turn: Option<usize>, pub protected_start_index: Option<usize>,
/// 判定結果。 /// 判定結果。
pub decision: PruneDecision, pub decision: PruneDecision,
} }
@ -70,10 +81,9 @@ pub type PruneObserver = Box<dyn Fn(&PruneEvaluation) + Send + Sync>;
/// Configuration for the Prune algorithm. /// Configuration for the Prune algorithm.
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PruneConfig { pub struct PruneConfig {
/// Number of recent turns to protect from pruning. /// Token budget at the history tail protected from pruning.
/// A "turn" starts at each user message. #[serde(default = "default_protected_tokens")]
#[serde(default = "default_protected_turns")] pub protected_tokens: u64,
pub protected_turns: usize,
/// Minimum token savings required to actually prune. If the prunable /// Minimum token savings required to actually prune. If the prunable
/// content is smaller than this, the caller should skip to avoid /// content is smaller than this, the caller should skip to avoid
@ -84,8 +94,8 @@ pub struct PruneConfig {
pub min_savings: u64, pub min_savings: u64,
} }
fn default_protected_turns() -> usize { fn default_protected_tokens() -> u64 {
3 8000
} }
fn default_min_savings() -> u64 { fn default_min_savings() -> u64 {
4096 4096
@ -94,25 +104,12 @@ fn default_min_savings() -> u64 {
impl Default for PruneConfig { impl Default for PruneConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
protected_turns: default_protected_turns(), protected_tokens: default_protected_tokens(),
min_savings: default_min_savings(), min_savings: default_min_savings(),
} }
} }
} }
/// Find indices where each "turn" begins.
///
/// A turn starts at every user message. Returns the indices of those
/// user messages in ascending order.
fn find_turn_starts(items: &[Item]) -> Vec<usize> {
items
.iter()
.enumerate()
.filter(|(_, item)| item.is_user_message())
.map(|(i, _)| i)
.collect()
}
/// Set `content = None` on each `Item::ToolResult` at the given indices. /// Set `content = None` on each `Item::ToolResult` at the given indices.
/// ///
/// Returns the number of items that were actually modified — items that /// Returns the number of items that were actually modified — items that
@ -121,36 +118,43 @@ fn find_turn_starts(items: &[Item]) -> Vec<usize> {
pub fn project(items: &mut [Item], indices: &[usize]) -> usize { pub fn project(items: &mut [Item], indices: &[usize]) -> usize {
let mut count = 0; let mut count = 0;
for &i in indices { for &i in indices {
if let Item::ToolResult { content, .. } = &mut items[i] { if let Item::ToolResult { content, .. } = &mut items[i]
if content.is_some() { && content.is_some()
*content = None; {
count += 1; *content = None;
} count += 1;
} }
} }
count count
} }
/// Indices of `Item::ToolResult { content: Some(_), .. }` that lie outside /// Indices of `Item::ToolResult { content: Some(_), .. }` that lie before
/// the last `protected_turns` turns. Pure: does not mutate `items`. /// the suffix protected by `protected_tokens`. Pure: does not mutate `items`.
/// ///
/// Returns an empty vector when there are too few turns or no prunable /// Returns an empty vector when token estimates are unavailable (`NoData`) or
/// candidates. /// no prunable candidates exist.
pub fn prunable_indices(items: &[Item], protected_turns: usize) -> Vec<usize> { pub fn prunable_indices(
evaluate_candidates(items, protected_turns).0 items: &[Item],
protected_tokens: u64,
token_estimates: &[TokenEstimate],
) -> Vec<usize> {
evaluate_candidates(items, protected_tokens, token_estimates).0
} }
/// Same as [`prunable_indices`] but also returns the index of the /// Same as [`prunable_indices`] but also returns the start index of the
/// `protected_turns` boundary (the turn-start item whose tail is /// protected suffix. `None` means the token boundary could not be determined
/// protected). `None` when too few turns exist for a boundary to be /// (currently because usage estimates were `NoData` or malformed).
/// defined. pub fn evaluate_candidates(
pub fn evaluate_candidates(items: &[Item], protected_turns: usize) -> (Vec<usize>, Option<usize>) { items: &[Item],
let turn_starts = find_turn_starts(items); protected_tokens: u64,
if turn_starts.len() <= protected_turns { token_estimates: &[TokenEstimate],
) -> (Vec<usize>, Option<usize>) {
let Some(protected_start) = protected_start_index(items, protected_tokens, token_estimates)
else {
return (Vec::new(), None); return (Vec::new(), None);
} };
let boundary = turn_starts[turn_starts.len() - protected_turns];
let candidates = items[..boundary] let candidates = items[..protected_start]
.iter() .iter()
.enumerate() .enumerate()
.filter_map(|(i, item)| match item { .filter_map(|(i, item)| match item {
@ -160,7 +164,38 @@ pub fn evaluate_candidates(items: &[Item], protected_turns: usize) -> (Vec<usize
_ => None, _ => None,
}) })
.collect(); .collect();
(candidates, Some(boundary)) (candidates, Some(protected_start))
}
fn protected_start_index(
items: &[Item],
protected_tokens: u64,
token_estimates: &[TokenEstimate],
) -> Option<usize> {
if token_estimates.len() != items.len() + 1 {
return None;
}
let total = token_estimates[items.len()];
if total.source == EstimateSource::NoData {
return None;
}
if protected_tokens == 0 {
return Some(items.len());
}
let mut protected_start = items.len();
for idx in (0..items.len()).rev() {
let prefix = token_estimates[idx];
if prefix.source == EstimateSource::NoData {
return None;
}
protected_start = idx;
let tail_tokens = total.tokens.saturating_sub(prefix.tokens);
if tail_tokens >= protected_tokens {
break;
}
}
Some(protected_start)
} }
#[cfg(test)] #[cfg(test)]
@ -185,17 +220,70 @@ mod tests {
items items
} }
fn measured_prefix(tokens: &[u64]) -> Vec<TokenEstimate> {
tokens
.iter()
.copied()
.map(|tokens| TokenEstimate {
tokens,
source: EstimateSource::Measured,
})
.collect()
}
fn uniform_estimates(items: &[Item], item_tokens: u64) -> Vec<TokenEstimate> {
let mut tokens = Vec::with_capacity(items.len() + 1);
for i in 0..=items.len() {
tokens.push(i as u64 * item_tokens);
}
measured_prefix(&tokens)
}
fn estimates_from_item_tokens(item_tokens: &[u64]) -> Vec<TokenEstimate> {
let mut prefix = Vec::with_capacity(item_tokens.len() + 1);
let mut acc = 0;
prefix.push(acc);
for tokens in item_tokens {
acc += tokens;
prefix.push(acc);
}
measured_prefix(&prefix)
}
fn no_data_estimates(items: &[Item]) -> Vec<TokenEstimate> {
(0..=items.len())
.map(|i| TokenEstimate {
tokens: i as u64,
source: if i == 0 {
EstimateSource::Measured
} else {
EstimateSource::NoData
},
})
.collect()
}
#[test] #[test]
fn no_candidates_when_too_few_turns() { fn no_candidates_when_estimate_has_no_data() {
let items = make_history(&[("turn1", vec![("summary1", Some("big content here"))])]);
let estimates = no_data_estimates(&items);
let (candidates, protected_start) = evaluate_candidates(&items, 10, &estimates);
assert!(candidates.is_empty());
assert_eq!(protected_start, None);
}
#[test]
fn no_candidates_when_history_fits_in_protected_tokens() {
let items = make_history(&[ let items = make_history(&[
("turn1", vec![("summary1", Some("big content here"))]), ("turn1", vec![("summary1", Some("big content here"))]),
("turn2", vec![("summary2", Some("more content"))]), ("turn2", vec![("summary2", Some("more content"))]),
]); ]);
assert!(prunable_indices(&items, 3).is_empty()); let estimates = uniform_estimates(&items, 10);
assert!(prunable_indices(&items, 10_000, &estimates).is_empty());
} }
#[test] #[test]
fn candidates_in_unprotected_turns() { fn candidates_before_token_protected_suffix() {
let big = "x".repeat(4096 * 4); let big = "x".repeat(4096 * 4);
let items = make_history(&[ let items = make_history(&[
("turn1", vec![("s1", Some(&big))]), ("turn1", vec![("s1", Some(&big))]),
@ -203,9 +291,39 @@ mod tests {
("turn3", vec![("s3", Some("keep me"))]), ("turn3", vec![("s3", Some("keep me"))]),
("turn4", vec![("s4", Some("keep me too"))]), ("turn4", vec![("s4", Some("keep me too"))]),
]); ]);
let candidates = prunable_indices(&items, 2); let estimates = uniform_estimates(&items, 10);
let candidates = prunable_indices(&items, 80, &estimates);
assert_eq!(candidates.len(), 2);
// suffix budget 80 tokens protects turn3+turn4 (8 items), so only s1/s2 are candidates.
for &i in &candidates {
if let Item::ToolResult { summary, .. } = &items[i] {
assert!(summary == "s1" || summary == "s2");
} else {
panic!("non tool-result selected");
}
}
}
#[test]
fn single_long_task_gets_candidates_without_multiple_user_turns() {
let big = "x".repeat(4096 * 8);
let items = make_history(&[(
"one long task",
vec![
("s1", Some(&big)),
("s2", Some(&big)),
("s3", Some(&big)),
("s4", Some(&big)),
],
)]);
// user + assistant are cheap; every ToolCall is cheap; every ToolResult is heavy.
let item_tokens = vec![1, 1, 1, 5_000, 1, 5_000, 1, 5_000, 1, 5_000];
let estimates = estimates_from_item_tokens(&item_tokens);
let (candidates, protected_start) = evaluate_candidates(&items, 8_000, &estimates);
assert_eq!(protected_start, Some(7));
assert_eq!(candidates.len(), 2); assert_eq!(candidates.len(), 2);
// 候補は turn1 と turn2 の ToolResult のみ
for &i in &candidates { for &i in &candidates {
if let Item::ToolResult { summary, .. } = &items[i] { if let Item::ToolResult { summary, .. } = &items[i] {
assert!(summary == "s1" || summary == "s2"); assert!(summary == "s1" || summary == "s2");
@ -223,7 +341,8 @@ mod tests {
("turn3", vec![]), ("turn3", vec![]),
("turn4", vec![]), ("turn4", vec![]),
]); ]);
assert!(prunable_indices(&items, 2).is_empty()); let estimates = uniform_estimates(&items, 10);
assert!(prunable_indices(&items, 20, &estimates).is_empty());
} }
#[test] #[test]
@ -235,7 +354,8 @@ mod tests {
("turn3", vec![("s3", Some("keep me"))]), ("turn3", vec![("s3", Some("keep me"))]),
("turn4", vec![("s4", Some("keep me too"))]), ("turn4", vec![("s4", Some("keep me too"))]),
]); ]);
let candidates = prunable_indices(&items, 2); let estimates = uniform_estimates(&items, 10);
let candidates = prunable_indices(&items, 80, &estimates);
let count = project(&mut items, &candidates); let count = project(&mut items, &candidates);
assert_eq!(count, 2); assert_eq!(count, 2);
@ -261,7 +381,7 @@ mod tests {
("turn1", vec![("s1", None)]), ("turn1", vec![("s1", None)]),
("turn2", vec![("s2", Some("hello"))]), ("turn2", vec![("s2", Some("hello"))]),
]); ]);
// Manually target s1 (index 3) even though it's already None. // Manually target s1 even though it's already None.
let target = items let target = items
.iter() .iter()
.position(|it| matches!(it, Item::ToolResult { summary, .. } if summary == "s1")) .position(|it| matches!(it, Item::ToolResult { summary, .. } if summary == "s1"))
@ -279,14 +399,15 @@ mod tests {
("turn3", vec![]), ("turn3", vec![]),
("turn4", vec![]), ("turn4", vec![]),
]); ]);
let candidates = prunable_indices(&items, 2); let estimates = uniform_estimates(&items, 10);
let candidates = prunable_indices(&items, 20, &estimates);
assert_eq!(project(&mut items, &candidates), 1); assert_eq!(project(&mut items, &candidates), 1);
// 2 周目: 候補は一度の prunable_indices 結果を使い回しても 0 件。 // 2 周目: 候補は一度の prunable_indices 結果を使い回しても 0 件。
assert_eq!(project(&mut items, &candidates), 0); assert_eq!(project(&mut items, &candidates), 0);
} }
#[test] #[test]
fn evaluate_candidates_returns_boundary_index() { fn evaluate_candidates_returns_protected_start_index() {
let big = "x".repeat(64); let big = "x".repeat(64);
let items = make_history(&[ let items = make_history(&[
("turn1", vec![("s1", Some(&big))]), ("turn1", vec![("s1", Some(&big))]),
@ -294,36 +415,37 @@ mod tests {
("turn3", vec![("s3", Some("keep"))]), ("turn3", vec![("s3", Some("keep"))]),
("turn4", vec![("s4", Some("keep too"))]), ("turn4", vec![("s4", Some("keep too"))]),
]); ]);
let (candidates, border) = evaluate_candidates(&items, 2); let estimates = uniform_estimates(&items, 10);
let (candidates, protected_start) = evaluate_candidates(&items, 80, &estimates);
assert_eq!(candidates.len(), 2); assert_eq!(candidates.len(), 2);
// protected_turns=2 → boundary は turn3 の user message 位置。 // protected_tokens=80 → protected suffix is turn3+turn4, starting at index 8.
// turn1: u/a/c/r (4) + turn2: u/a/c/r (4) = index 8 (turn3 の user)。 assert_eq!(protected_start, Some(8));
assert_eq!(border, Some(8));
} }
#[test] #[test]
fn evaluate_candidates_no_boundary_when_too_few_turns() { fn evaluate_candidates_reports_zero_start_when_everything_is_protected() {
let items = make_history(&[("only", vec![("s", Some("x"))])]); let items = make_history(&[("only", vec![("s", Some("x"))])]);
let (candidates, border) = evaluate_candidates(&items, 2); let estimates = uniform_estimates(&items, 10);
let (candidates, protected_start) = evaluate_candidates(&items, 10_000, &estimates);
assert!(candidates.is_empty()); assert!(candidates.is_empty());
assert!(border.is_none()); assert_eq!(protected_start, Some(0));
} }
#[test] #[test]
fn protected_turns_boundary_exact() { fn zero_protected_tokens_allows_all_tool_results_as_candidates() {
// 3 turns with protected_turns=2: only turn 1 is a candidate.
let big = "x".repeat(64); let big = "x".repeat(64);
let items = make_history(&[ let items = make_history(&[("turn1", vec![("s1", Some(&big)), ("s2", Some(&big))])]);
("turn1", vec![("s1", Some(&big))]), let estimates = uniform_estimates(&items, 10);
("turn2", vec![("s2", Some("protected"))]), let (candidates, protected_start) = evaluate_candidates(&items, 0, &estimates);
("turn3", vec![("s3", Some("also protected"))]), assert_eq!(protected_start, Some(items.len()));
]); assert_eq!(candidates.len(), 2);
let candidates = prunable_indices(&items, 2); }
assert_eq!(candidates.len(), 1);
if let Item::ToolResult { summary, .. } = &items[candidates[0]] { #[test]
assert_eq!(summary, "s1"); fn malformed_estimate_vector_is_treated_as_no_boundary() {
} else { let items = make_history(&[("turn1", vec![("s1", Some("x"))])]);
panic!("expected ToolResult at candidate index"); let (candidates, protected_start) = evaluate_candidates(&items, 10, &[]);
} assert!(candidates.is_empty());
assert_eq!(protected_start, None);
} }
} }

View File

@ -201,6 +201,10 @@ pub struct Worker<C: LlmClient, S: WorkerState = Mutable> {
tool_output_limits: Option<ToolOutputLimits>, tool_output_limits: Option<ToolOutputLimits>,
/// Prune configuration. `None` disables the prune projection. /// Prune configuration. `None` disables the prune projection.
prune_config: Option<crate::prune::PruneConfig>, prune_config: Option<crate::prune::PruneConfig>,
/// Callback that estimates prefix token counts, injected by higher
/// layers that own usage measurements. `None` disables the prune
/// projection.
token_estimator: Option<crate::prune::TokenEstimator>,
/// Callback that estimates token savings for a drop range, injected /// Callback that estimates token savings for a drop range, injected
/// by higher layers that own usage measurements. `None` disables /// by higher layers that own usage measurements. `None` disables
/// the prune projection. /// the prune projection.
@ -213,7 +217,7 @@ pub struct Worker<C: LlmClient, S: WorkerState = Mutable> {
cache_anchor: Option<usize>, cache_anchor: Option<usize>,
/// Conversation-scoped cache key, set by higher layers. Plumbed into /// Conversation-scoped cache key, set by higher layers. Plumbed into
/// [`Request::cache_key`] at request build time. Pod 側では /// [`Request::cache_key`] at request build time. Pod 側では
/// `SessionId` を渡す。 /// `SegmentId` を渡す。
cache_key: Option<String>, cache_key: Option<String>,
/// State marker /// State marker
_state: PhantomData<S>, _state: PhantomData<S>,
@ -434,6 +438,17 @@ impl<C: LlmClient, S: WorkerState> Worker<C, S> {
self.prune_config = config; self.prune_config = config;
} }
/// Inject the callback used to estimate prefix token counts for prune's
/// protected-token boundary.
///
/// The callback is invoked with the *request context* (a clone of
/// history). It must be pure/idempotent since it may be called once per
/// LLM request. Returning `NoData` estimates makes prune skip as if no
/// candidates existed.
pub fn set_token_estimator(&mut self, estimator: Option<crate::prune::TokenEstimator>) {
self.token_estimator = estimator;
}
/// Inject the callback used to estimate token savings for a prune /// Inject the callback used to estimate token savings for a prune
/// candidate range. /// candidate range.
/// ///
@ -983,18 +998,26 @@ impl<C: LlmClient, S: WorkerState> Worker<C, S> {
// prunable candidates whose estimated savings meet the // prunable candidates whose estimated savings meet the
// threshold. Worker does not own usage history itself; the // threshold. Worker does not own usage history itself; the
// estimator is injected by the layer that does. // estimator is injected by the layer that does.
if let (Some(config), Some(estimator)) = (&self.prune_config, &self.savings_estimator) { if let (Some(config), Some(token_estimator), Some(savings_estimator)) = (
let (candidates, border_turn) = &self.prune_config,
crate::prune::evaluate_candidates(&request_context, config.protected_turns); &self.token_estimator,
&self.savings_estimator,
) {
let token_estimates = token_estimator(&request_context);
let (candidates, protected_start_index) = crate::prune::evaluate_candidates(
&request_context,
config.protected_tokens,
&token_estimates,
);
let evaluation = if candidates.is_empty() { let evaluation = if candidates.is_empty() {
crate::prune::PruneEvaluation { crate::prune::PruneEvaluation {
candidate_count: 0, candidate_count: 0,
estimated_savings: 0, estimated_savings: 0,
border_turn, protected_start_index,
decision: crate::prune::PruneDecision::SkippedNoCandidates, decision: crate::prune::PruneDecision::SkippedNoCandidates,
} }
} else { } else {
let savings = estimator(&request_context, &candidates); let savings = savings_estimator(&request_context, &candidates);
if savings >= config.min_savings { if savings >= config.min_savings {
let pruned = crate::prune::project(&mut request_context, &candidates); let pruned = crate::prune::project(&mut request_context, &candidates);
if pruned > 0 { if pruned > 0 {
@ -1007,7 +1030,7 @@ impl<C: LlmClient, S: WorkerState> Worker<C, S> {
crate::prune::PruneEvaluation { crate::prune::PruneEvaluation {
candidate_count: candidates.len(), candidate_count: candidates.len(),
estimated_savings: savings, estimated_savings: savings,
border_turn, protected_start_index,
decision: crate::prune::PruneDecision::Fired { decision: crate::prune::PruneDecision::Fired {
pruned_count: pruned, pruned_count: pruned,
}, },
@ -1016,7 +1039,7 @@ impl<C: LlmClient, S: WorkerState> Worker<C, S> {
crate::prune::PruneEvaluation { crate::prune::PruneEvaluation {
candidate_count: candidates.len(), candidate_count: candidates.len(),
estimated_savings: savings, estimated_savings: savings,
border_turn, protected_start_index,
decision: crate::prune::PruneDecision::SkippedBelowMinSavings, decision: crate::prune::PruneDecision::SkippedBelowMinSavings,
} }
} }
@ -1256,6 +1279,7 @@ impl<C: LlmClient> Worker<C, Mutable> {
cancel_rx, cancel_rx,
tool_output_limits: None, tool_output_limits: None,
prune_config: None, prune_config: None,
token_estimator: None,
savings_estimator: None, savings_estimator: None,
prune_observer: None, prune_observer: None,
cache_anchor: None, cache_anchor: None,
@ -1519,6 +1543,7 @@ impl<C: LlmClient> Worker<C, Mutable> {
cancel_rx: self.cancel_rx, cancel_rx: self.cancel_rx,
tool_output_limits: self.tool_output_limits, tool_output_limits: self.tool_output_limits,
prune_config: self.prune_config, prune_config: self.prune_config,
token_estimator: self.token_estimator,
savings_estimator: self.savings_estimator, savings_estimator: self.savings_estimator,
prune_observer: self.prune_observer, prune_observer: self.prune_observer,
cache_anchor: self.cache_anchor, cache_anchor: self.cache_anchor,
@ -1605,6 +1630,7 @@ impl<C: LlmClient> Worker<C, Locked> {
cancel_rx: self.cancel_rx, cancel_rx: self.cancel_rx,
tool_output_limits: self.tool_output_limits, tool_output_limits: self.tool_output_limits,
prune_config: self.prune_config, prune_config: self.prune_config,
token_estimator: self.token_estimator,
savings_estimator: self.savings_estimator, savings_estimator: self.savings_estimator,
prune_observer: self.prune_observer, prune_observer: self.prune_observer,
cache_anchor: self.cache_anchor, cache_anchor: self.cache_anchor,

View File

@ -10,6 +10,7 @@ use std::collections::HashMap;
use std::num::NonZeroU32; use std::num::NonZeroU32;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use serde::de::Error as _;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::defaults; use crate::defaults;
@ -112,7 +113,7 @@ pub struct PermissionConfigPartial {
#[derive(Debug, Clone, Default, Serialize, Deserialize)] #[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct CompactionConfigPartial { pub struct CompactionConfigPartial {
#[serde(default)] #[serde(default)]
pub prune_protected_turns: Option<usize>, pub prune_protected_tokens: Option<u64>,
#[serde(default)] #[serde(default)]
pub prune_min_savings: Option<u64>, pub prune_min_savings: Option<u64>,
#[serde(default)] #[serde(default)]
@ -141,12 +142,40 @@ pub enum ResolveError {
RelativePath { field: &'static str, path: PathBuf }, RelativePath { field: &'static str, path: PathBuf },
} }
/// Reject manifest fields that were intentionally removed and must not be
/// silently swallowed by the general warn-and-ignore unknown-field policy.
pub(crate) fn reject_removed_manifest_fields(s: &str) -> Result<(), toml::de::Error> {
let value: toml::Value = toml::from_str(s)?;
if value
.get("compaction")
.and_then(toml::Value::as_table)
.is_some_and(|table| table.contains_key("prune_protected_turns"))
{
return Err(toml::de::Error::custom(
"unknown field in manifest: compaction.prune_protected_turns \
(removed; use compaction.prune_protected_tokens)",
));
}
if value
.get("memory")
.and_then(toml::Value::as_table)
.is_some_and(|table| table.contains_key("extract_worker_max_input_tokens"))
{
return Err(toml::de::Error::custom(
"unknown field in manifest: memory.extract_worker_max_input_tokens (removed)",
));
}
Ok(())
}
impl PodManifestConfig { impl PodManifestConfig {
/// Parse a partial manifest from a TOML string. Unknown top-level or /// Parse a partial manifest from a TOML string. Unknown top-level or
/// nested fields emit a `tracing::warn!` and are ignored; use /// nested fields emit a `tracing::warn!` and are ignored; use
/// `tracing_subscriber` with `WARN` enabled to surface them to the /// `tracing_subscriber` with `WARN` enabled to surface them to the
/// operator. /// operator. Removed fields that must not be silently ignored (currently
/// `compaction.prune_protected_turns`) are rejected before deserialization.
pub fn from_toml(s: &str) -> Result<Self, toml::de::Error> { pub fn from_toml(s: &str) -> Result<Self, toml::de::Error> {
reject_removed_manifest_fields(s)?;
let de = toml::Deserializer::parse(s)?; let de = toml::Deserializer::parse(s)?;
serde_ignored::deserialize(de, |path| { serde_ignored::deserialize(de, |path| {
tracing::warn!("unknown field in manifest: {}", path); tracing::warn!("unknown field in manifest: {}", path);
@ -263,9 +292,6 @@ impl MemoryConfig {
language: upper.language.or(self.language), language: upper.language.or(self.language),
extract_model: upper.extract_model.or(self.extract_model), extract_model: upper.extract_model.or(self.extract_model),
extract_threshold: upper.extract_threshold.or(self.extract_threshold), extract_threshold: upper.extract_threshold.or(self.extract_threshold),
extract_worker_max_input_tokens: upper
.extract_worker_max_input_tokens
.or(self.extract_worker_max_input_tokens),
extract_worker_max_turns: upper extract_worker_max_turns: upper
.extract_worker_max_turns .extract_worker_max_turns
.or(self.extract_worker_max_turns), .or(self.extract_worker_max_turns),
@ -339,7 +365,7 @@ impl PermissionConfigPartial {
impl CompactionConfigPartial { impl CompactionConfigPartial {
fn merge(self, upper: Self) -> Self { fn merge(self, upper: Self) -> Self {
Self { Self {
prune_protected_turns: upper.prune_protected_turns.or(self.prune_protected_turns), prune_protected_tokens: upper.prune_protected_tokens.or(self.prune_protected_tokens),
prune_min_savings: upper.prune_min_savings.or(self.prune_min_savings), prune_min_savings: upper.prune_min_savings.or(self.prune_min_savings),
compact_threshold: upper.compact_threshold.or(self.compact_threshold), compact_threshold: upper.compact_threshold.or(self.compact_threshold),
compact_request_threshold: upper compact_request_threshold: upper
@ -489,9 +515,9 @@ impl TryFrom<PodManifestConfig> for PodManifest {
validate_model_paths(cm, "compaction.model.auth.file")?; validate_model_paths(cm, "compaction.model.auth.file")?;
} }
Ok(CompactionConfig { Ok(CompactionConfig {
prune_protected_turns: c prune_protected_tokens: c
.prune_protected_turns .prune_protected_tokens
.unwrap_or(defaults::PRUNE_PROTECTED_TURNS), .unwrap_or(defaults::PRUNE_PROTECTED_TOKENS),
prune_min_savings: c.prune_min_savings.unwrap_or(defaults::PRUNE_MIN_SAVINGS), prune_min_savings: c.prune_min_savings.unwrap_or(defaults::PRUNE_MIN_SAVINGS),
compact_threshold: c.compact_threshold, compact_threshold: c.compact_threshold,
compact_request_threshold: c.compact_request_threshold, compact_request_threshold: c.compact_request_threshold,
@ -921,7 +947,7 @@ mod tests {
let lower = PodManifestConfig { let lower = PodManifestConfig {
compaction: Some(CompactionConfigPartial { compaction: Some(CompactionConfigPartial {
compact_threshold: Some(50_000), compact_threshold: Some(50_000),
prune_protected_turns: Some(5), prune_protected_tokens: Some(5_000),
..Default::default() ..Default::default()
}), }),
..Default::default() ..Default::default()
@ -937,7 +963,7 @@ mod tests {
let c = merged.compaction.unwrap(); let c = merged.compaction.unwrap();
assert_eq!(c.compact_threshold, Some(80_000)); assert_eq!(c.compact_threshold, Some(80_000));
// field from lower retained when upper has None // field from lower retained when upper has None
assert_eq!(c.prune_protected_turns, Some(5)); assert_eq!(c.prune_protected_tokens, Some(5_000));
} }
#[test] #[test]
@ -971,6 +997,45 @@ unknown_future_field = "tolerated"
assert_eq!(cfg.worker.max_tokens, Some(1000)); assert_eq!(cfg.worker.max_tokens, Some(1000));
} }
#[test]
fn from_toml_rejects_removed_prune_protected_turns_field() {
let bad = r#"
[compaction]
prune_protected_turns = 3
"#;
let err = PodManifestConfig::from_toml(bad).unwrap_err();
assert!(
err.to_string().contains("compaction.prune_protected_turns"),
"unexpected error: {err}"
);
}
#[test]
fn from_toml_rejects_removed_extract_worker_max_input_tokens_field() {
let bad = r#"
[memory]
extract_worker_max_input_tokens = 30000
"#;
let err = PodManifestConfig::from_toml(bad).unwrap_err();
assert!(
err.to_string()
.contains("memory.extract_worker_max_input_tokens"),
"unexpected error: {err}"
);
}
#[test]
fn from_toml_accepts_extract_worker_max_turns() {
let cfg = PodManifestConfig::from_toml(
r#"
[memory]
extract_worker_max_turns = 2
"#,
)
.unwrap();
assert_eq!(cfg.memory.unwrap().extract_worker_max_turns, Some(2));
}
#[test] #[test]
fn from_toml_accepts_worker_reasoning_string_or_integer() { fn from_toml_accepts_worker_reasoning_string_or_integer() {
let effort = PodManifestConfig::from_toml( let effort = PodManifestConfig::from_toml(

View File

@ -14,9 +14,9 @@ pub const TOOL_OUTPUT_MAX_BYTES: usize = 64 * 1024;
/// See [`crate::FileUploadLimits`]. /// See [`crate::FileUploadLimits`].
pub const FILE_UPLOAD_MAX_BYTES: usize = 256 * 1024; pub const FILE_UPLOAD_MAX_BYTES: usize = 256 * 1024;
/// Number of most-recent turns protected from pruning. See /// Token budget at the history tail protected from pruning. See
/// [`crate::CompactionConfig::prune_protected_turns`]. /// [`crate::CompactionConfig::prune_protected_tokens`].
pub const PRUNE_PROTECTED_TURNS: usize = 3; pub const PRUNE_PROTECTED_TOKENS: u64 = 8000;
/// Minimum estimated token savings required to trigger a prune. See /// Minimum estimated token savings required to trigger a prune. See
/// [`crate::CompactionConfig::prune_min_savings`]. /// [`crate::CompactionConfig::prune_min_savings`].
@ -59,11 +59,6 @@ pub const COMPACT_WORKER_MAX_TURNS: Option<u32> = Some(20);
/// default references. /// default references.
pub const COMPACT_DEFAULT_REFERENCE_COUNT: usize = 5; pub const COMPACT_DEFAULT_REFERENCE_COUNT: usize = 5;
/// Current prompt-occupancy cap for the memory extract worker's own
/// LLM requests. Exceeding this aborts the extract run (circuit-breaker
/// path). See [`crate::MemoryConfig::extract_worker_max_input_tokens`].
pub const MEMORY_EXTRACT_WORKER_MAX_INPUT_TOKENS: u64 = 30_000;
/// Optional maximum extract-worker tool-loop depth. `None` means unlimited. /// Optional maximum extract-worker tool-loop depth. `None` means unlimited.
/// See [`crate::MemoryConfig::extract_worker_max_turns`]. /// See [`crate::MemoryConfig::extract_worker_max_turns`].
pub const MEMORY_EXTRACT_WORKER_MAX_TURNS: Option<u32> = Some(8); pub const MEMORY_EXTRACT_WORKER_MAX_TURNS: Option<u32> = Some(8);

View File

@ -114,11 +114,6 @@ pub struct MemoryConfig {
/// the auto-extract trigger is dormant. /// the auto-extract trigger is dormant.
#[serde(default)] #[serde(default)]
pub extract_threshold: Option<u64>, pub extract_threshold: Option<u64>,
/// Current prompt-occupancy cap for the extract worker's own LLM
/// requests. Exceeding this aborts the extract run. `None` ⇒
/// [`defaults::MEMORY_EXTRACT_WORKER_MAX_INPUT_TOKENS`].
#[serde(default)]
pub extract_worker_max_input_tokens: Option<u64>,
/// Optional maximum extract-worker tool-loop depth. `None` leaves /// Optional maximum extract-worker tool-loop depth. `None` leaves
/// the worker unlimited; the default bounds runaway short-context /// the worker unlimited; the default bounds runaway short-context
/// loops. Falls through to /// loops. Falls through to
@ -337,9 +332,9 @@ pub enum ToolPermissionAction {
/// (full history summarisation). Omitting `[compaction]` disables both. /// (full history summarisation). Omitting `[compaction]` disables both.
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CompactionConfig { pub struct CompactionConfig {
/// Number of recent turns protected from pruning. /// Token budget at the history tail protected from pruning.
#[serde(default = "default_prune_protected_turns")] #[serde(default = "default_prune_protected_tokens")]
pub prune_protected_turns: usize, pub prune_protected_tokens: u64,
/// Minimum estimated token savings to trigger a prune. /// Minimum estimated token savings to trigger a prune.
#[serde(default = "default_prune_min_savings")] #[serde(default = "default_prune_min_savings")]
@ -393,8 +388,8 @@ pub struct CompactionConfig {
pub model: Option<ModelManifest>, pub model: Option<ModelManifest>,
} }
fn default_prune_protected_turns() -> usize { fn default_prune_protected_tokens() -> u64 {
defaults::PRUNE_PROTECTED_TURNS defaults::PRUNE_PROTECTED_TOKENS
} }
fn default_prune_min_savings() -> u64 { fn default_prune_min_savings() -> u64 {
defaults::PRUNE_MIN_SAVINGS defaults::PRUNE_MIN_SAVINGS
@ -415,7 +410,7 @@ fn default_compact_worker_max_turns() -> Option<u32> {
impl Default for CompactionConfig { impl Default for CompactionConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
prune_protected_turns: default_prune_protected_turns(), prune_protected_tokens: default_prune_protected_tokens(),
prune_min_savings: default_prune_min_savings(), prune_min_savings: default_prune_min_savings(),
compact_threshold: None, compact_threshold: None,
compact_request_threshold: None, compact_request_threshold: None,
@ -431,6 +426,7 @@ impl Default for CompactionConfig {
impl PodManifest { impl PodManifest {
/// Parse a manifest from a TOML string. /// Parse a manifest from a TOML string.
pub fn from_toml(s: &str) -> Result<Self, toml::de::Error> { pub fn from_toml(s: &str) -> Result<Self, toml::de::Error> {
config::reject_removed_manifest_fields(s)?;
toml::from_str(s) toml::from_str(s)
} }
} }
@ -581,7 +577,7 @@ model_id = "claude-sonnet-4-20250514"
let toml = format!("{MINIMAL_REQUIRED}\n[compaction]\ncompact_threshold = 80000\n"); let toml = format!("{MINIMAL_REQUIRED}\n[compaction]\ncompact_threshold = 80000\n");
let manifest = PodManifest::from_toml(&toml).unwrap(); let manifest = PodManifest::from_toml(&toml).unwrap();
let c = manifest.compaction.unwrap(); let c = manifest.compaction.unwrap();
assert_eq!(c.prune_protected_turns, 3); assert_eq!(c.prune_protected_tokens, 8000);
assert_eq!(c.prune_min_savings, 4096); assert_eq!(c.prune_min_savings, 4096);
assert_eq!(c.compact_threshold, Some(80000)); assert_eq!(c.compact_threshold, Some(80000));
assert_eq!(c.compact_request_threshold, None); assert_eq!(c.compact_request_threshold, None);
@ -589,6 +585,16 @@ model_id = "claude-sonnet-4-20250514"
assert_eq!(c.compact_worker_max_turns, Some(20)); assert_eq!(c.compact_worker_max_turns, Some(20));
} }
#[test]
fn reject_removed_prune_protected_turns_field() {
let toml = format!("{MINIMAL_REQUIRED}\n[compaction]\nprune_protected_turns = 3\n");
let err = PodManifest::from_toml(&toml).unwrap_err();
assert!(
err.to_string().contains("compaction.prune_protected_turns"),
"unexpected error: {err}"
);
}
#[test] #[test]
fn parse_compaction_worker_max_turns() { fn parse_compaction_worker_max_turns() {
let toml = format!( let toml = format!(

View File

@ -52,6 +52,10 @@ pub struct ModelManifest {
/// `default_capability` → scheme 既定の順で解決される。 /// `default_capability` → scheme 既定の順で解決される。
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub capability: Option<ModelCapability>, pub capability: Option<ModelCapability>,
/// モデルのコンテキストウィンドウ上限tokens。カタログ未掲載 / inline
/// モデルでもここで明示 override できる。
#[serde(default, skip_serializing_if = "Option::is_none")]
pub context_window: Option<u64>,
} }
impl ModelManifest { impl ModelManifest {
@ -65,6 +69,7 @@ impl ModelManifest {
model_id: upper.model_id.or(self.model_id), model_id: upper.model_id.or(self.model_id),
auth: upper.auth.or(self.auth), auth: upper.auth.or(self.auth),
capability: upper.capability.or(self.capability), capability: upper.capability.or(self.capability),
context_window: upper.context_window.or(self.context_window),
} }
} }
} }

View File

@ -24,7 +24,7 @@ pub struct Scope {
deny: Vec<ResolvedRule>, deny: Vec<ResolvedRule>,
} }
#[derive(Debug, Clone)] #[derive(Debug, Clone, PartialEq, Eq)]
struct ResolvedRule { struct ResolvedRule {
/// Absolute, canonicalized-or-normalized target directory/file. /// Absolute, canonicalized-or-normalized target directory/file.
target: PathBuf, target: PathBuf,
@ -217,6 +217,32 @@ impl Scope {
Self::from_config(&config) Self::from_config(&config)
} }
/// Build a new [`Scope`] with one matching deny rule removed for each
/// rule in `remove_deny`.
///
/// This is intentionally exact (after the same target resolution used
/// by [`Scope::from_config`]) rather than geometric: reclaiming a
/// delegated child must remove the deny layer that was added for that
/// child without broadening any explicit base deny that merely overlaps
/// the delegated path. Missing rules are ignored, making repeated
/// reclaim calls harmless.
pub fn with_removed_deny_rules(
&self,
remove_deny: impl IntoIterator<Item = ScopeRule>,
) -> Result<Self, ScopeError> {
let mut deny = self.deny.clone();
for rule in remove_deny {
let resolved = resolve_rule(&rule)?;
if let Some(idx) = deny.iter().position(|existing| existing == &resolved) {
deny.remove(idx);
}
}
Ok(Self {
allow: self.allow.clone(),
deny,
})
}
/// Human-readable grouping of allow rules, suitable for embedding in /// Human-readable grouping of allow rules, suitable for embedding in
/// LLM system prompts. Deny rules are intentionally omitted — they /// LLM system prompts. Deny rules are intentionally omitted — they
/// only cap effective permission and surface them would mislead the /// only cap effective permission and surface them would mislead the
@ -684,6 +710,44 @@ mod tests {
); );
} }
#[test]
fn with_removed_deny_rules_reclaims_one_matching_layer() {
let dir = TempDir::new().unwrap();
let sub = dir.path().join("sub");
std::fs::create_dir(&sub).unwrap();
let rule = ScopeRule {
target: sub.clone(),
permission: Permission::Write,
recursive: true,
};
let base = Scope::writable(dir.path())
.unwrap()
.with_added_deny_rules([rule.clone(), rule.clone()])
.unwrap();
let reclaimed_once = base.with_removed_deny_rules([rule.clone()]).unwrap();
assert_eq!(
reclaimed_once.permission_at(&sub.join("a.txt")),
Some(Permission::Read),
"one duplicate deny layer must remain"
);
let reclaimed_twice = reclaimed_once
.with_removed_deny_rules([rule.clone()])
.unwrap();
assert_eq!(
reclaimed_twice.permission_at(&sub.join("a.txt")),
Some(Permission::Write)
);
let reclaimed_again = reclaimed_twice.with_removed_deny_rules([rule]).unwrap();
assert_eq!(
reclaimed_again.permission_at(&sub.join("a.txt")),
Some(Permission::Write),
"missing rules are ignored for idempotent reclaim"
);
}
#[test] #[test]
fn shared_scope_load_returns_current_value() { fn shared_scope_load_returns_current_value() {
let dir = TempDir::new().unwrap(); let dir = TempDir::new().unwrap();

View File

@ -243,7 +243,7 @@ mod tests {
let (_id, _) = write_staging( let (_id, _) = write_staging(
&layout, &layout,
SourceRef { SourceRef {
session_id: "s".into(), segment_id: "s".into(),
range: [0, 1], range: [0, 1],
}, },
ExtractedPayload::default(), ExtractedPayload::default(),

View File

@ -260,7 +260,7 @@ mod tests {
let (id_a, _) = write_staging( let (id_a, _) = write_staging(
&layout, &layout,
SourceRef { SourceRef {
session_id: "s".into(), segment_id: "s".into(),
range: [0, 0], range: [0, 0],
}, },
ExtractedPayload::default(), ExtractedPayload::default(),
@ -269,7 +269,7 @@ mod tests {
let (id_b, _) = write_staging( let (id_b, _) = write_staging(
&layout, &layout,
SourceRef { SourceRef {
session_id: "s".into(), segment_id: "s".into(),
range: [1, 1], range: [1, 1],
}, },
ExtractedPayload::default(), ExtractedPayload::default(),

View File

@ -94,9 +94,9 @@ mod tests {
ExtractedPayload::default() ExtractedPayload::default()
} }
fn source(session_id: &str, range: [u64; 2]) -> SourceRef { fn source(segment_id: &str, range: [u64; 2]) -> SourceRef {
SourceRef { SourceRef {
session_id: session_id.into(), segment_id: segment_id.into(),
range, range,
} }
} }

View File

@ -305,7 +305,7 @@ mod tests {
fn flags_sources_overflow() { fn flags_sources_overflow() {
let (dir, layout) = workspace(); let (dir, layout) = workspace();
let many_sources: String = (0..15) let many_sources: String = (0..15)
.map(|i| format!(" - session_id: s{i}\n range: [{i}, {i}]\n")) .map(|i| format!(" - segment_id: s{i}\n range: [{i}, {i}]\n"))
.collect(); .collect();
write( write(
&dir.path().join(".insomnia/memory/decisions/big.md"), &dir.path().join(".insomnia/memory/decisions/big.md"),

View File

@ -69,4 +69,19 @@ mod tests {
assert!(s.contains("[ToolResult] ok")); assert!(s.contains("[ToolResult] ok"));
assert!(!s.contains("scratch")); assert!(!s.contains("scratch"));
} }
#[test]
fn tool_result_renders_summary_but_not_content() {
let huge_content = "raw-content-should-never-enter-extract-input".repeat(10_000);
let items = vec![Item::tool_result_with_content(
"c1",
"short summary kept for extraction",
huge_content.clone(),
)];
let s = build_extract_input(&items);
assert!(s.contains("[ToolResult] short summary kept for extraction"));
assert!(!s.contains("raw-content-should-never-enter-extract-input"));
assert!(!s.contains(&huge_content));
}
} }

View File

@ -13,7 +13,7 @@
//! session-store の `LogEntry::Extension`、domain `"memory.extract"`)は //! session-store の `LogEntry::Extension`、domain `"memory.extract"`)は
//! Pod 側が責務を持つ。 //! Pod 側が責務を持つ。
//! //!
//! 出力 JSON の wrap は [`write_staging`] が `source: { session_id, range }` //! 出力 JSON の wrap は [`write_staging`] が `source: { segment_id, range }`
//! を機械付与する形で担当し、LLM には source を推論させない。 //! を機械付与する形で担当し、LLM には source を推論させない。
mod input; mod input;

View File

@ -78,7 +78,7 @@ pub struct RequestEntry {
/// staging に書き出される 1 ファイル分のレコード。 /// staging に書き出される 1 ファイル分のレコード。
/// ///
/// `source` は Pod 側ラッパーが session_id と log entry range を /// `source` は Pod 側ラッパーが segment_id と log entry range を
/// 機械付与する。LLM はこのフィールドを見ない / 推論しない。 /// 機械付与する。LLM はこのフィールドを見ない / 推論しない。
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StagingRecord { pub struct StagingRecord {

View File

@ -10,7 +10,7 @@ use super::EXTRACT_DOMAIN;
/// として 1 回ずつ書かれ、最新の 1 件が現行 pointer として有効になる。 /// として 1 回ずつ書かれ、最新の 1 件が現行 pointer として有効になる。
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct ExtractPointerPayload { pub struct ExtractPointerPayload {
/// 直近 extract が処理した最後の session-store HashedEntry の index。 /// 直近 extract が処理した最後の session-store LogEntry の index。
/// 次回の `source.range.start` はこの値 + 1。 /// 次回の `source.range.start` はこの値 + 1。
pub processed_through_entry: usize, pub processed_through_entry: usize,
/// 直近 extract 時点の `history.len()`。次回入力は /// 直近 extract 時点の `history.len()`。次回入力は

View File

@ -71,7 +71,7 @@ mod tests {
let layout = WorkspaceLayout::new(tmp.path().to_path_buf()); let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
let source = SourceRef { let source = SourceRef {
session_id: "sess-1".into(), segment_id: "sess-1".into(),
range: [3, 7], range: [3, 7],
}; };
let payload = ExtractedPayload { let payload = ExtractedPayload {
@ -93,7 +93,7 @@ mod tests {
let written: StagingRecord = let written: StagingRecord =
serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap(); serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap();
assert_eq!(written.source.session_id, "sess-1"); assert_eq!(written.source.segment_id, "sess-1");
assert_eq!(written.source.range, [3, 7]); assert_eq!(written.source.range, [3, 7]);
assert_eq!(written.payload.decisions.len(), 1); assert_eq!(written.payload.decisions.len(), 1);
} }
@ -103,7 +103,7 @@ mod tests {
let tmp = tempfile::TempDir::new().unwrap(); let tmp = tempfile::TempDir::new().unwrap();
let layout = WorkspaceLayout::new(tmp.path().to_path_buf()); let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
let source = SourceRef { let source = SourceRef {
session_id: "sess".into(), segment_id: "sess".into(),
range: [0, 0], range: [0, 0],
}; };
let (_, path) = write_staging(&layout, source, ExtractedPayload::default()).unwrap(); let (_, path) = write_staging(&layout, source, ExtractedPayload::default()).unwrap();

View File

@ -10,7 +10,7 @@ pub use lint_common::Frontmatter;
/// `last_sources` arrays for traceability back to raw session logs. /// `last_sources` arrays for traceability back to raw session logs.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct SourceRef { pub struct SourceRef {
pub session_id: String, pub segment_id: String,
/// `[start_entry, end_entry]` inclusive range of session-store entry indices. /// `[start_entry, end_entry]` inclusive range of session-store entry indices.
pub range: [u64; 2], pub range: [u64; 2],
} }

View File

@ -60,7 +60,7 @@ impl Tool for ReadTool {
})?; })?;
let text = String::from_utf8_lossy(&bytes).into_owned(); let text = String::from_utf8_lossy(&bytes).into_owned();
if let Some(session_id) = self.usage_session_id.as_deref() { if let Some(segment_id) = self.usage_session_id.as_deref() {
let usage_slug = params.slug.as_deref().unwrap_or("summary"); let usage_slug = params.slug.as_deref().unwrap_or("summary");
let snapshot = usage::snapshot_record_from_bytes( let snapshot = usage::snapshot_record_from_bytes(
params.kind.record_kind(), params.kind.record_kind(),
@ -69,7 +69,7 @@ impl Tool for ReadTool {
); );
if let Err(err) = usage::append_use_event( if let Err(err) = usage::append_use_event(
&self.layout, &self.layout,
session_id.to_string(), segment_id.to_string(),
UsageSource::MemoryRead, UsageSource::MemoryRead,
vec![snapshot], vec![snapshot],
) { ) {
@ -140,9 +140,9 @@ pub fn read_tool(layout: WorkspaceLayout) -> ToolDefinition {
pub fn read_tool_with_usage( pub fn read_tool_with_usage(
layout: WorkspaceLayout, layout: WorkspaceLayout,
session_id: impl Into<String>, segment_id: impl Into<String>,
) -> ToolDefinition { ) -> ToolDefinition {
read_tool_inner(layout, Some(session_id.into())) read_tool_inner(layout, Some(segment_id.into()))
} }
fn read_tool_inner(layout: WorkspaceLayout, usage_session_id: Option<String>) -> ToolDefinition { fn read_tool_inner(layout: WorkspaceLayout, usage_session_id: Option<String>) -> ToolDefinition {

View File

@ -64,7 +64,7 @@ impl UsageRecordSnapshot {
pub struct UsageEvent { pub struct UsageEvent {
pub id: Uuid, pub id: Uuid,
pub occurred_at: DateTime<Utc>, pub occurred_at: DateTime<Utc>,
pub session_id: String, pub segment_id: String,
pub event: UsageEventKind, pub event: UsageEventKind,
pub source: UsageSource, pub source: UsageSource,
pub records: Vec<UsageRecordSnapshot>, pub records: Vec<UsageRecordSnapshot>,
@ -72,7 +72,7 @@ pub struct UsageEvent {
impl UsageEvent { impl UsageEvent {
pub fn new( pub fn new(
session_id: impl Into<String>, segment_id: impl Into<String>,
event: UsageEventKind, event: UsageEventKind,
source: UsageSource, source: UsageSource,
records: Vec<UsageRecordSnapshot>, records: Vec<UsageRecordSnapshot>,
@ -80,7 +80,7 @@ impl UsageEvent {
Self { Self {
id: Uuid::now_v7(), id: Uuid::now_v7(),
occurred_at: Utc::now(), occurred_at: Utc::now(),
session_id: session_id.into(), segment_id: segment_id.into(),
event, event,
source, source,
records, records,
@ -144,7 +144,7 @@ pub fn append_usage_event(layout: &WorkspaceLayout, event: &UsageEvent) -> io::R
/// Convenience for a successful explicit record read. /// Convenience for a successful explicit record read.
pub fn append_use_event( pub fn append_use_event(
layout: &WorkspaceLayout, layout: &WorkspaceLayout,
session_id: impl Into<String>, segment_id: impl Into<String>,
source: UsageSource, source: UsageSource,
records: Vec<UsageRecordSnapshot>, records: Vec<UsageRecordSnapshot>,
) -> io::Result<()> { ) -> io::Result<()> {
@ -153,14 +153,14 @@ pub fn append_use_event(
} }
append_usage_event( append_usage_event(
layout, layout,
&UsageEvent::new(session_id, UsageEventKind::Use, source, records), &UsageEvent::new(segment_id, UsageEventKind::Use, source, records),
) )
} }
/// Convenience for resident model-invocation exposure cost telemetry. /// Convenience for resident model-invocation exposure cost telemetry.
pub fn append_resident_exposure_event( pub fn append_resident_exposure_event(
layout: &WorkspaceLayout, layout: &WorkspaceLayout,
session_id: impl Into<String>, segment_id: impl Into<String>,
records: Vec<UsageRecordSnapshot>, records: Vec<UsageRecordSnapshot>,
) -> io::Result<()> { ) -> io::Result<()> {
if records.is_empty() { if records.is_empty() {
@ -169,7 +169,7 @@ pub fn append_resident_exposure_event(
append_usage_event( append_usage_event(
layout, layout,
&UsageEvent::new( &UsageEvent::new(
session_id, segment_id,
UsageEventKind::ResidentExposure, UsageEventKind::ResidentExposure,
UsageSource::ResidentInjection, UsageSource::ResidentInjection,
records, records,

View File

@ -4,7 +4,7 @@ use std::io;
use std::path::PathBuf; use std::path::PathBuf;
use manifest::ScopeRule; use manifest::ScopeRule;
use session_store::SessionId; use session_store::SegmentId;
/// Errors raised by the mutating pod-registry operations. /// Errors raised by the mutating pod-registry operations.
#[derive(Debug, thiserror::Error)] #[derive(Debug, thiserror::Error)]
@ -27,11 +27,11 @@ pub enum ScopeLockError {
#[error("pod `{0}` is not registered")] #[error("pod `{0}` is not registered")]
UnknownPod(String), UnknownPod(String),
#[error( #[error(
"session {session_id} is already held by pod `{pod_name}` at {}", "session {segment_id} is already held by pod `{pod_name}` at {}",
.socket.display() .socket.display()
)] )]
SessionConflict { SegmentConflict {
session_id: SessionId, segment_id: SegmentId,
pod_name: String, pod_name: String,
socket: PathBuf, socket: PathBuf,
}, },

View File

@ -27,11 +27,11 @@ pub use conflict::{
}; };
pub use error::ScopeLockError; pub use error::ScopeLockError;
pub use lifecycle::{ pub use lifecycle::{
ScopeAllocationGuard, SessionLockInfo, adopt_allocation, install_top_level, ScopeAllocationGuard, SegmentLockInfo, adopt_allocation, install_top_level,
install_top_level_with_deny, lookup_session, update_session, install_top_level_with_deny, lookup_segment, update_segment,
}; };
pub use mutate::{ pub use mutate::{
delegate_scope, reclaim_stale, reclaim_stale_with, register_pod, register_pod_with_deny, delegate_scope, reclaim_delegated_scope, reclaim_stale, reclaim_stale_with, register_pod,
release_pod, register_pod_with_deny, release_pod,
}; };
pub use table::{Allocation, LockFile, LockFileGuard, default_registry_path}; pub use table::{Allocation, LockFile, LockFileGuard, default_registry_path};

View File

@ -5,7 +5,7 @@
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use manifest::ScopeRule; use manifest::ScopeRule;
use session_store::SessionId; use session_store::SegmentId;
use crate::error::ScopeLockError; use crate::error::ScopeLockError;
use crate::mutate::release_pod; use crate::mutate::release_pod;
@ -45,9 +45,9 @@ pub fn install_top_level(
pid: u32, pid: u32,
socket: PathBuf, socket: PathBuf,
scope_allow: Vec<ScopeRule>, scope_allow: Vec<ScopeRule>,
session_id: SessionId, segment_id: SegmentId,
) -> Result<ScopeAllocationGuard, ScopeLockError> { ) -> Result<ScopeAllocationGuard, ScopeLockError> {
install_top_level_with_deny(pod_name, pid, socket, scope_allow, Vec::new(), session_id) install_top_level_with_deny(pod_name, pid, socket, scope_allow, Vec::new(), segment_id)
} }
/// Open the default lock file, register a top-level Pod with explicit /// Open the default lock file, register a top-level Pod with explicit
@ -59,7 +59,7 @@ pub fn install_top_level_with_deny(
socket: PathBuf, socket: PathBuf,
scope_allow: Vec<ScopeRule>, scope_allow: Vec<ScopeRule>,
scope_deny: Vec<ScopeRule>, scope_deny: Vec<ScopeRule>,
session_id: SessionId, segment_id: SegmentId,
) -> Result<ScopeAllocationGuard, ScopeLockError> { ) -> Result<ScopeAllocationGuard, ScopeLockError> {
let lock_path = default_registry_path()?; let lock_path = default_registry_path()?;
let mut guard = LockFileGuard::open(&lock_path)?; let mut guard = LockFileGuard::open(&lock_path)?;
@ -70,7 +70,7 @@ pub fn install_top_level_with_deny(
socket, socket,
scope_allow, scope_allow,
scope_deny, scope_deny,
session_id, segment_id,
)?; )?;
Ok(ScopeAllocationGuard { Ok(ScopeAllocationGuard {
pod_name, pod_name,
@ -83,14 +83,14 @@ pub fn install_top_level_with_deny(
/// ///
/// The spawning flow is two-stage: the spawner calls /// The spawning flow is two-stage: the spawner calls
/// [`crate::delegate_scope`] (with its own pid as a live placeholder, /// [`crate::delegate_scope`] (with its own pid as a live placeholder,
/// `session_id = None`), then exec's the child; the child, once /// `segment_id = None`), then exec's the child; the child, once
/// running, calls this function to rewrite the allocation's pid + /// running, calls this function to rewrite the allocation's pid +
/// session_id to its own and claim the [`ScopeAllocationGuard`] so /// segment_id to its own and claim the [`ScopeAllocationGuard`] so
/// the entry is released when the child exits. /// the entry is released when the child exits.
pub fn adopt_allocation( pub fn adopt_allocation(
pod_name: String, pod_name: String,
new_pid: u32, new_pid: u32,
session_id: SessionId, segment_id: SegmentId,
) -> Result<ScopeAllocationGuard, ScopeLockError> { ) -> Result<ScopeAllocationGuard, ScopeLockError> {
let lock_path = default_registry_path()?; let lock_path = default_registry_path()?;
let mut guard = LockFileGuard::open(&lock_path)?; let mut guard = LockFileGuard::open(&lock_path)?;
@ -99,7 +99,7 @@ pub fn adopt_allocation(
.find_mut(&pod_name) .find_mut(&pod_name)
.ok_or_else(|| ScopeLockError::UnknownPod(pod_name.clone()))?; .ok_or_else(|| ScopeLockError::UnknownPod(pod_name.clone()))?;
alloc.pid = new_pid; alloc.pid = new_pid;
alloc.session_id = Some(session_id); alloc.segment_id = Some(segment_id);
guard.save()?; guard.save()?;
Ok(ScopeAllocationGuard { Ok(ScopeAllocationGuard {
pod_name, pod_name,
@ -107,32 +107,32 @@ pub fn adopt_allocation(
}) })
} }
/// Rewrite the `session_id` recorded for `pod_name` to /// Rewrite the `segment_id` recorded for `pod_name` to
/// `new_session_id`. /// `new_segment_id`.
/// ///
/// The Pod's in-memory `session_id` can change underneath the /// The Pod's in-memory `segment_id` can change underneath the
/// allocation in two normal places: /// allocation in two normal places:
/// ///
/// - `Pod::compact` mints a fresh session and swaps it in. /// - `Pod::compact` mints a fresh session and swaps it in.
/// - `session_store::ensure_head_or_fork` auto-forks when another /// - `session_store::ensure_head_or_fork` auto-forks when another
/// writer has advanced the store head behind our back. /// writer has advanced the store head behind our back.
/// ///
/// Both paths must call this so subsequent [`lookup_session`] queries /// Both paths must call this so subsequent [`lookup_segment`] queries
/// find the live session id, not the old one. Without this update a /// find the live session id, not the old one. Without this update a
/// concurrent `restore_from_manifest(new_id)` would see "no live /// concurrent `restore_from_manifest(new_id)` would see "no live
/// writer" and proceed to register a competing allocation on the /// writer" and proceed to register a competing allocation on the
/// session this Pod just moved into. /// session this Pod just moved into.
/// ///
/// The lock is opened once and the allocation is rewritten inside the /// The lock is opened once and the allocation is rewritten inside the
/// guard, so the session_id collision check is atomic with the /// guard, so the segment_id collision check is atomic with the
/// rewrite. /// rewrite.
pub fn update_session(pod_name: &str, new_session_id: SessionId) -> Result<(), ScopeLockError> { pub fn update_segment(pod_name: &str, new_segment_id: SegmentId) -> Result<(), ScopeLockError> {
let lock_path = default_registry_path()?; let lock_path = default_registry_path()?;
let mut guard = LockFileGuard::open(&lock_path)?; let mut guard = LockFileGuard::open(&lock_path)?;
if let Some(other) = guard.data().find_by_session(new_session_id) { if let Some(other) = guard.data().find_by_segment(new_segment_id) {
if other.pod_name != pod_name { if other.pod_name != pod_name {
return Err(ScopeLockError::SessionConflict { return Err(ScopeLockError::SegmentConflict {
session_id: new_session_id, segment_id: new_segment_id,
pod_name: other.pod_name.clone(), pod_name: other.pod_name.clone(),
socket: other.socket.clone(), socket: other.socket.clone(),
}); });
@ -142,7 +142,7 @@ pub fn update_session(pod_name: &str, new_session_id: SessionId) -> Result<(), S
.data_mut() .data_mut()
.find_mut(pod_name) .find_mut(pod_name)
.ok_or_else(|| ScopeLockError::UnknownPod(pod_name.into()))?; .ok_or_else(|| ScopeLockError::UnknownPod(pod_name.into()))?;
alloc.session_id = Some(new_session_id); alloc.segment_id = Some(new_segment_id);
guard.save()?; guard.save()?;
Ok(()) Ok(())
} }
@ -150,25 +150,25 @@ pub fn update_session(pod_name: &str, new_session_id: SessionId) -> Result<(), S
/// Information about a Pod that currently holds an allocation for a /// Information about a Pod that currently holds an allocation for a
/// given session. /// given session.
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct SessionLockInfo { pub struct SegmentLockInfo {
pub pod_name: String, pub pod_name: String,
pub socket: PathBuf, pub socket: PathBuf,
pub pid: u32, pub pid: u32,
} }
/// Open the default lock file, reclaim stale entries, and return the /// Open the default lock file, reclaim stale entries, and return the
/// allocation currently writing to `session_id`, if any. /// allocation currently writing to `segment_id`, if any.
/// ///
/// Used by `Pod::restore_from_manifest` to refuse a resume that would /// Used by `Pod::restore_from_manifest` to refuse a resume that would
/// race a live writer on the same source session. /// race a live writer on the same source session.
pub fn lookup_session(session_id: SessionId) -> Result<Option<SessionLockInfo>, ScopeLockError> { pub fn lookup_segment(segment_id: SegmentId) -> Result<Option<SegmentLockInfo>, ScopeLockError> {
let lock_path = default_registry_path()?; let lock_path = default_registry_path()?;
let mut guard = LockFileGuard::open(&lock_path)?; let mut guard = LockFileGuard::open(&lock_path)?;
crate::mutate::reclaim_stale(&mut guard); crate::mutate::reclaim_stale(&mut guard);
Ok(guard Ok(guard
.data() .data()
.find_by_session(session_id) .find_by_segment(segment_id)
.map(|a| SessionLockInfo { .map(|a| SegmentLockInfo {
pod_name: a.pod_name.clone(), pod_name: a.pod_name.clone(),
socket: a.socket.clone(), socket: a.socket.clone(),
pid: a.pid, pid: a.pid,
@ -193,7 +193,7 @@ mod tests {
scope_allow: vec![write_rule("/tmp/child", true)], scope_allow: vec![write_rule("/tmp/child", true)],
scope_deny: Vec::new(), scope_deny: Vec::new(),
delegated_from: None, delegated_from: None,
session_id: None, segment_id: None,
}); });
g.save().unwrap(); g.save().unwrap();
} }
@ -267,12 +267,12 @@ mod tests {
s, s,
) )
.unwrap(); .unwrap();
let info = lookup_session(s).unwrap().expect("expected live writer"); let info = lookup_segment(s).unwrap().expect("expected live writer");
assert_eq!(info.pod_name, "live"); assert_eq!(info.pod_name, "live");
assert_eq!(info.socket, sock("live")); assert_eq!(info.socket, sock("live"));
drop(guard); drop(guard);
// After the guard's release, the lookup goes back to None. // After the guard's release, the lookup goes back to None.
assert!(lookup_session(s).unwrap().is_none()); assert!(lookup_segment(s).unwrap().is_none());
} }
#[test] #[test]
@ -289,10 +289,10 @@ mod tests {
original, original,
) )
.unwrap(); .unwrap();
update_session("p", updated).unwrap(); update_segment("p", updated).unwrap();
// lookup against the original is now empty, the updated id wins. // lookup against the original is now empty, the updated id wins.
assert!(lookup_session(original).unwrap().is_none()); assert!(lookup_segment(original).unwrap().is_none());
assert_eq!(lookup_session(updated).unwrap().unwrap().pod_name, "p"); assert_eq!(lookup_segment(updated).unwrap().unwrap().pod_name, "p");
} }
#[test] #[test]
@ -318,17 +318,17 @@ mod tests {
) )
.unwrap(); .unwrap();
// `a` cannot adopt b's live session id. // `a` cannot adopt b's live session id.
let err = update_session("a", s_b).unwrap_err(); let err = update_segment("a", s_b).unwrap_err();
match err { match err {
ScopeLockError::SessionConflict { ScopeLockError::SegmentConflict {
pod_name, pod_name,
session_id, segment_id,
.. ..
} => { } => {
assert_eq!(pod_name, "b"); assert_eq!(pod_name, "b");
assert_eq!(session_id, s_b); assert_eq!(segment_id, s_b);
} }
other => panic!("expected SessionConflict, got {other:?}"), other => panic!("expected SegmentConflict, got {other:?}"),
} }
} }
} }

View File

@ -5,7 +5,7 @@ use std::io;
use std::path::PathBuf; use std::path::PathBuf;
use manifest::{Permission, ScopeRule}; use manifest::{Permission, ScopeRule};
use session_store::SessionId; use session_store::SegmentId;
use crate::conflict::{find_conflict_owner, find_conflict_owners, is_within_effective_write}; use crate::conflict::{find_conflict_owner, find_conflict_owners, is_within_effective_write};
use crate::error::ScopeLockError; use crate::error::ScopeLockError;
@ -16,7 +16,7 @@ use crate::table::{Allocation, LockFileGuard};
/// conflicts so a crashed Pod's allocation doesn't block the new one. /// conflicts so a crashed Pod's allocation doesn't block the new one.
/// ///
/// Rejects when another live allocation is already writing to /// Rejects when another live allocation is already writing to
/// `session_id`, so two `restore_from_manifest` calls under different /// `segment_id`, so two `restore_from_manifest` calls under different
/// `pod_name`s cannot both grab the same session log. /// `pod_name`s cannot both grab the same session log.
pub fn register_pod( pub fn register_pod(
guard: &mut LockFileGuard, guard: &mut LockFileGuard,
@ -24,7 +24,7 @@ pub fn register_pod(
pid: u32, pid: u32,
socket: PathBuf, socket: PathBuf,
scope_allow: Vec<ScopeRule>, scope_allow: Vec<ScopeRule>,
session_id: SessionId, segment_id: SegmentId,
) -> Result<(), ScopeLockError> { ) -> Result<(), ScopeLockError> {
register_pod_with_deny( register_pod_with_deny(
guard, guard,
@ -33,7 +33,7 @@ pub fn register_pod(
socket, socket,
scope_allow, scope_allow,
Vec::new(), Vec::new(),
session_id, segment_id,
) )
} }
@ -56,15 +56,15 @@ pub fn register_pod_with_deny(
socket: PathBuf, socket: PathBuf,
scope_allow: Vec<ScopeRule>, scope_allow: Vec<ScopeRule>,
scope_deny: Vec<ScopeRule>, scope_deny: Vec<ScopeRule>,
session_id: SessionId, segment_id: SegmentId,
) -> Result<(), ScopeLockError> { ) -> Result<(), ScopeLockError> {
reclaim_stale(guard); reclaim_stale(guard);
if guard.data().find(&pod_name).is_some() { if guard.data().find(&pod_name).is_some() {
return Err(ScopeLockError::DuplicatePodName(pod_name)); return Err(ScopeLockError::DuplicatePodName(pod_name));
} }
if let Some(existing) = guard.data().find_by_session(session_id) { if let Some(existing) = guard.data().find_by_segment(segment_id) {
return Err(ScopeLockError::SessionConflict { return Err(ScopeLockError::SegmentConflict {
session_id, segment_id,
pod_name: existing.pod_name.clone(), pod_name: existing.pod_name.clone(),
socket: existing.socket.clone(), socket: existing.socket.clone(),
}); });
@ -99,7 +99,7 @@ pub fn register_pod_with_deny(
scope_allow, scope_allow,
scope_deny, scope_deny,
delegated_from: None, delegated_from: None,
session_id: Some(session_id), segment_id: Some(segment_id),
}); });
guard.save()?; guard.save()?;
Ok(()) Ok(())
@ -147,9 +147,9 @@ pub fn delegate_scope(
scope_allow, scope_allow,
scope_deny: Vec::new(), scope_deny: Vec::new(),
delegated_from: Some(spawner.into()), delegated_from: Some(spawner.into()),
// Pre-reservation. The child fills in its own session_id when // Pre-reservation. The child fills in its own segment_id when
// it calls `adopt_allocation` after the worker is built. // it calls `adopt_allocation` after the worker is built.
session_id: None, segment_id: None,
}); });
guard.save()?; guard.save()?;
Ok(()) Ok(())
@ -178,6 +178,55 @@ pub fn release_pod(guard: &mut LockFileGuard, pod_name: &str) -> Result<(), Scop
Ok(()) Ok(())
} }
/// Reclaim a child delegation back into its parent allocation.
///
/// This is idempotent: missing child allocations and missing deny entries are
/// ignored. For each delegated Write rule, at most one exact matching deny rule
/// is removed from the parent's `scope_deny`, preserving any duplicate explicit
/// base deny that was not owned by this child delegation.
pub fn reclaim_delegated_scope(
guard: &mut LockFileGuard,
parent: &str,
child: &str,
delegated_scope: &[ScopeRule],
) -> Result<(), ScopeLockError> {
let child_idx = guard
.data()
.allocations
.iter()
.position(|a| a.pod_name == child);
let removed_child_parent = child_idx
.map(|idx| guard.data().allocations[idx].delegated_from.clone())
.unwrap_or(None);
let child_exists = child_idx.is_some();
if child_exists {
if let Some(parent_alloc) = guard.data_mut().find_mut(parent) {
for rule in delegated_scope
.iter()
.filter(|rule| rule.permission == Permission::Write)
{
if let Some(idx) = parent_alloc.scope_deny.iter().position(|deny| deny == rule) {
parent_alloc.scope_deny.remove(idx);
}
}
}
}
if let Some(idx) = child_idx {
for alloc in guard.data_mut().allocations.iter_mut() {
if alloc.delegated_from.as_deref() == Some(child) {
alloc.delegated_from.clone_from(&removed_child_parent);
}
}
guard.data_mut().allocations.remove(idx);
}
guard.save()?;
Ok(())
}
/// Remove allocations whose PID is dead, reparenting children to the /// Remove allocations whose PID is dead, reparenting children to the
/// dead Pod's `delegated_from`. Idempotent and best-effort — I/O /// dead Pod's `delegated_from`. Idempotent and best-effort — I/O
/// errors on save are swallowed so a crashed Pod's entry never blocks /// errors on save are swallowed so a crashed Pod's entry never blocks
@ -436,6 +485,46 @@ mod tests {
assert!(g.data().find("b").is_none()); assert!(g.data().find("b").is_none());
} }
#[test]
fn reclaim_delegated_scope_removes_child_and_one_parent_deny_layer() {
let dir = TempDir::new().unwrap();
let path = dir.path().join("pods.json");
let mut g = open_empty(&path);
let delegated_rule = write_rule("/src/core", true);
register_pod_with_deny(
&mut g,
"a".into(),
std::process::id(),
sock("a"),
vec![write_rule("/src", true)],
vec![delegated_rule.clone(), delegated_rule.clone()],
sid(),
)
.unwrap();
register_pod(
&mut g,
"b".into(),
std::process::id(),
sock("b"),
vec![delegated_rule.clone()],
sid(),
)
.unwrap();
reclaim_delegated_scope(&mut g, "a", "b", std::slice::from_ref(&delegated_rule)).unwrap();
let a = g.data().find("a").unwrap();
assert_eq!(a.scope_deny, vec![delegated_rule.clone()]);
assert!(g.data().find("b").is_none());
reclaim_delegated_scope(&mut g, "a", "b", &[delegated_rule.clone()]).unwrap();
let a = g.data().find("a").unwrap();
assert_eq!(
a.scope_deny,
vec![delegated_rule],
"a repeated reclaim with no child allocation must not broaden an explicit duplicate base deny"
);
}
#[test] #[test]
fn reclaim_stale_reparents_and_removes_dead_entries() { fn reclaim_stale_reparents_and_removes_dead_entries() {
let dir = TempDir::new().unwrap(); let dir = TempDir::new().unwrap();
@ -587,8 +676,8 @@ mod tests {
shared_session, shared_session,
) )
.unwrap(); .unwrap();
// Second registration tries to grab the same session_id under // Second registration tries to grab the same segment_id under
// a different pod_name. Without the SessionConflict check both // a different pod_name. Without the SegmentConflict check both
// would succeed and race on the same jsonl. // would succeed and race on the same jsonl.
let err = register_pod( let err = register_pod(
&mut g, &mut g,
@ -600,15 +689,15 @@ mod tests {
) )
.unwrap_err(); .unwrap_err();
match err { match err {
ScopeLockError::SessionConflict { ScopeLockError::SegmentConflict {
session_id, segment_id,
pod_name, pod_name,
.. ..
} => { } => {
assert_eq!(session_id, shared_session); assert_eq!(segment_id, shared_session);
assert_eq!(pod_name, "first"); assert_eq!(pod_name, "first");
} }
other => panic!("expected SessionConflict, got {other:?}"), other => panic!("expected SegmentConflict, got {other:?}"),
} }
} }
} }

View File

@ -8,7 +8,7 @@ use std::path::{Path, PathBuf};
use fs4::fs_std::FileExt; use fs4::fs_std::FileExt;
use manifest::{ScopeRule, paths}; use manifest::{ScopeRule, paths};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use session_store::SessionId; use session_store::SegmentId;
/// On-disk representation of the allocation table. /// On-disk representation of the allocation table.
#[derive(Debug, Clone, Default, Serialize, Deserialize)] #[derive(Debug, Clone, Default, Serialize, Deserialize)]
@ -43,12 +43,12 @@ pub struct Allocation {
/// Name of the Pod that delegated scope to this one, or `None` for /// Name of the Pod that delegated scope to this one, or `None` for
/// a top-level Pod started directly by a human. /// a top-level Pod started directly by a human.
pub delegated_from: Option<String>, pub delegated_from: Option<String>,
/// Session ID this Pod is currently writing to. `None` means this /// Segment ID this Pod is currently writing to. `None` means this
/// is a pre-reservation made by a spawner via [`crate::delegate_scope`] /// is a pre-reservation made by a spawner via [`crate::delegate_scope`]
/// before the child has come up; the child fills it in at /// before the child has come up; the child fills it in at
/// [`crate::adopt_allocation`] time. /// [`crate::adopt_allocation`] time.
#[serde(default)] #[serde(default)]
pub session_id: Option<SessionId>, pub segment_id: Option<SegmentId>,
} }
impl LockFile { impl LockFile {
@ -60,12 +60,12 @@ impl LockFile {
self.allocations.iter_mut().find(|a| a.pod_name == pod_name) self.allocations.iter_mut().find(|a| a.pod_name == pod_name)
} }
/// Find the allocation currently writing to `session_id`. Skips /// Find the allocation currently writing to `segment_id`. Skips
/// pre-reservations whose `session_id` is still `None`. /// pre-reservations whose `segment_id` is still `None`.
pub fn find_by_session(&self, session_id: SessionId) -> Option<&Allocation> { pub fn find_by_segment(&self, segment_id: SegmentId) -> Option<&Allocation> {
self.allocations self.allocations
.iter() .iter()
.find(|a| a.session_id == Some(session_id)) .find(|a| a.segment_id == Some(segment_id))
} }
} }
@ -225,8 +225,8 @@ mod tests {
let dir = TempDir::new().unwrap(); let dir = TempDir::new().unwrap();
let path = dir.path().join("pods.json"); let path = dir.path().join("pods.json");
let mut g = open_empty(&path); let mut g = open_empty(&path);
// Pre-reservation: delegate_scope leaves session_id = None // Pre-reservation: delegate_scope leaves segment_id = None
// until adopt_allocation rewrites it. find_by_session must not // until adopt_allocation rewrites it. find_by_segment must not
// match those placeholders, otherwise a freshly-spawning child // match those placeholders, otherwise a freshly-spawning child
// would shadow itself before it has even chosen a session. // would shadow itself before it has even chosen a session.
register_pod( register_pod(
@ -249,13 +249,13 @@ mod tests {
.unwrap(); .unwrap();
let target_session = sid(); let target_session = sid();
// The placeholder allocation has session_id = None and must // The placeholder allocation has segment_id = None and must
// not be returned for any lookup. // not be returned for any lookup.
assert!(g.data().find_by_session(target_session).is_none()); assert!(g.data().find_by_segment(target_session).is_none());
// After adopt-style rewrite, the same allocation is now found. // After adopt-style rewrite, the same allocation is now found.
g.data_mut().find_mut("child").unwrap().session_id = Some(target_session); g.data_mut().find_mut("child").unwrap().segment_id = Some(target_session);
let found = g.data().find_by_session(target_session).unwrap(); let found = g.data().find_by_segment(target_session).unwrap();
assert_eq!(found.pod_name, "child"); assert_eq!(found.pod_name, "child");
} }
} }

View File

@ -6,12 +6,12 @@ use std::path::{Path, PathBuf};
use std::sync::{LazyLock, Mutex, MutexGuard}; use std::sync::{LazyLock, Mutex, MutexGuard};
use manifest::{Permission, ScopeRule}; use manifest::{Permission, ScopeRule};
use session_store::SessionId; use session_store::SegmentId;
use crate::table::LockFileGuard; use crate::table::LockFileGuard;
pub(crate) fn sid() -> SessionId { pub(crate) fn sid() -> SegmentId {
session_store::new_session_id() session_store::new_segment_id()
} }
/// Serialises tests that mutate runtime-dir env vars. The test /// Serialises tests that mutate runtime-dir env vars. The test

View File

@ -30,7 +30,7 @@ memory = { workspace = true }
workflow-crate = { package = "workflow", path = "../workflow" } workflow-crate = { package = "workflow", path = "../workflow" }
uuid = { workspace = true, features = ["v7"] } uuid = { workspace = true, features = ["v7"] }
session-metrics = { workspace = true } session-metrics = { workspace = true }
parking_lot = "0.12.5" arc-swap = "1.9.1"
[dev-dependencies] [dev-dependencies]
dotenv = "0.15.0" dotenv = "0.15.0"

View File

@ -54,7 +54,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut pod = Pod::from_manifest_toml(&toml, store).await?; let mut pod = Pod::from_manifest_toml(&toml, store).await?;
let manifest: &PodManifest = pod.manifest(); let manifest: &PodManifest = pod.manifest();
println!("Pod: {}", manifest.pod.name); println!("Pod: {}", manifest.pod.name);
println!("Session: {}", pod.session_id()); println!("Segment: {}", pod.segment_id());
// 4. Run a prompt // 4. Run a prompt
let result = pod.run_text("What is the capital of France?").await?; let result = pod.run_text("What is the capital of France?").await?;
@ -76,7 +76,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
} }
// 6. Session ID for potential restore // 6. Session ID for potential restore
println!("\nSession ID: {}", pod.session_id()); println!("\nSegment ID: {}", pod.segment_id());
Ok(()) Ok(())
} }

View File

@ -13,19 +13,24 @@
use llm_worker::Item; use llm_worker::Item;
use llm_worker::llm_client::client::LlmClient; use llm_worker::llm_client::client::LlmClient;
use llm_worker::prune::{PruneConfig, PruneDecision, PruneObserver, SavingsEstimator}; use llm_worker::prune::{
PruneConfig, PruneDecision, PruneObserver, SavingsEstimator, TokenEstimator,
};
use session_metrics::Metric; use session_metrics::Metric;
use session_store::Store; use session_store::Store;
use crate::Pod; use crate::Pod;
use crate::compact::token_counter::{EstimateSource, savings_for_prune_impl}; use crate::compact::token_counter::{
EstimateSource, savings_for_prune_impl, token_estimates_for_prune_impl,
};
impl<C: LlmClient, St: Store> Pod<C, St> { impl<C: LlmClient, St: Store> Pod<C, St> {
/// Enable prune projection on the underlying Worker. /// Enable prune projection on the underlying Worker.
/// ///
/// Registers the config and a savings-estimator closure on the Worker. /// Registers the config and token/savings-estimator closures on the Worker.
/// The estimator captures a shared handle to [`Pod::usage_history_handle`] /// The estimators combine persisted [`Pod::usage_history_handle`] records
/// so that every LLM request sees the latest measurements. /// with in-flight `UsageTracker` records so multi-request tool loops can
/// prune before the surrounding Pod run finishes.
/// ///
/// Measurement-less estimates (before the first LLM call, or immediately /// Measurement-less estimates (before the first LLM call, or immediately
/// after a compact) return `0` from the estimator, which naturally /// after a compact) return `0` from the estimator, which naturally
@ -37,9 +42,25 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
/// [`UsageTracker`] so the next `LlmUsage` can be paired with a /// [`UsageTracker`] so the next `LlmUsage` can be paired with a
/// `prune.post_request` metric carrying the same id. /// `prune.post_request` metric carrying the same id.
pub fn attach_prune(&mut self, config: PruneConfig) { pub fn attach_prune(&mut self, config: PruneConfig) {
let usage = self.usage_history_handle(); let usage_history_for_tokens = self.usage_history_handle();
let usage_tracker_for_tokens = self.usage_tracker_handle();
let token_estimator: TokenEstimator = Box::new(move |history: &[Item]| {
let mut snapshot = usage_history_for_tokens
.lock()
.expect("usage_history poisoned")
.clone();
snapshot.extend(usage_tracker_for_tokens.records());
token_estimates_for_prune_impl(history, &snapshot)
});
let usage_history_for_savings = self.usage_history_handle();
let usage_tracker_for_savings = self.usage_tracker_handle();
let estimator: SavingsEstimator = Box::new(move |history: &[Item], indices| { let estimator: SavingsEstimator = Box::new(move |history: &[Item], indices| {
let snapshot = usage.lock().expect("usage_history poisoned").clone(); let mut snapshot = usage_history_for_savings
.lock()
.expect("usage_history poisoned")
.clone();
snapshot.extend(usage_tracker_for_savings.records());
let est = savings_for_prune_impl(history, &snapshot, indices); let est = savings_for_prune_impl(history, &snapshot, indices);
match est.source { match est.source {
EstimateSource::NoData => 0, EstimateSource::NoData => 0,
@ -56,8 +77,9 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
.with_value(eval.estimated_savings as f64) .with_value(eval.estimated_savings as f64)
.with_correlation_id(&correlation_id) .with_correlation_id(&correlation_id)
.with_dimension("candidate_count", eval.candidate_count.to_string()); .with_dimension("candidate_count", eval.candidate_count.to_string());
if let Some(border) = eval.border_turn { if let Some(protected_start) = eval.protected_start_index {
metric = metric.with_dimension("border_turn", border.to_string()); metric =
metric.with_dimension("protected_start_index", protected_start.to_string());
} }
metrics.push(metric); metrics.push(metric);
usage_tracker.note_correlation_id(correlation_id); usage_tracker.note_correlation_id(correlation_id);
@ -66,17 +88,21 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
metrics.push(Metric::now("prune.skip").with_dimension("reason", "no_candidates")); metrics.push(Metric::now("prune.skip").with_dimension("reason", "no_candidates"));
} }
PruneDecision::SkippedBelowMinSavings => { PruneDecision::SkippedBelowMinSavings => {
metrics.push( let mut metric = Metric::now("prune.skip")
Metric::now("prune.skip") .with_dimension("reason", "below_min_savings")
.with_dimension("reason", "below_min_savings") .with_dimension("candidate_count", eval.candidate_count.to_string())
.with_dimension("candidate_count", eval.candidate_count.to_string()) .with_value(eval.estimated_savings as f64);
.with_value(eval.estimated_savings as f64), if let Some(protected_start) = eval.protected_start_index {
); metric =
metric.with_dimension("protected_start_index", protected_start.to_string());
}
metrics.push(metric);
} }
}); });
let worker = self.worker_mut(); let worker = self.worker_mut();
worker.set_prune_config(Some(config)); worker.set_prune_config(Some(config));
worker.set_token_estimator(Some(token_estimator));
worker.set_savings_estimator(Some(estimator)); worker.set_savings_estimator(Some(estimator));
worker.set_prune_observer(Some(observer)); worker.set_prune_observer(Some(observer));
} }
@ -90,7 +116,7 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
return; return;
}; };
let config = PruneConfig { let config = PruneConfig {
protected_turns: compaction.prune_protected_turns, protected_tokens: compaction.prune_protected_tokens,
min_savings: compaction.prune_min_savings, min_savings: compaction.prune_min_savings,
}; };
self.attach_prune(config); self.attach_prune(config);

View File

@ -132,6 +132,21 @@ fn tool_result_content_bytes(item: &Item) -> u64 {
item_bytes(item).saturating_sub(item_bytes(&cleared)) item_bytes(item).saturating_sub(item_bytes(&cleared))
} }
/// Prefix-boundary token estimates used by Prune to find its protected suffix.
///
/// Returns `history.len() + 1` entries where entry `i` estimates
/// `history[..i]`. This shares the same [`tokens_at`] accounting as compact's
/// retained-tail split and prune's savings estimate.
pub(crate) fn token_estimates_for_prune_impl(
history: &[Item],
records: &[UsageRecord],
) -> Vec<TokenEstimate> {
let prefix = prefix_bytes(history);
(0..=history.len())
.map(|idx| tokens_at(history, records, idx, &prefix))
.collect()
}
/// Prune 射影(`ToolResult.content = None`)で節約されるトークン数の推定。 /// Prune 射影(`ToolResult.content = None`)で節約されるトークン数の推定。
/// ///
/// `indices` は [`llm_worker::prune::prunable_indices`] が返す候補列を /// `indices` は [`llm_worker::prune::prunable_indices`] が返す候補列を
@ -278,6 +293,26 @@ mod tests {
} }
} }
#[test]
fn token_estimates_for_prune_returns_every_prefix_boundary() {
let history = vec![msg("a"), msg("b"), msg("c")];
let estimates = token_estimates_for_prune_impl(&history, &[record(3, 300)]);
assert_eq!(estimates.len(), history.len() + 1);
assert_eq!(estimates[0].tokens, 0);
assert_eq!(estimates[3].tokens, 300);
assert_eq!(estimates[3].source, EstimateSource::Measured);
}
#[test]
fn token_estimates_for_prune_propagates_no_data() {
let history = vec![msg("a"), msg("b")];
let estimates = token_estimates_for_prune_impl(&history, &[]);
assert_eq!(estimates.len(), history.len() + 1);
assert_eq!(estimates[0].source, EstimateSource::Measured);
assert_eq!(estimates[1].source, EstimateSource::NoData);
assert_eq!(estimates[2].source, EstimateSource::NoData);
}
#[test] #[test]
fn savings_for_prune_skips_non_toolresult_indices() { fn savings_for_prune_skips_non_toolresult_indices() {
let history = vec![msg("a"), msg("b"), msg("c")]; let history = vec![msg("a"), msg("b"), msg("c")];

View File

@ -3,7 +3,7 @@ use std::sync::Arc;
use llm_worker::WorkerError; use llm_worker::WorkerError;
use llm_worker::llm_client::client::LlmClient; use llm_worker::llm_client::client::LlmClient;
use session_store::Store; use session_store::{PodMetadataStore, Store};
use tokio::sync::{broadcast, mpsc, oneshot}; use tokio::sync::{broadcast, mpsc, oneshot};
use crate::ipc::alerter::Alerter; use crate::ipc::alerter::Alerter;
@ -11,7 +11,7 @@ use crate::ipc::notify_buffer::NotifyBuffer;
use crate::ipc::server::SocketServer; use crate::ipc::server::SocketServer;
use crate::pod::{Pod, PodError, PodRunResult, SystemItemCommitter}; use crate::pod::{Pod, PodError, PodRunResult, SystemItemCommitter};
use crate::runtime::dir::RuntimeDir; use crate::runtime::dir::RuntimeDir;
use crate::session_log_sink::SessionLogSink; use crate::segment_log_sink::SegmentLogSink;
use crate::shared_state::PodSharedState; use crate::shared_state::PodSharedState;
use crate::spawn::comm_tools::{ use crate::spawn::comm_tools::{
list_pods_tool, read_pod_output_tool, send_to_pod_tool, stop_pod_tool, list_pods_tool, read_pod_output_tool, send_to_pod_tool, stop_pod_tool,
@ -33,10 +33,10 @@ pub struct PodHandle {
pub shared_state: Arc<PodSharedState>, pub shared_state: Arc<PodSharedState>,
pub runtime_dir: Arc<RuntimeDir>, pub runtime_dir: Arc<RuntimeDir>,
pub alerter: Alerter, pub alerter: Alerter,
/// Session-log mirror + broadcast handle. The IPC server snapshots /// Segment-log mirror + broadcast handle. The IPC server snapshots
/// it on every new connection (Event::Snapshot) and forwards /// it on every new connection (Event::Snapshot) and forwards
/// subsequent commits (Event::Entry) on the receiver. /// subsequent commits (Event::Entry) on the receiver.
pub sink: SessionLogSink, pub sink: SegmentLogSink,
} }
impl PodHandle { impl PodHandle {
@ -132,7 +132,7 @@ impl PodController {
) -> Result<(PodHandle, ShutdownReceiver), std::io::Error> ) -> Result<(PodHandle, ShutdownReceiver), std::io::Error>
where where
C: LlmClient + Clone + 'static, C: LlmClient + Clone + 'static,
St: Store + Clone + 'static, St: Store + PodMetadataStore + Clone + Send + Sync + 'static,
{ {
// === 1. Initialization (channels / RuntimeDir / pod-immutable // === 1. Initialization (channels / RuntimeDir / pod-immutable
// snapshots / SpawnedPodRegistry / alerter attach / // snapshots / SpawnedPodRegistry / alerter attach /
@ -151,7 +151,20 @@ impl PodController {
let spawner_name = pod.manifest().pod.name.clone(); let spawner_name = pod.manifest().pod.name.clone();
let self_parent_socket = pod.callback_socket().cloned(); let self_parent_socket = pod.callback_socket().cloned();
let spawned_registry = SpawnedPodRegistry::new(runtime_dir.clone()); let loaded_registry = SpawnedPodRegistry::load_from_pod_state_with_reclaim(
runtime_dir.clone(),
pod.store().clone(),
spawner_name.clone(),
Some(pod.scope().clone()),
Some(pod.scope_change_sink()),
)
.await?;
let reclaimed_unreachable = loaded_registry.reclaimed_unreachable;
let spawned_registry = loaded_registry.registry;
if reclaimed_unreachable {
pod.persist_scope_snapshot()
.map_err(std::io::Error::other)?;
}
// Hand the alerter to the Pod so internal operations (compaction, // Hand the alerter to the Pod so internal operations (compaction,
// AGENTS.md ingestion during the first turn) can emit user-facing // AGENTS.md ingestion during the first turn) can emit user-facing
@ -214,7 +227,7 @@ impl PodController {
let greeting = build_greeting(&pod); let greeting = build_greeting(&pod);
let shared_state = Arc::new(PodSharedState::new( let shared_state = Arc::new(PodSharedState::new(
pod.manifest().pod.name.clone(), pod.manifest().pod.name.clone(),
pod.session_id(), pod.segment_id(),
manifest_toml.clone(), manifest_toml.clone(),
greeting, greeting,
)); ));
@ -430,7 +443,7 @@ where
let scope_handle = pod.scope().clone(); let scope_handle = pod.scope().clone();
let pwd = pod.pwd().to_path_buf(); let pwd = pod.pwd().to_path_buf();
let task_store = pod.task_store(); let task_store = pod.task_store();
let session_id_for_usage = pod.session_id().to_string(); let session_id_for_usage = pod.segment_id().to_string();
let scope_change_sink = pod.scope_change_sink(); let scope_change_sink = pod.scope_change_sink();
let memory_config = pod.manifest().memory.clone(); let memory_config = pod.manifest().memory.clone();
let spawner_name = pod.manifest().pod.name.clone(); let spawner_name = pod.manifest().pod.name.clone();
@ -639,9 +652,7 @@ async fn controller_loop<C, St>(
// sees the buffered notification(s) without a human // sees the buffered notification(s) without a human
// Run. // Run.
if shared_state.get_status() == PodStatus::Idle { if shared_state.get_status() == PodStatus::Idle {
pending = Some(PendingRun::RunForNotification( pending = Some(PendingRun::RunForNotification(protocol::InvokeKind::Notify));
protocol::InvokeKind::Notify,
));
} }
} }
@ -893,6 +904,10 @@ where
// `build_client` がここに到達する前に同じマニフェストで成功している // `build_client` がここに到達する前に同じマニフェストで成功している
// ため、カタログ解決も必ず通る。念のため失敗時は "unknown" に落とす。 // ため、カタログ解決も必ず通る。念のため失敗時は "unknown" に落とす。
let resolved = provider::catalog::resolve_model_manifest(&manifest.model).ok(); let resolved = provider::catalog::resolve_model_manifest(&manifest.model).ok();
let context_window = resolved
.as_ref()
.map(|cfg| cfg.context_window)
.unwrap_or(provider::catalog::DEFAULT_CONTEXT_WINDOW);
let (provider_name, model_id) = match resolved { let (provider_name, model_id) = match resolved {
Some(cfg) => { Some(cfg) => {
let name = match cfg.scheme { let name = match cfg.scheme {
@ -930,6 +945,8 @@ where
model: model_id, model: model_id,
scope_summary: pod.scope_snapshot().summary(), scope_summary: pod.scope_snapshot().summary(),
tools: tool_names, tools: tool_names,
context_window,
context_tokens: pod.total_tokens().tokens,
} }
} }
@ -950,7 +967,7 @@ fn worker_error_code(e: &PodError) -> ErrorCode {
mod tests { mod tests {
use super::*; use super::*;
use protocol::PodEvent; use protocol::PodEvent;
use protocol::stream::JsonLineReader; use protocol::stream::{JsonLineReader, JsonLineWriter};
use std::time::Duration; use std::time::Duration;
use tempfile::TempDir; use tempfile::TempDir;
use tokio::net::UnixListener; use tokio::net::UnixListener;
@ -992,7 +1009,7 @@ mod tests {
let (cancel_tx, cancel_rx) = mpsc::channel::<()>(1); let (cancel_tx, cancel_rx) = mpsc::channel::<()>(1);
let shared_state = Arc::new(PodSharedState::new( let shared_state = Arc::new(PodSharedState::new(
"child-pod".to_string(), "child-pod".to_string(),
session_store::new_session_id(), session_store::new_segment_id(),
String::new(), String::new(),
protocol::Greeting { protocol::Greeting {
pod_name: "child-pod".to_string(), pod_name: "child-pod".to_string(),
@ -1001,6 +1018,8 @@ mod tests {
model: String::new(), model: String::new(),
scope_summary: String::new(), scope_summary: String::new(),
tools: Vec::new(), tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
}, },
)); ));
let notify_buffer = NotifyBuffer::new(); let notify_buffer = NotifyBuffer::new();
@ -1028,7 +1047,26 @@ mod tests {
async fn recv_pod_event(listener: UnixListener, timeout: Duration) -> Option<PodEvent> { async fn recv_pod_event(listener: UnixListener, timeout: Duration) -> Option<PodEvent> {
let accept = async { let accept = async {
let (stream, _) = listener.accept().await.ok()?; let (stream, _) = listener.accept().await.ok()?;
let mut reader = JsonLineReader::new(stream); let (r, w) = stream.into_split();
let mut writer = JsonLineWriter::new(w);
writer
.write(&Event::Snapshot {
entries: Vec::new(),
greeting: protocol::Greeting {
pod_name: "parent".into(),
cwd: "/tmp".into(),
provider: "test".into(),
model: "test".into(),
scope_summary: String::new(),
tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
},
status: PodStatus::Idle,
})
.await
.ok()?;
let mut reader = JsonLineReader::new(r);
match reader.next::<Method>().await { match reader.next::<Method>().await {
Ok(Some(Method::PodEvent(e))) => Some(e), Ok(Some(Method::PodEvent(e))) => Some(e),
_ => None, _ => None,

View File

@ -11,9 +11,9 @@
//! happen at the front of `Pod::run` when //! happen at the front of `Pod::run` when
//! `worker.last_run_interrupted()` is set; see `Pod::apply_interrupt_prep`. //! `worker.last_run_interrupted()` is set; see `Pod::apply_interrupt_prep`.
use llm_worker::Item;
#[cfg(test)] #[cfg(test)]
use crate::prompt::catalog::PromptCatalog; use crate::prompt::catalog::PromptCatalog;
use llm_worker::Item;
/// Build synthetic `Item::ToolResult` items for every unanswered /// Build synthetic `Item::ToolResult` items for every unanswered
/// `Item::ToolCall` in `history`, preserving order. /// `Item::ToolCall` in `history`, preserving order.

View File

@ -27,7 +27,6 @@ use std::sync::Arc;
use protocol::{Method, PodEvent, ScopeRule}; use protocol::{Method, PodEvent, ScopeRule};
use crate::runtime::dir::SpawnedPodRecord; use crate::runtime::dir::SpawnedPodRecord;
use crate::runtime::pod_registry::{self, ScopeLockError};
use crate::spawn::comm_tools::connect_and_send; use crate::spawn::comm_tools::connect_and_send;
use crate::spawn::registry::SpawnedPodRegistry; use crate::spawn::registry::SpawnedPodRegistry;
@ -86,8 +85,8 @@ pub fn render_event(event: &PodEvent) -> String {
/// ///
/// - `TurnEnded` / `Errored`: no system work; the LLM handles the /// - `TurnEnded` / `Errored`: no system work; the LLM handles the
/// semantic response. /// semantic response.
/// - `ShutDown`: remove the child from `spawned_pods.json` and release /// - `ShutDown`: remove the child from `spawned_pods.json`, Pod state,
/// its scope allocation. Missing entries are swallowed. /// and reclaim its delegated scope/allocation. Missing entries are swallowed.
/// - `ScopeSubDelegated`: register the grandchild locally and re-emit /// - `ScopeSubDelegated`: register the grandchild locally and re-emit
/// upward to our own parent if we have one. Duplicate grandchild /// upward to our own parent if we have one. Duplicate grandchild
/// entries (re-delivery) are swallowed. /// entries (re-delivery) are swallowed.
@ -104,7 +103,6 @@ pub async fn apply_event_side_effects(
if let Err(e) = registry.remove(pod_name).await { if let Err(e) = registry.remove(pod_name).await {
tracing::warn!(error = %e, pod = %pod_name, "registry remove on ShutDown failed"); tracing::warn!(error = %e, pod = %pod_name, "registry remove on ShutDown failed");
} }
release_scope_silently(pod_name);
} }
PodEvent::ScopeSubDelegated { PodEvent::ScopeSubDelegated {
@ -145,28 +143,6 @@ pub async fn apply_event_side_effects(
} }
} }
fn release_scope_silently(pod_name: &str) {
let lock_path = match pod_registry::default_registry_path() {
Ok(p) => p,
Err(e) => {
tracing::warn!(error = %e, "default_registry_path failed");
return;
}
};
let mut guard = match pod_registry::LockFileGuard::open(&lock_path) {
Ok(g) => g,
Err(e) => {
tracing::warn!(error = %e, "LockFileGuard open failed");
return;
}
};
match pod_registry::release_pod(&mut guard, pod_name) {
Ok(()) => {}
Err(ScopeLockError::UnknownPod(_)) => {}
Err(e) => tracing::warn!(error = ?e, pod = %pod_name, "release_pod failed"),
}
}
fn reemit_scope_sub_delegated( fn reemit_scope_sub_delegated(
self_parent_socket: &Option<PathBuf>, self_parent_socket: &Option<PathBuf>,
self_name: &str, self_name: &str,

View File

@ -185,7 +185,9 @@ mod tests {
let item = build_system_item(&entry, &catalog).unwrap(); let item = build_system_item(&entry, &catalog).unwrap();
match item { match item {
SystemItem::PodEvent { event, body } => { SystemItem::PodEvent { event, body } => {
assert!(matches!(event, PodEvent::TurnEnded { ref pod_name } if pod_name == "child")); assert!(
matches!(event, PodEvent::TurnEnded { ref pod_name } if pod_name == "child")
);
assert!(body.contains("[Notification]")); assert!(body.contains("[Notification]"));
assert!(body.contains("`child`")); assert!(body.contains("`child`"));
} }

View File

@ -105,10 +105,10 @@ async fn handle_connection(stream: tokio::net::UnixStream, handle: PodHandle) {
match entry { match entry {
Ok(entry) => { Ok(entry) => {
let outbound = match entry { let outbound = match entry {
session_store::LogEntry::SessionStart { .. } => { session_store::LogEntry::SegmentStart { .. } => {
let value = serde_json::to_value(&entry) let value = serde_json::to_value(&entry)
.expect("LogEntry is Serialize"); .expect("LogEntry is Serialize");
Some(Event::SessionRotated { entry: value }) Some(Event::SegmentRotated { entry: value })
} }
session_store::LogEntry::SystemItem { item, .. } => { session_store::LogEntry::SystemItem { item, .. } => {
let value = serde_json::to_value(&item) let value = serde_json::to_value(&item)
@ -119,7 +119,7 @@ async fn handle_connection(stream: tokio::net::UnixStream, handle: PodHandle) {
Some(Event::InvokeStart { kind: trigger }) Some(Event::InvokeStart { kind: trigger })
} }
other => { other => {
// `SessionLogSink::is_live_relevant` keeps // `SegmentLogSink::is_live_relevant` keeps
// non-live-relevant variants off the // non-live-relevant variants off the
// broadcast lane; reaching here means the // broadcast lane; reaching here means the
// two are out of sync and we silently // two are out of sync and we silently

View File

@ -5,7 +5,7 @@ pub mod hook;
pub mod ipc; pub mod ipc;
pub mod prompt; pub mod prompt;
pub mod runtime; pub mod runtime;
pub mod session_log_sink; pub mod segment_log_sink;
pub mod shared_state; pub mod shared_state;
pub mod spawn; pub mod spawn;
pub mod workflow; pub mod workflow;
@ -31,5 +31,5 @@ pub use prompt::system::{SystemPromptContext, SystemPromptError, SystemPromptTem
pub use protocol::{ErrorCode, Event, Method, PodStatus, TurnResult}; pub use protocol::{ErrorCode, Event, Method, PodStatus, TurnResult};
pub use provider::{ProviderError, build_client}; pub use provider::{ProviderError, build_client};
pub use runtime::dir::RuntimeDir; pub use runtime::dir::RuntimeDir;
pub use session_log_sink::SessionLogSink; pub use segment_log_sink::SegmentLogSink;
pub use shared_state::PodSharedState; pub use shared_state::PodSharedState;

View File

@ -3,9 +3,9 @@ use std::path::{Path, PathBuf};
use std::process::ExitCode; use std::process::ExitCode;
use clap::Parser; use clap::Parser;
use manifest::{PodManifest, paths}; use manifest::{PodManifest, PodManifestConfig, paths};
use pod::{Pod, PodController, PodFactory, PromptLoader}; use pod::{Pod, PodController, PodFactory, PromptLoader};
use session_store::{FsStore, SessionId}; use session_store::{FsStore, PodMetadataStore, SegmentId, Store};
const USER_MANIFEST_ENV: &str = "INSOMNIA_USER_MANIFEST"; const USER_MANIFEST_ENV: &str = "INSOMNIA_USER_MANIFEST";
@ -47,13 +47,19 @@ struct Cli {
#[arg(long, value_name = "PATH", requires = "adopt")] #[arg(long, value_name = "PATH", requires = "adopt")]
callback: Option<PathBuf>, callback: Option<PathBuf>,
/// Resume or create a Pod by name. If name-keyed Pod state exists,
/// the active session/segment recorded there is restored; otherwise a
/// fresh top-level Pod is created with this name.
#[arg(long, value_name = "NAME", conflicts_with_all = ["session", "adopt"])]
pod: Option<String>,
/// Restore a Pod from an existing session. The Pod re-uses the /// Restore a Pod from an existing session. The Pod re-uses the
/// given session id and appends new turns to the same jsonl; /// given session id and appends new turns to the same jsonl;
/// concurrent writers are prevented by the pod-registry. /// concurrent writers are prevented by the pod-registry.
/// Mutually exclusive with `--adopt` (spawned children always start /// Mutually exclusive with `--adopt` (spawned children always start
/// fresh). /// fresh).
#[arg(long, value_name = "UUID", conflicts_with = "adopt")] #[arg(long, value_name = "UUID", conflicts_with_all = ["adopt", "pod"])]
session: Option<SessionId>, session: Option<SegmentId>,
} }
fn resolve_manifest(cli: &Cli) -> Result<(PodManifest, PromptLoader), String> { fn resolve_manifest(cli: &Cli) -> Result<(PodManifest, PromptLoader), String> {
@ -72,7 +78,7 @@ fn resolve_manifest_with_user_manifest_env(
"--manifest cannot be used when {USER_MANIFEST_ENV} is set" "--manifest cannot be used when {USER_MANIFEST_ENV} is set"
)); ));
} }
return load_single_manifest(path); return load_single_manifest(path, cli.pod.as_deref());
} }
let factory = build_factory_with_user_manifest_path(cli, user_manifest)?; let factory = build_factory_with_user_manifest_path(cli, user_manifest)?;
@ -91,14 +97,45 @@ fn user_manifest_path_from_env(value: Option<OsString>) -> Option<PathBuf> {
}) })
} }
fn load_single_manifest(path: &Path) -> Result<(PodManifest, PromptLoader), String> { fn load_single_manifest(
path: &Path,
pod_name_override: Option<&str>,
) -> Result<(PodManifest, PromptLoader), String> {
let toml = std::fs::read_to_string(path) let toml = std::fs::read_to_string(path)
.map_err(|e| format!("failed to read manifest {}: {e}", path.display()))?; .map_err(|e| format!("failed to read manifest {}: {e}", path.display()))?;
let manifest = PodManifest::from_toml(&toml) let manifest = match pod_name_override {
.map_err(|e| format!("failed to parse manifest {}: {e}", path.display()))?; Some(pod_name) => match PodManifest::from_toml(&toml) {
Ok(mut manifest) => {
manifest.pod.name = pod_name.to_string();
manifest
}
Err(_) => {
let base = PodManifestConfig::from_toml(&toml)
.map_err(|e| format!("failed to parse manifest {}: {e}", path.display()))?;
let overlay = PodManifestConfig::from_toml(&pod_name_overlay_toml(pod_name))
.expect("pod name overlay TOML is generated");
PodManifest::try_from(base.merge(overlay)).map_err(|e| {
format!(
"failed to resolve manifest {} with --pod: {e}",
path.display()
)
})?
}
},
None => PodManifest::from_toml(&toml)
.map_err(|e| format!("failed to parse manifest {}: {e}", path.display()))?,
};
Ok((manifest, PromptLoader::builtins_only())) Ok((manifest, PromptLoader::builtins_only()))
} }
fn pod_name_overlay_toml(pod_name: &str) -> String {
let mut pod = toml::value::Table::new();
pod.insert("name".into(), toml::Value::String(pod_name.to_string()));
let mut root = toml::value::Table::new();
root.insert("pod".into(), toml::Value::Table(pod));
toml::to_string(&toml::Value::Table(root)).expect("pod name overlay serialisation cannot fail")
}
fn build_factory_with_user_manifest_path( fn build_factory_with_user_manifest_path(
cli: &Cli, cli: &Cli,
user_manifest: Option<PathBuf>, user_manifest: Option<PathBuf>,
@ -129,6 +166,12 @@ fn build_factory_with_user_manifest_path(
.map_err(|e| format!("failed to parse overlay TOML: {e}"))?; .map_err(|e| format!("failed to parse overlay TOML: {e}"))?;
} }
if let Some(pod_name) = cli.pod.as_deref() {
factory = factory
.with_overlay_toml(&pod_name_overlay_toml(pod_name))
.map_err(|e| format!("failed to apply --pod overlay: {e}"))?;
}
Ok(factory) Ok(factory)
} }
@ -136,7 +179,7 @@ fn build_factory_with_user_manifest_path(
async fn main() -> ExitCode { async fn main() -> ExitCode {
let cli = Cli::parse(); let cli = Cli::parse();
let (manifest, loader) = match resolve_manifest(&cli) { let (mut manifest, loader) = match resolve_manifest(&cli) {
Ok(pair) => pair, Ok(pair) => pair,
Err(e) => { Err(e) => {
eprintln!("error: {e}"); eprintln!("error: {e}");
@ -185,14 +228,59 @@ async fn main() -> ExitCode {
return ExitCode::FAILURE; return ExitCode::FAILURE;
} }
} }
} else if let Some(source_session_id) = cli.session { } else if let Some(source_segment_id) = cli.session {
match Pod::restore_from_manifest(source_session_id, manifest, store, loader).await { let source_session_id = match store.lookup_session_of(source_segment_id) {
Ok(Some(sid)) => sid,
Ok(None) => {
eprintln!(
"error: --session {source_segment_id}: segment is not registered to any session"
);
return ExitCode::FAILURE;
}
Err(e) => {
eprintln!("error: lookup_session_of failed: {e}");
return ExitCode::FAILURE;
}
};
match Pod::restore_from_manifest(
source_session_id,
source_segment_id,
manifest,
store,
loader,
)
.await
{
Ok(p) => p, Ok(p) => p,
Err(e) => { Err(e) => {
eprintln!("error: failed to restore pod: {e}"); eprintln!("error: failed to restore pod: {e}");
return ExitCode::FAILURE; return ExitCode::FAILURE;
} }
} }
} else if let Some(pod_name) = cli.pod.as_deref() {
manifest.pod.name = pod_name.to_string();
match store.read_by_name(pod_name) {
Ok(Some(_)) => {
match Pod::restore_from_pod_metadata(pod_name, manifest, store, loader).await {
Ok(p) => p,
Err(e) => {
eprintln!("error: failed to restore pod {pod_name}: {e}");
return ExitCode::FAILURE;
}
}
}
Ok(None) => match Pod::from_manifest(manifest, store, loader).await {
Ok(p) => p,
Err(e) => {
eprintln!("error: failed to create pod {pod_name}: {e}");
return ExitCode::FAILURE;
}
},
Err(e) => {
eprintln!("error: failed to read pod state for {pod_name}: {e}");
return ExitCode::FAILURE;
}
}
} else { } else {
match Pod::from_manifest(manifest, store, loader).await { match Pod::from_manifest(manifest, store, loader).await {
Ok(p) => p, Ok(p) => p,
@ -348,6 +436,56 @@ permission = "write"
assert_eq!(manifest.pod.name, "from-env"); assert_eq!(manifest.pod.name, "from-env");
} }
#[test]
fn pod_flag_conflicts_with_session() {
let segment_id = session_store::new_segment_id();
let segment_id = segment_id.to_string();
let err =
Cli::try_parse_from(["pod", "--pod", "agent", "--session", &segment_id]).unwrap_err();
assert_eq!(err.kind(), clap::error::ErrorKind::ArgumentConflict);
}
#[test]
fn pod_flag_sets_requested_name_after_manifest_resolution() {
let tmp = TempDir::new().unwrap();
let manifest = tmp.path().join("manifest.toml");
write(&manifest, &manifest_toml("from-file", tmp.path()));
let cli = Cli::try_parse_from([
"pod",
"--manifest",
manifest.to_str().unwrap(),
"--pod",
"from-flag",
])
.unwrap();
let (manifest, _loader) = resolve_manifest_with_user_manifest_env(&cli, None).unwrap();
assert_eq!(manifest.pod.name, "from-flag");
}
#[test]
fn pod_flag_supplies_missing_name_for_single_manifest() {
let tmp = TempDir::new().unwrap();
let manifest = tmp.path().join("manifest.toml");
write(
&manifest,
&manifest_toml("unused", tmp.path()).replace("name = \"unused\"\n", ""),
);
let cli = Cli::try_parse_from([
"pod",
"--manifest",
manifest.to_str().unwrap(),
"--pod",
"from-flag",
])
.unwrap();
let (manifest, _loader) = resolve_manifest_with_user_manifest_env(&cli, None).unwrap();
assert_eq!(manifest.pod.name, "from-flag");
}
#[test] #[test]
fn manifest_mode_loads_single_file_with_minimal_prompt_loader() { fn manifest_mode_loads_single_file_with_minimal_prompt_loader() {
let tmp = TempDir::new().unwrap(); let tmp = TempDir::new().unwrap();

File diff suppressed because it is too large Load Diff

View File

@ -7,11 +7,11 @@ use tokio::fs;
use crate::shared_state::PodSharedState; use crate::shared_state::PodSharedState;
/// One spawned-child record persisted to `spawned_pods.json`. /// One spawned-child record mirrored to `spawned_pods.json`.
/// ///
/// Written by the spawner after a successful `SpawnPod` tool call so /// Written by the spawner after registry changes so runtime-local tools
/// `ListPods` (future ticket) and a restored spawner can enumerate /// have a materialised snapshot. Durable restore uses Pod state metadata;
/// their live children without re-querying `pods.json`. /// this file is not the authoritative source.
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SpawnedPodRecord { pub struct SpawnedPodRecord {
/// Spawned Pod's identity. /// Spawned Pod's identity.
@ -131,7 +131,7 @@ mod tests {
fn test_state() -> PodSharedState { fn test_state() -> PodSharedState {
PodSharedState::new( PodSharedState::new(
"test-pod".into(), "test-pod".into(),
session_store::new_session_id(), session_store::new_segment_id(),
"[pod]\nname = \"test-pod\"".into(), "[pod]\nname = \"test-pod\"".into(),
protocol::Greeting { protocol::Greeting {
pod_name: "test-pod".into(), pod_name: "test-pod".into(),
@ -140,6 +140,8 @@ mod tests {
model: "claude".into(), model: "claude".into(),
scope_summary: String::new(), scope_summary: String::new(),
tools: Vec::new(), tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
}, },
) )
} }

View File

@ -10,11 +10,11 @@
//! Atomicity contract: //! Atomicity contract:
//! //!
//! 1. Pod writes the entry to disk via the `Store`. //! 1. Pod writes the entry to disk via the `Store`.
//! 2. Pod calls [`SessionLogSink::publish`] which acquires the mirror //! 2. Pod calls [`SegmentLogSink::publish`] which acquires the mirror
//! mutex, pushes the entry, and fires `broadcast::send` — all under //! mutex, pushes the entry, and fires `broadcast::send` — all under
//! the same critical section. //! the same critical section.
//! //!
//! [`SessionLogSink::subscribe_with_snapshot`] takes the same mutex, //! [`SegmentLogSink::subscribe_with_snapshot`] takes the same mutex,
//! so the `(snapshot, receiver)` pair returned to a connecting client //! so the `(snapshot, receiver)` pair returned to a connecting client
//! splits the entry sequence cleanly: every entry shows up in exactly //! splits the entry sequence cleanly: every entry shows up in exactly
//! one of `snapshot` or on `receiver`. //! one of `snapshot` or on `receiver`.
@ -39,24 +39,24 @@ const BROADCAST_CAPACITY: usize = 256;
/// for read-only `subscribe_with_snapshot` access and keeps one for /// for read-only `subscribe_with_snapshot` access and keeps one for
/// its own write path. /// its own write path.
#[derive(Clone)] #[derive(Clone)]
pub struct SessionLogSink { pub struct SegmentLogSink {
inner: Arc<SinkInner>, inner: Arc<SinkInner>,
} }
struct SinkInner { struct SinkInner {
/// Full session log mirror in commit order. Reset on session swap /// Full session log mirror in commit order. Reset on session swap
/// (compaction / fork) via [`SessionLogSink::reset_with_initial`]. /// (compaction / fork) via [`SegmentLogSink::reset_with_initial`].
mirror: StdMutex<Vec<LogEntry>>, mirror: StdMutex<Vec<LogEntry>>,
/// Broadcast channel for live entry updates. The same `Sender` /// Broadcast channel for live entry updates. The same `Sender`
/// survives session swaps so existing subscribers keep their /// survives session swaps so existing subscribers keep their
/// receiver — they observe the swap as a freshly broadcast /// receiver — they observe the swap as a freshly broadcast
/// `LogEntry::SessionStart` and reset their view accordingly. /// `LogEntry::SegmentStart` and reset their view accordingly.
broadcast_tx: broadcast::Sender<LogEntry>, broadcast_tx: broadcast::Sender<LogEntry>,
} }
impl SessionLogSink { impl SegmentLogSink {
/// Create a fresh sink with an empty mirror. Used before any entry /// Create a fresh sink with an empty mirror. Used before any entry
/// has been written (deferred SessionStart) or as a placeholder in /// has been written (deferred SegmentStart) or as a placeholder in
/// tests. /// tests.
pub fn new() -> Self { pub fn new() -> Self {
let (broadcast_tx, _) = broadcast::channel(BROADCAST_CAPACITY); let (broadcast_tx, _) = broadcast::channel(BROADCAST_CAPACITY);
@ -89,7 +89,7 @@ impl SessionLogSink {
/// ///
/// Live broadcast fires only for entries that the streaming-event /// Live broadcast fires only for entries that the streaming-event
/// lane does not cover: /// lane does not cover:
/// - `LogEntry::SessionStart` → `Event::SessionRotated` on the wire. /// - `LogEntry::SegmentStart` → `Event::SegmentRotated` on the wire.
/// - `LogEntry::SystemItem` → `Event::SystemItem`. /// - `LogEntry::SystemItem` → `Event::SystemItem`.
/// - `LogEntry::Invoke` → `Event::InvokeStart`. /// - `LogEntry::Invoke` → `Event::InvokeStart`.
/// Everything else (AssistantItem, ToolResult, UserInput, TurnEnd, /// Everything else (AssistantItem, ToolResult, UserInput, TurnEnd,
@ -119,20 +119,18 @@ impl SessionLogSink {
fn is_live_relevant(entry: &LogEntry) -> bool { fn is_live_relevant(entry: &LogEntry) -> bool {
matches!( matches!(
entry, entry,
LogEntry::SessionStart { .. } LogEntry::SegmentStart { .. } | LogEntry::SystemItem { .. } | LogEntry::Invoke { .. }
| LogEntry::SystemItem { .. }
| LogEntry::Invoke { .. }
) )
} }
/// Atomically swap the mirror to `[initial]` and broadcast the new /// Atomically swap the mirror to `[initial]` and broadcast the new
/// session-start entry. Used during compaction / fork: the new /// session-start entry. Used during compaction / fork: the new
/// `LogEntry::SessionStart` is the first entry of the replacement /// `LogEntry::SegmentStart` is the first entry of the replacement
/// session, and existing subscribers transition by replaying it /// session, and existing subscribers transition by replaying it
/// like any other live entry. /// like any other live entry.
/// ///
/// Existing snapshot prefixes seen by old subscribers stay valid /// Existing snapshot prefixes seen by old subscribers stay valid
/// for the prior session; the new `SessionStart` on the broadcast /// for the prior session; the new `SegmentStart` on the broadcast
/// is the signal to reset their derived view. /// is the signal to reset their derived view.
pub fn reset_with_initial(&self, initial: LogEntry) { pub fn reset_with_initial(&self, initial: LogEntry) {
let mut mirror = self let mut mirror = self
@ -188,7 +186,7 @@ impl SessionLogSink {
} }
} }
impl Default for SessionLogSink { impl Default for SegmentLogSink {
fn default() -> Self { fn default() -> Self {
Self::new() Self::new()
} }
@ -198,11 +196,12 @@ impl Default for SessionLogSink {
mod tests { mod tests {
use super::*; use super::*;
use llm_worker::llm_client::RequestConfig; use llm_worker::llm_client::RequestConfig;
use session_store::session_log::now_millis; use session_store::segment_log::now_millis;
fn session_start() -> LogEntry { fn session_start() -> LogEntry {
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: now_millis(), ts: now_millis(),
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -220,13 +219,13 @@ mod tests {
#[test] #[test]
fn publish_then_subscribe_returns_history_in_snapshot() { fn publish_then_subscribe_returns_history_in_snapshot() {
let sink = SessionLogSink::new(); let sink = SegmentLogSink::new();
sink.publish(session_start()); sink.publish(session_start());
sink.publish(turn_end(1)); sink.publish(turn_end(1));
let (snapshot, mut rx) = sink.subscribe_with_snapshot(); let (snapshot, mut rx) = sink.subscribe_with_snapshot();
assert_eq!(snapshot.len(), 2); assert_eq!(snapshot.len(), 2);
assert!(matches!(snapshot[0], LogEntry::SessionStart { .. })); assert!(matches!(snapshot[0], LogEntry::SegmentStart { .. }));
assert!(matches!( assert!(matches!(
snapshot[1], snapshot[1],
LogEntry::TurnEnd { turn_count: 1, .. } LogEntry::TurnEnd { turn_count: 1, .. }
@ -246,7 +245,7 @@ mod tests {
#[test] #[test]
fn subscribe_then_publish_delivers_only_live_relevant_entries() { fn subscribe_then_publish_delivers_only_live_relevant_entries() {
let sink = SessionLogSink::new(); let sink = SegmentLogSink::new();
sink.publish(session_start()); sink.publish(session_start());
let (snapshot, mut rx) = sink.subscribe_with_snapshot(); let (snapshot, mut rx) = sink.subscribe_with_snapshot();
@ -270,7 +269,7 @@ mod tests {
#[test] #[test]
fn snapshot_and_live_never_overlap() { fn snapshot_and_live_never_overlap() {
let sink = SessionLogSink::new(); let sink = SegmentLogSink::new();
sink.publish(session_start()); sink.publish(session_start());
let (snapshot, mut rx) = sink.subscribe_with_snapshot(); let (snapshot, mut rx) = sink.subscribe_with_snapshot();
sink.publish(notification_entry("post-snapshot")); sink.publish(notification_entry("post-snapshot"));
@ -285,7 +284,7 @@ mod tests {
#[test] #[test]
fn reset_with_initial_clears_and_broadcasts() { fn reset_with_initial_clears_and_broadcasts() {
let sink = SessionLogSink::new(); let sink = SegmentLogSink::new();
sink.publish(session_start()); sink.publish(session_start());
sink.publish(turn_end(1)); sink.publish(turn_end(1));
@ -293,18 +292,18 @@ mod tests {
sink.reset_with_initial(session_start()); sink.reset_with_initial(session_start());
match rx.try_recv() { match rx.try_recv() {
Ok(LogEntry::SessionStart { .. }) => {} Ok(LogEntry::SegmentStart { .. }) => {}
other => panic!("expected SessionStart broadcast, got {other:?}"), other => panic!("expected SegmentStart broadcast, got {other:?}"),
} }
let (post_snapshot, _) = sink.subscribe_with_snapshot(); let (post_snapshot, _) = sink.subscribe_with_snapshot();
assert_eq!(post_snapshot.len(), 1); assert_eq!(post_snapshot.len(), 1);
assert!(matches!(post_snapshot[0], LogEntry::SessionStart { .. })); assert!(matches!(post_snapshot[0], LogEntry::SegmentStart { .. }));
} }
#[test] #[test]
fn replace_silent_does_not_broadcast() { fn replace_silent_does_not_broadcast() {
let sink = SessionLogSink::new(); let sink = SegmentLogSink::new();
sink.publish(session_start()); sink.publish(session_start());
let (_pre_snapshot, mut rx) = sink.subscribe_with_snapshot(); let (_pre_snapshot, mut rx) = sink.subscribe_with_snapshot();
@ -318,7 +317,7 @@ mod tests {
#[test] #[test]
fn with_initial_seeds_the_mirror() { fn with_initial_seeds_the_mirror() {
let sink = SessionLogSink::with_initial(vec![session_start(), turn_end(1)]); let sink = SegmentLogSink::with_initial(vec![session_start(), turn_end(1)]);
let (snapshot, _) = sink.subscribe_with_snapshot(); let (snapshot, _) = sink.subscribe_with_snapshot();
assert_eq!(snapshot.len(), 2); assert_eq!(snapshot.len(), 2);
} }

View File

@ -2,7 +2,7 @@ use std::sync::{OnceLock, RwLock};
use protocol::PodStatus; use protocol::PodStatus;
use serde_json::json; use serde_json::json;
use session_store::SessionId; use session_store::SegmentId;
use crate::fs_view::PodFsView; use crate::fs_view::PodFsView;
@ -28,7 +28,7 @@ pub struct KnowledgeCandidate {
/// greeting, and completion lookup hubs. /// greeting, and completion lookup hubs.
pub struct PodSharedState { pub struct PodSharedState {
pub pod_name: String, pub pod_name: String,
pub session_id: SessionId, pub segment_id: SegmentId,
pub manifest_toml: String, pub manifest_toml: String,
pub greeting: protocol::Greeting, pub greeting: protocol::Greeting,
pub status: RwLock<PodStatus>, pub status: RwLock<PodStatus>,
@ -46,13 +46,13 @@ pub struct PodSharedState {
impl PodSharedState { impl PodSharedState {
pub fn new( pub fn new(
pod_name: String, pod_name: String,
session_id: SessionId, segment_id: SegmentId,
manifest_toml: String, manifest_toml: String,
greeting: protocol::Greeting, greeting: protocol::Greeting,
) -> Self { ) -> Self {
Self { Self {
pod_name, pod_name,
session_id, segment_id,
manifest_toml, manifest_toml,
greeting, greeting,
status: RwLock::new(PodStatus::Idle), status: RwLock::new(PodStatus::Idle),
@ -123,7 +123,7 @@ impl PodSharedState {
let status = self.get_status(); let status = self.get_status();
json!({ json!({
"state": status, "state": status,
"session_id": self.session_id.to_string(), "segment_id": self.segment_id.to_string(),
"pod_name": self.pod_name, "pod_name": self.pod_name,
}) })
.to_string() .to_string()
@ -137,7 +137,7 @@ mod tests {
fn test_state() -> PodSharedState { fn test_state() -> PodSharedState {
PodSharedState::new( PodSharedState::new(
"test-pod".into(), "test-pod".into(),
session_store::new_session_id(), session_store::new_segment_id(),
"[pod]\nname = \"test-pod\"".into(), "[pod]\nname = \"test-pod\"".into(),
test_greeting(), test_greeting(),
) )
@ -151,6 +151,8 @@ mod tests {
model: "claude".into(), model: "claude".into(),
scope_summary: String::new(), scope_summary: String::new(),
tools: Vec::new(), tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
} }
} }
@ -176,7 +178,7 @@ mod tests {
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap(); let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["state"], "idle"); assert_eq!(parsed["state"], "idle");
assert_eq!(parsed["pod_name"], "test-pod"); assert_eq!(parsed["pod_name"], "test-pod");
assert!(parsed["session_id"].is_string()); assert!(parsed["segment_id"].is_string());
} }
#[test] #[test]

View File

@ -6,7 +6,7 @@
//! target's Unix socket, perform one method exchange, disconnect. //! target's Unix socket, perform one method exchange, disconnect.
//! //!
//! These tools only touch Pods listed in the spawner's //! These tools only touch Pods listed in the spawner's
//! `spawned_pods.json`; there is no machine-wide directory lookup, so //! `SpawnedPodRegistry`; there is no machine-wide directory lookup, so
//! the spawner can only reach its own descendants. //! the spawner can only reach its own descendants.
use std::path::Path; use std::path::Path;
@ -204,20 +204,17 @@ impl Tool for StopPodTool {
.ok_or_else(|| unknown_pod_err(&input.name))?; .ok_or_else(|| unknown_pod_err(&input.name))?;
// Best-effort Shutdown. The child's own `ScopeAllocationGuard` // Best-effort Shutdown. The child's own `ScopeAllocationGuard`
// releases the entry on clean exit; we also release explicitly // releases its entry on clean exit; the parent reclaim below is the
// below so callers can't observe a window where the scope is // authoritative operation for removing the child record and returning
// still registered but StopPod has returned. Duplicate release // delegated Write scope to the spawner.
// is harmless — `ScopeAllocationGuard`'s drop path swallows
// `UnknownPod` errors.
let _ = connect_and_send(&record.socket_path, &Method::Shutdown).await; let _ = connect_and_send(&record.socket_path, &Method::Shutdown).await;
let scope_summary = summarize_scope(&record); let scope_summary = summarize_scope(&record);
release_scope(&record.pod_name);
self.registry self.registry
.remove(&record.pod_name) .remove(&record.pod_name)
.await .await
.map_err(|e| ToolError::ExecutionFailed(format!("update spawned_pods.json: {e}")))?; .map_err(|e| ToolError::ExecutionFailed(format!("update spawned pod registry: {e}")))?;
Ok(ToolOutput { Ok(ToolOutput {
summary: format!( summary: format!(
@ -321,21 +318,52 @@ fn unknown_pod_err(name: &str) -> ToolError {
ToolError::InvalidArgument(format!("no spawned pod named `{name}`")) ToolError::InvalidArgument(format!("no spawned pod named `{name}`"))
} }
/// Connect with a timeout, write one `Method` line, flush, and close. /// Connect with a timeout, drain the server's connect-time snapshot,
/// Any socket error maps to an `io::Error`; the caller decides whether /// write one `Method` line, flush, and close.
/// to surface it to the LLM or treat it as "pod stopped". ///
/// The Pod socket protocol sends replayed alerts and an initial
/// `Event::Snapshot` before it starts reading client methods. Send-only
/// callers must consume that prefix; otherwise a large snapshot can block
/// the server's writer before it reaches the method-read branch. Any
/// socket error maps to an `io::Error`; the caller decides whether to
/// surface it to the LLM or treat it as "pod stopped".
pub(crate) async fn connect_and_send(socket: &Path, method: &Method) -> std::io::Result<()> { pub(crate) async fn connect_and_send(socket: &Path, method: &Method) -> std::io::Result<()> {
let stream = tokio::time::timeout(SOCKET_OP_TIMEOUT, UnixStream::connect(socket)) let stream = tokio::time::timeout(SOCKET_OP_TIMEOUT, UnixStream::connect(socket))
.await .await
.map_err(|_| std::io::Error::new(std::io::ErrorKind::TimedOut, "connect timed out"))??; .map_err(|_| std::io::Error::new(std::io::ErrorKind::TimedOut, "connect timed out"))??;
let (_r, w) = stream.into_split(); let (r, w) = stream.into_split();
let mut reader = JsonLineReader::new(r);
let mut writer = JsonLineWriter::new(w); let mut writer = JsonLineWriter::new(w);
drain_initial_snapshot(&mut reader).await?;
tokio::time::timeout(SOCKET_OP_TIMEOUT, writer.write(method)) tokio::time::timeout(SOCKET_OP_TIMEOUT, writer.write(method))
.await .await
.map_err(|_| std::io::Error::new(std::io::ErrorKind::TimedOut, "write timed out"))??; .map_err(|_| std::io::Error::new(std::io::ErrorKind::TimedOut, "write timed out"))??;
Ok(()) Ok(())
} }
async fn drain_initial_snapshot<R>(reader: &mut JsonLineReader<R>) -> std::io::Result<()>
where
R: tokio::io::AsyncBufRead + Unpin,
{
loop {
let event = tokio::time::timeout(SOCKET_OP_TIMEOUT, reader.next::<Event>())
.await
.map_err(|_| std::io::Error::new(std::io::ErrorKind::TimedOut, "read timed out"))??;
match event {
Some(Event::Snapshot { .. }) => return Ok(()),
Some(_) => continue,
None => {
return Err(std::io::Error::new(
std::io::ErrorKind::UnexpectedEof,
"pod closed connection before Snapshot event",
));
}
}
}
}
/// Failure modes distinguished by `SendToPod`. /// Failure modes distinguished by `SendToPod`.
enum SendRunError { enum SendRunError {
/// Target Pod responded with `Error { AlreadyRunning }` — the /// Target Pod responded with `Error { AlreadyRunning }` — the
@ -429,39 +457,43 @@ fn extract_assistant_text(entries: &[serde_json::Value]) -> String {
let mut out = String::new(); let mut out = String::new();
for value in entries { for value in entries {
// The wire payload is the JSON form of `session_store::LogEntry`. // The wire payload is the JSON form of `session_store::LogEntry`.
// Walk Assistant items inside each entry that can carry them: // Walk current singular assistant items and the seeded history in
// post-compaction `SessionStart.history` (seed) and per-LLM-call // post-compaction `SegmentStart` entries.
// `AssistantItems` deltas.
let Ok(entry) = serde_json::from_value::<LogEntry>(value.clone()) else { let Ok(entry) = serde_json::from_value::<LogEntry>(value.clone()) else {
continue; continue;
}; };
let logged_items = match entry { match entry {
LogEntry::SessionStart { history, .. } => history, LogEntry::SegmentStart { history, .. } => {
LogEntry::AssistantItems { items, .. } => items, for logged in history {
_ => continue, push_assistant_text(&mut out, logged);
};
for logged in logged_items {
let item: Item = logged.into();
if let Item::Message {
role: Role::Assistant,
content,
..
} = item
{
for part in content {
if let ContentPart::Text { text } = part {
if !out.is_empty() {
out.push_str("\n\n");
}
out.push_str(&text);
}
} }
} }
LogEntry::AssistantItem { item, .. } => push_assistant_text(&mut out, item),
_ => continue,
} }
} }
out out
} }
fn push_assistant_text(out: &mut String, logged: session_store::LoggedItem) {
let item: Item = logged.into();
if let Item::Message {
role: Role::Assistant,
content,
..
} = item
{
for part in content {
if let ContentPart::Text { text } = part {
if !out.is_empty() {
out.push_str("\n\n");
}
out.push_str(&text);
}
}
}
}
fn summarize_scope(record: &SpawnedPodRecord) -> String { fn summarize_scope(record: &SpawnedPodRecord) -> String {
if record.scope_delegated.is_empty() { if record.scope_delegated.is_empty() {
return "(none)".into(); return "(none)".into();
@ -481,16 +513,94 @@ fn summarize_scope(record: &SpawnedPodRecord) -> String {
parts.join(", ") parts.join(", ")
} }
/// Best-effort release of the pod's scope allocation. Swallows every #[cfg(test)]
/// error: the caller has already completed its user-visible side mod tests {
/// effects (Method::Shutdown was sent), and stale-reclaim will clean use super::*;
/// up whatever we couldn't.
fn release_scope(pod_name: &str) { use protocol::{Alert, AlertLevel, AlertSource, Greeting, PodEvent, PodStatus};
let Ok(lock_path) = pod_registry::default_registry_path() else { use tempfile::TempDir;
return; use tokio::net::UnixListener;
}; use tokio::task::JoinHandle;
let Ok(mut guard) = LockFileGuard::open(&lock_path) else {
return; fn snapshot(entries: Vec<serde_json::Value>) -> Event {
}; Event::Snapshot {
let _ = pod_registry::release_pod(&mut guard, pod_name); entries,
greeting: Greeting {
pod_name: "server".into(),
cwd: "/tmp".into(),
provider: "test".into(),
model: "test".into(),
scope_summary: String::new(),
tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
},
status: PodStatus::Idle,
}
}
fn serve_initial_events_then_method(
listener: UnixListener,
events: Vec<Event>,
) -> JoinHandle<Option<Method>> {
tokio::spawn(async move {
let (stream, _) = listener.accept().await.ok()?;
let (r, w) = stream.into_split();
let mut reader = JsonLineReader::new(r);
let mut writer = JsonLineWriter::new(w);
for event in events {
writer.write(&event).await.ok()?;
}
reader.next::<Method>().await.ok().flatten()
})
}
#[tokio::test]
async fn connect_and_send_drains_initial_alert_and_snapshot_before_method() {
let tmp = TempDir::new().unwrap();
let socket = tmp.path().join("pod.sock");
let listener = UnixListener::bind(&socket).unwrap();
let received = serve_initial_events_then_method(
listener,
vec![
Event::Alert(Alert {
level: AlertLevel::Warn,
source: AlertSource::Pod,
message: "replayed alert".into(),
timestamp_ms: 0,
}),
snapshot(Vec::new()),
],
);
connect_and_send(&socket, &Method::Shutdown).await.unwrap();
let method = received.await.unwrap().expect("expected method");
assert!(matches!(method, Method::Shutdown));
}
#[tokio::test]
async fn connect_and_send_delivers_method_after_large_initial_snapshot() {
let tmp = TempDir::new().unwrap();
let socket = tmp.path().join("pod.sock");
let listener = UnixListener::bind(&socket).unwrap();
let large_payload = "x".repeat(2 * 1024 * 1024);
let received = serve_initial_events_then_method(
listener,
vec![snapshot(vec![
serde_json::json!({ "payload": large_payload }),
])],
);
let expected = Method::PodEvent(PodEvent::TurnEnded {
pod_name: "child".into(),
});
connect_and_send(&socket, &expected).await.unwrap();
let method = received.await.unwrap().expect("expected method");
match method {
Method::PodEvent(PodEvent::TurnEnded { pod_name }) => assert_eq!(pod_name, "child"),
other => panic!("expected TurnEnded PodEvent, got {other:?}"),
}
}
} }

View File

@ -2,30 +2,53 @@
//! //!
//! `SpawnPod` writes here; the pod-comm tools (`SendToPod`, //! `SpawnPod` writes here; the pod-comm tools (`SendToPod`,
//! `ReadPodOutput`, `StopPod`, `ListPods`) read and mutate the same //! `ReadPodOutput`, `StopPod`, `ListPods`) read and mutate the same
//! instance. Persisted to `spawned_pods.json` in the spawner's runtime //! instance. Runtime write-through still materialises `spawned_pods.json`,
//! dir so a restarted spawner rebuilds its view from disk (future work //! but durable state lives in the spawner's Pod metadata.
//! — today only write-through is implemented).
//! //!
//! `ReadPodOutput` additionally owns a per-spawned-pod cursor here so //! `ReadPodOutput` additionally owns a per-spawned-pod cursor here so
//! two consecutive reads yield only new assistant text. The cursor is //! two consecutive reads yield only new assistant text. The cursor is
//! an item-index into the child's history; push-only history makes //! an item-index into the child's history; push-only history makes
//! index stable across reads. //! index stable across reads.
//! //!
//! The registry stays in-memory only for this Pod's lifetime — cursors //! Cursors intentionally do not persist; a restored registry starts with
//! intentionally do not persist. //! fresh read positions.
use std::collections::HashMap; use std::collections::HashMap;
use std::io; use std::io;
use std::path::Path;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration;
use manifest::{Permission, ScopeRule, SharedScope};
use session_store::{
PodMetadata, PodMetadataStore, PodScopeSnapshot, PodSpawnedChild, PodSpawnedScopeRule,
StoreError,
};
use tokio::net::UnixStream;
use tokio::sync::Mutex; use tokio::sync::Mutex;
use tracing::warn;
use crate::runtime::dir::{RuntimeDir, SpawnedPodRecord}; use crate::runtime::dir::{RuntimeDir, SpawnedPodRecord};
use crate::runtime::pod_registry;
type RegistryStateWriter = Arc<dyn Fn(&[SpawnedPodRecord]) -> io::Result<()> + Send + Sync>;
type ScopeChangeSink = Arc<dyn Fn(PodScopeSnapshot) + Send + Sync>;
const RESTORE_REACHABILITY_TIMEOUT: Duration = Duration::from_millis(500);
pub struct SpawnedPodRegistry { pub struct SpawnedPodRegistry {
records: Mutex<Vec<SpawnedPodRecord>>, records: Mutex<Vec<SpawnedPodRecord>>,
cursors: Mutex<HashMap<String, usize>>, cursors: Mutex<HashMap<String, usize>>,
runtime_dir: Arc<RuntimeDir>, runtime_dir: Arc<RuntimeDir>,
state_writer: Option<RegistryStateWriter>,
parent_name: Option<String>,
parent_scope: Option<SharedScope>,
scope_change_sink: Option<ScopeChangeSink>,
}
pub struct SpawnedPodRegistryLoad {
pub registry: Arc<SpawnedPodRegistry>,
pub reclaimed_unreachable: bool,
} }
impl SpawnedPodRegistry { impl SpawnedPodRegistry {
@ -34,18 +57,111 @@ impl SpawnedPodRegistry {
records: Mutex::new(Vec::new()), records: Mutex::new(Vec::new()),
cursors: Mutex::new(HashMap::new()), cursors: Mutex::new(HashMap::new()),
runtime_dir, runtime_dir,
state_writer: None,
parent_name: None,
parent_scope: None,
scope_change_sink: None,
})
}
/// Build a registry from the spawner's durable Pod state, pruning child
/// records whose socket path is already gone. The surviving list is
/// written through to both `spawned_pods.json` and Pod state so runtime
/// and durable views start aligned.
pub async fn load_from_pod_state<St>(
runtime_dir: Arc<RuntimeDir>,
store: St,
pod_name: String,
) -> io::Result<Arc<Self>>
where
St: PodMetadataStore + Clone + Send + Sync + 'static,
{
let loaded =
Self::load_from_pod_state_with_reclaim(runtime_dir, store, pod_name, None, None)
.await?;
Ok(loaded.registry)
}
pub async fn load_from_pod_state_with_reclaim<St>(
runtime_dir: Arc<RuntimeDir>,
store: St,
pod_name: String,
parent_scope: Option<SharedScope>,
scope_change_sink: Option<ScopeChangeSink>,
) -> io::Result<SpawnedPodRegistryLoad>
where
St: PodMetadataStore + Clone + Send + Sync + 'static,
{
let metadata = store.read_by_name(&pod_name).map_err(store_error_to_io)?;
let persisted_children = metadata
.as_ref()
.map(|m| m.spawned_children.clone())
.unwrap_or_default();
let mut records = Vec::with_capacity(persisted_children.len());
let mut pruned = false;
let mut pruned_records = Vec::new();
for child in &persisted_children {
let record = match record_from_pod_state(child) {
Ok(record) => record,
Err(err) => {
pruned = true;
warn!(
error = %err,
pod = %child.pod_name,
"dropping corrupt persisted spawned-pod record"
);
continue;
}
};
if is_reachable(&record.socket_path).await {
records.push(record);
} else {
pruned = true;
warn!(
pod = %record.pod_name,
socket = %record.socket_path.display(),
"dropping unreachable persisted spawned-pod record"
);
pruned_records.push(record);
}
}
runtime_dir.write_spawned_pods(&records).await?;
let state_writer = pod_state_writer(store, pod_name.clone());
if pruned || metadata.is_some() {
state_writer(&records)?;
}
let mut reclaimed_unreachable = false;
if parent_scope.is_some() {
for record in &pruned_records {
reclaim_record(&pod_name, parent_scope.as_ref(), None, record)?;
reclaimed_unreachable = true;
}
}
Ok(SpawnedPodRegistryLoad {
registry: Arc::new(Self {
records: Mutex::new(records),
cursors: Mutex::new(HashMap::new()),
runtime_dir,
state_writer: Some(state_writer),
parent_name: Some(pod_name),
parent_scope,
scope_change_sink,
}),
reclaimed_unreachable,
}) })
} }
/// Append a new record and persist the full list. Returns an I/O /// Append a new record and persist the full list. Returns an I/O
/// error if the persisted write fails; the in-memory state is still /// error if either persisted write fails; the in-memory state is still
/// updated in that case — the next successful write will reconcile. /// updated in that case — the next successful write will reconcile.
pub async fn add(&self, record: SpawnedPodRecord) -> io::Result<()> { pub async fn add(&self, record: SpawnedPodRecord) -> io::Result<()> {
let mut records = self.records.lock().await; let mut records = self.records.lock().await;
records.push(record); records.push(record);
self.runtime_dir self.persist_records(records.as_slice()).await
.write_spawned_pods(records.as_slice())
.await
} }
/// Look up a record by pod name. Cloned so callers can drop the lock. /// Look up a record by pod name. Cloned so callers can drop the lock.
@ -62,22 +178,37 @@ impl SpawnedPodRegistry {
self.records.lock().await.clone() self.records.lock().await.clone()
} }
/// Remove the record for `pod_name`, persist, and clear its cursor. /// Remove the record for `pod_name`, persist, clear its cursor, and
/// Returns the removed record (if any). /// reclaim any delegated Write scope owned by that child. Returns the
/// removed record (if any).
pub async fn remove(&self, pod_name: &str) -> io::Result<Option<SpawnedPodRecord>> { pub async fn remove(&self, pod_name: &str) -> io::Result<Option<SpawnedPodRecord>> {
let removed = { let removed = {
let mut records = self.records.lock().await; let mut records = self.records.lock().await;
let idx = records.iter().position(|r| r.pod_name == pod_name); let idx = records.iter().position(|r| r.pod_name == pod_name);
let removed = idx.map(|i| records.remove(i)); let removed = idx.map(|i| records.remove(i));
self.runtime_dir self.persist_records(records.as_slice()).await?;
.write_spawned_pods(records.as_slice())
.await?;
removed removed
}; };
self.cursors.lock().await.remove(pod_name); self.cursors.lock().await.remove(pod_name);
if let Some(record) = &removed {
self.reclaim_record(record)?;
}
Ok(removed) Ok(removed)
} }
fn reclaim_record(&self, record: &SpawnedPodRecord) -> io::Result<()> {
let Some(parent_name) = &self.parent_name else {
release_child_allocation(&record.pod_name)?;
return Ok(());
};
reclaim_record(
parent_name,
self.parent_scope.as_ref(),
self.scope_change_sink.as_ref(),
record,
)
}
/// Read-only cursor lookup. Returns 0 when no cursor has been set. /// Read-only cursor lookup. Returns 0 when no cursor has been set.
pub async fn cursor(&self, pod_name: &str) -> usize { pub async fn cursor(&self, pod_name: &str) -> usize {
self.cursors self.cursors
@ -94,4 +225,150 @@ impl SpawnedPodRegistry {
.await .await
.insert(pod_name.to_string(), cursor); .insert(pod_name.to_string(), cursor);
} }
async fn persist_records(&self, records: &[SpawnedPodRecord]) -> io::Result<()> {
self.runtime_dir.write_spawned_pods(records).await?;
if let Some(write_state) = &self.state_writer {
write_state(records)?;
}
Ok(())
}
}
fn pod_state_writer<St>(store: St, pod_name: String) -> RegistryStateWriter
where
St: PodMetadataStore + Clone + Send + Sync + 'static,
{
Arc::new(move |records| {
write_records_to_pod_state(&store, &pod_name, records).map_err(store_error_to_io)
})
}
fn reclaim_record(
parent_name: &str,
parent_scope: Option<&SharedScope>,
scope_change_sink: Option<&ScopeChangeSink>,
record: &SpawnedPodRecord,
) -> io::Result<()> {
let write_rules = record
.scope_delegated
.iter()
.filter(|rule| rule.permission == Permission::Write)
.cloned()
.collect::<Vec<_>>();
let lock_path = pod_registry::default_registry_path()
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
let mut guard = pod_registry::LockFileGuard::open(&lock_path)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
pod_registry::reclaim_delegated_scope(
&mut guard,
parent_name,
&record.pod_name,
&record.scope_delegated,
)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
if let Some(scope) = parent_scope {
scope
.update(|current| current.with_removed_deny_rules(write_rules))
.map_err(|err| io::Error::new(io::ErrorKind::InvalidInput, err))?;
if let Some(sink) = scope_change_sink {
let snapshot = scope.snapshot();
sink(PodScopeSnapshot {
allow: snapshot.allow_rules(),
deny: snapshot.deny_rules(),
});
}
}
Ok(())
}
fn release_child_allocation(pod_name: &str) -> io::Result<()> {
let lock_path = pod_registry::default_registry_path()
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
let mut guard = pod_registry::LockFileGuard::open(&lock_path)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
match pod_registry::release_pod(&mut guard, pod_name) {
Ok(()) | Err(pod_registry::ScopeLockError::UnknownPod(_)) => Ok(()),
Err(err) => Err(io::Error::new(io::ErrorKind::Other, err)),
}
}
fn write_records_to_pod_state<St>(
store: &St,
pod_name: &str,
records: &[SpawnedPodRecord],
) -> Result<(), StoreError>
where
St: PodMetadataStore,
{
let mut metadata = store
.read_by_name(pod_name)?
.unwrap_or_else(|| PodMetadata::new(pod_name, None));
metadata.spawned_children = records
.iter()
.map(record_to_pod_state)
.collect::<Result<Vec<_>, _>>()?;
store.write(&metadata)
}
fn record_to_pod_state(record: &SpawnedPodRecord) -> Result<PodSpawnedChild, serde_json::Error> {
Ok(PodSpawnedChild {
pod_name: record.pod_name.clone(),
socket_path: record.socket_path.clone(),
scope_delegated: record
.scope_delegated
.iter()
.map(|rule| PodSpawnedScopeRule {
target: rule.target.clone(),
permission: match rule.permission {
Permission::Read => "read".to_string(),
Permission::Write => "write".to_string(),
},
recursive: rule.recursive,
})
.collect(),
callback_address: record.callback_address.clone(),
})
}
fn record_from_pod_state(child: &PodSpawnedChild) -> Result<SpawnedPodRecord, serde_json::Error> {
Ok(SpawnedPodRecord {
pod_name: child.pod_name.clone(),
socket_path: child.socket_path.clone(),
scope_delegated: child
.scope_delegated
.iter()
.map(|rule| {
Ok(ScopeRule {
target: rule.target.clone(),
permission: match rule.permission.as_str() {
"read" => Permission::Read,
"write" => Permission::Write,
other => {
return Err(serde_json::Error::io(io::Error::new(
io::ErrorKind::InvalidData,
format!("invalid permission `{other}`"),
)));
}
},
recursive: rule.recursive,
})
})
.collect::<Result<Vec<_>, _>>()?,
callback_address: child.callback_address.clone(),
})
}
fn store_error_to_io(error: StoreError) -> io::Error {
io::Error::other(error)
}
async fn is_reachable(socket: &Path) -> bool {
tokio::time::timeout(RESTORE_REACHABILITY_TIMEOUT, UnixStream::connect(socket))
.await
.map(|result| result.is_ok())
.unwrap_or(false)
} }

View File

@ -107,7 +107,8 @@ pub struct SpawnPodTool {
spawner_pwd: PathBuf, spawner_pwd: PathBuf,
/// Shared registry of spawned children, also used by the /// Shared registry of spawned children, also used by the
/// pod-comm tools (`SendToPod` / `ReadPodOutput` / `StopPod` / /// pod-comm tools (`SendToPod` / `ReadPodOutput` / `StopPod` /
/// `ListPods`). Writes the list to `spawned_pods.json` on each add. /// `ListPods`). Writes the list to runtime and durable Pod state on
/// each add.
registry: Arc<SpawnedPodRegistry>, registry: Arc<SpawnedPodRegistry>,
/// THIS Pod's own parent-callback socket, if any. After a /// THIS Pod's own parent-callback socket, if any. After a
/// successful spawn we fire `PodEvent::ScopeSubDelegated` upward /// successful spawn we fire `PodEvent::ScopeSubDelegated` upward
@ -268,7 +269,7 @@ impl Tool for SpawnPodTool {
self.registry self.registry
.add(record) .add(record)
.await .await
.map_err(|e| ToolError::ExecutionFailed(format!("write spawned_pods.json: {e}")))?; .map_err(|e| ToolError::ExecutionFailed(format!("write spawned pod registry: {e}")))?;
// Notify this Pod's own parent so the grandparent can register // Notify this Pod's own parent so the grandparent can register
// the new grandchild directly. Fire-and-forget; top-level Pods // the new grandchild directly. Fire-and-forget; top-level Pods
@ -482,7 +483,7 @@ fn pod_registry_err_to_tool(e: ScopeLockError) -> ToolError {
| ScopeLockError::WriteConflict { .. } | ScopeLockError::WriteConflict { .. }
| ScopeLockError::DuplicatePodName(_) | ScopeLockError::DuplicatePodName(_)
| ScopeLockError::UnknownPod(_) | ScopeLockError::UnknownPod(_)
| ScopeLockError::SessionConflict { .. } => ToolError::InvalidArgument(e.to_string()), | ScopeLockError::SegmentConflict { .. } => ToolError::InvalidArgument(e.to_string()),
ScopeLockError::Io(_) => ToolError::ExecutionFailed(e.to_string()), ScopeLockError::Io(_) => ToolError::ExecutionFailed(e.to_string()),
} }
} }

View File

@ -17,7 +17,7 @@ use llm_worker::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEve
use llm_worker::llm_client::types::Item; use llm_worker::llm_client::types::Item;
use llm_worker::llm_client::{ClientError, LlmClient, Request}; use llm_worker::llm_client::{ClientError, LlmClient, Request};
use protocol::Event; use protocol::Event;
use session_store::FsStore; use session_store::{FsStore, LogEntry, PodMetadataStore, Store};
use tokio::sync::broadcast; use tokio::sync::broadcast;
use pod::Pod; use pod::Pod;
@ -158,7 +158,9 @@ async fn make_pod_with_manifest(
std::mem::forget(pwd_tmp); std::mem::forget(pwd_tmp);
let worker = Worker::new(client); let worker = Worker::new(client);
Pod::new(manifest, worker, store, pwd, scope).await.unwrap() let mut pod = Pod::new(manifest, worker, store, pwd, scope).await.unwrap();
pod.enable_pod_metadata_write_through().unwrap();
pod
} }
async fn make_pod(client: MockClient) -> Pod<MockClient, FsStore> { async fn make_pod(client: MockClient) -> Pod<MockClient, FsStore> {
@ -178,7 +180,7 @@ fn drain(rx: &mut broadcast::Receiver<Event>) -> Vec<Event> {
} }
/// Collect every system-message text that the post-compaction /// Collect every system-message text that the post-compaction
/// `SessionStart.history` carries, by reading the sink mirror directly. /// `SegmentStart.history` carries, by reading the sink mirror directly.
fn system_texts_in_sink_session_start( fn system_texts_in_sink_session_start(
pod: &pod::Pod< pod: &pod::Pod<
impl llm_worker::llm_client::client::LlmClient + Clone + 'static, impl llm_worker::llm_client::client::LlmClient + Clone + 'static,
@ -187,7 +189,7 @@ fn system_texts_in_sink_session_start(
) -> Vec<String> { ) -> Vec<String> {
let (entries, _rx) = pod.sink().subscribe_with_snapshot(); let (entries, _rx) = pod.sink().subscribe_with_snapshot();
for entry in entries.into_iter().rev() { for entry in entries.into_iter().rev() {
if let session_store::LogEntry::SessionStart { history, .. } = entry { if let session_store::LogEntry::SegmentStart { history, .. } = entry {
return history return history
.into_iter() .into_iter()
.filter_map(|logged| { .filter_map(|logged| {
@ -213,6 +215,129 @@ fn system_texts_in_sink_session_start(
Vec::new() Vec::new()
} }
/// Pod metadata starts with a reserved Session and no Segment, then becomes
/// active once the first SegmentStart is materialized by `run`.
#[tokio::test]
async fn pod_metadata_moves_from_pending_to_active_on_first_run() {
let client = MockClient::new(vec![single_text_events("hi")]);
let mut pod = make_pod(client).await;
let store = pod.store().clone();
let session_id = pod.session_id();
let initial_segment_id = pod.segment_id();
let pending = store
.read_by_name("test-pod")
.unwrap()
.expect("metadata should be initialized at Pod construction");
assert_eq!(pending.pod_name, "test-pod");
let pending_active = pending.active.expect("active session pointer missing");
assert_eq!(pending_active.session_id, session_id);
assert_eq!(pending_active.segment_id, None);
pod.run_text("first").await.unwrap();
let resolved = store
.read_by_name("test-pod")
.unwrap()
.expect("metadata should still exist after first run");
let active = resolved.active.expect("active session pointer missing");
assert_eq!(active.session_id, session_id);
assert_eq!(active.segment_id, Some(initial_segment_id));
}
/// Live auto-fork: when another writer extends the segment behind the
/// Pod's back, the next run's `ensure_segment_head` detects the
/// entry-count drift and branches into a fresh segment **within the same
/// Session**. The source segment is left immutable (no terminal marker
/// written back); the new segment records its parentage forward via
/// `SegmentStart.forked_from`.
#[tokio::test]
async fn concurrent_writer_drift_auto_forks_with_forked_from() {
// No compaction: keep run → run deterministic so each run consumes
// exactly one mock response and ensure_segment_head is the only fork
// trigger.
const NO_COMPACT_MANIFEST_TOML: &str = r#"
[pod]
name = "test-pod"
pwd = "./"
[model]
scheme = "anthropic"
model_id = "test-model"
[worker]
max_tokens = 100
[[scope.allow]]
target = "./"
permission = "write"
"#;
let client = MockClient::new(vec![
single_text_events("first"),
single_text_events("second"),
]);
let mut pod = make_pod_with_manifest(NO_COMPACT_MANIFEST_TOML, client).await;
pod.run_text("first").await.unwrap();
let store = pod.store().clone();
let session_id = pod.session_id();
let source_segment_id = pod.segment_id();
let source_len_before = store.read_all(session_id, source_segment_id).unwrap().len();
// Simulate a foreign writer appending to the same segment. This bumps
// the on-disk entry count past the Pod's own append tally without
// updating the Pod's `entries_written`.
store
.append(
session_id,
source_segment_id,
&LogEntry::UserInput {
ts: 9999,
segments: vec![protocol::Segment::text("interloper")],
},
)
.unwrap();
// Next run triggers ensure_segment_head, which sees the drift.
pod.run_text("second").await.unwrap();
// The Pod moved to a new segment in the same Session.
let new_segment_id = pod.segment_id();
assert_ne!(new_segment_id, source_segment_id);
assert_eq!(pod.session_id(), session_id, "auto-fork stays in-Session");
let metadata = store
.read_by_name("test-pod")
.unwrap()
.expect("metadata should exist after auto-fork");
let active = metadata.active.expect("active session pointer missing");
assert_eq!(active.session_id, session_id);
assert_eq!(active.segment_id, Some(new_segment_id));
// New segment records forked_from pointing at the source.
let new_entries = store.read_all(session_id, new_segment_id).unwrap();
match &new_entries[0] {
LogEntry::SegmentStart {
session_id: seg_session,
forked_from: Some(origin),
..
} => {
assert_eq!(*seg_session, session_id);
assert_eq!(origin.segment_id, source_segment_id);
}
other => panic!("expected SegmentStart with forked_from, got {other:?}"),
}
// Source segment is unchanged except for the foreign append — the
// auto-fork wrote no terminal marker back into it.
let source_after = store.read_all(session_id, source_segment_id).unwrap();
assert_eq!(source_after.len(), source_len_before + 1);
assert!(matches!(
source_after.last(),
Some(LogEntry::UserInput { .. })
));
}
#[tokio::test] #[tokio::test]
async fn compact_emits_session_start_carrying_summary_and_task_snapshot() { async fn compact_emits_session_start_carrying_summary_and_task_snapshot() {
let client = MockClient::new(vec![ let client = MockClient::new(vec![
@ -226,10 +351,20 @@ async fn compact_emits_session_start_carrying_summary_and_task_snapshot() {
pod.attach_event_tx(tx); pod.attach_event_tx(tx);
pod.run_text("first").await.unwrap(); pod.run_text("first").await.unwrap();
let session_id = pod.session_id();
pod.compact(10_000).await.unwrap(); pod.compact(10_000).await.unwrap();
let compacted_segment_id = pod.segment_id();
let metadata = pod
.store()
.read_by_name("test-pod")
.unwrap()
.expect("metadata should exist after compaction");
let active = metadata.active.expect("active session pointer missing");
assert_eq!(active.session_id, session_id);
assert_eq!(active.segment_id, Some(compacted_segment_id));
let system_texts = system_texts_in_sink_session_start(&pod); let system_texts = system_texts_in_sink_session_start(&pod);
// The post-compaction `SessionStart.history` carries the new system // The post-compaction `SegmentStart.history` carries the new system
// messages introduced by the compactor. Clients re-seed their view // messages introduced by the compactor. Clients re-seed their view
// from this entry alone, so it is the load-bearing payload. // from this entry alone, so it is the load-bearing payload.
assert!( assert!(
@ -289,11 +424,11 @@ async fn pre_run_compact_success_broadcasts_start_and_done() {
// CompactDone carries the new session id. // CompactDone carries the new session id.
let new_id_in_event = events.iter().find_map(|e| match e { let new_id_in_event = events.iter().find_map(|e| match e {
Event::CompactDone { new_session_id } => Some(*new_session_id), Event::CompactDone { new_segment_id } => Some(*new_segment_id),
_ => None, _ => None,
}); });
assert!(new_id_in_event.is_some(), "CompactDone missing"); assert!(new_id_in_event.is_some(), "CompactDone missing");
assert_eq!(new_id_in_event.unwrap(), pod.session_id()); assert_eq!(new_id_in_event.unwrap(), pod.segment_id());
} }
#[tokio::test] #[tokio::test]
@ -345,10 +480,10 @@ async fn mid_turn_compact_success_broadcasts_start_and_done() {
); );
let new_id_in_event = events.iter().find_map(|e| match e { let new_id_in_event = events.iter().find_map(|e| match e {
Event::CompactDone { new_session_id } => Some(*new_session_id), Event::CompactDone { new_segment_id } => Some(*new_segment_id),
_ => None, _ => None,
}); });
assert_eq!(new_id_in_event, Some(pod.session_id())); assert_eq!(new_id_in_event, Some(pod.segment_id()));
} }
/// Regression: `Pod::compact()` must reset the in-memory /// Regression: `Pod::compact()` must reset the in-memory
@ -520,9 +655,9 @@ async fn pre_run_compact_failure_broadcasts_start_and_failed() {
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Detached post-run memory jobs (`spawn_post_run_memory_jobs` / // Detached post-run memory jobs (`spawn_post_run_memory_jobs` /
// `wait_for_memory_jobs`). Covers the detach round-trip and the structural // `wait_for_memory_jobs`). Covers the detach round-trip and the structural
// invariant that the cloned memory-task Pod shares `SessionHead` with the // invariant that the cloned memory-task Pod shares `SegmentState` with the
// source Pod, so that `save_extension` from the background extract does not // source Pod, so that `save_extension` from the background extract does not
// leave the next turn's `save_user_input` looking at a stale head_hash. // leave the next turn's `save_user_input` looking at a stale session pointer.
const EXTRACT_NO_COMPACT_MANIFEST: &str = r#" const EXTRACT_NO_COMPACT_MANIFEST: &str = r#"
[pod] [pod]
@ -544,6 +679,27 @@ target = "./"
permission = "write" permission = "write"
"#; "#;
#[tokio::test]
async fn extract_large_unprocessed_range_does_not_abort_on_input_occupancy() {
let client = MockClient::new(vec![
text_events_with_usage("recorded", 1000),
write_extracted_tool_use_events("ec-large"),
single_text_events("done"),
]);
let mut pod = make_pod_with_manifest(EXTRACT_NO_COMPACT_MANIFEST, client).await;
let large_request = format!("remember this large slice: {}", "x ".repeat(200_000));
pod.run_text(&large_request).await.unwrap();
pod.try_post_run_extract().await.expect(
"large unprocessed extract ranges must reach the extract worker, not abort locally",
);
assert!(
pod.extract_pointer().is_some(),
"successful extract should advance the pointer even when the input range is large"
);
}
#[tokio::test] #[tokio::test]
async fn spawn_and_wait_drives_extract_to_completion() { async fn spawn_and_wait_drives_extract_to_completion() {
let client = MockClient::new(vec![ let client = MockClient::new(vec![
@ -570,9 +726,9 @@ async fn spawn_and_wait_drives_extract_to_completion() {
#[tokio::test] #[tokio::test]
async fn detached_extract_does_not_fork_session_log() { async fn detached_extract_does_not_fork_session_log() {
// Source pod and the cloned memory-task pod share `SessionHead` via // Source pod and the cloned memory-task pod share `SegmentState` via
// `Arc<AsyncMutex<_>>`. The detached extract advances head_hash through // `Arc<_>`. The detached extract advances the entry tally through
// `save_extension`; the next `run` must see that same head_hash so // `save_extension`; the next `run` must see that same tally so
// `ensure_head_or_fork` does not spawn a new session. // `ensure_head_or_fork` does not spawn a new session.
let client = MockClient::new(vec![ let client = MockClient::new(vec![
text_events_with_usage("hi", 1000), text_events_with_usage("hi", 1000),
@ -583,18 +739,18 @@ async fn detached_extract_does_not_fork_session_log() {
let mut pod = make_pod_with_manifest(EXTRACT_NO_COMPACT_MANIFEST, client).await; let mut pod = make_pod_with_manifest(EXTRACT_NO_COMPACT_MANIFEST, client).await;
pod.run_text("first").await.unwrap(); pod.run_text("first").await.unwrap();
let session_before = pod.session_id(); let session_before = pod.segment_id();
pod.spawn_post_run_memory_jobs(); pod.spawn_post_run_memory_jobs();
pod.wait_for_memory_jobs().await; pod.wait_for_memory_jobs().await;
pod.run_text("second").await.unwrap(); pod.run_text("second").await.unwrap();
let session_after = pod.session_id(); let session_after = pod.segment_id();
assert_eq!( assert_eq!(
session_before, session_after, session_before, session_after,
"detached extract's save_extension and the next turn's save_user_input \ "detached extract's save_extension and the next turn's save_user_input \
must share head_hash through SessionHead a fork here means the clone \ must share the entry tally through SegmentState a fork here means the \
carried its own head_hash" clone carried its own counter"
); );
} }

View File

@ -172,7 +172,7 @@ fn write_n_staging(layout: &WorkspaceLayout, n: usize) -> Vec<uuid::Uuid> {
let (id, _) = write_staging( let (id, _) = write_staging(
layout, layout,
SourceRef { SourceRef {
session_id: format!("s-{i}"), segment_id: format!("s-{i}"),
range: [i as u64, i as u64], range: [i as u64, i as u64],
}, },
ExtractedPayload::default(), ExtractedPayload::default(),

View File

@ -22,7 +22,7 @@ fn history_from_sink(handle: &PodHandle) -> Vec<Item> {
let mut items = Vec::new(); let mut items = Vec::new();
for entry in entries { for entry in entries {
match entry { match entry {
LogEntry::SessionStart { history, .. } => { LogEntry::SegmentStart { history, .. } => {
items.extend(history.into_iter().map(Item::from)); items.extend(history.into_iter().map(Item::from));
} }
LogEntry::UserInput { segments, .. } => { LogEntry::UserInput { segments, .. } => {
@ -35,14 +35,6 @@ fn history_from_sink(handle: &PodHandle) -> Vec<Item> {
LogEntry::SystemItem { item, .. } => { LogEntry::SystemItem { item, .. } => {
items.push(item.to_history_item()); items.push(item.to_history_item());
} }
LogEntry::AssistantItems { items: i, .. }
| LogEntry::ToolResults { items: i, .. }
| LogEntry::HookInjectedItems { items: i, .. } => {
items.extend(i.into_iter().map(Item::from));
}
LogEntry::SystemItems { items: si, .. } => {
items.extend(si.iter().map(|s| s.to_history_item()));
}
_ => {} _ => {}
} }
} }
@ -417,21 +409,23 @@ async fn events_are_broadcast() {
#[tokio::test] #[tokio::test]
async fn double_run_returns_error() { async fn double_run_returns_error() {
// Create a client that streams slowly // Keep the first turn in-flight until the test drops the handle. A
// finite stream can finish before the second Method reaches the
// controller in the full test suite, making this assertion racy.
let events = vec![ let events = vec![
LlmEvent::text_block_start(0), LlmEvent::text_block_start(0),
LlmEvent::text_delta(0, "slow..."), LlmEvent::text_delta(0, "slow..."),
// No stop/completed — the stream will end but without proper completion
]; ];
let client = MockClient::new(events); let client = MockClient::sequential(vec![MockResponse::Hang(events)]);
let pod = make_pod(client).await; let pod = make_pod(client).await;
let handle = spawn_controller(pod).await; let handle = spawn_controller(pod).await;
let mut rx = handle.subscribe(); let mut rx = handle.subscribe();
// Send first run // Send first run and wait until the controller has entered Running.
handle.send(Method::run_text("first")).await.unwrap(); handle.send(Method::run_text("first")).await.unwrap();
wait_for_status(&handle, PodStatus::Running).await;
// Immediately send second run (should get error) // Now the second run must be rejected by drive_turn's live Method arm.
handle.send(Method::run_text("second")).await.unwrap(); handle.send(Method::run_text("second")).await.unwrap();
// Look for the error event // Look for the error event
@ -777,13 +771,15 @@ async fn notify_while_idle_auto_starts_turn_and_injects_system_message() {
// not on the `event_tx` broadcast that `handle.subscribe()` taps. // not on the `event_tx` broadcast that `handle.subscribe()` taps.
// Verify the notification landed on the sink mirror instead. // Verify the notification landed on the sink mirror instead.
let (entries, _) = handle.sink.subscribe_with_snapshot(); let (entries, _) = handle.sink.subscribe_with_snapshot();
let saw_notify_in_mirror = entries.iter().any(|e| matches!( let saw_notify_in_mirror = entries.iter().any(|e| {
e, matches!(
session_store::LogEntry::SystemItem { e,
item: session_store::SystemItem::Notification { message, .. }, session_store::LogEntry::SystemItem {
.. item: session_store::SystemItem::Notification { message, .. },
} if message == "turn finished" ..
)); } if message == "turn finished"
)
});
assert!( assert!(
saw_notify_in_mirror, saw_notify_in_mirror,
"Method::Notify should commit a SystemItem::Notification entry; mirror = {entries:?}" "Method::Notify should commit a SystemItem::Notification entry; mirror = {entries:?}"
@ -865,16 +861,18 @@ async fn pod_event_turn_ended_while_idle_auto_starts_turn_and_injects_system_mes
// its Flush of the drain queue) runs afterwards. // its Flush of the drain queue) runs afterwards.
wait_for_status(&handle, PodStatus::Idle).await; wait_for_status(&handle, PodStatus::Idle).await;
let (entries, _) = handle.sink.subscribe_with_snapshot(); let (entries, _) = handle.sink.subscribe_with_snapshot();
let saw_pod_event_in_mirror = entries.iter().any(|e| matches!( let saw_pod_event_in_mirror = entries.iter().any(|e| {
e, matches!(
session_store::LogEntry::SystemItem { e,
item: session_store::SystemItem::PodEvent { session_store::LogEntry::SystemItem {
event: protocol::PodEvent::TurnEnded { pod_name }, item: session_store::SystemItem::PodEvent {
event: protocol::PodEvent::TurnEnded { pod_name },
..
},
.. ..
}, } if pod_name == "child"
.. )
} if pod_name == "child" });
));
assert!( assert!(
saw_pod_event_in_mirror, saw_pod_event_in_mirror,
"Method::PodEvent should commit a SystemItem::PodEvent entry" "Method::PodEvent should commit a SystemItem::PodEvent entry"

View File

@ -2,18 +2,18 @@
//! `ReadPodOutput`, `StopPod`, `ListPods`). //! `ReadPodOutput`, `StopPod`, `ListPods`).
//! //!
//! The real child Pod binary is not started. Instead each test stands //! The real child Pod binary is not started. Instead each test stands
//! up a mock `UnixListener` that speaks the socket protocol directly //! up a mock `UnixListener` that speaks the socket protocol directly:
//! (accepting `Method::Run` / `Method::GetHistory` / `Method::Shutdown` //! it emits the connect-time `Event::Snapshot`, accepts methods such as
//! and responding with `Event::History` when asked). This keeps the //! `Method::Run` / `Method::Shutdown`, and responds with the relevant
//! tests fast and independent of the LLM layer — the tools are exercised //! events when needed. This keeps the tests fast and independent of the
//! for their wire behaviour alone. //! LLM layer — the tools are exercised for their wire behaviour alone.
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::{Arc, LazyLock, Mutex}; use std::sync::{Arc, LazyLock, Mutex};
use llm_worker::llm_client::types::{ContentPart, Item, Role}; use llm_worker::llm_client::types::{ContentPart, Item, Role};
use llm_worker::tool::ToolOutput; use llm_worker::tool::ToolOutput;
use manifest::{Permission, ScopeRule}; use manifest::{Permission, Scope, ScopeRule, SharedScope};
use pod::runtime::dir::{RuntimeDir, SpawnedPodRecord}; use pod::runtime::dir::{RuntimeDir, SpawnedPodRecord};
use pod::runtime::pod_registry::{self, LockFileGuard}; use pod::runtime::pod_registry::{self, LockFileGuard};
use pod::spawn::comm_tools::{ use pod::spawn::comm_tools::{
@ -23,8 +23,10 @@ use pod::spawn::registry::SpawnedPodRegistry;
use protocol::stream::{JsonLineReader, JsonLineWriter}; use protocol::stream::{JsonLineReader, JsonLineWriter};
use protocol::{ErrorCode, Event, Greeting, Method}; use protocol::{ErrorCode, Event, Greeting, Method};
use serde_json::json; use serde_json::json;
use session_store::{FsStore, PodMetadataStore};
use tempfile::TempDir; use tempfile::TempDir;
use tokio::net::UnixListener; use tokio::net::UnixListener;
use tokio::sync::mpsc;
use tokio::task::JoinHandle; use tokio::task::JoinHandle;
/// Serialises env-mutating tests. The test harness runs tasks across /// Serialises env-mutating tests. The test harness runs tasks across
@ -113,20 +115,42 @@ async fn bind_mock_socket(dir: &Path, name: &str) -> (PathBuf, UnixListener) {
(socket, listener) (socket, listener)
} }
/// Accept one connection and read exactly one `Method` line from it. /// Minimal connect-time snapshot used by mock socket servers.
fn empty_snapshot() -> Event {
Event::Snapshot {
entries: Vec::new(),
greeting: Greeting {
pod_name: "child".into(),
cwd: "/tmp".into(),
provider: "anthropic".into(),
model: "x".into(),
scope_summary: String::new(),
tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
},
status: protocol::PodStatus::Idle,
}
}
/// Accept one connection, send the protocol's connect-time snapshot,
/// and read exactly one `Method` line from it.
/// The reader half is kept open; caller awaits the returned handle. /// The reader half is kept open; caller awaits the returned handle.
fn accept_one_method(listener: UnixListener) -> JoinHandle<Option<Method>> { fn accept_one_method(listener: UnixListener) -> JoinHandle<Option<Method>> {
tokio::spawn(async move { tokio::spawn(async move {
let (stream, _) = listener.accept().await.ok()?; let (stream, _) = listener.accept().await.ok()?;
let (r, _w) = stream.into_split(); let (r, w) = stream.into_split();
let mut reader = JsonLineReader::new(r); let mut reader = JsonLineReader::new(r);
let mut writer = JsonLineWriter::new(w);
writer.write(&empty_snapshot()).await.ok()?;
reader.next::<Method>().await.ok().flatten() reader.next::<Method>().await.ok().flatten()
}) })
} }
/// Accept one connection, read one `Method`, then write `response` /// Accept one connection, send the protocol's connect-time snapshot,
/// back. Used by `SendToPod` tests to mock the real controller's /// read one `Method`, then write `response` back. Used by `SendToPod`
/// `TurnStart` acknowledgement (or its `AlreadyRunning` rejection). /// tests to mock the real controller's `TurnStart` acknowledgement (or
/// its `AlreadyRunning` rejection).
fn accept_method_and_respond( fn accept_method_and_respond(
listener: UnixListener, listener: UnixListener,
response: Event, response: Event,
@ -136,6 +160,7 @@ fn accept_method_and_respond(
let (r, w) = stream.into_split(); let (r, w) = stream.into_split();
let mut reader = JsonLineReader::new(r); let mut reader = JsonLineReader::new(r);
let mut writer = JsonLineWriter::new(w); let mut writer = JsonLineWriter::new(w);
writer.write(&empty_snapshot()).await.ok()?;
let method = reader.next::<Method>().await.ok().flatten(); let method = reader.next::<Method>().await.ok().flatten();
if method.is_some() { if method.is_some() {
let _ = writer.write(&response).await; let _ = writer.write(&response).await;
@ -156,18 +181,18 @@ fn serve_history(listener: UnixListener, items: Vec<Item>) -> JoinHandle<()> {
}; };
let (_r, w) = stream.into_split(); let (_r, w) = stream.into_split();
let mut writer = JsonLineWriter::new(w); let mut writer = JsonLineWriter::new(w);
// Wrap the assistant items in a single let entries: Vec<serde_json::Value> = items
// `LogEntry::AssistantItems` entry — that's the only kind .iter()
// that contributes assistant text via `extract_assistant_text`. .map(|item| {
let logged: Vec<session_store::LoggedItem> = let entry = session_store::LogEntry::AssistantItem {
items.iter().map(session_store::LoggedItem::from).collect(); ts: 0,
let entry = session_store::LogEntry::AssistantItems { item: session_store::LoggedItem::from(item),
ts: 0, };
items: logged, serde_json::to_value(&entry).unwrap()
}; })
let entry_value = serde_json::to_value(&entry).unwrap(); .collect();
let event = Event::Snapshot { let event = Event::Snapshot {
entries: vec![entry_value], entries,
greeting: Greeting { greeting: Greeting {
pod_name: "child".into(), pod_name: "child".into(),
cwd: "/tmp".into(), cwd: "/tmp".into(),
@ -175,6 +200,8 @@ fn serve_history(listener: UnixListener, items: Vec<Item>) -> JoinHandle<()> {
model: "x".into(), model: "x".into(),
scope_summary: String::new(), scope_summary: String::new(),
tools: Vec::new(), tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
}, },
status: protocol::PodStatus::Idle, status: protocol::PodStatus::Idle,
}; };
@ -183,6 +210,34 @@ fn serve_history(listener: UnixListener, items: Vec<Item>) -> JoinHandle<()> {
}) })
} }
fn serve_pod_methods(listener: UnixListener) -> mpsc::Receiver<Method> {
let (tx, rx) = mpsc::channel(8);
tokio::spawn(async move {
loop {
let Ok((stream, _)) = listener.accept().await else {
return;
};
let (r, w) = stream.into_split();
let mut reader = JsonLineReader::new(r);
let mut writer = JsonLineWriter::new(w);
if writer.write(&empty_snapshot()).await.is_err() {
continue;
}
let Some(method) = reader.next::<Method>().await.ok().flatten() else {
continue;
};
let is_shutdown = matches!(method, Method::Shutdown);
if matches!(method, Method::Run { .. }) {
let _ = writer.write(&Event::TurnStart { turn: 1 }).await;
}
if tx.send(method).await.is_err() || is_shutdown {
return;
}
}
});
rx
}
fn assistant(text: &str) -> Item { fn assistant(text: &str) -> Item {
Item::Message { Item::Message {
id: None, id: None,
@ -328,45 +383,67 @@ async fn read_pod_output_reports_stopped_on_dead_socket() {
#[tokio::test] #[tokio::test]
async fn stop_pod_sends_shutdown_and_releases_scope() { async fn stop_pod_sends_shutdown_and_releases_scope() {
let _env = EnvGuard::acquire(); let _env = EnvGuard::acquire();
let (tmp, registry, rd) = setup_registry().await; let tmp = TempDir::new().unwrap();
let store_tmp = TempDir::new().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap();
let rd = Arc::new(RuntimeDir::create(tmp.path(), "spawner").await.unwrap());
let parent_scope = SharedScope::new(
Scope::writable(tmp.path())
.unwrap()
.with_added_deny_rules([ScopeRule {
target: tmp.path().to_path_buf(),
permission: Permission::Write,
recursive: true,
}])
.unwrap(),
);
unsafe { unsafe {
std::env::set_var("INSOMNIA_RUNTIME_DIR", tmp.path()); std::env::set_var("INSOMNIA_RUNTIME_DIR", tmp.path());
} }
let lock_path = tmp.path().join("pods.json"); let lock_path = tmp.path().join("pods.json");
// Seed pods.json with a top-level `spawner` allocation plus a // Seed pods.json with a restored top-level `spawner` allocation whose
// delegated `child` allocation — mimics what SpawnPod would have // scope_deny contains the delegated child path plus the live child
// done so StopPod has something to release. // allocation — mimics a parent resumed after SpawnPod.
{ {
let mut g = LockFileGuard::open(&lock_path).unwrap(); let mut g = LockFileGuard::open(&lock_path).unwrap();
pod_registry::register_pod( let rule = ScopeRule {
target: tmp.path().to_path_buf(),
permission: Permission::Write,
recursive: true,
};
pod_registry::register_pod_with_deny(
&mut g, &mut g,
"spawner".into(), "spawner".into(),
std::process::id(), std::process::id(),
"/tmp/spawner.sock".into(), "/tmp/spawner.sock".into(),
vec![ScopeRule { vec![rule.clone()],
target: tmp.path().to_path_buf(), vec![rule.clone()],
permission: Permission::Write, session_store::new_segment_id(),
recursive: true,
}],
session_store::new_session_id(),
) )
.unwrap(); .unwrap();
pod_registry::delegate_scope( pod_registry::register_pod(
&mut g, &mut g,
"spawner",
"child".into(), "child".into(),
std::process::id(), std::process::id(),
"/tmp/child.sock".into(), "/tmp/child.sock".into(),
vec![ScopeRule { vec![rule],
target: tmp.path().to_path_buf(), session_store::new_segment_id(),
permission: Permission::Write,
recursive: true,
}],
) )
.unwrap(); .unwrap();
} }
let loaded = SpawnedPodRegistry::load_from_pod_state_with_reclaim(
rd.clone(),
store.clone(),
"spawner".into(),
Some(parent_scope.clone()),
None,
)
.await
.unwrap();
let registry = loaded.registry;
let (socket, listener) = bind_mock_socket(tmp.path(), "child").await; let (socket, listener) = bind_mock_socket(tmp.path(), "child").await;
let received = accept_one_method(listener); let received = accept_one_method(listener);
register_child(&registry, "child", &socket, tmp.path()).await; register_child(&registry, "child", &socket, tmp.path()).await;
@ -381,12 +458,20 @@ async fn stop_pod_sends_shutdown_and_releases_scope() {
let method = received.await.unwrap().expect("expected shutdown"); let method = received.await.unwrap().expect("expected shutdown");
assert!(matches!(method, Method::Shutdown)); assert!(matches!(method, Method::Shutdown));
// Allocation for `child` is gone; `spawner` remains. // Allocation for `child` is gone; `spawner` remains and its restored
// dynamic deny layer has been reclaimed.
{ {
let g = LockFileGuard::open(&lock_path).unwrap(); let g = LockFileGuard::open(&lock_path).unwrap();
assert!(g.data().find("child").is_none(), "child still allocated"); assert!(g.data().find("child").is_none(), "child still allocated");
assert!(g.data().find("spawner").is_some(), "spawner missing"); let spawner = g.data().find("spawner").expect("spawner missing");
assert!(spawner.scope_deny.is_empty(), "deny not reclaimed");
} }
assert_eq!(
parent_scope
.snapshot()
.permission_at(&tmp.path().join("file.txt")),
Some(Permission::Write)
);
// spawned_pods.json now lists zero children. // spawned_pods.json now lists zero children.
let spawned = rd.path().join("spawned_pods.json"); let spawned = rd.path().join("spawned_pods.json");
@ -418,6 +503,214 @@ async fn stop_pod_succeeds_even_when_child_unreachable() {
assert!(registry.get("child").await.is_none()); assert!(registry.get("child").await.is_none());
} }
// ---------------------------------------------------------------------------
// Persistence / restore
// ---------------------------------------------------------------------------
#[tokio::test]
async fn restored_registry_uses_pod_state_without_runtime_file() {
let _env = EnvGuard::acquire();
let runtime_tmp = TempDir::new().unwrap();
let store_tmp = TempDir::new().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap();
unsafe {
std::env::set_var("INSOMNIA_RUNTIME_DIR", runtime_tmp.path());
}
let rd = Arc::new(
RuntimeDir::create(runtime_tmp.path(), "spawner")
.await
.unwrap(),
);
let registry =
SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".to_string())
.await
.unwrap();
let (socket, listener) = bind_mock_socket(runtime_tmp.path(), "child").await;
let mut received = serve_pod_methods(listener);
register_child(&registry, "child", &socket, runtime_tmp.path()).await;
std::fs::remove_file(rd.path().join("spawned_pods.json")).unwrap();
let restored =
SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".to_string())
.await
.unwrap();
let def = list_pods_tool(restored.clone());
let (_meta, tool) = def();
let output: ToolOutput = tool.execute("{}").await.unwrap();
assert!(output.summary.contains("1 pod"), "{}", output.summary);
let body = output.content.expect("restored ListPods should list child");
assert!(body.contains("child [alive]"), "body: {body}");
let def = send_to_pod_tool(restored.clone());
let (_meta, tool) = def();
let input = json!({ "name": "child", "message": "after restart" }).to_string();
tool.execute(&input).await.unwrap();
match received.recv().await.expect("expected Run") {
Method::Run { input } => match input.as_slice() {
[protocol::Segment::Text { content }] => assert_eq!(content, "after restart"),
other => panic!("expected single Text segment, got {other:?}"),
},
other => panic!("expected Run, got {other:?}"),
}
let def = stop_pod_tool(restored.clone());
let (_meta, tool) = def();
tool.execute(&json!({ "name": "child" }).to_string())
.await
.unwrap();
assert!(matches!(
received.recv().await.expect("expected Shutdown"),
Method::Shutdown
));
assert!(restored.get("child").await.is_none());
let metadata = store
.read_by_name("spawner")
.unwrap()
.expect("spawner metadata should remain");
assert!(metadata.spawned_children.is_empty());
let runtime_contents = std::fs::read_to_string(rd.path().join("spawned_pods.json")).unwrap();
let runtime_records: Vec<SpawnedPodRecord> = serde_json::from_str(&runtime_contents).unwrap();
assert!(runtime_records.is_empty());
}
#[tokio::test]
async fn load_from_pod_state_prunes_children_with_missing_sockets() {
let runtime_tmp = TempDir::new().unwrap();
let store_tmp = TempDir::new().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap();
let rd = Arc::new(
RuntimeDir::create(runtime_tmp.path(), "spawner")
.await
.unwrap(),
);
let registry =
SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".to_string())
.await
.unwrap();
let (live_socket, listener) = bind_mock_socket(runtime_tmp.path(), "alive").await;
let _server = serve_pod_methods(listener);
register_child(&registry, "alive", &live_socket, runtime_tmp.path()).await;
register_child(
&registry,
"missing",
&runtime_tmp.path().join("missing.sock"),
runtime_tmp.path(),
)
.await;
let restored =
SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".to_string())
.await
.unwrap();
assert!(restored.get("alive").await.is_some());
assert!(restored.get("missing").await.is_none());
let metadata = store
.read_by_name("spawner")
.unwrap()
.expect("spawner metadata should be written");
assert_eq!(metadata.spawned_children.len(), 1);
assert_eq!(metadata.spawned_children[0].pod_name, "alive");
}
#[tokio::test]
async fn load_from_pod_state_reclaims_pruned_child_scope_and_registry_deny() {
let _env = EnvGuard::acquire();
let runtime_tmp = TempDir::new().unwrap();
let store_tmp = TempDir::new().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap();
unsafe {
std::env::set_var("INSOMNIA_RUNTIME_DIR", runtime_tmp.path());
}
let rd = Arc::new(
RuntimeDir::create(runtime_tmp.path(), "spawner")
.await
.unwrap(),
);
let missing_rule = ScopeRule {
target: runtime_tmp.path().to_path_buf(),
permission: Permission::Write,
recursive: true,
};
{
let mut g = LockFileGuard::open(&runtime_tmp.path().join("pods.json")).unwrap();
pod_registry::register_pod_with_deny(
&mut g,
"spawner".into(),
std::process::id(),
"/tmp/spawner.sock".into(),
vec![missing_rule.clone()],
vec![missing_rule.clone()],
session_store::new_segment_id(),
)
.unwrap();
pod_registry::register_pod(
&mut g,
"missing".into(),
std::process::id(),
"/tmp/missing.sock".into(),
vec![missing_rule.clone()],
session_store::new_segment_id(),
)
.unwrap();
}
let parent_scope = SharedScope::new(
Scope::writable(runtime_tmp.path())
.unwrap()
.with_added_deny_rules([missing_rule.clone()])
.unwrap(),
);
let seed = SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".into())
.await
.unwrap();
seed.add(SpawnedPodRecord {
pod_name: "missing".into(),
socket_path: runtime_tmp.path().join("missing.sock"),
scope_delegated: vec![missing_rule.clone()],
callback_address: "/dev/null".into(),
})
.await
.unwrap();
let loaded = SpawnedPodRegistry::load_from_pod_state_with_reclaim(
rd.clone(),
store.clone(),
"spawner".into(),
Some(parent_scope.clone()),
None,
)
.await
.unwrap();
assert!(loaded.reclaimed_unreachable);
assert!(loaded.registry.get("missing").await.is_none());
assert_eq!(
parent_scope
.snapshot()
.permission_at(&runtime_tmp.path().join("file.txt")),
Some(Permission::Write)
);
let g = LockFileGuard::open(&runtime_tmp.path().join("pods.json")).unwrap();
assert!(g.data().find("missing").is_none());
assert!(g.data().find("spawner").unwrap().scope_deny.is_empty());
let metadata = store
.read_by_name("spawner")
.unwrap()
.expect("spawner metadata should remain");
assert!(metadata.spawned_children.is_empty());
let runtime_contents = std::fs::read_to_string(rd.path().join("spawned_pods.json")).unwrap();
let runtime_records: Vec<SpawnedPodRecord> = serde_json::from_str(&runtime_contents).unwrap();
assert!(runtime_records.is_empty());
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// ListPods // ListPods
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------

View File

@ -13,8 +13,8 @@ use pod::ipc::event::{apply_event_side_effects, fire_and_forget, render_event};
use pod::runtime::dir::{RuntimeDir, SpawnedPodRecord}; use pod::runtime::dir::{RuntimeDir, SpawnedPodRecord};
use pod::runtime::pod_registry::{self, LockFileGuard}; use pod::runtime::pod_registry::{self, LockFileGuard};
use pod::spawn::registry::SpawnedPodRegistry; use pod::spawn::registry::SpawnedPodRegistry;
use protocol::stream::JsonLineReader; use protocol::stream::{JsonLineReader, JsonLineWriter};
use protocol::{Method, Permission, PodEvent, ScopeRule}; use protocol::{Event, Greeting, Method, Permission, PodEvent, PodStatus, ScopeRule};
use tempfile::TempDir; use tempfile::TempDir;
use tokio::net::UnixListener; use tokio::net::UnixListener;
@ -76,11 +76,32 @@ fn clear_runtime_dir() {
} }
} }
/// Accept a single connection, read one `Method`, and return it. /// Minimal connect-time snapshot used by mock parent sockets.
fn empty_snapshot() -> Event {
Event::Snapshot {
entries: Vec::new(),
greeting: Greeting {
pod_name: "parent".into(),
cwd: "/tmp".into(),
provider: "test".into(),
model: "test".into(),
scope_summary: String::new(),
tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
},
status: PodStatus::Idle,
}
}
/// Accept a single connection, send the protocol's connect-time snapshot,
/// read one `Method`, and return it.
fn accept_one_method(listener: UnixListener) -> tokio::task::JoinHandle<Option<Method>> { fn accept_one_method(listener: UnixListener) -> tokio::task::JoinHandle<Option<Method>> {
tokio::spawn(async move { tokio::spawn(async move {
let (stream, _) = listener.accept().await.ok()?; let (stream, _) = listener.accept().await.ok()?;
let (reader, _writer) = stream.into_split(); let (reader, writer) = stream.into_split();
let mut w = JsonLineWriter::new(writer);
w.write(&empty_snapshot()).await.ok()?;
let mut r = JsonLineReader::new(reader); let mut r = JsonLineReader::new(reader);
r.next::<Method>().await.ok().flatten() r.next::<Method>().await.ok().flatten()
}) })
@ -358,7 +379,7 @@ async fn shutdown_releases_scope_allocation_when_present() {
std::process::id(), std::process::id(),
"/tmp/kid.sock".into(), "/tmp/kid.sock".into(),
vec![], vec![],
session_store::new_session_id(), session_store::new_segment_id(),
) )
.unwrap(); .unwrap();
std::mem::forget(guard); std::mem::forget(guard);

View File

@ -8,7 +8,7 @@
use std::sync::{LazyLock, Mutex}; use std::sync::{LazyLock, Mutex};
use pod::{Pod, PodError}; use pod::{Pod, PodError};
use session_store::{FsStore, SessionId, StoreError}; use session_store::{FsStore, PodActiveSegmentRef, PodMetadata, PodMetadataStore, StoreError};
const MINIMAL_MANIFEST_TOML: &str = r#" const MINIMAL_MANIFEST_TOML: &str = r#"
[pod] [pod]
@ -32,7 +32,97 @@ permission = "write"
static ENV_LOCK: LazyLock<Mutex<()>> = LazyLock::new(|| Mutex::new(())); static ENV_LOCK: LazyLock<Mutex<()>> = LazyLock::new(|| Mutex::new(()));
#[tokio::test] #[tokio::test]
async fn restore_from_manifest_rejects_unknown_session() { async fn restore_from_pod_metadata_rejects_missing_metadata() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let store_tmp = tempfile::tempdir().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap();
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
let result = Pod::restore_from_pod_metadata(
"restore-test",
manifest,
store,
pod::PromptLoader::builtins_only(),
)
.await;
match result {
Err(PodError::PodMetadataMissing { pod_name }) => assert_eq!(pod_name, "restore-test"),
Err(other) => panic!("expected PodMetadataMissing, got {other:?}"),
Ok(_) => panic!("expected missing pod metadata to fail"),
}
}
#[tokio::test]
async fn restore_from_pod_metadata_rejects_pending_segment() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let store_tmp = tempfile::tempdir().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap();
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
let session_id = session_store::new_session_id();
store
.write(&PodMetadata::new(
"restore-test",
Some(PodActiveSegmentRef::pending_segment(session_id)),
))
.unwrap();
let result = Pod::restore_from_pod_metadata(
"restore-test",
manifest,
store,
pod::PromptLoader::builtins_only(),
)
.await;
match result {
Err(PodError::PodMetadataPending {
pod_name,
session_id: actual,
}) => {
assert_eq!(pod_name, "restore-test");
assert_eq!(actual, session_id);
}
Err(other) => panic!("expected PodMetadataPending, got {other:?}"),
Ok(_) => panic!("expected pending pod metadata to fail"),
}
}
#[tokio::test]
async fn restore_from_pod_metadata_resolves_active_pointer_through_session_log() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let store_tmp = tempfile::tempdir().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap();
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
let session_id = session_store::new_session_id();
let segment_id = session_store::new_segment_id();
store
.write(&PodMetadata::new(
"restore-test",
Some(PodActiveSegmentRef::active_segment(session_id, segment_id)),
))
.unwrap();
let result = Pod::restore_from_pod_metadata(
"restore-test",
manifest,
store,
pod::PromptLoader::builtins_only(),
)
.await;
match result {
Err(PodError::Store(StoreError::NotFound(id))) => assert_eq!(id, segment_id),
Err(other) => panic!("expected Store(NotFound) from resolved segment, got {other:?}"),
Ok(_) => panic!("expected unknown resolved segment to fail"),
}
}
#[tokio::test]
async fn restore_from_manifest_rejects_unknown_segment() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner()); let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let store_tmp = tempfile::tempdir().unwrap(); let store_tmp = tempfile::tempdir().unwrap();
@ -42,67 +132,87 @@ async fn restore_from_manifest_rejects_unknown_session() {
// A freshly-minted id with no jsonl file at all → store returns // A freshly-minted id with no jsonl file at all → store returns
// NotFound, which `Pod::restore_from_manifest` surfaces verbatim // NotFound, which `Pod::restore_from_manifest` surfaces verbatim
// as `PodError::Store`. // as `PodError::Store`.
let unknown = session_store::new_session_id(); let unknown_sid = session_store::new_session_id();
let result = let unknown_seg = session_store::new_segment_id();
Pod::restore_from_manifest(unknown, manifest, store, pod::PromptLoader::builtins_only()) let result = Pod::restore_from_manifest(
.await; unknown_sid,
unknown_seg,
manifest,
store,
pod::PromptLoader::builtins_only(),
)
.await;
match result { match result {
Err(PodError::Store(StoreError::NotFound(id))) => assert_eq!(id, unknown), Err(PodError::Store(StoreError::NotFound(id))) => assert_eq!(id, unknown_seg),
Err(other) => panic!("expected Store(NotFound), got {other:?}"), Err(other) => panic!("expected Store(NotFound), got {other:?}"),
Ok(_) => panic!("expected unknown session to fail"), Ok(_) => panic!("expected unknown segment to fail"),
} }
} }
#[tokio::test] #[tokio::test]
async fn restore_from_manifest_rejects_empty_session_log() { async fn restore_from_manifest_rejects_empty_segment_log() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner()); let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let store_tmp = tempfile::tempdir().unwrap(); let store_tmp = tempfile::tempdir().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap(); let store = FsStore::new(store_tmp.path()).unwrap();
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap(); let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
// Pre-create an empty `<id>.jsonl` so `read_all` succeeds with no // Pre-create an empty `<sid>/<segid>.jsonl` so `read_all` succeeds
// entries. `collect_state` returns `head_hash = None`, which // with no entries. `collect_state` returns `entries_count = 0`,
// `restore_from_manifest` rejects with `SessionEmpty` *before* it // which `restore_from_manifest` rejects with `SegmentEmpty` *before*
// gets as far as building the LLM client — so the test does not // it gets as far as building the LLM client.
// need credentials or a runtime sandbox. let sid = session_store::new_session_id();
let id: SessionId = session_store::new_session_id(); let segid = session_store::new_segment_id();
let path = store_tmp.path().join(format!("{id}.jsonl")); let dir = store_tmp.path().join(sid.to_string());
std::fs::write(&path, b"").unwrap(); std::fs::create_dir_all(&dir).unwrap();
std::fs::write(dir.join(format!("{segid}.jsonl")), b"").unwrap();
let result = let result = Pod::restore_from_manifest(
Pod::restore_from_manifest(id, manifest, store, pod::PromptLoader::builtins_only()).await; sid,
segid,
manifest,
store,
pod::PromptLoader::builtins_only(),
)
.await;
match result { match result {
Err(PodError::SessionEmpty { session_id }) => assert_eq!(session_id, id), Err(PodError::SegmentEmpty { segment_id }) => assert_eq!(segment_id, segid),
Err(other) => panic!("expected SessionEmpty, got {other:?}"), Err(other) => panic!("expected SegmentEmpty, got {other:?}"),
Ok(_) => panic!("expected empty session log to fail"), Ok(_) => panic!("expected empty segment log to fail"),
} }
} }
#[tokio::test] #[tokio::test]
async fn restore_from_manifest_rejects_session_without_scope_snapshot() { async fn restore_from_manifest_rejects_segment_without_scope_snapshot() {
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner()); let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let store_tmp = tempfile::tempdir().unwrap(); let store_tmp = tempfile::tempdir().unwrap();
let store = FsStore::new(store_tmp.path()).unwrap(); let store = FsStore::new(store_tmp.path()).unwrap();
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap(); let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
let id = session_store::new_session_id(); let sid = session_store::new_session_id();
let state = session_store::SessionStartState { let segid = session_store::new_segment_id();
let state = session_store::SegmentStartState {
system_prompt: None, system_prompt: None,
config: &Default::default(), config: &Default::default(),
history: &[], history: &[],
}; };
session_store::create_session_with_id(&store, id, state).unwrap(); session_store::create_segment_with_ids(&store, sid, segid, state).unwrap();
let result = let result = Pod::restore_from_manifest(
Pod::restore_from_manifest(id, manifest, store, pod::PromptLoader::builtins_only()).await; sid,
segid,
manifest,
store,
pod::PromptLoader::builtins_only(),
)
.await;
match result { match result {
Err(PodError::SessionScopeMissing { session_id }) => assert_eq!(session_id, id), Err(PodError::SegmentScopeMissing { segment_id }) => assert_eq!(segment_id, segid),
Err(other) => panic!("expected SessionScopeMissing, got {other:?}"), Err(other) => panic!("expected SegmentScopeMissing, got {other:?}"),
Ok(_) => panic!("expected missing scope snapshot to fail"), Ok(_) => panic!("expected missing scope snapshot to fail"),
} }
} }

View File

@ -4,10 +4,10 @@
//! returns a long `ToolOutput.content`, then inspects the persisted //! returns a long `ToolOutput.content`, then inspects the persisted
//! session log to verify: //! session log to verify:
//! //!
//! - `prune.skip { reason: "no_candidates" }` lands when the protected-turn //! - `prune.skip { reason: "no_candidates" }` lands when usage estimates are
//! window covers the entire history. //! unavailable or the protected-token window covers all tool results.
//! - `prune.fire` lands once enough turns + usage measurements exist for //! - `prune.fire` lands once enough measured history exceeds the protected-token
//! the projection to actually apply. //! budget for the projection to actually apply.
//! - The fire metric and the immediately-following `prune.post_request` //! - The fire metric and the immediately-following `prune.post_request`
//! metric share the same `correlation_id`, so cache_read / cache_write //! metric share the same `correlation_id`, so cache_read / cache_write
//! from the LlmUsage that triggered the projection can be joined back //! from the LlmUsage that triggered the projection can be joined back
@ -26,9 +26,7 @@ use llm_worker::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEve
use llm_worker::llm_client::{ClientError, LlmClient, Request}; use llm_worker::llm_client::{ClientError, LlmClient, Request};
use llm_worker::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput}; use llm_worker::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use session_metrics::{DOMAIN, Metric, metrics_from_extensions}; use session_metrics::{DOMAIN, Metric, metrics_from_extensions};
use session_store::{ use session_store::{FsStore, LogEntry, SegmentId, SessionId, Store, StoreError, TraceEntry};
EntryHash, FsStore, HashedEntry, LogEntry, SessionId, Store, StoreError, TraceEntry,
};
use pod::{Pod, PodManifest}; use pod::{Pod, PodManifest};
@ -138,7 +136,7 @@ fn text_response_with_cache(text: &str, cache_read: u64, cache_write: u64) -> Ve
] ]
} }
fn manifest_toml(prune_protected_turns: usize, prune_min_savings: u64) -> String { fn manifest_toml(prune_protected_tokens: u64, prune_min_savings: u64) -> String {
format!( format!(
r#" r#"
[pod] [pod]
@ -153,7 +151,7 @@ model_id = "test-model"
max_tokens = 100 max_tokens = 100
[compaction] [compaction]
prune_protected_turns = {prune_protected_turns} prune_protected_tokens = {prune_protected_tokens}
prune_min_savings = {prune_min_savings} prune_min_savings = {prune_min_savings}
[[scope.allow]] [[scope.allow]]
@ -194,7 +192,7 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
// Run 1 (request 0): tool_use → triggers tool execution → request 1 // Run 1 (request 0): tool_use → triggers tool execution → request 1
// on the second iteration to produce the assistant reply. // on the second iteration to produce the assistant reply.
// Run 2 (request 2): plain assistant text. Prune evaluation here // Run 2 (request 2): plain assistant text. Prune evaluation here
// sees user1's tool_result outside the 1-protected-turn window and // sees user1's tool_result outside the protected-token suffix and
// should fire. // should fire.
let client = MockClient::new(vec![ let client = MockClient::new(vec![
tool_use_response("call-1", "big_tool"), tool_use_response("call-1", "big_tool"),
@ -203,6 +201,7 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
]); ]);
let (mut pod, _store_tmp, _pwd_tmp) = make_pod(manifest_toml(1, 1), client, "big_tool").await; let (mut pod, _store_tmp, _pwd_tmp) = make_pod(manifest_toml(1, 1), client, "big_tool").await;
let session_id = pod.session_id(); let session_id = pod.session_id();
let segment_id = pod.segment_id();
// Cloning the store handle to read the session log back after the // Cloning the store handle to read the session log back after the
// runs complete — the Pod retains its own copy. // runs complete — the Pod retains its own copy.
let store = pod.store().clone(); let store = pod.store().clone();
@ -210,7 +209,7 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
pod.run_text("first").await.unwrap(); pod.run_text("first").await.unwrap();
pod.run_text("second").await.unwrap(); pod.run_text("second").await.unwrap();
let state = session_store::restore(&store, session_id).unwrap(); let state = session_store::restore(&store, session_id, segment_id).unwrap();
let metrics = metrics_from_extensions(&state.extensions); let metrics = metrics_from_extensions(&state.extensions);
// Run 1 has 2 LLM iterations (tool loop), each evaluates prune with // Run 1 has 2 LLM iterations (tool loop), each evaluates prune with
@ -251,8 +250,8 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
"fire missing candidate_count: {fire:?}" "fire missing candidate_count: {fire:?}"
); );
assert!( assert!(
fire.dimensions.contains_key("border_turn"), fire.dimensions.contains_key("protected_start_index"),
"fire missing border_turn: {fire:?}" "fire missing protected_start_index: {fire:?}"
); );
assert!(fire.value.is_some(), "fire missing estimated_savings value"); assert!(fire.value.is_some(), "fire missing estimated_savings value");
let fire_id = fire let fire_id = fire
@ -278,6 +277,36 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
assert!(post.dimensions.contains_key("history_len")); assert!(post.dimensions.contains_key("history_len"));
} }
#[tokio::test]
async fn prune_metrics_fire_during_single_long_task_without_multiple_user_turns() {
let client = MockClient::new(vec![
tool_use_response("call-1", "big_tool"),
tool_use_response("call-2", "big_tool"),
tool_use_response("call-3", "big_tool"),
tool_use_response("call-4", "big_tool"),
text_response_with_cache("done", 100, 20),
]);
let (mut pod, _store_tmp, _pwd_tmp) = make_pod(manifest_toml(1, 1), client, "big_tool").await;
let session_id = pod.session_id();
let segment_id = pod.segment_id();
let store = pod.store().clone();
pod.run_text("one long task").await.unwrap();
let state = session_store::restore(&store, session_id, segment_id).unwrap();
let metrics = metrics_from_extensions(&state.extensions);
let fire_count = metrics.iter().filter(|m| m.name == "prune.fire").count();
assert!(
fire_count > 0,
"single-turn tool loop should produce prune.fire once old heavy ToolResults fall outside the protected-token suffix: {metrics:?}"
);
assert!(
metrics.iter().any(|m| {
m.name == "prune.fire" && m.dimensions.contains_key("protected_start_index")
})
);
}
/// `min_savings` set high enough that candidates exist but the estimated /// `min_savings` set high enough that candidates exist but the estimated
/// savings always fall short → the second run should record /// savings always fall short → the second run should record
/// `prune.skip { reason: "below_min_savings" }`. /// `prune.skip { reason: "below_min_savings" }`.
@ -289,14 +318,15 @@ async fn prune_metrics_record_below_min_savings_skip() {
text_response_with_cache("done", 0, 0), text_response_with_cache("done", 0, 0),
]); ]);
let (mut pod, _store_tmp, _pwd_tmp) = let (mut pod, _store_tmp, _pwd_tmp) =
make_pod(manifest_toml(1, u64::MAX), client, "big_tool").await; make_pod(manifest_toml(1, 1_000_000), client, "big_tool").await;
let session_id = pod.session_id(); let session_id = pod.session_id();
let segment_id = pod.segment_id();
let store = pod.store().clone(); let store = pod.store().clone();
pod.run_text("first").await.unwrap(); pod.run_text("first").await.unwrap();
pod.run_text("second").await.unwrap(); pod.run_text("second").await.unwrap();
let state = session_store::restore(&store, session_id).unwrap(); let state = session_store::restore(&store, session_id, segment_id).unwrap();
let metrics = metrics_from_extensions(&state.extensions); let metrics = metrics_from_extensions(&state.extensions);
let below = metrics let below = metrics
.iter() .iter()
@ -329,35 +359,60 @@ struct MetricFailingStore {
} }
impl Store for MetricFailingStore { impl Store for MetricFailingStore {
fn append(&self, id: SessionId, entry: &HashedEntry) -> Result<(), StoreError> { fn append(
if let LogEntry::Extension { domain, .. } = &entry.entry { &self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
if let LogEntry::Extension { domain, .. } = entry {
if domain == DOMAIN { if domain == DOMAIN {
return Err(StoreError::Io(std::io::Error::other("synthetic failure"))); return Err(StoreError::Io(std::io::Error::other("synthetic failure")));
} }
} }
self.inner.append(id, entry) self.inner.append(session_id, segment_id, entry)
} }
fn read_all(&self, id: SessionId) -> Result<Vec<HashedEntry>, StoreError> { fn read_all(
self.inner.read_all(id) &self,
session_id: SessionId,
segment_id: SegmentId,
) -> Result<Vec<LogEntry>, StoreError> {
self.inner.read_all(session_id, segment_id)
} }
fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError> { fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError> {
self.inner.list_sessions() self.inner.list_sessions()
} }
fn create_session( fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError> {
self.inner.list_segments(session_id)
}
fn lookup_session_of(&self, segment_id: SegmentId) -> Result<Option<SessionId>, StoreError> {
self.inner.lookup_session_of(segment_id)
}
fn create_segment(
&self, &self,
id: SessionId, session_id: SessionId,
entries: &[HashedEntry], segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> { ) -> Result<(), StoreError> {
self.inner.create_session(id, entries) self.inner.create_segment(session_id, segment_id, entries)
} }
fn exists(&self, id: SessionId) -> Result<bool, StoreError> { fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
self.inner.exists(id) self.inner.exists(session_id, segment_id)
} }
fn read_head_hash(&self, id: SessionId) -> Result<Option<EntryHash>, StoreError> { fn read_entry_count(
self.inner.read_head_hash(id) &self,
session_id: SessionId,
segment_id: SegmentId,
) -> Result<usize, StoreError> {
self.inner.read_entry_count(session_id, segment_id)
} }
fn append_trace(&self, id: SessionId, entry: &TraceEntry) -> Result<(), StoreError> { fn append_trace(
self.inner.append_trace(id, entry) &self,
session_id: SessionId,
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError> {
self.inner.append_trace(session_id, segment_id, entry)
} }
} }
@ -380,7 +435,7 @@ async fn metric_write_failure_emits_warn_alert_and_does_not_abort_run() {
// Even with a tool registered, this run will only emit // Even with a tool registered, this run will only emit
// `prune.skip { reason: "no_candidates" }` (one user message, // `prune.skip { reason: "no_candidates" }` (one user message,
// protected_turns=1 covers everything). That is enough to drive // protected token budget covers the only user message). That is enough to drive
// the failure path: at least one metric attempts to write. // the failure path: at least one metric attempts to write.
let client = MockClient::new(vec![text_response_with_cache("hi", 0, 0)]); let client = MockClient::new(vec![text_response_with_cache("hi", 0, 0)]);
let worker = Worker::new(client); let worker = Worker::new(client);
@ -393,11 +448,12 @@ async fn metric_write_failure_emits_warn_alert_and_does_not_abort_run() {
pod.attach_alerter(alerter); pod.attach_alerter(alerter);
let session_id = pod.session_id(); let session_id = pod.session_id();
let segment_id = pod.segment_id();
// Run completes successfully despite metric failure. // Run completes successfully despite metric failure.
pod.run_text("hello").await.unwrap(); pod.run_text("hello").await.unwrap();
// No metrics ended up in the log (writes were rejected). // No metrics ended up in the log (writes were rejected).
let state = session_store::restore(&store, session_id).unwrap(); let state = session_store::restore(&store, session_id, segment_id).unwrap();
let metrics = metrics_from_extensions(&state.extensions); let metrics = metrics_from_extensions(&state.extensions);
assert!(metrics.is_empty(), "metrics must drop on write failure"); assert!(metrics.is_empty(), "metrics must drop on write failure");
@ -453,9 +509,10 @@ permission = "write"
.await .await
.unwrap(); .unwrap();
let session_id = pod.session_id(); let session_id = pod.session_id();
let segment_id = pod.segment_id();
pod.run_text("hello").await.unwrap(); pod.run_text("hello").await.unwrap();
let state = session_store::restore(&store, session_id).unwrap(); let state = session_store::restore(&store, session_id, segment_id).unwrap();
let metrics = metrics_from_extensions(&state.extensions); let metrics = metrics_from_extensions(&state.extensions);
assert!( assert!(
metrics.is_empty(), metrics.is_empty(),

View File

@ -73,7 +73,7 @@ async fn setup_spawner(
permission: Permission::Write, permission: Permission::Write,
recursive: true, recursive: true,
}], }],
session_store::new_session_id(), session_store::new_segment_id(),
) )
.unwrap(); .unwrap();
// Leak the guard — the spawner allocation needs to outlive the // Leak the guard — the spawner allocation needs to outlive the

View File

@ -182,16 +182,19 @@ async fn session_start_state_captures_rendered_prompt() {
.unwrap(); .unwrap();
pod.run_text("hi").await.unwrap(); pod.run_text("hi").await.unwrap();
let entries = pod.store().read_all(pod.session_id()).unwrap(); let entries = pod
.store()
.read_all(pod.session_id(), pod.segment_id())
.unwrap();
let first = entries.first().expect("at least one entry"); let first = entries.first().expect("at least one entry");
match &first.entry { match first {
LogEntry::SessionStart { system_prompt, .. } => { LogEntry::SegmentStart { system_prompt, .. } => {
let sp = system_prompt.as_deref().expect("system prompt set"); let sp = system_prompt.as_deref().expect("system prompt set");
assert!(sp.starts_with("hello cwd=")); assert!(sp.starts_with("hello cwd="));
assert!(sp.contains(&pwd.display().to_string())); assert!(sp.contains(&pwd.display().to_string()));
assert!(sp.contains("## Working boundaries")); assert!(sp.contains("## Working boundaries"));
} }
other => panic!("expected SessionStart as first entry, got {other:?}"), other => panic!("expected SegmentStart as first entry, got {other:?}"),
} }
} }

View File

@ -226,9 +226,7 @@ pub enum Event {
/// `[File: …]`. /// `[File: …]`.
/// ///
/// One event per `LogEntry::SystemItem` commit. Disk-side and /// One event per `LogEntry::SystemItem` commit. Disk-side and
/// wire-side are 1:1 (singular variant); legacy `SystemItems` /// wire-side are 1:1.
/// entries from older sessions are read-only and never emitted on
/// this lane.
SystemItem { SystemItem {
item: serde_json::Value, item: serde_json::Value,
}, },
@ -363,24 +361,23 @@ pub enum Event {
/// ///
/// Live updates after the snapshot arrive through the streaming /// Live updates after the snapshot arrive through the streaming
/// events (`TextDelta` / `ToolCall*` / `ToolResult` / etc.) plus /// events (`TextDelta` / `ToolCall*` / `ToolResult` / etc.) plus
/// the two role-specific entry events /// role-specific entry events (`SegmentRotated` / `SystemItem`) —
/// (`SessionRotated` / `HookInjectedItems`) — there is no generic /// there is no generic "every committed entry" broadcast.
/// "every committed entry" broadcast.
Snapshot { Snapshot {
entries: Vec<serde_json::Value>, entries: Vec<serde_json::Value>,
greeting: Greeting, greeting: Greeting,
#[serde(default)] #[serde(default)]
status: PodStatus, status: PodStatus,
}, },
/// Server-side session log rotated to a fresh `SessionStart`. /// Server-side segment log rotated to a fresh `SegmentStart`.
/// ///
/// Fires on compaction and on auto-fork when the store head drifts /// Fires on compaction and on auto-fork when the store head drifts
/// from the live writer's cached head. Clients drop their derived /// from the live writer's cached head. Clients drop their derived
/// view and reseed from `entry.history` exactly the way they would /// view and reseed from `entry.history` exactly the way they would
/// from a connect-time `Snapshot`. /// from a connect-time `Snapshot`.
/// ///
/// Payload is the JSON form of `session_store::LogEntry::SessionStart`. /// Payload is the JSON form of `session_store::LogEntry::SegmentStart`.
SessionRotated { SegmentRotated {
entry: serde_json::Value, entry: serde_json::Value,
}, },
/// Current Pod controller status. Broadcast on every controller-level /// Current Pod controller status. Broadcast on every controller-level
@ -400,15 +397,15 @@ pub enum Event {
/// Pod has started compacting the current session. /// Pod has started compacting the current session.
/// ///
/// Fired immediately before a compaction run. Success is signalled by /// Fired immediately before a compaction run. Success is signalled by
/// `CompactDone` (with the new `SessionId`); failure by `CompactFailed`. /// `CompactDone` (with the new `SegmentId`); failure by `CompactFailed`.
/// Broadcast to all clients; not replayed to late subscribers. /// Broadcast to all clients; not replayed to late subscribers.
CompactStart, CompactStart,
/// Compaction completed and the session was rotated. /// Compaction completed and the session was rotated.
/// ///
/// `new_session_id` is the UUID of the freshly created session that /// `new_segment_id` is the UUID of the freshly created session that
/// replaced the old history. /// replaced the old history.
CompactDone { CompactDone {
new_session_id: uuid::Uuid, new_segment_id: uuid::Uuid,
}, },
/// Compaction failed. The session is unchanged. /// Compaction failed. The session is unchanged.
CompactFailed { CompactFailed {
@ -489,6 +486,12 @@ pub struct Greeting {
pub model: String, pub model: String,
pub scope_summary: String, pub scope_summary: String,
pub tools: Vec<String>, pub tools: Vec<String>,
/// Model context window in tokens. Always filled by the Pod greeting.
#[serde(default)]
pub context_window: u64,
/// Estimated current session context tokens at connect time.
#[serde(default)]
pub context_tokens: u64,
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@ -876,6 +879,8 @@ mod tests {
model: "claude".into(), model: "claude".into(),
scope_summary: "Writable:\n - /tmp".into(), scope_summary: "Writable:\n - /tmp".into(),
tools: vec!["Read".into()], tools: vec!["Read".into()],
context_window: 200_000,
context_tokens: 42_000,
}, },
status: PodStatus::Paused, status: PodStatus::Paused,
}; };
@ -886,22 +891,24 @@ mod tests {
assert_eq!(parsed["data"]["entries"][0]["kind"], "user_input"); assert_eq!(parsed["data"]["entries"][0]["kind"], "user_input");
assert_eq!(parsed["data"]["greeting"]["pod_name"], "test"); assert_eq!(parsed["data"]["greeting"]["pod_name"], "test");
assert_eq!(parsed["data"]["greeting"]["tools"][0], "Read"); assert_eq!(parsed["data"]["greeting"]["tools"][0], "Read");
assert_eq!(parsed["data"]["greeting"]["context_window"], 200_000);
assert_eq!(parsed["data"]["greeting"]["context_tokens"], 42_000);
assert_eq!(parsed["data"]["status"], "paused"); assert_eq!(parsed["data"]["status"], "paused");
} }
#[test] #[test]
fn event_session_rotated_roundtrip() { fn event_segment_rotated_roundtrip() {
let event = Event::SessionRotated { let event = Event::SegmentRotated {
entry: serde_json::json!({"kind": "session_start", "ts": 1, "history": []}), entry: serde_json::json!({"kind": "segment_start", "ts": 1, "history": []}),
}; };
let json = serde_json::to_string(&event).unwrap(); let json = serde_json::to_string(&event).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap(); let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["event"], "session_rotated"); assert_eq!(parsed["event"], "segment_rotated");
assert_eq!(parsed["data"]["entry"]["kind"], "session_start"); assert_eq!(parsed["data"]["entry"]["kind"], "segment_start");
let decoded: Event = serde_json::from_str(&json).unwrap(); let decoded: Event = serde_json::from_str(&json).unwrap();
match decoded { match decoded {
Event::SessionRotated { entry } => assert_eq!(entry["kind"], "session_start"), Event::SegmentRotated { entry } => assert_eq!(entry["kind"], "segment_start"),
other => panic!("expected SessionRotated, got {other:?}"), other => panic!("expected SegmentRotated, got {other:?}"),
} }
} }
@ -945,7 +952,13 @@ mod tests {
let json = r#"{"event":"snapshot","data":{"entries":[],"greeting":{"pod_name":"test","cwd":"/tmp","provider":"anthropic","model":"claude","scope_summary":"","tools":[]}}}"#; let json = r#"{"event":"snapshot","data":{"entries":[],"greeting":{"pod_name":"test","cwd":"/tmp","provider":"anthropic","model":"claude","scope_summary":"","tools":[]}}}"#;
let decoded: Event = serde_json::from_str(json).unwrap(); let decoded: Event = serde_json::from_str(json).unwrap();
match decoded { match decoded {
Event::Snapshot { status, .. } => assert_eq!(status, PodStatus::Idle), Event::Snapshot {
status, greeting, ..
} => {
assert_eq!(status, PodStatus::Idle);
assert_eq!(greeting.context_window, 0);
assert_eq!(greeting.context_tokens, 0);
}
other => panic!("expected Snapshot, got {other:?}"), other => panic!("expected Snapshot, got {other:?}"),
} }
} }
@ -1060,17 +1073,17 @@ mod tests {
#[test] #[test]
fn event_compact_done_roundtrip() { fn event_compact_done_roundtrip() {
let id = uuid::Uuid::parse_str("0192f0e8-4d84-7d6e-a000-000000000001").unwrap(); let id = uuid::Uuid::parse_str("0192f0e8-4d84-7d6e-a000-000000000001").unwrap();
let event = Event::CompactDone { new_session_id: id }; let event = Event::CompactDone { new_segment_id: id };
let json = serde_json::to_string(&event).unwrap(); let json = serde_json::to_string(&event).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap(); let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["event"], "compact_done"); assert_eq!(parsed["event"], "compact_done");
assert_eq!( assert_eq!(
parsed["data"]["new_session_id"], parsed["data"]["new_segment_id"],
"0192f0e8-4d84-7d6e-a000-000000000001" "0192f0e8-4d84-7d6e-a000-000000000001"
); );
let decoded: Event = serde_json::from_str(&json).unwrap(); let decoded: Event = serde_json::from_str(&json).unwrap();
match decoded { match decoded {
Event::CompactDone { new_session_id } => assert_eq!(new_session_id, id), Event::CompactDone { new_segment_id } => assert_eq!(new_segment_id, id),
other => panic!("expected CompactDone, got {other:?}"), other => panic!("expected CompactDone, got {other:?}"),
} }
} }

View File

@ -18,3 +18,4 @@
- `AuthRef::None` / `AuthRef::CodexOAuth` の解決 - `AuthRef::None` / `AuthRef::CodexOAuth` の解決
- `Scheme::required_auth()``ResolvedAuth` の妥当性検証(非対応組合せは構築エラー) - `Scheme::required_auth()``ResolvedAuth` の妥当性検証(非対応組合せは構築エラー)
- capability は manifest 明示 > model catalog > provider.default_capability > `Scheme::default_capability()` の順で解決 - capability は manifest 明示 > model catalog > provider.default_capability > `Scheme::default_capability()` の順で解決
- context window は manifest 明示 > model catalog > provider.default_context_window > builtin fallback の順で解決し、inline model でも `context_window` で override できる

View File

@ -22,6 +22,11 @@ use serde::{Deserialize, Serialize};
const BUILTIN_PROVIDERS: &str = include_str!("../../../resources/providers/builtin.toml"); const BUILTIN_PROVIDERS: &str = include_str!("../../../resources/providers/builtin.toml");
const BUILTIN_MODELS: &str = include_str!("../../../resources/models/builtin.toml"); const BUILTIN_MODELS: &str = include_str!("../../../resources/models/builtin.toml");
/// Conservative fallback used when neither the manifest nor catalogs specify
/// a model context window. Greeting still carries a concrete number, while
/// catalog / manifest metadata can override unknown or inline models.
pub const DEFAULT_CONTEXT_WINDOW: u64 = 200_000;
#[derive(Debug, thiserror::Error)] #[derive(Debug, thiserror::Error)]
pub enum CatalogError { pub enum CatalogError {
#[error("failed to read catalog at {path}: {source}")] #[error("failed to read catalog at {path}: {source}")]
@ -92,6 +97,10 @@ pub struct ProviderEntry {
/// 使う。 /// 使う。
#[serde(default)] #[serde(default)]
pub default_capability: Option<ModelCapability>, pub default_capability: Option<ModelCapability>,
/// モデルカタログ未登録モデルで使う既定の context window。省略時は
/// [`DEFAULT_CONTEXT_WINDOW`] を使う。
#[serde(default)]
pub default_context_window: Option<u64>,
} }
/// モデルカタログの 1 エントリ。 /// モデルカタログの 1 エントリ。
@ -107,6 +116,10 @@ pub struct ModelEntry {
/// `ProviderEntry::default_capability` にフォールバックする。 /// `ProviderEntry::default_capability` にフォールバックする。
#[serde(default)] #[serde(default)]
pub capability: Option<ModelCapability>, pub capability: Option<ModelCapability>,
/// モデル単位の context window。省略時は provider default → builtin
/// fallback にフォールバックする。
#[serde(default)]
pub context_window: Option<u64>,
} }
/// 解決済みモデル設定。`build_client` が消費する完成形。 /// 解決済みモデル設定。`build_client` が消費する完成形。
@ -117,6 +130,7 @@ pub struct ModelConfig {
pub model_id: String, pub model_id: String,
pub auth: AuthRef, pub auth: AuthRef,
pub capability: Option<ModelCapability>, pub capability: Option<ModelCapability>,
pub context_window: u64,
} }
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
@ -244,6 +258,8 @@ fn split_ref(s: &str) -> Option<(&str, &str)> {
/// auth は manifest 明示 > provider.auth_hint 由来、capability は /// auth は manifest 明示 > provider.auth_hint 由来、capability は
/// manifest 明示 > model catalog > provider.default_capability > /// manifest 明示 > model catalog > provider.default_capability >
/// `build_client` 側で)`Scheme::default_capability()`。 /// `build_client` 側で)`Scheme::default_capability()`。
/// context_window は manifest 明示 > model catalog > provider default >
/// [`DEFAULT_CONTEXT_WINDOW`]。
pub fn resolve_model_manifest(manifest: &ModelManifest) -> Result<ModelConfig, ResolveError> { pub fn resolve_model_manifest(manifest: &ModelManifest) -> Result<ModelConfig, ResolveError> {
let providers = load_providers().map_err(ResolveError::LoadProviders)?; let providers = load_providers().map_err(ResolveError::LoadProviders)?;
let models = load_models().map_err(ResolveError::LoadModels)?; let models = load_models().map_err(ResolveError::LoadModels)?;
@ -294,12 +310,18 @@ pub fn resolve_with_catalogs(
.and_then(|m| m.capability.clone()) .and_then(|m| m.capability.clone())
.or_else(|| provider.default_capability.clone()) .or_else(|| provider.default_capability.clone())
}); });
let context_window = manifest
.context_window
.or_else(|| model_entry.and_then(|m| m.context_window))
.or(provider.default_context_window)
.unwrap_or(DEFAULT_CONTEXT_WINDOW);
Ok(ModelConfig { Ok(ModelConfig {
scheme, scheme,
base_url, base_url,
model_id, model_id,
auth, auth,
capability, capability,
context_window,
}) })
} else { } else {
let scheme = manifest let scheme = manifest
@ -319,6 +341,7 @@ pub fn resolve_with_catalogs(
model_id, model_id,
auth, auth,
capability: manifest.capability.clone(), capability: manifest.capability.clone(),
context_window: manifest.context_window.unwrap_or(DEFAULT_CONTEXT_WINDOW),
}) })
} }
} }
@ -381,6 +404,20 @@ mod tests {
cfg.capability.is_some(), cfg.capability.is_some(),
"should fall back to provider.default_capability" "should fall back to provider.default_capability"
); );
assert_eq!(cfg.context_window, 200_000);
}
#[test]
fn context_window_manifest_overrides_catalog() {
let providers = load_builtin_providers().unwrap();
let models = load_builtin_models().unwrap();
let manifest = ModelManifest {
ref_: Some("anthropic/claude-sonnet-4-6".into()),
context_window: Some(123_456),
..Default::default()
};
let cfg = resolve_with_catalogs(&manifest, &providers, &models).unwrap();
assert_eq!(cfg.context_window, 123_456);
} }
#[test] #[test]
@ -461,6 +498,25 @@ mod tests {
assert_eq!(cfg.scheme, SchemeKind::Anthropic); assert_eq!(cfg.scheme, SchemeKind::Anthropic);
assert_eq!(cfg.model_id, "claude-sonnet-4-6"); assert_eq!(cfg.model_id, "claude-sonnet-4-6");
assert!(cfg.capability.is_none(), "no catalog hit for inline-only"); assert!(cfg.capability.is_none(), "no catalog hit for inline-only");
assert_eq!(cfg.context_window, DEFAULT_CONTEXT_WINDOW);
}
#[test]
fn resolve_inline_context_window_override() {
let providers = load_builtin_providers().unwrap();
let models = load_builtin_models().unwrap();
let manifest = ModelManifest {
scheme: Some(SchemeKind::Anthropic),
model_id: Some("claude-sonnet-4-6".into()),
auth: Some(AuthRef::ApiKey {
env: None,
file: Some(PathBuf::from("/tmp/sk")),
}),
context_window: Some(777_000),
..Default::default()
};
let cfg = resolve_with_catalogs(&manifest, &providers, &models).unwrap();
assert_eq!(cfg.context_window, 777_000);
} }
#[test] #[test]

View File

@ -186,6 +186,7 @@ mod tests {
file: None, file: None,
}, },
capability: None, capability: None,
context_window: 200_000,
} }
} }
@ -313,6 +314,7 @@ mod tests {
model_id: "llama3".into(), model_id: "llama3".into(),
auth: AuthRef::None, auth: AuthRef::None,
capability: None, capability: None,
context_window: 200_000,
}; };
assert!(build_client_from_config(&config).is_ok()); assert!(build_client_from_config(&config).is_ok());
} }

View File

@ -18,7 +18,7 @@
use std::collections::BTreeMap; use std::collections::BTreeMap;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use session_store::{EntryHash, SessionId, Store, StoreError, save_extension, session_log}; use session_store::{SegmentId, SessionId, Store, StoreError, save_extension, segment_log};
/// Domain tag used in `LogEntry::Extension` for all metrics records. /// Domain tag used in `LogEntry::Extension` for all metrics records.
pub const DOMAIN: &str = "metrics"; pub const DOMAIN: &str = "metrics";
@ -48,7 +48,7 @@ impl Metric {
pub fn now(name: impl Into<String>) -> Self { pub fn now(name: impl Into<String>) -> Self {
Self { Self {
name: name.into(), name: name.into(),
ts: session_log::now_millis(), ts: segment_log::now_millis(),
dimensions: BTreeMap::new(), dimensions: BTreeMap::new(),
value: None, value: None,
correlation_id: None, correlation_id: None,
@ -78,11 +78,11 @@ impl Metric {
pub fn record_metric( pub fn record_metric(
store: &impl Store, store: &impl Store,
session_id: SessionId, session_id: SessionId,
head_hash: &mut Option<EntryHash>, segment_id: SegmentId,
metric: &Metric, metric: &Metric,
) -> Result<(), StoreError> { ) -> Result<(), StoreError> {
let payload = serde_json::to_value(metric).expect("Metric serialization cannot fail"); let payload = serde_json::to_value(metric).expect("Metric serialization cannot fail");
save_extension(store, session_id, head_hash, DOMAIN, payload) save_extension(store, session_id, segment_id, DOMAIN, payload)
} }
/// `RestoredState.extensions` から metrics domain の payload を順に取り出し、 /// `RestoredState.extensions` から metrics domain の payload を順に取り出し、
@ -104,7 +104,7 @@ mod tests {
#[test] #[test]
fn metric_round_trip_via_json() { fn metric_round_trip_via_json() {
let metric = Metric::now("prune.fire") let metric = Metric::now("prune.fire")
.with_dimension("border_turn", "3") .with_dimension("protected_start_index", "3")
.with_dimension("candidate_count", "2") .with_dimension("candidate_count", "2")
.with_value(4096.0) .with_value(4096.0)
.with_correlation_id("abc-123"); .with_correlation_id("abc-123");

View File

@ -11,8 +11,6 @@ serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true } serde_json = { workspace = true }
uuid = { workspace = true, features = ["v7", "serde"] } uuid = { workspace = true, features = ["v7", "serde"] }
thiserror = { workspace = true } thiserror = { workspace = true }
sha2 = { workspace = true }
hex = "0.4.3"
protocol = { workspace = true } protocol = { workspace = true }
tracing.workspace = true tracing.workspace = true

View File

@ -20,7 +20,7 @@ Worker のセッション永続化を提供するクレート。追記専用の
### ログ ### ログ
- `LogEntry` — セッションログのエントリ型(`SessionStart`, `UserInput`, `AssistantItems`, `TurnEnd` など) - `LogEntry` — セッションログのエントリ型(`SegmentStart`, `UserInput`, `AssistantItem`, `ToolResult`, `SystemItem`, `TurnEnd` など)
- `RestoredState` — ログ再生で復元された状態 - `RestoredState` — ログ再生で復元された状態
- `collect_state()` — ログエントリ列から状態を復元する関数 - `collect_state()` — ログエントリ列から状態を復元する関数

View File

@ -2,7 +2,7 @@
//! //!
//! [`TraceEntry`] captures every LLM stream event verbatim for debugging //! [`TraceEntry`] captures every LLM stream event verbatim for debugging
//! and post-hoc analysis. Written to a separate `.trace.jsonl` file, //! and post-hoc analysis. Written to a separate `.trace.jsonl` file,
//! completely independent of the session log used for state restoration. //! completely independent of the segment log used for state restoration.
//! //!
//! Disabled by default. Enable via `SessionConfig::record_event_trace`. //! Disabled by default. Enable via `SessionConfig::record_event_trace`.

View File

@ -1,20 +1,33 @@
//! Filesystem-backed JSONL store. //! Filesystem-backed JSONL store.
//! //!
//! Layout: //! Layout:
//! - Session log: `{root}/{session_id}.jsonl` //! - Segment log: `{root}/{session_id}/{segment_id}.jsonl`
//! - Event trace: `{root}/{session_id}.trace.jsonl` //! - Event trace: `{root}/{session_id}/{segment_id}.trace.jsonl`
//! - Pod metadata: `{root}/pods/{pod_name}/metadata.json`
//!
//! The per-Session directory makes `list_segments(session_id)` an O(dir)
//! scan and gives the fork tree a visible grouping in the filesystem.
//!
//! Migration: this layout is incompatible with the pre-`session-grouping`
//! flat `{root}/{segment_id}.jsonl` form. Project policy is no
//! backward compatibility — discard `~/.insomnia/sessions/` (or whatever
//! `root` resolved to) before running the new code. `list_sessions`
//! ignores top-level files outside session directories, so leftover
//! flat files do not corrupt new sessions, but they are no longer
//! enumerable by the picker.
use crate::SessionId;
use crate::event_trace::TraceEntry; use crate::event_trace::TraceEntry;
use crate::session_log::{EntryHash, HashedEntry}; use crate::pod_metadata::{PodMetadata, PodMetadataStore, validate_pod_name};
use crate::segment_log::LogEntry;
use crate::store::{Store, StoreError}; use crate::store::{Store, StoreError};
use crate::{SegmentId, SessionId};
use std::fs; use std::fs;
use std::io::Write; use std::io::Write;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
/// Filesystem-backed JSONL store. /// Filesystem-backed JSONL store.
/// ///
/// Each session is stored as a single `.jsonl` file with one [`LogEntry`] /// Each segment is stored as a single `.jsonl` file with one [`LogEntry`]
/// per line. Writes use append mode for crash safety. /// per line. Writes use append mode for crash safety.
#[derive(Clone)] #[derive(Clone)]
pub struct FsStore { pub struct FsStore {
@ -30,16 +43,41 @@ impl FsStore {
Ok(Self { root }) Ok(Self { root })
} }
fn log_path(&self, id: SessionId) -> PathBuf { fn session_dir(&self, session_id: SessionId) -> PathBuf {
self.root.join(format!("{id}.jsonl")) self.root.join(session_id.to_string())
} }
fn trace_path(&self, id: SessionId) -> PathBuf { fn log_path(&self, session_id: SessionId, segment_id: SegmentId) -> PathBuf {
self.root.join(format!("{id}.trace.jsonl")) self.session_dir(session_id)
.join(format!("{segment_id}.jsonl"))
}
fn trace_path(&self, session_id: SessionId, segment_id: SegmentId) -> PathBuf {
self.session_dir(session_id)
.join(format!("{segment_id}.trace.jsonl"))
}
fn pods_dir(&self) -> PathBuf {
self.root.join("pods")
}
fn pod_dir(&self, pod_name: &str) -> Result<PathBuf, StoreError> {
validate_pod_name(pod_name)?;
Ok(self.pods_dir().join(pod_name))
}
fn pod_metadata_path(&self, pod_name: &str) -> Result<PathBuf, StoreError> {
Ok(self.pod_dir(pod_name)?.join("metadata.json"))
} }
fn append_line(&self, path: &Path, line: &str) -> Result<(), StoreError> { fn append_line(&self, path: &Path, line: &str) -> Result<(), StoreError> {
let mut file = fs::OpenOptions::new().create(true).append(true).open(path)?; if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
}
let mut file = fs::OpenOptions::new()
.create(true)
.append(true)
.open(path)?;
file.write_all(line.as_bytes())?; file.write_all(line.as_bytes())?;
file.write_all(b"\n")?; file.write_all(b"\n")?;
// Append-mode write is the durability boundary; an explicit // Append-mode write is the durability boundary; an explicit
@ -64,16 +102,60 @@ impl FsStore {
} }
} }
impl Store for FsStore { impl PodMetadataStore for FsStore {
fn append(&self, id: SessionId, entry: &HashedEntry) -> Result<(), StoreError> { fn write(&self, metadata: &PodMetadata) -> Result<(), StoreError> {
let line = serde_json::to_string(entry)?; let path = self.pod_metadata_path(&metadata.pod_name)?;
self.append_line(&self.log_path(id), &line) if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
}
let content = serde_json::to_vec_pretty(metadata)?;
fs::write(path, content)?;
Ok(())
} }
fn read_all(&self, id: SessionId) -> Result<Vec<HashedEntry>, StoreError> { fn read_by_name(&self, pod_name: &str) -> Result<Option<PodMetadata>, StoreError> {
let path = self.log_path(id); let path = self.pod_metadata_path(pod_name)?;
let content = match fs::read_to_string(path) {
Ok(content) => content,
Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(None),
Err(err) => return Err(StoreError::Io(err)),
};
Ok(Some(serde_json::from_str(&content)?))
}
fn delete_by_name(&self, pod_name: &str) -> Result<(), StoreError> {
let path = self.pod_metadata_path(pod_name)?;
match fs::remove_file(&path) {
Ok(()) => {}
Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(()),
Err(err) => return Err(StoreError::Io(err)),
}
if let Some(parent) = path.parent() {
let _ = fs::remove_dir(parent);
}
Ok(())
}
}
impl Store for FsStore {
fn append(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError> {
let line = serde_json::to_string(entry)?;
self.append_line(&self.log_path(session_id, segment_id), &line)
}
fn read_all(
&self,
session_id: SessionId,
segment_id: SegmentId,
) -> Result<Vec<LogEntry>, StoreError> {
let path = self.log_path(session_id, segment_id);
if !path.exists() { if !path.exists() {
return Err(StoreError::NotFound(id)); return Err(StoreError::NotFound(segment_id));
} }
let content = fs::read_to_string(&path)?; let content = fs::read_to_string(&path)?;
Self::parse_jsonl(&content) Self::parse_jsonl(&content)
@ -81,25 +163,77 @@ impl Store for FsStore {
fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError> { fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError> {
let mut sessions = Vec::new(); let mut sessions = Vec::new();
if !self.root.exists() {
return Ok(sessions);
}
for entry in fs::read_dir(&self.root)? { for entry in fs::read_dir(&self.root)? {
let entry = entry?;
if !entry.file_type()?.is_dir() {
continue;
}
if let Some(name) = entry.file_name().to_str() {
if let Ok(id) = name.parse::<SessionId>() {
sessions.push(id);
}
}
}
sessions.sort_by(|a, b| b.cmp(a));
Ok(sessions)
}
fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError> {
let dir = self.session_dir(session_id);
let mut segments = Vec::new();
if !dir.exists() {
return Ok(segments);
}
for entry in fs::read_dir(&dir)? {
let entry = entry?; let entry = entry?;
let path = entry.path(); let path = entry.path();
// Only match .jsonl files, not .trace.jsonl // Only match .jsonl files, not .trace.jsonl
let name = path.file_name().and_then(|n| n.to_str()).unwrap_or(""); let name = path.file_name().and_then(|n| n.to_str()).unwrap_or("");
if name.ends_with(".jsonl") && !name.ends_with(".trace.jsonl") { if name.ends_with(".jsonl") && !name.ends_with(".trace.jsonl") {
let stem = name.trim_end_matches(".jsonl"); let stem = name.trim_end_matches(".jsonl");
if let Ok(id) = stem.parse::<SessionId>() { if let Ok(id) = stem.parse::<SegmentId>() {
sessions.push(id); segments.push(id);
} }
} }
} }
// UUID v7: lexicographic sort = chronological sort, newest first // UUID v7: lexicographic sort = chronological sort, newest first
sessions.sort_by(|a, b| b.cmp(a)); segments.sort_by(|a, b| b.cmp(a));
Ok(sessions) Ok(segments)
} }
fn create_session(&self, id: SessionId, entries: &[HashedEntry]) -> Result<(), StoreError> { fn lookup_session_of(&self, segment_id: SegmentId) -> Result<Option<SessionId>, StoreError> {
let path = self.log_path(id); if !self.root.exists() {
return Ok(None);
}
let needle = format!("{segment_id}.jsonl");
for entry in fs::read_dir(&self.root)? {
let entry = entry?;
if !entry.file_type()?.is_dir() {
continue;
}
if entry.path().join(&needle).exists()
&& let Some(name) = entry.file_name().to_str()
&& let Ok(id) = name.parse::<SessionId>()
{
return Ok(Some(id));
}
}
Ok(None)
}
fn create_segment(
&self,
session_id: SessionId,
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError> {
let path = self.log_path(session_id, segment_id);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
}
let mut content = String::new(); let mut content = String::new();
for entry in entries { for entry in entries {
content.push_str(&serde_json::to_string(entry)?); content.push_str(&serde_json::to_string(entry)?);
@ -109,32 +243,30 @@ impl Store for FsStore {
Ok(()) Ok(())
} }
fn exists(&self, id: SessionId) -> Result<bool, StoreError> { fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
Ok(self.log_path(id).exists()) Ok(self.log_path(session_id, segment_id).exists())
} }
fn read_head_hash(&self, id: SessionId) -> Result<Option<EntryHash>, StoreError> { fn read_entry_count(
let path = self.log_path(id); &self,
session_id: SessionId,
segment_id: SegmentId,
) -> Result<usize, StoreError> {
let path = self.log_path(session_id, segment_id);
if !path.exists() { if !path.exists() {
return Err(StoreError::NotFound(id)); return Err(StoreError::NotFound(segment_id));
} }
let content = fs::read_to_string(&path)?; let content = fs::read_to_string(&path)?;
let last_line = content.lines().rev().find(|l| !l.trim().is_empty()); Ok(content.lines().filter(|l| !l.trim().is_empty()).count())
match last_line {
Some(line) => {
let entry: HashedEntry =
serde_json::from_str(line).map_err(|e| StoreError::Corrupt {
line: content.lines().count(),
message: e.to_string(),
})?;
Ok(Some(entry.hash))
}
None => Ok(None),
}
} }
fn append_trace(&self, id: SessionId, entry: &TraceEntry) -> Result<(), StoreError> { fn append_trace(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError> {
let line = serde_json::to_string(entry)?; let line = serde_json::to_string(entry)?;
self.append_line(&self.trace_path(id), &line) self.append_line(&self.trace_path(session_id, segment_id), &line)
} }
} }

View File

@ -2,35 +2,40 @@
//! //!
//! # Architecture //! # Architecture
//! //!
//! Sessions are recorded as a sequence of [`LogEntry`] values, one per line //! A [`Session`](SessionId) is a fork-tree of [`Segment`](SegmentId)s
//! in a `.jsonl` file. Reading the log and collecting entries reconstructs //! belonging to the same logical conversation. Each Segment is recorded
//! the full Worker state — no separate snapshots or checkpoints needed. //! as a sequence of [`LogEntry`] values, one per line in a `.jsonl`
//! file. Reading a segment log and collecting entries reconstructs the
//! Worker state at that segment — no separate snapshots or checkpoints
//! needed. Compaction and fork operations mint a fresh Segment within
//! the same Session.
//! //!
//! This crate provides free functions for persistence operations. //! This crate provides free functions for persistence operations.
//! The caller (typically Pod) holds the Worker directly and calls these //! The caller (typically Pod) holds the Worker directly and calls these
//! functions after state-mutating operations. //! functions after state-mutating operations.
//! //!
//! Debug-mode [`TraceEntry`] records capture raw stream events in a separate //! Debug-mode [`TraceEntry`] records capture raw stream events in a separate
//! `.trace.jsonl` file, independent of the session log. //! `.trace.jsonl` file, independent of the segment log.
//! //!
//! # Quick start //! # Quick start
//! //!
//! ```ignore //! ```ignore
//! use session_store::{create_session, restore, save_delta, FsStore, SessionStartState}; //! use session_store::{create_segment, restore, save_delta, FsStore, SegmentStartState};
//! //!
//! let store = FsStore::new("./sessions").await?; //! let store = FsStore::new("./sessions")?;
//! let (session_id, head_hash) = create_session(&store, SessionStartState { //! let (session_id, segment_id) = create_segment(&store, SegmentStartState {
//! system_prompt: None, //! system_prompt: None,
//! config: &config, //! config: &config,
//! history: &[], //! history: &[],
//! }).await?; //! })?;
//! ``` //! ```
pub mod event_trace; pub mod event_trace;
pub mod fs_store; pub mod fs_store;
pub mod logged_item; pub mod logged_item;
pub mod session; pub mod pod_metadata;
pub mod session_log; pub mod segment;
pub mod segment_log;
pub mod store; pub mod store;
pub mod system_item; pub mod system_item;
@ -39,24 +44,39 @@ pub use fs_store::FsStore;
pub use llm_worker::UsageRecord; pub use llm_worker::UsageRecord;
pub use llm_worker::llm_client::types::{ContentPart, Item, Role}; pub use llm_worker::llm_client::types::{ContentPart, Item, Role};
pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged}; pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged};
pub use session::{ pub use pod_metadata::{
SessionStartState, append_entry, append_entry_with_hash, append_system_item, PodActiveSegmentRef, PodMetadata, PodMetadataStore, PodSpawnedChild, PodSpawnedScopeRule,
classify_history_item, create_compacted_session, create_session, create_session_with_id, };
ensure_head_or_fork, fork, fork_at, restore, save_config_changed, save_delta, save_extension, pub use segment::{
SegmentStartState, append_entry, append_system_item, classify_history_item,
create_compacted_segment, create_segment, create_segment_with_ids, ensure_head_or_fork, fork,
fork_at, restore, restore_by_segment, save_config_changed, save_delta, save_extension,
save_pod_scope, save_run_completed, save_run_errored, save_turn_end, save_usage, save_pod_scope, save_run_completed, save_run_errored, save_turn_end, save_usage,
save_user_input, save_user_input,
}; };
pub use session_log::{ pub use segment_log::{
EntryHash, HashedEntry, LogEntry, POD_SCOPE_EXTENSION_DOMAIN, PodScopeSnapshot, RestoredState, LogEntry, POD_SCOPE_EXTENSION_DOMAIN, PodScopeSnapshot, RestoredState, SegmentOrigin,
SessionOrigin, build_chain, collect_state, compute_hash, collect_state,
}; };
pub use system_item::{SystemItem, render_pod_event};
pub use store::{Store, StoreError}; pub use store::{Store, StoreError};
pub use system_item::{SystemItem, render_pod_event};
/// Session identifier. UUID v7 (time-ordered, lexicographically sortable). /// Session identifier — the fork-tree root. UUID v7 (time-ordered).
///
/// All Segments belonging to the same Session share this ID. Compaction
/// and fork operations create a new Segment within the same Session, so
/// `WHERE session_id = ?` retrieves the full lineage.
pub type SessionId = uuid::Uuid; pub type SessionId = uuid::Uuid;
/// Segment identifier. UUID v7 (time-ordered, lexicographically sortable).
pub type SegmentId = uuid::Uuid;
/// Generate a new session ID. /// Generate a new session ID.
pub fn new_session_id() -> SessionId { pub fn new_session_id() -> SessionId {
uuid::Uuid::now_v7() uuid::Uuid::now_v7()
} }
/// Generate a new segment ID.
pub fn new_segment_id() -> SegmentId {
uuid::Uuid::now_v7()
}

View File

@ -0,0 +1,109 @@
//! Pod metadata persistence API.
//!
//! Pod metadata is a lightweight name-keyed pointer to the Session/Segment
//! currently active for a Pod. Conversation content remains in the segment log;
//! this metadata only records references needed by Pod-name resume/attach flows.
use crate::store::StoreError;
use crate::{SegmentId, SessionId};
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
/// Active Session/Segment pointer for a Pod.
///
/// `segment_id` is optional so callers can persist a reserved Session before
/// the first Segment ID is known. Once a segment exists, callers should rewrite
/// the metadata with `Some(segment_id)`.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct PodActiveSegmentRef {
pub session_id: SessionId,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub segment_id: Option<SegmentId>,
}
impl PodActiveSegmentRef {
/// Create a reference whose active Segment is not known yet.
pub fn pending_segment(session_id: SessionId) -> Self {
Self {
session_id,
segment_id: None,
}
}
/// Create a fully resolved active Session/Segment reference.
pub fn active_segment(session_id: SessionId, segment_id: SegmentId) -> Self {
Self {
session_id,
segment_id: Some(segment_id),
}
}
}
/// One delegated scope rule for a spawned child, kept local to
/// `session-store` so the persistence crate does not depend on manifest
/// scope types.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct PodSpawnedScopeRule {
pub target: PathBuf,
pub permission: String,
pub recursive: bool,
}
/// One child Pod spawned by this Pod and persisted with the spawner's
/// name-keyed Pod state.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct PodSpawnedChild {
pub pod_name: String,
pub socket_path: PathBuf,
pub scope_delegated: Vec<PodSpawnedScopeRule>,
pub callback_address: PathBuf,
}
/// Persistent metadata for a Pod name.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct PodMetadata {
pub pod_name: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub active: Option<PodActiveSegmentRef>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub spawned_children: Vec<PodSpawnedChild>,
}
impl PodMetadata {
/// Create Pod metadata for `pod_name`.
pub fn new(pod_name: impl Into<String>, active: Option<PodActiveSegmentRef>) -> Self {
Self {
pod_name: pod_name.into(),
active,
spawned_children: Vec::new(),
}
}
}
/// Sync persistence backend for Pod metadata.
///
/// The key is the Pod name. Missing state is not an error: `read_by_name`
/// returns `Ok(None)` for Pods that have never persisted metadata or whose
/// metadata was deleted.
pub trait PodMetadataStore: Send + Sync {
/// Create or replace metadata for its `pod_name` key.
fn write(&self, metadata: &PodMetadata) -> Result<(), StoreError>;
/// Read metadata by Pod name. Returns `None` when no metadata exists.
fn read_by_name(&self, pod_name: &str) -> Result<Option<PodMetadata>, StoreError>;
/// Delete metadata by Pod name. Missing metadata is a successful no-op.
fn delete_by_name(&self, pod_name: &str) -> Result<(), StoreError>;
}
pub(crate) fn validate_pod_name(pod_name: &str) -> Result<(), StoreError> {
if pod_name.is_empty()
|| pod_name == "."
|| pod_name == ".."
|| pod_name.contains('/')
|| pod_name.contains('\0')
{
return Err(StoreError::InvalidPodName(pod_name.to_string()));
}
Ok(())
}

View File

@ -0,0 +1,515 @@
//! Free functions for segment persistence operations.
//!
//! These functions record and restore segment state without owning a Worker.
//! The caller (typically Pod) holds the Worker directly and calls these
//! functions after state-mutating operations.
use crate::logged_item::{LoggedItem, to_logged};
use crate::segment_log::{self, LogEntry, PodScopeSnapshot, SegmentOrigin};
use crate::store::{Store, StoreError};
use crate::system_item::SystemItem;
use crate::{SegmentId, SessionId};
use llm_worker::WorkerResult;
use llm_worker::llm_client::RequestConfig;
use llm_worker::llm_client::types::Item;
use protocol::Segment;
/// State snapshot for creating a SegmentStart entry.
pub struct SegmentStartState<'a> {
pub system_prompt: Option<&'a str>,
pub config: &'a RequestConfig,
pub history: &'a [Item],
}
/// Create a new session + initial segment, writing the initial
/// `SegmentStart` entry. Returns the freshly minted `(session_id, segment_id)`.
pub fn create_segment(
store: &impl Store,
state: SegmentStartState<'_>,
) -> Result<(SessionId, SegmentId), StoreError> {
let session_id = crate::new_session_id();
let segment_id = crate::new_segment_id();
create_segment_with_ids(store, session_id, segment_id, state)?;
Ok((session_id, segment_id))
}
/// Write a fresh `SegmentStart` entry using pre-generated IDs.
///
/// Used by callers that need to reserve `(session_id, segment_id)`
/// synchronously but defer the initial log append (e.g. Pod, which
/// resolves a templated system prompt only at first turn).
pub fn create_segment_with_ids(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
state: SegmentStartState<'_>,
) -> Result<(), StoreError> {
let entry = LogEntry::SegmentStart {
ts: segment_log::now_millis(),
session_id,
system_prompt: state.system_prompt.map(String::from),
config: state.config.clone(),
history: to_logged(state.history),
forked_from: None,
compacted_from: None,
};
store.append(session_id, segment_id, &entry)
}
/// Create a compacted segment from an existing one. Inherits the source's
/// `session_id` so the compacted lineage stays within the same Session.
///
/// Records `compacted_from` provenance linking back to the source segment
/// at the turn boundary captured by `source_turn_count` (the most recent
/// completed turn in the source).
pub fn create_compacted_segment(
store: &impl Store,
state: SegmentStartState<'_>,
source_session_id: SessionId,
source_segment_id: SegmentId,
source_turn_count: usize,
) -> Result<SegmentId, StoreError> {
let segment_id = crate::new_segment_id();
let entry = LogEntry::SegmentStart {
ts: segment_log::now_millis(),
session_id: source_session_id,
system_prompt: state.system_prompt.map(String::from),
config: state.config.clone(),
history: to_logged(state.history),
forked_from: None,
compacted_from: Some(SegmentOrigin {
segment_id: source_segment_id,
at_turn_index: source_turn_count,
}),
};
store.append(source_session_id, segment_id, &entry)?;
Ok(segment_id)
}
/// Restore segment state from a stored log.
///
/// Returns the reconstructed state. The caller is responsible for
/// applying it to a Worker.
pub fn restore(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
) -> Result<crate::segment_log::RestoredState, StoreError> {
let entries = store.read_all(session_id, segment_id)?;
Ok(segment_log::collect_state(&entries))
}
/// Restore segment state when only the segment ID is known. Uses
/// [`Store::lookup_session_of`] to resolve the parent Session.
///
/// Shim for legacy entry points (`pod-cli --session <UUID>` etc.) that
/// receive a Segment ID without a Session ID.
pub fn restore_by_segment(
store: &impl Store,
segment_id: SegmentId,
) -> Result<crate::segment_log::RestoredState, StoreError> {
let session_id = store
.lookup_session_of(segment_id)?
.ok_or(StoreError::NotFound(segment_id))?;
restore(store, session_id, segment_id)
}
/// Live auto-fork on concurrent-writer detection.
///
/// Checks whether the store's on-disk entry count still matches the
/// writer's own append tally. If they match, the writer still owns the
/// segment tail and nothing happens. If the store has grown behind the
/// writer's back, another process appended to the same segment — so we
/// branch into a fresh segment within the same Session.
///
/// # Marker form
///
/// Detection is by **tail entry-count comparison**, not by writing a
/// terminal marker into the source segment. The source segment is left
/// completely immutable — identical to the past-fork ([`fork_at`])
/// invariant. The fork relationship is instead recorded forward on the
/// *new* segment's `SegmentStart.forked_from`, so the lineage is still
/// reconstructable from the logs alone (read each segment's
/// `SegmentStart`; follow `forked_from` / `compacted_from` backward).
/// Listing a parent's children is a cheap `list_segments(session_id)`
/// scan filtered on `forked_from.segment_id`.
///
/// `at_turn_index` is the writer's current `turn_count`: the fork seeds
/// the new segment with the writer's in-memory history (which reflects
/// state up to that turn), so that is the divergence point relative to
/// the now-diverged source segment.
///
/// Updates `segment_id` and `entries_written` in place when a fork occurs.
pub fn ensure_head_or_fork(
store: &impl Store,
session_id: SessionId,
segment_id: &mut SegmentId,
entries_written: &mut usize,
at_turn_index: usize,
state: SegmentStartState<'_>,
) -> Result<(), StoreError> {
let store_count = store.read_entry_count(session_id, *segment_id)?;
if store_count == *entries_written {
return Ok(());
}
let source_segment_id = *segment_id;
let fork_id = crate::new_segment_id();
let entry = LogEntry::SegmentStart {
ts: segment_log::now_millis(),
session_id,
system_prompt: state.system_prompt.map(String::from),
config: state.config.clone(),
history: to_logged(state.history),
forked_from: Some(SegmentOrigin {
segment_id: source_segment_id,
at_turn_index,
}),
compacted_from: None,
};
store.create_segment(session_id, fork_id, &[entry])?;
*segment_id = fork_id;
*entries_written = 1;
Ok(())
}
/// Log a `UserInput` entry from the original typed `Vec<Segment>`.
///
/// Submit-time entry. Pod calls this at the head of a `Run` turn before
/// the worker pushes its flattened user message into history; replay
/// derives the worker `Item::user_message` from these segments via
/// [`Segment::flatten_to_text`].
pub fn save_user_input(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
segments: Vec<Segment>,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::UserInput {
ts: segment_log::now_millis(),
segments,
},
)
}
/// Log the history delta — new items added since the previous snapshot.
///
/// Classifies items into AssistantItem / ToolResult entries automatically
/// (one entry per item). User messages are skipped
/// because they are persisted upfront via [`save_user_input`] at submit
/// time; the worker pushes a flattened copy into its history that
/// arrives here in `new_items` and would otherwise produce a duplicate
/// `UserInput` entry.
pub fn save_delta(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
new_items: &[Item],
) -> Result<(), StoreError> {
if new_items.is_empty() {
return Ok(());
}
let ts = segment_log::now_millis();
for item in new_items {
if item.is_user_message() {
// Already persisted by save_user_input at submit time.
continue;
}
let entry = classify_history_item(item, ts);
append_entry(store, session_id, segment_id, entry)?;
}
Ok(())
}
/// Map one history item to its singular `LogEntry` form. Used by the
/// fallback `save_delta` path and the controller's worker-callback
/// classifier so write classification lives in one place.
pub fn classify_history_item(item: &Item, ts: u64) -> LogEntry {
if item.is_tool_result() {
LogEntry::ToolResult {
ts,
item: LoggedItem::from(item),
}
} else if item.is_assistant_message() || item.is_tool_call() || item.is_reasoning() {
LogEntry::AssistantItem {
ts,
item: LoggedItem::from(item),
}
} else {
// Defensive: anything else (future Item kinds) routes through
// AssistantItem rather than getting silently dropped.
LogEntry::AssistantItem {
ts,
item: LoggedItem::from(item),
}
}
}
/// Append a single typed system item as `LogEntry::SystemItem`. Helper
/// for the Pod-side interceptor commit path; mirrors the per-item
/// commit shape used for assistant / tool result entries.
pub fn append_system_item(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
item: SystemItem,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::SystemItem {
ts: segment_log::now_millis(),
item,
},
)
}
/// Log a TurnEnd entry.
pub fn save_turn_end(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
turn_count: usize,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::TurnEnd {
ts: segment_log::now_millis(),
turn_count,
},
)
}
/// Log a `RunCompleted` entry — `run()` / `resume()` returned `Ok(WorkerResult)`.
pub fn save_run_completed(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
result: WorkerResult,
interrupted: bool,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::RunCompleted {
ts: segment_log::now_millis(),
interrupted,
result,
},
)
}
/// Log a `RunErrored` entry — `run()` / `resume()` returned `Err(WorkerError)`.
///
/// `WorkerError` is not `Serialize`, so the caller passes a lossy
/// `to_string()` rendering as `message`.
pub fn save_run_errored(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
message: String,
interrupted: bool,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::RunErrored {
ts: segment_log::now_millis(),
interrupted,
message,
},
)
}
/// Log an `LlmUsage` entry — 1 LLM リクエスト分の Usage スナップショット。
///
/// `history_len` は送信時の `history.len()`。`input_total_tokens` は
/// その prefix をプロバイダが実測した占有量(プロンプト全長)で、
/// プロバイダ別の正規化Anthropic では `input + cache_read + cache_creation`)を
/// 済ませた値を渡す。
pub fn save_usage(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
history_len: usize,
input_total_tokens: u64,
cache_read_tokens: u64,
cache_write_tokens: u64,
output_tokens: u64,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::LlmUsage {
ts: segment_log::now_millis(),
history_len,
input_total_tokens,
cache_read_tokens,
cache_write_tokens,
output_tokens,
},
)
}
/// Log an `Extension` entry — domain-tagged opaque payload.
///
/// session-store treats `payload` as an unstructured `serde_json::Value`.
/// Each domain is responsible for serializing into and folding out of it.
/// Use `RestoredState.extensions` to read entries back at restore time.
pub fn save_extension(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
domain: impl Into<String>,
payload: serde_json::Value,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::Extension {
ts: segment_log::now_millis(),
domain: domain.into(),
payload,
},
)
}
/// Log the Pod's latest runtime scope snapshot.
pub fn save_pod_scope(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
snapshot: &PodScopeSnapshot,
) -> Result<(), StoreError> {
let payload = serde_json::to_value(snapshot)?;
save_extension(
store,
session_id,
segment_id,
segment_log::POD_SCOPE_EXTENSION_DOMAIN,
payload,
)
}
/// Log a `ConfigChanged` entry.
pub fn save_config_changed(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
config: &RequestConfig,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
segment_id,
LogEntry::ConfigChanged {
ts: segment_log::now_millis(),
config: config.clone(),
},
)
}
/// Fork the current state into a brand-new Session (no parent lineage).
///
/// Use this for "start a fresh conversation from this state" — the
/// returned segment does not share `session_id` with any prior segment.
/// In-Session forks (live auto-fork / past-turn fork) go through
/// [`fork_at`] or [`ensure_head_or_fork`] instead.
pub fn fork(
store: &impl Store,
state: SegmentStartState<'_>,
) -> Result<(SessionId, SegmentId), StoreError> {
let session_id = crate::new_session_id();
let fork_id = crate::new_segment_id();
let entry = LogEntry::SegmentStart {
ts: segment_log::now_millis(),
session_id,
system_prompt: state.system_prompt.map(String::from),
config: state.config.clone(),
history: to_logged(state.history),
forked_from: None,
compacted_from: None,
};
store.create_segment(session_id, fork_id, &[entry])?;
Ok((session_id, fork_id))
}
/// Fork from a turn boundary in a stored segment log, keeping the new
/// segment in the same Session as `source_id`.
///
/// `at_turn_index` is the `turn_count` of the most recent completed
/// `TurnEnd` in the source segment that the fork should branch from.
/// Replay collects state up to and including that `TurnEnd`; entries
/// after it are not carried into the new segment.
///
/// # Invariant: the source segment is never mutated
///
/// Past-fork only reads the source and seeds a brand-new segment. It
/// writes no marker back into the source — exactly like live auto-fork
/// ([`ensure_head_or_fork`]). This keeps nested past-forks simple: a
/// fork of a fork just reads its own source and branches again, with no
/// marker-position bookkeeping to reconcile across the chain.
pub fn fork_at(
store: &impl Store,
source_session_id: SessionId,
source_id: SegmentId,
at_turn_index: usize,
) -> Result<SegmentId, StoreError> {
let entries = store.read_all(source_session_id, source_id)?;
let cut = if at_turn_index == 0 {
// Branch directly after the SegmentStart (or whatever opens the
// segment), before any turn completes.
entries
.iter()
.position(|e| !matches!(e, LogEntry::SegmentStart { .. }))
.unwrap_or(entries.len())
} else {
entries
.iter()
.position(|e| matches!(e, LogEntry::TurnEnd { turn_count, .. } if *turn_count == at_turn_index))
.map(|i| i + 1)
.unwrap_or(entries.len())
};
let state = segment_log::collect_state(&entries[..cut]);
let fork_id = crate::new_segment_id();
let entry = LogEntry::SegmentStart {
ts: segment_log::now_millis(),
session_id: source_session_id,
system_prompt: state.system_prompt,
config: state.config,
history: to_logged(&state.history),
forked_from: Some(SegmentOrigin {
segment_id: source_id,
at_turn_index,
}),
compacted_from: None,
};
store.create_segment(source_session_id, fork_id, &[entry])?;
Ok(fork_id)
}
/// Append a single `LogEntry`.
///
/// Lower-level dual of the `save_*` convenience wrappers in this module.
/// Use when the caller already builds the typed entry itself (e.g. when
/// it needs the same value for an in-memory mirror + broadcast).
pub fn append_entry(
store: &impl Store,
session_id: SessionId,
segment_id: SegmentId,
entry: LogEntry,
) -> Result<(), StoreError> {
store.append(session_id, segment_id, &entry)
}

View File

@ -1,98 +1,28 @@
//! Session log types for append-only JSONL persistence. //! Segment log types for append-only JSONL persistence.
//! //!
//! Each [`LogEntry`] represents a single state transition in a session, //! Each [`LogEntry`] represents a single state transition within one
//! serialized as one line in a `.jsonl` file. Reading all entries and //! segment, serialized as one line in a `.jsonl` file. Reading all
//! collecting them via [`collect_state`] reconstructs the full [`Worker`] state. //! entries and collecting them via [`collect_state`] reconstructs the
//! full [`Worker`] state at that segment.
//! //!
//! Entries are chained via [`EntryHash`]: each [`HashedEntry`] records the hash //! The on-disk format is one `LogEntry` per line — entries are positionally
//! of the previous entry, forming a tamper-evident append-only chain. This //! ordered. Fork lineage references between segments use turn-number indices
//! enables safe fork detection when multiple writers share a session. //! (`SegmentOrigin.at_turn_index`) rather than per-entry hashes.
use llm_worker::llm_client::types::{Item, RequestConfig}; use llm_worker::llm_client::types::{Item, RequestConfig};
use llm_worker::{UsageRecord, WorkerResult}; use llm_worker::{UsageRecord, WorkerResult};
use protocol::{InvokeKind, ScopeRule, Segment}; use protocol::{InvokeKind, ScopeRule, Segment};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use crate::logged_item::LoggedItem; use crate::logged_item::LoggedItem;
use crate::system_item::SystemItem; use crate::system_item::SystemItem;
/// SHA-256 hash identifying a specific log entry in the chain. /// A single segment log entry, serialized as one JSONL line.
///
/// Computed as `sha256(prev_hash_bytes || canonical_json(entry))`.
/// Displayed and serialized as a lowercase hex string.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct EntryHash([u8; 32]);
impl EntryHash {
pub fn as_bytes(&self) -> &[u8; 32] {
&self.0
}
pub fn to_hex(&self) -> String {
hex::encode(self.0)
}
pub fn from_hex(s: &str) -> Result<Self, hex::FromHexError> {
let mut buf = [0u8; 32];
hex::decode_to_slice(s, &mut buf)?;
Ok(Self(buf))
}
}
impl std::fmt::Display for EntryHash {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.to_hex())
}
}
impl Serialize for EntryHash {
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
serializer.serialize_str(&self.to_hex())
}
}
impl<'de> Deserialize<'de> for EntryHash {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
let s = String::deserialize(deserializer)?;
Self::from_hex(&s).map_err(serde::de::Error::custom)
}
}
/// Compute the hash for a log entry given its predecessor's hash.
pub fn compute_hash(prev: Option<&EntryHash>, entry: &LogEntry) -> EntryHash {
let mut hasher = Sha256::new();
// Feed prev_hash bytes (32 zero bytes if None).
match prev {
Some(h) => hasher.update(h.as_bytes()),
None => hasher.update([0u8; 32]),
}
// Canonical JSON of the entry.
let json = serde_json::to_string(entry).expect("LogEntry serialization cannot fail");
hasher.update(json.as_bytes());
EntryHash(hasher.finalize().into())
}
/// A [`LogEntry`] with hash-chain metadata.
///
/// This is the unit persisted to JSONL — one line per `HashedEntry`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HashedEntry {
pub hash: EntryHash,
pub prev_hash: Option<EntryHash>,
#[serde(flatten)]
pub entry: LogEntry,
}
/// A single session log entry, serialized as one JSONL line.
/// ///
/// Variants correspond to specific mutation points in `Worker`: /// Variants correspond to specific mutation points in `Worker`:
/// - `SessionStart` — always the first entry; captures initial state /// - `SegmentStart` — always the first entry; captures initial state
/// - `Invoke` — IDLE → active marker (start of a new self-driving cycle) /// - `Invoke` — IDLE → active marker (start of a new self-driving cycle)
/// - `UserInput` / `AssistantItems` / `ToolResults` / `HookInjectedItems` — history appends /// - `UserInput` / `AssistantItem` / `ToolResult` / `SystemItem` — history appends
/// - `TurnEnd` — AgentTurn boundary marker; carries the post-increment /// - `TurnEnd` — AgentTurn boundary marker; carries the post-increment
/// `turn_count`. With retry unimplemented today this fires once per /// `turn_count`. With retry unimplemented today this fires once per
/// `run()`/`resume()` (current callers persist a single TurnEnd at /// `run()`/`resume()` (current callers persist a single TurnEnd at
@ -103,19 +33,25 @@ pub struct HashedEntry {
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")] #[serde(tag = "kind", rename_all = "snake_case")]
pub enum LogEntry { pub enum LogEntry {
/// Session start. Always the first entry in a log. /// Segment start. Always the first entry in a segment log.
/// For forked sessions, `history` contains the seed state from the parent. /// For forked segments, `history` contains the seed state from the parent.
SessionStart { SegmentStart {
ts: u64, ts: u64,
/// Session this segment belongs to. Compaction / fork inherits
/// the source segment's session_id; only fresh "new conversation"
/// segments mint a new session_id.
session_id: crate::SessionId,
system_prompt: Option<String>, system_prompt: Option<String>,
config: RequestConfig, config: RequestConfig,
history: Vec<LoggedItem>, history: Vec<LoggedItem>,
/// Origin: forked from another session at a specific entry. /// Origin: forked from a sibling segment at a specific turn boundary.
/// The referenced segment is guaranteed to share `session_id`.
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
forked_from: Option<SessionOrigin>, forked_from: Option<SegmentOrigin>,
/// Origin: compacted from another session at a specific entry. /// Origin: compacted from a sibling segment at a specific turn boundary.
/// The referenced segment is guaranteed to share `session_id`.
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
compacted_from: Option<SessionOrigin>, compacted_from: Option<SegmentOrigin>,
}, },
/// IDLE → active marker. Records the start of a new self-driving /// IDLE → active marker. Records the start of a new self-driving
@ -137,7 +73,7 @@ pub enum LogEntry {
/// User input accepted at submit time. Carries the original typed /// User input accepted at submit time. Carries the original typed
/// `Vec<Segment>` so clients can re-render typed atoms (paste chips, /// `Vec<Segment>` so clients can re-render typed atoms (paste chips,
/// file/knowledge refs, workflow invocations) on session restore. /// file/knowledge refs, workflow invocations) on segment restore.
/// Replay flattens these into a `Item::user_message` for the worker /// Replay flattens these into a `Item::user_message` for the worker
/// history; the worker layer never sees segments directly. /// history; the worker layer never sees segments directly.
UserInput { ts: u64, segments: Vec<Segment> }, UserInput { ts: u64, segments: Vec<Segment> },
@ -158,23 +94,6 @@ pub enum LogEntry {
/// dispatch on `kind` for typed rendering. /// dispatch on `kind` for typed rendering.
SystemItem { ts: u64, item: SystemItem }, SystemItem { ts: u64, item: SystemItem },
/// Legacy plural form: kept **read-only** so old session logs still
/// open. New writes always use the singular `AssistantItem`. Items
/// are flattened on replay.
AssistantItems { ts: u64, items: Vec<LoggedItem> },
/// Legacy plural form: kept **read-only**. New writes use the
/// singular `ToolResult`.
ToolResults { ts: u64, items: Vec<LoggedItem> },
/// Legacy plural form: kept **read-only**. New writes use the
/// singular `SystemItem`.
SystemItems { ts: u64, items: Vec<SystemItem> },
/// Legacy pre-`SystemItem*` form. Deserialize-only. Items are
/// flattened to `Item::system_message` on replay.
HookInjectedItems { ts: u64, items: Vec<LoggedItem> },
/// Turn boundary. Records the turn count after increment. /// Turn boundary. Records the turn count after increment.
TurnEnd { ts: u64, turn_count: usize }, TurnEnd { ts: u64, turn_count: usize },
@ -235,13 +154,16 @@ pub enum LogEntry {
}, },
} }
/// Provenance reference to a parent session. /// Provenance reference to a parent segment.
///
/// `at_turn_index` is the `turn_count` value of the most recent
/// `TurnEnd` entry preceding the split point in the source segment.
/// A value of `0` means the split happened before any turn completed
/// (e.g. immediately after `SegmentStart`).
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct SessionOrigin { pub struct SegmentOrigin {
/// Session ID of the source session. pub segment_id: crate::SegmentId,
pub session_id: crate::SessionId, pub at_turn_index: usize,
/// Hash of the entry in the source session at the point of fork/compact.
pub at_hash: EntryHash,
} }
/// Domain used by Pod to persist its latest effective runtime scope. /// Domain used by Pod to persist its latest effective runtime scope.
@ -257,13 +179,19 @@ pub struct PodScopeSnapshot {
/// State collected from log entries. /// State collected from log entries.
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct RestoredState { pub struct RestoredState {
/// Session the replayed segment belongs to. Sourced from the
/// `SegmentStart` entry; `None` only if the log was empty (in which
/// case `entries_count == 0`).
pub session_id: Option<crate::SessionId>,
pub system_prompt: Option<String>, pub system_prompt: Option<String>,
pub config: RequestConfig, pub config: RequestConfig,
pub history: Vec<Item>, pub history: Vec<Item>,
pub turn_count: usize, pub turn_count: usize,
pub last_run_interrupted: bool, pub last_run_interrupted: bool,
/// Hash of the last entry in the chain (None if empty). /// Number of entries replayed. `0` means the segment log was empty.
pub head_hash: Option<EntryHash>, /// Writers track their own append count via the same counter so
/// `ensure_head_or_fork` can compare it with the on-disk count.
pub entries_count: usize,
/// LLM リクエストごとの Usage スナップショット時系列。 /// LLM リクエストごとの Usage スナップショット時系列。
/// `LogEntry::LlmUsage` を replay して時系列順に積まれる。 /// `LogEntry::LlmUsage` を replay して時系列順に積まれる。
/// 任意位置のトークン数推定に使う。 /// 任意位置のトークン数推定に使う。
@ -272,42 +200,45 @@ pub struct RestoredState {
/// session-store は domain を不透明扱いし、各ドメインが自前で fold する。 /// session-store は domain を不透明扱いし、各ドメインが自前で fold する。
pub extensions: Vec<(String, serde_json::Value)>, pub extensions: Vec<(String, serde_json::Value)>,
/// Latest runtime scope snapshot persisted by the Pod. `None` means /// Latest runtime scope snapshot persisted by the Pod. `None` means
/// the session predates scope persistence or the payload was corrupt. /// the segment predates scope persistence or the payload was corrupt.
pub pod_scope: Option<PodScopeSnapshot>, pub pod_scope: Option<PodScopeSnapshot>,
/// User submissions in original typed form, in submit order. /// User submissions in original typed form, in submit order.
/// One entry per `LogEntry::UserInput`; the K-th entry corresponds to /// One entry per `LogEntry::UserInput`; the K-th entry corresponds to
/// the K-th `Item::user_message` derived during replay (modulo /// the K-th `Item::user_message` derived during replay (modulo
/// pre-compaction history seeded via `SessionStart.history`, whose /// pre-compaction history seeded via `SegmentStart.history`, whose
/// original segments are not preserved). Used by clients to re-render /// original segments are not preserved). Used by clients to re-render
/// typed atoms (paste chips, refs) on session restore. /// typed atoms (paste chips, refs) on segment restore.
pub user_segments: Vec<Vec<Segment>>, pub user_segments: Vec<Vec<Segment>>,
} }
/// Replay a sequence of hashed entries to reconstruct worker state. /// Replay a sequence of log entries to reconstruct worker state.
pub fn collect_state(entries: &[HashedEntry]) -> RestoredState { pub fn collect_state(entries: &[LogEntry]) -> RestoredState {
let mut state = RestoredState { let mut state = RestoredState {
session_id: None,
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: Vec::new(), history: Vec::new(),
turn_count: 0, turn_count: 0,
last_run_interrupted: false, last_run_interrupted: false,
head_hash: None, entries_count: 0,
usage_history: Vec::new(), usage_history: Vec::new(),
extensions: Vec::new(), extensions: Vec::new(),
pod_scope: None, pod_scope: None,
user_segments: Vec::new(), user_segments: Vec::new(),
}; };
for hashed in entries { for entry in entries {
state.head_hash = Some(hashed.hash.clone()); state.entries_count += 1;
match &hashed.entry { match entry {
LogEntry::SessionStart { LogEntry::SegmentStart {
session_id,
system_prompt, system_prompt,
config, config,
history, history,
.. ..
} => { } => {
state.session_id = Some(*session_id);
state.system_prompt = system_prompt.clone(); state.system_prompt = system_prompt.clone();
state.config = config.clone(); state.config = config.clone();
state.history = history.iter().cloned().map(Item::from).collect(); state.history = history.iter().cloned().map(Item::from).collect();
@ -331,20 +262,6 @@ pub fn collect_state(entries: &[HashedEntry]) -> RestoredState {
LogEntry::SystemItem { item, .. } => { LogEntry::SystemItem { item, .. } => {
state.history.push(item.to_history_item()); state.history.push(item.to_history_item());
} }
LogEntry::AssistantItems { items, .. } => {
state.history.extend(items.iter().cloned().map(Item::from));
}
LogEntry::ToolResults { items, .. } => {
state.history.extend(items.iter().cloned().map(Item::from));
}
LogEntry::SystemItems { items, .. } => {
state
.history
.extend(items.iter().map(|si| si.to_history_item()));
}
LogEntry::HookInjectedItems { items, .. } => {
state.history.extend(items.iter().cloned().map(Item::from));
}
LogEntry::TurnEnd { turn_count, .. } => { LogEntry::TurnEnd { turn_count, .. } => {
state.turn_count = *turn_count; state.turn_count = *turn_count;
} }
@ -382,7 +299,7 @@ pub fn collect_state(entries: &[HashedEntry]) -> RestoredState {
Err(err) => { Err(err) => {
tracing::warn!( tracing::warn!(
error = %err, error = %err,
"discarding malformed pod.scope snapshot from session log" "discarding malformed pod.scope snapshot from segment log"
); );
} }
} }
@ -403,26 +320,6 @@ pub fn now_millis() -> u64 {
.as_millis() as u64 .as_millis() as u64
} }
/// Build a hash chain from plain `LogEntry` values.
///
/// Useful for tests and for seeding new sessions from a list of entries.
pub fn build_chain(entries: &[LogEntry]) -> Vec<HashedEntry> {
let mut chain = Vec::with_capacity(entries.len());
let mut prev: Option<EntryHash> = None;
for entry in entries {
let hash = compute_hash(prev.as_ref(), entry);
chain.push(HashedEntry {
hash: hash.clone(),
prev_hash: prev,
entry: entry.clone(),
});
prev = Some(hash);
}
chain
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@ -432,31 +329,32 @@ mod tests {
let state = collect_state(&[]); let state = collect_state(&[]);
assert!(state.history.is_empty()); assert!(state.history.is_empty());
assert_eq!(state.turn_count, 0); assert_eq!(state.turn_count, 0);
assert!(state.head_hash.is_none()); assert_eq!(state.entries_count, 0);
} }
#[test] #[test]
fn replay_session_start_sets_initial_state() { fn replay_segment_start_sets_initial_state() {
let entries = build_chain(&[LogEntry::SessionStart { let state = collect_state(&[LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: uuid::Uuid::nil(),
system_prompt: Some("You are helpful.".into()), system_prompt: Some("You are helpful.".into()),
config: RequestConfig::default().with_max_tokens(1024), config: RequestConfig::default().with_max_tokens(1024),
history: vec![Item::user_message("seed").into()], history: vec![Item::user_message("seed").into()],
forked_from: None, forked_from: None,
compacted_from: None, compacted_from: None,
}]); }]);
let state = collect_state(&entries);
assert_eq!(state.system_prompt.as_deref(), Some("You are helpful.")); assert_eq!(state.system_prompt.as_deref(), Some("You are helpful."));
assert_eq!(state.config.max_tokens, Some(1024)); assert_eq!(state.config.max_tokens, Some(1024));
assert_eq!(state.history.len(), 1); assert_eq!(state.history.len(), 1);
assert!(state.head_hash.is_some()); assert_eq!(state.entries_count, 1);
} }
#[test] #[test]
fn replay_full_turn() { fn replay_full_turn() {
let entries = build_chain(&[ let state = collect_state(&[
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -467,9 +365,9 @@ mod tests {
ts: 2000, ts: 2000,
segments: vec![Segment::text("Hello")], segments: vec![Segment::text("Hello")],
}, },
LogEntry::AssistantItems { LogEntry::AssistantItem {
ts: 3000, ts: 3000,
items: vec![Item::assistant_message("Hi!").into()], item: Item::assistant_message("Hi!").into(),
}, },
LogEntry::TurnEnd { LogEntry::TurnEnd {
ts: 3100, ts: 3100,
@ -481,7 +379,6 @@ mod tests {
result: WorkerResult::Finished, result: WorkerResult::Finished,
}, },
]); ]);
let state = collect_state(&entries);
assert_eq!(state.history.len(), 2); assert_eq!(state.history.len(), 2);
assert_eq!(state.turn_count, 1); assert_eq!(state.turn_count, 1);
assert!(!state.last_run_interrupted); assert!(!state.last_run_interrupted);
@ -489,9 +386,10 @@ mod tests {
#[test] #[test]
fn replay_with_tool_calls() { fn replay_with_tool_calls() {
let entries = build_chain(&[ let state = collect_state(&[
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -502,24 +400,23 @@ mod tests {
ts: 2000, ts: 2000,
segments: vec![Segment::text("Check weather")], segments: vec![Segment::text("Check weather")],
}, },
LogEntry::AssistantItems { LogEntry::AssistantItem {
ts: 3000, ts: 3000,
items: vec![Item::tool_call("call_1", "get_weather", r#"{"city":"Tokyo"}"#).into()], item: Item::tool_call("call_1", "get_weather", r#"{"city":"Tokyo"}"#).into(),
}, },
LogEntry::ToolResults { LogEntry::ToolResult {
ts: 3500, ts: 3500,
items: vec![Item::tool_result("call_1", "Sunny, 25C").into()], item: Item::tool_result("call_1", "Sunny, 25C").into(),
}, },
LogEntry::AssistantItems { LogEntry::AssistantItem {
ts: 4000, ts: 4000,
items: vec![Item::assistant_message("It's sunny in Tokyo!").into()], item: Item::assistant_message("It's sunny in Tokyo!").into(),
}, },
LogEntry::TurnEnd { LogEntry::TurnEnd {
ts: 4100, ts: 4100,
turn_count: 1, turn_count: 1,
}, },
]); ]);
let state = collect_state(&entries);
assert_eq!(state.history.len(), 4); assert_eq!(state.history.len(), 4);
assert!(state.history[1].is_tool_call()); assert!(state.history[1].is_tool_call());
assert!(state.history[2].is_tool_result()); assert!(state.history[2].is_tool_result());
@ -527,9 +424,10 @@ mod tests {
#[test] #[test]
fn replay_config_changed() { fn replay_config_changed() {
let entries = build_chain(&[ let state = collect_state(&[
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -541,52 +439,15 @@ mod tests {
config: RequestConfig::default().with_temperature(0.5), config: RequestConfig::default().with_temperature(0.5),
}, },
]); ]);
let state = collect_state(&entries);
assert_eq!(state.config.temperature, Some(0.5)); assert_eq!(state.config.temperature, Some(0.5));
} }
#[test]
fn hash_chain_is_deterministic() {
let raw = vec![
LogEntry::SessionStart {
ts: 1000,
system_prompt: None,
config: RequestConfig::default(),
history: vec![],
forked_from: None,
compacted_from: None,
},
LogEntry::UserInput {
ts: 2000,
segments: vec![Segment::text("Hello")],
},
];
let chain_a = build_chain(&raw);
let chain_b = build_chain(&raw);
assert_eq!(chain_a[0].hash, chain_b[0].hash);
assert_eq!(chain_a[1].hash, chain_b[1].hash);
}
#[test]
fn different_content_produces_different_hash() {
let entry_a = LogEntry::UserInput {
ts: 1000,
segments: vec![Segment::text("Hello")],
};
let entry_b = LogEntry::UserInput {
ts: 1000,
segments: vec![Segment::text("World")],
};
let hash_a = compute_hash(None, &entry_a);
let hash_b = compute_hash(None, &entry_b);
assert_ne!(hash_a, hash_b);
}
#[test] #[test]
fn replay_llm_usage_appends_to_usage_history() { fn replay_llm_usage_appends_to_usage_history() {
let entries = build_chain(&[ let state = collect_state(&[
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -605,9 +466,9 @@ mod tests {
cache_write_tokens: 0, cache_write_tokens: 0,
output_tokens: 10, output_tokens: 10,
}, },
LogEntry::AssistantItems { LogEntry::AssistantItem {
ts: 2200, ts: 2200,
items: vec![Item::assistant_message("yo").into()], item: Item::assistant_message("yo").into(),
}, },
LogEntry::LlmUsage { LogEntry::LlmUsage {
ts: 3100, ts: 3100,
@ -618,7 +479,6 @@ mod tests {
output_tokens: 5, output_tokens: 5,
}, },
]); ]);
let state = collect_state(&entries);
// history は LlmUsage で変化しない // history は LlmUsage で変化しない
assert_eq!(state.history.len(), 2); assert_eq!(state.history.len(), 2);
// usage_history は時系列順 // usage_history は時系列順
@ -631,10 +491,10 @@ mod tests {
#[test] #[test]
fn replay_without_llm_usage_keeps_usage_history_empty() { fn replay_without_llm_usage_keeps_usage_history_empty() {
// 既存ログ互換: LlmUsage entry が無くても collect_state は壊れない let state = collect_state(&[
let entries = build_chain(&[ LogEntry::SegmentStart {
LogEntry::SessionStart {
ts: 1000, ts: 1000,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -646,7 +506,6 @@ mod tests {
segments: vec![Segment::text("hi")], segments: vec![Segment::text("hi")],
}, },
]); ]);
let state = collect_state(&entries);
assert!(state.usage_history.is_empty()); assert!(state.usage_history.is_empty());
} }
@ -704,9 +563,10 @@ mod tests {
#[test] #[test]
fn replay_invoke_marker_does_not_mutate_state() { fn replay_invoke_marker_does_not_mutate_state() {
let entries = build_chain(&[ let state = collect_state(&[
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 0, ts: 0,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -730,16 +590,16 @@ mod tests {
trigger: InvokeKind::Notify, trigger: InvokeKind::Notify,
}, },
]); ]);
let state = collect_state(&entries);
assert_eq!(state.history.len(), 1); assert_eq!(state.history.len(), 1);
assert_eq!(state.turn_count, 1); assert_eq!(state.turn_count, 1);
} }
#[test] #[test]
fn replay_extension_collects_domain_payload_pairs() { fn replay_extension_collects_domain_payload_pairs() {
let entries = build_chain(&[ let state = collect_state(&[
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -762,7 +622,6 @@ mod tests {
payload: serde_json::json!({ "x": 1 }), payload: serde_json::json!({ "x": 1 }),
}, },
]); ]);
let state = collect_state(&entries);
// 順序保持で全件積まれる。fold は呼び出し側の責務。 // 順序保持で全件積まれる。fold は呼び出し側の責務。
assert_eq!(state.extensions.len(), 3); assert_eq!(state.extensions.len(), 3);
assert_eq!(state.extensions[0].0, "memory.extract"); assert_eq!(state.extensions[0].0, "memory.extract");
@ -794,22 +653,6 @@ mod tests {
} }
} }
#[test]
fn hash_hex_round_trip() {
let entry = LogEntry::SessionStart {
ts: 1000,
system_prompt: None,
config: RequestConfig::default(),
history: vec![],
forked_from: None,
compacted_from: None,
};
let hash = compute_hash(None, &entry);
let hex = hash.to_hex();
let parsed = EntryHash::from_hex(&hex).unwrap();
assert_eq!(hash, parsed);
}
/// Mixed segments survive a JSON round-trip through `LogEntry::UserInput`, /// Mixed segments survive a JSON round-trip through `LogEntry::UserInput`,
/// and `collect_state` derives `Item::user_message` from the flattened /// and `collect_state` derives `Item::user_message` from the flattened
/// text while preserving the original segments separately. This covers /// text while preserving the original segments separately. This covers
@ -834,12 +677,13 @@ mod tests {
ts: 4242, ts: 4242,
segments: segments.clone(), segments: segments.clone(),
}; };
// Hash + JSON round-trip preserves the variant byte-for-byte. // JSON round-trip preserves the variant byte-for-byte.
let json = serde_json::to_string(&entry).unwrap(); let json = serde_json::to_string(&entry).unwrap();
let parsed: LogEntry = serde_json::from_str(&json).unwrap(); let parsed: LogEntry = serde_json::from_str(&json).unwrap();
let entries = build_chain(&[ let state = collect_state(&[
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 1, ts: 1,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -848,7 +692,6 @@ mod tests {
}, },
parsed, parsed,
]); ]);
let state = collect_state(&entries);
// Worker history gets a flattened user_message item. // Worker history gets a flattened user_message item.
assert_eq!(state.history.len(), 1); assert_eq!(state.history.len(), 1);
match &state.history[0] { match &state.history[0] {

View File

@ -1,482 +0,0 @@
//! Free functions for session persistence operations.
//!
//! These functions record and restore session state without owning a Worker.
//! The caller (typically Pod) holds the Worker directly and calls these
//! functions after state-mutating operations.
use crate::SessionId;
use crate::logged_item::{LoggedItem, to_logged};
use crate::session_log::{self, EntryHash, HashedEntry, LogEntry, PodScopeSnapshot, SessionOrigin};
use crate::store::{Store, StoreError};
use crate::system_item::SystemItem;
use llm_worker::WorkerResult;
use llm_worker::llm_client::RequestConfig;
use llm_worker::llm_client::types::Item;
use protocol::Segment;
/// State snapshot for creating a SessionStart entry.
pub struct SessionStartState<'a> {
pub system_prompt: Option<&'a str>,
pub config: &'a RequestConfig,
pub history: &'a [Item],
}
/// Create a new session, writing the initial `SessionStart` entry.
///
/// Returns the new session ID and head hash.
pub fn create_session(
store: &impl Store,
state: SessionStartState<'_>,
) -> Result<(SessionId, EntryHash), StoreError> {
let session_id = crate::new_session_id();
let hash = create_session_with_id(store, session_id, state)?;
Ok((session_id, hash))
}
/// Write a fresh `SessionStart` entry using a pre-generated session ID.
///
/// Used by callers that need to reserve a session ID synchronously but
/// defer the initial log append (e.g. Pod, which resolves a templated
/// system prompt only at first turn). Returns the resulting head hash.
pub fn create_session_with_id(
store: &impl Store,
session_id: SessionId,
state: SessionStartState<'_>,
) -> Result<EntryHash, StoreError> {
let entry = LogEntry::SessionStart {
ts: session_log::now_millis(),
system_prompt: state.system_prompt.map(String::from),
config: state.config.clone(),
history: to_logged(state.history),
forked_from: None,
compacted_from: None,
};
let hash = session_log::compute_hash(None, &entry);
let hashed_entry = HashedEntry {
hash: hash.clone(),
prev_hash: None,
entry,
};
store.append(session_id, &hashed_entry)?;
Ok(hash)
}
/// Create a compacted session from an existing one.
///
/// Records `compacted_from` provenance linking back to the source session.
/// Returns the new session ID and head hash.
pub fn create_compacted_session(
store: &impl Store,
state: SessionStartState<'_>,
source_session_id: SessionId,
source_head_hash: EntryHash,
) -> Result<(SessionId, EntryHash), StoreError> {
let session_id = crate::new_session_id();
let entry = LogEntry::SessionStart {
ts: session_log::now_millis(),
system_prompt: state.system_prompt.map(String::from),
config: state.config.clone(),
history: to_logged(state.history),
forked_from: None,
compacted_from: Some(SessionOrigin {
session_id: source_session_id,
at_hash: source_head_hash,
}),
};
let hash = session_log::compute_hash(None, &entry);
let hashed_entry = HashedEntry {
hash: hash.clone(),
prev_hash: None,
entry,
};
store.append(session_id, &hashed_entry)?;
Ok((session_id, hash))
}
/// Restore session state from a stored log.
///
/// Returns the reconstructed state. The caller is responsible for
/// applying it to a Worker.
pub fn restore(
store: &impl Store,
session_id: SessionId,
) -> Result<crate::session_log::RestoredState, StoreError> {
let entries = store.read_all(session_id)?;
Ok(session_log::collect_state(&entries))
}
/// Check if the store's head still matches the expected head hash.
/// If not, auto-fork into a new session.
///
/// Updates `session_id` and `head_hash` in place when a fork occurs.
pub fn ensure_head_or_fork(
store: &impl Store,
session_id: &mut SessionId,
head_hash: &mut Option<EntryHash>,
state: SessionStartState<'_>,
) -> Result<(), StoreError> {
let store_head = store.read_head_hash(*session_id)?;
if store_head == *head_hash {
return Ok(());
}
let fork_id = crate::new_session_id();
let entry = LogEntry::SessionStart {
ts: session_log::now_millis(),
system_prompt: state.system_prompt.map(String::from),
config: state.config.clone(),
history: to_logged(state.history),
forked_from: None,
compacted_from: None,
};
let hash = session_log::compute_hash(None, &entry);
let hashed_entry = HashedEntry {
hash: hash.clone(),
prev_hash: None,
entry,
};
store.create_session(fork_id, &[hashed_entry])?;
*session_id = fork_id;
*head_hash = Some(hash);
Ok(())
}
/// Log a `UserInput` entry from the original typed `Vec<Segment>`.
///
/// Submit-time entry. Pod calls this at the head of a `Run` turn before
/// the worker pushes its flattened user message into history; replay
/// derives the worker `Item::user_message` from these segments via
/// [`Segment::flatten_to_text`].
pub fn save_user_input(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
segments: Vec<Segment>,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
head_hash,
LogEntry::UserInput {
ts: session_log::now_millis(),
segments,
},
)
}
/// Log the history delta — new items added since the previous snapshot.
///
/// Classifies items into AssistantItem / ToolResult / HookInjectedItems
/// entries automatically (one entry per item). User messages are skipped
/// because they are persisted upfront via [`save_user_input`] at submit
/// time; the worker pushes a flattened copy into its history that
/// arrives here in `new_items` and would otherwise produce a duplicate
/// `UserInput` entry.
pub fn save_delta(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
new_items: &[Item],
) -> Result<(), StoreError> {
if new_items.is_empty() {
return Ok(());
}
let ts = session_log::now_millis();
for item in new_items {
if item.is_user_message() {
// Already persisted by save_user_input at submit time.
continue;
}
let entry = classify_history_item(item, ts);
append_entry(store, session_id, head_hash, entry)?;
}
Ok(())
}
/// Map one history item to its singular `LogEntry` form. Used by the
/// fallback `save_delta` path and the controller's worker-callback
/// classifier so write classification lives in one place.
pub fn classify_history_item(item: &Item, ts: u64) -> LogEntry {
if item.is_tool_result() {
LogEntry::ToolResult {
ts,
item: LoggedItem::from(item),
}
} else if item.is_assistant_message() || item.is_tool_call() || item.is_reasoning() {
LogEntry::AssistantItem {
ts,
item: LoggedItem::from(item),
}
} else {
// Defensive: anything else (future Item kinds) routes through
// AssistantItem rather than getting silently dropped.
LogEntry::AssistantItem {
ts,
item: LoggedItem::from(item),
}
}
}
/// Append a single typed system item as `LogEntry::SystemItem`. Helper
/// for the Pod-side interceptor commit path; mirrors the per-item
/// commit shape used for assistant / tool result entries.
pub fn append_system_item(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
item: SystemItem,
) -> Result<EntryHash, StoreError> {
append_entry_with_hash(
store,
session_id,
head_hash,
LogEntry::SystemItem {
ts: session_log::now_millis(),
item,
},
)
}
/// Log a TurnEnd entry.
pub fn save_turn_end(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
turn_count: usize,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
head_hash,
LogEntry::TurnEnd {
ts: session_log::now_millis(),
turn_count,
},
)
}
/// Log a `RunCompleted` entry — `run()` / `resume()` returned `Ok(WorkerResult)`.
pub fn save_run_completed(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
result: WorkerResult,
interrupted: bool,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
head_hash,
LogEntry::RunCompleted {
ts: session_log::now_millis(),
interrupted,
result,
},
)
}
/// Log a `RunErrored` entry — `run()` / `resume()` returned `Err(WorkerError)`.
///
/// `WorkerError` is not `Serialize`, so the caller passes a lossy
/// `to_string()` rendering as `message`.
pub fn save_run_errored(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
message: String,
interrupted: bool,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
head_hash,
LogEntry::RunErrored {
ts: session_log::now_millis(),
interrupted,
message,
},
)
}
/// Log an `LlmUsage` entry — 1 LLM リクエスト分の Usage スナップショット。
///
/// `history_len` は送信時の `history.len()`。`input_total_tokens` は
/// その prefix をプロバイダが実測した占有量(プロンプト全長)で、
/// プロバイダ別の正規化Anthropic では `input + cache_read + cache_creation`)を
/// 済ませた値を渡す。
pub fn save_usage(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
history_len: usize,
input_total_tokens: u64,
cache_read_tokens: u64,
cache_write_tokens: u64,
output_tokens: u64,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
head_hash,
LogEntry::LlmUsage {
ts: session_log::now_millis(),
history_len,
input_total_tokens,
cache_read_tokens,
cache_write_tokens,
output_tokens,
},
)
}
/// Log an `Extension` entry — domain-tagged opaque payload.
///
/// session-store treats `payload` as an unstructured `serde_json::Value`.
/// Each domain is responsible for serializing into and folding out of it.
/// Use `RestoredState.extensions` to read entries back at restore time.
pub fn save_extension(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
domain: impl Into<String>,
payload: serde_json::Value,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
head_hash,
LogEntry::Extension {
ts: session_log::now_millis(),
domain: domain.into(),
payload,
},
)
}
/// Log the Pod's latest runtime scope snapshot.
pub fn save_pod_scope(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
snapshot: &PodScopeSnapshot,
) -> Result<(), StoreError> {
let payload = serde_json::to_value(snapshot)?;
save_extension(
store,
session_id,
head_hash,
session_log::POD_SCOPE_EXTENSION_DOMAIN,
payload,
)
}
/// Log a `ConfigChanged` entry.
pub fn save_config_changed(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
config: &RequestConfig,
) -> Result<(), StoreError> {
append_entry(
store,
session_id,
head_hash,
LogEntry::ConfigChanged {
ts: session_log::now_millis(),
config: config.clone(),
},
)
}
/// Fork the current state into a new session.
pub fn fork(store: &impl Store, state: SessionStartState<'_>) -> Result<SessionId, StoreError> {
let fork_id = crate::new_session_id();
let entry = LogEntry::SessionStart {
ts: session_log::now_millis(),
system_prompt: state.system_prompt.map(String::from),
config: state.config.clone(),
history: to_logged(state.history),
forked_from: None,
compacted_from: None,
};
let hash = session_log::compute_hash(None, &entry);
let hashed_entry = HashedEntry {
hash,
prev_hash: None,
entry,
};
store.create_session(fork_id, &[hashed_entry])?;
Ok(fork_id)
}
/// Fork from an arbitrary point in a stored session's log.
pub fn fork_at(
store: &impl Store,
source_id: SessionId,
at_hash: &EntryHash,
) -> Result<SessionId, StoreError> {
let entries = store.read_all(source_id)?;
let cut = entries
.iter()
.position(|e| &e.hash == at_hash)
.map(|i| i + 1)
.unwrap_or(entries.len());
let state = session_log::collect_state(&entries[..cut]);
let fork_id = crate::new_session_id();
let entry = LogEntry::SessionStart {
ts: session_log::now_millis(),
system_prompt: state.system_prompt,
config: state.config,
history: to_logged(&state.history),
forked_from: Some(session_log::SessionOrigin {
session_id: source_id,
at_hash: at_hash.clone(),
}),
compacted_from: None,
};
let hash = session_log::compute_hash(None, &entry);
let hashed_entry = HashedEntry {
hash,
prev_hash: None,
entry,
};
store.create_session(fork_id, &[hashed_entry])?;
Ok(fork_id)
}
/// Append a single `LogEntry`, chaining the hash and updating `head_hash`.
///
/// Lower-level dual of the `save_*` convenience wrappers in this module.
/// Use when the caller already builds the typed entry itself (e.g. when
/// it needs the same value for an in-memory mirror + broadcast).
pub fn append_entry(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
entry: LogEntry,
) -> Result<(), StoreError> {
append_entry_with_hash(store, session_id, head_hash, entry)?;
Ok(())
}
/// Same as [`append_entry`] but returns the freshly computed entry hash.
///
/// Used by paths that need the hash for downstream broadcast or mirror
/// updates (e.g. the Pod's `SessionLogSink`).
pub fn append_entry_with_hash(
store: &impl Store,
session_id: SessionId,
head_hash: &mut Option<EntryHash>,
entry: LogEntry,
) -> Result<EntryHash, StoreError> {
let hash = session_log::compute_hash(head_hash.as_ref(), &entry);
let hashed_entry = HashedEntry {
hash: hash.clone(),
prev_hash: head_hash.clone(),
entry,
};
store.append(session_id, &hashed_entry)?;
*head_hash = Some(hash.clone());
Ok(hash)
}

View File

@ -1,18 +1,19 @@
//! Persistence backend abstraction. //! Persistence backend abstraction.
//! //!
//! [`Store`] defines the sync interface for reading and writing session logs. //! [`Store`] defines the sync interface for reading and writing segment logs
//! Implementations handle the physical storage (filesystem, database, etc.). //! within a [`Session`](crate::SessionId). Implementations handle the
//! physical storage (filesystem, database, etc.).
//! //!
//! Sync (rather than async) is intentional: a session log append is a single //! Sync (rather than async) is intentional: a segment log append is a single
//! `< 1 KiB` line on local fs and completes well below a millisecond. Going //! `< 1 KiB` line on local fs and completes well below a millisecond. Going
//! through `tokio::fs` would force every caller — including `Worker`'s sync //! through `tokio::fs` would force every caller — including `Worker`'s sync
//! `on_history_append` callback — to bridge sync → async via a channel + //! `on_history_append` callback — to bridge sync → async via a channel +
//! drain task. Keeping the store sync lets the worker callback, Pod commit //! drain task. Keeping the store sync lets the worker callback, Pod commit
//! paths, and `PodInterceptor` all share one direct `append_entry` call. //! paths, and `PodInterceptor` all share one direct `append_entry` call.
use crate::SessionId;
use crate::event_trace::TraceEntry; use crate::event_trace::TraceEntry;
use crate::session_log::{EntryHash, HashedEntry}; use crate::segment_log::LogEntry;
use crate::{SegmentId, SessionId};
/// Errors from the persistence store. /// Errors from the persistence store.
#[derive(Debug, thiserror::Error)] #[derive(Debug, thiserror::Error)]
@ -23,38 +24,80 @@ pub enum StoreError {
#[error("serialization error: {0}")] #[error("serialization error: {0}")]
Serde(#[from] serde_json::Error), Serde(#[from] serde_json::Error),
#[error("session not found: {0}")] #[error("segment not found: {0}")]
NotFound(SessionId), NotFound(SegmentId),
#[error("log corrupted at line {line}: {message}")] #[error("log corrupted at line {line}: {message}")]
Corrupt { line: usize, message: String }, Corrupt { line: usize, message: String },
#[error("invalid pod name: {0}")]
InvalidPodName(String),
} }
/// Sync persistence backend for session logs. /// Sync persistence backend for segment logs.
/// ///
/// All methods take `&self` — implementations should use interior mutability /// All methods take `&self` — implementations should use interior mutability
/// (e.g., append-mode file handles) when needed. /// (e.g., append-mode file handles) when needed. Most read/write methods
/// take `(SessionId, SegmentId)` so segments can be physically grouped
/// per Session on disk (or per session_id in a DB).
pub trait Store: Send + Sync { pub trait Store: Send + Sync {
/// Append a single hashed entry to the session log. /// Append a single log entry to the segment log.
fn append(&self, id: SessionId, entry: &HashedEntry) -> Result<(), StoreError>; ///
/// One line per call. The kernel orders concurrent `O_APPEND` writes
/// for lines < `PIPE_BUF`, so user-space serialization is unnecessary.
fn append(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &LogEntry,
) -> Result<(), StoreError>;
/// Read all hashed entries for a session, in order. /// Read all log entries for a segment, in order.
fn read_all(&self, id: SessionId) -> Result<Vec<HashedEntry>, StoreError>; fn read_all(
&self,
session_id: SessionId,
segment_id: SegmentId,
) -> Result<Vec<LogEntry>, StoreError>;
/// List all session IDs, most recent first. /// List all session IDs, most recent first.
fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError>; fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError>;
/// Create a new session with initial entries. /// List segment IDs belonging to `session_id`, most recent first.
fn create_session(&self, id: SessionId, entries: &[HashedEntry]) -> Result<(), StoreError>; fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError>;
/// Check if a session exists. /// Look up which session a given segment belongs to. Returns `None`
fn exists(&self, id: SessionId) -> Result<bool, StoreError>; /// when the segment is not known to any session. Implementations
/// may scan storage; intended for shim entry points that receive a
/// segment ID without its session ID (e.g. legacy `--session <UUID>`).
fn lookup_session_of(&self, segment_id: SegmentId) -> Result<Option<SessionId>, StoreError>;
/// Read the hash of the last entry in a session (the head). /// Create a new segment within `session_id`, with initial entries.
fn create_segment(
&self,
session_id: SessionId,
segment_id: SegmentId,
entries: &[LogEntry],
) -> Result<(), StoreError>;
/// Check if a segment exists.
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError>;
/// Count entries currently stored for a segment.
/// ///
/// Returns `None` if the session is empty. /// Used by `ensure_head_or_fork` to detect concurrent writers:
fn read_head_hash(&self, id: SessionId) -> Result<Option<EntryHash>, StoreError>; /// if the on-disk count exceeds the writer's own append tally,
/// another process has extended the log.
fn read_entry_count(
&self,
session_id: SessionId,
segment_id: SegmentId,
) -> Result<usize, StoreError>;
/// Append a trace entry to the debug event trace file. /// Append a trace entry to the debug event trace file.
fn append_trace(&self, id: SessionId, entry: &TraceEntry) -> Result<(), StoreError>; fn append_trace(
&self,
session_id: SessionId,
segment_id: SegmentId,
entry: &TraceEntry,
) -> Result<(), StoreError>;
} }

View File

@ -29,7 +29,7 @@ use serde::{Deserialize, Serialize};
/// path / knowledge slug / workflow slug / etc.), plus a pre-rendered /// path / knowledge slug / workflow slug / etc.), plus a pre-rendered
/// `body` (where applicable) that is the exact `role:system` text the /// `body` (where applicable) that is the exact `role:system` text the
/// LLM actually saw at commit time. `body` is denormalised so that /// LLM actually saw at commit time. `body` is denormalised so that
/// session log replay reconstructs worker history byte-identical to /// segment log replay reconstructs worker history byte-identical to
/// what was on the wire — even when prompt overrides (e.g. custom /// what was on the wire — even when prompt overrides (e.g. custom
/// `notify_wrapper` template) re-shape the live rendering on a later /// `notify_wrapper` template) re-shape the live rendering on a later
/// resume. /// resume.

View File

@ -1,18 +1,33 @@
use llm_worker::WorkerResult; use llm_worker::WorkerResult;
use llm_worker::llm_client::types::{Item, RequestConfig}; use llm_worker::llm_client::types::{Item, RequestConfig};
use session_store::{ use session_store::{
FsStore, LogEntry, Store, TraceEntry, build_chain, collect_state, new_session_id, FsStore, LogEntry, PodActiveSegmentRef, PodMetadata, PodMetadataStore, Store, TraceEntry,
collect_state, new_segment_id, new_session_id,
}; };
fn nil_session_start(ts: u64, session_id: uuid::Uuid) -> LogEntry {
LogEntry::SegmentStart {
ts,
session_id,
system_prompt: None,
config: RequestConfig::default(),
history: vec![],
forked_from: None,
compacted_from: None,
}
}
#[test] #[test]
fn round_trip_write_and_read() { fn round_trip_write_and_read() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap(); let store = FsStore::new(dir.path()).unwrap();
let id = new_session_id(); let sid = new_session_id();
let segid = new_segment_id();
let raw = vec![ let entries = vec![
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: sid,
system_prompt: Some("You are helpful.".into()), system_prompt: Some("You are helpful.".into()),
config: RequestConfig::default().with_max_tokens(1024), config: RequestConfig::default().with_max_tokens(1024),
history: vec![], history: vec![],
@ -37,41 +52,34 @@ fn round_trip_write_and_read() {
result: WorkerResult::Finished, result: WorkerResult::Finished,
}, },
]; ];
let entries = build_chain(&raw);
// Write entries one by one
for entry in &entries { for entry in &entries {
store.append(id, entry).unwrap(); store.append(sid, segid, entry).unwrap();
} }
// Read back let read_back = store.read_all(sid, segid).unwrap();
let read_back = store.read_all(id).unwrap();
assert_eq!(read_back.len(), entries.len()); assert_eq!(read_back.len(), entries.len());
// Verify hashes survived round-trip
for (orig, read) in entries.iter().zip(read_back.iter()) {
assert_eq!(orig.hash, read.hash);
assert_eq!(orig.prev_hash, read.prev_hash);
}
// Replay and verify state
let state = collect_state(&read_back); let state = collect_state(&read_back);
assert_eq!(state.session_id, Some(sid));
assert_eq!(state.system_prompt.as_deref(), Some("You are helpful.")); assert_eq!(state.system_prompt.as_deref(), Some("You are helpful."));
assert_eq!(state.config.max_tokens, Some(1024)); assert_eq!(state.config.max_tokens, Some(1024));
assert_eq!(state.history.len(), 2); assert_eq!(state.history.len(), 2);
assert_eq!(state.turn_count, 1); assert_eq!(state.turn_count, 1);
assert!(!state.last_run_interrupted); assert!(!state.last_run_interrupted);
assert!(state.head_hash.is_some()); assert_eq!(state.entries_count, entries.len());
} }
#[test] #[test]
fn create_session_writes_all_entries() { fn create_segment_writes_all_entries() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap(); let store = FsStore::new(dir.path()).unwrap();
let id = new_session_id(); let sid = new_session_id();
let segid = new_segment_id();
let entries = build_chain(&[LogEntry::SessionStart { let entries = [LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: sid,
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![ history: vec![
@ -80,80 +88,75 @@ fn create_session_writes_all_entries() {
], ],
forked_from: None, forked_from: None,
compacted_from: None, compacted_from: None,
}]); }];
store.create_session(id, &entries).unwrap(); store.create_segment(sid, segid, &entries).unwrap();
let read_back = store.read_all(id).unwrap(); let read_back = store.read_all(sid, segid).unwrap();
assert_eq!(read_back.len(), 1); assert_eq!(read_back.len(), 1);
let state = collect_state(&read_back); let state = collect_state(&read_back);
assert_eq!(state.history.len(), 2); assert_eq!(state.history.len(), 2);
assert_eq!(state.session_id, Some(sid));
} }
#[test] #[test]
fn list_sessions_returns_newest_first() { fn list_sessions_and_segments() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap(); let store = FsStore::new(dir.path()).unwrap();
let id1 = new_session_id(); let sid_a = new_session_id();
// Small delay to ensure different UUID v7 timestamps
std::thread::sleep(std::time::Duration::from_millis(2)); std::thread::sleep(std::time::Duration::from_millis(2));
let id2 = new_session_id(); let sid_b = new_session_id();
let entries1 = build_chain(&[LogEntry::SessionStart { let seg_a1 = new_segment_id();
ts: 1000, let seg_a2 = new_segment_id();
system_prompt: None, let seg_b1 = new_segment_id();
config: RequestConfig::default(),
history: vec![],
forked_from: None,
compacted_from: None,
}]);
let entries2 = build_chain(&[LogEntry::SessionStart {
ts: 1001,
system_prompt: None,
config: RequestConfig::default(),
history: vec![],
forked_from: None,
compacted_from: None,
}]);
store.append(id1, &entries1[0]).unwrap(); store
store.append(id2, &entries2[0]).unwrap(); .append(sid_a, seg_a1, &nil_session_start(1, sid_a))
.unwrap();
store
.append(sid_a, seg_a2, &nil_session_start(2, sid_a))
.unwrap();
store
.append(sid_b, seg_b1, &nil_session_start(3, sid_b))
.unwrap();
let sessions = store.list_sessions().unwrap(); let sessions = store.list_sessions().unwrap();
assert_eq!(sessions.len(), 2); assert_eq!(sessions, vec![sid_b, sid_a]); // newest first
assert_eq!(sessions[0], id2); // newest first
assert_eq!(sessions[1], id1); let segs_a = store.list_segments(sid_a).unwrap();
assert!(segs_a.contains(&seg_a1) && segs_a.contains(&seg_a2));
assert_eq!(segs_a.len(), 2);
let segs_b = store.list_segments(sid_b).unwrap();
assert_eq!(segs_b, vec![seg_b1]);
} }
#[test] #[test]
fn exists_returns_correct_state() { fn exists_returns_correct_state() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap(); let store = FsStore::new(dir.path()).unwrap();
let id = new_session_id(); let sid = new_session_id();
let segid = new_segment_id();
assert!(!store.exists(id).unwrap()); assert!(!store.exists(sid, segid).unwrap());
let entries = build_chain(&[LogEntry::SessionStart { store
ts: 1000, .append(sid, segid, &nil_session_start(1000, sid))
system_prompt: None, .unwrap();
config: RequestConfig::default(),
history: vec![],
forked_from: None,
compacted_from: None,
}]);
store.append(id, &entries[0]).unwrap();
assert!(store.exists(id).unwrap()); assert!(store.exists(sid, segid).unwrap());
} }
#[test] #[test]
fn not_found_error_for_missing_session() { fn not_found_error_for_missing_segment() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap(); let store = FsStore::new(dir.path()).unwrap();
let id = new_session_id(); let sid = new_session_id();
let segid = new_segment_id();
let result = store.read_all(id); let result = store.read_all(sid, segid);
assert!(result.is_err()); assert!(result.is_err());
} }
@ -161,20 +164,13 @@ fn not_found_error_for_missing_session() {
fn trace_entries_in_separate_file() { fn trace_entries_in_separate_file() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap(); let store = FsStore::new(dir.path()).unwrap();
let id = new_session_id(); let sid = new_session_id();
let segid = new_segment_id();
// Write a log entry store
let entries = build_chain(&[LogEntry::SessionStart { .append(sid, segid, &nil_session_start(1000, sid))
ts: 1000, .unwrap();
system_prompt: None,
config: RequestConfig::default(),
history: vec![],
forked_from: None,
compacted_from: None,
}]);
store.append(id, &entries[0]).unwrap();
// Write a trace entry
let trace = TraceEntry { let trace = TraceEntry {
ts: 1500, ts: 1500,
turn: 0, turn: 0,
@ -182,26 +178,31 @@ fn trace_entries_in_separate_file() {
llm_worker::llm_client::event::PingEvent { timestamp: None }, llm_worker::llm_client::event::PingEvent { timestamp: None },
), ),
}; };
store.append_trace(id, &trace).unwrap(); store.append_trace(sid, segid, &trace).unwrap();
// Log should have 1 entry, unaffected by trace // Log should have 1 entry, unaffected by trace
let log = store.read_all(id).unwrap(); let log = store.read_all(sid, segid).unwrap();
assert_eq!(log.len(), 1); assert_eq!(log.len(), 1);
// Trace file should exist separately // Trace file should exist separately
let trace_path = dir.path().join(format!("{id}.trace.jsonl")); let trace_path = dir
.path()
.join(sid.to_string())
.join(format!("{segid}.trace.jsonl"));
assert!(trace_path.exists()); assert!(trace_path.exists());
} }
#[test] #[test]
fn read_head_hash_returns_last_entry_hash() { fn read_entry_count_matches_append_tally() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap(); let store = FsStore::new(dir.path()).unwrap();
let id = new_session_id(); let sid = new_session_id();
let segid = new_segment_id();
let entries = build_chain(&[ let entries = [
LogEntry::SessionStart { LogEntry::SegmentStart {
ts: 1000, ts: 1000,
session_id: sid,
system_prompt: None, system_prompt: None,
config: RequestConfig::default(), config: RequestConfig::default(),
history: vec![], history: vec![],
@ -212,12 +213,63 @@ fn read_head_hash_returns_last_entry_hash() {
ts: 2000, ts: 2000,
segments: vec![protocol::Segment::text("Hello")], segments: vec![protocol::Segment::text("Hello")],
}, },
]); ];
for entry in &entries { for entry in &entries {
store.append(id, entry).unwrap(); store.append(sid, segid, entry).unwrap();
} }
let head = store.read_head_hash(id).unwrap(); assert_eq!(store.read_entry_count(sid, segid).unwrap(), entries.len());
assert_eq!(head.as_ref(), Some(&entries[1].hash)); }
#[test]
fn lookup_session_of_finds_owning_session() {
let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let sid = new_session_id();
let segid = new_segment_id();
assert_eq!(store.lookup_session_of(segid).unwrap(), None);
store
.append(sid, segid, &nil_session_start(1, sid))
.unwrap();
assert_eq!(store.lookup_session_of(segid).unwrap(), Some(sid));
}
#[test]
fn pod_metadata_minimal_crud() {
let dir = tempfile::tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let pod_name = "worker-a";
let sid = new_session_id();
let segid = new_segment_id();
assert_eq!(store.read_by_name(pod_name).unwrap(), None);
let pending = PodMetadata::new(pod_name, Some(PodActiveSegmentRef::pending_segment(sid)));
store.write(&pending).unwrap();
assert_eq!(store.read_by_name(pod_name).unwrap(), Some(pending.clone()));
assert!(
dir.path()
.join("pods")
.join(pod_name)
.join("metadata.json")
.exists(),
"Pod metadata must live under <data_dir>/pods/<pod_name>/"
);
let resolved = PodMetadata::new(
pod_name,
Some(PodActiveSegmentRef::active_segment(sid, segid)),
);
store.write(&resolved).unwrap();
assert_eq!(store.read_by_name(pod_name).unwrap(), Some(resolved));
store.delete_by_name(pod_name).unwrap();
assert_eq!(store.read_by_name(pod_name).unwrap(), None);
// Delete is idempotent for missing metadata.
store.delete_by_name(pod_name).unwrap();
} }

View File

@ -9,7 +9,7 @@ use llm_worker::interceptor::{Interceptor, TurnEndAction};
use llm_worker::llm_client::event::{Event, ResponseStatus, StatusEvent}; use llm_worker::llm_client::event::{Event, ResponseStatus, StatusEvent};
use llm_worker::llm_client::types::{Item, RequestConfig}; use llm_worker::llm_client::types::{Item, RequestConfig};
use llm_worker::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput}; use llm_worker::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use session_store::{EntryHash, FsStore, LogEntry, SessionStartState, Store, collect_state}; use session_store::{FsStore, LogEntry, SegmentStartState, Store, collect_state};
// ============================================================================= // =============================================================================
// Helpers // Helpers
@ -96,7 +96,7 @@ async fn run_and_persist(
worker: Worker<MockLlmClient>, worker: Worker<MockLlmClient>,
store: &FsStore, store: &FsStore,
session_id: session_store::SessionId, session_id: session_store::SessionId,
head_hash: &mut Option<EntryHash>, segment_id: session_store::SegmentId,
input: &str, input: &str,
) -> (Worker<MockLlmClient>, llm_worker::WorkerResult) { ) -> (Worker<MockLlmClient>, llm_worker::WorkerResult) {
// Mirror Pod's run-entry contract: log the user input as segments // Mirror Pod's run-entry contract: log the user input as segments
@ -105,10 +105,9 @@ async fn run_and_persist(
session_store::save_user_input( session_store::save_user_input(
store, store,
session_id, session_id,
head_hash, segment_id,
vec![protocol::Segment::text(input)], vec![protocol::Segment::text(input)],
) )
.unwrap(); .unwrap();
let history_before = worker.history().len(); let history_before = worker.history().len();
@ -118,34 +117,28 @@ async fn run_and_persist(
let worker = locked.unlock(); let worker = locked.unlock();
let new_items = &worker.history()[history_before..]; let new_items = &worker.history()[history_before..];
session_store::save_delta(store, session_id, head_hash, new_items) session_store::save_delta(store, session_id, segment_id, new_items).unwrap();
session_store::save_turn_end(store, session_id, segment_id, worker.turn_count()).unwrap();
.unwrap();
session_store::save_turn_end(store, session_id, head_hash, worker.turn_count())
.unwrap();
match &result { match &result {
Ok(r) => { Ok(r) => {
session_store::save_run_completed( session_store::save_run_completed(
store, store,
session_id, session_id,
head_hash, segment_id,
r.clone(), r.clone(),
worker.last_run_interrupted(), worker.last_run_interrupted(),
) )
.unwrap(); .unwrap();
} }
Err(e) => { Err(e) => {
session_store::save_run_errored( session_store::save_run_errored(
store, store,
session_id, session_id,
head_hash, segment_id,
e.to_string(), e.to_string(),
worker.last_run_interrupted(), worker.last_run_interrupted(),
) )
.unwrap(); .unwrap();
} }
} }
@ -164,37 +157,35 @@ async fn session_run_logs_entries() {
let client = MockLlmClient::new(simple_text_events()); let client = MockLlmClient::new(simple_text_events());
let worker = Worker::new(client); let worker = Worker::new(client);
let (sid, head_hash) = session_store::create_session( let (sid, segid) = session_store::create_segment(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker.get_system_prompt(), system_prompt: worker.get_system_prompt(),
config: worker.request_config(), config: worker.request_config(),
history: worker.history(), history: worker.history(),
}, },
) )
.unwrap(); .unwrap();
let mut head_hash = Some(head_hash); let (worker, _) = run_and_persist(worker, &store, sid, segid, "Hi").await;
let (worker, _) = run_and_persist(worker, &store, sid, &mut head_hash, "Hi").await;
let _ = &worker; let _ = &worker;
let entries = store.read_all(sid).unwrap(); let entries = store.read_all(sid, segid).unwrap();
// SessionStart, UserInput, AssistantItems, TurnEnd, RunCompleted (at minimum) // SegmentStart, UserInput, AssistantItem, TurnEnd, RunCompleted (at minimum)
assert!( assert!(
entries.len() >= 4, entries.len() >= 4,
"expected at least 4 entries, got {}", "expected at least 4 entries, got {}",
entries.len() entries.len()
); );
// First entry is SessionStart // First entry is SegmentStart
assert!(matches!(&entries[0].entry, LogEntry::SessionStart { .. })); assert!(matches!(&entries[0], LogEntry::SegmentStart { .. }));
// Has a RunCompleted with Finished // Has a RunCompleted with Finished
let has_finished = entries.iter().any(|e| { let has_finished = entries.iter().any(|e| {
matches!( matches!(
&e.entry, e,
LogEntry::RunCompleted { LogEntry::RunCompleted {
result: llm_worker::WorkerResult::Finished, result: llm_worker::WorkerResult::Finished,
.. ..
@ -202,17 +193,6 @@ async fn session_run_logs_entries() {
) )
}); });
assert!(has_finished, "should have a Finished outcome"); assert!(has_finished, "should have a Finished outcome");
// Verify hash chain integrity
assert!(entries[0].prev_hash.is_none());
for i in 1..entries.len() {
assert_eq!(
entries[i].prev_hash.as_ref(),
Some(&entries[i - 1].hash),
"hash chain broken at entry {}",
i
);
}
} }
#[tokio::test] #[tokio::test]
@ -222,30 +202,36 @@ async fn session_restore_round_trip() {
let mut worker = Worker::new(client); let mut worker = Worker::new(client);
worker.set_system_prompt("You are helpful."); worker.set_system_prompt("You are helpful.");
let (sid, head_hash) = session_store::create_session( let (sid, segid) = session_store::create_segment(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker.get_system_prompt(), system_prompt: worker.get_system_prompt(),
config: worker.request_config(), config: worker.request_config(),
history: worker.history(), history: worker.history(),
}, },
) )
.unwrap(); .unwrap();
let mut head_hash = Some(head_hash);
let (worker, _) = run_and_persist(worker, &store, sid, &mut head_hash, "Hi").await; let (worker, _) = run_and_persist(worker, &store, sid, segid, "Hi").await;
let original_history_len = worker.history().len(); let original_history_len = worker.history().len();
let original_turn_count = worker.turn_count(); let original_turn_count = worker.turn_count();
// Restore // Restore
let state = session_store::restore(&store, sid).unwrap(); let state = session_store::restore(&store, sid, segid).unwrap();
assert_eq!(state.session_id, Some(sid));
assert_eq!(state.history.len(), original_history_len); assert_eq!(state.history.len(), original_history_len);
assert_eq!(state.turn_count, original_turn_count); assert_eq!(state.turn_count, original_turn_count);
assert_eq!(state.system_prompt.as_deref(), Some("You are helpful.")); assert_eq!(state.system_prompt.as_deref(), Some("You are helpful."));
assert_eq!(state.head_hash, head_hash); assert_eq!(
state.entries_count,
store.read_entry_count(sid, segid).unwrap()
);
// Shim by segment ID alone.
let by_segment = session_store::restore_by_segment(&store, segid).unwrap();
assert_eq!(by_segment.session_id, Some(sid));
} }
#[tokio::test] #[tokio::test]
@ -255,31 +241,28 @@ async fn session_run_with_tool_call() {
let mut worker = Worker::new(client); let mut worker = Worker::new(client);
worker.register_tool(weather_tool_definition()); worker.register_tool(weather_tool_definition());
let (sid, head_hash) = session_store::create_session( let (sid, segid) = session_store::create_segment(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker.get_system_prompt(), system_prompt: worker.get_system_prompt(),
config: worker.request_config(), config: worker.request_config(),
history: worker.history(), history: worker.history(),
}, },
) )
.unwrap(); .unwrap();
let mut head_hash = Some(head_hash);
let (_worker, _) = let (_worker, _) = run_and_persist(worker, &store, sid, segid, "What's the weather?").await;
run_and_persist(worker, &store, sid, &mut head_hash, "What's the weather?").await;
let entries = store.read_all(sid).unwrap(); let entries = store.read_all(sid, segid).unwrap();
let has_tool_results = entries let has_tool_results = entries
.iter() .iter()
.any(|e| matches!(&e.entry, LogEntry::ToolResult { .. })); .any(|e| matches!(e, LogEntry::ToolResult { .. }));
assert!(has_tool_results, "should have ToolResult entry"); assert!(has_tool_results, "should have ToolResult entry");
let has_assistant = entries let has_assistant = entries
.iter() .iter()
.any(|e| matches!(&e.entry, LogEntry::AssistantItem { .. })); .any(|e| matches!(e, LogEntry::AssistantItem { .. }));
assert!(has_assistant, "should have AssistantItem entry"); assert!(has_assistant, "should have AssistantItem entry");
} }
@ -293,26 +276,24 @@ async fn session_resume_after_pause() {
worker.register_tool(weather_tool_definition()); worker.register_tool(weather_tool_definition());
worker.set_interceptor(PausePolicy); worker.set_interceptor(PausePolicy);
let (sid, head_hash) = session_store::create_session( let (sid, segid) = session_store::create_segment(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker.get_system_prompt(), system_prompt: worker.get_system_prompt(),
config: worker.request_config(), config: worker.request_config(),
history: worker.history(), history: worker.history(),
}, },
) )
.unwrap(); .unwrap();
let mut head_hash = Some(head_hash);
let (_worker, result) = run_and_persist(worker, &store, sid, &mut head_hash, "Weather?").await; let (_worker, result) = run_and_persist(worker, &store, sid, segid, "Weather?").await;
assert!(matches!(result, llm_worker::WorkerResult::Paused)); assert!(matches!(result, llm_worker::WorkerResult::Paused));
// Check RunCompleted is Paused // Check RunCompleted is Paused
let entries = store.read_all(sid).unwrap(); let entries = store.read_all(sid, segid).unwrap();
let has_paused = entries.iter().any(|e| { let has_paused = entries.iter().any(|e| {
matches!( matches!(
&e.entry, e,
LogEntry::RunCompleted { LogEntry::RunCompleted {
result: llm_worker::WorkerResult::Paused, result: llm_worker::WorkerResult::Paused,
.. ..
@ -322,93 +303,95 @@ async fn session_resume_after_pause() {
assert!(has_paused, "should have Paused outcome"); assert!(has_paused, "should have Paused outcome");
// Restore state and verify // Restore state and verify
let state = session_store::restore(&store, sid).unwrap(); let state = session_store::restore(&store, sid, segid).unwrap();
assert!(state.last_run_interrupted); assert!(state.last_run_interrupted);
} }
#[tokio::test] #[tokio::test]
async fn session_fork_preserves_state() { async fn session_fork_creates_new_session() {
let (_dir, store) = make_store(); let (_dir, store) = make_store();
let client = MockLlmClient::new(simple_text_events()); let client = MockLlmClient::new(simple_text_events());
let mut worker = Worker::new(client); let mut worker = Worker::new(client);
worker.set_system_prompt("System prompt"); worker.set_system_prompt("System prompt");
let (sid, head_hash) = session_store::create_session( let (sid, segid) = session_store::create_segment(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker.get_system_prompt(), system_prompt: worker.get_system_prompt(),
config: worker.request_config(), config: worker.request_config(),
history: worker.history(), history: worker.history(),
}, },
) )
.unwrap(); .unwrap();
let mut head_hash = Some(head_hash);
let (worker, _) = run_and_persist(worker, &store, sid, &mut head_hash, "Hello").await; let (worker, _) = run_and_persist(worker, &store, sid, segid, "Hello").await;
let original_history_len = worker.history().len(); let original_history_len = worker.history().len();
let fork_id = session_store::fork( let (fork_sid, fork_segid) = session_store::fork(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker.get_system_prompt(), system_prompt: worker.get_system_prompt(),
config: worker.request_config(), config: worker.request_config(),
history: worker.history(), history: worker.history(),
}, },
) )
.unwrap(); .unwrap();
assert_ne!(fork_sid, sid, "`fork` mints a fresh Session");
// Fork should have a SessionStart with the current history // Fork should have a SegmentStart with the current history
let fork_entries = store.read_all(fork_id).unwrap(); let fork_entries = store.read_all(fork_sid, fork_segid).unwrap();
assert_eq!(fork_entries.len(), 1); assert_eq!(fork_entries.len(), 1);
assert!(matches!( assert!(matches!(&fork_entries[0], LogEntry::SegmentStart { .. }));
&fork_entries[0].entry,
LogEntry::SessionStart { .. }
));
let fork_state = collect_state(&fork_entries); let fork_state = collect_state(&fork_entries);
assert_eq!(fork_state.session_id, Some(fork_sid));
assert_eq!(fork_state.history.len(), original_history_len); assert_eq!(fork_state.history.len(), original_history_len);
assert_eq!(fork_state.system_prompt.as_deref(), Some("System prompt")); assert_eq!(fork_state.system_prompt.as_deref(), Some("System prompt"));
} }
#[tokio::test] #[tokio::test]
async fn session_fork_at_truncates() { async fn session_fork_at_truncates_within_session() {
let (_dir, store) = make_store(); let (_dir, store) = make_store();
let client = MockLlmClient::new(simple_text_events()); let client = MockLlmClient::new(simple_text_events());
let worker = Worker::new(client); let worker = Worker::new(client);
let (sid, head_hash) = session_store::create_session( let (sid, segid) = session_store::create_segment(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker.get_system_prompt(), system_prompt: worker.get_system_prompt(),
config: worker.request_config(), config: worker.request_config(),
history: worker.history(), history: worker.history(),
}, },
) )
.unwrap(); .unwrap();
let mut head_hash = Some(head_hash);
let (_worker, _) = run_and_persist(worker, &store, sid, &mut head_hash, "Hello").await; let (worker, _) = run_and_persist(worker, &store, sid, segid, "Hello").await;
let all_entries = store.read_all(sid).unwrap(); let all_entries = store.read_all(sid, segid).unwrap();
assert!(all_entries.len() > 2); assert!(all_entries.len() > 2);
// Fork at the hash of the 2nd entry (SessionStart + UserInput) // Fork at turn 1 (one completed turn). Stays in same Session.
let at_hash = &all_entries[1].hash; let fork_segid = session_store::fork_at(&store, sid, segid, worker.turn_count()).unwrap();
let fork_id = session_store::fork_at(&store, sid, at_hash).unwrap();
let fork_entries = store.read_all(fork_id).unwrap(); let fork_entries = store.read_all(sid, fork_segid).unwrap();
assert_eq!(fork_entries.len(), 1); // Just the new SessionStart assert_eq!(fork_entries.len(), 1); // Just the new SegmentStart
let fork_state = collect_state(&fork_entries); let fork_state = collect_state(&fork_entries);
// Should have the state from replaying only the first 2 entries assert_eq!(fork_state.session_id, Some(sid), "fork_at inherits Session");
let original_truncated_state = collect_state(&all_entries[..2]);
assert_eq!( // History at fork point should match history right after the TurnEnd in
fork_state.history.len(), // the source segment.
original_truncated_state.history.len() let turn_end_pos = all_entries
); .iter()
.position(|e| matches!(e, LogEntry::TurnEnd { turn_count, .. } if *turn_count == worker.turn_count()))
.expect("source segment has the matching TurnEnd");
let source_state_at_fork = collect_state(&all_entries[..=turn_end_pos]);
assert_eq!(fork_state.history.len(), source_state_at_fork.history.len());
// list_segments should show both source and fork in the same Session.
let segs = store.list_segments(sid).unwrap();
assert!(segs.contains(&segid));
assert!(segs.contains(&fork_segid));
} }
#[tokio::test] #[tokio::test]
@ -417,29 +400,25 @@ async fn session_config_changed_logged() {
let client = MockLlmClient::new(vec![]); let client = MockLlmClient::new(vec![]);
let mut worker = Worker::new(client); let mut worker = Worker::new(client);
let (sid, head_hash) = session_store::create_session( let (sid, segid) = session_store::create_segment(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker.get_system_prompt(), system_prompt: worker.get_system_prompt(),
config: worker.request_config(), config: worker.request_config(),
history: worker.history(), history: worker.history(),
}, },
) )
.unwrap(); .unwrap();
let mut head_hash = Some(head_hash);
// Modify config and log it // Modify config and log it
let new_config = RequestConfig::default().with_temperature(0.7); let new_config = RequestConfig::default().with_temperature(0.7);
worker.set_request_config(new_config.clone()); worker.set_request_config(new_config.clone());
session_store::save_config_changed(&store, sid, &mut head_hash, &new_config) session_store::save_config_changed(&store, sid, segid, &new_config).unwrap();
.unwrap();
let entries = store.read_all(sid).unwrap(); let entries = store.read_all(sid, segid).unwrap();
let has_config_changed = entries.iter().any(|e| { let has_config_changed = entries.iter().any(|e| {
matches!( matches!(
&e.entry, e,
LogEntry::ConfigChanged { config, .. } if config.temperature == Some(0.7) LogEntry::ConfigChanged { config, .. } if config.temperature == Some(0.7)
) )
}); });
@ -450,62 +429,140 @@ async fn session_config_changed_logged() {
async fn session_auto_forks_on_conflict() { async fn session_auto_forks_on_conflict() {
let (_dir, store) = make_store(); let (_dir, store) = make_store();
// Create a session // Create a segment
let client_a = MockLlmClient::new(simple_text_events()); let client_a = MockLlmClient::new(simple_text_events());
let worker_a = Worker::new(client_a); let worker_a = Worker::new(client_a);
let (original_sid, head_hash) = session_store::create_session( let (sid, original_segid) = session_store::create_segment(
&store, &store,
SessionStartState { SegmentStartState {
system_prompt: worker_a.get_system_prompt(), system_prompt: worker_a.get_system_prompt(),
config: worker_a.request_config(), config: worker_a.request_config(),
history: worker_a.history(), history: worker_a.history(),
}, },
) )
.unwrap(); .unwrap();
let mut session_id = original_sid; let mut segment_id = original_segid;
let mut head_hash = Some(head_hash); // Writer tracked: just the SegmentStart we wrote.
let mut entries_written: usize = 1;
// Simulate another Pod writing to the same session behind our back // Simulate another Pod writing to the same segment behind our back.
let extra_entry = LogEntry::UserInput { let extra_entry = LogEntry::UserInput {
ts: 9999, ts: 9999,
segments: vec![protocol::Segment::text("Interloper")], segments: vec![protocol::Segment::text("Interloper")],
}; };
let current_head = store.read_head_hash(original_sid).unwrap(); store.append(sid, original_segid, &extra_entry).unwrap();
let hash = session_store::compute_hash(current_head.as_ref(), &extra_entry);
let hashed = session_store::HashedEntry {
hash,
prev_hash: current_head,
entry: extra_entry,
};
store.append(original_sid, &hashed).unwrap();
// Now head_hash is stale — ensure_head_or_fork should auto-fork // Now the on-disk count exceeds our tally — ensure_head_or_fork should auto-fork.
session_store::ensure_head_or_fork( session_store::ensure_head_or_fork(
&store, &store,
&mut session_id, sid,
&mut head_hash, &mut segment_id,
SessionStartState { &mut entries_written,
/* at_turn_index */ 0,
SegmentStartState {
system_prompt: worker_a.get_system_prompt(), system_prompt: worker_a.get_system_prompt(),
config: worker_a.request_config(), config: worker_a.request_config(),
history: worker_a.history(), history: worker_a.history(),
}, },
) )
.unwrap(); .unwrap();
// session_id should now be different // segment_id should now be different but live in the same Session.
assert_ne!(session_id, original_sid); assert_ne!(segment_id, original_segid);
// The fork session should exist and have entries // The fork segment should exist and have entries
let fork_entries = store.read_all(session_id).unwrap(); let fork_entries = store.read_all(sid, segment_id).unwrap();
assert!(!fork_entries.is_empty()); assert!(!fork_entries.is_empty());
let fork_state = collect_state(&fork_entries);
assert_eq!(
fork_state.session_id,
Some(sid),
"auto-fork inherits Session"
);
// Original session should still have the interloper entry // The new segment records its lineage forward via forked_from; the
let original_entries = store.read_all(original_sid).unwrap(); // source segment is left immutable (no terminal marker written back).
match &fork_entries[0] {
LogEntry::SegmentStart {
forked_from: Some(origin),
..
} => {
assert_eq!(origin.segment_id, original_segid);
assert_eq!(origin.at_turn_index, 0);
}
other => panic!("expected SegmentStart with forked_from, got {other:?}"),
}
// Original segment should still have the interloper entry and NO
// terminal fork marker — it is byte-for-byte unchanged.
let original_entries = store.read_all(sid, original_segid).unwrap();
assert_eq!(
original_entries.len(),
2,
"source segment holds only SegmentStart + interloper UserInput"
);
let has_interloper = original_entries let has_interloper = original_entries
.iter() .iter()
.any(|e| matches!(&e.entry, LogEntry::UserInput { .. })); .any(|e| matches!(e, LogEntry::UserInput { .. }));
assert!(has_interloper); assert!(has_interloper);
} }
/// Nested past-fork: forking a segment that is itself a fork must not
/// require touching any ancestor. Each `fork_at` only reads its direct
/// source and seeds a new segment, so a chain of forks composes cleanly.
#[tokio::test]
async fn nested_past_fork_leaves_ancestors_immutable() {
let (_dir, store) = make_store();
let client = MockLlmClient::new(simple_text_events());
let worker = Worker::new(client);
let (sid, root_segid) = session_store::create_segment(
&store,
SegmentStartState {
system_prompt: worker.get_system_prompt(),
config: worker.request_config(),
history: worker.history(),
},
)
.unwrap();
let (worker, _) = run_and_persist(worker, &store, sid, root_segid, "Hello").await;
let root_before = store.read_all(sid, root_segid).unwrap();
// First past-fork at the completed turn.
let fork1 = session_store::fork_at(&store, sid, root_segid, worker.turn_count()).unwrap();
// Fork the fork (turn 0 = right after its SegmentStart seed).
let fork2 = session_store::fork_at(&store, sid, fork1, 0).unwrap();
// All three are distinct, all in the same Session.
assert_ne!(fork1, root_segid);
assert_ne!(fork2, fork1);
for seg in [root_segid, fork1, fork2] {
assert_eq!(
collect_state(&store.read_all(sid, seg).unwrap()).session_id,
Some(sid)
);
}
// The root and fork1 are untouched by forking their descendants.
assert_eq!(
store.read_all(sid, root_segid).unwrap().len(),
root_before.len()
);
let fork1_entries = store.read_all(sid, fork1).unwrap();
assert_eq!(
fork1_entries.len(),
1,
"fork1 is just its SegmentStart seed"
);
// fork2's lineage points at fork1, not the root.
match &store.read_all(sid, fork2).unwrap()[0] {
LogEntry::SegmentStart {
forked_from: Some(origin),
..
} => assert_eq!(origin.segment_id, fork1),
other => panic!("expected SegmentStart with forked_from, got {other:?}"),
}
}

View File

@ -4,12 +4,12 @@
//! `llm-worker` `Tool` infrastructure. Filesystem access is mediated by //! `llm-worker` `Tool` infrastructure. Filesystem access is mediated by
//! two orthogonal concerns: //! two orthogonal concerns:
//! //!
//! - [`ScopedFs`] — pod-lifetime, expresses the write-block boundary for //! - [`ScopedFs`] — Pod-process lifetime, expresses the write-block
//! the current scope. Derived from the manifest and shareable across //! boundary for the current scope. Derived from the manifest; not
//! sessions. //! persisted across Pod restart.
//! - [`Tracker`] — session-lifetime, enforces the "read before edit" //! - [`Tracker`] — Pod-process lifetime, enforces the "read before edit"
//! policy via content hashes and tracks the recency of touched files. //! policy via content hashes and tracks the recency of touched files.
//! Recreated fresh per session. //! Recreated fresh on each Pod start (including resume).
//! //!
//! The Pod layer owns both instances and passes them to //! The Pod layer owns both instances and passes them to
//! [`builtin_tools`] when registering tools on a `Worker`. //! [`builtin_tools`] when registering tools on a `Worker`.
@ -42,11 +42,11 @@ pub use tracker::Tracker;
pub use write::write_tool; pub use write::write_tool;
/// Register all builtin tools, wiring them to a shared `ScopedFs` /// Register all builtin tools, wiring them to a shared `ScopedFs`
/// (pod-lifetime) and `Tracker` (session-lifetime). /// (Pod-process lifetime) and `Tracker` (Pod-process lifetime).
/// ///
/// All returned factories share the same tracker instance so that /// All returned factories share the same tracker instance so that
/// `Read` / `Write` / `Edit` see a consistent history across tool /// `Read` / `Write` / `Edit` see a consistent history across tool
/// invocations within a single session. /// invocations within a single Pod run.
/// ///
/// `bash_output_dir` is where the Bash tool spills long outputs. The /// `bash_output_dir` is where the Bash tool spills long outputs. The
/// caller is responsible for adding that path to the readable scope /// caller is responsible for adding that path to the readable scope

View File

@ -1,8 +1,9 @@
//! Session-scoped TaskStore and builtin task tools. //! Session-lifetime TaskStore and builtin task tools.
//! //!
//! The store is Pod/session-lifetime state shared by the four Task* tools. It //! The store survives compaction and Pod restart — it is reconstructed
//! is reconstructed on resume by replaying TaskCreate / TaskUpdate tool-call //! on resume by replaying TaskCreate / TaskUpdate tool-call arguments
//! arguments from persisted history. //! from persisted history, so its effective lifetime is the
//! [`session_store::SessionId`] (the conversation), not the Pod process.
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
@ -251,24 +252,25 @@ struct TaskUpdateTool {
store: TaskStore, store: TaskStore,
} }
const CREATE_DESCRIPTION: &str = "Create a session-lifetime task for short-term current-work \ const CREATE_DESCRIPTION: &str = "Create a session-lifetime task only when user-visible \
tracking, not project management. Tasks are user-visible real-time status for work with a \ progress tracking is genuinely useful: multiple active tasks must be remembered, or the work \
concrete goal that needs multiple meaningful steps, such as implementation, debugging, \ will involve long edits, long-running commands, extended investigation, or interruption-prone \
investigation, or structured review. Do not create tasks for simple questions, brief answers, or \ coordination. Do not create a task just because a request has several steps, and do not create \
single-step actions. Input only `subject` and `description`; `taskid` is assigned automatically \ one for short questions, quick checks, single reviews, or one-off commands. Prefer updating an \
and initial `status` is `pending`."; existing active task over creating a duplicate. Input only `subject` and `description`; `taskid` \
is assigned automatically and initial `status` is `pending`.";
const LIST_DESCRIPTION: &str = "List every session-lifetime task, including completed and \ const LIST_DESCRIPTION: &str = "List every session-lifetime task, including completed and \
deleted entries. Tasks are user-visible real-time status for short-term current-work tracking. \ deleted entries. Tasks are user-visible real-time status for short-term current-work tracking. \
Takes an empty object as input."; Takes an empty object as input.";
const GET_DESCRIPTION: &str = "Get one session-lifetime task by `taskid`. Tasks are \ const GET_DESCRIPTION: &str = "Get one session-lifetime task by `taskid`. Tasks are \
user-visible real-time status for short-term current-work tracking. Returns an error if the task \ user-visible real-time status for short-term current-work tracking. Returns an error if the task \
does not exist."; does not exist.";
const UPDATE_DESCRIPTION: &str = "Update an existing session-lifetime task as progress changes \ const UPDATE_DESCRIPTION: &str = "Update an existing session-lifetime task when meaningful \
between meaningful steps. Tasks are user-visible real-time status for multi-step work; keep \ progress changes between substantial steps. Tasks are user-visible real-time status, so avoid \
status current with `pending`, `inprogress`, `completed`, or `deleted`. Provide `taskid` and at \ churn for trivial substeps. Keep status current with `pending`, `inprogress`, `completed`, or \
least one of `status`, `subject`, or `description`; deletion is logical (`status = deleted`). If \ `deleted`. Provide `taskid` and at least one of `status`, `subject`, or `description`; deletion is \
an unexpected problem blocks progress, do not force the next step: leave the task as-is, \ logical (`status = deleted`). If an unexpected problem blocks progress, do not force the next \
summarize the problem to the user, and end the turn."; step: leave the task as-is, summarize the problem to the user, and end the turn.";
#[async_trait] #[async_trait]
impl Tool for TaskCreateTool { impl Tool for TaskCreateTool {

View File

@ -1,4 +1,4 @@
//! Session-scoped tracker for file operations performed by the builtin //! Pod-lifetime tracker for file operations performed by the builtin
//! file-manipulation tools. //! file-manipulation tools.
//! //!
//! A `Tracker` serves two orthogonal purposes: //! A `Tracker` serves two orthogonal purposes:
@ -18,11 +18,13 @@
//! //!
//! # Lifetime //! # Lifetime
//! //!
//! A `Tracker` is **session-scoped**: the Pod layer creates a fresh //! A `Tracker` is **Pod-process scoped**: the Pod layer creates a fresh
//! instance at the start of each agent session and discards it when the //! instance at the start of each Pod run (including resume) and discards
//! session ends. The `ScopedFs` write boundary, by contrast, is //! it when the process exits — it is not persisted, so a resumed
//! pod-lifetime (derived from the manifest). The two are orthogonal and //! conversation starts with an empty read/edit history. The `ScopedFs`
//! the Pod wires them together when registering builtin tools. //! write boundary is likewise Pod-process scoped (derived from the
//! manifest). The two are orthogonal and the Pod wires them together
//! when registering builtin tools.
//! //!
//! ```no_run //! ```no_run
//! # use std::path::PathBuf; //! # use std::path::PathBuf;

View File

@ -22,4 +22,5 @@ pulldown-cmark = { version = "0.13.3", default-features = false }
llm-worker.workspace = true llm-worker.workspace = true
[dev-dependencies] [dev-dependencies]
tempfile = { workspace = true }
tools = { workspace = true } tools = { workspace = true }

View File

@ -57,6 +57,10 @@ pub struct App {
/// cache reads excluded). Reset on `RunEnd`. /// cache reads excluded). Reset on `RunEnd`.
pub run_upload_tokens: u64, pub run_upload_tokens: u64,
pub run_output_tokens: u64, pub run_output_tokens: u64,
/// Latest session context tokens reported by the Pod. This is the raw
/// `input_tokens` value and is independent from per-run upload totals.
pub session_context_tokens: u64,
pub context_window: u64,
pub turn_index: usize, pub turn_index: usize,
pub current_tool: Option<String>, pub current_tool: Option<String>,
pub input: InputBuffer, pub input: InputBuffer,
@ -100,6 +104,8 @@ impl App {
run_requests: 0, run_requests: 0,
run_upload_tokens: 0, run_upload_tokens: 0,
run_output_tokens: 0, run_output_tokens: 0,
session_context_tokens: 0,
context_window: 0,
turn_index: 0, turn_index: 0,
current_tool: None, current_tool: None,
input: InputBuffer::new(), input: InputBuffer::new(),
@ -483,7 +489,7 @@ impl App {
self.blocks.push(Block::UserMessage { segments }); self.blocks.push(Block::UserMessage { segments });
self.assistant_streaming = false; self.assistant_streaming = false;
} }
Event::SessionRotated { entry } => { Event::SegmentRotated { entry } => {
self.reset_for_rotation(); self.reset_for_rotation();
self.apply_log_entry_raw(&entry); self.apply_log_entry_raw(&entry);
self.assistant_streaming = false; self.assistant_streaming = false;
@ -502,9 +508,7 @@ impl App {
// for `tickets/invoke-turn-llmcall-semantics.md`; events flow // for `tickets/invoke-turn-llmcall-semantics.md`; events flow
// through to subscribers but the TUI currently derives its // through to subscribers but the TUI currently derives its
// turn header from `UserMessage` / `SystemItem` arrivals. // turn header from `UserMessage` / `SystemItem` arrivals.
Event::InvokeStart { .. } Event::InvokeStart { .. } | Event::LlmCallStart { .. } | Event::LlmCallEnd { .. } => {}
| Event::LlmCallStart { .. }
| Event::LlmCallEnd { .. } => {}
Event::TextDelta { text } => { Event::TextDelta { text } => {
self.append_assistant_text(&text); self.append_assistant_text(&text);
} }
@ -651,6 +655,7 @@ impl App {
output_tokens, output_tokens,
cache_read_input_tokens, cache_read_input_tokens,
} => { } => {
self.session_context_tokens = input_tokens.unwrap_or(0);
// Subtract the cache-hit portion so a tool loop that // Subtract the cache-hit portion so a tool loop that
// re-sends the same prefix on every request doesn't // re-sends the same prefix on every request doesn't
// re-count it. cache_creation stays in (it is full // re-count it. cache_creation stays in (it is full
@ -685,7 +690,8 @@ impl App {
started_at: Instant::now(), started_at: Instant::now(),
})); }));
} }
Event::CompactDone { new_session_id } => { Event::CompactDone { new_segment_id } => {
self.session_context_tokens = 0;
if let Some(evt) = self.last_streaming_compact_mut() { if let Some(evt) = self.last_streaming_compact_mut() {
let elapsed_secs = match evt { let elapsed_secs = match evt {
CompactEvent::Streaming { started_at } => { CompactEvent::Streaming { started_at } => {
@ -694,12 +700,12 @@ impl App {
_ => None, _ => None,
}; };
*evt = CompactEvent::Done { *evt = CompactEvent::Done {
new_session_id, new_segment_id,
elapsed_secs, elapsed_secs,
}; };
} else { } else {
self.blocks.push(Block::Compact(CompactEvent::Done { self.blocks.push(Block::Compact(CompactEvent::Done {
new_session_id, new_segment_id,
elapsed_secs: None, elapsed_secs: None,
})); }));
} }
@ -916,6 +922,8 @@ impl App {
/// produced. Followed by `Event::Entry` updates for anything /// produced. Followed by `Event::Entry` updates for anything
/// committed after the snapshot. /// committed after the snapshot.
fn restore_snapshot(&mut self, entries: &[serde_json::Value], greeting: protocol::Greeting) { fn restore_snapshot(&mut self, entries: &[serde_json::Value], greeting: protocol::Greeting) {
self.context_window = greeting.context_window;
self.session_context_tokens = greeting.context_tokens;
self.turn_index = 0; self.turn_index = 0;
self.blocks.clear(); self.blocks.clear();
self.cache = FileCache::new(); self.cache = FileCache::new();
@ -932,7 +940,7 @@ impl App {
} }
/// Drop the derived view in preparation for replaying a new /// Drop the derived view in preparation for replaying a new
/// `SessionStart` (compaction / fork). Greeting is preserved /// `SegmentStart` (compaction / fork). Greeting is preserved
/// because the Pod identity hasn't changed. /// because the Pod identity hasn't changed.
fn reset_for_rotation(&mut self) { fn reset_for_rotation(&mut self) {
let greeting = self.blocks.iter().find_map(|b| match b { let greeting = self.blocks.iter().find_map(|b| match b {
@ -958,7 +966,7 @@ impl App {
return; return;
}; };
match entry { match entry {
session_store::LogEntry::SessionStart { history, .. } => { session_store::LogEntry::SegmentStart { history, .. } => {
for logged in history { for logged in history {
let item: llm_worker::Item = logged.into(); let item: llm_worker::Item = logged.into();
let item_value = serde_json::to_value(&item).expect("Item is Serialize"); let item_value = serde_json::to_value(&item).expect("Item is Serialize");
@ -984,22 +992,6 @@ impl App {
let value = serde_json::to_value(&item).expect("SystemItem is Serialize"); let value = serde_json::to_value(&item).expect("SystemItem is Serialize");
self.apply_system_item(&value); self.apply_system_item(&value);
} }
session_store::LogEntry::AssistantItems { items, .. }
| session_store::LogEntry::ToolResults { items, .. }
| session_store::LogEntry::HookInjectedItems { items, .. } => {
for logged in items {
let item: llm_worker::Item = logged.into();
let item_value = serde_json::to_value(&item).expect("Item is Serialize");
self.push_history_item(&item_value);
}
}
session_store::LogEntry::SystemItems { items, .. } => {
for system_item in items {
let value =
serde_json::to_value(&system_item).expect("SystemItem is Serialize");
self.apply_system_item(&value);
}
}
// Non-history-bearing variants don't affect the block view. // Non-history-bearing variants don't affect the block view.
_ => {} _ => {}
} }
@ -1445,8 +1437,9 @@ mod completion_flow_tests {
#[test] #[test]
fn snapshot_renders_system_message_block_from_session_start() { fn snapshot_renders_system_message_block_from_session_start() {
let mut app = App::new("test".into()); let mut app = App::new("test".into());
let session_start = session_store::LogEntry::SessionStart { let session_start = session_store::LogEntry::SegmentStart {
ts: 1, ts: 1,
session_id: uuid::Uuid::nil(),
system_prompt: None, system_prompt: None,
config: Default::default(), config: Default::default(),
history: vec![session_store::LoggedItem::from( history: vec![session_store::LoggedItem::from(
@ -1525,15 +1518,15 @@ mod completion_flow_tests {
let id = uuid::Uuid::parse_str("12345678-1234-5678-1234-567812345678").unwrap(); let id = uuid::Uuid::parse_str("12345678-1234-5678-1234-567812345678").unwrap();
app.handle_pod_event(Event::CompactStart); app.handle_pod_event(Event::CompactStart);
app.handle_pod_event(Event::CompactDone { new_session_id: id }); app.handle_pod_event(Event::CompactDone { new_segment_id: id });
assert_eq!(compact_block_count(&app), 1); assert_eq!(compact_block_count(&app), 1);
assert!(matches!( assert!(matches!(
app.blocks.as_slice(), app.blocks.as_slice(),
[Block::Compact(CompactEvent::Done { [Block::Compact(CompactEvent::Done {
new_session_id, new_segment_id,
elapsed_secs: Some(_), elapsed_secs: Some(_),
})] if *new_session_id == id })] if *new_segment_id == id
)); ));
} }
@ -1587,9 +1580,68 @@ mod completion_flow_tests {
model: "test-model".into(), model: "test-model".into(),
scope_summary: String::new(), scope_summary: String::new(),
tools: Vec::new(), tools: Vec::new(),
context_window: 200_000,
context_tokens: 0,
} }
} }
#[test]
fn snapshot_initializes_context_usage() {
let mut app = App::new("test".into());
let mut greeting = test_greeting();
greeting.context_window = 123_000;
greeting.context_tokens = 45_000;
app.handle_pod_event(Event::Snapshot {
entries: Vec::new(),
greeting,
status: PodStatus::Idle,
});
assert_eq!(app.context_window, 123_000);
assert_eq!(app.session_context_tokens, 45_000);
}
#[test]
fn usage_updates_session_context_tokens_without_cache_discount() {
let mut app = App::new("test".into());
app.handle_pod_event(Event::Usage {
input_tokens: Some(42_000),
output_tokens: Some(9),
cache_read_input_tokens: Some(40_000),
});
assert_eq!(app.session_context_tokens, 42_000);
assert_eq!(app.run_upload_tokens, 2_000);
assert_eq!(app.run_output_tokens, 9);
}
#[test]
fn compact_done_resets_session_context_tokens() {
let mut app = App::new("test".into());
app.session_context_tokens = 42_000;
app.handle_pod_event(Event::CompactDone {
new_segment_id: uuid::Uuid::nil(),
});
assert_eq!(app.session_context_tokens, 0);
}
#[test]
fn turn_start_and_run_end_do_not_reset_session_context_tokens() {
let mut app = App::new("test".into());
app.session_context_tokens = 42_000;
app.handle_pod_event(Event::TurnStart { turn: 1 });
app.handle_pod_event(Event::RunEnd {
result: RunResult::Finished,
});
assert_eq!(app.session_context_tokens, 42_000);
}
#[test] #[test]
fn live_task_create_updates_task_store() { fn live_task_create_updates_task_store() {
let mut app = App::new("test".into()); let mut app = App::new("test".into());
@ -1686,33 +1738,41 @@ mod completion_flow_tests {
arguments: r#"{"subject":"live","description":""}"#.into(), arguments: r#"{"subject":"live","description":""}"#.into(),
}); });
let assistant_items_entry = serde_json::json!({ let assistant_item_entries = vec![
"kind": "assistant_items", serde_json::json!({
"ts": 1, "kind": "assistant_item",
"items": [ "ts": 1,
{ "item": {
"kind": "tool_call", "kind": "tool_call",
"call_id": "c1", "call_id": "c1",
"name": "TaskCreate", "name": "TaskCreate",
"arguments": r#"{"subject":"a","description":"A"}"#, "arguments": r#"{"subject":"a","description":"A"}"#,
}, },
{ }),
serde_json::json!({
"kind": "assistant_item",
"ts": 2,
"item": {
"kind": "tool_call", "kind": "tool_call",
"call_id": "c2", "call_id": "c2",
"name": "TaskCreate", "name": "TaskCreate",
"arguments": r#"{"subject":"b","description":"B"}"#, "arguments": r#"{"subject":"b","description":"B"}"#,
}, },
{ }),
serde_json::json!({
"kind": "assistant_item",
"ts": 3,
"item": {
"kind": "tool_call", "kind": "tool_call",
"call_id": "u1", "call_id": "u1",
"name": "TaskUpdate", "name": "TaskUpdate",
"arguments": r#"{"taskid":2,"status":"inprogress"}"#, "arguments": r#"{"taskid":2,"status":"inprogress"}"#,
}, },
], }),
}); ];
app.handle_pod_event(Event::Snapshot { app.handle_pod_event(Event::Snapshot {
greeting: test_greeting(), greeting: test_greeting(),
entries: vec![assistant_items_entry], entries: assistant_item_entries,
status: PodStatus::Running, status: PodStatus::Running,
}); });

View File

@ -83,7 +83,7 @@ pub enum CompactEvent {
Streaming { started_at: Instant }, Streaming { started_at: Instant },
/// Compaction ended cleanly with `CompactDone`. /// Compaction ended cleanly with `CompactDone`.
Done { Done {
new_session_id: uuid::Uuid, new_segment_id: uuid::Uuid,
elapsed_secs: Option<u64>, elapsed_secs: Option<u64>,
}, },
/// Compaction ended with `CompactFailed`. /// Compaction ended with `CompactFailed`.

View File

@ -10,9 +10,16 @@ mod task;
mod tool; mod tool;
mod ui; mod ui;
use std::future::Future;
use std::io; use std::io;
use std::path::PathBuf; use std::path::PathBuf;
use std::process::ExitCode; use std::process::ExitCode;
use std::sync::{
Arc,
atomic::{AtomicBool, Ordering},
};
use std::thread;
use std::time::Duration;
use crossterm::event::{ use crossterm::event::{
self, DisableBracketedPaste, DisableMouseCapture, EnableBracketedPaste, EnableMouseCapture, self, DisableBracketedPaste, DisableMouseCapture, EnableBracketedPaste, EnableMouseCapture,
@ -25,7 +32,8 @@ use crossterm::terminal::{
use protocol::{Method, PodStatus}; use protocol::{Method, PodStatus};
use ratatui::Terminal; use ratatui::Terminal;
use ratatui::backend::CrosstermBackend; use ratatui::backend::CrosstermBackend;
use session_store::SessionId; use session_store::SegmentId;
use tokio::sync::mpsc;
use client::PodClient; use client::PodClient;
@ -45,22 +53,27 @@ fn resolve_socket(pod_name: &str, override_path: Option<PathBuf>) -> PathBuf {
}) })
} }
#[derive(Debug)]
enum Mode { enum Mode {
Spawn, Spawn,
Attach { /// `tui <name>` / `tui --pod <name>`: attach to a live Pod by name if
/// possible; otherwise launch `pod --pod <name>` so the pod process
/// resumes from name-keyed state or creates a fresh same-name Pod.
PodName {
pod_name: String, pod_name: String,
socket_override: Option<PathBuf>, socket_override: Option<PathBuf>,
}, },
/// `tui -r` / `tui --resume`: open the session picker first, then /// `tui -r` / `tui --resume`: open the Pod picker, then attach to the
/// run the same name dialog as Spawn but in resume mode. /// selected live Pod or restore the selected stopped Pod by name.
Resume, Resume,
/// `tui --session <UUID>`: skip the picker, go straight to the /// `tui --session <UUID>`: skip the picker, go straight to the
/// resume name dialog with `id` baked in. /// resume name dialog with `id` baked in.
ResumeWithSession(SessionId), ResumeWithSession(SegmentId),
} }
#[derive(Debug)]
enum ParseError { enum ParseError {
Conflict, Conflict(&'static str),
InvalidSession(String), InvalidSession(String),
MissingValue(&'static str), MissingValue(&'static str),
} }
@ -68,7 +81,7 @@ enum ParseError {
impl std::fmt::Display for ParseError { impl std::fmt::Display for ParseError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self { match self {
Self::Conflict => write!(f, "--resume and --session are mutually exclusive"), Self::Conflict(message) => write!(f, "{message}"),
Self::InvalidSession(s) => write!(f, "invalid --session UUID: {s}"), Self::InvalidSession(s) => write!(f, "invalid --session UUID: {s}"),
Self::MissingValue(flag) => write!(f, "{flag} requires a value"), Self::MissingValue(flag) => write!(f, "{flag} requires a value"),
} }
@ -76,9 +89,18 @@ impl std::fmt::Display for ParseError {
} }
fn parse_args() -> Result<Mode, ParseError> { fn parse_args() -> Result<Mode, ParseError> {
let args: Vec<String> = std::env::args().skip(1).collect(); parse_args_from(std::env::args().skip(1))
}
fn parse_args_from<I, S>(args: I) -> Result<Mode, ParseError>
where
I: IntoIterator<Item = S>,
S: Into<String>,
{
let args: Vec<String> = args.into_iter().map(Into::into).collect();
let mut resume = false; let mut resume = false;
let mut session: Option<SessionId> = None; let mut session: Option<SegmentId> = None;
let mut pod: Option<String> = None;
let mut socket_override: Option<PathBuf> = None; let mut socket_override: Option<PathBuf> = None;
let mut positional: Option<String> = None; let mut positional: Option<String> = None;
@ -94,11 +116,16 @@ fn parse_args() -> Result<Mode, ParseError> {
.get(i + 1) .get(i + 1)
.ok_or(ParseError::MissingValue("--session"))?; .ok_or(ParseError::MissingValue("--session"))?;
session = Some( session = Some(
raw.parse::<SessionId>() raw.parse::<SegmentId>()
.map_err(|_| ParseError::InvalidSession(raw.clone()))?, .map_err(|_| ParseError::InvalidSession(raw.clone()))?,
); );
i += 2; i += 2;
} }
"--pod" => {
let raw = args.get(i + 1).ok_or(ParseError::MissingValue("--pod"))?;
pod = Some(raw.clone());
i += 2;
}
"--socket" => { "--socket" => {
let raw = args let raw = args
.get(i + 1) .get(i + 1)
@ -119,9 +146,27 @@ fn parse_args() -> Result<Mode, ParseError> {
} }
if resume && session.is_some() { if resume && session.is_some() {
return Err(ParseError::Conflict); return Err(ParseError::Conflict(
"--resume and --session are mutually exclusive",
));
}
if pod.is_some() && session.is_some() {
return Err(ParseError::Conflict(
"--pod and --session are mutually exclusive",
));
}
if pod.is_some() && resume {
return Err(ParseError::Conflict(
"--pod and --resume are mutually exclusive",
));
} }
if let Some(pod_name) = pod {
return Ok(Mode::PodName {
pod_name,
socket_override,
});
}
if let Some(id) = session { if let Some(id) = session {
return Ok(Mode::ResumeWithSession(id)); return Ok(Mode::ResumeWithSession(id));
} }
@ -129,7 +174,7 @@ fn parse_args() -> Result<Mode, ParseError> {
return Ok(Mode::Resume); return Ok(Mode::Resume);
} }
if let Some(pod_name) = positional { if let Some(pod_name) = positional {
return Ok(Mode::Attach { return Ok(Mode::PodName {
pod_name, pod_name,
socket_override, socket_override,
}); });
@ -159,10 +204,10 @@ async fn main() -> ExitCode {
let result = match mode { let result = match mode {
Mode::Spawn => run_spawn(None).await, Mode::Spawn => run_spawn(None).await,
Mode::Attach { Mode::PodName {
pod_name, pod_name,
socket_override, socket_override,
} => run_attach(pod_name, socket_override).await, } => run_pod_name(pod_name, socket_override).await,
Mode::Resume => run_resume().await, Mode::Resume => run_resume().await,
Mode::ResumeWithSession(id) => run_spawn(Some(id)).await, Mode::ResumeWithSession(id) => run_spawn(Some(id)).await,
}; };
@ -186,8 +231,8 @@ async fn main() -> ExitCode {
// SpawnError has already been painted into the inline // SpawnError has already been painted into the inline
// viewport's final frame, so it's already visible in the // viewport's final frame, so it's already visible in the
// user's scrollback — printing it again would be a noisy // user's scrollback — printing it again would be a noisy
// duplicate. Other errors (attach-mode failures, terminal // duplicate. Other errors (pod-name failures, terminal setup
// setup hiccups, etc.) need surfacing here. // hiccups, etc.) need surfacing here.
if e.downcast_ref::<spawn::SpawnError>().is_none() { if e.downcast_ref::<spawn::SpawnError>().is_none() {
eprintln!("tui: {e}"); eprintln!("tui: {e}");
} }
@ -196,27 +241,75 @@ async fn main() -> ExitCode {
} }
} }
async fn run_attach( async fn run_pod_name(
pod_name: String, pod_name: String,
socket_override: Option<PathBuf>, socket_override: Option<PathBuf>,
) -> Result<(), Box<dyn std::error::Error>> { ) -> Result<(), Box<dyn std::error::Error>> {
let socket_path = resolve_socket(&pod_name, socket_override); let preferred_socket = resolve_socket(&pod_name, socket_override.clone());
if let Some((_socket_path, client)) =
connect_live_pod(&pod_name, preferred_socket, socket_override.is_none()).await
{
let mut terminal = enter_fullscreen()?;
let mut app = App::new(pod_name);
app.connected = true;
return run_loop(&mut terminal, &mut app, client).await;
}
let ready = match spawn::run_pod_name(pod_name).await? {
SpawnOutcome::Ready(r) => r,
SpawnOutcome::Cancelled => return Ok(()),
};
let SpawnReady {
pod_name,
socket_path,
} = ready;
let mut terminal = enter_fullscreen()?; let mut terminal = enter_fullscreen()?;
run(&mut terminal, pod_name, &socket_path).await let result = run(&mut terminal, pod_name, &socket_path).await;
let _ = execute!(
terminal.backend_mut(),
DisableMouseCapture,
LeaveAlternateScreen
);
result
}
async fn connect_live_pod(
pod_name: &str,
preferred_socket: PathBuf,
allow_registry_fallback: bool,
) -> Option<(PathBuf, PodClient)> {
if let Ok(client) = PodClient::connect(&preferred_socket).await {
return Some((preferred_socket, client));
}
if !allow_registry_fallback {
return None;
}
let registry_socket = picker::live_socket_for_pod(pod_name)?;
if registry_socket == preferred_socket {
return None;
}
PodClient::connect(&registry_socket)
.await
.ok()
.map(|client| (registry_socket, client))
} }
async fn run_resume() -> Result<(), Box<dyn std::error::Error>> { async fn run_resume() -> Result<(), Box<dyn std::error::Error>> {
// Phase 1: pick a session in its own inline viewport, dropping the // Pick a Pod in its own inline viewport, dropping the viewport before
// viewport before the name dialog opens so each phase gets fresh // attaching/restoring so each phase gets fresh vertical room.
// vertical room. let (pod_name, socket_override) = match picker::run().await? {
let id = match picker::run().await? { PickerOutcome::Picked {
PickerOutcome::Picked(id) => id, pod_name,
socket_override,
} => (pod_name, socket_override),
PickerOutcome::Cancelled => return Ok(()), PickerOutcome::Cancelled => return Ok(()),
}; };
run_spawn(Some(id)).await run_pod_name(pod_name, socket_override).await
} }
async fn run_spawn(resume_from: Option<SessionId>) -> Result<(), Box<dyn std::error::Error>> { async fn run_spawn(resume_from: Option<SegmentId>) -> Result<(), Box<dyn std::error::Error>> {
let ready = match spawn::run(resume_from).await? { let ready = match spawn::run(resume_from).await? {
SpawnOutcome::Ready(r) => r, SpawnOutcome::Ready(r) => r,
SpawnOutcome::Cancelled => return Ok(()), SpawnOutcome::Cancelled => return Ok(()),
@ -274,11 +367,137 @@ async fn run(
Ok(()) Ok(())
} }
type TerminalEventResult = io::Result<TermEvent>;
const TERMINAL_POLL_INTERVAL: Duration = Duration::from_millis(50);
const TERMINAL_EVENT_DRAIN_LIMIT: usize = 64;
const POD_EVENT_DRAIN_LIMIT: usize = 32;
struct TerminalEventReader {
stop: Arc<AtomicBool>,
_thread: thread::JoinHandle<()>,
}
impl TerminalEventReader {
fn spawn() -> io::Result<(Self, mpsc::UnboundedReceiver<TerminalEventResult>)> {
let (tx, rx) = mpsc::unbounded_channel();
let stop = Arc::new(AtomicBool::new(false));
let thread_stop = Arc::clone(&stop);
let thread = thread::Builder::new()
.name("insomnia-tui-terminal-reader".to_string())
.spawn(move || read_terminal_events(thread_stop, tx))?;
Ok((
Self {
stop,
_thread: thread,
},
rx,
))
}
}
impl Drop for TerminalEventReader {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
}
}
fn read_terminal_events(stop: Arc<AtomicBool>, tx: mpsc::UnboundedSender<TerminalEventResult>) {
while !stop.load(Ordering::Relaxed) {
match event::poll(TERMINAL_POLL_INTERVAL) {
Ok(false) => {}
Ok(true) => {
let event = event::read();
let should_stop = event.is_err();
if tx.send(event).is_err() || should_stop {
break;
}
}
Err(e) => {
let _ = tx.send(Err(e));
break;
}
}
}
}
enum LoopInput<P> {
Terminal(TerminalEventResult),
Pod(Option<P>),
}
async fn next_loop_input<P, F>(
term_rx: &mut mpsc::UnboundedReceiver<TerminalEventResult>,
connected: bool,
pod_next: F,
) -> LoopInput<P>
where
F: Future<Output = Option<P>>,
{
tokio::select! {
biased;
term_event = term_rx.recv() => {
LoopInput::Terminal(term_event.unwrap_or_else(|| {
Err(io::Error::new(
io::ErrorKind::UnexpectedEof,
"terminal event reader stopped",
))
}))
}
event = pod_next, if connected => LoopInput::Pod(event),
}
}
async fn drain_terminal_events(
app: &mut App,
client: &mut PodClient,
term_rx: &mut mpsc::UnboundedReceiver<TerminalEventResult>,
) -> Result<bool, Box<dyn std::error::Error>> {
let mut handled = false;
for _ in 0..TERMINAL_EVENT_DRAIN_LIMIT {
match term_rx.try_recv() {
Ok(event) => {
handled = true;
handle_terminal_event(app, client, event?).await?;
if app.quit {
break;
}
}
Err(mpsc::error::TryRecvError::Empty) => break,
Err(mpsc::error::TryRecvError::Disconnected) => {
return Err(Box::new(io::Error::new(
io::ErrorKind::UnexpectedEof,
"terminal event reader stopped",
)));
}
}
}
Ok(handled)
}
fn drain_pod_events(app: &mut App, client: &mut PodClient) -> bool {
let mut handled = false;
for _ in 0..POD_EVENT_DRAIN_LIMIT {
match client.try_next_event() {
Some(ev) => {
handled = true;
app.handle_pod_event(ev);
}
None => break,
}
}
handled
}
async fn run_loop( async fn run_loop(
terminal: &mut Terminal<CrosstermBackend<io::Stdout>>, terminal: &mut Terminal<CrosstermBackend<io::Stdout>>,
app: &mut App, app: &mut App,
mut client: PodClient, mut client: PodClient,
) -> Result<(), Box<dyn std::error::Error>> { ) -> Result<(), Box<dyn std::error::Error>> {
let (_terminal_reader, mut term_rx) = TerminalEventReader::spawn()?;
terminal.draw(|f| ui::draw(f, app))?; terminal.draw(|f| ui::draw(f, app))?;
loop { loop {
@ -286,56 +505,28 @@ async fn run_loop(
break; break;
} }
// Drain any already-buffered Pod events in a bounded batch before let handled_term_event = drain_terminal_events(app, &mut client, &mut term_rx).await?;
// polling the terminal. This keeps status fresh without letting a
// busy event stream starve Ctrl-C / Ctrl-X input.
for _ in 0..32 {
match client.try_next_event() {
Some(ev) => app.handle_pod_event(ev),
None => break,
}
}
// Always give the terminal queue a non-blocking pass each frame.
// The awaited select below only waits after this pass found nothing.
let mut handled_term_event = false;
while event::poll(std::time::Duration::ZERO)? {
handled_term_event = true;
handle_terminal_event(app, &mut client, event::read()?).await?;
if app.quit {
break;
}
}
if app.quit { if app.quit {
break; break;
} }
if handled_term_event { let handled_pod_event = drain_pod_events(app, &mut client);
if handled_term_event || handled_pod_event {
terminal.draw(|f| ui::draw(f, app))?; terminal.draw(|f| ui::draw(f, app))?;
continue; continue;
} }
tokio::select! { match next_loop_input(&mut term_rx, app.connected, client.next_event()).await {
term_event = tokio::task::spawn_blocking(|| { LoopInput::Terminal(term_event) => {
if event::poll(std::time::Duration::from_millis(50))? { handle_terminal_event(app, &mut client, term_event?).await?;
event::read().map(Some)
} else {
Ok(None)
}
}) => {
if let Some(term_event) = term_event?? {
handle_terminal_event(app, &mut client, term_event).await?;
}
} }
event = client.next_event(), if app.connected => { LoopInput::Pod(event) => match event {
match event { Some(ev) => app.handle_pod_event(ev),
Some(ev) => app.handle_pod_event(ev), None => {
None => { app.connected = false;
app.connected = false; app.mark_orphan_compacts_incomplete();
app.mark_orphan_compacts_incomplete(); app.push_error("Connection lost");
app.push_error("Connection lost");
}
} }
} },
} }
terminal.draw(|f| ui::draw(f, app))?; terminal.draw(|f| ui::draw(f, app))?;
@ -612,3 +803,120 @@ fn handle_pause_or_quit(app: &mut App) -> Option<Method> {
app.push_error("Press Ctrl-C again within 3 s to exit the TUI (the Pod keeps running)."); app.push_error("Press Ctrl-C again within 3 s to exit the TUI (the Pod keeps running).");
None None
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_pod_name_mode() {
match parse_args_from(["--pod", "agent", "--socket", "/tmp/agent.sock"]).unwrap() {
Mode::PodName {
pod_name,
socket_override,
} => {
assert_eq!(pod_name, "agent");
assert_eq!(socket_override, Some(PathBuf::from("/tmp/agent.sock")));
}
_ => panic!("expected PodName mode"),
}
}
#[test]
fn parse_positional_name_uses_pod_name_mode() {
match parse_args_from(["agent"]).unwrap() {
Mode::PodName {
pod_name,
socket_override,
} => {
assert_eq!(pod_name, "agent");
assert_eq!(socket_override, None);
}
_ => panic!("expected PodName mode"),
}
}
#[test]
fn parse_rejects_pod_and_session() {
let segment_id = session_store::new_segment_id().to_string();
let err = parse_args_from(["--pod", "agent", "--session", &segment_id]).unwrap_err();
assert_eq!(
err.to_string(),
"--pod and --session are mutually exclusive"
);
}
#[tokio::test]
async fn terminal_event_is_selected_before_ready_pod_event() {
let (tx, mut rx) = mpsc::unbounded_channel();
tx.send(Ok(TermEvent::Key(KeyEvent::new(
KeyCode::Char('x'),
KeyModifiers::NONE,
))))
.unwrap();
match next_loop_input(&mut rx, true, std::future::ready(Some(()))).await {
LoopInput::Terminal(Ok(TermEvent::Key(key))) => {
assert_eq!(key.code, KeyCode::Char('x'));
}
_ => panic!("ready terminal input should win over a ready Pod event"),
}
}
#[tokio::test]
async fn terminal_event_is_preserved_after_pod_event_wins() {
let (tx, mut rx) = mpsc::unbounded_channel();
match next_loop_input(&mut rx, true, std::future::ready(Some(1_u8))).await {
LoopInput::Pod(Some(1)) => {}
_ => panic!("expected the first ready Pod event to win before any terminal input"),
}
tx.send(Ok(TermEvent::Key(KeyEvent::new(
KeyCode::Char('y'),
KeyModifiers::NONE,
))))
.unwrap();
match next_loop_input(&mut rx, true, std::future::ready(Some(2_u8))).await {
LoopInput::Terminal(Ok(TermEvent::Key(key))) => {
assert_eq!(key.code, KeyCode::Char('y'));
}
_ => panic!("queued terminal input should not be lost to subsequent Pod events"),
}
}
#[test]
fn running_status_still_allows_text_editing() {
let mut app = App::new("agent".to_string());
app.set_pod_status(PodStatus::Running);
assert!(
handle_key(
&mut app,
KeyEvent::new(KeyCode::Char('a'), KeyModifiers::NONE)
)
.is_none()
);
assert!(
handle_key(
&mut app,
KeyEvent::new(KeyCode::Char('c'), KeyModifiers::NONE)
)
.is_none()
);
assert!(handle_key(&mut app, KeyEvent::new(KeyCode::Left, KeyModifiers::NONE)).is_none());
assert!(
handle_key(
&mut app,
KeyEvent::new(KeyCode::Char('b'), KeyModifiers::NONE)
)
.is_none()
);
assert_eq!(
protocol::Segment::flatten_to_text(&app.input.submit_segments()),
"abc"
);
}
}

View File

@ -1,18 +1,18 @@
//! Inline-viewport "pick a session to restore" UX. //! Inline-viewport "pick a Pod to attach or restore" UX.
//! //!
//! Reads the most recent sessions from the configured store, lets the //! Reads live Pod allocations from the runtime registry and stopped Pod state
//! user pick one with the arrow keys, and returns the chosen //! from the session store's name-keyed metadata. Picking a live row attaches to
//! `SessionId`. Closes its inline viewport before returning so the //! its socket; picking a stopped row restores via `pod --pod <name>`.
//! caller can open a fresh viewport for the name dialog.
//!
//! The picker only handles selection. Forking, pod-registry checks, and
//! actual `pod` launch happen later in the resume flow.
use std::collections::{BTreeMap, HashMap};
use std::fs;
use std::io; use std::io;
use std::path::{Path, PathBuf};
use std::time::Duration; use std::time::Duration;
use client::PodClient;
use crossterm::event::{self, Event as TermEvent, KeyCode, KeyEventKind, KeyModifiers}; use crossterm::event::{self, Event as TermEvent, KeyCode, KeyEventKind, KeyModifiers};
use pod_registry::lookup_session; use pod_registry::{LockFileGuard, default_registry_path};
use ratatui::Terminal; use ratatui::Terminal;
use ratatui::backend::CrosstermBackend; use ratatui::backend::CrosstermBackend;
use ratatui::layout::{Constraint, Layout}; use ratatui::layout::{Constraint, Layout};
@ -21,7 +21,7 @@ use ratatui::text::{Line, Span};
use ratatui::widgets::Paragraph; use ratatui::widgets::Paragraph;
use ratatui::{Frame, TerminalOptions, Viewport}; use ratatui::{Frame, TerminalOptions, Viewport};
use session_store::{ use session_store::{
FsStore, HashedEntry, LogEntry, LoggedContentPart, LoggedItem, SessionId, Store, FsStore, LogEntry, LoggedContentPart, LoggedItem, PodMetadata, SegmentId, SessionId, Store,
}; };
const MAX_ROWS: usize = 10; const MAX_ROWS: usize = 10;
@ -31,7 +31,7 @@ const VIEWPORT_LINES: u16 = MAX_ROWS as u16 + 4;
pub enum PickerError { pub enum PickerError {
Io(io::Error), Io(io::Error),
Store(session_store::StoreError), Store(session_store::StoreError),
NoSessions, NoPods,
} }
impl std::fmt::Display for PickerError { impl std::fmt::Display for PickerError {
@ -39,9 +39,9 @@ impl std::fmt::Display for PickerError {
match self { match self {
Self::Io(e) => write!(f, "io error: {e}"), Self::Io(e) => write!(f, "io error: {e}"),
Self::Store(e) => write!(f, "session store error: {e}"), Self::Store(e) => write!(f, "session store error: {e}"),
Self::NoSessions => write!( Self::NoPods => write!(
f, f,
"no sessions found — start a fresh pod with `tui` and try again" "no pods found — start a fresh pod with `tui` and try again"
), ),
} }
} }
@ -62,41 +62,77 @@ impl From<session_store::StoreError> for PickerError {
} }
pub enum PickerOutcome { pub enum PickerOutcome {
Picked(SessionId), /// User picked a Pod. `socket_override` is set for live rows when the
/// runtime registry knows the exact socket path; stopped rows leave it
/// empty so the caller restores with `pod --pod <name>`.
Picked {
pod_name: String,
socket_override: Option<PathBuf>,
},
Cancelled, Cancelled,
} }
/// One row in the picker view. Rendered from the session log so the #[derive(Debug, Clone, Copy, PartialEq, Eq)]
/// user can recognise their session at a glance without parsing UUIDs. enum PodRowState {
Live,
Stopped,
Corrupt,
}
impl PodRowState {
fn label(self) -> &'static str {
match self {
Self::Live => "live",
Self::Stopped => "stopped",
Self::Corrupt => "corrupt",
}
}
fn style(self) -> Style {
match self {
Self::Live => Style::default()
.fg(Color::Green)
.add_modifier(Modifier::BOLD),
Self::Stopped => Style::default().fg(Color::Yellow),
Self::Corrupt => Style::default().fg(Color::Red).add_modifier(Modifier::BOLD),
}
}
}
/// One row in the Pod picker. The primary key is the Pod name; Session/Segment
/// IDs are included only as debug context.
#[derive(Debug, Clone)]
struct Row { struct Row {
id: SessionId, pod_name: String,
/// Last user / assistant snippet, or a `[corrupt]` placeholder. state: PodRowState,
preview: String, updated_at: u64,
/// `Some(pod_name)` when a live Pod currently holds an allocation active_session_id: Option<SessionId>,
/// for this session in `pods.json`. Picking such a row launches active_segment_id: Option<SegmentId>,
/// `pod --session <UUID>` which will fail with `SessionConflict` — preview: Option<String>,
/// the badge warns the user up-front. socket_path: Option<PathBuf>,
live_pod: Option<String>, }
#[derive(Debug)]
struct PodStateRecord {
pod_name: String,
state: Result<PodMetadata, String>,
}
#[derive(Debug, Clone)]
pub(crate) struct LivePodRecord {
pub pod_name: String,
pub socket_path: PathBuf,
pub segment_id: Option<SegmentId>,
} }
pub async fn run() -> Result<PickerOutcome, PickerError> { pub async fn run() -> Result<PickerOutcome, PickerError> {
let store = open_default_store()?; let store_dir = default_store_dir()?;
let ids = store.list_sessions()?; let store = FsStore::new(&store_dir)?;
if ids.is_empty() { let pod_states = read_pod_state_records(&store_dir)?;
return Err(PickerError::NoSessions); let live_pods = read_reachable_live_pod_records().await.unwrap_or_default();
} let rows = build_rows(&store, pod_states, live_pods)?;
let mut rows: Vec<Row> = Vec::with_capacity(MAX_ROWS); if rows.is_empty() {
for id in ids.into_iter().take(MAX_ROWS) { return Err(PickerError::NoPods);
let preview = build_preview(&store, id);
// Best-effort live check. A pods.json I/O hiccup downgrades
// the row to "no badge" rather than killing the picker — the
// user still gets to see the listing.
let live_pod = lookup_session(id).ok().flatten().map(|info| info.pod_name);
rows.push(Row {
id,
preview,
live_pod,
});
} }
let mut selected = 0usize; let mut selected = 0usize;
@ -106,9 +142,7 @@ pub async fn run() -> Result<PickerOutcome, PickerError> {
match poll_event()? { match poll_event()? {
None => continue, None => continue,
Some(Action::Up) => { Some(Action::Up) => {
if selected > 0 { selected = selected.saturating_sub(1);
selected -= 1;
}
} }
Some(Action::Down) => { Some(Action::Down) => {
if selected + 1 < rows.len() { if selected + 1 < rows.len() {
@ -117,7 +151,11 @@ pub async fn run() -> Result<PickerOutcome, PickerError> {
} }
Some(Action::Submit) => { Some(Action::Submit) => {
close_viewport(&mut terminal)?; close_viewport(&mut terminal)?;
return Ok(PickerOutcome::Picked(rows[selected].id)); let row = &rows[selected];
return Ok(PickerOutcome::Picked {
pod_name: row.pod_name.clone(),
socket_override: row.socket_path.clone(),
});
} }
Some(Action::Cancel) => { Some(Action::Cancel) => {
close_viewport(&mut terminal)?; close_viewport(&mut terminal)?;
@ -127,17 +165,9 @@ pub async fn run() -> Result<PickerOutcome, PickerError> {
} }
} }
/// Park the cursor at the very bottom of the picker's inline viewport /// Park the cursor at the very bottom of the picker's inline viewport and emit
/// and emit one newline before dropping the terminal. Without this the /// one newline before dropping the terminal. This keeps any next inline viewport
/// inline area is left with the cursor still inside it, so the next /// from drawing over the lower picker rows.
/// `Terminal::with_options(Inline(_))` call (the resume name dialog)
/// computes its own area starting from inside the picker — drawing the
/// new dialog on top of the lower picker rows.
///
/// Setting the cursor to `area.bottom() - 1` and writing `\r\n`
/// scrolls the terminal up exactly one row, so the next inline
/// viewport opens immediately below the picker rather than on top of
/// it.
fn close_viewport(terminal: &mut Terminal<CrosstermBackend<io::Stdout>>) -> io::Result<()> { fn close_viewport(terminal: &mut Terminal<CrosstermBackend<io::Stdout>>) -> io::Result<()> {
let area = terminal.get_frame().area(); let area = terminal.get_frame().area();
let last_row = area.bottom().saturating_sub(1); let last_row = area.bottom().saturating_sub(1);
@ -149,38 +179,271 @@ fn close_viewport(terminal: &mut Terminal<CrosstermBackend<io::Stdout>>) -> io::
Ok(()) Ok(())
} }
fn open_default_store() -> Result<FsStore, PickerError> { fn default_store_dir() -> Result<PathBuf, PickerError> {
let dir = manifest::paths::sessions_dir().ok_or_else(|| { manifest::paths::sessions_dir().ok_or_else(|| {
PickerError::Io(io::Error::new( PickerError::Io(io::Error::new(
io::ErrorKind::NotFound, io::ErrorKind::NotFound,
"could not resolve sessions directory \ "could not resolve sessions directory \
(set INSOMNIA_HOME, INSOMNIA_DATA_DIR, or HOME)", (set INSOMNIA_HOME, INSOMNIA_DATA_DIR, or HOME)",
)) ))
})?; })
Ok(FsStore::new(&dir)?)
} }
fn build_preview(store: &FsStore, id: SessionId) -> String { fn read_pod_state_records(store_dir: &Path) -> Result<Vec<PodStateRecord>, PickerError> {
match store.read_all(id) { let pods_dir = store_dir.join("pods");
Ok(entries) => last_message_preview(&entries).unwrap_or_else(|| "[empty]".to_string()), let mut records = Vec::new();
Err(_) => "[corrupt]".to_string(), if !pods_dir.exists() {
return Ok(records);
}
for entry in fs::read_dir(pods_dir)? {
let entry = entry?;
if !entry.file_type()?.is_dir() {
continue;
}
let pod_name = entry.file_name().to_string_lossy().to_string();
let path = entry.path().join("metadata.json");
let state = match fs::read_to_string(&path) {
Ok(content) => serde_json::from_str::<PodMetadata>(&content).map_err(|e| e.to_string()),
Err(e) => Err(e.to_string()),
};
records.push(PodStateRecord { pod_name, state });
}
Ok(records)
}
fn read_live_pod_records() -> Result<Vec<LivePodRecord>, io::Error> {
let path = default_registry_path()?;
let guard = LockFileGuard::open(&path)?;
Ok(guard
.data()
.allocations
.iter()
.map(|allocation| LivePodRecord {
pod_name: allocation.pod_name.clone(),
socket_path: allocation.socket.clone(),
segment_id: allocation.segment_id,
})
.collect())
}
async fn read_reachable_live_pod_records() -> Result<Vec<LivePodRecord>, io::Error> {
let records = read_live_pod_records()?;
let mut reachable = Vec::new();
for record in records {
if PodClient::connect(&record.socket_path).await.is_ok() {
reachable.push(record);
}
}
Ok(reachable)
}
pub(crate) fn live_socket_for_pod(pod_name: &str) -> Option<PathBuf> {
read_live_pod_records()
.ok()?
.into_iter()
.find(|pod| pod.pod_name == pod_name)
.map(|pod| pod.socket_path)
}
fn build_rows(
store: &FsStore,
pod_states: Vec<PodStateRecord>,
live_pods: Vec<LivePodRecord>,
) -> Result<Vec<Row>, PickerError> {
let mut rows_by_name: BTreeMap<String, Row> = BTreeMap::new();
let mut live_by_name: HashMap<String, LivePodRecord> = HashMap::new();
for live in live_pods {
let (active_session_id, active_segment_id, updated_at, preview) =
summarize_live_pod(store, &live);
rows_by_name.insert(
live.pod_name.clone(),
Row {
pod_name: live.pod_name.clone(),
state: PodRowState::Live,
updated_at,
active_session_id,
active_segment_id,
preview,
socket_path: Some(live.socket_path.clone()),
},
);
live_by_name.insert(live.pod_name.clone(), live);
}
for record in pod_states {
match record.state {
Ok(metadata) => {
let summary = summarize_metadata(store, &metadata);
let state = if live_by_name.contains_key(&record.pod_name) {
PodRowState::Live
} else {
PodRowState::Stopped
};
upsert_metadata_row(&mut rows_by_name, record.pod_name, metadata, summary, state);
}
Err(message) => {
rows_by_name.entry(record.pod_name.clone()).or_insert(Row {
pod_name: record.pod_name,
state: PodRowState::Corrupt,
updated_at: 0,
active_session_id: None,
active_segment_id: None,
preview: Some(format!("metadata: {}", trim_one_line(&message, 48))),
socket_path: None,
});
}
}
}
let mut rows: Vec<Row> = rows_by_name.into_values().collect();
rows.sort_by(|a, b| {
b.updated_at
.cmp(&a.updated_at)
.then_with(|| a.pod_name.cmp(&b.pod_name))
});
rows.truncate(MAX_ROWS);
Ok(rows)
}
fn upsert_metadata_row(
rows_by_name: &mut BTreeMap<String, Row>,
pod_name: String,
metadata: PodMetadata,
summary: SegmentSummary,
state: PodRowState,
) {
let active = metadata.active;
let active_session_id = active.as_ref().map(|a| a.session_id);
let active_segment_id = active.as_ref().and_then(|a| a.segment_id);
match rows_by_name.get_mut(&pod_name) {
Some(existing) => {
existing.state = state;
if summary.updated_at > existing.updated_at {
existing.updated_at = summary.updated_at;
}
if existing.active_session_id.is_none() {
existing.active_session_id = active_session_id;
}
if existing.active_segment_id.is_none() {
existing.active_segment_id = active_segment_id;
}
if existing.preview.is_none() {
existing.preview = summary.preview;
}
}
None => {
rows_by_name.insert(
pod_name.clone(),
Row {
pod_name,
state,
updated_at: summary.updated_at,
active_session_id,
active_segment_id,
preview: summary.preview,
socket_path: None,
},
);
}
} }
} }
/// Walk the log from the tail looking for the most recent user-message #[derive(Debug, Clone)]
/// or assistant-message entry, then render its first text fragment in struct SegmentSummary {
/// a single line. updated_at: u64,
fn last_message_preview(entries: &[HashedEntry]) -> Option<String> { preview: Option<String>,
for hashed in entries.iter().rev() { }
match &hashed.entry {
fn summarize_live_pod(
store: &FsStore,
live: &LivePodRecord,
) -> (Option<SessionId>, Option<SegmentId>, u64, Option<String>) {
let Some(segment_id) = live.segment_id else {
return (None, None, 0, None);
};
let session_id = store.lookup_session_of(segment_id).ok().flatten();
let Some(session_id) = session_id else {
return (None, Some(segment_id), 0, None);
};
let summary = summarize_segment(store, session_id, segment_id);
(
Some(session_id),
Some(segment_id),
summary.updated_at,
summary.preview,
)
}
fn summarize_metadata(store: &FsStore, metadata: &PodMetadata) -> SegmentSummary {
let Some(active) = metadata.active.as_ref() else {
return SegmentSummary {
updated_at: 0,
preview: None,
};
};
let Some(segment_id) = active.segment_id else {
return SegmentSummary {
updated_at: 0,
preview: Some("[pending segment]".to_string()),
};
};
summarize_segment(store, active.session_id, segment_id)
}
fn summarize_segment(
store: &FsStore,
session_id: SessionId,
segment_id: SegmentId,
) -> SegmentSummary {
match store.read_all(session_id, segment_id) {
Ok(entries) => SegmentSummary {
updated_at: last_entry_ts(&entries).unwrap_or(0),
preview: last_message_preview(&entries).or_else(|| Some("[empty]".to_string())),
},
Err(_) => SegmentSummary {
updated_at: 0,
preview: Some("[corrupt segment]".to_string()),
},
}
}
fn last_entry_ts(entries: &[LogEntry]) -> Option<u64> {
entries.iter().map(log_entry_ts).max()
}
fn log_entry_ts(entry: &LogEntry) -> u64 {
match entry {
LogEntry::SegmentStart { ts, .. }
| LogEntry::Invoke { ts, .. }
| LogEntry::UserInput { ts, .. }
| LogEntry::AssistantItem { ts, .. }
| LogEntry::ToolResult { ts, .. }
| LogEntry::SystemItem { ts, .. }
| LogEntry::TurnEnd { ts, .. }
| LogEntry::RunCompleted { ts, .. }
| LogEntry::RunErrored { ts, .. }
| LogEntry::ConfigChanged { ts, .. }
| LogEntry::LlmUsage { ts, .. }
| LogEntry::Extension { ts, .. } => *ts,
}
}
/// Walk the log from the tail looking for the most recent user-message or
/// assistant-message entry, then render its first text fragment in a single line.
fn last_message_preview(entries: &[LogEntry]) -> Option<String> {
for entry in entries.iter().rev() {
match entry {
LogEntry::UserInput { segments, .. } => { LogEntry::UserInput { segments, .. } => {
let text = protocol::Segment::flatten_to_text(segments); let text = protocol::Segment::flatten_to_text(segments);
if !text.is_empty() { if !text.is_empty() {
return Some(format!("user: {}", trim_one_line(&text, 60))); return Some(format!("user: {}", trim_one_line(&text, 60)));
} }
} }
LogEntry::AssistantItems { items, .. } => { LogEntry::AssistantItem { item, .. } => {
if let Some(text) = items.iter().find_map(first_text_logged) { if let Some(text) = first_text_logged(item) {
return Some(format!("assistant: {}", trim_one_line(&text, 60))); return Some(format!("assistant: {}", trim_one_line(&text, 60)));
} }
} }
@ -262,7 +525,7 @@ fn draw(f: &mut Frame<'_>, rows: &[Row], selected: usize) {
f.render_widget( f.render_widget(
Paragraph::new(Line::from(vec![Span::styled( Paragraph::new(Line::from(vec![Span::styled(
"resume pod pick a session", picker_title(),
Style::default().add_modifier(Modifier::BOLD), Style::default().add_modifier(Modifier::BOLD),
)])), )])),
layout[0], layout[0],
@ -278,7 +541,7 @@ fn draw(f: &mut Frame<'_>, rows: &[Row], selected: usize) {
Span::styled("[↑/↓]", Style::default().fg(Color::DarkGray)), Span::styled("[↑/↓]", Style::default().fg(Color::DarkGray)),
Span::raw(" select "), Span::raw(" select "),
Span::styled("[enter]", Style::default().fg(Color::Green)), Span::styled("[enter]", Style::default().fg(Color::Green)),
Span::raw(" pick "), Span::raw(" attach/restore "),
Span::styled("[esc]", Style::default().fg(Color::Yellow)), Span::styled("[esc]", Style::default().fg(Color::Yellow)),
Span::raw(" cancel"), Span::raw(" cancel"),
])), ])),
@ -286,9 +549,13 @@ fn draw(f: &mut Frame<'_>, rows: &[Row], selected: usize) {
); );
} }
fn picker_title() -> &'static str {
"resume pod pick a pod"
}
fn row_line(row: &Row, selected: bool) -> Line<'_> { fn row_line(row: &Row, selected: bool) -> Line<'_> {
let marker = if selected { "" } else { " " }; let marker = if selected { "" } else { " " };
let id_style = if selected { let name_style = if selected {
Style::default() Style::default()
.fg(Color::Cyan) .fg(Color::Cyan)
.add_modifier(Modifier::BOLD) .add_modifier(Modifier::BOLD)
@ -300,22 +567,246 @@ fn row_line(row: &Row, selected: bool) -> Line<'_> {
} else { } else {
Style::default().fg(Color::DarkGray) Style::default().fg(Color::DarkGray)
}; };
let mut spans = vec![ let mut spans = vec![
Span::raw(marker), Span::raw(marker),
Span::styled(short_session(row.id), id_style), Span::styled(row.pod_name.as_str(), name_style),
Span::raw(" "), Span::raw(" "),
Span::styled(format!("[{}]", row.state.label()), row.state.style()),
Span::raw(" "),
Span::styled(
format_updated_at(row.updated_at),
Style::default().fg(Color::DarkGray),
),
Span::raw(" "),
Span::styled(debug_ids(row), Style::default().fg(Color::DarkGray)),
]; ];
if let Some(ref pod_name) = row.live_pod { if let Some(preview) = row.preview.as_ref() {
spans.push(Span::styled( spans.push(Span::raw(" "));
format!("[live: {pod_name}] "), spans.push(Span::styled(preview.as_str(), preview_style));
Style::default().fg(Color::Red).add_modifier(Modifier::BOLD),
));
} }
spans.push(Span::styled(row.preview.clone(), preview_style));
Line::from(spans) Line::from(spans)
} }
fn short_session(id: SessionId) -> String { fn format_updated_at(updated_at: u64) -> String {
let s = id.to_string(); if updated_at == 0 {
s.chars().take(8).collect() "updated: —".to_string()
} else {
format!("updated: {updated_at}")
}
}
fn debug_ids(row: &Row) -> String {
let session = row
.active_session_id
.map(short_id)
.unwrap_or_else(|| "--------".to_string());
let segment = row
.active_segment_id
.map(short_id)
.unwrap_or_else(|| "--------".to_string());
format!("s:{session} g:{segment}")
}
fn short_id<T: ToString>(id: T) -> String {
id.to_string().chars().take(8).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use llm_worker::llm_client::types::RequestConfig;
use session_store::{PodActiveSegmentRef, PodMetadataStore, new_segment_id, new_session_id};
use tempfile::tempdir;
#[test]
fn pod_rows_are_sorted_by_active_segment_timestamp() {
let dir = tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let earlier_session = new_session_id();
let later_session = new_session_id();
let earlier_segment = new_segment_id();
let later_segment = new_segment_id();
append_start(&store, earlier_session, earlier_segment, 10);
append_user(
&store,
earlier_session,
earlier_segment,
100,
"old pod update",
);
append_start(&store, later_session, later_segment, 20);
append_user(&store, later_session, later_segment, 200, "new pod update");
let records = vec![
metadata_record("older", earlier_session, earlier_segment),
metadata_record("newer", later_session, later_segment),
];
let rows = build_rows(&store, records, vec![]).unwrap();
assert_eq!(rows[0].pod_name, "newer");
assert_eq!(rows[0].state, PodRowState::Stopped);
assert_eq!(rows[0].updated_at, 200);
assert_eq!(rows[0].preview.as_deref(), Some("user: new pod update"));
assert_eq!(rows[1].pod_name, "older");
}
#[test]
fn pod_rows_include_live_and_stopped_pods() {
let dir = tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let stopped_session = new_session_id();
let stopped_segment = new_segment_id();
let live_session = new_session_id();
let live_segment = new_segment_id();
append_start(&store, stopped_session, stopped_segment, 10);
append_user(
&store,
stopped_session,
stopped_segment,
50,
"stopped preview",
);
append_start(&store, live_session, live_segment, 20);
append_user(&store, live_session, live_segment, 70, "live preview");
let rows = build_rows(
&store,
vec![metadata_record("stopped", stopped_session, stopped_segment)],
vec![LivePodRecord {
pod_name: "live".to_string(),
socket_path: PathBuf::from("/tmp/live.sock"),
segment_id: Some(live_segment),
}],
)
.unwrap();
let live = rows.iter().find(|row| row.pod_name == "live").unwrap();
assert_eq!(live.state, PodRowState::Live);
assert_eq!(live.active_session_id, Some(live_session));
assert_eq!(
live.socket_path.as_deref(),
Some(Path::new("/tmp/live.sock"))
);
let stopped = rows.iter().find(|row| row.pod_name == "stopped").unwrap();
assert_eq!(stopped.state, PodRowState::Stopped);
assert_eq!(stopped.socket_path, None);
}
#[test]
fn corrupt_pod_state_is_rendered_as_corrupt_row() {
let dir = tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let rows = build_rows(
&store,
vec![PodStateRecord {
pod_name: "broken".to_string(),
state: Err("expected value".to_string()),
}],
vec![],
)
.unwrap();
assert_eq!(rows.len(), 1);
assert_eq!(rows[0].pod_name, "broken");
assert_eq!(rows[0].state, PodRowState::Corrupt);
assert!(
rows[0]
.preview
.as_deref()
.unwrap()
.contains("expected value")
);
}
#[test]
fn picker_title_names_pods_not_sessions() {
assert_eq!(picker_title(), "resume pod pick a pod");
}
fn metadata_record(
pod_name: &str,
session_id: SessionId,
segment_id: SegmentId,
) -> PodStateRecord {
PodStateRecord {
pod_name: pod_name.to_string(),
state: Ok(PodMetadata::new(
pod_name,
Some(PodActiveSegmentRef::active_segment(session_id, segment_id)),
)),
}
}
fn append_start(store: &FsStore, session_id: SessionId, segment_id: SegmentId, ts: u64) {
store
.append(
session_id,
segment_id,
&LogEntry::SegmentStart {
ts,
session_id,
system_prompt: None,
config: RequestConfig::default(),
history: vec![],
forked_from: None,
compacted_from: None,
},
)
.unwrap();
}
fn append_user(
store: &FsStore,
session_id: SessionId,
segment_id: SegmentId,
ts: u64,
text: &str,
) {
store
.append(
session_id,
segment_id,
&LogEntry::UserInput {
ts,
segments: vec![protocol::Segment::text(text)],
},
)
.unwrap();
}
#[test]
fn read_pod_state_records_reports_corrupt_metadata() {
let dir = tempdir().unwrap();
let pod_dir = dir.path().join("pods").join("broken");
fs::create_dir_all(&pod_dir).unwrap();
fs::write(pod_dir.join("metadata.json"), "{not-json").unwrap();
let records = read_pod_state_records(dir.path()).unwrap();
assert_eq!(records.len(), 1);
assert_eq!(records[0].pod_name, "broken");
assert!(records[0].state.is_err());
}
#[test]
fn read_pod_state_records_reads_metadata() {
let dir = tempdir().unwrap();
let store = FsStore::new(dir.path()).unwrap();
let session_id = new_session_id();
let segment_id = new_segment_id();
store
.write(&PodMetadata::new(
"agent",
Some(PodActiveSegmentRef::active_segment(session_id, segment_id)),
))
.unwrap();
let records = read_pod_state_records(dir.path()).unwrap();
assert_eq!(records.len(), 1);
assert_eq!(records[0].pod_name, "agent");
assert!(records[0].state.is_ok());
}
} }

View File

@ -28,7 +28,7 @@ use ratatui::style::{Color, Modifier, Style};
use ratatui::text::{Line, Span}; use ratatui::text::{Line, Span};
use ratatui::widgets::Paragraph; use ratatui::widgets::Paragraph;
use ratatui::{Frame, TerminalOptions, Viewport}; use ratatui::{Frame, TerminalOptions, Viewport};
use session_store::SessionId; use session_store::SegmentId;
const VIEWPORT_LINES: u16 = 6; const VIEWPORT_LINES: u16 = 6;
@ -46,7 +46,7 @@ pub enum SpawnOutcome {
pub enum SpawnError { pub enum SpawnError {
Io(io::Error), Io(io::Error),
Store(session_store::StoreError), Store(session_store::StoreError),
MissingResumeScope { session_id: SessionId }, MissingResumeScope { segment_id: SegmentId },
Spawn(client::SpawnError), Spawn(client::SpawnError),
} }
@ -55,9 +55,9 @@ impl std::fmt::Display for SpawnError {
match self { match self {
Self::Io(e) => write!(f, "io error: {e}"), Self::Io(e) => write!(f, "io error: {e}"),
Self::Store(e) => write!(f, "failed to read session log: {e}"), Self::Store(e) => write!(f, "failed to read session log: {e}"),
Self::MissingResumeScope { session_id } => write!( Self::MissingResumeScope { segment_id } => write!(
f, f,
"session {session_id} has no persisted scope snapshot; refusing resume without explicit scope" "session {segment_id} has no persisted scope snapshot; refusing resume without explicit scope"
), ),
Self::Spawn(e) => write!(f, "{e}"), Self::Spawn(e) => write!(f, "{e}"),
} }
@ -89,57 +89,19 @@ type InlineTerminal = Terminal<CrosstermBackend<io::Stdout>>;
/// Source session for a resume run. `None` = fresh spawn (current /// Source session for a resume run. `None` = fresh spawn (current
/// behaviour); `Some(id)` swaps the dialog into "Resume Pod" mode and /// behaviour); `Some(id)` swaps the dialog into "Resume Pod" mode and
/// passes `--session <id>` to the spawned `pod` child. /// passes `--session <id>` to the spawned `pod` child.
pub async fn run(resume_from: Option<SessionId>) -> Result<SpawnOutcome, SpawnError> { pub async fn run(resume_from: Option<SegmentId>) -> Result<SpawnOutcome, SpawnError> {
let cwd = std::env::current_dir().map_err(SpawnError::Io)?; let defaults = load_spawn_defaults()?;
// Run the same merge pod itself uses, then read what's missing
// off the result. We only look at `scope.allow` here — `pod.name`
// is intentionally an instance-level identifier and is always
// taken from the dialog regardless of what (if anything) a layer
// declared.
let user_layer = user_manifest_path()
.filter(|p| p.is_file())
.and_then(|p| load_layer(&p).ok());
let project_layer = find_project_manifest_from(&cwd).and_then(|p| load_layer(&p).ok());
let mut cascade = PodManifestConfig::builtin_defaults();
for layer in [user_layer.as_ref(), project_layer.as_ref()]
.into_iter()
.flatten()
{
cascade = cascade.merge(layer.clone());
}
let cascade_has_scope = !cascade.scope.allow.is_empty();
let scope_origin = match (
project_layer
.as_ref()
.is_some_and(|l| !l.scope.allow.is_empty()),
user_layer
.as_ref()
.is_some_and(|l| !l.scope.allow.is_empty()),
) {
(true, _) => ScopeOrigin::FromProject,
(false, true) => ScopeOrigin::FromUser,
(false, false) => ScopeOrigin::CwdDefault,
};
let default_name = cwd
.file_name()
.and_then(|s| s.to_str())
.map(sanitise_default_name)
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "pod".to_string());
let mut form = Form { let mut form = Form {
cwd: cwd.clone(), cwd: defaults.cwd.clone(),
cascade_has_scope, cascade_has_scope: defaults.cascade_has_scope,
scope_origin, scope_origin: defaults.scope_origin,
name_cursor: default_name.chars().count(), name_cursor: defaults.default_name.chars().count(),
name: default_name, name: defaults.default_name,
message: None, message: None,
editing: true, editing: true,
resume_from, resume_from,
resume_by_pod_name: false,
resume_scope: None, resume_scope: None,
}; };
@ -206,6 +168,105 @@ pub async fn run(resume_from: Option<SessionId>) -> Result<SpawnOutcome, SpawnEr
} }
} }
/// Launch `pod --pod <name>` without opening the name dialog. The child Pod
/// resolves persisted Pod metadata if present, or creates a fresh same-name Pod
/// with the usual TUI cwd-scope fallback.
pub async fn run_pod_name(pod_name: String) -> Result<SpawnOutcome, SpawnError> {
let defaults = load_spawn_defaults()?;
let mut form = form_for_pod_name(pod_name, defaults);
let overlay_toml = build_overlay_toml(&form);
let mut terminal = make_inline_terminal()?;
terminal.draw(|f| draw_form(f, &form))?;
match wait_for_ready(&mut terminal, &mut form, &overlay_toml).await {
Ok(ready) => {
form.message = Some((
format!("ready: {} attaching...", ready.pod_name),
MessageKind::Ok,
));
terminal.draw(|f| draw_form(f, &form))?;
drop(terminal);
Ok(SpawnOutcome::Ready(ready))
}
Err(e) => {
form.message = Some((e.to_string(), MessageKind::Error));
let _ = terminal.draw(|f| draw_form(f, &form));
drop(terminal);
Err(e)
}
}
}
struct SpawnDefaults {
cwd: PathBuf,
cascade_has_scope: bool,
scope_origin: ScopeOrigin,
default_name: String,
}
fn load_spawn_defaults() -> Result<SpawnDefaults, SpawnError> {
let cwd = std::env::current_dir().map_err(SpawnError::Io)?;
// Run the same merge pod itself uses, then read what's missing off the
// result. We only look at `scope.allow` here — `pod.name` is an
// instance-level identifier and is supplied by the dialog or `--pod`.
let user_layer = user_manifest_path()
.filter(|p| p.is_file())
.and_then(|p| load_layer(&p).ok());
let project_layer = find_project_manifest_from(&cwd).and_then(|p| load_layer(&p).ok());
let mut cascade = PodManifestConfig::builtin_defaults();
for layer in [user_layer.as_ref(), project_layer.as_ref()]
.into_iter()
.flatten()
{
cascade = cascade.merge(layer.clone());
}
let cascade_has_scope = !cascade.scope.allow.is_empty();
let scope_origin = match (
project_layer
.as_ref()
.is_some_and(|l| !l.scope.allow.is_empty()),
user_layer
.as_ref()
.is_some_and(|l| !l.scope.allow.is_empty()),
) {
(true, _) => ScopeOrigin::FromProject,
(false, true) => ScopeOrigin::FromUser,
(false, false) => ScopeOrigin::CwdDefault,
};
let default_name = cwd
.file_name()
.and_then(|s| s.to_str())
.map(sanitise_default_name)
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "pod".to_string());
Ok(SpawnDefaults {
cwd,
cascade_has_scope,
scope_origin,
default_name,
})
}
fn form_for_pod_name(pod_name: String, defaults: SpawnDefaults) -> Form {
Form {
cwd: defaults.cwd,
cascade_has_scope: defaults.cascade_has_scope,
scope_origin: defaults.scope_origin,
name_cursor: pod_name.chars().count(),
name: pod_name,
message: Some(("resuming pod...".to_string(), MessageKind::Progress)),
editing: false,
resume_from: None,
resume_by_pod_name: true,
resume_scope: None,
}
}
fn make_inline_terminal() -> io::Result<InlineTerminal> { fn make_inline_terminal() -> io::Result<InlineTerminal> {
let backend = CrosstermBackend::new(io::stdout()); let backend = CrosstermBackend::new(io::stdout());
Terminal::with_options( Terminal::with_options(
@ -279,6 +340,7 @@ async fn wait_for_ready(
overlay_toml: overlay_toml.to_string(), overlay_toml: overlay_toml.to_string(),
cwd, cwd,
resume_from: form.resume_from, resume_from: form.resume_from,
resume_by_pod_name: form.resume_by_pod_name,
}; };
let ready = spawn_pod(config, |line| { let ready = spawn_pod(config, |line| {
form.message = Some((line.to_string(), MessageKind::Progress)); form.message = Some((line.to_string(), MessageKind::Progress));
@ -321,7 +383,7 @@ fn build_overlay_toml(form: &Form) -> String {
toml::to_string(&toml::Value::Table(root)).expect("overlay serialisation cannot fail") toml::to_string(&toml::Value::Table(root)).expect("overlay serialisation cannot fail")
} }
async fn load_resume_scope(session_id: SessionId) -> Result<ScopeConfig, SpawnError> { async fn load_resume_scope(segment_id: SegmentId) -> Result<ScopeConfig, SpawnError> {
let store_dir = manifest::paths::sessions_dir().ok_or_else(|| { let store_dir = manifest::paths::sessions_dir().ok_or_else(|| {
io::Error::new( io::Error::new(
io::ErrorKind::NotFound, io::ErrorKind::NotFound,
@ -329,16 +391,17 @@ async fn load_resume_scope(session_id: SessionId) -> Result<ScopeConfig, SpawnEr
) )
})?; })?;
let store = session_store::FsStore::new(&store_dir)?; let store = session_store::FsStore::new(&store_dir)?;
let state = session_store::restore(&store, session_id)?; let state = session_store::restore_by_segment(&store, segment_id)?;
let snapshot = state let snapshot = state
.pod_scope .pod_scope
.ok_or(SpawnError::MissingResumeScope { session_id })?; .ok_or(SpawnError::MissingResumeScope { segment_id })?;
Ok(ScopeConfig { Ok(ScopeConfig {
allow: snapshot.allow, allow: snapshot.allow,
deny: snapshot.deny, deny: snapshot.deny,
}) })
} }
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum MessageKind { enum MessageKind {
Info, Info,
Ok, Ok,
@ -376,7 +439,10 @@ struct Form {
/// switches, the source session is shown to the user, and the /// switches, the source session is shown to the user, and the
/// child pod is launched with `--session <id>` so it restores /// child pod is launched with `--session <id>` so it restores
/// from `id` and appends to the same session log. /// from `id` and appends to the same session log.
resume_from: Option<SessionId>, resume_from: Option<SegmentId>,
/// When true, launch the child with `--pod <name>` so the pod process
/// resolves name-keyed state before falling back to fresh creation.
resume_by_pod_name: bool,
/// Scope snapshot recovered from the source session log. Set only for /// Scope snapshot recovered from the source session log. Set only for
/// resume runs, and serialized into the overlay instead of cwd-default /// resume runs, and serialized into the overlay instead of cwd-default
/// scope so resume does not silently broaden access. /// scope so resume does not silently broaden access.
@ -445,7 +511,7 @@ fn draw_form(f: &mut Frame<'_>, form: &Form) {
.split(area); .split(area);
let title_text = match form.resume_from { let title_text = match form.resume_from {
Some(id) => format!("resume pod session: {}", short_session(id)), Some(id) => format!("resume pod session: {}", short_segment(id)),
None => "spawn pod".to_string(), None => "spawn pod".to_string(),
}; };
let title = Paragraph::new(Line::from(vec![Span::styled( let title = Paragraph::new(Line::from(vec![Span::styled(
@ -473,7 +539,7 @@ fn draw_form(f: &mut Frame<'_>, form: &Form) {
/// First 8 hex digits of a UUID — short enough to skim, long enough /// First 8 hex digits of a UUID — short enough to skim, long enough
/// to disambiguate inside a 10-row picker. /// to disambiguate inside a 10-row picker.
pub(crate) fn short_session(id: SessionId) -> String { pub(crate) fn short_segment(id: SegmentId) -> String {
let s = id.to_string(); let s = id.to_string();
s.chars().take(8).collect() s.chars().take(8).collect()
} }
@ -556,10 +622,33 @@ mod tests {
message: None, message: None,
editing: true, editing: true,
resume_from: None, resume_from: None,
resume_by_pod_name: false,
resume_scope: None, resume_scope: None,
} }
} }
#[test]
fn pod_name_form_restores_or_creates_by_pod_name() {
let defaults = SpawnDefaults {
cwd: PathBuf::from("/work/example"),
cascade_has_scope: true,
scope_origin: ScopeOrigin::FromProject,
default_name: "ignored".to_string(),
};
let f = form_for_pod_name("agent".to_string(), defaults);
assert_eq!(f.name, "agent");
assert_eq!(f.name_cursor, "agent".chars().count());
assert_eq!(f.resume_from, None);
assert!(f.resume_by_pod_name);
assert!(f.resume_scope.is_none());
assert!(!f.editing);
assert_eq!(
f.message,
Some(("resuming pod...".to_string(), MessageKind::Progress))
);
}
#[test] #[test]
fn overlay_adds_scope_default_when_cascade_lacks_scope() { fn overlay_adds_scope_default_when_cascade_lacks_scope() {
let f = form("agent-1", false); let f = form("agent-1", false);
@ -584,7 +673,7 @@ mod tests {
#[test] #[test]
fn overlay_uses_resume_scope_snapshot() { fn overlay_uses_resume_scope_snapshot() {
let mut f = form("agent-r", false); let mut f = form("agent-r", false);
f.resume_from = Some(session_store::new_session_id()); f.resume_from = Some(session_store::new_segment_id());
f.resume_scope = Some(ScopeConfig { f.resume_scope = Some(ScopeConfig {
allow: vec![manifest::ScopeRule { allow: vec![manifest::ScopeRule {
target: PathBuf::from("/work/example"), target: PathBuf::from("/work/example"),

View File

@ -7,6 +7,7 @@
//! ──────────── separator ────────── //! ──────────── separator ──────────
//! status line (1 row) //! status line (1 row)
//! > input area (1 row in Phase 1) //! > input area (1 row in Phase 1)
//! actionbar (1 row)
//! ``` //! ```
//! //!
//! Every frame we walk the entire `App::blocks` vector, produce styled //! Every frame we walk the entire `App::blocks` vector, produce styled
@ -78,6 +79,7 @@ pub fn draw(frame: &mut Frame, app: &mut App) {
Constraint::Length(1), // separator Constraint::Length(1), // separator
Constraint::Length(1), // status Constraint::Length(1), // status
Constraint::Length(input_height), // input area Constraint::Length(input_height), // input area
Constraint::Length(1), // actionbar
]) ])
.split(area); .split(area);
@ -88,6 +90,7 @@ pub fn draw(frame: &mut Frame, app: &mut App) {
draw_separator(frame, chunks[3]); draw_separator(frame, chunks[3]);
draw_status(frame, app, chunks[4]); draw_status(frame, app, chunks[4]);
draw_input(frame, &input_render, chunks[5]); draw_input(frame, &input_render, chunks[5]);
draw_actionbar(frame, app, chunks[6]);
if let Some(state) = app.completion.as_ref().filter(|c| c.is_active()) { if let Some(state) = app.completion.as_ref().filter(|c| c.is_active()) {
draw_completion_popup(frame, state, chunks[5]); draw_completion_popup(frame, state, chunks[5]);
} }
@ -1019,10 +1022,10 @@ fn render_compact(lines: &mut Vec<Line<'static>>, evt: &CompactEvent, width: u16
) )
} }
CompactEvent::Done { CompactEvent::Done {
new_session_id, new_segment_id,
elapsed_secs, elapsed_secs,
} => { } => {
let short = new_session_id let short = new_segment_id
.to_string() .to_string()
.chars() .chars()
.take(8) .take(8)
@ -1074,6 +1077,20 @@ fn draw_separator(frame: &mut Frame, area: Rect) {
); );
} }
fn context_usage_text(app: &App) -> String {
let pct = if app.context_window == 0 {
0
} else {
((app.session_context_tokens as f64 / app.context_window as f64) * 100.0).round() as u64
};
format!(
"{} / {} ({}%)",
fmt_tokens(app.session_context_tokens),
fmt_tokens(app.context_window),
pct
)
}
fn draw_status(frame: &mut Frame, app: &App, area: Rect) { fn draw_status(frame: &mut Frame, app: &App, area: Rect) {
let conn = if app.connected { let conn = if app.connected {
Span::styled("", Style::default().fg(Color::Green)) Span::styled("", Style::default().fg(Color::Green))
@ -1124,7 +1141,15 @@ fn draw_status(frame: &mut Frame, app: &App, area: Rect) {
spans.push(Span::styled(" idle", Style::default().fg(Color::DarkGray))); spans.push(Span::styled(" idle", Style::default().fg(Color::DarkGray)));
} }
// Right-aligned mode / scroll indicator. let right_text = context_usage_text(app);
let right_line = Line::from(Span::styled(right_text, Style::default().fg(Color::Gray)))
.alignment(ratatui::layout::Alignment::Right);
frame.render_widget(Paragraph::new(Line::from(spans)), area);
frame.render_widget(Paragraph::new(right_line), area);
}
fn draw_actionbar(frame: &mut Frame, app: &App, area: Rect) {
let mut right: Vec<Span<'static>> = Vec::new(); let mut right: Vec<Span<'static>> = Vec::new();
if !app.scroll.follow_tail { if !app.scroll.follow_tail {
right.push(Span::styled( right.push(Span::styled(
@ -1137,8 +1162,6 @@ fn draw_status(frame: &mut Frame, app: &App, area: Rect) {
Style::default().fg(Color::DarkGray), Style::default().fg(Color::DarkGray),
)); ));
let right_line = Line::from(right).alignment(ratatui::layout::Alignment::Right); let right_line = Line::from(right).alignment(ratatui::layout::Alignment::Right);
frame.render_widget(Paragraph::new(Line::from(spans)), area);
frame.render_widget(Paragraph::new(right_line), area); frame.render_widget(Paragraph::new(right_line), area);
} }

View File

@ -38,6 +38,7 @@ Pod::try_pre_run_compact ← proactive
- **条件付き実行**: 推定トークン節約量が `min_savings` を超えた場合のみ。KV キャッシュの無駄な無効化を避ける - **条件付き実行**: 推定トークン節約量が `min_savings` を超えた場合のみ。KV キャッシュの無駄な無効化を避ける
- **リクエストコンテキストのみ操作**: history 本体は変更しない。Prune 状態を Pod が保持し、LLM リクエスト構築時に反映する - **リクエストコンテキストのみ操作**: history 本体は変更しない。Prune 状態を Pod が保持し、LLM リクエスト構築時に反映する
- **保護境界**: 直近 `prune_protected_tokens` 相当の suffix は残す。turn 数ではなく usage history 由来の token estimate で境界を引くため、単発の長い tool loop でも古い `ToolResult.content` が候補になる
- **冪等**: `content: None` のアイテムはスキップ - **冪等**: `content: None` のアイテムはスキップ
### ToolOutput の構造 ### ToolOutput の構造
@ -138,8 +139,9 @@ compact は fork と同じ構造。旧セッションを保全し、新 SessionI
[compaction] [compaction]
compact_threshold = 80000 # ターンの合間 (proactive) compact_threshold = 80000 # ターンの合間 (proactive)
compact_request_threshold = 90000 # リクエストの合間 (safety net) compact_request_threshold = 90000 # リクエストの合間 (safety net)
retained_tokens = 8000 # 直近保護トークン数 (Prune 済みで計測) prune_protected_tokens = 8000 # prune から保護する末尾 token budget
auto_read_budget = 8000 # compact worker の mark_read_required 合計上限 compact_retained_tokens = 8000 # compact 後に生のまま残す末尾 token budget
compact_auto_read_budget = 8000 # compact worker の mark_read_required 合計上限
compact_worker_max_input_tokens = 50000 # compact worker 自身の現在占有トークン上限 compact_worker_max_input_tokens = 50000 # compact worker 自身の現在占有トークン上限
compact_worker_max_turns = 20 # compact worker 自身の tool loop 上限 compact_worker_max_turns = 20 # compact worker 自身の tool loop 上限
``` ```

View File

@ -191,9 +191,9 @@ permission = "write"
# セクションを書いた時点で Prune は有効化、Compact は閾値が None なら無効。 # セクションを書いた時点で Prune は有効化、Compact は閾値が None なら無効。
# [compaction] # [compaction]
# #
# # 任意。デフォルト: 3 (`defaults::PRUNE_PROTECTED_TURNS`)。 # # 任意。デフォルト: 8000 (`defaults::PRUNE_PROTECTED_TOKENS`)。
# # pruning から保護する末尾ターン数 # # pruning から保護する末尾 token budget。turn 数ではなく usage estimate で境界を引く
# prune_protected_turns = 3 # prune_protected_tokens = 8000
# #
# # 任意。デフォルト: 4096 (`defaults::PRUNE_MIN_SAVINGS`)。 # # 任意。デフォルト: 4096 (`defaults::PRUNE_MIN_SAVINGS`)。
# # prune が発火するための最低節約 token 推定値。 # # prune が発火するための最低節約 token 推定値。
@ -263,10 +263,6 @@ permission = "write"
# # ※ memory tools と resident injection は extract_threshold が None でも動く。 # # ※ memory tools と resident injection は extract_threshold が None でも動く。
# extract_threshold = 30000 # extract_threshold = 30000
# #
# # 任意。デフォルト: 30000 (`defaults::MEMORY_EXTRACT_WORKER_MAX_INPUT_TOKENS`)。
# # extract worker 自身の現在占有 token cap (超過で abort)。
# extract_worker_max_input_tokens = 30000
#
# # 任意。デフォルト: 8 (`defaults::MEMORY_EXTRACT_WORKER_MAX_TURNS`)。 # # 任意。デフォルト: 8 (`defaults::MEMORY_EXTRACT_WORKER_MAX_TURNS`)。
# # extract worker 自身の tool loop 上限。Rust config で None の場合のみ無制限。 # # extract worker 自身の tool loop 上限。Rust config で None の場合のみ無制限。
# extract_worker_max_turns = 8 # extract_worker_max_turns = 8

View File

@ -99,13 +99,13 @@ pub enum LogEntry {
UserInput { ts: u64, item: Item }, UserInput { ts: u64, item: Item },
// アシスタント応答worker.rs:1040-1041 に対応) // アシスタント応答worker.rs:1040-1041 に対応)
AssistantItems { ts: u64, items: Vec<Item> }, AssistantItem { ts: u64, item: Item },
// ツール実行結果worker.rs:897-900, 1072-1076 に対応) // ツール実行結果worker.rs:897-900, 1072-1076 に対応)
ToolResults { ts: u64, items: Vec<Item> }, ToolResult { ts: u64, item: Item },
// Hook 注入 Itemsworker.rs:1055 ContinueWithMessages に対応) // typed system injection
HookInjectedItems { ts: u64, items: Vec<Item> }, SystemItem { ts: u64, item: SystemItem },
// ターン境界 // ターン境界
TurnEnd { ts: u64, turn_count: usize }, TurnEnd { ts: u64, turn_count: usize },
@ -126,7 +126,7 @@ pub enum LogEntry {
pub enum Outcome { Finished, Paused, Error { message: String } } pub enum Outcome { Finished, Paused, Error { message: String } }
``` ```
**Replay ロジック**: 全エントリ種別を走査し、`*Items` / `UserInput` → history に append、 **Replay ロジック**: 全エントリ種別を走査し、`AssistantItem` / `ToolResult` / `SystemItem` / `UserInput` → history に append、
`TurnEnd` → turn_count 更新、`CacheLocked` → locked_prefix_len 設定。 `TurnEnd` → turn_count 更新、`CacheLocked` → locked_prefix_len 設定。
### TraceEntryevent_trace.rs ### TraceEntryevent_trace.rs

View File

@ -122,7 +122,7 @@ Workflow 保護は専用 tool schema のトリックではなく Linter ルー
- **Trigger**: activity tokens の累積閾値cumulative input tokens since last pointer。tool call カウントは不採用(ツールカスタマイズ非依存・大小重みづけのため) - **Trigger**: activity tokens の累積閾値cumulative input tokens since last pointer。tool call カウントは不採用(ツールカスタマイズ非依存・大小重みづけのため)
- **実行主体**: 既存 compact と同じ Worker spawn 機構を再利用。Pod は立てない - **実行主体**: 既存 compact と同じ Worker spawn 機構を再利用。Pod は立てない
- **入力**: 前回 extract 以降の session log 範囲。処理済み境界の pointer は session log 側に保持し、寿命を session と揃える。session-store のドメイン純度を保つため、汎用拡張点 `LogEntry::Extension { domain, payload }`domain = `"memory.extract"`に寄せ、session-store は memory ドメインを知らない - **入力**: 前回 extract 以降の session log 範囲。処理済み境界の pointer は session log 側に保持し、寿命を session と揃える。session-store のドメイン純度を保つため、汎用拡張点 `LogEntry::Extension { domain, payload }`domain = `"memory.extract"`に寄せ、session-store は memory ドメインを知らない。Tool result は raw `content` ではなく表示用 `summary` だけを render し、巨大な tool output を extract input に載せない
- **出力**: JSON schema で**活動ログ**の候補配列を返す。Knowledge 等の派生物は consolidation が活動ログから導出するので、extract では純粋な「起きたこと」に絞る - **出力**: JSON schema で**活動ログ**の候補配列を返す。Knowledge 等の派生物は consolidation が活動ログから導出するので、extract では純粋な「起きたこと」に絞る
- `decisions`: 判断したこと(選択肢 + 選んだ + 根拠) - `decisions`: 判断したこと(選択肢 + 選んだ + 根拠)
- `discussions`: 議論したこと(トピック + 論点) - `discussions`: 議論したこと(トピック + 論点)
@ -131,6 +131,7 @@ Workflow 保護は専用 tool schema のトリックではなく Linter ルー
- **抽出対象がなければ空配列を返してよい**Hermes の "Nothing to save." と同系。頻繁発火を許容する前提) - **抽出対象がなければ空配列を返してよい**Hermes の "Nothing to save." と同系。頻繁発火を許容する前提)
- **書き込み先**: `memory/_staging/<id>.json` - **書き込み先**: `memory/_staging/<id>.json`
- LLM 出力(活動ログ JSONは pod 側ラッパーが `source: { session_id, range: [start_entry, end_entry] }` を**機械付与**して wrap。LLM には source を推論させない - LLM 出力(活動ログ JSONは pod 側ラッパーが `source: { session_id, range: [start_entry, end_entry] }` を**機械付与**して wrap。LLM には source を推論させない
- **実行保証**: extract worker 自身の input occupancy cap は設けない。未処理 range が大きい場合でも pointer 以降の最大範囲を渡し、LLM/API/tool failure のときだけ pointer を進めない
- **モデル**: `memory.extract_model`。軽量だが文脈理解できる中堅クラスHaiku / 4o-mini / Flash 相当)を想定 - **モデル**: `memory.extract_model`。軽量だが文脈理解できる中堅クラスHaiku / 4o-mini / Flash 相当)を想定
- **Compact との順序**: 同一 turn 完了後の post-run チェックで extract を **compact より前** に走らせる。compact は history を組み替えるので、extract の入力範囲session log 上の entry indexは compact 前のほうが安定する - **Compact との順序**: 同一 turn 完了後の post-run チェックで extract を **compact より前** に走らせる。compact は history を組み替えるので、extract の入力範囲session log 上の entry indexは compact 前のほうが安定する
- **並走防止 (extract 同士)**: Pod 上の `extract_in_flight` フラグで in-flight 中の新規 trigger を skip。完了時点で閾値超過していれば直ちに次回を発火し、新 pointer 以降の最大範囲を回収するpending 状態は保持しない=完了時の閾値再評価で coalesce 相当の挙動を成立させる) - **並走防止 (extract 同士)**: Pod 上の `extract_in_flight` フラグで in-flight 中の新規 trigger を skip。完了時点で閾値超過していれば直ちに次回を発火し、新 pointer 以降の最大範囲を回収するpending 状態は保持しない=完了時の閾値再評価で coalesce 相当の挙動を成立させる)

View File

@ -179,7 +179,7 @@ pattern = "*.env"
action = "deny" action = "deny"
[compaction] [compaction]
prune_protected_turns = 3 prune_protected_tokens = 8000
prune_min_savings = 4096 prune_min_savings = 4096
compact_threshold = 80000 compact_threshold = 80000
compact_request_threshold = 90000 compact_request_threshold = 90000

View File

@ -0,0 +1,23 @@
# Spawned Pod 完了通知が来ない経路の疑い
## 観測
Spawned Pod が完了しているにもかかわらず、parent 側に完了通知が来ないことがある。`ReadPodOutput` の assistant text 抽出バグは修正済みだが、完了通知そのものの `PodEvent` delivery path に別の構造的リスクが見つかった。
## 現行の通知経路
child の parent-originated turn が `Finished` になると、child controller の `drive_turn` が parent socket へ `PodEvent::TurnEnded` を fire-and-forget する。
parent 側は `Method::PodEvent` を受けると side effect を適用し、`NotifyBuffer` に typed event を積む。parent が idle なら `RunForNotification(PodEvent)` が auto-kick され、interceptor が `SystemItem::PodEvent` として history に commit し、LLM request に通知が乗る。
## 疑い
送信 helper `connect_and_send` は socket に接続して Method を 1 行 write し、応答を読まずに close する。一方、受信側 `SocketServer::handle_connection` は Method を読む前に alert snapshot と `Event::Snapshot` を client に write する。
この組み合わせでは、send-only client が読まない / read half を保持しないため、server 側が snapshot write で失敗または詰まり、Method を読む前に connection handler が終わる可能性がある。これが起きると child の `PodEvent::TurnEnded` は parent controller に到達せず、NotifyBuffer にも入らない。
影響は `PodEvent` だけでなく、`StopPod` が child に送る `Method::Shutdown` など `connect_and_send` 利用箇所全般に及ぶ可能性がある。
## 対応
`tickets/pod-event-callback-delivery.md` を作成した。callback / fire-and-forget Method delivery を server の initial snapshot write に阻害されない形へ修正し、大きな snapshot を持つ parent に対しても `PodEvent::TurnEnded` が届く regression test を追加する。

View File

@ -0,0 +1,15 @@
# ReadPodOutput が LogEntry schema の分岐に取り残され no new assistant text になる
## 観測
spawned Pod のレビュー出力について、operator が attach すると assistant 出力が見える一方、spawner 側の `ReadPodOutput``pod ... running; no new assistant text` を返した。
## 原因
`crates/pod/src/spawn/comm_tools.rs``extract_assistant_text` が、`Event::Snapshot` 内の `LogEntry` を独自に解釈していた。session log の標準形は `LogEntry::AssistantItem { item, .. }` だが、`ReadPodOutput` 側の抽出対象が古い複数 item 形式に寄ったままになっていたため、新しい assistant 出力が snapshot に存在しても取りこぼした。
これは entry hash の問題ではなく、`LogEntry` に対する派生操作が各所で独自実装され、後方互換 variant が残ったことで標準形への追従漏れを型で検出できなかった問題。
## 対応
`LogEntry` から古い複数 item 形式の後方互換 variant を削除し、`ReadPodOutput` / TUI / picker / tests を現在の singular entry だけに揃える。これにより schema 変更時に独自 match の取り残しが compile error として表面化する。

10
docs/tui-parts.md Normal file
View File

@ -0,0 +1,10 @@
```
gap |
task(if some)| 8 tasks - pending: 2, inprogress:1, completed:5
|-----------------------------------------------------------
status |● insomnia idle 42.1k / 200k (21%)
input |>
actionbar | ↑ scrolled [normal]
```
status 右端は常に session context usage を `<tokens> / <window> (<pct>%)` 形式で表示する。mode / scrolled などの操作状態は actionbar に寄せる。

View File

@ -2,42 +2,51 @@
[[model]] [[model]]
id = "claude-sonnet-4-6" id = "claude-sonnet-4-6"
provider = "anthropic" provider = "anthropic"
context_window = 200000
[[model]] [[model]]
id = "claude-sonnet-4-5" id = "claude-sonnet-4-5"
provider = "anthropic" provider = "anthropic"
context_window = 200000
[[model]] [[model]]
id = "claude-opus-4-1" id = "claude-opus-4-1"
provider = "anthropic" provider = "anthropic"
context_window = 200000
# Ollama local (capability is router-ish / ollama handles its own models) # Ollama local (capability is router-ish / ollama handles its own models)
[[model]] [[model]]
id = "llama3.1" id = "llama3.1"
provider = "ollama-local" provider = "ollama-local"
context_window = 128000
[[model]] [[model]]
id = "qwen2.5-coder" id = "qwen2.5-coder"
provider = "ollama-local" provider = "ollama-local"
context_window = 128000
# Codex OAuth (ChatGPT backend via Responses API) # Codex OAuth (ChatGPT backend via Responses API)
[[model]] [[model]]
id = "gpt-5-codex" id = "gpt-5-codex"
provider = "codex-oauth" provider = "codex-oauth"
context_window = 400000
capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } } capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } }
[[model]] [[model]]
id = "gpt-5" id = "gpt-5"
provider = "codex-oauth" provider = "codex-oauth"
context_window = 400000
capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } } capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } }
# OpenRouter # OpenRouter
[[model]] [[model]]
id = "anthropic/claude-sonnet-4" id = "anthropic/claude-sonnet-4"
provider = "openrouter" provider = "openrouter"
context_window = 200000
capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "budget_tokens", vision = true, prompt_caching = { kind = "auto" } } capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "budget_tokens", vision = true, prompt_caching = { kind = "auto" } }
[[model]] [[model]]
id = "openai/gpt-5" id = "openai/gpt-5"
provider = "openrouter" provider = "openrouter"
context_window = 400000
capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } } capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } }

View File

@ -5,6 +5,7 @@ scheme = "anthropic"
base_url = "https://api.anthropic.com" base_url = "https://api.anthropic.com"
auth_hint = { kind = "api_key", env = "INSOMNIA_API_KEY_ANTHROPIC" } auth_hint = { kind = "api_key", env = "INSOMNIA_API_KEY_ANTHROPIC" }
default_capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "budget_tokens", vision = true, prompt_caching = { kind = "explicit", max_breakpoints = 4 } } default_capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "budget_tokens", vision = true, prompt_caching = { kind = "explicit", max_breakpoints = 4 } }
default_context_window = 200000
[[provider]] [[provider]]
id = "ollama-local" id = "ollama-local"
@ -13,6 +14,7 @@ scheme = "anthropic"
base_url = "http://localhost:11434" base_url = "http://localhost:11434"
auth_hint = { kind = "none" } auth_hint = { kind = "none" }
default_capability = { tool_calling = "parallel", structured_output = "json_schema", vision = false, prompt_caching = { kind = "auto" } } default_capability = { tool_calling = "parallel", structured_output = "json_schema", vision = false, prompt_caching = { kind = "auto" } }
default_context_window = 128000
[[provider]] [[provider]]
id = "codex-oauth" id = "codex-oauth"
@ -20,6 +22,7 @@ display_name = "ChatGPT (Codex OAuth)"
scheme = "openai_responses" scheme = "openai_responses"
auth_hint = { kind = "codex_oauth" } auth_hint = { kind = "codex_oauth" }
default_capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } } default_capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } }
default_context_window = 400000
[[provider]] [[provider]]
id = "openrouter" id = "openrouter"
@ -28,3 +31,4 @@ scheme = "openai_chat"
base_url = "https://openrouter.ai/api/v1" base_url = "https://openrouter.ai/api/v1"
auth_hint = { kind = "api_key", env = "INSOMNIA_API_KEY_OPENROUTER" } auth_hint = { kind = "api_key", env = "INSOMNIA_API_KEY_OPENROUTER" }
default_capability = { tool_calling = "parallel", structured_output = "json_schema", vision = true, prompt_caching = { kind = "auto" } } default_capability = { tool_calling = "parallel", structured_output = "json_schema", vision = true, prompt_caching = { kind = "auto" } }
default_context_window = 200000

View File

@ -1,188 +0,0 @@
# 永続化層のセマンティック整理
## 背景
現在の永続化は `SessionId` 単位の append-only JSONL log を中心に構成されている。これは実装上は扱いやすい一方で、今後 Pod 単位永続化、compaction、fork、DB backend 追加などを進めるにあたり、以下の概念が混ざり始めている。
- ユーザー視点の「同じ会話 / 作業の継続単位」
- Pod 視点の「現在 active な会話状態」
- append-only log の物理的 / 復元上の単位
- compaction によって生成される新しい履歴系列
- fork の起点となる履歴中の境界
- runtime dir に置かれる一時状態と、data dir / DB に置く永続正本
特に、現在は compaction によって新しい `SessionId` が発行される。これは append-only log の低レベル単位としては自然だが、ユーザー視点では「同じ会話が継続している」とも見えるため、`Session` という名称・粒度が今後の設計上あいまいになり得る。
このチケットでは、実装変更に入る前に、永続化層のドメイン概念・名称・責務境界を整理する。
## 目的
- 永続化層で扱う概念を、ユーザー視点 / Pod 視点 / storage 視点に分けて定義する。
- `SessionId` が今後も適切な中心概念か、あるいは別概念に分解すべきかを判断する。
- compaction / fork / resume / Pod state / spawned child registry が、どの粒度のデータに属するかを決める。
- 将来 DB backend を追加しても歪みにくいデータ構造を設計する。
- 既存の session-store JSONL 実装から段階的に移行できる命名・API 境界を決める。
## 結論: Session / Segment / Entry の 3 階層
```
Session ← ユーザー視点の会話の家系fork tree 全体。Segment 群の grouping
└── Segment ← Compaction / Fork で切れる物理 append-only 単位(現在の SessionId 相当)
└── Entry ← 1 永続化イベント
```
セマンティクスの対応:
- **resume** = 同 Session 内の指定 leaf Segment に append
- **compaction** = 同 Session, new Segmentlineage: `compacted_from`
- **fork** (live / 過去いずれも) = 同 Session, new Segmentlineage: `forked_from`
- **branching は Session 内で完結**する。Segment 間が DAG、Segment 内は完全 linear。
- **Session は分岐ツリーを持つ静的な構造**。「今どの Segment に書いているか」(current active Segment) は Pod state 側が動的に保持する。
文脈合成 (merge / cherry-pick) は今後の要件として想定しないため、entry レベルの DAG任意の親 pointerは採用しない。Segment 内 linear + Segment 間 DAG の 2 階層に閉じることで、同 Segment 内は `(segment_id, seq)` の連番 PK で直線探索でき、Segment 間の lineage は粗粒度な DAG として軽量に管理できる。
fork で新しい Session を切らないのは、compaction で Segment を切るのと fork で Segment を切るのが対称な操作であり、ユーザー視点でも「同じ起源から派生した枝」として 1 つの単位で扱う方が自然なため。これにより Session 数が分岐で爆発せず、`WHERE session_id = ?` だけで fork tree 全体が取れる。
### Restore / Resume の API
Session が分岐を含むため、resume の指定は **(SessionId, leaf SegmentId) の組** で行う:
- TUI 経路: ユーザーは **Session を選択 → その Session 内の leaf を選択**
- 内部 API: restore は `(SessionId, SegmentId)` を取り、指定 Segment から replay する。leaf 以外を指定すれば read-only な過去状態の参照も可能。
- 「最後に active だった leaf」は Pod state が保持するので、TUI が初期選択候補として使える。
### 既存コードとの対応
| 既存 | 新名称 |
|---|---|
| `SessionId` | `SegmentId` にリネーム |
| (なし) | `SessionId` 新設Segment 群をまとめる、ユーザー視点 ID |
`session-store` crate の型・関数は Segment 中心の命名に揃える。crate 名自体を変えるかは別論点(中身は引き続き Segment 単位の append-only log を扱う)。
### llm-worker への影響
llm-worker は session 概念を持たない(`Worker` は `history` / `turn_count` / `RequestConfig` を持つだけで永続化への hook も無い。Session / Segment 階層の導入は llm-worker 層に染み出さず、影響範囲は `session-store` / `pod` / `pod-registry` / `pod-cli` に閉じる。
## Entry hash の廃止
現状、各 entry は SHA-256 hash chain (`prev_hash` → `hash`) を持つが、実際に効いている用途は 2 つだけ:
1. `ensure_head_or_fork` (pod.rs:1348) — store の末尾と Pod の保持する `head_hash` を比較し、不一致なら auto-fork。**末尾識別子があれば良い**hash chain そのものは要らない)。
2. `fork_at(source_id, at_hash)` (session.rs:425) — 過去 entry pointer から fork。`pod-session-fork.md` の入口仕様は turn number で、entry hash は内部 pointer に過ぎない。turn boundary (TurnEnd entry の index) で代替可能。
改竄検知 (tamper-evident chain) は宣伝されているがコード上は walk して verify するルートが無いため、削除しても regression にならない。
廃止に伴う対応:
- `HashedEntry` 廃止、JSONL は 1 行 1 `LogEntry`
- `SessionOrigin.at_hash``at_turn_index` (TurnEnd 由来) に置換。
- `ensure_head_or_fork` の検知ロジックは、Segment 末尾の terminal marker entry または末尾 seq 比較に置換(形式は実装時に決める)。
### 廃止前の足場 (前提)
本セクションを実装に移すタイミングでは、log writer が既に sync 化されていることを前提にする (`tickets/log-entry-singular-and-direct-commit.md`)。具体的には:
- `Store::append` / `read_all` 等が `std::fs` ベースの sync API
- `SessionLogWriter::append_entry()` が sync 関数
- `session_head` mutex は `parking_lot::Mutex` / `std::sync::Mutex`
- `LogCommand` / drain task / Flush バリアは既に撤廃済み
この状態で hash chain を廃止すると追加で取れる単純化:
- **`session_head` mutex そのものを撤去できる**。 hash chain が無いので「`head_hash` を直前 entry から取得して次に渡す」 という serialize 必須の依存が消える。 1 行 < `PIPE_BUF` (Linux 4KB) の `O_APPEND` write は kernel 側で atomic に直列化されるので、 user space で mutex を持つ必要が無い
- `session_head` が消えると Pod / interceptor / worker callback が writer ハンドルだけ持てば良くなる。 `Arc<SessionLogWriter>` は単に `Arc<Store> + sink` を抱えるだけの値で、 hot-path の競合がない
- `compute_hash` 呼び出しが消える分、 append が serialize + open + write + close の 3 syscall まで詰まる
つまり「sync 化」 が先に来て、 「hash 廃止」 で mutex まで消える、 という 2 段階の単純化になる。
## Fork: 2 種類の書き込み方
Session 境界の話ではなく **元 Segment への marker 書き込みの有無**で 2 種類を分ける。Session はどちらの場合も同じで、新 Segment が同 Session 内に生える。
- **live auto-fork**concurrent writer 検知)
- 元 Segment の末尾に terminal marker (`Forked { to: SegmentId }` 等) を append → 以降の writer は marker を見て新 Segment へ自動移動。
- CoW semantics: 元 Segment は immutable、生まれた Segment 同士は対等な兄弟。
- **過去 fork**UI で turn 選択)
- 元 Segment は **無変更**、replay して新 Segment を生やすだけ。
- 起点は `(source_segment_id, at_turn_index)`
- 元 Segment に書き込まないため、過去 fork を nested に重ねても解釈が単純。
両者を同じ marker で扱おうとすると、過去 fork から更に過去で fork した場合に元 Segment への marker 位置解釈が複雑化して破綻するため、過去 fork 側は元 Segment に触れない方針で固定する。
### fork の物理操作(物理コピーは不要)
「fork = 同 Session 内に Segment を増やす」と言っても、過去 Segment を丸ごとコピーする必要は無い:
- source segment の seq=N までを replay して得られた `Vec<Item>` を、新 Segment の seed entry (`SessionStart` 相当) に 1 回書き込む。
- compaction の transitive な圧縮効果がここに乗るsource segment の `SessionStart.history` に過去 Segment の compaction 結果が既に埋まっている)ので、**source segment 1 本だけ replay すれば fork seed が作れる**。さらに過去の Segment は touch しない。
- 新 Segment が持つ lineage 情報は `(parent_segment_id, fork_at_turn)` のメタデータのみ。
これは現状の `fork_at` (`session.rs:430-456`) の挙動と同じで、Session 階層が乗っても操作は変わらない。
## RDB backend を想定した概念モデル
将来 DB backend を追加しても歪みにくい形として、以下の関係を仮定する:
```
sessions (
id PK,
... -- ユーザー視点 metadata
)
segments (
id PK,
session_id FK,
parent_segment_id FK NULL, -- compaction / fork の元
fork_at_turn INT NULL,
origin_kind ENUM (new | compact | fork),
lineage_path ltree, -- 祖先・子孫の逆引き用 (materialized path)
...
)
entries (
segment_id FK,
seq INT,
kind, payload, ts,
PRIMARY KEY (segment_id, seq) -- 同 Segment 内 linear scan
)
```
- 同 Segment 内 entry は `(segment_id, seq)` PK で linear scan、surrogate identity なので hash 不要。
- Segment 間 lineage は `parent_segment_id` chain。深さは compaction / fork 回数のみSession あたり数〜数十)。
- Segment lineage の祖先・子孫逆引きは `lineage_path``<@` / `@>` で 1 index 引き。entry 単位の DAG ではないため materialized path のメンテコストも軽い。
- 通常 append は `lineage_path` 更新を伴わないSegment 生成時に確定)。
- 「Session の fork tree 全体」は `WHERE session_id = ?` で取れる。Session 単位の listing / GC が自然。
FsStore 実装はこの構造のサブセット相当として位置付ける1 Segment = 1 jsonl、`session_id` は Segment の metadata に持たせるか別ファイルに index する)。
## 残る検討事項
- pod.scope extension entry を Pod state 側に寄せるか、Segment log に残すか(`tickets/pod-persistent-state.md` 側と合わせて決定)。
- 撤廃の選択肢: (a) Segment log から削除し Pod state を唯一の正本にする / (b) snapshot 保持責務だけ Pod state に寄せ、scope 変更 event は Segment log に残す / (c) 現状維持で Pod state は Segment への参照のみ。
- live auto-fork の marker 形式 (terminal entry vs Segment 末尾 seq 比較)。
- pod-cli / TUI の `--session` 引数を Session 単位にするか Segment 単位にするか。debug 用 ID とユーザー向け ID の分離。
- `session-store` crate 名のリネーム要否。
## 完了条件
- 永続化層の主要概念と名称が文書化されている。
- compaction / fork / resume / Pod state のデータ粒度が決まっている。
- 現在の `SessionId` / session-store API をどう扱うか、維持・alias・rename・段階移行の方針が決まっている。
- DB backend を追加する場合の概念モデルが、最低限テーブル / relation 相当で説明できる。
- `tickets/pod-persistent-state.md` や fork 関連チケットに反映すべき前提が整理されている。
## 範囲外
- このチケット単体での大規模 rename 実装。
- DB backend の実装。
- UI の履歴表示 / branch 表示の詳細 UX。
- GC / retention policy の実装。
- Session を跨ぐ merge / cherry-pick。
## 関連
- `tickets/pod-persistent-state.md`
- `tickets/pod-session-fork.md`
- `crates/session-store/`
- `crates/pod/src/pod.rs`

View File

@ -0,0 +1,78 @@
# Pod: 過去 Pod の探索と restore ツール
## 背景
Pod state の永続化と `--pod <name>` resume が入ったことで、名前が分かっている Pod は復元できるようになった。一方で、AI / operator が「過去にどんな Pod があったか」「この名前の Pod は復元できるか」「live attach できるのか、restore が必要なのか」を機械的に調べる導線はまだない。
現在の `ListPods` は主に spawner が知っている spawned child の live/runtime registry を見るためのツールであり、永続化された全 Pod の探索や、過去 Pod の restore 導線としては不十分。今後 Pod 単位で作業を再開する運用を成立させるには、Pod state を正本として過去 Pod を列挙・確認・復元できる tool surface が必要。
ただし、ホスト上の全 Pod state がどの Pod / LLM からも見える設計にはしない。Pod 管理 tool は capability / visibility scope を持ち、呼び出し元が見る権限を持つ Pod だけを列挙・操作できる必要がある。
## 要件
- 永続化された Pod state から、可視性 scope 内の既知 Pod 一覧を取得する tool / protocol API を追加する。
- 実際の Method / tool 名は実装時に確定する。
- `session-store` の Pod state backend/FsStore を正本にし、runtime dir の `spawned_pods.json` を正本にしない。
- state が壊れている Pod や active segment 未確定の Pod は、全体失敗ではなく item 単位の状態として返せるようにする。
- 呼び出し元に可視性がない Pod は列挙結果に含めない。
- Pod 可視性の制御を設計する。
- 少なくとも「現在の parent が spawn した child」と「明示的に指定された Pod 名」は扱えるようにする。
- ホスト上の全 Pod を無条件に返す admin/global tool にはしない。
- visibility の根拠は Pod state / parent-child registry / manifest capability / explicit user selection のいずれかに寄せ、実装時に確定する。
- 可視でない Pod に対する detail / restore / attach は not visible / forbidden として、state missing とは区別する。
- 一覧 item には最低限以下を含める。
- `pod_name`
- active `SessionId` / `SegmentId`(未確定ならその状態)
- live socket / runtime が到達可能かどうか
- restore 可能かどうかと、restore に必要な名前
- spawned children が永続化されている場合は、その概要(件数や reachable 状態。詳細展開は別 API でもよい)
- Pod 名指定で詳細を取得できる API を用意する。
- active pointer
- restoreability
- live attach 可能性
- spawned child registry の概要
- 読めない state / 消えた socket / lock 衝突を区別したエラー
- Pod 名指定で restore / attach を開始できる tool 導線を用意する。
- live socket が到達可能なら attach 相当の扱いにする。
- 到達不能だが Pod state があるなら既存の `--pod <name>` / `Pod::restore_from_pod_metadata(...)` 経路で restore する。
- Pod state が存在しない名前を指定した場合に新規 Pod を作るか、明示エラーにするかは API ごとに曖昧にせず決める。探索・復元ツールとしては、意図しない新規作成を避けるため default はエラー寄りが望ましい。
- 既存の `--pod <name>` / `--session <UUID>` / spawned child 向け `ListPods` / `SendToPod` / `StopPod` と責務を混ぜない。
- `ListPods` は現在接続中の spawned child registry を見る用途として維持してよい。
- 過去 Pod の探索 API は Pod state を正本にする。
- live writer 二重起動防止、scope delegation、session lock の責務は既存 registry / lock に任せ、Pod state に lock 責務を追加しない。
- tool result として LLM に返す情報は通常の tool call 履歴に残る形にし、history に残らない context 差し込みで実現しない。
## 完了条件
- 永続化済み Pod のうち、呼び出し元の可視性 scope 内にあるものだけを Pod state から列挙できる。
- runtime dir の `spawned_pods.json` が存在しない状態でも、Pod state から可視 Pod を探索できる。
- Pod 名指定で詳細を取得し、live attach 可能 / restore 可能 / state 不在 / state 破損 / lock 衝突を区別できる。
- Pod 名指定の restore / attach tool が、到達可能 live Pod には attach し、到達不能だが state がある Pod には既存 restore 経路で復元できる。
- 既存の `ListPods` / `SendToPod` / `StopPod` / `--pod` / `--session` の挙動を壊さない。
- unit / integration test で以下を確認する。
- 複数 Pod metadata の列挙(可視 Pod のみ)
- 可視でない Pod が列挙されず、detail / restore / attach でも state missing と区別されること
- active segment 未確定 Pod の表示
- runtime file が消えても Pod state から探索できる
- socket 到達可否の反映
- restore / attach の分岐
- lock 衝突時に二重 writer を起動しない
- `cargo fmt --check`
- `cargo check --workspace`
- 関連 crate の tests少なくとも `cargo test -p pod`。tool surface を置く crate に応じて追加)
## 範囲外
- 過去 Pod 一覧の本格 UI / pickerTUI 側の Pod picker は `tickets/tui-pod-restore-picker.md` で扱う)
- fork tree の可視化
- transcript 全文検索 / semantic search
- Pod の自動再起動
- 古い Pod state の GC / retention policy
- session / segment 単位の新しい resume 引数
## 依存 / 関連
- Pod state backend / FsStore 実装
- Pod lifecycle write-through
- Pod 名単位の resume / attach 導線
- SpawnedPodRegistry の永続化と復元

View File

@ -1,84 +0,0 @@
# Pod: セッションログをバックエンドにした Pod 単位の永続化
## 背景
現在の永続化の主軸は session-store の append-only JSONL ログで、`SessionId` 単位に会話履歴・設定・scope snapshot・usage・拡張 payload を復元できる。一方で Pod 単位のランタイム状態は `<runtime_dir>/{pod_name}/` 配下の `status.json` / `history.json` / `spawned_pods.json` などに write-through されているが、runtime dir は再起動で消えてよい領域であり、Pod プロセスの寿命を超える復元ソースとしては扱えない。
特に spawned Pod の管理情報は `SpawnedPodRegistry` のコメントにもある通り、現状は runtime dir への write-through のみで、再起動した spawner が子 Pod 一覧を rebuild する future work になっている。
このチケットでは、既存の session-store を物理バックエンドとして利用しつつ、Pod 名をキーにした永続状態を追加し、Pod 単位で「最後にどの session を保持していたか」「spawned children をどう復元するか」を扱えるようにする。
## 方針
- session log は引き続き会話状態の唯一の復元ソースにする。
- `history.json` や runtime dir の snapshot を永続正本にはしない。
- LLM context に載せる新規 input は、既存方針通り先に worker history / session log に commit されている必要がある。
- Pod 単位の永続化は「Pod identity → session / child registry などへの参照」を保存する薄いメタデータ層として設計する。
- 会話本文を二重保存しない。
- active session だけでなく、compaction / fork / resume によってその Pod が辿ってきた過去 session を順序付きで保持する。これは UI の履歴表示、直近以前への復元、active session 変更の監査に使う。
- session-store の `Store` trait を拡張するか、隣接 trait / module を追加して、FsStore 以外の backend でも同じ形で実装できるようにする。
- FsStore のデフォルト layout は `<data_dir>/pods/` 配下など、`sessions/` と同じ data_dir 管理下に置く。
- runtime dir (`<runtime_dir>/{pod_name}/`) は引き続き socket / pid / status など一時状態専用。
- Pod lifecycle 上の write point を明確にする。
- Pod 作成時: pod name と allocated session id を記録。
- first run で `SessionStart` が materialize された後: active session / head を更新できる状態にする。
- compaction / fork / resume で active session が変わる場合: Pod state も同時に更新。
- `SpawnPod` / callback / `StopPod` による child registry 変更時: runtime dir だけでなく persistent Pod state にも write-through。
- 復元時は Pod state から active session を解決し、その session log を `restore_from_manifest` 相当の経路で復元する。
- session id を明示した resume は既存通り session を直接指定できる。
- Pod 名 resume は Pod state → active session → session restore の順に解決する。
- live writer 衝突は既存の pod-registry / session_id collision check を維持する。
## データ粒度の考え方
- ユーザー視点の会話継続単位と、内部の append-only log 単位を分けて扱う。
- ユーザー視点: Pod / thread / conversation のような安定 ID。compaction しても同じ会話として継続する。
- 内部 log 視点: session segment / revision / epoch のような履歴再構築単位。compaction や fork で新しい log root が必要なら新 ID になる。
- 現状の `SessionId` は内部 log 単位の性質が強い。compaction は履歴を要約済み prefix に置き換えて新しい append-only chain を始めるため、低レベルには「新 session」として扱うのは自然。ただし UX / データモデル上は「同じ Pod conversation の新 revision」と見せる。
- 将来 DB backend を追加する場合も、`Conversation/PodState` と `SessionSegment` を分ける形に寄せる。
- `pod_state.active_session_id` は現在 append 先の segment を指す。
- `pod_state.session_history[]` は Pod 視点で active だった segment の順序付き履歴。
- compaction / fork の構造的 lineage は session log の `SessionOrigin` または DB の relation として保持し、Pod state は「この Pod がどれを active にしたか」の操作履歴に留める。
## 要件
- Pod 名をキーに、少なくとも以下を永続化できること:
- active `SessionId`
- ordered session history: その Pod が active として保持してきた `SessionId` の時系列リスト
- 各 entry には最低限 `session_id` と遷移理由new / resume / compact / fork など)を持たせる
- compaction / fork の構造的な出自は session log の `SessionOrigin` を正本とし、Pod state 側は Pod 視点の active session 遷移履歴として扱う
- Pod manifest / scope 復元に必要な参照または snapshot の扱い(既存 session log の `pod.scope` snapshot と責務を重複させない)
- spawned children の registrypod name, socket path, delegated scope, callback address, child session id が必要なら含める)
- `SpawnedPodRegistry` が runtime dir の `spawned_pods.json` だけでなく、Pod 永続状態から初期化できること。
- `ListPods` / `SendToPod` / `ReadPodOutput` / `StopPod` は、復元後の spawner でも永続化された child registry を基に動作できること。
- ただし `ReadPodOutput` の read cursor は session-lifetime / in-memory のままでよい。永続化対象にしない。
- Pod の compaction により active session id が変わった場合、Pod 永続状態と pod-registry の session id が整合すること。
- 既存の `--session <UUID>` resume は壊さない。
- 新しい Pod 名単位 resume / attach の入口を決めること。
- 例: `pod --pod-state <name>` ではなく、既存 `pod.name` と manifest cascade から同名 Pod state を探す形など。
- CLI / TUI の最小導線を本チケット内で確定する。
## 完了条件
- `session-store` に Pod 単位メタデータを扱う backend API と FsStore 実装がある。
- Pod state が active session と ordered session history を保持し、new / resume / compaction / fork の遷移が順序付きで記録される。
- 新規 Pod 起動、resume、compaction、spawn / stop の各タイミングで Pod 永続状態が更新される。
- Pod プロセス再起動後、Pod 名から active session を復元し、会話を継続できる。
- spawner Pod の再起動後、永続化された spawned children 一覧から `ListPods` が復元され、到達可能な child に対して comm tools が使える。
- runtime dir は引き続き一時状態として扱われ、永続正本に依存しない。
- live writer の二重起動は既存 pod-registry / session lock と同等以上に防止される。
## 範囲外
- 会話履歴そのものの保存形式変更。
- session log の DB 化や remote backend 実装。
- Pod state の自動 GC / retention policy。
- TUI 上の高度な Pod 一覧 UI。最小限の resume / attach 導線を超える UX は別チケット。
- `ReadPodOutput` cursor の永続化。
## 関連
- `crates/session-store/`: 既存の session append-only backend。
- `crates/pod/src/runtime/dir.rs`: runtime dir の `history.json` / `spawned_pods.json`
- `crates/pod/src/spawn/registry.rs`: spawned children registry。現状は write-through のみで復元未実装。
- `tickets/pod-session-fork.md`: active session 切り替え設計との整合が必要。

Some files were not shown because too many files have changed in this diff Show More