Compare commits
66 Commits
be5e413b55
...
3f7de349c3
| Author | SHA1 | Date | |
|---|---|---|---|
| 3f7de349c3 | |||
| 1feb560ff9 | |||
| 902083bd38 | |||
| 221b1edd92 | |||
| 828004a5e2 | |||
| b26bc420f1 | |||
| fb7abb1b7c | |||
| 66996f902b | |||
| d62cd09c4d | |||
| a4f03e7688 | |||
| 5ade50dec5 | |||
| dbb6cca894 | |||
| 7c9abb37ad | |||
| 0535260c8a | |||
| c435635e5b | |||
| c78cd28b27 | |||
| abe890cda5 | |||
| fa04b03643 | |||
| 1bebd8b6df | |||
| 4dec7f916f | |||
| 8662ca404f | |||
| 5f2efeb75e | |||
| 820dea1873 | |||
| 8d5ee0f0b8 | |||
| 0d39170bbe | |||
| 16963d15f2 | |||
| 2cea02648f | |||
| f55503edc3 | |||
| d18e3a0256 | |||
| 5ba4be1c9b | |||
| 90d4c8f5ad | |||
| 61c9719da5 | |||
| 73fbdcc025 | |||
| 5fdc46db47 | |||
| 534c6f4cac | |||
| 5540ca3d0e | |||
| edfdca3457 | |||
| bd32f704b4 | |||
| e55fc9a834 | |||
| 7f6e3b949f | |||
| 0954a4804a | |||
| e9cc4b90dc | |||
| 8aca67c97c | |||
| d7eabb18c9 | |||
| b13c2735bb | |||
| 67f135fbc6 | |||
| a728b7045d | |||
| 3eabcb6a6d | |||
| ffcba3aa54 | |||
| eb752fb295 | |||
| ac3ee5fcbe | |||
| 46b0e20685 | |||
| 6a4ee37be8 | |||
| 3f3ead3b71 | |||
| a6cc05f74c | |||
| e4cda5d3f2 | |||
| dd9abfee2e | |||
| d7ff25b6a7 | |||
| 7577577c9f | |||
| 0d7c37f673 | |||
| d5c7330659 | |||
| 9c1f51b4f0 | |||
| 1d8a490504 | |||
| 6e791d8668 | |||
| d5dff6d17b | |||
| 35c15923db |
5
Cargo.lock
generated
5
Cargo.lock
generated
|
|
@ -2148,6 +2148,7 @@ checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
|
|||
name = "pod"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"arc-swap",
|
||||
"async-trait",
|
||||
"chrono",
|
||||
"clap",
|
||||
|
|
@ -2160,7 +2161,6 @@ dependencies = [
|
|||
"manifest",
|
||||
"memory",
|
||||
"minijinja",
|
||||
"parking_lot",
|
||||
"pod-registry",
|
||||
"protocol",
|
||||
"provider",
|
||||
|
|
@ -2977,12 +2977,10 @@ version = "0.1.0"
|
|||
dependencies = [
|
||||
"async-trait",
|
||||
"futures",
|
||||
"hex",
|
||||
"llm-worker",
|
||||
"protocol",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2 0.11.0",
|
||||
"tempfile",
|
||||
"thiserror 2.0.18",
|
||||
"tokio",
|
||||
|
|
@ -3655,6 +3653,7 @@ dependencies = [
|
|||
"serde",
|
||||
"serde_json",
|
||||
"session-store",
|
||||
"tempfile",
|
||||
"tokio",
|
||||
"toml",
|
||||
"tools",
|
||||
|
|
|
|||
|
|
@ -31,7 +31,6 @@ llm-worker-macros = { path = "crates/llm-worker-macros", version = "0.2" }
|
|||
manifest = { path = "crates/manifest" }
|
||||
lint-common = { path = "crates/lint-common" }
|
||||
memory = { path = "crates/memory" }
|
||||
workflow = { path = "crates/workflow" }
|
||||
pod-registry = { path = "crates/pod-registry" }
|
||||
protocol = { path = "crates/protocol" }
|
||||
provider = { path = "crates/provider" }
|
||||
|
|
|
|||
|
|
@ -14,3 +14,5 @@ Ticket を切るほどではないが、次に近所を触るときに合わせ
|
|||
- `crates/tui/src/app.rs:478-485` — bad workflow slug を含む `Method::Run` 送信時、`Event::UserMessage` の早期 broadcast で `turn_index += 1` されターンヘッダだけ残る ("ghost turn header")。次に TUI のターンヘッダ / エラー表示周りを触るときに整理。→ [tickets/pod-input-validate-internalize.md] の review 由来。
|
||||
- `crates/pod/src/controller.rs:944` — `worker_error_code` で `PodError::WorkflowResolve(_) => InvalidRequest` が post-commit な resolve エラー (`KnowledgeNotFound` 等) にも適用される。意味論的には妥当方向だが、resolve 系のエラー粒度を分けたくなったタイミングで再評価。
|
||||
- `crates/pod/tests/controller_test.rs` — `double_run_returns_error` がたまに失敗する flakiness を観測。`pod-interrupt-prep-internalize` 以前から存在する別件。次に controller_test の Run 連投系のタイミングを触るときに併せて原因を切り分け。
|
||||
- `crates/session-store/src/fs_store.rs:117-122` — `FsStore::read_entry_count` が `fs::read_to_string` で全文ロードしてから行数カウントするため O(n)。`ensure_head_or_fork` は run-start でしか呼ばれず現状は許容範囲だが、長期セッションが普通になった時点で `\n` バイト数の cheap count か末尾 seek に置き換える。
|
||||
- `crates/session-store/src/segment.rs:121` `ensure_head_or_fork` (free fn, test 専用・本番 caller ゼロ) と `crates/pod/src/pod.rs` `Pod::ensure_segment_head` (本番 inline) に live auto-fork の検知 + forked_from 記録が二重実装されている。entry-hash-abolish 以前からの重複で、両方独立にテスト済みだが drift 必至。session-store 側を本番から呼ぶ形に寄せるか free fn を畳むかは要設計判断。Pod state / fork 周辺を次に触るときに統合を検討。
|
||||
|
|
|
|||
5
TODO.md
5
TODO.md
|
|
@ -7,8 +7,7 @@
|
|||
- Pod: 空応答ターン (Submit 後 AI 応答ゼロで Pause/Cancel) を自動巻き戻し → [tickets/pod-empty-turn-rollback.md](tickets/pod-empty-turn-rollback.md)
|
||||
- Pod: 任意ターンからの Fork(複数ターン巻き戻しを汎用化) → [tickets/pod-session-fork.md](tickets/pod-session-fork.md)
|
||||
- Pod: Inbound PodEvent ハンドリングの重複を統合 → [tickets/pod-inbound-pod-event-dedup.md](tickets/pod-inbound-pod-event-dedup.md)
|
||||
- Pod: セッションログをバックエンドにした Pod 単位の永続化 → [tickets/pod-persistent-state.md](tickets/pod-persistent-state.md)
|
||||
- 永続化層のセマンティック整理 → [tickets/persistence-semantics.md](tickets/persistence-semantics.md)
|
||||
- Pod: 過去 Pod の探索と restore ツール → [tickets/pod-discovery-restore-tools.md](tickets/pod-discovery-restore-tools.md)
|
||||
- llm-worker のエラー耐性
|
||||
- ストリーム途中失敗時の継続 → [tickets/llm-worker-stream-continuation.md](tickets/llm-worker-stream-continuation.md)
|
||||
- llm-worker: history append を callback 経由の単一経路に閉じる → [tickets/worker-history-append-contract.md](tickets/worker-history-append-contract.md)
|
||||
|
|
@ -21,8 +20,6 @@
|
|||
- ユーザーマニフェストのモデル設定 wizard → [tickets/tui-user-model-setup.md](tickets/tui-user-model-setup.md)
|
||||
- spawn 失敗時に Pod の stderr が TUI に表示されない → [tickets/tui-spawn-error-surface.md](tickets/tui-spawn-error-surface.md)
|
||||
- 巻き戻されたターンの入力テキストを編集領域に復元 → [tickets/tui-empty-turn-restore.md](tickets/tui-empty-turn-restore.md)
|
||||
- セッションコンテキスト長 / ウィンドウ占有率の常時表示 → [tickets/tui-context-usage-indicator.md](tickets/tui-context-usage-indicator.md)
|
||||
- Prune: 保護境界を turn 数から末尾 token budget に置き換え → [tickets/prune-token-budget.md](tickets/prune-token-budget.md)
|
||||
- メモリ機構
|
||||
- extract / consolidation 監査ログ → [tickets/memory-audit-log.md](tickets/memory-audit-log.md)
|
||||
- セッション内 Task ツールの注意機構(無アクティビティで `<system-reminder>` ナッジ) → [tickets/session-todo-reminder.md](tickets/session-todo-reminder.md)
|
||||
|
|
|
|||
|
|
@ -34,6 +34,9 @@ pub struct SpawnConfig {
|
|||
/// `Some(id)` のとき `--session <id>` を付与し、当該セッションから
|
||||
/// resume させる。
|
||||
pub resume_from: Option<Uuid>,
|
||||
/// true のとき `--pod <pod_name>` を付与し、pod 側で name-keyed state
|
||||
/// があれば resume、なければ同名の新規 Pod として起動させる。
|
||||
pub resume_by_pod_name: bool,
|
||||
}
|
||||
|
||||
pub struct SpawnReady {
|
||||
|
|
@ -111,6 +114,9 @@ where
|
|||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::from(stderr_file))
|
||||
.process_group(0);
|
||||
if config.resume_by_pod_name {
|
||||
command.arg("--pod").arg(&config.pod_name);
|
||||
}
|
||||
if let Some(id) = config.resume_from {
|
||||
command.arg("--session").arg(id.to_string());
|
||||
}
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ pub(crate) struct ResponsesRequest {
|
|||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// 会話単位の安定キー。ChatGPT backend (codex-oauth) は明示キーが
|
||||
/// 無いとプロンプトキャッシュがほぼ効かない。pod 側は `SessionId`
|
||||
/// 無いとプロンプトキャッシュがほぼ効かない。pod 側は `SegmentId`
|
||||
/// を渡す。`Request::cache_key` が `None` のときはキー自体を送らない。
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub prompt_cache_key: Option<String>,
|
||||
|
|
|
|||
|
|
@ -492,7 +492,7 @@ pub struct Request {
|
|||
/// 会話単位の安定キー。`prompt_cache_key` として送られる
|
||||
/// (OpenAI Responses)。ChatGPT backend (codex-oauth) は明示キーが
|
||||
/// 無いと org/project ハッシュ衝突でプロンプトキャッシュが
|
||||
/// ほぼヒットしないため、pod 側で `SessionId` を渡す運用を想定。
|
||||
/// ほぼヒットしないため、pod 側で `SegmentId` を渡す運用を想定。
|
||||
/// `cache_anchor` と違い名前空間キーであり、`prefix anchor` とは
|
||||
/// 別の概念。`cache_anchor` を読まない provider と同じく、
|
||||
/// `prompt_cache_key` を持たない provider は無視する。
|
||||
|
|
|
|||
|
|
@ -11,12 +11,23 @@
|
|||
//! 射影の適用は上位層(`pod::prune_hook` 等)が LLM に送る一時コンテキスト
|
||||
//! に対してだけ行う。Worker の永続履歴は決して変更されない。
|
||||
//!
|
||||
//! `min_savings` 判定や savings 推定もこの crate には置かず、上位層が
|
||||
//! usage 履歴ベースのトークン会計と組み合わせて行う。
|
||||
//! 保護境界は末尾 token budget で決めるが、この crate は usage 履歴を
|
||||
//! 所有しない。prefix ごとの token 推定値と savings 推定は上位層から
|
||||
//! callback で注入される。
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::llm_client::types::Item;
|
||||
use crate::token_counter::{EstimateSource, TokenEstimate};
|
||||
|
||||
/// Callback that returns token estimates for every prefix boundary of the
|
||||
/// supplied request history.
|
||||
///
|
||||
/// The returned slice must have `history.len() + 1` entries where entry `i`
|
||||
/// estimates the token count of `history[..i]`. Returning a malformed vector,
|
||||
/// or estimates whose source is [`EstimateSource::NoData`], makes prune treat
|
||||
/// the request as having no candidates.
|
||||
pub type TokenEstimator = Box<dyn Fn(&[Item]) -> Vec<TokenEstimate> + Send + Sync>;
|
||||
|
||||
/// Callback that estimates the token savings for projecting the
|
||||
/// `ToolResult.content` out of `history[i]` for each `i` in `indices`.
|
||||
|
|
@ -35,16 +46,16 @@ pub type SavingsEstimator = Box<dyn Fn(&[Item], &[usize]) -> u64 + Send + Sync>;
|
|||
///
|
||||
/// Worker は LLM リクエストごとに 1 回 prune の評価をし、その結果を
|
||||
/// (observer が登録されていれば)この値で通知する。fire/skip の判定
|
||||
/// 結果と、判定材料になった候補数 / 推定 savings / 境界ターン位置を持つ。
|
||||
/// 結果と、判定材料になった候補数 / 推定 savings / 保護領域の先頭 index を持つ。
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PruneEvaluation {
|
||||
/// `prunable_indices` の長さ。`Skipped::NoCandidates` の時は 0。
|
||||
pub candidate_count: usize,
|
||||
/// 推定された savings (tokens)。`NoCandidates` の時は 0。
|
||||
pub estimated_savings: u64,
|
||||
/// `protected_turns` 境界に当たる turn-start アイテムの index。
|
||||
/// turn 数が `protected_turns` 以下で境界が決まらない場合は `None`。
|
||||
pub border_turn: Option<usize>,
|
||||
/// Token budget で保護される suffix の先頭 item index。
|
||||
/// usage 推定が `NoData` で境界が決まらない場合は `None`。
|
||||
pub protected_start_index: Option<usize>,
|
||||
/// 判定結果。
|
||||
pub decision: PruneDecision,
|
||||
}
|
||||
|
|
@ -70,10 +81,9 @@ pub type PruneObserver = Box<dyn Fn(&PruneEvaluation) + Send + Sync>;
|
|||
/// Configuration for the Prune algorithm.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct PruneConfig {
|
||||
/// Number of recent turns to protect from pruning.
|
||||
/// A "turn" starts at each user message.
|
||||
#[serde(default = "default_protected_turns")]
|
||||
pub protected_turns: usize,
|
||||
/// Token budget at the history tail protected from pruning.
|
||||
#[serde(default = "default_protected_tokens")]
|
||||
pub protected_tokens: u64,
|
||||
|
||||
/// Minimum token savings required to actually prune. If the prunable
|
||||
/// content is smaller than this, the caller should skip to avoid
|
||||
|
|
@ -84,8 +94,8 @@ pub struct PruneConfig {
|
|||
pub min_savings: u64,
|
||||
}
|
||||
|
||||
fn default_protected_turns() -> usize {
|
||||
3
|
||||
fn default_protected_tokens() -> u64 {
|
||||
8000
|
||||
}
|
||||
fn default_min_savings() -> u64 {
|
||||
4096
|
||||
|
|
@ -94,25 +104,12 @@ fn default_min_savings() -> u64 {
|
|||
impl Default for PruneConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
protected_turns: default_protected_turns(),
|
||||
protected_tokens: default_protected_tokens(),
|
||||
min_savings: default_min_savings(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Find indices where each "turn" begins.
|
||||
///
|
||||
/// A turn starts at every user message. Returns the indices of those
|
||||
/// user messages in ascending order.
|
||||
fn find_turn_starts(items: &[Item]) -> Vec<usize> {
|
||||
items
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, item)| item.is_user_message())
|
||||
.map(|(i, _)| i)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Set `content = None` on each `Item::ToolResult` at the given indices.
|
||||
///
|
||||
/// Returns the number of items that were actually modified — items that
|
||||
|
|
@ -121,36 +118,43 @@ fn find_turn_starts(items: &[Item]) -> Vec<usize> {
|
|||
pub fn project(items: &mut [Item], indices: &[usize]) -> usize {
|
||||
let mut count = 0;
|
||||
for &i in indices {
|
||||
if let Item::ToolResult { content, .. } = &mut items[i] {
|
||||
if content.is_some() {
|
||||
if let Item::ToolResult { content, .. } = &mut items[i]
|
||||
&& content.is_some()
|
||||
{
|
||||
*content = None;
|
||||
count += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
count
|
||||
}
|
||||
|
||||
/// Indices of `Item::ToolResult { content: Some(_), .. }` that lie outside
|
||||
/// the last `protected_turns` turns. Pure: does not mutate `items`.
|
||||
/// Indices of `Item::ToolResult { content: Some(_), .. }` that lie before
|
||||
/// the suffix protected by `protected_tokens`. Pure: does not mutate `items`.
|
||||
///
|
||||
/// Returns an empty vector when there are too few turns or no prunable
|
||||
/// candidates.
|
||||
pub fn prunable_indices(items: &[Item], protected_turns: usize) -> Vec<usize> {
|
||||
evaluate_candidates(items, protected_turns).0
|
||||
/// Returns an empty vector when token estimates are unavailable (`NoData`) or
|
||||
/// no prunable candidates exist.
|
||||
pub fn prunable_indices(
|
||||
items: &[Item],
|
||||
protected_tokens: u64,
|
||||
token_estimates: &[TokenEstimate],
|
||||
) -> Vec<usize> {
|
||||
evaluate_candidates(items, protected_tokens, token_estimates).0
|
||||
}
|
||||
|
||||
/// Same as [`prunable_indices`] but also returns the index of the
|
||||
/// `protected_turns` boundary (the turn-start item whose tail is
|
||||
/// protected). `None` when too few turns exist for a boundary to be
|
||||
/// defined.
|
||||
pub fn evaluate_candidates(items: &[Item], protected_turns: usize) -> (Vec<usize>, Option<usize>) {
|
||||
let turn_starts = find_turn_starts(items);
|
||||
if turn_starts.len() <= protected_turns {
|
||||
/// Same as [`prunable_indices`] but also returns the start index of the
|
||||
/// protected suffix. `None` means the token boundary could not be determined
|
||||
/// (currently because usage estimates were `NoData` or malformed).
|
||||
pub fn evaluate_candidates(
|
||||
items: &[Item],
|
||||
protected_tokens: u64,
|
||||
token_estimates: &[TokenEstimate],
|
||||
) -> (Vec<usize>, Option<usize>) {
|
||||
let Some(protected_start) = protected_start_index(items, protected_tokens, token_estimates)
|
||||
else {
|
||||
return (Vec::new(), None);
|
||||
}
|
||||
let boundary = turn_starts[turn_starts.len() - protected_turns];
|
||||
let candidates = items[..boundary]
|
||||
};
|
||||
|
||||
let candidates = items[..protected_start]
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter_map(|(i, item)| match item {
|
||||
|
|
@ -160,7 +164,38 @@ pub fn evaluate_candidates(items: &[Item], protected_turns: usize) -> (Vec<usize
|
|||
_ => None,
|
||||
})
|
||||
.collect();
|
||||
(candidates, Some(boundary))
|
||||
(candidates, Some(protected_start))
|
||||
}
|
||||
|
||||
fn protected_start_index(
|
||||
items: &[Item],
|
||||
protected_tokens: u64,
|
||||
token_estimates: &[TokenEstimate],
|
||||
) -> Option<usize> {
|
||||
if token_estimates.len() != items.len() + 1 {
|
||||
return None;
|
||||
}
|
||||
let total = token_estimates[items.len()];
|
||||
if total.source == EstimateSource::NoData {
|
||||
return None;
|
||||
}
|
||||
if protected_tokens == 0 {
|
||||
return Some(items.len());
|
||||
}
|
||||
|
||||
let mut protected_start = items.len();
|
||||
for idx in (0..items.len()).rev() {
|
||||
let prefix = token_estimates[idx];
|
||||
if prefix.source == EstimateSource::NoData {
|
||||
return None;
|
||||
}
|
||||
protected_start = idx;
|
||||
let tail_tokens = total.tokens.saturating_sub(prefix.tokens);
|
||||
if tail_tokens >= protected_tokens {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Some(protected_start)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
@ -185,17 +220,70 @@ mod tests {
|
|||
items
|
||||
}
|
||||
|
||||
fn measured_prefix(tokens: &[u64]) -> Vec<TokenEstimate> {
|
||||
tokens
|
||||
.iter()
|
||||
.copied()
|
||||
.map(|tokens| TokenEstimate {
|
||||
tokens,
|
||||
source: EstimateSource::Measured,
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn uniform_estimates(items: &[Item], item_tokens: u64) -> Vec<TokenEstimate> {
|
||||
let mut tokens = Vec::with_capacity(items.len() + 1);
|
||||
for i in 0..=items.len() {
|
||||
tokens.push(i as u64 * item_tokens);
|
||||
}
|
||||
measured_prefix(&tokens)
|
||||
}
|
||||
|
||||
fn estimates_from_item_tokens(item_tokens: &[u64]) -> Vec<TokenEstimate> {
|
||||
let mut prefix = Vec::with_capacity(item_tokens.len() + 1);
|
||||
let mut acc = 0;
|
||||
prefix.push(acc);
|
||||
for tokens in item_tokens {
|
||||
acc += tokens;
|
||||
prefix.push(acc);
|
||||
}
|
||||
measured_prefix(&prefix)
|
||||
}
|
||||
|
||||
fn no_data_estimates(items: &[Item]) -> Vec<TokenEstimate> {
|
||||
(0..=items.len())
|
||||
.map(|i| TokenEstimate {
|
||||
tokens: i as u64,
|
||||
source: if i == 0 {
|
||||
EstimateSource::Measured
|
||||
} else {
|
||||
EstimateSource::NoData
|
||||
},
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_candidates_when_too_few_turns() {
|
||||
fn no_candidates_when_estimate_has_no_data() {
|
||||
let items = make_history(&[("turn1", vec![("summary1", Some("big content here"))])]);
|
||||
let estimates = no_data_estimates(&items);
|
||||
let (candidates, protected_start) = evaluate_candidates(&items, 10, &estimates);
|
||||
assert!(candidates.is_empty());
|
||||
assert_eq!(protected_start, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_candidates_when_history_fits_in_protected_tokens() {
|
||||
let items = make_history(&[
|
||||
("turn1", vec![("summary1", Some("big content here"))]),
|
||||
("turn2", vec![("summary2", Some("more content"))]),
|
||||
]);
|
||||
assert!(prunable_indices(&items, 3).is_empty());
|
||||
let estimates = uniform_estimates(&items, 10);
|
||||
assert!(prunable_indices(&items, 10_000, &estimates).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn candidates_in_unprotected_turns() {
|
||||
fn candidates_before_token_protected_suffix() {
|
||||
let big = "x".repeat(4096 * 4);
|
||||
let items = make_history(&[
|
||||
("turn1", vec![("s1", Some(&big))]),
|
||||
|
|
@ -203,9 +291,39 @@ mod tests {
|
|||
("turn3", vec![("s3", Some("keep me"))]),
|
||||
("turn4", vec![("s4", Some("keep me too"))]),
|
||||
]);
|
||||
let candidates = prunable_indices(&items, 2);
|
||||
let estimates = uniform_estimates(&items, 10);
|
||||
let candidates = prunable_indices(&items, 80, &estimates);
|
||||
assert_eq!(candidates.len(), 2);
|
||||
// suffix budget 80 tokens protects turn3+turn4 (8 items), so only s1/s2 are candidates.
|
||||
for &i in &candidates {
|
||||
if let Item::ToolResult { summary, .. } = &items[i] {
|
||||
assert!(summary == "s1" || summary == "s2");
|
||||
} else {
|
||||
panic!("non tool-result selected");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn single_long_task_gets_candidates_without_multiple_user_turns() {
|
||||
let big = "x".repeat(4096 * 8);
|
||||
let items = make_history(&[(
|
||||
"one long task",
|
||||
vec![
|
||||
("s1", Some(&big)),
|
||||
("s2", Some(&big)),
|
||||
("s3", Some(&big)),
|
||||
("s4", Some(&big)),
|
||||
],
|
||||
)]);
|
||||
// user + assistant are cheap; every ToolCall is cheap; every ToolResult is heavy.
|
||||
let item_tokens = vec![1, 1, 1, 5_000, 1, 5_000, 1, 5_000, 1, 5_000];
|
||||
let estimates = estimates_from_item_tokens(&item_tokens);
|
||||
|
||||
let (candidates, protected_start) = evaluate_candidates(&items, 8_000, &estimates);
|
||||
|
||||
assert_eq!(protected_start, Some(7));
|
||||
assert_eq!(candidates.len(), 2);
|
||||
// 候補は turn1 と turn2 の ToolResult のみ
|
||||
for &i in &candidates {
|
||||
if let Item::ToolResult { summary, .. } = &items[i] {
|
||||
assert!(summary == "s1" || summary == "s2");
|
||||
|
|
@ -223,7 +341,8 @@ mod tests {
|
|||
("turn3", vec![]),
|
||||
("turn4", vec![]),
|
||||
]);
|
||||
assert!(prunable_indices(&items, 2).is_empty());
|
||||
let estimates = uniform_estimates(&items, 10);
|
||||
assert!(prunable_indices(&items, 20, &estimates).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -235,7 +354,8 @@ mod tests {
|
|||
("turn3", vec![("s3", Some("keep me"))]),
|
||||
("turn4", vec![("s4", Some("keep me too"))]),
|
||||
]);
|
||||
let candidates = prunable_indices(&items, 2);
|
||||
let estimates = uniform_estimates(&items, 10);
|
||||
let candidates = prunable_indices(&items, 80, &estimates);
|
||||
let count = project(&mut items, &candidates);
|
||||
assert_eq!(count, 2);
|
||||
|
||||
|
|
@ -261,7 +381,7 @@ mod tests {
|
|||
("turn1", vec![("s1", None)]),
|
||||
("turn2", vec![("s2", Some("hello"))]),
|
||||
]);
|
||||
// Manually target s1 (index 3) even though it's already None.
|
||||
// Manually target s1 even though it's already None.
|
||||
let target = items
|
||||
.iter()
|
||||
.position(|it| matches!(it, Item::ToolResult { summary, .. } if summary == "s1"))
|
||||
|
|
@ -279,14 +399,15 @@ mod tests {
|
|||
("turn3", vec![]),
|
||||
("turn4", vec![]),
|
||||
]);
|
||||
let candidates = prunable_indices(&items, 2);
|
||||
let estimates = uniform_estimates(&items, 10);
|
||||
let candidates = prunable_indices(&items, 20, &estimates);
|
||||
assert_eq!(project(&mut items, &candidates), 1);
|
||||
// 2 周目: 候補は一度の prunable_indices 結果を使い回しても 0 件。
|
||||
assert_eq!(project(&mut items, &candidates), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn evaluate_candidates_returns_boundary_index() {
|
||||
fn evaluate_candidates_returns_protected_start_index() {
|
||||
let big = "x".repeat(64);
|
||||
let items = make_history(&[
|
||||
("turn1", vec![("s1", Some(&big))]),
|
||||
|
|
@ -294,36 +415,37 @@ mod tests {
|
|||
("turn3", vec![("s3", Some("keep"))]),
|
||||
("turn4", vec![("s4", Some("keep too"))]),
|
||||
]);
|
||||
let (candidates, border) = evaluate_candidates(&items, 2);
|
||||
let estimates = uniform_estimates(&items, 10);
|
||||
let (candidates, protected_start) = evaluate_candidates(&items, 80, &estimates);
|
||||
assert_eq!(candidates.len(), 2);
|
||||
// protected_turns=2 → boundary は turn3 の user message 位置。
|
||||
// turn1: u/a/c/r (4) + turn2: u/a/c/r (4) = index 8 (turn3 の user)。
|
||||
assert_eq!(border, Some(8));
|
||||
// protected_tokens=80 → protected suffix is turn3+turn4, starting at index 8.
|
||||
assert_eq!(protected_start, Some(8));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn evaluate_candidates_no_boundary_when_too_few_turns() {
|
||||
fn evaluate_candidates_reports_zero_start_when_everything_is_protected() {
|
||||
let items = make_history(&[("only", vec![("s", Some("x"))])]);
|
||||
let (candidates, border) = evaluate_candidates(&items, 2);
|
||||
let estimates = uniform_estimates(&items, 10);
|
||||
let (candidates, protected_start) = evaluate_candidates(&items, 10_000, &estimates);
|
||||
assert!(candidates.is_empty());
|
||||
assert!(border.is_none());
|
||||
assert_eq!(protected_start, Some(0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn protected_turns_boundary_exact() {
|
||||
// 3 turns with protected_turns=2: only turn 1 is a candidate.
|
||||
fn zero_protected_tokens_allows_all_tool_results_as_candidates() {
|
||||
let big = "x".repeat(64);
|
||||
let items = make_history(&[
|
||||
("turn1", vec![("s1", Some(&big))]),
|
||||
("turn2", vec![("s2", Some("protected"))]),
|
||||
("turn3", vec![("s3", Some("also protected"))]),
|
||||
]);
|
||||
let candidates = prunable_indices(&items, 2);
|
||||
assert_eq!(candidates.len(), 1);
|
||||
if let Item::ToolResult { summary, .. } = &items[candidates[0]] {
|
||||
assert_eq!(summary, "s1");
|
||||
} else {
|
||||
panic!("expected ToolResult at candidate index");
|
||||
let items = make_history(&[("turn1", vec![("s1", Some(&big)), ("s2", Some(&big))])]);
|
||||
let estimates = uniform_estimates(&items, 10);
|
||||
let (candidates, protected_start) = evaluate_candidates(&items, 0, &estimates);
|
||||
assert_eq!(protected_start, Some(items.len()));
|
||||
assert_eq!(candidates.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_estimate_vector_is_treated_as_no_boundary() {
|
||||
let items = make_history(&[("turn1", vec![("s1", Some("x"))])]);
|
||||
let (candidates, protected_start) = evaluate_candidates(&items, 10, &[]);
|
||||
assert!(candidates.is_empty());
|
||||
assert_eq!(protected_start, None);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -201,6 +201,10 @@ pub struct Worker<C: LlmClient, S: WorkerState = Mutable> {
|
|||
tool_output_limits: Option<ToolOutputLimits>,
|
||||
/// Prune configuration. `None` disables the prune projection.
|
||||
prune_config: Option<crate::prune::PruneConfig>,
|
||||
/// Callback that estimates prefix token counts, injected by higher
|
||||
/// layers that own usage measurements. `None` disables the prune
|
||||
/// projection.
|
||||
token_estimator: Option<crate::prune::TokenEstimator>,
|
||||
/// Callback that estimates token savings for a drop range, injected
|
||||
/// by higher layers that own usage measurements. `None` disables
|
||||
/// the prune projection.
|
||||
|
|
@ -213,7 +217,7 @@ pub struct Worker<C: LlmClient, S: WorkerState = Mutable> {
|
|||
cache_anchor: Option<usize>,
|
||||
/// Conversation-scoped cache key, set by higher layers. Plumbed into
|
||||
/// [`Request::cache_key`] at request build time. Pod 側では
|
||||
/// `SessionId` を渡す。
|
||||
/// `SegmentId` を渡す。
|
||||
cache_key: Option<String>,
|
||||
/// State marker
|
||||
_state: PhantomData<S>,
|
||||
|
|
@ -434,6 +438,17 @@ impl<C: LlmClient, S: WorkerState> Worker<C, S> {
|
|||
self.prune_config = config;
|
||||
}
|
||||
|
||||
/// Inject the callback used to estimate prefix token counts for prune's
|
||||
/// protected-token boundary.
|
||||
///
|
||||
/// The callback is invoked with the *request context* (a clone of
|
||||
/// history). It must be pure/idempotent since it may be called once per
|
||||
/// LLM request. Returning `NoData` estimates makes prune skip as if no
|
||||
/// candidates existed.
|
||||
pub fn set_token_estimator(&mut self, estimator: Option<crate::prune::TokenEstimator>) {
|
||||
self.token_estimator = estimator;
|
||||
}
|
||||
|
||||
/// Inject the callback used to estimate token savings for a prune
|
||||
/// candidate range.
|
||||
///
|
||||
|
|
@ -983,18 +998,26 @@ impl<C: LlmClient, S: WorkerState> Worker<C, S> {
|
|||
// prunable candidates whose estimated savings meet the
|
||||
// threshold. Worker does not own usage history itself; the
|
||||
// estimator is injected by the layer that does.
|
||||
if let (Some(config), Some(estimator)) = (&self.prune_config, &self.savings_estimator) {
|
||||
let (candidates, border_turn) =
|
||||
crate::prune::evaluate_candidates(&request_context, config.protected_turns);
|
||||
if let (Some(config), Some(token_estimator), Some(savings_estimator)) = (
|
||||
&self.prune_config,
|
||||
&self.token_estimator,
|
||||
&self.savings_estimator,
|
||||
) {
|
||||
let token_estimates = token_estimator(&request_context);
|
||||
let (candidates, protected_start_index) = crate::prune::evaluate_candidates(
|
||||
&request_context,
|
||||
config.protected_tokens,
|
||||
&token_estimates,
|
||||
);
|
||||
let evaluation = if candidates.is_empty() {
|
||||
crate::prune::PruneEvaluation {
|
||||
candidate_count: 0,
|
||||
estimated_savings: 0,
|
||||
border_turn,
|
||||
protected_start_index,
|
||||
decision: crate::prune::PruneDecision::SkippedNoCandidates,
|
||||
}
|
||||
} else {
|
||||
let savings = estimator(&request_context, &candidates);
|
||||
let savings = savings_estimator(&request_context, &candidates);
|
||||
if savings >= config.min_savings {
|
||||
let pruned = crate::prune::project(&mut request_context, &candidates);
|
||||
if pruned > 0 {
|
||||
|
|
@ -1007,7 +1030,7 @@ impl<C: LlmClient, S: WorkerState> Worker<C, S> {
|
|||
crate::prune::PruneEvaluation {
|
||||
candidate_count: candidates.len(),
|
||||
estimated_savings: savings,
|
||||
border_turn,
|
||||
protected_start_index,
|
||||
decision: crate::prune::PruneDecision::Fired {
|
||||
pruned_count: pruned,
|
||||
},
|
||||
|
|
@ -1016,7 +1039,7 @@ impl<C: LlmClient, S: WorkerState> Worker<C, S> {
|
|||
crate::prune::PruneEvaluation {
|
||||
candidate_count: candidates.len(),
|
||||
estimated_savings: savings,
|
||||
border_turn,
|
||||
protected_start_index,
|
||||
decision: crate::prune::PruneDecision::SkippedBelowMinSavings,
|
||||
}
|
||||
}
|
||||
|
|
@ -1256,6 +1279,7 @@ impl<C: LlmClient> Worker<C, Mutable> {
|
|||
cancel_rx,
|
||||
tool_output_limits: None,
|
||||
prune_config: None,
|
||||
token_estimator: None,
|
||||
savings_estimator: None,
|
||||
prune_observer: None,
|
||||
cache_anchor: None,
|
||||
|
|
@ -1519,6 +1543,7 @@ impl<C: LlmClient> Worker<C, Mutable> {
|
|||
cancel_rx: self.cancel_rx,
|
||||
tool_output_limits: self.tool_output_limits,
|
||||
prune_config: self.prune_config,
|
||||
token_estimator: self.token_estimator,
|
||||
savings_estimator: self.savings_estimator,
|
||||
prune_observer: self.prune_observer,
|
||||
cache_anchor: self.cache_anchor,
|
||||
|
|
@ -1605,6 +1630,7 @@ impl<C: LlmClient> Worker<C, Locked> {
|
|||
cancel_rx: self.cancel_rx,
|
||||
tool_output_limits: self.tool_output_limits,
|
||||
prune_config: self.prune_config,
|
||||
token_estimator: self.token_estimator,
|
||||
savings_estimator: self.savings_estimator,
|
||||
prune_observer: self.prune_observer,
|
||||
cache_anchor: self.cache_anchor,
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ use std::collections::HashMap;
|
|||
use std::num::NonZeroU32;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde::de::Error as _;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::defaults;
|
||||
|
|
@ -112,7 +113,7 @@ pub struct PermissionConfigPartial {
|
|||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct CompactionConfigPartial {
|
||||
#[serde(default)]
|
||||
pub prune_protected_turns: Option<usize>,
|
||||
pub prune_protected_tokens: Option<u64>,
|
||||
#[serde(default)]
|
||||
pub prune_min_savings: Option<u64>,
|
||||
#[serde(default)]
|
||||
|
|
@ -141,12 +142,40 @@ pub enum ResolveError {
|
|||
RelativePath { field: &'static str, path: PathBuf },
|
||||
}
|
||||
|
||||
/// Reject manifest fields that were intentionally removed and must not be
|
||||
/// silently swallowed by the general warn-and-ignore unknown-field policy.
|
||||
pub(crate) fn reject_removed_manifest_fields(s: &str) -> Result<(), toml::de::Error> {
|
||||
let value: toml::Value = toml::from_str(s)?;
|
||||
if value
|
||||
.get("compaction")
|
||||
.and_then(toml::Value::as_table)
|
||||
.is_some_and(|table| table.contains_key("prune_protected_turns"))
|
||||
{
|
||||
return Err(toml::de::Error::custom(
|
||||
"unknown field in manifest: compaction.prune_protected_turns \
|
||||
(removed; use compaction.prune_protected_tokens)",
|
||||
));
|
||||
}
|
||||
if value
|
||||
.get("memory")
|
||||
.and_then(toml::Value::as_table)
|
||||
.is_some_and(|table| table.contains_key("extract_worker_max_input_tokens"))
|
||||
{
|
||||
return Err(toml::de::Error::custom(
|
||||
"unknown field in manifest: memory.extract_worker_max_input_tokens (removed)",
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl PodManifestConfig {
|
||||
/// Parse a partial manifest from a TOML string. Unknown top-level or
|
||||
/// nested fields emit a `tracing::warn!` and are ignored; use
|
||||
/// `tracing_subscriber` with `WARN` enabled to surface them to the
|
||||
/// operator.
|
||||
/// operator. Removed fields that must not be silently ignored (currently
|
||||
/// `compaction.prune_protected_turns`) are rejected before deserialization.
|
||||
pub fn from_toml(s: &str) -> Result<Self, toml::de::Error> {
|
||||
reject_removed_manifest_fields(s)?;
|
||||
let de = toml::Deserializer::parse(s)?;
|
||||
serde_ignored::deserialize(de, |path| {
|
||||
tracing::warn!("unknown field in manifest: {}", path);
|
||||
|
|
@ -263,9 +292,6 @@ impl MemoryConfig {
|
|||
language: upper.language.or(self.language),
|
||||
extract_model: upper.extract_model.or(self.extract_model),
|
||||
extract_threshold: upper.extract_threshold.or(self.extract_threshold),
|
||||
extract_worker_max_input_tokens: upper
|
||||
.extract_worker_max_input_tokens
|
||||
.or(self.extract_worker_max_input_tokens),
|
||||
extract_worker_max_turns: upper
|
||||
.extract_worker_max_turns
|
||||
.or(self.extract_worker_max_turns),
|
||||
|
|
@ -339,7 +365,7 @@ impl PermissionConfigPartial {
|
|||
impl CompactionConfigPartial {
|
||||
fn merge(self, upper: Self) -> Self {
|
||||
Self {
|
||||
prune_protected_turns: upper.prune_protected_turns.or(self.prune_protected_turns),
|
||||
prune_protected_tokens: upper.prune_protected_tokens.or(self.prune_protected_tokens),
|
||||
prune_min_savings: upper.prune_min_savings.or(self.prune_min_savings),
|
||||
compact_threshold: upper.compact_threshold.or(self.compact_threshold),
|
||||
compact_request_threshold: upper
|
||||
|
|
@ -489,9 +515,9 @@ impl TryFrom<PodManifestConfig> for PodManifest {
|
|||
validate_model_paths(cm, "compaction.model.auth.file")?;
|
||||
}
|
||||
Ok(CompactionConfig {
|
||||
prune_protected_turns: c
|
||||
.prune_protected_turns
|
||||
.unwrap_or(defaults::PRUNE_PROTECTED_TURNS),
|
||||
prune_protected_tokens: c
|
||||
.prune_protected_tokens
|
||||
.unwrap_or(defaults::PRUNE_PROTECTED_TOKENS),
|
||||
prune_min_savings: c.prune_min_savings.unwrap_or(defaults::PRUNE_MIN_SAVINGS),
|
||||
compact_threshold: c.compact_threshold,
|
||||
compact_request_threshold: c.compact_request_threshold,
|
||||
|
|
@ -921,7 +947,7 @@ mod tests {
|
|||
let lower = PodManifestConfig {
|
||||
compaction: Some(CompactionConfigPartial {
|
||||
compact_threshold: Some(50_000),
|
||||
prune_protected_turns: Some(5),
|
||||
prune_protected_tokens: Some(5_000),
|
||||
..Default::default()
|
||||
}),
|
||||
..Default::default()
|
||||
|
|
@ -937,7 +963,7 @@ mod tests {
|
|||
let c = merged.compaction.unwrap();
|
||||
assert_eq!(c.compact_threshold, Some(80_000));
|
||||
// field from lower retained when upper has None
|
||||
assert_eq!(c.prune_protected_turns, Some(5));
|
||||
assert_eq!(c.prune_protected_tokens, Some(5_000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -971,6 +997,45 @@ unknown_future_field = "tolerated"
|
|||
assert_eq!(cfg.worker.max_tokens, Some(1000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn from_toml_rejects_removed_prune_protected_turns_field() {
|
||||
let bad = r#"
|
||||
[compaction]
|
||||
prune_protected_turns = 3
|
||||
"#;
|
||||
let err = PodManifestConfig::from_toml(bad).unwrap_err();
|
||||
assert!(
|
||||
err.to_string().contains("compaction.prune_protected_turns"),
|
||||
"unexpected error: {err}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn from_toml_rejects_removed_extract_worker_max_input_tokens_field() {
|
||||
let bad = r#"
|
||||
[memory]
|
||||
extract_worker_max_input_tokens = 30000
|
||||
"#;
|
||||
let err = PodManifestConfig::from_toml(bad).unwrap_err();
|
||||
assert!(
|
||||
err.to_string()
|
||||
.contains("memory.extract_worker_max_input_tokens"),
|
||||
"unexpected error: {err}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn from_toml_accepts_extract_worker_max_turns() {
|
||||
let cfg = PodManifestConfig::from_toml(
|
||||
r#"
|
||||
[memory]
|
||||
extract_worker_max_turns = 2
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(cfg.memory.unwrap().extract_worker_max_turns, Some(2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn from_toml_accepts_worker_reasoning_string_or_integer() {
|
||||
let effort = PodManifestConfig::from_toml(
|
||||
|
|
|
|||
|
|
@ -14,9 +14,9 @@ pub const TOOL_OUTPUT_MAX_BYTES: usize = 64 * 1024;
|
|||
/// See [`crate::FileUploadLimits`].
|
||||
pub const FILE_UPLOAD_MAX_BYTES: usize = 256 * 1024;
|
||||
|
||||
/// Number of most-recent turns protected from pruning. See
|
||||
/// [`crate::CompactionConfig::prune_protected_turns`].
|
||||
pub const PRUNE_PROTECTED_TURNS: usize = 3;
|
||||
/// Token budget at the history tail protected from pruning. See
|
||||
/// [`crate::CompactionConfig::prune_protected_tokens`].
|
||||
pub const PRUNE_PROTECTED_TOKENS: u64 = 8000;
|
||||
|
||||
/// Minimum estimated token savings required to trigger a prune. See
|
||||
/// [`crate::CompactionConfig::prune_min_savings`].
|
||||
|
|
@ -59,11 +59,6 @@ pub const COMPACT_WORKER_MAX_TURNS: Option<u32> = Some(20);
|
|||
/// default references.
|
||||
pub const COMPACT_DEFAULT_REFERENCE_COUNT: usize = 5;
|
||||
|
||||
/// Current prompt-occupancy cap for the memory extract worker's own
|
||||
/// LLM requests. Exceeding this aborts the extract run (circuit-breaker
|
||||
/// path). See [`crate::MemoryConfig::extract_worker_max_input_tokens`].
|
||||
pub const MEMORY_EXTRACT_WORKER_MAX_INPUT_TOKENS: u64 = 30_000;
|
||||
|
||||
/// Optional maximum extract-worker tool-loop depth. `None` means unlimited.
|
||||
/// See [`crate::MemoryConfig::extract_worker_max_turns`].
|
||||
pub const MEMORY_EXTRACT_WORKER_MAX_TURNS: Option<u32> = Some(8);
|
||||
|
|
|
|||
|
|
@ -114,11 +114,6 @@ pub struct MemoryConfig {
|
|||
/// the auto-extract trigger is dormant.
|
||||
#[serde(default)]
|
||||
pub extract_threshold: Option<u64>,
|
||||
/// Current prompt-occupancy cap for the extract worker's own LLM
|
||||
/// requests. Exceeding this aborts the extract run. `None` ⇒
|
||||
/// [`defaults::MEMORY_EXTRACT_WORKER_MAX_INPUT_TOKENS`].
|
||||
#[serde(default)]
|
||||
pub extract_worker_max_input_tokens: Option<u64>,
|
||||
/// Optional maximum extract-worker tool-loop depth. `None` leaves
|
||||
/// the worker unlimited; the default bounds runaway short-context
|
||||
/// loops. Falls through to
|
||||
|
|
@ -337,9 +332,9 @@ pub enum ToolPermissionAction {
|
|||
/// (full history summarisation). Omitting `[compaction]` disables both.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct CompactionConfig {
|
||||
/// Number of recent turns protected from pruning.
|
||||
#[serde(default = "default_prune_protected_turns")]
|
||||
pub prune_protected_turns: usize,
|
||||
/// Token budget at the history tail protected from pruning.
|
||||
#[serde(default = "default_prune_protected_tokens")]
|
||||
pub prune_protected_tokens: u64,
|
||||
|
||||
/// Minimum estimated token savings to trigger a prune.
|
||||
#[serde(default = "default_prune_min_savings")]
|
||||
|
|
@ -393,8 +388,8 @@ pub struct CompactionConfig {
|
|||
pub model: Option<ModelManifest>,
|
||||
}
|
||||
|
||||
fn default_prune_protected_turns() -> usize {
|
||||
defaults::PRUNE_PROTECTED_TURNS
|
||||
fn default_prune_protected_tokens() -> u64 {
|
||||
defaults::PRUNE_PROTECTED_TOKENS
|
||||
}
|
||||
fn default_prune_min_savings() -> u64 {
|
||||
defaults::PRUNE_MIN_SAVINGS
|
||||
|
|
@ -415,7 +410,7 @@ fn default_compact_worker_max_turns() -> Option<u32> {
|
|||
impl Default for CompactionConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
prune_protected_turns: default_prune_protected_turns(),
|
||||
prune_protected_tokens: default_prune_protected_tokens(),
|
||||
prune_min_savings: default_prune_min_savings(),
|
||||
compact_threshold: None,
|
||||
compact_request_threshold: None,
|
||||
|
|
@ -431,6 +426,7 @@ impl Default for CompactionConfig {
|
|||
impl PodManifest {
|
||||
/// Parse a manifest from a TOML string.
|
||||
pub fn from_toml(s: &str) -> Result<Self, toml::de::Error> {
|
||||
config::reject_removed_manifest_fields(s)?;
|
||||
toml::from_str(s)
|
||||
}
|
||||
}
|
||||
|
|
@ -581,7 +577,7 @@ model_id = "claude-sonnet-4-20250514"
|
|||
let toml = format!("{MINIMAL_REQUIRED}\n[compaction]\ncompact_threshold = 80000\n");
|
||||
let manifest = PodManifest::from_toml(&toml).unwrap();
|
||||
let c = manifest.compaction.unwrap();
|
||||
assert_eq!(c.prune_protected_turns, 3);
|
||||
assert_eq!(c.prune_protected_tokens, 8000);
|
||||
assert_eq!(c.prune_min_savings, 4096);
|
||||
assert_eq!(c.compact_threshold, Some(80000));
|
||||
assert_eq!(c.compact_request_threshold, None);
|
||||
|
|
@ -589,6 +585,16 @@ model_id = "claude-sonnet-4-20250514"
|
|||
assert_eq!(c.compact_worker_max_turns, Some(20));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reject_removed_prune_protected_turns_field() {
|
||||
let toml = format!("{MINIMAL_REQUIRED}\n[compaction]\nprune_protected_turns = 3\n");
|
||||
let err = PodManifest::from_toml(&toml).unwrap_err();
|
||||
assert!(
|
||||
err.to_string().contains("compaction.prune_protected_turns"),
|
||||
"unexpected error: {err}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_compaction_worker_max_turns() {
|
||||
let toml = format!(
|
||||
|
|
|
|||
|
|
@ -52,6 +52,10 @@ pub struct ModelManifest {
|
|||
/// `default_capability` → scheme 既定の順で解決される。
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub capability: Option<ModelCapability>,
|
||||
/// モデルのコンテキストウィンドウ上限(tokens)。カタログ未掲載 / inline
|
||||
/// モデルでもここで明示 override できる。
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub context_window: Option<u64>,
|
||||
}
|
||||
|
||||
impl ModelManifest {
|
||||
|
|
@ -65,6 +69,7 @@ impl ModelManifest {
|
|||
model_id: upper.model_id.or(self.model_id),
|
||||
auth: upper.auth.or(self.auth),
|
||||
capability: upper.capability.or(self.capability),
|
||||
context_window: upper.context_window.or(self.context_window),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ pub struct Scope {
|
|||
deny: Vec<ResolvedRule>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
struct ResolvedRule {
|
||||
/// Absolute, canonicalized-or-normalized target directory/file.
|
||||
target: PathBuf,
|
||||
|
|
@ -217,6 +217,32 @@ impl Scope {
|
|||
Self::from_config(&config)
|
||||
}
|
||||
|
||||
/// Build a new [`Scope`] with one matching deny rule removed for each
|
||||
/// rule in `remove_deny`.
|
||||
///
|
||||
/// This is intentionally exact (after the same target resolution used
|
||||
/// by [`Scope::from_config`]) rather than geometric: reclaiming a
|
||||
/// delegated child must remove the deny layer that was added for that
|
||||
/// child without broadening any explicit base deny that merely overlaps
|
||||
/// the delegated path. Missing rules are ignored, making repeated
|
||||
/// reclaim calls harmless.
|
||||
pub fn with_removed_deny_rules(
|
||||
&self,
|
||||
remove_deny: impl IntoIterator<Item = ScopeRule>,
|
||||
) -> Result<Self, ScopeError> {
|
||||
let mut deny = self.deny.clone();
|
||||
for rule in remove_deny {
|
||||
let resolved = resolve_rule(&rule)?;
|
||||
if let Some(idx) = deny.iter().position(|existing| existing == &resolved) {
|
||||
deny.remove(idx);
|
||||
}
|
||||
}
|
||||
Ok(Self {
|
||||
allow: self.allow.clone(),
|
||||
deny,
|
||||
})
|
||||
}
|
||||
|
||||
/// Human-readable grouping of allow rules, suitable for embedding in
|
||||
/// LLM system prompts. Deny rules are intentionally omitted — they
|
||||
/// only cap effective permission and surface them would mislead the
|
||||
|
|
@ -684,6 +710,44 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn with_removed_deny_rules_reclaims_one_matching_layer() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let sub = dir.path().join("sub");
|
||||
std::fs::create_dir(&sub).unwrap();
|
||||
let rule = ScopeRule {
|
||||
target: sub.clone(),
|
||||
permission: Permission::Write,
|
||||
recursive: true,
|
||||
};
|
||||
let base = Scope::writable(dir.path())
|
||||
.unwrap()
|
||||
.with_added_deny_rules([rule.clone(), rule.clone()])
|
||||
.unwrap();
|
||||
|
||||
let reclaimed_once = base.with_removed_deny_rules([rule.clone()]).unwrap();
|
||||
assert_eq!(
|
||||
reclaimed_once.permission_at(&sub.join("a.txt")),
|
||||
Some(Permission::Read),
|
||||
"one duplicate deny layer must remain"
|
||||
);
|
||||
|
||||
let reclaimed_twice = reclaimed_once
|
||||
.with_removed_deny_rules([rule.clone()])
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
reclaimed_twice.permission_at(&sub.join("a.txt")),
|
||||
Some(Permission::Write)
|
||||
);
|
||||
|
||||
let reclaimed_again = reclaimed_twice.with_removed_deny_rules([rule]).unwrap();
|
||||
assert_eq!(
|
||||
reclaimed_again.permission_at(&sub.join("a.txt")),
|
||||
Some(Permission::Write),
|
||||
"missing rules are ignored for idempotent reclaim"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shared_scope_load_returns_current_value() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
|
|
|
|||
|
|
@ -243,7 +243,7 @@ mod tests {
|
|||
let (_id, _) = write_staging(
|
||||
&layout,
|
||||
SourceRef {
|
||||
session_id: "s".into(),
|
||||
segment_id: "s".into(),
|
||||
range: [0, 1],
|
||||
},
|
||||
ExtractedPayload::default(),
|
||||
|
|
|
|||
|
|
@ -260,7 +260,7 @@ mod tests {
|
|||
let (id_a, _) = write_staging(
|
||||
&layout,
|
||||
SourceRef {
|
||||
session_id: "s".into(),
|
||||
segment_id: "s".into(),
|
||||
range: [0, 0],
|
||||
},
|
||||
ExtractedPayload::default(),
|
||||
|
|
@ -269,7 +269,7 @@ mod tests {
|
|||
let (id_b, _) = write_staging(
|
||||
&layout,
|
||||
SourceRef {
|
||||
session_id: "s".into(),
|
||||
segment_id: "s".into(),
|
||||
range: [1, 1],
|
||||
},
|
||||
ExtractedPayload::default(),
|
||||
|
|
|
|||
|
|
@ -94,9 +94,9 @@ mod tests {
|
|||
ExtractedPayload::default()
|
||||
}
|
||||
|
||||
fn source(session_id: &str, range: [u64; 2]) -> SourceRef {
|
||||
fn source(segment_id: &str, range: [u64; 2]) -> SourceRef {
|
||||
SourceRef {
|
||||
session_id: session_id.into(),
|
||||
segment_id: segment_id.into(),
|
||||
range,
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -305,7 +305,7 @@ mod tests {
|
|||
fn flags_sources_overflow() {
|
||||
let (dir, layout) = workspace();
|
||||
let many_sources: String = (0..15)
|
||||
.map(|i| format!(" - session_id: s{i}\n range: [{i}, {i}]\n"))
|
||||
.map(|i| format!(" - segment_id: s{i}\n range: [{i}, {i}]\n"))
|
||||
.collect();
|
||||
write(
|
||||
&dir.path().join(".insomnia/memory/decisions/big.md"),
|
||||
|
|
|
|||
|
|
@ -69,4 +69,19 @@ mod tests {
|
|||
assert!(s.contains("[ToolResult] ok"));
|
||||
assert!(!s.contains("scratch"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tool_result_renders_summary_but_not_content() {
|
||||
let huge_content = "raw-content-should-never-enter-extract-input".repeat(10_000);
|
||||
let items = vec![Item::tool_result_with_content(
|
||||
"c1",
|
||||
"short summary kept for extraction",
|
||||
huge_content.clone(),
|
||||
)];
|
||||
|
||||
let s = build_extract_input(&items);
|
||||
assert!(s.contains("[ToolResult] short summary kept for extraction"));
|
||||
assert!(!s.contains("raw-content-should-never-enter-extract-input"));
|
||||
assert!(!s.contains(&huge_content));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
//! (session-store の `LogEntry::Extension`、domain `"memory.extract"`)は
|
||||
//! Pod 側が責務を持つ。
|
||||
//!
|
||||
//! 出力 JSON の wrap は [`write_staging`] が `source: { session_id, range }`
|
||||
//! 出力 JSON の wrap は [`write_staging`] が `source: { segment_id, range }`
|
||||
//! を機械付与する形で担当し、LLM には source を推論させない。
|
||||
|
||||
mod input;
|
||||
|
|
|
|||
|
|
@ -78,7 +78,7 @@ pub struct RequestEntry {
|
|||
|
||||
/// staging に書き出される 1 ファイル分のレコード。
|
||||
///
|
||||
/// `source` は Pod 側ラッパーが session_id と log entry range を
|
||||
/// `source` は Pod 側ラッパーが segment_id と log entry range を
|
||||
/// 機械付与する。LLM はこのフィールドを見ない / 推論しない。
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct StagingRecord {
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ use super::EXTRACT_DOMAIN;
|
|||
/// として 1 回ずつ書かれ、最新の 1 件が現行 pointer として有効になる。
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
||||
pub struct ExtractPointerPayload {
|
||||
/// 直近 extract が処理した最後の session-store HashedEntry の index。
|
||||
/// 直近 extract が処理した最後の session-store LogEntry の index。
|
||||
/// 次回の `source.range.start` はこの値 + 1。
|
||||
pub processed_through_entry: usize,
|
||||
/// 直近 extract 時点の `history.len()`。次回入力は
|
||||
|
|
|
|||
|
|
@ -71,7 +71,7 @@ mod tests {
|
|||
let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
|
||||
|
||||
let source = SourceRef {
|
||||
session_id: "sess-1".into(),
|
||||
segment_id: "sess-1".into(),
|
||||
range: [3, 7],
|
||||
};
|
||||
let payload = ExtractedPayload {
|
||||
|
|
@ -93,7 +93,7 @@ mod tests {
|
|||
|
||||
let written: StagingRecord =
|
||||
serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap();
|
||||
assert_eq!(written.source.session_id, "sess-1");
|
||||
assert_eq!(written.source.segment_id, "sess-1");
|
||||
assert_eq!(written.source.range, [3, 7]);
|
||||
assert_eq!(written.payload.decisions.len(), 1);
|
||||
}
|
||||
|
|
@ -103,7 +103,7 @@ mod tests {
|
|||
let tmp = tempfile::TempDir::new().unwrap();
|
||||
let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
|
||||
let source = SourceRef {
|
||||
session_id: "sess".into(),
|
||||
segment_id: "sess".into(),
|
||||
range: [0, 0],
|
||||
};
|
||||
let (_, path) = write_staging(&layout, source, ExtractedPayload::default()).unwrap();
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ pub use lint_common::Frontmatter;
|
|||
/// `last_sources` arrays for traceability back to raw session logs.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct SourceRef {
|
||||
pub session_id: String,
|
||||
pub segment_id: String,
|
||||
/// `[start_entry, end_entry]` inclusive range of session-store entry indices.
|
||||
pub range: [u64; 2],
|
||||
}
|
||||
|
|
|
|||
|
|
@ -60,7 +60,7 @@ impl Tool for ReadTool {
|
|||
})?;
|
||||
|
||||
let text = String::from_utf8_lossy(&bytes).into_owned();
|
||||
if let Some(session_id) = self.usage_session_id.as_deref() {
|
||||
if let Some(segment_id) = self.usage_session_id.as_deref() {
|
||||
let usage_slug = params.slug.as_deref().unwrap_or("summary");
|
||||
let snapshot = usage::snapshot_record_from_bytes(
|
||||
params.kind.record_kind(),
|
||||
|
|
@ -69,7 +69,7 @@ impl Tool for ReadTool {
|
|||
);
|
||||
if let Err(err) = usage::append_use_event(
|
||||
&self.layout,
|
||||
session_id.to_string(),
|
||||
segment_id.to_string(),
|
||||
UsageSource::MemoryRead,
|
||||
vec![snapshot],
|
||||
) {
|
||||
|
|
@ -140,9 +140,9 @@ pub fn read_tool(layout: WorkspaceLayout) -> ToolDefinition {
|
|||
|
||||
pub fn read_tool_with_usage(
|
||||
layout: WorkspaceLayout,
|
||||
session_id: impl Into<String>,
|
||||
segment_id: impl Into<String>,
|
||||
) -> ToolDefinition {
|
||||
read_tool_inner(layout, Some(session_id.into()))
|
||||
read_tool_inner(layout, Some(segment_id.into()))
|
||||
}
|
||||
|
||||
fn read_tool_inner(layout: WorkspaceLayout, usage_session_id: Option<String>) -> ToolDefinition {
|
||||
|
|
|
|||
|
|
@ -64,7 +64,7 @@ impl UsageRecordSnapshot {
|
|||
pub struct UsageEvent {
|
||||
pub id: Uuid,
|
||||
pub occurred_at: DateTime<Utc>,
|
||||
pub session_id: String,
|
||||
pub segment_id: String,
|
||||
pub event: UsageEventKind,
|
||||
pub source: UsageSource,
|
||||
pub records: Vec<UsageRecordSnapshot>,
|
||||
|
|
@ -72,7 +72,7 @@ pub struct UsageEvent {
|
|||
|
||||
impl UsageEvent {
|
||||
pub fn new(
|
||||
session_id: impl Into<String>,
|
||||
segment_id: impl Into<String>,
|
||||
event: UsageEventKind,
|
||||
source: UsageSource,
|
||||
records: Vec<UsageRecordSnapshot>,
|
||||
|
|
@ -80,7 +80,7 @@ impl UsageEvent {
|
|||
Self {
|
||||
id: Uuid::now_v7(),
|
||||
occurred_at: Utc::now(),
|
||||
session_id: session_id.into(),
|
||||
segment_id: segment_id.into(),
|
||||
event,
|
||||
source,
|
||||
records,
|
||||
|
|
@ -144,7 +144,7 @@ pub fn append_usage_event(layout: &WorkspaceLayout, event: &UsageEvent) -> io::R
|
|||
/// Convenience for a successful explicit record read.
|
||||
pub fn append_use_event(
|
||||
layout: &WorkspaceLayout,
|
||||
session_id: impl Into<String>,
|
||||
segment_id: impl Into<String>,
|
||||
source: UsageSource,
|
||||
records: Vec<UsageRecordSnapshot>,
|
||||
) -> io::Result<()> {
|
||||
|
|
@ -153,14 +153,14 @@ pub fn append_use_event(
|
|||
}
|
||||
append_usage_event(
|
||||
layout,
|
||||
&UsageEvent::new(session_id, UsageEventKind::Use, source, records),
|
||||
&UsageEvent::new(segment_id, UsageEventKind::Use, source, records),
|
||||
)
|
||||
}
|
||||
|
||||
/// Convenience for resident model-invocation exposure cost telemetry.
|
||||
pub fn append_resident_exposure_event(
|
||||
layout: &WorkspaceLayout,
|
||||
session_id: impl Into<String>,
|
||||
segment_id: impl Into<String>,
|
||||
records: Vec<UsageRecordSnapshot>,
|
||||
) -> io::Result<()> {
|
||||
if records.is_empty() {
|
||||
|
|
@ -169,7 +169,7 @@ pub fn append_resident_exposure_event(
|
|||
append_usage_event(
|
||||
layout,
|
||||
&UsageEvent::new(
|
||||
session_id,
|
||||
segment_id,
|
||||
UsageEventKind::ResidentExposure,
|
||||
UsageSource::ResidentInjection,
|
||||
records,
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ use std::io;
|
|||
use std::path::PathBuf;
|
||||
|
||||
use manifest::ScopeRule;
|
||||
use session_store::SessionId;
|
||||
use session_store::SegmentId;
|
||||
|
||||
/// Errors raised by the mutating pod-registry operations.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
|
|
@ -27,11 +27,11 @@ pub enum ScopeLockError {
|
|||
#[error("pod `{0}` is not registered")]
|
||||
UnknownPod(String),
|
||||
#[error(
|
||||
"session {session_id} is already held by pod `{pod_name}` at {}",
|
||||
"session {segment_id} is already held by pod `{pod_name}` at {}",
|
||||
.socket.display()
|
||||
)]
|
||||
SessionConflict {
|
||||
session_id: SessionId,
|
||||
SegmentConflict {
|
||||
segment_id: SegmentId,
|
||||
pod_name: String,
|
||||
socket: PathBuf,
|
||||
},
|
||||
|
|
|
|||
|
|
@ -27,11 +27,11 @@ pub use conflict::{
|
|||
};
|
||||
pub use error::ScopeLockError;
|
||||
pub use lifecycle::{
|
||||
ScopeAllocationGuard, SessionLockInfo, adopt_allocation, install_top_level,
|
||||
install_top_level_with_deny, lookup_session, update_session,
|
||||
ScopeAllocationGuard, SegmentLockInfo, adopt_allocation, install_top_level,
|
||||
install_top_level_with_deny, lookup_segment, update_segment,
|
||||
};
|
||||
pub use mutate::{
|
||||
delegate_scope, reclaim_stale, reclaim_stale_with, register_pod, register_pod_with_deny,
|
||||
release_pod,
|
||||
delegate_scope, reclaim_delegated_scope, reclaim_stale, reclaim_stale_with, register_pod,
|
||||
register_pod_with_deny, release_pod,
|
||||
};
|
||||
pub use table::{Allocation, LockFile, LockFileGuard, default_registry_path};
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
use std::path::{Path, PathBuf};
|
||||
|
||||
use manifest::ScopeRule;
|
||||
use session_store::SessionId;
|
||||
use session_store::SegmentId;
|
||||
|
||||
use crate::error::ScopeLockError;
|
||||
use crate::mutate::release_pod;
|
||||
|
|
@ -45,9 +45,9 @@ pub fn install_top_level(
|
|||
pid: u32,
|
||||
socket: PathBuf,
|
||||
scope_allow: Vec<ScopeRule>,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<ScopeAllocationGuard, ScopeLockError> {
|
||||
install_top_level_with_deny(pod_name, pid, socket, scope_allow, Vec::new(), session_id)
|
||||
install_top_level_with_deny(pod_name, pid, socket, scope_allow, Vec::new(), segment_id)
|
||||
}
|
||||
|
||||
/// Open the default lock file, register a top-level Pod with explicit
|
||||
|
|
@ -59,7 +59,7 @@ pub fn install_top_level_with_deny(
|
|||
socket: PathBuf,
|
||||
scope_allow: Vec<ScopeRule>,
|
||||
scope_deny: Vec<ScopeRule>,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<ScopeAllocationGuard, ScopeLockError> {
|
||||
let lock_path = default_registry_path()?;
|
||||
let mut guard = LockFileGuard::open(&lock_path)?;
|
||||
|
|
@ -70,7 +70,7 @@ pub fn install_top_level_with_deny(
|
|||
socket,
|
||||
scope_allow,
|
||||
scope_deny,
|
||||
session_id,
|
||||
segment_id,
|
||||
)?;
|
||||
Ok(ScopeAllocationGuard {
|
||||
pod_name,
|
||||
|
|
@ -83,14 +83,14 @@ pub fn install_top_level_with_deny(
|
|||
///
|
||||
/// The spawning flow is two-stage: the spawner calls
|
||||
/// [`crate::delegate_scope`] (with its own pid as a live placeholder,
|
||||
/// `session_id = None`), then exec's the child; the child, once
|
||||
/// `segment_id = None`), then exec's the child; the child, once
|
||||
/// running, calls this function to rewrite the allocation's pid +
|
||||
/// session_id to its own and claim the [`ScopeAllocationGuard`] so
|
||||
/// segment_id to its own and claim the [`ScopeAllocationGuard`] so
|
||||
/// the entry is released when the child exits.
|
||||
pub fn adopt_allocation(
|
||||
pod_name: String,
|
||||
new_pid: u32,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<ScopeAllocationGuard, ScopeLockError> {
|
||||
let lock_path = default_registry_path()?;
|
||||
let mut guard = LockFileGuard::open(&lock_path)?;
|
||||
|
|
@ -99,7 +99,7 @@ pub fn adopt_allocation(
|
|||
.find_mut(&pod_name)
|
||||
.ok_or_else(|| ScopeLockError::UnknownPod(pod_name.clone()))?;
|
||||
alloc.pid = new_pid;
|
||||
alloc.session_id = Some(session_id);
|
||||
alloc.segment_id = Some(segment_id);
|
||||
guard.save()?;
|
||||
Ok(ScopeAllocationGuard {
|
||||
pod_name,
|
||||
|
|
@ -107,32 +107,32 @@ pub fn adopt_allocation(
|
|||
})
|
||||
}
|
||||
|
||||
/// Rewrite the `session_id` recorded for `pod_name` to
|
||||
/// `new_session_id`.
|
||||
/// Rewrite the `segment_id` recorded for `pod_name` to
|
||||
/// `new_segment_id`.
|
||||
///
|
||||
/// The Pod's in-memory `session_id` can change underneath the
|
||||
/// The Pod's in-memory `segment_id` can change underneath the
|
||||
/// allocation in two normal places:
|
||||
///
|
||||
/// - `Pod::compact` mints a fresh session and swaps it in.
|
||||
/// - `session_store::ensure_head_or_fork` auto-forks when another
|
||||
/// writer has advanced the store head behind our back.
|
||||
///
|
||||
/// Both paths must call this so subsequent [`lookup_session`] queries
|
||||
/// Both paths must call this so subsequent [`lookup_segment`] queries
|
||||
/// find the live session id, not the old one. Without this update a
|
||||
/// concurrent `restore_from_manifest(new_id)` would see "no live
|
||||
/// writer" and proceed to register a competing allocation on the
|
||||
/// session this Pod just moved into.
|
||||
///
|
||||
/// The lock is opened once and the allocation is rewritten inside the
|
||||
/// guard, so the session_id collision check is atomic with the
|
||||
/// guard, so the segment_id collision check is atomic with the
|
||||
/// rewrite.
|
||||
pub fn update_session(pod_name: &str, new_session_id: SessionId) -> Result<(), ScopeLockError> {
|
||||
pub fn update_segment(pod_name: &str, new_segment_id: SegmentId) -> Result<(), ScopeLockError> {
|
||||
let lock_path = default_registry_path()?;
|
||||
let mut guard = LockFileGuard::open(&lock_path)?;
|
||||
if let Some(other) = guard.data().find_by_session(new_session_id) {
|
||||
if let Some(other) = guard.data().find_by_segment(new_segment_id) {
|
||||
if other.pod_name != pod_name {
|
||||
return Err(ScopeLockError::SessionConflict {
|
||||
session_id: new_session_id,
|
||||
return Err(ScopeLockError::SegmentConflict {
|
||||
segment_id: new_segment_id,
|
||||
pod_name: other.pod_name.clone(),
|
||||
socket: other.socket.clone(),
|
||||
});
|
||||
|
|
@ -142,7 +142,7 @@ pub fn update_session(pod_name: &str, new_session_id: SessionId) -> Result<(), S
|
|||
.data_mut()
|
||||
.find_mut(pod_name)
|
||||
.ok_or_else(|| ScopeLockError::UnknownPod(pod_name.into()))?;
|
||||
alloc.session_id = Some(new_session_id);
|
||||
alloc.segment_id = Some(new_segment_id);
|
||||
guard.save()?;
|
||||
Ok(())
|
||||
}
|
||||
|
|
@ -150,25 +150,25 @@ pub fn update_session(pod_name: &str, new_session_id: SessionId) -> Result<(), S
|
|||
/// Information about a Pod that currently holds an allocation for a
|
||||
/// given session.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SessionLockInfo {
|
||||
pub struct SegmentLockInfo {
|
||||
pub pod_name: String,
|
||||
pub socket: PathBuf,
|
||||
pub pid: u32,
|
||||
}
|
||||
|
||||
/// Open the default lock file, reclaim stale entries, and return the
|
||||
/// allocation currently writing to `session_id`, if any.
|
||||
/// allocation currently writing to `segment_id`, if any.
|
||||
///
|
||||
/// Used by `Pod::restore_from_manifest` to refuse a resume that would
|
||||
/// race a live writer on the same source session.
|
||||
pub fn lookup_session(session_id: SessionId) -> Result<Option<SessionLockInfo>, ScopeLockError> {
|
||||
pub fn lookup_segment(segment_id: SegmentId) -> Result<Option<SegmentLockInfo>, ScopeLockError> {
|
||||
let lock_path = default_registry_path()?;
|
||||
let mut guard = LockFileGuard::open(&lock_path)?;
|
||||
crate::mutate::reclaim_stale(&mut guard);
|
||||
Ok(guard
|
||||
.data()
|
||||
.find_by_session(session_id)
|
||||
.map(|a| SessionLockInfo {
|
||||
.find_by_segment(segment_id)
|
||||
.map(|a| SegmentLockInfo {
|
||||
pod_name: a.pod_name.clone(),
|
||||
socket: a.socket.clone(),
|
||||
pid: a.pid,
|
||||
|
|
@ -193,7 +193,7 @@ mod tests {
|
|||
scope_allow: vec![write_rule("/tmp/child", true)],
|
||||
scope_deny: Vec::new(),
|
||||
delegated_from: None,
|
||||
session_id: None,
|
||||
segment_id: None,
|
||||
});
|
||||
g.save().unwrap();
|
||||
}
|
||||
|
|
@ -267,12 +267,12 @@ mod tests {
|
|||
s,
|
||||
)
|
||||
.unwrap();
|
||||
let info = lookup_session(s).unwrap().expect("expected live writer");
|
||||
let info = lookup_segment(s).unwrap().expect("expected live writer");
|
||||
assert_eq!(info.pod_name, "live");
|
||||
assert_eq!(info.socket, sock("live"));
|
||||
drop(guard);
|
||||
// After the guard's release, the lookup goes back to None.
|
||||
assert!(lookup_session(s).unwrap().is_none());
|
||||
assert!(lookup_segment(s).unwrap().is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -289,10 +289,10 @@ mod tests {
|
|||
original,
|
||||
)
|
||||
.unwrap();
|
||||
update_session("p", updated).unwrap();
|
||||
update_segment("p", updated).unwrap();
|
||||
// lookup against the original is now empty, the updated id wins.
|
||||
assert!(lookup_session(original).unwrap().is_none());
|
||||
assert_eq!(lookup_session(updated).unwrap().unwrap().pod_name, "p");
|
||||
assert!(lookup_segment(original).unwrap().is_none());
|
||||
assert_eq!(lookup_segment(updated).unwrap().unwrap().pod_name, "p");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -318,17 +318,17 @@ mod tests {
|
|||
)
|
||||
.unwrap();
|
||||
// `a` cannot adopt b's live session id.
|
||||
let err = update_session("a", s_b).unwrap_err();
|
||||
let err = update_segment("a", s_b).unwrap_err();
|
||||
match err {
|
||||
ScopeLockError::SessionConflict {
|
||||
ScopeLockError::SegmentConflict {
|
||||
pod_name,
|
||||
session_id,
|
||||
segment_id,
|
||||
..
|
||||
} => {
|
||||
assert_eq!(pod_name, "b");
|
||||
assert_eq!(session_id, s_b);
|
||||
assert_eq!(segment_id, s_b);
|
||||
}
|
||||
other => panic!("expected SessionConflict, got {other:?}"),
|
||||
other => panic!("expected SegmentConflict, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ use std::io;
|
|||
use std::path::PathBuf;
|
||||
|
||||
use manifest::{Permission, ScopeRule};
|
||||
use session_store::SessionId;
|
||||
use session_store::SegmentId;
|
||||
|
||||
use crate::conflict::{find_conflict_owner, find_conflict_owners, is_within_effective_write};
|
||||
use crate::error::ScopeLockError;
|
||||
|
|
@ -16,7 +16,7 @@ use crate::table::{Allocation, LockFileGuard};
|
|||
/// conflicts so a crashed Pod's allocation doesn't block the new one.
|
||||
///
|
||||
/// Rejects when another live allocation is already writing to
|
||||
/// `session_id`, so two `restore_from_manifest` calls under different
|
||||
/// `segment_id`, so two `restore_from_manifest` calls under different
|
||||
/// `pod_name`s cannot both grab the same session log.
|
||||
pub fn register_pod(
|
||||
guard: &mut LockFileGuard,
|
||||
|
|
@ -24,7 +24,7 @@ pub fn register_pod(
|
|||
pid: u32,
|
||||
socket: PathBuf,
|
||||
scope_allow: Vec<ScopeRule>,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<(), ScopeLockError> {
|
||||
register_pod_with_deny(
|
||||
guard,
|
||||
|
|
@ -33,7 +33,7 @@ pub fn register_pod(
|
|||
socket,
|
||||
scope_allow,
|
||||
Vec::new(),
|
||||
session_id,
|
||||
segment_id,
|
||||
)
|
||||
}
|
||||
|
||||
|
|
@ -56,15 +56,15 @@ pub fn register_pod_with_deny(
|
|||
socket: PathBuf,
|
||||
scope_allow: Vec<ScopeRule>,
|
||||
scope_deny: Vec<ScopeRule>,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<(), ScopeLockError> {
|
||||
reclaim_stale(guard);
|
||||
if guard.data().find(&pod_name).is_some() {
|
||||
return Err(ScopeLockError::DuplicatePodName(pod_name));
|
||||
}
|
||||
if let Some(existing) = guard.data().find_by_session(session_id) {
|
||||
return Err(ScopeLockError::SessionConflict {
|
||||
session_id,
|
||||
if let Some(existing) = guard.data().find_by_segment(segment_id) {
|
||||
return Err(ScopeLockError::SegmentConflict {
|
||||
segment_id,
|
||||
pod_name: existing.pod_name.clone(),
|
||||
socket: existing.socket.clone(),
|
||||
});
|
||||
|
|
@ -99,7 +99,7 @@ pub fn register_pod_with_deny(
|
|||
scope_allow,
|
||||
scope_deny,
|
||||
delegated_from: None,
|
||||
session_id: Some(session_id),
|
||||
segment_id: Some(segment_id),
|
||||
});
|
||||
guard.save()?;
|
||||
Ok(())
|
||||
|
|
@ -147,9 +147,9 @@ pub fn delegate_scope(
|
|||
scope_allow,
|
||||
scope_deny: Vec::new(),
|
||||
delegated_from: Some(spawner.into()),
|
||||
// Pre-reservation. The child fills in its own session_id when
|
||||
// Pre-reservation. The child fills in its own segment_id when
|
||||
// it calls `adopt_allocation` after the worker is built.
|
||||
session_id: None,
|
||||
segment_id: None,
|
||||
});
|
||||
guard.save()?;
|
||||
Ok(())
|
||||
|
|
@ -178,6 +178,55 @@ pub fn release_pod(guard: &mut LockFileGuard, pod_name: &str) -> Result<(), Scop
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// Reclaim a child delegation back into its parent allocation.
|
||||
///
|
||||
/// This is idempotent: missing child allocations and missing deny entries are
|
||||
/// ignored. For each delegated Write rule, at most one exact matching deny rule
|
||||
/// is removed from the parent's `scope_deny`, preserving any duplicate explicit
|
||||
/// base deny that was not owned by this child delegation.
|
||||
pub fn reclaim_delegated_scope(
|
||||
guard: &mut LockFileGuard,
|
||||
parent: &str,
|
||||
child: &str,
|
||||
delegated_scope: &[ScopeRule],
|
||||
) -> Result<(), ScopeLockError> {
|
||||
let child_idx = guard
|
||||
.data()
|
||||
.allocations
|
||||
.iter()
|
||||
.position(|a| a.pod_name == child);
|
||||
let removed_child_parent = child_idx
|
||||
.map(|idx| guard.data().allocations[idx].delegated_from.clone())
|
||||
.unwrap_or(None);
|
||||
|
||||
let child_exists = child_idx.is_some();
|
||||
|
||||
if child_exists {
|
||||
if let Some(parent_alloc) = guard.data_mut().find_mut(parent) {
|
||||
for rule in delegated_scope
|
||||
.iter()
|
||||
.filter(|rule| rule.permission == Permission::Write)
|
||||
{
|
||||
if let Some(idx) = parent_alloc.scope_deny.iter().position(|deny| deny == rule) {
|
||||
parent_alloc.scope_deny.remove(idx);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(idx) = child_idx {
|
||||
for alloc in guard.data_mut().allocations.iter_mut() {
|
||||
if alloc.delegated_from.as_deref() == Some(child) {
|
||||
alloc.delegated_from.clone_from(&removed_child_parent);
|
||||
}
|
||||
}
|
||||
guard.data_mut().allocations.remove(idx);
|
||||
}
|
||||
|
||||
guard.save()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Remove allocations whose PID is dead, reparenting children to the
|
||||
/// dead Pod's `delegated_from`. Idempotent and best-effort — I/O
|
||||
/// errors on save are swallowed so a crashed Pod's entry never blocks
|
||||
|
|
@ -436,6 +485,46 @@ mod tests {
|
|||
assert!(g.data().find("b").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reclaim_delegated_scope_removes_child_and_one_parent_deny_layer() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let path = dir.path().join("pods.json");
|
||||
let mut g = open_empty(&path);
|
||||
let delegated_rule = write_rule("/src/core", true);
|
||||
register_pod_with_deny(
|
||||
&mut g,
|
||||
"a".into(),
|
||||
std::process::id(),
|
||||
sock("a"),
|
||||
vec![write_rule("/src", true)],
|
||||
vec![delegated_rule.clone(), delegated_rule.clone()],
|
||||
sid(),
|
||||
)
|
||||
.unwrap();
|
||||
register_pod(
|
||||
&mut g,
|
||||
"b".into(),
|
||||
std::process::id(),
|
||||
sock("b"),
|
||||
vec![delegated_rule.clone()],
|
||||
sid(),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
reclaim_delegated_scope(&mut g, "a", "b", std::slice::from_ref(&delegated_rule)).unwrap();
|
||||
let a = g.data().find("a").unwrap();
|
||||
assert_eq!(a.scope_deny, vec![delegated_rule.clone()]);
|
||||
assert!(g.data().find("b").is_none());
|
||||
|
||||
reclaim_delegated_scope(&mut g, "a", "b", &[delegated_rule.clone()]).unwrap();
|
||||
let a = g.data().find("a").unwrap();
|
||||
assert_eq!(
|
||||
a.scope_deny,
|
||||
vec![delegated_rule],
|
||||
"a repeated reclaim with no child allocation must not broaden an explicit duplicate base deny"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reclaim_stale_reparents_and_removes_dead_entries() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
|
|
@ -587,8 +676,8 @@ mod tests {
|
|||
shared_session,
|
||||
)
|
||||
.unwrap();
|
||||
// Second registration tries to grab the same session_id under
|
||||
// a different pod_name. Without the SessionConflict check both
|
||||
// Second registration tries to grab the same segment_id under
|
||||
// a different pod_name. Without the SegmentConflict check both
|
||||
// would succeed and race on the same jsonl.
|
||||
let err = register_pod(
|
||||
&mut g,
|
||||
|
|
@ -600,15 +689,15 @@ mod tests {
|
|||
)
|
||||
.unwrap_err();
|
||||
match err {
|
||||
ScopeLockError::SessionConflict {
|
||||
session_id,
|
||||
ScopeLockError::SegmentConflict {
|
||||
segment_id,
|
||||
pod_name,
|
||||
..
|
||||
} => {
|
||||
assert_eq!(session_id, shared_session);
|
||||
assert_eq!(segment_id, shared_session);
|
||||
assert_eq!(pod_name, "first");
|
||||
}
|
||||
other => panic!("expected SessionConflict, got {other:?}"),
|
||||
other => panic!("expected SegmentConflict, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ use std::path::{Path, PathBuf};
|
|||
use fs4::fs_std::FileExt;
|
||||
use manifest::{ScopeRule, paths};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use session_store::SessionId;
|
||||
use session_store::SegmentId;
|
||||
|
||||
/// On-disk representation of the allocation table.
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
|
|
@ -43,12 +43,12 @@ pub struct Allocation {
|
|||
/// Name of the Pod that delegated scope to this one, or `None` for
|
||||
/// a top-level Pod started directly by a human.
|
||||
pub delegated_from: Option<String>,
|
||||
/// Session ID this Pod is currently writing to. `None` means this
|
||||
/// Segment ID this Pod is currently writing to. `None` means this
|
||||
/// is a pre-reservation made by a spawner via [`crate::delegate_scope`]
|
||||
/// before the child has come up; the child fills it in at
|
||||
/// [`crate::adopt_allocation`] time.
|
||||
#[serde(default)]
|
||||
pub session_id: Option<SessionId>,
|
||||
pub segment_id: Option<SegmentId>,
|
||||
}
|
||||
|
||||
impl LockFile {
|
||||
|
|
@ -60,12 +60,12 @@ impl LockFile {
|
|||
self.allocations.iter_mut().find(|a| a.pod_name == pod_name)
|
||||
}
|
||||
|
||||
/// Find the allocation currently writing to `session_id`. Skips
|
||||
/// pre-reservations whose `session_id` is still `None`.
|
||||
pub fn find_by_session(&self, session_id: SessionId) -> Option<&Allocation> {
|
||||
/// Find the allocation currently writing to `segment_id`. Skips
|
||||
/// pre-reservations whose `segment_id` is still `None`.
|
||||
pub fn find_by_segment(&self, segment_id: SegmentId) -> Option<&Allocation> {
|
||||
self.allocations
|
||||
.iter()
|
||||
.find(|a| a.session_id == Some(session_id))
|
||||
.find(|a| a.segment_id == Some(segment_id))
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -225,8 +225,8 @@ mod tests {
|
|||
let dir = TempDir::new().unwrap();
|
||||
let path = dir.path().join("pods.json");
|
||||
let mut g = open_empty(&path);
|
||||
// Pre-reservation: delegate_scope leaves session_id = None
|
||||
// until adopt_allocation rewrites it. find_by_session must not
|
||||
// Pre-reservation: delegate_scope leaves segment_id = None
|
||||
// until adopt_allocation rewrites it. find_by_segment must not
|
||||
// match those placeholders, otherwise a freshly-spawning child
|
||||
// would shadow itself before it has even chosen a session.
|
||||
register_pod(
|
||||
|
|
@ -249,13 +249,13 @@ mod tests {
|
|||
.unwrap();
|
||||
|
||||
let target_session = sid();
|
||||
// The placeholder allocation has session_id = None and must
|
||||
// The placeholder allocation has segment_id = None and must
|
||||
// not be returned for any lookup.
|
||||
assert!(g.data().find_by_session(target_session).is_none());
|
||||
assert!(g.data().find_by_segment(target_session).is_none());
|
||||
|
||||
// After adopt-style rewrite, the same allocation is now found.
|
||||
g.data_mut().find_mut("child").unwrap().session_id = Some(target_session);
|
||||
let found = g.data().find_by_session(target_session).unwrap();
|
||||
g.data_mut().find_mut("child").unwrap().segment_id = Some(target_session);
|
||||
let found = g.data().find_by_segment(target_session).unwrap();
|
||||
assert_eq!(found.pod_name, "child");
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -6,12 +6,12 @@ use std::path::{Path, PathBuf};
|
|||
use std::sync::{LazyLock, Mutex, MutexGuard};
|
||||
|
||||
use manifest::{Permission, ScopeRule};
|
||||
use session_store::SessionId;
|
||||
use session_store::SegmentId;
|
||||
|
||||
use crate::table::LockFileGuard;
|
||||
|
||||
pub(crate) fn sid() -> SessionId {
|
||||
session_store::new_session_id()
|
||||
pub(crate) fn sid() -> SegmentId {
|
||||
session_store::new_segment_id()
|
||||
}
|
||||
|
||||
/// Serialises tests that mutate runtime-dir env vars. The test
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ memory = { workspace = true }
|
|||
workflow-crate = { package = "workflow", path = "../workflow" }
|
||||
uuid = { workspace = true, features = ["v7"] }
|
||||
session-metrics = { workspace = true }
|
||||
parking_lot = "0.12.5"
|
||||
arc-swap = "1.9.1"
|
||||
|
||||
[dev-dependencies]
|
||||
dotenv = "0.15.0"
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|||
let mut pod = Pod::from_manifest_toml(&toml, store).await?;
|
||||
let manifest: &PodManifest = pod.manifest();
|
||||
println!("Pod: {}", manifest.pod.name);
|
||||
println!("Session: {}", pod.session_id());
|
||||
println!("Segment: {}", pod.segment_id());
|
||||
|
||||
// 4. Run a prompt
|
||||
let result = pod.run_text("What is the capital of France?").await?;
|
||||
|
|
@ -76,7 +76,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|||
}
|
||||
|
||||
// 6. Session ID for potential restore
|
||||
println!("\nSession ID: {}", pod.session_id());
|
||||
println!("\nSegment ID: {}", pod.segment_id());
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
|
|
|||
|
|
@ -13,19 +13,24 @@
|
|||
|
||||
use llm_worker::Item;
|
||||
use llm_worker::llm_client::client::LlmClient;
|
||||
use llm_worker::prune::{PruneConfig, PruneDecision, PruneObserver, SavingsEstimator};
|
||||
use llm_worker::prune::{
|
||||
PruneConfig, PruneDecision, PruneObserver, SavingsEstimator, TokenEstimator,
|
||||
};
|
||||
use session_metrics::Metric;
|
||||
use session_store::Store;
|
||||
|
||||
use crate::Pod;
|
||||
use crate::compact::token_counter::{EstimateSource, savings_for_prune_impl};
|
||||
use crate::compact::token_counter::{
|
||||
EstimateSource, savings_for_prune_impl, token_estimates_for_prune_impl,
|
||||
};
|
||||
|
||||
impl<C: LlmClient, St: Store> Pod<C, St> {
|
||||
/// Enable prune projection on the underlying Worker.
|
||||
///
|
||||
/// Registers the config and a savings-estimator closure on the Worker.
|
||||
/// The estimator captures a shared handle to [`Pod::usage_history_handle`]
|
||||
/// so that every LLM request sees the latest measurements.
|
||||
/// Registers the config and token/savings-estimator closures on the Worker.
|
||||
/// The estimators combine persisted [`Pod::usage_history_handle`] records
|
||||
/// with in-flight `UsageTracker` records so multi-request tool loops can
|
||||
/// prune before the surrounding Pod run finishes.
|
||||
///
|
||||
/// Measurement-less estimates (before the first LLM call, or immediately
|
||||
/// after a compact) return `0` from the estimator, which naturally
|
||||
|
|
@ -37,9 +42,25 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
|
|||
/// [`UsageTracker`] so the next `LlmUsage` can be paired with a
|
||||
/// `prune.post_request` metric carrying the same id.
|
||||
pub fn attach_prune(&mut self, config: PruneConfig) {
|
||||
let usage = self.usage_history_handle();
|
||||
let usage_history_for_tokens = self.usage_history_handle();
|
||||
let usage_tracker_for_tokens = self.usage_tracker_handle();
|
||||
let token_estimator: TokenEstimator = Box::new(move |history: &[Item]| {
|
||||
let mut snapshot = usage_history_for_tokens
|
||||
.lock()
|
||||
.expect("usage_history poisoned")
|
||||
.clone();
|
||||
snapshot.extend(usage_tracker_for_tokens.records());
|
||||
token_estimates_for_prune_impl(history, &snapshot)
|
||||
});
|
||||
|
||||
let usage_history_for_savings = self.usage_history_handle();
|
||||
let usage_tracker_for_savings = self.usage_tracker_handle();
|
||||
let estimator: SavingsEstimator = Box::new(move |history: &[Item], indices| {
|
||||
let snapshot = usage.lock().expect("usage_history poisoned").clone();
|
||||
let mut snapshot = usage_history_for_savings
|
||||
.lock()
|
||||
.expect("usage_history poisoned")
|
||||
.clone();
|
||||
snapshot.extend(usage_tracker_for_savings.records());
|
||||
let est = savings_for_prune_impl(history, &snapshot, indices);
|
||||
match est.source {
|
||||
EstimateSource::NoData => 0,
|
||||
|
|
@ -56,8 +77,9 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
|
|||
.with_value(eval.estimated_savings as f64)
|
||||
.with_correlation_id(&correlation_id)
|
||||
.with_dimension("candidate_count", eval.candidate_count.to_string());
|
||||
if let Some(border) = eval.border_turn {
|
||||
metric = metric.with_dimension("border_turn", border.to_string());
|
||||
if let Some(protected_start) = eval.protected_start_index {
|
||||
metric =
|
||||
metric.with_dimension("protected_start_index", protected_start.to_string());
|
||||
}
|
||||
metrics.push(metric);
|
||||
usage_tracker.note_correlation_id(correlation_id);
|
||||
|
|
@ -66,17 +88,21 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
|
|||
metrics.push(Metric::now("prune.skip").with_dimension("reason", "no_candidates"));
|
||||
}
|
||||
PruneDecision::SkippedBelowMinSavings => {
|
||||
metrics.push(
|
||||
Metric::now("prune.skip")
|
||||
let mut metric = Metric::now("prune.skip")
|
||||
.with_dimension("reason", "below_min_savings")
|
||||
.with_dimension("candidate_count", eval.candidate_count.to_string())
|
||||
.with_value(eval.estimated_savings as f64),
|
||||
);
|
||||
.with_value(eval.estimated_savings as f64);
|
||||
if let Some(protected_start) = eval.protected_start_index {
|
||||
metric =
|
||||
metric.with_dimension("protected_start_index", protected_start.to_string());
|
||||
}
|
||||
metrics.push(metric);
|
||||
}
|
||||
});
|
||||
|
||||
let worker = self.worker_mut();
|
||||
worker.set_prune_config(Some(config));
|
||||
worker.set_token_estimator(Some(token_estimator));
|
||||
worker.set_savings_estimator(Some(estimator));
|
||||
worker.set_prune_observer(Some(observer));
|
||||
}
|
||||
|
|
@ -90,7 +116,7 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
|
|||
return;
|
||||
};
|
||||
let config = PruneConfig {
|
||||
protected_turns: compaction.prune_protected_turns,
|
||||
protected_tokens: compaction.prune_protected_tokens,
|
||||
min_savings: compaction.prune_min_savings,
|
||||
};
|
||||
self.attach_prune(config);
|
||||
|
|
|
|||
|
|
@ -132,6 +132,21 @@ fn tool_result_content_bytes(item: &Item) -> u64 {
|
|||
item_bytes(item).saturating_sub(item_bytes(&cleared))
|
||||
}
|
||||
|
||||
/// Prefix-boundary token estimates used by Prune to find its protected suffix.
|
||||
///
|
||||
/// Returns `history.len() + 1` entries where entry `i` estimates
|
||||
/// `history[..i]`. This shares the same [`tokens_at`] accounting as compact's
|
||||
/// retained-tail split and prune's savings estimate.
|
||||
pub(crate) fn token_estimates_for_prune_impl(
|
||||
history: &[Item],
|
||||
records: &[UsageRecord],
|
||||
) -> Vec<TokenEstimate> {
|
||||
let prefix = prefix_bytes(history);
|
||||
(0..=history.len())
|
||||
.map(|idx| tokens_at(history, records, idx, &prefix))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Prune 射影(`ToolResult.content = None`)で節約されるトークン数の推定。
|
||||
///
|
||||
/// `indices` は [`llm_worker::prune::prunable_indices`] が返す候補列を
|
||||
|
|
@ -278,6 +293,26 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn token_estimates_for_prune_returns_every_prefix_boundary() {
|
||||
let history = vec![msg("a"), msg("b"), msg("c")];
|
||||
let estimates = token_estimates_for_prune_impl(&history, &[record(3, 300)]);
|
||||
assert_eq!(estimates.len(), history.len() + 1);
|
||||
assert_eq!(estimates[0].tokens, 0);
|
||||
assert_eq!(estimates[3].tokens, 300);
|
||||
assert_eq!(estimates[3].source, EstimateSource::Measured);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn token_estimates_for_prune_propagates_no_data() {
|
||||
let history = vec![msg("a"), msg("b")];
|
||||
let estimates = token_estimates_for_prune_impl(&history, &[]);
|
||||
assert_eq!(estimates.len(), history.len() + 1);
|
||||
assert_eq!(estimates[0].source, EstimateSource::Measured);
|
||||
assert_eq!(estimates[1].source, EstimateSource::NoData);
|
||||
assert_eq!(estimates[2].source, EstimateSource::NoData);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn savings_for_prune_skips_non_toolresult_indices() {
|
||||
let history = vec![msg("a"), msg("b"), msg("c")];
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ use std::sync::Arc;
|
|||
|
||||
use llm_worker::WorkerError;
|
||||
use llm_worker::llm_client::client::LlmClient;
|
||||
use session_store::Store;
|
||||
use session_store::{PodMetadataStore, Store};
|
||||
use tokio::sync::{broadcast, mpsc, oneshot};
|
||||
|
||||
use crate::ipc::alerter::Alerter;
|
||||
|
|
@ -11,7 +11,7 @@ use crate::ipc::notify_buffer::NotifyBuffer;
|
|||
use crate::ipc::server::SocketServer;
|
||||
use crate::pod::{Pod, PodError, PodRunResult, SystemItemCommitter};
|
||||
use crate::runtime::dir::RuntimeDir;
|
||||
use crate::session_log_sink::SessionLogSink;
|
||||
use crate::segment_log_sink::SegmentLogSink;
|
||||
use crate::shared_state::PodSharedState;
|
||||
use crate::spawn::comm_tools::{
|
||||
list_pods_tool, read_pod_output_tool, send_to_pod_tool, stop_pod_tool,
|
||||
|
|
@ -33,10 +33,10 @@ pub struct PodHandle {
|
|||
pub shared_state: Arc<PodSharedState>,
|
||||
pub runtime_dir: Arc<RuntimeDir>,
|
||||
pub alerter: Alerter,
|
||||
/// Session-log mirror + broadcast handle. The IPC server snapshots
|
||||
/// Segment-log mirror + broadcast handle. The IPC server snapshots
|
||||
/// it on every new connection (Event::Snapshot) and forwards
|
||||
/// subsequent commits (Event::Entry) on the receiver.
|
||||
pub sink: SessionLogSink,
|
||||
pub sink: SegmentLogSink,
|
||||
}
|
||||
|
||||
impl PodHandle {
|
||||
|
|
@ -132,7 +132,7 @@ impl PodController {
|
|||
) -> Result<(PodHandle, ShutdownReceiver), std::io::Error>
|
||||
where
|
||||
C: LlmClient + Clone + 'static,
|
||||
St: Store + Clone + 'static,
|
||||
St: Store + PodMetadataStore + Clone + Send + Sync + 'static,
|
||||
{
|
||||
// === 1. Initialization (channels / RuntimeDir / pod-immutable
|
||||
// snapshots / SpawnedPodRegistry / alerter attach /
|
||||
|
|
@ -151,7 +151,20 @@ impl PodController {
|
|||
|
||||
let spawner_name = pod.manifest().pod.name.clone();
|
||||
let self_parent_socket = pod.callback_socket().cloned();
|
||||
let spawned_registry = SpawnedPodRegistry::new(runtime_dir.clone());
|
||||
let loaded_registry = SpawnedPodRegistry::load_from_pod_state_with_reclaim(
|
||||
runtime_dir.clone(),
|
||||
pod.store().clone(),
|
||||
spawner_name.clone(),
|
||||
Some(pod.scope().clone()),
|
||||
Some(pod.scope_change_sink()),
|
||||
)
|
||||
.await?;
|
||||
let reclaimed_unreachable = loaded_registry.reclaimed_unreachable;
|
||||
let spawned_registry = loaded_registry.registry;
|
||||
if reclaimed_unreachable {
|
||||
pod.persist_scope_snapshot()
|
||||
.map_err(std::io::Error::other)?;
|
||||
}
|
||||
|
||||
// Hand the alerter to the Pod so internal operations (compaction,
|
||||
// AGENTS.md ingestion during the first turn) can emit user-facing
|
||||
|
|
@ -214,7 +227,7 @@ impl PodController {
|
|||
let greeting = build_greeting(&pod);
|
||||
let shared_state = Arc::new(PodSharedState::new(
|
||||
pod.manifest().pod.name.clone(),
|
||||
pod.session_id(),
|
||||
pod.segment_id(),
|
||||
manifest_toml.clone(),
|
||||
greeting,
|
||||
));
|
||||
|
|
@ -430,7 +443,7 @@ where
|
|||
let scope_handle = pod.scope().clone();
|
||||
let pwd = pod.pwd().to_path_buf();
|
||||
let task_store = pod.task_store();
|
||||
let session_id_for_usage = pod.session_id().to_string();
|
||||
let session_id_for_usage = pod.segment_id().to_string();
|
||||
let scope_change_sink = pod.scope_change_sink();
|
||||
let memory_config = pod.manifest().memory.clone();
|
||||
let spawner_name = pod.manifest().pod.name.clone();
|
||||
|
|
@ -639,9 +652,7 @@ async fn controller_loop<C, St>(
|
|||
// sees the buffered notification(s) without a human
|
||||
// Run.
|
||||
if shared_state.get_status() == PodStatus::Idle {
|
||||
pending = Some(PendingRun::RunForNotification(
|
||||
protocol::InvokeKind::Notify,
|
||||
));
|
||||
pending = Some(PendingRun::RunForNotification(protocol::InvokeKind::Notify));
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -893,6 +904,10 @@ where
|
|||
// `build_client` がここに到達する前に同じマニフェストで成功している
|
||||
// ため、カタログ解決も必ず通る。念のため失敗時は "unknown" に落とす。
|
||||
let resolved = provider::catalog::resolve_model_manifest(&manifest.model).ok();
|
||||
let context_window = resolved
|
||||
.as_ref()
|
||||
.map(|cfg| cfg.context_window)
|
||||
.unwrap_or(provider::catalog::DEFAULT_CONTEXT_WINDOW);
|
||||
let (provider_name, model_id) = match resolved {
|
||||
Some(cfg) => {
|
||||
let name = match cfg.scheme {
|
||||
|
|
@ -930,6 +945,8 @@ where
|
|||
model: model_id,
|
||||
scope_summary: pod.scope_snapshot().summary(),
|
||||
tools: tool_names,
|
||||
context_window,
|
||||
context_tokens: pod.total_tokens().tokens,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -950,7 +967,7 @@ fn worker_error_code(e: &PodError) -> ErrorCode {
|
|||
mod tests {
|
||||
use super::*;
|
||||
use protocol::PodEvent;
|
||||
use protocol::stream::JsonLineReader;
|
||||
use protocol::stream::{JsonLineReader, JsonLineWriter};
|
||||
use std::time::Duration;
|
||||
use tempfile::TempDir;
|
||||
use tokio::net::UnixListener;
|
||||
|
|
@ -992,7 +1009,7 @@ mod tests {
|
|||
let (cancel_tx, cancel_rx) = mpsc::channel::<()>(1);
|
||||
let shared_state = Arc::new(PodSharedState::new(
|
||||
"child-pod".to_string(),
|
||||
session_store::new_session_id(),
|
||||
session_store::new_segment_id(),
|
||||
String::new(),
|
||||
protocol::Greeting {
|
||||
pod_name: "child-pod".to_string(),
|
||||
|
|
@ -1001,6 +1018,8 @@ mod tests {
|
|||
model: String::new(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
},
|
||||
));
|
||||
let notify_buffer = NotifyBuffer::new();
|
||||
|
|
@ -1028,7 +1047,26 @@ mod tests {
|
|||
async fn recv_pod_event(listener: UnixListener, timeout: Duration) -> Option<PodEvent> {
|
||||
let accept = async {
|
||||
let (stream, _) = listener.accept().await.ok()?;
|
||||
let mut reader = JsonLineReader::new(stream);
|
||||
let (r, w) = stream.into_split();
|
||||
let mut writer = JsonLineWriter::new(w);
|
||||
writer
|
||||
.write(&Event::Snapshot {
|
||||
entries: Vec::new(),
|
||||
greeting: protocol::Greeting {
|
||||
pod_name: "parent".into(),
|
||||
cwd: "/tmp".into(),
|
||||
provider: "test".into(),
|
||||
model: "test".into(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
},
|
||||
status: PodStatus::Idle,
|
||||
})
|
||||
.await
|
||||
.ok()?;
|
||||
let mut reader = JsonLineReader::new(r);
|
||||
match reader.next::<Method>().await {
|
||||
Ok(Some(Method::PodEvent(e))) => Some(e),
|
||||
_ => None,
|
||||
|
|
|
|||
|
|
@ -11,9 +11,9 @@
|
|||
//! happen at the front of `Pod::run` when
|
||||
//! `worker.last_run_interrupted()` is set; see `Pod::apply_interrupt_prep`.
|
||||
|
||||
use llm_worker::Item;
|
||||
#[cfg(test)]
|
||||
use crate::prompt::catalog::PromptCatalog;
|
||||
use llm_worker::Item;
|
||||
|
||||
/// Build synthetic `Item::ToolResult` items for every unanswered
|
||||
/// `Item::ToolCall` in `history`, preserving order.
|
||||
|
|
|
|||
|
|
@ -27,7 +27,6 @@ use std::sync::Arc;
|
|||
use protocol::{Method, PodEvent, ScopeRule};
|
||||
|
||||
use crate::runtime::dir::SpawnedPodRecord;
|
||||
use crate::runtime::pod_registry::{self, ScopeLockError};
|
||||
use crate::spawn::comm_tools::connect_and_send;
|
||||
use crate::spawn::registry::SpawnedPodRegistry;
|
||||
|
||||
|
|
@ -86,8 +85,8 @@ pub fn render_event(event: &PodEvent) -> String {
|
|||
///
|
||||
/// - `TurnEnded` / `Errored`: no system work; the LLM handles the
|
||||
/// semantic response.
|
||||
/// - `ShutDown`: remove the child from `spawned_pods.json` and release
|
||||
/// its scope allocation. Missing entries are swallowed.
|
||||
/// - `ShutDown`: remove the child from `spawned_pods.json`, Pod state,
|
||||
/// and reclaim its delegated scope/allocation. Missing entries are swallowed.
|
||||
/// - `ScopeSubDelegated`: register the grandchild locally and re-emit
|
||||
/// upward to our own parent if we have one. Duplicate grandchild
|
||||
/// entries (re-delivery) are swallowed.
|
||||
|
|
@ -104,7 +103,6 @@ pub async fn apply_event_side_effects(
|
|||
if let Err(e) = registry.remove(pod_name).await {
|
||||
tracing::warn!(error = %e, pod = %pod_name, "registry remove on ShutDown failed");
|
||||
}
|
||||
release_scope_silently(pod_name);
|
||||
}
|
||||
|
||||
PodEvent::ScopeSubDelegated {
|
||||
|
|
@ -145,28 +143,6 @@ pub async fn apply_event_side_effects(
|
|||
}
|
||||
}
|
||||
|
||||
fn release_scope_silently(pod_name: &str) {
|
||||
let lock_path = match pod_registry::default_registry_path() {
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %e, "default_registry_path failed");
|
||||
return;
|
||||
}
|
||||
};
|
||||
let mut guard = match pod_registry::LockFileGuard::open(&lock_path) {
|
||||
Ok(g) => g,
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %e, "LockFileGuard open failed");
|
||||
return;
|
||||
}
|
||||
};
|
||||
match pod_registry::release_pod(&mut guard, pod_name) {
|
||||
Ok(()) => {}
|
||||
Err(ScopeLockError::UnknownPod(_)) => {}
|
||||
Err(e) => tracing::warn!(error = ?e, pod = %pod_name, "release_pod failed"),
|
||||
}
|
||||
}
|
||||
|
||||
fn reemit_scope_sub_delegated(
|
||||
self_parent_socket: &Option<PathBuf>,
|
||||
self_name: &str,
|
||||
|
|
|
|||
|
|
@ -185,7 +185,9 @@ mod tests {
|
|||
let item = build_system_item(&entry, &catalog).unwrap();
|
||||
match item {
|
||||
SystemItem::PodEvent { event, body } => {
|
||||
assert!(matches!(event, PodEvent::TurnEnded { ref pod_name } if pod_name == "child"));
|
||||
assert!(
|
||||
matches!(event, PodEvent::TurnEnded { ref pod_name } if pod_name == "child")
|
||||
);
|
||||
assert!(body.contains("[Notification]"));
|
||||
assert!(body.contains("`child`"));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -105,10 +105,10 @@ async fn handle_connection(stream: tokio::net::UnixStream, handle: PodHandle) {
|
|||
match entry {
|
||||
Ok(entry) => {
|
||||
let outbound = match entry {
|
||||
session_store::LogEntry::SessionStart { .. } => {
|
||||
session_store::LogEntry::SegmentStart { .. } => {
|
||||
let value = serde_json::to_value(&entry)
|
||||
.expect("LogEntry is Serialize");
|
||||
Some(Event::SessionRotated { entry: value })
|
||||
Some(Event::SegmentRotated { entry: value })
|
||||
}
|
||||
session_store::LogEntry::SystemItem { item, .. } => {
|
||||
let value = serde_json::to_value(&item)
|
||||
|
|
@ -119,7 +119,7 @@ async fn handle_connection(stream: tokio::net::UnixStream, handle: PodHandle) {
|
|||
Some(Event::InvokeStart { kind: trigger })
|
||||
}
|
||||
other => {
|
||||
// `SessionLogSink::is_live_relevant` keeps
|
||||
// `SegmentLogSink::is_live_relevant` keeps
|
||||
// non-live-relevant variants off the
|
||||
// broadcast lane; reaching here means the
|
||||
// two are out of sync and we silently
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ pub mod hook;
|
|||
pub mod ipc;
|
||||
pub mod prompt;
|
||||
pub mod runtime;
|
||||
pub mod session_log_sink;
|
||||
pub mod segment_log_sink;
|
||||
pub mod shared_state;
|
||||
pub mod spawn;
|
||||
pub mod workflow;
|
||||
|
|
@ -31,5 +31,5 @@ pub use prompt::system::{SystemPromptContext, SystemPromptError, SystemPromptTem
|
|||
pub use protocol::{ErrorCode, Event, Method, PodStatus, TurnResult};
|
||||
pub use provider::{ProviderError, build_client};
|
||||
pub use runtime::dir::RuntimeDir;
|
||||
pub use session_log_sink::SessionLogSink;
|
||||
pub use segment_log_sink::SegmentLogSink;
|
||||
pub use shared_state::PodSharedState;
|
||||
|
|
|
|||
|
|
@ -3,9 +3,9 @@ use std::path::{Path, PathBuf};
|
|||
use std::process::ExitCode;
|
||||
|
||||
use clap::Parser;
|
||||
use manifest::{PodManifest, paths};
|
||||
use manifest::{PodManifest, PodManifestConfig, paths};
|
||||
use pod::{Pod, PodController, PodFactory, PromptLoader};
|
||||
use session_store::{FsStore, SessionId};
|
||||
use session_store::{FsStore, PodMetadataStore, SegmentId, Store};
|
||||
|
||||
const USER_MANIFEST_ENV: &str = "INSOMNIA_USER_MANIFEST";
|
||||
|
||||
|
|
@ -47,13 +47,19 @@ struct Cli {
|
|||
#[arg(long, value_name = "PATH", requires = "adopt")]
|
||||
callback: Option<PathBuf>,
|
||||
|
||||
/// Resume or create a Pod by name. If name-keyed Pod state exists,
|
||||
/// the active session/segment recorded there is restored; otherwise a
|
||||
/// fresh top-level Pod is created with this name.
|
||||
#[arg(long, value_name = "NAME", conflicts_with_all = ["session", "adopt"])]
|
||||
pod: Option<String>,
|
||||
|
||||
/// Restore a Pod from an existing session. The Pod re-uses the
|
||||
/// given session id and appends new turns to the same jsonl;
|
||||
/// concurrent writers are prevented by the pod-registry.
|
||||
/// Mutually exclusive with `--adopt` (spawned children always start
|
||||
/// fresh).
|
||||
#[arg(long, value_name = "UUID", conflicts_with = "adopt")]
|
||||
session: Option<SessionId>,
|
||||
#[arg(long, value_name = "UUID", conflicts_with_all = ["adopt", "pod"])]
|
||||
session: Option<SegmentId>,
|
||||
}
|
||||
|
||||
fn resolve_manifest(cli: &Cli) -> Result<(PodManifest, PromptLoader), String> {
|
||||
|
|
@ -72,7 +78,7 @@ fn resolve_manifest_with_user_manifest_env(
|
|||
"--manifest cannot be used when {USER_MANIFEST_ENV} is set"
|
||||
));
|
||||
}
|
||||
return load_single_manifest(path);
|
||||
return load_single_manifest(path, cli.pod.as_deref());
|
||||
}
|
||||
|
||||
let factory = build_factory_with_user_manifest_path(cli, user_manifest)?;
|
||||
|
|
@ -91,14 +97,45 @@ fn user_manifest_path_from_env(value: Option<OsString>) -> Option<PathBuf> {
|
|||
})
|
||||
}
|
||||
|
||||
fn load_single_manifest(path: &Path) -> Result<(PodManifest, PromptLoader), String> {
|
||||
fn load_single_manifest(
|
||||
path: &Path,
|
||||
pod_name_override: Option<&str>,
|
||||
) -> Result<(PodManifest, PromptLoader), String> {
|
||||
let toml = std::fs::read_to_string(path)
|
||||
.map_err(|e| format!("failed to read manifest {}: {e}", path.display()))?;
|
||||
let manifest = PodManifest::from_toml(&toml)
|
||||
let manifest = match pod_name_override {
|
||||
Some(pod_name) => match PodManifest::from_toml(&toml) {
|
||||
Ok(mut manifest) => {
|
||||
manifest.pod.name = pod_name.to_string();
|
||||
manifest
|
||||
}
|
||||
Err(_) => {
|
||||
let base = PodManifestConfig::from_toml(&toml)
|
||||
.map_err(|e| format!("failed to parse manifest {}: {e}", path.display()))?;
|
||||
let overlay = PodManifestConfig::from_toml(&pod_name_overlay_toml(pod_name))
|
||||
.expect("pod name overlay TOML is generated");
|
||||
PodManifest::try_from(base.merge(overlay)).map_err(|e| {
|
||||
format!(
|
||||
"failed to resolve manifest {} with --pod: {e}",
|
||||
path.display()
|
||||
)
|
||||
})?
|
||||
}
|
||||
},
|
||||
None => PodManifest::from_toml(&toml)
|
||||
.map_err(|e| format!("failed to parse manifest {}: {e}", path.display()))?,
|
||||
};
|
||||
Ok((manifest, PromptLoader::builtins_only()))
|
||||
}
|
||||
|
||||
fn pod_name_overlay_toml(pod_name: &str) -> String {
|
||||
let mut pod = toml::value::Table::new();
|
||||
pod.insert("name".into(), toml::Value::String(pod_name.to_string()));
|
||||
let mut root = toml::value::Table::new();
|
||||
root.insert("pod".into(), toml::Value::Table(pod));
|
||||
toml::to_string(&toml::Value::Table(root)).expect("pod name overlay serialisation cannot fail")
|
||||
}
|
||||
|
||||
fn build_factory_with_user_manifest_path(
|
||||
cli: &Cli,
|
||||
user_manifest: Option<PathBuf>,
|
||||
|
|
@ -129,6 +166,12 @@ fn build_factory_with_user_manifest_path(
|
|||
.map_err(|e| format!("failed to parse overlay TOML: {e}"))?;
|
||||
}
|
||||
|
||||
if let Some(pod_name) = cli.pod.as_deref() {
|
||||
factory = factory
|
||||
.with_overlay_toml(&pod_name_overlay_toml(pod_name))
|
||||
.map_err(|e| format!("failed to apply --pod overlay: {e}"))?;
|
||||
}
|
||||
|
||||
Ok(factory)
|
||||
}
|
||||
|
||||
|
|
@ -136,7 +179,7 @@ fn build_factory_with_user_manifest_path(
|
|||
async fn main() -> ExitCode {
|
||||
let cli = Cli::parse();
|
||||
|
||||
let (manifest, loader) = match resolve_manifest(&cli) {
|
||||
let (mut manifest, loader) = match resolve_manifest(&cli) {
|
||||
Ok(pair) => pair,
|
||||
Err(e) => {
|
||||
eprintln!("error: {e}");
|
||||
|
|
@ -185,14 +228,59 @@ async fn main() -> ExitCode {
|
|||
return ExitCode::FAILURE;
|
||||
}
|
||||
}
|
||||
} else if let Some(source_session_id) = cli.session {
|
||||
match Pod::restore_from_manifest(source_session_id, manifest, store, loader).await {
|
||||
} else if let Some(source_segment_id) = cli.session {
|
||||
let source_session_id = match store.lookup_session_of(source_segment_id) {
|
||||
Ok(Some(sid)) => sid,
|
||||
Ok(None) => {
|
||||
eprintln!(
|
||||
"error: --session {source_segment_id}: segment is not registered to any session"
|
||||
);
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("error: lookup_session_of failed: {e}");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
};
|
||||
match Pod::restore_from_manifest(
|
||||
source_session_id,
|
||||
source_segment_id,
|
||||
manifest,
|
||||
store,
|
||||
loader,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
eprintln!("error: failed to restore pod: {e}");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
}
|
||||
} else if let Some(pod_name) = cli.pod.as_deref() {
|
||||
manifest.pod.name = pod_name.to_string();
|
||||
match store.read_by_name(pod_name) {
|
||||
Ok(Some(_)) => {
|
||||
match Pod::restore_from_pod_metadata(pod_name, manifest, store, loader).await {
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
eprintln!("error: failed to restore pod {pod_name}: {e}");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(None) => match Pod::from_manifest(manifest, store, loader).await {
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
eprintln!("error: failed to create pod {pod_name}: {e}");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
},
|
||||
Err(e) => {
|
||||
eprintln!("error: failed to read pod state for {pod_name}: {e}");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
match Pod::from_manifest(manifest, store, loader).await {
|
||||
Ok(p) => p,
|
||||
|
|
@ -348,6 +436,56 @@ permission = "write"
|
|||
assert_eq!(manifest.pod.name, "from-env");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pod_flag_conflicts_with_session() {
|
||||
let segment_id = session_store::new_segment_id();
|
||||
let segment_id = segment_id.to_string();
|
||||
let err =
|
||||
Cli::try_parse_from(["pod", "--pod", "agent", "--session", &segment_id]).unwrap_err();
|
||||
assert_eq!(err.kind(), clap::error::ErrorKind::ArgumentConflict);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pod_flag_sets_requested_name_after_manifest_resolution() {
|
||||
let tmp = TempDir::new().unwrap();
|
||||
let manifest = tmp.path().join("manifest.toml");
|
||||
write(&manifest, &manifest_toml("from-file", tmp.path()));
|
||||
let cli = Cli::try_parse_from([
|
||||
"pod",
|
||||
"--manifest",
|
||||
manifest.to_str().unwrap(),
|
||||
"--pod",
|
||||
"from-flag",
|
||||
])
|
||||
.unwrap();
|
||||
|
||||
let (manifest, _loader) = resolve_manifest_with_user_manifest_env(&cli, None).unwrap();
|
||||
|
||||
assert_eq!(manifest.pod.name, "from-flag");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pod_flag_supplies_missing_name_for_single_manifest() {
|
||||
let tmp = TempDir::new().unwrap();
|
||||
let manifest = tmp.path().join("manifest.toml");
|
||||
write(
|
||||
&manifest,
|
||||
&manifest_toml("unused", tmp.path()).replace("name = \"unused\"\n", ""),
|
||||
);
|
||||
let cli = Cli::try_parse_from([
|
||||
"pod",
|
||||
"--manifest",
|
||||
manifest.to_str().unwrap(),
|
||||
"--pod",
|
||||
"from-flag",
|
||||
])
|
||||
.unwrap();
|
||||
|
||||
let (manifest, _loader) = resolve_manifest_with_user_manifest_env(&cli, None).unwrap();
|
||||
|
||||
assert_eq!(manifest.pod.name, "from-flag");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn manifest_mode_loads_single_file_with_minimal_prompt_loader() {
|
||||
let tmp = TempDir::new().unwrap();
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -7,11 +7,11 @@ use tokio::fs;
|
|||
|
||||
use crate::shared_state::PodSharedState;
|
||||
|
||||
/// One spawned-child record persisted to `spawned_pods.json`.
|
||||
/// One spawned-child record mirrored to `spawned_pods.json`.
|
||||
///
|
||||
/// Written by the spawner after a successful `SpawnPod` tool call so
|
||||
/// `ListPods` (future ticket) and a restored spawner can enumerate
|
||||
/// their live children without re-querying `pods.json`.
|
||||
/// Written by the spawner after registry changes so runtime-local tools
|
||||
/// have a materialised snapshot. Durable restore uses Pod state metadata;
|
||||
/// this file is not the authoritative source.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct SpawnedPodRecord {
|
||||
/// Spawned Pod's identity.
|
||||
|
|
@ -131,7 +131,7 @@ mod tests {
|
|||
fn test_state() -> PodSharedState {
|
||||
PodSharedState::new(
|
||||
"test-pod".into(),
|
||||
session_store::new_session_id(),
|
||||
session_store::new_segment_id(),
|
||||
"[pod]\nname = \"test-pod\"".into(),
|
||||
protocol::Greeting {
|
||||
pod_name: "test-pod".into(),
|
||||
|
|
@ -140,6 +140,8 @@ mod tests {
|
|||
model: "claude".into(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -10,11 +10,11 @@
|
|||
//! Atomicity contract:
|
||||
//!
|
||||
//! 1. Pod writes the entry to disk via the `Store`.
|
||||
//! 2. Pod calls [`SessionLogSink::publish`] which acquires the mirror
|
||||
//! 2. Pod calls [`SegmentLogSink::publish`] which acquires the mirror
|
||||
//! mutex, pushes the entry, and fires `broadcast::send` — all under
|
||||
//! the same critical section.
|
||||
//!
|
||||
//! [`SessionLogSink::subscribe_with_snapshot`] takes the same mutex,
|
||||
//! [`SegmentLogSink::subscribe_with_snapshot`] takes the same mutex,
|
||||
//! so the `(snapshot, receiver)` pair returned to a connecting client
|
||||
//! splits the entry sequence cleanly: every entry shows up in exactly
|
||||
//! one of `snapshot` or on `receiver`.
|
||||
|
|
@ -39,24 +39,24 @@ const BROADCAST_CAPACITY: usize = 256;
|
|||
/// for read-only `subscribe_with_snapshot` access and keeps one for
|
||||
/// its own write path.
|
||||
#[derive(Clone)]
|
||||
pub struct SessionLogSink {
|
||||
pub struct SegmentLogSink {
|
||||
inner: Arc<SinkInner>,
|
||||
}
|
||||
|
||||
struct SinkInner {
|
||||
/// Full session log mirror in commit order. Reset on session swap
|
||||
/// (compaction / fork) via [`SessionLogSink::reset_with_initial`].
|
||||
/// (compaction / fork) via [`SegmentLogSink::reset_with_initial`].
|
||||
mirror: StdMutex<Vec<LogEntry>>,
|
||||
/// Broadcast channel for live entry updates. The same `Sender`
|
||||
/// survives session swaps so existing subscribers keep their
|
||||
/// receiver — they observe the swap as a freshly broadcast
|
||||
/// `LogEntry::SessionStart` and reset their view accordingly.
|
||||
/// `LogEntry::SegmentStart` and reset their view accordingly.
|
||||
broadcast_tx: broadcast::Sender<LogEntry>,
|
||||
}
|
||||
|
||||
impl SessionLogSink {
|
||||
impl SegmentLogSink {
|
||||
/// Create a fresh sink with an empty mirror. Used before any entry
|
||||
/// has been written (deferred SessionStart) or as a placeholder in
|
||||
/// has been written (deferred SegmentStart) or as a placeholder in
|
||||
/// tests.
|
||||
pub fn new() -> Self {
|
||||
let (broadcast_tx, _) = broadcast::channel(BROADCAST_CAPACITY);
|
||||
|
|
@ -89,7 +89,7 @@ impl SessionLogSink {
|
|||
///
|
||||
/// Live broadcast fires only for entries that the streaming-event
|
||||
/// lane does not cover:
|
||||
/// - `LogEntry::SessionStart` → `Event::SessionRotated` on the wire.
|
||||
/// - `LogEntry::SegmentStart` → `Event::SegmentRotated` on the wire.
|
||||
/// - `LogEntry::SystemItem` → `Event::SystemItem`.
|
||||
/// - `LogEntry::Invoke` → `Event::InvokeStart`.
|
||||
/// Everything else (AssistantItem, ToolResult, UserInput, TurnEnd,
|
||||
|
|
@ -119,20 +119,18 @@ impl SessionLogSink {
|
|||
fn is_live_relevant(entry: &LogEntry) -> bool {
|
||||
matches!(
|
||||
entry,
|
||||
LogEntry::SessionStart { .. }
|
||||
| LogEntry::SystemItem { .. }
|
||||
| LogEntry::Invoke { .. }
|
||||
LogEntry::SegmentStart { .. } | LogEntry::SystemItem { .. } | LogEntry::Invoke { .. }
|
||||
)
|
||||
}
|
||||
|
||||
/// Atomically swap the mirror to `[initial]` and broadcast the new
|
||||
/// session-start entry. Used during compaction / fork: the new
|
||||
/// `LogEntry::SessionStart` is the first entry of the replacement
|
||||
/// `LogEntry::SegmentStart` is the first entry of the replacement
|
||||
/// session, and existing subscribers transition by replaying it
|
||||
/// like any other live entry.
|
||||
///
|
||||
/// Existing snapshot prefixes seen by old subscribers stay valid
|
||||
/// for the prior session; the new `SessionStart` on the broadcast
|
||||
/// for the prior session; the new `SegmentStart` on the broadcast
|
||||
/// is the signal to reset their derived view.
|
||||
pub fn reset_with_initial(&self, initial: LogEntry) {
|
||||
let mut mirror = self
|
||||
|
|
@ -188,7 +186,7 @@ impl SessionLogSink {
|
|||
}
|
||||
}
|
||||
|
||||
impl Default for SessionLogSink {
|
||||
impl Default for SegmentLogSink {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
|
|
@ -198,11 +196,12 @@ impl Default for SessionLogSink {
|
|||
mod tests {
|
||||
use super::*;
|
||||
use llm_worker::llm_client::RequestConfig;
|
||||
use session_store::session_log::now_millis;
|
||||
use session_store::segment_log::now_millis;
|
||||
|
||||
fn session_start() -> LogEntry {
|
||||
LogEntry::SessionStart {
|
||||
LogEntry::SegmentStart {
|
||||
ts: now_millis(),
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -220,13 +219,13 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn publish_then_subscribe_returns_history_in_snapshot() {
|
||||
let sink = SessionLogSink::new();
|
||||
let sink = SegmentLogSink::new();
|
||||
sink.publish(session_start());
|
||||
sink.publish(turn_end(1));
|
||||
|
||||
let (snapshot, mut rx) = sink.subscribe_with_snapshot();
|
||||
assert_eq!(snapshot.len(), 2);
|
||||
assert!(matches!(snapshot[0], LogEntry::SessionStart { .. }));
|
||||
assert!(matches!(snapshot[0], LogEntry::SegmentStart { .. }));
|
||||
assert!(matches!(
|
||||
snapshot[1],
|
||||
LogEntry::TurnEnd { turn_count: 1, .. }
|
||||
|
|
@ -246,7 +245,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn subscribe_then_publish_delivers_only_live_relevant_entries() {
|
||||
let sink = SessionLogSink::new();
|
||||
let sink = SegmentLogSink::new();
|
||||
sink.publish(session_start());
|
||||
|
||||
let (snapshot, mut rx) = sink.subscribe_with_snapshot();
|
||||
|
|
@ -270,7 +269,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn snapshot_and_live_never_overlap() {
|
||||
let sink = SessionLogSink::new();
|
||||
let sink = SegmentLogSink::new();
|
||||
sink.publish(session_start());
|
||||
let (snapshot, mut rx) = sink.subscribe_with_snapshot();
|
||||
sink.publish(notification_entry("post-snapshot"));
|
||||
|
|
@ -285,7 +284,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn reset_with_initial_clears_and_broadcasts() {
|
||||
let sink = SessionLogSink::new();
|
||||
let sink = SegmentLogSink::new();
|
||||
sink.publish(session_start());
|
||||
sink.publish(turn_end(1));
|
||||
|
||||
|
|
@ -293,18 +292,18 @@ mod tests {
|
|||
sink.reset_with_initial(session_start());
|
||||
|
||||
match rx.try_recv() {
|
||||
Ok(LogEntry::SessionStart { .. }) => {}
|
||||
other => panic!("expected SessionStart broadcast, got {other:?}"),
|
||||
Ok(LogEntry::SegmentStart { .. }) => {}
|
||||
other => panic!("expected SegmentStart broadcast, got {other:?}"),
|
||||
}
|
||||
|
||||
let (post_snapshot, _) = sink.subscribe_with_snapshot();
|
||||
assert_eq!(post_snapshot.len(), 1);
|
||||
assert!(matches!(post_snapshot[0], LogEntry::SessionStart { .. }));
|
||||
assert!(matches!(post_snapshot[0], LogEntry::SegmentStart { .. }));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replace_silent_does_not_broadcast() {
|
||||
let sink = SessionLogSink::new();
|
||||
let sink = SegmentLogSink::new();
|
||||
sink.publish(session_start());
|
||||
let (_pre_snapshot, mut rx) = sink.subscribe_with_snapshot();
|
||||
|
||||
|
|
@ -318,7 +317,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn with_initial_seeds_the_mirror() {
|
||||
let sink = SessionLogSink::with_initial(vec![session_start(), turn_end(1)]);
|
||||
let sink = SegmentLogSink::with_initial(vec![session_start(), turn_end(1)]);
|
||||
let (snapshot, _) = sink.subscribe_with_snapshot();
|
||||
assert_eq!(snapshot.len(), 2);
|
||||
}
|
||||
|
|
@ -2,7 +2,7 @@ use std::sync::{OnceLock, RwLock};
|
|||
|
||||
use protocol::PodStatus;
|
||||
use serde_json::json;
|
||||
use session_store::SessionId;
|
||||
use session_store::SegmentId;
|
||||
|
||||
use crate::fs_view::PodFsView;
|
||||
|
||||
|
|
@ -28,7 +28,7 @@ pub struct KnowledgeCandidate {
|
|||
/// greeting, and completion lookup hubs.
|
||||
pub struct PodSharedState {
|
||||
pub pod_name: String,
|
||||
pub session_id: SessionId,
|
||||
pub segment_id: SegmentId,
|
||||
pub manifest_toml: String,
|
||||
pub greeting: protocol::Greeting,
|
||||
pub status: RwLock<PodStatus>,
|
||||
|
|
@ -46,13 +46,13 @@ pub struct PodSharedState {
|
|||
impl PodSharedState {
|
||||
pub fn new(
|
||||
pod_name: String,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
manifest_toml: String,
|
||||
greeting: protocol::Greeting,
|
||||
) -> Self {
|
||||
Self {
|
||||
pod_name,
|
||||
session_id,
|
||||
segment_id,
|
||||
manifest_toml,
|
||||
greeting,
|
||||
status: RwLock::new(PodStatus::Idle),
|
||||
|
|
@ -123,7 +123,7 @@ impl PodSharedState {
|
|||
let status = self.get_status();
|
||||
json!({
|
||||
"state": status,
|
||||
"session_id": self.session_id.to_string(),
|
||||
"segment_id": self.segment_id.to_string(),
|
||||
"pod_name": self.pod_name,
|
||||
})
|
||||
.to_string()
|
||||
|
|
@ -137,7 +137,7 @@ mod tests {
|
|||
fn test_state() -> PodSharedState {
|
||||
PodSharedState::new(
|
||||
"test-pod".into(),
|
||||
session_store::new_session_id(),
|
||||
session_store::new_segment_id(),
|
||||
"[pod]\nname = \"test-pod\"".into(),
|
||||
test_greeting(),
|
||||
)
|
||||
|
|
@ -151,6 +151,8 @@ mod tests {
|
|||
model: "claude".into(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -176,7 +178,7 @@ mod tests {
|
|||
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(parsed["state"], "idle");
|
||||
assert_eq!(parsed["pod_name"], "test-pod");
|
||||
assert!(parsed["session_id"].is_string());
|
||||
assert!(parsed["segment_id"].is_string());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
//! target's Unix socket, perform one method exchange, disconnect.
|
||||
//!
|
||||
//! These tools only touch Pods listed in the spawner's
|
||||
//! `spawned_pods.json`; there is no machine-wide directory lookup, so
|
||||
//! `SpawnedPodRegistry`; there is no machine-wide directory lookup, so
|
||||
//! the spawner can only reach its own descendants.
|
||||
|
||||
use std::path::Path;
|
||||
|
|
@ -204,20 +204,17 @@ impl Tool for StopPodTool {
|
|||
.ok_or_else(|| unknown_pod_err(&input.name))?;
|
||||
|
||||
// Best-effort Shutdown. The child's own `ScopeAllocationGuard`
|
||||
// releases the entry on clean exit; we also release explicitly
|
||||
// below so callers can't observe a window where the scope is
|
||||
// still registered but StopPod has returned. Duplicate release
|
||||
// is harmless — `ScopeAllocationGuard`'s drop path swallows
|
||||
// `UnknownPod` errors.
|
||||
// releases its entry on clean exit; the parent reclaim below is the
|
||||
// authoritative operation for removing the child record and returning
|
||||
// delegated Write scope to the spawner.
|
||||
let _ = connect_and_send(&record.socket_path, &Method::Shutdown).await;
|
||||
|
||||
let scope_summary = summarize_scope(&record);
|
||||
release_scope(&record.pod_name);
|
||||
|
||||
self.registry
|
||||
.remove(&record.pod_name)
|
||||
.await
|
||||
.map_err(|e| ToolError::ExecutionFailed(format!("update spawned_pods.json: {e}")))?;
|
||||
.map_err(|e| ToolError::ExecutionFailed(format!("update spawned pod registry: {e}")))?;
|
||||
|
||||
Ok(ToolOutput {
|
||||
summary: format!(
|
||||
|
|
@ -321,21 +318,52 @@ fn unknown_pod_err(name: &str) -> ToolError {
|
|||
ToolError::InvalidArgument(format!("no spawned pod named `{name}`"))
|
||||
}
|
||||
|
||||
/// Connect with a timeout, write one `Method` line, flush, and close.
|
||||
/// Any socket error maps to an `io::Error`; the caller decides whether
|
||||
/// to surface it to the LLM or treat it as "pod stopped".
|
||||
/// Connect with a timeout, drain the server's connect-time snapshot,
|
||||
/// write one `Method` line, flush, and close.
|
||||
///
|
||||
/// The Pod socket protocol sends replayed alerts and an initial
|
||||
/// `Event::Snapshot` before it starts reading client methods. Send-only
|
||||
/// callers must consume that prefix; otherwise a large snapshot can block
|
||||
/// the server's writer before it reaches the method-read branch. Any
|
||||
/// socket error maps to an `io::Error`; the caller decides whether to
|
||||
/// surface it to the LLM or treat it as "pod stopped".
|
||||
pub(crate) async fn connect_and_send(socket: &Path, method: &Method) -> std::io::Result<()> {
|
||||
let stream = tokio::time::timeout(SOCKET_OP_TIMEOUT, UnixStream::connect(socket))
|
||||
.await
|
||||
.map_err(|_| std::io::Error::new(std::io::ErrorKind::TimedOut, "connect timed out"))??;
|
||||
let (_r, w) = stream.into_split();
|
||||
let (r, w) = stream.into_split();
|
||||
let mut reader = JsonLineReader::new(r);
|
||||
let mut writer = JsonLineWriter::new(w);
|
||||
|
||||
drain_initial_snapshot(&mut reader).await?;
|
||||
|
||||
tokio::time::timeout(SOCKET_OP_TIMEOUT, writer.write(method))
|
||||
.await
|
||||
.map_err(|_| std::io::Error::new(std::io::ErrorKind::TimedOut, "write timed out"))??;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn drain_initial_snapshot<R>(reader: &mut JsonLineReader<R>) -> std::io::Result<()>
|
||||
where
|
||||
R: tokio::io::AsyncBufRead + Unpin,
|
||||
{
|
||||
loop {
|
||||
let event = tokio::time::timeout(SOCKET_OP_TIMEOUT, reader.next::<Event>())
|
||||
.await
|
||||
.map_err(|_| std::io::Error::new(std::io::ErrorKind::TimedOut, "read timed out"))??;
|
||||
match event {
|
||||
Some(Event::Snapshot { .. }) => return Ok(()),
|
||||
Some(_) => continue,
|
||||
None => {
|
||||
return Err(std::io::Error::new(
|
||||
std::io::ErrorKind::UnexpectedEof,
|
||||
"pod closed connection before Snapshot event",
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Failure modes distinguished by `SendToPod`.
|
||||
enum SendRunError {
|
||||
/// Target Pod responded with `Error { AlreadyRunning }` — the
|
||||
|
|
@ -429,18 +457,25 @@ fn extract_assistant_text(entries: &[serde_json::Value]) -> String {
|
|||
let mut out = String::new();
|
||||
for value in entries {
|
||||
// The wire payload is the JSON form of `session_store::LogEntry`.
|
||||
// Walk Assistant items inside each entry that can carry them:
|
||||
// post-compaction `SessionStart.history` (seed) and per-LLM-call
|
||||
// `AssistantItems` deltas.
|
||||
// Walk current singular assistant items and the seeded history in
|
||||
// post-compaction `SegmentStart` entries.
|
||||
let Ok(entry) = serde_json::from_value::<LogEntry>(value.clone()) else {
|
||||
continue;
|
||||
};
|
||||
let logged_items = match entry {
|
||||
LogEntry::SessionStart { history, .. } => history,
|
||||
LogEntry::AssistantItems { items, .. } => items,
|
||||
match entry {
|
||||
LogEntry::SegmentStart { history, .. } => {
|
||||
for logged in history {
|
||||
push_assistant_text(&mut out, logged);
|
||||
}
|
||||
}
|
||||
LogEntry::AssistantItem { item, .. } => push_assistant_text(&mut out, item),
|
||||
_ => continue,
|
||||
};
|
||||
for logged in logged_items {
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn push_assistant_text(out: &mut String, logged: session_store::LoggedItem) {
|
||||
let item: Item = logged.into();
|
||||
if let Item::Message {
|
||||
role: Role::Assistant,
|
||||
|
|
@ -457,9 +492,6 @@ fn extract_assistant_text(entries: &[serde_json::Value]) -> String {
|
|||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn summarize_scope(record: &SpawnedPodRecord) -> String {
|
||||
|
|
@ -481,16 +513,94 @@ fn summarize_scope(record: &SpawnedPodRecord) -> String {
|
|||
parts.join(", ")
|
||||
}
|
||||
|
||||
/// Best-effort release of the pod's scope allocation. Swallows every
|
||||
/// error: the caller has already completed its user-visible side
|
||||
/// effects (Method::Shutdown was sent), and stale-reclaim will clean
|
||||
/// up whatever we couldn't.
|
||||
fn release_scope(pod_name: &str) {
|
||||
let Ok(lock_path) = pod_registry::default_registry_path() else {
|
||||
return;
|
||||
};
|
||||
let Ok(mut guard) = LockFileGuard::open(&lock_path) else {
|
||||
return;
|
||||
};
|
||||
let _ = pod_registry::release_pod(&mut guard, pod_name);
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
use protocol::{Alert, AlertLevel, AlertSource, Greeting, PodEvent, PodStatus};
|
||||
use tempfile::TempDir;
|
||||
use tokio::net::UnixListener;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
fn snapshot(entries: Vec<serde_json::Value>) -> Event {
|
||||
Event::Snapshot {
|
||||
entries,
|
||||
greeting: Greeting {
|
||||
pod_name: "server".into(),
|
||||
cwd: "/tmp".into(),
|
||||
provider: "test".into(),
|
||||
model: "test".into(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
},
|
||||
status: PodStatus::Idle,
|
||||
}
|
||||
}
|
||||
|
||||
fn serve_initial_events_then_method(
|
||||
listener: UnixListener,
|
||||
events: Vec<Event>,
|
||||
) -> JoinHandle<Option<Method>> {
|
||||
tokio::spawn(async move {
|
||||
let (stream, _) = listener.accept().await.ok()?;
|
||||
let (r, w) = stream.into_split();
|
||||
let mut reader = JsonLineReader::new(r);
|
||||
let mut writer = JsonLineWriter::new(w);
|
||||
for event in events {
|
||||
writer.write(&event).await.ok()?;
|
||||
}
|
||||
reader.next::<Method>().await.ok().flatten()
|
||||
})
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn connect_and_send_drains_initial_alert_and_snapshot_before_method() {
|
||||
let tmp = TempDir::new().unwrap();
|
||||
let socket = tmp.path().join("pod.sock");
|
||||
let listener = UnixListener::bind(&socket).unwrap();
|
||||
let received = serve_initial_events_then_method(
|
||||
listener,
|
||||
vec![
|
||||
Event::Alert(Alert {
|
||||
level: AlertLevel::Warn,
|
||||
source: AlertSource::Pod,
|
||||
message: "replayed alert".into(),
|
||||
timestamp_ms: 0,
|
||||
}),
|
||||
snapshot(Vec::new()),
|
||||
],
|
||||
);
|
||||
|
||||
connect_and_send(&socket, &Method::Shutdown).await.unwrap();
|
||||
|
||||
let method = received.await.unwrap().expect("expected method");
|
||||
assert!(matches!(method, Method::Shutdown));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn connect_and_send_delivers_method_after_large_initial_snapshot() {
|
||||
let tmp = TempDir::new().unwrap();
|
||||
let socket = tmp.path().join("pod.sock");
|
||||
let listener = UnixListener::bind(&socket).unwrap();
|
||||
let large_payload = "x".repeat(2 * 1024 * 1024);
|
||||
let received = serve_initial_events_then_method(
|
||||
listener,
|
||||
vec![snapshot(vec![
|
||||
serde_json::json!({ "payload": large_payload }),
|
||||
])],
|
||||
);
|
||||
let expected = Method::PodEvent(PodEvent::TurnEnded {
|
||||
pod_name: "child".into(),
|
||||
});
|
||||
|
||||
connect_and_send(&socket, &expected).await.unwrap();
|
||||
|
||||
let method = received.await.unwrap().expect("expected method");
|
||||
match method {
|
||||
Method::PodEvent(PodEvent::TurnEnded { pod_name }) => assert_eq!(pod_name, "child"),
|
||||
other => panic!("expected TurnEnded PodEvent, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,30 +2,53 @@
|
|||
//!
|
||||
//! `SpawnPod` writes here; the pod-comm tools (`SendToPod`,
|
||||
//! `ReadPodOutput`, `StopPod`, `ListPods`) read and mutate the same
|
||||
//! instance. Persisted to `spawned_pods.json` in the spawner's runtime
|
||||
//! dir so a restarted spawner rebuilds its view from disk (future work
|
||||
//! — today only write-through is implemented).
|
||||
//! instance. Runtime write-through still materialises `spawned_pods.json`,
|
||||
//! but durable state lives in the spawner's Pod metadata.
|
||||
//!
|
||||
//! `ReadPodOutput` additionally owns a per-spawned-pod cursor here so
|
||||
//! two consecutive reads yield only new assistant text. The cursor is
|
||||
//! an item-index into the child's history; push-only history makes
|
||||
//! index stable across reads.
|
||||
//!
|
||||
//! The registry stays in-memory only for this Pod's lifetime — cursors
|
||||
//! intentionally do not persist.
|
||||
//! Cursors intentionally do not persist; a restored registry starts with
|
||||
//! fresh read positions.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::io;
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use manifest::{Permission, ScopeRule, SharedScope};
|
||||
use session_store::{
|
||||
PodMetadata, PodMetadataStore, PodScopeSnapshot, PodSpawnedChild, PodSpawnedScopeRule,
|
||||
StoreError,
|
||||
};
|
||||
use tokio::net::UnixStream;
|
||||
use tokio::sync::Mutex;
|
||||
use tracing::warn;
|
||||
|
||||
use crate::runtime::dir::{RuntimeDir, SpawnedPodRecord};
|
||||
use crate::runtime::pod_registry;
|
||||
|
||||
type RegistryStateWriter = Arc<dyn Fn(&[SpawnedPodRecord]) -> io::Result<()> + Send + Sync>;
|
||||
type ScopeChangeSink = Arc<dyn Fn(PodScopeSnapshot) + Send + Sync>;
|
||||
|
||||
const RESTORE_REACHABILITY_TIMEOUT: Duration = Duration::from_millis(500);
|
||||
|
||||
pub struct SpawnedPodRegistry {
|
||||
records: Mutex<Vec<SpawnedPodRecord>>,
|
||||
cursors: Mutex<HashMap<String, usize>>,
|
||||
runtime_dir: Arc<RuntimeDir>,
|
||||
state_writer: Option<RegistryStateWriter>,
|
||||
parent_name: Option<String>,
|
||||
parent_scope: Option<SharedScope>,
|
||||
scope_change_sink: Option<ScopeChangeSink>,
|
||||
}
|
||||
|
||||
pub struct SpawnedPodRegistryLoad {
|
||||
pub registry: Arc<SpawnedPodRegistry>,
|
||||
pub reclaimed_unreachable: bool,
|
||||
}
|
||||
|
||||
impl SpawnedPodRegistry {
|
||||
|
|
@ -34,18 +57,111 @@ impl SpawnedPodRegistry {
|
|||
records: Mutex::new(Vec::new()),
|
||||
cursors: Mutex::new(HashMap::new()),
|
||||
runtime_dir,
|
||||
state_writer: None,
|
||||
parent_name: None,
|
||||
parent_scope: None,
|
||||
scope_change_sink: None,
|
||||
})
|
||||
}
|
||||
|
||||
/// Build a registry from the spawner's durable Pod state, pruning child
|
||||
/// records whose socket path is already gone. The surviving list is
|
||||
/// written through to both `spawned_pods.json` and Pod state so runtime
|
||||
/// and durable views start aligned.
|
||||
pub async fn load_from_pod_state<St>(
|
||||
runtime_dir: Arc<RuntimeDir>,
|
||||
store: St,
|
||||
pod_name: String,
|
||||
) -> io::Result<Arc<Self>>
|
||||
where
|
||||
St: PodMetadataStore + Clone + Send + Sync + 'static,
|
||||
{
|
||||
let loaded =
|
||||
Self::load_from_pod_state_with_reclaim(runtime_dir, store, pod_name, None, None)
|
||||
.await?;
|
||||
Ok(loaded.registry)
|
||||
}
|
||||
|
||||
pub async fn load_from_pod_state_with_reclaim<St>(
|
||||
runtime_dir: Arc<RuntimeDir>,
|
||||
store: St,
|
||||
pod_name: String,
|
||||
parent_scope: Option<SharedScope>,
|
||||
scope_change_sink: Option<ScopeChangeSink>,
|
||||
) -> io::Result<SpawnedPodRegistryLoad>
|
||||
where
|
||||
St: PodMetadataStore + Clone + Send + Sync + 'static,
|
||||
{
|
||||
let metadata = store.read_by_name(&pod_name).map_err(store_error_to_io)?;
|
||||
let persisted_children = metadata
|
||||
.as_ref()
|
||||
.map(|m| m.spawned_children.clone())
|
||||
.unwrap_or_default();
|
||||
|
||||
let mut records = Vec::with_capacity(persisted_children.len());
|
||||
let mut pruned = false;
|
||||
let mut pruned_records = Vec::new();
|
||||
for child in &persisted_children {
|
||||
let record = match record_from_pod_state(child) {
|
||||
Ok(record) => record,
|
||||
Err(err) => {
|
||||
pruned = true;
|
||||
warn!(
|
||||
error = %err,
|
||||
pod = %child.pod_name,
|
||||
"dropping corrupt persisted spawned-pod record"
|
||||
);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
if is_reachable(&record.socket_path).await {
|
||||
records.push(record);
|
||||
} else {
|
||||
pruned = true;
|
||||
warn!(
|
||||
pod = %record.pod_name,
|
||||
socket = %record.socket_path.display(),
|
||||
"dropping unreachable persisted spawned-pod record"
|
||||
);
|
||||
pruned_records.push(record);
|
||||
}
|
||||
}
|
||||
|
||||
runtime_dir.write_spawned_pods(&records).await?;
|
||||
let state_writer = pod_state_writer(store, pod_name.clone());
|
||||
if pruned || metadata.is_some() {
|
||||
state_writer(&records)?;
|
||||
}
|
||||
|
||||
let mut reclaimed_unreachable = false;
|
||||
if parent_scope.is_some() {
|
||||
for record in &pruned_records {
|
||||
reclaim_record(&pod_name, parent_scope.as_ref(), None, record)?;
|
||||
reclaimed_unreachable = true;
|
||||
}
|
||||
}
|
||||
|
||||
Ok(SpawnedPodRegistryLoad {
|
||||
registry: Arc::new(Self {
|
||||
records: Mutex::new(records),
|
||||
cursors: Mutex::new(HashMap::new()),
|
||||
runtime_dir,
|
||||
state_writer: Some(state_writer),
|
||||
parent_name: Some(pod_name),
|
||||
parent_scope,
|
||||
scope_change_sink,
|
||||
}),
|
||||
reclaimed_unreachable,
|
||||
})
|
||||
}
|
||||
|
||||
/// Append a new record and persist the full list. Returns an I/O
|
||||
/// error if the persisted write fails; the in-memory state is still
|
||||
/// error if either persisted write fails; the in-memory state is still
|
||||
/// updated in that case — the next successful write will reconcile.
|
||||
pub async fn add(&self, record: SpawnedPodRecord) -> io::Result<()> {
|
||||
let mut records = self.records.lock().await;
|
||||
records.push(record);
|
||||
self.runtime_dir
|
||||
.write_spawned_pods(records.as_slice())
|
||||
.await
|
||||
self.persist_records(records.as_slice()).await
|
||||
}
|
||||
|
||||
/// Look up a record by pod name. Cloned so callers can drop the lock.
|
||||
|
|
@ -62,22 +178,37 @@ impl SpawnedPodRegistry {
|
|||
self.records.lock().await.clone()
|
||||
}
|
||||
|
||||
/// Remove the record for `pod_name`, persist, and clear its cursor.
|
||||
/// Returns the removed record (if any).
|
||||
/// Remove the record for `pod_name`, persist, clear its cursor, and
|
||||
/// reclaim any delegated Write scope owned by that child. Returns the
|
||||
/// removed record (if any).
|
||||
pub async fn remove(&self, pod_name: &str) -> io::Result<Option<SpawnedPodRecord>> {
|
||||
let removed = {
|
||||
let mut records = self.records.lock().await;
|
||||
let idx = records.iter().position(|r| r.pod_name == pod_name);
|
||||
let removed = idx.map(|i| records.remove(i));
|
||||
self.runtime_dir
|
||||
.write_spawned_pods(records.as_slice())
|
||||
.await?;
|
||||
self.persist_records(records.as_slice()).await?;
|
||||
removed
|
||||
};
|
||||
self.cursors.lock().await.remove(pod_name);
|
||||
if let Some(record) = &removed {
|
||||
self.reclaim_record(record)?;
|
||||
}
|
||||
Ok(removed)
|
||||
}
|
||||
|
||||
fn reclaim_record(&self, record: &SpawnedPodRecord) -> io::Result<()> {
|
||||
let Some(parent_name) = &self.parent_name else {
|
||||
release_child_allocation(&record.pod_name)?;
|
||||
return Ok(());
|
||||
};
|
||||
reclaim_record(
|
||||
parent_name,
|
||||
self.parent_scope.as_ref(),
|
||||
self.scope_change_sink.as_ref(),
|
||||
record,
|
||||
)
|
||||
}
|
||||
|
||||
/// Read-only cursor lookup. Returns 0 when no cursor has been set.
|
||||
pub async fn cursor(&self, pod_name: &str) -> usize {
|
||||
self.cursors
|
||||
|
|
@ -94,4 +225,150 @@ impl SpawnedPodRegistry {
|
|||
.await
|
||||
.insert(pod_name.to_string(), cursor);
|
||||
}
|
||||
|
||||
async fn persist_records(&self, records: &[SpawnedPodRecord]) -> io::Result<()> {
|
||||
self.runtime_dir.write_spawned_pods(records).await?;
|
||||
if let Some(write_state) = &self.state_writer {
|
||||
write_state(records)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
fn pod_state_writer<St>(store: St, pod_name: String) -> RegistryStateWriter
|
||||
where
|
||||
St: PodMetadataStore + Clone + Send + Sync + 'static,
|
||||
{
|
||||
Arc::new(move |records| {
|
||||
write_records_to_pod_state(&store, &pod_name, records).map_err(store_error_to_io)
|
||||
})
|
||||
}
|
||||
|
||||
fn reclaim_record(
|
||||
parent_name: &str,
|
||||
parent_scope: Option<&SharedScope>,
|
||||
scope_change_sink: Option<&ScopeChangeSink>,
|
||||
record: &SpawnedPodRecord,
|
||||
) -> io::Result<()> {
|
||||
let write_rules = record
|
||||
.scope_delegated
|
||||
.iter()
|
||||
.filter(|rule| rule.permission == Permission::Write)
|
||||
.cloned()
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
let lock_path = pod_registry::default_registry_path()
|
||||
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
|
||||
let mut guard = pod_registry::LockFileGuard::open(&lock_path)
|
||||
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
|
||||
pod_registry::reclaim_delegated_scope(
|
||||
&mut guard,
|
||||
parent_name,
|
||||
&record.pod_name,
|
||||
&record.scope_delegated,
|
||||
)
|
||||
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
|
||||
|
||||
if let Some(scope) = parent_scope {
|
||||
scope
|
||||
.update(|current| current.with_removed_deny_rules(write_rules))
|
||||
.map_err(|err| io::Error::new(io::ErrorKind::InvalidInput, err))?;
|
||||
if let Some(sink) = scope_change_sink {
|
||||
let snapshot = scope.snapshot();
|
||||
sink(PodScopeSnapshot {
|
||||
allow: snapshot.allow_rules(),
|
||||
deny: snapshot.deny_rules(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn release_child_allocation(pod_name: &str) -> io::Result<()> {
|
||||
let lock_path = pod_registry::default_registry_path()
|
||||
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
|
||||
let mut guard = pod_registry::LockFileGuard::open(&lock_path)
|
||||
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
|
||||
match pod_registry::release_pod(&mut guard, pod_name) {
|
||||
Ok(()) | Err(pod_registry::ScopeLockError::UnknownPod(_)) => Ok(()),
|
||||
Err(err) => Err(io::Error::new(io::ErrorKind::Other, err)),
|
||||
}
|
||||
}
|
||||
|
||||
fn write_records_to_pod_state<St>(
|
||||
store: &St,
|
||||
pod_name: &str,
|
||||
records: &[SpawnedPodRecord],
|
||||
) -> Result<(), StoreError>
|
||||
where
|
||||
St: PodMetadataStore,
|
||||
{
|
||||
let mut metadata = store
|
||||
.read_by_name(pod_name)?
|
||||
.unwrap_or_else(|| PodMetadata::new(pod_name, None));
|
||||
metadata.spawned_children = records
|
||||
.iter()
|
||||
.map(record_to_pod_state)
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
store.write(&metadata)
|
||||
}
|
||||
|
||||
fn record_to_pod_state(record: &SpawnedPodRecord) -> Result<PodSpawnedChild, serde_json::Error> {
|
||||
Ok(PodSpawnedChild {
|
||||
pod_name: record.pod_name.clone(),
|
||||
socket_path: record.socket_path.clone(),
|
||||
scope_delegated: record
|
||||
.scope_delegated
|
||||
.iter()
|
||||
.map(|rule| PodSpawnedScopeRule {
|
||||
target: rule.target.clone(),
|
||||
permission: match rule.permission {
|
||||
Permission::Read => "read".to_string(),
|
||||
Permission::Write => "write".to_string(),
|
||||
},
|
||||
recursive: rule.recursive,
|
||||
})
|
||||
.collect(),
|
||||
callback_address: record.callback_address.clone(),
|
||||
})
|
||||
}
|
||||
|
||||
fn record_from_pod_state(child: &PodSpawnedChild) -> Result<SpawnedPodRecord, serde_json::Error> {
|
||||
Ok(SpawnedPodRecord {
|
||||
pod_name: child.pod_name.clone(),
|
||||
socket_path: child.socket_path.clone(),
|
||||
scope_delegated: child
|
||||
.scope_delegated
|
||||
.iter()
|
||||
.map(|rule| {
|
||||
Ok(ScopeRule {
|
||||
target: rule.target.clone(),
|
||||
permission: match rule.permission.as_str() {
|
||||
"read" => Permission::Read,
|
||||
"write" => Permission::Write,
|
||||
other => {
|
||||
return Err(serde_json::Error::io(io::Error::new(
|
||||
io::ErrorKind::InvalidData,
|
||||
format!("invalid permission `{other}`"),
|
||||
)));
|
||||
}
|
||||
},
|
||||
recursive: rule.recursive,
|
||||
})
|
||||
})
|
||||
.collect::<Result<Vec<_>, _>>()?,
|
||||
callback_address: child.callback_address.clone(),
|
||||
})
|
||||
}
|
||||
|
||||
fn store_error_to_io(error: StoreError) -> io::Error {
|
||||
io::Error::other(error)
|
||||
}
|
||||
|
||||
async fn is_reachable(socket: &Path) -> bool {
|
||||
tokio::time::timeout(RESTORE_REACHABILITY_TIMEOUT, UnixStream::connect(socket))
|
||||
.await
|
||||
.map(|result| result.is_ok())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -107,7 +107,8 @@ pub struct SpawnPodTool {
|
|||
spawner_pwd: PathBuf,
|
||||
/// Shared registry of spawned children, also used by the
|
||||
/// pod-comm tools (`SendToPod` / `ReadPodOutput` / `StopPod` /
|
||||
/// `ListPods`). Writes the list to `spawned_pods.json` on each add.
|
||||
/// `ListPods`). Writes the list to runtime and durable Pod state on
|
||||
/// each add.
|
||||
registry: Arc<SpawnedPodRegistry>,
|
||||
/// THIS Pod's own parent-callback socket, if any. After a
|
||||
/// successful spawn we fire `PodEvent::ScopeSubDelegated` upward
|
||||
|
|
@ -268,7 +269,7 @@ impl Tool for SpawnPodTool {
|
|||
self.registry
|
||||
.add(record)
|
||||
.await
|
||||
.map_err(|e| ToolError::ExecutionFailed(format!("write spawned_pods.json: {e}")))?;
|
||||
.map_err(|e| ToolError::ExecutionFailed(format!("write spawned pod registry: {e}")))?;
|
||||
|
||||
// Notify this Pod's own parent so the grandparent can register
|
||||
// the new grandchild directly. Fire-and-forget; top-level Pods
|
||||
|
|
@ -482,7 +483,7 @@ fn pod_registry_err_to_tool(e: ScopeLockError) -> ToolError {
|
|||
| ScopeLockError::WriteConflict { .. }
|
||||
| ScopeLockError::DuplicatePodName(_)
|
||||
| ScopeLockError::UnknownPod(_)
|
||||
| ScopeLockError::SessionConflict { .. } => ToolError::InvalidArgument(e.to_string()),
|
||||
| ScopeLockError::SegmentConflict { .. } => ToolError::InvalidArgument(e.to_string()),
|
||||
ScopeLockError::Io(_) => ToolError::ExecutionFailed(e.to_string()),
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ use llm_worker::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEve
|
|||
use llm_worker::llm_client::types::Item;
|
||||
use llm_worker::llm_client::{ClientError, LlmClient, Request};
|
||||
use protocol::Event;
|
||||
use session_store::FsStore;
|
||||
use session_store::{FsStore, LogEntry, PodMetadataStore, Store};
|
||||
use tokio::sync::broadcast;
|
||||
|
||||
use pod::Pod;
|
||||
|
|
@ -158,7 +158,9 @@ async fn make_pod_with_manifest(
|
|||
std::mem::forget(pwd_tmp);
|
||||
|
||||
let worker = Worker::new(client);
|
||||
Pod::new(manifest, worker, store, pwd, scope).await.unwrap()
|
||||
let mut pod = Pod::new(manifest, worker, store, pwd, scope).await.unwrap();
|
||||
pod.enable_pod_metadata_write_through().unwrap();
|
||||
pod
|
||||
}
|
||||
|
||||
async fn make_pod(client: MockClient) -> Pod<MockClient, FsStore> {
|
||||
|
|
@ -178,7 +180,7 @@ fn drain(rx: &mut broadcast::Receiver<Event>) -> Vec<Event> {
|
|||
}
|
||||
|
||||
/// Collect every system-message text that the post-compaction
|
||||
/// `SessionStart.history` carries, by reading the sink mirror directly.
|
||||
/// `SegmentStart.history` carries, by reading the sink mirror directly.
|
||||
fn system_texts_in_sink_session_start(
|
||||
pod: &pod::Pod<
|
||||
impl llm_worker::llm_client::client::LlmClient + Clone + 'static,
|
||||
|
|
@ -187,7 +189,7 @@ fn system_texts_in_sink_session_start(
|
|||
) -> Vec<String> {
|
||||
let (entries, _rx) = pod.sink().subscribe_with_snapshot();
|
||||
for entry in entries.into_iter().rev() {
|
||||
if let session_store::LogEntry::SessionStart { history, .. } = entry {
|
||||
if let session_store::LogEntry::SegmentStart { history, .. } = entry {
|
||||
return history
|
||||
.into_iter()
|
||||
.filter_map(|logged| {
|
||||
|
|
@ -213,6 +215,129 @@ fn system_texts_in_sink_session_start(
|
|||
Vec::new()
|
||||
}
|
||||
|
||||
/// Pod metadata starts with a reserved Session and no Segment, then becomes
|
||||
/// active once the first SegmentStart is materialized by `run`.
|
||||
#[tokio::test]
|
||||
async fn pod_metadata_moves_from_pending_to_active_on_first_run() {
|
||||
let client = MockClient::new(vec![single_text_events("hi")]);
|
||||
let mut pod = make_pod(client).await;
|
||||
let store = pod.store().clone();
|
||||
let session_id = pod.session_id();
|
||||
let initial_segment_id = pod.segment_id();
|
||||
|
||||
let pending = store
|
||||
.read_by_name("test-pod")
|
||||
.unwrap()
|
||||
.expect("metadata should be initialized at Pod construction");
|
||||
assert_eq!(pending.pod_name, "test-pod");
|
||||
let pending_active = pending.active.expect("active session pointer missing");
|
||||
assert_eq!(pending_active.session_id, session_id);
|
||||
assert_eq!(pending_active.segment_id, None);
|
||||
|
||||
pod.run_text("first").await.unwrap();
|
||||
|
||||
let resolved = store
|
||||
.read_by_name("test-pod")
|
||||
.unwrap()
|
||||
.expect("metadata should still exist after first run");
|
||||
let active = resolved.active.expect("active session pointer missing");
|
||||
assert_eq!(active.session_id, session_id);
|
||||
assert_eq!(active.segment_id, Some(initial_segment_id));
|
||||
}
|
||||
|
||||
/// Live auto-fork: when another writer extends the segment behind the
|
||||
/// Pod's back, the next run's `ensure_segment_head` detects the
|
||||
/// entry-count drift and branches into a fresh segment **within the same
|
||||
/// Session**. The source segment is left immutable (no terminal marker
|
||||
/// written back); the new segment records its parentage forward via
|
||||
/// `SegmentStart.forked_from`.
|
||||
#[tokio::test]
|
||||
async fn concurrent_writer_drift_auto_forks_with_forked_from() {
|
||||
// No compaction: keep run → run deterministic so each run consumes
|
||||
// exactly one mock response and ensure_segment_head is the only fork
|
||||
// trigger.
|
||||
const NO_COMPACT_MANIFEST_TOML: &str = r#"
|
||||
[pod]
|
||||
name = "test-pod"
|
||||
pwd = "./"
|
||||
|
||||
[model]
|
||||
scheme = "anthropic"
|
||||
model_id = "test-model"
|
||||
|
||||
[worker]
|
||||
max_tokens = 100
|
||||
|
||||
[[scope.allow]]
|
||||
target = "./"
|
||||
permission = "write"
|
||||
"#;
|
||||
let client = MockClient::new(vec![
|
||||
single_text_events("first"),
|
||||
single_text_events("second"),
|
||||
]);
|
||||
let mut pod = make_pod_with_manifest(NO_COMPACT_MANIFEST_TOML, client).await;
|
||||
|
||||
pod.run_text("first").await.unwrap();
|
||||
|
||||
let store = pod.store().clone();
|
||||
let session_id = pod.session_id();
|
||||
let source_segment_id = pod.segment_id();
|
||||
let source_len_before = store.read_all(session_id, source_segment_id).unwrap().len();
|
||||
|
||||
// Simulate a foreign writer appending to the same segment. This bumps
|
||||
// the on-disk entry count past the Pod's own append tally without
|
||||
// updating the Pod's `entries_written`.
|
||||
store
|
||||
.append(
|
||||
session_id,
|
||||
source_segment_id,
|
||||
&LogEntry::UserInput {
|
||||
ts: 9999,
|
||||
segments: vec![protocol::Segment::text("interloper")],
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
// Next run triggers ensure_segment_head, which sees the drift.
|
||||
pod.run_text("second").await.unwrap();
|
||||
|
||||
// The Pod moved to a new segment in the same Session.
|
||||
let new_segment_id = pod.segment_id();
|
||||
assert_ne!(new_segment_id, source_segment_id);
|
||||
assert_eq!(pod.session_id(), session_id, "auto-fork stays in-Session");
|
||||
let metadata = store
|
||||
.read_by_name("test-pod")
|
||||
.unwrap()
|
||||
.expect("metadata should exist after auto-fork");
|
||||
let active = metadata.active.expect("active session pointer missing");
|
||||
assert_eq!(active.session_id, session_id);
|
||||
assert_eq!(active.segment_id, Some(new_segment_id));
|
||||
|
||||
// New segment records forked_from pointing at the source.
|
||||
let new_entries = store.read_all(session_id, new_segment_id).unwrap();
|
||||
match &new_entries[0] {
|
||||
LogEntry::SegmentStart {
|
||||
session_id: seg_session,
|
||||
forked_from: Some(origin),
|
||||
..
|
||||
} => {
|
||||
assert_eq!(*seg_session, session_id);
|
||||
assert_eq!(origin.segment_id, source_segment_id);
|
||||
}
|
||||
other => panic!("expected SegmentStart with forked_from, got {other:?}"),
|
||||
}
|
||||
|
||||
// Source segment is unchanged except for the foreign append — the
|
||||
// auto-fork wrote no terminal marker back into it.
|
||||
let source_after = store.read_all(session_id, source_segment_id).unwrap();
|
||||
assert_eq!(source_after.len(), source_len_before + 1);
|
||||
assert!(matches!(
|
||||
source_after.last(),
|
||||
Some(LogEntry::UserInput { .. })
|
||||
));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn compact_emits_session_start_carrying_summary_and_task_snapshot() {
|
||||
let client = MockClient::new(vec![
|
||||
|
|
@ -226,10 +351,20 @@ async fn compact_emits_session_start_carrying_summary_and_task_snapshot() {
|
|||
pod.attach_event_tx(tx);
|
||||
|
||||
pod.run_text("first").await.unwrap();
|
||||
let session_id = pod.session_id();
|
||||
pod.compact(10_000).await.unwrap();
|
||||
let compacted_segment_id = pod.segment_id();
|
||||
let metadata = pod
|
||||
.store()
|
||||
.read_by_name("test-pod")
|
||||
.unwrap()
|
||||
.expect("metadata should exist after compaction");
|
||||
let active = metadata.active.expect("active session pointer missing");
|
||||
assert_eq!(active.session_id, session_id);
|
||||
assert_eq!(active.segment_id, Some(compacted_segment_id));
|
||||
|
||||
let system_texts = system_texts_in_sink_session_start(&pod);
|
||||
// The post-compaction `SessionStart.history` carries the new system
|
||||
// The post-compaction `SegmentStart.history` carries the new system
|
||||
// messages introduced by the compactor. Clients re-seed their view
|
||||
// from this entry alone, so it is the load-bearing payload.
|
||||
assert!(
|
||||
|
|
@ -289,11 +424,11 @@ async fn pre_run_compact_success_broadcasts_start_and_done() {
|
|||
|
||||
// CompactDone carries the new session id.
|
||||
let new_id_in_event = events.iter().find_map(|e| match e {
|
||||
Event::CompactDone { new_session_id } => Some(*new_session_id),
|
||||
Event::CompactDone { new_segment_id } => Some(*new_segment_id),
|
||||
_ => None,
|
||||
});
|
||||
assert!(new_id_in_event.is_some(), "CompactDone missing");
|
||||
assert_eq!(new_id_in_event.unwrap(), pod.session_id());
|
||||
assert_eq!(new_id_in_event.unwrap(), pod.segment_id());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
|
@ -345,10 +480,10 @@ async fn mid_turn_compact_success_broadcasts_start_and_done() {
|
|||
);
|
||||
|
||||
let new_id_in_event = events.iter().find_map(|e| match e {
|
||||
Event::CompactDone { new_session_id } => Some(*new_session_id),
|
||||
Event::CompactDone { new_segment_id } => Some(*new_segment_id),
|
||||
_ => None,
|
||||
});
|
||||
assert_eq!(new_id_in_event, Some(pod.session_id()));
|
||||
assert_eq!(new_id_in_event, Some(pod.segment_id()));
|
||||
}
|
||||
|
||||
/// Regression: `Pod::compact()` must reset the in-memory
|
||||
|
|
@ -520,9 +655,9 @@ async fn pre_run_compact_failure_broadcasts_start_and_failed() {
|
|||
// ---------------------------------------------------------------------------
|
||||
// Detached post-run memory jobs (`spawn_post_run_memory_jobs` /
|
||||
// `wait_for_memory_jobs`). Covers the detach round-trip and the structural
|
||||
// invariant that the cloned memory-task Pod shares `SessionHead` with the
|
||||
// invariant that the cloned memory-task Pod shares `SegmentState` with the
|
||||
// source Pod, so that `save_extension` from the background extract does not
|
||||
// leave the next turn's `save_user_input` looking at a stale head_hash.
|
||||
// leave the next turn's `save_user_input` looking at a stale session pointer.
|
||||
|
||||
const EXTRACT_NO_COMPACT_MANIFEST: &str = r#"
|
||||
[pod]
|
||||
|
|
@ -544,6 +679,27 @@ target = "./"
|
|||
permission = "write"
|
||||
"#;
|
||||
|
||||
#[tokio::test]
|
||||
async fn extract_large_unprocessed_range_does_not_abort_on_input_occupancy() {
|
||||
let client = MockClient::new(vec![
|
||||
text_events_with_usage("recorded", 1000),
|
||||
write_extracted_tool_use_events("ec-large"),
|
||||
single_text_events("done"),
|
||||
]);
|
||||
let mut pod = make_pod_with_manifest(EXTRACT_NO_COMPACT_MANIFEST, client).await;
|
||||
|
||||
let large_request = format!("remember this large slice: {}", "x ".repeat(200_000));
|
||||
pod.run_text(&large_request).await.unwrap();
|
||||
|
||||
pod.try_post_run_extract().await.expect(
|
||||
"large unprocessed extract ranges must reach the extract worker, not abort locally",
|
||||
);
|
||||
assert!(
|
||||
pod.extract_pointer().is_some(),
|
||||
"successful extract should advance the pointer even when the input range is large"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn spawn_and_wait_drives_extract_to_completion() {
|
||||
let client = MockClient::new(vec![
|
||||
|
|
@ -570,9 +726,9 @@ async fn spawn_and_wait_drives_extract_to_completion() {
|
|||
|
||||
#[tokio::test]
|
||||
async fn detached_extract_does_not_fork_session_log() {
|
||||
// Source pod and the cloned memory-task pod share `SessionHead` via
|
||||
// `Arc<AsyncMutex<_>>`. The detached extract advances head_hash through
|
||||
// `save_extension`; the next `run` must see that same head_hash so
|
||||
// Source pod and the cloned memory-task pod share `SegmentState` via
|
||||
// `Arc<_>`. The detached extract advances the entry tally through
|
||||
// `save_extension`; the next `run` must see that same tally so
|
||||
// `ensure_head_or_fork` does not spawn a new session.
|
||||
let client = MockClient::new(vec![
|
||||
text_events_with_usage("hi", 1000),
|
||||
|
|
@ -583,18 +739,18 @@ async fn detached_extract_does_not_fork_session_log() {
|
|||
let mut pod = make_pod_with_manifest(EXTRACT_NO_COMPACT_MANIFEST, client).await;
|
||||
|
||||
pod.run_text("first").await.unwrap();
|
||||
let session_before = pod.session_id();
|
||||
let session_before = pod.segment_id();
|
||||
|
||||
pod.spawn_post_run_memory_jobs();
|
||||
pod.wait_for_memory_jobs().await;
|
||||
|
||||
pod.run_text("second").await.unwrap();
|
||||
let session_after = pod.session_id();
|
||||
let session_after = pod.segment_id();
|
||||
|
||||
assert_eq!(
|
||||
session_before, session_after,
|
||||
"detached extract's save_extension and the next turn's save_user_input \
|
||||
must share head_hash through SessionHead — a fork here means the clone \
|
||||
carried its own head_hash"
|
||||
must share the entry tally through SegmentState — a fork here means the \
|
||||
clone carried its own counter"
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -172,7 +172,7 @@ fn write_n_staging(layout: &WorkspaceLayout, n: usize) -> Vec<uuid::Uuid> {
|
|||
let (id, _) = write_staging(
|
||||
layout,
|
||||
SourceRef {
|
||||
session_id: format!("s-{i}"),
|
||||
segment_id: format!("s-{i}"),
|
||||
range: [i as u64, i as u64],
|
||||
},
|
||||
ExtractedPayload::default(),
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ fn history_from_sink(handle: &PodHandle) -> Vec<Item> {
|
|||
let mut items = Vec::new();
|
||||
for entry in entries {
|
||||
match entry {
|
||||
LogEntry::SessionStart { history, .. } => {
|
||||
LogEntry::SegmentStart { history, .. } => {
|
||||
items.extend(history.into_iter().map(Item::from));
|
||||
}
|
||||
LogEntry::UserInput { segments, .. } => {
|
||||
|
|
@ -35,14 +35,6 @@ fn history_from_sink(handle: &PodHandle) -> Vec<Item> {
|
|||
LogEntry::SystemItem { item, .. } => {
|
||||
items.push(item.to_history_item());
|
||||
}
|
||||
LogEntry::AssistantItems { items: i, .. }
|
||||
| LogEntry::ToolResults { items: i, .. }
|
||||
| LogEntry::HookInjectedItems { items: i, .. } => {
|
||||
items.extend(i.into_iter().map(Item::from));
|
||||
}
|
||||
LogEntry::SystemItems { items: si, .. } => {
|
||||
items.extend(si.iter().map(|s| s.to_history_item()));
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
|
@ -417,21 +409,23 @@ async fn events_are_broadcast() {
|
|||
|
||||
#[tokio::test]
|
||||
async fn double_run_returns_error() {
|
||||
// Create a client that streams slowly
|
||||
// Keep the first turn in-flight until the test drops the handle. A
|
||||
// finite stream can finish before the second Method reaches the
|
||||
// controller in the full test suite, making this assertion racy.
|
||||
let events = vec![
|
||||
LlmEvent::text_block_start(0),
|
||||
LlmEvent::text_delta(0, "slow..."),
|
||||
// No stop/completed — the stream will end but without proper completion
|
||||
];
|
||||
let client = MockClient::new(events);
|
||||
let client = MockClient::sequential(vec![MockResponse::Hang(events)]);
|
||||
let pod = make_pod(client).await;
|
||||
let handle = spawn_controller(pod).await;
|
||||
let mut rx = handle.subscribe();
|
||||
|
||||
// Send first run
|
||||
// Send first run and wait until the controller has entered Running.
|
||||
handle.send(Method::run_text("first")).await.unwrap();
|
||||
wait_for_status(&handle, PodStatus::Running).await;
|
||||
|
||||
// Immediately send second run (should get error)
|
||||
// Now the second run must be rejected by drive_turn's live Method arm.
|
||||
handle.send(Method::run_text("second")).await.unwrap();
|
||||
|
||||
// Look for the error event
|
||||
|
|
@ -777,13 +771,15 @@ async fn notify_while_idle_auto_starts_turn_and_injects_system_message() {
|
|||
// not on the `event_tx` broadcast that `handle.subscribe()` taps.
|
||||
// Verify the notification landed on the sink mirror instead.
|
||||
let (entries, _) = handle.sink.subscribe_with_snapshot();
|
||||
let saw_notify_in_mirror = entries.iter().any(|e| matches!(
|
||||
let saw_notify_in_mirror = entries.iter().any(|e| {
|
||||
matches!(
|
||||
e,
|
||||
session_store::LogEntry::SystemItem {
|
||||
item: session_store::SystemItem::Notification { message, .. },
|
||||
..
|
||||
} if message == "turn finished"
|
||||
));
|
||||
)
|
||||
});
|
||||
assert!(
|
||||
saw_notify_in_mirror,
|
||||
"Method::Notify should commit a SystemItem::Notification entry; mirror = {entries:?}"
|
||||
|
|
@ -865,7 +861,8 @@ async fn pod_event_turn_ended_while_idle_auto_starts_turn_and_injects_system_mes
|
|||
// its Flush of the drain queue) runs afterwards.
|
||||
wait_for_status(&handle, PodStatus::Idle).await;
|
||||
let (entries, _) = handle.sink.subscribe_with_snapshot();
|
||||
let saw_pod_event_in_mirror = entries.iter().any(|e| matches!(
|
||||
let saw_pod_event_in_mirror = entries.iter().any(|e| {
|
||||
matches!(
|
||||
e,
|
||||
session_store::LogEntry::SystemItem {
|
||||
item: session_store::SystemItem::PodEvent {
|
||||
|
|
@ -874,7 +871,8 @@ async fn pod_event_turn_ended_while_idle_auto_starts_turn_and_injects_system_mes
|
|||
},
|
||||
..
|
||||
} if pod_name == "child"
|
||||
));
|
||||
)
|
||||
});
|
||||
assert!(
|
||||
saw_pod_event_in_mirror,
|
||||
"Method::PodEvent should commit a SystemItem::PodEvent entry"
|
||||
|
|
|
|||
|
|
@ -2,18 +2,18 @@
|
|||
//! `ReadPodOutput`, `StopPod`, `ListPods`).
|
||||
//!
|
||||
//! The real child Pod binary is not started. Instead each test stands
|
||||
//! up a mock `UnixListener` that speaks the socket protocol directly
|
||||
//! (accepting `Method::Run` / `Method::GetHistory` / `Method::Shutdown`
|
||||
//! and responding with `Event::History` when asked). This keeps the
|
||||
//! tests fast and independent of the LLM layer — the tools are exercised
|
||||
//! for their wire behaviour alone.
|
||||
//! up a mock `UnixListener` that speaks the socket protocol directly:
|
||||
//! it emits the connect-time `Event::Snapshot`, accepts methods such as
|
||||
//! `Method::Run` / `Method::Shutdown`, and responds with the relevant
|
||||
//! events when needed. This keeps the tests fast and independent of the
|
||||
//! LLM layer — the tools are exercised for their wire behaviour alone.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Arc, LazyLock, Mutex};
|
||||
|
||||
use llm_worker::llm_client::types::{ContentPart, Item, Role};
|
||||
use llm_worker::tool::ToolOutput;
|
||||
use manifest::{Permission, ScopeRule};
|
||||
use manifest::{Permission, Scope, ScopeRule, SharedScope};
|
||||
use pod::runtime::dir::{RuntimeDir, SpawnedPodRecord};
|
||||
use pod::runtime::pod_registry::{self, LockFileGuard};
|
||||
use pod::spawn::comm_tools::{
|
||||
|
|
@ -23,8 +23,10 @@ use pod::spawn::registry::SpawnedPodRegistry;
|
|||
use protocol::stream::{JsonLineReader, JsonLineWriter};
|
||||
use protocol::{ErrorCode, Event, Greeting, Method};
|
||||
use serde_json::json;
|
||||
use session_store::{FsStore, PodMetadataStore};
|
||||
use tempfile::TempDir;
|
||||
use tokio::net::UnixListener;
|
||||
use tokio::sync::mpsc;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
/// Serialises env-mutating tests. The test harness runs tasks across
|
||||
|
|
@ -113,20 +115,42 @@ async fn bind_mock_socket(dir: &Path, name: &str) -> (PathBuf, UnixListener) {
|
|||
(socket, listener)
|
||||
}
|
||||
|
||||
/// Accept one connection and read exactly one `Method` line from it.
|
||||
/// Minimal connect-time snapshot used by mock socket servers.
|
||||
fn empty_snapshot() -> Event {
|
||||
Event::Snapshot {
|
||||
entries: Vec::new(),
|
||||
greeting: Greeting {
|
||||
pod_name: "child".into(),
|
||||
cwd: "/tmp".into(),
|
||||
provider: "anthropic".into(),
|
||||
model: "x".into(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
},
|
||||
status: protocol::PodStatus::Idle,
|
||||
}
|
||||
}
|
||||
|
||||
/// Accept one connection, send the protocol's connect-time snapshot,
|
||||
/// and read exactly one `Method` line from it.
|
||||
/// The reader half is kept open; caller awaits the returned handle.
|
||||
fn accept_one_method(listener: UnixListener) -> JoinHandle<Option<Method>> {
|
||||
tokio::spawn(async move {
|
||||
let (stream, _) = listener.accept().await.ok()?;
|
||||
let (r, _w) = stream.into_split();
|
||||
let (r, w) = stream.into_split();
|
||||
let mut reader = JsonLineReader::new(r);
|
||||
let mut writer = JsonLineWriter::new(w);
|
||||
writer.write(&empty_snapshot()).await.ok()?;
|
||||
reader.next::<Method>().await.ok().flatten()
|
||||
})
|
||||
}
|
||||
|
||||
/// Accept one connection, read one `Method`, then write `response`
|
||||
/// back. Used by `SendToPod` tests to mock the real controller's
|
||||
/// `TurnStart` acknowledgement (or its `AlreadyRunning` rejection).
|
||||
/// Accept one connection, send the protocol's connect-time snapshot,
|
||||
/// read one `Method`, then write `response` back. Used by `SendToPod`
|
||||
/// tests to mock the real controller's `TurnStart` acknowledgement (or
|
||||
/// its `AlreadyRunning` rejection).
|
||||
fn accept_method_and_respond(
|
||||
listener: UnixListener,
|
||||
response: Event,
|
||||
|
|
@ -136,6 +160,7 @@ fn accept_method_and_respond(
|
|||
let (r, w) = stream.into_split();
|
||||
let mut reader = JsonLineReader::new(r);
|
||||
let mut writer = JsonLineWriter::new(w);
|
||||
writer.write(&empty_snapshot()).await.ok()?;
|
||||
let method = reader.next::<Method>().await.ok().flatten();
|
||||
if method.is_some() {
|
||||
let _ = writer.write(&response).await;
|
||||
|
|
@ -156,18 +181,18 @@ fn serve_history(listener: UnixListener, items: Vec<Item>) -> JoinHandle<()> {
|
|||
};
|
||||
let (_r, w) = stream.into_split();
|
||||
let mut writer = JsonLineWriter::new(w);
|
||||
// Wrap the assistant items in a single
|
||||
// `LogEntry::AssistantItems` entry — that's the only kind
|
||||
// that contributes assistant text via `extract_assistant_text`.
|
||||
let logged: Vec<session_store::LoggedItem> =
|
||||
items.iter().map(session_store::LoggedItem::from).collect();
|
||||
let entry = session_store::LogEntry::AssistantItems {
|
||||
let entries: Vec<serde_json::Value> = items
|
||||
.iter()
|
||||
.map(|item| {
|
||||
let entry = session_store::LogEntry::AssistantItem {
|
||||
ts: 0,
|
||||
items: logged,
|
||||
item: session_store::LoggedItem::from(item),
|
||||
};
|
||||
let entry_value = serde_json::to_value(&entry).unwrap();
|
||||
serde_json::to_value(&entry).unwrap()
|
||||
})
|
||||
.collect();
|
||||
let event = Event::Snapshot {
|
||||
entries: vec![entry_value],
|
||||
entries,
|
||||
greeting: Greeting {
|
||||
pod_name: "child".into(),
|
||||
cwd: "/tmp".into(),
|
||||
|
|
@ -175,6 +200,8 @@ fn serve_history(listener: UnixListener, items: Vec<Item>) -> JoinHandle<()> {
|
|||
model: "x".into(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
},
|
||||
status: protocol::PodStatus::Idle,
|
||||
};
|
||||
|
|
@ -183,6 +210,34 @@ fn serve_history(listener: UnixListener, items: Vec<Item>) -> JoinHandle<()> {
|
|||
})
|
||||
}
|
||||
|
||||
fn serve_pod_methods(listener: UnixListener) -> mpsc::Receiver<Method> {
|
||||
let (tx, rx) = mpsc::channel(8);
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
let Ok((stream, _)) = listener.accept().await else {
|
||||
return;
|
||||
};
|
||||
let (r, w) = stream.into_split();
|
||||
let mut reader = JsonLineReader::new(r);
|
||||
let mut writer = JsonLineWriter::new(w);
|
||||
if writer.write(&empty_snapshot()).await.is_err() {
|
||||
continue;
|
||||
}
|
||||
let Some(method) = reader.next::<Method>().await.ok().flatten() else {
|
||||
continue;
|
||||
};
|
||||
let is_shutdown = matches!(method, Method::Shutdown);
|
||||
if matches!(method, Method::Run { .. }) {
|
||||
let _ = writer.write(&Event::TurnStart { turn: 1 }).await;
|
||||
}
|
||||
if tx.send(method).await.is_err() || is_shutdown {
|
||||
return;
|
||||
}
|
||||
}
|
||||
});
|
||||
rx
|
||||
}
|
||||
|
||||
fn assistant(text: &str) -> Item {
|
||||
Item::Message {
|
||||
id: None,
|
||||
|
|
@ -328,45 +383,67 @@ async fn read_pod_output_reports_stopped_on_dead_socket() {
|
|||
#[tokio::test]
|
||||
async fn stop_pod_sends_shutdown_and_releases_scope() {
|
||||
let _env = EnvGuard::acquire();
|
||||
let (tmp, registry, rd) = setup_registry().await;
|
||||
let tmp = TempDir::new().unwrap();
|
||||
let store_tmp = TempDir::new().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
let rd = Arc::new(RuntimeDir::create(tmp.path(), "spawner").await.unwrap());
|
||||
let parent_scope = SharedScope::new(
|
||||
Scope::writable(tmp.path())
|
||||
.unwrap()
|
||||
.with_added_deny_rules([ScopeRule {
|
||||
target: tmp.path().to_path_buf(),
|
||||
permission: Permission::Write,
|
||||
recursive: true,
|
||||
}])
|
||||
.unwrap(),
|
||||
);
|
||||
unsafe {
|
||||
std::env::set_var("INSOMNIA_RUNTIME_DIR", tmp.path());
|
||||
}
|
||||
let lock_path = tmp.path().join("pods.json");
|
||||
|
||||
// Seed pods.json with a top-level `spawner` allocation plus a
|
||||
// delegated `child` allocation — mimics what SpawnPod would have
|
||||
// done so StopPod has something to release.
|
||||
// Seed pods.json with a restored top-level `spawner` allocation whose
|
||||
// scope_deny contains the delegated child path plus the live child
|
||||
// allocation — mimics a parent resumed after SpawnPod.
|
||||
{
|
||||
let mut g = LockFileGuard::open(&lock_path).unwrap();
|
||||
pod_registry::register_pod(
|
||||
let rule = ScopeRule {
|
||||
target: tmp.path().to_path_buf(),
|
||||
permission: Permission::Write,
|
||||
recursive: true,
|
||||
};
|
||||
pod_registry::register_pod_with_deny(
|
||||
&mut g,
|
||||
"spawner".into(),
|
||||
std::process::id(),
|
||||
"/tmp/spawner.sock".into(),
|
||||
vec![ScopeRule {
|
||||
target: tmp.path().to_path_buf(),
|
||||
permission: Permission::Write,
|
||||
recursive: true,
|
||||
}],
|
||||
session_store::new_session_id(),
|
||||
vec![rule.clone()],
|
||||
vec![rule.clone()],
|
||||
session_store::new_segment_id(),
|
||||
)
|
||||
.unwrap();
|
||||
pod_registry::delegate_scope(
|
||||
pod_registry::register_pod(
|
||||
&mut g,
|
||||
"spawner",
|
||||
"child".into(),
|
||||
std::process::id(),
|
||||
"/tmp/child.sock".into(),
|
||||
vec![ScopeRule {
|
||||
target: tmp.path().to_path_buf(),
|
||||
permission: Permission::Write,
|
||||
recursive: true,
|
||||
}],
|
||||
vec![rule],
|
||||
session_store::new_segment_id(),
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
let loaded = SpawnedPodRegistry::load_from_pod_state_with_reclaim(
|
||||
rd.clone(),
|
||||
store.clone(),
|
||||
"spawner".into(),
|
||||
Some(parent_scope.clone()),
|
||||
None,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let registry = loaded.registry;
|
||||
|
||||
let (socket, listener) = bind_mock_socket(tmp.path(), "child").await;
|
||||
let received = accept_one_method(listener);
|
||||
register_child(®istry, "child", &socket, tmp.path()).await;
|
||||
|
|
@ -381,12 +458,20 @@ async fn stop_pod_sends_shutdown_and_releases_scope() {
|
|||
let method = received.await.unwrap().expect("expected shutdown");
|
||||
assert!(matches!(method, Method::Shutdown));
|
||||
|
||||
// Allocation for `child` is gone; `spawner` remains.
|
||||
// Allocation for `child` is gone; `spawner` remains and its restored
|
||||
// dynamic deny layer has been reclaimed.
|
||||
{
|
||||
let g = LockFileGuard::open(&lock_path).unwrap();
|
||||
assert!(g.data().find("child").is_none(), "child still allocated");
|
||||
assert!(g.data().find("spawner").is_some(), "spawner missing");
|
||||
let spawner = g.data().find("spawner").expect("spawner missing");
|
||||
assert!(spawner.scope_deny.is_empty(), "deny not reclaimed");
|
||||
}
|
||||
assert_eq!(
|
||||
parent_scope
|
||||
.snapshot()
|
||||
.permission_at(&tmp.path().join("file.txt")),
|
||||
Some(Permission::Write)
|
||||
);
|
||||
|
||||
// spawned_pods.json now lists zero children.
|
||||
let spawned = rd.path().join("spawned_pods.json");
|
||||
|
|
@ -418,6 +503,214 @@ async fn stop_pod_succeeds_even_when_child_unreachable() {
|
|||
assert!(registry.get("child").await.is_none());
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Persistence / restore
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[tokio::test]
|
||||
async fn restored_registry_uses_pod_state_without_runtime_file() {
|
||||
let _env = EnvGuard::acquire();
|
||||
let runtime_tmp = TempDir::new().unwrap();
|
||||
let store_tmp = TempDir::new().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
unsafe {
|
||||
std::env::set_var("INSOMNIA_RUNTIME_DIR", runtime_tmp.path());
|
||||
}
|
||||
|
||||
let rd = Arc::new(
|
||||
RuntimeDir::create(runtime_tmp.path(), "spawner")
|
||||
.await
|
||||
.unwrap(),
|
||||
);
|
||||
let registry =
|
||||
SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".to_string())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let (socket, listener) = bind_mock_socket(runtime_tmp.path(), "child").await;
|
||||
let mut received = serve_pod_methods(listener);
|
||||
register_child(®istry, "child", &socket, runtime_tmp.path()).await;
|
||||
|
||||
std::fs::remove_file(rd.path().join("spawned_pods.json")).unwrap();
|
||||
|
||||
let restored =
|
||||
SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".to_string())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let def = list_pods_tool(restored.clone());
|
||||
let (_meta, tool) = def();
|
||||
let output: ToolOutput = tool.execute("{}").await.unwrap();
|
||||
assert!(output.summary.contains("1 pod"), "{}", output.summary);
|
||||
let body = output.content.expect("restored ListPods should list child");
|
||||
assert!(body.contains("child [alive]"), "body: {body}");
|
||||
|
||||
let def = send_to_pod_tool(restored.clone());
|
||||
let (_meta, tool) = def();
|
||||
let input = json!({ "name": "child", "message": "after restart" }).to_string();
|
||||
tool.execute(&input).await.unwrap();
|
||||
match received.recv().await.expect("expected Run") {
|
||||
Method::Run { input } => match input.as_slice() {
|
||||
[protocol::Segment::Text { content }] => assert_eq!(content, "after restart"),
|
||||
other => panic!("expected single Text segment, got {other:?}"),
|
||||
},
|
||||
other => panic!("expected Run, got {other:?}"),
|
||||
}
|
||||
|
||||
let def = stop_pod_tool(restored.clone());
|
||||
let (_meta, tool) = def();
|
||||
tool.execute(&json!({ "name": "child" }).to_string())
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(matches!(
|
||||
received.recv().await.expect("expected Shutdown"),
|
||||
Method::Shutdown
|
||||
));
|
||||
assert!(restored.get("child").await.is_none());
|
||||
|
||||
let metadata = store
|
||||
.read_by_name("spawner")
|
||||
.unwrap()
|
||||
.expect("spawner metadata should remain");
|
||||
assert!(metadata.spawned_children.is_empty());
|
||||
let runtime_contents = std::fs::read_to_string(rd.path().join("spawned_pods.json")).unwrap();
|
||||
let runtime_records: Vec<SpawnedPodRecord> = serde_json::from_str(&runtime_contents).unwrap();
|
||||
assert!(runtime_records.is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn load_from_pod_state_prunes_children_with_missing_sockets() {
|
||||
let runtime_tmp = TempDir::new().unwrap();
|
||||
let store_tmp = TempDir::new().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
let rd = Arc::new(
|
||||
RuntimeDir::create(runtime_tmp.path(), "spawner")
|
||||
.await
|
||||
.unwrap(),
|
||||
);
|
||||
let registry =
|
||||
SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".to_string())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let (live_socket, listener) = bind_mock_socket(runtime_tmp.path(), "alive").await;
|
||||
let _server = serve_pod_methods(listener);
|
||||
register_child(®istry, "alive", &live_socket, runtime_tmp.path()).await;
|
||||
register_child(
|
||||
®istry,
|
||||
"missing",
|
||||
&runtime_tmp.path().join("missing.sock"),
|
||||
runtime_tmp.path(),
|
||||
)
|
||||
.await;
|
||||
|
||||
let restored =
|
||||
SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".to_string())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(restored.get("alive").await.is_some());
|
||||
assert!(restored.get("missing").await.is_none());
|
||||
let metadata = store
|
||||
.read_by_name("spawner")
|
||||
.unwrap()
|
||||
.expect("spawner metadata should be written");
|
||||
assert_eq!(metadata.spawned_children.len(), 1);
|
||||
assert_eq!(metadata.spawned_children[0].pod_name, "alive");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn load_from_pod_state_reclaims_pruned_child_scope_and_registry_deny() {
|
||||
let _env = EnvGuard::acquire();
|
||||
let runtime_tmp = TempDir::new().unwrap();
|
||||
let store_tmp = TempDir::new().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
unsafe {
|
||||
std::env::set_var("INSOMNIA_RUNTIME_DIR", runtime_tmp.path());
|
||||
}
|
||||
let rd = Arc::new(
|
||||
RuntimeDir::create(runtime_tmp.path(), "spawner")
|
||||
.await
|
||||
.unwrap(),
|
||||
);
|
||||
let missing_rule = ScopeRule {
|
||||
target: runtime_tmp.path().to_path_buf(),
|
||||
permission: Permission::Write,
|
||||
recursive: true,
|
||||
};
|
||||
|
||||
{
|
||||
let mut g = LockFileGuard::open(&runtime_tmp.path().join("pods.json")).unwrap();
|
||||
pod_registry::register_pod_with_deny(
|
||||
&mut g,
|
||||
"spawner".into(),
|
||||
std::process::id(),
|
||||
"/tmp/spawner.sock".into(),
|
||||
vec![missing_rule.clone()],
|
||||
vec![missing_rule.clone()],
|
||||
session_store::new_segment_id(),
|
||||
)
|
||||
.unwrap();
|
||||
pod_registry::register_pod(
|
||||
&mut g,
|
||||
"missing".into(),
|
||||
std::process::id(),
|
||||
"/tmp/missing.sock".into(),
|
||||
vec![missing_rule.clone()],
|
||||
session_store::new_segment_id(),
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
let parent_scope = SharedScope::new(
|
||||
Scope::writable(runtime_tmp.path())
|
||||
.unwrap()
|
||||
.with_added_deny_rules([missing_rule.clone()])
|
||||
.unwrap(),
|
||||
);
|
||||
let seed = SpawnedPodRegistry::load_from_pod_state(rd.clone(), store.clone(), "spawner".into())
|
||||
.await
|
||||
.unwrap();
|
||||
seed.add(SpawnedPodRecord {
|
||||
pod_name: "missing".into(),
|
||||
socket_path: runtime_tmp.path().join("missing.sock"),
|
||||
scope_delegated: vec![missing_rule.clone()],
|
||||
callback_address: "/dev/null".into(),
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let loaded = SpawnedPodRegistry::load_from_pod_state_with_reclaim(
|
||||
rd.clone(),
|
||||
store.clone(),
|
||||
"spawner".into(),
|
||||
Some(parent_scope.clone()),
|
||||
None,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(loaded.reclaimed_unreachable);
|
||||
assert!(loaded.registry.get("missing").await.is_none());
|
||||
assert_eq!(
|
||||
parent_scope
|
||||
.snapshot()
|
||||
.permission_at(&runtime_tmp.path().join("file.txt")),
|
||||
Some(Permission::Write)
|
||||
);
|
||||
|
||||
let g = LockFileGuard::open(&runtime_tmp.path().join("pods.json")).unwrap();
|
||||
assert!(g.data().find("missing").is_none());
|
||||
assert!(g.data().find("spawner").unwrap().scope_deny.is_empty());
|
||||
let metadata = store
|
||||
.read_by_name("spawner")
|
||||
.unwrap()
|
||||
.expect("spawner metadata should remain");
|
||||
assert!(metadata.spawned_children.is_empty());
|
||||
let runtime_contents = std::fs::read_to_string(rd.path().join("spawned_pods.json")).unwrap();
|
||||
let runtime_records: Vec<SpawnedPodRecord> = serde_json::from_str(&runtime_contents).unwrap();
|
||||
assert!(runtime_records.is_empty());
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// ListPods
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
|
|||
|
|
@ -13,8 +13,8 @@ use pod::ipc::event::{apply_event_side_effects, fire_and_forget, render_event};
|
|||
use pod::runtime::dir::{RuntimeDir, SpawnedPodRecord};
|
||||
use pod::runtime::pod_registry::{self, LockFileGuard};
|
||||
use pod::spawn::registry::SpawnedPodRegistry;
|
||||
use protocol::stream::JsonLineReader;
|
||||
use protocol::{Method, Permission, PodEvent, ScopeRule};
|
||||
use protocol::stream::{JsonLineReader, JsonLineWriter};
|
||||
use protocol::{Event, Greeting, Method, Permission, PodEvent, PodStatus, ScopeRule};
|
||||
use tempfile::TempDir;
|
||||
use tokio::net::UnixListener;
|
||||
|
||||
|
|
@ -76,11 +76,32 @@ fn clear_runtime_dir() {
|
|||
}
|
||||
}
|
||||
|
||||
/// Accept a single connection, read one `Method`, and return it.
|
||||
/// Minimal connect-time snapshot used by mock parent sockets.
|
||||
fn empty_snapshot() -> Event {
|
||||
Event::Snapshot {
|
||||
entries: Vec::new(),
|
||||
greeting: Greeting {
|
||||
pod_name: "parent".into(),
|
||||
cwd: "/tmp".into(),
|
||||
provider: "test".into(),
|
||||
model: "test".into(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
},
|
||||
status: PodStatus::Idle,
|
||||
}
|
||||
}
|
||||
|
||||
/// Accept a single connection, send the protocol's connect-time snapshot,
|
||||
/// read one `Method`, and return it.
|
||||
fn accept_one_method(listener: UnixListener) -> tokio::task::JoinHandle<Option<Method>> {
|
||||
tokio::spawn(async move {
|
||||
let (stream, _) = listener.accept().await.ok()?;
|
||||
let (reader, _writer) = stream.into_split();
|
||||
let (reader, writer) = stream.into_split();
|
||||
let mut w = JsonLineWriter::new(writer);
|
||||
w.write(&empty_snapshot()).await.ok()?;
|
||||
let mut r = JsonLineReader::new(reader);
|
||||
r.next::<Method>().await.ok().flatten()
|
||||
})
|
||||
|
|
@ -358,7 +379,7 @@ async fn shutdown_releases_scope_allocation_when_present() {
|
|||
std::process::id(),
|
||||
"/tmp/kid.sock".into(),
|
||||
vec![],
|
||||
session_store::new_session_id(),
|
||||
session_store::new_segment_id(),
|
||||
)
|
||||
.unwrap();
|
||||
std::mem::forget(guard);
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
use std::sync::{LazyLock, Mutex};
|
||||
|
||||
use pod::{Pod, PodError};
|
||||
use session_store::{FsStore, SessionId, StoreError};
|
||||
use session_store::{FsStore, PodActiveSegmentRef, PodMetadata, PodMetadataStore, StoreError};
|
||||
|
||||
const MINIMAL_MANIFEST_TOML: &str = r#"
|
||||
[pod]
|
||||
|
|
@ -32,7 +32,97 @@ permission = "write"
|
|||
static ENV_LOCK: LazyLock<Mutex<()>> = LazyLock::new(|| Mutex::new(()));
|
||||
|
||||
#[tokio::test]
|
||||
async fn restore_from_manifest_rejects_unknown_session() {
|
||||
async fn restore_from_pod_metadata_rejects_missing_metadata() {
|
||||
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
|
||||
let store_tmp = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
|
||||
|
||||
let result = Pod::restore_from_pod_metadata(
|
||||
"restore-test",
|
||||
manifest,
|
||||
store,
|
||||
pod::PromptLoader::builtins_only(),
|
||||
)
|
||||
.await;
|
||||
|
||||
match result {
|
||||
Err(PodError::PodMetadataMissing { pod_name }) => assert_eq!(pod_name, "restore-test"),
|
||||
Err(other) => panic!("expected PodMetadataMissing, got {other:?}"),
|
||||
Ok(_) => panic!("expected missing pod metadata to fail"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn restore_from_pod_metadata_rejects_pending_segment() {
|
||||
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
|
||||
let store_tmp = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
|
||||
let session_id = session_store::new_session_id();
|
||||
store
|
||||
.write(&PodMetadata::new(
|
||||
"restore-test",
|
||||
Some(PodActiveSegmentRef::pending_segment(session_id)),
|
||||
))
|
||||
.unwrap();
|
||||
|
||||
let result = Pod::restore_from_pod_metadata(
|
||||
"restore-test",
|
||||
manifest,
|
||||
store,
|
||||
pod::PromptLoader::builtins_only(),
|
||||
)
|
||||
.await;
|
||||
|
||||
match result {
|
||||
Err(PodError::PodMetadataPending {
|
||||
pod_name,
|
||||
session_id: actual,
|
||||
}) => {
|
||||
assert_eq!(pod_name, "restore-test");
|
||||
assert_eq!(actual, session_id);
|
||||
}
|
||||
Err(other) => panic!("expected PodMetadataPending, got {other:?}"),
|
||||
Ok(_) => panic!("expected pending pod metadata to fail"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn restore_from_pod_metadata_resolves_active_pointer_through_session_log() {
|
||||
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
|
||||
let store_tmp = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
|
||||
let session_id = session_store::new_session_id();
|
||||
let segment_id = session_store::new_segment_id();
|
||||
store
|
||||
.write(&PodMetadata::new(
|
||||
"restore-test",
|
||||
Some(PodActiveSegmentRef::active_segment(session_id, segment_id)),
|
||||
))
|
||||
.unwrap();
|
||||
|
||||
let result = Pod::restore_from_pod_metadata(
|
||||
"restore-test",
|
||||
manifest,
|
||||
store,
|
||||
pod::PromptLoader::builtins_only(),
|
||||
)
|
||||
.await;
|
||||
|
||||
match result {
|
||||
Err(PodError::Store(StoreError::NotFound(id))) => assert_eq!(id, segment_id),
|
||||
Err(other) => panic!("expected Store(NotFound) from resolved segment, got {other:?}"),
|
||||
Ok(_) => panic!("expected unknown resolved segment to fail"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn restore_from_manifest_rejects_unknown_segment() {
|
||||
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
|
||||
let store_tmp = tempfile::tempdir().unwrap();
|
||||
|
|
@ -42,67 +132,87 @@ async fn restore_from_manifest_rejects_unknown_session() {
|
|||
// A freshly-minted id with no jsonl file at all → store returns
|
||||
// NotFound, which `Pod::restore_from_manifest` surfaces verbatim
|
||||
// as `PodError::Store`.
|
||||
let unknown = session_store::new_session_id();
|
||||
let result =
|
||||
Pod::restore_from_manifest(unknown, manifest, store, pod::PromptLoader::builtins_only())
|
||||
let unknown_sid = session_store::new_session_id();
|
||||
let unknown_seg = session_store::new_segment_id();
|
||||
let result = Pod::restore_from_manifest(
|
||||
unknown_sid,
|
||||
unknown_seg,
|
||||
manifest,
|
||||
store,
|
||||
pod::PromptLoader::builtins_only(),
|
||||
)
|
||||
.await;
|
||||
|
||||
match result {
|
||||
Err(PodError::Store(StoreError::NotFound(id))) => assert_eq!(id, unknown),
|
||||
Err(PodError::Store(StoreError::NotFound(id))) => assert_eq!(id, unknown_seg),
|
||||
Err(other) => panic!("expected Store(NotFound), got {other:?}"),
|
||||
Ok(_) => panic!("expected unknown session to fail"),
|
||||
Ok(_) => panic!("expected unknown segment to fail"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn restore_from_manifest_rejects_empty_session_log() {
|
||||
async fn restore_from_manifest_rejects_empty_segment_log() {
|
||||
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
|
||||
let store_tmp = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
|
||||
|
||||
// Pre-create an empty `<id>.jsonl` so `read_all` succeeds with no
|
||||
// entries. `collect_state` returns `head_hash = None`, which
|
||||
// `restore_from_manifest` rejects with `SessionEmpty` *before* it
|
||||
// gets as far as building the LLM client — so the test does not
|
||||
// need credentials or a runtime sandbox.
|
||||
let id: SessionId = session_store::new_session_id();
|
||||
let path = store_tmp.path().join(format!("{id}.jsonl"));
|
||||
std::fs::write(&path, b"").unwrap();
|
||||
// Pre-create an empty `<sid>/<segid>.jsonl` so `read_all` succeeds
|
||||
// with no entries. `collect_state` returns `entries_count = 0`,
|
||||
// which `restore_from_manifest` rejects with `SegmentEmpty` *before*
|
||||
// it gets as far as building the LLM client.
|
||||
let sid = session_store::new_session_id();
|
||||
let segid = session_store::new_segment_id();
|
||||
let dir = store_tmp.path().join(sid.to_string());
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
std::fs::write(dir.join(format!("{segid}.jsonl")), b"").unwrap();
|
||||
|
||||
let result =
|
||||
Pod::restore_from_manifest(id, manifest, store, pod::PromptLoader::builtins_only()).await;
|
||||
let result = Pod::restore_from_manifest(
|
||||
sid,
|
||||
segid,
|
||||
manifest,
|
||||
store,
|
||||
pod::PromptLoader::builtins_only(),
|
||||
)
|
||||
.await;
|
||||
|
||||
match result {
|
||||
Err(PodError::SessionEmpty { session_id }) => assert_eq!(session_id, id),
|
||||
Err(other) => panic!("expected SessionEmpty, got {other:?}"),
|
||||
Ok(_) => panic!("expected empty session log to fail"),
|
||||
Err(PodError::SegmentEmpty { segment_id }) => assert_eq!(segment_id, segid),
|
||||
Err(other) => panic!("expected SegmentEmpty, got {other:?}"),
|
||||
Ok(_) => panic!("expected empty segment log to fail"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn restore_from_manifest_rejects_session_without_scope_snapshot() {
|
||||
async fn restore_from_manifest_rejects_segment_without_scope_snapshot() {
|
||||
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
|
||||
let store_tmp = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(store_tmp.path()).unwrap();
|
||||
let manifest = pod::PodManifest::from_toml(MINIMAL_MANIFEST_TOML).unwrap();
|
||||
|
||||
let id = session_store::new_session_id();
|
||||
let state = session_store::SessionStartState {
|
||||
let sid = session_store::new_session_id();
|
||||
let segid = session_store::new_segment_id();
|
||||
let state = session_store::SegmentStartState {
|
||||
system_prompt: None,
|
||||
config: &Default::default(),
|
||||
history: &[],
|
||||
};
|
||||
session_store::create_session_with_id(&store, id, state).unwrap();
|
||||
session_store::create_segment_with_ids(&store, sid, segid, state).unwrap();
|
||||
|
||||
let result =
|
||||
Pod::restore_from_manifest(id, manifest, store, pod::PromptLoader::builtins_only()).await;
|
||||
let result = Pod::restore_from_manifest(
|
||||
sid,
|
||||
segid,
|
||||
manifest,
|
||||
store,
|
||||
pod::PromptLoader::builtins_only(),
|
||||
)
|
||||
.await;
|
||||
|
||||
match result {
|
||||
Err(PodError::SessionScopeMissing { session_id }) => assert_eq!(session_id, id),
|
||||
Err(other) => panic!("expected SessionScopeMissing, got {other:?}"),
|
||||
Err(PodError::SegmentScopeMissing { segment_id }) => assert_eq!(segment_id, segid),
|
||||
Err(other) => panic!("expected SegmentScopeMissing, got {other:?}"),
|
||||
Ok(_) => panic!("expected missing scope snapshot to fail"),
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,10 +4,10 @@
|
|||
//! returns a long `ToolOutput.content`, then inspects the persisted
|
||||
//! session log to verify:
|
||||
//!
|
||||
//! - `prune.skip { reason: "no_candidates" }` lands when the protected-turn
|
||||
//! window covers the entire history.
|
||||
//! - `prune.fire` lands once enough turns + usage measurements exist for
|
||||
//! the projection to actually apply.
|
||||
//! - `prune.skip { reason: "no_candidates" }` lands when usage estimates are
|
||||
//! unavailable or the protected-token window covers all tool results.
|
||||
//! - `prune.fire` lands once enough measured history exceeds the protected-token
|
||||
//! budget for the projection to actually apply.
|
||||
//! - The fire metric and the immediately-following `prune.post_request`
|
||||
//! metric share the same `correlation_id`, so cache_read / cache_write
|
||||
//! from the LlmUsage that triggered the projection can be joined back
|
||||
|
|
@ -26,9 +26,7 @@ use llm_worker::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEve
|
|||
use llm_worker::llm_client::{ClientError, LlmClient, Request};
|
||||
use llm_worker::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
|
||||
use session_metrics::{DOMAIN, Metric, metrics_from_extensions};
|
||||
use session_store::{
|
||||
EntryHash, FsStore, HashedEntry, LogEntry, SessionId, Store, StoreError, TraceEntry,
|
||||
};
|
||||
use session_store::{FsStore, LogEntry, SegmentId, SessionId, Store, StoreError, TraceEntry};
|
||||
|
||||
use pod::{Pod, PodManifest};
|
||||
|
||||
|
|
@ -138,7 +136,7 @@ fn text_response_with_cache(text: &str, cache_read: u64, cache_write: u64) -> Ve
|
|||
]
|
||||
}
|
||||
|
||||
fn manifest_toml(prune_protected_turns: usize, prune_min_savings: u64) -> String {
|
||||
fn manifest_toml(prune_protected_tokens: u64, prune_min_savings: u64) -> String {
|
||||
format!(
|
||||
r#"
|
||||
[pod]
|
||||
|
|
@ -153,7 +151,7 @@ model_id = "test-model"
|
|||
max_tokens = 100
|
||||
|
||||
[compaction]
|
||||
prune_protected_turns = {prune_protected_turns}
|
||||
prune_protected_tokens = {prune_protected_tokens}
|
||||
prune_min_savings = {prune_min_savings}
|
||||
|
||||
[[scope.allow]]
|
||||
|
|
@ -194,7 +192,7 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
|
|||
// Run 1 (request 0): tool_use → triggers tool execution → request 1
|
||||
// on the second iteration to produce the assistant reply.
|
||||
// Run 2 (request 2): plain assistant text. Prune evaluation here
|
||||
// sees user1's tool_result outside the 1-protected-turn window and
|
||||
// sees user1's tool_result outside the protected-token suffix and
|
||||
// should fire.
|
||||
let client = MockClient::new(vec![
|
||||
tool_use_response("call-1", "big_tool"),
|
||||
|
|
@ -203,6 +201,7 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
|
|||
]);
|
||||
let (mut pod, _store_tmp, _pwd_tmp) = make_pod(manifest_toml(1, 1), client, "big_tool").await;
|
||||
let session_id = pod.session_id();
|
||||
let segment_id = pod.segment_id();
|
||||
// Cloning the store handle to read the session log back after the
|
||||
// runs complete — the Pod retains its own copy.
|
||||
let store = pod.store().clone();
|
||||
|
|
@ -210,7 +209,7 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
|
|||
pod.run_text("first").await.unwrap();
|
||||
pod.run_text("second").await.unwrap();
|
||||
|
||||
let state = session_store::restore(&store, session_id).unwrap();
|
||||
let state = session_store::restore(&store, session_id, segment_id).unwrap();
|
||||
let metrics = metrics_from_extensions(&state.extensions);
|
||||
|
||||
// Run 1 has 2 LLM iterations (tool loop), each evaluates prune with
|
||||
|
|
@ -251,8 +250,8 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
|
|||
"fire missing candidate_count: {fire:?}"
|
||||
);
|
||||
assert!(
|
||||
fire.dimensions.contains_key("border_turn"),
|
||||
"fire missing border_turn: {fire:?}"
|
||||
fire.dimensions.contains_key("protected_start_index"),
|
||||
"fire missing protected_start_index: {fire:?}"
|
||||
);
|
||||
assert!(fire.value.is_some(), "fire missing estimated_savings value");
|
||||
let fire_id = fire
|
||||
|
|
@ -278,6 +277,36 @@ async fn prune_metrics_emit_skip_then_fire_with_post_request_join() {
|
|||
assert!(post.dimensions.contains_key("history_len"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn prune_metrics_fire_during_single_long_task_without_multiple_user_turns() {
|
||||
let client = MockClient::new(vec![
|
||||
tool_use_response("call-1", "big_tool"),
|
||||
tool_use_response("call-2", "big_tool"),
|
||||
tool_use_response("call-3", "big_tool"),
|
||||
tool_use_response("call-4", "big_tool"),
|
||||
text_response_with_cache("done", 100, 20),
|
||||
]);
|
||||
let (mut pod, _store_tmp, _pwd_tmp) = make_pod(manifest_toml(1, 1), client, "big_tool").await;
|
||||
let session_id = pod.session_id();
|
||||
let segment_id = pod.segment_id();
|
||||
let store = pod.store().clone();
|
||||
|
||||
pod.run_text("one long task").await.unwrap();
|
||||
|
||||
let state = session_store::restore(&store, session_id, segment_id).unwrap();
|
||||
let metrics = metrics_from_extensions(&state.extensions);
|
||||
let fire_count = metrics.iter().filter(|m| m.name == "prune.fire").count();
|
||||
assert!(
|
||||
fire_count > 0,
|
||||
"single-turn tool loop should produce prune.fire once old heavy ToolResults fall outside the protected-token suffix: {metrics:?}"
|
||||
);
|
||||
assert!(
|
||||
metrics.iter().any(|m| {
|
||||
m.name == "prune.fire" && m.dimensions.contains_key("protected_start_index")
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
/// `min_savings` set high enough that candidates exist but the estimated
|
||||
/// savings always fall short → the second run should record
|
||||
/// `prune.skip { reason: "below_min_savings" }`.
|
||||
|
|
@ -289,14 +318,15 @@ async fn prune_metrics_record_below_min_savings_skip() {
|
|||
text_response_with_cache("done", 0, 0),
|
||||
]);
|
||||
let (mut pod, _store_tmp, _pwd_tmp) =
|
||||
make_pod(manifest_toml(1, u64::MAX), client, "big_tool").await;
|
||||
make_pod(manifest_toml(1, 1_000_000), client, "big_tool").await;
|
||||
let session_id = pod.session_id();
|
||||
let segment_id = pod.segment_id();
|
||||
let store = pod.store().clone();
|
||||
|
||||
pod.run_text("first").await.unwrap();
|
||||
pod.run_text("second").await.unwrap();
|
||||
|
||||
let state = session_store::restore(&store, session_id).unwrap();
|
||||
let state = session_store::restore(&store, session_id, segment_id).unwrap();
|
||||
let metrics = metrics_from_extensions(&state.extensions);
|
||||
let below = metrics
|
||||
.iter()
|
||||
|
|
@ -329,35 +359,60 @@ struct MetricFailingStore {
|
|||
}
|
||||
|
||||
impl Store for MetricFailingStore {
|
||||
fn append(&self, id: SessionId, entry: &HashedEntry) -> Result<(), StoreError> {
|
||||
if let LogEntry::Extension { domain, .. } = &entry.entry {
|
||||
fn append(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entry: &LogEntry,
|
||||
) -> Result<(), StoreError> {
|
||||
if let LogEntry::Extension { domain, .. } = entry {
|
||||
if domain == DOMAIN {
|
||||
return Err(StoreError::Io(std::io::Error::other("synthetic failure")));
|
||||
}
|
||||
}
|
||||
self.inner.append(id, entry)
|
||||
self.inner.append(session_id, segment_id, entry)
|
||||
}
|
||||
fn read_all(&self, id: SessionId) -> Result<Vec<HashedEntry>, StoreError> {
|
||||
self.inner.read_all(id)
|
||||
fn read_all(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<Vec<LogEntry>, StoreError> {
|
||||
self.inner.read_all(session_id, segment_id)
|
||||
}
|
||||
fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError> {
|
||||
self.inner.list_sessions()
|
||||
}
|
||||
fn create_session(
|
||||
fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError> {
|
||||
self.inner.list_segments(session_id)
|
||||
}
|
||||
fn lookup_session_of(&self, segment_id: SegmentId) -> Result<Option<SessionId>, StoreError> {
|
||||
self.inner.lookup_session_of(segment_id)
|
||||
}
|
||||
fn create_segment(
|
||||
&self,
|
||||
id: SessionId,
|
||||
entries: &[HashedEntry],
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entries: &[LogEntry],
|
||||
) -> Result<(), StoreError> {
|
||||
self.inner.create_session(id, entries)
|
||||
self.inner.create_segment(session_id, segment_id, entries)
|
||||
}
|
||||
fn exists(&self, id: SessionId) -> Result<bool, StoreError> {
|
||||
self.inner.exists(id)
|
||||
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
|
||||
self.inner.exists(session_id, segment_id)
|
||||
}
|
||||
fn read_head_hash(&self, id: SessionId) -> Result<Option<EntryHash>, StoreError> {
|
||||
self.inner.read_head_hash(id)
|
||||
fn read_entry_count(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<usize, StoreError> {
|
||||
self.inner.read_entry_count(session_id, segment_id)
|
||||
}
|
||||
fn append_trace(&self, id: SessionId, entry: &TraceEntry) -> Result<(), StoreError> {
|
||||
self.inner.append_trace(id, entry)
|
||||
fn append_trace(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entry: &TraceEntry,
|
||||
) -> Result<(), StoreError> {
|
||||
self.inner.append_trace(session_id, segment_id, entry)
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -380,7 +435,7 @@ async fn metric_write_failure_emits_warn_alert_and_does_not_abort_run() {
|
|||
|
||||
// Even with a tool registered, this run will only emit
|
||||
// `prune.skip { reason: "no_candidates" }` (one user message,
|
||||
// protected_turns=1 covers everything). That is enough to drive
|
||||
// protected token budget covers the only user message). That is enough to drive
|
||||
// the failure path: at least one metric attempts to write.
|
||||
let client = MockClient::new(vec![text_response_with_cache("hi", 0, 0)]);
|
||||
let worker = Worker::new(client);
|
||||
|
|
@ -393,11 +448,12 @@ async fn metric_write_failure_emits_warn_alert_and_does_not_abort_run() {
|
|||
pod.attach_alerter(alerter);
|
||||
|
||||
let session_id = pod.session_id();
|
||||
let segment_id = pod.segment_id();
|
||||
// Run completes successfully despite metric failure.
|
||||
pod.run_text("hello").await.unwrap();
|
||||
|
||||
// No metrics ended up in the log (writes were rejected).
|
||||
let state = session_store::restore(&store, session_id).unwrap();
|
||||
let state = session_store::restore(&store, session_id, segment_id).unwrap();
|
||||
let metrics = metrics_from_extensions(&state.extensions);
|
||||
assert!(metrics.is_empty(), "metrics must drop on write failure");
|
||||
|
||||
|
|
@ -453,9 +509,10 @@ permission = "write"
|
|||
.await
|
||||
.unwrap();
|
||||
let session_id = pod.session_id();
|
||||
let segment_id = pod.segment_id();
|
||||
pod.run_text("hello").await.unwrap();
|
||||
|
||||
let state = session_store::restore(&store, session_id).unwrap();
|
||||
let state = session_store::restore(&store, session_id, segment_id).unwrap();
|
||||
let metrics = metrics_from_extensions(&state.extensions);
|
||||
assert!(
|
||||
metrics.is_empty(),
|
||||
|
|
|
|||
|
|
@ -73,7 +73,7 @@ async fn setup_spawner(
|
|||
permission: Permission::Write,
|
||||
recursive: true,
|
||||
}],
|
||||
session_store::new_session_id(),
|
||||
session_store::new_segment_id(),
|
||||
)
|
||||
.unwrap();
|
||||
// Leak the guard — the spawner allocation needs to outlive the
|
||||
|
|
|
|||
|
|
@ -182,16 +182,19 @@ async fn session_start_state_captures_rendered_prompt() {
|
|||
.unwrap();
|
||||
pod.run_text("hi").await.unwrap();
|
||||
|
||||
let entries = pod.store().read_all(pod.session_id()).unwrap();
|
||||
let entries = pod
|
||||
.store()
|
||||
.read_all(pod.session_id(), pod.segment_id())
|
||||
.unwrap();
|
||||
let first = entries.first().expect("at least one entry");
|
||||
match &first.entry {
|
||||
LogEntry::SessionStart { system_prompt, .. } => {
|
||||
match first {
|
||||
LogEntry::SegmentStart { system_prompt, .. } => {
|
||||
let sp = system_prompt.as_deref().expect("system prompt set");
|
||||
assert!(sp.starts_with("hello cwd="));
|
||||
assert!(sp.contains(&pwd.display().to_string()));
|
||||
assert!(sp.contains("## Working boundaries"));
|
||||
}
|
||||
other => panic!("expected SessionStart as first entry, got {other:?}"),
|
||||
other => panic!("expected SegmentStart as first entry, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -226,9 +226,7 @@ pub enum Event {
|
|||
/// `[File: …]`.
|
||||
///
|
||||
/// One event per `LogEntry::SystemItem` commit. Disk-side and
|
||||
/// wire-side are 1:1 (singular variant); legacy `SystemItems`
|
||||
/// entries from older sessions are read-only and never emitted on
|
||||
/// this lane.
|
||||
/// wire-side are 1:1.
|
||||
SystemItem {
|
||||
item: serde_json::Value,
|
||||
},
|
||||
|
|
@ -363,24 +361,23 @@ pub enum Event {
|
|||
///
|
||||
/// Live updates after the snapshot arrive through the streaming
|
||||
/// events (`TextDelta` / `ToolCall*` / `ToolResult` / etc.) plus
|
||||
/// the two role-specific entry events
|
||||
/// (`SessionRotated` / `HookInjectedItems`) — there is no generic
|
||||
/// "every committed entry" broadcast.
|
||||
/// role-specific entry events (`SegmentRotated` / `SystemItem`) —
|
||||
/// there is no generic "every committed entry" broadcast.
|
||||
Snapshot {
|
||||
entries: Vec<serde_json::Value>,
|
||||
greeting: Greeting,
|
||||
#[serde(default)]
|
||||
status: PodStatus,
|
||||
},
|
||||
/// Server-side session log rotated to a fresh `SessionStart`.
|
||||
/// Server-side segment log rotated to a fresh `SegmentStart`.
|
||||
///
|
||||
/// Fires on compaction and on auto-fork when the store head drifts
|
||||
/// from the live writer's cached head. Clients drop their derived
|
||||
/// view and reseed from `entry.history` exactly the way they would
|
||||
/// from a connect-time `Snapshot`.
|
||||
///
|
||||
/// Payload is the JSON form of `session_store::LogEntry::SessionStart`.
|
||||
SessionRotated {
|
||||
/// Payload is the JSON form of `session_store::LogEntry::SegmentStart`.
|
||||
SegmentRotated {
|
||||
entry: serde_json::Value,
|
||||
},
|
||||
/// Current Pod controller status. Broadcast on every controller-level
|
||||
|
|
@ -400,15 +397,15 @@ pub enum Event {
|
|||
/// Pod has started compacting the current session.
|
||||
///
|
||||
/// Fired immediately before a compaction run. Success is signalled by
|
||||
/// `CompactDone` (with the new `SessionId`); failure by `CompactFailed`.
|
||||
/// `CompactDone` (with the new `SegmentId`); failure by `CompactFailed`.
|
||||
/// Broadcast to all clients; not replayed to late subscribers.
|
||||
CompactStart,
|
||||
/// Compaction completed and the session was rotated.
|
||||
///
|
||||
/// `new_session_id` is the UUID of the freshly created session that
|
||||
/// `new_segment_id` is the UUID of the freshly created session that
|
||||
/// replaced the old history.
|
||||
CompactDone {
|
||||
new_session_id: uuid::Uuid,
|
||||
new_segment_id: uuid::Uuid,
|
||||
},
|
||||
/// Compaction failed. The session is unchanged.
|
||||
CompactFailed {
|
||||
|
|
@ -489,6 +486,12 @@ pub struct Greeting {
|
|||
pub model: String,
|
||||
pub scope_summary: String,
|
||||
pub tools: Vec<String>,
|
||||
/// Model context window in tokens. Always filled by the Pod greeting.
|
||||
#[serde(default)]
|
||||
pub context_window: u64,
|
||||
/// Estimated current session context tokens at connect time.
|
||||
#[serde(default)]
|
||||
pub context_tokens: u64,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
@ -876,6 +879,8 @@ mod tests {
|
|||
model: "claude".into(),
|
||||
scope_summary: "Writable:\n - /tmp".into(),
|
||||
tools: vec!["Read".into()],
|
||||
context_window: 200_000,
|
||||
context_tokens: 42_000,
|
||||
},
|
||||
status: PodStatus::Paused,
|
||||
};
|
||||
|
|
@ -886,22 +891,24 @@ mod tests {
|
|||
assert_eq!(parsed["data"]["entries"][0]["kind"], "user_input");
|
||||
assert_eq!(parsed["data"]["greeting"]["pod_name"], "test");
|
||||
assert_eq!(parsed["data"]["greeting"]["tools"][0], "Read");
|
||||
assert_eq!(parsed["data"]["greeting"]["context_window"], 200_000);
|
||||
assert_eq!(parsed["data"]["greeting"]["context_tokens"], 42_000);
|
||||
assert_eq!(parsed["data"]["status"], "paused");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn event_session_rotated_roundtrip() {
|
||||
let event = Event::SessionRotated {
|
||||
entry: serde_json::json!({"kind": "session_start", "ts": 1, "history": []}),
|
||||
fn event_segment_rotated_roundtrip() {
|
||||
let event = Event::SegmentRotated {
|
||||
entry: serde_json::json!({"kind": "segment_start", "ts": 1, "history": []}),
|
||||
};
|
||||
let json = serde_json::to_string(&event).unwrap();
|
||||
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(parsed["event"], "session_rotated");
|
||||
assert_eq!(parsed["data"]["entry"]["kind"], "session_start");
|
||||
assert_eq!(parsed["event"], "segment_rotated");
|
||||
assert_eq!(parsed["data"]["entry"]["kind"], "segment_start");
|
||||
let decoded: Event = serde_json::from_str(&json).unwrap();
|
||||
match decoded {
|
||||
Event::SessionRotated { entry } => assert_eq!(entry["kind"], "session_start"),
|
||||
other => panic!("expected SessionRotated, got {other:?}"),
|
||||
Event::SegmentRotated { entry } => assert_eq!(entry["kind"], "segment_start"),
|
||||
other => panic!("expected SegmentRotated, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -945,7 +952,13 @@ mod tests {
|
|||
let json = r#"{"event":"snapshot","data":{"entries":[],"greeting":{"pod_name":"test","cwd":"/tmp","provider":"anthropic","model":"claude","scope_summary":"","tools":[]}}}"#;
|
||||
let decoded: Event = serde_json::from_str(json).unwrap();
|
||||
match decoded {
|
||||
Event::Snapshot { status, .. } => assert_eq!(status, PodStatus::Idle),
|
||||
Event::Snapshot {
|
||||
status, greeting, ..
|
||||
} => {
|
||||
assert_eq!(status, PodStatus::Idle);
|
||||
assert_eq!(greeting.context_window, 0);
|
||||
assert_eq!(greeting.context_tokens, 0);
|
||||
}
|
||||
other => panic!("expected Snapshot, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
|
@ -1060,17 +1073,17 @@ mod tests {
|
|||
#[test]
|
||||
fn event_compact_done_roundtrip() {
|
||||
let id = uuid::Uuid::parse_str("0192f0e8-4d84-7d6e-a000-000000000001").unwrap();
|
||||
let event = Event::CompactDone { new_session_id: id };
|
||||
let event = Event::CompactDone { new_segment_id: id };
|
||||
let json = serde_json::to_string(&event).unwrap();
|
||||
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(parsed["event"], "compact_done");
|
||||
assert_eq!(
|
||||
parsed["data"]["new_session_id"],
|
||||
parsed["data"]["new_segment_id"],
|
||||
"0192f0e8-4d84-7d6e-a000-000000000001"
|
||||
);
|
||||
let decoded: Event = serde_json::from_str(&json).unwrap();
|
||||
match decoded {
|
||||
Event::CompactDone { new_session_id } => assert_eq!(new_session_id, id),
|
||||
Event::CompactDone { new_segment_id } => assert_eq!(new_segment_id, id),
|
||||
other => panic!("expected CompactDone, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -18,3 +18,4 @@
|
|||
- `AuthRef::None` / `AuthRef::CodexOAuth` の解決
|
||||
- `Scheme::required_auth()` と `ResolvedAuth` の妥当性検証(非対応組合せは構築エラー)
|
||||
- capability は manifest 明示 > model catalog > provider.default_capability > `Scheme::default_capability()` の順で解決
|
||||
- context window は manifest 明示 > model catalog > provider.default_context_window > builtin fallback の順で解決し、inline model でも `context_window` で override できる
|
||||
|
|
|
|||
|
|
@ -22,6 +22,11 @@ use serde::{Deserialize, Serialize};
|
|||
const BUILTIN_PROVIDERS: &str = include_str!("../../../resources/providers/builtin.toml");
|
||||
const BUILTIN_MODELS: &str = include_str!("../../../resources/models/builtin.toml");
|
||||
|
||||
/// Conservative fallback used when neither the manifest nor catalogs specify
|
||||
/// a model context window. Greeting still carries a concrete number, while
|
||||
/// catalog / manifest metadata can override unknown or inline models.
|
||||
pub const DEFAULT_CONTEXT_WINDOW: u64 = 200_000;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum CatalogError {
|
||||
#[error("failed to read catalog at {path}: {source}")]
|
||||
|
|
@ -92,6 +97,10 @@ pub struct ProviderEntry {
|
|||
/// 使う。
|
||||
#[serde(default)]
|
||||
pub default_capability: Option<ModelCapability>,
|
||||
/// モデルカタログ未登録モデルで使う既定の context window。省略時は
|
||||
/// [`DEFAULT_CONTEXT_WINDOW`] を使う。
|
||||
#[serde(default)]
|
||||
pub default_context_window: Option<u64>,
|
||||
}
|
||||
|
||||
/// モデルカタログの 1 エントリ。
|
||||
|
|
@ -107,6 +116,10 @@ pub struct ModelEntry {
|
|||
/// `ProviderEntry::default_capability` にフォールバックする。
|
||||
#[serde(default)]
|
||||
pub capability: Option<ModelCapability>,
|
||||
/// モデル単位の context window。省略時は provider default → builtin
|
||||
/// fallback にフォールバックする。
|
||||
#[serde(default)]
|
||||
pub context_window: Option<u64>,
|
||||
}
|
||||
|
||||
/// 解決済みモデル設定。`build_client` が消費する完成形。
|
||||
|
|
@ -117,6 +130,7 @@ pub struct ModelConfig {
|
|||
pub model_id: String,
|
||||
pub auth: AuthRef,
|
||||
pub capability: Option<ModelCapability>,
|
||||
pub context_window: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
|
|
@ -244,6 +258,8 @@ fn split_ref(s: &str) -> Option<(&str, &str)> {
|
|||
/// auth は manifest 明示 > provider.auth_hint 由来、capability は
|
||||
/// manifest 明示 > model catalog > provider.default_capability >
|
||||
/// (`build_client` 側で)`Scheme::default_capability()`。
|
||||
/// context_window は manifest 明示 > model catalog > provider default >
|
||||
/// [`DEFAULT_CONTEXT_WINDOW`]。
|
||||
pub fn resolve_model_manifest(manifest: &ModelManifest) -> Result<ModelConfig, ResolveError> {
|
||||
let providers = load_providers().map_err(ResolveError::LoadProviders)?;
|
||||
let models = load_models().map_err(ResolveError::LoadModels)?;
|
||||
|
|
@ -294,12 +310,18 @@ pub fn resolve_with_catalogs(
|
|||
.and_then(|m| m.capability.clone())
|
||||
.or_else(|| provider.default_capability.clone())
|
||||
});
|
||||
let context_window = manifest
|
||||
.context_window
|
||||
.or_else(|| model_entry.and_then(|m| m.context_window))
|
||||
.or(provider.default_context_window)
|
||||
.unwrap_or(DEFAULT_CONTEXT_WINDOW);
|
||||
Ok(ModelConfig {
|
||||
scheme,
|
||||
base_url,
|
||||
model_id,
|
||||
auth,
|
||||
capability,
|
||||
context_window,
|
||||
})
|
||||
} else {
|
||||
let scheme = manifest
|
||||
|
|
@ -319,6 +341,7 @@ pub fn resolve_with_catalogs(
|
|||
model_id,
|
||||
auth,
|
||||
capability: manifest.capability.clone(),
|
||||
context_window: manifest.context_window.unwrap_or(DEFAULT_CONTEXT_WINDOW),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
@ -381,6 +404,20 @@ mod tests {
|
|||
cfg.capability.is_some(),
|
||||
"should fall back to provider.default_capability"
|
||||
);
|
||||
assert_eq!(cfg.context_window, 200_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn context_window_manifest_overrides_catalog() {
|
||||
let providers = load_builtin_providers().unwrap();
|
||||
let models = load_builtin_models().unwrap();
|
||||
let manifest = ModelManifest {
|
||||
ref_: Some("anthropic/claude-sonnet-4-6".into()),
|
||||
context_window: Some(123_456),
|
||||
..Default::default()
|
||||
};
|
||||
let cfg = resolve_with_catalogs(&manifest, &providers, &models).unwrap();
|
||||
assert_eq!(cfg.context_window, 123_456);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -461,6 +498,25 @@ mod tests {
|
|||
assert_eq!(cfg.scheme, SchemeKind::Anthropic);
|
||||
assert_eq!(cfg.model_id, "claude-sonnet-4-6");
|
||||
assert!(cfg.capability.is_none(), "no catalog hit for inline-only");
|
||||
assert_eq!(cfg.context_window, DEFAULT_CONTEXT_WINDOW);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resolve_inline_context_window_override() {
|
||||
let providers = load_builtin_providers().unwrap();
|
||||
let models = load_builtin_models().unwrap();
|
||||
let manifest = ModelManifest {
|
||||
scheme: Some(SchemeKind::Anthropic),
|
||||
model_id: Some("claude-sonnet-4-6".into()),
|
||||
auth: Some(AuthRef::ApiKey {
|
||||
env: None,
|
||||
file: Some(PathBuf::from("/tmp/sk")),
|
||||
}),
|
||||
context_window: Some(777_000),
|
||||
..Default::default()
|
||||
};
|
||||
let cfg = resolve_with_catalogs(&manifest, &providers, &models).unwrap();
|
||||
assert_eq!(cfg.context_window, 777_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -186,6 +186,7 @@ mod tests {
|
|||
file: None,
|
||||
},
|
||||
capability: None,
|
||||
context_window: 200_000,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -313,6 +314,7 @@ mod tests {
|
|||
model_id: "llama3".into(),
|
||||
auth: AuthRef::None,
|
||||
capability: None,
|
||||
context_window: 200_000,
|
||||
};
|
||||
assert!(build_client_from_config(&config).is_ok());
|
||||
}
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@
|
|||
use std::collections::BTreeMap;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use session_store::{EntryHash, SessionId, Store, StoreError, save_extension, session_log};
|
||||
use session_store::{SegmentId, SessionId, Store, StoreError, save_extension, segment_log};
|
||||
|
||||
/// Domain tag used in `LogEntry::Extension` for all metrics records.
|
||||
pub const DOMAIN: &str = "metrics";
|
||||
|
|
@ -48,7 +48,7 @@ impl Metric {
|
|||
pub fn now(name: impl Into<String>) -> Self {
|
||||
Self {
|
||||
name: name.into(),
|
||||
ts: session_log::now_millis(),
|
||||
ts: segment_log::now_millis(),
|
||||
dimensions: BTreeMap::new(),
|
||||
value: None,
|
||||
correlation_id: None,
|
||||
|
|
@ -78,11 +78,11 @@ impl Metric {
|
|||
pub fn record_metric(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
segment_id: SegmentId,
|
||||
metric: &Metric,
|
||||
) -> Result<(), StoreError> {
|
||||
let payload = serde_json::to_value(metric).expect("Metric serialization cannot fail");
|
||||
save_extension(store, session_id, head_hash, DOMAIN, payload)
|
||||
save_extension(store, session_id, segment_id, DOMAIN, payload)
|
||||
}
|
||||
|
||||
/// `RestoredState.extensions` から metrics domain の payload を順に取り出し、
|
||||
|
|
@ -104,7 +104,7 @@ mod tests {
|
|||
#[test]
|
||||
fn metric_round_trip_via_json() {
|
||||
let metric = Metric::now("prune.fire")
|
||||
.with_dimension("border_turn", "3")
|
||||
.with_dimension("protected_start_index", "3")
|
||||
.with_dimension("candidate_count", "2")
|
||||
.with_value(4096.0)
|
||||
.with_correlation_id("abc-123");
|
||||
|
|
|
|||
|
|
@ -11,8 +11,6 @@ serde = { workspace = true, features = ["derive"] }
|
|||
serde_json = { workspace = true }
|
||||
uuid = { workspace = true, features = ["v7", "serde"] }
|
||||
thiserror = { workspace = true }
|
||||
sha2 = { workspace = true }
|
||||
hex = "0.4.3"
|
||||
protocol = { workspace = true }
|
||||
tracing.workspace = true
|
||||
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ Worker のセッション永続化を提供するクレート。追記専用の
|
|||
|
||||
### ログ
|
||||
|
||||
- `LogEntry` — セッションログのエントリ型(`SessionStart`, `UserInput`, `AssistantItems`, `TurnEnd` など)
|
||||
- `LogEntry` — セッションログのエントリ型(`SegmentStart`, `UserInput`, `AssistantItem`, `ToolResult`, `SystemItem`, `TurnEnd` など)
|
||||
- `RestoredState` — ログ再生で復元された状態
|
||||
- `collect_state()` — ログエントリ列から状態を復元する関数
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
//!
|
||||
//! [`TraceEntry`] captures every LLM stream event verbatim for debugging
|
||||
//! and post-hoc analysis. Written to a separate `.trace.jsonl` file,
|
||||
//! completely independent of the session log used for state restoration.
|
||||
//! completely independent of the segment log used for state restoration.
|
||||
//!
|
||||
//! Disabled by default. Enable via `SessionConfig::record_event_trace`.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,20 +1,33 @@
|
|||
//! Filesystem-backed JSONL store.
|
||||
//!
|
||||
//! Layout:
|
||||
//! - Session log: `{root}/{session_id}.jsonl`
|
||||
//! - Event trace: `{root}/{session_id}.trace.jsonl`
|
||||
//! - Segment log: `{root}/{session_id}/{segment_id}.jsonl`
|
||||
//! - Event trace: `{root}/{session_id}/{segment_id}.trace.jsonl`
|
||||
//! - Pod metadata: `{root}/pods/{pod_name}/metadata.json`
|
||||
//!
|
||||
//! The per-Session directory makes `list_segments(session_id)` an O(dir)
|
||||
//! scan and gives the fork tree a visible grouping in the filesystem.
|
||||
//!
|
||||
//! Migration: this layout is incompatible with the pre-`session-grouping`
|
||||
//! flat `{root}/{segment_id}.jsonl` form. Project policy is no
|
||||
//! backward compatibility — discard `~/.insomnia/sessions/` (or whatever
|
||||
//! `root` resolved to) before running the new code. `list_sessions`
|
||||
//! ignores top-level files outside session directories, so leftover
|
||||
//! flat files do not corrupt new sessions, but they are no longer
|
||||
//! enumerable by the picker.
|
||||
|
||||
use crate::SessionId;
|
||||
use crate::event_trace::TraceEntry;
|
||||
use crate::session_log::{EntryHash, HashedEntry};
|
||||
use crate::pod_metadata::{PodMetadata, PodMetadataStore, validate_pod_name};
|
||||
use crate::segment_log::LogEntry;
|
||||
use crate::store::{Store, StoreError};
|
||||
use crate::{SegmentId, SessionId};
|
||||
use std::fs;
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// Filesystem-backed JSONL store.
|
||||
///
|
||||
/// Each session is stored as a single `.jsonl` file with one [`LogEntry`]
|
||||
/// Each segment is stored as a single `.jsonl` file with one [`LogEntry`]
|
||||
/// per line. Writes use append mode for crash safety.
|
||||
#[derive(Clone)]
|
||||
pub struct FsStore {
|
||||
|
|
@ -30,16 +43,41 @@ impl FsStore {
|
|||
Ok(Self { root })
|
||||
}
|
||||
|
||||
fn log_path(&self, id: SessionId) -> PathBuf {
|
||||
self.root.join(format!("{id}.jsonl"))
|
||||
fn session_dir(&self, session_id: SessionId) -> PathBuf {
|
||||
self.root.join(session_id.to_string())
|
||||
}
|
||||
|
||||
fn trace_path(&self, id: SessionId) -> PathBuf {
|
||||
self.root.join(format!("{id}.trace.jsonl"))
|
||||
fn log_path(&self, session_id: SessionId, segment_id: SegmentId) -> PathBuf {
|
||||
self.session_dir(session_id)
|
||||
.join(format!("{segment_id}.jsonl"))
|
||||
}
|
||||
|
||||
fn trace_path(&self, session_id: SessionId, segment_id: SegmentId) -> PathBuf {
|
||||
self.session_dir(session_id)
|
||||
.join(format!("{segment_id}.trace.jsonl"))
|
||||
}
|
||||
|
||||
fn pods_dir(&self) -> PathBuf {
|
||||
self.root.join("pods")
|
||||
}
|
||||
|
||||
fn pod_dir(&self, pod_name: &str) -> Result<PathBuf, StoreError> {
|
||||
validate_pod_name(pod_name)?;
|
||||
Ok(self.pods_dir().join(pod_name))
|
||||
}
|
||||
|
||||
fn pod_metadata_path(&self, pod_name: &str) -> Result<PathBuf, StoreError> {
|
||||
Ok(self.pod_dir(pod_name)?.join("metadata.json"))
|
||||
}
|
||||
|
||||
fn append_line(&self, path: &Path, line: &str) -> Result<(), StoreError> {
|
||||
let mut file = fs::OpenOptions::new().create(true).append(true).open(path)?;
|
||||
if let Some(parent) = path.parent() {
|
||||
fs::create_dir_all(parent)?;
|
||||
}
|
||||
let mut file = fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(path)?;
|
||||
file.write_all(line.as_bytes())?;
|
||||
file.write_all(b"\n")?;
|
||||
// Append-mode write is the durability boundary; an explicit
|
||||
|
|
@ -64,16 +102,60 @@ impl FsStore {
|
|||
}
|
||||
}
|
||||
|
||||
impl Store for FsStore {
|
||||
fn append(&self, id: SessionId, entry: &HashedEntry) -> Result<(), StoreError> {
|
||||
let line = serde_json::to_string(entry)?;
|
||||
self.append_line(&self.log_path(id), &line)
|
||||
impl PodMetadataStore for FsStore {
|
||||
fn write(&self, metadata: &PodMetadata) -> Result<(), StoreError> {
|
||||
let path = self.pod_metadata_path(&metadata.pod_name)?;
|
||||
if let Some(parent) = path.parent() {
|
||||
fs::create_dir_all(parent)?;
|
||||
}
|
||||
let content = serde_json::to_vec_pretty(metadata)?;
|
||||
fs::write(path, content)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn read_all(&self, id: SessionId) -> Result<Vec<HashedEntry>, StoreError> {
|
||||
let path = self.log_path(id);
|
||||
fn read_by_name(&self, pod_name: &str) -> Result<Option<PodMetadata>, StoreError> {
|
||||
let path = self.pod_metadata_path(pod_name)?;
|
||||
let content = match fs::read_to_string(path) {
|
||||
Ok(content) => content,
|
||||
Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(None),
|
||||
Err(err) => return Err(StoreError::Io(err)),
|
||||
};
|
||||
Ok(Some(serde_json::from_str(&content)?))
|
||||
}
|
||||
|
||||
fn delete_by_name(&self, pod_name: &str) -> Result<(), StoreError> {
|
||||
let path = self.pod_metadata_path(pod_name)?;
|
||||
match fs::remove_file(&path) {
|
||||
Ok(()) => {}
|
||||
Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(()),
|
||||
Err(err) => return Err(StoreError::Io(err)),
|
||||
}
|
||||
if let Some(parent) = path.parent() {
|
||||
let _ = fs::remove_dir(parent);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl Store for FsStore {
|
||||
fn append(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entry: &LogEntry,
|
||||
) -> Result<(), StoreError> {
|
||||
let line = serde_json::to_string(entry)?;
|
||||
self.append_line(&self.log_path(session_id, segment_id), &line)
|
||||
}
|
||||
|
||||
fn read_all(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<Vec<LogEntry>, StoreError> {
|
||||
let path = self.log_path(session_id, segment_id);
|
||||
if !path.exists() {
|
||||
return Err(StoreError::NotFound(id));
|
||||
return Err(StoreError::NotFound(segment_id));
|
||||
}
|
||||
let content = fs::read_to_string(&path)?;
|
||||
Self::parse_jsonl(&content)
|
||||
|
|
@ -81,25 +163,77 @@ impl Store for FsStore {
|
|||
|
||||
fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError> {
|
||||
let mut sessions = Vec::new();
|
||||
if !self.root.exists() {
|
||||
return Ok(sessions);
|
||||
}
|
||||
for entry in fs::read_dir(&self.root)? {
|
||||
let entry = entry?;
|
||||
if !entry.file_type()?.is_dir() {
|
||||
continue;
|
||||
}
|
||||
if let Some(name) = entry.file_name().to_str() {
|
||||
if let Ok(id) = name.parse::<SessionId>() {
|
||||
sessions.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
sessions.sort_by(|a, b| b.cmp(a));
|
||||
Ok(sessions)
|
||||
}
|
||||
|
||||
fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError> {
|
||||
let dir = self.session_dir(session_id);
|
||||
let mut segments = Vec::new();
|
||||
if !dir.exists() {
|
||||
return Ok(segments);
|
||||
}
|
||||
for entry in fs::read_dir(&dir)? {
|
||||
let entry = entry?;
|
||||
let path = entry.path();
|
||||
// Only match .jsonl files, not .trace.jsonl
|
||||
let name = path.file_name().and_then(|n| n.to_str()).unwrap_or("");
|
||||
if name.ends_with(".jsonl") && !name.ends_with(".trace.jsonl") {
|
||||
let stem = name.trim_end_matches(".jsonl");
|
||||
if let Ok(id) = stem.parse::<SessionId>() {
|
||||
sessions.push(id);
|
||||
if let Ok(id) = stem.parse::<SegmentId>() {
|
||||
segments.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
// UUID v7: lexicographic sort = chronological sort, newest first
|
||||
sessions.sort_by(|a, b| b.cmp(a));
|
||||
Ok(sessions)
|
||||
segments.sort_by(|a, b| b.cmp(a));
|
||||
Ok(segments)
|
||||
}
|
||||
|
||||
fn create_session(&self, id: SessionId, entries: &[HashedEntry]) -> Result<(), StoreError> {
|
||||
let path = self.log_path(id);
|
||||
fn lookup_session_of(&self, segment_id: SegmentId) -> Result<Option<SessionId>, StoreError> {
|
||||
if !self.root.exists() {
|
||||
return Ok(None);
|
||||
}
|
||||
let needle = format!("{segment_id}.jsonl");
|
||||
for entry in fs::read_dir(&self.root)? {
|
||||
let entry = entry?;
|
||||
if !entry.file_type()?.is_dir() {
|
||||
continue;
|
||||
}
|
||||
if entry.path().join(&needle).exists()
|
||||
&& let Some(name) = entry.file_name().to_str()
|
||||
&& let Ok(id) = name.parse::<SessionId>()
|
||||
{
|
||||
return Ok(Some(id));
|
||||
}
|
||||
}
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
fn create_segment(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entries: &[LogEntry],
|
||||
) -> Result<(), StoreError> {
|
||||
let path = self.log_path(session_id, segment_id);
|
||||
if let Some(parent) = path.parent() {
|
||||
fs::create_dir_all(parent)?;
|
||||
}
|
||||
let mut content = String::new();
|
||||
for entry in entries {
|
||||
content.push_str(&serde_json::to_string(entry)?);
|
||||
|
|
@ -109,32 +243,30 @@ impl Store for FsStore {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
fn exists(&self, id: SessionId) -> Result<bool, StoreError> {
|
||||
Ok(self.log_path(id).exists())
|
||||
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError> {
|
||||
Ok(self.log_path(session_id, segment_id).exists())
|
||||
}
|
||||
|
||||
fn read_head_hash(&self, id: SessionId) -> Result<Option<EntryHash>, StoreError> {
|
||||
let path = self.log_path(id);
|
||||
fn read_entry_count(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<usize, StoreError> {
|
||||
let path = self.log_path(session_id, segment_id);
|
||||
if !path.exists() {
|
||||
return Err(StoreError::NotFound(id));
|
||||
return Err(StoreError::NotFound(segment_id));
|
||||
}
|
||||
let content = fs::read_to_string(&path)?;
|
||||
let last_line = content.lines().rev().find(|l| !l.trim().is_empty());
|
||||
match last_line {
|
||||
Some(line) => {
|
||||
let entry: HashedEntry =
|
||||
serde_json::from_str(line).map_err(|e| StoreError::Corrupt {
|
||||
line: content.lines().count(),
|
||||
message: e.to_string(),
|
||||
})?;
|
||||
Ok(Some(entry.hash))
|
||||
}
|
||||
None => Ok(None),
|
||||
}
|
||||
Ok(content.lines().filter(|l| !l.trim().is_empty()).count())
|
||||
}
|
||||
|
||||
fn append_trace(&self, id: SessionId, entry: &TraceEntry) -> Result<(), StoreError> {
|
||||
fn append_trace(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entry: &TraceEntry,
|
||||
) -> Result<(), StoreError> {
|
||||
let line = serde_json::to_string(entry)?;
|
||||
self.append_line(&self.trace_path(id), &line)
|
||||
self.append_line(&self.trace_path(session_id, segment_id), &line)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,35 +2,40 @@
|
|||
//!
|
||||
//! # Architecture
|
||||
//!
|
||||
//! Sessions are recorded as a sequence of [`LogEntry`] values, one per line
|
||||
//! in a `.jsonl` file. Reading the log and collecting entries reconstructs
|
||||
//! the full Worker state — no separate snapshots or checkpoints needed.
|
||||
//! A [`Session`](SessionId) is a fork-tree of [`Segment`](SegmentId)s
|
||||
//! belonging to the same logical conversation. Each Segment is recorded
|
||||
//! as a sequence of [`LogEntry`] values, one per line in a `.jsonl`
|
||||
//! file. Reading a segment log and collecting entries reconstructs the
|
||||
//! Worker state at that segment — no separate snapshots or checkpoints
|
||||
//! needed. Compaction and fork operations mint a fresh Segment within
|
||||
//! the same Session.
|
||||
//!
|
||||
//! This crate provides free functions for persistence operations.
|
||||
//! The caller (typically Pod) holds the Worker directly and calls these
|
||||
//! functions after state-mutating operations.
|
||||
//!
|
||||
//! Debug-mode [`TraceEntry`] records capture raw stream events in a separate
|
||||
//! `.trace.jsonl` file, independent of the session log.
|
||||
//! `.trace.jsonl` file, independent of the segment log.
|
||||
//!
|
||||
//! # Quick start
|
||||
//!
|
||||
//! ```ignore
|
||||
//! use session_store::{create_session, restore, save_delta, FsStore, SessionStartState};
|
||||
//! use session_store::{create_segment, restore, save_delta, FsStore, SegmentStartState};
|
||||
//!
|
||||
//! let store = FsStore::new("./sessions").await?;
|
||||
//! let (session_id, head_hash) = create_session(&store, SessionStartState {
|
||||
//! let store = FsStore::new("./sessions")?;
|
||||
//! let (session_id, segment_id) = create_segment(&store, SegmentStartState {
|
||||
//! system_prompt: None,
|
||||
//! config: &config,
|
||||
//! history: &[],
|
||||
//! }).await?;
|
||||
//! })?;
|
||||
//! ```
|
||||
|
||||
pub mod event_trace;
|
||||
pub mod fs_store;
|
||||
pub mod logged_item;
|
||||
pub mod session;
|
||||
pub mod session_log;
|
||||
pub mod pod_metadata;
|
||||
pub mod segment;
|
||||
pub mod segment_log;
|
||||
pub mod store;
|
||||
pub mod system_item;
|
||||
|
||||
|
|
@ -39,24 +44,39 @@ pub use fs_store::FsStore;
|
|||
pub use llm_worker::UsageRecord;
|
||||
pub use llm_worker::llm_client::types::{ContentPart, Item, Role};
|
||||
pub use logged_item::{LoggedContentPart, LoggedItem, LoggedRole, from_logged, to_logged};
|
||||
pub use session::{
|
||||
SessionStartState, append_entry, append_entry_with_hash, append_system_item,
|
||||
classify_history_item, create_compacted_session, create_session, create_session_with_id,
|
||||
ensure_head_or_fork, fork, fork_at, restore, save_config_changed, save_delta, save_extension,
|
||||
pub use pod_metadata::{
|
||||
PodActiveSegmentRef, PodMetadata, PodMetadataStore, PodSpawnedChild, PodSpawnedScopeRule,
|
||||
};
|
||||
pub use segment::{
|
||||
SegmentStartState, append_entry, append_system_item, classify_history_item,
|
||||
create_compacted_segment, create_segment, create_segment_with_ids, ensure_head_or_fork, fork,
|
||||
fork_at, restore, restore_by_segment, save_config_changed, save_delta, save_extension,
|
||||
save_pod_scope, save_run_completed, save_run_errored, save_turn_end, save_usage,
|
||||
save_user_input,
|
||||
};
|
||||
pub use session_log::{
|
||||
EntryHash, HashedEntry, LogEntry, POD_SCOPE_EXTENSION_DOMAIN, PodScopeSnapshot, RestoredState,
|
||||
SessionOrigin, build_chain, collect_state, compute_hash,
|
||||
pub use segment_log::{
|
||||
LogEntry, POD_SCOPE_EXTENSION_DOMAIN, PodScopeSnapshot, RestoredState, SegmentOrigin,
|
||||
collect_state,
|
||||
};
|
||||
pub use system_item::{SystemItem, render_pod_event};
|
||||
pub use store::{Store, StoreError};
|
||||
pub use system_item::{SystemItem, render_pod_event};
|
||||
|
||||
/// Session identifier. UUID v7 (time-ordered, lexicographically sortable).
|
||||
/// Session identifier — the fork-tree root. UUID v7 (time-ordered).
|
||||
///
|
||||
/// All Segments belonging to the same Session share this ID. Compaction
|
||||
/// and fork operations create a new Segment within the same Session, so
|
||||
/// `WHERE session_id = ?` retrieves the full lineage.
|
||||
pub type SessionId = uuid::Uuid;
|
||||
|
||||
/// Segment identifier. UUID v7 (time-ordered, lexicographically sortable).
|
||||
pub type SegmentId = uuid::Uuid;
|
||||
|
||||
/// Generate a new session ID.
|
||||
pub fn new_session_id() -> SessionId {
|
||||
uuid::Uuid::now_v7()
|
||||
}
|
||||
|
||||
/// Generate a new segment ID.
|
||||
pub fn new_segment_id() -> SegmentId {
|
||||
uuid::Uuid::now_v7()
|
||||
}
|
||||
|
|
|
|||
109
crates/session-store/src/pod_metadata.rs
Normal file
109
crates/session-store/src/pod_metadata.rs
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
//! Pod metadata persistence API.
|
||||
//!
|
||||
//! Pod metadata is a lightweight name-keyed pointer to the Session/Segment
|
||||
//! currently active for a Pod. Conversation content remains in the segment log;
|
||||
//! this metadata only records references needed by Pod-name resume/attach flows.
|
||||
|
||||
use crate::store::StoreError;
|
||||
use crate::{SegmentId, SessionId};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// Active Session/Segment pointer for a Pod.
|
||||
///
|
||||
/// `segment_id` is optional so callers can persist a reserved Session before
|
||||
/// the first Segment ID is known. Once a segment exists, callers should rewrite
|
||||
/// the metadata with `Some(segment_id)`.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct PodActiveSegmentRef {
|
||||
pub session_id: SessionId,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub segment_id: Option<SegmentId>,
|
||||
}
|
||||
|
||||
impl PodActiveSegmentRef {
|
||||
/// Create a reference whose active Segment is not known yet.
|
||||
pub fn pending_segment(session_id: SessionId) -> Self {
|
||||
Self {
|
||||
session_id,
|
||||
segment_id: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Create a fully resolved active Session/Segment reference.
|
||||
pub fn active_segment(session_id: SessionId, segment_id: SegmentId) -> Self {
|
||||
Self {
|
||||
session_id,
|
||||
segment_id: Some(segment_id),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One delegated scope rule for a spawned child, kept local to
|
||||
/// `session-store` so the persistence crate does not depend on manifest
|
||||
/// scope types.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct PodSpawnedScopeRule {
|
||||
pub target: PathBuf,
|
||||
pub permission: String,
|
||||
pub recursive: bool,
|
||||
}
|
||||
|
||||
/// One child Pod spawned by this Pod and persisted with the spawner's
|
||||
/// name-keyed Pod state.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct PodSpawnedChild {
|
||||
pub pod_name: String,
|
||||
pub socket_path: PathBuf,
|
||||
pub scope_delegated: Vec<PodSpawnedScopeRule>,
|
||||
pub callback_address: PathBuf,
|
||||
}
|
||||
|
||||
/// Persistent metadata for a Pod name.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct PodMetadata {
|
||||
pub pod_name: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub active: Option<PodActiveSegmentRef>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub spawned_children: Vec<PodSpawnedChild>,
|
||||
}
|
||||
|
||||
impl PodMetadata {
|
||||
/// Create Pod metadata for `pod_name`.
|
||||
pub fn new(pod_name: impl Into<String>, active: Option<PodActiveSegmentRef>) -> Self {
|
||||
Self {
|
||||
pod_name: pod_name.into(),
|
||||
active,
|
||||
spawned_children: Vec::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Sync persistence backend for Pod metadata.
|
||||
///
|
||||
/// The key is the Pod name. Missing state is not an error: `read_by_name`
|
||||
/// returns `Ok(None)` for Pods that have never persisted metadata or whose
|
||||
/// metadata was deleted.
|
||||
pub trait PodMetadataStore: Send + Sync {
|
||||
/// Create or replace metadata for its `pod_name` key.
|
||||
fn write(&self, metadata: &PodMetadata) -> Result<(), StoreError>;
|
||||
|
||||
/// Read metadata by Pod name. Returns `None` when no metadata exists.
|
||||
fn read_by_name(&self, pod_name: &str) -> Result<Option<PodMetadata>, StoreError>;
|
||||
|
||||
/// Delete metadata by Pod name. Missing metadata is a successful no-op.
|
||||
fn delete_by_name(&self, pod_name: &str) -> Result<(), StoreError>;
|
||||
}
|
||||
|
||||
pub(crate) fn validate_pod_name(pod_name: &str) -> Result<(), StoreError> {
|
||||
if pod_name.is_empty()
|
||||
|| pod_name == "."
|
||||
|| pod_name == ".."
|
||||
|| pod_name.contains('/')
|
||||
|| pod_name.contains('\0')
|
||||
{
|
||||
return Err(StoreError::InvalidPodName(pod_name.to_string()));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
515
crates/session-store/src/segment.rs
Normal file
515
crates/session-store/src/segment.rs
Normal file
|
|
@ -0,0 +1,515 @@
|
|||
//! Free functions for segment persistence operations.
|
||||
//!
|
||||
//! These functions record and restore segment state without owning a Worker.
|
||||
//! The caller (typically Pod) holds the Worker directly and calls these
|
||||
//! functions after state-mutating operations.
|
||||
|
||||
use crate::logged_item::{LoggedItem, to_logged};
|
||||
use crate::segment_log::{self, LogEntry, PodScopeSnapshot, SegmentOrigin};
|
||||
use crate::store::{Store, StoreError};
|
||||
use crate::system_item::SystemItem;
|
||||
use crate::{SegmentId, SessionId};
|
||||
use llm_worker::WorkerResult;
|
||||
use llm_worker::llm_client::RequestConfig;
|
||||
use llm_worker::llm_client::types::Item;
|
||||
use protocol::Segment;
|
||||
|
||||
/// State snapshot for creating a SegmentStart entry.
|
||||
pub struct SegmentStartState<'a> {
|
||||
pub system_prompt: Option<&'a str>,
|
||||
pub config: &'a RequestConfig,
|
||||
pub history: &'a [Item],
|
||||
}
|
||||
|
||||
/// Create a new session + initial segment, writing the initial
|
||||
/// `SegmentStart` entry. Returns the freshly minted `(session_id, segment_id)`.
|
||||
pub fn create_segment(
|
||||
store: &impl Store,
|
||||
state: SegmentStartState<'_>,
|
||||
) -> Result<(SessionId, SegmentId), StoreError> {
|
||||
let session_id = crate::new_session_id();
|
||||
let segment_id = crate::new_segment_id();
|
||||
create_segment_with_ids(store, session_id, segment_id, state)?;
|
||||
Ok((session_id, segment_id))
|
||||
}
|
||||
|
||||
/// Write a fresh `SegmentStart` entry using pre-generated IDs.
|
||||
///
|
||||
/// Used by callers that need to reserve `(session_id, segment_id)`
|
||||
/// synchronously but defer the initial log append (e.g. Pod, which
|
||||
/// resolves a templated system prompt only at first turn).
|
||||
pub fn create_segment_with_ids(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
state: SegmentStartState<'_>,
|
||||
) -> Result<(), StoreError> {
|
||||
let entry = LogEntry::SegmentStart {
|
||||
ts: segment_log::now_millis(),
|
||||
session_id,
|
||||
system_prompt: state.system_prompt.map(String::from),
|
||||
config: state.config.clone(),
|
||||
history: to_logged(state.history),
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
};
|
||||
store.append(session_id, segment_id, &entry)
|
||||
}
|
||||
|
||||
/// Create a compacted segment from an existing one. Inherits the source's
|
||||
/// `session_id` so the compacted lineage stays within the same Session.
|
||||
///
|
||||
/// Records `compacted_from` provenance linking back to the source segment
|
||||
/// at the turn boundary captured by `source_turn_count` (the most recent
|
||||
/// completed turn in the source).
|
||||
pub fn create_compacted_segment(
|
||||
store: &impl Store,
|
||||
state: SegmentStartState<'_>,
|
||||
source_session_id: SessionId,
|
||||
source_segment_id: SegmentId,
|
||||
source_turn_count: usize,
|
||||
) -> Result<SegmentId, StoreError> {
|
||||
let segment_id = crate::new_segment_id();
|
||||
let entry = LogEntry::SegmentStart {
|
||||
ts: segment_log::now_millis(),
|
||||
session_id: source_session_id,
|
||||
system_prompt: state.system_prompt.map(String::from),
|
||||
config: state.config.clone(),
|
||||
history: to_logged(state.history),
|
||||
forked_from: None,
|
||||
compacted_from: Some(SegmentOrigin {
|
||||
segment_id: source_segment_id,
|
||||
at_turn_index: source_turn_count,
|
||||
}),
|
||||
};
|
||||
store.append(source_session_id, segment_id, &entry)?;
|
||||
Ok(segment_id)
|
||||
}
|
||||
|
||||
/// Restore segment state from a stored log.
|
||||
///
|
||||
/// Returns the reconstructed state. The caller is responsible for
|
||||
/// applying it to a Worker.
|
||||
pub fn restore(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<crate::segment_log::RestoredState, StoreError> {
|
||||
let entries = store.read_all(session_id, segment_id)?;
|
||||
Ok(segment_log::collect_state(&entries))
|
||||
}
|
||||
|
||||
/// Restore segment state when only the segment ID is known. Uses
|
||||
/// [`Store::lookup_session_of`] to resolve the parent Session.
|
||||
///
|
||||
/// Shim for legacy entry points (`pod-cli --session <UUID>` etc.) that
|
||||
/// receive a Segment ID without a Session ID.
|
||||
pub fn restore_by_segment(
|
||||
store: &impl Store,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<crate::segment_log::RestoredState, StoreError> {
|
||||
let session_id = store
|
||||
.lookup_session_of(segment_id)?
|
||||
.ok_or(StoreError::NotFound(segment_id))?;
|
||||
restore(store, session_id, segment_id)
|
||||
}
|
||||
|
||||
/// Live auto-fork on concurrent-writer detection.
|
||||
///
|
||||
/// Checks whether the store's on-disk entry count still matches the
|
||||
/// writer's own append tally. If they match, the writer still owns the
|
||||
/// segment tail and nothing happens. If the store has grown behind the
|
||||
/// writer's back, another process appended to the same segment — so we
|
||||
/// branch into a fresh segment within the same Session.
|
||||
///
|
||||
/// # Marker form
|
||||
///
|
||||
/// Detection is by **tail entry-count comparison**, not by writing a
|
||||
/// terminal marker into the source segment. The source segment is left
|
||||
/// completely immutable — identical to the past-fork ([`fork_at`])
|
||||
/// invariant. The fork relationship is instead recorded forward on the
|
||||
/// *new* segment's `SegmentStart.forked_from`, so the lineage is still
|
||||
/// reconstructable from the logs alone (read each segment's
|
||||
/// `SegmentStart`; follow `forked_from` / `compacted_from` backward).
|
||||
/// Listing a parent's children is a cheap `list_segments(session_id)`
|
||||
/// scan filtered on `forked_from.segment_id`.
|
||||
///
|
||||
/// `at_turn_index` is the writer's current `turn_count`: the fork seeds
|
||||
/// the new segment with the writer's in-memory history (which reflects
|
||||
/// state up to that turn), so that is the divergence point relative to
|
||||
/// the now-diverged source segment.
|
||||
///
|
||||
/// Updates `segment_id` and `entries_written` in place when a fork occurs.
|
||||
pub fn ensure_head_or_fork(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: &mut SegmentId,
|
||||
entries_written: &mut usize,
|
||||
at_turn_index: usize,
|
||||
state: SegmentStartState<'_>,
|
||||
) -> Result<(), StoreError> {
|
||||
let store_count = store.read_entry_count(session_id, *segment_id)?;
|
||||
if store_count == *entries_written {
|
||||
return Ok(());
|
||||
}
|
||||
let source_segment_id = *segment_id;
|
||||
let fork_id = crate::new_segment_id();
|
||||
let entry = LogEntry::SegmentStart {
|
||||
ts: segment_log::now_millis(),
|
||||
session_id,
|
||||
system_prompt: state.system_prompt.map(String::from),
|
||||
config: state.config.clone(),
|
||||
history: to_logged(state.history),
|
||||
forked_from: Some(SegmentOrigin {
|
||||
segment_id: source_segment_id,
|
||||
at_turn_index,
|
||||
}),
|
||||
compacted_from: None,
|
||||
};
|
||||
store.create_segment(session_id, fork_id, &[entry])?;
|
||||
*segment_id = fork_id;
|
||||
*entries_written = 1;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Log a `UserInput` entry from the original typed `Vec<Segment>`.
|
||||
///
|
||||
/// Submit-time entry. Pod calls this at the head of a `Run` turn before
|
||||
/// the worker pushes its flattened user message into history; replay
|
||||
/// derives the worker `Item::user_message` from these segments via
|
||||
/// [`Segment::flatten_to_text`].
|
||||
pub fn save_user_input(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
segments: Vec<Segment>,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
LogEntry::UserInput {
|
||||
ts: segment_log::now_millis(),
|
||||
segments,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log the history delta — new items added since the previous snapshot.
|
||||
///
|
||||
/// Classifies items into AssistantItem / ToolResult entries automatically
|
||||
/// (one entry per item). User messages are skipped
|
||||
/// because they are persisted upfront via [`save_user_input`] at submit
|
||||
/// time; the worker pushes a flattened copy into its history that
|
||||
/// arrives here in `new_items` and would otherwise produce a duplicate
|
||||
/// `UserInput` entry.
|
||||
pub fn save_delta(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
new_items: &[Item],
|
||||
) -> Result<(), StoreError> {
|
||||
if new_items.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let ts = segment_log::now_millis();
|
||||
for item in new_items {
|
||||
if item.is_user_message() {
|
||||
// Already persisted by save_user_input at submit time.
|
||||
continue;
|
||||
}
|
||||
let entry = classify_history_item(item, ts);
|
||||
append_entry(store, session_id, segment_id, entry)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Map one history item to its singular `LogEntry` form. Used by the
|
||||
/// fallback `save_delta` path and the controller's worker-callback
|
||||
/// classifier so write classification lives in one place.
|
||||
pub fn classify_history_item(item: &Item, ts: u64) -> LogEntry {
|
||||
if item.is_tool_result() {
|
||||
LogEntry::ToolResult {
|
||||
ts,
|
||||
item: LoggedItem::from(item),
|
||||
}
|
||||
} else if item.is_assistant_message() || item.is_tool_call() || item.is_reasoning() {
|
||||
LogEntry::AssistantItem {
|
||||
ts,
|
||||
item: LoggedItem::from(item),
|
||||
}
|
||||
} else {
|
||||
// Defensive: anything else (future Item kinds) routes through
|
||||
// AssistantItem rather than getting silently dropped.
|
||||
LogEntry::AssistantItem {
|
||||
ts,
|
||||
item: LoggedItem::from(item),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Append a single typed system item as `LogEntry::SystemItem`. Helper
|
||||
/// for the Pod-side interceptor commit path; mirrors the per-item
|
||||
/// commit shape used for assistant / tool result entries.
|
||||
pub fn append_system_item(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
item: SystemItem,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
LogEntry::SystemItem {
|
||||
ts: segment_log::now_millis(),
|
||||
item,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log a TurnEnd entry.
|
||||
pub fn save_turn_end(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
turn_count: usize,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
LogEntry::TurnEnd {
|
||||
ts: segment_log::now_millis(),
|
||||
turn_count,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log a `RunCompleted` entry — `run()` / `resume()` returned `Ok(WorkerResult)`.
|
||||
pub fn save_run_completed(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
result: WorkerResult,
|
||||
interrupted: bool,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
LogEntry::RunCompleted {
|
||||
ts: segment_log::now_millis(),
|
||||
interrupted,
|
||||
result,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log a `RunErrored` entry — `run()` / `resume()` returned `Err(WorkerError)`.
|
||||
///
|
||||
/// `WorkerError` is not `Serialize`, so the caller passes a lossy
|
||||
/// `to_string()` rendering as `message`.
|
||||
pub fn save_run_errored(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
message: String,
|
||||
interrupted: bool,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
LogEntry::RunErrored {
|
||||
ts: segment_log::now_millis(),
|
||||
interrupted,
|
||||
message,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log an `LlmUsage` entry — 1 LLM リクエスト分の Usage スナップショット。
|
||||
///
|
||||
/// `history_len` は送信時の `history.len()`。`input_total_tokens` は
|
||||
/// その prefix をプロバイダが実測した占有量(プロンプト全長)で、
|
||||
/// プロバイダ別の正規化(Anthropic では `input + cache_read + cache_creation`)を
|
||||
/// 済ませた値を渡す。
|
||||
pub fn save_usage(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
history_len: usize,
|
||||
input_total_tokens: u64,
|
||||
cache_read_tokens: u64,
|
||||
cache_write_tokens: u64,
|
||||
output_tokens: u64,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
LogEntry::LlmUsage {
|
||||
ts: segment_log::now_millis(),
|
||||
history_len,
|
||||
input_total_tokens,
|
||||
cache_read_tokens,
|
||||
cache_write_tokens,
|
||||
output_tokens,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log an `Extension` entry — domain-tagged opaque payload.
|
||||
///
|
||||
/// session-store treats `payload` as an unstructured `serde_json::Value`.
|
||||
/// Each domain is responsible for serializing into and folding out of it.
|
||||
/// Use `RestoredState.extensions` to read entries back at restore time.
|
||||
pub fn save_extension(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
domain: impl Into<String>,
|
||||
payload: serde_json::Value,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
LogEntry::Extension {
|
||||
ts: segment_log::now_millis(),
|
||||
domain: domain.into(),
|
||||
payload,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log the Pod's latest runtime scope snapshot.
|
||||
pub fn save_pod_scope(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
snapshot: &PodScopeSnapshot,
|
||||
) -> Result<(), StoreError> {
|
||||
let payload = serde_json::to_value(snapshot)?;
|
||||
save_extension(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
segment_log::POD_SCOPE_EXTENSION_DOMAIN,
|
||||
payload,
|
||||
)
|
||||
}
|
||||
|
||||
/// Log a `ConfigChanged` entry.
|
||||
pub fn save_config_changed(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
config: &RequestConfig,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
segment_id,
|
||||
LogEntry::ConfigChanged {
|
||||
ts: segment_log::now_millis(),
|
||||
config: config.clone(),
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Fork the current state into a brand-new Session (no parent lineage).
|
||||
///
|
||||
/// Use this for "start a fresh conversation from this state" — the
|
||||
/// returned segment does not share `session_id` with any prior segment.
|
||||
/// In-Session forks (live auto-fork / past-turn fork) go through
|
||||
/// [`fork_at`] or [`ensure_head_or_fork`] instead.
|
||||
pub fn fork(
|
||||
store: &impl Store,
|
||||
state: SegmentStartState<'_>,
|
||||
) -> Result<(SessionId, SegmentId), StoreError> {
|
||||
let session_id = crate::new_session_id();
|
||||
let fork_id = crate::new_segment_id();
|
||||
let entry = LogEntry::SegmentStart {
|
||||
ts: segment_log::now_millis(),
|
||||
session_id,
|
||||
system_prompt: state.system_prompt.map(String::from),
|
||||
config: state.config.clone(),
|
||||
history: to_logged(state.history),
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
};
|
||||
store.create_segment(session_id, fork_id, &[entry])?;
|
||||
Ok((session_id, fork_id))
|
||||
}
|
||||
|
||||
/// Fork from a turn boundary in a stored segment log, keeping the new
|
||||
/// segment in the same Session as `source_id`.
|
||||
///
|
||||
/// `at_turn_index` is the `turn_count` of the most recent completed
|
||||
/// `TurnEnd` in the source segment that the fork should branch from.
|
||||
/// Replay collects state up to and including that `TurnEnd`; entries
|
||||
/// after it are not carried into the new segment.
|
||||
///
|
||||
/// # Invariant: the source segment is never mutated
|
||||
///
|
||||
/// Past-fork only reads the source and seeds a brand-new segment. It
|
||||
/// writes no marker back into the source — exactly like live auto-fork
|
||||
/// ([`ensure_head_or_fork`]). This keeps nested past-forks simple: a
|
||||
/// fork of a fork just reads its own source and branches again, with no
|
||||
/// marker-position bookkeeping to reconcile across the chain.
|
||||
pub fn fork_at(
|
||||
store: &impl Store,
|
||||
source_session_id: SessionId,
|
||||
source_id: SegmentId,
|
||||
at_turn_index: usize,
|
||||
) -> Result<SegmentId, StoreError> {
|
||||
let entries = store.read_all(source_session_id, source_id)?;
|
||||
let cut = if at_turn_index == 0 {
|
||||
// Branch directly after the SegmentStart (or whatever opens the
|
||||
// segment), before any turn completes.
|
||||
entries
|
||||
.iter()
|
||||
.position(|e| !matches!(e, LogEntry::SegmentStart { .. }))
|
||||
.unwrap_or(entries.len())
|
||||
} else {
|
||||
entries
|
||||
.iter()
|
||||
.position(|e| matches!(e, LogEntry::TurnEnd { turn_count, .. } if *turn_count == at_turn_index))
|
||||
.map(|i| i + 1)
|
||||
.unwrap_or(entries.len())
|
||||
};
|
||||
let state = segment_log::collect_state(&entries[..cut]);
|
||||
|
||||
let fork_id = crate::new_segment_id();
|
||||
let entry = LogEntry::SegmentStart {
|
||||
ts: segment_log::now_millis(),
|
||||
session_id: source_session_id,
|
||||
system_prompt: state.system_prompt,
|
||||
config: state.config,
|
||||
history: to_logged(&state.history),
|
||||
forked_from: Some(SegmentOrigin {
|
||||
segment_id: source_id,
|
||||
at_turn_index,
|
||||
}),
|
||||
compacted_from: None,
|
||||
};
|
||||
store.create_segment(source_session_id, fork_id, &[entry])?;
|
||||
Ok(fork_id)
|
||||
}
|
||||
|
||||
/// Append a single `LogEntry`.
|
||||
///
|
||||
/// Lower-level dual of the `save_*` convenience wrappers in this module.
|
||||
/// Use when the caller already builds the typed entry itself (e.g. when
|
||||
/// it needs the same value for an in-memory mirror + broadcast).
|
||||
pub fn append_entry(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entry: LogEntry,
|
||||
) -> Result<(), StoreError> {
|
||||
store.append(session_id, segment_id, &entry)
|
||||
}
|
||||
|
|
@ -1,98 +1,28 @@
|
|||
//! Session log types for append-only JSONL persistence.
|
||||
//! Segment log types for append-only JSONL persistence.
|
||||
//!
|
||||
//! Each [`LogEntry`] represents a single state transition in a session,
|
||||
//! serialized as one line in a `.jsonl` file. Reading all entries and
|
||||
//! collecting them via [`collect_state`] reconstructs the full [`Worker`] state.
|
||||
//! Each [`LogEntry`] represents a single state transition within one
|
||||
//! segment, serialized as one line in a `.jsonl` file. Reading all
|
||||
//! entries and collecting them via [`collect_state`] reconstructs the
|
||||
//! full [`Worker`] state at that segment.
|
||||
//!
|
||||
//! Entries are chained via [`EntryHash`]: each [`HashedEntry`] records the hash
|
||||
//! of the previous entry, forming a tamper-evident append-only chain. This
|
||||
//! enables safe fork detection when multiple writers share a session.
|
||||
//! The on-disk format is one `LogEntry` per line — entries are positionally
|
||||
//! ordered. Fork lineage references between segments use turn-number indices
|
||||
//! (`SegmentOrigin.at_turn_index`) rather than per-entry hashes.
|
||||
|
||||
use llm_worker::llm_client::types::{Item, RequestConfig};
|
||||
use llm_worker::{UsageRecord, WorkerResult};
|
||||
use protocol::{InvokeKind, ScopeRule, Segment};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
use crate::logged_item::LoggedItem;
|
||||
use crate::system_item::SystemItem;
|
||||
|
||||
/// SHA-256 hash identifying a specific log entry in the chain.
|
||||
///
|
||||
/// Computed as `sha256(prev_hash_bytes || canonical_json(entry))`.
|
||||
/// Displayed and serialized as a lowercase hex string.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
||||
pub struct EntryHash([u8; 32]);
|
||||
|
||||
impl EntryHash {
|
||||
pub fn as_bytes(&self) -> &[u8; 32] {
|
||||
&self.0
|
||||
}
|
||||
|
||||
pub fn to_hex(&self) -> String {
|
||||
hex::encode(self.0)
|
||||
}
|
||||
|
||||
pub fn from_hex(s: &str) -> Result<Self, hex::FromHexError> {
|
||||
let mut buf = [0u8; 32];
|
||||
hex::decode_to_slice(s, &mut buf)?;
|
||||
Ok(Self(buf))
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for EntryHash {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str(&self.to_hex())
|
||||
}
|
||||
}
|
||||
|
||||
impl Serialize for EntryHash {
|
||||
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
|
||||
serializer.serialize_str(&self.to_hex())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'de> Deserialize<'de> for EntryHash {
|
||||
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
|
||||
let s = String::deserialize(deserializer)?;
|
||||
Self::from_hex(&s).map_err(serde::de::Error::custom)
|
||||
}
|
||||
}
|
||||
|
||||
/// Compute the hash for a log entry given its predecessor's hash.
|
||||
pub fn compute_hash(prev: Option<&EntryHash>, entry: &LogEntry) -> EntryHash {
|
||||
let mut hasher = Sha256::new();
|
||||
|
||||
// Feed prev_hash bytes (32 zero bytes if None).
|
||||
match prev {
|
||||
Some(h) => hasher.update(h.as_bytes()),
|
||||
None => hasher.update([0u8; 32]),
|
||||
}
|
||||
|
||||
// Canonical JSON of the entry.
|
||||
let json = serde_json::to_string(entry).expect("LogEntry serialization cannot fail");
|
||||
hasher.update(json.as_bytes());
|
||||
|
||||
EntryHash(hasher.finalize().into())
|
||||
}
|
||||
|
||||
/// A [`LogEntry`] with hash-chain metadata.
|
||||
///
|
||||
/// This is the unit persisted to JSONL — one line per `HashedEntry`.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct HashedEntry {
|
||||
pub hash: EntryHash,
|
||||
pub prev_hash: Option<EntryHash>,
|
||||
#[serde(flatten)]
|
||||
pub entry: LogEntry,
|
||||
}
|
||||
|
||||
/// A single session log entry, serialized as one JSONL line.
|
||||
/// A single segment log entry, serialized as one JSONL line.
|
||||
///
|
||||
/// Variants correspond to specific mutation points in `Worker`:
|
||||
/// - `SessionStart` — always the first entry; captures initial state
|
||||
/// - `SegmentStart` — always the first entry; captures initial state
|
||||
/// - `Invoke` — IDLE → active marker (start of a new self-driving cycle)
|
||||
/// - `UserInput` / `AssistantItems` / `ToolResults` / `HookInjectedItems` — history appends
|
||||
/// - `UserInput` / `AssistantItem` / `ToolResult` / `SystemItem` — history appends
|
||||
/// - `TurnEnd` — AgentTurn boundary marker; carries the post-increment
|
||||
/// `turn_count`. With retry unimplemented today this fires once per
|
||||
/// `run()`/`resume()` (current callers persist a single TurnEnd at
|
||||
|
|
@ -103,19 +33,25 @@ pub struct HashedEntry {
|
|||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||
pub enum LogEntry {
|
||||
/// Session start. Always the first entry in a log.
|
||||
/// For forked sessions, `history` contains the seed state from the parent.
|
||||
SessionStart {
|
||||
/// Segment start. Always the first entry in a segment log.
|
||||
/// For forked segments, `history` contains the seed state from the parent.
|
||||
SegmentStart {
|
||||
ts: u64,
|
||||
/// Session this segment belongs to. Compaction / fork inherits
|
||||
/// the source segment's session_id; only fresh "new conversation"
|
||||
/// segments mint a new session_id.
|
||||
session_id: crate::SessionId,
|
||||
system_prompt: Option<String>,
|
||||
config: RequestConfig,
|
||||
history: Vec<LoggedItem>,
|
||||
/// Origin: forked from another session at a specific entry.
|
||||
/// Origin: forked from a sibling segment at a specific turn boundary.
|
||||
/// The referenced segment is guaranteed to share `session_id`.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
forked_from: Option<SessionOrigin>,
|
||||
/// Origin: compacted from another session at a specific entry.
|
||||
forked_from: Option<SegmentOrigin>,
|
||||
/// Origin: compacted from a sibling segment at a specific turn boundary.
|
||||
/// The referenced segment is guaranteed to share `session_id`.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
compacted_from: Option<SessionOrigin>,
|
||||
compacted_from: Option<SegmentOrigin>,
|
||||
},
|
||||
|
||||
/// IDLE → active marker. Records the start of a new self-driving
|
||||
|
|
@ -137,7 +73,7 @@ pub enum LogEntry {
|
|||
|
||||
/// User input accepted at submit time. Carries the original typed
|
||||
/// `Vec<Segment>` so clients can re-render typed atoms (paste chips,
|
||||
/// file/knowledge refs, workflow invocations) on session restore.
|
||||
/// file/knowledge refs, workflow invocations) on segment restore.
|
||||
/// Replay flattens these into a `Item::user_message` for the worker
|
||||
/// history; the worker layer never sees segments directly.
|
||||
UserInput { ts: u64, segments: Vec<Segment> },
|
||||
|
|
@ -158,23 +94,6 @@ pub enum LogEntry {
|
|||
/// dispatch on `kind` for typed rendering.
|
||||
SystemItem { ts: u64, item: SystemItem },
|
||||
|
||||
/// Legacy plural form: kept **read-only** so old session logs still
|
||||
/// open. New writes always use the singular `AssistantItem`. Items
|
||||
/// are flattened on replay.
|
||||
AssistantItems { ts: u64, items: Vec<LoggedItem> },
|
||||
|
||||
/// Legacy plural form: kept **read-only**. New writes use the
|
||||
/// singular `ToolResult`.
|
||||
ToolResults { ts: u64, items: Vec<LoggedItem> },
|
||||
|
||||
/// Legacy plural form: kept **read-only**. New writes use the
|
||||
/// singular `SystemItem`.
|
||||
SystemItems { ts: u64, items: Vec<SystemItem> },
|
||||
|
||||
/// Legacy pre-`SystemItem*` form. Deserialize-only. Items are
|
||||
/// flattened to `Item::system_message` on replay.
|
||||
HookInjectedItems { ts: u64, items: Vec<LoggedItem> },
|
||||
|
||||
/// Turn boundary. Records the turn count after increment.
|
||||
TurnEnd { ts: u64, turn_count: usize },
|
||||
|
||||
|
|
@ -235,13 +154,16 @@ pub enum LogEntry {
|
|||
},
|
||||
}
|
||||
|
||||
/// Provenance reference to a parent session.
|
||||
/// Provenance reference to a parent segment.
|
||||
///
|
||||
/// `at_turn_index` is the `turn_count` value of the most recent
|
||||
/// `TurnEnd` entry preceding the split point in the source segment.
|
||||
/// A value of `0` means the split happened before any turn completed
|
||||
/// (e.g. immediately after `SegmentStart`).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
||||
pub struct SessionOrigin {
|
||||
/// Session ID of the source session.
|
||||
pub session_id: crate::SessionId,
|
||||
/// Hash of the entry in the source session at the point of fork/compact.
|
||||
pub at_hash: EntryHash,
|
||||
pub struct SegmentOrigin {
|
||||
pub segment_id: crate::SegmentId,
|
||||
pub at_turn_index: usize,
|
||||
}
|
||||
|
||||
/// Domain used by Pod to persist its latest effective runtime scope.
|
||||
|
|
@ -257,13 +179,19 @@ pub struct PodScopeSnapshot {
|
|||
/// State collected from log entries.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RestoredState {
|
||||
/// Session the replayed segment belongs to. Sourced from the
|
||||
/// `SegmentStart` entry; `None` only if the log was empty (in which
|
||||
/// case `entries_count == 0`).
|
||||
pub session_id: Option<crate::SessionId>,
|
||||
pub system_prompt: Option<String>,
|
||||
pub config: RequestConfig,
|
||||
pub history: Vec<Item>,
|
||||
pub turn_count: usize,
|
||||
pub last_run_interrupted: bool,
|
||||
/// Hash of the last entry in the chain (None if empty).
|
||||
pub head_hash: Option<EntryHash>,
|
||||
/// Number of entries replayed. `0` means the segment log was empty.
|
||||
/// Writers track their own append count via the same counter so
|
||||
/// `ensure_head_or_fork` can compare it with the on-disk count.
|
||||
pub entries_count: usize,
|
||||
/// LLM リクエストごとの Usage スナップショット時系列。
|
||||
/// `LogEntry::LlmUsage` を replay して時系列順に積まれる。
|
||||
/// 任意位置のトークン数推定に使う。
|
||||
|
|
@ -272,42 +200,45 @@ pub struct RestoredState {
|
|||
/// session-store は domain を不透明扱いし、各ドメインが自前で fold する。
|
||||
pub extensions: Vec<(String, serde_json::Value)>,
|
||||
/// Latest runtime scope snapshot persisted by the Pod. `None` means
|
||||
/// the session predates scope persistence or the payload was corrupt.
|
||||
/// the segment predates scope persistence or the payload was corrupt.
|
||||
pub pod_scope: Option<PodScopeSnapshot>,
|
||||
/// User submissions in original typed form, in submit order.
|
||||
/// One entry per `LogEntry::UserInput`; the K-th entry corresponds to
|
||||
/// the K-th `Item::user_message` derived during replay (modulo
|
||||
/// pre-compaction history seeded via `SessionStart.history`, whose
|
||||
/// pre-compaction history seeded via `SegmentStart.history`, whose
|
||||
/// original segments are not preserved). Used by clients to re-render
|
||||
/// typed atoms (paste chips, refs) on session restore.
|
||||
/// typed atoms (paste chips, refs) on segment restore.
|
||||
pub user_segments: Vec<Vec<Segment>>,
|
||||
}
|
||||
|
||||
/// Replay a sequence of hashed entries to reconstruct worker state.
|
||||
pub fn collect_state(entries: &[HashedEntry]) -> RestoredState {
|
||||
/// Replay a sequence of log entries to reconstruct worker state.
|
||||
pub fn collect_state(entries: &[LogEntry]) -> RestoredState {
|
||||
let mut state = RestoredState {
|
||||
session_id: None,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: Vec::new(),
|
||||
turn_count: 0,
|
||||
last_run_interrupted: false,
|
||||
head_hash: None,
|
||||
entries_count: 0,
|
||||
usage_history: Vec::new(),
|
||||
extensions: Vec::new(),
|
||||
pod_scope: None,
|
||||
user_segments: Vec::new(),
|
||||
};
|
||||
|
||||
for hashed in entries {
|
||||
state.head_hash = Some(hashed.hash.clone());
|
||||
for entry in entries {
|
||||
state.entries_count += 1;
|
||||
|
||||
match &hashed.entry {
|
||||
LogEntry::SessionStart {
|
||||
match entry {
|
||||
LogEntry::SegmentStart {
|
||||
session_id,
|
||||
system_prompt,
|
||||
config,
|
||||
history,
|
||||
..
|
||||
} => {
|
||||
state.session_id = Some(*session_id);
|
||||
state.system_prompt = system_prompt.clone();
|
||||
state.config = config.clone();
|
||||
state.history = history.iter().cloned().map(Item::from).collect();
|
||||
|
|
@ -331,20 +262,6 @@ pub fn collect_state(entries: &[HashedEntry]) -> RestoredState {
|
|||
LogEntry::SystemItem { item, .. } => {
|
||||
state.history.push(item.to_history_item());
|
||||
}
|
||||
LogEntry::AssistantItems { items, .. } => {
|
||||
state.history.extend(items.iter().cloned().map(Item::from));
|
||||
}
|
||||
LogEntry::ToolResults { items, .. } => {
|
||||
state.history.extend(items.iter().cloned().map(Item::from));
|
||||
}
|
||||
LogEntry::SystemItems { items, .. } => {
|
||||
state
|
||||
.history
|
||||
.extend(items.iter().map(|si| si.to_history_item()));
|
||||
}
|
||||
LogEntry::HookInjectedItems { items, .. } => {
|
||||
state.history.extend(items.iter().cloned().map(Item::from));
|
||||
}
|
||||
LogEntry::TurnEnd { turn_count, .. } => {
|
||||
state.turn_count = *turn_count;
|
||||
}
|
||||
|
|
@ -382,7 +299,7 @@ pub fn collect_state(entries: &[HashedEntry]) -> RestoredState {
|
|||
Err(err) => {
|
||||
tracing::warn!(
|
||||
error = %err,
|
||||
"discarding malformed pod.scope snapshot from session log"
|
||||
"discarding malformed pod.scope snapshot from segment log"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -403,26 +320,6 @@ pub fn now_millis() -> u64 {
|
|||
.as_millis() as u64
|
||||
}
|
||||
|
||||
/// Build a hash chain from plain `LogEntry` values.
|
||||
///
|
||||
/// Useful for tests and for seeding new sessions from a list of entries.
|
||||
pub fn build_chain(entries: &[LogEntry]) -> Vec<HashedEntry> {
|
||||
let mut chain = Vec::with_capacity(entries.len());
|
||||
let mut prev: Option<EntryHash> = None;
|
||||
|
||||
for entry in entries {
|
||||
let hash = compute_hash(prev.as_ref(), entry);
|
||||
chain.push(HashedEntry {
|
||||
hash: hash.clone(),
|
||||
prev_hash: prev,
|
||||
entry: entry.clone(),
|
||||
});
|
||||
prev = Some(hash);
|
||||
}
|
||||
|
||||
chain
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
|
@ -432,31 +329,32 @@ mod tests {
|
|||
let state = collect_state(&[]);
|
||||
assert!(state.history.is_empty());
|
||||
assert_eq!(state.turn_count, 0);
|
||||
assert!(state.head_hash.is_none());
|
||||
assert_eq!(state.entries_count, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_session_start_sets_initial_state() {
|
||||
let entries = build_chain(&[LogEntry::SessionStart {
|
||||
fn replay_segment_start_sets_initial_state() {
|
||||
let state = collect_state(&[LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: Some("You are helpful.".into()),
|
||||
config: RequestConfig::default().with_max_tokens(1024),
|
||||
history: vec![Item::user_message("seed").into()],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
}]);
|
||||
let state = collect_state(&entries);
|
||||
assert_eq!(state.system_prompt.as_deref(), Some("You are helpful."));
|
||||
assert_eq!(state.config.max_tokens, Some(1024));
|
||||
assert_eq!(state.history.len(), 1);
|
||||
assert!(state.head_hash.is_some());
|
||||
assert_eq!(state.entries_count, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_full_turn() {
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let state = collect_state(&[
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -467,9 +365,9 @@ mod tests {
|
|||
ts: 2000,
|
||||
segments: vec![Segment::text("Hello")],
|
||||
},
|
||||
LogEntry::AssistantItems {
|
||||
LogEntry::AssistantItem {
|
||||
ts: 3000,
|
||||
items: vec![Item::assistant_message("Hi!").into()],
|
||||
item: Item::assistant_message("Hi!").into(),
|
||||
},
|
||||
LogEntry::TurnEnd {
|
||||
ts: 3100,
|
||||
|
|
@ -481,7 +379,6 @@ mod tests {
|
|||
result: WorkerResult::Finished,
|
||||
},
|
||||
]);
|
||||
let state = collect_state(&entries);
|
||||
assert_eq!(state.history.len(), 2);
|
||||
assert_eq!(state.turn_count, 1);
|
||||
assert!(!state.last_run_interrupted);
|
||||
|
|
@ -489,9 +386,10 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn replay_with_tool_calls() {
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let state = collect_state(&[
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -502,24 +400,23 @@ mod tests {
|
|||
ts: 2000,
|
||||
segments: vec![Segment::text("Check weather")],
|
||||
},
|
||||
LogEntry::AssistantItems {
|
||||
LogEntry::AssistantItem {
|
||||
ts: 3000,
|
||||
items: vec![Item::tool_call("call_1", "get_weather", r#"{"city":"Tokyo"}"#).into()],
|
||||
item: Item::tool_call("call_1", "get_weather", r#"{"city":"Tokyo"}"#).into(),
|
||||
},
|
||||
LogEntry::ToolResults {
|
||||
LogEntry::ToolResult {
|
||||
ts: 3500,
|
||||
items: vec![Item::tool_result("call_1", "Sunny, 25C").into()],
|
||||
item: Item::tool_result("call_1", "Sunny, 25C").into(),
|
||||
},
|
||||
LogEntry::AssistantItems {
|
||||
LogEntry::AssistantItem {
|
||||
ts: 4000,
|
||||
items: vec![Item::assistant_message("It's sunny in Tokyo!").into()],
|
||||
item: Item::assistant_message("It's sunny in Tokyo!").into(),
|
||||
},
|
||||
LogEntry::TurnEnd {
|
||||
ts: 4100,
|
||||
turn_count: 1,
|
||||
},
|
||||
]);
|
||||
let state = collect_state(&entries);
|
||||
assert_eq!(state.history.len(), 4);
|
||||
assert!(state.history[1].is_tool_call());
|
||||
assert!(state.history[2].is_tool_result());
|
||||
|
|
@ -527,9 +424,10 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn replay_config_changed() {
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let state = collect_state(&[
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -541,52 +439,15 @@ mod tests {
|
|||
config: RequestConfig::default().with_temperature(0.5),
|
||||
},
|
||||
]);
|
||||
let state = collect_state(&entries);
|
||||
assert_eq!(state.config.temperature, Some(0.5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_chain_is_deterministic() {
|
||||
let raw = vec![
|
||||
LogEntry::SessionStart {
|
||||
ts: 1000,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
},
|
||||
LogEntry::UserInput {
|
||||
ts: 2000,
|
||||
segments: vec![Segment::text("Hello")],
|
||||
},
|
||||
];
|
||||
let chain_a = build_chain(&raw);
|
||||
let chain_b = build_chain(&raw);
|
||||
assert_eq!(chain_a[0].hash, chain_b[0].hash);
|
||||
assert_eq!(chain_a[1].hash, chain_b[1].hash);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn different_content_produces_different_hash() {
|
||||
let entry_a = LogEntry::UserInput {
|
||||
ts: 1000,
|
||||
segments: vec![Segment::text("Hello")],
|
||||
};
|
||||
let entry_b = LogEntry::UserInput {
|
||||
ts: 1000,
|
||||
segments: vec![Segment::text("World")],
|
||||
};
|
||||
let hash_a = compute_hash(None, &entry_a);
|
||||
let hash_b = compute_hash(None, &entry_b);
|
||||
assert_ne!(hash_a, hash_b);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_llm_usage_appends_to_usage_history() {
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let state = collect_state(&[
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -605,9 +466,9 @@ mod tests {
|
|||
cache_write_tokens: 0,
|
||||
output_tokens: 10,
|
||||
},
|
||||
LogEntry::AssistantItems {
|
||||
LogEntry::AssistantItem {
|
||||
ts: 2200,
|
||||
items: vec![Item::assistant_message("yo").into()],
|
||||
item: Item::assistant_message("yo").into(),
|
||||
},
|
||||
LogEntry::LlmUsage {
|
||||
ts: 3100,
|
||||
|
|
@ -618,7 +479,6 @@ mod tests {
|
|||
output_tokens: 5,
|
||||
},
|
||||
]);
|
||||
let state = collect_state(&entries);
|
||||
// history は LlmUsage で変化しない
|
||||
assert_eq!(state.history.len(), 2);
|
||||
// usage_history は時系列順
|
||||
|
|
@ -631,10 +491,10 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn replay_without_llm_usage_keeps_usage_history_empty() {
|
||||
// 既存ログ互換: LlmUsage entry が無くても collect_state は壊れない
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let state = collect_state(&[
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -646,7 +506,6 @@ mod tests {
|
|||
segments: vec![Segment::text("hi")],
|
||||
},
|
||||
]);
|
||||
let state = collect_state(&entries);
|
||||
assert!(state.usage_history.is_empty());
|
||||
}
|
||||
|
||||
|
|
@ -704,9 +563,10 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn replay_invoke_marker_does_not_mutate_state() {
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let state = collect_state(&[
|
||||
LogEntry::SegmentStart {
|
||||
ts: 0,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -730,16 +590,16 @@ mod tests {
|
|||
trigger: InvokeKind::Notify,
|
||||
},
|
||||
]);
|
||||
let state = collect_state(&entries);
|
||||
assert_eq!(state.history.len(), 1);
|
||||
assert_eq!(state.turn_count, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_extension_collects_domain_payload_pairs() {
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let state = collect_state(&[
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -762,7 +622,6 @@ mod tests {
|
|||
payload: serde_json::json!({ "x": 1 }),
|
||||
},
|
||||
]);
|
||||
let state = collect_state(&entries);
|
||||
// 順序保持で全件積まれる。fold は呼び出し側の責務。
|
||||
assert_eq!(state.extensions.len(), 3);
|
||||
assert_eq!(state.extensions[0].0, "memory.extract");
|
||||
|
|
@ -794,22 +653,6 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hash_hex_round_trip() {
|
||||
let entry = LogEntry::SessionStart {
|
||||
ts: 1000,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
};
|
||||
let hash = compute_hash(None, &entry);
|
||||
let hex = hash.to_hex();
|
||||
let parsed = EntryHash::from_hex(&hex).unwrap();
|
||||
assert_eq!(hash, parsed);
|
||||
}
|
||||
|
||||
/// Mixed segments survive a JSON round-trip through `LogEntry::UserInput`,
|
||||
/// and `collect_state` derives `Item::user_message` from the flattened
|
||||
/// text while preserving the original segments separately. This covers
|
||||
|
|
@ -834,12 +677,13 @@ mod tests {
|
|||
ts: 4242,
|
||||
segments: segments.clone(),
|
||||
};
|
||||
// Hash + JSON round-trip preserves the variant byte-for-byte.
|
||||
// JSON round-trip preserves the variant byte-for-byte.
|
||||
let json = serde_json::to_string(&entry).unwrap();
|
||||
let parsed: LogEntry = serde_json::from_str(&json).unwrap();
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let state = collect_state(&[
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -848,7 +692,6 @@ mod tests {
|
|||
},
|
||||
parsed,
|
||||
]);
|
||||
let state = collect_state(&entries);
|
||||
// Worker history gets a flattened user_message item.
|
||||
assert_eq!(state.history.len(), 1);
|
||||
match &state.history[0] {
|
||||
|
|
@ -1,482 +0,0 @@
|
|||
//! Free functions for session persistence operations.
|
||||
//!
|
||||
//! These functions record and restore session state without owning a Worker.
|
||||
//! The caller (typically Pod) holds the Worker directly and calls these
|
||||
//! functions after state-mutating operations.
|
||||
|
||||
use crate::SessionId;
|
||||
use crate::logged_item::{LoggedItem, to_logged};
|
||||
use crate::session_log::{self, EntryHash, HashedEntry, LogEntry, PodScopeSnapshot, SessionOrigin};
|
||||
use crate::store::{Store, StoreError};
|
||||
use crate::system_item::SystemItem;
|
||||
use llm_worker::WorkerResult;
|
||||
use llm_worker::llm_client::RequestConfig;
|
||||
use llm_worker::llm_client::types::Item;
|
||||
use protocol::Segment;
|
||||
|
||||
/// State snapshot for creating a SessionStart entry.
|
||||
pub struct SessionStartState<'a> {
|
||||
pub system_prompt: Option<&'a str>,
|
||||
pub config: &'a RequestConfig,
|
||||
pub history: &'a [Item],
|
||||
}
|
||||
|
||||
/// Create a new session, writing the initial `SessionStart` entry.
|
||||
///
|
||||
/// Returns the new session ID and head hash.
|
||||
pub fn create_session(
|
||||
store: &impl Store,
|
||||
state: SessionStartState<'_>,
|
||||
) -> Result<(SessionId, EntryHash), StoreError> {
|
||||
let session_id = crate::new_session_id();
|
||||
let hash = create_session_with_id(store, session_id, state)?;
|
||||
Ok((session_id, hash))
|
||||
}
|
||||
|
||||
/// Write a fresh `SessionStart` entry using a pre-generated session ID.
|
||||
///
|
||||
/// Used by callers that need to reserve a session ID synchronously but
|
||||
/// defer the initial log append (e.g. Pod, which resolves a templated
|
||||
/// system prompt only at first turn). Returns the resulting head hash.
|
||||
pub fn create_session_with_id(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
state: SessionStartState<'_>,
|
||||
) -> Result<EntryHash, StoreError> {
|
||||
let entry = LogEntry::SessionStart {
|
||||
ts: session_log::now_millis(),
|
||||
system_prompt: state.system_prompt.map(String::from),
|
||||
config: state.config.clone(),
|
||||
history: to_logged(state.history),
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
};
|
||||
let hash = session_log::compute_hash(None, &entry);
|
||||
let hashed_entry = HashedEntry {
|
||||
hash: hash.clone(),
|
||||
prev_hash: None,
|
||||
entry,
|
||||
};
|
||||
store.append(session_id, &hashed_entry)?;
|
||||
Ok(hash)
|
||||
}
|
||||
|
||||
/// Create a compacted session from an existing one.
|
||||
///
|
||||
/// Records `compacted_from` provenance linking back to the source session.
|
||||
/// Returns the new session ID and head hash.
|
||||
pub fn create_compacted_session(
|
||||
store: &impl Store,
|
||||
state: SessionStartState<'_>,
|
||||
source_session_id: SessionId,
|
||||
source_head_hash: EntryHash,
|
||||
) -> Result<(SessionId, EntryHash), StoreError> {
|
||||
let session_id = crate::new_session_id();
|
||||
let entry = LogEntry::SessionStart {
|
||||
ts: session_log::now_millis(),
|
||||
system_prompt: state.system_prompt.map(String::from),
|
||||
config: state.config.clone(),
|
||||
history: to_logged(state.history),
|
||||
forked_from: None,
|
||||
compacted_from: Some(SessionOrigin {
|
||||
session_id: source_session_id,
|
||||
at_hash: source_head_hash,
|
||||
}),
|
||||
};
|
||||
let hash = session_log::compute_hash(None, &entry);
|
||||
let hashed_entry = HashedEntry {
|
||||
hash: hash.clone(),
|
||||
prev_hash: None,
|
||||
entry,
|
||||
};
|
||||
store.append(session_id, &hashed_entry)?;
|
||||
Ok((session_id, hash))
|
||||
}
|
||||
|
||||
/// Restore session state from a stored log.
|
||||
///
|
||||
/// Returns the reconstructed state. The caller is responsible for
|
||||
/// applying it to a Worker.
|
||||
pub fn restore(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
) -> Result<crate::session_log::RestoredState, StoreError> {
|
||||
let entries = store.read_all(session_id)?;
|
||||
Ok(session_log::collect_state(&entries))
|
||||
}
|
||||
|
||||
/// Check if the store's head still matches the expected head hash.
|
||||
/// If not, auto-fork into a new session.
|
||||
///
|
||||
/// Updates `session_id` and `head_hash` in place when a fork occurs.
|
||||
pub fn ensure_head_or_fork(
|
||||
store: &impl Store,
|
||||
session_id: &mut SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
state: SessionStartState<'_>,
|
||||
) -> Result<(), StoreError> {
|
||||
let store_head = store.read_head_hash(*session_id)?;
|
||||
if store_head == *head_hash {
|
||||
return Ok(());
|
||||
}
|
||||
let fork_id = crate::new_session_id();
|
||||
let entry = LogEntry::SessionStart {
|
||||
ts: session_log::now_millis(),
|
||||
system_prompt: state.system_prompt.map(String::from),
|
||||
config: state.config.clone(),
|
||||
history: to_logged(state.history),
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
};
|
||||
let hash = session_log::compute_hash(None, &entry);
|
||||
let hashed_entry = HashedEntry {
|
||||
hash: hash.clone(),
|
||||
prev_hash: None,
|
||||
entry,
|
||||
};
|
||||
store.create_session(fork_id, &[hashed_entry])?;
|
||||
*session_id = fork_id;
|
||||
*head_hash = Some(hash);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Log a `UserInput` entry from the original typed `Vec<Segment>`.
|
||||
///
|
||||
/// Submit-time entry. Pod calls this at the head of a `Run` turn before
|
||||
/// the worker pushes its flattened user message into history; replay
|
||||
/// derives the worker `Item::user_message` from these segments via
|
||||
/// [`Segment::flatten_to_text`].
|
||||
pub fn save_user_input(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
segments: Vec<Segment>,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
LogEntry::UserInput {
|
||||
ts: session_log::now_millis(),
|
||||
segments,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log the history delta — new items added since the previous snapshot.
|
||||
///
|
||||
/// Classifies items into AssistantItem / ToolResult / HookInjectedItems
|
||||
/// entries automatically (one entry per item). User messages are skipped
|
||||
/// because they are persisted upfront via [`save_user_input`] at submit
|
||||
/// time; the worker pushes a flattened copy into its history that
|
||||
/// arrives here in `new_items` and would otherwise produce a duplicate
|
||||
/// `UserInput` entry.
|
||||
pub fn save_delta(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
new_items: &[Item],
|
||||
) -> Result<(), StoreError> {
|
||||
if new_items.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let ts = session_log::now_millis();
|
||||
for item in new_items {
|
||||
if item.is_user_message() {
|
||||
// Already persisted by save_user_input at submit time.
|
||||
continue;
|
||||
}
|
||||
let entry = classify_history_item(item, ts);
|
||||
append_entry(store, session_id, head_hash, entry)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Map one history item to its singular `LogEntry` form. Used by the
|
||||
/// fallback `save_delta` path and the controller's worker-callback
|
||||
/// classifier so write classification lives in one place.
|
||||
pub fn classify_history_item(item: &Item, ts: u64) -> LogEntry {
|
||||
if item.is_tool_result() {
|
||||
LogEntry::ToolResult {
|
||||
ts,
|
||||
item: LoggedItem::from(item),
|
||||
}
|
||||
} else if item.is_assistant_message() || item.is_tool_call() || item.is_reasoning() {
|
||||
LogEntry::AssistantItem {
|
||||
ts,
|
||||
item: LoggedItem::from(item),
|
||||
}
|
||||
} else {
|
||||
// Defensive: anything else (future Item kinds) routes through
|
||||
// AssistantItem rather than getting silently dropped.
|
||||
LogEntry::AssistantItem {
|
||||
ts,
|
||||
item: LoggedItem::from(item),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Append a single typed system item as `LogEntry::SystemItem`. Helper
|
||||
/// for the Pod-side interceptor commit path; mirrors the per-item
|
||||
/// commit shape used for assistant / tool result entries.
|
||||
pub fn append_system_item(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
item: SystemItem,
|
||||
) -> Result<EntryHash, StoreError> {
|
||||
append_entry_with_hash(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
LogEntry::SystemItem {
|
||||
ts: session_log::now_millis(),
|
||||
item,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log a TurnEnd entry.
|
||||
pub fn save_turn_end(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
turn_count: usize,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
LogEntry::TurnEnd {
|
||||
ts: session_log::now_millis(),
|
||||
turn_count,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log a `RunCompleted` entry — `run()` / `resume()` returned `Ok(WorkerResult)`.
|
||||
pub fn save_run_completed(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
result: WorkerResult,
|
||||
interrupted: bool,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
LogEntry::RunCompleted {
|
||||
ts: session_log::now_millis(),
|
||||
interrupted,
|
||||
result,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log a `RunErrored` entry — `run()` / `resume()` returned `Err(WorkerError)`.
|
||||
///
|
||||
/// `WorkerError` is not `Serialize`, so the caller passes a lossy
|
||||
/// `to_string()` rendering as `message`.
|
||||
pub fn save_run_errored(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
message: String,
|
||||
interrupted: bool,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
LogEntry::RunErrored {
|
||||
ts: session_log::now_millis(),
|
||||
interrupted,
|
||||
message,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log an `LlmUsage` entry — 1 LLM リクエスト分の Usage スナップショット。
|
||||
///
|
||||
/// `history_len` は送信時の `history.len()`。`input_total_tokens` は
|
||||
/// その prefix をプロバイダが実測した占有量(プロンプト全長)で、
|
||||
/// プロバイダ別の正規化(Anthropic では `input + cache_read + cache_creation`)を
|
||||
/// 済ませた値を渡す。
|
||||
pub fn save_usage(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
history_len: usize,
|
||||
input_total_tokens: u64,
|
||||
cache_read_tokens: u64,
|
||||
cache_write_tokens: u64,
|
||||
output_tokens: u64,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
LogEntry::LlmUsage {
|
||||
ts: session_log::now_millis(),
|
||||
history_len,
|
||||
input_total_tokens,
|
||||
cache_read_tokens,
|
||||
cache_write_tokens,
|
||||
output_tokens,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log an `Extension` entry — domain-tagged opaque payload.
|
||||
///
|
||||
/// session-store treats `payload` as an unstructured `serde_json::Value`.
|
||||
/// Each domain is responsible for serializing into and folding out of it.
|
||||
/// Use `RestoredState.extensions` to read entries back at restore time.
|
||||
pub fn save_extension(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
domain: impl Into<String>,
|
||||
payload: serde_json::Value,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
LogEntry::Extension {
|
||||
ts: session_log::now_millis(),
|
||||
domain: domain.into(),
|
||||
payload,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Log the Pod's latest runtime scope snapshot.
|
||||
pub fn save_pod_scope(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
snapshot: &PodScopeSnapshot,
|
||||
) -> Result<(), StoreError> {
|
||||
let payload = serde_json::to_value(snapshot)?;
|
||||
save_extension(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
session_log::POD_SCOPE_EXTENSION_DOMAIN,
|
||||
payload,
|
||||
)
|
||||
}
|
||||
|
||||
/// Log a `ConfigChanged` entry.
|
||||
pub fn save_config_changed(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
config: &RequestConfig,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
LogEntry::ConfigChanged {
|
||||
ts: session_log::now_millis(),
|
||||
config: config.clone(),
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Fork the current state into a new session.
|
||||
pub fn fork(store: &impl Store, state: SessionStartState<'_>) -> Result<SessionId, StoreError> {
|
||||
let fork_id = crate::new_session_id();
|
||||
let entry = LogEntry::SessionStart {
|
||||
ts: session_log::now_millis(),
|
||||
system_prompt: state.system_prompt.map(String::from),
|
||||
config: state.config.clone(),
|
||||
history: to_logged(state.history),
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
};
|
||||
let hash = session_log::compute_hash(None, &entry);
|
||||
let hashed_entry = HashedEntry {
|
||||
hash,
|
||||
prev_hash: None,
|
||||
entry,
|
||||
};
|
||||
store.create_session(fork_id, &[hashed_entry])?;
|
||||
Ok(fork_id)
|
||||
}
|
||||
|
||||
/// Fork from an arbitrary point in a stored session's log.
|
||||
pub fn fork_at(
|
||||
store: &impl Store,
|
||||
source_id: SessionId,
|
||||
at_hash: &EntryHash,
|
||||
) -> Result<SessionId, StoreError> {
|
||||
let entries = store.read_all(source_id)?;
|
||||
let cut = entries
|
||||
.iter()
|
||||
.position(|e| &e.hash == at_hash)
|
||||
.map(|i| i + 1)
|
||||
.unwrap_or(entries.len());
|
||||
let state = session_log::collect_state(&entries[..cut]);
|
||||
|
||||
let fork_id = crate::new_session_id();
|
||||
let entry = LogEntry::SessionStart {
|
||||
ts: session_log::now_millis(),
|
||||
system_prompt: state.system_prompt,
|
||||
config: state.config,
|
||||
history: to_logged(&state.history),
|
||||
forked_from: Some(session_log::SessionOrigin {
|
||||
session_id: source_id,
|
||||
at_hash: at_hash.clone(),
|
||||
}),
|
||||
compacted_from: None,
|
||||
};
|
||||
let hash = session_log::compute_hash(None, &entry);
|
||||
let hashed_entry = HashedEntry {
|
||||
hash,
|
||||
prev_hash: None,
|
||||
entry,
|
||||
};
|
||||
store.create_session(fork_id, &[hashed_entry])?;
|
||||
Ok(fork_id)
|
||||
}
|
||||
|
||||
/// Append a single `LogEntry`, chaining the hash and updating `head_hash`.
|
||||
///
|
||||
/// Lower-level dual of the `save_*` convenience wrappers in this module.
|
||||
/// Use when the caller already builds the typed entry itself (e.g. when
|
||||
/// it needs the same value for an in-memory mirror + broadcast).
|
||||
pub fn append_entry(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
entry: LogEntry,
|
||||
) -> Result<(), StoreError> {
|
||||
append_entry_with_hash(store, session_id, head_hash, entry)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Same as [`append_entry`] but returns the freshly computed entry hash.
|
||||
///
|
||||
/// Used by paths that need the hash for downstream broadcast or mirror
|
||||
/// updates (e.g. the Pod's `SessionLogSink`).
|
||||
pub fn append_entry_with_hash(
|
||||
store: &impl Store,
|
||||
session_id: SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
entry: LogEntry,
|
||||
) -> Result<EntryHash, StoreError> {
|
||||
let hash = session_log::compute_hash(head_hash.as_ref(), &entry);
|
||||
let hashed_entry = HashedEntry {
|
||||
hash: hash.clone(),
|
||||
prev_hash: head_hash.clone(),
|
||||
entry,
|
||||
};
|
||||
store.append(session_id, &hashed_entry)?;
|
||||
*head_hash = Some(hash.clone());
|
||||
Ok(hash)
|
||||
}
|
||||
|
|
@ -1,18 +1,19 @@
|
|||
//! Persistence backend abstraction.
|
||||
//!
|
||||
//! [`Store`] defines the sync interface for reading and writing session logs.
|
||||
//! Implementations handle the physical storage (filesystem, database, etc.).
|
||||
//! [`Store`] defines the sync interface for reading and writing segment logs
|
||||
//! within a [`Session`](crate::SessionId). Implementations handle the
|
||||
//! physical storage (filesystem, database, etc.).
|
||||
//!
|
||||
//! Sync (rather than async) is intentional: a session log append is a single
|
||||
//! Sync (rather than async) is intentional: a segment log append is a single
|
||||
//! `< 1 KiB` line on local fs and completes well below a millisecond. Going
|
||||
//! through `tokio::fs` would force every caller — including `Worker`'s sync
|
||||
//! `on_history_append` callback — to bridge sync → async via a channel +
|
||||
//! drain task. Keeping the store sync lets the worker callback, Pod commit
|
||||
//! paths, and `PodInterceptor` all share one direct `append_entry` call.
|
||||
|
||||
use crate::SessionId;
|
||||
use crate::event_trace::TraceEntry;
|
||||
use crate::session_log::{EntryHash, HashedEntry};
|
||||
use crate::segment_log::LogEntry;
|
||||
use crate::{SegmentId, SessionId};
|
||||
|
||||
/// Errors from the persistence store.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
|
|
@ -23,38 +24,80 @@ pub enum StoreError {
|
|||
#[error("serialization error: {0}")]
|
||||
Serde(#[from] serde_json::Error),
|
||||
|
||||
#[error("session not found: {0}")]
|
||||
NotFound(SessionId),
|
||||
#[error("segment not found: {0}")]
|
||||
NotFound(SegmentId),
|
||||
|
||||
#[error("log corrupted at line {line}: {message}")]
|
||||
Corrupt { line: usize, message: String },
|
||||
|
||||
#[error("invalid pod name: {0}")]
|
||||
InvalidPodName(String),
|
||||
}
|
||||
|
||||
/// Sync persistence backend for session logs.
|
||||
/// Sync persistence backend for segment logs.
|
||||
///
|
||||
/// All methods take `&self` — implementations should use interior mutability
|
||||
/// (e.g., append-mode file handles) when needed.
|
||||
/// (e.g., append-mode file handles) when needed. Most read/write methods
|
||||
/// take `(SessionId, SegmentId)` so segments can be physically grouped
|
||||
/// per Session on disk (or per session_id in a DB).
|
||||
pub trait Store: Send + Sync {
|
||||
/// Append a single hashed entry to the session log.
|
||||
fn append(&self, id: SessionId, entry: &HashedEntry) -> Result<(), StoreError>;
|
||||
/// Append a single log entry to the segment log.
|
||||
///
|
||||
/// One line per call. The kernel orders concurrent `O_APPEND` writes
|
||||
/// for lines < `PIPE_BUF`, so user-space serialization is unnecessary.
|
||||
fn append(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entry: &LogEntry,
|
||||
) -> Result<(), StoreError>;
|
||||
|
||||
/// Read all hashed entries for a session, in order.
|
||||
fn read_all(&self, id: SessionId) -> Result<Vec<HashedEntry>, StoreError>;
|
||||
/// Read all log entries for a segment, in order.
|
||||
fn read_all(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<Vec<LogEntry>, StoreError>;
|
||||
|
||||
/// List all session IDs, most recent first.
|
||||
fn list_sessions(&self) -> Result<Vec<SessionId>, StoreError>;
|
||||
|
||||
/// Create a new session with initial entries.
|
||||
fn create_session(&self, id: SessionId, entries: &[HashedEntry]) -> Result<(), StoreError>;
|
||||
/// List segment IDs belonging to `session_id`, most recent first.
|
||||
fn list_segments(&self, session_id: SessionId) -> Result<Vec<SegmentId>, StoreError>;
|
||||
|
||||
/// Check if a session exists.
|
||||
fn exists(&self, id: SessionId) -> Result<bool, StoreError>;
|
||||
/// Look up which session a given segment belongs to. Returns `None`
|
||||
/// when the segment is not known to any session. Implementations
|
||||
/// may scan storage; intended for shim entry points that receive a
|
||||
/// segment ID without its session ID (e.g. legacy `--session <UUID>`).
|
||||
fn lookup_session_of(&self, segment_id: SegmentId) -> Result<Option<SessionId>, StoreError>;
|
||||
|
||||
/// Read the hash of the last entry in a session (the head).
|
||||
/// Create a new segment within `session_id`, with initial entries.
|
||||
fn create_segment(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entries: &[LogEntry],
|
||||
) -> Result<(), StoreError>;
|
||||
|
||||
/// Check if a segment exists.
|
||||
fn exists(&self, session_id: SessionId, segment_id: SegmentId) -> Result<bool, StoreError>;
|
||||
|
||||
/// Count entries currently stored for a segment.
|
||||
///
|
||||
/// Returns `None` if the session is empty.
|
||||
fn read_head_hash(&self, id: SessionId) -> Result<Option<EntryHash>, StoreError>;
|
||||
/// Used by `ensure_head_or_fork` to detect concurrent writers:
|
||||
/// if the on-disk count exceeds the writer's own append tally,
|
||||
/// another process has extended the log.
|
||||
fn read_entry_count(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> Result<usize, StoreError>;
|
||||
|
||||
/// Append a trace entry to the debug event trace file.
|
||||
fn append_trace(&self, id: SessionId, entry: &TraceEntry) -> Result<(), StoreError>;
|
||||
fn append_trace(
|
||||
&self,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
entry: &TraceEntry,
|
||||
) -> Result<(), StoreError>;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ use serde::{Deserialize, Serialize};
|
|||
/// path / knowledge slug / workflow slug / etc.), plus a pre-rendered
|
||||
/// `body` (where applicable) that is the exact `role:system` text the
|
||||
/// LLM actually saw at commit time. `body` is denormalised so that
|
||||
/// session log replay reconstructs worker history byte-identical to
|
||||
/// segment log replay reconstructs worker history byte-identical to
|
||||
/// what was on the wire — even when prompt overrides (e.g. custom
|
||||
/// `notify_wrapper` template) re-shape the live rendering on a later
|
||||
/// resume.
|
||||
|
|
|
|||
|
|
@ -1,18 +1,33 @@
|
|||
use llm_worker::WorkerResult;
|
||||
use llm_worker::llm_client::types::{Item, RequestConfig};
|
||||
use session_store::{
|
||||
FsStore, LogEntry, Store, TraceEntry, build_chain, collect_state, new_session_id,
|
||||
FsStore, LogEntry, PodActiveSegmentRef, PodMetadata, PodMetadataStore, Store, TraceEntry,
|
||||
collect_state, new_segment_id, new_session_id,
|
||||
};
|
||||
|
||||
fn nil_session_start(ts: u64, session_id: uuid::Uuid) -> LogEntry {
|
||||
LogEntry::SegmentStart {
|
||||
ts,
|
||||
session_id,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn round_trip_write_and_read() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let id = new_session_id();
|
||||
let sid = new_session_id();
|
||||
let segid = new_segment_id();
|
||||
|
||||
let raw = vec![
|
||||
LogEntry::SessionStart {
|
||||
let entries = vec![
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: sid,
|
||||
system_prompt: Some("You are helpful.".into()),
|
||||
config: RequestConfig::default().with_max_tokens(1024),
|
||||
history: vec![],
|
||||
|
|
@ -37,41 +52,34 @@ fn round_trip_write_and_read() {
|
|||
result: WorkerResult::Finished,
|
||||
},
|
||||
];
|
||||
let entries = build_chain(&raw);
|
||||
|
||||
// Write entries one by one
|
||||
for entry in &entries {
|
||||
store.append(id, entry).unwrap();
|
||||
store.append(sid, segid, entry).unwrap();
|
||||
}
|
||||
|
||||
// Read back
|
||||
let read_back = store.read_all(id).unwrap();
|
||||
let read_back = store.read_all(sid, segid).unwrap();
|
||||
assert_eq!(read_back.len(), entries.len());
|
||||
|
||||
// Verify hashes survived round-trip
|
||||
for (orig, read) in entries.iter().zip(read_back.iter()) {
|
||||
assert_eq!(orig.hash, read.hash);
|
||||
assert_eq!(orig.prev_hash, read.prev_hash);
|
||||
}
|
||||
|
||||
// Replay and verify state
|
||||
let state = collect_state(&read_back);
|
||||
assert_eq!(state.session_id, Some(sid));
|
||||
assert_eq!(state.system_prompt.as_deref(), Some("You are helpful."));
|
||||
assert_eq!(state.config.max_tokens, Some(1024));
|
||||
assert_eq!(state.history.len(), 2);
|
||||
assert_eq!(state.turn_count, 1);
|
||||
assert!(!state.last_run_interrupted);
|
||||
assert!(state.head_hash.is_some());
|
||||
assert_eq!(state.entries_count, entries.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn create_session_writes_all_entries() {
|
||||
fn create_segment_writes_all_entries() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let id = new_session_id();
|
||||
let sid = new_session_id();
|
||||
let segid = new_segment_id();
|
||||
|
||||
let entries = build_chain(&[LogEntry::SessionStart {
|
||||
let entries = [LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: sid,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![
|
||||
|
|
@ -80,80 +88,75 @@ fn create_session_writes_all_entries() {
|
|||
],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
}]);
|
||||
}];
|
||||
|
||||
store.create_session(id, &entries).unwrap();
|
||||
let read_back = store.read_all(id).unwrap();
|
||||
store.create_segment(sid, segid, &entries).unwrap();
|
||||
let read_back = store.read_all(sid, segid).unwrap();
|
||||
assert_eq!(read_back.len(), 1);
|
||||
|
||||
let state = collect_state(&read_back);
|
||||
assert_eq!(state.history.len(), 2);
|
||||
assert_eq!(state.session_id, Some(sid));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn list_sessions_returns_newest_first() {
|
||||
fn list_sessions_and_segments() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
|
||||
let id1 = new_session_id();
|
||||
// Small delay to ensure different UUID v7 timestamps
|
||||
let sid_a = new_session_id();
|
||||
std::thread::sleep(std::time::Duration::from_millis(2));
|
||||
let id2 = new_session_id();
|
||||
let sid_b = new_session_id();
|
||||
|
||||
let entries1 = build_chain(&[LogEntry::SessionStart {
|
||||
ts: 1000,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
}]);
|
||||
let entries2 = build_chain(&[LogEntry::SessionStart {
|
||||
ts: 1001,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
}]);
|
||||
let seg_a1 = new_segment_id();
|
||||
let seg_a2 = new_segment_id();
|
||||
let seg_b1 = new_segment_id();
|
||||
|
||||
store.append(id1, &entries1[0]).unwrap();
|
||||
store.append(id2, &entries2[0]).unwrap();
|
||||
store
|
||||
.append(sid_a, seg_a1, &nil_session_start(1, sid_a))
|
||||
.unwrap();
|
||||
store
|
||||
.append(sid_a, seg_a2, &nil_session_start(2, sid_a))
|
||||
.unwrap();
|
||||
store
|
||||
.append(sid_b, seg_b1, &nil_session_start(3, sid_b))
|
||||
.unwrap();
|
||||
|
||||
let sessions = store.list_sessions().unwrap();
|
||||
assert_eq!(sessions.len(), 2);
|
||||
assert_eq!(sessions[0], id2); // newest first
|
||||
assert_eq!(sessions[1], id1);
|
||||
assert_eq!(sessions, vec![sid_b, sid_a]); // newest first
|
||||
|
||||
let segs_a = store.list_segments(sid_a).unwrap();
|
||||
assert!(segs_a.contains(&seg_a1) && segs_a.contains(&seg_a2));
|
||||
assert_eq!(segs_a.len(), 2);
|
||||
|
||||
let segs_b = store.list_segments(sid_b).unwrap();
|
||||
assert_eq!(segs_b, vec![seg_b1]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn exists_returns_correct_state() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let id = new_session_id();
|
||||
let sid = new_session_id();
|
||||
let segid = new_segment_id();
|
||||
|
||||
assert!(!store.exists(id).unwrap());
|
||||
assert!(!store.exists(sid, segid).unwrap());
|
||||
|
||||
let entries = build_chain(&[LogEntry::SessionStart {
|
||||
ts: 1000,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
}]);
|
||||
store.append(id, &entries[0]).unwrap();
|
||||
store
|
||||
.append(sid, segid, &nil_session_start(1000, sid))
|
||||
.unwrap();
|
||||
|
||||
assert!(store.exists(id).unwrap());
|
||||
assert!(store.exists(sid, segid).unwrap());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn not_found_error_for_missing_session() {
|
||||
fn not_found_error_for_missing_segment() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let id = new_session_id();
|
||||
let sid = new_session_id();
|
||||
let segid = new_segment_id();
|
||||
|
||||
let result = store.read_all(id);
|
||||
let result = store.read_all(sid, segid);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
|
|
@ -161,20 +164,13 @@ fn not_found_error_for_missing_session() {
|
|||
fn trace_entries_in_separate_file() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let id = new_session_id();
|
||||
let sid = new_session_id();
|
||||
let segid = new_segment_id();
|
||||
|
||||
// Write a log entry
|
||||
let entries = build_chain(&[LogEntry::SessionStart {
|
||||
ts: 1000,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
}]);
|
||||
store.append(id, &entries[0]).unwrap();
|
||||
store
|
||||
.append(sid, segid, &nil_session_start(1000, sid))
|
||||
.unwrap();
|
||||
|
||||
// Write a trace entry
|
||||
let trace = TraceEntry {
|
||||
ts: 1500,
|
||||
turn: 0,
|
||||
|
|
@ -182,26 +178,31 @@ fn trace_entries_in_separate_file() {
|
|||
llm_worker::llm_client::event::PingEvent { timestamp: None },
|
||||
),
|
||||
};
|
||||
store.append_trace(id, &trace).unwrap();
|
||||
store.append_trace(sid, segid, &trace).unwrap();
|
||||
|
||||
// Log should have 1 entry, unaffected by trace
|
||||
let log = store.read_all(id).unwrap();
|
||||
let log = store.read_all(sid, segid).unwrap();
|
||||
assert_eq!(log.len(), 1);
|
||||
|
||||
// Trace file should exist separately
|
||||
let trace_path = dir.path().join(format!("{id}.trace.jsonl"));
|
||||
let trace_path = dir
|
||||
.path()
|
||||
.join(sid.to_string())
|
||||
.join(format!("{segid}.trace.jsonl"));
|
||||
assert!(trace_path.exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_head_hash_returns_last_entry_hash() {
|
||||
fn read_entry_count_matches_append_tally() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let id = new_session_id();
|
||||
let sid = new_session_id();
|
||||
let segid = new_segment_id();
|
||||
|
||||
let entries = build_chain(&[
|
||||
LogEntry::SessionStart {
|
||||
let entries = [
|
||||
LogEntry::SegmentStart {
|
||||
ts: 1000,
|
||||
session_id: sid,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
|
|
@ -212,12 +213,63 @@ fn read_head_hash_returns_last_entry_hash() {
|
|||
ts: 2000,
|
||||
segments: vec![protocol::Segment::text("Hello")],
|
||||
},
|
||||
]);
|
||||
];
|
||||
|
||||
for entry in &entries {
|
||||
store.append(id, entry).unwrap();
|
||||
store.append(sid, segid, entry).unwrap();
|
||||
}
|
||||
|
||||
let head = store.read_head_hash(id).unwrap();
|
||||
assert_eq!(head.as_ref(), Some(&entries[1].hash));
|
||||
assert_eq!(store.read_entry_count(sid, segid).unwrap(), entries.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lookup_session_of_finds_owning_session() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let sid = new_session_id();
|
||||
let segid = new_segment_id();
|
||||
|
||||
assert_eq!(store.lookup_session_of(segid).unwrap(), None);
|
||||
|
||||
store
|
||||
.append(sid, segid, &nil_session_start(1, sid))
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(store.lookup_session_of(segid).unwrap(), Some(sid));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pod_metadata_minimal_crud() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let pod_name = "worker-a";
|
||||
let sid = new_session_id();
|
||||
let segid = new_segment_id();
|
||||
|
||||
assert_eq!(store.read_by_name(pod_name).unwrap(), None);
|
||||
|
||||
let pending = PodMetadata::new(pod_name, Some(PodActiveSegmentRef::pending_segment(sid)));
|
||||
store.write(&pending).unwrap();
|
||||
assert_eq!(store.read_by_name(pod_name).unwrap(), Some(pending.clone()));
|
||||
assert!(
|
||||
dir.path()
|
||||
.join("pods")
|
||||
.join(pod_name)
|
||||
.join("metadata.json")
|
||||
.exists(),
|
||||
"Pod metadata must live under <data_dir>/pods/<pod_name>/"
|
||||
);
|
||||
|
||||
let resolved = PodMetadata::new(
|
||||
pod_name,
|
||||
Some(PodActiveSegmentRef::active_segment(sid, segid)),
|
||||
);
|
||||
store.write(&resolved).unwrap();
|
||||
assert_eq!(store.read_by_name(pod_name).unwrap(), Some(resolved));
|
||||
|
||||
store.delete_by_name(pod_name).unwrap();
|
||||
assert_eq!(store.read_by_name(pod_name).unwrap(), None);
|
||||
|
||||
// Delete is idempotent for missing metadata.
|
||||
store.delete_by_name(pod_name).unwrap();
|
||||
}
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ use llm_worker::interceptor::{Interceptor, TurnEndAction};
|
|||
use llm_worker::llm_client::event::{Event, ResponseStatus, StatusEvent};
|
||||
use llm_worker::llm_client::types::{Item, RequestConfig};
|
||||
use llm_worker::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
|
||||
use session_store::{EntryHash, FsStore, LogEntry, SessionStartState, Store, collect_state};
|
||||
use session_store::{FsStore, LogEntry, SegmentStartState, Store, collect_state};
|
||||
|
||||
// =============================================================================
|
||||
// Helpers
|
||||
|
|
@ -96,7 +96,7 @@ async fn run_and_persist(
|
|||
worker: Worker<MockLlmClient>,
|
||||
store: &FsStore,
|
||||
session_id: session_store::SessionId,
|
||||
head_hash: &mut Option<EntryHash>,
|
||||
segment_id: session_store::SegmentId,
|
||||
input: &str,
|
||||
) -> (Worker<MockLlmClient>, llm_worker::WorkerResult) {
|
||||
// Mirror Pod's run-entry contract: log the user input as segments
|
||||
|
|
@ -105,10 +105,9 @@ async fn run_and_persist(
|
|||
session_store::save_user_input(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
segment_id,
|
||||
vec![protocol::Segment::text(input)],
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
|
||||
let history_before = worker.history().len();
|
||||
|
|
@ -118,34 +117,28 @@ async fn run_and_persist(
|
|||
let worker = locked.unlock();
|
||||
|
||||
let new_items = &worker.history()[history_before..];
|
||||
session_store::save_delta(store, session_id, head_hash, new_items)
|
||||
|
||||
.unwrap();
|
||||
session_store::save_turn_end(store, session_id, head_hash, worker.turn_count())
|
||||
|
||||
.unwrap();
|
||||
session_store::save_delta(store, session_id, segment_id, new_items).unwrap();
|
||||
session_store::save_turn_end(store, session_id, segment_id, worker.turn_count()).unwrap();
|
||||
|
||||
match &result {
|
||||
Ok(r) => {
|
||||
session_store::save_run_completed(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
segment_id,
|
||||
r.clone(),
|
||||
worker.last_run_interrupted(),
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
}
|
||||
Err(e) => {
|
||||
session_store::save_run_errored(
|
||||
store,
|
||||
session_id,
|
||||
head_hash,
|
||||
segment_id,
|
||||
e.to_string(),
|
||||
worker.last_run_interrupted(),
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
|
|
@ -164,37 +157,35 @@ async fn session_run_logs_entries() {
|
|||
let client = MockLlmClient::new(simple_text_events());
|
||||
let worker = Worker::new(client);
|
||||
|
||||
let (sid, head_hash) = session_store::create_session(
|
||||
let (sid, segid) = session_store::create_segment(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
|
||||
let mut head_hash = Some(head_hash);
|
||||
let (worker, _) = run_and_persist(worker, &store, sid, &mut head_hash, "Hi").await;
|
||||
let (worker, _) = run_and_persist(worker, &store, sid, segid, "Hi").await;
|
||||
let _ = &worker;
|
||||
|
||||
let entries = store.read_all(sid).unwrap();
|
||||
let entries = store.read_all(sid, segid).unwrap();
|
||||
|
||||
// SessionStart, UserInput, AssistantItems, TurnEnd, RunCompleted (at minimum)
|
||||
// SegmentStart, UserInput, AssistantItem, TurnEnd, RunCompleted (at minimum)
|
||||
assert!(
|
||||
entries.len() >= 4,
|
||||
"expected at least 4 entries, got {}",
|
||||
entries.len()
|
||||
);
|
||||
|
||||
// First entry is SessionStart
|
||||
assert!(matches!(&entries[0].entry, LogEntry::SessionStart { .. }));
|
||||
// First entry is SegmentStart
|
||||
assert!(matches!(&entries[0], LogEntry::SegmentStart { .. }));
|
||||
|
||||
// Has a RunCompleted with Finished
|
||||
let has_finished = entries.iter().any(|e| {
|
||||
matches!(
|
||||
&e.entry,
|
||||
e,
|
||||
LogEntry::RunCompleted {
|
||||
result: llm_worker::WorkerResult::Finished,
|
||||
..
|
||||
|
|
@ -202,17 +193,6 @@ async fn session_run_logs_entries() {
|
|||
)
|
||||
});
|
||||
assert!(has_finished, "should have a Finished outcome");
|
||||
|
||||
// Verify hash chain integrity
|
||||
assert!(entries[0].prev_hash.is_none());
|
||||
for i in 1..entries.len() {
|
||||
assert_eq!(
|
||||
entries[i].prev_hash.as_ref(),
|
||||
Some(&entries[i - 1].hash),
|
||||
"hash chain broken at entry {}",
|
||||
i
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
|
@ -222,30 +202,36 @@ async fn session_restore_round_trip() {
|
|||
let mut worker = Worker::new(client);
|
||||
worker.set_system_prompt("You are helpful.");
|
||||
|
||||
let (sid, head_hash) = session_store::create_session(
|
||||
let (sid, segid) = session_store::create_segment(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
let mut head_hash = Some(head_hash);
|
||||
|
||||
let (worker, _) = run_and_persist(worker, &store, sid, &mut head_hash, "Hi").await;
|
||||
let (worker, _) = run_and_persist(worker, &store, sid, segid, "Hi").await;
|
||||
|
||||
let original_history_len = worker.history().len();
|
||||
let original_turn_count = worker.turn_count();
|
||||
|
||||
// Restore
|
||||
let state = session_store::restore(&store, sid).unwrap();
|
||||
let state = session_store::restore(&store, sid, segid).unwrap();
|
||||
|
||||
assert_eq!(state.session_id, Some(sid));
|
||||
assert_eq!(state.history.len(), original_history_len);
|
||||
assert_eq!(state.turn_count, original_turn_count);
|
||||
assert_eq!(state.system_prompt.as_deref(), Some("You are helpful."));
|
||||
assert_eq!(state.head_hash, head_hash);
|
||||
assert_eq!(
|
||||
state.entries_count,
|
||||
store.read_entry_count(sid, segid).unwrap()
|
||||
);
|
||||
|
||||
// Shim by segment ID alone.
|
||||
let by_segment = session_store::restore_by_segment(&store, segid).unwrap();
|
||||
assert_eq!(by_segment.session_id, Some(sid));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
|
@ -255,31 +241,28 @@ async fn session_run_with_tool_call() {
|
|||
let mut worker = Worker::new(client);
|
||||
worker.register_tool(weather_tool_definition());
|
||||
|
||||
let (sid, head_hash) = session_store::create_session(
|
||||
let (sid, segid) = session_store::create_segment(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
let mut head_hash = Some(head_hash);
|
||||
|
||||
let (_worker, _) =
|
||||
run_and_persist(worker, &store, sid, &mut head_hash, "What's the weather?").await;
|
||||
let (_worker, _) = run_and_persist(worker, &store, sid, segid, "What's the weather?").await;
|
||||
|
||||
let entries = store.read_all(sid).unwrap();
|
||||
let entries = store.read_all(sid, segid).unwrap();
|
||||
|
||||
let has_tool_results = entries
|
||||
.iter()
|
||||
.any(|e| matches!(&e.entry, LogEntry::ToolResult { .. }));
|
||||
.any(|e| matches!(e, LogEntry::ToolResult { .. }));
|
||||
assert!(has_tool_results, "should have ToolResult entry");
|
||||
|
||||
let has_assistant = entries
|
||||
.iter()
|
||||
.any(|e| matches!(&e.entry, LogEntry::AssistantItem { .. }));
|
||||
.any(|e| matches!(e, LogEntry::AssistantItem { .. }));
|
||||
assert!(has_assistant, "should have AssistantItem entry");
|
||||
}
|
||||
|
||||
|
|
@ -293,26 +276,24 @@ async fn session_resume_after_pause() {
|
|||
worker.register_tool(weather_tool_definition());
|
||||
worker.set_interceptor(PausePolicy);
|
||||
|
||||
let (sid, head_hash) = session_store::create_session(
|
||||
let (sid, segid) = session_store::create_segment(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
let mut head_hash = Some(head_hash);
|
||||
|
||||
let (_worker, result) = run_and_persist(worker, &store, sid, &mut head_hash, "Weather?").await;
|
||||
let (_worker, result) = run_and_persist(worker, &store, sid, segid, "Weather?").await;
|
||||
assert!(matches!(result, llm_worker::WorkerResult::Paused));
|
||||
|
||||
// Check RunCompleted is Paused
|
||||
let entries = store.read_all(sid).unwrap();
|
||||
let entries = store.read_all(sid, segid).unwrap();
|
||||
let has_paused = entries.iter().any(|e| {
|
||||
matches!(
|
||||
&e.entry,
|
||||
e,
|
||||
LogEntry::RunCompleted {
|
||||
result: llm_worker::WorkerResult::Paused,
|
||||
..
|
||||
|
|
@ -322,93 +303,95 @@ async fn session_resume_after_pause() {
|
|||
assert!(has_paused, "should have Paused outcome");
|
||||
|
||||
// Restore state and verify
|
||||
let state = session_store::restore(&store, sid).unwrap();
|
||||
let state = session_store::restore(&store, sid, segid).unwrap();
|
||||
assert!(state.last_run_interrupted);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn session_fork_preserves_state() {
|
||||
async fn session_fork_creates_new_session() {
|
||||
let (_dir, store) = make_store();
|
||||
let client = MockLlmClient::new(simple_text_events());
|
||||
let mut worker = Worker::new(client);
|
||||
worker.set_system_prompt("System prompt");
|
||||
|
||||
let (sid, head_hash) = session_store::create_session(
|
||||
let (sid, segid) = session_store::create_segment(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
let mut head_hash = Some(head_hash);
|
||||
|
||||
let (worker, _) = run_and_persist(worker, &store, sid, &mut head_hash, "Hello").await;
|
||||
let (worker, _) = run_and_persist(worker, &store, sid, segid, "Hello").await;
|
||||
|
||||
let original_history_len = worker.history().len();
|
||||
let fork_id = session_store::fork(
|
||||
let (fork_sid, fork_segid) = session_store::fork(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
assert_ne!(fork_sid, sid, "`fork` mints a fresh Session");
|
||||
|
||||
// Fork should have a SessionStart with the current history
|
||||
let fork_entries = store.read_all(fork_id).unwrap();
|
||||
// Fork should have a SegmentStart with the current history
|
||||
let fork_entries = store.read_all(fork_sid, fork_segid).unwrap();
|
||||
assert_eq!(fork_entries.len(), 1);
|
||||
assert!(matches!(
|
||||
&fork_entries[0].entry,
|
||||
LogEntry::SessionStart { .. }
|
||||
));
|
||||
assert!(matches!(&fork_entries[0], LogEntry::SegmentStart { .. }));
|
||||
|
||||
let fork_state = collect_state(&fork_entries);
|
||||
assert_eq!(fork_state.session_id, Some(fork_sid));
|
||||
assert_eq!(fork_state.history.len(), original_history_len);
|
||||
assert_eq!(fork_state.system_prompt.as_deref(), Some("System prompt"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn session_fork_at_truncates() {
|
||||
async fn session_fork_at_truncates_within_session() {
|
||||
let (_dir, store) = make_store();
|
||||
let client = MockLlmClient::new(simple_text_events());
|
||||
let worker = Worker::new(client);
|
||||
|
||||
let (sid, head_hash) = session_store::create_session(
|
||||
let (sid, segid) = session_store::create_segment(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
let mut head_hash = Some(head_hash);
|
||||
|
||||
let (_worker, _) = run_and_persist(worker, &store, sid, &mut head_hash, "Hello").await;
|
||||
let (worker, _) = run_and_persist(worker, &store, sid, segid, "Hello").await;
|
||||
|
||||
let all_entries = store.read_all(sid).unwrap();
|
||||
let all_entries = store.read_all(sid, segid).unwrap();
|
||||
assert!(all_entries.len() > 2);
|
||||
|
||||
// Fork at the hash of the 2nd entry (SessionStart + UserInput)
|
||||
let at_hash = &all_entries[1].hash;
|
||||
let fork_id = session_store::fork_at(&store, sid, at_hash).unwrap();
|
||||
// Fork at turn 1 (one completed turn). Stays in same Session.
|
||||
let fork_segid = session_store::fork_at(&store, sid, segid, worker.turn_count()).unwrap();
|
||||
|
||||
let fork_entries = store.read_all(fork_id).unwrap();
|
||||
assert_eq!(fork_entries.len(), 1); // Just the new SessionStart
|
||||
let fork_entries = store.read_all(sid, fork_segid).unwrap();
|
||||
assert_eq!(fork_entries.len(), 1); // Just the new SegmentStart
|
||||
|
||||
let fork_state = collect_state(&fork_entries);
|
||||
// Should have the state from replaying only the first 2 entries
|
||||
let original_truncated_state = collect_state(&all_entries[..2]);
|
||||
assert_eq!(
|
||||
fork_state.history.len(),
|
||||
original_truncated_state.history.len()
|
||||
);
|
||||
assert_eq!(fork_state.session_id, Some(sid), "fork_at inherits Session");
|
||||
|
||||
// History at fork point should match history right after the TurnEnd in
|
||||
// the source segment.
|
||||
let turn_end_pos = all_entries
|
||||
.iter()
|
||||
.position(|e| matches!(e, LogEntry::TurnEnd { turn_count, .. } if *turn_count == worker.turn_count()))
|
||||
.expect("source segment has the matching TurnEnd");
|
||||
let source_state_at_fork = collect_state(&all_entries[..=turn_end_pos]);
|
||||
assert_eq!(fork_state.history.len(), source_state_at_fork.history.len());
|
||||
|
||||
// list_segments should show both source and fork in the same Session.
|
||||
let segs = store.list_segments(sid).unwrap();
|
||||
assert!(segs.contains(&segid));
|
||||
assert!(segs.contains(&fork_segid));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
|
@ -417,29 +400,25 @@ async fn session_config_changed_logged() {
|
|||
let client = MockLlmClient::new(vec![]);
|
||||
let mut worker = Worker::new(client);
|
||||
|
||||
let (sid, head_hash) = session_store::create_session(
|
||||
let (sid, segid) = session_store::create_segment(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
let mut head_hash = Some(head_hash);
|
||||
|
||||
// Modify config and log it
|
||||
let new_config = RequestConfig::default().with_temperature(0.7);
|
||||
worker.set_request_config(new_config.clone());
|
||||
session_store::save_config_changed(&store, sid, &mut head_hash, &new_config)
|
||||
session_store::save_config_changed(&store, sid, segid, &new_config).unwrap();
|
||||
|
||||
.unwrap();
|
||||
|
||||
let entries = store.read_all(sid).unwrap();
|
||||
let entries = store.read_all(sid, segid).unwrap();
|
||||
let has_config_changed = entries.iter().any(|e| {
|
||||
matches!(
|
||||
&e.entry,
|
||||
e,
|
||||
LogEntry::ConfigChanged { config, .. } if config.temperature == Some(0.7)
|
||||
)
|
||||
});
|
||||
|
|
@ -450,62 +429,140 @@ async fn session_config_changed_logged() {
|
|||
async fn session_auto_forks_on_conflict() {
|
||||
let (_dir, store) = make_store();
|
||||
|
||||
// Create a session
|
||||
// Create a segment
|
||||
let client_a = MockLlmClient::new(simple_text_events());
|
||||
let worker_a = Worker::new(client_a);
|
||||
|
||||
let (original_sid, head_hash) = session_store::create_session(
|
||||
let (sid, original_segid) = session_store::create_segment(
|
||||
&store,
|
||||
SessionStartState {
|
||||
SegmentStartState {
|
||||
system_prompt: worker_a.get_system_prompt(),
|
||||
config: worker_a.request_config(),
|
||||
history: worker_a.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
let mut session_id = original_sid;
|
||||
let mut head_hash = Some(head_hash);
|
||||
let mut segment_id = original_segid;
|
||||
// Writer tracked: just the SegmentStart we wrote.
|
||||
let mut entries_written: usize = 1;
|
||||
|
||||
// Simulate another Pod writing to the same session behind our back
|
||||
// Simulate another Pod writing to the same segment behind our back.
|
||||
let extra_entry = LogEntry::UserInput {
|
||||
ts: 9999,
|
||||
segments: vec![protocol::Segment::text("Interloper")],
|
||||
};
|
||||
let current_head = store.read_head_hash(original_sid).unwrap();
|
||||
let hash = session_store::compute_hash(current_head.as_ref(), &extra_entry);
|
||||
let hashed = session_store::HashedEntry {
|
||||
hash,
|
||||
prev_hash: current_head,
|
||||
entry: extra_entry,
|
||||
};
|
||||
store.append(original_sid, &hashed).unwrap();
|
||||
store.append(sid, original_segid, &extra_entry).unwrap();
|
||||
|
||||
// Now head_hash is stale — ensure_head_or_fork should auto-fork
|
||||
// Now the on-disk count exceeds our tally — ensure_head_or_fork should auto-fork.
|
||||
session_store::ensure_head_or_fork(
|
||||
&store,
|
||||
&mut session_id,
|
||||
&mut head_hash,
|
||||
SessionStartState {
|
||||
sid,
|
||||
&mut segment_id,
|
||||
&mut entries_written,
|
||||
/* at_turn_index */ 0,
|
||||
SegmentStartState {
|
||||
system_prompt: worker_a.get_system_prompt(),
|
||||
config: worker_a.request_config(),
|
||||
history: worker_a.history(),
|
||||
},
|
||||
)
|
||||
|
||||
.unwrap();
|
||||
|
||||
// session_id should now be different
|
||||
assert_ne!(session_id, original_sid);
|
||||
// segment_id should now be different but live in the same Session.
|
||||
assert_ne!(segment_id, original_segid);
|
||||
|
||||
// The fork session should exist and have entries
|
||||
let fork_entries = store.read_all(session_id).unwrap();
|
||||
// The fork segment should exist and have entries
|
||||
let fork_entries = store.read_all(sid, segment_id).unwrap();
|
||||
assert!(!fork_entries.is_empty());
|
||||
let fork_state = collect_state(&fork_entries);
|
||||
assert_eq!(
|
||||
fork_state.session_id,
|
||||
Some(sid),
|
||||
"auto-fork inherits Session"
|
||||
);
|
||||
|
||||
// Original session should still have the interloper entry
|
||||
let original_entries = store.read_all(original_sid).unwrap();
|
||||
// The new segment records its lineage forward via forked_from; the
|
||||
// source segment is left immutable (no terminal marker written back).
|
||||
match &fork_entries[0] {
|
||||
LogEntry::SegmentStart {
|
||||
forked_from: Some(origin),
|
||||
..
|
||||
} => {
|
||||
assert_eq!(origin.segment_id, original_segid);
|
||||
assert_eq!(origin.at_turn_index, 0);
|
||||
}
|
||||
other => panic!("expected SegmentStart with forked_from, got {other:?}"),
|
||||
}
|
||||
|
||||
// Original segment should still have the interloper entry and NO
|
||||
// terminal fork marker — it is byte-for-byte unchanged.
|
||||
let original_entries = store.read_all(sid, original_segid).unwrap();
|
||||
assert_eq!(
|
||||
original_entries.len(),
|
||||
2,
|
||||
"source segment holds only SegmentStart + interloper UserInput"
|
||||
);
|
||||
let has_interloper = original_entries
|
||||
.iter()
|
||||
.any(|e| matches!(&e.entry, LogEntry::UserInput { .. }));
|
||||
.any(|e| matches!(e, LogEntry::UserInput { .. }));
|
||||
assert!(has_interloper);
|
||||
}
|
||||
|
||||
/// Nested past-fork: forking a segment that is itself a fork must not
|
||||
/// require touching any ancestor. Each `fork_at` only reads its direct
|
||||
/// source and seeds a new segment, so a chain of forks composes cleanly.
|
||||
#[tokio::test]
|
||||
async fn nested_past_fork_leaves_ancestors_immutable() {
|
||||
let (_dir, store) = make_store();
|
||||
let client = MockLlmClient::new(simple_text_events());
|
||||
let worker = Worker::new(client);
|
||||
|
||||
let (sid, root_segid) = session_store::create_segment(
|
||||
&store,
|
||||
SegmentStartState {
|
||||
system_prompt: worker.get_system_prompt(),
|
||||
config: worker.request_config(),
|
||||
history: worker.history(),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let (worker, _) = run_and_persist(worker, &store, sid, root_segid, "Hello").await;
|
||||
let root_before = store.read_all(sid, root_segid).unwrap();
|
||||
|
||||
// First past-fork at the completed turn.
|
||||
let fork1 = session_store::fork_at(&store, sid, root_segid, worker.turn_count()).unwrap();
|
||||
// Fork the fork (turn 0 = right after its SegmentStart seed).
|
||||
let fork2 = session_store::fork_at(&store, sid, fork1, 0).unwrap();
|
||||
|
||||
// All three are distinct, all in the same Session.
|
||||
assert_ne!(fork1, root_segid);
|
||||
assert_ne!(fork2, fork1);
|
||||
for seg in [root_segid, fork1, fork2] {
|
||||
assert_eq!(
|
||||
collect_state(&store.read_all(sid, seg).unwrap()).session_id,
|
||||
Some(sid)
|
||||
);
|
||||
}
|
||||
|
||||
// The root and fork1 are untouched by forking their descendants.
|
||||
assert_eq!(
|
||||
store.read_all(sid, root_segid).unwrap().len(),
|
||||
root_before.len()
|
||||
);
|
||||
let fork1_entries = store.read_all(sid, fork1).unwrap();
|
||||
assert_eq!(
|
||||
fork1_entries.len(),
|
||||
1,
|
||||
"fork1 is just its SegmentStart seed"
|
||||
);
|
||||
|
||||
// fork2's lineage points at fork1, not the root.
|
||||
match &store.read_all(sid, fork2).unwrap()[0] {
|
||||
LogEntry::SegmentStart {
|
||||
forked_from: Some(origin),
|
||||
..
|
||||
} => assert_eq!(origin.segment_id, fork1),
|
||||
other => panic!("expected SegmentStart with forked_from, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,12 +4,12 @@
|
|||
//! `llm-worker` `Tool` infrastructure. Filesystem access is mediated by
|
||||
//! two orthogonal concerns:
|
||||
//!
|
||||
//! - [`ScopedFs`] — pod-lifetime, expresses the write-block boundary for
|
||||
//! the current scope. Derived from the manifest and shareable across
|
||||
//! sessions.
|
||||
//! - [`Tracker`] — session-lifetime, enforces the "read before edit"
|
||||
//! - [`ScopedFs`] — Pod-process lifetime, expresses the write-block
|
||||
//! boundary for the current scope. Derived from the manifest; not
|
||||
//! persisted across Pod restart.
|
||||
//! - [`Tracker`] — Pod-process lifetime, enforces the "read before edit"
|
||||
//! policy via content hashes and tracks the recency of touched files.
|
||||
//! Recreated fresh per session.
|
||||
//! Recreated fresh on each Pod start (including resume).
|
||||
//!
|
||||
//! The Pod layer owns both instances and passes them to
|
||||
//! [`builtin_tools`] when registering tools on a `Worker`.
|
||||
|
|
@ -42,11 +42,11 @@ pub use tracker::Tracker;
|
|||
pub use write::write_tool;
|
||||
|
||||
/// Register all builtin tools, wiring them to a shared `ScopedFs`
|
||||
/// (pod-lifetime) and `Tracker` (session-lifetime).
|
||||
/// (Pod-process lifetime) and `Tracker` (Pod-process lifetime).
|
||||
///
|
||||
/// All returned factories share the same tracker instance so that
|
||||
/// `Read` / `Write` / `Edit` see a consistent history across tool
|
||||
/// invocations within a single session.
|
||||
/// invocations within a single Pod run.
|
||||
///
|
||||
/// `bash_output_dir` is where the Bash tool spills long outputs. The
|
||||
/// caller is responsible for adding that path to the readable scope
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
//! Session-scoped TaskStore and builtin task tools.
|
||||
//! Session-lifetime TaskStore and builtin task tools.
|
||||
//!
|
||||
//! The store is Pod/session-lifetime state shared by the four Task* tools. It
|
||||
//! is reconstructed on resume by replaying TaskCreate / TaskUpdate tool-call
|
||||
//! arguments from persisted history.
|
||||
//! The store survives compaction and Pod restart — it is reconstructed
|
||||
//! on resume by replaying TaskCreate / TaskUpdate tool-call arguments
|
||||
//! from persisted history, so its effective lifetime is the
|
||||
//! [`session_store::SessionId`] (the conversation), not the Pod process.
|
||||
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
|
|
@ -251,24 +252,25 @@ struct TaskUpdateTool {
|
|||
store: TaskStore,
|
||||
}
|
||||
|
||||
const CREATE_DESCRIPTION: &str = "Create a session-lifetime task for short-term current-work \
|
||||
tracking, not project management. Tasks are user-visible real-time status for work with a \
|
||||
concrete goal that needs multiple meaningful steps, such as implementation, debugging, \
|
||||
investigation, or structured review. Do not create tasks for simple questions, brief answers, or \
|
||||
single-step actions. Input only `subject` and `description`; `taskid` is assigned automatically \
|
||||
and initial `status` is `pending`.";
|
||||
const CREATE_DESCRIPTION: &str = "Create a session-lifetime task only when user-visible \
|
||||
progress tracking is genuinely useful: multiple active tasks must be remembered, or the work \
|
||||
will involve long edits, long-running commands, extended investigation, or interruption-prone \
|
||||
coordination. Do not create a task just because a request has several steps, and do not create \
|
||||
one for short questions, quick checks, single reviews, or one-off commands. Prefer updating an \
|
||||
existing active task over creating a duplicate. Input only `subject` and `description`; `taskid` \
|
||||
is assigned automatically and initial `status` is `pending`.";
|
||||
const LIST_DESCRIPTION: &str = "List every session-lifetime task, including completed and \
|
||||
deleted entries. Tasks are user-visible real-time status for short-term current-work tracking. \
|
||||
Takes an empty object as input.";
|
||||
const GET_DESCRIPTION: &str = "Get one session-lifetime task by `taskid`. Tasks are \
|
||||
user-visible real-time status for short-term current-work tracking. Returns an error if the task \
|
||||
does not exist.";
|
||||
const UPDATE_DESCRIPTION: &str = "Update an existing session-lifetime task as progress changes \
|
||||
between meaningful steps. Tasks are user-visible real-time status for multi-step work; keep \
|
||||
status current with `pending`, `inprogress`, `completed`, or `deleted`. Provide `taskid` and at \
|
||||
least one of `status`, `subject`, or `description`; deletion is logical (`status = deleted`). If \
|
||||
an unexpected problem blocks progress, do not force the next step: leave the task as-is, \
|
||||
summarize the problem to the user, and end the turn.";
|
||||
const UPDATE_DESCRIPTION: &str = "Update an existing session-lifetime task when meaningful \
|
||||
progress changes between substantial steps. Tasks are user-visible real-time status, so avoid \
|
||||
churn for trivial substeps. Keep status current with `pending`, `inprogress`, `completed`, or \
|
||||
`deleted`. Provide `taskid` and at least one of `status`, `subject`, or `description`; deletion is \
|
||||
logical (`status = deleted`). If an unexpected problem blocks progress, do not force the next \
|
||||
step: leave the task as-is, summarize the problem to the user, and end the turn.";
|
||||
|
||||
#[async_trait]
|
||||
impl Tool for TaskCreateTool {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
//! Session-scoped tracker for file operations performed by the builtin
|
||||
//! Pod-lifetime tracker for file operations performed by the builtin
|
||||
//! file-manipulation tools.
|
||||
//!
|
||||
//! A `Tracker` serves two orthogonal purposes:
|
||||
|
|
@ -18,11 +18,13 @@
|
|||
//!
|
||||
//! # Lifetime
|
||||
//!
|
||||
//! A `Tracker` is **session-scoped**: the Pod layer creates a fresh
|
||||
//! instance at the start of each agent session and discards it when the
|
||||
//! session ends. The `ScopedFs` write boundary, by contrast, is
|
||||
//! pod-lifetime (derived from the manifest). The two are orthogonal and
|
||||
//! the Pod wires them together when registering builtin tools.
|
||||
//! A `Tracker` is **Pod-process scoped**: the Pod layer creates a fresh
|
||||
//! instance at the start of each Pod run (including resume) and discards
|
||||
//! it when the process exits — it is not persisted, so a resumed
|
||||
//! conversation starts with an empty read/edit history. The `ScopedFs`
|
||||
//! write boundary is likewise Pod-process scoped (derived from the
|
||||
//! manifest). The two are orthogonal and the Pod wires them together
|
||||
//! when registering builtin tools.
|
||||
//!
|
||||
//! ```no_run
|
||||
//! # use std::path::PathBuf;
|
||||
|
|
|
|||
|
|
@ -22,4 +22,5 @@ pulldown-cmark = { version = "0.13.3", default-features = false }
|
|||
llm-worker.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = { workspace = true }
|
||||
tools = { workspace = true }
|
||||
|
|
|
|||
|
|
@ -57,6 +57,10 @@ pub struct App {
|
|||
/// cache reads excluded). Reset on `RunEnd`.
|
||||
pub run_upload_tokens: u64,
|
||||
pub run_output_tokens: u64,
|
||||
/// Latest session context tokens reported by the Pod. This is the raw
|
||||
/// `input_tokens` value and is independent from per-run upload totals.
|
||||
pub session_context_tokens: u64,
|
||||
pub context_window: u64,
|
||||
pub turn_index: usize,
|
||||
pub current_tool: Option<String>,
|
||||
pub input: InputBuffer,
|
||||
|
|
@ -100,6 +104,8 @@ impl App {
|
|||
run_requests: 0,
|
||||
run_upload_tokens: 0,
|
||||
run_output_tokens: 0,
|
||||
session_context_tokens: 0,
|
||||
context_window: 0,
|
||||
turn_index: 0,
|
||||
current_tool: None,
|
||||
input: InputBuffer::new(),
|
||||
|
|
@ -483,7 +489,7 @@ impl App {
|
|||
self.blocks.push(Block::UserMessage { segments });
|
||||
self.assistant_streaming = false;
|
||||
}
|
||||
Event::SessionRotated { entry } => {
|
||||
Event::SegmentRotated { entry } => {
|
||||
self.reset_for_rotation();
|
||||
self.apply_log_entry_raw(&entry);
|
||||
self.assistant_streaming = false;
|
||||
|
|
@ -502,9 +508,7 @@ impl App {
|
|||
// for `tickets/invoke-turn-llmcall-semantics.md`; events flow
|
||||
// through to subscribers but the TUI currently derives its
|
||||
// turn header from `UserMessage` / `SystemItem` arrivals.
|
||||
Event::InvokeStart { .. }
|
||||
| Event::LlmCallStart { .. }
|
||||
| Event::LlmCallEnd { .. } => {}
|
||||
Event::InvokeStart { .. } | Event::LlmCallStart { .. } | Event::LlmCallEnd { .. } => {}
|
||||
Event::TextDelta { text } => {
|
||||
self.append_assistant_text(&text);
|
||||
}
|
||||
|
|
@ -651,6 +655,7 @@ impl App {
|
|||
output_tokens,
|
||||
cache_read_input_tokens,
|
||||
} => {
|
||||
self.session_context_tokens = input_tokens.unwrap_or(0);
|
||||
// Subtract the cache-hit portion so a tool loop that
|
||||
// re-sends the same prefix on every request doesn't
|
||||
// re-count it. cache_creation stays in (it is full
|
||||
|
|
@ -685,7 +690,8 @@ impl App {
|
|||
started_at: Instant::now(),
|
||||
}));
|
||||
}
|
||||
Event::CompactDone { new_session_id } => {
|
||||
Event::CompactDone { new_segment_id } => {
|
||||
self.session_context_tokens = 0;
|
||||
if let Some(evt) = self.last_streaming_compact_mut() {
|
||||
let elapsed_secs = match evt {
|
||||
CompactEvent::Streaming { started_at } => {
|
||||
|
|
@ -694,12 +700,12 @@ impl App {
|
|||
_ => None,
|
||||
};
|
||||
*evt = CompactEvent::Done {
|
||||
new_session_id,
|
||||
new_segment_id,
|
||||
elapsed_secs,
|
||||
};
|
||||
} else {
|
||||
self.blocks.push(Block::Compact(CompactEvent::Done {
|
||||
new_session_id,
|
||||
new_segment_id,
|
||||
elapsed_secs: None,
|
||||
}));
|
||||
}
|
||||
|
|
@ -916,6 +922,8 @@ impl App {
|
|||
/// produced. Followed by `Event::Entry` updates for anything
|
||||
/// committed after the snapshot.
|
||||
fn restore_snapshot(&mut self, entries: &[serde_json::Value], greeting: protocol::Greeting) {
|
||||
self.context_window = greeting.context_window;
|
||||
self.session_context_tokens = greeting.context_tokens;
|
||||
self.turn_index = 0;
|
||||
self.blocks.clear();
|
||||
self.cache = FileCache::new();
|
||||
|
|
@ -932,7 +940,7 @@ impl App {
|
|||
}
|
||||
|
||||
/// Drop the derived view in preparation for replaying a new
|
||||
/// `SessionStart` (compaction / fork). Greeting is preserved
|
||||
/// `SegmentStart` (compaction / fork). Greeting is preserved
|
||||
/// because the Pod identity hasn't changed.
|
||||
fn reset_for_rotation(&mut self) {
|
||||
let greeting = self.blocks.iter().find_map(|b| match b {
|
||||
|
|
@ -958,7 +966,7 @@ impl App {
|
|||
return;
|
||||
};
|
||||
match entry {
|
||||
session_store::LogEntry::SessionStart { history, .. } => {
|
||||
session_store::LogEntry::SegmentStart { history, .. } => {
|
||||
for logged in history {
|
||||
let item: llm_worker::Item = logged.into();
|
||||
let item_value = serde_json::to_value(&item).expect("Item is Serialize");
|
||||
|
|
@ -984,22 +992,6 @@ impl App {
|
|||
let value = serde_json::to_value(&item).expect("SystemItem is Serialize");
|
||||
self.apply_system_item(&value);
|
||||
}
|
||||
session_store::LogEntry::AssistantItems { items, .. }
|
||||
| session_store::LogEntry::ToolResults { items, .. }
|
||||
| session_store::LogEntry::HookInjectedItems { items, .. } => {
|
||||
for logged in items {
|
||||
let item: llm_worker::Item = logged.into();
|
||||
let item_value = serde_json::to_value(&item).expect("Item is Serialize");
|
||||
self.push_history_item(&item_value);
|
||||
}
|
||||
}
|
||||
session_store::LogEntry::SystemItems { items, .. } => {
|
||||
for system_item in items {
|
||||
let value =
|
||||
serde_json::to_value(&system_item).expect("SystemItem is Serialize");
|
||||
self.apply_system_item(&value);
|
||||
}
|
||||
}
|
||||
// Non-history-bearing variants don't affect the block view.
|
||||
_ => {}
|
||||
}
|
||||
|
|
@ -1445,8 +1437,9 @@ mod completion_flow_tests {
|
|||
#[test]
|
||||
fn snapshot_renders_system_message_block_from_session_start() {
|
||||
let mut app = App::new("test".into());
|
||||
let session_start = session_store::LogEntry::SessionStart {
|
||||
let session_start = session_store::LogEntry::SegmentStart {
|
||||
ts: 1,
|
||||
session_id: uuid::Uuid::nil(),
|
||||
system_prompt: None,
|
||||
config: Default::default(),
|
||||
history: vec![session_store::LoggedItem::from(
|
||||
|
|
@ -1525,15 +1518,15 @@ mod completion_flow_tests {
|
|||
let id = uuid::Uuid::parse_str("12345678-1234-5678-1234-567812345678").unwrap();
|
||||
|
||||
app.handle_pod_event(Event::CompactStart);
|
||||
app.handle_pod_event(Event::CompactDone { new_session_id: id });
|
||||
app.handle_pod_event(Event::CompactDone { new_segment_id: id });
|
||||
|
||||
assert_eq!(compact_block_count(&app), 1);
|
||||
assert!(matches!(
|
||||
app.blocks.as_slice(),
|
||||
[Block::Compact(CompactEvent::Done {
|
||||
new_session_id,
|
||||
new_segment_id,
|
||||
elapsed_secs: Some(_),
|
||||
})] if *new_session_id == id
|
||||
})] if *new_segment_id == id
|
||||
));
|
||||
}
|
||||
|
||||
|
|
@ -1587,9 +1580,68 @@ mod completion_flow_tests {
|
|||
model: "test-model".into(),
|
||||
scope_summary: String::new(),
|
||||
tools: Vec::new(),
|
||||
context_window: 200_000,
|
||||
context_tokens: 0,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn snapshot_initializes_context_usage() {
|
||||
let mut app = App::new("test".into());
|
||||
let mut greeting = test_greeting();
|
||||
greeting.context_window = 123_000;
|
||||
greeting.context_tokens = 45_000;
|
||||
|
||||
app.handle_pod_event(Event::Snapshot {
|
||||
entries: Vec::new(),
|
||||
greeting,
|
||||
status: PodStatus::Idle,
|
||||
});
|
||||
|
||||
assert_eq!(app.context_window, 123_000);
|
||||
assert_eq!(app.session_context_tokens, 45_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn usage_updates_session_context_tokens_without_cache_discount() {
|
||||
let mut app = App::new("test".into());
|
||||
|
||||
app.handle_pod_event(Event::Usage {
|
||||
input_tokens: Some(42_000),
|
||||
output_tokens: Some(9),
|
||||
cache_read_input_tokens: Some(40_000),
|
||||
});
|
||||
|
||||
assert_eq!(app.session_context_tokens, 42_000);
|
||||
assert_eq!(app.run_upload_tokens, 2_000);
|
||||
assert_eq!(app.run_output_tokens, 9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn compact_done_resets_session_context_tokens() {
|
||||
let mut app = App::new("test".into());
|
||||
app.session_context_tokens = 42_000;
|
||||
|
||||
app.handle_pod_event(Event::CompactDone {
|
||||
new_segment_id: uuid::Uuid::nil(),
|
||||
});
|
||||
|
||||
assert_eq!(app.session_context_tokens, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn turn_start_and_run_end_do_not_reset_session_context_tokens() {
|
||||
let mut app = App::new("test".into());
|
||||
app.session_context_tokens = 42_000;
|
||||
|
||||
app.handle_pod_event(Event::TurnStart { turn: 1 });
|
||||
app.handle_pod_event(Event::RunEnd {
|
||||
result: RunResult::Finished,
|
||||
});
|
||||
|
||||
assert_eq!(app.session_context_tokens, 42_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn live_task_create_updates_task_store() {
|
||||
let mut app = App::new("test".into());
|
||||
|
|
@ -1686,33 +1738,41 @@ mod completion_flow_tests {
|
|||
arguments: r#"{"subject":"live","description":""}"#.into(),
|
||||
});
|
||||
|
||||
let assistant_items_entry = serde_json::json!({
|
||||
"kind": "assistant_items",
|
||||
let assistant_item_entries = vec![
|
||||
serde_json::json!({
|
||||
"kind": "assistant_item",
|
||||
"ts": 1,
|
||||
"items": [
|
||||
{
|
||||
"item": {
|
||||
"kind": "tool_call",
|
||||
"call_id": "c1",
|
||||
"name": "TaskCreate",
|
||||
"arguments": r#"{"subject":"a","description":"A"}"#,
|
||||
},
|
||||
{
|
||||
}),
|
||||
serde_json::json!({
|
||||
"kind": "assistant_item",
|
||||
"ts": 2,
|
||||
"item": {
|
||||
"kind": "tool_call",
|
||||
"call_id": "c2",
|
||||
"name": "TaskCreate",
|
||||
"arguments": r#"{"subject":"b","description":"B"}"#,
|
||||
},
|
||||
{
|
||||
}),
|
||||
serde_json::json!({
|
||||
"kind": "assistant_item",
|
||||
"ts": 3,
|
||||
"item": {
|
||||
"kind": "tool_call",
|
||||
"call_id": "u1",
|
||||
"name": "TaskUpdate",
|
||||
"arguments": r#"{"taskid":2,"status":"inprogress"}"#,
|
||||
},
|
||||
],
|
||||
});
|
||||
}),
|
||||
];
|
||||
app.handle_pod_event(Event::Snapshot {
|
||||
greeting: test_greeting(),
|
||||
entries: vec![assistant_items_entry],
|
||||
entries: assistant_item_entries,
|
||||
status: PodStatus::Running,
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -83,7 +83,7 @@ pub enum CompactEvent {
|
|||
Streaming { started_at: Instant },
|
||||
/// Compaction ended cleanly with `CompactDone`.
|
||||
Done {
|
||||
new_session_id: uuid::Uuid,
|
||||
new_segment_id: uuid::Uuid,
|
||||
elapsed_secs: Option<u64>,
|
||||
},
|
||||
/// Compaction ended with `CompactFailed`.
|
||||
|
|
|
|||
|
|
@ -10,9 +10,16 @@ mod task;
|
|||
mod tool;
|
||||
mod ui;
|
||||
|
||||
use std::future::Future;
|
||||
use std::io;
|
||||
use std::path::PathBuf;
|
||||
use std::process::ExitCode;
|
||||
use std::sync::{
|
||||
Arc,
|
||||
atomic::{AtomicBool, Ordering},
|
||||
};
|
||||
use std::thread;
|
||||
use std::time::Duration;
|
||||
|
||||
use crossterm::event::{
|
||||
self, DisableBracketedPaste, DisableMouseCapture, EnableBracketedPaste, EnableMouseCapture,
|
||||
|
|
@ -25,7 +32,8 @@ use crossterm::terminal::{
|
|||
use protocol::{Method, PodStatus};
|
||||
use ratatui::Terminal;
|
||||
use ratatui::backend::CrosstermBackend;
|
||||
use session_store::SessionId;
|
||||
use session_store::SegmentId;
|
||||
use tokio::sync::mpsc;
|
||||
|
||||
use client::PodClient;
|
||||
|
||||
|
|
@ -45,22 +53,27 @@ fn resolve_socket(pod_name: &str, override_path: Option<PathBuf>) -> PathBuf {
|
|||
})
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum Mode {
|
||||
Spawn,
|
||||
Attach {
|
||||
/// `tui <name>` / `tui --pod <name>`: attach to a live Pod by name if
|
||||
/// possible; otherwise launch `pod --pod <name>` so the pod process
|
||||
/// resumes from name-keyed state or creates a fresh same-name Pod.
|
||||
PodName {
|
||||
pod_name: String,
|
||||
socket_override: Option<PathBuf>,
|
||||
},
|
||||
/// `tui -r` / `tui --resume`: open the session picker first, then
|
||||
/// run the same name dialog as Spawn but in resume mode.
|
||||
/// `tui -r` / `tui --resume`: open the Pod picker, then attach to the
|
||||
/// selected live Pod or restore the selected stopped Pod by name.
|
||||
Resume,
|
||||
/// `tui --session <UUID>`: skip the picker, go straight to the
|
||||
/// resume name dialog with `id` baked in.
|
||||
ResumeWithSession(SessionId),
|
||||
ResumeWithSession(SegmentId),
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum ParseError {
|
||||
Conflict,
|
||||
Conflict(&'static str),
|
||||
InvalidSession(String),
|
||||
MissingValue(&'static str),
|
||||
}
|
||||
|
|
@ -68,7 +81,7 @@ enum ParseError {
|
|||
impl std::fmt::Display for ParseError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::Conflict => write!(f, "--resume and --session are mutually exclusive"),
|
||||
Self::Conflict(message) => write!(f, "{message}"),
|
||||
Self::InvalidSession(s) => write!(f, "invalid --session UUID: {s}"),
|
||||
Self::MissingValue(flag) => write!(f, "{flag} requires a value"),
|
||||
}
|
||||
|
|
@ -76,9 +89,18 @@ impl std::fmt::Display for ParseError {
|
|||
}
|
||||
|
||||
fn parse_args() -> Result<Mode, ParseError> {
|
||||
let args: Vec<String> = std::env::args().skip(1).collect();
|
||||
parse_args_from(std::env::args().skip(1))
|
||||
}
|
||||
|
||||
fn parse_args_from<I, S>(args: I) -> Result<Mode, ParseError>
|
||||
where
|
||||
I: IntoIterator<Item = S>,
|
||||
S: Into<String>,
|
||||
{
|
||||
let args: Vec<String> = args.into_iter().map(Into::into).collect();
|
||||
let mut resume = false;
|
||||
let mut session: Option<SessionId> = None;
|
||||
let mut session: Option<SegmentId> = None;
|
||||
let mut pod: Option<String> = None;
|
||||
let mut socket_override: Option<PathBuf> = None;
|
||||
let mut positional: Option<String> = None;
|
||||
|
||||
|
|
@ -94,11 +116,16 @@ fn parse_args() -> Result<Mode, ParseError> {
|
|||
.get(i + 1)
|
||||
.ok_or(ParseError::MissingValue("--session"))?;
|
||||
session = Some(
|
||||
raw.parse::<SessionId>()
|
||||
raw.parse::<SegmentId>()
|
||||
.map_err(|_| ParseError::InvalidSession(raw.clone()))?,
|
||||
);
|
||||
i += 2;
|
||||
}
|
||||
"--pod" => {
|
||||
let raw = args.get(i + 1).ok_or(ParseError::MissingValue("--pod"))?;
|
||||
pod = Some(raw.clone());
|
||||
i += 2;
|
||||
}
|
||||
"--socket" => {
|
||||
let raw = args
|
||||
.get(i + 1)
|
||||
|
|
@ -119,9 +146,27 @@ fn parse_args() -> Result<Mode, ParseError> {
|
|||
}
|
||||
|
||||
if resume && session.is_some() {
|
||||
return Err(ParseError::Conflict);
|
||||
return Err(ParseError::Conflict(
|
||||
"--resume and --session are mutually exclusive",
|
||||
));
|
||||
}
|
||||
if pod.is_some() && session.is_some() {
|
||||
return Err(ParseError::Conflict(
|
||||
"--pod and --session are mutually exclusive",
|
||||
));
|
||||
}
|
||||
if pod.is_some() && resume {
|
||||
return Err(ParseError::Conflict(
|
||||
"--pod and --resume are mutually exclusive",
|
||||
));
|
||||
}
|
||||
|
||||
if let Some(pod_name) = pod {
|
||||
return Ok(Mode::PodName {
|
||||
pod_name,
|
||||
socket_override,
|
||||
});
|
||||
}
|
||||
if let Some(id) = session {
|
||||
return Ok(Mode::ResumeWithSession(id));
|
||||
}
|
||||
|
|
@ -129,7 +174,7 @@ fn parse_args() -> Result<Mode, ParseError> {
|
|||
return Ok(Mode::Resume);
|
||||
}
|
||||
if let Some(pod_name) = positional {
|
||||
return Ok(Mode::Attach {
|
||||
return Ok(Mode::PodName {
|
||||
pod_name,
|
||||
socket_override,
|
||||
});
|
||||
|
|
@ -159,10 +204,10 @@ async fn main() -> ExitCode {
|
|||
|
||||
let result = match mode {
|
||||
Mode::Spawn => run_spawn(None).await,
|
||||
Mode::Attach {
|
||||
Mode::PodName {
|
||||
pod_name,
|
||||
socket_override,
|
||||
} => run_attach(pod_name, socket_override).await,
|
||||
} => run_pod_name(pod_name, socket_override).await,
|
||||
Mode::Resume => run_resume().await,
|
||||
Mode::ResumeWithSession(id) => run_spawn(Some(id)).await,
|
||||
};
|
||||
|
|
@ -186,8 +231,8 @@ async fn main() -> ExitCode {
|
|||
// SpawnError has already been painted into the inline
|
||||
// viewport's final frame, so it's already visible in the
|
||||
// user's scrollback — printing it again would be a noisy
|
||||
// duplicate. Other errors (attach-mode failures, terminal
|
||||
// setup hiccups, etc.) need surfacing here.
|
||||
// duplicate. Other errors (pod-name failures, terminal setup
|
||||
// hiccups, etc.) need surfacing here.
|
||||
if e.downcast_ref::<spawn::SpawnError>().is_none() {
|
||||
eprintln!("tui: {e}");
|
||||
}
|
||||
|
|
@ -196,27 +241,75 @@ async fn main() -> ExitCode {
|
|||
}
|
||||
}
|
||||
|
||||
async fn run_attach(
|
||||
async fn run_pod_name(
|
||||
pod_name: String,
|
||||
socket_override: Option<PathBuf>,
|
||||
) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let socket_path = resolve_socket(&pod_name, socket_override);
|
||||
let preferred_socket = resolve_socket(&pod_name, socket_override.clone());
|
||||
if let Some((_socket_path, client)) =
|
||||
connect_live_pod(&pod_name, preferred_socket, socket_override.is_none()).await
|
||||
{
|
||||
let mut terminal = enter_fullscreen()?;
|
||||
run(&mut terminal, pod_name, &socket_path).await
|
||||
let mut app = App::new(pod_name);
|
||||
app.connected = true;
|
||||
return run_loop(&mut terminal, &mut app, client).await;
|
||||
}
|
||||
|
||||
let ready = match spawn::run_pod_name(pod_name).await? {
|
||||
SpawnOutcome::Ready(r) => r,
|
||||
SpawnOutcome::Cancelled => return Ok(()),
|
||||
};
|
||||
let SpawnReady {
|
||||
pod_name,
|
||||
socket_path,
|
||||
} = ready;
|
||||
|
||||
let mut terminal = enter_fullscreen()?;
|
||||
let result = run(&mut terminal, pod_name, &socket_path).await;
|
||||
let _ = execute!(
|
||||
terminal.backend_mut(),
|
||||
DisableMouseCapture,
|
||||
LeaveAlternateScreen
|
||||
);
|
||||
result
|
||||
}
|
||||
|
||||
async fn connect_live_pod(
|
||||
pod_name: &str,
|
||||
preferred_socket: PathBuf,
|
||||
allow_registry_fallback: bool,
|
||||
) -> Option<(PathBuf, PodClient)> {
|
||||
if let Ok(client) = PodClient::connect(&preferred_socket).await {
|
||||
return Some((preferred_socket, client));
|
||||
}
|
||||
|
||||
if !allow_registry_fallback {
|
||||
return None;
|
||||
}
|
||||
let registry_socket = picker::live_socket_for_pod(pod_name)?;
|
||||
if registry_socket == preferred_socket {
|
||||
return None;
|
||||
}
|
||||
PodClient::connect(®istry_socket)
|
||||
.await
|
||||
.ok()
|
||||
.map(|client| (registry_socket, client))
|
||||
}
|
||||
|
||||
async fn run_resume() -> Result<(), Box<dyn std::error::Error>> {
|
||||
// Phase 1: pick a session in its own inline viewport, dropping the
|
||||
// viewport before the name dialog opens so each phase gets fresh
|
||||
// vertical room.
|
||||
let id = match picker::run().await? {
|
||||
PickerOutcome::Picked(id) => id,
|
||||
// Pick a Pod in its own inline viewport, dropping the viewport before
|
||||
// attaching/restoring so each phase gets fresh vertical room.
|
||||
let (pod_name, socket_override) = match picker::run().await? {
|
||||
PickerOutcome::Picked {
|
||||
pod_name,
|
||||
socket_override,
|
||||
} => (pod_name, socket_override),
|
||||
PickerOutcome::Cancelled => return Ok(()),
|
||||
};
|
||||
run_spawn(Some(id)).await
|
||||
run_pod_name(pod_name, socket_override).await
|
||||
}
|
||||
|
||||
async fn run_spawn(resume_from: Option<SessionId>) -> Result<(), Box<dyn std::error::Error>> {
|
||||
async fn run_spawn(resume_from: Option<SegmentId>) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let ready = match spawn::run(resume_from).await? {
|
||||
SpawnOutcome::Ready(r) => r,
|
||||
SpawnOutcome::Cancelled => return Ok(()),
|
||||
|
|
@ -274,11 +367,137 @@ async fn run(
|
|||
Ok(())
|
||||
}
|
||||
|
||||
type TerminalEventResult = io::Result<TermEvent>;
|
||||
|
||||
const TERMINAL_POLL_INTERVAL: Duration = Duration::from_millis(50);
|
||||
const TERMINAL_EVENT_DRAIN_LIMIT: usize = 64;
|
||||
const POD_EVENT_DRAIN_LIMIT: usize = 32;
|
||||
|
||||
struct TerminalEventReader {
|
||||
stop: Arc<AtomicBool>,
|
||||
_thread: thread::JoinHandle<()>,
|
||||
}
|
||||
|
||||
impl TerminalEventReader {
|
||||
fn spawn() -> io::Result<(Self, mpsc::UnboundedReceiver<TerminalEventResult>)> {
|
||||
let (tx, rx) = mpsc::unbounded_channel();
|
||||
let stop = Arc::new(AtomicBool::new(false));
|
||||
let thread_stop = Arc::clone(&stop);
|
||||
let thread = thread::Builder::new()
|
||||
.name("insomnia-tui-terminal-reader".to_string())
|
||||
.spawn(move || read_terminal_events(thread_stop, tx))?;
|
||||
|
||||
Ok((
|
||||
Self {
|
||||
stop,
|
||||
_thread: thread,
|
||||
},
|
||||
rx,
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TerminalEventReader {
|
||||
fn drop(&mut self) {
|
||||
self.stop.store(true, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
|
||||
fn read_terminal_events(stop: Arc<AtomicBool>, tx: mpsc::UnboundedSender<TerminalEventResult>) {
|
||||
while !stop.load(Ordering::Relaxed) {
|
||||
match event::poll(TERMINAL_POLL_INTERVAL) {
|
||||
Ok(false) => {}
|
||||
Ok(true) => {
|
||||
let event = event::read();
|
||||
let should_stop = event.is_err();
|
||||
if tx.send(event).is_err() || should_stop {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
let _ = tx.send(Err(e));
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
enum LoopInput<P> {
|
||||
Terminal(TerminalEventResult),
|
||||
Pod(Option<P>),
|
||||
}
|
||||
|
||||
async fn next_loop_input<P, F>(
|
||||
term_rx: &mut mpsc::UnboundedReceiver<TerminalEventResult>,
|
||||
connected: bool,
|
||||
pod_next: F,
|
||||
) -> LoopInput<P>
|
||||
where
|
||||
F: Future<Output = Option<P>>,
|
||||
{
|
||||
tokio::select! {
|
||||
biased;
|
||||
|
||||
term_event = term_rx.recv() => {
|
||||
LoopInput::Terminal(term_event.unwrap_or_else(|| {
|
||||
Err(io::Error::new(
|
||||
io::ErrorKind::UnexpectedEof,
|
||||
"terminal event reader stopped",
|
||||
))
|
||||
}))
|
||||
}
|
||||
event = pod_next, if connected => LoopInput::Pod(event),
|
||||
}
|
||||
}
|
||||
|
||||
async fn drain_terminal_events(
|
||||
app: &mut App,
|
||||
client: &mut PodClient,
|
||||
term_rx: &mut mpsc::UnboundedReceiver<TerminalEventResult>,
|
||||
) -> Result<bool, Box<dyn std::error::Error>> {
|
||||
let mut handled = false;
|
||||
for _ in 0..TERMINAL_EVENT_DRAIN_LIMIT {
|
||||
match term_rx.try_recv() {
|
||||
Ok(event) => {
|
||||
handled = true;
|
||||
handle_terminal_event(app, client, event?).await?;
|
||||
if app.quit {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Err(mpsc::error::TryRecvError::Empty) => break,
|
||||
Err(mpsc::error::TryRecvError::Disconnected) => {
|
||||
return Err(Box::new(io::Error::new(
|
||||
io::ErrorKind::UnexpectedEof,
|
||||
"terminal event reader stopped",
|
||||
)));
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(handled)
|
||||
}
|
||||
|
||||
fn drain_pod_events(app: &mut App, client: &mut PodClient) -> bool {
|
||||
let mut handled = false;
|
||||
for _ in 0..POD_EVENT_DRAIN_LIMIT {
|
||||
match client.try_next_event() {
|
||||
Some(ev) => {
|
||||
handled = true;
|
||||
app.handle_pod_event(ev);
|
||||
}
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
handled
|
||||
}
|
||||
|
||||
async fn run_loop(
|
||||
terminal: &mut Terminal<CrosstermBackend<io::Stdout>>,
|
||||
app: &mut App,
|
||||
mut client: PodClient,
|
||||
) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let (_terminal_reader, mut term_rx) = TerminalEventReader::spawn()?;
|
||||
|
||||
terminal.draw(|f| ui::draw(f, app))?;
|
||||
|
||||
loop {
|
||||
|
|
@ -286,56 +505,28 @@ async fn run_loop(
|
|||
break;
|
||||
}
|
||||
|
||||
// Drain any already-buffered Pod events in a bounded batch before
|
||||
// polling the terminal. This keeps status fresh without letting a
|
||||
// busy event stream starve Ctrl-C / Ctrl-X input.
|
||||
for _ in 0..32 {
|
||||
match client.try_next_event() {
|
||||
Some(ev) => app.handle_pod_event(ev),
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
|
||||
// Always give the terminal queue a non-blocking pass each frame.
|
||||
// The awaited select below only waits after this pass found nothing.
|
||||
let mut handled_term_event = false;
|
||||
while event::poll(std::time::Duration::ZERO)? {
|
||||
handled_term_event = true;
|
||||
handle_terminal_event(app, &mut client, event::read()?).await?;
|
||||
let handled_term_event = drain_terminal_events(app, &mut client, &mut term_rx).await?;
|
||||
if app.quit {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if app.quit {
|
||||
break;
|
||||
}
|
||||
if handled_term_event {
|
||||
let handled_pod_event = drain_pod_events(app, &mut client);
|
||||
if handled_term_event || handled_pod_event {
|
||||
terminal.draw(|f| ui::draw(f, app))?;
|
||||
continue;
|
||||
}
|
||||
|
||||
tokio::select! {
|
||||
term_event = tokio::task::spawn_blocking(|| {
|
||||
if event::poll(std::time::Duration::from_millis(50))? {
|
||||
event::read().map(Some)
|
||||
} else {
|
||||
Ok(None)
|
||||
match next_loop_input(&mut term_rx, app.connected, client.next_event()).await {
|
||||
LoopInput::Terminal(term_event) => {
|
||||
handle_terminal_event(app, &mut client, term_event?).await?;
|
||||
}
|
||||
}) => {
|
||||
if let Some(term_event) = term_event?? {
|
||||
handle_terminal_event(app, &mut client, term_event).await?;
|
||||
}
|
||||
}
|
||||
event = client.next_event(), if app.connected => {
|
||||
match event {
|
||||
LoopInput::Pod(event) => match event {
|
||||
Some(ev) => app.handle_pod_event(ev),
|
||||
None => {
|
||||
app.connected = false;
|
||||
app.mark_orphan_compacts_incomplete();
|
||||
app.push_error("Connection lost");
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
terminal.draw(|f| ui::draw(f, app))?;
|
||||
|
|
@ -612,3 +803,120 @@ fn handle_pause_or_quit(app: &mut App) -> Option<Method> {
|
|||
app.push_error("Press Ctrl-C again within 3 s to exit the TUI (the Pod keeps running).");
|
||||
None
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn parse_pod_name_mode() {
|
||||
match parse_args_from(["--pod", "agent", "--socket", "/tmp/agent.sock"]).unwrap() {
|
||||
Mode::PodName {
|
||||
pod_name,
|
||||
socket_override,
|
||||
} => {
|
||||
assert_eq!(pod_name, "agent");
|
||||
assert_eq!(socket_override, Some(PathBuf::from("/tmp/agent.sock")));
|
||||
}
|
||||
_ => panic!("expected PodName mode"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_positional_name_uses_pod_name_mode() {
|
||||
match parse_args_from(["agent"]).unwrap() {
|
||||
Mode::PodName {
|
||||
pod_name,
|
||||
socket_override,
|
||||
} => {
|
||||
assert_eq!(pod_name, "agent");
|
||||
assert_eq!(socket_override, None);
|
||||
}
|
||||
_ => panic!("expected PodName mode"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_rejects_pod_and_session() {
|
||||
let segment_id = session_store::new_segment_id().to_string();
|
||||
let err = parse_args_from(["--pod", "agent", "--session", &segment_id]).unwrap_err();
|
||||
assert_eq!(
|
||||
err.to_string(),
|
||||
"--pod and --session are mutually exclusive"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn terminal_event_is_selected_before_ready_pod_event() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
tx.send(Ok(TermEvent::Key(KeyEvent::new(
|
||||
KeyCode::Char('x'),
|
||||
KeyModifiers::NONE,
|
||||
))))
|
||||
.unwrap();
|
||||
|
||||
match next_loop_input(&mut rx, true, std::future::ready(Some(()))).await {
|
||||
LoopInput::Terminal(Ok(TermEvent::Key(key))) => {
|
||||
assert_eq!(key.code, KeyCode::Char('x'));
|
||||
}
|
||||
_ => panic!("ready terminal input should win over a ready Pod event"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn terminal_event_is_preserved_after_pod_event_wins() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
|
||||
match next_loop_input(&mut rx, true, std::future::ready(Some(1_u8))).await {
|
||||
LoopInput::Pod(Some(1)) => {}
|
||||
_ => panic!("expected the first ready Pod event to win before any terminal input"),
|
||||
}
|
||||
|
||||
tx.send(Ok(TermEvent::Key(KeyEvent::new(
|
||||
KeyCode::Char('y'),
|
||||
KeyModifiers::NONE,
|
||||
))))
|
||||
.unwrap();
|
||||
|
||||
match next_loop_input(&mut rx, true, std::future::ready(Some(2_u8))).await {
|
||||
LoopInput::Terminal(Ok(TermEvent::Key(key))) => {
|
||||
assert_eq!(key.code, KeyCode::Char('y'));
|
||||
}
|
||||
_ => panic!("queued terminal input should not be lost to subsequent Pod events"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn running_status_still_allows_text_editing() {
|
||||
let mut app = App::new("agent".to_string());
|
||||
app.set_pod_status(PodStatus::Running);
|
||||
|
||||
assert!(
|
||||
handle_key(
|
||||
&mut app,
|
||||
KeyEvent::new(KeyCode::Char('a'), KeyModifiers::NONE)
|
||||
)
|
||||
.is_none()
|
||||
);
|
||||
assert!(
|
||||
handle_key(
|
||||
&mut app,
|
||||
KeyEvent::new(KeyCode::Char('c'), KeyModifiers::NONE)
|
||||
)
|
||||
.is_none()
|
||||
);
|
||||
assert!(handle_key(&mut app, KeyEvent::new(KeyCode::Left, KeyModifiers::NONE)).is_none());
|
||||
assert!(
|
||||
handle_key(
|
||||
&mut app,
|
||||
KeyEvent::new(KeyCode::Char('b'), KeyModifiers::NONE)
|
||||
)
|
||||
.is_none()
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
protocol::Segment::flatten_to_text(&app.input.submit_segments()),
|
||||
"abc"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,18 +1,18 @@
|
|||
//! Inline-viewport "pick a session to restore" UX.
|
||||
//! Inline-viewport "pick a Pod to attach or restore" UX.
|
||||
//!
|
||||
//! Reads the most recent sessions from the configured store, lets the
|
||||
//! user pick one with the arrow keys, and returns the chosen
|
||||
//! `SessionId`. Closes its inline viewport before returning so the
|
||||
//! caller can open a fresh viewport for the name dialog.
|
||||
//!
|
||||
//! The picker only handles selection. Forking, pod-registry checks, and
|
||||
//! actual `pod` launch happen later in the resume flow.
|
||||
//! Reads live Pod allocations from the runtime registry and stopped Pod state
|
||||
//! from the session store's name-keyed metadata. Picking a live row attaches to
|
||||
//! its socket; picking a stopped row restores via `pod --pod <name>`.
|
||||
|
||||
use std::collections::{BTreeMap, HashMap};
|
||||
use std::fs;
|
||||
use std::io;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Duration;
|
||||
|
||||
use client::PodClient;
|
||||
use crossterm::event::{self, Event as TermEvent, KeyCode, KeyEventKind, KeyModifiers};
|
||||
use pod_registry::lookup_session;
|
||||
use pod_registry::{LockFileGuard, default_registry_path};
|
||||
use ratatui::Terminal;
|
||||
use ratatui::backend::CrosstermBackend;
|
||||
use ratatui::layout::{Constraint, Layout};
|
||||
|
|
@ -21,7 +21,7 @@ use ratatui::text::{Line, Span};
|
|||
use ratatui::widgets::Paragraph;
|
||||
use ratatui::{Frame, TerminalOptions, Viewport};
|
||||
use session_store::{
|
||||
FsStore, HashedEntry, LogEntry, LoggedContentPart, LoggedItem, SessionId, Store,
|
||||
FsStore, LogEntry, LoggedContentPart, LoggedItem, PodMetadata, SegmentId, SessionId, Store,
|
||||
};
|
||||
|
||||
const MAX_ROWS: usize = 10;
|
||||
|
|
@ -31,7 +31,7 @@ const VIEWPORT_LINES: u16 = MAX_ROWS as u16 + 4;
|
|||
pub enum PickerError {
|
||||
Io(io::Error),
|
||||
Store(session_store::StoreError),
|
||||
NoSessions,
|
||||
NoPods,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for PickerError {
|
||||
|
|
@ -39,9 +39,9 @@ impl std::fmt::Display for PickerError {
|
|||
match self {
|
||||
Self::Io(e) => write!(f, "io error: {e}"),
|
||||
Self::Store(e) => write!(f, "session store error: {e}"),
|
||||
Self::NoSessions => write!(
|
||||
Self::NoPods => write!(
|
||||
f,
|
||||
"no sessions found — start a fresh pod with `tui` and try again"
|
||||
"no pods found — start a fresh pod with `tui` and try again"
|
||||
),
|
||||
}
|
||||
}
|
||||
|
|
@ -62,41 +62,77 @@ impl From<session_store::StoreError> for PickerError {
|
|||
}
|
||||
|
||||
pub enum PickerOutcome {
|
||||
Picked(SessionId),
|
||||
/// User picked a Pod. `socket_override` is set for live rows when the
|
||||
/// runtime registry knows the exact socket path; stopped rows leave it
|
||||
/// empty so the caller restores with `pod --pod <name>`.
|
||||
Picked {
|
||||
pod_name: String,
|
||||
socket_override: Option<PathBuf>,
|
||||
},
|
||||
Cancelled,
|
||||
}
|
||||
|
||||
/// One row in the picker view. Rendered from the session log so the
|
||||
/// user can recognise their session at a glance without parsing UUIDs.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum PodRowState {
|
||||
Live,
|
||||
Stopped,
|
||||
Corrupt,
|
||||
}
|
||||
|
||||
impl PodRowState {
|
||||
fn label(self) -> &'static str {
|
||||
match self {
|
||||
Self::Live => "live",
|
||||
Self::Stopped => "stopped",
|
||||
Self::Corrupt => "corrupt",
|
||||
}
|
||||
}
|
||||
|
||||
fn style(self) -> Style {
|
||||
match self {
|
||||
Self::Live => Style::default()
|
||||
.fg(Color::Green)
|
||||
.add_modifier(Modifier::BOLD),
|
||||
Self::Stopped => Style::default().fg(Color::Yellow),
|
||||
Self::Corrupt => Style::default().fg(Color::Red).add_modifier(Modifier::BOLD),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One row in the Pod picker. The primary key is the Pod name; Session/Segment
|
||||
/// IDs are included only as debug context.
|
||||
#[derive(Debug, Clone)]
|
||||
struct Row {
|
||||
id: SessionId,
|
||||
/// Last user / assistant snippet, or a `[corrupt]` placeholder.
|
||||
preview: String,
|
||||
/// `Some(pod_name)` when a live Pod currently holds an allocation
|
||||
/// for this session in `pods.json`. Picking such a row launches
|
||||
/// `pod --session <UUID>` which will fail with `SessionConflict` —
|
||||
/// the badge warns the user up-front.
|
||||
live_pod: Option<String>,
|
||||
pod_name: String,
|
||||
state: PodRowState,
|
||||
updated_at: u64,
|
||||
active_session_id: Option<SessionId>,
|
||||
active_segment_id: Option<SegmentId>,
|
||||
preview: Option<String>,
|
||||
socket_path: Option<PathBuf>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct PodStateRecord {
|
||||
pod_name: String,
|
||||
state: Result<PodMetadata, String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub(crate) struct LivePodRecord {
|
||||
pub pod_name: String,
|
||||
pub socket_path: PathBuf,
|
||||
pub segment_id: Option<SegmentId>,
|
||||
}
|
||||
|
||||
pub async fn run() -> Result<PickerOutcome, PickerError> {
|
||||
let store = open_default_store()?;
|
||||
let ids = store.list_sessions()?;
|
||||
if ids.is_empty() {
|
||||
return Err(PickerError::NoSessions);
|
||||
}
|
||||
let mut rows: Vec<Row> = Vec::with_capacity(MAX_ROWS);
|
||||
for id in ids.into_iter().take(MAX_ROWS) {
|
||||
let preview = build_preview(&store, id);
|
||||
// Best-effort live check. A pods.json I/O hiccup downgrades
|
||||
// the row to "no badge" rather than killing the picker — the
|
||||
// user still gets to see the listing.
|
||||
let live_pod = lookup_session(id).ok().flatten().map(|info| info.pod_name);
|
||||
rows.push(Row {
|
||||
id,
|
||||
preview,
|
||||
live_pod,
|
||||
});
|
||||
let store_dir = default_store_dir()?;
|
||||
let store = FsStore::new(&store_dir)?;
|
||||
let pod_states = read_pod_state_records(&store_dir)?;
|
||||
let live_pods = read_reachable_live_pod_records().await.unwrap_or_default();
|
||||
let rows = build_rows(&store, pod_states, live_pods)?;
|
||||
if rows.is_empty() {
|
||||
return Err(PickerError::NoPods);
|
||||
}
|
||||
|
||||
let mut selected = 0usize;
|
||||
|
|
@ -106,9 +142,7 @@ pub async fn run() -> Result<PickerOutcome, PickerError> {
|
|||
match poll_event()? {
|
||||
None => continue,
|
||||
Some(Action::Up) => {
|
||||
if selected > 0 {
|
||||
selected -= 1;
|
||||
}
|
||||
selected = selected.saturating_sub(1);
|
||||
}
|
||||
Some(Action::Down) => {
|
||||
if selected + 1 < rows.len() {
|
||||
|
|
@ -117,7 +151,11 @@ pub async fn run() -> Result<PickerOutcome, PickerError> {
|
|||
}
|
||||
Some(Action::Submit) => {
|
||||
close_viewport(&mut terminal)?;
|
||||
return Ok(PickerOutcome::Picked(rows[selected].id));
|
||||
let row = &rows[selected];
|
||||
return Ok(PickerOutcome::Picked {
|
||||
pod_name: row.pod_name.clone(),
|
||||
socket_override: row.socket_path.clone(),
|
||||
});
|
||||
}
|
||||
Some(Action::Cancel) => {
|
||||
close_viewport(&mut terminal)?;
|
||||
|
|
@ -127,17 +165,9 @@ pub async fn run() -> Result<PickerOutcome, PickerError> {
|
|||
}
|
||||
}
|
||||
|
||||
/// Park the cursor at the very bottom of the picker's inline viewport
|
||||
/// and emit one newline before dropping the terminal. Without this the
|
||||
/// inline area is left with the cursor still inside it, so the next
|
||||
/// `Terminal::with_options(Inline(_))` call (the resume name dialog)
|
||||
/// computes its own area starting from inside the picker — drawing the
|
||||
/// new dialog on top of the lower picker rows.
|
||||
///
|
||||
/// Setting the cursor to `area.bottom() - 1` and writing `\r\n`
|
||||
/// scrolls the terminal up exactly one row, so the next inline
|
||||
/// viewport opens immediately below the picker rather than on top of
|
||||
/// it.
|
||||
/// Park the cursor at the very bottom of the picker's inline viewport and emit
|
||||
/// one newline before dropping the terminal. This keeps any next inline viewport
|
||||
/// from drawing over the lower picker rows.
|
||||
fn close_viewport(terminal: &mut Terminal<CrosstermBackend<io::Stdout>>) -> io::Result<()> {
|
||||
let area = terminal.get_frame().area();
|
||||
let last_row = area.bottom().saturating_sub(1);
|
||||
|
|
@ -149,38 +179,271 @@ fn close_viewport(terminal: &mut Terminal<CrosstermBackend<io::Stdout>>) -> io::
|
|||
Ok(())
|
||||
}
|
||||
|
||||
fn open_default_store() -> Result<FsStore, PickerError> {
|
||||
let dir = manifest::paths::sessions_dir().ok_or_else(|| {
|
||||
fn default_store_dir() -> Result<PathBuf, PickerError> {
|
||||
manifest::paths::sessions_dir().ok_or_else(|| {
|
||||
PickerError::Io(io::Error::new(
|
||||
io::ErrorKind::NotFound,
|
||||
"could not resolve sessions directory \
|
||||
(set INSOMNIA_HOME, INSOMNIA_DATA_DIR, or HOME)",
|
||||
))
|
||||
})?;
|
||||
Ok(FsStore::new(&dir)?)
|
||||
})
|
||||
}
|
||||
|
||||
fn build_preview(store: &FsStore, id: SessionId) -> String {
|
||||
match store.read_all(id) {
|
||||
Ok(entries) => last_message_preview(&entries).unwrap_or_else(|| "[empty]".to_string()),
|
||||
Err(_) => "[corrupt]".to_string(),
|
||||
fn read_pod_state_records(store_dir: &Path) -> Result<Vec<PodStateRecord>, PickerError> {
|
||||
let pods_dir = store_dir.join("pods");
|
||||
let mut records = Vec::new();
|
||||
if !pods_dir.exists() {
|
||||
return Ok(records);
|
||||
}
|
||||
|
||||
for entry in fs::read_dir(pods_dir)? {
|
||||
let entry = entry?;
|
||||
if !entry.file_type()?.is_dir() {
|
||||
continue;
|
||||
}
|
||||
let pod_name = entry.file_name().to_string_lossy().to_string();
|
||||
let path = entry.path().join("metadata.json");
|
||||
let state = match fs::read_to_string(&path) {
|
||||
Ok(content) => serde_json::from_str::<PodMetadata>(&content).map_err(|e| e.to_string()),
|
||||
Err(e) => Err(e.to_string()),
|
||||
};
|
||||
records.push(PodStateRecord { pod_name, state });
|
||||
}
|
||||
Ok(records)
|
||||
}
|
||||
|
||||
fn read_live_pod_records() -> Result<Vec<LivePodRecord>, io::Error> {
|
||||
let path = default_registry_path()?;
|
||||
let guard = LockFileGuard::open(&path)?;
|
||||
Ok(guard
|
||||
.data()
|
||||
.allocations
|
||||
.iter()
|
||||
.map(|allocation| LivePodRecord {
|
||||
pod_name: allocation.pod_name.clone(),
|
||||
socket_path: allocation.socket.clone(),
|
||||
segment_id: allocation.segment_id,
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
async fn read_reachable_live_pod_records() -> Result<Vec<LivePodRecord>, io::Error> {
|
||||
let records = read_live_pod_records()?;
|
||||
let mut reachable = Vec::new();
|
||||
for record in records {
|
||||
if PodClient::connect(&record.socket_path).await.is_ok() {
|
||||
reachable.push(record);
|
||||
}
|
||||
}
|
||||
Ok(reachable)
|
||||
}
|
||||
|
||||
pub(crate) fn live_socket_for_pod(pod_name: &str) -> Option<PathBuf> {
|
||||
read_live_pod_records()
|
||||
.ok()?
|
||||
.into_iter()
|
||||
.find(|pod| pod.pod_name == pod_name)
|
||||
.map(|pod| pod.socket_path)
|
||||
}
|
||||
|
||||
fn build_rows(
|
||||
store: &FsStore,
|
||||
pod_states: Vec<PodStateRecord>,
|
||||
live_pods: Vec<LivePodRecord>,
|
||||
) -> Result<Vec<Row>, PickerError> {
|
||||
let mut rows_by_name: BTreeMap<String, Row> = BTreeMap::new();
|
||||
let mut live_by_name: HashMap<String, LivePodRecord> = HashMap::new();
|
||||
|
||||
for live in live_pods {
|
||||
let (active_session_id, active_segment_id, updated_at, preview) =
|
||||
summarize_live_pod(store, &live);
|
||||
rows_by_name.insert(
|
||||
live.pod_name.clone(),
|
||||
Row {
|
||||
pod_name: live.pod_name.clone(),
|
||||
state: PodRowState::Live,
|
||||
updated_at,
|
||||
active_session_id,
|
||||
active_segment_id,
|
||||
preview,
|
||||
socket_path: Some(live.socket_path.clone()),
|
||||
},
|
||||
);
|
||||
live_by_name.insert(live.pod_name.clone(), live);
|
||||
}
|
||||
|
||||
for record in pod_states {
|
||||
match record.state {
|
||||
Ok(metadata) => {
|
||||
let summary = summarize_metadata(store, &metadata);
|
||||
let state = if live_by_name.contains_key(&record.pod_name) {
|
||||
PodRowState::Live
|
||||
} else {
|
||||
PodRowState::Stopped
|
||||
};
|
||||
upsert_metadata_row(&mut rows_by_name, record.pod_name, metadata, summary, state);
|
||||
}
|
||||
Err(message) => {
|
||||
rows_by_name.entry(record.pod_name.clone()).or_insert(Row {
|
||||
pod_name: record.pod_name,
|
||||
state: PodRowState::Corrupt,
|
||||
updated_at: 0,
|
||||
active_session_id: None,
|
||||
active_segment_id: None,
|
||||
preview: Some(format!("metadata: {}", trim_one_line(&message, 48))),
|
||||
socket_path: None,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut rows: Vec<Row> = rows_by_name.into_values().collect();
|
||||
rows.sort_by(|a, b| {
|
||||
b.updated_at
|
||||
.cmp(&a.updated_at)
|
||||
.then_with(|| a.pod_name.cmp(&b.pod_name))
|
||||
});
|
||||
rows.truncate(MAX_ROWS);
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
fn upsert_metadata_row(
|
||||
rows_by_name: &mut BTreeMap<String, Row>,
|
||||
pod_name: String,
|
||||
metadata: PodMetadata,
|
||||
summary: SegmentSummary,
|
||||
state: PodRowState,
|
||||
) {
|
||||
let active = metadata.active;
|
||||
let active_session_id = active.as_ref().map(|a| a.session_id);
|
||||
let active_segment_id = active.as_ref().and_then(|a| a.segment_id);
|
||||
|
||||
match rows_by_name.get_mut(&pod_name) {
|
||||
Some(existing) => {
|
||||
existing.state = state;
|
||||
if summary.updated_at > existing.updated_at {
|
||||
existing.updated_at = summary.updated_at;
|
||||
}
|
||||
if existing.active_session_id.is_none() {
|
||||
existing.active_session_id = active_session_id;
|
||||
}
|
||||
if existing.active_segment_id.is_none() {
|
||||
existing.active_segment_id = active_segment_id;
|
||||
}
|
||||
if existing.preview.is_none() {
|
||||
existing.preview = summary.preview;
|
||||
}
|
||||
}
|
||||
None => {
|
||||
rows_by_name.insert(
|
||||
pod_name.clone(),
|
||||
Row {
|
||||
pod_name,
|
||||
state,
|
||||
updated_at: summary.updated_at,
|
||||
active_session_id,
|
||||
active_segment_id,
|
||||
preview: summary.preview,
|
||||
socket_path: None,
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Walk the log from the tail looking for the most recent user-message
|
||||
/// or assistant-message entry, then render its first text fragment in
|
||||
/// a single line.
|
||||
fn last_message_preview(entries: &[HashedEntry]) -> Option<String> {
|
||||
for hashed in entries.iter().rev() {
|
||||
match &hashed.entry {
|
||||
#[derive(Debug, Clone)]
|
||||
struct SegmentSummary {
|
||||
updated_at: u64,
|
||||
preview: Option<String>,
|
||||
}
|
||||
|
||||
fn summarize_live_pod(
|
||||
store: &FsStore,
|
||||
live: &LivePodRecord,
|
||||
) -> (Option<SessionId>, Option<SegmentId>, u64, Option<String>) {
|
||||
let Some(segment_id) = live.segment_id else {
|
||||
return (None, None, 0, None);
|
||||
};
|
||||
let session_id = store.lookup_session_of(segment_id).ok().flatten();
|
||||
let Some(session_id) = session_id else {
|
||||
return (None, Some(segment_id), 0, None);
|
||||
};
|
||||
let summary = summarize_segment(store, session_id, segment_id);
|
||||
(
|
||||
Some(session_id),
|
||||
Some(segment_id),
|
||||
summary.updated_at,
|
||||
summary.preview,
|
||||
)
|
||||
}
|
||||
|
||||
fn summarize_metadata(store: &FsStore, metadata: &PodMetadata) -> SegmentSummary {
|
||||
let Some(active) = metadata.active.as_ref() else {
|
||||
return SegmentSummary {
|
||||
updated_at: 0,
|
||||
preview: None,
|
||||
};
|
||||
};
|
||||
let Some(segment_id) = active.segment_id else {
|
||||
return SegmentSummary {
|
||||
updated_at: 0,
|
||||
preview: Some("[pending segment]".to_string()),
|
||||
};
|
||||
};
|
||||
summarize_segment(store, active.session_id, segment_id)
|
||||
}
|
||||
|
||||
fn summarize_segment(
|
||||
store: &FsStore,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> SegmentSummary {
|
||||
match store.read_all(session_id, segment_id) {
|
||||
Ok(entries) => SegmentSummary {
|
||||
updated_at: last_entry_ts(&entries).unwrap_or(0),
|
||||
preview: last_message_preview(&entries).or_else(|| Some("[empty]".to_string())),
|
||||
},
|
||||
Err(_) => SegmentSummary {
|
||||
updated_at: 0,
|
||||
preview: Some("[corrupt segment]".to_string()),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn last_entry_ts(entries: &[LogEntry]) -> Option<u64> {
|
||||
entries.iter().map(log_entry_ts).max()
|
||||
}
|
||||
|
||||
fn log_entry_ts(entry: &LogEntry) -> u64 {
|
||||
match entry {
|
||||
LogEntry::SegmentStart { ts, .. }
|
||||
| LogEntry::Invoke { ts, .. }
|
||||
| LogEntry::UserInput { ts, .. }
|
||||
| LogEntry::AssistantItem { ts, .. }
|
||||
| LogEntry::ToolResult { ts, .. }
|
||||
| LogEntry::SystemItem { ts, .. }
|
||||
| LogEntry::TurnEnd { ts, .. }
|
||||
| LogEntry::RunCompleted { ts, .. }
|
||||
| LogEntry::RunErrored { ts, .. }
|
||||
| LogEntry::ConfigChanged { ts, .. }
|
||||
| LogEntry::LlmUsage { ts, .. }
|
||||
| LogEntry::Extension { ts, .. } => *ts,
|
||||
}
|
||||
}
|
||||
|
||||
/// Walk the log from the tail looking for the most recent user-message or
|
||||
/// assistant-message entry, then render its first text fragment in a single line.
|
||||
fn last_message_preview(entries: &[LogEntry]) -> Option<String> {
|
||||
for entry in entries.iter().rev() {
|
||||
match entry {
|
||||
LogEntry::UserInput { segments, .. } => {
|
||||
let text = protocol::Segment::flatten_to_text(segments);
|
||||
if !text.is_empty() {
|
||||
return Some(format!("user: {}", trim_one_line(&text, 60)));
|
||||
}
|
||||
}
|
||||
LogEntry::AssistantItems { items, .. } => {
|
||||
if let Some(text) = items.iter().find_map(first_text_logged) {
|
||||
LogEntry::AssistantItem { item, .. } => {
|
||||
if let Some(text) = first_text_logged(item) {
|
||||
return Some(format!("assistant: {}", trim_one_line(&text, 60)));
|
||||
}
|
||||
}
|
||||
|
|
@ -262,7 +525,7 @@ fn draw(f: &mut Frame<'_>, rows: &[Row], selected: usize) {
|
|||
|
||||
f.render_widget(
|
||||
Paragraph::new(Line::from(vec![Span::styled(
|
||||
"resume pod pick a session",
|
||||
picker_title(),
|
||||
Style::default().add_modifier(Modifier::BOLD),
|
||||
)])),
|
||||
layout[0],
|
||||
|
|
@ -278,7 +541,7 @@ fn draw(f: &mut Frame<'_>, rows: &[Row], selected: usize) {
|
|||
Span::styled("[↑/↓]", Style::default().fg(Color::DarkGray)),
|
||||
Span::raw(" select "),
|
||||
Span::styled("[enter]", Style::default().fg(Color::Green)),
|
||||
Span::raw(" pick "),
|
||||
Span::raw(" attach/restore "),
|
||||
Span::styled("[esc]", Style::default().fg(Color::Yellow)),
|
||||
Span::raw(" cancel"),
|
||||
])),
|
||||
|
|
@ -286,9 +549,13 @@ fn draw(f: &mut Frame<'_>, rows: &[Row], selected: usize) {
|
|||
);
|
||||
}
|
||||
|
||||
fn picker_title() -> &'static str {
|
||||
"resume pod pick a pod"
|
||||
}
|
||||
|
||||
fn row_line(row: &Row, selected: bool) -> Line<'_> {
|
||||
let marker = if selected { "▶ " } else { " " };
|
||||
let id_style = if selected {
|
||||
let name_style = if selected {
|
||||
Style::default()
|
||||
.fg(Color::Cyan)
|
||||
.add_modifier(Modifier::BOLD)
|
||||
|
|
@ -300,22 +567,246 @@ fn row_line(row: &Row, selected: bool) -> Line<'_> {
|
|||
} else {
|
||||
Style::default().fg(Color::DarkGray)
|
||||
};
|
||||
|
||||
let mut spans = vec![
|
||||
Span::raw(marker),
|
||||
Span::styled(short_session(row.id), id_style),
|
||||
Span::styled(row.pod_name.as_str(), name_style),
|
||||
Span::raw(" "),
|
||||
Span::styled(format!("[{}]", row.state.label()), row.state.style()),
|
||||
Span::raw(" "),
|
||||
Span::styled(
|
||||
format_updated_at(row.updated_at),
|
||||
Style::default().fg(Color::DarkGray),
|
||||
),
|
||||
Span::raw(" "),
|
||||
Span::styled(debug_ids(row), Style::default().fg(Color::DarkGray)),
|
||||
];
|
||||
if let Some(ref pod_name) = row.live_pod {
|
||||
spans.push(Span::styled(
|
||||
format!("[live: {pod_name}] "),
|
||||
Style::default().fg(Color::Red).add_modifier(Modifier::BOLD),
|
||||
));
|
||||
if let Some(preview) = row.preview.as_ref() {
|
||||
spans.push(Span::raw(" "));
|
||||
spans.push(Span::styled(preview.as_str(), preview_style));
|
||||
}
|
||||
spans.push(Span::styled(row.preview.clone(), preview_style));
|
||||
Line::from(spans)
|
||||
}
|
||||
|
||||
fn short_session(id: SessionId) -> String {
|
||||
let s = id.to_string();
|
||||
s.chars().take(8).collect()
|
||||
fn format_updated_at(updated_at: u64) -> String {
|
||||
if updated_at == 0 {
|
||||
"updated: —".to_string()
|
||||
} else {
|
||||
format!("updated: {updated_at}")
|
||||
}
|
||||
}
|
||||
|
||||
fn debug_ids(row: &Row) -> String {
|
||||
let session = row
|
||||
.active_session_id
|
||||
.map(short_id)
|
||||
.unwrap_or_else(|| "--------".to_string());
|
||||
let segment = row
|
||||
.active_segment_id
|
||||
.map(short_id)
|
||||
.unwrap_or_else(|| "--------".to_string());
|
||||
format!("s:{session} g:{segment}")
|
||||
}
|
||||
|
||||
fn short_id<T: ToString>(id: T) -> String {
|
||||
id.to_string().chars().take(8).collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use llm_worker::llm_client::types::RequestConfig;
|
||||
use session_store::{PodActiveSegmentRef, PodMetadataStore, new_segment_id, new_session_id};
|
||||
use tempfile::tempdir;
|
||||
|
||||
#[test]
|
||||
fn pod_rows_are_sorted_by_active_segment_timestamp() {
|
||||
let dir = tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let earlier_session = new_session_id();
|
||||
let later_session = new_session_id();
|
||||
let earlier_segment = new_segment_id();
|
||||
let later_segment = new_segment_id();
|
||||
|
||||
append_start(&store, earlier_session, earlier_segment, 10);
|
||||
append_user(
|
||||
&store,
|
||||
earlier_session,
|
||||
earlier_segment,
|
||||
100,
|
||||
"old pod update",
|
||||
);
|
||||
append_start(&store, later_session, later_segment, 20);
|
||||
append_user(&store, later_session, later_segment, 200, "new pod update");
|
||||
|
||||
let records = vec![
|
||||
metadata_record("older", earlier_session, earlier_segment),
|
||||
metadata_record("newer", later_session, later_segment),
|
||||
];
|
||||
let rows = build_rows(&store, records, vec![]).unwrap();
|
||||
|
||||
assert_eq!(rows[0].pod_name, "newer");
|
||||
assert_eq!(rows[0].state, PodRowState::Stopped);
|
||||
assert_eq!(rows[0].updated_at, 200);
|
||||
assert_eq!(rows[0].preview.as_deref(), Some("user: new pod update"));
|
||||
assert_eq!(rows[1].pod_name, "older");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pod_rows_include_live_and_stopped_pods() {
|
||||
let dir = tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let stopped_session = new_session_id();
|
||||
let stopped_segment = new_segment_id();
|
||||
let live_session = new_session_id();
|
||||
let live_segment = new_segment_id();
|
||||
|
||||
append_start(&store, stopped_session, stopped_segment, 10);
|
||||
append_user(
|
||||
&store,
|
||||
stopped_session,
|
||||
stopped_segment,
|
||||
50,
|
||||
"stopped preview",
|
||||
);
|
||||
append_start(&store, live_session, live_segment, 20);
|
||||
append_user(&store, live_session, live_segment, 70, "live preview");
|
||||
|
||||
let rows = build_rows(
|
||||
&store,
|
||||
vec![metadata_record("stopped", stopped_session, stopped_segment)],
|
||||
vec![LivePodRecord {
|
||||
pod_name: "live".to_string(),
|
||||
socket_path: PathBuf::from("/tmp/live.sock"),
|
||||
segment_id: Some(live_segment),
|
||||
}],
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let live = rows.iter().find(|row| row.pod_name == "live").unwrap();
|
||||
assert_eq!(live.state, PodRowState::Live);
|
||||
assert_eq!(live.active_session_id, Some(live_session));
|
||||
assert_eq!(
|
||||
live.socket_path.as_deref(),
|
||||
Some(Path::new("/tmp/live.sock"))
|
||||
);
|
||||
|
||||
let stopped = rows.iter().find(|row| row.pod_name == "stopped").unwrap();
|
||||
assert_eq!(stopped.state, PodRowState::Stopped);
|
||||
assert_eq!(stopped.socket_path, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn corrupt_pod_state_is_rendered_as_corrupt_row() {
|
||||
let dir = tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let rows = build_rows(
|
||||
&store,
|
||||
vec![PodStateRecord {
|
||||
pod_name: "broken".to_string(),
|
||||
state: Err("expected value".to_string()),
|
||||
}],
|
||||
vec![],
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(rows.len(), 1);
|
||||
assert_eq!(rows[0].pod_name, "broken");
|
||||
assert_eq!(rows[0].state, PodRowState::Corrupt);
|
||||
assert!(
|
||||
rows[0]
|
||||
.preview
|
||||
.as_deref()
|
||||
.unwrap()
|
||||
.contains("expected value")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn picker_title_names_pods_not_sessions() {
|
||||
assert_eq!(picker_title(), "resume pod pick a pod");
|
||||
}
|
||||
|
||||
fn metadata_record(
|
||||
pod_name: &str,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
) -> PodStateRecord {
|
||||
PodStateRecord {
|
||||
pod_name: pod_name.to_string(),
|
||||
state: Ok(PodMetadata::new(
|
||||
pod_name,
|
||||
Some(PodActiveSegmentRef::active_segment(session_id, segment_id)),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn append_start(store: &FsStore, session_id: SessionId, segment_id: SegmentId, ts: u64) {
|
||||
store
|
||||
.append(
|
||||
session_id,
|
||||
segment_id,
|
||||
&LogEntry::SegmentStart {
|
||||
ts,
|
||||
session_id,
|
||||
system_prompt: None,
|
||||
config: RequestConfig::default(),
|
||||
history: vec![],
|
||||
forked_from: None,
|
||||
compacted_from: None,
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
fn append_user(
|
||||
store: &FsStore,
|
||||
session_id: SessionId,
|
||||
segment_id: SegmentId,
|
||||
ts: u64,
|
||||
text: &str,
|
||||
) {
|
||||
store
|
||||
.append(
|
||||
session_id,
|
||||
segment_id,
|
||||
&LogEntry::UserInput {
|
||||
ts,
|
||||
segments: vec![protocol::Segment::text(text)],
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_pod_state_records_reports_corrupt_metadata() {
|
||||
let dir = tempdir().unwrap();
|
||||
let pod_dir = dir.path().join("pods").join("broken");
|
||||
fs::create_dir_all(&pod_dir).unwrap();
|
||||
fs::write(pod_dir.join("metadata.json"), "{not-json").unwrap();
|
||||
|
||||
let records = read_pod_state_records(dir.path()).unwrap();
|
||||
assert_eq!(records.len(), 1);
|
||||
assert_eq!(records[0].pod_name, "broken");
|
||||
assert!(records[0].state.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_pod_state_records_reads_metadata() {
|
||||
let dir = tempdir().unwrap();
|
||||
let store = FsStore::new(dir.path()).unwrap();
|
||||
let session_id = new_session_id();
|
||||
let segment_id = new_segment_id();
|
||||
store
|
||||
.write(&PodMetadata::new(
|
||||
"agent",
|
||||
Some(PodActiveSegmentRef::active_segment(session_id, segment_id)),
|
||||
))
|
||||
.unwrap();
|
||||
|
||||
let records = read_pod_state_records(dir.path()).unwrap();
|
||||
assert_eq!(records.len(), 1);
|
||||
assert_eq!(records[0].pod_name, "agent");
|
||||
assert!(records[0].state.is_ok());
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -28,7 +28,7 @@ use ratatui::style::{Color, Modifier, Style};
|
|||
use ratatui::text::{Line, Span};
|
||||
use ratatui::widgets::Paragraph;
|
||||
use ratatui::{Frame, TerminalOptions, Viewport};
|
||||
use session_store::SessionId;
|
||||
use session_store::SegmentId;
|
||||
|
||||
const VIEWPORT_LINES: u16 = 6;
|
||||
|
||||
|
|
@ -46,7 +46,7 @@ pub enum SpawnOutcome {
|
|||
pub enum SpawnError {
|
||||
Io(io::Error),
|
||||
Store(session_store::StoreError),
|
||||
MissingResumeScope { session_id: SessionId },
|
||||
MissingResumeScope { segment_id: SegmentId },
|
||||
Spawn(client::SpawnError),
|
||||
}
|
||||
|
||||
|
|
@ -55,9 +55,9 @@ impl std::fmt::Display for SpawnError {
|
|||
match self {
|
||||
Self::Io(e) => write!(f, "io error: {e}"),
|
||||
Self::Store(e) => write!(f, "failed to read session log: {e}"),
|
||||
Self::MissingResumeScope { session_id } => write!(
|
||||
Self::MissingResumeScope { segment_id } => write!(
|
||||
f,
|
||||
"session {session_id} has no persisted scope snapshot; refusing resume without explicit scope"
|
||||
"session {segment_id} has no persisted scope snapshot; refusing resume without explicit scope"
|
||||
),
|
||||
Self::Spawn(e) => write!(f, "{e}"),
|
||||
}
|
||||
|
|
@ -89,57 +89,19 @@ type InlineTerminal = Terminal<CrosstermBackend<io::Stdout>>;
|
|||
/// Source session for a resume run. `None` = fresh spawn (current
|
||||
/// behaviour); `Some(id)` swaps the dialog into "Resume Pod" mode and
|
||||
/// passes `--session <id>` to the spawned `pod` child.
|
||||
pub async fn run(resume_from: Option<SessionId>) -> Result<SpawnOutcome, SpawnError> {
|
||||
let cwd = std::env::current_dir().map_err(SpawnError::Io)?;
|
||||
|
||||
// Run the same merge pod itself uses, then read what's missing
|
||||
// off the result. We only look at `scope.allow` here — `pod.name`
|
||||
// is intentionally an instance-level identifier and is always
|
||||
// taken from the dialog regardless of what (if anything) a layer
|
||||
// declared.
|
||||
let user_layer = user_manifest_path()
|
||||
.filter(|p| p.is_file())
|
||||
.and_then(|p| load_layer(&p).ok());
|
||||
let project_layer = find_project_manifest_from(&cwd).and_then(|p| load_layer(&p).ok());
|
||||
|
||||
let mut cascade = PodManifestConfig::builtin_defaults();
|
||||
for layer in [user_layer.as_ref(), project_layer.as_ref()]
|
||||
.into_iter()
|
||||
.flatten()
|
||||
{
|
||||
cascade = cascade.merge(layer.clone());
|
||||
}
|
||||
let cascade_has_scope = !cascade.scope.allow.is_empty();
|
||||
|
||||
let scope_origin = match (
|
||||
project_layer
|
||||
.as_ref()
|
||||
.is_some_and(|l| !l.scope.allow.is_empty()),
|
||||
user_layer
|
||||
.as_ref()
|
||||
.is_some_and(|l| !l.scope.allow.is_empty()),
|
||||
) {
|
||||
(true, _) => ScopeOrigin::FromProject,
|
||||
(false, true) => ScopeOrigin::FromUser,
|
||||
(false, false) => ScopeOrigin::CwdDefault,
|
||||
};
|
||||
|
||||
let default_name = cwd
|
||||
.file_name()
|
||||
.and_then(|s| s.to_str())
|
||||
.map(sanitise_default_name)
|
||||
.filter(|s| !s.is_empty())
|
||||
.unwrap_or_else(|| "pod".to_string());
|
||||
pub async fn run(resume_from: Option<SegmentId>) -> Result<SpawnOutcome, SpawnError> {
|
||||
let defaults = load_spawn_defaults()?;
|
||||
|
||||
let mut form = Form {
|
||||
cwd: cwd.clone(),
|
||||
cascade_has_scope,
|
||||
scope_origin,
|
||||
name_cursor: default_name.chars().count(),
|
||||
name: default_name,
|
||||
cwd: defaults.cwd.clone(),
|
||||
cascade_has_scope: defaults.cascade_has_scope,
|
||||
scope_origin: defaults.scope_origin,
|
||||
name_cursor: defaults.default_name.chars().count(),
|
||||
name: defaults.default_name,
|
||||
message: None,
|
||||
editing: true,
|
||||
resume_from,
|
||||
resume_by_pod_name: false,
|
||||
resume_scope: None,
|
||||
};
|
||||
|
||||
|
|
@ -206,6 +168,105 @@ pub async fn run(resume_from: Option<SessionId>) -> Result<SpawnOutcome, SpawnEr
|
|||
}
|
||||
}
|
||||
|
||||
/// Launch `pod --pod <name>` without opening the name dialog. The child Pod
|
||||
/// resolves persisted Pod metadata if present, or creates a fresh same-name Pod
|
||||
/// with the usual TUI cwd-scope fallback.
|
||||
pub async fn run_pod_name(pod_name: String) -> Result<SpawnOutcome, SpawnError> {
|
||||
let defaults = load_spawn_defaults()?;
|
||||
let mut form = form_for_pod_name(pod_name, defaults);
|
||||
let overlay_toml = build_overlay_toml(&form);
|
||||
let mut terminal = make_inline_terminal()?;
|
||||
terminal.draw(|f| draw_form(f, &form))?;
|
||||
|
||||
match wait_for_ready(&mut terminal, &mut form, &overlay_toml).await {
|
||||
Ok(ready) => {
|
||||
form.message = Some((
|
||||
format!("ready: {} attaching...", ready.pod_name),
|
||||
MessageKind::Ok,
|
||||
));
|
||||
terminal.draw(|f| draw_form(f, &form))?;
|
||||
drop(terminal);
|
||||
Ok(SpawnOutcome::Ready(ready))
|
||||
}
|
||||
Err(e) => {
|
||||
form.message = Some((e.to_string(), MessageKind::Error));
|
||||
let _ = terminal.draw(|f| draw_form(f, &form));
|
||||
drop(terminal);
|
||||
Err(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct SpawnDefaults {
|
||||
cwd: PathBuf,
|
||||
cascade_has_scope: bool,
|
||||
scope_origin: ScopeOrigin,
|
||||
default_name: String,
|
||||
}
|
||||
|
||||
fn load_spawn_defaults() -> Result<SpawnDefaults, SpawnError> {
|
||||
let cwd = std::env::current_dir().map_err(SpawnError::Io)?;
|
||||
|
||||
// Run the same merge pod itself uses, then read what's missing off the
|
||||
// result. We only look at `scope.allow` here — `pod.name` is an
|
||||
// instance-level identifier and is supplied by the dialog or `--pod`.
|
||||
let user_layer = user_manifest_path()
|
||||
.filter(|p| p.is_file())
|
||||
.and_then(|p| load_layer(&p).ok());
|
||||
let project_layer = find_project_manifest_from(&cwd).and_then(|p| load_layer(&p).ok());
|
||||
|
||||
let mut cascade = PodManifestConfig::builtin_defaults();
|
||||
for layer in [user_layer.as_ref(), project_layer.as_ref()]
|
||||
.into_iter()
|
||||
.flatten()
|
||||
{
|
||||
cascade = cascade.merge(layer.clone());
|
||||
}
|
||||
let cascade_has_scope = !cascade.scope.allow.is_empty();
|
||||
|
||||
let scope_origin = match (
|
||||
project_layer
|
||||
.as_ref()
|
||||
.is_some_and(|l| !l.scope.allow.is_empty()),
|
||||
user_layer
|
||||
.as_ref()
|
||||
.is_some_and(|l| !l.scope.allow.is_empty()),
|
||||
) {
|
||||
(true, _) => ScopeOrigin::FromProject,
|
||||
(false, true) => ScopeOrigin::FromUser,
|
||||
(false, false) => ScopeOrigin::CwdDefault,
|
||||
};
|
||||
|
||||
let default_name = cwd
|
||||
.file_name()
|
||||
.and_then(|s| s.to_str())
|
||||
.map(sanitise_default_name)
|
||||
.filter(|s| !s.is_empty())
|
||||
.unwrap_or_else(|| "pod".to_string());
|
||||
|
||||
Ok(SpawnDefaults {
|
||||
cwd,
|
||||
cascade_has_scope,
|
||||
scope_origin,
|
||||
default_name,
|
||||
})
|
||||
}
|
||||
|
||||
fn form_for_pod_name(pod_name: String, defaults: SpawnDefaults) -> Form {
|
||||
Form {
|
||||
cwd: defaults.cwd,
|
||||
cascade_has_scope: defaults.cascade_has_scope,
|
||||
scope_origin: defaults.scope_origin,
|
||||
name_cursor: pod_name.chars().count(),
|
||||
name: pod_name,
|
||||
message: Some(("resuming pod...".to_string(), MessageKind::Progress)),
|
||||
editing: false,
|
||||
resume_from: None,
|
||||
resume_by_pod_name: true,
|
||||
resume_scope: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn make_inline_terminal() -> io::Result<InlineTerminal> {
|
||||
let backend = CrosstermBackend::new(io::stdout());
|
||||
Terminal::with_options(
|
||||
|
|
@ -279,6 +340,7 @@ async fn wait_for_ready(
|
|||
overlay_toml: overlay_toml.to_string(),
|
||||
cwd,
|
||||
resume_from: form.resume_from,
|
||||
resume_by_pod_name: form.resume_by_pod_name,
|
||||
};
|
||||
let ready = spawn_pod(config, |line| {
|
||||
form.message = Some((line.to_string(), MessageKind::Progress));
|
||||
|
|
@ -321,7 +383,7 @@ fn build_overlay_toml(form: &Form) -> String {
|
|||
toml::to_string(&toml::Value::Table(root)).expect("overlay serialisation cannot fail")
|
||||
}
|
||||
|
||||
async fn load_resume_scope(session_id: SessionId) -> Result<ScopeConfig, SpawnError> {
|
||||
async fn load_resume_scope(segment_id: SegmentId) -> Result<ScopeConfig, SpawnError> {
|
||||
let store_dir = manifest::paths::sessions_dir().ok_or_else(|| {
|
||||
io::Error::new(
|
||||
io::ErrorKind::NotFound,
|
||||
|
|
@ -329,16 +391,17 @@ async fn load_resume_scope(session_id: SessionId) -> Result<ScopeConfig, SpawnEr
|
|||
)
|
||||
})?;
|
||||
let store = session_store::FsStore::new(&store_dir)?;
|
||||
let state = session_store::restore(&store, session_id)?;
|
||||
let state = session_store::restore_by_segment(&store, segment_id)?;
|
||||
let snapshot = state
|
||||
.pod_scope
|
||||
.ok_or(SpawnError::MissingResumeScope { session_id })?;
|
||||
.ok_or(SpawnError::MissingResumeScope { segment_id })?;
|
||||
Ok(ScopeConfig {
|
||||
allow: snapshot.allow,
|
||||
deny: snapshot.deny,
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum MessageKind {
|
||||
Info,
|
||||
Ok,
|
||||
|
|
@ -376,7 +439,10 @@ struct Form {
|
|||
/// switches, the source session is shown to the user, and the
|
||||
/// child pod is launched with `--session <id>` so it restores
|
||||
/// from `id` and appends to the same session log.
|
||||
resume_from: Option<SessionId>,
|
||||
resume_from: Option<SegmentId>,
|
||||
/// When true, launch the child with `--pod <name>` so the pod process
|
||||
/// resolves name-keyed state before falling back to fresh creation.
|
||||
resume_by_pod_name: bool,
|
||||
/// Scope snapshot recovered from the source session log. Set only for
|
||||
/// resume runs, and serialized into the overlay instead of cwd-default
|
||||
/// scope so resume does not silently broaden access.
|
||||
|
|
@ -445,7 +511,7 @@ fn draw_form(f: &mut Frame<'_>, form: &Form) {
|
|||
.split(area);
|
||||
|
||||
let title_text = match form.resume_from {
|
||||
Some(id) => format!("resume pod session: {}", short_session(id)),
|
||||
Some(id) => format!("resume pod session: {}", short_segment(id)),
|
||||
None => "spawn pod".to_string(),
|
||||
};
|
||||
let title = Paragraph::new(Line::from(vec![Span::styled(
|
||||
|
|
@ -473,7 +539,7 @@ fn draw_form(f: &mut Frame<'_>, form: &Form) {
|
|||
|
||||
/// First 8 hex digits of a UUID — short enough to skim, long enough
|
||||
/// to disambiguate inside a 10-row picker.
|
||||
pub(crate) fn short_session(id: SessionId) -> String {
|
||||
pub(crate) fn short_segment(id: SegmentId) -> String {
|
||||
let s = id.to_string();
|
||||
s.chars().take(8).collect()
|
||||
}
|
||||
|
|
@ -556,10 +622,33 @@ mod tests {
|
|||
message: None,
|
||||
editing: true,
|
||||
resume_from: None,
|
||||
resume_by_pod_name: false,
|
||||
resume_scope: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pod_name_form_restores_or_creates_by_pod_name() {
|
||||
let defaults = SpawnDefaults {
|
||||
cwd: PathBuf::from("/work/example"),
|
||||
cascade_has_scope: true,
|
||||
scope_origin: ScopeOrigin::FromProject,
|
||||
default_name: "ignored".to_string(),
|
||||
};
|
||||
let f = form_for_pod_name("agent".to_string(), defaults);
|
||||
|
||||
assert_eq!(f.name, "agent");
|
||||
assert_eq!(f.name_cursor, "agent".chars().count());
|
||||
assert_eq!(f.resume_from, None);
|
||||
assert!(f.resume_by_pod_name);
|
||||
assert!(f.resume_scope.is_none());
|
||||
assert!(!f.editing);
|
||||
assert_eq!(
|
||||
f.message,
|
||||
Some(("resuming pod...".to_string(), MessageKind::Progress))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overlay_adds_scope_default_when_cascade_lacks_scope() {
|
||||
let f = form("agent-1", false);
|
||||
|
|
@ -584,7 +673,7 @@ mod tests {
|
|||
#[test]
|
||||
fn overlay_uses_resume_scope_snapshot() {
|
||||
let mut f = form("agent-r", false);
|
||||
f.resume_from = Some(session_store::new_session_id());
|
||||
f.resume_from = Some(session_store::new_segment_id());
|
||||
f.resume_scope = Some(ScopeConfig {
|
||||
allow: vec![manifest::ScopeRule {
|
||||
target: PathBuf::from("/work/example"),
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@
|
|||
//! ──────────── separator ──────────
|
||||
//! status line (1 row)
|
||||
//! > input area (1 row in Phase 1)
|
||||
//! actionbar (1 row)
|
||||
//! ```
|
||||
//!
|
||||
//! Every frame we walk the entire `App::blocks` vector, produce styled
|
||||
|
|
@ -78,6 +79,7 @@ pub fn draw(frame: &mut Frame, app: &mut App) {
|
|||
Constraint::Length(1), // separator
|
||||
Constraint::Length(1), // status
|
||||
Constraint::Length(input_height), // input area
|
||||
Constraint::Length(1), // actionbar
|
||||
])
|
||||
.split(area);
|
||||
|
||||
|
|
@ -88,6 +90,7 @@ pub fn draw(frame: &mut Frame, app: &mut App) {
|
|||
draw_separator(frame, chunks[3]);
|
||||
draw_status(frame, app, chunks[4]);
|
||||
draw_input(frame, &input_render, chunks[5]);
|
||||
draw_actionbar(frame, app, chunks[6]);
|
||||
if let Some(state) = app.completion.as_ref().filter(|c| c.is_active()) {
|
||||
draw_completion_popup(frame, state, chunks[5]);
|
||||
}
|
||||
|
|
@ -1019,10 +1022,10 @@ fn render_compact(lines: &mut Vec<Line<'static>>, evt: &CompactEvent, width: u16
|
|||
)
|
||||
}
|
||||
CompactEvent::Done {
|
||||
new_session_id,
|
||||
new_segment_id,
|
||||
elapsed_secs,
|
||||
} => {
|
||||
let short = new_session_id
|
||||
let short = new_segment_id
|
||||
.to_string()
|
||||
.chars()
|
||||
.take(8)
|
||||
|
|
@ -1074,6 +1077,20 @@ fn draw_separator(frame: &mut Frame, area: Rect) {
|
|||
);
|
||||
}
|
||||
|
||||
fn context_usage_text(app: &App) -> String {
|
||||
let pct = if app.context_window == 0 {
|
||||
0
|
||||
} else {
|
||||
((app.session_context_tokens as f64 / app.context_window as f64) * 100.0).round() as u64
|
||||
};
|
||||
format!(
|
||||
"{} / {} ({}%)",
|
||||
fmt_tokens(app.session_context_tokens),
|
||||
fmt_tokens(app.context_window),
|
||||
pct
|
||||
)
|
||||
}
|
||||
|
||||
fn draw_status(frame: &mut Frame, app: &App, area: Rect) {
|
||||
let conn = if app.connected {
|
||||
Span::styled("●", Style::default().fg(Color::Green))
|
||||
|
|
@ -1124,7 +1141,15 @@ fn draw_status(frame: &mut Frame, app: &App, area: Rect) {
|
|||
spans.push(Span::styled(" idle", Style::default().fg(Color::DarkGray)));
|
||||
}
|
||||
|
||||
// Right-aligned mode / scroll indicator.
|
||||
let right_text = context_usage_text(app);
|
||||
let right_line = Line::from(Span::styled(right_text, Style::default().fg(Color::Gray)))
|
||||
.alignment(ratatui::layout::Alignment::Right);
|
||||
|
||||
frame.render_widget(Paragraph::new(Line::from(spans)), area);
|
||||
frame.render_widget(Paragraph::new(right_line), area);
|
||||
}
|
||||
|
||||
fn draw_actionbar(frame: &mut Frame, app: &App, area: Rect) {
|
||||
let mut right: Vec<Span<'static>> = Vec::new();
|
||||
if !app.scroll.follow_tail {
|
||||
right.push(Span::styled(
|
||||
|
|
@ -1137,8 +1162,6 @@ fn draw_status(frame: &mut Frame, app: &App, area: Rect) {
|
|||
Style::default().fg(Color::DarkGray),
|
||||
));
|
||||
let right_line = Line::from(right).alignment(ratatui::layout::Alignment::Right);
|
||||
|
||||
frame.render_widget(Paragraph::new(Line::from(spans)), area);
|
||||
frame.render_widget(Paragraph::new(right_line), area);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ Pod::try_pre_run_compact ← proactive
|
|||
|
||||
- **条件付き実行**: 推定トークン節約量が `min_savings` を超えた場合のみ。KV キャッシュの無駄な無効化を避ける
|
||||
- **リクエストコンテキストのみ操作**: history 本体は変更しない。Prune 状態を Pod が保持し、LLM リクエスト構築時に反映する
|
||||
- **保護境界**: 直近 `prune_protected_tokens` 相当の suffix は残す。turn 数ではなく usage history 由来の token estimate で境界を引くため、単発の長い tool loop でも古い `ToolResult.content` が候補になる
|
||||
- **冪等**: `content: None` のアイテムはスキップ
|
||||
|
||||
### ToolOutput の構造
|
||||
|
|
@ -138,8 +139,9 @@ compact は fork と同じ構造。旧セッションを保全し、新 SessionI
|
|||
[compaction]
|
||||
compact_threshold = 80000 # ターンの合間 (proactive)
|
||||
compact_request_threshold = 90000 # リクエストの合間 (safety net)
|
||||
retained_tokens = 8000 # 直近保護トークン数 (Prune 済みで計測)
|
||||
auto_read_budget = 8000 # compact worker の mark_read_required 合計上限
|
||||
prune_protected_tokens = 8000 # prune から保護する末尾 token budget
|
||||
compact_retained_tokens = 8000 # compact 後に生のまま残す末尾 token budget
|
||||
compact_auto_read_budget = 8000 # compact worker の mark_read_required 合計上限
|
||||
compact_worker_max_input_tokens = 50000 # compact worker 自身の現在占有トークン上限
|
||||
compact_worker_max_turns = 20 # compact worker 自身の tool loop 上限
|
||||
```
|
||||
|
|
|
|||
|
|
@ -191,9 +191,9 @@ permission = "write"
|
|||
# セクションを書いた時点で Prune は有効化、Compact は閾値が None なら無効。
|
||||
# [compaction]
|
||||
#
|
||||
# # 任意。デフォルト: 3 (`defaults::PRUNE_PROTECTED_TURNS`)。
|
||||
# # pruning から保護する末尾ターン数。
|
||||
# prune_protected_turns = 3
|
||||
# # 任意。デフォルト: 8000 (`defaults::PRUNE_PROTECTED_TOKENS`)。
|
||||
# # pruning から保護する末尾 token budget。turn 数ではなく usage estimate で境界を引く。
|
||||
# prune_protected_tokens = 8000
|
||||
#
|
||||
# # 任意。デフォルト: 4096 (`defaults::PRUNE_MIN_SAVINGS`)。
|
||||
# # prune が発火するための最低節約 token 推定値。
|
||||
|
|
@ -263,10 +263,6 @@ permission = "write"
|
|||
# # ※ memory tools と resident injection は extract_threshold が None でも動く。
|
||||
# extract_threshold = 30000
|
||||
#
|
||||
# # 任意。デフォルト: 30000 (`defaults::MEMORY_EXTRACT_WORKER_MAX_INPUT_TOKENS`)。
|
||||
# # extract worker 自身の現在占有 token cap (超過で abort)。
|
||||
# extract_worker_max_input_tokens = 30000
|
||||
#
|
||||
# # 任意。デフォルト: 8 (`defaults::MEMORY_EXTRACT_WORKER_MAX_TURNS`)。
|
||||
# # extract worker 自身の tool loop 上限。Rust config で None の場合のみ無制限。
|
||||
# extract_worker_max_turns = 8
|
||||
|
|
|
|||
|
|
@ -99,13 +99,13 @@ pub enum LogEntry {
|
|||
UserInput { ts: u64, item: Item },
|
||||
|
||||
// アシスタント応答(worker.rs:1040-1041 に対応)
|
||||
AssistantItems { ts: u64, items: Vec<Item> },
|
||||
AssistantItem { ts: u64, item: Item },
|
||||
|
||||
// ツール実行結果(worker.rs:897-900, 1072-1076 に対応)
|
||||
ToolResults { ts: u64, items: Vec<Item> },
|
||||
ToolResult { ts: u64, item: Item },
|
||||
|
||||
// Hook 注入 Items(worker.rs:1055 ContinueWithMessages に対応)
|
||||
HookInjectedItems { ts: u64, items: Vec<Item> },
|
||||
// typed system injection
|
||||
SystemItem { ts: u64, item: SystemItem },
|
||||
|
||||
// ターン境界
|
||||
TurnEnd { ts: u64, turn_count: usize },
|
||||
|
|
@ -126,7 +126,7 @@ pub enum LogEntry {
|
|||
pub enum Outcome { Finished, Paused, Error { message: String } }
|
||||
```
|
||||
|
||||
**Replay ロジック**: 全エントリ種別を走査し、`*Items` / `UserInput` → history に append、
|
||||
**Replay ロジック**: 全エントリ種別を走査し、`AssistantItem` / `ToolResult` / `SystemItem` / `UserInput` → history に append、
|
||||
`TurnEnd` → turn_count 更新、`CacheLocked` → locked_prefix_len 設定。
|
||||
|
||||
### TraceEntry(event_trace.rs)
|
||||
|
|
|
|||
|
|
@ -122,7 +122,7 @@ Workflow 保護は専用 tool schema のトリックではなく Linter ルー
|
|||
|
||||
- **Trigger**: activity tokens の累積閾値(cumulative input tokens since last pointer)。tool call カウントは不採用(ツールカスタマイズ非依存・大小重みづけのため)
|
||||
- **実行主体**: 既存 compact と同じ Worker spawn 機構を再利用。Pod は立てない
|
||||
- **入力**: 前回 extract 以降の session log 範囲。処理済み境界の pointer は session log 側に保持し、寿命を session と揃える。session-store のドメイン純度を保つため、汎用拡張点 `LogEntry::Extension { domain, payload }`(domain = `"memory.extract"`)に寄せ、session-store は memory ドメインを知らない
|
||||
- **入力**: 前回 extract 以降の session log 範囲。処理済み境界の pointer は session log 側に保持し、寿命を session と揃える。session-store のドメイン純度を保つため、汎用拡張点 `LogEntry::Extension { domain, payload }`(domain = `"memory.extract"`)に寄せ、session-store は memory ドメインを知らない。Tool result は raw `content` ではなく表示用 `summary` だけを render し、巨大な tool output を extract input に載せない
|
||||
- **出力**: JSON schema で**活動ログ**の候補配列を返す。Knowledge 等の派生物は consolidation が活動ログから導出するので、extract では純粋な「起きたこと」に絞る
|
||||
- `decisions`: 判断したこと(選択肢 + 選んだ + 根拠)
|
||||
- `discussions`: 議論したこと(トピック + 論点)
|
||||
|
|
@ -131,6 +131,7 @@ Workflow 保護は専用 tool schema のトリックではなく Linter ルー
|
|||
- **抽出対象がなければ空配列を返してよい**(Hermes の "Nothing to save." と同系。頻繁発火を許容する前提)
|
||||
- **書き込み先**: `memory/_staging/<id>.json`
|
||||
- LLM 出力(活動ログ JSON)は pod 側ラッパーが `source: { session_id, range: [start_entry, end_entry] }` を**機械付与**して wrap。LLM には source を推論させない
|
||||
- **実行保証**: extract worker 自身の input occupancy cap は設けない。未処理 range が大きい場合でも pointer 以降の最大範囲を渡し、LLM/API/tool failure のときだけ pointer を進めない
|
||||
- **モデル**: `memory.extract_model`。軽量だが文脈理解できる中堅クラス(Haiku / 4o-mini / Flash 相当)を想定
|
||||
- **Compact との順序**: 同一 turn 完了後の post-run チェックで extract を **compact より前** に走らせる。compact は history を組み替えるので、extract の入力範囲(session log 上の entry index)は compact 前のほうが安定する
|
||||
- **並走防止 (extract 同士)**: Pod 上の `extract_in_flight` フラグで in-flight 中の新規 trigger を skip。完了時点で閾値超過していれば直ちに次回を発火し、新 pointer 以降の最大範囲を回収する(pending 状態は保持しない=完了時の閾値再評価で coalesce 相当の挙動を成立させる)
|
||||
|
|
|
|||
|
|
@ -179,7 +179,7 @@ pattern = "*.env"
|
|||
action = "deny"
|
||||
|
||||
[compaction]
|
||||
prune_protected_turns = 3
|
||||
prune_protected_tokens = 8000
|
||||
prune_min_savings = 4096
|
||||
compact_threshold = 80000
|
||||
compact_request_threshold = 90000
|
||||
|
|
|
|||
23
docs/report/2026-05-22-pod-event-callback-delivery.md
Normal file
23
docs/report/2026-05-22-pod-event-callback-delivery.md
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# Spawned Pod 完了通知が来ない経路の疑い
|
||||
|
||||
## 観測
|
||||
|
||||
Spawned Pod が完了しているにもかかわらず、parent 側に完了通知が来ないことがある。`ReadPodOutput` の assistant text 抽出バグは修正済みだが、完了通知そのものの `PodEvent` delivery path に別の構造的リスクが見つかった。
|
||||
|
||||
## 現行の通知経路
|
||||
|
||||
child の parent-originated turn が `Finished` になると、child controller の `drive_turn` が parent socket へ `PodEvent::TurnEnded` を fire-and-forget する。
|
||||
|
||||
parent 側は `Method::PodEvent` を受けると side effect を適用し、`NotifyBuffer` に typed event を積む。parent が idle なら `RunForNotification(PodEvent)` が auto-kick され、interceptor が `SystemItem::PodEvent` として history に commit し、LLM request に通知が乗る。
|
||||
|
||||
## 疑い
|
||||
|
||||
送信 helper `connect_and_send` は socket に接続して Method を 1 行 write し、応答を読まずに close する。一方、受信側 `SocketServer::handle_connection` は Method を読む前に alert snapshot と `Event::Snapshot` を client に write する。
|
||||
|
||||
この組み合わせでは、send-only client が読まない / read half を保持しないため、server 側が snapshot write で失敗または詰まり、Method を読む前に connection handler が終わる可能性がある。これが起きると child の `PodEvent::TurnEnded` は parent controller に到達せず、NotifyBuffer にも入らない。
|
||||
|
||||
影響は `PodEvent` だけでなく、`StopPod` が child に送る `Method::Shutdown` など `connect_and_send` 利用箇所全般に及ぶ可能性がある。
|
||||
|
||||
## 対応
|
||||
|
||||
`tickets/pod-event-callback-delivery.md` を作成した。callback / fire-and-forget Method delivery を server の initial snapshot write に阻害されない形へ修正し、大きな snapshot を持つ parent に対しても `PodEvent::TurnEnded` が届く regression test を追加する。
|
||||
15
docs/report/2026-05-22-read-pod-output-singular-log-entry.md
Normal file
15
docs/report/2026-05-22-read-pod-output-singular-log-entry.md
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
# ReadPodOutput が LogEntry schema の分岐に取り残され no new assistant text になる
|
||||
|
||||
## 観測
|
||||
|
||||
spawned Pod のレビュー出力について、operator が attach すると assistant 出力が見える一方、spawner 側の `ReadPodOutput` は `pod ... running; no new assistant text` を返した。
|
||||
|
||||
## 原因
|
||||
|
||||
`crates/pod/src/spawn/comm_tools.rs` の `extract_assistant_text` が、`Event::Snapshot` 内の `LogEntry` を独自に解釈していた。session log の標準形は `LogEntry::AssistantItem { item, .. }` だが、`ReadPodOutput` 側の抽出対象が古い複数 item 形式に寄ったままになっていたため、新しい assistant 出力が snapshot に存在しても取りこぼした。
|
||||
|
||||
これは entry hash の問題ではなく、`LogEntry` に対する派生操作が各所で独自実装され、後方互換 variant が残ったことで標準形への追従漏れを型で検出できなかった問題。
|
||||
|
||||
## 対応
|
||||
|
||||
`LogEntry` から古い複数 item 形式の後方互換 variant を削除し、`ReadPodOutput` / TUI / picker / tests を現在の singular entry だけに揃える。これにより schema 変更時に独自 match の取り残しが compile error として表面化する。
|
||||
10
docs/tui-parts.md
Normal file
10
docs/tui-parts.md
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
```
|
||||
gap |
|
||||
task(if some)| 8 tasks - pending: 2, inprogress:1, completed:5
|
||||
|-----------------------------------------------------------
|
||||
status |● insomnia idle 42.1k / 200k (21%)
|
||||
input |>
|
||||
actionbar | ↑ scrolled [normal]
|
||||
```
|
||||
|
||||
status 右端は常に session context usage を `<tokens> / <window> (<pct>%)` 形式で表示する。mode / scrolled などの操作状態は actionbar に寄せる。
|
||||
|
|
@ -2,42 +2,51 @@
|
|||
[[model]]
|
||||
id = "claude-sonnet-4-6"
|
||||
provider = "anthropic"
|
||||
context_window = 200000
|
||||
|
||||
[[model]]
|
||||
id = "claude-sonnet-4-5"
|
||||
provider = "anthropic"
|
||||
context_window = 200000
|
||||
|
||||
[[model]]
|
||||
id = "claude-opus-4-1"
|
||||
provider = "anthropic"
|
||||
context_window = 200000
|
||||
|
||||
# Ollama local (capability is router-ish / ollama handles its own models)
|
||||
[[model]]
|
||||
id = "llama3.1"
|
||||
provider = "ollama-local"
|
||||
context_window = 128000
|
||||
|
||||
[[model]]
|
||||
id = "qwen2.5-coder"
|
||||
provider = "ollama-local"
|
||||
context_window = 128000
|
||||
|
||||
# Codex OAuth (ChatGPT backend via Responses API)
|
||||
[[model]]
|
||||
id = "gpt-5-codex"
|
||||
provider = "codex-oauth"
|
||||
context_window = 400000
|
||||
capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } }
|
||||
|
||||
[[model]]
|
||||
id = "gpt-5"
|
||||
provider = "codex-oauth"
|
||||
context_window = 400000
|
||||
capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } }
|
||||
|
||||
# OpenRouter
|
||||
[[model]]
|
||||
id = "anthropic/claude-sonnet-4"
|
||||
provider = "openrouter"
|
||||
context_window = 200000
|
||||
capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "budget_tokens", vision = true, prompt_caching = { kind = "auto" } }
|
||||
|
||||
[[model]]
|
||||
id = "openai/gpt-5"
|
||||
provider = "openrouter"
|
||||
context_window = 400000
|
||||
capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } }
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ scheme = "anthropic"
|
|||
base_url = "https://api.anthropic.com"
|
||||
auth_hint = { kind = "api_key", env = "INSOMNIA_API_KEY_ANTHROPIC" }
|
||||
default_capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "budget_tokens", vision = true, prompt_caching = { kind = "explicit", max_breakpoints = 4 } }
|
||||
default_context_window = 200000
|
||||
|
||||
[[provider]]
|
||||
id = "ollama-local"
|
||||
|
|
@ -13,6 +14,7 @@ scheme = "anthropic"
|
|||
base_url = "http://localhost:11434"
|
||||
auth_hint = { kind = "none" }
|
||||
default_capability = { tool_calling = "parallel", structured_output = "json_schema", vision = false, prompt_caching = { kind = "auto" } }
|
||||
default_context_window = 128000
|
||||
|
||||
[[provider]]
|
||||
id = "codex-oauth"
|
||||
|
|
@ -20,6 +22,7 @@ display_name = "ChatGPT (Codex OAuth)"
|
|||
scheme = "openai_responses"
|
||||
auth_hint = { kind = "codex_oauth" }
|
||||
default_capability = { tool_calling = "parallel", structured_output = "json_schema", reasoning = "effort", vision = true, prompt_caching = { kind = "auto" } }
|
||||
default_context_window = 400000
|
||||
|
||||
[[provider]]
|
||||
id = "openrouter"
|
||||
|
|
@ -28,3 +31,4 @@ scheme = "openai_chat"
|
|||
base_url = "https://openrouter.ai/api/v1"
|
||||
auth_hint = { kind = "api_key", env = "INSOMNIA_API_KEY_OPENROUTER" }
|
||||
default_capability = { tool_calling = "parallel", structured_output = "json_schema", vision = true, prompt_caching = { kind = "auto" } }
|
||||
default_context_window = 200000
|
||||
|
|
|
|||
|
|
@ -1,188 +0,0 @@
|
|||
# 永続化層のセマンティック整理
|
||||
|
||||
## 背景
|
||||
|
||||
現在の永続化は `SessionId` 単位の append-only JSONL log を中心に構成されている。これは実装上は扱いやすい一方で、今後 Pod 単位永続化、compaction、fork、DB backend 追加などを進めるにあたり、以下の概念が混ざり始めている。
|
||||
|
||||
- ユーザー視点の「同じ会話 / 作業の継続単位」
|
||||
- Pod 視点の「現在 active な会話状態」
|
||||
- append-only log の物理的 / 復元上の単位
|
||||
- compaction によって生成される新しい履歴系列
|
||||
- fork の起点となる履歴中の境界
|
||||
- runtime dir に置かれる一時状態と、data dir / DB に置く永続正本
|
||||
|
||||
特に、現在は compaction によって新しい `SessionId` が発行される。これは append-only log の低レベル単位としては自然だが、ユーザー視点では「同じ会話が継続している」とも見えるため、`Session` という名称・粒度が今後の設計上あいまいになり得る。
|
||||
|
||||
このチケットでは、実装変更に入る前に、永続化層のドメイン概念・名称・責務境界を整理する。
|
||||
|
||||
## 目的
|
||||
|
||||
- 永続化層で扱う概念を、ユーザー視点 / Pod 視点 / storage 視点に分けて定義する。
|
||||
- `SessionId` が今後も適切な中心概念か、あるいは別概念に分解すべきかを判断する。
|
||||
- compaction / fork / resume / Pod state / spawned child registry が、どの粒度のデータに属するかを決める。
|
||||
- 将来 DB backend を追加しても歪みにくいデータ構造を設計する。
|
||||
- 既存の session-store JSONL 実装から段階的に移行できる命名・API 境界を決める。
|
||||
|
||||
## 結論: Session / Segment / Entry の 3 階層
|
||||
|
||||
```
|
||||
Session ← ユーザー視点の会話の家系(fork tree 全体。Segment 群の grouping)
|
||||
└── Segment ← Compaction / Fork で切れる物理 append-only 単位(現在の SessionId 相当)
|
||||
└── Entry ← 1 永続化イベント
|
||||
```
|
||||
|
||||
セマンティクスの対応:
|
||||
|
||||
- **resume** = 同 Session 内の指定 leaf Segment に append
|
||||
- **compaction** = 同 Session, new Segment(lineage: `compacted_from`)
|
||||
- **fork** (live / 過去いずれも) = 同 Session, new Segment(lineage: `forked_from`)
|
||||
- **branching は Session 内で完結**する。Segment 間が DAG、Segment 内は完全 linear。
|
||||
- **Session は分岐ツリーを持つ静的な構造**。「今どの Segment に書いているか」(current active Segment) は Pod state 側が動的に保持する。
|
||||
|
||||
文脈合成 (merge / cherry-pick) は今後の要件として想定しないため、entry レベルの DAG(任意の親 pointer)は採用しない。Segment 内 linear + Segment 間 DAG の 2 階層に閉じることで、同 Segment 内は `(segment_id, seq)` の連番 PK で直線探索でき、Segment 間の lineage は粗粒度な DAG として軽量に管理できる。
|
||||
|
||||
fork で新しい Session を切らないのは、compaction で Segment を切るのと fork で Segment を切るのが対称な操作であり、ユーザー視点でも「同じ起源から派生した枝」として 1 つの単位で扱う方が自然なため。これにより Session 数が分岐で爆発せず、`WHERE session_id = ?` だけで fork tree 全体が取れる。
|
||||
|
||||
### Restore / Resume の API
|
||||
|
||||
Session が分岐を含むため、resume の指定は **(SessionId, leaf SegmentId) の組** で行う:
|
||||
|
||||
- TUI 経路: ユーザーは **Session を選択 → その Session 内の leaf を選択**。
|
||||
- 内部 API: restore は `(SessionId, SegmentId)` を取り、指定 Segment から replay する。leaf 以外を指定すれば read-only な過去状態の参照も可能。
|
||||
- 「最後に active だった leaf」は Pod state が保持するので、TUI が初期選択候補として使える。
|
||||
|
||||
### 既存コードとの対応
|
||||
|
||||
| 既存 | 新名称 |
|
||||
|---|---|
|
||||
| `SessionId` | `SegmentId` にリネーム |
|
||||
| (なし) | `SessionId` 新設(Segment 群をまとめる、ユーザー視点 ID) |
|
||||
|
||||
`session-store` crate の型・関数は Segment 中心の命名に揃える。crate 名自体を変えるかは別論点(中身は引き続き Segment 単位の append-only log を扱う)。
|
||||
|
||||
### llm-worker への影響
|
||||
|
||||
llm-worker は session 概念を持たない(`Worker` は `history` / `turn_count` / `RequestConfig` を持つだけで永続化への hook も無い)。Session / Segment 階層の導入は llm-worker 層に染み出さず、影響範囲は `session-store` / `pod` / `pod-registry` / `pod-cli` に閉じる。
|
||||
|
||||
## Entry hash の廃止
|
||||
|
||||
現状、各 entry は SHA-256 hash chain (`prev_hash` → `hash`) を持つが、実際に効いている用途は 2 つだけ:
|
||||
|
||||
1. `ensure_head_or_fork` (pod.rs:1348) — store の末尾と Pod の保持する `head_hash` を比較し、不一致なら auto-fork。**末尾識別子があれば良い**(hash chain そのものは要らない)。
|
||||
2. `fork_at(source_id, at_hash)` (session.rs:425) — 過去 entry pointer から fork。`pod-session-fork.md` の入口仕様は turn number で、entry hash は内部 pointer に過ぎない。turn boundary (TurnEnd entry の index) で代替可能。
|
||||
|
||||
改竄検知 (tamper-evident chain) は宣伝されているがコード上は walk して verify するルートが無いため、削除しても regression にならない。
|
||||
|
||||
廃止に伴う対応:
|
||||
|
||||
- `HashedEntry` 廃止、JSONL は 1 行 1 `LogEntry`。
|
||||
- `SessionOrigin.at_hash` → `at_turn_index` (TurnEnd 由来) に置換。
|
||||
- `ensure_head_or_fork` の検知ロジックは、Segment 末尾の terminal marker entry または末尾 seq 比較に置換(形式は実装時に決める)。
|
||||
|
||||
### 廃止前の足場 (前提)
|
||||
|
||||
本セクションを実装に移すタイミングでは、log writer が既に sync 化されていることを前提にする (`tickets/log-entry-singular-and-direct-commit.md`)。具体的には:
|
||||
|
||||
- `Store::append` / `read_all` 等が `std::fs` ベースの sync API
|
||||
- `SessionLogWriter::append_entry()` が sync 関数
|
||||
- `session_head` mutex は `parking_lot::Mutex` / `std::sync::Mutex`
|
||||
- `LogCommand` / drain task / Flush バリアは既に撤廃済み
|
||||
|
||||
この状態で hash chain を廃止すると追加で取れる単純化:
|
||||
|
||||
- **`session_head` mutex そのものを撤去できる**。 hash chain が無いので「`head_hash` を直前 entry から取得して次に渡す」 という serialize 必須の依存が消える。 1 行 < `PIPE_BUF` (Linux 4KB) の `O_APPEND` write は kernel 側で atomic に直列化されるので、 user space で mutex を持つ必要が無い
|
||||
- `session_head` が消えると Pod / interceptor / worker callback が writer ハンドルだけ持てば良くなる。 `Arc<SessionLogWriter>` は単に `Arc<Store> + sink` を抱えるだけの値で、 hot-path の競合がない
|
||||
- `compute_hash` 呼び出しが消える分、 append が serialize + open + write + close の 3 syscall まで詰まる
|
||||
|
||||
つまり「sync 化」 が先に来て、 「hash 廃止」 で mutex まで消える、 という 2 段階の単純化になる。
|
||||
|
||||
## Fork: 2 種類の書き込み方
|
||||
|
||||
Session 境界の話ではなく **元 Segment への marker 書き込みの有無**で 2 種類を分ける。Session はどちらの場合も同じで、新 Segment が同 Session 内に生える。
|
||||
|
||||
- **live auto-fork**(concurrent writer 検知)
|
||||
- 元 Segment の末尾に terminal marker (`Forked { to: SegmentId }` 等) を append → 以降の writer は marker を見て新 Segment へ自動移動。
|
||||
- CoW semantics: 元 Segment は immutable、生まれた Segment 同士は対等な兄弟。
|
||||
- **過去 fork**(UI で turn 選択)
|
||||
- 元 Segment は **無変更**、replay して新 Segment を生やすだけ。
|
||||
- 起点は `(source_segment_id, at_turn_index)`。
|
||||
- 元 Segment に書き込まないため、過去 fork を nested に重ねても解釈が単純。
|
||||
|
||||
両者を同じ marker で扱おうとすると、過去 fork から更に過去で fork した場合に元 Segment への marker 位置解釈が複雑化して破綻するため、過去 fork 側は元 Segment に触れない方針で固定する。
|
||||
|
||||
### fork の物理操作(物理コピーは不要)
|
||||
|
||||
「fork = 同 Session 内に Segment を増やす」と言っても、過去 Segment を丸ごとコピーする必要は無い:
|
||||
|
||||
- source segment の seq=N までを replay して得られた `Vec<Item>` を、新 Segment の seed entry (`SessionStart` 相当) に 1 回書き込む。
|
||||
- compaction の transitive な圧縮効果がここに乗る(source segment の `SessionStart.history` に過去 Segment の compaction 結果が既に埋まっている)ので、**source segment 1 本だけ replay すれば fork seed が作れる**。さらに過去の Segment は touch しない。
|
||||
- 新 Segment が持つ lineage 情報は `(parent_segment_id, fork_at_turn)` のメタデータのみ。
|
||||
|
||||
これは現状の `fork_at` (`session.rs:430-456`) の挙動と同じで、Session 階層が乗っても操作は変わらない。
|
||||
|
||||
## RDB backend を想定した概念モデル
|
||||
|
||||
将来 DB backend を追加しても歪みにくい形として、以下の関係を仮定する:
|
||||
|
||||
```
|
||||
sessions (
|
||||
id PK,
|
||||
... -- ユーザー視点 metadata
|
||||
)
|
||||
|
||||
segments (
|
||||
id PK,
|
||||
session_id FK,
|
||||
parent_segment_id FK NULL, -- compaction / fork の元
|
||||
fork_at_turn INT NULL,
|
||||
origin_kind ENUM (new | compact | fork),
|
||||
lineage_path ltree, -- 祖先・子孫の逆引き用 (materialized path)
|
||||
...
|
||||
)
|
||||
|
||||
entries (
|
||||
segment_id FK,
|
||||
seq INT,
|
||||
kind, payload, ts,
|
||||
PRIMARY KEY (segment_id, seq) -- 同 Segment 内 linear scan
|
||||
)
|
||||
```
|
||||
|
||||
- 同 Segment 内 entry は `(segment_id, seq)` PK で linear scan、surrogate identity なので hash 不要。
|
||||
- Segment 間 lineage は `parent_segment_id` chain。深さは compaction / fork 回数のみ(Session あたり数〜数十)。
|
||||
- Segment lineage の祖先・子孫逆引きは `lineage_path` の `<@` / `@>` で 1 index 引き。entry 単位の DAG ではないため materialized path のメンテコストも軽い。
|
||||
- 通常 append は `lineage_path` 更新を伴わない(Segment 生成時に確定)。
|
||||
- 「Session の fork tree 全体」は `WHERE session_id = ?` で取れる。Session 単位の listing / GC が自然。
|
||||
|
||||
FsStore 実装はこの構造のサブセット相当として位置付ける(1 Segment = 1 jsonl、`session_id` は Segment の metadata に持たせるか別ファイルに index する)。
|
||||
|
||||
## 残る検討事項
|
||||
|
||||
- pod.scope extension entry を Pod state 側に寄せるか、Segment log に残すか(`tickets/pod-persistent-state.md` 側と合わせて決定)。
|
||||
- 撤廃の選択肢: (a) Segment log から削除し Pod state を唯一の正本にする / (b) snapshot 保持責務だけ Pod state に寄せ、scope 変更 event は Segment log に残す / (c) 現状維持で Pod state は Segment への参照のみ。
|
||||
- live auto-fork の marker 形式 (terminal entry vs Segment 末尾 seq 比較)。
|
||||
- pod-cli / TUI の `--session` 引数を Session 単位にするか Segment 単位にするか。debug 用 ID とユーザー向け ID の分離。
|
||||
- `session-store` crate 名のリネーム要否。
|
||||
|
||||
## 完了条件
|
||||
|
||||
- 永続化層の主要概念と名称が文書化されている。
|
||||
- compaction / fork / resume / Pod state のデータ粒度が決まっている。
|
||||
- 現在の `SessionId` / session-store API をどう扱うか、維持・alias・rename・段階移行の方針が決まっている。
|
||||
- DB backend を追加する場合の概念モデルが、最低限テーブル / relation 相当で説明できる。
|
||||
- `tickets/pod-persistent-state.md` や fork 関連チケットに反映すべき前提が整理されている。
|
||||
|
||||
## 範囲外
|
||||
|
||||
- このチケット単体での大規模 rename 実装。
|
||||
- DB backend の実装。
|
||||
- UI の履歴表示 / branch 表示の詳細 UX。
|
||||
- GC / retention policy の実装。
|
||||
- Session を跨ぐ merge / cherry-pick。
|
||||
|
||||
## 関連
|
||||
|
||||
- `tickets/pod-persistent-state.md`
|
||||
- `tickets/pod-session-fork.md`
|
||||
- `crates/session-store/`
|
||||
- `crates/pod/src/pod.rs`
|
||||
78
tickets/pod-discovery-restore-tools.md
Normal file
78
tickets/pod-discovery-restore-tools.md
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
# Pod: 過去 Pod の探索と restore ツール
|
||||
|
||||
## 背景
|
||||
|
||||
Pod state の永続化と `--pod <name>` resume が入ったことで、名前が分かっている Pod は復元できるようになった。一方で、AI / operator が「過去にどんな Pod があったか」「この名前の Pod は復元できるか」「live attach できるのか、restore が必要なのか」を機械的に調べる導線はまだない。
|
||||
|
||||
現在の `ListPods` は主に spawner が知っている spawned child の live/runtime registry を見るためのツールであり、永続化された全 Pod の探索や、過去 Pod の restore 導線としては不十分。今後 Pod 単位で作業を再開する運用を成立させるには、Pod state を正本として過去 Pod を列挙・確認・復元できる tool surface が必要。
|
||||
|
||||
ただし、ホスト上の全 Pod state がどの Pod / LLM からも見える設計にはしない。Pod 管理 tool は capability / visibility scope を持ち、呼び出し元が見る権限を持つ Pod だけを列挙・操作できる必要がある。
|
||||
|
||||
## 要件
|
||||
|
||||
- 永続化された Pod state から、可視性 scope 内の既知 Pod 一覧を取得する tool / protocol API を追加する。
|
||||
- 実際の Method / tool 名は実装時に確定する。
|
||||
- `session-store` の Pod state backend/FsStore を正本にし、runtime dir の `spawned_pods.json` を正本にしない。
|
||||
- state が壊れている Pod や active segment 未確定の Pod は、全体失敗ではなく item 単位の状態として返せるようにする。
|
||||
- 呼び出し元に可視性がない Pod は列挙結果に含めない。
|
||||
- Pod 可視性の制御を設計する。
|
||||
- 少なくとも「現在の parent が spawn した child」と「明示的に指定された Pod 名」は扱えるようにする。
|
||||
- ホスト上の全 Pod を無条件に返す admin/global tool にはしない。
|
||||
- visibility の根拠は Pod state / parent-child registry / manifest capability / explicit user selection のいずれかに寄せ、実装時に確定する。
|
||||
- 可視でない Pod に対する detail / restore / attach は not visible / forbidden として、state missing とは区別する。
|
||||
- 一覧 item には最低限以下を含める。
|
||||
- `pod_name`
|
||||
- active `SessionId` / `SegmentId`(未確定ならその状態)
|
||||
- live socket / runtime が到達可能かどうか
|
||||
- restore 可能かどうかと、restore に必要な名前
|
||||
- spawned children が永続化されている場合は、その概要(件数や reachable 状態。詳細展開は別 API でもよい)
|
||||
- Pod 名指定で詳細を取得できる API を用意する。
|
||||
- active pointer
|
||||
- restoreability
|
||||
- live attach 可能性
|
||||
- spawned child registry の概要
|
||||
- 読めない state / 消えた socket / lock 衝突を区別したエラー
|
||||
- Pod 名指定で restore / attach を開始できる tool 導線を用意する。
|
||||
- live socket が到達可能なら attach 相当の扱いにする。
|
||||
- 到達不能だが Pod state があるなら既存の `--pod <name>` / `Pod::restore_from_pod_metadata(...)` 経路で restore する。
|
||||
- Pod state が存在しない名前を指定した場合に新規 Pod を作るか、明示エラーにするかは API ごとに曖昧にせず決める。探索・復元ツールとしては、意図しない新規作成を避けるため default はエラー寄りが望ましい。
|
||||
- 既存の `--pod <name>` / `--session <UUID>` / spawned child 向け `ListPods` / `SendToPod` / `StopPod` と責務を混ぜない。
|
||||
- `ListPods` は現在接続中の spawned child registry を見る用途として維持してよい。
|
||||
- 過去 Pod の探索 API は Pod state を正本にする。
|
||||
- live writer 二重起動防止、scope delegation、session lock の責務は既存 registry / lock に任せ、Pod state に lock 責務を追加しない。
|
||||
- tool result として LLM に返す情報は通常の tool call 履歴に残る形にし、history に残らない context 差し込みで実現しない。
|
||||
|
||||
## 完了条件
|
||||
|
||||
- 永続化済み Pod のうち、呼び出し元の可視性 scope 内にあるものだけを Pod state から列挙できる。
|
||||
- runtime dir の `spawned_pods.json` が存在しない状態でも、Pod state から可視 Pod を探索できる。
|
||||
- Pod 名指定で詳細を取得し、live attach 可能 / restore 可能 / state 不在 / state 破損 / lock 衝突を区別できる。
|
||||
- Pod 名指定の restore / attach tool が、到達可能 live Pod には attach し、到達不能だが state がある Pod には既存 restore 経路で復元できる。
|
||||
- 既存の `ListPods` / `SendToPod` / `StopPod` / `--pod` / `--session` の挙動を壊さない。
|
||||
- unit / integration test で以下を確認する。
|
||||
- 複数 Pod metadata の列挙(可視 Pod のみ)
|
||||
- 可視でない Pod が列挙されず、detail / restore / attach でも state missing と区別されること
|
||||
- active segment 未確定 Pod の表示
|
||||
- runtime file が消えても Pod state から探索できる
|
||||
- socket 到達可否の反映
|
||||
- restore / attach の分岐
|
||||
- lock 衝突時に二重 writer を起動しない
|
||||
- `cargo fmt --check`
|
||||
- `cargo check --workspace`
|
||||
- 関連 crate の tests(少なくとも `cargo test -p pod`。tool surface を置く crate に応じて追加)
|
||||
|
||||
## 範囲外
|
||||
|
||||
- 過去 Pod 一覧の本格 UI / picker(TUI 側の Pod picker は `tickets/tui-pod-restore-picker.md` で扱う)
|
||||
- fork tree の可視化
|
||||
- transcript 全文検索 / semantic search
|
||||
- Pod の自動再起動
|
||||
- 古い Pod state の GC / retention policy
|
||||
- session / segment 単位の新しい resume 引数
|
||||
|
||||
## 依存 / 関連
|
||||
|
||||
- Pod state backend / FsStore 実装
|
||||
- Pod lifecycle write-through
|
||||
- Pod 名単位の resume / attach 導線
|
||||
- SpawnedPodRegistry の永続化と復元
|
||||
|
|
@ -1,84 +0,0 @@
|
|||
# Pod: セッションログをバックエンドにした Pod 単位の永続化
|
||||
|
||||
## 背景
|
||||
|
||||
現在の永続化の主軸は session-store の append-only JSONL ログで、`SessionId` 単位に会話履歴・設定・scope snapshot・usage・拡張 payload を復元できる。一方で Pod 単位のランタイム状態は `<runtime_dir>/{pod_name}/` 配下の `status.json` / `history.json` / `spawned_pods.json` などに write-through されているが、runtime dir は再起動で消えてよい領域であり、Pod プロセスの寿命を超える復元ソースとしては扱えない。
|
||||
|
||||
特に spawned Pod の管理情報は `SpawnedPodRegistry` のコメントにもある通り、現状は runtime dir への write-through のみで、再起動した spawner が子 Pod 一覧を rebuild する future work になっている。
|
||||
|
||||
このチケットでは、既存の session-store を物理バックエンドとして利用しつつ、Pod 名をキーにした永続状態を追加し、Pod 単位で「最後にどの session を保持していたか」「spawned children をどう復元するか」を扱えるようにする。
|
||||
|
||||
## 方針
|
||||
|
||||
- session log は引き続き会話状態の唯一の復元ソースにする。
|
||||
- `history.json` や runtime dir の snapshot を永続正本にはしない。
|
||||
- LLM context に載せる新規 input は、既存方針通り先に worker history / session log に commit されている必要がある。
|
||||
- Pod 単位の永続化は「Pod identity → session / child registry などへの参照」を保存する薄いメタデータ層として設計する。
|
||||
- 会話本文を二重保存しない。
|
||||
- active session だけでなく、compaction / fork / resume によってその Pod が辿ってきた過去 session を順序付きで保持する。これは UI の履歴表示、直近以前への復元、active session 変更の監査に使う。
|
||||
- session-store の `Store` trait を拡張するか、隣接 trait / module を追加して、FsStore 以外の backend でも同じ形で実装できるようにする。
|
||||
- FsStore のデフォルト layout は `<data_dir>/pods/` 配下など、`sessions/` と同じ data_dir 管理下に置く。
|
||||
- runtime dir (`<runtime_dir>/{pod_name}/`) は引き続き socket / pid / status など一時状態専用。
|
||||
- Pod lifecycle 上の write point を明確にする。
|
||||
- Pod 作成時: pod name と allocated session id を記録。
|
||||
- first run で `SessionStart` が materialize された後: active session / head を更新できる状態にする。
|
||||
- compaction / fork / resume で active session が変わる場合: Pod state も同時に更新。
|
||||
- `SpawnPod` / callback / `StopPod` による child registry 変更時: runtime dir だけでなく persistent Pod state にも write-through。
|
||||
- 復元時は Pod state から active session を解決し、その session log を `restore_from_manifest` 相当の経路で復元する。
|
||||
- session id を明示した resume は既存通り session を直接指定できる。
|
||||
- Pod 名 resume は Pod state → active session → session restore の順に解決する。
|
||||
- live writer 衝突は既存の pod-registry / session_id collision check を維持する。
|
||||
|
||||
## データ粒度の考え方
|
||||
|
||||
- ユーザー視点の会話継続単位と、内部の append-only log 単位を分けて扱う。
|
||||
- ユーザー視点: Pod / thread / conversation のような安定 ID。compaction しても同じ会話として継続する。
|
||||
- 内部 log 視点: session segment / revision / epoch のような履歴再構築単位。compaction や fork で新しい log root が必要なら新 ID になる。
|
||||
- 現状の `SessionId` は内部 log 単位の性質が強い。compaction は履歴を要約済み prefix に置き換えて新しい append-only chain を始めるため、低レベルには「新 session」として扱うのは自然。ただし UX / データモデル上は「同じ Pod conversation の新 revision」と見せる。
|
||||
- 将来 DB backend を追加する場合も、`Conversation/PodState` と `SessionSegment` を分ける形に寄せる。
|
||||
- `pod_state.active_session_id` は現在 append 先の segment を指す。
|
||||
- `pod_state.session_history[]` は Pod 視点で active だった segment の順序付き履歴。
|
||||
- compaction / fork の構造的 lineage は session log の `SessionOrigin` または DB の relation として保持し、Pod state は「この Pod がどれを active にしたか」の操作履歴に留める。
|
||||
|
||||
## 要件
|
||||
|
||||
- Pod 名をキーに、少なくとも以下を永続化できること:
|
||||
- active `SessionId`
|
||||
- ordered session history: その Pod が active として保持してきた `SessionId` の時系列リスト
|
||||
- 各 entry には最低限 `session_id` と遷移理由(new / resume / compact / fork など)を持たせる
|
||||
- compaction / fork の構造的な出自は session log の `SessionOrigin` を正本とし、Pod state 側は Pod 視点の active session 遷移履歴として扱う
|
||||
- Pod manifest / scope 復元に必要な参照または snapshot の扱い(既存 session log の `pod.scope` snapshot と責務を重複させない)
|
||||
- spawned children の registry(pod name, socket path, delegated scope, callback address, child session id が必要なら含める)
|
||||
- `SpawnedPodRegistry` が runtime dir の `spawned_pods.json` だけでなく、Pod 永続状態から初期化できること。
|
||||
- `ListPods` / `SendToPod` / `ReadPodOutput` / `StopPod` は、復元後の spawner でも永続化された child registry を基に動作できること。
|
||||
- ただし `ReadPodOutput` の read cursor は session-lifetime / in-memory のままでよい。永続化対象にしない。
|
||||
- Pod の compaction により active session id が変わった場合、Pod 永続状態と pod-registry の session id が整合すること。
|
||||
- 既存の `--session <UUID>` resume は壊さない。
|
||||
- 新しい Pod 名単位 resume / attach の入口を決めること。
|
||||
- 例: `pod --pod-state <name>` ではなく、既存 `pod.name` と manifest cascade から同名 Pod state を探す形など。
|
||||
- CLI / TUI の最小導線を本チケット内で確定する。
|
||||
|
||||
## 完了条件
|
||||
|
||||
- `session-store` に Pod 単位メタデータを扱う backend API と FsStore 実装がある。
|
||||
- Pod state が active session と ordered session history を保持し、new / resume / compaction / fork の遷移が順序付きで記録される。
|
||||
- 新規 Pod 起動、resume、compaction、spawn / stop の各タイミングで Pod 永続状態が更新される。
|
||||
- Pod プロセス再起動後、Pod 名から active session を復元し、会話を継続できる。
|
||||
- spawner Pod の再起動後、永続化された spawned children 一覧から `ListPods` が復元され、到達可能な child に対して comm tools が使える。
|
||||
- runtime dir は引き続き一時状態として扱われ、永続正本に依存しない。
|
||||
- live writer の二重起動は既存 pod-registry / session lock と同等以上に防止される。
|
||||
|
||||
## 範囲外
|
||||
|
||||
- 会話履歴そのものの保存形式変更。
|
||||
- session log の DB 化や remote backend 実装。
|
||||
- Pod state の自動 GC / retention policy。
|
||||
- TUI 上の高度な Pod 一覧 UI。最小限の resume / attach 導線を超える UX は別チケット。
|
||||
- `ReadPodOutput` cursor の永続化。
|
||||
|
||||
## 関連
|
||||
|
||||
- `crates/session-store/`: 既存の session append-only backend。
|
||||
- `crates/pod/src/runtime/dir.rs`: runtime dir の `history.json` / `spawned_pods.json`。
|
||||
- `crates/pod/src/spawn/registry.rs`: spawned children registry。現状は write-through のみで復元未実装。
|
||||
- `tickets/pod-session-fork.md`: active session 切り替え設計との整合が必要。
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user