Compare commits

..

23 Commits

Author SHA1 Message Date
59bf20f2cd
Merge branch 'tui-knowledge-completion' into develop 2026-05-12 15:43:29 +09:00
f7f59dd30c
docs(memory): fix knowledge dir path in collect_resident_knowledge doc 2026-05-12 15:07:39 +09:00
806440ac7a
docs(tickets): review tui knowledge completion (approve) 2026-05-12 14:56:30 +09:00
7b8eb3af8d
feat(pod): wire knowledge slugs into # completion 2026-05-12 14:45:46 +09:00
705c873097
docs(tickets): tui knowledge completion unimplemented fix 2026-05-12 14:40:37 +09:00
ae6c27a5c7
docs(tickets): define work item query strategy 2026-05-12 02:32:32 +09:00
03a577527a
docs(tickets): use timestamp work item ids 2026-05-12 02:07:29 +09:00
b4dff2835e
docs: add ai maintainer work item plan 2026-05-12 01:53:52 +09:00
e87a515474
docs(tickets): add lint-common crate ticket 2026-05-12 00:06:06 +09:00
9df6bd5fcb
merge: workflow crate extraction 2026-05-11 22:50:19 +09:00
6610ef8150
docs(tickets): complete workflow crate extraction 2026-05-11 22:50:06 +09:00
7159a66a60
review: workflow crate extraction 2026-05-11 22:49:50 +09:00
7db4146f3d
refactor: extract workflow crate 2026-05-11 22:49:07 +09:00
d8f29bcbcb
merge: anthropic assistant burst bundling 2026-05-11 22:24:36 +09:00
f444b387be
docs(tickets): complete anthropic assistant burst bundling 2026-05-11 22:23:53 +09:00
d18f536945
review: anthropic assistant burst bundling 2026-05-11 22:23:38 +09:00
19badfe8b7
fix: bundle anthropic assistant bursts 2026-05-11 22:22:36 +09:00
31f94bf791
merge: memory usage metrics 2026-05-11 21:46:24 +09:00
ac09bfcc21
docs(tickets): complete memory usage metrics 2026-05-11 21:46:19 +09:00
76f83a0894
review: memory usage metrics 2026-05-11 21:46:19 +09:00
d581a35426
feat: add memory usage event metrics 2026-05-11 21:29:48 +09:00
f69aa469f8
docs(tickets): complete memory phase naming cleanup 2026-05-11 17:16:36 +09:00
3d4d83db68
docs(tickets): simplify memory usage metrics 2026-05-11 16:54:23 +09:00
47 changed files with 2500 additions and 675 deletions

16
Cargo.lock generated
View File

@ -2164,6 +2164,7 @@ dependencies = [
"tools",
"tracing",
"uuid",
"workflow",
]
[[package]]
@ -4398,6 +4399,21 @@ dependencies = [
"wasmparser",
]
[[package]]
name = "workflow"
version = "0.1.0"
dependencies = [
"chrono",
"manifest",
"memory",
"serde",
"serde_json",
"serde_yaml",
"tempfile",
"thiserror 2.0.18",
"tracing",
]
[[package]]
name = "writeable"
version = "0.6.3"

View File

@ -15,6 +15,7 @@ members = [
"crates/tools",
"crates/tui",
"crates/memory",
"crates/workflow",
]
[workspace.package]
@ -28,6 +29,7 @@ llm-worker = { path = "crates/llm-worker", version = "0.2" }
llm-worker-macros = { path = "crates/llm-worker-macros", version = "0.2" }
manifest = { path = "crates/manifest" }
memory = { path = "crates/memory" }
workflow = { path = "crates/workflow" }
pod-registry = { path = "crates/pod-registry" }
protocol = { path = "crates/protocol" }
provider = { path = "crates/provider" }

View File

@ -1,8 +1,9 @@
- Workflow / Skills
- 内部 Worker / 内部 Pod の Workflow 化 → [tickets/internal-worker-workflow.md](tickets/internal-worker-workflow.md)
- 半自動開発運用 Workflow → [tickets/auto-maintain-workflow.md](tickets/auto-maintain-workflow.md)
- Workflow を memory crate から独立させる → [tickets/workflow-crate-extraction.md](tickets/workflow-crate-extraction.md)
- AI maintainer 用 WorkItem / Thread 抽象 → [tickets/maintainer-work-items.md](tickets/maintainer-work-items.md)
- Prompt / Workflow 評価メトリクスと改善 Offer → [tickets/prompt-eval-metrics.md](tickets/prompt-eval-metrics.md)
- memory / workflow 共通基盤Slug / frontmatter helpersを別 crate に切り出す → [tickets/lint-common-crate.md](tickets/lint-common-crate.md)
- Permission: allow-all 既定 policy への整理 → [tickets/permission-default-policy.md](tickets/permission-default-policy.md)
- Pod CLI: マニフェスト関連フラグの整理 → [tickets/pod-cli-manifest-flags.md](tickets/pod-cli-manifest-flags.md)
- Pod: 空応答ターン (Submit 後 AI 応答ゼロで Pause/Cancel) を自動巻き戻し → [tickets/pod-empty-turn-rollback.md](tickets/pod-empty-turn-rollback.md)
@ -13,7 +14,6 @@
- Exchange / Turn / Call セマンティクス整理 → [tickets/exchange-turn-call-semantics.md](tickets/exchange-turn-call-semantics.md)
- llm-worker のエラー耐性
- ストリーム途中失敗時の継続 → [tickets/llm-worker-stream-continuation.md](tickets/llm-worker-stream-continuation.md)
- llm-worker: Anthropic projection で assistant ターン内ブロックを 1 message に束ねる → [tickets/anthropic-assistant-burst-bundling.md](tickets/anthropic-assistant-burst-bundling.md)
- ネイティブ GUI クライアント MVP → [tickets/native-gui-mvp.md](tickets/native-gui-mvp.md)
- E2E テストハーネス(`tests/e2e/`、opt-in → [tickets/e2e-harness.md](tickets/e2e-harness.md)
- TUI 拡充
@ -25,10 +25,9 @@
- Manifest: Tool Output / File Upload 上限の分離とデフォルト緩和 → [tickets/manifest-output-upload-limits.md](tickets/manifest-output-upload-limits.md)
- Prune: 保護境界を turn 数から末尾 token budget に置き換え → [tickets/prune-token-budget.md](tickets/prune-token-budget.md)
- メモリ機構
- 使用頻度メトリクス + Knowledge 化候補レポート → [tickets/memory-usage-metrics.md](tickets/memory-usage-metrics.md)
- Phase 1/2 呼称を extract/consolidation に統一 → [tickets/memory-phase-naming.md](tickets/memory-phase-naming.md)
- extract / consolidation 監査ログ → [tickets/memory-audit-log.md](tickets/memory-audit-log.md)
- セッション内 Task ツールの注意機構(無アクティビティで `<system-reminder>` ナッジ) → [tickets/session-todo-reminder.md](tickets/session-todo-reminder.md)
- ワークスペースのメモリーをLintするヘッドレスCLI
- system-reminder 注入機構の汎用化2件目の利用者が出た時に検討。タグ形式 `<system-reminder>...</system-reminder>` の規約は session-todo-reminder で先行確立。注入された Item は worker.history に append する方針)
- Bashツールがファイル編集に常用されている問題をdesciptionで抑制
- 事前定義したManifestをProfile的に扱い、Orchestrator/Coder/Researcherで別々のモデル/設定を使わせる運用ができるようにする

View File

@ -242,10 +242,13 @@ impl AnthropicScheme {
/// - Tool calls are content parts within assistant messages
/// - Tool results are content parts within user messages
///
/// Each non-`Message` item produces exactly one content part, so
/// "last part for the item" is always well-defined. For breakpoint
/// `Message` items the output is forced into the array form so a
/// marker has a part to attach to.
/// Assistant-side items are accumulated until a user/system message or
/// tool result boundary so one logical assistant burst becomes one
/// Anthropic assistant message content array. Pending parts carry their
/// origin item index; when flushed, the final part for each item records
/// the `(msg_idx, part_idx)` used by breakpoint attachment. User/system
/// `Message` items keep the single-text shorthand unless a breakpoint
/// needs a concrete part to live on.
fn convert_items_to_messages(
&self,
items: &[Item],
@ -261,19 +264,6 @@ impl AnthropicScheme {
for (i, item) in items.iter().enumerate() {
match item {
Item::Message { role, content, .. } => {
flush_pending(
&mut messages,
&mut pending_assistant,
"assistant",
&mut locations,
);
flush_pending(&mut messages, &mut pending_user, "user", &mut locations);
let anthropic_role = match role {
Role::User | Role::System => "user",
Role::Assistant => "assistant",
};
let parts: Vec<AnthropicContentPart> = content
.iter()
.map(|p| match p {
@ -284,27 +274,43 @@ impl AnthropicScheme {
})
.collect();
let force_parts = breakpoints.contains(&i);
let msg_idx = messages.len();
match role {
Role::Assistant => {
flush_pending(&mut messages, &mut pending_user, "user", &mut locations);
pending_assistant.extend(parts.into_iter().map(|part| (i, part)));
}
Role::User | Role::System => {
flush_pending(
&mut messages,
&mut pending_assistant,
"assistant",
&mut locations,
);
flush_pending(&mut messages, &mut pending_user, "user", &mut locations);
// Preserve the single-text shorthand unless a
// breakpoint needs a concrete part to live on.
if parts.len() == 1 && !force_parts {
if let AnthropicContentPart::Text { text, .. } = &parts[0] {
let force_parts = breakpoints.contains(&i);
let msg_idx = messages.len();
// Preserve the single-text shorthand unless a
// breakpoint needs a concrete part to live on.
if parts.len() == 1 && !force_parts {
if let AnthropicContentPart::Text { text, .. } = &parts[0] {
messages.push(AnthropicMessage {
role: "user".to_string(),
content: AnthropicContent::Text(text.clone()),
});
continue;
}
}
let last_part_idx = parts.len().saturating_sub(1);
messages.push(AnthropicMessage {
role: anthropic_role.to_string(),
content: AnthropicContent::Text(text.clone()),
role: "user".to_string(),
content: AnthropicContent::Parts(parts),
});
continue;
locations[i] = Some((msg_idx, last_part_idx));
}
}
let last_part_idx = parts.len().saturating_sub(1);
messages.push(AnthropicMessage {
role: anthropic_role.to_string(),
content: AnthropicContent::Parts(parts),
});
locations[i] = Some((msg_idx, last_part_idx));
}
Item::ToolCall {
@ -626,6 +632,109 @@ mod tests {
out
}
#[test]
fn assistant_burst_bundles_reasoning_text_and_tool_call() {
let scheme = AnthropicScheme::new();
let request = Request::new()
.user("question?")
.item(Item::reasoning("thinking").with_signature("SIG-A"))
.item(Item::assistant_message("answer"))
.item(Item::tool_call("c1", "tool_a", r#"{"x":1}"#));
let req = scheme.build_request("claude-sonnet-4-20250514", &request, &cap_explicit());
assert_eq!(req.messages.len(), 2, "messages: {:?}", req.messages);
assert_eq!(req.messages[0].role, "user");
assert_eq!(req.messages[1].role, "assistant");
let AnthropicContent::Parts(parts) = &req.messages[1].content else {
panic!("assistant burst must be emitted as content parts");
};
assert_eq!(parts.len(), 3, "parts: {:?}", parts);
assert!(matches!(parts[0], AnthropicContentPart::Thinking { .. }));
assert!(matches!(parts[1], AnthropicContentPart::Text { .. }));
assert!(matches!(parts[2], AnthropicContentPart::ToolUse { .. }));
}
#[test]
fn tool_result_and_user_messages_bound_assistant_bursts() {
let scheme = AnthropicScheme::new();
let request = Request::new()
.user("question?")
.item(Item::reasoning("thinking").with_signature("SIG-A"))
.item(Item::assistant_message("answer"))
.item(Item::tool_call("c1", "tool_a", "{}"))
.item(Item::tool_result("c1", "result"))
.item(Item::assistant_message("final"))
.user("follow up");
let req = scheme.build_request("claude-sonnet-4-20250514", &request, &cap_explicit());
let roles: Vec<&str> = req.messages.iter().map(|msg| msg.role.as_str()).collect();
assert_eq!(
roles,
vec!["user", "assistant", "user", "assistant", "user"]
);
let AnthropicContent::Parts(first_assistant) = &req.messages[1].content else {
panic!("first assistant burst must be content parts");
};
assert_eq!(first_assistant.len(), 3);
assert!(matches!(
first_assistant[0],
AnthropicContentPart::Thinking { .. }
));
assert!(matches!(
first_assistant[1],
AnthropicContentPart::Text { .. }
));
assert!(matches!(
first_assistant[2],
AnthropicContentPart::ToolUse { .. }
));
let AnthropicContent::Parts(tool_result) = &req.messages[2].content else {
panic!("tool result must be content parts");
};
assert_eq!(tool_result.len(), 1);
assert!(matches!(
tool_result[0],
AnthropicContentPart::ToolResult { .. }
));
let AnthropicContent::Parts(second_assistant) = &req.messages[3].content else {
panic!("second assistant burst must be content parts");
};
assert_eq!(second_assistant.len(), 1);
assert!(matches!(
second_assistant[0],
AnthropicContentPart::Text { .. }
));
}
#[test]
fn assistant_message_breakpoint_maps_to_text_part_inside_burst() {
let scheme = AnthropicScheme::new();
let mut request = Request::new().items(vec![
Item::user_message("question?"),
Item::reasoning("thinking").with_signature("SIG-A"),
Item::assistant_message("answer"),
Item::tool_call("c1", "tool_a", "{}"),
Item::user_message("next"),
]);
request.cache_anchor = Some(2);
let req = scheme.build_request("claude-sonnet-4-20250514", &request, &cap_explicit());
let AnthropicContent::Parts(parts) = &req.messages[1].content else {
panic!("assistant burst must be content parts");
};
assert!(matches!(parts[0], AnthropicContentPart::Thinking { .. }));
assert!(matches!(parts[1], AnthropicContentPart::Text { .. }));
assert!(matches!(parts[2], AnthropicContentPart::ToolUse { .. }));
assert_eq!(part_cache_control(&parts[1]), Some(CacheControl::Ephemeral));
assert_eq!(part_cache_control(&parts[2]), Some(CacheControl::Ephemeral));
}
/// Convenience: a turn that ends with one assistant text, one tool
/// call/result pair, and a final assistant text. Produced at
/// `history[head..]` indices shown alongside, so tests can reason

View File

@ -6,8 +6,8 @@
//!
//! 1. consumed staging エントリ全文(`source` 込み)
//! 2. 既存 `memory/*` 全文summary / decisions / requests
//! 3. Knowledge 化候補レポート(メトリクス未完なら空
//! 4. 整理材料Linter Warn ベース、メトリクス未完なら明示 invoke 頻度なし
//! 3. Usage evidence report明示使用回数 + resident exposure cost
//! 4. 整理材料Linter Warn ベース、hard protection 判定はしない
//!
//! 既存 `knowledge/*` 本文は埋めず、agent に `KnowledgeQuery` 経由で引かせる
//! 設計(`docs/plan/memory.md` §retrieval 経路 / §Consolidation の Knowledge アクセス)。
@ -16,41 +16,15 @@ use std::fmt::Write;
use crate::consolidate::staging::StagingEntry;
use crate::consolidate::tidy::TidyHints;
use crate::usage::UsageReport;
use crate::workspace::{RecordKind, WorkspaceLayout};
/// Knowledge 化候補レポート。`tickets/memory-usage-metrics.md` の成果物が
/// 出るまでは空で渡す前提(`docs/plan/memory.md` §Knowledge 化候補レポート)。
/// 空入力時、統合 step は新規 Knowledge を作らず decisions / requests /
/// summary / 既存 Knowledge update に留まる。
#[derive(Debug, Default, Clone)]
pub struct KnowledgeCandidateReport {
/// 候補に上がった `(kind, slug, frequency_per_mtoken)` の三つ組。
/// 空配列を渡すと「候補なし」を意味する。
pub entries: Vec<KnowledgeCandidateEntry>,
}
#[derive(Debug, Clone)]
pub struct KnowledgeCandidateEntry {
pub source_kind: &'static str,
pub source_slug: String,
pub frequency_per_mtoken: f64,
}
impl KnowledgeCandidateReport {
pub fn empty() -> Self {
Self::default()
}
pub fn is_empty(&self) -> bool {
self.entries.is_empty()
}
}
/// consolidation sub-Worker の最初の user 入力。
pub fn build_consolidate_input(
layout: &WorkspaceLayout,
staging: &[StagingEntry],
tidy: &TidyHints,
candidates: &KnowledgeCandidateReport,
usage_report: &UsageReport,
) -> String {
let mut out = String::new();
out.push_str(
@ -68,8 +42,8 @@ pub fn build_consolidate_input(
out.push_str(&render_existing_memory_records(layout));
out.push('\n');
out.push_str("## Knowledge candidate report\n\n");
out.push_str(&render_candidate_report(candidates));
out.push_str("## Usage evidence report\n\n");
out.push_str(&render_usage_report(usage_report));
out.push('\n');
out.push_str("## Tidy hints\n\n");
@ -159,21 +133,16 @@ fn push_kind_records(out: &mut String, layout: &WorkspaceLayout, kind: RecordKin
}
}
fn render_candidate_report(report: &KnowledgeCandidateReport) -> String {
fn render_usage_report(report: &UsageReport) -> String {
if report.is_empty() {
return "(empty — usage metrics pipeline not populated. \
Do not create new Knowledge records this run.)\n"
return "(empty — no explicit memory/knowledge usage events recorded yet. \
Treat this as lack of evidence, not proof that records are unused.)\n"
.to_string();
}
let mut out = String::new();
for c in &report.entries {
let _ = writeln!(
&mut out,
"- {} `{}` — frequency {:.3} invokes/Mtoken",
c.source_kind, c.source_slug, c.frequency_per_mtoken
);
}
out
let json = serde_json::to_string_pretty(report).unwrap_or_else(|_| "{}".to_string());
format!(
"This report is evidence only. Do not make hard Knowledge-creation or tidy-protection decisions from it alone.\n\n```json\n{json}\n```\n"
)
}
/// Tidy hints の Markdown 描画。空ヒントなら "(none)" 1 行。
@ -229,8 +198,8 @@ pub fn render_tidy_hints(tidy: &TidyHints) -> String {
}
out.push_str(
"Explicit-invoke metrics (protection threshold) are not yet wired up; \
skip drop on long-standing records when uncertain.\n",
"Use the Usage evidence report as soft context only; \
require an explicit reason before deleting or heavily compressing records with recent use.\n",
);
out
}
@ -295,31 +264,27 @@ mod tests {
slugs: vec!["a".into(), "ab".into()],
}],
};
let report = KnowledgeCandidateReport::empty();
let report = UsageReport::empty();
let out = build_consolidate_input(&layout, &staging, &tidy, &report);
assert!(out.contains("Staging entries"));
assert!(out.contains("Existing memory records"));
assert!(out.contains("Knowledge candidate report"));
assert!(out.contains("Usage evidence report"));
assert!(out.contains("Tidy hints"));
assert!(out.contains("state of the world"));
assert!(out.contains("decision:dec"));
assert!(out.contains("Replaced decisions"));
assert!(out.contains("Sources overflow"));
assert!(out.contains("Similar slug clusters"));
assert!(out.contains("usage metrics pipeline not populated"));
assert!(out.contains("no explicit memory/knowledge usage events"));
}
#[test]
fn empty_inputs_render_placeholders() {
let dir = tempfile::TempDir::new().unwrap();
let layout = WorkspaceLayout::new(dir.path().to_path_buf());
let out = build_consolidate_input(
&layout,
&[],
&TidyHints::default(),
&KnowledgeCandidateReport::empty(),
);
let out =
build_consolidate_input(&layout, &[], &TidyHints::default(), &UsageReport::empty());
// Both staging and tidy show "(none)"; existing memory records too.
assert!(out.contains("Staging entries"));
assert!(out.contains("(none)"));

View File

@ -12,9 +12,8 @@
//! consumed ID 分の staging のみ削除し、占有ファイルを解放
//!
//! system prompt は Pod の `PromptCatalog`
//! (`PodPrompt::MemoryConsolidationSystem`) で管理される。Knowledge 化候補
//! レポートと使用頻度メトリクスは別チケットで供給される想定。本モジュール
//! 時点では空入力として扱い、prompt 側の説明だけ残しておく
//! (`PodPrompt::MemoryConsolidationSystem`) で管理される。Usage report は
//! 判断材料として渡すだけで、ここでは Knowledge 化や protection の hard decision はしない
//! `docs/plan/memory.md` §Consolidation / 整理材料)。
mod input;
@ -23,8 +22,8 @@ mod staging;
mod tidy;
pub use input::{
KnowledgeCandidateReport, build_consolidate_input, render_existing_memory_records,
render_staging_records, render_tidy_hints,
build_consolidate_input, render_existing_memory_records, render_staging_records,
render_tidy_hints,
};
pub use lock::{LockError, LockRecord, StagingLock};
pub use staging::{StagingEntry, list_staging_entries};

View File

@ -69,11 +69,6 @@ pub enum LintError {
#[error("body exceeds the size limit for this record kind: {actual} chars > {limit}")]
BodyTooLong { actual: usize, limit: usize },
#[error(
"write to a Workflow path is forbidden via the memory tool — Workflows are human-edited"
)]
WorkflowWriteForbidden,
#[error("slug `{0}` already exists; use the edit tool instead of creating a new record")]
SlugAlreadyExists(String),

View File

@ -13,21 +13,20 @@ pub mod linter;
pub mod resident;
pub mod schema;
pub mod scope;
pub mod skill;
pub mod slug;
pub mod tool;
pub mod workflow;
pub mod usage;
pub mod workspace;
pub use error::{LintError, LintWarning, MemoryError};
pub use extract::ExtractPointerPayload;
pub use linter::{LintReport, Linter};
pub use resident::{ResidentKnowledgeEntry, collect_resident_knowledge};
pub use resident::{ResidentKnowledgeEntry, collect_resident_knowledge, list_knowledge_slugs};
pub use scope::deny_write_rules;
pub use skill::{SKILL_FILENAME, SkillParseError, SkillRecord, load_skills_from_dir, parse_skill_md};
pub use slug::Slug;
pub use workflow::{
ResidentWorkflowEntry, ShadowedSkill, WORKFLOW_DESCRIPTION_HARD_CAP, WorkflowLoadError,
WorkflowRecord, WorkflowRegistry, WorkflowSource, load_workflows,
pub use usage::{
UsageEvent, UsageEventKind, UsageRecordSnapshot, UsageReport, UsageReportRecord, UsageSource,
append_resident_exposure_event, append_usage_event, append_use_event, build_usage_report,
snapshot_record_from_bytes, snapshot_record_from_layout,
};
pub use workspace::WorkspaceLayout;

View File

@ -1,5 +1,4 @@
//! Walks `<workspace>/memory/{decisions,requests}/`,
//! `<workspace>/workflow/`, and `<workspace>/knowledge/` to collect
//! Walks `<workspace>/memory/{decisions,requests}/` and `<workspace>/knowledge/` to collect
//! the slug set the linter needs for reference-integrity and
//! same-slug-duplication checks.
//!
@ -11,8 +10,7 @@ use std::io;
use std::path::Path;
use crate::schema::{
DecisionFrontmatter, KnowledgeFrontmatter, RequestFrontmatter, WorkflowFrontmatter,
split_frontmatter,
DecisionFrontmatter, KnowledgeFrontmatter, RequestFrontmatter, split_frontmatter,
};
use crate::slug::Slug;
use crate::workspace::{RecordKind, WorkspaceLayout};
@ -28,7 +26,6 @@ pub struct ExistingRecords {
decisions: HashMap<Slug, DecisionMeta>,
requests: HashSet<Slug>,
knowledge: HashSet<Slug>,
workflow: HashSet<Slug>,
}
#[derive(Debug, Clone)]
@ -42,7 +39,7 @@ impl ExistingRecords {
RecordKind::Decision => self.decisions.contains_key(slug),
RecordKind::Request => self.requests.contains(slug),
RecordKind::Knowledge => self.knowledge.contains(slug),
RecordKind::Workflow => self.workflow.contains(slug),
RecordKind::Workflow => false,
RecordKind::Summary => false,
}
}
@ -56,7 +53,7 @@ impl ExistingRecords {
RecordKind::Decision => self.decisions.keys().collect(),
RecordKind::Request => self.requests.iter().collect(),
RecordKind::Knowledge => self.knowledge.iter().collect(),
RecordKind::Workflow => self.workflow.iter().collect(),
RecordKind::Workflow => Vec::new(),
RecordKind::Summary => Vec::new(),
}
}
@ -82,10 +79,6 @@ pub fn scan_existing(layout: &WorkspaceLayout) -> io::Result<ExistingRecords> {
let _ = parse_silent::<KnowledgeFrontmatter>(path);
out.knowledge.insert(slug);
})?;
scan_dir(&layout.workflow_dir(), |path, slug| {
let _ = parse_silent::<WorkflowFrontmatter>(path);
out.workflow.insert(slug);
})?;
Ok(out)
}

View File

@ -23,9 +23,8 @@ use serde::de::DeserializeOwned;
use crate::error::{LintError, LintWarning};
use crate::schema::{
DecisionFrontmatter, KnowledgeFrontmatter, RequestFrontmatter, SummaryFrontmatter,
WorkflowFrontmatter, split_frontmatter,
split_frontmatter,
};
use crate::workflow::WORKFLOW_DESCRIPTION_HARD_CAP;
use crate::workspace::{ClassifiedPath, RecordKind, WorkspaceLayout};
pub use existing::{ExistingRecords, scan_existing};
@ -99,12 +98,6 @@ impl Linter {
}
};
// 2. Workflow paths are sub-Worker-forbidden at the tool layer.
if classified.kind == RecordKind::Workflow {
report.push_error(LintError::WorkflowWriteForbidden);
return report;
}
// 3. Frontmatter parse + kind-specific structural checks +
// size limits. Reference-integrity needs the existing
// record set, fetched once below.
@ -146,7 +139,9 @@ impl Linter {
RecordKind::Summary => {
self.check_kind::<SummaryFrontmatter>(content, &classified, &mut report);
}
RecordKind::Workflow => unreachable!("guarded above"),
RecordKind::Workflow => {
unreachable!("workflow paths are not classified by memory linter")
}
}
report
@ -240,59 +235,6 @@ impl Linter {
}
}
impl Linter {
/// Workflow record validator exposed for human-edit paths
/// (CLI / pre-commit). Not used by the memory tool, which rejects
/// workflow writes outright.
///
/// Verifies frontmatter shape, body size, and that every slug in
/// `requires` points at an existing Knowledge record under the
/// workspace's `knowledge/` directory.
pub fn lint_workflow(&self, content: &str) -> LintReport {
let mut report = LintReport::default();
let parsed = match parse_frontmatter::<WorkflowFrontmatter>(content) {
Ok(p) => p,
Err(e) => {
report.push_error(e);
return report;
}
};
size::check_body::<WorkflowFrontmatter>(parsed.body, &mut report);
// Mirror the loader's cap so human-edit paths fail fast instead
// of surfacing the same error only at Pod startup.
if parsed.frontmatter.model_invokation {
let actual = parsed.frontmatter.description.chars().count();
if actual > WORKFLOW_DESCRIPTION_HARD_CAP {
report.push_error(LintError::DescriptionTooLong {
actual,
limit: WORKFLOW_DESCRIPTION_HARD_CAP,
});
}
}
let existing = match existing::scan_existing(&self.layout) {
Ok(e) => e,
Err(e) => {
report.push_error(LintError::MalformedFrontmatter(format!(
"failed to scan existing records: {e}"
)));
return report;
}
};
for slug in &parsed.frontmatter.requires {
if !existing.contains(crate::workspace::RecordKind::Knowledge, slug) {
report.push_error(LintError::UnknownReference {
field: "requires",
kind: "knowledge",
slug: slug.to_string(),
});
}
}
report
}
}
struct Parsed<'a, F> {
frontmatter: F,
body: &'a str,
@ -332,22 +274,6 @@ mod tests {
(dir, linter)
}
#[test]
fn workflow_write_rejected() {
let (dir, linter) = workspace();
let path = dir.path().join(".insomnia/workflow/wf.md");
let content =
"---\ndescription: x\nmodel_invokation: false\nuser_invocable: true\n---\nbody"
.to_string();
let report = linter.lint(&path, &content, WriteMode::Create);
assert!(
report
.errors
.iter()
.any(|e| matches!(e, LintError::WorkflowWriteForbidden))
);
}
#[test]
fn outside_memory_tree_rejected() {
let (dir, linter) = workspace();
@ -499,83 +425,6 @@ mod tests {
);
}
#[test]
fn workflow_lint_accepts_valid_record() {
let (dir, linter) = workspace();
// Place a Knowledge record that the workflow will reference.
let kn = dir.path().join(".insomnia/knowledge/foo.md");
write(
&kn,
&format!(
"---\ncreated_at: {n}\nupdated_at: {n}\nkind: rule\ndescription: x\nmodel_invokation: false\nuser_invocable: true\nlast_sources: []\n---\n",
n = iso_now()
),
);
let wf = "---\ndescription: do thing\nmodel_invokation: false\nuser_invocable: true\nrequires: [foo]\n---\nstep 1\n".to_string();
let report = linter.lint_workflow(&wf);
assert!(!report.has_errors(), "got errors: {:?}", report.errors);
}
#[test]
fn workflow_lint_flags_unknown_requires() {
let (_dir, linter) = workspace();
let wf = "---\ndescription: x\nmodel_invokation: false\nuser_invocable: true\nrequires: [missing-knowledge]\n---\n".to_string();
let report = linter.lint_workflow(&wf);
assert!(report.errors.iter().any(|e| matches!(
e,
LintError::UnknownReference {
field: "requires",
kind: "knowledge",
..
}
)));
}
#[test]
fn workflow_lint_flags_long_description_when_model_invokation() {
let (_dir, linter) = workspace();
let desc = "x".repeat(crate::workflow::WORKFLOW_DESCRIPTION_HARD_CAP + 1);
let wf = format!(
"---\ndescription: {desc}\nmodel_invokation: true\nuser_invocable: true\n---\n"
);
let report = linter.lint_workflow(&wf);
assert!(
report
.errors
.iter()
.any(|e| matches!(e, LintError::DescriptionTooLong { .. })),
);
}
#[test]
fn workflow_lint_allows_long_description_when_not_model_invokation() {
let (_dir, linter) = workspace();
let desc = "x".repeat(crate::workflow::WORKFLOW_DESCRIPTION_HARD_CAP + 1);
let wf = format!(
"---\ndescription: {desc}\nmodel_invokation: false\nuser_invocable: true\n---\n"
);
let report = linter.lint_workflow(&wf);
assert!(
!report
.errors
.iter()
.any(|e| matches!(e, LintError::DescriptionTooLong { .. })),
);
}
#[test]
fn workflow_lint_collects_multiple_unknown_requires() {
let (_dir, linter) = workspace();
let wf = "---\ndescription: x\nmodel_invokation: false\nuser_invocable: true\nrequires: [a, b, c]\n---\n".to_string();
let report = linter.lint_workflow(&wf);
let unknown_count = report
.errors
.iter()
.filter(|e| matches!(e, LintError::UnknownReference { .. }))
.count();
assert_eq!(unknown_count, 3);
}
#[test]
fn similar_slugs_warns_on_cluster() {
let (dir, linter) = workspace();

View File

@ -1,8 +1,4 @@
//! Reference-integrity checks: `replaced_by` existence + cycle detection.
//!
//! `requires` (Workflow) is checked symmetrically when/if the Workflow
//! linter is invoked from a human-edit path; the memory tool itself
//! never writes Workflow records.
use std::collections::HashSet;

View File

@ -1,10 +1,14 @@
//! Collect resident-injection candidates from the workspace.
//! Workspace knowledge enumeration helpers.
//!
//! Walks `<workspace>/knowledge/*.md`, returns the records whose
//! frontmatter has `model_invokation: true` as `(slug, description)`
//! pairs sorted by slug. The Pod system-prompt assembler appends them
//! into the trailing section so descriptions sit next to the scope
//! summary and AGENTS.md.
//! Two surfaces, both walking `<workspace>/.insomnia/knowledge/*.md`:
//!
//! - [`collect_resident_knowledge`] — resident-injection candidates
//! (`model_invokation: true`) returned as `(slug, description)` pairs
//! for the Pod system-prompt assembler.
//! - [`list_knowledge_slugs`] — every slug whose file parses, regardless
//! of `model_invokation`. Used by the Pod IPC layer to answer TUI `#`
//! completion (`model_invokation` is a resident-injection flag, not a
//! user-visibility flag).
//!
//! Files that fail to read or parse are skipped silently — the Linter
//! enforces shape on write, so a malformed file here means external
@ -19,17 +23,40 @@ pub struct ResidentKnowledgeEntry {
pub description: String,
}
/// Walk `<workspace>/knowledge/*.md` and return entries whose
/// Walk `<workspace>/.insomnia/knowledge/*.md` and return entries whose
/// frontmatter has `model_invokation: true`, sorted by slug. A missing
/// `knowledge/` directory yields an empty vec.
/// directory yields an empty vec.
pub fn collect_resident_knowledge(layout: &WorkspaceLayout) -> Vec<ResidentKnowledgeEntry> {
let mut out: Vec<ResidentKnowledgeEntry> = Vec::new();
walk_knowledge(layout, |slug, fm| {
if fm.model_invokation {
out.push(ResidentKnowledgeEntry {
slug,
description: fm.description,
});
}
});
out.sort_by(|a, b| a.slug.cmp(&b.slug));
out
}
/// Walk `<workspace>/knowledge/*.md` and return every slug whose
/// frontmatter parses, sorted ascending. Does not filter on
/// `model_invokation`. A missing `knowledge/` directory yields an empty
/// vec.
pub fn list_knowledge_slugs(layout: &WorkspaceLayout) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
walk_knowledge(layout, |slug, _fm| out.push(slug));
out.sort();
out
}
fn walk_knowledge(layout: &WorkspaceLayout, mut visit: impl FnMut(String, KnowledgeFrontmatter)) {
let dir = layout.knowledge_dir();
let entries = match std::fs::read_dir(&dir) {
Ok(it) => it,
Err(_) => return Vec::new(),
Err(_) => return,
};
let mut out: Vec<ResidentKnowledgeEntry> = Vec::new();
for entry in entries.flatten() {
let path = entry.path();
if !path.is_file() {
@ -55,15 +82,8 @@ pub fn collect_resident_knowledge(layout: &WorkspaceLayout) -> Vec<ResidentKnowl
Ok(f) => f,
Err(_) => continue,
};
if fm.model_invokation {
out.push(ResidentKnowledgeEntry {
slug,
description: fm.description,
});
}
visit(slug, fm);
}
out.sort_by(|a, b| a.slug.cmp(&b.slug));
out
}
#[cfg(test)]
@ -164,4 +184,41 @@ mod tests {
let got = collect_resident_knowledge(&layout);
assert_eq!(got.len(), 1);
}
#[test]
fn list_slugs_missing_dir_returns_empty() {
let dir = TempDir::new().unwrap();
let layout = WorkspaceLayout::new(dir.path().to_path_buf());
assert!(list_knowledge_slugs(&layout).is_empty());
}
#[test]
fn list_slugs_returns_all_regardless_of_model_invokation() {
let (dir, layout) = setup();
write_knowledge(dir.path(), "alpha", "a", true, "");
write_knowledge(dir.path(), "beta", "b", false, "");
write_knowledge(dir.path(), "gamma", "g", true, "");
let got = list_knowledge_slugs(&layout);
assert_eq!(got, vec!["alpha", "beta", "gamma"]);
}
#[test]
fn list_slugs_skips_malformed_and_non_md() {
let (dir, layout) = setup();
write_knowledge(dir.path(), "good", "ok", true, "");
std::fs::write(
dir.path().join(".insomnia/knowledge/bad.md"),
"---\nthis is not yaml: : :\n---\nbody\n",
)
.unwrap();
std::fs::write(
dir.path().join(".insomnia/knowledge/note.txt"),
"not markdown\n",
)
.unwrap();
let got = list_knowledge_slugs(&layout);
assert_eq!(got, vec!["good"]);
}
}

View File

@ -10,11 +10,9 @@ mod decision;
mod knowledge;
mod request;
mod summary;
mod workflow;
pub use common::{Frontmatter, SourceRef, split_frontmatter};
pub use decision::{DecisionFrontmatter, DecisionStatus};
pub use knowledge::{KNOWLEDGE_DESCRIPTION_HARD_CAP, KnowledgeFrontmatter};
pub use request::RequestFrontmatter;
pub use summary::SummaryFrontmatter;
pub use workflow::WorkflowFrontmatter;

View File

@ -1,50 +0,0 @@
//! Workflow frontmatter schema.
//!
//! NOTE: Workflows are written by humans, not by the memory tool. The
//! linter only validates frontmatter when invoked directly (e.g. by a
//! future CLI / pre-commit hook). The memory write/edit tool rejects
//! `.insomnia/workflow/` paths outright via
//! [`LintError::WorkflowWriteForbidden`].
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use crate::schema::common::Frontmatter;
use crate::slug::Slug;
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct WorkflowFrontmatter {
/// Workflows do not require timestamps in the MVP. Human-authored files
/// may carry them; when absent the linter uses Unix epoch as a neutral
/// placeholder for the shared `Frontmatter` trait.
#[serde(default)]
pub updated_at: Option<DateTime<Utc>>,
#[serde(default)]
pub created_at: Option<DateTime<Utc>>,
pub description: String,
#[serde(default)]
pub model_invokation: bool,
#[serde(default = "default_user_invocable")]
pub user_invocable: bool,
#[serde(default)]
pub requires: Vec<Slug>,
}
fn default_user_invocable() -> bool {
true
}
fn epoch() -> DateTime<Utc> {
DateTime::<Utc>::from_timestamp(0, 0).expect("Unix epoch timestamp is valid")
}
impl Frontmatter for WorkflowFrontmatter {
const BODY_LIMIT: usize = 8000;
fn created_at(&self) -> DateTime<Utc> {
self.created_at.or(self.updated_at).unwrap_or_else(epoch)
}
fn updated_at(&self) -> DateTime<Utc> {
self.updated_at.unwrap_or_else(epoch)
}
}

View File

@ -13,17 +13,13 @@ use manifest::{Permission, ScopeRule};
use crate::workspace::WorkspaceLayout;
/// Build deny rules that strip Write permission from `<workspace>/memory/`,
/// `<workspace>/knowledge/`, and `<workspace>/workflow/`. Recursive —
/// every descendant is capped at Read for the generic tools.
///
/// Workflow files are human-edited on the host side; the generic CRUD
/// tools must not touch them.
/// Build deny rules that strip Write permission from `<workspace>/memory/`
/// and `<workspace>/knowledge/`. Recursive — every descendant is capped at
/// Read for the generic tools.
pub fn deny_write_rules(layout: &WorkspaceLayout) -> Vec<ScopeRule> {
vec![
deny_write(layout.memory_dir().as_path()),
deny_write(layout.knowledge_dir().as_path()),
deny_write(layout.workflow_dir().as_path()),
]
}
@ -41,14 +37,13 @@ mod tests {
use std::path::PathBuf;
#[test]
fn deny_targets_memory_knowledge_and_workflow() {
fn deny_targets_memory_and_knowledge() {
let layout = WorkspaceLayout::new(PathBuf::from("/ws"));
let rules = deny_write_rules(&layout);
assert_eq!(rules.len(), 3);
assert_eq!(rules.len(), 2);
assert_eq!(rules[0].target, PathBuf::from("/ws/.insomnia/memory"));
assert_eq!(rules[0].permission, Permission::Write);
assert!(rules[0].recursive);
assert_eq!(rules[1].target, PathBuf::from("/ws/.insomnia/knowledge"));
assert_eq!(rules[2].target, PathBuf::from("/ws/.insomnia/workflow"));
}
}

View File

@ -20,7 +20,7 @@ use crate::workspace::{RecordKind, WorkspaceLayout};
pub use edit::edit_tool;
pub use query::{QueryConfig, knowledge_query_tool, memory_query_tool};
pub use read::read_tool;
pub use read::{read_tool, read_tool_with_usage};
pub use write::write_tool;
/// Kinds the memory tools accept as input. `Workflow` is intentionally

View File

@ -568,6 +568,29 @@ mod tests {
assert!(records.is_empty(), "got records: {:?}", out.content);
}
#[tokio::test]
async fn query_hits_do_not_log_usage() {
let (dir, layout) = setup();
write_decision(dir.path(), "alpha", "needle line\n");
write_knowledge(
dir.path(),
"policy",
"policy",
"needle desc",
"needle body\n",
);
let (_, memory_tool) = memory_query_tool(layout.clone(), QueryConfig::default())();
let (_, knowledge_tool) = knowledge_query_tool(layout.clone(), QueryConfig::default())();
let inp = serde_json::json!({ "query": "needle" });
memory_tool.execute(&inp.to_string()).await.unwrap();
knowledge_tool.execute(&inp.to_string()).await.unwrap();
let report = crate::usage::build_usage_report(&layout).unwrap();
assert!(report.records.is_empty());
assert!(!layout.usage_events_path().exists());
}
#[tokio::test]
async fn memory_query_respects_result_limit() {
let (dir, layout) = setup();

View File

@ -12,6 +12,7 @@ use llm_worker::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use serde::Deserialize;
use crate::tool::MemoryToolKind;
use crate::usage::{self, UsageSource};
use crate::workspace::WorkspaceLayout;
const DESCRIPTION: &str = "Read a memory or knowledge record by `kind` + `slug`. \
@ -38,6 +39,7 @@ struct ReadParams {
struct ReadTool {
layout: WorkspaceLayout,
usage_session_id: Option<String>,
}
#[async_trait]
@ -58,6 +60,22 @@ impl Tool for ReadTool {
})?;
let text = String::from_utf8_lossy(&bytes).into_owned();
if let Some(session_id) = self.usage_session_id.as_deref() {
let usage_slug = params.slug.as_deref().unwrap_or("summary");
let snapshot = usage::snapshot_record_from_bytes(
params.kind.record_kind(),
usage_slug.to_string(),
&bytes,
);
if let Err(err) = usage::append_use_event(
&self.layout,
session_id.to_string(),
UsageSource::MemoryRead,
vec![snapshot],
) {
tracing::warn!(error = %err, "failed to append MemoryRead usage event");
}
}
let offset = params.offset.unwrap_or(0);
let limit = params.limit.unwrap_or(DEFAULT_LIMIT).max(1);
let rendered = render_numbered(&text, offset, limit);
@ -117,6 +135,17 @@ fn render_numbered(text: &str, offset: usize, limit: usize) -> Rendered {
}
pub fn read_tool(layout: WorkspaceLayout) -> ToolDefinition {
read_tool_inner(layout, None)
}
pub fn read_tool_with_usage(
layout: WorkspaceLayout,
session_id: impl Into<String>,
) -> ToolDefinition {
read_tool_inner(layout, Some(session_id.into()))
}
fn read_tool_inner(layout: WorkspaceLayout, usage_session_id: Option<String>) -> ToolDefinition {
Arc::new(move || {
let schema = schemars::schema_for!(ReadParams);
let schema_value = serde_json::to_value(schema).unwrap_or(serde_json::json!({}));
@ -125,6 +154,7 @@ pub fn read_tool(layout: WorkspaceLayout) -> ToolDefinition {
.input_schema(schema_value);
let tool: Arc<dyn Tool> = Arc::new(ReadTool {
layout: layout.clone(),
usage_session_id: usage_session_id.clone(),
});
(meta, tool)
})
@ -209,6 +239,27 @@ mod tests {
assert!(out.content.unwrap().contains("k"));
}
#[tokio::test]
async fn read_logs_explicit_use_when_usage_session_is_set() {
let (dir, layout) = setup();
let path = dir.path().join(".insomnia/memory/decisions/foo.md");
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(&path, "alpha\n").unwrap();
let (_, tool) = read_tool_with_usage(layout.clone(), "session-1")();
let inp = serde_json::json!({ "kind": "decision", "slug": "foo" });
tool.execute(&inp.to_string()).await.unwrap();
let report = usage::build_usage_report(&layout).unwrap();
assert_eq!(report.records.len(), 1);
let record = &report.records[0];
assert_eq!(record.kind, "decision");
assert_eq!(record.slug, "foo");
assert_eq!(record.use_count, 1);
assert_eq!(record.source_breakdown["MemoryRead"], 1);
assert_eq!(record.resident_exposure_count, 0);
}
#[tokio::test]
async fn missing_file_returns_execution_failed() {
let (_dir, layout) = setup();

View File

@ -2,7 +2,7 @@
//!
//! Creates or overwrites a memory or knowledge record by `(kind, slug)`.
//! Pre-write Linter validates frontmatter, slug uniqueness (Create only),
//! reference integrity, size limits, and the workflow-write ban. On any
//! reference integrity, size limits. On any
//! Linter error the tool returns `ToolError::InvalidArgument` with all
//! violations aggregated and the file is **not** written.

383
crates/memory/src/usage.rs Normal file
View File

@ -0,0 +1,383 @@
//! Workspace-local usage event log for memory / knowledge / workflow records.
//!
//! The log is append-only JSONL under the workspace's `.insomnia/` tree. It is
//! intentionally evidence-only: aggregation reports explicit context reads and
//! resident exposure cost telemetry, but it does not classify records as
//! Knowledge candidates or tidy-protected records.
use std::collections::{BTreeMap, HashMap};
use std::fs::{self, OpenOptions};
use std::io::{self, BufRead, Write};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::workspace::{RecordKind, WorkspaceLayout};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum UsageEventKind {
Use,
ResidentExposure,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
pub enum UsageSource {
MemoryRead,
KnowledgeRef,
WorkflowInvoke,
ResidentInjection,
}
impl UsageSource {
pub fn as_str(self) -> &'static str {
match self {
Self::MemoryRead => "MemoryRead",
Self::KnowledgeRef => "KnowledgeRef",
Self::WorkflowInvoke => "WorkflowInvoke",
Self::ResidentInjection => "ResidentInjection",
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageRecordSnapshot {
pub kind: String,
pub slug: String,
pub file_bytes: u64,
pub file_tokens_estimate: u64,
}
impl UsageRecordSnapshot {
pub fn from_bytes(kind: RecordKind, slug: impl Into<String>, bytes: &[u8]) -> Self {
Self {
kind: kind.as_str().to_string(),
slug: slug.into(),
file_bytes: bytes.len() as u64,
file_tokens_estimate: estimate_tokens(bytes.len()),
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageEvent {
pub id: Uuid,
pub occurred_at: DateTime<Utc>,
pub session_id: String,
pub event: UsageEventKind,
pub source: UsageSource,
pub records: Vec<UsageRecordSnapshot>,
}
impl UsageEvent {
pub fn new(
session_id: impl Into<String>,
event: UsageEventKind,
source: UsageSource,
records: Vec<UsageRecordSnapshot>,
) -> Self {
Self {
id: Uuid::now_v7(),
occurred_at: Utc::now(),
session_id: session_id.into(),
event,
source,
records,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageReport {
pub records: Vec<UsageReportRecord>,
}
impl UsageReport {
pub fn empty() -> Self {
Self {
records: Vec::new(),
}
}
pub fn is_empty(&self) -> bool {
self.records.is_empty()
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageReportRecord {
pub kind: String,
pub slug: String,
pub use_count: u64,
pub last_used_at: Option<DateTime<Utc>>,
pub source_breakdown: BTreeMap<String, u64>,
pub resident_exposure_count: u64,
pub estimated_tokens_per_injection: u64,
pub estimated_total_resident_exposure_tokens: u64,
}
#[derive(Debug, Default)]
struct Accumulator {
kind: String,
slug: String,
use_count: u64,
last_used_at: Option<DateTime<Utc>>,
source_breakdown: BTreeMap<String, u64>,
resident_exposure_count: u64,
last_resident_tokens: u64,
}
/// Append one usage event to the workspace-local JSONL log.
pub fn append_usage_event(layout: &WorkspaceLayout, event: &UsageEvent) -> io::Result<()> {
let path = layout.usage_events_path();
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
}
let line = serde_json::to_string(event)
.map_err(|err| io::Error::new(io::ErrorKind::InvalidData, err))?;
let mut file = OpenOptions::new().create(true).append(true).open(path)?;
writeln!(file, "{line}")?;
Ok(())
}
/// Convenience for a successful explicit record read.
pub fn append_use_event(
layout: &WorkspaceLayout,
session_id: impl Into<String>,
source: UsageSource,
records: Vec<UsageRecordSnapshot>,
) -> io::Result<()> {
if records.is_empty() {
return Ok(());
}
append_usage_event(
layout,
&UsageEvent::new(session_id, UsageEventKind::Use, source, records),
)
}
/// Convenience for resident model-invocation exposure cost telemetry.
pub fn append_resident_exposure_event(
layout: &WorkspaceLayout,
session_id: impl Into<String>,
records: Vec<UsageRecordSnapshot>,
) -> io::Result<()> {
if records.is_empty() {
return Ok(());
}
append_usage_event(
layout,
&UsageEvent::new(
session_id,
UsageEventKind::ResidentExposure,
UsageSource::ResidentInjection,
records,
),
)
}
/// Read a record from the workspace and build the snapshot stored in usage
/// events. `slug` should be `"summary"` for [`RecordKind::Summary`].
pub fn snapshot_record_from_layout(
layout: &WorkspaceLayout,
kind: RecordKind,
slug: &str,
) -> io::Result<UsageRecordSnapshot> {
let path = record_path(layout, kind, slug)?;
let bytes = fs::read(path)?;
Ok(UsageRecordSnapshot::from_bytes(
kind,
slug.to_string(),
&bytes,
))
}
pub fn snapshot_record_from_bytes(
kind: RecordKind,
slug: impl Into<String>,
bytes: &[u8],
) -> UsageRecordSnapshot {
UsageRecordSnapshot::from_bytes(kind, slug, bytes)
}
fn record_path(
layout: &WorkspaceLayout,
kind: RecordKind,
slug: &str,
) -> io::Result<std::path::PathBuf> {
match kind {
RecordKind::Summary => Ok(layout.summary_path()),
RecordKind::Decision => {
let slug = crate::Slug::parse(slug.to_string()).map_err(invalid_slug_error)?;
Ok(layout.decision_path(&slug))
}
RecordKind::Request => {
let slug = crate::Slug::parse(slug.to_string()).map_err(invalid_slug_error)?;
Ok(layout.request_path(&slug))
}
RecordKind::Workflow => {
let slug = crate::Slug::parse(slug.to_string()).map_err(invalid_slug_error)?;
Ok(layout.workflow_path(&slug))
}
RecordKind::Knowledge => {
let slug = crate::Slug::parse(slug.to_string()).map_err(invalid_slug_error)?;
Ok(layout.knowledge_path(&slug))
}
}
}
fn invalid_slug_error(err: crate::LintError) -> io::Error {
io::Error::new(io::ErrorKind::InvalidInput, err)
}
/// Aggregate the append-only usage log into per-record evidence.
pub fn build_usage_report(layout: &WorkspaceLayout) -> io::Result<UsageReport> {
let path = layout.usage_events_path();
if !path.exists() {
return Ok(UsageReport::empty());
}
let file = fs::File::open(path)?;
let reader = io::BufReader::new(file);
let mut acc: HashMap<(String, String), Accumulator> = HashMap::new();
for line in reader.lines() {
let line = line?;
if line.trim().is_empty() {
continue;
}
let event: UsageEvent = match serde_json::from_str(&line) {
Ok(event) => event,
Err(err) => {
tracing::warn!(error = %err, "Skipping malformed memory usage event log line");
continue;
}
};
for record in event.records {
let key = (record.kind.clone(), record.slug.clone());
let entry = acc.entry(key).or_insert_with(|| Accumulator {
kind: record.kind.clone(),
slug: record.slug.clone(),
..Accumulator::default()
});
match event.event {
UsageEventKind::Use => {
entry.use_count += 1;
let source = event.source.as_str().to_string();
*entry.source_breakdown.entry(source).or_insert(0) += 1;
entry.last_used_at = Some(
entry
.last_used_at
.map(|prev| prev.max(event.occurred_at))
.unwrap_or(event.occurred_at),
);
}
UsageEventKind::ResidentExposure => {
entry.resident_exposure_count += 1;
entry.last_resident_tokens = record.file_tokens_estimate;
}
}
}
}
let mut records: Vec<UsageReportRecord> = acc
.into_values()
.map(|a| UsageReportRecord {
kind: a.kind,
slug: a.slug,
use_count: a.use_count,
last_used_at: a.last_used_at,
source_breakdown: a.source_breakdown,
resident_exposure_count: a.resident_exposure_count,
estimated_tokens_per_injection: a.last_resident_tokens,
estimated_total_resident_exposure_tokens: a
.last_resident_tokens
.saturating_mul(a.resident_exposure_count),
})
.collect();
records.sort_by(|a, b| (&a.kind, &a.slug).cmp(&(&b.kind, &b.slug)));
Ok(UsageReport { records })
}
fn estimate_tokens(bytes: usize) -> u64 {
(bytes as u64).div_ceil(4)
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::TempDir;
fn setup() -> (TempDir, WorkspaceLayout) {
let dir = TempDir::new().unwrap();
let layout = WorkspaceLayout::new(dir.path().to_path_buf());
(dir, layout)
}
#[test]
fn aggregates_use_and_resident_exposure_separately() {
let (_dir, layout) = setup();
let decision = snapshot_record_from_bytes(RecordKind::Decision, "alpha", b"abcd");
let knowledge = snapshot_record_from_bytes(RecordKind::Knowledge, "policy", b"abcdefgh");
append_use_event(
&layout,
"session-a",
UsageSource::MemoryRead,
vec![decision.clone()],
)
.unwrap();
append_use_event(
&layout,
"session-a",
UsageSource::KnowledgeRef,
vec![knowledge.clone()],
)
.unwrap();
append_use_event(
&layout,
"session-b",
UsageSource::KnowledgeRef,
vec![knowledge.clone()],
)
.unwrap();
append_resident_exposure_event(&layout, "session-b", vec![knowledge]).unwrap();
let report = build_usage_report(&layout).unwrap();
let decision = report
.records
.iter()
.find(|r| r.kind == "decision" && r.slug == "alpha")
.unwrap();
assert_eq!(decision.use_count, 1);
assert_eq!(decision.source_breakdown["MemoryRead"], 1);
assert_eq!(decision.resident_exposure_count, 0);
assert!(decision.last_used_at.is_some());
let knowledge = report
.records
.iter()
.find(|r| r.kind == "knowledge" && r.slug == "policy")
.unwrap();
assert_eq!(knowledge.use_count, 2);
assert_eq!(knowledge.source_breakdown["KnowledgeRef"], 2);
assert_eq!(knowledge.resident_exposure_count, 1);
assert_eq!(knowledge.estimated_tokens_per_injection, 2);
assert_eq!(knowledge.estimated_total_resident_exposure_tokens, 2);
}
#[test]
fn resident_only_record_does_not_increment_use_count() {
let (_dir, layout) = setup();
let snapshot = snapshot_record_from_bytes(RecordKind::Knowledge, "policy", b"abcdefgh");
append_resident_exposure_event(&layout, "session", vec![snapshot]).unwrap();
let report = build_usage_report(&layout).unwrap();
let record = &report.records[0];
assert_eq!(record.use_count, 0);
assert!(record.last_used_at.is_none());
assert!(record.source_breakdown.is_empty());
assert_eq!(record.resident_exposure_count, 1);
}
}

View File

@ -33,6 +33,8 @@ const SUMMARY_FILE: &str = "summary.md";
const DECISIONS_DIR: &str = "decisions";
const REQUESTS_DIR: &str = "requests";
const STAGING_DIR: &str = "_staging";
const USAGE_DIR: &str = "_usage";
const USAGE_EVENTS_FILE: &str = "events.jsonl";
/// What kind of record a path under the memory tree represents.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@ -126,6 +128,14 @@ impl WorkspaceLayout {
self.memory_dir().join(STAGING_DIR)
}
pub fn usage_dir(&self) -> PathBuf {
self.memory_dir().join(USAGE_DIR)
}
pub fn usage_events_path(&self) -> PathBuf {
self.usage_dir().join(USAGE_EVENTS_FILE)
}
pub fn decision_path(&self, slug: &Slug) -> PathBuf {
self.decisions_dir().join(format!("{slug}.md"))
}
@ -143,9 +153,9 @@ impl WorkspaceLayout {
}
/// Classify a path under the memory tree. Returns `None` if the
/// path is not under `.insomnia/memory/`, `.insomnia/knowledge/`,
/// or `.insomnia/workflow/` of this workspace, or if it lives in
/// `_staging/` (which is opaque to the linter).
/// path is not under `.insomnia/memory/` or `.insomnia/knowledge/`
/// of this workspace, or if it lives in
/// `_staging/` / `_usage/` (opaque subsystem-owned trees).
///
/// On a conventional path that's *almost* a record but malformed
/// (e.g. `.insomnia/memory/decisions/Foo.md` with an invalid slug),
@ -154,14 +164,10 @@ impl WorkspaceLayout {
pub fn classify(&self, path: &Path) -> Result<Option<ClassifiedPath>, LintError> {
let memory = self.memory_dir();
let knowledge = self.knowledge_dir();
let workflow = self.workflow_dir();
if let Ok(rel) = path.strip_prefix(&knowledge) {
return Ok(Some(classify_kinded_md(rel, RecordKind::Knowledge, path)?));
}
if let Ok(rel) = path.strip_prefix(&workflow) {
return Ok(Some(classify_kinded_md(rel, RecordKind::Workflow, path)?));
}
let rel = match path.strip_prefix(&memory) {
Ok(r) => r,
Err(_) => return Ok(None),
@ -182,8 +188,8 @@ impl WorkspaceLayout {
slug: None,
}));
}
if first == STAGING_DIR {
// Linter opts out of `_staging/`; extract handles its schema.
if first == STAGING_DIR || first == USAGE_DIR {
// Linter opts out of subsystem-owned opaque trees.
return Ok(None);
}
@ -267,16 +273,6 @@ mod tests {
assert_eq!(cp.kind, RecordKind::Knowledge);
}
#[test]
fn classifies_workflow() {
let cp = layout()
.classify(&PathBuf::from("/ws/.insomnia/workflow/wf.md"))
.unwrap()
.unwrap();
assert_eq!(cp.kind, RecordKind::Workflow);
assert_eq!(cp.slug.unwrap().as_str(), "wf");
}
#[test]
fn workflow_under_memory_is_invalid_path() {
let err = layout()
@ -295,6 +291,14 @@ mod tests {
);
}
#[test]
fn usage_tree_is_opaque_to_classifier() {
let cp = layout()
.classify(&PathBuf::from("/ws/.insomnia/memory/_usage/events.jsonl"))
.unwrap();
assert!(cp.is_none());
}
#[test]
fn outside_returns_none() {
assert!(

View File

@ -27,6 +27,7 @@ fs4 = { workspace = true, features = ["sync"] }
libc = { workspace = true }
schemars = { workspace = true }
memory = { workspace = true }
workflow-crate = { package = "workflow", path = "../workflow" }
uuid = { workspace = true, features = ["v7"] }
session-metrics = { workspace = true }

View File

@ -176,6 +176,7 @@ impl PodController {
// `PodFsView` to the shared state once the latter exists.
let fs_for_view: tools::ScopedFs;
let task_store = pod.task_store();
let session_id_for_usage = pod.session_id().to_string();
let scope_change_sink = pod.scope_change_sink();
@ -334,7 +335,10 @@ impl PodController {
if let Some(mem) = memory_config.as_ref() {
let layout = memory::WorkspaceLayout::resolve(mem, &pwd_for_tools);
let query_cfg = memory::tool::QueryConfig::from(mem);
worker.register_tool(memory::tool::read_tool(layout.clone()));
worker.register_tool(memory::tool::read_tool_with_usage(
layout.clone(),
session_id_for_usage.clone(),
));
worker.register_tool(memory::tool::write_tool(layout.clone()));
worker.register_tool(memory::tool::edit_tool(layout.clone()));
worker.register_tool(memory::tool::memory_query_tool(layout.clone(), query_cfg));
@ -384,6 +388,12 @@ impl PodController {
.map(|slug| crate::shared_state::WorkflowCandidate { slug })
.collect(),
);
shared_state.set_knowledge(
pod.knowledge_completions()
.into_iter()
.map(|slug| crate::shared_state::KnowledgeCandidate { slug })
.collect(),
);
runtime_dir.write_manifest(&manifest_toml).await?;
runtime_dir.write_status(&shared_state).await?;
runtime_dir.write_history(&shared_state).await?;

View File

@ -102,7 +102,15 @@ async fn handle_connection(stream: tokio::net::UnixStream, handle: PodHandle) {
is_dir: c.is_dir,
})
.collect(),
protocol::CompletionKind::Knowledge => Vec::new(),
protocol::CompletionKind::Knowledge => handle
.shared_state
.list_knowledge_completions(&prefix)
.into_iter()
.map(|c| protocol::CompletionEntry {
value: c.slug,
is_dir: false,
})
.collect(),
protocol::CompletionKind::Workflow => handle
.shared_state
.list_workflow_completions(&prefix)

View File

@ -150,7 +150,7 @@ pub struct Pod<C: LlmClient, St: Store> {
prompts: Arc<PromptCatalog>,
/// Registry loaded from `<workspace>/.insomnia/workflow/*.md` when
/// memory is enabled. Missing memory config keeps this empty.
workflow_registry: memory::WorkflowRegistry,
workflow_registry: workflow_crate::WorkflowRegistry,
/// Memory workspace layout used by the workflow resolver to load required
/// Knowledge records by exact slug.
memory_layout: Option<memory::WorkspaceLayout>,
@ -323,7 +323,7 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
scope_allocation: None,
callback_socket: None,
prompts,
workflow_registry: memory::WorkflowRegistry::empty(),
workflow_registry: workflow_crate::WorkflowRegistry::empty(),
memory_layout: None,
inject_resident_knowledge: true,
pending_scope_snapshot: Arc::new(Mutex::new(None)),
@ -828,17 +828,19 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
return Ok(());
};
let alerter = self.alerter.clone();
let worker = self.worker.as_mut().expect("worker present");
// Materialise any pending tool factories so the template sees the
// full list of tool names. Redundant with the flush inside
// `Worker::lock()`; safe because `flush_pending` is idempotent.
worker.tool_server_handle().flush_pending();
let tool_names: Vec<String> = worker
.tool_server_handle()
.tool_definitions_sorted()
.into_iter()
.map(|d| d.name)
.collect();
let tool_names: Vec<String> = {
let worker = self.worker.as_mut().expect("worker present");
// Materialise any pending tool factories so the template sees the
// full list of tool names. Redundant with the flush inside
// `Worker::lock()`; safe because `flush_pending` is idempotent.
worker.tool_server_handle().flush_pending();
worker
.tool_server_handle()
.tool_definitions_sorted()
.into_iter()
.map(|d| d.name)
.collect()
};
let agents_md_read = read_agents_md(&self.pwd);
for warning in agents_md_read.warnings {
if let Some(n) = alerter.as_ref() {
@ -863,18 +865,20 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
} else {
None
};
let resident_workflows: Vec<memory::ResidentWorkflowEntry> =
let resident_workflows: Vec<workflow_crate::ResidentWorkflowEntry> =
if self.inject_resident_knowledge && self.memory_layout.is_some() {
self.workflow_registry.resident_entries()
} else {
Vec::new()
};
let resident_workflow_slice: Option<&[memory::ResidentWorkflowEntry]> =
let resident_workflow_slice: Option<&[workflow_crate::ResidentWorkflowEntry]> =
if self.inject_resident_knowledge && self.memory_layout.is_some() {
Some(&resident_workflows)
} else {
None
};
let resident_exposure_snapshots =
self.resident_exposure_snapshots(&resident, &resident_workflows);
let scope_snapshot = self.scope.snapshot();
let ctx = SystemPromptContext {
now: chrono::Utc::now(),
@ -889,7 +893,11 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
let rendered = template
.render(&ctx)
.map_err(|source| PodError::SystemPromptRender { source })?;
worker.set_system_prompt(rendered);
self.worker
.as_mut()
.expect("worker present")
.set_system_prompt(rendered);
self.append_resident_exposure_event(resident_exposure_snapshots);
Ok(())
}
@ -979,11 +987,13 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
}
self.user_segments.push(input.clone());
// Resolve `@<path>` refs and `/<slug>` workflow invocations to
// system messages stashed for the PodInterceptor to attach right
// after the user message. File failures are non-fatal alerts; explicit
// workflow invocation failures abort before the Worker sees the turn.
// Resolve `@<path>` refs, `#<slug>` Knowledge refs, and `/<slug>`
// workflow invocations to system messages stashed for the
// PodInterceptor to attach right after the user message. File and
// Knowledge failures are non-fatal alerts; explicit workflow invocation
// failures abort before the Worker sees the turn.
let mut attachments = self.resolve_file_refs(&input);
attachments.extend(self.resolve_knowledge_refs(&input));
attachments.extend(self.resolve_workflow_invocations(&input)?);
if !attachments.is_empty() {
*self
@ -1034,6 +1044,127 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
out
}
fn resolve_knowledge_refs(&self, segments: &[Segment]) -> Vec<Item> {
let Some(layout) = self.memory_layout.as_ref() else {
return Vec::new();
};
let mut out = Vec::new();
for seg in segments {
let Segment::KnowledgeRef { slug } = seg else {
continue;
};
let parsed = match memory::Slug::parse(slug.clone()) {
Ok(slug) => slug,
Err(e) => {
self.alert(
AlertLevel::Warn,
AlertSource::Pod,
format!("knowledge ref #{slug} has invalid slug: {e}"),
);
continue;
}
};
let path = layout.knowledge_path(&parsed);
let bytes = match std::fs::read(&path) {
Ok(bytes) => bytes,
Err(e) => {
self.alert(
AlertLevel::Warn,
AlertSource::Pod,
format!("knowledge ref #{slug} could not be read: {e}"),
);
continue;
}
};
let raw = String::from_utf8_lossy(&bytes).into_owned();
let body = match memory::schema::split_frontmatter(&raw) {
Ok((_yaml, body)) => body,
Err(e) => {
self.alert(
AlertLevel::Warn,
AlertSource::Pod,
format!("knowledge ref #{slug} has invalid frontmatter: {e}"),
);
continue;
}
};
let snapshot = memory::snapshot_record_from_bytes(
memory::workspace::RecordKind::Knowledge,
slug.clone(),
&bytes,
);
self.append_memory_use_event(memory::UsageSource::KnowledgeRef, vec![snapshot]);
out.push(Item::system_message(format!(
"[Knowledge #{}]\n{}",
slug,
body.trim_end()
)));
}
out
}
fn resident_exposure_snapshots(
&self,
knowledge: &[memory::ResidentKnowledgeEntry],
workflows: &[workflow_crate::ResidentWorkflowEntry],
) -> Vec<memory::UsageRecordSnapshot> {
let Some(layout) = self.memory_layout.as_ref() else {
return Vec::new();
};
let mut snapshots = Vec::new();
for entry in knowledge {
match memory::snapshot_record_from_layout(
layout,
memory::workspace::RecordKind::Knowledge,
&entry.slug,
) {
Ok(snapshot) => snapshots.push(snapshot),
Err(err) => {
warn!(knowledge = %entry.slug, error = %err, "failed to snapshot resident knowledge exposure")
}
}
}
for entry in workflows {
match memory::snapshot_record_from_layout(
layout,
memory::workspace::RecordKind::Workflow,
&entry.slug,
) {
Ok(snapshot) => snapshots.push(snapshot),
Err(err) => {
warn!(workflow = %entry.slug, error = %err, "failed to snapshot resident workflow exposure")
}
}
}
snapshots
}
fn append_memory_use_event(
&self,
source: memory::UsageSource,
records: Vec<memory::UsageRecordSnapshot>,
) {
let Some(layout) = self.memory_layout.as_ref() else {
return;
};
if let Err(err) =
memory::append_use_event(layout, self.session_id.to_string(), source, records)
{
warn!(error = %err, "failed to append memory usage event");
}
}
fn append_resident_exposure_event(&self, records: Vec<memory::UsageRecordSnapshot>) {
let Some(layout) = self.memory_layout.as_ref() else {
return;
};
if let Err(err) =
memory::append_resident_exposure_event(layout, self.session_id.to_string(), records)
{
warn!(error = %err, "failed to append resident exposure event");
}
}
fn resolve_workflow_invocations(
&self,
segments: &[Segment],
@ -1057,6 +1188,21 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
layout,
slug,
)?;
match memory::snapshot_record_from_layout(
layout,
memory::workspace::RecordKind::Workflow,
slug,
) {
Ok(snapshot) => {
self.append_memory_use_event(
memory::UsageSource::WorkflowInvoke,
vec![snapshot],
);
}
Err(err) => {
warn!(workflow = %slug, error = %err, "failed to snapshot workflow usage");
}
}
out.extend(items);
}
Ok(out)
@ -1074,8 +1220,8 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
let Segment::WorkflowInvoke { slug } = seg else {
continue;
};
let parsed =
memory::Slug::parse(slug.clone()).map_err(WorkflowResolveError::InvalidSlug)?;
let parsed = workflow_crate::Slug::parse(slug.clone())
.map_err(WorkflowResolveError::InvalidSlug)?;
let record = self
.workflow_registry
.get(&parsed)
@ -1091,6 +1237,13 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
self.workflow_registry.list_user_invocable("")
}
pub fn knowledge_completions(&self) -> Vec<String> {
self.memory_layout
.as_ref()
.map(memory::list_knowledge_slugs)
.unwrap_or_default()
}
/// Flatten a typed segment list into the single string the Worker
/// receives as the user message, and emit user-facing alerts for
/// segments that fall through to placeholder (knowledge / workflow
@ -1103,14 +1256,16 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
match seg {
Segment::Text { .. } | Segment::Paste { .. } | Segment::FileRef { .. } => {}
Segment::KnowledgeRef { slug } => {
self.alert(
AlertLevel::Warn,
AlertSource::Pod,
format!(
"knowledge ref #{slug} cannot be resolved \
(resolver not yet implemented); passed to LLM as placeholder"
),
);
if self.memory_layout.is_none() {
self.alert(
AlertLevel::Warn,
AlertSource::Pod,
format!(
"knowledge ref #{slug} cannot be resolved \
because memory is disabled; passed to LLM as placeholder"
),
);
}
}
Segment::WorkflowInvoke { .. } => {}
Segment::Unknown => {
@ -2139,7 +2294,10 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
// のKnowledgeアクセス (agent pulls knowledge through the search
// tool instead of via system-prompt residency).
let query_cfg = memory::tool::QueryConfig::from(memory_cfg);
worker.register_tool(memory::tool::read_tool(layout.clone()));
worker.register_tool(memory::tool::read_tool_with_usage(
layout.clone(),
self.session_id.to_string(),
));
worker.register_tool(memory::tool::write_tool(layout.clone()));
worker.register_tool(memory::tool::edit_tool(layout.clone()));
worker.register_tool(memory::tool::memory_query_tool(layout.clone(), query_cfg));
@ -2149,9 +2307,15 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
));
let tidy = consolidate::collect_tidy_hints(&layout);
let candidates = consolidate::KnowledgeCandidateReport::empty();
let usage_report = match memory::build_usage_report(&layout) {
Ok(report) => report,
Err(err) => {
warn!(error = %err, "failed to build memory usage report for consolidation");
memory::UsageReport::empty()
}
};
let input_text =
consolidate::build_consolidate_input(&layout, &entries, &tidy, &candidates);
consolidate::build_consolidate_input(&layout, &entries, &tidy, &usage_report);
let run_result = worker.run(input_text).await;
match run_result {
@ -2729,7 +2893,7 @@ pub enum PodError {
ConsolidationLock(#[source] memory::consolidate::LockError),
#[error("workflow load failed: {0}")]
WorkflowLoad(#[source] memory::WorkflowLoadError),
WorkflowLoad(#[source] workflow_crate::WorkflowLoadError),
#[error("workflow invocation failed: {0}")]
WorkflowResolve(#[from] WorkflowResolveError),
@ -2752,14 +2916,14 @@ struct PodCommon {
scope: Scope,
client: Box<dyn LlmClient>,
prompts: Arc<PromptCatalog>,
workflow_registry: memory::WorkflowRegistry,
workflow_registry: workflow_crate::WorkflowRegistry,
memory_layout: Option<memory::WorkspaceLayout>,
system_prompt_template: Option<SystemPromptTemplate>,
/// SKILL.md shadow events surfaced during workflow-registry build.
/// The Pod constructor drains these into the notify buffer right
/// after the Pod is materialised so the first LLM request observes
/// any skill ↔ workflow collisions.
skill_shadows: Vec<memory::ShadowedSkill>,
skill_shadows: Vec<workflow_crate::ShadowedSkill>,
}
/// Resolve pwd / scope / LLM client / prompt catalog from a validated
@ -2811,8 +2975,8 @@ fn prepare_pod_common_from_scope(
.as_ref()
.map(|mem| memory::WorkspaceLayout::resolve(mem, &pwd));
let mut workflow_registry = match memory_layout.as_ref() {
Some(layout) => memory::load_workflows(layout).map_err(PodError::WorkflowLoad)?,
None => memory::WorkflowRegistry::empty(),
Some(layout) => workflow_crate::load_workflows(layout).map_err(PodError::WorkflowLoad)?,
None => workflow_crate::WorkflowRegistry::empty(),
};
let skill_shadows = ingest_skills(&mut workflow_registry, manifest);
@ -2841,21 +3005,21 @@ fn prepare_pod_common_from_scope(
///
/// Skills come exclusively from the manifest's `[skills] directories`
/// list (resolved against the manifest base directory). Internal
/// Workflows already loaded via [`memory::load_workflows`] take priority
/// Workflows already loaded via [`workflow_crate::load_workflows`] take priority
/// over skills sharing the same slug; collisions are surfaced as
/// [`memory::ShadowedSkill`] events that the caller pushes onto the
/// [`workflow_crate::ShadowedSkill`] events that the caller pushes onto the
/// Pod's notification buffer.
fn ingest_skills(
registry: &mut memory::WorkflowRegistry,
registry: &mut workflow_crate::WorkflowRegistry,
manifest: &PodManifest,
) -> Vec<memory::ShadowedSkill> {
) -> Vec<workflow_crate::ShadowedSkill> {
let mut shadows = Vec::new();
let Some(skills_cfg) = manifest.skills.as_ref() else {
return shadows;
};
for dir in &skills_cfg.directories {
for skill in memory::load_skills_from_dir(dir) {
let source = memory::WorkflowSource::Skill { dir: dir.clone() };
for skill in workflow_crate::load_skills_from_dir(dir) {
let source = workflow_crate::WorkflowSource::Skill { dir: dir.clone() };
let record = skill.into_workflow_record(source);
if let Some(shadow) = registry.merge_skill(record) {
shadows.push(shadow);
@ -2867,7 +3031,7 @@ fn ingest_skills(
/// Drain skill-ingest shadow events into the Pod's notify buffer so the
/// first LLM request renders them as system-message attachments.
fn drain_skill_shadows<C, S>(pod: &Pod<C, S>, shadows: Vec<memory::ShadowedSkill>)
fn drain_skill_shadows<C, S>(pod: &Pod<C, S>, shadows: Vec<workflow_crate::ShadowedSkill>)
where
C: LlmClient,
S: Store,
@ -2891,6 +3055,9 @@ fn build_scope_with_memory(manifest: &PodManifest, pwd: &Path) -> Result<Scope,
if let Some(mem) = manifest.memory.as_ref() {
let layout = memory::WorkspaceLayout::resolve(mem, pwd);
scope_config.deny.extend(memory::deny_write_rules(&layout));
scope_config
.deny
.extend(workflow_crate::deny_write_rules(&layout));
}
scope_config.allow.extend(skill_dir_read_rules(manifest));
Scope::from_config(&scope_config).map_err(PodError::Scope)
@ -3058,7 +3225,7 @@ permission = "write"
#[test]
fn ingest_skills_returns_empty_when_skills_section_missing() {
let manifest = minimal_manifest_with_skills(vec![]);
let mut registry = memory::WorkflowRegistry::empty();
let mut registry = workflow_crate::WorkflowRegistry::empty();
let shadows = ingest_skills(&mut registry, &manifest);
assert!(shadows.is_empty());
assert!(registry.is_empty());
@ -3076,13 +3243,13 @@ permission = "write"
.unwrap();
let manifest = minimal_manifest_with_skills(vec![skills_root.clone()]);
let mut registry = memory::WorkflowRegistry::empty();
let mut registry = workflow_crate::WorkflowRegistry::empty();
let shadows = ingest_skills(&mut registry, &manifest);
// workspace skill `alpha` should be registered (no collision).
assert!(
registry
.get(&memory::Slug::parse("alpha").unwrap())
.get(&workflow_crate::Slug::parse("alpha").unwrap())
.is_some()
);
// No workflow exists to shadow `alpha`, so no shadow event for it.

View File

@ -18,10 +18,11 @@ use std::sync::Arc;
use chrono::{DateTime, SecondsFormat, Utc};
use manifest::Scope;
use memory::{ResidentKnowledgeEntry, ResidentWorkflowEntry};
use memory::ResidentKnowledgeEntry;
use minijinja::value::Value;
use minijinja::{Environment, ErrorKind, UndefinedBehavior};
use thiserror::Error;
use workflow_crate::ResidentWorkflowEntry;
use crate::prompt::catalog::{CatalogError, PromptCatalog};
use crate::prompt::loader::{LoaderError, PromptLoader, PromptRef};

View File

@ -12,6 +12,11 @@ pub struct WorkflowCandidate {
pub slug: String,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct KnowledgeCandidate {
pub slug: String,
}
/// Shared state between PodController and runtime directory.
///
/// Controller updates this in-memory; RuntimeDir writes it to disk.
@ -37,6 +42,7 @@ pub struct PodSharedState {
/// directly without spinning up a controller).
fs_view: OnceLock<PodFsView>,
workflows: OnceLock<Vec<WorkflowCandidate>>,
knowledge: OnceLock<Vec<KnowledgeCandidate>>,
}
impl PodSharedState {
@ -56,6 +62,7 @@ impl PodSharedState {
user_segments: RwLock::new(Vec::new()),
fs_view: OnceLock::new(),
workflows: OnceLock::new(),
knowledge: OnceLock::new(),
}
}
@ -88,6 +95,23 @@ impl PodSharedState {
.unwrap_or_default()
}
pub fn set_knowledge(&self, knowledge: Vec<KnowledgeCandidate>) {
let _ = self.knowledge.set(knowledge);
}
pub fn list_knowledge_completions(&self, prefix: &str) -> Vec<KnowledgeCandidate> {
self.knowledge
.get()
.map(|items| {
items
.iter()
.filter(|candidate| candidate.slug.starts_with(prefix))
.cloned()
.collect()
})
.unwrap_or_default()
}
pub fn user_segments(&self) -> Vec<Vec<Segment>> {
self.user_segments
.read()
@ -230,4 +254,38 @@ mod tests {
assert!(parsed.is_array());
assert_eq!(parsed[0]["role"], "assistant");
}
#[test]
fn knowledge_completions_empty_when_unset() {
let state = test_state();
assert!(state.list_knowledge_completions("").is_empty());
assert!(state.list_knowledge_completions("foo").is_empty());
}
#[test]
fn knowledge_completions_filter_by_prefix() {
let state = test_state();
state.set_knowledge(vec![
KnowledgeCandidate {
slug: "alpha".into(),
},
KnowledgeCandidate {
slug: "alphabet".into(),
},
KnowledgeCandidate {
slug: "beta".into(),
},
]);
let all = state.list_knowledge_completions("");
assert_eq!(all.len(), 3);
let alpha = state.list_knowledge_completions("alpha");
assert_eq!(
alpha
.iter()
.map(|c| c.slug.as_str())
.collect::<Vec<_>>(),
vec!["alpha", "alphabet"]
);
assert!(state.list_knowledge_completions("zzz").is_empty());
}
}

View File

@ -9,12 +9,13 @@
use std::fmt;
use llm_worker::Item;
use memory::WorkspaceLayout;
use memory::schema::split_frontmatter;
use memory::{Slug, WorkflowRegistry, WorkspaceLayout};
use workflow_crate::{Slug, WorkflowRegistry};
#[derive(Debug)]
pub enum WorkflowResolveError {
InvalidSlug(memory::LintError),
InvalidSlug(workflow_crate::WorkflowLintError),
NotFound {
slug: String,
},
@ -90,7 +91,7 @@ pub fn resolve_workflow_invocation(
let mut out = Vec::new();
for req in &record.requires {
let path = layout.knowledge_path(req);
let path = layout.knowledge_dir().join(format!("{req}.md"));
let raw = std::fs::read_to_string(&path).map_err(|source| {
if source.kind() == std::io::ErrorKind::NotFound {
WorkflowResolveError::KnowledgeNotFound {
@ -150,7 +151,7 @@ mod tests {
&dir.path().join(".insomnia/workflow/run-it.md"),
"---\ndescription: run\nrequires: [policy]\n---\nworkflow body\n",
);
let registry = memory::load_workflows(&layout).unwrap();
let registry = workflow_crate::load_workflows(&layout).unwrap();
(dir, layout, registry)
}
@ -174,7 +175,7 @@ mod tests {
&dir.path().join(".insomnia/workflow/hidden.md"),
"---\ndescription: hidden\nuser_invocable: false\n---\nbody\n",
);
let registry = memory::load_workflows(&layout).unwrap();
let registry = workflow_crate::load_workflows(&layout).unwrap();
let err = resolve_workflow_invocation(&registry, &layout, "hidden").unwrap_err();
assert!(matches!(err, WorkflowResolveError::NotUserInvocable { .. }));
}
@ -187,7 +188,7 @@ mod tests {
&dir.path().join(".insomnia/workflow/bad.md"),
"---\ndescription: bad\nrequires: [ghost]\n---\nbody\n",
);
let registry = memory::load_workflows(&layout).unwrap();
let registry = workflow_crate::load_workflows(&layout).unwrap();
let err = resolve_workflow_invocation(&registry, &layout, "bad").unwrap_err();
assert!(matches!(
err,

View File

@ -0,0 +1,18 @@
[package]
name = "workflow"
version = "0.1.0"
edition.workspace = true
license.workspace = true
[dependencies]
chrono = { version = "0.4", features = ["serde"] }
manifest = { workspace = true }
memory = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_yaml = "0.9.34"
thiserror = { workspace = true }
tracing = { workspace = true }
[dev-dependencies]
tempfile = { workspace = true }
serde_json = { workspace = true }

View File

@ -0,0 +1,39 @@
//! Errors raised by Workflow loading and linting.
use std::path::PathBuf;
use thiserror::Error;
/// A single Workflow linter violation.
#[derive(Debug, Clone, Error, PartialEq, Eq)]
pub enum WorkflowLintError {
#[error("invalid slug `{0}`: must match ^[a-z0-9](?:[a-z0-9-]{{0,62}}[a-z0-9])?$")]
InvalidSlug(String),
#[error("malformed frontmatter: {0}")]
MalformedFrontmatter(String),
#[error("frontmatter is missing or document is empty")]
MissingFrontmatter,
#[error("missing required frontmatter field: `{0}`")]
MissingField(&'static str),
#[error(
"Workflow with model_invokation: true cannot have description longer than {limit} chars (got {actual})"
)]
DescriptionTooLong { actual: usize, limit: usize },
#[error("body exceeds the Workflow size limit: {actual} chars > {limit}")]
BodyTooLong { actual: usize, limit: usize },
#[error("`{field}` references unknown {kind} slug `{slug}`")]
UnknownReference {
field: &'static str,
kind: &'static str,
slug: String,
},
#[error("path is not a valid Workflow location: {}", .0.display())]
InvalidPath(PathBuf),
}

View File

@ -0,0 +1,22 @@
//! Workflow records, loading, Agent Skill ingestion, and human-edit linting.
mod error;
mod linter;
mod schema;
mod scope;
mod skill;
mod slug;
mod workflow;
pub use error::WorkflowLintError;
pub use linter::{WorkflowLintReport, WorkflowLinter};
pub use schema::{WorkflowFrontmatter, split_frontmatter};
pub use scope::deny_write_rules;
pub use skill::{
SKILL_FILENAME, SkillParseError, SkillRecord, load_skills_from_dir, parse_skill_md,
};
pub use slug::{Slug, is_valid_slug};
pub use workflow::{
ResidentWorkflowEntry, ShadowedSkill, WORKFLOW_DESCRIPTION_HARD_CAP, WorkflowLoadError,
WorkflowRecord, WorkflowRegistry, WorkflowSource, load_workflows,
};

View File

@ -0,0 +1,223 @@
//! Human-edit linter for Workflow files.
use std::collections::HashSet;
use memory::WorkspaceLayout;
use crate::{Slug, WorkflowLintError};
use serde::de::DeserializeOwned;
use crate::schema::{WORKFLOW_BODY_LIMIT, WorkflowFrontmatter, split_frontmatter};
use crate::workflow::WORKFLOW_DESCRIPTION_HARD_CAP;
#[derive(Debug, Default, Clone)]
pub struct WorkflowLintReport {
pub errors: Vec<WorkflowLintError>,
}
impl WorkflowLintReport {
pub fn has_errors(&self) -> bool {
!self.errors.is_empty()
}
pub fn push_error(&mut self, err: WorkflowLintError) {
self.errors.push(err);
}
}
#[derive(Debug, Clone)]
pub struct WorkflowLinter {
layout: WorkspaceLayout,
}
impl WorkflowLinter {
pub fn new(layout: WorkspaceLayout) -> Self {
Self { layout }
}
pub fn layout(&self) -> &WorkspaceLayout {
&self.layout
}
/// Validate a human-authored Workflow document.
///
/// Verifies frontmatter shape, body size, resident description size, and
/// that every `requires` slug points at an existing Knowledge record.
pub fn lint(&self, content: &str) -> WorkflowLintReport {
let mut report = WorkflowLintReport::default();
let parsed = match parse_frontmatter::<WorkflowFrontmatter>(content) {
Ok(parsed) => parsed,
Err(err) => {
report.push_error(err);
return report;
}
};
let body_chars = parsed.body.chars().count();
if body_chars > WORKFLOW_BODY_LIMIT {
report.push_error(WorkflowLintError::BodyTooLong {
actual: body_chars,
limit: WORKFLOW_BODY_LIMIT,
});
}
if parsed.frontmatter.model_invokation {
let actual = parsed.frontmatter.description.chars().count();
if actual > WORKFLOW_DESCRIPTION_HARD_CAP {
report.push_error(WorkflowLintError::DescriptionTooLong {
actual,
limit: WORKFLOW_DESCRIPTION_HARD_CAP,
});
}
}
let knowledge = match scan_knowledge_slugs(&self.layout) {
Ok(knowledge) => knowledge,
Err(err) => {
report.push_error(WorkflowLintError::MalformedFrontmatter(format!(
"failed to scan existing Knowledge records: {err}"
)));
return report;
}
};
for slug in &parsed.frontmatter.requires {
if !knowledge.contains(slug) {
report.push_error(WorkflowLintError::UnknownReference {
field: "requires",
kind: "knowledge",
slug: slug.to_string(),
});
}
}
report
}
}
struct Parsed<'a, F> {
frontmatter: F,
body: &'a str,
}
fn parse_frontmatter<F: DeserializeOwned>(
content: &str,
) -> Result<Parsed<'_, F>, WorkflowLintError> {
let (yaml, body) = split_frontmatter(content)?;
let frontmatter = serde_yaml::from_str::<F>(yaml).map_err(|err| {
let msg = err.to_string();
if let Some(field) = parse_missing_field(&msg) {
WorkflowLintError::MissingField(field)
} else {
WorkflowLintError::MalformedFrontmatter(msg)
}
})?;
Ok(Parsed { frontmatter, body })
}
fn parse_missing_field(msg: &str) -> Option<&'static str> {
let needle = "missing field `";
let start = msg.find(needle)? + needle.len();
let end = msg[start..].find('`')? + start;
match &msg[start..end] {
"description" => Some("description"),
"model_invokation" => Some("model_invokation"),
"user_invocable" => Some("user_invocable"),
"requires" => Some("requires"),
_ => None,
}
}
fn scan_knowledge_slugs(layout: &WorkspaceLayout) -> std::io::Result<HashSet<Slug>> {
let mut out = HashSet::new();
let entries = match std::fs::read_dir(layout.knowledge_dir()) {
Ok(entries) => entries,
Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(out),
Err(err) => return Err(err),
};
for entry in entries {
let entry = entry?;
let path = entry.path();
if !path.is_file() || path.extension().and_then(|s| s.to_str()) != Some("md") {
continue;
}
let Some(stem) = path.file_stem().and_then(|s| s.to_str()) else {
continue;
};
if let Ok(slug) = Slug::parse(stem) {
out.insert(slug);
}
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::TempDir;
fn write(path: &std::path::Path, content: &str) {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).unwrap();
}
std::fs::write(path, content).unwrap();
}
fn workspace() -> (TempDir, WorkflowLinter) {
let dir = TempDir::new().unwrap();
let layout = WorkspaceLayout::new(dir.path().to_path_buf());
(dir, WorkflowLinter::new(layout))
}
#[test]
fn workflow_lint_accepts_valid_file() {
let (dir, linter) = workspace();
write(
&dir.path().join(".insomnia/knowledge/policy.md"),
"---\ndescription: p\n---\nbody",
);
let wf = "---\ndescription: run\nrequires: [policy]\n---\nbody";
let report = linter.lint(wf);
assert!(!report.has_errors(), "{:?}", report.errors);
}
#[test]
fn workflow_lint_rejects_missing_required_knowledge() {
let (_dir, linter) = workspace();
let wf = "---\ndescription: run\nrequires: [ghost]\n---\nbody";
let report = linter.lint(wf);
assert!(report.errors.iter().any(|err| matches!(
err,
WorkflowLintError::UnknownReference { field: "requires", kind: "knowledge", slug }
if slug == "ghost"
)));
}
#[test]
fn workflow_lint_enforces_resident_description_cap() {
let (_dir, linter) = workspace();
let desc = "x".repeat(WORKFLOW_DESCRIPTION_HARD_CAP + 1);
let wf = format!("---\ndescription: {desc}\nmodel_invokation: true\n---\nbody");
let report = linter.lint(&wf);
assert!(
report
.errors
.iter()
.any(|err| matches!(err, WorkflowLintError::DescriptionTooLong { .. }))
);
}
#[test]
fn workflow_lint_enforces_body_limit() {
let (_dir, linter) = workspace();
let body = "x".repeat(WORKFLOW_BODY_LIMIT + 1);
let wf = format!("---\ndescription: run\n---\n{body}");
let report = linter.lint(&wf);
assert!(
report
.errors
.iter()
.any(|err| matches!(err, WorkflowLintError::BodyTooLong { .. }))
);
}
}

View File

@ -0,0 +1,90 @@
//! Workflow frontmatter schema and frontmatter splitting helpers.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use crate::{Slug, WorkflowLintError};
pub const WORKFLOW_BODY_LIMIT: usize = 8000;
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct WorkflowFrontmatter {
/// Workflows do not require timestamps in the MVP. Human-authored files
/// may carry them.
#[serde(default)]
pub updated_at: Option<DateTime<Utc>>,
#[serde(default)]
pub created_at: Option<DateTime<Utc>>,
pub description: String,
#[serde(default)]
pub model_invokation: bool,
#[serde(default = "default_user_invocable")]
pub user_invocable: bool,
#[serde(default)]
pub requires: Vec<Slug>,
}
fn default_user_invocable() -> bool {
true
}
const FRONTMATTER_DELIM: &str = "---";
/// Split a markdown document into `(yaml_frontmatter, body)`.
pub fn split_frontmatter(content: &str) -> Result<(&str, &str), WorkflowLintError> {
let after_open = content
.strip_prefix(FRONTMATTER_DELIM)
.and_then(|s| s.strip_prefix('\n').or(Some(s)))
.ok_or(WorkflowLintError::MissingFrontmatter)?;
let mut yaml_end = None;
let mut byte_offset = 0usize;
for line in after_open.split_inclusive('\n') {
let trimmed = line.trim_end_matches('\n').trim_end_matches('\r');
if trimmed == FRONTMATTER_DELIM {
yaml_end = Some((byte_offset, byte_offset + line.len()));
break;
}
byte_offset += line.len();
}
let (yaml_end_excl, body_start) = yaml_end.ok_or_else(|| {
WorkflowLintError::MalformedFrontmatter("missing closing `---` line".to_string())
})?;
let yaml = &after_open[..yaml_end_excl];
let body = &after_open[body_start..];
Ok((yaml, body))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn splits_simple() {
let doc = "---\nfoo: 1\n---\nbody here\n";
let (y, b) = split_frontmatter(doc).unwrap();
assert_eq!(y, "foo: 1\n");
assert_eq!(b, "body here\n");
}
#[test]
fn no_leading_delim_errors() {
let err = split_frontmatter("hello").unwrap_err();
assert!(matches!(err, WorkflowLintError::MissingFrontmatter));
}
#[test]
fn no_closing_delim_errors() {
let err = split_frontmatter("---\nfoo: 1\nno close\n").unwrap_err();
assert!(matches!(err, WorkflowLintError::MalformedFrontmatter(_)));
}
#[test]
fn handles_empty_body() {
let doc = "---\nfoo: 1\n---\n";
let (_, b) = split_frontmatter(doc).unwrap();
assert_eq!(b, "");
}
}

View File

@ -0,0 +1,36 @@
//! Scope deny helpers for human-authored Workflow files.
use std::path::Path;
use manifest::{Permission, ScopeRule};
use memory::WorkspaceLayout;
/// Build deny rules that strip Write permission from
/// `<workspace>/.insomnia/workflow/` for generic CRUD tools.
pub fn deny_write_rules(layout: &WorkspaceLayout) -> Vec<ScopeRule> {
vec![deny_write(layout.workflow_dir().as_path())]
}
fn deny_write(target: &Path) -> ScopeRule {
ScopeRule {
target: target.to_path_buf(),
permission: Permission::Write,
recursive: true,
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
#[test]
fn deny_targets_workflow() {
let layout = WorkspaceLayout::new(PathBuf::from("/ws"));
let rules = deny_write_rules(&layout);
assert_eq!(rules.len(), 1);
assert_eq!(rules[0].target, PathBuf::from("/ws/.insomnia/workflow"));
assert_eq!(rules[0].permission, Permission::Write);
assert!(rules[0].recursive);
}
}

View File

@ -19,10 +19,9 @@ use serde::Deserialize;
use thiserror::Error;
use tracing::warn;
use crate::error::LintError;
use crate::schema::split_frontmatter;
use crate::slug::Slug;
use crate::workflow::{WORKFLOW_DESCRIPTION_HARD_CAP, WorkflowRecord, WorkflowSource};
use crate::{Slug, WorkflowLintError};
/// Filename within a skill directory carrying the frontmatter + body.
pub const SKILL_FILENAME: &str = "SKILL.md";
@ -34,6 +33,7 @@ pub const SKILL_FILENAME: &str = "SKILL.md";
/// `metadata` are documentary, while `allowed-tools` is recognised and
/// emits a warning until [`permission-extension-point.md`] lands.
#[derive(Debug, Clone, Deserialize)]
#[allow(dead_code)]
pub struct SkillFrontmatter {
pub name: String,
pub description: String,
@ -49,7 +49,7 @@ pub struct SkillFrontmatter {
/// Validated skill record. Constructed by [`parse_skill_md`] and converted
/// to a `WorkflowRecord` by the caller via the `Skill → Workflow`
/// projection in [`crate::workflow`].
/// projection in [`crate::WorkflowRecord`].
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SkillRecord {
pub slug: Slug,
@ -94,7 +94,7 @@ pub enum SkillParseError {
Frontmatter {
path: PathBuf,
#[source]
source: LintError,
source: WorkflowLintError,
},
#[error(
"SKILL.md `name` `{name}` does not match its directory name `{dir_name}` (at {})",
@ -109,7 +109,7 @@ pub enum SkillParseError {
InvalidName {
skill_md_path: PathBuf,
#[source]
source: LintError,
source: WorkflowLintError,
},
#[error("SKILL.md `description` must be non-empty (at {})", .skill_md_path.display())]
DescriptionEmpty { skill_md_path: PathBuf },
@ -150,7 +150,7 @@ pub fn parse_skill_md(skill_md_path: &Path) -> Result<SkillRecord, SkillParseErr
let frontmatter: SkillFrontmatter =
serde_yaml::from_str(yaml).map_err(|err| SkillParseError::Frontmatter {
path: skill_md_path.to_path_buf(),
source: LintError::MalformedFrontmatter(err.to_string()),
source: WorkflowLintError::MalformedFrontmatter(err.to_string()),
})?;
if frontmatter.allowed_tools.is_some() {
@ -344,7 +344,10 @@ mod tests {
"body",
);
let record = parse_skill_md(&path).unwrap();
assert_eq!(record.description.chars().count(), WORKFLOW_DESCRIPTION_HARD_CAP);
assert_eq!(
record.description.chars().count(),
WORKFLOW_DESCRIPTION_HARD_CAP
);
}
#[test]

146
crates/workflow/src/slug.rs Normal file
View File

@ -0,0 +1,146 @@
//! Slug type and validation.
//!
//! Syntax (agent-skills compatible):
//! ^[a-z0-9](?:[a-z0-9-]{0,62}[a-z0-9])?$
//! - 164 chars
//! - lowercase ASCII alphanumerics and `-`
//! - cannot start or end with `-`
//! - no consecutive `--`
use std::fmt;
use std::str::FromStr;
use serde::{Deserialize, Deserializer, Serialize};
use crate::WorkflowLintError;
const MIN_LEN: usize = 1;
const MAX_LEN: usize = 64;
/// Validated slug. Constructible only via [`Slug::parse`].
#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize)]
#[serde(transparent)]
pub struct Slug(String);
impl Slug {
/// Parse and validate. Returns [`WorkflowLintError::InvalidSlug`] on rejection.
pub fn parse(s: impl Into<String>) -> Result<Self, WorkflowLintError> {
let s = s.into();
if is_valid_slug(&s) {
Ok(Self(s))
} else {
Err(WorkflowLintError::InvalidSlug(s))
}
}
pub fn as_str(&self) -> &str {
&self.0
}
pub fn into_string(self) -> String {
self.0
}
}
impl fmt::Display for Slug {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.0)
}
}
impl AsRef<str> for Slug {
fn as_ref(&self) -> &str {
&self.0
}
}
impl FromStr for Slug {
type Err = WorkflowLintError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
Self::parse(s)
}
}
impl<'de> Deserialize<'de> for Slug {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let raw = String::deserialize(deserializer)?;
Self::parse(raw).map_err(serde::de::Error::custom)
}
}
/// Pure-fn predicate matching the agent-skills slug regex without
/// pulling in the `regex` crate.
pub fn is_valid_slug(s: &str) -> bool {
let bytes = s.as_bytes();
let len = bytes.len();
if len < MIN_LEN || len > MAX_LEN {
return false;
}
if !is_alnum_lower(bytes[0]) || !is_alnum_lower(bytes[len - 1]) {
return false;
}
let mut prev_dash = false;
for &b in bytes {
if b == b'-' {
if prev_dash {
return false;
}
prev_dash = true;
} else if is_alnum_lower(b) {
prev_dash = false;
} else {
return false;
}
}
true
}
fn is_alnum_lower(b: u8) -> bool {
b.is_ascii_digit() || b.is_ascii_lowercase()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn accepts_basic_slugs() {
for s in ["a", "ab", "abc-def", "x9", "a-b-c", "123", "a-1"] {
assert!(is_valid_slug(s), "expected `{s}` valid");
assert!(Slug::parse(s).is_ok());
}
}
#[test]
fn rejects_bad_slugs() {
for s in [
"", "-", "-foo", "foo-", "Foo", "foo_bar", "foo bar", "foo--bar", "foo.bar", "ä",
] {
assert!(!is_valid_slug(s), "expected `{s}` invalid");
assert!(Slug::parse(s).is_err());
}
}
#[test]
fn enforces_length_bounds() {
let too_long = "a".repeat(MAX_LEN + 1);
assert!(!is_valid_slug(&too_long));
let max = "a".repeat(MAX_LEN);
assert!(is_valid_slug(&max));
}
#[test]
fn deserializes_via_serde() {
let json = "\"valid-slug\"";
let slug: Slug = serde_json::from_str(json).unwrap();
assert_eq!(slug.as_str(), "valid-slug");
let bad = "\"BAD\"";
let err: Result<Slug, _> = serde_json::from_str(bad);
assert!(err.is_err());
}
}

View File

@ -12,10 +12,10 @@ use std::path::{Path, PathBuf};
use thiserror::Error;
use tracing::warn;
use crate::error::LintError;
use crate::schema::{WorkflowFrontmatter, split_frontmatter};
use crate::slug::Slug;
use crate::workspace::WorkspaceLayout;
use memory::WorkspaceLayout;
use crate::{Slug, WorkflowLintError};
/// Hard cap on Workflow descriptions that are advertised resident.
/// Mirrors agent-skills and resident Knowledge descriptions.
@ -167,9 +167,15 @@ pub enum WorkflowLoadError {
#[error("failed to read workflow file {}: {source}", .path.display())]
ReadFile { path: PathBuf, source: io::Error },
#[error("invalid workflow file name {}: {source}", .path.display())]
InvalidSlug { path: PathBuf, source: LintError },
InvalidSlug {
path: PathBuf,
source: WorkflowLintError,
},
#[error("invalid workflow frontmatter in {}: {source}", .path.display())]
Frontmatter { path: PathBuf, source: LintError },
Frontmatter {
path: PathBuf,
source: WorkflowLintError,
},
#[error(
"Workflow {} with model_invokation: true cannot have description longer than {limit} chars (got {actual})",
.path.display()
@ -281,12 +287,12 @@ fn warn_unknown_workflow_fields(path: &Path, yaml: &str) {
}
}
fn map_serde_workflow_error(err: serde_yaml::Error) -> LintError {
fn map_serde_workflow_error(err: serde_yaml::Error) -> WorkflowLintError {
let msg = err.to_string();
if let Some(field) = parse_missing_field(&msg) {
return LintError::MissingField(field);
return WorkflowLintError::MissingField(field);
}
LintError::MalformedFrontmatter(msg)
WorkflowLintError::MalformedFrontmatter(msg)
}
fn parse_missing_field(msg: &str) -> Option<&'static str> {
@ -416,9 +422,18 @@ mod tests {
#[test]
fn merge_skill_shadows_existing_workflow() {
let (dir, layout) = setup();
write_workflow(dir.path(), "shared", "description: Internal", "internal body");
write_workflow(
dir.path(),
"shared",
"description: Internal",
"internal body",
);
let mut reg = load_workflows(&layout).unwrap();
let skill_path = dir.path().join("user-skills").join("shared").join("SKILL.md");
let skill_path = dir
.path()
.join("user-skills")
.join("shared")
.join("SKILL.md");
std::fs::create_dir_all(skill_path.parent().unwrap()).unwrap();
std::fs::write(&skill_path, "ignored").unwrap();
let incoming = WorkflowRecord {
@ -435,8 +450,14 @@ mod tests {
};
let shadow = reg.merge_skill(incoming).expect("expected shadow");
assert_eq!(shadow.slug.as_str(), "shared");
assert!(matches!(shadow.kept_source, WorkflowSource::WorkspaceWorkflow));
assert!(matches!(shadow.shadowed_source, WorkflowSource::Skill { .. }));
assert!(matches!(
shadow.kept_source,
WorkflowSource::WorkspaceWorkflow
));
assert!(matches!(
shadow.shadowed_source,
WorkflowSource::Skill { .. }
));
// The kept record is still the workspace workflow.
let kept = reg.get(&Slug::parse("shared").unwrap()).unwrap();
assert!(matches!(kept.source, WorkflowSource::WorkspaceWorkflow));

369
docs/plan/ai-maintainer.md Normal file
View File

@ -0,0 +1,369 @@
# AI maintainer 運用設計
## 位置づけ
この文書は、insomnia を単なる Coding Agent ではなく、開発プロジェクトを継続的に運用する AI maintainer として使うための上位設計である。
`/auto-maintain` はこの設計の最初の実行形であり、TODO / tickets から小さな実装作業を選んで実装・レビューを orchestration する Workflow に留まる。本設計はその一段上で、設計相談、ticket 整理、実装委譲、レビュー、運用課題の記録、改善提案までを一つの maintainer loop として扱う。
ただし、これは unattended 自動開発ではない。最終判断、危険な権限拡大、push、未合意の要件変更は人間に戻す。
## 目的
AI maintainer の目的は、コードを書くことそのものではなく、開発状態を前に進めることである。
具体的には以下を担う。
- TODO / tickets / docs / git history から現在の開発状態を把握する
- 要件が十分に固まった作業を実装可能な単位へ落とす
- 設計判断が必要な作業を実装前に人間へ戻す
- 実装 Pod / reviewer Pod / 親 Pod 自身を使い分ける
- 実装結果を ticket の完了条件に照らして review する
- merge / ticket lifecycle / TODO 整理の順序と branch placement を守る
- 作業中に見つかった運用上の障壁を `docs/report/` に残す
- 繰り返し発生する作業を Workflow / Knowledge / ticket 候補として提案する
## 役割モデル
### Human maintainer
人間は設計上の最終責任者である。
主な責務:
- プロダクト方針、設計方針、優先順位の決定
- AI が提示した選択肢の採否
- scope / permission / history persistence など安全モデルに関わる判断
- push や外部公開に関わる判断
- AI maintainer の運用方針そのものの変更承認
### AI maintainer / orchestrator
親 Pod が担う役割。コードを書くこともできるが、主な責務は制御面である。
主な責務:
- 状態把握: `TODO.md`, `tickets/`, `docs/`, git history, worktree 状態を読む
- 作業選定: 実装可能な ticket と設計相談が必要な ticket を分ける
- 計画: 作業範囲、検証方法、child worktree / Pod scope を決める
- 委譲: 実装 Pod / reviewer Pod を spawn し、必要最小 scope を渡す
- 監督: Pod 出力、worktree diff、test 結果を確認する
- レビュー: ticket の背景・要件・完了条件に対して実装を確認する
- lifecycle: review artifact、ticket 完了、TODO 更新、merge の branch placement を守る
- 報告: 完了内容、検証結果、未解決事項、次の人間判断をまとめる
### Implementation Pod
狭い write scope を持つ作業者である。実装 Pod は ticket と worktree を渡され、その範囲で実装・検証・報告を行う。
禁止事項:
- `/auto-maintain` など上位 Workflow を実行しない
- ticket / TODO / review artifact / docs/report を勝手に編集しない
- git commit / merge / push をしない
- scope / permission / history persistence を勝手に再設計しない
### Reviewer Pod
原則 read-only の検証者である。実装 Pod とは分けるのが望ましいが、小さい作業や scope 衝突がある場合は親 Pod が review してよい。
主な確認項目:
- ticket の要件を満たしているか
- 既存設計を歪めていないか
- 不要な抽象化や過剰実装がないか
- test / build の結果が妥当か
- ticket lifecycle 上、どの branch に review / completion commit を置くべきか
## 状態と成果物
AI maintainer は、状態を会話内だけに閉じ込めない。継続的な運用に必要な情報は、用途ごとに既存のファイルへ残す。
### `TODO.md`
未完了 ticket の一覧。1 ticket = 1 行。作業選定の入口であり、完了したら feature branch 側で該当行を削除する。
AI maintainer は TODO と ticket の不整合を見つけた場合、勝手に ticket を作成・削除せず、人間に報告する。
### `tickets/`
実装可能な要件単位。作業の完了条件は ticket が基準になる。
AI maintainer は ticket を読むだけでなく、git history 上の作成・更新・削除も参照して前提を把握する。
ただし、`tickets/` は長い設計相談、実装 Pod の報告、review 指摘、修正依頼、lease、artifact を thread として扱うには弱い。長期的には WorkItem / Thread 抽象を導入し、ticket は WorkItem の linked artifact または backend view として扱う。
### WorkItem / Thread
AI maintainer の上位運用単位。`tickets/` より広く、design / feature / refactor / ops / investigation を含む project coordination record として扱う。
WorkItem は Git に依存しない domain model として定義し、初期 backend は repo-managed な `work-items/` または `issues/` directory にできるようにする。正本として残すのは description、acceptance criteria、discussion thread、decision、review、status history、linked branch / worktree / commit、durable artifact metadata である。
`.insomnia` に WorkItem 正本は置かない。`.insomnia/maintainer/` は Pod run、lease cache、polling cursor、temporary inbox、local-only trial log など runtime coordination state の置き場とする。
将来 network 越し workspace / remote maintainer hub を導入する場合も、AI maintainer は file path ではなく `WorkItemStore` / `LeaseStore` 相当の interface を通す。remote backend は後回しにするが、初期 file backend の時点で Git 前提の API に固定しない。
### `tickets/*.review.md`
review の判断根拠。review artifact は対象 feature branch 側に commit する。`develop` の first-parent に review / completion commit を直接置かない。
### `docs/plan/`
長期設計、概念設計、運用設計を置く。実装 ticket より上位の合意形成に使う。
本ファイルは `/auto-maintain` 単体ではなく、AI maintainer 全体の設計を置く場所である。
### `docs/report/`
実際の運用で発生した障壁や改善案を残す。明確な力不足、tool 問題、workflow 問題、ユーザーからの指示があった場合に作る。
report は愚痴ではなく、後から改善 ticket / Workflow 改訂 / Knowledge 化へつなげる観測記録である。
### `.insomnia/workflow/`
実行可能な手順。Workflow は人間が書く、または AI が提案して人間が承認する。AI が勝手に自律生成しない。
### memory / Knowledge
作業中の判断や設計を再利用可能な知識として扱う。ただし、turn を跨ぐ根拠を history に残さず context だけへ差し込むことは禁止する。
## 権限モード
AI maintainer の運用は、作業ごとに権限モードを明示する。
### Mode 0: Consultation
設計相談・方針整理のみ。ファイル編集しない。
使う場面:
- 要件が曖昧
- 複数の設計方針が自然に導ける
- ticket 化する前に合意したい
### Mode 1: Documentation / ticket maintenance
`docs/`, `tickets/`, `TODO.md` など制御面だけを編集する。実装コードは触らない。
使う場面:
- 設計文書作成
- ticket の詳細化
- report 作成
- TODO と ticket の整合確認
注意点:
- 新 ticket 作成や既存 ticket の大幅変更は、人間の合意後に行う
- 完了削除は対象 feature branch 側で行う
### Mode 2: Delegated implementation
child worktree を作り、実装 Pod に狭い write scope を渡して実装させる。親 Pod は orchestration と review を行う。
使う場面:
- 要件と完了条件が明確
- 影響範囲が限定されている
- test / build で確認可能
### Mode 3: Maintainer execution with explicit git authority
人間が明示的に許可した場合に、親 Pod が commit / merge / ticket 完了削除まで行う。
使う場面:
- 「レビューして問題なければマージして」など、明示的な実行指示がある
- 実装 branch / review / ticket lifecycle の配置が明確
制約:
- push は行わない
- unrelated dirty changes を commit に混ぜない
- review / completion commit は feature branch 側に置く
- `develop` first-parent には merge commit だけが載る形を目標にする
## 基本 loop
AI maintainer の基本 loop は以下である。
1. 状態把握
- `git status --short --branch`
- `TODO.md`
- 対象 ticket
- 関連 docs / report
- 既存 review artifact
2. 分類
- 実装可能
- 設計相談が必要
- ticket 整理が必要
- 運用上の問題として report すべき
3. 方針提示または実行
- 設計判断が必要なら、人間に質問する
- 実装可能なら、worktree / Pod / scope / test 方針を決める
4. 実装
- 原則 child worktree
- 実装 Pod に任せるか、親 Pod が直接行うかを選ぶ
- `.insomnia` は child worktree から除外する
5. 検証
- focused test
- 必要なら broader test
- `git diff --check`
- `cargo fmt --check` は既存 unrelated 差分を区別して扱う
6. Review
- ticket の背景・要件・完了条件に照らす
- 過剰実装や設計歪みを見る
- 必要なら修正依頼
7. Lifecycle
- implementation commit
- review commit
- ticket / review / TODO completion commit
- develop への merge
- first-parent 確認
8. 報告
- 実装概要
- 変更ファイル
- test 結果
- review 判断
- 未解決事項
- 残った dirty changes
## エスカレーション基準
以下では AI maintainer は実装を止め、人間に戻す。
- ticket の要件から複数の設計方針が自然に導ける
- 長期構造、crate boundary、protocol、permission、scope、history persistence に触れる
- prompt context 加工原則に関わる
- 新 ticket の作成、既存 ticket の大幅変更、ticket 完了削除について合意がない
- git commit / merge / push などの書き込み権限が明示されていない
- test 不能、再現不能、または作業範囲外の不具合に遭遇した
- child worktree に `.insomnia` が出てしまった
- SpawnedPod の完了状態が不明で、worktree 状態からも安全に判断できない
## Orchestration の制約
### SpawnedPod の完了は自動検知しない
現状、親 Pod は SpawnedPod の完了を push 通知として自動検知しない。完了確認は以下で行う。
- `ReadPodOutput` による polling
- Pod が出した完了報告
- `stopped` 状態
- worktree の `git status` / `git diff`
- build / test 結果
そのため、AI maintainer は「Pod が何も言わないが実は完了している」ケースと「Pod が止まっているが失敗した」ケースを区別するため、Pod 出力だけでなく worktree 状態を確認する。
### Scope は所有権として扱う
実装 Pod に write scope を渡している間、親 Pod は同じ path を編集しない。review artifact や ticket lifecycle を書く前に、必要なら `StopPod` して scope を回収する。
### main workspace は制御面
main workspace は orchestration / docs / ticket lifecycle / merge のための場所である。実装差分は原則 child worktree に隔離する。
## Git / ticket lifecycle
feature branch の ticket lifecycle は以下の形を正とする。
```text
* merge: <topic> # develop
|\
| * docs(tickets): complete <topic>
| * review: <topic>
| * feat/fix/refactor: <topic> # feature branch
|/
* previous develop
```
重要な検査:
- review / completion commit は feature branch 側にある
- `develop` first-parent に review / completion commit が直接載っていない
- unrelated dirty changes が staged / committed されていない
- ticket / review / TODO cleanup は merge 前に feature branch 側で完了している
## `/auto-maintain` との関係
`/auto-maintain` は、この設計の Mode 2 を安全側に制限した Workflow である。
現在の `/auto-maintain` は以下に留める。
- TODO / tickets を読む
- 将来的には WorkItemStore を入口にして、ticket は linked artifact または backend view として扱う
- 低リスク ticket を選ぶ
- worktree / implementation Pod / reviewer を orchestration する
- 完了候補を報告する
- 原則として commit / merge / ticket 完了削除は人間に戻す
一方、この文書が扱う AI maintainer は、ユーザーが明示的に許可した場合に Mode 3 として commit / merge / ticket lifecycle まで実行できる。ただし push はしない。
## 今後必要な機能
この設計を安定運用するには、以下が必要になる。
### WorkItemStore / LeaseStore
`tickets/` file を直接読むだけでなく、AI maintainer が WorkItem / Thread / Event / Artifact を扱うための store interface。初期実装は repo-managed file backend でよいが、network 越し workspace / remote maintainer hub へ差し替えられるよう、Git 操作や path layout を上位 Workflow に漏らさない。
Lease は thread の正本とは分け、`.insomnia/maintainer/` または local DB に置く runtime coordination state として扱う。remote hub を導入する場合は LeaseStore だけを先に集権化できる設計にする。
### Maintainer doctor
merge 前 / ticket 完了前に以下を検査するコマンド。
- current branch
- feature branch と develop の merge base
- review / completion commit の branch placement
- unrelated staged changes
- ticket / TODO の整合
- child worktree に `.insomnia` が含まれていないこと
### Pod completion tracking
SpawnedPod の状態を polling ではなく、親 Pod が扱いやすいイベントとして観測できる仕組み。
必要な情報:
- running / completed / failed / stopped
- 最終 assistant output
- 最終 tool result
- worktree path
- delegated scope
### Operation inbox / trial log
maintainer が見つけた改善候補、試走結果、未整理の気づきを一時的に集める場所。恒久的な ticket にする前の buffer として使う。
### Profile / role manifest
Orchestrator / Coder / Reviewer / Researcher で model・prompt・scope 既定値を分けるための manifest profile。人間が用途ごとに起動時設定を切り替えられるようにする。
### Workflow quality evaluation
Workflow 本文が実際に subagent に渡された時、どこで裁量補完が発生し、どこが曖昧だったかを構造化して report する仕組み。
## 非目標
当面やらないこと:
- 常駐 scheduler による unattended 開発
- AI による push
- AI による無承認の ticket 新規作成 / 大幅変更
- AI による Workflow 自律生成
- scope owner handoff の暗黙化
- history に残らない情報を context へ差し込む運用
## 現時点の方針
AI maintainer は「コードを書く agent」ではなく、「プロジェクト状態を読み、必要な作業単位へ分解し、適切な worker に委譲し、結果を検証し、履歴に残す agent」として扱う。
`/auto-maintain` はその最小実行単位であり、今後はこの文書を上位設計として、Workflow 本文・doctor・profile・Pod completion tracking を段階的に足していく。

View File

@ -1,31 +0,0 @@
# Anthropic projection: assistant ターン内ブロックを 1 message に束ねる
## 背景
`crates/llm-worker/src/llm_client/scheme/anthropic/request.rs``convert_items_to_messages` は、Worker が 1 ターンで生成する `[Reasoning, assistant_message, ToolCall]` の連列を、Anthropic wire 上で **複数の隣接した assistant message** に分割している。
具体的には:
- `Item::Reasoning``pending_assistant` に push
- 次の `Item::Message { Role::Assistant }` が到来すると `pending_assistant` を flush し、自分自身は別 message として messages に直 push
- 続く `Item::ToolCall` は再び `pending_assistant` に積まれ、turn 末で flush され 3 つ目の assistant message に
結果として 1 turn が `assistant[Thinking] / assistant[text] / assistant[tool_use]` の 3 message に展開される。
Anthropic Messages API は user/assistant の交互を要求し、同一論理 turn 内の thinking/text/tool_use は **1 つの assistant message の `content` 配列** に並べる仕様。新世代 Claude (Opus 4.5+/Sonnet 4.6+) で thinking signature を round-trip する際、隣接 assistant message に分かれていると signature の文脈が崩れて 400 になる懸念があるreasoning-history-persist のレビュー指摘)。
なお、本バグは reasoning-history-persist で導入されたものではなく、`assistant_message` + `tool_call` の組合せで以前から存在していた pre-existing な分割。Reasoning が同じ flush 経路を継承した形。
## 要件
- 同一論理ターンに属する `Item::Reasoning` / `Item::Message(Assistant)` / `Item::ToolCall` を、Anthropic wire 上の **1 つの assistant message の `content` 配列** に束ねる
- 順序は arrival 順 (= history 順)。Anthropic 仕様の典型は thinking → text → tool_use
- user / system role の `Item::Message``Item::ToolResult` を境界として assistant burst を区切る
- 既存の breakpoint (cache_control) 計算が壊れないこと: 各 item のオリジン index → (msg_idx, part_idx) マッピングは flush_pending 経由で記録されているので、Item::Message(Assistant) も pending を経由するように揃えれば自然に追従する
- Single-text 専用の `AnthropicContent::Text` shorthand は assistant burst 内 1 part のみのときに限定して維持するか、簡潔さのために常に `Parts` 形式に統一するかは実装時に判断
- 既存テスト群(`completed_turn`, `single_text_message_uses_text_shorthand_without_breakpoint`, `breakpoint_on_tool_result_head` 等)の意図を逸脱しないよう更新
## スコープ外
- モデル世代別の thinking keep/strip デフォルト分岐reasoning-history-persist のフォローアップ候補と同じ扱い)
- `clear_thinking_20251015` context-edit
- prune.rs の reasoning aware 化

View File

@ -0,0 +1,61 @@
# memory / workflow の共通基盤を別 crate に切り出す
## 背景
`tickets/workflow-crate-extraction.md`(完了済 / git log 参照)で Workflow を `crates/memory/` から `crates/workflow/` に切り出した際、依存方向を「workflow → memory`WorkspaceLayout` のみ)」に限定するため、本来共通であるべき型・関数を **両 crate にコピペで重複** させて済ませている。
具体的に重複しているもの:
- **`Slug` 型と `is_valid_slug`**: `crates/memory/src/slug.rs``crates/workflow/src/slug.rs` がエラー型(`LintError::InvalidSlug` / `WorkflowLintError::InvalidSlug`)以外完全に同じ。テストごと丸ごとコピー。
- **`split_frontmatter`**: `crates/memory/src/schema/common.rs``crates/workflow/src/schema.rs` に同等の実装。返すエラー型だけ違う。
- **YAML frontmatter の `MissingFrontmatter` / `MalformedFrontmatter` バリアント**: `LintError``WorkflowLintError` の両方に重複定義。
- **`Frontmatter` trait`created_at` / `updated_at` の統一アクセス)**: 現状 memory 側だけにあり、workflow 側の `WorkflowFrontmatter` は同 trait を実装していない。共通 crate に出るなら、workflow 側でも揃えられる。
memory / workflow どちらも agent-skills 互換のスラグ規約と Markdown + YAML frontmatter の同一フォーマットを採用しているため、これらは設計上「両者が共有すべき同一の概念」であって、別物として持つ理由はない。`tickets/workflow-crate-extraction.md` も完了条件と直交する形で「共有が必要なら共通部分を別 crate例: `crates/lint-common/`)に切る判断を行う」と前置きしており、抽出時にスキップした判断を本チケットで補う。
## 要件
### 新 crate の新設
memory / workflow 双方が依存する共通 crate を 1 つ立てる。crate 名は実装時に決める(候補: `lint-common`, `record-core`, `frontmatter` など。memory / workflow より下層に位置し、両者が import する。
新 crate が持つもの:
- `Slug` 型 + `is_valid_slug`agent-skills 互換規約)
- `split_frontmatter`YAML frontmatter / Markdown body 分離)
- 上記に紐づく共通エラー型(`InvalidSlug` / `MissingFrontmatter` / `MalformedFrontmatter`
- `Frontmatter` trait`BODY_LIMIT` / `created_at` / `updated_at` のアクセサ)
### memory / workflow からの重複削除
- `crates/memory/src/slug.rs``crates/workflow/src/slug.rs` を削除し、新 crate の `Slug` を再 export または直接 import する形に書き換える
- `crates/memory/src/schema/common.rs` 内の `split_frontmatter``crates/workflow/src/schema.rs` 内の `split_frontmatter` を新 crate のものに統合
- `LintError` / `WorkflowLintError``InvalidSlug` / `MissingFrontmatter` / `MalformedFrontmatter` バリアントは、共通エラー型を `#[from]` で包む形に揃えるか、共通エラー型をそのまま使う形に切り替える(実装時に判断)
- `WorkflowFrontmatter` も共通 `Frontmatter` trait を実装するように揃える(`BODY_LIMIT` を 8000 で踏襲)
### 依存方向
- 新 crate は memory / workflow / その他に依存しない(純粋なドメイン型のみ)
- memory / workflow 双方が新 crate を import する
- workflow → memory の `WorkspaceLayout` 依存は維持(このチケットの対象外)
## 範囲外
- linter 本体の共通化memory `Linter` と workflow `WorkflowLinter` の統合)
- `WorkspaceLayout` の memory crate からの切り出し
- `WorkflowFrontmatter` / `KnowledgeFrontmatter` 等のスキーマ変更
- agent-skills 互換規約自体の変更
## 完了条件
- 新 crate が `Slug` / `is_valid_slug` / `split_frontmatter` / 共通エラー型 / `Frontmatter` trait を提供している
- `crates/memory/src/slug.rs``crates/workflow/src/slug.rs` の重複コードが消えている(少なくとも一方からは)
- `split_frontmatter` の実装が 1 箇所に集約されている
- `WorkflowFrontmatter``Frontmatter` trait を実装している
- 既存テストmemory / workflow / podが新構造で通る
- 循環依存が無い
## 参照
- 直前: `tickets/workflow-crate-extraction.md`git log、`workflow-crate-extraction.review.md` で本件が見落とされた経緯あり)
- 関連: `tickets/internal-worker-workflow.md`(本チケット完了後に着手すると共通基盤が揃った状態で進められる)

View File

@ -0,0 +1,284 @@
# AI maintainer 用 WorkItem / Thread 抽象
## 背景
現在の開発運用は `TODO.md``tickets/*.md` を中心に回している。これは Git 履歴で要件と完了条件を追うには十分だが、AI maintainer が単なる Coding Agent を超えて運用を担うには弱い。
特に、設計相談、実装 Pod の作業報告、review 指摘、修正依頼、完了判断、Pod run、lease、artifact が ticket file / review file / 会話 / git log に分散し、thread として扱いづらい。
shiguredo/http3-rs の `issues/` directory のように、repository 内に issue / work item を置く運用は参考になる。一方で、同 repository の owner も指摘している通り、中央の `SEQUENCE` ファイルによる連番採番は並列 branch / worktree で conflict しやすい。将来的な network 越し workspace / remote coordination も想定すると、最初から Git directory や中央採番前提で API を固めるべきではない。
本チケットでは、`tickets/` を直ちに置き換えるのではなく、AI maintainer が扱う上位の **WorkItem / Thread / Event / Lease / Artifact** 抽象を設計し、最小 file backend を導入できる状態にする。
## 方針
- WorkItem / Thread の正本は `.insomnia` ではなく、project-visible な repo-managed 領域に置く
- `.insomnia` は local runtime state / memory / workflow / Pod run / lease cache の領域として分離する
- API / domain model は Git に依存しない形にする
- WorkItem ID は中央 `SEQUENCE` 連番ではなく、作成時刻ベースの衝突しにくい ID にする
- 初期 backend は repo 内 directory例: `work-items/` または `issues/`)でよい
- network 越し workspace / remote hub は後回しにするが、将来差し替え可能な interface を先に切る
- 既存 `tickets/` は当面維持し、WorkItem から link するか、file backend の一 view として扱えるようにする
## データ配置
初期設計では以下の分離を前提にする。
```text
repo/
work-items/ or issues/ # project-visible, git-managed coordination data
tickets/ # 当面の既存 ticket
docs/ # 設計・report
.insomnia/ # local agent/runtime state
memory/
workflow/
maintainer/
leases/
runs/
inbox/
```
### repo-managed に置くもの
- WorkItem description
- acceptance criteria
- discussion thread
- design decision
- review comment
- status history
- linked branch / worktree / commit
- durable artifact metadata
### `.insomnia` に置くもの
- Pod run state
- lease の local cache
- SpawnedPod polling cursor
- temporary inbox
- local-only trial log
- model / role runtime state
## WorkItem ID
WorkItem ID は identity のためだけに使い、priority や処理順序を背負わせない。`SEQUENCE` のような中央連番ファイルは、複数 branch / worktree / Pod が同時に WorkItem を作ると conflict しやすいため採用しない。
初期 file backend では、directory name を immutable ID として扱う。
```text
YYYYMMDD-HHMMSS-<slug>
YYYYMMDD-HHMMSS-<short-rand>-<slug> # 同一秒衝突を避けたい場合
```
例:
```text
20260510-184233-maintainer-work-items
20260510-184233-a1b2-maintainer-work-items
```
要件:
- lexical sort で概ね作成順になる
- 中央採番ファイルを更新しない
- collision 時は backend が短い random suffix や retry で解決する
- human-visible `slug` / `title` と immutable `id` を分ける
- priority / status / scheduling は `id` ではなく metadata で表す
- 将来 remote backend に移る場合も ID 生成責務は backend 側に閉じ込める
## WorkItem model
最低限、以下の概念を持つ。
```text
WorkItem
- id
- slug
- title
- status
- kind: feature | bug | refactor | design | ops | investigation
- priority / labels
- owner / assignee / current lease summary
- acceptance criteria
- linked ticket / docs / branches / worktrees / commits
- thread events
- artifacts
```
```text
ThreadEvent
- id
- work_item_id
- occurred_at
- author: human | orchestrator | pod:<name>
- role: comment | plan | decision | implementation_report | review | status_change | escalation | artifact
- reply_to?
- body
- links
```
```text
Lease
- id
- work_item_id
- holder
- scope hint
- worktree path
- expires_at
- status: active | released | expired
```
Lease はリアルタイム coordination に近いため、Git-managed thread の正本とは分ける。file backend 初期実装では `.insomnia/maintainer/leases/` か local DB を使ってよい。
## backend interface
AI maintainer / `/auto-maintain` は file path を直接前提にせず、概念上は store interface を通す。
```text
WorkItemStore
- list(filter)
- get(id)
- create(item)
- append_event(id, event)
- update_status(id, status)
- attach_artifact(id, artifact)
```
```text
LeaseStore
- acquire(work_item_id, holder, scope, ttl)
- refresh(lease_id)
- release(lease_id)
- list_active(filter)
```
初期 backend 候補:
- `file://<repo>/work-items`
- `file://<repo>/issues`
- `sqlite://<workspace>/.insomnia/maintainer/work-items.db`
将来 backend 候補:
- `http://maintainer-hub/...`
- `github://owner/repo/issues`
## Query / listing
初期 file backend の list / query は、index や DB を作らず全 `item.md` frontmatter scan で行う。
対象:
```text
work-items/{open,pending,closed}/*/item.md
```
`item.md` の frontmatter に query 用 metadata を集約する。
```yaml
---
id: 20260510-184233-a1b2-maintainer-work-items
slug: maintainer-work-items
title: AI maintainer 用 WorkItem / Thread 抽象
status: open
kind: design
priority: P2
labels: [maintainer, workflow]
created_at: 2026-05-10T18:42:33Z
updated_at: 2026-05-10T19:10:00Z
assignee: null
---
```
`WorkItemStore::list` / `query` は frontmatter だけを読み、summary を返す。
- status / kind / labels / priority / assignee で filter する
- title / slug / description excerpt の軽い substring query を提供する
- sort は priority / updated_at / created_at を metadata で行う
- `thread.jsonl` は一覧では読まず、`get(id)` / thread read 時だけ読む
- `updated_at` は item metadata として持ち、必要なら thread append 時に更新する
当面の件数では全件 scan で十分であり、AI に directory を探索させて候補を推測させない。index / SQLite / search daemon は件数増加、全文検索、remote backend 同期が必要になった時点で検討する。
### status の二重管理
file backend では directory と frontmatter の両方に status が出る。
```text
work-items/open/<id>/item.md
frontmatter: status: open
```
これは人間の `ls` と backend abstraction の両方を成立させるため許容する。ただし linter / doctor で一致確認する。
- `work-items/open/*/item.md``status: open`
- `work-items/pending/*/item.md``status: pending`
- `work-items/closed/*/item.md``status: closed`
不一致は warning ではなく error とする。
## 初期 file backend 案
```text
work-items/
open/
20260510-184233-maintainer-work-items/
item.md
thread.jsonl
artifacts/
review.md
test-log.txt
pending/
20260510-190102-transport-parameter-api/
item.md
thread.jsonl
artifacts/
closed/
20260510-201522-anthropic-burst-bundling/
item.md
thread.jsonl
resolution.md
artifacts/
```
`item.md` は human-readable な issue 本文(背景、根拠、完了条件、非目標など)を持つ。`thread.jsonl` は append-only を基本にし、AI maintainer が conversation / decision / review / status change を追いやすい形にする。`resolution.md` は close 時の解決方法や検証結果を、人間が読みやすい形でまとめる任意ファイルとする。
## `/auto-maintain` との関係
`/auto-maintain` は当面 `TODO.md` / `tickets/` を読むが、将来的には WorkItemStore を入口にする。
移行方針:
1. 既存 `tickets/` は維持
2. WorkItem 抽象と file backend schema を設計する
3. 新しい設計相談・並列作業・長い thread が必要な作業だけ WorkItem 化する
4. ticket は WorkItem の linked artifact として扱う
5. `work-items/open/` が安定したら、`TODO.md` は generated view または廃止候補にする
6. 十分に安定したら `tickets/` を WorkItem backend の view に寄せる
## 範囲外
- remote maintainer hub の実装
- index / SQLite / search daemon による query 最適化
- 既存 `tickets/` の即時移行
- 常駐 scheduler
- Pod lifecycle / completion tracking の完全実装
- scope owner handoff の再設計
## 完了条件
- WorkItem / Thread / Event / Lease / Artifact の domain model が docs に定義されている
- repo-managed coordination data と `.insomnia` local runtime state の分担が明文化されている
- `WorkItemStore` / `LeaseStore` 相当の interface 方針が決まっている
- list / query は初期実装では全 `item.md` frontmatter scan で行う方針になっている
- `thread.jsonl` は一覧では読まず、詳細 read 時だけ読む方針になっている
- directory status と frontmatter status の一致を linter / doctor で確認する方針になっている
- WorkItem ID scheme が中央連番ではなく timestamp-based になっている
- 初期 file backend の directory schema が決まっている
- `/auto-maintain` / AI maintainer が将来 WorkItemStore を入口にできる移行方針が書かれている
- network 越し workspace / remote hub は後回しにしつつ、backend 差し替え可能性を潰していない
## 参照
- `docs/plan/ai-maintainer.md`
- `tickets/auto-maintain-workflow.md`
- `docs/report/2026-05-10-ticket-lifecycle-branch-placement.md`

View File

@ -1,54 +0,0 @@
# Memory: extract / consolidation 呼称を extract / consolidation に統一
## 背景
メモリサブシステムは公開 API 側では既に `extract` / `consolidate` (consolidation) を識別子として採用している:
- モジュール: `crates/memory/src/extract/`, `crates/memory/src/consolidate/`
- Manifest フィールド: `extract_model`, `extract_threshold`, `extract_worker_max_input_tokens`, `consolidation_model`, `consolidation_threshold_files`, `consolidation_threshold_bytes` (`crates/manifest/src/lib.rs:99-131`)
- `pub fn` / `pub struct``phase` を含む識別子は 0 件、共通の `Phase` enum / trait も存在しない
一方、コメント・ログ・エラー文字列・LLM プロンプト・ドキュメントには「extract」「consolidation」という呼称が広く残っている。設計的に "phase" という抽象が不要であるにもかかわらず二重の語彙が並走している状態で、新規読者にとって「extract = extract、consolidation = consolidation」というマッピングを暗黙に要求している。
特に LLM プロンプト (`resources/prompts/internal/memory_*_system.md`, `crates/memory/src/consolidate/input.rs:57-59`) に "consolidation" が出現しているのは、モデルの行動を機能名ではなく序数で誘導している形になり、置き換えるとついでに改善になる。
## 方針
"extract" を "extract"、"consolidation" を "consolidation" に置き換える。コードに `phase` という共通抽象が無いことを名前の上でも明示する。
「consolidation 内部の `consolidation phase` / `tidy phase`」のように phase という語が **階層的に再利用されている表現**は単純置換すると壊れるので、その箇所だけ語彙ごと整理する(例: `consolidation step` / `tidy step`、または「consolidation の統合パート / 整理パート」など、文脈に応じて)。
## 要件
- Rust コード (`crates/`) 内の doc comment / 通常コメント / ログメッセージ / `thiserror``#[error]` 文字列 / テスト関数名・コメントから "extract" / "consolidation" を排除し、`extract` / `consolidation` に置き換える。
- LLM プロンプト用の固定文字列 (`crates/memory/src/consolidate/input.rs` の `consolidation input. Run the consolidation phase first ...`) を `consolidation` 系の語彙のみで再構成する。`resources/prompts/internal/memory_extract_system.md` と `resources/prompts/internal/memory_consolidation_system.md``phase` 言及も同様に置換する。
- `docs/plan/memory.md` と関連 plan / ref 文書 (`docs/plan/memory-effectiveness.md`, `docs/plan/memory-prompts.md`, `docs/ref/memory-systems.md`, `docs/ref/opencode-comparison.md` の memory 周辺) の "extract / consolidation" を改稿する。章立てや見出しに使われている場合は「## Extract」「## Consolidation」の構成に置き換える。
- `phase` という語を残してよいのは「(memory と無関係な) tool dispatch / TUI / worker 内部の局所的なフェーズ表現」(例: `crates/llm-worker/src/worker.rs:747,795`、`crates/tui/src/spawn.rs:148,182`、`crates/tui/src/main.rs:209`)。これは memory の話ではないので対象外。
- 後方互換 shim は不要(公開 API 名は変更しない、テキストの置き換えのみ)。
## 完了条件
- `git grep -i 'extract\|consolidation\|phase1\|phase2'` の結果から、memory サブシステムに紐づく hit が 0 になるTUI / tool dispatch / 一般語 "thinking phase" は除く)。
- LLM プロンプト 3 ファイル (`memory_extract_system.md`, `memory_consolidation_system.md`, `consolidate/input.rs` の inline テキスト) で序数表現が消え、機能名のみで指示が成立している。
- `docs/plan/memory.md` の見出し構造と本文が `Extract` / `Consolidation` ベースに揃っている。
## 範囲外
- 公開 API 名 (`extract_*`, `consolidation_*`) の rename。既に統一されているので変更しない。
- メモリの設計・挙動変更。純粋に呼称の整理のみ。
- TUI / worker / tool dispatch などメモリ外の "extract / 2" 言及。
- ファイル名・モジュール名の変更(既に `extract/` / `consolidate/`)。
## 影響範囲
- `crates/manifest/src/lib.rs`, `crates/manifest/src/defaults.rs`: 該当 doc comment。
- `crates/memory/src/extract/{mod,input,payload,pointer,staging,tool}.rs`: doc comment。
- `crates/memory/src/consolidate/{mod,input,lock,staging,tidy}.rs`: doc comment、ログ、`#[error]`、LLM プロンプト inline 文字列。
- `crates/memory/src/workspace.rs:186`: コメント 1 件。
- `crates/llm-worker/src/token_counter.rs:11`, `crates/pod/src/compact/token_counter.rs:154`: doc comment。
- `crates/pod/src/prompt/catalog.rs:64,66`: doc comment。
- `crates/pod/tests/compact_events_test.rs`: テスト関数名 (`compact_resets_extract_pointer_so_phase1_can_fire_again` 他) とコメント。
- `crates/pod/tests/consolidation_test.rs`: コメント。
- `resources/prompts/internal/memory_extract_system.md`, `resources/prompts/internal/memory_consolidation_system.md`: プロンプト本文。
- `docs/plan/memory.md`, `docs/plan/memory-effectiveness.md`, `docs/plan/memory-prompts.md`, `docs/ref/memory-systems.md`, `docs/ref/opencode-comparison.md` の該当箇所。
- `docs/manifest.toml`: 該当があれば。

View File

@ -1,60 +0,0 @@
# メモリ機構: 使用頻度メトリクス + Knowledge 化候補レポート
## 背景
`docs/plan/memory.md` §使用頻度メトリクス の実装。memory 検索ツール / Knowledge 検索ツール内に invoke 計測フックを入れ、時間単位ではなく累積 input token で正規化した頻度を算出する。consolidation 統合 step の Knowledge 新規作成 gate と consolidation 整理 step の保護閾値の両方で使われる。
## 要件
### 観測経路
- memory 検索ツール / Knowledge 検索ツール内に hook を挿入
- `#<slug>`slug 完全一致経路)、`/<slug>`workflow 側、将来接続)、明示検索呼び出しをすべて同じ経路に集約
### カウント対象
- **明示 invoke**: 検索ツール経由の読み取り / `#<slug>` / `/<slug>``n回 / Mtoken` でスコア化
- **`model_invokation` 注入**: 明示 invoke の分子には含めない。**コスト側**(注入した record に対する消費 input tokensとして別途記録。使われ率 ratio や ON/OFF 判断材料として後段で参照
- ファイル token 数
### 記録先
- staging と独立
- workspace 側に記録session データ喪失で統計が消えない)
- invoke event を UUID + Stats 形式
- 具体 schema / フォーマットは実装で決定
### 集計
- 累積方式: 最大 10 回前の invoke から現在までの時系列窓でフィルタして集計
- **Knowledge 化候補レポート**:
- 対象は `memory/*` 配下の recordextract 成果物の decisions / requests + 既存 knowledge
- 明示 invoke 頻度が閾値超過のものを列挙
- 同一 session 内の連続参照は 1 count に丸める
- 複数 session での再参照を要件とするspike 除外)
- 閾値は設定ファイルで tune
### 消費者
- consolidation Worker の統合 step 入力として候補レポートを渡す
- consolidation Worker の整理 step で保護閾値判定(明示 invoke frequency >= 1.0 invokes/Mtokenに使う
## 範囲外
- consolidation 整理 step の実装本体(`memory-consolidation` 側。本チケットは保護閾値判定に必要なメトリクスの提供まで)
- `model_invokation` ON/OFF の自動判定ロジック(将来検討)
- Shallow request の自動除外(将来検討)
## 完了条件
- 検索ツール呼び出しで invoke event が workspace 側に積まれる
- `model_invokation` 注入のコスト側集計が別口で積まれる
- 候補レポート API が consolidation Worker の起動時に呼べる
- 閾値未満の record は候補レポートに載らない
- 同一 session 内連続参照は 1 count に丸まる
## 参照
- `docs/plan/memory.md` §使用頻度メトリクス / §判断ルール / §retrieval 経路
- `tickets/memory-search-tools.md`hook 挿入点)
- `tickets/memory-consolidation.md`(統合 / 整理 両 step の消費者)

View File

@ -9,8 +9,8 @@ mizchi の empirical-prompt-tuning は、agent-facing な指示Skill / slash
- evaluator Pod の session id / history
- tool call / tool result
- usage tokens
- workflow / knowledge の明示 invoke 頻度(`tickets/memory-usage-metrics.md`
- `model_invokation` 常駐注入のコスト側指標
- workflow / knowledge の明示使用ログuse 回数、last used、source breakdown。`tickets/memory-usage-metrics.md`
- `model_invokation` 常駐注入の exposure cost 指標
- extract / consolidation による recurring pattern 抽出
- Workflow 自動書き込み禁止に基づく improvement offer
@ -93,9 +93,9 @@ retries
- evaluator self-report は consolidation extract の活動抽出対象になる
- repeated `General Fix Rule` は consolidation が recurring failure pattern として統合できる
- recurring pattern は即 Knowledge 化せず、使用頻度メトリクス gate を通す
- recurring pattern は即 Knowledge 化せず、明示使用ログと Doctor / prompt-eval の事後評価を通す
- Workflow 改善は `.insomnia/workflow/*.md` へ自動書き込みせず、Notification / report / ticket などの offer に留める
- `model_invokation` ON 判断では、明示 invoke 頻度と常駐コストに加えて、eval success rate / unclear point count / description-body consistency を判断材料にする
- `model_invokation` ON 判断では、明示使用ログと resident exposure cost に加えて、eval success rate / unclear point count / description-body consistency を判断材料にする
### 評価指標の解釈
@ -140,7 +140,6 @@ Claude Code 版の `tool_uses` を、insomnia では tool 種別ごとの偏り
## 参照
- `docs/external/zenn-mizchi-empirical-prompt-tuning.md`
- `/home/hare/ghq/github.com/mizchi/skills/empirical-prompt-tuning/SKILL.md`(外部参照。取り込み時は必要最小限に一般化する)
- `docs/plan/workflow.md`
- `docs/plan/memory.md`

View File

@ -0,0 +1,63 @@
# TUI `#` Knowledge 補完の未実装解消
## 背景
TUI 入力欄で `#` を打つと `CompletionKind::Knowledge` で Pod に `ListCompletions` を投げる導線は既にある(`crates/tui/src/input.rs`、`crates/tui/src/app.rs`)。一方 Pod 側 IPC は `crates/pod/src/ipc/server.rs:105`
```rust
protocol::CompletionKind::Knowledge => Vec::new(),
```
と無条件に空ベクタを返しており、ワークスペースに knowledge エントリがあっても TUI 上では候補が一切出ない。`#` で何も出ないのはフロントマター(`model_invokation`)の問題ではなく、補完経路そのものが未配線。
Workflow 側は同じ構造で既に動いている:
- `Pod::workflow_completions()``Vec<String>``crates/pod/src/pod.rs:1236`
- `PodController::start``PodSharedState::set_workflows()` に投入(`crates/pod/src/controller.rs:385`
- IPC が `shared_state.list_workflow_completions(prefix)` を引いて返す
Knowledge も同じ形に揃えれば足りる。
## 前提
- knowledge の物理レイアウトは `<workspace>/.insomnia/knowledge/<slug>.md``crates/memory/src/workspace.rs`)。
- memory クレートには `collect_resident_knowledge` があるが、これは `model_invokation: true` のみを返す resident-injection 用 API で、補完用途には不適。
- `#` 補完では「ユーザーが参照可能な knowledge slug すべて」を返したい(`model_invokation` は resident 注入対象のフラグであって参照可否ではない)。
- workflow の `list_user_invocable` に相当する「列挙 API」が memory クレートに無いので追加が要る。
## 方針
- memory クレートに「knowledge slug 一覧を返す」公開 API を追加する。`collect_resident_knowledge` とは別関数とし、`model_invokation` でフィルタしない。frontmatter が壊れているファイルは silently skip する(`collect_resident_knowledge` と同じく Linter が write 時に shape を保証する前提)。
- `PodSharedState` に knowledge 候補スロット(`workflows` と同形の `OnceLock<Vec<...>>`)を追加し、`PodController::start` で memory layout から列挙して投入する。memory layout 未設定の Pod`Pod::memory_layout` が `None`)では空のまま残す。
- IPC server の `CompletionKind::Knowledge` 分岐を、`shared_state.list_knowledge_completions(&prefix)` を引いて `CompletionEntry { value: slug, is_dir: false }` に詰める実装に置き換える。
- 補完結果の並びは slug 昇順、prefix マッチは `starts_with`workflow と揃える)。
## 要件
- TUI で `#` を入力した状態で、ワークスペース `.insomnia/knowledge/` 配下に存在する slug`model_invokation` の真偽に関わらず)が候補として列挙される。
- `#foo` のように prefix 入力中の場合、prefix にマッチする slug のみが返る。
- memory layout が無効な Pod`memory_layout: None`)では空ベクタが返り、エラーにはならない。
- 確定時の挙動(`replace_with_knowledge_ref` → `Segment::KnowledgeRef` 化、`#slug` チップ表示)は既存の TUI 側の実装をそのまま使う。Pod 側補完を埋めるだけで TUI 改修は不要。
- 単体テストで以下をカバーする
- 全件列挙(複数 slug、`model_invokation: true/false` 混在で全部返る)
- prefix フィルタ
- knowledge ディレクトリ不在時の空ベクタ
- 壊れた frontmatter / `.md` 以外のファイルがスキップされる
## 範囲外
- knowledge のフロントマター仕様変更や、補完候補に description / model_invokation を載せて TUI で表示する強化(`CompletionEntry.value` 以外の表示は別チケットで検討)
- workflow / file の補完経路への手入れ
- resident 注入経路(`collect_resident_knowledge` の挙動)の変更
## 参照
- 未実装箇所: `crates/pod/src/ipc/server.rs:105`
- mirror 対象: `crates/pod/src/pod.rs:1236``workflow_completions`)、`crates/pod/src/shared_state.rs:74-89`、`crates/pod/src/controller.rs:385-390`
- TUI 側既存導線: `crates/tui/src/input.rs:260`、`crates/tui/src/app.rs:281,315`
- 列挙対象: `crates/memory/src/workspace.rs``knowledge_dir`)、`crates/memory/src/resident.rs`(参考)
## Review
- 状態: Approve
- レビュー詳細: [./tui-knowledge-completion.review.md](./tui-knowledge-completion.review.md)
- 日付: 2026-05-12

View File

@ -0,0 +1,35 @@
# Review: TUI `#` Knowledge 補完の未実装解消
対象実装: `7b8eb3a feat(pod): wire knowledge slugs into # completion` (branch `tui-knowledge-completion`)
## 前提・要件の確認
要件4項目すべてを diff 上に対応コードと根拠付きで確認した。
- 「`.insomnia/knowledge/` 配下の slug が `model_invokation` の真偽に関わらず列挙される」: `memory::list_knowledge_slugs``walk_knowledge``model_invokation` フィルタなしで通す(`crates/memory/src/resident.rs:47-52`)。テスト `list_slugs_returns_all_regardless_of_model_invokation`(同 `:196-204`)が `true/false/true` の 3 件を全部返すことを確認している。
- 「`#foo` の prefix 入力中は prefix マッチのみが返る」: `PodSharedState::list_knowledge_completions``starts_with` で絞り込み(`crates/pod/src/shared_state.rs:102-113`)。テスト `knowledge_completions_filter_by_prefix`(同 `:262-287`)が `alpha` prefix で `alpha`/`alphabet` のみ返り、`zzz` で空、空 prefix で全件、を確認。
- 「memory_layout が None の Pod で空ベクタ、エラーにならない」: `Pod::knowledge_completions``memory_layout.as_ref().map(...).unwrap_or_default()` で短絡(`crates/pod/src/pod.rs:1240-1245`。controller も `Vec<String>` を素通しで `set_knowledge` に渡すだけで panic 経路なし(`crates/pod/src/controller.rs:391-396`。IPC 側も `OnceLock` 未 set で空を返す(テスト `knowledge_completions_empty_when_unset` `:256-260` で確認)。
- 「確定時挙動は既存 TUI のまま、Pod 側を埋めるだけ」: TUI クレートには変更なし。IPC server の `CompletionKind::Knowledge` 分岐のみが `Vec::new()` から実装に差し替えられている(`crates/pod/src/ipc/server.rs:105-113`)。`CompletionEntry` の `value` に slug、`is_dir: false` を詰める形は Workflow 分岐と完全に同形。
単体テスト 4 項目もすべて存在し、`cargo test -p memory --lib resident::` と `cargo test -p pod --lib shared_state::` でグリーン。
## アーキテクチャ・スコープ
- 列挙 API を `memory` クレート(低レベル workspace 操作の所在に追加し、Pod 層は Memory layout から「引いて詰めるだけ」というレイヤ分割を保っている。`llm-worker` を巻き込まない、higher-level は上層という方針に合致。
- `WorkflowCandidate` / `set_workflows` / `list_workflow_completions``KnowledgeCandidate` / `set_knowledge` / `list_knowledge_completions` がフィールド順・docstring の有無・実装行数まで揃っており、mirror 対象(`shared_state.rs:74-89`、`controller.rs:385-390`、`pod.rs:1236`)のスタイルと一貫している。
- `walk_knowledge` の共通化は `FnMut(String, KnowledgeFrontmatter)` 1 つの最小抽象で、2 呼び出し元の重複(`read_dir` のエラー早抜け、非 `.md` スキップ、frontmatter parse スキップを素直にまとめている。やりすぎIterator 化、ジェネリック化はしておらず、CLAUDE.md の「変更量を最小に」「設計を歪めない」に合致する。逆に共通化を見送って書き写すと 30 行程度の同形コード重複になるので、この程度の抽出は妥当。
- 範囲外は守られている。frontmatter スキーマ、`collect_resident_knowledge` の挙動(`model_invokation: true` のみ返す resident 注入用途、workflow/file 補完経路、TUI コードへの手入れは一切なし。
## 指摘事項
### Non-blocking / Follow-up
- なし。
### Nits
- `crates/memory/src/resident.rs:26-28``collect_resident_knowledge` の docstring が `<workspace>/knowledge/*.md` のままで、実 path `<workspace>/.insomnia/knowledge/*.md``list_knowledge_slugs` 側の docstring `:43-46` では正しく書かれている)と齟齬がある。本チケットの範囲外の既存記述だが、隣接行を編集したついでに同期しておくと整う。今回は追わなくてよい。
## 判断
Approve — ticket の前提・方針・要件・テスト 4 項目すべてに対応コードとパスする単体テストがあり、範囲外を踏まず、Workflow 経路と整合した最小実装になっている。

View File

@ -1,68 +0,0 @@
# Workflow を memory crate から独立させる
## 背景
`tickets/workflow-directory-layout.md` で Workflow の物理配置を `.insomnia/workflow/` に分離した。これにより Workflow は概念上 memory statesession-derived / generatedと別物として整理されたが、ソースコード上は依然として `crates/memory/` 配下に同居している:
- `crates/memory/src/workflow.rs``WorkflowRecord` / `WorkflowRegistry` / `WorkflowSource` / `load_workflows` / `WorkflowLoadError` / `WORKFLOW_DESCRIPTION_HARD_CAP` / `ResidentWorkflowEntry` / `ShadowedSkill`
- `crates/memory/src/schema/workflow.rs``WorkflowFrontmatter`
- `crates/memory/src/skill.rs`Skill → Workflow projection
- `crates/memory/src/linter/mod.rs::lint_workflow`(人間編集向けの workflow linter
- `crates/memory/src/error.rs::LintError::WorkflowWriteForbidden`
memory crate のドメインは「decisions / requests / summary / knowledge / staging / consolidation」に絞り、Workflow は独立した crate に出す。`tickets/internal-worker-workflow.md` で内部 Worker の Workflow 化が予定されており、bundled default や `internal_role` 追加の置き場として独立 crate がある方が自然。
## 要件
### crate の分離
`crates/workflow/` を新設し、上記の Workflow 関連型 / 関数 / スキーマ / Skill projection / human-edit linter を移す。
- 新 crate からは memory crate に依存しないか、`WorkspaceLayout` 経由で薄く依存するに留める
- `crates/memory/` から workflow 関連の `pub use` 再エクスポートは削除(呼び出し側が新 crate を直接 import する)
- Workflow 用の linter は memory crate の `Linter` を共有しないでよい場合は単独で持つ。共有が必要なら共通部分を別 crate例: `crates/lint-common/`)に切る判断を行う
### `WorkspaceLayout` の扱い
`workflow_dir()` / `workflow_path()` が memory crate に残るかは設計判断:
- memory crate に残し、workflow crate がそれを利用する形でよい
- 別 crate例: `crates/workspace-layout/`)に切り出す場合は memory / workflow 両方が参照する形にする
どちらでもよいが、結果として循環依存を生まないこと。
### 既存 use site の更新
- `crates/pod/``pod.rs` / `prompt/system.rs` / `workflow/mod.rs`
- `crates/tui/`
- その他 `memory::Workflow*` を import している箇所
これらが新 crate を import する形に書き換わる。
### Skill ingestion の所属
`SKILL.md` パーサと `WorkflowRecord` への projection は workflow crate に同居する。Skills は外部入力だが最終的に Workflow registry に流れるので、workflow crate を窓口にする方がレイヤとして自然。
### scope deny の整理
`crates/memory/src/scope.rs::deny_write_rules` は memory / knowledge / workflow の 3 ディレクトリを deny している。workflow crate 側で `.insomnia/workflow/` の deny を表明し、Pod 起動時に両方を合成する形にするか、あるいは scope deny は呼び出し側podで集約する形に再設計する。
## 範囲外
- Workflow の機能変更frontmatter schema 変更、resolver 改修等)
- bundled default Workflow 機構(`tickets/internal-worker-workflow.md` の対象)
- memory crate 内部の他モジュール再編
## 完了条件
- `crates/workflow/` crate が独立して存在し、`WorkflowRecord` / `WorkflowRegistry` / `load_workflows` / `WorkflowFrontmatter` / Skill projection / human-edit linter がそこに住む
- memory crate に workflow / skill 関連のソースが残っていないreexport も無し)
- 既存テストが新構造で通る
- 既存呼び出し側pod / tui 等)が新 crate を import する形に更新されている
- scope deny が memory / workflow を矛盾なく合成できる構成になっている
## 参照
- 直前: `tickets/workflow-directory-layout.md`git log
- 後続: `tickets/internal-worker-workflow.md`
- 関連: `docs/plan/workflow.md`