memory: implement flat extract staging

This commit is contained in:
Keisuke Hirata 2026-07-18 05:46:56 +09:00
parent 444e6bee42
commit 2352f34c4e
No known key found for this signature in database
14 changed files with 560 additions and 424 deletions

View File

@ -1,14 +1,16 @@
--- ---
title: 'Flat candidate staging record schemaを実装する' title: 'Flat candidate staging record schemaを実装する'
state: 'ready' state: 'closed'
created_at: '2026-07-17T18:07:40Z' created_at: '2026-07-17T18:07:40Z'
updated_at: '2026-07-17T18:10:00Z' updated_at: '2026-07-17T19:56:08Z'
assignee: null assignee: null
queued_by: 'yoi ticket'
queued_at: '2026-07-17T19:48:08Z'
--- ---
## 背景 ## 背景
旧 staging は extract run ごとの batch payload (`decisions` / `discussions` / `attempts` / `requests`) と entry-level source refs を前提にしていた。しかし、consolidation から見ると batch は session に紐いた不必要な集団になり、record ごとの discard / merge / promote / defer が曖昧になる。 旧 staging は extract run ごとの batch payload (`decisions` / `discussions` / `attempts` / `requests`) と entry-level source refs を前提にしていた。しかし、consolidation から見ると batch は session に紐いた不必要な集団になり、record ごとの discard / merge / promote / defer が曖昧になる。
新方針では、extract は記憶化を検討すべき candidate を kind taxonomy に基づいて切り出し、candidate ごとに flat staging record を作る。 新方針では、extract は記憶化を検討すべき candidate を kind taxonomy に基づいて切り出し、candidate ごとに flat staging record を作る。

View File

@ -0,0 +1,9 @@
Implemented flat candidate staging record schema.
The memory extract staging path now treats staging as flat candidate records: one extract run can produce zero or more staging files, and each staging file is one consolidation decision unit. The transitional `write_extracted` path now accepts `candidates[]`, and the host expands each candidate into a flat `StagingRecord` with schema version, ids, source, kind, claim, usefulness, optional staleness, bounded evidence, and source refs.
Validation passed:
- `cargo fmt --check`
- `cargo test -p memory`
- `cargo test -p worker`
- `nix build .#yoi`

View File

@ -13,4 +13,80 @@ LocalTicketBackend によって作成されました。
Created to replace the previous batch staging direction. This Ticket owns the new flat staging record schema: one candidate per staging record, with kinds `preference`, `working_assumption`, `constraint`, `decision`, `open_question`, and `lesson`. Created to replace the previous batch staging direction. This Ticket owns the new flat staging record schema: one candidate per staging record, with kinds `preference`, `working_assumption`, `constraint`, `decision`, `open_question`, and `lesson`.
---
<!-- event: state_changed author: "yoi ticket" at: 2026-07-17T19:48:08Z from: ready to: queued reason: queued field: state -->
## State changed
Ticket を `yoi ticket` が queued にしました。
---
<!-- event: state_changed author: "yoi ticket" at: 2026-07-17T19:48:08Z from: queued to: inprogress reason: cli_state field: state -->
## State changed
State changed to `inprogress`.
---
<!-- event: implementation_report author: hare at: 2026-07-17T19:56:08Z -->
## Implementation report
Implemented flat candidate staging record schema.
Changes:
- Replaced extract payload with `candidates[]` based on narrow candidate kinds: `preference`, `working_assumption`, `constraint`, `decision`, `open_question`, `lesson`.
- Added flat `StagingRecord` schema with `schema_version`, `id`, `extract_run_id`, record-level `source`, `kind`, `claim`, `why_useful`, optional `staleness`, bounded `evidence[]`, and `source_refs[]`.
- Changed staging writer to write one JSON file per candidate. Empty payload writes no files.
- Updated transitional `write_extracted` tool and extract prompt to candidate extraction instead of activity-log extraction.
- Updated consolidation staging/input paths and tests to read/render flat records.
- Updated worker extract path to handle multiple staged candidate records per extract run.
Validation:
- `cargo fmt --check`
- `cargo test -p memory`
- `cargo test -p worker`
- `nix build .#yoi`
---
<!-- event: state_changed author: "yoi ticket" at: 2026-07-17T19:56:08Z from: inprogress to: done reason: cli_state field: state -->
## State changed
State changed to `done`.
---
<!-- event: state_changed author: hare at: 2026-07-17T19:56:08Z from: done to: closed reason: closed field: state -->
## State changed
Ticket を closed にしました。
---
<!-- event: close author: hare at: 2026-07-17T19:56:08Z status: closed -->
## 完了
Implemented flat candidate staging record schema.
The memory extract staging path now treats staging as flat candidate records: one extract run can produce zero or more staging files, and each staging file is one consolidation decision unit. The transitional `write_extracted` path now accepts `candidates[]`, and the host expands each candidate into a flat `StagingRecord` with schema version, ids, source, kind, claim, usefulness, optional staleness, bounded evidence, and source refs.
Validation passed:
- `cargo fmt --check`
- `cargo test -p memory`
- `cargo test -p worker`
- `nix build .#yoi`
--- ---

View File

@ -206,7 +206,10 @@ pub fn render_tidy_hints(tidy: &TidyHints) -> String {
mod tests { mod tests {
use super::*; use super::*;
use crate::consolidate::tidy::{SimilarSlugCluster, SourcesOverflow}; use crate::consolidate::tidy::{SimilarSlugCluster, SourcesOverflow};
use crate::extract::{DecisionEntry, ExtractedPayload, write_staging}; use crate::extract::{
CandidateKind, ExtractedCandidate, ExtractedPayload, STAGING_SCHEMA_VERSION,
StagingEvidence, StagingRecord, write_staging,
};
use crate::schema::{EvidenceKind, SourceEvidenceRef, SourceRef}; use crate::schema::{EvidenceKind, SourceEvidenceRef, SourceRef};
use chrono::Utc; use chrono::Utc;
use std::path::Path; use std::path::Path;
@ -222,6 +225,18 @@ mod tests {
std::fs::write(p, content).unwrap(); std::fs::write(p, content).unwrap();
} }
fn candidate_payload(kind: CandidateKind, claim: &str) -> ExtractedPayload {
ExtractedPayload {
candidates: vec![ExtractedCandidate {
kind,
claim: claim.into(),
why_useful: "useful for consolidation".into(),
staleness: None,
evidence_ids: Vec::new(),
}],
}
}
#[test] #[test]
fn build_includes_all_sections_when_populated() { fn build_includes_all_sections_when_populated() {
let dir = tempfile::TempDir::new().unwrap(); let dir = tempfile::TempDir::new().unwrap();
@ -238,13 +253,13 @@ mod tests {
n = now() n = now()
), ),
); );
let (_id, _) = write_staging( let _written = write_staging(
&layout, &layout,
SourceRef { SourceRef {
segment_id: "s".into(), segment_id: "s".into(),
range: [0, 1], range: [0, 1],
}, },
ExtractedPayload::default(), candidate_payload(CandidateKind::Preference, "Prefer concise tickets"),
) )
.unwrap(); .unwrap();
let staging = crate::consolidate::staging::list_staging_entries(&layout); let staging = crate::consolidate::staging::list_staging_entries(&layout);
@ -281,17 +296,27 @@ mod tests {
fn staging_render_preserves_entry_source_refs() { fn staging_render_preserves_entry_source_refs() {
let dir = tempfile::TempDir::new().unwrap(); let dir = tempfile::TempDir::new().unwrap();
let layout = WorkspaceLayout::new(dir.path().to_path_buf()); let layout = WorkspaceLayout::new(dir.path().to_path_buf());
let (_id, _) = write_staging( std::fs::create_dir_all(layout.staging_dir()).unwrap();
&layout, let id = uuid::Uuid::now_v7();
SourceRef { let record = StagingRecord {
schema_version: STAGING_SCHEMA_VERSION,
id: id.to_string(),
extract_run_id: "run-1".into(),
source: SourceRef {
segment_id: "segment-record".into(), segment_id: "segment-record".into(),
range: [0, 10], range: [0, 10],
}, },
ExtractedPayload { kind: CandidateKind::Decision,
decisions: vec![DecisionEntry { claim: "Keep flat staging records".into(),
options: vec!["preserve".into()], why_useful: "consolidation input is lossless JSON".into(),
chosen: "preserve".into(), staleness: None,
rationale: "consolidation input is lossless JSON".into(), evidence: vec![StagingEvidence {
id: "ev-1".into(),
kind: EvidenceKind::new(EvidenceKind::MESSAGE),
entry_range: Some([3, 4]),
excerpt: Some("bounded excerpt".into()),
summary: Some("bounded summary".into()),
}],
source_refs: vec![SourceEvidenceRef { source_refs: vec![SourceEvidenceRef {
session_id: Some("session-1".into()), session_id: Some("session-1".into()),
segment_id: Some("segment-1".into()), segment_id: Some("segment-1".into()),
@ -301,9 +326,10 @@ mod tests {
label: Some("user message".into()), label: Some("user message".into()),
summary: Some("bounded summary".into()), summary: Some("bounded summary".into()),
}], }],
}], };
..Default::default() std::fs::write(
}, layout.staging_dir().join(format!("{id}.json")),
serde_json::to_string_pretty(&record).unwrap(),
) )
.unwrap(); .unwrap();
let staging = crate::consolidate::staging::list_staging_entries(&layout); let staging = crate::consolidate::staging::list_staging_entries(&layout);

View File

@ -200,7 +200,7 @@ fn pid_is_alive(_pid: u32) -> bool {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::extract::{ExtractedPayload, write_staging}; use crate::extract::{CandidateKind, ExtractedCandidate, ExtractedPayload, write_staging};
use crate::schema::SourceRef; use crate::schema::SourceRef;
fn make_layout() -> (tempfile::TempDir, WorkspaceLayout) { fn make_layout() -> (tempfile::TempDir, WorkspaceLayout) {
@ -210,6 +210,18 @@ mod tests {
(dir, layout) (dir, layout)
} }
fn candidate_payload(claim: &str) -> ExtractedPayload {
ExtractedPayload {
candidates: vec![ExtractedCandidate {
kind: CandidateKind::Lesson,
claim: claim.into(),
why_useful: "useful for test".into(),
staleness: None,
evidence_ids: Vec::new(),
}],
}
}
#[test] #[test]
fn acquire_writes_lock_file() { fn acquire_writes_lock_file() {
let (_dir, layout) = make_layout(); let (_dir, layout) = make_layout();
@ -257,24 +269,28 @@ mod tests {
#[test] #[test]
fn release_drops_consumed_entries_and_unlinks_lock() { fn release_drops_consumed_entries_and_unlinks_lock() {
let (_dir, layout) = make_layout(); let (_dir, layout) = make_layout();
let (id_a, _) = write_staging( let id_a = write_staging(
&layout, &layout,
SourceRef { SourceRef {
segment_id: "s".into(), segment_id: "s".into(),
range: [0, 0], range: [0, 0],
}, },
ExtractedPayload::default(), candidate_payload("a"),
) )
.unwrap(); .unwrap()
let (id_b, _) = write_staging( .remove(0)
.id;
let id_b = write_staging(
&layout, &layout,
SourceRef { SourceRef {
segment_id: "s".into(), segment_id: "s".into(),
range: [1, 1], range: [1, 1],
}, },
ExtractedPayload::default(), candidate_payload("b"),
) )
.unwrap(); .unwrap()
.remove(0)
.id;
let lock = StagingLock::acquire(&layout, std::process::id(), "worker", vec![id_a]).unwrap(); let lock = StagingLock::acquire(&layout, std::process::id(), "worker", vec![id_a]).unwrap();
let lock_path = lock.path().to_path_buf(); let lock_path = lock.path().to_path_buf();

View File

@ -116,11 +116,19 @@ pub fn list_staging_entries_snapshot(layout: &WorkspaceLayout) -> StagingEntries
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::extract::{ExtractedPayload, write_staging}; use crate::extract::{CandidateKind, ExtractedCandidate, ExtractedPayload, write_staging};
use crate::schema::SourceRef; use crate::schema::SourceRef;
fn empty_payload() -> ExtractedPayload { fn candidate_payload(claim: &str) -> ExtractedPayload {
ExtractedPayload::default() ExtractedPayload {
candidates: vec![ExtractedCandidate {
kind: CandidateKind::Lesson,
claim: claim.into(),
why_useful: "useful for later consolidation".into(),
staleness: None,
evidence_ids: Vec::new(),
}],
}
} }
fn source(segment_id: &str, range: [u64; 2]) -> SourceRef { fn source(segment_id: &str, range: [u64; 2]) -> SourceRef {
@ -135,9 +143,18 @@ mod tests {
let tmp = tempfile::TempDir::new().unwrap(); let tmp = tempfile::TempDir::new().unwrap();
let layout = WorkspaceLayout::new(tmp.path().to_path_buf()); let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
let (id1, _) = write_staging(&layout, source("s", [0, 1]), empty_payload()).unwrap(); let id1 = write_staging(&layout, source("s", [0, 1]), candidate_payload("one"))
let (id2, _) = write_staging(&layout, source("s", [2, 3]), empty_payload()).unwrap(); .unwrap()
let (id3, _) = write_staging(&layout, source("s", [4, 5]), empty_payload()).unwrap(); .remove(0)
.id;
let id2 = write_staging(&layout, source("s", [2, 3]), candidate_payload("two"))
.unwrap()
.remove(0)
.id;
let id3 = write_staging(&layout, source("s", [4, 5]), candidate_payload("three"))
.unwrap()
.remove(0)
.id;
let entries = list_staging_entries(&layout); let entries = list_staging_entries(&layout);
let ids: Vec<Uuid> = entries.iter().map(|e| e.id).collect(); let ids: Vec<Uuid> = entries.iter().map(|e| e.id).collect();
@ -148,13 +165,15 @@ mod tests {
fn skips_lock_file_and_counts_invalid_json() { fn skips_lock_file_and_counts_invalid_json() {
let tmp = tempfile::TempDir::new().unwrap(); let tmp = tempfile::TempDir::new().unwrap();
let layout = WorkspaceLayout::new(tmp.path().to_path_buf()); let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
let (_id, _) = write_staging(&layout, source("s", [0, 1]), empty_payload()).unwrap(); let _id = write_staging(&layout, source("s", [0, 1]), candidate_payload("kept"))
.unwrap()
.remove(0)
.id;
// Drop a non-UUID json file, an unparsable UUID-named json file, a // Drop a non-UUID json file, an unparsable UUID-named json file, an
// legacy source UUID-named json file, and a bare lock file alongside. // old batch-schema UUID-named json file, and a bare lock file alongside.
// Lock files are not `.json`; invalid `.json` files are surfaced // Lock files are not `.json`; invalid `.json` files are surfaced
// separately instead of being mistaken for an empty staging directory. // separately instead of being mistaken for an empty staging directory.
// Legacy `source.session_id` staging remains readable for compatibility.
std::fs::write(layout.staging_dir().join("not-a-uuid.json"), "{}").unwrap(); std::fs::write(layout.staging_dir().join("not-a-uuid.json"), "{}").unwrap();
let bad_id = Uuid::now_v7(); let bad_id = Uuid::now_v7();
std::fs::write(layout.staging_dir().join(format!("{bad_id}.json")), "{").unwrap(); std::fs::write(layout.staging_dir().join(format!("{bad_id}.json")), "{").unwrap();
@ -174,17 +193,12 @@ mod tests {
std::fs::write(layout.staging_dir().join(".consolidation.lock"), "{}").unwrap(); std::fs::write(layout.staging_dir().join(".consolidation.lock"), "{}").unwrap();
let entries = list_staging_entries(&layout); let entries = list_staging_entries(&layout);
assert_eq!(entries.len(), 2); assert_eq!(entries.len(), 1);
let snapshot = list_staging_entries_snapshot(&layout); let snapshot = list_staging_entries_snapshot(&layout);
assert_eq!(snapshot.entries.len(), 2); assert_eq!(snapshot.entries.len(), 1);
assert_eq!(snapshot.invalid_count, 2); assert_eq!(snapshot.invalid_count, 3);
assert!( assert_eq!(snapshot.entries[0].record.claim, "kept");
snapshot
.entries
.iter()
.any(|entry| entry.record.source.segment_id == "legacy-session")
);
} }
#[test] #[test]

View File

@ -11,8 +11,8 @@ use llm_engine::Item;
pub fn build_extract_input(items: &[Item]) -> String { pub fn build_extract_input(items: &[Item]) -> String {
let mut out = String::new(); let mut out = String::new();
out.push_str( out.push_str(
"Extract activity logs from the conversation slice below. \ "Extract memory candidates from the conversation slice below. \
Follow the system prompt's schema strictly and call `write_extracted` once.\n\n", Follow the system prompt's candidate schema strictly and call `write_extracted` once.\n\n",
); );
out.push_str("## Conversation slice\n"); out.push_str("## Conversation slice\n");
out.push_str(&render_items(items)); out.push_str(&render_items(items));

View File

@ -1,4 +1,4 @@
//! extract: 活動抽出。 //! extract: memory candidate extraction.
//! //!
//! 通常 Worker の post-run hook で発火する disposable Engine と、その //! 通常 Worker の post-run hook で発火する disposable Engine と、その
//! 出力を `<workspace>/.yoi/memory/_staging/<id>.json` に書き出す //! 出力を `<workspace>/.yoi/memory/_staging/<id>.json` に書き出す
@ -24,10 +24,11 @@ mod tool;
pub use input::build_extract_input; pub use input::build_extract_input;
pub use payload::{ pub use payload::{
AttemptEntry, DecisionEntry, DiscussionEntry, ExtractedPayload, RequestEntry, StagingRecord, CandidateKind, ExtractedCandidate, ExtractedPayload, STAGING_SCHEMA_VERSION, StagingEvidence,
StagingRecord,
}; };
pub use pointer::{ExtractPointerPayload, fold_pointer}; pub use pointer::{ExtractPointerPayload, fold_pointer};
pub use staging::{StagingError, write_staging}; pub use staging::{StagingWriteResult, write_staging};
pub use tool::{ExtractWorkerContext, write_extracted_tool}; pub use tool::{ExtractWorkerContext, write_extracted_tool};
/// session-store `LogEntry::Extension` で使う domain 名。 /// session-store `LogEntry::Extension` で使う domain 名。

View File

@ -1,254 +1,210 @@
//! extract 抽出の出力 schema。 //! extract staging schema.
//! //!
//! LLM は [`ExtractedPayload`] そのものrecord-level source 抜きを返し、Worker 側 //! Extract produces memory-candidate records, not activity-log batches. During
//! ラッパーが [`StagingRecord`] に組み立てて staging へ書き出す。 //! the transitional `write_extracted` path the model submits an
//! source は機械付与する契約 (`docs/plan/memory.md` §Extract)。 //! [`ExtractedPayload`] containing `candidates[]`; the host expands each
//! candidate into one [`StagingRecord`] and attaches record ids, extract-run ids,
//! record-level source, evidence snippets, and source anchors mechanically.
use schemars::JsonSchema; use schemars::JsonSchema;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::schema::{SourceEvidenceRef, SourceRef}; use crate::schema::{EvidenceKind, SourceEvidenceRef, SourceRef};
/// LLM が返す活動ログ候補の集合。すべて optional空配列は許容 /// Current flat staging schema version.
#[derive(Debug, Clone, Default, Serialize, Deserialize, JsonSchema)] pub const STAGING_SCHEMA_VERSION: u32 = 2;
pub struct ExtractedPayload {
#[serde(default)] /// Candidate kinds that extract is allowed to stage.
pub decisions: Vec<DecisionEntry>, #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(default)] #[serde(rename_all = "snake_case")]
pub discussions: Vec<DiscussionEntry>, pub enum CandidateKind {
#[serde(default)] Preference,
pub attempts: Vec<AttemptEntry>, WorkingAssumption,
#[serde(default)] Constraint,
pub requests: Vec<RequestEntry>, Decision,
OpenQuestion,
Lesson,
} }
impl ExtractedPayload { impl CandidateKind {
/// すべての配列が空であれば true。空ペイロードは pub fn as_str(&self) -> &'static str {
/// "Nothing to save" 扱いで staging への書き込みを省いてよい。 match self {
pub fn is_empty(&self) -> bool { CandidateKind::Preference => "preference",
self.decisions.is_empty() CandidateKind::WorkingAssumption => "working_assumption",
&& self.discussions.is_empty() CandidateKind::Constraint => "constraint",
&& self.attempts.is_empty() CandidateKind::Decision => "decision",
&& self.requests.is_empty() CandidateKind::OpenQuestion => "open_question",
CandidateKind::Lesson => "lesson",
}
} }
} }
/// 判断したこと(選択肢 + 選んだ + 根拠)。 /// Model-submitted candidate before host-side staging metadata is attached.
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct DecisionEntry { pub struct ExtractedCandidate {
/// 検討された選択肢の列挙。 /// Kind of candidate. This is intentionally a narrow taxonomy, not an
pub options: Vec<String>, /// activity-log category.
/// 採用された選択肢。 pub kind: CandidateKind,
pub chosen: String, /// Concise candidate claim.
/// 採用理由 / 根拠。 pub claim: String,
pub rationale: String, /// Why this may be useful for future work or consolidation.
/// Host-resolved anchors backing this individual claim. pub why_useful: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")] /// Optional invalidation / staleness hint.
#[schemars(skip)]
pub source_refs: Vec<SourceEvidenceRef>,
}
/// 議論したこと(トピック + 論点)。結論が出ていなくてもよい。
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct DiscussionEntry {
/// 議論の主題。
pub topic: String,
/// 主題の中で挙がった論点 / 観点。
pub points: Vec<String>,
/// Host-resolved anchors backing this individual claim.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
#[schemars(skip)]
pub source_refs: Vec<SourceEvidenceRef>,
}
/// 試したこと(試行 + 結果 + 成否)。
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct AttemptEntry {
/// 何を試したか。
pub action: String,
/// 試した結果。
pub result: String,
/// 試行が目的に対して成功したか。失敗 / 部分成功も含めて bool で表現する。
pub succeeded: bool,
/// Host-resolved anchors backing this individual claim.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
#[schemars(skip)]
pub source_refs: Vec<SourceEvidenceRef>,
}
/// ユーザー submit の構造化要約。
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct RequestEntry {
/// ユーザーの意図 / ゴール。
pub intent: String,
/// 対象ファイル / モジュール / 機能(任意)。
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub target: Option<String>, pub staleness: Option<String>,
/// 一文サマリ。 /// Evidence ids selected by the extract worker. The transitional
pub summary: String, /// `write_extracted` path may leave this empty until `session-explore`
/// Host-resolved anchors backing this individual claim. /// provides host-issued evidence ids.
#[serde(default, skip_serializing_if = "Vec::is_empty")] #[serde(default, skip_serializing_if = "Vec::is_empty")]
#[schemars(skip)] pub evidence_ids: Vec<String>,
pub source_refs: Vec<SourceEvidenceRef>,
} }
/// staging に書き出される 1 ファイル分のレコード。 /// Transitional extract output: zero or more flat candidates.
/// #[derive(Debug, Clone, Default, Serialize, Deserialize, JsonSchema)]
/// `source` は Worker 側ラッパーが segment_id と log entry range を pub struct ExtractedPayload {
/// 機械付与する。LLM はこのフィールドを見ない / 推論しない。 #[serde(default)]
pub candidates: Vec<ExtractedCandidate>,
}
impl ExtractedPayload {
pub fn is_empty(&self) -> bool {
self.candidates.is_empty()
}
}
/// Bounded evidence snippet copied into a flat staging record.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct StagingEvidence {
pub id: String,
pub kind: EvidenceKind,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub entry_range: Option<[u64; 2]>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub excerpt: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub summary: Option<String>,
}
/// One flat staging record. One record is one consolidation decision unit.
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StagingRecord { pub struct StagingRecord {
pub schema_version: u32,
pub id: String,
pub extract_run_id: String,
pub source: SourceRef, pub source: SourceRef,
#[serde(flatten)] pub kind: CandidateKind,
pub payload: ExtractedPayload, pub claim: String,
pub why_useful: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub staleness: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub evidence: Vec<StagingEvidence>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub source_refs: Vec<SourceEvidenceRef>,
}
impl StagingRecord {
pub fn from_candidate(
id: impl Into<String>,
extract_run_id: impl Into<String>,
source: SourceRef,
candidate: ExtractedCandidate,
evidence: Vec<StagingEvidence>,
source_refs: Vec<SourceEvidenceRef>,
) -> Self {
Self {
schema_version: STAGING_SCHEMA_VERSION,
id: id.into(),
extract_run_id: extract_run_id.into(),
source,
kind: candidate.kind,
claim: candidate.claim,
why_useful: candidate.why_useful,
staleness: candidate.staleness,
evidence,
source_refs,
}
}
} }
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::schema::{EvidenceKind, SourceEvidenceRef};
fn record_source() -> serde_json::Value { fn source() -> SourceRef {
serde_json::json!({ SourceRef {
"source": {
"segment_id": "seg-old",
"range": [1, 5]
}
})
}
#[test]
fn old_staging_json_without_entry_source_refs_deserializes() {
let mut raw = record_source();
raw["decisions"] = serde_json::json!([
{
"options": ["keep", "drop"],
"chosen": "keep",
"rationale": "compatible"
}
]);
raw["discussions"] = serde_json::json!([
{
"topic": "compatibility",
"points": ["missing source_refs should default empty"]
}
]);
raw["attempts"] = serde_json::json!([
{
"action": "deserialize old staging",
"result": "ok",
"succeeded": true
}
]);
raw["requests"] = serde_json::json!([
{
"intent": "preserve old JSON",
"summary": "old payload has no entry anchors"
}
]);
let record: StagingRecord = serde_json::from_value(raw).unwrap();
assert_eq!(record.source.segment_id, "seg-old");
assert!(record.payload.decisions[0].source_refs.is_empty());
assert!(record.payload.discussions[0].source_refs.is_empty());
assert!(record.payload.attempts[0].source_refs.is_empty());
assert!(record.payload.requests[0].source_refs.is_empty());
let serialized = serde_json::to_string(&record).unwrap();
assert!(!serialized.contains("\"source_refs\""));
}
#[test]
fn new_staging_json_roundtrips_entry_source_refs() {
let evidence = SourceEvidenceRef {
session_id: Some("session-1".into()),
segment_id: Some("segment-1".into()),
entry_range: Some([10, 12]),
evidence_id: Some("ev-1".into()),
evidence_kind: Some(EvidenceKind::new(EvidenceKind::TOOL_RESULT)),
label: Some("cargo test result".into()),
summary: Some("bounded host summary".into()),
};
let record = StagingRecord {
source: SourceRef {
segment_id: "segment-record".into(), segment_id: "segment-record".into(),
range: [0, 20], range: [0, 20],
}, }
payload: ExtractedPayload { }
decisions: vec![DecisionEntry {
options: vec!["a".into(), "b".into()], #[test]
chosen: "a".into(), fn extracted_payload_empty_when_no_candidates() {
rationale: "evidence-backed".into(), assert!(ExtractedPayload::default().is_empty());
source_refs: vec![evidence.clone()], let payload = ExtractedPayload {
candidates: vec![ExtractedCandidate {
kind: CandidateKind::Decision,
claim: "Use flat staging records".into(),
why_useful: "Consolidation can resolve candidates independently".into(),
staleness: None,
evidence_ids: Vec::new(),
}], }],
discussions: vec![DiscussionEntry {
topic: "anchor shape".into(),
points: vec!["entry refs roundtrip".into()],
source_refs: vec![SourceEvidenceRef {
evidence_kind: Some(EvidenceKind::new(EvidenceKind::MESSAGE)),
..Default::default()
}],
}],
attempts: vec![AttemptEntry {
action: "serialize".into(),
result: "contains source refs".into(),
succeeded: true,
source_refs: vec![SourceEvidenceRef {
evidence_kind: Some(EvidenceKind::new(EvidenceKind::FILE_REF)),
evidence_id: Some("file:crates/memory/src/extract/payload.rs".into()),
..Default::default()
}],
}],
requests: vec![RequestEntry {
intent: "keep anchors".into(),
target: Some("memory staging".into()),
summary: "entry-level anchors survive".into(),
source_refs: vec![SourceEvidenceRef {
evidence_kind: Some(EvidenceKind::new(EvidenceKind::TICKET_REF)),
evidence_id: Some("00001KXNYXNM6".into()),
..Default::default()
}],
}],
},
}; };
assert!(!payload.is_empty());
}
#[test]
fn flat_staging_record_roundtrips() {
let evidence = StagingEvidence {
id: "E001".into(),
kind: EvidenceKind::new(EvidenceKind::MESSAGE),
entry_range: Some([10, 12]),
excerpt: Some("extract candidate taxonomy".into()),
summary: Some("User and assistant discussed staging kinds".into()),
};
let source_ref = SourceEvidenceRef {
segment_id: Some("segment-1".into()),
entry_range: Some([10, 12]),
evidence_id: Some("E001".into()),
evidence_kind: Some(EvidenceKind::new(EvidenceKind::MESSAGE)),
label: Some("design discussion".into()),
summary: Some("bounded host summary".into()),
..Default::default()
};
let candidate = ExtractedCandidate {
kind: CandidateKind::Constraint,
claim: "Extract stages candidates but does not write Memory".into(),
why_useful: "Preserves extract/consolidation boundary".into(),
staleness: Some("Revisit if the execution boundary changes".into()),
evidence_ids: vec!["E001".into()],
};
let record = StagingRecord::from_candidate(
"stg-1",
"run-1",
source(),
candidate,
vec![evidence],
vec![source_ref],
);
let json = serde_json::to_string_pretty(&record).unwrap(); let json = serde_json::to_string_pretty(&record).unwrap();
assert!(json.contains("\"schema_version\": 2"));
assert!(json.contains("\"kind\": \"constraint\""));
assert!(json.contains("source_refs")); assert!(json.contains("source_refs"));
assert!(json.contains("tool_result")); assert!(json.contains("evidence"));
assert!(json.contains("entry_range"));
let parsed: StagingRecord = serde_json::from_str(&json).unwrap(); let parsed: StagingRecord = serde_json::from_str(&json).unwrap();
let source_ref = &parsed.payload.decisions[0].source_refs[0]; assert_eq!(parsed.schema_version, STAGING_SCHEMA_VERSION);
assert_eq!(source_ref.session_id.as_deref(), Some("session-1")); assert_eq!(parsed.kind, CandidateKind::Constraint);
assert_eq!(source_ref.segment_id.as_deref(), Some("segment-1")); assert_eq!(parsed.evidence[0].id, "E001");
assert_eq!(source_ref.entry_range, Some([10, 12])); assert_eq!(parsed.source_refs[0].evidence_id.as_deref(), Some("E001"));
assert_eq!(source_ref.evidence_id.as_deref(), Some("ev-1")); }
assert_eq!(
source_ref.evidence_kind.as_ref().map(EvidenceKind::as_str), #[test]
Some(EvidenceKind::TOOL_RESULT) fn candidate_kind_serializes_as_snake_case() {
); let json = serde_json::to_string(&CandidateKind::WorkingAssumption).unwrap();
assert_eq!(source_ref.label.as_deref(), Some("cargo test result")); assert_eq!(json, "\"working_assumption\"");
assert_eq!(source_ref.summary.as_deref(), Some("bounded host summary")); let parsed: CandidateKind = serde_json::from_str("\"open_question\"").unwrap();
assert_eq!( assert_eq!(parsed, CandidateKind::OpenQuestion);
parsed.payload.discussions[0].source_refs[0]
.evidence_kind
.as_ref()
.map(EvidenceKind::as_str),
Some(EvidenceKind::MESSAGE)
);
assert_eq!(
parsed.payload.attempts[0].source_refs[0]
.evidence_kind
.as_ref()
.map(EvidenceKind::as_str),
Some(EvidenceKind::FILE_REF)
);
assert_eq!(
parsed.payload.requests[0].source_refs[0]
.evidence_kind
.as_ref()
.map(EvidenceKind::as_str),
Some(EvidenceKind::TICKET_REF)
);
} }
} }

View File

@ -1,9 +1,12 @@
//! `<workspace>/.yoi/memory/_staging/<id>.json` への書き出しヘルパー。 //! extract staging writer.
//! //!
//! 1 件 1 ファイル、UUIDv7 命名(短命なので衝突回避と順序を兼ねる)。 //! Staging is flat: one file is one candidate and one consolidation decision
//! `source` を機械付与した [`StagingRecord`] 形式で保存する。 //! unit. The transitional extract tool still submits `ExtractedPayload` with
//! `candidates[]`; this writer expands it into one [`StagingRecord`] per
//! candidate.
use std::fs; use std::fs;
use std::io;
use std::path::PathBuf; use std::path::PathBuf;
use uuid::Uuid; use uuid::Uuid;
@ -12,104 +15,108 @@ use crate::extract::payload::{ExtractedPayload, StagingRecord};
use crate::schema::SourceRef; use crate::schema::SourceRef;
use crate::workspace::WorkspaceLayout; use crate::workspace::WorkspaceLayout;
/// staging 書き出し時のエラー。 /// Filesystem result for a single staged candidate.
#[derive(Debug, thiserror::Error)] #[derive(Debug, Clone)]
pub enum StagingError { pub struct StagingWriteResult {
#[error("failed to create staging dir {}: {source}", .path.display())] pub id: Uuid,
CreateDir { pub path: PathBuf,
path: PathBuf,
#[source]
source: std::io::Error,
},
#[error("failed to write staging file {}: {source}", .path.display())]
Write {
path: PathBuf,
#[source]
source: std::io::Error,
},
#[error("failed to serialize staging record: {0}")]
Serialize(#[from] serde_json::Error),
} }
/// `payload` を `source` で wrap して staging に書き出す。 /// Write one flat staging JSON file per extracted candidate.
/// ///
/// 戻り値は割り当てられた staging file の (id, path)。`payload` が /// Returns an empty vector when `payload` has no candidates.
/// 完全に空の場合は呼び出し側が事前に `is_empty()` で skip 推奨だが、
/// この関数は空でも正規に書き出す(仕様 §Extract で空配列許容と
/// 明記されており、書く / 書かないの判断は呼び出し側に委ねる)。
pub fn write_staging( pub fn write_staging(
layout: &WorkspaceLayout, layout: &WorkspaceLayout,
source: SourceRef, source: SourceRef,
payload: ExtractedPayload, payload: ExtractedPayload,
) -> Result<(Uuid, PathBuf), StagingError> { ) -> io::Result<Vec<StagingWriteResult>> {
let staging_dir = layout.staging_dir(); if payload.candidates.is_empty() {
fs::create_dir_all(&staging_dir).map_err(|source| StagingError::CreateDir { return Ok(Vec::new());
path: staging_dir.clone(), }
source,
})?;
let dir = layout.staging_dir();
fs::create_dir_all(&dir)?;
let extract_run_id = Uuid::now_v7().to_string();
let mut written = Vec::with_capacity(payload.candidates.len());
for candidate in payload.candidates {
let id = Uuid::now_v7(); let id = Uuid::now_v7();
let path = staging_dir.join(format!("{id}.json")); let record = StagingRecord::from_candidate(
let record = StagingRecord { source, payload }; id.to_string(),
let json = serde_json::to_string_pretty(&record)?; extract_run_id.clone(),
fs::write(&path, json).map_err(|source| StagingError::Write { source.clone(),
path: path.clone(), candidate,
source, Vec::new(),
})?; Vec::new(),
Ok((id, path)) );
let path = dir.join(format!("{}.json", id));
let bytes = serde_json::to_vec_pretty(&record).map_err(io::Error::other)?;
fs::write(&path, bytes)?;
written.push(StagingWriteResult { id, path });
}
Ok(written)
} }
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::extract::payload::{DecisionEntry, ExtractedPayload}; use crate::extract::payload::{CandidateKind, ExtractedCandidate};
#[test] fn layout() -> WorkspaceLayout {
fn writes_record_with_machine_attached_source() { let dir = tempfile::tempdir().unwrap();
let tmp = tempfile::TempDir::new().unwrap(); // leak tempdir for the duration of the test process; sufficient for unit tests
let layout = WorkspaceLayout::new(tmp.path().to_path_buf()); let path = dir.keep();
WorkspaceLayout::new(path)
}
let source = SourceRef { fn source() -> SourceRef {
segment_id: "sess-1".into(), SourceRef {
range: [3, 7], segment_id: "segment-1".into(),
}; range: [1, 3],
let payload = ExtractedPayload { }
decisions: vec![DecisionEntry { }
options: vec!["a".into(), "b".into()],
chosen: "a".into(),
rationale: "shorter".into(),
source_refs: Vec::new(),
}],
..Default::default()
};
let (id, path) = write_staging(&layout, source.clone(), payload).unwrap();
assert_eq!(path.parent().unwrap(), layout.staging_dir());
assert!(
path.file_name()
.unwrap()
.to_string_lossy()
.contains(&id.to_string())
);
let written: StagingRecord = fn candidate(kind: CandidateKind, claim: &str) -> ExtractedCandidate {
serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap(); ExtractedCandidate {
assert_eq!(written.source.segment_id, "sess-1"); kind,
assert_eq!(written.source.range, [3, 7]); claim: claim.into(),
assert_eq!(written.payload.decisions.len(), 1); why_useful: "useful for future consolidation".into(),
staleness: None,
evidence_ids: Vec::new(),
}
} }
#[test] #[test]
fn empty_payload_is_written_verbatim() { fn writes_one_file_per_candidate() {
let tmp = tempfile::TempDir::new().unwrap(); let layout = layout();
let layout = WorkspaceLayout::new(tmp.path().to_path_buf()); let payload = ExtractedPayload {
let source = SourceRef { candidates: vec![
segment_id: "sess".into(), candidate(CandidateKind::Preference, "Prefer implementation tickets"),
range: [0, 0], candidate(CandidateKind::Decision, "Use flat staging records"),
],
}; };
let (_, path) = write_staging(&layout, source, ExtractedPayload::default()).unwrap(); let results = write_staging(&layout, source(), payload).unwrap();
let written: StagingRecord = assert_eq!(results.len(), 2);
serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap(); assert_ne!(results[0].id, results[1].id);
assert!(written.payload.is_empty());
let first = fs::read_to_string(&results[0].path).unwrap();
let second = fs::read_to_string(&results[1].path).unwrap();
let first_record: StagingRecord = serde_json::from_str(&first).unwrap();
let second_record: StagingRecord = serde_json::from_str(&second).unwrap();
assert_eq!(first_record.kind, CandidateKind::Preference);
assert_eq!(second_record.kind, CandidateKind::Decision);
assert_eq!(first_record.extract_run_id, second_record.extract_run_id);
assert_eq!(first_record.source.segment_id, "segment-1");
}
#[test]
fn empty_payload_writes_nothing() {
let layout = layout();
let results = write_staging(&layout, source(), ExtractedPayload::default()).unwrap();
assert!(results.is_empty());
assert!(!layout.staging_dir().exists());
} }
} }

View File

@ -12,10 +12,10 @@ use llm_engine::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
use crate::extract::payload::ExtractedPayload; use crate::extract::payload::ExtractedPayload;
const WRITE_EXTRACTED_DESCRIPTION: &str = "Submit the final activity-log JSON for this slice. \ const WRITE_EXTRACTED_DESCRIPTION: &str = "Submit extracted memory-candidate JSON for this slice. \
Pass an object with `decisions`, `discussions`, `attempts`, and `requests` arrays (any may be empty). \ Pass an object with a `candidates` array. Each candidate must have `kind`, `claim`, and `why_useful`; \
Call this exactly once and end the turn. Do not include `source`, session metadata, or free-form prose \ `staleness` and `evidence_ids` are optional. Call this exactly once and end the turn. Do not include \
the wrapper attaches provenance mechanically."; record ids, source anchors, session metadata, or free-form prose the wrapper attaches staging metadata mechanically.";
/// extract sub-Engine の出力受け口。`ExtractedPayload` 1 件をホストする。 /// extract sub-Engine の出力受け口。`ExtractedPayload` 1 件をホストする。
#[derive(Debug, Default)] #[derive(Debug, Default)]
@ -63,11 +63,8 @@ impl Tool for WriteExtractedTool {
ToolError::InvalidArgument(format!("invalid write_extracted input: {e}")) ToolError::InvalidArgument(format!("invalid write_extracted input: {e}"))
})?; })?;
let summary = format!( let summary = format!(
"Recorded activity log: decisions={} discussions={} attempts={} requests={}", "Recorded memory candidates: candidates={}",
payload.decisions.len(), payload.candidates.len(),
payload.discussions.len(),
payload.attempts.len(),
payload.requests.len(),
); );
{ {
let mut guard = self let mut guard = self
@ -116,20 +113,17 @@ mod tests {
let ctx = Arc::new(ExtractWorkerContext::new()); let ctx = Arc::new(ExtractWorkerContext::new());
let tool: Arc<dyn Tool> = Arc::new(WriteExtractedTool { ctx: ctx.clone() }); let tool: Arc<dyn Tool> = Arc::new(WriteExtractedTool { ctx: ctx.clone() });
let input = serde_json::json!({ let input = serde_json::json!({
"decisions": [{ "candidates": [{
"options": ["a", "b"], "kind": "decision",
"chosen": "a", "claim": "Use flat staging",
"rationale": "test" "why_useful": "Consolidation can resolve candidates independently"
}], }]
"discussions": [],
"attempts": [],
"requests": []
}) })
.to_string(); .to_string();
let out = tool.execute(&input, Default::default()).await.unwrap(); let out = tool.execute(&input, Default::default()).await.unwrap();
assert!(out.summary.contains("decisions=1")); assert!(out.summary.contains("candidates=1"));
let payload = ctx.take_payload().unwrap(); let payload = ctx.take_payload().unwrap();
assert_eq!(payload.decisions.len(), 1); assert_eq!(payload.candidates.len(), 1);
assert_eq!(ctx.call_count(), 1); assert_eq!(ctx.call_count(), 1);
} }
@ -138,22 +132,21 @@ mod tests {
let ctx = Arc::new(ExtractWorkerContext::new()); let ctx = Arc::new(ExtractWorkerContext::new());
let tool: Arc<dyn Tool> = Arc::new(WriteExtractedTool { ctx: ctx.clone() }); let tool: Arc<dyn Tool> = Arc::new(WriteExtractedTool { ctx: ctx.clone() });
let first = let first = serde_json::json!({"candidates": []}).to_string();
serde_json::json!({"decisions": [], "discussions": [], "attempts": [], "requests": []})
.to_string();
tool.execute(&first, Default::default()).await.unwrap(); tool.execute(&first, Default::default()).await.unwrap();
let second = serde_json::json!({ let second = serde_json::json!({
"decisions": [], "candidates": [{
"discussions": [], "kind": "lesson",
"attempts": [{"action": "x", "result": "ok", "succeeded": true}], "claim": "Validation should use Nix build",
"requests": [] "why_useful": "Packaging can fail independently"
}]
}) })
.to_string(); .to_string();
tool.execute(&second, Default::default()).await.unwrap(); tool.execute(&second, Default::default()).await.unwrap();
let payload = ctx.take_payload().unwrap(); let payload = ctx.take_payload().unwrap();
assert_eq!(payload.attempts.len(), 1); assert_eq!(payload.candidates.len(), 1);
assert_eq!(ctx.call_count(), 2); assert_eq!(ctx.call_count(), 2);
} }

View File

@ -3172,15 +3172,15 @@ impl<C: LlmClient, St: Store> Worker<C, St> {
}); });
let source_segment_id = self.segment_state.segment_id(); let source_segment_id = self.segment_state.segment_id();
let staging_id = if payload.is_empty() { let staging_results = if payload.is_empty() {
String::new() Vec::new()
} else { } else {
let source = memory::schema::SourceRef { let source = memory::schema::SourceRef {
segment_id: source_segment_id.to_string(), segment_id: source_segment_id.to_string(),
range: [start_entry as u64, end_entry as u64], range: [start_entry as u64, end_entry as u64],
}; };
let (id, _) = match extract::write_staging(&layout, source, payload) { match extract::write_staging(&layout, source, payload) {
Ok(result) => result, Ok(results) => results,
Err(err) => { Err(err) => {
let usage = usage_capture let usage = usage_capture
.lock() .lock()
@ -3197,9 +3197,12 @@ impl<C: LlmClient, St: Store> Worker<C, St> {
); );
return Err(WorkerError::ExtractStaging(err)); return Err(WorkerError::ExtractStaging(err));
} }
}
}; };
id.to_string() let staging_id = staging_results
}; .first()
.map(|result| result.id.to_string())
.unwrap_or_default();
let pointer_payload = extract::ExtractPointerPayload { let pointer_payload = extract::ExtractPointerPayload {
processed_through_entry: end_entry, processed_through_entry: end_entry,
@ -3220,16 +3223,12 @@ impl<C: LlmClient, St: Store> Worker<C, St> {
.expect("extract_pointer poisoned") = Some(pointer_payload); .expect("extract_pointer poisoned") = Some(pointer_payload);
let mut extract_audit = extract_audit_base; let mut extract_audit = extract_audit_base;
if !staging_id.is_empty() { extract_audit.staging_count = staging_results.len();
extract_audit.staging_count = 1; for result in &staging_results {
extract_audit.staging_ids.push(staging_id.clone()); extract_audit.staging_ids.push(result.id.to_string());
extract_audit.staging_paths.push( extract_audit
layout .staging_paths
.staging_dir() .push(result.path.display().to_string());
.join(format!("{staging_id}.json"))
.display()
.to_string(),
);
} }
let usage = usage_capture let usage = usage_capture
.lock() .lock()
@ -4819,7 +4818,7 @@ pub enum WorkerError {
Skill(#[from] SkillClientError), Skill(#[from] SkillClientError),
#[error("memory extract staging write failed: {0}")] #[error("memory extract staging write failed: {0}")]
ExtractStaging(#[source] memory::extract::StagingError), ExtractStaging(#[source] std::io::Error),
#[error("memory consolidation lock acquisition failed: {0}")] #[error("memory consolidation lock acquisition failed: {0}")]
ConsolidationLock(#[source] memory::consolidate::LockError), ConsolidationLock(#[source] memory::consolidate::LockError),

View File

@ -24,7 +24,7 @@ use llm_engine::Engine;
use llm_engine::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEvent}; use llm_engine::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEvent};
use llm_engine::llm_client::{ClientError, LlmClient, Request}; use llm_engine::llm_client::{ClientError, LlmClient, Request};
use memory::WorkspaceLayout; use memory::WorkspaceLayout;
use memory::extract::{ExtractedPayload, write_staging}; use memory::extract::{CandidateKind, ExtractedCandidate, ExtractedPayload, write_staging};
use memory::schema::SourceRef; use memory::schema::SourceRef;
use session_store::FsStore; use session_store::FsStore;
use session_store::{CombinedStore, FsWorkerStore}; use session_store::{CombinedStore, FsWorkerStore};
@ -182,18 +182,32 @@ async fn make_worker_with(
.unwrap() .unwrap()
} }
fn staging_payload(claim: String) -> ExtractedPayload {
ExtractedPayload {
candidates: vec![ExtractedCandidate {
kind: CandidateKind::Lesson,
claim,
why_useful: "useful for consolidation trigger tests".into(),
staleness: None,
evidence_ids: Vec::new(),
}],
}
}
fn write_n_staging(layout: &WorkspaceLayout, n: usize) -> Vec<uuid::Uuid> { fn write_n_staging(layout: &WorkspaceLayout, n: usize) -> Vec<uuid::Uuid> {
let mut ids = Vec::new(); let mut ids = Vec::new();
for i in 0..n { for i in 0..n {
let (id, _) = write_staging( let id = write_staging(
layout, layout,
SourceRef { SourceRef {
segment_id: format!("s-{i}"), segment_id: format!("s-{i}"),
range: [i as u64, i as u64], range: [i as u64, i as u64],
}, },
ExtractedPayload::default(), staging_payload(format!("candidate-{i}")),
) )
.unwrap(); .unwrap()
.remove(0)
.id;
ids.push(id); ids.push(id);
} }
ids ids

View File

@ -1,40 +1,63 @@
You are the activity extractor for a Yoi memory subsystem. You are a Yoi memory extract worker.
Your single job: read the supplied conversation slice and emit a structured JSON record of "what happened" via the `write_extracted` tool. You are not consolidating, summarising, or generating memory — that is the consolidation worker's job. Your job is to read the supplied conversation slice and extract only memory candidates that may be worth later consolidation. Do not produce activity logs.
# Memory language ## Language
- `language`: `{{ language }}`. - `language`: `{{language}}`
- Write extracted fact strings (`rationale`, `topic`, `points`, `action`, `result`, `intent`, `summary`, etc.) in this language. - Write candidate claims, usefulness, and staleness text in this language.
- Preserve code identifiers, paths, command names, quoted user text, logs, and external proper nouns when translation would reduce fidelity. - Preserve literal identifiers, paths, commands, branch names, issue IDs, tool names, model names, and quoted user/system text as-is.
- If the configured language is unclear, use English.
# Hard rules Call `write_extracted` exactly once with an object of this shape:
- Call `write_extracted` exactly once. Do not narrate, ask questions, or send any other tool output. ```json
- The argument is an object with four arrays: `decisions`, `discussions`, `attempts`, `requests`. Any of them may be empty. If nothing in the slice is worth recording, call `write_extracted({"decisions": [], "discussions": [], "attempts": [], "requests": []})` and stop. {
- Do NOT include `source`, `session_id`, entry indices, timestamps, or any provenance metadata. The wrapper attaches them mechanically. "candidates": [
- Do NOT add free-form commentary, summaries, or explanatory prose outside the schema fields. {
"kind": "preference",
"claim": "...",
"why_useful": "...",
"staleness": "...",
"evidence_ids": []
}
]
}
```
# Extraction guidance Allowed candidate kinds:
- `decisions`: judgements made during the slice. Each entry needs `options` (the alternatives considered), `chosen` (what was picked), and `rationale` (why). - `preference`: durable user/workspace preference or working style, not a one-off instruction.
- `discussions`: topics that were debated. `topic` plus `points` (the considerations raised). Open / unresolved discussions are valid. - `working_assumption`: provisional assumption that affects future design/implementation and may later change.
- `attempts`: things that were tried. `action`, `result`, and a `succeeded` boolean. Partial success is `false` with the result text describing the partial outcome. - `constraint`: boundary, invariant, or prohibition that future work/review should respect.
- `requests`: structured summaries of user submissions. `intent` (what the user wants), optional `target` (file / module / feature), and a one-line `summary`. - `decision`: choice with alternatives/chosen/rationale; not a mere fact or progress note.
- `open_question`: unresolved question that affects follow-up work and has a concrete next action.
- `lesson`: reusable learning from validation/failure/attempts that can improve future work.
# Quality bar Required fields per candidate:
- Drop one-off chit-chat, shallow questions, and turn-by-turn progress noise. Keep entries with long-term reference value. - `kind`: one of the allowed candidate kinds.
- Do not duplicate content already captured by static project docs (AGENTS.md, plan documents) — those are not "what happened in this slice". - `claim`: concise statement of the candidate.
- Prefer concise, fact-shaped strings. Do not pad rationale or summary fields. - `why_useful`: why this candidate may be useful for future consolidation.
# Anti-noise rules Optional fields:
Authoritative project records (issue trackers, task boards, planning documents, changelogs, version-control history, generated reports) are the source of truth for their exact contents. Memory must not mirror those records verbatim or maintain a parallel state ledger, but it may capture durable project-management facts, workflow constraints, recurring patterns, and abstractions when they will help future work. - `staleness`: when this candidate should be revisited or invalidated.
- `evidence_ids`: leave empty in this transitional path unless host-issued evidence ids are present.
- `attempts`: skip actions whose only substance is maintaining an authoritative record or moving an item through an external lifecycle. Keep attempts for outcomes that are not captured by that record itself: build / test outcomes, external API responses, observed bug reproductions, design experiments, and process lessons that inform later judgement. Do not extract:
- `discussions`: skip transient triage that goes stale within the day — immediate scheduling, checklist-style state reads, or short-lived sequencing choices. Keep discussions whose points outlive the session (architectural trade-offs, durable process constraints, recurring workflow questions).
- `decisions`: the rationale must be a design / policy / process / approach reason, not "we did X in this session". Recording that an item was filed, completed, or moved through a lifecycle is NOT a decision; recording the durable policy or abstraction behind that workflow can be.
- Avoid copying titles, bodies, checklists, raw statees, or short-lived identifiers from authoritative project records. If a record is only meaningful as an exact state mirror or with a transient identifier, the record itself is probably session-local and should be skipped.
When you have produced the JSON, call `write_extracted` and end the turn. No follow-up text. - tool-call chronology;
- file read/write history;
- generic progress updates;
- current-focus updates;
- one-off chit-chat;
- resolved local confusion;
- assistant self-corrections without durable consequence;
- authoritative Ticket/docs/git facts copied verbatim;
- validation results unless they imply a reusable lesson, active blocker, or authority evidence;
- implementation details that belong only in commit diff.
Prefer no candidates over noisy candidates. If nothing is worth staging, call `write_extracted` with `{"candidates": []}`.
Do not include record ids, source anchors, session metadata, free-form prose, or raw tool output content. The host attaches staging metadata mechanically.