memory: implement flat extract staging
This commit is contained in:
parent
444e6bee42
commit
2352f34c4e
|
|
@ -1,14 +1,16 @@
|
|||
---
|
||||
title: 'Flat candidate staging record schemaを実装する'
|
||||
state: 'ready'
|
||||
state: 'closed'
|
||||
created_at: '2026-07-17T18:07:40Z'
|
||||
updated_at: '2026-07-17T18:10:00Z'
|
||||
updated_at: '2026-07-17T19:56:08Z'
|
||||
assignee: null
|
||||
queued_by: 'yoi ticket'
|
||||
queued_at: '2026-07-17T19:48:08Z'
|
||||
---
|
||||
|
||||
## 背景
|
||||
|
||||
旧 staging は extract run ごとの batch payload (`decisions` / `discussions` / `attempts` / `requests`) と entry-level source refs を前提にしていた。しかし、consolidation から見ると batch は session に紐いた不必要な集団になり、record ごとの discard / merge / promote / defer が曖昧になる。
|
||||
旧 staging は extract run ごとの batch payload (`decisions` / `discussions` / `attempts` / `requests`) と entry-level source refs を前提にしていた。しかし、consolidation から見ると batch は session に紐づいた不必要な集団になり、record ごとの discard / merge / promote / defer が曖昧になる。
|
||||
|
||||
新方針では、extract は記憶化を検討すべき candidate を kind taxonomy に基づいて切り出し、candidate ごとに flat staging record を作る。
|
||||
|
||||
|
|
|
|||
9
.yoi/tickets/00001KXRM6G0G/resolution.md
Normal file
9
.yoi/tickets/00001KXRM6G0G/resolution.md
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
Implemented flat candidate staging record schema.
|
||||
|
||||
The memory extract staging path now treats staging as flat candidate records: one extract run can produce zero or more staging files, and each staging file is one consolidation decision unit. The transitional `write_extracted` path now accepts `candidates[]`, and the host expands each candidate into a flat `StagingRecord` with schema version, ids, source, kind, claim, usefulness, optional staleness, bounded evidence, and source refs.
|
||||
|
||||
Validation passed:
|
||||
- `cargo fmt --check`
|
||||
- `cargo test -p memory`
|
||||
- `cargo test -p worker`
|
||||
- `nix build .#yoi`
|
||||
|
|
@ -13,4 +13,80 @@ LocalTicketBackend によって作成されました。
|
|||
Created to replace the previous batch staging direction. This Ticket owns the new flat staging record schema: one candidate per staging record, with kinds `preference`, `working_assumption`, `constraint`, `decision`, `open_question`, and `lesson`.
|
||||
|
||||
|
||||
---
|
||||
|
||||
<!-- event: state_changed author: "yoi ticket" at: 2026-07-17T19:48:08Z from: ready to: queued reason: queued field: state -->
|
||||
|
||||
## State changed
|
||||
|
||||
Ticket を `yoi ticket` が queued にしました。
|
||||
|
||||
|
||||
---
|
||||
|
||||
<!-- event: state_changed author: "yoi ticket" at: 2026-07-17T19:48:08Z from: queued to: inprogress reason: cli_state field: state -->
|
||||
|
||||
## State changed
|
||||
|
||||
State changed to `inprogress`.
|
||||
|
||||
|
||||
---
|
||||
|
||||
<!-- event: implementation_report author: hare at: 2026-07-17T19:56:08Z -->
|
||||
|
||||
## Implementation report
|
||||
|
||||
Implemented flat candidate staging record schema.
|
||||
|
||||
Changes:
|
||||
- Replaced extract payload with `candidates[]` based on narrow candidate kinds: `preference`, `working_assumption`, `constraint`, `decision`, `open_question`, `lesson`.
|
||||
- Added flat `StagingRecord` schema with `schema_version`, `id`, `extract_run_id`, record-level `source`, `kind`, `claim`, `why_useful`, optional `staleness`, bounded `evidence[]`, and `source_refs[]`.
|
||||
- Changed staging writer to write one JSON file per candidate. Empty payload writes no files.
|
||||
- Updated transitional `write_extracted` tool and extract prompt to candidate extraction instead of activity-log extraction.
|
||||
- Updated consolidation staging/input paths and tests to read/render flat records.
|
||||
- Updated worker extract path to handle multiple staged candidate records per extract run.
|
||||
|
||||
Validation:
|
||||
- `cargo fmt --check`
|
||||
- `cargo test -p memory`
|
||||
- `cargo test -p worker`
|
||||
- `nix build .#yoi`
|
||||
|
||||
|
||||
---
|
||||
|
||||
<!-- event: state_changed author: "yoi ticket" at: 2026-07-17T19:56:08Z from: inprogress to: done reason: cli_state field: state -->
|
||||
|
||||
## State changed
|
||||
|
||||
State changed to `done`.
|
||||
|
||||
|
||||
---
|
||||
|
||||
<!-- event: state_changed author: hare at: 2026-07-17T19:56:08Z from: done to: closed reason: closed field: state -->
|
||||
|
||||
## State changed
|
||||
|
||||
Ticket を closed にしました。
|
||||
|
||||
|
||||
---
|
||||
|
||||
<!-- event: close author: hare at: 2026-07-17T19:56:08Z status: closed -->
|
||||
|
||||
## 完了
|
||||
|
||||
Implemented flat candidate staging record schema.
|
||||
|
||||
The memory extract staging path now treats staging as flat candidate records: one extract run can produce zero or more staging files, and each staging file is one consolidation decision unit. The transitional `write_extracted` path now accepts `candidates[]`, and the host expands each candidate into a flat `StagingRecord` with schema version, ids, source, kind, claim, usefulness, optional staleness, bounded evidence, and source refs.
|
||||
|
||||
Validation passed:
|
||||
- `cargo fmt --check`
|
||||
- `cargo test -p memory`
|
||||
- `cargo test -p worker`
|
||||
- `nix build .#yoi`
|
||||
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -206,7 +206,10 @@ pub fn render_tidy_hints(tidy: &TidyHints) -> String {
|
|||
mod tests {
|
||||
use super::*;
|
||||
use crate::consolidate::tidy::{SimilarSlugCluster, SourcesOverflow};
|
||||
use crate::extract::{DecisionEntry, ExtractedPayload, write_staging};
|
||||
use crate::extract::{
|
||||
CandidateKind, ExtractedCandidate, ExtractedPayload, STAGING_SCHEMA_VERSION,
|
||||
StagingEvidence, StagingRecord, write_staging,
|
||||
};
|
||||
use crate::schema::{EvidenceKind, SourceEvidenceRef, SourceRef};
|
||||
use chrono::Utc;
|
||||
use std::path::Path;
|
||||
|
|
@ -222,6 +225,18 @@ mod tests {
|
|||
std::fs::write(p, content).unwrap();
|
||||
}
|
||||
|
||||
fn candidate_payload(kind: CandidateKind, claim: &str) -> ExtractedPayload {
|
||||
ExtractedPayload {
|
||||
candidates: vec![ExtractedCandidate {
|
||||
kind,
|
||||
claim: claim.into(),
|
||||
why_useful: "useful for consolidation".into(),
|
||||
staleness: None,
|
||||
evidence_ids: Vec::new(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn build_includes_all_sections_when_populated() {
|
||||
let dir = tempfile::TempDir::new().unwrap();
|
||||
|
|
@ -238,13 +253,13 @@ mod tests {
|
|||
n = now()
|
||||
),
|
||||
);
|
||||
let (_id, _) = write_staging(
|
||||
let _written = write_staging(
|
||||
&layout,
|
||||
SourceRef {
|
||||
segment_id: "s".into(),
|
||||
range: [0, 1],
|
||||
},
|
||||
ExtractedPayload::default(),
|
||||
candidate_payload(CandidateKind::Preference, "Prefer concise tickets"),
|
||||
)
|
||||
.unwrap();
|
||||
let staging = crate::consolidate::staging::list_staging_entries(&layout);
|
||||
|
|
@ -281,17 +296,27 @@ mod tests {
|
|||
fn staging_render_preserves_entry_source_refs() {
|
||||
let dir = tempfile::TempDir::new().unwrap();
|
||||
let layout = WorkspaceLayout::new(dir.path().to_path_buf());
|
||||
let (_id, _) = write_staging(
|
||||
&layout,
|
||||
SourceRef {
|
||||
std::fs::create_dir_all(layout.staging_dir()).unwrap();
|
||||
let id = uuid::Uuid::now_v7();
|
||||
let record = StagingRecord {
|
||||
schema_version: STAGING_SCHEMA_VERSION,
|
||||
id: id.to_string(),
|
||||
extract_run_id: "run-1".into(),
|
||||
source: SourceRef {
|
||||
segment_id: "segment-record".into(),
|
||||
range: [0, 10],
|
||||
},
|
||||
ExtractedPayload {
|
||||
decisions: vec![DecisionEntry {
|
||||
options: vec!["preserve".into()],
|
||||
chosen: "preserve".into(),
|
||||
rationale: "consolidation input is lossless JSON".into(),
|
||||
kind: CandidateKind::Decision,
|
||||
claim: "Keep flat staging records".into(),
|
||||
why_useful: "consolidation input is lossless JSON".into(),
|
||||
staleness: None,
|
||||
evidence: vec![StagingEvidence {
|
||||
id: "ev-1".into(),
|
||||
kind: EvidenceKind::new(EvidenceKind::MESSAGE),
|
||||
entry_range: Some([3, 4]),
|
||||
excerpt: Some("bounded excerpt".into()),
|
||||
summary: Some("bounded summary".into()),
|
||||
}],
|
||||
source_refs: vec![SourceEvidenceRef {
|
||||
session_id: Some("session-1".into()),
|
||||
segment_id: Some("segment-1".into()),
|
||||
|
|
@ -301,9 +326,10 @@ mod tests {
|
|||
label: Some("user message".into()),
|
||||
summary: Some("bounded summary".into()),
|
||||
}],
|
||||
}],
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
std::fs::write(
|
||||
layout.staging_dir().join(format!("{id}.json")),
|
||||
serde_json::to_string_pretty(&record).unwrap(),
|
||||
)
|
||||
.unwrap();
|
||||
let staging = crate::consolidate::staging::list_staging_entries(&layout);
|
||||
|
|
|
|||
|
|
@ -200,7 +200,7 @@ fn pid_is_alive(_pid: u32) -> bool {
|
|||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::extract::{ExtractedPayload, write_staging};
|
||||
use crate::extract::{CandidateKind, ExtractedCandidate, ExtractedPayload, write_staging};
|
||||
use crate::schema::SourceRef;
|
||||
|
||||
fn make_layout() -> (tempfile::TempDir, WorkspaceLayout) {
|
||||
|
|
@ -210,6 +210,18 @@ mod tests {
|
|||
(dir, layout)
|
||||
}
|
||||
|
||||
fn candidate_payload(claim: &str) -> ExtractedPayload {
|
||||
ExtractedPayload {
|
||||
candidates: vec![ExtractedCandidate {
|
||||
kind: CandidateKind::Lesson,
|
||||
claim: claim.into(),
|
||||
why_useful: "useful for test".into(),
|
||||
staleness: None,
|
||||
evidence_ids: Vec::new(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn acquire_writes_lock_file() {
|
||||
let (_dir, layout) = make_layout();
|
||||
|
|
@ -257,24 +269,28 @@ mod tests {
|
|||
#[test]
|
||||
fn release_drops_consumed_entries_and_unlinks_lock() {
|
||||
let (_dir, layout) = make_layout();
|
||||
let (id_a, _) = write_staging(
|
||||
let id_a = write_staging(
|
||||
&layout,
|
||||
SourceRef {
|
||||
segment_id: "s".into(),
|
||||
range: [0, 0],
|
||||
},
|
||||
ExtractedPayload::default(),
|
||||
candidate_payload("a"),
|
||||
)
|
||||
.unwrap();
|
||||
let (id_b, _) = write_staging(
|
||||
.unwrap()
|
||||
.remove(0)
|
||||
.id;
|
||||
let id_b = write_staging(
|
||||
&layout,
|
||||
SourceRef {
|
||||
segment_id: "s".into(),
|
||||
range: [1, 1],
|
||||
},
|
||||
ExtractedPayload::default(),
|
||||
candidate_payload("b"),
|
||||
)
|
||||
.unwrap();
|
||||
.unwrap()
|
||||
.remove(0)
|
||||
.id;
|
||||
|
||||
let lock = StagingLock::acquire(&layout, std::process::id(), "worker", vec![id_a]).unwrap();
|
||||
let lock_path = lock.path().to_path_buf();
|
||||
|
|
|
|||
|
|
@ -116,11 +116,19 @@ pub fn list_staging_entries_snapshot(layout: &WorkspaceLayout) -> StagingEntries
|
|||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::extract::{ExtractedPayload, write_staging};
|
||||
use crate::extract::{CandidateKind, ExtractedCandidate, ExtractedPayload, write_staging};
|
||||
use crate::schema::SourceRef;
|
||||
|
||||
fn empty_payload() -> ExtractedPayload {
|
||||
ExtractedPayload::default()
|
||||
fn candidate_payload(claim: &str) -> ExtractedPayload {
|
||||
ExtractedPayload {
|
||||
candidates: vec![ExtractedCandidate {
|
||||
kind: CandidateKind::Lesson,
|
||||
claim: claim.into(),
|
||||
why_useful: "useful for later consolidation".into(),
|
||||
staleness: None,
|
||||
evidence_ids: Vec::new(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
fn source(segment_id: &str, range: [u64; 2]) -> SourceRef {
|
||||
|
|
@ -135,9 +143,18 @@ mod tests {
|
|||
let tmp = tempfile::TempDir::new().unwrap();
|
||||
let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
|
||||
|
||||
let (id1, _) = write_staging(&layout, source("s", [0, 1]), empty_payload()).unwrap();
|
||||
let (id2, _) = write_staging(&layout, source("s", [2, 3]), empty_payload()).unwrap();
|
||||
let (id3, _) = write_staging(&layout, source("s", [4, 5]), empty_payload()).unwrap();
|
||||
let id1 = write_staging(&layout, source("s", [0, 1]), candidate_payload("one"))
|
||||
.unwrap()
|
||||
.remove(0)
|
||||
.id;
|
||||
let id2 = write_staging(&layout, source("s", [2, 3]), candidate_payload("two"))
|
||||
.unwrap()
|
||||
.remove(0)
|
||||
.id;
|
||||
let id3 = write_staging(&layout, source("s", [4, 5]), candidate_payload("three"))
|
||||
.unwrap()
|
||||
.remove(0)
|
||||
.id;
|
||||
|
||||
let entries = list_staging_entries(&layout);
|
||||
let ids: Vec<Uuid> = entries.iter().map(|e| e.id).collect();
|
||||
|
|
@ -148,13 +165,15 @@ mod tests {
|
|||
fn skips_lock_file_and_counts_invalid_json() {
|
||||
let tmp = tempfile::TempDir::new().unwrap();
|
||||
let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
|
||||
let (_id, _) = write_staging(&layout, source("s", [0, 1]), empty_payload()).unwrap();
|
||||
let _id = write_staging(&layout, source("s", [0, 1]), candidate_payload("kept"))
|
||||
.unwrap()
|
||||
.remove(0)
|
||||
.id;
|
||||
|
||||
// Drop a non-UUID json file, an unparsable UUID-named json file, a
|
||||
// legacy source UUID-named json file, and a bare lock file alongside.
|
||||
// Drop a non-UUID json file, an unparsable UUID-named json file, an
|
||||
// old batch-schema UUID-named json file, and a bare lock file alongside.
|
||||
// Lock files are not `.json`; invalid `.json` files are surfaced
|
||||
// separately instead of being mistaken for an empty staging directory.
|
||||
// Legacy `source.session_id` staging remains readable for compatibility.
|
||||
std::fs::write(layout.staging_dir().join("not-a-uuid.json"), "{}").unwrap();
|
||||
let bad_id = Uuid::now_v7();
|
||||
std::fs::write(layout.staging_dir().join(format!("{bad_id}.json")), "{").unwrap();
|
||||
|
|
@ -174,17 +193,12 @@ mod tests {
|
|||
std::fs::write(layout.staging_dir().join(".consolidation.lock"), "{}").unwrap();
|
||||
|
||||
let entries = list_staging_entries(&layout);
|
||||
assert_eq!(entries.len(), 2);
|
||||
assert_eq!(entries.len(), 1);
|
||||
|
||||
let snapshot = list_staging_entries_snapshot(&layout);
|
||||
assert_eq!(snapshot.entries.len(), 2);
|
||||
assert_eq!(snapshot.invalid_count, 2);
|
||||
assert!(
|
||||
snapshot
|
||||
.entries
|
||||
.iter()
|
||||
.any(|entry| entry.record.source.segment_id == "legacy-session")
|
||||
);
|
||||
assert_eq!(snapshot.entries.len(), 1);
|
||||
assert_eq!(snapshot.invalid_count, 3);
|
||||
assert_eq!(snapshot.entries[0].record.claim, "kept");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -11,8 +11,8 @@ use llm_engine::Item;
|
|||
pub fn build_extract_input(items: &[Item]) -> String {
|
||||
let mut out = String::new();
|
||||
out.push_str(
|
||||
"Extract activity logs from the conversation slice below. \
|
||||
Follow the system prompt's schema strictly and call `write_extracted` once.\n\n",
|
||||
"Extract memory candidates from the conversation slice below. \
|
||||
Follow the system prompt's candidate schema strictly and call `write_extracted` once.\n\n",
|
||||
);
|
||||
out.push_str("## Conversation slice\n");
|
||||
out.push_str(&render_items(items));
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
//! extract: 活動抽出。
|
||||
//! extract: memory candidate extraction.
|
||||
//!
|
||||
//! 通常 Worker の post-run hook で発火する disposable Engine と、その
|
||||
//! 出力を `<workspace>/.yoi/memory/_staging/<id>.json` に書き出す
|
||||
|
|
@ -24,10 +24,11 @@ mod tool;
|
|||
|
||||
pub use input::build_extract_input;
|
||||
pub use payload::{
|
||||
AttemptEntry, DecisionEntry, DiscussionEntry, ExtractedPayload, RequestEntry, StagingRecord,
|
||||
CandidateKind, ExtractedCandidate, ExtractedPayload, STAGING_SCHEMA_VERSION, StagingEvidence,
|
||||
StagingRecord,
|
||||
};
|
||||
pub use pointer::{ExtractPointerPayload, fold_pointer};
|
||||
pub use staging::{StagingError, write_staging};
|
||||
pub use staging::{StagingWriteResult, write_staging};
|
||||
pub use tool::{ExtractWorkerContext, write_extracted_tool};
|
||||
|
||||
/// session-store `LogEntry::Extension` で使う domain 名。
|
||||
|
|
|
|||
|
|
@ -1,254 +1,210 @@
|
|||
//! extract 抽出の出力 schema。
|
||||
//! extract staging schema.
|
||||
//!
|
||||
//! LLM は [`ExtractedPayload`] そのもの（record-level source 抜き）を返し、Worker 側
|
||||
//! ラッパーが [`StagingRecord`] に組み立てて staging へ書き出す。
|
||||
//! source は機械付与する契約 (`docs/plan/memory.md` §Extract)。
|
||||
//! Extract produces memory-candidate records, not activity-log batches. During
|
||||
//! the transitional `write_extracted` path the model submits an
|
||||
//! [`ExtractedPayload`] containing `candidates[]`; the host expands each
|
||||
//! candidate into one [`StagingRecord`] and attaches record ids, extract-run ids,
|
||||
//! record-level source, evidence snippets, and source anchors mechanically.
|
||||
|
||||
use schemars::JsonSchema;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::schema::{SourceEvidenceRef, SourceRef};
|
||||
use crate::schema::{EvidenceKind, SourceEvidenceRef, SourceRef};
|
||||
|
||||
/// LLM が返す活動ログ候補の集合。すべて optional（空配列は許容）。
|
||||
/// Current flat staging schema version.
|
||||
pub const STAGING_SCHEMA_VERSION: u32 = 2;
|
||||
|
||||
/// Candidate kinds that extract is allowed to stage.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum CandidateKind {
|
||||
Preference,
|
||||
WorkingAssumption,
|
||||
Constraint,
|
||||
Decision,
|
||||
OpenQuestion,
|
||||
Lesson,
|
||||
}
|
||||
|
||||
impl CandidateKind {
|
||||
pub fn as_str(&self) -> &'static str {
|
||||
match self {
|
||||
CandidateKind::Preference => "preference",
|
||||
CandidateKind::WorkingAssumption => "working_assumption",
|
||||
CandidateKind::Constraint => "constraint",
|
||||
CandidateKind::Decision => "decision",
|
||||
CandidateKind::OpenQuestion => "open_question",
|
||||
CandidateKind::Lesson => "lesson",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Model-submitted candidate before host-side staging metadata is attached.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct ExtractedCandidate {
|
||||
/// Kind of candidate. This is intentionally a narrow taxonomy, not an
|
||||
/// activity-log category.
|
||||
pub kind: CandidateKind,
|
||||
/// Concise candidate claim.
|
||||
pub claim: String,
|
||||
/// Why this may be useful for future work or consolidation.
|
||||
pub why_useful: String,
|
||||
/// Optional invalidation / staleness hint.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub staleness: Option<String>,
|
||||
/// Evidence ids selected by the extract worker. The transitional
|
||||
/// `write_extracted` path may leave this empty until `session-explore`
|
||||
/// provides host-issued evidence ids.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub evidence_ids: Vec<String>,
|
||||
}
|
||||
|
||||
/// Transitional extract output: zero or more flat candidates.
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct ExtractedPayload {
|
||||
#[serde(default)]
|
||||
pub decisions: Vec<DecisionEntry>,
|
||||
#[serde(default)]
|
||||
pub discussions: Vec<DiscussionEntry>,
|
||||
#[serde(default)]
|
||||
pub attempts: Vec<AttemptEntry>,
|
||||
#[serde(default)]
|
||||
pub requests: Vec<RequestEntry>,
|
||||
pub candidates: Vec<ExtractedCandidate>,
|
||||
}
|
||||
|
||||
impl ExtractedPayload {
|
||||
/// すべての配列が空であれば true。空ペイロードは
|
||||
/// "Nothing to save" 扱いで staging への書き込みを省いてよい。
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.decisions.is_empty()
|
||||
&& self.discussions.is_empty()
|
||||
&& self.attempts.is_empty()
|
||||
&& self.requests.is_empty()
|
||||
self.candidates.is_empty()
|
||||
}
|
||||
}
|
||||
|
||||
/// 判断したこと（選択肢 + 選んだ + 根拠）。
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct DecisionEntry {
|
||||
/// 検討された選択肢の列挙。
|
||||
pub options: Vec<String>,
|
||||
/// 採用された選択肢。
|
||||
pub chosen: String,
|
||||
/// 採用理由 / 根拠。
|
||||
pub rationale: String,
|
||||
/// Host-resolved anchors backing this individual claim.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
#[schemars(skip)]
|
||||
pub source_refs: Vec<SourceEvidenceRef>,
|
||||
}
|
||||
|
||||
/// 議論したこと（トピック + 論点）。結論が出ていなくてもよい。
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct DiscussionEntry {
|
||||
/// 議論の主題。
|
||||
pub topic: String,
|
||||
/// 主題の中で挙がった論点 / 観点。
|
||||
pub points: Vec<String>,
|
||||
/// Host-resolved anchors backing this individual claim.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
#[schemars(skip)]
|
||||
pub source_refs: Vec<SourceEvidenceRef>,
|
||||
}
|
||||
|
||||
/// 試したこと（試行 + 結果 + 成否）。
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct AttemptEntry {
|
||||
/// 何を試したか。
|
||||
pub action: String,
|
||||
/// 試した結果。
|
||||
pub result: String,
|
||||
/// 試行が目的に対して成功したか。失敗 / 部分成功も含めて bool で表現する。
|
||||
pub succeeded: bool,
|
||||
/// Host-resolved anchors backing this individual claim.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
#[schemars(skip)]
|
||||
pub source_refs: Vec<SourceEvidenceRef>,
|
||||
}
|
||||
|
||||
/// ユーザー submit の構造化要約。
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RequestEntry {
|
||||
/// ユーザーの意図 / ゴール。
|
||||
pub intent: String,
|
||||
/// 対象ファイル / モジュール / 機能（任意）。
|
||||
/// Bounded evidence snippet copied into a flat staging record.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct StagingEvidence {
|
||||
pub id: String,
|
||||
pub kind: EvidenceKind,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub target: Option<String>,
|
||||
/// 一文サマリ。
|
||||
pub summary: String,
|
||||
/// Host-resolved anchors backing this individual claim.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
#[schemars(skip)]
|
||||
pub source_refs: Vec<SourceEvidenceRef>,
|
||||
pub entry_range: Option<[u64; 2]>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub excerpt: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub summary: Option<String>,
|
||||
}
|
||||
|
||||
/// staging に書き出される 1 ファイル分のレコード。
|
||||
///
|
||||
/// `source` は Worker 側ラッパーが segment_id と log entry range を
|
||||
/// 機械付与する。LLM はこのフィールドを見ない / 推論しない。
|
||||
/// One flat staging record. One record is one consolidation decision unit.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct StagingRecord {
|
||||
pub schema_version: u32,
|
||||
pub id: String,
|
||||
pub extract_run_id: String,
|
||||
pub source: SourceRef,
|
||||
#[serde(flatten)]
|
||||
pub payload: ExtractedPayload,
|
||||
pub kind: CandidateKind,
|
||||
pub claim: String,
|
||||
pub why_useful: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub staleness: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub evidence: Vec<StagingEvidence>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub source_refs: Vec<SourceEvidenceRef>,
|
||||
}
|
||||
|
||||
impl StagingRecord {
|
||||
pub fn from_candidate(
|
||||
id: impl Into<String>,
|
||||
extract_run_id: impl Into<String>,
|
||||
source: SourceRef,
|
||||
candidate: ExtractedCandidate,
|
||||
evidence: Vec<StagingEvidence>,
|
||||
source_refs: Vec<SourceEvidenceRef>,
|
||||
) -> Self {
|
||||
Self {
|
||||
schema_version: STAGING_SCHEMA_VERSION,
|
||||
id: id.into(),
|
||||
extract_run_id: extract_run_id.into(),
|
||||
source,
|
||||
kind: candidate.kind,
|
||||
claim: candidate.claim,
|
||||
why_useful: candidate.why_useful,
|
||||
staleness: candidate.staleness,
|
||||
evidence,
|
||||
source_refs,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::schema::{EvidenceKind, SourceEvidenceRef};
|
||||
|
||||
fn record_source() -> serde_json::Value {
|
||||
serde_json::json!({
|
||||
"source": {
|
||||
"segment_id": "seg-old",
|
||||
"range": [1, 5]
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn old_staging_json_without_entry_source_refs_deserializes() {
|
||||
let mut raw = record_source();
|
||||
raw["decisions"] = serde_json::json!([
|
||||
{
|
||||
"options": ["keep", "drop"],
|
||||
"chosen": "keep",
|
||||
"rationale": "compatible"
|
||||
}
|
||||
]);
|
||||
raw["discussions"] = serde_json::json!([
|
||||
{
|
||||
"topic": "compatibility",
|
||||
"points": ["missing source_refs should default empty"]
|
||||
}
|
||||
]);
|
||||
raw["attempts"] = serde_json::json!([
|
||||
{
|
||||
"action": "deserialize old staging",
|
||||
"result": "ok",
|
||||
"succeeded": true
|
||||
}
|
||||
]);
|
||||
raw["requests"] = serde_json::json!([
|
||||
{
|
||||
"intent": "preserve old JSON",
|
||||
"summary": "old payload has no entry anchors"
|
||||
}
|
||||
]);
|
||||
|
||||
let record: StagingRecord = serde_json::from_value(raw).unwrap();
|
||||
|
||||
assert_eq!(record.source.segment_id, "seg-old");
|
||||
assert!(record.payload.decisions[0].source_refs.is_empty());
|
||||
assert!(record.payload.discussions[0].source_refs.is_empty());
|
||||
assert!(record.payload.attempts[0].source_refs.is_empty());
|
||||
assert!(record.payload.requests[0].source_refs.is_empty());
|
||||
let serialized = serde_json::to_string(&record).unwrap();
|
||||
assert!(!serialized.contains("\"source_refs\""));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn new_staging_json_roundtrips_entry_source_refs() {
|
||||
let evidence = SourceEvidenceRef {
|
||||
session_id: Some("session-1".into()),
|
||||
segment_id: Some("segment-1".into()),
|
||||
entry_range: Some([10, 12]),
|
||||
evidence_id: Some("ev-1".into()),
|
||||
evidence_kind: Some(EvidenceKind::new(EvidenceKind::TOOL_RESULT)),
|
||||
label: Some("cargo test result".into()),
|
||||
summary: Some("bounded host summary".into()),
|
||||
};
|
||||
let record = StagingRecord {
|
||||
source: SourceRef {
|
||||
fn source() -> SourceRef {
|
||||
SourceRef {
|
||||
segment_id: "segment-record".into(),
|
||||
range: [0, 20],
|
||||
},
|
||||
payload: ExtractedPayload {
|
||||
decisions: vec![DecisionEntry {
|
||||
options: vec!["a".into(), "b".into()],
|
||||
chosen: "a".into(),
|
||||
rationale: "evidence-backed".into(),
|
||||
source_refs: vec![evidence.clone()],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extracted_payload_empty_when_no_candidates() {
|
||||
assert!(ExtractedPayload::default().is_empty());
|
||||
let payload = ExtractedPayload {
|
||||
candidates: vec![ExtractedCandidate {
|
||||
kind: CandidateKind::Decision,
|
||||
claim: "Use flat staging records".into(),
|
||||
why_useful: "Consolidation can resolve candidates independently".into(),
|
||||
staleness: None,
|
||||
evidence_ids: Vec::new(),
|
||||
}],
|
||||
discussions: vec![DiscussionEntry {
|
||||
topic: "anchor shape".into(),
|
||||
points: vec!["entry refs roundtrip".into()],
|
||||
source_refs: vec![SourceEvidenceRef {
|
||||
evidence_kind: Some(EvidenceKind::new(EvidenceKind::MESSAGE)),
|
||||
..Default::default()
|
||||
}],
|
||||
}],
|
||||
attempts: vec![AttemptEntry {
|
||||
action: "serialize".into(),
|
||||
result: "contains source refs".into(),
|
||||
succeeded: true,
|
||||
source_refs: vec![SourceEvidenceRef {
|
||||
evidence_kind: Some(EvidenceKind::new(EvidenceKind::FILE_REF)),
|
||||
evidence_id: Some("file:crates/memory/src/extract/payload.rs".into()),
|
||||
..Default::default()
|
||||
}],
|
||||
}],
|
||||
requests: vec![RequestEntry {
|
||||
intent: "keep anchors".into(),
|
||||
target: Some("memory staging".into()),
|
||||
summary: "entry-level anchors survive".into(),
|
||||
source_refs: vec![SourceEvidenceRef {
|
||||
evidence_kind: Some(EvidenceKind::new(EvidenceKind::TICKET_REF)),
|
||||
evidence_id: Some("00001KXNYXNM6".into()),
|
||||
..Default::default()
|
||||
}],
|
||||
}],
|
||||
},
|
||||
};
|
||||
assert!(!payload.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn flat_staging_record_roundtrips() {
|
||||
let evidence = StagingEvidence {
|
||||
id: "E001".into(),
|
||||
kind: EvidenceKind::new(EvidenceKind::MESSAGE),
|
||||
entry_range: Some([10, 12]),
|
||||
excerpt: Some("extract candidate taxonomy".into()),
|
||||
summary: Some("User and assistant discussed staging kinds".into()),
|
||||
};
|
||||
let source_ref = SourceEvidenceRef {
|
||||
segment_id: Some("segment-1".into()),
|
||||
entry_range: Some([10, 12]),
|
||||
evidence_id: Some("E001".into()),
|
||||
evidence_kind: Some(EvidenceKind::new(EvidenceKind::MESSAGE)),
|
||||
label: Some("design discussion".into()),
|
||||
summary: Some("bounded host summary".into()),
|
||||
..Default::default()
|
||||
};
|
||||
let candidate = ExtractedCandidate {
|
||||
kind: CandidateKind::Constraint,
|
||||
claim: "Extract stages candidates but does not write Memory".into(),
|
||||
why_useful: "Preserves extract/consolidation boundary".into(),
|
||||
staleness: Some("Revisit if the execution boundary changes".into()),
|
||||
evidence_ids: vec!["E001".into()],
|
||||
};
|
||||
let record = StagingRecord::from_candidate(
|
||||
"stg-1",
|
||||
"run-1",
|
||||
source(),
|
||||
candidate,
|
||||
vec![evidence],
|
||||
vec![source_ref],
|
||||
);
|
||||
|
||||
let json = serde_json::to_string_pretty(&record).unwrap();
|
||||
assert!(json.contains("\"schema_version\": 2"));
|
||||
assert!(json.contains("\"kind\": \"constraint\""));
|
||||
assert!(json.contains("source_refs"));
|
||||
assert!(json.contains("tool_result"));
|
||||
assert!(json.contains("entry_range"));
|
||||
assert!(json.contains("evidence"));
|
||||
let parsed: StagingRecord = serde_json::from_str(&json).unwrap();
|
||||
|
||||
let source_ref = &parsed.payload.decisions[0].source_refs[0];
|
||||
assert_eq!(source_ref.session_id.as_deref(), Some("session-1"));
|
||||
assert_eq!(source_ref.segment_id.as_deref(), Some("segment-1"));
|
||||
assert_eq!(source_ref.entry_range, Some([10, 12]));
|
||||
assert_eq!(source_ref.evidence_id.as_deref(), Some("ev-1"));
|
||||
assert_eq!(
|
||||
source_ref.evidence_kind.as_ref().map(EvidenceKind::as_str),
|
||||
Some(EvidenceKind::TOOL_RESULT)
|
||||
);
|
||||
assert_eq!(source_ref.label.as_deref(), Some("cargo test result"));
|
||||
assert_eq!(source_ref.summary.as_deref(), Some("bounded host summary"));
|
||||
assert_eq!(
|
||||
parsed.payload.discussions[0].source_refs[0]
|
||||
.evidence_kind
|
||||
.as_ref()
|
||||
.map(EvidenceKind::as_str),
|
||||
Some(EvidenceKind::MESSAGE)
|
||||
);
|
||||
assert_eq!(
|
||||
parsed.payload.attempts[0].source_refs[0]
|
||||
.evidence_kind
|
||||
.as_ref()
|
||||
.map(EvidenceKind::as_str),
|
||||
Some(EvidenceKind::FILE_REF)
|
||||
);
|
||||
assert_eq!(
|
||||
parsed.payload.requests[0].source_refs[0]
|
||||
.evidence_kind
|
||||
.as_ref()
|
||||
.map(EvidenceKind::as_str),
|
||||
Some(EvidenceKind::TICKET_REF)
|
||||
);
|
||||
assert_eq!(parsed.schema_version, STAGING_SCHEMA_VERSION);
|
||||
assert_eq!(parsed.kind, CandidateKind::Constraint);
|
||||
assert_eq!(parsed.evidence[0].id, "E001");
|
||||
assert_eq!(parsed.source_refs[0].evidence_id.as_deref(), Some("E001"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn candidate_kind_serializes_as_snake_case() {
|
||||
let json = serde_json::to_string(&CandidateKind::WorkingAssumption).unwrap();
|
||||
assert_eq!(json, "\"working_assumption\"");
|
||||
let parsed: CandidateKind = serde_json::from_str("\"open_question\"").unwrap();
|
||||
assert_eq!(parsed, CandidateKind::OpenQuestion);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,9 +1,12 @@
|
|||
//! `<workspace>/.yoi/memory/_staging/<id>.json` への書き出しヘルパー。
|
||||
//! extract staging writer.
|
||||
//!
|
||||
//! 1 件 1 ファイル、UUIDv7 命名（短命なので衝突回避と順序を兼ねる）。
|
||||
//! `source` を機械付与した [`StagingRecord`] 形式で保存する。
|
||||
//! Staging is flat: one file is one candidate and one consolidation decision
|
||||
//! unit. The transitional extract tool still submits `ExtractedPayload` with
|
||||
//! `candidates[]`; this writer expands it into one [`StagingRecord`] per
|
||||
//! candidate.
|
||||
|
||||
use std::fs;
|
||||
use std::io;
|
||||
use std::path::PathBuf;
|
||||
|
||||
use uuid::Uuid;
|
||||
|
|
@ -12,104 +15,108 @@ use crate::extract::payload::{ExtractedPayload, StagingRecord};
|
|||
use crate::schema::SourceRef;
|
||||
use crate::workspace::WorkspaceLayout;
|
||||
|
||||
/// staging 書き出し時のエラー。
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum StagingError {
|
||||
#[error("failed to create staging dir {}: {source}", .path.display())]
|
||||
CreateDir {
|
||||
path: PathBuf,
|
||||
#[source]
|
||||
source: std::io::Error,
|
||||
},
|
||||
#[error("failed to write staging file {}: {source}", .path.display())]
|
||||
Write {
|
||||
path: PathBuf,
|
||||
#[source]
|
||||
source: std::io::Error,
|
||||
},
|
||||
#[error("failed to serialize staging record: {0}")]
|
||||
Serialize(#[from] serde_json::Error),
|
||||
/// Filesystem result for a single staged candidate.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct StagingWriteResult {
|
||||
pub id: Uuid,
|
||||
pub path: PathBuf,
|
||||
}
|
||||
|
||||
/// `payload` を `source` で wrap して staging に書き出す。
|
||||
/// Write one flat staging JSON file per extracted candidate.
|
||||
///
|
||||
/// 戻り値は割り当てられた staging file の (id, path)。`payload` が
|
||||
/// 完全に空の場合は呼び出し側が事前に `is_empty()` で skip 推奨だが、
|
||||
/// この関数は空でも正規に書き出す（仕様 §Extract で空配列許容と
|
||||
/// 明記されており、書く / 書かないの判断は呼び出し側に委ねる）。
|
||||
/// Returns an empty vector when `payload` has no candidates.
|
||||
pub fn write_staging(
|
||||
layout: &WorkspaceLayout,
|
||||
source: SourceRef,
|
||||
payload: ExtractedPayload,
|
||||
) -> Result<(Uuid, PathBuf), StagingError> {
|
||||
let staging_dir = layout.staging_dir();
|
||||
fs::create_dir_all(&staging_dir).map_err(|source| StagingError::CreateDir {
|
||||
path: staging_dir.clone(),
|
||||
source,
|
||||
})?;
|
||||
) -> io::Result<Vec<StagingWriteResult>> {
|
||||
if payload.candidates.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
|
||||
let dir = layout.staging_dir();
|
||||
fs::create_dir_all(&dir)?;
|
||||
let extract_run_id = Uuid::now_v7().to_string();
|
||||
let mut written = Vec::with_capacity(payload.candidates.len());
|
||||
|
||||
for candidate in payload.candidates {
|
||||
let id = Uuid::now_v7();
|
||||
let path = staging_dir.join(format!("{id}.json"));
|
||||
let record = StagingRecord { source, payload };
|
||||
let json = serde_json::to_string_pretty(&record)?;
|
||||
fs::write(&path, json).map_err(|source| StagingError::Write {
|
||||
path: path.clone(),
|
||||
source,
|
||||
})?;
|
||||
Ok((id, path))
|
||||
let record = StagingRecord::from_candidate(
|
||||
id.to_string(),
|
||||
extract_run_id.clone(),
|
||||
source.clone(),
|
||||
candidate,
|
||||
Vec::new(),
|
||||
Vec::new(),
|
||||
);
|
||||
let path = dir.join(format!("{}.json", id));
|
||||
let bytes = serde_json::to_vec_pretty(&record).map_err(io::Error::other)?;
|
||||
fs::write(&path, bytes)?;
|
||||
written.push(StagingWriteResult { id, path });
|
||||
}
|
||||
|
||||
Ok(written)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
use crate::extract::payload::{DecisionEntry, ExtractedPayload};
|
||||
use crate::extract::payload::{CandidateKind, ExtractedCandidate};
|
||||
|
||||
fn layout() -> WorkspaceLayout {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
// leak tempdir for the duration of the test process; sufficient for unit tests
|
||||
let path = dir.keep();
|
||||
WorkspaceLayout::new(path)
|
||||
}
|
||||
|
||||
fn source() -> SourceRef {
|
||||
SourceRef {
|
||||
segment_id: "segment-1".into(),
|
||||
range: [1, 3],
|
||||
}
|
||||
}
|
||||
|
||||
fn candidate(kind: CandidateKind, claim: &str) -> ExtractedCandidate {
|
||||
ExtractedCandidate {
|
||||
kind,
|
||||
claim: claim.into(),
|
||||
why_useful: "useful for future consolidation".into(),
|
||||
staleness: None,
|
||||
evidence_ids: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn writes_record_with_machine_attached_source() {
|
||||
let tmp = tempfile::TempDir::new().unwrap();
|
||||
let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
|
||||
|
||||
let source = SourceRef {
|
||||
segment_id: "sess-1".into(),
|
||||
range: [3, 7],
|
||||
};
|
||||
fn writes_one_file_per_candidate() {
|
||||
let layout = layout();
|
||||
let payload = ExtractedPayload {
|
||||
decisions: vec![DecisionEntry {
|
||||
options: vec!["a".into(), "b".into()],
|
||||
chosen: "a".into(),
|
||||
rationale: "shorter".into(),
|
||||
source_refs: Vec::new(),
|
||||
}],
|
||||
..Default::default()
|
||||
candidates: vec![
|
||||
candidate(CandidateKind::Preference, "Prefer implementation tickets"),
|
||||
candidate(CandidateKind::Decision, "Use flat staging records"),
|
||||
],
|
||||
};
|
||||
let (id, path) = write_staging(&layout, source.clone(), payload).unwrap();
|
||||
assert_eq!(path.parent().unwrap(), layout.staging_dir());
|
||||
assert!(
|
||||
path.file_name()
|
||||
.unwrap()
|
||||
.to_string_lossy()
|
||||
.contains(&id.to_string())
|
||||
);
|
||||
let results = write_staging(&layout, source(), payload).unwrap();
|
||||
assert_eq!(results.len(), 2);
|
||||
assert_ne!(results[0].id, results[1].id);
|
||||
|
||||
let written: StagingRecord =
|
||||
serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap();
|
||||
assert_eq!(written.source.segment_id, "sess-1");
|
||||
assert_eq!(written.source.range, [3, 7]);
|
||||
assert_eq!(written.payload.decisions.len(), 1);
|
||||
let first = fs::read_to_string(&results[0].path).unwrap();
|
||||
let second = fs::read_to_string(&results[1].path).unwrap();
|
||||
let first_record: StagingRecord = serde_json::from_str(&first).unwrap();
|
||||
let second_record: StagingRecord = serde_json::from_str(&second).unwrap();
|
||||
|
||||
assert_eq!(first_record.kind, CandidateKind::Preference);
|
||||
assert_eq!(second_record.kind, CandidateKind::Decision);
|
||||
assert_eq!(first_record.extract_run_id, second_record.extract_run_id);
|
||||
assert_eq!(first_record.source.segment_id, "segment-1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_payload_is_written_verbatim() {
|
||||
let tmp = tempfile::TempDir::new().unwrap();
|
||||
let layout = WorkspaceLayout::new(tmp.path().to_path_buf());
|
||||
let source = SourceRef {
|
||||
segment_id: "sess".into(),
|
||||
range: [0, 0],
|
||||
};
|
||||
let (_, path) = write_staging(&layout, source, ExtractedPayload::default()).unwrap();
|
||||
let written: StagingRecord =
|
||||
serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap();
|
||||
assert!(written.payload.is_empty());
|
||||
fn empty_payload_writes_nothing() {
|
||||
let layout = layout();
|
||||
let results = write_staging(&layout, source(), ExtractedPayload::default()).unwrap();
|
||||
assert!(results.is_empty());
|
||||
assert!(!layout.staging_dir().exists());
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -12,10 +12,10 @@ use llm_engine::tool::{Tool, ToolDefinition, ToolError, ToolMeta, ToolOutput};
|
|||
|
||||
use crate::extract::payload::ExtractedPayload;
|
||||
|
||||
const WRITE_EXTRACTED_DESCRIPTION: &str = "Submit the final activity-log JSON for this slice. \
|
||||
Pass an object with `decisions`, `discussions`, `attempts`, and `requests` arrays (any may be empty). \
|
||||
Call this exactly once and end the turn. Do not include `source`, session metadata, or free-form prose — \
|
||||
the wrapper attaches provenance mechanically.";
|
||||
const WRITE_EXTRACTED_DESCRIPTION: &str = "Submit extracted memory-candidate JSON for this slice. \
|
||||
Pass an object with a `candidates` array. Each candidate must have `kind`, `claim`, and `why_useful`; \
|
||||
`staleness` and `evidence_ids` are optional. Call this exactly once and end the turn. Do not include \
|
||||
record ids, source anchors, session metadata, or free-form prose — the wrapper attaches staging metadata mechanically.";
|
||||
|
||||
/// extract sub-Engine の出力受け口。`ExtractedPayload` 1 件をホストする。
|
||||
#[derive(Debug, Default)]
|
||||
|
|
@ -63,11 +63,8 @@ impl Tool for WriteExtractedTool {
|
|||
ToolError::InvalidArgument(format!("invalid write_extracted input: {e}"))
|
||||
})?;
|
||||
let summary = format!(
|
||||
"Recorded activity log: decisions={} discussions={} attempts={} requests={}",
|
||||
payload.decisions.len(),
|
||||
payload.discussions.len(),
|
||||
payload.attempts.len(),
|
||||
payload.requests.len(),
|
||||
"Recorded memory candidates: candidates={}",
|
||||
payload.candidates.len(),
|
||||
);
|
||||
{
|
||||
let mut guard = self
|
||||
|
|
@ -116,20 +113,17 @@ mod tests {
|
|||
let ctx = Arc::new(ExtractWorkerContext::new());
|
||||
let tool: Arc<dyn Tool> = Arc::new(WriteExtractedTool { ctx: ctx.clone() });
|
||||
let input = serde_json::json!({
|
||||
"decisions": [{
|
||||
"options": ["a", "b"],
|
||||
"chosen": "a",
|
||||
"rationale": "test"
|
||||
}],
|
||||
"discussions": [],
|
||||
"attempts": [],
|
||||
"requests": []
|
||||
"candidates": [{
|
||||
"kind": "decision",
|
||||
"claim": "Use flat staging",
|
||||
"why_useful": "Consolidation can resolve candidates independently"
|
||||
}]
|
||||
})
|
||||
.to_string();
|
||||
let out = tool.execute(&input, Default::default()).await.unwrap();
|
||||
assert!(out.summary.contains("decisions=1"));
|
||||
assert!(out.summary.contains("candidates=1"));
|
||||
let payload = ctx.take_payload().unwrap();
|
||||
assert_eq!(payload.decisions.len(), 1);
|
||||
assert_eq!(payload.candidates.len(), 1);
|
||||
assert_eq!(ctx.call_count(), 1);
|
||||
}
|
||||
|
||||
|
|
@ -138,22 +132,21 @@ mod tests {
|
|||
let ctx = Arc::new(ExtractWorkerContext::new());
|
||||
let tool: Arc<dyn Tool> = Arc::new(WriteExtractedTool { ctx: ctx.clone() });
|
||||
|
||||
let first =
|
||||
serde_json::json!({"decisions": [], "discussions": [], "attempts": [], "requests": []})
|
||||
.to_string();
|
||||
let first = serde_json::json!({"candidates": []}).to_string();
|
||||
tool.execute(&first, Default::default()).await.unwrap();
|
||||
|
||||
let second = serde_json::json!({
|
||||
"decisions": [],
|
||||
"discussions": [],
|
||||
"attempts": [{"action": "x", "result": "ok", "succeeded": true}],
|
||||
"requests": []
|
||||
"candidates": [{
|
||||
"kind": "lesson",
|
||||
"claim": "Validation should use Nix build",
|
||||
"why_useful": "Packaging can fail independently"
|
||||
}]
|
||||
})
|
||||
.to_string();
|
||||
tool.execute(&second, Default::default()).await.unwrap();
|
||||
|
||||
let payload = ctx.take_payload().unwrap();
|
||||
assert_eq!(payload.attempts.len(), 1);
|
||||
assert_eq!(payload.candidates.len(), 1);
|
||||
assert_eq!(ctx.call_count(), 2);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -3172,15 +3172,15 @@ impl<C: LlmClient, St: Store> Worker<C, St> {
|
|||
});
|
||||
|
||||
let source_segment_id = self.segment_state.segment_id();
|
||||
let staging_id = if payload.is_empty() {
|
||||
String::new()
|
||||
let staging_results = if payload.is_empty() {
|
||||
Vec::new()
|
||||
} else {
|
||||
let source = memory::schema::SourceRef {
|
||||
segment_id: source_segment_id.to_string(),
|
||||
range: [start_entry as u64, end_entry as u64],
|
||||
};
|
||||
let (id, _) = match extract::write_staging(&layout, source, payload) {
|
||||
Ok(result) => result,
|
||||
match extract::write_staging(&layout, source, payload) {
|
||||
Ok(results) => results,
|
||||
Err(err) => {
|
||||
let usage = usage_capture
|
||||
.lock()
|
||||
|
|
@ -3197,9 +3197,12 @@ impl<C: LlmClient, St: Store> Worker<C, St> {
|
|||
);
|
||||
return Err(WorkerError::ExtractStaging(err));
|
||||
}
|
||||
}
|
||||
};
|
||||
id.to_string()
|
||||
};
|
||||
let staging_id = staging_results
|
||||
.first()
|
||||
.map(|result| result.id.to_string())
|
||||
.unwrap_or_default();
|
||||
|
||||
let pointer_payload = extract::ExtractPointerPayload {
|
||||
processed_through_entry: end_entry,
|
||||
|
|
@ -3220,16 +3223,12 @@ impl<C: LlmClient, St: Store> Worker<C, St> {
|
|||
.expect("extract_pointer poisoned") = Some(pointer_payload);
|
||||
|
||||
let mut extract_audit = extract_audit_base;
|
||||
if !staging_id.is_empty() {
|
||||
extract_audit.staging_count = 1;
|
||||
extract_audit.staging_ids.push(staging_id.clone());
|
||||
extract_audit.staging_paths.push(
|
||||
layout
|
||||
.staging_dir()
|
||||
.join(format!("{staging_id}.json"))
|
||||
.display()
|
||||
.to_string(),
|
||||
);
|
||||
extract_audit.staging_count = staging_results.len();
|
||||
for result in &staging_results {
|
||||
extract_audit.staging_ids.push(result.id.to_string());
|
||||
extract_audit
|
||||
.staging_paths
|
||||
.push(result.path.display().to_string());
|
||||
}
|
||||
let usage = usage_capture
|
||||
.lock()
|
||||
|
|
@ -4819,7 +4818,7 @@ pub enum WorkerError {
|
|||
Skill(#[from] SkillClientError),
|
||||
|
||||
#[error("memory extract staging write failed: {0}")]
|
||||
ExtractStaging(#[source] memory::extract::StagingError),
|
||||
ExtractStaging(#[source] std::io::Error),
|
||||
|
||||
#[error("memory consolidation lock acquisition failed: {0}")]
|
||||
ConsolidationLock(#[source] memory::consolidate::LockError),
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ use llm_engine::Engine;
|
|||
use llm_engine::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEvent};
|
||||
use llm_engine::llm_client::{ClientError, LlmClient, Request};
|
||||
use memory::WorkspaceLayout;
|
||||
use memory::extract::{ExtractedPayload, write_staging};
|
||||
use memory::extract::{CandidateKind, ExtractedCandidate, ExtractedPayload, write_staging};
|
||||
use memory::schema::SourceRef;
|
||||
use session_store::FsStore;
|
||||
use session_store::{CombinedStore, FsWorkerStore};
|
||||
|
|
@ -182,18 +182,32 @@ async fn make_worker_with(
|
|||
.unwrap()
|
||||
}
|
||||
|
||||
fn staging_payload(claim: String) -> ExtractedPayload {
|
||||
ExtractedPayload {
|
||||
candidates: vec![ExtractedCandidate {
|
||||
kind: CandidateKind::Lesson,
|
||||
claim,
|
||||
why_useful: "useful for consolidation trigger tests".into(),
|
||||
staleness: None,
|
||||
evidence_ids: Vec::new(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
fn write_n_staging(layout: &WorkspaceLayout, n: usize) -> Vec<uuid::Uuid> {
|
||||
let mut ids = Vec::new();
|
||||
for i in 0..n {
|
||||
let (id, _) = write_staging(
|
||||
let id = write_staging(
|
||||
layout,
|
||||
SourceRef {
|
||||
segment_id: format!("s-{i}"),
|
||||
range: [i as u64, i as u64],
|
||||
},
|
||||
ExtractedPayload::default(),
|
||||
staging_payload(format!("candidate-{i}")),
|
||||
)
|
||||
.unwrap();
|
||||
.unwrap()
|
||||
.remove(0)
|
||||
.id;
|
||||
ids.push(id);
|
||||
}
|
||||
ids
|
||||
|
|
|
|||
|
|
@ -1,40 +1,63 @@
|
|||
You are the activity extractor for a Yoi memory subsystem.
|
||||
You are a Yoi memory extract worker.
|
||||
|
||||
Your single job: read the supplied conversation slice and emit a structured JSON record of "what happened" via the `write_extracted` tool. You are not consolidating, summarising, or generating memory — that is the consolidation worker's job.
|
||||
Your job is to read the supplied conversation slice and extract only memory candidates that may be worth later consolidation. Do not produce activity logs.
|
||||
|
||||
# Memory language
|
||||
## Language
|
||||
|
||||
- `language`: `{{ language }}`.
|
||||
- Write extracted fact strings (`rationale`, `topic`, `points`, `action`, `result`, `intent`, `summary`, etc.) in this language.
|
||||
- Preserve code identifiers, paths, command names, quoted user text, logs, and external proper nouns when translation would reduce fidelity.
|
||||
- `language`: `{{language}}`
|
||||
- Write candidate claims, usefulness, and staleness text in this language.
|
||||
- Preserve literal identifiers, paths, commands, branch names, issue IDs, tool names, model names, and quoted user/system text as-is.
|
||||
- If the configured language is unclear, use English.
|
||||
|
||||
# Hard rules
|
||||
Call `write_extracted` exactly once with an object of this shape:
|
||||
|
||||
- Call `write_extracted` exactly once. Do not narrate, ask questions, or send any other tool output.
|
||||
- The argument is an object with four arrays: `decisions`, `discussions`, `attempts`, `requests`. Any of them may be empty. If nothing in the slice is worth recording, call `write_extracted({"decisions": [], "discussions": [], "attempts": [], "requests": []})` and stop.
|
||||
- Do NOT include `source`, `session_id`, entry indices, timestamps, or any provenance metadata. The wrapper attaches them mechanically.
|
||||
- Do NOT add free-form commentary, summaries, or explanatory prose outside the schema fields.
|
||||
```json
|
||||
{
|
||||
"candidates": [
|
||||
{
|
||||
"kind": "preference",
|
||||
"claim": "...",
|
||||
"why_useful": "...",
|
||||
"staleness": "...",
|
||||
"evidence_ids": []
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
# Extraction guidance
|
||||
Allowed candidate kinds:
|
||||
|
||||
- `decisions`: judgements made during the slice. Each entry needs `options` (the alternatives considered), `chosen` (what was picked), and `rationale` (why).
|
||||
- `discussions`: topics that were debated. `topic` plus `points` (the considerations raised). Open / unresolved discussions are valid.
|
||||
- `attempts`: things that were tried. `action`, `result`, and a `succeeded` boolean. Partial success is `false` with the result text describing the partial outcome.
|
||||
- `requests`: structured summaries of user submissions. `intent` (what the user wants), optional `target` (file / module / feature), and a one-line `summary`.
|
||||
- `preference`: durable user/workspace preference or working style, not a one-off instruction.
|
||||
- `working_assumption`: provisional assumption that affects future design/implementation and may later change.
|
||||
- `constraint`: boundary, invariant, or prohibition that future work/review should respect.
|
||||
- `decision`: choice with alternatives/chosen/rationale; not a mere fact or progress note.
|
||||
- `open_question`: unresolved question that affects follow-up work and has a concrete next action.
|
||||
- `lesson`: reusable learning from validation/failure/attempts that can improve future work.
|
||||
|
||||
# Quality bar
|
||||
Required fields per candidate:
|
||||
|
||||
- Drop one-off chit-chat, shallow questions, and turn-by-turn progress noise. Keep entries with long-term reference value.
|
||||
- Do not duplicate content already captured by static project docs (AGENTS.md, plan documents) — those are not "what happened in this slice".
|
||||
- Prefer concise, fact-shaped strings. Do not pad rationale or summary fields.
|
||||
- `kind`: one of the allowed candidate kinds.
|
||||
- `claim`: concise statement of the candidate.
|
||||
- `why_useful`: why this candidate may be useful for future consolidation.
|
||||
|
||||
# Anti-noise rules
|
||||
Optional fields:
|
||||
|
||||
Authoritative project records (issue trackers, task boards, planning documents, changelogs, version-control history, generated reports) are the source of truth for their exact contents. Memory must not mirror those records verbatim or maintain a parallel state ledger, but it may capture durable project-management facts, workflow constraints, recurring patterns, and abstractions when they will help future work.
|
||||
- `staleness`: when this candidate should be revisited or invalidated.
|
||||
- `evidence_ids`: leave empty in this transitional path unless host-issued evidence ids are present.
|
||||
|
||||
- `attempts`: skip actions whose only substance is maintaining an authoritative record or moving an item through an external lifecycle. Keep attempts for outcomes that are not captured by that record itself: build / test outcomes, external API responses, observed bug reproductions, design experiments, and process lessons that inform later judgement.
|
||||
- `discussions`: skip transient triage that goes stale within the day — immediate scheduling, checklist-style state reads, or short-lived sequencing choices. Keep discussions whose points outlive the session (architectural trade-offs, durable process constraints, recurring workflow questions).
|
||||
- `decisions`: the rationale must be a design / policy / process / approach reason, not "we did X in this session". Recording that an item was filed, completed, or moved through a lifecycle is NOT a decision; recording the durable policy or abstraction behind that workflow can be.
|
||||
- Avoid copying titles, bodies, checklists, raw statees, or short-lived identifiers from authoritative project records. If a record is only meaningful as an exact state mirror or with a transient identifier, the record itself is probably session-local and should be skipped.
|
||||
Do not extract:
|
||||
|
||||
When you have produced the JSON, call `write_extracted` and end the turn. No follow-up text.
|
||||
- tool-call chronology;
|
||||
- file read/write history;
|
||||
- generic progress updates;
|
||||
- current-focus updates;
|
||||
- one-off chit-chat;
|
||||
- resolved local confusion;
|
||||
- assistant self-corrections without durable consequence;
|
||||
- authoritative Ticket/docs/git facts copied verbatim;
|
||||
- validation results unless they imply a reusable lesson, active blocker, or authority evidence;
|
||||
- implementation details that belong only in commit diff.
|
||||
|
||||
Prefer no candidates over noisy candidates. If nothing is worth staging, call `write_extracted` with `{"candidates": []}`.
|
||||
|
||||
Do not include record ids, source anchors, session metadata, free-form prose, or raw tool output content. The host attaches staging metadata mechanically.
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user