Hare/yoi
1
1
Fork 0
Code Actions Packages Releases Activity

Agents.mdを一定閾値でturncateする仕様を削除

Browse Source
This commit is contained in:
Keisuke Hirata 2026-04-23 01:34:25 +09:00
parent 26ce346a81
commit e21d2041ef
5 changed files with 55 additions and 193 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

190
crates/pod/src/agents_md.rs
View File

@ -1,32 +1,29 @@
//! `AGENTS.md` ingestion for system-prompt templates.
//!
//! Reads `AGENTS.md` directly under the Pod cwd and exposes its body
//! to the template engine through `SystemPromptContext.files.agents_md`.
//! to the template engine through `SystemPromptContext.agents_md`.
//! Nested / parent-directory AGENTS.md files are intentionally ignored;
//! subproject context is expressed by launching a Pod with that
//! directory as cwd.
//!
//! No size cap is applied here — the whole file is read and embedded.
//! System-prompt-size policing is the responsibility of a higher layer
//! (Usage-driven warning after the first LLM round-trip).
use std::fs::File;
use std::io::{ErrorKind, Read};
use std::fs;
use std::io::ErrorKind;
use std::path::Path;
use tracing::warn;
/// Hard cap on the bytes exposed to the template. Roughly 20-25k tokens,
/// well within typical provider rate limits.
pub(crate) const AGENTS_MD_LIMIT: usize = 64 * 1024;
/// Outcome of an `AGENTS.md` ingestion attempt.
///
/// `body` carries the text that should be handed to the template
/// engine (if any); `truncated` signals that the caller should append
/// the `PodPrompt::AgentsMdTruncationNotice` text. `warnings` are short
/// human-readable messages that Pod forwards to the user-facing
/// notification channel. The caller also gets `tracing::warn!` lines
/// for the developer log.
/// engine (if any); `warnings` are short human-readable messages that
/// Pod forwards to the user-facing notification channel. The caller
/// also gets `tracing::warn!` lines for the developer log.
pub(crate) struct AgentsMdResult {
pub body: Option<String>,
pub truncated: bool,
pub warnings: Vec<String>,
}
@ -35,107 +32,50 @@ pub(crate) struct AgentsMdResult {
/// via `AgentsMdResult::warnings` (user-facing).
///
/// - Absent: `body = None`, no warning.
/// - Over limit: first 64KB (UTF-8 char boundary) + truncation notice, warning.
/// - Non-UTF-8 or I/O error: `body = None`, warning.
pub(crate) fn read_agents_md(cwd: &Path) -> AgentsMdResult {
let path = cwd.join("AGENTS.md");
let mut warnings = Vec::new();
let file = match File::open(&path) {
Ok(f) => f,
Err(e) if e.kind() == ErrorKind::NotFound => {
return AgentsMdResult {
body: None,
truncated: false,
warnings,
};
}
Err(e) => {
warn!(path = %path.display(), error = %e, "failed to open AGENTS.md");
warnings.push(format!("failed to open AGENTS.md ({}): {}", path.display(), e));
return AgentsMdResult {
body: None,
truncated: false,
warnings,
};
}
};
// Read one extra byte beyond the limit so we can detect oversize
// regardless of what `metadata()` claims (pipes/procfs may lie).
let mut buf = Vec::new();
let read_limit = (AGENTS_MD_LIMIT as u64) + 1;
if let Err(e) = file.take(read_limit).read_to_end(&mut buf) {
warn!(path = %path.display(), error = %e, "failed to read AGENTS.md");
warnings.push(format!("failed to read AGENTS.md ({}): {}", path.display(), e));
return AgentsMdResult {
body: None,
truncated: false,
match fs::read_to_string(&path) {
Ok(body) => AgentsMdResult {
body: Some(body),
warnings,
};
}
let truncated = buf.len() > AGENTS_MD_LIMIT;
if truncated {
buf.truncate(AGENTS_MD_LIMIT);
}
// UTF-8 decoding must not depend on whether the file exceeded the
// size limit: the same "genuinely non-UTF-8" file should be rejected
// regardless of its size. The only case in which we tolerate an
// invalid tail is when truncation itself sliced through a multi-byte
// char — at most 3 bytes of the final (4-byte) code point can be
// orphaned that way. Anything worse means the file was already
// non-UTF-8 before truncation, and we reject it.
let text = match std::str::from_utf8(&buf) {
Ok(_) => {
// SAFETY path: buf is valid UTF-8 in its entirety.
String::from_utf8(buf).expect("validated above")
}
Err(e) if truncated && e.valid_up_to() >= AGENTS_MD_LIMIT - 3 => {
let valid_len = e.valid_up_to();
buf.truncate(valid_len);
String::from_utf8(buf).expect("valid_up_to prefix is valid UTF-8")
}
Err(e) => {
},
Err(e) if e.kind() == ErrorKind::NotFound => AgentsMdResult {
body: None,
warnings,
},
Err(e) if e.kind() == ErrorKind::InvalidData => {
warn!(path = %path.display(), error = %e, "AGENTS.md is not valid UTF-8");
warnings.push(format!(
"AGENTS.md ({}) is not valid UTF-8: {}",
path.display(),
e
));
return AgentsMdResult {
AgentsMdResult {
body: None,
truncated: false,
warnings,
};
}
}
Err(e) => {
warn!(path = %path.display(), error = %e, "failed to read AGENTS.md");
warnings.push(format!(
"failed to read AGENTS.md ({}): {}",
path.display(),
e
));
AgentsMdResult {
body: None,
warnings,
}
}
};
if truncated {
warn!(
path = %path.display(),
limit = AGENTS_MD_LIMIT,
"AGENTS.md exceeded size limit; truncating"
);
warnings.push(format!(
"AGENTS.md ({}) exceeded {} bytes; the tail was truncated",
path.display(),
AGENTS_MD_LIMIT
));
}
AgentsMdResult {
body: Some(text),
truncated,
warnings,
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
use tempfile::TempDir;
#[test]
@ -154,51 +94,17 @@ mod tests {
}
#[test]
fn oversized_file_is_truncated_with_notice() {
fn reads_large_file_verbatim() {
// Previously truncated at 64KB; now read whole. Size-policing
// is deferred to the Usage-driven warning layer.
let dir = TempDir::new().unwrap();
let body = "a".repeat(AGENTS_MD_LIMIT + 1024);
let body = "a".repeat(128 * 1024);
fs::write(dir.path().join("AGENTS.md"), &body).unwrap();
let result = read_agents_md(dir.path());
assert!(result.truncated);
let got = result.body.expect("some");
assert_eq!(got.len(), AGENTS_MD_LIMIT);
assert!(got.chars().all(|c| c == 'a'));
assert_eq!(result.warnings.len(), 1);
}
#[test]
fn exact_limit_is_not_truncated() {
let dir = TempDir::new().unwrap();
let body = "a".repeat(AGENTS_MD_LIMIT);
fs::write(dir.path().join("AGENTS.md"), &body).unwrap();
let result = read_agents_md(dir.path());
assert!(!result.truncated);
let got = result.body.expect("some");
assert_eq!(got.len(), AGENTS_MD_LIMIT);
assert_eq!(result.body.as_ref().map(String::len), Some(128 * 1024));
assert!(result.warnings.is_empty());
}
#[test]
fn truncation_respects_utf8_char_boundary() {
let dir = TempDir::new().unwrap();
// Fill up to just under the limit with ASCII, then append a
// multi-byte char that straddles the boundary.
let mut body = "a".repeat(AGENTS_MD_LIMIT - 1);
body.push('あ'); // 3 bytes → pushes total past the limit
body.push_str(&"b".repeat(128));
fs::write(dir.path().join("AGENTS.md"), &body).unwrap();
let result = read_agents_md(dir.path());
assert!(result.truncated);
let got = result.body.expect("some");
// The partial 'あ' must have been dropped, leaving only the ASCII prefix.
assert_eq!(got.len(), AGENTS_MD_LIMIT - 1);
assert!(got.chars().all(|c| c == 'a'));
assert_eq!(result.warnings.len(), 1);
}
#[test]
fn non_utf8_surfaces_warning() {
let dir = TempDir::new().unwrap();
@ -207,24 +113,4 @@ mod tests {
assert!(result.body.is_none());
assert_eq!(result.warnings.len(), 1);
}
#[test]
fn oversized_non_utf8_is_still_rejected() {
// Regression: a file that is genuinely non-UTF-8 must be rejected
// regardless of its size. Previously the truncation-recovery pop
// loop would silently accept a partial prefix of such files once
// they exceeded the limit.
let dir = TempDir::new().unwrap();
let body = vec![0xffu8; AGENTS_MD_LIMIT + 1024];
fs::write(dir.path().join("AGENTS.md"), body).unwrap();
assert!(read_agents_md(dir.path()).body.is_none());
}
#[test]
fn non_utf8_returns_none() {
let dir = TempDir::new().unwrap();
// Invalid UTF-8 start byte.
fs::write(dir.path().join("AGENTS.md"), [0xff, 0xfe, 0xfd]).unwrap();
assert!(read_agents_md(dir.path()).body.is_none());
}
}

13
crates/pod/src/pod.rs
View File

@ -538,23 +538,12 @@ impl<C: LlmClient, St: Store> Pod<C, St> {
);
}
}
let agents_md_body = match agents_md_read.body {
Some(mut body) if agents_md_read.truncated => {
let notice = self
.prompts
.agents_md_truncation_notice()
.map_err(PodError::PromptCatalog)?;
body.push_str(&notice);
Some(body)
}
other => other,
};
let ctx = SystemPromptContext {
now: chrono::Utc::now(),
cwd: &self.pwd,
scope: &self.scope,
tool_names,
agents_md: agents_md_body,
agents_md: agents_md_read.body,
prompts: &self.prompts,
};
let rendered = template

19
crates/pod/src/prompts.rs
View File

@ -75,9 +75,6 @@ pub enum PodPrompt {
/// Trailing `## Project instructions (AGENTS.md)` section, appended
/// after the scope summary when an AGENTS.md is present.
AgentsMdSection,
/// Tail note used when AGENTS.md exceeds the byte cap and is
/// truncated before being embedded in the system prompt.
AgentsMdTruncationNotice,
}
impl PodPrompt {
@ -89,7 +86,6 @@ impl PodPrompt {
Self::InterruptSystemNote => "interrupt_system_note",
Self::WorkingBoundariesSection => "working_boundaries_section",
Self::AgentsMdSection => "agents_md_section",
Self::AgentsMdTruncationNotice => "agents_md_truncation_notice",
}
}
@ -103,7 +99,6 @@ impl PodPrompt {
PodPrompt::InterruptSystemNote,
PodPrompt::WorkingBoundariesSection,
PodPrompt::AgentsMdSection,
PodPrompt::AgentsMdTruncationNotice,
];
pub const KEYS: &'static [&'static str] = &[
@ -113,7 +108,6 @@ impl PodPrompt {
"interrupt_system_note",
"working_boundaries_section",
"agents_md_section",
"agents_md_truncation_notice",
];
}
@ -323,11 +317,6 @@ impl PromptCatalog {
pub fn agents_md_section(&self, agents_md: &str) -> Result<String, CatalogError> {
self.render(PodPrompt::AgentsMdSection, single("agents_md", agents_md))
}
/// Render `PodPrompt::AgentsMdTruncationNotice` (no inputs).
pub fn agents_md_truncation_notice(&self) -> Result<String, CatalogError> {
self.render(PodPrompt::AgentsMdTruncationNotice, Value::UNDEFINED)
}
}
fn single(key: &'static str, value: &str) -> Value {
@ -476,14 +465,6 @@ mod tests {
assert!(out.contains("PROJECT DOCS"));
}
#[test]
fn agents_md_truncation_notice_matches_expected_form() {
let cat = PromptCatalog::builtins_only().unwrap();
let out = cat.agents_md_truncation_notice().unwrap();
assert!(out.contains("[truncated: AGENTS.md exceeded 64KB limit]"));
assert!(out.starts_with("\n\n"));
}
#[test]
fn user_pack_overrides_builtin() {
let tmp = TempDir::new().unwrap();

2
resources/prompts/internal.toml
View File

@ -34,5 +34,3 @@ agents_md_section = """\
{{ agents_md }}\
"""
agents_md_truncation_notice = "\n\n[truncated: AGENTS.md exceeded 64KB limit]"

24
resources/prompts/internal/compact_system.md
View File

@ -1,12 +1,17 @@
You are a context compaction assistant. Your job is to hand the next session a structured summary plus pointers to the files it actually needs.
You are a context compaction assistant. Your job is to hand the next session a structured summary plus pointers to the files it actually needs — not a narrative transcript of the conversation.
Tools you can call:
- `read_file(file_path, offset?, limit?)` — inspect referenced files before deciding.
- `mark_read_required(file_path, offset?, limit?)` — inject a file's contents into the next session as an auto-read system message. Counts against `auto_read_budget`.
- `add_reference(file_path)` — record a file path the next session should know about without embedding its contents.
- `write_summary(text)` — deliver the final structured summary. May be called multiple times; only the last call is kept.
## Workflow
Always finish by calling `write_summary`. Produce the summary in this exact format:
1. Use `read_file` to inspect referenced files before deciding what the next session needs. Prefer skimming over blind inclusion.
2. For files whose current contents are load-bearing for the active work, call `mark_read_required` to inject them into the next session. These count against the auto-read token budget — spend it deliberately.
3. For files the next session should know about but can fetch on demand, call `add_reference` to record the path without embedding contents.
4. Finish with `write_summary` carrying the final text. You may call it multiple times; only the last call is kept.
Stop nominating and close out with `write_summary` as soon as the auto-read budget is exhausted, or whenever further nominations would not change the next session's next step.
## Summary format
Produce the summary in this exact format:
## Completed Tasks
### (task name)
@ -28,4 +33,7 @@ Always finish by calling `write_summary`. Produce the summary in this exact form
## Current Work
(2–3 lines on what was happening just before compaction).
Keep code snippets and raw tool output OUT of the summary — that is what auto-read and references are for. Target 1000–2000 tokens.
## Constraints
- Keep code snippets and raw tool output OUT of the summary — that is what auto-read and references are for.
- Target 1000–2000 tokens for the summary text itself.