fix: make image tool results durably prunable

2026-08-11 02:44:37 +09:00
parent 38b8f26a50
commit 8d2b8b690f
15 changed files with 469 additions and 268 deletions
@@ -0,0 +1,49 @@
+# Request-local image injection bypassed durable history and pruning
+
+Date: 2026-08-06
+
+## Symptom
+
+The first `ViewImage` implementation stored image bytes in `ToolOutput.attachments` with
+`#[serde(skip)]`, projected them into a synthetic user image only while building one provider
+request, and then cleared the attachment from Worker history. The model could therefore answer
+from pixels that disappeared from the next turn, session restore, pruning, and compaction
+observation.
+
+## Why this was wrong
+
+Yoi's context policy requires new model-visible input to be appended and committed to
+`worker.history` before request construction. Pure pruning may alter a request-context clone
+because the projection is deterministic from durable history; request-local input injection is
+not equivalent.
+
+The transient image also changed the middle of the next prompt prefix without an explicit prune,
+which reduced prompt-cache reuse and left later turns without the evidence behind the assistant
+response.
+
+Local reference review confirmed the intended pattern:
+
+- Codex records `view_image` as image-bearing `FunctionCallOutput` content and explicitly tests
+  that no separate image message is injected.
+- OpenCode persists image file parts in Session state and makes media removal an explicit
+  compaction/pruning decision.
+
+## Resolution
+
+Image attachments are now durable ToolResult detail:
+
+- Image bytes are base64-serialized through normal Item/session-log persistence.
+- OpenAI Responses lowers them to `function_call_output` content items.
+- OpenAI Chat deterministically lowers the durable ToolResult to tool text followed by a user
+  image message while preserving parallel-tool-result ordering.
+- The existing ToolResult pruning projection removes both text detail and image attachments from
+  the request-context clone while retaining the summary and original persistent history.
+- Session exploration and compaction-facing projections expose only bounded attachment metadata,
+  never raw base64 as ordinary text.
+
+## Guardrail
+
+Do not use `#[serde(skip)]`, post-build clearing, or one-shot synthetic messages for input that can
+influence model output. Provider-specific synthetic messages are acceptable only when they are a
+stable projection of a committed history item and are regenerated identically until an explicit
+pruning or compaction boundary.