ticket: use base32 project record ids

2026-06-09 22:10:47 +09:00
parent 0803bc3725
commit 4203988d74
798 changed files with 477 additions and 105 deletions
@@ -0,0 +1,119 @@
+---
+title: "セッション解析ツールを追加する"
+state: 'closed'
+created_at: "2026-06-09T03:25:33Z"
+updated_at: '2026-06-09T07:40:53Z'
+queued_by: 'workspace-panel'
+queued_at: '2026-06-09T06:56:20Z'
+---
+
+## 背景
+
+ドッグフーディングのセッションが十分に濃くなり、Worker / Pod の挙動を体感だけで調整するのが難しくなっている。永続化された session JSONL から安定した指標を抽出できる基盤が必要。
+
+現時点の懸念:
+
+- prune / compaction によってモデルが有用な文脈を失い、同じファイルを繰り返し読んでいる可能性がある。
+- `Read` の繰り返しが、必要な検証なのか、避けられる文脈喪失なのかを観測したい。
+- `Edit` の `old_string` / `new_string` が大きくなり、出力トークンを不必要に増やしている可能性がある。
+- 広すぎる `Bash` / `Grep` / `Glob` / `Read` の結果が history / context を膨らませている可能性がある。
+- 同じファイルへの編集 churn から、非効率な workflow や tool guidance の問題を見つけたい。
+
+まずは再利用可能な解析ライブラリとして作り、その上に CLI surface を薄く載せる。挙動を「無駄」と断定するのではなく、観測事実と suspicious pattern を報告する。
+
+## ゴール
+
+専用 crate `session-analytics` を作り、session JSONL log を解析して tool / session metrics を抽出できるようにする。その後、薄い `yoi session analyze` CLI として公開する。
+
+## 要件
+
+### ライブラリ crate
+
+- 専用の解析 crate を追加する。例: `crates/session-analytics`。
+- 1つの session file を解析する再利用可能 API を提供する。将来的には複数 session の集約も可能にする。
+  - 例: `analyze_session(path) -> SessionReport` または同等の API。
+  - 大きい session file を不必要に全読みしなくて済むよう、可能な範囲で streaming parse する。
+- TUI rendering や Pod runtime side effect から独立させる。
+- 既存の session / log schema 型は適切に再利用してよいが、runtime crate が analytics に依存する形にはしない。
+
+### 最初に抽出する metrics
+
+- Tool usage summary:
+  - tool name / kind ごとの呼び出し回数。
+  - turn ごとの tool call 数。
+  - failed tool call 数。
+  - 検出可能なら、より具体的な tool があるのに `Bash` で file inspection しているような broad tool usage pattern。
+- File read duplication:
+  - file path ごとの repeated `Read`。
+  - 同じ path + offset/limit の repeated `Read`。
+  - repeated read の間に対象ファイルへの write/edit があったかどうか。
+  - 観測可能なら compaction / prune event 後の repeated read。
+- Edit / write churn:
+  - file path ごとの `Edit` / `Write` 回数。
+  - 同一ファイルへの repeated edit。
+  - `old_string` / `new_string` / edit argument 全体のおおよその byte size。
+  - 出力トークンを増やし得る大きな replacement argument。
+  - `replace_all` の使用。
+- Tool result size:
+  - 取得可能な範囲で output bytes / lines。
+  - truncated / saved Bash output indicator。
+  - large `Read` / `Grep` / `Bash` / web result の観測。
+- Context lifecycle correlation:
+  - session log に存在する prune / compaction 関連 event を記録する。
+  - その後の repeated read / tool call と相関を取る。ただし因果として断定しない。
+
+### Report model
+
+- 後続の CLI / TUI / report rendering に使える構造化 `SessionReport` を定義する。
+- summary total と suspicious-pattern diagnostics の両方を含める。
+- 最初の安定 interface は machine-readable output を優先する。
+- default report では user input / file content の raw dump を避ける。
+  - path、count、byte size、timestamp / turn index、必要なら hash を含める。
+  - raw snippet / content は default では出さず、必要なら後続で明示 opt-in にする。
+
+### CLI integration
+
+- product binary に薄い CLI entrypoint を追加する。例:
+
+```text
+yoi session analyze <session-jsonl-or-session-id> [--json]
+```
+
+- 初期版は明示 file path 指定だけでもよい。
+- 後続拡張候補:
+  - `--latest`
+  - `--pod <name>`
+  - directory aggregation
+  - Markdown / human summary output
+- CLI は library crate を呼び、analytics logic を重複実装しない。
+
+### 意味論と制約
+
+- repeated read を自動的に悪いものとして扱わない。
+  - mutation 後の再読や validation のための再読は適切な場合がある。
+  - 事実と bounded suspicious diagnostics として報告する。
+- 通常出力で raw session content を露出しない。
+- 解析は read-only にする。
+- session log は記録済み tool call / history の authority として扱う。ただし context prune の因果は、明示 event がない限り相関としてしか推定できない。
+
+## 非目標
+
+- この Ticket で Worker の pruning / compaction 挙動を変更すること。
+- analytics 用の full TUI dashboard を作ること。
+- metrics に基づいて prompt / tool guidance を自動修正すること。
+- LLM を使って session content を意味的に要約すること。
+- default report で secret や raw file content を露出すること。
+
+## 受け入れ条件
+
+- `session-analytics` crate が存在し、tool analytics に必要な current session JSONL entry を少なくとも parse できる。
+- library が tool counts、repeated reads、edit/write churn、edit argument sizes、large/truncated result observations を含む structured report を返す。
+- `yoi session analyze <path> --json` または同等の CLI が machine-readable output を出す。
+- report は default で raw user/file content を避ける。
+- tests が以下を cover する:
+  - intervening edit あり/なしの repeated reads。
+  - large edit argument sizing。
+  - tool failure counting。
+  - representative log entry がある場合の basic compaction/prune event correlation。
+  - malformed / unknown log entry を bounded diagnostics で扱うこと。
+- focused tests、`cargo fmt --check`、`git diff --check`、`target/debug/yoi ticket doctor` が通る。
@@ -0,0 +1,25 @@
+Implemented, reviewed, merged, and validated.
+
+Summary:
+- Added reusable `crates/session-analytics` library with `analyze_session(path) -> SessionReport`.
+- Added thin product CLI `yoi session analyze <SESSION_JSONL_PATH> --json` that delegates to the library.
+- Implemented tolerant streaming JSONL parsing with bounded malformed/unknown-entry diagnostics.
+- Report covers tool usage counts, calls per turn, failed tool counts, repeated reads with mutation/context-lifecycle observations, edit/write churn, `replace_all`, large edit/result observations, saved/truncated Bash outputs, and compaction/prune/context correlations.
+- Default report avoids raw user input, raw tool arguments, raw file contents, raw session snippets, and raw tool output content.
+- Tests use synthetic/minimal fixtures rather than private local sessions.
+- No Worker pruning/compaction, Pod protocol/history/session writer, TUI dashboard, prompt auto-fixer, or LLM summarization behavior was changed.
+
+Implementation:
+- Coder commit: `c1809b3 feat: add session analytics tooling`
+- Reviewer approved with no blocking findings.
+- Merge commit: `0d2a6a7 merge: add session analytics tooling`
+
+Validation after merge:
+- `cargo test -p session-analytics`
+- `cargo test -p yoi session_cli`
+- `cargo test -p yoi parse_session_analyze_uses_session_mode`
+- `cargo fmt --check`
+- `git diff --check`
+- `cargo run -q -p yoi -- ticket doctor`
+- `cargo check --workspace`
+- `nix build .#yoi`
@@ -0,0 +1,273 @@
+<!-- event: create author: LocalTicketBackend at: 2026-06-09T03:25:33Z -->
+
+## 作成
+
+LocalTicketBackend によって作成されました。
+
+---
+
+<!-- event: intake_summary author: intake at: 2026-06-09T04:43:46Z -->
+
+## Intake summary
+
+既存 Ticket は、session JSONL を read-only に解析する再利用可能な `session-analytics` crate と薄い `yoi session analyze <path> --json` CLI を追加する concrete work item として十分に具体化済み。初期対象 metrics、raw content を既定で出さない privacy/secret 境界、non-goals、受け入れ条件、検証項目が明記されており、Orchestrator が実装 routing を判断できる。関連する `prompt-eval-metrics` は評価/改善 offer 寄りであり、本 Ticket は低レベル session/tool analytics 基盤として重複ではない。
+
+---
+
+<!-- event: state_changed author: intake at: 2026-06-09T04:43:46Z from: planning to: ready reason: intake_ready field: workflow_state -->
+
+## State changed
+
+Intake で要件が実装 routing 可能な粒度まで整理済みであることを確認したため、planning から ready にします。
+
+---
+
+<!-- event: state_changed author: workspace-panel at: 2026-06-09T06:56:20Z from: ready to: queued reason: queued field: state -->
+
+## State changed
+
+Ticket を `workspace-panel` が queued にしました。
+
+
+---
+
+<!-- event: state_changed author: orchestrator at: 2026-06-09T06:59:15Z from: queued to: inprogress reason: orchestrator_acceptance field: state -->
+
+## State changed
+
+Accepted queued implementation after reading the Ticket body/thread and current workspace state. The Ticket has concrete library/CLI/report/privacy acceptance criteria and does not depend on the just-landed Objective work. Implementation can proceed with synthetic fixture tests and read-only analysis semantics.
+
+---
+
+<!-- event: decision author: orchestrator at: 2026-06-09T06:59:15Z -->
+
+## Decision
+
+Routing decision: implementation_ready
+
+Evidence checked:
+- Ticket body and Intake summary.
+- Current workspace state after Objective records merged, closed, and cleaned up.
+- Current worktree state: no active implementation worktree remains.
+- Current queued set: this is the only queued Ticket.
+
+Reason:
+- The Ticket is specific enough for implementation: add a reusable `session-analytics` crate, structured report model, and thin `yoi session analyze <path> --json` CLI.
+- The privacy boundary is explicit: default output must not dump raw user input, file contents, secrets, or raw session snippets.
+- The non-goals are clear: no pruning/compaction behavior changes, no TUI dashboard, no prompt/tool guidance auto-rewrite, no LLM semantic summarization.
+
+IntentPacket:
+
+Intent:
+- Add read-only session JSONL analytics that extracts structured tool/session metrics and suspicious-pattern diagnostics without exposing raw content by default.
+
+Binding decisions / invariants:
+- Add a reusable library crate, e.g. `crates/session-analytics`, with an API such as `analyze_session(path) -> SessionReport`.
+- Add a thin product CLI surface, e.g. `yoi session analyze <session-jsonl-or-path> --json`, that calls the library and avoids duplicating analytics logic.
+- Prefer streaming/bounded parsing for large JSONL files; do not unnecessarily read huge files into memory if line-by-line parsing is practical.
+- Analysis is read-only and has no runtime side effects.
+- The analytics crate must stay independent from TUI rendering and Pod runtime side effects; runtime crates must not depend on analytics.
+- Reuse current session/log schema types only where dependency direction remains healthy.
+- Structured report should include at least:
+  - tool usage counts by name/kind;
+  - tool calls per turn;
+  - failed tool call counts;
+  - repeated `Read` by path and by path+offset/limit;
+  - whether repeated reads had intervening write/edit observations;
+  - edit/write churn by path;
+  - edit/write argument byte-size estimates and large replacement diagnostics;
+  - `replace_all` usage;
+  - observable tool result sizes/large/truncated indicators where available;
+  - prune/compaction events and bounded correlation with subsequent repeated reads/tool calls;
+  - malformed/unknown entry diagnostics.
+- Suspicious diagnostics must be framed as observations, not automatic blame. Re-reading after mutation or validation may be correct.
+- Default CLI/report output must avoid raw user input/file contents/session snippets. It may include paths, counts, byte sizes, turn/timestamp indexes, and hashes if useful.
+- Tests should use synthetic/minimal session JSONL fixtures rather than relying on private local sessions.
+- Do not implement a full dashboard, prompt auto-fixer, semantic summarizer, or Worker pruning/compaction behavior changes.
+
+Implementation latitude:
+- Coder may choose exact report struct field names and JSON format if machine-readable and test-covered.
+- Coder may add a small JSONL adapter tolerant of unknown/malformed records with bounded diagnostics if current session log enum parsing is too strict.
+- Coder may keep initial path input file-only; `--latest`, `--pod`, directory aggregation, and Markdown output can remain future work.
+- If current session schema is too hard to reuse without unhealthy dependencies, implement a minimal analytics event parser for the observed JSONL shape and document the limitation.
+
+Escalate if:
+- Implementing this requires changing session writer/history schemas or Worker compaction behavior.
+- Avoiding raw-content exposure conflicts with required metrics.
+- Healthy crate dependency direction would require a larger session-schema extraction refactor.
+- Real local session data would need to be read for tests/validation; use synthetic fixtures instead unless explicitly authorized.
+
+Validation:
+- Focused unit tests for repeated reads with/without intervening edit/write.
+- Focused tests for large edit argument sizing and replace_all observations.
+- Focused tests for tool failure counting and tool calls per turn.
+- Focused tests for large/truncated result observations if representable in fixtures.
+- Focused tests for compaction/prune event correlation as correlation only.
+- Focused tests for malformed/unknown JSONL entries producing bounded diagnostics.
+- CLI JSON output test for `yoi session analyze <path> --json` or equivalent.
+- `cargo fmt --check`.
+- `git diff --check`.
+- `cargo run -q -p yoi -- ticket doctor`.
+- `cargo check --workspace`.
+- `nix build .#yoi`.
+
+---
+
+<!-- event: implementation_report author: orchestrator at: 2026-06-09T07:00:26Z -->
+
+## Implementation report
+
+Implementation routing started.
+
+Worktree/branch:
+- Worktree: `.worktree/session-analytics-tooling`
+- Branch: `session-analytics-tooling`
+- Base/routing commit: `c166140 ticket: route session analytics tooling`
+
+Spawned sibling Coder Pod:
+- `coder-session-analytics-tooling`
+- Scope: non-recursive read on parent workspace root plus recursive write limited to the child worktree.
+
+Routing context:
+- This Ticket is now the active implementation work.
+- Handoff explicitly requires synthetic/minimal session fixtures for validation and avoids reading private real local session contents.
+
+---
+
+<!-- event: implementation_report author: orchestrator at: 2026-06-09T07:26:46Z -->
+
+## Implementation report
+
+Coder implementation completed and is ready for sibling review.
+
+Coder Pod:
+- `coder-session-analytics-tooling`
+- Commit: `c1809b37b1816a73e18c7587b9952cf4fc7b43f3 feat: add session analytics tooling`
+- Worktree status before review: clean branch `session-analytics-tooling`
+- Stopped after collecting output to reclaim delegated worktree scope.
+
+Selected design:
+- Added `crates/session-analytics` with public `analyze_session(path) -> SessionReport`.
+- Parser is a tolerant `serde_json::Value` JSONL parser using `BufReader::lines()` instead of tightly coupling analytics to runtime schema enums.
+- Report avoids raw user input, raw tool arguments, raw file contents, and raw tool output snippets by default; it reports paths, counts, byte/line sizes, line/turn indexes, bounded diagnostics, and observation text.
+- Added thin CLI `yoi session analyze <SESSION_JSONL_PATH> --json` that delegates to the library crate.
+
+Implementation summary:
+- Tool usage summary: counts by tool name/kind, calls per turn, failed tool result count, and Bash file-inspection-like heuristic observations.
+- Repeated reads: by path and path+offset/limit, with intervening Edit/Write observation and post-context-lifecycle correlation.
+- Edit/write churn: counts by path, repeated edits, argument byte-size estimates, large replacement diagnostics, and `replace_all` observations.
+- Tool result size: output byte/line sizes, large result observations, and saved/truncated Bash result indicators.
+- Context lifecycle: observes `segment_start.compacted_from` and context/prune/compact-like extension events; correlation only, not causation.
+- Robustness: malformed/unknown JSONL entries produce bounded diagnostics; unreadable input file remains an error.
+
+Changed files:
+- `Cargo.toml`
+- `Cargo.lock`
+- `package.nix`
+- `crates/session-analytics/Cargo.toml`
+- `crates/session-analytics/src/lib.rs`
+- `crates/yoi/Cargo.toml`
+- `crates/yoi/src/main.rs`
+- `crates/yoi/src/session_cli.rs`
+
+Coder validation reported passed:
+- `cargo test -p session-analytics`
+- `cargo test -p yoi session_cli`
+- `cargo test -p yoi parse_session_analyze_uses_session_mode`
+- `cargo fmt --check`
+- `git diff --check`
+- `git diff --cached --check`
+- `cargo run -q -p yoi -- ticket doctor` -> `doctor: ok`
+- `cargo check --workspace`
+- `nix build .#yoi`
+- `result/bin/yoi session analyze --help`
+
+Residual risks noted by coder:
+- Parser is intentionally tolerant and may emit more diagnostics if future session JSONL shapes change.
+- Context lifecycle support is based on currently observable `segment_start.compacted_from` and context/prune/compact-like extension events.
+- Bash file-inspection detection is a lightweight heuristic and should be treated as an observation, not automatic blame.
+
+---
+
+<!-- event: review author: reviewer-session-analytics-tooling at: 2026-06-09T07:37:41Z status: approve -->
+
+## Review: approve
+
+## Review result: approve
+
+Reviewed commit `c1809b37b1816a73e18c7587b9952cf4fc7b43f3` on branch `session-analytics-tooling` in worktree `.worktree/session-analytics-tooling`.
+
+### Evidence
+
+- The implementation adds `crates/session-analytics` with public `analyze_session(path) -> SessionReport` and a tolerant JSONL parser using `BufReader::lines()` / per-line `serde_json::Value` parsing.
+- The product CLI adds a thin `yoi session analyze <SESSION_JSONL_PATH> --json` surface through `crates/yoi/src/session_cli.rs`; analytics logic remains in the library crate.
+- Changed files are limited to workspace/package metadata, the new analytics crate, and the top-level product CLI wiring. I did not find changes to Worker pruning/compaction, Pod protocol/history/session writer semantics, TUI dashboarding, prompt auto-fixing, or summarization behavior.
+- Dependency direction is healthy for this Ticket: `session-analytics` depends only on `serde`, `serde_json`, and `thiserror` plus test `tempfile`; `cargo tree -i session-analytics --workspace` shows the product `yoi` crate as the consumer, not runtime/TUI crates.
+- Privacy boundary checked with a synthetic fixture containing secret-like user/file/tool argument/tool output strings. JSON output did not include raw `Read` content, `Edit` old/new strings, Bash command arguments, or Bash result content; it emitted paths, counts, byte/line sizes, line/turn indexes, and bounded observations.
+- Malformed and unknown JSONL entries are counted and surfaced as bounded diagnostics instead of crashing the whole report; unreadable input remains an error.
+- Tests are synthetic/minimal fixtures in the new crate / CLI tests, not private local sessions.
+
+### Validation run
+
+Passed:
+
+- `cargo test -p session-analytics`
+- `cargo test -p yoi session_cli`
+- `cargo test -p yoi parse_session_analyze_uses_session_mode`
+- synthetic CLI JSON privacy/metrics assertion fixture created during review
+- `cargo fmt --check`
+- `git diff --check HEAD~1..HEAD`
+- `cargo check --workspace`
+- `nix build .#yoi`
+- `result/bin/yoi session analyze --help`
+- `git diff --exit-code` / `git status --short` clean after review
+
+### Residual risks / notes
+
+- Tool result size reporting is most detailed for large-result and truncated/saved Bash observations; this meets the Ticket acceptance focus, but future aggregation could add per-tool output-size histograms if analysis needs it.
+- Repeated-read and Bash-file-inspection diagnostics are correctly framed as observations/correlations rather than automatic blame.
+- The new top-level `session` subcommand reserves that literal bare token; explicit `--pod session` remains the escape hatch for a Pod named `session`.
+
+---
+
+<!-- event: state_changed author: hare at: 2026-06-09T07:40:53Z from: inprogress to: closed reason: closed field: state -->
+
+## State changed
+
+Ticket を closed にしました。
+
+
+---
+
+<!-- event: close author: hare at: 2026-06-09T07:40:53Z status: closed -->
+
+## 完了
+
+Implemented, reviewed, merged, and validated.
+
+Summary:
+- Added reusable `crates/session-analytics` library with `analyze_session(path) -> SessionReport`.
+- Added thin product CLI `yoi session analyze <SESSION_JSONL_PATH> --json` that delegates to the library.
+- Implemented tolerant streaming JSONL parsing with bounded malformed/unknown-entry diagnostics.
+- Report covers tool usage counts, calls per turn, failed tool counts, repeated reads with mutation/context-lifecycle observations, edit/write churn, `replace_all`, large edit/result observations, saved/truncated Bash outputs, and compaction/prune/context correlations.
+- Default report avoids raw user input, raw tool arguments, raw file contents, raw session snippets, and raw tool output content.
+- Tests use synthetic/minimal fixtures rather than private local sessions.
+- No Worker pruning/compaction, Pod protocol/history/session writer, TUI dashboard, prompt auto-fixer, or LLM summarization behavior was changed.
+
+Implementation:
+- Coder commit: `c1809b3 feat: add session analytics tooling`
+- Reviewer approved with no blocking findings.
+- Merge commit: `0d2a6a7 merge: add session analytics tooling`
+
+Validation after merge:
+- `cargo test -p session-analytics`
+- `cargo test -p yoi session_cli`
+- `cargo test -p yoi parse_session_analyze_uses_session_mode`
+- `cargo fmt --check`
+- `git diff --check`
+- `cargo run -q -p yoi -- ticket doctor`
+- `cargo check --workspace`
+- `nix build .#yoi`
+
+
+---