274 lines
14 KiB
Markdown
274 lines
14 KiB
Markdown
<!-- event: create author: LocalTicketBackend at: 2026-06-09T03:25:33Z -->
|
|
|
|
## 作成
|
|
|
|
LocalTicketBackend によって作成されました。
|
|
|
|
---
|
|
|
|
<!-- event: intake_summary author: intake at: 2026-06-09T04:43:46Z -->
|
|
|
|
## Intake summary
|
|
|
|
既存 Ticket は、session JSONL を read-only に解析する再利用可能な `session-analytics` crate と薄い `yoi session analyze <path> --json` CLI を追加する concrete work item として十分に具体化済み。初期対象 metrics、raw content を既定で出さない privacy/secret 境界、non-goals、受け入れ条件、検証項目が明記されており、Orchestrator が実装 routing を判断できる。関連する `prompt-eval-metrics` は評価/改善 offer 寄りであり、本 Ticket は低レベル session/tool analytics 基盤として重複ではない。
|
|
|
|
---
|
|
|
|
<!-- event: state_changed author: intake at: 2026-06-09T04:43:46Z from: planning to: ready reason: intake_ready field: workflow_state -->
|
|
|
|
## State changed
|
|
|
|
Intake で要件が実装 routing 可能な粒度まで整理済みであることを確認したため、planning から ready にします。
|
|
|
|
---
|
|
|
|
<!-- event: state_changed author: workspace-panel at: 2026-06-09T06:56:20Z from: ready to: queued reason: queued field: state -->
|
|
|
|
## State changed
|
|
|
|
Ticket を `workspace-panel` が queued にしました。
|
|
|
|
|
|
---
|
|
|
|
<!-- event: state_changed author: orchestrator at: 2026-06-09T06:59:15Z from: queued to: inprogress reason: orchestrator_acceptance field: state -->
|
|
|
|
## State changed
|
|
|
|
Accepted queued implementation after reading the Ticket body/thread and current workspace state. The Ticket has concrete library/CLI/report/privacy acceptance criteria and does not depend on the just-landed Objective work. Implementation can proceed with synthetic fixture tests and read-only analysis semantics.
|
|
|
|
---
|
|
|
|
<!-- event: decision author: orchestrator at: 2026-06-09T06:59:15Z -->
|
|
|
|
## Decision
|
|
|
|
Routing decision: implementation_ready
|
|
|
|
Evidence checked:
|
|
- Ticket body and Intake summary.
|
|
- Current workspace state after Objective records merged, closed, and cleaned up.
|
|
- Current worktree state: no active implementation worktree remains.
|
|
- Current queued set: this is the only queued Ticket.
|
|
|
|
Reason:
|
|
- The Ticket is specific enough for implementation: add a reusable `session-analytics` crate, structured report model, and thin `yoi session analyze <path> --json` CLI.
|
|
- The privacy boundary is explicit: default output must not dump raw user input, file contents, secrets, or raw session snippets.
|
|
- The non-goals are clear: no pruning/compaction behavior changes, no TUI dashboard, no prompt/tool guidance auto-rewrite, no LLM semantic summarization.
|
|
|
|
IntentPacket:
|
|
|
|
Intent:
|
|
- Add read-only session JSONL analytics that extracts structured tool/session metrics and suspicious-pattern diagnostics without exposing raw content by default.
|
|
|
|
Binding decisions / invariants:
|
|
- Add a reusable library crate, e.g. `crates/session-analytics`, with an API such as `analyze_session(path) -> SessionReport`.
|
|
- Add a thin product CLI surface, e.g. `yoi session analyze <session-jsonl-or-path> --json`, that calls the library and avoids duplicating analytics logic.
|
|
- Prefer streaming/bounded parsing for large JSONL files; do not unnecessarily read huge files into memory if line-by-line parsing is practical.
|
|
- Analysis is read-only and has no runtime side effects.
|
|
- The analytics crate must stay independent from TUI rendering and Pod runtime side effects; runtime crates must not depend on analytics.
|
|
- Reuse current session/log schema types only where dependency direction remains healthy.
|
|
- Structured report should include at least:
|
|
- tool usage counts by name/kind;
|
|
- tool calls per turn;
|
|
- failed tool call counts;
|
|
- repeated `Read` by path and by path+offset/limit;
|
|
- whether repeated reads had intervening write/edit observations;
|
|
- edit/write churn by path;
|
|
- edit/write argument byte-size estimates and large replacement diagnostics;
|
|
- `replace_all` usage;
|
|
- observable tool result sizes/large/truncated indicators where available;
|
|
- prune/compaction events and bounded correlation with subsequent repeated reads/tool calls;
|
|
- malformed/unknown entry diagnostics.
|
|
- Suspicious diagnostics must be framed as observations, not automatic blame. Re-reading after mutation or validation may be correct.
|
|
- Default CLI/report output must avoid raw user input/file contents/session snippets. It may include paths, counts, byte sizes, turn/timestamp indexes, and hashes if useful.
|
|
- Tests should use synthetic/minimal session JSONL fixtures rather than relying on private local sessions.
|
|
- Do not implement a full dashboard, prompt auto-fixer, semantic summarizer, or Worker pruning/compaction behavior changes.
|
|
|
|
Implementation latitude:
|
|
- Coder may choose exact report struct field names and JSON format if machine-readable and test-covered.
|
|
- Coder may add a small JSONL adapter tolerant of unknown/malformed records with bounded diagnostics if current session log enum parsing is too strict.
|
|
- Coder may keep initial path input file-only; `--latest`, `--pod`, directory aggregation, and Markdown output can remain future work.
|
|
- If current session schema is too hard to reuse without unhealthy dependencies, implement a minimal analytics event parser for the observed JSONL shape and document the limitation.
|
|
|
|
Escalate if:
|
|
- Implementing this requires changing session writer/history schemas or Worker compaction behavior.
|
|
- Avoiding raw-content exposure conflicts with required metrics.
|
|
- Healthy crate dependency direction would require a larger session-schema extraction refactor.
|
|
- Real local session data would need to be read for tests/validation; use synthetic fixtures instead unless explicitly authorized.
|
|
|
|
Validation:
|
|
- Focused unit tests for repeated reads with/without intervening edit/write.
|
|
- Focused tests for large edit argument sizing and replace_all observations.
|
|
- Focused tests for tool failure counting and tool calls per turn.
|
|
- Focused tests for large/truncated result observations if representable in fixtures.
|
|
- Focused tests for compaction/prune event correlation as correlation only.
|
|
- Focused tests for malformed/unknown JSONL entries producing bounded diagnostics.
|
|
- CLI JSON output test for `yoi session analyze <path> --json` or equivalent.
|
|
- `cargo fmt --check`.
|
|
- `git diff --check`.
|
|
- `cargo run -q -p yoi -- ticket doctor`.
|
|
- `cargo check --workspace`.
|
|
- `nix build .#yoi`.
|
|
|
|
---
|
|
|
|
<!-- event: implementation_report author: orchestrator at: 2026-06-09T07:00:26Z -->
|
|
|
|
## Implementation report
|
|
|
|
Implementation routing started.
|
|
|
|
Worktree/branch:
|
|
- Worktree: `.worktree/session-analytics-tooling`
|
|
- Branch: `session-analytics-tooling`
|
|
- Base/routing commit: `c166140 ticket: route session analytics tooling`
|
|
|
|
Spawned sibling Coder Pod:
|
|
- `coder-session-analytics-tooling`
|
|
- Scope: non-recursive read on parent workspace root plus recursive write limited to the child worktree.
|
|
|
|
Routing context:
|
|
- This Ticket is now the active implementation work.
|
|
- Handoff explicitly requires synthetic/minimal session fixtures for validation and avoids reading private real local session contents.
|
|
|
|
---
|
|
|
|
<!-- event: implementation_report author: orchestrator at: 2026-06-09T07:26:46Z -->
|
|
|
|
## Implementation report
|
|
|
|
Coder implementation completed and is ready for sibling review.
|
|
|
|
Coder Pod:
|
|
- `coder-session-analytics-tooling`
|
|
- Commit: `c1809b37b1816a73e18c7587b9952cf4fc7b43f3 feat: add session analytics tooling`
|
|
- Worktree status before review: clean branch `session-analytics-tooling`
|
|
- Stopped after collecting output to reclaim delegated worktree scope.
|
|
|
|
Selected design:
|
|
- Added `crates/session-analytics` with public `analyze_session(path) -> SessionReport`.
|
|
- Parser is a tolerant `serde_json::Value` JSONL parser using `BufReader::lines()` instead of tightly coupling analytics to runtime schema enums.
|
|
- Report avoids raw user input, raw tool arguments, raw file contents, and raw tool output snippets by default; it reports paths, counts, byte/line sizes, line/turn indexes, bounded diagnostics, and observation text.
|
|
- Added thin CLI `yoi session analyze <SESSION_JSONL_PATH> --json` that delegates to the library crate.
|
|
|
|
Implementation summary:
|
|
- Tool usage summary: counts by tool name/kind, calls per turn, failed tool result count, and Bash file-inspection-like heuristic observations.
|
|
- Repeated reads: by path and path+offset/limit, with intervening Edit/Write observation and post-context-lifecycle correlation.
|
|
- Edit/write churn: counts by path, repeated edits, argument byte-size estimates, large replacement diagnostics, and `replace_all` observations.
|
|
- Tool result size: output byte/line sizes, large result observations, and saved/truncated Bash result indicators.
|
|
- Context lifecycle: observes `segment_start.compacted_from` and context/prune/compact-like extension events; correlation only, not causation.
|
|
- Robustness: malformed/unknown JSONL entries produce bounded diagnostics; unreadable input file remains an error.
|
|
|
|
Changed files:
|
|
- `Cargo.toml`
|
|
- `Cargo.lock`
|
|
- `package.nix`
|
|
- `crates/session-analytics/Cargo.toml`
|
|
- `crates/session-analytics/src/lib.rs`
|
|
- `crates/yoi/Cargo.toml`
|
|
- `crates/yoi/src/main.rs`
|
|
- `crates/yoi/src/session_cli.rs`
|
|
|
|
Coder validation reported passed:
|
|
- `cargo test -p session-analytics`
|
|
- `cargo test -p yoi session_cli`
|
|
- `cargo test -p yoi parse_session_analyze_uses_session_mode`
|
|
- `cargo fmt --check`
|
|
- `git diff --check`
|
|
- `git diff --cached --check`
|
|
- `cargo run -q -p yoi -- ticket doctor` -> `doctor: ok`
|
|
- `cargo check --workspace`
|
|
- `nix build .#yoi`
|
|
- `result/bin/yoi session analyze --help`
|
|
|
|
Residual risks noted by coder:
|
|
- Parser is intentionally tolerant and may emit more diagnostics if future session JSONL shapes change.
|
|
- Context lifecycle support is based on currently observable `segment_start.compacted_from` and context/prune/compact-like extension events.
|
|
- Bash file-inspection detection is a lightweight heuristic and should be treated as an observation, not automatic blame.
|
|
|
|
---
|
|
|
|
<!-- event: review author: reviewer-session-analytics-tooling at: 2026-06-09T07:37:41Z status: approve -->
|
|
|
|
## Review: approve
|
|
|
|
## Review result: approve
|
|
|
|
Reviewed commit `c1809b37b1816a73e18c7587b9952cf4fc7b43f3` on branch `session-analytics-tooling` in worktree `.worktree/session-analytics-tooling`.
|
|
|
|
### Evidence
|
|
|
|
- The implementation adds `crates/session-analytics` with public `analyze_session(path) -> SessionReport` and a tolerant JSONL parser using `BufReader::lines()` / per-line `serde_json::Value` parsing.
|
|
- The product CLI adds a thin `yoi session analyze <SESSION_JSONL_PATH> --json` surface through `crates/yoi/src/session_cli.rs`; analytics logic remains in the library crate.
|
|
- Changed files are limited to workspace/package metadata, the new analytics crate, and the top-level product CLI wiring. I did not find changes to Worker pruning/compaction, Pod protocol/history/session writer semantics, TUI dashboarding, prompt auto-fixing, or summarization behavior.
|
|
- Dependency direction is healthy for this Ticket: `session-analytics` depends only on `serde`, `serde_json`, and `thiserror` plus test `tempfile`; `cargo tree -i session-analytics --workspace` shows the product `yoi` crate as the consumer, not runtime/TUI crates.
|
|
- Privacy boundary checked with a synthetic fixture containing secret-like user/file/tool argument/tool output strings. JSON output did not include raw `Read` content, `Edit` old/new strings, Bash command arguments, or Bash result content; it emitted paths, counts, byte/line sizes, line/turn indexes, and bounded observations.
|
|
- Malformed and unknown JSONL entries are counted and surfaced as bounded diagnostics instead of crashing the whole report; unreadable input remains an error.
|
|
- Tests are synthetic/minimal fixtures in the new crate / CLI tests, not private local sessions.
|
|
|
|
### Validation run
|
|
|
|
Passed:
|
|
|
|
- `cargo test -p session-analytics`
|
|
- `cargo test -p yoi session_cli`
|
|
- `cargo test -p yoi parse_session_analyze_uses_session_mode`
|
|
- synthetic CLI JSON privacy/metrics assertion fixture created during review
|
|
- `cargo fmt --check`
|
|
- `git diff --check HEAD~1..HEAD`
|
|
- `cargo check --workspace`
|
|
- `nix build .#yoi`
|
|
- `result/bin/yoi session analyze --help`
|
|
- `git diff --exit-code` / `git status --short` clean after review
|
|
|
|
### Residual risks / notes
|
|
|
|
- Tool result size reporting is most detailed for large-result and truncated/saved Bash observations; this meets the Ticket acceptance focus, but future aggregation could add per-tool output-size histograms if analysis needs it.
|
|
- Repeated-read and Bash-file-inspection diagnostics are correctly framed as observations/correlations rather than automatic blame.
|
|
- The new top-level `session` subcommand reserves that literal bare token; explicit `--pod session` remains the escape hatch for a Pod named `session`.
|
|
|
|
---
|
|
|
|
<!-- event: state_changed author: hare at: 2026-06-09T07:40:53Z from: inprogress to: closed reason: closed field: state -->
|
|
|
|
## State changed
|
|
|
|
Ticket を closed にしました。
|
|
|
|
|
|
---
|
|
|
|
<!-- event: close author: hare at: 2026-06-09T07:40:53Z status: closed -->
|
|
|
|
## 完了
|
|
|
|
Implemented, reviewed, merged, and validated.
|
|
|
|
Summary:
|
|
- Added reusable `crates/session-analytics` library with `analyze_session(path) -> SessionReport`.
|
|
- Added thin product CLI `yoi session analyze <SESSION_JSONL_PATH> --json` that delegates to the library.
|
|
- Implemented tolerant streaming JSONL parsing with bounded malformed/unknown-entry diagnostics.
|
|
- Report covers tool usage counts, calls per turn, failed tool counts, repeated reads with mutation/context-lifecycle observations, edit/write churn, `replace_all`, large edit/result observations, saved/truncated Bash outputs, and compaction/prune/context correlations.
|
|
- Default report avoids raw user input, raw tool arguments, raw file contents, raw session snippets, and raw tool output content.
|
|
- Tests use synthetic/minimal fixtures rather than private local sessions.
|
|
- No Worker pruning/compaction, Pod protocol/history/session writer, TUI dashboard, prompt auto-fixer, or LLM summarization behavior was changed.
|
|
|
|
Implementation:
|
|
- Coder commit: `c1809b3 feat: add session analytics tooling`
|
|
- Reviewer approved with no blocking findings.
|
|
- Merge commit: `0d2a6a7 merge: add session analytics tooling`
|
|
|
|
Validation after merge:
|
|
- `cargo test -p session-analytics`
|
|
- `cargo test -p yoi session_cli`
|
|
- `cargo test -p yoi parse_session_analyze_uses_session_mode`
|
|
- `cargo fmt --check`
|
|
- `git diff --check`
|
|
- `cargo run -q -p yoi -- ticket doctor`
|
|
- `cargo check --workspace`
|
|
- `nix build .#yoi`
|
|
|
|
|
|
---
|