<!-- event: create author: tickets.sh at: 2026-06-01T00:16:16Z -->

## Created

Created by tickets.sh create.

---

<!-- event: plan author: hare at: 2026-06-01T00:16:59Z -->

## Plan

## Investigation notes

- Representative session: `~/.insomnia/sessions/019e8042-be06-72e2-bc80-05afdfde4515/`.
- First segment: `019e8042-be06-72e2-bc80-05b98007803a.jsonl`.
- Compact segment: `019e8043-4d63-7231-b0c7-3d356e86665a.jsonl` with `compacted_from.at_turn_index = 1`.
- The compact path appears to be mid-turn request threshold yield, not prune itself:
  - `PodInterceptor::pre_llm_request()` checks `state.exceeds_request(current)`.
  - `PreRequestAction::Yield` becomes `WorkerResult::Yielded`.
  - `Pod::handle_worker_result()` runs `do_compact_and_resume()`.
- `prune.fire` observed in the same segment is useful context when reading the log, but prune is not the compact trigger and this ticket does not require prune behavior changes.

## Design constraint

Do not model system and history as exactly separable token domains unless the implementation can measure them as such. For compact thresholding, the stable property is whole request prompt occupancy.


---

<!-- event: plan author: hare at: 2026-06-01T00:41:18Z -->

## Plan

## Preflight classification

implementation-ready.

The ticket is a bounded bug fix in compact/request-threshold token accounting. The intended behavior is clear: compact thresholding should estimate whole request prompt occupancy and must not divide provider `input_total_tokens` by history-only bytes. Prune behavior is explicitly out of scope.

## Requirements sync

Observable completion:

- A fresh session / one prior usage record case with large fixed prompt overhead does not trigger request-threshold yield solely because history grew after the first measured request.
- Measured `input_total_tokens` remains authoritative for the exact measured request occupancy.
- Unmeasured request occupancy estimation uses a size measure that corresponds to the same whole request context being estimated, not history-only bytes.

Non-goals:

- Do not change prune behavior or prune savings policy for this ticket.
- Do not change compact thresholds or profile defaults as the fix.
- Do not alter session log schema unless the implementation finds it necessary and escalates first.

## Current code map

- `crates/llm-worker/src/token_counter.rs`: shared token estimate functions used for compact/request thresholding; current extrapolation uses history prefix bytes.
- `crates/pod/src/ipc/interceptor.rs`: `pre_llm_request` computes request-context estimate and yields when `request_threshold` is exceeded.
- `crates/pod/src/compact/token_counter.rs`: Pod-side wrappers and tests around token estimates; prune helpers exist here but are not in ticket scope.
- `crates/pod/src/compact/usage_tracker.rs`: captures usage records keyed by in-flight request history length.
- `crates/pod/src/compact/state.rs`: threshold semantics; should not need behavior changes.
- `crates/llm-worker/src/worker.rs`: request loop and prune projection before `pre_llm_request`; should not need lifecycle changes.

## Critical risks

- Fixing the estimate by simply raising thresholds or disabling request-threshold yield would hide the bug and is not acceptable.
- Splitting system/tool/history into separate exact token domains is not warranted unless the implementation can measure the same request shape consistently.
- Regression tests must exercise the one-measurement case, because that is where fixed prompt overhead previously dominated the inferred history rate.
- Reviewer should verify that prune behavior was not intentionally changed.

## Intent packet

Intent:
- Fix compact/request-threshold token occupancy estimation so whole prompt usage is not projected from history-only bytes.

Requirements:
- Treat exact usage records as authoritative for the measured request occupancy.
- Estimate unmeasured whole request occupancy using a request-size basis that corresponds to the whole request context, or a conservative fallback that does not allocate fixed prompt overhead to history bytes.
- Add regression coverage for first-turn/fresh-session overestimation.

Invariants:
- Compact remains triggered by threshold semantics, not by prune activity.
- Prune behavior is out of scope and should not be changed intentionally.
- Do not introduce a false exact system/history token split.
- Do not modify profile thresholds as the fix.

Escalate if:
- The clean fix requires session-log schema changes, provider request serialization changes, or durable migration.
- The implementation would change prune behavior or compact lifecycle semantics.

Validation:
- Focused Rust tests for `llm-worker` token counter and pod compact/interceptor behavior as applicable.
- `cargo test -p llm-worker token_counter` or narrower exact test target if available.
- `cargo test -p pod compact` or focused pod tests if touched.
- `cargo check --workspace` if focused tests pass and runtime is reasonable.
- `./tickets.sh doctor` in main workspace before finalization.


---

<!-- event: review author: hare at: 2026-06-01T00:59:20Z status: approve -->

## Review: approve

External reviewer recommendation: approve.

Summary:
- Exact UsageRecord matches remain authoritative.
- Extrapolation after the latest measurement no longer uses `latest_total_input_tokens / latest_history_bytes`.
- Interpolation between two measurements still uses measured deltas where fixed overhead cancels.
- One-measurement extrapolation uses a conservative byte fallback added to latest measured total.
- Regression coverage exists in `llm-worker` token counter tests and pod `pre_llm_request` behavior.
- No intentional prune policy/savings, threshold/default, session schema, or compact lifecycle changes were found.

Validation re-run by reviewer:
- `cargo test -p llm-worker token_counter` passed.
- `cargo test -p pod pre_llm_request_does_not_yield_from_single_measurement_history_rate_projection` passed.
- `git diff --check 3ea0058..HEAD` passed.
- `cargo fmt --check` passed.

Non-blocking follow-up:
- Some comments still describe extrapolation as a latest/final measurement rate even though the implementation is now latest measured incremental span or byte fallback. Reviewer classified this as documentation drift only, not a blocker.


---

<!-- event: close author: hare at: 2026-06-01T01:10:06Z status: closed -->

## Closed

Merged and completed.

Implementation:
- Merged branch `prompt-occupancy-token-estimator` into `develop` with `merge: prompt occupancy estimator`.
- `llm-worker` token counter extrapolation now keeps exact measured prompt occupancy authoritative and no longer extrapolates one-measurement growth via `total_input_tokens / history_bytes`.
- Extrapolation past the latest measurement uses a measured incremental span rate when available; otherwise it adds a conservative byte fallback for the unmeasured delta.
- Added pod interceptor regression coverage for the fresh-session / one-measurement overestimation case.

Validation after merge:
- `cargo test -p llm-worker token_counter` passed.
- `cargo test -p pod pre_llm_request_does_not_yield_from_single_measurement_history_rate_projection` passed.
- `./tickets.sh doctor` passed.

Review:
- External reviewer approved with no blockers.


---
