ticket: pod-store split direction

work-items/open/20260529-205844-session-pod-state-boundary/item.md

@@ -1,13 +1,13 @@
 ---
 id: 20260529-205844-session-pod-state-boundary
 slug: session-pod-state-boundary
-title: Split session log storage from Pod metadata storage
+title: Split Pod metadata into a dedicated pod-store crate
 status: open
 kind: task
 priority: P2
-labels: [session-store, pod, persistence, architecture]
+labels: [session-store, pod-store, pod, persistence, architecture]
 created_at: 2026-05-29T20:58:44Z
-updated_at: 2026-05-29T21:30:26Z
+updated_at: 2026-05-29T22:00:16Z
 assignee: null
 legacy_ticket: null
 ---
@@ -17,11 +17,22 @@ legacy_ticket: null
 The current persistence design intentionally has two durable surfaces:
 
 - append-only session/segment logs, which are the authority for conversation/history state and segment lineage;
-- name-keyed Pod metadata, which is supposed to be a thin pointer layer for Pod-name attach/restore and spawned-child bookkeeping.
+- name-keyed Pod metadata, which is the authority for Pod-name attach/restore pointers and durable spawned-child bookkeeping.
 
 That boundary has become blurry. The `session-store` crate is named and documented primarily as session persistence, but it also owns Pod metadata types, the `PodMetadataStore` trait, validation of Pod names, and the filesystem layout `{sessions_root}/pods/{pod_name}/metadata.json`. In addition, Pod metadata currently stores `spawned_children` and `resolved_manifest_snapshot`, while session logs also store Pod scope snapshots as `LogEntry::Extension` entries. This creates a risk that session-log authority, Pod-state authority, and runtime mirrors drift or become hard to reason about.
 
-The chosen direction for this ticket is to split the durable surfaces instead of documenting the current shape as acceptable. Pod metadata is not a child resource of the session log store: it should live under a Pod-state root such as `{data_dir}/pods/{pod_name}/metadata.json`, while segment logs remain under `{data_dir}/sessions/{session_id}/{segment_id}.jsonl`. Pod metadata may point at a `(SessionId, SegmentId)`, but the session log store must not own Pod metadata types or the Pod metadata filesystem layout.
+This happened because earlier implementation work treated `session-store` as a convenient place for every durable file under the sessions root. That shape should not be extended. The chosen direction for this ticket is to split the durable surfaces into separate crates/APIs: `session-store` remains the session/segment JSONL store, and a new `pod-store` crate owns Pod metadata, Pod-name validation, and the Pod metadata filesystem layout.
+
+## Decisions
+
+- Introduce a dedicated `pod-store` crate for durable Pod metadata/state.
+- Move Pod metadata storage from `{sessions_root}/pods/{pod_name}/metadata.json` to a top-level Pod-state root such as `{data_dir}/pods/{pod_name}/metadata.json`.
+- Do not provide backward compatibility or migration for the obsolete `{sessions_root}/pods` layout. Existing old-layout Pod metadata may be ignored/lost by this change.
+- Redesign the Pod metadata API where needed instead of preserving awkward `session-store`-shaped APIs.
+- Keep session logs as the authority for conversation/history replay and segment-bound runtime state.
+- Keep runtime mirrors such as sockets, lock-file allocations, and `spawned_pods.json` as live runtime views, not durable authority.
+
+Pod metadata may point at a `(SessionId, SegmentId)`, but the session log store must not own Pod metadata types or the Pod metadata filesystem layout. If sharing ID types directly causes an undesirable dependency, introduce a small shared ID module/crate or otherwise keep the dependency narrow; do not let `pod-store` pull in session replay concerns just to name a session pointer.
 
 Observed code points:
 
@@ -34,7 +45,7 @@ Observed code points:
 
 ## Goal
 
-Refactor the architectural boundary between session logs, Pod metadata/state, and runtime mirrors so the storage APIs and filesystem layout match their authority boundaries, without changing user-visible restore/attach semantics.
+Refactor the architectural boundary between session logs, Pod metadata/state, and runtime mirrors so the storage APIs, crate boundaries, and filesystem layout match their authority boundaries, without changing intended restore/attach semantics for newly written state.
 
 ## Desired boundary
 
@@ -46,38 +57,45 @@ The resulting design should make these responsibilities explicit:
   - request config / usage / metrics / memory extension records;
   - Pod runtime scope snapshots required to restore the same session without silently reclaiming delegated writes;
   - filesystem layout under the session log root, e.g. `{data_dir}/sessions/{session_id}/{segment_id}.jsonl` and associated trace logs.
-- Pod metadata authority:
+- Pod metadata authority, owned by `pod-store`:
+  - Pod-name validation and safe filesystem key rules;
   - name-keyed active `(SessionId, SegmentId)` pointer;
+  - pending/new Pod state if needed before a session segment is materialized;
   - resolved manifest snapshot needed for Pod-name restore when the source profile/manifest should not be re-evaluated;
-  - spawned-child registry state, if retained here, with a documented reason why it is Pod state rather than session state;
+  - spawned-child registry state, because it is current parent-Pod state rather than conversation history;
+  - delegated child scope records needed for prune/reclaim bookkeeping;
   - filesystem layout under a Pod-state root, e.g. `{data_dir}/pods/{pod_name}/metadata.json`, not below the session log root.
 - Runtime mirrors:
-  - sockets, lock-file allocations, and `spawned_pods.json` are live runtime views, not durable authority.
+  - sockets, lock-file allocations, and `spawned_pods.json` are live runtime views, not durable authority;
+  - socket paths and callback addresses, if retained in durable metadata, must be documented as last-known runtime hints rather than proof of liveness.
 
 ## Acceptance criteria
 
-- Audit every public type/function in `session-store` related to Pod metadata and classify whether it is genuinely session-log responsibility or Pod-state responsibility.
-- Move/split Pod metadata APIs out of the session log API surface so session log APIs do not expose Pod metadata concepts unnecessarily.
+- Create a new `pod-store` crate and move Pod metadata types/store traits/filesystem implementation into it.
+- Remove `pod_metadata` exports and Pod metadata filesystem ownership from `session-store`; update `session-store` crate/module docs so it describes session/segment logs rather than Pod metadata.
 - Move the durable Pod metadata layout out of `{sessions_root}/pods/{pod_name}/metadata.json` to a Pod-state root such as `{data_dir}/pods/{pod_name}/metadata.json`.
+- Do not implement compatibility fallback or migration for `{sessions_root}/pods`; tests should assert the old path is not read or written as an authority.
+- Redesign the Pod metadata API where useful. At minimum, avoid caller-side read-modify-write helpers that can silently drop unrelated fields; provide explicit update/merge operations or otherwise make field-preservation semantics safe and testable.
 - Update construction/configuration paths so callers pass distinct roots or distinct store handles for session logs and Pod metadata; sharing the same higher-level data directory is allowed, but the session log store must not own the Pod metadata subdirectory.
+- Update `pod`, `tui`, and other callers to depend on/use `pod-store` for Pod metadata instead of importing Pod metadata through `session-store` or reading metadata files directly.
+- Remove direct filesystem reads of `pods/*/metadata.json` outside the `pod-store` abstraction, especially in TUI Pod list/discovery paths.
 - Document the new boundary in code comments and/or crate/module docs, including why Pod metadata points to session IDs rather than being contained by the session store.
-- Remove direct filesystem reads of `pods/*/metadata.json` outside the Pod metadata store abstraction, especially in TUI Pod list/discovery paths.
-- Clarify the authority of `resolved_manifest_snapshot`: it belongs to Pod-name restore state unless a different Pod-state record is deliberately introduced; ensure restore paths follow the documented authority.
-- Clarify the authority of `spawned_children`: it belongs to Pod-state/durable child-registry state unless deliberately moved; ensure restore/prune/reclaim behavior follows the documented authority.
-- Ensure read-modify-write preservation of unrelated Pod metadata fields does not silently lose data when active pointer updates and spawned-child updates occur near each other; either make the update semantics explicit or add a safer merge/update API.
-- Preserve the current durable behavior unless deliberately changed:
-  - Pod-name restore resolves active metadata then restores the session log;
+- Clarify the authority of `resolved_manifest_snapshot`: it belongs to Pod-name restore state in `pod-store`; session JSONL `SegmentStart` config/system prompt remain the authority for replaying an existing segment.
+- Clarify the authority of `spawned_children`: it belongs to Pod-state/durable child-registry state in `pod-store`; child lifecycle messages shown to the model remain session JSONL history.
+- Clarify delegated scope handling: child delegated-scope records live in `pod-store` for child registry/reclaim, while effective parent scope snapshots remain in session JSONL for restore safety.
+- Preserve intended durable behavior for newly written state:
+  - Pod-name restore resolves active metadata from `pod-store` then restores the session log from `session-store`;
   - session restore uses session log state and scope snapshots;
   - runtime `spawned_pods.json` remains a mirror;
   - stopped or unreachable child Pod metadata is not deleted merely because its socket is gone.
-- Decide and document the migration stance for existing `{sessions_root}/pods` data. If migration is provided, it must be one-shot and must not keep the old path as a long-term fallback authority.
-- Add focused tests for the chosen split, including active pointer updates preserving spawned children / manifest snapshot, spawned-child updates preserving active pointer / manifest snapshot, and discovery/restore behavior when one durable surface exists without the other.
+- Add focused tests for the split, including active pointer updates preserving spawned children / manifest snapshot, spawned-child updates preserving active pointer / manifest snapshot, and discovery/restore behavior when one durable surface exists without the other.
 - Add or update tests that verify Pod metadata is read/written under the new Pod-state root and not under the session log root.
+- Run focused validation for `session-store`, `pod-store`, `pod`, and `tui`, plus `./tickets.sh doctor` and `git diff --check`.
 - Update any relevant docs or workflow notes if the persistence model changes.
 
 ## Non-goals
 
-- Do not redesign the session-log schema unless the audit proves it is necessary.
-- Do not keep backward compatibility for obsolete persistence layouts as a permanent fallback; a one-shot migration may be implemented only if explicitly chosen and documented.
+- Do not redesign the session-log schema unless the split proves it is necessary.
+- Do not preserve backward compatibility for obsolete `{sessions_root}/pods` metadata, and do not implement a permanent fallback or migration path.
 - Do not change live Pod registry lock semantics except where necessary to align with the clarified durable authority.
 - Do not implement broader database storage or transactional storage in this ticket; if the boundary audit reveals a need for transactions, record it as a follow-up unless a minimal update API suffices.