ticket: use base32 project record ids

This commit is contained in:
2026-06-09 22:10:47 +09:00
parent 0803bc3725
commit 4203988d74
798 changed files with 477 additions and 105 deletions
@@ -0,0 +1,46 @@
Implemented and merged local key-value secret store support.
Merged commits:
- `cc2c9a2 secrets: add local key store`
- `7ddf745 secrets: polish key manager and docs`
- `629159a merge: local secret store`
Review:
- Review approved in `c9e48b3 review: approve local secret store`.
- Focused follow-up review approved the docs example and key-manager terminal cleanup polish.
Summary:
- Added a provider-independent local `id -> value` secret store under the user data directory.
- Added id validation, atomic persistence, and lightweight at-rest obfuscation consistent with the ticket's modest security target.
- Added `insomnia keys` interactive TUI management for listing ids, setting values with masked display, deleting with confirmation, and quitting without displaying plaintext values.
- Wired provider `secret_ref` auth through the store.
- Added WebSearch `api_key_secret` and removed normal WebSearch/provider credential env configuration.
- Updated bundled resources and docs to point users to `insomnia keys` plus explicit secret refs.
- Left Codex OAuth behavior unchanged.
Validation after merge:
- `cargo fmt --check` — passed
- `cargo test -p secrets` — passed
- `cargo test -p manifest secret --lib` — passed
- `cargo test -p provider secret --lib` — passed
- `cargo test -p tools web::tests::search_requires_configuration --lib` — passed
- `cargo test -p tools web::tests::searches_brave_with_secret_ref --lib` — passed
- `cargo test -p tools web::tests::searches_brave_with_bounded_output --lib` — passed
- `cargo test -p tui keys::tests --lib` — passed
- `cargo test -p insomnia parse_keys_subcommand --bin insomnia` — passed
- `cargo check -p manifest -p provider -p tools -p tui -p insomnia` — passed
- `./tickets.sh doctor` — passed
- `git diff --check` — passed
Credential/env grep:
- `api_key_env`, `BRAVE_SEARCH_API_KEY`, `INSOMNIA_API_KEY`, and `default_env_var` are absent from `crates docs resources` after the merge.
- Remaining `sk-`/`secret-value`/`test-secret` hits are fake test values, docs/comments, or Codex OAuth test fixtures, not new persisted real credentials.
Caveat:
- The store should continue to be described as local obfuscation / limited at-rest protection, not a high-assurance password manager.
@@ -0,0 +1,34 @@
# Review: local key-value secret store implementation
Implementation reviewed on branch `manifest-profile-encrypted-secrets`.
Reviewed commits:
- `cc2c9a2 secrets: add local key store`
- `7ddf745 secrets: polish key manager and docs`
Verdict: approve.
Summary:
- Core provider-independent `id -> value` local secret store satisfies the ticket model.
- Store values are not persisted as casual plaintext and error/debug surfaces avoid secret values within the stated modest protection boundary.
- Provider auth now resolves explicit `secret_ref` values through the local store without env credential fallback.
- WebSearch uses explicit `api_key_secret` and no longer depends on `BRAVE_SEARCH_API_KEY` / `api_key_env` in the normal path.
- `insomnia keys` provides interactive list/add-set/delete management without displaying plaintext values.
- Codex OAuth behavior remains separate and unchanged.
- Follow-up review confirmed the docs credential example is schema-valid and key-manager terminal setup cleanup was added.
Validation reported by coder/reviewer:
- `cargo fmt --check`
- `cargo test -p secrets`
- focused manifest/provider/tools/tui/insomnia tests
- `cargo check -p manifest -p provider -p tools -p tui -p insomnia`
- `./tickets.sh doctor`
- `git diff --check`
- credential/env greps confirming `api_key_env`, `BRAVE_SEARCH_API_KEY`, `INSOMNIA_API_KEY`, and `default_env_var` are absent from crates/docs/resources
Remaining caveat:
- Continue to describe this as local obfuscation / limited at-rest protection, not a high-assurance password manager or OS-keychain-backed vault.
+148
View File
@@ -0,0 +1,148 @@
---
title: "Manifest/Profile: local key-value secret store"
state: "closed"
created_at: "2026-05-29T14:53:55Z"
updated_at: "2026-05-31T22:23:34Z"
---
## Background
Credential configuration still relies on process environment variables in important paths:
- provider API keys use `AuthRef::ApiKey { env, file }` and provider-default `INSOMNIA_API_KEY_*` names;
- WebSearch currently uses `web.search.api_key_env`;
- normal runtime intentionally does not load `.env` files.
This should not be solved by implicit `.env` loading. `.env` files are easy to leak into projects, do not solve profile-specific credential selection cleanly, and still expose secrets through process environments.
The desired replacement is a local key-value secret store plus explicit references from manifest/profile/tool configuration.
The security target is intentionally modest. This is not a high-assurance password manager. The goal is to avoid casual plaintext exposure and generic environment-variable scraping, not to defend against a local attacker with the user's account or process memory access.
## Intent
Implement a provider-independent local key-value secret store and use it as the normal credential path for provider and WebSearch credentials.
The logical store model is just:
```text
{
"anthropic/default" = "sk-..."
"web/brave/default" = "..."
}
```
The store must not know that a key is Anthropic, Brave, OpenAI, or any other provider-specific kind. Provider/model/tool configuration chooses which key to reference.
## Requirements
### Store model
- Add a local secret/key store that maps a validated string id to a secret string value.
- Keep the user-visible logical schema provider-independent: `id -> value`.
- Do not add provider-specific slots, credential kinds, or required metadata to the store schema.
- Technical envelope fields needed for versioning/nonce/ciphertext/checksum are allowed, but they must not become user-facing semantic metadata.
- Store data outside the repository, under the user data directory, e.g. `<data_dir>/secrets/store.json` or equivalent.
- Use atomic writes.
- Validate ids:
- reject empty ids;
- reject path traversal / absolute-path-like ids;
- reject control characters;
- bound length;
- allow a conservative useful set such as ASCII alnum plus `._/-`.
### Obfuscation / encryption stance
- Apply lightweight encryption or obfuscation at rest so the file is not a casual plaintext key dump.
- Do not claim strong local security guarantees.
- Do not introduce OS keychain dependency or interactive passphrase UX in this ticket.
- Do not store plaintext values in logs, work items, session history, diagnostics, or normal command output.
- Decryption/decoding failures must fail closed and name only the key id, not the value.
### `insomnia keys` TUI management
- Add `insomnia keys` as an interactive TUI key manager.
- The product CLI owner (`insomnia` crate) routes the subcommand.
- Use the TUI implementation crate for the terminal screen if practical.
- Minimum UI features:
- list key ids;
- add/set a key;
- delete a key with confirmation;
- quit.
- Do not display plaintext values in the list or normal screen output.
- During add/set, mask the value input or otherwise avoid echoing plaintext.
- Scriptable commands such as `insomnia keys set <id> --stdin` may be added if convenient, but the required user surface is the TUI manager.
### Config references / consumers
- Use explicit secret references from configuration.
- Existing `AuthRef::SecretRef { ref_ }` in manifest model should resolve through the new store for provider API keys.
- WebSearch must gain an explicit secret reference path so Brave search can be configured without `BRAVE_SEARCH_API_KEY` / `api_key_env`.
- Prefer a generic auth/secret-ref shape if it stays small.
- A focused `api_key_secret = "web/brave/default"` field is acceptable if it avoids a broad schema redesign.
- Secret refs are resolved at the consumer/runtime boundary only; resolved config/debug output must not contain plaintext.
- The store must not implicitly choose default keys based on provider name. No ambient lookup like "anthropic automatically reads anthropic/default" unless the profile/config explicitly references it.
### Env credential removal
- Do not load `.env` files.
- Do not add new credential environment variables.
- Do not keep migration/backward-compatibility behavior for credential env config in the normal profile path.
- Remove credential env configuration from normal provider/WebSearch use as part of this ticket.
- Docs and diagnostics should point users to `insomnia keys` + secret refs as the credential path.
### Codex OAuth relationship
- Codex OAuth is not part of this key-value secret store in this ticket.
- Current Codex OAuth intentionally interoperates with Codex CLI's `auth.json` file and refresh behavior; that file contains a structured token bundle, not a single provider API key string.
- Do not store or refresh Codex OAuth token bundles through the key-value store as part of this ticket.
- Do not change `CODEX_HOME` / `$HOME/.codex` lookup behavior in this ticket.
- A future Insomnia-owned Codex login/token store could be designed separately if needed, but it should be a dedicated OAuth token-store design, not an implicit use of the simple key-value API-key store.
## Phases within this ticket
1. Core store
- key-value store API;
- id validation;
- lightweight encrypted/obfuscated file format;
- atomic load/save;
- focused tests.
2. `insomnia keys` TUI manager
- list/add/delete;
- masked input;
- no plaintext display.
3. Provider integration
- implement provider `AuthRef::SecretRef` resolution through the store;
- keep plaintext in memory only;
- fail closed on missing/invalid/decode failures.
4. WebSearch integration
- add a secret-ref credential path;
- make Brave search usable without env credentials.
5. Docs and env removal
- update `docs/environment.md` and manifest/profile docs;
- document the modest security target honestly;
- point users to `insomnia keys` and secret refs as the credential path;
- remove credential env configuration from normal provider/WebSearch docs and code paths.
## Non-goals
- A high-assurance password manager.
- OS keychain integration.
- Passphrase prompt UX.
- Provider-specific secret-store schema.
- Automatic provider-name-to-secret-id lookup.
- Loading `.env` files.
- Changing Codex OAuth behavior. Codex OAuth remains an external structured token-source integration in this ticket.
- Reworking model/provider catalog ownership.
## Acceptance criteria
- A user can run `insomnia keys` and manage key ids interactively.
- The store persists key-value entries under the user data directory without plaintext values in the on-disk file.
- Store id validation rejects unsafe ids.
- Provider `AuthRef::SecretRef` resolves through the store and does not print/serialize plaintext.
- WebSearch can use a configured secret ref without exporting an environment variable.
- Missing key, invalid id, unreadable store, and decode/decrypt failure produce clear fail-closed errors naming only the key id.
- `docs/environment.md` no longer presents credential env vars as the normal path, removes normal provider/WebSearch credential env configuration, and documents the limited protection goal.
- Focused tests cover store round-trip, id validation, decode failure, provider secret-ref resolution, WebSearch secret-ref resolution, and no-plaintext debug/serialization paths where applicable.
- `cargo fmt --check`, relevant crate tests/checks, `./tickets.sh doctor`, and `git diff --check` pass.
+248
View File
@@ -0,0 +1,248 @@
<!-- event: create author: tickets.sh at: 2026-05-29T14:53:55Z -->
## Created
Created by tickets.sh create.
---
<!-- event: plan author: hare at: 2026-05-31T20:58:00Z -->
## Plan
Preflight/detailing status: requirements-sync-needed before implementation.
This ticket is the remaining credential/env cleanup boundary. It should not go directly to coding until the secret-store key-management decision is settled. The current code already has some typed-reference groundwork, but no runtime secret store exists.
Current code map:
- `crates/manifest/src/model.rs`
- `AuthRef::SecretRef { ref_ }` already exists in the manifest model and serializes as `auth = { kind = "secret_ref", ref = "..." }`.
- `AuthRef::ApiKey { env, file }` still documents and models env/file API-key sources.
- `SchemeKind::default_env_var()` still provides `INSOMNIA_API_KEY_*` defaults.
- `crates/provider/src/lib.rs`
- `AuthRef::ApiKey` resolves env first, then `auth.file`.
- `AuthRef::SecretRef` currently errors with "secret store references are not implemented yet".
- `crates/tools/src/web.rs`
- Brave WebSearch currently requires `web.search.api_key_env` and reads process env directly.
- `docs/environment.md`
- credential env vars are documented as migration compatibility, not the target supported configuration path.
Proposed target architecture:
- Add a small secret-store boundary independent from `manifest`, `provider`, and `tools` cycles.
- Candidate crate name: `secret-store` or `secrets`.
- Responsibilities: secret id validation, encrypted store read/write, metadata listing, redacted diagnostics, and test-only in-memory/key fixtures.
- Non-responsibilities: provider-specific auth semantics, profile selection, tool networking.
- Keep manifest/profile config as references only.
- Model auth should use the existing `AuthRef::SecretRef { ref_ }` shape rather than adding another model-auth syntax.
- WebSearch needs an explicit secret reference field, e.g. `web.search.api_key_secret = "web/brave/default"` or a typed `web.search.auth = { kind = "secret_ref", ref = "..." }`. Prefer the latter if it avoids another provider-specific one-off, but choose based on minimal config churn.
- Secret IDs should be logical names, not paths.
- Allow a conservative character set such as `[A-Za-z0-9._/-]`.
- Reject empty ids, absolute/path-traversal-like ids, control characters, and extremely long ids.
- IDs are diagnostics-safe; values are not.
- Secret values are resolved only at consumer/runtime boundary.
- Resolved manifests/profiles may contain secret refs, not plaintext.
- Provider/WebSearch consumers receive plaintext only in memory and must not serialize/log it.
Key-management decision required before implementation:
- Do not store an encryption key next to the encrypted store; that provides obfuscation, not meaningful encryption.
- Preferred direction to evaluate: OS keychain/credential manager as the wrapping-key source where available.
- Linux Secret Service / macOS Keychain / Windows Credential Manager through a maintained crate such as `keyring`, if acceptable for dependencies/packaging/headless use.
- If no OS key provider is available, fail closed with clear diagnostics rather than silently writing plaintext or adjacent keys.
- Tests should use an explicit test-only key provider/in-memory store without process-env mutation.
- If a passphrase fallback is desired later, make it explicit interactive/CLI UX; do not read passphrases from ambient env.
Suggested store layout:
- Store encrypted blobs outside the repository, under the user data directory, e.g. `<data_dir>/secrets/store.json` or `<data_dir>/secrets/<id>.json`.
- Use atomic write + fsync where the project already has or can add an atomic-write helper.
- Store metadata needed for listing without plaintext values:
- id
- created_at / updated_at
- optional description/provider/kind
- encryption metadata: algorithm, nonce, key provider/version
- Do not put encrypted blobs under work-items, memory, session logs, project `.insomnia/`, or generated reports.
Encryption requirements:
- Authenticated encryption only (AEAD), e.g. XChaCha20-Poly1305 or AES-GCM depending on dependency choice.
- Unique nonce per encryption.
- Include associated data that binds ciphertext to secret id and store metadata version.
- Decryption/auth failure is a fail-closed error naming the secret id only.
CLI/TUI management scope:
- Initial scope can be CLI-first under `insomnia secrets ...`; TUI management can be follow-up unless UX is trivial.
- Minimum CLI:
- `insomnia secrets set <id> --stdin [--description ...]`
- `insomnia secrets list`
- `insomnia secrets delete <id>`
- optionally `insomnia secrets rename <old> <new>` later, not required for MVP.
- Do not print plaintext by default. Avoid adding `show` unless protected by an explicit `--reveal` decision in a later ticket.
- Non-interactive `set --stdin` is required so scripts can load keys without shell env exports.
Consumer migration plan:
1. Implement secret store + model `AuthRef::SecretRef` resolution in `provider`.
2. Add WebSearch secret reference support and tests.
3. Add CLI management commands.
4. Update docs and examples to use secret refs as the normal path.
5. Convert env-var credential paths into migration diagnostics or compatibility-only one-file/debug behavior, then remove from normal profile path.
Important migration constraints:
- Do not load `.env` files.
- Do not keep env as a normal fallback once secret refs are available.
- If env inputs remain temporarily, diagnostics should say they are migration compatibility and point to `insomnia secrets set ...` / profile secret refs.
- Avoid changing Codex OAuth in this ticket unless a clear secret-store integration is needed; `CODEX_HOME` remains external compatibility.
Acceptance criteria additions:
- `AuthRef::SecretRef` can resolve through the encrypted store for provider API keys without exposing plaintext in resolved config/debug output.
- WebSearch can use a secret ref without `BRAVE_SEARCH_API_KEY` or `web.search.api_key_env`.
- CLI can add/list/delete a secret without printing plaintext.
- Missing secret, invalid id, unavailable key provider, and decryption/auth failure produce clear fail-closed errors naming only the reference.
- Tests cover:
- secret id validation;
- encrypted round-trip with test key provider;
- wrong-key/auth-failure diagnostics;
- provider `AuthRef::SecretRef` resolution;
- WebSearch secret ref resolution;
- no plaintext in `Debug`/serialization paths checked by focused assertions.
Recommended next step:
- Create a short spike/preflight sub-ticket specifically for key provider/dependency choice (`keyring` vs explicit passphrase vs other OS-backed provider). Once that decision is recorded, the implementation ticket can be split into:
1. secret-store crate + CLI management;
2. provider `AuthRef::SecretRef` integration;
3. WebSearch secret ref integration and env migration docs.
---
<!-- event: decision author: hare at: 2026-05-31T21:04:45Z -->
## Decision
Decision: keep the secret store as a provider-independent key-value store.
User decision:
- The secret store path should be completely limited to a simple key-value store.
- Do not hard-code provider-specific slots or provider-specific semantics into the store.
- Do not require metadata in the store schema.
- Conceptual model is:
```text
{
"anthropic/default" = "sk-..."
}
```
- Provider/model/tool configuration is responsible for choosing which secret key to reference.
- The store does not know whether a value is an Anthropic key, Brave key, OpenAI key, token, or anything else.
- Any provider-aware UX should be a higher-level helper that writes ordinary key-value entries and/or config references; it must not change the store schema.
Security stance:
- Use light encryption/obfuscation at rest if practical, but do not claim strong security guarantees.
- The goal is to avoid casual plaintext exposure in files, logs, work items, and accidental grep/cat output, not to defend against a local attacker with access to the user account.
- Avoid complicated key-management requirements such as OS keychain dependency as a prerequisite for this ticket unless a later explicit decision changes the security target.
- Documentation and diagnostics should be honest: this is an obfuscated/encrypted local key-value store with limited protection, not a high-assurance secret manager.
Implications for implementation planning:
- Remove the previous requirement for metadata such as provider/kind/description/created_at/updated_at unless the implementation needs internal versioning/encryption fields.
- Store format may still need technical envelope fields for version/nonce/ciphertext/checksum, but the user-visible logical schema is only `id -> value`.
- Secret id validation remains useful because ids are referenced from manifest/profile/tool config and diagnostics.
- Provider/WebSearch integration should resolve `secret_ref` by direct key lookup only.
---
<!-- event: plan author: hare at: 2026-05-31T21:19:29Z -->
## Plan
Preflight update: implementation-ready as a single phased ticket.
Finalized intent:
- Build a provider-independent local key-value secret store and wire it into provider/WebSearch credential resolution.
- Add `insomnia keys` as the required interactive TUI management surface.
- Keep the security claim modest: avoid casual plaintext exposure and generic env scraping; do not claim strong local security.
Settled decisions:
- Store model is user-visible `id -> value`; no provider-specific slots and no required metadata.
- Store may use technical envelope fields for version/nonce/ciphertext/checksum, but those are implementation details.
- No OS keychain, passphrase UX, or high-assurance key-management dependency in this ticket.
- No automatic provider-name-to-secret-id lookup. Config must explicitly reference a key id.
- This is one ticket with phases, not separate tickets.
Suggested implementation order:
1. Add a focused secret store crate/module with id validation, obfuscated/encrypted persistence, atomic writes, and tests.
2. Add `insomnia keys` TUI manager for list/add/delete with masked input.
3. Implement provider `AuthRef::SecretRef` resolution through the store.
4. Add WebSearch secret-ref configuration path.
5. Update docs and migration messaging.
Critical risks for coder/reviewer:
- Accidentally turning the store into provider-aware schema.
- Displaying plaintext in Debug/errors/TUI list output.
- Overstating security guarantees in docs.
- Adding ambient defaults that recreate the env-var problem in secret-store form.
- Expanding into OS keychain/passphrase design despite the settled modest threat model.
Validation should include focused tests for id validation, store round-trip, decode failure, provider secret-ref resolution, WebSearch secret-ref resolution, and no plaintext in display/debug paths where applicable.
---
<!-- event: decision author: hare at: 2026-05-31T21:23:46Z -->
## Decision
Decision update:
- Migration/backward compatibility for credential environment variables is not required.
- The implementation should remove credential env configuration from the normal provider/WebSearch path rather than keep env fallback as a compatibility layer.
- Docs should point to `insomnia keys` + explicit secret refs as the credential path.
Codex OAuth relationship:
- Codex OAuth should not be connected to this key-value API-key store in this ticket.
- The current Codex integration interoperates with Codex CLI's structured `auth.json` token bundle and refresh behavior.
- That is a different credential shape than `id -> secret string` API keys.
- Keep `CODEX_HOME` / `$HOME/.codex` behavior unchanged for this ticket.
- If Insomnia later owns Codex login/token storage, design it as a separate OAuth token-store feature, not as an implicit use of the simple key-value store.
---
<!-- event: review author: hare at: 2026-05-31T22:21:04Z status: approve -->
## Review: approve
# Review: local key-value secret store implementation
Implementation reviewed on branch `manifest-profile-encrypted-secrets`.
Reviewed commits:
- `cc2c9a2 secrets: add local key store`
- `7ddf745 secrets: polish key manager and docs`
Verdict: approve.
Summary:
- Core provider-independent `id -> value` local secret store satisfies the ticket model.
- Store values are not persisted as casual plaintext and error/debug surfaces avoid secret values within the stated modest protection boundary.
- Provider auth now resolves explicit `secret_ref` values through the local store without env credential fallback.
- WebSearch uses explicit `api_key_secret` and no longer depends on `BRAVE_SEARCH_API_KEY` / `api_key_env` in the normal path.
- `insomnia keys` provides interactive list/add-set/delete management without displaying plaintext values.
- Codex OAuth behavior remains separate and unchanged.
- Follow-up review confirmed the docs credential example is schema-valid and key-manager terminal setup cleanup was added.
Validation reported by coder/reviewer:
- `cargo fmt --check`
- `cargo test -p secrets`
- focused manifest/provider/tools/tui/insomnia tests
- `cargo check -p manifest -p provider -p tools -p tui -p insomnia`
- `./tickets.sh doctor`
- `git diff --check`
- credential/env greps confirming `api_key_env`, `BRAVE_SEARCH_API_KEY`, `INSOMNIA_API_KEY`, and `default_env_var` are absent from crates/docs/resources
Remaining caveat:
- Continue to describe this as local obfuscation / limited at-rest protection, not a high-assurance password manager or OS-keychain-backed vault.
---