Hare/yoi
1
1
Fork 0
Code Actions Packages Releases Activity

test: isolate e2e yoi subprocess env

Browse Source
This commit is contained in:
Keisuke Hirata 2026-06-14 01:02:19 +09:00
parent 13d0053036
commit 47efeb0143
No known key found for this signature in database
4 changed files with 263 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
.yoi/tickets/00001KV0TJVN5/artifacts/implementation-report.md
View File

@ -5,17 +5,29 @@ Files changed:
- Added a cached e2e binary provider using `OnceLock`.
- Preserves `YOI_E2E_BIN=<path>` as the explicit override and skips the default cargo build provider in that path.
- Default path runs `${CARGO:-cargo} build -p yoi --features e2e-test --bin yoi` from the workspace root, then returns the direct `target/{profile}/yoi` binary path for PTY spawning.
- Writes `target/e2e-artifacts/binary-provider.json` and emits diagnostics with provider, build command, and binary path.
- Writes `target/e2e-artifacts/binary-provider.json` and emits diagnostics with provider, build command, binary path, and tested-subprocess env policy.
- Expanded command-failure diagnostics to include command args.
- Follow-up: isolated tested `yoi` subprocess environments in both `PanelHarness::spawn` and fixture setup `run_yoi_capture` with `env_clear()` plus explicit allowlists only.
- Follow-up: recorded env policy in `run.json`, `binary-provider.json`, and per-fixture `fixture-commands.jsonl` artifacts.
- Follow-up: added a regression assertion that tested-subprocess policies use `env_clear`, do not allow `PATH`, and default-deny provider credentials (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`) and secret-like patterns.
- Follow-up: relative `YOI_E2E_BIN` values are resolved against the workspace root and must exist, so tested subprocess launch does not rely on `PATH` lookup.
- `tests/e2e/tests/panel.rs`
- Updated panel tests to use the fallible cached binary provider.
Env isolation policy:
- Cargo build provider remains a build-tool command and is not treated as the tested `yoi` subprocess.
- Tested `yoi` fixture setup commands receive only: `HOME`, `XDG_DATA_HOME`, `XDG_STATE_HOME`, `XDG_CONFIG_HOME`, `YOI_POD_RUNTIME_COMMAND`.
- Tested `yoi panel` commands receive only: fixture `HOME`, `XDG_DATA_HOME`, `XDG_STATE_HOME`, `XDG_CONFIG_HOME`, `TERM`, `YOI_TUI_TEST_EVENTS`, `YOI_POD_RUNTIME_COMMAND`, and `YOI_TUI_TEST_HOLD_BACKGROUND_TASK` when used.
- `PATH` is intentionally not passed to tested `yoi` subprocesses; the harness launches the already-resolved binary path directly.
- Host provider credentials / token / secret-like environment variables are default-denied. Future provider/LLM E2E should use fixture providers, canned servers, or explicit test env instead of inheriting host credentials.
Validation:
- `cargo fmt --check` — passed.
- `git diff --check` — passed.
- `cargo check -p yoi-e2e --all-targets --features e2e` — passed.
- `unset YOI_E2E_BIN && cargo test -p yoi-e2e --features e2e --test panel -- --nocapture` — passed; default provider built the current `yoi` binary and PTY-spawned `target/debug/yoi`.
- `YOI_E2E_BIN=/home/hare/Projects/yoi/.worktree/e2e-binary-provider/target/debug/yoi cargo test -p yoi-e2e --features e2e --test panel -- --nocapture` — passed; override provider path used without invoking the default cargo-build provider.
- `cargo test -p yoi-e2e --features e2e tested_yoi_env_policy_is_env_clear_allowlist -- --nocapture` — passed.
- `unset YOI_E2E_BIN && OPENAI_API_KEY=host-secret ANTHROPIC_API_KEY=host-secret GEMINI_API_KEY=host-secret cargo test -p yoi-e2e --features e2e --test panel -- --nocapture` — passed; default provider built the current `yoi` binary and tested `yoi` subprocesses used isolated env policy artifacts. Host provider env was present for the harness but is not inherited by tested `yoi` subprocesses because `env_clear()` is applied before the allowlist.
- `YOI_E2E_BIN=/home/hare/Projects/yoi/.worktree/e2e-binary-provider/target/debug/yoi OPENAI_API_KEY=host-secret ANTHROPIC_API_KEY=host-secret GEMINI_API_KEY=host-secret cargo test -p yoi-e2e --features e2e --test panel -- --nocapture` — passed; override provider path used without invoking the default cargo-build provider, and tested `yoi` subprocesses still used isolated env policy.
Remaining gaps:
- None known.

2
.yoi/tickets/00001KV0TJVN5/item.md
View File

@ -2,7 +2,7 @@
title: 'E2E harness が最新 yoi binary を自動 build して使うようにする'
state: 'inprogress'
created_at: '2026-06-13T15:46:07Z'
updated_at: '2026-06-13T15:54:18Z'
updated_at: '2026-06-13T16:01:51Z'
assignee: null
readiness: 'ready'
queued_by: 'yoi ticket'

41
.yoi/tickets/00001KV0TJVN5/thread.md
View File

@ -112,4 +112,45 @@ Remaining gaps:
- None known.
---
<!-- event: implementation_report author: hare at: 2026-06-13T16:01:51Z -->
## Implementation report
Implementation report for Ticket 00001KV0TJVN5
Files changed:
- `tests/e2e/src/lib.rs`
- Added a cached e2e binary provider using `OnceLock`.
- Preserves `YOI_E2E_BIN=<path>` as the explicit override and skips the default cargo build provider in that path.
- Default path runs `${CARGO:-cargo} build -p yoi --features e2e-test --bin yoi` from the workspace root, then returns the direct `target/{profile}/yoi` binary path for PTY spawning.
- Writes `target/e2e-artifacts/binary-provider.json` and emits diagnostics with provider, build command, binary path, and tested-subprocess env policy.
- Expanded command-failure diagnostics to include command args.
- Follow-up: isolated tested `yoi` subprocess environments in both `PanelHarness::spawn` and fixture setup `run_yoi_capture` with `env_clear()` plus explicit allowlists only.
- Follow-up: recorded env policy in `run.json`, `binary-provider.json`, and per-fixture `fixture-commands.jsonl` artifacts.
- Follow-up: added a regression assertion that tested-subprocess policies use `env_clear`, do not allow `PATH`, and default-deny provider credentials (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`) and secret-like patterns.
- Follow-up: relative `YOI_E2E_BIN` values are resolved against the workspace root and must exist, so tested subprocess launch does not rely on `PATH` lookup.
- `tests/e2e/tests/panel.rs`
- Updated panel tests to use the fallible cached binary provider.
Env isolation policy:
- Cargo build provider remains a build-tool command and is not treated as the tested `yoi` subprocess.
- Tested `yoi` fixture setup commands receive only: `HOME`, `XDG_DATA_HOME`, `XDG_STATE_HOME`, `XDG_CONFIG_HOME`, `YOI_POD_RUNTIME_COMMAND`.
- Tested `yoi panel` commands receive only: fixture `HOME`, `XDG_DATA_HOME`, `XDG_STATE_HOME`, `XDG_CONFIG_HOME`, `TERM`, `YOI_TUI_TEST_EVENTS`, `YOI_POD_RUNTIME_COMMAND`, and `YOI_TUI_TEST_HOLD_BACKGROUND_TASK` when used.
- `PATH` is intentionally not passed to tested `yoi` subprocesses; the harness launches the already-resolved binary path directly.
- Host provider credentials / token / secret-like environment variables are default-denied. Future provider/LLM E2E should use fixture providers, canned servers, or explicit test env instead of inheriting host credentials.
Validation:
- `cargo fmt --check` — passed.
- `git diff --check` — passed.
- `cargo check -p yoi-e2e --all-targets --features e2e` — passed.
- `cargo test -p yoi-e2e --features e2e tested_yoi_env_policy_is_env_clear_allowlist -- --nocapture` — passed.
- `unset YOI_E2E_BIN && OPENAI_API_KEY=host-secret ANTHROPIC_API_KEY=host-secret GEMINI_API_KEY=host-secret cargo test -p yoi-e2e --features e2e --test panel -- --nocapture` — passed; default provider built the current `yoi` binary and tested `yoi` subprocesses used isolated env policy artifacts. Host provider env was present for the harness but is not inherited by tested `yoi` subprocesses because `env_clear()` is applied before the allowlist.
- `YOI_E2E_BIN=/home/hare/Projects/yoi/.worktree/e2e-binary-provider/target/debug/yoi OPENAI_API_KEY=host-secret ANTHROPIC_API_KEY=host-secret GEMINI_API_KEY=host-secret cargo test -p yoi-e2e --features e2e --test panel -- --nocapture` — passed; override provider path used without invoking the default cargo-build provider, and tested `yoi` subprocesses still used isolated env policy.
Remaining gaps:
- None known.
---

211
tests/e2e/src/lib.rs
View File

@ -32,6 +32,23 @@ pub struct BinaryProviderInfo {
pub build_args: Vec<String>,
pub build_command: Option<String>,
pub profile: String,
pub tested_yoi_subprocess_env: TestedYoiEnvPolicy,
}
#[derive(Clone, Debug, Serialize)]
pub struct EnvPolicy {
pub env_clear: bool,
pub allowlist: Vec<String>,
pub path_allowed: bool,
pub provider_credentials_default_deny: Vec<String>,
pub secret_patterns_default_deny: Vec<String>,
pub note: String,
}
#[derive(Clone, Debug, Serialize)]
pub struct TestedYoiEnvPolicy {
pub fixture_setup: EnvPolicy,
pub panel: EnvPolicy,
}
impl BinaryProviderInfo {
@ -52,6 +69,65 @@ impl BinaryProviderInfo {
}
}
fn env_policy(allowlist: &[&str], note: &str) -> EnvPolicy {
EnvPolicy {
env_clear: true,
allowlist: allowlist.iter().map(|name| (*name).to_owned()).collect(),
path_allowed: false,
provider_credentials_default_deny: vec![
"OPENAI_API_KEY".to_owned(),
"ANTHROPIC_API_KEY".to_owned(),
"GEMINI_API_KEY".to_owned(),
],
secret_patterns_default_deny: vec![
"*_API_KEY".to_owned(),
"*_TOKEN".to_owned(),
"*_SECRET".to_owned(),
"*_CREDENTIAL*".to_owned(),
],
note: note.to_owned(),
}
}
fn fixture_setup_env_policy() -> EnvPolicy {
env_policy(
&[
"HOME",
"XDG_DATA_HOME",
"XDG_STATE_HOME",
"XDG_CONFIG_HOME",
"YOI_POD_RUNTIME_COMMAND",
],
"tested yoi fixture setup commands use env_clear and receive only fixture data/config homes plus the explicit runtime binary override",
)
}
fn panel_env_policy(include_hold_background_task: bool) -> EnvPolicy {
let mut allowlist = vec![
"HOME",
"XDG_DATA_HOME",
"XDG_STATE_HOME",
"XDG_CONFIG_HOME",
"TERM",
"YOI_TUI_TEST_EVENTS",
"YOI_POD_RUNTIME_COMMAND",
];
if include_hold_background_task {
allowlist.push("YOI_TUI_TEST_HOLD_BACKGROUND_TASK");
}
env_policy(
&allowlist,
"tested yoi panel subprocess uses env_clear and receives only fixture homes, terminal/test-observer variables, and the explicit runtime binary override",
)
}
fn tested_yoi_env_policy_overview() -> TestedYoiEnvPolicy {
TestedYoiEnvPolicy {
fixture_setup: fixture_setup_env_policy(),
panel: panel_env_policy(true),
}
}
#[derive(Debug)]
pub enum HarnessError {
Io(io::Error),
@ -218,6 +294,7 @@ impl PanelHarness {
fs::write(&artifacts.events_jsonl, "")?;
fs::write(&artifacts.input_log, "")?;
fs::write(&artifacts.output_log, "")?;
let env_policy = panel_env_policy(config.hold_background_task.is_some());
fs::write(
&artifacts.run_json,
serde_json::to_vec_pretty(&serde_json::json!({
@ -232,6 +309,7 @@ impl PanelHarness {
"rows": config.terminal_size.1,
},
"hold_background_task": config.hold_background_task,
"tested_yoi_env_policy": &env_policy,
}))?,
)?;
@ -244,6 +322,7 @@ impl PanelHarness {
.arg("panel")
.arg("--workspace")
.arg(&config.workspace)
.env_clear()
.env("YOI_TUI_TEST_EVENTS", &artifacts.events_jsonl)
.env("YOI_POD_RUNTIME_COMMAND", &config.binary)
.env("HOME", &config.home)
@ -641,14 +720,25 @@ pub fn yoi_binary_info() -> Result<BinaryProviderInfo> {
fn resolve_yoi_binary() -> Result<BinaryProviderInfo> {
if let Some(path) = std::env::var_os("YOI_E2E_BIN") {
let workspace_root = workspace_root()?;
let binary = PathBuf::from(path);
let binary = if binary.is_absolute() {
binary
} else {
workspace_root.join(binary)
};
if !binary.exists() {
return Err(HarnessError::MissingBinary(binary));
}
let info = BinaryProviderInfo {
provider: "YOI_E2E_BIN".to_owned(),
binary: PathBuf::from(path),
workspace_root: workspace_root()?,
binary,
workspace_root,
cargo: None,
build_args: Vec::new(),
build_command: None,
profile: test_profile(),
tested_yoi_subprocess_env: tested_yoi_env_policy_overview(),
};
info.log();
write_binary_provider_artifact(&info)?;
@ -695,6 +785,7 @@ fn resolve_yoi_binary() -> Result<BinaryProviderInfo> {
build_args: args,
build_command: Some(command),
profile: test_profile(),
tested_yoi_subprocess_env: tested_yoi_env_policy_overview(),
};
info.log();
write_binary_provider_artifact(&info)?;
@ -832,15 +923,21 @@ fn run_yoi_capture(
config: &Path,
args: &[&str],
) -> Result<String> {
let output = Command::new(binary)
let env_policy = fixture_setup_env_policy();
append_fixture_command_artifact(workspace, binary, args, &env_policy)?;
let mut command = Command::new(binary);
command
.args(args)
.current_dir(workspace)
.env_clear()
.env("HOME", home)
.env("XDG_DATA_HOME", data)
.env("XDG_STATE_HOME", state)
.env("XDG_CONFIG_HOME", config)
.env("YOI_POD_RUNTIME_COMMAND", binary)
.output()?;
.env("YOI_POD_RUNTIME_COMMAND", binary);
let output = command.output()?;
if !output.status.success() {
return Err(HarnessError::CommandFailed {
program: binary.to_path_buf(),
@ -855,6 +952,37 @@ fn run_yoi_capture(
Ok(text)
}
fn append_fixture_command_artifact(
workspace: &Path,
binary: &Path,
args: &[&str],
env_policy: &EnvPolicy,
) -> Result<()> {
let fixture_root = workspace.parent().ok_or_else(|| {
HarnessError::Protocol(format!(
"fixture workspace {} has no parent for artifacts",
workspace.display()
))
})?;
let artifacts_dir = fixture_root.join("artifacts");
fs::create_dir_all(&artifacts_dir)?;
let mut file = OpenOptions::new()
.append(true)
.create(true)
.open(artifacts_dir.join("fixture-commands.jsonl"))?;
serde_json::to_writer(
&mut file,
&serde_json::json!({
"ts_ms": now_ms(),
"binary": binary,
"args": args,
"tested_yoi_env_policy": env_policy,
}),
)?;
writeln!(file)?;
Ok(())
}
fn write_blocking_pod_metadata(data_home: &Path, pod_name: &str) -> Result<()> {
let dir = data_home.join("yoi").join("pods").join(pod_name);
fs::create_dir_all(&dir)?;
@ -907,3 +1035,76 @@ fn now_ms() -> u128 {
.map(|duration| duration.as_millis())
.unwrap_or_default()
}
#[cfg(test)]
mod tests {
use super::*;
fn assert_host_credentials_default_denied(policy: &EnvPolicy) {
assert!(
policy.env_clear,
"tested yoi subprocesses must use env_clear"
);
assert!(
!policy.path_allowed,
"tested yoi subprocesses should not inherit or allow PATH"
);
assert!(
!policy.allowlist.iter().any(|name| name == "PATH"),
"PATH must not be allowlisted for tested yoi subprocesses"
);
for name in ["OPENAI_API_KEY", "ANTHROPIC_API_KEY", "GEMINI_API_KEY"] {
assert!(
!policy.allowlist.iter().any(|allowed| allowed == name),
"{name} must not be allowlisted for tested yoi subprocesses"
);
assert!(
policy
.provider_credentials_default_deny
.iter()
.any(|denied| denied == name),
"{name} should be recorded as provider credential default-deny"
);
}
}
#[test]
fn tested_yoi_env_policy_is_env_clear_allowlist() {
let fixture = fixture_setup_env_policy();
assert_host_credentials_default_denied(&fixture);
assert_eq!(
fixture.allowlist,
[
"HOME",
"XDG_DATA_HOME",
"XDG_STATE_HOME",
"XDG_CONFIG_HOME",
"YOI_POD_RUNTIME_COMMAND",
]
);
let panel = panel_env_policy(false);
assert_host_credentials_default_denied(&panel);
assert_eq!(
panel.allowlist,
[
"HOME",
"XDG_DATA_HOME",
"XDG_STATE_HOME",
"XDG_CONFIG_HOME",
"TERM",
"YOI_TUI_TEST_EVENTS",
"YOI_POD_RUNTIME_COMMAND",
]
);
let panel_with_hold = panel_env_policy(true);
assert_host_credentials_default_denied(&panel_with_hold);
assert!(
panel_with_hold
.allowlist
.iter()
.any(|name| name == "YOI_TUI_TEST_HOLD_BACKGROUND_TASK")
);
}
}