feat: live reload runtime connections

This commit is contained in:
Keisuke Hirata 2026-07-04 01:53:51 +09:00
parent 296da3a440
commit b3dac8ea68
No known key found for this signature in database
19 changed files with 1040 additions and 336 deletions

View File

@ -1,9 +1,11 @@
--- ---
title: 'Live-reload Runtime connection registry changes' title: 'Live-reload Runtime connection registry changes'
state: 'planning' state: 'done'
created_at: '2026-07-03T15:59:48Z' created_at: '2026-07-03T15:59:48Z'
updated_at: '2026-07-03T15:59:48Z' updated_at: '2026-07-03T19:47:23Z'
assignee: null assignee: null
queued_by: 'yoi ticket'
queued_at: '2026-07-03T16:52:35Z'
--- ---
## 背景 ## 背景

View File

@ -4,4 +4,237 @@
LocalTicketBackend によって作成されました。 LocalTicketBackend によって作成されました。
---
<!-- event: intake_summary author: hare at: 2026-07-03T16:52:26Z -->
## Intake summary
Marked ready by `yoi ticket state`.
---
<!-- event: state_changed author: "yoi ticket" at: 2026-07-03T16:52:26Z from: planning to: ready reason: cli_state field: state -->
## State changed
Marked ready by `yoi ticket state`.
---
<!-- event: state_changed author: "yoi ticket" at: 2026-07-03T16:52:35Z from: ready to: queued reason: queued field: state -->
## State changed
Ticket を `yoi ticket` が queued にしました。
---
<!-- event: state_changed author: "yoi ticket" at: 2026-07-03T16:52:35Z from: queued to: inprogress reason: cli_state field: state -->
## State changed
State changed to `inprogress`.
---
<!-- event: implementation_report author: hare at: 2026-07-03T16:53:43Z -->
## Implementation report
実装報告。
- `RuntimeRegistry` を live mutation 可能な共有 registry にし、Settings の remote Runtime add/delete から `register_or_replace` / `unregister_if_idle` を呼ぶようにした。
- add は persisted config 書き込み前に remote runtime config を解決・構築し、成功後に active registry へ反映する。成功時は `restart_required = false``runtime_registry_applied` diagnostic を返す。
- delete は active registry 側に worker が残っている場合 `remote_runtime_delete_blocked` で拒否し、config は削除しない。worker が無ければ config と active registry の両方から削除する。
- Settings Test 表示は connected / verified / unchecked warning を分け、接続できた範囲が分かるようにした。
- `worker-runtime-rest-server` の default bind port を `127.0.0.1:38800` に固定し、default config コメントも同じ port に合わせた。
- remote runtime blocking HTTP client/request は async context 直下で Tokio runtime を drop しない boundary 経由にした。
Validation:
- `cargo fmt --check`
- `git diff --check`
- `cargo test -p yoi-workspace-server`
- `cargo test -p worker-runtime --features ws-server`
- `cd web/workspace && deno task check && deno task test`
Skipped:
- `nix build .#yoi --no-link` は重いため、ユーザー指示により未実行。
---
<!-- event: state_changed author: "yoi ticket" at: 2026-07-03T16:53:43Z from: inprogress to: done reason: cli_state field: state -->
## State changed
State changed to `done`.
---
<!-- event: implementation_report author: hare at: 2026-07-03T17:41:08Z -->
## Implementation report
Follow-up after manual Browser test.
Observed failure:
- Browser could connect/test `arcadia-test`, but worker creation failed with `workspace_worker_create_failed`: Runtime did not accept worker creation.
Root cause:
- The standalone `worker-runtime-rest-server` started from `worker-runtime` has no `WorkerExecutionBackend`; it can serve control/catalog endpoints but cannot spawn Workers.
- The connection test previously reported spawn as unknown, while runtime launch candidates projected remote worker creation as available.
Fix:
- Runtime summary now exposes `worker_creation_available` based on whether an execution backend is attached.
- Remote Runtime capability projection uses that flag for `can_spawn_worker` / `can_accept_input`.
- Settings connection test now marks `workers.spawn` incompatible with `remote_runtime_worker_creation_unavailable` when the Runtime has no execution backend.
- Added backend test covering a control-only Runtime with no execution backend.
Validation:
- `cargo fmt --check`
- `git diff --check`
- `cargo test -p worker-runtime --features ws-server`
- `cargo test -p yoi-workspace-server`
- `cd web/workspace && deno task check && deno task test`
Skipped:
- `nix build .#yoi --no-link` per earlier user instruction to skip heavy build.
---
<!-- event: implementation_report author: hare at: 2026-07-03T17:53:10Z -->
## Implementation report
Follow-up after `can_spawn_worker=false` review.
User feedback:
- A registered Runtime that cannot spawn workers should not exist as a usable Runtime.
Fix:
- Removed the control-only `worker-runtime` package binary surface by disabling Cargo autobin discovery and deleting `crates/worker-runtime/src/main.rs`.
- Added `worker-runtime-rest-server` under the `worker` package, where it can attach `WorkerRuntimeExecutionBackend` to the Runtime before serving REST/WebSocket endpoints.
- The server binary now starts a worker-backed Runtime, so `/v1/runtime` reports `worker_creation_available=true` for the intended standalone process.
- Retained fail-closed capability projection for old/control-only Runtime summaries so a process without execution backend is rejected as incompatible rather than exposed as a usable candidate.
New local command:
- `cargo run -p worker --features runtime-adapter --bin worker-runtime-rest-server -- --workspace .`
Validation:
- `cargo fmt --check`
- `git diff --check`
- `cargo check -p yoi`
- `cargo test -p worker-runtime --features ws-server`
- `cargo test -p worker --features runtime-adapter --bin worker-runtime-rest-server`
- `cargo test -p yoi-workspace-server`
- `cd web/workspace && deno task check && deno task test`
- Smoke: started worker-backed Runtime on `127.0.0.1:38801` and verified `/v1/runtime` contains `"worker_creation_available":true`.
Skipped:
- `nix build .#yoi --no-link` per earlier user instruction to skip heavy build.
---
<!-- event: implementation_report author: hare at: 2026-07-03T18:33:15Z -->
## Implementation report
Follow-up after dependency-boundary review.
User feedback:
- Putting the standalone Runtime server binary in `worker` was also wrong; `worker` should not depend on Runtime just to host the adapter.
Correction:
- Split the Runtime/Worker bridge into a dedicated `worker-runtime-adapter` crate.
- Moved `WorkerRuntimeExecutionBackend` / `ProfileRuntimeWorkerFactory` out of `worker` and into `worker-runtime-adapter`.
- Moved the `worker-runtime-rest-server` binary into `worker-runtime-adapter`.
- Removed the `worker` -> `worker-runtime` dependency and removed the `runtime-adapter` feature from `worker`.
- `worker-runtime` remains the Runtime domain/API crate; `worker` remains Worker implementation; `worker-runtime-adapter` is the only crate depending on both.
New local command:
- `cargo run -p worker-runtime-adapter --bin worker-runtime-rest-server -- --workspace .`
Validation:
- `cargo fmt --check`
- `git diff --check`
- `cargo check -p worker`
- `cargo check -p yoi`
- `cargo test -p worker-runtime --features ws-server`
- `cargo test -p worker-runtime-adapter`
- `cargo test -p yoi-workspace-server`
- `cd web/workspace && deno task check && deno task test`
Skipped:
- `nix build .#yoi --no-link` per earlier user instruction to skip heavy build.
---
<!-- event: implementation_report author: hare at: 2026-07-03T19:01:34Z -->
## Implementation report
Correction after dependency-boundary review.
User feedback:
- A separate bridge crate is unnecessary; the Runtime should bundle/manage Workers directly.
Correction:
- Removed `worker-runtime-adapter` crate entirely.
- Moved `WorkerRuntimeExecutionBackend` / `ProfileRuntimeWorkerFactory` into `worker-runtime::worker_backend`.
- Moved the `worker-runtime-rest-server` binary back into `worker-runtime`, but kept it worker-backed rather than the old control-only process.
- `worker` no longer depends on `worker-runtime`; `worker-runtime` now depends on `worker` and owns the Runtime process that bundles Workers.
- `workspace-server` now imports `worker_runtime::worker_backend::WorkerRuntimeExecutionBackend`.
Correct local command:
- `cargo run -p worker-runtime --features ws-server,fs-store --bin worker-runtime-rest-server -- --workspace .`
Validation:
- `cargo fmt --check`
- `git diff --check`
- `cargo check -p worker`
- `cargo check -p yoi`
- `cargo check -p worker-runtime --features ws-server,fs-store`
- `cargo test -p worker-runtime --features ws-server,fs-store`
- `cargo test -p yoi-workspace-server`
- `cd web/workspace && deno task check && deno task test`
Skipped:
- `nix build .#yoi --no-link` per earlier user instruction to skip heavy build.
---
<!-- event: implementation_report author: hare at: 2026-07-03T19:47:23Z -->
## Implementation report
Follow-up after Runtime generation path review.
User feedback:
- Runtime generation paths should not be duplicated; a Runtime that can be served over HTTP must be constructed explicitly with the intended Worker backend rather than implicitly from HTTP server config.
Fix:
- Removed `RuntimeHttpServer::bind`.
- Removed `serve_configured_runtime_http`.
- Removed the internal `runtime_from_config` path that built backend-less Runtime instances from `RuntimeHttpServerConfig`.
- `http_server` now serves only an already-constructed `Runtime` via `serve_runtime_http(runtime, listener, token)`.
Validation:
- `cargo fmt --check`
- `git diff --check`
- `cargo test -p worker-runtime --features ws-server,fs-store`
- `cargo test -p yoi-workspace-server`
Skipped:
- `nix build .#yoi --no-link` per earlier user instruction to skip heavy build.
--- ---

7
Cargo.lock generated
View File

@ -5904,7 +5904,6 @@ dependencies = [
"uuid", "uuid",
"wasmtime", "wasmtime",
"wat", "wat",
"worker-runtime",
"workflow", "workflow",
"yoi-plugin-pdk", "yoi-plugin-pdk",
] ]
@ -5913,16 +5912,22 @@ dependencies = [
name = "worker-runtime" name = "worker-runtime"
version = "0.1.0" version = "0.1.0"
dependencies = [ dependencies = [
"async-trait",
"axum", "axum",
"futures", "futures",
"llm-engine",
"manifest",
"protocol", "protocol",
"serde", "serde",
"serde_json", "serde_json",
"session-store",
"sha2 0.11.0", "sha2 0.11.0",
"tempfile",
"thiserror 2.0.18", "thiserror 2.0.18",
"tokio", "tokio",
"tokio-tungstenite 0.29.0", "tokio-tungstenite 0.29.0",
"tower", "tower",
"worker",
] ]
[[package]] [[package]]

View File

@ -4,29 +4,37 @@ description = "Embedded memory-backed Runtime API for Worker management"
version = "0.1.0" version = "0.1.0"
edition.workspace = true edition.workspace = true
license.workspace = true license.workspace = true
autobins = false
[[bin]] [[bin]]
name = "worker-runtime-rest-server" name = "worker-runtime-rest-server"
path = "src/main.rs" path = "src/main.rs"
required-features = ["http-server"] required-features = ["ws-server", "fs-store"]
[features] [features]
default = [] default = []
fs-store = ["dep:serde_json"] fs-store = ["dep:serde_json"]
http-server = ["dep:axum", "dep:serde_json", "dep:tokio", "dep:tower"] http-server = ["dep:axum", "dep:serde_json", "dep:tower"]
ws-server = ["http-server", "axum/ws", "dep:futures", "dep:protocol", "tokio/sync"] ws-server = ["http-server", "axum/ws", "dep:futures", "tokio/sync"]
[dependencies] [dependencies]
async-trait.workspace = true
axum = { workspace = true, optional = true } axum = { workspace = true, optional = true }
futures = { workspace = true, optional = true } futures = { workspace = true, optional = true }
protocol = { workspace = true, optional = true } manifest.workspace = true
protocol.workspace = true
serde = { workspace = true, features = ["derive"] } serde = { workspace = true, features = ["derive"] }
session-store.workspace = true
sha2.workspace = true sha2.workspace = true
serde_json = { workspace = true, optional = true } serde_json = { workspace = true, optional = true }
thiserror = { workspace = true } thiserror = { workspace = true }
tokio = { workspace = true, features = ["net", "rt"], optional = true } tokio = { workspace = true, features = ["net", "rt", "sync", "time"] }
tower = { workspace = true, features = ["util"], optional = true } tower = { workspace = true, features = ["util"], optional = true }
worker.workspace = true
[dev-dependencies] [dev-dependencies]
futures.workspace = true
llm-engine.workspace = true
tempfile.workspace = true
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] } tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
tokio-tungstenite.workspace = true tokio-tungstenite.workspace = true

View File

@ -12,11 +12,9 @@ use crate::catalog::{
}; };
use crate::config_bundle::{ConfigBundle, ConfigBundleAvailability, ConfigBundleSummary}; use crate::config_bundle::{ConfigBundle, ConfigBundleAvailability, ConfigBundleSummary};
use crate::error::RuntimeError; use crate::error::RuntimeError;
#[cfg(feature = "fs-store")]
use crate::fs_store::FsRuntimeStoreOptions;
use crate::identity::{RuntimeId, WorkerId, WorkerRef}; use crate::identity::{RuntimeId, WorkerId, WorkerRef};
use crate::interaction::{WorkerInput, WorkerInteractionAck}; use crate::interaction::{WorkerInput, WorkerInteractionAck};
use crate::management::{RuntimeLimits, RuntimeOptions, RuntimeSummary}; use crate::management::{RuntimeLimits, RuntimeSummary};
#[cfg(feature = "ws-server")] #[cfg(feature = "ws-server")]
use crate::observation::WorkerObservationCursor; use crate::observation::WorkerObservationCursor;
use crate::observation::{TranscriptProjection, TranscriptQuery}; use crate::observation::{TranscriptProjection, TranscriptQuery};
@ -40,6 +38,12 @@ use std::path::PathBuf;
use std::sync::Arc; use std::sync::Arc;
use tokio::net::TcpListener; use tokio::net::TcpListener;
const DEFAULT_RUNTIME_HTTP_PORT: u16 = 38800;
fn default_runtime_http_bind_addr() -> SocketAddr {
SocketAddr::from(([127, 0, 0, 1], DEFAULT_RUNTIME_HTTP_PORT))
}
/// v0 Runtime REST server configuration. /// v0 Runtime REST server configuration.
#[derive(Clone, PartialEq, Eq)] #[derive(Clone, PartialEq, Eq)]
pub struct RuntimeHttpServerConfig { pub struct RuntimeHttpServerConfig {
@ -63,7 +67,7 @@ pub struct RuntimeHttpServerConfig {
impl Default for RuntimeHttpServerConfig { impl Default for RuntimeHttpServerConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
bind_addr: SocketAddr::from(([127, 0, 0, 1], 0)), bind_addr: default_runtime_http_bind_addr(),
runtime_id: None, runtime_id: None,
display_name: None, display_name: None,
limits: RuntimeLimits::default(), limits: RuntimeLimits::default(),
@ -102,48 +106,6 @@ pub enum RuntimeHttpStoreSelection {
}, },
} }
/// Bound REST server instance.
pub struct RuntimeHttpServer {
runtime: Runtime,
local_token: Option<String>,
listener: TcpListener,
}
impl RuntimeHttpServer {
/// Build a Runtime from config and bind the configured address.
pub async fn bind(config: RuntimeHttpServerConfig) -> Result<Self, RuntimeHttpServerError> {
let runtime = runtime_from_config(&config)?;
let listener = TcpListener::bind(config.bind_addr).await?;
Ok(Self {
runtime,
local_token: config.local_token,
listener,
})
}
/// Address actually bound by the server.
pub fn local_addr(&self) -> Result<SocketAddr, RuntimeHttpServerError> {
Ok(self.listener.local_addr()?)
}
/// Runtime owned by this server.
pub fn runtime(&self) -> Runtime {
self.runtime.clone()
}
/// Serve requests until the axum server is stopped or returns an error.
pub async fn serve(self) -> Result<(), RuntimeHttpServerError> {
serve_runtime_http(self.runtime, self.listener, self.local_token).await
}
}
/// Convenience entry point: bind and serve a configured Runtime REST process API.
pub async fn serve_configured_runtime_http(
config: RuntimeHttpServerConfig,
) -> Result<(), RuntimeHttpServerError> {
RuntimeHttpServer::bind(config).await?.serve().await
}
/// Serve an existing Runtime on a pre-bound listener. /// Serve an existing Runtime on a pre-bound listener.
pub async fn serve_runtime_http( pub async fn serve_runtime_http(
runtime: Runtime, runtime: Runtime,
@ -194,27 +156,6 @@ pub fn runtime_http_router(runtime: Runtime, local_token: Option<String>) -> Rou
.layer(middleware::from_fn_with_state(state, require_local_token)) .layer(middleware::from_fn_with_state(state, require_local_token))
} }
fn runtime_from_config(
config: &RuntimeHttpServerConfig,
) -> Result<Runtime, RuntimeHttpServerError> {
match &config.store {
RuntimeHttpStoreSelection::Memory => Ok(Runtime::with_options(RuntimeOptions {
runtime_id: config.runtime_id.clone(),
display_name: config.display_name.clone(),
limits: config.limits.clone(),
})),
#[cfg(feature = "fs-store")]
RuntimeHttpStoreSelection::Fs { root } => {
Ok(Runtime::with_fs_store(FsRuntimeStoreOptions {
root: root.clone(),
runtime_id: config.runtime_id.clone(),
display_name: config.display_name.clone(),
limits: config.limits.clone(),
})?)
}
}
}
#[derive(Clone)] #[derive(Clone)]
struct RuntimeHttpState { struct RuntimeHttpState {
runtime: Runtime, runtime: Runtime,
@ -1134,7 +1075,7 @@ mod ws_tests {
WorkerExecutionResult, WorkerExecutionRunState, WorkerExecutionSpawnRequest, WorkerExecutionResult, WorkerExecutionRunState, WorkerExecutionSpawnRequest,
WorkerExecutionSpawnResult, WorkerExecutionSpawnResult,
}; };
use crate::interaction::WorkerInput; use crate::management::RuntimeOptions;
use futures::{SinkExt, StreamExt}; use futures::{SinkExt, StreamExt};
use std::sync::Arc; use std::sync::Arc;
use tokio_tungstenite::connect_async; use tokio_tungstenite::connect_async;

View File

@ -1,11 +1,10 @@
//! Embedded Runtime domain API for Worker management. //! Embedded Runtime domain API for Worker management.
//! //!
//! `worker-runtime` keeps its core independent from HTTP/WebSocket servers, //! `worker-runtime` owns the Runtime authority surface and, for the standalone
//! provider execution, and the existing Worker host. Filesystem persistence is //! process, wires that Runtime to the real Worker host. Filesystem persistence is
//! available only through the optional `fs-store` feature, and the minimal REST //! available only through the optional `fs-store` feature, and the REST/WebSocket
//! process adapter is available only through the optional `http-server` feature. //! server is available only through the optional `http-server` / `ws-server`
//! The crate defines the in-process Runtime authority surface that higher layers //! features.
//! can later adapt into registries or backend APIs.
pub mod catalog; pub mod catalog;
pub mod config_bundle; pub mod config_bundle;
@ -21,6 +20,7 @@ pub mod interaction;
pub mod management; pub mod management;
pub mod observation; pub mod observation;
mod runtime; mod runtime;
pub mod worker_backend;
#[cfg(feature = "fs-store")] #[cfg(feature = "fs-store")]
pub use fs_store::{FsRuntimeStore, FsRuntimeStoreOptions}; pub use fs_store::{FsRuntimeStore, FsRuntimeStoreOptions};

View File

@ -1,8 +1,8 @@
//! Minimal Runtime REST process wrapper. //! Worker-backed Runtime REST process wrapper.
//! //!
//! This binary is available only when the `http-server` feature is enabled. It //! This binary starts a Runtime command API with a real worker execution backend.
//! starts a Runtime-local command API intended for a trusted backend/proxy; //! A REST Runtime process that cannot spawn Workers is not a valid Runtime for the
//! browsers must not connect to this Runtime process directly. //! Workspace Browser.
use std::collections::VecDeque; use std::collections::VecDeque;
use std::env; use std::env;
@ -11,11 +11,16 @@ use std::fmt;
use std::net::SocketAddr; use std::net::SocketAddr;
use std::path::PathBuf; use std::path::PathBuf;
use std::process::ExitCode; use std::process::ExitCode;
use std::sync::Arc;
use worker_runtime::error::RuntimeError;
use worker_runtime::fs_store::FsRuntimeStoreOptions;
use worker_runtime::http_server::{ use worker_runtime::http_server::{
RuntimeHttpServer, RuntimeHttpServerConfig, RuntimeHttpServerError, RuntimeHttpStoreSelection, RuntimeHttpServerConfig, RuntimeHttpServerError, RuntimeHttpStoreSelection,
}; };
use worker_runtime::identity::RuntimeId; use worker_runtime::identity::RuntimeId;
use worker_runtime::worker_backend::{ProfileRuntimeWorkerFactory, WorkerRuntimeExecutionBackend};
use worker_runtime::{Runtime, RuntimeOptions};
fn main() -> ExitCode { fn main() -> ExitCode {
match run() { match run() {
@ -40,25 +45,77 @@ fn run() -> Result<(), ProcessError> {
}; };
let runtime = tokio::runtime::Builder::new_current_thread() let runtime = tokio::runtime::Builder::new_current_thread()
.enable_io() .enable_all()
.build()?; .build()?;
runtime.block_on(async move { runtime.block_on(async move {
let server = RuntimeHttpServer::bind(config).await?; let listener = tokio::net::TcpListener::bind(config.http.bind_addr).await?;
let local_addr = server.local_addr()?; let local_addr = listener.local_addr()?;
let worker_runtime = build_runtime(&config)?;
eprintln!( eprintln!(
"worker-runtime REST server listening on {local_addr}; intended client is a trusted backend/proxy, not a browser" "worker-runtime REST server listening on {local_addr}; intended client is a trusted backend/proxy, not a browser"
); );
server.serve().await worker_runtime::http_server::serve_runtime_http(
worker_runtime,
listener,
config.http.local_token,
)
.await
.map_err(ProcessError::from)
})?; })?;
Ok(()) Ok(())
} }
fn parse_args<I, S>(args: I) -> Result<Option<RuntimeHttpServerConfig>, ProcessError> fn build_runtime(config: &ProcessConfig) -> Result<Runtime, ProcessError> {
let mut factory = ProfileRuntimeWorkerFactory::new(config.workspace_root.clone())
.with_cwd(config.cwd.clone());
if let Some(store_dir) = config.worker_store_dir.clone() {
factory = factory.with_store_dir(store_dir);
}
if let Some(metadata_dir) = config.worker_metadata_dir.clone() {
factory = factory.with_worker_metadata_dir(metadata_dir);
}
if let Some(runtime_base_dir) = config.worker_runtime_base_dir.clone() {
factory = factory.with_runtime_base_dir(runtime_base_dir);
}
if let Some(profile) = config.profile.clone() {
factory = factory.with_profile(profile);
}
let backend =
Arc::new(WorkerRuntimeExecutionBackend::new(factory).map_err(ProcessError::WorkerAdapter)?);
match &config.http.store {
RuntimeHttpStoreSelection::Memory => {
Runtime::with_execution_backend(runtime_options_from_http(&config.http), backend)
.map_err(ProcessError::Runtime)
}
RuntimeHttpStoreSelection::Fs { root } => {
let mut options = FsRuntimeStoreOptions::new(root.clone());
options.runtime_id = config.http.runtime_id.clone();
options.display_name = config.http.display_name.clone();
options.limits = config.http.limits.clone();
Runtime::with_fs_store_and_execution_backend(options, backend)
.map_err(ProcessError::Runtime)
}
_ => Err(ProcessError::usage(
"unsupported Runtime catalog store selection".to_string(),
)),
}
}
fn runtime_options_from_http(config: &RuntimeHttpServerConfig) -> RuntimeOptions {
RuntimeOptions {
runtime_id: config.runtime_id.clone(),
display_name: config.display_name.clone(),
limits: config.limits.clone(),
}
}
fn parse_args<I, S>(args: I) -> Result<Option<ProcessConfig>, ProcessError>
where where
I: IntoIterator<Item = S>, I: IntoIterator<Item = S>,
S: Into<String>, S: Into<String>,
{ {
let mut config = RuntimeHttpServerConfig::default(); let mut config = ProcessConfig::default()?;
let mut store = StoreArg::Memory; let mut store = StoreArg::Memory;
let mut args = args.into_iter().map(Into::into).collect::<VecDeque<_>>(); let mut args = args.into_iter().map(Into::into).collect::<VecDeque<_>>();
@ -68,18 +125,39 @@ where
"--help" | "-h" => return Ok(None), "--help" | "-h" => return Ok(None),
"--bind" => { "--bind" => {
let value = take_value(&flag, inline_value, &mut args)?; let value = take_value(&flag, inline_value, &mut args)?;
config.bind_addr = value.parse::<SocketAddr>().map_err(|error| { config.http.bind_addr = value.parse::<SocketAddr>().map_err(|error| {
ProcessError::usage(format!("invalid --bind socket address `{value}`: {error}")) ProcessError::usage(format!("invalid --bind socket address `{value}`: {error}"))
})?; })?;
} }
"--runtime-id" => { "--runtime-id" => {
let value = take_value(&flag, inline_value, &mut args)?; let value = take_value(&flag, inline_value, &mut args)?;
config.runtime_id = Some(RuntimeId::new(value).ok_or_else(|| { config.http.runtime_id = Some(RuntimeId::new(value).ok_or_else(|| {
ProcessError::usage("--runtime-id must not be empty".to_string()) ProcessError::usage("--runtime-id must not be empty".to_string())
})?); })?);
} }
"--display-name" => { "--display-name" => {
config.display_name = Some(take_value(&flag, inline_value, &mut args)?); config.http.display_name = Some(take_value(&flag, inline_value, &mut args)?);
}
"--workspace" => {
config.workspace_root = PathBuf::from(take_value(&flag, inline_value, &mut args)?);
}
"--cwd" => {
config.cwd = PathBuf::from(take_value(&flag, inline_value, &mut args)?);
}
"--worker-store-dir" => {
config.worker_store_dir =
Some(PathBuf::from(take_value(&flag, inline_value, &mut args)?));
}
"--worker-metadata-dir" => {
config.worker_metadata_dir =
Some(PathBuf::from(take_value(&flag, inline_value, &mut args)?));
}
"--worker-runtime-base-dir" => {
config.worker_runtime_base_dir =
Some(PathBuf::from(take_value(&flag, inline_value, &mut args)?));
}
"--profile" => {
config.profile = Some(take_value(&flag, inline_value, &mut args)?);
} }
"--store" => { "--store" => {
let value = take_value(&flag, inline_value, &mut args)?; let value = take_value(&flag, inline_value, &mut args)?;
@ -106,7 +184,7 @@ where
"--local-token must not be empty when provided".to_string(), "--local-token must not be empty when provided".to_string(),
)); ));
} }
config.local_token = Some(value); config.http.local_token = Some(value);
} }
"--local-token-env" => { "--local-token-env" => {
let name = take_value(&flag, inline_value, &mut args)?; let name = take_value(&flag, inline_value, &mut args)?;
@ -120,14 +198,14 @@ where
"--local-token-env `{name}` resolved to an empty value" "--local-token-env `{name}` resolved to an empty value"
))); )));
} }
config.local_token = Some(value); config.http.local_token = Some(value);
} }
"--max-transcript-projection-items" => { "--max-transcript-projection-items" => {
config.limits.max_transcript_projection_items = config.http.limits.max_transcript_projection_items =
parse_usize_flag(&flag, take_value(&flag, inline_value, &mut args)?)?; parse_usize_flag(&flag, take_value(&flag, inline_value, &mut args)?)?;
} }
"--max-event-batch-items" => { "--max-event-batch-items" => {
config.limits.max_event_batch_items = config.http.limits.max_event_batch_items =
parse_usize_flag(&flag, take_value(&flag, inline_value, &mut args)?)?; parse_usize_flag(&flag, take_value(&flag, inline_value, &mut args)?)?;
} }
_ => { _ => {
@ -136,7 +214,10 @@ where
} }
} }
apply_store_selection(&mut config, store)?; apply_store_selection(&mut config.http, store)?;
if config.cwd.as_os_str().is_empty() {
config.cwd = config.workspace_root.clone();
}
Ok(Some(config)) Ok(Some(config))
} }
@ -180,31 +261,40 @@ fn apply_store_selection(
config.store = RuntimeHttpStoreSelection::Memory; config.store = RuntimeHttpStoreSelection::Memory;
Ok(()) Ok(())
} }
StoreArg::Fs { root } => apply_fs_store_selection(config, root), StoreArg::Fs { root } => {
} let root = root.ok_or_else(|| {
} ProcessError::usage("--store fs requires --fs-root <PATH>".to_string())
})?;
#[cfg(feature = "fs-store")]
fn apply_fs_store_selection(
config: &mut RuntimeHttpServerConfig,
root: Option<PathBuf>,
) -> Result<(), ProcessError> {
let root = root
.ok_or_else(|| ProcessError::usage("--store fs requires --fs-root <PATH>".to_string()))?;
config.store = RuntimeHttpStoreSelection::Fs { root }; config.store = RuntimeHttpStoreSelection::Fs { root };
Ok(()) Ok(())
} }
}
}
#[cfg(not(feature = "fs-store"))] #[derive(Clone, Debug, PartialEq, Eq)]
fn apply_fs_store_selection( struct ProcessConfig {
_config: &mut RuntimeHttpServerConfig, http: RuntimeHttpServerConfig,
root: Option<PathBuf>, workspace_root: PathBuf,
) -> Result<(), ProcessError> { cwd: PathBuf,
let _ = root; worker_store_dir: Option<PathBuf>,
Err(ProcessError::usage( worker_metadata_dir: Option<PathBuf>,
"fs store selection requires building worker-runtime with features `http-server,fs-store`" worker_runtime_base_dir: Option<PathBuf>,
.to_string(), profile: Option<String>,
)) }
impl ProcessConfig {
fn default() -> Result<Self, ProcessError> {
let workspace_root = env::current_dir()?;
Ok(Self {
http: RuntimeHttpServerConfig::default(),
cwd: workspace_root.clone(),
workspace_root,
worker_store_dir: None,
worker_metadata_dir: None,
worker_runtime_base_dir: None,
profile: None,
})
}
} }
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
@ -217,6 +307,8 @@ enum StoreArg {
enum ProcessError { enum ProcessError {
Usage(String), Usage(String),
Server(RuntimeHttpServerError), Server(RuntimeHttpServerError),
Runtime(RuntimeError),
WorkerAdapter(String),
Io(std::io::Error), Io(std::io::Error),
} }
@ -231,6 +323,8 @@ impl fmt::Display for ProcessError {
match self { match self {
Self::Usage(message) => message.fmt(f), Self::Usage(message) => message.fmt(f),
Self::Server(error) => error.fmt(f), Self::Server(error) => error.fmt(f),
Self::Runtime(error) => error.fmt(f),
Self::WorkerAdapter(error) => error.fmt(f),
Self::Io(error) => error.fmt(f), Self::Io(error) => error.fmt(f),
} }
} }
@ -239,8 +333,9 @@ impl fmt::Display for ProcessError {
impl Error for ProcessError { impl Error for ProcessError {
fn source(&self) -> Option<&(dyn Error + 'static)> { fn source(&self) -> Option<&(dyn Error + 'static)> {
match self { match self {
Self::Usage(_) => None, Self::Usage(_) | Self::WorkerAdapter(_) => None,
Self::Server(error) => Some(error), Self::Server(error) => Some(error),
Self::Runtime(error) => Some(error),
Self::Io(error) => Some(error), Self::Io(error) => Some(error),
} }
} }
@ -260,14 +355,20 @@ impl From<std::io::Error> for ProcessError {
fn usage() -> &'static str { fn usage() -> &'static str {
"Usage: worker-runtime-rest-server [OPTIONS]\n\n\ "Usage: worker-runtime-rest-server [OPTIONS]\n\n\
Starts the worker-runtime REST command API for a trusted backend/proxy.\n\ Starts a worker-backed Runtime REST command API for a trusted backend/proxy.\n\
Browsers must not connect to this Runtime process directly.\n\n\ Browsers must not connect to this Runtime process directly.\n\n\
Options:\n\ Options:\n\
--bind <ADDR> Bind socket address (default: 127.0.0.1:0)\n\ --bind <ADDR> Bind socket address (default: 127.0.0.1:38800)\n\
--runtime-id <ID> Runtime authority id (default: generated)\n\ --runtime-id <ID> Runtime authority id (default: generated)\n\
--display-name <NAME> Runtime display name\n\ --display-name <NAME> Runtime display name\n\
--store <memory|fs> Store selection (default: memory)\n\ --workspace <PATH> Workspace root used for spawned Workers (default: cwd)\n\
--fs-root <PATH> Filesystem store root; requires fs-store feature\n\ --cwd <PATH> Process cwd used for spawned Workers (default: workspace)\n\
--profile <SELECTOR> Force spawned Workers to use a Profile selector\n\
--worker-store-dir <PATH> Worker session store directory\n\
--worker-metadata-dir <PATH> Worker metadata directory\n\
--worker-runtime-base-dir <PATH> Worker controller runtime directory\n\
--store <memory|fs> Runtime catalog store selection (default: memory)\n\
--fs-root <PATH> Runtime catalog filesystem store root\n\
--local-token <TOKEN> Minimal local bearer token placeholder\n\ --local-token <TOKEN> Minimal local bearer token placeholder\n\
--local-token-env <ENV> Read local bearer token placeholder from env\n\ --local-token-env <ENV> Read local bearer token placeholder from env\n\
--max-transcript-projection-items <N> Override transcript projection limit\n\ --max-transcript-projection-items <N> Override transcript projection limit\n\
@ -279,68 +380,84 @@ Options:\n\
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn parses_memory_runtime_process_config() {
let config = parse_args([
"--bind",
"127.0.0.1:48181",
"--runtime-id=runtime-review",
"--display-name",
"review runtime",
"--store",
"memory",
"--local-token",
"local-placeholder",
"--max-transcript-projection-items",
"32",
"--max-event-batch-items=16",
])
.unwrap()
.unwrap();
assert_eq!(
config.bind_addr,
"127.0.0.1:48181".parse::<SocketAddr>().unwrap()
);
assert_eq!(
config.runtime_id.as_ref().map(RuntimeId::as_str),
Some("runtime-review")
);
assert_eq!(config.display_name.as_deref(), Some("review runtime"));
assert!(matches!(config.store, RuntimeHttpStoreSelection::Memory));
assert_eq!(config.local_token.as_deref(), Some("local-placeholder"));
assert_eq!(config.limits.max_transcript_projection_items, 32);
assert_eq!(config.limits.max_event_batch_items, 16);
}
#[cfg(feature = "fs-store")]
#[test]
fn parses_fs_store_runtime_process_config_when_feature_enabled() {
let config = parse_args(["--fs-root", "/tmp/yoi-worker-runtime-store"])
.unwrap()
.unwrap();
assert!(matches!(
config.store,
RuntimeHttpStoreSelection::Fs { ref root }
if root == &PathBuf::from("/tmp/yoi-worker-runtime-store")
));
}
#[cfg(not(feature = "fs-store"))]
#[test]
fn rejects_fs_store_runtime_process_config_without_feature() {
let error = parse_args(["--store", "fs", "--fs-root", "/tmp/store"]).unwrap_err();
assert!(
error
.to_string()
.contains("requires building worker-runtime with features")
);
}
#[test] #[test]
fn help_does_not_start_server() { fn help_does_not_start_server() {
assert!(parse_args(["--help"]).unwrap().is_none()); assert!(parse_args(["--help"]).unwrap().is_none());
} }
#[test]
fn default_bind_addr_is_stable_for_local_dev() {
let config = parse_args([] as [&str; 0]).unwrap().unwrap();
assert_eq!(
config.http.bind_addr,
"127.0.0.1:38800".parse::<SocketAddr>().unwrap()
);
}
#[test]
fn parses_memory_runtime_process_config() {
let config = parse_args([
"--bind",
"127.0.0.1:0",
"--runtime-id=runtime-alpha",
"--display-name",
"Runtime Alpha",
"--workspace",
"/tmp/workspace",
"--cwd",
"/tmp/workspace/subdir",
"--profile",
"builtin:coder",
"--local-token-env",
"TEST_RUNTIME_TOKEN",
]);
assert!(config.is_err());
unsafe {
env::set_var("TEST_RUNTIME_TOKEN", "secret-token");
}
let config = parse_args([
"--bind",
"127.0.0.1:0",
"--runtime-id=runtime-alpha",
"--display-name",
"Runtime Alpha",
"--workspace",
"/tmp/workspace",
"--cwd",
"/tmp/workspace/subdir",
"--profile",
"builtin:coder",
"--local-token-env",
"TEST_RUNTIME_TOKEN",
])
.unwrap()
.unwrap();
unsafe {
env::remove_var("TEST_RUNTIME_TOKEN");
}
assert_eq!(config.http.bind_addr, "127.0.0.1:0".parse().unwrap());
assert_eq!(
config.http.runtime_id.as_ref().map(ToString::to_string),
Some("runtime-alpha".to_string())
);
assert_eq!(config.http.display_name.as_deref(), Some("Runtime Alpha"));
assert_eq!(config.workspace_root, PathBuf::from("/tmp/workspace"));
assert_eq!(config.cwd, PathBuf::from("/tmp/workspace/subdir"));
assert_eq!(config.profile.as_deref(), Some("builtin:coder"));
assert_eq!(config.http.local_token.as_deref(), Some("secret-token"));
}
#[test]
fn parses_fs_store_runtime_process_config() {
let config = parse_args(["--store", "fs", "--fs-root", "/tmp/runtime-store"])
.unwrap()
.unwrap();
assert!(matches!(
config.http.store,
RuntimeHttpStoreSelection::Fs { ref root } if root == &PathBuf::from("/tmp/runtime-store")
));
}
} }

View File

@ -65,4 +65,6 @@ pub struct RuntimeSummary {
pub cancelled_worker_count: usize, pub cancelled_worker_count: usize,
pub diagnostic_count: usize, pub diagnostic_count: usize,
pub limits: RuntimeLimits, pub limits: RuntimeLimits,
#[serde(default)]
pub worker_creation_available: bool,
} }

View File

@ -161,6 +161,7 @@ impl Runtime {
cancelled_worker_count, cancelled_worker_count,
diagnostic_count: state.diagnostics.len(), diagnostic_count: state.diagnostics.len(),
limits: state.limits.clone(), limits: state.limits.clone(),
worker_creation_available: state.execution_backend.is_some(),
}) })
} }

View File

@ -14,6 +14,11 @@ use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex, mpsc}; use std::sync::{Arc, Mutex, mpsc};
use std::time::Duration; use std::time::Duration;
use crate::execution::{
WorkerExecutionBackend, WorkerExecutionHandle, WorkerExecutionOperation, WorkerExecutionResult,
WorkerExecutionRunState, WorkerExecutionSpawnRequest, WorkerExecutionSpawnResult,
};
use crate::interaction::{WorkerInput, WorkerInputKind};
use async_trait::async_trait; use async_trait::async_trait;
use manifest::paths; use manifest::paths;
use protocol::{Method, Segment, WorkerStatus}; use protocol::{Method, Segment, WorkerStatus};
@ -21,13 +26,8 @@ use session_store::FsStore;
use session_store::{CombinedStore, FsWorkerStore}; use session_store::{CombinedStore, FsWorkerStore};
use tokio::runtime::Runtime; use tokio::runtime::Runtime;
use tokio::sync::broadcast; use tokio::sync::broadcast;
use worker_runtime::execution::{
WorkerExecutionBackend, WorkerExecutionHandle, WorkerExecutionOperation, WorkerExecutionResult,
WorkerExecutionRunState, WorkerExecutionSpawnRequest, WorkerExecutionSpawnResult,
};
use worker_runtime::interaction::{WorkerInput, WorkerInputKind};
use crate::{Worker, WorkerController, WorkerHandle}; use worker::{Worker, WorkerController, WorkerHandle};
const DEFAULT_BACKEND_ID: &str = "worker-crate"; const DEFAULT_BACKEND_ID: &str = "worker-crate";
const RUNTIME_TASK_TIMEOUT: Duration = Duration::from_secs(10); const RUNTIME_TASK_TIMEOUT: Duration = Duration::from_secs(10);
@ -115,7 +115,7 @@ impl ProfileRuntimeWorkerFactory {
fn runtime_base_dir(&self) -> Result<PathBuf, String> { fn runtime_base_dir(&self) -> Result<PathBuf, String> {
self.runtime_base_dir self.runtime_base_dir
.clone() .clone()
.or_else(|| crate::runtime::dir::default_base().ok()) .or_else(|| worker::runtime::dir::default_base().ok())
.ok_or_else(|| "could not resolve worker runtime directory".to_string()) .ok_or_else(|| "could not resolve worker runtime directory".to_string())
} }
@ -124,14 +124,14 @@ impl ProfileRuntimeWorkerFactory {
} }
fn runtime_profile_value( fn runtime_profile_value(
profile: &worker_runtime::catalog::ProfileSelector, profile: &crate::catalog::ProfileSelector,
) -> Option<std::borrow::Cow<'_, str>> { ) -> Option<std::borrow::Cow<'_, str>> {
match profile { match profile {
worker_runtime::catalog::ProfileSelector::RuntimeDefault => None, crate::catalog::ProfileSelector::RuntimeDefault => None,
worker_runtime::catalog::ProfileSelector::Named(name) => { crate::catalog::ProfileSelector::Named(name) => {
Some(std::borrow::Cow::Borrowed(name.as_str())) Some(std::borrow::Cow::Borrowed(name.as_str()))
} }
worker_runtime::catalog::ProfileSelector::Builtin(name) => { crate::catalog::ProfileSelector::Builtin(name) => {
if name.starts_with("builtin:") { if name.starts_with("builtin:") {
Some(std::borrow::Cow::Borrowed(name.as_str())) Some(std::borrow::Cow::Borrowed(name.as_str()))
} else { } else {
@ -160,7 +160,7 @@ impl RuntimeWorkerFactory for ProfileRuntimeWorkerFactory {
) -> Result<WorkerHandle, String> { ) -> Result<WorkerHandle, String> {
let worker_name = Self::runtime_worker_name(&request); let worker_name = Self::runtime_worker_name(&request);
let profile = self.runtime_profile(&request); let profile = self.runtime_profile(&request);
let (mut manifest, loader) = crate::entrypoint::resolve_runtime_profile_manifest( let (mut manifest, loader) = worker::entrypoint::resolve_runtime_profile_manifest(
profile.as_deref(), profile.as_deref(),
&self.workspace_root, &self.workspace_root,
&worker_name, &worker_name,
@ -211,7 +211,7 @@ pub struct WorkerRuntimeExecutionBackend<F = ProfileRuntimeWorkerFactory> {
backend_id: String, backend_id: String,
factory: Arc<F>, factory: Arc<F>,
runtime: Mutex<Option<Runtime>>, runtime: Mutex<Option<Runtime>>,
workers: Mutex<HashMap<worker_runtime::identity::WorkerRef, RuntimeWorkerExecution>>, workers: Mutex<HashMap<crate::identity::WorkerRef, RuntimeWorkerExecution>>,
} }
impl WorkerRuntimeExecutionBackend<ProfileRuntimeWorkerFactory> { impl WorkerRuntimeExecutionBackend<ProfileRuntimeWorkerFactory> {
@ -461,7 +461,7 @@ where
}, },
WorkerExecutionRunState::Busy, WorkerExecutionRunState::Busy,
); );
if result.outcome != worker_runtime::execution::WorkerExecutionOutcome::Accepted { if result.outcome != crate::execution::WorkerExecutionOutcome::Accepted {
busy.store(false, Ordering::SeqCst); busy.store(false, Ordering::SeqCst);
} }
result result
@ -506,17 +506,17 @@ mod tests {
use std::pin::Pin; use std::pin::Pin;
use std::sync::atomic::{AtomicUsize, Ordering}; use std::sync::atomic::{AtomicUsize, Ordering};
use crate::Runtime as EmbeddedRuntime;
use crate::catalog::{ConfigBundleRef, CreateWorkerRequest, ProfileSelector};
use crate::identity::RuntimeId;
use crate::management::RuntimeOptions;
use crate::observation::{TranscriptQuery, TranscriptRole};
use async_trait::async_trait; use async_trait::async_trait;
use futures::Stream; use futures::Stream;
use llm_engine::Engine; use llm_engine::Engine;
use llm_engine::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEvent}; use llm_engine::llm_client::event::{Event as LlmEvent, ResponseStatus, StatusEvent};
use llm_engine::llm_client::{ClientError, LlmClient, Request}; use llm_engine::llm_client::{ClientError, LlmClient, Request};
use manifest::{Scope, WorkerManifest}; use manifest::{Scope, WorkerManifest};
use worker_runtime::Runtime as EmbeddedRuntime;
use worker_runtime::catalog::{CreateWorkerRequest, ProfileSelector, WorkerIntent};
use worker_runtime::identity::RuntimeId;
use worker_runtime::management::RuntimeOptions;
use worker_runtime::observation::{TranscriptQuery, TranscriptRole};
#[derive(Clone)] #[derive(Clone)]
struct MockClient { struct MockClient {
@ -619,17 +619,37 @@ mod tests {
] ]
} }
fn create_request(name: &str) -> CreateWorkerRequest { fn test_bundle() -> crate::config_bundle::ConfigBundle {
CreateWorkerRequest { crate::config_bundle::ConfigBundle {
intent: WorkerIntent::Role { metadata: crate::config_bundle::ConfigBundleMetadata {
role: name.to_string(), id: "adapter-test-bundle".to_string(),
purpose: None, digest: String::new(),
revision: "test".to_string(),
workspace_id: "adapter-test".to_string(),
created_at: "test".to_string(),
provenance: crate::config_bundle::ConfigBundleProvenance {
source: "test".to_string(),
detail: None,
}, },
},
profiles: vec![crate::config_bundle::ConfigProfileDescriptor {
selector: ProfileSelector::RuntimeDefault,
label: Some("adapter-test".to_string()),
}],
declarations: Vec::new(),
}
.with_computed_digest()
}
fn create_request(_name: &str) -> CreateWorkerRequest {
let bundle = test_bundle();
CreateWorkerRequest {
profile: ProfileSelector::RuntimeDefault, profile: ProfileSelector::RuntimeDefault,
config_bundle: None, config_bundle: ConfigBundleRef {
requested_capabilities: Vec::new(), id: bundle.metadata.id,
workspace_refs: Vec::new(), digest: bundle.metadata.digest,
mount_refs: Vec::new(), },
initial_input: None,
} }
} }
@ -637,14 +657,14 @@ mod tests {
fn builtin_profile_selector_is_not_double_prefixed() { fn builtin_profile_selector_is_not_double_prefixed() {
assert_eq!( assert_eq!(
ProfileRuntimeWorkerFactory::runtime_profile_value( ProfileRuntimeWorkerFactory::runtime_profile_value(
&worker_runtime::catalog::ProfileSelector::Builtin("coder".to_string()) &crate::catalog::ProfileSelector::Builtin("coder".to_string())
) )
.as_deref(), .as_deref(),
Some("builtin:coder") Some("builtin:coder")
); );
assert_eq!( assert_eq!(
ProfileRuntimeWorkerFactory::runtime_profile_value( ProfileRuntimeWorkerFactory::runtime_profile_value(
&worker_runtime::catalog::ProfileSelector::Builtin("builtin:coder".to_string()) &crate::catalog::ProfileSelector::Builtin("builtin:coder".to_string())
) )
.as_deref(), .as_deref(),
Some("builtin:coder") Some("builtin:coder")
@ -673,6 +693,7 @@ mod tests {
Arc::new(backend), Arc::new(backend),
) )
.unwrap(); .unwrap();
runtime.store_config_bundle(test_bundle()).unwrap();
let detail = runtime.create_worker(create_request("chat")).unwrap(); let detail = runtime.create_worker(create_request("chat")).unwrap();
runtime runtime
@ -700,7 +721,7 @@ mod tests {
let observations = runtime let observations = runtime
.read_worker_observation_events( .read_worker_observation_events(
&detail.worker_ref, &detail.worker_ref,
worker_runtime::observation::WorkerObservationCursor::zero(), crate::observation::WorkerObservationCursor::zero(),
) )
.unwrap(); .unwrap();
assert!( assert!(

View File

@ -7,7 +7,6 @@ autobins = false
[features] [features]
default = [] default = []
runtime-adapter = ["dep:worker-runtime"]
[dependencies] [dependencies]
async-trait = { workspace = true } async-trait = { workspace = true }
@ -19,7 +18,6 @@ mcp = { workspace = true }
protocol = { workspace = true } protocol = { workspace = true }
provider = { workspace = true } provider = { workspace = true }
client = { workspace = true } client = { workspace = true }
worker-runtime = { workspace = true, features = ["ws-server"], optional = true }
serde = { workspace = true, features = ["derive"] } serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true } serde_json = { workspace = true }
reqwest = { version = "0.13", default-features = false, features = ["blocking", "native-tls"] } reqwest = { version = "0.13", default-features = false, features = ["blocking", "native-tls"] }

View File

@ -225,8 +225,7 @@ fn load_profile(
Ok((resolved.manifest, PromptLoader::builtins_only())) Ok((resolved.manifest, PromptLoader::builtins_only()))
} }
#[cfg(feature = "runtime-adapter")] pub fn resolve_runtime_profile_manifest(
pub(crate) fn resolve_runtime_profile_manifest(
profile: Option<&str>, profile: Option<&str>,
workspace_root: &Path, workspace_root: &Path,
worker_name: &str, worker_name: &str,

View File

@ -10,8 +10,6 @@ pub(crate) mod in_flight;
pub mod ipc; pub mod ipc;
pub mod prompt; pub mod prompt;
pub mod runtime; pub mod runtime;
#[cfg(feature = "runtime-adapter")]
pub mod runtime_adapter;
pub mod segment_log_sink; pub mod segment_log_sink;
pub mod shared_state; pub mod shared_state;
mod shutdown_after_idle; mod shutdown_after_idle;

View File

@ -23,7 +23,7 @@ thiserror.workspace = true
ticket.workspace = true ticket.workspace = true
tokio = { workspace = true, features = ["fs", "macros", "net", "rt-multi-thread", "sync"] } tokio = { workspace = true, features = ["fs", "macros", "net", "rt-multi-thread", "sync"] }
tokio-tungstenite.workspace = true tokio-tungstenite.workspace = true
worker = { workspace = true, features = ["runtime-adapter"] } worker.workspace = true
worker-runtime = { workspace = true, features = ["ws-server", "fs-store"] } worker-runtime = { workspace = true, features = ["ws-server", "fs-store"] }
toml.workspace = true toml.workspace = true
tracing.workspace = true tracing.workspace = true

View File

@ -299,7 +299,9 @@ impl ResolvedWorkspaceBackendConfig {
} }
} }
fn resolve_remote_runtime(config: &RemoteRuntimeConfigFile) -> Result<RemoteRuntimeConfig> { pub(crate) fn resolve_remote_runtime(
config: &RemoteRuntimeConfigFile,
) -> Result<RemoteRuntimeConfig> {
if let Some(token_ref) = config.token_ref.as_deref() { if let Some(token_ref) = config.token_ref.as_deref() {
return Err(Error::Config(format!( return Err(Error::Config(format!(
"remote runtime `{}` uses token_ref `{token_ref}`, but secret ref resolution is not implemented for workspace backend config yet", "remote runtime `{}` uses token_ref `{token_ref}`, but secret ref resolution is not implemented for workspace backend config yet",

View File

@ -6,7 +6,11 @@ use reqwest::header::{AUTHORIZATION, CONTENT_TYPE};
use serde::de::DeserializeOwned; use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use std::{path::PathBuf, sync::Arc, time::Duration}; use std::{
path::PathBuf,
sync::{Arc, RwLock},
time::Duration,
};
use worker_runtime::catalog::{ use worker_runtime::catalog::{
ConfigBundleRef, CreateWorkerRequest, ProfileSelector, WorkerDetail as EmbeddedWorkerDetail, ConfigBundleRef, CreateWorkerRequest, ProfileSelector, WorkerDetail as EmbeddedWorkerDetail,
WorkerStatus as EmbeddedWorkerStatus, WorkerStatus as EmbeddedWorkerStatus,
@ -45,6 +49,22 @@ const MAX_HOST_SCAN: usize = 256;
const MAX_IDENTIFIER_LEN: usize = 120; const MAX_IDENTIFIER_LEN: usize = 120;
const ID_DIGEST_HEX_LEN: usize = 16; const ID_DIGEST_HEX_LEN: usize = 16;
fn run_blocking_http<T, F>(operation: F) -> T
where
T: Send + 'static,
F: FnOnce() -> T + Send + 'static,
{
match tokio::runtime::Handle::try_current() {
Ok(handle) if handle.runtime_flavor() == tokio::runtime::RuntimeFlavor::MultiThread => {
tokio::task::block_in_place(operation)
}
Ok(_) => std::thread::spawn(operation)
.join()
.expect("blocking HTTP thread panicked"),
Err(_) => operation(),
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct RuntimeDiagnostic { pub struct RuntimeDiagnostic {
pub code: String, pub code: String,
@ -643,31 +663,95 @@ pub trait WorkspaceWorkerRuntime: Send + Sync {
} }
} }
#[derive(Debug, Clone)]
pub enum RuntimeRegistryUnregisterResult {
Removed,
NotFound,
BlockedByWorkers {
worker_count: usize,
diagnostics: Vec<RuntimeDiagnostic>,
},
}
#[derive(Clone)] #[derive(Clone)]
pub struct RuntimeRegistry { pub struct RuntimeRegistry {
runtimes: Vec<Arc<dyn WorkspaceWorkerRuntime>>, runtimes: Arc<RwLock<Vec<Arc<dyn WorkspaceWorkerRuntime>>>>,
} }
impl RuntimeRegistry { impl RuntimeRegistry {
pub fn new(runtimes: Vec<Arc<dyn WorkspaceWorkerRuntime>>) -> Self { pub fn new(runtimes: Vec<Arc<dyn WorkspaceWorkerRuntime>>) -> Self {
Self { runtimes } Self {
runtimes: Arc::new(RwLock::new(runtimes)),
}
} }
pub fn for_workspace(embedded_runtime: EmbeddedWorkerRuntime) -> Self { pub fn for_workspace(embedded_runtime: EmbeddedWorkerRuntime) -> Self {
Self::new(vec![Arc::new(embedded_runtime)]) Self::new(vec![Arc::new(embedded_runtime)])
} }
pub fn register<R>(&mut self, runtime: R) pub fn register<R>(&self, runtime: R)
where where
R: WorkspaceWorkerRuntime + 'static, R: WorkspaceWorkerRuntime + 'static,
{ {
self.runtimes.push(Arc::new(runtime)); self.runtimes
.write()
.expect("runtime registry lock poisoned")
.push(Arc::new(runtime));
}
pub fn register_or_replace<R>(&self, runtime: R)
where
R: WorkspaceWorkerRuntime + 'static,
{
let runtime = Arc::new(runtime);
let runtime_id = runtime.runtime_id().to_string();
let mut runtimes = self
.runtimes
.write()
.expect("runtime registry lock poisoned");
runtimes.retain(|existing| existing.runtime_id() != runtime_id);
runtimes.push(runtime);
}
pub fn unregister_if_idle(
&self,
runtime_id: &str,
worker_scan_limit: usize,
) -> Result<RuntimeRegistryUnregisterResult, RuntimeRegistryError> {
validate_backend_identifier("runtime_id", runtime_id)?;
let runtime = self
.runtimes_snapshot()
.into_iter()
.find(|runtime| runtime.runtime_id() == runtime_id);
if let Some(runtime) = runtime {
let worker_list = runtime.list_workers(worker_scan_limit);
if !worker_list.items.is_empty() {
return Ok(RuntimeRegistryUnregisterResult::BlockedByWorkers {
worker_count: worker_list.items.len(),
diagnostics: worker_list.diagnostics,
});
}
} else {
return Ok(RuntimeRegistryUnregisterResult::NotFound);
}
let mut runtimes = self
.runtimes
.write()
.expect("runtime registry lock poisoned");
let before = runtimes.len();
runtimes.retain(|runtime| runtime.runtime_id() != runtime_id);
if runtimes.len() == before {
Ok(RuntimeRegistryUnregisterResult::NotFound)
} else {
Ok(RuntimeRegistryUnregisterResult::Removed)
}
} }
pub fn list_runtimes(&self, limit: usize) -> RuntimeList<RuntimeSummary> { pub fn list_runtimes(&self, limit: usize) -> RuntimeList<RuntimeSummary> {
let mut diagnostics = Vec::new(); let mut diagnostics = Vec::new();
let mut items = Vec::new(); let mut items = Vec::new();
for runtime in self.runtimes.iter().take(limit) { for runtime in self.runtimes_snapshot().iter().take(limit) {
let summary = runtime.runtime_summary(limit); let summary = runtime.runtime_summary(limit);
diagnostics.extend(summary.diagnostics.iter().cloned()); diagnostics.extend(summary.diagnostics.iter().cloned());
items.push(summary); items.push(summary);
@ -679,7 +763,7 @@ impl RuntimeRegistry {
pub fn list_hosts(&self, limit: usize) -> RuntimeList<HostSummary> { pub fn list_hosts(&self, limit: usize) -> RuntimeList<HostSummary> {
let mut items = Vec::new(); let mut items = Vec::new();
let mut diagnostics = Vec::new(); let mut diagnostics = Vec::new();
for runtime in &self.runtimes { for runtime in self.runtimes_snapshot() {
if items.len() >= limit { if items.len() >= limit {
break; break;
} }
@ -694,7 +778,7 @@ impl RuntimeRegistry {
pub fn list_workers(&self, limit: usize) -> RuntimeList<WorkerSummary> { pub fn list_workers(&self, limit: usize) -> RuntimeList<WorkerSummary> {
let mut items = Vec::new(); let mut items = Vec::new();
let mut diagnostics = Vec::new(); let mut diagnostics = Vec::new();
for runtime in &self.runtimes { for runtime in self.runtimes_snapshot() {
if items.len() >= limit { if items.len() >= limit {
break; break;
} }
@ -716,7 +800,7 @@ impl RuntimeRegistry {
let mut host_found = false; let mut host_found = false;
let mut diagnostics = Vec::new(); let mut diagnostics = Vec::new();
let mut items = Vec::new(); let mut items = Vec::new();
for runtime in &self.runtimes { for runtime in self.runtimes_snapshot() {
let host_list = runtime.list_hosts(MAX_HOST_SCAN); let host_list = runtime.list_hosts(MAX_HOST_SCAN);
diagnostics.extend(host_list.diagnostics); diagnostics.extend(host_list.diagnostics);
if !host_list.items.iter().any(|host| host.host_id == host_id) { if !host_list.items.iter().any(|host| host.host_id == host_id) {
@ -894,13 +978,23 @@ impl RuntimeRegistry {
}) })
} }
fn runtimes_snapshot(&self) -> Vec<Arc<dyn WorkspaceWorkerRuntime>> {
self.runtimes
.read()
.expect("runtime registry lock poisoned")
.clone()
}
fn runtime( fn runtime(
&self, &self,
runtime_id: &str, runtime_id: &str,
) -> Result<&Arc<dyn WorkspaceWorkerRuntime>, RuntimeRegistryError> { ) -> Result<Arc<dyn WorkspaceWorkerRuntime>, RuntimeRegistryError> {
self.runtimes self.runtimes
.read()
.expect("runtime registry lock poisoned")
.iter() .iter()
.find(|runtime| runtime.runtime_id() == runtime_id) .find(|runtime| runtime.runtime_id() == runtime_id)
.cloned()
.ok_or_else(|| RuntimeRegistryError::UnknownRuntime(runtime_id.to_string())) .ok_or_else(|| RuntimeRegistryError::UnknownRuntime(runtime_id.to_string()))
} }
} }
@ -1548,7 +1642,7 @@ impl RemoteRuntimeConfig {
display_name: display_name.into(), display_name: display_name.into(),
base_url: base_url.into(), base_url: base_url.into(),
bearer_token, bearer_token,
cached_capabilities: remote_runtime_capabilities(200, false), cached_capabilities: remote_runtime_capabilities(200, false, false),
cached_status: "configured".to_string(), cached_status: "configured".to_string(),
timeout: Duration::from_secs(10), timeout: Duration::from_secs(10),
} }
@ -1586,9 +1680,9 @@ impl RemoteWorkerRuntime {
pub fn new(config: RemoteRuntimeConfig) -> Result<Self, RuntimeRegistryError> { pub fn new(config: RemoteRuntimeConfig) -> Result<Self, RuntimeRegistryError> {
validate_backend_identifier("runtime_id", &config.runtime_id)?; validate_backend_identifier("runtime_id", &config.runtime_id)?;
let base_url = config.base_url.trim_end_matches('/').to_string(); let base_url = config.base_url.trim_end_matches('/').to_string();
let http = BlockingHttpClient::builder() let timeout = config.timeout;
.timeout(config.timeout) let http =
.build() run_blocking_http(move || BlockingHttpClient::builder().timeout(timeout).build())
.map_err(|err| RuntimeRegistryError::RuntimeOperationFailed { .map_err(|err| RuntimeRegistryError::RuntimeOperationFailed {
runtime_id: config.runtime_id.clone(), runtime_id: config.runtime_id.clone(),
code: "remote_runtime_client_build_failed".to_string(), code: "remote_runtime_client_build_failed".to_string(),
@ -1628,18 +1722,9 @@ impl RemoteWorkerRuntime {
format!("{base}/v1/workers/{worker_id}/events/ws") format!("{base}/v1/workers/{worker_id}/events/ws")
} }
fn authorize(&self, request: RequestBuilder) -> RequestBuilder {
let request = request.header(CONTENT_TYPE, "application/json");
if let Some(token) = self.bearer_token.as_deref() {
request.header(AUTHORIZATION, format!("Bearer {token}"))
} else {
request
}
}
fn get_json<T>(&self, path: &str) -> Result<T, RuntimeDiagnostic> fn get_json<T>(&self, path: &str) -> Result<T, RuntimeDiagnostic>
where where
T: DeserializeOwned, T: DeserializeOwned + Send + 'static,
{ {
self.send_json(self.http.get(self.endpoint(path))) self.send_json(self.http.get(self.endpoint(path)))
} }
@ -1647,19 +1732,27 @@ impl RemoteWorkerRuntime {
fn post_json<B, T>(&self, path: &str, body: &B) -> Result<T, RuntimeDiagnostic> fn post_json<B, T>(&self, path: &str, body: &B) -> Result<T, RuntimeDiagnostic>
where where
B: Serialize + ?Sized, B: Serialize + ?Sized,
T: DeserializeOwned, T: DeserializeOwned + Send + 'static,
{ {
self.send_json(self.http.post(self.endpoint(path)).json(body)) self.send_json(self.http.post(self.endpoint(path)).json(body))
} }
fn send_json<T>(&self, request: RequestBuilder) -> Result<T, RuntimeDiagnostic> fn send_json<T>(&self, request: RequestBuilder) -> Result<T, RuntimeDiagnostic>
where where
T: DeserializeOwned, T: DeserializeOwned + Send + 'static,
{ {
let response = self let runtime_id = self.runtime_id.clone();
.authorize(request) let bearer_token = self.bearer_token.clone();
run_blocking_http(move || {
let request = request.header(CONTENT_TYPE, "application/json");
let request = if let Some(token) = bearer_token.as_deref() {
request.header(AUTHORIZATION, format!("Bearer {token}"))
} else {
request
};
let response = request
.send() .send()
.map_err(|err| remote_reqwest_diagnostic(&self.runtime_id, err))?; .map_err(|err| remote_reqwest_diagnostic(&runtime_id, err))?;
let status = response.status(); let status = response.status();
if status.is_success() { if status.is_success() {
response.json::<T>().map_err(|err| { response.json::<T>().map_err(|err| {
@ -1668,17 +1761,14 @@ impl RemoteWorkerRuntime {
DiagnosticSeverity::Error, DiagnosticSeverity::Error,
format!( format!(
"Remote Runtime returned malformed JSON for '{}': {err}", "Remote Runtime returned malformed JSON for '{}': {err}",
self.runtime_id runtime_id
), ),
) )
}) })
} else { } else {
Err(remote_http_status_diagnostic( Err(remote_http_status_diagnostic(&runtime_id, status, response))
&self.runtime_id,
status,
response,
))
} }
})
} }
fn map_worker_summary(&self, summary: worker_runtime::catalog::WorkerSummary) -> WorkerSummary { fn map_worker_summary(&self, summary: worker_runtime::catalog::WorkerSummary) -> WorkerSummary {
@ -1790,7 +1880,11 @@ impl WorkspaceWorkerRuntime for RemoteWorkerRuntime {
} else { } else {
vec![self.host_id.clone()] vec![self.host_id.clone()]
}, },
capabilities: remote_runtime_capabilities(limit, true), capabilities: remote_runtime_capabilities(
limit,
true,
response.runtime.worker_creation_available,
),
diagnostics: vec![diagnostic( diagnostics: vec![diagnostic(
"remote_runtime_backend_proxy", "remote_runtime_backend_proxy",
DiagnosticSeverity::Info, DiagnosticSeverity::Info,
@ -1827,7 +1921,7 @@ impl WorkspaceWorkerRuntime for RemoteWorkerRuntime {
status: "configured".to_string(), status: "configured".to_string(),
observed_at: Utc::now().to_rfc3339(), observed_at: Utc::now().to_rfc3339(),
last_seen_at: None, last_seen_at: None,
capabilities: remote_runtime_capabilities(limit, true), capabilities: remote_runtime_capabilities(limit, true, false),
diagnostics: vec![diagnostic( diagnostics: vec![diagnostic(
"remote_runtime_backend_proxy", "remote_runtime_backend_proxy",
DiagnosticSeverity::Info, DiagnosticSeverity::Info,
@ -2489,14 +2583,18 @@ fn percent_encode(input: &str, keep: impl Fn(u8) -> bool) -> String {
encoded encoded
} }
fn remote_runtime_capabilities(limit: usize, available: bool) -> RuntimeCapabilitySummary { fn remote_runtime_capabilities(
limit: usize,
available: bool,
worker_creation_available: bool,
) -> RuntimeCapabilitySummary {
RuntimeCapabilitySummary { RuntimeCapabilitySummary {
can_list_hosts: true, can_list_hosts: true,
can_list_workers: available, can_list_workers: available,
can_get_worker: available, can_get_worker: available,
can_spawn_worker: available, can_spawn_worker: available && worker_creation_available,
can_stop_worker: available, can_stop_worker: available,
can_accept_input: available, can_accept_input: available && worker_creation_available,
has_workspace_fs: false, has_workspace_fs: false,
has_shell: false, has_shell: false,
has_git: false, has_git: false,
@ -3327,6 +3425,19 @@ mod tests {
); );
} }
#[tokio::test]
async fn remote_runtime_client_can_initialize_inside_tokio_context() {
let runtime = RemoteWorkerRuntime::new(RemoteRuntimeConfig::new(
"remote:async-init",
"Remote Async Init",
"http://127.0.0.1:9",
None,
))
.unwrap();
assert_eq!(runtime.runtime_id(), "remote:async-init");
}
#[test] #[test]
fn remote_runtime_registry_routes_commands_without_browser_secret_leaks() { fn remote_runtime_registry_routes_commands_without_browser_secret_leaks() {
let worker_json = worker_json("remote:primary", "worker-remote-1"); let worker_json = worker_json("remote:primary", "worker-remote-1");
@ -3362,7 +3473,7 @@ mod tests {
), ),
]); ]);
let secret = "secret-token-do-not-leak".to_string(); let secret = "secret-token-do-not-leak".to_string();
let mut registry = RuntimeRegistry::new(Vec::new()); let registry = RuntimeRegistry::new(Vec::new());
registry.register( registry.register(
RemoteWorkerRuntime::new(RemoteRuntimeConfig::new( RemoteWorkerRuntime::new(RemoteRuntimeConfig::new(
"remote:primary", "remote:primary",
@ -3559,7 +3670,7 @@ mod tests {
.to_string(), .to_string(),
), ),
]); ]);
let mut registry = RuntimeRegistry::new(Vec::new()); let registry = RuntimeRegistry::new(Vec::new());
registry.register( registry.register(
RemoteWorkerRuntime::new(RemoteRuntimeConfig::new( RemoteWorkerRuntime::new(RemoteRuntimeConfig::new(
"remote:primary", "remote:primary",
@ -3605,7 +3716,7 @@ mod tests {
401, 401,
json!({ "error": { "code": "unauthorized", "message": "bad token" } }).to_string(), json!({ "error": { "code": "unauthorized", "message": "bad token" } }).to_string(),
)]); )]);
let mut registry = RuntimeRegistry::new(Vec::new()); let registry = RuntimeRegistry::new(Vec::new());
registry.register( registry.register(
RemoteWorkerRuntime::new(RemoteRuntimeConfig::new( RemoteWorkerRuntime::new(RemoteRuntimeConfig::new(
"remote:primary", "remote:primary",

View File

@ -12,20 +12,20 @@ use chrono::{SecondsFormat, Utc};
use futures::StreamExt; use futures::StreamExt;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use tokio::net::TcpListener; use tokio::net::TcpListener;
use worker::runtime_adapter::WorkerRuntimeExecutionBackend; use worker_runtime::worker_backend::WorkerRuntimeExecutionBackend;
use crate::companion::{ use crate::companion::{
CompanionCancelRequest, CompanionConsole, CompanionMessageRequest, CompanionMessageResponse, CompanionCancelRequest, CompanionConsole, CompanionMessageRequest, CompanionMessageResponse,
CompanionStatusResponse, CompanionTranscriptProjection, CompanionStatusResponse, CompanionTranscriptProjection,
}; };
use crate::config::{RemoteRuntimeConfigFile, WorkspaceBackendConfigFile}; use crate::config::{RemoteRuntimeConfigFile, WorkspaceBackendConfigFile, resolve_remote_runtime};
use crate::hosts::{ use crate::hosts::{
ConfigBundleCheckResult, ConfigBundleSyncResult, DiagnosticSeverity, EmbeddedWorkerRuntime, ConfigBundleCheckResult, ConfigBundleSyncResult, DiagnosticSeverity, EmbeddedWorkerRuntime,
HostSummary, RemoteRuntimeConfig, RemoteWorkerRuntime, RuntimeDiagnostic, RuntimeRegistry, HostSummary, RemoteRuntimeConfig, RemoteWorkerRuntime, RuntimeDiagnostic, RuntimeRegistry,
RuntimeSummary, WorkerInputRequest, WorkerInputResult, WorkerLifecycleRequest, RuntimeRegistryUnregisterResult, RuntimeSummary, WorkerInputRequest, WorkerInputResult,
WorkerLifecycleResult, WorkerOperationState, WorkerSpawnAcceptanceRequirement, WorkerLifecycleRequest, WorkerLifecycleResult, WorkerOperationState,
WorkerSpawnIntent, WorkerSpawnRequest, WorkerSpawnResult, WorkerSummary, WorkerSpawnAcceptanceRequirement, WorkerSpawnIntent, WorkerSpawnRequest, WorkerSpawnResult,
WorkerTranscriptProjection, WorkerSummary, WorkerTranscriptProjection,
}; };
use crate::identity::WorkspaceIdentity; use crate::identity::WorkspaceIdentity;
use crate::observation::{ use crate::observation::{
@ -171,7 +171,7 @@ impl WorkspaceApi {
updated_at: config.workspace_created_at.clone(), updated_at: config.workspace_created_at.clone(),
}) })
.await?; .await?;
let mut runtime = RuntimeRegistry::for_workspace( let runtime = RuntimeRegistry::for_workspace(
EmbeddedWorkerRuntime::new_fs_store_with_execution_backend( EmbeddedWorkerRuntime::new_fs_store_with_execution_backend(
config.workspace_id.clone(), config.workspace_id.clone(),
config.embedded_runtime_store_root.clone(), config.embedded_runtime_store_root.clone(),
@ -782,7 +782,7 @@ async fn add_remote_runtime_connection(
"a remote Runtime connection with that id is already configured", "a remote Runtime connection with that id is already configured",
)); ));
} }
local_config.runtimes.remote.push(RemoteRuntimeConfigFile { let remote_config = RemoteRuntimeConfigFile {
id, id,
endpoint: request.endpoint.trim().to_string(), endpoint: request.endpoint.trim().to_string(),
display_name: request display_name: request
@ -792,9 +792,30 @@ async fn add_remote_runtime_connection(
.filter(|value| !value.is_empty()) .filter(|value| !value.is_empty())
.map(ToOwned::to_owned), .map(ToOwned::to_owned),
token_ref: None, token_ref: None,
}); };
let active_config = remote_runtime_config_from_file(&remote_config).map_err(|diagnostic| {
ApiError::with_diagnostics(
Error::RuntimeOperationFailed {
runtime_id: remote_config.id.clone(),
code: diagnostic.code.clone(),
message: diagnostic.message.clone(),
},
vec![diagnostic],
)
})?;
let active_runtime = RemoteWorkerRuntime::new(active_config).map_err(|err| err.into_error())?;
local_config.runtimes.remote.push(remote_config);
write_workspace_backend_config_for_settings(&api, &local_config)?; write_workspace_backend_config_for_settings(&api, &local_config)?;
let mut response = runtime_connection_mutation_response(&api, &local_config); api.runtime.register_or_replace(active_runtime);
let mut response = runtime_connection_mutation_response(
&api,
&local_config,
vec![settings_diagnostic(
"runtime_registry_applied",
DiagnosticSeverity::Info,
"Remote Runtime config was persisted and applied to the active Runtime registry without restarting the Workspace backend.",
)],
);
response.diagnostics.push(settings_diagnostic( response.diagnostics.push(settings_diagnostic(
"workspace_backend_config_rewritten", "workspace_backend_config_rewritten",
DiagnosticSeverity::Info, DiagnosticSeverity::Info,
@ -822,8 +843,44 @@ async fn delete_remote_runtime_connection(
if before == local_config.runtimes.remote.len() { if before == local_config.runtimes.remote.len() {
return Err(Error::UnknownRuntime(runtime_id).into()); return Err(Error::UnknownRuntime(runtime_id).into());
} }
match api
.runtime
.unregister_if_idle(&runtime_id, api.config.max_records.min(200))
.map_err(|err| err.into_error())?
{
RuntimeRegistryUnregisterResult::Removed | RuntimeRegistryUnregisterResult::NotFound => {}
RuntimeRegistryUnregisterResult::BlockedByWorkers {
worker_count,
diagnostics,
} => {
let mut diagnostics = diagnostics;
diagnostics.push(settings_diagnostic(
"remote_runtime_delete_blocked",
DiagnosticSeverity::Error,
format!(
"Remote Runtime '{runtime_id}' has {worker_count} active worker(s); stop or move them before deleting the connection."
),
));
return Err(ApiError::with_diagnostics(
Error::RuntimeOperationFailed {
runtime_id,
code: "remote_runtime_delete_blocked".to_string(),
message: "Remote Runtime connection has active workers".to_string(),
},
diagnostics,
));
}
}
write_workspace_backend_config_for_settings(&api, &local_config)?; write_workspace_backend_config_for_settings(&api, &local_config)?;
let mut response = runtime_connection_mutation_response(&api, &local_config); let mut response = runtime_connection_mutation_response(
&api,
&local_config,
vec![settings_diagnostic(
"runtime_registry_applied",
DiagnosticSeverity::Info,
"Remote Runtime config was removed from persisted config and the active Runtime registry without restarting the Workspace backend.",
)],
);
response.diagnostics.push(settings_diagnostic( response.diagnostics.push(settings_diagnostic(
"workspace_backend_config_rewritten", "workspace_backend_config_rewritten",
DiagnosticSeverity::Info, DiagnosticSeverity::Info,
@ -1277,16 +1334,13 @@ fn runtime_connection_settings_response(
fn runtime_connection_mutation_response( fn runtime_connection_mutation_response(
api: &WorkspaceApi, api: &WorkspaceApi,
local_config: &WorkspaceBackendConfigFile, local_config: &WorkspaceBackendConfigFile,
diagnostics: Vec<RuntimeDiagnostic>,
) -> RuntimeConnectionMutationResponse { ) -> RuntimeConnectionMutationResponse {
RuntimeConnectionMutationResponse { RuntimeConnectionMutationResponse {
workspace_id: api.config.workspace_id.clone(), workspace_id: api.config.workspace_id.clone(),
restart_required: true, restart_required: false,
remotes: remote_runtime_connection_summaries(api, local_config, true), remotes: remote_runtime_connection_summaries(api, local_config, false),
diagnostics: vec![settings_diagnostic( diagnostics,
"runtime_registry_restart_required",
DiagnosticSeverity::Warning,
"Runtime connection config changed; restart the Workspace backend for the live Runtime registry to use the new config.",
)],
} }
} }
@ -1443,6 +1497,18 @@ fn validate_public_runtime_id(runtime_id: &str) -> ApiResult<()> {
Ok(()) Ok(())
} }
fn remote_runtime_config_from_file(
remote: &RemoteRuntimeConfigFile,
) -> std::result::Result<RemoteRuntimeConfig, RuntimeDiagnostic> {
resolve_remote_runtime(remote).map_err(|err| {
settings_diagnostic(
"remote_runtime_apply_failed",
DiagnosticSeverity::Error,
err.to_string(),
)
})
}
async fn test_remote_runtime_config( async fn test_remote_runtime_config(
api: &WorkspaceApi, api: &WorkspaceApi,
remote: &RemoteRuntimeConfigFile, remote: &RemoteRuntimeConfigFile,
@ -1529,7 +1595,10 @@ async fn test_remote_runtime_config(
); );
} }
}; };
observation.available("runtime.summary", "Runtime summary was readable."); observation.available(
"runtime.summary",
"Connected: /v1/runtime responded with a recognized worker-runtime summary.",
);
let workers_url = match remote_probe_url(remote, "/v1/workers") { let workers_url = match remote_probe_url(remote, "/v1/workers") {
Ok(url) => url, Ok(url) => url,
@ -1544,7 +1613,10 @@ async fn test_remote_runtime_config(
match probe_remote_json(&client, workers_url, "workers.list", "Worker list").await { match probe_remote_json(&client, workers_url, "workers.list", "Worker list").await {
Ok(payload) => match serde_json::from_value::<RuntimeHttpWorkersResponse>(payload) { Ok(payload) => match serde_json::from_value::<RuntimeHttpWorkersResponse>(payload) {
Ok(workers) => { Ok(workers) => {
observation.available("workers.list", "Worker list was readable."); observation.available(
"workers.list",
"Verified: /v1/workers responded with a recognized worker list.",
);
Some(workers) Some(workers)
} }
Err(_) => { Err(_) => {
@ -1574,7 +1646,10 @@ async fn test_remote_runtime_config(
match remote_probe_url(remote, &path) { match remote_probe_url(remote, &path) {
Ok(url) => match probe_remote_json(&client, url, "workers.detail", "Worker detail").await { Ok(url) => match probe_remote_json(&client, url, "workers.detail", "Worker detail").await {
Ok(payload) => match serde_json::from_value::<RuntimeHttpWorkerResponse>(payload) { Ok(payload) => match serde_json::from_value::<RuntimeHttpWorkerResponse>(payload) {
Ok(_) => observation.available("workers.detail", "Worker detail was readable."), Ok(_) => observation.available(
"workers.detail",
"Verified: worker detail responded for an existing worker reported by the remote Runtime.",
),
Err(_) => observation.incompatible( Err(_) => observation.incompatible(
"workers.detail", "workers.detail",
settings_diagnostic( settings_diagnostic(
@ -1591,13 +1666,13 @@ async fn test_remote_runtime_config(
} else { } else {
observation.unknown( observation.unknown(
"workers.detail", "workers.detail",
"Worker detail compatibility could not be proven because the Runtime reported no workers during the lightweight probe.", "No connection problem found. Worker detail was not checked because the remote Runtime reported no workers during the lightweight probe.",
); );
} }
observation.available( observation.available(
"workers.events_ws.construct", "workers.events_ws.construct",
"Worker event websocket URL can be constructed from the configured HTTP(S) Runtime endpoint, but no websocket connection was opened during this lightweight test.", "Verified: worker event websocket URL can be constructed from the configured HTTP(S) Runtime endpoint. The lightweight test does not open a websocket stream.",
); );
let bundles_url = match remote_probe_url(remote, "/v1/config-bundles") { let bundles_url = match remote_probe_url(remote, "/v1/config-bundles") {
@ -1621,8 +1696,10 @@ async fn test_remote_runtime_config(
Ok(payload) => { Ok(payload) => {
match serde_json::from_value::<RuntimeHttpConfigBundlesResponse>(payload) { match serde_json::from_value::<RuntimeHttpConfigBundlesResponse>(payload) {
Ok(bundles) => { Ok(bundles) => {
observation observation.available(
.available("config_bundles.list", "Config-bundle list was readable."); "config_bundles.list",
"Verified: /v1/config-bundles responded with a recognized config-bundle list.",
);
Some(bundles) Some(bundles)
} }
Err(_) => { Err(_) => {
@ -1665,7 +1742,7 @@ async fn test_remote_runtime_config(
{ {
Ok(_) => observation.available( Ok(_) => observation.available(
"config_bundles.availability", "config_bundles.availability",
"Config-bundle availability was readable for an advertised bundle.", "Verified: config-bundle availability was confirmed for an advertised bundle.",
), ),
Err(_) => observation.incompatible( Err(_) => observation.incompatible(
"config_bundles.availability", "config_bundles.availability",
@ -1686,21 +1763,32 @@ async fn test_remote_runtime_config(
} else { } else {
observation.unknown( observation.unknown(
"config_bundles.availability", "config_bundles.availability",
"Config-bundle availability compatibility could not be proven because the Runtime advertised no bundles during the lightweight probe.", "No connection problem found. Config-bundle availability was not checked because the remote Runtime advertised no bundles during the lightweight probe.",
); );
} }
observation.unknown( if summary.runtime.worker_creation_available {
observation.available(
"workers.spawn", "workers.spawn",
"Worker spawn compatibility was not proven because the lightweight test does not create remote workers as a side effect.", "Verified: /v1/runtime reports worker creation is enabled by a Runtime execution backend. The lightweight test does not create a worker.",
); );
} else {
observation.incompatible(
"workers.spawn",
settings_diagnostic(
"remote_runtime_worker_creation_unavailable",
DiagnosticSeverity::Error,
"Connected to the Runtime, but worker creation is unavailable because this Runtime process has no execution backend attached.",
),
);
}
observation.unknown( observation.unknown(
"workers.input_dispatch", "workers.input_dispatch",
"Worker input dispatch compatibility was not proven because the lightweight test does not send model-visible input as a side effect.", "No connection problem found. Worker input dispatch was not checked because this lightweight test does not send model-visible input as a side effect.",
); );
observation.unknown( observation.unknown(
"config_bundles.sync", "config_bundles.sync",
"Config-bundle sync compatibility was not proven because the lightweight test does not upload bundles as a side effect.", "No connection problem found. Config-bundle sync was not checked because this lightweight test does not upload bundles as a side effect.",
); );
RemoteRuntimeTestResponse { RemoteRuntimeTestResponse {
@ -1709,11 +1797,14 @@ async fn test_remote_runtime_config(
checked_at, checked_at,
state: observation.state().to_string(), state: observation.state().to_string(),
protocol_version, protocol_version,
compatibility_basis: "Observed worker-runtime HTTP endpoints for summary, worker listing/detail when available, event websocket URL construction, and config-bundle listing/availability when available; side-effecting spawn/input/sync operations are reported unknown unless the Runtime API proves them.".to_string(), compatibility_basis: "Connected to /v1/runtime and verified non-side-effecting worker-runtime HTTP endpoints. No incompatible operation was found; warning items below are unproven optional or side-effecting checks, not connection failures.".to_string(),
capabilities: observation.capabilities, capabilities: observation.capabilities,
health_result: format!( health_result: format!(
"runtime_status={:?}; incompatible={}; unknown={}", "connected=true; runtime_status={:?}; available={}; incompatible={}; warnings={}",
summary.runtime.status, observation.incompatible_count, observation.unknown_count summary.runtime.status,
observation.available_count,
observation.incompatible_count,
observation.unknown_count
), ),
diagnostics: observation.diagnostics, diagnostics: observation.diagnostics,
} }
@ -1747,13 +1838,20 @@ fn remote_runtime_test_failed(
struct RuntimeCompatibilityObservation { struct RuntimeCompatibilityObservation {
capabilities: Vec<String>, capabilities: Vec<String>,
diagnostics: Vec<RuntimeDiagnostic>, diagnostics: Vec<RuntimeDiagnostic>,
available_count: usize,
incompatible_count: usize, incompatible_count: usize,
unknown_count: usize, unknown_count: usize,
} }
impl RuntimeCompatibilityObservation { impl RuntimeCompatibilityObservation {
fn available(&mut self, operation: &str, _message: &str) { fn available(&mut self, operation: &str, message: impl Into<String>) {
self.available_count += 1;
self.capabilities.push(format!("{operation}:available")); self.capabilities.push(format!("{operation}:available"));
self.diagnostics.push(settings_diagnostic(
format!("{operation}.available"),
DiagnosticSeverity::Info,
message,
));
} }
fn unknown(&mut self, operation: &str, message: impl Into<String>) { fn unknown(&mut self, operation: &str, message: impl Into<String>) {
@ -1775,8 +1873,6 @@ impl RuntimeCompatibilityObservation {
fn state(&self) -> &'static str { fn state(&self) -> &'static str {
if self.incompatible_count > 0 { if self.incompatible_count > 0 {
"incompatible" "incompatible"
} else if self.unknown_count > 0 {
"unknown"
} else { } else {
"compatible" "compatible"
} }
@ -2144,6 +2240,9 @@ impl IntoResponse for ApiError {
Error::RuntimeOperationFailed { code, .. } if code == "remote_runtime_unsupported" => { Error::RuntimeOperationFailed { code, .. } if code == "remote_runtime_unsupported" => {
StatusCode::NOT_IMPLEMENTED StatusCode::NOT_IMPLEMENTED
} }
Error::RuntimeOperationFailed { code, .. } if code.ends_with("_blocked") => {
StatusCode::CONFLICT
}
Error::RuntimeOperationFailed { code, .. } Error::RuntimeOperationFailed { code, .. }
if code.starts_with("workspace_settings_") if code.starts_with("workspace_settings_")
|| code.starts_with("invalid_") || code.starts_with("invalid_")
@ -2365,7 +2464,7 @@ mod tests {
} }
#[tokio::test] #[tokio::test]
async fn runtime_connection_settings_add_delete_persist_restart_required() { async fn runtime_connection_settings_add_delete_apply_live_registry() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let app = test_app(dir.path()).await; let app = test_app(dir.path()).await;
@ -2383,9 +2482,16 @@ mod tests {
}), }),
) )
.await; .await;
assert_eq!(added["restart_required"], true); assert_eq!(added["restart_required"], false);
assert_eq!(added["remotes"][0]["runtime_id"], "team-runtime"); assert_eq!(added["remotes"][0]["runtime_id"], "team-runtime");
assert_eq!(added["remotes"][0]["endpoint_configured"], true); assert_eq!(added["remotes"][0]["endpoint_configured"], true);
assert!(
added["diagnostics"]
.as_array()
.unwrap()
.iter()
.any(|diagnostic| diagnostic["code"] == "runtime_registry_applied")
);
let projected = serde_json::to_string(&added).unwrap(); let projected = serde_json::to_string(&added).unwrap();
assert!(!projected.contains("runtime.example.invalid")); assert!(!projected.contains("runtime.example.invalid"));
@ -2397,22 +2503,113 @@ mod tests {
"https://runtime.example.invalid" "https://runtime.example.invalid"
); );
let launch_options = get_json(app.clone(), "/api/workers/launch-options").await;
assert!(
launch_options["runtimes"]
.as_array()
.unwrap()
.iter()
.any(|runtime| runtime["runtime_id"] == "team-runtime")
);
let deleted = request_json( let deleted = request_json(
app, app.clone(),
"DELETE", "DELETE",
"/api/settings/runtime-connections/remotes/team-runtime", "/api/settings/runtime-connections/remotes/team-runtime",
None, None,
StatusCode::OK, StatusCode::OK,
) )
.await; .await;
assert_eq!(deleted["restart_required"], true); assert_eq!(deleted["restart_required"], false);
assert_eq!(deleted["remotes"].as_array().unwrap().len(), 0); assert_eq!(deleted["remotes"].as_array().unwrap().len(), 0);
let launch_options = get_json(app.clone(), "/api/workers/launch-options").await;
assert!(
!launch_options["runtimes"]
.as_array()
.unwrap()
.iter()
.any(|runtime| runtime["runtime_id"] == "team-runtime")
);
let persisted = WorkspaceBackendConfigFile::load_for_workspace(dir.path()).unwrap(); let persisted = WorkspaceBackendConfigFile::load_for_workspace(dir.path()).unwrap();
assert!(persisted.runtimes.remote.is_empty()); assert!(persisted.runtimes.remote.is_empty());
} }
#[tokio::test] #[tokio::test(flavor = "multi_thread")]
async fn runtime_connection_test_reports_observed_and_unknown_capabilities_without_endpoint_leak() async fn runtime_connection_delete_rejects_active_remote_workers() {
let (runtime, _worker_ref) = runtime_with_worker();
let runtime_listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let runtime_addr = runtime_listener.local_addr().unwrap();
tokio::spawn({
let runtime = runtime.clone();
async move {
worker_runtime::http_server::serve_runtime_http(runtime, runtime_listener, None)
.await
.unwrap()
}
});
let dir = tempfile::tempdir().unwrap();
let app = test_app(dir.path()).await;
let added = post_json(
app.clone(),
"/api/settings/runtime-connections/remotes",
serde_json::json!({
"runtime_id": "busy-runtime",
"display_name": "Busy Runtime",
"endpoint": format!("http://{runtime_addr}")
}),
)
.await;
assert_eq!(added["restart_required"], false);
let created = post_json(
app.clone(),
"/api/workers",
serde_json::json!({
"runtime_id": "busy-runtime",
"display_name": "Remote Test Worker",
"profile": "runtime_default",
"initial_text": ""
}),
)
.await;
assert_eq!(created["runtime_id"], "busy-runtime");
let workers = get_json(app.clone(), "/api/workers").await;
assert!(
workers["items"]
.as_array()
.unwrap()
.iter()
.any(|worker| worker["runtime_id"] == "busy-runtime"),
"expected remote runtime to report at least one worker, got {workers}"
);
let response = request_json(
app,
"DELETE",
"/api/settings/runtime-connections/remotes/busy-runtime",
None,
StatusCode::CONFLICT,
)
.await;
assert!(
response["message"]
.as_str()
.unwrap()
.contains("remote_runtime_delete_blocked")
);
assert!(
response["diagnostics"]
.as_array()
.unwrap()
.iter()
.any(|diagnostic| { diagnostic["code"] == "remote_runtime_delete_blocked" })
);
let persisted = WorkspaceBackendConfigFile::load_for_workspace(dir.path()).unwrap();
assert_eq!(persisted.runtimes.remote.len(), 1);
}
#[tokio::test(flavor = "multi_thread")]
async fn runtime_connection_test_reports_compatible_with_unknown_warnings_without_endpoint_leak()
{ {
let (runtime, _worker_ref) = runtime_with_worker(); let (runtime, _worker_ref) = runtime_with_worker();
let runtime_listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); let runtime_listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
@ -2449,7 +2646,7 @@ mod tests {
serde_json::json!({}), serde_json::json!({}),
) )
.await; .await;
assert_eq!(response["state"], "unknown"); assert_eq!(response["state"], "compatible");
let capabilities = response["capabilities"].as_array().unwrap(); let capabilities = response["capabilities"].as_array().unwrap();
assert!( assert!(
capabilities capabilities
@ -2464,14 +2661,14 @@ mod tests {
assert!( assert!(
capabilities capabilities
.iter() .iter()
.any(|value| value == "workers.spawn:unknown") .any(|value| value == "workers.spawn:available")
); );
assert!( assert!(
response["diagnostics"] response["diagnostics"]
.as_array() .as_array()
.unwrap() .unwrap()
.iter() .iter()
.any(|diagnostic| { diagnostic["code"] == "workers.spawn.unknown" }) .any(|diagnostic| { diagnostic["code"] == "workers.spawn.available" })
); );
let projected = serde_json::to_string(&response).unwrap(); let projected = serde_json::to_string(&response).unwrap();
assert!(!projected.contains(&endpoint)); assert!(!projected.contains(&endpoint));
@ -2479,6 +2676,60 @@ mod tests {
assert_eq!(response["protocol_version"], serde_json::Value::Null); assert_eq!(response["protocol_version"], serde_json::Value::Null);
} }
#[tokio::test(flavor = "multi_thread")]
async fn runtime_connection_test_marks_missing_execution_backend_incompatible() {
let runtime =
worker_runtime::Runtime::with_options(worker_runtime::RuntimeOptions::default());
let runtime_listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let runtime_addr = runtime_listener.local_addr().unwrap();
tokio::spawn(async move {
worker_runtime::http_server::serve_runtime_http(runtime, runtime_listener, None)
.await
.unwrap()
});
let dir = tempfile::tempdir().unwrap();
let endpoint = format!("http://{runtime_addr}");
WorkspaceBackendConfigFile {
runtimes: crate::config::WorkspaceBackendRuntimesConfig {
remote: vec![RemoteRuntimeConfigFile {
id: "control-only-runtime".to_string(),
display_name: Some("Control-only Runtime".to_string()),
endpoint,
token_ref: None,
}],
},
..WorkspaceBackendConfigFile::default()
}
.write_for_workspace(dir.path())
.unwrap();
let app = test_app(dir.path()).await;
let response = post_json(
app,
"/api/settings/runtime-connections/remotes/control-only-runtime/test",
serde_json::json!({}),
)
.await;
assert_eq!(response["state"], "incompatible");
assert!(
response["capabilities"]
.as_array()
.unwrap()
.iter()
.any(|value| { value == "workers.spawn:incompatible" })
);
assert!(
response["diagnostics"]
.as_array()
.unwrap()
.iter()
.any(|diagnostic| {
diagnostic["code"] == "remote_runtime_worker_creation_unavailable"
})
);
}
#[tokio::test] #[tokio::test]
async fn browser_worker_create_succeeds_and_preserves_unsupported_diagnostics() { async fn browser_worker_create_succeeds_and_preserves_unsupported_diagnostics() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();

View File

@ -46,6 +46,6 @@ max_records = 200
# #
# [[runtimes.remote]] # [[runtimes.remote]]
# id = "example" # id = "example"
# endpoint = "http://127.0.0.1:8790" # endpoint = "http://127.0.0.1:38800"
# display_name = "Example Runtime" # display_name = "Example Runtime"
# token_ref = "local:example-runtime-token" # token_ref = "local:example-runtime-token"

View File

@ -184,6 +184,13 @@
} }
} }
function capabilityOperations(test: RemoteRuntimeTestResponse, state: "available" | "unknown" | "incompatible"): string[] {
const suffix = `:${state}`;
return test.capabilities
.filter((capability) => capability.endsWith(suffix))
.map((capability) => capability.slice(0, -suffix.length));
}
function applyRuntimeMutation(data: RuntimeConnectionMutationResponse) { function applyRuntimeMutation(data: RuntimeConnectionMutationResponse) {
runtimeSettings = runtimeSettings runtimeSettings = runtimeSettings
? { ...runtimeSettings, remotes: data.remotes, diagnostics: data.diagnostics } ? { ...runtimeSettings, remotes: data.remotes, diagnostics: data.diagnostics }
@ -335,10 +342,18 @@
</div> </div>
{#if tests[remote.runtime_id]} {#if tests[remote.runtime_id]}
{@const test = tests[remote.runtime_id]} {@const test = tests[remote.runtime_id]}
{@const available = capabilityOperations(test, "available")}
{@const unchecked = capabilityOperations(test, "unknown")}
<div class="settings-test-result"> <div class="settings-test-result">
<strong>Test: {test.state}</strong> <strong>Test: {test.state}</strong>
<span>{test.health_result} · {test.checked_at}</span> <span>{test.health_result} · {test.checked_at}</span>
<p>{test.compatibility_basis}</p> <p>{test.compatibility_basis}</p>
{#if available.length > 0}
<p class="settings-test-verified">Verified areas: {available.join(', ')}</p>
{/if}
{#if unchecked.length > 0}
<p class="settings-test-verified">Unchecked warning areas: {unchecked.join(', ')}</p>
{/if}
{@render DiagnosticsList({ diagnostics: test.diagnostics })} {@render DiagnosticsList({ diagnostics: test.diagnostics })}
</div> </div>
{/if} {/if}