merge: compose T748 bounded execution with T743 public callers

docs/development/backend-db-execution.md

@@ -0,0 +1,115 @@
+# Backend bounded synchronous execution
+
+## Ownership and admission
+
+`ControlPlaneStore::db_executor()` returns a clone of the bounded execution owner.
+Use `executor.run(move || synchronous_operation(...)).await` with owned input and
+services. The closure owns the **whole** synchronous operation, including connection
+acquisition, mutex wait, the existing SQL transaction, commit, and ordered local
+publication. Do not split source authorization, receipts, CAS, audit, or saved
+results around an await. Do not change connection ownership or introduce a read pool.
+
+Production stores share one DB worker and a queue of 32 jobs. Admission uses a
+nonblocking queue send and a separate actual-job budget (worker count plus queue
+capacity); rejection means the operation was **not accepted**. HTTP maps capacity
+and executor unavailability to 503, not authentication failure or deletion conflict.
+Tests can inject a dedicated executor with explicit limits, without process-global
+settings or environment variables. Ordinary SQLite fixtures use `open_for_test`,
+`in_memory_for_test`, or `from_connection_for_test`: their clones share their own
+1-worker/32-queued owner, not the production singleton or another fixture's drain.
+Server/Workspace fixture constructors likewise inject a lazy `HostExecutors` owner;
+all child Workspaces and weak retention adapters use that same owner. Host and
+cleanup still have the production 2-worker/16-queued limits. Dropping the final
+sender closes its queue after accepted work finishes; idle fixture threads are
+not kept in a process-global fixture cache.
+
+The restore intent journal and Workspace deletion use
+`with_immediate_transaction`: read authority/journal state only after acquiring
+SQLite write ownership. A DEFERRED read-to-write upgrade can return BUSY without
+invoking SQLite's busy handler when another connection is writing. Moving these
+whole helpers to an offloaded caller must not create an automatic mutation retry.
+
+
+Accepted work is owned by the worker, not the Tokio waiter. Cancelling a waiter:
+
+- does not interrupt, resubmit, or retry SQL;
+- does not return its queue/job slot before execution and discarded-result cleanup;
+- does not release the request lifecycle lease before actual completion.
+
+Synchronous post-commit continuations inherit the admitted lifecycle source when
+submitting another bounded job. This is not implicit inheritance by detached Tokio
+tasks. An operation panic has an unknown outcome even if SQL already committed;
+return a structured unknown-outcome error and diagnostic, never retry the mutation.
+A cancelled fire-and-forget job still emits its diagnostic. `drain()` observes all
+jobs accepted before its barrier is first polled and waits for actual cleanup; it
+is not a queue close and does not include later admissions. Lifecycle callers must
+stop their producers/admission before destroying resources.
+
+## Runtime/network boundaries
+
+A synchronous Runtime workflow must not monopolize the sole DB worker while
+waiting for callbacks that themselves need DB authority. The Server's separate
+finite Host lane has two workers and 16 queued jobs. It uses the same accepted-job
+ownership mechanism, not an unbounded `spawn_blocking` queue.
+
+Pure DB stages use the DB lane. Existing mixed resource-lifecycle workflows can
+use the Host lane, with no Runtime I/O or await inside a SQL transaction. Where an
+owned workflow needs async resource locks, `run_host_workflow` polls it on a Host
+worker using the original Tokio handle; do not build a replacement runtime for
+cached services and do not submit to and await the Host lane from inside itself.
+Control/session guards needed by an accepted mutation belong to that actual job.
+
+Backend Job cleanup has a third finite lane (two workers, 16 queued jobs), so a
+long Run teardown wait cannot starve ordinary Host control/stop requests. It is
+admitted before its durable cleanup claim. Rejection leaves Pending/Failed
+ownership recoverable instead of abandoning an Executing claim. Ticket
+checker dispatch and timeout processing use bounded admission; overload leaves the
+saved reservation/deadline diagnosable. They are not automatically retried after an
+unknown DB outcome. Runtime teardown is still its own lifecycle fence, not proof
+of model processing.
+
+Remote subscription and Workdir authorization are async callbacks: their complete
+live binding check and fresh request signing are offloaded together. A queue error
+must not fall back to a configured bearer token. Synchronous CLI/migration callers
+and existing synchronous Runtime adapters retain their explicit blocking boundary.
+
+## Caller recipe (including assignment APIs)
+
+```rust
+let store = api.store.clone();
+let workspace_id = api.workspace_id().to_owned();
+let worker = worker.clone();
+let record = api.store.db_executor().run(move || {
+    store.get_worker_registry(&workspace_id, &worker)
+}).await?;
+```
+
+This is an illustration, not a new authority API. Assignment delivery scan,
+reserve, prepare, acknowledgement and start authorization must keep their existing
+whole transaction helpers. Additional assignment diagnostics, answer mutations,
+and Ticket queries use the same owned caller boundary; do not add a second ledger
+or reconstruct processing evidence from Worker liveness.
+
+## Regression evidence
+
+- `db_executor::tests`: real SQLite IMMEDIATE write contention while a current-thread
+  timer/control task progresses; accepted/cancelled queue bounds; rollback; panic
+  before/after commit; discarded reply cleanup and retained authority; child jobs.
+- `server::db_execution_tests`: HTTP Ticket writes and cold/warm Subjektiv scope
+  opening/mutation under a separate SQLite connection's write lock; overload;
+  cancelled HTTP jobs and Workspace deletion drain; same-budget fixture isolation,
+  saturated/cancelled Host jobs versus cleanup and paused Tokio time; restore intent
+  contention before the first authority read, with one persisted tuple and no retry.
+  The terminal checker fixture keeps its original Tokio driver alive through accepted cleanup,
+  then shuts down projection producers and drains actual DB jobs.
+- `worker_projection::tests`: real SQL contention, commit/payload publication,
+  ordered snapshot registration, and replacement draining a cancelled consumer's
+  accepted SQL before applying a new Runtime snapshot.
+- Auth/proof and Runtime subscription tests protect availability classification,
+  replay rejection, and no bearer fallback.
+- `workdir`'s `http-client` tests protect awaited fresh authorization and exact
+  request claims before any HTTP send.
+
+These fixtures use temporary authority data and observable synchronization, not
+live databases or a fixed sleep to guess completion. This change does not claim a
+latency benchmark or a measured production incident.