27 lines
3.6 KiB
Markdown
27 lines
3.6 KiB
Markdown
# 2026-01-02 LLM Streaming & Hooks Research
|
||
|
||
## Summary Table
|
||
| Topic | Key Takeaways | Sources |
|
||
| --- | --- | --- |
|
||
| Fine-grained tool streaming | Anthropic beta header `fine-grained-tool-streaming-2025-05-14` streams tool parameters without intermediate JSON validation; reduces latency but may emit invalid/partial JSON that callers must sanitize. | Anthropic Docs – Fine-grained tool streaming (https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/fine-grained-tool-streaming) [turn1search0]; AWS Bedrock Anthropic tool-use reference (https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages-tool-use.html) [turn1search5]; Anthropic release notes (https://docs.anthropic.com/en/release-notes/api) [turn1search3]
|
||
| Streaming SSE events | Anthropic SSE stream emits typed events (`message_start`, `content_block_start`, `content_block_delta`, etc.) that clients must map to internal state machines for deterministic playback. | Anthropic Streaming Messages (https://docs.anthropic.com/en/docs/build-with-claude/streaming) [turn1search6]
|
||
| Tool streaming ergonomics | LangChain Anthropic integration exposes `betas=["fine-grained-tool-streaming-2025-05-14"]` and warns about invalid JSON, reinforcing need for resilient parsers. | LangChain Anthropic integration guide (https://docs.langchain.com/oss/python/integrations/chat/anthropic) [turn1search8]
|
||
| Hook architectures | Claude Code hook lifecycle (SessionStart, UserPromptSubmit, Tool Use, etc.) keeps hooks non-blocking, context-injecting, and failure-isolated—useful template for worker hooks/routers. | Claude-Mem hook architecture overview (https://docs.claude-mem.ai/hooks-architecture) [turn0search0]; Claude Blog – configuring hooks (https://claude.com/blog/how-to-configure-hooks) [turn0search2]
|
||
|
||
## Detailed Notes
|
||
|
||
### Fine-grained Tool Streaming
|
||
- Anthropic’s beta header `fine-grained-tool-streaming-2025-05-14` enables parameter streams, shrinking first-byte latency from ~15s to ~3s in their example. Clients must prepare for partial JSON and wrap invalid payloads before echoing them back. [turn1search0]
|
||
- AWS Bedrock mirrors the same header, confirming availability on Claude Sonnet 4.5/4 and Opus 4. Their docs explicitly caution about invalid/partial JSON and show the request schema. [turn1search5]
|
||
- Anthropic’s June 11, 2025 release notes document the beta launch, signaling freshness and likely API stability requirements. [turn1search3]
|
||
|
||
### SSE Event Model
|
||
- Anthropic exposes SSE event names plus JSON `type` fields; implementers should parse events like `content_block_start`, `content_block_delta`, and `message_stop` to drive a deterministic timeline. This justifies a dedicated timeline router/state machine as in the spec. [turn1search6]
|
||
|
||
### Tool Streaming Ergonomics in SDKs
|
||
- LangChain’s Anthropic integration demonstrates how third-party SDKs surface the beta header and reiterates the need for error handling when incomplete JSON arrives because `max_tokens` can stop a stream mid-parameter. This informs library-level abstractions for `scheme` and `llm_client` layers. [turn1search8]
|
||
|
||
### Hook / Lifecycle Design Patterns
|
||
- Claude-Mem’s architecture treats hooks as lifecycle-triggered closures that must stay non-blocking, degrade gracefully, and respect security constraints (frozen configs, permission prompts). This maps closely to the proposed `Tools/Hooks` system. [turn0search0]
|
||
- Claude’s official hook configuration guide enumerates eight hook types (PreToolUse, PostToolUse, PermissionRequest, SessionStart, Stop, etc.) and their contexts, reinforcing the need for a trait/macro system to statically describe hooks and route events. [turn0search2]
|