Document Decodal components

doc/manual/souce/components.md

@@ -0,0 +1,94 @@
+# Components
+
+Decodal is split into a small runtime core and separate syntax tooling components.
+The public surface is intentionally organized by use case: execute Decodal with Rust or WebAssembly, edit Decodal on the Web with Lezer and CodeMirror, and integrate Decodal into general-purpose editors with Tree-sitter.
+
+## Runtime components
+
+### Rust crate
+
+The `decodal` crate is the primary Rust runtime and embedding API.
+It owns parsing, evaluation, materialization, diagnostics, host-provided values, and schema/decode traits.
+Rust applications should use this crate when they want to load Decodal source, evaluate it, or embed Decodal into a host program.
+
+Important paths:
+
+```text
+crates/decodal-core/
+crates/decodal-derive/
+```
+
+`decodal-derive` provides optional derive macros for Rust struct integration.
+It is a companion to the runtime crate rather than an editor or syntax-highlighting component.
+
+### WebAssembly package
+
+`decodal-wasm` exposes the runtime to browsers.
+The documentation site playground uses it to evaluate Decodal entirely in the browser.
+
+Important paths:
+
+```text
+crates/decodal-wasm/
+site/decodal-site/src/wasm/
+```
+
+The generated files in `site/decodal-site/src/wasm/` are committed so the site can build without requiring every consumer to run `wasm-pack` first.
+The WebAssembly package is for execution, not syntax highlighting.
+
+## Web editor components
+
+The Web playground editor uses CodeMirror 6 with a generated Lezer parser.
+This is the source of syntax highlighting, folding, indentation, and editor syntax tree behavior in the browser UI.
+
+Important paths:
+
+```text
+editors/lezer-decodal/decodal.grammar
+site/decodal-site/src/lib/codemirror/decodal-parser.js
+site/decodal-site/src/lib/codemirror/decodal-parser.terms.js
+site/decodal-site/src/lib/codemirror/decodal.js
+```
+
+The Lezer grammar is derived from the canonical grammar documentation, but it is not a literal copy of the EBNF.
+Precedence and token conflict handling are represented in the Lezer grammar in the form CodeMirror needs.
+
+## General editor components
+
+Tree-sitter is the portable editor-integration grammar.
+Editors such as Zed, Neovim, Helix, and Emacs should consume this component when they need Decodal parsing or highlighting outside the Web playground.
+
+Important paths:
+
+```text
+editors/tree-sitter-decodal/grammar.js
+editors/tree-sitter-decodal/queries/highlights.scm
+editors/tree-sitter-decodal/queries/locals.scm
+editors/tree-sitter-decodal/src/
+```
+
+The generated parser sources under `editors/tree-sitter-decodal/src/` are committed so editor integrations can consume the grammar without regenerating it first.
+
+## Canonical grammar
+
+The human-readable grammar lives in:
+
+```text
+doc/manual/souce/language/grammar.md
+```
+
+This EBNF is the language-level reference.
+The Rust parser, Lezer grammar, and Tree-sitter grammar should be kept aligned with it, but each implementation may encode precedence and recovery behavior in the form required by its parser generator or runtime.
+
+## What is not a public component
+
+The Rust lexer is an implementation detail of the Rust parser.
+Decodal does not expose a standalone public tokenizer API for editor tooling.
+Consumers that need syntax information should use the component that matches their environment:
+
+- Rust execution and embedding: `decodal`
+- Browser execution: `decodal-wasm`
+- Web editor syntax: Lezer / CodeMirror
+- General editor syntax: Tree-sitter
+
+This avoids having a separate token stream API whose behavior would have to be kept compatible with both runtime parsing and editor grammars.

doc/manual/souce/development.md

@@ -162,7 +162,7 @@ The generated parser files under `editors/tree-sitter-decodal/src/` are committe
 When the Decodal syntax changes:
 
 1. Update the canonical EBNF in `doc/manual/souce/language/grammar.md`.
-2. Update the Rust parser/tokenizer as needed.
+2. Update the Rust parser/lexer as needed.
 3. Update `editors/tree-sitter-decodal/grammar.js` and run `npx tree-sitter generate` / `npx tree-sitter test`.
 4. Update `editors/lezer-decodal/decodal.grammar` and regenerate the CodeMirror parser:
 

doc/manual/souce/index.md

@@ -6,7 +6,8 @@ Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェク
 ## 目次
 
 1. [Introduction](./introduction.md)
-2. [Language Specification](./language/index.md)
+2. [Components](./components.md)
+3. [Language Specification](./language/index.md)
    1. [Lexical Structure and Syntax](./language/syntax.md)
    2. [Value](./language/value/index.md)
       1. [String](./language/value/string.md)
@@ -35,7 +36,7 @@ Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェク
    9. [Materialization and Errors](./language/materialization-and-errors.md)
    10. [Naming Conventions](./language/naming.md)
    11. [Examples](./language/examples.md)
-3. [Implementation Design](./design/index.md)
+4. [Implementation Design](./design/index.md)
    1. [Execution Pipeline](./design/execution-pipeline.md)
    2. [Runtime Model](./design/runtime-model.md)
    3. [Thunk and Lazy Evaluation](./design/thunk-and-lazy-evaluation.md)
@@ -43,5 +44,5 @@ Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェク
    5. [Diagnostics and Fallback](./design/diagnostics-and-fallback.md)
    6. [Embedding API](./design/embedding-api.md)
    7. [Features](./design/features.md)
-4. [Development](./development.md)
-5. [Open Issues](./open-issues.md)
+5. [Development](./development.md)
+6. [Open Issues](./open-issues.md)

site/decodal-site/src/lib/docs.js

@@ -19,6 +19,7 @@ export const docs = Object.fromEntries(
 
 export const nav = [
   { title: 'Introduction', slug: 'introduction' },
+  { title: 'Components', slug: 'components' },
   {
     title: 'Language Specification',
     slug: 'language',