138 lines
5.3 KiB
Markdown
138 lines
5.3 KiB
Markdown
# Components
|
|
|
|
Decodal is split into a small runtime core and separate syntax tooling components.
|
|
The public surface is intentionally organized by use case: execute Decodal with Rust or WebAssembly, edit Decodal on the Web with Lezer and CodeMirror, and integrate Decodal into general-purpose editors with Tree-sitter.
|
|
|
|
## Runtime components
|
|
|
|
### Rust crate
|
|
|
|
The `decodal` crate is the primary Rust runtime and embedding API.
|
|
It owns parsing, evaluation, materialization, diagnostics, host-provided values, and schema/decode traits.
|
|
Rust applications should use this crate when they want to load Decodal source, evaluate it, or embed Decodal into a host program.
|
|
|
|
Important paths:
|
|
|
|
```text
|
|
crates/decodal-core/
|
|
crates/decodal-derive/
|
|
```
|
|
|
|
`decodal-derive` provides optional derive macros for Rust struct integration.
|
|
It is a companion to the runtime crate rather than an editor or syntax-highlighting component.
|
|
|
|
### WebAssembly package
|
|
|
|
`decodal-wasm` exposes the runtime to browsers.
|
|
The documentation site playground uses it to evaluate Decodal entirely in the browser.
|
|
|
|
Important paths:
|
|
|
|
```text
|
|
crates/decodal-wasm/
|
|
packages/decodal-wasm/
|
|
```
|
|
|
|
The npm package is `decodal-wasm`.
|
|
The JSR package is `@hare/decodal-wasm`.
|
|
|
|
The generated files in `packages/decodal-wasm/` are committed so the site can build without requiring every consumer to run `wasm-pack` first.
|
|
The WebAssembly package is for execution, not syntax highlighting.
|
|
|
|
## Language tools
|
|
|
|
Semantic editor integration lives in the host-configurable language service crate.
|
|
It depends only on the runtime and accepts the same `HostEnvironment` implementation used by a production application.
|
|
The LSP crate adapts that service to the Language Server Protocol over stdin/stdout.
|
|
It provides full document synchronization, semantic diagnostics, and whole-document formatting.
|
|
A host-specific LSP binary injects its loader and global schema configuration without reimplementing evaluation rules.
|
|
|
|
Important paths:
|
|
|
|
```text
|
|
crates/decodal-language-service/
|
|
crates/decodal-lsp/
|
|
```
|
|
|
|
The default `decodal-lsp` binary reads Decodal imports from the filesystem.
|
|
Embedded hosts can call its library entry point with a custom `LspEnvironment` to reuse structured imports and to make unsaved external documents, such as Markdown, visible to the loader.
|
|
|
|
Source formatting lives in a separate Rust language tools crate.
|
|
Keeping it separate prevents host-specific semantic services from inheriting the formatter's Tree-sitter and WebAssembly dependencies.
|
|
|
|
This component is responsible for operations that must preserve source text details such as comments and whitespace.
|
|
It is used by the CodeMirror package's bundled formatter WebAssembly and can also be used by an LSP adapter for formatting.
|
|
|
|
Important paths:
|
|
|
|
```text
|
|
crates/decodal-language-tools/
|
|
```
|
|
|
|
The current language tools crate exposes the formatter.
|
|
|
|
## Web editor components
|
|
|
|
The Web playground editor uses CodeMirror 6 with a generated Lezer parser.
|
|
This is the source of syntax highlighting, folding, indentation, editor syntax tree behavior, and the browser formatter command in the browser UI.
|
|
|
|
Important paths:
|
|
|
|
```text
|
|
editors/lezer-decodal/decodal.grammar
|
|
packages/decodal-codemirror/src/decodal.js
|
|
packages/decodal-codemirror/src/decodal-parser.js
|
|
packages/decodal-codemirror/src/decodal-parser.terms.js
|
|
packages/decodal-codemirror/src/format.js
|
|
packages/decodal-codemirror/wasm/
|
|
```
|
|
|
|
The npm package is `decodal-codemirror`.
|
|
The JSR package is `@hare/decodal-codemirror`.
|
|
|
|
The Lezer grammar is derived from the canonical grammar documentation, but it is not a literal copy of the EBNF.
|
|
Precedence and token conflict handling are represented in the Lezer grammar in the form CodeMirror needs.
|
|
|
|
## General editor components
|
|
|
|
Tree-sitter is the portable editor-integration grammar.
|
|
Editors such as Zed, Neovim, Helix, and Emacs should consume this component when they need Decodal parsing or highlighting outside the Web playground.
|
|
|
|
Important paths:
|
|
|
|
```text
|
|
editors/tree-sitter-decodal/grammar.js
|
|
editors/tree-sitter-decodal/queries/highlights.scm
|
|
editors/tree-sitter-decodal/queries/locals.scm
|
|
editors/tree-sitter-decodal/src/
|
|
```
|
|
|
|
The generated parser sources under `editors/tree-sitter-decodal/src/` are committed so editor integrations can consume the grammar without regenerating it first.
|
|
|
|
## Canonical grammar
|
|
|
|
The human-readable grammar lives in:
|
|
|
|
```text
|
|
doc/manual/souce/language/grammar.md
|
|
```
|
|
|
|
This EBNF is the language-level reference.
|
|
The Rust parser, Lezer grammar, and Tree-sitter grammar should be kept aligned with it, but each implementation may encode precedence and recovery behavior in the form required by its parser generator or runtime.
|
|
|
|
## What is not a public component
|
|
|
|
The Rust lexer is an implementation detail of the Rust parser.
|
|
Decodal does not expose a standalone public tokenizer API for editor tooling.
|
|
Consumers that need syntax information should use the component that matches their environment:
|
|
|
|
- Rust execution and embedding: `decodal`
|
|
- Browser execution: `decodal-wasm`
|
|
- Web formatting and editor syntax: Lezer / CodeMirror
|
|
- Semantic editor analysis: `decodal-language-service`
|
|
- Language Server Protocol integration: `decodal-lsp`
|
|
- Rust formatting: `decodal-language-tools`
|
|
- General editor syntax: Tree-sitter
|
|
|
|
This avoids having a separate token stream API whose behavior would have to be kept compatible with both runtime parsing and editor grammars.
|