243 lines
6.9 KiB
Markdown
243 lines
6.9 KiB
Markdown
# Development
|
|
|
|
This document describes the development workflow for Decodal itself.
|
|
|
|
## Rust checks
|
|
|
|
Run the normal Rust checks from the repository root.
|
|
|
|
```sh
|
|
cargo fmt --check
|
|
cargo test
|
|
cargo check -p decodal --no-default-features
|
|
nix flake check
|
|
```
|
|
|
|
Run the default stdio language server with:
|
|
|
|
```sh
|
|
cargo run -q -p decodal-lsp
|
|
```
|
|
|
|
The protocol integration tests use an in-memory LSP connection and can be run independently:
|
|
|
|
```sh
|
|
cargo test -p decodal-lsp
|
|
```
|
|
|
|
Regex support is optional and should be tested explicitly when touched.
|
|
|
|
```sh
|
|
cargo test -p decodal --features regex
|
|
cargo run -q -p decodal-cli --features regex -- examples/regex/main.dcdl
|
|
```
|
|
|
|
## crates.io release
|
|
|
|
The crates.io release contains `decodal`, `decodal-derive`, `decodal-language-service`, `decodal-language-tools`, and `decodal-lsp`.
|
|
`decodal-cli` and the Rust source crate `decodal-wasm` remain repository-only packages.
|
|
The generated WebAssembly package under `packages/decodal-wasm/` is published to npm and JSR.
|
|
The CodeMirror package bundles the generated formatter WebAssembly from `decodal-language-tools`.
|
|
The project is dual licensed as `MIT OR Apache-2.0`.
|
|
|
|
The authoritative validation commands, dependency order, and publish commands are maintained in `RELEASING.md` at the repository root.
|
|
|
|
## Web site and playground
|
|
|
|
The Astro documentation site and browser playground are kept in:
|
|
|
|
```text
|
|
site/decodal-site/
|
|
```
|
|
|
|
The site imports Markdown files from `doc/manual/souce/` and renders them as mdBook-style pages.
|
|
The playground loads `decodal-wasm` for evaluation and `decodal-codemirror` for editor features and source formatting.
|
|
|
|
Important files:
|
|
|
|
```text
|
|
site/decodal-site/src/pages/docs/[...slug].astro
|
|
site/decodal-site/src/pages/playground.astro
|
|
site/decodal-site/src/scripts/playground.js
|
|
site/decodal-site/src/scripts/playground-examples.js
|
|
site/decodal-site/src/layouts/ManualLayout.astro
|
|
site/decodal-site/src/lib/docs.js
|
|
site/decodal-site/src/lib/highlight.js
|
|
packages/decodal-codemirror/src/decodal.js
|
|
packages/decodal-codemirror/src/decodal-parser.js
|
|
packages/decodal-wasm/decodal_wasm.js
|
|
packages/decodal-wasm/decodal_wasm_bg.wasm
|
|
packages/decodal-codemirror/wasm/decodal_language_tools.js
|
|
packages/decodal-codemirror/wasm/decodal_language_tools_bg.wasm
|
|
crates/decodal-wasm/src/lib.rs
|
|
crates/decodal-language-tools/src/lib.rs
|
|
```
|
|
|
|
Build the WebAssembly packages before building the site:
|
|
|
|
```sh
|
|
cd site/decodal-site
|
|
npm install
|
|
npm run build:wasm
|
|
npm run build
|
|
```
|
|
|
|
`npm run build:wasm` builds both generated WebAssembly packages.
|
|
`npm run build:wasm:runtime` writes generated runtime files into `packages/decodal-wasm/`.
|
|
`npm run build:wasm:formatter` writes generated formatter files into `packages/decodal-codemirror/wasm/`.
|
|
`npm run build` builds only the static site from the current package files.
|
|
Use `npm run build:all` to regenerate both WebAssembly packages and then build the static site in one command.
|
|
|
|
These generated files are committed so the site can be built without requiring every consumer to regenerate the wasm packages first.
|
|
|
|
Publish the generated WebAssembly package from its package directory:
|
|
|
|
```sh
|
|
cd packages/decodal-wasm
|
|
npm publish
|
|
npx jsr publish
|
|
```
|
|
|
|
Run dry-runs first when preparing a release:
|
|
|
|
```sh
|
|
npm pack --dry-run
|
|
npx jsr publish --dry-run
|
|
```
|
|
|
|
The npm package name is `decodal-wasm`.
|
|
The JSR package name is `@hare/decodal-wasm`.
|
|
The npm package advertises `MIT OR Apache-2.0`; JSR metadata uses `MIT` because its publisher requires a single recognized license identifier. Both license files are included in the package.
|
|
|
|
The playground editor uses the `decodal-codemirror` package with the generated Lezer parser in `packages/decodal-codemirror/src/decodal-parser.js`.
|
|
The canonical grammar is documented in `doc/manual/souce/language/grammar.md`; regenerate the Lezer parser when that grammar or `editors/lezer-decodal/decodal.grammar` changes.
|
|
|
|
Publish the CodeMirror package from its package directory:
|
|
|
|
```sh
|
|
cd packages/decodal-codemirror
|
|
npm install
|
|
npm publish
|
|
npx jsr publish
|
|
```
|
|
|
|
Run dry-runs first when preparing a release:
|
|
|
|
```sh
|
|
npm install
|
|
npm pack --dry-run
|
|
npx jsr publish --dry-run
|
|
```
|
|
|
|
The npm package name is `decodal-codemirror`.
|
|
The JSR package name is `@hare/decodal-codemirror`.
|
|
As with `decodal-wasm`, JSR metadata uses `MIT` while both the MIT and Apache-2.0 license files remain included.
|
|
|
|
The documentation build still uses the lightweight JavaScript fallback highlighter so Astro can render Markdown without initializing WASM at build time.
|
|
|
|
To run the site locally:
|
|
|
|
```sh
|
|
cd site/decodal-site
|
|
npm run dev
|
|
```
|
|
|
|
Deploy the static site to Cloudflare Pages with Wrangler direct upload:
|
|
|
|
```sh
|
|
cd site/decodal-site
|
|
npm run deploy
|
|
```
|
|
|
|
The deploy script runs `npm run build` and then uploads `dist/` to the Pages project named `decodal-site` on branch `master`.
|
|
Cloudflare Pages treats this as production when the project production branch is `master`.
|
|
Use `CLOUDFLARE_PROJECT_NAME` when deploying to a differently named Pages project.
|
|
|
|
```sh
|
|
CLOUDFLARE_PROJECT_NAME=my-pages-project npm run deploy
|
|
```
|
|
|
|
If the committed WASM package must be regenerated before deploy, use:
|
|
|
|
```sh
|
|
npm run deploy:all
|
|
```
|
|
|
|
## Tree-sitter grammar
|
|
|
|
The Tree-sitter grammar is kept in:
|
|
|
|
```text
|
|
editors/tree-sitter-decodal/
|
|
```
|
|
|
|
Important files:
|
|
|
|
```text
|
|
editors/tree-sitter-decodal/grammar.js
|
|
editors/tree-sitter-decodal/queries/highlights.scm
|
|
editors/tree-sitter-decodal/queries/locals.scm
|
|
editors/tree-sitter-decodal/corpus/basic.txt
|
|
```
|
|
|
|
The grammar is intended to be portable across editors such as Zed, Neovim, Helix, and Emacs.
|
|
Zed support should consume this grammar rather than relying on a TextMate grammar.
|
|
|
|
## Tree-sitter commands
|
|
|
|
From the grammar directory:
|
|
|
|
```sh
|
|
cd editors/tree-sitter-decodal
|
|
npm install
|
|
npx tree-sitter generate
|
|
npx tree-sitter test
|
|
```
|
|
|
|
To inspect a parse tree:
|
|
|
|
```sh
|
|
npx tree-sitter parse ../../examples/advanced/main.dcdl
|
|
```
|
|
|
|
The generated parser files under `editors/tree-sitter-decodal/src/` are committed so editor integrations can consume the grammar without regenerating it first.
|
|
`node_modules/` is ignored and must not be committed.
|
|
|
|
## Updating the grammar
|
|
|
|
When the Decodal syntax changes:
|
|
|
|
1. Update the canonical EBNF in `doc/manual/souce/language/grammar.md`.
|
|
2. Update the Rust parser/lexer as needed.
|
|
3. Update `editors/tree-sitter-decodal/grammar.js` and run `npx tree-sitter generate` / `npx tree-sitter test`.
|
|
4. Update `editors/lezer-decodal/decodal.grammar` and regenerate the CodeMirror parser:
|
|
|
|
```sh
|
|
cd site/decodal-site
|
|
npx lezer-generator ../../editors/lezer-decodal/decodal.grammar -o ../../packages/decodal-codemirror/src/decodal-parser.js
|
|
```
|
|
|
|
5. Add or update corpus/tests/examples.
|
|
6. Run Rust and site checks from the repository root.
|
|
|
|
## Development shell
|
|
|
|
The Nix development shell includes Rust tooling, Node.js, Tree-sitter CLI tooling, and wasm-pack tooling.
|
|
|
|
```sh
|
|
nix develop
|
|
```
|
|
|
|
The shell provides:
|
|
|
|
- `cargo`
|
|
- `rustc`
|
|
- `rustfmt`
|
|
- `clippy`
|
|
- `node`
|
|
- `npm`
|
|
- `wasm-pack`
|
|
- `lld`
|
|
- `tree-sitter`
|
|
- `nixfmt`
|