6.4 KiB
Development
This document describes the development workflow for Decodal itself.
Rust checks
Run the normal Rust checks from the repository root.
cargo fmt --check
cargo test
cargo check -p decodal --no-default-features
nix flake check
Regex support is optional and should be tested explicitly when touched.
cargo test -p decodal --features regex
cargo run -q -p decodal-cli --features regex -- examples/regex/main.dcdl
crates.io release
The primary crates.io package is decodal, which contains the embeddable library.
decodal-derive provides optional derive macros for Rust struct integration and is published only when the derive crate changes.
Workspace support crates such as decodal-cli and the Rust source crate decodal-wasm are not published to crates.io.
The generated WebAssembly package under packages/decodal-wasm/ is published to npm and JSR.
The project is dual licensed as MIT OR Apache-2.0.
cargo publish -p decodal
Publish decodal-derive first only when that crate has a new version:
cargo publish -p decodal-derive
Before publishing, run:
cargo fmt --check
cargo test
cargo check -p decodal --no-default-features
cargo publish -p decodal --dry-run
If decodal-derive changed, also run cargo publish -p decodal-derive --dry-run.
Web site and playground
The Astro documentation site and browser playground are kept in:
site/decodal-site/
The site imports Markdown files from doc/manual/souce/ and renders them as mdBook-style pages.
The playground loads decodal-wasm and evaluates DCDL entirely in the browser.
Important files:
site/decodal-site/src/pages/docs/[...slug].astro
site/decodal-site/src/pages/playground.astro
site/decodal-site/src/scripts/playground.js
site/decodal-site/src/scripts/playground-examples.js
site/decodal-site/src/layouts/ManualLayout.astro
site/decodal-site/src/lib/docs.js
site/decodal-site/src/lib/highlight.js
packages/decodal-codemirror/src/decodal.js
packages/decodal-codemirror/src/decodal-parser.js
packages/decodal-wasm/decodal_wasm.js
packages/decodal-wasm/decodal_wasm_bg.wasm
crates/decodal-wasm/src/lib.rs
Build the WebAssembly package before building the site:
cd site/decodal-site
npm install
npm run build:wasm
npm run build
npm run build:wasm writes generated files into:
packages/decodal-wasm/
These generated files are committed so the site can be built without requiring every consumer to regenerate the wasm package first.
Publish the generated WebAssembly package from its package directory:
cd packages/decodal-wasm
npm publish
npx jsr publish
Run dry-runs first when preparing a release:
npm pack --dry-run
npx jsr publish --dry-run
The npm package name is decodal-wasm.
The JSR package name is @hare/decodal-wasm.
JSR currently expects a single SPDX license identifier in jsr.json; the WebAssembly package metadata uses MIT while the Rust workspace remains dual licensed as MIT OR Apache-2.0.
The playground editor uses the decodal-codemirror package with the generated Lezer parser in packages/decodal-codemirror/src/decodal-parser.js.
The canonical grammar is documented in doc/manual/souce/language/grammar.md; regenerate the Lezer parser when that grammar or editors/lezer-decodal/decodal.grammar changes.
Publish the CodeMirror package from its package directory:
cd packages/decodal-codemirror
npm install
npm publish
npx jsr publish
Run dry-runs first when preparing a release:
npm install
npm pack --dry-run
npx jsr publish --dry-run
The npm package name is decodal-codemirror.
The JSR package name is @hare/decodal-codemirror.
JSR currently expects a single SPDX license identifier in jsr.json; the CodeMirror package metadata uses MIT while the Rust workspace remains dual licensed as MIT OR Apache-2.0.
The documentation build still uses the lightweight JavaScript fallback highlighter so Astro can render Markdown without initializing WASM at build time.
To run the site locally:
cd site/decodal-site
npm run dev
Deploy the static site to Cloudflare Pages with Wrangler direct upload:
cd site/decodal-site
npm run deploy
The deploy script runs npm run build and then uploads dist/ to the Pages project named decodal-site on branch master.
Cloudflare Pages treats this as production when the project production branch is master.
Use CLOUDFLARE_PROJECT_NAME when deploying to a differently named Pages project.
CLOUDFLARE_PROJECT_NAME=my-pages-project npm run deploy
If the committed WASM package must be regenerated before deploy, use:
npm run deploy:wasm
Tree-sitter grammar
The Tree-sitter grammar is kept in:
editors/tree-sitter-decodal/
Important files:
editors/tree-sitter-decodal/grammar.js
editors/tree-sitter-decodal/queries/highlights.scm
editors/tree-sitter-decodal/queries/locals.scm
editors/tree-sitter-decodal/corpus/basic.txt
The grammar is intended to be portable across editors such as Zed, Neovim, Helix, and Emacs. Zed support should consume this grammar rather than relying on a TextMate grammar.
Tree-sitter commands
From the grammar directory:
cd editors/tree-sitter-decodal
npm install
npx tree-sitter generate
npx tree-sitter test
To inspect a parse tree:
npx tree-sitter parse ../../examples/advanced/main.dcdl
The generated parser files under editors/tree-sitter-decodal/src/ are committed so editor integrations can consume the grammar without regenerating it first.
node_modules/ is ignored and must not be committed.
Updating the grammar
When the Decodal syntax changes:
- Update the canonical EBNF in
doc/manual/souce/language/grammar.md. - Update the Rust parser/lexer as needed.
- Update
editors/tree-sitter-decodal/grammar.jsand runnpx tree-sitter generate/npx tree-sitter test. - Update
editors/lezer-decodal/decodal.grammarand regenerate the CodeMirror parser:
cd site/decodal-site
npx lezer-generator ../../editors/lezer-decodal/decodal.grammar -o ../../packages/decodal-codemirror/src/decodal-parser.js
- Add or update corpus/tests/examples.
- Run Rust and site checks from the repository root.
Development shell
The Nix development shell includes Rust tooling, Node.js, Tree-sitter CLI tooling, and wasm-pack tooling.
nix develop
The shell provides:
cargorustcrustfmtclippynodenpmwasm-packlldtree-sitternixfmt