# Development

This document describes the development workflow for Decodal itself.

## Rust checks

Run the normal Rust checks from the repository root.

```sh
cargo fmt --check
cargo test
cargo check -p decodal --no-default-features
nix flake check
```

Run the default stdio language server with:

```sh
cargo run -q -p decodal-lsp
```

The protocol integration tests use an in-memory LSP connection and can be run independently:

```sh
cargo test -p decodal-lsp
```

Regex support is optional and should be tested explicitly when touched.

```sh
cargo test -p decodal --features regex
cargo run -q -p decodal-cli --features regex -- examples/regex/main.dcdl
```

## crates.io release

The crates.io release contains `decodal`, `decodal-derive`, `decodal-language-service`, `decodal-language-tools`, and `decodal-lsp`.
`decodal-cli` and the Rust source crate `decodal-wasm` remain repository-only packages.
The generated WebAssembly package under `packages/decodal-wasm/` is published to npm and JSR.
The CodeMirror package bundles the generated formatter WebAssembly from `decodal-language-tools`.
The project is dual licensed as `MIT OR Apache-2.0`.

The authoritative validation commands, dependency order, and publish commands are maintained in `RELEASING.md` at the repository root.

## Web site and playground

The Astro documentation site and browser playground are kept in:

```text
site/decodal-site/
```

The site imports Markdown files from `doc/manual/souce/` and renders them as mdBook-style pages.
The playground loads `decodal-wasm` for evaluation and `decodal-codemirror` for editor features and source formatting.

Important files:

```text
site/decodal-site/src/pages/docs/[...slug].astro
site/decodal-site/src/pages/playground.astro
site/decodal-site/src/scripts/playground.js
site/decodal-site/src/scripts/playground-examples.js
site/decodal-site/src/layouts/ManualLayout.astro
site/decodal-site/src/lib/docs.js
site/decodal-site/src/lib/highlight.js
packages/decodal-codemirror/src/decodal.js
packages/decodal-codemirror/src/decodal-parser.js
packages/decodal-wasm/decodal_wasm.js
packages/decodal-wasm/decodal_wasm_bg.wasm
packages/decodal-codemirror/wasm/decodal_language_tools.js
packages/decodal-codemirror/wasm/decodal_language_tools_bg.wasm
crates/decodal-wasm/src/lib.rs
crates/decodal-language-tools/src/lib.rs
```

Build the WebAssembly packages before building the site:

```sh
cd site/decodal-site
npm install
npm run build:wasm
npm run build
```

`npm run build:wasm` builds both generated WebAssembly packages.
`npm run build:wasm:runtime` writes generated runtime files into `packages/decodal-wasm/`.
`npm run build:wasm:formatter` writes generated formatter files into `packages/decodal-codemirror/wasm/`.
`npm run build` builds only the static site from the current package files.
Use `npm run build:all` to regenerate both WebAssembly packages and then build the static site in one command.

These generated files are committed so the site can be built without requiring every consumer to regenerate the wasm packages first.

Publish the generated WebAssembly package from its package directory:

```sh
cd packages/decodal-wasm
npm publish
npx jsr publish
```

Run dry-runs first when preparing a release:

```sh
npm pack --dry-run
npx jsr publish --dry-run
```

The npm package name is `decodal-wasm`.
The JSR package name is `@hare/decodal-wasm`.
The npm package advertises `MIT OR Apache-2.0`; JSR metadata uses `MIT` because its publisher requires a single recognized license identifier. Both license files are included in the package.

The playground editor uses the `decodal-codemirror` package with the generated Lezer parser in `packages/decodal-codemirror/src/decodal-parser.js`.
The canonical grammar is documented in `doc/manual/souce/language/grammar.md`; regenerate the Lezer parser when that grammar or `editors/lezer-decodal/decodal.grammar` changes.

Publish the CodeMirror package from its package directory:

```sh
cd packages/decodal-codemirror
npm install
npm publish
npx jsr publish
```

Run dry-runs first when preparing a release:

```sh
npm install
npm pack --dry-run
npx jsr publish --dry-run
```

The npm package name is `decodal-codemirror`.
The JSR package name is `@hare/decodal-codemirror`.
As with `decodal-wasm`, JSR metadata uses `MIT` while both the MIT and Apache-2.0 license files remain included.

The documentation build still uses the lightweight JavaScript fallback highlighter so Astro can render Markdown without initializing WASM at build time.

To run the site locally:

```sh
cd site/decodal-site
npm run dev
```

Deploy the static site to Cloudflare Pages with Wrangler direct upload:

```sh
cd site/decodal-site
npm run deploy
```

The deploy script runs `npm run build` and then uploads `dist/` to the Pages project named `decodal-site` on branch `master`.
Cloudflare Pages treats this as production when the project production branch is `master`.
Use `CLOUDFLARE_PROJECT_NAME` when deploying to a differently named Pages project.

```sh
CLOUDFLARE_PROJECT_NAME=my-pages-project npm run deploy
```

If the committed WASM package must be regenerated before deploy, use:

```sh
npm run deploy:all
```

## Tree-sitter grammar

The Tree-sitter grammar is kept in:

```text
editors/tree-sitter-decodal/
```

Important files:

```text
editors/tree-sitter-decodal/grammar.js
editors/tree-sitter-decodal/queries/highlights.scm
editors/tree-sitter-decodal/queries/locals.scm
editors/tree-sitter-decodal/corpus/basic.txt
```

The grammar is intended to be portable across editors such as Zed, Neovim, Helix, and Emacs.
Zed support should consume this grammar rather than relying on a TextMate grammar.

## Tree-sitter commands

From the grammar directory:

```sh
cd editors/tree-sitter-decodal
npm install
npx tree-sitter generate
npx tree-sitter test
```

To inspect a parse tree:

```sh
npx tree-sitter parse ../../examples/advanced/main.dcdl
```

The generated parser files under `editors/tree-sitter-decodal/src/` are committed so editor integrations can consume the grammar without regenerating it first.
`node_modules/` is ignored and must not be committed.

## Updating the grammar

When the Decodal syntax changes:

1. Update the canonical EBNF in `doc/manual/souce/language/grammar.md`.
2. Update the Rust parser/lexer as needed.
3. Update `editors/tree-sitter-decodal/grammar.js` and run `npx tree-sitter generate` / `npx tree-sitter test`.
4. Update `editors/lezer-decodal/decodal.grammar` and regenerate the CodeMirror parser:

```sh
cd site/decodal-site
npx lezer-generator ../../editors/lezer-decodal/decodal.grammar -o ../../packages/decodal-codemirror/src/decodal-parser.js
```

5. Add or update corpus/tests/examples.
6. Run Rust and site checks from the repository root.

## Development shell

The Nix development shell includes Rust tooling, Node.js, Tree-sitter CLI tooling, and wasm-pack tooling.

```sh
nix develop
```

The shell provides:

- `cargo`
- `rustc`
- `rustfmt`
- `clippy`
- `node`
- `npm`
- `wasm-pack`
- `lld`
- `tree-sitter`
- `nixfmt`