Compare commits

81 Commits
Author SHA1 Message Date
Hare e0a9a8efb7 Bump packages for open value release 2026-08-14 08:03:25 +09:00
Hare 848c7f169f Add unknown ranges and open object values 2026-08-14 08:00:22 +09:00
Hare 8c50dd202d Bump packages for range refinement release 2026-08-14 06:29:48 +09:00
Hare cc6ab40807 Add associative map constraints and range refinement 2026-08-14 06:26:15 +09:00
Hare 8198b615a8 Clarify site build and deploy scripts 2026-08-13 19:53:38 +09:00
Hare 62aa81b91c Fix JSR license metadata 2026-08-13 19:45:04 +09:00
Hare 9c46635ee0 Bump Rust crates for breaking release 2026-08-13 19:40:08 +09:00
Hare 515adc2533 Unify formatting on canonical syntax 2026-08-13 19:31:21 +09:00
Hare d9cf24d0eb Prepare Decodal packages for release 2026-08-13 19:01:14 +09:00
Hare d28edcf041 Share host-aware completion across runtimes 2026-08-13 18:55:03 +09:00
Hare e862e53f3e Add project-aware playground completion 2026-08-13 18:07:24 +09:00
Hare 610358110f Add host-configurable LSP server 2026-08-12 05:42:45 +09:00
Hare bdaccd9803 Add host-configurable language service 2026-08-11 22:05:17 +09:00
Hare 08603dc4b5 Report structured import origins in diagnostics 2026-08-11 19:20:11 +09:00
Hare 165daada23 Add host-structured imports 2026-08-11 18:33:49 +09:00
Hare 46b2b3b1b6 Add required array element constraints 2026-08-11 17:24:04 +09:00
Hare 5a05da6fe4 Use non-underscore Astro asset directory 2026-07-10 04:36:22 +09:00
Hare aa0f464320 Bump CodeMirror package to 0.1.5 2026-07-10 04:30:38 +09:00
Hare fc1f3e0b25 Improve JSR package metadata 2026-07-10 04:19:00 +09:00
Hare 8eac5c5cb4 Bundle language tools with CodeMirror package 2026-07-09 21:25:04 +09:00
Hare 59cb8f31e4 Simplify package link labels 2026-07-09 20:50:09 +09:00
Hare 4eee2cd2c9 Make package links flex layout 2026-07-09 20:45:16 +09:00
Hare 44b6041e60 Compact package links on landing page 2026-07-09 20:36:57 +09:00
Hare a95afe0463 Align playground control heights 2026-07-09 19:36:15 +09:00
Hare 43ab6244ef Reduce button vertical padding 2026-07-09 19:34:33 +09:00
Hare 1d4503b015 Simplify playground action separator 2026-07-09 19:33:48 +09:00
Hare 250e29e00c Remove playground helper copy 2026-07-09 19:29:22 +09:00
Hare 1080081cec Reset state before loading playground examples 2026-07-09 19:27:19 +09:00
Hare 43f692e75c Add playground entry selector 2026-07-09 19:21:12 +09:00
Hare 5b26d071ca Run playground examples from entrypoint 2026-07-09 19:12:18 +09:00
Hare a653f0b7c6 Nest repeated playground fields 2026-07-09 19:06:28 +09:00
Hare 384d17b539 Replace playground examples 2026-07-09 19:02:11 +09:00
Hare 09a84264ab Format playground examples 2026-07-09 18:54:29 +09:00
Hare 0a8c261545 Load playground wasm assets through Vite 2026-07-09 18:50:26 +09:00
Hare a8a46998a8 Make build:wasm build all wasm packages 2026-07-09 18:46:24 +09:00
Hare e4a9f80617 Dedupe CodeMirror dependencies 2026-07-09 18:43:09 +09:00
Hare 0e2a7b35d2 Add language tools package 2026-07-09 18:34:41 +09:00
Hare ac99c4cb4e Revert "Add Decodal formatter"
This reverts commit 22f6bc1ab9.
2026-07-09 17:46:24 +09:00
Hare 22f6bc1ab9 Add Decodal formatter 2026-07-09 17:39:32 +09:00
Hare 16006d000a Move wasm package under packages 2026-07-09 04:13:08 +09:00
Hare e4b9247c45 Extract CodeMirror language package 2026-07-09 03:49:44 +09:00
Hare 375bc80a57 Add package links to landing page 2026-07-09 03:31:20 +09:00
Hare 335ecde816 Fix JSR wasm package license 2026-07-09 03:29:27 +09:00
Hare 4142fcd46a Prepare 0.1.2 release 2026-07-09 03:12:09 +09:00
Hare 2d06dc3277 Add source link to header 2026-07-09 03:00:30 +09:00
Hare 37237dc1f7 Add source link to footer 2026-07-09 02:59:32 +09:00
Hare 2966d5b979 Add site footer 2026-07-09 02:58:53 +09:00
Hare ea858ad28e Remove landing page eyebrow 2026-07-09 02:57:55 +09:00
Hare 871d342e15 Add site landing page 2026-07-09 02:55:50 +09:00
Hare 6d80c66144 Trim open issues to materialization 2026-07-09 02:46:28 +09:00
Hare 8e1cb01f52 Document Decodal components 2026-07-09 01:47:28 +09:00
Hare 26d8495631 Remove public tokenizer API 2026-07-09 01:09:40 +09:00
Hare c4332ffb9d Restore CodeMirror syntax highlighting 2026-07-09 00:38:31 +09:00
Hare 33ea8b94f9 Fix CodeMirror folding ranges 2026-07-09 00:04:34 +09:00
Hare 720c71157e Fold Decodal delimited blocks from opener 2026-07-08 23:55:04 +09:00
Hare ee031bd2e8 Add canonical grammar and CodeMirror playground 2026-07-08 23:41:26 +09:00
Hare a07f4c48aa Expose tokenizer for web highlighting 2026-07-08 19:50:04 +09:00
Hare fe127428f9 Update generated WASM package README 2026-06-26 01:26:23 +09:00
Hare 0b76a600b1 Make Pages deploy script portable 2026-06-26 01:25:22 +09:00
Hare ada3578a1d Expose derive macro behind feature 2026-06-26 00:59:39 +09:00
Hare b8404df047 Add typed Decodal derive support 2026-06-26 00:39:33 +09:00
Hare 87fef44c68 Make decodal the published library crate 2026-06-25 22:51:16 +09:00
Hare 6e2363632b Prepare crates.io release metadata 2026-06-25 22:25:01 +09:00
Hare c26f9ca50a Add playground example loader 2026-06-25 03:39:40 +09:00
Hare f928028007 Deploy Pages production branch by default 2026-06-25 03:12:37 +09:00
Hare d8af980f7d Add Cloudflare Pages deploy script 2026-06-25 02:49:15 +09:00
Hare a2a6dee025 Show source names in diagnostics 2026-06-22 18:05:26 +09:00
Hare cc5ab63922 Improve composition diagnostics 2026-06-22 17:37:51 +09:00
Hare 8b3df21cfa Document lightweight runtime boundaries 2026-06-19 22:48:33 +09:00
Hare 01ad6dca52 Add array concat operator 2026-06-19 01:01:04 +09:00
Hare 6da0ec4c77 Fix inline code escaping 2026-06-18 23:55:11 +09:00
Hare 683151f6bd Remove unimplemented operator note 2026-06-18 23:54:10 +09:00
Hare 9e728fb634 Document operator reference 2026-06-18 23:42:53 +09:00
Hare 2fe54bda62 Add logical and comparison expressions 2026-06-17 23:38:44 +09:00
Hare 3f7dd7c692 Add arithmetic expressions 2026-06-17 23:00:14 +09:00
Hare dc28cddbff Add dark theme support 2026-06-17 14:39:44 +09:00
Hare aa6e5c51c1 Number manual sidebar navigation 2026-06-17 13:37:53 +09:00
Hare 4ab47e5719 Add multi-file playground imports 2026-06-17 10:17:00 +09:00
Hare 19c9de1601 Add site syntax highlighting 2026-06-17 07:39:00 +09:00
Hare 0e873fbd51 Migrate documentation site to Astro 2026-06-17 00:22:32 +09:00
Hare 4020b7c2d5 Add Svelte docs site and WASM playground 2026-06-17 00:07:11 +09:00
154 changed files with 36294 additions and 3659 deletions
+6
View File
@@ -3,3 +3,9 @@
/.env
/.yoi
/result
node_modules
/dist
site/decodal-site/dist
# Astro
site/decodal-site/.astro
Generated
+357 -5
View File
@@ -12,25 +12,221 @@ dependencies = [
]
[[package]]
name = "decodal"
version = "0.1.0"
name = "bitflags"
version = "1.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "crossbeam-channel"
version = "0.5.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e"
dependencies = [
"decodal-core",
"crossbeam-utils",
]
[[package]]
name = "decodal-core"
version = "0.1.0"
name = "crossbeam-utils"
version = "0.8.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
[[package]]
name = "decodal"
version = "0.4.0"
dependencies = [
"decodal-derive",
"regex",
]
[[package]]
name = "decodal-cli"
version = "0.4.0"
dependencies = [
"decodal",
]
[[package]]
name = "decodal-derive"
version = "0.2.0"
dependencies = [
"decodal",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "decodal-language-service"
version = "0.4.0"
dependencies = [
"decodal",
]
[[package]]
name = "decodal-language-tools"
version = "0.4.0"
dependencies = [
"decodal",
"serde_json",
"wasm-bindgen",
]
[[package]]
name = "decodal-lsp"
version = "0.4.0"
dependencies = [
"decodal",
"decodal-language-service",
"decodal-language-tools",
"lsp-server",
"lsp-types",
"serde_json",
]
[[package]]
name = "decodal-wasm"
version = "0.4.0"
dependencies = [
"decodal",
"decodal-language-service",
"js-sys",
"serde-wasm-bindgen",
"serde_json",
"wasm-bindgen",
]
[[package]]
name = "fluent-uri"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "17c704e9dbe1ddd863da1e6ff3567795087b1eb201ce80d8fa81162e1516500d"
dependencies = [
"bitflags",
]
[[package]]
name = "futures-core"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e"
[[package]]
name = "futures-task"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd"
[[package]]
name = "futures-util"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc"
dependencies = [
"futures-core",
"futures-task",
"pin-project-lite",
"slab",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "js-sys"
version = "0.3.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03d04c30968dffe80775bd4d7fb676131cd04a1fb46d2686dbffbaec2d9dfd31"
dependencies = [
"cfg-if",
"futures-util",
"wasm-bindgen",
]
[[package]]
name = "log"
version = "0.4.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "lsp-server"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ee25a31f2e571e426eef2896179450cafc7e2f5be00d8a93b1c2d21c0ff7656"
dependencies = [
"crossbeam-channel",
"log",
"serde",
"serde_derive",
"serde_json",
]
[[package]]
name = "lsp-types"
version = "0.97.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53353550a17c04ac46c585feb189c2db82154fc84b79c7a66c96c2c644f66071"
dependencies = [
"bitflags",
"fluent-uri",
"serde",
"serde_json",
"serde_repr",
]
[[package]]
name = "memchr"
version = "2.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88904434abc2901f197fe8cc55f0445e7ded921dba5911dad2e2b39b48e663c4"
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "regex"
version = "1.12.4"
@@ -59,3 +255,159 @@ name = "regex-syntax"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "rustversion"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde-wasm-bindgen"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8302e169f0eddcc139c70f139d19d6467353af16f9fce27e8c30158036a1e16b"
dependencies = [
"js-sys",
"serde",
"wasm-bindgen",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "serde_json"
version = "1.0.150"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "serde_repr"
version = "0.1.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d3b1629de253c70a0508c3899572da79ca359fdab27c7920ff00406df418906"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.3",
]
[[package]]
name = "slab"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "wasm-bindgen"
version = "0.2.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ddb3f79143bced6de84270411622a2699cee572fc0875aeaf1e7867cf9fca1a"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e21a184b13fb19e157296e2c46056aec9092264fab83e4ba59e68c61b323c3d"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fecefd9c35bd935a20fc3fc344b5f29138961e4f47fb03297d88f2587afb5ebd"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn 2.0.117",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23939e44bb9a5d7576fa2b563dc2e136628f1224e88a8deed09e04858b77871f"
dependencies = [
"unicode-ident",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
+18 -7
View File
@@ -2,12 +2,23 @@
members = [
"crates/decodal-core",
"crates/decodal-cli",
"crates/decodal-wasm",
"crates/decodal-derive",
"crates/decodal-language-tools",
"crates/decodal-language-service",
"crates/decodal-lsp",
]
resolver = "3"
exclude = ["editors/tree-sitter-decodal"]
resolver = "2"
[profile.release]
opt-level = "z"
lto = true
codegen-units = 1
strip = true
panic = "abort"
[workspace.package]
version = "0.4.0"
edition = "2024"
rust-version = "1.85"
license = "MIT OR Apache-2.0"
repository = "https://gitea.hareworks.net/Hare/Decodal"
readme = "README.md"
[workspace.dependencies]
serde_json = "1"
wasm-bindgen = "0.2"
+173
View File
@@ -0,0 +1,173 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Decodal contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+109
View File
@@ -0,0 +1,109 @@
# Decodal
Decodal is a small deterministic DSL for describing, composing, validating, and materializing structured data.
It is designed around a lightweight Rust library:
- host-supplied source and structured-value imports through `ImportLoader`
- shared host environments for production and semantic editor tooling
- no filesystem access in the library core
- concrete and abstract values with constraints and defaults
- asymmetric range refinement with `narrower as wider`
- homogeneous associative-array schemas with `{...valueSchema}` and object rest constraints
- the safe top range `Unknown`, which remains abstract until refined, defaulted, or supplied concretely
- deterministic expression evaluation
- optional regex support behind a Cargo feature
- browser playground support through WebAssembly
## Library crate
Embedded hosts should depend on `decodal` and provide imports with an `ImportLoader`.
```toml
[dependencies]
decodal = "0.4.0"
```
## Derive support
For embedded Rust applications, Decodal can generate a schema and typed decoder from a Rust struct with the `derive` feature.
Map fields marked with `#[decodal(rest)]` receive additional object fields; `BTreeMap<String, Data>` corresponds to `...Unknown`.
```toml
[dependencies]
decodal = { version = "0.4.0", features = ["derive"] }
```
```rust
use decodal::{Decodal, DecodalDecode, DecodalSchema, Engine};
#[derive(Decodal)]
struct Service {
name: String,
#[decodal(gt = 443, default = 8443)]
port: i64,
#[decodal(rename = "feature.enable", default = true)]
feature_enabled: bool,
}
```
The derive implements:
- `DecodalSchema`, which produces a `Value` range for `Engine::bind_global`
- `DecodalDecode`, which converts materialized `Data` into the Rust struct
## CLI
A standalone CLI is kept in this repository as the `decodal-cli` workspace package.
It builds a `decodal` binary, but it is not the primary crates.io package.
Run a Decodal file from the repository:
```sh
cargo run -q -p decodal-cli -- examples/advanced/main.dcdl
```
Enable optional regex support when needed:
```sh
cargo run -q -p decodal-cli --features regex -- examples/regex/main.dcdl
```
## Language server
The `decodal-language-service` crate provides transport-independent evaluation and completion, while `decodal-lsp` exposes it through the Language Server Protocol with live semantic diagnostics and document formatting.
Its library API accepts the same host environment used for production evaluation, so application-defined globals and structured import routing remain available in the editor.
```toml
[dependencies]
decodal = "0.4.0"
decodal-language-service = "0.4.0"
decodal-lsp = "0.4.0"
```
Run the default filesystem-backed server from the repository:
```sh
cargo run -q -p decodal-lsp
```
Embedded applications can implement `decodal_lsp::LspEnvironment` for their existing `HostEnvironment` and construct it from the `InitializeParams` passed to `decodal_lsp::run_stdio`.
## Web playground
The static documentation site and browser playground live under:
```text
site/decodal-site/
```
Browser hosts can use `decodal-wasm`. Its `DecodalLanguageService` accepts JavaScript-owned `globals`, `loadImport`, and `completeImport` callbacks; filesystem and virtual-project policy remain outside the package.
## License
Licensed under either of:
- Apache License, Version 2.0
- MIT license
at your option.
+62
View File
@@ -0,0 +1,62 @@
# Releasing Decodal
Run all commands from the repository root. Publishing requires Cargo, npm, and JSR credentials; validation does not.
## Versions
- `decodal-derive`: `0.2.0`
- Other published Rust crates: `0.4.0`
- `decodal-wasm` npm/JSR package: `0.3.0`
- `decodal-codemirror`: `0.3.0`
The npm packages advertise the repository's dual `MIT OR Apache-2.0` license.
JSR metadata uses the single `MIT` identifier required by its publisher; both license files remain included in every package.
## Validate
```sh
cargo fmt --check
cargo test --workspace
cargo test -p decodal --no-default-features
cargo clippy --workspace --all-targets -- -D warnings
cargo clippy -p decodal-wasm --target wasm32-unknown-unknown -- -D warnings
cargo clippy -p decodal-language-tools --target wasm32-unknown-unknown -- -D warnings
npm --prefix site/decodal-site run build:wasm
npm --prefix packages/decodal-codemirror test
npm --prefix site/decodal-site test
npm --prefix site/decodal-site run build
deno check packages/decodal-wasm/mod.ts
npm pack --dry-run ./packages/decodal-wasm
deno publish --dry-run --config packages/decodal-wasm/jsr.json
npm pack --dry-run ./packages/decodal-codemirror
deno publish --dry-run --config packages/decodal-codemirror/jsr.json
```
## Publish Rust crates
Publish in dependency order. Wait for each crate to become available in the crates.io index before continuing.
Run the corresponding command with `--dry-run` immediately before each real publish.
```sh
cargo publish -p decodal-derive
cargo publish -p decodal
cargo publish -p decodal-language-service
cargo publish -p decodal-language-tools
cargo publish -p decodal-lsp
```
`decodal-cli` and the Rust `decodal-wasm` wrapper remain repository-only packages.
## Publish JavaScript packages
The WASM build regenerates and normalizes both committed npm/JSR packages before publishing.
```sh
npm --prefix site/decodal-site run build:wasm
npm publish ./packages/decodal-wasm
deno publish --config packages/decodal-wasm/jsr.json
npm publish ./packages/decodal-codemirror
deno publish --config packages/decodal-codemirror/jsr.json
```
After both registries accept the release, tag the release commit and update the deployed site.
+16 -4
View File
@@ -1,11 +1,23 @@
[package]
name = "decodal-cli"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Command-line interface for the Decodal data description language."
keywords = ["decodal", "dsl", "config", "cli"]
categories = ["command-line-utilities", "config"]
publish = false
[[bin]]
name = "decodal"
version = "0.1.0"
edition = "2024"
path = "src/main.rs"
[features]
default = []
regex = ["decodal-core/regex"]
regex = ["decodal/regex"]
[dependencies]
decodal-core = { path = "../decodal-core" }
decodal = { version = "0.4.0", path = "../decodal-core" }
+38 -25
View File
@@ -4,19 +4,28 @@ use std::{
process::ExitCode,
};
use decodal_core::{Data, Diagnostic, DiagnosticKind, Engine, LoadedSource, SourceLoader, Span};
use decodal::{
Data, Diagnostic, DiagnosticKind, Engine, ImportLoader, LoadedImport, SourceId, Span,
format_diagnostic_with,
};
struct SourceInput {
key: String,
name: String,
source: String,
}
fn main() -> ExitCode {
match run() {
Ok(()) => ExitCode::SUCCESS,
Err(error) => {
print_diagnostic(&error);
eprintln!("{error}");
ExitCode::FAILURE
}
}
}
fn run() -> Result<(), Diagnostic> {
fn run() -> Result<(), String> {
let mut args = env::args().skip(1);
let first = args.next().unwrap_or_else(|| String::from("--help"));
match first.as_str() {
@@ -25,7 +34,7 @@ fn run() -> Result<(), Diagnostic> {
Ok(())
}
"--version" | "-V" => {
println!("Decodal {}", decodal_core::version());
println!("Decodal {}", decodal::version());
Ok(())
}
"check" => {
@@ -51,15 +60,22 @@ fn run() -> Result<(), Diagnostic> {
}
}
fn materialize_path(path: &str) -> Result<Data, Diagnostic> {
let root = read_root_source(path)?;
fn materialize_path(path: &str) -> Result<Data, String> {
let root = read_root_source(path).map_err(format_raw_diagnostic)?;
let root_name = root.name.clone();
let mut engine = Engine::new(FsLoader);
let module = engine.add_root_source(root.key, root.name, &root.source)?;
let value = engine.eval_module(module)?;
engine.materialize(&value)
let module = engine
.add_root_source(root.key, root.name, &root.source)
.map_err(|error| format_diagnostic_with_root(&error, &root_name))?;
let value = engine
.eval_module(module)
.map_err(|error| engine.format_diagnostic(&error))?;
engine
.materialize(&value)
.map_err(|error| engine.format_diagnostic(&error))
}
fn read_root_source(path: &str) -> Result<LoadedSource, Diagnostic> {
fn read_root_source(path: &str) -> Result<SourceInput, Diagnostic> {
if path == "-" {
use std::io::Read;
let mut source = String::new();
@@ -72,7 +88,7 @@ fn read_root_source(path: &str) -> Result<LoadedSource, Diagnostic> {
format!("failed to read stdin: {error}"),
)
})?;
Ok(LoadedSource {
Ok(SourceInput {
key: String::from("<stdin>"),
name: String::from("<stdin>"),
source,
@@ -85,12 +101,12 @@ fn read_root_source(path: &str) -> Result<LoadedSource, Diagnostic> {
#[derive(Debug, Clone, Copy)]
struct FsLoader;
impl SourceLoader for FsLoader {
impl ImportLoader for FsLoader {
fn load(
&mut self,
current_key: Option<&str>,
specifier: &str,
) -> Result<LoadedSource, Diagnostic> {
) -> Result<LoadedImport, Diagnostic> {
let path = Path::new(specifier);
let path = if path.is_absolute() {
PathBuf::from(path)
@@ -102,11 +118,11 @@ impl SourceLoader for FsLoader {
} else {
PathBuf::from(path)
};
load_path(&path)
load_path(&path).map(|source| LoadedImport::source(source.key, source.name, source.source))
}
}
fn load_path(path: &Path) -> Result<LoadedSource, Diagnostic> {
fn load_path(path: &Path) -> Result<SourceInput, Diagnostic> {
let canonical = path.canonicalize().map_err(|error| {
Diagnostic::new(
DiagnosticKind::Import,
@@ -122,7 +138,7 @@ fn load_path(path: &Path) -> Result<LoadedSource, Diagnostic> {
)
})?;
let key = canonical.to_string_lossy().into_owned();
Ok(LoadedSource {
Ok(SourceInput {
name: key.clone(),
key,
source,
@@ -139,15 +155,12 @@ fn print_help() {
println!(" decodal - Read DCDL source from stdin");
}
fn print_diagnostic(error: &Diagnostic) {
eprintln!(
"error[{kind:?}] {source}:{start}..{end}: {message}",
kind = error.kind,
source = error.span.source.0,
start = error.span.start,
end = error.span.end,
message = error.message,
);
fn format_diagnostic_with_root(error: &Diagnostic, root_name: &str) -> String {
format_diagnostic_with(error, |source| (source == SourceId(0)).then_some(root_name))
}
fn format_raw_diagnostic(error: Diagnostic) -> String {
format_diagnostic_with(&error, |_| None)
}
fn print_data(data: &Data, indent: usize) {
+12 -3
View File
@@ -1,12 +1,21 @@
[package]
name = "decodal-core"
version = "0.1.0"
edition = "2024"
name = "decodal"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Parser, evaluator, and embedding API for the Decodal data description language."
keywords = ["decodal", "dsl", "config", "schema"]
categories = ["config", "parser-implementations"]
[features]
default = ["std"]
std = []
derive = ["dep:decodal-derive"]
regex = ["std", "dep:regex"]
[dependencies]
decodal-derive = { version = "0.2.0", path = "../decodal-derive", optional = true }
regex = { version = "1.10", default-features = false, features = ["std", "unicode-perl"], optional = true }
+6 -6
View File
@@ -1,14 +1,14 @@
use decodal_core::{Data, EmptyLoader, Engine, HostValue};
use decodal::{Data, EmptyLoader, Engine, Value};
fn main() -> decodal_core::Result<()> {
fn main() -> decodal::Result<()> {
let mut engine = Engine::new(EmptyLoader);
engine.bind_global(
"Service",
HostValue::object([
("name", HostValue::string_type()),
("port", HostValue::int_type().gt(443).default_int(8443)?),
("enabled", HostValue::bool_type().default_bool(true)?),
Value::object([
("name", Value::string_type()),
("port", Value::int_type().gt(443).default_int(8443)?),
("enabled", Value::bool_type().default_bool(true)?),
]),
)?;
@@ -0,0 +1,85 @@
use decodal::{Data, Diagnostic, DiagnosticKind, Engine, ImportLoader, LoadedImport, Span, Value};
const POST: &str = r#"---
title: Hello
draft: false
---
# Hello
This body stays as Markdown.
"#;
struct ContentLoader;
impl ImportLoader for ContentLoader {
fn load(
&mut self,
_current_key: Option<&str>,
specifier: &str,
) -> decodal::Result<LoadedImport> {
if specifier != "./post.md" {
return Err(Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
format!("unknown content import `{specifier}`"),
));
}
Ok(LoadedImport::value(
"content/post.md",
parse_markdown(POST)?,
))
}
}
fn parse_markdown(source: &str) -> decodal::Result<Value> {
let source = source.strip_prefix("---\n").ok_or_else(frontmatter_error)?;
let (frontmatter, body) = source.split_once("\n---\n").ok_or_else(frontmatter_error)?;
let mut fields = Vec::new();
for line in frontmatter.lines() {
let (name, value) = line.split_once(':').ok_or_else(frontmatter_error)?;
let value = match value.trim() {
"true" => Value::bool(true),
"false" => Value::bool(false),
value => Value::string(value),
};
fields.push((name.trim(), value));
}
Ok(Value::object([
("frontmatter", Value::object(fields)),
("body", Value::string(body)),
]))
}
fn frontmatter_error() -> Diagnostic {
Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
"invalid Markdown frontmatter",
)
}
fn main() -> decodal::Result<()> {
let mut engine = Engine::new(ContentLoader);
let module = engine.add_root_source(
"main.dcdl",
"main.dcdl",
r#"
let post = import "./post.md";
in {
title = post.frontmatter.title;
draft = post.frontmatter.draft;
body = post.body;
}
"#,
)?;
let value = engine.eval_module(module)?;
let data = engine.materialize(&value)?;
let Data::Object(fields) = data else {
panic!("expected imported Markdown to produce an object")
};
assert_eq!(fields[0].value, Data::String(String::from("Hello")));
assert_eq!(fields[1].value, Data::Bool(false));
assert!(matches!(fields[2].value, Data::String(_)));
Ok(())
}
+50 -1
View File
@@ -48,8 +48,17 @@ pub struct SpannedExpr {
pub enum Expr {
Literal(Literal),
Ident(String),
Object(Vec<Field>),
Object {
fields: Vec<Field>,
rest: Option<ObjectRest>,
},
Array(Vec<ExprId>),
ArrayConstraint {
item: ExprId,
},
MapConstraint {
value: ExprId,
},
Let {
bindings: Vec<Field>,
body: ExprId,
@@ -67,10 +76,21 @@ pub enum Expr {
params: Vec<Param>,
body: ExprId,
},
/// An explicitly parenthesized expression.
///
/// Keeping this node preserves the delimiters and their trivia for source
/// tools while evaluation remains equivalent to evaluating `expr`.
Parenthesized {
expr: ExprId,
},
Match {
scrutinee: ExprId,
arms: Vec<MatchArm>,
},
Unary {
op: UnaryOp,
expr: ExprId,
},
Binary {
op: BinaryOp,
lhs: ExprId,
@@ -80,6 +100,10 @@ pub enum Expr {
base: ExprId,
fallback: ExprId,
},
As {
narrower: ExprId,
wider: ExprId,
},
CompareConstraint {
op: CompareOp,
value: ExprId,
@@ -95,6 +119,12 @@ pub struct Field {
pub span: Span,
}
#[derive(Debug, Clone, PartialEq)]
pub struct ObjectRest {
pub value: ExprId,
pub span: Span,
}
#[derive(Debug, Clone, PartialEq)]
pub struct Param {
pub name: String,
@@ -117,8 +147,27 @@ pub enum Literal {
Bool(bool),
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum UnaryOp {
Neg,
Not,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum BinaryOp {
Add,
Sub,
Mul,
Div,
Concat,
Equal,
NotEqual,
Greater,
GreaterEqual,
Less,
LessEqual,
LogicalAnd,
LogicalOr,
And,
Patch,
}
+230 -66
View File
@@ -3,103 +3,240 @@ use alloc::{string::String, vec::Vec};
use crate::{
Diagnostic, DiagnosticKind, Span,
ast::CompareOp,
runtime::{Constraint, LiteralValue, PrimitiveType},
runtime::{Constraint, ConstraintEntry, LiteralValue, PrimitiveType},
};
pub fn normalize_constraints(
constraints: Vec<Constraint>,
constraints: Vec<ConstraintEntry>,
span: Span,
) -> crate::Result<Vec<Constraint>> {
let mut primitive = None;
let mut lower: Option<Bound> = None;
let mut upper: Option<Bound> = None;
) -> crate::Result<Vec<ConstraintEntry>> {
let mut primitive: Option<(PrimitiveType, Span)> = None;
let mut array_span: Option<Span> = None;
let mut map_span: Option<Span> = None;
let mut lower: Option<(Bound, Span)> = None;
let mut upper: Option<(Bound, Span)> = None;
let mut rest = Vec::new();
for constraint in constraints {
match constraint {
for entry in constraints {
match entry.constraint {
Constraint::Unknown => rest.push(ConstraintEntry {
constraint: Constraint::Unknown,
span: entry.span,
}),
Constraint::Type(next) => match primitive {
Some(current) if current != next => {
Some((current, current_span)) if current != next => {
return Err(Diagnostic::new(
DiagnosticKind::Conflict,
span,
"primitive type constraints conflict",
));
)
.with_label(current_span, "first primitive constraint")
.with_label(entry.span, "conflicting primitive constraint"));
}
Some(_) => {}
None => primitive = Some(next),
None => primitive = Some((next, entry.span)),
},
Constraint::Compare(op, value) => {
let number = Number::from_literal(&value).ok_or_else(|| {
Diagnostic::new(
DiagnosticKind::TypeMismatch,
span,
entry.span,
"comparison constraints require numeric literals",
)
})?;
match op {
CompareOp::Gt => merge_lower(&mut lower, Bound::new(number, false)),
CompareOp::Gte => merge_lower(&mut lower, Bound::new(number, true)),
CompareOp::Lt => merge_upper(&mut upper, Bound::new(number, false)),
CompareOp::Lte => merge_upper(&mut upper, Bound::new(number, true)),
CompareOp::Gt => {
merge_lower(&mut lower, (Bound::new(number, false), entry.span))
}
CompareOp::Gte => {
merge_lower(&mut lower, (Bound::new(number, true), entry.span))
}
CompareOp::Lt => {
merge_upper(&mut upper, (Bound::new(number, false), entry.span))
}
CompareOp::Lte => {
merge_upper(&mut upper, (Bound::new(number, true), entry.span))
}
CompareOp::Eq => {
merge_lower(&mut lower, Bound::new(number, true));
merge_upper(&mut upper, Bound::new(number, true));
merge_lower(&mut lower, (Bound::new(number, true), entry.span));
merge_upper(&mut upper, (Bound::new(number, true), entry.span));
}
}
}
Constraint::Regex(pattern) => rest.push(Constraint::Regex(pattern)),
Constraint::BuiltinPredicate(name) => rest.push(Constraint::BuiltinPredicate(name)),
Constraint::Regex(pattern) => rest.push(ConstraintEntry {
constraint: Constraint::Regex(pattern),
span: entry.span,
}),
Constraint::ArrayItems(item) => {
array_span.get_or_insert(entry.span);
rest.push(ConstraintEntry {
constraint: Constraint::ArrayItems(item),
span: entry.span,
});
}
Constraint::MapValues(value) => {
map_span.get_or_insert(entry.span);
rest.push(ConstraintEntry {
constraint: Constraint::MapValues(value),
span: entry.span,
});
}
Constraint::BuiltinPredicate(name) => rest.push(ConstraintEntry {
constraint: Constraint::BuiltinPredicate(name),
span: entry.span,
}),
}
}
if matches!(primitive, Some(PrimitiveType::String | PrimitiveType::Bool))
if let (Some((_, primitive_span)), Some(array_span)) = (primitive, array_span) {
return Err(Diagnostic::new(
DiagnosticKind::Conflict,
span,
"array element constraints conflict with primitive type constraints",
)
.with_label(primitive_span, "primitive type constraint")
.with_label(array_span, "array element constraint"));
}
if let (Some((_, primitive_span)), Some(map_span)) = (primitive, map_span) {
return Err(Diagnostic::new(
DiagnosticKind::Conflict,
span,
"map value constraints conflict with primitive type constraints",
)
.with_label(primitive_span, "primitive constraint")
.with_label(map_span, "map value constraint"));
}
if let (Some(array_span), Some(map_span)) = (array_span, map_span) {
return Err(Diagnostic::new(
DiagnosticKind::Conflict,
span,
"array and map constraints conflict",
)
.with_label(array_span, "array element constraint")
.with_label(map_span, "map value constraint"));
}
if let Some(array_span) = array_span
&& (lower.is_some() || upper.is_some())
{
return Err(Diagnostic::new(
let mut diagnostic = Diagnostic::new(
DiagnosticKind::Conflict,
span,
"numeric comparison constraints conflict with array element constraints",
)
.with_label(array_span, "array element constraint");
if let Some((_, lower_span)) = lower {
diagnostic = diagnostic.with_label(lower_span, "numeric comparison constraint");
}
if let Some((_, upper_span)) = upper {
diagnostic = diagnostic.with_label(upper_span, "numeric comparison constraint");
}
return Err(diagnostic);
}
if let Some(map_span) = map_span
&& (lower.is_some() || upper.is_some())
{
let mut diagnostic = Diagnostic::new(
DiagnosticKind::Conflict,
span,
"numeric comparison constraints conflict with map value constraints",
)
.with_label(map_span, "map value constraint");
if let Some((_, lower_span)) = lower {
diagnostic = diagnostic.with_label(lower_span, "numeric comparison constraint");
}
if let Some((_, upper_span)) = upper {
diagnostic = diagnostic.with_label(upper_span, "numeric comparison constraint");
}
return Err(diagnostic);
}
if matches!(
primitive,
Some((PrimitiveType::String | PrimitiveType::Bool, _))
) && (lower.is_some() || upper.is_some())
{
let mut diagnostic = Diagnostic::new(
DiagnosticKind::Conflict,
span,
"numeric comparison constraints conflict with non-numeric primitive type",
));
);
if let Some((_, primitive_span)) = primitive {
diagnostic = diagnostic.with_label(primitive_span, "non-numeric primitive constraint");
}
if let Some((_, lower_span)) = lower {
diagnostic = diagnostic.with_label(lower_span, "numeric comparison constraint");
}
if let Some((_, upper_span)) = upper {
diagnostic = diagnostic.with_label(upper_span, "numeric comparison constraint");
}
return Err(diagnostic);
}
if primitive == Some(PrimitiveType::Int)
if matches!(primitive, Some((PrimitiveType::Int, _)))
&& lower
.iter()
.chain(upper.iter())
.any(|bound| !matches!(bound.number, Number::Int(_)))
.any(|(bound, _)| !matches!(bound.number, Number::Int(_)))
{
return Err(Diagnostic::new(
let mut diagnostic = Diagnostic::new(
DiagnosticKind::Conflict,
span,
"Int comparison constraints must use integer literals",
));
);
if let Some((_, primitive_span)) = primitive {
diagnostic = diagnostic.with_label(primitive_span, "Int primitive constraint");
}
if let Some((bound, bound_span)) = lower {
if !matches!(bound.number, Number::Int(_)) {
diagnostic = diagnostic.with_label(bound_span, "non-integer comparison constraint");
}
}
if let Some((bound, bound_span)) = upper {
if !matches!(bound.number, Number::Int(_)) {
diagnostic = diagnostic.with_label(bound_span, "non-integer comparison constraint");
}
}
return Err(diagnostic);
}
ensure_bounds_non_empty(primitive, lower, upper, span)?;
let mut normalized = Vec::new();
if let Some(primitive) = primitive {
normalized.push(Constraint::Type(primitive));
if let Some((primitive, primitive_span)) = primitive {
normalized.push(ConstraintEntry {
constraint: Constraint::Type(primitive),
span: primitive_span,
});
}
if let Some(lower) = lower {
normalized.push(Constraint::Compare(
if let Some((lower, lower_span)) = lower {
normalized.push(ConstraintEntry {
constraint: Constraint::Compare(
if lower.inclusive {
CompareOp::Gte
} else {
CompareOp::Gt
},
lower.number.into_literal(),
));
),
span: lower_span,
});
}
if let Some(upper) = upper {
normalized.push(Constraint::Compare(
if let Some((upper, upper_span)) = upper {
normalized.push(ConstraintEntry {
constraint: Constraint::Compare(
if upper.inclusive {
CompareOp::Lte
} else {
CompareOp::Lt
},
upper.number.into_literal(),
));
),
span: upper_span,
});
}
normalized.extend(rest);
Ok(normalized)
@@ -147,18 +284,22 @@ impl Number {
}
}
fn merge_lower(current: &mut Option<Bound>, next: Bound) {
fn merge_lower(current: &mut Option<(Bound, Span)>, next: (Bound, Span)) {
match current {
None => *current = Some(next),
Some(current_bound) if is_stricter_lower(next, *current_bound) => *current_bound = next,
Some((current_bound, _)) if is_stricter_lower(next.0, *current_bound) => {
*current = Some(next)
}
Some(_) => {}
}
}
fn merge_upper(current: &mut Option<Bound>, next: Bound) {
fn merge_upper(current: &mut Option<(Bound, Span)>, next: (Bound, Span)) {
match current {
None => *current = Some(next),
Some(current_bound) if is_stricter_upper(next, *current_bound) => *current_bound = next,
Some((current_bound, _)) if is_stricter_upper(next.0, *current_bound) => {
*current = Some(next)
}
Some(_) => {}
}
}
@@ -178,28 +319,32 @@ fn is_stricter_upper(next: Bound, current: Bound) -> bool {
}
fn ensure_bounds_non_empty(
primitive: Option<PrimitiveType>,
lower: Option<Bound>,
upper: Option<Bound>,
primitive: Option<(PrimitiveType, Span)>,
lower: Option<(Bound, Span)>,
upper: Option<(Bound, Span)>,
span: Span,
) -> crate::Result<()> {
if primitive == Some(PrimitiveType::Int) {
let min = lower.map(int_lower_bound).unwrap_or(i128::from(i64::MIN));
let max = upper.map(int_upper_bound).unwrap_or(i128::from(i64::MAX));
if matches!(primitive, Some((PrimitiveType::Int, _))) {
let min = lower
.map(|(bound, _)| int_lower_bound(bound))
.unwrap_or(i128::from(i64::MIN));
let max = upper
.map(|(bound, _)| int_upper_bound(bound))
.unwrap_or(i128::from(i64::MAX));
if min > max {
return Err(empty_numeric_bounds(span));
return Err(empty_numeric_bounds(span, lower, upper));
}
return Ok(());
}
if let (Some(lower), Some(upper)) = (lower, upper) {
let lower_value = lower.number.as_f64();
let upper_value = upper.number.as_f64();
if let (Some((lower_bound, _)), Some((upper_bound, _))) = (lower, upper) {
let lower_value = lower_bound.number.as_f64();
let upper_value = upper_bound.number.as_f64();
if lower_value > upper_value {
return Err(empty_numeric_bounds(span));
return Err(empty_numeric_bounds(span, lower, upper));
}
if lower_value == upper_value && !(lower.inclusive && upper.inclusive) {
return Err(empty_numeric_bounds(span));
if lower_value == upper_value && !(lower_bound.inclusive && upper_bound.inclusive) {
return Err(empty_numeric_bounds(span, lower, upper));
}
}
Ok(())
@@ -227,12 +372,23 @@ fn int_upper_bound(bound: Bound) -> i128 {
}
}
fn empty_numeric_bounds(span: Span) -> Diagnostic {
Diagnostic::new(
fn empty_numeric_bounds(
span: Span,
lower: Option<(Bound, Span)>,
upper: Option<(Bound, Span)>,
) -> Diagnostic {
let mut diagnostic = Diagnostic::new(
DiagnosticKind::Conflict,
span,
String::from("numeric comparison constraints have an empty intersection"),
)
);
if let Some((_, lower_span)) = lower {
diagnostic = diagnostic.with_label(lower_span, "lower bound constraint");
}
if let Some((_, upper_span)) = upper {
diagnostic = diagnostic.with_label(upper_span, "upper bound constraint");
}
diagnostic
}
#[cfg(test)]
@@ -240,13 +396,20 @@ mod tests {
use super::*;
use crate::runtime::LiteralValue;
fn entry(constraint: Constraint) -> ConstraintEntry {
ConstraintEntry {
constraint,
span: Span::default(),
}
}
#[test]
fn detects_primitive_conflict() {
assert!(
normalize_constraints(
alloc::vec![
Constraint::Type(PrimitiveType::Int),
Constraint::Type(PrimitiveType::String),
entry(Constraint::Type(PrimitiveType::Int)),
entry(Constraint::Type(PrimitiveType::String)),
],
Span::default(),
)
@@ -259,9 +422,9 @@ mod tests {
assert!(
normalize_constraints(
alloc::vec![
Constraint::Type(PrimitiveType::Int),
Constraint::Compare(CompareOp::Gt, LiteralValue::Int(10)),
Constraint::Compare(CompareOp::Lt, LiteralValue::Int(11)),
entry(Constraint::Type(PrimitiveType::Int)),
entry(Constraint::Compare(CompareOp::Gt, LiteralValue::Int(10))),
entry(Constraint::Compare(CompareOp::Lt, LiteralValue::Int(5))),
],
Span::default(),
)
@@ -270,15 +433,16 @@ mod tests {
}
#[test]
fn keeps_regex_constraints_without_intersection_check() {
let constraints = normalize_constraints(
fn detects_non_integer_int_bound() {
assert!(
normalize_constraints(
alloc::vec![
Constraint::Regex(String::from("^a$")),
Constraint::Regex(String::from("^b$")),
entry(Constraint::Type(PrimitiveType::Int)),
entry(Constraint::Compare(CompareOp::Gt, LiteralValue::Float(1.5))),
],
Span::default(),
)
.unwrap();
assert_eq!(constraints.len(), 2);
.is_err()
);
}
}
+24 -1
View File
@@ -1,4 +1,4 @@
use alloc::string::String;
use alloc::{string::String, vec::Vec};
use crate::span::Span;
@@ -9,6 +9,14 @@ pub struct Diagnostic {
pub kind: DiagnosticKind,
pub span: Span,
pub message: String,
pub labels: Vec<DiagnosticLabel>,
pub notes: Vec<String>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DiagnosticLabel {
pub span: Span,
pub message: String,
}
impl Diagnostic {
@@ -17,12 +25,27 @@ impl Diagnostic {
kind,
span,
message: message.into(),
labels: Vec::new(),
notes: Vec::new(),
}
}
pub fn syntax(span: Span, message: impl Into<String>) -> Self {
Self::new(DiagnosticKind::Syntax, span, message)
}
pub fn with_label(mut self, span: Span, message: impl Into<String>) -> Self {
self.labels.push(DiagnosticLabel {
span,
message: message.into(),
});
self
}
pub fn with_note(mut self, message: impl Into<String>) -> Self {
self.notes.push(message.into());
self
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
-164
View File
@@ -1,164 +0,0 @@
use alloc::{boxed::Box, string::String, vec::Vec};
use crate::runtime::{Constraint, LiteralValue, PrimitiveType};
use crate::{CompareOp, Diagnostic, DiagnosticKind, Result, Span};
#[derive(Debug, Clone, PartialEq)]
pub enum HostValue {
String(String),
Int(i64),
Float(f64),
Bool(bool),
Array(Vec<HostValue>),
Object(Vec<HostField>),
Abstract {
constraints: Vec<Constraint>,
default: Option<Box<HostValue>>,
},
}
#[derive(Debug, Clone, PartialEq)]
pub struct HostField {
pub name: String,
pub value: HostValue,
}
impl HostValue {
pub fn string(value: impl Into<String>) -> Self {
Self::String(value.into())
}
pub fn int(value: i64) -> Self {
Self::Int(value)
}
pub fn float(value: f64) -> Self {
Self::Float(value)
}
pub fn bool(value: bool) -> Self {
Self::Bool(value)
}
pub fn array<I>(items: I) -> Self
where
I: IntoIterator<Item = HostValue>,
{
Self::Array(items.into_iter().collect())
}
pub fn object<I, N>(fields: I) -> Self
where
I: IntoIterator<Item = (N, HostValue)>,
N: Into<String>,
{
Self::Object(
fields
.into_iter()
.map(|(name, value)| HostField {
name: name.into(),
value,
})
.collect(),
)
}
pub fn string_type() -> Self {
Self::abstract_with_constraint(Constraint::Type(PrimitiveType::String))
}
pub fn int_type() -> Self {
Self::abstract_with_constraint(Constraint::Type(PrimitiveType::Int))
}
pub fn float_type() -> Self {
Self::abstract_with_constraint(Constraint::Type(PrimitiveType::Float))
}
pub fn bool_type() -> Self {
Self::abstract_with_constraint(Constraint::Type(PrimitiveType::Bool))
}
pub fn builtin_predicate(name: impl Into<String>) -> Self {
Self::abstract_with_constraint(Constraint::BuiltinPredicate(name.into()))
}
pub fn abstract_with_constraint(constraint: Constraint) -> Self {
Self::Abstract {
constraints: alloc::vec![constraint],
default: None,
}
}
pub fn with_constraint(mut self, constraint: Constraint) -> Self {
match &mut self {
Self::Abstract { constraints, .. } => constraints.push(constraint),
_ => {
self = Self::Abstract {
constraints: alloc::vec![constraint],
default: Some(Box::new(self)),
};
}
}
self
}
pub fn gt(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(CompareOp::Gt, LiteralValue::Int(value)))
}
pub fn gte(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(
CompareOp::Gte,
LiteralValue::Int(value),
))
}
pub fn lt(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(CompareOp::Lt, LiteralValue::Int(value)))
}
pub fn lte(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(
CompareOp::Lte,
LiteralValue::Int(value),
))
}
pub fn default(self, value: HostValue) -> Result<Self> {
match self {
Self::Abstract {
constraints,
default: None,
} => Ok(Self::Abstract {
constraints,
default: Some(Box::new(value)),
}),
Self::Abstract { .. } => Err(Diagnostic::new(
DiagnosticKind::DefaultConflict,
Span::default(),
"host value already has a default",
)),
concrete => Ok(Self::Abstract {
constraints: Vec::new(),
default: Some(Box::new(concrete)),
}),
}
}
pub fn default_string(self, value: impl Into<String>) -> Result<Self> {
self.default(Self::string(value))
}
pub fn default_int(self, value: i64) -> Result<Self> {
self.default(Self::int(value))
}
pub fn default_float(self, value: f64) -> Result<Self> {
self.default(Self::float(value))
}
pub fn default_bool(self, value: bool) -> Result<Self> {
self.default(Self::bool(value))
}
}
+34
View File
@@ -0,0 +1,34 @@
use crate::{Engine, ImportLoader, Result};
/// Host-owned configuration shared by runtime and language tooling.
///
/// Implementations create the import loader and install global bindings on a
/// fresh engine. Calling [`HostEnvironment::create_engine`] therefore gives
/// every consumer the same Decodal execution environment.
pub trait HostEnvironment {
type Loader: ImportLoader;
fn create_loader(&self) -> Self::Loader;
fn configure_engine(&self, _engine: &mut Engine<Self::Loader>) -> Result<()> {
Ok(())
}
fn create_engine(&self) -> Result<Engine<Self::Loader>> {
let mut engine = Engine::new(self.create_loader());
self.configure_engine(&mut engine)?;
Ok(engine)
}
}
impl<T: HostEnvironment + ?Sized> HostEnvironment for &T {
type Loader = T::Loader;
fn create_loader(&self) -> Self::Loader {
T::create_loader(*self)
}
fn configure_engine(&self, engine: &mut Engine<Self::Loader>) -> Result<()> {
T::configure_engine(*self, engine)
}
}
File diff suppressed because it is too large Load Diff
+167 -10
View File
@@ -22,6 +22,7 @@ pub enum TokenKind {
Match,
Import,
Default,
As,
Underscore,
LBrace,
RBrace,
@@ -32,18 +33,45 @@ pub enum TokenKind {
Semicolon,
Comma,
Dot,
Ellipsis,
Colon,
Equal,
EqualEqual,
Bang,
BangEqual,
Arrow,
Amp,
AmpAmp,
PipePipe,
Plus,
PlusPlus,
Minus,
Star,
Slash,
SlashSlash,
Gt,
Gte,
Lt,
Lte,
/// A line comment, including its leading `#` and excluding its newline.
Comment,
Eof,
}
/// Tokenizes source while retaining comments and source spans for tooling.
///
/// Whitespace remains available through the gaps between adjacent token spans,
/// making this a lossless syntax view when paired with the original source.
pub fn tokenize_source(source: &str) -> Result<Vec<Token>> {
tokenize_source_with_source_id(SourceId(0), source)
}
/// Tokenizes source with a caller-provided source identifier while retaining
/// comments and source spans for tooling.
pub fn tokenize_source_with_source_id(source_id: SourceId, source: &str) -> Result<Vec<Token>> {
Lexer::with_source_id(source_id, source).tokenize_with_comments()
}
pub struct Lexer<'a> {
source_id: SourceId,
source: &'a str,
@@ -52,6 +80,7 @@ pub struct Lexer<'a> {
}
impl<'a> Lexer<'a> {
#[cfg(test)]
pub fn new(source: &'a str) -> Self {
Self::with_source_id(SourceId(0), source)
}
@@ -66,10 +95,30 @@ impl<'a> Lexer<'a> {
}
pub fn tokenize(mut self) -> Result<Vec<Token>> {
self.tokenize_impl(false)
}
fn tokenize_with_comments(mut self) -> Result<Vec<Token>> {
self.tokenize_impl(true)
}
fn tokenize_impl(&mut self, include_comments: bool) -> Result<Vec<Token>> {
let mut tokens = Vec::new();
let mut previous = None;
loop {
let token = self.next_token()?;
self.skip_whitespace();
if self.peek() == Some(b'#') {
let comment = self.lex_comment();
if include_comments {
tokens.push(comment);
}
continue;
}
let token = self.next_non_ws_token(previous.as_ref())?;
let is_eof = token.kind == TokenKind::Eof;
if !is_eof {
previous = Some(token.kind.clone());
}
tokens.push(token);
if is_eof {
return Ok(tokens);
@@ -77,8 +126,7 @@ impl<'a> Lexer<'a> {
}
}
fn next_token(&mut self) -> Result<Token> {
self.skip_ws_and_comments();
fn next_non_ws_token(&mut self, previous: Option<&TokenKind>) -> Result<Token> {
let start = self.pos;
let Some(ch) = self.peek() else {
return Ok(Token {
@@ -122,8 +170,19 @@ impl<'a> Lexer<'a> {
}
b'.' => {
self.pos += 1;
if self.consume(b'.') {
if self.consume(b'.') {
TokenKind::Ellipsis
} else {
return Err(Diagnostic::syntax(
self.span(start, self.pos),
"expected third '.' in ellipsis",
));
}
} else {
TokenKind::Dot
}
}
b':' => {
self.pos += 1;
TokenKind::Colon
@@ -134,16 +193,57 @@ impl<'a> Lexer<'a> {
}
b'&' => {
self.pos += 1;
if self.consume(b'&') {
TokenKind::AmpAmp
} else {
TokenKind::Amp
}
}
b'|' => {
self.pos += 1;
if self.consume(b'|') {
TokenKind::PipePipe
} else {
return Err(Diagnostic::syntax(
self.span(start, self.pos),
"expected '|' after '|'",
));
}
}
b'+' => {
self.pos += 1;
if self.consume(b'+') {
TokenKind::PlusPlus
} else {
TokenKind::Plus
}
}
b'-' => {
self.pos += 1;
TokenKind::Minus
}
b'*' => {
self.pos += 1;
TokenKind::Star
}
b'=' => {
self.pos += 1;
if self.consume(b'>') {
TokenKind::Arrow
} else if self.consume(b'=') {
TokenKind::EqualEqual
} else {
TokenKind::Equal
}
}
b'!' => {
self.pos += 1;
if self.consume(b'=') {
TokenKind::BangEqual
} else {
TokenKind::Bang
}
}
b'>' => {
self.pos += 1;
if self.consume(b'=') {
@@ -164,6 +264,8 @@ impl<'a> Lexer<'a> {
self.pos += 1;
if self.consume(b'/') {
TokenKind::SlashSlash
} else if previous.is_some_and(token_can_end_expr) {
TokenKind::Slash
} else {
self.lex_regex(start)?
}
@@ -185,21 +287,23 @@ impl<'a> Lexer<'a> {
})
}
fn skip_ws_and_comments(&mut self) {
loop {
fn skip_whitespace(&mut self) {
while matches!(self.peek(), Some(b' ' | b'\t' | b'\r' | b'\n')) {
self.pos += 1;
}
if self.peek() == Some(b'#') {
}
fn lex_comment(&mut self) -> Token {
let start = self.pos;
while let Some(c) = self.peek() {
self.pos += 1;
if c == b'\n' {
break;
}
self.pos += 1;
}
continue;
}
break;
Token {
kind: TokenKind::Comment,
span: self.span(start, self.pos),
}
}
@@ -303,6 +407,7 @@ impl<'a> Lexer<'a> {
"match" => TokenKind::Match,
"import" => TokenKind::Import,
"default" => TokenKind::Default,
"as" => TokenKind::As,
_ => TokenKind::Ident(String::from(text)),
}
}
@@ -341,6 +446,23 @@ fn is_ident_continue(c: u8) -> bool {
c.is_ascii_alphanumeric() || c == b'_'
}
fn token_can_end_expr(kind: &TokenKind) -> bool {
matches!(
kind,
TokenKind::Ident(_)
| TokenKind::Int(_)
| TokenKind::Float(_)
| TokenKind::String(_)
| TokenKind::Regex(_)
| TokenKind::True
| TokenKind::False
| TokenKind::Underscore
| TokenKind::RBrace
| TokenKind::RBracket
| TokenKind::RParen
)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -353,4 +475,39 @@ mod tests {
assert_eq!(tokens[3].kind, TokenKind::Amp);
assert_eq!(tokens[4].kind, TokenKind::Gte);
}
#[test]
fn tokenizes_array_constraint_ellipsis() {
let tokens = Lexer::new("[...String]").tokenize().unwrap();
assert_eq!(tokens[0].kind, TokenKind::LBracket);
assert_eq!(tokens[1].kind, TokenKind::Ellipsis);
assert!(matches!(tokens[2].kind, TokenKind::Ident(_)));
assert_eq!(tokens[3].kind, TokenKind::RBracket);
}
#[test]
fn tooling_tokens_retain_comments_and_whitespace_gaps() {
let source = "value = 1; # trailing\n\n# leading\nnext = 2;";
let tokens = tokenize_source(source).unwrap();
let comments = tokens
.iter()
.filter(|token| token.kind == TokenKind::Comment)
.collect::<Vec<_>>();
assert_eq!(comments.len(), 2);
assert_eq!(
&source[comments[0].span.start as usize..comments[0].span.end as usize],
"# trailing"
);
assert!(
source[comments[0].span.end as usize..comments[1].span.start as usize].contains("\n\n")
);
let source_id = SourceId(7);
let identified = tokenize_source_with_source_id(source_id, "# comment").unwrap();
assert!(
identified
.iter()
.all(|token| token.span.source == source_id)
);
}
}
+20 -8
View File
@@ -5,24 +5,36 @@ extern crate alloc;
pub mod ast;
pub mod constraints;
pub mod diagnostic;
pub mod embedding;
pub mod environment;
pub mod eval;
pub mod lexer;
mod lexer;
pub mod module;
pub mod parser;
pub mod runtime;
pub mod span;
pub mod typed;
pub mod value;
pub use ast::{Ast, BinaryOp, CompareOp, Expr, ExprId, Field, Literal, Param};
pub use ast::{Ast, BinaryOp, CompareOp, Expr, ExprId, Field, Literal, ObjectRest, Param};
pub use constraints::normalize_constraints;
#[cfg(feature = "derive")]
pub use decodal_derive::Decodal;
pub use diagnostic::{Diagnostic, DiagnosticKind, Result};
pub use embedding::{HostField, HostValue};
pub use eval::Engine;
pub use lexer::{Lexer, Token, TokenKind};
pub use module::{EmptyLoader, LoadedSource, Module, SourceLoader};
pub use environment::HostEnvironment;
pub use eval::{Engine, format_diagnostic_with};
pub use lexer::{
Token as SyntaxToken, TokenKind as SyntaxTokenKind, tokenize_source,
tokenize_source_with_source_id,
};
pub use module::{EmptyLoader, ImportCandidate, ImportLoader, LoadedImport, Module};
pub use parser::{ParseOutput, Parser, SourceForm, parse_source, parse_source_with_source_id};
pub use runtime::{Constraint, Data, ExprRef, LiteralValue, ModuleId, PrimitiveType, RuntimeValue};
pub use runtime::{Constraint, Data, ExprRef, LiteralValue, ModuleId, PrimitiveType};
pub use span::{SourceId, Span};
pub use typed::{
DecodalDecode, DecodalRest, DecodalSchema, DecodeError, DecodeResult, IntoValue, data_at_path,
decode_path, prefix_decode_error,
};
pub use value::{Value, ValueField};
pub fn version() -> &'static str {
env!("CARGO_PKG_VERSION")
+72 -10
View File
@@ -1,7 +1,7 @@
use alloc::string::String;
use alloc::{string::String, vec::Vec};
use crate::{
Ast, ExprId, SourceForm, SourceId,
Ast, ExprId, SourceForm, SourceId, Value,
runtime::{EnvId, ThunkId},
};
@@ -17,22 +17,84 @@ pub struct Module {
pub root_thunk: ThunkId,
}
/// Content resolved by an [`ImportLoader`].
#[derive(Debug, Clone)]
pub struct LoadedSource {
pub key: String,
pub name: String,
pub source: String,
pub enum LoadedImport {
Source {
key: String,
name: String,
source: String,
},
Value {
key: String,
value: Value,
},
}
pub trait SourceLoader {
fn load(&mut self, current_key: Option<&str>, specifier: &str) -> crate::Result<LoadedSource>;
/// An import specifier offered by host-owned language tooling.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ImportCandidate {
pub specifier: String,
pub detail: Option<String>,
}
impl ImportCandidate {
pub fn new(specifier: impl Into<String>) -> Self {
Self {
specifier: specifier.into(),
detail: None,
}
}
pub fn with_detail(mut self, detail: impl Into<String>) -> Self {
self.detail = Some(detail.into());
self
}
}
impl LoadedImport {
pub fn source(
key: impl Into<String>,
name: impl Into<String>,
source: impl Into<String>,
) -> Self {
Self::Source {
key: key.into(),
name: name.into(),
source: source.into(),
}
}
pub fn value(key: impl Into<String>, value: Value) -> Self {
Self::Value {
key: key.into(),
value,
}
}
}
pub trait ImportLoader {
fn load(&mut self, current_key: Option<&str>, specifier: &str) -> crate::Result<LoadedImport>;
/// Returns host-resolvable import specifiers matching an unfinished prefix.
///
/// Runtime-only loaders may keep the default implementation. Hosts that
/// provide an editor should implement this from the same namespace used by
/// [`ImportLoader::load`].
fn complete_import(
&mut self,
_current_key: Option<&str>,
_prefix: &str,
) -> crate::Result<Vec<ImportCandidate>> {
Ok(Vec::new())
}
}
#[derive(Debug, Clone, Copy, Default)]
pub struct EmptyLoader;
impl SourceLoader for EmptyLoader {
fn load(&mut self, _current_key: Option<&str>, specifier: &str) -> crate::Result<LoadedSource> {
impl ImportLoader for EmptyLoader {
fn load(&mut self, _current_key: Option<&str>, specifier: &str) -> crate::Result<LoadedImport> {
Err(crate::Diagnostic::new(
crate::DiagnosticKind::Import,
crate::Span::default(),
+280 -12
View File
@@ -2,7 +2,10 @@ use alloc::{string::String, vec::Vec};
use crate::{
SourceId, Span,
ast::{Ast, BinaryOp, CompareOp, Expr, ExprId, Field, Literal, MatchArm, Param},
ast::{
Ast, BinaryOp, CompareOp, Expr, ExprId, Field, Literal, MatchArm, ObjectRest, Param,
UnaryOp,
},
diagnostic::{Diagnostic, Result},
lexer::{Lexer, Token, TokenKind},
};
@@ -38,7 +41,10 @@ pub struct Parser {
impl Parser {
pub fn new(tokens: Vec<Token>) -> Self {
Self {
tokens,
tokens: tokens
.into_iter()
.filter(|token| token.kind != TokenKind::Comment)
.collect(),
pos: 0,
ast: Ast::new(),
}
@@ -56,7 +62,7 @@ impl Parser {
})
.unwrap_or_else(|| self.peek().span);
(
self.ast.push(Expr::Object(fields), span),
self.ast.push(Expr::Object { fields, rest: None }, span),
SourceForm::Fields,
)
} else {
@@ -115,6 +121,110 @@ impl Parser {
let rhs = self.parse_expr(r_bp)?;
let span = self.ast.span(lhs).join(self.ast.span(rhs)).join(op_span);
lhs = match kind {
InfixKind::Add => self.ast.push(
Expr::Binary {
op: BinaryOp::Add,
lhs,
rhs,
},
span,
),
InfixKind::Sub => self.ast.push(
Expr::Binary {
op: BinaryOp::Sub,
lhs,
rhs,
},
span,
),
InfixKind::Mul => self.ast.push(
Expr::Binary {
op: BinaryOp::Mul,
lhs,
rhs,
},
span,
),
InfixKind::Div => self.ast.push(
Expr::Binary {
op: BinaryOp::Div,
lhs,
rhs,
},
span,
),
InfixKind::Concat => self.ast.push(
Expr::Binary {
op: BinaryOp::Concat,
lhs,
rhs,
},
span,
),
InfixKind::Equal => self.ast.push(
Expr::Binary {
op: BinaryOp::Equal,
lhs,
rhs,
},
span,
),
InfixKind::NotEqual => self.ast.push(
Expr::Binary {
op: BinaryOp::NotEqual,
lhs,
rhs,
},
span,
),
InfixKind::Greater => self.ast.push(
Expr::Binary {
op: BinaryOp::Greater,
lhs,
rhs,
},
span,
),
InfixKind::GreaterEqual => self.ast.push(
Expr::Binary {
op: BinaryOp::GreaterEqual,
lhs,
rhs,
},
span,
),
InfixKind::Less => self.ast.push(
Expr::Binary {
op: BinaryOp::Less,
lhs,
rhs,
},
span,
),
InfixKind::LessEqual => self.ast.push(
Expr::Binary {
op: BinaryOp::LessEqual,
lhs,
rhs,
},
span,
),
InfixKind::LogicalAnd => self.ast.push(
Expr::Binary {
op: BinaryOp::LogicalAnd,
lhs,
rhs,
},
span,
),
InfixKind::LogicalOr => self.ast.push(
Expr::Binary {
op: BinaryOp::LogicalOr,
lhs,
rhs,
},
span,
),
InfixKind::And => self.ast.push(
Expr::Binary {
op: BinaryOp::And,
@@ -138,6 +248,13 @@ impl Parser {
},
span,
),
InfixKind::As => self.ast.push(
Expr::As {
narrower: lhs,
wider: rhs,
},
span,
),
};
}
@@ -173,6 +290,28 @@ impl Parser {
TokenKind::Let => self.parse_let(token.span),
TokenKind::Match => self.parse_match(token.span),
TokenKind::Import => self.parse_import(token.span),
TokenKind::Minus => {
let expr = self.parse_expr(17)?;
let span = token.span.join(self.ast.span(expr));
Ok(self.ast.push(
Expr::Unary {
op: UnaryOp::Neg,
expr,
},
span,
))
}
TokenKind::Bang => {
let expr = self.parse_expr(17)?;
let span = token.span.join(self.ast.span(expr));
Ok(self.ast.push(
Expr::Unary {
op: UnaryOp::Not,
expr,
},
span,
))
}
TokenKind::Gt | TokenKind::Gte | TokenKind::Lt | TokenKind::Lte => {
let op = match token.kind {
TokenKind::Gt => CompareOp::Gt,
@@ -181,7 +320,7 @@ impl Parser {
TokenKind::Lte => CompareOp::Lte,
_ => unreachable!(),
};
let value = self.parse_expr(8)?;
let value = self.parse_expr(12)?;
let span = token.span.join(self.ast.span(value));
Ok(self.ast.push(Expr::CompareConstraint { op, value }, span))
}
@@ -190,28 +329,64 @@ impl Parser {
}
fn parse_object_after_lbrace(&mut self, start_span: Span) -> Result<ExprId> {
if self.consume_kind(&TokenKind::Ellipsis).is_some() {
let value = self.parse_expr(0)?;
self.expect_kind(
&TokenKind::RBrace,
"expected '}' after map value constraint",
)?;
let span = start_span.join(self.previous_span());
return Ok(self.ast.push(Expr::MapConstraint { value }, span));
}
let mut fields = Vec::new();
if self.consume_kind(&TokenKind::RBrace).is_some() {
return Ok(self
.ast
.push(Expr::Object(fields), start_span.join(self.previous_span())));
return Ok(self.ast.push(
Expr::Object { fields, rest: None },
start_span.join(self.previous_span()),
));
}
let mut rest = None;
loop {
fields.push(self.parse_field()?);
if self.consume_kind(&TokenKind::Semicolon).is_some() {
if self.consume_kind(&TokenKind::RBrace).is_some() {
break;
}
if let Some(ellipsis) = self.consume_kind(&TokenKind::Ellipsis) {
let value = self.parse_expr(0)?;
rest = Some(ObjectRest {
value,
span: ellipsis.join(self.ast.span(value)),
});
let _ = self.consume_kind(&TokenKind::Semicolon);
self.expect_kind(
&TokenKind::RBrace,
"expected '}' after object rest constraint",
)?;
break;
}
continue;
}
self.expect_kind(&TokenKind::RBrace, "expected ';' or '}' after object field")?;
break;
}
let span = start_span.join(self.previous_span());
Ok(self.ast.push(Expr::Object(fields), span))
Ok(self.ast.push(Expr::Object { fields, rest }, span))
}
fn parse_array_after_lbracket(&mut self, start_span: Span) -> Result<ExprId> {
if self.consume_kind(&TokenKind::Ellipsis).is_some() {
let item = self.parse_expr(0)?;
let _ = self.consume_kind(&TokenKind::Comma);
self.expect_kind(
&TokenKind::RBracket,
"expected ']' after array element constraint",
)?;
let span = start_span.join(self.previous_span());
return Ok(self.ast.push(Expr::ArrayConstraint { item }, span));
}
let mut items = Vec::new();
if self.consume_kind(&TokenKind::RBracket).is_some() {
return Ok(self
@@ -243,8 +418,9 @@ impl Parser {
}
let expr = self.parse_expr(0)?;
self.expect_kind(&TokenKind::RParen, "expected ')' after expression")?;
Ok(expr)
let end_span = self.expect_kind(&TokenKind::RParen, "expected ')' after expression")?;
let span = start_span.join(end_span);
Ok(self.ast.push(Expr::Parenthesized { expr }, span))
}
fn parse_params_after_lparen(&mut self) -> Result<Vec<Param>> {
@@ -403,9 +579,23 @@ impl Parser {
fn peek_infix(&self) -> Option<(InfixKind, u8, u8)> {
match self.peek_kind() {
TokenKind::As => Some((InfixKind::As, 0, 1)),
TokenKind::Default => Some((InfixKind::Default, 1, 2)),
TokenKind::SlashSlash => Some((InfixKind::Patch, 3, 4)),
TokenKind::Amp => Some((InfixKind::And, 5, 6)),
TokenKind::PipePipe => Some((InfixKind::LogicalOr, 7, 8)),
TokenKind::AmpAmp => Some((InfixKind::LogicalAnd, 9, 10)),
TokenKind::EqualEqual => Some((InfixKind::Equal, 11, 12)),
TokenKind::BangEqual => Some((InfixKind::NotEqual, 11, 12)),
TokenKind::Gt => Some((InfixKind::Greater, 11, 12)),
TokenKind::Gte => Some((InfixKind::GreaterEqual, 11, 12)),
TokenKind::Lt => Some((InfixKind::Less, 11, 12)),
TokenKind::Lte => Some((InfixKind::LessEqual, 11, 12)),
TokenKind::PlusPlus => Some((InfixKind::Concat, 12, 13)),
TokenKind::Plus => Some((InfixKind::Add, 13, 14)),
TokenKind::Minus => Some((InfixKind::Sub, 13, 14)),
TokenKind::Star => Some((InfixKind::Mul, 15, 16)),
TokenKind::Slash => Some((InfixKind::Div, 15, 16)),
_ => None,
}
}
@@ -498,6 +688,20 @@ impl Parser {
#[derive(Debug, Clone, Copy)]
enum InfixKind {
As,
Add,
Sub,
Mul,
Div,
Concat,
Equal,
NotEqual,
Greater,
GreaterEqual,
Less,
LessEqual,
LogicalAnd,
LogicalOr,
And,
Patch,
Default,
@@ -511,15 +715,79 @@ mod tests {
#[test]
fn parses_top_level_fields_as_object() {
let parsed = parse_source("port = Int & >= 1 default 8080;").unwrap();
assert!(matches!(parsed.ast.get(parsed.root).expr, Expr::Object(_)));
assert!(matches!(
parsed.ast.get(parsed.root).expr,
Expr::Object { .. }
));
}
#[test]
fn parses_object_dot_field() {
let parsed = parse_source("{ feature.enable = false; }").unwrap();
let Expr::Object(fields) = &parsed.ast.get(parsed.root).expr else {
let Expr::Object { fields, .. } = &parsed.ast.get(parsed.root).expr else {
panic!()
};
assert_eq!(fields[0].path, ["feature", "enable"]);
}
#[test]
fn parses_array_constraint() {
let parsed = parse_source("[...(String & /^api-/)]").unwrap();
let Expr::ArrayConstraint { item } = parsed.ast.get(parsed.root).expr else {
panic!()
};
let Expr::Parenthesized { expr } = parsed.ast.get(item).expr else {
panic!()
};
assert!(matches!(parsed.ast.get(expr).expr, Expr::Binary { .. }));
assert_eq!(parsed.ast.span(item), Span::new(SourceId(0), 4, 22));
}
#[test]
fn parser_accepts_the_public_lossless_token_stream() {
let tokens = crate::tokenize_source("value = (# note\n 1);").unwrap();
let parsed = Parser::new(tokens).parse().unwrap();
let Expr::Object { fields, .. } = &parsed.ast.get(parsed.root).expr else {
panic!()
};
assert!(matches!(
parsed.ast.get(fields[0].value).expr,
Expr::Parenthesized { .. }
));
}
#[test]
fn rejects_array_constraint_without_element_constraint() {
assert!(parse_source("[...]").is_err());
}
#[test]
fn parses_map_constraint() {
let parsed = parse_source("{...String}").unwrap();
let Expr::MapConstraint { value } = parsed.ast.get(parsed.root).expr else {
panic!()
};
assert!(matches!(parsed.ast.get(value).expr, Expr::Ident(_)));
}
#[test]
fn parses_object_rest_constraint_after_named_fields() {
let parsed = parse_source("{ hoge = Int; fuga = String; ...Unknown }").unwrap();
let Expr::Object { fields, rest } = &parsed.ast.get(parsed.root).expr else {
panic!()
};
assert_eq!(fields.len(), 2);
let rest = rest.as_ref().expect("object has a rest constraint");
assert!(matches!(parsed.ast.get(rest.value).expr, Expr::Ident(_)));
}
#[test]
fn parses_as_below_composition() {
let parsed = parse_source("value & override as Schema").unwrap();
let Expr::As { narrower, wider } = parsed.ast.get(parsed.root).expr else {
panic!()
};
assert!(matches!(parsed.ast.get(narrower).expr, Expr::Binary { .. }));
assert!(matches!(parsed.ast.get(wider).expr, Expr::Ident(_)));
}
}
+24 -2
View File
@@ -1,6 +1,6 @@
use alloc::{string::String, vec::Vec};
use crate::{ExprId, ast::CompareOp};
use crate::{ExprId, Span, ast::CompareOp};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct ModuleId(pub u32);
@@ -11,6 +11,9 @@ pub struct ExprRef {
pub expr: ExprId,
}
/// Evaluator-owned representation of a lazy concrete value or unresolved range.
///
/// Embedders normally construct [`crate::Value`] and consume [`Data`] instead.
#[derive(Debug, Clone, PartialEq)]
pub enum RuntimeValue {
Concrete(ConcreteValue),
@@ -31,12 +34,20 @@ pub enum ConcreteValue {
#[derive(Debug, Clone, PartialEq)]
pub struct ObjectValue {
pub fields: Vec<ObjectField>,
pub rest: Option<ObjectRest>,
}
#[derive(Debug, Clone, PartialEq)]
pub struct ObjectRest {
pub value: ThunkId,
pub span: Span,
}
#[derive(Debug, Clone, PartialEq)]
pub struct ObjectField {
pub name: String,
pub value: ThunkId,
pub span: Span,
}
#[derive(Debug, Clone, PartialEq)]
@@ -54,13 +65,22 @@ pub struct FunctionParam {
#[derive(Debug, Clone, PartialEq)]
pub struct AbstractValue {
pub constraints: Vec<Constraint>,
pub constraints: Vec<ConstraintEntry>,
pub default: Option<ThunkId>,
}
#[derive(Debug, Clone, PartialEq)]
pub struct ConstraintEntry {
pub constraint: Constraint,
pub span: Span,
}
#[derive(Debug, Clone, PartialEq)]
pub enum Constraint {
Unknown,
Type(PrimitiveType),
ArrayItems(ThunkId),
MapValues(ThunkId),
Compare(CompareOp, LiteralValue),
Regex(String),
BuiltinPredicate(String),
@@ -82,6 +102,7 @@ pub enum LiteralValue {
Bool(bool),
}
/// Fully materialized Decodal data.
#[derive(Debug, Clone, PartialEq)]
pub enum Data {
String(String),
@@ -108,6 +129,7 @@ pub struct EnvId(pub u32);
pub struct Thunk {
pub kind: ThunkKind,
pub state: ThunkState,
pub span: Span,
}
#[derive(Debug, Clone)]
+445
View File
@@ -0,0 +1,445 @@
use alloc::{collections::BTreeMap, format, string::String, vec::Vec};
use crate::{Data, Value, runtime::DataField};
pub trait DecodalSchema {
fn decodal_schema() -> Value;
}
pub trait DecodalDecode: Sized {
fn decodal_decode(data: &Data) -> DecodeResult<Self>;
}
pub trait IntoValue {
fn into_value(self) -> Value;
}
/// A map-like receiver for fields not named by a derived struct schema.
pub trait DecodalRest: Sized {
fn decodal_rest_schema() -> Value;
fn decodal_decode_rest(data: &Data, known_fields: &[&str]) -> DecodeResult<Self>;
}
pub type DecodeResult<T> = core::result::Result<T, DecodeError>;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DecodeError {
pub path: String,
pub message: String,
}
impl DecodeError {
pub fn new(path: impl Into<String>, message: impl Into<String>) -> Self {
Self {
path: path.into(),
message: message.into(),
}
}
pub fn at_type(path: &str, expected: &'static str) -> Self {
Self::new(path, format!("expected {expected}"))
}
}
impl core::fmt::Display for DecodeError {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
if self.path.is_empty() {
write!(f, "{}", self.message)
} else {
write!(f, "{}: {}", self.path, self.message)
}
}
}
#[cfg(feature = "std")]
impl std::error::Error for DecodeError {}
pub fn decode_path<T: DecodalDecode>(data: &Data, path: &str) -> DecodeResult<T> {
let value = data_at_path(data, path).ok_or_else(|| DecodeError::new(path, "missing field"))?;
T::decodal_decode(value).map_err(|error| prefix_error(path, error))
}
pub fn data_at_path<'a>(data: &'a Data, path: &str) -> Option<&'a Data> {
let mut current = data;
for part in path.split('.') {
if part.is_empty() {
return None;
}
let Data::Object(fields) = current else {
return None;
};
current = &fields.iter().find(|field| field.name == part)?.value;
}
Some(current)
}
pub fn prefix_decode_error(path: &str, error: DecodeError) -> DecodeError {
prefix_error(path, error)
}
fn prefix_error(path: &str, mut error: DecodeError) -> DecodeError {
if error.path.is_empty() {
error.path = String::from(path);
} else if !path.is_empty() {
error.path = format!("{path}.{}", error.path);
}
error
}
impl DecodalSchema for Data {
fn decodal_schema() -> Value {
Value::unknown()
}
}
impl DecodalDecode for Data {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
Ok(data.clone())
}
}
impl IntoValue for Data {
fn into_value(self) -> Value {
match self {
Data::String(value) => Value::string(value),
Data::Int(value) => Value::int(value),
Data::Float(value) => Value::float(value),
Data::Bool(value) => Value::bool(value),
Data::Array(items) => Value::array(items.into_iter().map(IntoValue::into_value)),
Data::Object(fields) => Value::object(
fields
.into_iter()
.map(|field| (field.name, field.value.into_value())),
),
}
}
}
impl DecodalSchema for String {
fn decodal_schema() -> Value {
Value::string_type()
}
}
impl DecodalDecode for String {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::String(value) => Ok(value.clone()),
_ => Err(DecodeError::at_type("", "String")),
}
}
}
impl IntoValue for String {
fn into_value(self) -> Value {
Value::string(self)
}
}
impl IntoValue for &str {
fn into_value(self) -> Value {
Value::string(self)
}
}
macro_rules! impl_int_decode {
($($ty:ty),* $(,)?) => {
$(
impl DecodalSchema for $ty {
fn decodal_schema() -> Value {
Value::int_type()
}
}
impl DecodalDecode for $ty {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Int(value) => <$ty>::try_from(*value)
.map_err(|_| DecodeError::new("", "integer value is out of range")),
_ => Err(DecodeError::at_type("", "Int")),
}
}
}
impl IntoValue for $ty {
fn into_value(self) -> Value {
Value::int(self as i64)
}
}
)*
};
}
impl_int_decode!(i8, i16, i32, i64, u8, u16, u32);
impl DecodalSchema for f64 {
fn decodal_schema() -> Value {
Value::float_type()
}
}
impl DecodalDecode for f64 {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Float(value) => Ok(*value),
Data::Int(value) => Ok(*value as f64),
_ => Err(DecodeError::at_type("", "Float")),
}
}
}
impl IntoValue for f64 {
fn into_value(self) -> Value {
Value::float(self)
}
}
impl DecodalSchema for f32 {
fn decodal_schema() -> Value {
Value::float_type()
}
}
impl DecodalDecode for f32 {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
f64::decodal_decode(data).map(|value| value as f32)
}
}
impl IntoValue for f32 {
fn into_value(self) -> Value {
Value::float(f64::from(self))
}
}
impl DecodalSchema for bool {
fn decodal_schema() -> Value {
Value::bool_type()
}
}
impl DecodalDecode for bool {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Bool(value) => Ok(*value),
_ => Err(DecodeError::at_type("", "Bool")),
}
}
}
impl IntoValue for bool {
fn into_value(self) -> Value {
Value::bool(self)
}
}
impl<T: DecodalSchema> DecodalSchema for Vec<T> {
fn decodal_schema() -> Value {
Value::array_of(T::decodal_schema())
}
}
impl<T: DecodalDecode> DecodalDecode for Vec<T> {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Array(items) => items
.iter()
.enumerate()
.map(|(index, item)| {
T::decodal_decode(item)
.map_err(|error| prefix_error(&format!("[{index}]"), error))
})
.collect(),
_ => Err(DecodeError::at_type("", "Array")),
}
}
}
impl<T: IntoValue> IntoValue for Vec<T> {
fn into_value(self) -> Value {
Value::array(self.into_iter().map(IntoValue::into_value))
}
}
impl<T: DecodalSchema> DecodalSchema for BTreeMap<String, T> {
fn decodal_schema() -> Value {
Value::map_of(T::decodal_schema())
}
}
impl<T: DecodalDecode> DecodalDecode for BTreeMap<String, T> {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Object(fields) => fields
.iter()
.map(|field| {
T::decodal_decode(&field.value)
.map(|value| (field.name.clone(), value))
.map_err(|error| prefix_error(&field.name, error))
})
.collect(),
_ => Err(DecodeError::at_type("", "Object")),
}
}
}
impl<T: IntoValue> IntoValue for BTreeMap<String, T> {
fn into_value(self) -> Value {
Value::object(
self.into_iter()
.map(|(name, value)| (name, value.into_value())),
)
}
}
impl<T: DecodalSchema + DecodalDecode> DecodalRest for BTreeMap<String, T> {
fn decodal_rest_schema() -> Value {
T::decodal_schema()
}
fn decodal_decode_rest(data: &Data, known_fields: &[&str]) -> DecodeResult<Self> {
let Data::Object(fields) = data else {
return Err(DecodeError::at_type("", "Object"));
};
fields
.iter()
.filter(|field| !known_fields.contains(&field.name.as_str()))
.map(decode_rest_field::<T>)
.collect()
}
}
#[cfg(feature = "std")]
impl<T: DecodalSchema> DecodalSchema for std::collections::HashMap<String, T> {
fn decodal_schema() -> Value {
Value::map_of(T::decodal_schema())
}
}
#[cfg(feature = "std")]
impl<T: DecodalDecode> DecodalDecode for std::collections::HashMap<String, T> {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Object(fields) => fields
.iter()
.map(|field| {
T::decodal_decode(&field.value)
.map(|value| (field.name.clone(), value))
.map_err(|error| prefix_error(&field.name, error))
})
.collect(),
_ => Err(DecodeError::at_type("", "Object")),
}
}
}
#[cfg(feature = "std")]
impl<T: IntoValue> IntoValue for std::collections::HashMap<String, T> {
fn into_value(self) -> Value {
let mut fields: Vec<_> = self
.into_iter()
.map(|(name, value)| (name, value.into_value()))
.collect();
fields.sort_by(|left, right| left.0.cmp(&right.0));
Value::object(fields)
}
}
#[cfg(feature = "std")]
impl<T: DecodalSchema + DecodalDecode> DecodalRest for std::collections::HashMap<String, T> {
fn decodal_rest_schema() -> Value {
T::decodal_schema()
}
fn decodal_decode_rest(data: &Data, known_fields: &[&str]) -> DecodeResult<Self> {
let Data::Object(fields) = data else {
return Err(DecodeError::at_type("", "Object"));
};
fields
.iter()
.filter(|field| !known_fields.contains(&field.name.as_str()))
.map(decode_rest_field::<T>)
.collect()
}
}
fn decode_rest_field<T: DecodalDecode>(field: &DataField) -> DecodeResult<(String, T)> {
T::decodal_decode(&field.value)
.map(|value| (field.name.clone(), value))
.map_err(|error| prefix_error(&field.name, error))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::runtime::{Constraint, DataField, PrimitiveType};
#[test]
fn string_maps_expose_map_schemas_and_decode_objects() {
assert!(matches!(
BTreeMap::<String, i64>::decodal_schema(),
Value::MapRange { value, .. }
if matches!(
*value,
Value::Range { ref constraints, .. }
if constraints == &[Constraint::Type(PrimitiveType::Int)]
)
));
let data = Data::Object(alloc::vec![DataField {
name: String::from("api"),
value: Data::Int(8080),
}]);
let decoded = BTreeMap::<String, i64>::decodal_decode(&data).unwrap();
assert_eq!(decoded.get("api"), Some(&8080));
}
#[test]
fn data_is_the_unknown_schema_and_round_trips_values() {
assert!(matches!(
Data::decodal_schema(),
Value::Range { ref constraints, .. }
if constraints == &[Constraint::Unknown]
));
let data = Data::Object(alloc::vec![DataField {
name: String::from("nested"),
value: Data::Array(alloc::vec![Data::Bool(true), Data::Int(2)]),
}]);
assert_eq!(Data::decodal_decode(&data).unwrap(), data);
assert!(matches!(data.into_value(), Value::Object { .. }));
}
#[test]
fn rest_maps_decode_only_fields_outside_the_known_domain() {
let data = Data::Object(alloc::vec![
DataField {
name: String::from("name"),
value: Data::String(String::from("api")),
},
DataField {
name: String::from("priority"),
value: Data::Int(3),
},
]);
let decoded = BTreeMap::<String, Data>::decodal_decode_rest(&data, &["name"]).unwrap();
assert_eq!(decoded.len(), 1);
assert_eq!(decoded.get("priority"), Some(&Data::Int(3)));
}
#[cfg(feature = "std")]
#[test]
fn hash_maps_can_receive_rest_fields() {
let data = Data::Object(alloc::vec![DataField {
name: String::from("region"),
value: Data::String(String::from("ap-northeast-1")),
}]);
let decoded =
std::collections::HashMap::<String, String>::decodal_decode_rest(&data, &[]).unwrap();
assert_eq!(
decoded.get("region").map(String::as_str),
Some("ap-northeast-1")
);
}
}
+333
View File
@@ -0,0 +1,333 @@
use alloc::{boxed::Box, string::String, vec::Vec};
use crate::runtime::{Constraint, LiteralValue, PrimitiveType};
use crate::{CompareOp, Diagnostic, DiagnosticKind, Result, Span};
/// A Decodal value supplied through the embedding API.
///
/// Unlike [`crate::Data`], a `Value` may contain unresolved ranges and
/// defaults. The evaluator converts it into its lazy runtime representation
/// when it is bound as a global or returned from an import loader.
#[derive(Debug, Clone, PartialEq)]
pub enum Value {
String(String),
Int(i64),
Float(f64),
Bool(bool),
Array(Vec<Value>),
ArrayRange {
item: Box<Value>,
constraints: Vec<Constraint>,
default: Option<Box<Value>>,
},
MapRange {
value: Box<Value>,
constraints: Vec<Constraint>,
default: Option<Box<Value>>,
},
Object {
fields: Vec<ValueField>,
rest: Option<Box<Value>>,
},
Range {
constraints: Vec<Constraint>,
default: Option<Box<Value>>,
},
}
/// A named field in a [`Value::Object`].
#[derive(Debug, Clone, PartialEq)]
pub struct ValueField {
pub name: String,
pub value: Value,
}
impl Value {
pub fn string(value: impl Into<String>) -> Self {
Self::String(value.into())
}
pub fn int(value: i64) -> Self {
Self::Int(value)
}
pub fn float(value: f64) -> Self {
Self::Float(value)
}
pub fn bool(value: bool) -> Self {
Self::Bool(value)
}
pub fn array<I>(items: I) -> Self
where
I: IntoIterator<Item = Value>,
{
Self::Array(items.into_iter().collect())
}
pub fn object<I, N>(fields: I) -> Self
where
I: IntoIterator<Item = (N, Value)>,
N: Into<String>,
{
Self::Object {
fields: fields
.into_iter()
.map(|(name, value)| ValueField {
name: name.into(),
value,
})
.collect(),
rest: None,
}
}
pub fn object_from_paths<I, N>(fields: I) -> Self
where
I: IntoIterator<Item = (N, Value)>,
N: Into<String>,
{
let mut root = Vec::new();
for (path, value) in fields {
insert_path(&mut root, &path.into(), value);
}
Self::Object {
fields: root,
rest: None,
}
}
pub fn object_from_paths_with_rest<I, N>(fields: I, rest: Value) -> Self
where
I: IntoIterator<Item = (N, Value)>,
N: Into<String>,
{
let mut root = Vec::new();
for (path, value) in fields {
insert_path(&mut root, &path.into(), value);
}
Self::Object {
fields: root,
rest: Some(Box::new(rest)),
}
}
pub fn object_with_rest<I, N>(fields: I, rest: Value) -> Self
where
I: IntoIterator<Item = (N, Value)>,
N: Into<String>,
{
Self::Object {
fields: fields
.into_iter()
.map(|(name, value)| ValueField {
name: name.into(),
value,
})
.collect(),
rest: Some(Box::new(rest)),
}
}
pub fn unknown() -> Self {
Self::range_with_constraint(Constraint::Unknown)
}
pub fn string_type() -> Self {
Self::range_with_constraint(Constraint::Type(PrimitiveType::String))
}
pub fn int_type() -> Self {
Self::range_with_constraint(Constraint::Type(PrimitiveType::Int))
}
pub fn float_type() -> Self {
Self::range_with_constraint(Constraint::Type(PrimitiveType::Float))
}
pub fn bool_type() -> Self {
Self::range_with_constraint(Constraint::Type(PrimitiveType::Bool))
}
pub fn array_of(item: Value) -> Self {
Self::ArrayRange {
item: Box::new(item),
constraints: Vec::new(),
default: None,
}
}
pub fn map_of(value: Value) -> Self {
Self::MapRange {
value: Box::new(value),
constraints: Vec::new(),
default: None,
}
}
pub fn builtin_predicate(name: impl Into<String>) -> Self {
Self::range_with_constraint(Constraint::BuiltinPredicate(name.into()))
}
pub fn range_with_constraint(constraint: Constraint) -> Self {
Self::Range {
constraints: alloc::vec![constraint],
default: None,
}
}
pub fn with_constraint(mut self, constraint: Constraint) -> Self {
match &mut self {
Self::Range { constraints, .. } => constraints.push(constraint),
Self::ArrayRange { constraints, .. } | Self::MapRange { constraints, .. } => {
constraints.push(constraint)
}
_ => {
self = Self::Range {
constraints: alloc::vec![constraint],
default: Some(Box::new(self)),
};
}
}
self
}
pub fn gt(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(CompareOp::Gt, LiteralValue::Int(value)))
}
pub fn gte(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(
CompareOp::Gte,
LiteralValue::Int(value),
))
}
pub fn lt(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(CompareOp::Lt, LiteralValue::Int(value)))
}
pub fn lte(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(
CompareOp::Lte,
LiteralValue::Int(value),
))
}
pub fn default(self, value: Value) -> Result<Self> {
match self {
Self::ArrayRange {
item,
constraints,
default: None,
} => Ok(Self::ArrayRange {
item,
constraints,
default: Some(Box::new(value)),
}),
Self::ArrayRange {
default: Some(_), ..
} => Err(Diagnostic::new(
DiagnosticKind::DefaultConflict,
Span::default(),
"value already has a default",
)),
Self::MapRange {
value: map_value,
constraints,
default: None,
} => Ok(Self::MapRange {
value: map_value,
constraints,
default: Some(Box::new(value)),
}),
Self::MapRange {
default: Some(_), ..
} => Err(Diagnostic::new(
DiagnosticKind::DefaultConflict,
Span::default(),
"value already has a default",
)),
Self::Range {
constraints,
default: None,
} => Ok(Self::Range {
constraints,
default: Some(Box::new(value)),
}),
Self::Range { .. } => Err(Diagnostic::new(
DiagnosticKind::DefaultConflict,
Span::default(),
"value already has a default",
)),
concrete => Ok(Self::Range {
constraints: Vec::new(),
default: Some(Box::new(concrete)),
}),
}
}
pub fn default_string(self, value: impl Into<String>) -> Result<Self> {
self.default(Self::string(value))
}
pub fn default_int(self, value: i64) -> Result<Self> {
self.default(Self::int(value))
}
pub fn default_float(self, value: f64) -> Result<Self> {
self.default(Self::float(value))
}
pub fn default_bool(self, value: bool) -> Result<Self> {
self.default(Self::bool(value))
}
}
impl ValueField {
pub fn new(name: impl Into<String>, value: Value) -> Self {
Self {
name: name.into(),
value,
}
}
}
fn insert_path(fields: &mut Vec<ValueField>, path: &str, value: Value) {
let mut parts = path.splitn(2, '.');
let Some(head) = parts.next().filter(|part| !part.is_empty()) else {
return;
};
if let Some(tail) = parts.next() {
if let Some(field) = fields.iter_mut().find(|field| field.name == head) {
if let Value::Object {
fields: children, ..
} = &mut field.value
{
insert_path(children, tail, value);
} else {
let mut children = Vec::new();
insert_path(&mut children, tail, value);
field.value = Value::Object {
fields: children,
rest: None,
};
}
} else {
let mut children = Vec::new();
insert_path(&mut children, tail, value);
fields.push(ValueField::new(
head,
Value::Object {
fields: children,
rest: None,
},
));
}
} else if let Some(field) = fields.iter_mut().find(|field| field.name == head) {
field.value = value;
} else {
fields.push(ValueField::new(head, value));
}
}
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "decodal-derive"
version = "0.2.0"
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Derive macro for generating Decodal schemas and typed decoders from Rust structs."
keywords = ["decodal", "derive", "proc-macro", "schema"]
categories = ["config", "development-tools::procedural-macro-helpers"]
[lib]
proc-macro = true
[dependencies]
proc-macro2 = "1"
quote = "1"
syn = { version = "2", features = ["full"] }
[dev-dependencies]
decodal = { path = "../decodal-core" }
+313
View File
@@ -0,0 +1,313 @@
use proc_macro::TokenStream;
use proc_macro2::TokenStream as TokenStream2;
use quote::{format_ident, quote};
use syn::{
Data, DeriveInput, Expr, Fields, LitStr, Result, Type, parse_macro_input, spanned::Spanned,
};
#[proc_macro_derive(Decodal, attributes(decodal))]
pub fn derive_decodal(input: TokenStream) -> TokenStream {
match expand_decodal(parse_macro_input!(input as DeriveInput)) {
Ok(tokens) => tokens.into(),
Err(error) => error.to_compile_error().into(),
}
}
fn expand_decodal(input: DeriveInput) -> Result<TokenStream2> {
let name = input.ident;
let fields = match input.data {
Data::Struct(data) => match data.fields {
Fields::Named(fields) => fields.named,
_ => {
return Err(syn::Error::new(
data.struct_token.span(),
"Decodal derive supports structs with named fields only",
));
}
},
_ => {
return Err(syn::Error::new(
name.span(),
"Decodal derive supports structs only",
));
}
};
let mut schema_fields = Vec::new();
let mut decode_fields = Vec::new();
let mut known_field_roots = Vec::new();
let mut rest_field = None;
for field in fields {
let ident = field.ident.expect("named field");
let ty = field.ty;
let attrs = FieldAttrs::from_attrs(&field.attrs, &ident.to_string())?;
if attrs.rest {
if attrs.rename_explicit || attrs.default.is_some() || !attrs.constraints.is_empty() {
return Err(syn::Error::new(
ident.span(),
"`rest` cannot be combined with rename, default, or field constraints",
));
}
if rest_field.is_some() {
return Err(syn::Error::new(
ident.span(),
"Decodal derive supports only one `rest` field",
));
}
rest_field = Some((ident, ty));
continue;
}
let path = attrs.rename.clone();
known_field_roots.push(
path.split('.')
.next()
.expect("field paths are non-empty")
.to_owned(),
);
let schema_value = schema_expr(&ty, &attrs)?;
let decode_value = decode_expr(&ty, &path, &attrs);
schema_fields.push(quote! {
(#path, { #schema_value })
});
decode_fields.push(quote! {
#ident: #decode_value
});
}
if let Some((ident, ty)) = &rest_field {
decode_fields.push(quote! {
#ident: <#ty as ::decodal::DecodalRest>::decodal_decode_rest(
data,
&[#(#known_field_roots),*],
)?
});
}
let schema_entries = if schema_fields.is_empty() {
quote! { ::core::iter::empty::<(&'static str, ::decodal::Value)>() }
} else {
quote! { [#(#schema_fields),*] }
};
let build_schema = if let Some((_, ty)) = &rest_field {
quote! {
::decodal::Value::object_from_paths_with_rest(
#schema_entries,
<#ty as ::decodal::DecodalRest>::decodal_rest_schema(),
)
}
} else {
quote! {
::decodal::Value::object_from_paths(#schema_entries)
}
};
let (impl_generics, ty_generics, where_clause) = input.generics.split_for_impl();
Ok(quote! {
impl #impl_generics ::decodal::DecodalSchema for #name #ty_generics #where_clause {
fn decodal_schema() -> ::decodal::Value {
#build_schema
}
}
impl #impl_generics ::decodal::DecodalDecode for #name #ty_generics #where_clause {
fn decodal_decode(data: &::decodal::Data) -> ::decodal::DecodeResult<Self> {
Ok(Self {
#(#decode_fields),*
})
}
}
})
}
#[derive(Default)]
struct FieldAttrs {
rename: String,
rename_explicit: bool,
rest: bool,
default: Option<DefaultAttr>,
constraints: Vec<ConstraintAttr>,
}
impl FieldAttrs {
fn from_attrs(attrs: &[syn::Attribute], fallback_name: &str) -> Result<Self> {
let mut output = Self {
rename: fallback_name.to_string(),
..Self::default()
};
for attr in attrs {
if !attr.path().is_ident("decodal") {
continue;
}
attr.parse_nested_meta(|meta| {
if meta.path.is_ident("rename") {
let value: LitStr = meta.value()?.parse()?;
output.rename = value.value();
output.rename_explicit = true;
return Ok(());
}
if meta.path.is_ident("rest") {
if meta.input.peek(syn::Token![=]) || meta.input.peek(syn::token::Paren) {
return Err(meta.error("`rest` does not accept a value"));
}
output.rest = true;
return Ok(());
}
if meta.path.is_ident("default") {
if meta.input.peek(syn::Token![=]) {
let expr: Expr = meta.value()?.parse()?;
output.default = Some(DefaultAttr::Expr(expr));
} else {
output.default = Some(DefaultAttr::Default);
}
return Ok(());
}
if let Some(kind) = ConstraintKind::from_path(&meta.path) {
let expr: Expr = meta.value()?.parse()?;
output.constraints.push(ConstraintAttr { kind, expr });
return Ok(());
}
Err(meta.error("unsupported decodal attribute"))
})?;
}
Ok(output)
}
}
enum DefaultAttr {
Default,
Expr(Expr),
}
struct ConstraintAttr {
kind: ConstraintKind,
expr: Expr,
}
#[derive(Clone, Copy)]
enum ConstraintKind {
Gt,
Gte,
Lt,
Lte,
}
impl ConstraintKind {
fn from_path(path: &syn::Path) -> Option<Self> {
if path.is_ident("gt") {
Some(Self::Gt)
} else if path.is_ident("gte") {
Some(Self::Gte)
} else if path.is_ident("lt") {
Some(Self::Lt)
} else if path.is_ident("lte") {
Some(Self::Lte)
} else {
None
}
}
fn method(self) -> syn::Ident {
match self {
Self::Gt => format_ident!("gt"),
Self::Gte => format_ident!("gte"),
Self::Lt => format_ident!("lt"),
Self::Lte => format_ident!("lte"),
}
}
}
fn schema_expr(ty: &Type, attrs: &FieldAttrs) -> Result<TokenStream2> {
let mut tokens = quote! {
let mut schema = <#ty as ::decodal::DecodalSchema>::decodal_schema();
};
for constraint in &attrs.constraints {
let method = constraint.kind.method();
let expr = &constraint.expr;
tokens.extend(quote! {
schema = schema.#method(#expr);
});
}
if let Some(default) = &attrs.default {
let default_expr = match default {
DefaultAttr::Default => quote! { <#ty as ::core::default::Default>::default() },
DefaultAttr::Expr(expr) => quote! { (#expr) },
};
tokens.extend(quote! {
schema = schema
.default(::decodal::IntoValue::into_value(#default_expr))
.expect("Decodal derive generated a valid default");
});
}
tokens.extend(quote! { schema });
Ok(tokens)
}
fn decode_expr(ty: &Type, path: &str, attrs: &FieldAttrs) -> TokenStream2 {
let missing = if let Some(default) = &attrs.default {
match default {
DefaultAttr::Default => quote! { <#ty as ::core::default::Default>::default() },
DefaultAttr::Expr(expr) => quote! { ::core::convert::Into::into(#expr) },
}
} else {
quote! { return Err(::decodal::DecodeError::new(#path, "missing field")); }
};
quote! {
if let Some(value) = ::decodal::data_at_path(data, #path) {
<#ty as ::decodal::DecodalDecode>::decodal_decode(value)
.map_err(|error| ::decodal::prefix_decode_error(#path, error))?
} else {
#missing
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn rejects_multiple_rest_fields() {
let input: DeriveInput = syn::parse_quote! {
struct Invalid {
#[decodal(rest)]
first: std::collections::BTreeMap<String, String>,
#[decodal(rest)]
second: std::collections::BTreeMap<String, String>,
}
};
assert!(
expand_decodal(input)
.unwrap_err()
.to_string()
.contains("only one `rest` field")
);
}
#[test]
fn rejects_rest_field_modifiers() {
let input: DeriveInput = syn::parse_quote! {
struct Invalid {
#[decodal(rest, default)]
extra: std::collections::BTreeMap<String, String>,
}
};
assert!(
expand_decodal(input)
.unwrap_err()
.to_string()
.contains("cannot be combined")
);
}
}
+268
View File
@@ -0,0 +1,268 @@
use std::collections::BTreeMap;
use decodal::{Constraint, Data, DecodalDecode, DecodalSchema, EmptyLoader, Engine, Value};
use decodal_derive::Decodal;
#[derive(Debug, PartialEq, Decodal)]
struct Service {
name: String,
#[decodal(gt = 443, default = 8443)]
port: i64,
#[decodal(default = true, rename = "feature.enable")]
feature_enabled: bool,
#[decodal(default)]
tags: Vec<String>,
}
#[derive(Debug, PartialEq, Decodal)]
struct Worker {
name: String,
#[decodal(default = true)]
enabled: bool,
}
#[derive(Debug, PartialEq, Decodal)]
struct Fleet {
workers: Vec<Worker>,
}
#[derive(Debug, PartialEq, Decodal)]
struct OpenConfig {
name: String,
#[decodal(default = true, rename = "feature.enabled")]
enabled: bool,
#[decodal(rest)]
extra: BTreeMap<String, Data>,
}
#[derive(Debug, PartialEq, Decodal)]
struct StringLabels {
name: String,
#[decodal(rest)]
labels: BTreeMap<String, String>,
}
#[derive(Debug, PartialEq, Decodal)]
struct OpenBag {
#[decodal(rest)]
values: BTreeMap<String, Data>,
}
#[test]
fn derives_schema_and_decode() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("Service", Service::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
Service & {
name = "api";
port = 9443;
feature.enable = false;
tags = ["web", "prod"];
}
"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let service = Service::decodal_decode(&data).unwrap();
assert_eq!(
service,
Service {
name: "api".into(),
port: 9443,
feature_enabled: false,
tags: vec!["web".into(), "prod".into()],
}
);
}
#[test]
fn defaults_are_available_in_schema() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("Service", Service::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
Service & {
name = "api";
}
"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let service = Service::decodal_decode(&data).unwrap();
assert_eq!(service.port, 8443);
assert!(service.feature_enabled);
assert!(service.tags.is_empty());
}
#[test]
fn decode_reports_field_path() {
let data = Data::Object(vec![]);
let error = Service::decodal_decode(&data).unwrap_err();
assert_eq!(error.path, "name");
}
#[test]
fn vec_schema_rejects_invalid_element_before_decode() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("Service", Service::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
Service & {
name = "api";
port = 9443;
tags = ["web", 1];
}
"#,
)
.unwrap();
let error = engine.eval_module(module).unwrap_err();
assert!(
error
.labels
.iter()
.any(|label| label.message.contains("array element [1]"))
);
}
#[test]
fn vec_schema_applies_nested_struct_defaults() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("Fleet", Fleet::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
Fleet & {
workers = [{ name = "api"; }];
}
"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let fleet = Fleet::decodal_decode(&data).unwrap();
assert_eq!(
fleet,
Fleet {
workers: vec![Worker {
name: "api".into(),
enabled: true,
}],
}
);
}
#[test]
fn rest_map_opens_the_derived_object_and_collects_additional_fields() {
let schema = OpenConfig::decodal_schema();
let Value::Object {
fields,
rest: Some(rest),
} = schema
else {
panic!("derived schema should have an object rest range")
};
assert_eq!(fields.len(), 2);
assert!(matches!(
*rest,
Value::Range { ref constraints, .. }
if constraints == &[Constraint::Unknown]
));
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("OpenConfig", OpenConfig::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
{
name = "api";
plugin = { name = "cache"; };
retries = 3;
} as OpenConfig
"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let config = OpenConfig::decodal_decode(&data).unwrap();
assert_eq!(config.name, "api");
assert!(config.enabled);
assert_eq!(config.extra.get("retries"), Some(&Data::Int(3)));
assert!(matches!(config.extra.get("plugin"), Some(Data::Object(_))));
assert!(!config.extra.contains_key("name"));
assert!(!config.extra.contains_key("feature"));
}
#[test]
fn typed_rest_map_constrains_each_additional_field() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("StringLabels", StringLabels::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"{ name = "service"; region = "ap-northeast-1"; tier = "edge"; } as StringLabels"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let labels = StringLabels::decodal_decode(&data).unwrap();
assert_eq!(labels.labels.get("region"), Some(&"ap-northeast-1".into()));
assert_eq!(labels.labels.get("tier"), Some(&"edge".into()));
let module = engine
.add_root_source(
"invalid",
"invalid",
r#"{ name = "service"; priority = 1; } as StringLabels"#,
)
.unwrap();
assert!(engine.eval_module(module).is_err());
}
#[test]
fn rest_only_struct_collects_every_field() {
let schema = OpenBag::decodal_schema();
assert!(matches!(
schema,
Value::Object {
ref fields,
rest: Some(_),
} if fields.is_empty()
));
let data = Data::Object(vec![decodal::runtime::DataField {
name: "answer".into(),
value: Data::Int(42),
}]);
let bag = OpenBag::decodal_decode(&data).unwrap();
assert_eq!(bag.values.get("answer"), Some(&Data::Int(42)));
}
@@ -0,0 +1,14 @@
[package]
name = "decodal-language-service"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Host-configurable semantic language service for Decodal."
keywords = ["decodal", "lsp", "language-server", "editor"]
categories = ["development-tools", "text-editors"]
[dependencies]
decodal = { version = "0.4.0", path = "../decodal-core" }
@@ -0,0 +1,913 @@
use std::collections::{BTreeMap, HashMap};
use decodal::{
Engine, HostEnvironment, ImportLoader, LoadedImport, Result, Value, runtime::RuntimeValue,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CompletionKind {
Keyword,
Constant,
Type,
Variable,
Namespace,
Property,
File,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CompletionItem {
pub label: String,
pub kind: CompletionKind,
pub detail: Option<String>,
pub priority: i32,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CompletionResult {
/// UTF-8 byte offset at which the unfinished token starts.
pub from: usize,
pub items: Vec<CompletionItem>,
}
#[derive(Debug, Clone, Default, PartialEq, Eq)]
struct FieldTree(BTreeMap<String, FieldTree>);
impl FieldTree {
fn new() -> Self {
Self::default()
}
fn is_empty(&self) -> bool {
self.0.is_empty()
}
fn insert(&mut self, name: String, fields: FieldTree) {
self.0.insert(name, fields);
}
fn get(&self, name: &str) -> Option<&FieldTree> {
self.0.get(name)
}
fn remove(&mut self, name: &str) -> Option<FieldTree> {
self.0.remove(name)
}
}
impl IntoIterator for FieldTree {
type Item = (String, FieldTree);
type IntoIter = std::collections::btree_map::IntoIter<String, FieldTree>;
fn into_iter(self) -> Self::IntoIter {
self.0.into_iter()
}
}
impl FromIterator<(String, FieldTree)> for FieldTree {
fn from_iter<T: IntoIterator<Item = (String, FieldTree)>>(iter: T) -> Self {
Self(iter.into_iter().collect())
}
}
pub(crate) fn complete<E: HostEnvironment>(
environment: &E,
key: &str,
source: &str,
position: usize,
explicit: bool,
) -> Result<Option<CompletionResult>> {
let position = floor_char_boundary(source, position.min(source.len()));
if let Some((from, prefix)) = unfinished_import(source, position) {
let mut loader = environment.create_loader();
let items = loader
.complete_import(Some(key), prefix)?
.into_iter()
.map(|candidate| CompletionItem {
label: candidate.specifier,
kind: CompletionKind::File,
detail: candidate.detail,
priority: 30,
})
.collect();
return Ok(Some(CompletionResult {
from,
items: unique_items(items),
}));
}
if in_string_or_comment(source, position) {
return Ok(None);
}
let tokens = tokenize(source);
let local_fields = collect_fields(&tokens);
let imports = collect_import_bindings(&tokens);
let globals = collect_globals(environment)?;
if let Some((from, path)) = member_path(source, position) {
let mut parts = path.split('.');
let Some(root) = parts.next() else {
return Ok(None);
};
let mut fields;
let detail;
if let Some(specifier) = imports.get(root) {
let mut loader = environment.create_loader();
let loaded = loader.load(Some(key), specifier)?;
detail = match &loaded {
LoadedImport::Source { name, .. } => name.clone(),
LoadedImport::Value { key, .. } => key.clone(),
};
fields = match loaded {
LoadedImport::Source { source, .. } => collect_fields(&tokenize(&source)),
LoadedImport::Value { value, .. } => fields_from_value(&value),
};
} else if let Some(local) = local_fields.get(root) {
fields = local.clone();
detail = String::from("local value");
} else if let Some(global) = globals.get(root) {
fields = global.clone();
detail = String::from("host global");
} else {
return Ok(None);
}
for part in parts {
let Some(nested) = fields.remove(part) else {
return Ok(None);
};
fields = nested;
}
let items = fields
.into_iter()
.map(|(label, children)| CompletionItem {
label,
kind: if children.is_empty() {
CompletionKind::Property
} else {
CompletionKind::Namespace
},
detail: Some(detail.clone()),
priority: 30,
})
.collect();
return Ok(Some(CompletionResult {
from,
items: unique_items(items),
}));
}
let word_from = word_start(source, position);
if word_from == position && !explicit {
return Ok(None);
}
let mut items = builtin_items();
for (label, children) in local_fields {
items.push(CompletionItem {
label,
kind: if children.is_empty() {
CompletionKind::Variable
} else {
CompletionKind::Namespace
},
detail: Some(if children.is_empty() {
String::from("local value")
} else {
String::from("local object")
}),
priority: 20,
});
}
for label in collect_parameters(&tokens) {
items.push(CompletionItem {
label,
kind: CompletionKind::Variable,
detail: Some(String::from("parameter")),
priority: 20,
});
}
for (label, specifier) in imports {
items.push(CompletionItem {
label,
kind: CompletionKind::Namespace,
detail: Some(specifier),
priority: 30,
});
}
for (label, children) in globals {
items.push(CompletionItem {
label,
kind: if children.is_empty() {
CompletionKind::Variable
} else {
CompletionKind::Namespace
},
detail: Some(String::from("host global")),
priority: 40,
});
}
Ok(Some(CompletionResult {
from: word_from,
items: unique_items(items),
}))
}
fn builtin_items() -> Vec<CompletionItem> {
[
("let", CompletionKind::Keyword, "local bindings", 5),
("in", CompletionKind::Keyword, "let body", 0),
("match", CompletionKind::Keyword, "pattern matching", 5),
("import", CompletionKind::Keyword, "load a module", 5),
("default", CompletionKind::Keyword, "fallback value", 0),
("as", CompletionKind::Keyword, "refine narrower as wider", 0),
("true", CompletionKind::Constant, "Bool", 0),
("false", CompletionKind::Constant, "Bool", 0),
("String", CompletionKind::Type, "string constraint", 5),
("Int", CompletionKind::Type, "integer constraint", 5),
("Float", CompletionKind::Type, "float constraint", 5),
("Bool", CompletionKind::Type, "boolean constraint", 5),
("Unknown", CompletionKind::Type, "unknown top range", 5),
]
.into_iter()
.map(|(label, kind, detail, priority)| CompletionItem {
label: label.into(),
kind,
detail: Some(detail.into()),
priority,
})
.collect()
}
fn collect_globals<E: HostEnvironment>(environment: &E) -> Result<FieldTree> {
let mut engine = environment.create_engine()?;
let mut fields = FieldTree::new();
for (name, value) in engine.global_values()? {
fields.insert(name, fields_from_runtime(&mut engine, &value, 0)?);
}
Ok(fields)
}
fn fields_from_runtime<L: ImportLoader>(
engine: &mut Engine<L>,
value: &RuntimeValue,
depth: usize,
) -> Result<FieldTree> {
if depth >= 32 {
return Ok(FieldTree::new());
}
let Some(fields) = engine.value_fields(value)? else {
return Ok(FieldTree::new());
};
let mut tree = FieldTree::new();
for (name, value) in fields {
tree.insert(name, fields_from_runtime(engine, &value, depth + 1)?);
}
Ok(tree)
}
fn fields_from_value(value: &Value) -> FieldTree {
match value {
Value::Object { fields, .. } => fields
.iter()
.map(|field| (field.name.clone(), fields_from_value(&field.value)))
.collect(),
Value::ArrayRange {
default: Some(value),
..
}
| Value::MapRange {
default: Some(value),
..
}
| Value::Range {
default: Some(value),
..
} => fields_from_value(value),
_ => FieldTree::new(),
}
}
fn unique_items(items: Vec<CompletionItem>) -> Vec<CompletionItem> {
let mut unique = HashMap::<String, CompletionItem>::new();
let mut order = Vec::new();
for item in items {
if !unique.contains_key(&item.label) {
order.push(item.label.clone());
}
let replace = unique
.get(&item.label)
.is_none_or(|previous| item.priority > previous.priority);
if replace {
unique.insert(item.label.clone(), item);
}
}
order
.into_iter()
.filter_map(|label| unique.remove(&label))
.collect()
}
#[derive(Debug, Clone, PartialEq, Eq)]
enum TokenKind {
Identifier(String),
String(String),
Symbol(char),
Arrow,
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct Token {
kind: TokenKind,
}
fn tokenize(source: &str) -> Vec<Token> {
let bytes = source.as_bytes();
let mut tokens = Vec::new();
let mut index = 0;
while index < bytes.len() {
match bytes[index] {
b' ' | b'\t' | b'\r' | b'\n' => index += 1,
b'#' => {
while index < bytes.len() && bytes[index] != b'\n' {
index += 1;
}
}
b'"' => {
index += 1;
let start = index;
let mut escaped = false;
while index < bytes.len() {
let byte = bytes[index];
if escaped {
escaped = false;
} else if byte == b'\\' {
escaped = true;
} else if byte == b'"' || byte == b'\n' {
break;
}
index += 1;
}
let value = unescape_string(&source[start..index]);
index = index.saturating_add(1);
tokens.push(Token {
kind: TokenKind::String(value),
});
}
byte if is_identifier_start(byte) => {
let start = index;
index += 1;
while index < bytes.len() && is_identifier_continue(bytes[index]) {
index += 1;
}
tokens.push(Token {
kind: TokenKind::Identifier(source[start..index].into()),
});
}
b'=' if bytes.get(index + 1) == Some(&b'>') => {
tokens.push(Token {
kind: TokenKind::Arrow,
});
index += 2;
}
byte => {
tokens.push(Token {
kind: TokenKind::Symbol(byte as char),
});
index += 1;
}
}
}
tokens
}
fn unescape_string(value: &str) -> String {
let mut chars = value.chars();
let mut unescaped = String::new();
while let Some(ch) = chars.next() {
if ch != '\\' {
unescaped.push(ch);
continue;
}
let Some(escaped) = chars.next() else {
unescaped.push('\\');
break;
};
unescaped.push(match escaped {
'n' => '\n',
'r' => '\r',
't' => '\t',
escaped => escaped,
});
}
unescaped
}
fn collect_fields(tokens: &[Token]) -> FieldTree {
let mut fields = FieldTree::new();
collect_fields_in(tokens, 0, tokens.len(), &mut fields);
fields
}
fn collect_fields_in(tokens: &[Token], start: usize, end: usize, fields: &mut FieldTree) {
let mut index = start;
while index < end {
let Some((path, equals)) = field_definition_at(tokens, index, end) else {
index += 1;
continue;
};
let value_start = equals + 1;
let value_end = definition_end(tokens, value_start, end);
let mut nested = &mut *fields;
for part in path {
nested = nested.0.entry(part).or_default();
}
collect_fields_in(tokens, value_start, value_end, nested);
index = value_end.saturating_add(1);
}
}
fn field_definition_at(tokens: &[Token], start: usize, end: usize) -> Option<(Vec<String>, usize)> {
let TokenKind::Identifier(first) = &tokens.get(start)?.kind else {
return None;
};
let mut path = vec![first.clone()];
let mut index = start + 1;
while index + 1 < end
&& matches!(tokens[index].kind, TokenKind::Symbol('.'))
&& matches!(tokens[index + 1].kind, TokenKind::Identifier(_))
{
let TokenKind::Identifier(part) = &tokens[index + 1].kind else {
unreachable!()
};
path.push(part.clone());
index += 2;
}
matches!(
tokens.get(index).map(|token| &token.kind),
Some(TokenKind::Symbol('='))
)
.then_some((path, index))
}
fn definition_end(tokens: &[Token], start: usize, end: usize) -> usize {
let mut delimiters = Vec::new();
for (index, token) in tokens.iter().enumerate().take(end).skip(start) {
match token.kind {
TokenKind::Symbol('(' | '[' | '{') => delimiters.push(token.kind.clone()),
TokenKind::Symbol(')') if matches!(delimiters.last(), Some(TokenKind::Symbol('('))) => {
delimiters.pop();
}
TokenKind::Symbol(']') if matches!(delimiters.last(), Some(TokenKind::Symbol('['))) => {
delimiters.pop();
}
TokenKind::Symbol('}') if matches!(delimiters.last(), Some(TokenKind::Symbol('{'))) => {
delimiters.pop();
}
TokenKind::Symbol(';') if delimiters.is_empty() => return index,
_ => {}
}
}
end
}
fn collect_import_bindings(tokens: &[Token]) -> BTreeMap<String, String> {
let mut imports = BTreeMap::new();
for window in tokens.windows(4) {
let (
TokenKind::Identifier(binding),
TokenKind::Symbol('='),
TokenKind::Identifier(keyword),
TokenKind::String(specifier),
) = (
&window[0].kind,
&window[1].kind,
&window[2].kind,
&window[3].kind,
)
else {
continue;
};
if keyword == "import" {
imports.insert(binding.clone(), specifier.clone());
}
}
imports
}
fn collect_parameters(tokens: &[Token]) -> Vec<String> {
let mut parameters = Vec::new();
for (close, token) in tokens.iter().enumerate() {
if !matches!(token.kind, TokenKind::Symbol(')'))
|| !matches!(
tokens.get(close + 1).map(|token| &token.kind),
Some(TokenKind::Arrow)
)
{
continue;
}
let Some(open) = matching_open_paren(tokens, close) else {
continue;
};
let mut depth = 0usize;
let mut segment_start = true;
for token in &tokens[open + 1..close] {
match &token.kind {
TokenKind::Symbol('(' | '[' | '{') => depth += 1,
TokenKind::Symbol(')' | ']' | '}') => depth = depth.saturating_sub(1),
TokenKind::Symbol(',') if depth == 0 => segment_start = true,
TokenKind::Identifier(name) if depth == 0 && segment_start => {
parameters.push(name.clone());
segment_start = false;
}
_ => {}
}
}
}
parameters
}
fn matching_open_paren(tokens: &[Token], close: usize) -> Option<usize> {
let mut depth = 0usize;
for index in (0..close).rev() {
match tokens[index].kind {
TokenKind::Symbol(')') => depth += 1,
TokenKind::Symbol('(') if depth == 0 => return Some(index),
TokenKind::Symbol('(') => depth -= 1,
_ => {}
}
}
None
}
fn unfinished_import(source: &str, position: usize) -> Option<(usize, &str)> {
let bytes = source.as_bytes();
let mut index = 0usize;
let mut previous_word = None;
while index < position {
match bytes[index] {
b' ' | b'\t' | b'\r' | b'\n' => index += 1,
b'#' => {
while index < position && bytes[index] != b'\n' {
index += 1;
}
previous_word = None;
}
b'"' => {
let from = index + 1;
index += 1;
let mut escaped = false;
while index < position {
let byte = bytes[index];
if escaped {
escaped = false;
} else if byte == b'\\' {
escaped = true;
} else if byte == b'"' || byte == b'\n' {
break;
}
index += 1;
}
if index == position && previous_word == Some("import") {
return Some((from, &source[from..position]));
}
index = index.saturating_add(1);
previous_word = None;
}
byte if is_identifier_start(byte) => {
let start = index;
index += 1;
while index < position && is_identifier_continue(bytes[index]) {
index += 1;
}
previous_word = Some(&source[start..index]);
}
_ => {
index += 1;
previous_word = None;
}
}
}
None
}
fn in_string_or_comment(source: &str, position: usize) -> bool {
let bytes = source.as_bytes();
let mut index = 0usize;
while index < position {
match bytes[index] {
b'#' => {
while index < position && bytes[index] != b'\n' {
index += 1;
}
if index == position {
return true;
}
}
b'"' => {
index += 1;
let mut escaped = false;
while index < position {
let byte = bytes[index];
if escaped {
escaped = false;
} else if byte == b'\\' {
escaped = true;
} else if byte == b'"' || byte == b'\n' {
break;
}
index += 1;
}
if index == position {
return true;
}
index = index.saturating_add(1);
}
_ => index += 1,
}
}
false
}
fn member_path(source: &str, position: usize) -> Option<(usize, &str)> {
let bytes = source.as_bytes();
let from = word_start(source, position);
if from == 0 || bytes[from - 1] != b'.' {
return None;
}
let mut base_start = from - 1;
while base_start > 0 {
let byte = bytes[base_start - 1];
if is_identifier_continue(byte) || byte == b'.' {
base_start -= 1;
} else {
break;
}
}
let base = &source[base_start..from - 1];
(!base.is_empty()
&& base
.split('.')
.all(|part| !part.is_empty() && is_identifier(part)))
.then_some((from, base))
}
fn word_start(source: &str, position: usize) -> usize {
let bytes = source.as_bytes();
let mut from = position;
while from > 0 && is_identifier_continue(bytes[from - 1]) {
from -= 1;
}
from
}
fn is_identifier(value: &str) -> bool {
let bytes = value.as_bytes();
bytes.first().is_some_and(|byte| is_identifier_start(*byte))
&& bytes[1..].iter().all(|byte| is_identifier_continue(*byte))
}
fn is_identifier_start(byte: u8) -> bool {
byte.is_ascii_alphabetic() || byte == b'_'
}
fn is_identifier_continue(byte: u8) -> bool {
is_identifier_start(byte) || byte.is_ascii_digit()
}
fn floor_char_boundary(source: &str, mut position: usize) -> usize {
while !source.is_char_boundary(position) {
position -= 1;
}
position
}
#[cfg(test)]
mod tests {
use std::collections::BTreeMap;
use decodal::{
Diagnostic, DiagnosticKind, EmptyLoader, ImportCandidate, LoadedImport, Span, Value,
};
use super::*;
#[derive(Clone, Default)]
struct TestEnvironment {
files: BTreeMap<String, String>,
}
#[derive(Clone, Default)]
struct TestLoader {
files: BTreeMap<String, String>,
}
impl ImportLoader for TestLoader {
fn load(&mut self, _current_key: Option<&str>, specifier: &str) -> Result<LoadedImport> {
if specifier == "./post.md" {
return Ok(LoadedImport::value(
"post.md",
Value::object([
(
"frontmatter",
Value::object([
("title", Value::string("Hello")),
("draft", Value::bool(false)),
]),
),
("body", Value::string("# Hello")),
]),
));
}
let key = resolve_virtual("main.dcdl", specifier);
let Some(source) = self.files.get(&key) else {
return Err(Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
"unknown import",
));
};
Ok(LoadedImport::source(key.clone(), key, source))
}
fn complete_import(
&mut self,
_current_key: Option<&str>,
prefix: &str,
) -> Result<Vec<ImportCandidate>> {
Ok(self
.files
.keys()
.map(|path| format!("./{path}"))
.filter(|path| path.starts_with(prefix))
.map(ImportCandidate::new)
.collect())
}
}
impl HostEnvironment for TestEnvironment {
type Loader = TestLoader;
fn create_loader(&self) -> Self::Loader {
TestLoader {
files: self.files.clone(),
}
}
fn configure_engine(&self, engine: &mut Engine<Self::Loader>) -> Result<()> {
engine.bind_global(
"App",
Value::object([(
"config",
Value::object([("enabled", Value::bool_type()), ("port", Value::int_type())]),
)]),
)?;
Ok(())
}
}
fn environment() -> TestEnvironment {
TestEnvironment {
files: BTreeMap::from([
(
"schemas/service.dcdl".into(),
"Service = { name = String; resources = { cpu = Int; }; };".into(),
),
(
"env/production.dcdl".into(),
"capacity = { cpu = 2000; };".into(),
),
(
"資料/service.dcdl".into(),
"Service = { label = String; };".into(),
),
]),
}
}
fn labels(result: CompletionResult) -> Vec<String> {
result.items.into_iter().map(|item| item.label).collect()
}
#[test]
fn completes_import_paths_from_the_host_loader() {
let source = "schema = import \"./sch";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(result.from, source.len() - 5);
assert!(labels(result).contains(&"./schemas/service.dcdl".into()));
}
#[test]
fn completes_nested_fields_from_imported_source() {
let source = "let schema = import \"./schemas/service.dcdl\"; in schema.Service.";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(labels(result), ["name", "resources"]);
}
#[test]
fn completes_structured_host_imports() {
let source = "let post = import \"./post.md\"; in post.frontmatter.";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(labels(result), ["draft", "title"]);
}
#[test]
fn preserves_unicode_import_specifiers() {
let source = "let schema = import \"./資料/service.dcdl\"; in schema.Service.";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(labels(result), ["label"]);
}
#[test]
fn completes_host_globals_from_the_evaluation_environment() {
let source = "App.config.";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(labels(result), ["enabled", "port"]);
}
#[test]
fn completes_language_locals_parameters_and_partial_members() {
let source = "let service = { port = 8080; }; in (value: Int) => service.po";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(result.from, source.len() - 2);
assert_eq!(labels(result), ["port"]);
let source = "let service = { port = 8080; }; in (value: Int) => val";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
let labels = labels(result);
assert!(labels.contains(&"String".into()));
assert!(labels.contains(&"Unknown".into()));
assert!(labels.contains(&"service".into()));
assert!(labels.contains(&"value".into()));
assert!(labels.contains(&"App".into()));
}
#[test]
fn suppresses_completions_in_strings_and_comments() {
let string = "value = \"hello world\"";
assert!(
complete(&environment(), "main.dcdl", string, string.len() - 1, false)
.unwrap()
.is_none()
);
let comment = "value = true; # hello";
assert!(
complete(&environment(), "main.dcdl", comment, comment.len(), false)
.unwrap()
.is_none()
);
}
#[test]
fn empty_environment_remains_supported() {
let result = complete(&EmptyEnvironment, "main.dcdl", "Str", 3, false)
.unwrap()
.unwrap();
assert!(labels(result).contains(&"String".into()));
let result = complete(&EmptyEnvironment, "main.dcdl", "Unk", 3, false)
.unwrap()
.unwrap();
assert!(labels(result).contains(&"Unknown".into()));
}
struct EmptyEnvironment;
impl HostEnvironment for EmptyEnvironment {
type Loader = EmptyLoader;
fn create_loader(&self) -> Self::Loader {
EmptyLoader
}
}
fn resolve_virtual(current: &str, specifier: &str) -> String {
let mut parts = current.split('/').collect::<Vec<_>>();
parts.pop();
for part in specifier.split('/') {
match part {
"" | "." => {}
".." => {
parts.pop();
}
part => parts.push(part),
}
}
parts.join("/")
}
}
+204
View File
@@ -0,0 +1,204 @@
use decodal::{Data, Diagnostic, HostEnvironment, Result};
mod completion;
pub use completion::{CompletionItem, CompletionKind, CompletionResult};
/// Semantic tooling backed by the same host environment as production.
pub struct LanguageService<E> {
environment: E,
}
impl<E> LanguageService<E> {
pub fn new(environment: E) -> Self {
Self { environment }
}
pub fn environment(&self) -> &E {
&self.environment
}
pub fn environment_mut(&mut self) -> &mut E {
&mut self.environment
}
pub fn into_environment(self) -> E {
self.environment
}
}
impl<E: HostEnvironment> LanguageService<E> {
/// Runs the production parse, evaluation, and materialization pipeline.
pub fn evaluate(
&self,
key: impl Into<String>,
name: impl Into<String>,
source: &str,
) -> Result<Data> {
let mut engine = self.environment.create_engine()?;
let module = engine.add_root_source(key, name, source)?;
let value = engine.eval_module(module)?;
engine.materialize(&value)
}
/// Evaluates a document and exposes failures in an editor-friendly form.
///
/// The evaluator currently stops at the first failure, so an invalid
/// analysis contains one diagnostic. The collection leaves room for a
/// future diagnostic-accumulation pass without changing this API.
pub fn analyze(
&self,
key: impl Into<String>,
name: impl Into<String>,
source: &str,
) -> SemanticAnalysis {
match self.evaluate(key, name, source) {
Ok(data) => SemanticAnalysis {
data: Some(data),
diagnostics: Vec::new(),
},
Err(diagnostic) => SemanticAnalysis {
data: None,
diagnostics: vec![diagnostic],
},
}
}
/// Completes the document using the same host globals and import loader as
/// production evaluation.
pub fn complete(
&self,
key: impl AsRef<str>,
source: &str,
position: usize,
explicit: bool,
) -> Result<Option<CompletionResult>> {
completion::complete(&self.environment, key.as_ref(), source, position, explicit)
}
}
#[derive(Debug, Clone, PartialEq)]
pub struct SemanticAnalysis {
pub data: Option<Data>,
pub diagnostics: Vec<Diagnostic>,
}
impl SemanticAnalysis {
pub fn is_valid(&self) -> bool {
self.diagnostics.is_empty()
}
}
#[cfg(test)]
mod tests {
use decodal::{
Data, Diagnostic, DiagnosticKind, Engine, ImportLoader, LoadedImport, Span, Value,
};
use super::*;
const ROOT: &str = r#"Post & import "./post.md""#;
struct ContentEnvironment {
draft: Value,
}
struct ContentLoader {
draft: Value,
}
impl ImportLoader for ContentLoader {
fn load(
&mut self,
_current_key: Option<&str>,
specifier: &str,
) -> decodal::Result<LoadedImport> {
if specifier != "./post.md" {
return Err(Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
"unknown content import",
));
}
Ok(LoadedImport::value(
"content/post.md",
Value::object([
(
"frontmatter",
Value::object([
("title", Value::string("Hello")),
("draft", self.draft.clone()),
]),
),
("body", Value::string("# Hello")),
]),
))
}
}
impl HostEnvironment for ContentEnvironment {
type Loader = ContentLoader;
fn create_loader(&self) -> Self::Loader {
ContentLoader {
draft: self.draft.clone(),
}
}
fn configure_engine(&self, engine: &mut Engine<Self::Loader>) -> decodal::Result<()> {
engine.bind_global(
"Post",
Value::object([
(
"frontmatter",
Value::object([
("title", Value::string_type()),
("draft", Value::bool_type()),
]),
),
("body", Value::string_type()),
]),
)?;
Ok(())
}
}
fn evaluate_direct(environment: &ContentEnvironment) -> decodal::Result<Data> {
let mut engine = environment.create_engine()?;
let module = engine.add_root_source("main.dcdl", "main.dcdl", ROOT)?;
let value = engine.eval_module(module)?;
engine.materialize(&value)
}
#[test]
fn injected_environment_matches_direct_evaluation() {
let environment = ContentEnvironment {
draft: Value::bool(false),
};
let direct = evaluate_direct(&environment).unwrap();
let service = LanguageService::new(&environment);
let analysis = service.analyze("main.dcdl", "main.dcdl", ROOT);
assert!(analysis.is_valid());
assert_eq!(analysis.data, Some(direct));
}
#[test]
fn injected_environment_preserves_import_diagnostics() {
let service = LanguageService::new(ContentEnvironment {
draft: Value::string("maybe"),
});
let analysis = service.analyze("main.dcdl", "main.dcdl", ROOT);
assert!(!analysis.is_valid());
assert_eq!(analysis.data, None);
assert_eq!(
analysis.diagnostics[0].message,
"expected Bool, found String"
);
assert_eq!(
analysis.diagnostics[0].notes,
["imported `content/post.md` at `frontmatter.draft` supplied a value of type String"]
);
}
}
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "decodal-language-tools"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Source-level language tooling for Decodal."
keywords = ["decodal", "formatter", "lsp", "language-tools"]
categories = ["development-tools", "text-processing"]
[lib]
crate-type = ["cdylib", "rlib"]
[dependencies]
decodal = { version = "0.4.0", path = "../decodal-core" }
serde_json.workspace = true
wasm-bindgen.workspace = true
[package.metadata.wasm-pack.profile.release]
wasm-opt = false
+838
View File
@@ -0,0 +1,838 @@
use std::{error::Error, fmt};
use decodal::{
Ast, BinaryOp, CompareOp, Expr, ExprId, Field, ObjectRest, Param, SourceForm, Span,
SyntaxToken, SyntaxTokenKind,
ast::{MatchArm, UnaryOp},
parse_source, tokenize_source,
};
use wasm_bindgen::prelude::*;
const INDENT: usize = 4;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct FormatError {
message: String,
}
impl FormatError {
fn new(message: impl Into<String>) -> Self {
Self {
message: message.into(),
}
}
pub fn message(&self) -> &str {
&self.message
}
}
impl fmt::Display for FormatError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.message)
}
}
impl Error for FormatError {}
/// Formats a complete Decodal source with the canonical parser and syntax tokens.
///
/// Native and WebAssembly callers execute this exact implementation. Comments
/// are recovered from the lossless token stream while expression structure and
/// precedence come from the same AST used by evaluation.
pub fn format_source(source: &str) -> Result<String, FormatError> {
let parsed = parse_source(source)
.map_err(|diagnostic| FormatError::new(format!("parse error: {}", diagnostic.message)))?;
let tokens = tokenize_source(source)
.map_err(|diagnostic| FormatError::new(format!("lex error: {}", diagnostic.message)))?;
Ok(Formatter::new(source, &parsed.ast, tokens).format(parsed.root, parsed.source_form))
}
#[wasm_bindgen(js_name = formatSource)]
pub fn format_source_json(source: &str) -> String {
match format_source(source) {
Ok(formatted) => serde_json::json!({ "ok": true, "source": formatted }).to_string(),
Err(error) => serde_json::json!({ "ok": false, "error": error.message() }).to_string(),
}
}
struct Formatter<'a> {
source: &'a str,
ast: &'a Ast,
tokens: Vec<SyntaxToken>,
comments: Vec<Span>,
}
impl<'a> Formatter<'a> {
fn new(source: &'a str, ast: &'a Ast, tokens: Vec<SyntaxToken>) -> Self {
let comments = tokens
.iter()
.filter(|token| token.kind == SyntaxTokenKind::Comment)
.map(|token| token.span)
.collect();
Self {
source,
ast,
tokens,
comments,
}
}
fn format(&self, root: ExprId, source_form: SourceForm) -> String {
let mut out = String::new();
if source_form == SourceForm::Fields {
let Expr::Object { fields, .. } = &self.ast.get(root).expr else {
unreachable!("field-form sources parse to an object")
};
self.write_field_list(&mut out, fields, 0, self.source.len(), 0);
} else {
let span = self.ast.span(root);
self.write_between(&mut out, 0, span.start as usize, None, 0);
self.write_expr(&mut out, root, 0, 0);
self.write_between(
&mut out,
span.end as usize,
self.source.len(),
Some(span.end as usize),
0,
);
}
trim_trailing_whitespace(&mut out);
out.push('\n');
out
}
fn write_field_list(
&self,
out: &mut String,
fields: &[Field],
start: usize,
end: usize,
indent: usize,
) {
let mut cursor = start;
let mut previous_end = None;
for field in fields {
let field_start = field.span.start as usize;
self.write_between(out, cursor, field_start, previous_end, indent);
write_indent(out, indent);
self.write_field(out, field, indent);
out.push(';');
cursor = field.span.end as usize;
previous_end = Some(cursor);
}
self.write_between(out, cursor, end, previous_end, indent);
}
fn write_field(&self, out: &mut String, field: &Field, indent: usize) {
let value_start = self.ast.span(field.value).start as usize;
if self.has_comment_between(field.span.start as usize, value_start) {
out.push_str(self.raw(field.span));
return;
}
for (index, part) in field.path.iter().enumerate() {
if index > 0 {
out.push('.');
}
out.push_str(part);
}
out.push_str(" = ");
self.write_expr(out, field.value, indent, 0);
}
fn write_expr(&self, out: &mut String, id: ExprId, indent: usize, parent_prec: u8) {
let node = self.ast.get(id);
if self.has_comment(node.span)
&& !matches!(
node.expr,
Expr::Object { .. } | Expr::Array(_) | Expr::Let { .. } | Expr::Match { .. }
)
{
out.push_str(self.raw(node.span));
return;
}
let precedence = self.precedence(id);
let parenthesize = precedence < parent_prec;
if parenthesize {
out.push('(');
}
match &node.expr {
Expr::Literal(_) | Expr::Ident(_) | Expr::RegexConstraint(_) | Expr::Wildcard => {
out.push_str(self.raw(node.span));
}
Expr::Object { fields, rest } => {
self.write_object(out, node.span, fields, rest.as_ref(), indent)
}
Expr::Array(items) => self.write_array(out, node.span, items, indent),
Expr::ArrayConstraint { item } => {
out.push_str("[...");
self.write_expr(out, *item, indent, 0);
out.push(']');
}
Expr::MapConstraint { value } => {
out.push_str("{...");
self.write_expr(out, *value, indent, 0);
out.push('}');
}
Expr::Let { bindings, body } => self.write_let(out, node.span, bindings, *body, indent),
Expr::Import(_) => {
out.push_str("import ");
let raw = self.raw(node.span);
out.push_str(raw.strip_prefix("import").unwrap_or(raw).trim());
}
Expr::Path { base, field } => {
self.write_expr(out, *base, indent, precedence);
out.push('.');
out.push_str(field);
}
Expr::Call { callee, args } => {
self.write_expr(out, *callee, indent, precedence);
out.push('(');
for (index, argument) in args.iter().enumerate() {
if index > 0 {
out.push_str(", ");
}
self.write_expr(out, *argument, indent, 0);
}
out.push(')');
}
Expr::Function { params, body } => self.write_function(out, params, *body, indent),
Expr::Parenthesized { expr } => {
out.push('(');
self.write_expr(out, *expr, indent, 0);
out.push(')');
}
Expr::Match { scrutinee, arms } => {
self.write_match(out, node.span, *scrutinee, arms, indent)
}
Expr::Unary { op, expr } => {
out.push_str(match op {
UnaryOp::Neg => "-",
UnaryOp::Not => "!",
});
self.write_expr(out, *expr, indent, precedence);
}
Expr::Binary { op, lhs, rhs } => {
self.write_expr(out, *lhs, indent, precedence);
out.push(' ');
out.push_str(binary_operator(*op));
out.push(' ');
self.write_expr(out, *rhs, indent, precedence + 1);
}
Expr::Default { base, fallback } => {
self.write_expr(out, *base, indent, precedence);
out.push_str(" default ");
self.write_expr(out, *fallback, indent, precedence + 1);
}
Expr::As { narrower, wider } => {
self.write_expr(out, *narrower, indent, precedence);
out.push_str(" as ");
self.write_expr(out, *wider, indent, precedence + 1);
}
Expr::CompareConstraint { op, value } => {
out.push_str(compare_operator(*op));
out.push(' ');
self.write_expr(out, *value, indent, precedence);
}
}
if parenthesize {
out.push(')');
}
}
fn write_object(
&self,
out: &mut String,
span: Span,
fields: &[Field],
rest: Option<&ObjectRest>,
indent: usize,
) {
let (start, end) =
self.delimited_range(span, SyntaxTokenKind::LBrace, SyntaxTokenKind::RBrace);
if fields.is_empty() && rest.is_none() && !self.has_comment_between(start, end) {
out.push_str("{}");
return;
}
out.push_str("{\n");
let field_end = rest.map_or(end, |rest| rest.span.start as usize);
self.write_field_list(out, fields, start, field_end, indent + INDENT);
if let Some(rest) = rest {
write_indent(out, indent + INDENT);
out.push_str("...");
let value_start = self.ast.span(rest.value).start as usize;
let ellipsis = self
.tokens_between(rest.span.start as usize, value_start)
.find(|token| token.kind == SyntaxTokenKind::Ellipsis)
.expect("parsed object rest constraints contain `...`");
if self.has_comment_between(ellipsis.span.end as usize, value_start) {
self.write_between(
out,
ellipsis.span.end as usize,
value_start,
Some(ellipsis.span.end as usize),
indent + INDENT,
);
write_indent(out, indent + INDENT);
}
self.write_expr(out, rest.value, indent + INDENT, 0);
self.write_between(
out,
rest.span.end as usize,
end,
Some(rest.span.end as usize),
indent + INDENT,
);
}
write_indent(out, indent);
out.push('}');
}
fn write_array(&self, out: &mut String, span: Span, items: &[ExprId], indent: usize) {
let (start, end) =
self.delimited_range(span, SyntaxTokenKind::LBracket, SyntaxTokenKind::RBracket);
if items.is_empty() && !self.has_comment_between(start, end) {
out.push_str("[]");
return;
}
let inline = !self.has_comment_between(start, end)
&& items.iter().all(|item| self.is_inline_expr(*item));
if inline {
out.push('[');
for (index, item) in items.iter().enumerate() {
if index > 0 {
out.push_str(", ");
}
self.write_expr(out, *item, indent, 0);
}
out.push(']');
return;
}
out.push_str("[\n");
let mut cursor = start;
let mut previous_end = None;
for item in items {
let item_span = self.ast.span(*item);
self.write_between(
out,
cursor,
item_span.start as usize,
previous_end,
indent + INDENT,
);
write_indent(out, indent + INDENT);
self.write_expr(out, *item, indent + INDENT, 0);
out.push(',');
cursor = item_span.end as usize;
previous_end = Some(cursor);
}
self.write_between(out, cursor, end, previous_end, indent + INDENT);
write_indent(out, indent);
out.push(']');
}
fn write_let(
&self,
out: &mut String,
span: Span,
bindings: &[Field],
body: ExprId,
indent: usize,
) {
let body_span = self.ast.span(body);
let in_token = self
.tokens_between(span.start as usize, body_span.start as usize)
.rev()
.find(|token| token.kind == SyntaxTokenKind::In)
.expect("parsed let expressions contain `in`");
let let_token = self
.tokens_between(span.start as usize, span.end as usize)
.find(|token| token.kind == SyntaxTokenKind::Let)
.expect("parsed let expressions contain `let`");
out.push_str("let\n");
self.write_field_list(
out,
bindings,
let_token.span.end as usize,
in_token.span.start as usize,
indent + INDENT,
);
write_indent(out, indent);
out.push_str("in");
self.write_between(
out,
in_token.span.end as usize,
body_span.start as usize,
Some(in_token.span.end as usize),
indent + INDENT,
);
write_indent(out, indent + INDENT);
self.write_expr(out, body, indent + INDENT, 0);
}
fn write_function(&self, out: &mut String, params: &[Param], body: ExprId, indent: usize) {
out.push('(');
for (index, parameter) in params.iter().enumerate() {
if index > 0 {
out.push_str(", ");
}
out.push_str(&parameter.name);
if let Some(constraint) = parameter.constraint {
out.push_str(": ");
self.write_expr(out, constraint, indent, 0);
}
}
out.push_str(") =>");
if self.is_inline_expr(body) {
out.push(' ');
self.write_expr(out, body, indent, 0);
} else {
out.push('\n');
write_indent(out, indent + INDENT);
self.write_expr(out, body, indent + INDENT, 0);
}
}
fn write_match(
&self,
out: &mut String,
span: Span,
scrutinee: ExprId,
arms: &[MatchArm],
indent: usize,
) {
out.push_str("match ");
self.write_expr(out, scrutinee, indent, 0);
let scrutinee_end = self.ast.span(scrutinee).end as usize;
let open = self
.tokens_between(scrutinee_end, span.end as usize)
.find(|token| token.kind == SyntaxTokenKind::LBrace)
.expect("parsed match expressions contain `{`");
let close = self
.tokens_between(open.span.end as usize, span.end as usize)
.rev()
.find(|token| token.kind == SyntaxTokenKind::RBrace)
.expect("parsed match expressions contain `}`");
let start = open.span.end as usize;
let end = close.span.start as usize;
if self.has_comment_between(scrutinee_end, open.span.start as usize) {
self.write_between(
out,
scrutinee_end,
open.span.start as usize,
Some(scrutinee_end),
indent,
);
write_indent(out, indent);
out.push('{');
} else {
out.push_str(" {");
}
if arms.is_empty() && !self.has_comment_between(start, end) {
out.push('}');
return;
}
out.push('\n');
let mut cursor = start;
let mut previous_end = None;
for arm in arms {
self.write_between(
out,
cursor,
arm.span.start as usize,
previous_end,
indent + INDENT,
);
write_indent(out, indent + INDENT);
if self.has_comment_between(
self.ast.span(arm.pattern).end as usize,
self.ast.span(arm.body).start as usize,
) {
out.push_str(self.raw(arm.span));
} else {
self.write_expr(out, arm.pattern, indent + INDENT, 0);
out.push_str(": ");
self.write_expr(out, arm.body, indent + INDENT, 0);
}
out.push(';');
cursor = arm.span.end as usize;
previous_end = Some(cursor);
}
self.write_between(out, cursor, end, previous_end, indent + INDENT);
write_indent(out, indent);
out.push('}');
}
fn write_between(
&self,
out: &mut String,
start: usize,
end: usize,
previous_end: Option<usize>,
indent: usize,
) {
let mut cursor = start.min(end);
for comment in self.comments_between(start, end) {
let comment_start = comment.start as usize;
let trailing = previous_end.is_some_and(|previous| {
!out.ends_with('\n') && self.same_line(previous, comment_start)
});
if trailing {
out.push(' ');
out.push_str(self.raw(*comment));
out.push('\n');
} else {
ensure_line_break(out);
if self.has_blank_line(cursor, comment_start) {
ensure_blank_line(out);
}
write_indent(out, indent);
out.push_str(self.raw(*comment));
out.push('\n');
}
cursor = comment.end as usize;
}
if previous_end.is_some() {
ensure_line_break(out);
}
if self.has_blank_line(cursor, end) {
ensure_blank_line(out);
}
}
fn precedence(&self, id: ExprId) -> u8 {
match &self.ast.get(id).expr {
Expr::As { .. } => 0,
Expr::Default { .. } => 1,
Expr::Binary { op, .. } => match op {
BinaryOp::Patch => 3,
BinaryOp::And => 5,
BinaryOp::LogicalOr => 7,
BinaryOp::LogicalAnd => 9,
BinaryOp::Equal
| BinaryOp::NotEqual
| BinaryOp::Greater
| BinaryOp::GreaterEqual
| BinaryOp::Less
| BinaryOp::LessEqual => 11,
BinaryOp::Concat => 12,
BinaryOp::Add | BinaryOp::Sub => 13,
BinaryOp::Mul | BinaryOp::Div => 15,
},
Expr::Unary { .. } | Expr::CompareConstraint { .. } => 17,
Expr::Call { .. } | Expr::Path { .. } => 19,
Expr::Parenthesized { .. } => 20,
_ => 20,
}
}
fn is_inline_expr(&self, id: ExprId) -> bool {
let node = self.ast.get(id);
if self.has_comment(node.span) {
return false;
}
match &node.expr {
Expr::Literal(_)
| Expr::Ident(_)
| Expr::RegexConstraint(_)
| Expr::Wildcard
| Expr::Import(_)
| Expr::CompareConstraint { .. } => true,
Expr::Path { base, .. } => self.is_inline_expr(*base),
Expr::Call { callee, args } => {
self.is_inline_expr(*callee)
&& args.iter().all(|argument| self.is_inline_expr(*argument))
}
Expr::Unary { expr, .. } => self.is_inline_expr(*expr),
Expr::Binary { lhs, rhs, .. } => self.is_inline_expr(*lhs) && self.is_inline_expr(*rhs),
Expr::Default { base, fallback } => {
self.is_inline_expr(*base) && self.is_inline_expr(*fallback)
}
Expr::Array(items) => items.iter().all(|item| self.is_inline_expr(*item)),
Expr::ArrayConstraint { item } => self.is_inline_expr(*item),
Expr::MapConstraint { value } => self.is_inline_expr(*value),
Expr::Parenthesized { expr } => self.is_inline_expr(*expr),
Expr::As { narrower, wider } => {
self.is_inline_expr(*narrower) && self.is_inline_expr(*wider)
}
Expr::Object { .. } | Expr::Let { .. } | Expr::Function { .. } | Expr::Match { .. } => {
false
}
}
}
fn delimited_range(
&self,
span: Span,
open_kind: SyntaxTokenKind,
close_kind: SyntaxTokenKind,
) -> (usize, usize) {
let open = self
.tokens_between(span.start as usize, span.end as usize)
.find(|token| same_kind(&token.kind, &open_kind))
.expect("parsed delimited expressions contain an opening token");
let close = self
.tokens_between(open.span.end as usize, span.end as usize)
.rev()
.find(|token| same_kind(&token.kind, &close_kind))
.expect("parsed delimited expressions contain a closing token");
(open.span.end as usize, close.span.start as usize)
}
fn tokens_between(
&self,
start: usize,
end: usize,
) -> impl DoubleEndedIterator<Item = &SyntaxToken> {
self.tokens.iter().filter(move |token| {
token.span.start as usize >= start && token.span.end as usize <= end
})
}
fn comments_between(&self, start: usize, end: usize) -> impl Iterator<Item = &Span> {
self.comments
.iter()
.filter(move |comment| comment.start as usize >= start && comment.end as usize <= end)
}
fn has_comment(&self, span: Span) -> bool {
self.has_comment_between(span.start as usize, span.end as usize)
}
fn has_comment_between(&self, start: usize, end: usize) -> bool {
self.comments_between(start, end).next().is_some()
}
fn same_line(&self, start: usize, end: usize) -> bool {
!self.slice(start, end).contains('\n')
}
fn has_blank_line(&self, start: usize, end: usize) -> bool {
self.slice(start, end)
.bytes()
.filter(|byte| *byte == b'\n')
.count()
>= 2
}
fn raw(&self, span: Span) -> &'a str {
self.slice(span.start as usize, span.end as usize).trim()
}
fn slice(&self, start: usize, end: usize) -> &'a str {
&self.source[start.min(self.source.len())..end.min(self.source.len())]
}
}
fn binary_operator(operator: BinaryOp) -> &'static str {
match operator {
BinaryOp::Add => "+",
BinaryOp::Sub => "-",
BinaryOp::Mul => "*",
BinaryOp::Div => "/",
BinaryOp::Concat => "++",
BinaryOp::Equal => "==",
BinaryOp::NotEqual => "!=",
BinaryOp::Greater => ">",
BinaryOp::GreaterEqual => ">=",
BinaryOp::Less => "<",
BinaryOp::LessEqual => "<=",
BinaryOp::LogicalAnd => "&&",
BinaryOp::LogicalOr => "||",
BinaryOp::And => "&",
BinaryOp::Patch => "//",
}
}
fn compare_operator(operator: CompareOp) -> &'static str {
match operator {
CompareOp::Gt => ">",
CompareOp::Gte => ">=",
CompareOp::Lt => "<",
CompareOp::Lte => "<=",
CompareOp::Eq => "==",
}
}
fn same_kind(left: &SyntaxTokenKind, right: &SyntaxTokenKind) -> bool {
core::mem::discriminant(left) == core::mem::discriminant(right)
}
fn write_indent(out: &mut String, indent: usize) {
out.extend(core::iter::repeat_n(' ', indent));
}
fn ensure_line_break(out: &mut String) {
if !out.is_empty() && !out.ends_with('\n') {
out.push('\n');
}
}
fn ensure_blank_line(out: &mut String) {
if !out.is_empty() && !out.ends_with("\n\n") {
ensure_line_break(out);
out.push('\n');
}
}
fn trim_trailing_whitespace(out: &mut String) {
while out.ends_with(char::is_whitespace) {
out.pop();
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn formats_fields_and_preserves_comments() {
let source =
"Server={\n# host\nhost=String default \"localhost\"; # trailing\n\nport=Int&>0;\n};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"Server = {\n # host\n host = String default \"localhost\"; # trailing\n\n port = Int & > 0;\n};\n"
);
}
#[test]
fn formats_arrays_and_match_comments() {
let source = "result=match x{# c\n_: [1,{a=2;}];};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"result = match x {\n # c\n _: [\n 1,\n {\n a = 2;\n },\n ];\n};\n"
);
}
#[test]
fn formats_array_constraints() {
let source = "tags=[...String];ports=[...(Int&>=1)];";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"tags = [...String];\nports = [...(Int & >= 1)];\n"
);
}
#[test]
fn formats_map_constraints_and_range_ascription() {
let source =
"config={service={port=8080;};}as{service={port=Int;};};labels={a=1;b=2;}as{...Int};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"config = {\n service = {\n port = 8080;\n };\n} as {\n service = {\n port = Int;\n };\n};\nlabels = {\n a = 1;\n b = 2;\n} as {...Int};\n"
);
assert_eq!(format_source(&formatted).unwrap(), formatted);
}
#[test]
fn formats_let_functions_and_precedence_from_the_canonical_ast() {
let source = "value=let x=1+2*3;in(a:Int)=>{result=(x+a)*2;};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"value = let\n x = 1 + 2 * 3;\nin\n (a: Int) =>\n {\n result = (x + a) * 2;\n };\n"
);
}
#[test]
fn formats_unknown_and_object_rest_constraints() {
let source =
"value={hoge=Int;fuga=String;# remaining fields\n...# unconstrained\nUnknown};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"value = {\n hoge = Int;\n fuga = String; # remaining fields\n ... # unconstrained\n Unknown\n};\n"
);
}
#[test]
fn preserves_regex_and_escaped_strings() {
let source = r#"value={pattern=/^api\/.+$/;text="a\n\"b";};"#;
let formatted = format_source(source).unwrap();
assert!(formatted.contains(r#"pattern = /^api\/.+$/;"#));
assert!(formatted.contains(r#"text = "a\n\"b";"#));
}
#[test]
fn preserves_comments_inside_grouping_and_between_structural_tokens() {
let source = "value = (# grouped\n 1 + 2); result = match x # before brace\n { _ # before body\n : value; };";
let formatted = format_source(source).unwrap();
assert!(formatted.contains("(# grouped\n 1 + 2)"));
assert!(formatted.contains("match x # before brace\n{"));
assert!(formatted.contains("_ # before body\n : value;"));
assert_eq!(format_source(&formatted).unwrap(), formatted);
}
#[test]
fn wasm_export_returns_the_same_formatter_result() {
let source = "value={a=1;};";
let expected = format_source(source).unwrap();
let output: serde_json::Value = serde_json::from_str(&format_source_json(source)).unwrap();
assert_eq!(output["source"], expected);
}
#[test]
fn formats_all_repository_examples_idempotently() {
let examples = [
(
"advanced/main",
include_str!("../../../examples/advanced/main.dcdl"),
),
(
"advanced/profiles",
include_str!("../../../examples/advanced/profiles.dcdl"),
),
(
"advanced/schema",
include_str!("../../../examples/advanced/schema.dcdl"),
),
(
"arithmetic",
include_str!("../../../examples/arithmetic.dcdl"),
),
(
"array-concat",
include_str!("../../../examples/array-concat.dcdl"),
),
(
"array-constraint",
include_str!("../../../examples/array-constraint.dcdl"),
),
("basic", include_str!("../../../examples/basic.dcdl")),
(
"import/main",
include_str!("../../../examples/import/main.dcdl"),
),
(
"import/schema",
include_str!("../../../examples/import/schema.dcdl"),
),
("logical", include_str!("../../../examples/logical.dcdl")),
(
"map-ascription",
include_str!("../../../examples/map-ascription.dcdl"),
),
(
"regex/main",
include_str!("../../../examples/regex/main.dcdl"),
),
];
for (name, source) in examples {
let once = format_source(source).unwrap_or_else(|error| panic!("{name}: {error}"));
let twice = format_source(&once).unwrap_or_else(|error| {
panic!("{name} reformatted to invalid source: {error}\n{once}")
});
assert_eq!(once, twice, "{name}");
}
}
}
+19
View File
@@ -0,0 +1,19 @@
[package]
name = "decodal-lsp"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Language Server Protocol implementation for Decodal."
keywords = ["decodal", "lsp", "language-server", "editor"]
categories = ["development-tools", "text-editors"]
[dependencies]
decodal = { version = "0.4.0", path = "../decodal-core" }
decodal-language-service = { version = "0.4.0", path = "../decodal-language-service" }
decodal-language-tools = { version = "0.4.0", path = "../decodal-language-tools" }
lsp-server = "0.10"
lsp-types = "0.97"
serde_json.workspace = true
File diff suppressed because it is too large Load Diff
+11
View File
@@ -0,0 +1,11 @@
use std::process::ExitCode;
fn main() -> ExitCode {
match decodal_lsp::run_stdio(|_| Ok(decodal_lsp::FileSystemEnvironment::default())) {
Ok(()) => ExitCode::SUCCESS,
Err(error) => {
eprintln!("decodal-lsp failed: {error}");
ExitCode::FAILURE
}
}
}
+28
View File
@@ -0,0 +1,28 @@
[package]
name = "decodal-wasm"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Host-configurable Decodal evaluator and language service for JavaScript runtimes."
keywords = ["decodal", "wasm", "dsl", "config"]
categories = ["wasm", "config"]
publish = false
[lib]
crate-type = ["cdylib", "rlib"]
[dependencies]
decodal = { version = "0.4.0", path = "../decodal-core" }
decodal-language-service = { version = "0.4.0", path = "../decodal-language-service" }
serde_json.workspace = true
wasm-bindgen.workspace = true
[target.'cfg(target_arch = "wasm32")'.dependencies]
js-sys = "0.3"
serde-wasm-bindgen = "0.6"
[package.metadata.wasm-pack.profile.release]
wasm-opt = false
+263
View File
@@ -0,0 +1,263 @@
#[cfg(target_arch = "wasm32")]
use decodal::HostEnvironment;
use decodal::{Data, EmptyLoader, Engine, SourceId, format_diagnostic_with};
#[cfg(target_arch = "wasm32")]
use decodal_language_service::{CompletionKind, LanguageService};
use wasm_bindgen::prelude::*;
#[cfg(any(target_arch = "wasm32", test))]
mod value;
#[cfg(target_arch = "wasm32")]
mod web_environment;
#[cfg(target_arch = "wasm32")]
use web_environment::JsEnvironment;
/// Evaluates one standalone Decodal source with no host globals or imports.
#[wasm_bindgen]
pub fn evaluate(source: &str) -> String {
encode_result(evaluate_inner(source))
}
/// A browser-facing language service configured entirely by its JavaScript host.
///
/// The host owns globals, import loading, and import completion. This keeps
/// filesystem, network, and virtual-project policy outside the WASM package.
#[cfg(target_arch = "wasm32")]
#[wasm_bindgen(js_name = DecodalLanguageService)]
pub struct WebLanguageService {
environment: JsEnvironment,
}
#[cfg(target_arch = "wasm32")]
#[wasm_bindgen(js_class = DecodalLanguageService)]
impl WebLanguageService {
#[wasm_bindgen(constructor)]
pub fn new(options: JsValue) -> Result<WebLanguageService, JsValue> {
Ok(Self {
environment: JsEnvironment::from_options(options)?,
})
}
/// Evaluates a source using the injected globals and import loader.
pub fn evaluate(&self, key: &str, name: &str, source: &str) -> String {
encode_result(evaluate_with_environment(
&self.environment,
key,
name,
source,
))
}
/// Completes a source using the same injected environment as evaluation.
///
/// Positions and returned ranges are UTF-16 offsets, matching browser
/// editors and the Language Server Protocol.
pub fn complete(&self, key: &str, source: &str, position: usize, explicit: bool) -> String {
encode_completion(&self.environment, key, source, position, explicit)
}
}
fn encode_result(result: Result<String, String>) -> String {
match result {
Ok(output) => serde_json::json!({ "ok": true, "output": output }).to_string(),
Err(error) => serde_json::json!({ "ok": false, "error": error }).to_string(),
}
}
fn evaluate_inner(source: &str) -> Result<String, String> {
let mut engine = Engine::new(EmptyLoader);
let module = match engine.add_root_source("playground", "playground", source) {
Ok(module) => module,
Err(error) => return Err(format_diagnostic_with_root(&error, "playground")),
};
let value = match engine.eval_module(module) {
Ok(value) => value,
Err(error) => return Err(engine.format_diagnostic(&error)),
};
let data = match engine.materialize(&value) {
Ok(data) => data,
Err(error) => return Err(engine.format_diagnostic(&error)),
};
Ok(format_data(&data, 0))
}
#[cfg(target_arch = "wasm32")]
fn evaluate_with_environment(
environment: &JsEnvironment,
key: &str,
name: &str,
source: &str,
) -> Result<String, String> {
let mut engine = environment
.create_engine()
.map_err(|diagnostic| diagnostic.message)?;
let module = match engine.add_root_source(key, name, source) {
Ok(module) => module,
Err(diagnostic) => return Err(engine.format_diagnostic(&diagnostic)),
};
let value = match engine.eval_module(module) {
Ok(value) => value,
Err(diagnostic) => return Err(engine.format_diagnostic(&diagnostic)),
};
let data = match engine.materialize(&value) {
Ok(data) => data,
Err(diagnostic) => return Err(engine.format_diagnostic(&diagnostic)),
};
Ok(format_data(&data, 0))
}
#[cfg(target_arch = "wasm32")]
fn encode_completion(
environment: &JsEnvironment,
key: &str,
source: &str,
position: usize,
explicit: bool,
) -> String {
let byte_position = utf16_offset_to_byte(source, position);
let service = LanguageService::new(environment);
let completion = match service.complete(key, source, byte_position, explicit) {
Ok(completion) => completion,
Err(diagnostic) => {
return serde_json::json!({
"ok": false,
"error": diagnostic.message,
})
.to_string();
}
};
let completion = completion.map(|completion| {
let from = byte_offset_to_utf16(source, completion.from);
let options = completion
.items
.into_iter()
.map(|item| {
serde_json::json!({
"label": item.label,
"kind": completion_kind_name(item.kind),
"detail": item.detail,
"priority": item.priority,
})
})
.collect::<Vec<_>>();
serde_json::json!({ "from": from, "options": options })
});
serde_json::json!({ "ok": true, "completion": completion }).to_string()
}
#[cfg(target_arch = "wasm32")]
fn completion_kind_name(kind: CompletionKind) -> &'static str {
match kind {
CompletionKind::Keyword => "keyword",
CompletionKind::Constant => "constant",
CompletionKind::Type => "type",
CompletionKind::Variable => "variable",
CompletionKind::Namespace => "namespace",
CompletionKind::Property => "property",
CompletionKind::File => "file",
}
}
#[cfg(any(target_arch = "wasm32", test))]
fn utf16_offset_to_byte(source: &str, offset: usize) -> usize {
let mut utf16 = 0usize;
for (byte, ch) in source.char_indices() {
if utf16 >= offset {
return byte;
}
let next = utf16 + ch.len_utf16();
if next > offset {
return byte;
}
utf16 = next;
}
source.len()
}
#[cfg(any(target_arch = "wasm32", test))]
fn byte_offset_to_utf16(source: &str, mut offset: usize) -> usize {
offset = offset.min(source.len());
while !source.is_char_boundary(offset) {
offset = offset.saturating_sub(1);
}
source[..offset].encode_utf16().count()
}
fn format_diagnostic_with_root(diagnostic: &decodal::Diagnostic, root_name: &str) -> String {
format_diagnostic_with(diagnostic, |source| {
(source == SourceId(0)).then_some(root_name)
})
}
fn format_data(data: &Data, indent: usize) -> String {
match data {
Data::String(value) => json_string(value),
Data::Int(value) => value.to_string(),
Data::Float(value) => value.to_string(),
Data::Bool(value) => value.to_string(),
Data::Array(items) => {
if items.is_empty() {
return String::from("[]");
}
let mut out = String::from("[\n");
for (index, item) in items.iter().enumerate() {
out.push_str(&" ".repeat(indent + 2));
out.push_str(&format_data(item, indent + 2));
if index + 1 != items.len() {
out.push(',');
}
out.push('\n');
}
out.push_str(&" ".repeat(indent));
out.push(']');
out
}
Data::Object(fields) => {
if fields.is_empty() {
return String::from("{}");
}
let mut out = String::from("{\n");
for (index, field) in fields.iter().enumerate() {
out.push_str(&" ".repeat(indent + 2));
out.push_str(&json_string(&field.name));
out.push_str(": ");
out.push_str(&format_data(&field.value, indent + 2));
if index + 1 != fields.len() {
out.push(',');
}
out.push('\n');
}
out.push_str(&" ".repeat(indent));
out.push('}');
out
}
}
}
fn json_string(value: &str) -> String {
serde_json::to_string(value).expect("strings are always JSON-serializable")
}
#[cfg(test)]
mod tests {
use super::{byte_offset_to_utf16, evaluate_inner, utf16_offset_to_byte};
#[test]
fn evaluates_standalone_sources() {
assert_eq!(
evaluate_inner("value = 1;").unwrap(),
"{\n \"value\": 1\n}"
);
}
#[test]
fn converts_web_utf16_offsets() {
let source = "a😀β";
assert_eq!(utf16_offset_to_byte(source, 0), 0);
assert_eq!(utf16_offset_to_byte(source, 1), 1);
assert_eq!(utf16_offset_to_byte(source, 3), 5);
assert_eq!(byte_offset_to_utf16(source, 5), 3);
assert_eq!(byte_offset_to_utf16(source, source.len()), 4);
}
}
+320
View File
@@ -0,0 +1,320 @@
use decodal::{CompareOp, Constraint, LiteralValue, PrimitiveType, Value};
use serde_json::{Map, Value as JsonValue};
pub(crate) fn from_json(value: &JsonValue) -> Result<Value, String> {
match value {
JsonValue::Null => Err(String::from("null is not a Decodal value")),
JsonValue::Bool(value) => Ok(Value::bool(*value)),
JsonValue::Number(value) => number(value),
JsonValue::String(value) => Ok(Value::string(value)),
JsonValue::Array(items) => items
.iter()
.enumerate()
.map(|(index, value)| {
from_json(value).map_err(|error| format!("array item {index}: {error}"))
})
.collect::<Result<Vec<_>, _>>()
.map(Value::array),
JsonValue::Object(fields) => object(fields),
}
}
fn number(value: &serde_json::Number) -> Result<Value, String> {
if let Some(value) = value.as_i64() {
return Ok(Value::int(value));
}
if let Some(value) = value.as_u64() {
return i64::try_from(value)
.map(Value::int)
.map_err(|_| String::from("integer value is outside the signed 64-bit range"));
}
value
.as_f64()
.filter(|value| value.is_finite())
.map(Value::float)
.ok_or_else(|| String::from("invalid numeric value"))
}
fn object(fields: &Map<String, JsonValue>) -> Result<Value, String> {
let Some(descriptor) = fields.get("$decodal") else {
return fields
.iter()
.map(|(name, value)| {
from_json(value)
.map(|value| (name.clone(), value))
.map_err(|error| format!("field `{name}`: {error}"))
})
.collect::<Result<Vec<_>, _>>()
.map(Value::object);
};
let descriptor = descriptor
.as_str()
.ok_or_else(|| String::from("`$decodal` must be a descriptor name"))?;
let constraints = parse_constraints(fields.get("constraints"))?;
let default = fields
.get("default")
.map(from_json)
.transpose()?
.map(Box::new);
match descriptor {
"Unknown" => {
let mut all_constraints = vec![Constraint::Unknown];
all_constraints.extend(constraints);
Ok(Value::Range {
constraints: all_constraints,
default,
})
}
"String" | "Int" | "Float" | "Bool" => {
let primitive = match descriptor {
"String" => PrimitiveType::String,
"Int" => PrimitiveType::Int,
"Float" => PrimitiveType::Float,
"Bool" => PrimitiveType::Bool,
_ => unreachable!(),
};
let mut all_constraints = vec![Constraint::Type(primitive)];
all_constraints.extend(constraints);
Ok(Value::Range {
constraints: all_constraints,
default,
})
}
"Array" => {
let item = fields
.get("item")
.ok_or_else(|| String::from("Array descriptor requires `item`"))?;
Ok(Value::ArrayRange {
item: Box::new(from_json(item)?),
constraints,
default,
})
}
"Map" => {
let value = fields
.get("value")
.ok_or_else(|| String::from("Map descriptor requires `value`"))?;
Ok(Value::MapRange {
value: Box::new(from_json(value)?),
constraints,
default,
})
}
"Object" => {
if !constraints.is_empty() {
return Err(String::from(
"Object descriptor does not accept `constraints`; constrain its fields or rest range",
));
}
if default.is_some() {
return Err(String::from(
"Object descriptor does not accept `default`; place defaults on its fields",
));
}
let object_fields = fields
.get("fields")
.and_then(JsonValue::as_object)
.ok_or_else(|| String::from("Object descriptor requires object `fields`"))?;
let value_fields = object_fields
.iter()
.map(|(name, value)| {
from_json(value)
.map(|value| (name.clone(), value))
.map_err(|error| format!("field `{name}`: {error}"))
})
.collect::<Result<Vec<_>, _>>()?;
let rest = fields
.get("rest")
.ok_or_else(|| String::from("Object descriptor requires `rest`"))?;
Ok(Value::object_with_rest(value_fields, from_json(rest)?))
}
"Range" => Ok(Value::Range {
constraints,
default,
}),
name => Err(format!("unknown Decodal value descriptor `{name}`")),
}
}
fn parse_constraints(value: Option<&JsonValue>) -> Result<Vec<Constraint>, String> {
let Some(value) = value else {
return Ok(Vec::new());
};
let items = value
.as_array()
.ok_or_else(|| String::from("`constraints` must be an array"))?;
items
.iter()
.enumerate()
.map(|(index, value)| {
parse_constraint(value).map_err(|error| format!("constraint {index}: {error}"))
})
.collect()
}
fn parse_constraint(value: &JsonValue) -> Result<Constraint, String> {
let fields = value
.as_object()
.ok_or_else(|| String::from("constraint must be an object"))?;
let kind = required_string(fields, "kind")?;
match kind {
"unknown" => Ok(Constraint::Unknown),
"type" => match required_string(fields, "value")? {
"String" => Ok(Constraint::Type(PrimitiveType::String)),
"Int" => Ok(Constraint::Type(PrimitiveType::Int)),
"Float" => Ok(Constraint::Type(PrimitiveType::Float)),
"Bool" => Ok(Constraint::Type(PrimitiveType::Bool)),
value => Err(format!("unknown primitive type `{value}`")),
},
"compare" => {
let operation = match required_string(fields, "op")? {
">" => CompareOp::Gt,
">=" => CompareOp::Gte,
"<" => CompareOp::Lt,
"<=" => CompareOp::Lte,
operation => return Err(format!("unknown comparison operator `{operation}`")),
};
let value = fields
.get("value")
.ok_or_else(|| String::from("compare constraint requires `value`"))?;
Ok(Constraint::Compare(operation, literal(value)?))
}
"regex" => Ok(Constraint::Regex(
required_string(fields, "value")?.to_owned(),
)),
"predicate" => Ok(Constraint::BuiltinPredicate(
required_string(fields, "value")?.to_owned(),
)),
kind => Err(format!("unknown constraint kind `{kind}`")),
}
}
fn literal(value: &JsonValue) -> Result<LiteralValue, String> {
match value {
JsonValue::String(value) => Ok(LiteralValue::String(value.clone())),
JsonValue::Bool(value) => Ok(LiteralValue::Bool(*value)),
JsonValue::Number(value) => match number(value)? {
Value::Int(value) => Ok(LiteralValue::Int(value)),
Value::Float(value) => Ok(LiteralValue::Float(value)),
_ => unreachable!(),
},
_ => Err(String::from("comparison value must be a primitive literal")),
}
}
fn required_string<'a>(fields: &'a Map<String, JsonValue>, name: &str) -> Result<&'a str, String> {
fields
.get(name)
.and_then(JsonValue::as_str)
.ok_or_else(|| format!("constraint requires string `{name}`"))
}
#[cfg(test)]
mod tests {
use decodal::{Constraint, LiteralValue, PrimitiveType, Value};
use super::from_json;
#[test]
fn converts_plain_js_shaped_values() {
let value = serde_json::json!({
"frontmatter": { "draft": false },
"body": "# Hello",
});
assert_eq!(
from_json(&value).unwrap(),
Value::object([
("body", Value::string("# Hello")),
(
"frontmatter",
Value::object([("draft", Value::bool(false))]),
),
])
);
}
#[test]
fn converts_schema_descriptors() {
let value = serde_json::json!({
"$decodal": "Array",
"item": {
"$decodal": "Int",
"constraints": [{ "kind": "compare", "op": ">", "value": 0 }],
},
"default": [1, 2],
});
assert_eq!(
from_json(&value).unwrap(),
Value::ArrayRange {
item: Box::new(Value::Range {
constraints: vec![
Constraint::Type(PrimitiveType::Int),
Constraint::Compare(decodal::CompareOp::Gt, LiteralValue::Int(0)),
],
default: None,
}),
constraints: Vec::new(),
default: Some(Box::new(Value::array([Value::int(1), Value::int(2),]))),
}
);
}
#[test]
fn converts_map_schema_descriptors() {
let value = serde_json::json!({
"$decodal": "Map",
"value": { "$decodal": "Bool" },
"default": {},
});
assert_eq!(
from_json(&value).unwrap(),
Value::MapRange {
value: Box::new(Value::bool_type()),
constraints: Vec::new(),
default: Some(Box::new(Value::object([] as [(&str, Value); 0]))),
}
);
}
#[test]
fn converts_unknown_and_open_object_descriptors() {
let value = serde_json::json!({
"$decodal": "Object",
"fields": {
"name": { "$decodal": "String" },
},
"rest": { "$decodal": "Unknown" },
});
assert_eq!(
from_json(&value).unwrap(),
Value::object_with_rest([("name", Value::string_type())], Value::unknown(),)
);
}
#[test]
fn converts_general_range_descriptors() {
let value = serde_json::json!({
"$decodal": "Range",
"constraints": [{ "kind": "predicate", "value": "available" }],
"default": true,
});
assert_eq!(
from_json(&value).unwrap(),
Value::Range {
constraints: vec![Constraint::BuiltinPredicate("available".into())],
default: Some(Box::new(Value::bool(true))),
}
);
}
#[test]
fn rejects_null_values() {
assert!(
from_json(&serde_json::Value::Null)
.unwrap_err()
.contains("null")
);
}
}
+209
View File
@@ -0,0 +1,209 @@
use std::collections::BTreeMap;
use decodal::{
Diagnostic, DiagnosticKind, Engine, HostEnvironment, ImportCandidate, ImportLoader,
LoadedImport, Span, Value as DecodalValue,
};
use js_sys::{Function, Reflect};
use serde_json::Value;
use wasm_bindgen::{JsCast, JsValue};
use crate::value;
pub(crate) struct JsEnvironment {
globals: BTreeMap<String, DecodalValue>,
load_import: Option<Function>,
complete_import: Option<Function>,
}
impl JsEnvironment {
pub(crate) fn from_options(options: JsValue) -> Result<Self, JsValue> {
if !options.is_null() && !options.is_undefined() && !options.is_object() {
return Err(JsValue::from_str(
"DecodalLanguageService options must be an object",
));
}
let globals = property(&options, "globals")?;
let globals = if globals.is_null() || globals.is_undefined() {
BTreeMap::new()
} else {
let globals: Value = serde_wasm_bindgen::from_value(globals).map_err(|error| {
JsValue::from_str(&format!("failed to read `globals`: {error}"))
})?;
let globals = globals.as_object().ok_or_else(|| {
JsValue::from_str("DecodalLanguageService `globals` must be an object")
})?;
globals
.iter()
.map(|(name, value)| {
value::from_json(value)
.map(|value| (name.clone(), value))
.map_err(|error| {
JsValue::from_str(&format!("invalid global `{name}`: {error}"))
})
})
.collect::<Result<BTreeMap<_, _>, _>>()?
};
Ok(Self {
globals,
load_import: optional_function(&options, "loadImport")?,
complete_import: optional_function(&options, "completeImport")?,
})
}
}
impl HostEnvironment for JsEnvironment {
type Loader = JsLoader;
fn create_loader(&self) -> Self::Loader {
JsLoader {
load_import: self.load_import.clone(),
complete_import: self.complete_import.clone(),
}
}
fn configure_engine(&self, engine: &mut Engine<Self::Loader>) -> decodal::Result<()> {
for (name, value) in &self.globals {
engine.bind_global(name, value.clone())?;
}
Ok(())
}
}
pub(crate) struct JsLoader {
load_import: Option<Function>,
complete_import: Option<Function>,
}
impl ImportLoader for JsLoader {
fn load(
&mut self,
current_key: Option<&str>,
specifier: &str,
) -> decodal::Result<LoadedImport> {
let callback = self.load_import.as_ref().ok_or_else(|| {
import_error(format!(
"no JavaScript `loadImport` callback is configured for `{specifier}`"
))
})?;
let current_key = current_key.map(JsValue::from_str).unwrap_or(JsValue::NULL);
let specifier = JsValue::from_str(specifier);
let loaded = callback
.call2(&JsValue::UNDEFINED, &current_key, &specifier)
.map_err(|error| import_error(js_error_message(error)))?;
let loaded: Value = serde_wasm_bindgen::from_value(loaded)
.map_err(|error| import_error(format!("invalid `loadImport` result: {error}")))?;
loaded_import(&loaded).map_err(import_error)
}
fn complete_import(
&mut self,
current_key: Option<&str>,
prefix: &str,
) -> decodal::Result<Vec<ImportCandidate>> {
let Some(callback) = &self.complete_import else {
return Ok(Vec::new());
};
let current_key = current_key.map(JsValue::from_str).unwrap_or(JsValue::NULL);
let prefix = JsValue::from_str(prefix);
let candidates = callback
.call2(&JsValue::UNDEFINED, &current_key, &prefix)
.map_err(|error| import_error(js_error_message(error)))?;
let candidates: Value = serde_wasm_bindgen::from_value(candidates)
.map_err(|error| import_error(format!("invalid `completeImport` result: {error}")))?;
import_candidates(&candidates).map_err(import_error)
}
}
fn loaded_import(value: &Value) -> Result<LoadedImport, String> {
let fields = value
.as_object()
.ok_or_else(|| String::from("`loadImport` must return an object"))?;
let kind = string_field(fields, "kind")?;
let key = string_field(fields, "key")?;
match kind {
"source" => {
let source = string_field(fields, "source")?;
let name = fields.get("name").and_then(Value::as_str).unwrap_or(key);
Ok(LoadedImport::source(key, name, source))
}
"value" => {
let value = fields
.get("value")
.ok_or_else(|| String::from("value import requires `value`"))?;
let value = value::from_json(value)
.map_err(|error| format!("invalid imported value: {error}"))?;
Ok(LoadedImport::value(key, value))
}
kind => Err(format!("unknown `loadImport` result kind `{kind}`")),
}
}
fn import_candidates(value: &Value) -> Result<Vec<ImportCandidate>, String> {
value
.as_array()
.ok_or_else(|| String::from("`completeImport` must return an array"))?
.iter()
.enumerate()
.map(|(index, value)| match value {
Value::String(specifier) => Ok(ImportCandidate::new(specifier)),
Value::Object(fields) => {
let mut candidate = ImportCandidate::new(string_field(fields, "specifier")?);
if let Some(detail) = fields.get("detail") {
let detail = detail.as_str().ok_or_else(|| {
format!("import candidate {index} `detail` must be a string")
})?;
candidate = candidate.with_detail(detail);
}
Ok(candidate)
}
_ => Err(format!(
"import candidate {index} must be a string or object"
)),
})
.collect()
}
fn string_field<'a>(
fields: &'a serde_json::Map<String, Value>,
name: &str,
) -> Result<&'a str, String> {
fields
.get(name)
.and_then(Value::as_str)
.ok_or_else(|| format!("`loadImport` result requires string `{name}`"))
}
fn property(options: &JsValue, name: &str) -> Result<JsValue, JsValue> {
if options.is_null() || options.is_undefined() {
return Ok(JsValue::UNDEFINED);
}
Reflect::get(options, &JsValue::from_str(name))
}
fn optional_function(options: &JsValue, name: &str) -> Result<Option<Function>, JsValue> {
let value = property(options, name)?;
if value.is_null() || value.is_undefined() {
return Ok(None);
}
value
.dyn_into::<Function>()
.map(Some)
.map_err(|_| JsValue::from_str(&format!("`{name}` must be a function")))
}
fn js_error_message(value: JsValue) -> String {
if let Some(message) = value.as_string() {
return message;
}
Reflect::get(&value, &JsValue::from_str("message"))
.ok()
.and_then(|message| message.as_string())
.unwrap_or_else(|| String::from("JavaScript import callback failed"))
}
fn import_error(message: impl Into<String>) -> Diagnostic {
Diagnostic::new(DiagnosticKind::Import, Span::default(), message)
}
+2
View File
@@ -4,11 +4,13 @@ pkgs.mkShell {
cargo
clippy
git
lld
nodejs
nixfmt
rustc
rustfmt
tree-sitter
wasm-pack
];
shellHook = ''
+140
View File
@@ -0,0 +1,140 @@
# Components
Decodal is split into a small runtime core and separate syntax tooling components.
The public surface is intentionally organized by use case: execute Decodal with Rust or WebAssembly, edit Decodal on the Web with Lezer and CodeMirror, and integrate Decodal into general-purpose editors with Tree-sitter.
## Runtime components
### Rust crate
The `decodal` crate is the primary Rust runtime and embedding API.
It owns parsing, evaluation, materialization, diagnostics, host-provided values, and schema/decode traits.
Rust applications should use this crate when they want to load Decodal source, evaluate it, or embed Decodal into a host program.
Important paths:
```text
crates/decodal-core/
crates/decodal-derive/
```
`decodal-derive` provides optional derive macros for Rust struct integration.
It is a companion to the runtime crate rather than an editor or syntax-highlighting component.
### WebAssembly package
`decodal-wasm` exposes the runtime to browsers.
The documentation site playground uses it to evaluate Decodal entirely in the browser.
Important paths:
```text
crates/decodal-wasm/
packages/decodal-wasm/
```
The npm package is `decodal-wasm`.
The JSR package is `@hare/decodal-wasm`.
The generated files in `packages/decodal-wasm/` are committed so the site can build without requiring every consumer to run `wasm-pack` first.
The WebAssembly package is for execution, not syntax highlighting.
## Language tools
Semantic editor integration lives in the host-configurable language service crate.
It depends only on the runtime and accepts the same `HostEnvironment` implementation used by a production application.
The LSP crate adapts that service to the Language Server Protocol over stdin/stdout.
It provides full document synchronization, semantic diagnostics, and whole-document formatting.
A host-specific LSP binary injects its loader and global schema configuration without reimplementing evaluation rules.
Important paths:
```text
crates/decodal-language-service/
crates/decodal-lsp/
```
The default `decodal-lsp` binary reads Decodal imports from the filesystem.
Embedded hosts can call its library entry point with a custom `LspEnvironment` to reuse structured imports and to make unsaved external documents, such as Markdown, visible to the loader.
Source formatting lives in a separate Rust language tools crate.
It uses the canonical Decodal AST together with the runtime lexer's lossless syntax tokens, so native LSP and WebAssembly callers execute the same formatter implementation.
This component is responsible for operations that must preserve source text details such as comments and whitespace.
It is used by the CodeMirror package's bundled formatter WebAssembly and can also be used by an LSP adapter for formatting.
It does not depend on Tree-sitter or Lezer.
Important paths:
```text
crates/decodal-language-tools/
```
The current language tools crate exposes the formatter.
## Web editor components
The Web playground editor uses CodeMirror 6 with a generated Lezer parser.
Lezer provides syntax highlighting, folding, indentation, and editor syntax tree behavior. The browser formatter command calls the canonical Rust formatter compiled to WebAssembly.
Important paths:
```text
editors/lezer-decodal/decodal.grammar
packages/decodal-codemirror/src/decodal.js
packages/decodal-codemirror/src/decodal-parser.js
packages/decodal-codemirror/src/decodal-parser.terms.js
packages/decodal-codemirror/src/format.js
packages/decodal-codemirror/wasm/
```
The npm package is `decodal-codemirror`.
The JSR package is `@hare/decodal-codemirror`.
The Lezer grammar is derived from the canonical grammar documentation, but it is not a literal copy of the EBNF.
Precedence and token conflict handling are represented in the Lezer grammar in the form CodeMirror needs.
## General editor components
Tree-sitter is the portable editor-integration grammar.
Editors such as Zed, Neovim, Helix, and Emacs should consume this component when they need Decodal parsing or highlighting outside the Web playground.
Important paths:
```text
editors/tree-sitter-decodal/grammar.js
editors/tree-sitter-decodal/queries/highlights.scm
editors/tree-sitter-decodal/queries/locals.scm
editors/tree-sitter-decodal/src/
```
The generated parser sources under `editors/tree-sitter-decodal/src/` are committed so editor integrations can consume the grammar without regenerating it first.
## Canonical grammar
The human-readable grammar lives in:
```text
doc/manual/souce/language/grammar.md
```
This EBNF is the language-level reference.
The Rust parser, Lezer grammar, and Tree-sitter grammar should be kept aligned with it, but each implementation may encode precedence and recovery behavior in the form required by its parser generator or runtime.
## Syntax token API
The `decodal` crate exposes `tokenize_source`, `tokenize_source_with_source_id`, `SyntaxToken`, and `SyntaxTokenKind` for source-preserving tooling.
Comments have explicit tokens, while whitespace is represented by gaps between token spans and can be recovered from the original source.
The evaluator parser and formatter therefore share one lexical definition without making Tree-sitter an upstream dependency.
Consumers should otherwise use the component matching their environment:
- Rust execution and embedding: `decodal`
- Browser execution: `decodal-wasm`
- Web formatting and editor syntax: canonical formatter / CodeMirror / Lezer
- Semantic editor analysis: `decodal-language-service`
- Language Server Protocol integration: `decodal-lsp`
- Rust formatting: `decodal-language-tools`
- General editor syntax: Tree-sitter
Tree-sitter and Lezer remain downstream editor grammars and are not dependencies of the runtime formatter.
@@ -1,6 +1,6 @@
# Composition and Materialization
`&``//``default`、materialize は、runtime value の variant に基づいて処理する。
`&``//``as``default`、materialize は、runtime value の variant に基づいて処理する。
## `&`
@@ -59,8 +59,9 @@ Config = MyConfig & {
## default の合成
初期方針では、`&` による異なる default 同士の合成は conflict とする。
同じ default は同一候補として扱ってよい
`&` で片方だけが default を持つ場合、その default を保持する。
同じ thunk 由来の default 同士は同一 default として保持する
異なる default 同士は conflict とする。
```text
merge_default(None, None) -> None
@@ -88,7 +89,24 @@ patch(a: RuntimeValue, b: RuntimeValue) -> RuntimeValue
- 配列、scalar、function は右辺置換とする。
`//` は制約を保持するための演算子ではない。
制約を満たす具体化に`&` を使う。
対称な制約合成には `&` を使い、左辺が右辺より狭いことを確認しながら合成する場合`as` を使う。
## `as`
`as` は narrower と wider の向きを固定した範囲包含確認と合成である。
```text
apply_as(narrower: RuntimeValue, wider: RuntimeValue)
-> RuntimeValue | Diagnostic
```
abstract / abstract では、左辺 constraints が右辺 constraints を包含することを確認し、左辺を結果にする。
concrete / abstract では、concrete value が右辺 constraints を満たすことを確認する。
object / object では右辺を field domain とする。左辺にしかない field は診断し、両側にある field は再帰的に `apply_as` し、右辺にしかない field は thunk を force せずそのまま結果へ残す。
この扱いは default の有無に依存しない。default は包含判定の根拠でも `as` の補完値でもなく、後の materialize だけが選択する。
`ArrayItems(wider)` は concrete array の全要素を、`MapValues(wider)` は concrete object の全 field value を同じ関数で絞り込む。
## materialize
@@ -117,3 +135,9 @@ Abstract { constraints, default: None }
materialize は default を採用する唯一の段階である。
通常評価中に明示値が得られた場合、default は採用されない。
## Diagnostic context
Composition and materialization keep source spans for object fields, constraints, defaults, and thunks where possible.
When a conflict occurs, the diagnostic should identify the operation span and related participant spans, such as the left field, right field, constraint, or default value.
For object materialization errors, diagnostics also include the field path being processed when known.
@@ -10,10 +10,22 @@ Diagnostic {
kind: DiagnosticKind
span: Span
message: String
notes: Vec<Note>
labels: Vec<DiagnosticLabel>
notes: Vec<String>
}
DiagnosticLabel {
span: Span
message: String
}
```
`span` は primary location を示す。
表示時には `Span.source` を source id のまま出すのではなく、可能な限り file path や virtual file name に解決する。
`labels` は同じ error に関係する追加 location を示す。
`notes` は source location を持たない semantic context を示す。
合成や materialize の失敗では、衝突した constraint、value、default、または処理中 field path を label に含める。
代表的な diagnostic kind:
- syntax error
@@ -27,6 +39,9 @@ Diagnostic {
- match failure
- materialization failure
Structured imports use semantic provenance rather than synthetic source spans.
Constraint failures report the imported value's stable key and logical field or array path alongside the source span of the Decodal constraint that rejected it.
## エラーは値ではない
評価失敗は `RuntimeValue` ではなく `Diagnostic` を返す。
@@ -38,6 +53,18 @@ Result<RuntimeValue, Diagnostic>
これにより、制約違反、未定義識別子、循環依存、import 失敗などが通常値として流れることを避ける。
## 合成と materialize の diagnostic
合成や materialize の失敗は、以下を示す。
1. どの段階で失敗したか: composition、patch、materialization
2. どの field path を処理中だったか
3. どの constraint、value、default が衝突したか
4. なぜ合成または materialize できないか
例えば default が constraint を満たさない場合は、constraint の位置と default value の位置の両方を label として持つ。
object field の合成で concrete value が衝突する場合は、左辺 field と右辺 field の位置を label として持つ。
## `try / catch` は core に入れない
汎用 `try / catch` は core に入れない。
@@ -47,21 +74,6 @@ fallback は有限で明示的な仕組みに限定する。
- `default`: 未指定値の fallback。
- `match`: 有限 pattern に基づく分岐。
- optional import: ファイル不存在など、限定された失敗だけを fallback 可能にする候補。
- optional field access: field 不在だけを fallback 可能にする候補。
- union / tagged schema: 複数 schema の選択を明示的に表す将来候補。
## optional fallback の扱い
optional import や optional field access を導入する場合も、捕捉できる失敗は限定する。
例として optional import は、ファイル不存在だけを fallback 可能にし、parse error や import 先の制約違反は diagnostic として報告する方がよい。
```text
optional import:
file not found -> fallback
parse error -> diagnostic
eval error -> diagnostic
```
この方針により、fallback は通常の値選択として扱い、エラー内容に依存した実行時分岐は避ける。
`Unknown` は明示的に書く最上位のabstract rangeであり、評価エラーを包む値ではない。
field 不在や未解決 identifier は `Unknown` へ変換せず、diagnostic として報告する。
+185 -24
View File
@@ -1,7 +1,7 @@
# Embedding API
Decodal core can be embedded without giving the core crate access to a filesystem.
The host supplies imported sources through `SourceLoader` and may also provide global bindings through the host prelude API.
The host supplies imports through `ImportLoader` and may also provide global bindings through the host prelude API.
## Host prelude
@@ -17,23 +17,23 @@ let / function env
```
Module top-level bindings shadow prelude bindings.
Primitive type names such as `String`, `Int`, `Float`, and `Bool` are handled before environment lookup, so they are reserved and cannot be shadowed by host bindings.
Primitive type names such as `String`, `Int`, `Float`, and `Bool`, plus the top range `Unknown`, are handled before environment lookup, so they are reserved and cannot be shadowed by host bindings.
## Global bindings
The host can bind values before adding or evaluating user sources.
```rust
use decodal_core::{EmptyLoader, Engine, HostValue};
use decodal::{EmptyLoader, Engine, Value};
let mut engine = Engine::new(EmptyLoader);
engine.bind_global(
"Service",
HostValue::object([
("name", HostValue::string_type()),
("port", HostValue::int_type().gt(443).default_int(8443)?),
("enabled", HostValue::bool_type().default_bool(true)?),
Value::object([
("name", Value::string_type()),
("port", Value::int_type().gt(443).default_int(8443)?),
("enabled", Value::bool_type().default_bool(true)?),
]),
)?;
```
@@ -41,38 +41,55 @@ engine.bind_global(
A user source can then refer to `Service` without importing it.
```dcdl
Service & {
{
name = "api";
port = 9443;
}
} as Service
```
## HostValue
## Value
`HostValue` is the public builder-facing value representation for embedding.
`Value` is the public builder-facing value representation for embedding.
It keeps host code from constructing internal `ThunkId` or `ObjectValue` values directly.
```text
HostValue =
Value =
String
Int
Float
Bool
Array(Vec<HostValue>)
Object(Vec<HostField>)
Abstract { constraints, default }
Array(Vec<Value>)
ArrayRange { item, constraints, default }
MapRange { value, constraints, default }
Object { fields, rest: Option<Value> }
Range { constraints, default }
```
When a host value is bound, the engine internalizes it into `RuntimeValue` and allocates value thunks for object fields, array items, and defaults.
When a value is bound by the host, the engine internalizes it into `RuntimeValue` and allocates value thunks for object fields, array items, and defaults.
## Abstract host objects
`Value::array_of(item)` builds an array constraint with a required element schema.
There is no `Value` constructor for an unconstrained abstract array.
A host-provided schema object is represented as a concrete object structure whose fields may contain abstract values.
`Value::map_of(value)` builds a map constraint whose arbitrary object field values must satisfy `value`.
`BTreeMap<String, T>` and, with `std`, `HashMap<String, T>` implement `DecodalSchema`, `DecodalDecode`, and `IntoValue` using this representation.
`Value::unknown()` builds the top abstract range. `Value::object_with_rest(fields, rest)` builds an object with named fields and a range for all remaining fields.
```rust
HostValue::object([
("name", HostValue::string_type()),
("port", HostValue::int_type().gt(443).default_int(8443)?),
Value::object_with_rest(
[("enabled", Value::bool_type().default_bool(true)?)],
Value::unknown(),
)
```
## Objects containing ranges
A host-provided schema object is represented as a concrete object structure whose fields may contain ranges.
```rust
Value::object([
("name", Value::string_type()),
("port", Value::int_type().gt(443).default_int(8443)?),
])
```
@@ -87,11 +104,155 @@ Concrete(Object {
This matches the runtime model used for Decodal source-defined schema objects.
## SourceLoader and prelude together
## Typed Rust integration
`SourceLoader` and host prelude bindings are independent mechanisms.
Hosts can enable the `derive` feature on `decodal` to keep a Rust struct, the Decodal schema, and the decoded result in sync.
The derive implements two traits from the `decodal` crate:
- Use `SourceLoader` when user sources should explicitly import host-provided modules.
- `DecodalSchema`: builds a `Value` schema that can be passed to `Engine::bind_global`.
- `DecodalDecode`: decodes materialized `Data` back into the Rust type.
An explicitly marked map field can receive the open portion of an object.
```rust
use std::collections::BTreeMap;
use decodal::{Data, Decodal};
#[derive(Decodal)]
struct OpenConfig {
enabled: bool,
#[decodal(rest)]
extra: BTreeMap<String, Data>,
}
```
The schema generated for `extra` is `...Unknown`. A typed receiver such as `BTreeMap<String, String>` generates `...String` instead. During decode, named top-level fields are excluded and all remaining fields are collected into the map.
Only one `#[decodal(rest)]` field is allowed. It must implement `DecodalRest`; the core provides implementations for `BTreeMap<String, T>` and, with `std`, `HashMap<String, T>`. `rename`, `default`, and field constraints cannot be combined with `rest`. Without a rest receiver, a derived struct remains a closed object range.
```rust
use decodal::{Decodal, DecodalDecode, DecodalSchema, EmptyLoader, Engine};
#[derive(Decodal)]
struct Service {
name: String,
#[decodal(gt = 443, default = 8443)]
port: i64,
#[decodal(rename = "feature.enable", default = true)]
feature_enabled: bool,
}
let mut engine = Engine::new(EmptyLoader);
engine.bind_global("Service", Service::decodal_schema())?;
let value = engine.eval_module(module)?;
let data = engine.materialize(&value)?;
let service = Service::decodal_decode(&data)?;
```
Supported field attributes are intentionally small:
- `rename = "path.to.field"`
- `default`
- `default = value`
- numeric constraints: `gt`, `gte`, `lt`, `lte`
The derive does not add host callbacks or reflection.
It only generates schema construction and typed decoding code.
## ImportLoader and prelude together
`ImportLoader` and host prelude bindings are independent mechanisms.
- Use `ImportLoader` when user sources should explicitly import host-provided sources or values.
- Use prelude bindings when host-provided schemas or constants should be globally available.
Both mechanisms share the same runtime evaluator, thunk model, and materialization rules.
## Shared host environment
An embedded application can implement `HostEnvironment` to keep loader creation and global binding setup in one place.
Both production evaluation and semantic editor tooling create their engines from this environment, preventing the editor from drifting onto a separate validation path.
```rust
use decodal::{Engine, HostEnvironment};
struct AppEnvironment;
impl HostEnvironment for AppEnvironment {
type Loader = ContentLoader;
fn create_loader(&self) -> Self::Loader {
ContentLoader::new()
}
fn configure_engine(
&self,
engine: &mut Engine<Self::Loader>,
) -> decodal::Result<()> {
engine.bind_global("Site", site_schema())?;
Ok(())
}
}
let environment = AppEnvironment;
let mut engine = environment.create_engine()?;
```
The semantic service accepts the same environment by value:
```rust
use decodal_language_service::LanguageService;
let service = LanguageService::new(&environment);
let analysis = service.analyze("site.dcdl", "site.dcdl", source);
```
Each analysis uses a fresh engine and runs the normal parse, evaluate, and materialize pipeline.
An environment may create loaders backed by shared filesystem, database, or editor-overlay state when repeated analysis needs a current workspace snapshot.
The LSP adapter constructs that environment after receiving the client's initialization parameters:
```rust
use decodal_lsp::{LspEnvironment, run_stdio};
impl LspEnvironment for AppEnvironment {}
run_stdio(|initialize| {
let _ = initialize;
Ok(AppEnvironment)
})?;
```
`LspEnvironment` adds document lifecycle hooks on top of `HostEnvironment`.
A `run_stdio` environment factory always receives the client's `InitializeParams`; the host decides whether to use its workspace folders, initialization options, and capabilities or ignore them.
A host that keeps unsaved buffers in shared state can update them from `open_document`, `change_document`, and `close_document`; every subsequent diagnostic pass creates the normal import loader from that updated environment.
Synchronized non-Decodal documents are passed through these hooks but are not evaluated as Decodal roots, so a loader can parse an unsaved Markdown file into `Value` and immediately revalidate the open Decodal documents that import it.
## Structured imports
`ImportLoader::load` returns either `LoadedImport::Source` or `LoadedImport::Value`.
The value variant carries a `Value`, allowing a host to parse non-Decodal resources such as Markdown into an application-specific structure.
```text
import "./post.md"
-> host parses content
-> LoadedImport::Value {
key: "content/post.md",
value: { frontmatter: {...}, body: "..." }
}
-> Engine internalizes Value
-> normal composition and materialization
```
The core does not select content types or bundle Markdown/frontmatter parsers.
The loader owns path resolution, media or extension dispatch, parsing rules, and parse diagnostics.
The stable loader key is also used to cache structured imports.
When a structured value fails a Decodal constraint, the diagnostic keeps the Decodal constraint span and identifies the imported value by its stable key and logical value path, such as `content/post.md` and `frontmatter.draft`.
`Value` does not need source spans: syntax diagnostics for the external format remain the loader's responsibility, while cross-value validation reports semantic provenance.
`load` is the single import hook: loaders dispatch by extension, media type, or another host-defined rule and return the appropriate variant directly.
+22 -9
View File
@@ -16,13 +16,17 @@ demand-driven evaluation
├─ load imported module on demand
├─ evaluate expression
├─ compose `&`
─ patch `//`
─ patch `//`
└─ validate `as`
materialize
data / diagnostics
```
Production applications and semantic language services can construct this pipeline through the same host-defined `HostEnvironment`.
The environment creates the `ImportLoader` and installs host globals before the root module is registered.
## lexer / parser
lexer / parser は source を AST に変換する。
@@ -66,14 +70,16 @@ ModuleRegistry:
処理系は、まず root module を parse / desugar して registry に登録する。
import 先 module は、この段階で全て読み込む必要はない。
import expression が評価されたとき、処理系は `SourceLoader` に現在の module key と import specifier を渡す。
loader は module key、表示名、source text を返す。
module registry は key が未登録なら対象 module を parse / desugar して登録する。
登録された module は module root thunk を持つ。
同じ module が複数回 import された場合は、同じ `ModuleId` を返す
import expression が評価されたとき、処理系は `ImportLoader::load` に現在の module key と import specifier を渡す。
loader は module key、表示名、および DCDL source text または構造化済み `Value` を返す。
DCDL source の場合、module registry は key が未登録なら対象 module を parse / desugar して登録する。
登録された source module は module root thunk を持つ。
構造化値の場合、host が返した `Value` を runtime value に internalize した root thunk を key でキャッシュする
同じ key が複数回 import された場合は、同じ source module または structured root thunk を使う。
つまり import は module を即時評価しない。
つまり source import は module を即時評価しない。
module を読み込み、module root を thunk として登録するだけにする。
structured import も root value を thunk として保持し、object field や array item の既存 thunk model を利用する。
AST の `ExprId` は module-local である。
そのため runtime が保持する式参照は `ExprRef { module, expr }` として module-qualified にする。
@@ -100,9 +106,16 @@ eval(A // B):
a = eval(A)
b = eval(B)
patch(a, b)
eval(A as S):
narrower = eval(A)
wider = eval(S)
apply_as(narrower, wider)
```
`&` は制約を保った合成を行い、`//` は右辺優先の deep patch を行う。
`&` は制約を保った対称な合成を行い、`//` は右辺優先の deep patch を行う。
`as` は左辺が右辺より狭いことを再帰的に確認する。左辺で絞られた部分は左辺を使い、右辺だけの部分は abstract のまま保持する。
default は `as` では選択せず、materialize まで遅延する。
詳細は [Composition and Materialization](./composition-and-materialization.md) に置く。
## resolver / binder
@@ -135,7 +148,7 @@ resolver / binder を追加すると、以下を早期に診断しやすくな
- import path の一部静的解決
- span 付き diagnostic の精度向上
ただし、Decodal の制約検証は独立した type checking pass ではなく、`&` の合成時materialize 時に行う。
ただし、Decodal の制約検証は独立した type checking pass ではなく、`&` の合成時`as` の適用時、materialize 時に行う。
## materialize
+39 -5
View File
@@ -1,10 +1,44 @@
# Features
Decodal keeps the embedded core small by making heavy functionality opt-in.
Decodal embedded use と小さい runtime を優先する。
言語機能を追加するときは、値の合成・検証・materialization に直接必要なものを core に残し、重い依存や高度な推論は optional feature または外部 tooling に分ける。
## Core feature boundary
Core に入れる機能は、基本的に deterministic な value transformation に限る。
- arithmetic / logical / comparison operators
- array concat
- object / constraint composition
- asymmetric range refinement, `Unknown`, map constraints, and object rest constraints
- default materialization
- pure function evaluation
- host supplied import evaluation
Core に入れないものは以下である。
- filesystem / network / environment access
- time / random
- mutation
- reflection or existence probing
- arbitrary host function calls
- symbolic constraint solving beyond simple normalization
未解決 identifier や missing field は明示的な `Unknown` range とは異なり、diagnostic になる。
この方針により、存在チェックや optional chaining のような dynamic object inspection は core language の対象外とする。
## Constraint reasoning
Constraint normalization は軽量な範囲に留める。
primitive type conflict や明らかな numeric bound conflict は合成時に検出してよい。
一方で、symbolic arithmetic、boolean algebra、regex intersection、array length dependent typing のような重い推論は行わない。
評価済みの concrete value に対する検証は runtime / materialization で行う。
静的に完全な型検査フェーズを増やすのではなく、parse、evaluate、compose、materialize の各段階で自然に分かる error を diagnostic として返す。
## Core defaults
`decodal-core` defaults to `std` only.
`decodal` defaults to `std` only.
```toml
[features]
@@ -13,7 +47,7 @@ std = []
regex = ["std", "dep:regex"]
```
Building `decodal-core` with `--no-default-features` keeps the core in `no_std + alloc` mode and avoids optional dependencies.
Building `decodal` with `--no-default-features` keeps the core in `no_std + alloc` mode and avoids optional dependencies.
## Regex
@@ -21,7 +55,7 @@ Regex constraints are implemented behind the `regex` feature.
When the feature is disabled, regex constraints parse and compose, but validating a concrete value against them returns an unsupported feature diagnostic.
```sh
cargo run -q -p decodal --features regex -- examples/regex/main.dcdl
cargo run -q -p decodal-cli --features regex -- examples/regex/main.dcdl
```
Regex constraints are accumulated during `&` composition.
@@ -30,5 +64,5 @@ Concrete strings must match every regex constraint attached to the abstract valu
## CLI features
`decodal-cli` exposes a matching `regex` feature that enables `decodal-core/regex`.
`decodal-cli` exposes a matching `regex` feature that enables `decodal/regex`.
The feature is not enabled by default so the default CLI binary remains small.
+1 -1
View File
@@ -6,7 +6,7 @@
## 方針
初期処理系は AST interpreter として実装する。
bytecode VM や JIT ではなく、AST を demand-driven に評価することで、遅延評価、循環参照、`default``&``//` の意味論を小さく実装する。
bytecode VM や JIT ではなく、AST を demand-driven に評価することで、遅延評価、循環参照、`default``&``//``as` の意味論を小さく実装する。
## 構成
+46 -2
View File
@@ -14,6 +14,10 @@ RuntimeValue =
`Concrete` は明示的な値である。
`Abstract` は、まだ具体値に確定していない制約付きの値である。
Decodal は `Unknown` を、すべてのDecodal値を包含する最上位の abstract range として持つ。
これは存在する具体値の型情報を消す `Any` ではない。具体値を `Unknown` に対して検証した場合は具体値を保持し、`Unknown` のままmaterializeしようとした場合は diagnostic になる。
識別子や field が解決できない状態とは区別し、それらは従来どおりその場で diagnostic になる。
## ConcreteValue
```text
@@ -32,7 +36,16 @@ object は concrete structure として扱う。
```text
ObjectValue:
fields: Map<Symbol, ThunkId>
fields: Map<Symbol, ObjectField>
rest: Option<ObjectRest>
ObjectField:
value: ThunkId
span: Span
ObjectRest:
value: ThunkId
span: Span
```
例えば以下の schema object は、object 自体は concrete だが、field の値は abstract value になる。
@@ -57,9 +70,14 @@ Concrete(Object {
```text
AbstractValue {
constraints: Vec<Constraint>
constraints: Vec<ConstraintEntry>
default: Option<ThunkId>
}
ConstraintEntry {
constraint: Constraint
span: Span
}
```
`default``AbstractValue` にだけ存在する。
@@ -88,19 +106,45 @@ port = 8000;
Concrete(Int(8000))
```
## Runtime scope
Decodal runtime は application runtime ではなく、pure value evaluator である。
同じ source、同じ import results、同じ host globals が与えられた場合、評価結果は決定的である。
runtime が扱う責務は以下に限る。
- expression を評価する。
- thunk を必要に応じて force する。
- concrete / abstract value を合成する。
- materialize 時に constraint を検証する。
runtime は filesystem、network、environment variable、time、random、mutation を扱わない。
core における import は host supplied source または structured value を受け取る境界であり、filesystem access ではない。
## Constraint
constraint は concrete value とは別の型として扱う。
```text
Constraint =
Unknown
Type(PrimitiveType)
ArrayItems(ThunkId)
MapValues(ThunkId)
Compare(Op, Literal)
Regex(Pattern)
BuiltinPredicate(Symbol)
ObjectConstraint(...)
```
`ArrayItems` は配列そのものの型と、すべての要素へ合成する schema thunk を表す。
配列用の primitive type は持たず、抽象配列には必ず要素制約が必要である。
`MapValues` は object の key 集合を制限せず、すべての field value へ適用する schema thunk を表す。
連想配列は materialize 後も `Data::Object` になり、別の data variant は持たない。
`ObjectValue.rest` は名前付きfieldを持つobjectの残余field rangeを表す。rest自体はfieldを生成せず、materializeは実在するfieldだけを出力する。
初期実装では、object の形は主に `Concrete(Object)` の field に `Abstract` を置くことで表現する。
object 全体にかかる constraint は必要になった時点で追加する。
@@ -84,5 +84,5 @@ module root や field は thunk として保持され、参照されたときだ
関数引数は thunk として関数の environment に束縛する。
関数本体で引数が参照されたときだけ force する。
任意の関数呼び出し結果グローバルに memoize する必要はない。
関数呼び出し結果そのものはグローバルに memoize ない。
field に束縛された関数呼び出し結果は、その field thunk の評価結果として memoize される。
+161 -10
View File
@@ -9,15 +9,158 @@ Run the normal Rust checks from the repository root.
```sh
cargo fmt --check
cargo test
cargo check -p decodal-core --no-default-features
cargo check -p decodal --no-default-features
nix flake check
```
Run the default stdio language server with:
```sh
cargo run -q -p decodal-lsp
```
The protocol integration tests use an in-memory LSP connection and can be run independently:
```sh
cargo test -p decodal-lsp
```
Regex support is optional and should be tested explicitly when touched.
```sh
cargo test -p decodal-core --features regex
cargo run -q -p decodal --features regex -- examples/regex/main.dcdl
cargo test -p decodal --features regex
cargo run -q -p decodal-cli --features regex -- examples/regex/main.dcdl
```
## crates.io release
The crates.io release contains `decodal`, `decodal-derive`, `decodal-language-service`, `decodal-language-tools`, and `decodal-lsp`.
`decodal-cli` and the Rust source crate `decodal-wasm` remain repository-only packages.
The generated WebAssembly package under `packages/decodal-wasm/` is published to npm and JSR.
The CodeMirror package bundles the generated formatter WebAssembly from `decodal-language-tools`.
The project is dual licensed as `MIT OR Apache-2.0`.
The authoritative validation commands, dependency order, and publish commands are maintained in `RELEASING.md` at the repository root.
## Web site and playground
The Astro documentation site and browser playground are kept in:
```text
site/decodal-site/
```
The site imports Markdown files from `doc/manual/souce/` and renders them as mdBook-style pages.
The playground loads `decodal-wasm` for evaluation and `decodal-codemirror` for editor features and source formatting.
Important files:
```text
site/decodal-site/src/pages/docs/[...slug].astro
site/decodal-site/src/pages/playground.astro
site/decodal-site/src/scripts/playground.js
site/decodal-site/src/scripts/playground-examples.js
site/decodal-site/src/layouts/ManualLayout.astro
site/decodal-site/src/lib/docs.js
site/decodal-site/src/lib/highlight.js
packages/decodal-codemirror/src/decodal.js
packages/decodal-codemirror/src/decodal-parser.js
packages/decodal-wasm/decodal_wasm.js
packages/decodal-wasm/decodal_wasm_bg.wasm
packages/decodal-codemirror/wasm/decodal_language_tools.js
packages/decodal-codemirror/wasm/decodal_language_tools_bg.wasm
crates/decodal-wasm/src/lib.rs
crates/decodal-language-tools/src/lib.rs
```
Build the WebAssembly packages before building the site:
```sh
cd site/decodal-site
npm install
npm run build:wasm
npm run build
```
`npm run build:wasm` builds both generated WebAssembly packages.
`npm run build:wasm:runtime` writes generated runtime files into `packages/decodal-wasm/`.
`npm run build:wasm:formatter` writes generated formatter files into `packages/decodal-codemirror/wasm/`.
`npm run build` builds only the static site from the current package files.
Use `npm run build:all` to regenerate both WebAssembly packages and then build the static site in one command.
These generated files are committed so the site can be built without requiring every consumer to regenerate the wasm packages first.
Publish the generated WebAssembly package from its package directory:
```sh
cd packages/decodal-wasm
npm publish
npx jsr publish
```
Run dry-runs first when preparing a release:
```sh
npm pack --dry-run
npx jsr publish --dry-run
```
The npm package name is `decodal-wasm`.
The JSR package name is `@hare/decodal-wasm`.
The npm package advertises `MIT OR Apache-2.0`; JSR metadata uses `MIT` because its publisher requires a single recognized license identifier. Both license files are included in the package.
The playground editor uses the `decodal-codemirror` package with the generated Lezer parser in `packages/decodal-codemirror/src/decodal-parser.js`.
The canonical grammar is documented in `doc/manual/souce/language/grammar.md`; regenerate the Lezer parser when that grammar or `editors/lezer-decodal/decodal.grammar` changes.
Publish the CodeMirror package from its package directory:
```sh
cd packages/decodal-codemirror
npm install
npm publish
npx jsr publish
```
Run dry-runs first when preparing a release:
```sh
npm install
npm pack --dry-run
npx jsr publish --dry-run
```
The npm package name is `decodal-codemirror`.
The JSR package name is `@hare/decodal-codemirror`.
As with `decodal-wasm`, JSR metadata uses `MIT` while both the MIT and Apache-2.0 license files remain included.
The documentation build still uses the lightweight JavaScript fallback highlighter so Astro can render Markdown without initializing WASM at build time.
To run the site locally:
```sh
cd site/decodal-site
npm run dev
```
Deploy the static site to Cloudflare Pages with Wrangler direct upload:
```sh
cd site/decodal-site
npm run deploy
```
The deploy script runs `npm run build` and then uploads `dist/` to the Pages project named `decodal-site` on branch `master`.
Cloudflare Pages treats this as production when the project production branch is `master`.
Use `CLOUDFLARE_PROJECT_NAME` when deploying to a differently named Pages project.
```sh
CLOUDFLARE_PROJECT_NAME=my-pages-project npm run deploy
```
If the committed WASM package must be regenerated before deploy, use:
```sh
npm run deploy:all
```
## Tree-sitter grammar
@@ -64,16 +207,22 @@ The generated parser files under `editors/tree-sitter-decodal/src/` are committe
When the Decodal syntax changes:
1. Update `grammar.js`.
2. Run `npx tree-sitter generate`.
3. Add or update corpus tests in `corpus/`.
4. Update highlight queries in `queries/highlights.scm` if token names changed.
5. Run `npx tree-sitter test`.
6. Run Rust checks from the repository root if the language parser or examples also changed.
1. Update the canonical EBNF in `doc/manual/souce/language/grammar.md`.
2. Update the Rust parser/lexer as needed.
3. Update `editors/tree-sitter-decodal/grammar.js` and run `npx tree-sitter generate` / `npx tree-sitter test`.
4. Update `editors/lezer-decodal/decodal.grammar` and regenerate the CodeMirror parser:
```sh
cd site/decodal-site
npx lezer-generator ../../editors/lezer-decodal/decodal.grammar -o ../../packages/decodal-codemirror/src/decodal-parser.js
```
5. Add or update corpus/tests/examples.
6. Run Rust and site checks from the repository root.
## Development shell
The Nix development shell includes Rust tooling, Node.js, and Tree-sitter CLI tooling.
The Nix development shell includes Rust tooling, Node.js, Tree-sitter CLI tooling, and wasm-pack tooling.
```sh
nix develop
@@ -87,5 +236,7 @@ The shell provides:
- `clippy`
- `node`
- `npm`
- `wasm-pack`
- `lld`
- `tree-sitter`
- `nixfmt`
+8 -6
View File
@@ -6,7 +6,8 @@ Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェク
## 目次
1. [Introduction](./introduction.md)
2. [Language Specification](./language/index.md)
2. [Components](./components.md)
3. [Language Specification](./language/index.md)
1. [Lexical Structure and Syntax](./language/syntax.md)
2. [Value](./language/value/index.md)
1. [String](./language/value/string.md)
@@ -25,8 +26,9 @@ Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェク
9. [Match](./language/expression/match.md)
10. [Import](./language/expression/import.md)
11. [Composition](./language/expression/composition.md)
12. [Default](./language/expression/default.md)
13. [String Interpolation](./language/expression/string-interpolation.md)
12. [Range Refinement](./language/expression/ascription.md)
13. [Default](./language/expression/default.md)
14. [String Interpolation](./language/expression/string-interpolation.md)
4. [Constraints and Defaults](./language/constraints-and-defaults.md)
5. [Composition Operators](./language/operators.md)
6. [Functions](./language/functions.md)
@@ -35,7 +37,7 @@ Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェク
9. [Materialization and Errors](./language/materialization-and-errors.md)
10. [Naming Conventions](./language/naming.md)
11. [Examples](./language/examples.md)
3. [Implementation Design](./design/index.md)
4. [Implementation Design](./design/index.md)
1. [Execution Pipeline](./design/execution-pipeline.md)
2. [Runtime Model](./design/runtime-model.md)
3. [Thunk and Lazy Evaluation](./design/thunk-and-lazy-evaluation.md)
@@ -43,5 +45,5 @@ Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェク
5. [Diagnostics and Fallback](./design/diagnostics-and-fallback.md)
6. [Embedding API](./design/embedding-api.md)
7. [Features](./design/features.md)
4. [Development](./development.md)
5. [Open Issues](./open-issues.md)
5. [Development](./development.md)
6. [Open Issues](./open-issues.md)
@@ -82,12 +82,27 @@ Int & >= 10 & <= 10 # OK
最小の組み込み制約は以下である。
```dcdl
Unknown
String
Int
Float
Bool
```
`Unknown` はすべてのDecodal値を含む最上位rangeである。検査を無効化する `Any` ではなく、具体的な値またはより狭いrangeがまだ決まっていないことを表す。
```dcdl
Int & Unknown # Int
42 as Unknown # 42
Unknown as Int # エラー
```
`Unknown` 自体は具体値を持たないためmaterializeできない。defaultを与えるか、具体値で絞り込む必要がある。
```dcdl
Unknown default {} # {}
```
追加の述語制約はライブラリまたは組み込みとして提供できる。
```dcdl
@@ -124,6 +139,43 @@ String & /^a$/ & /^b$/
Host = IPv4Address;
```
## 配列制約
配列制約には要素制約が必須であり、`[...T]` と書く。
```dcdl
Names = [...String];
PositiveInts = [...(Int & > 0)];
```
複数の配列制約を `&` で合成した場合、各 concrete 要素をすべての要素 range で絞り込む。
要素が object range の場合、右辺にしかない field は default の有無にかかわらず abstract のまま各要素へ残る。
左辺にしかない field は右辺の field domain 外なのでエラーになる。
要素制約のない `Array` primitive type は存在しない。
## 連想配列制約
連想配列制約は `{...T}` と書き、object の任意の field value を `T` に対して絞り込む。
```dcdl
Services = {...{
port = Int;
enabled = Bool default true;
}};
```
key は schema で列挙せず、空 object も許容する。
固定 object field と任意 key の value constraint は、末尾の `...T` で混在できる。
```dcdl
{
enabled = Bool default true;
...Unknown
}
```
このrest constraintは明示されていないfieldだけに適用され、field自体は生成しない。
## default
`default` は制約ではない。
@@ -188,11 +240,12 @@ RuntimeValue =
## default の合成
同じフィールドに複数の `default` 合成された場合の詳細規則は未確定である。
現時点の単純な方針は以下である。
`default` 合成規則は以下である。
- `&` による default 同士の衝突はエラーにする。
- 同一 default 値は許可してよい
- `//` による patch では右辺 default が左辺 default を置き換える。
- `&` で片方だけが default を持つ場合、その default を保持する。
- `&` で両方が異なる default を持つ場合、conflict になる
- `Abstract & Concrete` が成功した場合、結果は concrete value になり default は消える。
- `//` では右辺が左辺を置き換える。object/object の場合は field ごとに再帰 patch されるため、右辺 field の default が左辺 field の default を置き換える。
この方針により、`&` は制約を保った合成、`//` は上書き操作として説明できる。
`default` thunk は materialize 時に必要になった時点で評価する。
評価された default value は、同じ abstract value に残っている制約を満たす必要がある。
+1 -1
View File
@@ -74,5 +74,5 @@ Error 評価失敗
関数引数は thunk として渡せる。
関数本体内で引数が参照されたときに評価する。
任意の関数呼び出し結果グローバルに memoize することは必須ではない。
関数呼び出し結果そのものはグローバルに memoize ない。
ただし、フィールドに束縛された呼び出し結果は、そのフィールド thunk の評価結果として memoize される。
+39 -15
View File
@@ -5,7 +5,6 @@
## 基本的な設定スキーマ
```dcdl
rec {
Host = IPv4Address;
Port = Int & >= 1 & <= 65535;
@@ -19,7 +18,6 @@ rec {
fuga = Int default 10;
};
};
}
NewConfig = MyConfig & {
host = "127.0.0.1";
@@ -33,28 +31,54 @@ disabled_config = NewConfig & {
enabled_config = NewConfig;
```
## 関数と文字列生成
## 配列スキーマ
```dcdl
Services = [...{
name = String;
port = Int default 8080;
}];
[
{ name = "api"; },
{ name = "worker"; port = 9000; },
] as Services
```
抽象配列には要素制約が必須である。
この例では、1 番目の要素の `port``8080` に materialize される。
## 連想配列と範囲絞り込み
```dcdl
Service = {
port = Int;
enabled = Bool default true;
};
services = {
api = { port = 8080; };
worker = { port = 8081; enabled = false; };
} as {...Service};
```
`api``worker` は任意の key であり、それぞれの value は `Service` より狭い範囲へ絞り込まれる。
`as` の結果では `api.enabled` は abstract のまま残り、最終的に結果全体を materialize した時点で default の `true` が使われる。
## 関数と制約
```dcdl
let
maybe_hw = String & /Hello! .*/;
part = {
greet = String;
target = String;
};
mk_hw = (part: part) =>
maybe_hw & "${part.greet}! ${part.target}";
Port = Int & >= 1 & <= 65535;
add_offset = (base: Port, offset: Int) => base + offset;
in
mk_hw({
greet = "Hello";
target = "World";
})
add_offset(8000, 80)
```
評価結果:
```text
"Hello! World"
8080
```
## match
@@ -0,0 +1,43 @@
# Arithmetic Expression
Decodal supports arithmetic over concrete numeric values.
```dcdl
{
workers = 2 + 2;
timeout = 30.0 / 2;
port = 8000 + 80;
negative = -1;
}
```
## Operators
- `+` addition
- `-` subtraction
- `*` multiplication
- `/` division
- unary `-` negation
`*` and `/` bind tighter than `+` and `-`.
Parentheses can be used to make grouping explicit.
```dcdl
2 + 3 * 4 # 14
(2 + 3) * 4 # 20
```
## Numeric behavior
Arithmetic requires concrete `Int` or `Float` operands.
`Int + Int`, `Int - Int`, and `Int * Int` produce `Int` when no overflow occurs.
Mixed `Int` / `Float` arithmetic produces `Float`.
Division always produces `Float`.
Division by zero and integer overflow are evaluation errors.
Arithmetic expressions can be used anywhere a concrete numeric expression is expected, including defaults and numeric constraints.
```dcdl
port = Int & > 4000 + 42 default 8080;
```
+61 -5
View File
@@ -7,9 +7,65 @@ array expression は、順序付きの値の列を表す。
["a", "b", "c"]
```
## 未確定事項
配列リテラルは concrete value であり、要素型を揃える必要はない。
- 配列要素の制約表現。
- 異種配列を許可するか。
- `//` による patch を右辺置換だけにするか。
- append / prepend / remove などの操作を提供するか。
```dcdl
["api", 8080, true]
```
## Array constraint
配列制約は `[` の直後に ellipsis を置き、その後へ要素制約を記述する。
```dcdl
Names = [...String];
Ports = [...(Int & >= 1 & <= 65535)];
```
`[...T]` は、すべての要素が `T` を満たす長さ 0 以上の配列を表す。
空配列は任意の配列制約を満たす。
```dcdl
[...String] & []
[...String] & ["api", "worker"]
```
要素制約は object range にもできる。
右辺だけにある default 付き field は、配列へ制約を合成した時点では abstract のまま各要素に残る。
その配列を後から materialize した場合にだけ default が使われる。
```dcdl
Services = [...{
name = String;
enabled = Bool default true;
}];
Services & [{ name = "api"; }]
```
配列制約を concrete array に適用すると、各要素は要素 range に対して `as` と同じ規則で絞り込まれる。
object の要素 range では左辺にしかない field がエラーになり、右辺にしかない field は abstract のまま残る。
要素制約のない抽象配列型は提供しない。
旧来の `Array` primitive type は使用できず、`[...T]``T` は必須である。
`[String]` は配列制約ではなく、未解決の `String` 制約を 1 要素に持つ concrete array になる。
長さ制約、位置別の tuple 制約、unique 制約は現在サポートしない。
## Array concat
`++` は concrete array 同士を連結する。
```dcdl
base = ["read", "write"];
extra = ["admin"];
roles = base ++ extra;
```
`roles` は以下と同じ値になる。
```dcdl
["read", "write", "admin"]
```
`++` は配列要素を変換しない。
左辺の要素の後に右辺の要素が並ぶ。
@@ -0,0 +1,75 @@
# Range Refinement
`narrower as wider` は、左辺が右辺より具体的で狭い範囲であることを確認し、両者を非対称に合成する演算である。
```dcdl
Int & > 10 as Int & > 0
```
この例は成功し、結果は左辺の狭い範囲 `Int & > 10` のままになる。逆向きの `Int as Int & > 0` は、左辺が右辺を満たすとは限らないため失敗する。
## Object
object では、右辺を field domain として左辺の field を確認する。
- 左辺にしかない field は、右辺の domain 外なのでエラーになる。
- 両側にある field は、左辺が右辺より狭いかを再帰的に確認し、左辺の狭い結果を使う。
- 右辺にしかない field は、default の有無に関係なく未評価のまま結果へ残す。
- 結果の field order は右辺の順序に従う。
```dcdl
partial = {
port = 8080;
} as {
host = String;
port = Int;
enabled = Bool default true;
};
```
`partial.port``8080` に具体化される。`partial.host``partial.enabled` は右辺由来の abstract field のままであり、`as``enabled` の default を選択も force もしない。
その後 `partial` 全体を materialize すれば、通常の materialize 規則が適用される。この例では unresolved な `host` がエラーになり、`host` も具体化されていれば `enabled` の default がその時点で利用される。
## Default
default は範囲包含の根拠ではない。両側に同じ field がある場合、結果には左辺を使うため、右辺の default を左辺へ注入しない。
```dcdl
Int & > 0 as (Int default 1)
```
結果は default のない `Int & > 0` である。一方、右辺だけに残る object field は field 全体を保持するため、その abstract value が元から持つ default も保持される。
## Map and array ranges
`{...T}` は任意の object key を許可し、各 value が `T` より狭いことを確認する。
```dcdl
ports = {
http = 80;
https = 443;
} as {...(Int & >= 1 & <= 65535)};
```
`[...T]` を右辺に使う場合も、すべての array element を `T` に対して絞り込む。
## Concrete right-hand ranges
primitive constraint や合成 constraint は通常どおり値を検証する。
右辺が concrete scalar または array literal の場合は、左辺も同じ値または同じ長さ・要素構造である必要がある。
function は右辺の範囲として使用できない。
左辺は concrete value に限らない。処理系が包含を確認できる constraint 同士であれば abstract value も使用できる。primitive type、numeric bound、同一 regex / predicate、array/map の要素範囲は包含確認の対象になる。
関数 parameter の `name: range` も、引数が force された時点で `as` と同じ絞り込み規則を使う。
## Precedence
`as``default` より低い、最も低い優先順位を持ち、左結合である。
```dcdl
narrow & overrides as Wider
```
これは `(narrow & overrides) as Wider` と解釈される。
@@ -11,6 +11,9 @@ Port = Int & >= 1 & <= 65535;
Config = MyConfig & { port = 8000; };
```
object 同士では片側だけにある field も保持するため、`&` は方向を持たない。
左辺が右辺より狭いことを確認し、右辺を field domain として合成したい場合は [`as`](./ascription.md) を使う。
## `//`
`//` は右辺優先の構造的 patch を行う。
@@ -3,15 +3,15 @@
function call expression は、関数値を引数に適用する式である。
```dcdl
mk_hw({
greet = "Hello";
target = "World";
})
increment(41)
```
## 評価
引数は thunk として渡せる。
関数本体内で引数が参照されたときに評価する。
parameter に range が指定されている場合、引数は `narrower as wider` と同じ規則で絞り込まれる。
parameter 側だけにある field は abstract のまま残り、default はこの時点では選択されない。
任意の関数呼び出し結果グローバルに memoize することは必須ではない。
関数呼び出し結果そのものはグローバルに memoize ない。
フィールドに束縛された呼び出し結果は、そのフィールド thunk の評価結果として memoize される。
@@ -3,11 +3,7 @@
function expression は、引数を受け取り式を返す値である。
```dcdl
(part: {
greet = String;
target = String;
}) =>
"${part.greet}! ${part.target}"
(value: Int) => value + 1
```
関数仕様の詳細は [関数](../functions.md) に置く。
@@ -10,13 +10,16 @@ Expr
├─ identifier
├─ path reference
├─ object
├─ map constraint
├─ array
├─ array constraint
├─ function
├─ function call
├─ let
├─ match
├─ import
├─ composition
├─ range refinement (`as`)
├─ default
└─ string interpolation
```
+3 -5
View File
@@ -4,12 +4,10 @@ let expression は、ローカル束縛を作る。
```dcdl
let
part = {
greet = "Hello";
target = "World";
};
base = 8000;
offset = 80;
in
"${part.greet}! ${part.target}"
base + offset
```
## 評価
@@ -0,0 +1,41 @@
# Logical and Comparison Expressions
Decodal supports boolean logic over concrete `Bool` values and comparison over concrete scalar values.
```dcdl
{
is_prod = env == "prod";
high_port = port > 9000;
enabled = is_prod && high_port;
disabled = !enabled;
}
```
## Logical operators
- `!expr` negates a concrete `Bool`.
- `lhs && rhs` returns boolean AND.
- `lhs || rhs` returns boolean OR.
`&&` and `||` short-circuit: the right-hand side is evaluated only when needed.
Logical operands must evaluate to concrete `Bool` values.
## Comparison operators
- `==`
- `!=`
- `<`
- `<=`
- `>`
- `>=`
`==` and `!=` compare concrete scalar values: `String`, `Bool`, `Int`, and `Float`.
`Int` and `Float` can be compared to each other numerically.
Ordering operators `<`, `<=`, `>`, and `>=` compare concrete numeric values only.
They are separate from prefix comparison constraints such as `> 443`.
```dcdl
port = Int & > 443 default 9443;
is_high = port > 9000;
```
@@ -37,3 +37,52 @@ MyConfig = {
};
}
```
## Map constraint
連想配列の制約は、object の任意の field value に同じ schema を適用する。
```dcdl
Services = {...{
port = Int;
enabled = Bool default true;
}};
```
`{...T}` は key の集合を固定せず、すべての value が `T` を満たす object を表す。
空 object も許容される。runtime と materialize 後の表現は通常の object と共通であり、別の map value variant は持たない。
```dcdl
services = {
api = { port = 8080; };
worker = { port = 8081; enabled = false; };
} as Services;
```
各 entry は `as` によって右辺の field domain 内へ絞り込まれるため、左辺に `port``enabled` 以外の field があればエラーになる。
右辺にしかない field は default の有無にかかわらず abstract のまま残る。
host から渡した object は識別子構文に収まらない文字列 key も保持できるが、DCDL source の object field name は通常の識別子に限られる。
## Object rest constraint
固定 field と任意 key の value range は、一つの object に混在できる。
```dcdl
OpenConfig = {
enabled = Bool default true;
...Unknown
};
```
`...T` は明示されていない残余 field にだけ適用する。明示 field にはそれぞれの field range を適用し、rest range を重ねて適用しない。
rest constraint は field を生成せず、materialize 時には実際に存在する追加 field だけを出力する。
```dcdl
{
enabled = false;
plugin = { name = "cache"; };
} as OpenConfig
```
`...T` は object の末尾に一つだけ書ける。省略した object は閉じており、`as` の左辺に未宣言 field があればエラーになる。
名前付き field を持たない `{...T}` は従来どおり抽象的な map constraint であり、単独でmaterializeするにはdefaultまたは具体値が必要になる。
@@ -12,4 +12,5 @@ config.feature_hoge.enable
左側の式を object として評価し、指定された field を参照する。
参照先 field は必要になるまで評価されない。
存在しない field への参照をエラーにするか、open schema として扱うかは未確定である。
存在しない field への参照は diagnostic になる。
明示的な `Unknown` range は存在しないfieldを表さないため、fieldの有無を曖昧にはしない。
@@ -1,12 +1,8 @@
# String Interpolation Expression
# String Interpolation
string interpolation は、文字列内に式を埋め込む候補機能である
文字列補間は初期実装には含めない
```dcdl
"${part.greet}! ${part.target}"
```
Decodal の string literal は、現時点では literal text として扱う。
式を埋め込む構文は定義しない。
## 評価
補間式の評価タイミングは通常の遅延評価に従う。
文字列補間を初期実装に含めるかは未確定である。
必要になった場合は、文字列連結や明示的な formatting function として別途設計する。
+20 -52
View File
@@ -5,54 +5,48 @@
## 構文
```dcdl
(part: {
greet = String;
target = String;
}) =>
"${part.greet}! ${part.target}"
(value: Int) => value + 1
```
関数呼び出しは通常の呼び出し構文で行う。
```dcdl
mk_hw({
greet = "Hello";
target = "World";
})
let
increment = (value: Int) => value + 1;
in
increment(41)
```
複数引数の構文は候補として以下を想定する。
複数引数も指定できる。
```dcdl
(
input_a: {
hoge = Int & >= 0;
},
input_b: {
fuga = 20;
}
) =>
(input_a: { hoge = Int & >= 0; }, input_b: { fuga = Int; }) =>
{
# ...
hoge = input_a.hoge;
fuga = input_b.fuga;
}
```
## 関数の意味
関数は値として扱える
ただし、最終データとして関数値を出力できるかどうかは別途定める。
設定ファイルを materialize する段階では、未適用の関数値は出力不能な値として扱うのが自然である。
関数は runtime value として扱えるが、最終データとして materialize することはできない
未適用の関数値が materialize 対象に残っている場合は diagnostic になる。
関数値は opaque であり、関数値同士の等価性は提供しない。
`&` で関数値同士を合成すると conflict になる。
## 評価方針
関数は以下の方針を基本とする。
関数は以下の方針で評価する。
- 関数は純粋である。
- 関数はレキシカルスコープを持つ。
- 関数は定義時の環境を参照として保持する。
- 引数は必要になるまで評価ない。
- フィールドに束縛された関数呼び出し結果は、そのフィールド評価結果として memoize される。
- 任意の関数呼び出しそのものグローバルに memoize することは必須ではない。
- 引数は thunk として渡され、必要になるまで評価されない。
- parameter range がある引数は、force 時に `as` と同じ範囲包含・絞り込み規則で合成される。
- 関数呼び出し結果そのものグローバルに memoize ない。
- フィールドに束縛された関数呼び出し結果は、そのフィールド thunk の評価結果として memoize される。
- 再帰的な依存は thunk cycle として diagnostic になる。
関数値の内部モデル例:
@@ -63,29 +57,3 @@ Function {
env: EnvRef
}
```
## 関数と重さ
関数の文法・パーサー自体は大きくない。
実装上の重さは、主に評価モデルと意味論から発生する。
注意点:
- クロージャが環境を掴む。
- 引数を lazy にするか strict にするかを決める必要がある。
- 関数呼び出し結果をどこまで memoize するかを決める必要がある。
- 再帰関数を許可するかを決める必要がある。
- 関数値同士の `&` をどう扱うかを決める必要がある。
- 関数値を object に入れたとき、materialize 可能かを決める必要がある。
- import 循環と関数適用が絡んだときの cycle detection が必要になる。
軽量に保つため、初期仕様では以下の制限を検討できる。
- 関数は pure。
- lexical closure は許可。
- 引数は thunk として渡す。
- 関数本体は必要になるまで評価しない。
- 関数値は opaque。
- 関数同士の `&` は、同一関数参照以外は conflict。
- 関数値は最終データとして出力できない。
- 再帰は cycle error として扱う、または v1 では禁止する。
+119
View File
@@ -0,0 +1,119 @@
# Grammar
This page is the canonical grammar reference for Decodal syntax.
Parser implementations such as the Rust parser, Tree-sitter grammar, and Lezer grammar should follow this grammar and may add implementation-specific precedence annotations where needed.
## Lexical grammar
```ebnf
source_character = ? any Unicode scalar value ? ;
newline = "\n" | "\r\n" | "\r" ;
space = " " | "\t" | newline ;
comment = "#" , { ? any character except newline ? } ;
digit = "0" … "9" ;
letter = "A" … "Z" | "a" … "z" ;
identifier = letter , { letter | digit | "_" } ;
integer = digit , { digit } ;
float = digit , { digit } , "." , digit , { digit } ;
string = '"' , { string_character | escape } , '"' ;
string_character = ? any character except '"', "\\", or newline ? ;
escape = "\\" , source_character ;
regex = "/" , regex_character , { regex_character } , "/" ;
regex_character = escape | ? any character except "/", "\\", or newline ? ;
```
Whitespace and comments separate tokens and are otherwise ignored by the parser.
## Syntactic grammar
```ebnf
module = { statement } ;
statement = field_definition , [ ";" ]
| expression , [ ";" ] ;
expression = as_expression ;
as_expression = default_expression , { "as" , default_expression } ;
default_expression = patch_expression , [ "default" , default_expression ] ;
patch_expression = compose_expression , { "//" , compose_expression } ;
compose_expression = logical_or_expression , { "&" , logical_or_expression } ;
logical_or_expression = logical_and_expression , { "||" , logical_and_expression } ;
logical_and_expression = comparison_expression , { "&&" , comparison_expression } ;
comparison_expression = concat_expression , [ comparison_operator , concat_expression ] ;
concat_expression = additive_expression , { "++" , additive_expression } ;
additive_expression = multiplicative_expression , { ( "+" | "-" ) , multiplicative_expression } ;
multiplicative_expression = unary_expression , { ( "*" | "/" ) , unary_expression } ;
unary_expression = [ "!" | "-" ] , postfix_expression ;
postfix_expression = primary_expression , { call_suffix | path_suffix } ;
call_suffix = "(" , [ argument_list ] , ")" ;
path_suffix = "." , identifier ;
primary_expression = literal
| identifier
| comparison_constraint
| map_constraint
| object
| array_constraint
| array
| let_expression
| function_expression
| match_expression
| import_expression
| "(" , expression , ")" ;
literal = string | integer | float | "true" | "false" | regex ;
comparison_operator = "==" | "!=" | "<" | "<=" | ">" | ">=" ;
comparison_constraint = ( "<" | "<=" | ">" | ">=" ) , expression ;
object = "{" , [ field_definition , { ";" , field_definition }
, [ ";" , object_rest ] , [ ";" ] ] , "}" ;
object_rest = "..." , expression ;
map_constraint = "{" , "..." , expression , "}" ;
field_definition = field_path , "=" , expression ;
field_path = identifier , { "." , identifier } ;
array = "[" , [ expression , { "," , expression } , [ "," ] ] , "]" ;
array_constraint = "[" , "..." , expression , [ "," ] , "]" ;
let_expression = "let" , { field_definition , ";" } , "in" , expression ;
function_expression = "(" , [ parameter_list ] , ")" , "=>" , expression ;
parameter_list = parameter , { "," , parameter } , [ "," ] ;
parameter = identifier , [ ":" , expression ] ;
match_expression = "match" , expression , "{" , [ match_arm , { ";" , match_arm } , [ ";" ] ] , "}" ;
match_arm = pattern , ":" , expression ;
pattern = "_" | expression ;
import_expression = "import" , string ;
argument_list = expression , { "," , expression } , [ "," ] ;
```
## Precedence
Precedence is highest first.
1. function call and field path reference
2. unary `!` and `-`
3. `*` and `/`
4. `+` and `-`
5. `++`
6. `==`, `!=`, `<`, `<=`, `>`, `>=`
7. `&&`
8. `||`
9. `&`
10. `//`
11. `default`
12. `as`
Binary operators are left-associative except `default`, which is right-associative.
## Tooling mapping
Syntax tooling should derive token categories from this grammar rather than making a tool-specific grammar canonical.
Tree-sitter and Lezer grammars are implementation artifacts that follow this page.
@@ -95,4 +95,5 @@ match value {
通常の式はエラー内容に基づいて分岐できない。
汎用 `try / catch` は core には含めない。
fallback は `default``match`、および将来的な optional import / optional field access のような限定された仕組みで表現する。
fallback は `default``match` で表現する。
optional import や optional field access は core には含めない。
@@ -29,17 +29,80 @@ result = schema;
```
この場合、`result` の右辺の `schema` は同じ module の top-level field `schema` を参照する。
通常の object literal 内の field を暗黙に recursive scope にするかは別仕様とする
通常の object literal 内の field は、その object 内の sibling field を暗黙には識別子として参照できない
object 内の値を参照する場合は、外側で束縛された値や明示的な path reference を使う。
## SourceLoader
## ImportLoader
`import` specifier の解決は処理系 core ではなく host 側の `SourceLoader` が行う。
`import` specifier の解決は処理系 core ではなく host 側の `ImportLoader` が行う。
CLI では、specifier を現在の module path からの相対 path として解決する。
組み込み利用では、resource table や static source map など、filesystem 以外の loader を使える。
module cache の key は loader が返す安定 key を使う。
CLI では canonical path を key とする。
### 構造化 import
`ImportLoader::load` は DCDL source または host が構築した `Value` を import 結果として返す。
Markdown、JSON、TOML などの解釈規則は core に固定せず、loader がファイル種別を判定して構造化する。
```rust
use decodal::{Value, ImportLoader, LoadedImport};
struct ContentLoader;
impl ImportLoader for ContentLoader {
fn load(
&mut self,
current_key: Option<&str>,
specifier: &str,
) -> decodal::Result<LoadedImport> {
if specifier.ends_with(".md") {
let markdown = read_content(current_key, specifier)?;
let parsed = parse_frontmatter(&markdown)?;
return Ok(LoadedImport::value(
parsed.key,
Value::object([
("frontmatter", parsed.frontmatter),
("body", Value::string(parsed.body)),
]),
));
}
let source = read_dcdl(current_key, specifier)?;
Ok(LoadedImport::source(
source.key,
specifier,
source.text,
))
}
}
```
`read_content``parse_frontmatter` は host 独自の処理であり、Decodal core は Markdown や YAML parser に依存しない。
上の loader を使うと、DCDL 側から次のように扱える。
```dcdl
Post = {
frontmatter = {
title = String;
draft = Bool default false;
};
body = String;
};
post = Post & import "./hello.md";
title = post.frontmatter.title;
body = post.body;
```
`LoadedImport::Value` は通常の concrete runtime value に internalize される。
そのため、path reference、object composition、constraint validation、materialize は source 由来の値と同じ規則を使う。
安定した `key` が同じ構造化 import は、engine 内で同じ値としてキャッシュされる。
`load` が唯一の import hook である。
loader は拡張子、media type、または host 独自の規則で振り分け、対応する `LoadedImport` variant を直接返す。
## 循環 import
モジュール間に循環参照があっても、必要なフィールドの依存関係が循環していなければ評価できる。
@@ -92,4 +155,5 @@ Module func
- ファイルが読めない。
- import 先の構文解析に失敗する。
- import 先の評価で必要な値がエラーになる。
- host による非 DCDL content の読み込みまたは構造化に失敗する。
- 実装が禁止する import 循環に該当する。
+117 -12
View File
@@ -1,6 +1,81 @@
# 合成演算子
# 演算子
この章では、`&``//` の意味を定義する。
この章では、Decodal の演算子の意味を定義する。
## 演算子一覧
| 演算子 | 形 | 種類 | 対象 | 結果 / 意味 |
|---|---|---|---|---|
| `.` | `object.field` | field reference | object / abstract object | field value |
| call | `fn(arg)` | function call | function | function result |
| `!` | `!expr` | unary logical | concrete `Bool` | concrete `Bool` |
| `-` | `-expr` | unary arithmetic | concrete `Int` / `Float` | negated number |
| `*` | `lhs * rhs` | arithmetic | concrete `Int` / `Float` | numeric product |
| `/` | `lhs / rhs` | arithmetic | concrete `Int` / `Float` | `Float` quotient |
| `+` | `lhs + rhs` | arithmetic | concrete `Int` / `Float` | numeric sum |
| `-` | `lhs - rhs` | arithmetic | concrete `Int` / `Float` | numeric difference |
| `++` | `lhs ++ rhs` | array concat | concrete arrays | concatenated array |
| `==` | `lhs == rhs` | equality | concrete scalar | concrete `Bool` |
| `!=` | `lhs != rhs` | equality | concrete scalar | concrete `Bool` |
| `<` | `lhs < rhs` | ordering | concrete `Int` / `Float` | concrete `Bool` |
| `<=` | `lhs <= rhs` | ordering | concrete `Int` / `Float` | concrete `Bool` |
| `>` | `lhs > rhs` | ordering | concrete `Int` / `Float` | concrete `Bool` |
| `>=` | `lhs >= rhs` | ordering | concrete `Int` / `Float` | concrete `Bool` |
| `>` | `> value` | comparison constraint | numeric constraint value | abstract constraint |
| `>=` | `>= value` | comparison constraint | numeric constraint value | abstract constraint |
| `<` | `< value` | comparison constraint | numeric constraint value | abstract constraint |
| `<=` | `<= value` | comparison constraint | numeric constraint value | abstract constraint |
| `&&` | `lhs && rhs` | logical | concrete `Bool` | short-circuit AND |
| `||` | `lhs || rhs` | logical | concrete `Bool` | short-circuit OR |
| `&` | `lhs & rhs` | composition | value / constraint / object | constraint-preserving composition |
| `//` | `lhs // rhs` | patch | object / value | right-biased structural patch |
| `default` | `base default fallback` | default | abstract value | materialization fallback |
| `as` | `narrower as wider` | range refinement | value / constraint / structure | narrower result plus untouched right-only ranges |
`concrete scalar``String``Bool``Int``Float` を指す。
## 優先順位
優先順位は高い順に以下である。
1. 関数呼び出しとフィールド参照
2. unary `!` `-`
3. `*` `/`
4. `+` `-`
5. `++`
6. `==` `!=` `<` `<=` `>` `>=`
7. `&&`
8. `||`
9. `&`
10. `//`
11. `default`
12. `as`
同じ優先順位の二項演算子は左結合である。
`default` は右結合である。
## Arithmetic operators
`+` `-` `*` `/` は具体的な `Int` / `Float` に対する四則演算である。
詳しくは [Arithmetic Expression](./expression/arithmetic.md) を参照する。
## Array concat operator
`++` は concrete array 同士を連結する演算子である。
要素は変換されず、左辺の要素の後に右辺の要素が並ぶ。
```dcdl
["read", "write"] ++ ["admin"]
```
## Logical and comparison operators
`!` `&&` `||` は concrete `Bool` に対する論理演算である。
`&&``||` は短絡評価される。
`==` `!=` は concrete scalar value を比較する。
`<` `<=` `>` `>=` は concrete numeric value を比較する。
詳しくは [Logical and Comparison Expressions](./expression/logical-and-comparison.md) を参照する。
## `&`: 制約合成
@@ -18,6 +93,7 @@ A & B
- 両方が異なる具体値なら conflict になる。
- 両方が object なら、フィールドごとに合成する。
- 同じフィールドが両方にある場合、そのフィールド値を `&` で合成する。
- 片方にしかないフィールドは、そのまま結果に保持する。
- 矛盾が発生した場合はエラーになる。
例:
@@ -110,26 +186,49 @@ Patched = Base // {
## object 全体の置換
`//` が deep patch である場合、object 全体を置き換えたいときの escape hatch が必要になる。
`//` では object/object は常に deep patch される。
object field 全体を特別に置き換えるための `replace(...)` 構文や組み込み関数は core には含めない。
候補として、`replace(...)` を組み込み関数として提供する
object 全体を別構造にしたい場合は、patch 対象より外側で値を作り直す
## `as`: range refinement
`as` は左辺が右辺より具体的で狭い範囲であることを確認する、非対称な絞り込み演算子である。
```dcdl
Replaced = Base // {
feature_hoge = replace({
enable = false;
});
Refined = {
port = 8000;
} as {
port = Int & >= 1 & <= 65535;
host = String;
enabled = Bool default true;
};
```
この場合、`feature_hoge` は再帰 patch されず、右辺の object に丸ごと置き換わる。
`port` は左辺の `8000` に具体化される。
右辺にしかない `host``enabled` は abstract のまま結果へ残る。default の有無は、右辺だけの field を保持するかどうかに影響しない。
`as` 自体は default を選択せず、後から結果を materialize した場合だけ通常の default 規則が働く。
## `&``//` の使い分け
左辺にしかない object field は右辺の field domain 外なのでエラーになる。
両側にある nested object、array constraint、map constraint は同じ規則で再帰的に絞り込む。
`&` は制約を満たす具体化に使う
abstract range 同士も包含を確認できる
```dcdl
ValidConfig = MyConfig & {
Int & > 10 as Int & > 0 # 成功し、Int & > 10 を返す
Int as Int & > 0 # 失敗
```
`&` は対称な合成であり、片方だけにある object field を保持する。
したがって、左右に方向を持つ絞り込みを `&` で代用しない。
詳細は [Range Refinement](./expression/ascription.md) を参照する。
## `&``//``as` の使い分け
`&` は制約や部分構造を失わず、対称に合成する。
```dcdl
Combined = MyConfig & {
port = 8000;
};
```
@@ -144,3 +243,9 @@ ModifiedSchema = MyConfig // {
`//` は右辺優先の patch であり、左辺の制約を常に保持するとは限らない。
制約を保持したい場合は `&` を使う。
左辺が右辺より狭いことを検証しながら合成する場合は `as` を使う。
```dcdl
RefinedConfig = NarrowConfig as WideConfig;
```
+11 -8
View File
@@ -1,7 +1,7 @@
# 構文と字句
この章では、表層構文の方針をまとめる。
厳密な EBNF は未確定であり、今後このファイルに詳細化する。
厳密な構文は [Grammar](./grammar.md) の EBNF を参照する。
## 言語名と拡張子
@@ -93,9 +93,9 @@ config.feature_hoge.enable
}
```
## 予約語候補
## 予約語
以下は予約語または予約構文として扱う候補である
以下は予約語として扱う
```text
let
@@ -103,24 +103,27 @@ in
match
import
default
as
true
false
rec
```
`rec` の扱いは未確定である。
## 演算子
主要な演算子は以下である。
```text
+ - * / 四則演算
++ 配列結合
! && || 論理演算
== != < <= > >= 比較式
& 制約合成
// patch 合成
default fallback 指定
as 左辺から右辺への範囲包含確認と絞り込み
=> 関数
. フィールド参照 / ドットパス定義
... 配列または連想配列の値制約
```
演算子の優先順位は未確定である。
詳細は [合成演算子](./operators.md) で定義する。
演算子の優先順位は [合成演算子](./operators.md) で定義する。
+6 -3
View File
@@ -9,7 +9,10 @@ ratio = Float;
threshold = Float default 0.5;
```
## 未確定事項
## Int との関係
`Int``Float` の暗黙変換を許可するかは未確定である。
軽量実装では、両者を明確に分ける方が単純である。
`Int``Float` は primitive type constraint としては別の型である。
`Float` constraint は concrete `Float` を要求し、`Int` constraint は concrete `Int` を要求する。
数値演算や比較式では `Int``Float` を同じ numeric value として扱える。
混在した四則演算の結果は `Float` になる。
+3
View File
@@ -9,9 +9,12 @@ name = String;
retry = Int default 3;
ratio = Float;
enable = Bool default true;
tags = [...String];
```
現在の primitive type は `String``Int``Float``Bool` である。
`Unknown` はprimitive typeではなく、すべてのDecodal値を含む最上位の抽象rangeである。具体値またはdefaultがない `Unknown` はmaterializeできない。
配列は primitive type ではなく、必須の要素制約を持つ `[...T]` で表現する。
各型の個別仕様へのリンクは [Manual Index](../../index.md) に集約する。
primitive type と制約合成の詳細は [制約と default](../constraints-and-defaults.md) も参照する。
+2 -1
View File
@@ -17,4 +17,5 @@ greeting = String default "hello";
message = String & /Hello! .*/;
```
正規表現制約を必須機能にするかは未確定である。
正規表現制約の検証は Rust crate の `regex` feature で有効化される。
feature が無効な場合、正規表現制約は具体 string に対して検証できず diagnostic になる。
+14 -51
View File
@@ -1,60 +1,23 @@
# 未確定事項
今後決める必要がある事項を管理する。
詳細化するときは、各項目を該当する仕様ファイルへ移動または反映する
実装または仕様方針が固まった項目は、該当する仕様ファイルへ反映してここから外す
## 構文
## Materialize target and host projection
- 正式な字句・構文仕様
- 演算子の優先順位
- `rec` の扱い。
- コメント構文を `#` のみにするか。
Decodal の評価結果は、制約・default・関数値を含む中間値になり得る
そのため、Decodal 単独で常に「最終成果物」を一意に決めるのではなく、host 側が期待する型や出力形式を与えて materialize / decode する経路を明確にする必要がある
## 型・制約
決めること:
- 配列要素の制約表現
- object の open/closed schema の扱い
- 正規表現を必須機能にするか optional feature にするか。
- 代表的な組み込み述語の範囲
- Rust API で評価結果の field/path を選択して decode / materialize できるようにするか
- `decodal-derive` の struct schema と評価結果を合成して decode する経路を、主要な materialize path として位置づけるか
- CLI / WASM では target path を指定して JSON-compatible value へ materialize する形にするか。
- 制約や関数値が残った値を出力したい場合、materialize ではなく inspect/debug API として分けるか
## default
現時点の案:
- `default` 同士の conflict 解決規則
- `&` による default 合成の厳密な規則
- `//` による default 置換の厳密な規則
- default thunk の評価失敗をどの段階で報告するか
## 演算子
- `//` による制約・default の置換詳細。
- `replace(...)` を採用するか、別構文を設けるか。
- 配列に対する patch 操作を右辺置換だけにするか。
- 配列 append / prepend / remove などを提供するか。
## 関数
- 関数値の最終出力可否。
- 再帰関数を許可するか。
- 関数同士の `&` の扱い。
- 関数値の等価性。
- 関数呼び出し結果の memoize 範囲。
## 評価
- thunk のエラー memoize 方針。
- import cache の単位。
- 循環 import の診断メッセージ。
- materialize 対象の範囲指定方法。
## match
- match の網羅性チェックを行うか。
- 到達不能分岐を警告するか。
- パターン構文の範囲。
## エラー処理
- optional import を導入するか。
- optional field access を導入するか。
- optional fallback が捕捉できる失敗の範囲。
- エラー報告に制約由来の説明をどこまで含めるか。
- Rust では `evaluate -> select field/path -> expected schema と合成 -> decode` を主経路にする
- CLI / WASM では、明示された target path または module 全体を JSON-compatible value として materialize する
- materialize できない unresolved abstract value、default のない制約値、関数値は diagnostic にする
- Decodal 言語内には materialize 構文を追加せず、host API / CLI / WASM の責務として扱う
+18
View File
@@ -0,0 +1,18 @@
# lezer-decodal
Lezer implementation of the Decodal grammar for CodeMirror 6.
The canonical grammar is documented in:
```text
../../doc/manual/souce/language/grammar.md
```
Regenerate the CodeMirror parser from the site directory:
```sh
cd ../../site/decodal-site
npx lezer-generator ../../editors/lezer-decodal/decodal.grammar -o ../../packages/decodal-codemirror/src/decodal-parser.js
```
The generated parser is committed with the `decodal-codemirror` package because both the playground and external Web editors consume it.
+190
View File
@@ -0,0 +1,190 @@
@top Source { Statement* }
Statement {
FieldDefinition Semicolon? |
Expression Semicolon?
}
Expression { AsExpression }
AsExpression {
DefaultExpression |
AsExpression !as As DefaultExpression
}
DefaultExpression {
PatchExpression |
PatchExpression !default Default DefaultExpression
}
PatchExpression {
ComposeExpression |
PatchExpression !patch SlashSlash ComposeExpression
}
ComposeExpression {
LogicalOrExpression |
ComposeExpression !compose Amp LogicalOrExpression
}
LogicalOrExpression {
LogicalAndExpression |
LogicalOrExpression !or PipePipe LogicalAndExpression
}
LogicalAndExpression {
ComparisonExpression |
LogicalAndExpression !and AmpAmp ComparisonExpression
}
ComparisonExpression {
ConcatExpression |
ConcatExpression !compare CompareOperator ConcatExpression
}
ConcatExpression {
AdditiveExpression |
ConcatExpression !concat PlusPlus AdditiveExpression
}
AdditiveExpression {
MultiplicativeExpression |
AdditiveExpression !add (Plus | Minus) MultiplicativeExpression
}
MultiplicativeExpression {
UnaryExpression |
MultiplicativeExpression !multiply (Star | Slash) UnaryExpression
}
UnaryExpression {
PostfixExpression |
(Bang | Minus) !unary UnaryExpression
}
PostfixExpression {
PrimaryExpression |
PostfixExpression !call CallSuffix |
PostfixExpression !path Dot Identifier
}
CallSuffix { LParen ArgumentList? RParen }
ArgumentList { Expression (Comma Expression)* Comma? }
PrimaryExpression {
Literal |
Identifier |
ComparisonConstraint |
MapConstraint |
Object |
ArrayConstraint |
Array |
LetExpression |
FunctionExpression |
MatchExpression |
ImportExpression |
LParen Expression RParen
}
Literal { String | Integer | Float | True | False | Regex }
CompareOperator { EqualEqual | BangEqual | Lt | Lte | Gt | Gte }
ComparisonConstraint { (Lt | Lte | Gt | Gte) Expression }
Object {
LBrace
(FieldDefinition (Semicolon FieldDefinition)* (Semicolon ObjectRest)? Semicolon?)?
RBrace
}
ObjectRest { Ellipsis Expression }
MapConstraint { LBrace Ellipsis Expression RBrace }
FieldDefinition { FieldPath Equal Expression }
FieldPath { Identifier !fieldPath (Dot Identifier)* }
Array { LBracket (Expression (Comma Expression)* Comma?)? RBracket }
ArrayConstraint { LBracket Ellipsis Expression Comma? RBracket }
LetExpression { Let FieldDefinitionList In Expression }
FieldDefinitionList { (FieldDefinition Semicolon)* }
FunctionExpression { LParen ParameterList? RParen Arrow Expression }
ParameterList { Parameter (Comma Parameter)* Comma? }
Parameter { Identifier Colon Expression }
MatchExpression { Match Expression LBrace (MatchArm (Semicolon MatchArm)* Semicolon?)? RBrace }
MatchArm { Pattern Colon Expression }
Pattern { Underscore | Expression }
ImportExpression { Import String }
As { @specialize<Identifier, "as"> }
@precedence {
fieldPath @left,
as @left,
default @right,
patch @left,
compose @left,
or @left,
and @left,
compare @left,
concat @left,
add @left,
multiply @left,
unary,
call @left,
path @left
}
@tokens {
Let { "let" }
In { "in" }
Match { "match" }
Import { "import" }
Default { "default" }
True { "true" }
False { "false" }
Identifier { $[A-Za-z_] $[A-Za-z0-9_]* }
Integer { $[0-9]+ }
Float { $[0-9]+ "." $[0-9]+ }
String { '"' (!["\\\n] | "\\" _)* '"' }
Regex { "/" (![/\\\n] | "\\" _)+ "/" }
Comment { "#" ![\n]* }
LBrace { "{" }
RBrace { "}" }
LBracket { "[" }
RBracket { "]" }
LParen { "(" }
RParen { ")" }
Semicolon { ";" }
Comma { "," }
Dot { "." }
Ellipsis { "..." }
Colon { ":" }
Equal { "=" }
EqualEqual { "==" }
Bang { "!" }
BangEqual { "!=" }
Arrow { "=>" }
Amp { "&" }
AmpAmp { "&&" }
PipePipe { "||" }
Plus { "+" }
PlusPlus { "++" }
Minus { "-" }
Star { "*" }
Slash { "/" }
SlashSlash { "//" }
Gt { ">" }
Gte { ">=" }
Lt { "<" }
Lte { "<=" }
Underscore { "_" }
whitespace { @whitespace+ }
@precedence { Let, In, Match, Import, Default, True, False, Float, Integer, Underscore, Identifier }
}
@skip { whitespace | Comment }
+2 -2
View File
@@ -6,7 +6,7 @@ license = "MIT"
readme = "README.md"
keywords = ["incremental", "parsing", "tree-sitter", "decodal"]
categories = ["parsing", "text-editors"]
repository = "https://github.com/tree-sitter/tree-sitter-decodal"
repository = "https://gitea.hareworks.net/Hare/Decodal"
edition = "2021"
autoexamples = false
@@ -17,7 +17,7 @@ include = ["bindings/rust/*", "grammar.js", "queries/*", "src/*"]
path = "bindings/rust/lib.rs"
[dependencies]
tree-sitter = ">=0.22.6"
tree-sitter = "0.22.6"
[build-dependencies]
cc = "1.0.87"
+12
View File
@@ -0,0 +1,12 @@
# tree-sitter-decodal
Tree-sitter implementation of the Decodal grammar.
The canonical grammar is documented in:
```text
../../doc/manual/souce/language/grammar.md
```
This grammar is an editor/tooling implementation of that EBNF reference.
Generated parser files under `src/` are committed.
@@ -74,3 +74,165 @@ base // {
body: (literal (string)))
(match_arm
body: (literal (string))))))))
==================
Arithmetic
==================
{
value = 1 + 2 * 3;
grouped = (1 + 2) / -3;
}
---
(source_file
(object
(field_definition
path: (field_path (identifier))
value: (binary_expression
left: (literal (integer))
right: (binary_expression
left: (literal (integer))
right: (literal (integer)))))
(field_definition
path: (field_path (identifier))
value: (binary_expression
left: (parenthesized_expression
(binary_expression
left: (literal (integer))
right: (literal (integer))))
right: (unary_expression
operand: (literal (integer)))))))
==================
Logical and comparison
==================
{
enabled = env == "prod" && replicas > 1;
disabled = !enabled || false;
}
---
(source_file
(object
(field_definition
path: (field_path (identifier))
value: (binary_expression
left: (binary_expression
left: (identifier)
right: (literal (string)))
right: (binary_expression
left: (identifier)
right: (literal (integer)))))
(field_definition
path: (field_path (identifier))
value: (binary_expression
left: (unary_expression
operand: (identifier))
right: (literal (boolean))))))
==================
Array concat
==================
{
roles = ["read"] ++ ["write", "admin"];
ports = [8000 + 80] ++ [9443];
}
---
(source_file
(object
(field_definition
path: (field_path (identifier))
value: (binary_expression
left: (array
(literal (string)))
right: (array
(literal (string))
(literal (string)))))
(field_definition
path: (field_path (identifier))
value: (binary_expression
left: (array
(binary_expression
left: (literal (integer))
right: (literal (integer))))
right: (array
(literal (integer)))))))
==================
Array constraints
==================
{
tags = [...String];
ports = [...(Int & >= 1),];
}
---
(source_file
(object
(field_definition
path: (field_path (identifier))
value: (array_constraint
element: (identifier)))
(field_definition
path: (field_path (identifier))
value: (array_constraint
element: (parenthesized_expression
(binary_expression
left: (identifier)
right: (comparison_constraint
value: (integer))))))))
==================
Map constraint and ascription
==================
{
services = {
api = { port = 8080; };
} as {...{ port = Int; }};
}
---
(source_file
(object
(field_definition
path: (field_path (identifier))
value: (as_expression
narrower: (object
(field_definition
path: (field_path (identifier))
value: (object
(field_definition
path: (field_path (identifier))
value: (literal (integer))))))
wider: (map_constraint
value: (object
(field_definition
path: (field_path (identifier))
value: (identifier))))))))
==================
Unknown object rest constraint
==================
{
value = {
hoge = Int;
fuga = String;
...Unknown
};
}
---
(source_file
(object
(field_definition
path: (field_path (identifier))
value: (object
(field_definition
path: (field_path (identifier))
value: (identifier))
(field_definition
path: (field_path (identifier))
value: (identifier))
(object_rest
value: (identifier))))))
+90 -9
View File
@@ -1,9 +1,20 @@
// Tree-sitter implementation of the canonical EBNF grammar in
// doc/manual/souce/language/grammar.md.
const PREC = {
AS: 0,
DEFAULT: 1,
PATCH: 2,
AND: 3,
CALL: 7,
PATH: 8,
COMPOSE: 3,
OR: 4,
LOGICAL_AND: 5,
COMPARE: 6,
CONCAT: 7,
ADD: 8,
MUL: 9,
UNARY: 10,
CALL: 11,
PATH: 12,
};
function commaSep(rule) {
@@ -43,7 +54,9 @@ module.exports = grammar({
$.identifier,
$.regex_literal,
$.comparison_constraint,
$.map_constraint,
$.object,
$.array_constraint,
$.array,
$.let_expression,
$.function_expression,
@@ -52,8 +65,10 @@ module.exports = grammar({
$.parenthesized_expression,
$.call_expression,
$.path_expression,
$.unary_expression,
$.binary_expression,
$.default_expression,
$.as_expression,
),
literal: $ => choice(
@@ -84,7 +99,24 @@ module.exports = grammar({
object: $ => seq(
'{',
semiSep($.field_definition),
optional(seq(
$.field_definition,
repeat(seq(';', $.field_definition)),
optional(seq(';', $.object_rest)),
optional(';'),
)),
'}',
),
object_rest: $ => seq(
'...',
field('value', $._expression),
),
map_constraint: $ => seq(
'{',
'...',
field('value', $._expression),
'}',
),
@@ -105,12 +137,20 @@ module.exports = grammar({
']',
),
let_expression: $ => seq(
array_constraint: $ => seq(
'[',
'...',
field('element', $._expression),
optional(','),
']',
),
let_expression: $ => prec.right(seq(
'let',
repeat(seq($.field_definition, ';')),
'in',
field('body', $._expression),
),
)),
function_expression: $ => prec.right(seq(
'(',
@@ -159,13 +199,48 @@ module.exports = grammar({
field('field', $.identifier),
)),
comparison_constraint: $ => prec(6, seq(
comparison_constraint: $ => prec.right(PREC.UNARY + 1, seq(
field('operator', choice('>', '>=', '<', '<=')),
field('value', choice($.integer, $.float)),
field('value', $._expression),
)),
unary_expression: $ => prec(PREC.UNARY, seq(
field('operator', choice('-', '!')),
field('operand', $._expression),
)),
binary_expression: $ => choice(
prec.left(PREC.AND, seq(
prec.left(PREC.OR, seq(
field('left', $._expression),
field('operator', token(prec(2, '||'))),
field('right', $._expression),
)),
prec.left(PREC.LOGICAL_AND, seq(
field('left', $._expression),
field('operator', token(prec(2, '&&'))),
field('right', $._expression),
)),
prec.left(PREC.COMPARE, seq(
field('left', $._expression),
field('operator', choice('==', '!=', '>', '>=', '<', '<=')),
field('right', $._expression),
)),
prec.left(PREC.CONCAT, seq(
field('left', $._expression),
field('operator', token(prec(2, '++'))),
field('right', $._expression),
)),
prec.left(PREC.ADD, seq(
field('left', $._expression),
field('operator', choice('+', '-')),
field('right', $._expression),
)),
prec.left(PREC.MUL, seq(
field('left', $._expression),
field('operator', choice('*', '/')),
field('right', $._expression),
)),
prec.left(PREC.COMPOSE, seq(
field('left', $._expression),
field('operator', '&'),
field('right', $._expression),
@@ -182,5 +257,11 @@ module.exports = grammar({
'default',
field('fallback', $._expression),
)),
as_expression: $ => prec.left(PREC.AS, seq(
field('narrower', $._expression),
'as',
field('wider', $._expression),
)),
},
});
@@ -12,11 +12,23 @@
"match"
"import"
"default"
"as"
] @keyword
[
"&"
"//"
"+"
"-"
"*"
"/"
"++"
"..."
"&&"
"||"
"!"
"=="
"!="
"=>"
"="
">"
@@ -42,7 +54,7 @@
] @punctuation.delimiter
((identifier) @type.builtin
(#match? @type.builtin "^(String|Int|Float|Bool)$"))
(#match? @type.builtin "^(String|Int|Float|Bool|Unknown)$"))
(field_definition
path: (field_path (identifier) @property))
+444 -8
View File
@@ -91,10 +91,18 @@
"type": "SYMBOL",
"name": "comparison_constraint"
},
{
"type": "SYMBOL",
"name": "map_constraint"
},
{
"type": "SYMBOL",
"name": "object"
},
{
"type": "SYMBOL",
"name": "array_constraint"
},
{
"type": "SYMBOL",
"name": "array"
@@ -127,6 +135,10 @@
"type": "SYMBOL",
"name": "path_expression"
},
{
"type": "SYMBOL",
"name": "unary_expression"
},
{
"type": "SYMBOL",
"name": "binary_expression"
@@ -134,6 +146,10 @@
{
"type": "SYMBOL",
"name": "default_expression"
},
{
"type": "SYMBOL",
"name": "as_expression"
}
]
},
@@ -286,6 +302,27 @@
]
}
},
{
"type": "CHOICE",
"members": [
{
"type": "SEQ",
"members": [
{
"type": "STRING",
"value": ";"
},
{
"type": "SYMBOL",
"name": "object_rest"
}
]
},
{
"type": "BLANK"
}
]
},
{
"type": "CHOICE",
"members": [
@@ -311,6 +348,48 @@
}
]
},
"object_rest": {
"type": "SEQ",
"members": [
{
"type": "STRING",
"value": "..."
},
{
"type": "FIELD",
"name": "value",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
},
"map_constraint": {
"type": "SEQ",
"members": [
{
"type": "STRING",
"value": "{"
},
{
"type": "STRING",
"value": "..."
},
{
"type": "FIELD",
"name": "value",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "STRING",
"value": "}"
}
]
},
"field_definition": {
"type": "SEQ",
"members": [
@@ -423,7 +502,47 @@
}
]
},
"array_constraint": {
"type": "SEQ",
"members": [
{
"type": "STRING",
"value": "["
},
{
"type": "STRING",
"value": "..."
},
{
"type": "FIELD",
"name": "element",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "CHOICE",
"members": [
{
"type": "STRING",
"value": ","
},
{
"type": "BLANK"
}
]
},
{
"type": "STRING",
"value": "]"
}
]
},
"let_expression": {
"type": "PREC_RIGHT",
"value": 0,
"content": {
"type": "SEQ",
"members": [
{
@@ -459,6 +578,7 @@
}
}
]
}
},
"function_expression": {
"type": "PREC_RIGHT",
@@ -712,7 +832,7 @@
},
"call_expression": {
"type": "PREC_LEFT",
"value": 7,
"value": 11,
"content": {
"type": "SEQ",
"members": [
@@ -782,7 +902,7 @@
},
"path_expression": {
"type": "PREC_LEFT",
"value": 8,
"value": 12,
"content": {
"type": "SEQ",
"members": [
@@ -810,8 +930,8 @@
}
},
"comparison_constraint": {
"type": "PREC",
"value": 6,
"type": "PREC_RIGHT",
"value": 11,
"content": {
"type": "SEQ",
"members": [
@@ -843,19 +963,44 @@
{
"type": "FIELD",
"name": "value",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
},
"unary_expression": {
"type": "PREC",
"value": 10,
"content": {
"type": "SEQ",
"members": [
{
"type": "FIELD",
"name": "operator",
"content": {
"type": "CHOICE",
"members": [
{
"type": "SYMBOL",
"name": "integer"
"type": "STRING",
"value": "-"
},
{
"type": "SYMBOL",
"name": "float"
"type": "STRING",
"value": "!"
}
]
}
},
{
"type": "FIELD",
"name": "operand",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
@@ -863,6 +1008,268 @@
"binary_expression": {
"type": "CHOICE",
"members": [
{
"type": "PREC_LEFT",
"value": 4,
"content": {
"type": "SEQ",
"members": [
{
"type": "FIELD",
"name": "left",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "FIELD",
"name": "operator",
"content": {
"type": "TOKEN",
"content": {
"type": "PREC",
"value": 2,
"content": {
"type": "STRING",
"value": "||"
}
}
}
},
{
"type": "FIELD",
"name": "right",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
},
{
"type": "PREC_LEFT",
"value": 5,
"content": {
"type": "SEQ",
"members": [
{
"type": "FIELD",
"name": "left",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "FIELD",
"name": "operator",
"content": {
"type": "TOKEN",
"content": {
"type": "PREC",
"value": 2,
"content": {
"type": "STRING",
"value": "&&"
}
}
}
},
{
"type": "FIELD",
"name": "right",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
},
{
"type": "PREC_LEFT",
"value": 6,
"content": {
"type": "SEQ",
"members": [
{
"type": "FIELD",
"name": "left",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "FIELD",
"name": "operator",
"content": {
"type": "CHOICE",
"members": [
{
"type": "STRING",
"value": "=="
},
{
"type": "STRING",
"value": "!="
},
{
"type": "STRING",
"value": ">"
},
{
"type": "STRING",
"value": ">="
},
{
"type": "STRING",
"value": "<"
},
{
"type": "STRING",
"value": "<="
}
]
}
},
{
"type": "FIELD",
"name": "right",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
},
{
"type": "PREC_LEFT",
"value": 7,
"content": {
"type": "SEQ",
"members": [
{
"type": "FIELD",
"name": "left",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "FIELD",
"name": "operator",
"content": {
"type": "TOKEN",
"content": {
"type": "PREC",
"value": 2,
"content": {
"type": "STRING",
"value": "++"
}
}
}
},
{
"type": "FIELD",
"name": "right",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
},
{
"type": "PREC_LEFT",
"value": 8,
"content": {
"type": "SEQ",
"members": [
{
"type": "FIELD",
"name": "left",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "FIELD",
"name": "operator",
"content": {
"type": "CHOICE",
"members": [
{
"type": "STRING",
"value": "+"
},
{
"type": "STRING",
"value": "-"
}
]
}
},
{
"type": "FIELD",
"name": "right",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
},
{
"type": "PREC_LEFT",
"value": 9,
"content": {
"type": "SEQ",
"members": [
{
"type": "FIELD",
"name": "left",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "FIELD",
"name": "operator",
"content": {
"type": "CHOICE",
"members": [
{
"type": "STRING",
"value": "*"
},
{
"type": "STRING",
"value": "/"
}
]
}
},
{
"type": "FIELD",
"name": "right",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
},
{
"type": "PREC_LEFT",
"value": 3,
@@ -966,6 +1373,35 @@
}
]
}
},
"as_expression": {
"type": "PREC_LEFT",
"value": 0,
"content": {
"type": "SEQ",
"members": [
{
"type": "FIELD",
"name": "narrower",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
},
{
"type": "STRING",
"value": "as"
},
{
"type": "FIELD",
"name": "wider",
"content": {
"type": "SYMBOL",
"name": "_expression"
}
}
]
}
}
},
"extras": [
File diff suppressed because it is too large Load Diff
+9815 -2830
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -16,7 +16,7 @@ This example exercises multiple current Decodal features together:
Run it with:
```sh
cargo run -q -p decodal -- examples/advanced/main.dcdl
cargo run -q -p decodal-cli -- examples/advanced/main.dcdl
```
The entrypoint is `main.dcdl`.
+7
View File
@@ -0,0 +1,7 @@
{
workers = 2 + 2;
memory_gib = 1.5 * 4;
port = 9000 + 443;
timeout_seconds = 30 / 2;
negative_offset = -3;
}
+6
View File
@@ -0,0 +1,6 @@
{
base_roles = ["read", "write"];
extra_roles = ["admin"];
roles = ["read", "write"] ++ ["admin"];
ports = [8000 + 80] ++ [9000 + 443];
}
+11
View File
@@ -0,0 +1,11 @@
let
Service = {
name = String;
enabled = Bool default true;
};
Services = [...Service];
in
Services & [
{ name = "api"; },
{ name = "worker"; enabled = false; },
]
+11
View File
@@ -0,0 +1,11 @@
let
env = "prod";
replicas = 3;
in
{
is_prod = env == "prod";
scaled = replicas > 1;
enabled = env == "prod" && replicas > 1;
disabled = !(env == "prod" && replicas > 1);
safe = env != "dev" || replicas >= 1;
}
+15
View File
@@ -0,0 +1,15 @@
let
Service = {
port = Int & >= 1 & <= 65535;
enabled = Bool default true;
};
in
{
api = {
port = 8080;
};
worker = {
port = 8081;
enabled = false;
};
} as {...Service}
+11
View File
@@ -0,0 +1,11 @@
OpenConfig = {
enabled = Bool default true;
...Unknown
};
config = {
enabled = false;
plugin = {
name = "cache";
};
} as OpenConfig;
+1 -1
View File
@@ -3,7 +3,7 @@
Regex constraints are optional. Run this example with the `regex` feature enabled:
```sh
cargo run -q -p decodal --features regex -- examples/regex/main.dcdl
cargo run -q -p decodal-cli --features regex -- examples/regex/main.dcdl
```
Without the feature, regex validation returns an unsupported feature diagnostic.
+1
View File
@@ -29,6 +29,7 @@
apps.default = mkApp "decodal" "Run Decodal";
apps.decodal = mkApp "decodal" "Run Decodal";
apps.decodal-lsp = mkApp "decodal-lsp" "Run the Decodal language server";
checks.default = decodal;
+5 -2
View File
@@ -24,7 +24,7 @@ let
in
rustPlatform.buildRustPackage {
pname = "decodal";
version = "0.1.0";
version = "0.1.2";
src = lib.cleanSourceWith {
src = srcRoot;
@@ -37,7 +37,9 @@ rustPlatform.buildRustPackage {
cargoBuildFlags = [
"-p"
"decodal"
"decodal-cli"
"-p"
"decodal-lsp"
];
doInstallCheck = true;
@@ -45,6 +47,7 @@ rustPlatform.buildRustPackage {
runHook preInstallCheck
test -x "$out/bin/decodal"
test -x "$out/bin/decodal-lsp"
"$out/bin/decodal" >/dev/null
runHook postInstallCheck
+173
View File
@@ -0,0 +1,173 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Decodal contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+74
View File
@@ -0,0 +1,74 @@
# decodal-codemirror
CodeMirror 6 language support for Decodal.
This package provides the Lezer parser metadata, highlighting tags, indentation rules, block folding, and formatter command used by the Decodal Web playground.
It does not include the Decodal runtime evaluator.
Use `decodal-wasm` when you want to evaluate Decodal in a browser.
## Install
```sh
npm install decodal-codemirror
```
JSR:
```sh
deno add jsr:@hare/decodal-codemirror
```
## Usage
```js
import { basicSetup } from 'codemirror';
import { EditorView } from '@codemirror/view';
import { decodal } from 'decodal-codemirror';
new EditorView({
doc: 'Server = { port = Int default 8080; };',
extensions: [
basicSetup,
decodal(),
],
parent: document.querySelector('#editor'),
});
```
## Exports
```js
import {
decodal,
decodalLanguage,
decodalHighlightStyle,
} from 'decodal-codemirror';
```
`decodal()` accepts a small options object:
```js
decodal({ highlight: false })
```
Use this when you want the language support without the bundled highlight style.
## Formatting
The formatter is exposed as a separate subpath so plain syntax highlighting does not force callers to initialize WebAssembly.
```js
import {
formatDecodal,
formatDecodalCommand,
initDecodalFormatter,
} from 'decodal-codemirror/format';
import formatterWasm from 'decodal-codemirror/wasm/decodal_language_tools_bg.wasm?url';
await initDecodalFormatter(formatterWasm);
const result = formatDecodal('Server={port=Int default 8080;};');
if (result.ok) console.log(result.source);
formatDecodalCommand(view);
```

Some files were not shown because too many files have changed in this diff Show More