Compare commits

72 Commits
Author SHA1 Message Date
Hare e5c73c647d Add SEO and agent-readable documentation 2026-08-14 21:10:52 +09:00
Hare cda32cc260 Add bilingual manual and localized docs routes 2026-08-14 11:16:37 +09:00
Hare 3645a2cc2d Add bilingual homepage and self-hosted fonts 2026-08-14 11:00:11 +09:00
Hare e50458595b Redesign site and streamline playground 2026-08-14 10:41:37 +09:00
Hare 6653607ed3 Improve documentation navigation and code blocks 2026-08-14 08:37:03 +09:00
Hare e0970acdcc Refocus manual on public language usage 2026-08-14 08:17:25 +09:00
Hare e0a9a8efb7 Bump packages for open value release 2026-08-14 08:03:25 +09:00
Hare 848c7f169f Add unknown ranges and open object values 2026-08-14 08:00:22 +09:00
Hare 8c50dd202d Bump packages for range refinement release 2026-08-14 06:29:48 +09:00
Hare cc6ab40807 Add associative map constraints and range refinement 2026-08-14 06:26:15 +09:00
Hare 8198b615a8 Clarify site build and deploy scripts 2026-08-13 19:53:38 +09:00
Hare 62aa81b91c Fix JSR license metadata 2026-08-13 19:45:04 +09:00
Hare 9c46635ee0 Bump Rust crates for breaking release 2026-08-13 19:40:08 +09:00
Hare 515adc2533 Unify formatting on canonical syntax 2026-08-13 19:31:21 +09:00
Hare d9cf24d0eb Prepare Decodal packages for release 2026-08-13 19:01:14 +09:00
Hare d28edcf041 Share host-aware completion across runtimes 2026-08-13 18:55:03 +09:00
Hare e862e53f3e Add project-aware playground completion 2026-08-13 18:07:24 +09:00
Hare 610358110f Add host-configurable LSP server 2026-08-12 05:42:45 +09:00
Hare bdaccd9803 Add host-configurable language service 2026-08-11 22:05:17 +09:00
Hare 08603dc4b5 Report structured import origins in diagnostics 2026-08-11 19:20:11 +09:00
Hare 165daada23 Add host-structured imports 2026-08-11 18:33:49 +09:00
Hare 46b2b3b1b6 Add required array element constraints 2026-08-11 17:24:04 +09:00
Hare 5a05da6fe4 Use non-underscore Astro asset directory 2026-07-10 04:36:22 +09:00
Hare aa0f464320 Bump CodeMirror package to 0.1.5 2026-07-10 04:30:38 +09:00
Hare fc1f3e0b25 Improve JSR package metadata 2026-07-10 04:19:00 +09:00
Hare 8eac5c5cb4 Bundle language tools with CodeMirror package 2026-07-09 21:25:04 +09:00
Hare 59cb8f31e4 Simplify package link labels 2026-07-09 20:50:09 +09:00
Hare 4eee2cd2c9 Make package links flex layout 2026-07-09 20:45:16 +09:00
Hare 44b6041e60 Compact package links on landing page 2026-07-09 20:36:57 +09:00
Hare a95afe0463 Align playground control heights 2026-07-09 19:36:15 +09:00
Hare 43ab6244ef Reduce button vertical padding 2026-07-09 19:34:33 +09:00
Hare 1d4503b015 Simplify playground action separator 2026-07-09 19:33:48 +09:00
Hare 250e29e00c Remove playground helper copy 2026-07-09 19:29:22 +09:00
Hare 1080081cec Reset state before loading playground examples 2026-07-09 19:27:19 +09:00
Hare 43f692e75c Add playground entry selector 2026-07-09 19:21:12 +09:00
Hare 5b26d071ca Run playground examples from entrypoint 2026-07-09 19:12:18 +09:00
Hare a653f0b7c6 Nest repeated playground fields 2026-07-09 19:06:28 +09:00
Hare 384d17b539 Replace playground examples 2026-07-09 19:02:11 +09:00
Hare 09a84264ab Format playground examples 2026-07-09 18:54:29 +09:00
Hare 0a8c261545 Load playground wasm assets through Vite 2026-07-09 18:50:26 +09:00
Hare a8a46998a8 Make build:wasm build all wasm packages 2026-07-09 18:46:24 +09:00
Hare e4a9f80617 Dedupe CodeMirror dependencies 2026-07-09 18:43:09 +09:00
Hare 0e2a7b35d2 Add language tools package 2026-07-09 18:34:41 +09:00
Hare ac99c4cb4e Revert "Add Decodal formatter"
This reverts commit 22f6bc1ab9.
2026-07-09 17:46:24 +09:00
Hare 22f6bc1ab9 Add Decodal formatter 2026-07-09 17:39:32 +09:00
Hare 16006d000a Move wasm package under packages 2026-07-09 04:13:08 +09:00
Hare e4b9247c45 Extract CodeMirror language package 2026-07-09 03:49:44 +09:00
Hare 375bc80a57 Add package links to landing page 2026-07-09 03:31:20 +09:00
Hare 335ecde816 Fix JSR wasm package license 2026-07-09 03:29:27 +09:00
Hare 4142fcd46a Prepare 0.1.2 release 2026-07-09 03:12:09 +09:00
Hare 2d06dc3277 Add source link to header 2026-07-09 03:00:30 +09:00
Hare 37237dc1f7 Add source link to footer 2026-07-09 02:59:32 +09:00
Hare 2966d5b979 Add site footer 2026-07-09 02:58:53 +09:00
Hare ea858ad28e Remove landing page eyebrow 2026-07-09 02:57:55 +09:00
Hare 871d342e15 Add site landing page 2026-07-09 02:55:50 +09:00
Hare 6d80c66144 Trim open issues to materialization 2026-07-09 02:46:28 +09:00
Hare 8e1cb01f52 Document Decodal components 2026-07-09 01:47:28 +09:00
Hare 26d8495631 Remove public tokenizer API 2026-07-09 01:09:40 +09:00
Hare c4332ffb9d Restore CodeMirror syntax highlighting 2026-07-09 00:38:31 +09:00
Hare 33ea8b94f9 Fix CodeMirror folding ranges 2026-07-09 00:04:34 +09:00
Hare 720c71157e Fold Decodal delimited blocks from opener 2026-07-08 23:55:04 +09:00
Hare ee031bd2e8 Add canonical grammar and CodeMirror playground 2026-07-08 23:41:26 +09:00
Hare a07f4c48aa Expose tokenizer for web highlighting 2026-07-08 19:50:04 +09:00
Hare fe127428f9 Update generated WASM package README 2026-06-26 01:26:23 +09:00
Hare 0b76a600b1 Make Pages deploy script portable 2026-06-26 01:25:22 +09:00
Hare ada3578a1d Expose derive macro behind feature 2026-06-26 00:59:39 +09:00
Hare b8404df047 Add typed Decodal derive support 2026-06-26 00:39:33 +09:00
Hare 87fef44c68 Make decodal the published library crate 2026-06-25 22:51:16 +09:00
Hare 6e2363632b Prepare crates.io release metadata 2026-06-25 22:25:01 +09:00
Hare c26f9ca50a Add playground example loader 2026-06-25 03:39:40 +09:00
Hare f928028007 Deploy Pages production branch by default 2026-06-25 03:12:37 +09:00
Hare d8af980f7d Add Cloudflare Pages deploy script 2026-06-25 02:49:15 +09:00
245 changed files with 27818 additions and 8205 deletions
+1
View File
@@ -9,3 +9,4 @@ site/decodal-site/dist
# Astro
site/decodal-site/.astro
site/decodal-site/.wrangler
Generated
+195 -10
View File
@@ -11,6 +11,12 @@ dependencies = [
"memchr",
]
[[package]]
name = "bitflags"
version = "1.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]]
name = "bumpalo"
version = "3.20.3"
@@ -24,34 +30,167 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "decodal"
version = "0.1.0"
name = "crossbeam-channel"
version = "0.5.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e"
dependencies = [
"decodal-core",
"crossbeam-utils",
]
[[package]]
name = "decodal-core"
version = "0.1.0"
name = "crossbeam-utils"
version = "0.8.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
[[package]]
name = "decodal"
version = "0.4.0"
dependencies = [
"decodal-derive",
"regex",
]
[[package]]
name = "decodal-wasm"
version = "0.1.0"
name = "decodal-cli"
version = "0.4.0"
dependencies = [
"decodal-core",
"decodal",
]
[[package]]
name = "decodal-derive"
version = "0.2.0"
dependencies = [
"decodal",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "decodal-language-service"
version = "0.4.0"
dependencies = [
"decodal",
]
[[package]]
name = "decodal-language-tools"
version = "0.4.0"
dependencies = [
"decodal",
"serde_json",
"wasm-bindgen",
]
[[package]]
name = "decodal-lsp"
version = "0.4.0"
dependencies = [
"decodal",
"decodal-language-service",
"decodal-language-tools",
"lsp-server",
"lsp-types",
"serde_json",
]
[[package]]
name = "decodal-wasm"
version = "0.4.0"
dependencies = [
"decodal",
"decodal-language-service",
"js-sys",
"serde-wasm-bindgen",
"serde_json",
"wasm-bindgen",
]
[[package]]
name = "fluent-uri"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "17c704e9dbe1ddd863da1e6ff3567795087b1eb201ce80d8fa81162e1516500d"
dependencies = [
"bitflags",
]
[[package]]
name = "futures-core"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e"
[[package]]
name = "futures-task"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd"
[[package]]
name = "futures-util"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc"
dependencies = [
"futures-core",
"futures-task",
"pin-project-lite",
"slab",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "js-sys"
version = "0.3.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03d04c30968dffe80775bd4d7fb676131cd04a1fb46d2686dbffbaec2d9dfd31"
dependencies = [
"cfg-if",
"futures-util",
"wasm-bindgen",
]
[[package]]
name = "log"
version = "0.4.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "lsp-server"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ee25a31f2e571e426eef2896179450cafc7e2f5be00d8a93b1c2d21c0ff7656"
dependencies = [
"crossbeam-channel",
"log",
"serde",
"serde_derive",
"serde_json",
]
[[package]]
name = "lsp-types"
version = "0.97.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53353550a17c04ac46c585feb189c2db82154fc84b79c7a66c96c2c644f66071"
dependencies = [
"bitflags",
"fluent-uri",
"serde",
"serde_json",
"serde_repr",
]
[[package]]
name = "memchr"
version = "2.8.2"
@@ -64,6 +203,12 @@ version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "proc-macro2"
version = "1.0.106"
@@ -124,6 +269,18 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde-wasm-bindgen"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8302e169f0eddcc139c70f139d19d6467353af16f9fce27e8c30158036a1e16b"
dependencies = [
"js-sys",
"serde",
"wasm-bindgen",
]
[[package]]
@@ -143,7 +300,7 @@ checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
"syn 2.0.117",
]
[[package]]
@@ -159,6 +316,23 @@ dependencies = [
"zmij",
]
[[package]]
name = "serde_repr"
version = "0.1.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d3b1629de253c70a0508c3899572da79ca359fdab27c7920ff00406df418906"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.3",
]
[[package]]
name = "slab"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "syn"
version = "2.0.117"
@@ -170,6 +344,17 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
@@ -208,7 +393,7 @@ dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn",
"syn 2.0.117",
"wasm-bindgen-shared",
]
+17 -7
View File
@@ -3,12 +3,22 @@ members = [
"crates/decodal-core",
"crates/decodal-cli",
"crates/decodal-wasm",
"crates/decodal-derive",
"crates/decodal-language-tools",
"crates/decodal-language-service",
"crates/decodal-lsp",
]
resolver = "3"
exclude = ["editors/tree-sitter-decodal"]
resolver = "2"
[profile.release]
opt-level = "z"
lto = true
codegen-units = 1
strip = true
panic = "abort"
[workspace.package]
version = "0.4.0"
edition = "2024"
rust-version = "1.85"
license = "MIT OR Apache-2.0"
repository = "https://gitea.hareworks.net/Hare/Decodal"
readme = "README.md"
[workspace.dependencies]
serde_json = "1"
wasm-bindgen = "0.2"
+173
View File
@@ -0,0 +1,173 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Decodal contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+109
View File
@@ -0,0 +1,109 @@
# Decodal
Decodal is a small deterministic DSL for describing, composing, validating, and materializing structured data.
It is designed around a lightweight Rust library:
- host-supplied source and structured-value imports through `ImportLoader`
- shared host environments for production and semantic editor tooling
- no filesystem access in the library core
- concrete and abstract values with constraints and defaults
- asymmetric range refinement with `narrower as wider`
- homogeneous associative-array schemas with `{...valueSchema}` and object rest constraints
- the safe top range `Unknown`, which remains abstract until refined, defaulted, or supplied concretely
- deterministic expression evaluation
- optional regex support behind a Cargo feature
- browser playground support through WebAssembly
## Library crate
Embedded hosts should depend on `decodal` and provide imports with an `ImportLoader`.
```toml
[dependencies]
decodal = "0.4.0"
```
## Derive support
For embedded Rust applications, Decodal can generate a schema and typed decoder from a Rust struct with the `derive` feature.
Map fields marked with `#[decodal(rest)]` receive additional object fields; `BTreeMap<String, Data>` corresponds to `...Unknown`.
```toml
[dependencies]
decodal = { version = "0.4.0", features = ["derive"] }
```
```rust
use decodal::{Decodal, DecodalDecode, DecodalSchema, Engine};
#[derive(Decodal)]
struct Service {
name: String,
#[decodal(gt = 443, default = 8443)]
port: i64,
#[decodal(rename = "feature.enable", default = true)]
feature_enabled: bool,
}
```
The derive implements:
- `DecodalSchema`, which produces a `Value` range for `Engine::bind_global`
- `DecodalDecode`, which converts materialized `Data` into the Rust struct
## CLI
A standalone CLI is kept in this repository as the `decodal-cli` workspace package.
It builds a `decodal` binary, but it is not the primary crates.io package.
Run a Decodal file from the repository:
```sh
cargo run -q -p decodal-cli -- examples/advanced/main.dcdl
```
Enable optional regex support when needed:
```sh
cargo run -q -p decodal-cli --features regex -- examples/regex/main.dcdl
```
## Language server
The `decodal-language-service` crate provides transport-independent evaluation and completion, while `decodal-lsp` exposes it through the Language Server Protocol with live semantic diagnostics and document formatting.
Its library API accepts the same host environment used for production evaluation, so application-defined globals and structured import routing remain available in the editor.
```toml
[dependencies]
decodal = "0.4.0"
decodal-language-service = "0.4.0"
decodal-lsp = "0.4.0"
```
Run the default filesystem-backed server from the repository:
```sh
cargo run -q -p decodal-lsp
```
Embedded applications can implement `decodal_lsp::LspEnvironment` for their existing `HostEnvironment` and construct it from the `InitializeParams` passed to `decodal_lsp::run_stdio`.
## Web playground
The static documentation site and browser playground live under:
```text
site/decodal-site/
```
Browser hosts can use `decodal-wasm`. Its `DecodalLanguageService` accepts JavaScript-owned `globals`, `loadImport`, and `completeImport` callbacks; filesystem and virtual-project policy remain outside the package.
## License
Licensed under either of:
- Apache License, Version 2.0
- MIT license
at your option.
+62
View File
@@ -0,0 +1,62 @@
# Releasing Decodal
Run all commands from the repository root. Publishing requires Cargo, npm, and JSR credentials; validation does not.
## Versions
- `decodal-derive`: `0.2.0`
- Other published Rust crates: `0.4.0`
- `decodal-wasm` npm/JSR package: `0.3.0`
- `decodal-codemirror`: `0.3.0`
The npm packages advertise the repository's dual `MIT OR Apache-2.0` license.
JSR metadata uses the single `MIT` identifier required by its publisher; both license files remain included in every package.
## Validate
```sh
cargo fmt --check
cargo test --workspace
cargo test -p decodal --no-default-features
cargo clippy --workspace --all-targets -- -D warnings
cargo clippy -p decodal-wasm --target wasm32-unknown-unknown -- -D warnings
cargo clippy -p decodal-language-tools --target wasm32-unknown-unknown -- -D warnings
npm --prefix site/decodal-site run build:wasm
npm --prefix packages/decodal-codemirror test
npm --prefix site/decodal-site test
npm --prefix site/decodal-site run build
deno check packages/decodal-wasm/mod.ts
npm pack --dry-run ./packages/decodal-wasm
deno publish --dry-run --config packages/decodal-wasm/jsr.json
npm pack --dry-run ./packages/decodal-codemirror
deno publish --dry-run --config packages/decodal-codemirror/jsr.json
```
## Publish Rust crates
Publish in dependency order. Wait for each crate to become available in the crates.io index before continuing.
Run the corresponding command with `--dry-run` immediately before each real publish.
```sh
cargo publish -p decodal-derive
cargo publish -p decodal
cargo publish -p decodal-language-service
cargo publish -p decodal-language-tools
cargo publish -p decodal-lsp
```
`decodal-cli` and the Rust `decodal-wasm` wrapper remain repository-only packages.
## Publish JavaScript packages
The WASM build regenerates and normalizes both committed npm/JSR packages before publishing.
```sh
npm --prefix site/decodal-site run build:wasm
npm publish ./packages/decodal-wasm
deno publish --config packages/decodal-wasm/jsr.json
npm publish ./packages/decodal-codemirror
deno publish --config packages/decodal-codemirror/jsr.json
```
After both registries accept the release, tag the release commit and update the deployed site.
+16 -4
View File
@@ -1,11 +1,23 @@
[package]
name = "decodal-cli"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Command-line interface for the Decodal data description language."
keywords = ["decodal", "dsl", "config", "cli"]
categories = ["command-line-utilities", "config"]
publish = false
[[bin]]
name = "decodal"
version = "0.1.0"
edition = "2024"
path = "src/main.rs"
[features]
default = []
regex = ["decodal-core/regex"]
regex = ["decodal/regex"]
[dependencies]
decodal-core = { path = "../decodal-core" }
decodal = { version = "0.4.0", path = "../decodal-core" }
+16 -10
View File
@@ -4,11 +4,17 @@ use std::{
process::ExitCode,
};
use decodal_core::{
Data, Diagnostic, DiagnosticKind, Engine, LoadedSource, SourceId, SourceLoader, Span,
use decodal::{
Data, Diagnostic, DiagnosticKind, Engine, ImportLoader, LoadedImport, SourceId, Span,
format_diagnostic_with,
};
struct SourceInput {
key: String,
name: String,
source: String,
}
fn main() -> ExitCode {
match run() {
Ok(()) => ExitCode::SUCCESS,
@@ -28,7 +34,7 @@ fn run() -> Result<(), String> {
Ok(())
}
"--version" | "-V" => {
println!("Decodal {}", decodal_core::version());
println!("Decodal {}", decodal::version());
Ok(())
}
"check" => {
@@ -69,7 +75,7 @@ fn materialize_path(path: &str) -> Result<Data, String> {
.map_err(|error| engine.format_diagnostic(&error))
}
fn read_root_source(path: &str) -> Result<LoadedSource, Diagnostic> {
fn read_root_source(path: &str) -> Result<SourceInput, Diagnostic> {
if path == "-" {
use std::io::Read;
let mut source = String::new();
@@ -82,7 +88,7 @@ fn read_root_source(path: &str) -> Result<LoadedSource, Diagnostic> {
format!("failed to read stdin: {error}"),
)
})?;
Ok(LoadedSource {
Ok(SourceInput {
key: String::from("<stdin>"),
name: String::from("<stdin>"),
source,
@@ -95,12 +101,12 @@ fn read_root_source(path: &str) -> Result<LoadedSource, Diagnostic> {
#[derive(Debug, Clone, Copy)]
struct FsLoader;
impl SourceLoader for FsLoader {
impl ImportLoader for FsLoader {
fn load(
&mut self,
current_key: Option<&str>,
specifier: &str,
) -> Result<LoadedSource, Diagnostic> {
) -> Result<LoadedImport, Diagnostic> {
let path = Path::new(specifier);
let path = if path.is_absolute() {
PathBuf::from(path)
@@ -112,11 +118,11 @@ impl SourceLoader for FsLoader {
} else {
PathBuf::from(path)
};
load_path(&path)
load_path(&path).map(|source| LoadedImport::source(source.key, source.name, source.source))
}
}
fn load_path(path: &Path) -> Result<LoadedSource, Diagnostic> {
fn load_path(path: &Path) -> Result<SourceInput, Diagnostic> {
let canonical = path.canonicalize().map_err(|error| {
Diagnostic::new(
DiagnosticKind::Import,
@@ -132,7 +138,7 @@ fn load_path(path: &Path) -> Result<LoadedSource, Diagnostic> {
)
})?;
let key = canonical.to_string_lossy().into_owned();
Ok(LoadedSource {
Ok(SourceInput {
name: key.clone(),
key,
source,
+12 -3
View File
@@ -1,12 +1,21 @@
[package]
name = "decodal-core"
version = "0.1.0"
edition = "2024"
name = "decodal"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Parser, evaluator, and embedding API for the Decodal data description language."
keywords = ["decodal", "dsl", "config", "schema"]
categories = ["config", "parser-implementations"]
[features]
default = ["std"]
std = []
derive = ["dep:decodal-derive"]
regex = ["std", "dep:regex"]
[dependencies]
decodal-derive = { version = "0.2.0", path = "../decodal-derive", optional = true }
regex = { version = "1.10", default-features = false, features = ["std", "unicode-perl"], optional = true }
+6 -6
View File
@@ -1,14 +1,14 @@
use decodal_core::{Data, EmptyLoader, Engine, HostValue};
use decodal::{Data, EmptyLoader, Engine, Value};
fn main() -> decodal_core::Result<()> {
fn main() -> decodal::Result<()> {
let mut engine = Engine::new(EmptyLoader);
engine.bind_global(
"Service",
HostValue::object([
("name", HostValue::string_type()),
("port", HostValue::int_type().gt(443).default_int(8443)?),
("enabled", HostValue::bool_type().default_bool(true)?),
Value::object([
("name", Value::string_type()),
("port", Value::int_type().gt(443).default_int(8443)?),
("enabled", Value::bool_type().default_bool(true)?),
]),
)?;
@@ -0,0 +1,85 @@
use decodal::{Data, Diagnostic, DiagnosticKind, Engine, ImportLoader, LoadedImport, Span, Value};
const POST: &str = r#"---
title: Hello
draft: false
---
# Hello
This body stays as Markdown.
"#;
struct ContentLoader;
impl ImportLoader for ContentLoader {
fn load(
&mut self,
_current_key: Option<&str>,
specifier: &str,
) -> decodal::Result<LoadedImport> {
if specifier != "./post.md" {
return Err(Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
format!("unknown content import `{specifier}`"),
));
}
Ok(LoadedImport::value(
"content/post.md",
parse_markdown(POST)?,
))
}
}
fn parse_markdown(source: &str) -> decodal::Result<Value> {
let source = source.strip_prefix("---\n").ok_or_else(frontmatter_error)?;
let (frontmatter, body) = source.split_once("\n---\n").ok_or_else(frontmatter_error)?;
let mut fields = Vec::new();
for line in frontmatter.lines() {
let (name, value) = line.split_once(':').ok_or_else(frontmatter_error)?;
let value = match value.trim() {
"true" => Value::bool(true),
"false" => Value::bool(false),
value => Value::string(value),
};
fields.push((name.trim(), value));
}
Ok(Value::object([
("frontmatter", Value::object(fields)),
("body", Value::string(body)),
]))
}
fn frontmatter_error() -> Diagnostic {
Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
"invalid Markdown frontmatter",
)
}
fn main() -> decodal::Result<()> {
let mut engine = Engine::new(ContentLoader);
let module = engine.add_root_source(
"main.dcdl",
"main.dcdl",
r#"
let post = import "./post.md";
in {
title = post.frontmatter.title;
draft = post.frontmatter.draft;
body = post.body;
}
"#,
)?;
let value = engine.eval_module(module)?;
let data = engine.materialize(&value)?;
let Data::Object(fields) = data else {
panic!("expected imported Markdown to produce an object")
};
assert_eq!(fields[0].value, Data::String(String::from("Hello")));
assert_eq!(fields[1].value, Data::Bool(false));
assert!(matches!(fields[2].value, Data::String(_)));
Ok(())
}
+27 -1
View File
@@ -48,8 +48,17 @@ pub struct SpannedExpr {
pub enum Expr {
Literal(Literal),
Ident(String),
Object(Vec<Field>),
Object {
fields: Vec<Field>,
rest: Option<ObjectRest>,
},
Array(Vec<ExprId>),
ArrayConstraint {
item: ExprId,
},
MapConstraint {
value: ExprId,
},
Let {
bindings: Vec<Field>,
body: ExprId,
@@ -67,6 +76,13 @@ pub enum Expr {
params: Vec<Param>,
body: ExprId,
},
/// An explicitly parenthesized expression.
///
/// Keeping this node preserves the delimiters and their trivia for source
/// tools while evaluation remains equivalent to evaluating `expr`.
Parenthesized {
expr: ExprId,
},
Match {
scrutinee: ExprId,
arms: Vec<MatchArm>,
@@ -84,6 +100,10 @@ pub enum Expr {
base: ExprId,
fallback: ExprId,
},
As {
narrower: ExprId,
wider: ExprId,
},
CompareConstraint {
op: CompareOp,
value: ExprId,
@@ -99,6 +119,12 @@ pub struct Field {
pub span: Span,
}
#[derive(Debug, Clone, PartialEq)]
pub struct ObjectRest {
pub value: ExprId,
pub span: Span,
}
#[derive(Debug, Clone, PartialEq)]
pub struct Param {
pub name: String,
+86
View File
@@ -11,12 +11,18 @@ pub fn normalize_constraints(
span: Span,
) -> crate::Result<Vec<ConstraintEntry>> {
let mut primitive: Option<(PrimitiveType, Span)> = None;
let mut array_span: Option<Span> = None;
let mut map_span: Option<Span> = None;
let mut lower: Option<(Bound, Span)> = None;
let mut upper: Option<(Bound, Span)> = None;
let mut rest = Vec::new();
for entry in constraints {
match entry.constraint {
Constraint::Unknown => rest.push(ConstraintEntry {
constraint: Constraint::Unknown,
span: entry.span,
}),
Constraint::Type(next) => match primitive {
Some((current, current_span)) if current != next => {
return Err(Diagnostic::new(
@@ -61,6 +67,20 @@ pub fn normalize_constraints(
constraint: Constraint::Regex(pattern),
span: entry.span,
}),
Constraint::ArrayItems(item) => {
array_span.get_or_insert(entry.span);
rest.push(ConstraintEntry {
constraint: Constraint::ArrayItems(item),
span: entry.span,
});
}
Constraint::MapValues(value) => {
map_span.get_or_insert(entry.span);
rest.push(ConstraintEntry {
constraint: Constraint::MapValues(value),
span: entry.span,
});
}
Constraint::BuiltinPredicate(name) => rest.push(ConstraintEntry {
constraint: Constraint::BuiltinPredicate(name),
span: entry.span,
@@ -68,6 +88,72 @@ pub fn normalize_constraints(
}
}
if let (Some((_, primitive_span)), Some(array_span)) = (primitive, array_span) {
return Err(Diagnostic::new(
DiagnosticKind::Conflict,
span,
"array element constraints conflict with primitive type constraints",
)
.with_label(primitive_span, "primitive type constraint")
.with_label(array_span, "array element constraint"));
}
if let (Some((_, primitive_span)), Some(map_span)) = (primitive, map_span) {
return Err(Diagnostic::new(
DiagnosticKind::Conflict,
span,
"map value constraints conflict with primitive type constraints",
)
.with_label(primitive_span, "primitive constraint")
.with_label(map_span, "map value constraint"));
}
if let (Some(array_span), Some(map_span)) = (array_span, map_span) {
return Err(Diagnostic::new(
DiagnosticKind::Conflict,
span,
"array and map constraints conflict",
)
.with_label(array_span, "array element constraint")
.with_label(map_span, "map value constraint"));
}
if let Some(array_span) = array_span
&& (lower.is_some() || upper.is_some())
{
let mut diagnostic = Diagnostic::new(
DiagnosticKind::Conflict,
span,
"numeric comparison constraints conflict with array element constraints",
)
.with_label(array_span, "array element constraint");
if let Some((_, lower_span)) = lower {
diagnostic = diagnostic.with_label(lower_span, "numeric comparison constraint");
}
if let Some((_, upper_span)) = upper {
diagnostic = diagnostic.with_label(upper_span, "numeric comparison constraint");
}
return Err(diagnostic);
}
if let Some(map_span) = map_span
&& (lower.is_some() || upper.is_some())
{
let mut diagnostic = Diagnostic::new(
DiagnosticKind::Conflict,
span,
"numeric comparison constraints conflict with map value constraints",
)
.with_label(map_span, "map value constraint");
if let Some((_, lower_span)) = lower {
diagnostic = diagnostic.with_label(lower_span, "numeric comparison constraint");
}
if let Some((_, upper_span)) = upper {
diagnostic = diagnostic.with_label(upper_span, "numeric comparison constraint");
}
return Err(diagnostic);
}
if matches!(
primitive,
Some((PrimitiveType::String | PrimitiveType::Bool, _))
+7
View File
@@ -10,6 +10,7 @@ pub struct Diagnostic {
pub span: Span,
pub message: String,
pub labels: Vec<DiagnosticLabel>,
pub notes: Vec<String>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -25,6 +26,7 @@ impl Diagnostic {
span,
message: message.into(),
labels: Vec::new(),
notes: Vec::new(),
}
}
@@ -39,6 +41,11 @@ impl Diagnostic {
});
self
}
pub fn with_note(mut self, message: impl Into<String>) -> Self {
self.notes.push(message.into());
self
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
-164
View File
@@ -1,164 +0,0 @@
use alloc::{boxed::Box, string::String, vec::Vec};
use crate::runtime::{Constraint, LiteralValue, PrimitiveType};
use crate::{CompareOp, Diagnostic, DiagnosticKind, Result, Span};
#[derive(Debug, Clone, PartialEq)]
pub enum HostValue {
String(String),
Int(i64),
Float(f64),
Bool(bool),
Array(Vec<HostValue>),
Object(Vec<HostField>),
Abstract {
constraints: Vec<Constraint>,
default: Option<Box<HostValue>>,
},
}
#[derive(Debug, Clone, PartialEq)]
pub struct HostField {
pub name: String,
pub value: HostValue,
}
impl HostValue {
pub fn string(value: impl Into<String>) -> Self {
Self::String(value.into())
}
pub fn int(value: i64) -> Self {
Self::Int(value)
}
pub fn float(value: f64) -> Self {
Self::Float(value)
}
pub fn bool(value: bool) -> Self {
Self::Bool(value)
}
pub fn array<I>(items: I) -> Self
where
I: IntoIterator<Item = HostValue>,
{
Self::Array(items.into_iter().collect())
}
pub fn object<I, N>(fields: I) -> Self
where
I: IntoIterator<Item = (N, HostValue)>,
N: Into<String>,
{
Self::Object(
fields
.into_iter()
.map(|(name, value)| HostField {
name: name.into(),
value,
})
.collect(),
)
}
pub fn string_type() -> Self {
Self::abstract_with_constraint(Constraint::Type(PrimitiveType::String))
}
pub fn int_type() -> Self {
Self::abstract_with_constraint(Constraint::Type(PrimitiveType::Int))
}
pub fn float_type() -> Self {
Self::abstract_with_constraint(Constraint::Type(PrimitiveType::Float))
}
pub fn bool_type() -> Self {
Self::abstract_with_constraint(Constraint::Type(PrimitiveType::Bool))
}
pub fn builtin_predicate(name: impl Into<String>) -> Self {
Self::abstract_with_constraint(Constraint::BuiltinPredicate(name.into()))
}
pub fn abstract_with_constraint(constraint: Constraint) -> Self {
Self::Abstract {
constraints: alloc::vec![constraint],
default: None,
}
}
pub fn with_constraint(mut self, constraint: Constraint) -> Self {
match &mut self {
Self::Abstract { constraints, .. } => constraints.push(constraint),
_ => {
self = Self::Abstract {
constraints: alloc::vec![constraint],
default: Some(Box::new(self)),
};
}
}
self
}
pub fn gt(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(CompareOp::Gt, LiteralValue::Int(value)))
}
pub fn gte(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(
CompareOp::Gte,
LiteralValue::Int(value),
))
}
pub fn lt(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(CompareOp::Lt, LiteralValue::Int(value)))
}
pub fn lte(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(
CompareOp::Lte,
LiteralValue::Int(value),
))
}
pub fn default(self, value: HostValue) -> Result<Self> {
match self {
Self::Abstract {
constraints,
default: None,
} => Ok(Self::Abstract {
constraints,
default: Some(Box::new(value)),
}),
Self::Abstract { .. } => Err(Diagnostic::new(
DiagnosticKind::DefaultConflict,
Span::default(),
"host value already has a default",
)),
concrete => Ok(Self::Abstract {
constraints: Vec::new(),
default: Some(Box::new(concrete)),
}),
}
}
pub fn default_string(self, value: impl Into<String>) -> Result<Self> {
self.default(Self::string(value))
}
pub fn default_int(self, value: i64) -> Result<Self> {
self.default(Self::int(value))
}
pub fn default_float(self, value: f64) -> Result<Self> {
self.default(Self::float(value))
}
pub fn default_bool(self, value: bool) -> Result<Self> {
self.default(Self::bool(value))
}
}
+34
View File
@@ -0,0 +1,34 @@
use crate::{Engine, ImportLoader, Result};
/// Host-owned configuration shared by runtime and language tooling.
///
/// Implementations create the import loader and install global bindings on a
/// fresh engine. Calling [`HostEnvironment::create_engine`] therefore gives
/// every consumer the same Decodal execution environment.
pub trait HostEnvironment {
type Loader: ImportLoader;
fn create_loader(&self) -> Self::Loader;
fn configure_engine(&self, _engine: &mut Engine<Self::Loader>) -> Result<()> {
Ok(())
}
fn create_engine(&self) -> Result<Engine<Self::Loader>> {
let mut engine = Engine::new(self.create_loader());
self.configure_engine(&mut engine)?;
Ok(engine)
}
}
impl<T: HostEnvironment + ?Sized> HostEnvironment for &T {
type Loader = T::Loader;
fn create_loader(&self) -> Self::Loader {
T::create_loader(*self)
}
fn configure_engine(&self, engine: &mut Engine<Self::Loader>) -> Result<()> {
T::configure_engine(*self, engine)
}
}
File diff suppressed because it is too large Load Diff
+101 -18
View File
@@ -22,6 +22,7 @@ pub enum TokenKind {
Match,
Import,
Default,
As,
Underscore,
LBrace,
RBrace,
@@ -32,6 +33,7 @@ pub enum TokenKind {
Semicolon,
Comma,
Dot,
Ellipsis,
Colon,
Equal,
EqualEqual,
@@ -51,9 +53,25 @@ pub enum TokenKind {
Gte,
Lt,
Lte,
/// A line comment, including its leading `#` and excluding its newline.
Comment,
Eof,
}
/// Tokenizes source while retaining comments and source spans for tooling.
///
/// Whitespace remains available through the gaps between adjacent token spans,
/// making this a lossless syntax view when paired with the original source.
pub fn tokenize_source(source: &str) -> Result<Vec<Token>> {
tokenize_source_with_source_id(SourceId(0), source)
}
/// Tokenizes source with a caller-provided source identifier while retaining
/// comments and source spans for tooling.
pub fn tokenize_source_with_source_id(source_id: SourceId, source: &str) -> Result<Vec<Token>> {
Lexer::with_source_id(source_id, source).tokenize_with_comments()
}
pub struct Lexer<'a> {
source_id: SourceId,
source: &'a str,
@@ -62,6 +80,7 @@ pub struct Lexer<'a> {
}
impl<'a> Lexer<'a> {
#[cfg(test)]
pub fn new(source: &'a str) -> Self {
Self::with_source_id(SourceId(0), source)
}
@@ -76,10 +95,26 @@ impl<'a> Lexer<'a> {
}
pub fn tokenize(mut self) -> Result<Vec<Token>> {
self.tokenize_impl(false)
}
fn tokenize_with_comments(mut self) -> Result<Vec<Token>> {
self.tokenize_impl(true)
}
fn tokenize_impl(&mut self, include_comments: bool) -> Result<Vec<Token>> {
let mut tokens = Vec::new();
let mut previous = None;
loop {
let token = self.next_token(previous.as_ref())?;
self.skip_whitespace();
if self.peek() == Some(b'#') {
let comment = self.lex_comment();
if include_comments {
tokens.push(comment);
}
continue;
}
let token = self.next_non_ws_token(previous.as_ref())?;
let is_eof = token.kind == TokenKind::Eof;
if !is_eof {
previous = Some(token.kind.clone());
@@ -91,8 +126,7 @@ impl<'a> Lexer<'a> {
}
}
fn next_token(&mut self, previous: Option<&TokenKind>) -> Result<Token> {
self.skip_ws_and_comments();
fn next_non_ws_token(&mut self, previous: Option<&TokenKind>) -> Result<Token> {
let start = self.pos;
let Some(ch) = self.peek() else {
return Ok(Token {
@@ -136,7 +170,18 @@ impl<'a> Lexer<'a> {
}
b'.' => {
self.pos += 1;
TokenKind::Dot
if self.consume(b'.') {
if self.consume(b'.') {
TokenKind::Ellipsis
} else {
return Err(Diagnostic::syntax(
self.span(start, self.pos),
"expected third '.' in ellipsis",
));
}
} else {
TokenKind::Dot
}
}
b':' => {
self.pos += 1;
@@ -242,21 +287,23 @@ impl<'a> Lexer<'a> {
})
}
fn skip_ws_and_comments(&mut self) {
loop {
while matches!(self.peek(), Some(b' ' | b'\t' | b'\r' | b'\n')) {
self.pos += 1;
fn skip_whitespace(&mut self) {
while matches!(self.peek(), Some(b' ' | b'\t' | b'\r' | b'\n')) {
self.pos += 1;
}
}
fn lex_comment(&mut self) -> Token {
let start = self.pos;
while let Some(c) = self.peek() {
if c == b'\n' {
break;
}
if self.peek() == Some(b'#') {
while let Some(c) = self.peek() {
self.pos += 1;
if c == b'\n' {
break;
}
}
continue;
}
break;
self.pos += 1;
}
Token {
kind: TokenKind::Comment,
span: self.span(start, self.pos),
}
}
@@ -360,6 +407,7 @@ impl<'a> Lexer<'a> {
"match" => TokenKind::Match,
"import" => TokenKind::Import,
"default" => TokenKind::Default,
"as" => TokenKind::As,
_ => TokenKind::Ident(String::from(text)),
}
}
@@ -427,4 +475,39 @@ mod tests {
assert_eq!(tokens[3].kind, TokenKind::Amp);
assert_eq!(tokens[4].kind, TokenKind::Gte);
}
#[test]
fn tokenizes_array_constraint_ellipsis() {
let tokens = Lexer::new("[...String]").tokenize().unwrap();
assert_eq!(tokens[0].kind, TokenKind::LBracket);
assert_eq!(tokens[1].kind, TokenKind::Ellipsis);
assert!(matches!(tokens[2].kind, TokenKind::Ident(_)));
assert_eq!(tokens[3].kind, TokenKind::RBracket);
}
#[test]
fn tooling_tokens_retain_comments_and_whitespace_gaps() {
let source = "value = 1; # trailing\n\n# leading\nnext = 2;";
let tokens = tokenize_source(source).unwrap();
let comments = tokens
.iter()
.filter(|token| token.kind == TokenKind::Comment)
.collect::<Vec<_>>();
assert_eq!(comments.len(), 2);
assert_eq!(
&source[comments[0].span.start as usize..comments[0].span.end as usize],
"# trailing"
);
assert!(
source[comments[0].span.end as usize..comments[1].span.start as usize].contains("\n\n")
);
let source_id = SourceId(7);
let identified = tokenize_source_with_source_id(source_id, "# comment").unwrap();
assert!(
identified
.iter()
.all(|token| token.span.source == source_id)
);
}
}
+19 -7
View File
@@ -5,24 +5,36 @@ extern crate alloc;
pub mod ast;
pub mod constraints;
pub mod diagnostic;
pub mod embedding;
pub mod environment;
pub mod eval;
pub mod lexer;
mod lexer;
pub mod module;
pub mod parser;
pub mod runtime;
pub mod span;
pub mod typed;
pub mod value;
pub use ast::{Ast, BinaryOp, CompareOp, Expr, ExprId, Field, Literal, Param};
pub use ast::{Ast, BinaryOp, CompareOp, Expr, ExprId, Field, Literal, ObjectRest, Param};
pub use constraints::normalize_constraints;
#[cfg(feature = "derive")]
pub use decodal_derive::Decodal;
pub use diagnostic::{Diagnostic, DiagnosticKind, Result};
pub use embedding::{HostField, HostValue};
pub use environment::HostEnvironment;
pub use eval::{Engine, format_diagnostic_with};
pub use lexer::{Lexer, Token, TokenKind};
pub use module::{EmptyLoader, LoadedSource, Module, SourceLoader};
pub use lexer::{
Token as SyntaxToken, TokenKind as SyntaxTokenKind, tokenize_source,
tokenize_source_with_source_id,
};
pub use module::{EmptyLoader, ImportCandidate, ImportLoader, LoadedImport, Module};
pub use parser::{ParseOutput, Parser, SourceForm, parse_source, parse_source_with_source_id};
pub use runtime::{Constraint, Data, ExprRef, LiteralValue, ModuleId, PrimitiveType, RuntimeValue};
pub use runtime::{Constraint, Data, ExprRef, LiteralValue, ModuleId, PrimitiveType};
pub use span::{SourceId, Span};
pub use typed::{
DecodalDecode, DecodalRest, DecodalSchema, DecodeError, DecodeResult, IntoValue, data_at_path,
decode_path, prefix_decode_error,
};
pub use value::{Value, ValueField};
pub fn version() -> &'static str {
env!("CARGO_PKG_VERSION")
+72 -10
View File
@@ -1,7 +1,7 @@
use alloc::string::String;
use alloc::{string::String, vec::Vec};
use crate::{
Ast, ExprId, SourceForm, SourceId,
Ast, ExprId, SourceForm, SourceId, Value,
runtime::{EnvId, ThunkId},
};
@@ -17,22 +17,84 @@ pub struct Module {
pub root_thunk: ThunkId,
}
/// Content resolved by an [`ImportLoader`].
#[derive(Debug, Clone)]
pub struct LoadedSource {
pub key: String,
pub name: String,
pub source: String,
pub enum LoadedImport {
Source {
key: String,
name: String,
source: String,
},
Value {
key: String,
value: Value,
},
}
pub trait SourceLoader {
fn load(&mut self, current_key: Option<&str>, specifier: &str) -> crate::Result<LoadedSource>;
/// An import specifier offered by host-owned language tooling.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ImportCandidate {
pub specifier: String,
pub detail: Option<String>,
}
impl ImportCandidate {
pub fn new(specifier: impl Into<String>) -> Self {
Self {
specifier: specifier.into(),
detail: None,
}
}
pub fn with_detail(mut self, detail: impl Into<String>) -> Self {
self.detail = Some(detail.into());
self
}
}
impl LoadedImport {
pub fn source(
key: impl Into<String>,
name: impl Into<String>,
source: impl Into<String>,
) -> Self {
Self::Source {
key: key.into(),
name: name.into(),
source: source.into(),
}
}
pub fn value(key: impl Into<String>, value: Value) -> Self {
Self::Value {
key: key.into(),
value,
}
}
}
pub trait ImportLoader {
fn load(&mut self, current_key: Option<&str>, specifier: &str) -> crate::Result<LoadedImport>;
/// Returns host-resolvable import specifiers matching an unfinished prefix.
///
/// Runtime-only loaders may keep the default implementation. Hosts that
/// provide an editor should implement this from the same namespace used by
/// [`ImportLoader::load`].
fn complete_import(
&mut self,
_current_key: Option<&str>,
_prefix: &str,
) -> crate::Result<Vec<ImportCandidate>> {
Ok(Vec::new())
}
}
#[derive(Debug, Clone, Copy, Default)]
pub struct EmptyLoader;
impl SourceLoader for EmptyLoader {
fn load(&mut self, _current_key: Option<&str>, specifier: &str) -> crate::Result<LoadedSource> {
impl ImportLoader for EmptyLoader {
fn load(&mut self, _current_key: Option<&str>, specifier: &str) -> crate::Result<LoadedImport> {
Err(crate::Diagnostic::new(
crate::DiagnosticKind::Import,
crate::Span::default(),
+127 -11
View File
@@ -2,7 +2,10 @@ use alloc::{string::String, vec::Vec};
use crate::{
SourceId, Span,
ast::{Ast, BinaryOp, CompareOp, Expr, ExprId, Field, Literal, MatchArm, Param, UnaryOp},
ast::{
Ast, BinaryOp, CompareOp, Expr, ExprId, Field, Literal, MatchArm, ObjectRest, Param,
UnaryOp,
},
diagnostic::{Diagnostic, Result},
lexer::{Lexer, Token, TokenKind},
};
@@ -38,7 +41,10 @@ pub struct Parser {
impl Parser {
pub fn new(tokens: Vec<Token>) -> Self {
Self {
tokens,
tokens: tokens
.into_iter()
.filter(|token| token.kind != TokenKind::Comment)
.collect(),
pos: 0,
ast: Ast::new(),
}
@@ -56,7 +62,7 @@ impl Parser {
})
.unwrap_or_else(|| self.peek().span);
(
self.ast.push(Expr::Object(fields), span),
self.ast.push(Expr::Object { fields, rest: None }, span),
SourceForm::Fields,
)
} else {
@@ -242,6 +248,13 @@ impl Parser {
},
span,
),
InfixKind::As => self.ast.push(
Expr::As {
narrower: lhs,
wider: rhs,
},
span,
),
};
}
@@ -316,28 +329,64 @@ impl Parser {
}
fn parse_object_after_lbrace(&mut self, start_span: Span) -> Result<ExprId> {
if self.consume_kind(&TokenKind::Ellipsis).is_some() {
let value = self.parse_expr(0)?;
self.expect_kind(
&TokenKind::RBrace,
"expected '}' after map value constraint",
)?;
let span = start_span.join(self.previous_span());
return Ok(self.ast.push(Expr::MapConstraint { value }, span));
}
let mut fields = Vec::new();
if self.consume_kind(&TokenKind::RBrace).is_some() {
return Ok(self
.ast
.push(Expr::Object(fields), start_span.join(self.previous_span())));
return Ok(self.ast.push(
Expr::Object { fields, rest: None },
start_span.join(self.previous_span()),
));
}
let mut rest = None;
loop {
fields.push(self.parse_field()?);
if self.consume_kind(&TokenKind::Semicolon).is_some() {
if self.consume_kind(&TokenKind::RBrace).is_some() {
break;
}
if let Some(ellipsis) = self.consume_kind(&TokenKind::Ellipsis) {
let value = self.parse_expr(0)?;
rest = Some(ObjectRest {
value,
span: ellipsis.join(self.ast.span(value)),
});
let _ = self.consume_kind(&TokenKind::Semicolon);
self.expect_kind(
&TokenKind::RBrace,
"expected '}' after object rest constraint",
)?;
break;
}
continue;
}
self.expect_kind(&TokenKind::RBrace, "expected ';' or '}' after object field")?;
break;
}
let span = start_span.join(self.previous_span());
Ok(self.ast.push(Expr::Object(fields), span))
Ok(self.ast.push(Expr::Object { fields, rest }, span))
}
fn parse_array_after_lbracket(&mut self, start_span: Span) -> Result<ExprId> {
if self.consume_kind(&TokenKind::Ellipsis).is_some() {
let item = self.parse_expr(0)?;
let _ = self.consume_kind(&TokenKind::Comma);
self.expect_kind(
&TokenKind::RBracket,
"expected ']' after array element constraint",
)?;
let span = start_span.join(self.previous_span());
return Ok(self.ast.push(Expr::ArrayConstraint { item }, span));
}
let mut items = Vec::new();
if self.consume_kind(&TokenKind::RBracket).is_some() {
return Ok(self
@@ -369,8 +418,9 @@ impl Parser {
}
let expr = self.parse_expr(0)?;
self.expect_kind(&TokenKind::RParen, "expected ')' after expression")?;
Ok(expr)
let end_span = self.expect_kind(&TokenKind::RParen, "expected ')' after expression")?;
let span = start_span.join(end_span);
Ok(self.ast.push(Expr::Parenthesized { expr }, span))
}
fn parse_params_after_lparen(&mut self) -> Result<Vec<Param>> {
@@ -529,6 +579,7 @@ impl Parser {
fn peek_infix(&self) -> Option<(InfixKind, u8, u8)> {
match self.peek_kind() {
TokenKind::As => Some((InfixKind::As, 0, 1)),
TokenKind::Default => Some((InfixKind::Default, 1, 2)),
TokenKind::SlashSlash => Some((InfixKind::Patch, 3, 4)),
TokenKind::Amp => Some((InfixKind::And, 5, 6)),
@@ -637,6 +688,7 @@ impl Parser {
#[derive(Debug, Clone, Copy)]
enum InfixKind {
As,
Add,
Sub,
Mul,
@@ -663,15 +715,79 @@ mod tests {
#[test]
fn parses_top_level_fields_as_object() {
let parsed = parse_source("port = Int & >= 1 default 8080;").unwrap();
assert!(matches!(parsed.ast.get(parsed.root).expr, Expr::Object(_)));
assert!(matches!(
parsed.ast.get(parsed.root).expr,
Expr::Object { .. }
));
}
#[test]
fn parses_object_dot_field() {
let parsed = parse_source("{ feature.enable = false; }").unwrap();
let Expr::Object(fields) = &parsed.ast.get(parsed.root).expr else {
let Expr::Object { fields, .. } = &parsed.ast.get(parsed.root).expr else {
panic!()
};
assert_eq!(fields[0].path, ["feature", "enable"]);
}
#[test]
fn parses_array_constraint() {
let parsed = parse_source("[...(String & /^api-/)]").unwrap();
let Expr::ArrayConstraint { item } = parsed.ast.get(parsed.root).expr else {
panic!()
};
let Expr::Parenthesized { expr } = parsed.ast.get(item).expr else {
panic!()
};
assert!(matches!(parsed.ast.get(expr).expr, Expr::Binary { .. }));
assert_eq!(parsed.ast.span(item), Span::new(SourceId(0), 4, 22));
}
#[test]
fn parser_accepts_the_public_lossless_token_stream() {
let tokens = crate::tokenize_source("value = (# note\n 1);").unwrap();
let parsed = Parser::new(tokens).parse().unwrap();
let Expr::Object { fields, .. } = &parsed.ast.get(parsed.root).expr else {
panic!()
};
assert!(matches!(
parsed.ast.get(fields[0].value).expr,
Expr::Parenthesized { .. }
));
}
#[test]
fn rejects_array_constraint_without_element_constraint() {
assert!(parse_source("[...]").is_err());
}
#[test]
fn parses_map_constraint() {
let parsed = parse_source("{...String}").unwrap();
let Expr::MapConstraint { value } = parsed.ast.get(parsed.root).expr else {
panic!()
};
assert!(matches!(parsed.ast.get(value).expr, Expr::Ident(_)));
}
#[test]
fn parses_object_rest_constraint_after_named_fields() {
let parsed = parse_source("{ hoge = Int; fuga = String; ...Unknown }").unwrap();
let Expr::Object { fields, rest } = &parsed.ast.get(parsed.root).expr else {
panic!()
};
assert_eq!(fields.len(), 2);
let rest = rest.as_ref().expect("object has a rest constraint");
assert!(matches!(parsed.ast.get(rest.value).expr, Expr::Ident(_)));
}
#[test]
fn parses_as_below_composition() {
let parsed = parse_source("value & override as Schema").unwrap();
let Expr::As { narrower, wider } = parsed.ast.get(parsed.root).expr else {
panic!()
};
assert!(matches!(parsed.ast.get(narrower).expr, Expr::Binary { .. }));
assert!(matches!(parsed.ast.get(wider).expr, Expr::Ident(_)));
}
}
+14
View File
@@ -11,6 +11,9 @@ pub struct ExprRef {
pub expr: ExprId,
}
/// Evaluator-owned representation of a lazy concrete value or unresolved range.
///
/// Embedders normally construct [`crate::Value`] and consume [`Data`] instead.
#[derive(Debug, Clone, PartialEq)]
pub enum RuntimeValue {
Concrete(ConcreteValue),
@@ -31,6 +34,13 @@ pub enum ConcreteValue {
#[derive(Debug, Clone, PartialEq)]
pub struct ObjectValue {
pub fields: Vec<ObjectField>,
pub rest: Option<ObjectRest>,
}
#[derive(Debug, Clone, PartialEq)]
pub struct ObjectRest {
pub value: ThunkId,
pub span: Span,
}
#[derive(Debug, Clone, PartialEq)]
@@ -67,7 +77,10 @@ pub struct ConstraintEntry {
#[derive(Debug, Clone, PartialEq)]
pub enum Constraint {
Unknown,
Type(PrimitiveType),
ArrayItems(ThunkId),
MapValues(ThunkId),
Compare(CompareOp, LiteralValue),
Regex(String),
BuiltinPredicate(String),
@@ -89,6 +102,7 @@ pub enum LiteralValue {
Bool(bool),
}
/// Fully materialized Decodal data.
#[derive(Debug, Clone, PartialEq)]
pub enum Data {
String(String),
+445
View File
@@ -0,0 +1,445 @@
use alloc::{collections::BTreeMap, format, string::String, vec::Vec};
use crate::{Data, Value, runtime::DataField};
pub trait DecodalSchema {
fn decodal_schema() -> Value;
}
pub trait DecodalDecode: Sized {
fn decodal_decode(data: &Data) -> DecodeResult<Self>;
}
pub trait IntoValue {
fn into_value(self) -> Value;
}
/// A map-like receiver for fields not named by a derived struct schema.
pub trait DecodalRest: Sized {
fn decodal_rest_schema() -> Value;
fn decodal_decode_rest(data: &Data, known_fields: &[&str]) -> DecodeResult<Self>;
}
pub type DecodeResult<T> = core::result::Result<T, DecodeError>;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DecodeError {
pub path: String,
pub message: String,
}
impl DecodeError {
pub fn new(path: impl Into<String>, message: impl Into<String>) -> Self {
Self {
path: path.into(),
message: message.into(),
}
}
pub fn at_type(path: &str, expected: &'static str) -> Self {
Self::new(path, format!("expected {expected}"))
}
}
impl core::fmt::Display for DecodeError {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
if self.path.is_empty() {
write!(f, "{}", self.message)
} else {
write!(f, "{}: {}", self.path, self.message)
}
}
}
#[cfg(feature = "std")]
impl std::error::Error for DecodeError {}
pub fn decode_path<T: DecodalDecode>(data: &Data, path: &str) -> DecodeResult<T> {
let value = data_at_path(data, path).ok_or_else(|| DecodeError::new(path, "missing field"))?;
T::decodal_decode(value).map_err(|error| prefix_error(path, error))
}
pub fn data_at_path<'a>(data: &'a Data, path: &str) -> Option<&'a Data> {
let mut current = data;
for part in path.split('.') {
if part.is_empty() {
return None;
}
let Data::Object(fields) = current else {
return None;
};
current = &fields.iter().find(|field| field.name == part)?.value;
}
Some(current)
}
pub fn prefix_decode_error(path: &str, error: DecodeError) -> DecodeError {
prefix_error(path, error)
}
fn prefix_error(path: &str, mut error: DecodeError) -> DecodeError {
if error.path.is_empty() {
error.path = String::from(path);
} else if !path.is_empty() {
error.path = format!("{path}.{}", error.path);
}
error
}
impl DecodalSchema for Data {
fn decodal_schema() -> Value {
Value::unknown()
}
}
impl DecodalDecode for Data {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
Ok(data.clone())
}
}
impl IntoValue for Data {
fn into_value(self) -> Value {
match self {
Data::String(value) => Value::string(value),
Data::Int(value) => Value::int(value),
Data::Float(value) => Value::float(value),
Data::Bool(value) => Value::bool(value),
Data::Array(items) => Value::array(items.into_iter().map(IntoValue::into_value)),
Data::Object(fields) => Value::object(
fields
.into_iter()
.map(|field| (field.name, field.value.into_value())),
),
}
}
}
impl DecodalSchema for String {
fn decodal_schema() -> Value {
Value::string_type()
}
}
impl DecodalDecode for String {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::String(value) => Ok(value.clone()),
_ => Err(DecodeError::at_type("", "String")),
}
}
}
impl IntoValue for String {
fn into_value(self) -> Value {
Value::string(self)
}
}
impl IntoValue for &str {
fn into_value(self) -> Value {
Value::string(self)
}
}
macro_rules! impl_int_decode {
($($ty:ty),* $(,)?) => {
$(
impl DecodalSchema for $ty {
fn decodal_schema() -> Value {
Value::int_type()
}
}
impl DecodalDecode for $ty {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Int(value) => <$ty>::try_from(*value)
.map_err(|_| DecodeError::new("", "integer value is out of range")),
_ => Err(DecodeError::at_type("", "Int")),
}
}
}
impl IntoValue for $ty {
fn into_value(self) -> Value {
Value::int(self as i64)
}
}
)*
};
}
impl_int_decode!(i8, i16, i32, i64, u8, u16, u32);
impl DecodalSchema for f64 {
fn decodal_schema() -> Value {
Value::float_type()
}
}
impl DecodalDecode for f64 {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Float(value) => Ok(*value),
Data::Int(value) => Ok(*value as f64),
_ => Err(DecodeError::at_type("", "Float")),
}
}
}
impl IntoValue for f64 {
fn into_value(self) -> Value {
Value::float(self)
}
}
impl DecodalSchema for f32 {
fn decodal_schema() -> Value {
Value::float_type()
}
}
impl DecodalDecode for f32 {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
f64::decodal_decode(data).map(|value| value as f32)
}
}
impl IntoValue for f32 {
fn into_value(self) -> Value {
Value::float(f64::from(self))
}
}
impl DecodalSchema for bool {
fn decodal_schema() -> Value {
Value::bool_type()
}
}
impl DecodalDecode for bool {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Bool(value) => Ok(*value),
_ => Err(DecodeError::at_type("", "Bool")),
}
}
}
impl IntoValue for bool {
fn into_value(self) -> Value {
Value::bool(self)
}
}
impl<T: DecodalSchema> DecodalSchema for Vec<T> {
fn decodal_schema() -> Value {
Value::array_of(T::decodal_schema())
}
}
impl<T: DecodalDecode> DecodalDecode for Vec<T> {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Array(items) => items
.iter()
.enumerate()
.map(|(index, item)| {
T::decodal_decode(item)
.map_err(|error| prefix_error(&format!("[{index}]"), error))
})
.collect(),
_ => Err(DecodeError::at_type("", "Array")),
}
}
}
impl<T: IntoValue> IntoValue for Vec<T> {
fn into_value(self) -> Value {
Value::array(self.into_iter().map(IntoValue::into_value))
}
}
impl<T: DecodalSchema> DecodalSchema for BTreeMap<String, T> {
fn decodal_schema() -> Value {
Value::map_of(T::decodal_schema())
}
}
impl<T: DecodalDecode> DecodalDecode for BTreeMap<String, T> {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Object(fields) => fields
.iter()
.map(|field| {
T::decodal_decode(&field.value)
.map(|value| (field.name.clone(), value))
.map_err(|error| prefix_error(&field.name, error))
})
.collect(),
_ => Err(DecodeError::at_type("", "Object")),
}
}
}
impl<T: IntoValue> IntoValue for BTreeMap<String, T> {
fn into_value(self) -> Value {
Value::object(
self.into_iter()
.map(|(name, value)| (name, value.into_value())),
)
}
}
impl<T: DecodalSchema + DecodalDecode> DecodalRest for BTreeMap<String, T> {
fn decodal_rest_schema() -> Value {
T::decodal_schema()
}
fn decodal_decode_rest(data: &Data, known_fields: &[&str]) -> DecodeResult<Self> {
let Data::Object(fields) = data else {
return Err(DecodeError::at_type("", "Object"));
};
fields
.iter()
.filter(|field| !known_fields.contains(&field.name.as_str()))
.map(decode_rest_field::<T>)
.collect()
}
}
#[cfg(feature = "std")]
impl<T: DecodalSchema> DecodalSchema for std::collections::HashMap<String, T> {
fn decodal_schema() -> Value {
Value::map_of(T::decodal_schema())
}
}
#[cfg(feature = "std")]
impl<T: DecodalDecode> DecodalDecode for std::collections::HashMap<String, T> {
fn decodal_decode(data: &Data) -> DecodeResult<Self> {
match data {
Data::Object(fields) => fields
.iter()
.map(|field| {
T::decodal_decode(&field.value)
.map(|value| (field.name.clone(), value))
.map_err(|error| prefix_error(&field.name, error))
})
.collect(),
_ => Err(DecodeError::at_type("", "Object")),
}
}
}
#[cfg(feature = "std")]
impl<T: IntoValue> IntoValue for std::collections::HashMap<String, T> {
fn into_value(self) -> Value {
let mut fields: Vec<_> = self
.into_iter()
.map(|(name, value)| (name, value.into_value()))
.collect();
fields.sort_by(|left, right| left.0.cmp(&right.0));
Value::object(fields)
}
}
#[cfg(feature = "std")]
impl<T: DecodalSchema + DecodalDecode> DecodalRest for std::collections::HashMap<String, T> {
fn decodal_rest_schema() -> Value {
T::decodal_schema()
}
fn decodal_decode_rest(data: &Data, known_fields: &[&str]) -> DecodeResult<Self> {
let Data::Object(fields) = data else {
return Err(DecodeError::at_type("", "Object"));
};
fields
.iter()
.filter(|field| !known_fields.contains(&field.name.as_str()))
.map(decode_rest_field::<T>)
.collect()
}
}
fn decode_rest_field<T: DecodalDecode>(field: &DataField) -> DecodeResult<(String, T)> {
T::decodal_decode(&field.value)
.map(|value| (field.name.clone(), value))
.map_err(|error| prefix_error(&field.name, error))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::runtime::{Constraint, DataField, PrimitiveType};
#[test]
fn string_maps_expose_map_schemas_and_decode_objects() {
assert!(matches!(
BTreeMap::<String, i64>::decodal_schema(),
Value::MapRange { value, .. }
if matches!(
*value,
Value::Range { ref constraints, .. }
if constraints == &[Constraint::Type(PrimitiveType::Int)]
)
));
let data = Data::Object(alloc::vec![DataField {
name: String::from("api"),
value: Data::Int(8080),
}]);
let decoded = BTreeMap::<String, i64>::decodal_decode(&data).unwrap();
assert_eq!(decoded.get("api"), Some(&8080));
}
#[test]
fn data_is_the_unknown_schema_and_round_trips_values() {
assert!(matches!(
Data::decodal_schema(),
Value::Range { ref constraints, .. }
if constraints == &[Constraint::Unknown]
));
let data = Data::Object(alloc::vec![DataField {
name: String::from("nested"),
value: Data::Array(alloc::vec![Data::Bool(true), Data::Int(2)]),
}]);
assert_eq!(Data::decodal_decode(&data).unwrap(), data);
assert!(matches!(data.into_value(), Value::Object { .. }));
}
#[test]
fn rest_maps_decode_only_fields_outside_the_known_domain() {
let data = Data::Object(alloc::vec![
DataField {
name: String::from("name"),
value: Data::String(String::from("api")),
},
DataField {
name: String::from("priority"),
value: Data::Int(3),
},
]);
let decoded = BTreeMap::<String, Data>::decodal_decode_rest(&data, &["name"]).unwrap();
assert_eq!(decoded.len(), 1);
assert_eq!(decoded.get("priority"), Some(&Data::Int(3)));
}
#[cfg(feature = "std")]
#[test]
fn hash_maps_can_receive_rest_fields() {
let data = Data::Object(alloc::vec![DataField {
name: String::from("region"),
value: Data::String(String::from("ap-northeast-1")),
}]);
let decoded =
std::collections::HashMap::<String, String>::decodal_decode_rest(&data, &[]).unwrap();
assert_eq!(
decoded.get("region").map(String::as_str),
Some("ap-northeast-1")
);
}
}
+333
View File
@@ -0,0 +1,333 @@
use alloc::{boxed::Box, string::String, vec::Vec};
use crate::runtime::{Constraint, LiteralValue, PrimitiveType};
use crate::{CompareOp, Diagnostic, DiagnosticKind, Result, Span};
/// A Decodal value supplied through the embedding API.
///
/// Unlike [`crate::Data`], a `Value` may contain unresolved ranges and
/// defaults. The evaluator converts it into its lazy runtime representation
/// when it is bound as a global or returned from an import loader.
#[derive(Debug, Clone, PartialEq)]
pub enum Value {
String(String),
Int(i64),
Float(f64),
Bool(bool),
Array(Vec<Value>),
ArrayRange {
item: Box<Value>,
constraints: Vec<Constraint>,
default: Option<Box<Value>>,
},
MapRange {
value: Box<Value>,
constraints: Vec<Constraint>,
default: Option<Box<Value>>,
},
Object {
fields: Vec<ValueField>,
rest: Option<Box<Value>>,
},
Range {
constraints: Vec<Constraint>,
default: Option<Box<Value>>,
},
}
/// A named field in a [`Value::Object`].
#[derive(Debug, Clone, PartialEq)]
pub struct ValueField {
pub name: String,
pub value: Value,
}
impl Value {
pub fn string(value: impl Into<String>) -> Self {
Self::String(value.into())
}
pub fn int(value: i64) -> Self {
Self::Int(value)
}
pub fn float(value: f64) -> Self {
Self::Float(value)
}
pub fn bool(value: bool) -> Self {
Self::Bool(value)
}
pub fn array<I>(items: I) -> Self
where
I: IntoIterator<Item = Value>,
{
Self::Array(items.into_iter().collect())
}
pub fn object<I, N>(fields: I) -> Self
where
I: IntoIterator<Item = (N, Value)>,
N: Into<String>,
{
Self::Object {
fields: fields
.into_iter()
.map(|(name, value)| ValueField {
name: name.into(),
value,
})
.collect(),
rest: None,
}
}
pub fn object_from_paths<I, N>(fields: I) -> Self
where
I: IntoIterator<Item = (N, Value)>,
N: Into<String>,
{
let mut root = Vec::new();
for (path, value) in fields {
insert_path(&mut root, &path.into(), value);
}
Self::Object {
fields: root,
rest: None,
}
}
pub fn object_from_paths_with_rest<I, N>(fields: I, rest: Value) -> Self
where
I: IntoIterator<Item = (N, Value)>,
N: Into<String>,
{
let mut root = Vec::new();
for (path, value) in fields {
insert_path(&mut root, &path.into(), value);
}
Self::Object {
fields: root,
rest: Some(Box::new(rest)),
}
}
pub fn object_with_rest<I, N>(fields: I, rest: Value) -> Self
where
I: IntoIterator<Item = (N, Value)>,
N: Into<String>,
{
Self::Object {
fields: fields
.into_iter()
.map(|(name, value)| ValueField {
name: name.into(),
value,
})
.collect(),
rest: Some(Box::new(rest)),
}
}
pub fn unknown() -> Self {
Self::range_with_constraint(Constraint::Unknown)
}
pub fn string_type() -> Self {
Self::range_with_constraint(Constraint::Type(PrimitiveType::String))
}
pub fn int_type() -> Self {
Self::range_with_constraint(Constraint::Type(PrimitiveType::Int))
}
pub fn float_type() -> Self {
Self::range_with_constraint(Constraint::Type(PrimitiveType::Float))
}
pub fn bool_type() -> Self {
Self::range_with_constraint(Constraint::Type(PrimitiveType::Bool))
}
pub fn array_of(item: Value) -> Self {
Self::ArrayRange {
item: Box::new(item),
constraints: Vec::new(),
default: None,
}
}
pub fn map_of(value: Value) -> Self {
Self::MapRange {
value: Box::new(value),
constraints: Vec::new(),
default: None,
}
}
pub fn builtin_predicate(name: impl Into<String>) -> Self {
Self::range_with_constraint(Constraint::BuiltinPredicate(name.into()))
}
pub fn range_with_constraint(constraint: Constraint) -> Self {
Self::Range {
constraints: alloc::vec![constraint],
default: None,
}
}
pub fn with_constraint(mut self, constraint: Constraint) -> Self {
match &mut self {
Self::Range { constraints, .. } => constraints.push(constraint),
Self::ArrayRange { constraints, .. } | Self::MapRange { constraints, .. } => {
constraints.push(constraint)
}
_ => {
self = Self::Range {
constraints: alloc::vec![constraint],
default: Some(Box::new(self)),
};
}
}
self
}
pub fn gt(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(CompareOp::Gt, LiteralValue::Int(value)))
}
pub fn gte(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(
CompareOp::Gte,
LiteralValue::Int(value),
))
}
pub fn lt(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(CompareOp::Lt, LiteralValue::Int(value)))
}
pub fn lte(self, value: i64) -> Self {
self.with_constraint(Constraint::Compare(
CompareOp::Lte,
LiteralValue::Int(value),
))
}
pub fn default(self, value: Value) -> Result<Self> {
match self {
Self::ArrayRange {
item,
constraints,
default: None,
} => Ok(Self::ArrayRange {
item,
constraints,
default: Some(Box::new(value)),
}),
Self::ArrayRange {
default: Some(_), ..
} => Err(Diagnostic::new(
DiagnosticKind::DefaultConflict,
Span::default(),
"value already has a default",
)),
Self::MapRange {
value: map_value,
constraints,
default: None,
} => Ok(Self::MapRange {
value: map_value,
constraints,
default: Some(Box::new(value)),
}),
Self::MapRange {
default: Some(_), ..
} => Err(Diagnostic::new(
DiagnosticKind::DefaultConflict,
Span::default(),
"value already has a default",
)),
Self::Range {
constraints,
default: None,
} => Ok(Self::Range {
constraints,
default: Some(Box::new(value)),
}),
Self::Range { .. } => Err(Diagnostic::new(
DiagnosticKind::DefaultConflict,
Span::default(),
"value already has a default",
)),
concrete => Ok(Self::Range {
constraints: Vec::new(),
default: Some(Box::new(concrete)),
}),
}
}
pub fn default_string(self, value: impl Into<String>) -> Result<Self> {
self.default(Self::string(value))
}
pub fn default_int(self, value: i64) -> Result<Self> {
self.default(Self::int(value))
}
pub fn default_float(self, value: f64) -> Result<Self> {
self.default(Self::float(value))
}
pub fn default_bool(self, value: bool) -> Result<Self> {
self.default(Self::bool(value))
}
}
impl ValueField {
pub fn new(name: impl Into<String>, value: Value) -> Self {
Self {
name: name.into(),
value,
}
}
}
fn insert_path(fields: &mut Vec<ValueField>, path: &str, value: Value) {
let mut parts = path.splitn(2, '.');
let Some(head) = parts.next().filter(|part| !part.is_empty()) else {
return;
};
if let Some(tail) = parts.next() {
if let Some(field) = fields.iter_mut().find(|field| field.name == head) {
if let Value::Object {
fields: children, ..
} = &mut field.value
{
insert_path(children, tail, value);
} else {
let mut children = Vec::new();
insert_path(&mut children, tail, value);
field.value = Value::Object {
fields: children,
rest: None,
};
}
} else {
let mut children = Vec::new();
insert_path(&mut children, tail, value);
fields.push(ValueField::new(
head,
Value::Object {
fields: children,
rest: None,
},
));
}
} else if let Some(field) = fields.iter_mut().find(|field| field.name == head) {
field.value = value;
} else {
fields.push(ValueField::new(head, value));
}
}
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "decodal-derive"
version = "0.2.0"
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Derive macro for generating Decodal schemas and typed decoders from Rust structs."
keywords = ["decodal", "derive", "proc-macro", "schema"]
categories = ["config", "development-tools::procedural-macro-helpers"]
[lib]
proc-macro = true
[dependencies]
proc-macro2 = "1"
quote = "1"
syn = { version = "2", features = ["full"] }
[dev-dependencies]
decodal = { path = "../decodal-core" }
+313
View File
@@ -0,0 +1,313 @@
use proc_macro::TokenStream;
use proc_macro2::TokenStream as TokenStream2;
use quote::{format_ident, quote};
use syn::{
Data, DeriveInput, Expr, Fields, LitStr, Result, Type, parse_macro_input, spanned::Spanned,
};
#[proc_macro_derive(Decodal, attributes(decodal))]
pub fn derive_decodal(input: TokenStream) -> TokenStream {
match expand_decodal(parse_macro_input!(input as DeriveInput)) {
Ok(tokens) => tokens.into(),
Err(error) => error.to_compile_error().into(),
}
}
fn expand_decodal(input: DeriveInput) -> Result<TokenStream2> {
let name = input.ident;
let fields = match input.data {
Data::Struct(data) => match data.fields {
Fields::Named(fields) => fields.named,
_ => {
return Err(syn::Error::new(
data.struct_token.span(),
"Decodal derive supports structs with named fields only",
));
}
},
_ => {
return Err(syn::Error::new(
name.span(),
"Decodal derive supports structs only",
));
}
};
let mut schema_fields = Vec::new();
let mut decode_fields = Vec::new();
let mut known_field_roots = Vec::new();
let mut rest_field = None;
for field in fields {
let ident = field.ident.expect("named field");
let ty = field.ty;
let attrs = FieldAttrs::from_attrs(&field.attrs, &ident.to_string())?;
if attrs.rest {
if attrs.rename_explicit || attrs.default.is_some() || !attrs.constraints.is_empty() {
return Err(syn::Error::new(
ident.span(),
"`rest` cannot be combined with rename, default, or field constraints",
));
}
if rest_field.is_some() {
return Err(syn::Error::new(
ident.span(),
"Decodal derive supports only one `rest` field",
));
}
rest_field = Some((ident, ty));
continue;
}
let path = attrs.rename.clone();
known_field_roots.push(
path.split('.')
.next()
.expect("field paths are non-empty")
.to_owned(),
);
let schema_value = schema_expr(&ty, &attrs)?;
let decode_value = decode_expr(&ty, &path, &attrs);
schema_fields.push(quote! {
(#path, { #schema_value })
});
decode_fields.push(quote! {
#ident: #decode_value
});
}
if let Some((ident, ty)) = &rest_field {
decode_fields.push(quote! {
#ident: <#ty as ::decodal::DecodalRest>::decodal_decode_rest(
data,
&[#(#known_field_roots),*],
)?
});
}
let schema_entries = if schema_fields.is_empty() {
quote! { ::core::iter::empty::<(&'static str, ::decodal::Value)>() }
} else {
quote! { [#(#schema_fields),*] }
};
let build_schema = if let Some((_, ty)) = &rest_field {
quote! {
::decodal::Value::object_from_paths_with_rest(
#schema_entries,
<#ty as ::decodal::DecodalRest>::decodal_rest_schema(),
)
}
} else {
quote! {
::decodal::Value::object_from_paths(#schema_entries)
}
};
let (impl_generics, ty_generics, where_clause) = input.generics.split_for_impl();
Ok(quote! {
impl #impl_generics ::decodal::DecodalSchema for #name #ty_generics #where_clause {
fn decodal_schema() -> ::decodal::Value {
#build_schema
}
}
impl #impl_generics ::decodal::DecodalDecode for #name #ty_generics #where_clause {
fn decodal_decode(data: &::decodal::Data) -> ::decodal::DecodeResult<Self> {
Ok(Self {
#(#decode_fields),*
})
}
}
})
}
#[derive(Default)]
struct FieldAttrs {
rename: String,
rename_explicit: bool,
rest: bool,
default: Option<DefaultAttr>,
constraints: Vec<ConstraintAttr>,
}
impl FieldAttrs {
fn from_attrs(attrs: &[syn::Attribute], fallback_name: &str) -> Result<Self> {
let mut output = Self {
rename: fallback_name.to_string(),
..Self::default()
};
for attr in attrs {
if !attr.path().is_ident("decodal") {
continue;
}
attr.parse_nested_meta(|meta| {
if meta.path.is_ident("rename") {
let value: LitStr = meta.value()?.parse()?;
output.rename = value.value();
output.rename_explicit = true;
return Ok(());
}
if meta.path.is_ident("rest") {
if meta.input.peek(syn::Token![=]) || meta.input.peek(syn::token::Paren) {
return Err(meta.error("`rest` does not accept a value"));
}
output.rest = true;
return Ok(());
}
if meta.path.is_ident("default") {
if meta.input.peek(syn::Token![=]) {
let expr: Expr = meta.value()?.parse()?;
output.default = Some(DefaultAttr::Expr(expr));
} else {
output.default = Some(DefaultAttr::Default);
}
return Ok(());
}
if let Some(kind) = ConstraintKind::from_path(&meta.path) {
let expr: Expr = meta.value()?.parse()?;
output.constraints.push(ConstraintAttr { kind, expr });
return Ok(());
}
Err(meta.error("unsupported decodal attribute"))
})?;
}
Ok(output)
}
}
enum DefaultAttr {
Default,
Expr(Expr),
}
struct ConstraintAttr {
kind: ConstraintKind,
expr: Expr,
}
#[derive(Clone, Copy)]
enum ConstraintKind {
Gt,
Gte,
Lt,
Lte,
}
impl ConstraintKind {
fn from_path(path: &syn::Path) -> Option<Self> {
if path.is_ident("gt") {
Some(Self::Gt)
} else if path.is_ident("gte") {
Some(Self::Gte)
} else if path.is_ident("lt") {
Some(Self::Lt)
} else if path.is_ident("lte") {
Some(Self::Lte)
} else {
None
}
}
fn method(self) -> syn::Ident {
match self {
Self::Gt => format_ident!("gt"),
Self::Gte => format_ident!("gte"),
Self::Lt => format_ident!("lt"),
Self::Lte => format_ident!("lte"),
}
}
}
fn schema_expr(ty: &Type, attrs: &FieldAttrs) -> Result<TokenStream2> {
let mut tokens = quote! {
let mut schema = <#ty as ::decodal::DecodalSchema>::decodal_schema();
};
for constraint in &attrs.constraints {
let method = constraint.kind.method();
let expr = &constraint.expr;
tokens.extend(quote! {
schema = schema.#method(#expr);
});
}
if let Some(default) = &attrs.default {
let default_expr = match default {
DefaultAttr::Default => quote! { <#ty as ::core::default::Default>::default() },
DefaultAttr::Expr(expr) => quote! { (#expr) },
};
tokens.extend(quote! {
schema = schema
.default(::decodal::IntoValue::into_value(#default_expr))
.expect("Decodal derive generated a valid default");
});
}
tokens.extend(quote! { schema });
Ok(tokens)
}
fn decode_expr(ty: &Type, path: &str, attrs: &FieldAttrs) -> TokenStream2 {
let missing = if let Some(default) = &attrs.default {
match default {
DefaultAttr::Default => quote! { <#ty as ::core::default::Default>::default() },
DefaultAttr::Expr(expr) => quote! { ::core::convert::Into::into(#expr) },
}
} else {
quote! { return Err(::decodal::DecodeError::new(#path, "missing field")); }
};
quote! {
if let Some(value) = ::decodal::data_at_path(data, #path) {
<#ty as ::decodal::DecodalDecode>::decodal_decode(value)
.map_err(|error| ::decodal::prefix_decode_error(#path, error))?
} else {
#missing
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn rejects_multiple_rest_fields() {
let input: DeriveInput = syn::parse_quote! {
struct Invalid {
#[decodal(rest)]
first: std::collections::BTreeMap<String, String>,
#[decodal(rest)]
second: std::collections::BTreeMap<String, String>,
}
};
assert!(
expand_decodal(input)
.unwrap_err()
.to_string()
.contains("only one `rest` field")
);
}
#[test]
fn rejects_rest_field_modifiers() {
let input: DeriveInput = syn::parse_quote! {
struct Invalid {
#[decodal(rest, default)]
extra: std::collections::BTreeMap<String, String>,
}
};
assert!(
expand_decodal(input)
.unwrap_err()
.to_string()
.contains("cannot be combined")
);
}
}
+268
View File
@@ -0,0 +1,268 @@
use std::collections::BTreeMap;
use decodal::{Constraint, Data, DecodalDecode, DecodalSchema, EmptyLoader, Engine, Value};
use decodal_derive::Decodal;
#[derive(Debug, PartialEq, Decodal)]
struct Service {
name: String,
#[decodal(gt = 443, default = 8443)]
port: i64,
#[decodal(default = true, rename = "feature.enable")]
feature_enabled: bool,
#[decodal(default)]
tags: Vec<String>,
}
#[derive(Debug, PartialEq, Decodal)]
struct Worker {
name: String,
#[decodal(default = true)]
enabled: bool,
}
#[derive(Debug, PartialEq, Decodal)]
struct Fleet {
workers: Vec<Worker>,
}
#[derive(Debug, PartialEq, Decodal)]
struct OpenConfig {
name: String,
#[decodal(default = true, rename = "feature.enabled")]
enabled: bool,
#[decodal(rest)]
extra: BTreeMap<String, Data>,
}
#[derive(Debug, PartialEq, Decodal)]
struct StringLabels {
name: String,
#[decodal(rest)]
labels: BTreeMap<String, String>,
}
#[derive(Debug, PartialEq, Decodal)]
struct OpenBag {
#[decodal(rest)]
values: BTreeMap<String, Data>,
}
#[test]
fn derives_schema_and_decode() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("Service", Service::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
Service & {
name = "api";
port = 9443;
feature.enable = false;
tags = ["web", "prod"];
}
"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let service = Service::decodal_decode(&data).unwrap();
assert_eq!(
service,
Service {
name: "api".into(),
port: 9443,
feature_enabled: false,
tags: vec!["web".into(), "prod".into()],
}
);
}
#[test]
fn defaults_are_available_in_schema() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("Service", Service::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
Service & {
name = "api";
}
"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let service = Service::decodal_decode(&data).unwrap();
assert_eq!(service.port, 8443);
assert!(service.feature_enabled);
assert!(service.tags.is_empty());
}
#[test]
fn decode_reports_field_path() {
let data = Data::Object(vec![]);
let error = Service::decodal_decode(&data).unwrap_err();
assert_eq!(error.path, "name");
}
#[test]
fn vec_schema_rejects_invalid_element_before_decode() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("Service", Service::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
Service & {
name = "api";
port = 9443;
tags = ["web", 1];
}
"#,
)
.unwrap();
let error = engine.eval_module(module).unwrap_err();
assert!(
error
.labels
.iter()
.any(|label| label.message.contains("array element [1]"))
);
}
#[test]
fn vec_schema_applies_nested_struct_defaults() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("Fleet", Fleet::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
Fleet & {
workers = [{ name = "api"; }];
}
"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let fleet = Fleet::decodal_decode(&data).unwrap();
assert_eq!(
fleet,
Fleet {
workers: vec![Worker {
name: "api".into(),
enabled: true,
}],
}
);
}
#[test]
fn rest_map_opens_the_derived_object_and_collects_additional_fields() {
let schema = OpenConfig::decodal_schema();
let Value::Object {
fields,
rest: Some(rest),
} = schema
else {
panic!("derived schema should have an object rest range")
};
assert_eq!(fields.len(), 2);
assert!(matches!(
*rest,
Value::Range { ref constraints, .. }
if constraints == &[Constraint::Unknown]
));
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("OpenConfig", OpenConfig::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"
{
name = "api";
plugin = { name = "cache"; };
retries = 3;
} as OpenConfig
"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let config = OpenConfig::decodal_decode(&data).unwrap();
assert_eq!(config.name, "api");
assert!(config.enabled);
assert_eq!(config.extra.get("retries"), Some(&Data::Int(3)));
assert!(matches!(config.extra.get("plugin"), Some(Data::Object(_))));
assert!(!config.extra.contains_key("name"));
assert!(!config.extra.contains_key("feature"));
}
#[test]
fn typed_rest_map_constrains_each_additional_field() {
let mut engine = Engine::new(EmptyLoader);
engine
.bind_global("StringLabels", StringLabels::decodal_schema())
.unwrap();
let module = engine
.add_root_source(
"test",
"test",
r#"{ name = "service"; region = "ap-northeast-1"; tier = "edge"; } as StringLabels"#,
)
.unwrap();
let value = engine.eval_module(module).unwrap();
let data = engine.materialize(&value).unwrap();
let labels = StringLabels::decodal_decode(&data).unwrap();
assert_eq!(labels.labels.get("region"), Some(&"ap-northeast-1".into()));
assert_eq!(labels.labels.get("tier"), Some(&"edge".into()));
let module = engine
.add_root_source(
"invalid",
"invalid",
r#"{ name = "service"; priority = 1; } as StringLabels"#,
)
.unwrap();
assert!(engine.eval_module(module).is_err());
}
#[test]
fn rest_only_struct_collects_every_field() {
let schema = OpenBag::decodal_schema();
assert!(matches!(
schema,
Value::Object {
ref fields,
rest: Some(_),
} if fields.is_empty()
));
let data = Data::Object(vec![decodal::runtime::DataField {
name: "answer".into(),
value: Data::Int(42),
}]);
let bag = OpenBag::decodal_decode(&data).unwrap();
assert_eq!(bag.values.get("answer"), Some(&Data::Int(42)));
}
@@ -0,0 +1,14 @@
[package]
name = "decodal-language-service"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Host-configurable semantic language service for Decodal."
keywords = ["decodal", "lsp", "language-server", "editor"]
categories = ["development-tools", "text-editors"]
[dependencies]
decodal = { version = "0.4.0", path = "../decodal-core" }
@@ -0,0 +1,913 @@
use std::collections::{BTreeMap, HashMap};
use decodal::{
Engine, HostEnvironment, ImportLoader, LoadedImport, Result, Value, runtime::RuntimeValue,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CompletionKind {
Keyword,
Constant,
Type,
Variable,
Namespace,
Property,
File,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CompletionItem {
pub label: String,
pub kind: CompletionKind,
pub detail: Option<String>,
pub priority: i32,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CompletionResult {
/// UTF-8 byte offset at which the unfinished token starts.
pub from: usize,
pub items: Vec<CompletionItem>,
}
#[derive(Debug, Clone, Default, PartialEq, Eq)]
struct FieldTree(BTreeMap<String, FieldTree>);
impl FieldTree {
fn new() -> Self {
Self::default()
}
fn is_empty(&self) -> bool {
self.0.is_empty()
}
fn insert(&mut self, name: String, fields: FieldTree) {
self.0.insert(name, fields);
}
fn get(&self, name: &str) -> Option<&FieldTree> {
self.0.get(name)
}
fn remove(&mut self, name: &str) -> Option<FieldTree> {
self.0.remove(name)
}
}
impl IntoIterator for FieldTree {
type Item = (String, FieldTree);
type IntoIter = std::collections::btree_map::IntoIter<String, FieldTree>;
fn into_iter(self) -> Self::IntoIter {
self.0.into_iter()
}
}
impl FromIterator<(String, FieldTree)> for FieldTree {
fn from_iter<T: IntoIterator<Item = (String, FieldTree)>>(iter: T) -> Self {
Self(iter.into_iter().collect())
}
}
pub(crate) fn complete<E: HostEnvironment>(
environment: &E,
key: &str,
source: &str,
position: usize,
explicit: bool,
) -> Result<Option<CompletionResult>> {
let position = floor_char_boundary(source, position.min(source.len()));
if let Some((from, prefix)) = unfinished_import(source, position) {
let mut loader = environment.create_loader();
let items = loader
.complete_import(Some(key), prefix)?
.into_iter()
.map(|candidate| CompletionItem {
label: candidate.specifier,
kind: CompletionKind::File,
detail: candidate.detail,
priority: 30,
})
.collect();
return Ok(Some(CompletionResult {
from,
items: unique_items(items),
}));
}
if in_string_or_comment(source, position) {
return Ok(None);
}
let tokens = tokenize(source);
let local_fields = collect_fields(&tokens);
let imports = collect_import_bindings(&tokens);
let globals = collect_globals(environment)?;
if let Some((from, path)) = member_path(source, position) {
let mut parts = path.split('.');
let Some(root) = parts.next() else {
return Ok(None);
};
let mut fields;
let detail;
if let Some(specifier) = imports.get(root) {
let mut loader = environment.create_loader();
let loaded = loader.load(Some(key), specifier)?;
detail = match &loaded {
LoadedImport::Source { name, .. } => name.clone(),
LoadedImport::Value { key, .. } => key.clone(),
};
fields = match loaded {
LoadedImport::Source { source, .. } => collect_fields(&tokenize(&source)),
LoadedImport::Value { value, .. } => fields_from_value(&value),
};
} else if let Some(local) = local_fields.get(root) {
fields = local.clone();
detail = String::from("local value");
} else if let Some(global) = globals.get(root) {
fields = global.clone();
detail = String::from("host global");
} else {
return Ok(None);
}
for part in parts {
let Some(nested) = fields.remove(part) else {
return Ok(None);
};
fields = nested;
}
let items = fields
.into_iter()
.map(|(label, children)| CompletionItem {
label,
kind: if children.is_empty() {
CompletionKind::Property
} else {
CompletionKind::Namespace
},
detail: Some(detail.clone()),
priority: 30,
})
.collect();
return Ok(Some(CompletionResult {
from,
items: unique_items(items),
}));
}
let word_from = word_start(source, position);
if word_from == position && !explicit {
return Ok(None);
}
let mut items = builtin_items();
for (label, children) in local_fields {
items.push(CompletionItem {
label,
kind: if children.is_empty() {
CompletionKind::Variable
} else {
CompletionKind::Namespace
},
detail: Some(if children.is_empty() {
String::from("local value")
} else {
String::from("local object")
}),
priority: 20,
});
}
for label in collect_parameters(&tokens) {
items.push(CompletionItem {
label,
kind: CompletionKind::Variable,
detail: Some(String::from("parameter")),
priority: 20,
});
}
for (label, specifier) in imports {
items.push(CompletionItem {
label,
kind: CompletionKind::Namespace,
detail: Some(specifier),
priority: 30,
});
}
for (label, children) in globals {
items.push(CompletionItem {
label,
kind: if children.is_empty() {
CompletionKind::Variable
} else {
CompletionKind::Namespace
},
detail: Some(String::from("host global")),
priority: 40,
});
}
Ok(Some(CompletionResult {
from: word_from,
items: unique_items(items),
}))
}
fn builtin_items() -> Vec<CompletionItem> {
[
("let", CompletionKind::Keyword, "local bindings", 5),
("in", CompletionKind::Keyword, "let body", 0),
("match", CompletionKind::Keyword, "pattern matching", 5),
("import", CompletionKind::Keyword, "load a module", 5),
("default", CompletionKind::Keyword, "fallback value", 0),
("as", CompletionKind::Keyword, "refine narrower as wider", 0),
("true", CompletionKind::Constant, "Bool", 0),
("false", CompletionKind::Constant, "Bool", 0),
("String", CompletionKind::Type, "string constraint", 5),
("Int", CompletionKind::Type, "integer constraint", 5),
("Float", CompletionKind::Type, "float constraint", 5),
("Bool", CompletionKind::Type, "boolean constraint", 5),
("Unknown", CompletionKind::Type, "unknown top range", 5),
]
.into_iter()
.map(|(label, kind, detail, priority)| CompletionItem {
label: label.into(),
kind,
detail: Some(detail.into()),
priority,
})
.collect()
}
fn collect_globals<E: HostEnvironment>(environment: &E) -> Result<FieldTree> {
let mut engine = environment.create_engine()?;
let mut fields = FieldTree::new();
for (name, value) in engine.global_values()? {
fields.insert(name, fields_from_runtime(&mut engine, &value, 0)?);
}
Ok(fields)
}
fn fields_from_runtime<L: ImportLoader>(
engine: &mut Engine<L>,
value: &RuntimeValue,
depth: usize,
) -> Result<FieldTree> {
if depth >= 32 {
return Ok(FieldTree::new());
}
let Some(fields) = engine.value_fields(value)? else {
return Ok(FieldTree::new());
};
let mut tree = FieldTree::new();
for (name, value) in fields {
tree.insert(name, fields_from_runtime(engine, &value, depth + 1)?);
}
Ok(tree)
}
fn fields_from_value(value: &Value) -> FieldTree {
match value {
Value::Object { fields, .. } => fields
.iter()
.map(|field| (field.name.clone(), fields_from_value(&field.value)))
.collect(),
Value::ArrayRange {
default: Some(value),
..
}
| Value::MapRange {
default: Some(value),
..
}
| Value::Range {
default: Some(value),
..
} => fields_from_value(value),
_ => FieldTree::new(),
}
}
fn unique_items(items: Vec<CompletionItem>) -> Vec<CompletionItem> {
let mut unique = HashMap::<String, CompletionItem>::new();
let mut order = Vec::new();
for item in items {
if !unique.contains_key(&item.label) {
order.push(item.label.clone());
}
let replace = unique
.get(&item.label)
.is_none_or(|previous| item.priority > previous.priority);
if replace {
unique.insert(item.label.clone(), item);
}
}
order
.into_iter()
.filter_map(|label| unique.remove(&label))
.collect()
}
#[derive(Debug, Clone, PartialEq, Eq)]
enum TokenKind {
Identifier(String),
String(String),
Symbol(char),
Arrow,
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct Token {
kind: TokenKind,
}
fn tokenize(source: &str) -> Vec<Token> {
let bytes = source.as_bytes();
let mut tokens = Vec::new();
let mut index = 0;
while index < bytes.len() {
match bytes[index] {
b' ' | b'\t' | b'\r' | b'\n' => index += 1,
b'#' => {
while index < bytes.len() && bytes[index] != b'\n' {
index += 1;
}
}
b'"' => {
index += 1;
let start = index;
let mut escaped = false;
while index < bytes.len() {
let byte = bytes[index];
if escaped {
escaped = false;
} else if byte == b'\\' {
escaped = true;
} else if byte == b'"' || byte == b'\n' {
break;
}
index += 1;
}
let value = unescape_string(&source[start..index]);
index = index.saturating_add(1);
tokens.push(Token {
kind: TokenKind::String(value),
});
}
byte if is_identifier_start(byte) => {
let start = index;
index += 1;
while index < bytes.len() && is_identifier_continue(bytes[index]) {
index += 1;
}
tokens.push(Token {
kind: TokenKind::Identifier(source[start..index].into()),
});
}
b'=' if bytes.get(index + 1) == Some(&b'>') => {
tokens.push(Token {
kind: TokenKind::Arrow,
});
index += 2;
}
byte => {
tokens.push(Token {
kind: TokenKind::Symbol(byte as char),
});
index += 1;
}
}
}
tokens
}
fn unescape_string(value: &str) -> String {
let mut chars = value.chars();
let mut unescaped = String::new();
while let Some(ch) = chars.next() {
if ch != '\\' {
unescaped.push(ch);
continue;
}
let Some(escaped) = chars.next() else {
unescaped.push('\\');
break;
};
unescaped.push(match escaped {
'n' => '\n',
'r' => '\r',
't' => '\t',
escaped => escaped,
});
}
unescaped
}
fn collect_fields(tokens: &[Token]) -> FieldTree {
let mut fields = FieldTree::new();
collect_fields_in(tokens, 0, tokens.len(), &mut fields);
fields
}
fn collect_fields_in(tokens: &[Token], start: usize, end: usize, fields: &mut FieldTree) {
let mut index = start;
while index < end {
let Some((path, equals)) = field_definition_at(tokens, index, end) else {
index += 1;
continue;
};
let value_start = equals + 1;
let value_end = definition_end(tokens, value_start, end);
let mut nested = &mut *fields;
for part in path {
nested = nested.0.entry(part).or_default();
}
collect_fields_in(tokens, value_start, value_end, nested);
index = value_end.saturating_add(1);
}
}
fn field_definition_at(tokens: &[Token], start: usize, end: usize) -> Option<(Vec<String>, usize)> {
let TokenKind::Identifier(first) = &tokens.get(start)?.kind else {
return None;
};
let mut path = vec![first.clone()];
let mut index = start + 1;
while index + 1 < end
&& matches!(tokens[index].kind, TokenKind::Symbol('.'))
&& matches!(tokens[index + 1].kind, TokenKind::Identifier(_))
{
let TokenKind::Identifier(part) = &tokens[index + 1].kind else {
unreachable!()
};
path.push(part.clone());
index += 2;
}
matches!(
tokens.get(index).map(|token| &token.kind),
Some(TokenKind::Symbol('='))
)
.then_some((path, index))
}
fn definition_end(tokens: &[Token], start: usize, end: usize) -> usize {
let mut delimiters = Vec::new();
for (index, token) in tokens.iter().enumerate().take(end).skip(start) {
match token.kind {
TokenKind::Symbol('(' | '[' | '{') => delimiters.push(token.kind.clone()),
TokenKind::Symbol(')') if matches!(delimiters.last(), Some(TokenKind::Symbol('('))) => {
delimiters.pop();
}
TokenKind::Symbol(']') if matches!(delimiters.last(), Some(TokenKind::Symbol('['))) => {
delimiters.pop();
}
TokenKind::Symbol('}') if matches!(delimiters.last(), Some(TokenKind::Symbol('{'))) => {
delimiters.pop();
}
TokenKind::Symbol(';') if delimiters.is_empty() => return index,
_ => {}
}
}
end
}
fn collect_import_bindings(tokens: &[Token]) -> BTreeMap<String, String> {
let mut imports = BTreeMap::new();
for window in tokens.windows(4) {
let (
TokenKind::Identifier(binding),
TokenKind::Symbol('='),
TokenKind::Identifier(keyword),
TokenKind::String(specifier),
) = (
&window[0].kind,
&window[1].kind,
&window[2].kind,
&window[3].kind,
)
else {
continue;
};
if keyword == "import" {
imports.insert(binding.clone(), specifier.clone());
}
}
imports
}
fn collect_parameters(tokens: &[Token]) -> Vec<String> {
let mut parameters = Vec::new();
for (close, token) in tokens.iter().enumerate() {
if !matches!(token.kind, TokenKind::Symbol(')'))
|| !matches!(
tokens.get(close + 1).map(|token| &token.kind),
Some(TokenKind::Arrow)
)
{
continue;
}
let Some(open) = matching_open_paren(tokens, close) else {
continue;
};
let mut depth = 0usize;
let mut segment_start = true;
for token in &tokens[open + 1..close] {
match &token.kind {
TokenKind::Symbol('(' | '[' | '{') => depth += 1,
TokenKind::Symbol(')' | ']' | '}') => depth = depth.saturating_sub(1),
TokenKind::Symbol(',') if depth == 0 => segment_start = true,
TokenKind::Identifier(name) if depth == 0 && segment_start => {
parameters.push(name.clone());
segment_start = false;
}
_ => {}
}
}
}
parameters
}
fn matching_open_paren(tokens: &[Token], close: usize) -> Option<usize> {
let mut depth = 0usize;
for index in (0..close).rev() {
match tokens[index].kind {
TokenKind::Symbol(')') => depth += 1,
TokenKind::Symbol('(') if depth == 0 => return Some(index),
TokenKind::Symbol('(') => depth -= 1,
_ => {}
}
}
None
}
fn unfinished_import(source: &str, position: usize) -> Option<(usize, &str)> {
let bytes = source.as_bytes();
let mut index = 0usize;
let mut previous_word = None;
while index < position {
match bytes[index] {
b' ' | b'\t' | b'\r' | b'\n' => index += 1,
b'#' => {
while index < position && bytes[index] != b'\n' {
index += 1;
}
previous_word = None;
}
b'"' => {
let from = index + 1;
index += 1;
let mut escaped = false;
while index < position {
let byte = bytes[index];
if escaped {
escaped = false;
} else if byte == b'\\' {
escaped = true;
} else if byte == b'"' || byte == b'\n' {
break;
}
index += 1;
}
if index == position && previous_word == Some("import") {
return Some((from, &source[from..position]));
}
index = index.saturating_add(1);
previous_word = None;
}
byte if is_identifier_start(byte) => {
let start = index;
index += 1;
while index < position && is_identifier_continue(bytes[index]) {
index += 1;
}
previous_word = Some(&source[start..index]);
}
_ => {
index += 1;
previous_word = None;
}
}
}
None
}
fn in_string_or_comment(source: &str, position: usize) -> bool {
let bytes = source.as_bytes();
let mut index = 0usize;
while index < position {
match bytes[index] {
b'#' => {
while index < position && bytes[index] != b'\n' {
index += 1;
}
if index == position {
return true;
}
}
b'"' => {
index += 1;
let mut escaped = false;
while index < position {
let byte = bytes[index];
if escaped {
escaped = false;
} else if byte == b'\\' {
escaped = true;
} else if byte == b'"' || byte == b'\n' {
break;
}
index += 1;
}
if index == position {
return true;
}
index = index.saturating_add(1);
}
_ => index += 1,
}
}
false
}
fn member_path(source: &str, position: usize) -> Option<(usize, &str)> {
let bytes = source.as_bytes();
let from = word_start(source, position);
if from == 0 || bytes[from - 1] != b'.' {
return None;
}
let mut base_start = from - 1;
while base_start > 0 {
let byte = bytes[base_start - 1];
if is_identifier_continue(byte) || byte == b'.' {
base_start -= 1;
} else {
break;
}
}
let base = &source[base_start..from - 1];
(!base.is_empty()
&& base
.split('.')
.all(|part| !part.is_empty() && is_identifier(part)))
.then_some((from, base))
}
fn word_start(source: &str, position: usize) -> usize {
let bytes = source.as_bytes();
let mut from = position;
while from > 0 && is_identifier_continue(bytes[from - 1]) {
from -= 1;
}
from
}
fn is_identifier(value: &str) -> bool {
let bytes = value.as_bytes();
bytes.first().is_some_and(|byte| is_identifier_start(*byte))
&& bytes[1..].iter().all(|byte| is_identifier_continue(*byte))
}
fn is_identifier_start(byte: u8) -> bool {
byte.is_ascii_alphabetic() || byte == b'_'
}
fn is_identifier_continue(byte: u8) -> bool {
is_identifier_start(byte) || byte.is_ascii_digit()
}
fn floor_char_boundary(source: &str, mut position: usize) -> usize {
while !source.is_char_boundary(position) {
position -= 1;
}
position
}
#[cfg(test)]
mod tests {
use std::collections::BTreeMap;
use decodal::{
Diagnostic, DiagnosticKind, EmptyLoader, ImportCandidate, LoadedImport, Span, Value,
};
use super::*;
#[derive(Clone, Default)]
struct TestEnvironment {
files: BTreeMap<String, String>,
}
#[derive(Clone, Default)]
struct TestLoader {
files: BTreeMap<String, String>,
}
impl ImportLoader for TestLoader {
fn load(&mut self, _current_key: Option<&str>, specifier: &str) -> Result<LoadedImport> {
if specifier == "./post.md" {
return Ok(LoadedImport::value(
"post.md",
Value::object([
(
"frontmatter",
Value::object([
("title", Value::string("Hello")),
("draft", Value::bool(false)),
]),
),
("body", Value::string("# Hello")),
]),
));
}
let key = resolve_virtual("main.dcdl", specifier);
let Some(source) = self.files.get(&key) else {
return Err(Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
"unknown import",
));
};
Ok(LoadedImport::source(key.clone(), key, source))
}
fn complete_import(
&mut self,
_current_key: Option<&str>,
prefix: &str,
) -> Result<Vec<ImportCandidate>> {
Ok(self
.files
.keys()
.map(|path| format!("./{path}"))
.filter(|path| path.starts_with(prefix))
.map(ImportCandidate::new)
.collect())
}
}
impl HostEnvironment for TestEnvironment {
type Loader = TestLoader;
fn create_loader(&self) -> Self::Loader {
TestLoader {
files: self.files.clone(),
}
}
fn configure_engine(&self, engine: &mut Engine<Self::Loader>) -> Result<()> {
engine.bind_global(
"App",
Value::object([(
"config",
Value::object([("enabled", Value::bool_type()), ("port", Value::int_type())]),
)]),
)?;
Ok(())
}
}
fn environment() -> TestEnvironment {
TestEnvironment {
files: BTreeMap::from([
(
"schemas/service.dcdl".into(),
"Service = { name = String; resources = { cpu = Int; }; };".into(),
),
(
"env/production.dcdl".into(),
"capacity = { cpu = 2000; };".into(),
),
(
"資料/service.dcdl".into(),
"Service = { label = String; };".into(),
),
]),
}
}
fn labels(result: CompletionResult) -> Vec<String> {
result.items.into_iter().map(|item| item.label).collect()
}
#[test]
fn completes_import_paths_from_the_host_loader() {
let source = "schema = import \"./sch";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(result.from, source.len() - 5);
assert!(labels(result).contains(&"./schemas/service.dcdl".into()));
}
#[test]
fn completes_nested_fields_from_imported_source() {
let source = "let schema = import \"./schemas/service.dcdl\"; in schema.Service.";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(labels(result), ["name", "resources"]);
}
#[test]
fn completes_structured_host_imports() {
let source = "let post = import \"./post.md\"; in post.frontmatter.";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(labels(result), ["draft", "title"]);
}
#[test]
fn preserves_unicode_import_specifiers() {
let source = "let schema = import \"./資料/service.dcdl\"; in schema.Service.";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(labels(result), ["label"]);
}
#[test]
fn completes_host_globals_from_the_evaluation_environment() {
let source = "App.config.";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(labels(result), ["enabled", "port"]);
}
#[test]
fn completes_language_locals_parameters_and_partial_members() {
let source = "let service = { port = 8080; }; in (value: Int) => service.po";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
assert_eq!(result.from, source.len() - 2);
assert_eq!(labels(result), ["port"]);
let source = "let service = { port = 8080; }; in (value: Int) => val";
let result = complete(&environment(), "main.dcdl", source, source.len(), false)
.unwrap()
.unwrap();
let labels = labels(result);
assert!(labels.contains(&"String".into()));
assert!(labels.contains(&"Unknown".into()));
assert!(labels.contains(&"service".into()));
assert!(labels.contains(&"value".into()));
assert!(labels.contains(&"App".into()));
}
#[test]
fn suppresses_completions_in_strings_and_comments() {
let string = "value = \"hello world\"";
assert!(
complete(&environment(), "main.dcdl", string, string.len() - 1, false)
.unwrap()
.is_none()
);
let comment = "value = true; # hello";
assert!(
complete(&environment(), "main.dcdl", comment, comment.len(), false)
.unwrap()
.is_none()
);
}
#[test]
fn empty_environment_remains_supported() {
let result = complete(&EmptyEnvironment, "main.dcdl", "Str", 3, false)
.unwrap()
.unwrap();
assert!(labels(result).contains(&"String".into()));
let result = complete(&EmptyEnvironment, "main.dcdl", "Unk", 3, false)
.unwrap()
.unwrap();
assert!(labels(result).contains(&"Unknown".into()));
}
struct EmptyEnvironment;
impl HostEnvironment for EmptyEnvironment {
type Loader = EmptyLoader;
fn create_loader(&self) -> Self::Loader {
EmptyLoader
}
}
fn resolve_virtual(current: &str, specifier: &str) -> String {
let mut parts = current.split('/').collect::<Vec<_>>();
parts.pop();
for part in specifier.split('/') {
match part {
"" | "." => {}
".." => {
parts.pop();
}
part => parts.push(part),
}
}
parts.join("/")
}
}
+204
View File
@@ -0,0 +1,204 @@
use decodal::{Data, Diagnostic, HostEnvironment, Result};
mod completion;
pub use completion::{CompletionItem, CompletionKind, CompletionResult};
/// Semantic tooling backed by the same host environment as production.
pub struct LanguageService<E> {
environment: E,
}
impl<E> LanguageService<E> {
pub fn new(environment: E) -> Self {
Self { environment }
}
pub fn environment(&self) -> &E {
&self.environment
}
pub fn environment_mut(&mut self) -> &mut E {
&mut self.environment
}
pub fn into_environment(self) -> E {
self.environment
}
}
impl<E: HostEnvironment> LanguageService<E> {
/// Runs the production parse, evaluation, and materialization pipeline.
pub fn evaluate(
&self,
key: impl Into<String>,
name: impl Into<String>,
source: &str,
) -> Result<Data> {
let mut engine = self.environment.create_engine()?;
let module = engine.add_root_source(key, name, source)?;
let value = engine.eval_module(module)?;
engine.materialize(&value)
}
/// Evaluates a document and exposes failures in an editor-friendly form.
///
/// The evaluator currently stops at the first failure, so an invalid
/// analysis contains one diagnostic. The collection leaves room for a
/// future diagnostic-accumulation pass without changing this API.
pub fn analyze(
&self,
key: impl Into<String>,
name: impl Into<String>,
source: &str,
) -> SemanticAnalysis {
match self.evaluate(key, name, source) {
Ok(data) => SemanticAnalysis {
data: Some(data),
diagnostics: Vec::new(),
},
Err(diagnostic) => SemanticAnalysis {
data: None,
diagnostics: vec![diagnostic],
},
}
}
/// Completes the document using the same host globals and import loader as
/// production evaluation.
pub fn complete(
&self,
key: impl AsRef<str>,
source: &str,
position: usize,
explicit: bool,
) -> Result<Option<CompletionResult>> {
completion::complete(&self.environment, key.as_ref(), source, position, explicit)
}
}
#[derive(Debug, Clone, PartialEq)]
pub struct SemanticAnalysis {
pub data: Option<Data>,
pub diagnostics: Vec<Diagnostic>,
}
impl SemanticAnalysis {
pub fn is_valid(&self) -> bool {
self.diagnostics.is_empty()
}
}
#[cfg(test)]
mod tests {
use decodal::{
Data, Diagnostic, DiagnosticKind, Engine, ImportLoader, LoadedImport, Span, Value,
};
use super::*;
const ROOT: &str = r#"Post & import "./post.md""#;
struct ContentEnvironment {
draft: Value,
}
struct ContentLoader {
draft: Value,
}
impl ImportLoader for ContentLoader {
fn load(
&mut self,
_current_key: Option<&str>,
specifier: &str,
) -> decodal::Result<LoadedImport> {
if specifier != "./post.md" {
return Err(Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
"unknown content import",
));
}
Ok(LoadedImport::value(
"content/post.md",
Value::object([
(
"frontmatter",
Value::object([
("title", Value::string("Hello")),
("draft", self.draft.clone()),
]),
),
("body", Value::string("# Hello")),
]),
))
}
}
impl HostEnvironment for ContentEnvironment {
type Loader = ContentLoader;
fn create_loader(&self) -> Self::Loader {
ContentLoader {
draft: self.draft.clone(),
}
}
fn configure_engine(&self, engine: &mut Engine<Self::Loader>) -> decodal::Result<()> {
engine.bind_global(
"Post",
Value::object([
(
"frontmatter",
Value::object([
("title", Value::string_type()),
("draft", Value::bool_type()),
]),
),
("body", Value::string_type()),
]),
)?;
Ok(())
}
}
fn evaluate_direct(environment: &ContentEnvironment) -> decodal::Result<Data> {
let mut engine = environment.create_engine()?;
let module = engine.add_root_source("main.dcdl", "main.dcdl", ROOT)?;
let value = engine.eval_module(module)?;
engine.materialize(&value)
}
#[test]
fn injected_environment_matches_direct_evaluation() {
let environment = ContentEnvironment {
draft: Value::bool(false),
};
let direct = evaluate_direct(&environment).unwrap();
let service = LanguageService::new(&environment);
let analysis = service.analyze("main.dcdl", "main.dcdl", ROOT);
assert!(analysis.is_valid());
assert_eq!(analysis.data, Some(direct));
}
#[test]
fn injected_environment_preserves_import_diagnostics() {
let service = LanguageService::new(ContentEnvironment {
draft: Value::string("maybe"),
});
let analysis = service.analyze("main.dcdl", "main.dcdl", ROOT);
assert!(!analysis.is_valid());
assert_eq!(analysis.data, None);
assert_eq!(
analysis.diagnostics[0].message,
"expected Bool, found String"
);
assert_eq!(
analysis.diagnostics[0].notes,
["imported `content/post.md` at `frontmatter.draft` supplied a value of type String"]
);
}
}
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "decodal-language-tools"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Source-level language tooling for Decodal."
keywords = ["decodal", "formatter", "lsp", "language-tools"]
categories = ["development-tools", "text-processing"]
[lib]
crate-type = ["cdylib", "rlib"]
[dependencies]
decodal = { version = "0.4.0", path = "../decodal-core" }
serde_json.workspace = true
wasm-bindgen.workspace = true
[package.metadata.wasm-pack.profile.release]
wasm-opt = false
+838
View File
@@ -0,0 +1,838 @@
use std::{error::Error, fmt};
use decodal::{
Ast, BinaryOp, CompareOp, Expr, ExprId, Field, ObjectRest, Param, SourceForm, Span,
SyntaxToken, SyntaxTokenKind,
ast::{MatchArm, UnaryOp},
parse_source, tokenize_source,
};
use wasm_bindgen::prelude::*;
const INDENT: usize = 4;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct FormatError {
message: String,
}
impl FormatError {
fn new(message: impl Into<String>) -> Self {
Self {
message: message.into(),
}
}
pub fn message(&self) -> &str {
&self.message
}
}
impl fmt::Display for FormatError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.message)
}
}
impl Error for FormatError {}
/// Formats a complete Decodal source with the canonical parser and syntax tokens.
///
/// Native and WebAssembly callers execute this exact implementation. Comments
/// are recovered from the lossless token stream while expression structure and
/// precedence come from the same AST used by evaluation.
pub fn format_source(source: &str) -> Result<String, FormatError> {
let parsed = parse_source(source)
.map_err(|diagnostic| FormatError::new(format!("parse error: {}", diagnostic.message)))?;
let tokens = tokenize_source(source)
.map_err(|diagnostic| FormatError::new(format!("lex error: {}", diagnostic.message)))?;
Ok(Formatter::new(source, &parsed.ast, tokens).format(parsed.root, parsed.source_form))
}
#[wasm_bindgen(js_name = formatSource)]
pub fn format_source_json(source: &str) -> String {
match format_source(source) {
Ok(formatted) => serde_json::json!({ "ok": true, "source": formatted }).to_string(),
Err(error) => serde_json::json!({ "ok": false, "error": error.message() }).to_string(),
}
}
struct Formatter<'a> {
source: &'a str,
ast: &'a Ast,
tokens: Vec<SyntaxToken>,
comments: Vec<Span>,
}
impl<'a> Formatter<'a> {
fn new(source: &'a str, ast: &'a Ast, tokens: Vec<SyntaxToken>) -> Self {
let comments = tokens
.iter()
.filter(|token| token.kind == SyntaxTokenKind::Comment)
.map(|token| token.span)
.collect();
Self {
source,
ast,
tokens,
comments,
}
}
fn format(&self, root: ExprId, source_form: SourceForm) -> String {
let mut out = String::new();
if source_form == SourceForm::Fields {
let Expr::Object { fields, .. } = &self.ast.get(root).expr else {
unreachable!("field-form sources parse to an object")
};
self.write_field_list(&mut out, fields, 0, self.source.len(), 0);
} else {
let span = self.ast.span(root);
self.write_between(&mut out, 0, span.start as usize, None, 0);
self.write_expr(&mut out, root, 0, 0);
self.write_between(
&mut out,
span.end as usize,
self.source.len(),
Some(span.end as usize),
0,
);
}
trim_trailing_whitespace(&mut out);
out.push('\n');
out
}
fn write_field_list(
&self,
out: &mut String,
fields: &[Field],
start: usize,
end: usize,
indent: usize,
) {
let mut cursor = start;
let mut previous_end = None;
for field in fields {
let field_start = field.span.start as usize;
self.write_between(out, cursor, field_start, previous_end, indent);
write_indent(out, indent);
self.write_field(out, field, indent);
out.push(';');
cursor = field.span.end as usize;
previous_end = Some(cursor);
}
self.write_between(out, cursor, end, previous_end, indent);
}
fn write_field(&self, out: &mut String, field: &Field, indent: usize) {
let value_start = self.ast.span(field.value).start as usize;
if self.has_comment_between(field.span.start as usize, value_start) {
out.push_str(self.raw(field.span));
return;
}
for (index, part) in field.path.iter().enumerate() {
if index > 0 {
out.push('.');
}
out.push_str(part);
}
out.push_str(" = ");
self.write_expr(out, field.value, indent, 0);
}
fn write_expr(&self, out: &mut String, id: ExprId, indent: usize, parent_prec: u8) {
let node = self.ast.get(id);
if self.has_comment(node.span)
&& !matches!(
node.expr,
Expr::Object { .. } | Expr::Array(_) | Expr::Let { .. } | Expr::Match { .. }
)
{
out.push_str(self.raw(node.span));
return;
}
let precedence = self.precedence(id);
let parenthesize = precedence < parent_prec;
if parenthesize {
out.push('(');
}
match &node.expr {
Expr::Literal(_) | Expr::Ident(_) | Expr::RegexConstraint(_) | Expr::Wildcard => {
out.push_str(self.raw(node.span));
}
Expr::Object { fields, rest } => {
self.write_object(out, node.span, fields, rest.as_ref(), indent)
}
Expr::Array(items) => self.write_array(out, node.span, items, indent),
Expr::ArrayConstraint { item } => {
out.push_str("[...");
self.write_expr(out, *item, indent, 0);
out.push(']');
}
Expr::MapConstraint { value } => {
out.push_str("{...");
self.write_expr(out, *value, indent, 0);
out.push('}');
}
Expr::Let { bindings, body } => self.write_let(out, node.span, bindings, *body, indent),
Expr::Import(_) => {
out.push_str("import ");
let raw = self.raw(node.span);
out.push_str(raw.strip_prefix("import").unwrap_or(raw).trim());
}
Expr::Path { base, field } => {
self.write_expr(out, *base, indent, precedence);
out.push('.');
out.push_str(field);
}
Expr::Call { callee, args } => {
self.write_expr(out, *callee, indent, precedence);
out.push('(');
for (index, argument) in args.iter().enumerate() {
if index > 0 {
out.push_str(", ");
}
self.write_expr(out, *argument, indent, 0);
}
out.push(')');
}
Expr::Function { params, body } => self.write_function(out, params, *body, indent),
Expr::Parenthesized { expr } => {
out.push('(');
self.write_expr(out, *expr, indent, 0);
out.push(')');
}
Expr::Match { scrutinee, arms } => {
self.write_match(out, node.span, *scrutinee, arms, indent)
}
Expr::Unary { op, expr } => {
out.push_str(match op {
UnaryOp::Neg => "-",
UnaryOp::Not => "!",
});
self.write_expr(out, *expr, indent, precedence);
}
Expr::Binary { op, lhs, rhs } => {
self.write_expr(out, *lhs, indent, precedence);
out.push(' ');
out.push_str(binary_operator(*op));
out.push(' ');
self.write_expr(out, *rhs, indent, precedence + 1);
}
Expr::Default { base, fallback } => {
self.write_expr(out, *base, indent, precedence);
out.push_str(" default ");
self.write_expr(out, *fallback, indent, precedence + 1);
}
Expr::As { narrower, wider } => {
self.write_expr(out, *narrower, indent, precedence);
out.push_str(" as ");
self.write_expr(out, *wider, indent, precedence + 1);
}
Expr::CompareConstraint { op, value } => {
out.push_str(compare_operator(*op));
out.push(' ');
self.write_expr(out, *value, indent, precedence);
}
}
if parenthesize {
out.push(')');
}
}
fn write_object(
&self,
out: &mut String,
span: Span,
fields: &[Field],
rest: Option<&ObjectRest>,
indent: usize,
) {
let (start, end) =
self.delimited_range(span, SyntaxTokenKind::LBrace, SyntaxTokenKind::RBrace);
if fields.is_empty() && rest.is_none() && !self.has_comment_between(start, end) {
out.push_str("{}");
return;
}
out.push_str("{\n");
let field_end = rest.map_or(end, |rest| rest.span.start as usize);
self.write_field_list(out, fields, start, field_end, indent + INDENT);
if let Some(rest) = rest {
write_indent(out, indent + INDENT);
out.push_str("...");
let value_start = self.ast.span(rest.value).start as usize;
let ellipsis = self
.tokens_between(rest.span.start as usize, value_start)
.find(|token| token.kind == SyntaxTokenKind::Ellipsis)
.expect("parsed object rest constraints contain `...`");
if self.has_comment_between(ellipsis.span.end as usize, value_start) {
self.write_between(
out,
ellipsis.span.end as usize,
value_start,
Some(ellipsis.span.end as usize),
indent + INDENT,
);
write_indent(out, indent + INDENT);
}
self.write_expr(out, rest.value, indent + INDENT, 0);
self.write_between(
out,
rest.span.end as usize,
end,
Some(rest.span.end as usize),
indent + INDENT,
);
}
write_indent(out, indent);
out.push('}');
}
fn write_array(&self, out: &mut String, span: Span, items: &[ExprId], indent: usize) {
let (start, end) =
self.delimited_range(span, SyntaxTokenKind::LBracket, SyntaxTokenKind::RBracket);
if items.is_empty() && !self.has_comment_between(start, end) {
out.push_str("[]");
return;
}
let inline = !self.has_comment_between(start, end)
&& items.iter().all(|item| self.is_inline_expr(*item));
if inline {
out.push('[');
for (index, item) in items.iter().enumerate() {
if index > 0 {
out.push_str(", ");
}
self.write_expr(out, *item, indent, 0);
}
out.push(']');
return;
}
out.push_str("[\n");
let mut cursor = start;
let mut previous_end = None;
for item in items {
let item_span = self.ast.span(*item);
self.write_between(
out,
cursor,
item_span.start as usize,
previous_end,
indent + INDENT,
);
write_indent(out, indent + INDENT);
self.write_expr(out, *item, indent + INDENT, 0);
out.push(',');
cursor = item_span.end as usize;
previous_end = Some(cursor);
}
self.write_between(out, cursor, end, previous_end, indent + INDENT);
write_indent(out, indent);
out.push(']');
}
fn write_let(
&self,
out: &mut String,
span: Span,
bindings: &[Field],
body: ExprId,
indent: usize,
) {
let body_span = self.ast.span(body);
let in_token = self
.tokens_between(span.start as usize, body_span.start as usize)
.rev()
.find(|token| token.kind == SyntaxTokenKind::In)
.expect("parsed let expressions contain `in`");
let let_token = self
.tokens_between(span.start as usize, span.end as usize)
.find(|token| token.kind == SyntaxTokenKind::Let)
.expect("parsed let expressions contain `let`");
out.push_str("let\n");
self.write_field_list(
out,
bindings,
let_token.span.end as usize,
in_token.span.start as usize,
indent + INDENT,
);
write_indent(out, indent);
out.push_str("in");
self.write_between(
out,
in_token.span.end as usize,
body_span.start as usize,
Some(in_token.span.end as usize),
indent + INDENT,
);
write_indent(out, indent + INDENT);
self.write_expr(out, body, indent + INDENT, 0);
}
fn write_function(&self, out: &mut String, params: &[Param], body: ExprId, indent: usize) {
out.push('(');
for (index, parameter) in params.iter().enumerate() {
if index > 0 {
out.push_str(", ");
}
out.push_str(&parameter.name);
if let Some(constraint) = parameter.constraint {
out.push_str(": ");
self.write_expr(out, constraint, indent, 0);
}
}
out.push_str(") =>");
if self.is_inline_expr(body) {
out.push(' ');
self.write_expr(out, body, indent, 0);
} else {
out.push('\n');
write_indent(out, indent + INDENT);
self.write_expr(out, body, indent + INDENT, 0);
}
}
fn write_match(
&self,
out: &mut String,
span: Span,
scrutinee: ExprId,
arms: &[MatchArm],
indent: usize,
) {
out.push_str("match ");
self.write_expr(out, scrutinee, indent, 0);
let scrutinee_end = self.ast.span(scrutinee).end as usize;
let open = self
.tokens_between(scrutinee_end, span.end as usize)
.find(|token| token.kind == SyntaxTokenKind::LBrace)
.expect("parsed match expressions contain `{`");
let close = self
.tokens_between(open.span.end as usize, span.end as usize)
.rev()
.find(|token| token.kind == SyntaxTokenKind::RBrace)
.expect("parsed match expressions contain `}`");
let start = open.span.end as usize;
let end = close.span.start as usize;
if self.has_comment_between(scrutinee_end, open.span.start as usize) {
self.write_between(
out,
scrutinee_end,
open.span.start as usize,
Some(scrutinee_end),
indent,
);
write_indent(out, indent);
out.push('{');
} else {
out.push_str(" {");
}
if arms.is_empty() && !self.has_comment_between(start, end) {
out.push('}');
return;
}
out.push('\n');
let mut cursor = start;
let mut previous_end = None;
for arm in arms {
self.write_between(
out,
cursor,
arm.span.start as usize,
previous_end,
indent + INDENT,
);
write_indent(out, indent + INDENT);
if self.has_comment_between(
self.ast.span(arm.pattern).end as usize,
self.ast.span(arm.body).start as usize,
) {
out.push_str(self.raw(arm.span));
} else {
self.write_expr(out, arm.pattern, indent + INDENT, 0);
out.push_str(": ");
self.write_expr(out, arm.body, indent + INDENT, 0);
}
out.push(';');
cursor = arm.span.end as usize;
previous_end = Some(cursor);
}
self.write_between(out, cursor, end, previous_end, indent + INDENT);
write_indent(out, indent);
out.push('}');
}
fn write_between(
&self,
out: &mut String,
start: usize,
end: usize,
previous_end: Option<usize>,
indent: usize,
) {
let mut cursor = start.min(end);
for comment in self.comments_between(start, end) {
let comment_start = comment.start as usize;
let trailing = previous_end.is_some_and(|previous| {
!out.ends_with('\n') && self.same_line(previous, comment_start)
});
if trailing {
out.push(' ');
out.push_str(self.raw(*comment));
out.push('\n');
} else {
ensure_line_break(out);
if self.has_blank_line(cursor, comment_start) {
ensure_blank_line(out);
}
write_indent(out, indent);
out.push_str(self.raw(*comment));
out.push('\n');
}
cursor = comment.end as usize;
}
if previous_end.is_some() {
ensure_line_break(out);
}
if self.has_blank_line(cursor, end) {
ensure_blank_line(out);
}
}
fn precedence(&self, id: ExprId) -> u8 {
match &self.ast.get(id).expr {
Expr::As { .. } => 0,
Expr::Default { .. } => 1,
Expr::Binary { op, .. } => match op {
BinaryOp::Patch => 3,
BinaryOp::And => 5,
BinaryOp::LogicalOr => 7,
BinaryOp::LogicalAnd => 9,
BinaryOp::Equal
| BinaryOp::NotEqual
| BinaryOp::Greater
| BinaryOp::GreaterEqual
| BinaryOp::Less
| BinaryOp::LessEqual => 11,
BinaryOp::Concat => 12,
BinaryOp::Add | BinaryOp::Sub => 13,
BinaryOp::Mul | BinaryOp::Div => 15,
},
Expr::Unary { .. } | Expr::CompareConstraint { .. } => 17,
Expr::Call { .. } | Expr::Path { .. } => 19,
Expr::Parenthesized { .. } => 20,
_ => 20,
}
}
fn is_inline_expr(&self, id: ExprId) -> bool {
let node = self.ast.get(id);
if self.has_comment(node.span) {
return false;
}
match &node.expr {
Expr::Literal(_)
| Expr::Ident(_)
| Expr::RegexConstraint(_)
| Expr::Wildcard
| Expr::Import(_)
| Expr::CompareConstraint { .. } => true,
Expr::Path { base, .. } => self.is_inline_expr(*base),
Expr::Call { callee, args } => {
self.is_inline_expr(*callee)
&& args.iter().all(|argument| self.is_inline_expr(*argument))
}
Expr::Unary { expr, .. } => self.is_inline_expr(*expr),
Expr::Binary { lhs, rhs, .. } => self.is_inline_expr(*lhs) && self.is_inline_expr(*rhs),
Expr::Default { base, fallback } => {
self.is_inline_expr(*base) && self.is_inline_expr(*fallback)
}
Expr::Array(items) => items.iter().all(|item| self.is_inline_expr(*item)),
Expr::ArrayConstraint { item } => self.is_inline_expr(*item),
Expr::MapConstraint { value } => self.is_inline_expr(*value),
Expr::Parenthesized { expr } => self.is_inline_expr(*expr),
Expr::As { narrower, wider } => {
self.is_inline_expr(*narrower) && self.is_inline_expr(*wider)
}
Expr::Object { .. } | Expr::Let { .. } | Expr::Function { .. } | Expr::Match { .. } => {
false
}
}
}
fn delimited_range(
&self,
span: Span,
open_kind: SyntaxTokenKind,
close_kind: SyntaxTokenKind,
) -> (usize, usize) {
let open = self
.tokens_between(span.start as usize, span.end as usize)
.find(|token| same_kind(&token.kind, &open_kind))
.expect("parsed delimited expressions contain an opening token");
let close = self
.tokens_between(open.span.end as usize, span.end as usize)
.rev()
.find(|token| same_kind(&token.kind, &close_kind))
.expect("parsed delimited expressions contain a closing token");
(open.span.end as usize, close.span.start as usize)
}
fn tokens_between(
&self,
start: usize,
end: usize,
) -> impl DoubleEndedIterator<Item = &SyntaxToken> {
self.tokens.iter().filter(move |token| {
token.span.start as usize >= start && token.span.end as usize <= end
})
}
fn comments_between(&self, start: usize, end: usize) -> impl Iterator<Item = &Span> {
self.comments
.iter()
.filter(move |comment| comment.start as usize >= start && comment.end as usize <= end)
}
fn has_comment(&self, span: Span) -> bool {
self.has_comment_between(span.start as usize, span.end as usize)
}
fn has_comment_between(&self, start: usize, end: usize) -> bool {
self.comments_between(start, end).next().is_some()
}
fn same_line(&self, start: usize, end: usize) -> bool {
!self.slice(start, end).contains('\n')
}
fn has_blank_line(&self, start: usize, end: usize) -> bool {
self.slice(start, end)
.bytes()
.filter(|byte| *byte == b'\n')
.count()
>= 2
}
fn raw(&self, span: Span) -> &'a str {
self.slice(span.start as usize, span.end as usize).trim()
}
fn slice(&self, start: usize, end: usize) -> &'a str {
&self.source[start.min(self.source.len())..end.min(self.source.len())]
}
}
fn binary_operator(operator: BinaryOp) -> &'static str {
match operator {
BinaryOp::Add => "+",
BinaryOp::Sub => "-",
BinaryOp::Mul => "*",
BinaryOp::Div => "/",
BinaryOp::Concat => "++",
BinaryOp::Equal => "==",
BinaryOp::NotEqual => "!=",
BinaryOp::Greater => ">",
BinaryOp::GreaterEqual => ">=",
BinaryOp::Less => "<",
BinaryOp::LessEqual => "<=",
BinaryOp::LogicalAnd => "&&",
BinaryOp::LogicalOr => "||",
BinaryOp::And => "&",
BinaryOp::Patch => "//",
}
}
fn compare_operator(operator: CompareOp) -> &'static str {
match operator {
CompareOp::Gt => ">",
CompareOp::Gte => ">=",
CompareOp::Lt => "<",
CompareOp::Lte => "<=",
CompareOp::Eq => "==",
}
}
fn same_kind(left: &SyntaxTokenKind, right: &SyntaxTokenKind) -> bool {
core::mem::discriminant(left) == core::mem::discriminant(right)
}
fn write_indent(out: &mut String, indent: usize) {
out.extend(core::iter::repeat_n(' ', indent));
}
fn ensure_line_break(out: &mut String) {
if !out.is_empty() && !out.ends_with('\n') {
out.push('\n');
}
}
fn ensure_blank_line(out: &mut String) {
if !out.is_empty() && !out.ends_with("\n\n") {
ensure_line_break(out);
out.push('\n');
}
}
fn trim_trailing_whitespace(out: &mut String) {
while out.ends_with(char::is_whitespace) {
out.pop();
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn formats_fields_and_preserves_comments() {
let source =
"Server={\n# host\nhost=String default \"localhost\"; # trailing\n\nport=Int&>0;\n};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"Server = {\n # host\n host = String default \"localhost\"; # trailing\n\n port = Int & > 0;\n};\n"
);
}
#[test]
fn formats_arrays_and_match_comments() {
let source = "result=match x{# c\n_: [1,{a=2;}];};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"result = match x {\n # c\n _: [\n 1,\n {\n a = 2;\n },\n ];\n};\n"
);
}
#[test]
fn formats_array_constraints() {
let source = "tags=[...String];ports=[...(Int&>=1)];";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"tags = [...String];\nports = [...(Int & >= 1)];\n"
);
}
#[test]
fn formats_map_constraints_and_range_ascription() {
let source =
"config={service={port=8080;};}as{service={port=Int;};};labels={a=1;b=2;}as{...Int};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"config = {\n service = {\n port = 8080;\n };\n} as {\n service = {\n port = Int;\n };\n};\nlabels = {\n a = 1;\n b = 2;\n} as {...Int};\n"
);
assert_eq!(format_source(&formatted).unwrap(), formatted);
}
#[test]
fn formats_let_functions_and_precedence_from_the_canonical_ast() {
let source = "value=let x=1+2*3;in(a:Int)=>{result=(x+a)*2;};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"value = let\n x = 1 + 2 * 3;\nin\n (a: Int) =>\n {\n result = (x + a) * 2;\n };\n"
);
}
#[test]
fn formats_unknown_and_object_rest_constraints() {
let source =
"value={hoge=Int;fuga=String;# remaining fields\n...# unconstrained\nUnknown};";
let formatted = format_source(source).unwrap();
assert_eq!(
formatted,
"value = {\n hoge = Int;\n fuga = String; # remaining fields\n ... # unconstrained\n Unknown\n};\n"
);
}
#[test]
fn preserves_regex_and_escaped_strings() {
let source = r#"value={pattern=/^api\/.+$/;text="a\n\"b";};"#;
let formatted = format_source(source).unwrap();
assert!(formatted.contains(r#"pattern = /^api\/.+$/;"#));
assert!(formatted.contains(r#"text = "a\n\"b";"#));
}
#[test]
fn preserves_comments_inside_grouping_and_between_structural_tokens() {
let source = "value = (# grouped\n 1 + 2); result = match x # before brace\n { _ # before body\n : value; };";
let formatted = format_source(source).unwrap();
assert!(formatted.contains("(# grouped\n 1 + 2)"));
assert!(formatted.contains("match x # before brace\n{"));
assert!(formatted.contains("_ # before body\n : value;"));
assert_eq!(format_source(&formatted).unwrap(), formatted);
}
#[test]
fn wasm_export_returns_the_same_formatter_result() {
let source = "value={a=1;};";
let expected = format_source(source).unwrap();
let output: serde_json::Value = serde_json::from_str(&format_source_json(source)).unwrap();
assert_eq!(output["source"], expected);
}
#[test]
fn formats_all_repository_examples_idempotently() {
let examples = [
(
"advanced/main",
include_str!("../../../examples/advanced/main.dcdl"),
),
(
"advanced/profiles",
include_str!("../../../examples/advanced/profiles.dcdl"),
),
(
"advanced/schema",
include_str!("../../../examples/advanced/schema.dcdl"),
),
(
"arithmetic",
include_str!("../../../examples/arithmetic.dcdl"),
),
(
"array-concat",
include_str!("../../../examples/array-concat.dcdl"),
),
(
"array-constraint",
include_str!("../../../examples/array-constraint.dcdl"),
),
("basic", include_str!("../../../examples/basic.dcdl")),
(
"import/main",
include_str!("../../../examples/import/main.dcdl"),
),
(
"import/schema",
include_str!("../../../examples/import/schema.dcdl"),
),
("logical", include_str!("../../../examples/logical.dcdl")),
(
"map-ascription",
include_str!("../../../examples/map-ascription.dcdl"),
),
(
"regex/main",
include_str!("../../../examples/regex/main.dcdl"),
),
];
for (name, source) in examples {
let once = format_source(source).unwrap_or_else(|error| panic!("{name}: {error}"));
let twice = format_source(&once).unwrap_or_else(|error| {
panic!("{name} reformatted to invalid source: {error}\n{once}")
});
assert_eq!(once, twice, "{name}");
}
}
}
+19
View File
@@ -0,0 +1,19 @@
[package]
name = "decodal-lsp"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Language Server Protocol implementation for Decodal."
keywords = ["decodal", "lsp", "language-server", "editor"]
categories = ["development-tools", "text-editors"]
[dependencies]
decodal = { version = "0.4.0", path = "../decodal-core" }
decodal-language-service = { version = "0.4.0", path = "../decodal-language-service" }
decodal-language-tools = { version = "0.4.0", path = "../decodal-language-tools" }
lsp-server = "0.10"
lsp-types = "0.97"
serde_json.workspace = true
File diff suppressed because it is too large Load Diff
+11
View File
@@ -0,0 +1,11 @@
use std::process::ExitCode;
fn main() -> ExitCode {
match decodal_lsp::run_stdio(|_| Ok(decodal_lsp::FileSystemEnvironment::default())) {
Ok(()) => ExitCode::SUCCESS,
Err(error) => {
eprintln!("decodal-lsp failed: {error}");
ExitCode::FAILURE
}
}
}
+18 -5
View File
@@ -1,15 +1,28 @@
[package]
name = "decodal-wasm"
version = "0.1.0"
edition = "2024"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
description = "Host-configurable Decodal evaluator and language service for JavaScript runtimes."
keywords = ["decodal", "wasm", "dsl", "config"]
categories = ["wasm", "config"]
publish = false
[lib]
crate-type = ["cdylib", "rlib"]
[dependencies]
decodal-core = { path = "../decodal-core" }
serde_json = "1"
wasm-bindgen = "0.2"
decodal = { version = "0.4.0", path = "../decodal-core" }
decodal-language-service = { version = "0.4.0", path = "../decodal-language-service" }
serde_json.workspace = true
wasm-bindgen.workspace = true
[target.'cfg(target_arch = "wasm32")'.dependencies]
js-sys = "0.3"
serde-wasm-bindgen = "0.6"
[package.metadata.wasm-pack.profile.release]
wasm-opt = false
+150 -143
View File
@@ -1,25 +1,67 @@
use std::collections::BTreeMap;
use decodal_core::{
Data, Diagnostic, DiagnosticKind, EmptyLoader, Engine, LoadedSource, SourceId, SourceLoader,
Span, format_diagnostic_with,
};
#[cfg(target_arch = "wasm32")]
use decodal::HostEnvironment;
use decodal::{Data, EmptyLoader, Engine, SourceId, format_diagnostic_with};
#[cfg(target_arch = "wasm32")]
use decodal_language_service::{CompletionKind, LanguageService};
use wasm_bindgen::prelude::*;
#[cfg(any(target_arch = "wasm32", test))]
mod value;
#[cfg(target_arch = "wasm32")]
mod web_environment;
#[cfg(target_arch = "wasm32")]
use web_environment::JsEnvironment;
/// Evaluates one standalone Decodal source with no host globals or imports.
#[wasm_bindgen]
pub fn evaluate(source: &str) -> String {
encode_result(evaluate_inner(source))
}
#[wasm_bindgen(js_name = evaluateProject)]
pub fn evaluate_project(entry: &str, files_json: &str) -> String {
encode_result(evaluate_project_inner(entry, files_json))
/// A browser-facing language service configured entirely by its JavaScript host.
///
/// The host owns globals, import loading, and import completion. This keeps
/// filesystem, network, and virtual-project policy outside the WASM package.
#[cfg(target_arch = "wasm32")]
#[wasm_bindgen(js_name = DecodalLanguageService)]
pub struct WebLanguageService {
environment: JsEnvironment,
}
#[cfg(target_arch = "wasm32")]
#[wasm_bindgen(js_class = DecodalLanguageService)]
impl WebLanguageService {
#[wasm_bindgen(constructor)]
pub fn new(options: JsValue) -> Result<WebLanguageService, JsValue> {
Ok(Self {
environment: JsEnvironment::from_options(options)?,
})
}
/// Evaluates a source using the injected globals and import loader.
pub fn evaluate(&self, key: &str, name: &str, source: &str) -> String {
encode_result(evaluate_with_environment(
&self.environment,
key,
name,
source,
))
}
/// Completes a source using the same injected environment as evaluation.
///
/// Positions and returned ranges are UTF-16 offsets, matching browser
/// editors and the Language Server Protocol.
pub fn complete(&self, key: &str, source: &str, position: usize, explicit: bool) -> String {
encode_completion(&self.environment, key, source, position, explicit)
}
}
fn encode_result(result: Result<String, String>) -> String {
match result {
Ok(output) => format!("{{\"ok\":true,\"output\":{}}}", json_string(&output)),
Err(error) => format!("{{\"ok\":false,\"error\":{}}}", json_string(&error)),
Ok(output) => serde_json::json!({ "ok": true, "output": output }).to_string(),
Err(error) => serde_json::json!({ "ok": false, "error": error }).to_string(),
}
}
@@ -40,106 +82,109 @@ fn evaluate_inner(source: &str) -> Result<String, String> {
Ok(format_data(&data, 0))
}
fn evaluate_project_inner(entry: &str, files_json: &str) -> Result<String, String> {
let raw_files: BTreeMap<String, String> = serde_json::from_str(files_json)
.map_err(|error| format!("failed to read playground files: {error}"))?;
let mut files = BTreeMap::new();
for (path, source) in raw_files {
let path = normalize_path(&path).ok_or_else(|| format!("invalid file path `{path}`"))?;
files.insert(path, source);
}
let entry = normalize_path(entry).ok_or_else(|| format!("invalid entry path `{entry}`"))?;
let source = files
.get(&entry)
.cloned()
.ok_or_else(|| format!("entry file `{entry}` was not found"))?;
let mut engine = Engine::new(VirtualLoader { files });
let module = match engine.add_root_source(entry.clone(), entry.clone(), &source) {
#[cfg(target_arch = "wasm32")]
fn evaluate_with_environment(
environment: &JsEnvironment,
key: &str,
name: &str,
source: &str,
) -> Result<String, String> {
let mut engine = environment
.create_engine()
.map_err(|diagnostic| diagnostic.message)?;
let module = match engine.add_root_source(key, name, source) {
Ok(module) => module,
Err(error) => return Err(format_diagnostic_with_root(&error, &entry)),
Err(diagnostic) => return Err(engine.format_diagnostic(&diagnostic)),
};
let value = match engine.eval_module(module) {
Ok(value) => value,
Err(error) => return Err(engine.format_diagnostic(&error)),
Err(diagnostic) => return Err(engine.format_diagnostic(&diagnostic)),
};
let data = match engine.materialize(&value) {
Ok(data) => data,
Err(error) => return Err(engine.format_diagnostic(&error)),
Err(diagnostic) => return Err(engine.format_diagnostic(&diagnostic)),
};
Ok(format_data(&data, 0))
}
#[derive(Debug, Clone)]
struct VirtualLoader {
files: BTreeMap<String, String>,
}
impl SourceLoader for VirtualLoader {
fn load(
&mut self,
current_key: Option<&str>,
specifier: &str,
) -> decodal_core::Result<LoadedSource> {
let key = resolve_import(current_key, specifier).ok_or_else(|| {
Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
format!("invalid import path `{specifier}`"),
)
})?;
let source = self.files.get(&key).cloned().ok_or_else(|| {
Diagnostic::new(
DiagnosticKind::Import,
Span::default(),
format!("import `{specifier}` resolved to `{key}`, but that file does not exist"),
)
})?;
Ok(LoadedSource {
key: key.clone(),
name: key,
source,
})
}
}
fn resolve_import(current_key: Option<&str>, specifier: &str) -> Option<String> {
if specifier.starts_with('/') {
return normalize_path(specifier);
}
let mut base = String::new();
if let Some(current_key) = current_key {
if let Some((parent, _file)) = current_key.rsplit_once('/') {
base.push_str(parent);
base.push('/');
#[cfg(target_arch = "wasm32")]
fn encode_completion(
environment: &JsEnvironment,
key: &str,
source: &str,
position: usize,
explicit: bool,
) -> String {
let byte_position = utf16_offset_to_byte(source, position);
let service = LanguageService::new(environment);
let completion = match service.complete(key, source, byte_position, explicit) {
Ok(completion) => completion,
Err(diagnostic) => {
return serde_json::json!({
"ok": false,
"error": diagnostic.message,
})
.to_string();
}
}
base.push_str(specifier);
normalize_path(&base)
};
let completion = completion.map(|completion| {
let from = byte_offset_to_utf16(source, completion.from);
let options = completion
.items
.into_iter()
.map(|item| {
serde_json::json!({
"label": item.label,
"kind": completion_kind_name(item.kind),
"detail": item.detail,
"priority": item.priority,
})
})
.collect::<Vec<_>>();
serde_json::json!({ "from": from, "options": options })
});
serde_json::json!({ "ok": true, "completion": completion }).to_string()
}
fn normalize_path(path: &str) -> Option<String> {
let mut parts = Vec::new();
let normalized = path.replace('\\', "/");
for part in normalized.split('/') {
match part {
"" | "." => {}
".." => {
parts.pop()?;
}
part => parts.push(part),
#[cfg(target_arch = "wasm32")]
fn completion_kind_name(kind: CompletionKind) -> &'static str {
match kind {
CompletionKind::Keyword => "keyword",
CompletionKind::Constant => "constant",
CompletionKind::Type => "type",
CompletionKind::Variable => "variable",
CompletionKind::Namespace => "namespace",
CompletionKind::Property => "property",
CompletionKind::File => "file",
}
}
#[cfg(any(target_arch = "wasm32", test))]
fn utf16_offset_to_byte(source: &str, offset: usize) -> usize {
let mut utf16 = 0usize;
for (byte, ch) in source.char_indices() {
if utf16 >= offset {
return byte;
}
let next = utf16 + ch.len_utf16();
if next > offset {
return byte;
}
utf16 = next;
}
if parts.is_empty() {
None
} else {
Some(parts.join("/"))
}
source.len()
}
fn format_diagnostic_with_root(diagnostic: &decodal_core::Diagnostic, root_name: &str) -> String {
#[cfg(any(target_arch = "wasm32", test))]
fn byte_offset_to_utf16(source: &str, mut offset: usize) -> usize {
offset = offset.min(source.len());
while !source.is_char_boundary(offset) {
offset = offset.saturating_sub(1);
}
source[..offset].encode_utf16().count()
}
fn format_diagnostic_with_root(diagnostic: &decodal::Diagnostic, root_name: &str) -> String {
format_diagnostic_with(diagnostic, |source| {
(source == SourceId(0)).then_some(root_name)
})
@@ -191,66 +236,28 @@ fn format_data(data: &Data, indent: usize) -> String {
}
fn json_string(value: &str) -> String {
let mut out = String::from("\"");
for ch in value.chars() {
match ch {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
ch if ch.is_control() => {
use core::fmt::Write;
let _ = write!(out, "\\u{:04x}", ch as u32);
}
ch => out.push(ch),
}
}
out.push('"');
out
serde_json::to_string(value).expect("strings are always JSON-serializable")
}
#[cfg(test)]
mod tests {
use super::{evaluate_project_inner, normalize_path, resolve_import};
use super::{byte_offset_to_utf16, evaluate_inner, utf16_offset_to_byte};
#[test]
fn normalizes_virtual_paths() {
fn evaluates_standalone_sources() {
assert_eq!(
normalize_path("/schemas/../main.dcdl"),
Some("main.dcdl".into())
);
assert_eq!(normalize_path("../main.dcdl"), None);
}
#[test]
fn resolves_imports_relative_to_current_file() {
assert_eq!(
resolve_import(Some("schemas/service.dcdl"), "./types.dcdl"),
Some("schemas/types.dcdl".into())
evaluate_inner("value = 1;").unwrap(),
"{\n \"value\": 1\n}"
);
}
#[test]
fn evaluates_project_imports() {
let files = r#"{
"main.dcdl":"let dep = import \"./schemas/service.dcdl\"; in dep.Service & { port = 9443; }",
"schemas/service.dcdl":"Service = { name = String default \"api\"; port = Int & > 443 default 8443; }"
}"#;
let output = evaluate_project_inner("main.dcdl", files).unwrap();
assert!(output.contains("\"port\": 9443"));
}
#[test]
fn project_diagnostics_use_virtual_file_names() {
let files = r#"{
"main.dcdl":"let dep = import \"./schemas/service.dcdl\"; in dep.Service & { port = 80; }",
"schemas/service.dcdl":"Service = { port = Int & > 443 default 8443; }"
}"#;
let error = evaluate_project_inner("main.dcdl", files).unwrap_err();
assert!(error.contains("main.dcdl:"));
assert!(error.contains("schemas/service.dcdl:"));
assert!(!error.contains("source 0:"));
assert!(!error.contains("source 1:"));
fn converts_web_utf16_offsets() {
let source = "a😀β";
assert_eq!(utf16_offset_to_byte(source, 0), 0);
assert_eq!(utf16_offset_to_byte(source, 1), 1);
assert_eq!(utf16_offset_to_byte(source, 3), 5);
assert_eq!(byte_offset_to_utf16(source, 5), 3);
assert_eq!(byte_offset_to_utf16(source, source.len()), 4);
}
}
+320
View File
@@ -0,0 +1,320 @@
use decodal::{CompareOp, Constraint, LiteralValue, PrimitiveType, Value};
use serde_json::{Map, Value as JsonValue};
pub(crate) fn from_json(value: &JsonValue) -> Result<Value, String> {
match value {
JsonValue::Null => Err(String::from("null is not a Decodal value")),
JsonValue::Bool(value) => Ok(Value::bool(*value)),
JsonValue::Number(value) => number(value),
JsonValue::String(value) => Ok(Value::string(value)),
JsonValue::Array(items) => items
.iter()
.enumerate()
.map(|(index, value)| {
from_json(value).map_err(|error| format!("array item {index}: {error}"))
})
.collect::<Result<Vec<_>, _>>()
.map(Value::array),
JsonValue::Object(fields) => object(fields),
}
}
fn number(value: &serde_json::Number) -> Result<Value, String> {
if let Some(value) = value.as_i64() {
return Ok(Value::int(value));
}
if let Some(value) = value.as_u64() {
return i64::try_from(value)
.map(Value::int)
.map_err(|_| String::from("integer value is outside the signed 64-bit range"));
}
value
.as_f64()
.filter(|value| value.is_finite())
.map(Value::float)
.ok_or_else(|| String::from("invalid numeric value"))
}
fn object(fields: &Map<String, JsonValue>) -> Result<Value, String> {
let Some(descriptor) = fields.get("$decodal") else {
return fields
.iter()
.map(|(name, value)| {
from_json(value)
.map(|value| (name.clone(), value))
.map_err(|error| format!("field `{name}`: {error}"))
})
.collect::<Result<Vec<_>, _>>()
.map(Value::object);
};
let descriptor = descriptor
.as_str()
.ok_or_else(|| String::from("`$decodal` must be a descriptor name"))?;
let constraints = parse_constraints(fields.get("constraints"))?;
let default = fields
.get("default")
.map(from_json)
.transpose()?
.map(Box::new);
match descriptor {
"Unknown" => {
let mut all_constraints = vec![Constraint::Unknown];
all_constraints.extend(constraints);
Ok(Value::Range {
constraints: all_constraints,
default,
})
}
"String" | "Int" | "Float" | "Bool" => {
let primitive = match descriptor {
"String" => PrimitiveType::String,
"Int" => PrimitiveType::Int,
"Float" => PrimitiveType::Float,
"Bool" => PrimitiveType::Bool,
_ => unreachable!(),
};
let mut all_constraints = vec![Constraint::Type(primitive)];
all_constraints.extend(constraints);
Ok(Value::Range {
constraints: all_constraints,
default,
})
}
"Array" => {
let item = fields
.get("item")
.ok_or_else(|| String::from("Array descriptor requires `item`"))?;
Ok(Value::ArrayRange {
item: Box::new(from_json(item)?),
constraints,
default,
})
}
"Map" => {
let value = fields
.get("value")
.ok_or_else(|| String::from("Map descriptor requires `value`"))?;
Ok(Value::MapRange {
value: Box::new(from_json(value)?),
constraints,
default,
})
}
"Object" => {
if !constraints.is_empty() {
return Err(String::from(
"Object descriptor does not accept `constraints`; constrain its fields or rest range",
));
}
if default.is_some() {
return Err(String::from(
"Object descriptor does not accept `default`; place defaults on its fields",
));
}
let object_fields = fields
.get("fields")
.and_then(JsonValue::as_object)
.ok_or_else(|| String::from("Object descriptor requires object `fields`"))?;
let value_fields = object_fields
.iter()
.map(|(name, value)| {
from_json(value)
.map(|value| (name.clone(), value))
.map_err(|error| format!("field `{name}`: {error}"))
})
.collect::<Result<Vec<_>, _>>()?;
let rest = fields
.get("rest")
.ok_or_else(|| String::from("Object descriptor requires `rest`"))?;
Ok(Value::object_with_rest(value_fields, from_json(rest)?))
}
"Range" => Ok(Value::Range {
constraints,
default,
}),
name => Err(format!("unknown Decodal value descriptor `{name}`")),
}
}
fn parse_constraints(value: Option<&JsonValue>) -> Result<Vec<Constraint>, String> {
let Some(value) = value else {
return Ok(Vec::new());
};
let items = value
.as_array()
.ok_or_else(|| String::from("`constraints` must be an array"))?;
items
.iter()
.enumerate()
.map(|(index, value)| {
parse_constraint(value).map_err(|error| format!("constraint {index}: {error}"))
})
.collect()
}
fn parse_constraint(value: &JsonValue) -> Result<Constraint, String> {
let fields = value
.as_object()
.ok_or_else(|| String::from("constraint must be an object"))?;
let kind = required_string(fields, "kind")?;
match kind {
"unknown" => Ok(Constraint::Unknown),
"type" => match required_string(fields, "value")? {
"String" => Ok(Constraint::Type(PrimitiveType::String)),
"Int" => Ok(Constraint::Type(PrimitiveType::Int)),
"Float" => Ok(Constraint::Type(PrimitiveType::Float)),
"Bool" => Ok(Constraint::Type(PrimitiveType::Bool)),
value => Err(format!("unknown primitive type `{value}`")),
},
"compare" => {
let operation = match required_string(fields, "op")? {
">" => CompareOp::Gt,
">=" => CompareOp::Gte,
"<" => CompareOp::Lt,
"<=" => CompareOp::Lte,
operation => return Err(format!("unknown comparison operator `{operation}`")),
};
let value = fields
.get("value")
.ok_or_else(|| String::from("compare constraint requires `value`"))?;
Ok(Constraint::Compare(operation, literal(value)?))
}
"regex" => Ok(Constraint::Regex(
required_string(fields, "value")?.to_owned(),
)),
"predicate" => Ok(Constraint::BuiltinPredicate(
required_string(fields, "value")?.to_owned(),
)),
kind => Err(format!("unknown constraint kind `{kind}`")),
}
}
fn literal(value: &JsonValue) -> Result<LiteralValue, String> {
match value {
JsonValue::String(value) => Ok(LiteralValue::String(value.clone())),
JsonValue::Bool(value) => Ok(LiteralValue::Bool(*value)),
JsonValue::Number(value) => match number(value)? {
Value::Int(value) => Ok(LiteralValue::Int(value)),
Value::Float(value) => Ok(LiteralValue::Float(value)),
_ => unreachable!(),
},
_ => Err(String::from("comparison value must be a primitive literal")),
}
}
fn required_string<'a>(fields: &'a Map<String, JsonValue>, name: &str) -> Result<&'a str, String> {
fields
.get(name)
.and_then(JsonValue::as_str)
.ok_or_else(|| format!("constraint requires string `{name}`"))
}
#[cfg(test)]
mod tests {
use decodal::{Constraint, LiteralValue, PrimitiveType, Value};
use super::from_json;
#[test]
fn converts_plain_js_shaped_values() {
let value = serde_json::json!({
"frontmatter": { "draft": false },
"body": "# Hello",
});
assert_eq!(
from_json(&value).unwrap(),
Value::object([
("body", Value::string("# Hello")),
(
"frontmatter",
Value::object([("draft", Value::bool(false))]),
),
])
);
}
#[test]
fn converts_schema_descriptors() {
let value = serde_json::json!({
"$decodal": "Array",
"item": {
"$decodal": "Int",
"constraints": [{ "kind": "compare", "op": ">", "value": 0 }],
},
"default": [1, 2],
});
assert_eq!(
from_json(&value).unwrap(),
Value::ArrayRange {
item: Box::new(Value::Range {
constraints: vec![
Constraint::Type(PrimitiveType::Int),
Constraint::Compare(decodal::CompareOp::Gt, LiteralValue::Int(0)),
],
default: None,
}),
constraints: Vec::new(),
default: Some(Box::new(Value::array([Value::int(1), Value::int(2),]))),
}
);
}
#[test]
fn converts_map_schema_descriptors() {
let value = serde_json::json!({
"$decodal": "Map",
"value": { "$decodal": "Bool" },
"default": {},
});
assert_eq!(
from_json(&value).unwrap(),
Value::MapRange {
value: Box::new(Value::bool_type()),
constraints: Vec::new(),
default: Some(Box::new(Value::object([] as [(&str, Value); 0]))),
}
);
}
#[test]
fn converts_unknown_and_open_object_descriptors() {
let value = serde_json::json!({
"$decodal": "Object",
"fields": {
"name": { "$decodal": "String" },
},
"rest": { "$decodal": "Unknown" },
});
assert_eq!(
from_json(&value).unwrap(),
Value::object_with_rest([("name", Value::string_type())], Value::unknown(),)
);
}
#[test]
fn converts_general_range_descriptors() {
let value = serde_json::json!({
"$decodal": "Range",
"constraints": [{ "kind": "predicate", "value": "available" }],
"default": true,
});
assert_eq!(
from_json(&value).unwrap(),
Value::Range {
constraints: vec![Constraint::BuiltinPredicate("available".into())],
default: Some(Box::new(Value::bool(true))),
}
);
}
#[test]
fn rejects_null_values() {
assert!(
from_json(&serde_json::Value::Null)
.unwrap_err()
.contains("null")
);
}
}
+209
View File
@@ -0,0 +1,209 @@
use std::collections::BTreeMap;
use decodal::{
Diagnostic, DiagnosticKind, Engine, HostEnvironment, ImportCandidate, ImportLoader,
LoadedImport, Span, Value as DecodalValue,
};
use js_sys::{Function, Reflect};
use serde_json::Value;
use wasm_bindgen::{JsCast, JsValue};
use crate::value;
pub(crate) struct JsEnvironment {
globals: BTreeMap<String, DecodalValue>,
load_import: Option<Function>,
complete_import: Option<Function>,
}
impl JsEnvironment {
pub(crate) fn from_options(options: JsValue) -> Result<Self, JsValue> {
if !options.is_null() && !options.is_undefined() && !options.is_object() {
return Err(JsValue::from_str(
"DecodalLanguageService options must be an object",
));
}
let globals = property(&options, "globals")?;
let globals = if globals.is_null() || globals.is_undefined() {
BTreeMap::new()
} else {
let globals: Value = serde_wasm_bindgen::from_value(globals).map_err(|error| {
JsValue::from_str(&format!("failed to read `globals`: {error}"))
})?;
let globals = globals.as_object().ok_or_else(|| {
JsValue::from_str("DecodalLanguageService `globals` must be an object")
})?;
globals
.iter()
.map(|(name, value)| {
value::from_json(value)
.map(|value| (name.clone(), value))
.map_err(|error| {
JsValue::from_str(&format!("invalid global `{name}`: {error}"))
})
})
.collect::<Result<BTreeMap<_, _>, _>>()?
};
Ok(Self {
globals,
load_import: optional_function(&options, "loadImport")?,
complete_import: optional_function(&options, "completeImport")?,
})
}
}
impl HostEnvironment for JsEnvironment {
type Loader = JsLoader;
fn create_loader(&self) -> Self::Loader {
JsLoader {
load_import: self.load_import.clone(),
complete_import: self.complete_import.clone(),
}
}
fn configure_engine(&self, engine: &mut Engine<Self::Loader>) -> decodal::Result<()> {
for (name, value) in &self.globals {
engine.bind_global(name, value.clone())?;
}
Ok(())
}
}
pub(crate) struct JsLoader {
load_import: Option<Function>,
complete_import: Option<Function>,
}
impl ImportLoader for JsLoader {
fn load(
&mut self,
current_key: Option<&str>,
specifier: &str,
) -> decodal::Result<LoadedImport> {
let callback = self.load_import.as_ref().ok_or_else(|| {
import_error(format!(
"no JavaScript `loadImport` callback is configured for `{specifier}`"
))
})?;
let current_key = current_key.map(JsValue::from_str).unwrap_or(JsValue::NULL);
let specifier = JsValue::from_str(specifier);
let loaded = callback
.call2(&JsValue::UNDEFINED, &current_key, &specifier)
.map_err(|error| import_error(js_error_message(error)))?;
let loaded: Value = serde_wasm_bindgen::from_value(loaded)
.map_err(|error| import_error(format!("invalid `loadImport` result: {error}")))?;
loaded_import(&loaded).map_err(import_error)
}
fn complete_import(
&mut self,
current_key: Option<&str>,
prefix: &str,
) -> decodal::Result<Vec<ImportCandidate>> {
let Some(callback) = &self.complete_import else {
return Ok(Vec::new());
};
let current_key = current_key.map(JsValue::from_str).unwrap_or(JsValue::NULL);
let prefix = JsValue::from_str(prefix);
let candidates = callback
.call2(&JsValue::UNDEFINED, &current_key, &prefix)
.map_err(|error| import_error(js_error_message(error)))?;
let candidates: Value = serde_wasm_bindgen::from_value(candidates)
.map_err(|error| import_error(format!("invalid `completeImport` result: {error}")))?;
import_candidates(&candidates).map_err(import_error)
}
}
fn loaded_import(value: &Value) -> Result<LoadedImport, String> {
let fields = value
.as_object()
.ok_or_else(|| String::from("`loadImport` must return an object"))?;
let kind = string_field(fields, "kind")?;
let key = string_field(fields, "key")?;
match kind {
"source" => {
let source = string_field(fields, "source")?;
let name = fields.get("name").and_then(Value::as_str).unwrap_or(key);
Ok(LoadedImport::source(key, name, source))
}
"value" => {
let value = fields
.get("value")
.ok_or_else(|| String::from("value import requires `value`"))?;
let value = value::from_json(value)
.map_err(|error| format!("invalid imported value: {error}"))?;
Ok(LoadedImport::value(key, value))
}
kind => Err(format!("unknown `loadImport` result kind `{kind}`")),
}
}
fn import_candidates(value: &Value) -> Result<Vec<ImportCandidate>, String> {
value
.as_array()
.ok_or_else(|| String::from("`completeImport` must return an array"))?
.iter()
.enumerate()
.map(|(index, value)| match value {
Value::String(specifier) => Ok(ImportCandidate::new(specifier)),
Value::Object(fields) => {
let mut candidate = ImportCandidate::new(string_field(fields, "specifier")?);
if let Some(detail) = fields.get("detail") {
let detail = detail.as_str().ok_or_else(|| {
format!("import candidate {index} `detail` must be a string")
})?;
candidate = candidate.with_detail(detail);
}
Ok(candidate)
}
_ => Err(format!(
"import candidate {index} must be a string or object"
)),
})
.collect()
}
fn string_field<'a>(
fields: &'a serde_json::Map<String, Value>,
name: &str,
) -> Result<&'a str, String> {
fields
.get(name)
.and_then(Value::as_str)
.ok_or_else(|| format!("`loadImport` result requires string `{name}`"))
}
fn property(options: &JsValue, name: &str) -> Result<JsValue, JsValue> {
if options.is_null() || options.is_undefined() {
return Ok(JsValue::UNDEFINED);
}
Reflect::get(options, &JsValue::from_str(name))
}
fn optional_function(options: &JsValue, name: &str) -> Result<Option<Function>, JsValue> {
let value = property(options, name)?;
if value.is_null() || value.is_undefined() {
return Ok(None);
}
value
.dyn_into::<Function>()
.map(Some)
.map_err(|_| JsValue::from_str(&format!("`{name}` must be a function")))
}
fn js_error_message(value: JsValue) -> String {
if let Some(message) = value.as_string() {
return message;
}
Reflect::get(&value, &JsValue::from_str("message"))
.ok()
.and_then(|message| message.as_string())
.unwrap_or_else(|| String::from("JavaScript import callback failed"))
}
fn import_error(message: impl Into<String>) -> Diagnostic {
Diagnostic::new(DiagnosticKind::Import, Span::default(), message)
}
+69
View File
@@ -0,0 +1,69 @@
# Developing Decodal
Development notes live here rather than in the public manual. The manual describes released language behavior and public integration APIs; implementation plans and unresolved design work belong in issues or pull requests.
## Repository map
- `crates/decodal-core`: parser, evaluator, public Rust API, and tests.
- `crates/decodal-derive`: `Decodal` derive macros.
- `crates/decodal-language-service`: host-configurable semantic analysis and completion.
- `crates/decodal-language-tools`: source-preserving formatter and tooling.
- `crates/decodal-lsp`: LSP transport and document synchronization.
- `crates/decodal-wasm`: WebAssembly bindings.
- `packages/decodal-wasm`: generated npm and JSR runtime package.
- `packages/decodal-codemirror`: CodeMirror language support and formatter WebAssembly.
- `editors/lezer-decodal` and `editors/tree-sitter-decodal`: downstream editor grammars.
- `site/decodal-site`: documentation site and playground.
- `doc/manual/source`: English public manual consumed by the site.
- `doc/manual/source/jp`: Japanese translation of the public manual.
## Runtime architecture
The host owns I/O. `ImportLoader` turns an import specifier into DCDL source or a structured `Value`, and `HostEnvironment` installs the same loader and globals for production and editor tooling.
Evaluation proceeds through parse, lazy expression evaluation, and materialization. Object fields, module roots, imports, arguments, and defaults are represented by memoized thunks. A thunk re-entered while it is being evaluated produces a cycle diagnostic.
Runtime values distinguish concrete values from abstract ranges. Composition and `as` preserve that distinction; materialization selects defaults and rejects unresolved ranges or functions. `Data` is the output-only representation.
Diagnostics retain Decodal source spans. Structured host values use their stable import key and logical value path rather than synthetic source spans.
## Checks
Run the release validation commands documented in [`RELEASING.md`](../../RELEASING.md). For a focused change, run the tests for the affected crate or package first, then the full workspace and site checks before release.
The documentation site can be checked without regenerating committed WebAssembly artifacts:
```sh
npm --prefix site/decodal-site test
npm --prefix site/decodal-site run build
```
Use `npm --prefix site/decodal-site run build:wasm` when Rust bindings or formatter behavior changed.
## Documentation site delivery
The canonical site origin is `https://decodal.hareworks.net`, configured through Astro's `site` option. A normal site build generates the sitemap, robots policy, social metadata, structured data, 404 page, and source-backed Markdown endpoints. `/llms.txt` indexes the individual English and Japanese Markdown pages; `/llms-full.txt` and `/ja/llms-full.txt` provide complete manuals. These resources are built directly from `doc/manual/source` and do not use Cloudflare's HTML-to-Markdown conversion.
Deploy the already prepared site with:
```sh
npm --prefix site/decodal-site run deploy
```
Cloudflare Pages `_redirects` handles legacy paths within the site, but not hostname redirects. In the Cloudflare account, configure a Bulk Redirect from `https://decodal-site.pages.dev/` to `https://decodal.hareworks.net/` with status `301`, subpath matching, path-suffix preservation, and query-string preservation enabled. This prevents the Pages project hostname from becoming a second public origin.
## Syntax changes
When syntax changes:
1. Update the public EBNF in `doc/manual/source/language/grammar.md` and the relevant language page.
2. Update the Rust lexer/parser and tests.
3. Update the Lezer and Tree-sitter grammars and regenerate their committed outputs.
4. Update formatter, playground, and language-service tests as applicable.
5. Run the validation commands in `RELEASING.md`.
The Rust parser is authoritative for evaluation. Lezer and Tree-sitter are downstream editor integrations and are not runtime or formatter dependencies.
## Releases
Package versions, validation commands, publish order, and registry commands are maintained only in [`RELEASING.md`](../../RELEASING.md).
@@ -1,125 +0,0 @@
# Composition and Materialization
`&``//``default`、materialize は、runtime value の variant に基づいて処理する。
## `&`
`&` は制約を保った合成である。
```text
compose_and(a: RuntimeValue, b: RuntimeValue) -> RuntimeValue | Diagnostic
```
基本規則:
```text
Abstract(a) & Abstract(b)
-> Abstract {
constraints: a.constraints + b.constraints,
default: merge_default(a.default, b.default)
}
Abstract(a) & Concrete(v)
-> if satisfies(v, a.constraints) then Concrete(v)
else constraint diagnostic
Concrete(v) & Abstract(a)
-> if satisfies(v, a.constraints) then Concrete(v)
else constraint diagnostic
Concrete(Object(a)) & Concrete(Object(b))
-> Concrete(Object(fieldwise_and(a, b)))
Concrete(a) & Concrete(b)
-> if a == b then Concrete(a)
else conflict diagnostic
```
`Abstract & Concrete` が成功した場合、default は消える。
明示値があるなら fallback は不要だからである。
## object の合成
object は concrete structure だが、field の値は thunk 経由で concrete / abstract のどちらにもなりうる。
object 同士の `&` は field ごとに再帰合成する。
```dcdl
MyConfig = {
host = String;
port = Int default 8080;
};
Config = MyConfig & {
host = "localhost";
};
```
`host``Abstract(String) & Concrete("localhost")` として検証され、成功すれば `Concrete("localhost")` になる。
`port` は右辺に明示値がないため、`Abstract(Int, default 8080)` のまま残る。
## default の合成
初期方針では、`&` による異なる default 同士の合成は conflict とする。
同じ default は同一候補として扱ってよい。
```text
merge_default(None, None) -> None
merge_default(Some(a), None) -> Some(a)
merge_default(None, Some(b)) -> Some(b)
merge_default(Some(a), Some(b)) -> if same(a, b) then Some(a) else conflict
```
`//` では右辺 default が左辺 default を置き換える。
## `//`
`//` は右辺優先の deep patch である。
```text
patch(a: RuntimeValue, b: RuntimeValue) -> RuntimeValue
```
基本規則:
- object / object は field ごとに再帰 patch する。
- object / object 以外は右辺で置き換える。
- 左辺にしかない field は保持する。
- 右辺にしかない field は追加する。
- 配列、scalar、function は右辺置換とする。
`//` は制約を保持するための演算子ではない。
制約を満たす具体化には `&` を使う。
## materialize
materialize は runtime value を出力可能な `Data` に変換する。
```text
materialize(RuntimeValue) -> Data | Diagnostic
```
処理規則:
```text
Concrete(String/Int/Float/Bool) -> Data
Concrete(Array(items)) -> each item を force して materialize
Concrete(Object(fields)) -> each field を force して materialize
Concrete(Function) -> materialize 不能
Abstract { constraints, default: Some(d) }
-> force(d)
-> result が constraints を満たすか検証
-> materialize(result)
Abstract { constraints, default: None }
-> 未解決 abstract value として diagnostic
```
materialize は default を採用する唯一の段階である。
通常評価中に明示値が得られた場合、default は採用されない。
## Diagnostic context
Composition and materialization keep source spans for object fields, constraints, defaults, and thunks where possible.
When a conflict occurs, the diagnostic should identify the operation span and related participant spans, such as the left field, right field, constraint, or default value.
For object materialization errors, diagnostics also include the field path being processed when known.
@@ -1,74 +0,0 @@
# Diagnostics and Fallback
エラーは runtime value ではなく diagnostic として扱う。
処理系はエラー内容に基づく汎用的な実行時分岐を提供しない。
## Diagnostic
```text
Diagnostic {
kind: DiagnosticKind
span: Span
message: String
labels: Vec<DiagnosticLabel>
}
DiagnosticLabel {
span: Span
message: String
}
```
`span` は primary location を示す。
表示時には `Span.source` を source id のまま出すのではなく、可能な限り file path や virtual file name に解決する。
`labels` は同じ error に関係する追加 location を示す。
合成や materialize の失敗では、衝突した constraint、value、default、または処理中 field path を label に含める。
代表的な diagnostic kind:
- syntax error
- unresolved identifier
- type mismatch
- constraint violation
- composition conflict
- default conflict
- cycle dependency
- import failure
- match failure
- materialization failure
## エラーは値ではない
評価失敗は `RuntimeValue` ではなく `Diagnostic` を返す。
そのため、通常の式はエラー内容に基づいて分岐できない。
```text
Result<RuntimeValue, Diagnostic>
```
これにより、制約違反、未定義識別子、循環依存、import 失敗などが通常値として流れることを避ける。
## 合成と materialize の diagnostic
合成や materialize の失敗は、以下を示す。
1. どの段階で失敗したか: composition、patch、materialization
2. どの field path を処理中だったか
3. どの constraint、value、default が衝突したか
4. なぜ合成または materialize できないか
例えば default が constraint を満たさない場合は、constraint の位置と default value の位置の両方を label として持つ。
object field の合成で concrete value が衝突する場合は、左辺 field と右辺 field の位置を label として持つ。
## `try / catch` は core に入れない
汎用 `try / catch` は core に入れない。
エラーを制御フローとして扱うと、どの失敗を捕捉できるか、捕捉後の thunk state をどう扱うか、制約違反を握りつぶしてよいか、といった仕様が重くなる。
fallback は有限で明示的な仕組みに限定する。
- `default`: 未指定値の fallback。
- `match`: 有限 pattern に基づく分岐。
Decodal は `unknown` / `any` を持たないため、field 不在や未解決 identifier を fallback 可能な通常値として扱わない。
それらは diagnostic として報告する。
-97
View File
@@ -1,97 +0,0 @@
# Embedding API
Decodal core can be embedded without giving the core crate access to a filesystem.
The host supplies imported sources through `SourceLoader` and may also provide global bindings through the host prelude API.
## Host prelude
`Engine` owns a prelude environment.
Bindings in this environment are visible from every module loaded by the engine.
```text
prelude env
module root env
let / function env
```
Module top-level bindings shadow prelude bindings.
Primitive type names such as `String`, `Int`, `Float`, and `Bool` are handled before environment lookup, so they are reserved and cannot be shadowed by host bindings.
## Global bindings
The host can bind values before adding or evaluating user sources.
```rust
use decodal_core::{EmptyLoader, Engine, HostValue};
let mut engine = Engine::new(EmptyLoader);
engine.bind_global(
"Service",
HostValue::object([
("name", HostValue::string_type()),
("port", HostValue::int_type().gt(443).default_int(8443)?),
("enabled", HostValue::bool_type().default_bool(true)?),
]),
)?;
```
A user source can then refer to `Service` without importing it.
```dcdl
Service & {
name = "api";
port = 9443;
}
```
## HostValue
`HostValue` is the public builder-facing value representation for embedding.
It keeps host code from constructing internal `ThunkId` or `ObjectValue` values directly.
```text
HostValue =
String
Int
Float
Bool
Array(Vec<HostValue>)
Object(Vec<HostField>)
Abstract { constraints, default }
```
When a host value is bound, the engine internalizes it into `RuntimeValue` and allocates value thunks for object fields, array items, and defaults.
## Abstract host objects
A host-provided schema object is represented as a concrete object structure whose fields may contain abstract values.
```rust
HostValue::object([
("name", HostValue::string_type()),
("port", HostValue::int_type().gt(443).default_int(8443)?),
])
```
Conceptually this becomes:
```text
Concrete(Object {
name -> Thunk(Abstract { constraints: [String], default: none })
port -> Thunk(Abstract { constraints: [Int, > 443], default: 8443 })
})
```
This matches the runtime model used for Decodal source-defined schema objects.
## SourceLoader and prelude together
`SourceLoader` and host prelude bindings are independent mechanisms.
- Use `SourceLoader` when user sources should explicitly import host-provided modules.
- Use prelude bindings when host-provided schemas or constants should be globally available.
Both mechanisms share the same runtime evaluator, thunk model, and materialization rules.
@@ -1,150 +0,0 @@
# Execution Pipeline
処理系は、source を AST に変換し、必要な値だけを AST interpreter で評価する。
```text
source
lexer / parser
desugar
register root module
demand-driven evaluation
├─ force thunk
├─ load imported module on demand
├─ evaluate expression
├─ compose `&`
└─ patch `//`
materialize
data / diagnostics
```
## lexer / parser
lexer / parser は source を AST に変換する。
構文エラーはこの段階で diagnostic として報告する。
AST は arena に格納し、式や pattern は `ExprId``PatternId` のような ID で参照する。
## desugar
desugar は、意味論を単純にするための表層構文変換を行う。
例として、dot-path field は nested object に変換できる。
```dcdl
{
feature_hoge.enable = false;
}
```
```dcdl
{
feature_hoge = {
enable = false;
};
}
```
この変換により、評価器は object field の再帰構造だけを扱えばよい。
## module registry
module registry は、読み込んだ module を loader が返す安定 key で管理する。
CLI では canonical path を key とする。
組み込み利用では、resource name や static source table の key を使える。
```text
ModuleRegistry:
ModuleKey -> ModuleId
```
処理系は、まず root module を parse / desugar して registry に登録する。
import 先 module は、この段階で全て読み込む必要はない。
import expression が評価されたとき、処理系は `SourceLoader` に現在の module key と import specifier を渡す。
loader は module key、表示名、source text を返す。
module registry は key が未登録なら対象 module を parse / desugar して登録する。
登録された module は module root thunk を持つ。
同じ module が複数回 import された場合は、同じ `ModuleId` を返す。
つまり import は module を即時評価しない。
module を読み込み、module root を thunk として登録するだけにする。
AST の `ExprId` は module-local である。
そのため runtime が保持する式参照は `ExprRef { module, expr }` として module-qualified にする。
## demand-driven evaluation
評価器は、必要になった thunk だけを force する。
未参照の field、let binding、import 先の field は評価しない。
この方式により、module 間に循環 import があっても、実際に force された thunk の依存が循環しない限り評価できる。
## composition
composition は、module 全体に後からかける global pass ではない。
`&``//` の式を評価するときに、demand-driven evaluation の中で呼ばれる演算である。
```text
eval(A & B):
a = eval(A)
b = eval(B)
compose_and(a, b)
eval(A // B):
a = eval(A)
b = eval(B)
patch(a, b)
```
`&` は制約を保った合成を行い、`//` は右辺優先の deep patch を行う。
詳細は [Composition and Materialization](./composition-and-materialization.md) に置く。
## resolver / binder
resolver / binder は初期実装では必須ではない。
評価時に environment lookup を行えば、識別子参照は実装できる。
ただし、将来的には optional phase として追加できる余地を残す。
```text
source
lexer / parser
desugar
resolver / binder
register root module
demand-driven evaluation
```
resolver / binder を追加すると、以下を早期に診断しやすくなる。
- 未定義識別子
- shadowing の扱い
- reserved word の扱い
- symbol interning
- import path の一部静的解決
- span 付き diagnostic の精度向上
ただし、Decodal の制約検証は独立した type checking pass ではなく、`&` の合成時や materialize 時に行う。
## materialize
通常の評価結果は runtime value であり、抽象値や default を含みうる。
外部へ出力するときだけ materialize を行い、出力可能な data に変換する。
materialize は以下を行う。
- 必要な thunk を force する。
- abstract value の default を必要に応じて force する。
- concrete value が constraint を満たすか検証する。
- 未解決の abstract value、function value などを diagnostic にする。
-67
View File
@@ -1,67 +0,0 @@
# Features
Decodal は embedded use と小さい runtime を優先する。
言語機能を追加するときは、値の合成・検証・materialization に直接必要なものを core に残し、重い依存や高度な推論は optional feature または外部 tooling に分ける。
## Core feature boundary
Core に入れる機能は、基本的に deterministic な value transformation に限る。
- arithmetic / logical / comparison operators
- array concat
- object / constraint composition
- default materialization
- pure function evaluation
- host supplied import evaluation
Core に入れないものは以下である。
- filesystem / network / environment access
- time / random
- mutation
- reflection or existence probing
- arbitrary host function calls
- symbolic constraint solving beyond simple normalization
未解決 identifier や missing field は `unknown` として流れず、diagnostic になる。
この方針により、存在チェックや optional chaining のような dynamic object inspection は core language の対象外とする。
## Constraint reasoning
Constraint normalization は軽量な範囲に留める。
primitive type conflict や明らかな numeric bound conflict は合成時に検出してよい。
一方で、symbolic arithmetic、boolean algebra、regex intersection、array length dependent typing のような重い推論は行わない。
評価済みの concrete value に対する検証は runtime / materialization で行う。
静的に完全な型検査フェーズを増やすのではなく、parse、evaluate、compose、materialize の各段階で自然に分かる error を diagnostic として返す。
## Core defaults
`decodal-core` defaults to `std` only.
```toml
[features]
default = ["std"]
std = []
regex = ["std", "dep:regex"]
```
Building `decodal-core` with `--no-default-features` keeps the core in `no_std + alloc` mode and avoids optional dependencies.
## Regex
Regex constraints are implemented behind the `regex` feature.
When the feature is disabled, regex constraints parse and compose, but validating a concrete value against them returns an unsupported feature diagnostic.
```sh
cargo run -q -p decodal --features regex -- examples/regex/main.dcdl
```
Regex constraints are accumulated during `&` composition.
The implementation does not try to prove whether the intersection of two regex constraints is empty.
Concrete strings must match every regex constraint attached to the abstract value.
## CLI features
`decodal-cli` exposes a matching `regex` feature that enables `decodal-core/regex`.
The feature is not enabled by default so the default CLI binary remains small.
-19
View File
@@ -1,19 +0,0 @@
# 処理系設計
この章では、言語仕様を実装するための処理系モデルを定義する。
言語仕様そのものは [Language Specification](../language/index.md) に置き、この章では AST interpreter、遅延評価、thunk、runtime value、materialize の実装方針を扱う。
## 方針
初期処理系は AST interpreter として実装する。
bytecode VM や JIT ではなく、AST を demand-driven に評価することで、遅延評価、循環参照、`default``&``//` の意味論を小さく実装する。
## 構成
1. [Execution Pipeline](./execution-pipeline.md)
2. [Runtime Model](./runtime-model.md)
3. [Thunk and Lazy Evaluation](./thunk-and-lazy-evaluation.md)
4. [Composition and Materialization](./composition-and-materialization.md)
5. [Diagnostics and Fallback](./diagnostics-and-fallback.md)
6. [Embedding API](./embedding-api.md)
7. [Features](./features.md)
-163
View File
@@ -1,163 +0,0 @@
# Runtime Model
処理系の評価結果は、具体値と抽象値を区別した runtime value として扱う。
`value``constraints``default` を横並びに持つ構造にはしない。
## RuntimeValue
```text
RuntimeValue =
Concrete(ConcreteValue)
Abstract(AbstractValue)
```
`Concrete` は明示的な値である。
`Abstract` は、まだ具体値に確定していない制約付きの値である。
Decodal は `unknown``any``null` のような「存在するが意味が未確定な値」を runtime value として持たない。
識別子や field が解決できない場合は、その場で diagnostic になる。
未解決値を後続の演算へ流して推論することはしない。
## ConcreteValue
```text
ConcreteValue =
String(String)
Int(i64)
Float(f64)
Bool(bool)
Array(Vec<ThunkId>)
Object(ObjectValue)
Function(FunctionValue)
```
object は concrete structure として扱う。
ただし、各 field の中身は concrete value でも abstract value でもよい。
```text
ObjectValue:
fields: Map<Symbol, ObjectField>
ObjectField:
value: ThunkId
span: Span
```
例えば以下の schema object は、object 自体は concrete だが、field の値は abstract value になる。
```dcdl
MyConfig = {
host = String;
port = Int default 8080;
};
```
概念的には以下である。
```text
Concrete(Object {
host -> Thunk(Abstract { constraints: [String], default: none })
port -> Thunk(Abstract { constraints: [Int], default: 8080 })
})
```
## AbstractValue
```text
AbstractValue {
constraints: Vec<ConstraintEntry>
default: Option<ThunkId>
}
ConstraintEntry {
constraint: Constraint
span: Span
}
```
`default``AbstractValue` にだけ存在する。
明示的な concrete value がある場合、default は保持しない。
```dcdl
port = Int default 8080;
```
これは以下の runtime value になる。
```text
Abstract {
constraints: [Type(Int)]
default: Some(Thunk(8080))
}
```
```dcdl
port = 8000;
```
これは以下である。
```text
Concrete(Int(8000))
```
## Runtime scope
Decodal runtime は application runtime ではなく、pure value evaluator である。
同じ source、同じ import sources、同じ host globals が与えられた場合、評価結果は決定的である。
runtime が扱う責務は以下に限る。
- expression を評価する。
- thunk を必要に応じて force する。
- concrete / abstract value を合成する。
- materialize 時に constraint を検証する。
runtime は filesystem、network、environment variable、time、random、mutation を扱わない。
core における import は host supplied source を受け取る境界であり、filesystem access ではない。
## Constraint
constraint は concrete value とは別の型として扱う。
```text
Constraint =
Type(PrimitiveType)
Compare(Op, Literal)
Regex(Pattern)
BuiltinPredicate(Symbol)
ObjectConstraint(...)
```
初期実装では、object の形は主に `Concrete(Object)` の field に `Abstract` を置くことで表現する。
object 全体にかかる constraint は必要になった時点で追加する。
## Data
materialize 後の出力可能な値は runtime value とは別型にする。
```text
Data =
String(String)
Int(i64)
Float(f64)
Bool(bool)
Array(Vec<Data>)
Object(Map<Symbol, Data>)
```
`Function`、未解決の `Abstract`、未評価の thunk は `Data` にはならない。
## 命名
実装内部では `RuntimeValue` を短く `Value` と呼んでもよい。
ただし、materialize 後の出力値とは区別する。
推奨する区別:
```text
RuntimeValue / Value 言語内部の評価結果。Abstract を含む。
ConcreteValue 明示的な具体値。
AbstractValue constraint と default を持つ抽象値。
Data 外部へ出力可能な最終データ。
```
@@ -1,88 +0,0 @@
# Thunk and Lazy Evaluation
thunk は、まだ評価していない式をあとで評価できるように包んだ遅延計算である。
この処理系では、循環検出と memoize の単位として thunk を使う。
## Thunk
```text
Thunk {
expr: ExprRef
env: EnvId
state: ThunkState
}
ExprRef {
module: ModuleId
expr: ExprId
}
ThunkState =
Unevaluated
Evaluating
Evaluated(RuntimeValue)
Error(Diagnostic)
```
`expr` は評価対象の AST node を module-qualified に指す。
`ExprId` は module-local な ID なので、runtime では `ModuleId` と組み合わせた `ExprRef` を保持する。
`env` は、その式を評価するときに使う lexical environment を指す。
式だけではなく environment も保持するのは、遅延評価された式が定義時の名前解決文脈を必要とするためである。
## force
thunk を評価する操作を force と呼ぶ。
```text
force(thunk):
Unevaluated -> Evaluating -> Evaluated(value)
Evaluated(value) -> value
Evaluating -> cycle diagnostic
Error(diagnostic) -> diagnostic
```
一度 `Evaluated` になった thunk は memoize される。
同じ thunk を複数回 force しても、式は一度だけ評価される。
## 循環検出
評価中の thunk を再度 force しようとした場合は循環依存である。
```dcdl
{
a = b;
b = a;
}
```
`a` を force すると、`a -> b -> a` と戻る。
このとき `a``Evaluating` なので cycle diagnostic を返す。
## 遅延評価の単位
thunk は主に以下に使う。
- module root
- object field
- let binding
- function argument
- default expression
object は field ごとに thunk を持つ。
そのため、object の一部だけが必要な場合、他の field は評価されない。
## module import
import は module を登録するが、module 全体を即時評価しない。
module root や field は thunk として保持され、参照されたときだけ force される。
これにより、module 間に循環 import があっても、force された thunk の依存が循環しなければ評価できる。
## function call
関数引数は thunk として関数の environment に束縛する。
関数本体で引数が参照されたときだけ force する。
任意の関数呼び出し結果をグローバルに memoize する必要はない。
field に束縛された関数呼び出し結果は、その field thunk の評価結果として memoize される。
-138
View File
@@ -1,138 +0,0 @@
# Development
This document describes the development workflow for Decodal itself.
## Rust checks
Run the normal Rust checks from the repository root.
```sh
cargo fmt --check
cargo test
cargo check -p decodal-core --no-default-features
nix flake check
```
Regex support is optional and should be tested explicitly when touched.
```sh
cargo test -p decodal-core --features regex
cargo run -q -p decodal --features regex -- examples/regex/main.dcdl
```
## Web site and playground
The Astro documentation site and browser playground are kept in:
```text
site/decodal-site/
```
The site imports Markdown files from `doc/manual/souce/` and renders them as mdBook-style pages.
The playground loads `decodal-wasm` and evaluates DCDL entirely in the browser.
Important files:
```text
site/decodal-site/src/pages/docs/[...slug].astro
site/decodal-site/src/pages/playground.astro
site/decodal-site/src/layouts/ManualLayout.astro
site/decodal-site/src/lib/docs.js
crates/decodal-wasm/src/lib.rs
```
Build the WebAssembly package before building the site:
```sh
cd site/decodal-site
npm install
npm run build:wasm
npm run build
```
`npm run build:wasm` writes generated files into:
```text
site/decodal-site/src/wasm/
```
These generated files are committed so the site can be built without requiring every consumer to regenerate the wasm package first.
To run the site locally:
```sh
cd site/decodal-site
npm run dev
```
## Tree-sitter grammar
The Tree-sitter grammar is kept in:
```text
editors/tree-sitter-decodal/
```
Important files:
```text
editors/tree-sitter-decodal/grammar.js
editors/tree-sitter-decodal/queries/highlights.scm
editors/tree-sitter-decodal/queries/locals.scm
editors/tree-sitter-decodal/corpus/basic.txt
```
The grammar is intended to be portable across editors such as Zed, Neovim, Helix, and Emacs.
Zed support should consume this grammar rather than relying on a TextMate grammar.
## Tree-sitter commands
From the grammar directory:
```sh
cd editors/tree-sitter-decodal
npm install
npx tree-sitter generate
npx tree-sitter test
```
To inspect a parse tree:
```sh
npx tree-sitter parse ../../examples/advanced/main.dcdl
```
The generated parser files under `editors/tree-sitter-decodal/src/` are committed so editor integrations can consume the grammar without regenerating it first.
`node_modules/` is ignored and must not be committed.
## Updating the grammar
When the Decodal syntax changes:
1. Update `grammar.js`.
2. Run `npx tree-sitter generate`.
3. Add or update corpus tests in `corpus/`.
4. Update highlight queries in `queries/highlights.scm` if token names changed.
5. Run `npx tree-sitter test`.
6. Run Rust checks from the repository root if the language parser or examples also changed.
## Development shell
The Nix development shell includes Rust tooling, Node.js, Tree-sitter CLI tooling, and wasm-pack tooling.
```sh
nix develop
```
The shell provides:
- `cargo`
- `rustc`
- `rustfmt`
- `clippy`
- `node`
- `npm`
- `wasm-pack`
- `lld`
- `tree-sitter`
- `nixfmt`
-47
View File
@@ -1,47 +0,0 @@
# Decodal Manual
このディレクトリには、Decodal のマニュアル文書を置く。
Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェクト名である。
## 目次
1. [Introduction](./introduction.md)
2. [Language Specification](./language/index.md)
1. [Lexical Structure and Syntax](./language/syntax.md)
2. [Value](./language/value/index.md)
1. [String](./language/value/string.md)
2. [Int](./language/value/int.md)
3. [Float](./language/value/float.md)
4. [Bool](./language/value/bool.md)
3. [Expression](./language/expression/index.md)
1. [Literal](./language/expression/literal.md)
2. [Identifier](./language/expression/identifier.md)
3. [Path Reference](./language/expression/path-reference.md)
4. [Object](./language/expression/object.md)
5. [Array](./language/expression/array.md)
6. [Function](./language/expression/function.md)
7. [Function Call](./language/expression/function-call.md)
8. [Let](./language/expression/let.md)
9. [Match](./language/expression/match.md)
10. [Import](./language/expression/import.md)
11. [Composition](./language/expression/composition.md)
12. [Default](./language/expression/default.md)
13. [String Interpolation](./language/expression/string-interpolation.md)
4. [Constraints and Defaults](./language/constraints-and-defaults.md)
5. [Composition Operators](./language/operators.md)
6. [Functions](./language/functions.md)
7. [Modules and Imports](./language/modules-and-imports.md)
8. [Evaluation Semantics](./language/evaluation.md)
9. [Materialization and Errors](./language/materialization-and-errors.md)
10. [Naming Conventions](./language/naming.md)
11. [Examples](./language/examples.md)
3. [Implementation Design](./design/index.md)
1. [Execution Pipeline](./design/execution-pipeline.md)
2. [Runtime Model](./design/runtime-model.md)
3. [Thunk and Lazy Evaluation](./design/thunk-and-lazy-evaluation.md)
4. [Composition and Materialization](./design/composition-and-materialization.md)
5. [Diagnostics and Fallback](./design/diagnostics-and-fallback.md)
6. [Embedding API](./design/embedding-api.md)
7. [Features](./design/features.md)
4. [Development](./development.md)
5. [Open Issues](./open-issues.md)
-94
View File
@@ -1,94 +0,0 @@
# Introduction
このマニュアルは、Decodal の目的、設計方針、言語仕様をまとめる。
Decodal は **Deferred Constraint Data Language**、略称 **DCDL** のプロジェクト名である。
ファイル拡張子は `.dcdl` とする。
Decodal は、設定値・スキーマ・制約・派生設定を同じ式体系で扱い、組み込み環境でも実装しやすい小さな言語核を提供することを目指す。
## 目的
一般的な設定ファイルでは、値の記述、スキーマ定義、デフォルト値、派生設定、バリデーションが別々の仕組みとして扱われやすい。
この言語では、それらを単一の式体系に寄せる。
例えば、以下のように制約と値を同じ構文で合成できる。
```dcdl
Port = Int & >= 1 & <= 65535;
NarrowedPort = Port & > 443;
MyConfig = {
host = String;
port = NarrowedPort default 8080;
};
Config = MyConfig & {
host = "127.0.0.1";
port = 8000;
};
```
`MyConfig` は設定の形と制約を表し、`Config` はそこへ具体値を合成した設定を表す。
具体値は対応する制約を満たす必要がある。
## 設計目標
- データ記述とスキーマ記述を同じ構文で表現できる。
- 制約を `&` で合成し、値が制約を満たすか検証できる。
- 設定値やスキーマを `//` で構造的に patch できる。
- `default` により、最終評価時の fallback 値を定義できる。
- フィールド単位の遅延評価により、未使用値の評価を避ける。
- import 循環があっても、必要なフィールド依存が循環していなければ評価できる。
- 処理系を組み込み向けに小さく保てるよう、意味論を明示的かつ決定的にする。
## 非目標
初期仕様では以下を必須にしない。
- 高度な型推論。
- match の完全な網羅性検査。
- 到達不能分岐の静的検査。
- 任意の関数呼び出し結果のグローバル memoize。
- 正規表現エンジンの必須搭載。
- 汎用 `try / catch` の core 搭載。
- 完全なプログラミング言語としての汎用性。
## 中心概念
この言語の中心概念は以下である。
- 値と制約を同じ式として扱う。
- `&` で制約を保った合成を行う。
- `//` で右辺優先の構造的 patch を行う。
- `default` は制約ではなく、最終評価時の fallback として扱う。
- フィールド単位で遅延評価する。
- import 循環は、実際に必要なフィールド依存が循環しない限り許容する。
## 組み込み向けの方針
この言語は、汎用プログラミング言語を目指すものではない。
主対象は、設定、スキーマ、制約、派生データの記述である。
そのため、言語核は「値・制約・構造の合成」と「遅延評価」に寄せる。
便利な機能であっても、実装サイズ・評価モデル・エラー決定性を大きく複雑にするものは optional feature または将来拡張として扱う。
## ドキュメント構成
言語仕様の解説は [Language Specification](./language/index.md) にまとめる。
`language/` 配下には、構文、値、式、制約、演算子、評価意味論など、言語仕様そのものの説明だけを置く。
処理系の設計は [Implementation Design](./design/index.md) にまとめる。
ここでは、AST interpreter、runtime value、thunk、合成処理、materialize、diagnostic の扱いを説明する。
主な章は以下である。
- [Value](./language/value/index.md): `String``Int``Float``Bool` などの値・プリミティブ制約。
- [Expression](./language/expression/index.md): literal、object、array、function、let、match、import などの式。
- [Constraints and Defaults](./language/constraints-and-defaults.md): 制約と `default` の意味。
- [Composition Operators](./language/operators.md): `&``//` の意味。
- [Evaluation Semantics](./language/evaluation.md): 遅延評価、thunk、循環検出。
- [Materialization and Errors](./language/materialization-and-errors.md): 最終評価とエラー分類。
- [Runtime Model](./design/runtime-model.md): concrete value と abstract value の内部表現。
- [Thunk and Lazy Evaluation](./design/thunk-and-lazy-evaluation.md): 遅延計算と循環検出の処理系モデル。
未確定事項は [Open Issues](./open-issues.md) に集約する。
@@ -1,198 +0,0 @@
# 制約と default
この章では、制約と `default` の意味を定義する。
## 制約
制約は、値が満たすべき条件を表す。
```dcdl
Int
String
>= 1
<= 65535
/Hello! .*/
```
制約は `&` により合成できる。
```dcdl
Port = Int & >= 1 & <= 65535;
NarrowedPort = Port & > 443;
```
制約合成の意味は、すべての制約を同時に満たすことである。
```text
A & B = A と B の両方を満たす値または制約
```
矛盾する制約はエラーになる。
```dcdl
Int & String # エラー
> 10 & < 5 # エラー
Int & > 10 & < 11 # エラー。整数値の候補が存在しない
```
## 制約の正規化
`&` によって abstract value 同士を合成した場合、処理系は軽量に判定できる制約を正規化する。
正規化対象:
- primitive type 制約。
- 数値比較制約。
primitive type 制約は、異なる型が同時に要求された場合 conflict になる。
```dcdl
Int & Float
Int & String
```
数値比較制約は上下限として正規化される。
```dcdl
Int & >= 1 & <= 65535 & > 443
```
これは概念的に以下へ正規化される。
```text
Type(Int)
> 443
<= 65535
```
上下限の交差が空であれば conflict になる。
`Int` 制約がある場合は、整数候補が存在するかも判定する。
```dcdl
> 10 & < 5 # conflict
Int & > 10 & < 11 # conflict
Int & >= 10 & <= 10 # OK
```
`Int` の比較制約は整数リテラルを使う。
`Float` の比較制約は整数リテラルまたは浮動小数リテラルを使える。
## 組み込み制約
最小の組み込み制約は以下である。
```dcdl
String
Int
Float
Bool
```
追加の述語制約はライブラリまたは組み込みとして提供できる。
```dcdl
IPv4Address
```
## 正規表現制約
正規表現リテラルは文字列制約として使える。
```dcdl
Host = /^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/;
```
正規表現制約は積み重ね可能である。
複数の正規表現制約が同じ abstract value に付与された場合、具体文字列はすべての正規表現制約に一致しなければならない。
```dcdl
String & /^a/ & /z$/
```
処理系は、正規表現制約同士の交差が空であるかを合成時に判定する必要はない。
つまり、以下は合成時には conflict にならず、具体値検証時に失敗する。
```dcdl
String & /^a$/ & /^b$/
```
正規表現エンジンは optional feature にできる。
正規表現 feature が無効な処理系では、正規表現制約の検証は unsupported feature diagnostic になる。
軽量実装では代表的な制約を組み込み述語として提供してもよい。
```dcdl
Host = IPv4Address;
```
## default
`default` は制約ではない。
`default` は、最終評価時に明示値が存在しない場合だけ使われる fallback 値である。
```dcdl
port = NarrowedPort default 8080;
```
これは概念的には以下を表す。
```text
Abstract {
constraints: [NarrowedPort]
default: 8080
}
```
明示値が合成された場合、`default` は採用されない。
```dcdl
MyConfig = {
port = NarrowedPort default 8080;
};
Config = MyConfig & {
port = 8000;
};
```
この場合、最終値は `8000` である。
`8080` は評価されない、または評価されても採用されない。
明示値がない場合、最終 materialize 時に `default` が採用される。
採用された default 値は、同じフィールドに定義された制約を満たす必要がある。
```text
Abstract {
constraints: [NarrowedPort]
default: 8080
}
finalize => 8080 が NarrowedPort を満たせば成功
```
## default の内部表現
`default` は abstract value に付随する fallback thunk として保持できる。
これにより、default 値自体も必要になるまで評価しない。
```text
RuntimeValue =
Concrete(ConcreteValue)
Abstract {
constraints: Vec<Constraint>
default: Option<Thunk>
}
```
明示値は `Concrete` として表現し、`default` を保持しない。
`Abstract & Concrete` が成功した場合、制約検証後に `Concrete` になり、default は消える。
## default の合成
同じフィールドに複数の `default` が合成された場合の詳細規則は未確定である。
現時点の単純な方針は以下である。
- `&` による default 同士の衝突はエラーにする。
- 同一 default 値は許可してよい。
- `//` による patch では右辺 default が左辺 default を置き換える。
この方針により、`&` は制約を保った合成、`//` は上書き操作として説明できる。
-78
View File
@@ -1,78 +0,0 @@
# 遅延評価
この言語はフィールド単位で遅延評価する。
## 基本方針
```dcdl
{
schema = {
hoge = String;
};
result = expensive(schema);
}
```
`schema` のみが必要な場合、`result` は評価されない。
## thunk
各フィールドや let 束縛は thunk として保持できる。
```text
Thunk {
expr: ExprId
env: EnvRef
state: Unevaluated | Evaluating | Evaluated(Value) | Error
}
```
評価済み thunk は memoize する。
同じフィールドを複数回参照しても、評価は一度だけでよい。
## 評価状態
thunk は以下の状態を持つ。
```text
Unevaluated 未評価
Evaluating 評価中
Evaluated 評価済み
Error 評価失敗
```
`Evaluating` の thunk を再度評価しようとした場合、循環依存として扱う。
## 循環検出
```dcdl
{
a = b + 1;
b = a + 1;
}
```
この場合、`a` または `b` を評価すると循環エラーになる。
一方、同じモジュール内または import 間に循環があっても、評価対象のフィールドが循環していなければ成功する。
## 評価と materialize の分離
通常の評価では、制約や default を含む中間値が残ることがある。
外部へデータとして出力する段階で materialize を行う。
この分離により、以下が可能になる。
- スキーマを値として扱う。
- default を必要になるまで評価しない。
- import されたモジュールの未使用フィールドを評価しない。
- 制約だけのフィールドを中間状態として保持する。
## 関数呼び出しとの関係
関数引数は thunk として渡せる。
関数本体内で引数が参照されたときに評価する。
任意の関数呼び出し結果をグローバルに memoize することは必須ではない。
ただし、フィールドに束縛された呼び出し結果は、そのフィールド thunk の評価結果として memoize される。
-142
View File
@@ -1,142 +0,0 @@
# 例
この章には、仕様を説明するための例を置く。
## 基本的な設定スキーマ
```dcdl
rec {
Host = IPv4Address;
Port = Int & >= 1 & <= 65535;
NarrowedPort = Port & > 443;
MyConfig = {
host = Host;
port = NarrowedPort default 8080;
feature_hoge = {
enable = Bool default true;
fuga = Int default 10;
};
};
}
NewConfig = MyConfig & {
host = "127.0.0.1";
port = 8000;
};
disabled_config = NewConfig & {
feature_hoge.enable = false;
};
enabled_config = NewConfig;
```
## 関数と文字列生成
```dcdl
let
maybe_hw = String & /Hello! .*/;
part = {
greet = String;
target = String;
};
mk_hw = (part: part) =>
maybe_hw & "${part.greet}! ${part.target}";
in
mk_hw({
greet = "Hello";
target = "World";
})
```
評価結果:
```text
"Hello! World"
```
## match
```dcdl
(
input_a: {
hoge = Int & >= 0;
},
input_b: {
fuga = 20;
}
) =>
let
inputs = {
a = input_a;
b = input_b;
};
in
{
foo = match inputs.a.hoge {
>= 20: {
value = 200;
};
>= 10: {
value = 100;
};
_: {
value = 300;
};
};
}
```
`match` は上から順に評価されるため、広い条件より狭い条件を先に書く。
## deep patch
```dcdl
Base = {
feature_hoge = {
enable = Bool default true;
fuga = Int default 10;
};
};
Patched = Base // {
feature_hoge.enable = false;
};
```
`Patched` は以下に相当する。
```dcdl
{
feature_hoge = {
enable = false;
fuga = Int default 10;
};
}
```
## 循環 import
```dcdl
# main.n
{
schema = {
hoge = String;
};
result = (import ./func.n)(schema);
}
```
```dcdl
# func.n
(input: (import ./main.n).schema) =>
{
# ...
}
```
`func.n``main.n` を import しているが、参照しているのは `main.schema` である。
`main.schema``main.result` に依存していなければ、この循環 import は成立する。
@@ -1,27 +0,0 @@
# Array Expression
array expression は、順序付きの値の列を表す。
```dcdl
[1, 2, 3]
["a", "b", "c"]
```
## Array concat
`++` は concrete array 同士を連結する。
```dcdl
base = ["read", "write"];
extra = ["admin"];
roles = base ++ extra;
```
`roles` は以下と同じ値になる。
```dcdl
["read", "write", "admin"]
```
`++` は配列要素を変換しない。
左辺の要素の後に右辺の要素が並ぶ。
@@ -1,15 +0,0 @@
# Default Expression
`default` expression は、明示値が存在しない場合に materialize 時に採用される fallback を指定する。
```dcdl
port = Int default 8080;
```
`default` は制約ではない。
詳細は [制約と default](../constraints-and-defaults.md) に置く。
## 評価
fallback 値は thunk として保持できる。
明示値がある場合、default は採用されない。
@@ -1,17 +0,0 @@
# Function Call Expression
function call expression は、関数値を引数に適用する式である。
```dcdl
mk_hw({
greet = "Hello";
target = "World";
})
```
## 評価
引数は thunk として渡せる。
関数本体内で引数が参照されたときに評価する。
任意の関数呼び出し結果をグローバルに memoize することは必須ではない。
@@ -1,18 +0,0 @@
# Function Expression
function expression は、引数を受け取り式を返す値である。
```dcdl
(part: {
greet = String;
target = String;
}) =>
"${part.greet}! ${part.target}"
```
関数仕様の詳細は [関数](../functions.md) に置く。
## 評価
関数は定義時の環境を参照として保持する。
関数本体は、関数値の生成時ではなく呼び出し時に評価される。
@@ -1,14 +0,0 @@
# Identifier Expression
identifier expression は、現在の環境に束縛された名前を参照する式である。
```dcdl
Port
MyConfig
mkConfig
```
## 評価
識別子は、対応する束縛の thunk を参照する。
束縛が存在しない場合は未定義識別子エラーになる。
@@ -1,15 +0,0 @@
# Import Expression
import expression は、外部ファイルを読み込み、そのファイルの評価結果を返す。
```dcdl
import ./config.n
import "./config.n"
```
import 仕様の詳細は [モジュールと import](../modules-and-imports.md) に置く。
## 評価
import 先はモジュール単位で読み込まれる。
ただし、各 field は thunk として保持され、必要になるまで評価されない。
@@ -1,18 +0,0 @@
# Let Expression
let expression は、ローカル束縛を作る。
```dcdl
let
part = {
greet = "Hello";
target = "World";
};
in
"${part.greet}! ${part.target}"
```
## 評価
let 束縛は thunk として保持される。
参照されない束縛は評価されない。
@@ -1,39 +0,0 @@
# Object Expression
object expression は、名前付き field の集合を表す。
```dcdl
{
host = "127.0.0.1";
port = 8000;
}
```
object は設定値にもスキーマにも使う。
```dcdl
MyConfig = {
host = String;
port = Int default 8080;
};
```
## Dot-path Field
ネストした field はドットパスでも定義できる。
```dcdl
{
feature_hoge.enable = false;
}
```
これは以下と同じ構造を表す。
```dcdl
{
feature_hoge = {
enable = false;
};
}
```
@@ -1,12 +0,0 @@
# String Interpolation Expression
string interpolation は、文字列内に式を埋め込む候補機能である。
```dcdl
"${part.greet}! ${part.target}"
```
## 評価
補間式の評価タイミングは通常の遅延評価に従う。
文字列補間を初期実装に含めるかは未確定である。
-91
View File
@@ -1,91 +0,0 @@
# 関数
関数は構造を受け取り、構造を返す式として扱う。
## 構文
```dcdl
(part: {
greet = String;
target = String;
}) =>
"${part.greet}! ${part.target}"
```
関数呼び出しは通常の呼び出し構文で行う。
```dcdl
mk_hw({
greet = "Hello";
target = "World";
})
```
複数引数の構文は候補として以下を想定する。
```dcdl
(
input_a: {
hoge = Int & >= 0;
},
input_b: {
fuga = 20;
}
) =>
{
# ...
}
```
## 関数の意味
関数は値として扱える。
ただし、最終データとして関数値を出力できるかどうかは別途定める。
設定ファイルを materialize する段階では、未適用の関数値は出力不能な値として扱うのが自然である。
## 評価方針
関数は以下の方針を基本とする。
- 関数は純粋である。
- 関数はレキシカルスコープを持つ。
- 関数は定義時の環境を参照として保持する。
- 引数は必要になるまで評価しない。
- フィールドに束縛された関数呼び出し結果は、そのフィールド評価結果として memoize される。
- 任意の関数呼び出しそのものをグローバルに memoize することは必須ではない。
関数値の内部モデル例:
```text
Function {
params: Vec<Param>
body: ExprId
env: EnvRef
}
```
## 関数と重さ
関数の文法・パーサー自体は大きくない。
実装上の重さは、主に評価モデルと意味論から発生する。
注意点:
- クロージャが環境を掴む。
- 引数を lazy にするか strict にするかを決める必要がある。
- 関数呼び出し結果をどこまで memoize するかを決める必要がある。
- 再帰関数を許可するかを決める必要がある。
- 関数値同士の `&` をどう扱うかを決める必要がある。
- 関数値を object に入れたとき、materialize 可能かを決める必要がある。
- import 循環と関数適用が絡んだときの cycle detection が必要になる。
軽量に保つため、初期仕様では以下の制限を検討できる。
- 関数は pure。
- lexical closure は許可。
- 引数は thunk として渡す。
- 関数本体は必要になるまで評価しない。
- 関数値は opaque。
- 関数同士の `&` は、同一関数参照以外は conflict。
- 関数値は最終データとして出力できない。
- 再帰は cycle error として扱う、または v1 では禁止する。
-12
View File
@@ -1,12 +0,0 @@
# 言語仕様
このディレクトリは、Decodal / DCDL の仕様本文を章ごとに分割して管理する。
目次はマニュアル直下の [Manual Index](../index.md) に集約する。
このファイルは `Language Specification` 章の入口としてだけ使う。
## Language
言語仕様は、構文、値、式、制約、合成演算子、関数、モジュール、評価意味論、materialize、エラーを定義する。
詳細な章構成と各ファイルへのリンクは [Manual Index](../index.md) を参照する。
@@ -1,98 +0,0 @@
# materialize とエラー
通常の評価では、制約や default を含む中間値が残ることがある。
外部へデータとして出力する段階では、materialize を行う。
## materialize の責務
materialize は以下を行う。
- 必要なフィールドを評価する。
- 明示値がない abstract value に default を適用する。
- 採用された値が制約を満たすか検証する。
- default を持たない未解決の abstract value をエラーにする。
- 未適用の関数など、データとして出力できない値をエラーにする。
## 例
```dcdl
MyConfig = {
host = String;
port = Int default 8080;
};
```
`MyConfig` を materialize すると、`host` は具体値も default もないためエラーになる。
`port``8080` が採用される。
```dcdl
Config = MyConfig & {
host = "localhost";
};
```
`Config` を materialize すると以下になる。
```dcdl
{
host = "localhost";
port = 8080;
}
```
## default の適用
`default` は materialize 時にのみ fallback として採用される。
```text
Abstract {
constraints: [Int]
default: 8080
}
```
この abstract value を materialize すると、`8080` が採用され、`Int` を満たすか検証される。
明示値がある場合、default は採用しない。
明示値は concrete value として表現され、default を保持しない。
```text
Concrete(Int(9000))
```
この値の最終値は `9000` である。
## エラー分類
代表的なエラー:
- 構文エラー
- 未定義識別子
- 型不一致
- 制約違反
- `&` の conflict
- `default` の conflict
- 循環依存
- import 失敗
- match の非網羅による失敗
- materialize 不能な値の出力
## match の失敗
`match` に fallback 分岐がなく、どの分岐にも一致しなかった場合はエラーになる。
```dcdl
match value {
>= 10: "large";
}
```
`value``10` 未満であれば失敗する。
## エラーは値ではない
エラーは runtime value ではなく diagnostic として扱う。
通常の式はエラー内容に基づいて分岐できない。
汎用 `try / catch` は core には含めない。
fallback は `default``match`、および将来的な optional import / optional field access のような限定された仕組みで表現する。
@@ -1,95 +0,0 @@
# モジュールと import
`import` は外部ファイルを読み込み、そのファイルの評価結果を返す。
## 構文
```dcdl
import "./config.dcdl"
```
import specifier は文字列リテラルとする。
パスリテラル構文は採用しない。
## モジュール
import 先はモジュール単位で読み込まれる。
ただし、モジュール全体を即時評価する必要はない。
各フィールドは thunk として保持され、必要になったときだけ評価される。
top-level に field 定義列を書いた module は、recursive module scope を作る。
つまり、top-level field は同じ module の他の top-level field から識別子として参照できる。
```dcdl
schema = {
hoge = String;
};
result = schema;
```
この場合、`result` の右辺の `schema` は同じ module の top-level field `schema` を参照する。
通常の object literal 内の field を暗黙に recursive scope にするかは別仕様とする。
## SourceLoader
`import` specifier の解決は処理系 core ではなく host 側の `SourceLoader` が行う。
CLI では、specifier を現在の module path からの相対 path として解決する。
組み込み利用では、resource table や static source map など、filesystem 以外の loader を使える。
module cache の key は loader が返す安定 key を使う。
CLI では canonical path を key とする。
## 循環 import
モジュール間に循環参照があっても、必要なフィールドの依存関係が循環していなければ評価できる。
例:
```dcdl
# main.dcdl
{
schema = {
hoge = String;
};
result = (import "./func.dcdl")(schema);
}
```
```dcdl
# func.dcdl
(input: (import "./main.dcdl").schema) =>
{
# ...
}
```
`func.dcdl``main.dcdl` を import しているが、参照しているのは `main.schema` である。
`main.schema``main.result` に依存していなければ、この循環 import は成立する。
## import の評価単位
実装上は、以下の単位で管理するのが自然である。
```text
Module main
schema -> thunk
result -> thunk
Module func
root -> thunk
```
各 thunk は一度だけ評価して memoize する。
評価中に同じ thunk へ戻った場合は循環依存としてエラーにする。
## import 失敗
以下は import 失敗として扱う。
- ファイルが存在しない。
- ファイルが読めない。
- import 先の構文解析に失敗する。
- import 先の評価で必要な値がエラーになる。
- 実装が禁止する import 循環に該当する。
-27
View File
@@ -1,27 +0,0 @@
# 命名規約
具体値と抽象値がグラデーションになるため、大文字・小文字による厳密な意味分けは設けない。
ただし、読みやすさのために慣習を定める。
## 推奨規約
- object 値: `lower_snake`
- 関数: `lowerCamel`
- 組み込み型・抽象的な制約名: `UpperCamel`
例:
```dcdl
IPv4Address
MyConfig
new_config
mkConfig
```
## 厳密な規則にしない理由
この言語では、値・制約・スキーマ・派生設定が同じ式体系に乗る。
そのため、ある名前が「具体値」か「抽象的な制約」かは文脈によってグラデーションになる。
大文字なら型、小文字なら値、のような厳密な規則を設けると、実際の利用に対して過剰に硬くなる可能性がある。
-15
View File
@@ -1,15 +0,0 @@
# Float
`Float` は浮動小数値を表す primitive type constraint である。
## 例
```dcdl
ratio = Float;
threshold = Float default 0.5;
```
## 未確定事項
`Int``Float` の暗黙変換を許可するかは未確定である。
軽量実装では、両者を明確に分ける方が単純である。
-20
View File
@@ -1,20 +0,0 @@
# String
`String` は文字列値を表す primitive type constraint である。
## 例
```dcdl
name = String;
greeting = String default "hello";
```
## 制約合成
`String` は文字列制約と合成できる。
```dcdl
message = String & /Hello! .*/;
```
正規表現制約を必須機能にするかは未確定である。
-60
View File
@@ -1,60 +0,0 @@
# 未確定事項
今後決める必要がある事項を管理する。
詳細化するときは、各項目を該当する仕様ファイルへ移動または反映する。
## 構文
- 正式な字句・構文仕様。
- 演算子の優先順位。
- `rec` の扱い。
- コメント構文を `#` のみにするか。
## 型・制約
- 配列要素の制約表現。
- object の open/closed schema の扱い。
- 正規表現を必須機能にするか optional feature にするか。
- 代表的な組み込み述語の範囲。
## default
- `default` 同士の conflict 解決規則。
- `&` による default 合成の厳密な規則。
- `//` による default 置換の厳密な規則。
- default thunk の評価失敗をどの段階で報告するか。
## 演算子
- `//` による制約・default の置換詳細。
- `replace(...)` を採用するか、別構文を設けるか。
- 配列に対する patch 操作を右辺置換だけにするか。
- 配列 append / prepend / remove などを提供するか。
## 関数
- 関数値の最終出力可否。
- 再帰関数を許可するか。
- 関数同士の `&` の扱い。
- 関数値の等価性。
- 関数呼び出し結果の memoize 範囲。
## 評価
- thunk のエラー memoize 方針。
- import cache の単位。
- 循環 import の診断メッセージ。
- materialize 対象の範囲指定方法。
## match
- match の網羅性チェックを行うか。
- 到達不能分岐を警告するか。
- パターン構文の範囲。
## エラー処理
- optional import を導入するか。
- optional field access を導入するか。
- optional fallback が捕捉できる失敗の範囲。
- エラー報告に制約由来の説明をどこまで含めるか。
+81
View File
@@ -0,0 +1,81 @@
# Packages and Integrations
Decodal provides separate packages for execution, host embedding, language services, and editor integration.
## Rust
### `decodal`
The runtime and embedding API for Rust applications.
It handles parsing, evaluation, and materialization of source; host globals; and imports containing either source or structured values.
```toml
[dependencies]
decodal = "0.4"
```
Enable the `derive` feature to generate a Decodal schema and decoder from a Rust struct.
```toml
[dependencies]
decodal = { version = "0.4", features = ["derive"] }
```
Enable the `regex` feature to validate regular-expression constraints.
This feature requires `std`.
### `decodal-language-service`
Provides transport-independent semantic evaluation and completion.
It accepts an application's `HostEnvironment` directly, allowing production and editor evaluation to share global bindings and import rules.
### `decodal-lsp`
Connects the language service to the Language Server Protocol.
In addition to a stdio server, it exposes a library API that constructs an application-specific environment from the client's `InitializeParams`.
### `decodal-language-tools`
Provides shared tools for source text, including the formatter.
The Rust, LSP, and WebAssembly integrations all use the same formatter implementation.
## JavaScript and WebAssembly
### `decodal-wasm`
Provides the evaluator and language service for browsers and other WebAssembly runtimes.
```sh
npm install decodal-wasm
```
It is also published on JSR as `@hare/decodal-wasm`.
Pass `globals`, `loadImport`, and `completeImport` to `DecodalLanguageService` so JavaScript-owned environments are shared by evaluation and completion.
### `decodal-codemirror`
Language support for CodeMirror 6.
It provides syntax highlighting, folding, indentation, and integration with the Decodal formatter.
```sh
npm install decodal-codemirror
```
It is also published on JSR as `@hare/decodal-codemirror`.
Combine it with the `decodal-wasm` language service when semantic evaluation and completion are required.
## Editor syntax
The Tree-sitter grammar is an integration for parsing and highlighting in editors that use Tree-sitter.
Tree-sitter is not required by the Decodal runtime, formatter, or LSP.
Choose packages according to the integration you need:
- Execution and embedding in Rust: `decodal`
- Execution and semantic tooling in a browser: `decodal-wasm`
- CodeMirror 6: `decodal-codemirror`
- Transport-independent Rust language service: `decodal-language-service`
- General editor clients: `decodal-lsp`
- Syntax grammar for editors that use Tree-sitter: the Tree-sitter integration
See [Embedding](./embedding.md) for concrete environment setup.
+193
View File
@@ -0,0 +1,193 @@
# Embedding
The Decodal runtime does not read filesystems, networks, or environment variables directly.
The host injects global bindings and import resolution, then receives the evaluation result as `Data` or an application type.
## Rust runtime
`Engine` owns an `ImportLoader` and global bindings.
Use `EmptyLoader` for source that does not require external resources.
```rust
use decodal::{EmptyLoader, Engine, Value};
let mut engine = Engine::new(EmptyLoader);
engine.bind_global(
"Service",
Value::object([
("name", Value::string_type()),
("port", Value::int_type().gt(443).default_int(8443)?),
("enabled", Value::bool_type().default_bool(true)?),
]),
)?;
let module = engine.add_root_source(
"service.dcdl",
"service.dcdl",
r#"{ name = "api"; port = 9443; } as Service"#,
)?;
let value = engine.eval_module(module)?;
let data = engine.materialize(&value)?;
```
`Value` is the public type for both concrete values and host-defined ranges.
Its main constructors are:
- `Value::string`, `int`, `float`, `bool`, `array`, and `object`: concrete values.
- `Value::string_type`, `int_type`, `float_type`, and `bool_type`: primitive ranges.
- `Value::unknown`: the top range, `Unknown`.
- `Value::array_of`: an array range that applies one range to every element.
- `Value::map_of`: an associative-array range that applies one range to every field value.
- `Value::object_with_rest`: an object with named fields and a range for remaining fields.
## Typed Rust integration
The `Decodal` derive from the `derive` feature generates `DecodalSchema` and `DecodalDecode` implementations from a Rust struct.
```rust
use decodal::{Decodal, DecodalDecode, DecodalSchema};
#[derive(Decodal)]
struct Service {
name: String,
#[decodal(gt = 443, default = 8443)]
port: i64,
#[decodal(rename = "feature.enable", default = true)]
feature_enabled: bool,
}
```
`DecodalSchema::decodal_schema()` returns a `Value` suitable for `Engine::bind_global`.
Materialized `Data` can be converted to a Rust type with `DecodalDecode::decodal_decode`.
For a struct that accepts additional fields, mark a map field with `#[decodal(rest)]`.
```rust
use std::collections::BTreeMap;
use decodal::{Data, Decodal};
#[derive(Decodal)]
struct OpenConfig {
enabled: bool,
#[decodal(rest)]
extra: BTreeMap<String, Data>,
}
```
`BTreeMap<String, Data>` corresponds to `...Unknown`, while `BTreeMap<String, String>` corresponds to `...String`.
Named fields are excluded from the rest map.
Only one `#[decodal(rest)]` field is allowed per struct, and it cannot be combined with `rename`, `default`, or field constraints.
## Imports
`ImportLoader::load` returns either `LoadedImport::Source` or `LoadedImport::Value`.
The host controls specifier resolution, reading from the filesystem or other storage, and content-type detection.
`LoadedImport::Source` supplies DCDL source.
`LoadedImport::Value` is intended for structured host-defined representations of formats such as Markdown, JSON, and TOML.
```text
import "./post.md"
-> host reads and parses Markdown
-> LoadedImport::Value {
key: "content/post.md",
value: { frontmatter: {...}, body: "..." }
}
-> normal Decodal validation and materialization
```
A stable `key` identifies an import and supplies diagnostic provenance.
If a structured value violates a Decodal constraint, the diagnostic identifies the import key, logical value path, and source span of the violated DCDL constraint.
The loader reports parse errors and positions in the external format itself.
## Shared host environment
`HostEnvironment` combines loader construction and global-binding setup.
Passing the same environment to the runtime, language service, and LSP keeps runtime and editor evaluation rules aligned.
```rust
use decodal::{Engine, HostEnvironment};
struct AppEnvironment;
impl HostEnvironment for AppEnvironment {
type Loader = ContentLoader;
fn create_loader(&self) -> Self::Loader {
ContentLoader::new()
}
fn configure_engine(
&self,
engine: &mut Engine<Self::Loader>,
) -> decodal::Result<()> {
engine.bind_global("Site", site_schema())?;
Ok(())
}
}
```
Construct transport-independent tooling with `decodal_language_service::LanguageService::new(&environment)`.
For LSP integration, implement `decodal_lsp::LspEnvironment` and start the server from an environment factory.
```rust
use decodal_lsp::{LspEnvironment, run_stdio};
impl LspEnvironment for AppEnvironment {}
run_stdio(|initialize| {
let _ = initialize;
Ok(AppEnvironment)
})?;
```
The factory receives the client's `InitializeParams` once, before the server responds with its capabilities.
Use the `LspEnvironment` document lifecycle hooks to reflect unsaved DCDL, Markdown, and other resources in a host-owned overlay.
## JavaScript and browser environments
The `DecodalLanguageService` in `decodal-wasm` does not assume a filesystem or virtual project.
JavaScript supplies `globals`, `loadImport`, and `completeImport`.
```js
import init, { DecodalLanguageService } from 'decodal-wasm';
await init();
const files = {
'schema.dcdl': 'Server = { port = Int; };',
};
const service = new DecodalLanguageService({
globals: {
App: {
enabled: { $decodal: 'Bool', default: true },
},
},
loadImport(_currentKey, specifier) {
const key = specifier.replace(/^\.\//, '');
return { kind: 'source', key, name: key, source: files[key] };
},
completeImport() {
return ['./schema.dcdl'];
},
});
const result = service.evaluate('main.dcdl', 'main.dcdl', 'App');
```
JavaScript primitives, arrays, and plain objects become concrete values.
Use `$decodal` descriptors for ranges:
- `{ $decodal: 'String' | 'Int' | 'Float' | 'Bool' }`
- `{ $decodal: 'Unknown' }`
- `{ $decodal: 'Array', item }`
- `{ $decodal: 'Map', value }`
- `{ $decodal: 'Object', fields, rest }`
- `{ $decodal: 'Range', constraints, default }`
Import callbacks are synchronous.
Preload or cache asynchronous inputs such as network resources before evaluation.
+42
View File
@@ -0,0 +1,42 @@
# Decodal Manual
This manual describes the Decodal language and the public APIs for embedding it in applications.
Decodal is the project name for the Deferred Constraint Data Language, abbreviated DCDL.
## Contents
1. [Introduction](./introduction.md)
2. [Language Specification](./language/index.md)
1. [Lexical Structure and Syntax](./language/syntax.md)
2. [Grammar](./language/grammar.md)
3. [Value](./language/value/index.md)
1. [String](./language/value/string.md)
2. [Int](./language/value/int.md)
3. [Float](./language/value/float.md)
4. [Bool](./language/value/bool.md)
4. [Expression](./language/expression/index.md)
1. [Literal](./language/expression/literal.md)
2. [Identifier](./language/expression/identifier.md)
3. [Path Reference](./language/expression/path-reference.md)
4. [Object](./language/expression/object.md)
5. [Array](./language/expression/array.md)
6. [Function](./language/expression/function.md)
7. [Function Call](./language/expression/function-call.md)
8. [Let](./language/expression/let.md)
9. [Match](./language/expression/match.md)
10. [Import](./language/expression/import.md)
11. [Composition](./language/expression/composition.md)
12. [Range Refinement](./language/expression/ascription.md)
13. [Default](./language/expression/default.md)
14. [Arithmetic](./language/expression/arithmetic.md)
15. [Logical and Comparison](./language/expression/logical-and-comparison.md)
5. [Constraints and Defaults](./language/constraints-and-defaults.md)
6. [Operators](./language/operators.md)
7. [Functions](./language/functions.md)
8. [Modules and Imports](./language/modules-and-imports.md)
9. [Evaluation Semantics](./language/evaluation.md)
10. [Materialization and Errors](./language/materialization-and-errors.md)
11. [Naming](./language/naming.md)
12. [Examples](./language/examples.md)
3. [Embedding](./embedding.md)
4. [Packages and Integrations](./components.md)
+54
View File
@@ -0,0 +1,54 @@
# Introduction
Decodal is the **Deferred Constraint Data Language**, abbreviated **DCDL**.
Its source files use the `.dcdl` extension.
Decodal is a language for describing configuration values, schemas, constraints, and derived configuration in one expression system, producing validated structured data.
## Purpose
In conventional configuration systems, values, schemas, defaults, derived settings, and validation are often handled by separate mechanisms.
Decodal treats them as composable expressions.
```dcdl
Port = Int & >= 1 & <= 65535;
Service = {
host = String;
port = Port default 8080;
};
Config = {
host = "127.0.0.1";
port = 8000;
} as Service;
```
`Service` describes a range of accepted values.
Each value in `Config` is validated against `Service`, while ranges that exist only in `Service` remain abstract in the result.
During materialization, explicit values take precedence and `default` supplies values for ranges that remain unspecified.
## Core concepts
- Values and constraints are expressions that can be referenced and composed in the same way.
- `&` performs symmetric composition and preserves constraints from both sides.
- `as` verifies that the left side is more specific and narrower than the right side, then combines them.
- `//` performs a right-biased structural patch, including overrides.
- `default` is not a constraint. It is a fallback selected only during materialization.
- Object fields, function arguments, and imports are evaluated when they are needed.
- `Unknown` is the top range that accepts any concrete value, but it cannot be materialized while it remains abstract.
## Scope
Decodal is specialized for describing configuration, schemas, constraints, and derived data.
General-purpose state mutation, time, randomness, network access, and `try / catch` for treating errors as values are not language features.
The host provides filesystem and external-format access. Given the same source, import results, and global bindings, Decodal evaluation is deterministic.
Regular-expression constraints depend on the runtime's `regex` feature.
Decodal does not provide advanced type inference, exhaustiveness checking for `match`, or static detection of unreachable branches.
## Continue reading
- [Language Specification](./language/index.md): syntax, values, expressions, constraints, evaluation, and materialization.
- [Embedding](./embedding.md): Rust and JavaScript execution, host environments, structured imports, and language services.
- [Packages and Integrations](./components.md): crates and JavaScript packages for each use case.
+81
View File
@@ -0,0 +1,81 @@
# Packages and Integrations
Decodal は実行、ホスト組み込み、言語サービス、エディタ統合を用途別の package として提供する。
## Rust
### `decodal`
Rust アプリケーション向けの runtime と embedding API である。
source の parse・evaluate・materialize、host global、source または structured value の import を扱う。
```toml
[dependencies]
decodal = "0.4"
```
Rust struct から Decodal schema と decoder を生成する場合は `derive` feature を有効にする。
```toml
[dependencies]
decodal = { version = "0.4", features = ["derive"] }
```
正規表現制約を検証する場合は `regex` feature を有効にする。
この feature は `std` を必要とする。
### `decodal-language-service`
transport に依存しない semantic evaluation と completion を提供する。
アプリケーションの `HostEnvironment` をそのまま受け取るため、本番評価と編集時評価で global bindings と import 規則を共有できる。
### `decodal-lsp`
language service を Language Server Protocol に接続する。
stdio server のほか、client の `InitializeParams` からアプリケーション固有の environment を構築する library API を提供する。
### `decodal-language-tools`
source formatter など、source text を扱う共通ツールを提供する。
formatter は Rust、LSP、WebAssembly の各統合から同じ実装を利用できる。
## JavaScript and WebAssembly
### `decodal-wasm`
browser と WebAssembly 対応 runtime 向けの evaluator と language service である。
```sh
npm install decodal-wasm
```
JSR では `@hare/decodal-wasm` として提供される。
`DecodalLanguageService``globals``loadImport``completeImport` を渡すことで、JavaScript が所有する environment を評価と補完で共有できる。
### `decodal-codemirror`
CodeMirror 6 向けの language support である。
syntax highlighting、folding、indentation、Decodal formatter との統合を提供する。
```sh
npm install decodal-codemirror
```
JSR では `@hare/decodal-codemirror` として提供される。
semantic evaluation と completion が必要な場合は `decodal-wasm` の language service と組み合わせる。
## Editor syntax
Tree-sitter grammar は、Tree-sitter を採用するエディタで構文解析と highlighting を行うための統合である。
Decodal runtime、formatter、LSP の利用には Tree-sitter は必要ない。
用途ごとの選択は次の通りである。
- Rust での実行と組み込み: `decodal`
- browser での実行と semantic tooling: `decodal-wasm`
- CodeMirror 6: `decodal-codemirror`
- transport 非依存の Rust language service: `decodal-language-service`
- 一般的な editor client: `decodal-lsp`
- Tree-sitter 採用 editor の syntax grammar: Tree-sitter integration
具体的な environment の構築方法は [Embedding](./embedding.md) を参照する。
+193
View File
@@ -0,0 +1,193 @@
# Embedding
Decodal runtime は filesystem、network、environment variable を直接読み込まない。
ホストは global bindings と import の解決方法を注入し、評価結果を `Data` または application type として受け取る。
## Rust runtime
`Engine``ImportLoader` と global bindings を持つ。
外部 resource を必要としない source には `EmptyLoader` を使える。
```rust
use decodal::{EmptyLoader, Engine, Value};
let mut engine = Engine::new(EmptyLoader);
engine.bind_global(
"Service",
Value::object([
("name", Value::string_type()),
("port", Value::int_type().gt(443).default_int(8443)?),
("enabled", Value::bool_type().default_bool(true)?),
]),
)?;
let module = engine.add_root_source(
"service.dcdl",
"service.dcdl",
r#"{ name = "api"; port = 9443; } as Service"#,
)?;
let value = engine.eval_module(module)?;
let data = engine.materialize(&value)?;
```
`Value` は concrete value と host-defined range の両方を表す公開型である。
主な constructor は次の通りである。
- `Value::string``int``float``bool``array``object`: concrete value。
- `Value::string_type``int_type``float_type``bool_type`: primitive range。
- `Value::unknown`: 最上位 range `Unknown`
- `Value::array_of`: 全要素へ同じ range を適用する array range。
- `Value::map_of`: 全 field value へ同じ range を適用する associative-array range。
- `Value::object_with_rest`: named fields と残りの field range を持つ object。
## Typed Rust integration
`derive` feature の `Decodal` derive は、Rust struct から `DecodalSchema``DecodalDecode` を生成する。
```rust
use decodal::{Decodal, DecodalDecode, DecodalSchema};
#[derive(Decodal)]
struct Service {
name: String,
#[decodal(gt = 443, default = 8443)]
port: i64,
#[decodal(rename = "feature.enable", default = true)]
feature_enabled: bool,
}
```
`DecodalSchema::decodal_schema()``Engine::bind_global` に渡せる `Value` を返す。
materialize 済みの `Data``DecodalDecode::decodal_decode` で Rust type に変換できる。
追加 field を受け取る struct では、map field に `#[decodal(rest)]` を付ける。
```rust
use std::collections::BTreeMap;
use decodal::{Data, Decodal};
#[derive(Decodal)]
struct OpenConfig {
enabled: bool,
#[decodal(rest)]
extra: BTreeMap<String, Data>,
}
```
`BTreeMap<String, Data>``...Unknown``BTreeMap<String, String>``...String` に対応する。
named fields は rest map から除外される。
`#[decodal(rest)]` は struct ごとに1つだけ指定でき、`rename``default`、field constraints とは併用できない。
## Imports
`ImportLoader::load``LoadedImport::Source` または `LoadedImport::Value` を返す。
ホストは specifier の解決、filesystem や他の storage からの読み込み、content type の判定をすべて管理する。
`LoadedImport::Source` は DCDL source を返す。
`LoadedImport::Value` は Markdown、JSON、TOML などをホスト独自の規則で構造化して返す用途に使える。
```text
import "./post.md"
-> host reads and parses Markdown
-> LoadedImport::Value {
key: "content/post.md",
value: { frontmatter: {...}, body: "..." }
}
-> normal Decodal validation and materialization
```
stable `key` は import の同一性と diagnostic の provenance に使われる。
構造化した値が Decodal constraint に違反した場合、diagnostic は import key と logical value path、および違反した DCDL constraint の source span を示す。
外部形式そのものの parse error と位置情報は loader が報告する。
## Shared host environment
`HostEnvironment` は loader の作成と global binding の設定を1つにまとめる。
同じ environment を runtime、language service、LSP へ渡すことで、実行時と編集時の評価規則を一致させられる。
```rust
use decodal::{Engine, HostEnvironment};
struct AppEnvironment;
impl HostEnvironment for AppEnvironment {
type Loader = ContentLoader;
fn create_loader(&self) -> Self::Loader {
ContentLoader::new()
}
fn configure_engine(
&self,
engine: &mut Engine<Self::Loader>,
) -> decodal::Result<()> {
engine.bind_global("Site", site_schema())?;
Ok(())
}
}
```
transport 非依存の tooling は `decodal_language_service::LanguageService::new(&environment)` で構築する。
LSP integration では `decodal_lsp::LspEnvironment` を実装し、environment factory から server を起動する。
```rust
use decodal_lsp::{LspEnvironment, run_stdio};
impl LspEnvironment for AppEnvironment {}
run_stdio(|initialize| {
let _ = initialize;
Ok(AppEnvironment)
})?;
```
factory は server capability の応答前に client の `InitializeParams` を一度受け取る。
`LspEnvironment` の document lifecycle hooks を使うと、未保存の DCDL、Markdown、その他の resource を host-owned overlay に反映できる。
## JavaScript and browser environments
`decodal-wasm``DecodalLanguageService` は filesystem や仮想 project を仮定しない。
JavaScript 側が `globals``loadImport``completeImport` を提供する。
```js
import init, { DecodalLanguageService } from 'decodal-wasm';
await init();
const files = {
'schema.dcdl': 'Server = { port = Int; };',
};
const service = new DecodalLanguageService({
globals: {
App: {
enabled: { $decodal: 'Bool', default: true },
},
},
loadImport(_currentKey, specifier) {
const key = specifier.replace(/^\.\//, '');
return { kind: 'source', key, name: key, source: files[key] };
},
completeImport() {
return ['./schema.dcdl'];
},
});
const result = service.evaluate('main.dcdl', 'main.dcdl', 'App');
```
JavaScript の primitive、array、plain object は concrete value になる。
range は `$decodal` descriptor で指定する。
- `{ $decodal: 'String' | 'Int' | 'Float' | 'Bool' }`
- `{ $decodal: 'Unknown' }`
- `{ $decodal: 'Array', item }`
- `{ $decodal: 'Map', value }`
- `{ $decodal: 'Object', fields, rest }`
- `{ $decodal: 'Range', constraints, default }`
import callbacks は同期 API である。
network resource などの非同期入力は、評価の前に preload または cache しておく。
+42
View File
@@ -0,0 +1,42 @@
# Decodal Manual
このマニュアルでは、Decodal の言語仕様と、アプリケーションへ組み込むための公開 API を説明する。
Decodal は Deferred Constraint Data Language、略称 DCDL のプロジェクト名である。
## 目次
1. [Introduction](./introduction.md)
2. [Language Specification](./language/index.md)
1. [Lexical Structure and Syntax](./language/syntax.md)
2. [Grammar](./language/grammar.md)
3. [Value](./language/value/index.md)
1. [String](./language/value/string.md)
2. [Int](./language/value/int.md)
3. [Float](./language/value/float.md)
4. [Bool](./language/value/bool.md)
4. [Expression](./language/expression/index.md)
1. [Literal](./language/expression/literal.md)
2. [Identifier](./language/expression/identifier.md)
3. [Path Reference](./language/expression/path-reference.md)
4. [Object](./language/expression/object.md)
5. [Array](./language/expression/array.md)
6. [Function](./language/expression/function.md)
7. [Function Call](./language/expression/function-call.md)
8. [Let](./language/expression/let.md)
9. [Match](./language/expression/match.md)
10. [Import](./language/expression/import.md)
11. [Composition](./language/expression/composition.md)
12. [Range Refinement](./language/expression/ascription.md)
13. [Default](./language/expression/default.md)
14. [Arithmetic](./language/expression/arithmetic.md)
15. [Logical and Comparison](./language/expression/logical-and-comparison.md)
5. [Constraints and Defaults](./language/constraints-and-defaults.md)
6. [Operators](./language/operators.md)
7. [Functions](./language/functions.md)
8. [Modules and Imports](./language/modules-and-imports.md)
9. [Evaluation Semantics](./language/evaluation.md)
10. [Materialization and Errors](./language/materialization-and-errors.md)
11. [Naming](./language/naming.md)
12. [Examples](./language/examples.md)
3. [Embedding](./embedding.md)
4. [Packages and Integrations](./components.md)
+54
View File
@@ -0,0 +1,54 @@
# Introduction
Decodal は **Deferred Constraint Data Language**、略称 **DCDL** である。
ファイル拡張子には `.dcdl` を使う。
Decodal は、設定値・スキーマ・制約・派生設定を同じ式体系で記述し、検証済みの構造化データを得るための言語である。
## 目的
一般的な設定システムでは、値、スキーマ、デフォルト値、派生設定、バリデーションが別々の仕組みになりやすい。
Decodal では、それらを合成可能な式として扱う。
```dcdl
Port = Int & >= 1 & <= 65535;
Service = {
host = String;
port = Port default 8080;
};
Config = {
host = "127.0.0.1";
port = 8000;
} as Service;
```
`Service` は許容する値の範囲を表す。
`Config` の各値は `Service` によって検証され、`Service` にだけ存在する範囲は抽象的なまま結果へ残る。
materialize 時には明示値が優先され、値がない範囲には `default` が使われる。
## 中心概念
- 値と制約は同じ式として参照・合成できる。
- `&` は両辺の制約を保つ対称な合成を行う。
- `as` は左辺が右辺より具体的で狭いことを検証しながら合成する。
- `//` は右辺優先の構造的な patch を行う。
- `default` は制約ではなく、materialize 時にだけ選ばれる fallback である。
- object field、関数引数、import は必要になった時点で評価される。
- `Unknown` は任意の具体値を受け入れる最上位 range だが、抽象的なまま materialize はできない。
## 適用範囲
Decodal は、設定、スキーマ、制約、派生データの記述に特化している。
汎用的な状態変更、時刻・乱数・ネットワークアクセス、例外を値として捕捉する `try / catch` は言語機能に含まれない。
filesystem や外部形式の読み込みはホストが提供し、Decodal の評価は同じ source・import 結果・global bindings に対して決定的に動作する。
正規表現制約は利用する runtime の `regex` feature に依存する。
高度な型推論、match の網羅性検査、到達不能分岐の静的検査は提供しない。
## 次に読む章
- [Language Specification](./language/index.md): 構文、値、式、制約、評価、materialization。
- [Embedding](./embedding.md): Rust・JavaScript からの実行、host environment、structured import、language service。
- [Packages and Integrations](./components.md): 用途ごとに選ぶ crate と JavaScript package。
@@ -0,0 +1,163 @@
# 制約と default
constraint は、値が満たすべき範囲を表す。
```dcdl
Int
String
>= 1
<= 65535
/Hello! .*/
```
constraint は `&` で合成できる。
```dcdl
Port = Int & >= 1 & <= 65535;
NarrowedPort = Port & > 443;
```
合成結果は両辺を同時に満たす範囲になる。
両立しない constraint は conflict になる。
```dcdl
Int & String # conflict
> 10 & < 5 # conflict
Int & > 10 & < 11 # conflict: integer candidate does not exist
```
## Primitive and comparison constraints
組み込みの primitive range は次の通りである。
```dcdl
String
Int
Float
Bool
```
異なる primitive type を `&` で合成すると conflict になる。
数値比較 constraint は上下限として合成され、空の範囲になる場合は conflict になる。
```dcdl
Int & >= 1 & <= 65535 & > 443
```
`Int` の比較 constraint は integer literal を使う。
`Float` の比較 constraint は integer または float literal を使える。
## Unknown
`Unknown` はすべての Decodal value を含む最上位 range である。
検査を無効化して具体値の情報を消す `Any` ではなく、値の範囲がまだ絞られていないことを表す。
```dcdl
Int & Unknown # Int
42 as Unknown # 42
Unknown as Int # conflict
```
`Unknown` 自体は concrete value を持たないため materialize できない。
concrete value で絞り込むか `default` を指定する必要がある。
```dcdl
Unknown default {}
```
## Regex constraints
regex literal は string constraint である。
```dcdl
Host = /^api-[0-9]+$/;
```
複数の regex constraint を合成した場合、concrete string はすべてに一致する必要がある。
```dcdl
String & /^a/ & /z$/
```
regex constraint 同士の交差は合成時には判定されない。
そのため、次の範囲は合成時には conflict にならず、concrete value の検証時に失敗する。
```dcdl
String & /^a$/ & /^b$/
```
Rust runtime では regex engine を `regex` feature で有効にする。
feature が無効な場合、regex constraint の concrete value 検証は unsupported feature diagnostic になる。
## Array constraints
array constraint は `[...T]` と書き、すべての要素へ `T` を適用する。
要素 constraint は必須である。
```dcdl
Names = [...String];
PositiveInts = [...(Int & > 0)];
```
concrete array と合成した場合、各要素は `T` に対して `as` と同じ規則で絞り込まれる。
空 array は任意の array constraint を満たす。
要素 constraint が object range の場合、右辺にだけある field は abstract のまま各要素へ残る。
左辺にしかない field は右辺の field domain 外なので conflict になる。
## Associative-array and object rest constraints
associative-array constraint は `{...T}` と書き、object の任意の field value へ `T` を適用する。
```dcdl
Services = {...{
port = Int;
enabled = Bool default true;
}};
```
key は列挙されず、空 object も許容される。
named field を持つ object の末尾へ `...T` を書くと、列挙されていない field だけに `T` を適用できる。
```dcdl
{
enabled = Bool default true;
...Unknown
}
```
rest constraint は field を生成しない。
実在する追加 field の検証にだけ使われる。
## default
`default` は constraint ではない。
materialize 時に concrete value がない場合だけ使われる fallback である。
```dcdl
Port = Int & >= 1 & <= 65535;
Service = {
port = Port default 8080;
};
```
明示値が合成された場合は明示値が使われ、`default` は評価されない。
```dcdl
Config = {
port = 9000;
} as Service;
```
明示値がない場合、materialize 時に `8080` が採用され、`Port` を満たすか検証される。
## default composition
- `&` で片方だけが `default` を持つ場合、その `default` は保持される。
- `&` で両辺が異なる `default` を持つ場合は conflict になる。
- constraint と concrete value の `&` が成功した場合、結果は concrete value になり `default` は残らない。
- `//` は右辺優先なので、同じ field では右辺の値または `default` が左辺を置き換える。
- `as` は右辺の `default` を左辺へ注入しない。ただし右辺にだけ存在する field は、その field が持つ `default` とともに abstract なまま結果へ残る。
`default` expression は materialize 時に必要になった時点で評価され、同じ range の constraint を満たす必要がある。
@@ -0,0 +1,48 @@
# 遅延評価
Decodal は値を必要になった時点で評価する。
## 遅延評価される値
次の値は参照または materialize されるまで評価されない。
- module の root と top-level field
- object field
- `let` binding
- function argument
- `default` expression
- import 先の値
同じ binding を複数回参照した場合、その評価結果は再利用される。
このため、未参照の field や function argument にある失敗は、値が必要になるまで発生しない。
```dcdl
let
safe = 42;
unused = missing_name;
in
safe
```
この式は `unused` を参照しないため `42` になる。
## 循環検出
評価中の値が自身へ再び依存した場合は cycle diagnostic になる。
```dcdl
{
a = b + 1;
b = a + 1;
}
```
module や import の参照関係自体が循環していても、実際に評価される field の依存関係が循環していなければ評価できる。
## 評価と materialize
通常の評価結果には、constraint、`Unknown``default`、function などの abstract value が残り得る。
外部へ concrete data として取り出すときに materialize を行う。
この分離により、schema を値として合成し、必要な field だけを評価し、`default` の選択を出力時まで遅らせられる。
詳細は [materialize とエラー](./materialization-and-errors.md) を参照する。
+166
View File
@@ -0,0 +1,166 @@
# 例
この章では、Decodal の主要な記法を組み合わせた例を示す。
## 基本的な設定スキーマ
```dcdl
Host = String;
Port = Int & >= 1 & <= 65535;
NarrowedPort = Port & > 443;
MyConfig = {
host = Host;
port = NarrowedPort default 8080;
feature_hoge = {
enable = Bool default true;
fuga = Int default 10;
};
};
NewConfig = MyConfig & {
host = "127.0.0.1";
port = 8000;
};
disabled_config = NewConfig & {
feature_hoge.enable = false;
};
enabled_config = NewConfig;
```
## 配列スキーマ
```dcdl
Services = [...{
name = String;
port = Int default 8080;
}];
[
{ name = "api"; },
{ name = "worker"; port = 9000; },
] as Services
```
抽象配列には要素制約が必須である。
この例では、1 番目の要素の `port``8080` に materialize される。
## 連想配列と範囲絞り込み
```dcdl
Service = {
port = Int;
enabled = Bool default true;
};
services = {
api = { port = 8080; };
worker = { port = 8081; enabled = false; };
} as {...Service};
```
`api``worker` は任意の key であり、それぞれの value は `Service` より狭い範囲へ絞り込まれる。
`as` の結果では `api.enabled` は abstract のまま残り、最終的に結果全体を materialize した時点で default の `true` が使われる。
## 関数と制約
```dcdl
let
Port = Int & >= 1 & <= 65535;
add_offset = (base: Port, offset: Int) => base + offset;
in
add_offset(8000, 80)
```
評価結果:
```text
8080
```
## match
```dcdl
(
input_a: {
hoge = Int & >= 0;
},
input_b: {
fuga = 20;
}
) =>
let
inputs = {
a = input_a;
b = input_b;
};
in
{
foo = match inputs.a.hoge {
>= 20: {
value = 200;
};
>= 10: {
value = 100;
};
_: {
value = 300;
};
};
}
```
`match` は上から順に評価されるため、広い条件より狭い条件を先に書く。
## deep patch
```dcdl
Base = {
feature_hoge = {
enable = Bool default true;
fuga = Int default 10;
};
};
Patched = Base // {
feature_hoge.enable = false;
};
```
`Patched` は以下に相当する。
```dcdl
{
feature_hoge = {
enable = false;
fuga = Int default 10;
};
}
```
## 循環 import
```dcdl
# main.dcdl
{
schema = {
hoge = String;
};
result = (import "./func.dcdl")(schema);
}
```
```dcdl
# func.dcdl
(input: (import "./main.dcdl").schema) =>
{
# ...
}
```
`func.dcdl``main.dcdl` を import しているが、参照しているのは `main.schema` である。
`main.schema``main.result` に依存していなければ、この循環 import は成立する。
@@ -0,0 +1,43 @@
# 算術式
Decodal は具体的な数値に対する算術演算をサポートする。
```dcdl
{
workers = 2 + 2;
timeout = 30.0 / 2;
port = 8000 + 80;
negative = -1;
}
```
## 演算子
- `+`: 加算
- `-`: 減算
- `*`: 乗算
- `/`: 除算
- 単項 `-`: 符号反転
`*``/``+``-` より高い優先順位を持つ。
括弧を使ってグループ化を明示できる。
```dcdl
2 + 3 * 4 # 14
(2 + 3) * 4 # 20
```
## 数値の扱い
算術演算には具体的な `Int` または `Float` の operand が必要である。
`Int + Int``Int - Int``Int * Int` は、overflow が発生しない場合に `Int` を返す。
`Int``Float` が混在する演算は `Float` を返す。
除算は常に `Float` を返す。
ゼロ除算と整数 overflow は評価エラーになる。
算術式は、default や数値制約を含め、具体的な数値式が必要な任意の場所で使える。
```dcdl
port = Int & > 4000 + 42 default 8080;
```
@@ -0,0 +1,71 @@
# Array Expression
array expression は、順序付きの値の列を表す。
```dcdl
[1, 2, 3]
["a", "b", "c"]
```
配列リテラルは concrete value であり、要素型を揃える必要はない。
```dcdl
["api", 8080, true]
```
## Array constraint
配列制約は `[` の直後に ellipsis を置き、その後へ要素制約を記述する。
```dcdl
Names = [...String];
Ports = [...(Int & >= 1 & <= 65535)];
```
`[...T]` は、すべての要素が `T` を満たす長さ 0 以上の配列を表す。
空配列は任意の配列制約を満たす。
```dcdl
[...String] & []
[...String] & ["api", "worker"]
```
要素制約は object range にもできる。
右辺だけにある default 付き field は、配列へ制約を合成した時点では abstract のまま各要素に残る。
その配列を後から materialize した場合にだけ default が使われる。
```dcdl
Services = [...{
name = String;
enabled = Bool default true;
}];
Services & [{ name = "api"; }]
```
配列制約を concrete array に適用すると、各要素は要素 range に対して `as` と同じ規則で絞り込まれる。
object の要素 range では左辺にしかない field がエラーになり、右辺にしかない field は abstract のまま残る。
要素制約のない抽象配列型は提供しない。
旧来の `Array` primitive type は使用できず、`[...T]``T` は必須である。
`[String]` は配列制約ではなく、未解決の `String` 制約を 1 要素に持つ concrete array になる。
配列制約は要素範囲だけを表す。長さ制約、位置別の tuple 制約、unique 制約は持たない。
## Array concat
`++` は concrete array 同士を連結する。
```dcdl
base = ["read", "write"];
extra = ["admin"];
roles = base ++ extra;
```
`roles` は以下と同じ値になる。
```dcdl
["read", "write", "admin"]
```
`++` は配列要素を変換しない。
左辺の要素の後に右辺の要素が並ぶ。
@@ -0,0 +1,75 @@
# Range Refinement
`narrower as wider` は、左辺が右辺より具体的で狭い範囲であることを確認し、両者を非対称に合成する演算である。
```dcdl
Int & > 10 as Int & > 0
```
この例は成功し、結果は左辺の狭い範囲 `Int & > 10` のままになる。逆向きの `Int as Int & > 0` は、左辺が右辺を満たすとは限らないため失敗する。
## Object
object では、右辺を field domain として左辺の field を確認する。
- 左辺にしかない field は、右辺の domain 外なのでエラーになる。
- 両側にある field は、左辺が右辺より狭いかを再帰的に確認し、左辺の狭い結果を使う。
- 右辺にしかない field は、default の有無に関係なく未評価のまま結果へ残す。
- 結果の field order は右辺の順序に従う。
```dcdl
partial = {
port = 8080;
} as {
host = String;
port = Int;
enabled = Bool default true;
};
```
`partial.port``8080` に具体化される。`partial.host``partial.enabled` は右辺由来の abstract field のままであり、`as``enabled` の default を選択も force もしない。
その後 `partial` 全体を materialize すれば、通常の materialize 規則が適用される。この例では unresolved な `host` がエラーになり、`host` も具体化されていれば `enabled` の default がその時点で利用される。
## Default
default は範囲包含の根拠ではない。両側に同じ field がある場合、結果には左辺を使うため、右辺の default を左辺へ注入しない。
```dcdl
Int & > 0 as (Int default 1)
```
結果は default のない `Int & > 0` である。一方、右辺だけに残る object field は field 全体を保持するため、その abstract value が元から持つ default も保持される。
## Map and array ranges
`{...T}` は任意の object key を許可し、各 value が `T` より狭いことを確認する。
```dcdl
ports = {
http = 80;
https = 443;
} as {...(Int & >= 1 & <= 65535)};
```
`[...T]` を右辺に使う場合も、すべての array element を `T` に対して絞り込む。
## Concrete right-hand ranges
primitive constraint や合成 constraint は通常どおり値を検証する。
右辺が concrete scalar または array literal の場合は、左辺も同じ値または同じ長さ・要素構造である必要がある。
function は右辺の範囲として使用できない。
左辺は concrete value に限らない。包含関係を判定できる constraint 同士であれば abstract value も使用できる。primitive type、numeric bound、同一 regex / predicate、array/map の要素範囲は包含確認の対象になる。
関数 parameter の `name: range` も、引数が force された時点で `as` と同じ絞り込み規則を使う。
## Precedence
`as``default` より低い、最も低い優先順位を持ち、左結合である。
```dcdl
narrow & overrides as Wider
```
これは `(narrow & overrides) as Wider` と解釈される。
@@ -11,6 +11,9 @@ Port = Int & >= 1 & <= 65535;
Config = MyConfig & { port = 8000; };
```
object 同士では片側だけにある field も保持するため、`&` は方向を持たない。
左辺が右辺より狭いことを確認し、右辺を field domain として合成したい場合は [`as`](./ascription.md) を使う。
## `//`
`//` は右辺優先の構造的 patch を行う。
@@ -21,4 +24,4 @@ Patched = Base // {
};
```
詳細は [合成演算子](../operators.md) に置く
詳細は [合成演算子](../operators.md) を参照する
@@ -0,0 +1,15 @@
# Default Expression
`default` expression は、明示値が存在しない場合に materialize 時に採用される fallback を指定する。
```dcdl
port = Int default 8080;
```
`default` は制約ではない。
詳細は [制約と default](../constraints-and-defaults.md) を参照する。
## 評価
fallback expression は materialize 時に必要になった場合だけ評価される。
明示値がある場合、`default` は評価も採用もされない。
@@ -0,0 +1,14 @@
# Function Call Expression
function call expression は、関数値を引数に適用する式である。
```dcdl
increment(41)
```
## 評価
引数は関数本体から参照された時点で評価される。
同じ引数を複数回参照した場合は評価結果が再利用される。
parameter に range が指定されている場合、引数は `narrower as wider` と同じ規則で絞り込まれる。
parameter 側だけにある field は abstract のまま残り、default はこの時点では選択されない。
@@ -0,0 +1,14 @@
# Function Expression
function expression は、引数を受け取り式を返す値である。
```dcdl
(value: Int) => value + 1
```
関数仕様の詳細は [関数](../functions.md) を参照する。
## 評価
関数は定義された lexical scope の bindings を参照する。
関数本体は、関数値の生成時ではなく呼び出し時に評価される。
@@ -0,0 +1,14 @@
# Identifier Expression
identifier expression は、lexical scope に束縛された名前を参照する式である。
```dcdl
Port
MyConfig
mkConfig
```
## 評価
識別子は対応する binding の値を必要になった時点で評価する。
束縛が存在しない場合は未定義識別子エラーになる。
@@ -0,0 +1,14 @@
# Import Expression
import expression は、ホストが解決した DCDL module または structured value を返す。
```dcdl
import "./config.dcdl"
```
specifier は string literal であり、path や resource name としての解釈はホストが定義する。
詳しくは [モジュールと import](../modules-and-imports.md) を参照する。
## 評価
import 先は遅延評価され、参照されない field は評価されない。
@@ -10,15 +10,17 @@ Expr
├─ identifier
├─ path reference
├─ object
├─ map constraint
├─ array
├─ array constraint
├─ function
├─ function call
├─ let
├─ match
├─ import
├─ composition
├─ default
└─ string interpolation
├─ range refinement (`as`)
└─ default
```
各式の個別仕様へのリンクは [Manual Index](../../index.md) に集約する。
@@ -0,0 +1,16 @@
# Let Expression
let expression は、ローカル束縛を作る。
```dcdl
let
base = 8000;
offset = 80;
in
base + offset
```
## 評価
binding は参照された時点で評価される。
参照されない binding は評価されず、同じ binding を複数回参照した場合は評価結果が再利用される。
@@ -0,0 +1,41 @@
# 論理式と比較式
Decodal は具体的な `Bool` に対する論理演算と、具体的な scalar value に対する比較をサポートする。
```dcdl
{
is_prod = env == "prod";
high_port = port > 9000;
enabled = is_prod && high_port;
disabled = !enabled;
}
```
## 論理演算子
- `!expr` は具体的な `Bool` を反転する。
- `lhs && rhs` は論理積を返す。
- `lhs || rhs` は論理和を返す。
`&&``||` は短絡評価され、右辺は必要な場合にだけ評価される。
論理演算の operand は具体的な `Bool` へ評価される必要がある。
## 比較演算子
- `==`
- `!=`
- `<`
- `<=`
- `>`
- `>=`
`==``!=` は、`String``Bool``Int``Float` の具体的な scalar value を比較する。
`Int``Float` は数値として相互に比較できる。
順序比較演算子 `<``<=``>``>=` は、具体的な数値だけを比較する。
これらは `> 443` のような prefix comparison constraint とは別のものである。
```dcdl
port = Int & > 443 default 9443;
is_high = port > 9000;
```
@@ -0,0 +1,88 @@
# Object Expression
object expression は、名前付き field の集合を表す。
```dcdl
{
host = "127.0.0.1";
port = 8000;
}
```
object は設定値にもスキーマにも使う。
```dcdl
MyConfig = {
host = String;
port = Int default 8080;
};
```
## Dot-path Field
ネストした field はドットパスでも定義できる。
```dcdl
{
feature_hoge.enable = false;
}
```
これは以下と同じ構造を表す。
```dcdl
{
feature_hoge = {
enable = false;
};
}
```
## Map constraint
連想配列の制約は、object の任意の field value に同じ schema を適用する。
```dcdl
Services = {...{
port = Int;
enabled = Bool default true;
}};
```
`{...T}` は key の集合を固定せず、すべての value が `T` を満たす object を表す。
空 object も許容される。materialize 後は named fields を持つ object と同じ object data になる。
```dcdl
services = {
api = { port = 8080; };
worker = { port = 8081; enabled = false; };
} as Services;
```
各 entry は `as` によって右辺の field domain 内へ絞り込まれるため、左辺に `port``enabled` 以外の field があればエラーになる。
右辺にしかない field は default の有無にかかわらず abstract のまま残る。
host から渡した object は識別子構文に収まらない文字列 key も保持できるが、DCDL source の object field name は通常の識別子に限られる。
## Object rest constraint
固定 field と任意 key の value range は、一つの object に混在できる。
```dcdl
OpenConfig = {
enabled = Bool default true;
...Unknown
};
```
`...T` は明示されていない残余 field にだけ適用する。明示 field にはそれぞれの field range を適用し、rest range を重ねて適用しない。
rest constraint は field を生成せず、materialize 時には実際に存在する追加 field だけを出力する。
```dcdl
{
enabled = false;
plugin = { name = "cache"; };
} as OpenConfig
```
`...T` は object の末尾に一つだけ書ける。省略した object は閉じており、`as` の左辺に未宣言 field があればエラーになる。
named field を持たない `{...T}` は abstract な map constraint であり、単独で materialize するには `default` または concrete value が必要になる。
@@ -12,4 +12,5 @@ config.feature_hoge.enable
左側の式を object として評価し、指定された field を参照する。
参照先 field は必要になるまで評価されない。
存在しない field への参照をエラーにするか、open schema として扱うかは未確定である。
存在しない field への参照は diagnostic になる。
明示的な `Unknown` range は存在しないfieldを表さないため、fieldの有無を曖昧にはしない。
@@ -0,0 +1,41 @@
# 関数
関数は値を受け取り、値を返す純粋な式である。
## 構文
```dcdl
(value: Int) => value + 1
```
関数呼び出しは通常の呼び出し構文で行う。
```dcdl
let
increment = (value: Int) => value + 1;
in
increment(41)
```
複数の parameter を指定できる。
```dcdl
(input_a: { hoge = Int & >= 0; }, input_b: { fuga = Int; }) =>
{
hoge = input_a.hoge;
fuga = input_b.fuga;
}
```
parameter range は省略できる。
range がある場合、引数は参照された時点で `as` と同じ規則によって検証・絞り込みされる。
## Scope and evaluation
関数は lexical scope を持ち、定義された場所の bindings を参照する。
引数は遅延評価され、関数本体から参照されない引数は評価されない。
再帰的な field または argument の依存は cycle diagnostic になる。
関数は中間値として参照・呼び出しできるが、data として materialize できない。
未適用の関数が materialize 対象に残っている場合は diagnostic になる。
関数値同士の等価性は定義されず、`&` で関数値同士を合成すると conflict になる。

Some files were not shown because too many files have changed in this diff Show More