Bindings for the dash shell as a library
Project description
libdash is a fork of the Linux Kernel's dash
shell that builds a linkable library with extra exposed interfaces. The primary use of libdash is to parse shell scripts, but it could be used for more.
The Python bindings are packaged as the libdash
PyPi package.
The OCaml bindings---packaged as the libdash
OPAM package---include two executables, shell_to_json
and json_to_shell
, which let you conveniently parse POSIX shell scripts into a JSON AST.
What are the dependencies?
The C code for dash should build on a wide variety of systems; it requires libtool
and autotools
(aclocal
, autoheader
, automake
, autoconf
). The library may not build on platforms with esoteric linkers; it's been tested on macOS and Linux.
The Python and OCaml bindings depend on being able to build the C code. See libdash.opam
for details on the OCaml code's dependencies, which includes the build-time external dependencies. Python wheels have no need for these build-time dependencies, but building from a Python source distribution will only succeed when libtool
and autotools
are present.
The CI scripts (in .github/workflows/build.yml
) give build details.
How to build libdash
from source
Python
Run python3 setup.py install
. On macOS, you must first install the build dependencies via brew install libtool autoconf automake
.
You can test the Python bindings by running:
cd python; make test
OCaml
Install the OPAM file: opam pin add .
or opam install .
. This will build the OCaml library and install it in your OPAM repository. There are tests in another directory; they will only build when libdash is actually installed.
You can test the OCaml bindings by running:
cd ocaml; make test
Testing
The tests use test/round_trip.sh
to ensure that every tester file in test/tests
round-trips correctly through parsing and pretty printing.
Additionally, you can run tests that compare the OCaml and Python implementations (after you've installed them both):
cd test; make
How to use the parser
For Python, see python/rt.py
, an example tool that does a round-trip: shell syntax to AST back to shell syntax.
For OCaml, see ocaml/shell_to_json.ml
, a tool that parses shell syntax and produces JSON (using the atdgen bindings).
The ideal low-level interface to use is parsecmd_safe
in parser.c
; you'll need to ensure that dash's initialization routines have been called and that the stack marks are managed correctly. Parsing the POSIX shell is a complicated affair: beyond the usual locale issues, aliases affect the lexer, so one must use setalias
and unalias
to manage any aliases that ought to exist.
How work with the parsed nodes
The general AST is described in nodes.h
. There are some tricky invariants around the precise formatting of control codes; the OCaml code shows some examples of working with the args
fields in ocaml/ast.ml
, which converts the C AST to an OCaml AST.
The OCaml tools shell_to_json
and json_to_shell
will produce JSON ASTs, allowing you to work with these ASTs in any language.
Pretty printing
The pretty printer does its best to produce valid shell scripts, but it's possible to manually construct AST nodes that don't directly correspond to valid scripts.
For example, the Python AST [[['Q', [['C', 34]]]]]
represents a quoted field containing a double quote character. Translated literally, this would yield the string """
, which is not a valid shell script. The pretty printer will instead automatically escape the inner quote, rendering "\""
.
While the printer tries to get things right either way, you should use escapes to signal to the printer when to escape: you should use the Python AST [[['Q', [['E', 34]]]]]
to mark the inner double quote as escaped.
Known issues
We currently do not escape the character !
(exclamation point). In an interactive shell, !
is likely treated as a history substitution (and so should be escaped), but in a non-interactive shell, !
is treated normally. We currently cater to non-interactive shells; eventually this behavior will be controllable.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for libdash-0.3.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9d8b53ce4d0aa1346c6aff6c9e27f0221d1ff1bfb2bbfda37fa36645a1c353aa |
|
MD5 | 0941a7894b6bb6d2a25fbc3860ff62a2 |
|
BLAKE2b-256 | 7412de866393e0cff79c2e87e260a842d143afa109c3ea059a3bdc7404412b23 |
Hashes for libdash-0.3.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6e7fc4465e7775f22a84dbf590ae66e214e05fc6ab96faeb8461eddd424f3fa6 |
|
MD5 | b870571927f48042026258a047493ea8 |
|
BLAKE2b-256 | 8c6565c1fadf525a2110eb2ba87e6f42df34209d19c43220cb0ce3aa03fad168 |
Hashes for libdash-0.3.1-pp39-pypy39_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d95bdc6d5ac3db020c3f6abc6dd8673e066da3338528caa07980bebfbb4587f5 |
|
MD5 | 3b30692042a074fa7f0db58568ba343c |
|
BLAKE2b-256 | 21d0fb558e6d7599c0b18b9457ad0bb11b5d5b81c6de08e6cd970102e33da667 |
Hashes for libdash-0.3.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 563e2554ae31ba3e7afb20fb2d600381982f7ca62c8414e77749a489c25116e6 |
|
MD5 | 3ba4fb070929bfc3e6fb0d71483c828b |
|
BLAKE2b-256 | 54b8ef4aff2ef3f20060f52377ed086a0b002dde29f2396e67e9377fc83b8758 |
Hashes for libdash-0.3.1-pp38-pypy38_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a82e5998826d762abd0f2b7e37a6847bd6bb668408383f04606c2127079b4583 |
|
MD5 | 4b2cad42c2cbffcf4c6592a3f0a1d97b |
|
BLAKE2b-256 | 7c12569021fd0cce56ff2fe24a6d69aadf1b662b1078f66dcdfaaf6dcd4c0f78 |
Hashes for libdash-0.3.1-pp38-pypy38_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75886db8a394aaac3a9364b146525ac09a87b28b4873a188098b308d675b09f2 |
|
MD5 | d5db6adf875f96cfd7f91af677058bd5 |
|
BLAKE2b-256 | bf13b4f5a2a97b4192d4f8ff2cf7f534c4b8072070b8e01966cd12b71d255c29 |
Hashes for libdash-0.3.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 113ede2865b7134f6aa16fc6bf5759daf68414dd603282b2cb23f47c90698be5 |
|
MD5 | 4cd1d6eb7ce38dfdd06d78074a63ae58 |
|
BLAKE2b-256 | 9ab5aeafdf1879273db167b82b5151f5b5154089e73422f34104b92606ff56bf |
Hashes for libdash-0.3.1-pp37-pypy37_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ac35cfdde4ce109fc170b280f73c9b48b98b3145ece8fb840e8614820368e847 |
|
MD5 | f5f18c560320641563bd2a51295df088 |
|
BLAKE2b-256 | a38b2700686085d94ab53892ce5fa51c592ee847182bab87909164bcb586f6f3 |
Hashes for libdash-0.3.1-pp37-pypy37_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a45d1e3f2996f9d3b21d5082495cdb67fccae24b41d86d02eb2fff3bd33e205 |
|
MD5 | 792eb64db5c69b9e74fd45a34d6b3c3a |
|
BLAKE2b-256 | e07c62776653019e0f7b0fb49e060631af22604b55418c180f78ad60efe55c3a |
Hashes for libdash-0.3.1-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 88912a7df9e4f40c4517d58a9131ea47900db80a11707ea60f8050a22dd9c588 |
|
MD5 | abca7adc21f4263d2c1abd48d75917f1 |
|
BLAKE2b-256 | ffac64189a623ea1e9f997df5783e08653a177687ddc219fef95c6b70424ee6f |
Hashes for libdash-0.3.1-cp310-cp310-musllinux_1_1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a4772e4212e2f886c6f0452cfa88ad86f5b28e57108be24159cdb1e3ba5f96d2 |
|
MD5 | aba7dae2de6d5a196096a4c35d158f7f |
|
BLAKE2b-256 | 8d562ff01ebd1be9076babab12ca1b7dbac64b080298dccc8a1f26488df82b39 |
Hashes for libdash-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1c12e8bcfddca5940491e0263eac75fe9f19f16e2ebb6f2907139910fdd3ff58 |
|
MD5 | 77e3c7d51a6c9cce74e7bbef72533859 |
|
BLAKE2b-256 | 68adbcddcc31c0042055b9f5c31b2a8a82f9a830cfbbdb8e4069fde9e45f3a76 |
Hashes for libdash-0.3.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 25abfd270e12109f1a9bdcfa3c2a23eee3a48cb3136782f9e1862e73d67c4586 |
|
MD5 | 768e6d128d791dd6aca55b45fae000af |
|
BLAKE2b-256 | 4b37aee087155916becf7000745788f024a521394272d215abd8eb955d5b4629 |
Hashes for libdash-0.3.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9316353606c3991d21bd7a1e127f86257eb3b6cb2a6d7b8d413c0865fa937527 |
|
MD5 | 76ea6485c3f6de286a0ed6f729650323 |
|
BLAKE2b-256 | b70a3da43772819125dc7b3c2ad524adb316eb92a0eaebd085de0e59c8e2952c |
Hashes for libdash-0.3.1-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c2657223a647dc948af543574959b46c5d529d7ace5e6c58a220f4e560900895 |
|
MD5 | fa9940331846e01eef4b8e44b4920081 |
|
BLAKE2b-256 | d12b8bf69dfe97dc612966ce96c52ad85a079fc193aef26aad74e52886809d84 |
Hashes for libdash-0.3.1-cp39-cp39-musllinux_1_1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 524131037043062229be7838a6e6e026f48ec5b2bb5056b8cf9ce74389bf2f21 |
|
MD5 | f5a9de6cc950f36fe697de1ba2301e3a |
|
BLAKE2b-256 | c8b2b8916feaf0b66958938ac0f45221fc960ff8dc346097ce2fefb4ad3cbdb2 |
Hashes for libdash-0.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 256c32eadfb266bae3acece54b387ae6e6f8211885b35ec278a0c84e128e7ca6 |
|
MD5 | f7563078529a6d454ec4c31f20e677a7 |
|
BLAKE2b-256 | 60c4bc31f72a994de719493206711972e026128e75e161a7e4c41502cd9a723f |
Hashes for libdash-0.3.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 546102cd650f37d20dc6b00ee9b8bfd2190bda162e2a254e1442cafd98701973 |
|
MD5 | 5e6e031ec32c96eaae38839ef2c1267d |
|
BLAKE2b-256 | fbc2f395372ecdd3b690514a1992f3b26365e8806732a62a9ccafdf1b23511bd |
Hashes for libdash-0.3.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b4ec3a7f7e018d8908ad0e2b0f2c157817d8e387a8b6c94dea7b162ec4b91d1 |
|
MD5 | e4db34d13ad32b00f7510e88e69b47eb |
|
BLAKE2b-256 | b03e160947c244e8508f3ba00e8b41d6ec8dfa296a96aac16b8dca2e0e87bbbc |
Hashes for libdash-0.3.1-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7074af45335082aa04ab723459af3b00018e281bdf85bdf17f230cbdfb465431 |
|
MD5 | 1302cb5de48e864ad08d26b002073bcf |
|
BLAKE2b-256 | 234aa1a69f306c439df70c27f2d40a2097efdb09bd819bf50867594c4d7aa865 |
Hashes for libdash-0.3.1-cp38-cp38-musllinux_1_1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d911d69708e16e6da248597099e87a7004da1d56b2e922227e471f4630069f05 |
|
MD5 | 551b1632fcbf0eef0e81cbeb2537a4d8 |
|
BLAKE2b-256 | f63eabcb27f30bc9df2d2137e37f3ddc212e7348f23532a77bd7a5794dff6cf3 |
Hashes for libdash-0.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 21aa445c52e866dc65ff3c45ad59c08ad7197bdb18dcbaece8acd33538dd6c0d |
|
MD5 | 73330fa58e0159fb48980ffa0fb7abc5 |
|
BLAKE2b-256 | 02f12144d6257a94d1c5a1a31046b070480fa34f7a3f72e17e42b1f5a91f934d |
Hashes for libdash-0.3.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 82194e7b5d9c4f0e65afc9b84a681054ed527ac8bca174ad40e98ddfca55d5d1 |
|
MD5 | ee1a2aaf0b1781661c5379e62a46317a |
|
BLAKE2b-256 | dc70aae720202d54b50a338f01f49123adf013dff419bc88e97893a405b3e4f9 |
Hashes for libdash-0.3.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1c517acb2ba95f199aa5120930da65b43b1c9638d5e7e1e5063d6ae9bb9b2bb6 |
|
MD5 | 27c2482e781f1cb31b5bbc091263a9a6 |
|
BLAKE2b-256 | 19d5c86603cc355aad1d62f619c2bfba10b0fcb77fc8b5e7d75705c4a07c7fc9 |
Hashes for libdash-0.3.1-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a765bccb1ed3b76b5d002b5aca6c97693b8a7dec88a3d862e039aab468b4fd92 |
|
MD5 | 8b6be26d38857294e7d46f9bea65ed83 |
|
BLAKE2b-256 | 039a52f291f0be0f5ed609b160747881ec5a0f5a56ae177955102843d8711426 |
Hashes for libdash-0.3.1-cp37-cp37m-musllinux_1_1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9bab0b0edc810eb3058e897ce3e6eece4b949b9853b2b6f97b5e2a4ecedbfb1e |
|
MD5 | aa01d903bc27f74db8ce2271f9b1a546 |
|
BLAKE2b-256 | 9a80436ad4ffcab4808a33f3ff1806599f4117feaa2008a46b1c4fdf8a2ccf18 |
Hashes for libdash-0.3.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1420b33dcc07b8fc10ac94f21519c48b8a446fdee6593a7ba899550aa59e4dae |
|
MD5 | 2da85ea70003b2ad2792a221948009c7 |
|
BLAKE2b-256 | 9e8f24cf7f0679e05fc3481f2fd8902d94c7b33da77f7d9ae05d97713e7f0288 |
Hashes for libdash-0.3.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b8ec0c863caf70aa4660f17e600a937b266238fd67c5f92b3fe33d696f791ed3 |
|
MD5 | 56ce0ff79417188f54185378cd3df04a |
|
BLAKE2b-256 | aceb4a9a7a1668eaf2c13668f430a0cee44782dc5ad6bde440e5f94e694a73a7 |
Hashes for libdash-0.3.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8e5c36cf5eeb970d0a1f8d16888c29af5be677497ee52bc32b6dff263599ffc2 |
|
MD5 | a9e0ac2dd6965da5254099d5f71b26e6 |
|
BLAKE2b-256 | 8cb8f688e70709ef5573c82bc2f96373b8bb2a1d1fadf9ead78c97af1883f4e4 |