Skip to main content

A flexible Lisp reader.

Project description

#+title: sxpyr: a flexible Lisp reader for Python

Parse s-expressions, edn, and a variety of lisp dialects.

  • Introduction Currently supports something approximating the union of Common Lisp, Scheme, Racket, Clojure, Hy, Fennel, and Emacs Lisp.

Since sxpyr is designed to support a variety of different dialects, a second pass is almost always needed in order to match the output of native readers. This second pass is a good place to raise additional syntax errors for the dialect you are working with. Some dialects have already been implemented.

Due to deeper differences between dialects, in some cases it is necessary to configure the reader in advance. This happens in cases where a particular form is valid in two different dialects and one or both parses are sufficiently divergent that it is not possible to recover the correct parse for the other dialect.

At the moment the reader cannot be modified at runtime. This means that it cannot be used to parse named readtables or things like Racket's #lang at-exp. In the future it might be possible to pass the generator through to the functions that handle dispatch macros and make it possible to register new reader macros, but that is well out of scope at the moment.

Originally this library was intended to be vendored, however since then it is significantly increased in complexity. A stripped down version that is suitable for parsing a minimal sane dialect that can be used as a configuration language may be added in the future.

  • Usage #+begin_src bash git clone https://github.com/tgbugs/sxpyr.git pushd sxpyr pip install --user -e .[cli] python -m sxpyr.cli parse test/data/nested-splicing-unsyntax.rkt test/data/plist-test.sxpr popd #+end_src

  • Testing In order to run the tests you will need the following repos.

  • sbcl
  • ccl
  • emacs
  • org-mode
  • clojure
  • spec.alpha
  • code.specs.alpha
  • tawny-owl
  • hy
  • Fennel
  • txr

For now you will also need the following installed on your system.

  • xemacs
  • app-xemacs/xemacs-packages-all
  • guile
  • gambit
  • Fuzzing You will need a copy of afl. #+begin_src bash pip install --user -e .[fuzz] pushd test mkdir initial-inputs cp data/plist-test.sxpr initial-inputs/ # other seed files can be added as well py-afl-fuzz -m 400 -i initial-inputs/ -o fuzzing-results/ -- python -m sxpyr.cli parse --fuzz #+end_src

#+begin_src bash python -m sxpyr.cli parse test/fuzzing-results/crashes/* #+end_src

Note that initial inputs cannot be symlinks becuase afl creates hardlinks to the input files.

Also note that afl seems to die after 175k execs?

Sometimes it also seems that some errors are escaping the except block and making it be classified as crashes?

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sxpyr-0.0.4.tar.gz (84.2 kB view details)

Uploaded Source

Built Distribution

sxpyr-0.0.4-py2.py3-none-any.whl (32.8 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file sxpyr-0.0.4.tar.gz.

File metadata

  • Download URL: sxpyr-0.0.4.tar.gz
  • Upload date:
  • Size: 84.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.4

File hashes

Hashes for sxpyr-0.0.4.tar.gz
Algorithm Hash digest
SHA256 68a070b4996eb514cb878a209e15fa4797389a7b10e6172cb21353f02f5140f9
MD5 7e6695c7fc81f5a02a055aec8dd2fe19
BLAKE2b-256 63df014c20a5c4b1e8335a58c8e59d1d88389804ab153df9a117644dad2eb995

See more details on using hashes here.

File details

Details for the file sxpyr-0.0.4-py2.py3-none-any.whl.

File metadata

  • Download URL: sxpyr-0.0.4-py2.py3-none-any.whl
  • Upload date:
  • Size: 32.8 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.4

File hashes

Hashes for sxpyr-0.0.4-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 b0e4d824055b8ae4e426ebd5e2e61976f973fdcff28af901f38c261dedb3a142
MD5 e4bd4e11fce7abe81a89bd288d893541
BLAKE2b-256 9063134e7feb2b913b81f038d6853b6bcb37feccf29d4af084e1428a2c665aad

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page