Skip to main content

Transforms tokens into original source code (while preserving whitespace).

Project description

untokenize transforms tokens into source code. Unlike the standard library’s tokenize.untokenize(), it preserves the original whitespace between tokens.

Build status

Usage

import untokenize
source_code = untokenize.untokenize(tokens)

Tests

To run the unit tests:

$ ./test_untokenize.py

There is also an acid test. It tokenizes Python code and confirms that the code generated by untokenize exactly matches the original source code from before tokenization:

$ ./test_acid.py

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

untokenize-0.1.1.tar.gz (3.1 kB view details)

Uploaded Source

File details

Details for the file untokenize-0.1.1.tar.gz.

File metadata

  • Download URL: untokenize-0.1.1.tar.gz
  • Upload date:
  • Size: 3.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for untokenize-0.1.1.tar.gz
Algorithm Hash digest
SHA256 3865dbbbb8efb4bb5eaa72f1be7f3e0be00ea8b7f125c69cbd1f5fda926f37a2
MD5 50d325dff09208c624cc603fad33bb0d
BLAKE2b-256 f746e7cea8159199096e1df52da20a57a6665da80c37fb8aeb848a3e47442c32

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page