Lexer generation
Project description
Lexit
Lexit is an open source lexer generator written in Python3.6 using new features like NamedTuple
, type hinting and __init_subclass__
hook.
JSON lexer example
from lexit import Lexer
class JsonLexer(Lexer):
NUMBER = r'-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+-]?\d+)?'
STRING = r'"(\\\"|\\\\|[^"\n])*?"i?'
L_BRACE = r'{'
R_BRACE = r'}'
L_BRACKET = r'\['
R_BRACKET = r'\]'
TRUE = r'true'
FALSE = r'false'
NULL = r'null'
COMMA = r','
COLON = r':'
ignore = r'\s+'
tokens = list(JsonLexer.lex('{"hello": "world"}'))
Requirements
- The only requirement is Python3.6+
- For testing the
pytest
library is used
Installation
pip install lexit
Error handling
try:
tokens = list(JsonLexer.lex('${"hello": "world"}'))
except LexerError as e:
print(e.pretty())
exit(1)
# Will produce the following output:
No match for character '$' in line 1 column 1
${"hello": "world"}
^
Design decisions
- Should be easy to use
- Longest match priority (
++
always wins over+
despite of the order in which the tokens are defined in the lexer class) - Self-describing errors for humans (it's should be obvious what happened and when)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
lexit-0.0.1.tar.gz
(3.3 kB
view details)
Built Distribution
lexit-0.0.1-py3-none-any.whl
(3.4 kB
view details)
File details
Details for the file lexit-0.0.1.tar.gz
.
File metadata
- Download URL: lexit-0.0.1.tar.gz
- Upload date:
- Size: 3.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
400119d224e272fda8af809100de074f7d81bd292d5a2af3212d8f64b85ee4e7
|
|
MD5 |
2d57120df102a8d106e8dcc21d7e47cf
|
|
BLAKE2b-256 |
772deeed45dd47b608d0c1f33b3a2b6f48376df96d100e7836d51b08884bf60f
|
File details
Details for the file lexit-0.0.1-py3-none-any.whl
.
File metadata
- Download URL: lexit-0.0.1-py3-none-any.whl
- Upload date:
- Size: 3.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
abf3eff2630d7a440cb222b4db4fd07e04aefb9ef3d7313cd44d7a1e912124df
|
|
MD5 |
12fada084394153c16fece2f5d4f32ee
|
|
BLAKE2b-256 |
eaf047d7c907918bac5e2882a2815130a62a447814476e093df412390c487ca0
|