Lexer generation
Project description
Lexit
Lexit is an open source lexer generator written in Python3.6 using new features like NamedTuple
, type hinting and __init_subclass__
hook.
JSON lexer example
from lexit import Lexer class JsonLexer(Lexer): NUMBER = r'-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+-]?\d+)?' STRING = r'"(\\\"|\\\\|[^"\n])*?"i?' L_BRACE = r'{' R_BRACE = r'}' L_BRACKET = r'\[' R_BRACKET = r'\]' TRUE = r'true' FALSE = r'false' NULL = r'null' COMMA = r',' COLON = r':' ignore = r'\s+' tokens = list(JsonLexer.lex('{"hello": "world"}'))
Requirements
- The only requirement is Python3.6+
- For testing the
pytest
library is used
Installation
pip install lexit
Error handling
try:
tokens = list(JsonLexer.lex('${"hello": "world"}'))
except LexerError as e:
print(e.pretty())
exit(1)
# Will produce the following output:
No match for character '$' in line 1 column 1
${"hello": "world"}
^
Design decisions
- Should be easy to use
- Longest match priority (
++
always wins over+
despite of the order in which the tokens are defined in the lexer class) - Self-describing errors for humans (it's should be obvious what happened and when)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size lexit-0.0.1-py3-none-any.whl (3.4 kB) | File type Wheel | Python version py3 | Upload date | Hashes View |
Filename, size lexit-0.0.1.tar.gz (3.3 kB) | File type Source | Python version None | Upload date | Hashes View |