Simple library to scan or tokenize text.
Project description
nr.parsing.core
A simple library for scanning and lexing text which makes it easy to implement new DSLs (domain specific languages).
Example (Scanning)
from nr.parsing.core import Scanner
sc = Scanner('abc + 42.0')
assert sc.getmatch(r'\w+') == 'abc'
sc.match('\s*')
assert sc.char == '+'; sc.next()
sc.match('\s*')
assert sc.getmatch(r'\d+(?:\.\d*)') == '42.0'
assert sc.char == '' # eof
Example (Lexing)
from nr.parsing.core import Scanner, Lexer, Regex, Charset
rules = [
Charset('ws', ' ', skip=True),
Charset('op', '+'),
Regex('num', r'\d+(?:\.\d*)', group=0),
Regex('id', r'\w+', group=0),
]
for tok in Lexer(Scanner('abc + 42.0'), rules):
print(tok.type, tok.value)
Copyright © 2020 Niklas Rosenstein
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
nr.parsing.core-1.0.1.tar.gz
(9.8 kB
view hashes)
Built Distribution
Close
Hashes for nr.parsing.core-1.0.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6ac8aefa13490ee322672502b22f2fc8d9b621f53ab7f9cd943dd26b953bd42c |
|
MD5 | 28978d2eb1c1b2a58d6a999aae46fdcf |
|
BLAKE2b-256 | 690efb8543c8ae50aacd8bc03567b4c029ca869e8c2abbe0c617ef34aed8b04e |