Skip to main content

Python wrapper for libparsing, a PEG-based parsing library written in C

Project description

libparsing is a parsing element grammar (PEG) library written in C with Python bindings. It offers good performance while allowing for a lot of flexibility. It is mainly intended to be used to create programming languages and software engineering tools.

As opposed to more traditional parsing techniques, the grammar is not compiled but constructed using an API that allows for the dynamic update of the grammar.

The parser does not do any tokeninzation, instead, an input stream is consumed and parsing elements are dynamically asked to match the next element of it. Once parsing elements match, the resulting matched input is processed and an action is triggered.

libparsing supports the following features:

  • backtracking, ie. going back in the input stream if a match is not found
  • cherry-picking, ie. skipping unrecognized input
  • contextual rules, ie. a rule that will match or not depending on external variables

Parsing elements are usually slower than compiled or FSM-based parsers as they trade performance for flexibility. It’s probably not a great idea to use libparsing if the parsing has to happen as fast as possible (ie. a protocol implementation), but it is a great use for programming languages, as it opens up the door to dynamic syntax plug-ins and multiple language embedding.

If you’re interested in PEG, you can start reading Brian Ford’s original article. Projects such as PEG/LEG by Ian Piumarta ,OMeta by Alessandro Warth or Haskell’s Parsec library are of particular interest in the field.

Here is a short example of what creating a simple grammar looks like in Python:

g = Grammar()
s = g.symbols
g.token("WS",       "\s+")
g.token("NUMBER",   "\d+(\.\d+)?")
g.token("VARIABLE", "\w+")
g.token("OPERATOR", "[\/\+\-\*]")"Value",     s.NUMBER, s.VARIABLE)
g.rule("Suffix",     s.OPERATOR._as("operator"), s.Value._as("value"))
g.rule("Expression", s.Value, s.Suffix.zeroOrMore())
match = g.parseString("10 + 20 / 5")

and the equivalent code in C

Grammar* g = Grammar_new()
SYMBOL(WS,         TOKEN("\\s+"))
SYMBOL(NUMBER,     TOKEN("\\d+(\\.\\d+)?"))
SYMBOL(OPERATOR,   GROUP("[\\/\\+\\-\\*]"))
SYMBOL(Suffix,     RULE(_AS(_S(OPERATOR), "operator"), _AS(_S(Value), "value")))
SYMBOL(Expression, RULE(_S(Value), _MO(Suffix))
g->axiom = s_Expression;
Match* match = Grammar_parseString(g, "10 + 20 / 5")

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for libparsing, version 0.9.2
Filename, size File type Python version Upload date Hashes
Filename, size libparsing-0.9.2.linux-x86_64.tar.gz (280.0 kB) File type Dumb Binary Python version 2.7 Upload date Hashes View
Filename, size libparsing-0.9.2.tar.gz (33.3 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page