EBNFParser 0.1.4

very powerful and optional parser framework for python

EBNFParser

Parse Many, Any, Every

HomePage

Multi-Language-Versions

• Python Project (v0.1.4)

  • What’s new in EBNFParser 0.1.4

• C# Project(unfinished)

---

An Introduce to EBNFParser

EBNFParser seems to be a parser framework for EBNF, however, it’s just what I want to do at the beginning, and so far this framework is much more powerful than what I used to expect it to be.

As a result, I prefer to call it EEBNF(which means Extented(Extented Backus-Naur Form)).

What’s more, tokenizer is automatically generated from EEBNF, as well as the parsers for your DSL.

Here are some grammars for EEBNF, and they’re quite easy to be learnt.

I’m going to write the parsers for Lisp quickly to tell you how to use EBNFParser.

The reason why I choose Lisp is that its EEBNF codes is very very short.

Atom    := R'[^\(\)\s\`]+' # use Regex
# define a literal parser. `Atom::= R'[^\(\)\s\']+'` is ok, but the ast parsed from the two is a little different with each other.
Expr  Throw NEWLINE ::= Atom | Quote | '(' (NEWLINE* Expr* NEWLINE*)* ')'
Quote ::= '`' Expr
NEWLINE := R'\n'
Stmt Throw NEWLINE ::= (NEWLINE* Expr* NEWLINE*)*

Okay, now a parser for Lisp is finished! Let’s save this file as lisp.eebnf. Just

• download CPython 3.6(If you’re in China, go to Tsinghua Tuna Mirror

  and choose the corresponding installer for you OS.

• download EBNFParser pip install EBNFParser.

• type this codes

  parserGenerator ./lisp.eebnf ./LispParser.py -test True -comment True

And now there should be two files(testLang.py, LispParser.py) automatically generated near by lisp.eebnf.

• feel free to try any lisp codes as <Lisp Codes> with following command.

  python testLang.py Expr "(+ 1 (+ b a))" -o test1
  Stmt[Expr['('[(]
  
        Expr[Atom[+]
  
        ]
        Expr[Atom[1]
  
        ]
        Expr['('[(]
  ...

  Then see the results at test1.json, test1Ast.

  A complete EEBNF for Lisp can be found at Grammar.

  Here are more examples given at the following sections.

Each example has the same structure like:

• grammar. The only file you have to write.

• parser.py. Parser and token generated by EBNFParser.

• testLang.py. To do some testing easily.

• testn.json. The n-th testing result in JSON format.

• testnAst. The n-th testing result in S-Expr format.

Some Examples

• Lisp

  • Grammar See ./tests/Python/Lang/Lisp/grammar.

  Expr  Throw NEWLINE ::= Atom | Quote | '(' NEWLINE* Expr* NEWLINE* ')'
  Quote   ::= '`' Expr
  Atom    := R'\S+'
  NEWLINE := R'\n'
  Stmt Throw NEWLINE  ::= (NEWLINE* Expr* NEWLINE*)*

  • testCodes See ./testpy.sh.

  parserGenerator tests/Python/Lang/Lisp/grammar tests/Python/Lang/Lisp/parser.py -test True
  python tests/Python/Lang/Lisp/testLang.py Stmt "(set r 1) (define a b (+ a (+ r 1)))"  -o tests/Python/Lang/Lisp/test1

  • Result

    • JSON.

      See ./tests/Python/Lang/Lisp/test1.json.

      {
      "name": "Stmt",
      "value": [
      {
          "name": "Expr",
          "value": [
              {
                  "name": "'('",
                  "value": "(",
                  "meta": [
                      0,
                      1,
                      "<input>"
                  ]
              },
              {
                  "name": "Expr",
                  "value": [
                      {
                          "name": "Atom",
                          "value": "set",
                          "meta": [
                              0,
                              2,
                              "<input>"
                          ]
                      }
                  ]
      // ...

    • Ast See ./tests/Python/Lang/Lisp/test1Ast.

    Stmt[Expr['('[(]
    
            Expr[Atom[set]
    
            ]
            Expr[Atom[r]
    
            ]
            Expr[Atom[1]
    
            ]
            ')'[)]
    
        ]
        Expr['('[(]
    
            Expr[Atom[define]
    
            ]
            Expr[Atom[a]
    
            ]
            Expr[Atom[b]
    
            ]
            Expr['('[(]
    
                Expr[Atom[+]
    
                ]
                Expr[Atom[a]
    
                ]
                Expr['('[(]
    
                        Expr[Atom[+]
    
                        ]
    ...

• Python(Just Expression)

• ExtraPy Language

• EBNF(bootstrap)

• CmLang

• JSON

• XML

• See more at tests.

Usage

• Requirement(for Python version)

  • Python 3.6.x+

• Command

  • Generate parsers from Grammar file.

  parserGenerator <grammarFile> <outputParser>
      -lang <lang>
      -comment <comment>
      -multiline <multiline>

  • Specification

    • grammarFile the ebnf(eebnf) filename.

    • outputParser the parser filename which ends with .py.

    • lang(optional) your language name.

    • comment True or False. Default to be False.

    • multiline True or False. Default to be False.

Parser-Generator

• For Python

  It is implemented by using bootstrap EBNF gramamr.

• BootstrapParser

• BootstrapAst

• BootstrapCompile/Code Generator

Will support C# sooner.

---

License

GPL