EBNFParser 0.1.2

very powerful and optional parser framework for python

EBNFParser

Parse Many, Any, Every

HomePage

Multi-Language-Versions

• Python Project (v0.1.1)

  • What’s new in EBNFParser 0.1.2

• C# Project (unfinished)

---

An Introduce to EBNFParser

EBNFParser seems to be a parser framework for EBNF, however, it’s just what I want to do at the beginning, and so far this framework is much more powerful than what I used to expect it to be.

As a result, I prefer to call it EEBNF(which means Extented(Extented Backus-Naur Form)).

What’s more, tokenizer is automatically generated from EEBNF, as well as the parsers for your DSL.

Here are some grammars for EEBNF, and they’re quite easy to be learnt.

I’m going to write the parsers for Lisp quickly to tell you how to use EBNFParser.

The reason why I choose Lisp is that its EEBNF codes is very very short.

Atom  := R'[^\(\)\s]+' # use Regex
# define a literal parser. `Atom::= R'[^\(\)\s]+'` is ok, but the ast parsed from the two is a little different with each other.
Left  := '('
Right := ')'
SExpr ::= Atom | Left SExpr* Right

Okay, now a parser for Lisp is finished! Let’s save this file as lisp.eebnf.

Just

• download CPython 3.6(If you’re in China, go to Tsinghua Tuna Mirror and choose the corresponding installer for you OS.

• download EBNFParser pip install EBNFParser.

• type this codes parserGenerator.py lisp.eebnf ./parser.py -test True -comment True

And now there should be two files(testLang.py, parser.py) automatically generated near by lisp.eebnf. - feel free to try any lisp codes as <Lisp Codes> with following command.

python testLang.py SExpr `<Lisp Codes>`
# python testLang.py SExpr "(+ 1 +(2 3))" -o test1
SExpr[Left[(]

  SExpr[Atom[+]

  ]
  SExpr[Atom[1]
    ...
]
# python testLang.py SExpr "(define a x y (+ x y))" -o test2
SExpr[Left[(]

  SExpr[Atom[define]

  ]
  SExpr[Atom[a]

  ]
    ...
]

See the results at test1.json, test1Ast, test2.json, test2Ast. A complete EEBNF for Lisp can be found at [Grammar](./tests/Python/Lang/Lisp/grammar). Here are more examples given at the following sections.

Each example has the same structure like:

• grammar. The only file you have to write.

• parser.py.Parser and token generated by EBNFParser.

• testLang.py. To do some testing easily.

• testn.json. The n-th testing result in JSON format.

• testnAst. The n-th testing result in S-Expr format.

Some Examples

• Lisp

  • Grammar See ./tests/Python/Lang/Lisp/grammar.

    Expr  Throw NEWLINE ::= Atom | Quote | '(' NEWLINE* Expr* NEWLINE* ')'
    Quote   ::= '`' Expr
    Atom    := R'\S+'
    NEWLINE := R'\n'
    Stmt Throw NEWLINE  ::= (NEWLINE* Expr* NEWLINE*)*

  • testCodes See ./testpy.sh.

    export PYTHONPATH="Python"
    python Python/parserGenerator.py tests/Python/Lang/Lisp/grammar tests/Python/Lang/Lisp/parser.py -test True
    python tests/Python/Lang/Lisp/testLang.py Stmt "(set r 1) (define a b (+ a (+ r 1)))"  -o tests/Python/Lang/Lisp/test1

  • Result

    • JSON.

      See ./tests/Python/Lang/Lisp/test1.json.

      ...
      {
      "name": "Stmt",
      "value": [
          {
              "name": "Expr",
              "value": [
                  {
                      "name": "'('",
                      "value": "(",
                      "meta": {
                          "rowIdx": 0,
                          "hasParsed": 1,
                          "fileName": "<input>"
                      }
                  },
                  {
                      "name": "Expr",
                      "value": [
                          {
                              "name": "Atom",
                              "value": "set",
                              "meta": {
                                  "rowIdx": 0,
                                  "hasParsed": 2,
                                  "fileName": "<input>"
                              }
                          }
                      ],
      ...

    • Ast See ./tests/Python/Lang/Lisp/test1Ast.

      Stmt[Expr['('[(]
      
          Expr[Atom[set]
      
          ]
          Expr[Atom[r]
      
          ]
          Expr[Atom[1]
      
          ]
          ')'[)]
      
      ]
      Expr['('[(]
      
          Expr[Atom[define]
      
          ]
          Expr[Atom[a]
      
          ]
          Expr[Atom[b]
      
          ]
          Expr['('[(]
      
              Expr[Atom[+]
      
              ]
              Expr[Atom[a]
      
              ]
              Expr['('[(]
      
                      Expr[Atom[+]
      
                      ]
                      Expr[Atom[r]
      
                      ]
                      Expr[Atom[1]
      
                      ]
                      ')'[)]
      
              ]
              ')'[)]
      
          ]
          ')'[)]
      
      ]
      ]

• Python(Just Expression)

• ExtraPy Language

• EBNF(bootstrap)

• CmLang

• JSON

• XML

Usage

• Requirement(for Python version)

  • Python 3.6.x

Feel free to clone this project and make parsers for your own language quickly and easily. - Command

1. move to the root of project directory.

2. cd Python/

3. write an EBNF file to define the grammars for your own language.

4. see the following codes. Also, you can find out more details in each example listed above. Or you can see testpy.sh.

python parserGenerator.py <EBNF filename> <outputParser.py filename> -test True
python <outputParser.py filename> "<codes of your language>" -o <JSON filename>

However, if you download EBNFParser with pip, you can use it more conveniently. - Command

pip install EBNFParser
parserGenerator.py <EBNF filename> <outputParser.py filename>     python <outputParser.py filename> "<codes of your language>" -o <JSON filename>

Parser-Generator

• Python

It is implemented by using bootstrap EBNF gramamr.

• BootstrapParser

• BootstrapAst

• BootstrapCompile/Code Generator

Will support C# sooner.

---

License

GPL