openexpressions 1.4

A easy to use and expandable expression parser

OpenExpressions

The Most Simple And Expandable Python Expression Library

OpenExpressions is a lightweight, open-source, and easily integrated expression library.

Features

• Parse and Evaluate Expressions With Built-In and Self-Implemented Functionality
• Inspect Abstract Syntax Tree generated by the Parser
• Quickly Create New Operators and Operands by Inheriting from Predefined Abstract Classes
• Default Modes Available for Math or Boolean Expressions
• Empty Mode Available for Clean Slate Start for New Expression Parser

OpenExpressions generates a context-free grammar from the given operators and operands and then creates an LR(1) Automata and Parse Table to quickly generate and Abstract Syntax Tree which represents the expression provided. The expression can then be evaluated from the leaf nodes up and the generated structure of the nodes determines the order of operations.

The goal of this project is to provide a better alternative to the native eval function that python provides for evaluating expressions. Although it can provide the functionality, the security risks can quickly become an issue if not used carefully. Additionally, using custom operations in eval must be done with function calls which can quickly become verbose in longer expressions OpenExpressions' options to create new Unary and Binary operators allows for simpler expression syntax and more readable expressions. It also resolves the security risks associated with eval as all the expression is not executed as code but instead interpreted as the available operators and operands.

Installation

OpenExpressions is available on PyPi and easily installable on Python 3.3+ using pip

pip install openexpressions

Source code is available at this Github Repository

Module info is available at PyPi Page

Documentation

Basic Usage

The basic usage of this library consists of using the preset of the Parser. Currently, the 2 presets are: math (default) and boolean

from openexpressions.Parser import Parser

if __name__ == "__main__":
    # create the parser
    math_parser = Parser() # or explicitly with math_parser = Parser(mode="math")
    # using the parser, store the parsed expression
    expression = math_parser.parse("1 + (2 ** a ** b - 4 / -c)")
    # expression is now a Tree of ExpressionNodes, each one is an Operator or Operand
    
    # evaluate the expression by passing in a context we would like for variables
    value_0 = expression.eval({'a': 1, 'b': 3, 'c': -5})
    # we can reuse the same expression to evaluate with a different context
    value_1 = expression.eval({'a': 0.1, 'b': 9, 'c': 18.2})
    
    # you can also reuse the same parser for multiple expression strings
    other_expression = math_parser.parse("i * like * math")
    value_2 = other_expression.eval({'i': 5, 'like': 32, 'math': 64})

Here's an equivalent usage for the boolean mode

from openexpressions.Parser import Parser

if __name__ == "__main__":
    # create the parser
    bool_parser = Parser(mode="boolean")
    # using the parser, store the parsed expression
    expression = bool_parser.parse("1 & a | 0 | (b ^ c)")
    # expression is now a Tree of ExpressionNodes, each one is an Operator or Operand
    
    # evaluate the expression by passing in a context we would like for variables
    value_0 = expression.eval({'a': 1, 'b': 0, 'c': 1})
    # we can reuse the same expression to evaluate with a different context
    value_1 = expression.eval({'a': 0, 'b': 1, 'c': 0})
    
    # you can also reuse the same parser for multiple expression strings
    other_expression = bool_parser.parse("i | like | boolean")
    value_2 = other_expression.eval({'i': 1, 'like': 1, 'boolean': 1})

With the preset modes, you can use the following default Operators and Operands

Built-Ins

Source code for built-in operators and operands found here

This provides an overview of each of the default values, to better understand the significance of each field, taking a look at the Expression Node Declarations

Math Mode Built-Ins

Operands

• Int - An Integer Literal - identifier regex: \d+(?!\.)
• Float - An Floating Point Literal - identifier regex: \d*\.\d+
• Var - A Variable Usage - identifier regex: [a-zA-Z]\w*

Unary Operators (UnOp)

• Neg - Arithmetic Negation - identifier regex: - order value: 70000

Binary Operators (BinOp)

• Add - Arithmetic Addition - identifier regex: \+ order value: 100000
• Sub - Arithmetic Subtraction - identifier regex: - order value: 100000
• Mult - Arithmetic Multiplication - identifier regex: \* order value: 90000
• Div - _Arithmetic Division - identifier regex: / order value: 90000
• IntDiv - Integer Division - identifier regex: // order value: 90000
• Mod - Arithmetic Modulo - identifier regex: % order value: 90000
• Pow - Arithmetic Exponentiation - identifier regex: \*\* order value: 80000

Poly Operators (PolyOp)

• Sum - Summation Operator - identifier regex: SUM

• number of sub-expressions: 4
• first sub-expression: counter variable
• second sub-expression: lower bound count
• third sub-expression: upper bound count
• fourth sub-expression: evaluated expression

• Prod - Product Notation Operator - identifier regex: PROD

• number of sub-expressions: 4
• first sub-expression: counter variable
• second sub-expression: lower bound count
• third sub-expression: upper bound count
• fourth sub-expression: evaluated expression

Wrap Operators (WrapOp)

• Paren - Parenthetical Expression - left identifier regex: \( - right identifier regex: \)
• Abs - Absolute Value Expression - left identifier regex: \| - right identifier regex: \|

Boolean Mode Built-Ins

Operands

• BoolVal - A Boolean Literal - identifier regex: (0 | 1)
• BoolVar - A Boolean Variable - identifier regex: [a-zA-Z]\w*

Unary Operators (UnOp)

• Not - Arithmetic Negation - identifier regex: ~ order value: 70000

Binary Operators (BinOp)

• BitAnd - Logical And - identifier regex: & order value: 90000
• BitOr - Logical Or - identifier regex: | order value: 100000
• BitXOr - Logical XOr - identifier regex: \^ order value: 80000

Wrap Operators (WrapOp)

• Paren - Parenthetical Expression - left identifier regex: \( - right identifier regex: \)

Advanced Usage

Advanced usage of this library consists of creating custom operators and operands, here we'll go over templates and an example usage of this feature. The current options are to create new Operands, UnOps, BinOps, PolyOps, and WrapOps by inheriting from their respective abstract classes.

Custom Operand

A custom operand can be made by using the following template

from openexpressions.ExpressionNodes import Operand

class NEW_OPERAND(Operand):
    #REQUIRED - must be a python re library compatible regex
    identifier = r"0"
    #OPTIONAL - used by some operators to determine behavior.
    negation_inclusive = False # used by Pow to determine if result should be negative
    
    def __init__(self, image): # will be passed the raw string image of the operand match
        super().__init__(image) # super will store passed value in self.val
    #REQUIRED - must accept a context object
    def eval(context=None):
        #RETURN WHAT THIS OPERAND SHOULD EVALUATE TO
        return(self.val)

The parser will scan for a regex match of the given identifier and the matched string will be passed to the constructor of this class as "image". Since operands are the leaves of the expression tree, there will be no children of this ExpressionNode.

Custom Unary Operator

A custom unary operator can be made by using the following template

from openexpressions.ExpressionNodes import UnOp

class NEW_UNOP(UnOp):
    #REQUIRED - must be a python re library compatible regex
        #Ensure that operators match only 1 non-zero length substring to prevent issues
        #In other words, only use characters and not any other regex functionality
    identifier = r"UNOP"
    #OPTIONAL - used by some operators to determine behavior.
    negation_inclusive = False # used by Pow to determine if result should be negative
    
    def __init__(self, expression) -> None:
        super().__init__(expression) # super will store passed expression in self.expr
    def eval(self, context=None):
        #PERFORM OPERATION ON self.expr EVALUATION AND RETURN NEW VALUE
        return(do_something(self.expr.eval(context)))

The parser will scan for a regex match of the given identifier and the appropriate following expression will be passed into the constructor as "expression."

Custom Binary Operator

A custom binary operator can be made by using the following template

from openexpressions.ExpressionNodes import BinOp

class NEW_BINOP(BinOp):
    #REQUIRED - must be a python re library compatible regex
        #Ensure that operators match only 1 non-zero length substring to prevent issues
        #In other words, only use characters and not any other regex functionality
    identifier = r"BINOP"
    #OPTIONAL - used by some operators to determine behavior.
    negation_inclusive = False # used by Pow to determine if result should be negative
    
    def __init__(self, left_expression, right_expression) -> None:
        # store expressions in self.left and self.right
        super().__init__(left_expression, right_expression) 
    def eval(self, context=None):
        #PERFORM OPERATION ON self.left and self.right, EVALUATE AND RETURN NEW VALUE
        return(do_something(self.left.eval(context), self.right.eval(context)))

The parser will scan for a regex match of the given identifier and the appropriate preceding and following expressions will be passed into the constructor as "left_expression" and "right_expression" respectively.

Custom Poly Operator

A custom poly operator can be made by using the following template

from openexpressions.ExpressionNodes import PolyOp

class NEW_POLYOP(PolyOp):
    #REQUIRED - must be a python re library compatible regex
        #Ensure that operators match only 1 non-zero length substring to prevent issues
        #In other words, only use characters and not any other regex functionality
    identifier = r"POLYOP"
    #REQUIRED - must be an integer which represents the number of required fields
    num_fields = 3
    #OPTIONAL - used by some operators to determine behavior.
    negation_inclusive = False # used by Pow to determine if result should be negative
    
    def __init__(self, op_1, op_2, op_3) -> None:
         # super will store passed parameters in self.ops with the leftmost being index 0
         # and each subsequence paramter being at the index after the previous one
        super().__init__(op_1, op_2, op_3)
        # op_1 stored in self.op[0]
        # op_2 stored in self.op[1]
        # op_3 stored in self.op[2]
    def eval(self, context=None):
        #PERFORM OPERATION ON all OPERANDS EVALUATION AND RETURN NEW VALUE
        return(do_something(self.op[0].eval(context), self.op[1].eval(context), self.op[2].eval(context)))

The parser will scan for a regex match of the given identifier and followed by a parenthetical expression with "num_field" sub-expressions which are delimited by commas.

Custom Wrap Operator

A custom wrap operator can be made by using the following template

from openexpressions.ExpressionNodes import WrapOp

class NEW_WRAPOP(WrapOp):
    #REQUIRED - must be a python re library compatible regex
        #Ensure that operators match only 1 non-zero length substring to prevent issues
        #In other words, only use characters and not any other regex functionality
    left_identifier = r"LEFT_WRAPOP_BOUND"
    right_identifier = r"RIGHT_WRAPOP_BOUND"
    #OPTIONAL - used by some operators to determine behavior.
    negation_inclusive = False # used by Pow to determine if result should be negative
    
    def __init__(self, expression) -> None:
        super().__init__(expression) # super will store passed expression in self.expr
    def eval(self, context=None):
        #PERFORM OPERATION ON self.expr EVALUATION AND RETURN NEW VALUE
        return(do_something(self.expr.eval(context)))

The parser will scan for a regex match of the given left_identifier followed by the right_identifier and the expression between them will be passed into the constructor as "expression."

Create a Parser With Custom Operands and Operators

After creating your operands, you need to create a parser that uses them by passing them in the constructor of the parser. This can be done as shown in the following example.

Note: The parser creates an expression tree that evaluates the given expression from lowest to highest order value operators.

from openexpressions.Parser import Parser
from openexpressions.ExpressionNodes import UnOp, BinOp, PolyOp, WrapOp

# CREATE DECLARATIONS OF CUSTOM OPERANDS AND OPERATORS HERE

if __name__ == "__main__":
    # when a mode is selected, custom operators and operands are appended 
    # on top of the provided built-in ones for that mode. If you want a empty
    # start, use mode = "empty".
    
    # UnOps must be passed in as a tuple of 2 elements
        # index 0: Class
        # index 1: order value
        
    # BinOps must be passed in as a tuple of 3 elements
        # index 0: Class
        # index 1: order value
        # index 2: reversed boolean (order of evaluation for equal priority operations)
            # when reversed = False -> left to right
            # when reverse = True -> right to left
            
    # PolyOps and WrapOps can simply be passed in as a Class
    
    # Operands can simply be passed in as a Class
    
    parser = Parser(mode="math"
            custom_operators=[
                    (UnOp1, 100), (UnOp2, 99)...,
                    (BinOp1, 50, True), (BinOp2, 49, False)...
                    PolyOp1, PolyOp2..., WrapOp1, WrapOp2...
                ]
            custom_operands=[
                    Operand1, Operand2...
                ]
            )

Warnings About Custom Operators and Operands

Since this library relies on LR(1) parsing, there are restrictions on the types of context-free grammars that can be parsed using an LR(1) parser. To ensure that the parser works as expected, make sure to follow the following rules.

• Priority Values Must be Non-Negative
• Each Priority Value Must contain exclusively UnOps OR BinOps
• Each Priority Value Must contain exclusively Reversed OR Non-Reversed BinOps
• Each Priority Value Must contain exclusively Operators with unique identifiers

Full-Scale Example

To get a better idea of the capabilities of the customization for this library, lets take a look at an example setup for 3 dimensional vector expressions.

from openexpressions.Parser import Parser
from openexpressions.ExpressionNodes import UnOp,BinOp,PolyOp,WrapOp,Operand,Var,Paren
import math
import re

class Vector(Operand):
    identifier = r"<-?(\d+.\d+|\d+), -?(\d+.\d+|\d+), -?(\d+.\d+|\d+)>"
    def __init__(self, image) -> None:
        super().__init__(tuple(float(i) for i in re.findall(r"-?(\d+.\d+|\d+)", image)))
    def eval(self, context=None):
        return(self.val)

class Negate(UnOp):
    identifier = r"-"
    def __init__(self, expression) -> None:
        super().__init__(expression)
    def eval(self, context=None):
        return(tuple(-i for i in self.expr.eval(context)))

class CrossProduct(BinOp):
    identifier = r"X"
    def __init__(self, left_expression, right_expression) -> None:
        super().__init__(left_expression, right_expression)
    def eval(self, context=None):
        lv = self.left.eval(context)
        rv = self.right.eval(context)
        return((lv[1] * rv[2] - lv[2] * rv[1], lv[2] * rv[0] - lv[0] * rv[2], lv[0] * rv[1] - lv[1] * rv[0]))

class DotProduct(BinOp):
    identifier = r"\."
    def __init__(self, left_expression, right_expression) -> None:
        super().__init__(left_expression, right_expression)
    def eval(self, context=None):
        lv = self.left.eval(context)
        rv = self.right.eval(context)
        s = 0
        for i in range(3):
            s += lv[i] * rv[i]
        return(s)

class Magnitude(WrapOp):
    left_identifier = r"\|"
    right_identifier = r"\|"
    def __init__(self, expression) -> None:
        super().__init__(expression)
    def eval(self, context=None):
        v = self.expr.eval(context)
        s = 0
        for i in range(3):
            s += abs(v[i]) ** 2
        return(math.sqrt(s))

class Select(PolyOp):
    identifier = r"SELECT"
    num_fields = 3

    def __init__(self, one, two, three) -> None:
        super().__init__(one, two, three)
    def eval(self, context=None):
        one = self.ops[0].eval(context)
        two = self.ops[1].eval(context)
        three = self.ops[2].eval(context)
        return((one[0], two[1], three[2]))


if __name__ == "__main__":
    vector_parser = Parser(mode="empty",
                           custom_operands=(Vector, Var),
                           custom_operators=((Negate, 10), (CrossProduct, 20, False), (DotProduct, 20, False), Magnitude, Select, Paren))
    print(vector_parser.parse("-(SELECT(<0, -991, -992>, <-990, 1, -992>, <-990, -991, 2>) X <3, 4, 5>) . a").eval({'a': (-6, 7, -8)}))

Using this example, we can see the power of this library. It's capabilities extend beyond just numbers and it's generic nature can quickly be used to create entire new expression systems to help simplify representing them in your code.

Note: Notice that Var and Paren we imported from openexpressions.ExpressionNodes and then passed as an extra operand and operator respectively. Since we were using the "empty" mode, we can reuse these operators as they fulfilled the purpose we wanted.

License

This library operates under the MIT License

Contact

If discover some issue within this library or would like to collaborate on it's development or some other project, please reach out to me at [e-mail]. I am always happy to chat!