Skip to main content

Library to convert those MongoDB queries to something else, like a python expresion, a function or a Django Q object tree to be used with a ORM query.

Project description

Build Status Coverage Status PYPI Package PYPI Package

Library to convert those MongoDB queries to something else, like a python expresion, a function or a Django Q object tree to be used with a ORM query.

For now, only supports flat operations. No subdocuments. It might work but results are undefined/buggy. Could be fixed though …

Installation

pip install mongoql-conv

Or:

pip install mongoql-conv[django]

API

to_string

>>> from mongoql_conv import to_string

>>> to_string({"myfield": 1})
"row['myfield'] == 1"

>>> to_string({})
'True'

>>> set(to_string({"field1": 1, "field2": 2}).split(' and ')) == {"(row['field2'] == 2)", "(row['field1'] == 1)"}
True

>>> to_string({"myfield": 1}, object_name='item')
"item['myfield'] == 1"

>>> to_string({"myfield": {"$in": [1, 2]}})
"row['myfield'] in {1, 2}"

>>> to_string({"myfield": {"$in": {1: 2}}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_string({"myfield": {"$and": []}})
'True'

to_string: Supported operators

to_string: Supported operators: Arithmetic

  • $gt:

    >>> to_string({"myfield": {"$gt": 1}})
    "row['myfield'] > 1"
    >>> to_string({"myfield": {"$gt": [1]}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part [1]. Expected one of: int, long, float, str, unicode, bool, None.
    
  • $gte:

    >>> to_string({"myfield": {"$gte": 1}})
    "row['myfield'] >= 1"
    
  • $lt:

    >>> to_string({"myfield": {"$lt": 1}})
    "row['myfield'] < 1"
    
  • $lte:

    >>> to_string({"myfield": {"$lte": 1}})
    "row['myfield'] <= 1"
    
  • $eq:

    >>> to_string({"myfield": {"$eq": 1}})
    "row['myfield'] == 1"
    >>> to_string({"myfield": 1})
    "row['myfield'] == 1"
    
  • $ne:

    >>> to_string({"myfield": {"$ne": 1}})
    "row['myfield'] != 1"
    
  • $mod:

    >>> to_string({"myfield": {"$mod": [2, 1]}})
    "row['myfield'] % 2 == 1"
    >>> to_string({"myfield": {"$mod": [2, 1, 3]}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
    >>> to_string({"myfield": {"$mod": 2}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
    >>> to_string({"myfield": {"$mod": (2, 1)}})
    "row['myfield'] % 2 == 1"
    

to_string: Supported operators: Containers

  • $in:

    >>> to_string({"myfield": {"$in": (1, 2, 3)}})
    "row['myfield'] in {1, 2, 3}"
    
  • $nin:

    >>> to_string({"myfield": {"$nin": [1, 2, 3]}})
    "row['myfield'] not in {1, 2, 3}"
    >>> to_string({"myfield": {"$nin": {1: 2}}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
    
  • $size:

    >>> to_string({"myfield": {"$size": 3}})
    "len(row['myfield']) == 3"
    >>> to_string({"myfield": {"$size": "3"}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part '3'. Expected one of: int, long.
    
  • $all:

    >>> to_string({"myfield": {"$all": [1, 2, 3]}})
    "set(row['myfield']) >= {1, 2, 3}"
    >>> to_string({"myfield": {"$all": 1}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
    
  • $exists:

    >>> to_string({"myfield": {"$exists": True}})
    "'myfield' in row"
    >>> to_string({"myfield": {"$exists": False}})
    "'myfield' not in row"
    

to_string: Supported operators: Boolean operators

  • $or:

    >>> to_string({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]})
    "(row['bubu'] > 1) or (row['bubu'] < 2)"
    >>> to_string({'$or': "invalid value"})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    
  • $and:

    >>> to_string({'$and':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]})
    "(row['bubu'] > 1) and (row['bubu'] < 2)"
    >>> to_string({'$or': "invalid value"})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    
  • $*nesting*:

    >>> to_string({'$and': [
    ...     {"bubu": {"$gt": 1}},
    ...     {'$or': [
    ...         {'bubu': {'$lt': 2}},
    ...         {'$and': [
    ...             {'bubu': {'$lt': 3}},
    ...             {'bubu': {'$lt': 4}},
    ...         ]}
    ...     ]}
    ... ]})
    "(row['bubu'] > 1) and ((row['bubu'] < 2) or ((row['bubu'] < 3) and (row['bubu'] < 4)))"
    

to_string: Supported operators: Regular expressions

  • $regex:

    >>> to_string({"myfield": {"$regex": 'a'}})
    "re.search('a', row['myfield'], 0)"
    
    >>> to_string({"bubu": {"$regex": ".*x"}}, object_name='X')
    "re.search('.*x', X['bubu'], 0)"
    
    >>> to_string({"myfield": {"$regex": 'a', "$options": 'i'}})
    "re.search('a', row['myfield'], 2)"
    
    >>> closure = {}
    >>> to_string({"bubu": {"$regex": ".*x"}}, closure=closure), closure
    ("var0.search(row['bubu'])", {'var0': "re.compile('.*x', 0)"})
    
    >>> to_string({"myfield": {"$regex": 'junk('}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid regular expression 'junk(': unbalanced parenthesis
    
    >>> to_string({"myfield": {"$regex": 'a', 'junk': 'junk'}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
    
    >>> set(to_string({"myfield": {"$regex": 'a', '$nin': ['aaa']}}).split(' and ')) == {
    ...     "(re.search('a', row['myfield'], 0))",
    ...     "(row['myfield'] not in {'aaa'})"
    ... }
    True
    
    >>> to_string({"bubu": {"$regex": ".*", "$options": "junk"}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
    
    >>> to_string({"bubu": {"$options": "i"}})
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
    

to_func

>>> from mongoql_conv import to_func

>>> to_func({"myfield": 1}).source
"lambda item: (item['myfield'] == 1) # compiled from {'myfield': 1}"

>>> to_func({}).source
'lambda item: (True) # compiled from {}'

>>> list(filter(to_func({"myfield": 1}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}]

>>> list(filter(to_func({}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": [1, 2]}}).source
"lambda item, var0={1, 2}: (item['myfield'] in var0) # compiled from {'myfield': {'$in': [1, 2]}}"

>>> list(filter(to_func({"myfield": {"$in": [1, 2]}}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": {1: 2}}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_func({"myfield": {"$and": []}}).source
"lambda item: (True) # compiled from {'myfield': {'$and': []}}"

>>> list(filter(to_func({"myfield": {"$and": []}}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

to_func: Supported operators

to_func: Supported operators: Arithmetic

  • $gt:

    >>> to_func({"myfield": {"$gt": 1}}).source
    "lambda item: (item['myfield'] > 1) # compiled from {'myfield': {'$gt': 1}}"
    >>> to_func({"myfield": {"$gt": [1]}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part [1]. Expected one of: int, long, float, str, unicode, bool, None.
    
    >>> list(filter(to_func({"myfield": {"$gt": 1}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 2}, {'myfield': 3}, {'myfield': 4}]
    
  • $gte:

    >>> to_func({"myfield": {"$gte": 1}}).source
    "lambda item: (item['myfield'] >= 1) # compiled from {'myfield': {'$gte': 1}}"
    
    >>> list(filter(to_func({"myfield": {"$gte": 2}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 2}, {'myfield': 3}, {'myfield': 4}]
    
  • $lt:

    >>> to_func({"myfield": {"$lt": 1}}).source
    "lambda item: (item['myfield'] < 1) # compiled from {'myfield': {'$lt': 1}}"
    
    >>> list(filter(to_func({"myfield": {"$lt": 1}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 0}]
    
  • $lte:

    >>> to_func({"myfield": {"$lte": 1}}).source
    "lambda item: (item['myfield'] <= 1) # compiled from {'myfield': {'$lte': 1}}"
    
    >>> list(filter(to_func({"myfield": {"$lte": 1}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 0}, {'myfield': 1}]
    
  • $eq:

    >>> to_func({"myfield": {"$eq": 1}}).source
    "lambda item: (item['myfield'] == 1) # compiled from {'myfield': {'$eq': 1}}"
    >>> to_func({"myfield": 1}).source
    "lambda item: (item['myfield'] == 1) # compiled from {'myfield': 1}"
    
    >>> list(filter(to_func({"myfield": {"$eq": 2}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 2}]
    
  • $ne:

    >>> to_func({"myfield": {"$ne": 1}}).source
    "lambda item: (item['myfield'] != 1) # compiled from {'myfield': {'$ne': 1}}"
    
    >>> list(filter(to_func({"myfield": {"$ne": 2}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 0}, {'myfield': 1}, {'myfield': 3}, {'myfield': 4}]
    
  • $mod:

    >>> to_func({"myfield": {"$mod": [2, 1]}}).source
    "lambda item: (item['myfield'] % 2 == 1) # compiled from {'myfield': {'$mod': [2, 1]}}"
    >>> to_func({"myfield": {"$mod": [2, 1, 3]}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
    
    >>> to_func({"myfield": {"$mod": 2}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
    
    >>> to_func({"myfield": {"$mod": (2, 1)}}).source
    "lambda item: (item['myfield'] % 2 == 1) # compiled from {'myfield': {'$mod': (2, 1)}}"
    
    >>> list(filter(to_func({"myfield": {"$mod": (2, 1)}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 1}, {'myfield': 3}]
    

to_func: Supported operators: Containers

  • $in:

    >>> to_func({"myfield": {"$in": (1, 2, 3)}}).source
    "lambda item, var0={1, 2, 3}: (item['myfield'] in var0) # compiled from {'myfield': {'$in': (1, 2, 3)}}"
    
    >>> list(filter(to_func({"myfield": {"$in": (1, 2, 3)}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 1}, {'myfield': 2}, {'myfield': 3}]
    
  • $nin:

    >>> to_func({"myfield": {"$nin": [1, 2, 3]}}).source
    "lambda item, var0={1, 2, 3}: (item['myfield'] not in var0) # compiled from {'myfield': {'$nin': [1, 2, 3]}}"
    
    >>> to_func({"myfield": {"$nin": {1: 2}}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
    
    >>> list(filter(to_func({"myfield": {"$nin": (1, 2, 3)}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 0}, {'myfield': 4}]
    
  • $size:

    >>> to_func({"myfield": {"$size": 3}}).source
    "lambda item: (len(item['myfield']) == 3) # compiled from {'myfield': {'$size': 3}}"
    
    >>> to_func({"myfield": {"$size": "3"}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part '3'. Expected one of: int, long.
    
    >>> list(filter(to_func({"myfield": {"$size": 3}}), [{"myfield": 'x'*i} for i in range(5)]))
    [{'myfield': 'xxx'}]
    
    >>> list(filter(to_func({"myfield": {"$size": 3}}), [{"myfield": list(range(i))} for i in range(5)]))
    [{'myfield': [0, 1, 2]}]
    
  • $all:

    >>> to_func({"myfield": {"$all": [1, 2, 3]}}).source
    "lambda item, var0={1, 2, 3}: (set(item['myfield']) >= var0) # compiled from {'myfield': {'$all': [1, 2, 3]}}"
    
    >>> to_func({"myfield": {"$all": 1}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
    
    >>> list(filter(to_func({"myfield": {"$all": [3, 4]}}), [{"myfield": list(range(i))} for i in range(7)]))
    [{'myfield': [0, 1, 2, 3, 4]}, {'myfield': [0, 1, 2, 3, 4, 5]}]
    
  • $exists:

    >>> to_func({"myfield": {"$exists": True}}).source
    "lambda item: ('myfield' in item) # compiled from {'myfield': {'$exists': True}}"
    
    >>> to_func({"myfield": {"$exists": False}}).source
    "lambda item: ('myfield' not in item) # compiled from {'myfield': {'$exists': False}}"
    
    >>> list(filter(to_func({"$or": [{"field1": {"$exists": True}}, {"field2": {"$exists": False}}]}), [{"field%s" % i: i} for i in range(5)]))
    [{'field0': 0}, {'field1': 1}, {'field3': 3}, {'field4': 4}]
    

to_func: Supported operators: Boolean operators

  • $or:

    >>> to_func({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}).source
    "lambda item: ((item['bubu'] > 1) or (item['bubu'] < 2)) # compiled from {'$or': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
    
    >>> to_func({'$or': "invalid value"}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    
    >>> list(filter(to_func({'$or': [{"bubu": {"$gt": 3}}, {'bubu': {'$lt': 2}}]}), [{"bubu": i} for i in range(5)]))
    [{'bubu': 0}, {'bubu': 1}, {'bubu': 4}]
    
  • $and:

    >>> to_func({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}).source
    "lambda item: ((item['bubu'] > 1) and (item['bubu'] < 2)) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
    >>> to_func({'$or': "invalid value"}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    >>> list(filter(to_func({'$and': [{"bubu": {"$lt": 3}}, {'bubu': {'$gt': 1}}]}), [{"bubu": i} for i in range(5)]))
    [{'bubu': 2}]
    
  • $*nesting*:

    >>> to_func({'$and': [
    ...     {"bubu": {"$gt": 1}},
    ...     {'$or': [
    ...         {'bubu': {'$lt': 2}},
    ...         {'$and': [
    ...             {'bubu': {'$lt': 3}},
    ...             {'bubu': {'$lt': 4}},
    ...         ]}
    ...     ]}
    ... ]}).source
    "lambda item: ((item['bubu'] > 1) and ((item['bubu'] < 2) or ((item['bubu'] < 3) and (item['bubu'] < 4)))) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'$or': [{'bubu': {'$lt': 2}}, {'$and': [{'bubu': {'$lt': 3}}, {'bubu': {'$lt': 4}}]}]}]}"
    

to_func: Supported operators: Regular expressions

  • $regex:

    >>> to_func({"myfield": {"$regex": 'a'}}).source
    "lambda item, var0=re.compile('a', 0): (var0.search(item['myfield'])) # compiled from {'myfield': {'$regex': 'a'}}"
    
    >>> to_func({"myfield": {"$regex": 'a', "$options": 'i'}}).source
    "lambda item, var0=re.compile('a', 2): (var0.search(item['myfield'])) # compiled from {'myfield': {...}}"
    
    >>> to_func({"myfield": {"$regex": 'junk('}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid regular expression 'junk(': unbalanced parenthesis
    
    >>> to_func({"myfield": {"$regex": 'a', 'junk': 'junk'}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
    
    >>> import re
    >>> set(re.match(r'(.*): \((.*) and (.*)\)',
    ...     to_func({"myfield": {"$regex": 'a', '$nin': ['aaa']}}, use_arguments=False).source
    ... ).groups()) == {
    ...     'lambda item',
    ...     "(item['myfield'] not in {'aaa'})",
    ...     "(re.search('a', item['myfield'], 0))"
    ... }
    True
    
    >>> to_func({"bubu": {"$regex": ".*", "$options": "junk"}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
    
    >>> to_func({"bubu": {"$options": "i"}}).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
    
    >>> import string
    >>> list(filter(to_func({"myfield": {"$regex": '[a-c]', "$options": 'i'}}), [{"myfield": i} for i in string.ascii_letters]))
    [{'myfield': 'a'}, {'myfield': 'b'}, {'myfield': 'c'}, {'myfield': 'A'}, {'myfield': 'B'}, {'myfield': 'C'}]
    
    >>> list(filter(to_func({"myfield": {"$regex": '[a-c]', "$nin": ['c']}}), [{"myfield": i} for i in string.ascii_letters]))
    [{'myfield': 'a'}, {'myfield': 'b'}]
    
    >>> total = len(string.ascii_letters)
    >>> 2 * len(list(filter(
    ...     to_func({"myfield": {"$regex": '[a-z]'}}),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == total
    True
    
    >>> len(list(filter(
    ...     to_func({"myfield": {"$regex": '[a-z]', '$options': 'i'}}),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == total
    True
    
    >>> len(list(filter(
    ...     to_func({"myfield": {"$regex": '[^\d]'}}),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == total
    True
    

to_func (lax mode)

>>> from mongoql_conv import LaxNone
>>> LaxNone < 1, LaxNone > 1, LaxNone == 0, LaxNone < 0, LaxNone > 0
(False, False, False, False, False)

>>> from mongoql_conv import to_func

>>> to_func({"myfield": 1}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': 1}"

>>> to_func({}, lax=True).source
'lambda item: (True) # compiled from {}'

>>> list(filter(to_func({"bogus": 1}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[]

>>> list(filter(to_func({}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": [1, 2]}}, lax=True).source
"lambda item, var0={1, 2}: ('myfield' in item and item.get('myfield', LaxNone) in var0) # compiled from {'myfield': {'$in': [1, 2]}}"

>>> list(filter(to_func({"bogus": {"$in": [1, 2]}}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[]

>>> to_func({"myfield": {"$in": {1: 2}}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_func({"myfield": {"$and": []}}, lax=True).source
"lambda item: (True) # compiled from {'myfield': {'$and': []}}"

>>> list(filter(to_func({"bogus": {"$and": []}}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

to_func (lax mode): Supported operators

to_func (lax mode): Supported operators: Arithmetic

  • $gt:

    >>> to_func({"myfield": {"$gt": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) > 1) # compiled from {'myfield': {'$gt': 1}}"
    >>> to_func({"myfield": {"$gt": [1]}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part [1]. Expected one of: int, long, float, str, unicode, bool, None.
    
    >>> list(filter(to_func({"bogus": {"$gt": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $gte:

    >>> to_func({"myfield": {"$gte": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) >= 1) # compiled from {'myfield': {'$gte': 1}}"
    
    >>> list(filter(to_func({"bogus": {"$gte": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $lt:

    >>> to_func({"myfield": {"$lt": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) < 1) # compiled from {'myfield': {'$lt': 1}}"
    
    >>> list(filter(to_func({"bogus": {"$lt": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $lte:

    >>> to_func({"myfield": {"$lte": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) <= 1) # compiled from {'myfield': {'$lte': 1}}"
    
    >>> list(filter(to_func({"bogus": {"$lte": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $eq:

    >>> to_func({"myfield": {"$eq": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': {'$eq': 1}}"
    >>> to_func({"myfield": 1}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': 1}"
    
    >>> list(filter(to_func({"bogus": {"$eq": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $ne:

    >>> to_func({"myfield": {"$ne": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) != 1) # compiled from {'myfield': {'$ne': 1}}"
    
    >>> list(filter(to_func({"bogus": {"$ne": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $mod:

    >>> to_func({"myfield": {"$mod": [2, 1]}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) % 2 == 1) # compiled from {'myfield': {'$mod': [2, 1]}}"
    >>> to_func({"myfield": {"$mod": [2, 1, 3]}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
    
    >>> to_func({"myfield": {"$mod": 2}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
    
    >>> to_func({"myfield": {"$mod": (2, 1)}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) % 2 == 1) # compiled from {'myfield': {'$mod': (2, 1)}}"
    
    >>> list(filter(to_func({"bogus": {"$mod": (2, 1)}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    

to_func (lax mode): Supported operators: Containers

  • $in:

    >>> to_func({"myfield": {"$in": (1, 2, 3)}}, lax=True).source
    "lambda item, var0={1, 2, 3}: ('myfield' in item and item.get('myfield', LaxNone) in var0) # compiled from {'myfield': {'$in': (1, 2, 3)}}"
    
    >>> list(filter(to_func({"bogus": {"$in": (1, 2, 3)}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $nin:

    >>> to_func({"myfield": {"$nin": [1, 2, 3]}}, lax=True).source
    "lambda item, var0={1, 2, 3}: ('myfield' not in item or item.get('myfield', LaxNone) not in var0) # compiled from {'myfield': {'$nin': [1, 2, 3]}}"
    
    >>> to_func({"myfield": {"$nin": {1: 2}}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
    
    >>> list(filter(to_func({"bogus": {"$nin": (1, 2, 3)}}, lax=True), [{"myfield": i} for i in range(3)]))
    [{'myfield': 0}, {'myfield': 1}, {'myfield': 2}]
    
  • $size:

    >>> to_func({"myfield": {"$size": 3}}, lax=True).source
    "lambda item: (len(item.get('myfield', LaxNone)) == 3) # compiled from {'myfield': {'$size': 3}}"
    
    >>> to_func({"myfield": {"$size": "3"}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part '3'. Expected one of: int, long.
    
    >>> list(filter(to_func({"bogus": {"$size": 3}}, lax=True), [{"myfield": 'x'*i} for i in range(5)]))
    []
    
    >>> list(filter(to_func({"bogus": {"$size": 3}}, lax=True), [{"myfield": list(range(i))} for i in range(5)]))
    []
    
  • $all:

    >>> to_func({"myfield": {"$all": [1, 2, 3]}}, lax=True).source
    "lambda item, var0={1, 2, 3}: (set(item.get('myfield', LaxNone)) >= var0) # compiled from {'myfield': {'$all': [1, 2, 3]}}"
    
    >>> to_func({"myfield": {"$all": 1}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
    
    >>> list(filter(to_func({"bogus": {"$all": [3, 4]}}, lax=True), [{"myfield": list(range(i))} for i in range(7)]))
    []
    
  • $exists:

    >>> to_func({"myfield": {"$exists": True}}, lax=True).source
    "lambda item: ('myfield' in item) # compiled from {'myfield': {'$exists': True}}"
    
    >>> to_func({"myfield": {"$exists": False}}, lax=True).source
    "lambda item: ('myfield' not in item) # compiled from {'myfield': {'$exists': False}}"
    
    >>> list(filter(to_func({"$or": [{"bogus": {"$exists": True}}]}, lax=True), [{"field%s" % i: i} for i in range(5)]))
    []
    

to_func (lax mode): Supported operators: Boolean operators

  • $or:

    >>> to_func({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}, lax=True).source
    "lambda item: ((item.get('bubu', LaxNone) > 1) or (item.get('bubu', LaxNone) < 2)) # compiled from {'$or': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
    
    >>> to_func({'$or': "invalid value"}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    
    >>> list(filter(to_func({'$or': [{"bogus": {"$gt": 3}}, {'bogus': {'$lt': 2}}]}, lax=True), [{"bubu": i} for i in range(5)]))
    []
    
  • $and:

    >>> to_func({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}, lax=True).source
    "lambda item: ((item.get('bubu', LaxNone) > 1) and (item.get('bubu', LaxNone) < 2)) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
    >>> to_func({'$or': "invalid value"}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    >>> list(filter(to_func({'$and': [{"bogus": {"$lt": 3}}, {'bogus': {'$gt': 1}}]}, lax=True), [{"bubu": i} for i in range(5)]))
    []
    
  • $*nesting*:

    >>> to_func({'$and': [
    ...     {"bubu": {"$gt": 1}},
    ...     {'$or': [
    ...         {'bubu': {'$lt': 2}},
    ...         {'$and': [
    ...             {'bubu': {'$lt': 3}},
    ...             {'bubu': {'$lt': 4}},
    ...         ]}
    ...     ]}
    ... ]}, lax=True).source
    "lambda item: ((item.get('bubu', LaxNone) > 1) and ((item.get('bubu', LaxNone) < 2) or ((item.get('bubu', LaxNone) < 3) and (item.get('bubu', LaxNone) < 4)))) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'$or': [{'bubu': {'$lt': 2}}, {'$and': [{'bubu': {'$lt': 3}}, {'bubu': {'$lt': 4}}]}]}]}"
    

to_func (lax mode): Supported operators: Regular expressions

  • $regex:

    >>> to_func({"myfield": {"$regex": 'a'}}, lax=True).source
    "lambda item, var0=re.compile('a', 0): (var0.search(item.get('myfield', ''))) # compiled from {'myfield': {'$regex': 'a'}}"
    
    >>> to_func({"myfield": {"$regex": 'a', "$options": 'i'}}, lax=True).source
    "lambda item, var0=re.compile('a', 2): (var0.search(item.get('myfield', ''))) # compiled from {'myfield': {...}}"
    
    >>> to_func({"myfield": {"$regex": 'junk('}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid regular expression 'junk(': unbalanced parenthesis
    
    >>> to_func({"myfield": {"$regex": 'a', 'junk': 'junk'}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
    
    >>> set(re.match(r'(.*): \((.*) and (.*)\)',
    ...     to_func({"myfield": {"$regex": 'a', '$nin': ['aaa']}}, lax=True, use_arguments=False).source
    ... ).groups()) == {
    ...     "lambda item",
    ...     "(re.search('a', item.get('myfield', ''), 0))",
    ...     "('myfield' not in item or item.get('myfield', LaxNone) not in {'aaa'})"
    ... }
    True
    
    >>> to_func({"bubu": {"$regex": ".*", "$options": "junk"}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
    
    >>> to_func({"bubu": {"$options": "i"}}, lax=True).source
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
    
    >>> import string
    >>> list(filter(to_func({"bogus": {"$regex": '[a-c]', "$options": 'i'}}, lax=True), [{"myfield": i} for i in string.ascii_letters]))
    []
    
    >>> list(filter(to_func({"bogus": {"$regex": '[a-c]', "$nin": ['c']}}, lax=True), [{"myfield": i} for i in string.ascii_letters]))
    []
    
    >>> total = len(string.ascii_letters)
    >>> 2 * len(list(filter(
    ...     to_func({"bougs": {"$regex": '[a-z]'}}, lax=True),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == 0
    True
    
    >>> len(list(filter(
    ...     to_func({"bogus": {"$regex": '[a-z]', '$options': 'i'}}, lax=True),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == 0
    True
    
    >>> len(list(filter(
    ...     to_func({"bougs": {"$regex": '[^\d]'}}, lax=True),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == 0
    True
    

to_Q

Compiles down to a Django Q object tree:

>>> from mongoql_conv.django import to_Q
>>> print(to_Q({"myfield": 1}))
(AND: ('myfield', 1))

>>> print(to_Q({}))
(AND: )

>>> from test_app.models import MyModel
>>> MyModel.objects.clean_and_create([(i, i) for i in range(5)])
>>> MyModel.objects.filter(to_Q({"field1": 1}))
[<MyModel: field1=1, field2='1'>]

>>> MyModel.objects.filter(to_Q({"field1": 1, "field2": 1}))
[<MyModel: field1=1, field2='1'>]

>>> print(to_Q({"myfield": {"$in": [1, 2]}}))
(AND: ('myfield__in', [1, 2]))

>>> MyModel.objects.filter(to_Q({"field1": {"$in": [1, 2]}}))
[<MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>]

>>> print(to_Q({"myfield": {"$in": {1: 2}}}))
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> print(to_Q({"myfield": {"$and": []}}))
(AND: )

>>> MyModel.objects.filter(to_Q({"field1": {"$and": []}}))
[<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]

to_Q: Supported operators

to_Q: Supported operators: Arithmetic

  • $gt:

    >>> print(to_Q({"myfield": {"$gt": 1}}))
    (AND: ('myfield__gt', 1))
    
    >>> MyModel.objects.filter(to_Q({"field1": {"$gt": 2}}))
    [<MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    
  • $gte:

    >>> print(to_Q({"myfield": {"$gte": 1}}))
    (AND: ('myfield__gte', 1))
    
    >>> MyModel.objects.filter(to_Q({"field1": {"$gte": 2}}))
    [<MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    
  • $lt:

    >>> print(to_Q({"myfield": {"$lt": 1}}))
    (AND: ('myfield__lt', 1))
    
    >>> MyModel.objects.filter(to_Q({"field1": {"$lt": 1}}))
    [<MyModel: field1=0, field2='0'>]
    
  • $lte:

    >>> print(to_Q({"myfield": {"$lte": 1}}))
    (AND: ('myfield__lte', 1))
    
    >>> MyModel.objects.filter(to_Q({"field1": {"$lte": 1}}))
    [<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>]
    
  • $eq:

    >>> print(to_Q({"myfield": {"$eq": 1}}))
    (AND: ('myfield', 1))
    
    >>> MyModel.objects.filter(to_Q({"field1": 1}))
    [<MyModel: field1=1, field2='1'>]
    
    >>> print(to_Q({"myfield": 1}))
    (AND: ('myfield', 1))
    
    >>> MyModel.objects.filter(to_Q({"field1": {"$eq": 1}}))
    [<MyModel: field1=1, field2='1'>]
    
  • $ne:

    >>> str(to_Q({"myfield": {"$ne": 1}})) in ["(NOT (AND: ('myfield', 1)))", "(AND: (NOT (AND: ('myfield', 1))))"]
    True
    >>> MyModel.objects.filter(to_Q({"field1": {"$ne": 1}}))
    [<MyModel: field1=0, field2='0'>, <MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    
  • $mod:

    >>> print(to_Q({"myfield": {"$mod": [2, 1]}}))
    Traceback (most recent call last):
    ...
    InvalidQuery: DjangoVisitor doesn't support operator '$mod'
    

to_Q: Supported operators: Containers

  • $in:

    >>> print(to_Q({"myfield": {"$in": (1, 2, 3)}}))
    (AND: ('myfield__in', (1, 2, 3)))
    
    >>> MyModel.objects.filter(to_Q({"field1": {"$in": (1, 2)}}))
    [<MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>]
    
  • $nin:

    >>> str(to_Q({"myfield": {"$nin": [1, 2, 3]}})) in ["(NOT (AND: ('myfield__in', [1, 2, 3])))", "(AND: (NOT (AND: ('myfield__in', [1, 2, 3]))))"]
    True
    
    >>> MyModel.objects.filter(to_Q({"field1": {"$nin": (1, 2)}}))
    [<MyModel: field1=0, field2='0'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    
  • $size:

    >>> print(to_Q({"myfield": {"$size": 3}}))
    Traceback (most recent call last):
    ...
    InvalidQuery: DjangoVisitor doesn't support operator '$size'
    
  • $all:

    >>> print(to_Q({"myfield": {"$all": [1, 2, 3]}}))
    Traceback (most recent call last):
    ...
    InvalidQuery: DjangoVisitor doesn't support operator '$all'
    
  • $exists:

    >>> print(to_Q({"myfield": {"$exists": True}}))
    Traceback (most recent call last):
    ...
    InvalidQuery: DjangoVisitor doesn't support operator '$exists'
    

to_Q: Supported operators: Boolean operators

  • $or:

    >>> print(to_Q({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}))
    (OR: ('bubu__gt', 1), ('bubu__lt', 2))
    
    >>> MyModel.objects.filter(to_Q({'$or': [{"field1": {"$gt": 3}}, {'field1': {'$lt': 2}}]}))
    [<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>, <MyModel: field1=4, field2='4'>]
    
  • $and:

    >>> print(to_Q({'$and':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}))
    (AND: ('bubu__gt', 1), ('bubu__lt', 2))
    
    >>> MyModel.objects.filter(to_Q({'$and': [{"field1": {"$gt": 1}}, {'field1': {'$lt': 3}}]}))
    [<MyModel: field1=2, field2='2'>]
    
  • $*nesting*:

    >>> print(to_Q({'$and': [
    ...     {"bubu": {"$gt": 1}},
    ...     {'$or': [
    ...         {'bubu': {'$lt': 2}},
    ...         {'$and': [
    ...             {'bubu': {'$lt': 3}},
    ...             {'bubu': {'$lt': 4}},
    ...         ]}
    ...     ]}
    ... ]}))
    (AND: ('bubu__gt', 1), (OR: ('bubu__lt', 2), (AND: ('bubu__lt', 3), ('bubu__lt', 4))))
    
    >>> MyModel.objects.filter(to_Q({'$and': [
    ...     {"field1": {"$gt": 1}},
    ...     {'$or': [
    ...         {'field2': {'$lt': 2}},
    ...         {'$and': [
    ...             {'field2': {'$lt': 5}},
    ...             {'field2': {'$gt': 2}},
    ...         ]}
    ...     ]}
    ... ]}))
    [<MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    

to_Q: Supported operators: Regular expressions

  • $regex:

    >>> print(to_Q({"myfield": {"$regex": 'a'}}))
    (AND: ('myfield__regex', 'a'))
    
    >>> print(to_Q({"myfield": {"$regex": 'a', "$options": 'i'}}))
    (AND: ('myfield__iregex', 'a'))
    
    >>> print(to_Q({"myfield": {"$regex": 'junk('}}))
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid regular expression 'junk(': unbalanced parenthesis
    
    >>> print(to_Q({"myfield": {"$regex": 'a', 'junk': 'junk'}}))
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
    
    >>> "('myfield__regex', 'a')" in str(to_Q({"myfield": {"$regex": 'a', '$nin': ['aaa']}}))
    True
    >>> "(NOT (AND: ('myfield__in', ['aaa'])))" in str(to_Q({"myfield": {"$regex": 'a', '$nin': ['aaa']}}))
    True
    
    >>> print(to_Q({"bubu": {"$regex": ".*", "$options": "mxs"}}))
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part 'mxs'. Unsupported regex option 'm'. Only i are supported !
    
    >>> print(to_Q({"bubu": {"$options": "i"}}))
    Traceback (most recent call last):
    ...
    InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
    
    >>> MyModel.objects.clean_and_create([(None, "prefix__"+i) for i in string.ascii_letters])
    >>> MyModel.objects.filter(to_Q({"field2": {"$regex": '[a-b]', "$options": 'i'}}))
    [<MyModel: field1=None, field2='prefix__a'>, <MyModel: field1=None, field2='prefix__b'>, <MyModel: field1=None, field2='prefix__A'>, <MyModel: field1=None, field2='prefix__B'>]
    
    >>> MyModel.objects.filter(to_Q({"field2": {"$regex": '[a-c]', "$nin": ['prefix__c']}}))
    [<MyModel: field1=None, field2='prefix__a'>, <MyModel: field1=None, field2='prefix__b'>]
    
    >>> total = MyModel.objects.count()
    
    >>> total == 2 * MyModel.objects.filter(to_Q({"field2": {"$regex": '__[a-z]'}})).count()
    True
    
    >>> total == MyModel.objects.filter(to_Q({"field2": {"$regex": '__[a-z]', '$options': 'i'}})).count()
    True
    
    >>> total == MyModel.objects.filter(to_Q({"field2": {"$regex": '[^\d]'}})).count()
    True
    

Extending (implementing a custom visitor)

There are few requirements for a visitor. Fist, you need to be able to render boolean $and:

>>> from mongoql_conv import BaseVisitor
>>> class MyVisitor(BaseVisitor):
...     def __init__(self, object_name):
...         self.object_name = object_name
...     def visit_foobar(self, value, field_name, context):
...         return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
>>> MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
Traceback (most recent call last):
...
TypeError: Can't instantiate abstract class MyVisitor with abstract methods render_and

This is the minimal code to have a custom generator:

>>> class MyVisitor(BaseVisitor):
...     def __init__(self, object_name):
...         self.object_name = object_name
...     def visit_foobar(self, value, field_name, context):
...         return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
...     def render_and(self, parts, field_name, context):
...         return ' & '.join(parts)
>>> MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
"foobar(obj['field'], 'test')"

Ofcourse, it won’t do much:

>>> MyVisitor('obj').visit({'field': {'$ne': 'test'}})
Traceback (most recent call last):
...
InvalidQuery: MyVisitor doesn't support operator '$ne'

Take a look at ExprVisitor too see all the methods you should implement.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for mongoql-conv, version 0.4.1
Filename, size File type Python version Upload date Hashes
Filename, size mongoql_conv-0.4.1-py2.py3-none-any.whl (16.0 kB) File type Wheel Python version 2.7 Upload date Hashes View hashes
Filename, size mongoql-conv-0.4.1.tar.gz (25.0 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page