Build Status Coverage Status PYPI Package PYPI Package

Library to convert those MongoDB queries to something else, like a python expresion, a function or a Django Q object tree to be used with a ORM query.

For now, only supports flat operations. No subdocuments. It might work but results are undefined/buggy. Could be fixed though …

Installation

pip install mongoql-conv

Or:

pip install mongoql-conv[django]

API

to_string

>>> from mongoql_conv import to_string

>>> to_string({"myfield": 1})
"row['myfield'] == 1"

>>> to_string({})
'True'

>>> set(to_string({"field1": 1, "field2": 2}).split(' and ')) == {"(row['field2'] == 2)", "(row['field1'] == 1)"}
True

>>> to_string({"myfield": 1}, object_name='item')
"item['myfield'] == 1"

>>> to_string({"myfield": {"$in": [1, 2]}})
"row['myfield'] in {1, 2}"

>>> to_string({"myfield": {"$in": {1: 2}}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_string({"myfield": {"$and": []}})
'True'

to_string: Supported operators

to_string: Supported operators: Arithmetic

  • $gt:

    >>> to_string({"myfield": {"$gt": 1}})
"row['myfield'] > 1"
>>> to_string({"myfield": {"$gt": [1]}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part [1]. Expected one of: int, long, float, str, unicode, bool, None.

  • $gte:

    >>> to_string({"myfield": {"$gte": 1}})
"row['myfield'] >= 1"

  • $lt:

    >>> to_string({"myfield": {"$lt": 1}})
"row['myfield'] < 1"

  • $lte:

    >>> to_string({"myfield": {"$lte": 1}})
"row['myfield'] <= 1"

  • $eq:

    >>> to_string({"myfield": {"$eq": 1}})
"row['myfield'] == 1"
>>> to_string({"myfield": 1})
"row['myfield'] == 1"

  • $ne:

    >>> to_string({"myfield": {"$ne": 1}})
"row['myfield'] != 1"

  • $mod:

    >>> to_string({"myfield": {"$mod": [2, 1]}})
"row['myfield'] % 2 == 1"
>>> to_string({"myfield": {"$mod": [2, 1, 3]}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
>>> to_string({"myfield": {"$mod": 2}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
>>> to_string({"myfield": {"$mod": (2, 1)}})
"row['myfield'] % 2 == 1"

to_string: Supported operators: Containers

  • $in:

    >>> to_string({"myfield": {"$in": (1, 2, 3)}})
"row['myfield'] in {1, 2, 3}"

  • $nin:

    >>> to_string({"myfield": {"$nin": [1, 2, 3]}})
"row['myfield'] not in {1, 2, 3}"
>>> to_string({"myfield": {"$nin": {1: 2}}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

  • $size:

    >>> to_string({"myfield": {"$size": 3}})
"len(row['myfield']) == 3"
>>> to_string({"myfield": {"$size": "3"}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part '3'. Expected one of: int, long.

  • $all:

    >>> to_string({"myfield": {"$all": [1, 2, 3]}})
"set(row['myfield']) >= {1, 2, 3}"
>>> to_string({"myfield": {"$all": 1}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.

  • $exists:

    >>> to_string({"myfield": {"$exists": True}})
"'myfield' in row"
>>> to_string({"myfield": {"$exists": False}})
"'myfield' not in row"

to_string: Supported operators: Boolean operators

  • $or:

    >>> to_string({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]})
"(row['bubu'] > 1) or (row['bubu'] < 2)"
>>> to_string({'$or': "invalid value"})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.

  • $and:

    >>> to_string({'$and':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]})
"(row['bubu'] > 1) and (row['bubu'] < 2)"
>>> to_string({'$or': "invalid value"})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.

  • $*nesting*:

    >>> to_string({'$and': [
...     {"bubu": {"$gt": 1}},
...     {'$or': [
...         {'bubu': {'$lt': 2}},
...         {'$and': [
...             {'bubu': {'$lt': 3}},
...             {'bubu': {'$lt': 4}},
...         ]}
...     ]}
... ]})
"(row['bubu'] > 1) and ((row['bubu'] < 2) or ((row['bubu'] < 3) and (row['bubu'] < 4)))"

to_string: Supported operators: Regular expressions

  • $regex:

    >>> to_string({"myfield": {"$regex": 'a'}})
"re.search('a', row['myfield'], 0)"

>>> to_string({"bubu": {"$regex": ".*x"}}, object_name='X')
"re.search('.*x', X['bubu'], 0)"

>>> to_string({"myfield": {"$regex": 'a', "$options": 'i'}})
"re.search('a', row['myfield'], 2)"

>>> closure = {}
>>> to_string({"bubu": {"$regex": ".*x"}}, closure=closure), closure
("var0.search(row['bubu'])", {'var0': "re.compile('.*x', 0)"})

>>> to_string({"myfield": {"$regex": 'junk('}})
Traceback (most recent call last):
...
InvalidQuery: Invalid regular expression 'junk(': unbalanced parenthesis

>>> to_string({"myfield": {"$regex": 'a', 'junk': 'junk'}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.

>>> set(to_string({"myfield": {"$regex": 'a', '$nin': ['aaa']}}).split(' and ')) == {
...     "(re.search('a', row['myfield'], 0))",
...     "(row['myfield'] not in {'aaa'})"
... }
True

>>> to_string({"bubu": {"$regex": ".*", "$options": "junk"}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !

>>> to_string({"bubu": {"$options": "i"}})
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.

to_func

>>> from mongoql_conv import to_func

>>> to_func({"myfield": 1}).source
"lambda item: (item['myfield'] == 1) # compiled from {'myfield': 1}"

>>> to_func({}).source
'lambda item: (True) # compiled from {}'

>>> list(filter(to_func({"myfield": 1}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}]

>>> list(filter(to_func({}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": [1, 2]}}).source
"lambda item, var0={1, 2}: (item['myfield'] in var0) # compiled from {'myfield': {'$in': [1, 2]}}"

>>> list(filter(to_func({"myfield": {"$in": [1, 2]}}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": {1: 2}}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_func({"myfield": {"$and": []}}).source
"lambda item: (True) # compiled from {'myfield': {'$and': []}}"

>>> list(filter(to_func({"myfield": {"$and": []}}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

to_func: Supported operators

to_func: Supported operators: Arithmetic

  • $gt:

    >>> to_func({"myfield": {"$gt": 1}}).source
"lambda item: (item['myfield'] > 1) # compiled from {'myfield': {'$gt': 1}}"
>>> to_func({"myfield": {"$gt": [1]}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part [1]. Expected one of: int, long, float, str, unicode, bool, None.

>>> list(filter(to_func({"myfield": {"$gt": 1}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 2}, {'myfield': 3}, {'myfield': 4}]

  • $gte:

    >>> to_func({"myfield": {"$gte": 1}}).source
"lambda item: (item['myfield'] >= 1) # compiled from {'myfield': {'$gte': 1}}"

>>> list(filter(to_func({"myfield": {"$gte": 2}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 2}, {'myfield': 3}, {'myfield': 4}]

  • $lt:

    >>> to_func({"myfield": {"$lt": 1}}).source
"lambda item: (item['myfield'] < 1) # compiled from {'myfield': {'$lt': 1}}"

>>> list(filter(to_func({"myfield": {"$lt": 1}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 0}]

  • $lte:

    >>> to_func({"myfield": {"$lte": 1}}).source
"lambda item: (item['myfield'] <= 1) # compiled from {'myfield': {'$lte': 1}}"

>>> list(filter(to_func({"myfield": {"$lte": 1}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 0}, {'myfield': 1}]

  • $eq:

    >>> to_func({"myfield": {"$eq": 1}}).source
"lambda item: (item['myfield'] == 1) # compiled from {'myfield': {'$eq': 1}}"
>>> to_func({"myfield": 1}).source
"lambda item: (item['myfield'] == 1) # compiled from {'myfield': 1}"

>>> list(filter(to_func({"myfield": {"$eq": 2}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 2}]

  • $ne:

    >>> to_func({"myfield": {"$ne": 1}}).source
"lambda item: (item['myfield'] != 1) # compiled from {'myfield': {'$ne': 1}}"

>>> list(filter(to_func({"myfield": {"$ne": 2}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 0}, {'myfield': 1}, {'myfield': 3}, {'myfield': 4}]

  • $mod:

    >>> to_func({"myfield": {"$mod": [2, 1]}}).source
"lambda item: (item['myfield'] % 2 == 1) # compiled from {'myfield': {'$mod': [2, 1]}}"
>>> to_func({"myfield": {"$mod": [2, 1, 3]}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.

>>> to_func({"myfield": {"$mod": 2}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 2. Expected one of: list, tuple.

>>> to_func({"myfield": {"$mod": (2, 1)}}).source
"lambda item: (item['myfield'] % 2 == 1) # compiled from {'myfield': {'$mod': (2, 1)}}"

>>> list(filter(to_func({"myfield": {"$mod": (2, 1)}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 1}, {'myfield': 3}]

to_func: Supported operators: Containers

  • $in:

    >>> to_func({"myfield": {"$in": (1, 2, 3)}}).source
"lambda item, var0={1, 2, 3}: (item['myfield'] in var0) # compiled from {'myfield': {'$in': (1, 2, 3)}}"

>>> list(filter(to_func({"myfield": {"$in": (1, 2, 3)}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 1}, {'myfield': 2}, {'myfield': 3}]

  • $nin:

    >>> to_func({"myfield": {"$nin": [1, 2, 3]}}).source
"lambda item, var0={1, 2, 3}: (item['myfield'] not in var0) # compiled from {'myfield': {'$nin': [1, 2, 3]}}"

>>> to_func({"myfield": {"$nin": {1: 2}}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> list(filter(to_func({"myfield": {"$nin": (1, 2, 3)}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 0}, {'myfield': 4}]

  • $size:

    >>> to_func({"myfield": {"$size": 3}}).source
"lambda item: (len(item['myfield']) == 3) # compiled from {'myfield': {'$size': 3}}"

>>> to_func({"myfield": {"$size": "3"}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part '3'. Expected one of: int, long.

>>> list(filter(to_func({"myfield": {"$size": 3}}), [{"myfield": 'x'*i} for i in range(5)]))
[{'myfield': 'xxx'}]

>>> list(filter(to_func({"myfield": {"$size": 3}}), [{"myfield": list(range(i))} for i in range(5)]))
[{'myfield': [0, 1, 2]}]

  • $all:

    >>> to_func({"myfield": {"$all": [1, 2, 3]}}).source
"lambda item, var0={1, 2, 3}: (set(item['myfield']) >= var0) # compiled from {'myfield': {'$all': [1, 2, 3]}}"

>>> to_func({"myfield": {"$all": 1}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.

>>> list(filter(to_func({"myfield": {"$all": [3, 4]}}), [{"myfield": list(range(i))} for i in range(7)]))
[{'myfield': [0, 1, 2, 3, 4]}, {'myfield': [0, 1, 2, 3, 4, 5]}]

  • $exists:

    >>> to_func({"myfield": {"$exists": True}}).source
"lambda item: ('myfield' in item) # compiled from {'myfield': {'$exists': True}}"

>>> to_func({"myfield": {"$exists": False}}).source
"lambda item: ('myfield' not in item) # compiled from {'myfield': {'$exists': False}}"

>>> list(filter(to_func({"$or": [{"field1": {"$exists": True}}, {"field2": {"$exists": False}}]}), [{"field%s" % i: i} for i in range(5)]))
[{'field0': 0}, {'field1': 1}, {'field3': 3}, {'field4': 4}]

to_func: Supported operators: Boolean operators

  • $or:

    >>> to_func({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}).source
"lambda item: ((item['bubu'] > 1) or (item['bubu'] < 2)) # compiled from {'$or': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"

>>> to_func({'$or': "invalid value"}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.

>>> list(filter(to_func({'$or': [{"bubu": {"$gt": 3}}, {'bubu': {'$lt': 2}}]}), [{"bubu": i} for i in range(5)]))
[{'bubu': 0}, {'bubu': 1}, {'bubu': 4}]

  • $and:

    >>> to_func({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}).source
"lambda item: ((item['bubu'] > 1) and (item['bubu'] < 2)) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
>>> to_func({'$or': "invalid value"}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
>>> list(filter(to_func({'$and': [{"bubu": {"$lt": 3}}, {'bubu': {'$gt': 1}}]}), [{"bubu": i} for i in range(5)]))
[{'bubu': 2}]

  • $*nesting*:

    >>> to_func({'$and': [
...     {"bubu": {"$gt": 1}},
...     {'$or': [
...         {'bubu': {'$lt': 2}},
...         {'$and': [
...             {'bubu': {'$lt': 3}},
...             {'bubu': {'$lt': 4}},
...         ]}
...     ]}
... ]}).source
"lambda item: ((item['bubu'] > 1) and ((item['bubu'] < 2) or ((item['bubu'] < 3) and (item['bubu'] < 4)))) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'$or': [{'bubu': {'$lt': 2}}, {'$and': [{'bubu': {'$lt': 3}}, {'bubu': {'$lt': 4}}]}]}]}"

to_func: Supported operators: Regular expressions

  • $regex:

    >>> to_func({"myfield": {"$regex": 'a'}}).source
"lambda item, var0=re.compile('a', 0): (var0.search(item['myfield'])) # compiled from {'myfield': {'$regex': 'a'}}"

>>> to_func({"myfield": {"$regex": 'a', "$options": 'i'}}).source
"lambda item, var0=re.compile('a', 2): (var0.search(item['myfield'])) # compiled from {'myfield': {...}}"

>>> to_func({"myfield": {"$regex": 'junk('}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid regular expression 'junk(': unbalanced parenthesis

>>> to_func({"myfield": {"$regex": 'a', 'junk': 'junk'}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.

>>> import re
>>> set(re.match(r'(.*): \((.*) and (.*)\)',
...     to_func({"myfield": {"$regex": 'a', '$nin': ['aaa']}}, use_arguments=False).source
... ).groups()) == {
...     'lambda item',
...     "(item['myfield'] not in {'aaa'})",
...     "(re.search('a', item['myfield'], 0))"
... }
True

>>> to_func({"bubu": {"$regex": ".*", "$options": "junk"}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !

>>> to_func({"bubu": {"$options": "i"}}).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.

>>> import string
>>> list(filter(to_func({"myfield": {"$regex": '[a-c]', "$options": 'i'}}), [{"myfield": i} for i in string.ascii_letters]))
[{'myfield': 'a'}, {'myfield': 'b'}, {'myfield': 'c'}, {'myfield': 'A'}, {'myfield': 'B'}, {'myfield': 'C'}]

>>> list(filter(to_func({"myfield": {"$regex": '[a-c]', "$nin": ['c']}}), [{"myfield": i} for i in string.ascii_letters]))
[{'myfield': 'a'}, {'myfield': 'b'}]

>>> total = len(string.ascii_letters)
>>> 2 * len(list(filter(
...     to_func({"myfield": {"$regex": '[a-z]'}}),
...     [{"myfield": i} for i in string.ascii_letters]
... ))) == total
True

>>> len(list(filter(
...     to_func({"myfield": {"$regex": '[a-z]', '$options': 'i'}}),
...     [{"myfield": i} for i in string.ascii_letters]
... ))) == total
True

>>> len(list(filter(
...     to_func({"myfield": {"$regex": '[^\d]'}}),
...     [{"myfield": i} for i in string.ascii_letters]
... ))) == total
True

to_func (lax mode)

>>> from mongoql_conv import LaxNone
>>> LaxNone < 1, LaxNone > 1, LaxNone == 0, LaxNone < 0, LaxNone > 0
(False, False, False, False, False)

>>> from mongoql_conv import to_func

>>> to_func({"myfield": 1}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': 1}"

>>> to_func({}, lax=True).source
'lambda item: (True) # compiled from {}'

>>> list(filter(to_func({"bogus": 1}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[]

>>> list(filter(to_func({}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": [1, 2]}}, lax=True).source
"lambda item, var0={1, 2}: ('myfield' in item and item.get('myfield', LaxNone) in var0) # compiled from {'myfield': {'$in': [1, 2]}}"

>>> list(filter(to_func({"bogus": {"$in": [1, 2]}}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[]

>>> to_func({"myfield": {"$in": {1: 2}}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_func({"myfield": {"$and": []}}, lax=True).source
"lambda item: (True) # compiled from {'myfield': {'$and': []}}"

>>> list(filter(to_func({"bogus": {"$and": []}}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

to_func (lax mode): Supported operators

to_func (lax mode): Supported operators: Arithmetic

  • $gt:

    >>> to_func({"myfield": {"$gt": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) > 1) # compiled from {'myfield': {'$gt': 1}}"
>>> to_func({"myfield": {"$gt": [1]}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part [1]. Expected one of: int, long, float, str, unicode, bool, None.

>>> list(filter(to_func({"bogus": {"$gt": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
[]

  • $gte:

    >>> to_func({"myfield": {"$gte": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) >= 1) # compiled from {'myfield': {'$gte': 1}}"

>>> list(filter(to_func({"bogus": {"$gte": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
[]

  • $lt:

    >>> to_func({"myfield": {"$lt": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) < 1) # compiled from {'myfield': {'$lt': 1}}"

>>> list(filter(to_func({"bogus": {"$lt": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
[]

  • $lte:

    >>> to_func({"myfield": {"$lte": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) <= 1) # compiled from {'myfield': {'$lte': 1}}"

>>> list(filter(to_func({"bogus": {"$lte": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
[]

  • $eq:

    >>> to_func({"myfield": {"$eq": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': {'$eq': 1}}"
>>> to_func({"myfield": 1}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': 1}"

>>> list(filter(to_func({"bogus": {"$eq": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
[]

  • $ne:

    >>> to_func({"myfield": {"$ne": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) != 1) # compiled from {'myfield': {'$ne': 1}}"

>>> list(filter(to_func({"bogus": {"$ne": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
[]

  • $mod:

    >>> to_func({"myfield": {"$mod": [2, 1]}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) % 2 == 1) # compiled from {'myfield': {'$mod': [2, 1]}}"
>>> to_func({"myfield": {"$mod": [2, 1, 3]}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.

>>> to_func({"myfield": {"$mod": 2}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 2. Expected one of: list, tuple.

>>> to_func({"myfield": {"$mod": (2, 1)}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) % 2 == 1) # compiled from {'myfield': {'$mod': (2, 1)}}"

>>> list(filter(to_func({"bogus": {"$mod": (2, 1)}}, lax=True), [{"myfield": i} for i in range(5)]))
[]

to_func (lax mode): Supported operators: Containers

  • $in:

    >>> to_func({"myfield": {"$in": (1, 2, 3)}}, lax=True).source
"lambda item, var0={1, 2, 3}: ('myfield' in item and item.get('myfield', LaxNone) in var0) # compiled from {'myfield': {'$in': (1, 2, 3)}}"

>>> list(filter(to_func({"bogus": {"$in": (1, 2, 3)}}, lax=True), [{"myfield": i} for i in range(5)]))
[]

  • $nin:

    >>> to_func({"myfield": {"$nin": [1, 2, 3]}}, lax=True).source
"lambda item, var0={1, 2, 3}: ('myfield' not in item or item.get('myfield', LaxNone) not in var0) # compiled from {'myfield': {'$nin': [1, 2, 3]}}"

>>> to_func({"myfield": {"$nin": {1: 2}}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> list(filter(to_func({"bogus": {"$nin": (1, 2, 3)}}, lax=True), [{"myfield": i} for i in range(3)]))
[{'myfield': 0}, {'myfield': 1}, {'myfield': 2}]

  • $size:

    >>> to_func({"myfield": {"$size": 3}}, lax=True).source
"lambda item: (len(item.get('myfield', LaxNone)) == 3) # compiled from {'myfield': {'$size': 3}}"

>>> to_func({"myfield": {"$size": "3"}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part '3'. Expected one of: int, long.

>>> list(filter(to_func({"bogus": {"$size": 3}}, lax=True), [{"myfield": 'x'*i} for i in range(5)]))
[]

>>> list(filter(to_func({"bogus": {"$size": 3}}, lax=True), [{"myfield": list(range(i))} for i in range(5)]))
[]

  • $all:

    >>> to_func({"myfield": {"$all": [1, 2, 3]}}, lax=True).source
"lambda item, var0={1, 2, 3}: (set(item.get('myfield', LaxNone)) >= var0) # compiled from {'myfield': {'$all': [1, 2, 3]}}"

>>> to_func({"myfield": {"$all": 1}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.

>>> list(filter(to_func({"bogus": {"$all": [3, 4]}}, lax=True), [{"myfield": list(range(i))} for i in range(7)]))
[]

  • $exists:

    >>> to_func({"myfield": {"$exists": True}}, lax=True).source
"lambda item: ('myfield' in item) # compiled from {'myfield': {'$exists': True}}"

>>> to_func({"myfield": {"$exists": False}}, lax=True).source
"lambda item: ('myfield' not in item) # compiled from {'myfield': {'$exists': False}}"

>>> list(filter(to_func({"$or": [{"bogus": {"$exists": True}}]}, lax=True), [{"field%s" % i: i} for i in range(5)]))
[]

to_func (lax mode): Supported operators: Boolean operators

  • $or:

    >>> to_func({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}, lax=True).source
"lambda item: ((item.get('bubu', LaxNone) > 1) or (item.get('bubu', LaxNone) < 2)) # compiled from {'$or': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"

>>> to_func({'$or': "invalid value"}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.

>>> list(filter(to_func({'$or': [{"bogus": {"$gt": 3}}, {'bogus': {'$lt': 2}}]}, lax=True), [{"bubu": i} for i in range(5)]))
[]

  • $and:

    >>> to_func({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}, lax=True).source
"lambda item: ((item.get('bubu', LaxNone) > 1) and (item.get('bubu', LaxNone) < 2)) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
>>> to_func({'$or': "invalid value"}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
>>> list(filter(to_func({'$and': [{"bogus": {"$lt": 3}}, {'bogus': {'$gt': 1}}]}, lax=True), [{"bubu": i} for i in range(5)]))
[]

  • $*nesting*:

    >>> to_func({'$and': [
...     {"bubu": {"$gt": 1}},
...     {'$or': [
...         {'bubu': {'$lt': 2}},
...         {'$and': [
...             {'bubu': {'$lt': 3}},
...             {'bubu': {'$lt': 4}},
...         ]}
...     ]}
... ]}, lax=True).source
"lambda item: ((item.get('bubu', LaxNone) > 1) and ((item.get('bubu', LaxNone) < 2) or ((item.get('bubu', LaxNone) < 3) and (item.get('bubu', LaxNone) < 4)))) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'$or': [{'bubu': {'$lt': 2}}, {'$and': [{'bubu': {'$lt': 3}}, {'bubu': {'$lt': 4}}]}]}]}"

to_func (lax mode): Supported operators: Regular expressions

  • $regex:

    >>> to_func({"myfield": {"$regex": 'a'}}, lax=True).source
"lambda item, var0=re.compile('a', 0): (var0.search(item.get('myfield', ''))) # compiled from {'myfield': {'$regex': 'a'}}"

>>> to_func({"myfield": {"$regex": 'a', "$options": 'i'}}, lax=True).source
"lambda item, var0=re.compile('a', 2): (var0.search(item.get('myfield', ''))) # compiled from {'myfield': {...}}"

>>> to_func({"myfield": {"$regex": 'junk('}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid regular expression 'junk(': unbalanced parenthesis

>>> to_func({"myfield": {"$regex": 'a', 'junk': 'junk'}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.

>>> set(re.match(r'(.*): \((.*) and (.*)\)',
...     to_func({"myfield": {"$regex": 'a', '$nin': ['aaa']}}, lax=True, use_arguments=False).source
... ).groups()) == {
...     "lambda item",
...     "(re.search('a', item.get('myfield', ''), 0))",
...     "('myfield' not in item or item.get('myfield', LaxNone) not in {'aaa'})"
... }
True

>>> to_func({"bubu": {"$regex": ".*", "$options": "junk"}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !

>>> to_func({"bubu": {"$options": "i"}}, lax=True).source
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.

>>> import string
>>> list(filter(to_func({"bogus": {"$regex": '[a-c]', "$options": 'i'}}, lax=True), [{"myfield": i} for i in string.ascii_letters]))
[]

>>> list(filter(to_func({"bogus": {"$regex": '[a-c]', "$nin": ['c']}}, lax=True), [{"myfield": i} for i in string.ascii_letters]))
[]

>>> total = len(string.ascii_letters)
>>> 2 * len(list(filter(
...     to_func({"bougs": {"$regex": '[a-z]'}}, lax=True),
...     [{"myfield": i} for i in string.ascii_letters]
... ))) == 0
True

>>> len(list(filter(
...     to_func({"bogus": {"$regex": '[a-z]', '$options': 'i'}}, lax=True),
...     [{"myfield": i} for i in string.ascii_letters]
... ))) == 0
True

>>> len(list(filter(
...     to_func({"bougs": {"$regex": '[^\d]'}}, lax=True),
...     [{"myfield": i} for i in string.ascii_letters]
... ))) == 0
True

to_Q

Compiles down to a Django Q object tree:

>>> from mongoql_conv.django import to_Q
>>> print(to_Q({"myfield": 1}))
(AND: ('myfield', 1))

>>> print(to_Q({}))
(AND: )

>>> from test_app.models import MyModel
>>> MyModel.objects.clean_and_create([(i, i) for i in range(5)])
>>> MyModel.objects.filter(to_Q({"field1": 1}))
[<MyModel: field1=1, field2='1'>]

>>> MyModel.objects.filter(to_Q({"field1": 1, "field2": 1}))
[<MyModel: field1=1, field2='1'>]

>>> print(to_Q({"myfield": {"$in": [1, 2]}}))
(AND: ('myfield__in', [1, 2]))

>>> MyModel.objects.filter(to_Q({"field1": {"$in": [1, 2]}}))
[<MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>]

>>> print(to_Q({"myfield": {"$in": {1: 2}}}))
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> print(to_Q({"myfield": {"$and": []}}))
(AND: )

>>> MyModel.objects.filter(to_Q({"field1": {"$and": []}}))
[<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]

to_Q: Supported operators

to_Q: Supported operators: Arithmetic

  • $gt:

    >>> print(to_Q({"myfield": {"$gt": 1}}))
(AND: ('myfield__gt', 1))

>>> MyModel.objects.filter(to_Q({"field1": {"$gt": 2}}))
[<MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]

  • $gte:

    >>> print(to_Q({"myfield": {"$gte": 1}}))
(AND: ('myfield__gte', 1))

>>> MyModel.objects.filter(to_Q({"field1": {"$gte": 2}}))
[<MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]

  • $lt:

    >>> print(to_Q({"myfield": {"$lt": 1}}))
(AND: ('myfield__lt', 1))

>>> MyModel.objects.filter(to_Q({"field1": {"$lt": 1}}))
[<MyModel: field1=0, field2='0'>]

  • $lte:

    >>> print(to_Q({"myfield": {"$lte": 1}}))
(AND: ('myfield__lte', 1))

>>> MyModel.objects.filter(to_Q({"field1": {"$lte": 1}}))
[<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>]

  • $eq:

    >>> print(to_Q({"myfield": {"$eq": 1}}))
(AND: ('myfield', 1))

>>> MyModel.objects.filter(to_Q({"field1": 1}))
[<MyModel: field1=1, field2='1'>]

>>> print(to_Q({"myfield": 1}))
(AND: ('myfield', 1))

>>> MyModel.objects.filter(to_Q({"field1": {"$eq": 1}}))
[<MyModel: field1=1, field2='1'>]

  • $ne:

    >>> str(to_Q({"myfield": {"$ne": 1}})) in ["(NOT (AND: ('myfield', 1)))", "(AND: (NOT (AND: ('myfield', 1))))"]
True
>>> MyModel.objects.filter(to_Q({"field1": {"$ne": 1}}))
[<MyModel: field1=0, field2='0'>, <MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]

  • $mod:

    >>> print(to_Q({"myfield": {"$mod": [2, 1]}}))
Traceback (most recent call last):
...
InvalidQuery: DjangoVisitor doesn't support operator '$mod'

to_Q: Supported operators: Containers

  • $in:

    >>> print(to_Q({"myfield": {"$in": (1, 2, 3)}}))
(AND: ('myfield__in', (1, 2, 3)))

>>> MyModel.objects.filter(to_Q({"field1": {"$in": (1, 2)}}))
[<MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>]

  • $nin:

    >>> str(to_Q({"myfield": {"$nin": [1, 2, 3]}})) in ["(NOT (AND: ('myfield__in', [1, 2, 3])))", "(AND: (NOT (AND: ('myfield__in', [1, 2, 3]))))"]
True

>>> MyModel.objects.filter(to_Q({"field1": {"$nin": (1, 2)}}))
[<MyModel: field1=0, field2='0'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]

  • $size:

    >>> print(to_Q({"myfield": {"$size": 3}}))
Traceback (most recent call last):
...
InvalidQuery: DjangoVisitor doesn't support operator '$size'

  • $all:

    >>> print(to_Q({"myfield": {"$all": [1, 2, 3]}}))
Traceback (most recent call last):
...
InvalidQuery: DjangoVisitor doesn't support operator '$all'

  • $exists:

    >>> print(to_Q({"myfield": {"$exists": True}}))
Traceback (most recent call last):
...
InvalidQuery: DjangoVisitor doesn't support operator '$exists'

to_Q: Supported operators: Boolean operators

  • $or:

    >>> print(to_Q({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}))
(OR: ('bubu__gt', 1), ('bubu__lt', 2))

>>> MyModel.objects.filter(to_Q({'$or': [{"field1": {"$gt": 3}}, {'field1': {'$lt': 2}}]}))
[<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>, <MyModel: field1=4, field2='4'>]

  • $and:

    >>> print(to_Q({'$and':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}))
(AND: ('bubu__gt', 1), ('bubu__lt', 2))

>>> MyModel.objects.filter(to_Q({'$and': [{"field1": {"$gt": 1}}, {'field1': {'$lt': 3}}]}))
[<MyModel: field1=2, field2='2'>]

  • $*nesting*:

    >>> print(to_Q({'$and': [
...     {"bubu": {"$gt": 1}},
...     {'$or': [
...         {'bubu': {'$lt': 2}},
...         {'$and': [
...             {'bubu': {'$lt': 3}},
...             {'bubu': {'$lt': 4}},
...         ]}
...     ]}
... ]}))
(AND: ('bubu__gt', 1), (OR: ('bubu__lt', 2), (AND: ('bubu__lt', 3), ('bubu__lt', 4))))

>>> MyModel.objects.filter(to_Q({'$and': [
...     {"field1": {"$gt": 1}},
...     {'$or': [
...         {'field2': {'$lt': 2}},
...         {'$and': [
...             {'field2': {'$lt': 5}},
...             {'field2': {'$gt': 2}},
...         ]}
...     ]}
... ]}))
[<MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]

to_Q: Supported operators: Regular expressions

  • $regex:

    >>> print(to_Q({"myfield": {"$regex": 'a'}}))
(AND: ('myfield__regex', 'a'))

>>> print(to_Q({"myfield": {"$regex": 'a', "$options": 'i'}}))
(AND: ('myfield__iregex', 'a'))

>>> print(to_Q({"myfield": {"$regex": 'junk('}}))
Traceback (most recent call last):
...
InvalidQuery: Invalid regular expression 'junk(': unbalanced parenthesis

>>> print(to_Q({"myfield": {"$regex": 'a', 'junk': 'junk'}}))
Traceback (most recent call last):
...
InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.

>>> "('myfield__regex', 'a')" in str(to_Q({"myfield": {"$regex": 'a', '$nin': ['aaa']}}))
True
>>> "(NOT (AND: ('myfield__in', ['aaa'])))" in str(to_Q({"myfield": {"$regex": 'a', '$nin': ['aaa']}}))
True

>>> print(to_Q({"bubu": {"$regex": ".*", "$options": "mxs"}}))
Traceback (most recent call last):
...
InvalidQuery: Invalid query part 'mxs'. Unsupported regex option 'm'. Only i are supported !

>>> print(to_Q({"bubu": {"$options": "i"}}))
Traceback (most recent call last):
...
InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.

>>> MyModel.objects.clean_and_create([(None, "prefix__"+i) for i in string.ascii_letters])
>>> MyModel.objects.filter(to_Q({"field2": {"$regex": '[a-b]', "$options": 'i'}}))
[<MyModel: field1=None, field2='prefix__a'>, <MyModel: field1=None, field2='prefix__b'>, <MyModel: field1=None, field2='prefix__A'>, <MyModel: field1=None, field2='prefix__B'>]

>>> MyModel.objects.filter(to_Q({"field2": {"$regex": '[a-c]', "$nin": ['prefix__c']}}))
[<MyModel: field1=None, field2='prefix__a'>, <MyModel: field1=None, field2='prefix__b'>]

>>> total = MyModel.objects.count()

>>> total == 2 * MyModel.objects.filter(to_Q({"field2": {"$regex": '__[a-z]'}})).count()
True

>>> total == MyModel.objects.filter(to_Q({"field2": {"$regex": '__[a-z]', '$options': 'i'}})).count()
True

>>> total == MyModel.objects.filter(to_Q({"field2": {"$regex": '[^\d]'}})).count()
True

Extending (implementing a custom visitor)

There are few requirements for a visitor. Fist, you need to be able to render boolean $and:

>>> from mongoql_conv import BaseVisitor
>>> class MyVisitor(BaseVisitor):
...     def __init__(self, object_name):
...         self.object_name = object_name
...     def visit_foobar(self, value, field_name, context):
...         return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
>>> MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
Traceback (most recent call last):
...
TypeError: Can't instantiate abstract class MyVisitor with abstract methods render_and

This is the minimal code to have a custom generator:

>>> class MyVisitor(BaseVisitor):
...     def __init__(self, object_name):
...         self.object_name = object_name
...     def visit_foobar(self, value, field_name, context):
...         return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
...     def render_and(self, parts, field_name, context):
...         return ' & '.join(parts)
>>> MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
"foobar(obj['field'], 'test')"

Ofcourse, it won’t do much:

>>> MyVisitor('obj').visit({'field': {'$ne': 'test'}})
Traceback (most recent call last):
...
InvalidQuery: MyVisitor doesn't support operator '$ne'

Take a look at ExprVisitor too see all the methods you should implement.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ionel from gravatar.com ionel

Credits

Author: Ionel Cristian Mărieș

License

BSD License (BSD)

Tags

mongo mongodb django orm query conversion converter

Classifiers

Metadata

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ionel from gravatar.com ionel

Credits

Author: Ionel Cristian Mărieș

License

BSD License (BSD)

Tags

mongo mongodb django orm query conversion converter

Classifiers

Release files for mongoql-conv 0.4.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mongoql-conv 0.4.1
File Size Uploaded Provenance Actions
mongoql-conv-0.4.1.tar.gz 25.0 kB Without provenance Details for mongoql-conv-0.4.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for mongoql-conv 0.4.1
File Interpreter ABI Platform Actions
mongoql_conv-0.4.1-py2.py3-none-any.whl 16.0 kB Without provenance Python 2, Python 3 none any Details for mongoql_conv-0.4.1-py2.py3-none-any.whl

Total release size: 41.0 kB

Release files / mongoql-conv-0.4.1.tar.gz

Download URL mongoql-conv-0.4.1.tar.gz
Size 25.0 kB
Tags Source
SHA-256 checksum
How to use checksums 		0f9697faed8ccea4fac39f0d8be23445f4a6e860692a1e23375208988a17aa32
BLAKE2b-256 checksum
How to use checksums 		1d875cdfb17cd811267e61f5c99a7d7e66980e274d7dc0cfe2aec0baa0225df2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing? 		No

Release files / mongoql_conv-0.4.1-py2.py3-none-any.whl

Download URL mongoql_conv-0.4.1-py2.py3-none-any.whl
Size 16.0 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums 		f643dba47909e38414d39e1a10f21bf66d7d86bcb1ca0b327cd7d9f76f07352d
BLAKE2b-256 checksum
How to use checksums 		46d8524f40dcb1612750f29b9fea41e992877ad71e30a76d31ba4fbb844b3bc0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing? 		No

Security for mongoql-conv 0.4.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page