django-sub-query 0.1

Django app which uses SQL sub-queries to solve some ORM limitations

Django app which uses SQL sub-queries to solve some ORM limitations

• Free software: MIT license

• GitHub: https://github.com/miki725/django-sub-query

Why?

Django ORM is pretty awesome however it has some limitations. One of such limitations is the ability to use sub-queries. Actually thats not 100% true since Django ORM can use nested queries however those subqueries are for either aggregates or are manually provided by the user via QuerySet.extra(). Such types of subqueries however sometimes are not good enough. One such scenario is when you need to sort by one column however and at the same time you need to use distinct on a different column. The usual solution is to use subquery where inner query will use distinct and outer query will do the sorting:

SELECT *
FROM (
  SELECT DISTINCT ON ("table"."foo") <lots of columns here>
  FROM "table"
) "table"
ORDER BY "table"."bar" ASC;

The above however is not supported by vanilla Django ORM. You can of course use raw SQL queries however that is not desired since then you loose all of the ORM power (e.g. pagination, etc).

One hack to still use ORM and yet use subquery is use Django ORM to construct inner query and then manually add outer query:

query = Model.objects.filter(...).distinct(...).query
sql, params = query.sql_with_params()
Model.objects.raw(
    'SELECT * FROM ({}) "table" ORDER BY "table"."foo"'.format(sql),
    params
)

This approach however is also not desired since due to:

• not able to use select_related() when related table has similar column name(s)

• still difficult to do pagination

• more logic required to be able to do .count()

Solution

As you saw above, currently it is difficult to use sub-queries in Django however conceptually the solution is pretty simply since we should be able to simply wrap SQL generated by Django ORM within an outer query. This is exactly what this library does. It hooks up directly into Django’s as_sql() ORM method which is responsible for generating SQL and adds outer query when necessary conditions are met.

Installing

You can install django-sub-query using pip:

$ pip install django-sub-query

Using

Since this library changes a few bits in inner-workings in Django ORM, there is no simple way to use this library other then completely change database engine:

DATABASES = {
    'default': {
        'ENGINE': 'sub_query.db.backends.postgis',
        ...
    },
}

Currently only postgis backend is supported however please feel free to open an issue to add support for other backends (or contribute implementation!).

Once the database engine is changed, then you have to make sure the queryset you are using is a subclass of SubQueryGeoQuerySet:

class ExampleModel(models.Model):
    ...
    objects = SubQueryGeoQuerySet.as_manager()

Testing

To run the tests you need to install testing requirements first:

$ make install

Then to run tests, you can use manage.py or simply use Makefile command:

$ python manage.py test
# or
$ make test

History

0.1.0 (2015-01-25)

• First release on PyPI.

Credits

Development Lead

• Miroslav Shubernetskiy - https://github.com/miki725

Contributors

None yet. Why not be the first?

License

The MIT License (MIT)

Copyright (c) 2015, Miroslav Shubernetskiy

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.