Skip to main content

Tree queries with explicit opt-in, without configurability

Project description

CI Status

Query Django model trees using adjacency lists and recursive common table expressions. Supports PostgreSQL, sqlite3 (3.8.3 or higher) and MariaDB (10.2.2 or higher) and MySQL (8.0 or higher, if running without ONLY_FULL_GROUP_BY).

Supports Django 3.2 or better, Python 3.8 or better. See the GitHub actions build for more details.

Features and limitations

  • Supports only integer and UUID primary keys (for now).

  • Allows specifying ordering among siblings.

  • Uses the correct definition of depth, where root nodes have a depth of zero.

  • The parent foreign key must be named "parent" at the moment (but why would you want to name it differently?)

  • The fields added by the common table expression always are tree_depth, tree_path and tree_ordering. The names cannot be changed. tree_depth is an integer, tree_path an array of primary keys and tree_ordering an array of values used for ordering nodes within their siblings. Note that the contents of the tree_path and tree_ordering are subject to change. You shouldn’t rely on their contents.

  • Besides adding the fields mentioned above the package only adds queryset methods for ordering siblings and filtering ancestors and descendants. Other features may be useful, but will not be added to the package just because it’s possible to do so.

  • Little code, and relatively simple when compared to other tree management solutions for Django. No redundant values so the only way to end up with corrupt data is by introducing a loop in the tree structure (making it a graph). The TreeNode abstract model class has some protection against this.

  • Supports only trees with max. 50 levels on MySQL/MariaDB, since those databases do not support arrays and require us to provide a maximum length for the tree_path and tree_ordering upfront.

  • Performance optimization: The library automatically detects simple cases (single field ordering, no tree filters, no custom tree fields) and uses an optimized CTE that avoids creating a rank table, significantly improving performance for basic tree queries.

Here’s a blog post offering some additional insight (hopefully) into the reasons for django-tree-queries’ existence.

Usage

  • Install django-tree-queries using pip.

  • Extend tree_queries.models.TreeNode or build your own queryset and/or manager using tree_queries.query.TreeQuerySet. The TreeNode abstract model already contains a parent foreign key for your convenience and also uses model validation to protect against loops.

  • Call the with_tree_fields() queryset method if you require the additional fields respectively the CTE.

  • Call the order_siblings_by("field_name") queryset method if you want to order tree siblings by a specific model field. Note that Django’s standard order_by() method isn’t supported – nodes are returned according to the depth-first search algorithm.

  • Use tree_filter() and tree_exclude() for better performance when working with large tables - these filter the base table before building the tree structure.

  • Use tree_fields() to aggregate ancestor field values into arrays.

  • Create a manager using TreeQuerySet.as_manager(with_tree_fields=True) if you want to add tree fields to queries by default.

  • Until documentation is more complete I’ll have to refer you to the test suite for additional instructions and usage examples, or check the recipes below.

Recipes

Basic models

The following two examples both extend the TreeNode which offers a few agreeable utilities and a model validation method that prevents loops in the tree structure. The common table expression could be hardened against such loops but this would involve a performance hit which we don’t want – this is a documented limitation (non-goal) of the library after all.

Basic tree node

from tree_queries.models import TreeNode

class Node(TreeNode):
    name = models.CharField(max_length=100)

Tree node with ordering among siblings

Nodes with the same parent may be ordered among themselves. The default is to order siblings by their primary key but that’s not always very useful.

from tree_queries.models import TreeNode

class Node(TreeNode):
    name = models.CharField(max_length=100)
    position = models.PositiveIntegerField(default=0)

    class Meta:
        ordering = ["position"]

Add custom methods to queryset

from tree_queries.models import TreeNode
from tree_queries.query import TreeQuerySet

class NodeQuerySet(TreeQuerySet):
    def active(self):
        return self.filter(is_active=True)

class Node(TreeNode):
    is_active = models.BooleanField(default=True)

    objects = NodeQuerySet.as_manager()

Querying the tree

All examples assume the Node class from above.

Basic usage

# Basic usage, disregards the tree structure completely.
nodes = Node.objects.all()

# Fetch nodes in depth-first search order. All nodes will have the
# tree_path, tree_ordering and tree_depth attributes.
nodes = Node.objects.with_tree_fields()

# Fetch any node.
node = Node.objects.order_by("?").first()

# Fetch direct children and include tree fields. (The parent ForeignKey
# specifies related_name="children")
children = node.children.with_tree_fields()

# Fetch all ancestors starting from the root.
ancestors = node.ancestors()

# Fetch all ancestors including self, starting from the root.
ancestors_including_self = node.ancestors(include_self=True)

# Fetch all ancestors starting with the node itself.
ancestry = node.ancestors(include_self=True).reverse()

# Fetch all descendants in depth-first search order, including self.
descendants = node.descendants(include_self=True)

# Temporarily override the ordering by siblings.
nodes = Node.objects.order_siblings_by("id")

# Revert to a queryset without tree fields (improves performance).
nodes = Node.objects.with_tree_fields().without_tree_fields()

Filtering tree subsets

IMPORTANT: For large tables, always use tree_filter() or tree_exclude() to limit which nodes are processed by the recursive CTE. Without these filters, the database evaluates the entire table, which can be extremely slow.

# Get a specific tree from a forest by filtering on root category
product_tree = Node.objects.with_tree_fields().tree_filter(category="products")

# Get organizational chart for a specific department
engineering_tree = Node.objects.with_tree_fields().tree_filter(department="engineering")

# Exclude entire trees/sections you don't need
content_trees = Node.objects.with_tree_fields().tree_exclude(category="archived")

# Chain multiple tree filters for more specific trees
recent_products = (Node.objects.with_tree_fields()
                  .tree_filter(category="products")
                  .tree_filter(created_date__gte=datetime.date.today()))

# Get descendants within a filtered tree subset
product_descendants = (Node.objects.with_tree_fields()
                      .tree_filter(category="products")
                      .descendants(some_product_node))

# Filter by site/tenant in multi-tenant applications
site_content = Node.objects.with_tree_fields().tree_filter(site_id=request.site.id)

Performance note: tree_filter() and tree_exclude() filter the base table before the recursive CTE processes relationships, dramatically improving performance for large datasets compared to using regular filter() after with_tree_fields(). Best used for selecting complete trees or tree sections rather than scattered nodes.

Note that the tree queryset doesn’t support all types of queries Django supports. For example, updating all descendants directly isn’t supported. The reason for that is that the recursive CTE isn’t added to the UPDATE query correctly. Workarounds often include moving the tree query into a subquery:

# Doesn't work:
node.descendants().update(is_active=False)

# Use this workaround instead:
Node.objects.filter(pk__in=node.descendants()).update(is_active=False)

Filter by depth

If you only want nodes from the top two levels:

nodes = Node.objects.with_tree_fields().extra(
    where=["__tree.tree_depth <= %s"],
    params=[1],
)

Aggregating ancestor fields

Use tree_fields() to aggregate values from ancestor nodes into arrays. This is useful for collecting paths, permissions, categories, or any field that should be inherited down the tree hierarchy.

# Aggregate names from all ancestors into an array
nodes = Node.objects.with_tree_fields().tree_fields(
    tree_names="name",
)
# Each node now has a tree_names attribute: ['root', 'parent', 'current']

# Aggregate multiple fields
nodes = Node.objects.with_tree_fields().tree_fields(
    tree_names="name",
    tree_categories="category",
    tree_permissions="permission_level",
)

# Build a full path string from ancestor names
nodes = Node.objects.with_tree_fields().tree_fields(tree_names="name")
for node in nodes:
    full_path = " > ".join(node.tree_names)  # "Root > Section > Subsection"

# Combine with tree filtering for better performance
active_nodes = (Node.objects.with_tree_fields()
                .tree_filter(is_active=True)
                .tree_fields(tree_names="name"))

The aggregated fields contain values from all ancestors (root to current node) in hierarchical order, including the current node itself.

Form fields

django-tree-queries ships a model field and some form fields which augment the default foreign key field and the choice fields with a version where the tree structure is visualized using dashes etc. Those fields are tree_queries.fields.TreeNodeForeignKey, tree_queries.forms.TreeNodeChoiceField, tree_queries.forms.TreeNodeMultipleChoiceField.

Templates

django-tree-queries includes template tags to help render tree structures in Django templates. These template tags are designed to work efficiently with tree querysets and respect queryset boundaries.

Setup

Add tree_queries to your INSTALLED_APPS setting:

INSTALLED_APPS = [
    # ... other apps
    'tree_queries',
]

Then load the template tags in your template:

{% load tree_queries %}

tree_info filter

The tree_info filter provides detailed information about each node’s position in the tree structure. It’s useful when you need fine control over the tree rendering.

{% load tree_queries %}
<ul>
{% for node, structure in nodes|tree_info %}
    {% if structure.new_level %}<ul><li>{% else %}</li><li>{% endif %}
    {{ node.name }}
    {% for level in structure.closed_levels %}</li></ul>{% endfor %}
{% endfor %}
</ul>

The filter returns tuples of (node, structure_info) where structure_info contains:

  • new_level: True if this node starts a new level, False otherwise

  • closed_levels: List of levels that close after this node

  • ancestors: List of ancestor node representations from root to immediate parent

Example showing ancestor information:

{% for node, structure in nodes|tree_info %}
    {{ node.name }}
    {% if structure.ancestors %}
        (Path: {% for ancestor in structure.ancestors %}{{ ancestor }}{% if not forloop.last %} > {% endif %}{% endfor %})
    {% endif %}
{% endfor %}

recursetree tag

The recursetree tag provides recursive rendering similar to django-mptt’s recursetree tag, but optimized for django-tree-queries. It only considers nodes within the provided queryset and doesn’t make additional database queries.

Basic usage:

{% load tree_queries %}
<ul>
{% recursetree nodes %}
    <li>
        {{ node.name }}
        {% if children %}
            <ul>{{ children }}</ul>
        {% endif %}
    </li>
{% endrecursetree %}
</ul>

The recursetree tag provides these context variables within the template:

  • node: The current tree node

  • children: Rendered HTML of child nodes (from the queryset)

  • is_leaf: True if the node has no children in the queryset

Using is_leaf for conditional rendering:

{% recursetree nodes %}
    <div class="{% if is_leaf %}leaf-node{% else %}branch-node{% endif %}">
        <span class="node-name">{{ node.name }}</span>
        {% if children %}
            <div class="children">{{ children }}</div>
        {% elif is_leaf %}
            <span class="leaf-indicator">🍃</span>
        {% endif %}
    </div>
{% endrecursetree %}

Advanced example with depth information:

{% recursetree nodes %}
    <div class="node depth-{{ node.tree_depth }}"
         data-id="{{ node.pk }}"
         data-has-children="{{ children|yesno:'true,false' }}">
        <h{{ node.tree_depth|add:1 }}>{{ node.name }}</h{{ node.tree_depth|add:1 }}>
        {% if children %}
            <div class="node-children">{{ children }}</div>
        {% endif %}
    </div>
{% endrecursetree %}

Working with limited querysets

Both template tags respect queryset boundaries and work efficiently with filtered or limited querysets:

# Only nodes up to depth 2
limited_nodes = Node.objects.with_tree_fields().extra(
    where=["__tree.tree_depth <= %s"], params=[2]
)

# Only specific branches
branch_nodes = Node.objects.descendants(some_node, include_self=True)

When using these limited querysets:

  • recursetree will only render nodes from the queryset

  • is_leaf reflects whether nodes have children in the queryset, not in the full tree

  • No additional database queries are made

  • Nodes whose parents aren’t in the queryset are treated as root nodes

Example with depth-limited queryset:

<!-- Template -->
{% recursetree limited_nodes %}
    <li>
        {{ node.name }}
        {% if is_leaf %}
            <small>(leaf in limited view)</small>
        {% endif %}
        {{ children }}
    </li>
{% endrecursetree %}

This is particularly useful for creating expandable tree interfaces or rendering only portions of large trees for performance.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django_tree_queries-0.20.0.tar.gz (18.1 kB view details)

Uploaded Source

Built Distribution

django_tree_queries-0.20.0-py3-none-any.whl (21.1 kB view details)

Uploaded Python 3

File details

Details for the file django_tree_queries-0.20.0.tar.gz.

File metadata

  • Download URL: django_tree_queries-0.20.0.tar.gz
  • Upload date:
  • Size: 18.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-httpx/0.28.1

File hashes

Hashes for django_tree_queries-0.20.0.tar.gz
Algorithm Hash digest
SHA256 5439d8bf9f4eafab66175d23ddd4e0a639b1be47cfc357561ff740ef7b6f4df3
MD5 54a9bc2be71fd7e46cfaf030cc3f2613
BLAKE2b-256 803d86e979dc25c15bf0b7187c5c063bf2ef1296cca9515b7542379b9d11f8bb

See more details on using hashes here.

File details

Details for the file django_tree_queries-0.20.0-py3-none-any.whl.

File metadata

File hashes

Hashes for django_tree_queries-0.20.0-py3-none-any.whl
Algorithm Hash digest
SHA256 56e14105a894b801e41da82ef9e82151d06b1f481cfe39f792c08b6c87c49fce
MD5 ecceb7514e2dcab6bd8527ad0ea3890a
BLAKE2b-256 a47fcbc208164239904b4a00065eec327d02071ef6c950c8f2d2288150b5ab33

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page