Denorm
Denormalized and aggregated tables for PostgreSQL.
Keywords: PostgreSQL, denormalization, aggregation, incremental view maintenance, materialized view
Overview
Dernom is similar to PostgreSQL's
REFRESH MATERIALIZED VIEW,
except that it updates materialized table incrementally.
How it works: Define the query, the tables, and their relationships in JSON. Denorm generates the SQL DDL statements that create the necessary functions and triggers. Apply the generated SQL statements to the database. Now, the materialized target is automatically kept in sync with the source tables.
Features
- Efficient incremental updates
- Arbitrarily complex SQL features and expressions
- Configurable consistency
- Deadlock-free
Install
Pip
pip3 install denorm
Usage
For CLI usage, see Usage.
Operations
Denorm has two operations:
Aggregate
Create a materalized aggregate of a single table.
See Aggregate.
Join
Create a materialized join of several tables.
See Join.
Performance
Materialized views exchange slower write performance for higher read performance.
While it's impossible to escape the fundamental trade-off, Denorm is as fast or faster than hand-written triggers. It uses statement-level transitions tables to make batch updates especially efficient.
In deferred mode, Denorm uses temp tables to defer updates until the end of the
transaction. Using temp tables and ON DELETE COMMIT reduces I/O overhead and
obviates the need for vacuuming. Since PostgreSQL does not support global
temporary tables, the tables are created as necessary for each session. Thus the
first saliant update in a session may have several millseconds of overhead as
the trigger creates the temporary tables. Pool connections to reduce overhead,
and vacuum reguarly to prevent system tables from bloating.
Migration
Denorm does not generate migration scripts.
Consider a tool like migra to help generate migration scripts.
Limitations
Denorm mangles names for generated objects, Long IDs and table names may run into the PostgreSQL limit of 63 characters for identifiers.
Developing
Install: make install
Generate JSONSchema: make schema
Test: make test
Generate documentation: make doc
Format: make format
Publish
- Update denorm/version.py.
- Create commit
Version <version>. - Tag
v<version>. - Push master branch and tag.
- Publish to PyPI:
make publish.
3 maintainers
Metadata
3 maintainers
Release files for denorm 3.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| denorm-3.0.0.tar.gz | 28.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| denorm-3.0.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 60.0 kB
Release files / denorm-3.0.0.tar.gz
| Download URL | denorm-3.0.0.tar.gz |
|---|---|
| Size | 28.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
582d81a67b2c35d17b493bcdbded4fe394e3c2c3166fae4cadad9260259ff8b1
|
|
BLAKE2b-256 checksum How to use checksums |
d21743b2abccf0ed2f13f1f1112e698142ae38be96e595ecd63c74a7bdee8ac8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.14.4
|
Release files / denorm-3.0.0-py3-none-any.whl
| Download URL | denorm-3.0.0-py3-none-any.whl |
|---|---|
| Size | 31.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1718ccf8620aea81bd86a47defc18b3bab8ba4035104ad038286f81f79f9f26f
|
|
BLAKE2b-256 checksum How to use checksums |
9e1c68c8d0a0f35d55a9eaafcac515f58ff1739166a01d95354ffad1b1f5fdfa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.14.4
|
