Benchling uses SQLAlchemy and psycopg2 to talk to PostgreSQL. To save on round-trip latency, we batch our inserts using this code.
Typically, creating and flushing N models in SQLAlchemy does N roundtrips to the database if the model has an autoincrementing primary key. This module improves creating N models to only require 2 roundtrips, without requiring any other changes to your code.
Is this for me?
You may find use for this module if:
- You are using SQLAlchemy
- You are using Postgres
- You sometimes need to create several models at once and care about performance
Usage
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy_batch_inserts import enable_batch_inserting
engine = create_engine("postgresql+psycopg2://postgres@localhost", executemany_mode="values") # SQLAlchemy < 1.3.7 needs use_batch_mode=True instead
Session = sessionmaker(bind=engine)
session = Session()
enable_batch_inserting(session)
If you use Flask-SQLALchemy,
from flask_sqlalchemy import SignallingSession
from sqlalchemy_batch_inserts import enable_batch_inserting
# Make sure that you've specified executemany_mode or use_batch_mode when creating your engine! Otherwise
# this library will not have any effect.
enable_batch_inserting(SignallingSession)
Acknowledgements
This is all possible thanks to @dvarrazzo's psycopg2 execute_batch and @zzzeek's SQLAlchemy support for the same and helpful advice on the mailing list.
Metadata
Release files for sqlalchemy-batch-inserts 0.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| sqlalchemy_batch_inserts-0.0.4.tar.gz | 6.0 kB | Without provenance | Details for sqlalchemy_batch_inserts-0.0.4.tar.gz |
Release files / sqlalchemy_batch_inserts-0.0.4.tar.gz
| Download URL | sqlalchemy_batch_inserts-0.0.4.tar.gz |
|---|---|
| Size | 6.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c4a9b3b5b195caec4390922310510e74a6605efd7a11b351806f74169101d616
|
|
BLAKE2b-256 checksum How to use checksums |
8c38b4b630fd00686b1a39222608662178c53ef7b34d9fd3bb914114c7ff1c0b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/50.3.1 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.8.6
|
Security for sqlalchemy-batch-inserts 0.0.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.