lsst-daf-butler
LSST Data Access framework described in arXiv:2206.14941. Please cite the SPIE paper when using this software.
This is a Python 3 only package (we assume Python 3.11 or higher).
- SPIE paper from 2024: Converting Rubin Observatory's data butler to a client/server architecture (Tech note)
- SPIE Paper from 2022: The Vera C. Rubin Observatory Data Butler and Pipeline Execution System (arXiv) [Primary reference]
- ADASS paper from 2019: Abstracting the Storage and Retrieval of Image Data at the LSST.
- Early design note: DMTN-056
PyPI: lsst-daf-butler
This software is dual licensed under the GNU General Public License (version 3 of the License, or (at your option) any later version, and also under a 3-clause BSD license). Recipients may choose which of these licenses to use; please see the files gpl-3.0.txt and/or bsd_license.txt, respectively.
Arrow Memory Leaks
From version 18 of arrow we have seen significant memory leaks when accessing parquet files using the default memory allocator.
If you see such leaks the workaround is to set the ARROW_DEFAULT_MEMORY_POOL environment variable to jemalloc following the advice from apache/arrow#45882.
For EUPS users this variable is automatically set.
Managing dependencies
This package uses uv to manage the versions of libraries used for CI, Docker builds, and local development.
Upgrading the versions of libraries
Run the following command to refresh uv.lock with the latest available
versions of all libraries:
uv lock --upgrade
Temporarily pinning another LSST package to a development branch
If you are working on a PR that requires unreleased changes from another LSST
repository, you can temporarily pin the new version by opening pyproject.toml
and finding the library in the override-dependencies declaration under
tool.uv. Replace main with the ticket branch, e.g.:
override-dependencies = [
"lsst-utils @ git+https://github.com/lsst/utils@tickets/DM-xxxxx",
...
]
Then run:
uv lock --upgrade-package <package name>
to pull in the updated version for testing, where <package name> is
lsst-utils or whatever package you are overriding.
(You can also just do uv lock --upgrade if you don't mind upgrading all the
other libraries at the same time.)
You will need to re-do this lock update any time you change the code, so that it resolves to the new version.
After the other library's development branch has been merged, reset the
branch to main in pyproject.toml and run the lock update again to pull in
the released version.
1 maintainer
Metadata
1 maintainer
Release files for lsst-daf-butler 30.2026.4100
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| lsst_daf_butler-30.2026.4100.tar.gz | 1.3 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| lsst_daf_butler-30.2026.4100-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 2.8 MB
Release files / lsst_daf_butler-30.2026.4100.tar.gz
| Download URL | lsst_daf_butler-30.2026.4100.tar.gz |
|---|---|
| Size | 1.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bb23dd9a26d011d44b4016c1bbf00716ea37d343bc0fa0f2f270f660b0ad132d
|
|
BLAKE2b-256 checksum How to use checksums |
fbea6fa3c2d561ab29c2da045d94aa958499df63d5c9acd528eddd1d3e4cbedc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / lsst_daf_butler-30.2026.4100-py3-none-any.whl
| Download URL | lsst_daf_butler-30.2026.4100-py3-none-any.whl |
|---|---|
| Size | 1.4 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5c5c1cf8e2a00d10839c91ea75cac25f83daa1a77498767d608db9bffec03cf7
|
|
BLAKE2b-256 checksum How to use checksums |
1abd6b0f5815fb521e805aac67d5fd603df20a0e9b5a7579b147c90bdcc92ba7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
