This release is a pre-release and may not be stable for production use.
science_catalogs
science_catalogs is a reusable Python library for building science-ready
catalogs with LSDB-oriented workflows. The package focuses on the reusable core
of the processing stack:
- column selection
- column transformations
- row filtering
- output materialization to memory, partitioned files, or HATS catalogs
The package is published on PyPI as science-catalogs and imported in Python as
science_catalogs.
Installation
pip install science-catalogs
For local development:
pip install -e '.[dev]'
Or, if you prefer a requirements file for a full developer environment including build and PyPI publication tools:
pip install -r requirements-dev.txt
Main API
from science_catalogs import (
build_catalog,
materialize_catalog,
materialize_lsdb_catalog,
open_lsdb_catalog,
prepare_catalog,
write_catalog,
)
Beta API
The beta public API is:
prepare_catalogmaterialize_catalogwrite_catalogmaterialize_lsdb_catalogopen_lsdb_catalogbuild_catalog
Legacy names based on pipeline are not part of the beta API.
Usage
Prepare a catalog from a catalog-processing YAML configuration:
from science_catalogs import prepare_catalog
prepared = prepare_catalog("configs/catalog.yml")
Materialize the processed data in memory and keep track of the written output paths:
from science_catalogs import materialize_catalog
result = materialize_catalog(prepared, "./output")
frame = result["data"]
paths = result["path"]
Write the result to disk. The write mode follows the output configuration,
including HATS when output.save_as: hats is selected:
from science_catalogs import write_catalog
written_paths = write_catalog(prepared, "./output")
Open the final result as an LSDB catalog after writing HATS output:
from science_catalogs import materialize_lsdb_catalog
result = materialize_lsdb_catalog(prepared, "./output")
catalog = result["data"]
hats_path = result["path"]
Execute the full flow from configuration and persist parquet output in one call:
from science_catalogs import build_catalog
paths = build_catalog("configs/catalog.yml", output_dir="./output")
Or force a HATS artifact from the same flow:
hats_path = build_catalog(
"configs/catalog.yml",
output_dir="./output",
output_format="hats",
)
If you already have a HATS catalog on disk, you can open it directly:
from science_catalogs import open_lsdb_catalog
catalog = open_lsdb_catalog("./output/my_catalog")
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file science_catalogs-0.1.0b6.tar.gz.
File metadata
- Download URL: science_catalogs-0.1.0b6.tar.gz
- Upload date:
- Size: 275.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c3fa956245040c77be322daa414556235c1432b60f03e69e29029ed7c1456b2a
|
|
| MD5 |
d721dbef1c9ac77917a0a15e68732356
|
|
| BLAKE2b-256 |
a94037daf671ad0a0946d32b0c938e2ab99b3100dc401ff545acebb204945afb
|
Provenance
The following attestation bundles were made for science_catalogs-0.1.0b6.tar.gz:
Publisher:
publish-to-pypi.yml on linea-it/science_catalogs
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
science_catalogs-0.1.0b6.tar.gz -
Subject digest:
c3fa956245040c77be322daa414556235c1432b60f03e69e29029ed7c1456b2a - Sigstore transparency entry: 2206748433
- Sigstore integration time:
-
Permalink:
linea-it/science_catalogs@57e8f449b32325e9e057978e6a50140b3fb2fd29 -
Branch / Tag:
refs/tags/v0.1.0b6 - Owner: https://github.com/linea-it
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish-to-pypi.yml@57e8f449b32325e9e057978e6a50140b3fb2fd29 -
Trigger Event:
release
-
Statement type:
File details
Details for the file science_catalogs-0.1.0b6-py3-none-any.whl.
File metadata
- Download URL: science_catalogs-0.1.0b6-py3-none-any.whl
- Upload date:
- Size: 20.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e1ce0cf26fc5d429124d1cdeabdff43acaf35e4813f37cdd79e67ba20fcfeca1
|
|
| MD5 |
2d967109846a4fab0cbac06e6a35c86d
|
|
| BLAKE2b-256 |
3ad5b5d823e95d9c9ee62f0cb76294a0a6899e67f9c5f3b8e4f7c1c7c7fd34a2
|
Provenance
The following attestation bundles were made for science_catalogs-0.1.0b6-py3-none-any.whl:
Publisher:
publish-to-pypi.yml on linea-it/science_catalogs
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
science_catalogs-0.1.0b6-py3-none-any.whl -
Subject digest:
e1ce0cf26fc5d429124d1cdeabdff43acaf35e4813f37cdd79e67ba20fcfeca1 - Sigstore transparency entry: 2206748470
- Sigstore integration time:
-
Permalink:
linea-it/science_catalogs@57e8f449b32325e9e057978e6a50140b3fb2fd29 -
Branch / Tag:
refs/tags/v0.1.0b6 - Owner: https://github.com/linea-it
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish-to-pypi.yml@57e8f449b32325e9e057978e6a50140b3fb2fd29 -
Trigger Event:
release
-
Statement type:
