nested-pandas
An extension of pandas for efficient representation of nested associated datasets.
Nested-Pandas extends the pandas package with tooling and support for nested dataframes packed into values of top-level dataframe columns. Pyarrow is used internally to aid in scalability and performance.
Nested-Pandas allows data like this:
To instead be represented like this:
Where the nested data is represented as nested dataframes:
# Each row of "object_nf" now has it's own sub-dataframe of matched rows from "source_df"
object_nf.loc[0]["nested_sources"]
Allowing powerful and straightforward operations, like:
# Compute the mean flux for each row of "object_nf"
import numpy as np
def mean_flux(row):
"""Calculates the mean flux for each object"""
return np.mean(row["nested_sources.flux"])
object_nf.map_rows(mean_flux, output_names="mean_flux")
Nested-Pandas is motivated by time-domain astronomy use cases, where we see
typically two levels of information, information about astronomical objects and
then an associated set of N measurements of those objects. Nested-Pandas offers
a performant and memory-efficient package for working with these types of datasets.
Core advantages being:
- hierarchical column access
- efficient packing of nested information into inputs to custom user functions
- avoiding costly groupby operations
This is a LINCC Frameworks project - find more information about LINCC Frameworks here.
Acknowledgements
This project is supported by Schmidt Sciences.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file nested_pandas-0.7.0.tar.gz.
File metadata
- Download URL: nested_pandas-0.7.0.tar.gz
- Upload date:
- Size: 1.2 MB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
4383441bc757bf53b6db6d7d576e3494565ccada6493b6bee6b237ded9ccb770
|
|
| MD5 |
e480463876f12196c42475a10781bd02
|
|
| BLAKE2b-256 |
9d03d00a1d067024364d7493dedc30cf82c8efda18f1a4340f2645e40905cb17
|
Provenance
The following attestation bundles were made for nested_pandas-0.7.0.tar.gz:
Publisher:
publish-to-pypi.yml on lincc-frameworks/nested-pandas
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
nested_pandas-0.7.0.tar.gz -
Subject digest:
4383441bc757bf53b6db6d7d576e3494565ccada6493b6bee6b237ded9ccb770 - Sigstore transparency entry: 2129507408
- Sigstore integration time:
-
Permalink:
lincc-frameworks/nested-pandas@aac6ed3c4490f6133d8f365d57f5109029a60971 -
Branch / Tag:
refs/tags/v0.7.0 - Owner: https://github.com/lincc-frameworks
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish-to-pypi.yml@aac6ed3c4490f6133d8f365d57f5109029a60971 -
Trigger Event:
release
-
Statement type:
File details
Details for the file nested_pandas-0.7.0-py3-none-any.whl.
File metadata
- Download URL: nested_pandas-0.7.0-py3-none-any.whl
- Upload date:
- Size: 81.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
910f6fe425e879e68c529654608cda6bbd19c66873df3f9e376a12f4f74bfd46
|
|
| MD5 |
bfa5be4852aa54289c7fc862977722c4
|
|
| BLAKE2b-256 |
a1b61e2ab0cb518d64038fca6b30553219f4b113e910286eabe22aa28c4a1d8a
|
Provenance
The following attestation bundles were made for nested_pandas-0.7.0-py3-none-any.whl:
Publisher:
publish-to-pypi.yml on lincc-frameworks/nested-pandas
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
nested_pandas-0.7.0-py3-none-any.whl -
Subject digest:
910f6fe425e879e68c529654608cda6bbd19c66873df3f9e376a12f4f74bfd46 - Sigstore transparency entry: 2129507744
- Sigstore integration time:
-
Permalink:
lincc-frameworks/nested-pandas@aac6ed3c4490f6133d8f365d57f5109029a60971 -
Branch / Tag:
refs/tags/v0.7.0 - Owner: https://github.com/lincc-frameworks
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish-to-pypi.yml@aac6ed3c4490f6133d8f365d57f5109029a60971 -
Trigger Event:
release
-
Statement type:
