Cell painting gallery data handling and validation
Project description
Cell painting gallery data handling and validation
Getting started
Install cpgdata
package
pip install cpgdata
Sync pre-generated index files
cpg index sync -o "path to save index files"
Example of using the index for filtering files to download from the Cell painting gallery
from pathlib import Path
from pprint import pprint
import polars as pl
from cpgdata.utils import download_s3_files, parallel
index_dir = Path("path to dir containing index files")
index_files = [file for file in index_dir.glob("*.parquet")]
df = pl.scan_parquet(index_files)
df = (
df
.filter(pl.col("dataset_id").eq("cpg0016-jump"))
.filter(pl.col("source_id").eq("source_4"))
.filter(pl.col("leaf_node").str.contains("Cells.csv"))
.select(pl.col("key"))
.collect()
)
# print first 10 results
pprint(df.to_dicts()[0:10])
# Download filtered files
download_keys = list(df.to_dict()["key"])
parallel(download_keys, download_s3_files, ["cellpainting-gallery", Path("path to save downloaded files")], jobs=20)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
cpgdata-0.4.0.tar.gz
(11.4 kB
view hashes)
Built Distribution
cpgdata-0.4.0-py3-none-any.whl
(15.1 kB
view hashes)