Skip to main content

A helper package to download example datasets used in various publications and deep-learning algorithms, including data featured in N2V, P(P)N2V, DivNoising, HDN, EmbedSeg, etc.

Project description

CAREamics Portfolio

License PyPI Python Version CI Datasets CI codecov

A helper package based on pooch allowing downloading various example datasets used in publications by the Jug lab, including data featured in N2V, P(P)N2V, DivNoising, HDN, EmbedSeg, etc.

The complete list of datasets can be found in datasets.json.

CAREamics-portfolio tooling was generated using pydev-guide/pyrepo-copier.


To install the portfolio in your conda environment, simply use pip:

$ pip install careamics-portfolio


Follow the example notebook for details on how to use the package.

The portfolio can be instantiated as follow:

from careamics_portfolio import PortfolioManager

portfolio = PortfolioManager()

You can explore the different datasets easily:


Finally, you can download the dataset of your choice:

from pathlib import Path

data_path = Path('data')

# to the path of your choice

# or to your system's cache

By default, if you do not pass path to the download() method, all datasets will be saved in your system's cache. New queries to download will not cause the files to be downloaded again (thanks pooch!!).

Important: if you download all datasets of interest using the same path, pooch will maintain a regsitry of files and you will not have to download them again!

Add a dataset to the portfolio

There are a few steps to follow in order to add a new dataset to the repository:

:white_check_mark: 1 - Create a PortfolioEntry child class

:white_check_mark: 2 - Instantiate the portfolio entry in an IterablePortfolio

:white_check_mark: 3 - Update registry.txt

:white_check_mark: 4 - Make sure all tests pass

Note: To run the tests, you will need to have pytest installed. You can create an environment with careamics-portfolio and pytest by running:

pip install "careamics-portfolio[test]"

1 - Create a portfolio entry

To add a dataset, subclass a PortfolioEntry and enter the following information (preferably in one of the current categories, e.g.

class MyDataset(PortfolioEntry):
    def __init__(self) -> None:
            portfolio="Denoising", # for instance
            hash="953a815333805a423b7342971289h10121263917019bd16cc3341", # sha256
            description="Description of the dataset.",
            license="CC-BY 3.0",
            citation="Citation of the dataset",
                "/folder/in/the/zip": ["file1.tif", "file2.tif"], # folder can be "."
            size=13.0, # size in MB
            tags=["tag1", "tag2"],

To obtain sha256 hash of your file, you can run the following code and read out the sha256 from the pooch prompt:

import pooch

url = ""
pooch.retrieve(url, known_hash=None)

Likewise, to get the size in MB of your file:

import os

os.path.getsize(file_path) / 1024 / 1024

2 - Add the entry to a portfolio

Add the file class to one of the categories (e.g. denoising) in

class Denoising(IterablePortfolio):
    def __init__(self) -> None:
        self._N2V_BSD68 = N2V_BSD68()
        self._N2V_SEM = N2V_SEM()
        self._N2V_RGB = N2V_RGB()
        self._flywing = Flywing()

        # add your dataset as a private attribute
        self._myDataset = MyDataset()


    # and add a public getter
    def MyDataset(self) -> MyDataset:
        return self._myDataset

3 - Update registry

Finally, update the registry by running the following pythons script:

python scripts/

or run:

from careamics_portfolio import update_registry

The datasets.json file is updated using:

python scripts/

4 - Verify that all tests pass

Verify that all tests pass, it can take a while:


Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

careamics_portfolio-0.0.12.tar.gz (25.8 kB view hashes)

Uploaded Source

Built Distribution

careamics_portfolio-0.0.12-py3-none-any.whl (19.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page