Skip to main content

Split Folders Build Status PyPI PyPI - Python Version

Split folders with files (e.g. images) into train, validation and test (dataset) folders.

The input folder shoud have the following format:

input/
    class1/
        img1.jpg
        img2.jpg
        ...
    class2/
        imgWhatever.jpg
        ...
    ...

In order to give you this:

output/
    train/
        class1/
            img1.jpg
            ...
        class2/
            imga.jpg
            ...
    val/
        class1/
            img2.jpg
            ...
        class2/
            imgb.jpg
            ...
    test/
        class1/
            img3.jpg
            ...
        class2/
            imgc.jpg
            ...

This should get you started to do some serious deep learning on your data. Read here why it's a good idea to split your data intro three different sets.

  • You may only split into a training and validation set.
  • The data gets split before it gets shuffled.
  • A seed lets you reproduce the splits.
  • Works on any file types.
  • Allows randomized oversampling for imbalanced datasets.
  • (Should) work on all operating systems.

Install

pip install split-folders

If you are working with a large amount of files, you may want to get a progress bar. Install tqdm in order to get updates when copying the files into the new folders.

pip install split-folders tqdm

Usage

You you can use split_folders as Python module or as a Command Line Interface (CLI).

If your datasets is balanced (each class has the same number of samples), choose ratio otherwise fixed. NB: oversampling is turned off by default.

Module

import split_folders

# Split with a ratio.
# To only split into training and validation set, set a tuple to `ratio`, i.e, `(.8, .2)`.
split_folders.ratio('input_folder', output="output", seed=1337, ratio=(.8, .1, .1)) # default values

# Split val/test with a fixed number of items e.g. 100 for each set.
# To only split into training and validation set, use a single number to `fixed`, i.e., `10`.
split_folders.fixed('input_folder', output="output", seed=1337, fixed=(100, 100), oversample=False) # default values

CLI

Usage:
    split_folders folder_with_images [--output] [--ratio] [--fixed] [--seed] [--oversample]
Options:
    --output     path to the output folder. defaults to `output`. Get created if non-existent.
    --ratio      the ratio to split. e.g. for train/val/test `.8 .1 .1` or for train/val `.8 .2`.
    --fixed      set the absolute number of items per validation/test set. The remaining items constitute
                 the training set. e.g. for train/val/test `100 100` or for train/val `100`.
    --seed       set seed value for shuffling the items. defaults to 1337.
    --oversample enable oversampling of imbalanced datasets, works only with --fixed.
Example:
    split_folders imgs --ratio .8 .1 .1

License

MIT.

Metadata

Release files for split-folders 0.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for split-folders 0.3.1
File Size Uploaded Provenance Actions
split_folders-0.3.1.tar.gz 4.7 kB Without provenance Details for split_folders-0.3.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for split-folders 0.3.1
File Interpreter ABI Platform Actions
split_folders-0.3.1-py3-none-any.whl Python 3 none any Details for split_folders-0.3.1-py3-none-any.whl

Total release size: 11.0 kB

Release files / split_folders-0.3.1.tar.gz

Download URL split_folders-0.3.1.tar.gz
Size 4.7 kB
Tags Source
SHA-256 checksum
How to use checksums
98f32fbff02702529db3c11e5f7c049fb030a7911876b653a40796a2ae3401b6
BLAKE2b-256 checksum
How to use checksums
4335196590b7054028e68d6796884f3157713e092d66e6e74cd4afdeb4b898ea
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.3

Release files / split_folders-0.3.1-py3-none-any.whl

Download URL split_folders-0.3.1-py3-none-any.whl
Size 6.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
0252f36a93af05cb93080e4236aa602ff59af4e1ab62932a7545ac5ab5097827
BLAKE2b-256 checksum
How to use checksums
206729dda743e6d23ac1ea3d16704d8bbb48d65faf3f1b1eaf53153b3da56c56
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.3

Security for split-folders 0.3.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page