Easily turn a set of image urls to an image dataset
Project description
img2dataset
Easily turn a set of image urls to an image dataset
Install
pip install img2dataset
Usage
First get some image url list. For example:
echo 'https://placekitten.com/200/305' >> myimglist.txt
echo 'https://placekitten.com/200/304' >> myimglist.txt
echo 'https://placekitten.com/200/303' >> myimglist.txt
Then, run the tool:
img2dataset --url_list=myimglist.txt --output_folder=output_folder --thread_count=64 --image_size=256
The tool will then automatically download the urls, resize them, and store them with that format:
- output_folder
- 0
- 0.jpg
- 1.jpg
- 2.jpg
- 0
with each number being the position in the list. The subfolders avoids having too many files in a single folder.
This can then easily be fed into machine learning training or any other use case.
Road map
This tool work as it. However in the future goals will include:
- WebDataset format option
- support for multiple input files
- support for csv or parquet files as input
- more resizing options (currently resizing with borders, other options are often useful)
For development
Either locally, or in gitpod (do export PIP_USER=false
there)
Setup a virtualenv:
python3 -m venv .env
source .env/bin/activate
pip install -e .
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
img2dataset-1.0.1.tar.gz
(4.2 kB
view hashes)
Built Distribution
Close
Hashes for img2dataset-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d9d815c0a6c135257b925b72faba071d600adfc62706a73afa777356bd747e1 |
|
MD5 | 4d16d6b3c02ddc2fbc154ffd46af7f33 |
|
BLAKE2b-256 | 19f6253a8bf4cb9c45b15df39a301b503d6de9a54fb28e9d20b1bfd19458ca61 |