Skip to main content

build-status coverage-status pypi-reference pypi-downloads

Melon

Melon is a lightweight package meant to simplify data processing for Deep Learning.
It removes the need for boilerplate code to pre-process the data prior to (model) training, testing and inference.
It aims at standardizing data serialization and manipulation approaches.

The default formats align with the requirements by frameworks such as Tensorflow / PyTorch / Keras.
The tool also provides various level of customizations depending on the use-case.

Installation

Install and update using pip:

$ pip install melon

Supported in Python >= 3.4.0

Examples

Images

With default options:

from melon import ImageReader

def train():
    source_dir = "resources/images"
    reader = ImageReader(source_dir)
    X, Y = reader.read()
    ...
    with tf.Session() as s:
        s.run(..., feed_dict = {X_placeholder: X, Y_placeholder: Y})
source_dir directory should contain images that need to be read. See sample directory for reference.
In the sample directory there is an optional labels.txt file that is described in Labeling.

Since number of images may be too large to fit into memory the tool supports batch-processing.

from melon import ImageReader

def train():
    source_dir = "resources/images"
    options = { "batch_size": 32 }
    reader = ImageReader(source_dir, options)
    while reader.has_next():
        X, Y = reader.read()
        ...
This reads images in the batches of 32 until all images are read. If batch_size is not specified then reader.read() will read all images.

With custom options:

from melon import ImageReader

def train():
    source_dir = "resources/images"
    options = { "data_format": "channels_last", "normalize": False }
    reader = ImageReader(source_dir, options)
    ...
This changes format of data to channels-last (each sample will be Height x Width x Channel) and doesn’t normalize the data. See options for available options.

Options

Images

width

Width of the output (pixels). default: 255

height

Height of the output (pixels). default: 255

batch_size

Batch size of each read. default: All images in a directory

data_format

Format of the images data

channels_first - Channel x Height x Width (default)
channels_last - Height x Width x Channel
label_format

Format of the labels data

one_hot - as a matrix, with one-hot vector per image (default)
label - as a vector, with a single label per image
normalize

Normalize data. default: True

num_threads - number of threads for parallel processing

default: Number of cores of the machine

Labeling

In supervised learning each image needs to be mapped to a label.
While the tool supports reading images without labels (e.g. for inference) it also provides a way to label them.

Generating labels file

To generate labels file use the following command:
$ melon generate
> Source dir:
After providing source directory the tool will generate labels file in that directory with blank labels.
Final step is to add a label to each row in the generated file.

For reference see sample labels:
#legend
pedestrian:0
cat:1
parrot:2
car:3
apple tree:4

#map
img275.jpg:1
img324.jpg:2
img551.jpg:3
img928.jpg:1
img999.png:0
img736.png:4
#legend section is optional but #map section is required to map a label to an image.

Format of the labels

Label’s output format can be specified in Custom options. It defaults to one-hot format.

Roadmap

  • Support for video data (Q1 2019)

  • Support for reading from AWS S3 (Q2 2019)

Metadata

Release files for melon 0.1.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for melon 0.1.2.1
File Size Uploaded
melon-0.1.2.1.tar.gz 8.5 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for melon 0.1.2.1
File Interpreter ABI Platform
melon-0.1.2.1-py3-none-any.whl Python 3 none any Details

Total release size: 25.0 kB

Release files / melon-0.1.2.1.tar.gz

Download URL melon-0.1.2.1.tar.gz
Size 8.5 kB
Tags Source
SHA-256 checksum
How to use checksums
061acb12ad3cbf629c720f667d5d7ab66690dd27e0bc46b33a2003e9bc0a4eee
BLAKE2b-256 checksum
How to use checksums
3e7654c0eb0679926caa5227c71f16465b554b6e3e6318b70f094383400ea100
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.29.0 CPython/3.7.2

Release files / melon-0.1.2.1-py3-none-any.whl

Download URL melon-0.1.2.1-py3-none-any.whl
Size 16.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
faab1df00c249a6f8a4f2180af041f808a506bb2015a0cbefd867494b90958eb
BLAKE2b-256 checksum
How to use checksums
e4bb5b777e4aacb73c0304a08253328b6d3117ad9697b0438010137c3564353a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.29.0 CPython/3.7.2

Release history Release notifications | RSS feed

This release

0.1.2.1 This release

2 release files

0.1.2

2 release files

0.1.1

2 release files

0.1.0

2 release files

0.0.6

2 release files

0.0.5

2 release files

0.0.4

2 release files

0.0.3

2 release files

0.0.2

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page