Batching is a set of tools to format data for training sequence models
Project description
Batching
Batching is a set of tools to format data for training sequence models.
Installation
$ pip install batching
Example usage
Example script exists in sample.py
# Metadata for batch info - including batch IDs and mappings to storage resouces like filenames
storage_meta = StorageMeta(validation_split=0.2)
# Storage for batch data - Memory, Files, S3
storage = BatchStorageMemory(storage_meta)
# Create batches - configuration contains feature names, windowing config, timeseries spacing
batch_generator = Builder(storage,
feature_set,
look_back,
look_forward,
batch_seconds,
batch_size=128)
batch_generator.generate_and_save_batches(list_of_dataframes)
# Generator for feeding batches to training - tf.keras.model.fit_generator
train_generator = BatchGenerator(storage)
validation_generator = BatchGenerator(storage, is_validation=True)
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(1, activation='sigmoid')
model.compile(loss=tf.keras.losses.binary_crossentropy,
optimizer=tf.keras.optimizers.Adam(),
metrics=['accuracy'])
model.fit_generator(train_generator,
validation_data=validation_generator,
epochs=epochs)
License
- MIT license
- Copyright 2015 © FVCproductions.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
batching-1.1.0.tar.gz
(8.7 kB
view hashes)