Skip to main content

Batching

Batching is a set of tools to format data for training sequence models.

Build Status Coverage Status

Installation

$ pip install batching

Example usage

Example script exists in sample.py

# Metadata for batch info - including batch IDs and mappings to storage resouces like filenames
storage_meta = StorageMeta(validation_split=0.2)

# Storage for batch data - Memory, Files, S3
storage = BatchStorageMemory(storage_meta)

# Create batches - configuration contains feature names, windowing config, timeseries spacing
batch_generator = Builder(storage, 
                          feature_set, 
                          look_back, 
                          look_forward, 
                          batch_seconds, 
                          batch_size=128)
batch_generator.generate_and_save_batches(list_of_dataframes)

# Generator for feeding batches to training - tf.keras.model.fit_generator
train_generator = BatchGenerator(storage)
validation_generator = BatchGenerator(storage, is_validation=True)

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(1, activation='sigmoid')
model.compile(loss=tf.keras.losses.binary_crossentropy, 
              optimizer=tf.keras.optimizers.Adam(), 
              metrics=['accuracy'])
model.fit_generator(train_generator,
                    validation_data=validation_generator,
                    epochs=epochs)

License

License

Metadata

Release files for batching 1.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for batching 1.2.0
File Size Uploaded Provenance Actions
batching-1.2.0.tar.gz 9.9 kB Without provenance Details for batching-1.2.0.tar.gz

Release files / batching-1.2.0.tar.gz

Download URL batching-1.2.0.tar.gz
Size 9.9 kB
Tags Source
SHA-256 checksum
How to use checksums
8ef11879c4edf320db72a1dd70203f23f28bd399a1eafaf76adc4779f80cda05
BLAKE2b-256 checksum
How to use checksums
45294525e96a0d128488e2572464839a85c8a124b697f6e64d089d625a6d07d5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.8.12

Security for batching 1.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page