keras-mixed-sequence

Lazily loading mixed sequences using Keras Sequence, focused on multi-task models.

How do I install this package?

As usual, just download it using pip:

pip install keras_mixed_sequence

Usage examples

Example for traditional single-task models

First of all let’s create a simple single-task model:

from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Sequential

model = Sequential([
    Dense(1, activation="relu")
])
model.compile(
    optimizer="nadam",
    loss="relu"
)

Then we proceed to load or otherwise create the training data. Here there will be listed, in the future, some custom Sequence objects that have been created for the purpose of being used alongside this library.

X = either_a_numpy_array_or_sequence_for_input
y = either_a_numpy_array_or_sequence_for_output

Now we combine the training data using the MixedSequence object.

from keras_mixed_sequence import MixedSequence

sequence = MixedSequence(
    X, y,
    batch_size=batch_size
)

Finally, we can train the model:

from multiprocessing import cpu_count

model.fit_generator(
    sequence,
    steps_per_epoch=sequence.steps_per_epoch,
    epochs=2,
    verbose=0,
    use_multiprocessing=True,
    workers=cpu_count(),
    shuffle=True
)

Example for multi-task models

First of all let’s create a simple multi-taks model:

from tensorflow.keras.models import Model
from tensorflow.keras.layers import Dense, Input

inputs = Input(shape=(10,))

output1 = Dense(
    units=10,
    activation="relu",
    name="output1"
)(inputs)
output2 = Dense(
    units=10,
    activation="relu",
    name="output2"
)(inputs)

model = Model(
    inputs=inputs,
    outputs=[output1, output2],
    name="my_model"
)

model.compile(
    optimizer="nadam",
    loss="MSE"
)

X = either_a_numpy_array_or_sequence_for_input
y1 = either_a_numpy_array_or_sequence_for_output1
y2 = either_a_numpy_array_or_sequence_for_output2

Now we combine the training data using the MixedSequence object.

from keras_mixed_sequence import MixedSequence

sequence = MixedSequence(
    x=X,
    y={
        "output1": y1,
        "output2": y2
    },
    batch_size=batch_size
)

Finally, we can train the model:

from multiprocessing import cpu_count

model.fit_generator(
    sequence,
    steps_per_epoch=sequence.steps_per_epoch,
    epochs=2,
    verbose=0,
    use_multiprocessing=True,
    workers=cpu_count(),
    shuffle=True
)

Metadata

Release files for keras-mixed-sequence 1.0.29

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for keras-mixed-sequence 1.0.29
File	Size	Uploaded
keras_mixed_sequence-1.0.29.tar.gz	7.1 kB	Feb 3, 2024	Details

Release files / keras_mixed_sequence-1.0.29.tar.gz

Download URL	keras_mixed_sequence-1.0.29.tar.gz
Size	7.1 kB
Tags	Source
SHA-256 checksum How to use checksums	`4d49f4325988dccd9d1f6d1ea53e1fc0be8a4479545387f79db7d9245ed66e91`
BLAKE2b-256 checksum How to use checksums	`26970f66d9fa1579eaded44a0c572c8e2e9d2daa89f5d911d45f0307e8d2be73`
Upload date	Feb 3, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.8.0 pkginfo/1.8.2 readme-renderer/34.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.63.1 importlib-metadata/4.11.3 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.7.9`