Skip to main content

Tensorflow Keras utilities for reducing boilerplate code.

Project description

Build Status Coverage Status Version Status Python Versions Downloads

params-flow provides an alternative style for defining your Keras model or layer configuration in order to reduce the boilerplate code related to passing and (de)serializing your model/layer configuration arguments.

params-flow encourages this:

import params_flow

class MyDenseLayer(params_flow.Layer):      # using params_flow Layer/Model instead of Keras ones
  class Params(params_flow.Layer.Params):   # extend one or more base Params configurations
    num_outputs = None                      # declare all configuration arguments
    activation = "gelu"                     #   provide or override super() defaults
                                            # do not define an __init__()

  def build(self, in_shape):
    self.kernel = self.add_variable("kernel",
                                    [int(in_shape[-1]),
                                     self.params.num_outputs])     # access config arguments

which would be sufficient to pass the right configuration arguments to the super layer/model, as well as take care of (de)serialization, so you can concentrate on the build() or call() implementations, instead of writing boilerplate code like this:

from tf.keras.layers import Layer

class MyDenseLayer(Layer):
  def __init__(self,
               num_outputs,            # put all of the layer configuration in the constructor
               activation = "gelu",    #     provide defaults
               **kwargs):              # allow base layer configuration to be passed to super
    self.num_outputs = num_outputs
    self.activation = activation
    super().__init__(**kwargs)

  def build(self, in_shape):
    self.kernel = self.add_variable("kernel",
                                    [int(in_shape[-1]),
                                     self.num_outputs])      # access config arguments

  def get_config(self):                # serialize layer configuration, __init__() is the deserializer
    config = {
      'num_outputs': self.num_outputs,
      'activation': self.activation
    }
    base_config = super().get_config()
    return dict(list(base_config.items())) + list(config.items())

LICENSE

MIT. See License File.

Install

params-flow is on the Python Package Index (PyPI):

pip install params-flow

Usage

params-flow provides a Layer and Model base classes that help reducing common boilerplate code in your custom Keras layers and models.

When subclassing a Keras Model or Layer, each configuration parameter has to be provided as an argument in __init__(). Keras relies on both __init__() and get_config() to make a model/layer serializable.

While python idiomatic this style of defining your Keras models/layers results in a lot of boilerplate code. params-flow provides an alternative by encapsulating all those __init__() configuration arguments in a dedicated Params instance (Params is kind of a “type-safe” python dict - see kpe/py-params). The model/layer specific configuration needs to be declared as a nested Model.Params/Layer.Params subclass, and your model/layer have to subclass params_flow.Model/params_flow.Layer instead of the Keras ones:

class BertEmbeddingsLayer(Layer):
  class Params(PositionEmbeddingLayer.Params):
    vocab_size              = None
    token_type_vocab_size   = 2
    hidden_size             = 768
    use_position_embeddings = True

class TransformerEncoderLayer(Layer):
  class Params(TransformerSelfAttentionLayer.Params,
               ProjectionLayer.Params):
    intermediate_size       = 3072
    intermediate_activation = "gelu"

this allows you to declare the model’s configuration by simply extending the Params of the underlying layers:

class BertModel(Model):
  class Params(BertEmbeddingsLayer.Params,
               TransformerEncoderLayer.Params):
    pass

N.B. The two code excerpts above are taken from kpe/bert-for-tf2, so check there for the details of a non-trivial params-flow based implementation (of BERT).

Resources

  • kpe/py-params - A “type-safe” dict class for python.
  • kpe/bert-for-tf2 - BERT implementation using the TensorFlow 2 Keras API with the help of params-flow for reducing some of the common Keras boilerplate code needed when passing parameters to custom layers.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for params-flow, version 0.6.2
Filename, size & hash File type Python version Upload date
params-flow-0.6.2.tar.gz (11.1 kB) View hashes Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page