a crf layer for tensorflow 2 keras
Project description
tf2crf
- a simple CRF layer for tensorflow 2 keras
- support keras masking
Install
$ pip install tf2crf
Attention
A dense layer is needed before the CRF layer to convert inputs to shape (batch_size, timesteps, num_classes). The 'num_class' is how many tags or catogories the model predicts.
Tips
tensorflow >= 2.1.0 Recommmend use the latest tensorflow-addons which is compatiable with your tf version.
Example
from tf2CRF import CRF
from tensorflow.keras.layers import Input, Embedding, Bidirectional, GRU, Dense
from tensorflow.keras.models import Model
inputs = Input(shape=(None,), dtype='int32')
output = Embedding(100, 40, trainable=True, mask_zero=True)(inputs)
output = Bidirectional(GRU(64, return_sequences=True))(output)
output = Dense(9, activation=None)(output)
crf = CRF(dtype='float32')
output = crf(output)
model = Model(inputs, output)
model.compile(loss=crf.loss, optimizer='adam', metrics=[crf.accuracy])
x = [[5, 2, 3] * 3] * 10
y = [[1, 2, 3] * 3] * 10
model.fit(x=x, y=y, epochs=2, batch_size=2)
model.save('model')
Supoort for tensorflow mixed precision training
Currently these is a bug in tensorflow-addons.text.crf, which causes a dtype error when using miex precision. To correctly use mixed precison, you need to modify the line 488 of tensorflow_addons/text/crf.py to:
crf_fwd_cell = CrfDecodeForwardRnnCell(transition_params, dtype=inputs.dtype)
Example
from tf2CRF import CRF
from tensorflow.keras.layers import Input, Embedding, Bidirectional, GRU, Dense
from tensorflow.keras.models import Model
from tensorflow.keras.mixed_precision import experimental as mixed_precision
policy = mixed_precision.Policy('mixed_float16')
mixed_precision.set_policy(policy)
inputs = Input(shape=(None,), dtype='int32')
output = Embedding(100, 40, trainable=True, mask_zero=True)(inputs)
output = Bidirectional(GRU(64, return_sequences=True))(output)
output = Dense(9, activation=None)(output)
crf = CRF(dtype='float32')
output = crf(output)
model = Model(inputs, output)
model.compile(loss=crf.loss, optimizer='adam', metrics=[crf.accuracy])
x = [[5, 2, 3] * 3] * 10
y = [[1, 2, 3] * 3] * 10
model.fit(x=x, y=y, epochs=2, batch_size=2)
How to save the model
Currently, Loading the model directly is not supported. So you need to load the model weights instead. For example:
tf.keras.models.save_model(model, '1')
model.load_weights('1/variables/variables')
or
model.save_weights('model.h5')
model.load_weights('model.h5')
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tf2crf-0.1.27-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d049326c5266a28ca9bf3de61400f08e008792e9b982499fdeed03f3b079b220 |
|
MD5 | 208051fed36682dc2caefabfe97e80fb |
|
BLAKE2b-256 | 76cbb6a9e23fdbfc877ef6032f0f05bd95b922d32d57d23e36a5d17a10e9dafc |