Skip to main content

A bert sentence encoding tool

Project description

This is a bert sentence encoding tool.

How to use it

install package

pip install --index-url https://pypi.python.org/simple/ bert-sent-encoding==0.1.3

or

git clone ssh://git@gitlab.leihuo.netease.com:32200/shaojianzhi/bert-sent-encoding.git
cd bert-sent-encoding
python setup.py install

then use it

from bert_sent_encoding import bert_sent_encoding # 1st line
bse = bert_sent_encoding(model_path='bert_sent_encoding/model/chinese_L-12_H-768_A-12', seq_length=64, batch_size=8) # 2nd line
vector = bse.get_vector('你好', word_vector=False, layer=-1)   # 3rd line 1. get vector of string
vectors = bse.get_vector(['你好', '哈哈'], word_vector=False, layer=-1)  # 4th line 2. get vector list of strings
bse.write_txt2vector(input_file, output_file, word_vector=False, layer=-1)   # 5th line 3. get and write vectors of strings

for 2nd line:

bse = bert_sent_encoding(model_path='bert_sent_encoding/model/chinese_L-12_H-768_A-12', seq_length=64, batch_size=8)
*model_path is required, seq_length and batch_size are optional

for 3rd, 4th and 5th lines

vector = bse.get_vector('你好', word_vector=False, layer=-1)   # 3rd line 1. get vector of string
vectors = bse.get_vector(['你好', '哈哈'], word_vector=False, layer=-1)  # 4th line 2. get vector list of strings
bse.write_txt2vector(input_file, output_file, word_vector=False, layer=-1)   # 5th line 3. get and write vectors of strings
*word_vector and layer are optional*

for 5th line:

bse.write_txt2vector(input_file, output_file)   # 3. get and write vectors of strings

path of input_file and output_file are defined by user and below is content of input_file:

the first line text
the second line text
...

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
bert_sent_encoding-0.1.4-py3-none-any.whl (51.0 kB) Copy SHA256 hash SHA256 Wheel py3
bert_sent_encoding-0.1.4.tar.gz (45.5 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page