A bert sentence encoding tool
Project description
This is a bert sentence encoding tool.
## How to use it
### install package
git clone ssh://git@gitlab.leihuo.netease.com:32200/shaojianzhi/bert-sent-encoding.git
cd bert-sent-encoding
python setup.py install
### then use it
from bert_sent_encoding import bert_sent_encoding
bse = bert_sent_encoding(model_path='bert_sent_encoding/model/chinese_L-12_H-768_A-12', seq_length=64, batch_size=8)
(model_path is required, seq_length and batch_size are optional)
vector = bse.get_vector('你好') # 1. get vector of string
vectors = bse.get_vector(['你好', '哈哈']) # 2. get vector list of strings
bse.write_txt2vector(input_file, output_file) # 3. get and write vectors of strings
### for
bse = bert_sent_encoding(model_path='bert_sent_encoding/model/chinese_L-12_H-768_A-12', seq_length=64, batch_size=8)
(model_path is required, seq_length and batch_size are optional)
**input_file**:
the first line text
the second line text
...
## How to use it
### install package
git clone ssh://git@gitlab.leihuo.netease.com:32200/shaojianzhi/bert-sent-encoding.git
cd bert-sent-encoding
python setup.py install
### then use it
from bert_sent_encoding import bert_sent_encoding
bse = bert_sent_encoding(model_path='bert_sent_encoding/model/chinese_L-12_H-768_A-12', seq_length=64, batch_size=8)
(model_path is required, seq_length and batch_size are optional)
vector = bse.get_vector('你好') # 1. get vector of string
vectors = bse.get_vector(['你好', '哈哈']) # 2. get vector list of strings
bse.write_txt2vector(input_file, output_file) # 3. get and write vectors of strings
### for
bse = bert_sent_encoding(model_path='bert_sent_encoding/model/chinese_L-12_H-768_A-12', seq_length=64, batch_size=8)
(model_path is required, seq_length and batch_size are optional)
**input_file**:
the first line text
the second line text
...
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
bert_sent_encoding-0.1.1.tar.gz
(22.0 kB
view hashes)
Built Distributions
Close
Hashes for bert_sent_encoding-0.1.1-py3.6.egg
Algorithm | Hash digest | |
---|---|---|
SHA256 | aaab792a7566577094059da08d0d5544418db2947debcd1245e97e9336e205bf |
|
MD5 | 6d7cb8a091804acb73280a653cffdaad |
|
BLAKE2b-256 | 6a0802543e31b25a81d3c41e40069c19ce912349de0baba5349d604efea5dd37 |
Close
Hashes for bert_sent_encoding-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e448b6f85eca68075a50c14cc029af781bc0746a90cdf4b32ff0e9846afa31e7 |
|
MD5 | c587394cdd441a33cb5e9f3bffdf4b45 |
|
BLAKE2b-256 | 088e9b5fa6e0839fcc6809eb29b3054e457e6eac4b2f1adea7df846a52e7bfee |