Skip to main content

get_aligned_BERT_emb

Get the aligned BERT embedding for sequence labeling tasks

Installing as a dependency

pip install aligned-bert-embedder

Installing dependencies

conda env create -f environment.yml

Example of usage from cmd (not recommended):

python -m aligned_bert_embedder embed aligned_bert_embedder/configs/snip.yml aligned_bert_embedder/texts/triple.txt

Example of usage from code (preferable)

from aligned_bert_embedder import AlignedBertEmbedder

embeddings = AlignedBertEmbedder(config).embed(
  (
    (
      'First', `sentence`, `or`, `other`, `context`, `chunk`
    ),
    (
      `Second`, `sentence`
    )
  )
)

The following is the content of the original README.md file from the developer repo.

Why this repo?

In the origin script extract_features.py in BERT, tokens may be splited into pieces as follows:

orig_tokens = ["John", "Johanson", "'s",  "house"]
bert_tokens = ["[CLS]", "john", "johan", "##son", "'", "s", "house", "[SEP]"]
orig_to_tok_map = [1, 2, 4, 6]

We investigate 3 align strategies (first, mean and max) to maintain an original-to-tokenized alignment. Take the "Johanson -> johan, ##son" as example:

  • first: take the representation of johan as the whole word Johanson
  • mean: take the reduce_mean value of representations of johan and ##son as the whole word Johanson
  • max: take the reduce_max value of representations of johan and ##son as the whole word Johanson

How to use this repo?

sh run.sh input_file outout_file BERT_BASE_DIR
# For example:
sh run.sh you_data you_data.bert path/to/bert/uncased_L-12_H-768_A-12 

You can modify layers and align_strategies in the run.sh.

How to load the output embeddings?

After the above procedure, you are expected to get a output file of contextual embeddings (e.g., your_data_6_mean). Then you can load this file like conventional word embeddings. For example in a python script:

with open("your_data_6_mean", "r", encoding="utf-8") as bert_f"
    for line in bert_f:
        bert_vec = [[float(value) for value in token.split()] for token in line.strip().split("|||")] 

Metadata

Release files for aligned-bert-embedder 0.81

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for aligned-bert-embedder 0.81
File Interpreter ABI Platform Actions
aligned_bert_embedder-0.81-py3-none-any.whl Python 3 none any Details for aligned_bert_embedder-0.81-py3-none-any.whl

Release files / aligned_bert_embedder-0.81-py3-none-any.whl

Download URL aligned_bert_embedder-0.81-py3-none-any.whl
Size 27.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
756edce37afd31d3275c103fe639b4872a41ff115b66a61dc083a9ef43c420ef
BLAKE2b-256 checksum
How to use checksums
eb736044b174146aa8e522cd6729604cd0e7191d8cbbafadbdf4e6bf2e2af629
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0.post20200210 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.6

Security for aligned-bert-embedder 0.81

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.81 This release

1 release file No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page