iterative-bert
An iterative refinement BERT encoder based on Tiny Recursive Models.
Installation
pip install iterative-bert
Usage
Load a pre-trained encoder from HuggingFace Hub
from iterative_bert.model import IterativeBert
# Load encoder from HuggingFace Hub
encoder = IterativeBert.from_pretrained("your-username/your-model")
# Run inference
import torch
input_ids = torch.tensor([[101, 2054, 2003, 2023, 102]]) # Example tokens
attention_mask = torch.ones_like(input_ids)
outputs = encoder(input_ids, attention_mask=attention_mask)
hidden_states = outputs.last_hidden_state
Create a new encoder from config
from iterative_bert.model import IterativeBert, IterativeBertConfig
config = IterativeBertConfig(
vocab_size=30522,
hidden_size=768,
num_hidden_layers=1,
num_attention_heads=12,
intermediate_size=3072,
h_cycles=1,
l_cycles=8,
use_rope=True,
)
encoder = IterativeBert(config)
Features
- Iterative Refinement: Applies transformer layers multiple times with residual connections
- RoPE Support: Rotary Position Embeddings for better length generalization
- Flash Attention: Optional Flash Attention 2/3 support for efficiency
- HuggingFace Compatible: Works with
from_pretrainedandsave_pretrained
License
Apache 2.0
Metadata
Release files for iterative-bert 0.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| iterative_bert-0.1.2.tar.gz | 34.5 kB | Without provenance | Details for iterative_bert-0.1.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| iterative_bert-0.1.2-py3-none-any.whl | Python 3 | none | any | Details for iterative_bert-0.1.2-py3-none-any.whl |
Total release size: 73.3 kB
Release files / iterative_bert-0.1.2.tar.gz
| Download URL | iterative_bert-0.1.2.tar.gz |
|---|---|
| Size | 34.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
575588df0f81e24c042cf6c653be4f5b826ca16354ef6e4c6e7999d2d6fd1f43
|
|
BLAKE2b-256 checksum How to use checksums |
a90b9949aa1f6b5f9b1c3a414a059a3996254ae4b4150e27dac8f9a6802183da
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.28 {"installer":{"name":"uv","version":"0.9.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"CachyOS Linux","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / iterative_bert-0.1.2-py3-none-any.whl
| Download URL | iterative_bert-0.1.2-py3-none-any.whl |
|---|---|
| Size | 38.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
df430763fff68077daa0b5cc6270b8cfeabe8d81292e59de63954712964c2c62
|
|
BLAKE2b-256 checksum How to use checksums |
c5da1338eac5da8594a23f79eb1b4d70e31af395565b624cedd883555e061ffa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.28 {"installer":{"name":"uv","version":"0.9.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"CachyOS Linux","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for iterative-bert 0.1.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.