license: mit library_name: rumi tags:
- rumi
- language-model
- text-generation
- decoder-only-transformer
- from-scratch pipeline_tag: text-generation
RUMI-8M
RUMI-8M is a small decoder-only Transformer language model trained from scratch as part of the RUMI project.
Model Information
- Model name: RUMI-8M
- Parameters: 8,459,904
- Architecture: Decoder-only Transformer
- Vocabulary size: 16,000
- Context length: 256
- Embedding dimension: 192
- Transformer layers: 12
- Attention heads: 6
- Feed-forward dimension: 768
- Dropout: 0.1
- Weight tying: enabled
- Tokenizer: Byte-Level BPE
Important
RUMI-8M is a base language model.
It is not instruction-tuned and is not intended to behave like ChatGPT. Its primary capability is language-model text continuation.
Load RUMI-8M
Install the package dependencies:
pip install torch tokenizers huggingface_hub
1 maintainer
Metadata
1 maintainer
Release files for rumi-llm 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| rumi_llm-1.0.0.tar.gz | 8.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| rumi_llm-1.0.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 16.2 kB
Release files / rumi_llm-1.0.0.tar.gz
| Download URL | rumi_llm-1.0.0.tar.gz |
|---|---|
| Size | 8.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a78f9db5c795009f5b0f845aedd7e1a5c48185d5cdb29499d0b5df8aa5ccebd8
|
|
BLAKE2b-256 checksum How to use checksums |
3a780960c6ebf19f515b5d5c22a99abd856ef93d9d3cfc2db40d2f19de0ae544
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.9
|
Release files / rumi_llm-1.0.0-py3-none-any.whl
| Download URL | rumi_llm-1.0.0-py3-none-any.whl |
|---|---|
| Size | 8.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e0eb46c568d455e3559f08bc05e1bd8444970ab75841e1972f7225455d97a87a
|
|
BLAKE2b-256 checksum How to use checksums |
0e1ace5f2dea3bf89bb8a5126fdd0565544fffc4638020edfcdf6018f2b343dd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.9
|
