Skip to main content

Supernova

Supernova NepaliFast V5 tokenizer.

Installation

pip install supernova

Usage

from supernova import Tokenizer

tokenizer = Tokenizer("v5")

tokens = tokenizer.encode("नेपाल राम्रो छ।")
text = tokenizer.decode(tokens)

print(tokens)
print(text)

Metadata

Release files for supernova-v5 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for supernova-v5 0.1.1
File Interpreter ABI Platform
supernova_v5-0.1.1-py3-none-any.whl Python 3 none any Details

Release files / supernova_v5-0.1.1-py3-none-any.whl

Download URL supernova_v5-0.1.1-py3-none-any.whl
Size 14.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a849a9d0f89d53b3df86bb64947f7c9733b783072c79fd7e7f03e3f513466cea
BLAKE2b-256 checksum
How to use checksums
a49b3f1bb73d2c41888fa079f1853057be59c7cb71659958c17b6e08f0fd55b7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.15

Release history Release notifications | RSS feed

This release

0.1.1 This release

1 release file

0.1.0

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page