Installation

uv add gyokusai

ruff check --select I --fix . && ruff format .

@software{shippole2026gyokusai,
  title        = {Gyokusai: A Web-Scale Text Processing Library},
  author       = {Shippole, Enrico and Hu, Andrew and Li, Owen and Huang, Thomas and Jitsev, Jenia},
  year         = {2026},
  url          = {https://github.com/teraflop-ai/gyokusai},
  version      = {0.1.1},
  note         = {Python package available on PyPI at \url{https://pypi.org/project/gyokusai/}}
}

@misc{paster2023openwebmath,
    title={OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text},
    author={Keiran Paster and Marco Dos Santos and Zhangir Azerbayev and Jimmy Ba},
    year={2023},
    eprint={2310.06786},
    archivePrefix={arXiv},
    primaryClass={cs.AI}
}

@misc{nemo_curator,
  title = {NeMo Curator: GPU-Accelerated Data Curation for Training AI Models},
  author = {NVIDIA},
  year = {2024},
  url = {https://github.com/NVIDIA-NeMo/Curator}
}

@article{parmar2024nemotron4,
  title = {Nemotron-4 15B Technical Report},
  author = {Parmar, Jupinder and Satheesh, Shrimai and others},
  journal = {arXiv preprint arXiv:2402.16819},
  year = {2024}
}

Metadata

Release files for gyokusai 0.1.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gyokusai 0.1.10
File	Size	Uploaded
gyokusai-0.1.10.tar.gz	38.0 kB	Sep 26, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for gyokusai 0.1.10
File	Interpreter	ABI	Platform	Reset
gyokusai-0.1.10-py3-none-any.whl 57.7 kB Sep 26, 2026	Python 3	none	any	Details

Total release size: 95.7 kB

Release files / gyokusai-0.1.10.tar.gz

Download URL	gyokusai-0.1.10.tar.gz
Size	38.0 kB
Tags	Source
SHA-256 checksum How to use checksums	`31a858746a2d379a8af3b511263922e457147fd1e29b61168d7b573d60ae8362`
BLAKE2b-256 checksum How to use checksums	`fbc1c90d4665523b7b43716cc4c8d48520cea890b82d7bc1b12303ee8b5436c7`
Upload date	Sep 26, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.11.31 {"installer":{"name":"uv","version":"0.11.31","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}`

Release files / gyokusai-0.1.10-py3-none-any.whl

Download URL	gyokusai-0.1.10-py3-none-any.whl
Size	57.7 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`9ddcf2a5471700ad497bd7b1cd6c3c3423f5f1e5384a134dadd1e410e5238931`
BLAKE2b-256 checksum How to use checksums	`8d7fc4acf8bf5d0114591afae030cea35a08263892d0b432a17116fa99c8e00a`
Upload date	Sep 26, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.11.31 {"installer":{"name":"uv","version":"0.11.31","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}`