A Python package for infini-gram
Project description
infini-gram
To use this package, please check out its documentation here: https://infini-gram.io/pkg_doc
To learn more about infini-gram:
- Paper: https://arxiv.org/abs/2401.17377
- Project Home: https://infini-gram.io
- Web Interface: https://infini-gram.io/demo
- API Endpoint: https://infini-gram.io/api_doc
License
This package is licensed under the UW Academic Software License. Use by universities and non-profit institutions is allowed. Commercial use is not allowed. A copy of the license is enclosed with the package distribution.
The suffix array implementation is adapted from Lee et al. (2021), which is distributed under Apache-2.0.
Citation
If you find infini-gram useful, please kindly cite our paper:
@article{Liu2024InfiniGram,
title={Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens},
author={Liu, Jiacheng and Min, Sewon and Zettlemoyer, Luke and Choi, Yejin and Hajishirzi, Hannaneh},
journal={arXiv preprint arXiv:2401.17377},
year={2024}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
File details
Details for the file infini_gram-2.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 3.3 MB
- Tags: CPython 3.12, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | c7ee08e35363ce2eafcba4d0d5352285d48ecc610eb0d3968d86f17923c1f767 |
|
MD5 | 4962344143cec59f88834f924f8a9006 |
|
BLAKE2b-256 | 76174649becbe1bfe8fa8d2edfd58607ad047f689aa38b498ce89e0c29e19208 |
File details
Details for the file infini_gram-2.1.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl
- Upload date:
- Size: 3.4 MB
- Tags: CPython 3.12, manylinux: glibc 2.17+ i686
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 23d0dd8c7b917c448fbe1006d398d7bc2abd74169303bfc33ac5cf6fbbe876fc |
|
MD5 | 72b926350aaadb1af3e7076171f46d2f |
|
BLAKE2b-256 | 96e08c10d2ed4993990c9e81a4182bc753d7848423820195f75a0113d28f51ba |
File details
Details for the file infini_gram-2.1.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 3.3 MB
- Tags: CPython 3.11, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 90e050bedc54922c5c8af6eff3878677d782be96b2d5c92f4b9867cb7fc8f7d8 |
|
MD5 | 639113d31040537488e9bbac054f7543 |
|
BLAKE2b-256 | bbc7839c23b7da8a33893081dccd770d7bbcbf1d5009a761b6d51a2f0f7190a4 |
File details
Details for the file infini_gram-2.1.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl
- Upload date:
- Size: 3.4 MB
- Tags: CPython 3.11, manylinux: glibc 2.17+ i686
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 357e264e34683e0f48eda0cf6b88607918d5c41ab0430edfc4c421a71699d965 |
|
MD5 | c23d7438f5b6690791189768c8c5cca0 |
|
BLAKE2b-256 | eb5bd04a10c657897673cfcd7e262fd9c5b6c056e657a54aba0a20fbd8c49d4b |
File details
Details for the file infini_gram-2.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 3.3 MB
- Tags: CPython 3.10, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e745d38b80929c283bf488cb2d95a4349673891daf06bc83e203bbed8235285 |
|
MD5 | 184dfa650542f148f5a697fdbb932c5b |
|
BLAKE2b-256 | c3bb0d1673a8f9acad3ed85e61852e81595bc26b1f051f40bb55ebf7410152ab |
File details
Details for the file infini_gram-2.1.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
- Upload date:
- Size: 3.4 MB
- Tags: CPython 3.10, manylinux: glibc 2.17+ i686
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1d3b68bb8b9945dec785f1e38f49613b566e14b8bda02b96aee1cb4f0cfa9ee2 |
|
MD5 | 846f9aa7722361bd7f923d593d1c54df |
|
BLAKE2b-256 | eaf21ac24b56a305b507ece3fa60d4bdea37f6df8bdc787bc5fd509b6c319e89 |
File details
Details for the file infini_gram-2.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 3.3 MB
- Tags: CPython 3.9, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 71087915d76a36bb0a7530b27bb2687af64aa49ea56bb0781592e32a853b33b9 |
|
MD5 | 083c9b6b05057a97f44048c0e7751b76 |
|
BLAKE2b-256 | 5fc8ef3b517e62f3c347ec5bc9030022c76a0121c8d62440124553efe0d90ced |
File details
Details for the file infini_gram-2.1.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl
- Upload date:
- Size: 3.4 MB
- Tags: CPython 3.9, manylinux: glibc 2.17+ i686
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0a482240d37d8285f88f88beb3fb2d9c8138024db3f0e494a2e4512dbc53343e |
|
MD5 | 223547d08edaf2ce81d176c55255c054 |
|
BLAKE2b-256 | c95204c64f60d943f221776ebc43d5d441aec2960dba9e5c3e6855de9ace2829 |
File details
Details for the file infini_gram-2.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 3.3 MB
- Tags: CPython 3.8, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 99b020b08844fd44830220b2b7f314c5c1760f0bc2cd5afda7c350d94bcc9058 |
|
MD5 | 40d179b363fac6999dfa55de7a95dfec |
|
BLAKE2b-256 | af14069b40852b724178308bbb8e37a06072544c4a5b3d9f8d0453eb2d304135 |
File details
Details for the file infini_gram-2.1.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl
.
File metadata
- Download URL: infini_gram-2.1.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl
- Upload date:
- Size: 3.4 MB
- Tags: CPython 3.8, manylinux: glibc 2.17+ i686
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4ef0c0483e2b5019eeeed19bab4041d646f921ea1878cb3ccb8872561fe6f6c9 |
|
MD5 | de9764cfb1cc9282a247e8d392e608a3 |
|
BLAKE2b-256 | a3eb1c4a4cd1e34798452f69764e50fab3309bd3f62b784dbf0a8b6a23561d6f |