No project description provided
Project description
Whispool
Fast in memory word frequency count, with Rust
Installation
pip install whispool
Basic usage
We assume we have n sentences, with each sentence broken into words. And each sentence is
# each sentence could have diffrent length of words
sentence = [
['This', 'is', 'a', 'sentence'],
['This', 'is', 'another', 'sentence'],
['This', 'is', 'a', 'third', 'sentence', 'here'],
...
]
# each info list has to be IN THE SAME LENGTH
info = list([
['id1', 'category1', 'sci-fi', 'movie','Jan.'],
['id2', 'category1', 'romance', 'movie','Jan.'],
['id3', 'category2', 'sci-fi', 'tv','Feb.'],
...
])
Put in data
Now we put in data
from whispool import Whispool
from pathlib import Path
whisper_multi = Whispool(
directory=Path("test_data"), threads=2, capacity=200)
whisper_multi.consume(sentence, info)
whisper_multi.consume(sentence_batch2, info_batch2)
...
whisper_multi.consume(sentence_batchN, info_batchN)
Fetch the result data
Now we want to see the hot key words of Sci-Fi movies in January
.
df = whisper_multi.final_stats(
[None, None, 'sci-fi', 'movie', 'Jan.'],
top_n=2)
Build from rust
maturin build --release
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distributions
Close
Hashes for whispool-0.1.2-pp37-pypy37_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 79e28aac9bf9b84a7a483ed2363346679b4c0559ab7b4557a4f5bb4ba0eed12d |
|
MD5 | 2cd6b8cc935ae3e95c1279e7bca7a7bf |
|
BLAKE2b-256 | 3d08d6f278d8676639908b355f58571c45059d146633bc3d2d57dcea70736fc5 |
Close
Hashes for whispool-0.1.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 47fd7cec322d1d17f3e904e5ad355ca3e5bd601e84ea5bf8453f4d41ee8cb0b0 |
|
MD5 | 6120ff63424c4d58996175ead0bf40e6 |
|
BLAKE2b-256 | dd405342d06b2ece759efeb68b5b6eee01e268878c25e8c8e4b2450fd74d73a0 |
Close
Hashes for whispool-0.1.2-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 388041bc1e60f1f436170fe323bd997bf60c2d70288107c4e6ae68194b0dab31 |
|
MD5 | d69b7c9e7504a1b19b4bc5fa7caf4695 |
|
BLAKE2b-256 | 86ca8cb1c7a11b1ff881707e36d34af92906175a29a5e824a80c990dc5c0c38d |
Close
Hashes for whispool-0.1.2-cp310-cp310-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38c9dea439988ff021a3648474ed71bfa786cb8f23422b177ee6ec033b449b4f |
|
MD5 | e9163b1e3408f529595b98f747f02ce6 |
|
BLAKE2b-256 | e6b10dc9292ecbc6ff3eada07e2ecc20175efd1c0d74b62f919036f00d35683b |
Close
Hashes for whispool-0.1.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8e09198159eb08b8245e5412e956972b370c9b5e0d5db6e3d58430395fad627 |
|
MD5 | a4b19ca490354aba51ba93f36fd9f37c |
|
BLAKE2b-256 | 858e359a64e0eb1511afad33ee66b170ec9af960a6d9ed58d9b4751608d59e8a |
Close
Hashes for whispool-0.1.2-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1552a0a7393abad0d6f5fd6a2b44da1abb4d8aca5665a471385ec5aaa7f761f3 |
|
MD5 | 570c8f8395e47555fb73cced08146cb3 |
|
BLAKE2b-256 | d96368e08b7e9a90729faa89f7e59070b070c51180aa9b835c268b6a213ac8bc |
Close
Hashes for whispool-0.1.2-cp39-cp39-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d6d9e29d5dc4c1537498c50aeec30a693bcf3d063fc399f81f52de31cdb0efa1 |
|
MD5 | 5fa1ea15ce0d2819f71521ec2013370c |
|
BLAKE2b-256 | ee944fa0518d7424b0e9bf81459afb241f87ee172313ae2be5e28028b7e114b9 |
Close
Hashes for whispool-0.1.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9567dde86050b133e9321418226cf022ae299d73c9311eb7c7d32f2f08ec381 |
|
MD5 | 2ebdbb0fab034fb26d1c4e3dd6173623 |
|
BLAKE2b-256 | aacebcf5c7919d2c9da869e9a068a5fd46f925fa31915a2e20225dafe819be04 |
Close
Hashes for whispool-0.1.2-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 220ad8067a440bac1f86aea17fc0d5512b0d6b68950d021bcd26c7728866875d |
|
MD5 | 5e5d35ff12647557b8ff328c0b028dc0 |
|
BLAKE2b-256 | 35be13aa9b02534614a1d687f7395b7a55e97975ec55934affd87721f3247ff2 |
Close
Hashes for whispool-0.1.2-cp38-cp38-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb912558e7cd616d444ca7ce3b5a53b82b24dc07984f7f7d2b7a545207d487ec |
|
MD5 | c80fecff6dbae1263321c9c3a5eece62 |
|
BLAKE2b-256 | 9b1b23516b419725cf963f736577b8e8535875bd22472dd2e31191d55e1f3219 |
Close
Hashes for whispool-0.1.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f06891f3a4894f09da60c68ccb035e80e526e77d0e5b86d6ec0c33f6e3b58024 |
|
MD5 | 688ed652951342422eb287e5413acd82 |
|
BLAKE2b-256 | 8330c3e390264609355911372fd47b70170bc7aa99882b1a7e0d5a00244777bd |
Close
Hashes for whispool-0.1.2-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ed285c6e23f5bae075be47a562a291d8f40bb109fc71b1aaaa4bf7f24013b992 |
|
MD5 | 699a0f7b4e324f1ac076da0b8e8ae856 |
|
BLAKE2b-256 | 5926406156eea3c45273af559d17982a87139af1718a498633e47ece92f01d8a |
Close
Hashes for whispool-0.1.2-cp37-cp37m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a58df3ff9d44e926555feeb1a065c331c6cafeac5021264bf3b1b547fb5e3391 |
|
MD5 | 7b68b2888dbec20e0751d24877b785d4 |
|
BLAKE2b-256 | ab8c74274ee06da48856621bf3d7236c7d92bd402afeb75675300f9d8ebc2ddb |
Close
Hashes for whispool-0.1.2-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3cd2e124188c2e0d6aab190906fab38048b1b558b757e80a64f57c819271bd5 |
|
MD5 | 486c21ba8d9f624d1cae9c3fb27ff2dd |
|
BLAKE2b-256 | 83c61e4566bcc985cb9a06a9883ea8abb771dfa519fd4c68947ae6c642f15fa8 |
Close
Hashes for whispool-0.1.2-cp36-cp36m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f92fbe6d0224c8c3085d8bb4f7838e3c7b8a9c079706e511892290d228d84db5 |
|
MD5 | 20f603c426efaf0d64b232af2d59bcbf |
|
BLAKE2b-256 | 005c36f460f9eb272f9980c4a146a9c31e4c8183642ecf2b4c1743add6a0653a |
Close
Hashes for whispool-0.1.2-cp36-cp36m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d67e16ddc99881ab2c9052d0870d7355ec81d22908c119315c09c5377774c309 |
|
MD5 | b2706ad65ad40202bf9ea2fd058081ae |
|
BLAKE2b-256 | cde425a90a85918300e525aceb20d5f9997ddef7a7abb79e760d28a9dfde557d |