Temporal Wikipedia Blocks (TWB)
Temporal Wikipedia Blocks (TWB) is a powerful Python package designed to process the extensive edit history of Wikipedia pages into easily manageable and memory-friendly blocks. The package is specifically developed to enable efficient parallelization and composition of these blocks to facilitate faster processing and analysis of large Wikipedia datasets. The original design of this package is to build other Wikipedia-oriented datasets on top of it.
The package works by dividing the Wikipedia edit history into temporal blocks, which are essentially subsets of the complete dataset that are based on time intervals. These blocks can then be easily processed and analyzed without the need to load the entire dataset into memory.
Installation
The package is available on PyPI and can be installed using pip:
pip install twb-project
Benefits
- Efficient: The package is designed to be memory-friendly and can be easily parallelized to process large datasets.
- Fast: The package is designed to be fast and can be easily optimized to process large datasets.
- Flexible: The package is designed to be flexible and can be easily extended to support other types of blocks.
- Composable: The package is designed to be composable and can be easily combined with other packages to build other datasets.
Specification
- Default compression method: ZStandard.
1 maintainer
Release files for twb-project 1.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| twb_project-1.1.2.tar.gz | 22.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| twb_project-1.1.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 50.7 kB
Release files / twb_project-1.1.2.tar.gz
| Download URL | twb_project-1.1.2.tar.gz |
|---|---|
| Size | 22.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
949469485d31a01021185f83a2385cde4cab0df065e5f70b83a125910bb10304
|
|
BLAKE2b-256 checksum How to use checksums |
2ca184658a9e867828027bfe171d8ad9a9605f2cb3ff9631907c868edfa99880
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.4.0 CPython/3.9.13 Darwin/22.4.0
|
Release files / twb_project-1.1.2-py3-none-any.whl
| Download URL | twb_project-1.1.2-py3-none-any.whl |
|---|---|
| Size | 28.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2a3da7124d4eb69194c168bd48a5466bd35472eaf4a3de3d07f0d5722b347f2d
|
|
BLAKE2b-256 checksum How to use checksums |
67b630a3abd1303104d5596b01bd50325ce89200c245685e1610bee5ba5cfdd6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.4.0 CPython/3.9.13 Darwin/22.4.0
|
