Tools for building streamcorpus objects, such as those used in TREC.
Project description
streamcorpus_pipeline is a document processing pipeline that assembles streamcorpus objects from raw data sets.
The streamcorpus_pipeline python module contains tools for processing streamcorpus.StreamItem objects stored in Chunks. It includes transform functions for getting clean_html, clean_visible, creating labels from hyperlinks to particular sites (e.g. Wikipedia), and taggers like LingPipe, Serif, and Factorie, which make Tokens and Sentences.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
streamcorpus_pipeline-0.5.10.tar.gz
(455.6 kB
view hashes)
Built Distribution
Close
Hashes for streamcorpus_pipeline-0.5.10.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 2b78f8b28f2d4172118104796075cb4ab040ba272016558f0df9db7579a48ab1 |
|
| MD5 | 4fb444a00c744f935c785f7ba328933c |
|
| BLAKE2b-256 | 1915ec967e2e50db1ded624d66588c2a055c11eae3706e67d7b97a6cd7e01a66 |
Close
Hashes for streamcorpus_pipeline-0.5.10-py2.7.egg
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 40ab8fbdee0af6576b2c969f742b15eee667abec4e6bddb146e36a2021814cfe |
|
| MD5 | 0590215ed593f19a2cfdd200c2791f7e |
|
| BLAKE2b-256 | b6a3166ec8e12096dff10722c75fabeb07aef51b2d82935a40a6e69696332d97 |
