Skip to main content

Statistics for each tag's set of key phrases

Project description

PyPI version PyPI pyversions PyPI license

A concise yet efficient implementation for computing the statistics of each tag's set of key phrases over input lines in Python 3. One of the major applications is for sentiment analysis, where each tag is a sentiment with the respective key phrases describing the sentiment.

How it Works

A trie structure is constructed to index all the key phrases. Then each line is matched towards the index to compute the respective statistics. The time complexity is $O(m^2 \cdot n)$, where $m$ is the maximum number of words in each line and $n$ is the number of lines.

Installation

This package is available on PyPI. Just use pip3 install -U TagStats to install it.

Usage

You can simply call compute(content, tagDict), where content is a list of lines and tagDict is a dictionary with each tag name as key and the respective set of key phrases as value.

from tagstats import compute

print(compute(
    [
        "a b c",
        "b c",
        "a c e"
    ],
    {
        "+": ["a b", "a c"],
        "-": ["b c"]
    }
))

The output is a dictionary with each tag name as key and the respective totaled frequencies for lines as value.

{'+': [1, 0, 1], '-': [1, 1, 0]}

You can change the default tokenizer, by specifying a compiled regex as separator to tagstats.tagstats.tokenizer. You can disable the tokenizer to allow matching over word boundaries, by specifying None.

You can pre-build the index by calling index(tagDict), and later reuse it more than once as an optional parameter of compute(content, tagDict, index).

Tip

I strongly encourage using PyPy instead of CPython to run the script for best performance.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

TagStats-0.1.2.tar.gz (2.7 kB view details)

Uploaded Source

File details

Details for the file TagStats-0.1.2.tar.gz.

File metadata

  • Download URL: TagStats-0.1.2.tar.gz
  • Upload date:
  • Size: 2.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for TagStats-0.1.2.tar.gz
Algorithm Hash digest
SHA256 40115d0e9bb89a2c6b70a2478300f399ce085aa62122171e52bc8e2b7bfb5f3d
MD5 dd4deeff4eec6dfcd36ba1d8323f7a15
BLAKE2b-256 0a30a9fe524890f702b287d923c50fbf6d801297051869c3d8d929234f968825

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page