glovpy

Package for interfacing Stanford's C GloVe implementation from Python.

Installation

Install glovpy from PyPI:

pip install glovpy

Additionally the first time you import glopy it will build GloVe from scratch on your system.

Requirements

We highly recommend that you use a Unix-based system, preferably a variant of Debian. The package needs git, make and a C compiler (clang or gcc) installed.

Otherwise the implementation is as barebones as it gets, only the standard library and gensim are being used (gensim only for producing KeyedVectors).

Example Usage

Here's a quick example of how to train GloVe on 20newsgroups using Gensim's tokenizer.

from gensim.utils import tokenize
from sklearn.datasets import fetch_20newsgroups

from glovpy import GloVe

texts = fetch_20newsgroups().data
corpus = [list(tokenize(text, lowercase=True, deacc=True)) for text in texts]

model = GloVe(vector_size=25)
model.train(corpus)

for word, similarity in model.wv.most_similar("god"):
    print(f"{word}, sim: {similarity}")

API Reference

class glovpy.GloVe(vector_size, window_size, symmetric, distance_weighting, alpha, min_count, iter, initial_learning_rate, threads, memory)

Wrapper around the original C implementation of GloVe.

Parameters

Attributes

Methods

glovpy.GloVe.train(tokens)

Train the model on a stream of texts.

glovpy.utils.reusable(gen_func)

Function decorator that turns your generator function into an iterator, thereby making it reusable. You can use this if you want to reuse a generator function so that multiple passes can be made.

Parameters

Returns

Example usage

Here's how to stream a very long file line by line in a reusable manner.

from gensim.utils import tokenize
from glovpy.utils import reusable
from glovpy import GloVe

@reusable
def stream_lines():
    with open("very_long_text_file.txt") as f:
        for line in f:
            yield list(tokenize(line))

model = GloVe()
model.train(stream_lines())