This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Rabbit of Caerbannog

Well, that’s no ordinary rabbit - that’s the most foul, cruel, and bad-tempered rodent you ever set eyes on!

—Tim the Enchanter

This module is a high-level interface for the Vowpal Wabbit machine learning system. Currently it relies on the wabbit_wappa module for lower-level interaction, but strives to provide a more high-level object-oriented interface.

There are currently 3 kinds of Rabbit`s you can `import from caerbannog:

Your standard rabbit instance. By default runs Vowpal Wabbit using pipes for stdin/stdout
Runs Vowpal Wabbit in active learning mode, using TCP socket
The initializer expects the argument fp which is an open file with ‘wt’ mode. the inputs fed to teach will be written to this file for offline processing.

Movie Review Sentiments - Active learning demo with caerbannog

Import relevant modules here. We are using the ActiveRabbit for active online learning

from caerbannog import ActiveRabbit, Example
from itertools import islice
import random
import nltk
from nltk.corpus import movie_reviews

Create an active bunny, with active mellowness of 0.01

rabbit = ActiveRabbit(loss_function='logistic', active_mellowness=0.01)

Load the documents from NLTK movie review corpus (note that you need to download these first by For each document, make a tuple (document_words, category) where category is either ‘pos’ or ‘neg’ and document_words is a list of words from tokenizer.

documents = [(list(movie_reviews.words(fileid)), category)
              for category in movie_reviews.categories()
              for fileid in movie_reviews.fileids(category)]

The feature extractor function. First filters out all tokens that are non-alphanumeric. Then make the ‘w’ namespace consist of all the words in the review; ‘n’ consists of 2-4 ngrams of the document words.

def document_features(document_words):
    document_words = list(filter(str.isalnum, document_words))
    example = Example()
    ngrams = set()
    for j in range(2, 5):
        ngrams.update('_'.join(i) for i in nltk.ngrams(document_words, j))

    return example

Vowpal Wabbit expects labels to be -1 and 1 for logistic binary classifier

def convert_sent(sent):
    return {'pos': 1, 'neg': -1}[sent]

Convert the sentiment value and extract features.

examples = [ (convert_sent(sent), document_features(doc)) for (doc, sent) in documents ]

Train with 1500 first examples and keep the remaining ones for verification

teach, test = examples[:1500], examples[1500:]

Teach the filter. We ask for prediction for each example; if the importance is over 1 we “label” the example and teach it to the classifier. We repeat the classification 40 times to ensure that the classifier has had enough to adjust the weights.

taught = 0
predicted = 0
labelled = set()
for i in range(10):
    for sent, ex in teach:
        predicted += 1
        if rabbit.predict(example=ex).importance >= 1:
            rabbit.teach(label=sent, example=ex)
            taught += 1

print("Predicted {}, taught {} (ratio {}). {} unique inputs labelled"
      .format(predicted, taught, taught/predicted, len(labelled)))
Predicted 15000, taught 1057 (ratio 0.07046666666666666). 1042 unique inputs labelled

Test with the testing set. For each correctly labelled example, increase the counter

correct = 0
for sent, ex in test:
    prediction = rabbit.predict(example=ex)
    if prediction.label == sent:
        correct += 1

print("{} inputs predicted. {} correct; ratio {}".format(len(test), correct, correct / len(test)))
500 inputs predicted. 418 correct; ratio 0.836


MIT license.

Release History

Release History


This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
caerbannog-0.1.tar.gz (5.1 kB) Copy SHA256 Checksum SHA256 Source Oct 29, 2014

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting