hover 0.3.1

Data annotation done right: easy, fun, hyper-productive, and inducing insight.

Imagine editing a picture layer by layer, not pixel by pixel, nor by splashing paint.

We can apply this idea to datasets.

---

Hover is a machine teaching library that enables intuitive and effecient supervision. In other words, it provides a map where you hover over and label your data... differently. For instance, you can:

• :seedling: annotate an intuitively selected group of data points at a time
• :ferris_wheel: throw a model in the loop and exploit active learning
• :whale: cross-check with Snorkel-based distant supervision

Check out @phurwicz/hover-binder for a complete list of demo apps.

:flags: Latest Announcements

• Dec 12 Common usage workflows are now available in hover.recipes. Much cleaner code!
• Dec 6 Video tutorials coming soon! The first one is projected to be available by Dec 15.

:flight_departure: Quick Start

Step 0: load your dataset and compute its 2-d embedding

from hover.core.dataset import SupervisableTextDataset

dataset = SupervisableTextDataset(
    # 'raw' contains the data to be supervised
    raw_dictl=[{"content": "this is great"}],
    # train/dev/test sets can be empty
    # train_dictl=[],
    dev_dictl=[{"content": "this is awesome", "mark": "POSITIVE"}],
    test_dictl=[{"content": "this is meh", "mark": "NEGATIVE"}],
    # specify feature/label keys
    feature_key="content",
    label_key="mark",
)

# define a vectorizer for your feature, then call dimensionality reduction
nlp = spacy.load('en')
vectorizer = lambda text: nlp(text).vector
dataset.compute_2d_embedding(vectorizer, "umap")

Step 1: choose a recipe (or create your own with examples)

from hover.recipes.experimental import (
    simple_annotator,
    active_learning,
    snorkel_crosscheck,
)

handle = simple_annotator(dataset)

Step 2: fire it up

Hover uses bokeh to deliver its annotation interface:

# Option 1: in Jupyter
from bokeh.io import show, output_notebook
output_notebook()
show(handle)

# Option 2: in app.py (`bokeh serve app.py` in the command line)
from bokeh.io import curdoc
doc = curdoc()
handle(doc)

# Option 3: elsewhere as an embedded app
from bokeh.server.server import Server
server = Server({'my-app': handle})
server.start()

:package: Installation

Python: 3.6+

OS: tested on Mac & Linux

To get the latest release version, you can use pip:

pip install hover

Installation through conda is not yet available. Please open an issue if you would like conda or conda-forge support.

:flamingo: Features

Here we attempt a quick comparison with a few other packages that do machine teaching:

Package	Hover	Prodigy	Snorkel
Core idea	supervise like editing a picture	scriptable active learning	programmatic distant supervision
Annotates per	batch of just the size you find right	piece predicted to be the most valuable	the whole dataset as long as it fits in
Supports	all classification (text only atm)	text & images, audio, vidio, & more	text classification (for the most part)
Status	open-source	proprietary	open-source
Devs	indie	Explosion AI	Stanford / Snorkel AI
Related	many imports of the awesome Bokeh	builds on the Thinc/SpaCy stack	Variants: Snorkel Drybell, MeTaL, DeepDive
Vanilla usage	define a vectorizer and annotate away	choose a base model and annotate away	define labeling functions and apply away
Advanced usage	combine w/ active learning & snorkel	patterns / transformers / custom models	transforming / slicing functions
Hardcore usage	exploit hover.core templates	custom @prodigy.recipe	the upcoming Snorkel Flow

Hover claims the best deal of scale vs. precision thanks to

• the flexibility to use, or not use, any technique beyond annotating on a "map";
• the speed, or coarseness, of annotation being literally at your fingertips;
• the interaction between multiple "maps" that each serves a different but connected purpose.

:book: Resources

Documentation

• Contains API references of the most crucial components.
• A lot more is on the way! (video tutorials, for example)

Dependencies

• ./requirements-test.txt lists additional dependencies for the test suite.
• ./requirements-dev.txt lists recommended packages for developers.