hover 0.4.0

Data annotation done right: easy, fun, hyper-productive, and inducing insight.

Imagine editing a picture layer by layer, not pixel by pixel, nor by splashing paint.

We can apply this idea to datasets.

---

Hover is a machine teaching library that enables intuitive and effecient supervision. In other words, it provides a map where you hover over and label your data... differently. For instance, you can:

• :seedling: annotate an intuitively selected group of data points at a time
• :ferris_wheel: throw a model in the loop and exploit active learning
• :whale: cross-check with Snorkel-based distant supervision

Check out @phurwicz/hover-binder for a complete list of demo apps.

:flags: Latest Announcements

• Dec 26 Major update coming soon:

  • Low-level APIs in bokeh.core.explorer will undergo significant refacorization. Watch out if you've been using them directly!
  • There will be a change log :sunglasses:

• Dec 16 We decided to make notebook tutorials instead of videos, because

  • hover is now embeddable in Jupyter notebooks, and
  • active development tends to outdate video tutorials rather quickly.

• Dec 12 Common usage workflows are now available in hover.recipes. Much cleaner code!

:flight_departure: Quick Start

Step 0: load your dataset

from hover.core.dataset import SupervisableTextDataset

dataset = SupervisableTextDataset(
    raw_dictl=[{"content": "this is great"}],                  # the raw data to be supervised
    # train_dictl=[],                                          # train/dev/test sets can be empty
    dev_dictl=[{"content": "this is awesome", "mark": "A"}],
    test_dictl=[{"content": "this is meh", "mark": "B"}],
    feature_key="content",                                     # specify feature/label keys
    label_key="mark",
)

# define a vectorizer for your feature, then call dimensionality reduction
import spacy
nlp = spacy.load('en')
vectorizer = lambda text: nlp(text).vector # we recommend wrapping a @lru_cache around this
dataset.compute_2d_embedding(vectorizer, "umap")

Step 1: choose a recipe

(or create your own with examples)

from hover.recipes import simple_annotator

handle = simple_annotator(dataset)

Step 2: fire it up

Hover uses bokeh to deliver its annotation interface:

option 1: in Jupyter

from bokeh.io import show, output_notebook
output_notebook()
show(handle)

option 2: with bokeh serve

from bokeh.io import curdoc
doc = curdoc()
handle(doc)

option 3: elsewhere as an embedded app

from bokeh.server.server import Server
server = Server({'/my-app': handle})
server.start()

:package: Installation

Python: 3.6+

OS: tested on Mac & Linux

To get the latest release version, you can use pip:

pip install hover

Installation through conda is not yet available. Please open an issue if you would like conda or conda-forge support.

:flamingo: Features

Here we attempt a quick comparison with a few other packages that do machine teaching:

Package	Hover	Prodigy	Snorkel
Core idea	supervise like editing a picture	scriptable active learning	programmatic distant supervision
Annotates per	batch of just the size you find right	piece predicted to be the most valuable	the whole dataset as long as it fits in
Supports	all classification (text only atm)	text & images, audio, vidio, & more	text classification (for the most part)
Status	open-source	proprietary	open-source
Devs	indie	Explosion AI	Stanford / Snorkel AI
Related	many imports of the awesome Bokeh	builds on the Thinc/SpaCy stack	Variants: Snorkel Drybell, MeTaL, DeepDive
Vanilla usage	define a vectorizer and annotate away	choose a base model and annotate away	define labeling functions and apply away
Advanced usage	combine w/ active learning & snorkel	patterns / transformers / custom models	transforming / slicing functions
Hardcore usage	exploit hover.core templates	custom @prodigy.recipe	the upcoming Snorkel Flow

Hover claims the best deal of scale vs. precision thanks to

• the flexibility to use, or not use, any technique beyond annotating on a "map";
• the speed, or coarseness, of annotation being literally at your fingertips;
• the interaction between multiple "maps" that each serves a different but connected purpose.

:book: Resources

Documentation

• Contains API references of the most crucial components.
• A lot more is on the way!

:bell: Remarks

Shoutouts

• Thanks to Bokeh because hover would not exist without linked plots and callbacks.
• Thanks to Philip Vollet for sharing hover with the community!

Dependencies

• ./requirements-dev.txt lists recommended packages for developers.