Data annotation done right: easy, fun, hyper-productive, and inducing insight.
Project description
Imagine editing a picture layer by layer, not pixel by pixel, nor by splashing paint.
We can apply this idea to datasets.
Hover
is a machine teaching library that enables intuitive and effecient supervision. In other words, it provides a map where you hover over and label your data... differently. For instance, you can:
:seedling: annotate an intuitively selected group of data points at a time
:ferris_wheel: throw a model in the loop and exploit active learning
:whale: cross-check with Snorkel-based distant supervision
Check out @phurwicz/hover-binder for a complete list of demo apps.
:flags: Latest Announcements
- Dec 12 Common usage workflows are now available in
hover.recipes
. Much cleaner code! - Dec 6 Video tutorials coming soon! The first one is projected to be available by Dec 15.
:flight_departure: Quick Start
Step 0: load your dataset and compute its 2-d embedding
from hover.core.dataset import SupervisableTextDataset
dataset = SupervisableTextDataset(
# 'raw' contains the data to be supervised
raw_dictl=[{"content": "this is great"}],
# train/dev/test sets can be empty
# train_dictl=[],
dev_dictl=[{"content": "this is awesome", "mark": "POSITIVE"}],
test_dictl=[{"content": "this is meh", "mark": "NEGATIVE"}],
# specify feature/label keys
feature_key="content",
label_key="mark",
)
# define a vectorizer for your feature, then call dimensionality reduction
nlp = spacy.load('en')
vectorizer = lambda text: nlp(text).vector
dataset.compute_2d_embedding(vectorizer, "umap")
Step 1: choose a recipe (or create your own with
examples
)
from hover.recipes.experimental import (
simple_annotator,
active_learning,
snorkel_crosscheck,
)
handle = simple_annotator(dataset)
Step 2: fire it up
Hover
uses bokeh
to deliver its annotation interface:
# Option 1: in Jupyter
from bokeh.io import show, output_notebook
output_notebook()
show(handle)
# Option 2: in app.py (`bokeh serve app.py` in the command line)
from bokeh.io import curdoc
doc = curdoc()
handle(doc)
# Option 3: elsewhere as an embedded app
from bokeh.server.server import Server
server = Server({'my-app': handle})
server.start()
:package: Installation
Python: 3.6+
OS: tested on Mac & Linux
To get the latest release version, you can use pip
:
pip install hover
Installation through conda
is not yet available. Please open an issue if you would like conda
or conda-forge
support.
:flamingo: Features
Here we attempt a quick comparison with a few other packages that do machine teaching:
Package | Hover |
Prodigy |
Snorkel |
---|---|---|---|
Core idea | supervise like editing a picture | scriptable active learning | programmatic distant supervision |
Annotates per | batch of just the size you find right | piece predicted to be the most valuable | the whole dataset as long as it fits in |
Supports | all classification (text only atm) | text & images, audio, vidio, & more | text classification (for the most part) |
Status | open-source | proprietary | open-source |
Devs | indie | Explosion AI | Stanford / Snorkel AI |
Related | many imports of the awesome Bokeh |
builds on the Thinc /SpaCy stack |
Variants: Snorkel Drybell , MeTaL , DeepDive |
Vanilla usage | define a vectorizer and annotate away | choose a base model and annotate away | define labeling functions and apply away |
Advanced usage | combine w/ active learning & snorkel | patterns / transformers / custom models | transforming / slicing functions |
Hardcore usage | exploit hover.core templates |
custom @prodigy.recipe | the upcoming Snorkel Flow |
Hover
claims the best deal of scale vs. precision thanks to
- the flexibility to use, or not use, any technique beyond annotating on a "map";
- the speed, or coarseness, of annotation being literally at your fingertips;
- the interaction between multiple "maps" that each serves a different but connected purpose.
:book: Resources
Documentation
- Contains API references of the most crucial components.
- A lot more is on the way! (video tutorials, for example)
Dependencies
./requirements-test.txt
lists additional dependencies for the test suite../requirements-dev.txt
lists recommended packages for developers.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.