Nucleus
https://dashboard.scale.com/nucleus
Aggregate metrics in ML are not good enough. To improve production ML, you need to understand their qualitative failure modes, fix them by gathering more data, and curate diverse scenarios.
Scale Nucleus helps you:
- Visualize your data
- Curate interesting slices within your dataset
- Review and manage annotations
- Measure and debug your model performance
Nucleus is a new way—the right way—to develop ML models, helping us move away from the concept of one dataset and towards a paradigm of collections of scenarios.
Installation
$ pip install scale-nucleus
CLI installation
We recommend installing the CLI via pipx (https://pypa.github.io/pipx/installation/). This makes sure that
the CLI does not interfere with you system packages and is accessible from your favorite terminal.
For MacOS:
brew install pipx
pipx ensurepath
pipx install scale-nucleus
# Optional installation of shell completion (for bash, zsh or fish)
nu install-completions
Otherwise, install via pip (requires pip 19.0 or later):
python3 -m pip install --user pipx
python3 -m pipx ensurepath
python3 -m pipx install scale-nucleus
# Optional installation of shell completion (for bash, zsh or fish)
nu install-completions
Common issues/FAQ
Outdated Client
Nucleus is iterating rapidly and as a result we do not always perfectly preserve backwards compatibility with older versions of the client. If you run into any unexpected error, it's a good idea to upgrade your version of the client by running
pip install --upgrade scale-nucleus
Usage
For the most up to date documentation, reference: https://dashboard.scale.com/nucleus/docs/api?language=python.
For Developers
Clone from github and install as editable
git clone git@github.com:scaleapi/nucleus-python-client.git
cd nucleus-python-client
pip3 install poetry
poetry install
Please install the pre-commit hooks by running the following command:
poetry run pre-commit install
When releasing a new version please add release notes to the changelog in CHANGELOG.md.
Best practices for testing: (1). Please run pytest from the root directory of the repo, i.e.
poetry run pytest tests/test_dataset.py
(2) To skip slow integration tests that have to wait for an async job to start.
poetry run pytest -n auto -m "not integration"
Note: "-n auto" is used for pytest-xdist parallelization
Pydantic Models
Prefer using Pydantic models rather than creating raw dictionaries or dataclasses to send or receive over the wire as JSONs. Pydantic is created with data validation in mind and provides very clear error messages when it encounters a problem with the payload.
The Pydantic model(s) should mirror the payload to send. To represent a JSON payload that looks like this:
{
"example_json_with_info": {
"metadata": {
"frame": 0
},
"reference_id": "frame0",
"url": "s3://example/scale_nucleus/2021/lidar/0038711321865000.json",
"type": "pointcloud"
},
"example_image_with_info": {
"metadata": {
"author": "Picasso"
},
"reference_id": "frame0",
"url": "s3://bucket/0038711321865000.jpg",
"type": "image"
}
}
Could be represented as the following structure. Note that the field names map to the JSON keys and the usage of field
validators (@validator).
import os.path
from pydantic import BaseModel, validator
from typing import Literal
class JsonWithInfo(BaseModel):
metadata: dict # any dict is valid
reference_id: str
url: str
type: Literal["pointcloud", "recipe"]
@validator("url")
def has_json_extension(cls, v):
if not v.endswith(".json"):
raise ValueError(f"Expected '.json' extension got {v}")
return v
class ImageWithInfo(BaseModel):
metadata: dict # any dict is valid
reference_id: str
url: str
type: Literal["image", "mask"]
@validator("url")
def has_valid_extension(cls, v):
valid_extensions = {".jpg", ".jpeg", ".png", ".tiff"}
_, extension = os.path.splitext(v)
if extension not in valid_extensions:
raise ValueError(f"Expected extension in {valid_extensions} got {v}")
return v
class ExampleNestedModel(BaseModel):
example_json_with_info: JsonWithInfo
example_image_with_info: ImageWithInfo
# Usage:
import requests
payload = requests.get("/example")
parsed_model = ExampleNestedModel.parse_obj(payload.json())
requests.post("example/post_to", json=parsed_model.dict())
Migrating to Pydantic
- When migrating an interface from a dictionary use
nucleus.pydantic_base.DictCompatibleModel. That allows you to get the benefits of Pydantic but maintaints backwards compatibility with a Python dictionary by delegating__getitem__to fields. - When migrating a frozen dataclass use
nucleus.pydantic_base.ImmutableModel. That is a base class set up to be immutable after initialization.
Updating documentation: We use Sphinx to autogenerate our API Reference from docstrings.
To test your local docstring changes, run the following commands from the repository's root directory:
poetry shell
cd docs
sphinx-autobuild . ./_build/html --watch ../nucleus
sphinx-autobuild will spin up a server on localhost (port 8000 by default) that will watch for and automatically rebuild a version of the API reference based on your local docstring changes.
Custom Metrics using Shapely in scale-validate
Certain metrics use Shapely and rasterio which is added as optional dependencies.
pip install scale-nucleus[metrics]
Note that you might need to install a local GEOS package since Shapely doesn't provide binaries bundled with GEOS for every platform.
#Mac OS
brew install geos
# Ubuntu/Debian flavors
apt-get install libgeos-dev
To develop it locally use
poetry install --extras metrics
1 maintainer
Release files for scale-nucleus 0.22.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| scale_nucleus-0.22.2.tar.gz | 181.3 kB | Details |
Built distributions (wheels)
Total release size: 3.7 MB
Release files / scale_nucleus-0.22.2.tar.gz
| Download URL | scale_nucleus-0.22.2.tar.gz |
|---|---|
| Size | 181.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4fcfb3849593259f2e40fa15e4dc5cfddba166853c4b2ac9cd8b986f1808a2b9
|
|
BLAKE2b-256 checksum How to use checksums |
e6b6a0571805035d37d9e3bc84eee61baf485bcbfb6092f86ba5cafbf267739d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp314-cp314-win_amd64.whl
| Download URL | scale_nucleus-0.22.2-cp314-cp314-win_amd64.whl |
|---|---|
| Size | 229.7 kB |
| Tags | CPython 3.14 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
f18f056028f15fc7e17310b700e8abd8704143fab8f2a6af659df9d4f2286d7d
|
|
BLAKE2b-256 checksum How to use checksums |
f3994680c799b5a9f58767979c2dded8a50cccbcee2e0024d82abc54bd6f8a0b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
| Download URL | scale_nucleus-0.22.2-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl |
|---|---|
| Size | 243.5 kB |
| Tags | CPython 3.14 Linux glibc 2.28+ x86-64 Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
6b62d821bf36b18962d5e07fb56897fef69e3b934e6a35f66bf2502900dc5fd5
|
|
BLAKE2b-256 checksum How to use checksums |
c0ee294bc34e644f1d6e5355b4e06d4930785d35450704132022751b52653b8b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp314-cp314-macosx_11_0_arm64.whl
| Download URL | scale_nucleus-0.22.2-cp314-cp314-macosx_11_0_arm64.whl |
|---|---|
| Size | 227.1 kB |
| Tags | CPython 3.14 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
a8bf7fc8f92ba895ede9cab0c0f04f06c5099fb2b4ff6cbdf38829893fa591de
|
|
BLAKE2b-256 checksum How to use checksums |
d5836012f60a07038403b6541d3d37b4288b89dfd19a02f8ccf012b01a39da7e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp313-cp313-win_amd64.whl
| Download URL | scale_nucleus-0.22.2-cp313-cp313-win_amd64.whl |
|---|---|
| Size | 229.9 kB |
| Tags | CPython 3.13 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
93dbfe90f4f4f4ef28983681f23bae30b0f1f49c43c7a7b604f71075eb8af3e2
|
|
BLAKE2b-256 checksum How to use checksums |
921b3d1585e10e62bd5a5f72cce0c2b4a6adabdb8453048644ec0a107971ff8f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
| Download URL | scale_nucleus-0.22.2-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl |
|---|---|
| Size | 243.5 kB |
| Tags | CPython 3.13 Linux glibc 2.28+ x86-64 Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
a820af23f2d7b81bb4181743709c8ad385f4a41b43cccf40c9fa06cac89da81f
|
|
BLAKE2b-256 checksum How to use checksums |
43445dbda57ac39e10ca0299c3d87c2685d5bf3646445ed435ec00c22266f0e0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp313-cp313-macosx_11_0_arm64.whl
| Download URL | scale_nucleus-0.22.2-cp313-cp313-macosx_11_0_arm64.whl |
|---|---|
| Size | 227.1 kB |
| Tags | CPython 3.13 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
486f53a8210c9aeea6a92dc655a3ecec01372cbd1717d831c4b4d1218624e3e0
|
|
BLAKE2b-256 checksum How to use checksums |
e2b0aa69880dd91af56b508518ea5f48843ec912bfa389ccdb878b4093f9134c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp312-cp312-win_amd64.whl
| Download URL | scale_nucleus-0.22.2-cp312-cp312-win_amd64.whl |
|---|---|
| Size | 229.9 kB |
| Tags | CPython 3.12 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
30a80a77477fae021577276d5a5e645747725a13d786d1af6f97e477abfdebb5
|
|
BLAKE2b-256 checksum How to use checksums |
edc00b10d3d8a9397c66d3e4061aced35f2badc85046c0c6c2c74d85e71a7042
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
| Download URL | scale_nucleus-0.22.2-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl |
|---|---|
| Size | 243.4 kB |
| Tags | CPython 3.12 Linux glibc 2.28+ x86-64 Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
611937d4f6d5695f1c63fe1f763628ede1f2588b39a8f588fcc4a24b4d499b9f
|
|
BLAKE2b-256 checksum How to use checksums |
d75a82a7033bc8ca60590dae4c1223d7caf23658f210cbdb3adad31174782ebe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp312-cp312-macosx_11_0_arm64.whl
| Download URL | scale_nucleus-0.22.2-cp312-cp312-macosx_11_0_arm64.whl |
|---|---|
| Size | 227.1 kB |
| Tags | CPython 3.12 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
ce2a78dab4109d112397bd0309be17a1a91a366c053a38b5ec1d5b3945775266
|
|
BLAKE2b-256 checksum How to use checksums |
688bfb7cac17d2c1d0ed623d4bb33c93c93b7ccb851762c3cd410b1395750e16
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp311-cp311-win_amd64.whl
| Download URL | scale_nucleus-0.22.2-cp311-cp311-win_amd64.whl |
|---|---|
| Size | 229.9 kB |
| Tags | CPython 3.11 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
7fe7509a571e4c49c42776257b9c858ee26a8261f2751d80682699712619dd01
|
|
BLAKE2b-256 checksum How to use checksums |
9ca005742421f29a0556e2d29a57cde5a1f4fd70d6d66f7bf3b075ee2391596a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
| Download URL | scale_nucleus-0.22.2-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl |
|---|---|
| Size | 243.0 kB |
| Tags | CPython 3.11 Linux glibc 2.28+ x86-64 Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
9e5ede2dc6a8521e16fb466fff66a9f9af164c9fddf97b3a026414c135e5c4f8
|
|
BLAKE2b-256 checksum How to use checksums |
152e637eeb80335e3cf106cb0f9fd4d33a6ec92f4ba99b5a40071baaf6c1a794
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp311-cp311-macosx_11_0_arm64.whl
| Download URL | scale_nucleus-0.22.2-cp311-cp311-macosx_11_0_arm64.whl |
|---|---|
| Size | 227.1 kB |
| Tags | CPython 3.11 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
1c0023c26cf5662dd852644f84ad013d17d5010c46a53348ec4740b3c39de42c
|
|
BLAKE2b-256 checksum How to use checksums |
153ec0d33dd00d43a4d92c3459d6b4aabdc8ee12d96817028f46173ecaefb1e0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp310-cp310-win_amd64.whl
| Download URL | scale_nucleus-0.22.2-cp310-cp310-win_amd64.whl |
|---|---|
| Size | 229.9 kB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
522ac58aa48262defb4639b5eb48ac47c285dcf2e8574b637dffdd45d189b3a9
|
|
BLAKE2b-256 checksum How to use checksums |
1b37938531ce4e9206cc9221617fa97b87dc5230a313e18aee574ea3ef52e0a9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
| Download URL | scale_nucleus-0.22.2-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl |
|---|---|
| Size | 242.7 kB |
| Tags | CPython 3.10 Linux glibc 2.28+ x86-64 Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
5b60061184a8e1b192d199fce4e98bebdd076a49657269259e915370b46208e0
|
|
BLAKE2b-256 checksum How to use checksums |
e46b2bfeaf34c93eef080ccbe3393806928ef0404a5cc6054192229402763f7a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
Release files / scale_nucleus-0.22.2-cp310-cp310-macosx_11_0_arm64.whl
| Download URL | scale_nucleus-0.22.2-cp310-cp310-macosx_11_0_arm64.whl |
|---|---|
| Size | 227.1 kB |
| Tags | CPython 3.10 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
8567e401b1b241c974f642249c8025fe772022f53b9023916a586677d9b75765
|
|
BLAKE2b-256 checksum How to use checksums |
acf2888f0284385b7031fa7f0e40414389f403edb8982c73a181bd17b2237490
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.4.2 CPython/3.10.21 Linux/7.0.0-1009-aws
|
