zeno-client 0.1.8

Python client for creating new Zeno projects and uploading data.

Zeno Python Client

The Zeno Python client lets you create and manage Zeno projects from Python.

Example

A Zeno project has a base dataset and any number of systems (AI models) to evaluate on a dataset. The following example shows how to upload a dataset and a system to a Zeno project.

from zeno_client import ZenoClient
import pandas as pd

# Create a Zeno client with your API key
client = ZenoClient("YOUR_API_KEY")

# Create a project with a specific data renderer, e.g. "text-classification" or "image-classification"
# See view options at https://zenoml.com/docs/views/
project = zeno_client.create_project(
    "test project", 
    view="text-classification",
    metrics=[ZenoMetric(name="accuracy", type="mean", columns=["correct"])]
)
# Upload a simple dataset
# You need to provide at least an id column.
# Your dataframe can contain additional metadata which will be usable in Zeno.
df = pd.DataFrame({
    "id": [1, 2, 3],
    "text": ["Zeno", "of", "Elea"],
    "label": ["A", "B", "B"]
})
project.upload_dataset(df, id_column="id", label_column="label", data_column="text")

# Upload a system to the project
# ... run inference on your model ...
df_out = pd.DataFrame({"id": [1, 2, 3], "output": ["A", "B", "A"]})
df_out['correct'] = df_out['output'] == df['label']
project.upload_system("System A", df_out, output_column="output", id_column="id")

See the examples directory for more in-depth examples.

API Documentation

Documentation generated with pydoc-markdown.

pydoc-markdown -I zeno_client -m client --render-toc > docs.md

Table of Contents

• client
  • ZenoMetric
  • ZenoProject
    • __init__
    • upload_dataset
    • upload_system
  • ZenoClient
    • __init__
    • create_project
    • get_project

client

Functions to upload data to Zeno's backend.

ZenoMetric Objects

class ZenoMetric(BaseModel)

A metric to calculate for a Zeno project.

Attributes:

• id int - The ID of the metric. -1 if not set.
• name str - The name of the metric.
• type str - The type of metric to calculate.
• columns list[str] - The columns to calculate the metric on.

ZenoProject Objects

class ZenoProject()

Provides data upload functionality for a Zeno project.

Attributes:

• api_key str - The API key to authenticate uploads with.
• project_uuid str - The ID of the project to add data to.
• endpoint str - The base URL of the Zeno backend.

__init__

def __init__(api_key: str, project_uuid: str, endpoint: str = DEFAULT_BACKEND)

Initialize the Project object for API upload calls.

Arguments:

• api_key str - the API key to authenticate uploads with.
• project_uuid str - the ID of the project to add data to.
• endpoint str, optional - the base URL of the Zeno backend.

upload_dataset

def upload_dataset(df: pd.DataFrame,
                   id_column: str,
                   label_column: Optional[str] = None,
                   data_column: Optional[str] = None)

Upload a dataset to a Zeno project.

Arguments:

• df pd.DataFrame - The dataset to upload.
• id_column str - The name of the column containing the instance IDs.
• label_column str | None, optional - The name of the column containing the instance labels. Defaults to None.
• data_column str | None, optional - The name of the column containing the raw data. Only works for small text data. Defaults to None.

upload_system

def upload_system(system_name: str, df: pd.DataFrame, output_column: str,
                  id_column: str)

Upload a system to a Zeno project.

Arguments:

• df pd.DataFrame - The dataset to upload.
• system_name str - The name of the system to upload.
• output_column str - The name of the column containing the system output.
• id_column str - The name of the column containing the instance IDs.

ZenoClient Objects

class ZenoClient()

Client class for data upload functionality to Zeno.

Attributes:

• api_key str - The API key to authenticate uploads with.
• endpoint str - The base URL of the Zeno backend.

__init__

def __init__(api_key, endpoint=DEFAULT_BACKEND) -> None

Initialize the ZenoClient object for API upload calls.

Arguments:

• api_key str - the API key to authenticate uploads with.
• endpoint str, optional - the base URL of the Zeno backend. Defaults to DEFAULT_BACKEND.

create_project

def create_project(name: str,
                   view: str,
                   metrics: List[ZenoMetric] = [],
                   data_url: str = "",
                   calculate_histogram_metrics: bool = True,
                   samples_per_page: int = 10,
                   public: bool = True) -> ZenoProject

Creates an empty project in Zeno's backend.

Arguments:

• name str - The name of the project to be created. The project will be created under the current user, e.g. username/name.
• project - str,
• view str - The view to use for the project.
• metrics list[ZenoMetric], optional - The metrics to calculate for the project. Defaults to [].
• data_url str, optional - The URL to the data to use for the project. Defaults to "".
• calculate_histogram_metrics bool, optional - Whether to calculate histogram metrics. Defaults to True.
• samples_per_page int, optional - The number of samples to show per page. Defaults to 10.
• public bool, optional - Whether the project is public. Defaults to False.

Returns:

ZenoProject | None: The created project object or None if the project could not be created.

Raises:

• ValidationError - If the config does not match the ProjectConfig schema.
• HTTPError - If the project could not be created.

get_project

def get_project(project_name: str) -> ZenoProject

Get a project object by its name. Names are split into owner/project_name.

Arguments:

• project_name str - The owner/project_name of the project to get.

Returns:

Project | None: The project object or None if the project could not be found.

Raises:

• HTTPError - If the project could not be found.