A CLI and SDK for interacting with the Efemarai ML testing platform.
Project description
A CLI and SDK for interacting with the Efemarai ML testing platform.
Setup
Install with
pip install -U efemarai
then run
$ efemarai init
and following the instructions to connect your account.
Example Usage
Working with projects:
import efemarai as ef
session = ef.Session()
# List all projects
for project in session.projects:
print(project.name)
# Create a new project
project = session.create_project(
name="Aircraft Detection",
description="Exmample object detection project",
problem_type="ObjectDetection",
)
# Load an existing one
project = session.project("Aircraft Detection")
# List project models
for model in project.models:
print(model.name)
Working with stress tests:
# Create a new stress test
test = project.create_stress_test(
name="Test via SDK",
model=project.model("COCO YOLOR-P6"),
domain=project.domain("Aircraft Domain"),
dataset="Aircraft Detection - Train", # Just a name also works
)
# Load an existing stress test
test = project.stress_test("Test via SDK")
# Download dataset with discovered vulnerabilities
dataset_filepath = test.vulnerabilities_dataset()
# Check test run state
print(f"Running: {test.running} Failed: {test.failed} Finished: {test.finished}")
Models, domains and datasets can be easily created programatically, but
they require quite a few configuration paramaters to be provided. That's
why the most convenient way to create a project with multiple models, domains
and datasets is to put everything into a config file (see e.g.
examples/aircraft_project.yaml
) and then just load it with:
result = ef.Session().load("examples/aircraft_project.yaml")
# access the created entities
project = result["project"]
models = result["models"]
domains = result["domains"]
datasets = result["datasets"]
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
efemarai-0.2.2.tar.gz
(26.9 kB
view hashes)
Built Distribution
efemarai-0.2.2-py3-none-any.whl
(31.6 kB
view hashes)