AfriLink SDK

One-line access to GPUs for training, finetuning, and inference — from any notebook.

pip install afrilink-sdk

Works in: Google Colab · Kaggle · Jupyter · VS Code · any Python 3.8+ environment

Contents

1. 60-second quickstart
2. Which method should I use?
3. Backends: k3s vs. opentoken-a100
4. Authentication
5. Guides by task — Pretrain · Finetune · Generate · Custom containers
6. Working with your output model
7. API reference
8. Hardware & billing
9. Model & dataset registry
10. Troubleshooting
11. Built-in help
12. Architecture (advanced)

60-second quickstart

This example uses the default k3s backend — no environment variable needed.

from afrilink import AfriLinkClient

client = AfriLinkClient()
client.authenticate(api_key="afk_live_…")  # or set AFRILINK_API_KEY

job = client.pretrain(
    kind="image-classify",
    model="resnet18",
    data="./images/",
    params={"epochs": 5, "num_classes": 2},
    gpus=1,
)
result = job.run(wait=True)
print(result["status"])

Every other capability (finetune, generate, custom containers) follows this same client → job → job.run() pattern. See Guides by task.

Which method should I use?

Rule of thumb: train from zero → pretrain(). Adapt existing weights → finetune(). Frozen answers → generate(). Need a bespoke environment → build_and_train().

Backends

AfriLink can run your job on two different backends. This choice affects auth, output retrieval, and billing, so it's worth understanding up front rather than discovering it mid-job.

If you don't set AFRILINK_BACKEND, you are on k3s. Every quickstart below states which backend it's using.

Authentication

As of v0.8.x, auth is a single stateless API key — no passwords, no certificate refresh, no SSH key management on your side.

Get a key

1. Sign up at dataspires.com.
2. Profile → AfriLink SDK keys → Create new key. Copy the afk_live_… value — it's shown once.
3. Store it as AFRILINK_API_KEY.

Set the key

client = AfriLinkClient()
client.authenticate()   # resolves from secret / env / argument, in that order

What happens at auth time (~1–2 sec total)

The JWT lives in memory for the kernel lifetime — nothing is written to disk. Rotate a key by revoking it on the dashboard and minting a new one.

Guides by task

Pretrain

(train from scratch — YOLO, CNN, or transformer)

On the default k3s backend:

job = client.pretrain(
    kind="image-classify",
    model="resnet18",
    data="./images/",
    params={"epochs": 5, "num_classes": 2},
    gpus=1,
)
result = job.run(wait=True)

YOLO example, explicitly on opentoken-a100:

import os
os.environ["AFRILINK_BACKEND"] = "opentoken-a100"

job = client.pretrain(
    kind="yolo-detect",
    weights="yolo11n.pt",       # or scratch=True for random init
    data="./dataset/",
    data_config="data.yaml",
    gpus=1,
    time_limit="02:00:00",
)
result = job.run(wait=True)
client.download_model(result["job_id"], "./yolo-out")

Curated containers resolved from kind=:

Need a stack outside these? → Custom containers.

K3s output retrieval

On k3s, your script writes to /workspace/job/output/; the orchestrator syncs it to S3 on completion.

from pathlib import Path
job.k8s_runner.download_output(job.job_id, Path("./my-outputs"))

Requires a deployed orchestrator exposing GET /api/v1/jobs. Full spec: dataspires.com/docs.

Finetune

(LoRA/QLoRA adaptation of an LLM/VLM — runs in the afrilink-finetune container)

import pandas as pd

data = pd.DataFrame({"text": ["Below is an instruction...\n\n### Response:\n..."]})

job = client.finetune(
    model="qwen2.5-0.5b",
    training_mode="low",     # low | medium | high — see table below
    data=data,
    gpus=1,
    time_limit="01:00:00",
)
result = job.run(wait=True)
if result["status"] == "completed":
    client.download_model(result["job_id"], "./my-model")

Generate

(frozen inference — no weight updates)

job = client.generate(
    model="smolvlm-256m",
    data="./manifest_dir/",
    outputs=["text", "confidence"],   # add "hidden_states" for analysis
    max_new_tokens=64,
    temperature=0.0,
)
result = job.run(wait=True)

Custom containers

For frameworks or model versions the curated containers don't cover. build_and_train() requires AFRILINK_BACKEND=opentoken-a100 (not available on the default k3s backend).

spec = dict(
    base_image="pytorch",       # preset — see table below
    pip_packages=["transformers>=4.45", "accelerate>=0.34", "peft>=0.13"],
    apt_packages=["git"],
    model_source={"kind": "huggingface", "id": "Qwen/Qwen2.5-0.5B-Instruct"},
)

# check cache first — avoids a ~5 min rebuild for a spec you've already built
hit = client.find_existing_image(**spec)

result = client.build_and_train(
    **spec,
    script="my_train.py",
    gpus=1,
    time_limit_hours=0.5,
    reuse_existing_image=True,   # default
)
client.download_model(result["run"]["job_id"], "./output")

base_image presets:

model_source kinds: huggingface (id, optional revision/subfolder), url, git, gs, s3, or omit to load the model yourself in-script. Models are fetched at runtime, not baked into the image — keeps images ~2GB instead of 7+GB. Gated HF models (Llama, Gemma) work automatically if you add HUGGINGFACE_TOKEN as a notebook secret.

Cache key includes base_image, pip_packages, apt_packages, index URLs, and model_source. It excludes script, env, extra_files, and job/user IDs — so two runs with an identical environment spec but different training scripts share a cached image.

Lifecycle: the built image lives permanently in Artifact Registry (that's what cache hits read from); the local copy on the compute node is deleted at the end of each build_and_train() call unless cleanup_image_after=False.

Working with your model

Convert to GGUF for Ollama / llama.cpp

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
model = PeftModel.from_pretrained(base, "./my-model")
merged = model.merge_and_unload()
merged.save_pretrained("./my-model-merged")
AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B").save_pretrained("./my-model-merged")

# python convert_hf_to_gguf.py ./my-model-merged --outfile my-model.gguf --outtype f16
# ./llama-quantize my-model.gguf my-model-q4.gguf Q4_K_M
# ollama create my-model -f Modelfile && ollama run my-model

Publish to Hugging Face Hub

from huggingface_hub import HfApi

api = HfApi(token="hf_...")
api.create_repo("your-username/my-finetuned-model", exist_ok=True)
api.upload_folder(folder_path="./my-model", repo_id="your-username/my-finetuned-model")  # adapter only

API reference

AfriLinkClient

TrainJob / FinetuneJob

Returned by pretrain() / finetune() / build_and_train().

Result shape — opentoken-a100:

{
  "job_id": "a1b2c3d4",
  "status": "completed",
  "output_dir": "/mnt/data/sdk-jobs/a1b2c3d4/output",
  "billing": {"total_gpu_minutes": 5.0, "total_cost_usd": 0.1667, "rate_per_gpu_hour": 0.60}
}

Result shape — k3s:

{"job_id": "a1b2c3d4", "status": "completed", "billing": null}

On failure, also includes "error": "<last 200 log lines>".

Async job discovery (k3s):

result = job.run(wait=False)
for entry in client.list_jobs():
    print(entry["job_id"], entry["state"], entry.get("reason"))

status = client.get_job_status(result["job_id"])  # works even after a notebook restart

list_jobs(include_completed=True) includes completed jobs for up to 3 days.

Hardware & billing

opentoken-a100 node — this backend name refers to the OpenToken node; the GPU it actually gives you access to is a 1× NVIDIA Tesla V100 32GB · 12 CPU cores · 84GB RAM · 774GB storage · CUDA 13.0 pre-installed, containerized via --gpus all.

Billing: $0.60/GPU-hour on opentoken-a100, billed per completed GPU-minute (1-minute minimum), deducted automatically from your DataSpires balance. Invoices: dataspires.com/dashboard/billing. Cloud Build time for custom images is absorbed by the platform — you only pay for GPU time. k3s billing is not yet wired (tracked on the roadmap).

Model & dataset registry

client.list_available_models()                # all
client.list_available_models(size="tiny")      # tiny | small | medium | large
client.list_available_datasets()
client.get_model_requirements("qwen2.5-0.5b", "low")

Anything else → custom containers with model_source=.

Troubleshooting

Built-in help

Query the inline reference manual from any cell — no internet required:

import afrilink

afrilink.docs("help")       # recommended
afrilink.docs("quickstart")
afrilink.docs("auth")
afrilink.docs("finetune")
afrilink.docs("pretrain")
afrilink.docs("generate")
afrilink.docs("specs")
afrilink.docs("datasets")
afrilink.docs("billing")

# Quoted slash form also works:
afrilink / "help"

Architecture (advanced)

High-level flow: your notebook authenticates against api.dataspires.com (a Cloudflare Worker), which exchanges your API key for a short-lived Supabase JWT, a compute-node SSH key, a GCP service-account key (for custom builds), and a registry pull token. Job execution then proceeds either over SSH to the opentoken-a100 node (Docker, running on the V100) or via the K3s orchestrator REST API (Kubernetes), with custom images built out-of-band on Cloud Build and pushed to a private Artifact Registry.

Notebook (Colab / Kaggle / Local)         api.dataspires.com (Cloudflare Worker)
+---------------------+                   +---------------------------+
| AfriLink SDK        | --- POST -----→   | exchange afk_live_… for:  |
|  client.authenticate()                  |  - Supabase JWT (billing) |
|                     | ←-- response ---  |  - V100 SSH key (in-mem)  |
+---------------------+                   |  - GCP SA key (build)     |
     |        ↓                           |  - GHCR PAT (image pulls) |
     |   (in-memory state)                +---------------------------+
     |
     ↓
+---------------------+        SSH        +---------------------+
| docker_runner.py    | ----------------→ | OpenToken V100 32GB |
|  - prepare_job_dir  |   /mnt/data/      |  Docker daemon      |
|  - upload via SCP   |    sdk-jobs/      |  (containerd at     |
|  - docker run --gpus=all                |   /mnt/data/)       |
|  - docker inspect (poll)                +---------------------+
+---------------------+
     |
     ↓ build path
+---------------------+   Cloud Build    +---------------------+
| build.py            | --→ submit job → | europe-west4-       |
|  - generate Docker- |     (anadrome)   | docker.pkg.dev/...  |
|    file from spec   |                  |  afrilink-user-     |
|  - tar build context|                  |  images/<user>/<job>|
|  - upload to GCS    |                  +---------------------+
+---------------------+                            |
                                                   ↓ docker pull
                                              (V100 fetches image,
                                               runs it, deletes
                                               local layer at end)

Full diagram and internals: dataspires.com/docs.

Links

• Docs: dataspires.com/docs
• Repository / Issues: github.com/DataSpires/afrilink-sdk
• License: MIT