AfriLink SDK

Version: 0.9.2

One-line access to GPUs for training, finetuning, and inference — from any notebook.

pip install afrilink-sdk

Works in: Google Colab · Kaggle · Jupyter · VS Code · any Python 3.8+ environment

Contents

1. 60-second quickstart
2. Authentication
3. Which method should I use?
4. Backend (k3s)
5. Guides by task — Pretrain · Finetune · Generate · Custom containers
6. Working with your output model
7. API reference
8. Hardware & billing
9. Model & dataset registry
10. Troubleshooting
11. Built-in help

60-second quickstart

This example uses the default k3s backend — no environment variable needed.

from afrilink import AfriLinkClient

client = AfriLinkClient()
client.authenticate(api_key="afk_live_…")  # or set AFRILINK_API_KEY

job = client.pretrain(
    kind="image-classify",
    model="resnet18",
    data="./images/",
    params={"epochs": 5, "num_classes": 2},
    gpus=1,
)
result = job.run(wait=True)
print(result["status"])

Every other capability (finetune, generate, custom containers) follows this same client → job → job.run() pattern. See Guides by task.

Authentication

Auth is a single stateless API key — no passwords, no certificate refresh, no SSH key management on your side.

Get a key

1. Sign up at dataspires.com.
2. Profile → AfriLink SDK keys → Create new key. Copy the afk_live_… value — it's shown once.
3. Store it as AFRILINK_API_KEY.

Set the key

client = AfriLinkClient()
client.authenticate()   # resolves from secret / env / argument, in that order

What happens at auth time

Your API key is exchanged at api.dataspires.com for a short-lived Supabase JWT used for billing writes. The JWT lives in memory for the kernel lifetime — nothing is written to disk. Rotate a key by revoking it on the dashboard and minting a new one.

Which method should I use?

Rule of thumb: train from zero → pretrain(). Adapt existing weights → finetune(). Frozen answers → generate(). Need a bespoke environment → build_and_train().

client.train(), adapt(), and initialize() are deprecated in 0.9 and will be removed in 1.0.0.

Backend (k3s)

The pilot and student path uses the DataSpires K3s GPU cluster (default). You do not need to set AFRILINK_BACKEND.

Guides by task

Pretrain

(train from scratch — YOLO, CNN, or transformer)

Image classification:

job = client.pretrain(
    kind="image-classify",
    model="resnet18",
    data="./images/",
    params={"epochs": 5, "num_classes": 2},
    gpus=1,
)
result = job.run(wait=True)

YOLO detection:

job = client.pretrain(
    kind="yolo-detect",
    weights="yolo11n.pt",       # or scratch=True for random init
    data="./dataset/",
    data_config="data.yaml",
    gpus=1,
    time_limit="02:00:00",
)
result = job.run(wait=True)

Curated containers resolved from kind=:

Need a stack outside these? → Custom containers.

K3s output retrieval

On k3s, your script writes to /workspace/job/output/; the orchestrator syncs it to S3 on completion.

from pathlib import Path
job.k8s_runner.download_output(job.job_id, Path("./my-outputs"))

Requires a deployed orchestrator exposing GET /api/v1/jobs. Full spec: dataspires.com/docs.

Finetune

(LoRA/QLoRA adaptation of an LLM/VLM — runs in the afrilink-finetune container)

import pandas as pd

data = pd.DataFrame({"text": ["Below is an instruction...\n\n### Response:\n..."]})

job = client.finetune(
    model="qwen2.5-0.5b",
    training_mode="low",     # low | medium | high — see table below
    data=data,
    gpus=1,
    time_limit="01:00:00",
)
result = job.run(wait=True)
if result["status"] == "completed":
    client.download_model(result["job_id"], "./my-model")

For VLMs, pass task= (e.g. captioning / VQA) with image+text data. See afrilink/finetune in the built-in help.

Generate

(frozen inference — no weight updates)

job = client.generate(
    model="smolvlm-256m",
    data="./manifest_dir/",
    outputs=["text", "confidence"],   # add "hidden_states" for analysis
    max_new_tokens=64,
    temperature=0.0,
)
result = job.run(wait=True)

Custom containers

For frameworks or model versions the curated containers don't cover.

spec = dict(
    base_image="pytorch",       # preset — see table below
    pip_packages=["transformers>=4.45", "accelerate>=0.34", "peft>=0.13"],
    apt_packages=["git"],
    model_source={"kind": "huggingface", "id": "Qwen/Qwen2.5-0.5B-Instruct"},
)

# check cache first — avoids a ~5 min rebuild for a spec you've already built
hit = client.find_existing_image(**spec)

result = client.build_and_train(
    **spec,
    script="my_train.py",
    gpus=1,
    time_limit_hours=0.5,
    reuse_existing_image=True,   # default
)
client.download_model(result["run"]["job_id"], "./output")

base_image presets:

model_source kinds: huggingface (id, optional revision/subfolder), url, git, gs, s3, or omit to load the model yourself in-script. Models are fetched at runtime, not baked into the image — keeps images ~2GB instead of 7+GB. Gated HF models (Llama, Gemma) work automatically if you add HUGGINGFACE_TOKEN as a notebook secret.

Cache key includes base_image, pip_packages, apt_packages, index URLs, and model_source. It excludes script, env, extra_files, and job/user IDs — so two runs with an identical environment spec but different training scripts share a cached image.

Lifecycle: the built image lives permanently in Artifact Registry (that's what cache hits read from); the local copy on the compute node is deleted at the end of each build_and_train() call unless cleanup_image_after=False.

Working with your model

Convert to GGUF for Ollama / llama.cpp

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
model = PeftModel.from_pretrained(base, "./my-model")
merged = model.merge_and_unload()
merged.save_pretrained("./my-model-merged")
AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B").save_pretrained("./my-model-merged")

# python convert_hf_to_gguf.py ./my-model-merged --outfile my-model.gguf --outtype f16
# ./llama-quantize my-model.gguf my-model-q4.gguf Q4_K_M
# ollama create my-model -f Modelfile && ollama run my-model

Publish to Hugging Face Hub

from huggingface_hub import HfApi

api = HfApi(token="hf_...")
api.create_repo("your-username/my-finetuned-model", exist_ok=True)
api.upload_folder(folder_path="./my-model", repo_id="your-username/my-finetuned-model")  # adapter only

API reference

AfriLinkClient

TrainJob / FinetuneJob / GenerateJob

Returned by pretrain() / finetune() / generate() / build_and_train().

Result shape — k3s:

{"job_id": "a1b2c3d4", "status": "completed", "billing": null}

On failure, also includes "error": "<last 200 log lines>".

Async job discovery:

result = job.run(wait=False)
for entry in client.list_jobs():
    print(entry["job_id"], entry["state"], entry.get("reason"))

status = client.get_job_status(result["job_id"])  # works even after a notebook restart

list_jobs(include_completed=True) includes completed jobs for up to 3 days.

Hardware & billing

Jobs run on the DataSpires k3s GPU cluster. Prefer small models and training_mode="low" (QLoRA) for workshop / assignment budgets.

Billing: wall-clock GPU usage is charged against your DataSpires balance when billing is enabled for the job path. Invoices: dataspires.com/dashboard/billing. Cloud Build time for custom images is absorbed by the platform — you only pay for GPU time. k3s result objects currently return billing: null until cluster billing is fully wired.

Model & dataset registry

client.list_available_models()                # all
client.list_available_models(size="tiny")      # tiny | small | medium | large
client.list_available_datasets()
client.get_model_requirements("qwen2.5-0.5b", "low")

Anything else → custom containers with model_source=.

Troubleshooting

Built-in help

Query the inline reference manual from any cell — no internet required:

import afrilink

afrilink/help          # index
afrilink/quickstart
afrilink/auth
afrilink/choose
afrilink/finetune
afrilink/pretrain
afrilink/generate
afrilink/training      # deprecated train() notes
afrilink/specs
afrilink/datasets
afrilink/billing

Links

• Docs: dataspires.com/docs
• Repository / Issues: github.com/DataSpires/afrilink-sdk
• License: MIT