DataSpires SDK
Version: 0.1.24
Train, fine-tune, and run models on DataSpires GPUs from any notebook.
pip install dataspires
Works in: Google Colab · Kaggle · Jupyter · VS Code · any Python 3.9+ environment
Latest (0.1.24)
- Security:
authenticate()no longer receives or stores infrastructure credentials; the SDK supports the DataSpires cluster (k3s) only; removed legacy CINECA, A100 and Cloud Build paths.
Contents
- Quickstart
- Authentication
- Which method should I use?
- Models & data sources
- Resource guide
- Jobs
- Guides by task — Pretrain · Finetune · Generate · Evaluate
- Custom containers
- Job patterns — Checkpointing · Long-running jobs · Handling failures
- Working with your model
- Deploy
- API reference
- Hardware
- Model & dataset registry
- Troubleshooting
- Built-in help
Quickstart
Fine-tune a model on a Hugging Face dataset in a few lines:
from dataspires import DataSpiresClient
client = DataSpiresClient()
client.authenticate()
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="low",
data="tatsu-lab/alpaca",
gpus=1,
)
result = job.run(wait=True)
if result["status"] == "completed":
job.download("./my-model")
else:
print(result.get("error") or result.get("reason"))
pretrain() and generate() follow the same pattern: create a job, then call job.run().
Authentication
Get an API key
- Sign in at dataspires.com.
- Go to Profile → SDK keys → Create new key, then copy the
dsp_live_…value. It is shown only once. - Save it as
DATASPIRES_API_KEY.
Set the key
| Environment | How |
|---|---|
| Google Colab | Secrets → DATASPIRES_API_KEY |
| Kaggle | Add-ons → Secrets → DATASPIRES_API_KEY |
| Terminal | export DATASPIRES_API_KEY=dsp_live_… |
| In code | client.authenticate(api_key="dsp_live_…") |
client = DataSpiresClient()
client.authenticate()
authenticate() keeps only your identity and credit balance. It raises an error if the key is invalid or revoked. To rotate a key, revoke it in the dashboard and create a new one.
Gated Hugging Face models
Some registry models (for example Llama or Gemma) are gated on Hugging Face: you must accept the model's license on its Hugging Face page, then provide a token. finetune() and generate() check this before the job is submitted and raise a clear error if the model needs a token and none is found.
- Accept the license at
https://huggingface.co/<model-id>(see the model'shuggingface_idin the registry, e.g.client.list_models()). - Create a token at huggingface.co/settings/tokens.
- Save it as
HUGGINGFACE_TOKEN:
| Environment | How |
|---|---|
| Google Colab | Secrets → HUGGINGFACE_TOKEN |
| Kaggle | Add-ons → Secrets → HUGGINGFACE_TOKEN |
| Terminal | export HUGGINGFACE_TOKEN=hf_… |
Which method should I use?
| Your goal | Method | You provide |
|---|---|---|
| Train a detector, CNN, or transformer from scratch | client.pretrain() |
architecture, kind, data, optional init, config, tokenizer |
| Adapt an existing LLM or vision-language model | client.finetune() |
model, training_mode, data, optional task |
| Run a model without changing its weights | client.generate() |
model, prompts or an image manifest |
| Score a text model | client.evaluate() |
model or a finished LoRA source_job_id, plus tasks or data |
| Serve a finished job over HTTP | client.deploy() |
job_id, optional artifact, name; architecture for a CNN model.pt |
| Run your own script in a container you define | client.build_and_train() |
script or script_content, pip_packages, optional base_image, model_source, data |
Models & data sources
Pass a registry name, a Hugging Face ID, a URL, a Google Drive file, or local data. Remote sources are downloaded directly by the job, so large datasets never pass through your notebook. Local files and DataFrames are uploaded for you.
model=
| Input | Example |
|---|---|
| Registry name | Any catalog alias (e.g. qwen2.5-0.5b, ministral-3b, florence-2-base, smolvlm-256m) |
| Hugging Face ID (optional revision) | Qwen/Qwen2.5-0.5B, Qwen/Qwen2.5-0.5B@main |
| Hugging Face URL | https://huggingface.co/Qwen/Qwen2.5-0.5B |
| Google Drive file | gdrive:FILE_ID or a Drive share link |
data=
| Input | Example |
|---|---|
| In-memory | Python list, pandas DataFrame, Hugging Face Dataset |
| Local file or folder | ./train.jsonl, ./corpus/ |
| Hugging Face dataset | tatsu-lab/alpaca |
| Dataset with config | Salesforce/wikitext#wikitext-2-raw-v1 |
| HTTPS file | https://example.com/train.jsonl (jsonl, csv, parquet, txt, zip) |
| Google Drive file | gdrive:FILE_ID or a Drive share link |
tokenizer= (transformer pretraining)
Accepts gpt2, any Hugging Face tokenizer ID (optionally with @revision), a Hugging Face URL, or a Google Drive file containing tokenizer files.
Preview how a source resolves
from dataspires import resolve_model, resolve_data, resolve_tokenizer
resolve_model(model="Qwen/Qwen2.5-0.5B")
resolve_data(data="Salesforce/wikitext#wikitext-2-raw-v1")
resolve_tokenizer("gpt2")
Dataset format (finetune)
A single text column works best. These common layouts are converted automatically:
| Columns | Converted to |
|---|---|
messages (chat format) |
text |
user + final (optional analysis) |
Instruction-style text |
instruction + output, or prompt + completion |
Instruction-style text |
Google Drive
Pass a Drive file the same way as any other source:
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="low",
data="gdrive:FILE_ID",
gpus=1,
)
- Shared files (“Anyone with the link”) work immediately.
- Private files open a Google sign-in window the first time. After you approve read-only access, the job continues and future jobs reuse your sign-in.
For headless environments, set GOOGLE_DRIVE_TOKEN instead. To turn off the sign-in prompt, set DATASPIRES_GDRIVE_AUTO_CONNECT=0. To sign out, call client.disconnect_google_drive().
To check a file before submitting a job:
client.inspect_source("gdrive:FILE_ID")
Current limitations
| Feature | Status |
|---|---|
| Google Drive folders | Share a .zip of the folder |
| Resume from a Hub, HTTPS, or Drive checkpoint | Coming soon — resume from a previous job today |
Custom YOLO/CNN weights= from Hub or Drive |
Coming soon — built-in weight names are supported |
Resource guide
Check whether a model fits before you start a run:
est = client.estimate_model("qwen2.5-0.5b", training_mode="low", gpus=1)
print(est["fits"], est["min_vram_gb"], est["parameters_b"])
fits |
Meaning |
|---|---|
True |
Expected to fit with this mode and GPU count |
False |
Likely too large — try training_mode="low" or more GPUs |
"unknown" |
Model size not known — start with low on 1 GPU |
If a configuration is unlikely to fit, the SDK shows a warning before submitting.
Jobs
Jobs run asynchronously on the DataSpires GPU cluster.
| Lifecycle | Submit, then wait or poll with list_jobs() |
| Status | status on job.run(), list_jobs(), and get_job_status(). Values: queued, running, submitted, completed, failed, timeout, cancelled, missing |
| Outputs | job.download() or client.download_model() |
| Multi-GPU | Available on request with gpus= |
| GPU | Optional. Omit it to run on DataSpires' own V100 32GB. Pass gpu_type= ("v100", "l4", "4090", "5090", "a100") for the cheapest GPU of that type, or offering= with an id from client.gpus(). See Choosing a GPU. |
| Maximum duration | Set per workspace or cohort. Pass time_limit="HH:MM:SS" within that limit. |
Guides by task
Pretrain
Image classification (CNN):
job = client.pretrain(
architecture="cnn",
kind="resnet18",
init="pretrained",
data="./images/", # train/<class>/ + optional val/<class>/; or config val_split=
config={
"n_epoch": 5,
"num_classes": 2,
"batch_size": 32,
"seed": 42,
# "val_split": 0.2, # if no val/ folder
},
gpus=1,
time_limit="02:00:00",
)
result = job.run(wait=True)
job.export_metrics("./run.csv") # includes val_loss / val_acc when val is set
Object detection:
Use data.yaml with both train: and val: (Ultralytics evaluates each epoch).
job = client.pretrain(
architecture="detector",
kind="detect", # or "segment", "pose"
init="pretrained",
weights="yolo11n.pt",
data="./dataset/",
data_config="data.yaml",
config={"n_epoch": 100, "imgsz": 640, "batch_size": 16, "seed": 42},
)
Detector runs report training loss.
Language model from scratch:
job = client.pretrain(
architecture="transformer",
kind="gpt-decoder-tiny",
init="scratch",
tokenizer="gpt2",
data="roneneldan/TinyStories", # or a local folder such as "./corpus/"
config={"max_steps": 200, "n_epoch": 1, "batch_size": 2, "seed": 42},
)
print(job.estimate())
architecture |
kind |
|---|---|
detector |
detect, segment, pose |
cnn |
resnet18, resnet50 |
transformer |
gpt-decoder-tiny, qwen-decoder-tiny, llama-3-decoder-tiny, vit-tiny |
Download results:
job.download("./my-outputs")
job.download("./checkpoints", include="checkpoints")
Finetune
import pandas as pd
data = pd.DataFrame({"text": ["Below is an instruction...\n\n### Response:\n..."]})
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="low", # low | medium | high | custom
data=data,
gpus=1,
time_limit="01:00:00",
)
result = job.run(wait=True)
if result["status"] == "completed":
job.download("./my-model")
| Mode | Strategy | Precision |
|---|---|---|
low |
QLoRA (rank 8) | 4-bit |
medium |
LoRA (rank 16) | 8-bit on 1 GPU, bf16 on more |
high |
LoRA (rank 64) | bf16 |
custom |
Your settings via config= |
bf16, fp16, 4bit, or 8bit |
For full control, use training_mode="custom" with a config:
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="custom",
data=data,
config={
"batch_size": 2,
"learning_rate": 2e-4,
"scheduler": "cosine",
"n_epoch": 3,
"precision": "bf16",
"lora": {"r": 16, "alpha": 32, "dropout": 0.05},
"max_seq_length": 2048,
"optimizer": "adamw_torch",
"n_checkpoint": 100,
"n_evals": 0,
"seed": 42,
},
)
gpus, gpu_type, and time_limit are passed as arguments, not inside config. Reported accuracy is token-level accuracy.
When n_evals is greater than 0, choose how evaluation data is supplied:
Hub dataset with a built-in val split — used automatically:
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="custom",
data="glue",
config={
"batch_size": 2,
"learning_rate": 2e-4,
"scheduler": "cosine",
"n_epoch": 1,
"precision": "bf16",
"n_evals": 50,
},
)
Local val file or in-memory val data:
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="custom",
data="./train.jsonl",
val_data="./val.jsonl", # or a DataFrame / list of rows
config={
"batch_size": 2,
"learning_rate": 2e-4,
"scheduler": "cosine",
"n_epoch": 1,
"precision": "bf16",
"n_evals": 50,
},
)
Hold out part of the training data (when there is no separate val split):
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="low",
data="tatsu-lab/alpaca",
val_split=0.1,
)
For vision-language models, use a registry VLM alias or a Hugging Face VLM id with task="captioning", task="vqa", or task="chat", plus a folder with train.jsonl and images/ (or a DataFrame with image and response). Florence 2 and chat-style VLMs share the same dataset shape; the job passes model_architecture to the container loader. See dataspires.docs("finetune").
Generate
Run a model without changing its weights:
job = client.generate(
model="Qwen/Qwen2.5-0.5B",
data=[{"text": "Write one short sentence about GPUs."}],
max_new_tokens=48,
gpus=1,
)
result = job.run(wait=True)
if result["status"] == "completed":
job.download("./generate-output")
Generated text is saved to the output folder. Use job.get_logs() to view progress while the job runs.
With a vision-language model and a local image manifest:
job = client.generate(
model="smolvlm-256m",
data="./manifest_dir/",
outputs=["text", "confidence"],
max_new_tokens=64,
temperature=0.0,
)
result = job.run(wait=True)
if result["status"] == "completed":
job.download("./vlm-output")
Evaluate
Score a text model on a benchmark, or on questions you write. The weights do not change.
job = client.evaluate(
model="Qwen/Qwen2.5-0.5B",
tasks="gsm8k",
max_samples=20,
)
result = job.run(wait=True)
if result["status"] == "completed":
print(job.get_results())
job.export_results("./eval_results.json")
tasks="gsm8k" scores grade-school math. max_samples=20 scores 20 questions instead of the full set. Add few_shot=5 when the model should see five answered examples before each question.
Score more than one benchmark in the same job:
job = client.evaluate(
model="Qwen/Qwen2.5-0.5B",
tasks=["gsm8k", "hellaswag"],
max_samples=20,
)
Your own questions
Each row needs a question and the answer you expect. A reply counts when it matches the answer, or when it starts with the answer. A JSONL file works the same way: data="./exam.jsonl".
job = client.evaluate(
model="Qwen/Qwen2.5-0.5B",
data=[
{"question": "The capital of France is", "answer": "Paris"},
{"question": "2 + 2 =", "answer": "4"},
],
)
result = job.run(wait=True)
A model you already trained
Pass the finished text LoRA job. The adapter is applied to the base model, then scored the same way.
job = client.evaluate(
source_job_id=train.job_id,
tasks="gsm8k",
max_samples=20,
)
Use a finished LoRA training job. Its output must include output/final/adapter_config.json. An evaluate job, a full fine-tune, and a vision model do not.
evaluate() is for text models. Classifiers, detectors, and vision-language models keep the scores from training.
evaluate() takes the same gpu_type= and offering= as training. dtype="auto" (the default) scores in bfloat16 on GPUs that support it and float16 on the V100.
Custom containers
Run your own script on the cluster. build_and_train() builds a container from a base image and your dependencies, then starts the job. Install with pip install dataspires and call authenticate() first.
result = client.build_and_train(
base_image="pytorch",
pip_packages=["transformers", "datasets"],
apt_packages=["git"],
script="./train.py",
extra_files=["./helper.py"],
model_source={"kind": "huggingface", "id": "Qwen/Qwen2.5-0.5B"},
data="./train.jsonl",
gpus=1,
time_limit_hours=1.0,
)
run = result["run"]
print(run["status"], run["job_id"])
print(result["build"]["image"])
if run["status"] == "completed":
client.download_model(run["job_id"], "./out")
Pass the script as a string with script_content= instead of script=. That string is saved as /workspace/user/main.py.
Base image
base_image |
Image |
|---|---|
pytorch (default) |
pytorch/pytorch:2.5.0-cuda12.4-cudnn9-runtime |
pytorch-2.4 |
pytorch/pytorch:2.4.0-cuda12.4-cudnn9-runtime |
pytorch-cpu |
pytorch/pytorch:2.5.0-cpu-runtime |
cuda-12.4 |
nvidia/cuda:12.4.0-runtime-ubuntu22.04 |
ultralytics |
ultralytics/ultralytics:latest |
Any other value is used as a full image:tag.
Paths inside the container
| You pass | Where it lands |
|---|---|
script="./train.py" |
/workspace/user/train.py, then python /workspace/user/train.py |
script_content= |
/workspace/user/main.py |
extra_files= |
/workspace/user/<filename> |
data="./train.jsonl" |
/workspace/job/input/train.jsonl, also the AFRILINK_DATA environment variable |
model_source |
Downloaded when the job starts, under /workspace/models/ |
data= is a local file or folder. Download a remote dataset from inside your script. Pass entrypoint= to replace the default python command. env= bakes non-secret variables into the image. HUGGINGFACE_TOKEN is read from your environment when the job starts.
Weights stay out of the image. A Hugging Face id org/name is saved at /workspace/models/org__name. Include huggingface_hub or transformers in pip_packages. For a gated model, set HUGGINGFACE_TOKEN the same way as for finetune().
{"kind": "huggingface", "id": "Qwen/Qwen2.5-0.5B"}
{"kind": "url", "url": "https://example.com/weights.tar.gz"}
{"kind": "git", "url": "https://github.com/you/repo.git", "revision": "main"}
{"kind": "gs", "uri": "gs://bucket/path/"}
{"kind": "s3", "uri": "s3://bucket/path/"}
A git source needs git in apt_packages. A gs:// or s3:// source needs that cloud CLI in the image.
Write files you want back to /workspace/job/output/final/. client.download_model(job_id, "./out") collects that folder.
Build only, or reuse an image
build_image(...) publishes the image and returns its reference. Start the job with build_and_train(). find_existing_image(...) returns the matching image, or None when the cache is empty.
A new build runs when the base image, pip packages, apt packages, or model source changes. After a script-only edit, pass reuse_existing_image=False (or force=True to build_image()).
While it runs
wait=True (the default) blocks until the job finishes.
| Field | Meaning |
|---|---|
result["build"]["image"] |
Image reference |
result["build"]["status"] |
success or cached |
result["run"]["job_id"] |
Training job id |
result["run"]["status"] |
completed, failed, timeout, or submitted |
result["run"]["logs_tail"] |
Last log lines when wait=True |
Submit and check later:
result = client.build_and_train(
script="./train.py",
pip_packages=["transformers"],
data="./train.jsonl",
wait=False,
)
status = client.get_job_status(result["run"]["job_id"])
Job patterns
Cross-cutting workflows that apply after you pick a task (pretrain, finetune, or generate).
Checkpointing and resuming
Save checkpoints during a run, then continue from a previous job:
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="low",
data=data,
time_limit="08:00:00",
checkpoint={"every": 200, "keep": 2},
)
result = job.run(wait=True)
if result["status"] != "completed":
resumed = client.finetune(
model="qwen2.5-0.5b",
training_mode="low",
data=data,
resume_from=job, # or a job ID
)
resumed.run(wait=True)
| Path | Contents |
|---|---|
output/checkpoints/checkpoint-N/ |
Snapshots saved during training |
output/final/ |
Final weights |
List checkpoints with client.list_checkpoints(job_id). See dataspires.docs("checkpoints").
Long-running jobs
Notebook sessions can end before a long job finishes. Submit without waiting, save the job ID, and check back later:
job.run(wait=False)
job_id = job.job_id
# Later, in the same or a new session
status = client.get_job_status(job_id)
print(status["status"])
if status["status"] == "completed":
client.download_model(job_id, "./my-model")
Jobs keep running on the cluster after your notebook disconnects.
Handling failures
job.run(wait=True) returns a result instead of raising when a job fails. Check result["status"] before downloading. Authentication and validation errors raise immediately.
Working with your model
The steps below apply to finetune() outputs. Fine-tuning produces a LoRA adapter, which is loaded on top of the base model.
Convert to GGUF for Ollama or llama.cpp
Merge the adapter into the base model first:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
model = PeftModel.from_pretrained(base, "./my-model")
merged = model.merge_and_unload()
merged.save_pretrained("./my-model-merged")
AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B").save_pretrained("./my-model-merged")
Then convert ./my-model-merged with llama.cpp's convert_hf_to_gguf.py.
Publish to Hugging Face Hub
from huggingface_hub import HfApi
api = HfApi()
# Publish the adapter
api.upload_folder(folder_path="./my-model", repo_id="you/my-adapter", repo_type="model")
# Or publish full merged weights
api.upload_folder(folder_path="./my-model-merged", repo_id="you/my-model", repo_type="model")
Deploy
After a training job finishes, client.deploy() gives you a URL you can call from a notebook or an app. Use the same API key you trained with. The first call after the endpoint has been idle can take a few minutes while the model loads. Later calls are quick. Call client.undeploy() when you are done.
dep = client.deploy(job.job_id, artifact="final", name="my-model")
print(dep.url)
artifact is "final" for the finished weights, "checkpoint-N" for a saved checkpoint, or "best" / "last" for a detector. Check dep.kind and use the matching call below.
Text model
Qwen, Ministral, and a full fine-tune.
print(dep.predict(prompt="Say hi in three words.", max_tokens=32))
print(dep.chat(
messages=[{"role": "user", "content": "Say hi in three words."}],
max_tokens=32,
))
curl -sS -X POST "$URL/v1/predict" \
-H "Authorization: Bearer $DATASPIRES_API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt":"Say hi in three words.","max_tokens":32}'
$URL is dep.url.
Vision-language model
Florence and other models that read an image. Pass a file path.
print(dep.predict(
prompt="Describe this image in one short sentence.",
image="./sample.png",
max_tokens=32,
))
Image classifier
For a CNN such as ResNet, pass the architecture you trained.
dep = client.deploy(
cnn_job.job_id,
artifact="final",
name="resnet",
architecture="resnet18",
)
print(dep.predict(image="./sample.png", top_k=5))
architecture is one of resnet18, resnet50, resnet101, efficientnet_b0, mobilenet_v3_small.
Object detector
dep = client.deploy(detect_job.job_id, artifact="best", name="detector")
print(dep.predict(image="./photo.jpg", conf=0.25))
chat() is only for text and vision-language models. Classifiers and detectors use predict().
Stop a deployment
client.deployments()
client.undeploy(dep.deployment_id)
API reference
DataSpiresClient
| Method | Description |
|---|---|
authenticate(api_key=None) |
Authenticate with your API key |
pretrain(architecture, kind, init, config, gpu_type, …) |
Create a pretraining job |
finetune(model, training_mode, data, config, gpu_type, …) |
Create a fine-tuning job |
generate(model, data, outputs, gpu_type, …) |
Create an inference job |
evaluate(model=None, source_job_id=None, tasks=None, data=None, few_shot=0, max_samples=None) |
Score a text model. See Evaluate |
gpus() |
GPUs you can use now, with specs and live prices. See Choosing a GPU |
build_and_train(script, pip_packages, base_image, …) |
Build a custom container and run it |
build_image(script, pip_packages, base_image, …) |
Build a custom container without starting a job |
find_existing_image(base_image, pip_packages, …) |
Look up a previously built image |
estimate_model(model, training_mode, gpus) |
Estimate memory and fit |
inspect_source(ref) |
Check how a source resolves and whether it is accessible |
connect_google_drive() / disconnect_google_drive() |
Sign in to or out of Google Drive |
download_model(job_id, local_dir, include=("final",)) |
Download job outputs |
list_checkpoints(job_id) |
List saved checkpoints |
list_available_models() / list_available_datasets() |
Browse the registry |
get_model_requirements(model, training_mode) |
Recommended GPUs and memory |
cancel_job(job_id) |
Cancel a job |
list_jobs(include_completed=False) |
List your jobs |
get_job_status(job_id) |
Get the status of a job |
deploy(job_id, artifact="final", name=None, max_replicas=1, architecture=None) |
Serve a finished job. See Deploy |
deployments() |
List your deployments |
deployment(deployment_id) |
Get one deployment |
deployment_update(deployment_id, max_replicas=) |
Change the replica cap (1–4) |
undeploy(deployment_id) |
Delete a deployment |
Job objects
| Member | Description |
|---|---|
run(wait=True, poll_interval=30) |
Submit the job; with wait=True, block until it finishes |
cancel() |
Cancel the job |
get_logs(tail=100) |
Recent log lines |
get_metrics() |
Training metrics |
export_metrics(path) |
Save metrics as .csv or .json |
download(local_dir, include=("final",)) |
Download outputs |
checkpoints() |
List saved checkpoints |
job_id, status |
Job ID and current status |
Fine-tuning jobs also provide validate(). Pretraining jobs provide estimate().
Source helpers
| Helper | Description |
|---|---|
resolve_model |
Resolve a model reference |
resolve_data |
Resolve a data reference |
resolve_tokenizer |
Resolve a tokenizer reference |
estimate_model |
Same as client.estimate_model |
Hardware
Start with a small model and training_mode="low", and use estimate_model before trying larger models.
| Model size | Mode | Single GPU |
|---|---|---|
| 0.5B – 3B | low, medium | Yes |
| 3B – 7B | low | Yes |
| 7B | medium | Yes |
| 13B | low | Tight |
| 30B+ | low | Unlikely |
On-demand jobs run on one GPU. Maximum duration is set per workspace or cohort.
Choosing a GPU
client.gpus() lists the GPUs you can use right now, with GPU memory, the minimum RAM and vCPUs that come with each, and the live price per GPU-hour:
client.gpus()
- Omit
gpu_typeandofferingto run on DataSpires' own V100 32GB (marked*in the list). It runs one job at a time, so your job may wait for it. gpu_type="l4"runs on the cheapest L4, preferring one that can start now.offering="od-l4-2"runs on exactly that GPU from the list.
The price is locked when you submit and printed with the job. Submitting needs credits for one hour at that price; you pay for the time the job runs. A job that has not started within 24 hours is cancelled without charge. Members of a pilot group who omit gpu_type get any eligible GPU in their group.
gpu_type |
Hardware | bf16 |
|---|---|---|
"v100" |
NVIDIA V100 32GB | No (evaluate() switches to fp16) |
"l4" |
NVIDIA L4 24GB | Yes |
"4090" |
NVIDIA RTX 4090 24GB | Yes |
"5090" |
NVIDIA RTX 5090 32GB | Yes |
"a100" |
NVIDIA A100 80GB | Yes |
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="low",
data=data,
gpu_type="l4", # or offering="od-l4-2"
)
print(job.estimated_cost_usd()) # the most it can cost at today's price
Model & dataset registry
Browse curated models and datasets:
client.list_available_models()
client.list_available_models(size="tiny")
client.list_available_datasets()
client.get_model_requirements("qwen2.5-0.5b", "low")
Any public Hugging Face model or dataset also works. See Models & data sources.
Troubleshooting
| Issue | Cause | Solution |
|---|---|---|
API key auth failed |
Invalid or revoked key | Create a new key and update DATASPIRES_API_KEY |
Job stays queued |
Cluster at capacity | The job starts automatically when capacity is available. A job still queued after 24 hours is cancelled automatically without charge; submit it again or pick another GPU. |
No v100 GPU is offered right now |
That GPU type is not for sale at the moment | Run client.gpus() and pick another gpu_type or offering |
| Not enough credits to submit | Submitting needs one hour at the GPU's price | Top up on Billing, or pick a cheaper GPU from client.gpus() |
Dataset has no 'text' column |
Unsupported format | Add a text column or use a supported layout |
| Drive file not accessible | File is private or not shared with you | Sign in when prompted, or share the file as “Anyone with the link” |
| Drive folder link rejected | Folders are not supported | Share a .zip of the folder |
| Google sign-in unavailable | Drive sign-in is not configured in this environment | Set GOOGLE_DRIVE_TOKEN, or see Google Drive setup |
| HTTPS download fails | URL is not publicly reachable | Use a public https:// URL |
| Gated Hugging Face model fails | Missing token | Set HUGGINGFACE_TOKEN |
download_model() returns nothing |
Job did not finish | Check the status, or download with include="checkpoints" |
Job ends with DeadlineExceeded |
Reached time_limit |
Resume with resume_from=job |
Notebook disconnected during wait=True |
Session ended | See Long-running jobs |
| First deploy call is slow | The model is loading | Wait. dep.predict() and dep.chat() retry for you |
chat() raises |
The model is a classifier or a detector | Use predict(image=...) |
| Script edit did not show up in the job | The previous image was reused | Call build_and_train(..., reuse_existing_image=False) |
Built-in help
import dataspires
dataspires.docs("help")
dataspires.docs("finetune")
dataspires.docs("pretrain")
dataspires.docs("checkpoints")
dataspires.docs("evaluate")
dataspires.docs("deploy")
Metadata
Release files for dataspires 0.1.24
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| dataspires-0.1.24.tar.gz | 196.2 kB | Without provenance | Details for dataspires-0.1.24.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| dataspires-0.1.24-py3-none-any.whl | Python 3 | none | any | Details for dataspires-0.1.24-py3-none-any.whl |
Total release size: 376.6 kB
Release files / dataspires-0.1.24.tar.gz
| Download URL | dataspires-0.1.24.tar.gz |
|---|---|
| Size | 196.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
14473e922cd99b2956df4b1ffc3a98bdf63856de7e4cfc55b8f9b5be74ea7666
|
|
BLAKE2b-256 checksum How to use checksums |
82ced28dd766da3333f3c929b71620f3629ff5fa87c6d84b6a13a37e6898bce4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.14.4
|
Release files / dataspires-0.1.24-py3-none-any.whl
| Download URL | dataspires-0.1.24-py3-none-any.whl |
|---|---|
| Size | 180.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3a6bd5c07c8a44cbc01e07e8bac1c5b4f3efb002cddc1b830e5f84b083f46b3c
|
|
BLAKE2b-256 checksum How to use checksums |
a6ac54f5b936ee3b0adcceaae79e83f6cc1053696b73127cd0b4d01907ed533e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.14.4
|
Security for dataspires 0.1.24
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.