Teepin Python SDK

The Python SDK of Teepin Web Services (TWS): Teepin Storage, Teepin Inference, and compute. A synchronous client on requests and an asynchronous one on aiohttp, for Python 3.9 and later.

pip install teepin

Or from source:

git clone https://github.com/FlashbackAi/teepin-python
cd teepin-python
pip install -e .

Quick start

from teepin import Client

client = Client(api_key="tpk_...")  # or set TEEPIN_API_KEY

client.storage.create_bucket("my-bucket")
obj = client.storage.upload_file("my-bucket", "videos/launch.mp4", "launch.mp4")
print(obj.key, obj.size, obj.etag)

client.storage.download_file("my-bucket", "videos/launch.mp4", "copy.mp4")

upload_file sends a file of any size (up to 5 TiB): up to 16 MiB in one request, otherwise in parts sent 8 at a time. Each part is retried on its own, and the stored object is checked against the bytes sent, so a corrupted transfer is never reported as success. download_file fetches large objects as parallel ranges, all pinned to the same version of the object, and writes a temporary file that is renamed into place only when the download is complete and verified.

The asynchronous client has the same methods:

import asyncio
from teepin import AsyncClient

async def main():
    async with AsyncClient() as client:
        objs = await asyncio.gather(
            *(client.storage.upload_file("my-bucket", f"logs/{n}", n) for n in ["a.log", "b.log"])
        )

asyncio.run(main())

Configuration

Argument	Environment	Default
api_key	TEEPIN_API_KEY	none
base_url	TEEPIN_BASE_URL	https://api.teepin.com
timeout		60 s for each API call that is not a transfer
transfer_timeout		300 s without a byte before a transfer is cut off
max_retries		4
inference_max_retries		2, for Teepin Inference (models, chat)
create(..., timeout=)		600 s for a chat answer (a stream: its headers)
project_id		when api_key is a signed-in session's token rather than a project key

Client(base_url, api_key, timeout) keeps the argument order of version 0.1.

Retries

A request that fails for a passing reason is sent again after a wait with full jitter, or the server's Retry-After. It is always retried on 429, 502 and 503. It is retried on 500, 504 or a broken connection only when sending it twice is harmless (reads, deletes, uploads that replace). Requests that cannot be repeated (start an upload, rename, complete) are retried only when they never reached the server. Errors record the number of tries in err.attempts.

Teepin Inference has its own rules. The server bills a chat completion once it has the model's answer, even if the client has gone, so a chat request is sent again only when it certainly was not billed:

• retried: a 429; Teepin's own 500 internal_error, 502 upstream_error, 503 model_unavailable or 503 billing_unavailable (told by the OpenAI-style error body); a connection that was never opened (refused, or DNS failed);
• never retried: a timeout, or a connection reset or cut after the request was sent; a 504; a 5xx whose body is not Teepin's (a load balancer's HTML page); 400, 401, 402, 403, 404, 413; a stream once it has started.

Reads (models.list, models.attestation) are retried on any 429, 5xx or connection error. Retries go up to inference_max_retries times (2; 0 turns them off), waiting from 0.5 s up to 8 s with full jitter, or the server's Retry-After up to 30 s.

A chat request waits up to 600 s for its answer, whatever the client's timeout; create(..., timeout=120) changes it for one call. For a stream it is the wait for the headers; after them a stream is cut off only by transfer_timeout seconds without a byte.

When an answer is lost after the server acted, the SDK recovers rather than failing:

• A bucket that turns out to exist on a retry is fetched.
• A delete or cancel whose object is gone on a retry succeeds.
• A rename whose source is gone on a retry returns the destination.
• A completed upload whose answer was lost is confirmed by comparing the object's ETag with the one computed locally.

Storage

Every route of the storage API is one method; the list is under Reference.

Upload

s = client.storage

# bytes, or a binary file from where it stands
s.upload("b", "data.bin", b"...", content_type="application/octet-stream", metadata={"owner": "ana"})

# progress, part size, parallelism
s.upload_file("b", "big.iso", "big.iso", part_size=64 << 20, concurrency=16,
              progress=lambda done, total: print(f"{done}/{total}"))

# resume: continue the newest unfinished upload of the same key
from teepin import UploadError
try:
    s.upload_file("b", "big.iso", "big.iso", resume=True)
except UploadError as e:
    print("left open to resume:", e.upload_id, e.__cause__)

# a stream of unknown size (a pipe, a socket; or an async iterable with AsyncClient)
import sys
s.upload_stream("b", "backup.tar", sys.stdin.buffer)

With resume=True, parts the server already holds are skipped when their size and MD5 match the file. If the upload fails, it is left open for 7 days to be resumed instead of being cancelled.

Download

s.download_file("b", "big.iso", "/tmp/big.iso")     # parallel ranges, temp file + rename
data = s.download_bytes("b", "small.json")           # into memory
with open("out.log", "wb") as f:
    s.download_stream("b", "app.log", f)               # in order, to any writable

with s.get_object("b", "app.log", range=(0, 1023)) as r:   # one request, streamed
    head = r.read()

A range that breaks off is continued from where it stopped. If the object is replaced mid-download, ObjectChangedError is raised. An object stored in one request is checked against its SHA-256 (ChecksumMismatchError).

Listing

page = s.list_objects("b", prefix="photos/", delimiter="/", limit=100)
page.objects, page.prefixes, page.next_cursor   # pass next_cursor as cursor= for the next page

for obj in s.iter_objects("b", "photos/"):      # every object, pages fetched as needed
    print(obj.key, obj.size)

ETags

The ETag of an object is its SHA-256; for an upload in parts it is the SHA-256 of the parts' SHA-256s with -<parts>. ETags are unquoted. To tell whether a local file matches an object:

from teepin import compute_etag, part_size_for

obj = s.stat_object("b", "big.iso")
if obj.part_count:
    # uploaded by upload() with the default part size
    local = compute_etag("big.iso", part_size_for(obj.size), obj.part_count)
else:
    local = compute_etag("big.iso")
same = local == obj.etag

For an object in parts, pass the part size it was uploaded with. part_size_for(size, want) gives the size upload picks.

Signed links

Every method sends your API key, so call them from a backend you control. To let a browser download or upload one object, sign a link and hand it only that:

link = s.presign_download("b", "report.pdf", ttl_seconds=900, attachment=True)   # up to 24 h
up = s.presign_upload("b", "avatars/ana.png", ttl_seconds=600, max_bytes=5 << 20, content_type="image/png")
# the browser sends: PUT up.url with up.headers, once

get_signed, head_signed and put_signed use such links the way a browser would, without the API key (handy for tests).

Errors

Every error is a teepin.TeepinError. API answers raise the subclass for their status and code. Each carries status_code, code (for example object_not_found), message and request_id (quote it to support).

Error	When
NotFoundError	404: bucket, object, upload
ConflictError	409: bucket_exists, bucket_not_empty, object_exists
PreconditionFailedError	412: if_match / if_none_match
TooLargeError	413, or object_too_large
NoCreditError	402 insufficient_credit
OnHoldError	402 storage_on_hold; delete_after says when held data goes
UnauthorizedError, ForbiddenError	401, 403 (e.g. a key without storage:write)
APIConnectionError	no answer: the network, a timeout
ObjectChangedError	the object was replaced during a download
ChecksumMismatchError	the bytes do not match the object's checksum
UploadError	an upload in parts failed; upload_id, cause in __cause__

S3 tools

Teepin Storage also has an S3-compatible endpoint, so code and tools written for S3 work with an S3 access key (created in the console; an API key does not work there):

import boto3
from botocore.config import Config

s3 = boto3.client("s3", endpoint_url="https://s3.teepin.com", region_name="teepin",
                  aws_access_key_id="...", aws_secret_access_key="...",
                  config=Config(s3={"addressing_style": "path"}))
s3.upload_file("big.bin", "my-bucket", "big.bin")

Tested with boto3 1.36; see examples/s3_boto3.py.

Inference

One OpenAI-compatible API for every model; the model is chosen per request. The API key needs the inference:invoke permission (and may be limited to some models).

from teepin import Client

client = Client(api_key="tpk_...")

for m in client.models.list():          # the models this key may call
    print(m.id, m.display_name, m.pricing.input_per_million_tokens, m.confidential)
    if m.attestation:                   # a confidential model not fully verified right now
        print("  ", m.attestation.level, m.attestation.warning)

reply = client.chat.completions.create(
    model="teepin/llama-3.3-70b",
    messages=[{"role": "user", "content": "What is a GPU?"}],
    max_tokens=200,
)
print(reply.choices[0].message.content)
print(reply.usage.total_tokens, reply.request_id, reply.attestation, reply.rate_limit)

Only the arguments you give are sent: max_tokens or max_completion_tokens, temperature, top_p, top_k, min_p, stop, seed, n, frequency_penalty, presence_penalty, repetition_penalty, logit_bias, logprobs, top_logprobs, tools, tool_choice, response_format, stream_options, user and chat_template_kwargs (for example {"enable_thinking": False}). extra_body={...} sends fields this SDK does not know yet. Messages are plain dicts, and content may be OpenAI content parts ({"type": "image_url", "image_url": {"url": ...}}).

Every result keeps the JSON it was read from in .raw, so a field the SDK does not know yet is still there (reply.raw["new_field"]).

Streaming

with client.chat.completions.create(
    model="teepin/llama-3.3-70b",
    messages=[{"role": "user", "content": "Write a haiku."}],
    stream=True,
    stream_options={"include_usage": True},   # usage on the last chunk
) as stream:
    print(stream.request_id, stream.attestation)  # known as soon as the headers arrive
    for chunk in stream:
        if chunk.choices:
            print(chunk.choices[0].delta.content or "", end="")
print(stream.usage)

The stream ends at [DONE]. If the model fails mid-stream, or the stream ends without [DONE], iterating raises StreamError (code is the server's, for example stream_error, or stream_ended_early). Leaving the with early closes the connection.

Attestation

A confidential model is served in a hardware-attested enclave. Each answer says how far the enclave was verified in attestation (the X-Teepin-Attestation header): verified, verified-older-release or not-verified, which is a warning, never an error. The proof itself, to check yourself:

proof = client.models.attestation("teepin/confidential-llama")   # a dict, as the provider gives it

It raises NotFoundError with code attestation_not_supported for a model that is not confidential, and model_not_found for one that does not exist or the key may not call.

Async

import asyncio
from teepin import AsyncClient

async def main():
    async with AsyncClient() as client:
        models = await client.models.list()
        reply = await client.chat.completions.create(
            model=models[0].id, messages=[{"role": "user", "content": "Hello"}]
        )
        print(reply.choices[0].message.content)

        stream = await client.chat.completions.create(
            model=models[0].id, messages=[{"role": "user", "content": "Hello"}], stream=True
        )
        async with stream:
            async for chunk in stream:
                print(chunk.choices[0].delta.content or "", end="")

asyncio.run(main())

Inference errors

Errors carry status, code, type, message, request_id and retry_after (seconds, when the server sent Retry-After).

Error	When
BadRequestError	400 invalid_request
UnauthorizedError	401 unauthorized
NoCreditError (also InsufficientCreditError)	402 insufficient_credit: add credit in the console
ForbiddenError (also PermissionDeniedError)	403 missing_scope (no inference:invoke), forbidden
NotFoundError	404 model_not_found, attestation_not_supported
TooLargeError	413 body_too_large
RateLimitError	429 rate_limit_exceeded, too_many_concurrent_requests, rate_limited
APIError	500 internal_error, 502 upstream_error, 503 model_unavailable, billing_unavailable
APIConnectionError	no answer: the network, a timeout (a chat request is not retried after a timeout)
StreamError	a stream that failed after it started

Compute

from teepin import CreateInstanceRequest, PortMapping

instance = client.instances.create(CreateInstanceRequest(
    name="pytorch-training", image="pytorch/pytorch:latest", gpu_vram="25GB",
    ports=[PortMapping(container=8080, public=80)],
))
client.instances.list()
client.instances.get(instance.id)
client.instances.metrics(instance.id)
client.instances.list_types()
client.instances.delete(instance.id)

Reference

client.storage (on AsyncClient, the same methods, awaited):

Method	Route
get_hold()	GET /v1/storage/hold
create_bucket(name)	POST /v1/storage/buckets
list_buckets()	GET /v1/storage/buckets
get_bucket(name)	GET /v1/storage/buckets/{bucket}
delete_bucket(name)	DELETE /v1/storage/buckets/{bucket}
list_objects(bucket, prefix, delimiter, cursor, limit), iter_objects(bucket, prefix)	GET .../objects
delete_objects(bucket, keys)	POST .../objects/delete
put_object(bucket, key, body, content_type, metadata, if_match, if_none_match)	PUT .../object
stat_object(bucket, key)	GET .../object
delete_object(bucket, key)	DELETE .../object
get_object(bucket, key, range, if_none_match, if_match)	GET .../object/content
head_object(bucket, key)	HEAD .../object/content
rename_object(bucket, from_key, to_key, overwrite)	POST .../object/rename
copy_object(bucket, key, source_bucket, source_key, if_match, if_none_match)	POST .../object/copy
presign_download(bucket, key, ttl_seconds, attachment)	POST .../object/download-url
presign_upload(bucket, key, ttl_seconds, max_bytes, content_type)	POST .../object/upload-url
create_upload(bucket, key, content_type, metadata)	POST .../uploads
list_uploads(bucket, prefix)	GET .../uploads
abort_upload(bucket, upload_id)	DELETE .../uploads/{id}
list_parts(bucket, upload_id)	GET .../uploads/{id}/parts
upload_part(bucket, upload_id, number, data)	PUT .../uploads/{id}/parts/{n}
complete_upload(bucket, upload_id, parts)	POST .../uploads/{id}/complete
get_signed(url, range), head_signed(url)	GET, HEAD /v1/storage/d/{token}
put_signed(upload, body)	PUT /v1/storage/u/{token}

client.models and client.chat.completions (on AsyncClient, awaited):

Method	Route
models.list()	GET /v1/models
models.attestation(model_id)	GET /v1/models/attestation?model=<id>
chat.completions.create(model, messages, stream, ...)	POST /v1/chat/completions

High level: upload, upload_file, upload_stream, download, download_file, download_bytes, download_stream. Helpers: teepin.compute_etag, teepin.part_size_for.

Development

pip install -e ".[dev]"
pytest                 # against an in-process fake of the API, with injected faults
black src tests

tests/e2e_drill.py runs the SDK against the real storage API; teepin-core's TestStorageAPI_PythonSDK drill starts it (see pkg/api/storage_python_sdk_drill_test.go).

Examples

• examples/storage_sync.py: buckets, a large upload with resume, a parallel download, listing, a signed link
• examples/storage_async.py: several uploads at once, an async stream, reading back
• examples/s3_boto3.py: boto3 against s3.teepin.com
• examples/inference_sync.py: models, a chat answer, a stream, an attestation proof
• examples/inference_async.py: several chat requests at once, an async stream
• examples/basic_sync.py, examples/basic_async.py: compute instances

License

Apache License 2.0