Skip to main content

Execute Python and Shell code on remote GPU sessions

Project description

Clouditia SDK

Execute Python and Shell code on remote GPU sessions.

Clouditia SDK provides a simple Python interface to run code on remote GPU-powered containers. Perfect for machine learning, deep learning, and any GPU-accelerated workloads.

PyPI version Python 3.7+ License: MIT

Installation

pip install clouditia

Quick Start

from clouditia import GPUSession

# Connect to your GPU session
session = GPUSession("ck_your_api_key")

# Execute Python code on the remote GPU
result = session.run("""
import torch
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"GPU: {torch.cuda.get_device_name(0)}")
""")

print(result.output)

Features

  • Python Execution: Run Python code on remote GPUs
  • Shell Commands: Execute shell commands on the GPU pod
  • Variable Transfer: Send and retrieve variables between local and remote
  • Async Jobs: Submit long-running tasks with real-time log monitoring
  • Jupyter Magic: Use %%clouditia magic in notebooks
  • Decorator Support: Use @session.remote to run functions on GPU

Table of Contents

  1. Getting Your API Key
  2. Basic Usage
  3. Executing Python Code
  4. Shell Commands
  5. Variable Transfer
  6. Remote Functions (Decorator)
  7. Async Jobs (Long-Running Tasks)
  8. Jupyter Magic
  9. Error Handling
  10. API Reference

Getting Your API Key

  1. Log in to clouditia.com
  2. Start a GPU session
  3. Go to API Keys in your session dashboard
  4. Generate a new API key (starts with ck_ or sk_)

Basic Usage

Connect to a Session

from clouditia import GPUSession

# Create a session with your API key
session = GPUSession("ck_your_api_key_here")

# Verify the connection
info = session.verify()
print(f"Connected to: {info['session_name']}")
print(f"GPU: {info['gpu_type']}")
print(f"Credit remaining: {info['user_credit']}€")

Using the connect() Function

from clouditia import connect

session = connect("ck_your_api_key")
result = session.run("print('Hello from GPU!')")

Executing Python Code

Simple Execution

# Run Python code and get the output
result = session.run("print('Hello from the GPU!')")
print(result.output)  # "Hello from the GPU!"

# Check if execution was successful
if result.success:
    print("Code executed successfully!")
else:
    print(f"Error: {result.error}")

Getting Return Values

# The last expression is captured as result
result = session.run("2 + 2")
print(result.result)  # "4"

result = session.run("[i**2 for i in range(5)]")
print(result.result)  # "[0, 1, 4, 9, 16]"

Multi-line Code

result = session.run("""
import torch
import torch.nn as nn

# Create a simple model
model = nn.Linear(10, 5).cuda()
x = torch.randn(32, 10).cuda()
output = model(x)

print(f"Input shape: {x.shape}")
print(f"Output shape: {output.shape}")
print(f"Model parameters: {sum(p.numel() for p in model.parameters())}")
""")

print(result.output)

Using exec() for Side Effects

# exec() is for code that doesn't need a return value
session.exec("import torch")
session.exec("model = torch.nn.Linear(10, 5).cuda()")
session.exec("optimizer = torch.optim.Adam(model.parameters())")

Shell Commands

Execute shell commands on the remote GPU pod:

# List files
result = session.shell("ls -la /workspace")
print(result.output)

# Check current directory
result = session.shell("pwd")
print(result.output)

# Create directories and files
result = session.shell("mkdir -p /workspace/models && ls /workspace")
print(result.output)

# Chain multiple commands
result = session.shell("cd /workspace && mkdir data && ls -la")
print(result.output)

# Check disk space
result = session.shell("df -h")
print(result.output)

# Check memory
result = session.shell("free -h")
print(result.output)

# Install packages
result = session.shell("pip install transformers datasets")
print(result.output)

# Download files
result = session.shell("wget https://example.com/data.zip -O /workspace/data.zip")
print(result.output)

Checking Exit Codes

result = session.shell("ls /nonexistent")
print(f"Exit code: {result.exit_code}")
print(f"Success: {result.success}")

Variable Transfer

Sending Variables to GPU

# Send local data to the remote session
data = [1, 2, 3, 4, 5]
session.set("my_data", data)

# Use it in remote code
session.run("print(f'Data: {my_data}')")
session.run("print(f'Sum: {sum(my_data)}')")

Retrieving Variables from GPU

# Compute something on the GPU
session.run("""
import torch
tensor = torch.randn(100, 100).cuda()
result_stats = {
    'mean': tensor.mean().item(),
    'std': tensor.std().item(),
    'shape': list(tensor.shape)
}
""")

# Get the result locally
stats = session.get("result_stats")
print(f"Mean: {stats['mean']:.4f}")
print(f"Std: {stats['std']:.4f}")
print(f"Shape: {stats['shape']}")

Sending Complex Objects

import numpy as np

# Send numpy arrays
arr = np.random.randn(100, 100)
session.set("numpy_array", arr)

# Send dictionaries
config = {
    "learning_rate": 0.001,
    "batch_size": 32,
    "epochs": 100
}
session.set("config", config)

# Use in remote code
session.run("""
import torch
tensor = torch.from_numpy(numpy_array).cuda()
print(f"Learning rate: {config['learning_rate']}")
""")

Remote Functions (Decorator)

Use the @session.remote decorator to run functions on the GPU:

from clouditia import GPUSession

session = GPUSession("ck_your_api_key")

@session.remote
def compute_on_gpu(data, power=2):
    import torch
    tensor = torch.tensor(data, device='cuda', dtype=torch.float32)
    result = tensor ** power
    return result.cpu().tolist()

# Call the function - it runs on the remote GPU!
result = compute_on_gpu([1, 2, 3, 4, 5], power=2)
print(result)  # [1.0, 4.0, 9.0, 16.0, 25.0]

Remote Function with Model

@session.remote
def train_step(batch_data, learning_rate=0.01):
    import torch
    import torch.nn as nn

    # Create model (or load from checkpoint)
    model = nn.Sequential(
        nn.Linear(len(batch_data), 64),
        nn.ReLU(),
        nn.Linear(64, 1)
    ).cuda()

    optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

    # Training step
    x = torch.tensor(batch_data, dtype=torch.float32).cuda()
    output = model(x)
    loss = output.sum()

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    return {"loss": loss.item()}

# Call it like a normal function
result = train_step([1.0, 2.0, 3.0, 4.0], learning_rate=0.001)
print(f"Loss: {result['loss']}")

Async Remote Functions

@session.remote(async_mode=True)
def long_training():
    import torch
    for epoch in range(100):
        print(f"Epoch {epoch}/100")
        # ... training code ...
    return {"status": "completed"}

# Returns an AsyncJob instead of waiting
job = long_training()
print(f"Job submitted: {job.job_id}")

# Wait for completion
result = job.wait(show_logs=True)

Async Jobs (Long-Running Tasks)

For tasks that take hours or days, use async jobs:

Submitting a Job

# Submit a long-running job
job = session.submit("""
import torch
import time

print("Starting training...")
for epoch in range(100):
    print(f"Epoch {epoch + 1}/100")
    time.sleep(1)  # Simulate training

print("Training complete!")
torch.save({'epoch': 100}, '/workspace/checkpoint.pt')
""", name="my_training")

print(f"Job ID: {job.job_id}")

Monitoring Progress

import time

# Poll for status
while not job.is_done():
    status = job.status()
    print(f"Status: {status}")

    # View recent logs
    if status == "running":
        logs = job.logs(tail=10)
        print(logs)

    time.sleep(30)

print("Job finished!")

Real-Time Log Streaming

# View logs as they come in
while job.is_running():
    new_logs = job.logs(new_only=True)
    if new_logs.strip():
        print(new_logs, end='')
    time.sleep(5)

Waiting for Completion

# Wait with live log output
result = job.wait(show_logs=True)

# Or wait with timeout
try:
    result = job.wait(timeout=3600)  # 1 hour max
except TimeoutError:
    print("Job taking too long, cancelling...")
    job.cancel()

Getting Results

# Get the final result
result = job.result()

if result.success:
    print("Job completed successfully!")
    print(result.output)
else:
    print(f"Job failed: {result.error}")

Listing Jobs

# List all jobs
jobs = session.jobs()
for j in jobs:
    print(f"{j.name}: {j.status()}")

# List only running jobs
running_jobs = session.jobs(status="running")

# List completed jobs
completed_jobs = session.jobs(status="completed", limit=5)

Cancelling Jobs

if job.is_running():
    job.cancel()
    print("Job cancelled")

Shell Jobs

# Submit a shell command as an async job
job = session.submit(
    "pip install transformers && python /workspace/train.py",
    name="install_and_train",
    job_type="shell"
)

Jupyter Magic

Use Clouditia directly in Jupyter notebooks with magic commands.

Loading the Extension

# In a Jupyter cell
%load_ext clouditia

# Set your API key
CLOUDITIA_API_KEY = "ck_your_api_key"

Running Code on GPU

%%clouditia
import torch
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"GPU: {torch.cuda.get_device_name(0)}")

x = torch.randn(1000, 1000, device='cuda')
y = torch.randn(1000, 1000, device='cuda')
z = torch.matmul(x, y)
print(f"Result shape: {z.shape}")

Specifying API Key Directly

%%clouditia ck_your_api_key
print("Hello from GPU!")

Async Mode in Jupyter

%%clouditia --async
for epoch in range(100):
    print(f"Epoch {epoch}")
    # ... training code ...

# The job is submitted and _clouditia_job variable is set
# Check job status
_clouditia_job.status()

# View logs
print(_clouditia_job.logs())

Utility Magic Commands

# Check session status
%clouditia_status

# List recent jobs
%clouditia_jobs

# List only running jobs
%clouditia_jobs running

Error Handling

The SDK provides specific exceptions for different error types:

from clouditia import (
    GPUSession,
    ClouditiaError,
    AuthenticationError,
    SessionError,
    ExecutionError,
    TimeoutError,
    CommandBlockedError
)

session = GPUSession("ck_your_api_key")

try:
    result = session.run("some_code()")
except AuthenticationError:
    print("Invalid API key")
except SessionError:
    print("Session not running or not accessible")
except ExecutionError as e:
    print(f"Code execution failed: {e}")
except TimeoutError:
    print("Execution timed out - consider using async jobs")
except CommandBlockedError:
    print("Command blocked by security filters")
except ClouditiaError as e:
    print(f"General error: {e}")

Using raise_for_status()

result = session.run("some_code()")
result.raise_for_status()  # Raises ExecutionError if failed
print(result.output)

API Reference

GPUSession

GPUSession(
    api_key: str,
    base_url: str = "https://clouditia.com/code-editor",
    timeout: int = 120,
    poll_interval: int = 5
)

Methods:

Method Description
verify() Verify API key and get session info
run(code, timeout=None) Execute Python code
exec(code, timeout=None) Execute without return value
shell(command, timeout=None) Execute shell command
set(name, value) Send variable to remote
get(name) Retrieve variable from remote
submit(code, name=None, job_type="python") Submit async job
jobs(status=None, limit=10) List jobs
gpu_info() Get GPU information
remote(func) Decorator for remote functions

ExecutionResult

ExecutionResult(
    output: str,      # stdout output
    result: Any,      # last expression value
    error: str,       # error message if failed
    exit_code: int,   # process exit code
    success: bool     # True if successful
)

Methods:

Method Description
raise_for_status() Raise exception if failed
to_dict() Convert to dictionary

AsyncJob

AsyncJob(session, job_id, name=None)

Methods:

Method Description
status() Get current status
is_done() Check if finished
is_running() Check if running
is_pending() Check if pending
logs(tail=50, new_only=False) Get logs
result() Get final result
cancel() Cancel the job
wait(timeout=None, show_logs=False) Wait for completion
get_info() Get detailed job info

Configuration

Environment Variables

You can set the API key via environment variable:

export CLOUDITIA_API_KEY="ck_your_api_key"
import os
from clouditia import GPUSession

session = GPUSession(os.environ["CLOUDITIA_API_KEY"])

Custom Base URL

session = GPUSession(
    "ck_your_api_key",
    base_url="https://custom.clouditia.com/code-editor"
)

Timeouts

# Set default timeout (seconds)
session = GPUSession("ck_your_api_key", timeout=300)

# Or per-request
result = session.run("long_computation()", timeout=600)

Examples

Training a Neural Network

from clouditia import GPUSession

session = GPUSession("ck_your_api_key")

# Submit training job
job = session.submit("""
import torch
import torch.nn as nn
import torch.optim as optim

# Create model
model = nn.Sequential(
    nn.Linear(784, 256),
    nn.ReLU(),
    nn.Linear(256, 10)
).cuda()

optimizer = optim.Adam(model.parameters(), lr=0.001)
criterion = nn.CrossEntropyLoss()

# Training loop
for epoch in range(10):
    # Simulated batch
    x = torch.randn(64, 784).cuda()
    y = torch.randint(0, 10, (64,)).cuda()

    optimizer.zero_grad()
    output = model(x)
    loss = criterion(output, y)
    loss.backward()
    optimizer.step()

    print(f"Epoch {epoch+1}/10, Loss: {loss.item():.4f}")

# Save model
torch.save(model.state_dict(), '/workspace/model.pt')
print("Training complete!")
""", name="mnist_training")

# Wait with live logs
result = job.wait(show_logs=True)

Data Processing Pipeline

# Create workspace
session.shell("mkdir -p /workspace/data /workspace/output")

# Download data
session.shell("cd /workspace/data && wget https://example.com/data.csv")

# Process data
result = session.run("""
import pandas as pd

# Load and process data
df = pd.read_csv('/workspace/data/data.csv')
print(f"Loaded {len(df)} rows")

# Process...
df_processed = df.dropna()
print(f"After cleaning: {len(df_processed)} rows")

# Save
df_processed.to_csv('/workspace/output/processed.csv', index=False)
print("Saved to /workspace/output/processed.csv")
""")

print(result.output)

Support


License

MIT License - see LICENSE for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

clouditia-1.0.0.tar.gz (24.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

clouditia-1.0.0-py3-none-any.whl (21.4 kB view details)

Uploaded Python 3

File details

Details for the file clouditia-1.0.0.tar.gz.

File metadata

  • Download URL: clouditia-1.0.0.tar.gz
  • Upload date:
  • Size: 24.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for clouditia-1.0.0.tar.gz
Algorithm Hash digest
SHA256 b12978c6d85c85d16e28d81c7b37e3732ce2f78fb2239f6c733cf8502a2f11b8
MD5 a05b8d289a93903d6ceb73cfa9833203
BLAKE2b-256 c4caf6451427b5e150878732ef98f24d4629438507c19ed95e030a99619c57be

See more details on using hashes here.

File details

Details for the file clouditia-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: clouditia-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 21.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for clouditia-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 51e282eb6e8e04ee98221c00bd63074c8546dcd0470451cf80670c1a4a3eed12
MD5 dd5658de827c6b7883bb359f3e76504c
BLAKE2b-256 947ebe37fe0dfe0ad72015f6947efb3171fd2d2fee0be12132eace3eb058e49c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page