aidsc 0.1.5

Package for Inferencing AI at DSC

AIDSC: Lightweight Classroom Inference Client

aidsc is a Python client designed for students and researchers to perform LLM inference on classroom clusters. It acts as a gateway to the Zeus control plane, allowing you to run models on high-performance classroom hardware from your local machine.

🚀 Key Features

• Classroom-Ready: Built specifically for inferencing classroom machines running LLMs.
• Seamless Remote Access: Automatic SSH tunneling via PAMD to bridge the gap to internal networks.
• Smart Connectivity: Checks for existing local tunnels before prompting for credentials.
• Persistent Credentials: Optional encrypted-style saving to .env (git-ignored automatically).
• Lightweight & Typed: Pydantic-validated payloads ensuring your requests always match the Zeus API.

---

🌐 Connectivity Logic

The client automatically resolves connectivity in this order:

1. Local Check: Checks localhost:32553. If healthy, it proceeds immediately.
2. Environment/Dotenv: Looks for PAMD_USER and PAMD_PASS in your system or .env file.
3. Interactive Setup: If credentials are missing or the port is closed, it clears the console and guides you through establishing an SSH tunnel.
4. Background Persistence: After establishing a tunnel, it asks if you'd like to keep it running in the background after your script exits.

---

🛠️ Usage Reference

1. Initialization

from aidsc import aidsc

client = aidsc(
    local_base_url="http://127.0.0.1:32553", # Override local Zeus URL
    infer_timeout=600.0,                      # Global timeout for inference calls (seconds)
    dotenv_path=".env"                       # Custom path to .env file
)

# Ensure connectivity (checks local port, prompts for SSH if needed)
client.ensure_ready()

2. Request Parameters

The LanguageInferenceRequest object allows fine-grained control over the job:

from aidsc import LanguageArgs, LanguageInferenceRequest

req = LanguageInferenceRequest(
    args=LanguageArgs(
        model="gpt-oss-20b",           # The target model name
        questions=["Recipe for rice?"], # List of prompts/questions
    ),
    max_new_tokens=512,                # Tokens to generate per response
    max_context_length=8192,           # Context window (alias: max_model_len)
    max_server_uptime=604800,          # How long the server can stay alive (seconds)
    total_concurrent_models=1,         # Instances of the model to run
    max_num_seqs=8                     # Max sequences for batching
)

3. Inference & Overrides

You can override specific parameters during the infer() call:

response = client.infer(
    req,
    server=117,             # Target a specific machine ID (e.g., 117)
    max_new_tokens=1024,    # Override the request's max_new_tokens
    client_timeout=300      # Specific timeout for this request
)

---

⚙️ Configuration Reference

Environment Variables

These can be set in your OS or in a .env file in your project root.

Variable	Default	Description
PAMD_USER	-	Your FSU/PAMD username (e.g., abc20z).
PAMD_PASS	-	Your PAMD password (used for sshpass).
aidsc_ZEUS_LOCAL_URL	http://127.0.0.1:32553	The local port where the tunnel maps.
aidsc_ZEUS_REMOTE_URL	http://144.174.11.196:32553	Internal IP of the Zeus orchestration server.

Client Settings (aidsc())

Parameter	Type	Description
local_base_url	str	URL to reach Zeus (default: aidsc_ZEUS_LOCAL_URL).
infer_timeout	float	Max time to wait for an inference result (default: 600.0).
dotenv_path	str	Custom path to load .env from.

Request Settings (LanguageInferenceRequest)

Field	Default	Description
max_new_tokens	512	Max tokens generated for each question.
max_context_length	8192	Total tokens (prompt + completion) allowed.
max_server_uptime	604800	Kill server after this many seconds of idle.
total_concurrent_models	1	Number of engine instances to spin up.
max_num_seqs	8	Maximum number of concurrent sequences.

---

🏗️ Project Architecture

• client.py: The high-level API. Handles the "Automatic SSH" logic, process detachment, and console UI.
• models.py: Pydantic definitions for all request types.
• connectivity.py: Low-level logic for checking local port health.
• zeus_client.py: The underlying HTTP client for the Zeus REST API.

---

🔒 Security Note

The client uses sshpass to handle password-based SSH connections to the PAMD jump host. Your .env file is automatically added to .gitignore when you run setup_interactive() to prevent accidental exposure of your credentials.