worldjen 0.1.0

WorldJen Python SDK for running evaluation pipelines on video and world models

WorldJen Python SDK

Python SDK for running WorldJen evaluation on video and world models.

Requires Python 3.10+.

Configuration

Set your API key and options via worldjen.config() or environment variables:

• WORLDJEN_API_KEY — API key (required for run creation and uploads)
• WORLDJEN_VIDEO_DIR — Directory for cached prompts and run outputs (default: ~/.worldjen/data/videos)

import worldjen

worldjen.config(api_key="your-api-key")
# or use env vars and call config() only to override
worldjen.config(video_dir="/path/to/videos")

Usage

Dimensions

Use the Dimensions enum for evaluation dimensions:

from worldjen import Dimensions

dims = [Dimensions.SUBJECT_CONSISTENCY, Dimensions.SCENE_CONSISTENCY]
# or strings: dims = ["subject_consistency", "scene_consistency"]

Running a pipeline

Your pipeline must be either:

• A callable: pipeline(prompt, **kwargs) -> list[frames]
• An object with a generate method: pipeline.generate(prompt, **kwargs) -> list[frames]
• An object with an infer method: pipeline.infer(prompt, **kwargs) -> list[frames]

Frames can be PIL Images or numpy arrays (HWC, any dtype supported by the SDK’s video writer).

import worldjen

def my_pipeline(prompt, **kwargs):
    # Generate frames from prompt (text-to-video)
    ...
    return frames  # list of PIL or numpy arrays

result = worldjen.run(
    my_pipeline,
    dimensions=[worldjen.Dimensions.SUBJECT_CONSISTENCY],
    run_name="my-eval-run",
    model_id="my-org/my-model-checkpoint-xyz"
    wait_for_evals=True,
    # optional pipeline kwargs:
    # num_frames=16,
)

print(result.run_id, result.status, result.video_paths, result.eval_results)
if result.error_message:
    print("Error:", result.error_message)

On any failure (run creation, pipeline error, upload), the run is marked failed and RunResult is returned with status="failed" and error_message set.

API

• worldjen.config(api_key=..., video_dir=..., api_url=..., timeout=...) — Set or override config; returns the config object.
• worldjen.run(pipeline, dimensions, run_name=None, wait_for_evals=True, **pipeline_kwargs) — Creates a run, generates videos, uploads videos, then optionally poll until evals are ready. Returns RunResult(run_id, video_paths, output_dir, eval_results, status, error_message).