Allows people without massive GPUs to easily run graphcast on remote runpod servers
Project description
Remote Graphcast
Run graphcast on a runpod GPU server. Output is saved to s3. Shouldn't cost more than $0.6 USD for a 10 day forcast.
Order of operations
- Your input is validated
- A secure runpod GPU pod is spun up on your runpod account
- Graphcast is installed into that gpu and forecasts of your chosen length are generated for each timestamp, this takes around 10 minutes for a 10 day forcast
- These forecasts are saved to your chosen s3 bucket, roughly 6.5GB for 10 days of forecast
- The runpod pod is terminated
- The program exits
Requirements
- python 3.8+ and pip
- cds.climate credentials
- an AWS s3 bucket, free tier should be fine
- S3 credentials to go with the bucket
- Runpod credentials
Installation
pip install remote-graphcast
Example Code
from remote_graphcast import remote_cast
remote_cast(
aws_access_key_id='YOUR_AWS_ACCESS_KEY_ID',
aws_secret_access_key='YOUR_AWS_SECRET_ACCESS_KEY',
aws_bucket='YOUR_AWS_BUCKET_NAME',
cds_url='https://cds.climate.copernicus.eu/api/v2', # this is probably your CDS URL
cds_key='YOUR_CDS_KEY',
forcast_list="[{'start': '2023122518', 'hours_to_forcast': 48}]",
# dates to forcast from, note the weird quasi-JSON format, of this string, use single quotes instead of double quotes
# select a date in the future and it will raise an error without spinning up anything
runpod_key='YOUR_RUNPOD_KEY',
gpu_type_id="NVIDIA A100-SXM4-80GB", # graphcast needs at least 61GB GPU ram (unless you want to quantize)
container_disk_in_gb=50, # you'll need around 40GB per 10 day forcast + a healthy 10GB buffer
)
# internally this function will keep polling the pod it spins up and the s3 bucket until it sees that all forcasts
# are complete, then it will return
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.