Allows people without massive GPUs to easily run graphcast on remote runpod servers
Project description
Remote Graphcast
Run graphcast on a runpod GPU. Output is saved to s3. Shouldn't cost more than $0.2 for a 10 day forcast.
Order of operations
- Input is validated
- A secure runpod GPU pod is spun up on your account
- Graphcast is installed into that gpu and forcasts of your chosen length are generated for each timestamp, this takes around 10 minutes for a 10 day forcast
- These forcasts are saved to your chosen s3 bucket, roughly 6.5GB for 10 days of forcast
- The runpod pod is terminated
- The program exits
Requirements
- python 3.10+ and pip
- cds.climate credentials
- an s3 bucket
- S3 credentials to go with the bucket
- Runpod credentials
Installation
Example Code
from remote_graphcast import remote_cast
remote_cast(
aws_access_key_id='YOUR_AWS_ACCESS_KEY_ID',
aws_secret_access_key='YOUR_AWS_SECRET_ACCESS_KEY',
aws_bucket='YOUR_AWS_BUCKET_NAME',
cds_url='https://cds.climate.copernicus.eu/api/v2', # this is probably your CDS URL
cds_key='YOUR_CDS_KEY',
forcast_list="[{'start': '2023122518', 'hours_to_forcast': 48}]",
# dates to forcast from, note the weird quasi-JSON format, of this string, use single quotes instead of double quotes
# select a date in the future and it will raise an error without spinning up anything
runpod_key='YOUR_RUNPOD_KEY',
gpu_type_id="NVIDIA A100-SXM4-80GB", # graphcast needs at least 61GB GPU ram
container_disk_in_gb=50, # you'll need around 40GB per 10 day forcast + a healthy 10GB buffer
)
# internally this function will keep polling the pod it spins up and the s3 bucket until it sees that all forcasts
# are complete, then it will return
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
remote-graphcast-0.0.7.tar.gz
(7.5 kB
view hashes)