A DAG library to manage hyperparameter search and ablations
Project description
hpdag
Usage:
from hpdag import DAG,Node,Branch
dataset = Node("dataset")
lr = Node("lr")
with DAG() as dag:
datasets = Branch(
dataset("the_pile") >> lr(0.001), #for example, one dataset might require specific settings than the others
dataset( "c4") >> lr(0.01),
)
ablations = Branch( #do a type of ablation on each dataset
Node("use_glu")(True,False), #run the experiment with and without the glu
Node("positional_enc")("alibi","rotary"), #run the experiment with two different positional encodings
)
sizes = Node("size")("7b","3b") #run the experiment with two different sizes
datasets >> ablations >>sizes
print(dag)
Output:
Task(dataset=the_pile, lr=0.001, use_glu=True, size=7b)
Task(dataset=the_pile, lr=0.001, use_glu=True, size=3b)
Task(dataset=the_pile, lr=0.001, use_glu=False, size=7b)
Task(dataset=the_pile, lr=0.001, use_glu=False, size=3b)
Task(dataset=the_pile, lr=0.001, positional_enc=alibi, size=7b)
Task(dataset=the_pile, lr=0.001, positional_enc=alibi, size=3b)
Task(dataset=the_pile, lr=0.001, positional_enc=rotary, size=7b)
Task(dataset=the_pile, lr=0.001, positional_enc=rotary, size=3b)
Task(dataset=c4, lr=0.01, use_glu=True, size=7b)
Task(dataset=c4, lr=0.01, use_glu=True, size=3b)
Task(dataset=c4, lr=0.01, use_glu=False, size=7b)
Task(dataset=c4, lr=0.01, use_glu=False, size=3b)
Task(dataset=c4, lr=0.01, positional_enc=alibi, size=7b)
Task(dataset=c4, lr=0.01, positional_enc=alibi, size=3b)
Task(dataset=c4, lr=0.01, positional_enc=rotary, size=7b)
Task(dataset=c4, lr=0.01, positional_enc=rotary, size=3b)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hpdag-0.2.tar.gz
(3.0 kB
view details)
Built Distribution
hpdag-0.2-py3-none-any.whl
(3.2 kB
view details)
File details
Details for the file hpdag-0.2.tar.gz
.
File metadata
- Download URL: hpdag-0.2.tar.gz
- Upload date:
- Size: 3.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5a7a2e086110c71f13ce9ae37f29f972e9292b590caa2e2b3dda8347369741f7 |
|
MD5 | 75299eea63f9cd84931b263caafac223 |
|
BLAKE2b-256 | a27ff9b09a35da853908bb545f66a9209627971f48ab7941e28255d1455bb20a |
File details
Details for the file hpdag-0.2-py3-none-any.whl
.
File metadata
- Download URL: hpdag-0.2-py3-none-any.whl
- Upload date:
- Size: 3.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 71e63bf4f1b236a029b8fce2131d98bf54d8f228cddf8fa1ff53c57e9a2113a9 |
|
MD5 | 7fb0a6c29cdd2e27bccb5afa300142a2 |
|
BLAKE2b-256 | 6e51e111b54dfded7f6704bdee97ccdfb1e96c28e284171969472a002a1e3898 |