Skip to main content

A DAG library to manage hyperparameter search and ablations

Project description

hpdag

Usage:

from hpdag import DAG,Node,Branch
dataset = Node("dataset")
lr = Node("lr")
with DAG() as dag:
    datasets = Branch(
        dataset("the_pile") >> lr(0.001), #for example, one dataset might require specific settings than the others
        dataset( "c4") >> lr(0.01),
        )
    ablations = Branch( #do a type of ablation on each dataset
            Node("use_glu")(True,False), #run the experiment with and without the glu
            Node("positional_enc")("alibi","rotary"), #run the experiment with two different positional encodings
            )
    sizes = Node("size")("7b","3b") #run the experiment with two different sizes
    datasets >> ablations >>sizes
print(dag)

Output:

Task(dataset=the_pile, lr=0.001, use_glu=True, size=7b)
Task(dataset=the_pile, lr=0.001, use_glu=True, size=3b)
Task(dataset=the_pile, lr=0.001, use_glu=False, size=7b)
Task(dataset=the_pile, lr=0.001, use_glu=False, size=3b)
Task(dataset=the_pile, lr=0.001, positional_enc=alibi, size=7b)
Task(dataset=the_pile, lr=0.001, positional_enc=alibi, size=3b)
Task(dataset=the_pile, lr=0.001, positional_enc=rotary, size=7b)
Task(dataset=the_pile, lr=0.001, positional_enc=rotary, size=3b)
Task(dataset=c4, lr=0.01, use_glu=True, size=7b)
Task(dataset=c4, lr=0.01, use_glu=True, size=3b)
Task(dataset=c4, lr=0.01, use_glu=False, size=7b)
Task(dataset=c4, lr=0.01, use_glu=False, size=3b)
Task(dataset=c4, lr=0.01, positional_enc=alibi, size=7b)
Task(dataset=c4, lr=0.01, positional_enc=alibi, size=3b)
Task(dataset=c4, lr=0.01, positional_enc=rotary, size=7b)
Task(dataset=c4, lr=0.01, positional_enc=rotary, size=3b)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hpdag-0.2.tar.gz (3.0 kB view details)

Uploaded Source

Built Distribution

hpdag-0.2-py3-none-any.whl (3.2 kB view details)

Uploaded Python 3

File details

Details for the file hpdag-0.2.tar.gz.

File metadata

  • Download URL: hpdag-0.2.tar.gz
  • Upload date:
  • Size: 3.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.10

File hashes

Hashes for hpdag-0.2.tar.gz
Algorithm Hash digest
SHA256 5a7a2e086110c71f13ce9ae37f29f972e9292b590caa2e2b3dda8347369741f7
MD5 75299eea63f9cd84931b263caafac223
BLAKE2b-256 a27ff9b09a35da853908bb545f66a9209627971f48ab7941e28255d1455bb20a

See more details on using hashes here.

File details

Details for the file hpdag-0.2-py3-none-any.whl.

File metadata

  • Download URL: hpdag-0.2-py3-none-any.whl
  • Upload date:
  • Size: 3.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.10

File hashes

Hashes for hpdag-0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 71e63bf4f1b236a029b8fce2131d98bf54d8f228cddf8fa1ff53c57e9a2113a9
MD5 7fb0a6c29cdd2e27bccb5afa300142a2
BLAKE2b-256 6e51e111b54dfded7f6704bdee97ccdfb1e96c28e284171969472a002a1e3898

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page