tfmx 0.6.3

Serve Transformers in one line

tfmx

Install

pip install tfmx --upgrade

Commands

Set GPU control state:

gpu_fan -cs a:1

Set GPU power limit:

# M-X
gpu_pow -pm a:1 && gpu_pow -pl 0:220

# M-A GPU-1/2
# gpu_pow -pm a:1 && gpu_pow -pl a:220
gpu_pow -pm a:1 && gpu_pow -pl 0,1:160

Set GPU fan speed:

gpu_fan -fs a:100

Set GPU monitor with curve:

# gpu_mon -c "a:30-50/50-65/60-80/75-100;3,7:25-100" -s

Run tei compose and machine:

tei_compose up && sleep 20 && tei_machine run --perf-track

Run tei benchmark:

tei_benchmark -E "http://localhost:28800" -n 1000000 run