llmwalk 0.1.0

Explore likely completions as a prompt tree.

llmwalk

Explore the answer-space for any prompt and any MLX-supported model.

TODO: gif

Instead of sampling from the possible tokens each step, llmwalk branches out and completes all of the branches the sampler would consider based on --top-k, --top-p and --temperature, ranking the results by probability as it goes.

The tree is walked prioritising the most likely branches, until it finds -n branches and then it stops. It doesn't enumerate all possibilities, just enough to know for sure it has found the -n most likely branches.

Usage

• uvx llmwalk -p "In what year was Barack Obama born?"
• uvx llmwalk -p "Write a haiku about compilers" -n 5
• uvx llmwalk -p "Give me one word: " --top-k 200 --temperature 0.7

Options

• -p, --prompt TEXT: Prompt to score (wrapped with the model’s chat template).
• -m, --model MODEL: MLX-LM model identifier or path (default: mlx-community/Llama-3.2-1B-Instruct-4bit).
• -n N: Number of answers to show. The search stops once it has N finished answers and no unfinished branch can beat the worst of those N.
• --min-probability FLOAT: Any branch whose cumulative probability falls below this is marked finished (low_probability) and not expanded further.
• --top-k INT: At each step, expand at most k next tokens (highest probability).
• --top-p FLOAT: Nucleus cutoff applied within the top-k tokens at each step (keep adding tokens until cumulative probability ≥ p).
• --temperature FLOAT: Softmax temperature applied when computing per-step probabilities (1.0 is the model distribution; must be > 0).
• --stats-interval SECONDS: How often to refresh the live view (<= 0 disables periodic refresh; still renders at start/end).