Visualizing the Paths Not Taken
It’s easy to judge a large language model (LLM) by the one answer it gives to you. It’s also misleading. From all the options the LLM has to complete a given prompt, every prompt has the option to fan out into dozens of good or bad continuations. We usually see just one: the “winner” picked by the Token Sampler. The rest vanish into a black box.
This project was born from a simple question: what is the complete set of token paths the model didn’t go?
How the Token Sampler works
From a given input prompt, the LLM generates a probability value for every token in its vocabulary. This value indicates how good of a choice the token is as the next token. Since most of those probalities are close to zero, only a small list of tokens remain for final consideration.
In the end, one token must be selected - but it’s not necessarily the one with the highest probability.
There’s an element of randomness, so some token we’d consider a better choice might actually be discarded.
The graph of all possible options
The graph captures all the final candidates, including the selected token with their probabilities. It further explores the “could have beens”, by following every token path. This way, we can browse and evaluate the otherwise invisible responses, that the LLM discarded.
It provides a great way to understand how LLMs generate their outputs.
Here are some interactive vizualizations for different prompts and models. You can clearly see issues with specific models and prompts, that otherwise would be incomprehensible.
You can expand/collapse nodes, hover, zoom and pan.
Example Prompt #1 “A watch gear train consists of” - Vizualization qwen3 8b - Vizualization Granite 3.1 - Vizualization phi 3.5
Example Prompt #2 “Respond with only one single word! Is 13 a prime number?” - Vizualization qwen3 8b - Vizualization Granite 3.1 - Vizualization phi 3.5
Example Prompt #3 “On a beautiful day, the sky is” - Vizualization qwen3 8b - Vizualization Granite 3.1 - Vizualization phi 3.5
Example Prompt #4 “What is the root of 18225 and the root of 250047?” - Vizualization qwen3 8b - Vizualization Granite 3.1 - Vizualization phi 3.5
What do the different color codings mean?
| Viz Element | What it tells you |
|---|---|
| Green Border Dot | The token that was selected out of all its siblings |
| Red Border Dot | Token that was not selected, albeit having higher probability |
| Orange Border Dot | Token with lower, but still reasonably high relative probability, but also not chosen |
| Yellow Border Dot | Token with lower relative probability, also not chosen |
| Grey Border Dot | Token with overall low probability, not chosen |
| Light Blue Filled Dot | Can be expanded to next token sampling run |