A CREATURE THAT GUESSES THE NEXT WORD
Meet the
Word Oracle.
It doesn't understand.
It only dreams what comes next.
SCROLL TO GO DEEPER • EVERYTHING IS ALIVE
BEGIN THE JOURNEY ↓
LEVEL 1 — THE ONLY THING IT EVER DOES
The model does not "know" words.
It only produces numbers that add up to 100%.
This is the entire foundation of how every LLM works.
THE PROMPT YOU GIVE IT
Change it and watch the math move
Raw Scores (what the model really thinks before math)
LOGITS
These numbers can be any positive or negative value. They are meaningless by themselves.
After the Mathematics
PROBABILITIES • SUM = 1.000
Sum of all probabilities = 1.000000
TEMPERATURE (how sharp or fuzzy the math becomes)
1.00
0.1 → very confident (peaky)
2.5 → very creative (flat)
THE ACTUAL MATH THAT HAPPENS (Softmax)
probability(word) = exp( logit(word) / temperature ) / Σ exp( all_logits / temperature )
This is the complete core idea. Everything else in LLMs is just making these numbers better.
LEVEL 2 • THE ACTUAL ALGORITHM
We don't just "break" words.
We run statistics until we have a useful vocabulary of numbers.
Tiny Training Corpus (edit me)
Current Vocabulary (learned merges)
PAIR FREQUENCIES (the actual counts the algorithm uses)
Type any word → see it turned into numbers
CURRENT TOKENIZATION (with learned merges)
Integer IDs the model actually receives:
THE MATH: At every step we count how many times every adjacent pair of symbols appears.
We merge the pair with the highest count. This is repeated until we reach desired vocabulary size.
The final mapping turns any text into a sequence of integers.
LEVEL 3 • MEANING BECOMES SPACE
Every word lives somewhere in a hidden universe.
Words that feel similar live near each other. The creature discovered this on its own.
The Constellation of Meaning DRAG THE WORDS
Do the famous trick
LIVE VECTOR MATH (when you use the buttons)
Drag words to explore geometry • Buttons show the actual arithmetic
LEVEL 4 • THE CORE MECHANISM
This is the actual calculation that lets every word look at every other word.
YOUR SENTENCE (edit to see the math change)
Q, K, V for each token (tiny 4D vectors for demo)
Step 1: QKᵀ scores (raw alignment)
Step 2: Scaled by 1/√dₖ (prevents softmax from becoming too sharp)
Step 3: Softmax → Attention weights
Temperature 1.0
Step 4: Output = Attention weights × V
Visual summary of where each token is looking
LEVEL 5 • DECISIONS ARE NOT CHOICES
It never picks one word.
It grows a whole field of possibilities.
The Possibility Field
DRAG THE WEATHER SLIDER
The capital of France is
TEMPERATURE — THE WEATHER OF THE DREAM
LEVEL 6 • HOW THE CREATURE BECAME WISE
We showed it millions of books
and said "guess the next word" over and over.
The Learning Garden
Every time it guesses wrong, it adjusts its dreams slightly.
LOSS: 2.84
LEVEL 7 • THE INNER MACHINE
This is what actually happens
inside every time it thinks.
You are now controlling the real mechanisms. Everything here is mathematically faithful.
The Living Transformer
TOKENS ARRIVE
They become vectors in 4096-dimensional space
ATTENTION FLOWS
Every position looks at every other position with learned eyes
FEED-FORWARD DREAM
Each position thinks deeply about what it saw
PROBABILITY RIVER
The final shape of what comes next
YOU HAVE NOW TOUCHED THE ACTUAL MACHINE THAT POWERS EVERY MODERN AI
Every time you use ChatGPT, Claude, Gemini, or any modern AI, this exact process (with much bigger vectors and hundreds of layers) is happening in milliseconds.
YOU MADE IT TO THE BOTTOM
You didn't read about LLMs.
You played with them.
That feeling you have right now? That's what it feels like to actually understand something.