What Happens Inside an LLM When You Type “Hello”?

You open ChatGPT, Claude, Gemini, or another AI assistant.You type: HelloYou hit Enter.A moment later, the AI responds: Hello! How can I help you today?It looks simple.But behind those few words, a surprisingly complex process has taken place. Your computer did not simply send the word “Hello” to a database and retrieve a pre-written answer. Instead, the text passed through several stages involving tokenization, embeddings, neural networks, attention, probabilities, and token generation.At the center of this process is the Transformer architecture, introduced in the 2017 research paper Attention Is All You Need. The paper proposed an architecture based on attention rather than recurrent or convolutional sequence processing, making it much more suitable for parallel computation.So, what actually happens after you type “Hello”?Let's follow the journey.1. It Starts With Your TextThe first thing you provide is ordinary human language.For example:Hello, how are you?Humans naturally see this as a sentence made up of words. An LLM does not.Computers ultimately work with numbers, so before the model can process your sentence, the text needs to be converted into a numerical representation.This begins with tokenization.2. Your Sentence Is Broken Into TokensAn LLM does not necessarily treat every word as one unit. Instead, it breaks text into smaller pieces called tokens.A token can be a complete word, part of a word, punctuation, or another frequently occurring piece of text. The exact tokenization depends on the model and tokenizer.For example, a sentence such as:Hello, how are you?might be represented by a tokenizer as something conceptually similar to:"Hello" → token "," → token " how" → token " are" → token " you" → token "?" → token The exact tokens and token IDs vary between models.This is important because the model does not directly process the word “Hello” as humans understand it. It processes a sequence of token IDs. You can think of tokenization as the entry gate into the LLM.Human language enters one side. Numbers come out the other.3. Tokens Become VectorsToken IDs are still not enough for a neural network to work with effectively. The model converts these tokens into numerical vectors called embeddings.You can think of an embedding as a long list of numbers representing information about a token.For example, instead of thinking:Hello the model might internally work with something conceptually like:[0.21, -0.14, 0.87, 0.03, ...] Real models use much larger vectors, so the actual representation is far more complex.The important idea is this:Text → Tokens → Numerical vectorsThese vectors are what the neural network actually processes.4. Now the Transformer Takes OverThis is where one of the most important ideas in modern AI appears:The Transformer.The Transformer architecture was introduced in 2017 in the paper Attention Is All You Need. The original paper was designed for sequence-to-sequence tasks such as machine translation, but its architecture later became extremely influential in language modeling.Modern LLMs such as GPT-style models use Transformer-based designs, although today's systems can differ substantially from the original 2017 Transformer.So what makes the Transformer so useful?The answer is Attention.5. Attention Helps the Model Understand ContextConsider this sentence:The animal didn't cross the street because it was too tired.What does “it” refer to?A human reader can usually understand that “it” refers to the animal. But for a machine, understanding relationships between different parts of a sentence is a difficult problem.This is where self-attention becomes important. Self-attention allows the model to examine relationships between tokens in the context. When processing one token, the model can assign different levels of importance to other tokens.For example:The → animal → didn't → cross → the → street → because → it → was → tired ↑ ↑ └──── relationship ──────┘ The model can learn that some tokens are more relevant to understanding “it” than others. This doesn't mean the model consciously “understands” the sentence like a human. Rather, its learned numerical operations produce representations that capture useful relationships between tokens.6. But What Does “Attention” Actually Do?This is where the mathematics comes in.Self-attention uses three important components:Query (Q)Key (K)Value (V)A simplified version of the attention calculation is:Attention(Q, K, V) = softmax(QKᵀ / √dₖ)V Don't let the equation scare you. The basic idea is relatively simple.You can think of:Query: “What information am I looking for?”Key: “What information do I contain?”Value: “What information should I provide if I am relevant?”The model calculates how strongly different tokens should interact with each other and then combines information accordingly.This allows the model to create a richer representation of each token based on its surrounding context.7. Why Multiple Attention Heads?A Transformer does not rely on just one attention calculation. It uses multi-head attention. You can imagine several attention heads looking at the same sentence from different perspectives.One head might learn relationships related to grammar. Another might focus on nearby words. Another might capture longer-range relationships. Another might learn patterns that are useful for meaning.These aren't manually assigned jobs. During training, the model learns useful patterns for its attention heads. This is one reason Transformers can represent complex relationships in language.8. How Did the Model Learn All of This?At this point, you might wonder:“Where did the model learn what words and sentences usually look like?”The answer is training. Before you ever type “Hello,” the model has already gone through a large-scale training process.One major stage is called pre-training.The basic task can be surprisingly simple:Predict the next token.For example:The sky is ___ The model might assign probabilities such as:blue → 0.72 clear → 0.12 dark → 0.05 green → 0.01 ... The model compares its prediction with the training data and adjusts its parameters. This happens again and again across enormous amounts of training data. Over time, the model learns statistical patterns in language. It doesn't memorize a dictionary of answers in the way a traditional database does.Instead, its parameters encode patterns learned during training.9. From a Base Model to an AI AssistantA model trained only to predict the next token isn't automatically a helpful chatbot.Imagine giving a base language model:Explain photosynthesis.It might continue the text rather than respond in the conversational format you expect. To make models more useful, additional training can teach them to follow instructions. This is commonly called instruction tuning or instruction fine-tuning.The training data can contain examples such as:User: Explain photosynthesis in simple English. Assistant: Photosynthesis is the process by which... By learning from many such examples, the model becomes better at following instructions and producing useful responses.This is one of the steps that helps turn a general language model into something that behaves more like an AI assistant.10. Now You Type “Hello”Let's return to our original example.You type:HelloThe system converts the text into tokens. Those tokens are converted into numerical representations. The Transformer processes them.Attention mechanisms help create contextual representations.The model then produces a probability distribution for what should come next.For example, conceptually:"!" → 0.35 "," → 0.20 " there" → 0.12 "!" ... → ... " How" → 0.08 " there!" → ... These numbers are only illustrative. The actual distribution is much larger and depends on the model, context, and system instructions. The model then selects a token.Suppose it chooses:!Now the sequence becomes:Hello!The model runs the generation process again to determine what should come next.Perhaps:HowThen:canThen:IThen:helpAnd so on. The answer is generated token by token. This is one of the most important things to understand about LLMs. The model doesn't generate the entire paragraph as one giant decision. It repeatedly predicts what token should come next.11. Where Do Temperature and Top-p Come In?There is another interesting part of generation. The model doesn't always have to choose the single highest-probability token. Generation can use sampling strategies that influence how predictable or varied the output is. Two commonly discussed settings are temperature and top-p.TemperatureTemperature controls how sharply the model's probability distribution is adjusted during sampling.A lower temperature generally makes the output more predictable. A higher temperature generally allows more variation.For example:Low temperature → safer / more predictable choices Higher temperature → more varied / unpredictable choices This doesn't mean a higher temperature automatically makes an answer more intelligent or creative. It simply changes the sampling behavior.Top-pTop-p, also called nucleus sampling, limits sampling to a set of tokens whose combined probability reaches a chosen threshold.For example, with a hypothetical top-p of 0.9, the model considers the smallest group of likely tokens whose cumulative probability reaches approximately 90%.The model can then sample from that group instead of considering extremely unlikely tokens. Together, techniques like these can influence the balance between predictable and varied generation.12. Why Doesn't the Model Just Give the Same Answer Every Time?Because generation can involve probability.Suppose the model considers:A → 60% B → 25% C → 10% D → 5% Always selecting A would produce deterministic behavior. Sampling can sometimes allow B or C to be selected as well. That means the same prompt can produce different responses depending on the model, settings, context, and other factors. This is one reason an LLM isn't simply a search engine with a giant database of pre-written answers.13. And This Is Where Hallucinations Come FromHere's an important consequence. An LLM is fundamentally trained to model patterns and generate likely continuations. That doesn't mean every generated statement is guaranteed to be factually correct.The model can produce a sentence that sounds extremely convincing while being wrong.For example, it might confidently provide:an incorrect date,a nonexistent research paper,a wrong statistic,or a fictional citation.Why?Because producing fluent language and verifying truth are different problems. This is why LLM outputs should be checked when accuracy matters. The model can be excellent at generating language while still making factual mistakes.14. So What Actually Happened When We Typed “Hello”?Let's summarize the entire journey.You type: "Hello" ↓ Tokenization ↓ Token IDs ↓ Embeddings ↓ Transformer ↓ Self-Attention ↓ Neural Network Layers ↓ Probability Distribution ↓ Token Sampling ↓ "Hello!" ↓ Next token ↓ "How can I..." ↓ More tokens ↓ Final response What looks like a simple conversation is actually a long chain of numerical operations happening extremely quickly.15. The Bigger PictureThe remarkable part is that the fundamental idea behind today's LLM revolution traces back to a 2017 paper.Attention Is All You Need introduced the Transformer architecture as a simpler architecture based on attention rather than recurrence and convolution for the sequence-transduction setting studied in that work.The original research was not written as a guide for today's chatbots. Yet the ideas introduced there became extremely influential in the development of modern language models.That is what makes the Transformer story so interesting. It is not simply a story about an old neural-network architecture.It is a story about how a relatively simple idea—allowing a model to efficiently consider relationships between elements of a sequence—helped shape the modern era of generative AI.Conclusion: The Next Time You Type “Hello”The next time you open an AI assistant and type:Helloremember that the model doesn't simply “read” your message and search for a response.Your text is converted into tokens.Those tokens become numerical representations.Transformer layers process them.Attention helps the model incorporate contextual relationships.The model calculates probabilities for possible next tokens.A generation process selects tokens one after another.And eventually, those tokens become the sentence you see on your screen.All of that happens between you pressing Enter and the AI saying hello back.That simple interaction is one of the most fascinating examples of how mathematics, computer science, and large-scale computing have come together to create modern AI.The next time an LLM says “Hello,” you will know that there is a lot more happening behind those five letters than meets the eye.