Transformers and Attention in Plain Words

Almost every modern language model is a transformer, and the key idea inside a transformer is called attention.

The Problem Attention Solves

Read this sentence: "Ali could not lift the bag because it was heavy."

You know at once that "it" means the bag, not Ali. To understand one word, you looked at other words in the sentence. Older AI models read text one word at a time from left to right and often lost track of earlier words in long sentences.

In 2017, researchers introduced the transformer. Its main trick: for every word, the model can look at all the other words at the same time and decide which ones matter most. That is attention.

Attention as a Classroom

Imagine every word is a student in a classroom.

• Each student asks a question: "Who here helps explain me?" This is the query.
• Each student also wears a name badge describing what they offer. This is the key.
• Each student holds some information to share. This is the value.

A student compares their question with everyone's badge. The better the match, the more attention they pay to that student's information. Then they mix what they heard into a new, better understanding of themselves.

Attention Weights

The model gives each word a score, then turns the scores into weights that add up to 1 using a function called softmax. Here are made-up scores for how much "it" should look at each word:

import numpy as np

def softmax(scores):
    e = np.exp(scores - np.max(scores))
    return e / e.sum()

words = ["Ali", "could", "not", "lift", "the", "bag", "because", "it", "was", "heavy"]
# Made-up scores: how much the word "it" should look at each word
scores = np.array([1.2, 0.1, 0.2, 0.6, 0.1, 2.8, 0.2, 0.3, 0.1, 0.9])
weights = softmax(scores)
for w, a in zip(words, weights):
    print(f"{w:<8} {a:.2f} " + "#" * round(a * 40))
print("Total:", round(weights.sum(), 2))

Ali      0.11 ####
could    0.04 #
not      0.04 ##
lift     0.06 ##
the      0.04 #
bag      0.53 #####################
because  0.04 ##
it       0.04 ##
was      0.04 #
heavy    0.08 ###
Total: 1.0

More than half of the attention goes to "bag". In a real model, these scores are not typed by hand. They are learned during training.

The Real Calculation, Small Size

Inside a transformer, each token is a vector. The model multiplies it by three learned matrices to get the query, key and value vectors. Then:

1. Score = query of one token compared (dot product) with the key of every token
2. Divide by the square root of the vector size (this keeps numbers stable)
3. Softmax turns scores into weights
4. The new vector = weighted mix of all value vectors

import numpy as np

def softmax(x):
    e = np.exp(x - x.max(axis=-1, keepdims=True))
    return e / e.sum(axis=-1, keepdims=True)

np.random.seed(0)
tokens = ["I", "love", "Karachi"]
X = np.random.rand(3, 4)          # 3 tokens, each a vector of 4 numbers

Wq, Wk, Wv = (np.random.rand(4, 4) for _ in range(3))
Q, K, V = X @ Wq, X @ Wk, X @ Wv  # queries, keys, values

scores = Q @ K.T / np.sqrt(K.shape[1])
weights = softmax(scores)          # each row adds up to 1
output = weights @ V               # a new, context-aware vector per token

print("Attention weights:")
print(np.round(weights, 2))
print("Output shape:", output.shape)

Attention weights:
[[0.29 0.23 0.48]
 [0.29 0.23 0.48]
 [0.29 0.22 0.49]]
Output shape: (3, 4)

Each row shows how much one token looks at the three tokens. Here the matrices are random, so the weights do not mean anything yet. Training adjusts these matrices over billions of examples until the attention becomes useful.

The Rest of the Transformer

A full model repeats a block many times (dozens of layers). Each block has:

At the end, the model turns the last token's vector into a probability for every possible next token, and one is chosen. For how that choice works (temperature and sampling), see How LLMs Generate Text. For neural networks in general, see Neural Networks Simply Explained.

Why This Matters to You

• Attention costs grow with length. Comparing every token with every other token is why long inputs are slower and more expensive.
• Placement matters. Put your most important instructions clearly, near the start or the end, not hidden in the middle of a long document.
• The model works on tokens, not facts. It is very good with patterns in language, which is both its power and the reason it can sound right while being wrong.

Exercise