what's a transformer?

before transformers, sequence models (RNNs, LSTMs) processed tokens one at a time, which was slow to train and bad at long-range dependencies. the transformer replaces recurrence with self-attention: each token computes weighted relationships to every other token in the input in parallel. this is the “Attention Is All You Need” thesis.

the architecture is massively parallelizable on GPUs, which is the technical precondition for scaling to GPT-scale models. without parallelism, you can't train trillion-parameter models in any reasonable wall-clock time.

transformers now power GPT, Claude, Gemini, Llama (see LLM); and, through “diffusion transformers” (DiT), they also underlie Stable Diffusion 3, Sora, Veo, and most modern image / video generators (see diffusion model). vision transformers (ViT) replaced CNNs for many image-understanding tasks.

detection-side, most AI-text detectors are themselves fine-tuned transformers, scoring whether text looks like a transformer wrote it.

“transformer” is the architecture; LLM is what you get when you train a large transformer on text. the same architecture trained on pixels yields image models, on audio yields speech models.

self-attention has a quadratic cost in sequence length. context windows used to be small because of this, and long-context models (1M+ tokens) require workarounds like sparse attention, sliding windows, and mixture-of-experts routing.

most current AI detectors share architecture with what they're detecting. they're transformers trained to classify transformer output. when a new generator architecture ships, detectors lag until they retrain on the new style.

the common misconception is “transformers are how AI thinks.” they're a pattern-matching architecture optimized for next-token prediction. they don't reason, plan, or have beliefs in any rigorous sense. the appearance of reasoning is a learned behavior from training on large amounts of data.