In the early days, engineers used recurrent neural networks (RNNs) to process text one token at a time, passing each output back into the network. Variants like LSTM and GRU added gates to hold onto important information longer, but two limits remained: you couldn’t parallelize across tokens, so training time scaled linearly with sequence length, and gradients either vanished or exploded as they propagated through long chains. Those issues left models blind to long-range dependencies in a text. Bahdanau’s 2014 paper grafted an attention mechanism onto RNNs, letting the model look back at all prior tokens when generating each new one. It improved results but still suffered from sequential execution. Then in 2017, Vaswani et al. dropped recurrence altogether. Their transformer used self-attention layers that compute relationships between every token pair simultaneously via matrix multiplications. That change removed the sequential bottleneck and long-range decay at once, letting hardware exploit massive parallelism. The original paper, “Attention Is All You Need,” targeted translation, but its architecture—stacks of identical encoder and decoder blocks topped by a prediction head—proved broadly useful. Google split the transformer in two. The encoder-only version became BERT in 2018, trained to predict masked words in enormous text corpora. It showed deep contextual understanding and quickly powered Google Search’s ranking backend. On the other side, decoder-only models drove generative work. OpenAI’s GPT-1 in mid-2018 combined a transformer decoder with a two-step recipe: massive unsupervised pretraining on unlabeled text followed by fine-tuning on smaller, task-specific data. GPT-2 arrived in 2019 with ten times more parameters and training data; GPT-3 in 2020 pushed scale another hundredfold and demonstrated zero-shot and few-shot task performance, often bypassing explicit fine-tuning altogether. The shift from sequential RNNs to fully parallel transformers underpins modern LLMs. Self-attention lets models track dependencies across thousands of tokens in one pass. Encoders excel at understanding and feature extraction, decoders at generation. And the pretrain-then-fine-tune paradigm has become standard, with model size and data scale driving improvements across translation, summarization, question answering and beyond.