Large language models today resemble Leonard Shelby in Memento: they store vast facts in frozen weights but can’t form new memories after training. To compensate, developers use chat histories, retrieval systems and system prompts—external scaffolding that sits outside the model’s core. These tactics work for tasks where the answer already exists in data, but they fall short in novel discovery, adversarial settings or when knowledge is too tacit to phrase in prompts. True learning requires updating the model’s parameters, compressing new information into its inner wiring rather than juggling it externally. Research on continual learning aims to give models that capability. Instead of relying solely on in-context tricks, engineers would let models adjust their weights post-deployment. Meanwhile, non-parametric advances like state space models (SSMs) extend context windows far beyond a few thousand tokens. By interleaving memory layers and attention heads, labs hope to push coherent reasoning from a 20-step agent loop to tens of thousands of steps. But even an infinite context window is just a gigantic filing cabinet—it retrieves but never integrates. Updating weights remains the only way for a system to truly “know” something new.