Researchers discovered that language models fail on long conversations due to the removal of initial tokens, which act as "attention sinks" that stabilize attention distribution. Their solution, StreamingLLM, retains these tokens permanently, allowing models to process sequences of over 4 million tokens effectively. This approach has been integrated into major frameworks like HuggingFace and OpenAI's latest models.