Reproducibility in LLM inference stumbles not because models themselves change, but because of how we do the math under the hood. Everyone points to floating-point non-associativity plus thread scheduling on GPUs—the idea that (a + b) + c can differ from a + (b + c) if threads arrive out of order. Yet when you run the same matrix multiply on a GPU thousands of times, you get bit-identical results. That tells us the usual “concurrency + floating point” story isn’t the whole picture. Digging into GPU kernels reveals the real culprit: nondeterministic algorithm choices inside libraries like cuBLAS or cuDNN. These libraries pick between different reduction or tiling methods at runtime—sometimes based on workspace size, sometimes on hardware generation. Each method sums floating-point values in a slightly different order. Even if each kernel is itself deterministic once chosen, the library’s internal logic can pick A-then-B today and B-then-A tomorrow, so your end-to-end LLM call still varies. To fix this, you must pin down every moving part. That means setting environment variables (for example, CUBLAS_WORKSPACE_CONFIG) or library flags to force a single, deterministic algorithm. In PyTorch you’ll also turn on torch.use_deterministic_algorithms(True) and seed CUDA’s RNG. Once you lock algorithm selection and eliminate any atomic-add races, running the same prompt through ChatGPT-style inference yields the exact same tokens every time. No more surprises, just reproducible outputs.