#floating-point
Every note tagged #floating-point, newest first — or browse the full archive.
Temperature zero is not determinism: your logits depend on who else is in the batch
Enumerating every ordering of a signed sum shows that reordering moves the result by one ulp — far too little to explain why temperature-zero endpoints return different paragraphs. The rest of the story is batch invariance, bfloat16, and argmax.
Two runs, two answers: floating-point sums aren't associative
Run the same simulation on four cores and eight and the total energy disagrees in the twelfth digit. That is not a bug in your code — it is floating-point addition refusing to be associative, and the fix is not to make the hardware lie but to know your noise floor.