Same question, different answers
Why a model can surprise you, even at temperature 0.
A model writes one token at a time, picking each by weighted chance. Temperature sets how adventurous those picks are.
Asked 12 times at each temperature: “Suggest a name for a new app that helps students practise coding interviews.”
temperature4 different answers
- CodePrep8
- CodeSprout2
- CodeCrest1
- CodeReps1
Real data from gemini-3.5-flash-lite, 2026-10-16
Even at temperature 0, the same question got 4 different answers in 12 tries. An agent that passed a test once has proven very little.
Your agent answered a test question correctly. What should you do before shipping?