Stopping safely
Read · 1 of 3
Who stops the loop?
while True trusts the model to say it's done. Usually it does. Sometimes it doesn't: it keeps searching for something that isn't there, or retries a failing tool again and again, and every call costs money.
Always cap the number of turns. When the cap is hit, stop cleanly and say so. A clear "I couldn't finish" beats a silent, expensive spin.
Read this lesson as text
Models can loop forever. Your code decides when enough is enough.
Who stops the loop?
while True trusts the model to say it's done. Usually it does. Sometimes it doesn't: it keeps searching for something that isn't there, or retries a failing tool again and again, and every call costs money.
Always cap the number of turns. When the cap is hit, stop cleanly and say so. A clear "I couldn't finish" beats a silent, expensive spin.
Which one can spin forever?
# A
while True:
reply = call_model(messages)
if reply.stop_reason == "end_turn":
break
run_tools(reply)
# B
for turn in range(10):
reply = call_model(messages)
if reply.stop_reason == "end_turn":
break
run_tools(reply)Which loop can run (and bill you) forever?
- A: it only stops when the model decides to
- B: range(10) is too small
- Neither: models always finish
Show the answer
A: it only stops when the model decides to. A has no limit of its own. If the model never returns end_turn, it never stops. B gives up after 10 calls, which is what production agents do (with a much more generous limit).
Exercise: A turn limit
Input: the turn limit, then a question. This stand-in model searches forever when the question contains forever.
Make the loop stop after that many model calls. If the model answered, print the answer; otherwise print Turn limit (N) reached without an answer.