Tokens and cost

You pay per token, every time

Models read and write tokens: pieces of words. In English, a token is roughly 4 characters.

You pay for input tokens (everything you send) and output tokens (everything it writes), priced per million.

Here's the catch: since every call re-sends the whole conversation, call 10 pays again for calls 1 to 9. Cost grows much faster than the conversation does. Agents, which make many calls per task, feel this most.

Resending adds up

sent = 0
history = 0
for turn in range(3):
    history += 100
    sent += history
print(sent)

Each turn adds 100 tokens, and every call sends the whole history. How many tokens were sent in total?

• 300
• 600
• 100

Exercise: What did that cost?

First line: the price in dollars per million input tokens and per million output tokens.

Then n, then n lines new_input output for each call. Every call sends all earlier input and output again, plus its own new input.

Print input=X output=Y cost=$Z with the total tokens sent and received, and the cost to 4 decimal places.