Messages and roles

The model has no memory

A language model is a function: give it a list of messages, it returns the next message. That's all.

It remembers nothing between calls. A chat only feels continuous because your code sends the whole conversation again, every time.

Each message has a role: user (you, or your program) or assistant (the model). Instructions that apply to the whole conversation go in a separate system prompt.

Building a conversation

messages = []
messages.append({"role": "user", "content": "What's 2 + 2?"})
messages.append({"role": "assistant", "content": "4"})
messages.append({"role": "user", "content": "Times 10?"})
print(len(messages), messages[-1]["role"])
print([m["role"] for m in messages])

1. Line 1: A conversation starts as an empty list.
2. Line 2: The user's question.
3. Line 3: The model's reply goes in the list too, as role `assistant`.
4. Line 4: "Times 10?" only makes sense because the earlier messages are sent along with it.
5. Line 5: Three messages; the last one is the user's.
6. Line 6: Roles alternate: user, assistant, user.

Who spoke last?

history = [
    {"role": "user", "content": "Summarise this"},
    {"role": "assistant", "content": "Sure, paste it"},
    {"role": "user", "content": "Here it is"},
]
history.append({"role": "assistant", "content": "It says hi."})
print(len(history), history[1]["content"])

What does this print?

• 4 Sure, paste it
• 3 Sure, paste it
• 4 Summarise this

Exercise: Fix the turns

The API expects roles to alternate. When two messages in a row share a role, merge them into one.

Input: n, then n lines role|text. Print each message after merging as role: text, joining merged texts with / .