Large Language Models· Lesson 4 of 7
- 1How an LLM Works Inside8 min
- 2The Context Window — the Model's Short-Term Memory7 min
- 3Temperature & Sampling — Controlling Randomness7 min
- 4System vs User Messages — Who's Speaking7 min
- 5Choosing a Model — Bigger Isn't Always Better8 min
- 6What LLMs Can't Do7 min
- 7Cost & Latency — How Your Bill Adds Up7 min
System vs User Messages — Who's Speaking
In one line · A conversation is an array of role-tagged messages: system sets behavior and persona, user is the user's input, assistant is the model's reply.
Most behavior control happens in the system message — miss the roles and you'll try to steer from the wrong place.
Like a play: system is the actor's instruction sheet ('you're a polite assistant who answers in Arabic'), while user and assistant are the dialogue itself.
Like the briefing a call-center agent gets before a shift: those rules hold all day while customers (the user) come and go on top of them. To change the agent's behavior you edit the briefing, not nag each customer individually.
Put 'always answer in short bullets' in the system message and every later reply is bulleted — without repeating the request each turn.
When your app calls the API, it sends an array of messages, each tagged with a role. Before reaching the model, that array is flattened into one text stream with special markers separating the roles — and the model is trained to give system instructions more weight and authority than user content. So the separation isn't a hard security wall; it's a trained bias: strong most of the time, but craftily phrased user content can overcome it (that's the idea behind prompt injection). The assistant messages get recorded into history too, so the model sees its own past replies as context in later turns.
The array that actually gets sent — persistent rules in system, the changing user input in user:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a support agent for Nour ISP. "
"Answer in Egyptian Arabic, max 3 sentences. "
"Never promise a refund."},
{"role": "user", "content": "النت فاصل عندي من ساعتين، هتعوضوني إزاي؟"},
],
)🧪 Try it yourself — swap the system message and compare: take one user question ('explain recursion to me') and run it 3 times with 3 different system messages: (1) 'you teach 10-year-olds', (2) 'you are a strict tech-company interviewer', (3) no system message at all. Compare tone, depth, and length. Then try breaking a rule: put 'never use code examples' in system and see whether you can talk it out of that from the user side — you'll feel first-hand that the separation is a bias, not a wall.
- Stuffing everything into the user message. Persistent rules (persona, language, format, prohibitions) belong in system — in user messages they weaken and get forgotten as the chat grows.
- Trusting the system message as a security boundary. Instructions like 'never reveal the price' can be broken by a clever user message — real security rules are enforced in application code, not in the prompt.
- Forgetting the whole history is resent each turn. If you change the system message in a new release, live sessions carrying old history can behave differently than you expect.
Use the system message for anything that must hold across the whole conversation: persona and tone, language, output format, topic boundaries, and tool-handling instructions. Keep the user message for the variable content only: the user's question, the data, the text to process.
Don't rely on system messages to enforce security or legal rules — that's the job of code and guardrails outside the model. Don't let it balloon into an essay covering every eventuality: piles of conflicting instructions weaken compliance, and you pay for every token of it on every call. And data coming from a user or the internet belongs in a user message — never inject it into the system message.