System Prompts, Temperature, and Model Settings: Controlling LLM Behavior
Prompt text is only half of control. Message roles and decoding parameters decide whether your feature feels precise or creative—and whether it stays on policy. Product teams should treat these settings as part of the prompt version.
System vs developer vs user messages
- System / developer — durable rules: persona, safety, output schema, tool policy
- User — the end-user request or task input
- Assistant — prior model replies in multi-turn chat
Put invariants in the system prompt. Putting critical safety only in the user message is fragile when users override instructions.
Temperature
Higher temperature increases randomness among likely tokens.
- 0–0.2 — extraction, classification, deterministic transforms, most production tools
- 0.3–0.7 — drafting, brainstorming, UX copy variants
- 0.8+ — creative exploration; rare in business workflows
If outputs must match a schema, low temperature plus constrained decoding beats “please be consistent” alone.
Top-p (nucleus sampling)
Top-p limits sampling to a probability mass. Many teams either tune temperature or top-p—not both aggressively. Start with defaults; change one knob at a time while evaluating.
Max tokens and stop sequences
- Set max tokens to cap cost and runaway answers
- Use stop sequences for custom delimiters in older completion APIs
- For chat APIs, structure with clear section headers instead of exotic stops when possible
System prompt design tips
- Short, prioritized rules (models overweight early and late content—still keep noise low)
- Explicit tool and refusal policies
- Output contract first if the feature is machine-consumed
- Version the system prompt in git next to the feature flag
Multi-turn memory hygiene
Long chats accumulate contradictions. Summarize state into a structured “working memory” object and resend it instead of infinite raw history when building agents.
Recommended starting defaults
| Use case | Temp | Notes |
|---|---|---|
| Data extraction | 0 | JSON schema + validation |
| Support draft replies | 0.3 | Style examples in few-shot |
| Marketing variants | 0.7 | Human picks winner |
| Code generation | 0–0.2 | Tests required |
Frequently asked questions
Why did the model ignore my system prompt?
Long user content can dominate; conflicting instructions exist; or the model is weaker at instruction hierarchy. Shorten, reorder, and reinforce critical rules near the task.
Work with Auroviq
Auroviq (AuroviQ) builds production AI features, agent workflows, and LLM integrations for product companies in the UK, Netherlands, Singapore, and India. We combine prompt design with architecture, evals, and secure delivery—not demos that break in production.
Work with Auroviq — custom software & AI for product teams.