Definition
An instruct model is a base model that has been further trained (via Supervised Fine-Tuning and/or RLHF) to understand and follow natural language instructions reliably. It responds to user requests rather than just completing text. Also called a "chat model" or "assistant model."
The Gap Between Base and Instruct
A base model trained to predict the next token knows a lot — but doesn't know it's supposed to be helpful. Fine-tuning on instruction-response pairs bridges this gap:
`
Base model sees: "Summarize this article: [article]..."
→ continues generating more article text
Instruct model sees: "Summarize this article: [article]..."
→ generates a concise summary
`
How Instruct Models Are Created
Step 1: Supervised Fine-Tuning (SFT)
- Collect a dataset of (instruction, ideal response) pairs
- Fine-tune the base model to reproduce the ideal responses
- Dataset is human-written or GPT-generated and human-verified
- Result: model learns the instruction-following format
- Human raters compare model outputs and rank them
- Train a reward model to predict human preferences
- Use PPO (or DPO, ORPO) to optimize the model toward higher-reward outputs
- Result: model becomes more helpful, honest, and harmless
- Instruct model: single-turn task execution focus
- Chat model: multi-turn conversational focus
- Persona definition: "You are an expert data scientist"
- Behavioral constraints: "Always respond in French"
- Safety rules: "Do not discuss competitors"
- Tool/context injection: "You have access to the following tools: ..."
- Base Model, Fine-Tuning, RLHF, System Prompt, User Prompt, Alignment, SFT
Step 2: RLHF / Preference Optimization (Optional but Standard)
Instruct Format (Chat Template)
Most instruct models use a structured prompt format:
`
<|system|>
You are a helpful assistant.
<|user|>
What is photosynthesis?
<|assistant|>
Photosynthesis is the process by which plants...
`
Each model family has its own chat template (ChatML, Llama-3, Mistral, Gemma, etc.)
Key Capabilities Gained Through Instruction Tuning
| Capability | Description |
|------------|-------------|
| Instruction following | Executes explicit user requests |
| Multi-turn conversation | Maintains context across turns |
| Formatting | Follows "respond in JSON", "use bullet points" |
| Role-playing | Adopts personas when instructed |
| Refusals | Declines harmful or out-of-scope requests |
| Conciseness | Answers the question without rambling |
Quality Dimensions of Instruct Models
| Dimension | Description |
|-----------|-------------|
| Helpfulness | Does it actually answer the question? |
| Harmlessness | Does it avoid dangerous outputs? |
| Honesty | Does it express uncertainty appropriately? |
| Instruction adherence | Does it follow all constraints in the prompt? |
| Format compliance | Does it match requested output format? |
Popular Instruct Models
| Model | Base | Notes |
|-------|------|-------|
| GPT-4o | GPT-4 (base) | OpenAI, closed |
| Claude 3.5 Sonnet | Claude (base) | Anthropic, via API |
| LLaMA 3.1 Instruct | LLaMA 3.1 | Meta, open weights |
| Mistral Instruct | Mistral | Open weights, efficient |
| Gemma 2 Instruct | Gemma 2 | Google, open weights |
Instruct Model vs. Chat Model
These terms are used interchangeably in practice. Some make a distinction:
In practice, most modern models are both.
System Prompt Role
Instruct models accept a system prompt that shapes their behavior:
Tradeoffs of Instruction Tuning
| Pro | Con |
|-----|-----|
| Much more useful for users | Can reduce raw capability vs. base |
| Safer behavior | May be overly cautious / refuse valid requests |
| Consistent format | May lose some creative/unexpected completions |
| Better at following constraints | Alignment tax on some benchmarks |