Fine-tuning
Fine-tuning adapts a model's behavior using examples, usually to improve style, format, or narrow task performance.
Use fine-tuning when you need consistent behavior across many similar tasks, not when you only need new facts.
Use when
- Stable output format
- Repeated domain-specific classification
- Tone and style adaptation
- High-volume tasks where prompt length is expensive
Avoid when
- Frequently changing private knowledge
- One-off product experiments
- Missing document retrieval
- Unclear evaluation criteria
What fine-tuning changes
Fine-tuning is best understood as behavior shaping. It teaches a model to respond in a more consistent way for a repeated task: classification, extraction, support tone, formatting, or domain-specific language.
It is not a good first answer for changing facts. If the issue is that the model lacks current product documentation, RAG or long context is usually simpler and easier to update.
Before fine-tuning
You need examples, a clear target behavior, and an evaluation set. Without that, fine-tuning turns into expensive guessing. A strong prompt baseline should exist first, because it gives you a reference point for measuring improvement.
Common mistakes
- Fine-tuning to add facts that change every week.
- Starting without an evaluation set.
- Using low-quality examples from inconsistent human output.
- Ignoring the operational cost of retraining.
Next decision
If you need better behavior, compare fine-tuning with prompt engineering and structured output. If you need better knowledge, compare it with RAG.