Fine-tuning
Continuing to train a pre-trained model on your specific data to teach it a domain, style or task. Less common now that prompting and RAG are stronger.
· Reviewed by senior engineers
Fine-tuning takes a pre-trained model and continues training it on your data. The goal varies: teach a specific style or tone, encode domain knowledge the base model lacks, adapt the model to a structured output format, or improve performance on a narrow task where prompting plateaus. Techniques range from full fine-tuning (expensive, large data needed) to parameter-efficient methods like LoRA and QLoRA that update only small adapter weights.
The pendulum has swung. In 2022 fine-tuning was the default for anything serious. By 2024-2026 it's a tool to reach for sparingly because prompting, RAG and bigger base models have closed most of the gap at a fraction of the operational cost. Fine-tuning still wins for narrow style adaptation, when you must run smaller cheaper models in production, or when you need to encode behaviour that prompts can't reliably express.
The pitfalls are catastrophic forgetting (the model loses general capability while learning your task), data quality (garbage in, fine-tuned garbage out), and operational drag (every new base model means re-fine-tuning). Evals matter even more after fine-tuning than before because the failure modes are subtle.
Devinsta recommends starting with prompting and RAG, escalating to fine-tuning only when there's a clear ceiling you can't break and a maintenance budget to keep the model current.