Full fine-tune vs LoRA for specialist work

Who this page is for

Read this when someone wants to “do a proper full fine-tune” so the project looks thorough, or when a LoRA run already exists and a sponsor wants to spend more. Typical readers are a platform owner staring at checkpoint size, a specialist who has a written bar the adapter already meets, and a finance partner who has not yet heard the forgetting risk. This is not a recipe for either method. It will not publish an llm.build accuracy gap.

If you do not yet have labelled pairs, read supervised fine-tuning. If you want the adapter handover, read LoRA. If memory is the only reason the base will not fit, read QLoRA. This URL owns the fork: when touching every weight has a written reason, and when it is extra cost.

Default to LoRA when the language is already there

Most insurance and retail jobs need format, terminology and restraint, not a new foundation model. Hu et al. built LoRA for that shape: freeze the pretrained weights, train low-rank updates, hand over a small artefact. You can keep the same base for more than one adapter. You spend less GPU memory than a full update. That is the default for specialist work on this site.

Default is not a quality ranking we will publish. Default means you need a written reason to leave it. “We have budget left” is not a reason. “The vendor demo used a full update” is not a reason. “Full sounds more serious in a steering pack” is not a reason.

QLoRA sits on the LoRA side

Dettmers et al. published QLoRA as a 4-bit frozen base plus LoRA adapters so a large model fits on less GPU memory. That is a memory story. It is not a third quality tier above LoRA and below a full update. If VRAM is the issue, read the QLoRA page. If the issue is “should we touch every weight?”, you are on this page.

People collapse the three acronyms into a ladder. Refuse the ladder. LoRA and QLoRA are adapter methods. A full update is a different artefact, a different GPU bill, and a different forgetting conversation. Write which one you mean.

A full update needs a written reason

Full fine-tuning can still be in scope when LoRA cannot move a written metric and the task is stable enough to justify the cost. It uses more memory, risks more forgetting of behaviour you still need, and produces a heavier artefact. You write a new copy of the base (or most of it). Operators then have to store, licence, and incident that copy.

The written reason should name the failed LoRA metric, the unused pack that showed the miss, and what a full update is supposed to move. If you cannot name those three, you are spending the larger run to look thorough. We will not open with a full fine-tune for that motive.

• Failed LoRA on a named metric, with unused-test notes attached.
• A domain shift the frozen base cannot absorb with adapters.
• Budget for compute, forgetting risk, and a heavier handover.
• A serving plan that can actually host the new copy of the base.

What changes in the run

A full update writes a new copy of the base (or most of it). Checkpoint size, GPU memory and forgetting risk all go up. A LoRA run writes adapter files and leaves the foundation frozen; merge only if the operating plan is a single task. QLoRA writes the same kind of adapter against a 4-bit load of that frozen base. None of those paths change the checkpoint licence or where inference runs.

If LoRA already meets the unused-test bar, the full update is extra cost. If LoRA failed that bar, write the failed metric down before you spend the larger run. A full update that overfits a hundred letters will still fail unused work. Touching every weight is not regularisation for bad labels.

Insurance: LoRA already meets the letter bar

Retail: failed LoRA on stubborn attributes

Forgetting is part of the bill

A full update can move a stubborn task and can also erase behaviour you still need: restraint on exceptions, format on a second product, language the frozen base already had. Adapters localise the change. That is one reason they are the default. If you still choose a full update, write which behaviours must not regress and how you will notice.

We will not invent a forgetting number. We will ask for unused test documents that cover the old behaviours as well as the new ones. If you only score the stubborn attribute and ignore the letter format you already shipped, you will discover the regression in production.

Extra cost has a shape, not a vibe

When LoRA already meets the letter bar, the extra cost of a full update is not abstract. You pay more GPU hours. You store a larger image. You accept a wider blast radius if the run overfits. You spend specialist time re-checking behaviours the adapter already held. That time is usually better spent on the next document family or on unused SKUs you do not have yet.

When LoRA failed fibre and care-code on a written metric, the extra cost can be justified. Write the miss, write the budget, write the forgetting checks. Then the larger run is a scoped job. Without those three sentences it is still a prestige spend, just wearing a failed-adapter story.

“Make it ours” is not a method

Sponsors reach for a full update because a heavier checkpoint feels like ownership. Ownership is the licence, the operator, and the unused-test notes. A new copy of someone else’s weights is still someone else’s grant if you distribute. Read the family page. Do not buy forgetting risk to feel like the model has changed nationality.

If the real need is a file you can take into an air-gapped box, a merged LoRA image can be that file. If the real need is several specialists on one base, separate adapters are that file. Neither need is a reason to touch every weight. Write the need in the operating plan first. The method is a consequence of that plan, not a way to look like the plan exists.

Licence and serving do not move with the method

Updating every weight does not buy you a new licence. A full Llama fine-tune is still a derivative of the Llama materials if you distribute it. An Apache Qwen full update still wants notices. Read the family page. This comparison will not reprint those clauses.

Serving a full update is a larger image. Serving LoRA is an adapter plus a base, or a merge. Write the serving plan in the same pack as the reason for the method. Operators who only receive a mystery blob cannot tell a full update from a merge from a vendor export.

When a full update is the right conversation

Use it when LoRA failed a written bar on unused work, the task is stable, you accept cost and forgetting risk, and someone will operate the heavier artefact. Then the run is a different training job with a different handover. It is not a trophy.

Skip it when LoRA already meets the bar, when the labels still disagree, when memory was the only issue (use QLoRA), or when the brief is only “make it ours”. Those are reasons to keep adapters or to go back to readiness.

Handover for either path

For LoRA, the pack is the adapter, the base revision, load or merge recipe, licence line, and unused-test notes. For a full update, the pack is the new checkpoint, the base it started from, the failed-LoRA metric that justified the spend, forgetting checks, licence line, and unused-test notes. Include the sentence that explains why adapters were not enough — or why they were, and you did not spend the larger run.

If you cannot show that sentence, you do not have a method handover. You have a larger file. We will ask for the reason before we treat a full update as an engagement artefact. A heavier checkpoint without the failed-adapter metric is extra cost wearing a filename. Write the metric or keep the adapter. Prestige is not a metric.

Sources

1. Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (Accessed September 2026). LoRA freezes the pretrained weights and trains low-rank update matrices, reducing the number of trainable parameters for adaptation.
2. Dettmers et al., QLoRA: Efficient Finetuning of Quantized LLMs (Accessed September 2026). QLoRA stores the frozen base model in 4-bit precision and trains LoRA adapters, with NF4, double quantisation and paged optimisers as the published method.

Questions we hear first