Overview
Parameter-efficient finetuning, or PEFT, is really about a simple question: when adapting a large pretrained model, do we actually need to move all of the weights? In many cases, the answer is no. A pretrained model already contains a lot of general language structure and world knowledge, so downstream adaptation often looks less like rewriting the whole model and more like injecting a compact task-specific update.
That is the framing I found most useful in this lecture. Full supervised finetuning changes the entire parameter space, while PEFT methods try to preserve the base model and only learn a lightweight mechanism for steering it toward a new task. The technical differences across adapters, prefix tuning, LoRA, and QLoRA all come from where that extra trainable capacity is inserted and how the update is represented.
Why move beyond prompting
The lecture starts from the usual critique of prompt-based learning: it can be inefficient, brittle, sensitive to phrasing, and unclear as a long-term adaptation strategy. Prompting is often enough for quick experimentation, but it is a weak substitute for actually teaching a model how to behave on a task. Instruction tuning matters because it aligns the model more directly with the behavior we want rather than relying on increasingly clever prompt engineering.
This distinction feels important in practice. If a model is consistently weak at domain-specific question answering, summarization, or structured generation, then better prompts may help at the margins, but they do not really solve the adaptation problem. Finetuning does. The harder question is just how much of the model we need to finetune.
Figure 1. From pretraining to finetuning: a large model is first trained on broad data, then adapted to a narrower downstream task with smaller task-specific data.
SFT versus PEFT
In full supervised finetuning, every parameter in the pretrained model can be updated. Formally, if the pretrained model has parameters (W), then finetuning simply optimizes all of (W) on a downstream objective. This gives the model maximum flexibility, but it is also the most expensive option in both memory and compute, and it becomes increasingly impractical as model size grows.
PEFT changes the setup by freezing most or all of the original parameters and only optimizing a much smaller set of new or selected parameters. So instead of learning a full new model, we learn a constrained adaptation on top of the old one. Conceptually, this means the pretrained weights continue to store general capabilities, while the new trainable mechanism captures the task-specific deviation.
I think this is the cleanest way to read the full finetuning versus PEFT distinction: full finetuning says the whole model is plastic, whereas PEFT says the base model is mostly fixed and adaptation should be expressed through a narrow channel.
Adapter tuning
Adapters are probably the most literal example of inserting trainable structure directly into the model architecture. Instead of changing the existing transformer weights everywhere, we splice in small neural modules, usually after the multi-head attention block, the feed-forward block, or both. The original model remains frozen, and only these inserted modules are trained.
Technically, the usual adapter design is a bottleneck MLP. A hidden state (h) is first projected down into a lower-dimensional space, passed through a nonlinearity, and then projected back up to the original hidden size. The adapter output is then merged back with the main residual stream. So the model architecture is literally augmented with an extra trainable path, but the path is deliberately small so that the parameter count stays low.
What I like about adapters is that they make the PEFT idea very concrete. The model does not overwrite its original knowledge; it keeps the original computation and learns a side module that nudges the representation at specific points in the network. If I imagine adapting a base LLM for policy memo writing, the adapter is not retraining the whole system to speak policy language from scratch. It is learning a narrow correction layer that reshapes intermediate representations toward that style and task.
Figure 2. Adapter tuning inserts small bottleneck modules into the transformer block while keeping the pretrained backbone frozen.
Prefix tuning
Prefix tuning also adds trainable parameters without fully changing the pretrained weights, but the mechanism is different. Instead of inserting a small module inside the transformer block, prefix tuning learns a sequence of continuous vectors that function like virtual tokens. These vectors are prepended to the model’s internal computation and condition how later tokens are processed.
So in a technical sense, prefix tuning does not just append extra text to the input the way prompt engineering does. It learns trainable prefix states that the model attends to, which means the adaptation happens inside the hidden representation space rather than in ordinary discrete language. The base model is frozen, but its attention behavior is altered because these learned prefixes become part of the context the model sees.
This is why prefix tuning feels like an intermediate case between prompting and architectural modification. It does not edit the original transformer weights, but it still changes the model’s internal computation by inserting learned vectors into the processing pipeline. If I wanted one base summarization model to produce either short executive summaries or more technical research summaries, different learned prefixes could steer those behaviors without requiring separate full finetuned models.
Figure 3. Prefix-style conditioning can be understood through the attention mechanism: learned prefix representations alter the context that later tokens attend to, steering a frozen model without updating its main weights.
LoRA
I replicate the LoRA paper here.
LoRA takes a more algebraic route. Instead of inserting a standalone module like an adapter, it targets the weight update itself. Suppose a pretrained linear layer has weight matrix (W). In full finetuning, we would directly learn a dense update (\Delta W). LoRA assumes that this update can be well-approximated by a low-rank decomposition, so instead of learning (\Delta W) freely, it learns (\Delta W = BA), where (A) and (B) are much smaller matrices with low rank.
This means LoRA does not replace the original weight matrix. It keeps (W) frozen and injects a trainable low-rank path alongside it. Operationally, the layer behaves like (Wx + BAx): the frozen pretrained transformation still does most of the work, and the low-rank term adds a compact task-specific correction. That is why LoRA is best understood not as a full new module inserted into the architecture, but as a constrained parameterization of the update itself.
I think LoRA is elegant because it reframes adaptation as a structural claim about the update space. It says the downstream task does not need an arbitrary full-rank perturbation of the original model; a low-dimensional adjustment is often enough. For a writing assistant tuned to my own note-taking style, that means the model’s general linguistic competence can remain intact while a small low-rank update captures preferences about tone, terminology, and response format.
In practice, LoRA is often attractive not just because it is theoretically parameter-efficient, but because it may require updating only around 1 to 10 percent of parameters instead of the full model. Another practical detail is that LoRA is typically attached to specific linear projections inside the transformer, such as the QKV projections, output projections, and MLP layers, rather than being applied uniformly to every parameter. So the low-rank update is not only constrained in rank but also targeted in where it is inserted.
Figure 4. LoRA represents a large weight update with the product of much smaller low-rank matrices, reducing the number of trainable parameters needed for adaptation.
QLoRA
QLoRA keeps the LoRA update mechanism but compresses the frozen base model even more aggressively. The lecture emphasizes quantizing the transformer weights to 4-bit precision and using paged optimizers to manage memory. In other words, the base model becomes cheaper to store and run during finetuning, while the trainable part is still the LoRA adapter rather than the full original model.
The important point here is that QLoRA is not a different answer to the question of how to represent the update. It still relies on low-rank adaptation. What changes is the systems setup around it: the frozen backbone is quantized so that very large models can be finetuned under tighter hardware constraints. That is why QLoRA feels less like a new conceptual family and more like a practical extension of LoRA for limited-memory settings.
Figure 5. QLoRA keeps LoRA-style low-rank updates but stores the frozen backbone in 4-bit precision, which sharply reduces memory requirements during finetuning.
Distillation
Distillation solves a different problem from the PEFT methods above. Adapters, prefix tuning, LoRA, and QLoRA are mostly about how to adapt a large model cheaply. Distillation is about how to transfer capability from a large teacher model into a smaller student model so that deployment becomes cheaper.
The teacher-student framework matters because the most expensive part of LLM use is often inference rather than training alone. A large model may be acceptable for experimentation, but too slow or costly for production. Distillation tries to preserve useful behavior while shrinking the deployed model. For language models, this is especially interesting because some reasoning ability can transfer surprisingly well even when the student is much smaller.
How I now separate the methods
At this point, the distinctions feel much clearer to me on the technical side. Adapters insert small trainable modules directly into the transformer architecture. Prefix tuning inserts trainable continuous prefixes that alter the model’s internal context and attention behavior. LoRA does not insert a full new block in the same way; instead, it parameterizes the weight update itself as a low-rank matrix product attached to existing linear layers. QLoRA keeps that same low-rank idea but quantizes the frozen backbone so the whole setup fits within a much smaller memory budget.
That is also why these methods feel related but not identical. Adapters and prefix tuning are easiest to understand as augmentation mechanisms added to the model’s computation graph. LoRA is more precise to describe as a constrained reparameterization of the update. Distillation then sits outside this family a bit, because it is really about compressing capability into a smaller student rather than only making finetuning cheaper.
One data-side insight that cuts across all of these methods is that the finetuning data itself matters as much as the mechanism. Finetuning can mix reasoning-style traces with general chat data, so the model picks up the target behavior without becoming narrowly specialized at the cost of losing other capabilities. In other words, the choice of adapter, prefix, or low-rank update decides how the model is adapted, but the data mixture decides what it ends up being good at.