Fine-tuning

01

Teaching the model itself

Imagine a company that wants an AI to summarize its call center logs. The summaries have to follow the company format: three fields, "Reason / Action / Follow-up", written in the company's own tone and terms. If you just ask an already-trained LLM to summarize, the content is fine, but the format and tone come out a little different every time.

There are three main ways to fix that. You can add a format description and examples to the prompt (prompt engineering), look up the material the model needs and attach it (RAG), or train the model itself a bit more (fine-tuning). In the scene below, switch between the methods and watch what changes.

The first two only change the text you send to the model. The numbers inside the model, its weights, stay the same. Fine-tuning is the opposite: it changes the weights. You start from a model that has already finished pre-training and train it a little more on your data, nudging those numbers. That takes far less data and money than building a model from scratch. It's a lot like hiring someone experienced and only teaching them how your company does things.

If you want to know how models learn and what weights actually are, see the training chapter of How LLMs work.

To change the weights, the model needs something to learn from. What you get out of fine-tuning is decided almost entirely by your training data.

02

Preparing the training data

The most common kind of fine-tuning trains on examples that pair an input with the correct answer. For our company, one example is a single call log plus a well-written summary of it in the company format. You usually prepare these as a file with one example per line, in a format like JSONL.

{"input": "Customer says they were charged twice…", "output": "Reason: double charge / Action: refund filed / Follow-up: confirmation text within 3 days"}
{"input": "Customer wants to change their delivery address…", "output": "Reason: address change / Action: updated before shipping / Follow-up: none"}

The model looks at these examples and learns "for this kind of input, answer like this." That also means it copies any mistakes hiding in the examples. In the scene below, try cleaning up the examples you've collected.

What good training data looks like

• Consistent format: if the answers are formatted differently from example to example, the model learns to be inconsistent too.
• Correct answers: if wrong summaries slip in, the model learns the wrong way as well. Quality matters more than quantity.
• Close to real inputs: the examples should look like the call logs you'll actually get, in length and in kind.
• No duplicates: if the same example shows up several times, the model over-memorizes that one example.
• Sensitive data removed: a model can memorize training data and repeat it to someone else word for word. Remove or mask personal data like names and phone numbers before training.

How much data do you need? It depends on the task, but if you're mainly teaching a format, a few hundred to a few thousand well-made examples are often enough to make a difference. You also hold back part of what you collect and don't train on it. This is the validation set, and you use it to check how training went on examples the model has never seen. You'll see how it's used in the next chapter.

Once the data is ready, you run the training. Before you do, you have to set a few values, and if you set them badly, training goes off track.