Evaluation metrics for AI models

01

What goes wrong when you only look at accuracy

A card company builds a model to spot fraudulent transactions. They test it and get 95% accuracy. That means it got 95 out of 100 right, which sounds pretty good. But what if only 5 of those 100 transactions were fraud in the first place?

In the scene below, each cell is one transaction, and the red cells are the real fraud cases. Any cell the model flags as "fraud" pops forward. Switch between the two models.

The model that always answers "normal" hasn't learned a thing, yet its accuracy is 95%. It didn't catch a single fraud case. The model that was actually trained has lower accuracy, 93%, but it caught 4 of the 5 fraud cases. The second model is the one this company needs.

Accuracy is the share of all predictions that were right. When one outcome vastly outnumbers the other — what's called imbalanced data — you can get high accuracy just by always guessing the common outcome. Most problems where the thing you're looking for is rare work like this: fraud detection, disease diagnosis, defect inspection.

So what should you look at instead? It starts with sorting the model's hits and misses into four groups.

02

Confusion matrix, precision, recall and F1

When a model picks one of two answers, like "spam or not spam", every result lands in one of four boxes. The class you're trying to find (spam) is called positive, and the other one (normal email) is called negative. Then you put true in front if the model was right, and false if it was wrong.

Here's the part that trips people up. Say the model calls a normal email spam. It said "positive" and was wrong, so that's a false positive (FP). If it lets a spam email through as normal, that's a false negative (FN). The second word is what the model said; the first word tells you whether it was right.

This table is called a confusion matrix. From it, you get two key ratios.

A classification model usually gives a score between 0 and 1, and calls something positive if the score is above a threshold you choose. The scene below lines up 20 emails by score. Try moving the threshold.

Raise the threshold and the model only flags what it's sure about, so precision goes up — but more spam slips through, so recall goes down. Lower the threshold and the opposite happens. Usually, pushing one up pulls the other down. So you set the threshold based on which mistake costs more. For something like cancer screening, where missing a case is unacceptable, you favor recall. For a spam filter that must never bury an important email, you favor precision.

The F1 score lets you look at both at once. It's the harmonic mean of precision and recall, so if one is high and the other is low, F1 comes out low too. It's a common way to compare models with a single number on imbalanced data.

But if precision and recall change with every threshold, how do you compare two models before you've picked a threshold?

03

ROC curve and AUC

Lower the threshold bit by bit from 1 to 0, and at each step measure two things. The true positive rate (TPR) is the share of real spam you caught. The false positive rate (FPR) is the share of normal emails you wrongly filtered. TPR is the same thing as recall from chapter 2. Plot FPR across and TPR up, join the dots, and you get the ROC curve.

A good model catches spam quickly while only filtering a few normal emails by mistake, so its curve shoots up toward the top-left. A model that guesses at random follows the diagonal. The area under the curve is called the AUC (area under the curve). In the scene below, the area filled by the columns is the AUC.

AUC runs from 0.5 (random guessing) to 1 (perfect). It sums up, in one number, how well the model separates positives from negatives without committing to a single threshold. That makes it handy for comparing several models before you've chosen a threshold. When you actually put a model into service, though, you still have to pick one threshold, and then you're back to precision and recall from chapter 2.

So far the models have been guessing a category, like "spam or not". A model that predicts a number, like sales or a house price, can't simply be right or wrong. You need to measure how far off it was.