Project: an image classifier

Two ways into the same idea

Before You Start

The Core Idea

The track project is an image classifier for classes you collect yourself. You collect and split the images, build two versions with the same settings, find where each one fails, then write the report.

1Collect and split by session

Pick 3 to 5 classes you can photograph with your phone in two days, like date varieties. Collect 60 to 100 images per class with different lighting, backgrounds and angles, since variety matters more than count.

Look at the picture: the marked step is shooting the test set under other conditions.

torchvision reads image classes straight from folder names. Split early and never move an image between folders afterwards.

dates/
  train/
    sukkari/   *.jpg
    ajwa/      *.jpg
    khalas/    *.jpg
  val/
    sukkari/ ajwa/ khalas/
  test/
    sukkari/ ajwa/ khalas/

Shoot the test set on a different day or in a different place from training. If everything is shot on the same table, the model may learn the table rather than the dates, and look excellent until someone tries it on another table.

2Two versions, same settings

Version one is a small convolutional network from scratch, the baseline. Version two is ResNet18 with a frozen backbone and a new head.

Look at the picture: they differ only in the model. Same split, augmentation, epochs and seed, or you cannot tell what caused the difference.

It needs torchvision, your images and a weight download, so it is here to read and to run on your machine.

import time

import torch
from torch import nn
from torchvision import datasets, transforms
from torchvision.models import ResNet18_Weights, resnet18

norm = transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
train_tf = transforms.Compose([transforms.RandomResizedCrop(224, scale=(0.7, 1.0)),
                               transforms.ColorJitter(0.2, 0.2), transforms.ToTensor(), norm])
eval_tf = transforms.Compose([transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(), norm])

train = datasets.ImageFolder("dates/train", train_tf)
val = datasets.ImageFolder("dates/val", eval_tf)
train_dl = torch.utils.data.DataLoader(train, batch_size=32, shuffle=True, num_workers=2)
val_dl = torch.utils.data.DataLoader(val, batch_size=64)

torch.manual_seed(0)
model = resnet18(weights=ResNet18_Weights.DEFAULT)
model.requires_grad_(False)
model.fc = nn.Linear(model.fc.in_features, len(train.classes))
optimizer = torch.optim.Adam(model.fc.parameters(), lr=1e-3)

start = time.perf_counter()
for epoch in range(8):
    model.train()
    for xb, yb in train_dl:
        loss = nn.functional.cross_entropy(model(xb), yb)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    model.eval()
    with torch.no_grad():
        correct = sum((model(xb).argmax(1) == yb).sum().item() for xb, yb in val_dl)
    print(f"epoch {epoch} val acc {correct / len(val):.3f}")
print(f"training time: {time.perf_counter() - start:.0f}s")

Everything in this file appeared in this track: preprocessing and normalisation from the first lesson, augmentation without flips, freezing and swapping the head, the five training lines, and eval with no_grad for evaluation. Copy it for the from-scratch version and change only the model definition and the optimiser.

3Where the model fails

The confusion matrix tells you which two classes the model mixes up. Then you look at the wrong images yourself, and usually find one recurring reason.

Look at the picture: overall accuracy hides that one class is much weaker than the rest.

Before your images, try the analysis on the digit images with a simple classifier. The same lines work on any model's predictions.

import numpy as np
from sklearn.datasets import load_digits
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC

X, y = load_digits(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, train_size=300, random_state=0, stratify=y)
pred = SVC().fit(X_tr, y_tr).predict(X_te)
cm = confusion_matrix(y_te, pred)
print(f"accuracy: {(pred == y_te).mean():.3f}")
per_class = cm.diagonal() / cm.sum(axis=1)
print("worst classes:", [(int(c), round(float(per_class[c]), 2)) for c in np.argsort(per_class)[:3]])
off = cm.copy()
np.fill_diagonal(off, 0)
i, j = np.unravel_index(off.argmax(), off.shape)
print(f"most common confusion: true {i} predicted as {j}, {off[i, j]} times")

95.3% overall, but 3 is right only 86% of the time, and the most common confusion is 3 read as 8, eight times. Visually it makes sense: a 3 is an 8 with an open left side. In your project you will find the same: "ajwa and khalas get mixed up in dark photos", a sentence that leads to a specific fix, like more dim-light images.

4The report

The report compares both versions with numbers measured on the same machine. Then it judges: is the difference worth it? A version two points better and ten times slower at prediction may not suit a phone.

Look at the picture: the numbers that go in the table, then a paragraph on the two most common error types, with an image and a hypothesis for each.

How It Works, Step by Step: how to build the classifier and compare its two versions

Worked example

Choose three to five classes you can photograph with your phone in two days, like date varieties, coffee types or the leaves of your garden plants. Collect 60 to 100 images per class, with different lighting, backgrounds and angles.

Try it without code

Myth vs Reality

Try it yourself

Quick review cards

Check Your Understanding