The CNN Behind Our Vision Models: ResNet-152, Multi-Label, on SageMaker

What a CNN is, why we used ResNet-152, the exact training settings, and how its scores become answers in our app.

Our system uses cameras to understand what’s in front of it: how many objects are in view, how close they are, and whether something is blocking the left side. Each photo can have several labels at once. To do that, we use a convolutional neural network (CNN) called ResNet-152, trained with SageMaker’s built-in image-classification algorithm.

This post explains how that model works, in plain terms first and then with our real settings. You don’t need a deep-learning background to follow it.

What a CNN does

A CNN looks at a photo through lots of small filters. Each filter slides across the image looking for one simple pattern.

• Early layers find edges and color changes.
• Middle layers combine those into shapes and textures.
• Later layers combine shapes into parts of objects.
• The final layer turns all of that into one score per label.

Nobody programs those filters by hand. The network learns them from labeled photos during training.

Why ResNet-152

More layers let a network learn more complex patterns. But very deep networks used to be hard to train: the learning signal got weaker as it passed back through dozens of layers.

ResNet fixed this with shortcut connections. Each block of layers learns a small change and adds it to its input, and the input also skips straight past the block. That shortcut keeps the signal strong, so networks can be very deep. ResNet-152 has 152 layers.

In our settings, that’s one line: num_layers: "152".

We didn’t start from zero: transfer learning.

Training 152 layers from scratch needs millions of photos. We have tens of thousands. So we use transfer learning:

• use_pretrained_model: "1" starts from a ResNet already trained on ImageNet, a huge public photo dataset. It already knows edges, shapes, and textures.
• Our training then teaches it our labels on our photos.

We also often go one step further and start from our own previous model instead of ImageNet. That takes about 10 passes over the data (epochs) instead of 50, because the model already knows most of the job.

One photo, many labels

Most image tutorials pick one label per photo: cat or dog. Ours need several at once, like “few objects," “close range,” and “blocked on the left."

• multi_label: "1" tells the algorithm that each photo can have several labels.
• The model returns a score between 0 and 1 for every label, in the same order as our label map.

Our exact training settings

These are the defaults every model starts from (all values are strings, because SageMaker requires that):

const BASE_HYPERPARAMETERS = {
  num_layers: "152",           // ResNet-152
  image_shape: "3,224,224",    // colour photos resized to 224 × 224
  use_pretrained_model: "1",   // start from ImageNet weights
  multi_label: "1",            // several labels per photo
  epochs: "50",                // 10 when building on our previous model
  mini_batch_size: "32",
  optimizer: "sgd",
  learning_rate: "0.001",
  momentum: "0.9",
  weight_decay: "0.0001",
  augmentation_type: "crop_color_transform",
  use_weighted_loss: "0",
  precision_dtype: "float32",
  top_k: "2",                  // must be 2 or more, or the job fails
  early_stopping: "false",
  checkpoint_frequency: "1",
};

In plain English:

• 224 × 224 color photos. Every photo is resized to the size the network expects.
• 32 photos per step. The model adjusts its filters after each batch of 32.
• Learning rate 0.001 with SGD and momentum. Small, steady steps. Momentum keeps it moving in a consistent direction.
• Weight decay. A small penalty that discourages the model from memorising.
• crop_color_transform. During training, photos are randomly cropped and their colours slightly changed, so the model copes with real-world variety like blur or poor lighting.
• use_weighted_loss: "0". We deliberately don't let the model compensate for rare labels. Instead, we show the imbalance to our team, so they go and label more photos of the rare ones.
• No early stopping. Every run trains the full schedule, so runs are comparable.

Two things are filled in automatically for every run: num_classes (how many labels the model has) and num_training_samples. Nobody can override them, because a wrong value produces a model that gives confident, wrong answers.

Get Nishara Ramasinghe’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Training runs on one ml.g4dn.xlarge, a single NVIDIA T4 GPU.

From scores to answers

The model returns a list of scores. Our app turns them into answers with two simple rules.

Rule 1: Some labels are a group, and only one can win. “Many objects” and “few objects” can’t both be true. So within a group, the highest score wins:

for (const group of groups) {
  const present = group.filter((label) => probMap.has(label));
  if (present.length === 0) continue;
  let maxLabel = present[0], maxProb = -1;
  for (const label of present) {
    const p = probMap.get(label) ?? 0;
    if (p > maxProb) { maxProb = p; maxLabel = label; }
  }
  const hasWinner = maxProb > 0;
  for (const label of present) labels.set(label, hasWinner && label === maxLabel);
}

Rule 2: other labels stand alone, with a 0.5 threshold. “Low lighting” is either true or not, so it’s true when its score is 0.5 or more.

Two edge cases we learned the hard way:

• If every score in a group is 0, nobody wins. That’s the model saying “none of these," not “the first one."
• A label the model doesn’t have shouldn’t read as "no." An older model trained before a label existed returns no score for it. For those labels, the app shows “can’t tell” instead of quietly answering “no” without the model ever looking.

What we’d try next

ResNet-152 is a big network. A smaller depth, such as ResNet-50, might answer faster and cost less to serve, possibly with similar accuracy on our labels. We haven’t tested that yet. With the training pipeline in place, it’s now a one-setting experiment.

Checklist if you’re using the built-in image classifier

1. One label or several per photo? Set multi_label to match.
2. Do you have millions of photos? If not, keep use_pretrained_model on.
3. Is top_k 2 or more? A value of 1 fails the job.
4. Do group labels use “highest score wins," and standalone labels a threshold?
5. Does a label the model doesn’t have to read as “can’t tell," not “no”?

Final thoughts

You don’t need to design a CNN to use one well. The built-in algorithm gave us a proven 152-layer network that already understood photos. Our work was choosing sensible settings and being careful about how its scores turn into answers in the app.

References

• Image classification: MXNet (SageMaker built-in algorithm)
• Image classification hyperparameters
• Deep Residual Learning for Image Recognition (He et al., the ResNet paper)
• Amazon EC2 G4 instances

If this helped, give it a clap and follow for more practical breakdowns on cloud and AI.