Author avatar

Dr. Emmanuel Tsukerman

Build Your First Image Classification Experiment

Dr. Emmanuel Tsukerman

  • Jun 17, 2020
  • 13 Min read
  • 182 Views
  • Jun 17, 2020
  • 13 Min read
  • 182 Views
Data
Data Analytics
Artificial Intelligence
Artificial Intelligence (AI)

Introduction

Working knowledge of neural networks, TensorFlow and image classification are essential tools in the arsenal of any data scientist, even for those whose area of application is outside of computer vision. Indeed, the technology of Convolutional Neural Networks (CNNs) has found applications in areas ranging from speech recognition to malware detection and even to understanding climate. This guide will help you onboard onto using these tools at lightning speed by guiding you through a fun starter project of classifying cats and dogs, and then point the way to subsequent steps to becoming an expert on image classification.

Code for this guide is available on Github.

Accessing the Dataset

If you haven’t registered yet to Kaggle, head on over to Kaggle and create an account. Open the settings to your account and click Create New API Token:

Accessing the Kaggle API

This will download a Kaggle API json file, which you’ll want to place at ~/.kaggle/kaggle.json (or, for a typical Windows setup, at C:\Users\<Windows-username>\.kaggle\kaggle.json).

The Kaggle API is a convenient way to access datasets. For more information on Kaggle’s API, see Kaggle API Github page.

If your environment lacks the Kaggle pip library, install it by running:

1
pip install kaggle
bash

Now, you can use the Kaggle APIs. In your environment, run:

1
kaggle competitions download -c dogs-vs-cats
bash

This will download the “Dogs vs. Cats” dataset.

Cats and Dogs Dataset

Next, you will unzip the dataset and, for clarity, remove unneeded data.

1
2
3
!unzip train.zip
!mv train data
!rm test1.zip sampleSubmission.csv train.zip
bash

You now have a dataset consisting of cat and dog images.

Exploring the Data

Next, you’ll perform some data exploration. Set a variable pointing to the dataset’s location.

1
DATASET_LOCATION = "data"
python

Collect the labels and filenames of the dataset.

1
2
3
4
5
6
7
8
9
10
import os

filenames = os.listdir(DATASET_LOCATION)
classes = []
for filename in filenames:
    image_class = filename.split(".")[0]
    if image_class == "dog":
        classes.append(1)
    else:
        classes.append(0)
python

Read the dataset into a pandas dataframe for convenient access.

1
2
3
4
import pandas as pd

df = pd.DataFrame({"filename": filenames, "category": classes})
df["category"] = df["category"].replace({0: "cat", 1: "dog"})
python

You can see you now have labels for the files:

1
df.head()
python

Dataset preview

In addition, you can see that the dataset is balanced, and consists of 12,500 images of cats and dogs, each:

1
df.category.value_counts()
python

Dataset balanced

The following code block will display a random image:

1
2
3
4
5
6
7
import random
from keras.preprocessing.image import load_img
import matplotlib.pyplot as plt

sample1 = random.choice(filenames)
image1 = load_img(DATASET_LOCATION + "/" + sample1)
plt.imshow(image1)
python

Random Image1

Here is code to preview another datapoint:

1
2
3
sample2 = random.choice(filenames)
image2 = load_img(DATASET_LOCATION + "/" + sample2)
plt.imshow(image2)
python

Random Image2

The sizes of the images are not uniform:

Image sizes

This is important because for a standard neural network classifier, the sizes of the inputs must be identical. Fortunately, this is not a problem, as you will rescale all images to the same size. Specify the desired size:

1
2
3
4
IMAGE_WIDTH = 64
IMAGE_HEIGHT = 64
IMAGE_SIZE = (IMAGE_WIDTH, IMAGE_HEIGHT)
INPUT_SHAPE = (IMAGE_WIDTH, IMAGE_HEIGHT, 1)
python

You now have a good sense of what the dataset consists of.

Instantiating a Convolutional Neural Network (CNN) Classifier

Next, you will specify the architecture of a neural network that you will use to classify the images. The architecture you will use is a simple, standard CNN meant to serve as a starting point. The library that implements the CNN is called Keras, and is a high-level API that can use lower-level neural network libraries, such as TensorFlow, under the hood.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), activation="relu", input_shape=INPUT_SHAPE))
model.add(Conv2D(64, (3, 3), activation="relu"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation="relu"))
model.add(Dropout(0.5))
model.add(Dense(2, activation="softmax"))

model.compile(
    loss=keras.losses.categorical_crossentropy,
    optimizer=keras.optimizers.Adadelta(),
    metrics=["accuracy"],
)
python

The model consists of convolutional layers, max pooling layers, dense layers and dropout layers. It utilizes categorical cross-entropy as a loss function, and Adadelta as its optimizer.

Preprocessing the Dataset

To train the model on the data, and to be able to assess its performance as it trains, split the dataset into a training and testing set:

1
2
3
from sklearn.model_selection import train_test_split

train_df, test_df = train_test_split(df, test_size=0.20, random_state=42)
python

Keras also has very convenient methods to perform data augmentation and reading images from directories. Data augmentation is a procedure in which existing data is used to generate new data. For example, you can take an existing image and flip it to create another data point. That is what an ImageDataGenerator allows you to do.

1
2
3
4
5
6
7
8
9
10
11
from keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(
    rotation_range=15,
    rescale=1.0 / 255,
    shear_range=0.1,
    zoom_range=0.2,
    horizontal_flip=True,
    width_shift_range=0.1,
    height_shift_range=0.1,
)
python

The flow_from_dataframe method efficiently reads and preprocesses files of a directory.

1
2
3
4
5
6
7
8
9
10
11
BATCH_SIZE = 16
train_generator = train_datagen.flow_from_dataframe(
    train_df,
    DATASET_LOCATION,
    x_col="filename",
    y_col="category",
    target_size=IMAGE_SIZE,
    class_mode="categorical",
    batch_size=BATCH_SIZE,
    color_mode="grayscale",
)
python

Flow from directory

Create a similar generator for the test images

Test generator

For illustrative purposes, create a generator for a single image and display the corresponding augmented data.

1
2
3
4
5
6
7
8
9
10
example_df = train_df.sample(n=1)
example_generator = train_datagen.flow_from_dataframe(
    example_df,
    DATASET_LOCATION,
    x_col="filename",
    y_col="category",
    target_size=IMAGE_SIZE,
    class_mode="categorical",
    color_mode="grayscale",
)
python

Example generator

Plot the augmented data.

1
2
3
4
5
6
7
8
9
10
plt.figure(figsize=(12, 12))
for i in range(0, 15):
    plt.subplot(5, 3, i + 1)
    for X_batch, Y_batch in example_generator:
        image = X_batch[0]
        image = image.reshape(IMAGE_SIZE)
        plt.imshow(image)
        break
plt.tight_layout()
plt.show()
python

Augmented data

Note that throughout, you have greyscaled the data. The purpose of this is to reduce the size of the data, and therefore the corresponding computational burden. You have now setup the data preprocessing part of your training pipeline.

Training the CNN Using a GPU

If you have access to a GPU, this is the time to enable it. A GPU accelerates most Deep Learning computations (i.e., computations for neural networks with many layers). If you are using a Google Colab notebook, click on Runtime and then on Change runtime type. Under Hardware accelerator, select GPU and then SAVE.

GPU

You now are running your notebook with a GPU enabled. Go ahead and train your model .

1
2
3
4
5
6
7
8
EPOCHS = 10
history = model.fit_generator(
    train_generator,
    epochs=EPOCHS,
    validation_data=test_generator,
    validation_steps=test_df.shape[0] // BATCH_SIZE,
    steps_per_epoch=train_df.shape[0] // BATCH_SIZE,
)
python

This might take a bit of time, but with the simplifications you have made, such as small greyscale images, it shouldn’t be much longer than 10 minutes. Once finished, you should see something like this:

Performance

Looking at val_accuracy, you can see that the classifier attains about 78% accuracy on the testing set, which is a good starting point for further improvements. Looking at the accuracy, which is the training accuracy, you can see that it stays close to the val_accuracy. The implication is that your network is not overfitting, meaning that it is not inferring rules that aren't true. You will see suggestions for improvement in a later section.

It’s nice to see the classifier in action. Take some six samples to observe:

1
2
3
4
5
6
7
8
9
10
11
12
13
NUM_SAMPLES = 6
sample_test_df = test_df.head(NUM_SAMPLES).reset_index(drop=True)
sample_test_datagen = ImageDataGenerator(rescale=1.0 / 255)
sample_test_generator = sample_test_datagen.flow_from_dataframe(
    sample_test_df,
    DATASET_LOCATION,
    x_col="filename",
    y_col="category",
    target_size=IMAGE_SIZE,
    class_mode="categorical",
    batch_size=BATCH_SIZE,
    color_mode="grayscale",
)
python

Predict on these samples:

1
2
3
4
predict = model.predict_generator(sample_test_generator)
import numpy as np

predictions = np.argmax(predict, axis=-1)
python

And display the results:

1
2
3
4
5
6
7
8
9
10
plt.figure(figsize=(12, 24))
for index, row in sample_test_df.iterrows():
    filename = row["filename"]
    prediction = predictions[index]
    img = load_img(DATASET_LOCATION + "/" + filename)
    plt.subplot(6, 3, index + 1)
    plt.imshow(img)
    plt.xlabel(prediction)
plt.tight_layout()
plt.show()
python

Prediction

As you can see, the classifier performs with some mistakes. But that's okay for a first prototype.

Conclusion

You have accomplished much in this guide, taking a set of images and constructing a classifier that can recognize these as images of cats or dogs. There is also a lot of exciting content to learn, from transfer learning to segmentation, in your path to expertise in image classification. Following the directions below, you will reach your destination in no time.

Improving Your Current Classifier

The following steps will yield improvements to your classifier on this problem:

  • Since the classifier is not overfitting it makes sense to experiment with training it for longer
  • Add additional layers to the network
  • Incorporate all three color channels of images
  • Perform additional data augmentation
  • Perform a hyperparameter search
  • Apply transfer learning (e.g., start with a pre-trained professionally-made model like VGG16 and use it on this problem). You can learn this technique in the course Building Image Classification Solutions Using Keras and Transfer Learning.

Additional Ways You Can Practice and Hone Your Skills of Image Classification

Working on different problems like these will allow you to expand the variety of problems you can solve:

Happy learning!

2