## #014 TF Implementing LeNet-5 in TensorFlow 2.0

*Highlights*: In this post we will show how to implement a foundamental Convolutional Neural Network like \(LeNet-5\) in TensorFlow. The LeNet-5 architecture was invented by Yann LeCun in 1998 and was the first Convolutional Neural Network.

Tutorial Overview:

### 1. Theory recapitulation

The goal of \(LeNet-5 \) was to recognize handwritten digits. So, it takes as an input \(32\times32\times1 \) image. It is a grayscale image, thus the number of channels is \(1 \). Below we can see an arhitecture of this network.

In this post we will implement this model for the Fashion MNIST dataset, which serves as a direct drop-in replacement for the original MNIST dataset and is a little bit more complicated than the main one with digits. Original MNIST is nowadays considered as too easy. In the last two decades, many researchers proposed successful solutions for original MNIST. You can check direct comparison results here.

Because this network is primarily designed for MNIST dataset, it performs significantly better on it. With small changes, it can achieve that accuracy on the Fashion MNIST dataset. However, in this post we will stick to the original architecture of the network.

About LeNet-5 architecture you can read a detailed theoretical post here.

Let’us summarize layers of LeNet-5 architecture.

Layer Type | Feature Map | Size | Kernel Size | Stride | Activation |
---|---|---|---|---|---|

Image | 1 | 32×32 | – | – | – |

Convolution | 6 | 28×28 | 5×5 | 1 | tanh |

Average Pooling | 6 | 14×14 | 2×2 | 2 | – |

Convolution | 16 | 10×10 | 5×5 | 1 | tanh |

Average Pooling | 16 | 5×5 | 2×2 | 2 | – |

Fully Connected | – | 120 | – | – | tanh |

Fully Connected | – | 84 | – | – | tanh |

Fully Connected | – | 10 | – | – | softmax |

### 2. Implementation in TensorFlow

The interactive Colab notebook can be found at the following link

For practice, you can try to change a dataset by replacing *fashion_mnist* with *mnist*, *cifar10* or other.

Let’s start with importing all necessary libraries. After imports, we can use imported module to load data. The * load_data()* function will automatically download and split our data into training and test sets.

We can check the shape of new data and see that our images are 28×28 pixels, so we need to add a new axis, which will represent a number of channels. Also, it is important to do one-hot encoding of labels and normalization of input images.

Now, it is time to start using TensorFlow 2.0 in order to build our convolutional neural network. The easiest way to do this is by using the Sequential API. We will wrap it in a class called *LeNet*. The input is an image, and the output will be a class probability vector.

In tf.keras, Sequential model represents the linear stack of layers and in this case it follows the network architecture.

After creating a model, we need to train its parameters to make it powerful. Letâ€™s train the model for a given number of epochs. We can see the progress of training in TensorBoard.

Not bad for only 20 epochs.

After this, we can make some predictions and visualize it. With *predict_classes()* function we can get the exact class from the network instead of probability values. It is the same as using the *numpy.argmax()*. We will display the false predictions with red and the correct ones will be signified with blue.

### Summary

So, here we have learned how to develop and train LeNet-5 in Tensorflow 2.0. In the next post we will continue with implementations of popular Convolutional Neural Networks and learn how to implement **AlexNet** in TensorFlow 2.0.