Neural Networks - Going beyond the buzz

How a neural network can identify a handwritten digit

The use of neural networks has reinforced the scopes of machine learning. It has become almost common knowledge among people who have some interest in these matters that neural networks are designed to emulate the biological neurons - the unitary components of the nervous systems in sentient beings. When a system of neural networks uses a large number of layers it is called a deep neural network and the form of machine learning that uses these networks is called deep learning.

Deep learning has caused a small scale revolution in terms of image recognition, computer vision, and natural language processing which in turn has positively affected and influenced hundreds of academic as well as industrial bodies across the globe.

The information I have provided so far comes in the neural network buzz package. I am counting on you to have already gathered this much. What we will do henceforth is unlock the mysteries of a neural network to a limited extent. My goal is neither to waste your time nor to overwhelm you with technical jargon. I just want you to take a look inside a neural network and finally take away something of value.

What is a neural network?

It is difficult to define a neural network in easily comprehensible terms. Just grasp this much that it is a network consisting of multiple layers of neurons where all the neurons in one layer are connected to every neuron in the subsequent layer. If this sounds like gibberish, that is what we will untangle.

The image of a handwritten digit

What does an image contain? It contains pixels. So let us take the image that has 28x28 pixels which means a total of 784 pixels. Our goal is to conceptualize a neural network that can take these 784 pixels as the input and provide one digit as the output. Let the digit here be 9. A shaky 9 written in bright white, projected on a black backdrop.

The neurons

As you know, the neural network is supposed to emulate the nervous system, the smallest unit of this network is also called a neuron. You do not need to think of the neuron as a cell with certain properties, that will complicate things. Just think of it as an empty sphere which can contain a number.

The structure of the network

The input layer

The network starts with a neuron assigned for each pixel of the image we are trying to identify. That means our little network starts with 784 neurons which make the first layer of the network. Now, each one of these neurons holds a number that corresponds to the grayscale value of the pixel on a scale of 0 to 1. 0 for the darkest pixels and 1 for the brightest. Our digit, as you may remember, is written in bright white. This number inside each neuron is called activation.

The output layer

We need a digit in the output so the output layer consists of 10 neurons each corresponding to a digit between 0 to 9. The activations in these neurons show how much the system thinks that the given image corresponds with a certain digit on a scale of 0 to 1. The neuron with the highest activation wins.

The hidden layers

Let us take two hidden layers and let us arbitrarily set the number of neurons in each of those two layers to 16. The activations in the first layer determine the activation in the second layer and so on. In action

If you feed in an image that lights up all 784 neurons in the input layer according to the brightness of the pixel it is associated with, a pattern of activations is created. That pattern of activations causes some very specific pattern in the next layer which in turn causes some pattern in the following layer. This finally leads to a pattern of activations in the output layer. The brightest neuron in the output layer is the network’s choice for what digit the image represents.

What I am not telling you

I promised limited access inside the living room of a network. There is much more fun to it. How does the activation pattern in the second layer actually form? What are the parameters? What sort of math goes into all these? How do the activation patterns actually lead to the recognition of the digit nine, which is just a loop up top with an adjacent line on its right side? If you are bugged by these questions, that is good; seek and you shall find. An artificial intelligence course in Delhi can come in real handy.

Start as an awestruck enthusiast, you may just end up spinning a career out of this.

The use of neural networks has reinforced the scopes of machine learning. It has become almost common knowledge among people who have some interest in these matters that neural networks are designed to emulate the biological neurons.