[ad_1]

Batch normalization is basically standardizing or normalizing the output of the previous Convolutional Network layer by subtracting each value by its mean and dividing by the standard deviation of the same.

In simple normalization of image, we have single mean and variance value for each RGB channels, or the no. of channels present in an image whereas in batch normalization we have global mean and global standard deviation which is calculated for each channel for each mini batch of images. This will reduce the computational time as calculating mean or each of the image is a tedious task. Thus, global mean and standard deviation will act as a standard value for the batch of images.

Considering a single image of 6×6 with single channel:

The mean of the image is given by:

Where w=width of image, h= height of image, x( i,j)=pixel value

W=h=6. So, using the pixel values and using equation 1, Mean=**17.5**

The Standard deviation is calculated as given below:

Using eq. 2, Standard deviation= **10.388**

To get the normalized image, simply subtract each pixel value by the mean and divide by the standard deviation as given below.

Similarly, for batch normalization considering the batch of 32 training images the above equations will be modified as:

Where N=batch size or in our case 32.

The new global mean=mean/N =0.547

Global standard deviation = 1.836

The above calculations are for single channel, similarly the normalization can be done for all the channels.

The Batch Normalization has many no. of batches so which batch will provide with the beast results depends upon the concepts of the Moving-Averages and Moving Standard Deviations.

The equation describing the final step which is normalizing each pixel value of a set of images is given as:

Where b= batch size

are Batch normalization parameters that are used for learnable mean, learnable standard deviation respectively and Epsilon which is kept constant so in case of Standard deviation becomes zero the non-zero value in the denominator will help in providing the value rather that tending it to infinity.

**Advantages of using Batch Normalization:**

**1)** Since Batch Normalization reduce noise and hence it reduces overfitting.

**2)** Increases the learning rates.

**3)** Helps in improving the output of the activation layer by regulating its input as Batch Normalization is used before the activation layer.

**4)** Decrease computational time and hence train the large sized dataset faster.

**Conclusion: **Batch Normalization provides with the better accuracy and can be used in CNN networks before the activation layer but use less dropout values if using Batch Normalization.

** Stay Tuned** for the new articles on Neural Networks.

[ad_2]

Source link