What is batch normalization and its benefit?

What is batch normalization and its benefit?

• A. Increases model capacity by adding parameters.
• B. Normalizes layer inputs to stabilize and speed up training, reducing internal covariate shift.
• C. Reduces dataset size by trimming features.
• D. Alters activation functions for nonlinearity.

Answer: (B)

Batch normalization normalizes the inputs of each layer within a mini-batch, stabilizing their distribution and speeding up training. It does this by computing the batch’s mean and variance, normalizing the inputs to zero mean and unit variance, and then applying learnable scale and shift parameters to preserve the layer’s expressive power. This reduces internal covariate shift—the changing distribution of layer inputs as learning progresses—making optimization more efficient and allowing higher learning rates. The result is faster convergence, more stable gradients, and often improved performance, with the model using running statistics during inference. It’s not about increasing capacity, trimming features, or altering activation functions; those are different concepts.

Batch normalization normalizes the inputs of each layer within a mini-batch, stabilizing their distribution and speeding up training. It does this by computing the batch’s mean and variance, normalizing the inputs to zero mean and unit variance, and then applying learnable scale and shift parameters to preserve the layer’s expressive power. This reduces internal covariate shift—the changing distribution of layer inputs as learning progresses—making optimization more efficient and allowing higher learning rates. The result is faster convergence, more stable gradients, and often improved performance, with the model using running statistics during inference. It’s not about increasing capacity, trimming features, or altering activation functions; those are different concepts.