On learning feedforward neural networks with noise injection into inputs

Abstract

Injecting noise to the inputs during the training of feedforward neural networks (FNN) can improve their generalization performance remarkably. Reported works justify this fact arguing that noise injection is equivalent to a smoothing regularization with the input noise variance playing the role of the regularization parameter. The success of this approach depends on the appropriate choice of the input noise variance. However, it is often not known a priori if the degree of smoothness imposed on the FNN mapping is consistent with the unknown function to be approximated. In order to have a better control over this smoothing effect, a cost function putting in balance the smoothed fitting induc..