Growing and Learning Algorithms of Radial Basis Function Networks

Abstract

Radial Basis Function (RBF) network is a type of artificial neural network, which uses the Gaussian kernel activation function. It has a fixed three-layer architecture. The RBF network is easier to be designed and trained than traditional neural networks, and they can also act as an universal approximator. They have good generalization properties and can respond well for patterns which are not used for training. RBF networks have strong tolerance to input noise, which enhances the stability of the designed systems. Therefore, RBF network can be considered as a valid alternative for nonlinear system design. This work presents an improved second order algorithm for training RBF networks. The output weights, the centers, widths, and input weights are adjusted during the training process. More accurate results will be obtained by increasing variable dimensions. Taking the advantages of fast convergence and powerful search ability of second order algorithms, the proposed algorithms can reach smaller training and testing errors with a much less number of RBF units. A new error correction algorithm is proposed for the incremental design of radial basis function networks. In this algorithm the number of RBF units is increased one by one until the training evaluation reaches desired accuracy. The initial center of the newly added RBF unit is properly selected based on the location of highest peak/lowest valley in the error surface; while for the other RBF units, the initial conditions are copied from the training results of the last step. Parameter adjustments, including weights, centers, and widths are performed by the Levenberg Marquardt algorithm. This algorithm is very efficient to design a compact network by comparing with other sequential algorithms in constructing radial basis function networks. The duplicate patterns test and the noise patterns test are applied to show the robustness of the proposed algorithm.