Desktop version

Home arrow Computer Science

  • Increase font
  • Decrease font

<<   CONTENTS   >>


The past literature survey indicates that multilayer perceptions are widely used in neural networks in comparison to single-layer perception. It is due to the inefficiency of single-layer perceptions in the solution of complex problems. However, multilayer perception does present several problems as discussed in the following.

  • • Multilayer perceptions are difficult to implement, but the usage of commercially available software can solve these problems to a certain degree (e.g., [17, 81]).
  • • The second main issue with the multilayer perception theoiy is in the decision of network architecture. The network architecture depends on the number of hidden layers, which, in turn, depends on training data. The training data vaiy depending on the problem, so there are no specific rules to decide. The BP network may face the convergence issue in case of lesser number of nodes in the hidden layers. However, the large number of nodes cause network overfitting the training data resulting in poor performance.

• The network leams all the training patterns, which leads to overfitting. This overtrained network will lead to an inaccurate prediction of the pattern, which generates clatter. If used for extrapolation, multilayer perceptions may perform poorly. However, for the interpolation, they may exhibit better performance.

Apart from problems associated with a multilayer perception, the BP network has its inherent disadvantages. The first drawback is the slow learning rate. In the case of a BP network, the number of iterations is enormous (Carpenter and Barthelemy, 1994). In addition, the sensitiveness of convergence rate is observed for parameters such as momentum ratio and learning rate. The selection of appropriate values of these parameters is strongly influenced by the nature of the problem [3,99].


Although the reduction in the number of hidden layers may reduce the noise, a network is trained to overlook a small noise in a particular situation. The overfitting of the training data is one of the significant causes of concern for the perception. It can be avoided by subdividing the network into several units such as the following.

  • • Training set
  • • Test set
  • • Validation set

The validation set determines the capability of the network to generalize itself during training. The training is discontinued as generalization reaches the maximum value. This process is called as early stopping and is widely adopted in training of multilayer perception network. The basic concept of the early stopping technique is as follows.

  • • The data available for training are divided into data for training and validation in the ratio of 2:1.
  • • The training set is trained and evaluated to determine the error after the end of each cycle.
  • • The training process is terminated as soon as the error on the validation set exceeds the previous cycle.
  • • The weights are kept unchanged at each cycle.

Early stopping is preferred due to its greater efficiency overgeneralization techniques.

The other ways to reduce overfitting are (i) reducing the parameter size which is termed as “greedy constructive learning,” and (ii) sharing of weights to the reduced size of each parameter by employing techniques such as regularization, weight decay, etc.

There are several other solutions that can be adopted are the following.

  • • The functions employed in training the multilayer perception should be smooth so that input data become sensitive to the output data. Nevertheless, the used training data should be extensive in number and should cover a wide range of parameters. The adopted training data should represent all the problems to which the multilayer perception will be subjected.
  • • In case the training data adopted are based on extreme or uncommon events, then it would reduce the accuracy of the network. This problem can be rectified by a technique called V-fold cross-validation. In this technique, the data are randomly divided in the form of V-shaped individual subsets. After that, training of multilayer perception is done to determine its performance. Several other approaches to overcome this problem can be adopted, such as data preprocessing, adjusting the training algorithm to ignore the well- defined patterns.
  • • For problems with less complexity, a single hidden layer is required to generate approximate results.

The issues related to the BP networks can be solved as follows.

  • • The learning rate and momentum parameter need to be carefully chosen based on the problem. LeCun et al. [57] prescribe following thumb mles for the process. The momentum parameter needs to be increased, and the learning rate needs to be decreased for convergence of the network. In addition, the learning rate should be decreased if network fluctuates in the vicinity of the solution.
  • • More sophisticated algorithms can be adopted so that the necessity of sensitive training parameters is eliminated. Many alternatives such as log sig functions are available, but bounded functions are used for BP networks.
  • • The rescaling of input data is done between 0 and 1. Then, initial network weights are adopted randomly, but input parameter exhibiting lesser variance is preferred.
  • • The standardization of inputs is done by division of input data by the respective standard deviation.
  • • BP is slow, and this problem can be solved by reducing the dimensionality of the input data. This is done employing the process of feature selection, in which the redundant variables are removed from the input data [82].


The solution presented in the previous section has the following advantages as enumerated under

  • • It improves accuracy.
  • • Reduces the computation time.
  • • Less storage space is required.
  • • It reduces the cost of future measurements.
  • • It improves data and model understanding.

There is no specific rule to obtain minima in case of early stopping technique. There may be more than one minima in case of real problems, and consequently, no particular criterion is prescribed to judge a minimum of iterations for the desired accuracy. The selection technique requires a large subset of data, which should extensively cover all types of problems; otherwise, it may induce errors in the network. Moreover, the dimensions of the input parameters should also be extensive.


A large number of problems are encountered in the field of structural engineering, and review of past literature study shows that multilayer percepti on is a very useful tool, which can be used to solve these complex problems. The multilayer perceptions cany a distinct advantage that they are simple, easy to implement, and can be effectively applied to nonlinear problems. These are distmct advantages over nonlinear systems. The multilayer perceptions are easy to perform with the development of sophisticated software. Nevertheless, multilayer percepti ons do cany some of the inherent disadvantages such as the decision of network architecture and number of layers, which varies for each type of problem. Moreover, overfitting is also one of the major problems encountered in multilayer perception theory. These problems can be solved to a large extent to facilitate the easy implementation of multilayer perceptions.


The other domains of civil engineering such as traffic engineering, disaster mitigation, environmental engineering, etc., can be explored. Nevertheless, more techniques of forecasting can be adopted in conjunction with the ANN. Finally, more case studies can be involved.


  • backpropogation network
  • conjugate gradient network
  • structural engineering
  • artificial neural network.
<<   CONTENTS   >>

Related topics