### Use intuitive arguments to justify these four heuristics

Assignment Help Basic Computer Science
##### Reference no: EM131108996

In Jacobs (1988), the following heuristics are proposed to accelerate the convergence of on-line back-propagation learning:

(i) Every adjustable network parameter of the cost function should have its own learn ingrate parameter.

(ii) Every learning-rate parameter should be allowed to vary from one iteration to the next.

(iii) When the derivative of the cost function with respect to a synaptic weight has the same algebraic sign for several consecutive iterations of the algorithm the learning-rate parameter for that particular weight should be increased.

(iv) When the algebraic sign of the cost function with respect to a particular synaptic weight alternates for several consecutive iterations of the algorithm, the learning-rate parameter for that weight should be decreased.
These four heuristics satisfy the locality constraint of the back-propagation algorithm.

(a) Use intuitive arguments to justify these four heuristics.

(b) The inclusion of a momentum in the weight update of the back-propagation algorithm may be viewed as a mechanism for satisfying heuristics (iii) and (iv). Demonstrate the validity of this statement.

#### Define the weight-error vector

An alternative to the mean-square error (MSE) formulation of the learning curve that we sometimes find in the literature is the mean-square deviation (MSD) learning curve. D

#### Algorithm for computing a set of synaptic weights

Use the back-propagation algorithm for computing a set of synaptic weights and bias levels for a neural network structured as in Fig. 4.8 to solve the XOR problem. Assume th

#### Research professional associations and summarize

Research professional associations and summarize one. Based on previous coursework on networking and/or new online research, locate at least two professional associations th

#### Human resource management challenges

A Kenya-based chain of supermarkets has been having human resource management challenges relating to cultural differences. This has caused bad blood between the Kenyan retai

#### Labour education and employee development schemes

Write an essay of 1500--2000 words. "Labour education and employee development schemes provide the best hope for empowerment for workers and for the democratization of work.

#### Neural network involving a single hidden neuron

Figure P4.1 shows a neural network involving a single hidden neuron for solving the XOR problem; this network may be viewed as an alternative to that considered in Section 4

#### Investigate the difference that would be made

The momentum constant α is normally assigned a positive value in the range 0 α ≤ 1. Investigate the difference that would be made in the behavior of Eq. (4.43) with respect

#### Explore the way in which the inclusion of the momentum

Explore the way in which the inclusion of the momentum constant α influences the learning process, with particular reference to the number of steps required for convergence