Use intuitive arguments to justify these four heuristics

Assignment Help Basic Computer Science
Reference no: EM131108996

In Jacobs (1988), the following heuristics are proposed to accelerate the convergence of on-line back-propagation learning:

(i) Every adjustable network parameter of the cost function should have its own learn ingrate parameter.

(ii) Every learning-rate parameter should be allowed to vary from one iteration to the next.

(iii) When the derivative of the cost function with respect to a synaptic weight has the same algebraic sign for several consecutive iterations of the algorithm the learning-rate parameter for that particular weight should be increased.

(iv) When the algebraic sign of the cost function with respect to a particular synaptic weight alternates for several consecutive iterations of the algorithm, the learning-rate parameter for that weight should be decreased.
These four heuristics satisfy the locality constraint of the back-propagation algorithm.

(a) Use intuitive arguments to justify these four heuristics.

(b) The inclusion of a momentum in the weight update of the back-propagation algorithm may be viewed as a mechanism for satisfying heuristics (iii) and (iv). Demonstrate the validity of this statement.

Reference no: EM131108996

Define the weight-error vector

An alternative to the mean-square error (MSE) formulation of the learning curve that we sometimes find in the literature is the mean-square deviation (MSD) learning curve. D

Algorithm for computing a set of synaptic weights

Use the back-propagation algorithm for computing a set of synaptic weights and bias levels for a neural network structured as in Fig. 4.8 to solve the XOR problem. Assume th

Research professional associations and summarize

Research professional associations and summarize one. Based on previous coursework on networking and/or new online research, locate at least two professional associations th

Human resource management challenges

A Kenya-based chain of supermarkets has been having human resource management challenges relating to cultural differences. This has caused bad blood between the Kenyan retai

Labour education and employee development schemes

Write an essay of 1500--2000 words. "Labour education and employee development schemes provide the best hope for empowerment for workers and for the democratization of work.

Neural network involving a single hidden neuron

Figure P4.1 shows a neural network involving a single hidden neuron for solving the XOR problem; this network may be viewed as an alternative to that considered in Section 4

Investigate the difference that would be made

The momentum constant α is normally assigned a positive value in the range 0 α ≤ 1. Investigate the difference that would be made in the behavior of Eq. (4.43) with respect

Explore the way in which the inclusion of the momentum

Explore the way in which the inclusion of the momentum constant α influences the learning process, with particular reference to the number of steps required for convergence

Reviews

Write a Review

 
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd