Learning weights in perceptrons, Computer Engineering

Learning Weights in Perceptrons - Artificial neural network

In detail we will look at the learning method for weights in multi-layer networks next chapter. The following description of learning in perceptions will help explain what is going on in the multilayer case. We are in a machine learning set up so we can expect the job to be to learn a objective function which categorizes examples into categories, at least given a set of training examples supplied with their right categorizations. A little thought will be needed in order to select the right way of thinking regarding the examples as input to a set of input units, but, due to the simple nature of a preceptor, there is not much option for the rest of the architecture.

In order to produce a perceptron able to perform our categorization task, we have to use the examples to train the weights among the input units and the output unit, and to train the threshold. To make simple the routine, we think of the threshold as a special weight, that comes from a special input node that always outputs one. Thus, we think of our perceptron like this:

Then, we say that the output from the perceptron is +1 if the weighted sum from all the input units (including the special one) is bigger than zero, and it outputs -1 otherwise. We see that weight w0 is easily the threshold value. Though, thinking of the network like this means in the same way we may train w0 as we train all the other weights.

The weights are first assigned randomly and training instance is used 1 after another to tweak the weights in the network. In the training set all instance are used and the whole process (using all the examples again) is iterated till all examples are exactly categorized by the network. The tweaking is called as the perceptron training rule, and is following like: If the training example, E, is exactly categorized by the network, no tweaking is carried out,then. If E is misclassified, then every specific weight is tweaked by adding on a tiny value, Δ. Imagine we are trying to calculate weight wi, which is between the xi, i-th input unit and the output unit. Then,  given  that  the  network  should  have  calculated  the  target  value  t(E)  for example E, but in fact calculated the observed value o(E), then Δ is calculated as:

Δ = η (t (E) - o (E)) xi

Notice that η is a fixed positive constant called the learning rate. By ignoring η briefly, we see that the value Δ that we add on to our weight wi is calculated by multiplying the input value xi by t (E) - o (E). t(E) - o(E) will either be -2 or  +2, because perceptrons output only -1 or +1, and t(E) can't be equal to o(E), otherwise we would not be doing any tweaking. Thus, we may think of t(E) - o(E) as a movement in a specific numerical direction, for example, negative or positive. This direction will be like that, if the whole sum, S, was too low to get over the threshold and produce the right categorisation, then the contribution to S from wi * xi will be increased. On the other hand, if S is too high, the contribution from wi * xi is reduced. Because t (E) - o (E) is multiplied by xi, then if xi is a high value (positive or negative), the change to the weight will be larger. To get a better comfort for why this direction correction works, it is a good idea to do some simple calculations by hand.

 

272_Perceptrons1.png

η simply controls how far the correction should go at 1 time, and is generally set to be a fairly low value, for example, 0.1. The weight learning problem may be seen as finding the global minimum error, calculated as the proportion of miscategorised training examples, over a space where every input values may vary. So, it is possible to move too far in a direction and improve one specific weight to the detriment of the whole sum: while the sum can work for the training example being looked at, it can no longer be a good value for categorizing all the examples correct. Due to this reason, η limit the amount of movement possible. If a large movement is really required for a weight, then it will happen over a series of iterations through the example set. For Sometimes, η is set to decay as the number of such type of iterations through the complete set of training examples increases, so that it may move more slowly towards the global minimum in order not to overshoot in 1 direction. This type of gradient descent is at the heart of the learning algorithm for multi- layered networks, as discussed in the next chapter.

Perceptrons with step functions have restricted abilities when it comes to the range of concepts that may be learned, as discussed in a later section. One way to get better matters is to replace the threshold function with a linear unit, so that the network outputs a real value, rather than a -1or +1. This enables us to utilize another rule, called the delta rule, which is also depending on gradient descent. We do not look at this rule here, because the back propagation  learning  method  for  multi-layer  networks  is same.

Posted Date: 10/2/2012 6:01:37 AM | Location : United States







Related Discussions:- Learning weights in perceptrons, Assignment Help, Ask Question on Learning weights in perceptrons, Get Answer, Expert's Help, Learning weights in perceptrons Discussions

Write discussion on Learning weights in perceptrons
Your posts are moderated
Related Questions
Q. What is Block size of cache memory? Block Size: Block size means unit of data (few memory words) exchanged between main and cache memory. As block size increases from very

Task 1:   Methodologies are 'regarded as a recommended series of steps and procedures to be followed in the course of developing an information system' and were introduced to im

Explain about Magnetic tape      a) They are mostly used for keeping files of data  b )It is one of the cheapest and slowest methods for storage

Q. What is basic working of Modem? A modem (modulator/demodulator) is an electronic device that takes digital data as a serial stream of bits and produces a modulated carrier s

How are problems of clock skew minimized? Clock skew, when done right, can also benefit a circuit. This can be intentionally introduced to reduce the clock period, at that the

What is a linked list? Linked list: A linked list is a self referential structure which having a member field that point to the similar structure type. In simple term, a link

What is a disk drive? The electro mechanical mechanism that spins the disk and moves the read/write heads known as disk drive.


Define parity bit and List its types. The most common error detection code used is the parity bit. Parity bit is a extra bit contained with a binary message to make the total n

What are vectored interrupts? To decrease the time involved in the polling process, a device requesting an interrupt may recognize itself directly to the processor. Then the pr