Over fitting considerations - artificial intelligence, Computer Engineering

Over fitting Considerations - artificial intelligence

Left  unexamined ,  back  propagation  in  multi-layer  networks  may  be very susceptible  to over fitting itself to the training examples. The following graph plots the error on the training and test set as the number of weight updates increases. It is error prone of networks left to train unchecked.

810_Over fitting Considerations.png

Alarmingly, even though the error on the training set continues to slowly decrease, the error on the test set essentially begins to increase towards the end. It is clearly over fitting, and it relates to the network starting to find and fine-tune to idiosyncrasies in the data, rather than to general properties. Given this phenomena, it would not be wise to use some sort of threshold for the error as the termination condition for back propagation.

In the cases where the number of training examples is high, one antidote to over fitting is to crack the training examples into a set to use to train the weight and a set to hold back as an internal validation set. This is a mini-test set, which may be used to keep the network in check: if the error on the validation set reaches minima and then start to increase, then it could be over fitting in beginning to occur.

Note that (time permitting) it is good giving the training algorithm the advantage of the doubt as much as possible. That is, in the validation set, the error may also go through local minima, and it is unwise to stop training as soon as the validation set error begin to increase, as a better minima can be achieved later on. Of course, if the minima are never bettered, then the network which is in final presented by the learning algorithm should be re-wound to be the 1 which produced the minimum on the validation set.

Another way around over fitting is to decrease each weight by a little weight decay factor during each epoch. Learned networks with large (negative or positive) weights tend to have over fitted the data, because larger weights are needed to accommodate outliers in the data. Thus, keeping the weights low with a weight decay factor can help to steer the network from over fitting.

Posted Date: 10/2/2012 6:39:54 AM | Location : United States







Related Discussions:- Over fitting considerations - artificial intelligence, Assignment Help, Ask Question on Over fitting considerations - artificial intelligence, Get Answer, Expert's Help, Over fitting considerations - artificial intelligence Discussions

Write discussion on Over fitting considerations - artificial intelligence
Your posts are moderated
Related Questions
Pruning and Sorting: This means we can test where each hypothesis explains as entails a common example that we can associate to a hypothesis a set of positive elements in whic

Q. Registers used in IA-64 architecture ? Registers: The IA-64 architecture comprises a very generous set of registers. It has an 82-bit floating point as well as 64-bit inte

Explain about CSMA/CD It is an access method used mainly with LANs configured in a bus topology. With CSMA/CD, any station (node) can transmit a message to any other  statio

What is the File-system security Most commercial servers use a powerful operating system as base that provides good methods of file-system security. Some servers require additi

Reduced Instruction Set Computer (RISC): As we discussed before most of the modern CPUs are of the GPR (General Purpose Register) type. A few instances of such type of CPUs ar

Develop a web traffic plan - suppose that there is no/minimal advertising budget so you will require to look at branding choices & site design, word-of-mouth and publicity as your

Q. Define Wide Area Network? Wide Area Network (WAN): It takes a large geographical area and frequently owned by a state. Data transfer rate is low (few KBPS to 10 MBPS) and er

Give detail explanation about Arithmetic Micro-operations These micro-operations perform some essential arithmetic operations on numeric data stored in the registers. These bas

The next type of input is a Pull Down List. With this type you have to employ in place of and it also has a closing tag. This control is used when we have a

1's complement representation of decimal number of -17 by using 8 bit representation is ? Ans. (17) 10 = (10001) 2 In 8 bit = 00010001 1's Complement = 11101110