Learning abilities of perceptrons - ann, Computer Engineering

Learning Abilities of Perceptrons - Artificial intelligence

Computational learning theory is the study of what concepts specific learning schemes (representation and method) can and cannot learn. We do not look at this in detail, but a popular example, at first highlighted in a very powerful book by Minsky and Papert involves perceptrons. It has been mathematically verified that the above technique for learning perceptron weights will converge to a ideal classifier for learning tasks where the target concept is linearly separable.

To understand what is and what is not a linearly separable target function, we take simplest functions of all, Boolean functions. These take 2 inputs, which are either -1 or 1 and output either a -1 or a 1. Note down that, in other contexts, the values 1and 0 are used instead of 1 and -1. As an instance function, the AND Boolean function outputs a 1 only if both inputs are 1, whereas the OR function just outputs a 1 if either inputs are 1. Clearly, these relate to the connectives we studied in first order logic. The following 2 perceptrons may represent the AND and OR Boolean functions respectively:

 

1931_Learning Abilities of Perceptrons.png

One of the main impacts of Minsky and Papert's book was to highlight the issue that perceptions can't learn a specific boolean function called XOR. This function output 1 if the 2 inputs are not the similar. To see why XOR can't be learned, try and write down a perception to do the task. The following diagram highlights the notion of linear reparability in Boolean functions, which describe why they cannot be learned by perceptions:

In every case, we have plotted the values taken by the Boolean function when the inputs are specific values: (-1,-1) ;(1,-1);(-1,1) and (1,1). For the AND function, there is just one place where a 1 is plotted, namely when both inputs are 1. It meant that we could draw the dotted line to separate the output -1s from the 1s. We were able to draw a same line in the OR case. Because we may draw these lines, we say that these functions are linearly separable. Notice that it is impossible to draw such as line for the XOR plot: wherever you try, you never get a clear split into 1s and -1s.

 

396_Learning Abilities of Perceptrons1.png

The dotted lines may be seen as the threshold in perceptrons: if the weighted sum S falls below it, then the perceptron outputs 1 value, and if S falls above it, the other output is produced. It is not issue how the weights are organized; the threshold will yet be a line on the graph.  Therefore, functions which are not linearly separable can't be represented by perceptrons.

Notice that this result extends to functions over any number of variables, which may take  in  any  input,  but  which  produce  a  Boolean  output  (and  so could,  in principle be learned by a perceptron). In the following two graphs, For example, the function takes in 2 inputs (like Boolean functions), but the input may be over a range of values. The concept on the left may be learned by a perceptron, whereas the concept on the correct cannot:

As an exercise, draw in the separating (threshold) line in the left hand plot.

 

2105_Learning Abilities of Perceptrons2.png

Unluckily, the disclosure in Minsky and Papert's book that perceptrons can't learn even sach type of simple function was taken the incorrect way: people believed it represented a basic flaw in the use of ANNs to perform learning job. This lead to a winter of ANN research within Artificial Intelligence, which lasted over a decade. In realism, perceptrons were being studied in order to achived insights into more complexed architectures with hidden layers, which don't have the restriction that perceptrons have. No one ever advised that perceptrons would be eventually used to solve real world learning problems. luckily, people studying ANNs within other sciences (notably neuro-science) revived interest in the study of ANNs. For more facts of computational learning theory, see chapter 7 of Tom Mitchell's machine learning book.

Posted Date: 10/2/2012 6:07:36 AM | Location : United States







Related Discussions:- Learning abilities of perceptrons - ann, Assignment Help, Ask Question on Learning abilities of perceptrons - ann, Get Answer, Expert's Help, Learning abilities of perceptrons - ann Discussions

Write discussion on Learning abilities of perceptrons - ann
Your posts are moderated
Related Questions
Common Functions of Interrupts: An Interrupt transfers control to the interrupt service routine, generally through the interrupt vector table, which contains the addresses

Explain FDM and show how CCITT standards help in building the base band? Frequency Division Multiplexing: This is the process of combining some information channels through s

Explain about CSMA/CD It is an access method used mainly with LANs configured in a bus topology. With CSMA/CD, any station (node) can transmit a message to any other  statio

Vector Processing  A vector is an ordered set of the similar type of scalar data items. The scalar item can be a floating point number, a logical value or an integer. Vector pr


What are priority Queues? There are many queues in which we can insert items or delete items from any of the position based on some property. Now, those queues based on the pro

What are the Process states? By the courses of implementation, processes change state. Status of a process is express by its present activity. Dissimilar practical states of

Describe the Assume - Assembler directives ASSUME: This directive would be used to map the segment register names with memory addresses.  Syntax is as below: ASSUME SS:

Find the Regular Grammar for the following Regular Expression: a(a+b)*(ab*+ba*)b.

What is data hazard? Any condition that causes the pipeline to stall is known as a hazard. A data hazard is any condition in which either the source or destination operands of