Learning abilities of perceptrons - ann, Computer Engineering

Learning Abilities of Perceptrons - Artificial intelligence

Computational learning theory is the study of what concepts specific learning schemes (representation and method) can and cannot learn. We do not look at this in detail, but a popular example, at first highlighted in a very powerful book by Minsky and Papert involves perceptrons. It has been mathematically verified that the above technique for learning perceptron weights will converge to a ideal classifier for learning tasks where the target concept is linearly separable.

To understand what is and what is not a linearly separable target function, we take simplest functions of all, Boolean functions. These take 2 inputs, which are either -1 or 1 and output either a -1 or a 1. Note down that, in other contexts, the values 1and 0 are used instead of 1 and -1. As an instance function, the AND Boolean function outputs a 1 only if both inputs are 1, whereas the OR function just outputs a 1 if either inputs are 1. Clearly, these relate to the connectives we studied in first order logic. The following 2 perceptrons may represent the AND and OR Boolean functions respectively:


1931_Learning Abilities of Perceptrons.png

One of the main impacts of Minsky and Papert's book was to highlight the issue that perceptions can't learn a specific boolean function called XOR. This function output 1 if the 2 inputs are not the similar. To see why XOR can't be learned, try and write down a perception to do the task. The following diagram highlights the notion of linear reparability in Boolean functions, which describe why they cannot be learned by perceptions:

In every case, we have plotted the values taken by the Boolean function when the inputs are specific values: (-1,-1) ;(1,-1);(-1,1) and (1,1). For the AND function, there is just one place where a 1 is plotted, namely when both inputs are 1. It meant that we could draw the dotted line to separate the output -1s from the 1s. We were able to draw a same line in the OR case. Because we may draw these lines, we say that these functions are linearly separable. Notice that it is impossible to draw such as line for the XOR plot: wherever you try, you never get a clear split into 1s and -1s.


396_Learning Abilities of Perceptrons1.png

The dotted lines may be seen as the threshold in perceptrons: if the weighted sum S falls below it, then the perceptron outputs 1 value, and if S falls above it, the other output is produced. It is not issue how the weights are organized; the threshold will yet be a line on the graph.  Therefore, functions which are not linearly separable can't be represented by perceptrons.

Notice that this result extends to functions over any number of variables, which may take  in  any  input,  but  which  produce  a  Boolean  output  (and  so could,  in principle be learned by a perceptron). In the following two graphs, For example, the function takes in 2 inputs (like Boolean functions), but the input may be over a range of values. The concept on the left may be learned by a perceptron, whereas the concept on the correct cannot:

As an exercise, draw in the separating (threshold) line in the left hand plot.


2105_Learning Abilities of Perceptrons2.png

Unluckily, the disclosure in Minsky and Papert's book that perceptrons can't learn even sach type of simple function was taken the incorrect way: people believed it represented a basic flaw in the use of ANNs to perform learning job. This lead to a winter of ANN research within Artificial Intelligence, which lasted over a decade. In realism, perceptrons were being studied in order to achived insights into more complexed architectures with hidden layers, which don't have the restriction that perceptrons have. No one ever advised that perceptrons would be eventually used to solve real world learning problems. luckily, people studying ANNs within other sciences (notably neuro-science) revived interest in the study of ANNs. For more facts of computational learning theory, see chapter 7 of Tom Mitchell's machine learning book.

Posted Date: 10/2/2012 6:07:36 AM | Location : United States

Related Discussions:- Learning abilities of perceptrons - ann, Assignment Help, Ask Question on Learning abilities of perceptrons - ann, Get Answer, Expert's Help, Learning abilities of perceptrons - ann Discussions

Write discussion on Learning abilities of perceptrons - ann
Your posts are moderated
Related Questions
E-Commerce is not suitable for Online job searching.

What is the purpose of DOTNET? Dot Net Is a collection of products like C#, ASp.Net, Vb.Net, XML. The purpose of .Net is that we can simply upgrade and degrade our programs whi

What are the Advantages of carrying out simulations -  saves cost (instead of doing the real thing) -  Safer (scenarios tried out on the simulation first before used in real

Q. What is Functions of Input - Output Interface? An I/O interface is bridge between processor and I/O devices. It controls data exchange between external devices and main mem

Broad Band ISDN handles data rate of about (A) 64 kbps                                      (B) 100 mbps (C) 5.4 mbps                                    (D) 2.048 mbps

THE DIFFERENCE ENGINE - Pascaline The difference enigne was based on the mathematical principle of finite differences and was used to solve calculations based on large numbers

Give some examples of malicious data.  In May 2002, the Norton Anti-Virus software for Windows operating systems detects about 61000 malicious programs. Some of them are named

Explain the techniques, which are used to gain on the number pairs. In rural areas, subscribers are usually dispersed. This is both unessential and expensive to give a dedicate

Explain Clone process. A clone process is generated using primitive type clone by duplicating its parent process. However unlike traditional processes it might be share its con