Entropy - learning decision trees, Computer Engineering

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."

Posted Date: 1/11/2013 6:38:23 AM | Location : United States







Related Discussions:- Entropy - learning decision trees, Assignment Help, Ask Question on Entropy - learning decision trees, Get Answer, Expert's Help, Entropy - learning decision trees Discussions

Write discussion on Entropy - learning decision trees
Your posts are moderated
Related Questions
What are the lists of signal available? Terminating and suspending method Physical circumstances Available for the Programmer Fault in power supply

Von Neumann architecture was first majoranticipated structure for general-purpose computer. However before consideringmain components of von Neumann architecture let us first elabo

a ship is loaded with a stock of 3 items each unit of item n has a wt wn and vn the max cargo wt the ship can take is the details of 3items are item(n):1 2 3 wt(wn):2 3 1 value(vn)

1.  The network shown in figure 2 uses neurons with:             (a) Unipolar Binary;             (b) Bipolar Binary. Calculate a table of responses to all four possi

What is Parsing? Parsing: It is the process of analyzing a text, made of an order of tokens, to find out its grammatical structure regarding a given formal grammar. Parsing

Q. Illustrate Programming Based on Message Passing? Since we know programming model based on message passing employs high level programming languages such as C/C++ along with a

what are the questionnaries and observation of work site for banking system?

What are the Kernel threads Kernel threads are supported directly by the operating system .Thread creation; scheduling and management are done by the operating system. Thus the

What is independent process? A process is independent it cannot affect or be affected by the other processes implementing in the system. Any process does not share data with ot

Arithmetic Pipelines The method of pipelining can be applied to a variety of complex and slow arithmetic operations to speed up the processing time. The pipelines used for arit