Entropy - learning decision trees, Computer Engineering

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."

Posted Date: 1/11/2013 6:38:23 AM | Location : United States







Related Discussions:- Entropy - learning decision trees, Assignment Help, Ask Question on Entropy - learning decision trees, Get Answer, Expert's Help, Entropy - learning decision trees Discussions

Write discussion on Entropy - learning decision trees
Your posts are moderated
Related Questions
How to manage the web based projects? Many project management applications contain additional functions useful in the management of group projects. These features may contain g

Suppose that the working register W contains the value 0x4F, the register FSR contains the value 0x2B; the register with address 0x2B contains 0x2F and the instruction ADDWF INDF,

Design two matlab algorithms for enlarging the 256x256 images into 512x512 images by using bilinear and bicubic interpolations   a)  Evaluate the interpolated images with the

For a non blocking crossbar configuration, taking N as the number of subscribers, there will be               number of cross points and _             number of switches for est

What is a file? A file is a named collection of related information that is recorded on secondary storage. A file having either programs or data. A file has sure "structure" b

Disadvantages of random scan display - Just by wire-frame, it is almost impossible to create images with shaded objects or areas filled with a given colour. - In case

Write about TSR TPA also holds TSR (terminate and stay resident) programs which remain in memory in an active state until activated by a hot-key sequence or another event like

Q. Explain about Programmable Logic Array? Until now individual gates are considered as fundamental building blocks from that different logic functions can be derived. With the

Explain the differences between macros and subroutines. Macros Vs Subroutines (i) Macros are pre-processor directives which are processed before the source program is pass

Q. Advantages of using clusters? Parallel and distributed applications Decreased turnaround time Balanced loads Utilization of more powerful hosts Access to