Entropy - learning decision trees, Computer Engineering

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."

Posted Date: 1/11/2013 6:38:23 AM | Location : United States

Related Discussions:- Entropy - learning decision trees, Assignment Help, Ask Question on Entropy - learning decision trees, Get Answer, Expert's Help, Entropy - learning decision trees Discussions

Write discussion on Entropy - learning decision trees
Your posts are moderated
Related Questions
What are the differences between a Database index and a match code? Match code can have fields from several tables whereas an index can have fields from only one table.

Give the difference between multiprogramming and multiprocessing. A multiprocessing system is a computer hardware configuration which contains more than one independent proc

Instruction Issue degree : The major concept in superscalar processing is how many instructions we can issue per cycle. If we issue k number of instructions per cycle in a supersca

Explain SR Latch with NAND Gate? SR Latch has two useful states: Set state, when output Q=1 and Q'=0. Reset state, when output Q=0 and Q'=1.Output Qand Q' are normally

What are value types and reference types?  Value type - bool, byte, chat, decimal, double, enum, float, int, long, sbyte, short, strut, uint, ulong, ushort. Value types are sto

CIM and Holonic Manufacturing System The improvement of computer integrated manufacturing has demonstrated that the automation has the same potential such that of computers in

Erlang is used to (A) Measure busy period                (B) Give total busy period in minutes (C)  Measure average call rate       (D) Indicate total call period Ans

Introduction to object oriented Analysis & design: tools In case of OOAD, Unified Modelling Language (UML) is a well accepted language. It is used for constructing, visualizin

If for a short period of time circuits goes to some dissimilar logic level then it is assumed to have then it is known as static hazard e.g. If the final logic value of output of a

c program??????????