Avoiding Over fitting - Artificial intelligence

As  we  discussed  in  the last  lecture,  over fitting  is  a  normal  problem  in machine learning. Decision trees suffer from that, because they are skilled to stop when  they  have  completely  classify  all  the  training  data,  i.e.,  every  branch  is complete  just  far sufficient to  suitably  categories  the  examples important to  the branch. Various ways to overcoming over fitting in decision trees have been used. As summarized by Tom Mitchell, these attempts fit into two types:

·       Stop growing the tree before it reaches excellence.

• Allow the tree to complete grow, and then post-prune some of the branches from it.

The second approach has been found to be more useable in practice. Both methods boil down to the question of shaping the right tree size. See Chapter 3 of Tom Mitchell's book for more detailed explanation of over fitting  prevention in decision tree knowledge.