K-nearest neighbor for text classification, Computer Engineering

Assignment Help:

Assignment 2: K-nearest neighbor for text classification.

The goal of text classification is to identify the topic for a piece of text (news article, web-blog, etc.). Text classification has obvious utility in the age of information overload, and it has become a popular turf for applying machine learning algorithms. In this project, you will have the opportunity to implement k-nearest neighbor and apply it to text classification on the well known Reuter news collection.

1.       Download the dataset from my website, which is created from the original collection and contains a training file, a test file, the topics, and the format for train/test.

2.       Implement the k-nearest neighbor algorithm for text classification. Your goal is to predict the topic for each news article in the test set. Try the following distance or similarity measures with their corresponding representations.

a.        Hamming distance: each document is represented as a boolean vector, where each bit represents whether the corresponding word appears in the document.

b.       Euclidean distance: each document is represented as a numeric vector, where each number represents how many times the corresponding word appears in the document (it could be zero).

c.         Cosine similarity with TF-IDF weights (a popular metric in information retrieval): each document is represented by a numeric vector as in (b). However, now each number is the TF-IDF weight for the corresponding word (as defined below). The similarity between two documents is the dot product of their corresponding vectors, divided by the product of their norms.

3.        Let w be a word, d be a document, and N(d,w) be the number of occurrences of w in d (i.e., the number in the vector in (b)). TF stands for term frequency, and TF(d,w)=N(d,w)/W(d), where W(d) is the total number of words in d. IDF stands for inverted document frequency, and IDF(d,w)=log(D/C(w)), where D is the total number of documents, and C(w) is the total number of documents that contains the word w; the base for the logarithm is irrelevant, you can use e or 2. The TF-IDF weight for w in d is TF(d,w)*IDF(d,w); this is the number you should put in the vector in (c). TF-IDF is a clever heuristic to take into account of the "information content" that each word conveys, so that frequent words like "the" is discounted and document-specific ones are amplified. You can find more details about it online or in standard IR text.

4.       You should try k = 1, k = 3 and k = 5 with each of the representations above. Notice that with a distance measure, the k-nearest neighborhoods are the ones with the smallest distance from the test point, whereas with a similarity measure, they are the ones with the highest similarity scores.

 

 


Related Discussions:- K-nearest neighbor for text classification

Define the key features of hop field neural networks, a. Define the Key fea...

a. Define the Key features of Hop field Neural Networks. b. Compare and contrast among Neural Networks and Expert systems in terms of knowledge representation, acquisition and e

Define formal language grammar, Define formal language grammar? A forma...

Define formal language grammar? A formal language grammar is a set of formation rules which describe that strings made by the alphabet of a formal language are syntactically va

What do you mean by true complement method, Q. What do you mean by True Com...

Q. What do you mean by True Complement Method? Explain in detail. Q. Show IEEE format for (12.75)10. Q. What are the various ways to represent Negative Numbers in computer sy

Explain status and control registers, Q. Explain Status and Control Registe...

Q. Explain Status and Control Registers? For control of numerous operations several registers are used. These registers can't be used in data manipulation though content of som

Define e-cash, What do you understand by E-cash? E-Cash Ecash is ...

What do you understand by E-cash? E-Cash Ecash is a cash which is showed by two models. One is the on-line form of e-cash which permits for the completion of all types of

How a shift register can be used as a ring counter, Explain how a shift reg...

Explain how a shift register can be used as a ring counter giving the wave forms at the output of the flipflops. Ans: Shift Register as a Ring Counter: A Ring Counter is a

Performance and issues in pipelining-speedup, Performance and Issues in Pip...

Performance and Issues in Pipelining Speedup:   Firstly, we get the speedup factor that is we see how much speed up performance we get through pipelining. First we get the

Determine the object oriented principles, Determine the Object oriented pri...

Determine the Object oriented principles Many latest applications are being developed based on object oriented principles such as methods, classes, and inheritance. To fulfil

How do you make an image clickable in HTML, How do you make an image clicka...

How do you make an image clickable in HTML? Give an example. To create an image or text clickable hyperlinks are used, that use the and tags. Such tag has different paramet

Generation of erp products, 1. How does this new generation of ERP products...

1. How does this new generation of ERP products differ from traditional solutions? 2. What are the major external forces driving competition in the ERP industry? Cloud comput

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd