K-nearest neighbor for text classification, Computer Engineering

Assignment 2: K-nearest neighbor for text classification.

The goal of text classification is to identify the topic for a piece of text (news article, web-blog, etc.). Text classification has obvious utility in the age of information overload, and it has become a popular turf for applying machine learning algorithms. In this project, you will have the opportunity to implement k-nearest neighbor and apply it to text classification on the well known Reuter news collection.

1.       Download the dataset from my website, which is created from the original collection and contains a training file, a test file, the topics, and the format for train/test.

2.       Implement the k-nearest neighbor algorithm for text classification. Your goal is to predict the topic for each news article in the test set. Try the following distance or similarity measures with their corresponding representations.

a.        Hamming distance: each document is represented as a boolean vector, where each bit represents whether the corresponding word appears in the document.

b.       Euclidean distance: each document is represented as a numeric vector, where each number represents how many times the corresponding word appears in the document (it could be zero).

c.         Cosine similarity with TF-IDF weights (a popular metric in information retrieval): each document is represented by a numeric vector as in (b). However, now each number is the TF-IDF weight for the corresponding word (as defined below). The similarity between two documents is the dot product of their corresponding vectors, divided by the product of their norms.

3.        Let w be a word, d be a document, and N(d,w) be the number of occurrences of w in d (i.e., the number in the vector in (b)). TF stands for term frequency, and TF(d,w)=N(d,w)/W(d), where W(d) is the total number of words in d. IDF stands for inverted document frequency, and IDF(d,w)=log(D/C(w)), where D is the total number of documents, and C(w) is the total number of documents that contains the word w; the base for the logarithm is irrelevant, you can use e or 2. The TF-IDF weight for w in d is TF(d,w)*IDF(d,w); this is the number you should put in the vector in (c). TF-IDF is a clever heuristic to take into account of the "information content" that each word conveys, so that frequent words like "the" is discounted and document-specific ones are amplified. You can find more details about it online or in standard IR text.

4.       You should try k = 1, k = 3 and k = 5 with each of the representations above. Notice that with a distance measure, the k-nearest neighborhoods are the ones with the smallest distance from the test point, whereas with a similarity measure, they are the ones with the highest similarity scores.

 

 

Posted Date: 5/29/2013 5:15:54 AM | Location : United States







Related Discussions:- K-nearest neighbor for text classification, Assignment Help, Ask Question on K-nearest neighbor for text classification, Get Answer, Expert's Help, K-nearest neighbor for text classification Discussions

Write discussion on K-nearest neighbor for text classification
Your posts are moderated
Related Questions
Unlink() is a function for file system handling. It will easily delete the file in context.   Unset() is a function for variable management. It will create a variable undefin

How does CPU know that an interrupt has taken place? There needs to be a line or a register or status word in CPU which can be increased on occurrence of interrupt situation.

What are the Advantages of store program control (I) Easy to maintain (II) Easy to control (III) Wide range of services can be provided to customers (IV) Flexible

Students are needed to work in group of 3 and make an Android mobile application falling under the following categories: Multimedia o    Eg: Camera app, mp3 player, ga

a.  Sketch the excitation diagram indicating the last states and next states. b. Build the circuit using a Synchronous Counter with JK FF and NAND gates only. Replicate the circ

What is the max no of match code Id's that can be defined for one Match code object? A match code Id is a single character ID that can be a letter or a number.

We download data which arrives to me in a column. I need to use it in a complex sheet that requires data in a row. How can we convert the column data into row data?   Ans) H

Q. Write a menu driven program to find 9's and 10's complement of a decimal number using file. Perform necessary validation with proper message that entered numbers must be de

Combinatorial and Scheduling Problems: One class of problems is concerned with specifying optimal scheduled. A classical example is the Travelling Salesperson Problem where

What do you mean by system calls? System calls give the interface among a process and the operating system. When a system call is executed, it is treated as by the hardware as