K-nearest neighbor for text classification, Computer Engineering

Assignment Help:

Assignment 2: K-nearest neighbor for text classification.

The goal of text classification is to identify the topic for a piece of text (news article, web-blog, etc.). Text classification has obvious utility in the age of information overload, and it has become a popular turf for applying machine learning algorithms. In this project, you will have the opportunity to implement k-nearest neighbor and apply it to text classification on the well known Reuter news collection.

1.       Download the dataset from my website, which is created from the original collection and contains a training file, a test file, the topics, and the format for train/test.

2.       Implement the k-nearest neighbor algorithm for text classification. Your goal is to predict the topic for each news article in the test set. Try the following distance or similarity measures with their corresponding representations.

a.        Hamming distance: each document is represented as a boolean vector, where each bit represents whether the corresponding word appears in the document.

b.       Euclidean distance: each document is represented as a numeric vector, where each number represents how many times the corresponding word appears in the document (it could be zero).

c.         Cosine similarity with TF-IDF weights (a popular metric in information retrieval): each document is represented by a numeric vector as in (b). However, now each number is the TF-IDF weight for the corresponding word (as defined below). The similarity between two documents is the dot product of their corresponding vectors, divided by the product of their norms.

3.        Let w be a word, d be a document, and N(d,w) be the number of occurrences of w in d (i.e., the number in the vector in (b)). TF stands for term frequency, and TF(d,w)=N(d,w)/W(d), where W(d) is the total number of words in d. IDF stands for inverted document frequency, and IDF(d,w)=log(D/C(w)), where D is the total number of documents, and C(w) is the total number of documents that contains the word w; the base for the logarithm is irrelevant, you can use e or 2. The TF-IDF weight for w in d is TF(d,w)*IDF(d,w); this is the number you should put in the vector in (c). TF-IDF is a clever heuristic to take into account of the "information content" that each word conveys, so that frequent words like "the" is discounted and document-specific ones are amplified. You can find more details about it online or in standard IR text.

4.       You should try k = 1, k = 3 and k = 5 with each of the representations above. Notice that with a distance measure, the k-nearest neighborhoods are the ones with the smallest distance from the test point, whereas with a similarity measure, they are the ones with the highest similarity scores.

 

 


Related Discussions:- K-nearest neighbor for text classification

Which instruction is use in each process before proceeding, Before proceedi...

Before proceeding with its execution, each process must acquire all the resources it needs is called ? Ans. Hold and Wait is requires in each process should acquire all the res

What is the difference among choice and list, A Choice is displayed in a co...

A Choice is displayed in a compact form that needs you to pull it down to see the list of available choices and only one item may be selected from a choice. A List may be shown in

Characteristics of a pipeline, What are the main characteristics of a pipel...

What are the main characteristics of a pipeline? Ans: The main characteristics of a pipeline are: a) The speedup or efficiency gain by suing a pipeline depends on the numbe

What is the difference among conventional and HD T.V, a. Write down the sal...

a. Write down the salient difference among conventional T.V. and High Definition TV (HDTV) and also highlight the various benefits of digital representation for video. b. Video

Give the solutions for critical section problem, Give the solutions for cri...

Give the solutions for critical section problem? A solution to the critical section problem must specify the subsequent requirements. (a) Mutual exclusion (b) Progress (c)

Automated reasoning - first-order logic, Automated Reasoning - first-order ...

Automated Reasoning - first-order logic: The topic known as "Automated Reasoning" in "AI"concentrates mostly on deductive reasoning, here new facts are logically deduced from

Linux, Discuss the risks of having a single root user and how more limited ...

Discuss the risks of having a single root user and how more limited management abilities can be given to others users on Linux/UNIX systems

Quantitative analysis, solve the primal problem using duality and determine...

solve the primal problem using duality and determine the primal and dual solution P= 300x1 + 300x2 S.T. 2x1 + 3x2 => 13 3x1 = 2x2 => 15 x1,x2 => 0

What is rom, What is ROM? Read only memory [ROM] is used for storing pr...

What is ROM? Read only memory [ROM] is used for storing programs that are permanently resident in the computer and for tables of constants that do not change in value as the pr

Explain the fundamentals of c programming, Explain the Fundamentals of C Pr...

Explain the Fundamentals of C Programming? The C is a general purpose structured programming language. The C was developed and first implemented by the Dennis Ritchie at Bell L

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd