K-nearest neighbor for text classification, Computer Engineering

Assignment Help:

Assignment 2: K-nearest neighbor for text classification.

The goal of text classification is to identify the topic for a piece of text (news article, web-blog, etc.). Text classification has obvious utility in the age of information overload, and it has become a popular turf for applying machine learning algorithms. In this project, you will have the opportunity to implement k-nearest neighbor and apply it to text classification on the well known Reuter news collection.

1.       Download the dataset from my website, which is created from the original collection and contains a training file, a test file, the topics, and the format for train/test.

2.       Implement the k-nearest neighbor algorithm for text classification. Your goal is to predict the topic for each news article in the test set. Try the following distance or similarity measures with their corresponding representations.

a.        Hamming distance: each document is represented as a boolean vector, where each bit represents whether the corresponding word appears in the document.

b.       Euclidean distance: each document is represented as a numeric vector, where each number represents how many times the corresponding word appears in the document (it could be zero).

c.         Cosine similarity with TF-IDF weights (a popular metric in information retrieval): each document is represented by a numeric vector as in (b). However, now each number is the TF-IDF weight for the corresponding word (as defined below). The similarity between two documents is the dot product of their corresponding vectors, divided by the product of their norms.

3.        Let w be a word, d be a document, and N(d,w) be the number of occurrences of w in d (i.e., the number in the vector in (b)). TF stands for term frequency, and TF(d,w)=N(d,w)/W(d), where W(d) is the total number of words in d. IDF stands for inverted document frequency, and IDF(d,w)=log(D/C(w)), where D is the total number of documents, and C(w) is the total number of documents that contains the word w; the base for the logarithm is irrelevant, you can use e or 2. The TF-IDF weight for w in d is TF(d,w)*IDF(d,w); this is the number you should put in the vector in (c). TF-IDF is a clever heuristic to take into account of the "information content" that each word conveys, so that frequent words like "the" is discounted and document-specific ones are amplified. You can find more details about it online or in standard IR text.

4.       You should try k = 1, k = 3 and k = 5 with each of the representations above. Notice that with a distance measure, the k-nearest neighborhoods are the ones with the smallest distance from the test point, whereas with a similarity measure, they are the ones with the highest similarity scores.

 

 


Related Discussions:- K-nearest neighbor for text classification

Determine a positive logic system logic state level, In a positive logic sy...

In a positive logic system, logic state 1 corresponds to ? Ans. For positive digital logic, we choose two voltages levels. Higher voltage shows logic 1 and a lower voltage sho

Name some of synthesizable and non-synthesizable constructs, Can you list o...

Can you list out some of synthesizable and non-synthesizable constructs? not synthesizable->>>> initial ignored for synthesis. delays  ignored for synthesis. ev

Er diagrams, how can I draw er diagram for sell & storage section of a drug...

how can I draw er diagram for sell & storage section of a drugstore?

Explain management information base variables, Explain MIB (Management Info...

Explain MIB (Management Information Base) variables. MIB is a set of named items which an SNMP agent knows. To control or monitor a remote computer, a manager should fetch or s

Illustrate the execute cycle, Q. Illustrate the Execute Cycle? The fetc...

Q. Illustrate the Execute Cycle? The fetch and indirect cycles include a small, fixed sequence of micro-operations. Every one of these cycles has fixed sequence of micro-operat

Subtract the numbers using 1's complements, Subtraction 11001-10110 using 1...

Subtraction 11001-10110 using 1's complement Ans. 11001 - 10110 01001 is 1' s Compliment of 10110, so     1 1 0 0 1 +  0 1 0 0 1 ------------------    1 0 0 0 1 0    Add 1

Functions to remove common walls, We must also be able to remove common wal...

We must also be able to remove common walls between two cells. Write the function removewalls that accepts two cells and removes the wall that is common between the two (hint: any

Receive a message, Receive a message. tid is integer task identifier of tra...

Receive a message. tid is integer task identifier of transmitting process supplied by user and msgtag is message tag supplied by user( must be non negative integer). The process re

List-processing without using suppress-dialog, What happens if we use Leave...

What happens if we use Leave to list-processing without using Suppress-Dialog? If we don't use Suppress-Dialog to next screen will be viewed but as empty, when the user presse

Homework, Use linear activation function and Hebbian learning for a SLFF ne...

Use linear activation function and Hebbian learning for a SLFF network to attempt to learn the following two sets of patterns. Why can the patterns be learned, or why can they not

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd