Create algorithm for similarities between documents, Data Structure & Algorithms

Assignment Help:

Here is a diagram showing similarities between documents; this is an actual set of physics lab assignments from a large university.

2442_similarities between documents.png

Each node (square) in the graph is a document. Each edge (line) connects two documents based on the number of 6-word phrases they have in common. To reduce noise, only documents sharing more than 200 6-word phrases are shown. The red square is the lab manual, and the brown squares are two sample lab reports that were distributed in class. (Apparently many students 'borrowed' heavily from these documents.) But if we look carefully, we notice that several papers share a large number of phrases in common with each other and with no other documents. For example, a pair at the top left share 718, a pair at the top right show 882, and a pair on the lower left share 863 6-word phrases in common. It's likely that those people turned in essentially the same lab report or copied from each other. Your program will read through a directory full of files and identify pairs of files that share more than 200 6-word phrases, as a means of detecting copied work.

It's important to understand what we mean by "6-word phrases." It's not just looking at 6 words, then the next 6, etc.; after all, even plagiarists are smarter than that. A 6-word phrase is a word and the following 5 words, for each word with at least 5 words after it. So, for example, the text:

Now is the time for all good men to come to the aid of their country.

Contains the 6-word phrases:

Now is the time for all

is the time for all good

the time for all good men

time for all good men to

...and so on.

Thus, a single extended passage can generate many duplicates. On the other hand, a sentence that happens to begin with "Now is the time..." will be less likely to generate more than a few 'hits' as duplicate phrases. For our purposes, upper- or lower-case doesn't matter, nor does punctuation.


Related Discussions:- Create algorithm for similarities between documents

Complexity of quick sort, Q. What do you mean by the best case complexity o...

Q. What do you mean by the best case complexity of quick sort and outline why it is so. How would its worst case behaviour arise?

What is a spanning tree of a graph, What is a Spanning tree of a graph? ...

What is a Spanning tree of a graph? A Spanning Tree is any tree having of vertices of graph tree and some edges of graph is known as a spanning tree.

Efficiency of linear search, Efficiency of Linear Search How much numbe...

Efficiency of Linear Search How much number of comparisons is there in this search in searching for a particular element? The number of comparisons based upon where the reco

Explain dijkstra''s algorithm, Explain Dijkstra's algorithm Dijkstra's ...

Explain Dijkstra's algorithm Dijkstra's algorithm: This problem is concerned with finding the least cost path from an originating node in a weighted graph to a destination node

File organization, Define File organization''s and it''s types

Define File organization''s and it''s types

All pairs shortest paths algorithm, In the last section, we discussed regar...

In the last section, we discussed regarding shortest path algorithm that starts with a single source and determines shortest path to all vertices in the graph. In this section, we

Size of stack, The size of stack was declared as ten. Thus, stack cannot ho...

The size of stack was declared as ten. Thus, stack cannot hold more than ten elements. The major operations which can be performed onto a stack are push and pop. However, in a prog

Construct a minimum spanning tree, Construct G for α, n, and W given as com...

Construct G for α, n, and W given as command line parameters. Throw away edges that have an asymmetric relation between nodes. That is, if A is connected to B, but B is not connect

Comparisions and assignments in worst case, Q. Calculate that how many key ...

Q. Calculate that how many key comparisons and assignments an insertion sort makes in its worst case?        Ans: The worst case performance occurs in insertion

Big o notation, This notation gives an upper bound for a function to within...

This notation gives an upper bound for a function to within a constant factor. Given Figure illustrates the plot of f(n) = O(g(n)) depend on big O notation. We write f(n) = O(g(n))

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd