Create algorithm for similarities between documents, Data Structure & Algorithms

Assignment Help:

Here is a diagram showing similarities between documents; this is an actual set of physics lab assignments from a large university.

2442_similarities between documents.png

Each node (square) in the graph is a document. Each edge (line) connects two documents based on the number of 6-word phrases they have in common. To reduce noise, only documents sharing more than 200 6-word phrases are shown. The red square is the lab manual, and the brown squares are two sample lab reports that were distributed in class. (Apparently many students 'borrowed' heavily from these documents.) But if we look carefully, we notice that several papers share a large number of phrases in common with each other and with no other documents. For example, a pair at the top left share 718, a pair at the top right show 882, and a pair on the lower left share 863 6-word phrases in common. It's likely that those people turned in essentially the same lab report or copied from each other. Your program will read through a directory full of files and identify pairs of files that share more than 200 6-word phrases, as a means of detecting copied work.

It's important to understand what we mean by "6-word phrases." It's not just looking at 6 words, then the next 6, etc.; after all, even plagiarists are smarter than that. A 6-word phrase is a word and the following 5 words, for each word with at least 5 words after it. So, for example, the text:

Now is the time for all good men to come to the aid of their country.

Contains the 6-word phrases:

Now is the time for all

is the time for all good

the time for all good men

time for all good men to

...and so on.

Thus, a single extended passage can generate many duplicates. On the other hand, a sentence that happens to begin with "Now is the time..." will be less likely to generate more than a few 'hits' as duplicate phrases. For our purposes, upper- or lower-case doesn't matter, nor does punctuation.


Related Discussions:- Create algorithm for similarities between documents

Efficient way of storing two symmetric matrices, Explain an efficient way o...

Explain an efficient way of storing two symmetric matrices of the same order in memory. A n-square matrix array is said to be symmetric if a[j][k]=a[k][j] for all j and k. For

Determine the types of java, Determine the types of JAVA Java has two p...

Determine the types of JAVA Java has two parts... 1. Core language -- variables, arrays, objects o Java Virtual Machine (JVM) runs the core language o Core language is

Tic Tac Toe game , Book to refer: Introduction to Algorithms, 3rd Ed, by Cl...

Book to refer: Introduction to Algorithms, 3rd Ed, by Clifford Stein, Thomas H. Cormen, Ronald Rivest, Charles E. Leiserson Question: Tic Tac Toe game -Design a GUI and implement

Efficient algorithms.., implementation of fast fourier transforms for non p...

implementation of fast fourier transforms for non power of 2

Compare and contrast various sorting techniques, Q. Compare and contrast va...

Q. Compare and contrast various sorting techniques or methods with respect to the memory space and the computing time.

Linked lists - implementation, The Linked list is a chain of structures whe...

The Linked list is a chain of structures wherein each structure contains data in addition to pointer, which stores the address (link) of the next logical structure in the list.

Aa-trees, Red-Black trees have introduced a new property in the binary sear...

Red-Black trees have introduced a new property in the binary search tree that means an extra property of color (red, black). However, as these trees grow, in their operations such

Hashing and hash functions, Q. Describe the term hashing. Explain any two u...

Q. Describe the term hashing. Explain any two usually used hash functions. Explain one method of collision resolution.

Calculation of time complexity, Example: Assume the following of code: ...

Example: Assume the following of code: x = 4y + 3 z = z + 1 p = 1 As we have been seen, x, y, z and p are all scalar variables & the running time is constant irrespective

Double linked list, In a doubly linked list, also called as 2 way list, eac...

In a doubly linked list, also called as 2 way list, each node is divided into 3 parts. The first part is called previous pointer field. It contains the address of the preceding

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd