Create algorithm for similarities between documents, Data Structure & Algorithms

Assignment Help:

Here is a diagram showing similarities between documents; this is an actual set of physics lab assignments from a large university.

2442_similarities between documents.png

Each node (square) in the graph is a document. Each edge (line) connects two documents based on the number of 6-word phrases they have in common. To reduce noise, only documents sharing more than 200 6-word phrases are shown. The red square is the lab manual, and the brown squares are two sample lab reports that were distributed in class. (Apparently many students 'borrowed' heavily from these documents.) But if we look carefully, we notice that several papers share a large number of phrases in common with each other and with no other documents. For example, a pair at the top left share 718, a pair at the top right show 882, and a pair on the lower left share 863 6-word phrases in common. It's likely that those people turned in essentially the same lab report or copied from each other. Your program will read through a directory full of files and identify pairs of files that share more than 200 6-word phrases, as a means of detecting copied work.

It's important to understand what we mean by "6-word phrases." It's not just looking at 6 words, then the next 6, etc.; after all, even plagiarists are smarter than that. A 6-word phrase is a word and the following 5 words, for each word with at least 5 words after it. So, for example, the text:

Now is the time for all good men to come to the aid of their country.

Contains the 6-word phrases:

Now is the time for all

is the time for all good

the time for all good men

time for all good men to

...and so on.

Thus, a single extended passage can generate many duplicates. On the other hand, a sentence that happens to begin with "Now is the time..." will be less likely to generate more than a few 'hits' as duplicate phrases. For our purposes, upper- or lower-case doesn't matter, nor does punctuation.


Related Discussions:- Create algorithm for similarities between documents

Explain binary search tree, Binary search tree. A binary search tree is...

Binary search tree. A binary search tree is a binary tree that is either empty or in which every node having a key that satisfies the following conditions: - All keys (if an

An undirected graph g with n vertices and e edges, An undirected graph G wi...

An undirected graph G with n vertices and e edges is shown by adjacency list.  What is the time required to generate all the connected components? O (e+n)

Explain how two dimensional arrays are represented in memory, Explain how t...

Explain how two dimensional arrays are represented in memory. Representation of two-dimensional arrays in memory:- Let grades be a 2-D array as grades [3][4]. The array will

Dijkstras algorithm, Djikstra's algorithm (named after it is discovered by ...

Djikstra's algorithm (named after it is discovered by Dutch computer scientist E.W. Dijkstra) resolves the problem of finding the shortest path through a point in a graph (the sour

Graph traversal schemes, Various graph traversal schemes Graph Traversa...

Various graph traversal schemes Graph Traversal Scheme. In many problems we wish to investigate all the vertices in a graph in some systematic order. In graph we often do no

Sorting, explain quick sort algorithm

explain quick sort algorithm

Write a program to create a hashed file, Write a program to create a hashed...

Write a program to create a hashed file that stores the records currently in the file " data_2013 ". Records should use the same fixed-length schema given previously, and should ag

Algorithm for the selection sort, Q. Give the algorithm for the selection s...

Q. Give the algorithm for the selection sort. Describe the behaviours of selection sort when the input given is already sorted.

An algorithm to insert a node in beginning of linked list, Q. Write down an...

Q. Write down an algorithm to insert a node in the beginning of the linked list.                         Ans: /* structure containing a link part and link part

Design and implement a software system, In this assignment, you are invited...

In this assignment, you are invited to design and implement a software system for catalogue sale. A catalogue is organised in a tree structure. Each node of the catalogue tree repr

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd