Write a program that can facilitate searching by indexing text files according to words. In this task, you are given a large text file, sample.txt, which you will need to index the words stored in them.
To do this, you will separate out the words in the text file and index them according to their frequencies. Your program shall count the number of unique words and store them in an appropriate Standard Template Library container. The words are to be normalized to lower-case so that we do not have to deal with case-sensitivity. Your program will ignore the following:
- Punctuations
- Numerical numbers (1, 2, etc., but 'one', 'two' are to be treated as words)
Next, your program shall generate two output files, index.txt, and common.txt. At the start of the program, you shall prompt user to enter the threshold number. This number determines if the unique words are to be stored in index.txt or common.txt.
Unique words with frequency greater or equal than the threshold are to be stored in common.txt. Likewise, unique words with frequency less than the threshold are to be stored in index.txt.
As an illustration, suppose a text file, sample.txt, contains the following:
Give us a break! It is a beautiful day. We do not want to do programming today. Do you want to go to the beach with us?
At program starts:
Enter threshold number: 2
The above indicates that user enters 2 for threshold number. Your program shall generate the two output files with following content (words sorted in ascending order):
index.txt
Total words: 15
beach 1
beautiful 1
break 1
day 1
give 1
go 1
is 1
it 1
not 1
programming 1
the 1
today 1
we 1
with 1
you 1
common.txt
Total words: 5
a 2
do 3
to 3
us 2
want 2