Reference no: EM131422208

Assignment

1. Consider this dictionary: {"CAT", "COUNT", "DOG", "DONKEY", "ELEPHANT" }

Term-ID1	Offset
1
2
3
4
5

a) Complete this table assuming "dictionary as a string"

b) Create a second dictionary consisting of each word reversed (e.g. CAT -> TAC ). Show the dictionary as a string.

Term-ID2	Offset
1
2
3
4
5

c) Complete this table using your reversed dictionary string

d) Using your two dictionaries, show how you can determine the words that satisfy the wildcard query C*T

Consider the following documents:

Doc1: the wood table Doc2: they made the wood Doc3: the table is made of steel Doc4: wood table or steel table

 

 

 

Using a shingle size 2, compute the Jaccard coefficient of:

(Doc1, Doc2)

(Doc1, Doc3)

(Doc1, Doc4)

Based upon your results, Doc1 is most similar to ____?

1. Crawl-delay: 10
2. User-agent: crawlerbot
3. Disallow: /includes
4. Disallow: /misc
5. Disallow: /setup
6. Allow: /misc/*.jpg
7. User-agent: *
8. Disallow: /setup
2. Using this robots.txt file

a) What does line 1 mean?

b) What is the difference between line 2 and 7?

c) Should any crawler access the file /setup/help.txt?

d) Should the crawler "mybot" access the file /a/b.htm?

3. Consider the following text:

This tree is just one of many older-growth trees in the forest. Forests in Texas, can be over 100 years-old before they are considered "old". Trees can be over 200 years.

a) What punctuation can be removed to determine terms?

b) What stop words can be removed?

c) Which tokens can be converted to lower case?