Need help in file search engine crawler, PHP Web Programming

Assignment Help:

Need help in File search engine crawler

Want a web crawler to gather data on an continuous basis from different file hosting services and filter it.

1) Data will be used to provide a files search engine to users. I don't want a clone of any similar kind website but infect something better than that one.

2) Process of data get-together will be a continuous process and will be running on the servers 24x7.

3) Data must be properly filtered to erase duplicate entries and new entries of the same data should merge.

I want the crawler itself part of the websites

It has to crawl multiple websites through given xml feed. Crawler must detect all links to files on rapidshare.com, 4shared, uploading as well as all other file sharing hosts. After detection it must add those links found to some kind of database (prefer MySQL) with addition information found about file Like Meta description, size, title, date if available.

Desired Skills are PHP5, cURL, MySQL Programming


Related Discussions:- Need help in file search engine crawler

Definition lists, Another type of list is a definition list. Definition lis...

Another type of list is a definition list. Definition lists contain a heading & the text show below that. IGNOU <

Basics of hypertext markup language, In the form of hypermedia documents, W...

In the form of hypermedia documents, Web pages or materials accessed by the Internet can be located anywhere in the world. Regardless of where they originated, most of the Web d

Fix problem with html5 contenteditable field, Fix problem with html5 conten...

Fix problem with html5 contentEditable field I have a issue with a modest contentEditable field in html5 that I want to fix 1) If I establish a contentEditable field in html5

Want support to make membership opt-in page, Want support to make Membershi...

Want support to make Membership Opt-in Page When fresh members sign up for Femworking membership they do consequently through a PayPal button. It works great however what I'm mi

I desire help for website development, website development 1. You want t...

website development 1. You want to have ideal about different time zone at the world, instance, USA have few different time zone, Los Angeles, New York, and USA change time twic

Need help for a proposal for electronic medical record, Proposal for electr...

Proposal for electronic medical record I'm development an online electronic medical record custody application for a third country as well as need a proposal for funding. I want

Want simple/basic word press website created, Want SIMPLE/BASIC Word Press ...

Want SIMPLE/BASIC Word Press Website Created In want of a US based freelancer to create a very simple/basic word press website in the next 24 hours. Brand Identity completed as

How can images be optimized for use on web pages, Question: (a) Write ...

Question: (a) Write the HTML tags for the following text: Software List A. Module 1 • Adobe Illustrator CS3 B. Module 2 • Adobe Photoshop CS3 C. Module 3 • Maya •

Network layer of osi model, This layer is responsible for routing a packet ...

This layer is responsible for routing a packet in the subnet i.e., from source to destination nodes across multiple nodes in the similar network or across multiple networks. This l

Develop a javascript code to generate a pull down menu box, 1. Develop...

1. Develop a program to display a multiplication table. 2. Develop a code to generate a scrolling text in a text box. 3. Develop a JavaScript code to generate

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd