View Full Version : Search Engine URL Scraper

06-24-2009, 03:39 PM
1) Project Details: (be as specific as possible):
I am looking for a URL scraper similar to the one mentioned on: http://www.codingforums.com/archive/index.php/t-156958.html

Specifically, I need to be able to upload a list of keywords. This can be done via a csv file or just copy and paste into a text field. I need to be able to process multiple keywords at a time, I don't want to do this one by one.

I need it to scrape the top 100, 250, 500 or 1,000 results (let me pick) in Google, Yahoo, MSN, Bing, Ask (let me pick which ones or all). I see that on the scraper mentioned above, it also included the country type for the search engine (.com, .ca, .co.uk, .cn, .com.au, etc) I'd like this option also for each search engine if possible.

The top 100 results should be written to the screen, and all results to a csv file that I can download.

For the output here is what I need:

1) Keyword phrase used to find the site (since multiple keywords can be searched at once via the upload, I'll need to know which keyword found the site)
2) Search Engine
3) Title of website
4) URL of website
5) Root URL of website (not sure how hard this would be)

2) Payment Amount:
Quote Please

3) Payment method/ details (Paypal, check? Timeline?):

As soon as possible

4) Additional Info (about project or potential bidders):

I'm new to this forum, but you can see on www.WickedFire.com that I pay on time for projects.

it career
06-26-2009, 08:59 AM
Are you sure it is not illegal ?

06-26-2009, 03:15 PM
Well Google provides an API, I'm sure its not too illegal but might be abusive, depending on what scale this will be implemented.