...

View Full Version : Create a website Crawler



treeleaf20
08-23-2011, 04:34 PM
All,
I'm looking to do something similar to how Google indexes web pages. For example, if I find a link that says Contact Us I'd like to get that information from the website. Any ideas or links out there that people have used to obtain this type of information?

Thanks in advance!

ajay1831
12-10-2011, 10:23 AM
can any one tell me the source code in php for a web crawler that crawl for all links ...thanks in advance..

Jinxy
12-11-2011, 12:22 AM
can any one tell me the source code in php for a web crawler that crawl for all links ...thanks in advance..

This might get you started. This will check all the links on a page and tell you if they are good.

http://pastebin.com/5uaRuDG2



EZ Archive Ads Plugin for vBulletin Copyright 2006 Computer Help Forum