Create a website Crawler
I'm looking to do something similar to how Google indexes web pages. For example, if I find a link that says Contact Us I'd like to get that information from the website. Any ideas or links out there that people have used to obtain this type of information?
Thanks in advance!