Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 3 of 3
  1. #1
    Regular Coder
    Join Date
    Oct 2009
    Posts
    438
    Thanks
    9
    Thanked 7 Times in 7 Posts

    Create a website Crawler

    All,
    I'm looking to do something similar to how Google indexes web pages. For example, if I find a link that says Contact Us I'd like to get that information from the website. Any ideas or links out there that people have used to obtain this type of information?

    Thanks in advance!

  • #2
    New to the CF scene
    Join Date
    Dec 2011
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts
    can any one tell me the source code in php for a web crawler that crawl for all links ...thanks in advance..

  • #3
    New Coder
    Join Date
    Feb 2007
    Posts
    86
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Quote Originally Posted by ajay1831 View Post
    can any one tell me the source code in php for a web crawler that crawl for all links ...thanks in advance..
    This might get you started. This will check all the links on a page and tell you if they are good.

    http://pastebin.com/5uaRuDG2


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •