Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 5 of 5
  1. #1
    New Coder
    Join Date
    Oct 2008
    Posts
    43
    Thanks
    5
    Thanked 2 Times in 2 Posts

    Arrow Help, building own search engine

    Hi,

    I have been using Google Custom Search to build search engines (MP3/AVI) but i'm tired of using them, so i want to build my own search engine but i always fail in the crawler/spider part. Can anyone point some directions? Good PHP/PERL search engines.

    I know it isn't dificult to achieve but i lack of knowledge / logic.

    The crawler/spider
    1) visit a site,
    2) get the html text,
    3) extract the links,
    4) If on DB, exclude or delay the index

    Thanks for all the help provided

  • #2
    Regular Coder patryk's Avatar
    Join Date
    Oct 2012
    Location
    /dev/couch
    Posts
    398
    Thanks
    2
    Thanked 64 Times in 64 Posts
    check out this project http://www.openwebspider.org/
    or if you really want to build your own spider then this might be a good place to start: http://www.ibm.com/developerworks/library/l-spider/
    Last edited by patryk; 04-18-2013 at 09:57 AM.

    -------------------------------------------------------------------------------
    "Real Programmers can write assembly code in any language" - Larry Wall

  • #3
    New Coder
    Join Date
    Oct 2008
    Posts
    43
    Thanks
    5
    Thanked 2 Times in 2 Posts

    Question

    Quote Originally Posted by patryk View Post
    check out this project http://www.openwebspider.org/
    or if you really want to build your own spider then this might be a good place to start: http://www.ibm.com/developerworks/library/l-spider/
    Installed XAMMP, and have a shot with OpenWebSpider

    OpenWebSpider v0.1.4 didn't work, unable to configure it properly. So i downgrade to version v0.6 because it has a tutorial in the site and i was able to create DBs, but, not able to index anything.

    Any other sugestions 2012 or 2013?

    Thanks

  • #4
    New to the CF scene
    Join Date
    May 2013
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts
    This is a good one, i never tried it before but i'm sure i can get it easily...!

  • #5
    New Coder
    Join Date
    Mar 2013
    Posts
    26
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks pytrk i must try this out i was also searching for it .


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •