Help, building own search engine
I have been using Google Custom Search to build search engines (MP3/AVI) but i'm tired of using them, so i want to build my own search engine but i always fail in the crawler/spider part. Can anyone point some directions? Good PHP/PERL search engines.
I know it isn't dificult to achieve but i lack of knowledge / logic.
1) visit a site,
2) get the html text,
3) extract the links,
4) If on DB, exclude or delay the index
Thanks for all the help provided
check out this project http://www.openwebspider.org/
or if you really want to build your own spider then this might be a good place to start: http://www.ibm.com/developerworks/library/l-spider/
OpenWebSpider v0.1.4 didn't work, unable to configure it properly. So i downgrade to version v0.6 because it has a tutorial in the site and i was able to create DBs, but, not able to index anything.
Any other sugestions 2012 or 2013?
This is a good one, i never tried it before but i'm sure i can get it easily...!
Thanks pytrk i must try this out i was also searching for it .
|All times are GMT +1. The time now is 01:50 PM.|
Powered by vBulletin®
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.