View Full Version : Trying to build a search bot, web crawler

08-16-2006, 03:06 PM
I'm trying to develop my own search engine, which will autimatically crawl the web and index the sites. Does anyone have any idea the main idea behind this method and how it can be achieved? Any software which I can start from etc.