allow all bots on robot.txt?

[raptrex]

is it safe to allow all bots to search your site?
what should i not allow
im runing a forum using fluxbb and im not what sure what i shouldnt allow the bots to search

[gnomeontherun]

Well not all bots play by the rules, so they can search your entire site regardless if you put that file on your site. Most of the good search sites do play by the rules though. However I think that its good to hide administration folders, cgi-bin, logs, and any other folders that the general public doesn't see or need to see.

[newagesmb]

No problem if all spiders search your site, but we are using robot.txt is to disallow some pages, files or images of a particular site. You have the right to show in which much pages spiders want to see. We have done some works in our Web Development Company (http://www.newagesmb.com/)Site.

A file written and stored in the root directory of a website that restricts the Search Engine spiders from indexing certain pages of the website. This file is used to disallow certain spiders from seeing files that you not want them to see. You can also prevent a certain spider to look at any of the web pages through this file.