Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 8 of 8
  1. #1
    Regular Coder
    Join Date
    Jun 2002
    Location
    England
    Posts
    172
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Bots / Crawlers / Spiders

    Does anyone know where I can find a list of bots spiders and crawlers, and their hostnames?
    \o/

  • #2
    Regular Coder
    Join Date
    Jun 2002
    Location
    Canada
    Posts
    253
    Thanks
    0
    Thanked 0 Times in 0 Posts
    This may be a starting place...

    http://www.philsearch.de/watch.htm

  • #3
    Regular Coder
    Join Date
    Jun 2002
    Location
    NL
    Posts
    421
    Thanks
    0
    Thanked 0 Times in 0 Posts

  • #4
    Regular Coder
    Join Date
    Jun 2002
    Location
    NL
    Posts
    421
    Thanks
    0
    Thanked 0 Times in 0 Posts
    pb&j - didn't see your post. Cool list

  • #5
    Regular Coder
    Join Date
    Jun 2002
    Location
    England
    Posts
    172
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks for your replies.

    Some of those pages haven't been updated since 1998 or 2000 so not very reliable really
    \o/

  • #6
    Regular Coder
    Join Date
    Nov 2002
    Location
    Bristol, UK
    Posts
    932
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Have a look through this site. All the info you need, ever - and up-to-date!

    http://www.webmasterworld.com/home.htm

    ::] krycek [::
    ithium | SOAPI | SDP | PTPScript manual
    "ithium is a non-profit webhost, which is pretty much unique. The mission of ithium is to provide free hosting resources for worthwhile and needy non-profit projects, which otherwise may not be able to obtain such facilities. The money from commercial customers goes to maintain ithium's servers and further development."

  • #7
    Senior Coder
    Join Date
    Jun 2002
    Location
    ColoRockyz
    Posts
    1,646
    Thanks
    1
    Thanked 0 Times in 0 Posts
    What's really nice about the major SE bots is that they change their alog every 3 months. Enjoy!

    Stealth=banned
    Zoobie or not Zoobie...That is the problem.
    <body onUnload="flush( ! )">

  • #8
    Senior Coder
    Join Date
    Aug 2002
    Location
    Kansas City, Kansas
    Posts
    1,518
    Thanks
    0
    Thanked 2 Times in 2 Posts
    wat is a bot, spider, and crawler?


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •