Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 2 of 2
  1. #1
    Regular Coder
    Join Date
    Jul 2004
    Posts
    133
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Subdomains and restricted access Web pages

    This topic is related to SEO; I am not sure to post it here, but i am doing that.
    I wanna ask, if some Web pages are set user login permission ( may be HTTP authentication on Apache/Linux ); then how'll SE robots/spider crawl/index the keywords? ( afterall, only valid cookie/session values will have the access! )

    Secondly, how'll SE crawl subdomain, take e.g., 'abc.def.domain.com'
    Still, webmasters use forums.domain.com mostly.
    Actually, I can't see if any robots, visits my subdomains, when I see 'Awstats log' for my Web site.
    ( SEs ask/allow for domain submission, not subdomain submission )

    So if domain.com/forums is better than forums.domain.com, they what is the need of subdomains, unless ftp access ( and main domain's images/css need relative-linking protection ) limitations are desired.

    Please discuss.

  • #2
    Regular Coder
    Join Date
    Feb 2005
    Location
    UK
    Posts
    133
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by anshul
    This topic is related to SEO; I am not sure to post it here, but i am doing that.
    I wanna ask, if some Web pages are set user login permission ( may be HTTP authentication on Apache/Linux ); then how'll SE robots/spider crawl/index the keywords? ( afterall, only valid cookie/session values will have the access! )
    If your site is passworded using .htaccess or something similar, then robots will not be able to spider the site, unless you write an exception into your authentication (i.e. allow all users who have the useragent of "GoogleBot", although that creates a bit of a security hole)

    Secondly, how'll SE crawl subdomain, take e.g., 'abc.def.domain.com'
    Still, webmasters use forums.domain.com mostly.
    Actually, I can't see if any robots, visits my subdomains, when I see 'Awstats log' for my Web site.
    ( SEs ask/allow for domain submission, not subdomain submission )

    So if domain.com/forums is better than forums.domain.com, they what is the need of subdomains, unless ftp access ( and main domain's images/css need relative-linking protection ) limitations are desired.

    Please discuss.
    Subdomains are just other domains, there's no difference (to a browser/spider etc) between forums.mydomain.com and www.mydomain.com. To get spiders to index your sub domain, just link to it like you do for any other domain. The more incoming links you have to a domain, the more pages you're going to get spidered/the better you're going to do in search engines for the pages that do get spidered.


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •