Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 10 of 10
  1. #1
    Regular Coder
    Join Date
    Jul 2002
    Location
    Earth
    Posts
    113
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Search Engines browser pwd protected sites

    How do you enable a search engine (such as google or something like that) to search pwd protected sites. Is there a code to enter into a page or something?
    Life is funny, especially when you're poor.

  • #2
    Regular Coder Feyd's Avatar
    Join Date
    May 2002
    Location
    Los Angeles, CA Maxim: Subvert Society
    Posts
    404
    Thanks
    0
    Thanked 0 Times in 0 Posts
    You don't...

    It defeats the purpose. However, you could do some overly complex things that just would be ridiculously ornate for this task...like detecting the footprint of SE robots and serving them different robots.txt file and removing the password protection so that only robots can index that page. Or backdooring your own password protected area (so why have the password protection to begin with).

    If you are talking about useless password protection (like virtually any password protection done on the client side, ie. javascript, hidden URLs, form tricks, etc.), then you might have some more options. However, if you are talking about real password protection, such as htpasswd or login forms and sessions/cookies based on login, its kind of pointless.

    Maybe if you explain why it needs to be password protected and why (at the same time) it needs to be searchable via external SEs, we can give you some better architecture philosophy on this...
    Moderator, Perl/CGI Forum
    shadowstorm.net - subvert society

  • #3
    Regular Coder
    Join Date
    Jul 2002
    Location
    Earth
    Posts
    113
    Thanks
    0
    Thanked 0 Times in 0 Posts
    why do you keep telling me that my idea and/or need is 'pointless'

    i have the pwd protection for stories those who are on my mailinglist can read. and to promote memebership (and also make it easier for those ON the mailinglist to search) I would like to have those stories indexed.

    i did however consider taking the pwd off so that the spiders could do their stuff, but no idea when google or any others would spider it.

    why have the mailinglist? because some books have adult content and that way, if they sign up to view them, they know what they are viewing.

    a simple no will be fine.

    thank you for your time.

    good day
    Life is funny, especially when you're poor.

  • #4
    Regular Coder Feyd's Avatar
    Join Date
    May 2002
    Location
    Los Angeles, CA Maxim: Subvert Society
    Posts
    404
    Thanks
    0
    Thanked 0 Times in 0 Posts
    You misunderstand...

    It is pointless to try and get password protected pages to be indexed...spiders can't index them if they are password protected...see what I mean? In order to get them to be indexed, you have to loosen up (virtually remove entirely) your password protection. It's like a person wanting to break the record for hand walking after they cut off their hands...kinda going the wrong way, sparky!

    This is why I asked what specifically you were trying to do since there may be simpler ways you have not considered...

    For instance, create a gallery of headlines/abstracts (brief descriptions) outside of your password protected directory than is public to spiders and you'll get some indexing (if you link from this gallery to the articles, spidering won't happen, but that page will show up with search possibilities on articles that haven't been searched), which could all point to your registration/membership area.
    Moderator, Perl/CGI Forum
    shadowstorm.net - subvert society

  • #5
    Regular Coder
    Join Date
    Jul 2002
    Location
    Earth
    Posts
    113
    Thanks
    0
    Thanked 0 Times in 0 Posts
    not a bad idea. i don't suppose you know php, huh? well if you do, please do not hesitate to help me out in my php topic. *grin*

    anywho, i see what you mean. that way keywords would point to key files.

    not a bad idea. i'm still looking for a cheap if not 'free' PDF indexing search engine. so far i use google but you know how that is.

    Thanks.

    I think i have a good idea of what i'm going to do.


    Life is funny, especially when you're poor.

  • #6
    Regular Coder
    Join Date
    Jun 2002
    Posts
    185
    Thanks
    0
    Thanked 0 Times in 0 Posts
    In addition to Feyd's suggestion, you could also add a search engine to your site. Most web servers have one built in, like Index Server in Microsoft IIS.

    This would allow you to index your site locally and your members could search it once they've logged in.
    Last edited by BrainJar; 07-14-2002 at 07:51 PM.

  • #7
    Senior Coder joh6nn's Avatar
    Join Date
    Jun 2002
    Location
    72 W. 48' 57" , 41 N. 32' 04"
    Posts
    1,887
    Thanks
    0
    Thanked 1 Time in 1 Post
    it's always a good idea to add a seach utility to your site ( or, well, i can imagine that there are times when this could be bad, but none occur to me at the moment ). however, that doesn't really server his purpose at the moment. he wants search engines to index his content, so that people will come to his site, but he wants people to have to sign up, to get in.

    in which case, as far as i know, Feyd's way is the only one that will work for all search engines.
    bluemood | devedge | devmo | MS Dev Library | WebMonkey | the Guide

    i am a loser geek, crazy with an evil streak,
    yes i do believe there is a violent thing inside of me.

  • #8
    Regular Coder
    Join Date
    Jul 2002
    Location
    Earth
    Posts
    113
    Thanks
    0
    Thanked 0 Times in 0 Posts
    hi, i'm a girl.


    well some of the books don't require membership, and those ones are indexed. the only thing is that some are in PDF format, so ya know, i'd need a search engine that supports that. google is the only thing that's even slightly adeqet.

    luckily.

    (

    but the idea of getting my own search engine on my site is a great one though!

    thanks a lot. if you have any other suggestions on how to make this easier for me, i'd really appreciate it.
    Life is funny, especially when you're poor.

  • #9
    Senior Coder
    Join Date
    Jun 2002
    Location
    near Oswestry
    Posts
    4,508
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Atomz search can index PDF documents

  • #10
    Regular Coder
    Join Date
    Apr 2007
    Location
    Griffith. Australia
    Posts
    138
    Thanks
    4
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Feyd View Post
    spiders can't index them if they are password protected...see what I mean? In order to get them to be indexed, you have to loosen up (virtually remove entirely) your password protection.
    For the purposes of any other poor sod wanting to know if anything has changed in recent years it has. Namely with Google but if you use Google Webmaster Tools you can specify your site to be indexed and you can also provide the login details for the site to index password protected pages. It is still not recommended as providing a publicly accessible list of your protected pages is like cutting yourself shaving while swimming in the shark tank with a gallon of chum down your jocks but the service is still available through Google. It supports HTTP authentication, Form based authentication as well as cookie and URL based authentication.


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •