Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 9 of 9
  1. #1
    Supreme Master coder! Philip M's Avatar
    Join Date
    Jun 2002
    Location
    London, England
    Posts
    17,730
    Thanks
    202
    Thanked 2,508 Times in 2,486 Posts

    Question Blank HTTP_REFERER

    I hope I have posted this in the right forum.

    On my site I have an on-line order form which transmits using
    a modified version of Formmail.

    For security reasons, if a form is submitted with the HTTP_REFERER blank (or not my domain) then access is denied and an error message is returned. I also record details of submissions (successful or otherwise) in an error file. (Some failed submissions arise because the form is not correctly completed).

    Here is a fragment of the PERL script:-

    open (ERRORFILE, '>>Errors.fil' );
    print ERRORFILE ("$date\n");
    print ERRORFILE ("$ENV{'HTTP_REFERER'}\n");
    print ERRORFILE ("$ENV{'REMOTE_ADDR'}\n");
    print ERRORFILE ("$ENV{'REMOTE_HOST'}\n");
    print ERRORFILE ("$ENV{'HTTP_USER_AGENT'}\n\n");
    close (ERRORFILE);

    Examining the Errorfile shows that sometimes customers fill in the
    form properly but access is denied them as the HTTP-REFERER is blank.

    Here is an example:-

    Sunday, October 6, 2002 at 19:41:58

    62.252.224.8
    cache5-leed.server.ntli.net
    Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; Hotbar 4.1.4.0; .NET CLR 1.0.3705)

    Access was denied and an error message screen was displayed.

    The customer then did something to correct the situation and re-submitted his
    form successfully:-

    Sunday, October 6, 2002 at 19:42:24
    http://www.mydomain.co.uk/orderform.html
    62.252.224.8
    cache5-leed.server.ntli.net
    Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; Hotbar 4.1.4.0; .NET CLR 1.0.3705)

    Note that it took only 26 seconds for the customer to read the error message screen (several lines of text), and do whatever he did to reinstate the HTTP_REFERER.

    My question is:-

    a) How does it arise that the HTTP_REFERER in the browser
    address box can be a blank?

    b) What did my customer do to re-instate this in his browser?

    Is it something to do with the browser configuration, a firewall, or
    anti-virus program?

    Any comments or advice would be appreciated.
    Last edited by Philip M; 10-08-2002 at 07:18 PM.

  • #2
    Regular Coder Feyd's Avatar
    Join Date
    May 2002
    Location
    Los Angeles, CA Maxim: Subvert Society
    Posts
    404
    Thanks
    0
    Thanked 0 Times in 0 Posts
    It is probably a firewall or browser privacy standards. HTTP_REFERER is contained within the browser and you shouldn't ever really rely on it in order to perform an action, at the very least you should have the ability to detect if there is no refererer and use 'not referred' or something similiar instead of erroring out.

    As for 're-instating' it, that seems a bit odd...does your error page alter the referer variable with other information onerror? Is a form re-presented to the user that does not contain this check? Maybe the user turned off their firewall in order to process the request (unlikely). It might help to get a little more information...
    Moderator, Perl/CGI Forum
    shadowstorm.net - subvert society

  • #3
    Supreme Master coder! Philip M's Avatar
    Join Date
    Jun 2002
    Location
    London, England
    Posts
    17,730
    Thanks
    202
    Thanked 2,508 Times in 2,486 Posts
    Feyd - I now learn that there are browsers such as Opera and "privacy tools" such as Webwasher where the user can elect not to transmit the referer URL (why should he wish to do this???) but the user can quickly switch this feature off. I guess this is what has happened here.

    On my site I found that someone was stealing my bandwith and using my formmail.pl to send spam. I therefore altered the script so as to only accept form submissions from a defined referer page (not a blank), and send mail only to one address (mine!). this has cured the problem. My ISP has made all users upgrade to formmail.pl version 1.92 as the previous version had security flaws in it.

  • #4
    New to the CF scene
    Join Date
    Oct 2002
    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hi


    REFERER = a embeded string sent by the browser, stating the last page the browser viewed!


    If you trusting [REFERER] for any reason that is important, you should not, because it can be faked very easy!



    1. Some browsers limit access to not allow REFERER to be passed!

    2. Type a address in the address bar will not pass the REFERER

    3. open a new browser window will not pass the REFERER, because REFERER = NULL


    F!

  • #5
    Regular Coder
    Join Date
    Sep 2002
    Location
    self.location
    Posts
    181
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Philip, you're right about Opera... I believe recent Mozilla builds can also withhold the Referer but don't quote me on that (going by memory).

    As Feyd pointed out, firewalls can also suppress the Referer, as does Norton's Personal Firewall.

    "(why should he wish to do this???)" - because some users do not want to be tracked across domains.

  • #6
    New Coder
    Join Date
    Oct 2002
    Location
    Lancaster, UK
    Posts
    26
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Philip M,

    It's VERY easy to spoof the HTTP_REFERER var and therefore fool your script.

    1 easy way:

    open telnet, and try the following:

    telnet www.yoursite.com 80 (press return)
    GET /index.htm HTTP/1.0 (press return)
    Referer: http://www.anyurlyouwant.com (press return)
    (press return again)

    You'll see the referrer you inputted in your access_log.

    It's much more secure to hard code the email address you want to send the form to into the script, and don't bother about checking the referrer.

    And yes, I know this from experience

  • #7
    Supreme Master coder! Philip M's Avatar
    Join Date
    Jun 2002
    Location
    London, England
    Posts
    17,730
    Thanks
    202
    Thanked 2,508 Times in 2,486 Posts
    Originally posted by toolkit
    Philip M,

    It's VERY easy to spoof the HTTP_REFERER var and therefore fool your script.

    1 easy way:

    open telnet, and try the following:

    telnet www.yoursite.com 80 (press return)
    GET /index.htm HTTP/1.0 (press return)
    Referer: http://www.anyurlyouwant.com (press return)
    (press return again)

    You'll see the referrer you inputted in your access_log.

    It's much more secure to hard code the email address you want to send the form to into the script, and don't bother about checking the referrer.

    And yes, I know this from experience

    Thanks, Toolkit. You have explained how to spoof the referer, (which I did not know about) and have to say that I am well aware of the need to hard code the recipient of the email into the perl script.

    But I am still not entirely clear why my perfectly ordinary customer(s) who are not computer experts or Telnet users, nor are they trying to spoof anything, can end up with a blank referer (and hence have their form submissions rejected).

    Yes, Opera offers the option of masking the referer, but they are not using Opera! Likewise Webwasher or similar 'privacy tools' as far as I can see.

  • #8
    Regular Coder
    Join Date
    Jul 2002
    Location
    London, UK
    Posts
    126
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hi Philip M,

    If the referrer variable is blank, it's probably because a user wasn't referred.

    If a user was just to type in the address of your form without being referred to it via a link then the browser would not have any HTTP_REFERER header to send.

    Also, as pointed out, there are some browsers which will not provide the header, and some firewalls that will block it.

    Also, don't assume that your user wasn'ta computer expert.. he may have known enough to realise what was going wrong and sort it out.

  • #9
    Supreme Master coder! Philip M's Avatar
    Join Date
    Jun 2002
    Location
    London, England
    Posts
    17,730
    Thanks
    202
    Thanked 2,508 Times in 2,486 Posts
    Originally posted by Mouldy_Goat
    Hi Philip M,

    If the referrer variable is blank, it's probably because a user wasn't referred.

    If a user was just to type in the address of your form without being referred to it via a link then the browser would not have any HTTP_REFERER header to send.

    Thanks, Mouldy_Goat! I am now in an area where I am woefully ignorant, but I have to say that in fact my form submits fine when I simply type in the form's URL as opposed to reach it via the index or another page.

    Surely the referer is the page (URL) of the form itself?


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •