Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 9 of 9
  1. #1
    New Coder
    Join Date
    Jun 2012
    Posts
    47
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Escaping text (characters) that can potentially be a security risk

    I would like bit of advice regarding escaping text (characters) that can potentially be a security risk.
    When dealing with a typical website form containing a comment box or (multi-line textarea).

    One tactic is to create a white list of allowable characters –

    PHP Code:
    preg_match("/^[a-zA-Z0-9!?£%@\-,.&;: ]{1,1000}$/"$comment_box
    however this means if someone uses an apostrophe or quote marks the form submission will fail.
    So, is it better to allow such characters and replace them with empty space before they can do damage.

    PHP Code:
    str_replace(array("\\n""\\r""'""\""), ''$_POST["comment_box"]); 
    Also is it wise to increase the chance of success by informing the user via a tool tip exactly what characters are allowed?

    I have done lots of research, however I have found that opinions tend to be contradictory.

  2. #2
    Master Coder Dormilich's Avatar
    Join Date
    Jan 2010
    Location
    Behind the Wall
    Posts
    5,801
    Thanks
    26
    Thanked 602 Times in 595 Posts
    I would like bit of advice regarding escaping text (characters) that can potentially be a security risk.
    You escape the text according to the context change. e.g. htmlspecialchars() for inserting into HTML, mysqli_real_escape_string() for inserting into a database via mysqli, etc.

    There is no single method that fits it all.
    The computer is always right. The computer is always right. The computer is always right. Take it from someone who has programmed for over ten years: not once has the computational mechanism of the machine malfunctioned.
    André Behrens, NY Times Software Developer

  3. #3
    Senior Coder deathshadow's Avatar
    Join Date
    Feb 2016
    Location
    Keene, NH
    Posts
    3,319
    Thanks
    4
    Thanked 477 Times in 465 Posts
    Quote Originally Posted by Dormilich View Post
    mysqli_real_escape_string() for inserting into a database via mysqli, etc.
    ... and really if you 'need' to do string escaping for SQL, you're likely doing it ALL WRONG; a decade out of date on methodology. That's what we have prepared statements for... you know, 80%+ of the reason we were supposed to stop using the mysql_ functions and the outdated outmoded insecure dumbass garbage practice of derping variables into query strings?

    But yeah, IF you need to escape to markup, use htmlspecial chars... and in the rare corner cases where you have to put a variable into a query string (which means a table name and... well there's ... ok, tablenames) you use the real_escape_string trash.

    Though really if you're doing your ->prepare() like a good little doobie, you really shouldn't be doing anything that needs escaping for SQL.

    -- edit -- oh, and don't forget urlencode. Right tool for the right job. Bottom line, don't brute force things with regex until you're sure the language doesn't already have that functionality.
    Last edited by deathshadow; Jan 29th, 2019 at 05:48 PM.
    “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies.” – C.A.R. Hoare, The 1980 ACM Turing Award Lecture
    http://www.cutcodedown.com

  4. #4
    New Coder
    Join Date
    Jun 2012
    Posts
    47
    Thanks
    1
    Thanked 0 Times in 0 Posts
    Hi Dormilich

    Regarding creating a whitelist of characters for content to be included in an email

    [!£$%^&*():@;',.#?]

    can you tell me which ones, if any, you would automatically exclude.

    Also what is you opinion on informing the user prior to filling in the form what characters are acceptable.

  5. #5
    New Coder
    Join Date
    Jun 2012
    Posts
    47
    Thanks
    1
    Thanked 0 Times in 0 Posts
    Hi Deathshadow

    Thanks for the advice, It's just that when you read in web security books ETC how horrendous it is to include certain characters
    in form submissions the best approach is to use maximum security measures such as whitelists. However as you said the best
    answer is having a more in depth understanding of the language.

  6. #6
    Master Coder Dormilich's Avatar
    Join Date
    Jan 2010
    Location
    Behind the Wall
    Posts
    5,801
    Thanks
    26
    Thanked 602 Times in 595 Posts
    Quote Originally Posted by maxwell5 View Post
    Regarding creating a whitelist of characters for content to be included in an email
    None. If you use an email library, that has its escaping functions for content built-in.
    The computer is always right. The computer is always right. The computer is always right. Take it from someone who has programmed for over ten years: not once has the computational mechanism of the machine malfunctioned.
    André Behrens, NY Times Software Developer

  7. #7
    Senior Coder deathshadow's Avatar
    Join Date
    Feb 2016
    Location
    Keene, NH
    Posts
    3,319
    Thanks
    4
    Thanked 477 Times in 465 Posts
    Quote Originally Posted by maxwell5 View Post
    Regarding creating a whitelist of characters for content to be included in an email
    Do you mean an e-mail address, or the stuff you plug into an e-mail for sending?

    In the case of the address, PHP provides filter_var which can check for valid mail characters. I then usually check the lengths of the parts and do a reverse MX/A check on it. It's a amazing how many spam bots use made up domains that don't even resolve DNS.

    Code:
    function isValidEmail($address) {
    	if (filter_var($address,FILTER_VALIDATE_EMAIL)==FALSE) {
    		return false;
    	}
    	/* explode out local and domain */
    	list($local, $domain) = explode('@', $address);
    	
    	$localLength = strlen($local);
    	$domainLength = strlen($domain);
    	
    	return (
    		/* check for proper lengths */
    		($localLength > 0 && $localLength < 65) &&
    		($domainLength > 3 && $domainLength < 256) &&
    		(
    			checkdnsrr($domain, 'MX') ||
    			checkdnsrr($domain, 'A')
    		)
    	);
    }
    Mind you I use that to check if it's valid, NOT to escape it for sending. Generally if you have to escape to send a e-mail address you're doing it wrong.

    Now, if you mean the mail itself, there isn't ANYTHING you can put into the body of a e-mail that needs escaping. In a simple mail everything after the first "\r\n\r\n" (yes, twice) is automatically body content. If you are sending multipart a long randomized hash is used that it is highly unlikely (near impossible) for anyone at a client side form would ever be able to guess/predict. That php's mail() function automatically applies that /r/n/r/n for you when it pastes the header to the body for sending, it's a non-issue.

    Which just leaves the other stuff you might put into a mail heading. If you're going to construct a to/cc/bcc/reply-to from client-side data, the ONLY characters you really should have to worry about are /r, /n, and <> -- and rather than escaping them, I would reject the form and make the end user type it again. When it comes to the subject header, the only thing you would/should really worry about is /r/n, and really if you're using a text INPUT tag for subject and they send you carriage returns and linefeeds, reject the form altogether as a hacking attempt. Just die right there.

    In fact as a rule of thumb if I find either carriage returns or linefeeds in any field that's going to be plugged into a mail header string, I treat it as a hacking attempt and immediately stop right there.
    “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies.” – C.A.R. Hoare, The 1980 ACM Turing Award Lecture
    http://www.cutcodedown.com

  8. #8
    New Coder
    Join Date
    Jun 2012
    Posts
    47
    Thanks
    1
    Thanked 0 Times in 0 Posts
    Hi Deathshadow


    My current code for processing a users email address is as follows -
    Is there anything missing security wise or anything you would suggest to add?
    Also should I also be using the isValidEmail() function you suggest?

    PHP Code:
    if($_POST['Your_Email_Address'] != "") {
                    
    $sender filter_var($_POST['Your_Email_Address'], FILTER_SANITIZE_EMAIL);  
              
    if (!
    filter_var($senderFILTER_VALIDATE_EMAIL)) {      
           
    $errors[] = "Your email address is not valid";             
    }     
    } else {             

    $errors[] = 'Please enter your email address';       
     }

    if(
    strlen($sender) > 200){ 

              
    $errors[] = 'Maximum length of 200 characters has been exceeded';                        
    } else {          
                   
    $sender spam_scrubber($sender);        
    $sender space_remover($sender);           
     }



    function 
    spam_scrubber($value) {    
           
     
    // List of very bad values:        
    $very_bad = array('to:''cc:''bcc:''content-type:''mime-version:''multipart-mixed:''content-transfer-encoding:');              
      
    // If any of the very bad strings are in        
     // the submitted value, return an empty string:  
         
     
    foreach ($very_bad as $v) {            

    if (
    stripos($value$v) !== false) return '';     
      
     }   
    // Replace any newline characters with spaces:    
       
     
    $value str_replace(array( "\r""\n""%0a""%0d""\\n""\\r""'""\""), ' '$value);               
     
    // Return the value:        return trim($value);        


    // End of spam_scrubber() function.    


    function space_remover($text) { 
          return 
    preg_replace('/\s+/'' '$text);


  9. #9
    Senior Coder deathshadow's Avatar
    Join Date
    Feb 2016
    Location
    Keene, NH
    Posts
    3,319
    Thanks
    4
    Thanked 477 Times in 465 Posts
    There is no real reason to call the sanitize, and 90% of what's after the mail filter is pointless. You check if the e-mail is valid. The message body requires no sanitation... leaving just the user's name and the mail subject... and PHP's mail sanitizes subject for you (that's why it's a separate field on the mail() function!)

    In the case of those or anything else you would plug into your mail header, the ONLY thing you need to check for is /r and /n. That's it. You don't need to do more.

    Do not "remove", do not "sanitize" -- test for the presence of either \r or \n, and if present reject the form as invalid. In fact, if for any field you would plug into your header string contains a \r or \n, I would not just reject it, I'd treat it as a hacking attempt possibly even reporting the IP address to something like fail2ban. This is because those fields should be coming from INPUT tags which don't allow for /r or /n in them in the first place!

    You shouldn't have to check for to:, for:, whatever because unless there's a \r\n preceding it there is no way for them to turn into header values. Period. You certainly don't need to test for URI escaped characters.

    Remember, an e-mail is just a plaintext file where it starts with headers of:

    Code:
    FieldName: data\r\n
    FieldName: data\r\n
    FieldName: data\r\n
    \r\n
    Message can contain anything
    Anything after that colon before the ending /r/n is fair game and does NOT create a new heading. Only \r\n can do that. There is ZERO reason to be testing for any of that stuff you have there. Just like anything (unless you set multipart) after the double \r\n is automatically ALL the message... and all multipart does is add a hash and formatting for the parts inside the message.

    So even if you had:

    Code:
    Subject: To:[email protected]
    
    Message here
    It would just transmit the subject as "To:[email protected]" and NOT actually mail to that address!

    Just use my isValidEmail function to test if the address is valid. If not spit the form back at them.

    Then test if any fields you are going to plug into the header contain \r or \n, if so bomb out assuming that someone is trying to hack the form.

    Do not even worry about the content of the message area.

    That's it, that's all.

    For example if you had a form like this:

    Code:
    <form action="contact" method="post" id="mail">
    	<h2>Contact Us</h2>
    	<fieldset>
    		<label for="mail_name">Name:</label>
    		<input
    			id="mail_name"
    			name="name"
    			type="text"
    			required
    		><br>
    		<label for="mail_eMail">E-Mail Address:</label>
    		<input
    			id="mail_eMail"
    			name="email"
    			type="text"
    			required
    		><br>
    		<label for="mail_subject">Subject:</label>
    		<input
    			id="mail_subject"
    			name="subject"
    			type="text"
    			required
    		><br>
    		<label for="mail_message">Message Body:</label>
    		<textarea
    			id="mail_message"
    			name="message"
    			rows="6"
    			required
    		></textarea><br>
    		<button>Send Mail</button>
    	</fieldset>
    </form>
    ... and you wanted to build it as a normal plaintext e-mail...

    Code:
    $errors = [];
    
    if (empty($_POST['name']))
    	$errors['name'] = 'You must enter a name';
    else {
    	if (preg_match('/[\r\n]/', $_POST['name']))
    		die('Hacking Attempt Detected');
    	if (preg_match('/[<>]/', $_POST['name']))
    		$errors['name'] = 'You cannot use < or > in your name';
    	// the above is because < and > have special meaning in form/reply-to/to/cc/bcc
    }
    
    if (empty($_POST['email']))
    	$errors['email'] = 'You must enter an E-Mail address';
    else if (!isValidEmail($_POST['email']))
    	$errors['email'] = 'Invalid E-mail address';
    
    if (empty($_POST['subject']))
    	$errors['subject'] = 'You must enter a subject';
    
    if (empty($_POST['message']))
    	$errors['message'] = 'You must enter a message';
    	
    if ($errors) {
    	// re-show the form here. You can even use the $errors array
    	// to display the error messages next to their appropriate fields.
    } else {
    	$header =
    		'From: ' . $_POST['name'] . ' <' . $_POST['email'] . ">\r\n" .
    		'Reply-To: ' . $_POST['email'] . "\r\n" .
    		'X-Mailer: PHP/' . phpversion() . "\r\n" .
    		'Content-Type: text/plain';
    	// thankfully mail() sanitizes subject for us!
    	mail(
    		'[email protected]',
    		$_POST['subject'],
    		$_POST['message'],
    		$header
    	);
    }
    That's all you really need to validate/check. Don't "sanitize" and blindly trust the result. Test and reject instead.

    Though if you build the message as a HTML e-mail, then you need to htmlspecialchars the contents of the message body... obviously.
    “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies.” – C.A.R. Hoare, The 1980 ACM Turing Award Lecture
    http://www.cutcodedown.com


 

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •