Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 9 of 9
  1. #1
    New Coder
    Join Date
    Apr 2007
    Posts
    64
    Thanks
    4
    Thanked 0 Times in 0 Posts

    Exempt the Facebook crawler from HTTPS redirect

    I redirected my site to SSL and I've lost all Facebook likes of my non-SSL index page. I added meta property="og:url" to my index page. According to Facebook official page I need Exempt the Facebook crawler from HTTPS redirect. How can I do this?

    .htaccess file:
    Code:
    RewriteEngine On
    RewriteCond %{SERVER_PORT} 80
    RewriteRule ^(.*)$ https://www.aymavisi.org/$1 [R=301,L]
    Added to my index file:
    Code:
    <meta property="og:url" content="http://www.aymavisi.org" />
    Last edited by tunayx; Jan 22nd, 2017 at 12:48 PM.

  2. #2
    New Coder
    Join Date
    Apr 2007
    Posts
    64
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Is it possible?

  3. #3
    Senior Coder Arbitrator's Avatar
    Join Date
    Mar 2006
    Location
    Splendora, Texas, United States of America
    Posts
    3,625
    Thanks
    43
    Thanked 317 Times in 311 Posts
    Quote Originally Posted by tunayx View Post
    I redirected my site to SSL and I've lost all Facebook likes of my non-SSL index page. I added meta property="og:url" to my index page. According to Facebook official page I need Exempt the Facebook crawler from HTTPS redirect. How can I do this?
    Presumably, you can do this by using another rewritecond directive:

    Code:
    rewriteengine on
    rewritecond %{HTTPS}           "^off$"
    rewritecond %{HTTP_USER_AGENT} "^facebookexternalhit/1\.1(?: \(\+http://www\.facebook\.com/externalhit_uatext\.php\))?$" [nocase]
    rewriterule ^(.*)$             "https://%{HTTP_HOST}/$1"                                                                 [redirect=301,last]
    (I wrote a quick regular expression based on the UA strings at https://developers.facebook.com/docs...asters/crawler.)

    Edit: I changed the directive involving SERVER_PORT to use the HTTPS server variable, added quotation marks around the regular expressions because of the space in the user agent regex, made the parentheses non-capturing, added a [nocase] flag, and used the %{HTTP_HOST} server variable in place of the host name.
    Last edited by Arbitrator; Jan 24th, 2017 at 04:44 AM. Reason: See the post.
    For every complex problem, there is an answer that is clear, simple, and wrong.

    CSS Specifications

  4. #4
    New Coder
    Join Date
    Apr 2007
    Posts
    64
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Arbitrator View Post
    Presumably, you can do this by using another rewritecond directive:

    Code:
    rewriteengine on
    rewritecond %{HTTPS}           "^off$"
    rewritecond %{HTTP_USER_AGENT} "^facebookexternalhit/1\.1(?: \(\+http://www\.facebook\.com/externalhit_uatext\.php\))?$" [nocase]
    rewriterule ^(.*)$             "https://%{HTTP_HOST}/$1"                                                                 [redirect=301,last]
    (I wrote a quick regular expression based on the UA strings at https://developers.facebook.com/docs...asters/crawler.)

    Edit: I changed the directive involving SERVER_PORT to use the HTTPS server variable, added quotation marks around the regular expressions because of the space in the user agent regex, made the parentheses non-capturing, added a [nocase] flag, and used the %{HTTP_HOST} server variable in place of the host name.
    I've replaced your code with mine but I get Could Not Follow Redirect Path error while debugging on Facebook:
    https://developers.facebook.com/tool...w.aymavisi.org
    Also you code doesn't force https for my website, every page is now also view able as http.
    Last edited by tunayx; Jan 24th, 2017 at 09:58 AM.

  5. #5
    Senior Coder Arbitrator's Avatar
    Join Date
    Mar 2006
    Location
    Splendora, Texas, United States of America
    Posts
    3,625
    Thanks
    43
    Thanked 317 Times in 311 Posts
    Quote Originally Posted by tunayx View Post
    I've replaced your code with mine but I get Could Not Follow Redirect Path error while debugging on Facebook:
    https://developers.facebook.com/tool...w.aymavisi.org
    Also you code doesn't force https for my website, every page is now also view able as http.
    Sorry, it looks like there's an error in my code; I assumed that the rewriterule test string will always contain one or more characters when it can contain zero characters. This should work assuming I have the UA string correct:

    Code:
    rewriteengine on
    rewritecond "%{HTTPS}"           "!=on"
    rewritecond "%{HTTP_USER_AGENT}" "^facebookexternalhit/1\.1(?: \(\+http://www\.facebook\.com/externalhit_uatext\.php\))?$" [nocase]
    rewriterule "^.*$"               "https://%{HTTP_HOST}%{REQUEST_URI}"                                                      [redirect=308]
    Re-add the last flag if needed and change the redirect code from 308 if desired.

    Edit: This still isn't correct because you want to not redirect if the Facebook crawler is being used. Give me a moment for a followup.
    Last edited by Arbitrator; Jan 24th, 2017 at 08:00 PM.
    For every complex problem, there is an answer that is clear, simple, and wrong.

    CSS Specifications

  6. #6
    Senior Coder Arbitrator's Avatar
    Join Date
    Mar 2006
    Location
    Splendora, Texas, United States of America
    Posts
    3,625
    Thanks
    43
    Thanked 317 Times in 311 Posts
    OK, take three:

    Code:
    rewriteengine on
    rewritecond "%{HTTPS}"           "!=on"
    rewritecond "%{HTTP_USER_AGENT}" "!^facebookexternalhit/1\.1(?: \(\+http://www\.facebook\.com/externalhit_uatext\.php\))?$" [nocase]
    rewriterule "^.*$"               "https://%{HTTP_HOST}%{REQUEST_URI}"                                                       [redirect=308]
    Apparently, you can simply negate a regular expression with ! (as opposed to !=).
    For every complex problem, there is an answer that is clear, simple, and wrong.

    CSS Specifications

  7. Users who have thanked Arbitrator for this post:

    tunayx (Jan 24th, 2017)

  8. #7
    New Coder
    Join Date
    Apr 2007
    Posts
    64
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Arbitrator View Post
    OK, take three:

    Code:
    rewriteengine on
    rewritecond "%{HTTPS}"           "!=on"
    rewritecond "%{HTTP_USER_AGENT}" "!^facebookexternalhit/1\.1(?: \(\+http://www\.facebook\.com/externalhit_uatext\.php\))?$" [nocase]
    rewriterule "^.*$"               "https://%{HTTP_HOST}%{REQUEST_URI}"                                                       [redirect=308]
    Apparently, you can simply negate a regular expression with ! (as opposed to !=).
    Worked perfect. Thank you very much Arbitrator!

  9. #8
    New to the CF scene
    Join Date
    Aug 2017
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello,
    I have the same issue and it doesn't work for me.
    Here is my code.

    RewriteEngine On
    RewriteCond %{SERVER_PORT} 80
    RewriteCond %{HTTP_USER_AGENT} !(Facebot|facebookexternalhit/1.1) [NC]
    RewriteRule ^(.*)$ https://www.mysite.com/$1 [R=301,L]

    I'm using Wordpress and the meta property looks like this after migration.

    <meta property="og:url" content="http://www.mysite.com/article/" />

    I have no idea why my code is not working. Can you help ? Thanks.

  10. #9
    New to the CF scene
    Join Date
    Aug 2017
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Sorry I made a mistake.

    The meta property looks like : <meta property="og:url" content="httpS://www.mysite.com/article/" />


 

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •