Reaching out to others! Free & Open Source Software, Kannada, L10n, L18n Data Science, Cloud Computing & more…

Fix CSF blocking Google bots

If you install on your server along with you might see it blocking google bot for some specific rules. At times you might find it offensive to remove some of the rules from the configuration just to allow Google bots.

 

Here is what you should be doing to make Google Bots crawl through your sites again.

Just add the following rules to mod_security configuration file and restart apache.

# GoogleBot by user-agent…
SecRule HTTP_USER_AGENT “Google” nolog,allow
SecRule HTTP_USER_AGENT “Googlebot” nolog,allow
SecRule HTTP_USER_AGENT “GoogleBot” nolog,allow
SecRule HTTP_USER_AGENT “googlebot” nolog,allow
SecRule HTTP_USER_AGENT “Googlebot-Image” nolog,allow
##
SecRule HTTP_USER_AGENT “AdsBot-Google” nolog,allow
SecRule HTTP_USER_AGENT “Googlebot-Image/1.0″ nolog,allow
SecRule HTTP_USER_AGENT “Googlebot/2.1″ nolog,allow
SecRule HTTP_USER_AGENT “Googlebot/Test” nolog,allow
SecRule HTTP_USER_AGENT “Mediapartners-Google/2.1″ nolog,allow
SecRule HTTP_USER_AGENT “Mediapartners-Google*” nolog,allow
SecRule HTTP_USER_AGENT “msnbot” nolog,allow

Enjoy!

Tags: , , , ,

Enable Clean URL for Drupal

Read this manual if you’re not sure about enabling Clean URL for your brand new Drupal installation. Or simply create a .htaccess and add the following lines of code. That’s it. If you’re working with Drupal 6.x it automatically detects CleanURL’s working. enjoy!

RewriteEngine on
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]