Knowledge
Detect Googlebot visits using nginx
#nginx
How to detect when and how often Googlebot visits your website using a few configuration lines in nginx.
Published by Mark van Eijk on November 18, 2024 · 1 minute read
Detecting Googlebot
It's very easy to detect the Googlebut using the user agent in nginx, here we use the map
directive to set the variable $googlebot
to yes
or keep it empty depending on the given user agent:
map $http_user_agent $googlebot {
default "";
"~*googlebot" "yes";
}
Logging the requests
When $googlebot
is filled, we want to log the request in a log file. This can be done using the access_log
directive:
access_log /var/www/logs/googlebot.log bots if=$googlebot;
That's it!
You can read here how you can log multiple bots how to log multiple bots for your virtual host in nginx.
Subscribe to our newsletter
Do you want to receive regular updates with fresh and exclusive content to learn more about web development, hosting, security and performance? Subscribe now!
Related articles
Server Side Includes (SSI) in nginx
How to detect when and how often Googlebot visits your website using a few configuration lines in nginx.
Configure Content Security Policy with nonce using nginx
How to detect when and how often Googlebot visits your website using a few configuration lines in nginx.