Skip to content
    HadoSEO

    [ search crawler · Google ]

    What is Googlebot?

    Googlebot is Google's search crawler. It fetches pages so they can be indexed and ranked in Google Search.

    In robots.txt it answers to the Googlebot token — rules below — though robots.txt binds only crawlers that choose to honor it. Its intent is search index: stores your pages so they can show up in search results.

    See whether Googlebot actually visits your site. Upload a week of server logs to the free log file analyzer, or connect crawl analytics for verified visits by page, live.

    search indexverifiable by reverse DNS

    IP ranges last updated Nov 8, 2026

    Updated · HadoSEO

    [ technical facts ]

    operator
    Google
    intent
    search index
    user agent contains
    googlebot
    robots.txt token
    Googlebot
    verification
    reverse DNSverified
    reverse DNS
    .googlebot.com, .google.com
    last updated
    Nov 8, 2026

    [ user agent ]

    The Googlebot user agent.

    the User-Agent contains (case-insensitive)
    googlebot

    The full User-Agent string includes a version number that changes over time; Google's documentation lists the current one. A matching user agent is a claim, not proof — verify the IP below.

    [ verify ]

    How to verify a real Googlebot request.

    Forward-confirmed reverse DNS: the IP's hostname must end in .googlebot.com or .google.com, and that hostname must resolve back to the same IP.

    1. 01Check the User-Agent header contains "googlebot" (case-insensitive). That is only a claim: anyone can send it.
    2. 02Run a reverse DNS lookup on the IP. The hostname must end in .googlebot.com or .google.com.
    3. 03Run a forward lookup on that hostname. It must return the original IP (forward-confirmed reverse DNS).
    4. 04Google also publishes an IP list you can check instead: https://developers.google.com/static/crawling/ipranges/common-crawlers.json.

    Skip the lookups: the free bot verifier checks one IP against the same registry, and the log file analyzer checks every Googlebot request in a week of access logs. Requests that fail are flagged as spoofed, not hidden.

    [ robots.txt ]

    Block or allow Googlebot in robots.txt.

    Add one of these groups to the robots.txt at your domain root. The token matches case-insensitively. Google runs other crawlers with their own tokens, so a rule for Googlebot leaves Google-Extended (training) untouched.

    block Googlebot
    User-agent: Googlebot
    Disallow: /
    allow Googlebot
    User-agent: Googlebot
    Allow: /

    robots.txt only binds crawlers that choose to follow it. A scraper sending Googlebot's user agent from its own servers won't, which is why HadoSEO checks the IP behind every request.

    [ questions ]

    Googlebot, answered.

    [ free scan ]

    How much of your site can AI actually see?

    No signup required · results in ~60 seconds