Glossary

What is Crawling?

Crawling is Googlebot following links to discover and download pages. It comes before indexing, but not everything crawled gets indexed.

Updated: 05.11.2026 · 1 min read · Türkçe

Contents
  1. Why does it matter?
  2. Example
  3. What should you do?
  4. Guides that use this term
  5. Related terms

Why does it matter?

A page that cannot be crawled can never be indexed; server errors and robots.txt blocks stop crawling.

Example

Example: a link on your home page leads to a new category page. Googlebot follows the link and downloads the category page; that is crawling.

What should you do?

Follow daily crawl counts and error rates in the Search Console crawl stats report.

Guides that use this term

Frequently asked questions

What is Crawling?

Crawling is Googlebot following links to discover and download pages. It comes before indexing, but not everything crawled gets indexed.

Why does Crawling matter?

A page that cannot be crawled can never be indexed; server errors and robots.txt blocks stop crawling.

What should I do about Crawling?

Follow daily crawl counts and error rates in the Search Console crawl stats report.

Related pages