Beamfar
Sign inStart the 14-day trial

What is indexing in search?

Last updated .

Indexing is the step after crawling: the search engine analyses a fetched page and stores it in its index, from which results are drawn. A page that is crawled is not necessarily indexed, and a page that is indexed is not necessarily ranked.

Definition
Indexing
The process by which a search engine stores a crawled page in its index so it can appear in results.

What decides it

Directives such as noindex, the canonical URL the engine selects among duplicates, the quality and uniqueness of the content, the status code, and whether the page can be rendered. Google treats redirects, rel=canonical and sitemap inclusion as signals of different strength when choosing the canonical, and indexes that URL.

Why agencies watch it

The common outages in search are indexing outages: a stray noindex, a robots.txt block, a canonical to the wrong host or a migration that left old URLs unredirected. The site works for visitors while its visibility falls.

How to look

Search Console’s URL Inspection tool shows for one URL whether it is indexed, which URL Google chose as the canonical and whether a noindex rule or a robots.txt block applies. A site:example.com search gives only a rough picture. After a release, inspect one URL per template (home page, article, product), because a template-level mistake affects every page built from it, and the indexed count falls only days later.

A page can be indexed under another URL than the one you expect when the engine picks a different canonical. The inspection tool shows both the declared and the chosen one, and a difference between them is worth reading.

What to check

The indexed page count and the reasons for excluded pages in Search Console, and a regular check that the indexing directives on key templates are what you intend.

Sources

Read on . Standards and vendor pages change; confirm the current text before you rely on it.

Guides that use this term

  • CrawlingCrawling is how a search engine fetches pages by following links and sitemaps. What affects it, why robots.txt and server errors change crawl rate and what crawl budget means.
  • NoindexNoindex is a robots directive, set in a meta tag or header, that tells search engines not to keep a page in their index. How Google reads it and its common failure.
  • Canonical URLThe canonical URL is the address a search engine treats as the main version of a page that exists at several URLs. How rel=canonical, redirects and sitemaps signal it.
  • SitemapAn XML sitemap lists the URLs a site wants search engines to know about. Google’s size limits, how lastmod is used and what belongs in the file.

Know first. Know why. Tell your client before they call.

Paste your client list. Both regions start checking.

Start the 14-day trialWhat Beamfar watches

No card for the trial.