Is this page indexable? Check noindex, robots.txt and canonical.
Enter a page address. Beamfar reads the page once, the way a search engine would start, and shows whether it asks not to be indexed, whether robots.txt blocks it, and which address it calls canonical.
What the check reads
shop.alder.example/sale
- Answers
200, no redirectFine - Robots meta tag
noindexProblem - X-Robots-Tag
not sentFine - robots.txt
allows the pageFine - Canonical
this pageFine
What the check looks at
The same probes Beamfar uses to monitor a client site, run once, from this server.
- Does it answer?
- The status of the final page. A page that does not answer with a success cannot be indexed.
- Redirects
- Every step from the address you typed to the final page.
- noindex
- The robots meta tag in the page and the
X-Robots-Tagheader: either one can ask search engines to drop the page. - robots.txt
- Whether the site’s robots.txt blocks this page for Google’s crawler or for every crawler.
- Canonical
- The address the page names as its canonical one, and whether it points somewhere else.
How to read the result
Each reading gets one of the words of the free check, with its mark.
Fine on every row means nothing on the page itself keeps it out of search results. Whether it ranks is another question.
Problem on noindex or robots.txt means search engines are told to stay away: after a deploy this is the most common reason a whole site disappears from Google within days.
Look at this on canonical means the page names another address as the real one: right for duplicates, wrong when a staging or old domain is named.
Common reasons a page is not indexable
- noindex left over from staging
- the CMS setting “discourage search engines” or a staging template went live. Remove the tag or the header and ask for a new crawl.
- robots.txt from the development site
Disallow: /was copied to production. Replace it with the production rules.- Canonical to the wrong host
- after a migration the canonical still names the old domain or the staging host. Point it at the page’s own live address.
- Redirect chains
- several hops, or a loop, after a migration. Redirect old addresses straight to their final page.
What happens to the address you enter
One check, then nothing kept. Only public websites, only the standard web ports.
- Nothing is stored
- The address is used for this one check and then forgotten: no database row, no log line, no cookie, no link to share the result. Only rate-limit counters are kept: hashed counts per network range and overall, for at most two days, never an address of a site.
- Only public websites
- Addresses on private networks, loopback, link-local and cloud-metadata ranges are refused, and so are redirects to them. Only the standard web ports 80 and 443 are used. Each name is looked up once and the connection goes only to the address that passed. Checks are limited per network and overall, share the budget of the free site check, and need a short puzzle to be solved by your browser.
The site you check will see a few requests from Beamfar-probe in its logs (the page, each redirect hop and robots.txt), from this server. Beamfar reads these signals on every check of a client site and alerts you when one changes into a risk.
Questions
Is this indexability checker free?
Yes. No account and no payment. It shares the rate limits of the free site check.
Does it tell me whether Google has indexed the page?
No. It reads what the page and robots.txt tell search engines. Whether Google indexed it is in Google Search Console.
Which crawler’s robots.txt rules does it read?
The rules for Google’s crawler and for every crawler (the asterisk group), as robots.txt applies them to this page’s path.
Does it run JavaScript?
No. It reads the page as sent, like a first crawl does. A noindex added later by a script is not seen.
Is the address I enter stored?
No. It is used for this one check only. Only hashed rate-limit counters are kept, for at most two days.
Guides that go with this page
Longer reads on the same ground, in plain words, each with its sources.
- Noindex after a deploy: how sites vanish from GoogleHow a leftover staging noindex removes a site from search, how Google reads it and a deploy checklist.
- robots.txt mistakes that hurt a client siteThe robots.txt mistakes made on launch day and what Google and RFC 9309 say about errors, wildcards and caching.
- Canonical tag mistakes on client sitesThe canonical link errors that point search at the wrong URL and a checklist for catching them after every release.
- Redirect chains after a site migrationHow chains and loops appear after a migration, how Google treats each status code and how to repair them.
More free tools
One live check each, no account. The free site check runs all of them at once.
- SSL certificate checkerWho issued a site’s certificate, which name it covers, when it expires and the days left.
- Domain expiry checkerWhen a domain name’s registration ends, from the registry’s public RDAP record.
- DNS lookupThe address, alias, name server and mail records of a name, from public resolvers.
- Security headers checkerHSTS, Content-Security-Policy, nosniff, frame protection, Referrer-Policy and more.
- The free site checkEvery check at once: the page, redirects, certificate, DNS, search settings and the domain registration date.
Know the day a client site drops out of search.
Beamfar watches noindex, robots.txt and canonical on every check and alerts you when one turns risky.
No card for the trial.