Technical SEO: Three Requirements, Not a Sixty-Point Audit
Google publishes exactly three minimum technical requirements. It also says most sites should not read its crawl budget guide, and gives the page count.
Google publishes three minimum technical requirements. A typical technical SEO audit reports sixty findings. The gap between those numbers is where most of the money in this field is spent.
The three are worth knowing exactly, because everything beyond them is optimisation rather than eligibility, and because the documentation is unusually candid about which of its own guides most sites should skip.
This page sets out the requirements, the size thresholds at which the advanced advice starts applying, and the one configuration mistake that keeps pages in the index against their owner’s wishes.
The three requirements, verbatim
They appear together on one page, with an explicit caveat attached.
Requirement one. Googlebot is not blocked. The documentation adds that Google only indexes pages that are accessible to the public and which do not block its crawler from crawling them.
Requirement two. The page works, meaning that Google receives an HTTP 200 success status code.
Requirement three. The page has indexable content, defined as textual content in a file type Google Search supports, that does not violate the spam policies.
The caveat, verbatim. Just because a page meets these requirements does not mean that a page will be indexed; indexing isn’t guaranteed.
And discoverability, from the companion page. Some pages are known because Google has already visited them, others are discovered when Google extracts a link from a known page, and still others when you submit a sitemap.
What that gives you. A four-item checklist: reachable by a link or a sitemap, not blocked, returns 200, has real content. Everything in a sixty-point audit that is not one of those four is a refinement, and should be justified as such.
The guide Google tells you not to read
The crawl budget documentation is the clearest example of the platform being more relaxed than the industry built on it.
The opening, verbatim. That the guide describes how to optimise crawling of very large and frequently updated sites, and that if your site does not have a large number of pages that change rapidly, or if your pages seem to be crawled the same day that they are published, you do not need to read this guide.
What it says to do instead. For Google Search specifically, keeping your sitemap up to date and checking the page indexing report regularly is adequate.
The thresholds, verbatim. Large sites of 1 million or more unique pages with content that changes moderately often, about once a week. Medium or larger sites of 10,000 or more unique pages with very rapidly changing content, daily. Plus sites with a large portion of URLs classified as discovered but not currently indexed.
The disclaimer attached to those numbers. That they are a rough estimate to help you classify your site, and these are not exact thresholds.
Why this matters commercially. Crawl budget optimisation is a standard line item in technical SEO proposals. For a B2B site with a few hundred pages, the platform’s own documentation says it does not apply.
The honest exception. The third condition. If Search Console shows a large share of your URLs as discovered and not indexed, you have a real problem, regardless of your page count.
One configuration error accounts for a large share of “why is this still in Google” tickets, and the documentation explains it precisely.
What robots.txt does, verbatim. It is used mainly to avoid overloading your site with requests, and it is not a mechanism for keeping a web page out of Google.
What happens if you use it that way. A page disallowed in robots.txt can still have its URL appear in Google Search results if other pages link to it with descriptive text, because Google can index the URL without visiting the page.
The clause that explains the whole failure, verbatim. That robots meta tags and X-Robots-Tag headers are discovered when a URL is crawled, so if a page is disallowed from crawling in robots.txt, then any information about indexing or serving rules will not be found and will therefore be ignored.
Stated as a rule. If an indexing rule must be followed, the URL containing that rule cannot be disallowed from crawling.
So the common sequence is self-defeating. Add noindex to a page, then block it in robots.txt to be doubly safe. The block prevents Google from ever seeing the noindex, and the page stays.
What to do instead, in the documentation’s own words. To keep a page out of Google, block indexing with noindex and allow Google to crawl the URL. Or password-protect it, or remove it entirely.
The corollary for audits. A finding of “page not blocked in robots.txt” is not automatically a problem. On any page carrying a noindex, it is a requirement.
Sitemaps, with the number attached
Another item that appears in every audit and has a published threshold.
The general position, verbatim. If your site’s pages are properly linked, Google can usually discover most of your site. A sitemap helps search engines discover URLs but does not guarantee that all items in it will be crawled and indexed. In most cases a site will benefit from having one.
When you might not need one, verbatim. If your site is small, by which they mean about 500 pages or fewer, counting only pages you think need to be in search results.
When you likely do. A large site, a new site with few external links, or a site with a lot of rich media or appearing in news.
The technical limits. 50 MB uncompressed, or 50,000 URLs, per sitemap file.
How to read that in a B2B context. Most B2B sites are well under 500 pages. A sitemap is still worth having because it is free and it feeds the indexing report, but its absence is not the cause of your traffic problem.
And what it never does. Force indexing. The documentation is explicit that inclusion in a sitemap guarantees nothing.
About 500 pages, counting only what you want in results. And it guarantees nothing either way. Source : Google sitemap documentation (2026)
What the documentation says does nothing
Short list, and the boundary of it is worth knowing.
Meta keywords, verbatim. That the meta keywords tag is not used by Google Search, and it has no effect on indexing and ranking at all. That is documented, quotable and unambiguous.
Keyword density. Not addressed anywhere in the search documentation. The widely repeated statement that it is not a factor comes from spokesperson remarks in public forums, not from a page you can cite.
Third-party authority scores. Same situation. The well-known statement that Google does not use them is a spokesperson position expressed publicly, and it is not in the documentation.
Why that distinction matters. In a client argument, “Google’s documentation states it has no effect at all” and “a Google employee said so on a forum” are different weights of evidence. Only one of the three items above carries the first.
What that implies for audits generally. Ask which claims in the report cite documentation, which cite a spokesperson, and which cite nothing. That single question usually reduces sixty findings to a workable list.
And one thing worth saying plainly. None of this means technical work is unnecessary. It means the requirements are few, the thresholds are published, and most of what fills an audit is optimisation that should be justified against the traffic it can plausibly move.
Ask which findings cite documentation, which cite a spokesperson, and which cite nothing. Sixty becomes a workable list. Source : Google Search Central documentation (2026)
What to do with an audit you have been handed
Six steps, and the first two dispose of most of it.
Check the four eligibility items first. Reachable, not blocked, returns 200, has content. If any page you care about fails one of those, that is the whole priority list until it is fixed.
Sort every remaining finding by tier. Documented, spokesperson, or nothing. Then work only the first group until it is empty.
Check the discovered-but-not-indexed report. This is the one crawl condition that applies at any size, and it is the only crawl-related finding on a small site that is likely to be real.
Verify that no page you want removed is blocked from crawling. The single most common cause of a page refusing to leave the index.
Ignore crawl budget unless you cross the published thresholds. A million pages changing weekly, or ten thousand changing daily. If you have four hundred pages, this is not your problem and paying for it is a transfer, not an investment.
Then ask what the remaining items are expected to move. Not whether they are best practice. What traffic, on which pages, by when. Anything that cannot answer goes to the bottom.
Three published requirements: Googlebot not blocked, HTTP 200, indexable content. Plus discoverability by link or sitemap.
And an explicit caveat: meeting them does not guarantee indexing.
The crawl budget guide tells most sites not to read it, and says keeping the sitemap current and checking the indexing report is adequate.
Its thresholds are 1,000,000+ pages changing weekly or 10,000+ changing daily, described as a rough estimate rather than exact.
A sitemap may be unnecessary below about 500 pages that you want in search results. Limits are 50 MB or 50,000 URLs per file.
robots.txt does not remove pages. A blocked URL can still appear if other pages link to it.
And blocking hides your noindex. Rules on a page cannot be found if the page cannot be crawled.
Only meta keywords is documented as having no effect. Keyword density and authority scores are spokesperson positions, not documentation.
Fix the four eligibility items, then sort the rest by what it cites. Book a diagnostic, or see how we approach B2B websites.
Frequently asked questions
What does Google actually require technically?
Three things: that Googlebot is not blocked, that the page returns an HTTP 200 success status, and that the page has indexable content. The documentation adds that meeting these does not guarantee indexing.
Do I need to worry about crawl budget?
Almost certainly not. The guide opens by saying that if your site does not have a large number of rapidly changing pages, or if pages are crawled the day they are published, you do not need to read it.
What size of site does crawl budget apply to?
Large sites of 1 million or more unique pages changing about weekly, or medium and larger sites of 10,000 or more unique pages changing daily. Google calls these a rough estimate, not exact thresholds.
Do I need a sitemap?
Google says you might not if your site is small, defining small as about 500 pages or fewer, counting only pages you think need to be in search results. Below that, good internal linking usually suffices.
Why is my blocked page still in Google?
Because blocking in robots.txt prevents crawling, not indexing. Google states that a disallowed URL can still appear in results if other pages link to it, and that it cannot see a noindex rule on a page it is not allowed to crawl.
So how do I actually remove a page?
Use noindex and allow Google to crawl the URL, or password-protect it, or remove it. The documentation is explicit that indexing rules on a page cannot be followed if the page is disallowed from crawling.
Do meta keywords do anything?
No. The documentation states that the meta keywords tag is not used by Google Search and has no effect on indexing and ranking at all.
What about keyword density and domain authority?
Neither appears in the search documentation at all. The widely quoted statements about them come from spokesperson remarks in public forums, not from any documented position you can cite.