noindex

Noindex is a directive used in search engine optimization to prevent a specific web page from appearing in a search engine’s index. It is implemented through metadata or HTTP headers that signal search engines not to include the page in their results. Noindex is distinct from crawling controls and is not set through the robots.txt file, which governs crawling rather than indexing.

Common implementations include a robots meta tag placed in the HTML head, such as <meta name="robots" content="noindex">, or a page-specific directive like <meta name="googlebot" content="noindex">. Another method is the X-Robots-Tag HTTP header, for example X-Robots-Tag: noindex, which can apply to non-HTML content as well. These signals can be used for individual pages or entire sections of a site.

Use cases for noindex include staging or development pages, private or paywalled content, pages with duplicate content, or pages whose visibility in search results should be limited. It is commonly paired with other directives such as nofollow, but should not be used to block crawling when the goal is to keep a page out of the index while still allowing users to access it.

Limitations and timing vary by engine. Not all crawlers honor every noindex signal, and deindexing typically requires a re-crawl. If a page is disallowed by robots.txt, it may be crawled but not indexed, or indexed based on signals from external links; conversely, a noindex directive aims to prevent indexing regardless of crawlability. Results can take days to weeks to reflect the change.