Meta robots and X-Robots-Tag
type: TechArticle
description: Every page must have an explicit, correct indexing policy — either implicit (default index, follow) on public pages, or an explicit noindex / X-Robots-Tag on staging, admin, thin, or private content. Get this wrong and you either disappear from search or expose what you didn't mean to.
language: en
is part of: Web Specification Studio
author: Web Specification Studio
publisher: Web Specification Studio
license: https://creativecommons.org/licenses/by/4.0/
date modified: 2026-05-29T09:55:11.000Z
citation: HTML Living Standard — Standard metadata names: robots
  type: CreativeWork
  publisher: WHATWG
citation: Google Search Central — Robots meta tag, data-nosnippet, and X-Robots-Tag
  type: CreativeWork
  publisher: Google Search Central
citation: Bing Webmaster — Robots meta tag
  type: CreativeWork
  publisher: Bing
citation: MDN —
  type: CreativeWork
  publisher: MDN

Web Specification Studio
type: WebSite

Web Specification Studio
type: Organization

breadcrumb
type: BreadcrumbList
item list element: Home
  type: ListItem
  position: 1
  item: https://webspecification.com/
item list element: Spec
  type: ListItem
  position: 2
  item: https://webspecification.com/spec/
item list element: SEO
  type: ListItem
  position: 3
  item: https://webspecification.com/spec/seo/
item list element: Meta robots and X-Robots-Tag
  type: ListItem
  position: 4
  item: https://webspecification.com/spec/seo/meta-robots/