←  All terms
S
SEO & AEO

sitemap.xml

An XML sitemap lists relevant URLs and optional metadata to help search engines discover content efficiently.

An XML sitemap provides search engines with a structured list of important URLs and can include metadata such as the date a page was last modified. It supports content discovery, especially on large or complex sites, but does not replace strong internal linking or a clear information architecture. The concept is closely related to Crawling and Indexing.

Read more

Crawling

Indexing

robots.txt

hreflang

FAQ

sitemap.xml

What is the difference between sitemap.xml and robots.txt?
The robots.txt file defines which areas crawlers should not fetch, while the sitemap.xml names pages that should be discovered. One restricts, the other recommends. The robots.txt can also point to the sitemap location. Contradictions, such as blocked URLs listed in the sitemap, should be avoided.
Does a sitemap.xml guarantee that pages get indexed?
No. A sitemap only suggests URLs as candidates for crawling. Whether they enter the index depends on quality, duplication, and relevance. It is a hint, not an instruction. Pages that stay out of the index despite being listed usually point to content or technical problems.
Which URLs belong in a sitemap.xml, and which do not?
Indexable pages that serve as the canonical version and return status code 200 belong in it. Redirects, error pages, URLs excluded with noindex, and duplicates do not. A sitemap full of unwanted addresses sends conflicting signals and makes it harder to evaluate how a site is indexed.
Does every website need a sitemap.xml?
Not strictly. Small, well-linked sites are usually discovered completely without one. A sitemap becomes useful with many pages, new domains, few external links, or content that is hard to reach through internal links. The effort is low, and the benefit grows with the complexity of the site structure.