XML Sitemap
Why you need one: a URL map for crawlers
An XML sitemap is a file that lists a site’s page URLs and each URL’s last-modified date in XML format and hands it to search engines. Google Search Central documentation defines it as “a file where you provide information about the pages, videos, and other files on your site, and the relationships between them,” and says search engines read this file to crawl the site more efficiently. Google (official) It’s essentially a map you hand directly to crawlers for URLs that are hard to discover through links alone, such as newly published articles or deep pages with few internal links.
The same document is also clear about its limits. It makes plain that a sitemap only helps with URL “discovery” and doesn’t guarantee that every item in it will be crawled and indexed. Google (official) Crawling and indexing are separate stages, and a sitemap is a hint that helps with the first stage, discovery.
Here Google’s official position and industry practice diverge slightly. Google says that if a site has about 500 pages or fewer and is well linked internally, it can discover most pages even without a sitemap. Google (official) Industry tools like Yoast, on the other hand, generate a sitemap by default for almost every site regardless of size. Yoast Since the cost of maintaining a sitemap is practically zero, in practice it’s safer to have one even for a small site.
Basic structure and limits
A sitemap is an XML file that follows the sitemaps.org protocol. According to the specification, the root element urlset and each url entry’s loc (address) are required, while lastmod (last modified date), changefreq (change frequency) and priority are optional. sitemaps.org specification
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://example.com/</loc>
<lastmod>2026-06-15</lastmod>
</url>
<url>
<loc>https://example.com/portfolio/project-a/</loc>
<lastmod>2026-05-02</lastmod>
</url>
</urlset>
The limits are also set in the specification: up to 50,000 URLs or 50MB uncompressed per file, and Google Search Central documentation gives the same limits. Google (official) If you exceed them, split the sitemap into several files and add a separate sitemap index file that lists them.
lastmod: the only optional field Google uses
The specification also includes the priority and changefreq fields, but Google Search Central documentation states flatly that Google “ignores” these two values. Google (official) Of the optional fields, lastmod is the only value Google actually uses, and even then with the caveat that it’s trusted only “if it’s consistently and verifiably accurate.” Google (official)
Google explained this trust condition in detail on the Search Central Blog in June 2023. If lastmod doesn’t match reality, such as a page changed seven years ago showing yesterday’s date, search engines eventually stop trusting all of that site’s last-modified values. Google official blog On this point, industry observations don’t conflict with the official position: Yoast notes that both Google and Bing emphasize accurate lastmod values and recommends reflecting when meaningful changes to the main content, structured data or links happened, not trivial edits like updating the copyright year. Yoast
If you stamp every URL’s lastmod with the current time each time the sitemap is generated, you also lose the signal asking for quick recrawling of pages you actually updated. The same blog says it’s fine to use lastmod only for pages you’re sure about and omit it for unclear ones. Google official blog
How to submit: Search Console and robots.txt
Google’s documentation describes two submission routes. Google (official) Google Search Console: submit the URL in its Sitemaps report, or robots.txt add a single Sitemap: directive line to the file. According to the Search Console Help Center, the Sitemaps report shows submission history, the number of pages discovered from the sitemap and processing errors, so it also serves as a diagnostic tool. Search Console Help Center
The ping endpoint used in the past (notifying sitemap updates via an HTTP request) was deprecated by Google in June 2023 and now returns 404. Google official blog The same blog explains that old plugins that keep calling ping do no harm but have no effect.
Generating one in WordPress: build-stage checkpoints
In WordPress, there are three ways to generate one.
| Method | Path | Characteristics |
|---|---|---|
| Core default (WP 5.5+) | /wp-sitemap.xml | Generated automatically. Limited fine-grained control |
| SEO plugin (Yoast, etc.) | /sitemap_index.xml | An index structure bundling sub-sitemaps by post type and taxonomy. Automatically excludes noindex pages |
| Custom theme implementation | Any path | Full control to fit special structures such as custom post types and multilingual setups |
The items to check when building a custom theme come straight from official recommendations. First, custom post types you want shown should be included in the sitemap, and pages set to noindex should be left out. Yoast says pages set to noindex are automatically excluded from the sitemap, and offers this rule of thumb: “if it’s not a good result when a visitor lands on that URL, it shouldn’t be in the sitemap.” Yoast Second, as Google recommends, list only canonical (representative) URLs, as complete absolute URLs. Google states that it “will try to crawl the URLs exactly as listed.” Google (official) Third, check that lastmod reflects the actual post modification date (post_modified).
Common mistakes
- Adding non-ASCII (e.g., Korean) URLs as-is without encoding: the sitemap protocol requires all URLs to be percent-encoded according to RFC-3986. For example, a Korean path such as
/블로그/(meaning “blog”) must be written encoded, as/%EB%B8%94%EB%A1%9C%EA%B7%B8/. Sitemap protocol This is a common mistake on Korean-language sites that use Korean characters directly in URLs. Since Google crawls URLs exactly as written in the sitemap, adding them raw can cause a mismatch with the actual address and drop them from the index. Google (official) - Putting effort into tuning priority and changefreq values: Google Search Central documentation clearly says it ignores both fields, so that time is better spent on lastmod accuracy. Google (official)
- Stamping the same date on every URL’s lastmod: a sign that the generation time was recorded in bulk. Google’s blog warns that lastmod values that don’t match reality eventually lose trust. Google official blog
- Listing non-canonical addresses: URLs with parameters, relative paths and mixed http/https addresses are typical. Google’s documentation recommends including only canonical URLs as complete absolute URLs. Google (official)
- Including noindex pages or redirecting URLs: a contradiction, putting addresses that aren’t meant to be indexed into a list asking for indexing. Yoast also treats excluding noindex pages from the sitemap as standard behavior. Yoast
- Creating a sitemap but never submitting it: crawlers only know where it is once you submit it in Search Console or register it in robots.txt. Old code that relies on the deprecated ping endpoint also needs checking.
References
- Learn about sitemaps – Google Search Central
- Build and submit a sitemap – Google Search Central
- Sitemaps ping endpoint is going away – Google Search Central Blog
- Sitemaps report – Search Console Help
- Sitemaps XML format – sitemaps.org
- What is an XML sitemap and why should you have one? – Yoast
- Google and Bing stress the importance of lastmod in XML sitemaps – Yoast
Jul.2026By Studio JT