Published: August 1, 2026 | Category: SEO Services to Technical SEO | Reading Time: ~20 minutes
An XML sitemap is a structured file that lists the URLs a site owner considers important for search engines to discover. It is best understood as a discovery aid and a signal of intended canonical URLs—not a command that guarantees crawling, indexing or rankings. A clean sitemap can help search engines find important content efficiently, while an inaccurate sitemap can send contradictory signals. This 2026 guide explains sitemap structure, submission, maintenance and troubleshooting, including the indexing issues that matter most in Google Search Console.
1. What XML Sitemaps Actually Do (and Don’t Do)
What They Do
A sitemap gives search engines an organized list of URLs the site wants discovered. It can be particularly useful for large websites, recently launched sites, sites with frequently changing content, and sites where some important pages are not naturally easy to discover through internal links alone.
What They Don’t Do
Being listed in a sitemap does not guarantee that a URL will be indexed or rank. Search engines still independently evaluate crawlability, canonicalization, duplication, content usefulness and other quality signals. If Google reports a URL as “Crawled – currently not indexed,” repeatedly resubmitting the same sitemap is not a substitute for diagnosing why Google is choosing not to index the page.
2. What Should and Shouldn’t Be in Your Sitemap
Include Canonical, Indexable URLs
A sitemap should normally contain the preferred canonical version of each important page that you genuinely want indexed. Align sitemap inclusion with canonical tags, internal links and indexation directives so the site sends consistent signals.
Exclude Noindexed, Duplicate and Low-Value URLs
Pages carrying a noindex directive, duplicate URL variations, thin utility pages, empty archives and unnecessary parameter URLs generally should not appear in a sitemap. Including them creates a contradiction between “discover/index this URL” signals and the page’s actual indexing intent.
Exclude Redirected and Broken URLs
Sitemap entries should normally resolve directly to successful, intended destinations. URLs that redirect or return 404/410/5xx responses should be investigated and removed from the sitemap when they are no longer intended indexable URLs.
3. Sitemap Structure Best Practices
Sitemap Indexes for Large Sites
Large sites benefit from a sitemap index: a master file referencing separate sitemap files for logical content groups such as posts, pages, products or categories. This makes maintenance easier and can also make Search Console diagnosis more useful because problems can be isolated by content type instead of treating the entire site as one undifferentiated URL set.
Splitting Large Sitemaps
Do not attempt to force an extremely large URL inventory into one file. Follow sitemap protocol limits and use separate files grouped in a sensible way. The objective is not merely technical compliance; a logical structure makes auditing and indexation monitoring easier.
Accurate Last-Modified Dates
Where your sitemap includes lastmod, it should reflect meaningful content changes rather than being artificially refreshed on every request. Accurate modification information can help crawlers understand which URLs have genuinely changed.
Priority and Change-Frequency Tags
Older sitemap implementations often emphasize priority and changefreq. These should not be treated as powerful modern ranking or crawling controls. Spend your effort on accurate URL inclusion, canonical consistency, useful internal linking and genuine modification data rather than trying to fine-tune legacy hint values.
4. Submitting and Discovering Your Sitemap
Google Search Console Submission
Submitting the sitemap through Google Search Console provides a direct and trackable way to tell Google where the file is located and to monitor whether it can be processed successfully.
Referencing the Sitemap in Robots.txt
A sitemap location can also be referenced in robots.txt, providing another standard discovery path for crawlers. This complements rather than replaces Search Console monitoring.
Do You Need to Resubmit After Every New Post?
Normally, no. A properly generated dynamic sitemap should update as content changes, and search engines can revisit an already known sitemap. Manual resubmission is more relevant after a sitemap URL change, significant restructuring or when troubleshooting a genuine processing problem.
5. Monitoring Sitemaps in Search Console
Sitemap Coverage Is a Diagnostic Starting Point
If many URLs associated with a sitemap are not indexed, use that pattern as a starting point rather than as proof of one particular problem. Review Page Indexing reasons and representative URLs to determine whether the difference reflects redirects, canonical selection, intentional exclusions, duplicates, technical problems or other indexing decisions. The sitemap may reveal a pattern without identifying its cause.
Use the Page Indexing Report With Sitemap Data
When many submitted URLs remain excluded, inspect Google’s specific exclusion categories and example URLs. “Discovered – currently not indexed” and “Crawled – currently not indexed” require different investigation from “Excluded by noindex,” redirects, 404s or server errors.
Sitemap Processing Errors
Malformed XML, unreachable sitemap URLs and protocol/size problems are sitemap-specific technical failures. Correct these first because Google cannot use a sitemap reliably if it cannot fetch or parse it.
6. Dynamic Sitemap Generation and Maintenance
For an actively maintained WordPress or other CMS site, sitemap generation should generally be automated. New canonical content should appear when appropriate, while retired, redirected or noindexed URLs should disappear. Periodically verify the generated output instead of assuming a plugin or CMS configuration remains correct forever.
Audit After Major Site Changes
Recheck sitemaps after migrations, permalink changes, taxonomy cleanup, large content consolidations, plugin changes or major publishing projects. These are common moments for sitemap contents to drift away from the site’s intended indexable URL set.
7. Sitemaps for Different Content Types
Standard Page and Post Sitemaps
These cover ordinary indexable pages and articles and are commonly generated automatically by modern CMS platforms.
Image Sitemaps
Sites where image discovery is strategically important may benefit from image-specific sitemap information, especially where important images are otherwise difficult for crawlers to discover.
Video Sitemaps
Sites publishing substantial original video can use video sitemap information when appropriate to improve discovery and provide additional video metadata.
News Sitemaps
News sitemaps serve a specialized use case for qualifying, time-sensitive news publishers. Most ordinary business websites do not need one.
8. Pagination, Archives and Utility URLs
Do not automatically include every URL WordPress or another CMS can generate. Paginated archives, author archives, search-result pages, empty tags and other utility URLs should be evaluated according to whether they provide standalone search value and are intentionally indexable. Sitemap inclusion should match that decision.
9. Common Sitemap Mistakes
- Including every technically existing URL rather than curating URLs intended for indexing.
- Including redirected, broken, noindexed or non-canonical URLs, which creates contradictory signals.
- Leaving obsolete URLs in dynamically generated sitemaps after content consolidation.
- Assuming sitemap submission guarantees indexing instead of addressing content quality and duplication.
- Artificially changing modification dates without meaningful page updates.
- Ignoring Search Console processing errors.
- Using one poorly organized sitemap for a very large site when a sitemap index would make management clearer.
- Treating sitemap setup as a one-time task rather than periodically verifying the generated URL set.
10. XML Sitemap Troubleshooting Workflow
- Confirm the sitemap URL loads successfully.
- Confirm the XML is valid and Search Console can process it.
- Compare sitemap URLs with the site’s intended canonical URLs.
- Remove redirected, broken and intentionally noindexed entries.
- Check whether important canonical pages are missing.
- Review Search Console Page Indexing reasons for excluded submitted URLs.
- Inspect representative examples rather than assuming every excluded URL has the same cause.
- Audit internal linking and orphan pages.
- Identify duplicate, near-duplicate and low-value content clusters.
- Recheck the sitemap after fixes and allow search engines time to recrawl and reprocess the site.
Frequently Asked Questions
Does submitting a sitemap guarantee that all listed pages will be indexed?
No. A sitemap helps discovery and communicates intended URLs, but Google and other search engines independently decide whether a page should be indexed.
How often should a sitemap be resubmitted?
A properly automated sitemap usually does not need manual resubmission every time content changes. Resubmission can be useful after major restructuring, a sitemap-location change or when resolving a processing issue.
Should paginated archive pages be included?
Only when those URLs are intentionally indexable and provide useful standalone search value. Do not include pagination merely because the CMS generates it.
Should 404 URLs remain in the sitemap?
No, not when they represent genuinely removed URLs. A sitemap should describe the current intended indexable URL set rather than act as a historical inventory.
Why are sitemap URLs crawled but not indexed?
There is no single cause. Review Google’s reported indexing status and inspect representative URLs for technical accessibility, canonical signals, duplication, content overlap, internal discovery and other relevant evidence. Avoid assuming that a particular exclusion label proves a content-quality problem, and do not repeatedly resubmit the sitemap as a substitute for diagnosis.
Can My Advisers help audit and optimize a sitemap?
Yes. Our Technical SEO Services guide explains professional crawling and indexation work. You can also use the 100+ Point SEO Audit Checklist for a DIY review or submit your requirements for help with a specific indexing problem.
11. XML Sitemap Limits and Scaling Strategy
Sitemap files have protocol limits, so large websites should use multiple files and a sitemap index instead of trying to maintain one enormous document. Logical segmentation is more useful than arbitrary splitting because it lets teams compare indexing patterns by content type, market, language or business unit.
For example, an e-commerce site might separate products, categories, editorial content and store locations. A publisher might separate articles by section or publication period. The goal is operational visibility: when one group begins showing unexpected indexing problems, a segmented sitemap makes the issue easier to isolate.
12. Sitemap Indexes for Enterprise Websites
A sitemap index is a master XML file listing individual sitemap files. It reduces maintenance complexity and gives search engines a clear directory of the site’s sitemap structure.
Large organisations should document ownership for each sitemap group and ensure new content types are added intentionally. A template launched without sitemap integration can create orphaned indexable content even when the pages are technically crawlable.
13. WordPress XML Sitemap Management
Modern WordPress can generate native sitemaps, and popular SEO plugins can generate their own versions. Avoid running overlapping sitemap systems unless there is a deliberate reason, because duplicates can confuse site owners and complicate troubleshooting even when search engines can process them.
Review the output after installing or removing SEO plugins. Confirm that the sitemap location referenced in robots.txt and Search Console matches the sitemap you actually intend to maintain.
14. WooCommerce and Product Sitemaps
WooCommerce sites should review product, product-category and other taxonomy sitemaps carefully. Out-of-stock or discontinued products may remain indexable depending on the business model, while thin automatically generated attribute archives may not deserve sitemap inclusion.
Do not remove products from the sitemap simply because inventory is temporarily unavailable if the product page remains useful and is expected to return. Sitemap strategy should follow indexation intent, not short-term stock fluctuations.
15. Sitemap Hygiene After Content Deletion
When content is deleted or consolidated, the sitemap should update accordingly. Old URLs that now redirect or return 404/410 responses should not remain indefinitely in current sitemaps.
If a plugin continues listing retired URLs, investigate caching, plugin configuration or stale generated files. A sitemap should reflect the live site, not become a historical archive of every URL ever published.
16. Sitemap Hygiene After URL Changes
When slugs, categories or permalink structures change, ensure the new canonical URLs replace the old ones in the sitemap. Old addresses can remain accessible through redirects, but the sitemap should normally contain only the final destination.
After a large URL migration, compare the sitemap against the redirect map and crawl results. This helps identify cases where the sitemap still lists legacy URLs or where new destinations are missing.
17. Sitemap and Canonical Tag Consistency
One of the most important technical checks is whether sitemap URLs self-canonicalize or otherwise point to the intended canonical version. If a sitemap lists URL A but the page declares URL B as canonical, the site is sending mixed signals.
Mixed canonical signals are especially common when tracking parameters, faceted URLs, printer-friendly versions or legacy hostname variants are accidentally included. Align sitemaps, internal links and canonical tags wherever possible.
18. Sitemap and Internal Linking
A sitemap should not compensate for poor internal architecture. Important pages should still be discoverable through normal navigation and contextual links. If a valuable page appears only in the sitemap and has no meaningful internal links, it may be technically discoverable but poorly integrated into the site.
Use sitemap audits to uncover orphan pages, then decide whether those pages deserve stronger internal linking, consolidation or removal.
19. Sitemap and Robots.txt Conflicts
Check whether URLs listed in the sitemap are blocked by robots.txt. This creates an obvious inconsistency because the sitemap invites discovery while robots rules prevent crawling.
There can be legitimate edge cases, but most business websites should keep sitemap URLs crawlable so search engines can evaluate their content and directives.
20. Sitemap and Noindex Conflicts
A noindexed URL generally should not remain in an XML sitemap if the intent is to keep it out of search. Remove the URL from the sitemap while leaving it crawlable long enough for search engines to process the noindex directive when needed.
Large-scale mismatches between sitemap inclusion and noindex settings often indicate poor plugin configuration or conflicting SEO workflows.
21. Sitemap and Redirect Conflicts
Redirecting URLs should normally be excluded from active sitemaps. Listing them wastes crawl attention and creates unnecessary ambiguity about which version is preferred.
Audit sitemap response codes in bulk. Any substantial number of 3xx responses deserves investigation, especially after migrations or taxonomy changes.
22. Sitemap and 404/410 Errors
Broken URLs in a sitemap should be treated as maintenance issues. They may result from deleted content, failed imports, incorrect permalink generation or stale caches.
Remove genuinely retired URLs and fix sitemap generation logic so the errors do not return. If the missing content has a valid replacement, implement the appropriate redirect separately.
23. Sitemap and Server Errors
If sitemap URLs frequently return 5xx errors, the problem is larger than sitemap hygiene. Investigate hosting, application stability, database load and caching layers. Search engines cannot reliably crawl content that intermittently fails.
Monitor both the sitemap file itself and the URLs it lists. A perfectly valid XML document is not useful if many listed pages are unavailable.
24. Sitemap Compression and File Delivery
Large sitemap files can be compressed where supported, which reduces transfer size. Ensure compressed files remain accessible to crawlers and are served with correct headers.
Do not introduce unnecessary complexity for small sites. Compression is most useful when sitemap files are genuinely large.
25. Lastmod: When It Helps and When It Misleads
The lastmod value is useful when it reflects meaningful page changes. Automatically resetting it every time a site cache clears or a minor template changes creates noisy signals.
For editorial content, update lastmod when the article itself materially changes. For product pages, meaningful stock, price or specification changes may also justify updates depending on the implementation.
26. Multilingual and International Sitemaps
International websites can organize sitemaps by language or market when that improves operational clarity. Ensure localized URLs are indexable, canonical and internally linked appropriately.
If hreflang is implemented, it can be included in HTML or sitemaps. Whichever method you use, annotations should be accurate and reciprocal where required. Sitemap segmentation does not replace proper international SEO architecture.
27. Image Sitemap Strategy
Image sitemap information is most useful when images are strategically important and may be difficult to discover through standard page HTML. Examples include image-heavy galleries, portfolios and media libraries.
For ordinary business pages, good HTML image implementation, descriptive context and accessible alt text often provide sufficient discovery. Use image sitemap extensions where they add real value rather than by default.
28. Video Sitemap Strategy
Video sitemaps can provide additional metadata about important original videos, including duration, thumbnail and publication information where supported. They are most relevant for sites where video is a meaningful content product.
Ensure video pages themselves are useful and indexable. A video sitemap cannot compensate for thin pages with little context or poor accessibility.
29. News Sitemap Strategy
News sitemaps are specialized and should be used only by qualifying publishers producing recent news content. They are not a general-purpose SEO feature for ordinary blogs or business websites.
Using a specialized sitemap format without meeting the actual use case does not create a ranking advantage.
30. Sitemap Monitoring Dashboard
For larger sites, maintain a simple dashboard comparing submitted URLs, indexed URLs, exclusions and errors by sitemap group. This reveals trends that site-wide totals can hide.
A decline in indexation for one product category or language section can become visible quickly when sitemaps are segmented logically.
31. Sitemap Audit Spreadsheet
A practical audit sheet can include sitemap file, URL, response code, indexability, canonical destination, robots status, internal-link count, lastmod value and Search Console indexing status.
For large datasets, summarize by pattern instead of manually reviewing every URL. The objective is to identify systemic issues such as redirected URLs, noindex conflicts or entire templates missing from the sitemap.
32. Sitemap Troubleshooting After a Migration
After a domain or platform migration, confirm that the new sitemap contains final URLs only and that legacy sitemap files either redirect appropriately or are retired cleanly. Submit the new sitemap in Search Console and monitor indexing by section.
Cross-check the redirect map, canonicals and internal links. A migration can technically succeed at the redirect level while still sending conflicting sitemap signals.
33. Sitemap Troubleshooting After a Plugin Change
SEO-plugin changes can alter sitemap locations, included post types, taxonomy settings and URL formatting. Before deactivating an SEO plugin, record the existing sitemap setup and Search Console submissions.
After the change, validate the new sitemap structure and update references if the URL changed. Avoid leaving Search Console pointed to obsolete sitemap files indefinitely.
34. Sitemap Troubleshooting for “Discovered – Currently Not Indexed”
This status means Google knows about the URL but has not yet crawled it, or has not crawled it recently enough to index. Review whether the site is large, whether internal linking is weak, whether many low-value URLs compete for crawl attention and whether the affected pages are strategically important.
Do not repeatedly resubmit the sitemap expecting the label to disappear. Improve discovery, site quality and crawl efficiency based on evidence.
35. Sitemap Troubleshooting for “Crawled – Currently Not Indexed”
If Google crawled a page but chose not to index it, focus on the page and its relationship to the rest of the site. Review duplication, near-duplication, thin value, search intent, internal linking and canonical signals.
Again, sitemap resubmission does not solve the underlying reason for non-indexation.
36. Sitemap Troubleshooting for Duplicate Canonical Issues
When Google chooses a different canonical from the one declared, compare the sitemap URL, canonical tag, redirects, internal links and duplicate versions. Ensure the preferred URL receives the strongest consistent signals.
Do not try to force canonical selection solely through sitemap inclusion while other site signals point elsewhere.
37. Sitemap Governance for Growing Websites
Assign responsibility for sitemap configuration. Developers may own technical generation, SEO teams may define indexation strategy and editors may control which content types are published. Without clear ownership, sitemap drift can go unnoticed.
Document changes to post types, taxonomies, multilingual architecture and plugin settings that affect sitemap output.
38. 30-Day Sitemap Cleanup Plan
Days 1–5: inventory sitemap files and validate accessibility.
Days 6–10: crawl all listed URLs and identify redirects, errors, noindex pages and canonical conflicts.
Days 11–15: compare sitemaps against internal linking and intended indexation.
Days 16–20: fix generation rules, stale caches and missing important URLs.
Days 21–25: review Search Console indexing patterns by sitemap group.
Days 26–30: document governance and establish recurring validation.
39. Advanced XML Sitemap Checklist
- Sitemap file loads successfully.
- XML is valid.
- Only canonical, indexable URLs are included.
- No redirected URLs remain.
- No 404, 410 or 5xx URLs remain.
- No noindexed URLs remain.
- Robots.txt does not block intended sitemap URLs.
- Important pages are not missing.
- Lastmod values reflect real changes.
- Large sites use logical sitemap indexes.
- WordPress plugins are not generating conflicting sitemap systems.
- Sitemaps align with canonicals and internal links.
- Search Console submissions point to current sitemap locations.
- Migration changes are validated.
40. Additional XML Sitemap FAQs
Should every page on my site be in the sitemap?
No. Include pages you genuinely want search engines to discover and consider for indexing. Utility, duplicate and intentionally excluded pages usually should not be included.
Can I have multiple sitemap files?
Yes. Large sites commonly use multiple files organized through a sitemap index.
Should redirected URLs stay in the sitemap?
No. Replace them with the final canonical destination.
Can a sitemap fix orphan pages?
It can help discovery, but important pages should still have meaningful internal links and architectural context.
Can a sitemap force Google to crawl faster?
No. It helps discovery, but crawl scheduling remains under the search engine’s control.
Final Takeaway
A sitemap is most useful when it accurately mirrors the site’s indexing strategy: canonical, valuable URLs in; obsolete, redirected, duplicate and intentionally excluded URLs out. Treat sitemap data as part of a wider crawling and indexation system involving internal links, canonical signals, content quality and Search Console evidence—not as a button that forces Google to index every URL.
Discover more from My Advisers
Subscribe to get the latest posts sent to your email.