Create and Submit a XML Sitemap

Speeding up Sitemap Submissions for New Pages

Currently there are several ways to submit your pages through Google Console and keeping all the new and old pages together under one sitemap without having to add additional sitemaps.

 

For new websites wanting to get new pages indexed quickly it can sometimes take months for your pages to be crawled. One of the first places to look is in your Google Console to see if your new pages under "Page Indexing" are showing "Crawled - currently not indexed These pages aren't indexed or served on Google". This can be a headache for new websites trying to rank their pages quickly. Before I go over a way around this let’s better understand what options are out there currently:

 

Create New Sitemap:
You can create a new sitemap and simply add your new URLs to a separate sitemap and submit through Google Console.

 

Resubmit Your Sitemap:
You can resubmit your sitemap which means when you add new pages to a sitemap and then you submit it by sending a Ping or HTTP request. Unfortunately, sitemaps ping endpoint is going away. This sitemap protocol defines an unauthenticated REST method for submitting sitemaps to search engines:

 

Read it here dated June 26, 2023: Sitemaps ping endpoint is going away

 

Sitemap Index File:
Using a Sitemap index file is for those who exceed the limits set which you can separate the new files that are below the required limit. Then you can use a sitemap index file to submit them all together.

 

Not all of these may be the best options for newly submitted pages. Changing how you submit might be better. First in Google Console look at the Page Indexing and the Discovery will most likely show that the issue is sitemap related. The reason is that even though the pages are being crawled they not being displayed in Google because the pages are not reflected on the "Last read" sitemap. You can confirm this by going to the Sitemap tab and checking the "Last read" column to see when the sitemap was last submitted. If Google is waiting for the sitemap that includes your new pages then you have to wait until it is included in the submission.

 

Here is one method I use that works well. Remove all of the sitemaps that include the new pages you are wanting to be indexed. Now simply resubmit them and it will immediately reflect today’s date as the "Last read". This will prevent you from having to wait months for a sitemap to be resubmitted.

 

Crawled - currently not indexed:

 

 

Removing Sitemap:

 

 

Sitemap Last Read:

 

 

XML sitemap is a structured file designed for search engines

XML sitemaps are an essential component of search engine optimization (SEO) and website management. They serve as a roadmap for search engines, helping them discover and index your website's content effectively. In this comprehensive guide, we will delve into the process of generating XML sitemaps, their importance in SEO, best practices, and tools to simplify the task.

 

An XML sitemap is a file that lists all the URLs (Uniform Resource Locators) of a website's pages, along with additional metadata like when they were last updated and their priority. This file is intended for search engines, such as Google and Bing, to crawl and index a website's content more efficiently. It provides search engine spiders with a structured overview of your site, ensuring that no page is missed during the indexing process.

 

XML sitemaps are crucial for several reasons:

 

Improved Crawling: XML sitemaps provide search engine bots with a clear path to all of your site's pages, ensuring that no valuable content goes unnoticed. This can help speed up the indexing process.

 

Freshness Signals: By including the last modification date for each URL, you can signal to search engines which pages are updated frequently. This information can influence how often search engines revisit your site.

 

Priority and Importance: XML sitemaps allow you to assign priority levels to different pages. This doesn't guarantee higher rankings, but it helps search engines understand which pages you consider most important.

 

Error Identification: If there are issues with your website, such as broken links or inaccessible pages, an XML sitemap can highlight these problems, enabling you to fix them promptly.

 

Enhanced SEO: XML sitemaps are part of a broader SEO strategy. When combined with other on-page and off-page optimization efforts, they contribute to better search engine rankings.

 

Generating XML sitemaps may seem complex, but following best practices can simplify the process and ensure their effectiveness:

 

Organize Your Content: Ensure your website's content is well-structured. This includes having a logical URL structure and clear site navigation. The organization of your content will directly impact the quality of your XML sitemap.

 

Use the Right Format: XML is the standard format for sitemaps. It's human-readable and machine-parseable, making it ideal for search engines. Ensure your sitemap adheres to XML formatting rules.

 

Prioritize Pages: Assign priority values to your pages using values between 0.0 and 1.0, with 1.0 indicating the highest priority. Reserve high priority for your most crucial pages.

 

Update Frequencies: Specify how often your pages are updated. Options include "always," "hourly," "daily," "weekly," "monthly," "yearly," and "never." Be honest in your assessments, as misleading information can harm your site's credibility.

 

Include All Important Pages: Your XML sitemap should include all your website's important pages, including blog posts, product pages, and landing pages. Exclude pages like login screens, thank-you pages, or other non-essential pages.

 

Limit Sitemap Size: Keep your XML sitemap's size manageable, ideally under 50,000 URLs and 50MB in size. For larger websites, use multiple sitemaps and create a sitemap index file to link them.

 

Validate Your Sitemap: Before submitting your XML sitemap to search engines, use validation tools like Google's Search Console or third-party tools to ensure it's error-free and correctly formatted.

 

Submit to Search Engines: After creating and validating your XML sitemap, submit it to search engines through their respective webmaster tools. This step ensures that search engines are aware of your sitemap and can use it for crawling and indexing.

 

Keep Sitemaps Updated: Regularly update your XML sitemap whenever you add or remove pages from your website. This ensures that search engines always have an accurate representation of your site's content.

 

Monitor Performance: Use webmaster tools to monitor how search engines are using your sitemap. This can provide insights into crawling behavior and potential issues with your site.

 

Several tools and plugins are available to help you generate XML sitemaps effortlessly. Some popular options include:

 

Yoast SEO (WordPress): If your website is powered by WordPress, the Yoast SEO plugin can generate XML sitemaps automatically. It also offers customization options for sitemap content and priorities.

 

SpiderNow: This web-based SEO tool can crawl your website and generate XML sitemaps. It's particularly useful for larger websites with complex structures.

 

Google XML Sitemaps (WordPress): Another WordPress-specific plugin, this tool generates XML sitemaps and automatically notifies search engines when changes are made to your site.

 

XML-sitemaps.com: This online tool is free and can generate XML sitemaps for websites with up to 500 pages. It's a quick solution for smaller sites.

 

Ahrefs: A comprehensive SEO tool like Ahrefs provides XML sitemap generation as part of its suite of SEO features. It's suitable for in-depth SEO analysis and management.

 

Generating XML sitemaps is a crucial step in optimizing your website for search engines. When done correctly, it ensures that your content is discovered and indexed efficiently, ultimately improving your site's visibility in search results. By following best practices and utilizing available tools, you can streamline the process and reap the benefits of a well-structured XML sitemap in your SEO strategy. Remember that sitemaps are just one aspect of SEO, and they work best when combined with other optimization techniques to enhance your website's overall performance in search engine rankings.