Building Dynamic Sitemaps in NextJS
Introduction
This article focuses on building dynamic, server-side rendered sitemaps for NextJS.
It’s ideal if you have:
- A massive site with thousands of pages that will take too long to statically render at build time
- These pages are thus SSR’ed or ISR’ed instead at request time.
- The pages change frequently and the site is not rebuilt often
- You need an accurate sitemap to provide to crawlers each time they visit
It’s not as daunting as it may seem, plus you’ll only have to implement it once and forget about it!
If you’re website only has a handful of pages or pages that are statically rendered at build time, this article is not for you. Consider using next-sitemap instead, or manually add a sitemap.
Prerequisites
- We’ll be using TypeScript
- We’ll be using the new App Router, but it can very easily be adopted for the Pages Router too
A full demo (including bonus content) can be found here: https://github.com/arnovanstaden/nextjs-dynamic-sitemaps
Basic Example
For our example, we’ll use the JSONPlaceholder API and build a sitemap for the posts, with a slug structure of /post/{id}.
The process of building a dynamic sitemap can be divided into 3 easy steps:
- Collecting the Dynamic Slugs/Paths
- Building the Sitemap XML
- Creating the Sitemap API Route
1. Collecting the Dynamic Slugs/Paths
If you manage a website of this size, chances are using a Headless CMS of some sort to manage the content, and rendering those pages behind dynamic routes. This is the step where you need to fetch those pages.
We create a new file to fetch the posts from our “CMS” and return the data we need for our sitemap:
src/lib/cms/index.ts
interface Post {
userId: number,
id: number,
title: string,
body: string,
}
export const getAllPosts = (): Promise<Post> => {
try {
return fetch('https://jsonplaceholder.typicode.com/posts')
.then(response => response.json())
} catch (e) {
throw e;
}
}2. Building the Sitemap XML
Next, we need to create a function to build the XML that we will return from our API Route:
src/lib/sitemap/index.ts
/**
* Update this with the domain of your site
*/
const Config = {
domain: 'https://www.test.com'
}
export interface Page {
slug: string,
}
export const createSitemapUrlSet = (pages: Page[]): string => {
const urlSet = pages.map((page) => (
`<url>
<loc>${Config.domain}${page.slug}</loc>
</url>`
)).join('');
return (
`<urlset
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:image="http://www.google.com/schemas/sitemap-image/1.1"
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
${urlSet}
</urlset>`
);
};Because this is a generic function and we might add new paths later (/authors, /collections etc.) to pass to this function, we need to add another small function that takes the data from our CMS and builds the slugs that match the pages in our website.
If your data comes from a Headless CMS, you probably already have a slug field and don’t need this step
src/lib/cms/index.ts
export const getAllPostPaths = async (): Promise<Page[]> => {
const blogPosts = await getAllPosts();
return blogPosts.map((post) => ({
slug: `/post/${post.id}`,
}))
}3. Creating the Sitemap API Route
Create a new API Route for the sitemap, where we fetch the posts and generate the sitemap XML:
src/app/sitemap.xml/route.ts
export const GET = async (): Promise<Response> => {
const blogPostPaths = await getAllPostPaths();
const sitemap = createSitemapUrlSet(blogPostPaths);
return new Response(sitemap, {
headers: {
'Content-Type': 'text/xml'
}
});
};If you start your server and navigate to http://localhost:3000/sitemap.xml, you should see the basic sitemap:
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<url>
<loc>https://www.test.com/post/1</loc>
</url>
<url>
<loc>https://www.test.com/post/2</loc>
</url>
...
</urlset>Additional Considerations
Last Modified Date
Your Headless CMS probably also returns some metadata about pages, such as when they were last updated. This is great to add to your sitemap so crawlers can know what changed since they last visited.
This can be done by updating your sitemap generator to include the <lastmod> tag :
src/lib/sitemap/index.ts
/**
* Update this with the domain of your site
*/
const Config = {
domain: 'https://www.test.com'
}
export interface Page {
slug: string,
lastmod: string, // Add This
}
export const createSitemapUrlSet = (pages: Page[]): string => {
const urlSet = pages.map((page) => (
`<url>
<loc>${Config.domain}${page.slug}</loc>
<lastmod>${page.lastmod}</lastmod> // Add This
</url>`
)).join('');
return (
`<urlset
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:image="http://www.google.com/schemas/sitemap-image/1.1"
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
${urlSet}
</urlset>`
);
};The date needs to be in W3C Datetime format.
Splitting out Large Sitemaps
Sitemaps are only allowed up to 50,000 URLs, so if you have more pages than that, or just want to split your sitemap by concern, you should consider splitting your sitemap into several small sitemaps.
Source: https://www.sitemaps.org/protocol.html
You can serve your sitemap index at /sitemap.xml containing the links to the smaller sitemaps (e.g./sitemap/posts.xml etc.) and use Dynamic Route Segments to serve individual sitemaps at these paths.
Conclusion
Implementing dynamic sitemaps in NextJS, as we’ve explored, ensures your website’s content remains consistently accessible. Even as content creators update or add new pages, your sitemap will always be up-to-date for every crawl.
