Skip to main content

How to Generate Dynamic XML Sitemaps in Next.js 14 App Router

 Migrating a production application from the Next.js Pages directory to the App Router brings significant performance benefits, particularly with React Server Components (RSC). However, it often breaks established SEO workflows.

The most common point of failure during this migration is the sitemap.xml.

In the legacy Pages directory, developers relied on getServerSideProps inside a pages/sitemap.xml.js file to construct XML strings manually. In Next.js 14, this approach throws errors. The file system routing has changed, and simply returning an XML string is no longer the optimized standard.

This guide details the architectural shift in Next.js 14 and provides a production-ready implementation for generating dynamic, database-driven sitemaps using the native sitemap.ts convention.

The Root Cause: Why Legacy Sitemaps Fail

To fix the problem, we must understand the architectural change. The Next.js 14 App Router moves away from Node.js-style request/response handlers as the default for page rendering.

1. The Disappearance of getServerSideProps

The App Router utilizes Server Components. Data fetching happens directly inside the component or via fetch requests that are deduped and cached. The concept of a "page prop" passed from a server-side method is obsolete in this context.

2. Route Handlers vs. Special Files

While you could technically create a app/sitemap.xml/route.ts file and manually return an XML response with specific headers, Next.js 14 introduces a Special File Convention: sitemap.ts.

Using sitemap.ts allows the framework to:

  • Automatically handle XML serialization.
  • Manage Content-Type headers (application/xml).
  • Optimize caching strategies specifically for crawler directives.
  • Type-check your return values against the Sitemap protocol.

The Solution: Dynamic sitemap.ts Implementation

We will create a sitemap that combines static routes (like /about) with dynamic routes fetched from an external CMS or database (like /blog/[slug]).

Prerequisites:

  • Next.js 13.4+ or 14
  • TypeScript (highly recommended for strict typing)

Step 1: Create the API Service

First, ensure you have a robust method to fetch your data. In a real-world scenario, you want to fetch only the fields necessary for the sitemap (slug and updated date) to reduce database load.

// lib/blog-service.ts

// Simulating a database fetch. 
// In production, replace this with Prisma, Sanity, Contentful, etc.

export type BlogPost = {
  slug: string;
  updatedAt: string;
};

export async function getBlogPostsForSitemap(): Promise<BlogPost[]> {
  // Simulating network delay
  // await new Promise((resolve) => setTimeout(resolve, 100));

  // Example return data
  return [
    { slug: 'nextjs-14-seo-guide', updatedAt: '2024-03-15T09:00:00.000Z' },
    { slug: 'react-server-components-deep-dive', updatedAt: '2024-02-20T14:30:00.000Z' },
    { slug: 'mastering-typescript-generics', updatedAt: '2024-01-10T08:15:00.000Z' },
  ];
}

Step 2: Implement app/sitemap.ts

Create a file named sitemap.ts in the root of your app directory. This file must export a default function.

This function supports async/await, allowing us to perform database operations before rendering the sitemap.

// app/sitemap.ts
import { MetadataRoute } from 'next';
import { getBlogPostsForSitemap } from '@/lib/blog-service';

// Ideally, pull this from your environment variables
const BASE_URL = process.env.NEXT_PUBLIC_SITE_URL || 'https://www.example.com';

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  // 1. Fetch dynamic data
  const posts = await getBlogPostsForSitemap();

  // 2. Map dynamic data to the sitemap format
  const blogEntries: MetadataRoute.Sitemap = posts.map((post) => ({
    url: `${BASE_URL}/blog/${post.slug}`,
    lastModified: new Date(post.updatedAt),
    changeFrequency: 'weekly',
    priority: 0.7,
  }));

  // 3. Define static pages manually
  const staticEntries: MetadataRoute.Sitemap = [
    {
      url: `${BASE_URL}`,
      lastModified: new Date(),
      changeFrequency: 'daily',
      priority: 1,
    },
    {
      url: `${BASE_URL}/about`,
      lastModified: new Date(),
      changeFrequency: 'monthly',
      priority: 0.8,
    },
    {
      url: `${BASE_URL}/contact`,
      lastModified: new Date(),
      changeFrequency: 'yearly',
      priority: 0.5,
    },
  ];

  // 4. Return the combined array
  return [...staticEntries, ...blogEntries];
}

Deep Dive: How It Works

When a search engine bot (Googlebot, Bingbot) requests https://yoursite.com/sitemap.xml, Next.js intercepts this request via the file convention.

Type Safety with MetadataRoute.Sitemap

By strictly typing the return with MetadataRoute.Sitemap, we ensure conformity to the XML Sitemap protocol. TypeScript will throw a build-time error if you attempt to add invalid properties or malformed dates.

Automatic XML Generation

You no longer need to write boilerplate XML strings like <urlset xmlns="...">. Next.js takes the array of objects returned by your function and compiles a valid XML response with the correct namespaces.

Caching and Revalidation

This is the most critical aspect for performance.

By default, if your sitemap.ts does not use dynamic functions (like reading headers or cookies), Next.js will cache this file at build time.

If your blog updates frequently, you need to ensure the sitemap reflects new content. You can export a revalidate constant from the file to control the cache lifetime (Incremental Static Regeneration).

// app/sitemap.ts

// Update the sitemap at most once every hour
export const revalidate = 3600; 

export default async function sitemap() { ... }

Handling Edge Cases: The 50,000 URL Limit

A standard sitemap file has a hard limit of 50,000 URLs or 50MB in size. If you are building a large-scale e-commerce site or a user-generated content platform, a single array will fail.

Next.js 14 supports Sitemap Generation (splitting) natively.

If your sitemap function needs to handle pagination, you can use the generateSitemaps function.

// app/sitemap.ts

// Generate sitemaps for IDs 0 through 4 (splitting into 5 files)
export async function generateSitemaps() {
  return [
    { id: 0 }, 
    { id: 1 }, 
    { id: 2 }, 
    { id: 3 },
    { id: 4 }
  ]
}

export default async function sitemap({ id }: { id: number }): Promise<MetadataRoute.Sitemap> {
  // Fetch only the subset of data based on the ID
  // e.g., limit 10000 offset (id * 10000)
  const posts = await getBlogPostsSubset(id); 

  return posts.map((post) => ({
    url: `${BASE_URL}/blog/${post.slug}`,
    lastModified: post.updatedAt,
  }));
}

This will automatically generate sitemap/0.xml, sitemap/1.xml, etc., and a sitemap index file at sitemap.xml.

Common Pitfalls to Avoid

1. Hardcoding Domains

Never hardcode https://localhost:3000 in your sitemap logic. Use environment variables (NEXT_PUBLIC_SITE_URL or VERCEL_URL) to ensure the sitemap generates valid canonical URLs in production, staging, and local environments.

2. Ignoring robots.ts

A sitemap is useless if crawlers can't find it. While Google usually checks /sitemap.xml by default, it is a best practice to declare it explicitly in a robots.txt file.

Next.js 14 also provides a convention for this:

// app/robots.ts
import { MetadataRoute } from 'next';

export default function robots(): MetadataRoute.Robots {
  const BASE_URL = process.env.NEXT_PUBLIC_SITE_URL;

  return {
    rules: {
      userAgent: '*',
      allow: '/',
      disallow: ['/private/', '/admin/'],
    },
    sitemap: `${BASE_URL}/sitemap.xml`,
  };
}

3. Ignoring lastModified

Google relies heavily on lastModified to determine if it needs to recrawl a page. If you omit this, or if you set it to new Date() for every single request regardless of content changes, you waste your "Crawl Budget." Always map this to the actual updatedAt field from your database.

Conclusion

Migrating to the Next.js 14 App Router requires a shift in mental models from "File Responses" to "Data Exports." By leveraging app/sitemap.ts, you remove the complexity of manual XML handling while gaining type safety and better integration with the framework's caching layer.

Ensure your environment variables are set, your robots.ts points to the sitemap, and your revalidation strategies match your content update frequency for optimal search engine visibility.

Labels: