Next.js SEO: Metadata, Sitemaps and Rendering Choices That Work

Next.js gives you good tools for search-friendly pages, but it does not apply them for you. A new App Router project ships a charset tag and a viewport tag and little else. Titles, descriptions, canonical links, a sitemap, a robots.txt file and correct 404 behaviour are all things you add on purpose. When they are missing or wrong, the site still looks fine in a browser while search engines see duplicate or thin pages.

This guide is for developers and technical site owners who run a Next.js site with the App Router. You will set page metadata with the Metadata API, give every page a canonical URL, generate a sitemap and robots.txt from code, return real 404 responses, and make redirects permanent where they should be. You will also learn how to check the raw HTML a crawler receives, which is the test that matters.

Nothing here makes a page rank. These steps remove technical obstacles so search engines can crawl, render and understand what you publish. The examples use example.com and TypeScript.

Before you begin

Access: you need the source code, a way to run a production build locally (next build then next start), and a deploy path you control.
Router type: this guide covers the App Router (the app/ directory). The Pages Router uses different APIs for metadata, so check which one your project uses first.
Risk: low for metadata, moderate for redirects and robots rules, because a wrong rule can hide pages or loop. Work on a branch and test the production build, not only next dev.
Back up: commit current work first. Save a copy of your live robots.txt and your current sitemap so you can compare before and after.

How Next.js delivers metadata and content

Pages and layouts in the App Router are Server Components by default. Next.js renders them on the server, and the HTML is sent to the browser as a fast, non-interactive preview before JavaScript makes it interactive (Next.js documentation, Server and Client Components). That is good for SEO, because the content exists in the first response.

Google processes JavaScript pages in three phases: crawling, rendering and indexing. Content that only appears after client-side code runs can still be seen, but only after a render step. Google recommends server-side rendering or pre-rendering, partly because not every bot runs JavaScript. For the wider picture, read JavaScript SEO and rendering basics.

The practical rule: keep titles, descriptions, canonical links, headings, main content and internal links in the server-rendered output. Use Client Components for interaction, not for the content you want found.

Step-by-step instructions

Step 1: Set a site-wide metadata base and title template

Why: relative URLs in metadata fail the build unless Next.js knows your domain, and a title template keeps naming consistent.

Action: in app/layout.tsx, export a metadata object with metadataBase and a title template.

import type { Metadata } from 'next'

export const metadata: Metadata = {
  metadataBase: new URL('https://www.example.com'),
  title: {
    default: 'Example Store',
    template: '%s | Example Store',
  },
  description: 'Handmade goods shipped from our workshop.',
}

Two rules from the documentation matter here. A title.default is required whenever you add a title.template, and the template applies to child segments, not to the segment that defines it. A page that needs a title with no suffix can use title: { absolute: '...' }.

Confirm: use a relative field such as alternates.canonical: '/' in the same file. The build passes only because metadataBase exists. Without it, Next.js raises a build error for a relative URL in a URL-based field.

Step 2: Give every page its own title, description and canonical

Why: duplicated titles and missing canonicals are the most common metadata faults. Our guide on writing titles and meta descriptions covers the writing; this step covers the wiring.

Action: in a static page, export metadata. In a dynamic route, export generateMetadata.

// app/products/[slug]/page.tsx
import type { Metadata } from 'next'
import { getProduct } from '@/lib/products'

type Props = { params: Promise<{ slug: string }> }

export async function generateMetadata({ params }: Props): Promise<Metadata> {
  const { slug } = await params
  const product = await getProduct(slug)
  return {
    title: product.name,
    description: product.summary,
    alternates: { canonical: `/products/${slug}` },
  }
}

With metadataBase set, the canonical resolves to a full URL in a <link rel="canonical"> tag. Know these limits: you cannot export both metadata and generateMetadata from the same segment, and both are supported only in Server Components. If a page needs client features, keep page.tsx as a Server Component and move the interactive part to a separate file marked 'use client'.

Merging is shallow. If a layout sets openGraph with a title and description, and a page sets openGraph with only a title, the page's object replaces the layout's. The description disappears. Share common fields through an exported variable and spread it.

Confirm: view the page source of a product URL and find one <title>, one description and one canonical that matches the address you want indexed. For the reasoning behind canonicals, see canonical URLs and duplicate pages.

Step 3: Control indexing with the robots metadata field

Why: pages such as internal search results or thank-you pages should stay out of the index, and robots.txt is the wrong tool for that (it blocks crawling, not indexing).

Action: set the robots field on those routes.

export const metadata: Metadata = {
  robots: { index: false, follow: true },
}

Next.js writes this as <meta name="robots" content="noindex, follow" />. Do not set it in a shared layout unless you mean it for every child page.

Confirm: fetch the page source and look for the robots meta tag. Search for it on pages you want indexed too: it must be absent there.

Step 4: Generate the sitemap from your data

Why: a sitemap lists the canonical URLs you want crawled, and generating it from your data keeps it current.

Action: create app/sitemap.ts. It returns an array of objects with a url and optional lastModified, changeFrequency, priority, alternates, images and videos fields.

import type { MetadataRoute } from 'next'
import { getAllProducts } from '@/lib/products'

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const products = await getAllProducts()
  return [
    { url: 'https://www.example.com/' },
    ...products.map((p) => ({
      url: `https://www.example.com/products/${p.slug}`,
      lastModified: p.updatedAt,
    })),
  ]
}

Only set lastModified from a real edit date. Google says it uses lastmod only when the value is consistently and verifiably accurate, and that it ignores priority and changefreq. Our guide on sitemap tags explains more, so leave the last two out.

The file is a special Route Handler that is cached by default unless it uses a request-time API. For large catalogues, export generateSitemaps to split output into several files. Next.js serves them at paths like /product/sitemap/1.xml. A single sitemap is limited to 50,000 URLs and 50 MB uncompressed, so split before you reach that. You then need an index; see sitemap indexes for large websites.

Confirm: open /sitemap.xml on the production build. It returns XML, every URL is absolute and uses your canonical host, and no URL redirects or returns an error.

Step 5: Generate robots.txt and point it at the sitemap

Why: robots.txt tells crawlers which paths they may fetch and where your sitemap lives.

Action: create app/robots.ts.

import type { MetadataRoute } from 'next'

export default function robots(): MetadataRoute.Robots {
  return {
    rules: { userAgent: '*', allow: '/', disallow: '/api/' },
    sitemap: 'https://www.example.com/sitemap.xml',
  }
}

Next.js turns this into User-Agent, Allow, Disallow and Sitemap lines. Do not disallow /_next/ paths that carry your CSS and JavaScript. Google does not render JavaScript from blocked files, so blocking them can hide content. Read robots.txt blocking search engines before adding broad rules.

Confirm: /robots.txt loads with HTTP 200 and lists the right sitemap URL. Test an important page URL with the URL Inspection tool in Search Console.

Step 6: Return real 404 responses and permanent redirects

Why: a missing page that answers 200 is a soft 404, which wastes crawl effort and clutters reports.

Action for 404s: call notFound() from a route when the record does not exist, and provide app/not-found.tsx for the page design. Know the limit: before streaming starts Next.js returns a 404 status, but after streaming has started the status stays 200 while the not-found UI renders. In both cases Next.js adds <meta name="robots" content="noindex" />. So check your status codes on the production build for every dynamic route rather than assuming.

Action for redirects: add them in next.config.js.

module.exports = {
  async redirects() {
    return [
      { source: '/old-blog/:slug', destination: '/blog/:slug', permanent: true },
    ]
  },
}

permanent: true sends a 308 and permanent: false sends a 307. Google treats 301 and 308 as permanent redirects and 302, 303 and 307 as temporary, so use permanent: true only when the move will not be reversed. Keep each redirect a single hop (see redirect chains and loops).

Confirm: curl -I https://www.example.com/old-blog/hello shows 308 and a location header pointing at the final URL, and a made-up product slug returns 404.

Step 7: Verify what crawlers actually receive

Why: the browser shows the finished page; the crawler's first view is the HTML response.

Action: after next build and next start, run curl -s https://www.example.com/products/blue-mug and read the output. Look for the title, description, canonical, robots tag, <h1>, main text and <a href> links to related pages. Google discovers links only in <a> elements with an href, so click handlers on buttons do not count. Then run URL Inspection in Search Console and view the rendered HTML.

One streaming detail: on dynamically rendered pages, Next.js can stream metadata after the initial UI, appending it to the body. For bots that expect metadata in the <head>, such as Bingbot and Twitterbot, it keeps metadata in the head. Prerendered pages resolve metadata at build time. The htmlLimitedBots option in next.config can turn streaming off, at the cost of possibly slower responses.

Confirm: each important page shows its metadata and content in the response or in the rendered HTML from Search Console.

Final verification checklist

metadataBase is set and the production build passes.
Every indexable page has a unique title, a description and a canonical on your preferred host.
Pages meant to stay out of search carry a noindex robots tag and are not blocked in robots.txt.
/sitemap.xml returns valid XML with only canonical, working URLs.
/robots.txt returns 200 and includes the Sitemap: line.
Missing records return a 404 status on the production build.
Moved URLs return a single permanent redirect to the final address.
The raw HTML of key pages contains content and normal <a href> links.

Common mistakes to avoid

Testing only in next dev. Development behaviour differs from production. Build and run the production server before you check.
Relying on client-side code for titles or content. Use Server Components and the Metadata API so the first HTML has them.
Overwriting nested metadata by accident. A page-level openGraph replaces the layout's whole object. Spread shared values.
Blocking a page in robots.txt and expecting noindex to work. Crawlers must fetch the page to see the tag. Pick one mechanism per URL.
Filling lastModified with the build date. Use real content edit dates or leave the field out.
Trusting notFound() blindly. Confirm the status code for streamed pages and dynamic routes.
Using permanent: true for a short campaign redirect. Browsers and search engines treat it as lasting. Use false when the move is temporary.
Putting links in buttons or scripts. Use real anchors with an href.

Frequently asked questions

Is server rendering required for Google to index my Next.js site?

Not strictly. Google renders JavaScript. But it recommends server-side rendering or pre-rendering, and relying on a client-only shell adds a render step and a failure risk. Server-rendered content is the safer default.

Will a sitemap get all my pages indexed?

No. Google says a sitemap helps discovery, but it does not promise that every listed URL is crawled or indexed. Quality, duplication and access still decide the outcome.

Should I use `metadata` or `generateMetadata`?

Use the static metadata object when values do not depend on request data. Use generateMetadata when titles or descriptions come from a database, CMS or route parameters. You cannot export both from one segment.

Why does my 404 page return status 200?

If the response had already started streaming when notFound() ran, the status stays 200 even though the not-found UI renders. Next.js still adds a noindex tag. Check your routes in production and restructure data loading if the status matters for that route.

Official references

Next step: See how Crawvix checks a website and capture a baseline once scanning opens. This guide provides technical information, not a guarantee of rankings, traffic or index inclusion.