Skip to main content
This guide shows how to configure search engine indexing for your Sabo app: robots rules, a dynamic sitemap, site-wide metadata (Open Graph/Twitter), and JSON‑LD structured data.

What’s included

  • robots: src/app/robots.ts
  • sitemap: src/app/sitemap.ts
  • Site-wide metadata (Open Graph, Twitter, robots): src/app/layout.tsx
  • JSON‑LD (structured data, Organization): src/app/layout.tsx
Always use your production domain in SEO outputs (e.g., https://yourdomain.com), and restrict indexing in preview environments.

Key concepts at a glance

  • robots.txt: A plain-text file that tells crawlers which paths they may crawl or must avoid, and where your sitemap lives.
  • sitemap.xml: A machine-readable list of your site’s URLs (and optional metadata like last modified). Helps crawlers discover content faster.
  • Site-wide metadata (Open Graph, Twitter, robots): Meta tags that control link previews (title, description, image) and crawling behavior.
  • JSON‑LD (structured data): Embedded JSON that describes your pages to search engines (e.g., Organization, BlogPosting) to enable rich results.

robots.txt

File: src/app/robots.ts
src/app/robots.ts
1

Set your domain

Change the sitemap URL to your production domain.
Visit /robots.txt and confirm the sitemap points to https://yourdomain.com/sitemap.xml.
2

Control indexing

Add items to disallow to prevent indexing of private/auth pages. For preview deploys, disallow everything or use a noindex robots policy via site metadata (see below).

sitemap.xml

File: src/app/sitemap.ts
src/app/sitemap.ts
1

Set baseUrl

Replace https://demo.getsabo.com with your production domain (must match robots.ts).
2

Add dynamic routes

The example shows blog/changelog from MDX. Add other collections (products, docs, etc.) following the same pattern.
3

Test the output

Visit /sitemap.xml and verify all routes appear with correct timestamps.
The file is sitemap.ts but Next.js serves it as /sitemap.xml automatically. Use as const for TypeScript strict mode.

Site-wide metadata (Open Graph, Twitter, robots)

File: src/app/layout.tsx
src/app/layout.tsx
1

Set metadataBase

Update metadataBase to your production domain. This becomes the base for all relative URLs in meta tags.
2

Customize all fields

Update title, description, keywords, and social tags (openGraph, twitter) to match your brand.
3

Add OG images

Create 1200×630 images in public/og/. Keep titles/descriptions consistent across openGraph and twitter.
For preview/staging environments, conditionally set robots.index: false to prevent accidental indexing.

Route-level metadata

For specific pages, you can export page-level metadata (or generateMetadata) to override the defaults.
Use alternates.canonical to avoid duplicate content when pages can be reached by multiple URLs.

JSON‑LD (Structured data)

File: src/app/layout.tsx (Organization). You can add per‑page JSON‑LD (e.g., BlogPosting) in page files.
src/app/layout.tsx
1

Customize fields

Update name, url, logo, and description. Add social profile URLs to sameAs array.
2

Add to layout

Place the script in <head> of your root layout. The .replace(/</g, "\\u003c") prevents script injection.
For blog posts, add page-level JSON-LD with @type: "BlogPosting" including author, datePublished, and image.

Verification and testing

1

Manual checks

  • Visit /robots.txt and /sitemap.xml
  • View page source for meta tags and JSON‑LD
2

Tools

  • Google Rich Results Test (structured data)
  • OpenGraph Preview tools (OG tags)
  • Twitter Card Validator
  • Search Console/Bing Webmaster for submission and coverage
3

Common issues

  • Wrong domain in metadataBase or sitemap
  • Preview builds accidentally indexed (set noindex)
  • OG images missing or wrong path
Deployed site exposes correct robots and sitemap, has consistent OG/Twitter meta, and validates in rich results tests.