This guide shows how to configure search engine indexing for your Sabo app: robots rules, a dynamic sitemap, site-wide metadata (Open Graph/Twitter), and JSON‑LD structured data.
What’s included
- robots:
src/app/robots.ts - sitemap:
src/app/sitemap.ts - Site-wide metadata (Open Graph, Twitter, robots):
src/app/layout.tsx - JSON‑LD (structured data, Organization):
src/app/layout.tsx
Key concepts at a glance
- robots.txt: A plain-text file that tells crawlers which paths they may crawl or must avoid, and where your sitemap lives.
- sitemap.xml: A machine-readable list of your site’s URLs (and optional metadata like last modified). Helps crawlers discover content faster.
- Site-wide metadata (Open Graph, Twitter, robots): Meta tags that control link previews (title, description, image) and crawling behavior.
- JSON‑LD (structured data): Embedded JSON that describes your pages to search engines (e.g., Organization, BlogPosting) to enable rich results.
robots.txt
File:src/app/robots.ts
src/app/robots.ts
Set your domain
Change the
sitemap URL to your production domain.Visit
/robots.txt and confirm the sitemap points to https://yourdomain.com/sitemap.xml.sitemap.xml
File:src/app/sitemap.ts
src/app/sitemap.ts
Add dynamic routes
The example shows blog/changelog from MDX. Add other collections (products, docs, etc.) following the same pattern.
The file is
sitemap.ts but Next.js serves it as /sitemap.xml automatically. Use as const for TypeScript strict mode.Site-wide metadata (Open Graph, Twitter, robots)
File:src/app/layout.tsx
src/app/layout.tsx
Set metadataBase
Update
metadataBase to your production domain. This becomes the base for all relative URLs in meta tags.Customize all fields
Update
title, description, keywords, and social tags (openGraph, twitter) to match your brand.Route-level metadata
For specific pages, you can export page-level metadata (orgenerateMetadata) to override the defaults.
JSON‑LD (Structured data)
File:src/app/layout.tsx (Organization). You can add per‑page JSON‑LD (e.g., BlogPosting) in page files.
src/app/layout.tsx
Verification and testing
Tools
- Google Rich Results Test (structured data)
- OpenGraph Preview tools (OG tags)
- Twitter Card Validator
- Search Console/Bing Webmaster for submission and coverage
Deployed site exposes correct robots and sitemap, has consistent OG/Twitter meta, and validates in rich results tests.