feat: serve markdown to agents via Accept content negotiation

[sriramveeraghanta]

Goal

Return a markdown representation of a page when an agent sends Accept: text/markdown, while HTML stays the default for browsers — Cloudflare's "Markdown for Agents" pattern, implemented Vercel-natively (docs.plane.so is on Vercel, not Cloudflare-proxied).

What was actually broken

The repo was already 95% there — buildEnd() copies a .md twin of every page into dist/, and Vercel serves those with Content-Type: text/markdown (verified live). But the Accept: text/markdown negotiation never fired:

Vercel applies vercel.json rewrites after the filesystem. Every clean URL (/foo) already resolves to an existing /foo.html, so the rewrite to /foo.md was permanently shadowed. Redeploying would not have fixed it.

The fix has to run before the filesystem. The only Vercel mechanisms that do are redirects and Routing/Edge Middleware — middleware was chosen for transparent same-URL negotiation.

Changes

File	Change
middleware.ts (new)	On Accept: text/markdown, transparently rewrites /foo → /foo.md (same URL, no redirect). Matcher skips assets/fonts/extensioned paths; root / → /index.md.
package.json	+ @vercel/functions (rewrite/next helpers)
vercel.json	Removed the dead, shadowed rewrites block
docs/.vitepress/config.ts	Updated two stale comments to point at middleware.ts

Cache safety: Vercel's CDN already includes the Accept header in its cache key by default, so the HTML and markdown variants of a URL cache as separate entries — no Vary needed, no risk of agents' markdown being served to browsers.

Verified locally

• pnpm build passes — 117 .md files + llms.txt still emit
• rewrite/next signatures match usage (read from .d.ts); esbuild bundles cleanly
• pnpm check:format clean

Needs deploy to confirm

Middleware doesn't run under vitepress preview, so end-to-end behavior must be checked on a Vercel preview/prod deploy:

# Markdown at the SAME url, no redirect
curl -sI -H "Accept: text/markdown" https://docs.plane.so/core-concepts/issues/overview | grep -iE "http|content-type"   # 200 + text/markdown

# Browsers still get HTML
curl -sI https://docs.plane.so/core-concepts/issues/overview | grep -i content-type   # text/html

# Official check
curl -s -X POST https://isitagentready.com/api/scan -H "Content-Type: application/json" \
  -d '{"url":"https://docs.plane.so"}' | grep -o '"markdownNegotiation":{[^}]*}'   # "status":"pass"

Tradeoff

The middleware runs as a small fluid-compute function on every page request (before the cache), returning next() instantly for browsers. That's the cost of transparent same-URL negotiation vs. a pure-static redirect.

Summary by CodeRabbit

• New Features
  • Added content negotiation support to serve markdown versions of pages for requests accepting text/markdown format, enabling better integration with AI agents and tools.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jun 19, 2026 2:36pm

[coderabbitai]

Warning

Review limit reached

@sriramveeraghanta, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 47 minutes and 37 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7426f766-f66a-4386-853a-cebc544a236f

📥 Commits

Reviewing files that changed from the base of the PR and between 2548b54 and 2a111af.

📒 Files selected for processing (1)

• middleware.ts

📝 Walkthrough

Walkthrough

Markdown content negotiation is migrated from a vercel.json rewrites rule to a new middleware.ts file. The middleware inspects the Accept header for text/markdown and rewrites the request path to the corresponding .md file; otherwise it passes through. The @vercel/functions package is added, the old vercel.json rewrites block is removed, and inline comments in config.ts are updated to reflect the new architecture.

Changes

Markdown Content Negotiation via Middleware

Layer / File(s)	Summary
Middleware implementation and dependency middleware.ts, package.json, vercel.json	Adds middleware.ts exporting a default middleware function and config.matcher; the middleware checks Accept: text/markdown and rewrites to the .md URL twin (handling /, trailing slashes, bare paths). Adds @vercel/functions (^3.7.1) to dependencies. Removes the now-superseded vercel.json rewrites array that performed the same mapping via routing config.
config.ts comment updates docs/.vitepress/config.ts	Rewords inline comments to clarify that the llmstxt plugin owns only llms.txt/llms-full.txt, and that buildEnd copies source .md files into dist/ for middleware.ts to serve via Accept: text/markdown negotiation.

Sequence Diagram(s)

sequenceDiagram
  participant Agent as Agent/Client
  participant Middleware as middleware.ts
  participant Edge as Vercel Edge
  participant Static as dist/*.md

  Agent->>Middleware: GET /docs/page (Accept: text/markdown)
  Middleware->>Middleware: Strip trailing slash → /docs/page.md
  Middleware->>Edge: rewrite(/docs/page.md)
  Edge->>Static: Fetch static .md file
  Static-->>Agent: 200 text/markdown

  Agent->>Middleware: GET /docs/page (Accept: text/html)
  Middleware->>Edge: next() — no rewrite
  Edge-->>Agent: 200 text/html

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

• devanshuplane
• danciaclara

Poem

🐇 A rewrite rule once lived in JSON so plain,
Now middleware hops through the Accept refrain.
"text/markdown?" it sniffs, then rewrites with glee,
Serving .md twins for agents to see.
The old rule is gone — the rabbit leaps free! 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat: serve markdown to agents via Accept content negotiation' directly and clearly summarizes the main change: implementing content negotiation to serve markdown files to AI agents.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/markdown-for-agents

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@middleware.ts`:
- Around line 27-30: The Accept header negotiation at line 28 uses a simple
substring check with includes() that doesn't properly handle case variants,
quality values (q-values), or proper media range parsing. Replace the substring
check with proper media type parsing logic that parses the Accept header into
individual media ranges with their quality values, checks if text/markdown is
present and not explicitly unacceptable (where q=0 means reject), and performs
case-insensitive matching. This ensures the Accept negotiation follows HTTP
content negotiation standards and correctly handles cases like
"text/markdown;q=0" which should be treated as unacceptable.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 78da4a5d-67f9-4786-b725-672adefa483d

📥 Commits

Reviewing files that changed from the base of the PR and between 389a8aa and 2548b54.

⛔ Files ignored due to path filters (1)

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

📒 Files selected for processing (4)

• docs/.vitepress/config.ts
• middleware.ts
• package.json
• vercel.json

💤 Files with no reviewable changes (1)

• vercel.json