Skip to main content

> GUIDE // FOUNDATIONS

Markdown Negotiation for AI Agents

Serving high-density, token-efficient Markdown representations to LLMs via HTTP Accept header negotiation.

Executive Overview

Learn how to configure HTTP content negotiation so requests with Accept: text/markdown receive a clean, token-efficient Markdown document while human browser visitors receive standard HTML.

1. The Negotiation Flow

When an AI agent (such as Claude, Perplexity, or GPTBot) requests a web page with Accept: text/markdown, edge middleware rewrites the internal request to a markdown synthesizer:

  1. Header Inspection: Check for Accept: text/markdown.
  2. AST Transformation: Extract semantic article body, headings, and code blocks while stripping layout navigation.
  3. Response Headers: Serve with Content-Type: text/markdown; charset=utf-8 and Vary: Accept.

Frequently Asked Questions

Why is the Vary: Accept header critical?

Without Vary: Accept, intermediate CDNs and browser caches might cache the Markdown version and serve it to human visitors, or cache HTML and serve it to AI agents.

AI Summary

Learn how to configure HTTP content negotiation so requests with Accept: text/markdown receive a clean, token-efficient Markdown document while human browser visitors receive standard HTML.