Skip to main content

How to fix the PageSpeed warning "llms.txt does not follow the recommendations"

Google PageSpeed Insights and Lighthouse can report "llms.txt does not follow the recommendations" with the detail "It does not appear that the file contains any links". The warning means the checker received a file with no Markdown links in it. With the Agentic Commerce extension for Magento 2 that has one of the causes below. Check them in this order.

1. Check what a crawler actually receives​

Fetch the file the way a crawler does, from a computer outside your server (a request made from the server itself skips your CDN and firewall):

curl -s -D - -A "Mozilla/5.0 (compatible; Google-InspectionTool/1.0)" https://example.com/llms.txt | head -20

A correct response is 200 with Content-Type: text/plain; charset=utf-8, followed by the instructions text with links such as - [UCP Discovery](https://example.com/.well-known/ucp).

If the response is text/html, a 403, or a "checking your browser" page, a bot protection layer is answering instead of your store: Cloudflare Bot Fight Mode or WAF rules, Anubis, Sucuri, or a hosting firewall. It serves crawlers a challenge they cannot solve, so the checker sees a page with no links.

note

Opening /llms.txt in a browser does not show this problem. A browser solves the challenge automatically and then displays the real file.

Allow these paths through the protection layer:

  • /llms.txt and /agents.md
  • /.well-known/ucp
  • /sitemap_agentic_discovery.xml
  • /agent/*

For Cloudflare rules, Bot Fight Mode and how to test the result, follow the same steps as in the MCP Server guide for protected stores, using the paths above. Run the curl check again afterwards.

If the crawler receives the plain-text file but it has no links, open the template:

  • Go to Stores -> Configuration -> Mirasvit Extensions -> Agentic Commerce.
  • Expand the Agent Instructions Template group and click Preview.

The endpoint links come from {{ agentic.info }}. It lists the UCP discovery and MCP endpoints while Enable UCP Merchant Profile or Enable Generic MCP Endpoint is Yes, and the product JSON endpoints while Enable Public JSON Read API is Yes (all in the HTTP Surfaces group). With all three off, or with {{ agentic.info }} removed from a customized LLMs Template, the file has no endpoint links.

The categories and policies sections add links only when they have entries: categories appear when the store has active categories, and policies when pages are chosen in Policy Pages. Choose your policy pages there, or add a link to the template yourself. See How to customize the agent instructions.

3. Upgrade the extension​

Older versions of the extension wrote the endpoint list as plain URLs, not as Markdown links, and the checker counts only links. If the preview shows the endpoint URLs but no [...](...) links, upgrade the extension to the latest version.