How to use the warmer with Cloudflare or a CDN
The warmer sends ordinary HTTP requests to your store URLs from the server where Magento cron runs. This page explains how these requests work together with Cloudflare, another CDN, or a load balancer in front of your store.
Two separate caches
When your store is behind Cloudflare or another CDN, pages are cached in two places:
| Cache | Where it is | What fills it |
|---|---|---|
| CDN cache | Cloudflare (or other CDN) edge servers | visitor requests that pass through the CDN |
| Magento full page cache | your server (Built-in cache, Redis, or Varnish) | any request that reaches Magento, including warmer requests |
The warmer fills and checks the Magento full page cache only. It does not warm the CDN cache and does not need any special settings to work with a CDN. Both caches work as separate layers: when the CDN does not have a page, it requests the page from your server, and a warmed page is then returned quickly from the Magento cache.
A page cached by the CDN can still be uncached in Magento, so the warmer still requests it. See How to reduce the number of warming requests.
Warming requests are blocked by Cloudflare
If the store domain points to Cloudflare, warming requests go through Cloudflare too. Cloudflare bot protection can treat them as bot traffic and block them with the 403 Forbidden response. In this case pages stay Pending, and the 403 errors appear in var/log/cache-warmer.log when Log warmer requests is enabled.
To fix this, allow the IP address of your Magento server in Cloudflare:
- Log in to the Cloudflare dashboard and select your domain.
- Go to Security -> WAF -> Tools.
- In IP Access Rules, enter the public IP address of your Magento server, select the Allow action and This website, and click Add.
If several servers run Magento cron, add the IP address of each of them.
Another way is to send warming requests directly to your server, so they do not go through Cloudflare at all. See the section below.
Send warming requests directly to your server
The extension has no setting for a separate warming address: it requests the same URLs that visitors open. The server resolves the store domain the usual way, so warming requests can go out to the internet, through the CDN or load balancer, and come back to the same server.
On cloud hosting, such as AWS, this traffic can add costs (for example, NAT gateway or load balancer traffic) and extra load on the CDN. To keep warming requests inside your network, make the store domain point to the internal address of your web server, only for the server where Magento cron runs:
-
/etc/hosts: on the server that runs Magento cron, add a line with the private IP address of your web server (or internal load balancer) and the store domain:10.0.1.25 www.example.comAdd every domain the warmer requests, for example, the domains of all store views.
-
AWS Route 53 private hosted zone: create a private hosted zone for the store domain, attach it to the VPC of your servers, and add a record that points the domain to the internal address. Servers inside the VPC then use the internal address, while visitors still use the public one.
The web server at the internal address must serve the store for this domain, including HTTPS when your store URLs use https://. The warmer does not check the SSL certificate, so the internal address can use the same certificate as the public site.
Warming requests that do not go through the CDN or load balancer reach the server without the headers these services add (for example, X-Forwarded-Proto). If your web server or Varnish relies on such headers, check that pages warmed this way get the Cached status and that visitors are served the cached version. Use How to verify if pages are cached.