After battling with terrible response time on three WordPress sites on the same hosting account for three days, I think I’ve finally gotten to the source of the issue: the wp-cron.php file. This file is how WordPress runs scheduled tasks, and is intended to run silently in the background without affecting load times. However, as I found out, this isn’t always the case.
From what I can tell from reading various posts on other forums, wp-cron.php is designed to function in a manner that is “non-blocking,” that is it should allow other processes be kicked of and run before it completes. However this is not always achievable, and if it can’t carry out its action ideally, it will fall back to a blocking system that prevents other processes from running until it completes.
If this occurs, it can create an unacceptably long wait times on initial loads while wp-cron.php completes whatever backlog of tasks have accumulated since the last time it was called. I noticed up to 45-60 seconds of wait time on my initial loads while this problem was in full swing.
I should note that I also found some interpretations that suggest the problem may not be with wp-cron.php falling back to a blocking execution, but instead process throttling on the part of the hosting provider. This could happen if your hosting provider limits the number of concurrent processes that can run on your account, and the wp-cron.php processes prevent subsequent processes from starting until there’s room.
This problem was exacerbated when I tried running caching plugins (W3 Total Cache and WP Super Cache specifically) with “preload mode” or cache priming turned on. These modes essentially create long backlogs of processes for wp-cron.php, so hitting the site for the first time after a long period of inactivity caused a ridiculous delay.
Once I made the connection between the response lag and wp-cron.php I was able to find some good suggestions to fix it. The idea behind it is to prevent wp-cron.php from being called when pages load, and instead to tie it to a recurring cron job.
Step one is to insert a simple line into wp-config.php that removes the wp-cron.php binding from every page load:
Once this is done, we need to create a cron job that will call wp-cron.php at a set interval. I saw a few suggestions of how to implement this, including using a simple wget command and calling the server’s php binary and feeding it the URL of wp-cron.php. I had issues getting either of these methods working, so I built a similar but slightly different method.
I created a php file that made a cURL request to wp-cron.php. This effectively acts like any normal web request to that file, and allows it to execute any tasks than have been accumulating since it was last called:
$ch = curl_init();
$url = "http://yourserver/wp-cron.php";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
I set this file to run once every 15 minutes, and my site response times have been excellent ever since.
I should make one more note, there are some admin functions that relay on wp-cron.php, and usually call it immediately after they’ve been run. So it’s possible that, if you implement this change, that some admin functions you’re used to seeing take place immediately will now be put off until your cron job next calls wp-cron.php.