Drupal Cron Job Problems

Submitted by Stefan Schneider on Fri, 01/14/2011 - 18:47

Drupal 6 cron jobs perform periodical tasks for a Drupal site. One reason to get in trouble is the following error message in the Drupal DB log:

...
Message: Attempting to re-run cron while it is already running.
Serverity: warning

This means that Drupal thinks a second cron job has been started while the first one hasn't finished.

The root causes circle around the initial cron job which took to much time:

  • Shared hosting environment
  • Scheduled jobs have planned and kicked off before the first one could finish
  • Problems with one of the tasks in the cron job led to a hanging cron job which didn't complete

Problem Analysis Strategy

Check the Schedule for the Cron Job

  1. Use the system log (http://www.yoursite.com/admin/reports/dblog) and identify the past point of times when cron jobs have been kicked off
  2. Consider to increase the period in between cron jobs. Only very few sites require the cron job to run every few minutes. Most low volume sites will do fine with a daily cron job
  3. Make sure that you didn't schedule two sources which kick of cron jobs at a time!

This step is rather unlikely to lead to a solution. It's however a low risk and low cost check and intervention

Do less in the individual Tasks in a Cron Job

Many modules (search, visitor info, xml site map etc.) allow to configure the number of items to be processed in a single task. There are good chances that the daily changes on your site don't justify to scan your entire site every day for all items.

  1. Lower all known settings to reasonable levels

This action is as well a low impact, low risk activity.

Increase the php "max_execution_time"

This action will not be applicable in shared hosting environments.

It will most likely be a temporary relief only. It will not address the root cause of the problem

Disable the problematic Module in Question

This action is suitable if the module didn't provide any benefit to your site anyhow.

This is a valid diagnostic activity to understand the root cause. It may be a temporary relief in order to keep the site working

Analyse the problem which the module has or try to use a different module from Drupal.org

Resetting the Error Condition

It is very likely that the first cron job is already dead and any new job refuses to run since the old job didn't release the semaphor (A blocking variable).

Create a Database Backup

This is a must. The following operations have the chance to corrupt the system if there are remaining active cron jobs.

Remove Semaphor Variables

There are two variables cron_semaphore and cron_last in the table variable which can delete. They will be recreated on demand

dmysql> select * from variable where name = 'cron_semaphore';
+----------------+---------------+
| name           | value         |
+----------------+---------------+
| cron_semaphore | i:1212121212; |
+----------------+---------------+
1 row in set (0.00 sec)
mysql> delete from variable where name = 'cron_semaphore';
Query OK, 1 row affected (0.00 sec)
mysql> select * from variable where name = 'cron_last';
+----------------+---------------+
| name           | value         |
+----------------+---------------+
| cron_semaphore | i:3434343434; |
+----------------+---------------+
1 row in set (0.00 sec)
mysql> delete from variable where name = 'cron_last';
Query OK, 1 row affected (0.00 sec)

Empty Cashes

Some state may be cached. Call flush your cashes

  1. Call page http://www.yoursite.com/admin/settings/performance
  2. Click on "Clear cached data"

Rerun Cron Job manually

You're now able to try again to process your cron job again. The status page http://www.yoursite.com/admin/reports/status will pretend that a cron job never has been run.

  1. Try to eliminate the root cause
  2. Run cron job manually through the page: http://www.yoursite.com/admin/reports/status/run-cron

Identifying the problematic Task

The system log will not tell you why the cron job got stuck. The following code modification will list each individual task which has been started by the cron job. The last task listed in the db log (http://www.yoursite.com/admin/reports/dblog)  is most likely the trouble maker:

Edit file includes/module.inc and change the function module_invoke_all by adding the following line:

foreach (module_implements($hook) as $module) {
  $function = $module .'_'. $hook;
  if ($hook == 'cron') watchdog('cron', "hit $module cron");   // add line to log in db log
  ...
}

This change will the lead to db log entries which look as follows (successful cron job):

You will want to have a closer look at the last entry in the db log. This will be the module which is the trouble maker. There's a chance that the cron job is simply running to long for your hoster. Go through the modules and check the time they need and consider to disable some of them. You may not need all of them.

Alternatives

The supercron module is the alternative to tame the cron tasks. It has the following per task features:

  • Enable and disable the task
  • Check the runtime and the average run time of the task(very informative)
  • Allows to invoke individual task by hand.
  • Allows to limit cron initiatore to a given IP address

It allows as well to limit access to the cron job. This helps to avoid Denial-of-Service attacks.

Update: This module has been EOLed (End of Life) for security reasons (see this note).

I'm using Cron Control as of now as a replacement:

  • enable and disable task
  • execution time of tasks need to be checked through the database log
  • no fine grained access control for cron jib submitter IP address

References

This solution strategy has been developed using material from people who understand the subject better than me. Check out the following links to get an even better understanding of the subject:

 

 

  • 16640 views