Problems with jobqueue.pl falling over

View: New views
3 Messages — Rating Filter:   Alert me  

Problems with jobqueue.pl falling over

by Owen Taylor :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

So, after adding '-d' to the jobqueue.pl options on Max's advice, I was
able to figure out why the mail queue daemon has been falling over over
the last few days.

It's my fault (sort of).

The reason that the problem started showing up now, is that in order to
add links to reviews to outgoing mails for Splinter, I used the
 
 mailer-before_send

hook. (It's not ideal, but its the only hook available in Bugzilla 3.4
to affect outgoing mail, and I want to keep Splinter working with
Bugzilla 3.4 and not require new hook additions.)

Now, if anything in mailer-before_send uses Bugzilla->dbh, then a
database connection is going to be opened in jobqueue.pl and left open.
We had mysqld configured with:

 wait_timeout=820

(About 15 minutes), so if it the splinter hook didn't get run for 15
minutes, then the handle would be closed, and all the outgoing mail
would start producing errors like:

job failed: An error occurred processing hook mailer-before_send in extension
./extensions/splinter: DBD::mysql::st execute failed: MySQL server has
gone away [for Statement ... ]

As a temporary workaround for this I upped the wait timeout to 3600,
and hopefully that will make the problem basically go away for us, but
it's not really the right fix. I have no real idea what the right fix is.

(Moving the Splinter code to a new hook that gets run when formatting
the mail rather than sending the mail would be a fix, but as above I'd
rather not do that to maintain compatibility with released Bugzilla.
And mailer-before_send hooks *should* be able to use Bugzilla->dbh,
it seems to me.)

Any ideas, Max?

- Owen


_______________________________________________
gnome-bugsquad mailing list
gnome-bugsquad@...
http://mail.gnome.org/mailman/listinfo/gnome-bugsquad

Re: Problems with jobqueue.pl falling over

by Max Kanat-Alexander :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On 10/21/2009 08:36 AM, Owen Taylor wrote:
> Now, if anything in mailer-before_send uses Bugzilla->dbh, then a
> database connection is going to be opened in jobqueue.pl and left open.

        Ah, normally Bugzilla reconnects automatically:

        http://search.cpan.org/dist/DBD-mysql/lib/DBD/mysql.pm#mysql_auto_reconnect

        But because jobqueue is a command-line script, it doesn't have that
variable set! We should set that variable. In fact, please file a bug
upstream about it.

        -Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla and Perl Services. Everything Else, too.
       
_______________________________________________
gnome-bugsquad mailing list
gnome-bugsquad@...
http://mail.gnome.org/mailman/listinfo/gnome-bugsquad

Re: Problems with jobqueue.pl falling over

by Olav Vitters-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I've put the following into a daily cron:
  /sbin/service bugzilla-queue restart


--
Regards,
Olav
_______________________________________________
gnome-bugsquad mailing list
gnome-bugsquad@...
http://mail.gnome.org/mailman/listinfo/gnome-bugsquad