Streaming Status Updates for a Long-running Run Mode

[eric.berg]

I've written a cgiapp that runs about 20 java processes in the
background and I'd like to update the web page that submitted the
request (via ajax) to run them with status, incidcating when each
process has completed.

Currently, my rm method just sits there running the subprocesses (using
Paralell::ForkManager, btw) until it's done.  This could be upwards of
15 minutes, so the initial request is treated as failed.

How can I update my web page with status from this run mode while the
subprocesses are running?

Thanks!

Eric
_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[eric.berg]

I've written a cgiapp that runs about 20 java processes in the
background and I'd like to update the web page that submitted the
request (via Ajax) to run them with status, indicating when each process
has completed.

Currently, my rm method just sits there running the subprocesses (using
Paralell::ForkManager, btw) until it's done.  This could be upwards of
15 minutes, so the initial request is treated as failed.

How can I update my web page with status from this run mode while the
subprocesses are running?

Thanks!

Eric
_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[Michael Peters]

On 10/07/2009 10:33 AM, eric.berg@... wrote:

> How can I update my web page with status from this run mode while the
> subprocesses are running?

There's a couple of different ways this can be done. The best is
definitely not the easiest but it means creating a separate offline job
queue that can run these processes and keep their status in a shared
location (like a DB, etc). Then your rm just adds a job to the queue and
returns. Then when you're checking on the status via Ajax you need to
have another rm that simply checks on the status of the job and returns
a flag that could mean success, failure, or try again later (still working).

Unfortunately, since you are using Ajax, you can't use the standard NPH
approach and just periodically print something to the browser while it's
working. Ajax requests don't return (in the javascript) until the entire
request is done and it will probably timeout on you.

--
Michael Peters
Plus Three, LP

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[eric.berg]

Thanks, Michael.  Good point about AJAX not returning until the entire
request is done.

The approach I'm working on now is to have a global status hash for each
file being processed, which I'll update as I run each subprocess and
when each completes.  Then I've thrown in a quick run mode that simply
sends that hash back as JSON and I'm putting some logic into my page to
periodically make a request for this runmode so I can update the page
with current status info.

On a related note, is there a way to have cgiapp send content back to
the client during the processing of a run mode instead of at the end
when the rm returns?

Eric

> -----Original Message-----
> From: cgiapp-bounces@...
> [mailto:cgiapp-bounces@...] On Behalf Of Michael Peters
> Sent: Wednesday, October 07, 2009 11:28 AM
> To: CGI Application
> Subject: Re: [cgiapp] Streaming Status Updates for a
> Long-running Run Mode
>
> On 10/07/2009 10:33 AM, eric.berg@... wrote:
>
> > How can I update my web page with status from this run mode
> while the
> > subprocesses are running?
>
> There's a couple of different ways this can be done. The best is
> definitely not the easiest but it means creating a separate
> offline job
> queue that can run these processes and keep their status in a shared
> location (like a DB, etc). Then your rm just adds a job to
> the queue and
> returns. Then when you're checking on the status via Ajax you need to
> have another rm that simply checks on the status of the job
> and returns
> a flag that could mean success, failure, or try again later
> (still working).
>
> Unfortunately, since you are using Ajax, you can't use the
> standard NPH
> approach and just periodically print something to the browser
> while it's
> working. Ajax requests don't return (in the javascript) until
> the entire
> request is done and it will probably timeout on you.
>
> --
> Michael Peters
> Plus Three, LP

_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[Michael Peters]

On 10/07/2009 11:35 AM, eric.berg@... wrote:

> The approach I'm working on now is to have a global status hash for each
> file being processed, which I'll update as I run each subprocess and
> when each completes.  Then I've thrown in a quick run mode that simply
> sends that hash back as JSON and I'm putting some logic into my page to
> periodically make a request for this runmode so I can update the page
> with current status info.

I really wouldn't do that. Don't tied up your web server for long
running tasks just so that you can wait to show a status to the user.
Even something as simple as Unix "at" for a simple queue would be better.

> On a related note, is there a way to have cgiapp send content back to
> the client during the processing of a run mode instead of at the end
> when the rm returns?

No, not for Ajax. And it's not a limitation of cgiapp, but of HTTP/Ajax.
For non-Ajax you use Non-Parsed-Headers (NPH) which means you tell C::A
to not send headers and you then instead print them yourselves. And then
periodically print more things to the client (like some JS that updates
a progress bar, etc). But like I said in the other email, this won't
work for Ajax.

--
Michael Peters
Plus Three, LP

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[eric.berg]

> -----Original Message-----
> From: cgiapp-bounces@...
> [mailto:cgiapp-bounces@...] On Behalf Of Michael Peters
> Sent: Wednesday, October 07, 2009 11:44 AM
> To: CGI Application
> Subject: Re: [cgiapp] Streaming Status Updates for a
> Long-running Run Mode
>
> On 10/07/2009 11:35 AM, eric.berg@... wrote:
>

> > The approach I'm working on now is to have a global status  hash for
each
> > file being processed, which I'll update as I run each subprocess and
> > when each completes.  Then I've thrown in a quick run mode  that
simply
> > sends that hash back as JSON and I'm putting some logic  into my
page to
> > periodically make a request for this runmode so I can  update the
page
> > with current status info.
>
> I really wouldn't do that. Don't tied up your web server for long
> running tasks just so that you can wait to show a status to the user.
> Even something as simple as Unix "at" for a simple queue
> would be better.

I wouldn't do that either, but I've had no end of issues with running
subprocesses from Apache2.  Among them is that of the environment's not
being passed to the subprocess, the way apache messes with STDOUT, and
the fact that it insists on starting CGI's (mod_perl handlers) in the
directory in which they are invoked, regardless of the current working
directory of the mod_perl handler.

I just ran across the 'at' solution, which I like quite a bit, however
it turns out that we do not receive the email for the users that the web
server runs under and 'at' sends error and other confirmation
information via email, so I have no idea at this point why my
subprocesses aren't running.  Yes.  It's heinous to tie up apache with
running subprocesses, but regardless of whether I'm using
Apache2::SubProcess->spawn_proc_prog() or IPC::Run3, I still get screwed
one way or another.

I wrote a wrapper for running via a number of different methods when
this problem first emerged after migrating to apache2.  I used
Module::Pluggable, so all I had to do was to implement a runner plugin
that uses 'at', which was nice, but it's failing and I'll have to wait
until I get our IT crew to understand and help with the email problem.
(don't ask)

>
> > On a related note, is there a way to have cgiapp send content back
to
> > the client during the processing of a run mode instead of at the end
> > when the rm returns?
>
> No, not for Ajax. And it's not a limitation of cgiapp, but of
HTTP/Ajax.
> For non-Ajax you use Non-Parsed-Headers (NPH) which means you  tell
C::A
> to not send headers and you then instead print them  yourselves. And
then
> periodically print more things to the client (like some JS  that
updates
> a progress bar, etc). But like I said in the other email, this won't
> work for Ajax.

Understood that you can't do this with Ajax, but how would you do it
with cgiapp in a non-ajax context?

>
> --
> Michael Peters
> Plus Three, LP
_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[Michael Peters]

On 10/07/2009 12:24 PM, eric.berg@... wrote:

> I wouldn't do that either, but I've had no end of issues with running
> subprocesses from Apache2.

Me: Doctor it hurts when I try to run subprocesses from Apache2.
Doctor: Don't do that :)

What I try to do is have an external process which looks for jobs to do
and then does them. Then my web app simply puts things in the job db
table and the job queue picks them up and processes them, putting the
status (complete, failed, working, etc) back in the db table. Then my
web processes that check the status of jobs checks to see if it's done
and then shows that in the browser.

Almost all web applications that I know have some things that should be
delegated to an external job queue. Even something as simple as sending
an email shouldn't be done from the web processes (could get stuck on
DNS lookups or SMTP waiting, etc).

> I just ran across the 'at' solution, which I like quite a bit, however
> it turns out that we do not receive the email for the users that the web
> server runs under and 'at' sends error and other confirmation
> information via email, so I have no idea at this point why my
> subprocesses aren't running.

If you want to go the "at" route instead of writing your own queue (or
using an existing queue) then have "at" call a Perl script that wraps
around the real process doing the work. Then it could set a failed
status and error messages in the DB or something that you could have
access to. Obviously if this wrapper fails it would have the same
problems, but that should be pretty infrequent since your wrapper could
be really simple.

> Understood that you can't do this with Ajax, but how would you do it
> with cgiapp in a non-ajax context?

Something like this rough code should work

$self->header_type('none');
print "Content-type: text/html\n\n";
$print $main_content; # but don't close the <html> tag yet

while($not_done) {
    # do some work
    ...
    # periodically print so the browser doesn't close the socket
    print "\0";

    # or print some progress that the already printed document knows
    # how to deal with
    print "<script type="text/javascript">update_progress(10)</script>";
}

print "</html>";

return;

--
Michael Peters
Plus Three, LP

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[eric.berg]

 

> -----Original Message-----
> From: cgiapp-bounces@...
> [mailto:cgiapp-bounces@...] On Behalf Of Michael Peters
> Sent: Wednesday, October 07, 2009 12:36 PM
> To: CGI Application
> Subject: Re: [cgiapp] Streaming Status Updates for a
> Long-running Run Mode
>
> On 10/07/2009 12:24 PM, eric.berg@... wrote:
>
> > I wouldn't do that either, but I've had no end of issues
> with running
> > subprocesses from Apache2.
>
> Me: Doctor it hurts when I try to run subprocesses from Apache2.
> Doctor: Don't do that :)

We've got a ton of legacy code that executes subprocesses for many
things.  I've been migrating them to use the process runner wrapper so
that I can just swap out a back-end for running, but...well...it's
legacy and is in constant conflict with all of the new stuff I have to
do.

 

> What I try to do is have an external process which looks for
> jobs to do
> and then does them. Then my web app simply puts things in the job db
> table and the job queue picks them up and processes them, putting the
> status (complete, failed, working, etc) back in the db table. Then my
> web processes that check the status of jobs checks to see if
> it's done
> and then shows that in the browser.
>
> Almost all web applications that I know have some things that
> should be
> delegated to an external job queue. Even something as simple
> as sending
> an email shouldn't be done from the web processes (could get stuck on
> DNS lookups or SMTP waiting, etc).

What do you use for your external job queue?

>
> > I just ran across the 'at' solution, which I like quite a
> bit, however
> > it turns out that we do not receive the email for the users
> that the web
> > server runs under and 'at' sends error and other confirmation
> > information via email, so I have no idea at this point why my
> > subprocesses aren't running.
>
> If you want to go the "at" route instead of writing your own
> queue (or
> using an existing queue) then have "at" call a Perl script that wraps
> around the real process doing the work. Then it could set a failed
> status and error messages in the DB or something that you could have
> access to. Obviously if this wrapper fails it would have the same
> problems, but that should be pretty infrequent since your
> wrapper could
> be really simple.
>
> > Understood that you can't do this with Ajax, but how would you do it
> > with cgiapp in a non-ajax context?
>
> Something like this rough code should work
>
> $self->header_type('none');
> print "Content-type: text/html\n\n";
> $print $main_content; # but don't close the <html> tag yet
>
> while($not_done) {
>     # do some work
>     ...
>     # periodically print so the browser doesn't close the socket
>     print "\0";
>
>     # or print some progress that the already printed document knows
>     # how to deal with
>     print "<script
> type="text/javascript">update_progress(10)</script>";
> }
>
> print "</html>";
>
> return;

Great!  Just what I was looking for.  The $self->header_type('none'),
printing your own headers, and using print makes perfect sense for
"getting around" the cgiapp "restrictions".

Thanks again, Michael.
Eric

>
> --
> Michael Peters
> Plus Three, LP
>
> #####  CGI::Application community mailing list  ################
> ##                                                            ##
> ##  To unsubscribe, or change your message delivery options,  ##
> ##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
> ##                                                            ##
> ##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
> ##  Wiki:          http://cgiapp.erlbaum.net/                 ##
> ##                                                            ##
> ################################################################
>
>

_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[Michael Peters]

On 10/07/2009 01:39 PM, eric.berg@... wrote:

> What do you use for your external job queue?

At $work we use a custom job queue that fits into our specific
environment. But there are some generic ones like Gearman and The Schwartz.

--
Michael Peters
Plus Three, LP

#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################

[Emmanuel Seyman-2]

* eric.berg@... [07/10/2009 21:05] :
>
> What do you use for your external job queue?

Please please take a look at TheSchwartz.
http://search.cpan.org/~bradfitz/TheSchwartz-1.07/

Emmanuel


#####  CGI::Application community mailing list  ################
##                                                            ##
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp    ##
##                                                            ##
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:          http://cgiapp.erlbaum.net/                 ##
##                                                            ##
################################################################