« Return to Thread: does anyone have idea on how to run multiple sequential jobs with bash script

RE: does anyone have idea on how to run multiple sequential jobs with bash script

by Haijun Cao-2 :: Rate this Message:

Reply to Author | View in Thread

Ted,

I find cascading very similar to pig, do you care to provide your comment here? If map reduce programmers are to go to the next level (scripting/query language), which way to go?

Thanks
Haijun
 

-----Original Message-----
From: Ted Dunning [mailto:ted.dunning@...]
Sent: Wednesday, June 11, 2008 2:16 PM
To: core-user@...
Subject: Re: does anyone have idea on how to run multiple sequential jobs with bash script

Just a quick plug for Cascading.  Our team uses cascading quite a bit and
found it to be a simpler way to write map reduce jobs.  The guys using it
find it very helpful.

On Wed, Jun 11, 2008 at 1:31 PM, Chris K Wensel <chris@...> wrote:

>
> Depending on the nature of your jobs, Cascading has built in a topological
> scheduler. It will schedule all your work as their dependencies are
> satisfied. Dependencies being source data and inter-job intermediate data.
>
> http://www.cascading.org
>
>
>


--
ted

 « Return to Thread: does anyone have idea on how to run multiple sequential jobs with bash script