is anyone using threaded automake?

View: New views
3 Messages — Rating Filter:   Alert me  

is anyone using threaded automake?

by Ralf Wildenhues :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Since the 1.11 release I have not seen one bug report about
$AUTOMAKE_JOBS aka. automake running several Perl threads to
update several Makefile.in files concurrently, for speed.
If you have more than a couple of Makefile.am files and a SMP
system, then it should help.

Is anyone using it?  Has anyone seen issues with it?

Bugs could manifest in unstable Makefile.in content,
i.e., running it a second time creates a different file,
and/or unstable warning/error output from automake, thus by
definition be hard to detect and hard to debug.  Which is
why I'm asking whether I should assume "it just works" or
"nobody uses it" or "there is not enough use to expose bugs".

Thanks,
Ralf



Re: is anyone using threaded automake?

by jasonsewall :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Sun, Nov 1, 2009 at 6:45 AM, Ralf Wildenhues <Ralf.Wildenhues@...> wrote:
> Since the 1.11 release I have not seen one bug report about
> $AUTOMAKE_JOBS aka. automake running several Perl threads to
> update several Makefile.in files concurrently, for speed.
> If you have more than a couple of Makefile.am files and a SMP
> system, then it should help.
>
> Is anyone using it?  Has anyone seen issues with it?

I didn't even know it existed, but I just tried it out and it seems to
generate correct output for a project I have with 12 Makefile.am
files. I have a quad core machine and set AUTOMAKE_JOBS=4.

The performance gains are fairly small, however:
$ AUTOMAKE_JOBS=1 time automake
1.31user 0.19system 0:01.51elapsed 100%CPU

$ AUTOMAKE_JOBS=4 time automake
1.62user 0.24system 0:01.14elapsed 163%CPU

And they are imperceptible with autoreconf.

> Bugs could manifest in unstable Makefile.in content,
> i.e., running it a second time creates a different file,
> and/or unstable warning/error output from automake, thus by
> definition be hard to detect and hard to debug.  Which is
> why I'm asking whether I should assume "it just works" or
> "nobody uses it" or "there is not enough use to expose bugs".

I made two copies of the project, ran AUTOMAKE_JOBS=1 autoreconf -i in
one and AUTOMAKE_JOBS=20 autoreconf -i in another and diff -r said the
results were the same.

FWIW,
Jason



Re: is anyone using threaded automake?

by Ralf Wildenhues :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Jason,

* Jason Sewall wrote on Sun, Nov 01, 2009 at 06:28:31PM CET:

> On Sun, Nov 1, 2009 at 6:45 AM, Ralf Wildenhues wrote:
> >
> > Is anyone using it?  Has anyone seen issues with it?
>
> I didn't even know it existed, but I just tried it out and it seems to
> generate correct output for a project I have with 12 Makefile.am
> files. I have a quad core machine and set AUTOMAKE_JOBS=4.
>
> The performance gains are fairly small, however:
> $ AUTOMAKE_JOBS=1 time automake
> 1.31user 0.19system 0:01.51elapsed 100%CPU
>
> $ AUTOMAKE_JOBS=4 time automake
> 1.62user 0.24system 0:01.14elapsed 163%CPU

Yeah, the perl thread spawning really has high overhead, and the output
synchronization isn't all that performant either.  I did get quite
acceptable speedup out of a 150-makefile project.

> And they are imperceptible with autoreconf.

Oh well.

Thanks for testing,
Ralf