[build] Performance tests

[Vicente Botet Escriba]

Hi,

I would like to execute some performance test with Boost.Build. Is there a way to force the execution of the tests without building the executables?

Are there some examples of performance tests for some Boost libraries?

Thanks,
Vicente
_____________________
Vicente Juan Botet Escribá
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Vladimir Prus-3]

vicente.botet wrote:

> Hi,
>
> I would like to execute some performance test with Boost.Build. Is there a way to force the
> execution of the tests without building the executables?

You should be able to do this:

        exe hello : hello.cpp ;
        run-output hello.output : hello ;
        time hello.time : hello.output ;

The first metatarget just builds exe, as usual. The second runs the executable
(capturing the output). The third records the time spend to run the executed.

You might need to 'import testing ; ' on top.

> Are there some examples of performance tests for some Boost libraries?

I am not aware of such.

Hope this helps,
Volodya



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Vicente Botet Escriba]

----- Original Message -----
From: "Vladimir Prus" <vladimir@...>
To: <boost@...>
Sent: Wednesday, November 04, 2009 4:55 PM
Subject: Re: [boost] [build] Performance tests

>
> vicente.botet wrote:
>
>> Hi,
>>
>> I would like to execute some performance test with Boost.Build. Is there a way to force the
>> execution of the tests without building the executables?
>
> You should be able to do this:
>
>        exe hello : hello.cpp ;
>        run-output hello.output : hello ;
>        time hello.time : hello.output ;
>
> The first metatarget just builds exe, as usual. The second runs the executable
> (capturing the output). The third records the time spend to run the executed.
>
> You might need to 'import testing ; ' on top.
>
>> Are there some examples of performance tests for some Boost libraries?
>
> I am not aware of such.
>
> Hope this helps,

Thanks ofor the hint. I didn't know run-output and time. Are these commands documented?

This allows me to get the time but in order to get again the time I need to force to build all with -a. I would like to build all the executables targets, and then be able to run performance tests several times.

BTW, I need to pass some args when running the program. I do this now as




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Vicente Botet Escriba]

Sorry, the last post was not complet.

----- Original Message -----
From: "Vladimir Prus" <vladimir@...>
To: <boost@...>
Sent: Wednesday, November 04, 2009 4:55 PM
Subject: Re: [boost] [build] Performance tests

>
> vicente.botet wrote:
>
>> Hi,
>>
>> I would like to execute some performance test with Boost.Build. Is there a way to force the
>> execution of the tests without building the executables?
>
> You should be able to do this:
>
>        exe hello : hello.cpp ;
>        run-output hello.output : hello ;
>        time hello.time : hello.output ;
>
> The first metatarget just builds exe, as usual. The second runs the executable
> (capturing the output). The third records the time spend to run the executed.
>
> You might need to 'import testing ; ' on top.
>
>> Are there some examples of performance tests for some Boost libraries?
>
> I am not aware of such.
>
> Hope this helps,

Thanks for the hint. I didn't know run-output and time.

This allows me to get the time but in order to get again the time I need to force to build all with -a.

$ bjam -q perf
...patience...
...found 2444 targets...
...updating 5 targets...
gcc.archive /sandbox/stm/branches/vbe/libs/stm/build/bin/gcc-3.4.4/debug/link-st
atic/threading-multi/libboost_STM.a
gcc.link bin/gcc-3.4.4/debug/threading-multi/perf_counter.exe
testing.capture-output bin/gcc-3.4.4/debug/threading-multi/perf_counter.run
testing.time bin/gcc-3.4.4/debug/threading-multi/perf_counter.time
user: [perf_counter.run] 0.106000
system: [perf_counter.run] 0.122000
...updated 5 targets...

$ bjam -q perf
...patience...
...found 2444 targets...

$ bjam -a perf
...patience...
...found 2444 targets...
...updating 15 targets...
gcc.compile.c++ bin/gcc-3.4.4/debug/threading-multi/counter.o
gcc.compile.c++ /sandbox/stm/branches/vbe/libs/stm/build/bin/gcc-3.4.4/debug/lin
k-static/threading-multi/bloom_filter.o
gcc.compile.c++ /sandbox/stm/branches/vbe/libs/stm/build/bin/gcc-3.4.4/debug/lin
k-static/threading-multi/contention_manager.o
gcc.compile.c++ /sandbox/stm/branches/vbe/libs/stm/build/bin/gcc-3.4.4/debug/lin
k-static/threading-multi/transaction.o
gcc.compile.c++ /sandbox/stm/branches/vbe/libs/stm/build/bin/gcc-3.4.4/debug/lin
k-static/threading-multi/except_and_back_off_on_abort_notice_cm.o
gcc.archive /sandbox/stm/branches/vbe/libs/stm/build/bin/gcc-3.4.4/debug/link-st
atic/threading-multi/libboost_STM.a
gcc.compile.c++ /boost_1_39_0/bin.v2/libs/thread/build/gcc-3.4.4/debug/link-stat
ic/threading-multi/pthread/thread.o
gcc.compile.c++ /boost_1_39_0/bin.v2/libs/thread/build/gcc-3.4.4/debug/link-stat
ic/threading-multi/pthread/exceptions.o
gcc.compile.c++ /boost_1_39_0/bin.v2/libs/thread/build/gcc-3.4.4/debug/link-stat
ic/threading-multi/pthread/once.o
gcc.archive /boost_1_39_0/bin.v2/libs/thread/build/gcc-3.4.4/debug/link-static/t
hreading-multi/libboost_thread-gcc34-mt-d-1_39.a
gcc.link bin/gcc-3.4.4/debug/threading-multi/perf_counter.exe
testing.capture-output bin/gcc-3.4.4/debug/threading-multi/perf_counter.run
testing.time bin/gcc-3.4.4/debug/threading-multi/perf_counter.time
user: [perf_counter.run] 0.122000
system: [perf_counter.run] 0.108000
...updated 15 targets...


I would like to build all the executables targets once, and then be able to run performance tests several times. Is this possible?

BTW, I need to pass some args when running the test program. I do this now as

    alias dir_invalid
        :
            [ run stm : -bench rbtree -dir -threads 4 -inserts 100 : : : rbtree_dir_t4_i100 ]
        ...
        ;

How can I integrate run-output and time in a test suite?

Thanks for all,
Vicente


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Robert Ramey]

vicente.botet wrote:

> Thanks ofor the hint. I didn't know run-output and time. Are these
> commands documented?
>
> This allows me to get the time but in order to get again the time I
> need to force to build all with -a. I would like to build all the
> executables targets, and then be able to run performance tests
> several times.
>
> BTW, I need to pass some args when running the program. I do this now
> as

I don't know if this is useful to you, but the serialization library
has a a subdirectory "performance" where it uses bjam
to run performance/profiling tests.  It's lacking a good
set of tests, but the infrastructure might be interesting
to look at.

Robert Ramey



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Vladimir Prus-3]

vicente.botet wrote:

>
> ----- Original Message -----
> From: "Vladimir Prus" <vladimir@...>
> To: <boost@...>
> Sent: Wednesday, November 04, 2009 4:55 PM
> Subject: Re: [boost] [build] Performance tests
>
>
>>
>> vicente.botet wrote:
>>
>>> Hi,
>>>
>>> I would like to execute some performance test with Boost.Build. Is there a way to force the
>>> execution of the tests without building the executables?
>>
>> You should be able to do this:
>>
>>        exe hello : hello.cpp ;
>>        run-output hello.output : hello ;
>>        time hello.time : hello.output ;
>>
>> The first metatarget just builds exe, as usual. The second runs the executable
>> (capturing the output). The third records the time spend to run the executed.
>>
>> You might need to 'import testing ; ' on top.
>>
>>> Are there some examples of performance tests for some Boost libraries?
>>
>> I am not aware of such.
>>
>> Hope this helps,
>
> Thanks ofor the hint. I didn't know run-output and time. Are these commands documented?

Unfortunately not.

> This allows me to get the time but in order to get again the time I need to force to build all
> with -a. I would like to build all the executables targets, and then be able to run performance
> tests several times.
>
> BTW, I need to pass some args when running the program. I do this now as
>
>
>
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Vladimir Prus-3]

vicente.botet wrote:

[sorry for previously posting an incomplete draft of this response]

>> You should be able to do this:
>>
>>        exe hello : hello.cpp ;
>>        run-output hello.output : hello ;
>>        time hello.time : hello.output ;
>>
>> The first metatarget just builds exe, as usual. The second runs the executable
>> (capturing the output). The third records the time spend to run the executed.
>>
>> You might need to 'import testing ; ' on top.
>>
>>> Are there some examples of performance tests for some Boost libraries?
>>
>> I am not aware of such.
>>
>> Hope this helps,
>
> Thanks for the hint. I didn't know run-output and time.
>
> This allows me to get the time but in order to get again the time I need to force to build all
> with -a.
>
> $ bjam -q perf
> ...patience...
> ...found 2444 targets...
> ...updating 5 targets...
> gcc.archive /sandbox/stm/branches/vbe/libs/stm/build/bin/gcc-3.4.4/debug/link-st
> atic/threading-multi/libboost_STM.a
> gcc.link bin/gcc-3.4.4/debug/threading-multi/perf_counter.exe
> testing.capture-output bin/gcc-3.4.4/debug/threading-multi/perf_counter.run
> testing.time bin/gcc-3.4.4/debug/threading-multi/perf_counter.time
> user: [perf_counter.run] 0.106000
> system: [perf_counter.run] 0.122000
> ...updated 5 targets...

> I would like to build all the executables targets once, and then be able to run performance tests
> several times. Is this possible?

It's now possible -- using trunk version of Boost.Build. Add

        always hello.output ;

> BTW, I need to pass some args when running the test program. I do this now as
>
>     alias dir_invalid
>         :
>             [ run stm : -bench rbtree -dir -threads 4 -inserts 100 : : : rbtree_dir_t4_i100 ]
>         ...
>         ;
>
> How can I integrate run-output and time in a test suite?

You might want to try:

        run stm : -bench rbtree -dir -threads 4 -inserts 100 : : : rbtree_dir_t4_i100
          time rbtree_dir_t4_i100.time : rbtree_dir_t4_i100.output ;

You can also create a helper function, run-with-time, like so:

rule run-with-time ( sources + : args * : input-files * : requirements * : target-name ? :
    default-build * )
{
    local t = [ run $(sources) : $(args) : $(input-files) : $(requirements) : $(target-name) ] ;
    local name = [ $(t).name ] ;
    time $(name).time : $(name) ;
}

Beware that the above is untested (but should work)

- Volodya

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost