Running just a single test

View: New views
2 Messages — Rating Filter:   Alert me  

Running just a single test

by Nathan Gallaher :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


I'm trying to debug a failure exposed by the test suite.  I've been running
opencv/bin/cvtest and my failure happens somewhere in the middle. I'm
wondering if there's a way to get these testers to just run the specific
test that I'm interested in, so I don't have to wait for all the other tests
to run while I'm debugging.   I poked around in CvTest but didn't see any
immediately obvious way to do this.   The --help for cvtest says something
about a config file, but I don't see any documentation or examples of this
immediately.

Thanks,

~Nate


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Opencvlibrary-devel mailing list
Opencvlibrary-devel@...
https://lists.sourceforge.net/lists/listinfo/opencvlibrary-devel

Re: Running just a single test

by Vadim Pisarevsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello Nathan,

Good question. I was going to provide some description of the configuration files, but never actually did it.

First, you run cvtest:

./cvtest -w -f _mycfg.yml

that will generate the configuration file _mycfg.yml with some common and test-specific parameters. The name of the file can be arbitrary, extension should be .yml or .xml, .yml is easier to read and easier to modify. Normally, you will only need to modify the common parameters. For example, you found that morph-erode fails. Then you can add

tests: "morph-er*"

to the "common" section. Beware that the file should not include TABs, use only spaces for formatting, and the indentation does matter, YAML is similar to Python in that. You can put several comma-separated test name patterns, each of those may end with "*". Another example:

tests: "m*, calibratecamera"  # run all the tests starting with m and also the calibratecamera test.

If you do not remember the exact test name, run

./cvtest -l # lists all the available tests

Now run the test:

./cvtest -f _mycfg.yml

it will run the specified tests. The order is always alphabetical, regardless of the order of the patterns in "tests" parameter.
Suppose that morph-erode failed again. If not, run cvtest several times, each time it will generate another set of test cases. When it failed, open
_mycfg.log (that is, the file that has the same base name as the configuration file, but .log extension), you will see the detailed information about the failed test, most important of which is seed, 16-digit hexadecimal number without "0x" prefix This is the Random Number Generator start value before running the test case which failed. To reproduce the test, add the parameter seed into the section common:

seed: "0123456789abcdef"

where 0123456789abcdef should be replaced with the seed value from .log file.

If you run the test again:
./cvtest -f _mycfg.yml,

the test should fail again on the very first test case (of course, if the failed test (morph-erode) was executed first). Now you can debug the code.

Regards,
Vadim

On Fri, Oct 16, 2009 at 10:21 PM, Nathan Gallaher <ngallaher@...> wrote:

I'm trying to debug a failure exposed by the test suite.  I've been running
opencv/bin/cvtest and my failure happens somewhere in the middle. I'm
wondering if there's a way to get these testers to just run the specific
test that I'm interested in, so I don't have to wait for all the other tests
to run while I'm debugging.   I poked around in CvTest but didn't see any
immediately obvious way to do this.   The --help for cvtest says something
about a config file, but I don't see any documentation or examples of this
immediately.

Thanks,

~Nate


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Opencvlibrary-devel mailing list
Opencvlibrary-devel@...
https://lists.sourceforge.net/lists/listinfo/opencvlibrary-devel


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Opencvlibrary-devel mailing list
Opencvlibrary-devel@...
https://lists.sourceforge.net/lists/listinfo/opencvlibrary-devel