|
View:
New views
8 Messages
—
Rating Filter:
Alert me
|
|
|
JMock not thread safe - how does one do multi-threaded testing?Hi,
The current policy of not supporting multi-threaded tests has just bitten me. We've got a spanking new CI server with multiple cores and are now getting build failures; I think due to JMock not being thread-safe; specifically the Mockery class which is declared as an instance of my test fixture and used throughout each @Test method. java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at java.util.ArrayList.ensureCapacity(ArrayList.java:170) at java.util.ArrayList.add(ArrayList.java:351) at org.jmock.Mockery.dispatch(Mockery.java:219) at org.jmock.Mockery.access$000(Mockery.java:43) at org.jmock.Mockery$MockObject.invoke(Mockery.java:258) at org.jmock.internal.InvocationDiverter.invoke(InvocationDiverter.java:27) at org.jmock.internal.FakeObjectMethods.invoke(FakeObjectMethods.java:38) at org.jmock.lib.JavaReflectionImposteriser$1.invoke(JavaReflectionImposteriser.java:33) at $Proxy9.getProperty(Unknown Source) at com.example.mediatranscoder.MediaTranscoderRequestContext.getStringProperty(MediaTranscoderRequestContext.java:274) at com.example.mediatranscoder.MediaTranscoderRequestContext.getStringProperty(MediaTranscoderRequestContext.java:122) at com.example.mediatranscoder.MediaAssetInfo.getHost(MediaAssetInfo.java:68) at com.example.mediatranscoder.MediaAssetInfo.toString(MediaAssetInfo.java:99) at com.example.mediatranscoder.media.locator.MediaAssetLocatorCacheManager.retrieveFromCache(MediaAssetLocatorCacheManager.java:38) at com.example.mediatranscoder.media.locator.ConcurrentCacheAccessTest$4.run(ConcurrentCacheAccessTest.java:278) at com.example.mediatranscoder.media.locator.ConcurrentTestUtils$1.run(ConcurrentTestUtils.java:74) The code being tested is effectively a singleton cache manager which takes a cache key and Callable block to populate a cache entry if there isn't one already. It's fairly heavily based on the cache given in Goetz et al [1]. The tests where I'm seeing failures are the ones where I'm using an Executor to simulate lots of concurrent requests trying to retrieve the same item from the cache. One of the requests should win and populate the cache. All others should block until the cache has been successfully populated. I'm fairly happy that my code is actually thread-safe and doing the right thing (of course, I may be wrong!), so I could mark the tests as @Ignore for now. But that's not a great place to be. I don't want someone (including me) to make a change which breaks the working code, etc. I think I can provide a small-ish sample test if required which demonstrates the problem (eventually, if you have enough CPUs). What suggestions do you have for testing this scenario? Cheers, James [1] http://jcip.net/listings/Memoizer.java --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: JMock not thread safe - how does one do multi-threaded testing?You might want to see Nat's slides on testing async code
http://www.natpryce.com/articles/000755.html and his series of articles about JMock and Threads. You'll probably also need to pull the latest code from the subversion repository. S. On 1 Apr 2009, at 11:27, James Abley wrote: > The current policy of not supporting multi-threaded tests has just > bitten me. We've got a spanking new CI server with multiple cores and > are now getting build failures; I think due to JMock not being > thread-safe; specifically the Mockery class which is declared as an > instance of my test fixture and used throughout each @Test method. Steve Freeman Winner of the Agile Alliance Gordon Pask award 2006 http://www.m3p.co.uk M3P Limited. Registered office. 2 Church Street, Burnham, Bucks, SL1 7HZ. Company registered in England & Wales. Number 03689627 --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: JMock not thread safe - how does one do multi-threaded testing?JMock is thread-safe if only one thread enters the Mockery at a time.
(E.g. it doesn't use thread-local variables or any magic like that). So, if your code is meant to guarantee that only one thread calls a mock object at a time, but you're seeing the Mockery fail because of race-conditions, then your code is not thread safe. If you want multiple threads to enter the Mockery at once, you should use the SynchronisingImposteriser that is currently in Subversion head but will be in the upcoming 2.6.0 release. --Nat 2009/4/1 James Abley <james.abley@...>: > Hi, > > The current policy of not supporting multi-threaded tests has just > bitten me. We've got a spanking new CI server with multiple cores and > are now getting build failures; I think due to JMock not being > thread-safe; specifically the Mockery class which is declared as an > instance of my test fixture and used throughout each @Test method. > > java.lang.ArrayIndexOutOfBoundsException > at java.lang.System.arraycopy(Native Method) > at java.util.ArrayList.ensureCapacity(ArrayList.java:170) > at java.util.ArrayList.add(ArrayList.java:351) > at org.jmock.Mockery.dispatch(Mockery.java:219) > at org.jmock.Mockery.access$000(Mockery.java:43) > at org.jmock.Mockery$MockObject.invoke(Mockery.java:258) > at org.jmock.internal.InvocationDiverter.invoke(InvocationDiverter.java:27) > at org.jmock.internal.FakeObjectMethods.invoke(FakeObjectMethods.java:38) > at org.jmock.lib.JavaReflectionImposteriser$1.invoke(JavaReflectionImposteriser.java:33) > at $Proxy9.getProperty(Unknown Source) > at com.example.mediatranscoder.MediaTranscoderRequestContext.getStringProperty(MediaTranscoderRequestContext.java:274) > at com.example.mediatranscoder.MediaTranscoderRequestContext.getStringProperty(MediaTranscoderRequestContext.java:122) > at com.example.mediatranscoder.MediaAssetInfo.getHost(MediaAssetInfo.java:68) > at com.example.mediatranscoder.MediaAssetInfo.toString(MediaAssetInfo.java:99) > at com.example.mediatranscoder.media.locator.MediaAssetLocatorCacheManager.retrieveFromCache(MediaAssetLocatorCacheManager.java:38) > at com.example.mediatranscoder.media.locator.ConcurrentCacheAccessTest$4.run(ConcurrentCacheAccessTest.java:278) > at com.example.mediatranscoder.media.locator.ConcurrentTestUtils$1.run(ConcurrentTestUtils.java:74) > > The code being tested is effectively a singleton cache manager which > takes a cache key and Callable block to populate a cache entry if > there isn't one already. It's fairly heavily based on the cache given > in Goetz et al [1]. The tests where I'm seeing failures are the ones > where I'm using an Executor to simulate lots of concurrent requests > trying to retrieve the same item from the cache. One of the requests > should win and populate the cache. All others should block until the > cache has been successfully populated. > > I'm fairly happy that my code is actually thread-safe and doing the > right thing (of course, I may be wrong!), so I could mark the tests as > @Ignore for now. But that's not a great place to be. I don't want > someone (including me) to make a change which breaks the working code, > etc. I think I can provide a small-ish sample test if required which > demonstrates the problem (eventually, if you have enough CPUs). > > What suggestions do you have for testing this scenario? > > Cheers, > > James > > [1] http://jcip.net/listings/Memoizer.java > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > -- http://www.natpryce.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: JMock not thread safe - how does one do multi-threaded testing?2009/4/1 Nat Pryce <nat.pryce@...>:
> JMock is thread-safe if only one thread enters the Mockery at a time. That's an interesting definition of thread-safe - only safe when one thread uses it. :-D > (E.g. it doesn't use thread-local variables or any magic like that). > So, if your code is meant to guarantee that only one thread calls a > mock object at a time, but you're seeing the Mockery fail because of > race-conditions, then your code is not thread safe. > > If you want multiple threads to enter the Mockery at once, you should > use the SynchronisingImposteriser that is currently in Subversion head > but will be in the upcoming 2.6.0 release. > > --Nat I guess that's what I'm seeing. I'll write some tests against subversion trunk HEAD and see if I can get it working with the current code, or maybe come back to you with a possible new feature request. Thanks for the fast responses; this isn't blocking me at all; it's a minor annoyance that I'd like to scratch. Cheers, James > > 2009/4/1 James Abley <james.abley@...>: >> Hi, >> >> The current policy of not supporting multi-threaded tests has just >> bitten me. We've got a spanking new CI server with multiple cores and >> are now getting build failures; I think due to JMock not being >> thread-safe; specifically the Mockery class which is declared as an >> instance of my test fixture and used throughout each @Test method. >> >> java.lang.ArrayIndexOutOfBoundsException >> at java.lang.System.arraycopy(Native Method) >> at java.util.ArrayList.ensureCapacity(ArrayList.java:170) >> at java.util.ArrayList.add(ArrayList.java:351) >> at org.jmock.Mockery.dispatch(Mockery.java:219) >> at org.jmock.Mockery.access$000(Mockery.java:43) >> at org.jmock.Mockery$MockObject.invoke(Mockery.java:258) >> at org.jmock.internal.InvocationDiverter.invoke(InvocationDiverter.java:27) >> at org.jmock.internal.FakeObjectMethods.invoke(FakeObjectMethods.java:38) >> at org.jmock.lib.JavaReflectionImposteriser$1.invoke(JavaReflectionImposteriser.java:33) >> at $Proxy9.getProperty(Unknown Source) >> at com.example.mediatranscoder.MediaTranscoderRequestContext.getStringProperty(MediaTranscoderRequestContext.java:274) >> at com.example.mediatranscoder.MediaTranscoderRequestContext.getStringProperty(MediaTranscoderRequestContext.java:122) >> at com.example.mediatranscoder.MediaAssetInfo.getHost(MediaAssetInfo.java:68) >> at com.example.mediatranscoder.MediaAssetInfo.toString(MediaAssetInfo.java:99) >> at com.example.mediatranscoder.media.locator.MediaAssetLocatorCacheManager.retrieveFromCache(MediaAssetLocatorCacheManager.java:38) >> at com.example.mediatranscoder.media.locator.ConcurrentCacheAccessTest$4.run(ConcurrentCacheAccessTest.java:278) >> at com.example.mediatranscoder.media.locator.ConcurrentTestUtils$1.run(ConcurrentTestUtils.java:74) >> >> The code being tested is effectively a singleton cache manager which >> takes a cache key and Callable block to populate a cache entry if >> there isn't one already. It's fairly heavily based on the cache given >> in Goetz et al [1]. The tests where I'm seeing failures are the ones >> where I'm using an Executor to simulate lots of concurrent requests >> trying to retrieve the same item from the cache. One of the requests >> should win and populate the cache. All others should block until the >> cache has been successfully populated. >> >> I'm fairly happy that my code is actually thread-safe and doing the >> right thing (of course, I may be wrong!), so I could mark the tests as >> @Ignore for now. But that's not a great place to be. I don't want >> someone (including me) to make a change which breaks the working code, >> etc. I think I can provide a small-ish sample test if required which >> demonstrates the problem (eventually, if you have enough CPUs). >> >> What suggestions do you have for testing this scenario? >> >> Cheers, >> >> James >> >> [1] http://jcip.net/listings/Memoizer.java >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > > > -- > http://www.natpryce.com > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: JMock not thread safe - how does one do multi-threaded testing?2009/4/1 James Abley <james.abley@...>:
> 2009/4/1 Nat Pryce <nat.pryce@...>: >> JMock is thread-safe if only one thread enters the Mockery at a time. > > That's an interesting definition of thread-safe - only safe when one > thread uses it. :-D > >> (E.g. it doesn't use thread-local variables or any magic like that). >> So, if your code is meant to guarantee that only one thread calls a >> mock object at a time, but you're seeing the Mockery fail because of >> race-conditions, then your code is not thread safe. >> >> If you want multiple threads to enter the Mockery at once, you should >> use the SynchronisingImposteriser that is currently in Subversion head >> but will be in the upcoming 2.6.0 release. >> >> --Nat > > I guess that's what I'm seeing. I'll write some tests against > subversion trunk HEAD and see if I can get it working with the current > code, or maybe come back to you with a possible new feature request. > > Thanks for the fast responses; this isn't blocking me at all; it's a > minor annoyance that I'd like to scratch. > > Cheers, > > James Works perfectly, plus gives me nice failures if I try to use the non-thread-safe version. Lovely job. package org.jmock.test.unit.lib.concurrent; import static org.junit.Assert.*; import java.util.ArrayList; import java.util.List; import java.util.concurrent.Callable; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap; import java.util.concurrent.CountDownLatch; import java.util.concurrent.ExecutionException; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; import java.util.concurrent.FutureTask; import java.util.concurrent.atomic.AtomicInteger; import org.jmock.Expectations; import org.jmock.Mockery; import org.jmock.integration.junit4.JMock; import org.jmock.integration.junit4.JUnit4Mockery; import org.jmock.lib.concurrent.SynchronisingImposteriser; import org.junit.Test; import org.junit.runner.RunWith; @RunWith(JMock.class) public class ConcurrentRequestsTest { /** * JMock context object. */ private final Mockery context = new JUnit4Mockery() { { /* Need to use the SynchronisingImposteriser otherwise it will fail when use in a multi-threaded situation. */ setImposteriser(new SynchronisingImposteriser()); } }; @Test public void onlyASingleClientPopulatesTheCache() throws Exception { final Cache cache = new Cache(); int clientRequests = 100; final Integer key = Integer.valueOf(43); ExecutorService exec = Executors.newFixedThreadPool(clientRequests); final CountDownLatch startGate = new CountDownLatch(1); final CountDownLatch endGate = new CountDownLatch(clientRequests); List<Future<Integer>> results = new ArrayList<Future<Integer>>(); for (int i = 0; i < clientRequests; ++i) { final Request request = context.mock(Request.class, "request-" + i); context.checking(new Expectations() { { allowing(request).getKey(); will(returnValue(key)); allowing(request).getCallable(); will(returnValue(new Callable<Integer>() { public Integer call() throws Exception { /* * Simulate an operation taking a little while to complete. */ Thread.sleep(10); return request.getKey(); } })); } }); FutureTask<Integer> requestThreadCall = new FutureTask<Integer>(new Callable<Integer>() { public Integer call() throws Exception { startGate.await(); try { return cache.retrieveFromCache(request); } finally { endGate.countDown(); } } }); results.add(requestThreadCall); exec.execute(requestThreadCall); } startGate.countDown(); endGate.await(); for (Future<Integer> result : results) { assertEquals(key, result.get()); } assertEquals("Only a single cache miss", 1, cache.missCount.intValue()); assertEquals("everything else was a cache hit or waited for task completion", clientRequests - 1, cache.hitCount.intValue() + cache.waitingCount.intValue()); } static interface Request { Integer getKey(); Callable<Integer> getCallable(); } static class Cache { private final ConcurrentMap<Integer, Future<Integer>> cache; private final AtomicInteger hitCount; private final AtomicInteger missCount; private final AtomicInteger waitingCount; Cache() { this.cache = new ConcurrentHashMap<Integer, Future<Integer>>(); this.hitCount = new AtomicInteger(); this.missCount = new AtomicInteger(); this.waitingCount = new AtomicInteger(); } Integer retrieveFromCache(final Request request) throws InterruptedException, ExecutionException { Future<Integer> value = cache.get(request.getKey()); if (value == null) { FutureTask<Integer> thunk = new FutureTask<Integer>(request.getCallable()); value = cache.putIfAbsent(request.getKey(), thunk); if (value == null) { cacheMiss(); thunk.run(); value = thunk; } else { cacheHitOrWaitingForCompletion(); } } else { cacheHit(); } return value.get(); } private void cacheHit() { this.hitCount.incrementAndGet(); } private void cacheHitOrWaitingForCompletion() { this.waitingCount.incrementAndGet(); } private void cacheMiss() { this.missCount.incrementAndGet(); } } } Every time I think I've found a missing feature, I should just check what's in Subversion since you've always already implemented it! Thanks, James --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: JMock not thread safe - how does one do multi-threaded testing?2009/4/1 James Abley <james.abley@...>:
> 2009/4/1 Nat Pryce <nat.pryce@...>: >> JMock is thread-safe if only one thread enters the Mockery at a time. > > That's an interesting definition of thread-safe - only safe when one > thread uses it. :-D To be pedantic, I said it's safe (by default) when called from multiple threads, as long as something else is doing the synchronisation to ensure only one thread is active in the Mockery at a time. There are APIs (Swing, for example) for which even this is not the case: you are only allowed to use objects on the same thread that created them because of hidden dependencies stored in thread-local storage. Nasty! --Nat -- http://www.natpryce.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: JMock not thread safe - how does one do multi-threaded testing?2009/4/1 James Abley <james.abley@...>:
> Works perfectly, plus gives me nice failures if I try to use the > non-thread-safe version. Lovely job. Great! We've made quite an effort ensuring that failure messages are helpful when multithreaded tests fail too. If you can find room for improvement, please make suggestions. --Nat --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: JMock not thread safe - how does one do multi-threaded testing?A quick heads-up. The API has changed. It's now more convenient if you
use multiple threads and mock concrete classes. However, you'll need to change your code if you're using a build from Svn. --Nat www.natpryce.com On 1 Apr 2009, at 23:14, James Abley <james.abley@...> wrote: > 2009/4/1 James Abley <james.abley@...>: >> 2009/4/1 Nat Pryce <nat.pryce@...>: >>> JMock is thread-safe if only one thread enters the Mockery at a >>> time. >> >> That's an interesting definition of thread-safe - only safe when one >> thread uses it. :-D >> >>> (E.g. it doesn't use thread-local variables or any magic like that). >>> So, if your code is meant to guarantee that only one thread calls a >>> mock object at a time, but you're seeing the Mockery fail because of >>> race-conditions, then your code is not thread safe. >>> >>> If you want multiple threads to enter the Mockery at once, you >>> should >>> use the SynchronisingImposteriser that is currently in Subversion >>> head >>> but will be in the upcoming 2.6.0 release. >>> >>> --Nat >> >> I guess that's what I'm seeing. I'll write some tests against >> subversion trunk HEAD and see if I can get it working with the >> current >> code, or maybe come back to you with a possible new feature request. >> >> Thanks for the fast responses; this isn't blocking me at all; it's a >> minor annoyance that I'd like to scratch. >> >> Cheers, >> >> James > > Works perfectly, plus gives me nice failures if I try to use the > non-thread-safe version. Lovely job. > > package org.jmock.test.unit.lib.concurrent; > > import static org.junit.Assert.*; > > import java.util.ArrayList; > import java.util.List; > import java.util.concurrent.Callable; > import java.util.concurrent.ConcurrentHashMap; > import java.util.concurrent.ConcurrentMap; > import java.util.concurrent.CountDownLatch; > import java.util.concurrent.ExecutionException; > import java.util.concurrent.ExecutorService; > import java.util.concurrent.Executors; > import java.util.concurrent.Future; > import java.util.concurrent.FutureTask; > import java.util.concurrent.atomic.AtomicInteger; > > import org.jmock.Expectations; > import org.jmock.Mockery; > import org.jmock.integration.junit4.JMock; > import org.jmock.integration.junit4.JUnit4Mockery; > import org.jmock.lib.concurrent.SynchronisingImposteriser; > import org.junit.Test; > import org.junit.runner.RunWith; > > @RunWith(JMock.class) > public class ConcurrentRequestsTest { > > /** > * JMock context object. > */ > private final Mockery context = new JUnit4Mockery() { > { > /* Need to use the SynchronisingImposteriser otherwise it > will fail when use in a multi-threaded situation. */ > setImposteriser(new SynchronisingImposteriser()); > } > }; > > @Test > public void onlyASingleClientPopulatesTheCache() throws Exception { > final Cache cache = new Cache(); > > int clientRequests = 100; > > final Integer key = Integer.valueOf(43); > > ExecutorService exec = > Executors.newFixedThreadPool(clientRequests); > > final CountDownLatch startGate = new CountDownLatch(1); > final CountDownLatch endGate = new > CountDownLatch(clientRequests); > > List<Future<Integer>> results = new > ArrayList<Future<Integer>>(); > > for (int i = 0; i < clientRequests; ++i) { > final Request request = context.mock(Request.class, > "request-" + i); > > context.checking(new Expectations() { > { > allowing(request).getKey(); > will(returnValue(key)); > > allowing(request).getCallable(); > will(returnValue(new Callable<Integer>() { > > public Integer call() throws Exception { > > /* > * Simulate an operation taking a little > while to complete. > */ > Thread.sleep(10); > return request.getKey(); > } > })); > } > }); > > FutureTask<Integer> requestThreadCall = new > FutureTask<Integer>(new Callable<Integer>() { > > public Integer call() throws Exception { > startGate.await(); > try { > return cache.retrieveFromCache(request); > } finally { > endGate.countDown(); > } > } > }); > > results.add(requestThreadCall); > exec.execute(requestThreadCall); > } > > startGate.countDown(); > endGate.await(); > > for (Future<Integer> result : results) { > assertEquals(key, result.get()); > } > > assertEquals("Only a single cache miss", 1, > cache.missCount.intValue()); > assertEquals("everything else was a cache hit or waited for > task completion", clientRequests - 1, > cache.hitCount.intValue() + > cache.waitingCount.intValue()); > } > > static interface Request { > > Integer getKey(); > > Callable<Integer> getCallable(); > } > > static class Cache { > > private final ConcurrentMap<Integer, Future<Integer>> cache; > private final AtomicInteger hitCount; > private final AtomicInteger missCount; > private final AtomicInteger waitingCount; > > Cache() { > this.cache = new ConcurrentHashMap<Integer, > Future<Integer>>(); > this.hitCount = new AtomicInteger(); > this.missCount = new AtomicInteger(); > this.waitingCount = new AtomicInteger(); > } > > Integer retrieveFromCache(final Request request) throws > InterruptedException, ExecutionException { > Future<Integer> value = cache.get(request.getKey()); > > if (value == null) { > FutureTask<Integer> thunk = new > FutureTask<Integer>(request.getCallable()); > > value = cache.putIfAbsent(request.getKey(), thunk); > > if (value == null) { > cacheMiss(); > thunk.run(); > value = thunk; > } else { > cacheHitOrWaitingForCompletion(); > } > } else { > cacheHit(); > } > > return value.get(); > } > > private void cacheHit() { > this.hitCount.incrementAndGet(); > } > > private void cacheHitOrWaitingForCompletion() { > this.waitingCount.incrementAndGet(); > } > > private void cacheMiss() { > this.missCount.incrementAndGet(); > } > } > } > > Every time I think I've found a missing feature, I should just check > what's in Subversion since you've always already implemented it! > > Thanks, > > James > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
| Free embeddable forum powered by Nabble | Forum Help |