Testing Order

I’ve been working a bit more on Concutest-JUnit. Even though I thought I was done, when I ran the tests on different machines, I was still getting errors. Clearly, there were timing problems. I think I got to the root of the problem: Sometimes the main thread continued before the test thread had written the uncaught exception to a field, so success was assumed. I switched the architecture of the “old” style to wait/notify as well.

Then I noticed that in some cases, I was calling notify too early. I called it before I actually threw the exception, so the same problem described above could occur. I rearranged the code everywhere, both in the “old” style and the “new” style portions. I did some refactoring too to improve code reuse.

Now I’m running into another problem: In the “old” 3.8.2-style, where methods are prefixed with “test“, I run the tearDown method before I check if threads are still alive. This allows tearDown to end the “infinite” loop in TimeoutTest. In the “new” 4.0-style code, methods tagged with the @After annotation, the equivalents to tearDown methods, aren’t run until after the check is made if threads are still alive (or more precisely: it’s not guaranteed that they are run before), so an @After method is not always able to kill the “infinite” loop.

What’s the right behavior here? Unfortunately, I’m thinking that tearDown and @After methods are still part of the test, as they perform necessary clean-up. So I guess I’ll have to do some pretty complicated restructuring in the “new” style portion of the code. Or maybe I’ll postpone that for now and accept that there’s a difference in the way “old” and “new” tests behave.


About Mathias

Software development engineer. Principal developer of DrJava. Recent Ph.D. graduate from the Department of Computer Science at Rice University.
This entry was posted in Concurrent Unit Testing. Bookmark the permalink.

Leave a Reply