I’ve just made a new release of ConcJUnit: 20100112. This release contains an important addition that I had been thinking about for a while.
ConcJUnit already detected failures and uncaught exceptions in the event thread, but since the event thread is a daemon thread started by the system, it is not part of the “join” and “lucky” checks. It is not required to join the event thread with the test’s main thread (in fact, it’s not really possible, since there is no way to terminate the event thread), and that also means that I do not check if the event thread finished processing.
That was a problem, of course. It was possible to put a Runnable on the event queue and then end the test, resulting in a success, even though the code in the Runnable may still fail.
I have now added code that checks if the event thread is running, and if that is the case, we check if the event queue is empty, as it should be. If it isn’t, then we issue an EventThreadStillProcessingError, which is a subclass of a NoJoinError.
However, an empty event queue does not guarantee that the event thread has finished processing. There way still be a Runnable executing in the event thread; there just aren’t any more events queued up after it. So we need to make sure that there isn’t an event being processed right now; I currently do that by adding a “token” Runnable to the event queue and requiring that it executes immediately.
But even if the token Runnable gets to execute right away, it isn’t guaranteed that the test’s events have all been processed. We also have to ensure that after we checked that the event queue was empty and before the token Runnable began executing, no additional Runnable objects were added to the event queue. Those additional events would have been added after the token Runnable, so inside the token Runnable we check again if the event queue is empty.
If the token Runnable executed immediately and the event queue is still empty when it executes, then we know that there aren’t anymore events that need to be processed and that could fail, and the test has completed in its entirety. We already checked that there aren’t any more regular threads around that could add new events, and the event thread has finished processing all Runnables.
When a EventThreadStillProcessingError is generated, then the event thread is still executing code belonging to a test that has just ended. I didn’t want to let the test wait for the event thread to finish, I want to allow it to move on. This is sort of a greedy process: Let’s run as many tests as we can. Waiting for the event thread, if not specified by the test code, could mean waiting forever if the event thread is deadlocked. However, if it isn’t deadlocked, and code there is still making progress, then it may also still fail after a new test has already started to execute. The event thread is shared among all tests, so I had to make sure that a failure in the event thread caused by a Runnable belonging to the previous test doesn’t fail the test currently executing.
I do this by maintaining a field containing the test thread group belonging to the current Runnable in the event thread. If the event thread is not running yet, then that field is updated directly at the beginning of each test. If the event thread is running, then we want all Runnables already in the event queue to still notify the test group of the previous test. All future Runnables, however, should use the current test’s thread group, so we update the field in an “update” Runnable which will execute when all Runnables belonging to the previous test have been processed.
Of course, if the event thread is deadlocked, then the update Runnable will never execute. This isn’t any worse, though, than JUnit already behaves. ConcJUnit will still allow tests that do not need the event thread to execute. When a test is executed that requires the event thread, for example by executing a invokeAndWait, then that test will hang. But at least there will have been a EventThreadStillProcessingError pointing out which test is responsible for it.
Another small problem is that once an EventThreadStillProcessingError has been generated, I do not want all following tests to also fail with that error. The event thread may still contain Runnables from the first test that was flawed that way, for example if the event thread is deadlocked. Therefore, once an EventThreadStillProcessingError has occurred, checking whether the event thread has finished is disabled. That means that only one EventThreadStillProcessingError will ever be generated. Later tests that also have this problem may go undetected until the first test has been fixed.
Footnotes:
