Simple Test for Delay Testing

Even though I haven’t implemented inserting the delays at most of the sync points, i.e. not at monitorenter, monitorexit, Object.wait, Object.notify, Object.notifyAll, and Thread.join, I have already written my first test. I only have delays at Thread.start, Thread.run, Thread.exit, but I wanted some buggy program to test the idea already.

It was actually pretty hard to come up with a simple program that was incorrectly synchronized and whose flaw would be detected by execution with random delays. Most flaws that came to my mind were race conditions without synchronization. Here, however, we assume that the program is race-free.

The program that I came up with now uses two threads, and one thread assumes that the other thread has finished its work, but this dependency is not expressed in code (using a Thread.join).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
import junit.framework.TestCase;
/**
 * A multithreaded test case exhibiting a problem with synchronization:
 * One thread is dependent on the completion of another thread's work,
 * but this is not expressed in code.
 */

public class SyncProblem2 extends TestCase {
    private Character lock = new Character('1');
    private volatile int sharedInt = 0;
    public void testUnexpressedDependency() {
        Thread worker = new Thread(new Runnable() {
            public void run() {
                System.out.println("Worker thread is running");
                lengthyProcessing("Worker thread", 50, '+');
                synchronized(lock) {
                    sharedInt = sharedInt + 1;
                    System.out.println("sharedInt == 1 == "+sharedInt);
                }
            }
        });
        worker.start();
        lengthyProcessing("Main thread", 10000, '.');
        synchronized(lock) {
            sharedInt = sharedInt - 1;
            System.out.println("sharedInt == 0 == "+sharedInt);
            assertEquals("The shared integer should be back to 0",
                         0, sharedInt);
        }
    }
    private void lengthyProcessing(String threadName, int iterations,
                                   char progressChar) {
        System.out.print(threadName+" is doing work");
        for(int i=0; i<iterations; ++i) {
            System.out.print(progressChar);
            System.out.flush();
        }
        System.out.println();
    }
}

The two threads share a variable int sharedInt that starts out at 0. Access to it is protected by the field lock, so there aren’t any race conditions (not that they’d matter here anyway).

The worker thread spends some time doing something (simulated by the lengthyProcessing method), then increments sharedInt, and finishes.

The main thread starts the worker thread, also spends some time doing something, and then decrements sharedInt. The amount of work is set up so that under normal conditions, the worker thread will have finished and the value of sharedInt will be 1. The main thread asserts that after decrementing sharedInt, its value is back at 0.

I have added two targets, run-normal and run-delay to my Ant script to run programs and test cases more easily. If I’m using the former target, the program executes as described:

mgricken@manifold ~/research/Concutest/ClassLoader
$ ant run-normal -Dtest-class-name=SyncProblem2
...
BUILD SUCCESSFUL
Total time: 1 second

With run-delay, however, delays get inserted, in particular at the beginning of the worker thread’s run method. This delays incrementing sharedInt long enough so that the main thread can decrement it first, resulting in a value of -1 and a failed assertion:

mgricken@manifold ~/research/Concutest/ClassLoader
$ ant run-delay -Dtest-class-name=SyncProblem2
...
    [junit] Testcase: testUnexpressedDependency(SyncProblem2):  FAILED
    [junit] The shared integer should be back to 0 expected:<0> but was:<-1>
    [junit] junit.framework.AssertionFailedError: The shared integer should be back to 0
            expected:<0> but was:<-1>
    [junit]     at SyncProblem2.testUnexpressedDependency(SyncProblem2.java:33)
BUILD FAILED
/Users/mgricken/Documents/Research/Concutest/ClassLoader/build.xml:708: Test SyncProblem2 failed

This is just a really simple example, though. I need more examples with more complex flaws, especially when I add delays to more places. I also need to figure out how long these delays should be on average. Unfortunately, I don’t think there’s a simple answer. I also need to think about which situations count as “concurrency” some more:

  1. several threads actually running at the same time
  2. several threads being spawned in the same test
  3. any reference to the Thread class in the code

The first one would be the most restrictive notion and accept many tests as non-concurrent, just because by chance threads didn’t overlap; the last one would be very conservative and treat threads as potentially concurrent just because they use threading code. I hope I’ll get a chance to discuss this with Corky soon.

Share

About Mathias

Software development engineer. Principal developer of DrJava. Recent Ph.D. graduate from the Department of Computer Science at Rice University.
This entry was posted in Concurrent Unit Testing. Bookmark the permalink.

Leave a Reply