A Concurrent Affair | Jenny and Mathias' work and play. | Page 36

Poster: Practical Tools for Testing Concurrent Programs

Posted on October 6, 2006 by Mathias

Practical Tools for Testing Concurrent Programs

Where: Rice University Computer Science Department, Corporate Affiliates Meeting 2006
When: October 5, 2006

In our experience with developing production programs in Java,
unit testing has proven effective in assuring the reliability of code with a single thread of control. Unfortunately, unit testing has proved much less effective in assuring the reliability of code with multiple threads of control, often simply because the JUnit testing framework silently ignores failures in auxiliary threads. Java libraries and user programs frequently make assumptions about the threading context in which they execute, but these assumptions are rarely enforced by the actual code and typically only appear in program documentation. Since thread scheduling is non-deterministic, a unit test can succeed on one run and fail on the next, or repeatedly succeed on one platform and occasionally fail on another.

To improve test-driven development for concurrent programs, we are developing

an extension of the JUnit framework, actively supporting the developer by treating tests that could silently ignore failures in auxiliary threads as test errors;
a lightweight annotation language, which can be used to specify and check the threading properties of both existing and new code; and
a testing framework that can execute unit tests according to a specified set of recorded or generated schedules, elevating the unit testing of concurrent programs to a rigorous, deterministic process.

Posted in Concurrent Unit Testing, Publications | Leave a comment

Print This Post

Temporary URL Changes

Posted on October 5, 2006 by Mathias

I’ve temporarily redirected www.concutest.org to the Concutest page on my Rice website, and this blog is now accessible at www.concurrentaffair.org.

Posted in Concurrent Unit Testing, Uncategorized | Leave a comment

Print This Post

We’re Being Naughty…

Posted on October 5, 2006 by Mathias

I just ran the DrJava unit tests with Concutest-JUnit. There were a few failures that I don’t quite understand — they seem to have to do with RMI — but in general, things look good. But we are being naughty… “The test did not perform a join on all spawned threads.”

   [junit] Testsuite: edu.rice.cs.drjava.ui.InteractionsPaneTest
   [junit] Tests run: 14, Failures: 0, Errors: 2, Time elapsed: 7.047 sec

   [junit] ------------- Standard Error -----------------
   [junit] Thread Testing System.in
(edu.rice.cs.drjava.ui.InteractionsPaneTest$16) is still alive:
state=RUNNABLE
   [junit] Thread Timer-0 (java.util.TimerThread) is still alive: state=WAITING
   [junit]     java.lang.Object.wait(Native Method)
   [junit]     java.lang.Object.wait(Object.java:474)
   [junit]     java.util.TimerThread.mainLoop(Timer.java:483)
   [junit]     java.util.TimerThread.run(Timer.java:462)
   [junit] Thread Wait for Interactions to Exit Thread
(edu.rice.cs.util.newjvm.AbstractMasterJVM$1) is still alive:
state=RUNNABLE
   [junit]     java.lang.ProcessImpl.waitFor(Native Method)
   [junit]
edu.rice.cs.util.newjvm.AbstractMasterJVM$1.run(AbstractMasterJVM.java:197)
   [junit] ------------- ---------------- ---------------
   [junit] Testcase:
testSystemIn(edu.rice.cs.drjava.ui.InteractionsPaneTest): Caused an
ERROR
   [junit] The test did not perform a join on all spawned threads.
   [junit] junit.framework.TestCase$MultithreadedTestError: The test
did not perform a join on all spawned threads.


   [junit] Testcase:
testPromptListClearedOnReset(edu.rice.cs.drjava.ui.InteractionsPaneTest):
 Caused an ERROR
   [junit] The test did not perform a join on all spawned threads.
   [junit] junit.framework.TestCase$MultithreadedTestError: The test
did not perform a join on all spawned threads."

And many more of those…

Posted in Concurrent Unit Testing, DrJava | Leave a comment

Print This Post

Preliminary Versions of Concutest-JUnit

Posted on October 5, 2006 by Mathias

I have made preliminary versions of Concutest-JUnit, our modified version of JUnit 4.1, available on the Concutest page of my Rice website.

In this modified version, I have corrected a number of defects of JUnit. The two most important improvements are:

Automatic exception handling in auxiliary threads.
Requiring the main thread to wait until all threads it has spawned have finished.

While I haven’t extensively tested the modified version, it is designed as a drop-in replacement of JUnit 4.1. If you are using JUnit 4.1, simply replace the original junit.jar file with the concutest-junit.jar file.

Posted in Concurrent Unit Testing | Leave a comment

Print This Post

Corporate Affiliates Meeting 2006 Poster

Posted on October 2, 2006 by Mathias

I just finished creating a new poster for the 2006 Corporate Affiliates Meeting of my department. It is available for download on the Concutest page of my Rice website (direct download).

I have a lot more to finish up, but several things are actually close to being released!

Posted in Concurrent Unit Testing, Research | Leave a comment

Print This Post

Trying to Categorize

Posted on September 28, 2006 by Mathias

I’m way behind on my schedule, originally I was supposed to have my data sets and poster done today already, but I am making progress. I’m trying to categorize all these thread checker violations, to lump them together somehow.

So here is some raw data. When running the unit tests for DrJava, branched off revision 3993, there are 3796 occurrences of thread violations. Of these, only 1115 are unique, as determined by the complete stack trace and violation criteria. When looking at just the offending lines, the top of the stack traces, then there are only 78 places left. That’s quite a reduction.

The revision of DrJava that I’m currently working with has 128,827 lines of code. I have selected a few other revisions of DrJava, basically one every six months, starting in January 2004, but I’m not even sure I can build some of these really old releases. It may also make more sense to stick to versions that we actually released, either as beta or stable.

Besides working on the thread checker, I’ve also improved the grading script and instructions for COMP 311, the course I’m TAing this semester for the last time. It was always a bit tricky to see who was working with whom, who had not submitted yet, and who had used exactly how many slip days, so all of that is automated now, too. I’ve been up for nearly 24 hours now, so I think it’s time for a beer, a shower, and a quick nap.

Posted in Concurrent Unit Testing, Uncategorized | Leave a comment

Print This Post

Generic Class May Not Extend `Throwable`

Posted on September 28, 2006 by Mathias

Another bit of Java trivia I just learned: A generic class may not implement Throwable. Interesting, huh? I guess Throwable and exceptions have a special meaning in Java and representation in the JVM, so there are probably some limitations, but I still find this irritating.

I was trying to write a wrapper exception that extended RuntimeException for the purpose of moving a checked exception out of a method that does not have a throws clause for that checked exception in a controlled fashion. This is what I tried:

public class WrappedException extends RuntimeException { private T _wrapped; public WrappedException(String message, T cause) { super(message, cause); _wrapped = cause; } public WrappedException(T cause) { super(cause); _wrapped = cause; } public T wrapped() { return _wrapped; } }

It would have allowed me to do the following:

public someMethod() { // no throws clause try { ... } catch(IOException e) { // checked exception throw new WrappedException(e); } }

public otherMethod() throws IOException { try { someMethod(); } catch(WrappedException we) { throw we.wrapped(); } }

But I guess it all makes sense… It’s erasure at work. If you look at the catch clause, you just know this can’t work in Java: How will the JVM be able to distinguish a WrappedException<IOException> from a WrappedException<ClassNotFoundException>? It just can’t, with everything erased.

Posted in Research | Leave a comment

Print This Post

javac Is Ugly But Easy

Posted on September 27, 2006 by Mathias

Even though I’d sworn I wouldn’t look at the LAPT again until I’ve worked some more on the thread checker and the poster I have to make about it, I nonetheless looked at the source code of javac, got it to compile without a hitch — as promised by James — and began to study it.

I’m in the process of tracing local variable annotations from the parser to the ClassWriter. I haven’t made my entire way through yet — in some places, javac is written in incredibly ugly ways, I think, even though they have tried to use visitors, which I’ve been able to employ with great benefits already. I can definitely say that local variable annotations are parsed in javac’s parser (so there’s no reason Sun’s apt should not have access to them), and the annotations stay within the AST at least until just before the code generation.

In the code generation, I assume they get stripped. The MethodDef class has an array of parameters, which may contain annotations and which is read out and written into the class file, but it does not have a list of local variables. At least I haven’t found one yet, even though there ought to be one for the LocalVariableTable and LocalVariableTypeTable. I’ll have to find the place where the information needs to be passed on. Then it should be a very easy task to write out the right annotations. I think once I’ve found the right place, I’ll probably just need to modify two or three files in a few places.

All of that makes me wonder even more why Sun didn’t support annotations on local variables in the first place… at all!

Posted in xajavac | Leave a comment

Print This Post

Modifying javac to do LAPT’s Job?

Posted on September 25, 2006 by Mathias

I think I may have started on the wrong end with my LAPT tool. Both Corky and James told me today that javac is written in Java, is relatively easy to build, and written in a pretty clean way, so maybe I should have created a modified version of javac instead.

However, I first have to check the licensing terms. I don’t know if I could freely distribute the tool then. So I may need to keep my tool around, just clean it up a little. James says I can distribute it in binary form for research purposes only, and in source form as long as I don’t violate Sun’s intellectual property. That probably means that whoever uses my source code (which will probably contain the source of javac) will have to agree to the Sun Community Source License (SCSL) first. But that’s ok with me.

One of the reasons I started from the wrong end was probably that I thought I could do it almost purely by working with class files. When I think of Java, I mostly think if class files now, because that’s how I mostly deal with it now, I don’t actually think of text. I figured I could just pull the line numbers out of the class file, pull the annotations out of the source, and add them to the class file. After about half a day, though, I realized that simply copying some text out of the source wasn’t enough, and I actually needed to parse the Java file. That’s where ANTLR came in, which fortunately was pretty easy to use.

Then I realized that just looking at the AST that ANTLR gives me isn’t enough to create the attribute that I wanted to add. Class names in the AST aren’t fully qualified, but class names in class files always are. I didn’t want to write my own implementation of the algorithm to determine the fully qualified class name, because I’m sure I would have made some mistake, and since I’d realized that annotations on parameters do make it in the class file and are accessible at runtime, I came up with the idea of adding dummy methods to the Java file and turn the annotated local variables into annotated parameters of that dummy method. Then I’d compile the Java file with the additional dummy methods, open the class file, and transfer the annotations to the class file of the original Java file.

Yeah… This is probably the ugliest hack I’ve ever written, and because of these unforeseen problems of parsing and needing fully qualified class names, it took way longer than I expected.

I still think it should be so much easier to do inside javac itself, so I’ll definitely look at it, now that I’ve been told it’s not that hard to work with.

Posted in xajavac | Leave a comment

Print This Post

So Where Were We?

Posted on September 25, 2006 by Mathias

I haven’t written in a while. I also haven’t really worked in a while. I did some work Friday until the early afternoon, then went home, and after a week of not being able to sleep for more than maybe two or three hours every night, the sleep came. And it stayed for a while. With it, unfortunately, came dehydration and congestion, as I didn’t take my allergy medication, and therefore, a headache.

Tonight (or rather: during this night), I was able to put the pre-finishing touches on LAPT. It works in many simple cases already. The biggest hurdle that I didn’t expect was that I had to transfer constant pool items from the annotated class file to the original class file. Since annotations on local variables usually completely get dropped, their class names, constants, classes, and annotations they contain also get dropped from the constant pool.

I should probably define the format of the LocalVariableAnnotationsTable I settled on. It’s a mixture of the LocalVariableTypeTable and the RuntimeInvisibleAnnotations attributes:

LocalVariableAnnotationsTable\_attribute { u2 attribute\_name\_index; u4 attribute\_length; u1 local\_variable\_annotations\_table\_length; { u2 start\_pc; u2 length; u2 name\_index; u2 index; u2 num\_annotations; annotation annotations[num\_annotations]; } local\_variable\_annotation\_table[local\_variable\_annotations\_table\_length]; }

So it really still is an array of arrays of annotations. I’ve just decided to separate it from the LocalVariableTable and let it contain information about the variable, like the PC range it’s valid and especially the name, itself. This is important since the LocalVariableTable might not even exist if a program is compiled without debug information.

Another thing to think about, of course, is reflection support. Right now, at least the annotations make it into the class file, but they’re not available at runtime. What I would need for that is an additional method in Constructor and Method, one that mirrors getParameterAnnotations(). In principle, this isn’t too hard to do.

Now I’m still having some problems with annotations and array initializers, and annotations containing annotations themselves, and there is a big caveat: Annotations that a programmer wants to place on a local variable also have to be appropriate for parameters. That means the following very narrow annotation definition will not work:

@Target({ ElementType.LOCAL_VARIABLE}) public @interface Bogus { ... }

To use LAPT, the programmer will have to expand it to

@Target({ ElementType.LOCAL_VARIABLE, ElementType.PARAMETER}) public @interface Bogus { ... }

I’m going to start making this tool available and known. I would even like Sun to incorporate this attribute. I just keep thinking how much easier it would have been for Sun to do this directly within the compiler. Ideally, this would end up in a JSR and then in a future version of Java. I’ll have to come up with examples that demonstrate the use of annotations on local variables, even though it should be quite obvious.

I think I’ll have to put working on this tool aside for a while. There are other things that I absolutely need to work on.

Posted in xajavac | Leave a comment

Print This Post

Appearances Can Be Deceiving

Posted on September 21, 2006 by Mathias

The title is true in at least two ways:

My LAPT tool is turning out to be more difficult to do than I thought, particularly because of class resolution, i.e. finding the fully-qualified class name for a class.

So I’m going to take another route and let javac do most of the work: I’m creating a temporary copy of the file that contains additional dummy methods. These dummy methods have annotated parameters for each local variable of the original method. As we all know, parameter annotations make it into the class file. So then I compile the temporary file, pull out the parameter annotations, change the name slightly, and that’s it.

Or so I thought: The JVM specs don’t specify the order of the local variables in the LocalVariableTable, so if I want to let the annotations be indexed by the entries in that table, I need to put them in the same order. I’ve considered just creating an AnnotatedLocalVariableTable that is a completely new format (my original intention was to reuse the RuntimeInvisibleParameterAnnotation) and is independent of the LocalVariableTable. Another benefit would be that the code can be compiled without the -g switch for debug information. Debug information is only necessary when compiling the temporary file.

Right now I’d say I’m two thirds done with this project. It has also already taken me three halves as much time ;-)

What prevented me from potentially finishing LAPT tonight was a bug in DrJava that had popped up: [ 1561534 ] Debug Stepping Over not working with keyboard input. When the debugger was active, the program would hang when reading from System.in. Outside of the debugger, everything was fine; in fact, the framed input box integrated into the Interactions Pane looked very nice.

Corky, my advisor, assigned the bug to me, and that confused the hell out of me. Bug assignments are rare here, in fact I don’t remember if he has ever done that before. That made me assume that fixing this problem was urgent, perhaps because the University of
Waterloo was an important user of DrJava, so working on the issue couldn’t wait until the regular Monday meeting when we would normally look at problem like this as a group

After running into a few dead ends because of files that still existed but weren’t in use anymore, I finally found the place where DrJava hung: It was a completion monitor, waiting for a set() call. That call never came. The strange thing was — and I didn’t notice at first — that our snazzy input box never appeared on the Interactions Pane. Without that input box, the user can’t hit Enter, and the listener will never call set(). I think additional eyes and brains at the group meeting could have helped here.

So, after spending about four hours, I realized this was not a debugger error at all. After five hours I had realized that the problem was the way the input box was inserted into the interactions document. Due to the different prompt during debugging, it was inserted in an invalid place, causing it not to show up and give the user a chance to hit enter. And after almost seven hours I’m close to checking my bug fix into the repository. And there was nothing wrong with the debugger.

To insert the component, which is represented by a plain-old string ("[component]" in this case), our own document class was bypassed and the Swing document accessed directly, probably because creating it was inconvenient at the time to create a proper style string for the input box and let our own document class know about it.

As of revision 4010, the input box will appear and stepping/resuming will work. When the program finishes, sometimes there are two "> >" symbols appearing now, so I suggest someone with better knowledge of the parts of the program that contained the bug should take another glance at the way the prompts are printed.

I also removed the two files edu/rice/cs/util/swing/InputBox.java and src/edu/rice/cs/util/swing/PopupConsole.java that are not in use anymore except as red herrings during my initial attempts to locate the bug.

This wasn’t exactly how I had planned to spend my evening (finishing on LAPT) and night (sleeping). I’ll give the latter a shot now, but my insomnia has been so bad lately, my eyes are dry.

PS: I am so f… antastically awesome.

Update

I think at the time I wrote the response to the bug report, I got carried away by my lack of sleep and confusion about why I was being assigned the bug, that I inappropriately let my emotions show. I apologize for my unprofessional attitude.

Posted in DrJava, xajavac | Leave a comment

Print This Post

Determining the Meaning of a Type Name

Posted on September 20, 2006 by Mathias

Condensed rules for determining the meaning of a type name from the Java Language Specification, 3rd edition.

Scope of a Type

The scope of a declaration is the region of the program within which the entity declared by the declaration can be referred to using a simple name (provided it is visible (Â§6.3.1)). A declaration is said to be in scope at a particular point in a program if and only if the declaration’s scope includes that point.

The scoping rules for various constructs are given in the sections that describe those constructs. For convenience, the rules are repeated here:

The scope of a type imported by a single-type-import declaration (Â§7.5.1) or a type-import-on-demand declaration (Â§7.5.2) is all the class and interface type declarations (Â§7.6) in the compilation unit in which the import declaration appears.

The scope of a member imported by a single-static-import declaration (Â§7.5.3) or a static-import-on-demand declaration (Â§7.5.4) is all the class and interface type declarations (Â§7.6) in the compilation unit in which the import declaration appears.

The scope of a top level type is all type declarations in the package in which the top level type is declared.

The scope of a declaration of a member m declared in or inherited by a class type C is the entire body of C, including any nested type declarations.

The scope of the declaration of a member m declared in or inherited by an interface type I is the entire body of I, including any nested type declarations.

I wish I had an ordering for that.

Meaning of Type Names

The meaning of a name classified as a TypeName is determined as follows.

Simple Type Names
If a type name consists of a single Identifier, then the identifier must occur in the scope of exactly one visible declaration of a type with this name, or a compile-time error occurs. The meaning of the type name is that type.

Qualified Type Names
If a type name is of the form Q.Id, then Q must be either a type name or a package name. If Id names exactly one type that is a member of the type or package denoted by Q, then the qualified type name denotes that type. If Id does not name a member type (Â§8.5, Â§9.5) within Q, or the member type named Id within Q is not accessible (Â§6.6), or Id names more than one member type within Q, then a compile-time error occurs.

The example:

package wnj.test; class Test { public static void main(String[] args) { java.util.Date date = new java.util.Date(System.currentTimeMillis()); System.out.println(date.toLocaleString()); } }

produced the following output the first time it was run:
Sun Jan 21 22:56:29 1996
In this example the name java.util.Date must denote a type, so we first use the procedure recursively to determine if java.util is an accessible type or a package, which it is, and then look to see if the type Date is accessible in this package.

All of this lookup seems more difficult than I would want it to be, particularly because of inheritance.

Posted in Research | Leave a comment

Print This Post

Local Variable Annotations

Posted on September 19, 2006 by Mathias

Ever since I found out that annotations on local variables are allowed but do not show up in class files nor at runtime, I was severely disappointed. To me, this is a big gap in an otherwise very interesting system. The thread checker annotations begin to look like a secondary type system that is put in place on top of Java’s primary type system, but I can’t apply it everywhere because I’m not aware if local variables are annotated. I was particularly shocked that not even apt, Sun’s annotations processing tool, can get me access to annotations on local variables.

Sun’s position that “there’s just no way we could put them in the class file” is absolutely bogus. Just make up another attribute, like the LocalVariableTable or LineNumberTable. Or… like a RuntimeInvisibleParameterAnnotation attribute, which can contain annotations for a method’s parameters. How are parameters and local variables so different? The RuntimeInvisibleParameterAnnotation attribute is basically a jagged array of arrays of annotations: The first dimension is the number of parameters of the method. The second dimension is the number of annotations on that parameter, which might be zero.

What I propose, and have already implemented as part of my concurrent unit testing class file framework, is to use the same format of the RuntimeInvisibleParameterAnnotation attribute for local variables: The first dimension is the number of the local variable, as specified in the LocalVariableTable. The second dimension again is the number of annotations on the local variable.

I have started working on a small processing tool, lapt, that parses Java files in conjunction with class files that have already been generated from them. First, I tried to do the parsing of the Java files myself, just with the debug information present in the class files, but I’ve realized that will not be enough; there would be many corner cases in which I could not accurately detect a method’s beginning, etc.

Fortunately, I’ve found a pretty accessible parser generator, ANTLR, and a rather good looking Java 1.5 grammar written by Michael Studman. I still need to look at Scott Wisniewski’s Updated Java 1.5 Grammar, which is supposed to be an improvement on Studman’s grammar.

Right now, my lapt can parse in a Java file and print out the Java source code of the annotations on the local variables. The challenge now is to insert them in the right format into the class file. In general, that is pretty easy, even though there are a few quirks in how the AST is generated, particularly when it comes to dotted class and member names.

What I’m currently unable to do is find the right fully-qualified class name for an annotations class. I’m making educated guesses if it’s used anywhere else in the class file, but that’s not enough. I’ll have to read up on the details of class resolution and provide a class path. I’d imagine it would go something like

inner class of the current class
inner class of any of the containing classes
any class specifically imported
any class on one of the imported paths (sort of imports intersected with class path)

The other idea that I had was to let javac do all of that. Instead of generating the annotations attributes myself byte by byte, I would create a temporary file with dummy methods that immediately follow the original methods, and those dummy methods would have the original method’s local variables as parameters. Parameters can be annotated, no problem. javac compiles the file, and I simply steal the annotations from the dummy methods, rename the attribute and attach them to the original method.

I think that could work quite easily actually, but it involves more text processing, something I always view as imprecise, especially since ANTLR somehow manage to lose line and column information, and it also involves a compile. So I think I’ll first take a stab at generating the annotations bytewise by doing the correct class lookup myself. It just has to be spelled out somewhere.

This is really only a small side project, an enabler for a full-blown thread checker, but I think many people could potentially love lapt, and it would show Sun that they realy did get it wrong. Hey, maybe I an even get an JSR out of this. I had hoped to finish this in a couple of days, but obviously it’s taking a little longer, but especilly on Monday, I was a code monster. 10 AM before the noon meeting, 2 PM after COMP 311, completely forgetting the COMP 617 seminar, at the office until midnight. At home, microwave meals, and more work until 4 AM. Out of bed at 8 AM, on campus at 10 PM…

On the DrJava front, I looked over Dan’s shoulder to find out what it takes to make a release. Yes, our 2nd update to the stable release is out: drjava-stable-20060918-1737 release. I also looked at a few bug reports, and there really seems to be something funny going on when we’re working across networked filesystems, not only APF, but also on our very own Windows labs, which use SAMBA, I believe.

One bug report was easy to close, misunderstanding of threading and synchronization, but another one was about memory usage. I added code to the About and Error Window dialogs now to display the used/free/total/maximum amounts of memory that the VM can give us, and in the process of testing that fixed another bug in command line handling.

When I now open the About dialog, look at how much memory is used, then open a project, close it and look at how much memory is used, it’s about the same. It seems like we have a major memory leak. However… I inserted a Runtime.gc() call to induce garbage collection before showing the memory statistics, and after a while, memory went down again. I guess I’m just not understanding what gets collected. I don’t think we have a reason to panic, though.

After doing the auto-grading for COMP 311 Project 1 by myself on Saturday morning, on Monday Dave, Felipe and I met for about an hour and a half and discussed grading. Felipe finished his third yesterday, I finished mine today. Our plan is to look over the completed reports one more time for better consistency and then send them out, on Friday at the latest.

Tired man, out. Four our of sleep last night… I’m just worried that I’ll wake up in the night again, can’t go back to sleep, start working, and then I’ll get tired when it’s time to go to campus.

Posted in DrJava, xajavac | Leave a comment

Print This Post

Problems

Posted on September 18, 2006 by Mathias

Lately 1&1, the hosting service that my blog runs. on has had a few problems. I got the funniest/strangest/most annoying mixture of error messages (“500”, “too many files open”, “CGI error -1”, …), so I couldn’t post exactly when I wanted to post.

I’ve had other problems too, though. For DrJava, I’ve tried to detect if it is running on a tablet PC OS to present the “Enable ‘Forcefully Quit DrJava’?” dialog just before quitting, but only on tablet PCs. I couldn’t do it; not from within Java. Of the ways presented by Microsoft, the only option available to Java was checking for the existence of inkobj.dll, and that file turned out to exist on Windows 2003 Server as well, and this method wouldn’t be Vista-proof anyway. So I’ve just given up on this issue. Now I’ve definitely put way over one man-week into the tablet PC problem *sigh*

On the thread checker front, I noticed last Monday morning — a week ago — that some methods would crash, often with a VerifyError that complained about incorrect PC values. I couldn’t explain it… That’s serious business; it’s basically one form of malformed class file. On Saturday morning, I finally figured out. It was a slow progress, though, because there was so much to do this week and I was sick from Tuesday to Thursday. My first step was to extend ClassDumper, my custom version of javap to perform a check for every method whether the PC values actually fall on instruction boundaries. In some cases, all static methods, they didn’t. That was clue number 1. When tracing through my program, I noticed that some bytecode might be inserted, assuming that further bytecode would get inserted too, but that wasn’t the case because I excluded static methods from ONLY\_AFTER\_REALIZE.

There was a second problem, though, and this was what made the diagnosis so complex. In some cases, Java complained that it expected an array or object instance on the stack. Why was it giving me that error in a few cases, but generally it was working? Well, I inserted these checks always at the very beginning of the method. In case of constructors, though, that would be before the this() or super() call, so aload_0 or this wasn’t properly initialized yet and was probably still null. Now I’m treating constructors differently and insert the this() or super() call.

On Friday, I just had to overcome my sickness to deal with an abstract submission for my upcoming poster. I wanted to make sure Corky got a chance to read my suggested abstract. Here is the submitted version:

Title: Practical Tools for Testing for Large Concurrent Programs

Authors: Professor Robert “Corky” Cartwright and Mathias Ricken

Abstract:

In our experience with developing production programs in Java, unit testing has proven effective in assuring the reliability of code with a single thread of control. Unfortunately, unit testing has proved much less effective in assuring the reliability of code with multiple threads of control, often simply because the JUnit testing framework silently ignores failures in auxiliary threads. Java libraries and user programs frequently make assumptions about the threading context in which they execute, but these assumptions are rarely enforced by the actual code and typically only appear in program documentation. Since thread scheduling is non-deterministic, a unit
test can succeed on one run and fail on the next, or repeatedly succeed on one platform and occasionally fail on another.

To improve test-driven development for concurrent programs, we are developing

an extension of the JUnit framework, actively supporting the developer by treating tests that could silently ignore failures in auxiliary threads as test errors;

a lightweight annotation language, which can be used to specify and check the threading properties of both existing and new code; and

a testing framework that can execute unit tests according to a specified set of recorded or generated schedules, elevating the unit testing of concurrent programs to a rigorous, deterministic process.

As you can see, it puts a lot more weight on practical tools that are within my reach, like an extension of JUnit that helps the developer create multithreaded tests that don’t vacuously succeed, e.g. when no default exception handler is installed to catch exceptions (unit test failures are exceptions) in auxiliary threads. My proposal goes further than that, though; I want to force the user to join with all threads that have been created, thereby guaranteeing that failures in other threads cannot be missed. In the example below, the test would likely succeed even with an exception handler installed:

public void testNoJoin() { (new Thread(new Runnable() { public void run() { /* some lengthy computation */ fail("unconditionally"); } })).start(); }

I also changed the error window in DrJava to also contain the configuration file information (basically the preference settings) in addition to the stack trace and Java properties. I’ve been happy to see that many users actually include this data, so more data can only help us. If users actually paste that information, too, now we don’t have to ask anymore “Do you have ‘automatically close comments’ enabled?”.

And finally I also had to rewrite the submission script for COMP 311, the class I TA this semester, as students were running into problems with submitting a larger (relatively small; about 50 were enough) number of files. The scripts were written in Python by James Sasitorn. I think he’s done a great job; there are some rough edges, but hey, the grading scripts do work! Where else do you see that? There were some changes to the Owlnet environment this semester that have tripped us up a few times, so I did the first round of grading myself this weekend, and the next time around I’ll show the other TAs how to do it, and then we’ll start rotating. The autograding is a great simplifier, but it’s not trivial and still took me about 5 hours to do Saturday morning.

Anyway, my TA work for COMP 311 really amuses me because it really emphasizes one mantra that I’ve had for a long time: Language doesn’t matter. Students can submit their assignments in Java, OCaml and Scheme, so I have to be relatively fluent in all of them, and the scripts were written in Python. I’ve been using and maintaining the scripts for two years now, and I never formally learned it. Does that matter? No. I see many “newbies” to computer science/software engineering ask questions like “Should I still learn Visual C++ 7 when VC++ 8 is almost out the door?” and freak out because there are so many programs they should be familiar with, and so many certification courses they could take. It doesn’t matter if you truly understand what you’re doing. </rant>

Posted in Concurrent Unit Testing, DrJava, Uncategorized | Leave a comment

Print This Post

To Do for DrJava and the Thread Checker

Posted on September 11, 2006 by Mathias

Even though I’ve implemented the lock file scheme Peter suggested, because of limited feedback from tablet users and the inconvenience non-tablet users will experience, we’ve decided not to use it. The problem is that the dialog asking to enable “forcefully quit” would appear every time after DrJava has been killed, not only when it was related to the tablet problem, and I guess we don’t want to do that.

What I’m going to do instead is try to detect the Windows tablet PC operating system, and only if it has been detected and “forcefully quit” is disabled, show a similar dialog. These changes won’t go into the updated stable we are going to release soon. It’s not critical; I put the “forcefully quit” option in already, so we just need to highlight it in the release notes and on our website.

I’ve also noted that the thread checker is broken as of this morning. I made a bunch of small commits and must have broken something. I need to figure out what happened. (Update: It seems like I introduced the problem with changelist 596).

As to giving Swing model classes the ONLY\_AFTER\_REALIZED ability, one idea that I got during our group meeting today, besides somehow tagging models and views that belong together, is to check whether model code has been accessed by the event thread yet. I don’t know if this will work, but it’s possible that Swing doesn’t access the model until the view has been realized. In that case, the semantics of ONLY\_AFTER\_REALIZED for model classes would be that it’s ok for any thread to access it until the event thread accesses it; after that, it’s event thread-only. It’s a reasonable assumption, but I need to see if the Java API will let me do that.

Posted in Concurrent Unit Testing, DrJava | Leave a comment

Print This Post

Thread Checker Log for DrJava

Posted on September 11, 2006 by Mathias

I’ve run the thread checker with the annotations inside DrJava and a very small set of annotations for Swing (I’ve namely made the java.swing.table and java.swing.tree model packages event thread-only and the table, tree and JComponent classes event thread-only after they have been realized.

Running the unit tests generates a lot of of violations, but most of them (I’m not willing to say all just yet) are caused by two things:

GUI model methods or methods that we have designated as event thread-only in our code get called during GUI setup:

Thread Violation: OnlyRunBy
Current thread 'main', id 1, group 'main' did not match
the event thread
at edu.rice.cs.drjava.model.AbstractGlobalModel.setActiveDocument
    (AbstractGlobalModel.java:4126)
at edu.rice.cs.drjava.model.AbstractGlobalModel.setActiveFirstDocument
    (AbstractGlobalModel.java:4202)
at edu.rice.cs.drjava.model.AbstractGlobalModel._init
    (AbstractGlobalModel.java:401)
at edu.rice.cs.drjava.model.AbstractGlobalModel.
    (AbstractGlobalModel.java:344)
at edu.rice.cs.drjava.model.DefaultGlobalModel.
    (DefaultGlobalModel.java:194)
at edu.rice.cs.drjava.ui.MainFrame.
    (MainFrame.java:2563)

We are calling those methods from within our unit tests and are simulating things that normally should occur on the event thread:

Thread Violation: OnlyRunBy
Current thread 'main', id 1, group 'main' did not match
the event thread
at edu.rice.cs.drjava.model.AbstractGlobalModel.setActiveDocument
    (AbstractGlobalModel.java:4126)
at edu.rice.cs.drjava.model.AbstractGlobalModel.newFile
    (AbstractGlobalModel.java:1055)
at edu.rice.cs.drjava.model.AbstractGlobalModel.closeFiles
    (AbstractGlobalModel.java:1734)
at edu.rice.cs.drjava.model.AbstractGlobalModel.closeFile
    (AbstractGlobalModel.java:1697)
at edu.rice.cs.drjava.CommandLineTest.checkFile
    (CommandLineTest.java:404)
at edu.rice.cs.drjava.CommandLineTest.testRelativePath
    (CommandLineTest.java:339)

These two reports aren’t strictly bugs. The first thing I will try to address by somehow tying model classes into the “after realized” scheme. For the second report, we should probably remodel our unit tests.

Posted in Concurrent Unit Testing, DrJava | Leave a comment

Print This Post

Realize Realized :)

Posted on September 10, 2006 by Mathias

I had to make a few changes, but at least for subclasses of java.lang.Component, there’s now an option to allow any thread before it has been realized, but only the event thread after that.

I replaced the eventThread member of the @ThreadDesc and @OnlyRunBy annotations to an enumeration OnlyRunBy.EVENT\_THREAD with the possible values NO, ONLY_AFTER\_REALIZED and ONLY, in increasingly strict order. If two of them are merged together, the strictest one will be picked, and it is a subtyping error to go from weaker to stronger, e.g. from ONLY\_AFTER\_REALIZED to ONLY, which corresponds exactly to a previous change from false to true.

Lets look at an example program:

public class ThreadCheckSample2 { public void run() { JFrame frame = new JFrame("ThreadCheckSample2") { @OnlyRunBy(@ThreadDesc(eventThread= OnlyRunBy.EVENT\_THREAD.ONLY\_AFTER\_REALIZED)) protected void processWindowEvent(WindowEvent e) { super.processWindowEvent(e); if (WindowEvent.WINDOW_CLOSING == e.getID()) { System.exit(0); } } }; JPanel panel = new JPanel(); JButton button = new JButton("Button") { @OnlyRunBy(@ThreadDesc(eventThread= OnlyRunBy.EVENT\_THREAD.ONLY\_AFTER\_REALIZED)) public void addActionListener(ActionListener l) { super.addActionListener(l); } }; button.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent e) { handleButton(); } }); panel.add(button); frame.getContentPane().add(panel); frame.pack(); frame.setVisible(true); handleButton(); button.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent e) { System.out.println("2nd handler"); } }); }


    @OnlyRunBy(@ThreadDesc(eventThread=OnlyRunBy.EVENT\_THREAD.ONLY))

    private void handleButton() {

        System.out.println("Button was pushed");

    }

public static void main(String[] args) { (new ThreadCheckSample2()).run(); } }

This generates the thread checker log:

----------------------------------------
Log opened 2006-08-10, 15:09:23 CST (GMT-5)
Thread Violation: OnlyRunBy
        Current thread 'main', id 1, group 'main' did not match
        the event thread
        at sample.threadCheck.ThreadCheckSample2.handleButton (ThreadCheckSample2.java:40)
        at sample.threadCheck.ThreadCheckSample2.run (ThreadCheckSample2.java:30)
        at sample.threadCheck.ThreadCheckSample2.main (ThreadCheckSample2.java:44)

Thread Violation: OnlyRunBy
        Current thread 'main', id 1, group 'main' did not match
        the event thread
        at sample.threadCheck.ThreadCheckSample2$2.addActionListener (ThreadCheckSample2.java:18)
        at sample.threadCheck.ThreadCheckSample2.run (ThreadCheckSample2.java:31)
        at sample.threadCheck.ThreadCheckSample2.main (ThreadCheckSample2.java:44)

The really interesting thing here is that I have annotated the addActionListener method of the button to allow only the event thread after the button has been realized. The first call to addActionListener in line 21 does not generate a violation, but the second call in line 31 does, because the call to JFrame.pack has realized the button.

This is pretty sweet already.

Update

Because determining if a component has been realized requires… a component, I have restricted the ONLY\_AFTER\_REALIZED to non-static methods in subclasses of java.awt.Component. But what should I do with static methods if the ONLY\_AFTER\_REALIZED annotation is on class-level? Right now I throw an exception, but I don’t think that’s right. But do I want to tighten the rules (ONLY) or relax them (NO)? Right now, I’m thinking I’ll print out a warning but then relax the annotation to NO.

I ran the DrJava unit tests again, and because I fixed a bug, this time I got a lot of reports of the same problem. I may want to think about removing duplicates and the ability to suppress certain reports.

Posted in Concurrent Unit Testing | Leave a comment

Print This Post

Realize

Posted on September 10, 2006 by Mathias

I talked to Corky about Sun’s new recommendation to put all GUI code on the event thread. He said he had tried that for a while with DrJava, but he ran into a lot of complications. The splash screen didn’t show up anymore, for example. We both also agreed that, regardless of what Sun says, most code isn’t written that way and is probably still ok. We want to generate as few error messages as possible, so I should probably still find a way to allow non-event threads to interact with Swing components as long as they have not been realized.

Fortunately, I’ve found there’s an easy way to tell if an AWT component has been realized: To make that happen, Component.addNotify is called. There’s even a method that should be able to let me know exactly if that has already happened: Component.isDisplayable. It should therefore be fairly easy to modify the thread checks to only complain about execution by a non-event thread if Component.isDisplayable returns true.

The question I’m trying to answer right now, though, is: Whose isDisplayable do I call? In many cases, that should be obvious: If the method is in a subclass of java.awt.Component, then it will be this.isDisplayable(), but what if it’s not a subclass? Both javax.swing.table.TableModel and javax.swing.tree.TreeModel should only be executed by the event thread, yet they are model classes and therefore not subclasses of Component.

Initially, I’ll probably treat them as always unsafe. Then there are some refinements that I can make:

Treat them as safe until any GUI component has been realized.
Treat them as safe until a GUI component of the correct class, i.e. TableModel or TreeModel has been realized.
Treat them as safe until a specific instance of a GUI component has been realized.

The first two are relatively easy to do: The thread checker will just keep flags that are initially false, and once the first addNotify of the specific class is called, the flag is set to true. The annotations can then be limited to be active only when that flag is true. This can be generalized to the notions of “until” and “after”, corresponding to the values of false and true of a flag.

Ideally, though, you’d like to tie a certain instance of a model class to a certain instance of a view component. Here’s a rough sketch:

The @OnlyRunBy annotation is essentially unchanged, except that it can also be applied to fields, local variables, and parameters. The @ThreadDesc annotation adds a when member, which is an array of @Condition annotations. @Condition annotation itself looks like this:

public @interface Condition { String until() default ""; String after() default ""; }

It allows the programmer to specify that a @ThreadDesc annotation is either active until a certain named condition has been met, or that is only active after that has happened (only one should be specified at a time). Whether all @Conditions declared in a @ThreadDesc have to apply or just one, that is up for debate.

A condition specified by a name is set to true when an annotated GUI component is realized (or when any other method is called). This requires the @ConditionTrigger annotation:

@Retention(RetentionPolicy.CLASS) @Target({ElementType.FIELD,ElementType.LOCAL_VARIABLE, ElementType.PARAMETER,ElementType.TYPE}) public @interface ConditionTrigger { String name(); String method(); }

Now we could link together both the model and the view component by using both the @Condition and the @ConditionTrigger annotations:

@ConditionTrigger(name="some-unique-name", method="addNotify()V") private JTable _table;

@OnlyRunBy(@ThreadDesc(eventThread=true, when=@Condition(after="some-unique-name"))) private MyTableModel _model;

It should be obvious that since these annotations can be applied to fields, local variables, and parameters and not only to types, methods and constructors, inserting the correct bytecode is a lot harder. In order to insert the checks in a method, I would have to know the annotations at all the usage sites.

I think at least for the @OnlyRunBy checks I should go for call-site instrumentation, i.e. inserting the checks where the call to a method is made, instead of putting all the checks in the method that is called. The @ConditionTriggers have to go into the method that is supposed to do the trigger, e.g. into the void JTable.addNotify() method in the example above. Right now, I don’t see a way around making two passes over the classes. Another problem is that due to Sun’s incompetence (sorry, I can’t say it any other way) annotations on local variables do not get put in class files, so I won’t see them.

I’ll ask Corky how general I should make this. This is a few days or weeks away anyway.

Update

I just had the idea of showing how incompetent Sun was by taking their apt (Annotation Processing Tool) and writing a small processor that reads Java source and adds a new attribute to code attributes, describing annotations on local variables. Guess what? Not even apt is able to deal with annotations on local variables. Why in the world did Sun allow annotations where they can’t be used in any way at all? Please, if someone knows, I would love to know!

Now there’s a clumsy way around that… I introduce another annotation that can be applied to constructors and methods, i.e. anywhere where locals can appear, that links the name of the local variable to the annotations it should carry:

@Retention(RetentionPolicy.CLASS) @Target({ElementType.CONSTRUCTOR,ElementType.METHOD}) @interface LocalVariable { String name(); OnlyRunBy[] onlyRunBy() default {}; NotRunBy[] notRunBy() default {}; }

Now we can do the glorious deed of indirectly attaching a @OnlyRunBy annotation to a local variable in a way that’s visible in the class file:

@LocalVariable(name="myModel", onlyRunBy=@OnlyRunBy( @ThreadDesc(eventThread=true, when= @Condition(after="some-unique-name")))) public void run() { ... MyTableModel myModel = new MyTableModel(); }

Thanks, Sun. Really pretty.

Posted in Concurrent Unit Testing, xajavac | Leave a comment

Print This Post

DrJava and the Tablets

Posted on September 7, 2006 by Mathias

I implemented an algorithm suggested by Peter Centgraf today that attempts to detect if DrJava did not close properly the last time it ran. This is still related to bug [ 1550220 ].

On startup, DrJava creates a lock file in the temporary directory (containing the user name in the file name so different users can be distinguished); if the file already existed, then there might have been a crash. DrJava now attempts to acquire a FileLock for that file; if that fails, then another instance of DrJava is currently running.

If the file existed and no other instance is running, the user is presented with a Yes/No dialog box with the text “DrJava’s lock file was not deleted the last time DrJava ran, which may have been due to
DrJava failing to exit properly. If you remember DrJava hanging after you tried to close it we suggest you enable the ‘Forcefully Quit DrJava’ option. Would you like to enable that option?”, and if the user chooses “Yes”, the option is enabled.

On closing DrJava, the lock is relesed and the file deleted by the process that originally created it. This is done in a shutdown hook to make it happen “as late as possible”, i.e. after the call to System.exit. Recall that there is no problem running code until System.exit is called, after which DrJava hangs, so calling the code to release the lock and close the file before System.exit is utterly pointless.

Peter looked at my implementation and still found a few problems:

If the user launches DrJava too soon after killing the hung instance, the lock might not be freed before the check runs. I think the user may have to dismiss the “Send error report” dialog before re-launching.

I’m not sure I can do anything about that. That seems to be Windows.

${java.io.tmpdir} is in the user profile on Windows XP, so this will not be local for a roaming profiles setup.

I don’t know of a platform-independent way to find a local temporary directory, though.

The dialog box presented to the user is singularly unhelpful, since it doesn’t describe the consequences of the choice or provide a reasonable alternative. Since the worst thing that can happen is that a few temp files won’t be cleaned up, I think it would be safe to silently enable “Forcefully quit” if it is needed. Windows users suffer from runaway temp files anyway, so DrJava won’t be making a significant impact.

The truth is: I just have no idea what using Runtime.halt instead of System.exit can do, so I would only suggest it to those users that are likely to suffer from exactly this problem. If you have to kill DrJava at any other time, the lock file will also still be there, but in that case, enabling “forcefully quit” would not be the right choice.

We’ll see what the others think of these issues.

When I had that implemented, I used the lock file for an experiment of mine to implement feature request [ 1518709 ]: I wrote the file name of the “MasterRemote” stub into the lock file. That way, a second instance of DrJava can connect to an existing instance. I implemented a -connect command line switch that tells DrJava to open the files specified on the command line in the instance of DrJava that created the lock file.

Example:

java -jar drjava-lockfile3.jar
java -jar drjava-lockfile3.jar -connect Test.java

This opens one copy of DrJava, then tells DrJava to open Test.java in the first instance. Only one copy of DrJava will be open at the end.

Unfortunately, because of the strange way file locking is done by the OS, this does not work on Windows: The second instance of DrJava cannot read the lock file. This was just an experiment. If we want to do this across all platforms, we probably have to use the RMI registry.

Posted in DrJava | Leave a comment

Print This Post

Quick Report

Posted on September 6, 2006 by Mathias

I wasn’t able to do much on Tuesday for personal reasons, and today, on Wednesday, I’ve been feeling sick, but I did write the code for the file instrumentor to unpack jars to a temporary directory. That involved more things I expected; when instrumentation strategies took a classpath as parameter, I had to replace references to jar files with references to directories, for example. Interestingly, the unpacking did not improve things at all, in fact it is between 25 to 50 percent slower. I think that may be because now I’m dealing with a lot of small files, and that is actually worse than processing the large jar. To speed things up, I guess I have to read all annotations when I open a class file, but that also means coming up with a better caching strategy. It’s not important enough to think about that now.

One idea I just had was to check out an older version of DrJava, perhaps from Summer 2004 or Spring 2005, and apply the thread checking techniques to it. I think DrJava had more concurrency problems then, and it would be interesting to see some kind of progression.

I also found out that Corky wants me to do another poster for the Corporate Affiliates meeting this year, just like during the last two years. I enjoy making posters, but it is time-consuming; on the other hand, while I still don’t have anything close to final results, I have a lot more to tell than in the past.

Posted in Concurrent Unit Testing, Research | Leave a comment

Print This Post