A Concurrent Affair

I’ve been working on a custom class loader that does the instrumentation required for the thread checker on the fly. I’ve got most of it working now, even though I ran into a bunch of weird bugs that had me confused for several hours: I was using incorrectly formatted class names, names in the format “Lsome/package/name/MyClass$InnerClass;", as they are used in descriptors and annotations for class constant pool info items that expect class names of the format "some/package/name/MyClass$InnerClass". For some reason, this ran without any “Illegal class name” ClassNotFoundExceptions when I instrumented everything offline, but when I did the instrumentation on the fly, the exception was thrown.

The problem I’m facing now concerns auto-generated compound annotations, and also only in the on-the-fly version: I’m getting InvocationTargetExceptions when the auto-generated predicate method is trying to make a call to a member predicate method that is in a private or package-private class.

java.lang.IllegalAccessError: tried to access class
sample.threadCheck.predicate.PrintPrimitive from class
auto.sample.threadCheck.predicate.AndPrintStringsPred

Here, auto.sample.threadCheck.predicate.AndPrintStringsPred is the automatically generated class with the predicate for a @Combine-style annotation. sample.threadCheck.predicate.PrintPrimitive is a package-private class that contains the predicate method for the PrintStringAnnotation, which is a member of AndPrintStringsPred.

Right now, I see two ways around this: Either I use some reflection magic and allow access to private entities, or I change the placement and naming of auto-generated classes. Allowing access to private entities, if not done carefully, might alter the behavior of the program that is running. I have to check if I can somehow enable access just when I need it and disable it afterwards. That might be the easiest way.

The other way, of course, would be to generate the predicate classes so that they do have access to the private classes they need. However, I don’t believe that’s possible in all cases. What if an auto-generated class needs to refer to two package-private classes that reside in two different packages? I think I’ll go with the former route, even though I don’t like the prospect of enabling and disabling reflection features frequently.

In general, though, the good news is that the custom class loader is very easy to use. It’s almost the same as running something using the java command: Right now, the class to use is edu.rice.cunit.CUnitRun, which can do on-the-fly instrumentation for any kind of instrumentation strategy, but I’ll probably wrap that in a dedicated class, perhaps something like edu.rice.cunit.TCRun.

Instead of writing java MyProg, right now I have to write

java edu.rice.cs.cunit.CUnitRun
     -i edu.rice.cs.cunit.instrumentors.threadCheck.CompoundThreadCheckStrategy
     MyProg

This looks quite cumbersome, but it’s actually pretty simple. CUnitRun is basically a replacement for javac, the -i argument with the following class name specifies the instrumentation strategy to run, in this case the Thread Checker strategy, and the normal javac arguments follow after that. There are a few more flags for debugging and finer-grained control, but they don’t really matter right now. With a dedicated class to run the Thread Checker instrumentation on-the-fly, the command will boil down to this:

java edu.rice.cs.cunit.TCRun MyProg

Simple enough, I think.

I’m getting to the point where my tools are stable and interesting enough to make them available, so soon I’ll have to create a website that is more suitable as a product website than my current research website. I’m planning to make the website look similar to the DrJava website, though I still have to decide where to run it. It will probably be on the CS server, but I’ll have to find a good way to point the concutest.org domain name there. I don’t like the redirection that I’m using right now; it was just what my private hoster 1&1 provided.

With a website goes a logo, so today I asked around what symbol people associated with “concurrency” or “multithreading”. Someone suggested using a “§”; he was looking for a symbol that “doubled up in some way”. Justin suggested depicting the dining philosophers: A circle with five dots on the outside, arrows going from one dot to the next, and a lambda inside.

The dots and arrows made me come up with an idea myself: Make the lambda part of a graph representing the possible scheduling choices. I’m not sure if that’s actually discernable in the logo I created, but that doesn’t matter. I think I like it enough to use it, at least for now. I also tried to make it similar to the DrScheme logo, which was probably the grandfather of PLT logos.

So, without further ado, the Concutest logo:

Again, I wasn’t feeling too well, was kept busy with other things, and therefore couldn’t work as much on my Concutest project as much as I wanted to. Earlier I had written that there were problems with arrays. This was only true for arrays of annotations; arrays of other values, i.e. primitive data, enumerations, Strings and Class constants, worked correctly before, I just had a few doubts after I observed the behavior of arrays of annotations.

Arrays of annotations were the only type of array that caused a problem because all other arrays are actually passed as arrays; arrays of annotations are not: The classes that represent annotations are only interfaces, i.e. there is no built-in way to construct these values, which makes passing an array of them more difficult. I could have created my own classes that implement these annotation interfaces, then created an array of them, then created instances of the new classes to fill the array, and then pass the array, but that seemed more difficult than sticking with my original approach, “flattening out” all all parameters.

The problem with flattening all parameters is that the predicate method for the same annotation may have to have a differing number of parameters, if the annotation contains an array somewhere and two uses of the annotation have different numbers of elements in the array. For now, I have decided that I will just create several distinct predicate methods, one for each combination of array sizes. A last problem that I didn’t see coming until very late was that I had to distinguish between arrays nested in different elements of another array, so for arrays, the index is encoded in the method name as well.

Let’s consider the following annotations, predicates and annotated methods:

class PrintPrimitive {
public static boolean checkString(Object thisO, String value) {
System.out.println("thisO = "+thisO+", value:String = "+value);
return false;
}
}

@PredicateLink(value=PrintPrimitive.class, method="checkString")
@interface PrintStringAnnotation {
String value();
}

This is the basic @PredicateLink-type annotation, containing a String as value, and its user-written predicate method.

Below are the @Combine-type annotations that combine annotations using Boolean operations: @AndPrintStrings has two arrays of @PrintStringAnnotation annotations, and all elements get combined using AND. @AndPrintStrings therefore is a @Combine-style annotation that combines @PredicateLink-style annotations.

The annotation below that, @AndCombines, is another @Combine-style annotation, but it contains two arrays of @AndPrintStrings annotation, so @AndCombines combines several @Combine-style annotations into one.

@Combine(Combine.Mode.AND)
@interface AndPrintStrings {
PrintStringAnnotation[] arr1();
PrintStringAnnotation[] arr2();
}

@Combine(Combine.Mode.AND)
@interface AndCombines {
AndPrintStrings[] arr3();
AndPrintStrings[] arr4();
}

When the @AndPrintStrings or @AndCombines annotations are used, then predicate methods must be generated. Consider the following uses:

@AndCombines(arr3={}, arr4={})
public void testAnnArrayxxxx() {}

@AndCombines(
arr3={@AndPrintStrings(
arr1={@PrintStringAnnotation("1a11001")},
arr2={@PrintStringAnnotation("11a1001")}),
@AndPrintStrings(
arr1={@PrintStringAnnotation("111a001")},
arr2={})},
arr4={@AndPrintStrings(
arr1={@PrintStringAnnotation("12a1001"),
@PrintStringAnnotation("12b1001")},
arr2={@PrintStringAnnotation("101022a"),
@PrintStringAnnotation("101022b")})})
public void testAnnArray111022() {}

All of these methods use the same annotation, @AndCombines, so they should all invoke the same predicate method. However, if we look at the data stored inside these annotations, we realize that the number of data items differs:

1. testAnnArrayxxxx contains two empty arrays
2. testAnnArrayxx00 contains an empty array and an array with one @AndPrintStrings, but that annotation contains two empty arrays.
3. testAnnArrayxx02 contains an empty array and an array with one @AndPrintStrings, and that annotation contains an empty array and an array with two @PrintStringAnnotation.
4. testAnnArray1122 contains two arrays with one @AndPrintStrings annotation each. These nested annotations contain arrays with one or two @PrintStringAnnotation, respectively.
5. testAnnArray111022 contains two arrays, the first with two and the second with one @AndPrintStrings annotation, respectively. These nested annotations contain two arrays with one @PrintStringAnnotation each; one array with one @PrintStringAnnotation and an empty array; and two arrays with two @PrintStringAnnotation each, respectively.

It is easy to see that the first two uses of the annotation contain no actual data, the third two, the fourth six, and the last seven data elements. My program now creates five different predicate methods whose names contain name-size pairs for all arrays. The following predicate methods are created for the five annotation uses above (all of them begin with public static boolean):

1. check$arr3$$$0$arr4$$$0
(java.lang.Object a)
2. check$arr3$$$0$arr4$$$1$arr4$$0$arr2$$$0$arr4$$0$arr1$$$0
(java.lang.Object a)
3. check$arr3$$$0$arr4$$$1$arr4$$0$arr2$$$2$arr4$$0$arr1$$$0
(java.lang.Object a, java.lang.String b, java.lang.String c)
4. check$arr3$$$1$arr3$$0$arr2$$$1$arr3$$0$arr1$$$1
$arr4$$$1$arr4$$0$arr2$$$2$arr4$$0$arr1$$$2
(java.lang.Object a, java.lang.String b, java.lang.String c,
java.lang.String d, java.lang.String e, java.lang.String f,
java.lang.String g)
5. check$arr3$$$2$arr3$$0$arr2$$$0$arr3$$0$arr1$$$1
$arr3$$1$arr2$$$1$arr3$$1$arr1$$$1$arr4$$$1$arr4$$0$arr2$$$2
$arr4$$0$arr1$$$2
(java.lang.Object a, java.lang.String b, java.lang.String c,
java.lang.String d, java.lang.String e, java.lang.String f,
java.lang.String g, java.lang.String h)

The methods begin with check, followed by name-size pairs separated by $. The name consists of the annotation members that lead to the array being described, where the individual members are again separated by $. If an annotation is an element in an array, then the index in the array is indicated after the member’s name by $$ followed by an integer. The end of the name is described by $$$, which is followed by an integer describing the size. In the third example, $arr4$$0$arr2$$$2 means that the array arr2 contained in element 0 of the arr4 array has size 2. Again, like so many times before, I have chosen the $ character because the JVM allows it in identifiers but the Java language does not; this avoids all possible clashes with regular Java identifiers.

Now my @Combine-type annotations work the way I want them to work. So what is left to do? I need to convert the tests that I have so far into unit tests; right now, they are more “eyeball tests”: I eyeball whether the results are correct. Unfortunately, writing unit tests that involve compilation, instrumentation and execution of other Java source are rather painful to write. But I’ll do it — eventually.

I also want to provide an alternative way of using the thread checker’s instrumentation strategy, allowing the user to bypass the highly flexible but somewhat complicated FileInstrumentor and FileInstrumentorLauncher.

I also have to write up a concise, comprehensive guide with many examples to demonstrate all the ways the thread checker annotations can be used. For now, I’m happy, though, and I have other things to do. I need to grade, and somehow Kooprey, my object-oriented parser generator, got infected with code rot. Dr. Wong wants to present it to his COMP 202 class tomorrow (=today, it’s 5:12 AM), so I need to figure out what’s going on there.

There are still a few unsolved problems, most of them concerning arrays of annotations, or arrays in general. Some concern arrays that have a different size than the default value, some concern arrays with zero elements, for which it is more difficult to get the element type. Mostly, I have pondered how to deal with arrays of annotations, though, but I haven’t actually implemented anything yet. I have considered passing the members of annotation array elements as individual arrays, but that creates problems too:

@PredicateLink(MyPredicates.class)
@interface MyAnnotation {
String value;
int i;
}

}
// auto-generated predicate for MyCombined:
// boolean check(Object thisO, String[] value, int[] i);


In this example, instead of passing an array of MyAnnotation, I could pass two arrays, one array of String and one array of int. That has the advantage that I don’t have to use RUNTIME retention or create my own runtime representation for the data in MyAnnotation. This gets problematic, though, when the array elements contain arrays themselves, i.e. when there’s an array in MyAnnotation:

@PredicateLink(MyPredicates.class)
@interface MyAnnotation {
String[] value;
int i;
}

}
// auto-generated predicate for MyCombined:
// boolean check(Object thisO, String[][] value$value, int[] value$i);


This isn’t too bad yet, except that such an array of arrays is becoming rather painful to create. The $ is just there to help distinguish members from different fields. It gets worse, though, when the array is an array of annotations:

@PredicateLink(MyPredicates.class)
@interface MyAnnotation {
String[] value;
int i;
}

@Combine
@interface MyCombinedCombined {
MyCombined[] value();
MyAnnotation other();
}
// auto-generated predicate for MyCombined:
// boolean check(Object thisO,
// String[][][] value$value$value, int[][] value$value$i,
// String[][] value$other$value, int[] value$other$i,
// String[] other$value, int other$i);


Do I really want to do this? Here it also becomes more apparent why the $ is needed, but that’s not the problem, I handle that already. I don’t think separating arrays this way is such a good idea. Another thing I could do would be to automatically generate a storage data structure for an annotation used in an array, something like a struct in C++, i.e. for MyAnnotation I would generate a MyAnnotationStruct that just has two public fields, String[] value and int i. Then I could more easily pass the data as array, the way it was intended.

At this point, however, I’m starting to wonder if maybe I should just use RUNTIME retention instead. I should really write a version that uses RUNTIME retention and reflection to do everything. It may solve some problems more elegantly, like this one, and at least serve as a way to compare performance.

I think my first cut will be the “code bloat” approach, though: I will generate a new predicate for each new array size. To accommodate for multiple arrays in several places, I can simply enumerate the arrays along a preorder walk and include index-size pairs in the predicate method name: check$1-2$2-0$3-0 would be the appropriate predicate for an annotation whose first array has two elements and whose second and third array have zero elements.

It looks like I just finished implementing the @PredicateLink-type annotations. Values from annotations are extracted, put on the stack, and then the right predicate method is called. It returns true or false, and that value is passed to a method in the Thread Checker. If the return value was false, then a violation is logged.

I don’t have time right now to test it more because I have a 10 AM appointment and I’ve worked all through the night, but I believe I have implemented this for all possible data types allowed in @PredicateLink-type annotations, i.e. all types except for annotations and arrays of annotations.

I haven’t started with the @Combine-type annotations yet. I still have to think about where I will put the auto-generated predicate methods, or if I’m going to do the entire thing differently. I’m still pleased with my progress.

I’ve enhanced the parser of the Thread Checker to the point where it can correctly check the structure of compound predicate annotations and build a data structure that contains all the relevant information, including default values. This is a strange kind of parser, though: It parses Java class files, which are already represented in a kind of AST, and creates another AST that’s more suitable for my needs, one that omits a lot of irrelevant information, but collects pertinent data from all over the program in one place.

For now, I have decided that predicate annotations that represent the combination of other predicate annotations have to be marked with the meta-annotation @Combine. As parameter to this meta-annotation, you can specify the Boolean operation, or mode, that will be used to combine the member predicate annotations. Possible modes are @Combine(Combine.Mode.OR), @Combine(Combine.Mode.AND), and @Combine(Combine.Mode.NOT); if no mode is specified, then “or” is assumed.

The behavior of @Combine(Combine.Mode.NOT) may be a bit peculiar: The Boolean operation “not” is really intended to be applied to only one operand. However, I decided not to limit the number of member predicate annotations to one when “not” is selected as mode. Instead, I decided that the result of each member predicate annotation will be inverted, and then the inverses will be combined using “and”.

The rationale for this is that the default mode of combination is “or”: a OR b OR c. De Morgan’s law tells us that NOT (a OR b OR c) is (NOT a) AND (NOT b) AND (NOT c). Therefore, if more than one predicate annotation is combined using “not”, it makes sense to combined the inverses using “and”.

For simplicity’s sake, I have also decided that the two meta-annotations that designate an annotation as Thread Checker annotation, namely @PredicateLink and @Combine, are mutually exclusive. You can either define a new predicate annotation that uses a predicate method, or you can combine existing predicate annotations into compound annotations, but you cannot do both at the same time.

A result of this simplification is that I disallowed annotations and arrays of annotations in @PredicateLink-type annotations, while @Combine-type annotations may only contain other ThreadChecker annotations or arrays of ThreadChecker annotations as members.

To help me debug the parser, I have again enhanced the instrumentor that generates XML output. Thread Checker annotations that combine other annotations use a <combine> tag instead of a <predicate> tag, but the remainder of the output is identical.

To make sure that I correctly read in all the required data, including default values, I can optionally output the entire set of values in a <values> tag nested beneath the <combine> or <predicate> tags. I still don’t think these values belong in the XML file, but right now they are helpful for debugging.

I still haven’t written any code that inserts bytecode into methods to actually call the predicate methods and log violations. That will be one of the more difficult tasks I’ve faced with the thread checker. One obstacle, for example, is that I have to add all the constant pool items that are used in the annotation to the constant pool of the class where the annotation is being used; if default values are present, then some constant pool items may not be present yet.

Another thing I have to consider is how I will actually deal with compound predicate annotations. The regualr kind of predicate annotation that actually has a link to a predicate method will be relatively easy to deal with: I set up the parameters, I make the call, I check the return value. But for compound predicate annotations, there is no single predicate method, there are several. And in the most complicated case, a compound predicate annotation may be comprised of predicate annotations that are compounds themselves, so it’s not even guaranteed that the member predicate annotations have associated predicate methods.

Right now, I think that I may automatically generate these predicate methods. That way, even when a compound is comprised of other compounds, the members have an auto-generated predicate method, and I can treat them the same as regular predicate annotations that have a user-written predicate method. But where do I put these auto-generated predicate methods? What name do I give them, and how do I find them?

I think I’ll sleep over those questions.

I think I have found an acceptable, relatively easy solution for compound predicate annotations. There are two problems:

• Each element can only have one annotation of each type, so it’s impossible to prefix a method with @OnlyThreadWithName("foo") @OnlyThreadWithName("bar").
• There is no subtyping for annotations, so two annotations of different types do not have a common subtype, and I can’t specify an array of annotations that can contain annotations of different types.

Because of these problems, I at first couldn’t figure out how to produce compound annotations that combine other annotations using “and” and “or”, and perhaps “not”. I really would have liked to be able to write the following to specify that method fee() could be run either by a thread with name “foo” or by a thread with name “bar” that is also the event thread:

@Or({@OnlyThreadWithName("foo"),
@And({@OnlyEventThread,
@OnlyThreadWithName("bar")})})
void fee() { ... }

But there’s no way to do that. I was close to giving up on finding an easy solution, because it’s not really necessary. I could always just write a new annotation and write a new predicate method that just does exactly this or-and combination. But it just looked like there should be an easier way…

I think I found it. Earlier, I wondered if I had programmed handling annotations as member values in annotations correctly, and how those values would be passed to the predicate method. In my new idea, they won’t be passed to a predicate method: An annotation member value instead represents the composition of that annotation’s predicate method with whatever else is in the annotation. Whether the composition is done using “and” or “or” is decided using a meta-annotation; by default, it is “or”, because I expect that will occur more often.

That means that you still have to write new annotations, but at least you do not have to write new predicates. The above annotation can now actually be written as:

@And
@interface MyAndAnnotation {
OnlyEventThread a1()
default @OnlyEventThread;
OnlyThreadWithName a2()
default @OnlyThreadWithName("bar");
}
@interface MyCompoundAnnotation {
OnlyThreadWithName a1()
default @OnlyThreadWithName("foo");
MyAndAnnotation a2()
default @MyAndAnnotation();
}

@MyCompoundAnnotation
void fee() { ... }

It’s still a little bit of work, because the compound cannot be declared just where it is used, but I think this is still very acceptable. After the annotation has been defined once, it can be reused, and with default parameters, it becomes very short.

I’ll now add the restriction that annotation member values themselves have to be Thread Checker annotations, i.e. they have to have a @PredicateLink meta-annotation. I’ll also have to check for the @And and @Or meta-annotations. Then I’ll do normal processing of the member values, except for annotations and arrays of annotations. If there are member values left, then I will use those to make the kind of predicate method call that I am making now.

For all annotation member values and all the annotations in array members, I will call their predicate methods and combine the results together using whatever mode was selected. That also gets combined with the result of the original predicate call from normal data.

If a predicate annotation has only annotations or arrays of annotations, and no normal data, then it is not required to have a @PredicateLink, even though I need something that tells me this is an anntotation for the Thread Checker. Maybe I’ll use a meta-annotation like @Compound(CompoundMode.AND) and @Compound(CompoundMode.OR) instead and then require an annotation for the Thread Checker to either have a @PredicateLink or @Compound meta-annotation.

I’m convinced this will work, it will work because quite frankly… I fell asleep again in the middle of this sentence. By the way, here are pictures of my scribbling on the office whiteboard: 1 2 3.

Last night, just before I passed out into comatose sleep after being up all “Wednursday” (Wednesday + Thursday ;)) again, I wrote code to also export arrays and annotations, i.e. data with structure inside, as XML that reflects the structure. Before, I was just dumping it out as a string. I also tested it, but I really wasn’t sure I could trust my coding OR testing skills after being up that long, so I just revisited it, and the code seems to be good.

You may ask why I’m putting so much importance on correctly exporting these annotations to an XML format. Right now, I don’t have code that imports that XML data back, in fact, I don’t even have the code that inserts the runtime checks. So why bother with exporting?

Looking at the XML is just the easiest way for me to make sure that I am gathering the data correctly and that I’m putting it together the right way. Yeah, writing this code took some time too, but it’s a convenient and reassuring stop on the way to what I need to do, and since importing the XML will be important later, this effort isn’t even wasted.

Here’s an example of XML generated from predicate annotations:

The Java code below contains the methods and some of the annotations and predicates. It demonstrates how different kinds of data are processed:

• The method void foo() is annotated with just a marker annotation; the XML for it simply has a <predicate> tag that specifies the class of the annotation.
• The method void bar() has an annotation with a string and a boolean, but boolean has a default value, so it’s not specified in the Java code. In the XML, in addition to the <predicate> tag, there are two nested <arg> tags that hold name-type-value triples for both the string and the boolean.
• The methods void arr() and void arr2() both have annotations with an array of strings, but void arr2() uses the default value. For arrays, I’ve decided that there will still be nested <arg> tags that hold name-type-value triples, but the value is the size of the array, and the actual elements are stored in nested <element> tags.
• The methods void ann()void ann2() have an annotation that itself has an annotation as member; void ann2() again uses the default value. For annotations, the value attribute of the <arg> tag contains the class name of the annotation, and then the data of the members is placed in <arg> tags that are themselves nested in the <arg> tag of the annotation.
• The methods void enu()void enu2() are pretty simple, but they have an annotation with an enum as member, and enums print out a little funny. I guess I could split it up into a class-value pair.
• The last two methods, void arrAnn()void arrAnn2(), finally have an annotation that contains an array of annotations. The XML they generate is a result of what has been explained above, I just wanted to make sure it works correctly. At this time I also noticed that arrays with more than one dimension (like String[][]) aren’t allowed inside annotations; that was another case I was worried about.

// class with predicate methods for structured data
class Checks {
public static boolean checkArray(Object thisObject, String[] value) {
return true;
}
public static boolean checkAnnot(Object thisObject, PredicateLink value /*???*/) {
return true;
}
public static boolean checkEnum(Object thisObject, RetentionPolicy value) {
return true;
}
}

@ArrayAnnotTest
public void arrAnn2() { }
}

Of course, the annotations @ArrayTest, @AnnotTest, @EnumTest and @ArrayAnnotTest don’t have any meaning when it comes to concurrency testing. They’re just there so I can test how arrays, annotations and enums as member values get dealt with.

There’s still one thing where I don’t quite know if what I did was right: The @AnnotTest annotation has a member that is an annotation itself, PredicateLink. According to the conventions for the predicates, the predicate method Checks.checkAnnot has to accept an Object for this and one argument for each value in the annotation. But what is the correct type for this annotation? Is it PredicateLink? How do I call that method? Checks.checkAnnotation(this, @PredicateLink(Object.class)); doesn’t work, and neither does Checks.checkAnnotation(this, new PredicateLink(Object.class));, because PredicateLink is an interface.

This is something that I have to figure out when I write the code that actually calls the predicates. I also think that I may want to change the way default values are exported. After thinking about it a little, I don’t think default values should appear in the XML for members whose values aren’t specified. The XML should correspond exactly to what’s written in the source code. It’s either that, or all default values should appear, and currently they don’t get exported when an annotation is a member value.

I need to get going, COMP 311 and my office hours are ahead of me…. We promised to send out the grades on Wednesday, but not all students have been graded. On Thursday night, I finally decided to send out the grades we have and not wait for consistency as planned. So now the ones that don’t have their grades yet will probably complain to me, and the ones that did get their grades will complain about certain deductions. Yay.

During the night and this morning, I worked on implementing the new annotation syntax for predicates. I got to a point where the program reads annotations on classes and method, checks if they contain the @PredicateLink meta-annotation, loads the class file specified, looks for a static method returning a boolean with the given name, and checks that the method takes an Object as first argument, and then an argument with the correct parameter and name (if available) for each value in the annotation. I haven’t extensively tested this yet, but it’s supposed to work both with and without a LocalVariableTable: If a LocalVariableTable is present, then it uses the name and the type to match values in the annotation to parameters in the predicate method; if it’s not available, then it just uses the type and requires that the parameters are in the same order as the values in the annotation. If all that works out, it creates a data structure that contains the predicate class and method name and basically name-value pairs, still in annotation element format. This also includes default values if the value of an element is not specified.

I extended the ScanThreadCheckStrategy to also include XML nodes for the predicate annotations; the data in the annotation is output as a list of name-type-value triples, I don’t yet handle structured data correctly: They should be nested nodes to make parsing easier. Right now, they just get printed as a flat string.

I also haven’t written any code that actually inserts any bytecode into class files. I have written the annotations, predicates, and the method that actually logs the violations, though. The annotations and predicates are able to express the following invariants:

• Only executed by the event thread
• Only executed by threads whose name is s
• Only executed by threads whose name matches the regular expression r
• Only executed by threads whose group name is s
• Only executed by threads whose group name matches the regular expression r
• Only executed by the thread stored in the field specified by class (or class name) c and field name s
• Only executed by threads whose name is stored in the field specified by class (or class name) c and field name s
• Only executed by threads whose group name is stored in the field specified by class (or class name) c and field name s

The last three predicates are written using reflection. Unfortunately, I have yet to find a way to actually combine these annotations into compounds. Right now, the executing the following method m() in a thread named “foo” (or “bar”, or any thread at all, actually) will log a violation, because the invariants must be satisfied, i.e. they are combined using “and”: m() may only be run by a thread whose name is both “foo” and “bar”, which is impossible.

@OnlyThreadWithName("foo")
class Test {
@OnlyThreadWithName("bar")
public static void m() {
...
}
}

That’s still a big problem, as is the fact that I can only use an annotation once per annotated element, which is why I put one of the two @OnlyThreadWithName on the class. If only annotations allowed subtyping…

Here is a description of the syntax for the Thread Checker annotations that use predicates. The syntax was formulated to make using the annotations as easy as possible for the “user programmer”; it should just boil down to a single line, one annotation, with as few values as possible, for example:

@OnlyEventThread
public void foo() {
// only allowed to be run by event thread
}

@OnlyThreadWithName("auxThread")
public void bar() {
// only allowed to be run by thread with name "auxThread"
}

To allow this and make the system extensible, we provide a meta-annotation that signals to the instrumentor that a certain annotation on a class, method or constructor is a Thread Checker annotation. It also specifies what method should be executed as predicate. This meta-annotation is currently called @PredicateLink:

public @interface PredicateLink {
Class value();
String method() default "check";
}

The class is specified using a class’ .class constant. This makes my job much easier, since the Java compiler will completely resolve the class. The method name is optional; if it isn’t specified, we assume the name of the predicate method is “check”.

This annotation is then applied to the annotations that are actually being used to annotate classes, methods and constructors. Here are two examples, one annotation to specify that only the event thread may execute something, and another one to specify the allowed thread by name:

@PredicateLink(ThreadCheckPredicates.class)
public @interface OnlyEventThread { }

@PredicateLink(value = ThreadCheckPredicates.class, method = "checkName")
public @interface OnlyThreadWithName {
String value();
boolean regex() default false;
}

The first annotation is just a marker annotation, i.e. it does not contain any data. The @PredicateLink meta-annotation tells us that this is indeed an annotation for the Thread Checker and that the method to execute is in the ThreadCheckPredicates class. Since no method name is given, the method name “check” is assumed.

The second annotation contains two values, a string and a boolean that controls whether the string is treated as a regular expression or as a plain string. That boolean is optional and defaults to false, though, making it very easy to use if the regular expression behavior is not desired. The @PredicateLink meta-annotation links this annotation with the ThreadCheckPredicates.checkName method.

The methods that serve as predicates have several requirements:

1. They need to be static. That also means that they cannot be in an inner (non-static) class. We may be able to relax this requirement later and allow non-static method as long as there’s a zero-ary constructor or a singleton field with a particular name.
2. They need to return boolean.
3. They need to accept at least one argument for the value of this (or null if used in a static context), and one additional argument with the correct type and name for each member value of the annotation that uses the predicate method.

Here is an example of the ThreadCheckPredicates class that contains the predicate methods for the two annotations defined above:

class ThreadCheckPredicates {
public static boolean check(Object thisObject) {
return EventQueue.isDispatchThread();
}
public static boolean checkName(Object thisObject, String value, boolean regex) {
if (regex) {
return Thread.currentThread().getName().matches(value);
}
else {
return Thread.currentThread().getName().equals(value);
}
}
}

Both methods are static and return boolean. The first method, check has just one parameter of type Object since the annotation linked to it, @OnlyEventThread is just a marker annotation without data. That parameter will contain the value of this or null, depending on whether it’s used in a non-static or static context. The second method, checkName, has three parameters: The first one is for the value of this or null again; the second parameter is String value and the third is boolean regex since the annotation that links to the predicate has two member values with the corresponding types and names.

I’m not sure yet, but I think I’ll insist that the names of the method parameters match the names of the member values in the annotation. The alternative would be to say that the parameters must be listed in the same order as the member values in the annotation, and while that’s easier for me to program, it seems a lot more brittle in actual use.

Using predicate methods written in Java adds a lot of flexibility to the system. The meta-annotation that provides the link to the annotation simplifies the actual annotation used by the “user programmer” to the bare minimum. There are still a few things that could be better. For example, accessing fields could be easier. To access fields, the predicate methods have to be enclosed by the scope the fields are defined in, or we have to resort to reflection. Reflection is powerful but also slow.

Accessing local variables directly seems impossible right now. I just noticed that the compiler fails to compile an annotation interface inside an anonymous inner class, even when I’ve relaxed the requirement that predicate methods must be static. When I compile the following, I get a NullPointerException inside the compiler and an “modifier interface not allowed here” error:

public void fuzz() {
final String name = "fuzzThread";
Thread t = new Thread(new Runnable() {
class AnonymousInnerPredicate {
public boolean check(Object thisObject) {
return (Thread.currentThread().getName().equals(name));
}
}

@OnlyThreadWithNameInVariable
public void run() {
// only allowed to be run by a thread with name
// equal to string in local variable
}
}, name);
}

The current alternative to this is, of course, to use reflection: The name of the local variable is prefixed with “val$”. That way, the predicate method can be declared outside. However, because an anonymous inner class is anonymous, we can’t get a .class constant at compile time, so the lookup has to be by name, which isn’t nearly as nice and can’t be automated, because the numbering of the anonymous inner class depends on the location in the source file:

public void fuzz() {
final String name = "fuzzThread";
Thread t = new Thread(new Runnable() {
@OnlyThreadWithNameInField(fieldClassName="TCTest3\$1",
fieldName="val\$name",
name="fuzzThread")
public void run() {
// only allowed to be run by a thread with name
// equal to string in local variable
}
}, name);
}

For situations like this, it would just be nice to allow actual Java code in a string in the annotation, i.e. choose the fourth option I described:

public void fuzz() {
final String name = "fuzzThread";
Thread t = new Thread(new Runnable() {
@CodeThreadPredicate("return Thread.currentThread().getName().equals(name)")
public void run() {
// only allowed to be run by a thread with name
// equal to string in local variable
}
}, name);
}

I know this is possible. I’m pretty sure I could do it. But I also know that it’s a can of worms. So it’ll remain closed for now.

Corky has made the thread checker a core part of the project now. In our group meeting on Monday, we briefly and with little time discussed how to make them more general and easier to use. The result was something I had already considered of when I thought about how to link model and view for ONLY\_AFTER\_REALIZED: Predicates written in Java, i.e. allowing calls to Java methods that return boolean. Somehow it seems like I didn’t write this idea down, which is a shame; that’s exactly why I keep this blog.

At that time, I saw four different options:

1. Defining the semantics of ONLY_AFTER_REALIZED for non-component classes as “any thread can call this method until the event thread calls it the first time; after that, only the event thread may call the method”.
2. Linking model and view classes with a HashMap<String,Boolean> by letting the view set the value indexed by some unique name to true and defining ONLY_AFTER_REALIZED as “allow any thread as long the value indexed by the unique name is false; once it is true, only the event thread is allowed”. That’s what I described in the earlier posting, and I considered generalizing it to allow Boolean expressions.
3. Specifying a Java method by name in the annotation and calling it. This is essentially what we decided to do now.
4. Allowing arbitrary Java code as a string in the annotation, extracting, compiling and then inserting it in the right place. This seemed like overkill, though, since pretty much everything can be done by calling a method. If I could find a way to easily access the variables in the surrounding context, then this might have some benefits over just calling a method.

I have listed these in order of increasing difficulty. I would have tried the first option first, but I’m not sure it would have worked well and it isn’t very flexible, so I planned to implement the second option anyway. I decided against the last two options because of their complexity, and especially because allowing the execution of arbitrary code might have a large impact on the threading behavior. I still think that the predicates should be as simple as possible, and I don’t know if the use of reflection, for example, is a good idea, even though it would open many doors.

In our meeting, the differences between what Corky had in mind and what I proposed (after the Boolean expression option had been shot down as not powerful enough) were mostly syntactic and superficial. He wanted something that’s as easy and short as possible for the “user programmer”, while it could be a little more complicated for the “library programmer”.

I guess he has a point here: All my current annotations can be a bit lengthy, because they require a primary annotation (@OnlyRunBy or @NotRunBy) and then another annotation, @ThreadDesc, as data for the primary annotation. The big advantage of doing it this way is that annotations can specify an arbitrary number of thread descriptions because the primary annotations contain an array of @ThreadDesc. Since any annotation may only appear once in the list of annotations in front of whatever is being annotated, without this array it will be difficult to specify compound predicates that use “and” and “or” to combine simpler predicates, e.g. “allow thread with name ‘foo’ and thread with name ‘bar'”.

Unfortunately, annotation interfaces cannot extend another interface, so there’s absolutely no subtyping for annotations, and arrays of annotations must specify the type of the annotation precisely. It’s impossible to say “make this an array of any annotation” or introduce a common super-annotation that all Thread Checker annotations must extend. I’ll have to think about that and discuss it with Corky and the group.