ReLooper for DrJava?

Posted on February 27, 2010 by Mathias

I received a message from Danny Dig, the project lead of the ReLooper project. ReLooper is a refactoring tool that enables Java programmers to parallelize loops over arrays or vectors using Java’s upcoming ParallelArray framework. I assume Danny searched for all Java projects on SourceForge and contacted all “expert programmers” of those projects.

I know Corky received the message, too, and we both came to the conclusion that there
aren’t many parts suitable for this kind of optimization. It would be nice if indenting and “Find All” were the operations could be sped up.

With indenting, there is the problem that indenting one line depends on how the previous lines were indented. I’m not sure how this would be split up and parallelized. “Find all” is in theory easier to parallelize, but right now, it is tied very much into our single-document model. Changing this would be a good thing — it would allow us to cancel a “Find all” operation, too, — but it is a delicate task.

From what I understand, ReLooper transforms ASTs in Eclipse. It also seems like it needs an array to work on (at least automatically). I don’t know if it is able to cut through classes like ArrayList to the underlying array. The string representation in DefinitionsDocuments may be suitable, but then again the list structure in the reduced model is connected to it.

ReLooper evokes a similar response in me as Dave Patterson’s talk did: This may work in many programs that really crunch data, but I expect will break down when those algorithms need to be tied together and connected to a GUI.

Share
Posted in DrJava, Ramblings | Leave a comment

Print This Post Print This Post  

Short, Non-Technical Talks Are Hard

Posted on February 25, 2010 by Mathias

I just gave my talk Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction to the students of the Thesis Writing Seminar of the School of Engineering here at Rice. It went very well, but I’ve got to say that this was probably the most difficult talk I’ve had to prepare.

It is a 7-minute talk to an audience of smart engineering graduate students who generally are in all kinds of different fields. You can’t really use jargon or assume too much background knowledge and be too technical, otherwise you might lose your audience. You also can’t spend too much time explaining concepts. And at eight minutes, you get cut off.

I’ve really had to cut the number of slides down, gloss over a few things, and unfortunately also ignore some achievements that I think are really cool, like the reachability check of the join graph, which prevents “lucky” threads. Because of the extremely short time, practicing the talk pays off a lot, because your explanations just have to work. You don’t have time to start over.

The talk went very well and received unanimously positive feedback. What people liked the most was that it was virtually jargon-free, that the animations were simple and helpful, and that I was able to simplify the concepts and make them accessible to non-computer scientists without talking down to them.

Again, I finished working on the presentation only an hour before the talk, and my delivery definitely could have benefited from more time to practice. I also still haven’t figured out how to take a sip of water while giving a talk, but seven minutes fortunately were not long enough to develop a serious case of dry mouth.

Interestingly, Rui Zhang’s talk about software transactional memory was next, and my talk served as a good introduction.

Share
Posted in Concurrent Unit Testing, Graduate School | Leave a comment

Print This Post Print This Post  

Presentation: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction

Posted on February 25, 2010 by Mathias

Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction
(PowerPoint, view embedded)

Where: Rice University School of Engineering, Thesis Writing Seminar
When: February 25, 2010

A 7-minute jargon-free introduction to the problems of testing concurrent programs, and a proposal for a solution, with non-computer science engineering graduate students as intended audience.

Share
Posted in Publications | 1 Comment

Print This Post Print This Post  

1.6.0_18 Seems to Have Fixed Bug

Posted on February 18, 2010 by Mathias

Java’s latest version, JDK 1.6.0 Update 18, seems to have fixed the most frequent DrJava bug report, which Corky determined to be Sun’s problem (Oracle’s problem now). We have been filing all of the bug reports as duplicates of bug 2831821″Caret updating is broken”.

So far, there hasn’t been a single bug report describing the same symptoms that used 1.6.0_18. Here are links to SourceForge tracker searches:

That covers all 35 duplicate bug reports we have received.

It’s difficult to definitively check that a bug that only happened sporadically has indeed been fixed, and fixed by someone else without any involvement by us, but I think I’ll close the bug report.

Share
Posted in DrJava | Leave a comment

Print This Post Print This Post  

New Mint Release: r15116

Posted on February 17, 2010 by Mathias

I just created a new release of Mint and DrJava with Mint: February 17, 2010 (r15116). The release is available from the Mint implementation page:

There were some small changes in the approximation of weak separability to make the implementation match improvements in the paper more closely. It also disallows stitching in a Code as a statement, because we can’t statically know if we have to discard the result in a statement context or not.

(Re-posted from The Java Mint Blog.)

Share
Posted in Mint | Leave a comment

Print This Post Print This Post  

<clinit> in The Thread of First Use

Posted on February 16, 2010 by Mathias

Question: In which thread does <clinit> run?

Answer: is being executed in whatever thread uses the class first.

package testers;

import java.awt.EventQueue;

public class CLInit {
public static void main(String[] args) throws Exception {
new CLInit2().doSomething();
EventQueue.invokeAndWait(new Runnable() {
public void run() {
new CLInit2().doSomething();
new CLInit3().doSomething();
}
});
new CLInit3().doSomething();
}
}

class CLInit2 {
static {
System.out.println("CLInit2.: "+Thread.currentThread());
}

public void doSomething() {
System.out.println("CLInit2.doSomething");
}
}

class CLInit3 {
static {
System.out.println("CLInit3.: "+Thread.currentThread());
}

public void doSomething() {
System.out.println("CLInit3.doSomething");
}
}

produces the output

$ java testers.CLInit
CLInit2.<CLInit>: Thread[main,5,main]
CLInit2.doSomething
CLInit2.doSomething
CLInit3.<CLInit>: Thread[AWT-EventQueue-0,6,main]
CLInit3.doSomething
CLInit3.doSomething

As you can see, the class initializer for CLInit2 is run in the main thread, but the one for CLInit3 is executed in the event thread.

The implication of this is that class initializers may run in the event thread (or auxiliary threads) and therefore have to be considered when I compute the methods that are reachable in auxiliary threads or the event thread.

Share
Posted in Concurrent Unit Testing | 1 Comment

Print This Post Print This Post  

Anonymous Inner Class Surprise

Posted on February 16, 2010 by Mathias

I had no idea that the following code was allowed:

public class AICSurprise {
public static void main(String[] args) {
new Object() {
public void foo() { System.out.println("foo"); }
}.foo();
}
}

I thought foo() could not be accessed from outside the class, because the class was an anonymous extension of Object. That’s demonstrated here:

public class AICSurprise {
public static void main(String[] args) {
Object o = new Object() {
public void foo() { System.out.println("foo"); }
};
o.foo(); // error
}
}

Note that Object is the most precise type we can give for o. Somehow there are special rules for accessing an anonymous inner class directly when it is being created.

Did you know this? After working with Java for nearly ten years, I still didn’t.

Share
Posted in Ramblings | Leave a comment

Print This Post Print This Post  

Code<?> Escaped As Statement

Posted on February 16, 2010 by Mathias

A few days ago, I changed the Mint compiler to allow non-void code objects to be spliced in where statement form is required:

Code x = <| 1 |>;
Code c = <| { `x } |>;

This essentially generates the code below, which allows the 1 to be discarded:

<| { { Object temp = 1; } } |>;

Eddy pointed out that there is still a problem: Since I do a static type check on the type argument of Code<T>, the proper form cannot be determined if there is a wildcard Code<?>. We can check at run-time whether a code object is in statement form or whether it is an expression whose result needs to be discarded, but we cannot simply write an if-statement, because one of the alternatives would always be syntactically incorrect:

<| { if (c.isStat) { `c; // already in statement form } else { Object temp = `c; // expression, discard result } } |>;

If c.isStat is true, then the else-branch is syntactically incorrect; if c.isStat is false, then there is a syntax error in the then-branch. Eddy pointed out that we can handle the Code<?> case, though, because the two branches are in future-stage code. That is, we can turn

<| { `(c); } |>

into

if (c.isStat) {
<| { `(c); } |>
} else {
<| Object temp = `c; |>
}

Unfortunately, this gets more complicated if there is more than one Code<?> being escaped in. We would have 2^n cases, where n is the number of Code<?> objects being stitched in:

<| { `(c); `(d); } |>

would have to be turned into

if (c.isStat && d.isStat) {
<| { `(c); `(d); } |>
} else if (c.isStat && !d.isStat) {
<| { `(c); { Object temp = `d; } } |>
} else if (!c.isStat && d.isStat) {
<| { { Object temp = `c; } `(d); } |>
} else {
<| { Object temp = `c; } { Object temp = `d; } |>
}

I think for now I will change the compiler so that escaping a Code<?> in as a statement generates a compiler error. It seems like this is an unimportant corner case, and I don’t want to spend my time on something that is rarely beneficial. Again, this only happens for Code<?>, and only when escaping into bracket with a code block where statement form is required by Java rules.

Share
Posted in Mint | Leave a comment

Print This Post Print This Post  

<clinit> in Which Thread?

Posted on February 16, 2010 by Mathias

Question: In which thread does <clinit> run?

Answer tomorrow.

Share
Posted in Concurrent Unit Testing | Leave a comment

Print This Post Print This Post  

The Free Java Book

Posted on February 15, 2010 by Mathias

Corky just informed me that there is a talk by Daniel L. Schuster at SIGCSE 2010 about Java, games, and the Free Java Book. I’ll definitely check out the talk and the book. I’ll also examine how concurrency is handled in the ACM Java Library.

Daniel is kind enough to state the following in his book:

Dr. Java is the preferred development environment for the Free Java Book. It is an easy to use but powerful editor and development environment. It is available via download and is free. Free as in $0.00! It is available for Windows and Mac.

Thanks! I added a link to the Free Java Book to the list of DrJava user sites.

Share
Posted in Concurrent Unit Testing, DrJava | Leave a comment

Print This Post Print This Post  

New Mint Release: r15085

Posted on February 12, 2010 by Mathias

I just created a new release of Mint and DrJava with Mint: February 12, 2010 (r15085). The release is available from the Mint implementation page:

The two changes since the last release involve CSP values and escapes in bracket statements. Instead of storing CSP values in a CSP table, we now store CSP values in individual fields. This reduces the overhead by eliminating an array lookup. The serializer benchmark benefited from this, for example.

We now also allow non-void escapes in bracket statements. For example, the following code is now allowed:

Code x = <| 1 |>;
Code c = <| { `x } |>;

In the past, this caused an error because 1 was not a statement. The Mint compiler now accepts this and discards the result of the escape if it is used in statement form, but it does emit a warning.

(Re-posted from The Java Mint Blog.)

Share
Posted in Mint | 1 Comment

Print This Post Print This Post  

Mint Talk Video

Posted on February 12, 2010 by Mathias

The video recording of the Mint talk I gave on Monday is now available on vimeo:

Mint: A Multi-stage Extension of Java (Mathias Ricken) from Mathias Ricken on Vimeo.

Share
Posted in Mint, Pictures | Leave a comment

Print This Post Print This Post  

Pictures from Yesterday’s Talk

Posted on February 9, 2010 by Mathias

Here are a few pictures from yesterday’s talk about Mint.

Multi-stage Programming MSP in Java

MSP in Java 2 Staged power Function

Weak Separability Thank You

Share
Posted in Graduate School, Mint, Pictures | Leave a comment

Print This Post Print This Post  

Mint Talk in COMP 600

Posted on February 8, 2010 by Mathias

I just gave the first presentation about Mint in the COMP 600 graduate seminar at Rice. The slides for “Mint: A Multi-stage Extension of Java” are available (PowerPoint, PDF, view embedded). This weekend, I was kicking myself why I had volunteered for this talk out of turn, but it was good practice, and I’m a philanthropist and got the other grad students pizza that way.

Thanks for the opportunity to speak and for the good feedback!

Finally, the Java Mint Blog has a new URL: www.javamint.org. We currently also have some problems with the implementation URL; for now, you can use http://mint.concutest.org as a resource hub for everything Mint-related.

Share
Posted in Graduate School, Mint | 1 Comment

Print This Post Print This Post  

Presentation: Mint: A Multi-stage Extension of Java

Posted on February 8, 2010 by Mathias

Mint: A Multi-stage Extension of Java
(PowerPoint, PDF, view embedded)
Video recording available on vimeo.

Where: Rice University Computer Science Department, COMP 600 Graduate Seminar
When: February 8, 2010

Multi-stage programming (MSP) provides a safe way of generating code
at run-time. In mostly-functional languages like MetaOCaml, this has
been used to reduce the performance penalties of abstractions such as
loops, recursion or interpretation. The main advantage of MSP compared
to other techniques, such as string or LISP quotations, is that MSP
guarantees type safety for the generated code statically, at the time
the program is compiled.

Unfortunately, MSP is difficult to combine with imperative features
found in most mainstream languages like Java. The central problem is
“scope extrusion”, which may accidentally move variables outside the
scopes in which they are bound, leading to run-time errors in the
generated code. This problem can be prevented if code in escapes (or
“anti-quotes”) is “weakly separable”, i.e. the computational effects
occurring inside an escape that are visible from the outside do not
involve code.

We have formalized a type system, based on Lightweight Java, that uses
weak separability to prevent scope extrusion, and we have proved that
the type system is sound. We have also developed an implementation
called Mint to demonstrate the expressivity of the type system and the
performance benefits MSP can provide in an imperative setting. Since
our implementation extends the Java language, our work is accessible
to mainstream programmers.

This talk is based on work done in collaboration with Edwin Westbrook,
Jun Inoue, Yilong Yao, Tamer Abdelatif, and Walid Taha. A paper titled
“Mint: Java Multi-stage Programming Using Weak Separability” has been
accepted for publication in the Proceedings of the 2010 ACM SIGPLAN
Conference on Programming Language Design and Implementation (PLDI
2010).

Share
Posted in Mint, Publications | 2 Comments

Print This Post Print This Post  

Paper: Mint: Java Multi-stage Programming Using Weak Separability

Posted on January 31, 2010 by Mathias

Mint: Java Multi-stage Programming Using Weak Separability

2010 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2010)

Multi-stage programming (MSP) provides a disciplined approach to run-time code generation. In the purely functional setting, it has been shown how MSP can be used to reduce the overhead of abstractions, allowing clean, maintainable code without paying performance penalties. Unfortunately, MSP is difficult to combine with imperative features, which are prevalent in mainstream languages. The central difficulty is scope extrusion, wherein free variables can inadvertently be moved outside the scopes of their binders. This paper proposes a new approach to combining MSP with imperative features that occupies a “sweet spot” in the design space in terms of how well useful MSP applications can be expressed and how easy it is for programmers to understand. The key insight is that escapes (or “anti quotes”) must be weakly separable from the rest of the code, i.e. the computational effects occurring inside an escape that are visible outside the escape are guaranteed to not contain code. To demonstrate the feasibility of this approach, we formalize a type system based on Lightweight Java which we prove sound, and we also provide an implementation, called Mint, to validate both the expressivity of the type system and the effect of staging on the performance of Java programs.

Share
Posted in Mint, Publications | 2 Comments

Print This Post Print This Post  

Java Bug 4396719 Work-Around?

Posted on January 31, 2010 by Mathias

It seems like I can get around the garbage collector bug if I use -XX:+UseConcMarkSweepGC JVM argument.

Now I’m having more Soot problems, but they actually make sense. In addition to being cross-platform, DrJava is also “cross-JDK version”: We support JDK versions 5 and 6 and OpenJDK version 6. In the past, we also supported JDK version 4 and various versions of JSR-14 (adding generics to Java 4).

The adapter to the JDK 5 compiler is compiled with and linked against JDK 5; the adapter to the JDK 6 compiler is compiled with and linked against JDK 6. At runtime, we will never use a JDK 5 adapter with a JDK 6 compiler, and vice versa. But when I’m generating the call graph with JDK 6, Soot is now also looking at the JDK 5 adapter.

The JDK 5 com.sun.tools.javac.util.Log class had a String getText(String a, Object[] b) method, but that has disappeared in JDK 6. Therefore, Soot is not finding a method it expects.

I guess I will have to remove the cross-version capability for the purpose of the call graph creation.

Update

On Windows and Mac, using -XX:+UseConcMarkSweepGC actually seems to help. On Linux, I’m still getting a segfault.

Share
Posted in Concurrent Unit Testing, DrJava | 1 Comment

Print This Post Print This Post  

Soot Build Instructions

Posted on January 30, 2010 by Mathias

While trying to create a call graph of DrJava using Soot, I ran into some problems that were supposed to have been fixed in the Soot nightly builds already. Unfortunately, the nightly builds I could find were all many months old. So I built Soot from scratch from the Subversion repository, which wasn’t an easy task.

Here are some instructions for building Soot:

  1. Make sure you have Ant ant the JDK (5 or newer) installed.
  2. Check out and download all of these code bases in the same directory.
  3. Download Polyglot from http://www.sable.mcgill.ca/soot/soot_download.html: 
    wget http://www.sable.mcgill.ca/software/polyglotclasses-1.3.5.jar
  4. Check out Soot from https://svn.sable.mcgill.ca/soot/soot/trunk: 
    svn checkout https://svn.sable.mcgill.ca/soot/soot/trunk soot-dev
  5. Check out Jasmin from https://svn.sable.mcgill.ca/soot/jasmin/trunk: 
    svn checkout https://svn.sable.mcgill.ca/soot/jasmin/trunk jasmin-dev
  6. Check out JastAddJ from http://svn.jastadd.org/projects/trunk/JastAddJ: 
    svn co http://svn.jastadd.org/projects/trunk/JastAddJ
  7. Check out JastAddExtensions from http://svn.jastadd.org/projects/trunk/JastAddExtensions: 
    svn co http://svn.jastadd.org/projects/trunk/JastAddExtensions
  8. You should now have the following directory structure: 
    .
|-- polyglotclasses-1.3.5.jar
|-- JastAddExtensions
|   |-- ...
|   |-- SootJastAddJ
|   `-- ...
|-- JastAddJ
|   `-- ...
|-- jasmin-dev
|   |-- ...
|   |-- lib
|   |   `...
|   |-- ant.settings.template
|   |-- build.xml
|   `-- ...
`-- soot-dev
    |-- ...
    |-- lib
    |   `...
    |-- ant.settings.template
    |-- build.xml
    `-- ...
  9. Copy jasmin-dev/ant.settings.template to jasmin-dev/ant.settings: 
    cp jasmin-dev/ant.settings.template jasmin-dev/ant.settings
  10. Edit jasmin-dev/ant.settings so it contains the following lines: 
    java_cup.jar=../polyglotclasses-1.3.5.jar
jasmin.version=trunk
  11. Change into the jasmin-dev directory, build the jasminclasses-trunk.jar file, and change back out of the directory: 
    cd jasmin-dev
ant jasmin-jar
cd ..
  12. The freshly built Jasmin jar file is jasmin-dev/lib/jasminclasses-trunk.jar.
  13. Change into the JastAddExtensions/SootJastAddJ directory, regenerate the JastAdd-generated files for Soot, and change back out of the directory: 
    cd JastAddExtensions/SootJastAddJ
ant gen
cd ../..
  14. Copy soot-dev/ant.settings.template to soot-dev/ant.settings: 
    cp soot-dev/ant.settings.template soot-dev/ant.settings
  15. Edit soot-dev/ant.settings so it contains the following lines: 
    jastaddfrontend.loc=../JastAddExtensions/SootJastAddJ
polyglot.jar=../polyglotclasses-1.3.5.jar
jasmin.jar=../jasmin-dev/lib/jasminclasses-trunk.jar
  16. Change into the soot-dev directory and build the sootclasses-trunk.jar file, and change back out of the directory: 
    cd soot-dev
ant classesjar
cd ..

    Note: I received a compiler error here. I had to insert the second cast into soot-dev/src/soot/jbco/bafTransformations/FindDuplicateSequences.java to make line 282
    Unit jump = (Unit) units.getSuccOf(next.get(next.size() - 1));

    17. read
    Unit jump = (Unit) units.getSuccOf((Unit)(next.get(next.size() - 1)));

  17. The freshly built Soot jar file is soot-dev/lib/sootclasses-trunk.jar.

Have fun with fresh Soot.

Here’s a bash script that does it all (except insert the cast): build-soot.sh.zip

#!/bin/bash

# Make sure you have Ant ant the JDK (5 or newer) installed.
# Check out and download all of these code bases in the same directory.
# Download Polyglot from http://www.sable.mcgill.ca/soot/soot_download.html:
wget http://www.sable.mcgill.ca/software/polyglotclasses-1.3.5.jar

# Check out Soot from https://svn.sable.mcgill.ca/soot/soot/trunk:
svn checkout https://svn.sable.mcgill.ca/soot/soot/trunk soot-dev

# Check out Jasmin from https://svn.sable.mcgill.ca/soot/jasmin/trunk:
svn checkout https://svn.sable.mcgill.ca/soot/jasmin/trunk jasmin-dev

# Check out JastAddJ from http://svn.jastadd.org/projects/trunk/JastAddJ:
svn co http://svn.jastadd.org/projects/trunk/JastAddJ

# Check out JastAddExtensions from
# http://svn.jastadd.org/projects/trunk/JastAddExtensions:
svn co http://svn.jastadd.org/projects/trunk/JastAddExtensions

# Copy jasmin-dev/ant.settings.template to jasmin-dev/ant.settings:
# Edit jasmin-dev/ant.settings so it contains the following lines:
# java_cup.jar=../polyglotclasses-1.3.5.jar
# jasmin.version=trunk
cat jasmin-dev/ant.settings.template | sed \
-e "s/.*\(java_cup.jar=\).*/\1..\/polyglotclasses-1.3.5.jar/" \
-e "s/.*\(jasmin.version=\).*/\1trunk/" \
> jasmin-dev/ant.settings

# Change into the jasmin-dev directory, build the
# jasminclasses-trunk.jar file, and change back out of the directory:
cd jasmin-dev
ant jasmin-jar
cd ..

# The freshly built Jasmin jar file is
# jasmin-dev/lib/jasminclasses-trunk.jar. Change into the
# JastAddExtensions/SootJastAddJ directory, regenerate the
# JastAdd-generated files for Soot, and change back out of the
# directory:
cd JastAddExtensions/SootJastAddJ
ant gen
cd ../..

# Copy soot-dev/ant.settings.template to soot-dev/ant.settings:
# Edit soot-dev/ant.settings so it contains the following lines:
# jastaddfrontend.loc=../JastAddExtensions/SootJastAddJ
# polyglot.jar=../polyglotclasses-1.3.5.jar
# jasmin.jar=../jasmin-dev/lib/jasminclasses-trunk.jar
cat soot-dev/ant.settings.template | sed \
-e "s/.*\(jastaddfrontend.loc=\).*/\1..\/JastAddExtensions\/SootJastAddJ/" \
-e "s/.*\(polyglot.jar=\).*/\1..\/polyglotclasses-1.3.5.jar/" \
-e "s/.*\(jasmin.jar=\).*/\1..\/jasmin-dev\/lib\/jasminclasses-trunk.jar/" \
> soot-dev/ant.settings

# Change into the soot-dev directory and build the
# sootclasses-trunk.jar file, and change back out of the directory:
cd soot-dev
ant classesjar
cd ..

cp soot-dev/lib/sootclasses-trunk.jar .

Share
Posted in Concurrent Unit Testing | 1 Comment

Print This Post Print This Post  

A HotSpot Java Error, a Bus Error and a Segmentation Fault

Posted on January 30, 2010 by Mathias

When I run Soot to create a call graph of DrJava, I get the aforementioned HotSpot Java error on Windows with Java 1.6.0_18.

On Mac OS X 10.4 with Java 1.5.0_19, I get a “Bus error” (Java exit status 138).
On Red Hat Enterprise Linux 5 with Java 1.6.0_18, I get a “Segmentation fault” (Java exit status 139).

Fun, fun! At least it’s consistently not working. They probably all have the same cause.

Share
Posted in Concurrent Unit Testing, Ramblings | Leave a comment

Print This Post Print This Post  

Mint Paper Accepted to PLDI 2010

Posted on January 30, 2010 by Mathias

I’m happy to report that our Mint paper “Mint: Java Multi-stage Programming Using Weak Separability” has been accepted to PLDI 2010 in Toronto!

I couldn’t find the deadline for submitting the camera-ready version yet, and the reviewer and referee comments aren’t final yet, but I’ll let you know once I know more.

The only disappointing issue is that I might not be able to go to Toronto. My US student visa expires in May, which isn’t a problem as long as I don’t leave the country ;)

Share
Posted in Mint | Leave a comment

Print This Post Print This Post