A Concurrent Affair

Mint Talk at Purdue

Mathias — Thu, 04 Mar 2010 22:49:55 +0000

I’m visiting my friend Luke at Purdue after SIGCSE, and we arranged for me to give a talk to the CS department.

If you have the chance, come grill me on Monday, March 15, 2010 at 3:30 PM.

Computer Science Colloquia
Mint: A Multi-stage Extension of Java
Mr. Mathias Ricken
Rice University
LWSN 3102 A/B
Monday, March 15, 2010
3:30 p.m. – 4:30 p.m.

Complete Description:
Abstract:
Multi-stage programming (MSP) provides a safe way of generating code at run-time. In mostly-functional languages like MetaOCaml, this has been used to reduce the performance penalties of abstractions such as loops, recursion or interpretation. The main advantage of MSP compared to other techniques, such as string or LISP quotations, is that MSP guarantees type safety for the generated code statically, at the time the program is compiled.

Unfortunately, MSP is difficult to combine with imperative features found in most mainstream languages like Java. The central problem is “scope extrusion”, which may accidentally move variables outside the scopes in which they are bound, leading to run-time errors in the generated code. This problem can be prevented if code in escapes (or “anti-quotes”) is “weakly separable”, i.e. the computational effects occurring inside an escape that are visible from the outside do not involve code.

We have formalized a type system, based on Lightweight Java, that uses weak separability to prevent scope extrusion, and we have proved that the type system is sound. We have also developed an implementation called Mint to demonstrate the expressivity of the type system and the performance benefits MSP can provide in an imperative setting. Since our implementation extends the Java language, our work is accessible to mainstream programmers.

This talk is based on work will be presented at the Conference on Programming Language Design and Implementation (PLDI 2010). More information is available at http://mint.concutest.org/

Bio:
Mathias Ricken is a doctoral candidate in the Programming Languages Team at Rice University and one of the principal developers of the DrJava integrated development environment. His research interests include concurrent programming, extending the Java language, and computer science education. He is the developer of the Concutest concurrent unit testing framework and has created various experimental extensions of Java to address, for instance, programming with meta-data. Currently, Mathias is contributing to Mint, a multi-stage extension of Java that allows safe and expressive statically typed program generation and specialization in an imperative language setting.

Faculty and students wishing to speak privately with Mr. Ricken should contact Jan Vitek at jv@cs.purdue.edu.

I’m a bit relieved that the sentence “He also has a weakness for Lady Gaga videos.” was left out of the bio.

10.4 Programs, A Dying Breed

Mathias — Tue, 02 Mar 2010 17:38:45 +0000

MacOS 10.4-compatible programs really are a dying breed. First, VLC stopped supporting 10.4 in new versions (last 10.4-compatible version is release 0.9.10), then Eclipse stopped supporting 10.4, and now VirtualBox has dropped support for 10.4 as well (last 10.4-compatible version is 3.0.12). I guess it’s high time for a new MacBook (Pro), but I want to wait for the next update.

You may ask why I don’t just install MacOS 10.5 or 10.6. As I explained before, because Apple doesn’t support Java 6 on 32-bit Macs. I have a first-generation Intel MacBook (Core Duo, not Core 2 Duo), and I need Java 6.

Review: DrJava – A Simple IDE For Beginners

Mathias — Mon, 01 Mar 2010 02:52:58 +0000

I found a nice review of DrJava today, written by Julian Jaic.

DrJava is a simple and light IDE for Java development. It is a free, open source IDE primarily designed for students. But it also includes powerful features for more advanced users. It has integrated debuggers, support unit testing with Junit and tools to generate javadocs. DrJava is available for multiple platforms, including Windows, Linux, and Macintosh.

DrJava is an ongoing project at Rice University, Texas, developed and maintain by students. DrJava shares the goal of providing a preparatory training environment that decreases the fear factor experienced by beginning students. This project provides an interface that is simple, interactive, and with a focus on the language.

REPL – Read Eval Print Loop

The eminent feature of DrJava is the ‘interactions pane,’ a ‘read-eval-print loop’ (REPL) for evaluating Java expressions and statements interactively. Functional languages such as ‘Scheme’ have long used Read Eval Print Loop. That make easy program development. Users experiment with Java constructs by typing an expression or statement will compile immediately without having a complete java code. REPL makes it possible to completely avoid the difficulties of Java I/O at the introductory level by using parameters and function return values. The interface is text based, and does require using Java syntax. This design was deliberately chosen so that only a single medium is required for dealing with program development, in contrast to the combination of UML and Java source code. Interactions can easily process graphic objects.

User Interface

DrJava keep up a focus on Java and it gives two components, a definitions pane and an interactions pane in a ’single window environment’. In definitions pane, when a class is compiled, it is directly available in the interactions pane for another uses. The Read Eval Print Loop features of the interactions pane boost understandable functional programming in association with the object-oriented model. The interactions pane keeps a history for comfortably recollecting the already inserted commands. This decreases the typing needed when going for experiential evaluations.

A Lift Current Interaction to Definitions command copies an interaction into the editor. This gives a comfortable way to move experiential tests into a JUnit test to make them repeatable. Since each class method can be executed independently, the interactions pane is an impressive tool for small testing and troubleshooting. It also contains a debugger that helps adding breakpoints and explaining watches. In debug mode when execution is break up at a breakpoint, the interactions pane can be used to check or modify state variables using Java statements and expressions. Again the design approach was to maintain an easy and steady interface throughout. It’s compiler shows all syntax errors by parsing the file. Clicking an error message highlights the error in the source code.

Dr.Java’s editor supplies automatic braces /parenthesis matching, comment /quotation highlighting and indentation that is refreshed in every keypress. A complete Java class can properly formated by an ‘Indent lines’ command. The editor supports several documents but does not arrange files into projects. To organize files, it is necessary to use the operating system’s file management facilities. Another features of DrJava contain built-in support for JUnit test cases creation of javadoc and it’s preview.

DrJava’s developers suggest that the IDE is useful beyond the beginner level, and cite advanced courses at Rice University that feature projects for extending the tool. One such project produced a DrJava plugin available for the Eclipse IDE that should ease the transition to the more powerful features of a professional IDE.

IDE’s are very important tools for easing the entry of students into software development courses. Dr.Java serve easy interfaces especially designed for teaching. It encourage cumulative development and associative experimentation, which will possibly the most important feature of this IDE. DrJava provide excellent options for simple entry level software development in Java. DrJava has a cleaner and easier interface that manages a focus on the Java language.

Julian is a professional content writer on software development and now working at Telious Technologies.

Article Source: http://EzineArticles.com/?expert=Julian_Jaic

Thanks, Julian! Actually, DrJava does have a project facility.

ReLooper for DrJava?

Mathias — Sat, 27 Feb 2010 14:37:52 +0000

I received a message from Danny Dig, the project lead of the ReLooper project. ReLooper is a refactoring tool that enables Java programmers to parallelize loops over arrays or vectors using Java’s upcoming ParallelArray framework. I assume Danny searched for all Java projects on SourceForge and contacted all “expert programmers” of those projects.

I know Corky received the message, too, and we both came to the conclusion that there aren’t many parts suitable for this kind of optimization. It would be nice if indenting and “Find All” were the operations could be sped up.

With indenting, there is the problem that indenting one line depends on how the previous lines were indented. I’m not sure how this would be split up and parallelized. “Find all” is in theory easier to parallelize, but right now, it is tied very much into our single-document model. Changing this would be a good thing — it would allow us to cancel a “Find all” operation, too, — but it is a delicate task.

From what I understand, ReLooper transforms ASTs in Eclipse. It also seems like it needs an array to work on (at least automatically). I don’t know if it is able to cut through classes like ArrayList to the underlying array. The string representation in DefinitionsDocuments may be suitable, but then again the list structure in the reduced model is connected to it.

ReLooper evokes a similar response in me as Dave Patterson’s talk did: This may work in many programs that really crunch data, but I expect will break down when those algorithms need to be tied together and connected to a GUI.

Short, Non-Technical Talks Are Hard

Mathias — Fri, 26 Feb 2010 00:43:20 +0000

I just gave my talk Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction to the students of the Thesis Writing Seminar of the School of Engineering here at Rice. It went very well, but I’ve got to say that this was probably the most difficult talk I’ve had to prepare.

It is a 7-minute talk to an audience of smart engineering graduate students who generally are in all kinds of different fields. You can’t really use jargon or assume too much background knowledge and be too technical, otherwise you might lose your audience. You also can’t spend too much time explaining concepts. And at eight minutes, you get cut off.

I’ve really had to cut the number of slides down, gloss over a few things, and unfortunately also ignore some achievements that I think are really cool, like the reachability check of the join graph, which prevents “lucky” threads. Because of the extremely short time, practicing the talk pays off a lot, because your explanations just have to work. You don’t have time to start over.

The talk went very well and received unanimously positive feedback. What people liked the most was that it was virtually jargon-free, that the animations were simple and helpful, and that I was able to simplify the concepts and make them accessible to non-computer scientists without talking down to them.

Again, I finished working on the presentation only an hour before the talk, and my delivery definitely could have benefited from more time to practice. I also still haven’t figured out how to take a sip of water while giving a talk, but seven minutes fortunately were not long enough to develop a serious case of dry mouth.

Interestingly, Rui Zhang’s talk about software transactional memory was next, and my talk served as a good introduction.

Presentation: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction

Mathias — Fri, 26 Feb 2010 00:25:57 +0000

Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction
(PowerPoint, view embedded)

Where: Rice University School of Engineering, Thesis Writing Seminar
When: February 25, 2010

A 7-minute jargon-free introduction to the problems of testing concurrent programs, and a proposal for a solution, with non-computer science engineering graduate students as intended audience.

1.6.0_18 Seems to Have Fixed Bug

Mathias — Thu, 18 Feb 2010 18:11:07 +0000

Java’s latest version, JDK 1.6.0 Update 18, seems to have fixed the most frequent DrJava bug report, which Corky determined to be Sun’s problem (Oracle’s problem now). We have been filing all of the bug reports as duplicates of bug 2831821″Caret updating is broken”.

So far, there hasn’t been a single bug report describing the same symptoms that used 1.6.0_18. Here are links to SourceForge tracker searches:

“1.6.0_18″ and “Caret updating is broken” in comments: 0 items
no “1.6.0_18″ and “Caret updating is broken” in comments: 35 items
“1.6.0_17″ and “Caret updating is broken” in comments: 10 items
“1.6.0_16″ and “Caret updating is broken” in comments: 17 items
“1.6.0_15″ and “Caret updating is broken” in comments: 3 items
“1.6.0_14″ and “Caret updating is broken” in comments: 3 items
Bugs 2867539 and 2867540 use 1.6.0_16 but for some reason don’t appear in the searches above.
“Caret updating is broken” in comments: 35 items

That covers all 35 duplicate bug reports we have received.

It’s difficult to definitively check that a bug that only happened sporadically has indeed been fixed, and fixed by someone else without any involvement by us, but I think I’ll close the bug report.

New Mint Release: r15116

Mathias — Thu, 18 Feb 2010 03:44:48 +0000

I just created a new release of Mint and DrJava with Mint: February 17, 2010 (r15116). The release is available from the Mint implementation page:

There were some small changes in the approximation of weak separability to make the implementation match improvements in the paper more closely. It also disallows stitching in a Code as a statement, because we can’t statically know if we have to discard the result in a statement context or not.

(Re-posted from The Java Mint Blog.)

in The Thread of First Use

Mathias — Wed, 17 Feb 2010 05:21:16 +0000

Question: In which thread does run?

Answer: is being executed in whatever thread uses the class first.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

package testers;

import java.awt.EventQueue;

public class CLInit {
public static void main(String[] args) throws Exception {
new CLInit2().doSomething();
EventQueue.invokeAndWait(new Runnable() {
public void run() {
new CLInit2().doSomething();
new CLInit3().doSomething();
}
});
new CLInit3().doSomething();
}
}

class CLInit2 {
static {
System.out.println("CLInit2.: "+Thread.currentThread());
}

public void doSomething() {
System.out.println("CLInit2.doSomething");
}
}

class CLInit3 {
static {
System.out.println("CLInit3.: "+Thread.currentThread());
}

public void doSomething() {
System.out.println("CLInit3.doSomething");
}
}

produces the output

$ java testers.CLInit
CLInit2.: Thread[main,5,main]
CLInit2.doSomething
CLInit2.doSomething
CLInit3.: Thread[AWT-EventQueue-0,6,main]
CLInit3.doSomething
CLInit3.doSomething

As you can see, the class initializer for CLInit2 is run in the main thread, but the one for CLInit3 is executed in the event thread.

The implication of this is that class initializers may run in the event thread (or auxiliary threads) and therefore have to be considered when I compute the methods that are reachable in auxiliary threads or the event thread.

Anonymous Inner Class Surprise

Mathias — Tue, 16 Feb 2010 19:51:46 +0000

I had no idea that the following code was allowed:

1
2
3
4
5
6
7

public class AICSurprise {
public static void main(String[] args) {
new Object() {
public void foo() { System.out.println("foo"); }
}.foo();
}
}

I thought foo() could not be accessed from outside the class, because the class was an anonymous extension of Object. That’s demonstrated here:

1
2
3
4
5
6
7
8

public class AICSurprise {
public static void main(String[] args) {
Object o = new Object() {
public void foo() { System.out.println("foo"); }
};
o.foo(); // error
}
}

Note that Object is the most precise type we can give for o. Somehow there are special rules for accessing an anonymous inner class directly when it is being created.

Did you know this? After working with Java for nearly ten years, I still didn’t.

Code Escaped As Statement

Mathias — Tue, 16 Feb 2010 17:48:31 +0000

A few days ago, I changed the Mint compiler to allow non-void code objects to be spliced in where statement form is required:

1 2	Code<Integer> x = <\| 1 \|>; Code<Void> c = <\| { `x } \|>;

This essentially generates the code below, which allows the 1 to be discarded:

1	<\| { { Object temp = 1; } } \|>;

Eddy pointed out that there is still a problem: Since I do a static type check on the type argument of Code, the proper form cannot be determined if there is a wildcard Code. We can check at run-time whether a code object is in statement form or whether it is an expression whose result needs to be discarded, but we cannot simply write an if-statement, because one of the alternatives would always be syntactically incorrect:

1
2
3
4
5

<| { if (c.isStat) {
`c; // already in statement form
} else {
Object temp = `c; // expression, discard result
} } |>;

If c.isStat is true, then the else-branch is syntactically incorrect; if c.isStat is false, then there is a syntax error in the then-branch. Eddy pointed out that we can handle the Code case, though, because the two branches are in future-stage code. That is, we can turn

1	<\| { `(c); } \|>

into

1
2
3
4
5

if (c.isStat) {
<| { `(c); } |>
} else {
<| Object temp = `c; |>
}

Unfortunately, this gets more complicated if there is more than one Code being escaped in. We would have cases, where is the number of Code objects being stitched in:

1	<\| { `(c); `(d); } \|>

would have to be turned into

1
2
3
4
5
6
7
8
9

if (c.isStat && d.isStat) {
<| { `(c); `(d); } |>
} else if (c.isStat && !d.isStat) {
<| { `(c); { Object temp = `d; } } |>
} else if (!c.isStat && d.isStat) {
<| { { Object temp = `c; } `(d); } |>
} else {
<| { Object temp = `c; } { Object temp = `d; } |>
}

I think for now I will change the compiler so that escaping a Code in as a statement generates a compiler error. It seems like this is an unimportant corner case, and I don’t want to spend my time on something that is rarely beneficial. Again, this only happens for Code, and only when escaping into bracket with a code block where statement form is required by Java rules.

in Which Thread?

Mathias — Tue, 16 Feb 2010 06:58:11 +0000

Question: In which thread does run?

Answer tomorrow.

The Free Java Book

Mathias — Mon, 15 Feb 2010 18:32:05 +0000

Corky just informed me that there is a talk by Daniel L. Schuster at SIGCSE 2010 about Java, games, and the Free Java Book. I’ll definitely check out the talk and the book. I’ll also examine how concurrency is handled in the ACM Java Library.

Daniel is kind enough to state the following in his book:

Dr. Java is the preferred development environment for the Free Java Book. It is an easy to use but powerful editor and development environment. It is available via download and is free. Free as in $0.00! It is available for Windows and Mac.

Thanks! I added a link to the Free Java Book to the list of DrJava user sites.

New Mint Release: r15085

Mathias — Fri, 12 Feb 2010 15:28:12 +0000

I just created a new release of Mint and DrJava with Mint: February 12, 2010 (r15085). The release is available from the Mint implementation page:

The two changes since the last release involve CSP values and escapes in bracket statements. Instead of storing CSP values in a CSP table, we now store CSP values in individual fields. This reduces the overhead by eliminating an array lookup. The serializer benchmark benefited from this, for example.

We now also allow non-void escapes in bracket statements. For example, the following code is now allowed:

1 2	Code<Integer> x = <\| 1 \|>; Code<Void> c = <\| { `x } \|>;

In the past, this caused an error because 1 was not a statement. The Mint compiler now accepts this and discards the result of the escape if it is used in statement form, but it does emit a warning.

(Re-posted from The Java Mint Blog.)

Mint Talk Video

Mathias — Fri, 12 Feb 2010 06:29:15 +0000

The video recording of the Mint talk I gave on Monday is now available on vimeo:

Mint: A Multi-stage Extension of Java (Mathias Ricken) from Mathias Ricken on Vimeo.

Pictures from Yesterday’s Talk

Mathias — Tue, 09 Feb 2010 18:49:00 +0000

Here are a few pictures from yesterday’s talk about Mint.

Mint Talk in COMP 600

Mathias — Mon, 08 Feb 2010 20:39:06 +0000

I just gave the first presentation about Mint in the COMP 600 graduate seminar at Rice. The slides for “Mint: A Multi-stage Extension of Java” are available (PowerPoint, PDF, view embedded). This weekend, I was kicking myself why I had volunteered for this talk out of turn, but it was good practice, and I’m a philanthropist and got the other grad students pizza that way.

Thanks for the opportunity to speak and for the good feedback!

Finally, the Java Mint Blog has a new URL: www.javamint.org. We currently also have some problems with the implementation URL; for now, you can use http://mint.concutest.org as a resource hub for everything Mint-related.

Presentation: Mint: A Multi-stage Extension of Java

Mathias — Mon, 08 Feb 2010 20:27:21 +0000

Mint: A Multi-stage Extension of Java
(PowerPoint, PDF, view embedded)
Video recording available on vimeo.

Where: Rice University Computer Science Department, COMP 600 Graduate Seminar
When: February 8, 2010

Multi-stage programming (MSP) provides a safe way of generating code at run-time. In mostly-functional languages like MetaOCaml, this has been used to reduce the performance penalties of abstractions such as loops, recursion or interpretation. The main advantage of MSP compared to other techniques, such as string or LISP quotations, is that MSP guarantees type safety for the generated code statically, at the time the program is compiled.

Unfortunately, MSP is difficult to combine with imperative features found in most mainstream languages like Java. The central problem is “scope extrusion”, which may accidentally move variables outside the scopes in which they are bound, leading to run-time errors in the generated code. This problem can be prevented if code in escapes (or “anti-quotes”) is “weakly separable”, i.e. the computational effects occurring inside an escape that are visible from the outside do not involve code.

We have formalized a type system, based on Lightweight Java, that uses weak separability to prevent scope extrusion, and we have proved that the type system is sound. We have also developed an implementation called Mint to demonstrate the expressivity of the type system and the performance benefits MSP can provide in an imperative setting. Since our implementation extends the Java language, our work is accessible to mainstream programmers.

This talk is based on work done in collaboration with Edwin Westbrook, Jun Inoue, Yilong Yao, Tamer Abdelatif, and Walid Taha. A paper titled “Mint: Java Multi-stage Programming Using Weak Separability” has been accepted for publication in the Proceedings of the 2010 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2010).

Paper: Mint: Java Multi-stage Programming Using Weak Separability

Mathias — Mon, 01 Feb 2010 01:21:58 +0000

Mint: Java Multi-stage Programming Using Weak Separability

2010 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2010)

Multi-stage programming (MSP) provides a disciplined approach to run-time code generation. In the purely functional setting, it has been shown how MSP can be used to reduce the overhead of abstractions, allowing clean, maintainable code without paying performance penalties. Unfortunately, MSP is difﬁcult to combine with imperative features, which are prevalent in mainstream languages. The central difﬁculty is scope extrusion, wherein free variables can inadvertently be moved outside the scopes of their binders. This paper proposes a new approach to combining MSP with imperative features that occupies a “sweet spot” in the design space in terms of how well useful MSP applications can be expressed and how easy it is for programmers to understand. The key insight is that escapes (or “anti-quotes”) must be weakly separable from the rest of the code, i.e. the computational effects occurring inside an escape that are visible outside the escape are guaranteed to not contain code. To demonstrate the feasibility of this approach, we formalize a type system based on Lightweight Java which we prove sound, and we also provide an implementation, called Mint, to validate both the expressivity of the system and the performance gains attainable by using MSP in this setting.

Java Bug 4396719 Work-Around?

Mathias — Sun, 31 Jan 2010 17:00:54 +0000

It seems like I can get around the garbage collector bug if I use -XX:+UseConcMarkSweepGC JVM argument.

Now I’m having more Soot problems, but they actually make sense. In addition to being cross-platform, DrJava is also “cross-JDK version”: We support JDK versions 5 and 6 and OpenJDK version 6. In the past, we also supported JDK version 4 and various versions of JSR-14 (adding generics to Java 4).

The adapter to the JDK 5 compiler is compiled with and linked against JDK 5; the adapter to the JDK 6 compiler is compiled with and linked against JDK 6. At runtime, we will never use a JDK 5 adapter with a JDK 6 compiler, and vice versa. But when I’m generating the call graph with JDK 6, Soot is now also looking at the JDK 5 adapter.

The JDK 5 com.sun.tools.javac.util.Log class had a String getText(String a, Object[] b) method, but that has disappeared in JDK 6. Therefore, Soot is not finding a method it expects.

I guess I will have to remove the cross-version capability for the purpose of the call graph creation.

Update

On Windows and Mac, using -XX:+UseConcMarkSweepGC actually seems to help. On Linux, I’m still getting a segfault.