A Concurrent Affair

Getting the lexing right for the syntax highlighting will be a little
bit more work than I thought. Right now, DrJava tokenizes numbers like
3.14 separately as “3” “.” “14”, and then it highlights “3” and “14”
as numbers. That approach never was quite right, but I think it gets
even uglier with the new literals.

I’m continuing the process of giving the compiler adapters more
responsibility. The first step that I made for supporting HJ and Mint
was to put the list of keywords into the adapters. I was now going to
put the highlight determination into the adapters, but it seems like I
may have to put tokenization there as well.

Of course, that fits well with plans of eventually supporting more languages, and languages that differ more.

While adding syntax highlighting for underscores and binary literals introduced in Java 7 to DrJava, I discovered that a few other numbers aren’t syntax-highlighted correctly. Some of these problems go back to the beginning of Java, others go back to Java 5, when the obscure hexadecimal floating-point literal was introduced.

Here are the numbers that give DrJava’s syntax highlighter trouble:

[cc lang=”java”]float f = 6.022137e+23f;
f = 2.f;
f = .3f;

double d = 2.;
d = .3;
d = 1e-9d;

double x = 0x1.8p1;
x = 0x1.fffffffffffffp1023;
x = 0x1.0P-1074;
x = 0x0.0000000000001P-1022;[/cc]

Did you know about these hexadecimal floating-point literals? I didn’t.

I created a bug report for these Java 5 highlighting bugs.

References:

• Underscores in Numeric Literals
• Binary Literals
• Hexadecimal Floating Point Literals
• Java 5 Literals

I just looked through the new language features in Java 7, and experimented with them in DrJava.

None of the changes are supported in the Interactions Pane, and there will be a lot of required work to make these things work there. We’ll have to change the parser, change the type checker, change the interpreter, and so on.

There are, however, smaller changes that we should do to provide better support in the Definitions Pane:

1. Binary literals [cci lang=”java”]int i = 0b100000;[/cci] are not syntax-highlighted correctly.
2. Numeric literals with an underscore in them [cci lang=”java]int i = 1_000; double d = 3.141_5927;[/cci] are not syntax-highlighted correctly.
3. In some cases, try-with-resources is indented badly:
   [cc lang=”java”] // badly indented
   public static void tryWithResources2(String f) {
   try (InputStream is = new FileInputStream(new File(f));
   InputStream is2 = new FileInputStream(new File(f))) {
   is.read();
   }
   catch(IOException e) {
   e.printStackTrace();
   }
   }[/cc]

Everything else looks pretty good. Here’s the sample class that I used:

[cc lang=”java”]import java.io.*;
import java.util.*;

public class Java7Samples {
public static void main(String[] args) {
stringSwitch(args[0]);
intSwitch(new Integer(args[1]));
tryWithResources(args[2]);
tryWithResources2(args[2]);
multiCatch(new Integer(args[3]));

// bad integer syntax highlighting
int i = 0b100000;
System.out.println(i);

// bad numeric syntax highlighting
i = 1_000_000;
System.out.println(i);
i = 2__000__000;
System.out.println(i);
// int illegal = _1000;
// int illegal2 = 1000_;
double d = 123_456.789_012;
System.out.println(d);
}

public static void stringSwitch(String arg) {
switch(arg) {
case “one”:
System.out.println(“case one”);
break;
case “two”:
case “three”:
System.out.println(“case two or three”);
break;
default:
System.out.println(“default case”);
}
}
public static void intSwitch(int arg) {
switch(arg) {
case 1:
System.out.println(“case 1”);
break;
case 2:
case 3:
System.out.println(“case 2 or 3”);
break;
default:
System.out.println(“default case”);
}
}

public static void tryWithResources(String f) {
try (InputStream is = new FileInputStream(new File(f))) {
is.read();
}
catch(IOException e) {
e.printStackTrace();
}
}

// badly indented
public static void tryWithResources2(String f) {
try (InputStream is = new FileInputStream(new File(f));
InputStream is2 = new FileInputStream(new File(f))) {
is.read();
}
catch(IOException e) {
e.printStackTrace();
}
}

public static class Ex1 extends RuntimeException { }
public static class Ex2 extends RuntimeException { }
public static class Ex3or4 extends RuntimeException { }
public static class Ex3 extends Ex3or4 { }
public static class Ex4 extends Ex3or4 { }

public static void doSomething(int i) throws Ex1, Ex2, Ex3, Ex4 {
switch(i) {
case 1:
throw new Ex1();
case 2:
throw new Ex2();
case 3:
throw new Ex3();
case 4:
throw new Ex4();
}
}

public static void handleEx1Or2(RuntimeException e) {
System.out.println(“handlEx1Or2”);
}

public static void handleEx3Or4(Ex3or4 e) {
System.out.println(“handlEx3Or4”);
}

public static void multiCatch(int i) {
try {
doSomething(i);
}
catch(Ex1 | Ex2 ex) {
handleEx1Or2(ex);
}
catch(Ex3 | Ex4 ex) {
handleEx3Or4(ex);
}
}

public static void multiCatch2(int i) {
try {
doSomething(i);
}
catch(Ex1 |
Ex2 ex) {
handleEx1Or2(ex);
}
catch(Ex3 |
Ex4 ex) {
handleEx3Or4(ex);
}
}

public static Map diamond() {
return new HashMap<>();
}

public static void diamond2() {
Map map = new HashMap<>();
}
}[/cc]

Update

I filed bugs 3404387 and 3404389.

We have made a new development release available: drjava-beta-20110822-r5448. You can download it from the DrJava website or from SourceForge.

Available for download at http://drjava.org.

DrJava is a lightweight programming environment for Java designed to foster test-driven software development. It includes an intelligent program editor, an interactions pane for evaluating program text, a source level debugger, and a unit testing tool.

In addition to bug fixes, this beta release includes a two new features introduced after the last development release:

1. The debugger has been revised to interface properly with functional language level files. The source line numbers shown in the debugger now consistently correspond to the source files open in DrJava, even when the source files are language level files. During tracing, line numbers corresponding to augmented code that does not appear in language level files are mapped to -1.
2. A compiler adapter for Java 7 has been added to DrJava. Unfortunately, the adapter does not find the Java 7 compiler on Windows unless DrJava is started from a JDK installation rather than a JRE installation. As a result, running this release on Windows with Java 7 will typically invoke the bundled Eclipse Java compiler rather than the Oracle/Sun Java 7 compiler.

   Note: Java 1.4 compatibility has been dropped. To use DrJava, you will need Java 5 or newer.

New features since the last release:

• better integration of the functional languaage level with the debugger
• a Java 7 compiler adapter.

Bug fixes since last release:

• the code has been revised to more closely conform with the synchronization invariants governing the design.

New features since the last stable release:

• better integration of the functional languaage level with the debugger
• a Java 7 compiler adapter
• refactored language level
• improved compiler support, including support for Habanero Java
• added support for copying the Compiler Errors pane to clipboard
• preferences like indent level can now be stored on a per-project basis (see “Project Properties”, “Advanced”).

Bug fixes since the last stable release:

• the code has been revised to more closely conform with the synchronization invariants governing the design
• applet viewer not started in event thread anymore to avoid exception
• Static type checking for primitive casts in Interactions pane.

I think I’ve got DrJava working with the Java 7 compiler the way it should, even when started with the JRE (as long as the JDK is installed as well). That was the situation that both Corky and I got into when we double-clicked on DrJava.

Please give the weekly build drjava-weekly-20110822-r5448.jar a shot.

In the original JDK7 compiler adapter, I followed the instructions and
used the [cci lang=”java”]javax.tools.ToolProvider.getSystemJavaCompiler()[/cci] method to get the Java compiler. Unfortunately, when a user starts DrJava with
the JRE, the tools.jar file is not on the boot classpath, and the [cci lang=”java”]getSystemJavaCompiler()[/cci] method will not find the
[cci lang=”java”]com.sun.tools.javac.api.JavacTool[/cci] class, even when we later provide the tools.jar file to the class loader. The [cci lang=”java”]javax.tools[/cci] package cannot be shadowed, so we can’t load one version of it for one compiler and a different version for another compiler.

Instead of using [cci lang=”java”]ToolProvider[/cci], I now do a
[cci lang=”java”]Class.forName(“com.sun.tools.javac.api.JavacTool”).newInstance()[/cci] directly. That’s exactly what [cci lang=”java”]ToolProvider[/cci] is doing too, but we can shadow the [cci lang=”java”]com.sun.tools.javac.*[/cci] packages and have control over the classloader.

I’m sorry that it took this long. I had a long day on Friday and I’m on call this coming week; I wanted to make sure I have time to goof off and enjoy the beautiful weather this weekend.

If other issues come up, please let me know.