A Concurrent Affair

With my Google Alert set for “DrJava”, I was notified about another website that mentions DrJava. It’s a blog post about the Alice summer camp. Here’s an excerpt:

The Alice Summer camps concluded last week. Starting from this week, we have the Java, Robotics and Graphics summer camps for High School children. The students who attend this Robotics summer camp are those who attended the Alice summer camp last year.

Well what do I say? I have enjoyed this week a lot. Students are taught Java on a very interactive IDE called Dr.Java. Also they are taught Java concepts using “Pictures”. In the sense, they have to write programs to modify pictures. “For” loop is taught in context of producing a mirror image of a picture, “If” loops are taught in the context of reducing the “red eye” effect in an image. I feel this is the best approach to teach children programming because they are excited working with pictures. So they have both fun and learning. However since it is just one week of Java, it is not much that they can do but definitely a good starting point to encourage them in the field of computer science.

We also got a response to a comment I left:

@Mathias Ricken:

I am very glad that you took the time to drop in a comment. Dr. Java was very user-friendly and easy to use by High School children. We have not had any problems till now. Kudos to the Dr. Java team of developers.

Thanks for the offer to help. I will surely get in touch with you if we need any help in the future.

I’m always happy when instructors choose DrJava, the best little IDE I know.

This past week was a bit chaotic. Fortunately, the problems about the new interpreter that I had bemoaned in the last posting are going away at a fast pace. Dan really dug into his work on DrJava again, so I’m not worried that we’ll still be plagued by embarrassing bugs in the Interactions Pane anymore by the time we make the next release.

There were also three problems with unintended commits to the trunk that should have gone into a branch. I reverted one of them immediately, but against my gut feeling, the consensus about the other commits was that we don’t revert but instead try to fix it. We’re making progress, but I still have an uneasy feeling. In the case of an unintended large commit, the best thing to do, in my opinion, is always a quick revert. Now, five days later, it would of course be a lot of work to backtrack to the point where the unintended commit was made, and then redo all the changes that were intended, but I have this lingering feeling that, while we’re probably running very close to our intended path, we may not actually be on it.

Out of the three problems, at least one of them seems to have gone away, at least when I run it on Windows in a VM on my MacBook. When I get home from the office, I’ll re-run the tests on a real Windows machine.

There was also a very strange problem with dozens of reported RejectedExecutionExceptions. Corky managed to find out that this was a bug known to Sun. I’m still not sure if it’s good to leave our code in a state where these exceptions can occur, even though they are not our fault. Before the unintended commits, these problems did not occur. I am going with the group consensus, though, and will try to not worry too much.

I have also begun to update the documentation for DrJava, in preparation of the new release. I don’t exactly know what other developers should add to the documentation (and it’s unlikely they’ll add anythin — in close to three years, I’m the only one to ever have updated it), but here are the new features I introduced since the last stable, features that have to be documented:

• Auto-Import in the Interactions Pane
• Detachable Tabbed Panes
• Detachable Debug Panel
• In the “Auto-Complete” dialog, there is now a button to complete the class with the fully qualified class name, e.g. java.lang.Object instead of just Object
• The External Process facility in all its glory, including the string substitution language I developed for it. This will be a long and major addition to the documentation.
• The “Follow File” operation and pane
• The “Check for New Version” dialog
• The “Send System Information to DrJava Developers” dialog

The problems with the interpreter that has been included since the last development release are much worse than I thought. Today, I realized that the for problem issue reported in [ 1954403 ] Pre-Increment/Pre-Decrement Throws Exception in For Loop is much worse than I initially thought.

It’s not only pre-increment/pre-Decrement, but also post-increment, post-decrement, assignment, add-and-assign, and subtract-and-assign. THERE IS ABSOLUTELY NO WAY TO WRITE A NORMAL FOR LOOP!

> for(int i=0;i<10;i++) System.out.println(i);
0
> for(int i=0;i<10;i++) System.out.println(i);
0
> for(int i=0;i<10;i+=1) System.out.println(i);
0
> for(int i=0;i<10;i=i+1) System.out.println(i);
0

This issue is so critical, it will kill DrJava. The problem with 1.6.0 Update 4 is bad, and it needs to be taken care of because it inconveniences users, but at least there is a simple work-around.

The problem with the for loop, and many of the other interpreter problems, however, will make most users abandon DrJava. Our user base will run away from DrJava. We need to deal with these problems before we make the next release.

If we can’t fix these bugs, I think we need to roll back to the old interpreter. It doesn’t do generics, but at least it does simple Java statements correctly! We need to know now which of the bugs we can fix within the next month.

It is embarrassing that we released a development version, and we weren’t aware of any of these problems. It’s even more embarrassing that the development release has been out there since April, and we haven’t realized how bad the problems were. What happened to our unit test coverage for DynamicJava? I assumed we had unit tests for it. How did we not notice that something as simple as a for loop does not work?

Why haven’t any of the bugs that have been on the tasks list since January been fixed? It’s almost ridiculous to call this blog posting “Problems with the New Interpreter”. The interpreter isn’t new, it’s half a year old! I’m terribly disappointed by all these problems and how they are not being handled. I have worked too hard on DrJava to see it fail because of sloppy code and lack of action. We have about a month before we should release the next beta, and the developer who created the new interpreter has been AWOL for the last two weeks.

It’s really pissing me off. I can’t release bug fixes since the version with my bug fixes and additional features is so horribly crippled by the buggy new interpreter.

As per Corky’s request at today’s JavaPLT group meeting, here is the list of interpreter bugs. The bugs are in the order of importance, as I perceive it.

I labeled those bugs that absolutely must be fixed before the next release as “CRITICAL”. If one of those bugs cannot be fixed in time, I think we should revert to the old interpreter and put the new interpreter into a branch, so we can merge it later, when the bugs have been taken care of.

The bugs labeled “IMPORTANT” are bad, but perhaps not bad enough to make us revert to the old interpreter, as long as the “CRITICAL” bugs have been fixed.

The bugs in the “WOULD BE NICE” section can be annoying or confusing, but I don’t think many people use them.

CRITICAL:

1. [ 1954403 ] Pre-Increment/Pre-Decrement Throws Exception in For Loo
   for loops don’t work at all! It’s not only pre-increment and pre-decrement, post-increment and post-decrement, assignment, add-and-assign and subtract-and-assign do not work either! This is horrible.
2. [ 1914486 ] length field of array types not supported in new interpreter
3. [ 1914487 ] private constructors in classes are accessible
   Not only private constructors are accessible, private and protected methods and fields are accessible as well!
4. [ 1986669 ] Case-Insensitivity Problem with Class Members
5. [ 1992394 ] Interpreter does not support methods with varargs
6. [ 1151966 ] Interactions pane doesn’t process nulls in varargs correctly

IMPORTANT:

7. [ 1004730 ] NullPointerException when passing null paramter
8. [ 995715 ] local methods, return value
9. [ 1266716 ] array declaration in interface yields error
10. [ 1585210 ] “Run Document’s Main Method” allows non-void return type

WOULD BE NICE IF FIXED BEFORE NEXT RELEASE:

11. [ 992535 ] Inner enum declaration fails
12. [ 994688 ] Incorrect DynamicJava error for inner class instantiation
13. [ 1080869 ] Class.forName() doesn’t work in interactions pane
14. [ 1239169 ] Reflection in Interactions
15. [ 1953827 ] Invalid escape character causes error

Update: I checked the line counts for unit tests, the old interpreter had over over 9000 lines of test code, the new interpreter has only about 4500! The number of tests should increase over time, not decrease. Leveraging the old tests is crucial, especially in a task like replacing the old interpreter with the new interpreter. The old interpreter’s tests would have ensured that the new interpreter can do everything the old one did, and then some.

Update: I’m in the process of reverting the changes made by the developer of the new interpreter, just so we have a stable revision to release in case we need it. Now I’m noticing how much worse our test coverage got with his changes. In addition to cutting the interpreter tests in half, as mentioned above, he also deleted two test files, while he kept other test files but commented out the tests in them so as to not cause suspicion. Sneaky!

Deleted:

src/edu/rice/cs/drjava/model/repl/JavaDebugInterpreterTest.java
src/edu/rice/cs/drjava/model/repl/newjvm/InterpreterJVMTest.java

Tests removed:

src/edu/rice/cs/drjava/model/debug/jpda/DebugTest.java - 9 tests removed
src/edu/rice/cs/drjava/model/debug/jpda/DebugContextTest.java - 8 tests removed
src/edu/rice/cs/drjava/model/GlobalModelOtherTest.java - 2 tests removed
src/edu/rice/cs/drjava/model/repl/JavaInterpreterTest.java - 26 tests removed

The developer has now pledged to fix the bugs in the next couple of weeks, but I have created a branch and will try to revert to the old interpreter nonetheless. Better safe than sorry, for “Wer sich auf andere verlaesst, ist verlassen” (who relies on others ends up left alone).

Using Google Alerts, I found a nice, short review of DrJava on a blog called “Return to Java”:

Dr.Java is an interesting little IDE that helps newcomers develop Java code. Although aimed at newcomers, it seems to have lots of advanced features. It could be a useful alternative to the larger IDEs like Eclipse and NetBeans.
I found it useful to create small projects and play around with compiling and easily interacting with Java code, without the hassle of creating a large project in a full featured IDE.

Thank you kindly!

For a while now, we’ve had a problem with the Java compiler, starting with JDK1.6.0 Update 4. While I’m not the expert on this, it seems like the compiler can no longer use *.exe files on the class path, even if they are just a wrapper around a *.jar file, which is how we create our Windows *.exe file.

Strangely enough, the class loader continues to accept *.exe files, as did the compiler in the past. Whatever Sun did to the Java compiler, now the compiler and class loader have divergent behaviors, and we have therefore considered this a bug on Sun’s part and filed a bug report. The bug report, however, has not been accepted (yet), and therefore isn’t accessible anywhere. We also don’t know if it has been rejected since Sun doesn’t provide notifications for rejected bugs. So, after several months, we have grudgingly accepted that we need to deal with this problem, even though there’s a simple work-around: Just use the *.jar file.

What we plan to do now is to extract a temporary copy of JUnit from our DrJava executable, and put that file instead of the DrJava *.exe file on the class path for compilation. Of course, we only do that if the user is running Windows, the DrJava application is a *.exe file, and the compiler being used is JDK1.6.0 Update 4 or newer.

We still need to exactly figure out when to create that file and when to remove it. Of course, we want to pay the smallest performance penalty possible. There are basically two options

1. Create the file before every compile, delete the file after every compile.
2. Create the file at DrJava startup, delete the file during DrJava shutdown.

The first option keeps the temporary directory very clean, but it will probably also be quite time-consuming, particularly on networked drives often found at universities. The second option is much less time-consuming, but it might leave a copy of the extracted file in the temporary directory if DrJava shuts down in an unclean way.

The compromise that we’ll probably attempt is to create temporary file names with a known, unique (or at least rare) naming scheme, e.g. drjava-junit-java6u4-problem-#.jar, where the # stands for a sequential number.

On DrJava startup, we will create a temporary file drjava-junit-java6u4-problem-1.jar, overwriting an already existing file with the same name if one exists. If we cannot overwrite the file because it is locked by another process, we will increase the sequential number until we have found a file that can be overwritten or does not exist.

On DrJava termination, we will attempt to delete all files in the temporary directory that match the pattern drjava-junit-java6u4-problem-#.jar: The file the current instance of DrJava created, and all other files left behind by other instances of Drjava.

This approach combines the low overhead of option 2) with a relatively clean temporary directory, which is a feature of option 1).

We may have to consider that the user could switch from one compiler version to a different one while running DrJava, so we may need to perform the above actions not only on startup, but also when the compiler is changed.

We are planning to have this code in place for an upcoming beta release for testing, and ready to use for the next stable version to be released at the beginning of the next school year.

Today I submitted four new feature requests for DrJava:

1. [ 2002071 ] Allow User-Specified Javadoc Links
2. [ 2002195 ] Next/Previous Match Keys for Current RegionsTreePanel
3. [ 2002096 ] Export Pane Contents to Clipboard
4. [ 2002161 ] Project-Persistent Find All Panels

The first one was a feature that I’ve been wanting for a long time, since I just can’t remember the names of any of the classes and methods in the PLT utilities, a library I usually don’t have checked out as source code.

The second feature request is something that I had thought about for a while as well: I often use F3/Shift-F3 to go to the next or previous match of the string currently entered in the “Find/Replace” pane. However, this only works for that word, so I often have to switch back to a second word, then back to the first word! There should be equivalents to the F3/Shift-F3 shortcuts that operate on the matches in the currently selected “Find All” pane, or the “Breakpoints” or “Bookmarks” pane, since they’re all RegionsTreePanel subclasses.

The final two feature requests actually arose from writing the previous blog post (which was an email to the mailing list first): I wanted an easier way to extract the “Find All” results, and after I had sent the email and continued my work on another issue, I realized that I’ll have to run these searches again once the other developers have responded to my email and we have decided on a course of action.

Ok, I have neglected the blog for a while, partly out of necessity as I was on vacation and without internet access, partly out of laziness. Neither of those reasons implies I didn’t do anything, though (yes, I work during my vacation because I enjoy my work), but I’ll fill in the gaps in a later post.

I was just going through some of the // TODO items I had left in the DrJava codebase, and I came across a problem with the “Import Package” checkbox in the “Auto Import Class” dialog in the Interactions Pane (the dialog is displayed if a class name is not recognized because it has not been imported yet).

For this checkbox, I have used Java’s setMnemonic('p'). On Windows, that means you press Alt+P, and the state of the checkbox toggles. On the Mac, however, it toggles AND inserts a Greek lower-case pi character. I searched for other uses of setMnemonic() in DrJava, and we have a bunch of them:

• A similar checkbox controls whether the Standard Java API should be considered as well for auto-completion in the Definitions Pane (Ctrl-Space or Command-Space). Pressing Alt-J on the Mac, toggles the checkbox and inserts a Greek upper-case delta.
• All the menus in DrJava’s main windw (File, Edit, Tools, Project, Debug, Language Level and Help) have mnemonics set to respond to Alt-F/E/T/P/D/L/H. The behavior on the Mac is totally erratic:
  • Alt-F displays the File menu, albeit in an awkward place, but also inserts a florin, which is a stylized lower-case f and apparently a currency symbol.
  • Alt-E does not display the Edit menu, but instead inserts an accent acute and waits for a suitable second letter to be entered to create accented letters.
  • Alt-T displays the Tools menu in an awkward place, but also inserts a dagger.
  • Alt-P displays the Project menu in an awkward place and inserts the lower-case pi, as reported above.
  • Alt-D displays the Debug menu in an awkward place and inserts what could be a lower-case delta.
  • Alt-L displays the Language Level menu in an awkward place and inserts a logical not.
  • Alt-H displays the Help menu in an awkward place and inserts what I believe is a diacritic dot, although it does not wait for a second letter, as Alt-E.
• In a currently unused class, ScrollableListSelectionDialog, there are mnemonics Alt-A and Alt-N for the “Select All” and “Select None” buttons, respectively.
  • Since there is no text input here, Alt-A, which would enter a Swedish/Danish a-circle, actually invokes the button’s action.
  • Alt-N, however does not do anything. Just like Alt-E, if text were entered, it inserts a tilde and then waits for a second suitable letter to which the tilde will belong.

A good reference for the different Alt key combinations is at: Everything2 Special Alt key characters & accents.

As a summary, Java mnemonic on Mac OS are broken, and we need to do something about it. There are several options:

1. We attempt to block the symbols from being inserted, but at least for Alt-E and Alt-N, possibly for other key combinations as well, the treatment might very well be a hard-wired Mac OS feature. Blocking these symbols would also make DrJava pretty much unusable for users who program in languages that use diacritics: German, French, and the Scandinavian languages, just to name a few.
2. We ditch mnemonics on the Mac completely. Even though I miss the ability to open menus like a File menu using Alt-F as can be done Windows, I have never seen a Mac application that does that. So not using these mnemonics on Mac OS would probably make DrJava more compliant with Apple’s user interface guidelines.
   There’s one caveat: We have to make sure that every feature that was accessible using a mnemonic is still accessible in some way on the Mac, probably using the mouse. However, I think that already is the case.
3. We use a different key for mnemonics on the Mac, e.g. Ctrl. This could work in most cases, since keyboard shortcuts like Ctrl-S to save a file on Windows are mapped to Command-S on the Mac. I checked, and the only places where we seem to have hard-coded the Ctrl, even on the Mac, is in the following places:
   • Find/Replace panel: Ctrl-Enter inserts a newline instead of invoking the “Find Next” action; Ctrl-Tab inserts a tab instead of switching focus to the next component
   • Interactions pane: Ctrl-Enter, same as above.
   • Ctrl-Backtick/Ctrl-Shift-Backtick to switch back and forth between recent documents.

   These cannot be replaced since Command-Tab toggles between MacOS applications and Command-Backtick between individual windows. Using Ctrl-Enter even though Command-Enter would work seems more consistent than using Ctrl-Tab to insert a tab, but Command-Enter to insert a newline.
   I don’t know how difficult to implement this option is because setMnemonic() asks either for a character like 'p' or for a VK_xxx constant, but what mask key to use can’t be specified.

Considering these three options (that I can think of — please chime in if you have a different idea), I believe we should choose option 2) and disable mnemonics on the Mac.

It would still be very interesting to figure out how we can use the Mac OS-standard Command-Backtick/Command-Shift-Backtick to switch between documents.

Today I implemented a feature that allows the user to double-click on the output of an external process, and if the text underneath the cursor happens to be a file name, perhaps followed by a line number, then DrJava will jump to that file and line. This helps tremendously when using external compilers.

There seems to be a problem when I want to do a javac -sourcepath ${project.root} -d ${project.build.dir} ${drjava.all.files;sep=" "} on the codebase of DrJava; in that case, DrJava hangs. The same command line works for smaller projects, and compiling DrJava using ant jar also works, so I believe it is a problem with the length of the command line that gets generated.

Unfortunately, javac does not have a command line switch to take the files to compile from a text file on disk, and piping does not work DrJava’s external process pane.

I also discovered that “Find All” highlights aren’t removed anymore; something was apparently optimized away in the last few weeks. And indeed, when I looked at it, I discovered that in revision 4482, a loop calling removeRegion for every region removed by clearRegions() and removeRegions(OpenDefinitionsDocument) had been removed and replaced by a simple call to clear() of the underlying data structure. This did not notify the listeners anymore that regions were removed, causing the highlights never to be removed.

Another issue fixed that bothered me a lot. After all, “Find All” is one of my most prominent additions.

I just finished adding a bunch of new properties for external processes:

• ${gt;op1="5";op2="7"} — greater than
• ${lt;op1="5";op2="7"} — less than
• ${gte;op1="5";op2="7"} — greater than or equal
• ${lte;op1="5";op2="7"} — less than or equal
• ${eq;op1="5";op2="7"} — equal
• ${add;op1="5";op2="7"} — addition
• ${sub;op1="5";op2="7"} — subtraction
• ${mul;op1="5";op2="7"} — multiplication
• ${div;op1="5";op2="7"} — division
• ${strlen;op="abcde"} — string length
• ${count;op="abc;def;ghi";sep=";"} — count items in list string

I also moved a lot of the code out of MainFrame.java; that file is already big enough.

PS: I’m tremendously upset that Rice started the graduation fireworks early, and I missed the presidential reception. That’s what I get for working at 10 PM the night before graduation :(

I just committed a few changes that improve the way “Open Java API Javadoc” works. In the “Preferences” dialog, you can set the Javadoc version to use to “use compiler version”. When this option is selected, DrJava will open the Javadoc page of the same version as the compiler currently selected in the “Compiler Output” tab on the bottom right of the screen.

If JDK 5.0\_13 is used as compiler, putting the cursor on File and selecting “Open Java API Javadoc for Word Under Cursor” will open http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html. But if JDK 1.4.2 is the compiler that DrJava uses, it will open http://java.sun.com/j2se/1.4.2/docs/api/java/io/File.html.

It’s these small details that help a lot and avoid confusion. Thank you, Dr. Wong, for this suggestion.