Confusing Hudson Test Failure

After moving our Hudson server to a new server, we started to experience a very confusing test failure in the Java Language Levels module.

The test that failed was AdvancedLevelTest.testShouldBeErrors. It compiles the files in javalanglevels/testFiles/forAdvancedLevelTest/shouldBreak/ and makes sure that they all convert with errors.

I had tried to figure out what causes this, since we haven’t changed the code base, but I failed to see the connection. Here is a summary of the facts:

  • The test failed on denmark.cs on the local partition.
  • The test passed on denmark.cs on the network file system.
  • The test passed on finland.cs on the network file system.
  • The test passed on finland.cs on the /local partition.

From all I can tell, the software (JDK, Ant, JavaCC, etc.) we are using on denmark.cs and finland.cs is the same. It definitely is the same when I switch on denmark.cs from /local to ~.

In the end, it turned out to be a badly written test. It didn’t create a new LanguageLevelConverter for each file to be converted, so the (expected) errors from a previous conversion counted towards the errors of following conversions. There were three files that converted without errors.

For some reason, the order in which the files in javalanglevels/testFiles/forAdvancedLevelTest/shouldBreak/ are enumerated and compiled differs on denmark’s local file system. On denmark’s local file system, one of those files that converts without error is processed first, letting the test fail. On other systems, the three tests aren’t processed until later, and the test succeeds even though it shouldn’t.

I fixed the test, but I don’t know the JLL well enough to decide if this is just a problem with the test, or if there is something wrong with the converter.

This was a very confusing, almost mysterious test failure. It was made so hard to figure out by the combination of a badly written test and the unexpected difference in file enumeration order. The latter really should not matter, but the flaw in the test case made it significant.

There’s still no version 1.328 at this time that fixes the mail configuration bug (issues #4586 and #4596), but other than that, Hudson seems to be fully operational on denmark.cs now.


Hudson 1.328 is available now, and I have already updated.


About Mathias

Software development engineer. Principal developer of DrJava. Recent Ph.D. graduate from the Department of Computer Science at Rice University.
This entry was posted in DrJava, Ramblings. Bookmark the permalink.

Leave a Reply